Big Data Engineer - ML Analytics & Search

Big Data Engineer - ML Analytics & Search

BMW Group

Munich, Germany

What awaits you?

• You design and build high-performance search and query pipelines over PB-scale MDF4 and MCAP data lakes, enabling ML engineers to find relevant driving scenarios, sensor conditions, and edge cases across billions of records in seconds;
• Furthermore, you build and operate indexing and cataloguing systems for automotive sensor data, including metadata extraction, signal-level indexing, scene tagging, and embedding-based similarity search;
• You implement distributed compute pipelines for large-scale data evaluation, such as batch statistics, distribution analysis, annotation coverage reports, and data-quality scoring;
• In addition, you build fast analytical queries that enable interactive exploration on top of raw data;
• You develop dataset assembly pipelines that automatically assemble, version, and register training and evaluation datasets;
• You optimise for cost and performance through intelligent partitioning, tiered storage, caching strategies, and query pushdown to minimise scan volumes over PB-scale data;
• You operate observability stacks for data pipelines, including query latency dashboards, pipeline health, and data freshness monitors.

What should you bring along?

• University degree in Computer Science, Engineering, or a related field;
• 3–5 years of experience in big data or data engineering with a focus on analytics and search over very large datasets;
• Strong Python and SQL skills, with experience in at least one distributed compute framework;
• Experience with columnar or analytical storage and query optimisation at PB scale;
• Familiarity with search and indexing technologies, including full-text search, vector/embedding search or metadata catalogues;
• Production experience with Kubernetes and AWS/Azure/Google Cloud, as well as hands-on experience with infrastructure-as-code;
• Experience with automotive measurement data (MDF4/ASAM MDF or MCAP) as well as with embedding-based retrieval, dataset management tools, stream processing, or graph-based metadata systems.