Senior Data Engineer
First seen by Alion on Nov 1, 2026.
Senior Data Engineer to build the pipelines that turn raw multimodal capture — audio, egocentric video and sensor streams from 20+ countries — into versioned, quality-checked, client-ready datasets for frontier AI and robotics teams.
Location: Bengaluru, Karnataka, India · Employment: full-time, on-site · Experience: 5+ years
WHAT YOU WILL DO
Design and build batch and streaming pipelines that ingest raw audio, video and sensor (IMU) data from the global capture network into cloud storage. • Build processing stages for unstructured media: transcoding, segmentation, metadata extraction, deduplication and joins with annotation and contributor data. • Own data quality: automated validation, QC gates and anomaly detection so that bad captures are caught before they reach a client. • Build dataset versioning, lineage and manifests so any delivered dataset can be reproduced exactly. • Package and deliver datasets in the formats client teams ask for (Parquet, WebDataset, Hugging Face datasets, MCAP and similar), on time and to specification.
WHAT YOU NEED
5+ years of data engineering, building production pipelines end to end
Strong Python and SQ
• Distributed processing at scale — Spark, Ray, Beam or Dask
• Workflow orchestration (Airflow, Dagster or Prefect) and cloud object storage on AWS or GCP
• Has worked with large unstructured or media data (audio, video, images, sensor logs), not only tabular warehouse data