Lead Data Engineer
First seen by Alion on Oct 13, 2026.
Lead Data Engineer | 9+ Years | Bangalore/Hyderabad
We're hiring a Lead Data Engineer to drive our enterprise Data Warehouse modernization to Google BigQuery & GCP!
Location: Bangalore / Hyderabad
Experience: 9+ years in Data Engineering/EDW, with 4+ years in a Lead/Principal role
Short joiners preferred
What you'll do:
Lead migration & modernization of legacy Oracle EDW/Exadata workloads to BigQuery and modern Lakehouse architectures
Design, build, and optimize large-scale batch & real-time ETL/ELT pipelines using Apache Spark (PySpark/Scala), Python, and Cloud Composer (Airflow)
Implement CDC and continuous replication from Oracle to BigQuery using Oracle GoldenGate, Kafka, and CDC frameworks
Establish end-to-end DataOps automation — CI/CD, data quality/reconciliation checks, and Infrastructure-as-Code (Terraform)
Champion AI/LLM-driven development — using tools like GitHub Copilot, Claude, ChatGPT, and Gemini for code generation, SQL refactoring, testing, and debugging
Set engineering standards, lead code/architecture reviews, and mentor teams on data engineering and AI-augmented workflows
What we're looking for:
9+ years in Data Engineering/Enterprise Data Warehousing, incl. 4+ years in a lead/principal capacity
Deep expertise in BigQuery — architecture, partitioning, clustering, slot optimization, cost governance & security (policy tags, row/column access policies)
Strong Oracle & SQL background — Exadata, advanced PL/SQL, complex data modeling (Star/Snowflake, Data Vault, SCDs), query tuning
Advanced hands-on experience with Apache Spark (PySpark/Scala) and Python for distributed data processing
Strong experience with Oracle GoldenGate, Kafka, or CDC streaming architectures
Proven track record automating data workflows and CI/CD pipelines (GitHub Actions, Jenkins, Terraform)
Demonstrated hands-on use of AI/LLMs in daily development — prompt engineering for code/SQL generation, automated testing, and code modernization