Data Engineering Lead - Modernization/Migration
First seen by Alion on Nov 14, 2026.
Work Location: Guindy Olympia Tech
Work Mode: Hybrid - 3 days in the office.
Department: Product (AI) - Candidates are not expected to have AI experience.
Role: Senior Lead/Architect - Data Engineering.
Experience: 8 to 14 years.
Primary Skill sets:
- Databricks Lakehouse architecture: PySpark, Delta Lake, Delta Live Tables / Lakeflow, Unity Catalog, Workflows, Databricks SQL, CDC, Schema evolution, Security and governance.
- Azure Data Factory (ADF) pipelines: Copy activities, Mapping Data Flows, Metadata-driven pipelines, Dynamic pipelines, Integration Runtime, Self-hosted IR, Trigger mechanisms, Error handling, Retry frameworks, Dependency management, CI/CD.
- Enterprise Data Architecture: Modern Data Lakehouse, Data Warehouse, Medallion architecture, Dimensional modelling, Star/Snowflake schemas, Data Vault, Data mesh, Batch and streaming architecture, CDC, Event-driven pipelines, Master/reference data, Data governance, Data lineage, Data quality, Metadata management.
- Metadata-driven: Ingestion, Dynamic pipeline generation, Automated schema discovery, Automated table creation, Incremental load framework, CDC framework, Data quality framework, Pipeline monitoring, Exception handling, Automated reconciliation, Data validation, Deployment automation, Environment provisioning.
- Legacy ETL modernisation: Data warehouse migration, Data lake migration, ETL - ELT transformation, On-premises - Azure migration, Legacy database - Lakehouse migration, Batch - near-real-time modernisation, stored procedure modernisation, Legacy ETL code conversion.
Notice Period: Immediate Joiner / Early Joiner - 0 days to 15 days.
Skills
Data Engineering, Databricks, PySpark, Azure Data Factory, Data Warehousing, ETL, Delta Lake, Data Governance, Data Quality, DataLake, Data Migration