Lead Data Engineer

The core requirements for the job include the following:

Data Engineering Core:

• Expert-level SQL, complex joins, window functions, CTEs, incremental merge patterns, and performance tuning.
• Strong Python, production ingestion pipelines, REST API clients, error handling, logging, and retry logic.
• dbt (any adapter), model structure, incremental materialisations, tests, macros, and packages.
• Medallion architecture, deep understanding of Bronze, Silver, and Gold layer responsibilities and boundaries.
• Data modelling, canonical modelling, dimensional modelling, star schema, SCD Type 2 and the distinction between integration models and reporting models.

Cloud and Platform:

• Hands-on delivery experience on at least one major cloud, AWS (S3 Redshift, Glue, Athena), GCP (BigQuery, Dataproc, Cloud Storage), or Azure (Fabric, Synapse, ADLS), with working knowledge of the others.
• Lakehouse platforms, Delta Lake, Apache Iceberg, or similar, and how data is physically stored, partitioned, and queried.
• Infrastructure fundamentals, networking, secrets management, managed identity or IAM roles, and container-based compute.

Engineering Practices:

• Git and GitHub, trunk-based branching, pull requests, code review, and CI/CD automation.
• Data quality, dbt tests, quarantine patterns, control tables, and schema drift detection.
• 5 to 7 years of hands-on data engineering experience across ingestion, transformation, modelling, and client delivery.