Lead Data Engineer

We are looking for an experienced Lead Data Engineer to lead the design, development, and optimisation of scalable data platforms and data engineering solutions. The role requires strong hands-on expertise in modern data engineering technologies, along with the ability to drive technical initiatives and mentor engineering teams.

Responsibilities:

• Lead the design and development of scalable and reliable data pipelines and data platforms.
• Own data engineering initiatives end-to-end, from requirement gathering and architecture to development, deployment, and monitoring.
• Design, develop, and optimise ETL/ELT pipelines for large-scale data.
• Work closely with business, analytics, product, and technology teams to understand data requirements and deliver effective solutions.
• Design and implement data models, data warehouses, data lakes, and scalable data architectures.
• Ensure data quality, availability, security, governance, and performance.
• Identify opportunities to improve data architecture, pipeline performance, automation, and operational efficiency.
• Lead technical discussions, define engineering best practices, and mentor team members.
• Troubleshoot complex data platform and production issues and drive timely resolution.

Requirements:

• Strong experience in ETL/ELT, data pipelines, data modelling, and data warehousing.
• Hands-on experience with Big Data technologies and distributed data processing.
• Strong experience with SQL and database technologies.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Experience with data orchestration/workflow management tools such as Airflow or equivalent.
• Good understanding of Data Lake / Data Warehouse / Lakehouse architectures.
• Experience working with large-scale datasets and building highly scalable data solutions.
• Strong understanding of data engineering architecture, performance optimisation, and data governance.
• Proven experience in technical leadership, mentoring, and stakeholder management.
• Good-to-have experience with Kafka or other real-time/streaming technologies.
• Exposure to Databricks, Snowflake, BigQuery, Redshift, or similar platforms.
• Experience with cloud-native data engineering solutions.
• Exposure to AI/ML data pipelines or GenAI data infrastructure.
• Experience in the Telecom or other large-scale enterprise technology environments.
• Knowledge of CI/CD, Docker, or Kubernetes.