Principal Data Engineer

We are seeking a highly experienced Principal Data Engineer / Senior Data Architect to lead the design, architecture, and scalability of modern data platforms. This role requires deep expertise in building enterprise-grade data ecosystems across cloud, real-time, batch, governance, and AI-ready infrastructure. You will own the end-to-end data architecture vision from ingestion and transformation to analytics, ML enablement, governance, and cost optimisation.

The candidate will have responsibilities across the following functions:

Data Architecture and Platform Design:

• Architect scalable data lakehouse and warehouse platforms
• Define canonical data models and domain-driven data architecture
• Design medallion (Bronze/Silver/Gold) architectures
• Establish data standards, governance, and security frameworks
• Evaluate and select cloud-native data tools

Engineering and Implementation Leadership:

• Build high-performance ETL/ELT pipelines (batch + streaming)
• Architect real-time data pipelines using event-driven systems
• Optimise large-scale distributed data processing systems
• Implement CI/CD for data engineering workflows
• Ensure reliability, scalability, and observability

Cloud and Modern Data Stack Expertise:

• Hands-on experience with Amazon Web Services, Microsoft Azure, or Google Cloud
• Strong expertise in Databricks, Snowflake, Apache Kafka, and Airflow.

Analytics and AI Enablement:

• Design data platforms optimised for BI and AI workloads
• Enable feature stores and ML-ready datasets
• Support GenAI and advanced analytics use cases
• Ensure high data quality and lineage tracking

Governance, Security and Cost Optimisation:

• Implement RBAC, encryption, and data masking
• Design cost-aware architectures
• Monitor usage and optimise cloud spend
• Define data SLAs and reliability frameworks.

Leadership and Strategy:

• Lead and mentor data engineering teams
• Conduct architecture reviews
• Partner with product, AI, and business teams
• Define long-term data strategy and roadmap
• Drive technical excellence and coding standards.

Requirements:

• 8+ years of experience in data engineering/data architecture.
• Proven experience designing large-scale data platforms.
• Deep expertise in distributed systems.
• Strong SQL and Python/Scala skills.
• Experience handling TB-PB scale datasets.
• Strong understanding of data modelling (dimensional + data vault).
• Experience with real-time and streaming architecture.
• Experience leading technical teams.
• Experience in FinTech, SaaS, or enterprise consulting.
• Experience building data engineering practice from scratch.
• Exposure to AI/ML infrastructure.
• Certifications in AWS/Azure/GCP.
• Experience with multi-cloud environments.

Skills

ETL, airflow, data architecture, data engineering, databricks, kafka, python, scala, snowflake, sql