Data Engineering & Analytics Lead

Introduction:

We are looking for a strong Data Engineering & Analytics professional with 10+ years of experience in building data pipelines, data models and analytics solutions, with hands-on expertise in Databricks, SQL, PySpark, Python and Power BI.

Responsibilities:

- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Python, and Azure Data Factory.

- Build and optimize enterprise-grade data solutions using Databricks Lakehouse architecture and Delta Lake.

- Develop high-performance data transformation workflows and implement data quality controls.

- Create and maintain logical and physical data models to support reporting, analytics, and business intelligence requirements.

- Write and optimize complex SQL queries, stored procedures, views, and data transformation logic.

- Design and implement Medallion Architecture (Bronze, Silver, Gold layers) for modern analytics platforms.

- Develop interactive dashboards, reports, and analytical solutions using Power BI.

- Collaborate with business stakeholders to understand reporting and analytics requirements.

- Ensure data governance, metadata management, and data cataloging best practices are followed.

- Implement performance optimization strategies across data pipelines, Databricks workloads, and reporting solutions.

- Leverage AI/LLM-enabled tools for data analysis, insight generation, and analytics acceleration.

- Work closely with cross-functional teams including Data Architects, Business Analysts, and Product Owners.

- Support CI/CD and version control practices for data engineering solutions.

Primary Skills:

- Databricks (Delta Lake, Lakehouse, data transformation, performance optimization)

- SQL (advanced queries, joins, window functions, performance tuning, data transformation)

- PySpark (transformations, joins, aggregations, window functions, deduplication, optimization)

- Python

- ETL / ELT Pipeline Development

- Data Modeling & Schema Design

- Power BI (data modelling, dashboards, analytics)

- Azure Data Factory (ADF)

- Data Lake / Lakehouse Architecture / Medallion Architecture

- AI-assisted Data Analysis & Analytics

Secondary Skills:

- ADLS Gen2

- Data Quality & Validation

- Git / Version Control

- CI/CD Fundamentals

- Data Governance / Metadata / Data Cataloguing

- Exposure to Process / Operational Analytics or Process Intelligence is a plus

Skills

Databricks, PySpark, Python, SQL, Power BI, Azure Data Factory, Data Engineering, Data Modeling, ETL, DataLake, Analytics