Azure Data Engineer

Role : Azure Data Engineer

Location : Gurgaon / Bangalore (Gurgaon Preferred)

Experience : 4-5 Years

Duration : 12 Months

Work Mode : Hybrid

Employment Type : FTE

Role Overview :

We are looking for a skilled Azure Data Engineer with strong hands-on experience in Azure Data Services, Databricks, PySpark, Azure Data Factory, Data Lakes, and enterprise data integration. The role involves designing, developing, and optimizing scalable data platforms and pipelines that support analytics, AI, and Agentic AI use cases. The ideal candidate should be comfortable working independently on enterprise-scale data engineering and Lakehouse solutions.

Key Responsibilities :

- Design, develop, and maintain scalable Azure-based data platforms and pipelines.

- Build data engineering solutions using Azure Data Factory, ADLS Gen2, Azure Databricks, PySpark, and Apache Spark.

- Design and implement Medallion Architecture across Bronze, Silver, and Gold layers.

- Develop robust ETL/ELT pipelines and enterprise data integration solutions.

- Implement batch, streaming, and event-driven data ingestion pipelines.

- Design and implement CDC, incremental processing, upsert/merge, and idempotent processing mechanisms.

- Integrate data from SAP, CRM systems, Microsoft platforms, REST APIs, SaaS applications, and third-party systems.

- Design and implement Lakehouse and ODS architectures.

- Work with Azure Synapse Analytics, Azure SQL, and Microsoft Fabric.

- Develop effective data models to support analytics, AI, and downstream applications.

- Implement CI/CD pipelines using Azure DevOps and/or GitHub Actions.

- Ensure high standards of data quality, governance, metadata management, and data lineage.

- Optimize data pipelines and workloads for performance, scalability, reliability, and cost efficiency.

- Collaborate with data scientists, AI engineers, architects, and application teams to deliver data solutions for modern analytics and AI workloads.

Must-Have Skills & Experience :

- 4-5 years of experience in Data Engineering, with strong hands-on Azure experience.

- Strong expertise in Azure Data Engineering.

- Hands-on experience with :

1. Azure Data Factory (ADF)

2. Azure Data Lake Storage Gen2 (ADLS)

3. Azure Databricks

4. PySpark / Apache Spark

5. Azure Synapse Analytics

6. Azure SQL

7. Microsoft Fabric

- Strong understanding and implementation experience with Medallion Architecture (Bronze/Silver/Gold).

- Strong ETL/ELT and enterprise data integration experience.

- Experience with batch, streaming, and event-driven data ingestion.

- Hands-on experience with CDC, incremental processing, upsert/merge, and idempotent processing.

- Experience integrating data from SAP, CRM, Microsoft platforms, REST APIs, SaaS, and third-party systems.

- Strong knowledge of data modeling, Lakehouse, and ODS architecture.

- Experience with Git and CI/CD, preferably Azure DevOps or GitHub Actions.

- Understanding of data quality, governance, metadata, and lineage.

- Strong analytical and problem-solving skills.

AI & Modern Data Platform Exposure :

- Good exposure to :

1. Azure AI Foundry

2. Azure OpenAI

3. Microsoft Copilot Studio

4. Designing AI-ready and API-ready data models

5. Supporting AI, Agentic AI, analytics, and microservices platforms

Good to Have :

- Experience with Azure Cosmos DB.

- Knowledge of Azure Event Hubs.

- Experience with Azure Functions.

- Knowledge of Logic Apps and Service Bus.

- Experience with near real-time data processing.

- Experience integrating Retail, SAP, and CRM data.

- Cloud migration experience.

- Azure Data Engineer certification.

Ideal Candidate :

The ideal candidate is a hands-on Azure Data Engineer capable of independently designing, developing, and optimizing enterprise-scale data pipelines and Lakehouse platforms. The candidate should have strong expertise in Databricks, PySpark, ADF, ADLS, and the broader Azure data ecosystem, along with an understanding of modern data platforms supporting analytics, AI, and Agentic AI workloads.

Skills

Azure, Data Engineering, Azure Databricks, Azure Data Factory, PySpark, Medallion Architecture, ETL, Data Integration, Data Modeling, Microsoft Fabric