Senior Data Engineer

Sr Data Engineer Azure

**Experience:**

8-10 years of IT experience, with a minimum of 4-5 years specifically focused on Azure data Services .

**Location:**

Gurugram

**About the Role:**

We are seeking a highly skilled Senior Azure Data Engineer to lead the design, development, and optimization of our enterprise-grade cloud data platform. You will be responsible for building scalable, resilient data pipelines that transform raw data from diverse sources into high-quality, actionable insights. As a senior member of the team, you will drive architectural decisions, mentor junior engineers, and ensure our data infrastructure aligns with the latest industry standards for performance, security, and cost-efficiency.

Key Responsibilities-->

Architecture & Design: Design end-to-end cloud data architectures using the Medallion (Bronze/Silver/Gold) architecture and Lakehouse patterns to support both batch and real-time processing.

Data Integration (ETL/ELT): Build complex, metadata-driven pipelines using Azure Data Factory (ADF) and Synapse Pipelines. Implement event-based and tumbling window triggers for seamless ingestion.

Big Data Processing: Develop high-performance data transformation logic using Azure Databricks and PySpark. Optimize Spark jobs through efficient partitioning, caching, and shuffle tuning.

Microsoft Fabric Implementation: Lead the transition or integration of existing workloads into Microsoft Fabric, leveraging OneLake, Dataflows Gen2, and Fabric Notebooks.

Data Modeling: Create robust dimensional models (Star/Snowflake schemas) within Azure Synapse SQL Pools and Fabric Data Warehouses to empower Power BI reporting.

Governance & Security: Implement strict RBAC, data masking, and encryption. Utilize Azure Purview for data lineage, cataloging, and compliance.

DevOps & Automation: Establish CI/CD pipelines using Azure DevOps or GitHub Actions for automated deployment of infrastructure (Terraform/Bicep) and code.

Performance Tuning: Monitor and optimize cloud spend and query performance across Synapse, Databricks, and Azure SQL databases.

Technical Requirements - Must-Have Skills

Core Azure Services - Azure Data Factory, Azure Databricks, Azure Synapse Analytics, ADLS Gen2.

Languages - Expert-level SQL and Python (PySpark, Pandas).

Emerging Tech Hands-on experience with Microsoft Fabric and OneLake.

Storage - Delta Lake, Parquet, Avro, Azure SQL, Cosmos DB.

Streaming - Azure Event Hubs, Stream Analytics, or Spark Structured Streaming.

Tools/DevOps - Git, Azure DevOps, Terraform/ARM, Power BI (Semantic Models).

Qualifications

Experience: 8-10 years in Data Engineering, with at least 5+ years focused specifically on the Azure ecosystem.

Education: Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field.

Certifications: (Preferred) DP-203 (Azure Data Engineer Associate) or Microsoft Certified: Fabric Analytics Engineer Associate.

Soft Skills: Strong ability to translate complex business requirements into technical roadmaps; experience mentoring junior staff.