Data Engineering Expert

Role : Microsoft Fabric Data Engineering Expert

Skills - Data Engineering - Microsoft Fabric - Azure - Power BI

Experience : 12+ Years

Work Mode : Hybrid - 3 days a week from the office

Client Office Locations :

Please source only from nearby locations. Candidates from distant locations may be considered if they confirm they can commute 3 days a week. Exact office locations will be shared post-shortlisting.

Ahmedabad : 1 office

Bengaluru : 2 offices

Chennai : 2 offices

Position Overview :

We are looking for an experienced Microsoft Fabric Data Engineering Expert with strong hands-on experience in data engineering, Microsoft Fabric, Azure, Medallion Architecture, and Power BI. The ideal candidate should be capable of designing and implementing scalable, secure, and high-performance data platforms using Microsoft Fabric and Azure technologies.

The candidate will develop modern data solutions leveraging Fabric Lakehouse, Data Warehouse, Data Factory, Spark, Notebooks, Pipelines, Dataflows Gen2, OneLake, and Power BI, while following best practices around Medallion Architecture (Bronze, Silver, Gold).

Strong technical and communication skills, with proven experience in client-facing roles.

Key Responsibilities :

- Design and implement end-to-end Microsoft Fabric data engineering solutions.

- Develop scalable data platforms using Microsoft Fabric OneLake, Lakehouse, Data Warehouse, Data Factory, Spark, and Notebooks.

- Design and implement Medallion Architecture with clearly defined Bronze, Silver, and Gold layers.

- Build robust and reusable data ingestion and transformation pipelines using Fabric Data Factory, Pipelines, Dataflows Gen2, Spark, and SQL.

- Integrate data from various sources including Azure services, relational databases, APIs, files, SaaS applications, and enterprise systems.

- Implement batch and near-real-time data processing solutions where required.

- Develop PySpark, Spark SQL, and T-SQL-based transformations and data engineering workloads.

- Design optimized Delta Lake tables, including partitioning, schema management, and performance optimization.

- Implement incremental data loading, CDC, SCD Type 1 / Type 2, data reconciliation, and data quality frameworks.

- Design Gold-layer data models optimized for Power BI reporting and analytics.

- Work closely with BI developers and business stakeholders to translate analytical requirements into scalable data models.

- Develop and optimize Power BI semantic models, datasets, and reports where required.

- Apply best practices for Power BI performance, star schema modeling, DAX, row-level security, and incremental refresh.

- Implement security, governance, monitoring, lineage, and access controls across Fabric and Azure.

- Work with Microsoft Purview and Fabric governance capabilities for data discovery, cataloging, lineage, and compliance.

- Implement CI/CD and deployment practices using Git integration, deployment pipelines, Azure DevOps, or equivalent DevOps tooling.

- Monitor pipeline and workload performance and troubleshoot failures and data-quality issues.

- Optimize Fabric workloads for performance, scalability, reliability, and cost.

- Provide technical leadership and contribute to architecture decisions, coding standards, and engineering best practices.

- Mentor junior and mid-level data engineers and conduct technical reviews.

Required Technical Skills :

1. Microsoft Fabric :

- Strong hands-on experience with Microsoft Fabric, with an excellent understanding of :

1. OneLake, Lakehouse, and Fabric Data Warehouse

2. Data Factory / Pipelines, Notebooks, and Spark

3. Dataflows Gen2, Delta Lake, Shortcuts, and Direct Lake

4. Fabric monitoring and optimization

- Experience designing production-grade Fabric solutions.

2. Medallion Architecture :

- Strong practical understanding of Bronze, Silver, and Gold architecture.

- Experience designing data flows across the different layers, including :

1. Raw/landing data management, cleansing, and standardization

2. Conformed dimensions, business transformations, fact and dimension modeling

3. Slowly Changing Dimensions, incremental processing, data quality and reconciliation

- Ability to determine when data should reside in a Lakehouse versus Fabric Warehouse.

3. Data Engineering :

- Strong hands-on experience with PySpark, Spark SQL, SQL / T-SQL, and Python.

- Strong understanding of ETL / ELT concepts and modern data engineering patterns.

- Experience with batch processing, incremental loads, CDC, SCD Type 1 & 2, data validation, error handling, and metadata-driven / parameterized pipelines.

- Strong knowledge of relational and dimensional data modeling.

- Experience working with large-volume datasets and performance optimization.

4. Azure :

- Good working knowledge and hands-on experience with :

1. Azure Data Lake Storage Gen2, Data Factory, Synapse Analytics

2. Azure SQL Database, SQL Managed Instance, Key Vault, Functions

3. Azure DevOps, Microsoft Entra ID, Microsoft Purview

4. Azure Monitor / Log Analytics

5. Power BI :

- Strong understanding of Power BI architecture and semantic modeling.

- Hands-on experience developing enterprise-grade Power BI solutions.

- Strong knowledge of DAX, Power Query / M, star schema, semantic models, Direct Lake, Import & DirectQuery, Row-Level Security, Incremental Refresh, and Performance Analyzer.

- Understanding of how to design the Fabric data layer to support high-performing Power BI reporting.

6. Architecture & Engineering :

- Ability to design end-to-end modern data platforms.

- Strong understanding of Lakehouse and Data Warehouse patterns.

Skills

Microsoft Fabric, Data Engineering, PySpark, SQL, Medallion Architecture, Data Ingestion, Data Warehousing, Azure Data Factory, Data Quality, Data Modeling, Azure