Senior Data Engineer - Microsoft Fabric

Experience:

- Overall 6+ years of experience in Azure Data Engineering and Data Warehousing.

- At least 2-3 years of hands-on experience with Microsoft Fabric.

Technical Skills:

- Strong understanding of Data Warehousing concepts, architecture, and best practices.

- Hands-on experience in designing and implementing Enterprise Data Warehouse (EDW) solutions.

- Good experience in Dimensional Data Modeling using Star and Snowflake schema techniques.

- Strong knowledge of ETL/ELT design and implementation.

Microsoft Fabric Components:

- Hands-on experience with:

- Lakehouse

- Data Factory

- Data Pipelines

- Notebooks (PySpark)

- Warehouse

- Semantic Models

- Power BI Reporting (optional)

Data Engineering & Optimization:

- Developing data transformation logic using PySpark and SQL.

- Strong SQL programming and query optimization skills.

- Experience working with Delta Lake tables and Medallion Architecture (Bronze, Silver, Gold).

- Experience with Metadata-Driven ETL/ELT Frameworks.

- Understanding of Incremental Data Loading, CDC (Change Data Capture), and SCD (Slowly Changing Dimensions).

- Experience in data validation, reconciliation, and end-to-end testing.

- Strong understanding of data quality frameworks and validation techniques.

- Experience in performance tuning and optimization of ETL pipelines and SQL queries.

- Ability to troubleshoot and resolve production data issues within SLA timelines.

- Experience in monitoring data pipelines, job failures, and root cause analysis.

Azure & DevOps (Optional):

- Hands-on experience with Azure DevOps for source code management and CI/CD implementation.

- Experience creating and maintaining build and release pipelines.

- Knowledge of Git branching strategies and version control best practices.

- Experience in deployment automation across Development, UAT, and Production environments.

Data Integration:

- Experience integrating data from multiple source systems such as SQL Server, Oracle, SAP, REST APIs, and flat files.

- Experience with incremental and full-load data ingestion patterns.

- Knowledge of handling structured and semi-structured data (JSON, XML, CSV, Parquet).

Project Experience:

- Hands-on experience in at least one end-to-end Data Warehouse implementation project, from requirements gathering through production deployment and hypercare support.

- Experience participating in solution design, technical discussions, and architecture reviews.

- Ability to estimate development effort and prepare technical design documents.

Soft Skills:

- Strong analytical and problem-solving skills.

- Excellent debugging and troubleshooting abilities.

- Good communication and stakeholder management skills.

- Ability to work independently and collaboratively within Agile/Scrum teams.

- Willingness to mentor junior team members and participate in knowledge-sharing sessions.

Preferred Skills:

- Microsoft Fabric Certification (DP-700 or equivalent) is preferred.

- Knowledge of Power BI and Semantic Models.

- Exposure to Azure Data Factory, Azure Synapse Analytics, or Databricks/PySpark is mandatory.

- Experience working in Retail, Manufacturing, or Financial Services domains is desirable.

Skills

Microsoft Fabric, Data Warehousing, PySpark, SQL, Data Modeling, ETL, Power BI, BI Reporting, Semantic Modeling