Senior Specialist - Data Engineering

The Data Lead will architect build and optimize enterprise data warehousing and analytics solutions using Microsoft Fabric with deep expertise in data modeling pipeline engineering semantic modeling realtime data processing governance and analytics delivery.

The role requires strong handson technical leadership and the ability to convert business requirements into scalable governed data solutions that drive measurable business outcomes.

Engage with business stakeholders to understand analytical operational and compliance needs

Convert business requirements into functional design sourcetotarget mappings transformation logic and technical specifications

Validate requirements against enterprise data models and recommend architecture patterns Lakehouse Warehouse Realtime Hub

Data Modeling Fabric Semantic Layer

Design build and govern Fabric Semantic Models Direct Lake Import DQ Hybrid modes

Define enterprisewide canonical models shared dimensions hierarchies KPIs and reusable DAX measures

Optimize semantic models for performance using aggregations incremental refresh and partitioning strategies

Enable certified datasets semantic governance and rolelevel security within Fabric

ETLELT Engineering with Fabric Pipelines Dataflows Notebooks

Build ingestion and transformation processes using Data Factory pipelines Dataflows Gen2 Warehouse pipelines and PySpark notebooks

Maintain metadatadriven ETL patterns and reusable frameworks for ingestion harmonization and transformation

Fabric Notebook Engineering

Use Fabric Notebooks to perform

PySpark transformations

Delta Lake optimization Zorder vacuuming partitioning

Data validation and data quality checks

ML feature engineering and lightweight model operations

Automate notebook execution via pipelines triggers and Fabric scheduling

Integrate notebooks with Lakehouse tables Warehouse tables and ML model outputs

Near RealTime NRT Data Processing in Fabric

Design and implement nearrealtime data ingestion pipelines using

Fabric RealTime Hub

Event Streams

KQL Databases

Streaming Dataflows

Build streaming transformations and realtime analytical models leveraging Kusto Query Language KQL and PySpark Structured Streaming

Ensure lowlatency ingestion to LakehouseWarehouse for downstream consumption

Optimize realtime workloads for durability recovery and performance under high throughput

Build dashboards and semantic models that support nearrealtime refresh scenarios

Performance Optimization

Optimize SQL queries Lakehouse Delta tables semantic models DAX expressions and Power BI datasets

Review and tune pipeline throughput notebook execution performance and refresh schedules

Improve Direct Lake performance by optimizing storage layouts file size distributions and columns

Perform workload monitoring using Fabric capacity metrics and logs

Power BI Development Visualization

Build highquality Power BI dashboards and enterprise reports integrated with centralized semantic models

Develop DAX calculations KPIs UXUI standards drillthroughs and rowlevel security

Drive semantic model reuse and promote governed gold datasets

Governance Security Compliance Purview Integration

Implement data governance and cataloging using Microsoft Purview for Fabric assets Manage lineage tracking glossary management classification and metadata enrichment

Define enterprise security controls RBAC masking PII handling encryption retention policies

Ensure compliance with GDPR CCPA HIPAA SOX and internal audit controls

Govern Fabric workspace structure capacity usage data certification processes and lifecycle management

Technical Leadership/ Program Delivery

Lead data engineers BI developers and analysts across multiple initiatives

Review designs STTMs code semantic models and performance benchmarks

Own sprint planning estimation milestone tracking and stakeholder communication

Promote documentation technical standards reusable design frameworks and automation

Technical Expertise

8 years of data engineering or BI experience with 2 years in Microsoft Fabric

Expert in data warehousing design dimensional modeling semantic modeling and data governance

Strong handson experience with

Fabric Lakehouses Warehouses

Fabric Semantic Models Direct Lake Import Hybrid

RealTime Hub Event Streams KQL Databases

Notebooks PySpark data transformations optimization

Data Factory pipelines Dataflows Gen2

Power BI modeling DAX report development

Solid understanding of Delta Lake Spark performance tuning and workload optimization

Expertise in implementing sourcetotarget mappings transformation logic and validation rules

Preferred Skills

Azure Synapse ADF ADLS Databricks experience

Knowledge of DataOps