Mudit Khandelwal

Created a governed landing-to-Lakehouse data-quality framework covering Landing, Bronze, Silver and Gold layers. Developed configurable Delta Live Tables expectations for schema checks, null handling, referential integrity, range validation and deduplication. Integrated an LLM approach to generate human-readable explanations for data-quality rule violations to reduce incidents.

Designed and built batch and near-real-time data pipelines using Azure Databricks, PySpark and Azure Data Factory for enterprise reporting across multiple business units. Reworked ingestion and incremental loading logic to improve data quality and reduce pipeline failures. Implemented Lakehouse patterns with Medallion architecture and optimized Spark performance to cut execution time. Set up governance with Unity Catalog and supported production workloads with troubleshooting for distributed pipelines.

Built a metadata-driven ingestion framework using reusable Azure Data Factory pipelines for onboarding multiple source tables for pricing analytics. Implemented CDC and watermark-based incremental loading to reduce full-load data transfer volume. Improved orchestration scalability and performance through dynamic pipeline patterns and parallel processing.