Senior Lead Data Engineer - R01562061

Role Overview

We are seeking an experienced Senior Data Engineer to design and build next-generation data platforms leveraging AI Copilot capabilities and Microsoft Fabric. This role focuses on enabling intelligent, scalable, and automated data workflows while supporting advanced analytics and AI-driven use cases. You will play a key role in modernizing data architecture and embedding AI-assisted development practices across the data lifecycle.

Key Responsibilities

• Design, build, and maintain scalable data pipelines and data platforms using Microsoft Fabric.
• Develop and optimize data ingestion, transformation, and orchestration workflows.
• Leverage AI Copilot tools to accelerate development, automate data engineering tasks, and improve productivity.
• Implement and manage Lakehouse architectures and unified data environments.
• Collaborate with data scientists and analysts to enable AI/ML and advanced analytics use cases.
• Ensure data quality, governance, security, and compliance across all pipelines.
• Optimize performance and cost efficiency of data workloads.
• Integrate data from multiple sources (structured, semi-structured, and streaming).
• Contribute to data modeling, metadata management, and lineage tracking.
• Support CI/CD pipelines, testing, and deployment of data solutions.
• Mentor junior engineers and promote best practices in data engineering and AI-assisted development.

Required Skills & Experience

• 6+ years of experience in data engineering or related roles.
• Strong hands-on experience with Microsoft Fabric (Data Factory, Lakehouse, Synapse, or equivalent components).
• Experience working with AI Copilot tools (e.g., GitHub Copilot, Azure AI Copilot) in development workflows.
• Proficiency in Python and/or Scala.
• Strong experience with SQL and large-scale data processing.
• Experience with Spark and distributed data systems.
• Solid understanding of data warehousing and lakehouse architectures.
• Familiarity with Azure ecosystem (Azure Data Factory, Azure Synapse, Azure Data Lake).
• Experience with data governance, security, and compliance frameworks.
• Knowledge of CI/CD pipelines, DevOps practices, and version control (Git).
• Experience enabling AI-driven data platforms or intelligent data workflows.
• Familiarity with real-time/streaming data processing.
• Knowledge of MLOps and integration with machine learning pipelines.
• Experience with Power BI or other data visualization tools.
• Exposure to enterprise data platform migrations or modernization programs.