Azure Data Engineer

At ProArch, we partner with businesses around the world to turn big ideas into better outcomes through IT services that span cybersecurity, cloud, data, AI, and app development. We’re 400+ team members strong across 3 countries (we call ourselves ProArchians)-and here’s what connects us all:

• A love for solving real business problems
• A belief in doing what’s right

What’s it like to work here?

• You’ll keep growing. You’ll work alongside domain experts who love to share what they know.
• You’ll be supported, heard, and trusted to make an impact.
• You’ll take on projects that touch industries, communities, and lives.
• You’ll have the time to focus on what matters most in your life outside of work.

At ProArch, you’ll be part of teams that design and deliver technology solutions solving real business challenges for our clients. With services spanning AI, Data, Application Development, Cybersecurity, Cloud & Infrastructure, and Industry Solutions, your work may involve building intelligent applications, securing business-critical systems, or supporting cloud migrations and infrastructure modernization.

Every role here contributes to shaping outcomes for global clients and driving meaningful impact. You’ll collaborate with experts across data, AI, engineering, cloud, cybersecurity, and infrastructure-solving complex problems with creativity, precision, and purpose. You’ll join a culture rooted in technology, curiosity, and continuous learning. A place where we move fast, trust you to make an impact, encourage innovation, and support your growth.

Job Summary:

ProArch is seeking a Azure Data Engineer with 5-8 years of experience to join our Data & Analytics practice. In this role, you will design, develop, and optimize scalable cloud-based data solutions that support enterprise analytics, reporting, and AI initiatives. You will work with large-scale datasets using Azure Data Platform technologies, build high-performance data pipelines, and collaborate directly with business and client stakeholders to deliver reliable, secure, and scalable data solutions.

The ideal candidate has strong expertise in SQL, Python, Azure Synapse Analytics, Azure Data Factory, Azure Data Lake, and Microsoft Fabric, along with solid experience in data modelling, DataOps, and client-facing engagements.

Responsibilities:

• Build and orchestrate data ingestion workflows to integrate data from multiple sources, including databases, APIs, enterprise applications, and cloud storage. Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Microsoft Fabric.
• Develop and optimize data transformation workflows using Azure Databricks, Apache Spark, PySpark, Python, and SQL.
• Implement and support Microsoft Fabric data engineering solutions, including data ingestion, transformation, lakehouse development, and migration of legacy data pipelines.
• Design and implement modern data lakehouse architectures using Azure Data Lake Storage Gen2, Delta Lake, and Medallion Architecture (Bronze, Silver, and Gold layers).
• Develop data models, including fact and dimension tables, to support analytical reporting and enterprise data warehousing.
• Optimize SQL queries, data models, and pipeline execution for performance, scalability, and cost efficiency.
• Implement data quality checks, validation rules, error handling, and monitoring to ensure data accuracy, completeness, and reliability.
• Apply data governance, metadata management, and data lineage practices using appropriate Azure and Microsoft data governance tools.
• Implement version control, automated deployments, and CI/CD pipelines using GitHub, Azure DevOps, or similar tools.
• Collaborate with business analysts, data architects, BI developers, and other stakeholders to translate business requirements into robust data solutions.
• Troubleshoot production data issues, improve pipeline reliability, and maintain technical documentation.
• Contribute to cloud data modernization initiatives, including migration of existing ETL workloads to Azure and Microsoft Fabric.

Requirements

1. Azure Data Factory (ADF)

• Strong hands-on experience developing and managing ADF pipelines.
• Experience with data ingestion, pipeline orchestration, scheduling, parameterization, and reusable pipeline frameworks.
• Ability to implement data integration, transformation workflows, error handling, and monitoring.

2. Microsoft Fabric

• Hands-on experience with Microsoft Fabric data engineering capabilities.
• Experience developing data pipelines and working with Fabric Lakehouse and related data processing capabilities.
• Understanding of data ingestion, transformation, Delta-based storage, and end-to-end data engineering workflows.
• Experience migrating or modernizing existing data pipelines on Microsoft Fabric is desirable.

3. Azure Databricks and PySpark

• Experience developing scalable data processing solutions using Azure Databricks, Apache Spark, PySpark, and Python.
• Knowledge of distributed data processing, data transformations, and performance optimization.
• Experience implementing Medallion Architecture and working with Delta Lake.

4. SQL and Data Warehousing

• Strong SQL skills, including complex queries, joins, aggregations, stored procedures, and query optimization.
• Experience with relational databases such as Azure SQL, SQL Server, or Azure Synapse Analytics.
• Knowledge of dimensional modeling, fact and dimension tables, and analytical data warehouse design.

5. Azure Data Storage and Integration

• Experience with Azure Data Lake Storage Gen2 (ADLS Gen2).
• Understanding of structured and semi-structured data formats, including Parquet and Delta.
• Experience integrating data from databases, APIs, files, and enterprise applications.

6. Data Quality, Governance, and DevOps

• Experience implementing data validation, reconciliation, and quality checks.
• Understanding of data governance, metadata management, and data lineage.
• Experience with Git-based version control and CI/CD automation using GitHub, Azure DevOps, Jenkins, or equivalent tools.

Preferred Skills:

• Experience with Microsoft Purview and data governance frameworks.
• Exposure to Power BI and analytical reporting requirements.
• Experience with Snowflake or other cloud data platforms.
• Familiarity with AWS data services or cross-cloud data migration.
• Understanding of GDPR and data privacy requirements.
• Experience in healthcare, retail, or other enterprise data environments.
• Knowledge of reusable ETL frameworks, automated testing, and production support practices.