Azure Data Engineer

We are looking for an experienced Azure Data Engineer with 7+ years of overall IT experience and at least 3–4 years of strong hands-on expertise in Azure Data Engineering. The ideal candidate should have excellent proficiency in T-SQL, SQL programming, Python-based ETL development, and data transformation using Azure Databricks.

The candidate will be responsible for designing, developing, testing, and maintaining robust and high-performance data pipelines using Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage Gen2 (ADLS Gen2), and Microsoft Fabric.

The role requires expertise in data warehouse modelling methodologies, automated data pipeline testing, source control, Azure DevOps (ADO), deployment pipelines, and data governance practices. The candidate must be analytical, self-motivated, delivery-focused, and capable of collaborating with business and technical stakeholders.

• Design, develop, and maintain scalable, reliable, and high-performance ETL/ELT data pipelines.

• Perform data analysis, exploration, validation, and transformation using T-SQL and advanced SQL programming.

• Develop ETL processes and data-wrangling solutions using Python and Azure Databricks.

• Build and manage data integration workflows using Azure Data Factory, Azure Synapse Analytics, and ADLS Gen2.

• Work with Azure data services and Microsoft Fabric to support data engineering and analytics requirements.

• Develop and optimise SQL stored procedures, functions, and queries for improved performance.

• Implement data warehouse models using Kimball methodology, dimensional modelling, and Data Vault 2.0 principles.

• Write and execute automated tests to validate data pipelines, transformations, data quality, and processing accuracy.

• Identify and resolve data pipeline failures, performance bottlenecks, and data integrity issues.

• Manage source code using version control tools and Azure DevOps (ADO).

• Support CI/CD and deployment pipelines across development, testing, and production environments.

• Apply data governance, metadata management, and data lineage practices using tools such as Microsoft Purview.

• Collaborate with business stakeholders, analysts, architects, and development teams to translate requirements into technical solutions.

• Ensure data solutions align with current and future business processes, standards, and governance requirements.

• Maintain technical documentation and communicate project progress, issues, and solutions effectively.

• Take ownership of deliverables, manage competing priorities, and ensure timely delivery of quality data engineering solutions.

• 7+ years of overall IT experience.

• Minimum 3–4 years of expert-level hands-on experience in Azure Data Engineering.

• Strong experience in SQL and T-SQL for data analysis, data exploration, and query optimisation.

• Proficiency in Python for ETL development and data wrangling, particularly using Azure Databricks.

• Hands-on experience with Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, and/or ADLS Gen2.

• Experience developing, maintaining, and optimising robust ETL/ELT data pipelines.

• Strong SQL programming skills, including stored procedures and functions.

• Experience in data warehouse modelling methodologies, including Kimball, dimensional modelling, and Data Vault 2.0.

• Experience writing automated tests for data pipelines and performing data quality validation.

• Knowledge of Microsoft Fabric and Azure data products.

• Experience with source control and Azure DevOps (ADO).

• Understanding of CI/CD, deployment pipelines, and release management.

• Awareness of data governance tools and practices, such as Microsoft Purview.

• Strong analytical, problem-solving, and troubleshooting skills.

• Ability to translate business requirements into scalable and efficient data solutions.

• Excellent communication and stakeholder management skills.

• Ability to work independently, manage multiple priorities, and deliver within deadlines.