Azure Data Engineer
First seen by Alion on Aug 19, 2026.
We are looking for an experienced Azure Data Engineer with 8 to 12 years of experience in designing, developing, and maintaining scalable data solutions on Microsoft Azure. The ideal candidate should have strong expertise in Azure Databricks, Azure Data Factory, ADLS, SQL, Python/PySpark, and ETL processes, along with an understanding of ERP data and financial data environments.
Key Responsibilities :
- Design, develop, and maintain scalable data pipelines and ETL/ELT workflows on Azure.
- Develop data transformation and processing solutions using Azure Databricks, Python, PySpark, and SQL.
- Build and optimize Databricks notebooks for data transformation, cleansing, aggregation, and table creation.
- Develop and maintain data ingestion pipelines using Azure Data Factory (ADF).
- Work with Azure Data Lake Storage (ADLS) for scalable storage and management of structured and unstructured data.
- Integrate data from multiple sources, including ERP systems, databases, APIs, and enterprise applications.
- Understand ERP data structures, business processes, and data flows and translate them into scalable data solutions.
- Implement data quality, validation, reconciliation, and error-handling processes across data pipelines.
- Optimize data pipelines and queries for performance, scalability, reliability, and cost efficiency.
- Work with Alteryx and Power BI to support data analysis, reporting, and visualization requirements.
- Collaborate with business, analytics, finance, risk, and technology teams to understand requirements and deliver data solutions.
- Support data governance, security, audit, and compliance requirements.
- Troubleshoot data pipeline and production issues and perform root-cause analysis.
- Create technical documentation covering data pipelines, transformations, data flows, and operational procedures.
- Contribute to cloud modernization and data platform improvement initiatives.
Required Skills & Experience :
- 8 to 12 years of experience in Data Engineering, ETL, or related data technologies.
- Strong hands-on experience with Microsoft Azure data services.
- Strong expertise in Azure Databricks and Databricks Notebooks.
- Proficiency in SQL and strong programming experience in Python and/or PySpark.
- Hands-on experience with Azure Data Factory (ADF).
- Strong experience with Azure Data Lake Storage (ADLS).
- Strong understanding of ETL/ELT concepts, data integration, data transformation, and data warehousing.
- Experience designing and implementing data pipelines using multiple data sources.
- Good understanding of Azure cloud services, security, networking, and data platform architecture.
- Experience working with ERP data and ERP data flows.
- Hands-on experience with Alteryx and Power BI for data analysis and visualization.
- Strong understanding of data quality, reconciliation, validation, and performance optimization.
- Good analytical, problem-solving, and stakeholder-management skills.
Skills
Azure, Data Engineering, ADLS, Azure Databricks, Azure Data Factory, Python, SQL, PySpark, ETL, Data Ingestion, Data Pipeline