Azure Data Engineer
First seen by Alion on Sep 3, 2026.
Required Skills:
Azure Databricks, PySpark, SQL, dbt, and Airflow, Medallion Architecture (Bronze/Silver/Gold), data modelling skills (Star/Snowflake schemas).
Job Description:
Skills:
- 5 to 8 years of experience in Azure Data Engineering.
- Strong expertise in Azure Databricks, PySpark, SQL, dbt, and Airflow.
- Experience building scalable ETL/ELT pipelines on Azure Data Lake.
- Hands-on experience with Medallion Architecture (Bronze/Silver/Gold).
- Strong data modelling skills (Star/Snowflake schemas).
- Knowledge of CI/CD, GitHub/Azure DevOps, and data quality/governance practices.
- Good understanding of analytics, reporting, and AI/ML data platforms.
- Databricks certification is required.
- DBT Experience required.
Key Responsibilities :
- Design and implement robust end-to-end data pipelines using Azure Databricks and PySpark to process large-scale structured and unstructured data for downstream consumption.
- Orchestrate complex data workflows using Apache Airflow to ensure timely delivery and reliability of data across the enterprise.
- Develop and optimize sophisticated SQL queries and stored procedures to support high-performance reporting and data retrieval requirements.
- Partner with cross-functional engineering teams to troubleshoot data quality issues and implement automated monitoring solutions that maintain system integrity.
- Lead the migration and modernization of legacy data systems to the Azure cloud, enhancing scalability and reducing operational overhead for the organization.
Required Skillset :
- Demonstrated expertise in building distributed data processing systems using Azure Databricks and PySpark, with a deep understanding of performance tuning and cluster management.
- Proven ability to design complex ETL/ELT workflows using Airflow, ensuring seamless integration across diverse data sources.
- Advanced proficiency in SQL for data modeling, complex transformations, and performance optimization within large-scale database environments.
- Strong interpersonal skills with the ability to translate technical data challenges into clear, actionable insights for non-technical stakeholders.
- A Bachelors or Masters degree in Computer Science, Engineering, or a related quantitative field, paired with 5 - 7 years of hands-on experience in data engineering.
- Adaptability to thrive in a hybrid work environment, maintaining high levels of productivity and collaboration across distributed teams in Bangalore and Hyderabad.
Skills
Data Engineering, Azure, DataLake, Azure Databricks, PySpark, SQL, Data Build Tool, Apache Airflow, Data Modeling, Snowflake DB, Data Pipeline