Azure Databricks Engineer
First seen by Alion on Sep 2, 2026.
Role Overview :
We are seeking a seasoned Azure Databricks Engineer to join our high-performing data engineering team in Bangalore/Chennai. In this role, you will architect and maintain robust data pipelines, transforming complex raw data into actionable insights that drive strategic decision-making. You will collaborate closely with cross-functional teams, including data scientists, business analysts, and product stakeholders, to ensure data integrity and performance at scale. By leveraging your deep technical expertise, you will play a pivotal role in optimizing our cloud data infrastructure, directly impacting the efficiency of our analytics platforms and the quality of data-driven outcomes for our global clients.
Key Responsibilities :
- Design and implement scalable end-to-end data pipelines using Azure Data Factory and Azure Databricks to support high-volume data processing requirements.
- Develop complex data transformation logic using PySpark and Apache Spark to ensure high-performance data ingestion and cleaning for downstream consumption.
- Optimize SQL queries and Spark jobs to improve system performance, reduce latency, and minimize cloud infrastructure costs for the organization.
- Partner with data architects and business stakeholders to translate complex business requirements into technical data solutions that meet evolving organizational needs.
- Maintain rigorous standards for data quality, security, and governance across all Azure-based data platforms to ensure compliance and reliability.
- Mentor junior team members and conduct code reviews to foster a culture of technical excellence and continuous improvement within the engineering team.
Required Skillset :
- Demonstrated expertise in building and managing large-scale data solutions within the Azure ecosystem, with a deep understanding of Azure Databricks and Azure Data Factory.
- Advanced proficiency in PySpark and Apache Spark for distributed computing, coupled with the ability to write highly optimized SQL for complex data manipulation.
- Strong analytical and problem-solving abilities, with a proven track record of troubleshooting performance bottlenecks in distributed data environments.
- Excellent communication skills, with the ability to articulate technical concepts clearly to non-technical stakeholders and collaborate effectively in a hybrid work environment.
- A proactive mindset toward learning new technologies and adapting to changing business requirements in a fast-paced, professional setting.
- A Bachelors or Masters degree in Computer Science, Information Technology, or a related quantitative field, supported by 6-12 years of relevant industry experience.
Skills
Data Engineering, Azure Databricks, Azure Data Factory, PySpark, SQL, Data Management, Data Services