Databricks PySpark Developer
First seen by Alion on Sep 27, 2026.
Role Summary :
We are looking for an experienced Databricks PySpark Developer to design, develop, and enhance scalable data pipelines.
The role requires strong hands-on expertise in PySpark and Databricks, along with experience working on cloud data platforms and ETL solutions.
Key Responsibilities :
- Design, develop, and maintain scalable data pipelines using Databricks and PySpark
- Understand existing data pipelines and implement enhancements based on project requirements
- Develop robust ETL/data processing workflows
- Collaborate with technical leads, data engineers, product managers, and business stakeholders
- Translate business requirements into effective technical solutions
- Troubleshoot, optimize, and improve data pipeline performance
- Work with distributed data processing and cloud-based data platforms
- Independently manage assigned development activities and deliver within timelines
- Provide regular updates on development progress and technical deliverables
Mandatory Skills :
- 5 - 7 years of relevant IT experience
- Strong hands-on experience with PySpark
- Strong hands-on experience with Databricks
- Good experience in ETL/Data Engineering
- Experience with at least one Cloud Data Platform
- Experience with Kafka / Pub-Sub for streaming data processing
- Experience with Apache Airflow for data pipeline orchestration
- Strong understanding of data engineering concepts
Good to Have :
- Experience with GCP
- Hands-on experience with GCP Data Services
- Experience in the Healthcare domain
- Experience working with large-scale data processing and cloud data pipelines
Candidate Profile :
- Strong problem-solving and analytical skills
- Good communication and stakeholder collaboration skills
- Ability to work independently with minimal supervision
- Comfortable collaborating with teams across India and the US
Skills
PySpark, Databricks, ETL, Data Engineering, Apache Airflow, Kafka, Cloud, Data Management, Data Pipeline, Spark