Databricks Data Engineer
First seen by Alion on Oct 29, 2026. Customized Energy Solutions scores B on the Alion truth index.
We are seeking a Databricks Data Engineer to join our growing data engineering team in Pune, India. CES is open to considering qualified candidates for either a direct-hire, full-time opportunity or a contract-to-hire arrangement, depending on the candidate's experience, availability, and preferred employment structure.
This role will play a key part in a large-scale modernization initiative to migrate a complex, enterprise-grade Microsoft SQL Server data warehouse ecosystem to the Databricks Lakehouse Platform.
The ideal candidate has strong hands-on experience across Databricks data engineering capabilities, with a core focus on building scalable, reliable data pipelines and analytics workloads. Exposure to Databricks AI/ML capabilities is a plus.
Key Responsibilities
- Design, build, and optimize scalable data pipelines using Databricks, Apache Spark, Delta Lake, and Unity Catalog.
- Participate in the migration of an approximately 20TB on-premises Microsoft SQL Server data warehouse to Databricks.
- Convert and modernize hundreds of SQL Server tables, thousands of SSIS jobs, and downstream SSRS and SSAS workloads.
- Re-engineer legacy SSIS ETL processes into Databricks notebooks, workflows, and orchestration frameworks.
- Support the migration and redesign of SSAS cube-based analytics into Databricks SQL, Delta tables, and modern semantic models.
- Implement data quality, validation, reconciliation, audit, and migration controls throughout the modernization process.
- Optimize Spark performance, partitioning strategies, query performance, and cost efficiency across large-scale datasets.
- Apply data governance, security, lineage, and access-control standards using Unity Catalog.
- Collaborate with analytics, BI, and AI/ML teams to enable downstream reporting, analytics, and advanced data use cases.
- Contribute to reusable frameworks, technical documentation, engineering standards, and platform best practices.
Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 4–7 years of overall data engineering experience.
- Hands-on experience with Databricks on AWS preferred; Azure Databricks experience is also acceptable.
- Strong proficiency in Spark, including PySpark and/or Scala, and SQL.
- Proven experience participating in the migration of on-premises SQL Server data warehouses to cloud-based data platforms.
- Experience converting SSIS-based ETL pipelines into Spark-based data engineering solutions.
- Experience working within structured migration or enterprise modernization initiatives rather than solely lift-and-shift projects.
- Strong understanding of data warehousing concepts, dimensional modeling, and analytical workloads.
- Experience with Delta Lake, incremental processing patterns, and data versioning.
- Familiarity with Databricks Workflows, Jobs, and production-grade deployments.
- Practical experience with performance tuning and large-volume data processing.
Preferred Skills
- Experience modernizing SSAS cube-based reporting solutions into Lakehouse-native semantic models.
- Exposure to Databricks SQL Warehouses and BI integrations, with Power BI experience preferred.
- Working knowledge of Medallion architecture and cloud-native data engineering practices aligned with Databricks best practices.
- Familiarity with MLflow, feature engineering, feature stores, or other AI/ML capabilities within Databricks.
- Databricks certification is a plus.
Position Details
- Location: Pune, Maharashtra
- Employment Type: Direct Hire or Contract-to-Hire
- Work Arrangement: Hybrid, with a minimum of three days per week in the Pune office, preferably Tuesday, Wednesday, and Thursday
Additional Information
Customized Energy Solutions India offers competitive compensation along with performance-based incentives tied to individual and company performance. Our benefits are designed to support both professional and personal well-being and include comprehensive medical insurance coverage for associates and eligible dependents, generous paid time off, and opportunities for professional development through training and skill-building programs.
We are committed to creating an environment where associates are supported, challenged, and provided opportunities to grow and develop their careers over the long term.