Data Engineer
First seen by Alion on Aug 13, 2026.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines using Azure Databricks and Azure Data Factory.
- Build robust ETL/ELT workflows for batch and real-time data processing.
- Develop data transformations using PySpark, Python, and SQL.
- Design and implement data lake/lakehouse and data warehouse solutions.
- Integrate data from multiple structured and unstructured sources.
- Optimize data pipelines, queries, and Spark workloads for performance and cost.
- Implement data quality, validation, monitoring, and error-handling mechanisms.
- Work with cloud-based data storage and Azure data services.
- Collaborate with architects, data scientists, analysts, and application teams.
- Follow engineering best practices around CI/CD, security, governance, and documentation.
Required Skills :
- 8 to 15 years of experience in data engineering.
- Strong hands-on experience with Azure Databricks.
- Strong hands-on experience with Azure Data Factory (ADF).
- Excellent PySpark, Python, and SQL skills.
- Strong understanding of ETL/ELT concepts and data pipeline architecture.
- Experience with Data Lake / Lakehouse architecture.
- Experience with data modeling and performance optimization.
- Good understanding of Microsoft Azure data services.
- Experience working in enterprise-scale data environments.
Skills
Data Engineering, ETL, Python, PySpark, SQL, DataLake, Data Warehousing, Data Pipeline, Azure Databricks, Data Modeling