Lead Data Engineer - Java/Python
First seen by Alion on Sep 28, 2026.
Key Responsibilities:
- Design, develop, and maintain efficient, reliable, and scalable data pipelines using modern data engineering tools and best practices.
- Build and optimize data flows between internal and external data sources and destinations.
- Implement data quality checks, monitoring, and alerting mechanisms.
- Write clean, efficient, maintainable, and production-ready code.
- Follow coding standards and engineering best practices.
- Participate in code reviews and contribute to technical documentation.
- Troubleshoot and resolve data pipeline failures, data quality issues, and performance bottlenecks.
- Collaborate with cross-functional teams to support analytics and data-driven business initiatives.
Required Technical Skills:
- Strong proficiency in SQL and data modelling.
- Hands-on experience with Python or Java.
- Proven experience with Apache Spark for large-scale data processing.
- Strong experience with Apache Airflow for workflow orchestration.
- Experience with modern data platforms and cloud data warehouses, particularly Databricks and/or Snowflake.
- Experience with at least one major cloud platform: AWS, GCP, or Azure.
- Strong understanding of data engineering concepts, ETL processes, and data warehousing principles.
- Good understanding of scalable data pipeline architecture and performance optimization.
Good to Have:
- Databricks certification.
- Snowflake certification.
- Experience working across multiple cloud platforms.
- Exposure to modern data engineering and analytics ecosystems.
Skills
Apache Spark, Apache Airflow, Python, Java, Databricks, AWS, Azure, Snowflake DB, Google Cloud Platform, Data Engineering, Data Pipeline