Lead Data Engineer
First seen by Alion on Sep 8, 2026.
About the Role :
Founded in the year 2017, CoffeeBeans specialises in offering high end consulting services in technology, product, and processes. We are seeking an experienced Data Engineer to join our data team. In this role, you will be responsible for designing, building, and maintaining scalable data pipelines and platforms that enable advanced analytics and data-driven decision making across the organisation.
Key Responsibilities :
- Design, develop, and maintain efficient, reliable, and scalable data pipelines using modern data engineer tools and best practices.
- Build and optimise data flows between various internal and external data sources and destination.
- Implement and maintain data quality checks, monitoring, and alerting systems.
- Write efficient and maintainable code following best practices and coding standards.
- Participate in code reviews and contribute to technical documentation.
- Troubleshoot and resolve data pipeline issues and performance bottlenecks.
Required Technical Skills :
- Strong proficiency in SQL and data modelling.
- Hands-on experience with Python or Java programming.
- Proven experience with Apache Spark for large-scale data processing.
- Experience with Apache Airflow for workflow orchestration.
- Expertise in working with modern data platforms and warehouses, particularly Databricks and/or Snowflake (good to have Snowflake / databricks certification).
- Experience working with at least one major cloud provider (AWS, GCP, or Azure).
- Strong understanding of data engineering concepts, ETL processes, and data warehousing principles.
Skills
Data Engineering, SQL, Python, Apache Spark, Apache Airflow, Databricks, Snowflake DB, AWS, ETL, Data Modeling, PySpark