Senior Developer
Confirmed on the employer's own hiring board on Oct 14, 2026. First seen by Alion on Oct 14, 2026.
Job Summary
We are looking for a Junior GCP Data Engineer with 3-5 years of experience who has strong hands-on expertise in Google Cloud Platform (GCP), BigQuery, SQL, Python, Apache Airflow, and Astronomer. The ideal candidate should be technically strong in designing and developing cloud-based data pipelines, building Airflow DAGs from scratch, implementing data transformation logic, and working with large-scale datasets in BigQuery.The candidate should have a solid understanding of GCP data services, data engineering best practices, CI/CD, orchestration, monitoring, and production support. This role requires someone who can independently develop data engineering components, troubleshoot complex data and pipeline issues, and contribute to the design and optimization of scalable data solutions on GCP
Key Responsibilities
Design, develop, test, and maintain data pipelines and ETL/ELT workflows on GCP.Develop Apache Airflow DAGs from scratch, including task dependencies, scheduling, parameterization, error handling, retries, monitoring, and alerting.Build and manage production-grade Airflow workflows using Astronomer.Develop and optimize complex BigQuery SQL queries, views, tables, stored procedures, and transformation logic.Design efficient BigQuery solutions with focus on performance, scalability, maintainability, and query optimization.Develop reusable data processing and automation components using Python and SQL.Integrate data from multiple upstream systems and make curated datasets available for downstream applications, analytics, and reporting.Work with relevant GCP services for data ingestion, storage, transformation, orchestration, and monitoring.Build and maintain CI/CD pipelines for automated deployment of DAGs, SQL, Python code, and other data engineering components across environments.Implement appropriate logging, monitoring, exception handling, data validation, and pipeline failure recovery mechanisms.Perform root cause analysis and troubleshoot data quality, pipeline, performance, and production issues.Follow coding standards, Git-based version control, peer review, testing, and deployment best practices.Collaborate with data engineers, analysts, BI developers, architects, and business teams to translate requirements into scalable technical solutions.Prepare and maintain technical documentation covering pipelines, DAGs, datasets, dependencies, transformation logic, and deployment procedures.
Skill Requirements
3-5 years of overall experience in Data Engineering, Cloud Data Engineering, or similar roles.Strong hands-on experience with Google Cloud Platform (GCP) and its data engineering ecosystem.Excellent hands-on knowledge of Google BigQuery, including:Advanced SQLComplex joins and transformationsViews and materialized viewsStored proceduresPartitioning and clusteringQuery performance optimizationData modelling and analytical datasetsStrong experience with Apache Airflow, with demonstrated ability to design and write DAGs from scratch.Hands-on experience with Astronomer for developing, deploying, and managing Airflow workloads.Strong proficiency in SQL with the ability to write and optimize complex queries independently.Good hands-on programming experience in Python, particularly for data processing, automation, APIs, and pipeline development.Good understanding of relevant GCP services such as Cloud Storage, Pub/Sub, Dataflow, Cloud Run, Cloud Functions, Secret Manager and IAM.Hands-on understanding of CI/CD concepts and automated deployment pipelines, preferably using Git-based development workflows.Experience with Git, branching strategies, pull requests, code reviews, and version control.Good understanding of ETL/ELT architecture, data warehousing, data modelling, data quality, and pipeline orchestration.Ability to troubleshoot complex technical issues across SQL, Python, BigQuery, Airflow, and GCP services.Understanding of cloud security, IAM/service accounts, secrets management, and environment-specific configurations.Exposure to Terraform/IaC, Docker, APIs, and containerized deployments would be an added advantage.Strong problem-solving mindset with the ability to work independently on technical development while collaborating effectively within an Agile delivery team.