Senior Data Engineer
First seen by Alion on Aug 31, 2026.
Key Responsibilities :
- Design and develop batch and real-time data pipelines on GCP.
- Build scalable ETL/ELT pipelines using Dataflow, BigQuery, and Cloud Composer.
- Ingest data from databases, APIs, files, SaaS applications, and streaming sources.
- Design and optimize BigQuery datasets, tables, partitioning, and clustering.
- Develop data transformations using SQL, Python, Apache Beam, Spark, dbt, or Dataform.
- Implement workflow orchestration and dependency management using Apache Airflow / Cloud Composer.
- Implement data quality checks, reconciliation, lineage, monitoring, and validation processes.
- Configure and manage GCP IAM, service accounts, and least-privilege access.
- Optimize BigQuery and Dataflow workloads for performance, scalability, and cloud cost efficiency.
- Build and manage infrastructure using Terraform.
- Participate in CI/CD implementation and deployment processes.
- Troubleshoot and resolve production data-pipeline failures and performance issues.
- Collaborate with Data Architects, Data Analysts, Data Scientists, and application engineering teams.
- Ensure data pipelines are reliable, secure, scalable, and maintainable.
Required Skills:
- 6+ years of experience in Data Engineering.
- Strong hands-on experience with Google Cloud Platform (GCP).
- Expertise in BigQuery, Dataflow, and Cloud Composer/Airflow.
- Strong SQL and Python programming skills.
- Experience with Apache Beam / Spark.
- Experience with ETL/ELT pipeline development and data integration.
- Knowledge of dbt / Dataform.
- Strong understanding of data warehousing, data modelling, partitioning, and clustering.
- Experience with data quality, reconciliation, lineage, monitoring, and troubleshooting.
- Hands-on experience with GCP IAM and service accounts.
- Experience with Terraform and Infrastructure as Code (IaC).
- Understanding of CI/CD and DevOps practices.
Preferred Candidate Profile:
- Strong experience designing production-grade, scalable data platforms on GCP.
- Experience handling both batch and streaming data workloads.
- Strong understanding of cloud cost optimization and performance tuning.
- Excellent analytical and problem-solving skills.
- Ability to work effectively with technical and business stakeholders.
- Strong communication and collaboration skills.
Skills
Data Engineering, Google Cloud Platform, BigQuery, Python, SQL, Data Management, Data Pipeline, Spark, Data Build Tool