GCP Data Engineer
First seen by Alion on Oct 7, 2026.
GCP Data Engineer - Python / PySpark
Experience : 5.5 - 9 Years
Openings : 8
Role Overview :
We are looking for experienced Data Engineers with strong expertise in Python or PySpark, SQL, GCP cloud services, data processing, data warehousing, and data pipelines. The ideal candidate should have hands-on experience with BigQuery, Dataflow or Dataproc, Pub/Sub, Apache Kafka, Airflow/Cloud Composer, and Terraform.
Key Responsibilities :
- Design, develop, and maintain scalable data pipelines.
- Develop data processing solutions using Python and/or PySpark.
- Develop complex SQL queries and optimize data processing workloads.
- Work with GCP data services including BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Functions.
- Build and maintain data ingestion and transformation pipelines.
- Develop and manage workflows using Airflow, Astronomer, or Cloud Composer.
- Work with Apache Kafka for real-time data processing and streaming.
- Implement infrastructure automation using Terraform.
- Monitor data pipelines and troubleshoot production issues.
- Develop monitoring dashboards and ensure pipeline reliability.
- Collaborate with Data Scientists, Software Engineers, business stakeholders, and other technical teams.
Mandatory Technical Skills :
- SQL - minimum 4 years
- Python or PySpark - minimum 4 years
- GCP - minimum 3 years
- BigQuery
- Dataflow or Dataproc
- Pub/Sub
- Scheduled Queries
- Cloud Functions
- Monitoring Tools / Dashboards
- Apache Kafka
- Terraform scripting - minimum 2 years
- Airflow / Astronomer / Cloud Composer - minimum 2 years
Good to Have :
- Data Fusion, Tekton Pipeline, CI/CD Pipeline, GCP Secret Manager
- Data/ML solutions and advanced analytics exposure
- Prompt Engineering, Generative AI, LLM, Agentic AI, LangGraph, CrewAI, OpenAI SDK, Google ADK
Soft Skills :
- Strong written and verbal communication skills
- Strong presentation and stakeholder management skills
- Excellent analytical and problem-solving abilities
- Strong collaboration and interpersonal skills
Skills
Data Engineering, Python, PySpark, SQL, BigQuery, Dataproc, Google Cloud Platform, Kafka, Apache Airflow, Data Pipeline