Data Engineer
First seen by Alion on Sep 30, 2026.
Data Engineer
Experience: 6+ years in data engineering, ETL/ELT, analytics engineering, data warehouse development, or big data pipelines; minimum 4 years on GCP cloud data platforms preferred.
What we're looking for:
- Strong SQL and Python.
- Data modelling, warehousing concepts, partitions, clustering, indexing, optimization.
- ETL/ELT pipeline design and production support.
- Batch and streaming pipeline fundamentals.
- Orchestration using Airflow, Composer, Control-M, ADF, dbt, or equivalent.
- Big data processing using Spark, PySpark, Beam, Dataflow, Dataproc, Databricks, EMR, or equivalent.
- Cloud storage and data lake concepts.
- Data quality, reconciliation, schema validation, lineage, monitoring, and alerting.
- CI/CD for data pipelines and code versioning.
- Ability to explain scale scenarios, not just tool definitions.
Technical Skills:
- Python and advanced SQL.
- Data quality checks, reconciliation, alerting, retries, and backfills.
- Production pipeline monitoring and troubleshooting.
GCP Skills:
- Mandatory: BigQuery: datasets, tables, views, partitioning, clustering, SQL optimisation, cost awareness.
- Option 1: Dataflow / Apache Beam or equivalent streaming/batch framework.
- Option 2: Dataproc / PySpark or strong Spark experience.
- Pub/Sub or equivalent event streaming.
- Cloud Composer / Airflow DAG development.
Nice to have:
- Dataform, dbt, CI/CD for data.
- BigQuery ML.
- Dataflow templates.
- Looker / LookML basics.
- Terraform for data infrastructure.
Skills
BigQuery, Python, SQL, Dataproc, PySpark, Apache Airflow, Google Cloud Platform, Big Data, Data Infrastructure