First seen by Alion on Nov 15, 2026.
Mandatory Skill Set:
- Strong hands-on expertise in Apache Spark and Scala for building and optimizing large-scale data pipelines.
- Experience working with high-volume batch and streaming workloads across multiple data sources.
- Strong experience with Hive and distributed data processing.
- Hands-on experience with Kafka and event/streaming-based architectures.
- Strong programming experience in Scala and/or Python.
- Experience designing, developing and troubleshooting large-scale ETL/data pipelines.
- Strong understanding of data modelling, data quality, performance tuning and distributed systems.
- Experience working with at least one major cloud platform - GCP, AWS or Azure.
- Experience with BigQuery and GCP services is preferred.
- Experience with ETL workflows and orchestration tools such as Oozie or equivalent.
- Hands-on experience with Git, Jenkins and CI/CD practices, including automated build, testing and deployment pipelines.
- Ability to independently troubleshoot complex data, performance and production issues.
Preferred / Added Advantage:
- Strong Google Cloud Platform (GCP) experience.
- Exposure to BigQuery, Google Cloud Storage (GCS), Dataproc and Pub/Sub.
- Experience migrating on-premise Spark/Hive workloads to cloud platforms.
- Experience with large-scale data migration and reconciliation.
- Knowledge of data quality and observability frameworks.
- Experience working with multiple upstream and downstream integrations.
- Exposure to multiple cloud environments.
What would you do here:
- Design, develop and optimize large-scale data pipelines using Apache Spark and Scala.
- Work with high-volume batch and streaming workloads across multiple data sources.
- Develop and support data processing solutions using Hive, Kafka and GCP services.
- Troubleshoot complex data, performance and production issues with minimal supervision.
- Contribute to cloud migration and platform modernization initiatives.
- Build and maintain ETL workflows and data orchestration pipelines.
- Perform performance tuning and optimization of Spark jobs and large datasets.
- Develop reusable, maintainable and production-ready data solutions.
- Implement and maintain CI/CD pipelines and engineering automation.
- Collaborate with engineering, product, analytics and platform teams to deliver end-to-end solutions.
- Participate in code reviews, testing, production deployments and operational support.
- Quickly understand existing systems and take ownership of assigned deliverables.
Skills
Data Engineering, Apache Spark, Scala, BigQuery, Kafka, Hive, Google Cloud Platform

