AWS Data Engineer
First seen by Alion on Sep 2, 2026.
Key Responsibilities :
- Design and develop scalable batch and real-time data pipelines on AWS.
- Build and maintain robust ETL/ELT pipelines using AWS data services.
- Develop data processing and transformation workflows using Python and PySpark.
- Ingest data from databases, APIs, files, SaaS applications, and other sources.
- Work with Amazon S3, AWS Glue, Redshift, Athena, Lambda, and other relevant AWS services.
- Develop and optimize SQL queries for large datasets.
- Design and implement data lakes, data warehouses, and data models.
- Implement data quality, validation, monitoring, and reconciliation processes.
- Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
- Troubleshoot and resolve data pipeline failures and production issues.
- Implement appropriate security, IAM, and access-control practices for data platforms.
- Collaborate with Data Architects, Data Scientists, Analysts, and application teams.
- Participate in code reviews, testing, deployment, and CI/CD activities.
Required Skills :
- 5 - 10 years of experience in Data Engineering.
- Strong hands-on experience with AWS Data Engineering.
- Strong knowledge of AWS Glue and Amazon S3.
- Strong proficiency in Python, PySpark, and SQL.
- Experience developing ETL/ELT pipelines.
- Good understanding of data lakes, data warehouses, and data modelling.
- Experience with AWS services such as Redshift, Athena, Lambda, and CloudWatch.
- Strong understanding of distributed data processing.
- Experience with data quality, monitoring, and troubleshooting.
- Good understanding of AWS IAM and cloud security.
- Knowledge of Git and CI/CD practices.
Preferred Candidate Profile :
- Experience building production-grade and scalable data platforms on AWS.
- Hands-on experience handling large datasets using PySpark and AWS Glue.
- Strong understanding of cloud data architecture and best practices.
- Experience with streaming technologies such as Kafka/Kinesis is an advantage.
- Exposure to Terraform or Infrastructure as Code (IaC) is preferred.
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration skills.
Skills
AWS, Data Engineering, ETL, Python, PySpark, SaaS, DataLake, Data Warehousing, SQL, Data Modeling