AWS Data Engineer

We are looking for skilled Data Engineers with hands-on experience in AWS Data Engineering, distributed data processing, and cloud-based data solutions. The ideal candidates should have strong expertise in building scalable data pipelines, working with Amazon Redshift, Apache Spark, PySpark, Kafka, Airflow, Python, and SQL. Candidates should be capable of developing, maintaining, and optimizing ETL/ELT workflows, data warehouses, and real-time data ingestion solutions.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using AWS services, particularly Amazon Redshift.
• Build and optimize ETL/ELT workflows for data ingestion, transformation, and processing.
• Develop distributed data processing solutions using Apache Spark and PySpark.
• Implement real-time data streaming and ingestion solutions using Apache Kafka and MQTT.
• Design and maintain Redshift-based data warehouses, data marts, and analytical data models.
• Implement workflow orchestration, scheduling, and monitoring using Apache Airflow.
• Write optimized and reusable Python and SQL code for data processing.
• Ensure data quality, reliability, scalability, and performance across data pipelines.
• Troubleshoot pipeline failures and optimize data processing workflows.
• Collaborate with cross-functional teams to deliver reliable data engineering solutions.

Mandatory Skills:

• 3-9 years of relevant experience in Data Engineering.
• Strong hands-on experience in AWS Data Engineering.
• Experience developing and managing data pipelines using Amazon Redshift.
• Proficiency in Apache Spark and PySpark.
• Strong programming skills in Python and SQL.
• Hands-on experience with Apache Airflow.
• Experience with Apache Kafka for data streaming and ingestion.
• Understanding of ETL/ELT, Data Warehousing, Data Marts, and analytical data modeling.
• Ability to build and maintain scalable, production-ready data pipelines.

Good to Have:

• MQTT and real-time data ingestion.
• Tableau or other data visualization tools.
• Manufacturing industry experience.
• Industrial IoT (IIoT) and sensor-based data ingestion.

Preferred Candidate Profile:

• Strong analytical and problem-solving skills.
• Ability to independently develop and maintain data pipelines.
• Understanding of cloud-native data engineering architectures.
• Good communication and collaboration skills.
• Experience troubleshooting and optimizing enterprise data workflows.

Skills

AWS, PySpark, Redshift, Spark