AWS Data Engineer

Key Responsibilities

• Design, develop, and optimize scalable ETL/ELT pipelines for data ingestion and transformation.
• Build and maintain robust data pipelines using AWS cloud services.
• Partner with Business Intelligence (BI), AI/ML, and Infrastructure teams to deliver reliable and scalable data platform solutions.
• Develop efficient SQL queries and Python scripts for data processing, automation, and analytics.
• Create and maintain technical documentation, including solution designs, data flow diagrams, and operational guides.
• Develop comprehensive unit tests and perform code reviews to ensure high-quality, reliable, and maintainable code.
• Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
• Work on multiple projects simultaneously while managing priorities in a fast-paced environment.
• Follow coding standards, best practices, and data governance guidelines.

Required Skills & Qualifications

• 3-5 years of hands-on experience in Data Engineering.
• Strong experience in designing and developing ETL/ELT pipelines.
• Proficiency in SQL and Python programming.
• Hands-on experience with AWS services, including:
  • Amazon Redshift
  • AWS Lambda
  • Amazon S3
  • AWS Glue (preferred)
  • AWS IAM
  • Amazon CloudWatch
• Experience with workflow orchestration tools such as Apache Airflow (preferred).
• Understanding of data warehousing concepts and dimensional data modeling.
• Knowledge of version control systems such as Git.
• Experience writing unit tests and following software development best practices.
• Strong analytical, problem-solving, and debugging skills.
• Excellent communication and documentation skills.
• Ability to manage multiple tasks and work effectively in a collaborative environment.

Preferred Skills

• Experience with CI/CD pipelines and DevOps practices.
• Knowledge of Spark, PySpark, or Databricks is an advantage.
• Exposure to AI/ML data pipelines and feature engineering.
• Familiarity with Agile/Scrum development methodologies.
• AWS Certification (Developer Associate or Data Engineer Associate) is a plus.

Key Competencies

• AWS Data Engineering
• ETL/ELT Development
• SQL & Python
• Amazon Redshift
• AWS Lambda
• Amazon S3
• Data Warehousing
• Unit Testing
• Technical Documentation
• Problem Solving
• Team Collaboration
• Multitasking
• Agile Development

Educational Qualification

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• Relevant AWS certifications will be an added advantage.