Data Engineer - AWS

We are seeking a hands-on AWS Cloud and Data Platform Engineer to actively develop and scale a serverless, AWS-based data and BI platform. The role involves building event-driven data pipelines, API integrations, and cloud-native services using AWS Lambda, modern ETL patterns, and strong engineering best practices. You will work closely with cross-functional teams across application engineering, IT, OT, and DevOps to accelerate the delivery of a production-grade analytics and integration platform.

The candidate will have responsibilities across the following functions:

AWS Cloud Engineering:

• Design, develop, and operate serverless solutions using AWS Lambda, API Gateway, S3 EventBridge, SNS/SQS.
• Implement secure networking using VPCs, IAM roles, and security best practices.
• Monitor and optimise solutions using CloudWatch metrics, logs, and alarms.

Data Pipelines and ETL:

• Build and enhance serverless data pipelines using AWS Lambda and Glue.
• Design event-driven and API-based ingestion architectures.
• Perform data modelling and transformations for analytics and BI use cases.

API Development and Integration:

• Design and implement RESTful APIs using an API Gateway and Lambda.
• Integrate with internal and external systems using secure API and webhook-based patterns.
• Define and maintain OpenAPI / Swagger specifications.

DevOps and Infrastructure as Code:

• Provision and manage infrastructure using Terraform or AWS CDK.
• Build and maintain CI/CD pipelines using GitHub Actions or AWS CodePipeline.
• Automate deployments and enforce repeatable, reliable delivery processes.

Observability and Dashboarding:

• Develop Grafana dashboards for system, pipeline, and business metrics.
• Implement CloudWatch-based monitoring, alerting, and logging.
• Visualise and analyse time-series data to support operational and BI insights.

Security and Governance:

• Implement IAM modelling, secrets management, encryption, and audit logging.
• Enforce cloud security, governance, and compliance best practices.

Cross-Functional Collaboration:

• Produce clear documentation and architecture diagrams.
• Collaborate with Application Software Engineering, IT, OT, and DevOps teams.
• Participate in design discussions and technical decision-making.

Requirements:

• 5-8 years of overall experience in software engineering, cloud engineering, or data engineering.
• Strong hands-on experience with AWS serverless services (Lambda, API Gateway, EventBridge, S3 SNS/SQS).
• Experience building data pipelines and ETL workflows in cloud-native architectures.
• Proficiency in Python and/or Node.js for backend and Lambda development.
• Experience with Infrastructure-as-Code (Terraform or AWS CDK).
• CI/CD experience with GitHub Actions, CodePipeline, or similar tools.
• Strong understanding of API design, integration patterns, and security.
• Experience with Grafana and CloudWatch for observability and dashboards.
• Experience with Time-series data platforms (InfluxDB or similar).

Skills

AWS, AWS Lambda, Amazon Web Services, ETL, Lambda