Data Engineer

Job Description

ilities

• Build Data Pipelines: Develop, maintain and troubleshoot data pipelines using Python, SQL, Snowflake and dbt.

• Manage Snowflake: Create and maintain tables, views and access permissions, while improving performance.

• Kafka & Streaming: Work with Kafka topics, schemas and connectors to manage real-time data movement.

• Manage AWS Infrastructure: Use Terraform to deploy and maintain AWS resources and other cloud platform services.

• Automate Processes: Write Python scripts, create automated tests and manage CI/CD deployments using GitHub Actions.

• Fix Production Issues: Monitor pipelines and platform services, investigate failures, resolve incidents and participate in on-call support.

• Maintain Security and Reliability: Follow security standards, manage access controls and improve platform stability and costs.

• Participate in the on-call rota, diagnose production issues, coordinate incident response, and drive post-incident actions that prevent recurrence.

• Maintain runbooks, architecture records and operational documentation for the services and workflows you own.

• Contribute to agile planning, technical design and code reviews, and collaborate effectively with engineers and stakeholders across distributed teams.

Knowledge, skills and experience required

Our core platform stack includes Python and SQL; Snowflake and dbt; Confluent Cloud (Kafka, Schema Registry, Flink SQL and ksqlDB); AWS; Terraform; MongoDB Atlas; SnapLogic; Datadog and PagerDuty; GitHub Actions; Docker and ECS/Fargate; and JSON Schema data contracts with policy-as-code controls.

Required:

• A university degree in computer science, engineering or a related discipline, or equivalent practical experience.

• At least 4-5 years of hands-on Data Engineering

• Strong Python and SQL skills, with experience writing production code.

• Practical experience with Snowflake and confluent for data transformation

• Hands-on experience with AWS and Terraform.

• Understanding of Kafka, including topics, consumers and data streaming.

• Experience with Git, CI/CD and automated testing.

• Ability to troubleshoot data pipelines and production issues independently.

• Understanding of cloud security, monitoring and access management.

• Good communication, problem-solving and teamwork skills.

Nice to have:

• Stream processing with Flink SQL or ksqlDB; Java for Flink user-defined functions is an advantage.

• Administration or integration experience with MongoDB Atlas, SnapLogic or AWS DMS, including deployment, monitoring, migrations, backup and recovery.

• Observability, incident response and delivery automation using Datadog, PagerDuty and GitHub Actions.

• Docker, ECS/Fargate and AWS networking, including PrivateLink or Transit Gateway, with a focus on resilience, capacity and cost.

• JSON Schema, data contracts and policy-as-code controls using OPA or Conftest.

• AI-assisted software development tools and practices that improve delivery speed and quality, supported by appropriate review, testing, security and governance controls.

We welcome applicants with diverse perspectives, career paths and equivalent experience.