Senior Data DevOps Engineer

Responsibilities

• Design, build, and operate services and pipelines that power analytics data ingestion, schema management, and delivery into Databricks
• Modernize legacy platform components by migrating services, libraries, and deployment workflows to modern Java and application standards
• Build automation, CI/CD flows, and scenario/integration test coverage to make platform changes safe and repeatable
• Design and maintain production infrastructure in AWS using Infrastructure as Code tooling such as Terraform
• Improve data quality through stronger schema validation, metadata capture, lineage tracking, and operational guardrails
• Build tooling and paved paths that help internal customers migrate to new ingestion and schema standards
• Collaborate with cross-functional teams to prepare design docs, rollout plans, and implementation strategies for platform changes
• Monitor live systems, investigate incidents, and be part of the on-call team that provides 3rd line support to live analytics services
• Support performance, reliability, and cost optimization across ingestion and processing flows

Required qualifications

• Minimum of 5 years commercial work experience in software engineering, data engineering, platform engineering, or a related field
• Bachelor's or higher degree in Computer Science, Software Engineering, or a related field
• Coding skills, with commercial experience in one of the following languages - Java, Python, Golang
• Knowledge in Infrastructure as Code tooling, e.g. Terraform
• Experience with CI/CD tooling, e.g. GitHub Actions, Jenkins, Docker
• Experience with streaming and schema-driven systems, e.g. Kafka, Protobuf, JSON Schema, or schema registries
• Commercial experience with Airflow and DBT (Data Build Tool)
• Commercial experience with Databricks
• Commercial experience with  Spark/PySpark
• Effective communication and teamwork skills

Nice to have

• Experience with DatabricksDelta Live Tables and Unity Catalog
• Experience with AWS especially Kafka/MSK-based data flows, S3, IAM and VPC/networking.
• Experience with Buf, Protobuf tooling, or JSON-to-Protobuf migration work
• Familiarity with Golang for internal tooling or automation
• Experience with infrastructure monitoring and on-call practices using tools like Datadog and PagerDuty
• Demonstrated ability to troubleshoot and optimize complex ETL pipelines

Tech stack

AWS, Airflow, Databricks, CI/CD, Golang, Python, Terraform, DBT, Spark, Kafka, Protobuf, JSON