Senior Software Engineer - End-to-End Python Microservices for Large-Scale ML Data

What awaits you/ Job Profile

i

• Own microservices end to end: API design, Python implementation, testing, deployment on AWS, monitoring and day-to-day operations

• Build the services behind our large-scale ML data processing, which work on petabytes of data and feed machine learning training and evaluation workflows
• Design clean, versioned and well-documented APIs for data-heavy services that handle high request volumes
• Shape the service architecture using domain-driven design and proven architectural patterns
• Make sure services stay reliable, observable and scalable in production, and take part in incident response and root-cause analysis
• Mentor other software engineers and help raise engineering standards across the team through code reviews, design discussions and best practices

What should you bring along

• A track record of taking services from design through to production, and then running them
• Solid judgement on architecture and trade-offs, especially around scalability, resilience, maintainability and cost
• Passion for clean code, good design and software craftsmanship
• Enjoyment in mentoring and growing other engineers
• Active collaboration with the Technical Lead, product owners, ML engineers and other feature team members
• Willingness to raise technical debt and risks early with the leads
• Good communication skills, with the ability to describe problems and solutions precisely
• High-quality deliverables that need few review comments and produce few defects

Must have technical skill

• 8+ years of overall software engineering experience, including at least 4 years building and operating production microservices with full end-to-end ownership
• Strong Python skills (FastAPI, Flask or Django), including async programming, typing, testing (pytest) and packaging
• Strong grounding in software design: domain-driven design, clean/hexagonal architecture, SOLID and common design patterns
• Experience with API design: REST/OpenAPI, versioning, authentication/authorization and backward compatibility
• Experience building services in the data domain, such as data APIs, ingestion or processing pipelines, or high-throughput backends, ideally at terabyte-to-petabyte scale
• A proven ability to scale services for high request volumes through caching, queuing, concurrency and performance tuning
• Experience with automated testing strategies (unit, integration and contract testing) and CI/CD using GitHub Actions or Jenkins
• Experience with Docker and deploying containerized services on AWS
• Experience working in an Agile environment: principles, timeboxing, roles and ceremonies

Good to have technical skills

• Experience working with large code bases
• Experience with TypeScript and some frontend work (React or similar), including building and deploying frontend applications
• Experience with ML data pipelines or ML platforms, such as dataset management and feature/data processing for training
• Hands-on AWS experience: ECS/EKS, Lambda, API Gateway, RDS/DynamoDB, S3, SQS/SNS/Kinesis
• Experience with observability tools such as CloudWatch, Prometheus/Grafana, OpenTelemetry or Datadog
• Experience with load testing (Locust or k6) and capacity planning