Machine Learning Engineer – Lumen Enterprise Models (SWE-focused LLMs)

Job title: Machine Learning Engineer - Lumen Enterprise Models (SWE-focused LLMs)

Location: London; full in-office working as default

Start date: ASAP

Reports to: CEO

Compensation: Salary and Equity options available for discussion

Application deadline: 08/11/26 at 23:59 UK time

___________________________________________________________________________

Help train the software engineers of the future

Cosine is building autonomous software-engineering systems for organisations where reliability, security and control matter most.

Our Lumen Enterprise models are designed to work across real codebases. They need to understand unfamiliar architectures, make changes across multiple files, use tools, run tests and produce software that engineers can maintain and trust.

We’re looking for a Machine Learning Engineer to help train and improve these models.

You’ll work across the full training stack, including pre-training, supervised fine-tuning and reinforcement learning. This is a truly unique role that will provide exposure across all areas of the model training lifecycle. WE’re looking for someone who is just as happy to profile the training loop, identifying performance bottlenecks and memory leaks, as implementing new RL algorithms.

This is a hands-on role for someone who enjoys working close to the model and cares about the quality of the software it produces.

___________________________________________________________________________

The role

You’ll work closely with research, infrastructure and product to answer questions such as:

• What should Lumen learn next?

• Which data and environments will teach it most effectively?

• How should we reward correct, maintainable and useful software?

• How do we train models to work across large repositories and long contexts?

• How do we know when a model is genuinely better for engineers?

The models you train will write code, use development tools, run tests and receive feedback from real software-engineering environments. Your work will directly influence how Lumen reasons about code, navigates unfamiliar systems and improves through experience.

___________________________________________________________________________

Why this role matters

The best coding models are not simply the ones that generate plausible code.

They need to understand context, respect existing architecture, use tools effectively, test their work and recover when their first attempt fails. They need to produce changes that work in the real world, not just on a benchmark.

That behaviour has to be trained.

You’ll work directly on the data, environments, rewards and evaluations that shape Lumen’s capabilities. Your work will influence the models used by engineers and enterprise teams working on complex and security-sensitive software systems.

This is an opportunity to work on frontier model training while staying close to a concrete, high-value outcome: building AI that can genuinely do software engineering.

___________________________________________________________________________

What you’ll do

• Work across the full training lifecycle, running pre-training, supervised fine-tuning and reinforcement-learning experiments.

• Optimize training methods at scale on frontier models.

• Diagnose and fix performance bottlenecks and communication overhead in distributed training environments.

• Train models on long-context, multi-file and repository-level tasks.

• Develop RL and RLVR loops using signals from execution, tests, linters, static analysis and other engineering feedback, turning fresh ideas into scalable, reproducible code.

• Investigate model failure modes such as poor planning, brittle fixes, reward hacking and regressions.

• Collaborate with infrastructure, platform and data teams.

___________________________________________________________________________

What we’re looking for

You may come from machine learning research, applied ML, model training or a closely related engineering background.

You should have:

• Strong experience training deep-learning models in a production or research environment.

• Deep practical experience with PyTorch or a comparable framework.

• Experience with distributed training and multi-GPU systems (FSDP, Deepspeed, …)

• Strong software-engineering fundamentals and a high bar for correctness and maintainability.

• Clear communication and the ability to explain technical trade-offs.

• Experience with large language models is important. You should have worked meaningfully on at least one of, pre-training, SFT, preference optimisation, RLHF or RLVR.

We care more about depth of understanding and evidence of strong technical judgement than about having used every tool listed. Even if you don’t tick every box, please consider applying!

A bonus if you have

• Experience with mixture-of-experts (MoE) architectures.

• Experience with CUDA.

• Familiarity with vLLM, SGLang or other high-performance inference systems.

• Experience running jobs on SLURM, Kubernetes or managed GPU platforms.

• Experience with code-model benchmarks such as SWE-bench, HumanEval or internal repository-level evaluations.

• Contributions to open-source model-training, RL or evaluation tools.

___________________________________________________________________________

Why join Cosine

• Direct impact: Your work directly shapes the next generations of Lumen Enterprise SWE models that engineers use every day.

• Real scale: You’ll work with large, modern open-source models, long context lengths, and multi-node training runs.

• Full-stack ML engineering: From custom PyTorch code and distributed systems to data curation, RL design and MLOps.

• Research + pragmatism: You’ll stay close to the latest literature in SFT, and code LLMs, but you’ll be judged by shipped improvements, not just ideas.

If this sounds like a fit, this is a role where you can meaningfully push the frontier of open-source-based software engineering models.

___________________________________________________________________________

Cosine is an equal opportunity employer.

We value diverse backgrounds, perspectives, and ways of thinking, and we’re committed to creating an inclusive and respectful workplace.

We encourage applications from anyone who meets the role requirements, even if you don’t meet every single qualification. If you need reasonable adjustments at any stage of the hiring process, we’re happy to discuss them.

___________________________________________________________________________

Compensation, Benefits & Ways of Working

We’re an in-office team, five days a week, by design. We believe the work we’re doing benefits from being together, collaborating closely, and building shared context.

What you can expect:

• Competitive salary, benchmarked to the market

• Equity / share options, so you share in the upside you help create

• 30 days’ holiday + bank holidays

• Work hard in the office, collaborate closely, and switch off properly

• Dog-friendly office - bring your dog to work

• Weekly team breakfast & lunch

• Monthly socials

• Pension

• High-quality equipment to do your best work

We care about focus, sustainability, and doing great work - not performative overwork. We value people who show up, contribute thoughtfully, collaborate well with their colleagues, and then go home.

This role won’t suit everyone. But if you’re looking for a fast-moving environment, close collaboration, and a team that takes both the work and work-life balance seriously, it’s a great place to be.

___________________________________________________________________________

Agency & Data Protection Notice

To comply with UK GDPR and our internal data-protection and equal-opportunity obligations, we only accept candidate applications and agency submissions via our Applicant Tracking System (ATS). This ensures appropriate privacy notices, lawful processing, auditability, and consistent retention controls.

Any CVs or candidate details received outside the ATS (including via email, Slack, or direct message) will be treated as unsolicited, will not be considered as part of the recruitment process, and will not give rise to any fee or payment obligation.