Staff Software Engineer – ML Inference Runtime

Job Overview:

We have a phenomenal opportunity to help shape the future of Machine Learning and Artificial Intelligence on Arm platforms. Arm technology is used wherever computing matters. Are you passionate about ML systems and keen to influence how generative AI and other machine-learning workloads run efficiently on next-generation hardware?

Then we should talk!

We are growing our ML framework integrations team and seek dedicated and motivated engineers to join us. You will work with teams across Arm and with leading technology companies building products based on Arm technologies. You will share ideas, learn from outstanding engineers, and make contributions that have a real impact. This role offers the opportunity to provide develop the software that connects widely used machine-learning frameworks and runtimes to Arm acceleration technologies.

Responsibilities:

Responsibilities include developing and optimizing C++, C, and Python software that integrates machine-learning platforms and execution environments with Arm hardware acceleration. This work entails integration with open-source LLM software such as llama.cpp. You will design and implement integrations between ML frameworks, runtimes, and hardware-acceleration technologies, as well as analyse and improve the performance of ML and LLM workloads. The role also involves leading significant work from design through delivery, mentoring junior engineers, and supporting technical decisions.

We are looking for someone with strong analytical and problem-solving skills who enjoys finding innovative solutions to complex technical challenges. You will be comfortable working in a fast-paced environment and collaborating across complementary teams to achieve shared goals.

Required Skills and Experience :

• Strong C++, C, and Python programming and software-design skills
• Experience delivering significant software features across multiple teams
• Familiarity with Git, GitLab, continuous integration, and automated testing
• Effective technical leadership and communication skills
• Knowledge of LLM inference, including tokenization, attention, KV caches, batching and quantisation

“Nice To Have” Skills and Experience :

• Experience with llama.cpp or other open-source ML or LLM runtimes
• Experience with profiling and optimizing ML workloads
• Experience with compiler or ML graph technologies
• Experience with Vulkan, GPU compute, or other hardware-acceleration APIs
• Experience using AI-assisted software-development tools effectively

In Return:

All Arm employees are provided with vital training to succeed in their respective roles, as well as a friendly and high-performance working environment.

You will be working with a bunch of enthusiastic and brilliant colleagues. We are proud to have a set of behaviours that reflect our DEI (Diversity, Equity & Inclusion) culture and guide our decisions, defining how we work together to shape extraordinary!

Please note that no relocation package is available for this role.

|Develop C++, C and Python integrations connecting ML frameworks and LLM runtimes like llama.cpp with Arm acceleration technologies, optimising AI workloads for next-generation hardware.!

Our 10x Mindset

At Arm, we believe progress happens when people are empowered to think bigger and push beyond what seems possible. Our 10x mindset is about curiosity, ambition and creating impact that will be used by millions. We learn fast, prioritise collaboration and turn bold ideas into real technology. We look for people who are inspired by this way of working and want to grow in an environment where bold ideas are welcomed. Discover how the 10x mindset shapes life at Arm: https://careers.arm.com/10x-mindset

Use of AI in Your Application

At Arm, we embrace the benefits AI can bring, including in recruitment. You may use AI to support your preparation, such as improving the clarity of written responses, practicing interview questions, or learning more about our industry. However, your application and interview responses must reflect your own thinking, experience, and abilities. We want to understand your authentic perspective, creativity, and approach to problem solving. As such, you may not use AI to complete assigned tasks as part of any assessments. If you plan to use AI-supported or assistive technologies as part of a reasonable adjustment, please contact [email protected] so we can support you.

Recruitment Adjustments and Accommodations

At Arm, we want to build extraordinary teams. If you need an adjustment or an accommodation during the recruitment process, please email [email protected]. To note, by sending us the requested information, you consent to its use by Arm to arrange for appropriate accommodations. All accommodation or adjustment requests will be treated with confidentiality, and information concerning these requests will only be disclosed as necessary to provide the accommodation. Although this is not an exhaustive list, examples of support include breaks between interviews, having documents read aloud, or office accessibility. Please email us about anything we can do to accommodate you during the recruitment process.

Hybrid Working

Arm’s approach to hybrid working is designed to create a working environment that supports both high performance and personal wellbeing. We believe in bringing people together face to face to enable us to work at pace, whilst recognizing the value of flexibility. Within that framework, we empower groups/teams to determine their own hybrid working patterns, depending on the work and the team’s needs. Details of what this means for each role will be shared upon application. In some cases, the flexibility we can offer is limited by local legal, regulatory, tax, or other considerations, and where this is the case, we will collaborate with you to find the best solution. Please talk to us to find out more about what this could look like for you.

Equal Opportunities

Arm is an equal opportunity employer, committed to providing an environment of mutual respect where equal opportunities are available to all applicants and colleagues. We are a diverse organization of dedicated and innovative individuals, and don’t discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.