Software Engineer - AI Evaluation
Confirmed on the employer's own hiring board on Oct 12, 2026. First seen by Alion on Oct 12, 2026.
The Role
AI is at the heart of what Board Intelligence builds. Boards and senior leaders rely on our AI to support their most important decisions, so the quality of its output has to be close to perfect.
We are looking for a Software Engineer to join our AI team as its dedicated engineer. The team includes prompt engineers and subject matter experts, and you will build the systems they need. Your first priority is our AI evaluation suite: the tools we use to measure and improve the quality of AI output across our products. Over time, you may move on to building one of our AI-powered products.
You do not need deep AI or machine learning experience; the team will help you get up to speed quickly. What matters is that you are a strong full-stack engineer who is comfortable in ambiguity and happy to build largely independently, in a field where good practice is still being defined.
Main Responsibilities
Design and build our AI evaluation suite, the systems we use to measure and improve the quality of AI output
Build across the full stack yourself, from data pipelines and backend services to the front-end tools our experts use
Work closely with prompt engineers and subject matter experts, turning their knowledge into tests, metrics and tooling
Help define how we measure AI quality, in a field where practices are still forming
Make evaluation processes and results automated as part of our SDLC, so product teams can ship with confidence
Over time, contribute to building our AI-powered products
Person Specification
Upper mid-level to senior professional software engineering experience
Experience building AI powered products or tooling
Full-stack experience across front end and back end
Comfort working independently and in ambiguity, setting your own direction
Experience collaborating with non-engineers and turning their expertise into working software
A strong focus on quality, testing and attention to detail
Experience using AI coding tools in your day-to-day work
Fluent written and spoken English
Experience with building evaluation suites for AI products
Experience with Python and TypeScript
Experience building internal tools, testing frameworks or data pipelines
Experience at a startup or in a small team where you built things end to end
After three months: the first version of the evaluation suite is running and our engineering teams are using it
After six months: evaluations cover all of our AI products and are integrated into the SDLC
After twelve months: evaluation is a standard part of how we ship AI, and you are a trusted voice on AI quality across engineering
We are looking for a strong all-round engineer who is comfortable working independently, enjoys ambiguity and wants to work at the forefront of AI quality.
Required Skills and Experience
Nice to haves
What good looks like in this role