Middle QA Automation Engineer (Python/TypeScript, AI Systems)
First seen by Alion on Nov 9, 2026.
We are looking for a QA Automation Engineer experienced in Python or TypeScript test frameworks to join a cross-functional team building an Agentic AI platform. You will prove that a system built on non-deterministic LLM output does what it claims.
Project: a platform for a leading UK energy and utilities provider that makes AI coding agents follow the company’s engineering standards.
Responsibilities:
Design, build and maintain automated test suites for the standards pipeline, the coding-agent plugin and the requirements tool, integrated into CI
Create fixture sets of good and bad code for each rule and measure the accuracy of the checks
Test the plugin end to end in Claude Code and GitHub Copilot
Find cases where an agent bypasses a rule and document them clearly for the team
Collaborate with developers and DevOps to keep quality high across releases
Requirements:
3+ years of experience in QA or test automation
Python or TypeScript test frameworks (pytest, Playwright or similar)
A clear approach to testing non-deterministic output
Hands-on use of AI coding agents and an understanding of where they cut corners
At least 6 hours of overlap with UK working hours
Upper-Intermediate English or higher
Nice to have:
LLM evaluation tools or custom graders
Security testing experience
What We Offer:
Transparent strategy and thoughtful management that keep everyone aligned
Space for personal growth and career development without limits
A culture of continuous learning: internal courses, workshops, and English classes twice a week
Skill up events where we share experience and stay on top of trends
Competitive perks: flexible schedule, a strong compensation package, support for professional certifications, and generous vacation policy
Team spirit that thrives through regular events, gatherings, and celebrations