QA Tech Analyst 12m FTC

Closing Date for Applications

22nd September 2026

Salary Scale: £44,389 - £45,000 per annum

What is the purpose of this job?

RDG’s Quality Assurance Practice is looking for a Technical QA AI Analyst to join the Quasar team. This is a pivotal role blending traditional technical QA - building and documenting system knowledge in the format prescribed by Quasar, maintaining clear traceability between Architectural and QA components, and conducting technical and acceptance testing - with assurance of the AI-enabled solutions being adopted across RDG’s systems.

The role will define how AI solutions are tested and assured: validating LLM-based features, ML models and retrieval-augmented (RAG) pipelines, evaluating AI outputs for accuracy, relevance, consistency and safety, detecting hallucinations and bias, assuring data quality, and testing the robustness and guardrails that keep AI features safe for the railway and its customers.

As the rail industry undergoes considerable change, with transformational programmes across smart ticketing and digital services and a growing adoption of AI across RDG, this role is pivotal in ensuring both technical systems and AI solutions are rigorously assured before release. The role will liaise with Service Assurance, Architecture, CCI, data and technology teams, and technical and management teams from TOCs, suppliers and other industry stakeholders.

What can I expect to do in this job?

This isn’t an exhaustive list, but things you can expect to be involved with include:

· Build and document system knowledge in Confluence in the Quasar-prescribed format - process diagrams, mind maps and requirements that make system components, end-to-end data flows and dependencies clear to all team members.

· Own requirement traceability for assigned systems - baseline and re-baseline traceability matrices, keep coverage accurate through ConnectALL mappings, and support sign-off of requirement acceptance for Quasar.

· Develop and execute test strategies for AI-enabled functionality (LLM features, ML models, RAG pipelines), with acceptance criteria and test oracles suited to non-deterministic systems.

· Evaluate AI outputs for accuracy, relevance, consistency and safety - identifying hallucinations, bias and fairness issues, and managing them through standard defect management.

· Validate AI/ML performance using appropriate metrics (accuracy, precision, recall, F1), assure training and evaluation data quality, and monitor for data drift.

· Conduct adversarial and robustness testing on AI features - red-teaming, prompt injection, edge cases - and validate guardrails and human-in-the-loop controls.

· Contribute to AI governance evidence - risk assessments, explainability documentation, model validation records, and post-deployment monitoring of AI-enabled services.

· Create and deliver all key QA deliverables - test plans, strategies, scope, test objective matrices, test cases, reports and defect management - across waterfall and agile projects.

· Maintain and build functional and non-functional testing using approved automation tools - Selenium, Robot Framework, Python, ReadyAPI, SauceLabs, LoadRunner - automating regression to improve time to market.

· Manage automation scripts and code bases through GitHub and GitLab, supporting testing in a CI/CD and DevOps model, and use AI tools to augment test generation, coverage and productivity.

· Ensure supplier and third-party test delivery (including AI features) is in line with RDG policy and standards, working collaboratively with vendors for release and product quality.

· Report daily testing status to the Test Manager and stakeholders, escalate delays proactively, and share knowledge across the team on technical and AI testing subjects.

Who will my key contacts be?

Internal:

· RDG’s Service Assurance (QA and Accreditation) and Quasar teams.

· RDG Architecture and CCI, including AI governance and data functions.

· RDG Technology Services, RDG IT, and Programme and Project Management teams.

· Internal functions including Audit, Customer Service, and Finance teams.

External:

· TOCs and other stakeholders.

· Technology suppliers, service providers, and AI/ML solution providers.

· Department for Transport.

· Transport for London.

· All related consultants and contracting arrangements within a project or programme of work.

Requirements

What experience, skills and knowledge do I need?

· Proven experience of manual and automation testing of end-to-end solutions, estimation, reporting and defect management, in both waterfall and agile environments, with a strong acceptance test background.

· Experience testing or assuring AI-enabled solutions - LLM features, ML models or RAG pipelines - including output evaluation, hallucination and bias detection, and guardrail validation.

· A good understanding of AI/ML fundamentals: model training and evaluation metrics (accuracy, precision, recall, F1), data quality and drift, LLM concepts (context, prompts, grounding), and testing non-deterministic systems.

· Working knowledge of test tools - JIRA, Confluence, ALM - and hands-on automation experience with PyCharm, Selenium, Robot Framework, Python, ReadyAPI, SauceLabs or LoadRunner.

· Demonstrable knowledge of CI/CD and DevOps models and code management through GitHub and GitLab.

· A keen understanding of system analysis - able to draft clear, testable requirements and acceptance criteria, create process diagrams and mind maps (MS Visio or equivalent), and analyse multiple data sets.

· Ability to simplify and communicate technical requirements, AI behaviour and limitations to non-technical audiences.

· Proficiency in a risk-based approach to testing, requirement traceability and static assurance across all testing stages.

· Bachelor’s degree in Information Technology or related Engineering discipline, with ISTQB/ISEB certification; ISTQB Certified Tester - AI Testing (CT-AI) is highly desirable.

· Most importantly, have an attitude and enthusiasm to drive change, teamwork, enjoy learning, be passionate about what you do, and be driven to grow the established practice into a stable and sustainable service.

Benefits

Why Work for RDG?

We offer a highly competitive package, including:

• 75% discount on rail travel (for work and leisure), plus international rail discounts
• 30 Days annual leave (plus buy/sell options and additional leave for key life events)
• Two Personal Development Days are granted to employees annually
• Interest-free Season Ticket Loan for commuting costs
• Enhanced family leave - 30 weeks full pay for maternity, adoption, surrogacy, and shared parental leave
• Pension scheme - up to 8.64% employer contribution
• Subsidised Private Medical Insurance (Vitality PPP) and Health Cash Plan (Healthshield)
• Access to our flexible benefits portal - a central hub for accessing retail discounts, shopping, dining, travel, and entertainment perks
• Flexible working arrangements available