Test Manager, AI

The Test Manager - AI is responsible for leading the organisation's quality engineering and AI testing strategy, driving the adoption of innovative testing practices, intelligent automation, and AI-enabled quality solutions across the technology landscape. Operating within a complex enterprise environment, the role provides testing leadership across multiple programs, ensuring the delivery of high-quality, reliable, scalable, and secure solutions that align with business objectives.

This role is accountable for shaping and advancing next-generation quality engineering capabilities by integrating Artificial Intelligence (AI) and Machine Learning (ML) into the testing lifecycle. The Test Manager - AI will champion the transformation of traditional testing approaches through the implementation of intelligent, predictive, and self-optimising testing frameworks that enhance efficiency, quality, and delivery outcomes.

Leading global testing teams and collaborating closely with business and technology stakeholders, the role fosters a culture of quality, innovation, and continuous improvement while driving testing maturity, operational excellence, and the adoption of industry-leading practices across the organisation.

Key Accountabilities and main responsibilities

Strategic Focus

• Formulate the strategic vision and technical roadmap for integrating AI and Generative AI into the testing lifecycle. Oversee the deployment of intelligent frameworks optimizing automated test case generation, autonomous test execution, and computer-vision visual validations.
• Manage and accountable for a Portfolio of programs/projects to ensure successful delivery and taking Business on the quality journey.
• Strong due diligence and analysis skills towards test solution designing.
• Oversee the modernization of scalable enterprise test automation frameworks embedded seamlessly into high-velocity CI/CD continuous delivery pipelines.
• Deploy AI/ML analytical tools to evaluate historical defect logs, identifying risk patterns, predicting system regressions, and proactively adjusting test focus areas.
• Formulate an effective, practical but robust Test Strategy by coordinating with all relevant stakeholders considering a highly complex financial services environment.
• Assess any quality gaps and keep the stakeholders informed in timely manner and suggest solution to resolve any blockers.
• Identify areas of improvement and establish a continuous improvement process.

Operational Management

• Manage the Full E2E testing lifecycle of assigned projects in a multi-vendor and in distributed delivery environment.
• Evaluate, proof-of-concept, and manage commercial relationships with emerging AI testing vendors, maximizing ROI across tools, frameworks, and resource allocations.
• Review program/project level business requirements, technical specifications, end to end Business Processes and solution design documents, highlighting any gaps or potential improvements.
• Define and deploy defect preventive process and measures in distributed delivery model.
• Defect Management activities (prioritization, including, conducting defect triage meeting with business, technology & interface partners); work with relevant stakeholders to agree on workaround for outstanding defects and defects resolutions.
• Participate in the planning of UAT, PVT for Programs and Projects (including approach and scope of PVT).
• Facilitates the development, implementation, and adherence of testing department standards in partnership with other test team leaders.
• Establish key performance indicators and quality metrics to measure and report on the effectiveness of the testing process.
• Communicate quality metrics to relevant stakeholders.
• Review reports produced by the testing teams, provide guidance, evolve process, and actively participate in resolving issues/roadblocks to testing progress.
• Contribute to continuous improvement initiatives by identifying opportunities to enhance AI testing frameworks, processes, and automation capabilities.
• Provide guidance and knowledge sharing to project teams on AI testing methodologies, Responsible AI validation, test automation approaches, and emerging industry practices.
• Mentor the testing team and contribute to building AI testing capabilities across teams through coaching, standards adoption, and continuous learning initiatives.

People Leadership

• Support the project team in leveraging testing tools to improve team productivity and effectiveness (Test Management, Defect Management, and Test Automation, Performance Testing, AI Driven Testing).
• Work within a team contributing to:
• Timely delivery of assigned tasks and project outcomes.
• Effective use of technology to solve business challenges and improve operational efficiency.
• Innovation and continuous improvement through the development of practical and scalable solutions.
• Motivate the team to consistently deliver quality results through collaboration, accountability, and innovation.

Governance & Risk

• Review and sign-off of testing artefacts (Test Plans and Test Summary Reports), liaising with respective project team members (including Development teams, Test teams, Business, Solution Designers, Architects, IT, PMs, and other stakeholders.)
• Governance and management of vendor achievement against Service Level Agreements, measuring performance against established metrics and evaluating third-party operations for quality and adherence to agreed schedule and budget requirements to align delivered Services with business IT needs and key stakeholder expectations.
• Manage and document the quality assurance activities with internal reporting and audits.
• Validate adherence to Responsible AI principles, regulatory requirements, organizational policies, and governance standards.
• Conduct and support the assessment and reporting of AI-related risks, including model bias, explainability, fairness, and data quality concerns.
• Ensure maintenance of appropriate test evidence, validation reports, defect records, and audit documentation to support governance and compliance requirements.
• Escalate risks, issues, and control gaps that may impact AI solution quality, regulatory compliance, or business outcomes.

The above list of key accountabilities is not an exhaustive list and may change from time-to-time based on business needs.

Experience & Personal Attributes

Experience

• Tertiary qualifications in Information Technology, Business, or a related discipline
• 14+ years of authentic experience in Software Testing, Quality Assurance, and Engineering.
• Minimum of 5 years of experience in direct managerial or organizational leadership capacities, directly managing distributed engineering teams in complex E2E integrated financial platform.
• Practical experience utilizing Generative AI models (e.g., OpenAI API, GitHub Copilot) for building intelligent synthetic data, structural test scaffolding, and automating framework maintenance.
• Knowledge or functional experience in testing and handling validation patterns specific to AI/ML software ecosystems.
• Extensive experience in managing a multi-vendor engagement.
• Extensive experience in building process, practices, and test team in distributed delivery model
• Strong E2E test management background within a fast-paced business and in complex IT application landscape.
• Strong commercial / business acumen
• Experience in CI/CD pipeline and tools such as Azure Devops
• Accounting/Superannuation/Insurance/BFSI highly desirable.
• Core automation experience in Selenium WebDriver, Playwright, Cypress, or Appium.
• Experience in API testing, Mobile, App testing highly desirable.
• Certification in Domain/Testing/Delivery methodology like ISTQB, Agile, ASFA highly desirable
• Working experience working in an Agile scrum environment, expert in JIRA and Confluence or similar tools
• Hands on experience in improving Testing & QA capabilities with a strong awareness of industry developments and best practice.
• Managing the mobilization and operation of Testing services.
• A strong background in financial services & pensions industry will be highly advantageous.

Personal Attributes

• Demonstrate strong thought leadership, strategic thinking, and effective communication in driving and uplifting Quality through all stages of the SDLC - and within the Test practice.
• Strong influential skills with an emphasis on building quality output, and continuous improvement
• Effective verbal and written communication skills, needed to communicate with global teams/stakeholders.
• Practical and simple problem-solving approach.
• Capable of managing high pressure and short timelines, with the willingness to be flexible
• Ability to handle multiple tasks and projects simultaneously and prioritize them