Senior AI QA Engineer
First seen by Alion on Sep 26, 2026.
About the Role:
We are looking for experienced QA professionals who understand how to test AI systems and can take ownership of AI quality and evaluation. We are looking for ownership-driven QA professionals who can solve problems, challenge AI behavior and improve quality rather than simply execute test cases.
Key Responsibilities:
- Create AI test scenarios, datasets and evaluation frameworks
- AI response evaluation covering accuracy, relevance, consistency and hallucinations
- Perform AI safety, bias and reliability testing
- Own QA strategy and improve testing processes
Tech Stack & Requirements:
- 7 - 12 years of overall QA/Testing experience
- Hands-on experience with AI/ML/GenAI/LLM application testing
- Strong Manual and Automation Testing background across Web, API and Mobile
- Strong Python-based automation and API testing skills
- Experience with Postman, BrowserStack/TestMu AI or similar tools
- Good understanding of Prompt Engineering and ability to assess the impact of prompt changes
- Agile development experience
- Strong analytical, exploratory and problem-solving skills
- Strong communication and stakeholder management skills
- Healthcare domain experience will be an added advantage.
Skills
Python, Manual Testing, API Testing, Artificial Intelligence, Testing, Automation Testing, Application Testing, AI/ML Testing