AI Quality Engineer
First seen by Alion on Aug 31, 2026.
About the Company:
Greetings from Teamware Solutions, a division of Quantum Leap Consulting Pvt. Ltd. Teamware Solutions is a 20+ year industry leader providing IT staffing and consulting services, supporting 4000+ professionals across India, the USA, the Middle East, and APAC for leading GCC clients.
About the Role:
We are hiring an AI Quality Engineer / QE Consultant Playwright & GenAI Testing.
Location: Bangalore.
Work Model: Hybrid.
Experience: 5 to 8 years.
Notice Period: Immediate 15 days.
Responsibilities:
We are seeking a highly motivated Quality Engineering (QE) Consultant with 46 years of experience in software testing, automation, and emerging AI technologies. This role will focus on ensuring the quality, reliability, safety, and performance of AI-powered solutions.
- AI/GenAI Testing: Test LLM applications, AI agents, copilots, and conversational systems for accuracy, relevance, groundedness, hallucinations, consistency, and safety.
- Playwright Automation: Build and maintain Playwright frameworks for UI, API, and E2E automation.
- API Testing: Perform API testing using Postman, REST Assured, and Playwright.
- AI Evaluation: Define AI quality metrics such as relevancy, semantic similarity, groundedness, precision/recall, and human evaluation.
- CI/CD: Integrate automation into CI/CD pipelines and support shift-left QE practices.
- Collaboration: Work with Product, Engineering, Data Science, and AI teams to identify risks and improve quality.
Technical Skills :
- Strong expertise in Playwright automation.
- Hands-on experience with API testing and automation.
- Experience validating AI/LLM-based applications and AI agents.
- Familiarity with M365 Copilot, OpenAI/Azure OpenAI, Claude, Gemini, and Agentic AI frameworks.
- Knowledge of test management tools such as Azure DevOps (ADO) or Jira.
Preferred Skills :
- Exposure to Python or TypeScript.
- Experience with AI evaluation frameworks (RAG evaluation, Prompt evaluation, LLM benchmarking).
- Knowledge of Responsible AI principles and model risk management.
Education :
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
Skills
Playwright Testing, API Testing, Artificial Intelligence, Software Quality Assurance, Python, AI/ML Testing, Generative AI