2,294,097open jobs
145,712companies
270,389added this week
Browse all
Location
In office (Bengaluru, Chennai)
Experience
3+ years exp

First seen by Alion on Nov 16, 2026.

Overview
Company
Impact
Profile match
Your Partner to Transform Data to Intelligence for Business Outcomes. Data, Platform, AI and Gen AI, Analytics and Security.

Job Description - AI Testing Engineer

Experience: 3 - 5 Years

Location: Chennai / Bangalore

Job Summary:

We are looking for an AI Testing Engineer with experience in software testing, Python automation, and validation of AI-powered applications.

Key Responsibilities:

- Perform functional, API, integration, and regression testing of AI applications.

- Validate AI/ML and Generative AI outputs for accuracy and reliability.

- Test LLM responses, prompts, and hallucinations.

- Develop automated test scripts using Python and PyTest.

- Identify defects and collaborate with development teams to resolve issues.

Required Skills:

- 3 - 5 years of experience in software testing.

- Strong Python and test automation skills.

- Experience in API testing using Postman.

- Knowledge of AI/ML, Generative AI, and LLM testing.

- Hands-on experience in evaluation testing (Eval Testing) of AI/LLM applications, including response quality, accuracy, relevance, and consistency.

- Understanding of SQL, Git, and defect-tracking tools such as Jira.

Preferred:

- Hands-on experience with RAG applications and AI evaluation frameworks such as DeepEval or Ragas.

Skills

Automation Testing, Software Quality Assurance, Python, Generative AI, LLM, Testing, PyTest, API Testing, Machine Learning

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,294,097 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

QA
Similar stack
Same company
Bengaluru
Integration Engineer 2 hours ago
In office · 4+ years exp · Gurgaon
Python
Databases
Apache Kafka
DevOps
Rest API
SOAP
Apply
In office · 5+ years exp · Chennai
Python
Java
DevOps
CI/CD
Analytics
ETL/ELT
Management
Agile
Apply
Sr Software Tester 2 hours ago
Hybrid · Full-Time · 4+ years exp · Bachelor's Degree · Pune
DevOps
Rest API
Management
Jira
Agile
Scrum
QA
Postman
SoapUI
Apply
In office · Full-Time · 5+ years exp · Bachelor's Degree · Pune
Python
Java
TypeScript
AI/ML
Copilot
Claude
DevOps
Rest API
Azure
CI/CD
AWS
Amazon EC2
SOAP
Apache HTTP Server
Management
Agile
Scrum
QA
JMeter
Playwright
MedTech
CAPA
Apply
Remote (India) · Full-Time · 5+ years exp · Bengaluru
Python
JavaScript
TypeScript
SQL
C#
Databases
MS SQL
AI/ML
NLP
Frontend
Angular
DevOps
Rest API
GitHub Actions
Azure
CI/CD
Jenkins
Self-Healing
QA
Selenium
Playwright
Apply
In office · Full-Time · Bachelor's Degree · Bengaluru
Python
C++
Apply
In office · 2+ years exp · Bachelor's Degree · Bengaluru
SQL
DevOps
Unix
Management
Oracle ERP
Apply
In office · Bachelor's Degree · Bengaluru
DevOps
Splunk
GCP
Incident Management
Linux
Windows
Unix
Cybersecurity
IBM QRadar
PKI
SIEM
Management
ServiceNow
Apply
In office · 5+ years exp · Bachelor's Degree · Bengaluru
Management
Oracle ERP
Apply
In office · Bachelor's Degree · Bengaluru
Analytics
Master Data Management
Management
Agile
Scrum
Oracle ERP
Apply
See all jobs
This is one of many
2,294,097 more open roles from verified company boards, updated every day.