AdvaRisk - QA Lead - Automation Testing
Smart Apply
Similar jobs
Location
In office (Pune)
Seniority
Staff
Employment
Full-Time
First seen by Alion on Nov 13, 2026.
Overview
Company
Impact
Profile match
Our mission Make lending fast, simple & trustworthy AdvaRisk is focused on digitizing the current mortgage based lending process to make the decisioning fast, simple & trustworthy.
Key Roles & Responsibilities
- Own and drive the end-to-end test automation strategy, defining the roadmap, frameworks, tooling and standards for UI, API, backend and integration testing.
- Lead, mentor and manage the QA team, owning delivery, prioritization, test coverage and quality outcomes.
- Work closely with engineering, product, data science/ML and client service teams to clarify requirements and define acceptance criteria.
- Create and maintain test plans, test scenarios, test data, traceability and release-level quality reports.
- Triage defects, assess business and customer impact, and coordinate defect prioritization with relevant teams.
- Test AI/ML model integrations end to end, including request/response contracts, preprocessing and post-processing, timeout/retry behavior, failure handling and fallback paths.
- Build model-focused test suites using representative and edge-case datasets; assess output correctness against agreed reference labels or acceptance criteria, and track metrics such as precision/recall where applicable.
- Validate LLM-enabled workflows for groundedness, consistency, hallucination risks, prompt/input sensitivity and regression when prompts, models or versions change; use defined evaluation criteria rather than subjective checks alone.
- Check model-service latency, throughput, rate limits, availability, version compatibility and behavior under malformed, incomplete or unexpected inputs.
- Perform end-to-end, system integration and regression testing across UI, backend services, APIs, databases and third party/model services.
- Establish performance baselines and report findings with actionable recommendations for capacity and scalability improvements.
- Plan and execute performance, load, stress, spike and endurance tests; analyze throughput, latency, concurrency, resource utilization and bottlenecks.
- Work with developers to reproduce issues, verify fixes and prevent regressions throughout the release cycle.
- Own test-suite execution in non-production environments and integrate automated checks into CI/CD pipelines where appropriate.
- Strong experience testing web applications across UI, backend services, APIs, databases and end-to-end workflows.
- Strong QA fundamentals with substantial hands-on automation experience using Python; able to design, build, maintain and scale automation frameworks rather than only execute existing scripts.
- People leadership experience, including leading, mentoring and driving the output of a team of 3 or more QA engineers.
- Strong API testing and debugging skills, including REST APIs, response/schema validation, authentication, error handling and service-to-service integration.
- Experience with performance-testing tools such as JMeter, Locust, k6 or equivalent, and ability to interpret results and identify bottlenecks.
- Working knowledge of AI/ML model evaluation and integration testing; able to define test datasets, expected outcomes, thresholds and regression checks in collaboration with data science/ML engineers.
- Familiarity with testing LLM-based features, including prompt/version regression, groundedness and response-quality evaluation, structured-output validation and safe failure handling.
- Comfortable validating data and service behavior using SQL, logs, metrics and traces; familiarity with CI/CD-based automated test execution is an advantage.
- Good understanding of front-end technologies and browser-based testing, including JavaScript, HTML and CSS.
- Hands-on experience building and maintaining automation frameworks using tools such as Pytest, Robot Framework, Selenium, Cypress or equivalent; experience with API automation is essential.
- Experience working in Agile/Scrum teams and participating in sprint planning, refinement, reviews and release readiness.
- A strong quality mindset and a track record of promoting automation-first, risk-based testing and continuous quality improvement.
- Clear communication skills and the ability to work across engineering, product, QA and AI/ML teams to define and deliver quality outcomes.
- Be excited by and have a positive outlook to navigate ambiguity.
- Passion for results and excellence.
- Team player.
- Must be able to get the job done by working collaboratively with others.
- Inquisitive and analytical mind; out-of-the-box thinking.
- Prioritize among competing opportunities, balancing consumer needs with business and product priorities, and clearly articulate the rationale behind product decisions.
- Straightforward and professional.
- Good communicator.
- Maintain high energy and motivate the team.
- A do-it-yourself orientation, consistent with the company's roll-up-the-sleeves culture.
- Proactive.
Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,276,528 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Free forever. No card. Under a minute.
Your match
How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.
Recommended for you based on this role
QA
Similar stack
Same company
Pune
QA Engineering Team Lead
1 hour ago
Remote (India) · 5+ years exp · India
AI/ML
Multi-Agent Systems
Management
Agile
Scrum
QA
Selenium
Cypress
Playwright
Appium
Apply
Senior Staff SDET
1 hour ago
In office · Full-Time · 8+ years exp · Pune
Python
Go
Java
TypeScript
C#
Mobile
JUnit
DevOps
CI/CD
Chaos Engineering
IAM
Cybersecurity
Okta
Shift-Left Security
Microsoft Entra ID
Active Directory
Management
ServiceNow
QA
TestNG
Selenium
Cypress
Playwright
Rest-Assured
Pytest
Pact
Apply
Apply
Lead
4 hours ago
In office · Full-Time · 6+ years exp · Bengaluru
JavaScript
TypeScript
Databases
Snowflake
Frontend
React.js
DevOps
Rest API
Azure
AWS
Cybersecurity
Microsoft Entra ID
Management
Power Automate
Power Apps
SharePoint
Apply
Lead Data - QA
3 hours ago
In office · 6+ years exp · Hyderabad
Python
SQL
Databases
Snowflake
Amazon Redshift
DevOps
AWS
Cybersecurity
HIPAA
Analytics
ETL/ELT
Apply
.Net Selenium SDET QA
7 hours ago
≈ $11k – $27k per year (Estimated) · Hybrid · 5+ years exp · Pune
SQL
C#
C#
.NET
Databases
DynamoDB
DevOps
Rest API
GitHub Actions
CI/CD
Jenkins
AWS
Docker
AWS Lambda
Amazon EC2
Amazon CloudWatch
API Gateway
Cybersecurity
Shift-Left Security
QA
Selenium
Apply
Senior Test Automation Engineer
7 hours ago
≈ $11k – $27k per year (Estimated) · Hybrid · 5+ years exp · Bachelor's Degree · Pune
Python
JavaScript
DevOps
CI/CD
Git
Management
Agile
Scrum
QA
Cucumber
WebDriverIO
Apply
Senior System Engineer- Digital Platform
6 hours ago
In office · Full-Time · Bachelor's Degree · Gimo · Pune
SQL
Databases
Redis
Db2
DevOps
Rest API
Azure
Jenkins
Docker
Kubernetes
TCP/IP
IoT
MQTT
OPC UA
Management
Jira
Apply
Condition Monitoring/ Asset Reliability
6 hours ago
In office · 2+ years exp · Pune
AI/ML
Anomaly Detection
DevOps
Prometheus
SpaceTech
FMEA
Apply
Forward Deployed Engineer
6 hours ago
In office · Bengaluru · Mumbai · Pune
Python
AI/ML
Prompt Engineering
Function Calling
AI Agents
LLM
RAG
Structured Outputs
LLM Guardrails
DevOps
Rest API
AWS
Apply
This is one of many
2,276,528 more open roles from verified company boards, updated every day.