{"id":113366,"url":"https://alion.io/job/quantiphi-associate-lead-testing-qa-mlops","title":"Associate Lead - Testing (QA + MLOps)","company":{"id":6155,"name":"Quantiphi","domain":"quantiphi.com","url":"https://alion.io/company/quantiphi","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"A","score":88,"open_postings":71,"ghost_share":0,"stale_share":0.493,"repost_share":0,"time_to_fill_p50_days":30,"computed_at":"2026-10-17T05:46:00Z"}},"role":"QA","role_family":"QA","seniority":"lead","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India","Mumbai, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":17000,"max_usd":50000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":8},"experience_years_min":10,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Amazon CloudWatch","optional":false},{"name":"Amazon Kinesis","optional":false},{"name":"Amazon S3","optional":false},{"name":"Amazon SageMaker","optional":false},{"name":"API Gateway","optional":false},{"name":"AWS","optional":false},{"name":"AWS Bedrock","optional":false},{"name":"AWS Glue","optional":false},{"name":"AWS Lambda","optional":false},{"name":"CI/CD","optional":false},{"name":"CloudFormation","optional":false},{"name":"Embeddings","optional":false},{"name":"Feature Store","optional":false},{"name":"Git","optional":false},{"name":"Hallucination","optional":false},{"name":"IAM","optional":false},{"name":"LangChain","optional":false},{"name":"LLM","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"Multi-Agent Systems","optional":false},{"name":"NLP","optional":false},{"name":"Prompt Engineering","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"RAG","optional":false},{"name":"Scikit-learn","optional":false},{"name":"Synthetic Data","optional":false},{"name":"TensorFlow","optional":false},{"name":"Terraform","optional":false}],"status":"live","first_seen_at":"2026-03-24T00:00:00Z","employer_posted_date":"2026-03-24","last_verified_at":"2026-10-17T08:16:41Z","board_verified":true,"closed_at":null,"days_open":207,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":207},"description":"While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.\nIf working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!\nRole: Lead/Associate Lead - QA + MLOps & Generative AI\nExperience: 10+ years\nLocation: Mumbai/Bangalore (Hybrid)\nKey Responsibilities:\nAI/ML & GenAI Testing Strategy (AWS Ecosystem)\nDefine testing approaches for AI systems built on AWS services such as:\nAmazon SageMaker\n\nAmazon Bedrock\n\nAWS Lambda\n\nAmazon API Gateway\n\nAmazon Kinesis\n\nAWS Glue\n\nAmazon S3\n\nAmazon CloudWatch\n\nDesign validation frameworks covering:\nModel accuracy & performance validation\n\nData drift & concept drift detection\n\nHallucination detection for LLMs\n\nPrompt robustness testing\n\nRAG validation (retrieval accuracy + grounding)\n\nBias & fairness validation\n\nSafety & toxicity testing\n\nMLOps Quality Engineering (AWS-Centric)\nValidate the end-to-end ML lifecycle including:\nData ingestion & feature pipelines\n\nModel training & hyperparameter tuning\n\nModel versioning & registry\n\nDeployment validation\n\nCanary & blue/green release validation\n\nWork with AWS-native services such as:\nSageMaker Pipelines\n\nSageMaker Model Monitor\n\nSageMaker Feature Store\n\nBedrock model evaluation workflows\n\nCloudWatch-based observability\n\nImplement CI/CD quality gates for ML pipelines integrated with AWS DevOps tools.\nGenAI & Agentic AI Testing\nDefine quality engineering approaches for:\nLLM-based applications using Amazon Bedrock\n\nPrompt engineering validation\n\nMulti-agent orchestration testing\n\nChatbot & Voice bot conversational testing\n\nIntent classification validation\n\nConversation drift & fallback validation\n\nAPI contract validation for LLM integrations\n\nBuild reusable evaluation harnesses for:\nBLEU / ROUGE scoring\n\nEmbedding similarity scoring\n\nResponse consistency\n\nSafety scoring frameworks\n\nFramework & Capability Development\nDesign reusable AI testing accelerators\n\nCreate AWS-aligned AI test automation frameworks (Python-first)\n\nDevelop synthetic data generation strategies\n\nEstablish AI quality scorecards\n\nBuild an internal AI QA Center of Excellence\n\nClient Engagement & Leadership\nLead AI/ML quality strategy workshops\n\nPerform AI risk & readiness assessments\n\nPresent quality architecture to CXOs\n\nDrive QA transformation programs\n\nMentor QA teams on AWS-based AI testing\n\nOwn delivery for AI testing engagements end-to-end\n\nMust have skills:\nTesting Expertise\n8-12+ years in Quality Engineering\n\nStrong test strategy, automation & governance experience\n\nExperience leading QA transformation initiatives\n\nExperience building frameworks from scratch AI/ML & GenAI Expertise\n\nDeep understanding of ML lifecycle\n\nExperience testing ML models (NLP preferred)\n\nHands-on experience validating LLM applications\n\nStrong understanding of:\n\nPrompt engineering\n\nRAG architecture\n\nEmbeddings\n\nBias & explainability AWS AI/ML Expertise\n\nHands-on experience with:\n\nAmazon SageMaker (training, deployment, monitoring)\n\nAmazon Bedrock (LLM integration & evaluation)\n\nS3-based data pipelines\n\nAWS IAM (security validation)\n\nCloudWatch monitoring\n\nLambda & API Gateway integrations\n\nAWS CI/CD (CodePipeline / CodeBuild preferred)\n\nUnderstanding of:\nInfrastructure as Code (Terraform / CloudFormation)\n\nObservability in AI systems\n\nCost monitoring for ML workloads\n\nTechnical Skills\nPython (mandatory)\n\nExperience with ML libraries (Scikit-learn, TensorFlow, PyTorch)\n\nExperience with LLM frameworks (LangChain, etc.)\n\nAPI & automation testing frameworks\n\nGit-based workflows\n\nLeadership & Communication\n\nStrong client-facing communication\n\nExperience leading QA teams\n\nAbility to create strategy decks & solution proposals\n\nStrong stakeholder management\n\nIf you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!","description_format":"text","description_chars":4040,"description_truncated":false,"requirements":{"experience_years_min":10,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["AI Consulting & Integration","Document AI","Cloud Consulting & Migration","Data Engineering & Migration Services"],"lifecycle":[{"event":"open","at":"2026-03-24T00:00:00Z"}],"visa":[],"liveness":{"score":9,"band":"cold","label":"Long shot","p_open":1,"p_active":0.337,"p_room":0.28,"age_days":207,"expected_fill_days":30,"reasons":["conf:1","velocity","win:tail","crowd:"],"computed_at":"2026-10-17T05:46:00Z"},"pay":null,"html_url":"https://alion.io/job/quantiphi-associate-lead-testing-qa-mlops","json_url":"https://alion.io/job/quantiphi-associate-lead-testing-qa-mlops.json","meta":{"generated_at":"2026-10-17T12:35:31Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":16702,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":6155},"rest":"https://alion.io/mcp/rest/get_company?id=6155"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fquantiphi-associate-lead-testing-qa-mlops"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fquantiphi-associate-lead-testing-qa-mlops"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fquantiphi-associate-lead-testing-qa-mlops"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/quantiphi-associate-lead-testing-qa-mlops\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fquantiphi-associate-lead-testing-qa-mlops"}]}