{"id":1307264,"url":"https://alion.io/job/doublu-qa-ai-engineer","title":"QA AI Engineer","company":{"id":3812749,"name":"DoublU","domain":"godoublu.com","url":"https://alion.io/company/doublu","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"QA","role_family":"QA","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Noida, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":24000,"max_usd":60000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":599},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"CI/CD","optional":false},{"name":"Embeddings","optional":false},{"name":"GraphQL","optional":false},{"name":"Hallucination","optional":false},{"name":"Knowledge Graph","optional":false},{"name":"LLM","optional":false},{"name":"RAG","optional":false},{"name":"Rest API","optional":false},{"name":"Semantic Search","optional":false},{"name":"Semantic Search","optional":false},{"name":"SQL","optional":false},{"name":"AWS","optional":true},{"name":"Azure","optional":true},{"name":"Databricks","optional":true},{"name":"Fabric AI","optional":true},{"name":"Microsoft Fabric","optional":true},{"name":"Python","optional":true},{"name":"Snowflake","optional":true}],"status":"live","first_seen_at":"2026-09-26T13:24:01Z","employer_posted_date":null,"last_verified_at":"2026-09-26T13:24:01Z","board_verified":false,"closed_at":null,"days_open":1,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":1},"description":"QA AI Engineer\n\nRole Overview:\n\nWe are looking for a QA AI Engineer to design and execute quality engineering strategies for Generative AI, RAG, Agentic AI, AI assistants, semantic search, and AI-enabled data solutions.\n\nThe role will focus on validating AI solutions across data quality, model outputs, retrieval accuracy, response quality, reliability, performance, security, and functional correctness.\n\nThe ideal candidate should have a strong understanding of AI/ML concepts and experience testing modern AI applications and data-driven solutions in enterprise environments.\n\nExperience in a Healthcare Payer environment will be an added advantage.\n\nKey Responsibilities:\n\nAI & GenAI Testing:\n\n- Design and execute test strategies for Generative AI, RAG, Agentic AI, AI assistants, and semantic search solutions.\n\n- Validate AI-generated responses for accuracy, relevance, consistency, completeness, and factual correctness.\n\n- Develop test scenarios covering different user queries, prompts, contexts, and business use cases.\n\n- Evaluate LLM responses against defined business and quality benchmarks.\n\n- Test prompt variations, context handling, hallucination scenarios, and edge cases.\n\n- Validate AI workflows involving multiple agents, tools, APIs, and data sources.\n\n- Support the creation of AI evaluation frameworks and quality metrics.\n\nRAG & Vector Search Testing:\n\n- Validate RAG pipelines across ingestion, chunking, embedding, retrieval, context generation, and response generation.\n\n- Test vector databases, embeddings, similarity search, and retrieval accuracy.\n\n- Evaluate precision, recall, relevance, and ranking of retrieved information.\n\n- Identify issues related to incorrect, incomplete, outdated, or irrelevant context.\n\n- Validate semantic search and knowledge retrieval capabilities.\n\n- Test knowledge graph-based retrieval and AI-assisted data consumption where applicable.\n\nData Quality & Validation:\n\n- Design test cases to validate data accuracy, completeness, consistency, uniqueness, and integrity.\n\n- Validate data pipelines and datasets used by AI/ML applications.\n\n- Test data ingestion, transformation, enrichment, and processing workflows.\n\n- Identify data quality issues that could impact model or AI application performance.\n\n- Validate data lineage and source-to-target mappings.\n\n- Support testing of AI-ready datasets and semantic data layers.\n\nAI Model & Application Validation:\n\n- Validate AI/ML models and applications against functional and business requirements.\n\n- Test model outputs across different datasets, scenarios, and user personas.\n\n- Perform regression testing for AI models and GenAI applications following model, prompt, or data changes.\n\n- Evaluate model performance and identify degradation over time.\n\n- Test model behaviour for boundary conditions, unexpected inputs, and failure scenarios.\n\n- Support validation of explainability, transparency, fairness, and responsible AI requirements.\n\nAPI & Integration Testing:\n\n- Test REST, GraphQL, event-driven, and streaming APIs supporting AI applications.\n\n- Validate API responses, data contracts, authentication, authorization, error handling, and performance.\n\n- Test integrations between AI applications, databases, vector stores, data platforms, and enterprise systems.\n\n- Perform integration and end-to-end testing across distributed AI workflows.\n\nPerformance & Reliability Testing:\n\n- Conduct performance and scalability testing for AI applications and data services.\n\n- Validate response time, throughput, concurrency, and system stability.\n\n- Identify performance bottlenecks across APIs, data pipelines, retrieval systems, and AI services.\n\n- Test application behaviour under high-volume and high-concurrency scenarios.\n\n- Validate reliability and recovery mechanisms for AI-enabled applications.\n\nAutomation & Quality Engineering:\n\n- Develop and maintain automated test frameworks for AI and data applications.\n\n- Automate functional, regression, API, data validation, and integration test scenarios.\n\n- Build reusable test utilities and validation frameworks.\n\n- Integrate automated testing into CI/CD pipelines.\n\n- Maintain test data, test suites, test reports, and quality dashboards.\n\n- Support continuous quality improvement across AI development lifecycle.\n\nAI Governance & Responsible AI:\n\n- Validate AI solutions against defined security, governance, privacy, and responsible AI requirements.\n\n- Test for sensitive data exposure, inappropriate responses, and policy violations.\n\n- Support testing related to AI explainability and traceability.\n\n- Validate compliance requirements applicable to enterprise and healthcare data.\n\n- Document quality risks, defects, test outcomes, and mitigation recommendations.\n\nTech Stack:\n\n- Generative AI / LLMs, RAG, Agentic AI, Vector databases, Embeddings, Knowledge graphs, Semantic search\n\n- Python, SQL, REST / GraphQL APIs\n\n- Azure / AWS, Databricks, Snowflake, Microsoft Fabric\n\n- AI/ML testing frameworks, CI/CD and test automation tools\n\nHealthcare Domain (Good to Have):\n\n- Experience working with Healthcare Payer data or applications (Claims, Membership, Provider, Care Management, Utilization Management, Healthcare analytics, Medical or pharmacy data).\n\n- Understanding of healthcare data privacy, governance, and regulatory requirements.\n\nSkills\nPython, Automation Testing, SQL, API Testing, Software Quality Assurance, Generative AI, RAG, AI/ML Testing, Data Quality, Validation Testing, Functional Testing","description_format":"text","description_chars":5489,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-09-26T14:00:00Z"}],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":1,"expected_fill_days":23,"reasons":["seen:1","velocity","win:early","comp:attention"],"computed_at":"2026-09-28T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/doublu-qa-ai-engineer","json_url":"https://alion.io/job/doublu-qa-ai-engineer.json","meta":{"generated_at":"2026-09-28T06:25:41Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","usage":{"tier":"crawler","counted_by":"address","units_charged":1,"used_today":4463,"day_limit":5000,"remaining_today":537,"minute_limit":60,"resets_at":"2026-09-29T00:00:00Z"}}}