Senior QA Engineer – Agentic AI Testing & Evaluation

About the Role Dailoqa is looking for an experienced Senior QA Engineer - Agentic AI Testing & Evaluation to own the quality, testing, and evaluation strategy for our Broccoli Platform. This role will be responsible for establishing robust testing frameworks for AI agents, multi-agent workflows, orchestration layers, tools, and enterprise-grade AI applications. The ideal candidate will have deep expertise in test automation, quality engineering, AI/ML validation, security testing, and performance engineering, with a strong understanding of Large Language Models (LLMs), Agentic AI systems, and modern DevOps practices. You will work closely with Capability Leads, AI Architects, Product Teams, and Platform Engineers to ensure that Broccoli delivers reliable, secure, scalable, and high-performing AI solutions.

Requirements Required Skills &

Qualifications Technical Expertise 7-10 years of experience in Quality Assurance, Test Automation, or Quality Engineering. Strong experience with automation frameworks using Python, Java, or JavaScript/TypeScript. Hands-on experience with: Selenium, Playwright, Cypress, or Robot Framework API testing tools such as Postman, Rest Assured, or Karate CI/CD platforms including Azure DevOps, GitHub Actions, Jenkins, or GitLab Experience with cloud platforms such as Azure, AWS, or Google Cloud. Agentic AI & LLM Testing Understanding of: Large Language Models (LLMs) Retrieval-Augmented Generation (RAG) AI Agents and Agent Orchestration Frameworks Prompt Engineering and Evaluation Experience with AI evaluation platforms and frameworks is preferred. Knowledge of hallucination detection, safety validation, guardrails, and model benchmarking. Performance & Security Testing Experience with performance testing tools such as JMeter, k6, or Locust. Knowledge of application security testing and threat modeling. Familiarity with AI security testing, adversarial testing, and governance controls.

Soft Skills Strong analytical and problem-solving skills. Excellent stakeholder management and communication abilities. Experience working in cross-functional Agile teams. Ability to lead quality initiatives in fast-paced, innovation-driven environments.

Preferred

Qualifications Experience testing AI-powered enterprise applications. Exposure to Microsoft Azure AI, OpenAI, LangChain, LangGraph, CrewAI, Semantic Kernel, or similar agent frameworks. Certifications such as: ISTQB Advanced Test Analyst/Test Manager Certified Agile Tester Azure AI Engineer Associate Azure DevOps Engineer Expert Experience working with BFSI, enterprise transformation, or regulated industry environments.