QA Engineer - AI Applications

We're building AI systems that change how industries work. As a QA Engineer, you'll own quality across product tracks, working closely with engineers and clients to make sure what we ship actually works.

Responsibilities:

• Own quality for an AI product track from requirements to release.
• Test LLM features, RAG pipelines, and multi-agent workflows.
• Build automated test suites to catch regressions in AI outputs.
• Run adversarial tests to find failure modes before users do.
• Work with engineers and clients to define what 'good' looks like.
• Plug quality gates into CI/CD pipelines.
• Track bugs to closure and flag risks clearly.

Requirements:

• 2-3 years in QA or software testing, at least 1 year on AI/ML products.
• Can write Python to automate tests and evaluate model outputs.
• Have tested LLM prompts, outputs, hallucinations, and edge cases.
• Know REST API testing, pytest, and basic CI/CD.
• Think in failure modes and communicate findings clearly.
• Bonus: exposure to vector DBs, RAG, or AI bias testing.
• Tech Stack: Python, pytest, REST APIs, Postman, GitHub Actions, LLM APIs, LangChain, Vector DBs, AWS, Postgres, and Redis.

Good to have:

• DeepEval LLM evaluation framework (hallucination, relevancy, toxicity testing).
• RAG evaluation framework specifically for RAG pipelines.

Skills

AI, LLM, ML, Python, QA, pytest, testing