SDET - AI

We're hiring a Senior Software Development Engineer in Test (SDET) AI Testing to work on cutting-edge generative AI, LLMs, RAG, and agentic AI solutions. This is an exciting opportunity to build scalable AI testing frameworks, automate LLM evaluations, and drive AI quality engineering for enterprise-grade AI products.

Responsibilities:

• Design and execute testing strategies for LLMs, RAG, and Agentic AI applications.
• Build automated AI evaluation pipelines using DeepEval, OpenAI Evals, and custom evaluation frameworks.
• Validate AI quality metrics, including hallucination detection, faithfulness, answer relevance, toxicity, bias, safety, latency, and cost.
• Develop robust automation frameworks for APIs and AI-powered applications.
• Integrate AI testing into CI/CD pipelines.
• Test multi-agent workflows, tool calling, memory management, and prompt effectiveness.
• Perform performance, reliability, regression, and end-to-end testing for AI systems.

Requirements:

• 4-8 years of experience in software testing/SDET with strong automation expertise.
• Hands-on experience testing LLMs, generative AI, RAG, or agentic AI applications.
• Experience with DeepEval, OpenAI Evals, prompt testing, or AI evaluation frameworks.
• Strong programming skills in Python (preferred) or Java.
• Experience with API testing, PyTest/Robot Framework, REST APIs, and microservices.
• Familiarity with LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, and vector databases like Pinecone, FAISS, ChromaDB, or Weaviate.
• Exposure to cloud platforms (AWS/Azure/GCP), Docker, Kubernetes, and CI/CD tools.

Nice to Have:

• Experience with AI observability tools such as Langfuse, Arize AI, or Weights and Biases.
• Knowledge of responsible AI, AI governance, AI safety, MLOps, or AI red teaming.
• Experience testing multimodal AI systems.