SDET 1

We are seeking a hands-on, high-ownership SDET to scale the automated quality engineering and evaluation pipelines for our autonomous multi-agent procurement platform.

Our architecture pairs probabilistic AI agent reasoning loops (written in Python) with a deterministic, high-throughput distributed execution engine (written in Go). In this role, you will go beyond traditional UI/API automation to test multi-agent state transitions, non-deterministic LLM tool-calling accuracy, asynchronous workflows, and enterprise ERP integration points.

Responsibilities:

• API and Workflow Test Automation: Design, write, and maintain robust end-to-end automation test suites for asynchronous REST/gRPC backend microservices, event streams, and distributed workflow activities.
• Agentic and LLM Evaluation: Build automated evaluation harnesses (using frameworks like DeepEval, Ragas, Langfuse, or custom PyTest suites) to benchmark agent reasoning accuracy, hallucination rates, context retrieval precision, and schema conformity.
• Schema and Contract Testing: Implement strict contract testing verifying that agent tool calls conform to strongly typed contracts when interfacing with core Go execution services.
• Mocking and Integration Sandboxes: Create mock services and virtualized sandboxes to simulate external enterprise procurement software and third-party vendor APIs for deterministic integration testing.
• State and Concurrency Verification: Write automated test scenarios validating state checkpointing, retries, and Human-in-the-Loop (HITL) pause/resume logic across long-running procurement cycles.
• CI/CD Integration: Embed automated test execution, linting, regression sweeps, and LLM evaluation benchmarks into CI/CD pipelines to catch regressions prior to deployment.
• Bug Triage and Observability: Investigate test failures using distributed tracing, log aggregators, and metrics to quickly differentiate between backend infrastructure bugs and probabilistic LLM drifts.

Requirements:

• Experience: 2 to 4 years of hands-on experience in software automation, backend testing, or software development in testing.
• Strong Coding Fundamentals: Proficiency in Python for authoring automated test frameworks, utility scripts, and test harnesses.
• API and Systems Testing: Solid hands-on experience testing distributed microservices, REST APIs, and event-driven architectures (Kafka, RabbitMQ, or message queues).
• Data and Storage Testing: Practical experience writing SQL queries, validating relational schemas in PostgreSQL, and checking cached states in Redis.
• Understanding of LLM/AI Concepts: Working familiarity with LLM fundamentals (prompt validation, tool/function calling, RAG pipelines, and automated evals like ground truth comparison).
• DevOps and Containers: Comfortable working with Docker, Linux/Unix CLI, Git workflows, and CI/CD automation pipelines.

Preferred Qualifications:

• Exposure to reading or testing services written in Go (Golang).
• Hands-on experience testing durable execution platforms like Temporal or Cadence.
• Prior domain exposure to enterprise SaaS, B2B procurement, supply-chain logistics, or fintech transaction systems.

Skills

SDET, Software Development Engineer in Test