AI Engineer

Responsibilities:

• Design and implement asynchronous multi-agent orchestration for Dhruva, our AI agent.
• Own end-to-end latency from user message to AI response.
• Build resilient inference-calling pipelines that gracefully degrade under load.
• Implement intelligent request routing and load balancing across AI workloads and providers.
• Migrate critical AI conversation flow from monolith to dedicated services.
• Implement WebSocket/streaming infrastructure for real-time chat.
• Design circuit breakers and fallback strategies for AI model failures.
• Build comprehensive observability for AI system performance, latency, error rates, and fallback triggers.
• Optimise credit data retrieval and caching strategies feeding into AI conversations.

Requirements:

• 3-5 years building production systems handling > 10k concurrent users.
• Proven experience with async/event-driven architectures (not just REST APIs).
• Hands-on experience scaling AI inference calls in production, routing, batching client-side, and retries.
• Deep understanding of caching strategies (Redis, in-memory, CDN).
• Experience with message queues and real-time communication protocols (WebSocket, SSE, Kafka or similar).

AI-Specific Expertise:

• Built systems integrating multiple LLM/AI models or providers in production, with fallback between them.
• Experience working against AI model serving endpoints (TensorFlow Serving, Triton, vLLM, or hosted LLM APIs).
• Understanding of inference optimisation from the application side: batching, response caching, prompt/context size management.
• Knowledge of conversation state management and context handling across multi-turn, multi-agent flows.
• Has debugged production issues under high AI inference load.