Software Engineer 3 (Gen AI)

You'll be part of the team building Exotel's AI and speech platform, powering products like Conversation Quality Analysis, AI Voice Agents, and AI Chat Agents. This means working across large language models, speech processing, and distributed backend systems to process and understand millions of enterprise conversations at scale. This is an engineering role first. You'll design, build, deploy, and operate production services and own them end-to-end.

Responsibilities:

• Build and optimise LLM-powered pipelines for analysing conversations at scale, managing cost, latency, and quality trade-offs across model providers.
• Work on speech processing: improving transcription accuracy, handling multilingual audio, and solving real-world audio quality challenges.
• Design and build AI agents that reason over enterprise data and deliver actionable answers with appropriate guardrails.
• Develop and scale high-throughput, multi-tenant backend services with a focus on reliability and performance.
• Own cost and observability for AI workloads: usage tracking, cost attribution, and quality monitoring. Model optimisation and self-hosting are active areas of investment.
• Take end-to-end ownership of the software development lifecycle: requirements, design, development, testing, deployment, and monitoring.

Requirements:

• Bachelor's or master's degree in computer science or equivalent.
• 5-8 years of software engineering experience.
• Proficiency in at least one backend language (Python, Java, Go, or similar). Ability to pick up new languages quickly; the language matters less than the engineering thinking behind it.
• Strong understanding of data structures, algorithms, multi-threading, and concurrency.
• Good understanding of software engineering concepts: design patterns, modularity, scalability.
• Experience with microservices architecture and distributed systems: designing, building, and operating them.
• Experience designing and developing RESTful APIs and async job-processing architectures.
• Production experience with databases, caching layers, message brokers, and search/analytics systems, including data modelling and scaling.
• Hands-on experience with LLM APIs (OpenAI, Gemini, Azure, or similar): prompt engineering, structured output, cost management. If you haven't worked with LLMs yet but have strong engineering fundamentals and a willingness to learn, that works too.
• Experience with major cloud platforms (AWS, GCP, or Azure).
• Experience with containers and orchestration (Docker, Kubernetes) and CI/CD pipelines.
• Strong analytical and problem-solving skills.
• Excellent written and verbal communication skills.
• Team player, comfortable working across teams (product, data, infrastructure) in a fast-paced environment.

Good-to-have:

• Understanding of RAG patterns: embeddings, vector stores, and retrieval strategies.
• Comfortable with Linux, shell scripting, and developing Linux-based applications.
• Familiarity with monitoring and observability tools such as Grafana, Kibana, and Elasticsearch.
• Strong networking fundamentals: DNS, load balancing, proxies, firewalls.
• Experience with ASR/TTS engines: Whisper-family models, VAD, speaker diarisation, alignment, and common failure modes.
• Experience working with audio pipelines: IP streaming, format handling, noise reduction, streaming vs batch processing.
• Experience with graph databases and graph query languages.
• Familiarity with columnar analytics stores for large-scale analytical queries.
• Experience building multi-tenant SaaS with tenant isolation and per-tenant configuration.
• Exposure to LLM observability and cost tracking tooling.
• Experience self-hosting or fine-tuning open-weight models.