Software Platform Engineer II

Job Summary

We are seeking a highly skilled Senior AI Backend Engineer to design, build, and productionize backend systems and ML/LLM inference pipelines within cloud-native environments. This role focuses on developing and operating production-grade AI agents powered by frontier models while ensuring scalability, reliability, and operational excellence.The ideal candidate will have strong expertise in backend engineering, model serving, and AI platform development, with hands-on experience in Agentic AI frameworks such as LangGraph and LangChain. You will be responsible for building the orchestration layer around large language models, including tooling, context management, workflow automation, guardrails, and evaluation frameworks.Experience with Azure cloud services, Azure Kubernetes Service (AKS), and cloud-native operations is highly desirable.

Key Responsibilities:

AI & Backend Engineering:

• Design, develop, and maintain scalable backend APIs and microservices using Python, FastAPI, and asynchronous programming patterns.
• Build and deploy production-ready Agentic AI workflows using LangGraph and LangChain.
• Develop intelligent agent systems incorporating tool calling, multi-step reasoning, context and memory management, retries, guardrails, and orchestration logic.
• Operate and optimize production AI agents leveraging frontier models such as GPT, Claude, and Gemini, ensuring optimal performance, cost efficiency, and reliability.
• Implement advanced LLM capabilities including Retrieval-Augmented Generation (RAG), vector search, embeddings, prompt engineering, and context management.

Model Engineering & MLOps:

• Package, version, deploy, and monitor ML/LLM models for low-latency, batch, and asynchronous inference workloads.
• Optimize model serving infrastructure using frameworks such as MLflow, LangSmith, or equivalent platforms.
• Implement model monitoring, evaluation pipelines, and performance benchmarking processes.

Event-Driven Architecture & Integrations:

• Build resilient event-driven systems utilizing Kafka, Azure Event Hubs, or similar messaging platforms.
• Develop idempotent consumers, replay mechanisms, and backfill strategies to ensure data consistency and reliability.
• Design scalable integration frameworks connecting AI services with enterprise systems.

Cloud & Platform Engineering:

• Containerize applications using Docker and deploy workloads on Kubernetes environments, including Azure Kubernetes Service (AKS).
• Create and manage Kubernetes manifests, Helm charts, and deployment configurations.
• Troubleshoot and optimize containerized workloads, health checks, autoscaling, and resource utilization.

Reliability & Operational Excellence:

• Implement observability solutions including logging, monitoring, metrics, and distributed tracing.
• Define and track Service Level Objectives (SLOs) and Service Level Indicators (SLIs).
• Lead incident response, root-cause analysis, and remediation efforts.
• Ensure secure, scalable, and highly available AI platform operations.

DevOps & Quality Engineering:

• Develop automated testing frameworks, CI/CD pipelines, and release automation processes.
• Collaborate with security and platform teams to implement code quality controls and security scanning.
• Promote engineering best practices, test-driven development, and operational excellence.

Cross-Functional Collaboration:

• Work closely with frontend engineering teams using React, Next.js, and TypeScript to deliver end-to-end AI solutions.
• Collaborate with product, data science, and platform engineering teams to deliver scalable AI capabilities.

Required Qualifications:

Technical Skills:

• Strong proficiency in Python and modern backend frameworks, preferably FastAPI.
• Hands-on experience building and operating production AI agents using frontier large language models.
• Expertise in Agentic AI orchestration, including workflow management, tool execution, memory systems, context handling, and guardrails.
• Strong experience with LangGraph and/or LangChain frameworks.
• Deep understanding of model serving, inference optimization, serialization, batching, and monitoring.
• Experience with vector databases and embedding technologies such as FAISS, Milvus, Pinecone, or similar platforms.
• Knowledge of Retrieval-Augmented Generation (RAG) architectures and implementation patterns.
• Strong foundation in distributed systems, concurrency, caching, rate limiting, and reliability engineering.
• Experience with Docker and Kubernetes administration, deployment, and troubleshooting.
• Proficiency with CI/CD tools such as Azure DevOps and GitHub Actions.
• Experience working with data pipelines, feature engineering, and data quality management practices.
• Working knowledge of React and JavaScript/TypeScript for effective collaboration with frontend teams.
• Excellent communication skills and strong written English.

Cloud & Infrastructure Requirements:

• Hands-on experience with Microsoft Azure services and Azure Kubernetes Service (AKS).
• Experience with Azure DevOps and cloud-native deployment practices.
• Familiarity with Redis, Cosmos DB, Azure Container Registry (ACR), and similar managed services.

Preferred Qualifications:

• Experience with Model Context Protocol (MCP) and AI tool/server integrations.
• Knowledge of load testing, performance tuning, and scalability optimization.
• Experience with model drift detection, explainability tools, MLflow Registry, or LangSmith.
• Exposure to multi-agent architectures, fine-tuning techniques, and structured output generation.
• Infrastructure as Code (IaC) experience using Terraform, Bicep, or equivalent technologies.