Senior AI Platform Engineer

Responsibilities:

• Design, build, and operate a production-grade AI Gateway / shared LLM infrastructure layer serving multiple downstream consumers.
• Implement multi-provider LLM routing across Anthropic, Gemini, OpenAI, and other providers, with intelligent fallback logic.
• Build usage and cost monitoring, quota management, rate limiting, and centralized logging for LLM traffic.
• Establish evaluation pipelines, prompt/version management, guardrails, and PII protection mechanisms.
• Build and maintain MCP-based tool-use and agent infrastructure to support downstream agentic applications.
• Own production operations for the platform — reliability, scaling, incident response, and performance tuning.
• Partner with applied ML and product teams to expose LLM capabilities as reliable, governed infrastructure.

Requirements:

• Strong, hands-on Python engineering skills, with production-grade code quality.
• Deep experience with GCP, GKE, and Kubernetes for deploying and scaling services.
• Proven experience building multi-provider LLM routing systems (Anthropic, Gemini, OpenAI, etc. ).
• Experience implementing usage/cost monitoring, quotas, rate limits, logging, and fallback mechanisms in production.
• Experience building evaluation frameworks, prompt/version management, guardrails, and PII protection.
• Hands-on experience with MCP, tool-use patterns, and agent infrastructure.
• Strong production operations background — this must be demonstrable, not POC-only.
• Avoid: prompt engineers, POC-only consultants, architecture-only profiles with no hands-on ops experience.

Skills

K8s, Kubernetes, LLM, Large Language Model, Python