Senior AI / Python Engineer

Role Summary

We are hiring a Senior AI / Python Engineer for one of the AI Initiative, which delivers an enterprise MCP (Model Context Protocol) platform connecting ChatGPT Enterprise to governed data in Databricks Unity Catalog. You will design and build core platform components, including the orchestration layer, account and entity resolution, and observability. You will also enhance the MCP server, engineer and evaluate prompts and agent instructions, enable new features, and own upgrades, releases, and production quality.

Key Responsibilities

Platform development

• Design and build the multi-agent orchestration layer: intent classification, semantic routing (embedding similarity with LLM fallback), agent handoffs, and clarification flows.
• Build and evolve account and entity resolution: alias indexes, fuzzy and semantic matching, batch index publishing with versioned swaps, and shadow-to-enforce rollouts, delivered as versioned, tested Python packages.
• Enhance the MCP server (FastMCP): tools, agents, and new features from requirements through release.
• Write design documents before coding and document trade-offs across accuracy, latency, and cost.

Prompt engineering and agent configuration

• Develop and refine prompts, agent instructions, tool descriptions, and semantic examples; reduce hallucination and tool-selection errors.
• Onboard and maintain agents: configuration, vector stores and recipe files, retrieval tuning, and SQL logic against Databricks Unity Catalog.

Observability, evaluation, and quality

• Instrument agents and tools with Langfuse (or equivalent) tracing; track prompt versions, token cost, latency, and quality per agent.
• Maintain evaluation suites (e.g., Promptfoo) with LLM-as-judge grading; add regression tests for every prompt or agent change and gate releases on the results.
• Own the audit and activity pipeline (CloudWatch, S3, Databricks); use logs, traces, and user feedback to improve routing, prompts, and retrieval.

Platform lifecycle and operations

• Plan and run dependency upgrades (OpenAI Agents SDK, openai, pydantic, FastMCP): breaking-change analysis, compatibility testing, phased rollout, and rollback.
• Manage CI/CD (GitHub Actions), nonprod-to-prod promotion, configuration and secrets, and Kubernetes (EKS) rollouts.
• Integrate external services and MCPs (e.g., Salesforce/MuleSoft, APISIX-based MCPs).
• Triage and resolve production issues; write and maintain runbooks.

Scale, security, and governance

• Optimize performance and cost: concurrency, rate limiting, connection pooling, Redis caching, retries and timeouts, model selection, token usage, and multi-pod consistency.
• Implement enterprise controls: OAuth (Azure AD) with user impersonation (OBO), row-level security compatibility, input validation, prompt-injection defenses, PII handling, and audit logging.

Collaboration

• Work with the OpenAI team, AWS/EKS infrastructure, security owners, and business users.
• Use AI-assisted development and review tools while owning design decisions and verifying every change with tests.

Required Qualifications

• 6+ years of professional software engineering experience, with strong Python (3.10+): async programming, typing, packaging, and testing (pytest).
• Hands-on experience building LLM applications: prompt engineering, tool/function calling, and multi-agent or orchestration design.
• Proven ownership of a production system component (e.g., orchestration, routing, matching, or search) from design through operations.
• Experience with entity matching, NLP, or retrieval (fuzzy matching, embeddings, RAG).
• API and security fundamentals: REST, OAuth 2.0 / OIDC.
• Working knowledge of AWS (EKS, S3, CloudWatch, DynamoDB, Redis/ElastiCache), Docker, and Kubernetes.
• Strong SQL and a cloud data platform (Databricks or similar), including notebooks/PySpark.
• Git and CI/CD (GitHub Actions); track record of managing breaking dependency upgrades safely.
• Clear written and verbal communication; able to work independently in an ambiguous, fast-moving environment.

Preferred Qualifications

• Experience with Model Context Protocol (MCP), FastMCP, OpenAI Agents SDK, or comparable agent frameworks (LangChain / LangGraph).
• Experience with LLM observability and evaluation tools (Langfuse, OpenTelemetry, Promptfoo).
• Familiarity with Databricks Unity Catalog governance, row/column-level security, and metric views or semantic layers.
• Background in enterprise security: Azure AD / Entra ID, identity propagation, secrets management.
• Experience optimizing cost and latency of LLM workloads.
• Life sciences, commercial, or enterprise data domain experience.

Technology Stack

Python 3.11 · FastMCP · OpenAI Agents SDK / ChatGPT Enterprise · Databricks Unity Catalog · AWS (EKS, CloudWatch, S3, DynamoDB, Redis, ALB, WAF) · Azure AD OAuth · Langfuse · Promptfoo · GitHub Actions · Docker / Kubernetes