Full - Stack Engineer - AI

We're hiring a Senior AI Engineer to own slices of Raptor's backend and AI layer, end-to-end retrieval, agent orchestration, evals, observability, and the admin tooling our internal users rely on. This role is ~70% Python backend and AI engineering and ~30% frontend (admin consoles, eval dashboards, and BU-facing configuration UIs). If you want to build chat UIs as your day job, this isn't the role. If you want to make production chatbots actually work and have the data to prove they do, read on.

Responsibilities:

• Build and own AI features end-to-end: retrieval, agent/router logic, prompt design, tool-calling, and post-processing from prototype to production traffic.
• Improve retrieval quality through chunking, embeddings, hybrid search, reranking, and grounded generation against BU knowledge bases. Measure hit rate, not vibes.
• Run the eval loop offline: eval harnesses, golden sets, LLM-as-judge graders, and regression suites. Every prompt or model change ships behind an eval delta.
• LLM observability instruments session-level traces, latency percentiles, retrieval hit rate, hallucination signals, intent misrouting, dead ends, and escalations.
• Latency and cost optimization: streaming, caching, model routing, batching, and prompt compression. Cost-per-session is a first-class metric.
• Backend services: Python services (FastAPI, async workers) that handle real production load with proper error handling, retries, and graceful degradation.
• Spec-driven development for non-trivial features: write a short design spec (problem, options, chosen approach, evaluations, and rollout) before writing code. We use AI coding tools heavily but reject vibe-coding: specs and evals are how we keep speed without losing the plot.
• Partner with PMs, conversation designers, and BU stakeholders to translate ambiguous product intent into a working system. Push back when the data says the proposal won't work.
• Admin and eval tooling React/TypeScript surfaces for our team and BU stakeholders to configure flows, inspect sessions, review eval runs, and ship changes safely.
• Guardrails and AI safety input/output safety checks, PII handling, prompt-injection defenses, and refusal patterns appropriate to each BU context.
• On-call and production support: debug live incidents, root-cause regressions, and own postmortems

Requirements:

• 5-8 years of software engineering experience, with at least 2 years building production LLM/AI applications that served real users (not demos).
• Strong Python backend experience with FastAPI or similar, async/await, Pydantic, background workers, and testing discipline.
• Retrieval/RAG in production vector stores (pgvector, Pinecone, Weaviate, or similar), hybrid search, and reranking.
• Hands-on with at least one agent/orchestration framework (LangChain, LangGraph, LlamaIndex, or custom equivalent) and a clear point of view on when *not* to use multi-agent patterns.
• LLM evals golden sets, regression harnesses, LLM-as-judge, and gating releases on eval deltas.
• LLM observability: LangSmith, Langfuse, Arize, OpenTelemetry for LLMs, or equivalent homegrown tracing.
• SQL and warehouse data (BigQuery, Snowflake, Redshift, or similar) chatbot work is data work.
• Working frontend skills in React + TypeScript sufficient to ship internal tooling and dashboards.
• Spec-driven development habit: comfortable writing short design docs and evals before code, using AI coding tools as accelerants rather than replacements for thinking.
• Bachelor's degree in CS or engineering, or equivalent practical experience.

Nice to have:

• Experience shipping conversational AI for Japanese-language users or other non-English markets.
• Experience with messaging platforms (WhatsApp Business, Slack, in-app chat) or telephony/voice channels.
• Fine-tuning, adapters, or distillation for cost/latency wins.
• Prompt versioning and feature-flagged prompt rollouts.

Tech you'll work with:

• Backend / AI: Python, FastAPI, async workers, LangChain/LangGraph, Frontier LLM APIs, pgvector / vector search, and Postgres.
• Observability & evals: LLM tracing tools, custom eval harnesses, OpenTelemetry.
• Frontend: React, TypeScript, a modern component library that is pragmatic, not religious.
• Data: SQL on the warehouse and event instrumentation.
• Infra: Containerized services, CI/CD, on-call rotation.

Skills

AI, Artificial Intelligence, LLM, Python, Rag, Retrieval