AI/ML Architect

Role : AI/ML Architect (L7)

Mission :

Own the architecture and engineering quality of our agentic AI systems. Bring deep, hands-on expertise in agent behaviour, evaluation, and information retrieval, and turn that expertise into buildable designs, reference implementations, and standards that the engineering pods can ship reliably.

Context :

We are a small, high-leverage AI organization with two complementary disciplines : AI Engineering and Platform Engineering. Our commercial thesis is vendor-neutral and accelerator-led : reusable components that fill gaps in or complement hyperscaler offerings. The AI/ML Architect is the senior technical authority for the intelligence and behaviour of our agents, working as a peer to the Platform Architect, who owns the runtime and platform substrate.

What you'll do :

- Own agentic-system architecture. Define how agents reason, plan, use tools, maintain state, recover from failure, collaborate, and involve humans.

- Design harness behaviour. Specify and prototype the control loop around the model : context assembly, planning and execution, tool selection, memory, retries, reflection, escalation, checkpoints, and termination conditions.

- Make quality measurable. Build the evaluation strategy for non-deterministic systems across offline test sets, simulations, trajectory and tool-use evaluation, online experiments, human review, and production monitoring.

- Architect retrieval and context. Design retrieval, ranking, context assembly, grounding, citation, permissions, and freshness patterns that remain reliable across enterprise knowledge sources and changing data.

- Turn intent into buildable specifications. Produce architecture decisions, behavioural specifications, interface contracts, evaluation plans, and reference implementations precise enough for teams to implement consistently.

- Prototype and de-risk. Write production-quality proofs of concept for difficult agent behaviours, retrieval strategies, or evaluation methods before a pod commits to an approach.

- Guard execution fidelity. Lead design reviews, identify behavioural and quality drift early, make evidence-based trade-offs, and keep implementation aligned with the intended architecture.

- Raise the engineering bar. Mentor senior engineers, establish reusable patterns and review standards, and serve as the VP's depth partner on AI/ML feasibility, risk, and opportunity.

Core depth requirement :

- Agentic engineering and harness behaviour : control loops, planning and execution, state and memory, tool use, structured outputs, error recovery, long-running tasks, human-in-the-loop patterns, multi-agent coordination, and safe autonomy.

- Complex evaluations : task-success and trajectory evaluation, tool-call correctness, groundedness, retrieval quality, safety, latency and cost, simulation, regression suites, online measurement, human calibration, and careful use of model-based judges.

- Information retrieval and context engineering : ingestion and indexing, hybrid and semantic retrieval, metadata and permission filtering, ranking and reranking, graph or relationship-aware retrieval, context compression, grounding, citations, and retrieval evaluation.

- Production AI quality : prompt and model selection, guardrails, observability and tracing, auditability, failure analysis, and the cost-latency-quality trade-offs.

Must-have qualifications :

- Approximately 8+ years building production software or ML systems, with several years architecting complex systems.

- Demonstrated hands-on delivery of production agentic systems used by real users.

- Deep practical knowledge of agent harnesses or runtimes.

- Evidence of designing sophisticated evaluation systems for non-deterministic, tool-using, or retrieval-augmented applications.

- Strong information-retrieval architecture experience.

- Strong Python and software-engineering ability.

- Technical authority to guide senior engineers without relying on positional management.

- Strong written communication skills.

Skills

Python, Machine Learning, Artificial Intelligence, Generative AI, Conversational AI, AI Architecture, RAG, LLM