ML Infrastructure Engineer
About the Role
This is a hands-on infrastructure engineering role at an early-stage enterprise AI company building a context and data governance layer for AI agents in highly regulated industries. You will own the inference and model-serving infrastructure end to end, ensuring AI agents run reliably, accurately, and at scale in production environments where performance is non-negotiable.
What You'll Do
Design, build, and operate inference and model-serving infrastructure from development through production deployment.
Scale systems to support AI agents running reliably under increasing concurrency and production load.
Identify and resolve infrastructure bottlenecks in close collaboration with ML and platform engineering teams.
Optimize systems for latency, throughput, and reliability at scale.
What We're Looking For
5 or more years building and operating machine learning inference systems, model-serving platforms, or ML infrastructure in production environments.
Hands-on experience designing and scaling inference serving infrastructure using tools such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom systems.
Strong systems engineering fundamentals with expertise in distributed systems, containerization, and orchestration (Docker, Kubernetes).
Demonstrated ability to optimize production ML systems for latency, throughput, and reliability under high concurrency.
Experience with cloud infrastructure platforms such as AWS, GCP, or Azure for deploying and managing ML workloads.
Proficiency with monitoring, observability, and debugging tools such as Prometheus, Grafana, ELK, or distributed tracing frameworks.
Proficiency in at least one systems programming or backend language: Python, Go, Rust, C++, or Java.
Experience with knowledge graphs, semantic search, or graph databases (e.g., Neo4j, Amazon Neptune) is a plus.
Familiarity with agentic AI systems, autonomous agents, or multi-step reasoning pipelines is a plus.
Experience with enterprise data infrastructure, data pipelines, or data integration platforms is a plus.
Location
This role is on-site in San Mateo, California. Visa sponsorship is not available.