2,445,015open jobs
150,239companies
295,749added this week
Browse all
Salary
$250k – $350k per year
Location
In office (San Francisco)
Seniority
Staff · 5+ years exp
Employment
Full-Time

Confirmed on the employer's own hiring board on Nov 20, 2026. First seen by Alion on Nov 20, 2026. LiteLLM scores A on the Alion truth index.

Overview
Company
Impact
Profile match
LiteLLM is an open-source AI gateway and Python SDK that gives developers a single OpenAI-compatible interface to more than 100 large language model providers. Built by BerriAI, it handles model routing, fallbacks, spend tracking, budgets and rate limits, and can be self-hosted, including in air-gapped environments. The company is headquartered in San Francisco, California, United States.
Backed by Y Combinator

Staff Backend Engineer

LiteLLM is the world’s most popular open-source AI Gateway, used by Adobe, Netflix, NASA, NVIDIA and much of the Fortune 500. Every LLM call these companies make runs through our code, so when we break, they break. We’re a team of about 20, and we’re hiring senior and staff backend engineers to own the core of the gateway and keep it fast and reliable as traffic grows.

About the role

You’ll own critical parts of the LiteLLM proxy: routing, auth, caching, spend tracking, guardrails and observability. The job is to make these systems correct, fast and dependable at 1B+ requests a month. That means hunting down silent failures, fixing the root cause in code rather than adding another dashboard, and refactoring components that keep breaking without changing how they behave. You’ll set the bar for how our mostly early-career team writes, tests and ships backend code.

What you’ll do

  • Own core proxy components end to end, from design through production, including the incidents

  • Find and remove whole classes of bugs, like race conditions, stale cache state across replicas, and errors that get silently swallowed instead of shown to users

  • Make sure every guardrail and policy check (e.g. apply_guardrail) is logged, traceable and fails loudly when it should

  • Build guardrails that run inside the gateway and block common attacks on LLM APIs, MCP servers and agents

  • Keep our observability integrations (Datadog, Splunk, Prometheus, OpenTelemetry) accurate at 1B+ requests a month, with minimal latency overhead and no memory leaks (e.g. from Prometheus metric cardinality)

  • Help migrate performance-critical paths from Python to Rust without breaking behavior

  • Raise the bar for the team on code review, testing and debugging

What we’re looking for

  • 5+ years building backend systems in production, with real ownership of something mission-critical: payments, gateways, core APIs, data stores, or anything else where downtime hurts customers

  • Strong Python, and experience with async or high-throughput services (asyncio, FastAPI or similar)

  • A track record of making production systems more reliable through code: refactors, re-architectures, migrations, or fixing a root cause for good

  • An excellent debugger across services, databases (Postgres, Redis) and distributed systems

  • High ownership, meaning you take responsibility for the customer impact of what you ship

  • BS/MS in Computer Science or a related field, or equivalent proof you’re exceptional

Nice to have

  • Rust, Go or C++ in production

  • Time at a fast-growing startup with a large, real codebase

  • Experience with LLM APIs, proxies or API gateways

  • Open-source contributions

Why LiteLLM

  • Your code runs in the critical path for some of the biggest companies in the world

  • Real ownership of core systems on a small, senior-leaning team

  • Profitable, growing fast, backed by Kleiner Perkins

  • $250-$350k base + equity, health, dental and vision. Onsite in San Francisco.

About LiteLLM

LiteLLM is a Python SDK and proxy server for calling 100+ LLM APIs in the OpenAI format, with 60K+ GitHub stars.

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,445,015 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Backend
Remote, many countries
Similar stack
Same company
San Francisco
$78k – $156k per year · In office · Full-Time · 6+ years exp · Bachelor's Degree · United States
Java
SQL
Java
Maven
Spring Boot
Hibernate
Databases
PostgreSQL
Apache Kafka
DevOps
Rest API
Azure
AWS
Management
Confluence
Agile
MedTech
HL7
FHIR
Design Controls
Apply
$99k – $199k per year · In office · Full-Time · 7+ years exp · High School Diploma · United States
Python
JavaScript
Java
SQL
C#
C++
Java
Maven
Spring Boot
Hibernate
C#
.NET
Databases
PostgreSQL
Apache Kafka
DevOps
Rest API
Azure
AWS
Management
Confluence
Agile
MedTech
HL7
FHIR
Apply
Senior Java Lead 17 min ago
$142k – $213k per year · Hybrid · Full-Time · 6+ years exp · Bachelor's Degree · New York
Java
Java
Spring Boot
Databases
Couchbase
Apache Kafka
AI/ML
AI Agents
DevOps
OpenShift
Kubernetes
Management
Agile
Apply
$114k – $171k per year · Hybrid · Full-Time · Bachelor's Degree · Tampa
JavaScript
Java
TypeScript
SQL
Perl
Java
Maven
Spring Boot
Hibernate
Gradle
Databases
Oracle
Apache Kafka
AI/ML
Copilot
Prompt Engineering
AI Agents
RAG
Devin
Agentic Workflows
Frontend
Angular
JQuery
Mobile
JUnit
DevOps
Rest API
Ansible
Red Hat
OpenShift
Helm
CI/CD
Jenkins
Git
Docker
Kubernetes
Service Mesh
Tekton
GitHub
API Gateway
Linux
SOAP
Apache HTTP Server
Cybersecurity
CyberArk
SonarQube
Checkmarx
SBOM
Dependabot
Renovate
Management
Confluence
Jira
Agile
Apply
$97k – $145k per year · Hybrid · Full-Time · 5+ years exp · Bachelor's Degree · Tampa
Java
Kotlin
SQL
Java
Spring Framework
Maven
Spring Boot
Hibernate
Hazelcast
Gradle
Kotlin
Mockito
Databases
PostgreSQL
Redis
Oracle
MS SQL
Apache Kafka
Mobile
JUnit
DevOps
Rest API
Splunk
OpenShift
Prometheus
CI/CD
Jenkins
Git
Docker
Kubernetes
Grafana
TeamCity
GitHub
Cybersecurity
SonarQube
Management
Agile
Scrum
Apply
up to $43k per year (net) · Remote (likely EAEU) · 5+ years exp · Volgograd
JavaScript
Java
TypeScript
SQL
Java
Vert.x
Databases
PostgreSQL
Redis
ActiveMQ
Frontend
Svelte
Tailwind CSS
DevOps
GitHub
Cybersecurity
PCI DSS
Web3
Ethereum
Apply
Remote (likely EAEU) · 2+ years exp · Moscow
Java
Kotlin
SQL
Java
Spring Framework
Maven
Spring Boot
Hibernate
Gradle
Flyway
Liquibase
Databases
PostgreSQL
Redis
RabbitMQ
Apache Kafka
Mobile
JUnit
DevOps
Rest API
gRPC
CI/CD
Git
Docker
Kubernetes
SOAP
Apply
$28k per year · Remote (EAEU) · Minsk
Rust
Solidity
Solidity
Alloy
Web3
Foundry
Polygon
Solana
Aave
Uniswap
Ethereum
DeFi
MakerDAO
Apply
Remote (likely EAEU) · 3+ years exp · Saint Petersburg
JavaScript
PHP
TypeScript
SQL
PHP
Yii
Laravel
Databases
PostgreSQL
ClickHouse
AI/ML
AI Agents
Apply
$8.5k per year (net) · Remote (likely EAEU) · Full-Time · Ufa
Java
DevOps
Git
Apply
$185k – $344k per year · In office · Full-Time · 5+ years exp · Bachelor's Degree · South San Francisco
Python
JavaScript
TypeScript
Databases
PostgreSQL
Snowflake
AI/ML
LangChain
AI Agents
LLM
RAG
Machine Learning
DevOps
CI/CD
AWS
Life Sciences
Veeva
Apply
Android Engineer 1 hour ago
Remote (Nigeria) · Lagos
Python
Go
Java
Kotlin
Ruby
Java
RxJava
Kotlin
Kotlin Coroutines
Ruby
Ruby on Rails
RSpec
Sidekiq
Databases
MySQL
PostgreSQL
DynamoDB
Google BigQuery
Amazon Aurora
Mobile
Jetpack Compose
Clean Architecture
DevOps
GCP
GitHub Actions
CI/CD
AWS
Kubernetes
Amazon S3
Amazon Kinesis
Apply
In office · Full-Time · São Paulo
Python
Bash
Databases
MySQL
PostgreSQL
Oracle
DevOps
Terraform
Ansible
VMWare
Packer
GitOps
AWS
Docker
Kubernetes
Platform Engineering
Amazon EC2
OpenStack
GitLab
Linux
TCP/IP
VPN
Apply
$50k – $60k per year · In office · Internship · High School Diploma · South San Francisco
Python
MATLAB
AI/ML
PubMed
Analytics
Microsoft Excel
Management
Google Calendar
Agile
Apply
In office · Full-Time · Bachelor's Degree · San Antonio
DevOps
Splunk
Analytics
Tableau
Management
Agile
Apply
Staff Engineer, AI 11 days ago
$250k – $350k per year · In office · Full-Time · 3+ years exp · San Francisco
Python
Rust
Python
FastAPI
Databases
PostgreSQL
Redis
AI/ML
LiteLLM
LLM
OpenRouter
OpenAI
Anthropic
Vercel AI SDK
Machine Learning
Analytics
Fivetran
Apply
Product Engineer 4 months ago
$200k – $260k per year · In office · Full-Time · Bachelor's Degree · San Francisco
Python
Python
FastAPI
AI/ML
LiteLLM
LLM
Apply
$200k – $260k per year · In office · Full-Time · 2+ years exp · Bachelor's Degree · San Francisco
Python
Go
Rust
C++
Python
FastAPI
Asyncio
Databases
PostgreSQL
Redis
AI/ML
Model Context Protocol
LiteLLM
LLM
OpenAI
LLM Guardrails
DevOps
Splunk
OpenTelemetry
Datadog
Prometheus
GitHub
Apply
ML Engineer 9 days ago
$200k – $250k per year · In office · Full-Time · 1+ year exp · San Francisco
Python
Python
FastAPI
Databases
PostgreSQL
Redis
AI/ML
Model Context Protocol
LiteLLM
LLM
OpenAI
Anthropic
Apply
$250k – $325k per year · In office · Full-Time · San Francisco
AI/ML
LiteLLM
DevOps
GitHub
Apply
$200k – $275k per year · Remote (United States) · 5+ years exp · Bachelor's Degree · San Francisco
Python
AI/ML
Cursor
Claude Code
AI Agents
Apply
Customer Engineer 6 hours ago
$115k – $130k per year · Equity 0.1–0.2% · In office · Full-Time · San Francisco
Apply
In office · 2+ years exp · Master's Degree · San Francisco
Apply
In office · 1+ year exp · High School Diploma · San Francisco
Apply
In office · 1+ year exp · San Francisco
Apply
See all jobs
This is one of many
2,445,015 more open roles from verified company boards, updated every day.