2,446,146open jobs
150,239companies
296,799added this week
Browse all
Salary
$200k – $260k per year
Location
In office (San Francisco)
Seniority
Senior · 2+ years exp
Employment
Full-Time

Confirmed on the employer's own hiring board on Nov 20, 2026. First seen by Alion on Mar 31, 2026. LiteLLM scores A on the Alion truth index.

Overview
Company
Impact
Profile match
LiteLLM is an open-source AI gateway and Python SDK that gives developers a single OpenAI-compatible interface to more than 100 large language model providers. Built by BerriAI, it handles model routing, fallbacks, spend tracking, budgets and rate limits, and can be self-hosted, including in air-gapped environments. The company is headquartered in San Francisco, California, United States.
Backed by Y Combinator

Senior Backend Engineer

LiteLLM is the world’s most popular open-source AI Gateway, used by Adobe, Netflix, NASA, NVIDIA and much of the Fortune 500. Every LLM call these companies make runs through our code, so when we break, they break. We’re a team of about 20, and we’re hiring senior and staff backend engineers to own the core of the gateway and keep it fast and reliable as traffic grows.

About the role

You’ll own critical parts of the LiteLLM proxy: routing, auth, caching, spend tracking, guardrails and observability. The job is to make these systems correct, fast and dependable at 1B+ requests a month. That means hunting down silent failures, fixing the root cause in code rather than adding another dashboard, and refactoring components that keep breaking without changing how they behave. You’ll set the bar for how our mostly early-career team writes, tests and ships backend code.

What you’ll do

  • Own core proxy components end to end, from design through production, including the incidents

  • Find and remove whole classes of bugs, like race conditions, stale cache state across replicas, and errors that get silently swallowed instead of shown to users

  • Make sure every guardrail and policy check (e.g. apply_guardrail) is logged, traceable and fails loudly when it should

  • Build guardrails that run inside the gateway and block common attacks on LLM APIs, MCP servers and agents

  • Keep our observability integrations (Datadog, Splunk, Prometheus, OpenTelemetry) accurate at 1B+ requests a month, with minimal latency overhead and no memory leaks (e.g. from Prometheus metric cardinality)

  • Help migrate performance-critical paths from Python to Rust without breaking behavior

  • Raise the bar for the team on code review, testing and debugging

What we’re looking for

  • 2+ years building backend systems in production, with real ownership of something mission-critical: payments, gateways, core APIs, data stores, or anything else where downtime hurts customers

  • Strong Python, and experience with async or high-throughput services (asyncio, FastAPI or similar)

  • A track record of making production systems more reliable through code: refactors, re-architectures, migrations, or fixing a root cause for good

  • An excellent debugger across services, databases (Postgres, Redis) and distributed systems

  • High ownership, meaning you take responsibility for the customer impact of what you ship

  • BS/MS in Computer Science or a related field, or equivalent proof you’re exceptional

Nice to have

  • Rust, Go or C++ in production

  • Time at a fast-growing startup with a large, real codebase

  • Experience with LLM APIs, proxies or API gateways

  • Open-source contributions

Why LiteLLM

  • Your code runs in the critical path for some of the biggest companies in the world

  • Real ownership of core systems on a small, senior-leaning team

  • Profitable, growing fast, backed by Kleiner Perkins

  • $220-$260k base + equity, health, dental and vision. Onsite in San Francisco.

About LiteLLM

LiteLLM is a Python SDK and proxy server for calling 100+ LLM APIs in the OpenAI format, with 60K+ GitHub stars.

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,446,146 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Backend
Remote, many countries
Similar stack
Same company
San Francisco
$78k – $156k per year · In office · Full-Time · 6+ years exp · Bachelor's Degree · United States
Java
SQL
Java
Maven
Spring Boot
Hibernate
Databases
PostgreSQL
Apache Kafka
DevOps
Rest API
Azure
AWS
Management
Confluence
Agile
MedTech
HL7
FHIR
Design Controls
Apply
$99k – $199k per year · In office · Full-Time · 7+ years exp · High School Diploma · United States
Python
JavaScript
Java
SQL
C#
C++
Java
Maven
Spring Boot
Hibernate
C#
.NET
Databases
PostgreSQL
Apache Kafka
DevOps
Rest API
Azure
AWS
Management
Confluence
Agile
MedTech
HL7
FHIR
Apply
$97k – $145k per year · Hybrid · Full-Time · 5+ years exp · Bachelor's Degree · Tampa
Java
Kotlin
SQL
Java
Spring Framework
Maven
Spring Boot
Hibernate
Hazelcast
Gradle
Kotlin
Mockito
Databases
PostgreSQL
Redis
Oracle
MS SQL
Apache Kafka
Mobile
JUnit
DevOps
Rest API
Splunk
OpenShift
Prometheus
CI/CD
Jenkins
Git
Docker
Kubernetes
Grafana
TeamCity
GitHub
Cybersecurity
SonarQube
Management
Agile
Scrum
Apply
In office · Full-Time · Austin
Python
JavaScript
Java
Ruby
Node JS
Python
Flask
Django
Java
Spring Boot
Ruby
Ruby on Rails
Databases
MySQL
DevOps
Prometheus
Git
Docker
Kubernetes
Grafana
Cybersecurity
Checkmarx
Sonatype Nexus IQ
Management
Confluence
Jira
Apply
In office · Full-Time · New York
JavaScript
TypeScript
AI/ML
AI Agents
Frontend
React.js
DevOps
Windows
Apply
up to $43k per year (net) · Remote (likely EAEU) · 5+ years exp · Volgograd
JavaScript
Java
TypeScript
SQL
Java
Vert.x
Databases
PostgreSQL
Redis
ActiveMQ
Frontend
Svelte
Tailwind CSS
DevOps
GitHub
Cybersecurity
PCI DSS
Web3
Ethereum
Apply
Remote (likely EAEU) · 2+ years exp · Moscow
Java
Kotlin
SQL
Java
Spring Framework
Maven
Spring Boot
Hibernate
Gradle
Flyway
Liquibase
Databases
PostgreSQL
Redis
RabbitMQ
Apache Kafka
Mobile
JUnit
DevOps
Rest API
gRPC
CI/CD
Git
Docker
Kubernetes
SOAP
Apply
$28k per year · Remote (EAEU) · Minsk
Rust
Solidity
Solidity
Alloy
Web3
Foundry
Polygon
Solana
Aave
Uniswap
Ethereum
DeFi
MakerDAO
Apply
Remote (likely EAEU) · 3+ years exp · Saint Petersburg
JavaScript
PHP
TypeScript
SQL
PHP
Yii
Laravel
Databases
PostgreSQL
ClickHouse
AI/ML
AI Agents
Apply
$8.5k per year (net) · Remote (likely EAEU) · Full-Time · Ufa
Java
DevOps
Git
Apply
$200k – $275k per year · Remote (United States) · 5+ years exp · Bachelor's Degree · San Francisco
Python
AI/ML
Cursor
Claude Code
AI Agents
Apply
Customer Engineer 7 hours ago
$115k – $130k per year · Equity 0.1–0.2% · In office · Full-Time · San Francisco
Apply
In office · 2+ years exp · Master's Degree · San Francisco
Apply
In office · 1+ year exp · High School Diploma · San Francisco
Apply
In office · 1+ year exp · San Francisco
Apply
See all jobs
This is one of many
2,446,146 more open roles from verified company boards, updated every day.