{"id":62216,"url":"https://alion.io/job/litellm-senior-backend-engineer","title":"Senior Backend Engineer","company":{"id":20631,"name":"LiteLLM","domain":"litellm.ai","url":"https://alion.io/company/litellm","size_band":"11-50","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"A","score":100,"open_postings":16,"ghost_share":0,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":46,"computed_at":"2026-11-20T05:46:30Z"}},"role":"Backend","role_family":"Backend","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Francisco, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":200000,"max":260000,"currency":"USD","period":"year","gross":null,"usd_annual":260000},"salary_estimate":null,"experience_years_min":2,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Asyncio","optional":false},{"name":"Datadog","optional":false},{"name":"FastAPI","optional":false},{"name":"LiteLLM","optional":false},{"name":"LLM","optional":false},{"name":"LLM Guardrails","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"OpenTelemetry","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Prometheus","optional":false},{"name":"Python","optional":false},{"name":"Redis","optional":false},{"name":"Rust","optional":false},{"name":"Splunk","optional":false},{"name":"C++","optional":true},{"name":"GitHub","optional":true},{"name":"Go","optional":true},{"name":"OpenAI","optional":true}],"status":"live","first_seen_at":"2026-03-31T16:13:56Z","employer_posted_date":"2026-11-20","last_verified_at":"2026-11-20T19:38:18Z","board_verified":true,"closed_at":null,"days_open":234,"trust":{"level":"stale","repost_count":2,"flags":["stale"],"days_open":167},"description":"Senior Backend Engineer\nLiteLLM is the world’s most popular open-source AI Gateway, used by Adobe, Netflix, NASA, NVIDIA and much of the Fortune 500. Every LLM call these companies make runs through our code, so when we break, they break. We’re a team of about 20, and we’re hiring senior and staff backend engineers to own the core of the gateway and keep it fast and reliable as traffic grows.\nAbout the role\nYou’ll own critical parts of the LiteLLM proxy: routing, auth, caching, spend tracking, guardrails and observability. The job is to make these systems correct, fast and dependable at 1B+ requests a month. That means hunting down silent failures, fixing the root cause in code rather than adding another dashboard, and refactoring components that keep breaking without changing how they behave. You’ll set the bar for how our mostly early-career team writes, tests and ships backend code.\nWhat you’ll do\nOwn core proxy components end to end, from design through production, including the incidents\n\nFind and remove whole classes of bugs, like race conditions, stale cache state across replicas, and errors that get silently swallowed instead of shown to users\n\nMake sure every guardrail and policy check (e.g. apply_guardrail) is logged, traceable and fails loudly when it should\n\nBuild guardrails that run inside the gateway and block common attacks on LLM APIs, MCP servers and agents\n\nKeep our observability integrations (Datadog, Splunk, Prometheus, OpenTelemetry) accurate at 1B+ requests a month, with minimal latency overhead and no memory leaks (e.g. from Prometheus metric cardinality)\n\nHelp migrate performance-critical paths from Python to Rust without breaking behavior\n\nRaise the bar for the team on code review, testing and debugging\n\nWhat we’re looking for\n2+ years building backend systems in production, with real ownership of something mission-critical: payments, gateways, core APIs, data stores, or anything else where downtime hurts customers\n\nStrong Python, and experience with async or high-throughput services (asyncio, FastAPI or similar)\n\nA track record of making production systems more reliable through code: refactors, re-architectures, migrations, or fixing a root cause for good\n\nAn excellent debugger across services, databases (Postgres, Redis) and distributed systems\n\nHigh ownership, meaning you take responsibility for the customer impact of what you ship\n\nBS/MS in Computer Science or a related field, or equivalent proof you’re exceptional\n\nNice to have\nRust, Go or C++ in production\n\nTime at a fast-growing startup with a large, real codebase\n\nExperience with LLM APIs, proxies or API gateways\n\nOpen-source contributions\n\nWhy LiteLLM\nYour code runs in the critical path for some of the biggest companies in the world\n\nReal ownership of core systems on a small, senior-leaning team\n\nProfitable, growing fast, backed by Kleiner Perkins\n\n$220-$260k base + equity, health, dental and vision. Onsite in San Francisco.\n\nAbout LiteLLM\nLiteLLM is a Python SDK and proxy server for calling 100+ LLM APIs in the OpenAI format, with 60K+ GitHub stars.","description_format":"text","description_chars":3089,"description_truncated":false,"requirements":{"experience_years_min":2,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Equity"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["LLM & Generative AI","AI Compute & Inference","AI Development Tools"],"lifecycle":[{"event":"open","at":"2026-03-31T16:13:56Z"},{"event":"close","at":"2026-08-18T15:44:18Z"},{"event":"reopen","at":"2026-08-30T05:19:01Z"},{"event":"close","at":"2026-09-15T00:21:12Z"},{"event":"reopen","at":"2026-11-20T19:38:18Z"}],"visa":[],"liveness":{"score":4,"band":"cold","label":"Long shot","p_open":1,"p_active":0.136,"p_room":0.28,"age_days":234,"expected_fill_days":46,"reasons":["conf:3","repost:2","win:tail","crowd:"],"computed_at":"2026-11-20T22:50:17Z"},"pay":{"stated_usd_annual":260000,"is_top_pay":true},"html_url":"https://alion.io/job/litellm-senior-backend-engineer","json_url":"https://alion.io/job/litellm-senior-backend-engineer.json","meta":{"generated_at":"2026-11-20T22:50:17Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":30297,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-21T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":20631},"rest":"https://alion.io/mcp/rest/get_company?id=20631"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Flitellm-senior-backend-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Flitellm-senior-backend-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Flitellm-senior-backend-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/litellm-senior-backend-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Flitellm-senior-backend-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":62216}},{"tool":"search_jobs","arguments":{"company":"litellm.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":62216},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}