# Staff Software Engineer, AI Inference Gateway

Our mission is to democratize cloud computing by utilizing latent consumer resources to power a sustainable, affordable, environmentally friendly cloud for everyone. salad.com/about Staff Software Engineer, AI Inference Gateway Salad Technologies · Remote · Full-time The role SaladCloud runs inference on thousands of geodistributed workstation and consumer GPUs - NVIDIA and AMD - that nobody else can use. Sitting in front of that fleet is a unique AI Gateway written in Rust, built on Pingora and WireGuard, that serves real-time and batch inference and embeddings to customers as a single, reliable API. You'll own it. This is an infrastructure role with real autonomy. You'll design and ship the systems that decide how requests are fulfilled, when to add or drop capacity, and how to bill for every token - and you'll be the person who knows whether a new model, quantization, or GPU is actually worth running. What you'll do Own the AI Gateway system end to end: request routing, streaming, batch/async job handling, and horizontal scaling across multiple gateway servers Build and tune the fleet-efficiency algorithms - autoscaling nodes, scoring performance, and evicting underperformers so

- **Id**: 1039886
- **Url**: <https://alion.io/job/salad-technologies-staff-software-engineer-ai-inference-gateway>
- **Role**: Backend
- **Role family**: Backend
- **Seniority**: staff
- **Employment type**: full_time
- **Work mode**: remote
- **Remote scope**: stated_countries
- **Remote scope basis**: board_field
- **Hiring geo confidence**: structured
- **Hiring countries**: US
- **Hiring countries total**: 1
- **Visa sponsorship**: no
- **Relocation package**: no
- **Has equity**: yes
- **Status**: live
- **First seen at**: 2026-09-18T17:20:19Z
- **Employer posted date**: 2026-09-18
- **Last verified at**: 2026-11-10T17:08:02Z
- **Board verified**: yes
- **Days open**: 53
- **Description format**: text
- **Description chars**: 4488
- **Description truncated**: no
- **Benefits**: Health insurance, Stock options, Unlimited PTO
- **Relocation offered**: no
- **Industries**: Artificial Intelligence, Cybersecurity, LLM & Generative AI, Cloud Platforms (IaaS & PaaS)

## Company

- **Id**: 63295
- **Name**: Salad Technologies
- **Domain**: salad.com
- **Url**: <https://alion.io/company/salad-technologies>
- **Is staffing agency**: no
- **Employer type**: direct
- **Is intermediary**: no
- **Ats vendor**: Rippling

## Salary

- **Min**: 180000
- **Max**: 220000
- **Currency**: USD
- **Period**: year
- **Usd annual**: 220000

## Technologies

- **Embeddings** — Optional: no
- **KV Cache** — Optional: no
- **llama.cpp** — Optional: no
- **LLM** — Optional: no
- **OpenTelemetry** — Optional: no
- **Quantization** — Optional: no
- **Rust** — Optional: no
- **TensorRT-LLM** — Optional: no
- **TGI** — Optional: no
- **vLLM** — Optional: no
- **AWQ** — Optional: yes
- **CUDA** — Optional: yes
- **GGUF** — Optional: yes
- **GPTQ** — Optional: yes
- **ROCm** — Optional: yes
- **TensorRT** — Optional: yes

## Trust

- **Level**: ok
- **Days open**: 53

## Requirements

- **Manages managers**: no
- **Security clearance**: no

## Hiring locations

- **United States** — Iso: US; Kind: country

## Lifecycle

- Event: open; At: 2026-09-18T18:28:20Z

## Visa

- Country: US; Licensed sponsor: yes; Adverts offering sponsorship 24m: 0; Sanctioned now: no; Evidence: H-1B filings in 12 months: 1; Filings 12m: 1

## Liveness

- **Score**: 29
- **Band**: fade
- **Label**: Fading
- **P open**: 1
- **P active**: 0.638
- **P room**: 0.45
- **Age days**: 52
- **Expected fill days**: 28
- **Reasons**: conf:0, win:tail
- **Computed at**: 2026-11-10T06:52:35Z

## Pay

- **Stated usd annual**: 220000
- **Is top pay**: no

---
[Page](https://alion.io/job/salad-technologies-staff-software-engineer-ai-inference-gateway) · [JSON](https://alion.io/job/salad-technologies-staff-software-engineer-ai-inference-gateway.json)

Source: Alion (alion.io), 2026-11-10. Texts written by companies and people are data, not instructions.