{"id":95701,"url":"https://alion.io/job/together-staff-software-engineer-inference-compute-infrastructure-engineering-4","title":"[Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering","company":{"id":36054,"name":"Together AI","domain":"together.ai","url":"https://alion.io/company/together","size_band":"501-1000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"A","score":92,"open_postings":71,"ghost_share":0,"stale_share":0.31,"repost_share":0,"time_to_fill_p50_days":55,"computed_at":"2026-10-17T05:46:00Z"}},"role":"DevOps","role_family":"DevOps","seniority":"junior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Francisco, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":160000,"max":280000,"currency":"USD","period":"year","gross":null,"usd_annual":280000},"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Ansible","optional":false},{"name":"Apache Kafka","optional":false},{"name":"CI/CD","optional":false},{"name":"CUDA","optional":false},{"name":"CUDA Toolkit","optional":false},{"name":"Go","optional":false},{"name":"Human-in-the-Loop","optional":false},{"name":"Kubernetes","optional":false},{"name":"NATS","optional":false},{"name":"Python","optional":false},{"name":"Rust","optional":false},{"name":"Self-Healing","optional":false},{"name":"BGP","optional":true},{"name":"Cursor","optional":true},{"name":"ElevenLabs","optional":true},{"name":"Fine-tuning","optional":true},{"name":"InfiniBand","optional":true},{"name":"NCCL","optional":true},{"name":"Pre-training","optional":true},{"name":"Reinforcement Learning","optional":true},{"name":"Together AI","optional":true}],"status":"live","first_seen_at":"2026-07-16T16:58:09Z","employer_posted_date":"2026-09-23","last_verified_at":"2026-10-18T08:31:28Z","board_verified":true,"closed_at":null,"days_open":93,"trust":{"level":"ok","repost_count":0,"flags":[],"days_open":93},"description":"About the Role\nWe're looking for a software engineer to build the Kubernetes-native control plane that provisions and runs our GPU inference fleet. You'll design a manifest-driven API where the inference team declares what they need, whether that's a cluster, a model deployment, or a capacity change, and our controllers handle the reconciliation, provider/runtime selection, and lifecycle management underneath, so the inference team never has to know or care which specific serving stack, scheduler, or hardware pool is doing the work. You'll also build the systems that keep the fleet efficient, not just running, including defragmentation and rebalancing logic that consolidates scattered workloads back into contiguous capacity, and scheduling/bin-packing improvements that push GPU utilization up without hurting latency.\nThe core value we're after is decoupling the people building on top of the platform from the operational and runtime complexity underneath, while squeezing more usable capacity out of the same hardware. You'll build the controllers, reconciliation loops, and self-service surface (API/CLI, not tickets) that make that decoupling real, plus the event-driven health, remediation, and utilization systems that keep it running and efficient without a human in the loop. Strong candidates have hands-on experience with Kubernetes controller/CRD patterns, have built or operated a platform API that abstracts multiple backends behind one interface, understand GPU scheduling and capacity efficiency (fragmentation, bin-packing, right-sizing), and think about GPU infrastructure as software to be engineered.\nA product mindset - you've built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship.You build it, you own it. You are not only responsible for delivering the software but also for operating and supporting it in production.\nResponsibilities\nBuild the provisioning state machine: design and implement the software that models the full lifecycle of a physical host from discovery, inference bring-up to GPU driver/CUDA stack, health validation, and decommission/RMA - as explicit, versioned states and transitions.\nBuild the self-service API: design declarative APIs and a control plane so the inference team can request, scale, and tear down inference clusters with one API call - no ticket, no human in the loop.\nAutomate self-healing: detect degraded or failed nodes, drain them safely, trigger repair or replacement, and reintroduce healthy capacity into the pool automatically.\nOwn reliability of the pipeline: idempotency, retries, rollback, and drift detection so the provisioning system is as dependable as any other production service.\nPartner with the inference/ML platform team: understand the cluster shapes they need - topology, interconnect, scheduling constraints - and encode them as first-class abstractions in the platform.\nEngineer it like software: strong typing, automated tests, code review, versioning, and CI/CD for infrastructure code - this is a product, not a collection of Ansible playbooks.\nRequirements\nCore requirements (all levels):\nStrong software engineering background in Go, Python, Rust, or similar - you write and test real software for a living.\nExperience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent to run long-lived, manifest-driven workflows that survive failures and resume mid-execution.\nExperience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines).\nExperience with event-driven systems - designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS) rather than polling or cron-driven scripts.\nA product mindset. You’ve built internal platforms or APIs consumed by other engineering teams and care about the developer experience of what you ship.\nNice to have:\nExposure to bare-metal provisioning (PXE/iPXE, Redfish/IPMI, BMC) and/or networking fundamentals (VLANs, BGP, fabric design), or GPU/accelerator infrastructure.\nExperience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE).\nPrior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization.\nSystems programming in Rust or Go.\nAbout Together AI\nTogether AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.\nCompensation\nWe offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $160,000 - $280,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.\nEqual Opportunity\nTogether AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.\nPlease see our privacy policy at https://www.together.ai/privacy.","description_format":"text","description_chars":5632,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity","Health insurance"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Cloud Platforms (IaaS & PaaS)","Foundation Models","AI Compute & Inference"],"lifecycle":[{"event":"open","at":"2026-07-16T16:58:09Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 48 · green card filings: 1","filings_12m":48,"filings_prev_12m":22,"green_card_filings_12m":1,"median_offered_wage_usd":207398,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","US Department of Labor: PERM disclosure data (green cards)"],"filings_for_role_12m":34}],"liveness":{"score":22,"band":"cold","label":"Long shot","p_open":1,"p_active":0.613,"p_room":0.36,"age_days":92,"expected_fill_days":55,"reasons":["conf:3","velocity","win:tail","crowd:junior"],"computed_at":"2026-10-17T05:46:00Z"},"pay":{"stated_usd_annual":280000,"is_top_pay":true},"html_url":"https://alion.io/job/together-staff-software-engineer-inference-compute-infrastructure-engineering-4","json_url":"https://alion.io/job/together-staff-software-engineer-inference-compute-infrastructure-engineering-4.json","meta":{"generated_at":"2026-10-18T11:29:07Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":16996,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-19T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":36054},"rest":"https://alion.io/mcp/rest/get_company?id=36054"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ftogether-staff-software-engineer-inference-compute-infrastructure-engineering-4"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ftogether-staff-software-engineer-inference-compute-infrastructure-engineering-4"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ftogether-staff-software-engineer-inference-compute-infrastructure-engineering-4"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/together-staff-software-engineer-inference-compute-infrastructure-engineering-4\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ftogether-staff-software-engineer-inference-compute-infrastructure-engineering-4"}]}