2,461,507open jobs
149,660companies
318,431added this week
Browse all
Salary
$260k – $300k per year
Location
Remote (United States)
Seniority
Staff · 7+ years exp
Visa
H-1B filings in 12 months: 62 · for this role: 44 · green card filings: 6
Employment
Full-Time

Confirmed on the employer's own hiring board on Nov 22, 2026. First seen by Alion on Sep 23, 2026. Together AI scores B on the Alion truth index.

Overview
Company
Impact
Profile match
Together AI is an American AI infrastructure company headquartered in San Francisco that runs a cloud platform for training, fine-tuning and serving open-source generative AI models. Founded in 2022 by Vipul Ved Prakash, Ce Zhang, Chris Re and Percy Liang, with Tri Dao as chief scientist, it offers GPU clusters, inference APIs and fine-tuning services to developers and enterprises. It hires infrastructure and inference reliability engineers, researchers, field marketing managers, recruiters and finance staff, with further offices in Amsterdam, London and India.

About the Role

Together AI is building the AI Native Cloud, an end-to-end platform for the full generative AI lifecycle, combining the fastest LLM inference engine with state-of-the-art GPU cloud infrastructure. The Together Cloud team builds the [Together GPU Clusters](https://www.together.ai/gpu-clusters) flagship IaaS product that provides high-performance, AI-ready GPU clusters through a self-serve cloud console, along with the virtualized infrastructure layer powering Together's inference, RL, and fine-tuning products.

As a Staff Software Engineer focusing on AI Compute in the Together Cloud org, you will set technical direction for and build major components of the next generation AI cloud platform - a highly available, global cloud infrastructure with cutting-edge virtualization of the latest ML hardware: GB300s/VRs, BlueField DPUs, InfiniBand and dual/quad-plane RoCEv2 fabrics. That virtualized computing platform powers our own SaaS products - inference, RL, and fine-tuning - and serves external cloud customers through self-serve offerings such as on-demand/reserved Kubernetes/Slurm clusters, across dozens of data centers and hundreds of thousands of GPUs.

This is an architect-and-build role. Fully automated bootstrapping of GPU data centers, high-performance virtualization of GPU compute and DC networking without compromising isolation or portability, and fault-tolerant decentralized control planes - you'll set the architecture for these across our global and in-DC services, and be a key owner of the hardest parts, in the code as well as the design. Your designs will span the IaaS layer of a greenfield Vera Rubin data center up to the global management plane that schedules capacity across all of them. At this level the job is as much leverage as code: the standards you set and the engineers you grow decide how fast the rest of Together Cloud ships.

Responsibilities

  • Own the GPU and network virtualization stack: the hypervisor, kernel, and SDN work that keeps GPU compute and DC networking high-performance, portable, and strongly isolated across heterogeneous hardware.
  • Own the in-DC IaaS layer: architect and roadmap the services, Kubernetes operators, and libraries that provision and manage compute, storage, and networks in our data centers - VMs, parallel filesystems, VPCs, and InfiniBand partitions; lead its build-out for a new Vera Rubin data center with thousands of GPUs, from hardware bring-up to customer-facing API.
  • Design the GPU scheduling and global management plane: the distributed control plane behind on-demand and reserved clusters across dozens of data centers, including the systems that scale per-cluster limits and automate the onboarding of new capacity.
  • Architect monitoring and automated remediation for fault tolerance: the strategy for automated detection, isolation, and recovery of failed nodes that keeps distributed pretraining and large-scale inference running through hardware failures.
  • Set technical direction across teams: lead design reviews, resolve cross-cutting architectural disagreements, unblock cross-team dependencies and integration risks, and define the standards other engineers build against - measured in cluster reliability, time-to-first-GPU on new capacity, and quality at scale.
  • Grow the team: mentor senior and junior engineers, deepen the team's expertise in virtualization, DC networking, and GPU infrastructure, and help raise the hiring bar for Together Cloud.
  • Set the engineering bar: create the testing frameworks, tools, and developer documentation that make our systems robust and usable by other teams, and shape the core, open-source Together AI platform.

To be successful you'll need to be deeply technical and an excellent communicator - expert software development fundamentals, deep systems knowledge and troubleshooting instincts, and the leadership and diplomacy skills to align teams that don't report to you. Much of this work starts ambiguous, and we expect you to define the scope yourself and drive it to production.

Requirements

  • 7+ years of professional software development experience, with expert-level proficiency in at least one backend language (Golang desired), writing high-performance, well-tested, production-quality code.
  • Track record of owning the architecture of large distributed systems from blank page to production at scale, including the judgment calls that could not be reversed cheaply.
  • Deep experience building and operating globally distributed, high-performance microservice architectures across one or more cloud providers (AWS, Azure, GCP).
  • Expert systems knowledge across compute, networking, and storage - including concurrency, memory management, performant I/O, and scale at a global level.
  • Demonstrated technical leadership beyond your own commits: mentoring senior engineers, leading design reviews, and driving alignment across teams that do not report to you.
  • Excellent communication and diplomacy skills - able to write design docs that settle arguments, and to work effectively with technical and non-technical stakeholders.
  • Experience building and operating reliable, customer-facing production systems at scale, and owning the infrastructure automation (Terraform, Ansible), observability (Prometheus, Grafana), and CI/CD (GitHub Actions, ArgoCD) that keep them healthy.

Preferred Qualifications 

  • Deep Kubernetes internals experience, such as implementing non-trivial Kubernetes operators, device/storage/network plugins, custom schedulers, or patches to Kubernetes itself
  • Deep experience with VMs/hypervisors, such as QEMU/KVM, cloud-hypervisor, VFIO, virtio, PCIE passthrough, Kubevirt, SR-IOV
  • Deep experience with DC networking tech + solutions, such as VLAN, VXLAN, VPN, VPC, OVS/OVN
  • Experience with Cluster API or similar
  • Experience working on high-performance compute, networking, and/or storage
  • Experience virtualizing GPUs and/or InfiniBand
  • Experience building IaaS or PaaS systems at scale
  • Experience with DPUs/SmartNICs
  • GPU programming, NCCL, CUDA knowledge

About Together AI

Together AI is a research-driven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society, and together we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI, and our team has been behind technological advancement such as FlashAttention, Hyena, FlexGen, and RedPajama. We invite you to join a passionate group of researchers in our journey in building the next generation AI infrastructure.

Compensation

We offer competitive compensation, startup equity, health insurance, and other benefits, as well as flexibility in terms of remote work. The US base salary range for this full-time position is: $260,000 - $300,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Application Policy

Please read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.

Equal Opportunity

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Please see our privacy policy at  https://www.together.ai/privacy  

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,461,507 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Backend
Similar stack
Same company
San Francisco
$145k – $250k per year · In office · Full-Time · 4+ years exp · Bachelor's Degree · Seattle
Python
Java
Rust
C++
AI/ML
AI Agents
LLM
Apply
$231k – $340k per year · In office · Full-Time · 4+ years exp · San Francisco
JavaScript
TypeScript
Frontend
React.js
DevOps
Windows
Apply
$200k – $300k per year · In office · Full-Time · 5+ years exp · New York
Python
JavaScript
TypeScript
Node JS
Databases
PostgreSQL
RabbitMQ
Apache Kafka
AI/ML
AI Agents
Recommender Systems
Apply
$200k – $300k per year · In office · Full-Time · 2+ years exp · San Francisco
TypeScript
Databases
PostgreSQL
Redis
ClickHouse
AI/ML
AI Agents
Machine Learning
DevOps
Terraform
GCP
Azure
AWS
Docker
AWS Step Functions
Apply
$140k – $200k per year · In office · Full-Time · 3+ years exp · New York
JavaScript
TypeScript
Node JS
Databases
PostgreSQL
AI/ML
AI Agents
Recommender Systems
Frontend
React.js
Mobile
React Native
Apply
In office · Full-Time · 15+ years exp · Hyderabad
Python
TypeScript
AI/ML
AI Agents
LLM
LLM Guardrails
DevOps
Terraform
Ansible
GCP
Azure DevOps
GitHub Actions
Azure
CI/CD
GitOps
Kubernetes
Platform Engineering
Chaos Engineering
SLI/SLO/SLA
GitHub
Cybersecurity
Wiz
CIS Benchmarks
Shift-Left Security
SBOM
SLSA
Sigstore
Cosign
Apply
$169k – $205k per year · Hybrid · Full-Time · 8+ years exp · Bachelor's Degree · Tampa · Jersey City · Tempe
Python
Java
TypeScript
SQL
Java
Maven
Spring Boot
Hibernate
Spring Cloud
Databases
MySQL
PostgreSQL
Db2
Oracle
Apache Kafka
AI/ML
Copilot
LangGraph
AutoGen
LangChain
Claude
Model Context Protocol
Embeddings
Prompt Engineering
AI Agents
AWS Bedrock
CrewAI
LLM
RAG
OpenAI
Anthropic
A2A
Semantic Search
Agentic Workflows
Frontend
GraphQL
DevOps
Rest API
Splunk
GitHub Actions
Datadog
CI/CD
Jenkins
Git
AWS
Docker
Kubernetes
Amazon EKS
AWS Lambda
Amazon EC2
Amazon S3
IAM
Amazon CloudWatch
API Gateway
Cybersecurity
Keycloak
Analytics
Tableau
Power BI
Management
Agile
Scrum
Apply
In office · Full-Time · Islamabad
Python
JavaScript
SQL
C#
DevOps
Rest API
GCP
Azure
Git
AWS
Management
Agile
Apply
Hybrid · Full-Time · Bachelor's Degree · Hyderabad
Apex
Apex
Lightning Web Components
Gearset
Copado
Visualforce
AI/ML
AI Agents
DevOps
Rest API
GCP
CI/CD
SOAP
Management
Agile
Scrum
Kanban
Apply
Hybrid · Full-Time · High School Diploma · Hyderabad
Python
PowerShell
Bash
AI/ML
AI Agents
DevOps
Terraform
Azure
Windows Server
Git
AWS
Docker
Kubernetes
Amazon EC2
GitHub
GitLab
IAM
Windows
Management
ITIL
ITSM
Apply
$250k – $300k per year · Remote (United States) · Full-Time · 5+ years exp · San Francisco
Python
Go
Rust
TypeScript
Databases
NATS
Apache Kafka
AI/ML
Cursor
ElevenLabs
Fine-tuning
Reinforcement Learning
Function Calling
AI Agents
LLM
RAG
Together AI
Pre-training
Semantic Search
Knowledge Graph
DevOps
Prometheus
GitOps
ArgoCD
Kubernetes
Grafana
Incident Management
Apply
$160k – $230k per year · In office · Full-Time · 4+ years exp · San Francisco
Python
Go
JavaScript
TypeScript
AI/ML
Cursor
ElevenLabs
Fine-tuning
Reinforcement Learning
AI Agents
Together AI
Pre-training
Agentic Workflows
Frontend
Tailwind CSS
Next.js
React.js
Radix UI
shadcn/ui
DevOps
CI/CD
Grafana
Apply
$250k – $330k per year · Remote (United States) · Full-Time · 8+ years exp · Bachelor's Degree · San Francisco
Python
Rust
C++
Databases
MinIO
AI/ML
vLLM
SGLang
TensorRT
TensorRT-LLM
Together AI
InfiniBand
KV Cache
DevOps
Terraform
Ansible
Helm
Prometheus
GitOps
ArgoCD
Kubernetes
Grafana
Thanos
Amazon S3
HPC
Linux
Apply
$200k – $280k per year · Remote (United States) · Full-Time · San Francisco
Python
Databases
PostgreSQL
Redis
ClickHouse
AI/ML
Cursor
ElevenLabs
Fine-tuning
Reinforcement Learning
LLM
Anomaly Detection
Time Series
Together AI
Pre-training
DevOps
Terraform
Ansible
GCP
Helm
OpenTelemetry
Prometheus
Azure
CI/CD
GitOps
AWS
Docker
Kubernetes
Grafana
Chaos Engineering
Service Mesh
SLI/SLO/SLA
Apply
$200k – $290k per year · In office · Full-Time · 5+ years exp · Bachelor's Degree · San Francisco
Python
Rust
TypeScript
AI/ML
Cursor
vLLM
ElevenLabs
Fine-tuning
Reinforcement Learning
Multimodal AI
SGLang
Together AI
CUDA
Triton
Pre-training
NCCL
InfiniBand
NVLink
DevOps
Kubernetes
SLI/SLO/SLA
HPC
Apply
≈ $155k – $286k per year (Estimated) · In office · 6+ years exp · Bachelor's Degree · San Francisco
Python
Julia
Databases
Snowflake
AI/ML
Machine Learning
DevOps
GCP
CI/CD
Management
Agile
Apply
$300k – $400k per year · Hybrid · Full-Time · 5+ years exp · PhD · San Francisco · Bellevue
Python
AI/ML
Machine Learning
DevOps
CI/CD
Apply
Software Engineer 1 day ago
In office · Full-Time · San Francisco
AI/ML
Copilot
Function Calling
AI Agents
RAG
Design
SolidWorks
SpaceTech
CATIA
Siemens NX
Apply
$130k – $170k per year · In office · Full-Time · 2+ years exp · San Francisco
Python
JavaScript
TypeScript
Node JS
Node JS
Prisma
Databases
Supabase
AI/ML
Embeddings
TensorFlow
PyTorch
LLM
Recommender Systems
Machine Learning
Apply
$48k per year · In office · Full-Time · Bachelor's Degree · New York · Waltham · San Francisco · Seattle
Apply
See all jobs
This is one of many
2,461,507 more open roles from verified company boards, updated every day.