{"id":3361434,"url":"https://alion.io/job/cerebras-systems-senior-software-engineer-ml-api-ai-inference","title":"Senior Software Engineer, ML API – AI Inference","company":{"id":31,"name":"Cerebras Systems","domain":"cerebras.ai","url":"https://alion.io/company/cerebras-systems","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"A","score":88,"open_postings":114,"ghost_share":0.123,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":97,"computed_at":"2026-10-30T05:48:23Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Sunnyvale, United States","Toronto, Canada"],"countries":["US","CA"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":209000,"max_usd":403000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":1332},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"C++","optional":false},{"name":"Cerebras","optional":false},{"name":"CI/CD","optional":false},{"name":"Function Calling","optional":false},{"name":"Go","optional":false},{"name":"gRPC","optional":false},{"name":"Hugging Face","optional":false},{"name":"Kubernetes","optional":false},{"name":"Linux","optional":false},{"name":"Machine Learning","optional":false},{"name":"Multimodal AI","optional":false},{"name":"OpenAI","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"Rest API","optional":false},{"name":"Rust","optional":false},{"name":"Structured Outputs","optional":false},{"name":"Tokenization","optional":false},{"name":"vLLM","optional":false},{"name":"CUDA","optional":true},{"name":"KV Cache","optional":true},{"name":"LLM","optional":true},{"name":"Quantization","optional":true},{"name":"ROCm","optional":true},{"name":"SGLang","optional":true},{"name":"TensorRT","optional":true},{"name":"TensorRT-LLM","optional":true},{"name":"Transformers","optional":true},{"name":"Triton","optional":true}],"status":"live","first_seen_at":"2026-10-27T19:51:12Z","employer_posted_date":"2026-10-27","last_verified_at":"2026-10-30T19:46:38Z","board_verified":true,"closed_at":null,"days_open":3,"trust":{"level":"ok","repost_count":0,"flags":["company_stale"],"days_open":1},"description":"Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services. \nThis order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation.\nCerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.\nAbout the Role\nCerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine.\nWe are hiring a Senior Software Engineer to build and evolve the ML API layer that makes this heterogeneous serving system accessible, reliable, and easy to use. You will own significant features and components across our inference APIs, model integration layer, and request-routing services, delivering a consistent experience across models and accelerator backends.\nThis role sits at the intersection of machine learning systems, API design, model serving, and distributed systems. You will translate evolving model capabilities and customer requirements into clear designs, lead implementation across multiple components, and carry features through validation, rollout, and production operation. Your work will help ensure that streaming, sampling, tool use, structured outputs, multimodal inputs, and model configuration behave correctly and consistently.\nYou will work closely with model enablement, compiler, runtime, cloud infrastructure, product, and customer-facing teams, as well as customers directly. This is a hands-on individual-contributor role with opportunities to lead small project teams, mentor engineers, and improve the quality and reliability of production inference software.\nResponsibilities\nOwn features end to end. Design, implement, test, and operate features for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and emerging inference capabilities.\n\nLead multi-component delivery. Write clear design documents, evaluate technical tradeoffs, break projects into deliverable milestones, and coordinate implementation with a small group of engineers and partner teams.\n\nDeliver consistent API behavior. Implement clear request and response semantics across GPU prefill, Cerebras decode, and other inference backends. Maintain compatibility with widely adopted inference interfaces and evolve owned APIs through validation, versioning, and deprecation best practices.\n\nEnable new models and capabilities. Integrate foundation models, tokenizers, chat templates, generation configuration, sampling methods, and model-specific features into the serving platform.\n\nIntegrate serving components. Extend inference services and their integration with frameworks and runtimes such as vLLM, PyTorch, Hugging Face libraries, LLGuidance, and Cerebras runtime components.\n\nSupport disaggregated inference. Implement routing, state-transfer integration, error handling, retries, cancellation, and request lifecycle management to coordinate GPU prefill with Cerebras decode.\n\nDeliver measurable performance improvements. Diagnose bottlenecks and improve time to first token, streaming latency, throughput, and resource efficiency through changes to tokenization, serialization, batching, scheduling, and communication.\n\nOwn correctness and release quality. Build conformance tests, model-validation suites, numerical comparisons, integration tests, and performance benchmarks that catch regressions in owned features and support safe releases.\n\nStrengthen production reliability. Improve service metrics, tracing, logging, alerts, and diagnostic tools. Investigate production incidents, resolve root causes, and follow through on improvements to resilience and incident response.\n\nImprove customer and developer experience. Partner with product, support, and solutions teams to understand user needs and make practical tradeoffs. Improve configuration, documentation, examples, and debugging workflows.\n\nRaise the engineering quality bar. Provide thoughtful design and code reviews, mentor less-experienced engineers, share domain knowledge, and proactively remove delivery roadblocks.\n\nMinimum Qualifications\n5+ years of software engineering experience, including independent ownership of production services, distributed systems, or substantial software components.\n\nStrong programming ability in Python and proficiency in Go, C++, Rust, or another language used to build production services.\n\nExperience designing and delivering features that span multiple components, including documenting designs, evaluating tradeoffs, and coordinating implementation with other engineers.\n\nExperience building and evolving production APIs, with sound practices for validation, error handling, compatibility, and testing. Familiarity with REST, gRPC, or streaming protocols.\n\nExperience integrating software across service, framework, runtime, or infrastructure boundaries.\n\nExperience operating services on Linux using containers, Kubernetes or comparable orchestration systems, and CI/CD.\n\nAbility to diagnose correctness, reliability, and performance issues across distributed components and deliver measurable improvements.\n\nStrong communication and collaboration skills, including experience leading technical delivery for a small project and supporting other engineers through reviews or mentorship.\n\nAbility to turn ambiguous requirements into maintainable software while balancing customer needs, delivery timelines, and operational quality.\n\nBachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.\n\nPreferred Qualifications\nExperience building or operating production ML or LLM inference APIs, including OpenAI-compatible interfaces.\n\nExperience extending or contributing to vLLM, SGLang, PyTorch, Hugging Face Transformers, Triton, TensorRT-LLM, or another ML systems project.\n\nUnderstanding of tokenization, chat templates, generation configuration, logits processing, stopping criteria, tool calling, structured generation, or constrained decoding.\n\nExperience with disaggregated prefill/decode architectures, KV-cache transfer, prefix caching, chunked prefill, or inference request scheduling.\n\nExperience building API conformance, model-quality, numerical-comparison, determinism, or performance-regression test systems.\n\nExperience with multi-model or multi-tenant serving, including routing, admission control, quotas, rate limiting, or capacity-aware scheduling.\n\nExperience developing performance-sensitive or highly concurrent services in Go, C++, Rust, or a similar language.\n\nFamiliarity with GPU inference, AMD Instinct accelerators, ROCm, CUDA, collective communication, or multi-node model serving.\n\nFamiliarity with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4.\n\nExperience building SDKs, developer tools, configuration systems, or self-service ML workflows, or collaborating directly with customers and open-source communities.\n\nWhy Join Cerebras\nPeople who are serious about software make their own hardware. At Cerebras, we have built a breakthrough architecture that is unlocking new opportunities for the AI industry. With dozens of model releases and rapid growth, we’ve reached an inflection point in our business. Members of our team tell us there are five main reasons they joined Cerebras:\nBuild a breakthrough AI platform beyond the constraints of the GPU.\n\nPublish and open source their cutting-edge AI research.\n\nWork on one of the fastest AI supercomputers in the world.\n\nEnjoy job stability with startup vitality.\n\nOur simple, non-corporate work culture that respects individual beliefs.\n\nFind out more about what it's like to work at Cerebras here!\nApply today and become part of the forefront of groundbreaking advancements in AI!\nCerebras Systems is committed to creating an equal and diverse environment and is proud to be an equal opportunity employer. We celebrate different backgrounds, perspectives, and skills. We believe inclusive teams build better products and companies. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them.\nThis website or its third-party tools process personal data. For more details, click here to review our CCPA disclosure notice.","description_format":"text","description_chars":8804,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Continuous learning"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"},{"name":"Canada","iso":"CA","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Servers & Data Center Hardware","AI Compute & Inference","AI Chips & Accelerators"],"lifecycle":[{"event":"open","at":"2026-10-27T21:30:27Z"}],"visa":[{"country":"CA","licensed_sponsor":true,"evidence":"Positions approved through LMIA in 12 months: 2","filings_12m":2,"filings_prev_12m":8,"green_card_filings_12m":null,"median_offered_wage_usd":null,"route":"High Wage; Global Talent Stream","cap_exempt":false,"checked_at":"2026-10-03T21:10:52+00:00","sources":["ESDC: employers issued a positive LMIA"],"filings_for_role_12m":2}],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":2,"expected_fill_days":97,"reasons":["conf:3","velocity","win:early","comp:brand"],"computed_at":"2026-10-30T05:48:23Z"},"pay":null,"html_url":"https://alion.io/job/cerebras-systems-senior-software-engineer-ml-api-ai-inference","json_url":"https://alion.io/job/cerebras-systems-senior-software-engineer-ml-api-ai-inference.json","meta":{"generated_at":"2026-10-30T21:56:54Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":12514,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-31T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":31},"rest":"https://alion.io/mcp/rest/get_company?id=31"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcerebras-systems-senior-software-engineer-ml-api-ai-inference"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcerebras-systems-senior-software-engineer-ml-api-ai-inference"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcerebras-systems-senior-software-engineer-ml-api-ai-inference"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/cerebras-systems-senior-software-engineer-ml-api-ai-inference\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcerebras-systems-senior-software-engineer-ml-api-ai-inference"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3361434}},{"tool":"search_jobs","arguments":{"company":"cerebras.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3361434},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}