{"id":14209,"url":"https://alion.io/job/baseten-software-engineer-voice-ai-inference-runtime","title":"Software Engineer - Voice AI (Inference Runtime)","company":{"id":4399,"name":"Baseten","domain":"baseten.co","url":"https://alion.io/company/baseten","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"B","score":72,"open_postings":17,"ghost_share":0,"stale_share":0.882,"repost_share":0.059,"time_to_fill_p50_days":92,"computed_at":"2026-10-19T05:45:15Z"}},"role":"Backend","role_family":"Backend","seniority":null,"employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Francisco, United States","New York, United States","Toronto, Canada","Montreal, Canada","Seattle, United States"],"countries":["US","CA"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":165000,"max":330000,"currency":"USD","period":"year","gross":null,"usd_annual":330000},"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Baseten","optional":false},{"name":"Cursor","optional":false},{"name":"Fine-tuning","optional":false},{"name":"Notion","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"Text-to-Speech","optional":false},{"name":"Voice Agents","optional":false},{"name":"Whisper","optional":false},{"name":"Claude Code","optional":true},{"name":"Docker","optional":true},{"name":"Kubernetes","optional":true},{"name":"Machine Learning","optional":true},{"name":"ONNX","optional":true},{"name":"OpenAI Codex","optional":true},{"name":"PyTorch","optional":true},{"name":"TensorRT","optional":true},{"name":"vLLM","optional":true}],"status":"live","first_seen_at":"2026-04-23T23:48:25Z","employer_posted_date":"2026-04-23","last_verified_at":"2026-10-19T14:25:32Z","board_verified":true,"closed_at":null,"days_open":178,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":178},"description":"ABOUT BASETEN\nBaseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products.\nTHE ROLE\nVoice is becoming the internet’s next interface, but a production-grade Voice AI system is\"hard to build\". You’ll join a small founding team of Baseten Voice AI, focused on bringing state-of-the-art open source models into production for Voice AI customers across productivity, customer service, clinical conversation, creator tools, education, and more. You’ll make a meaningful impact on people’s daily lives and help reshape these industries.\nThis is a high-impact, high-ownership role. You will be the primary owner of Baseten Voice AI - our in-house inference stack to power Voice AI models - from product roadmap through engineering implementation. You’ll partner closely with Forward Deployed Engineers, Model Performance Engineers, and sister engineering teams to push the boundaries of Voice AI.\nEXAMPLE INITIATIVES\nDevelop world-class model serving stack for state-of-the-art open-source voice models - reduce end-to-end and tail latency (p95/p99), increase throughput, and improve GPU efficiency via profiling, runtime tuning, and server-level optimizations.\n\nBuild large-scale, real-time infrastructure for multi-model voice agents - orchestrate STT, TTS, and agent components with streaming I/O to meet customer SLOs.\n\nDesign tight training and inference iteration loops for voice model customization - enable fast evaluation, safe rollout, and rapid experimentation for custom voice model development.\n\nPast projects:\nBaseten leads Coval’s voice AI benchmark\n\nHow Notion achieved 2-3x lower latency with AI Meeting Notes\n\nCost-efficient, high-performance TTS with Qwen3-TTS\n\nFine-tuning Qwen3-TTS for high-quality voice cloning\n\nThe fastest Whisper - with streaming and diarization\n\nNVIDIA Nemotron 3 Diarization: real-time speaker labels at a cent per audio hour\n\nRESPONSIBILITIES\nOwn and lead Voice AI product areas end-to-end - from architecture and system design through implementation, rollout, and long-term production operations.\n\nDesign, build, and operate real-time, large-scale, high-performance model serving systems for STT, TTS, and voice agent workloads for mission-critical customer deployments\n\nDrive cross-team collaboration with sister engineering teams to solve full-stack technical problems, align on priorities, and coordinate end-to-end delivery across the product surface area\n\nMentor teammates through code reviews, design docs, and technical leadership.\n\nREQUIREMENTS\nBachelor's degree or higher in Computer Science or related field\n\nProven track record owning production-grade real-time, large-scale systems where tail latency (p99) matters.\n\nProficient coding abilities in one or more popular programming or scripting languages; Python proficiency is a plus.\n\nGood taste in product, particularly developer-oriented tools\n\nInterest in ML/AI infrastructure and willingness to learn\n\nStrong collaboration and communication skills\n\nComfortable using AI coding assistants (e.g., Claude Code, Codex, Cursor) as a daily productivity multiplier - as an AI-native company, we see this as a must-have skill.\n\nNICE TO HAVE\nExperience implementing pipeline-level model runtime optimizations such as dynamic batching, async scheduling, or decode-side throughput improvements.\n\nExperience building developer platforms: SDKs, CLIs, APIs, and self-serve workflows for ML or infrastructure products.\n\nExperience with containerization and orchestration technologies (Docker, Kubernetes), service meshes, or distributed scheduling.\n\nFamiliarity with speech/audio ML models (STT, TTS, speech-to-speech)\n\nFamiliarity with model-serving runtimes (vLLM, TensorRT, ONNX).\n\nFamiliarity with systems-level performance profiling across host-device boundaries (e.g. PyTorch Profiler), diagnosing GPU utilization issues\n\nExposure to customer-facing engineering: pre-sales prototyping, technical discovery, or working directly with customers to ship solutions.\n\nBENEFITS\nCompetitive compensation, including meaningful equity\n\n(U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents\n\nFlexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)\n\nPaid parental leave\n\nFertility and family-building stipend through Carrot\n\n(U.S. only) Company-facilitated 401(k)\n\nExposure to a variety of ML startups, offering unparalleled learning and networking opportunities.\n\nApply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you.\nAt Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status.\nWe are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).","description_format":"text","description_chars":5736,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Equity","Parental leave","Vision insurance"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"},{"name":"Canada","iso":"CA","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","MLOps","AI Compute & Inference"],"lifecycle":[{"event":"open","at":"2026-04-23T23:48:25Z"}],"visa":[],"liveness":{"score":9,"band":"cold","label":"Long shot","p_open":1,"p_active":0.246,"p_room":0.36,"age_days":178,"expected_fill_days":92,"reasons":["conf:3","stale_co","velocity","win:tail","crowd:brand"],"computed_at":"2026-10-19T05:45:15Z"},"pay":{"stated_usd_annual":330000,"is_top_pay":true},"html_url":"https://alion.io/job/baseten-software-engineer-voice-ai-inference-runtime","json_url":"https://alion.io/job/baseten-software-engineer-voice-ai-inference-runtime.json","meta":{"generated_at":"2026-10-19T17:44:09Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":13519,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-20T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4399},"rest":"https://alion.io/mcp/rest/get_company?id=4399"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fbaseten-software-engineer-voice-ai-inference-runtime"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fbaseten-software-engineer-voice-ai-inference-runtime"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fbaseten-software-engineer-voice-ai-inference-runtime"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/baseten-software-engineer-voice-ai-inference-runtime\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fbaseten-software-engineer-voice-ai-inference-runtime"}]}