{"id":1020011,"url":"https://alion.io/job/furiosa-production-engineer-llm-serving","title":"Production Engineer, LLM Serving","company":{"id":31683,"name":"FuriosaAI","domain":"furiosa.ai","url":"https://alion.io/company/furiosa","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"A","score":100,"open_postings":38,"ghost_share":0,"stale_share":0,"repost_share":0.026,"time_to_fill_p50_days":33,"computed_at":"2026-11-12T10:38:37Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"middle","employment_type":null,"work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["Seoul, South Korea"],"countries":["KR"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":3,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"C++","optional":false},{"name":"Istio","optional":false},{"name":"Kubernetes","optional":false},{"name":"KV Cache","optional":false},{"name":"Linux","optional":false},{"name":"LLM","optional":false},{"name":"Python","optional":false},{"name":"Rust","optional":false},{"name":"Service Mesh","optional":false},{"name":"SLI/SLO/SLA","optional":false},{"name":"GitOps","optional":true},{"name":"Grafana","optional":true},{"name":"OpenTelemetry","optional":true},{"name":"Prometheus","optional":true},{"name":"SGLang","optional":true},{"name":"Speculative Decoding","optional":true},{"name":"TensorRT","optional":true},{"name":"TensorRT-LLM","optional":true},{"name":"vLLM","optional":true}],"status":"live","first_seen_at":"2026-09-18T02:05:10Z","employer_posted_date":"2026-10-21","last_verified_at":"2026-11-12T14:36:46Z","board_verified":true,"closed_at":null,"days_open":55,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":55},"description":"About FuriosaAI\nFuriosaAI builds high-performance, high-efficiency AI compute for the Inference Era. Founded in 2017 by veteran semiconductor and AI algorithm engineers, Furiosa operates globally with offices in Korea and Silicon Valley, along with a compiler-focused R&D lab in Lisbon. \nOur vision is to make AI computing sustainable, enabling access to powerful AI for everyone on Earth. We solve the AI hardware energy and operational cost crisis at the architectural level, rather than through brute force, building the world's first truly AI-native compute platform to unlock the full potential of artificial intelligence for every enterprise.\nProduction Engineer, LLM Serving\nLocation: Seoul, South Korea (Hybrid)\nAbout the Job\nOwns the reliability, observability, and service-level performance of production LLM serving services on FuriosaAI's RNGD NPUs. You will improve latency, throughput, and resource efficiency through workload-driven measurement, bottleneck analysis, and software engineering.\nOur stack uses llm-d and Furiosa-LLM, with Istio for traffic management in Kubernetes-based deployments and Mooncake for L3 KV-cache storage. You will own the serving service layer, collaborating with infrastructure, inference engine, runtime, and compiler teams on improvements that span their components.\nKey Responsibilities\nOperates and improves RNGD-based LLM serving services, defining SLIs/SLOs and production readiness criteria to guide reliability and capacity decisions.\nBuilds metrics, logs, traces, dashboards, and actionable alerts across the serving request path, tracking availability, errors, time to first token (TTFT), inter-token and end-to-end latency, throughput, and NPU utilization.\nDesigns benchmarks and load tests reflecting production input/output lengths, concurrency, and traffic patterns. Uses latency distributions, throughput, and utilization to establish baselines, detect regressions, and plan serving capacity.\nDiagnoses bottlenecks across routing, queueing, batching, caching, networking, and inference execution. Improves service components and configurations to increase throughput and resource efficiency while meeting latency SLOs.\nImproves deployment topologies, autoscaling, health checks, graceful degradation, and Istio traffic policies for serving workloads, partnering with the infrastructure team on shared Kubernetes and service mesh capabilities.\nAutomates serving deployments and validates changes through performance tests, canary rollouts, and rollback procedures, collaborating with the build and release team on release artifacts.\nParticipates in on-call and incident response, from mitigation to root-cause analysis and preventive improvements. Builds runbooks and recovery automation to reduce operational toil.\nMinimum Qualifications\nBachelor's degree in Computer Science or a related field, or equivalent practical experience, with 3+ years developing and operating production services.\nStrong programming skills in Rust, C++, Python, or Go, with the ability to read, instrument, and improve existing production code.\nHands-on experience operating services in Kubernetes and solid understanding of Linux, containers, networking, concurrency, and distributed systems.\nUnderstanding of LLM serving concepts, including prefill/decode, batching, KV-cache management, and request scheduling, and their impact on performance.\nExperience profiling production services and analyzing telemetry to identify bottlenecks across the request path and validate improvements in tail latency, throughput, and resource efficiency.\nExperience building observability systems and using metrics, logs, and traces to diagnose production incidents, restore services, and implement preventive improvements.\nAbility to communicate quantitative findings and collaborate across teams to deliver measurable production improvements.\nPreferred Qualifications\nExperience operating LLM serving systems using llm-d, vLLM, SGLang, TensorRT-LLM, or similar technologies.\nExperience optimizing request routing, load balancing, queueing, or autoscaling, and translating workload measurements into capacity plans or serving-cost estimates.\nExperience with inference optimizations such as continuous batching, prefix KV-cache reuse, speculative decoding, distributed inference, or kernel optimization.\nExperience with Istio traffic management, infrastructure as code, or GitOps, including automated rollout and rollback workflows.\nExperience with Prometheus, Grafana, OpenTelemetry, and reliability practices such as SLO design, error budgets, and post-incident reviews.\nExperience operating GPU/NPU workloads in on-premises or cloud environments and diagnosing issues across drivers, runtime, orchestration, and applications.\nExperience developing software in Rust.\nWhy Join FuriosaAI\nThe defining bottleneck of the AI era is building the right hardware and software stack to run it at global scale. Furiosa is solving this challenge holistically from the ground up.\nWith our flagship chip, RNGD, in mass production today and our next-generation platform in development with Broadcom, we are proving that full-stack, tensor-native compute is the future of AI infrastructure. This is a pivotal moment to join our team, right as we accelerate our global expansion.\nAt Furiosa, you will:\nSolve AI’s Most Urgent Challenge. Help build the high-performance, energy-efficient inference hardware and software required to fulfill the promise of advanced AI.\nPioneer Full-Stack Co-Design. Work with teams that are architecting solutions from silicon up through the compiler (featuring innovations like Tensor Contraction Language and Virtual ISA) and serving frameworks.\nShip Real-World Silicon, Software, and Solutions. Turn breakthrough technology into commercial deployment. RNGD is in mass production with TSMC and running live enterprise workloads for global leaders like LG AI Research and Samsung SDS.\nPartner With the Industry's Best. Collaborate across an elite global ecosystem that includes TSMC, Broadcom, SK Hynix, and GUC.\nDo Your Life’s Best Work. Join a brilliant, low-ego, mission-driven team in a high-trust environment that values autonomy, intellectual curiosity, and shared ambition. \nContact\n ","description_format":"text","description_chars":6249,"description_truncated":false,"requirements":{"experience_years_min":3,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"South Korea","iso":"KR","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Processors, MCUs & AI Chips","AI Chips & Accelerators"],"lifecycle":[{"event":"open","at":"2026-09-18T04:06:37Z"}],"visa":[],"liveness":{"score":29,"band":"fade","label":"Fading","p_open":1,"p_active":0.653,"p_room":0.45,"age_days":55,"expected_fill_days":33,"reasons":["conf:1","velocity","win:tail"],"computed_at":"2026-11-12T10:38:37Z"},"pay":null,"html_url":"https://alion.io/job/furiosa-production-engineer-llm-serving","json_url":"https://alion.io/job/furiosa-production-engineer-llm-serving.json","meta":{"generated_at":"2026-11-12T20:16:52Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":42829,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-13T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":31683},"rest":"https://alion.io/mcp/rest/get_company?id=31683"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ffuriosa-production-engineer-llm-serving"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ffuriosa-production-engineer-llm-serving"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ffuriosa-production-engineer-llm-serving"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/furiosa-production-engineer-llm-serving\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ffuriosa-production-engineer-llm-serving"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":1020011}},{"tool":"search_jobs","arguments":{"company":"furiosa.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":1020011},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}