{"id":3041051,"url":"https://alion.io/job/people-in-ai-senior-principal-software-engineer-llm-inference-optimization","title":"Senior Principal Software Engineer – LLM Inference Optimization","company":{"id":3862080,"name":"People In AI","domain":"peopleinai.com","url":"https://alion.io/company/people-in-ai","size_band":null,"is_staffing_agency":true,"employer_type":"agency","is_intermediary":false,"listed_via":null,"ats_vendor":"Schema","truth_index":{"grade":"B","score":75,"open_postings":16,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-30T05:48:23Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"lead","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":230000,"max":280000,"currency":"USD","period":"year","gross":null,"usd_annual":280000},"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"C++","optional":false},{"name":"CUDA","optional":false},{"name":"KV Cache","optional":false},{"name":"LLM","optional":false},{"name":"Quantization","optional":false},{"name":"SGLang","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"vLLM","optional":false},{"name":"AWQ","optional":true},{"name":"Azure","optional":true},{"name":"GPTQ","optional":true},{"name":"llama.cpp","optional":true},{"name":"Machine Learning","optional":true},{"name":"Rust","optional":true},{"name":"TensorRT","optional":true}],"status":"live","first_seen_at":"2026-10-21T22:43:55Z","employer_posted_date":"2026-10-21","last_verified_at":"2026-10-29T22:25:24Z","board_verified":true,"closed_at":null,"days_open":8,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":8},"description":"Senior Principal Software Engineer\n(LLM Inference, CUDA & GPU Performance Optimization)\nCompensation: $230,000-$280,000 base + annual bonus opportunity + potential equity\nLocation: Remote - United States\nThe Company\nAn established global AI technology company building intelligent, real-time conversational experiences for the automotive industry. Its technology is deployed at significant commercial scale through partnerships with major vehicle manufacturers worldwide, with continued investment in advanced generative AI and high-performance inference infrastructure.\nThe Opportunity\nWe are looking for a Senior Principal Software Engineer who understands what it takes to make large language models run faster, more efficiently, and reliably in production.\nThis is a hands-on engineering position focused on pushing the boundaries of LLM inference performance through low-level GPU optimization, inference runtime development, and advanced memory management. Your work will directly influence product responsiveness, infrastructure economics, and the performance of real-world AI systems operating at scale.\nThe Role\nYou will take ownership of identifying and removing performance bottlenecks in production LLM inference systems, with a particular focus on cloud-hosted workloads and real-time response requirements.\nWorking alongside an established AI infrastructure team, you will optimize inference engines, develop CUDA kernels, and improve GPU efficiency to deliver measurable reductions in latency and improvements in throughput.\nThis is not a conventional MLOps, infrastructure maintenance, or LLM application development role. We are looking for someone who can get deep into inference engine internals and make the underlying systems materially faster.\nWhat You'll Do\nOptimize LLM inference performance across production systems, improving latency, tokens/sec, throughput, and GPU utilization.\n\nDevelop and tune CUDA kernels, including attention implementations and other performance-critical inference operations.\n\nExtend and optimize inference engines such as vLLM and other high-performance runtimes, identifying opportunities for low-level code improvements.\n\nImprove GPU memory efficiency through KV cache optimization, memory layout design, paging, and cache-aware execution strategies.\n\nOptimize prefill and decode performance, particularly for workloads involving substantial contextual input and short, latency-sensitive responses.\n\nEvaluate advanced optimization techniques, including quantization, continuous batching, speculative decoding, and distributed inference strategies.\n\nBenchmark and deploy production improvements, delivering measurable gains in inference performance, stability, and cost efficiency.\n\nWhat You'll Bring\nSignificant hands-on experience with production ML or LLM inference systems, including direct responsibility for performance optimization.\n\nStrong C++ programming experience, ideally five or more years, with the ability to understand, modify, and optimize complex low-level code.\n\nPractical expertise in CUDA development, GPU profiling, and kernel-level optimization.\n\nDeep understanding of GPU architecture, memory bandwidth, memory hierarchies, concurrency, and the trade-offs involved in high-performance inference.\n\nExperience with inference frameworks such as vLLM, TensorRT-LLM, SGLang, or comparable runtimes, ideally including modifications beyond standard configuration.\n\nA track record of diagnosing performance bottlenecks and producing measurable improvements in latency, throughput, or infrastructure efficiency.\n\nAn independent engineering mindset, strong technical judgment, and the ability to operate effectively in a fast-moving, globally distributed team.\n\nWhat This Role Requires\nReal production inference experience: You must have worked directly on deploying or optimizing inference workloads beyond proof-of-concept or research environments.\n\nLow-level engineering capability: Strong C++ and hands-on CUDA experience, with the ability to work directly with performance-critical code.\n\nGPU performance expertise: A practical understanding of memory bandwidth, compute utilization, caching, and inference bottlenecks.\n\nDemonstrable technical ownership: The ability to explain what you personally built or optimized, why you made specific engineering decisions, and what measurable results you achieved.\n\nHands-on problem-solving: Comfort investigating complex systems, validating assumptions, and implementing improvements rather than relying exclusively on existing frameworks or third-party services.\n\nTech Stack\nLanguages: C++, CUDA; Rust exposure is a plus.\n\nInference Engines: vLLM, TensorRT-LLM, llama.cpp, QAIRT; SGLang and other emerging inference runtimes.\n\nGPU Performance: Custom CUDA kernels, GPU profiling, attention optimization, memory bandwidth tuning.\n\nQuantization: INT8, INT4, FP4, FP8, mixed precision, AWQ, GPTQ.\n\nInference Optimization: KV cache optimization, prefix caching, continuous batching, speculative decoding, prefill/decode optimization.\n\nInfrastructure: Cloud-hosted LLM inference, Azure and other cloud environments, distributed inference architectures.\n\nWhy Join\nSolve technically challenging problems with direct impact. Work deep inside LLM inference systems, optimizing the actual execution of models rather than building wrappers around them.\n\nSee your engineering work translate into measurable results. Improvements in latency, throughput, and GPU efficiency directly affect real-time AI product performance and commercial economics.\n\nCombine established scale with meaningful technical ownership. Join a commercially proven AI business investing in advanced inference capabilities, with a remote US position and the opportunity to shape critical technology.\n\nAbout People in AI\nPeople in AI is a specialist recruitment partner connecting exceptional AI, machine learning, data, and engineering talent with some of the most ambitious technology companies in the world. We work closely with founders, technical leaders, and hiring teams to represent opportunities accurately and help candidates assess genuine fit.","description_format":"text","description_chars":6129,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-21T22:43:55Z"}],"visa":[],"liveness":{"score":37,"band":"fade","label":"Fading","p_open":1,"p_active":0.368,"p_room":1,"age_days":8,"expected_fill_days":28,"reasons":["conf:7","agency","stale_co","velocity","win:early"],"computed_at":"2026-10-30T05:48:23Z"},"pay":{"stated_usd_annual":280000,"is_top_pay":true},"html_url":"https://alion.io/job/people-in-ai-senior-principal-software-engineer-llm-inference-optimization","json_url":"https://alion.io/job/people-in-ai-senior-principal-software-engineer-llm-inference-optimization.json","meta":{"generated_at":"2026-10-30T06:42:24Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":9393,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-31T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3862080},"rest":"https://alion.io/mcp/rest/get_company?id=3862080"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fpeople-in-ai-senior-principal-software-engineer-llm-inference-optimization"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fpeople-in-ai-senior-principal-software-engineer-llm-inference-optimization"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fpeople-in-ai-senior-principal-software-engineer-llm-inference-optimization"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/people-in-ai-senior-principal-software-engineer-llm-inference-optimization\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fpeople-in-ai-senior-principal-software-engineer-llm-inference-optimization"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3041051}},{"tool":"search_jobs","arguments":{"company":"peopleinai.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3041051},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}