{"id":1158330,"url":"https://alion.io/job/nvidia-senior-system-software-engineer-local-ai-2","title":"Senior System Software Engineer - Local AI","company":{"id":6,"name":"NVIDIA","domain":"nvidia.com","url":"https://alion.io/company/nvidia","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"B","score":82,"open_postings":592,"ghost_share":0.019,"stale_share":0.66,"repost_share":0.034,"time_to_fill_p50_days":33,"computed_at":"2026-10-17T05:46:00Z"}},"role":"DevOps","role_family":"DevOps","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Pune, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":25000,"max_usd":53000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":43},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"C++","optional":false},{"name":"CUDA","optional":false},{"name":"CUDA Toolkit","optional":false},{"name":"DirectX","optional":false},{"name":"Edge AI","optional":false},{"name":"Ignite","optional":false},{"name":"Knowledge Distillation","optional":false},{"name":"KV Cache","optional":false},{"name":"llama.cpp","optional":false},{"name":"LLM","optional":false},{"name":"LocalAI","optional":false},{"name":"Machine Learning","optional":false},{"name":"MLIR","optional":false},{"name":"Model Distillation","optional":false},{"name":"PyTorch","optional":false},{"name":"PyTorch C++","optional":false},{"name":"Quantization","optional":false},{"name":"Speech Recognition","optional":false},{"name":"TensorRT","optional":false},{"name":"Text-to-Speech","optional":false},{"name":"vLLM","optional":false},{"name":"Windows","optional":false}],"status":"live","first_seen_at":"2026-07-25T00:00:00Z","employer_posted_date":"2026-09-28","last_verified_at":"2026-10-17T21:30:23Z","board_verified":true,"closed_at":null,"days_open":84,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":84},"description":"NVIDIA has continuously reinvented itself for more than two decades. The invention of the GPU in 1999 fueled the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing. GPU-powered deep learning helped ignite the era of modern AI, establishing GPUs as the foundation of intelligent applications across productivity, gaming, and creative workflows, and reinforcing NVIDIA's position as a leading AI computing company.\nMore recently, there is a growing focus on running AI models locally, closer to where data is generated. This approach reduces latency, enables real-time processing, and addresses privacy concerns by minimizing the need to send data to centralized servers. As technology continues to evolve, client-side AI will play an increasingly important role in shaping the digital landscape. The LocalAI team is seeking a Senior Systems Software Engineer to lead the architecture of efficient on-device AI software for RTX and DGX-class systems. The role focuses on delivering high-performance local inference with low latency, optimized memory utilization, robust infrastructure, and practical deployment on resource-constrained platforms\nWhat You'll Be Doing:\nPartner with NVIDIA's software, research, architecture, and product leadership, as well as external partners such as Microsoft, to align technical requirements and strategic priorities, fostering the AI ecosystem on RTX and DGX PCs. \nBuild and optimize the local AI inference stack for RTX, RTX Pro, and DGX GPUs, with a focus on performance, stability, and scalability across diverse hardware architectures. \nArchitect and develop modern inference runtimes and execution stacks using frameworks such as llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT-RTX, supporting LLM, vision-language, TTS, ASR, and diffusion-based AI workloads. \nPerform end-to-end optimization of AI models, data pipelines, and inference runtimes to maximize performance on current and next-generation GPU architectures. Apply model optimization techniques, including quantization, pruning, sparsity, and distillation, to enable efficient deployment of large models on local and edge devices. \nLead system-level debugging, performance tuning, and performance-accuracy trade-off analysis; develop infrastructure for performance and accuracy sweeps; analyse results to identify gaps and drive fixes; and establish engineering guidelines to accelerate bring-up and ensure production readiness of new models and inference backends. \nMentor other engineers and review architecture proposals spanning several groups, acting as a force multiplier for technical quality and consistency across the broader LocalAI organization. \nWhat we need to see:\n5+ Years of experience with Bachelor’s, Master’s, or PhD in Computer Science, Software Engineering, Mathematics, or related field, or equivalent experience. \nExcellent C++ programming and debugging skills, with a strong foundation in data structures, algorithms, and machine learning. \nProven experience architecting and optimizing AI inference pipelines and applications using ML/DL frameworks such as Llama.cpp, vLLM, PyTorch, Windows ML, DXCGC, and TensorRT. \nDeep understanding of inference backends and runtime internals, including scheduling, memory management, KV-cache behavior, graph execution, quantization, and hardware-aware optimization. \nDemonstrated ability to establish technical direction and influence outcomes across multiple teams and organizations without relying on formal management authority. \nStrong analytical and problem-solving skills, with the ability to manage multiple priorities effectively in a fast-paced environment. \nExcellent written and verbal communication skills, enabling effective collaboration across engineering teams and management. \nWays to stand out from the crowd:\nDeep understanding of modern machine learning, deep neural network, and generative AI techniques, with exposure to AOT graph-compilation systems (TensorRT builder, MLIR-based compilers) \nConsistent track record of delivering end-to-end products in multinational companies with geographically distributed teams. \nExpert-level low-level systems and GPU kernel programming, CUDA, and development of high-performance systems. \nSignificant contributions to open-source inference runtimes, model tooling, or performance infrastructure, as a maintainer or core contributor. \nHands-on experience building applications using frameworks and APIs such as llama.cpp, PyTorch, TensorRT, Vulkan, DirectX, and vLLM. \nWe're a top employer recognized for innovation, growth, and commitment to diversity as an equal opportunity workplace. We offer competitive salaries, a generous benefits package, and the opportunity to work alongside some of the industry's most talented and forward-thinking professionals. As our engineering teams continue to grow rapidly, we're looking for creative, self-driven engineers with a passion for technology to join us.","description_format":"text","description_chars":4986,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Servers & Data Center Hardware","Computer Components","AI Chips & Accelerators"],"lifecycle":[{"event":"open","at":"2026-09-23T22:28:52Z"}],"visa":[],"liveness":{"score":10,"band":"cold","label":"Long shot","p_open":1,"p_active":0.356,"p_room":0.28,"age_days":84,"expected_fill_days":33,"reasons":["conf:1","stale_co","velocity","win:tail","crowd:brand"],"computed_at":"2026-10-17T05:46:00Z"},"pay":null,"html_url":"https://alion.io/job/nvidia-senior-system-software-engineer-local-ai-2","json_url":"https://alion.io/job/nvidia-senior-system-software-engineer-local-ai-2.json","meta":{"generated_at":"2026-10-17T22:04:33Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":7070,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":6},"rest":"https://alion.io/mcp/rest/get_company?id=6"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fnvidia-senior-system-software-engineer-local-ai-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fnvidia-senior-system-software-engineer-local-ai-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fnvidia-senior-system-software-engineer-local-ai-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/nvidia-senior-system-software-engineer-local-ai-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fnvidia-senior-system-software-engineer-local-ai-2"}]}