{"id":2034938,"url":"https://alion.io/job/microsoft-principal-ai-software-engineer","title":"Principal AI Software Engineer","company":{"id":18,"name":"Microsoft","domain":"microsoft.com","url":"https://alion.io/company/microsoft","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Eightfold","truth_index":{"grade":"B","score":75,"open_postings":86,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-12T06:55:09Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Washington, United States","Mountain View, United States","Hillsboro, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":142800,"max":274800,"currency":"USD","period":"year","gross":null,"usd_annual":274800},"salary_estimate":null,"experience_years_min":12,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Azure","optional":false},{"name":"C#","optional":false},{"name":"C++","optional":false},{"name":"JavaScript","optional":false},{"name":"KV Cache","optional":false},{"name":"LLM","optional":false},{"name":"OneDrive","optional":false},{"name":"Python","optional":false},{"name":"CUDA","optional":true},{"name":"CUDA Toolkit","optional":true},{"name":"Linux","optional":true},{"name":"NCCL","optional":true},{"name":"SGLang","optional":true},{"name":"TensorRT","optional":true},{"name":"TensorRT-LLM","optional":true},{"name":"vLLM","optional":true}],"status":"live","first_seen_at":"2026-09-16T20:49:53Z","employer_posted_date":"2026-10-07","last_verified_at":"2026-10-12T05:27:13Z","board_verified":true,"closed_at":null,"days_open":25,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":25},"description":"OverviewDo you want to be at the forefront of innovating the latest hardware and systems designs to propel Microsoft’s cloud growth? Are you seeking a unique career opportunity that combines technical capabilities, cross team collaboration, with business insight and strategy?\nThe SPARC organization is responsible for strategy, planning and architecture pathfinding, and manages Azure’s hardware roadmap from architecture concept, through production for Microsoft’s current and future offerings. The CSA team within SPARC is at the forefront of systems architecture and technology pathfinding spanning compute, memory, storage, and system interconnects. Drawing on deep insights of workloads, emerging technology trends, and focused industry engagements, CSA team’s charter is to define and evaluate novel systems architecture innovations through hardware/software co-design and advance them through technical readiness for productization.\nJoin our Compute System Architecture (CSA) team within the System Planning and Architecture (SPARC) organization in Azure Hardware Systems & Infrastructure (AHSI). AHSI is the team behind Microsoft’s expanding cloud business, responsible for delivering the hardware systems and infrastructure for cloud computing across Microsoft Azure, Bing, MSN, Office 365, OneDrive, Skype, Teams and Xbox Live.\nThe CSA team is seeking a Principal AI Software Engineer !\nResponsibilities\nLead full system software prototyping to develop capable proof-of-concepts to evaluate hardware/software co-designed capabilities for memory TCO reduction such as through memory tiering/pooling and overcommit solutions for Azure usages and deployment scenarios.\nDevelop deep insights through workload characterization and correlation to identify systems optimization opportunities.\nCollaborate with diverse workload experts across Microsoft and partner ISVs to engineer TCO-optimized solutions for Azure general-purpose and specialized compute fleet.\nInfluence and shapehardware architecture and industry alignment, targeting three-to-six-year timeframe, with data-driven analysis, insights and recommendations.\nLead characterization and optimization of Large Language Model (LLM) inference workloads with focus on KV Cache capacity, placement, migration, and utilization across GPU HBM, host DRAM, CXL memory expansion/pooling, SSD, and emerging memory tiers.\nDevelop proof-of-concepts and evaluation frameworks to assess memory-tiering architectures for AI inference, including CXL pooled memory, memory expansion solutions, context-memory platforms, SSD-backed cache tiers, and hardware/software co-designed approaches for reducing inference TCO.\nDesign and execute workload characterization studies for agentic, multi-turn, coding, reasoning, and long-context AI workloads to quantify memory consumption, latency, throughput, token efficiency, and system utilization.\nAnalyze end-to-end data movement across GPU, CPU, storage, and networking subsystems, identifying optimization opportunities within GPU Direct Storage (GDS), GPU Direct RDMA (GDR), peer-to-peer memory transfers, and distributed inference pipelines.\nDevelop software prototypes, framework extensions, and instrumentation to evaluate KV Cache offload, prefetching, migration, compression, deduplication, and memory-overcommit techniques.\nBuild performance models and simulation frameworks to predict the impact of memory hierarchy innovations on large-scale inference deployments.\nQualifications\nRequired/minimum qualifications\nBachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.\nOther Qualifications:\nAbility to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.\nPreferred Qualifications:\n12+years of experiencein systems software (OS kernel, memory management, I/O stacks, Virtualization) with demonstrated track record ofsuccess, guiding architecture and software enabling.\n10+ years of experience leading significant hardware/software co-design projects involving CPU and/or systems architecture and influencing technical direction.\nDeep expertise in Linux kernel internals, memory management, I/O subsystems, NUMA, DMA, and GPU/CPU/storage/network data paths.\nHands-on experience with NVIDIA GPU software stacks including CUDA, NCCL, GPUDirect Storage (GDS), and GPUDirect RDMA (GDR).\nUnderstanding of AI inference infrastructure, large GPU clusters, inference serving architectures, and workload performance optimization.\nExperience characterizing and optimizing KV Cache intensive workloads including long-context, agentic, multi-turn, coding, and reasoning workloads.\nExperience with inference frameworks such as vLLM, SGLang, and TensorRT-LLM.\nFamiliarity with KV Cache technologies including LMCache, SGLang HiCache, cache offload, cache sharing, and memory tiering approaches.\nExperience designing or extending inference runtimes, scheduling systems, memory management components, or KV Cache subsystems.\nUnderstanding of disaggregated prefill/decode architectures, distributed inference serving, and multi-node cache-sharing topologies.\nExperience with CXL memory expansion, memory pooling, and memory tiering solutions in large-scale deployments.\nSoftware development skills in C/C++, Python, CUDA, and distributed systems.\nSkilled in partnering and influencing architects, hardware engineers, and software leads\nAbility to manage through ambiguity, bringing clarity and results orientation to engage and energize collaborators and stakeholders\nCollaboration skills, teamwork, and sense of presumed responsibility\nVerbal and written communication skills, and ability to articulate and engage with both technical and non-technical stakeholders at all levels.\nExperience leading and driving complex projectswith respect and integrity, including thosewith multiple workstreams spanning different business and technical disciplines.\nIntellectual curiosity and passion about learning and deploying new technologies.\nProblem-solvingskills, analytical capabilities, and attention to detail\nSoftware Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.\nCertain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:\nhttps://careers.microsoft.com/us/en/us-corporate-pay\nThis position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.\nMicrosoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.","description_format":"text","description_chars":7880,"description_truncated":false,"requirements":{"experience_years_min":12,"management_years_min":10,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Console Games","Cloud Platforms (IaaS & PaaS)","Gaming Consoles","Developer Tools"],"lifecycle":[{"event":"open","at":"2026-10-07T15:32:12Z"}],"visa":[],"liveness":{"score":28,"band":"fade","label":"Fading","p_open":1,"p_active":0.514,"p_room":0.55,"age_days":25,"expected_fill_days":19,"reasons":["conf:1","stale_co","velocity","win:tail","comp:brand"],"computed_at":"2026-10-12T06:55:09Z"},"pay":{"stated_usd_annual":274800,"is_top_pay":true},"html_url":"https://alion.io/job/microsoft-principal-ai-software-engineer","json_url":"https://alion.io/job/microsoft-principal-ai-software-engineer.json","meta":{"generated_at":"2026-10-12T11:41:06Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":18131,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-13T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":18},"rest":"https://alion.io/mcp/rest/get_company?id=18"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fmicrosoft-principal-ai-software-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fmicrosoft-principal-ai-software-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fmicrosoft-principal-ai-software-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/microsoft-principal-ai-software-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fmicrosoft-principal-ai-software-engineer"}]}