{"id":4059161,"url":"https://alion.io/job/firmus-technologies-senior-platform-reliability-engineer-fabric-and-interconnect-2","title":"Senior Platform Reliability Engineer, Fabric and Interconnect","company":{"id":63306,"name":"Firmus Technologies","domain":"firmus.ai","url":"https://alion.io/company/firmus-technologies","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":null},"role":"DevOps","role_family":"DevOps","seniority":"senior","employment_type":null,"work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["San Francisco, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"InfiniBand","optional":false},{"name":"NVLink","optional":false},{"name":"Python","optional":false},{"name":"Self-Healing","optional":false}],"status":"live","first_seen_at":"2026-09-10T03:10:30Z","employer_posted_date":"2026-11-11","last_verified_at":"2026-11-11T11:52:42Z","board_verified":true,"closed_at":null,"days_open":62,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":62},"description":"Firmus Technologies \nFirmus Technologies is a global leader pioneering the development and operation of efficient AI infrastructure across Asia Pacific. \nFounded in Australia in 2019, our mission is to create the most efficient AI infrastructure by combining cutting-edge technology with a steadfast commitment to sustainability. At Firmus, we are unique in our approach. We design, build, and operatea new class of digital infrastructure - the AI Factory. Through our model-to-grid technology approach, we have pushed the boundaries of multi-generational liquid cooling systems, energy management, AI software orchestration, and construction. For our customers, this approach allows us to make every watt count and deliver low-cost AI tokens globally. \nFirmus AI Cloud \nOur large-scale GPU cloud platform, Firmus AI Cloud, is purpose-built to deliver energy-efficient AI compute at scale to customers. \nIt empowers developers, enterprises, educational institutions, and government users to train and deploy AI models with unmatched efficiency and cost savings. With an ever-growing suite of services and applications, we are committed to delivering a cloud experience that is market-leading, proprietary, and built to scale. \nWhy you’ll love working here\nAs an NVIDIA Cloud and Engineering partner in Asia Pacific, you will gain skills, experience,and exposure across the AI industry and be part of shaping what this industry looks like for decades to come. \nWe are founder-led, not a big corporate. Decisions happen fast, our leaders are accessible, and there's minimum bureaucracy between you and the work. \nOwnership comes early. Whatever your role, you will have a direct line to outcomes, helping shape how the business grows as we scale nationally across a long-term, large-scale roadmap.\nWork alongside founders and experts in AI infrastructure, energy systems and next-generation compute. \nWhat we build here has impact beyond the business. Our AI Factories are designed to operate as assets to the energy grid to actively strengthen the communities and regions they operate in rather than drawing from them. \nConsidering applying? You don't need a perfect background to join our team. If you're driven and curious, there's a path for you. We back our people to grow into new domains and take on challenges beyond their previous experience.\nAI FactoryOS Operations\nAI FactoryOS is Firmus' proprietary operating system for the AI Factory. It governs GPU telemetry, cooling, power and grid interaction as one integrated layer, so that every Firmus site can be optimised and monitored as a single system.\nAI FactoryOS Operations runs that platform in production and owns the 24/7 reliability of AI FactoryOS, Firmus AI Cloud and the platforms built on them, together with the service levels the estate is measured against.\nThe remit is an engineering one. The function builds the guarded automation, remediation and operational tooling that turn manual response into a software-defined capability, and builds and operates the shared services the estate's own operation depends on. The function works closely with the engineering teams that build the platform, supplying the production evidence that shapes what they fix and what they build next.\nRole Summary\nFirmus runs large-scale, state-of-the-art AI infrastructure built on the latest generation of GPU rack-scale systems and operated as one estate to power the next generation of AI innovation. The Senior Platform Reliability Engineer, Fabric and Interconnect, owns the reliability of the fabrics this estate runs on: the GPUto-GPU interconnect domains, the high-performance network fabrics carrying training and inference traffic, and the DPU-based host networking that binds compute to the rest of the platform.\nThis is a hands-on senior role with deep technical expertise. Automation is a first-class part of the role: the team builds and maintains the guarded automation and remediation tooling that turn manual fabric response into a self-healing capability, and the role engages fabric vendors at engineering level, reproducing faults to their standard and holding them to their answers.\nKey Responsibilities\nResponsible for the reliable operation, automation and continuous improvement of the estate's GPU interconnect and network fabrics (for example NVLink and NVSwitch domains, InfiniBand and Spectrum-X).\nBuild and maintain the guarded automation and remediation tooling for fabric faults, contributing to the software-driven remediation of AI clusters, including fault isolation and fabric reconvergence.\nDiagnose and tune performance across the interconnect stack, from application collective communication down to link level, working with technologies including NVLink, InfiniBand, RoCE and congestion control tuning.\nOperate DPU-based host networking across the fleet, including offload path configuration and driver and firmware compatibility.\nExecute firmware upgrade waves, fabric expansions and capacity changes to the supported paths and scaling patterns defined by AI Infrastructure, owning the production window, the staged or canary path, verification against declared success criteria, and rollback execution.\nProvide the deepest technical expertise for fabric and interconnect faults, correlating a collective communication failure to a specific physical link and diagnosing the faults that require internals-level knowledge to root cause, and driving the permanent fix to closure through AI Infrastructure.\nLead vendor escalations at engineering level, reproducing faults to the vendor's standard and pushing back credibly when a diagnosis does not explain the observed behaviour.\nLead technical recovery during major fabric incidents, drive the changes that remove repeat causes, share the follow-the-sun on-call roster, and mentor the engineers who carry frontline diagnosis, documenting operational procedures, runbooks and performance results.\nSkills & Experience\nRequired Skills\nStrong skills in high-performance networking and systems engineering, with 8+ years of experience including substantial ownership of production network or interconnect infrastructure in a 24/7 environment.\nDeep operational experience with high-performance GPU interconnect fabrics (for example NVLink and NVSwitch), including domain topology and failure diagnosis.\nExtensive experience with high-performance networking fabrics (for example InfiniBand or RoCE-based Ethernet such as NVIDIA Spectrum-X), including routing internals and congestion control tuning.\nExperience with DPU or SmartNIC-based host networking, including offload paths and driver and firmware coordination.\nExperience planning and executing firmware upgrade waves and capacity expansions on production fabrics with defined rollback.\nStrong skills in infrastructure automation and infrastructure-as-code practices, with change delivered through peer review and progressive rollout.\nPractical experience with scripting or programming for operational automation and tooling, such as Python, Go or Bash.\nProven ability to act as a senior escalation point in production, including major incident response, on-call participation, vendor escalation at engineering level, and the production of runbooks that others can execute successfully.\nClear technical judgement and communication skills, with the ability to explain complex fabric failures to engineers and non-specialists.\nPreferred Experience\nExperience operating fabrics for large-scale distributed training or inference workloads.\nExperience with NVIDIA rack-scale or multi-node GPU systems and their interconnect topology.\nExperience in a multi-tenant service provider, cloud or colocation environment.\nKnowledge of data centre and hardware fundamentals, including cabling and optics, firmware management and hardware fault workflows.\nRelevant vendor certification.\nA Bachelor's degree in computer science, engineering or a related discipline, or an equivalent combination of relevant experience and training.\nLocation & Reporting \nReporting directly to the Head of Firmus AI FactoryOS\nLocation: San Francisco Bay Area, CA, US\nHybrid working model, so must be based in San Francisco Bay Area or be willing to relocate\nEmployment Basis \nFull-time \nDiversity \nAt Firmus, we are committed to building a diverse and inclusive workplace.We encourage applications from candidates of all backgrounds who are passionate about creating a more sustainable future through innovative engineering solutions. Join us in our mission to revolutionize the AI industry through sustainable practices and cutting-edge engineering. Apply now to be part of shaping the future of sustainable AI infrastructure.","description_format":"text","description_chars":8682,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Hybrid work"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":true,"industries":["Data Centers & Colocation","Cloud Platforms (IaaS & PaaS)","AI Compute & Inference"],"lifecycle":[{"event":"open","at":"2026-11-11T07:13:18Z"}],"visa":[],"liveness":{"score":48,"band":"ok","label":"Likely open","p_open":1,"p_active":0.797,"p_room":0.6,"age_days":62,"expected_fill_days":72,"reasons":["conf:1","win:late","crowd:"],"computed_at":"2026-11-11T13:48:06Z"},"pay":null,"html_url":"https://alion.io/job/firmus-technologies-senior-platform-reliability-engineer-fabric-and-interconnect-2","json_url":"https://alion.io/job/firmus-technologies-senior-platform-reliability-engineer-fabric-and-interconnect-2.json","meta":{"generated_at":"2026-11-11T13:48:06Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":25294,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-12T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":63306},"rest":"https://alion.io/mcp/rest/get_company?id=63306"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ffirmus-technologies-senior-platform-reliability-engineer-fabric-and-interconnect-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ffirmus-technologies-senior-platform-reliability-engineer-fabric-and-interconnect-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ffirmus-technologies-senior-platform-reliability-engineer-fabric-and-interconnect-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/firmus-technologies-senior-platform-reliability-engineer-fabric-and-interconnect-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ffirmus-technologies-senior-platform-reliability-engineer-fabric-and-interconnect-2"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4059161}},{"tool":"search_jobs","arguments":{"company":"firmus.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4059161},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}