{"id":2341569,"url":"https://alion.io/job/goaly-founding-senior-ai-infrastructure-engineer","title":"Founding Senior AI Infrastructure Engineer","company":{"id":4416231,"name":"Goaly","domain":"goaly.ai","url":"https://alion.io/company/goaly-2","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":null},"role":"DevOps","role_family":"DevOps","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Palo Alto, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":143000,"max_usd":273000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":1716},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"C++","optional":false},{"name":"CUDA","optional":false},{"name":"DeepSpeed","optional":false},{"name":"FSDP","optional":false},{"name":"HPC","optional":false},{"name":"InfiniBand","optional":false},{"name":"JAX","optional":false},{"name":"Kubernetes","optional":false},{"name":"Linux","optional":false},{"name":"Machine Learning","optional":false},{"name":"Megatron-LM","optional":false},{"name":"Mixture of Experts","optional":false},{"name":"NCCL","optional":false},{"name":"NVLink","optional":false},{"name":"Post-training","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"Ray","optional":false},{"name":"Reinforcement Learning","optional":false},{"name":"Rust","optional":false},{"name":"SGLang","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"Triton","optional":false},{"name":"vLLM","optional":false},{"name":"TensorRT","optional":true}],"status":"live","first_seen_at":"2026-08-28T04:01:41Z","employer_posted_date":"2026-08-28","last_verified_at":"2026-10-21T13:08:11Z","board_verified":true,"closed_at":null,"days_open":54,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":54},"description":"About us\nWe’re building toward a world where every company can become its own AI lab.\nGoaly is a stealth AI startup founded by ex-Meta MSL engineers and researchers. Our mission is to dramatically lower the cost, time, and talent barriers to building proprietary AI - and make each generation of models faster and cheaper to build than the last.\nBacked by leading AI investors and endorsed by frontier AI researchers and builders, we’re looking for exceptional new grads who want to work on hard, foundational AI systems problems with outsized ownership from day one.\nAbout the role\nRunning modern AI workloads at scale creates systems problems that rarely fit within a single layer of the stack. A slowdown that appears in a training job may originate in a GPU kernel, collective communication, data movement, container runtime, storage path, scheduler, or interaction between model architecture and hardware topology.\nYou will identify these bottlenecks and build systems that improve the throughput, efficiency, and robustness of our largest distributed workloads. Your scope will span post-training, agentic reinforcement learning, model training, rollout inference, and the GPU cluster platform beneath them. You will work closely with researchers and systems engineers, develop a quantitative understanding of performance, and turn one-off investigations into durable infrastructure improvements.\nThis role is a strong fit for an exceptional systems or performance engineer who wants to work on frontier AI infrastructure. Deep ML experience is valuable but not required; we care most about a track record of solving difficult systems problems at scale and the ability to become fluent in new parts of the ML stack quickly.\nWhat you'll do\nProfile end-to-end AI workloads and identify limiting resources across model code, GPU kernels, memory, collective communication, networking, storage, orchestration, and environment execution.\n\nBuild low-latency, high-throughput sampling and inference systems for large language models, including batching, scheduling, caching, load balancing, and efficient weight updates.\n\nOptimize GPU execution through kernel and graph profiling, memory-layout improvements, reduced-precision computation, communication overlap, compilation, and targeted CUDA or Triton work.\n\nImprove distributed training and reinforcement-learning performance across heterogeneous GPU and CPU workloads, variable-length rollouts, complex network topologies, and changing model architectures.\n\nDesign quantitative performance and capacity models that predict bottlenecks, explain scaling behavior, guide hardware and topology choices, and prioritize engineering work.\n\nBuild scheduling and load-balancing mechanisms that improve accelerator utilization while respecting memory, locality, topology, latency, and fault-domain constraints.\n\nDesign fault-tolerant distributed systems that detect failures early, isolate their impact, recover efficiently, and preserve correctness during long-running jobs.\n\nInvestigate difficult production issues such as kernel-level stalls, network-latency spikes, collective timeouts, memory fragmentation, stragglers, and performance regressions in containerized environments.\n\nDevelop benchmarks, profiling tools, performance dashboards, and regression tests that make system behavior visible and allow improvements to be measured under realistic workloads.\n\nPartner with researchers to understand new models and algorithms, remove infrastructure constraints from the experimental loop, and translate successful optimizations into reusable platform capabilities.\n\nWhat success looks like\nWithin your first three months, you have built a quantitative understanding of at least one critical workload, identified its dominant bottlenecks, and shipped a measurable performance or reliability improvement.\n\nWithin six to twelve months, you have delivered sustained gains in throughput, accelerator utilization, latency, scaling efficiency, or cost across real training, rollout, or inference workloads.\n\nPerformance investigations become faster and more rigorous because the team has better benchmarks, models, profiles, and observability-not just undocumented fixes.\n\nLarge distributed jobs run predictably across complex hardware and network topologies, and failures or regressions can be detected, explained, and recovered from with minimal researcher intervention.\n\nYou may be a good fit if you have\nSignificant software-engineering, distributed-systems, high-performance-computing, or ML-infrastructure experience, particularly with performance-critical systems operating at large scale.\n\nExceptional programming and debugging ability in Python and at least one systems language such as C++, Rust, or Go.\n\nStrong systems fundamentals, including operating systems, concurrency, memory, networking, storage, scheduling, containerization, and failure handling.\n\nA track record of using measurement to solve ambiguous performance problems: forming hypotheses, designing representative benchmarks, reading profiles and traces, identifying root causes, and validating improvements under production conditions.\n\nThe ability to reason across abstraction boundaries, from model architecture and framework execution to accelerator behavior, distributed runtimes, cluster topology, and infrastructure services.\n\nA results-oriented mindset, flexibility about where in the stack to work, and a willingness to take ownership beyond a narrowly defined job description.\n\nStrong communication and collaboration skills, including the ability to work directly with researchers, explain complex systems behavior clearly, and turn repeated investigations into maintainable tools and abstractions.\n\nInterest in developing deep expertise in machine learning systems, even if your prior work has been primarily in distributed systems, HPC, operating systems, compilers, databases, or networking.\n\nStrong pluses\nExperience building or operating high-performance, large-scale ML training, post-training, or inference systems.\n\nExperience with GPU or accelerator programming, including CUDA, Triton, custom kernels, low-precision computation, memory optimization, or compiler stacks.\n\nFamiliarity with ML framework internals and distributed stacks such as PyTorch, JAX, torch.distributed, FSDP, Megatron, DeepSpeed, Ray, vLLM, SGLang, or TensorRT-LLM.\n\nKnowledge of NCCL or RCCL, RDMA, InfiniBand, RoCE, NVLink, collective algorithms, network topology, or topology-aware workload placement.\n\nExperience with Linux or operating-system internals, container runtimes, Kubernetes, cluster schedulers, storage systems, or production observability.\n\nFamiliarity with transformer architectures, language-model training, reinforcement learning, online sampling, or the performance characteristics of mixture-of-experts models.\n\nMeaningful contributions to open-source systems, ML frameworks, compilers, kernels, networking software, or performance tooling.\n\nHow we work\nMission first. We choose work for its impact on the mission and take responsibility for the outcome, not just our assigned tasks.\n\nHigh agency. We identify what is missing, form a plan, and move without waiting for perfect clarity.\n\nSpeed with rigor. We ship, measure, and iterate quickly while protecting correctness, safety, and reliability.\n\nFlexible scope. We cross team and technical boundaries when that is the fastest way to solve the real problem.\n\nLow ego, high standards. We give direct feedback, change our minds when the evidence changes, and help the whole team win.\n\nContinuous learning. The stack changes quickly; we are willing to learn unfamiliar systems, methods, and domains as the work demands.\n\nA note on qualifications. We care more about exceptional evidence than a perfect keyword match. If the work excites you and you can show unusual strength, learning speed, or ownership, we encourage you to apply even if your background does not match every preferred qualification.\nEqual opportunity\nWe are an equal opportunity employer. We consider qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, disability, veteran status, genetic information, or any other characteristic protected by applicable law. We provide reasonable accommodations for candidates who need them during the hiring process.","description_format":"text","description_chars":8388,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Continuous learning"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-12T11:26:53Z"}],"visa":[],"liveness":{"score":20,"band":"cold","label":"Long shot","p_open":1,"p_active":0.559,"p_room":0.35,"age_days":54,"expected_fill_days":27,"reasons":["conf:2","win:tail"],"computed_at":"2026-10-21T05:46:15Z"},"pay":null,"html_url":"https://alion.io/job/goaly-founding-senior-ai-infrastructure-engineer","json_url":"https://alion.io/job/goaly-founding-senior-ai-infrastructure-engineer.json","meta":{"generated_at":"2026-10-21T14:40:10Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":20922,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-22T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4416231},"rest":"https://alion.io/mcp/rest/get_company?id=4416231"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fgoaly-founding-senior-ai-infrastructure-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fgoaly-founding-senior-ai-infrastructure-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fgoaly-founding-senior-ai-infrastructure-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/goaly-founding-senior-ai-infrastructure-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fgoaly-founding-senior-ai-infrastructure-engineer"}]}