{"id":1990108,"url":"https://alion.io/job/stackyak-senior-ai-inference-engineer","title":"Senior AI Inference Engineer","company":{"id":3905444,"name":"StackYak","domain":"stackyak.io","url":"https://alion.io/company/stackyak","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"inferred_payroll_markers","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":[],"countries":[],"hiring_countries":["US"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":113000,"max_usd":250000,"period":"year","method":"global_role_seniority_cell","sample_n":1986},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Linux","optional":false},{"name":"Mixture of Experts","optional":false},{"name":"Python","optional":false},{"name":"Quantization","optional":false},{"name":"SGLang","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"vLLM","optional":false},{"name":"TensorRT","optional":true}],"status":"live","first_seen_at":"2026-10-07T02:19:33Z","employer_posted_date":null,"last_verified_at":"2026-10-07T02:19:33Z","board_verified":false,"closed_at":null,"days_open":5,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":5},"description":"About StackYak\nStackYak is building the infrastructure layer for AI.\nWe are an early-stage, funded company building software that brings compute, GPU infrastructure, networking, and inference together into one product. The opportunity is large, the market is moving quickly, and we are building for real production workloads from the start.\nThis is not internal IT. This is not a slow-moving infrastructure team maintaining someone else's platform. The infrastructure is the product.\nWe are a small, senior team with very little bureaucracy. This is a founding role in its discipline. You will be the first person here whose primary responsibility is the systems layer beneath the product, and the shape it takes will largely be yours to decide.\nTreat this document as a starting point rather than a boundary. The people who do well here take ground early and are not asked to give it back.\nWe move quickly. We do not have months for someone to learn the fundamentals of their discipline. You should already be very good at what you do, be able to ramp into adjacent areas quickly, and be comfortable operating without perfect requirements or neatly defined boundaries.\nThe RoleWe need someone who can take a model, a set of GPUs, and a production requirement and determine how that model should actually run.\nHand you a frontier open-weight model — dense or mixture-of-experts, tens to hundreds of billions of parameters — along with the hardware actually available and the latency a customer actually needs, and you should be able to make the calls that follow: precision, quantization, tensor and pipeline and expert parallelism, memory strategy, batching, concurrency, topology, and serving runtime. Then defend them, measure them, and operate them.\nDoing that well is the baseline. What makes the role interesting is the question sitting underneath it.\nServing expertise is scarce, slow to acquire, and currently lives in the heads of a small number of people who have done it enough times to have the instinct. We do not accept that it has to stay that way. A great deal of what those people know is reasoning, and reasoning can be written down, tested, and eventually carried out by something other than a person at two in the morning. How far that can be pushed is genuinely open, and you would be one of the people finding out.\nThis is not a research position and it is not an architecture-only role. You will benchmark, deploy, debug, tune, automate, and operate real inference systems — and then carry them in production.\nWhat You Will OwnNot tasks. Outcomes, and the authority that comes with them.\n\nHow models run. Placement, parallelism, precision, and memory strategy across single-GPU, multi-GPU, and multi-node — and the reasoning behind each choice, in a form someone other than you can audit.\n\nThe serving runtimes. vLLM, SGLang, TensorRT-LLM, and whatever displaces them: which one we use where, and what we do when the one we picked turns out to be wrong.\n\nWhat we are willing to promise. Concurrency limits, latency and TTFT targets, throughput under load. You own the evidence behind a number before it becomes a commitment to a customer.\n\nBenchmarking as an institution. Methodology, harness, reproducibility — and the standing to say that a result does not mean what someone wants it to mean.\n\nModel lifecycle in production. Loading, startup, health, upgrades, capacity, and failure handling, all of it happening while customers are connected.\n\nMulti-tenant versus dedicated serving, and where the line between them actually falls.\n\nInference incidents, including the ones where the serving runtime turns out to be innocent and the cause is GPU, driver, host, or network.\n\nTurning all of the above into software, so that it stops being institutional memory with a single point of failure.\n\nWhat Success Looks LikeYou can be given a model, a hardware target, and a workload profile and quickly produce a deployment that is sensible, measurable, reproducible, and ready to operate.\nThen you help us make that expertise programmatic.\nYou will help us answer questions such as:\n\nWhat hardware can this model run on?\n\nWhat is the right precision or quantization strategy?\n\nHow should we split it across GPUs or nodes?\n\nWhat serving runtime should we use?\n\nHow much concurrency can we safely offer?\n\nHow should we trade latency against throughput for this particular workload?\n\nHow does StackYak make those decisions automatically instead of needing an expert every time?\n\nWhat We NeedThe bar is what you have already done, not what you could learn. You should have done most of this:\n\nRun large language models in production, under real load, with someone depending on them.\n\nServed a model across multiple GPUs, and across multiple nodes.\n\nSized a model against hardware and been right about whether it would fit.\n\nChosen a quantization and precision strategy with a real quality and cost consequence attached to the choice.\n\nTuned batching and concurrency past the point where the easy wins ran out.\n\nMeasured latency, TTFT, throughput, utilization, and cost — and defended the numbers to someone motivated to disbelieve them.\n\nWorked in NVIDIA and/or AMD inference environments.\n\nWritten Python that other people run in production: tooling and product, not scripts.\n\nDebugged Linux and systems problems below the container boundary.\n\nFound the root cause of an inference failure that was not in the serving runtime.\n\nYou Will Be Especially Strong If\nYou have run inference infrastructure at an AI company, inference provider, GPU cloud, hyperscaler, or serious internal AI platform.\n\nYou have worked across multiple GPU generations and vendors, and can say concretely how that changed your decisions.\n\nYou understand distributed inference and the networking implications of multi-node serving.\n\nYou have benchmarked and compared serving runtimes rather than treating one framework as the answer to every problem.\n\nYou can explain why a deployment is configured the way it is instead of repeating a vendor recipe.\n\nYou have automated deployment decisions, or built schedulers, placement systems, capacity planners, or similar infrastructure.\n\nYou build and test things outside your assigned roadmap because you want to know how they actually work.\n\nThis Is Probably Not For You If\nYour background is primarily model training or ML research, with little production serving behind it.\n\nYou have put a model behind an API but have never had to reason about GPU memory, parallelism, topology, and concurrency.\n\nYou treat vLLM defaults as an architecture.\n\nYou are interested in benchmarks but not in operating the systems they describe.\n\nYou want to optimize kernels all day and have little interest in reliability or productization.\n\nYou would rather stay the person who knows how to configure it than encode that knowledge into software that makes you unnecessary.\n\nYou prefer writing recommendations to implementing them.\n\nYou need narrowly defined ownership, or a long runway before taking responsibility.\n\nHow We Work\nSmall, senior team with direct access to the founders.\n\nStrong opinions, loosely held.\n\nEveryone is expected to participate in technical decisions.\n\nEveryone shares responsibility for production and on-call.\n\nWe value people who can move between design, implementation, debugging, and operations.\n\nWe care much more about what you have built and operated than degrees, certifications, or academic credentials.\n\nWe expect people to leave ego at the door, argue the technical case, make a decision, and then execute.\n\nWe are remote and distributed across time zones. Whether a role is an employment or a contract engagement depends on where you are, and we work that out at offer.\n\nHiring here is a few real conversations with the people you would actually work with, not a recruiter screen followed by a panel of strangers.\n\nWe are hiring across inference, infrastructure, and networking. The boundaries between the three are blurry on purpose. If you sit between two of them, say so.\n\nThis is an early-stage startup. The pace is high, the problems are hard, and the scope will change as we grow.\n\nCompensationCompetitive compensation plus meaningful equity. Exact structure will depend on location, engagement model, and experience.\nA Note For AgenciesWe are not using external recruiters or agencies for this role, and we will not be persuaded otherwise by an email. We do not want your spam. We will not read the CVs you send, we will not reply to your follow-up, and no candidate you put in front of us creates a fee obligation of any kind. Do not contact us.\nOriginally posted on Himalayas","description_format":"text","description_chars":8634,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["AI Agents"],"lifecycle":[{"event":"open","at":"2026-10-07T05:28:00Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":5,"expected_fill_days":29,"reasons":["seen:5","win:early"],"computed_at":"2026-10-12T06:55:09Z"},"pay":null,"html_url":"https://alion.io/job/stackyak-senior-ai-inference-engineer","json_url":"https://alion.io/job/stackyak-senior-ai-inference-engineer.json","meta":{"generated_at":"2026-10-12T20:07:35Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":32574,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-13T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3905444},"rest":"https://alion.io/mcp/rest/get_company?id=3905444"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fstackyak-senior-ai-inference-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fstackyak-senior-ai-inference-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fstackyak-senior-ai-inference-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/stackyak-senior-ai-inference-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fstackyak-senior-ai-inference-engineer"}]}