{"id":3194724,"url":"https://alion.io/job/tensorx-senior-inference-engineer","title":"Senior Inference Engineer","company":{"id":2153149,"name":"TensorX","domain":"tensorx.ai","url":"https://alion.io/company/tensorx","size_band":"11-50","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"HireHive","truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Dublin, Ireland"],"countries":["IE"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":88000,"max_usd":189000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":11},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"API Gateway","optional":false},{"name":"Claude Code","optional":false},{"name":"Grafana","optional":false},{"name":"Kubernetes","optional":false},{"name":"KV Cache","optional":false},{"name":"LLM","optional":false},{"name":"Loki","optional":false},{"name":"OpenAI","optional":false},{"name":"OpenAI Codex","optional":false},{"name":"Prometheus","optional":false},{"name":"Python","optional":false},{"name":"SGLang","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"vLLM","optional":false},{"name":"Gateway API","optional":true},{"name":"InfiniBand","optional":true},{"name":"TensorRT","optional":true}],"status":"live","first_seen_at":"2026-05-09T19:32:25Z","employer_posted_date":"2026-05-09","last_verified_at":"2026-10-27T14:56:05Z","board_verified":true,"closed_at":null,"days_open":170,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":170},"description":"Overview\nTensorX is a sovereign AI infrastructure platform headquartered in Dublin. We run frontier open-weight large language models on our own NVIDIA Blackwell GPUs in European datacentres, under EU jurisdiction. Customers reach them through a drop-in OpenAI-compatible API. Nothing they send is retained after the request completes. We serve regulated enterprises in finance, healthcare and government as well as developers and AI platforms. We help them adopt AI without compromising on data privacy, compliance or performance.\nWe are looking for a Senior Inference Engineer to join our growing engineering team. Reporting to the CTO, you will own the serving layer between our API gateway and the inference engines across more than one site. This covers admission, routing, the prefill/decode split, KV cache movement across the GPU fabric and autoscaling.\nOur customers send very long contexts at high concurrency, often in bursts. The number that decides our margin is not how fast one engine runs. It is how many requests the whole fleet answers inside a latency target when a burst arrives. We run NVIDIA Dynamo in production for disaggregated serving on Kubernetes and are moving the rest of the fleet onto it. We are an AI-native team. Tools such as Claude Code and Codex are part of our daily workflow and materially accelerate how we build and operate systems.\nYou will work side by side with our GPU Performance Team, who own the inside of the engines: kernels, engine patches and per-model tuning. They make each worker faster. You make the fleet of workers behave. Our platform team builds and maintains the Kubernetes clusters and hosts. Our backend team owns the API gateway. You will work with all three every day.\nThis is a high-impact senior individual contributor role spanning disaggregated serving, routing, autoscaling and multi-site operations.\nResponsibilities\nDisaggregated serving - Run and extend our NVIDIA Dynamo deployment. Own the split between prefill and decode workers. Move capacity between them as the traffic shape changes.\n\nAdmission & routing - Own the router and KV-aware routing so requests land where their prefix is already cached. Build admission control so a burst queues instead of crashing a worker. The GPU Performance Team tunes cache behaviour inside the engine and contributes to the router code.\n\nAutoscaling & capacity - Scale prefill and decode against latency targets and scale down when traffic drops. Decide how many GPUs each model pool needs and where it runs. Base these decisions on traffic data and the parallelism layouts the GPU Performance Team validates.\n\nKV cache transfer - Own cross-node KV cache movement (NIXL, Mooncake or similar). Test it across nodes so the network is part of every measurement. Add custom telemetry where the stock tools do not see.\n\nMulti-site serving - Run several sites behind one front door, each with its own local serving plane. Keep a path open to non-NVIDIA pools through llm-d when we need one.\n\nObservability - Instrument the serving layer with per-model metrics that mean something: time to first token, inter-token latency, queue depth, cache hit rate and errors. Work with Prometheus, Grafana and Loki alongside GPU telemetry.\n\nRollouts & pre-production - Roll out serving-layer changes with zero downtime: drain, verify outside the pool, return, repeat. Share the pre-production gate with the GPU Performance Team so no change reaches a customer unmeasured.\n\nData retention - Make sure nothing a customer sends is retained after the request completes. Enforce this by design in the serving path, not only by policy.\n\nReliability & incidents - Own the serving-layer runbook. Lead the response when the fleet, rather than one engine, misbehaves. Write up each incident so it does not happen twice.\n\nSkills & Experience\n5+ years of professional experience in distributed systems, ML infrastructure or production serving. A meaningful portion of this should be on GPU workloads\n\nHands-on experience running large language models in production with vLLM, SGLang or TensorRT-LLM. You understand how prefill, decode, KV cache and batching behave under load\n\nExperience with NVIDIA Dynamo or a comparable disaggregated serving system (e.g. llm-d)\n\nKubernetes in production, including deployments, rollouts, operators or controllers and GPU scheduling\n\nStrong distributed systems fundamentals: queueing, backpressure, admission control, routing and retries under burst load\n\nExperience instrumenting production systems (e.g. Prometheus, Grafana, Loki) and using that data to guide tuning decisions\n\nFamiliarity with benchmarking methodology: one variable at a time, a pass or fail bar set before the run and results someone else can reproduce\n\nProficiency in Python and comfort reading systems code in other languages\n\nComfortable using AI-assisted development tools (e.g. Claude Code, Codex) as part of your daily workflow\n\nA clear and concise communicator who thrives in ambiguity and can articulate technical decisions to both technical and non-technical audiences\n\nNice to Have\nExperience with NIXL, Mooncake, LMCache or other KV cache transfer and offload layers\n\nExperience debugging performance across a GPU fabric (RDMA, RoCE or InfiniBand)\n\nGateway API Inference Extension or similar Kubernetes-native inference routing\n\nMulti-site or multi-cluster operations\n\nContributions to open-source inference or serving projects\n\nWhy This Role\nDisaggregated serving is live here, not on a roadmap. You will extend a Dynamo deployment that already carries production traffic.\n\nThe fleet is ours. Our own NVIDIA B300 GPUs in Dublin and Helsinki. You are not renting time on someone else's cluster.\n\nThe traffic is real. Long-context, bursty, high-concurrency workloads that break assumptions benchmarks never test.\n\nThe results are real. Our inference stack answers roughly twice as many requests inside the latency target as a standard configuration. We measured this on the same hardware with the same production traffic patterns.\n\nYou shape the architecture. Dynamo is live, but the multi-site serving layer around it is still being designed. Your decisions become the architecture.\n\nEducation & Qualifications\nBSc/MSc in Computer Science, Software Engineering, Electrical Engineering or a related technical discipline OR equivalent practical experience\n\nRemuneration\nHighly competitive package, dependent on experience\n\n25 days paid annual leave\n\nHybrid working from our centrally located Dublin office, with remote flexibility\n\nFree inference tokens!\n\n******* NO AGENCY ASSISTANCE REQUIRED *******","description_format":"text","description_chars":6600,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":true},"security_clearance":false,"languages":[]},"benefits":["Annual leave","Hybrid work"],"hiring_locations":[{"name":"Ireland","iso":"IE","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["AI Compute & Inference","AI Chips & Accelerators"],"lifecycle":[{"event":"open","at":"2026-10-24T03:54:25Z"}],"visa":[],"liveness":{"score":7,"band":"cold","label":"Long shot","p_open":1,"p_active":0.242,"p_room":0.28,"age_days":170,"expected_fill_days":30,"reasons":["conf:3","win:tail","crowd:"],"computed_at":"2026-10-27T05:46:45Z"},"pay":null,"html_url":"https://alion.io/job/tensorx-senior-inference-engineer","json_url":"https://alion.io/job/tensorx-senior-inference-engineer.json","meta":{"generated_at":"2026-10-27T17:11:23Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":6742,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-28T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":2153149},"rest":"https://alion.io/mcp/rest/get_company?id=2153149"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ftensorx-senior-inference-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ftensorx-senior-inference-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ftensorx-senior-inference-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/tensorx-senior-inference-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ftensorx-senior-inference-engineer"}]}