{"id":962061,"url":"https://alion.io/job/telnyx-inference-infrastructure-architect-remote","title":"Inference Infrastructure Architect (Remote)","company":{"id":13786,"name":"Telnyx","domain":"telnyx.com","url":"https://alion.io/company/telnyx","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"B","score":84,"open_postings":14,"ghost_share":0,"stale_share":0.429,"repost_share":0,"time_to_fill_p50_days":97,"computed_at":"2026-11-02T06:13:15Z"}},"role":"DevOps","role_family":"DevOps","seniority":"staff","employment_type":null,"work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"posting_text","remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["China"],"countries":["CN"],"hiring_countries":["CN"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":39000,"max_usd":93000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":926},"experience_years_min":null,"visa_sponsorship":true,"relocation_package":false,"has_equity":false,"technologies":[{"name":"CUDA","optional":false},{"name":"DeepSeek","optional":false},{"name":"Envoy","optional":false},{"name":"Gateway API","optional":false},{"name":"GitOps","optional":false},{"name":"Grafana","optional":false},{"name":"Kubernetes","optional":false},{"name":"KV Cache","optional":false},{"name":"Linux","optional":false},{"name":"LLM","optional":false},{"name":"LoRA","optional":false},{"name":"Mixture of Experts","optional":false},{"name":"OpenAI","optional":false},{"name":"OpenStack","optional":false},{"name":"OpenTelemetry","optional":false},{"name":"Prometheus","optional":false},{"name":"Python","optional":false},{"name":"Quantization","optional":false},{"name":"Ray","optional":false},{"name":"Ray Serve","optional":false},{"name":"Safetensors","optional":false},{"name":"SGLang","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"vLLM","optional":false},{"name":"Fine-tuning","optional":true},{"name":"LLM Evaluation","optional":true},{"name":"PEFT","optional":true}],"status":"live","first_seen_at":"2026-09-16T11:02:28Z","employer_posted_date":"2026-09-21","last_verified_at":"2026-11-02T20:22:50Z","board_verified":true,"closed_at":null,"days_open":47,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":47},"description":"About Telnyx\nTelnyx is an industry leader that's not just imagining the future of global connectivity-we're building it. From architecting and amplifying the reach of a private, global, multi-cloud IP network, to bringing hyperlocal edgetechnology right to your fingertips through intuitive APIs, we're shaping a new era of seamless interconnection between people, devices, and applications.\nWe're driven by a desire to transform and modernize what's antiquated, automate the manual, and solve real-world problems through innovative connectivity solutions. As a testament to our success, we're proud to stand as a financially stable and profitable company. Our robust profitability allows us not only to invest in pioneering technologies but also to foster an environment of continuous learning and growth for our team.\nOur collective vision is a world where borderless connectivity fuels limitless innovation. By joining us, you can be part of laying the foundations for this interconnected future. We're currently seeking passionate individuals who are excited about the opportunity to contribute to an industry-shaping company while growing their own skills and careers.\nInference Infrastructure Architect\nSenior / Staff · Remote - mainland China · Founding China team\nTelnyx runs its own B300 fleet - our hardware, in our facilities, operated from the metal up, and expanding globally. This role exists to turn that fleet into useful throughput: Telnyx's own AI-agent traffic on the voice and messaging network, and external customers' inference behind the token gateway, dedicated deployments and tuned models.\nYou have two mandates:\nOperate and expand the fleet efficiently. Maximize useful inference throughput per GPU-dollar while meeting latency and reliability SLOs, and continuously reduce cost per token.\nMake the product run on it. Serverless inference for the open-weight catalog, and dedicated, tuned model deployments for enterprises.\nThe stack is open source, bare metal to OpenAI-compatible endpoint. You work upstream in it.\nWhat you'll build\n Serverless serving pools. vLLM / SGLang engines with continuous batching, prefix caching, low-precision serving (FP8, FP4, INT4) and MoE expert parallelism, serving the head of the open-weight catalog at multi-tenant scale.\n The fleet layer. llm-d / NVIDIA Dynamo on the Kubernetes Gateway API: KV-cache-aware routing, prefill/decode disaggregation, KV tiering (Mooncake), and multi-LoRA serving so hundreds of customer adapters share one base pool.\nThe platform under it. Kubernetes on bare metal: GPU Operator, topology-aware scheduling, LeaderWorkerSet, Kueue; Kata Containers for isolated dedicated tenants; bare-metal lifecycle with OpenStack Ironic.\n Weight logistics and elasticity. P2P model distribution (Dragonfly, safetensors streaming), warm pools, and autoscaling driven by inference metrics - queue depth, KV occupancy, TTFT - never CPU.\nThe dedicated tier. Per-tenant pools, GPU-hour metering, latency SLOs, private networking; and the serving side of the model-tuning loop: adapter versioning, canary rollout, rollback on outcome regression.\nObservability and economics. DCGM and engine metrics into Prometheus / OpenTelemetry; capacity planning grounded in roofline math - bandwidth-bound decode, batching curves, utilization versus cost per token.\nThe stack you'll work in\nSome of this is committed direction: Kubernetes on bare metal, vLLM / SGLang, an OpenAI-compatible endpoint. Much of the rest is candidates you'll evaluate. You'll select, benchmark and integrate the components that earn their place in production. We value depth in the core serving stack and sound architectural judgment, not prior experience with every project listed.\nEngines: vLLM · SGLang\n Serving techniques: prefill/decode disaggregation · wide expert parallelism · speculative decoding (MTP, EAGLE-3) · low precision (FP8, FP4, INT4)\nMoE & attention libraries: DeepEP · DeepGEMM · EPLB · FlashMLA\nRouting & serving: llm-d · NVIDIA Dynamo · Kubernetes Gateway API · Envoy · Ray Serve / KubeRay\nKV cache & weights: Mooncake · HiCache · 3FS · Dragonfly\nScheduling & GPU sharing: Kubernetes · Volcano · Kueue · LeaderWorkerSet · HAMi\nIsolation & bare metal: Kata Containers · OpenStack Ironic · NVIDIA GPU Operator\nObservability: Prometheus · Grafana · DCGM · OpenTelemetry\nWhat we look for\nYou have owned production LLM serving under meaningful traffic and latency constraints, and can explain the architecture, the bottlenecks you diagnosed, the changes you made, and the resulting improvements in latency, reliability or cost. Scale on the order of thousands of GPUs or millions of requests a day is a strong signal, not a hard requirement.\nKubernetes on GPU fleets, operated end to end: GPU Operator, device plugins, node pools and topology-aware placement, gang scheduling (LeaderWorkerSet, Kueue), GitOps rollouts, and the ability to debug why a pod landed where it did.\nDeep operational command of vLLM or SGLang: deploying, tuning and upgrading it (parallelism with TP / EP, quantization, batch and KV-cache settings, prefix caching, disaggregation), and knowing which knob moves which metric.\nPerformance engineering at the system level: you read engine and DCGM metrics, reason from roofline, and size deployments with numbers.\nPython and Go for automation; real Linux, networking and storage depth. You don't need to write CUDA; you need to know when a problem is one, and route it upstream.\nYou work with the community in English and Chinese, and write runbooks and design docs people actually read.\nExperience we especially value\nOperated inference platforms at the scale of Ant Group, Alibaba Cloud / PAI, Tencent, ByteDance, Baidu, Huawei, DaoCloud, Moonshot, DeepSeek, or comparable teams.\nContributor to, or heavy production user of, any of: vLLM, SGLang, llm-d, NVIDIA Dynamo, Ray / KubeRay, Kata Containers, Volcano, Kueue, HAMi, Dragonfly, Mooncake, Envoy.\nPresence in the CNCF / OpenInfra community: talks at KubeCon China, OpenInfra Days, vLLM or SGLang meetups.\nNice to have\nFine-tuning / RL infrastructure: LoRA pipelines, evaluation harnesses, champion/challenger rollout.\nReal-time voice latency work: sub-second time-to-first-token budgets on live calls.\nMulti-region deployments and data-residency requirements.\nWhat we offer\nThe newest hardware, ours from the metal up. A globally expanding B300 fleet that Telnyx builds and operates end to end. Bring your experience optimizing demanding production workloads to a fleet where you shape the architecture from bare metal to customer endpoint.\nOwn the platform, build the team. A greenfield inference platform where you set the pattern others follow. As a founding member of our China team, you also get a say in who joins next and how the team works.\nOpen-source first. Upstream contribution is part of the job, and conference travel is supported.\nBased in mainland China, no relocation required. You work remotely with a global, async-friendly team. Should you ever choose to move, we have hiring entities in the Netherlands, the United States, Ireland and Saudi Arabia, and we sponsor visas.\nWhen you apply\nInclude a brief description of an inference system you personally improved: the bottleneck, your intervention, and the measured result. An anonymized example is welcome.","description_format":"text","description_chars":7314,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Continuous learning"],"hiring_locations":[{"name":"China","iso":"CN","kind":"country"}],"hiring_excludes":[],"relocation_offered":true,"industries":["Unified Communications","Conversational AI","Telecom Infrastructure","AI Compute & Inference"],"lifecycle":[{"event":"open","at":"2026-09-16T11:02:41Z"}],"visa":[],"liveness":{"score":53,"band":"ok","label":"Likely open","p_open":1,"p_active":0.735,"p_room":0.72,"age_days":46,"expected_fill_days":97,"reasons":["conf:3","velocity","win:mid","crowd:"],"computed_at":"2026-11-02T06:13:15Z"},"pay":null,"html_url":"https://alion.io/job/telnyx-inference-infrastructure-architect-remote","json_url":"https://alion.io/job/telnyx-inference-infrastructure-architect-remote.json","meta":{"generated_at":"2026-11-02T22:06:59Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":31144,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-03T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":13786},"rest":"https://alion.io/mcp/rest/get_company?id=13786"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ftelnyx-inference-infrastructure-architect-remote"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ftelnyx-inference-infrastructure-architect-remote"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ftelnyx-inference-infrastructure-architect-remote"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/telnyx-inference-infrastructure-architect-remote\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ftelnyx-inference-infrastructure-architect-remote"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":962061}},{"tool":"search_jobs","arguments":{"company":"telnyx.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":962061},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}