{"id":116838,"url":"https://alion.io/job/parasail-senior-inference-reliability-engineer","title":"Senior Inference Reliability Engineer","company":{"id":7031,"name":"Parasail","domain":"parasail.com","url":"https://alion.io/company/parasail","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Mateo, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":161000,"max_usd":306000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":1213},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Anomaly Detection","optional":false},{"name":"C++","optional":false},{"name":"CI/CD","optional":false},{"name":"Java","optional":false},{"name":"Kubernetes","optional":false},{"name":"Linux","optional":false},{"name":"LLM","optional":false},{"name":"Platform Engineering","optional":false},{"name":"Python","optional":false},{"name":"Quantization","optional":false},{"name":"Rust","optional":false},{"name":"SGLang","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"SRE","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"Triton","optional":false},{"name":"vLLM","optional":false},{"name":"TensorRT","optional":true}],"status":"live","first_seen_at":"2026-07-29T18:13:06Z","employer_posted_date":"2026-09-25","last_verified_at":"2026-10-18T00:30:35Z","board_verified":true,"closed_at":null,"days_open":80,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":80},"description":"Parasail is redefining AI infrastructure by enabling seamless deployment across a distributed network of GPUs, optimizing for cost, performance, and flexibility. Our mission is to empower AI developers with a fast, cost-efficient, and scalable cloud experience-free from vendor lock-in and designed for the next generation of AI workloads.\nThe Senior/Staff Inference Reliability Engineer will own the end-to-end reliability and production performance of customer inference workloads. This role sits at the intersection of inference platform engineering, LLM performance, and infrastructure reliability.\nYou will ensure that customer endpoints meet expectations for availability, latency, throughput, quality, and cost. When an endpoint degrades, you will follow the problem across the entire serving path-from APIs, routing, scheduling, and autoscaling through model servers, GPUs, networking, and underlying infrastructure-and drive it through resolution.\nThis is not a traditional DevOps role focused only on clusters and deployments. It is a production systems role for an engineer who enjoys investigating ambiguous performance problems, building diagnostic tooling, and turning recurring incidents into durable platform improvements.\nPrior LLM-inference experience is valuable but not required. We are looking for someone with deep production systems experience who can quickly learn inference-specific technologies and metrics.\nWhat You Will Own\nEnd-to-End Inference Reliability\nOwn the production health of customer inference workloads, including availability, request success, time to first token, inter-token latency, throughput, and operational efficiency.\nEstablish clear service-level indicators, objectives, performance baselines, and escalation paths for production endpoints.\nDetection and Observability\nBuild the telemetry, dashboards, alerts, and automated diagnostics needed to detect meaningful endpoint degradation before customers report it.\nCreate visibility across the full inference-serving path, including request queues, routing, scheduling, model servers, GPU utilization, networking, storage, and provider infrastructure.\nProduction Investigation\nLead the investigation of complex latency, throughput, capacity, and reliability regressions.\nDetermine whether an issue originates in customer traffic patterns, platform services, inference-engine configuration, GPU hardware, networking, storage, or an external infrastructure provider.\nRemain accountable for the customer outcome while partnering with the appropriate engineering teams to implement the fix.\nIncident Response and Prevention\nHelp lead customer-impacting incidents and establish effective operational practices for acknowledgement, diagnosis, recovery, and communication.\nConvert significant incidents into automated tests, safeguards, runbooks, capacity controls, anomaly detection, and platform improvements.\nPerformance and Capacity\nPartner with the LLM Performance team to validate that engine-level optimizations deliver measurable improvements in production.\nAnalyze workload behavior, capacity requirements, utilization, tail latency, and cost efficiency across heterogeneous GPU providers and hardware.\nHelp ensure that customer performance requirements are met without consuming unnecessary infrastructure capacity.\nProduction Feedback Loop\nIdentify recurring patterns across incidents, workloads, and customer escalations.\nTranslate those findings into improvements to the inference platform, reliability architecture, deployment processes, observability, and product roadmap.\nTechnical Abilities\nProduction Systems\nDeep experience operating critical, customer-facing or business-critical production systems.\nAbility to reason about service health across multiple layers rather than treating infrastructure availability as the complete customer outcome.\nReliability Engineering\nExperience defining and operating service-level indicators and objectives, building actionable observability, leading incidents, performing failure analysis, and reducing mean time to detection and recovery.\nPerformance Diagnosis\nStrong understanding of latency, throughput, queueing, resource contention, capacity, workload distribution, and tail-performance behavior.\nDemonstrated ability to diagnose difficult production regressions and isolate bottlenecks across applications and infrastructure.\nDistributed Systems\nKnowledge of distributed-systems principles, including fault tolerance, scheduling, routing, load balancing, capacity management, consistency, and failure recovery.\nCloud and Infrastructure\nStrong experience with Kubernetes, Linux, networking, storage, cloud infrastructure, and containerized production environments.\nExperience operating across multiple cloud providers, regions, hardware configurations, or infrastructure suppliers is especially valuable.\nSoftware Engineering\nAbility to write production-quality software and build internal tooling, instrumentation, automation, and diagnostic systems.\nProficiency in languages such as Python, Go, Java, C++, or Rust.\nAI and Inference Systems\nExperience with GPUs, ML infrastructure, model serving, vLLM, SGLang, Triton, TensorRT-LLM, or similar technologies is valuable but not required.\nYou should be excited to develop expertise in concepts such as time to first token, inter-token latency, continuous batching, KV caching, speculative decoding, quantization, and tokens per GPU.\nQualifications\n5+ years of experience in production engineering, site reliability engineering, infrastructure engineering, distributed systems, ML infrastructure, database reliability, or performance engineering.\nDemonstrated ownership of a critical production service or workload.\nExperience diagnosing complex latency, throughput, capacity, or reliability problems across multiple system layers.\nStrong software-engineering ability beyond infrastructure configuration and CI/CD automation.\nHands-on experience building observability, automation, diagnostic tooling, or production safeguards.\nStrong communication and technical leadership skills, including the ability to coordinate incident resolution across engineering teams.\nExperience with Kubernetes, Linux, networking, and cloud-native infrastructure.\nWhile prior LLM-inference experience is not required, a demonstrated ability to learn unfamiliar systems and develop deep technical expertise is essential.\nWhat Success Looks Like\nWithin your first six months, you will help establish clear end-to-end ownership and observability for Parasail’s production inference workloads.\nSuccess means:\nMaterial endpoint regressions are increasingly detected before customers report them.\n\nEngineers can quickly determine which layer of the system is responsible for a production issue.\n\nCustomer-impacting incidents are acknowledged, diagnosed, and resolved faster.\n\nProduction endpoints consistently meet defined availability, latency, throughput, and efficiency objectives.\n\nRecurring failure modes are converted into automated detection, safeguards, and durable platform improvements.\n\nInfrastructure and inference capacity are used efficiently while preserving customer performance and model quality.\n\nWhat You Bring to the Table\nThis role is pivotal in building a new operational discipline for AI infrastructure.\nYou may come from GPU infrastructure, ML platforms, databases, streaming systems, search, low-latency services, distributed storage, or large-scale production engineering. You do not need to have previously held the title “Inference Reliability Engineer.”\nWhat matters is that you have owned important production systems, can investigate problems that span organizational and technical boundaries, and continue following an issue until the customer experience is restored.\nIf you are passionate about production systems, performance, reliability, and learning the cutting edge of AI infrastructure, we are excited to welcome you aboard.","description_format":"text","description_chars":7939,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Cloud Platforms (IaaS & PaaS)","AI Compute & Inference"],"lifecycle":[{"event":"open","at":"2026-07-29T18:13:06Z"}],"visa":[],"liveness":{"score":18,"band":"cold","label":"Long shot","p_open":1,"p_active":0.627,"p_room":0.28,"age_days":79,"expected_fill_days":39,"reasons":["conf:0","velocity","win:tail","crowd:"],"computed_at":"2026-10-17T05:46:00Z"},"pay":null,"html_url":"https://alion.io/job/parasail-senior-inference-reliability-engineer","json_url":"https://alion.io/job/parasail-senior-inference-reliability-engineer.json","meta":{"generated_at":"2026-10-18T03:14:37Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":5118,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-19T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":7031},"rest":"https://alion.io/mcp/rest/get_company?id=7031"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fparasail-senior-inference-reliability-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fparasail-senior-inference-reliability-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fparasail-senior-inference-reliability-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/parasail-senior-inference-reliability-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fparasail-senior-inference-reliability-engineer"}]}