{"id":1079714,"url":"https://alion.io/job/jpmorganchase-senior-lead-software-engineer-llm-ops-platform-reliability","title":"Senior Lead Software Engineer - LLM Ops Platform Reliability","company":{"id":257,"name":"JPMorganChase","domain":"jpmorganchase.com","url":"https://alion.io/company/jpmorganchase","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Oracle","truth_index":{"grade":"A","score":99,"open_postings":546,"ghost_share":0.013,"stale_share":0.136,"repost_share":0.114,"time_to_fill_p50_days":8,"computed_at":"2026-10-19T05:45:15Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Glasgow, United Kingdom"],"countries":["GB"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":128000,"max_usd":236000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":20},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"CI/CD","optional":false},{"name":"Edge AI","optional":false},{"name":"Incident Management","optional":false},{"name":"Kubernetes","optional":false},{"name":"Machine Learning","optional":false},{"name":"Multi-Agent Systems","optional":false},{"name":"Python","optional":false},{"name":"Quantization","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"SRE","optional":false},{"name":"Tool Use","optional":false},{"name":"Amazon SageMaker","optional":true},{"name":"CrewAI","optional":true},{"name":"Hallucination","optional":true},{"name":"KServe","optional":true},{"name":"LangChain","optional":true},{"name":"LangGraph","optional":true},{"name":"OpenTelemetry","optional":true},{"name":"RAG","optional":true},{"name":"Ray","optional":true},{"name":"Ray Serve","optional":true},{"name":"Triton","optional":true},{"name":"Triton Inference Server","optional":true},{"name":"vLLM","optional":true}],"status":"live","first_seen_at":"2026-07-27T10:07:41Z","employer_posted_date":"2026-07-27","last_verified_at":"2026-10-19T20:55:32Z","board_verified":true,"closed_at":null,"days_open":84,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":84},"description":"Help shape how AI systems run reliably in production at scale. In this role, you'll build and operate large language model serving infrastructure, bringing strong engineering fundamentals and site reliability practices to cutting-edge AI platforms. You'll work hands-on with cloud and Kubernetes-based deployments, deep observability, and cost-aware performance tuning. If you enjoy solving hard production problems and making platforms measurably better, you'll find meaningful impact and growth here.\nAs a Senior Lead Software Engineer at JPMorganChase within the AI and Machine Learning Platform team, you will build and scale AI infrastructure that modernizes traditional infrastructure management and site reliability engineering through applied AI. You will own the reliability, performance, and cost-efficiency of the large language model inference platform end to end. You will operate large language model serving stacks in production at scale, with deep instrumentation and strong operational rigor. You will partner across engineering to deliver secure software, improve stability, and lead incident response and continuous improvement.\nJob responsibilities\nDesign, develop, troubleshoot, and deliver secure, high-quality production software and services for AI infrastructure\nBuild backend services and APIs that enable reliable operation of AI infrastructure in production environments\nOperate and scale large language model serving infrastructure, including model hosting, request routing, continuous batching, and cache optimization\nDeploy, host, and lifecycle-manage open-source and proprietary large language models on cloud-based container orchestration platforms and on-premises GPU clusters using reproducible infrastructure as code and continuous delivery pipelines\nImplement observability across logs, metrics, and traces with dashboards and actionable alerting for large language model and GPU workloads\nTune GPU and accelerator capacity, autoscaling, and cost efficiency for large language model inference workloads using performance optimization techniques such as quantization, parallelism, and speculative decoding\nLead reliability engineering for large language model endpoints through capacity planning, load and soak testing, safe rollouts, failover, and incident response for outages and model-quality regressions\nParticipate in on-call rotations, lead incident triage and mitigation, and produce clear post-incident root-cause analyses and follow-up actions\nIdentify recurring operational issues and automate remediation to improve platform stability and developer experience\nBuild and maintain multi-agent systems with strong orchestration, including planning, coordination, tool-calling, state and memory management, and workflow control where appropriate\nDrives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team. \nRequired qualifications, capabilities, and skills\nHands-on experience with system design, application development, testing, and operational stability in production environments\nAdvanced proficiency in Python for building production-grade services and tooling\nProficiency with automation and continuous delivery methods\nHands-on experience with cloud infrastructure platforms and infrastructure-as-code tooling for delivery and lifecycle management\nStrong understanding of site reliability engineering practices, including incident management, root-cause analysis, runbooks, and reliability patterns\nPractical knowledge of observability and instrumentation across metrics, logs, and traces\nHands-on experience with Kubernetes and container-based orchestration platforms, including managed cloud variants\nExperience hosting and serving large language models on cloud-based infrastructure and local GPU environments\nKnowledge of large language model reliability and risk considerations, including latency and throughput trade-offs, model versioning, prompt and response logging, and safe rollout patterns\nHands-on experience using enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, test creation, troubleshooting, or documentation) with demonstrated ability to critically evaluate, validate, and refine AI-generated outputs for correctness, performance, and security\nUnderstanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; ability to guide peers on safe and effective usage within team practices\nPreferred qualifications, capabilities, and skills\nExperience operating large language model inference servers such as vLLM and llm-d (or directly equivalent model serving stacks) in production\nExperience developing generative AI applications, AI agents, vector search, and retrieval-augmented generation patterns\nExperience building AI agents using orchestration frameworks such as LangChain, LangGraph, CrewAI, or similar platforms\nExperience operating or integrating model serving platforms such as KServe, Ray Serve, or NVIDIA Triton Inference Server alongside other large language model serving stacks\nFamiliarity with Amazon SageMaker JumpStart, SageMaker Endpoints, and Amazon Bedrock for managed model hosting\nExperience with online large language model quality monitoring, including hallucination detection, toxicity filtering, and drift detection using open telemetry conventions\nContributions to open-source large language model serving or inference projects,(vLLM, llm-d, Ray, KServe, Triton)","description_format":"text","description_chars":5927,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Commercial & Retail Banks","Wealth Management & Financial Advisors","Asset Management & Funds","Investment Banking & M&A Advisory"],"lifecycle":[{"event":"open","at":"2026-09-20T16:22:29Z"},{"event":"close","at":"2026-09-21T08:08:39Z"},{"event":"reopen","at":"2026-09-21T17:09:00Z"},{"event":"close","at":"2026-09-21T23:18:07Z"},{"event":"reopen","at":"2026-09-22T03:16:21Z"},{"event":"close","at":"2026-09-22T19:41:14Z"},{"event":"reopen","at":"2026-09-23T01:41:21Z"},{"event":"close","at":"2026-09-23T18:18:30Z"},{"event":"reopen","at":"2026-09-24T01:47:00Z"},{"event":"close","at":"2026-09-24T16:28:32Z"},{"event":"reopen","at":"2026-09-25T00:37:46Z"},{"event":"close","at":"2026-09-25T22:48:56Z"},{"event":"reopen","at":"2026-09-26T05:39:06Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 3,429 · green card filings: 933","filings_12m":3429,"filings_prev_12m":3601,"green_card_filings_12m":933,"median_offered_wage_usd":160366,"route":null,"cap_exempt":false,"checked_at":"2026-10-15T11:24:14+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","US Department of Labor: PERM disclosure data (green cards)"],"filings_for_role_12m":2086}],"liveness":{"score":7,"band":"cold","label":"Long shot","p_open":1,"p_active":0.251,"p_room":0.28,"age_days":83,"expected_fill_days":8,"reasons":["conf:0","velocity","win:tail","crowd:brand"],"computed_at":"2026-10-19T05:45:15Z"},"pay":null,"html_url":"https://alion.io/job/jpmorganchase-senior-lead-software-engineer-llm-ops-platform-reliability","json_url":"https://alion.io/job/jpmorganchase-senior-lead-software-engineer-llm-ops-platform-reliability.json","meta":{"generated_at":"2026-10-19T22:21:43Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":1321,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-20T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":257},"rest":"https://alion.io/mcp/rest/get_company?id=257"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fjpmorganchase-senior-lead-software-engineer-llm-ops-platform-reliability"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fjpmorganchase-senior-lead-software-engineer-llm-ops-platform-reliability"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fjpmorganchase-senior-lead-software-engineer-llm-ops-platform-reliability"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/jpmorganchase-senior-lead-software-engineer-llm-ops-platform-reliability\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fjpmorganchase-senior-lead-software-engineer-llm-ops-platform-reliability"}]}