{"id":107261,"url":"https://alion.io/job/arcadia-staff-applied-ai-engineer-product-agent-performance","title":"Staff Applied AI Engineer, Product & Agent Performance","company":{"id":3655,"name":"Arcadia","domain":"arcadia.com","url":"https://alion.io/company/arcadia","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Lever","truth_index":{"grade":"A","score":90,"open_postings":15,"ghost_share":0,"stale_share":0.2,"repost_share":0,"time_to_fill_p50_days":70,"computed_at":"2026-10-25T05:49:26Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"staff","employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["United States"],"countries":["US"],"hiring_countries":["US"],"hiring_countries_total":1,"salary":{"min":175000,"max":200000,"currency":"USD","period":"year","gross":null,"usd_annual":200000},"salary_estimate":null,"experience_years_min":4,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"AI Agents","optional":false},{"name":"Amazon SageMaker","optional":false},{"name":"AWS","optional":false},{"name":"AWS Bedrock","optional":false},{"name":"Human-in-the-Loop","optional":false},{"name":"LLM","optional":false},{"name":"Multi-Agent Systems","optional":false},{"name":"RAG","optional":false}],"status":"live","first_seen_at":"2026-08-20T17:12:39Z","employer_posted_date":"2026-08-20","last_verified_at":"2026-10-25T08:19:42Z","board_verified":true,"closed_at":null,"days_open":65,"trust":{"level":"ok","repost_count":0,"flags":[],"days_open":65},"description":"Arcadia is the most trusted healthcare platform powering outcomes. We transform complex healthcare data into trusted intelligence, helping providers, payers, and life sciences organizations act with clarity, make confident decisions, and achieve measurable clinical, operational, and financial outcomes.\nBuilt on a comprehensive data foundation spanning tens of millions of patient lives, Arcadia combines advanced analytics and responsible AI to surface meaningful insights, coordinate action, and improve performance at scale. Our approach to AI and automation is governed and transparent - designed to strengthen human expertise, not replace judgment or obscure responsibility.\nHundreds of organizations rely on Arcadia to improve cost, quality, and outcomes. Backed by Nordic Capital, we continue to invest in our platform, AI capabilities, and people as we pursue our purpose: helping healthcare deliver better outcomes for every person, every community, and every generation.\nWhy This Role Is Important to Arcadia\nArcadia’s data and analytics platform is used by hundreds of health systems, ACOs, payers, and life sciences organizations, touching tens of millions of patient lives. This role owns how our agentic capabilities perform at that same scale: accurate, transparent about their own confidence, and safe for the clinicians, care teams, and patients who depend on them.\nAs a staff-level individual contributor, you will own the product-layer decisions that shape agent behavior, including prompting, retrieval and context, memory and state, evaluation, and escalation, while partnering with Product and Engineering on the systems that support them. Your work will help Arcadia make evidence-based launch decisions and scale responsible AI that is steerable, trustworthy, and ready for real healthcare workflows.\nWhat Success Looks Like\nIn 3 months\nYou have establisheda production-grounded baseline for priority agentic workflows, with documented failure modes, severity-weighted evaluationrubrics, and a clear measurement plan\nYou have mapped the current retrieval, context, memory, and escalation patterns and identifiedthe highest-value opportunities to improve reliability, calibration, and cost\nYou have earned trust across Product and Engineering by turning production evidence into clear, actionable recommendations\nIn 6 months\nProduction-representative evaluation suites and regression checks inform model-change decisions for priority agentic workflows\nYou have delivered measurable improvements in accuracy, reliability, steerability, latency, or cost for one or more priority workflows\nHuman-review and escalation behavior hasbeen validatedunder adversarial and edge-case conditions, with decision criteria and ownership boundaries clearly documented\nIn 12 months\nArcadia has a repeatable product-layer AI performance practice that moves from production failure to diagnosis, experiment, evaluation, and release decision\nHigh-severity regressions are caught earlier, and agentbehavior is more transparent, calibrated, and trustworthy at scale\nModel cards, intended-use guidance, limitations, and performance documentation are current and useful to product and customer-facing teams\nWhat You'll Be Doing\nDesign and iterate onagent behavior across real, live workflows, including long-horizon, multi-turn agentic tasks\nDesign retrieval and context architecture so the right source data reaches a model in the right structureand agents remaingrounded in real data rather than filling gaps with assumptions\nDesign memory and state handling across multi-turn and multi-agent flows, determiningwhat is carried forward, summarized, or dropped and why\nCreate context and prompt templates that combine few-shot examples, structured formatting, and reasoning scaffolding for consistent agent behavior\nImprove performance through prompting, tool-use strategy, and context construction, validatedthrough direct experimentation rather than guesswork\nBuild and run evaluations against real production conditions to measure performance, regressions, failure modes, and edge cases\nAuthor evaluation rubrics, quality heuristics, and thresholds that weight failures by severity and cost, not just frequency, and monitorthose measures against production behavior\nDesign and validateescalation paths that route agents to human review based on confidence and uncertainty while preserving safety and consistency under adversarial and edge-case conditions\nDesign for cost-aware performance alongside latency, reliability, and accuracy through efficient context construction and tool-call economy\nEvaluate and sign off on model changes by baselining current behavior, running comparative evaluations, and making the go/no-go call before a change reaches a customer\nMaintain product-level AI documentation, including model cards, intended use, limitations, and known failure modes, so customer-facing teams work from actual agent behavior\nPartner closely with Product and product managersto ensure agents are not just capable, but steerable, trustworthy, and ready to scale\nWhat You'll Bring\nBachelors degree in Engineering or related field and/or masters of Data Science or related field\n4+ years of production software engineering experience, including 3+ years of hands-on ownership of ML, LLM, or agentic systems in production, with direct experience in healthcare, finance, or another regulated industry\nDemonstrated ability to diagnose why an agent failed, correctly attribute the fix to instruction, retrieval, context, or memory design, and weigh failures by severity and cost rather than frequency alone\nHands-on experience with RAG architecture, production-grounded evaluation frameworks, and fallback or human-in-the-loop logic for automated systems\nWorking familiarity with AWS AI/ML services, including Bedrock and SageMaker, sufficient to build and evaluate effectively in Arcadia’s environment\nEvidence-led judgment and the credibility to push back on launch decisions, paired with a builder’s instinct to run the experiment and move from a production failure to a fix\nWould Love for You to Have\nExperience applying AI to healthcare data or workflows where safety, transparency, and calibrated uncertainty directly affect care teams or patients\nExperience with long-horizon, multi-turn or multi-agent workflows and product-level AI documentation such as model cards\nWhat You'll Get\nThe opportunity to define how agent performance, safety, and readiness are measured for productionhealthcare workflows\nMeaningful ownership across prompts, context, memory, evaluations, and escalation patterns at product scale\nA cross-functional role translating production evidence into AI improvements used across Arcadia’s platform\nA mission-driven company working to improve how patients receive care\nA flexible, remote-friendly culture with personality and heart\nEmployee-driven programs and initiatives for personal and professional development\nMembership in the talented, energized, diverse, and purpose-driven Arcadian community","description_format":"text","description_chars":7033,"description_truncated":false,"requirements":{"experience_years_min":4,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Professional development"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Solar Energy","Energy Management"],"lifecycle":[{"event":"open","at":"2026-08-20T17:12:39Z"}],"visa":[],"liveness":{"score":54,"band":"ok","label":"Likely open","p_open":1,"p_active":0.903,"p_room":0.6,"age_days":65,"expected_fill_days":70,"reasons":["conf:1","velocity","win:late","crowd:"],"computed_at":"2026-10-25T05:49:26Z"},"pay":{"stated_usd_annual":200000,"is_top_pay":false},"html_url":"https://alion.io/job/arcadia-staff-applied-ai-engineer-product-agent-performance","json_url":"https://alion.io/job/arcadia-staff-applied-ai-engineer-product-agent-performance.json","meta":{"generated_at":"2026-10-25T08:57:34Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":12821,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-26T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3655},"rest":"https://alion.io/mcp/rest/get_company?id=3655"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Farcadia-staff-applied-ai-engineer-product-agent-performance"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Farcadia-staff-applied-ai-engineer-product-agent-performance"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Farcadia-staff-applied-ai-engineer-product-agent-performance"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/arcadia-staff-applied-ai-engineer-product-agent-performance\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Farcadia-staff-applied-ai-engineer-product-agent-performance"}]}