{"id":801807,"url":"https://alion.io/job/verahealth-ai-eval-engineer","title":"AI / Eval Engineer","company":{"id":688527,"name":"Vera Health","domain":"verahealth.ai","url":"https://alion.io/company/verahealth","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"D","score":46,"open_postings":10,"ghost_share":0.9,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-01T05:45:00Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Francisco, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":153000,"max_usd":335000,"period":"year","method":"role_country_seniority_unknown","sample_n":2986},"experience_years_min":null,"visa_sponsorship":true,"relocation_package":false,"has_equity":false,"technologies":[],"status":"live","first_seen_at":"2026-03-31T06:27:06Z","employer_posted_date":"2026-03-31","last_verified_at":"2026-10-02T02:39:27Z","board_verified":true,"closed_at":null,"days_open":184,"trust":{"level":"ghost","repost_count":0,"flags":["stale","company_stale"],"days_open":183},"description":"About Vera Health\nVera is the fastest-growing clinical app for healthcare professionals in the world. Five people built it.\nWe’re already operating at a scale most teams never reach, with a team most people would think is too small to pull it off. That’s by design. Everyone here runs at a level that compresses what would normally take entire teams. We keep it that way by only hiring people who raise the standard the moment they join.\nWe’re building the founding team. The people who join now will define how this company operates for the next decade, and shape the future of healthcare.\nWe move at a pace that feels unreasonable to most, and completely natural to the right people. If that pace feels natural to you, you belong here.\nThe Role\nYou can’t improve what you can’t measure.\nEvery answer on Vera is a clinical decision being made by a clinician. That’s the bar. Not “mostly correct,” not “good on average.” We need to know exactly how the Vera behaves, where it holds up, where it breaks, and why. Think of the system like a chessboard with millions of different combinations. Different models, prompts, retrieval strategies, ranking, parameters, all interacting at once. Your job is to understand that space and push it in the right direction by optimizing every part of it, down to the smallest parameter.\nThis isn’t about writing eval scripts and calling it done. It’s about going a step further, understanding how the system behaves across specialties, clinical contexts, and edge cases, and improving it so that answers hold up no matter the setting.\nYou’ll be working on a complex search and reasoning engine, where the goal is simple but hard: make it better for every question, every specialty, every clinical setting.\nWe’re building something that doesn’t exist yet.\nWhat You’ll Do\nOptimize every layer of the system, from model behavior to retrieval to ranking to the smallest parameters\n\nTake a system with thousands of moving parts and figure out which ones actually drive better answers\n\nPush performance forward by testing, breaking, and refining how outputs are generated\n\nGo deep on failures, not just what went wrong, but what pattern it reveals\n\nTune how the system behaves across different specialties, question types, and edge cases\n\nWork directly with the founders on the highest-leverage improvements\n\nWork with leading healthcare institutions to validate how the system performs in practice\n\nWhat We’re Looking For\nYou don’t trust outputs you can’t explain\n\nYou care about being right, not just being close\n\nYou like working in messy spaces where there’s no obvious path\n\nYou move fast, but you don’t compromise when the stakes are high\n\nYou have strong technical depth in LLMs, search, or similar systems\n\nYou’re excited to work at the edge of what’s currently being done\n\nHow we work\nShip, then improve. We get things in front of users fast and let real feedback drive iteration. A feature in production teaches you more than a month of planning.\nOwn it completely. There's no handoff culture here. You take a problem and see it through, from first principles to production.\nAlways look out for the user. Every decision comes back to the clinician on the other end. If it doesn't make their day better, it doesn't ship.\nWhy Vera\nRocketship traction: 100%+ MoM growth, built by a team of 5\n\nReal impact: Every feature you build helps clinicians make better decisions and save lives\n\nBest time to join: We’ve just raised our Series A and are entering the most important phase of the company, where a small team defines everything that comes next\n\nDetails\nLocation: San Francisco (strongly preferred). For exceptional candidates, remote or relocation support (O-1A / H-1B sponsorship) available.\n\nCompensation: Competitive salary + meaningful early-stage equity. Details shared in process.\n\nEmployment: Full-time","description_format":"text","description_chars":3848,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":true,"industries":["Health Care","Urgent & Emergency Care","Medical AI","Clinical Decision Support"],"lifecycle":[{"event":"open","at":"2026-09-12T08:11:40Z"}],"liveness":{"score":3,"band":"cold","label":"Long shot","p_open":1,"p_active":0.092,"p_room":0.28,"age_days":183,"expected_fill_days":30,"reasons":["conf:1","stale_co","ghost","win:tail","crowd:"],"computed_at":"2026-10-01T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/verahealth-ai-eval-engineer","json_url":"https://alion.io/job/verahealth-ai-eval-engineer.json","meta":{"generated_at":"2026-10-02T03:08:58Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","usage":{"tier":"crawler","counted_by":"address","units_charged":1,"used_today":4342,"day_limit":5000,"remaining_today":658,"minute_limit":60,"resets_at":"2026-10-03T00:00:00Z"}}}