{"id":461693,"url":"https://alion.io/job/theagi-research-engineer-evals","title":"Research Engineer - Evals","company":{"id":183962,"name":"The AGI Company","domain":"theagi.company","url":"https://alion.io/company/theagi","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"B","score":75,"open_postings":6,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-28T05:59:10Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Francisco, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":163000,"max_usd":362000,"period":"year","method":"role_country_seniority_unknown","sample_n":4425},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Computer Use","optional":false},{"name":"Function Calling","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"OpenAI","optional":false}],"status":"live","first_seen_at":"2026-05-27T16:52:12Z","employer_posted_date":"2026-05-27","last_verified_at":"2026-10-28T06:15:45Z","board_verified":true,"closed_at":null,"days_open":153,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":153},"description":"Think Different. Build the Future.\nOur Mission\nBuild everyday AGI. Trustworthy, consumer-grade agents that redefine human-AI collaboration for millions. Software shouldn’t wait for commands; it should partner with you, amplifying what you can do every single day.\nWhy AGI, Inc.\nWe’re a stealth team of elite founders and AI researchers, with backgrounds spanning Stanford, OpenAI, and DeepMind. We’re industry leaders in mobile and computer-use agents, bringing these capabilities to consumer scale.\nGrounded in years of agent research, our AI is designed with trustworthiness and reliability as core pillars, not afterthoughts.\nWe are supported by tier-1 investors who funded the first generation of AI giants; now they’re backing us to build the next: everyday AGI. (Watch the demo)\nIf you see possibility where others see limits, read on.\nYou decide what \"better\" means.\nModels, agents, and product features all ship behind one question: did this actually get better? Without a strong evals function, the lab ships vibes. With one, every training run, every prompt change, every agent capability moves a number we trust - and the team makes decisions on real signal, not the loudest opinion in the room.\nYou'll build the eval harness for AGI - across model capability, agentic behavior, on-device performance, and end-user experience. You'll set the bar for what counts as \"shipped\" and protect it from the gravity of product deadlines.\nTasks you will own\nThe eval suites that gate every model and agent release - capability, behavior, regressions, and human-rated rubrics that catch what automated evals miss\n\nThe dashboards and tooling that make researcher experiment loops fast and leadership decisions easy\n\nThe bar - what counts as ready to ship, and how we know\n\nAreas where you will assist\nResearch, by making sure what we measure is what we want\n\nProduct engineers, by instrumenting real-user behavior on real devices\n\nPartnerships, by translating \"did it get better\" into language an OEM partner can hold us to\n\nSkills you'll be expected to teach\nHow to measure non-deterministic systems - agent eval, tool use, long-horizon tasks, multilingual behavior\n\nHow to push back on a metric that's being gamed without breaking the team\n\nSkills you'll be expected to learn\nOn-device perf trade-offs and how they show up in real-user evals\n\nWhat QA-ing AI at OEM scale actually looks like\n\nThe realities of shipping consumer agents to production partners\n\nTimeline of success\nAfter 30 days - You've audited every eval we run today and produced a sharp doc on what's good, what's noise, and what's missing. You've fixed the most embarrassing gap.\nAfter 60 days - You've stood up a new eval surface - agentic, on-device, or behavioral - and the team is making real decisions on its output. Researchers come to you before launching a run, not after.\nAfter 90 days - Releases now ship against your eval bar, not a vibe-check. You've caught a regression that would have shipped, and cleared a launch the team was nervous about. You're shaping the research roadmap by surfacing where we're flat, where we're climbing, and where we're lying to ourselves.\nCompensation\nCompetitive cash and meaningful equity. Top-tier relocation and immigration support. SF, in person.\nHow to apply\nSend a link to an eval, benchmark, or measurement system you built - and one paragraph on what decision it changed. Plus your resume or LinkedIn. Every exceptional candidate hears back within 48 hours.","description_format":"text","description_chars":3478,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":true,"industries":["AI Agents"],"lifecycle":[{"event":"open","at":"2026-09-06T11:17:20Z"}],"visa":[],"liveness":{"score":5,"band":"cold","label":"Long shot","p_open":1,"p_active":0.18,"p_room":0.28,"age_days":153,"expected_fill_days":23,"reasons":["conf:4","win:tail","crowd:"],"computed_at":"2026-10-28T05:59:10Z"},"pay":null,"html_url":"https://alion.io/job/theagi-research-engineer-evals","json_url":"https://alion.io/job/theagi-research-engineer-evals.json","meta":{"generated_at":"2026-10-28T09:49:10Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":12825,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-29T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":183962},"rest":"https://alion.io/mcp/rest/get_company?id=183962"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ftheagi-research-engineer-evals"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ftheagi-research-engineer-evals"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ftheagi-research-engineer-evals"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/theagi-research-engineer-evals\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ftheagi-research-engineer-evals"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":461693}},{"tool":"search_jobs","arguments":{"company":"theagi.company","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":461693},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}