{"id":2135641,"url":"https://alion.io/job/apple-research-scientist-engineer-foundation-model-evaluation","title":"Research Scientist / Engineer, Foundation Model Evaluation","company":{"id":12,"name":"Apple","domain":"apple.com","url":"https://alion.io/company/apple","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Apple Jobs","truth_index":{"grade":"A","score":93,"open_postings":893,"ghost_share":0,"stale_share":0.075,"repost_share":0,"time_to_fill_p50_days":138,"computed_at":"2026-10-12T06:55:09Z"}},"role":"Data Science","role_family":"Data Science","seniority":"middle","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Cupertino, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":172000,"max_usd":324000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":408},"experience_years_min":3,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"AI Agents","optional":false},{"name":"JAX","optional":false},{"name":"Machine Learning","optional":false},{"name":"NLP","optional":false},{"name":"Pre-training","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"Reward Modeling","optional":false}],"status":"closed","first_seen_at":"2026-03-25T22:20:03Z","employer_posted_date":"2026-10-11","last_verified_at":"2026-10-11T23:11:48Z","board_verified":false,"closed_at":"2026-10-11T23:11:48Z","days_open":200,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":199},"description":"We build frontier foundation models that power intelligent experiences at Apple. Our team works across the full training lifecycle: including pre-training foundation models, and developing mid-training approaches that bridge general capability and task-specific performance. What makes our work distinct is that we're engineering models specifically for Apple silicon and optimized for experiences that are private, personal, and deeply integrated into the OS. We're solving frontier problems in reward modeling to resist reward hacking, handling sparse and delayed rewards in agentic settings, and aligning models reliably across the spectrum from open-ended creative tasks to precise, action-taking workflows. If you're drawn to hard problems where the research and the product are inseparable, this is the team.\nDescription\nThis is a hands-on role focused on the models that power Apple products used daily by over a\nbillion people. You will design evaluation systems where the outcome is not just a score, but an\nactionable signal - one that drives model improvement and predicts real user experience.\nWorking alongside model training and product teams, you will close the loop between evaluation\nand improvement.\nOur work spans three areas:\nFrontier capability assessment: benchmarking against the state of the art in reasoning,\ncode, knowledge, and agentic workflows\nProduct-aligned evaluation: measuring model quality in ways that reflect real user\nexperience\nEvaluation-to-training integration: feeding actionable insights back into the model\ndevelopment cycle\nYou may focus on one area or work across multiple, depending on your background and\ninterests.\nWe build frontier foundation models that power intelligent experiences at Apple. Our team works across the full training lifecycle: including pre-training foundation models, and developing mid-training approaches that bridge general capability and task-specific performance. What makes our work distinct is that we're engineering models specifically for Apple silicon and optimized for experiences that are private, personal, and deeply integrated into the OS. We're solving frontier problems in reward modeling to resist reward hacking, handling sparse and delayed rewards in agentic settings, and aligning models reliably across the spectrum from open-ended creative tasks to precise, action-taking workflows. If you're drawn to hard problems where the research and the product are inseparable, this is the team.\nMinimum Qualifications\n3+ years of experience in AI model evaluation, NLP, or a related area (e.g., natural language generation, information retrieval, or conversational AI)\nStrong fundamentals in machine learning, natural language processing, and statistical analysis\nProficiency in Python and experience with ML frameworks (PyTorch, JAX, or equivalent)\nDemonstrated ability to translate research insights into practical implementations\nStrong experimental design skills: ability to design rigorous comparisons and draw valid conclusions from results\nClear technical communication: ability to distill evaluation results into actionable recommendations for cross-functional partners\nMS or PhD in Computer Science, Machine Learning, Natural Language Processing or a related technical field. Equivalent practical experience will be considered.\nPreferred Qualifications\nPhD in Computer Science, Machine Learning, NLP, or a related field\nDirect experience evaluating large language models, e.g. benchmark design, model-based judging\nTrack record of collaborating with model training and data teams to turn evaluation findings into training improvements\nExperience building reusable evaluation tooling or analysis frameworks adopted across teams\nFamiliarity with human evaluation methodology and experience partnering with annotation teams or vendors to assess model quality","description_format":"text","description_chars":3848,"description_truncated":false,"requirements":{"experience_years_min":3,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"master","optional":true},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Operating Systems","Laptops","Smartwatches & Fitness Trackers","Tablets"],"lifecycle":[{"event":"open","at":"2026-10-09T01:35:03Z"},{"event":"close","at":"2026-10-09T01:39:59Z"},{"event":"reopen","at":"2026-10-09T06:23:17Z"},{"event":"close","at":"2026-10-11T12:23:15Z"},{"event":"reopen","at":"2026-10-11T19:23:20Z"},{"event":"close","at":"2026-10-11T23:11:48Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 6,350 · green card filings: 37","filings_12m":6350,"filings_prev_12m":6527,"green_card_filings_12m":37,"median_offered_wage_usd":177355,"route":null,"cap_exempt":false,"checked_at":"2026-10-03T21:08:04+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","US Department of Labor: PERM disclosure data (green cards)"],"filings_for_role_12m":239}],"liveness":null,"pay":null,"html_url":"https://alion.io/job/apple-research-scientist-engineer-foundation-model-evaluation","json_url":"https://alion.io/job/apple-research-scientist-engineer-foundation-model-evaluation.json","meta":{"generated_at":"2026-10-12T14:24:23Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":22652,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-13T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":12},"rest":"https://alion.io/mcp/rest/get_company?id=12"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fapple-research-scientist-engineer-foundation-model-evaluation"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fapple-research-scientist-engineer-foundation-model-evaluation"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fapple-research-scientist-engineer-foundation-model-evaluation"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/apple-research-scientist-engineer-foundation-model-evaluation\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fapple-research-scientist-engineer-foundation-model-evaluation"}]}