Research Scientist / Engineer, Foundation Model Evaluation
Confirmed on the employer's own hiring board on Oct 11, 2026. First seen by Alion on Mar 25, 2026. Apple scores B on the Alion truth index.
We build frontier foundation models that power intelligent experiences at Apple. Our team works across the full training lifecycle: including pre-training foundation models, and developing mid-training approaches that bridge general capability and task-specific performance. What makes our work distinct is that we're engineering models specifically for Apple silicon and optimized for experiences that are private, personal, and deeply integrated into the OS. We're solving frontier problems in reward modeling to resist reward hacking, handling sparse and delayed rewards in agentic settings, and aligning models reliably across the spectrum from open-ended creative tasks to precise, action-taking workflows. If you're drawn to hard problems where the research and the product are inseparable, this is the team.
Description
This is a hands-on role focused on the models that power Apple products used daily by over a
billion people. You will design evaluation systems where the outcome is not just a score, but an
actionable signal - one that drives model improvement and predicts real user experience.
Working alongside model training and product teams, you will close the loop between evaluation
and improvement.
Our work spans three areas:
- Frontier capability assessment: benchmarking against the state of the art in reasoning,
code, knowledge, and agentic workflows
- Product-aligned evaluation: measuring model quality in ways that reflect real user
experience
- Evaluation-to-training integration: feeding actionable insights back into the model
development cycle
You may focus on one area or work across multiple, depending on your background and
interests.
We build frontier foundation models that power intelligent experiences at Apple. Our team works across the full training lifecycle: including pre-training foundation models, and developing mid-training approaches that bridge general capability and task-specific performance. What makes our work distinct is that we're engineering models specifically for Apple silicon and optimized for experiences that are private, personal, and deeply integrated into the OS. We're solving frontier problems in reward modeling to resist reward hacking, handling sparse and delayed rewards in agentic settings, and aligning models reliably across the spectrum from open-ended creative tasks to precise, action-taking workflows. If you're drawn to hard problems where the research and the product are inseparable, this is the team.
Minimum Qualifications
3+ years of experience in AI model evaluation, NLP, or a related area (e.g., natural language generation, information retrieval, or conversational AI)
Strong fundamentals in machine learning, natural language processing, and statistical analysis
Proficiency in Python and experience with ML frameworks (PyTorch, JAX, or equivalent)
Demonstrated ability to translate research insights into practical implementations
Strong experimental design skills: ability to design rigorous comparisons and draw valid conclusions from results
Clear technical communication: ability to distill evaluation results into actionable recommendations for cross-functional partners
MS or PhD in Computer Science, Machine Learning, Natural Language Processing or a related technical field. Equivalent practical experience will be considered.
Preferred Qualifications
PhD in Computer Science, Machine Learning, NLP, or a related field
Direct experience evaluating large language models, e.g. benchmark design, model-based judging
Track record of collaborating with model training and data teams to turn evaluation findings into training improvements
Experience building reusable evaluation tooling or analysis frameworks adopted across teams
Familiarity with human evaluation methodology and experience partnering with annotation teams or vendors to assess model quality