2,386,326open jobs
148,846companies
289,929added this week
Browse all
Salary
≈ $229k – $441k per year (Estimated)
Location
In office (San Diego)
Seniority
Senior · 3+ years exp
Visa
H-1B filings in 12 months: 6,545 · for this role: 2,993 · green card filings: 214

Confirmed on the employer's own hiring board on Nov 19, 2026. First seen by Alion on Aug 3, 2026. Apple scores B on the Alion truth index.

Overview
Company
Impact
Profile match
Apple is an American multinational technology company founded in 1976 by Steve Jobs, Steve Wozniak and Ronald Wayne, and headquartered in Cupertino, California. It designs and sells consumer hardware including the iPhone, Mac, iPad, Apple Watch, AirPods and Vision Pro, together with the operating systems and silicon that run them. A growing services division built around the App Store, iCloud, Apple Music, Apple TV+ and Apple Pay now contributes a large share of profit, making Apple one of the most valuable companies in the world.

The Apple Intelligence Platform Experience Validation team builds the tooling and automation that keeps Apple Intelligence features high-quality before they ship. We are looking for a Senior SDET to lead the design and implementation of automated model evaluation: standing up LLM-as-a-judge in existing and new pipelines, and building the infrastructure that catches model regressions before they reach human evaluation or the live on population.

This is a hands-on, senior individual-contributor role. You will own eval automation as a discipline across the team, partnering with modeling, framework, and infrastructure teams to make model quality a first-class, continuously measured signal.

Description

You will build and maintain model level, component or end-to-end evaluation coverage for the generative features our team validates. Your job is to leverage LLM judge scoring output quality in automation, ensuring reliable, repeatable eval jobs that run that produce actionable signal.

The kinds of problems you will work on include:

* Image / visual generation: validating model output and its associated classification metadata, and detecting quality or behavior regressions across model updates.

* Natural-language generation: evaluating whether generated artifacts and responses match user intent, moving at-desk LLM judges into a scalable and repeatable automation environment.

* Correctness beyond string matching: replacing exact-match checks for open-ended or factual responses with an LLM-as-judge stage integrated into the pipeline.

* Generated insights and summaries: assessing whether model-generated content is sensible and good enough to surface to users.

You will decide when a component-level check (an API or CLI that exercises the model against its framework) is sufficient and when a full end-to-end user flow is required, and you will build the tooling for both.

Minimum Qualifications

BS in Computer Science, Mathematics, or a related field (or equivalent practical experience)

Three years of relevant industry experience in test automation, software development, or related areas.

Preferred Qualifications

Strong practical knowledge of Python, including data-pipeline fluency (JSON/YAML, REST APIs).

Hands-on experience with LLM-as-a-judge evaluation and rubric design, or a strong demonstrated ability to ramp into it quickly.

Strong software engineering fundamentals - able to define atomic, composable components and build maintainable pipelines and tooling, not just scripts.

Strong debugging and triage skills; able to separate genuine regressions from infrastructure or rubric noise.

Strong knowledge of the software development lifecycle, testing methodologies, and QA processes.

Excellent written and verbal communication; able to document clearly and describe quality signal to modeling and leadership audiences.

Ability to lead work across varying priorities and partner multi-functionally with modeling, framework, and infrastructure teams.

Experience building on-device tooling and device/model eval infrastructure.

Experience integrating with CI/CD and job orchestration systems, and comfort deploying tooling as reusable libraries.

Familiarity with generative model behavior - image generation, NLP, or LLM output evaluation.

Experience curating and reasoning about large datasets; comfort manually inspecting data (Jupyter or similar) to build intuition and drive next steps.

Awareness of dataset bias and fairness considerations in evaluation.

Experience with database/query tooling (e.g., SQL) and dashboards/visualization for reporting quality trends.

Experience with Xcode is a bonus.

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,386,326 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

AI/ML
Remote, many countries
Similar stack
Same company
San Diego
AI Engineer 1 hour ago
$190k – $240k per year · Equity 1–3% · In office · Full-Time · 3+ years exp · San Francisco
Python
JavaScript
AI/ML
Fine-tuning
Function Calling
AI Agents
Context Engineering
Browser Agents
Frontend
Tailwind CSS
Next.js
React.js
Radix UI
shadcn/ui
Mobile
Twilio
DevOps
AWS
Apply
Senior AI Engineer 1 day ago
$130k – $150k per year · Equity · In office · Full-Time · Bachelor's Degree · New York
Databases
Databricks
AI/ML
Model Context Protocol
LLM
Human-in-the-Loop
LLM Guardrails
Copilot Studio
DevOps
Azure DevOps
Azure
CI/CD
Git
Cybersecurity
Microsoft Entra ID
Apply
In office · Full-Time · Master's Degree · San Diego
Python
AI/ML
PyTorch
Time Series
Machine Learning
DevOps
Linux
Apply
In office · Full-Time · Bachelor's Degree · Boston · Arlington · London
Python
AI/ML
LangGraph
LangChain
LLM
RAG
Machine Learning
DevOps
Azure
CI/CD
Apply
In office · 5+ years exp · Bachelor's Degree · Cupertino
AI/ML
Claude Code
AI Agents
OpenAI Codex
Post-training
Machine Learning
Apply
$22k – $44k per year (net) · Remote (EAEU) · 3+ years exp · Yekaterinburg
Python
JavaScript
TypeScript
SQL
Node JS
Python
FastAPI
Django
AI/ML
AI Agents
LLM
RAG
DevOps
Rest API
CI/CD
Git
Docker
GitHub
Apply
$55k – $88k per year (net) · Remote (EU) · Full-Time · Tallinn
Python
PowerShell
AI/ML
AI Agents
AWS Bedrock
LLM
LLM Guardrails
DevOps
Terraform
CI/CD
AWS
Docker
Kubernetes
Amazon EKS
Apply
$49k – $79k per year · Remote (EU) · Full-Time · Riga
Python
PowerShell
AI/ML
AI Agents
AWS Bedrock
LLM
LLM Guardrails
DevOps
Terraform
CI/CD
AWS
Docker
Kubernetes
Amazon EKS
Apply
$30k – $36k per year · Remote (LATAM, Europe, Africa, US time zones hours) · Full-Time · 5+ years exp
JavaScript
TypeScript
Node JS
Node JS
Nest.JS
Prisma
AI/ML
Claude
Claude Code
Model Context Protocol
Function Calling
AI Agents
OpenAI Codex
Multi-Agent Systems
Frontend
Turborepo
DevOps
GitHub Actions
CI/CD
AWS
Docker
AWS Fargate
Amazon ECS
Cybersecurity
ISO 27001
SOC 2
Apply
$54k – $60k per year · Remote (LATAM, Europe, Africa, US time zones hours) · Contractor · 5+ years exp · Bachelor's Degree
Python
TypeScript
C#
C#
.NET
Databases
Databricks
Amazon Redshift
AI/ML
Claude
Model Context Protocol
AI Agents
AWS Bedrock
LLM
AWS Bedrock AgentCore
Human-in-the-Loop
Structured Outputs
DevOps
AWS
Management
Zapier
Apply
≈ $212k – $409k per year (Estimated) · In office · 3+ years exp · Bachelor's Degree · San Diego
Python
Swift
AI/ML
Fine-tuning
JAX
Multimodal AI
AI Agents
TensorFlow
PyTorch
LLM
LLM Evaluation
Mobile
Core ML
Apply
In office · Full-Time · 6+ years exp · High School Diploma · San Diego
Chips/EDA
Photolithography
Apply
Remote (United States) · Los Angeles · Portland · San Diego · Seattle
AI/ML
Claude
Marketing
Google Ads
Apply
In office · San Diego
Apply
≈ $156k – $295k per year (Estimated) · In office · 8+ years exp · Bachelor's Degree · San Diego
MATLAB
Chips/EDA
JMP
Minitab
Design
SolidWorks
SpaceTech
CATIA
GD&T
Life Sciences
Design of Experiments
Apply
See all jobs
This is one of many
2,386,326 more open roles from verified company boards, updated every day.