LLM Evaluation Jobs - Remote & On-site

LLM Evaluation jobs hiring now at vetted startups and product companies on Alion, updated daily. Average salary around $238,075/year. Compare remote, hybrid and on-site positions with relocation and visa sponsorship. Browse and apply today.

AI/ML
Data Science
Backend
Frontend
Mobile
DevOps
Web3
Games
Hardware
Robotics
Security
QA
Leadership
Networking
Product
Design
Analytics
Support
Solutions
Enterprise Apps
Quantum

Lam Research

lamresearch.com
lamresearch.com • HQ: Fremont, United States • Semiconductor Equipment • 1001-5000 employees • Est. 1980 • NASDAQ: LRCX • Trust C • Hiring 1010, 6 new
HQ: Fremont, United States • Semiconductor Equipment • 1001-5000 employees • Est. 1980 • NASDAQ: LRCX • Trust C • Hiring 1010, 6 new
Lam Research is an American semiconductor equipment company founded in 1980 that specialises in the deposition and etch steps used to build transistors and memory cells. Its tools are central to three-dimensional NAND manufacturing, where hundreds of layers must be etched through in high aspect ratio channels, and to the atomic layer deposition and selective etch processes required at advanced logic and DRAM nodes. Headquartered in Fremont, California and listed on Nasdaq, it also runs a large installed base business supplying spares, upgrades and service, which smooths the sharp cycles in equipment purchasing.
Douglas Bettinger · Chief Financial Officer · decision-maker
Timothy Archer · President and Chief Executive Officer · decision-maker
David Lam · Founder · founder
No careers mailbox known
Likely inactiveReposted ×6 Visa sponsor 9 months ago Long shot · 6%

Senior AI Observability engineer

$92k – $211k per year • Senior • Hybrid (Fremont, United States) • Bachelor's Degree • Senior
LangGraph
AutoGen
Model Context Protocol
Embeddings
Prompt Engineering
Function Calling
AI Agents
Semantic Kernel
LLM
RAG
Hallucination
Anomaly Detection
Time Series Forecasting
LLMOps
Human-in-the-Loop
LLM Evaluation
LLM Guardrails
Tool Use
Machine Learning
LangChain
DevOps
GCP
OpenTelemetry
Azure
CI/CD
AWS
Platform Engineering
AIOps
HPC
TCP/IP
DNS
BGP
OSPF
MPLS
Cybersecurity
Least Privilege
Analytics
A/B Testing
Management
ServiceNow
ITSM
+29
Apply

Vantaca

vantaca.com
vantaca.com • HQ: Wilmington, United States • PropTech • Property Management • 51-200 employees • Est. 2015 • Startup • Trust B • Hiring 11, 1 new
HQ: Wilmington, United States • PropTech • Property Management • 51-200 employees • Est. 2015 • Startup • Trust B • Hiring 11, 1 new
Simplify HOA management with software that handles everything from financials to communication. Enhance homeowner experience with the best HOA software!
Ben Currin · Executive Officer · decision-maker
Careers mailbox is known
Open 277 daysTop 25% pay Visa sponsor 9 months ago Long shot · 11%

Senior Backend Engineer — AI Voice Agent Platform (All Levels)

$200k – $260k per year • Senior • 6+ years exp • Hybrid (Palo Alto, United States) • Full-Time • Senior
Function Calling
AI Agents
Text-to-Speech
Voice Agents
Tool Use
Machine Learning
LangGraph
LangChain
LLM
Anthropic
OpenAI Agents SDK
LiveKit
LLM Evaluation
Vercel AI SDK
Mobile
Twilio
DevOps
WebSockets
OpenTelemetry
WebRTC
Cybersecurity
Least Privilege
+11
Apply

GreenLite

greenlite.com
greenlite.com • HQ: New York, United States • Construction Management Software • 51-200 employees • Est. 2022 • Startup • Trust B • Hiring 9, 2 new
HQ: New York, United States • Construction Management Software • 51-200 employees • Est. 2022 • Startup • Trust B • Hiring 9, 2 new
Get expert permitting guidance "*" indicates required fields Company This field is for validation purposes and should be left unchanged. First Name * Last Name * EMAIL * Company * Phone Number How did you hear about us?
James Gallagher · Executive Officer, Director · decision-maker
Benjamin Allen · Executive Officer, Director · decision-maker
Hyun Cho · Executive Officer, Director · decision-maker
Salen Churi · Director
No careers mailbox known
Open 277 daysVerified live · 3 hours ago Visa sponsor 9 months ago Long shot · 7%

Staff AI Engineer

≈ $168k – $344k per year (Estimated) • Staff+ • 7+ years exp • Hybrid (New York, United States) • Full-Time • Staff
Python
TypeScript
Databases
Pinecone
AI/ML
Copilot
LangGraph
Model Context Protocol
Function Calling
AI Agents
AWS Bedrock
RAG
AWS Bedrock AgentCore
AWS Strands Agents
Human-in-the-Loop
LLM Guardrails
Agentic Workflows
Tool Use
AutoGen
LangChain
LlamaIndex
CrewAI
LLM
LLM Evaluation
DevOps
CI/CD
AWS
Amazon EKS
Amazon S3
Amazon ECS
Kubernetes
Cybersecurity
Least Privilege
+21
Apply

Webook

webook.com
webook.com • HQ: Riyadh, Saudi Arabia • Event Management • Ticketing • 501-1000 employees • Est. 2022 • Trust D • Hiring 79
HQ: Riyadh, Saudi Arabia • Event Management • Ticketing • 501-1000 employees • Est. 2022 • Trust D • Hiring 79
webook is a Saudi event ticketing and experience booking platform based in Riyadh that sells tickets for concerts, sports, festivals and attractions and serves event organisers and venue operators. It has handled ticketing for some of the largest mega events in the kingdom, reports cumulative sales of more than 2 billion, and is expanding abroad with teams in Jordan, Morocco and Portugal. It hires software, DevOps and AI automation engineers, prompt engineers, sales and partnership staff, contact center agents and leads, HR and office managers, legal analysts and Tamheer interns in Saudi Arabia.
No named decision-maker yet.
No careers mailbox known
Likely inactiveVerified live · 1 hour ago 9 months ago Long shot · 5%

Prompt Engineer (LLM Systems, Evals & Safety)

Amman • Remote (Jordan) • Bachelor's Degree
Python
TypeScript
AI/ML
Function Calling
LLM
RAG
LLM Evaluation
Tool Use
LangChain
Analytics
A/B Testing
+1
Apply

DatologyAI

datologyai.com
datologyai.com • HQ: Redwood City, United States • Machine Learning • AI Research Labs • AI Training Data & Annotation • 11-50 employees • Est. 2023 • Startup • Trust C • Hiring 12, 1 new
HQ: Redwood City, United States • Machine Learning • AI Research Labs • AI Training Data & Annotation • 11-50 employees • Est. 2023 • Startup • Trust C • Hiring 12, 1 new
Headquartered in Redwood City, California, DatologyAI is an artificial intelligence startup that builds automated data curation platforms for training foundation models. The company provides software tools that automatically select and prepare optimal training datasets, allowing organizations to train better deep learning models with less data and reduced computing costs. Founded in 2023, the firm helps machine learning teams enhance overall AI performance by efficiently identifying and filtering out noisy data at scale.
No named decision-maker yet.
Careers mailbox is known
Likely inactiveTop 25% pay Visa sponsor 9 months ago Long shot · 5%

Solutions Engineer (AI/ML, Pre-Sales)

$230k – $300k per year • Middle • 4+ years exp • Hybrid (San Mateo, United States) • Relocation • Full-Time • Middle
Python
AI/ML
Spark
Reinforcement Learning
VLM
PyTorch
LLM
Ray
Synthetic Data
Hugging Face
SFT
Post-training
Pre-training
LLM Evaluation
DevOps
GCP
Azure
AWS
Docker
Kubernetes
+10
Apply

Prompt Health

prompthealth.com
prompthealth.com • HQ: San Francisco, United States • Revenue Cycle & Medical Billing • Practice Management Software • EHR & Clinical Software • 51-200 employees • Est. 2017 • Likely startup • Trust A • Hiring 38, 1 new
HQ: San Francisco, United States • Revenue Cycle & Medical Billing • Practice Management Software • EHR & Clinical Software • 51-200 employees • Est. 2017 • Likely startup • Trust A • Hiring 38, 1 new
Prompt Health is a healthcare software company that builds an electronic medical record and practice platform for outpatient rehabilitation therapy clinics. Its product combines clinical documentation, scheduling, patient intake and revenue cycle management, including billing and claims tools that clinics use to raise revenue per provider. The company is headquartered in San Francisco, California, United States, with a team in Hoboken, New Jersey.
No named decision-maker yet.
No careers mailbox known
Open 284 daysTop 25% pay 9 months ago Long shot · 7%

Senior Full Stack AI Engineer (Rapid Prototyping & Analytics )

$160k – $220k per year • Equity • Senior • 4+ years exp • Remote (United States) • Full-Time • Senior
Python
Python
SQLAlchemy
FastAPI
Pydantic
Databases
PostgreSQL
AI/ML
LLM
RAG
Human-in-the-Loop
Structured Outputs
LLM Guardrails
Streamlit
Hybrid Search
LLM Evaluation
DevOps
AWS
Docker
Amazon EC2
Amazon S3
+9
Apply

Lyzr

lyzr.ai
lyzr.ai • HQ: Jersey City, United States • AI Agents • 201-500 employees • Est. 2023 • Startup • Trust B • Hiring 31, 5 new
HQ: Jersey City, United States • AI Agents • 201-500 employees • Est. 2023 • Startup • Trust B • Hiring 31, 5 new
Most AI agent projects never reach production. Lyzr provides the platform and expertise to deploy governed AI agents at scale.
Surendirasiva Selvaraj · Executive Officer, Director · decision-maker
Henry Ford III · Director
Madhu Iyer · Director
Careers mailbox is known
Likely inactiveVerified live · 8 hours ago 9 months ago Long shot · 6%

Core Platform Engineer

In office (Bengaluru, India) • Full-Time
Python
SQL
Go
Rust
Python
FastAPI
AI/ML
Embeddings
Function Calling
AI Agents
LLM
RAG
LLM Evaluation
LLM Guardrails
Mobile
Clean Architecture
DevOps
Azure
AWS
Docker
Platform Engineering
CI/CD
+10
Apply

Reducto

reducto.ai
reducto.ai • HQ: San Francisco, United States • Machine Learning • Custom Software Development • AI Development Tools • Document AI • 501-1000 employees • Est. 2023 • Startup • Trust A • Hiring 37
HQ: San Francisco, United States • Machine Learning • Custom Software Development • AI Development Tools • Document AI • 501-1000 employees • Est. 2023 • Startup • Trust A • Hiring 37
Reducto turns complex documents into structured data for teams building AI products. Its platform parses PDFs, scans and forms through a single model and API instead of a multi-tool pipeline, and is used by companies from startups to large enterprises. The company is based in San Francisco.
No named decision-maker yet.
Careers mailbox is known
Open 325 daysTop 25% pay Visa sponsor 11 months ago Long shot · 11%

Forward Deployed Engineer

$200k – $300k per year • Middle • 3+ years exp • In office (San Francisco, United States) • Full-Time • Middle
Python
AI/ML
AI Agents
Streamlit
LLM Evaluation
OCR
Apply

Lotus

lotus.ai
lotus.ai • HQ: Salt Lake City, United States • Medical AI • Telemedicine & Virtual Care • 201-500 employees • Startup • Trust B • Hiring 3
HQ: Salt Lake City, United States • Medical AI • Telemedicine & Virtual Care • 201-500 employees • Startup • Trust B • Hiring 3
Lotus AI gives you infinite personalized care powered by your health data, AI trained on leading medical evidence, and real doctors you can trust.
No named decision-maker yet.
Careers mailbox is known
Open 326 daysVerified live · 1 hour ago 11 months ago Long shot · 4%

Research Engineer

≈ $163k – $358k per year (Estimated) • In office (San Francisco, United States) • Full-Time
Python
SQL
AI/ML
LoRA
vLLM
Quantization
JAX
Function Calling
AI Agents
TensorRT
QLoRA
PyTorch
LLM
RAG
Hallucination
Synthetic Data
Red Teaming
LLM Evaluation
Agentic Workflows
Tool Use
RLHF
Multimodal AI
Knowledge Distillation
Langfuse
PEFT
Model Distillation
RLAIF
DevOps
Docker
Amazon ECS
QA
Sentry
+21
Apply

Vmax

vmax.ai
vmax.ai • Reinforcement Learning • Machine Learning • Artificial Intelligence • Startup • Trust B • Hiring 7
Reinforcement Learning • Machine Learning • Artificial Intelligence • Startup • Trust B • Hiring 7
Vmax is automating reinforcement learning. We transform proprietary data and evals into new sets of environments. We refine agents on new examples of the tasks they are intended to perform.
No named decision-maker yet.
No careers mailbox known
Open 345 daysTop 25% pay 11 months ago Long shot · 9%

Member of Technical Staff - Applied RL

$300k – $500k per year • Staff+ • Hybrid • Full-Time • Staff
Python
AI/ML
Reinforcement Learning
JAX
PyTorch
Post-training
LLM Evaluation
Apply

VMAX

vmax.com
vmax.com • HQ: San Francisco, United States • Advertising • 1-10 employees • Trust B • Hiring 7
HQ: San Francisco, United States • Advertising • 1-10 employees • Trust B • Hiring 7
Boost your ad revenue with VMax’s powerful ad server solutions built for publishers. Get advanced targeting, faster delivery, real-time analytics & multi-format support.
No named decision-maker yet.
No careers mailbox known
Open 345 daysTop 25% pay 11 months ago Long shot · 9%

Member of Technical Staff - Applied RL

$300k – $500k per year • Staff+ • Hybrid (San Francisco, United States) • Staff
Python
AI/ML
Reinforcement Learning
JAX
PyTorch
Post-training
LLM Evaluation
Apply

Clarity

clarity.agency
clarity.agency • HQ: London, United Kingdom • Customer Support & Help Desk Software • Conversational AI • Likely startup • Trust B • Hiring 4
HQ: London, United Kingdom • Customer Support & Help Desk Software • Conversational AI • Likely startup • Trust B • Hiring 4
Clarity is a software company that uses artificial intelligence to help customer service teams handle interactions faster and with more personalised experiences. Its job ads cover full-stack web development with React, FastAPI and Elasticsearch, generative AI engineering, data engineering and product management. The company is headquartered in London, United Kingdom.
No named decision-maker yet.
No careers mailbox known
Open 346 daysVerified live · 2 hours ago 11 months ago Long shot · 5%

GenAI Engineer

≈ $103k – $191k per year (Estimated) • Senior • 7+ years exp • In office (London, United Kingdom) • Full-Time • Senior
Python
Clarity
Python
FastAPI
Databases
PostgreSQL
AI/ML
Cursor
Claude Code
AI Agents
LLM
RAG
Google ADK
Anomaly Detection
Time Series Forecasting
Human-in-the-Loop
LLM Evaluation
LLM Guardrails
Tool Use
Multimodal AI
DevOps
gRPC
GCP
Azure
CI/CD
AWS
Management
Dropbox
+15
Apply

Duvo

duvo.ai
duvo.ai • HQ: London, United Kingdom • AI Agents • Workflow & Process Automation (RPA, BPM) • 51-200 employees • Startup • Trust A • Hiring 15
HQ: London, United Kingdom • AI Agents • Workflow & Process Automation (RPA, BPM) • 51-200 employees • Startup • Trust A • Hiring 15
Duvo is the shared layer that understands how processes run, orchestrates execution across your systems, and measures the business outcome.
No named decision-maker yet.
Careers mailbox is known
Open 359 daysTop 25% pay Visa sponsor 12 months ago Long shot · 11%

AI Platform Engineer (EU/UK Based - Remote)

$124k – $247k per year • Remote (EU, United Kingdom, CET hours) • Full-Time
Python
TypeScript
Databases
PostgreSQL
Redis
AI/ML
Prompt Engineering
Function Calling
LLM Evaluation
Computer Use
Tool Use
AI Agents
DevOps
Terraform
GCP
Datadog
Docker
Kubernetes
Google GKE
Google Cloud Run
+9
Apply

Distyl AI

distyl.ai
distyl.ai • HQ: San Francisco, United States • AI Consulting & Integration • 201-500 employees • Est. 2022 • Startup • Trust B • Hiring 26, 3 new
HQ: San Francisco, United States • AI Consulting & Integration • 201-500 employees • Est. 2022 • Startup • Trust B • Hiring 26, 3 new
Distyl AI deploys generative artificial intelligence systems inside large enterprises. Its teams build and operate workflow specific applications. The company works with Fortune 500 operations groups.
No named decision-maker yet.
Careers mailbox is known
Likely inactiveTop 25% pay Visa sponsor 1 year ago Long shot · 6%

Applied AI Researcher, System Discovery

$150k – $250k per year • Hybrid (San Francisco, New York, United States) • Full-Time
Cursor
ChatGPT
Fine-tuning
Multimodal AI
AI Agents
Perplexity
LLM Evaluation
Apply

The General Intelligence Company

generalintelligencecompany.com
generalintelligencecompany.com • HQ: New York, United States • Corporate Events • Artificial Intelligence • 1-10 employees • Est. 2025 • Startup • Hiring 3
HQ: New York, United States • Corporate Events • Artificial Intelligence • 1-10 employees • Est. 2025 • Startup • Hiring 3
The General Intelligence Company Of New York. Agentic companies are on the horizon, and we're building them.
No named decision-maker yet.
No careers mailbox known
Open 382 daysTop 25% pay 1 year ago Long shot · 4%

Applied AI Engineer - Agent

$250k – $300k per year • Middle • 4+ years exp • In office (New York, United States) • Full-Time • Middle
Python
Python
Celery
Databases
PostgreSQL
Redis
Supabase
AI/ML
Embeddings
Prompt Engineering
AI Agents
LLM
RAG
Hallucination
LLM Evaluation
LLM Guardrails
Tool Use
DevOps
OpenTelemetry
Datadog
Management
Linear
Slack
Notion
Google Workspace
Gmail
+13
Apply

Cellbyte

cellbyte.com
cellbyte.com • HQ: Munich, Germany • AI Agents • Drug Discovery AI • 11-50 employees • Startup • Trust B • Hiring 4
HQ: Munich, Germany • AI Agents • Drug Discovery AI • 11-50 employees • Startup • Trust B • Hiring 4
AI Agents for Pharmaceutical Drug Launches. Pricing & Market Access professionals use Cellbyte to receive answers in seconds instead of weeks.
Moreira Felix · Managing Directors · decision-maker
Daniel Vidal · Managing Directors · decision-maker
Careers mailbox is known
Open 414 daysVerified live · 2 hours ago 1 year ago Long shot · 7%

AI Engineer Intern / Working Student

Intern • In office (Munich, Germany) • Part-Time • Intern
Python
SQL
AI/ML
LLM
Structured Outputs
LLM Evaluation
Analytics
ETL/ELT
Apply

Neon Health

neonhealth.com
neonhealth.com • HQ: San Francisco, United States • AI Agents • 201-500 employees • Startup • Trust D • Hiring 11
HQ: San Francisco, United States • AI Agents • 201-500 employees • Startup • Trust D • Hiring 11
Neon’s AI workforce jumps over the hurdles that block patient access—getting patients the treatments they need twice as fast and 80% cheaper.
No named decision-maker yet.
Careers mailbox is known
Likely inactiveTop 25% pay 1 year ago Long shot · 3%

Software Engineer

$132k – $202k per year • Junior • 2+ years exp • In office (San Francisco, United States) • Full-Time • Junior
TypeScript
AI/ML
Copilot
Cursor
Claude
RLHF
AI Agents
LLM
LLM Evaluation
LangChain
Prompt Engineering
NLP
Structured Outputs
Multi-Agent Systems
DevOps
Git
Management
ServiceNow
+7
Apply

get_unblocked

getunblocked.com
getunblocked.com • HQ: Vancouver, Canada • Machine Learning • Custom Software Development • 1-10 employees • Est. 2021 • Startup • Hiring 2
HQ: Vancouver, Canada • Machine Learning • Custom Software Development • 1-10 employees • Est. 2021 • Startup • Hiring 2
Unblocked is the context layer for agentic software development. It reasons across code, conversations, issues, docs, product, and production systems so agents can complete work correctly with less human intervention.
No named decision-maker yet.
Careers mailbox is known
Open 507 daysVerified live · 1 hour ago 1 year ago Long shot · 5%

Senior AI Engineer

≈ $81k – $146k per year (Estimated) • Senior • In office (Vancouver, Canada) • Full-Time • Senior
Cursor
Claude Code
Model Context Protocol
Embeddings
AI Agents
LLM
Reranking
OpenAI
Anthropic
Context Engineering
LLM Evaluation
Fine-tuning
Knowledge Distillation
LLM Guardrails
Model Distillation
+7
Apply

Chakra

chakra.dev
chakra.dev • HQ: New York, United States • AI Agents • AI Training Data & Annotation • 1-10 employees • Startup • Hiring 3
HQ: New York, United States • AI Agents • AI Training Data & Annotation • 1-10 employees • Startup • Hiring 3
Deterministic RL environments and trajectory datasets for CUA training. Pixel-perfect multi-layer clones, temporal integrity, automated verifiable task generation.
No named decision-maker yet.
Careers mailbox is known
Open 528 daysVerified live · 1 hour ago 1 year ago Long shot · 9%

Member of Technical Staff, Applied Research

≈ $173k – $353k per year (Estimated) • Staff+ • In office (New York, United States) • Master's Degree • Full-Time • Staff
Python
Python
FastAPI
AI/ML
Fine-tuning
Reinforcement Learning
Function Calling
PyTorch
LLM
SFT
GRPO
Post-training
FSDP
LLM Evaluation
Tool Use
Reward Modeling
Machine Learning
DevOps
AWS
Management
Airtable
Stripe
+10
Apply

Skillable

skillable.com
skillable.com • HQ: New Port Richey, United States • IT Training & Certification • Learning Management Systems • 51-200 employees • Est. 2004 • Hiring 3
HQ: New Port Richey, United States • IT Training & Certification • Learning Management Systems • 51-200 employees • Est. 2004 • Hiring 3
Skillable provides cloud lab and virtual environments for hands-on IT and technical skills training.
No named decision-maker yet.
No careers mailbox known
Open 625 daysVerified live · 2 hours ago 1 year ago Long shot · 5%

Senior Software Engineer

$130k – $170k per year • Senior • 8+ years exp • Remote (United States) • Bachelor's Degree • Full-Time • Senior
JavaScript
TypeScript
C#
Node JS
C#
Entity Framework Core
Node JS
Express
Databases
MS SQL
AI/ML
Copilot
Claude
Model Context Protocol
AI Agents
LLM Evaluation
Frontend
Next.js
React.js
Management
Agile
+7
Apply
Machines can read this selection · checked 15:33 UTC
LLM Evaluation Jobs - Remote & On-site
Frequently asked questions
How many LLM Evaluation jobs are available on Alion?
There are currently 761 active listings from 478 hiring companies on Alion, refreshed daily.
What salary do LLM Evaluation jobs pay?
The average salary is around $238,075 per year based on the current listings.
Are remote options available for LLM Evaluation jobs?
Yes, many of these roles offer remote or hybrid work, and you can filter listings by remote, relocation and visa sponsorship.
How do I apply for LLM Evaluation jobs?
Open any listing on Alion and apply directly with the employer through the provided link - no lengthy sign-up required.