LLM Evaluation Jobs - Remote & On-site

LLM Evaluation jobs hiring now at vetted startups and product companies on Alion, updated daily. Average salary around $238,075/year. Compare remote, hybrid and on-site positions with relocation and visa sponsorship. Browse and apply today.

AI/ML
Software
Data
Security
DevOps & Cloud
Leadership
Product
Design
Fintech
Hardware & Chips
Defense
Aerospace
Biotech
Energy
Robotics
Automotive
MedTech
Science

Lyzr

lyzr.ai
lyzr.ai • HQ: Jersey City, United States • AI Agents • 201-500 employees • Est. 2023 • Startup • Trust B • Hiring 30
HQ: Jersey City, United States • AI Agents • 201-500 employees • Est. 2023 • Startup • Trust B • Hiring 30
Most AI agent projects never reach production. Lyzr provides the platform and expertise to deploy governed AI agents at scale.
Surendirasiva Selvaraj · Executive Officer, Director · decision-maker
Henry Ford III · Director
Madhu Iyer · Director
Careers mailbox is known
Likely inactiveVerified live · 1 day ago 9 months ago Long shot · 6%

Core Platform Engineer

In office (Bengaluru, India) • Full-Time
SQL
Python
Rust
Go
Python
FastAPI
AI/ML
Embeddings
Function Calling
LLM Evaluation
LLM Guardrails
RAG
LLM
AI Agents
Mobile
Clean Architecture
DevOps
Platform Engineering
Docker
Azure
AWS
CI/CD
+10
Apply

Reducto

reducto.ai
reducto.ai • HQ: San Francisco, United States • Machine Learning • Custom Software Development • AI Development Tools • Document AI • 501-1000 employees • Est. 2023 • Startup • Trust A • Hiring 33
HQ: San Francisco, United States • Machine Learning • Custom Software Development • AI Development Tools • Document AI • 501-1000 employees • Est. 2023 • Startup • Trust A • Hiring 33
Reducto turns complex documents into structured data for teams building AI products. Its platform parses PDFs, scans and forms through a single model and API instead of a multi-tool pipeline, and is used by companies from startups to large enterprises. The company is based in San Francisco.
No named decision-maker yet.
Careers mailbox is known
Open 332 daysTop 25% pay Visa sponsor 11 months ago Long shot · 11%

Forward Deployed Engineer

$200k – $300k per year • Middle • 3+ years exp • In office (San Francisco, United States) • Full-Time • Middle
Python
AI/ML
Streamlit
LLM Evaluation
OCR
Apply

Lotus

lotus.ai
lotus.ai • HQ: Salt Lake City, United States • Medical AI • Telemedicine & Virtual Care • 201-500 employees • Startup • Hiring 3, 1 new
HQ: Salt Lake City, United States • Medical AI • Telemedicine & Virtual Care • 201-500 employees • Startup • Hiring 3, 1 new
Lotus AI gives you infinite personalized care powered by your health data, AI trained on leading medical evidence, and real doctors you can trust.
No named decision-maker yet.
Careers mailbox is known
Open 333 daysVerified live · 3 hours ago 11 months ago Long shot · 4%

Research Engineer

≈ $162k – $359k per year (Estimated) • In office (San Francisco, United States) • Full-Time
SQL
Python
AI/ML
LoRA
vLLM
Quantization
JAX
Function Calling
TensorRT
QLoRA
Hallucination
Synthetic Data
Red Teaming
LLM Evaluation
Agentic Workflows
PyTorch
RAG
LLM
AI Agents
RLHF
Multimodal AI
Knowledge Distillation
Langfuse
PEFT
Model Distillation
RLAIF
DevOps
Amazon ECS
Docker
QA
Sentry
MedTech
FHIR
+21
Apply

Clarity

clarity.agency
clarity.agency • HQ: London, United Kingdom • Customer Support & Help Desk Software • Conversational AI • Likely startup • Trust B • Hiring 4
HQ: London, United Kingdom • Customer Support & Help Desk Software • Conversational AI • Likely startup • Trust B • Hiring 4
Clarity is a software company that uses artificial intelligence to help customer service teams handle interactions faster and with more personalised experiences. Its job ads cover full-stack web development with React, FastAPI and Elasticsearch, generative AI engineering, data engineering and product management. The company is headquartered in London, United Kingdom.
No named decision-maker yet.
No careers mailbox known
Open 353 daysVerified live · 2 hours ago 11 months ago Long shot · 6%

GenAI Engineer

≈ $103k – $193k per year (Estimated) • Senior • 7+ years exp • In office (London, United Kingdom) • Full-Time • Senior
Clarity
Python
Python
FastAPI
Databases
PostgreSQL
AI/ML
Google ADK
Anomaly Detection
Time Series
Human-in-the-Loop
LLM Evaluation
LLM Guardrails
Cursor
Claude Code
RAG
LLM
AI Agents
Multimodal AI
DevOps
gRPC
GCP
Azure
CI/CD
AWS
+13
Apply

Distyl AI

distyl.ai
distyl.ai • HQ: San Francisco, United States • AI Consulting & Integration • 201-500 employees • Est. 2022 • Startup • Trust B • Hiring 27, 1 new
HQ: San Francisco, United States • AI Consulting & Integration • 201-500 employees • Est. 2022 • Startup • Trust B • Hiring 27, 1 new
Distyl AI deploys generative artificial intelligence systems inside large enterprises. Its teams build and operate workflow specific applications. The company works with Fortune 500 operations groups.
No named decision-maker yet.
Careers mailbox is known
Likely inactiveTop 25% pay Visa sponsor 1 year ago Long shot · 6%

Applied AI Researcher, System Discovery

$150k – $250k per year • Hybrid (San Francisco, New York, United States) • Full-Time
Fine-tuning
Multimodal AI
Perplexity
LLM Evaluation
Cursor
ChatGPT
Apply

Pear VC

pear.vc
pear.vc • HQ: San Francisco, United States • Venture Capital • 51-200 employees • Est. 2013 • Trust A • Hiring 100, 6 new
HQ: San Francisco, United States • Venture Capital • 51-200 employees • Est. 2013 • Trust A • Hiring 100, 6 new
Pear VC is a venture capital firm specializing in pre-seed and seed-stage investments in startups. Founded by Pejman Nozad and Mar Hershenson, the firm provides hands-on support in areas such as product development, recruiting, fundraising, and go-to-market strategy. Pear has backed companies including DoorDash, Gusto, Aurora Solar, Vanta, and Guardant Health.
No named decision-maker yet.
Careers mailbox is known
Verified live · 3 hours ago 1 year ago Long shot · 15%

AI / ML Engineer - Known

≈ $170k – $330k per year (Estimated) • Middle • 3+ years exp • In office (San Francisco, United States) • Full-Time • Middle
Python
AI/ML
Embeddings
JAX
Hugging Face
Human-in-the-Loop
LLM Guardrails
Agentic Workflows
TensorFlow
PyTorch
LLM
Machine Learning
AI Agents
Reinforcement Learning
LLM Evaluation
Voice Agents
Analytics
A/B Testing
+8
Apply

The General Intelligence Company

generalintelligencecompany.com
generalintelligencecompany.com • HQ: New York, United States • Corporate Events • Artificial Intelligence • 1-10 employees • Est. 2025 • Startup • Trust B • Hiring 3
HQ: New York, United States • Corporate Events • Artificial Intelligence • 1-10 employees • Est. 2025 • Startup • Trust B • Hiring 3
The General Intelligence Company Of New York. Agentic companies are on the horizon, and we're building them.
No named decision-maker yet.
No careers mailbox known
Open 389 daysTop 25% pay 1 year ago Long shot · 5%

Applied AI Engineer - Agent

$250k – $300k per year • Middle • 4+ years exp • In office (New York, United States) • Full-Time • Middle
Python
Python
Celery
Databases
Supabase
Redis
PostgreSQL
AI/ML
Embeddings
Hallucination
LLM Evaluation
LLM Guardrails
Prompt Engineering
RAG
LLM
AI Agents
DevOps
OpenTelemetry
Datadog
Management
Linear
Notion
+9
Apply

Cellbyte

cellbyte.com
cellbyte.com • HQ: Munich, Germany • AI Agents • Drug Discovery AI • 11-50 employees • Startup • Trust B • Hiring 4
HQ: Munich, Germany • AI Agents • Drug Discovery AI • 11-50 employees • Startup • Trust B • Hiring 4
AI Agents for Pharmaceutical Drug Launches. Pricing & Market Access professionals use Cellbyte to receive answers in seconds instead of weeks.
Moreira Felix · Managing Directors · decision-maker
Daniel Vidal · Managing Directors · decision-maker
Careers mailbox is known
Open 421 daysVerified live · 17 min ago 1 year ago Long shot · 6%

AI Engineer Intern / Working Student

Intern • In office (Munich, Germany) • Part-Time • Intern
SQL
Python
AI/ML
Structured Outputs
LLM Evaluation
LLM
Analytics
ETL/ELT
Apply

Neon Health

neonhealth.com
neonhealth.com • HQ: San Francisco, United States • AI Agents • 201-500 employees • Startup • Trust D • Hiring 11
HQ: San Francisco, United States • AI Agents • 201-500 employees • Startup • Trust D • Hiring 11
Neon’s AI workforce jumps over the hurdles that block patient access—getting patients the treatments they need twice as fast and 80% cheaper.
No named decision-maker yet.
Careers mailbox is known
Likely inactiveTop 25% pay 1 year ago Long shot · 3%

Software Engineer

$132k – $202k per year • Junior • 2+ years exp • In office (San Francisco, United States) • Full-Time • Junior
TypeScript
AI/ML
RLHF
LLM Evaluation
Cursor
Claude
Copilot
LLM
AI Agents
NLP
Structured Outputs
Multi-Agent Systems
LangChain
Prompt Engineering
DevOps
Git
Management
ServiceNow
+7
Apply

get_unblocked

getunblocked.com
getunblocked.com • HQ: Vancouver, Canada • Machine Learning • Custom Software Development • 1-10 employees • Est. 2021 • Startup • Hiring 2
HQ: Vancouver, Canada • Machine Learning • Custom Software Development • 1-10 employees • Est. 2021 • Startup • Hiring 2
Unblocked is the context layer for agentic software development. It reasons across code, conversations, issues, docs, product, and production systems so agents can complete work correctly with less human intervention.
No named decision-maker yet.
Careers mailbox is known
Open 514 daysVerified live · 2 hours ago 1 year ago Long shot · 9%

Senior AI Engineer

≈ $86k – $149k per year (Estimated) • Senior • In office (Vancouver, Canada) • Full-Time • Senior
Embeddings
Reranking
Anthropic
Context Engineering
LLM Evaluation
Cursor
Model Context Protocol
OpenAI
Claude Code
LLM
AI Agents
Fine-tuning
Knowledge Distillation
LLM Guardrails
Model Distillation
+7
Apply

Chakra

chakra.dev
chakra.dev • HQ: New York, United States • AI Agents • AI Training Data & Annotation • 1-10 employees • Startup • Hiring 2
HQ: New York, United States • AI Agents • AI Training Data & Annotation • 1-10 employees • Startup • Hiring 2
Deterministic RL environments and trajectory datasets for CUA training. Pixel-perfect multi-layer clones, temporal integrity, automated verifiable task generation.
No named decision-maker yet.
Careers mailbox is known
Open 535 daysVerified live · 2 hours ago 1 year ago Long shot · 9%

Member of Technical Staff, Applied Research

≈ $175k – $351k per year (Estimated) • Staff+ • In office (New York, United States) • Master's Degree • Full-Time • Staff
Python
Python
FastAPI
AI/ML
Fine-tuning
Reinforcement Learning
Function Calling
SFT
GRPO
Post-training
FSDP
LLM Evaluation
Reward Modeling
PyTorch
LLM
Machine Learning
DevOps
AWS
Management
Airtable
+8
Apply

Mactores

mactores.com
mactores.com • HQ: Seattle, United States • Cloud Consulting & Migration • Data Engineering & Migration Services • 201-500 employees • Est. 2008 • Trust D • Hiring 29
HQ: Seattle, United States • Cloud Consulting & Migration • Data Engineering & Migration Services • 201-500 employees • Est. 2008 • Trust D • Hiring 29
Mactores is a data and cloud consulting firm that helps businesses build and modernise data platforms, with services covering data strategy assessments, data lake and warehouse migration, analytics, machine learning and process automation, largely on Amazon Web Services. Founded in 2008, it serves enterprise clients in the United States and runs a large delivery team in Mumbai, India. The company is headquartered in Seattle, Washington, United States.
No named decision-maker yet.
No careers mailbox known
Likely inactiveVerified live · 2 min ago 2 years ago Long shot · 3%

Generative AI Forward Deployed Engineer

≈ $19k – $40k per year (Estimated) • Middle • 3+ years exp • Mumbai • Remote (India) • Full-Time • Middle • English
TypeScript
Python
Rust
C#
C++
Java
C#
.NET
Databases
DynamoDB
Amazon Redshift
Amazon Aurora
Apache Kafka
AI/ML
LangGraph
LoRA
Fine-tuning
RLHF
Function Calling
QLoRA
AWS Bedrock
CrewAI
Reranking
Amazon SageMaker
DPO
AWS Strands Agents
LLM Evaluation
Multi-Agent Systems
Prompt Engineering
RAG
AI Agents
PEFT
Knowledge Graph
LangChain
Machine Learning
DevOps
AWS Lambda
Amazon Kinesis
API Gateway
CI/CD
AWS
Cybersecurity
PCI DSS
HIPAA
SOC 2
+32
Apply

Bronco AI

bronco.ai
bronco.ai • HQ: San Francisco, United States • Custom Software Development • 1-10 employees • Est. 2022 • Startup • Trust B • Hiring 3
HQ: San Francisco, United States • Custom Software Development • 1-10 employees • Est. 2022 • Startup • Trust B • Hiring 3
AI for chip design and verification. Bronco debugs simulations, brings up testbenches, generates RTL, and closes coverage to win the race to tape-out.
No named decision-maker yet.
No careers mailbox known
Open 808 daysVerified live · 2 hours ago 2 years ago Long shot · 4%

Founding AI Engineer

≈ $166k – $367k per year (Estimated) • Equity • In office (San Francisco, United States) • Full-Time
Python
AI/ML
RLHF
Reinforcement Learning
Function Calling
DPO
Post-training
LLM Evaluation
RLAIF
Reward Modeling
PyTorch
LLM
AI Agents
+4
Apply
Machines can read this selection · checked 02:37 UTC
LLM Evaluation Jobs - Remote & On-site
Frequently asked questions
How many LLM Evaluation jobs are available on Alion?
There are currently 761 active listings from 478 hiring companies on Alion, refreshed daily.
What salary do LLM Evaluation jobs pay?
The average salary is around $238,075 per year based on the current listings.
Are remote options available for LLM Evaluation jobs?
Yes, many of these roles offer remote or hybrid work, and you can filter listings by remote, relocation and visa sponsorship.
How do I apply for LLM Evaluation jobs?
Open any listing on Alion and apply directly with the employer through the provided link - no lengthy sign-up required.