LLM Evaluation Jobs - Remote & On-site

LLM Evaluation jobs hiring now at vetted startups and product companies on Alion, updated daily. Average salary around $230,281/year. Compare remote, hybrid and on-site positions with relocation and visa sponsorship. Browse and apply today.

AI/ML
Data Science
Backend
Frontend
Mobile
DevOps
Web3
Games
Hardware
Robotics
Security
QA
Executive
Networking
Product
Design
Analytics
Support
Enterprise Apps
Quantum

Vosyn

vosyn.ai
Vosyn is an artificial intelligence technology company based in Toronto, Canada, and founded in 2023. The company provides a real-time localization platform that utilizes proprietary multimodal large language models to translate and dub video, audio, and text content while preserving original tone and cultural nuance. It operates globally, serving content creators, media organizations, and enterprises through its consumer applications and developer APIs.
vosyn.ai • HQ: Toronto, Canada • Software • Video Production • Media & Entertainment • Natural Language Processing • Speech & Audio AI • Artificial Intelligence • Professional Services • Translation Services • 51-200 employees • Est. 2023
HQ: Toronto, Canada • Software • Video Production • Media & Entertainment • Natural Language Processing • Speech & Audio AI • Artificial Intelligence • Professional Services • Translation Services • 51-200 employees • Est. 2023
Intern • Remote (Canada) • Master's Degree • Internship • Intern
JavaScript
Node JS
Python
Databases
pgvector
Pinecone
PostgreSQL
Supabase
Weaviate
AI/ML
Claude
Claude Code
Cursor
Embeddings
Hybrid Search
Model Context Protocol
Prompt Engineering
RAG
Windsurf
LLM Evaluation
AI Agents
Function Calling
Anthropic
Frontend
Next.js
React.js
Mobile
Firebase
DevOps
Git
Vector
GitHub
Management
Notion
Apply
Report

ClickUp

clickup.com
ClickUp is a software company headquartered in San Diego, California, and was founded in 2017. The company provides an all-in-one productivity platform that integrates project management, document collaboration, whiteboards, spreadsheets, and AI-powered workflow automation into a single application. It serves millions of users across diverse industries worldwide, ranging from small startups to large enterprise organizations, and operates as a privately held venture-backed firm.
clickup.com • HQ: San Diego, United States • Business Process Automation (BPA) • AI Agents • Artificial Intelligence • Communication & Collaboration • Productivity Software • Software • Project Management • 501-1000 employees • Est. 2017
HQ: San Diego, United States • Business Process Automation (BPA) • AI Agents • Artificial Intelligence • Communication & Collaboration • Productivity Software • Software • Project Management • 501-1000 employees • Est. 2017
Top 25% payVerified live · 1 day ago 3 months ago

Senior Software Engineer, Internally Deployed Products

$160k – $210k per year • Senior • 5+ years exp • Remote (United States) • Full-Time • Senior
Node JS
Python
TypeScript
JavaScript
Databases
Google BigQuery
AI/ML
AI Agents
AutoGen
AWS Bedrock
Claude
Copilot
CrewAI
Cursor
LangGraph
LLM
Model Context Protocol
Prompt Engineering
LangChain
Anthropic
AWS Bedrock AgentCore
LLM Evaluation
OpenAI
Frontend
Next.js
React.js
DevOps
AWS
AWS Fargate
AWS Lambda
CI/CD
GCP
Google Cloud Run
Platform Engineering
Rest API
Terraform
Amazon CloudWatch
Amazon ECS
API Gateway
GitHub
IAM
Cybersecurity
Okta
Analytics
ETL/ELT
Management
ClickUp
Slack
Marketing
HubSpot
Salesforce
Apply
Report

Hopper

hopper.com
Hopper is a travel technology company headquartered in Montreal, Canada, and founded in 2007. The company provides a mobile-first platform for booking flights, hotels, car rentals, and vacation rentals, utilizing data-driven price prediction algorithms and financial technology products. It operates globally with a focus on the North American market, serving over 120 million users through its mobile application and B2B partnerships.
hopper.com • HQ: Montreal, Canada • Vacation Rentals • Predictive Analytics • Artificial Intelligence • FinTech • Financial Services • Travel Technology • Travel Agencies • Travel & Tourism • Est. 2007
HQ: Montreal, Canada • Vacation Rentals • Predictive Analytics • Artificial Intelligence • FinTech • Financial Services • Travel Technology • Travel Agencies • Travel & Tourism • Est. 2007
Top 25% payVerified live · 2 hours ago 3 months ago

Principal Product Manager - Pricing and Personalization

$150k – $350k per year • Equity • Lead • 5+ years expWashingtonChicago • Remote (United States) • Full-Time • Principal
SQL
AI/ML
Anomaly Detection
LLM Evaluation
Apply
$138k – $264k per year (Estimated) • Equity • Lead • 5+ years expMontreal • Remote (Canada) • Full-Time • Principal • English • French
SQL
AI/ML
Anomaly Detection
LLM
Transformers
LLM Evaluation
Apply
Verified live · 2 hours ago 6 months ago

Principal Product Manager - Pricing and Personalization

$142k – $277k per year (Estimated) • Equity • Lead • 5+ years expBostonMiamiAustin • Remote (United States) • Full-Time • Principal
SQL
AI/ML
Anomaly Detection
LLM Evaluation
Apply
Report

Hark

hark.com
Hark was an online digital entertainment platform best known for its extensive library of short audio soundbites, video clips, and pop culture quotes. Launched in 2007, the website allowed users to browse, create, and share playable soundboards featuring memorable lines from movies, television shows, and political figures. While it grew into a popular destination for viral sound clips during the late 2000s and early 2010s, the platform has since ceased its original operations.
hark.com • HQ: San Jose, United States • Machine Learning • Artificial Intelligence
HQ: San Jose, United States • Machine Learning • Artificial Intelligence
Top 25% payVerified live · 1 hour ago 3 months ago

Lead Audio ML Engineer

$120k – $300k per year • Lead • 3+ years expIn office (San Jose, United States) • Full-Time • Staff
Knowledge Distillation
Multimodal AI
PyTorch
Quantization
TensorFlow
LLM Evaluation
AI Agents
Speech Recognition
Apply
Report

Wirescreen

wirescreen.ai
WireScreen is the leading risk intelligence platform on Chinese corporations, revealing investments and relationships that shape global markets and national security.
wirescreen.ai • New York • Washington • Homeland Security • National Security • Data & Analytics • Est. 2019
New York • Washington • Homeland Security • National Security • Data & Analytics • Est. 2019
$150k – $195k per year • Middle • 4+ years expRemote/Hybrid (New York, United States) • Full-Time • Middle
Python
SQL
Python
FastAPI
pySpark
AI/ML
Anomaly Detection
Embeddings
NLP
Spark
Knowledge Graph
LLM Evaluation
AI Agents
Model Context Protocol
Computer Vision
Fine-tuning
NumPy
Scikit-learn
Semantic Search
OCR
Semantic Search
DevOps
Docker
Kubernetes
Terraform
Apply
Report

Semiconductor Engineering

semiengineering.com
Semiconductor Engineering is a technical media and research organization based in San Jose, California, and founded in 2013. The company provides in-depth news, analysis, and technical papers focused on the design, manufacturing, testing, and integration of advanced semiconductor technologies. It operates as a global information hub for chip engineers and industry professionals, offering newsletters, webinars, and specialized research reports.
semiengineering.com • HQ: San Jose, United States • Semiconductor Manufacturing • Manufacturing • Science & Engineering • Publishing • Media & Entertainment • Electrical & Electronic Engineering • Semiconductors • Hardware • 1001-5000 employees • Est. 2013
HQ: San Jose, United States • Semiconductor Manufacturing • Manufacturing • Science & Engineering • Publishing • Media & Entertainment • Electrical & Electronic Engineering • Semiconductors • Hardware • 1001-5000 employees • Est. 2013
Verified live · 7 hours ago 3 months ago

Principal AI Forward Deployment Engineer

$137k – $293k per year (Estimated) • Lead • 10+ years expIn office (Cary, United States) • Relocation • PhD • Full-Time • Principal
Claude
Gemini
Llama
LLM
Qwen
RAG
LangChain
PEFT
AI Agents
AutoGen
Claude Code
Cursor
Fine-tuning
LangGraph
LoRA
Triton
vLLM
LLM Evaluation
Model Context Protocol
TGI
DevOps
HPC
Chips/EDA
Cadence Allegro
Apply
Report

Horizon3

horizon3.ai
Horizon3.ai is a cybersecurity company that specializes in autonomous threat detection and proactive security validation. Its flagship platform, NodeZero, acts as an autonomous penetration testing solution that continuously discovers, tests, and verifies exploitable vulnerabilities across an organization's internal and external infrastructure. By simulating real-world attacker tactics, Horizon3.ai enables IT and security teams to fix critical security gaps before adversaries can exploit them.
horizon3.ai • Chicago • Amsterdam • San Francisco • Las Vegas • Seattle • Artificial Intelligence • Cybersecurity • Penetration Testing
Chicago • Amsterdam • San Francisco • Las Vegas • Seattle • Artificial Intelligence • Cybersecurity • Penetration Testing
Top 25% payVerified live · 4 hours ago 3 months ago

Senior Software Engineer, Agentic Systems

$169k – $208k per year • Senior • 5+ years exp • Remote (United States) • Full-Time • Senior
Python
Databases
Neo4j
AI/ML
AWS Bedrock
Function Calling
LLM
AI Agents
Context Engineering
Structured Outputs
Fine-tuning
GRPO
LLM Evaluation
SFT
DevOps
AWS
Apply
Report

Arthur

arthur.ai
Arthur unites security, finance, and the teams shipping AI on one control plane for AI you can trust. Measure, monitor, and govern enterprise AI at scale.
arthur.ai • New York • Peripherals • Data & Analytics • Artificial Intelligence
New York • Peripherals • Data & Analytics • Artificial Intelligence
Top 25% payVerified live · 3 hours ago 3 months ago

Forward Deployed Engineer

$170k – $220k per year • Remote/Hybrid • Full-Time
SQL
AI/ML
AI Agents
LLM Guardrails
LLM
LLM Evaluation
Apply
Report

Accellor

accellor.com
Accellor is an AI-first digital transformation and technology services firm headquartered in Frisco, Texas, and founded in 2018. The company develops purpose-built artificial intelligence solutions, including agentic AI for supply chain optimization, AI-assisted software engineering, and intelligent automation for sectors like healthcare and financial services. It operates globally with a focus on scaling AI from proof of concept to enterprise-wide production through strategic partnerships with major technology providers like Salesforce and Microsoft.
accellor.com • HQ: Frisco, United States • Health Care • Business Process Automation (BPA) • Software • AI Agents • LLM & Generative AI • Artificial Intelligence • Information Technology • IT Consulting • Est. 2018
HQ: Frisco, United States • Health Care • Business Process Automation (BPA) • Software • AI Agents • LLM & Generative AI • Artificial Intelligence • Information Technology • IT Consulting • Est. 2018
Verified live · 1 day ago 3 months ago

AI Principal Engineer

$162k – $346k per year (Estimated) • Lead • 10+ years expIn office (San Francisco, United States) • Principal
C++
Go
Java
Python
Rust
TypeScript
C++
PyTorch C++
TensorFlow C++
AI/ML
AI Agents
ChatGPT
CUDA Toolkit
Function Calling
JAX
LLM
Multimodal AI
PyTorch
Ray
Reinforcement Learning
TensorFlow
vLLM
CUDA
Triton
Context Engineering
LLM Evaluation
NCCL
OpenAI
OpenAI Codex
Post-training
Pre-training
Hallucination
RAG
DevOps
Kubernetes
Platform Engineering
CI/CD
Docker
Grafana
OpenTelemetry
Prometheus
Terraform
Apply
Report

Regard

regard.com
Regard is a clinical artificial intelligence company whose software reads a patient's full chart and suggests diagnoses that clinicians may have missed. Founded in 2017 in Los Angeles, it writes the supporting note text directly into the electronic health record. Hospitals use it to improve both patient safety and documentation accuracy.
regard.com • HQ: Los Angeles, United States • Medical AI • Artificial Intelligence • Est. 2017
HQ: Los Angeles, United States • Medical AI • Artificial Intelligence • Est. 2017
Top 25% payVerified live · 2 days ago 3 months ago

Full Stack Software Engineer

$160k – $190k per year • Middle • 3+ years expRemote/Hybrid (New York, United States) • Relocation • Bachelor's Degree • Full-Time • Middle
Go
JavaScript
Python
TypeScript
Python
Celery
FastAPI
Flask
Databases
PostgreSQL
Redis
AI/ML
Claude
Claude Code
Cursor
Langfuse
LLM
LLM Evaluation
OpenAI
OpenAI Codex
LangChain
Prompt Engineering
RAG
Anthropic
Frontend
React.js
DevOps
AWS
Kubernetes
CI/CD
Docker
GitLab CI
GitLab
Apply
Report

Vosker

vosker.com
Outdoor surveillance camera that works anywhere, no need for wi-fi or electricity! Solar powered and weather-resistant, Vosker cellular camera.
vosker.com • HQ: Victoriaville, Canada • Smart Home Devices • Consumer Goods • Security Products
HQ: Victoriaville, Canada • Smart Home Devices • Consumer Goods • Security Products
Verified live · 1 day ago 3 months ago

Machine Learning Engineer - Defendec/Reconeyez

Senior • 5+ years expIn office (Tallinn, Estonia) • Full-Time • Senior
Python
Databases
NATS
AI/ML
Computer Vision
Fine-tuning
Label Studio
LLM
PyTorch
TensorRT
vLLM
VLM
PEFT
Triton
LLM Evaluation
AI Agents
Anomaly Detection
Comet ML
LoRA
Multimodal AI
RAG
Triton Inference Server
Model Context Protocol
DevOps
gRPC
Apply
Report

Docker

docker.com
Docker is a leading software platform that enables developers to build, share, and run applications inside lightweight, isolated containers. By packaging code together with all its dependencies, the technology ensures that applications run consistently across any computing environment - from local development machines to large-scale cloud infrastructure. The platform accelerates modern software development, simplifying microservices architecture, automated deployment pipelines, and cross-team collaboration.
docker.com • HQ: Palo Alto, United States • Software • Information Technology • Virtualization • DevOps • Est. 2011
HQ: Palo Alto, United States • Software • Information Technology • Virtualization • DevOps • Est. 2011
Verified live · 2 hours ago 3 months ago

ML Engineer

$109k – $235k per year (Estimated) • Senior • 5+ years expPalo AltoSeattle • Remote (United States, Canada) • Visa sponsorship • Bachelor's Degree • Full-Time • Senior
AI Agents
Anomaly Detection
Fine-tuning
LLM
Prompt Engineering
Feature Store
LLM Evaluation
LLM Guardrails
Model Context Protocol
DevOps
Docker
Platform Engineering
Apply
Verified live · 2 hours ago 3 months ago

Staff ML Engineer

$168k – $300k per year (Estimated) • Staff+ • 8+ years expPalo AltoSeattle • Remote (United States, Canada) • Visa sponsorship • Bachelor's Degree • Full-Time • Staff
AI Agents
Anomaly Detection
Fine-tuning
LLM
Prompt Engineering
Feature Store
LLM Evaluation
LLM Guardrails
Model Context Protocol
DevOps
Docker
Platform Engineering
Apply
Report

Trend Micro

trendmicro.com
Trend Micro is a global cybersecurity software company headquartered in Tokyo, Japan, and founded in 1988. The company develops enterprise security software for cloud environments, networks, and endpoints, including threat intelligence and risk management platforms. It operates as a publicly listed entity with 62 global offices, serving thousands of enterprise organizations across 185 countries.
trendmicro.com • HQ: Tokyo, Japan • Cybersecurity AI • Network Security • Endpoint Security • Cloud Security • Threat Intelligence • Artificial Intelligence • Cybersecurity • 1001-5000 employees • Est. 1988
HQ: Tokyo, Japan • Cybersecurity AI • Network Security • Endpoint Security • Cloud Security • Threat Intelligence • Artificial Intelligence • Cybersecurity • 1001-5000 employees • Est. 1988
Verified live · 1 day ago 3 months ago

(Sr.) Applied AI Engineer

Senior • In office (Taipei, Taiwan) • Full-Time • Senior
C++
Go
Java
Python
AI/ML
AI Agents
Claude
Claude Code
Fine-tuning
Hallucination
LLM
Context Engineering
Edge AI
LLM Evaluation
SFT
Function Calling
CUDA
CUDA Toolkit
TensorRT
Triton
NVIDIA NeMo
DevOps
AWS
Azure
Docker
GCP
Kubernetes
Terraform
Apply
Report

Marble

marble.onl
Andrew Marble. Evaluating AI Agents as Products, Aug 15, 2026.
marble.onl • Zurich • Artificial Intelligence • LLM & Generative AI
Zurich • Artificial Intelligence • LLM & Generative AI
$118k – $250k per year (Estimated) • Executive • Remote/Hybrid • Cofounder • Architect
Python
TypeScript
JavaScript
Python
Django
FastAPI
Flask
Databases
pgvector
PostgreSQL
Supabase
AI/ML
CNN
Computer Vision
LLM
RAG
AI Agents
Human-in-the-Loop
LLM Evaluation
Structured Outputs
Frontend
Angular
Mobile
Ionic
DevOps
AWS
Cloudflare
Apply
Report

Minerva

minerva.ai
minerva.ai • New York
New York
Top 25% payVerified live · 2 days ago 3 months ago

Staff Data Engineer

$200k – $250k per year • Staff+ • 5+ years expIn office (New York, United States) • Relocation • Full-Time • Staff
Python
SQL
Databases
Amazon Redshift
ElasticSearch
Google BigQuery
MySQL
PostgreSQL
Snowflake
Apache Kafka
AI/ML
AI Agents
dbt
LLM Guardrails
Model Context Protocol
OpenAI
Claude
Claude Code
Cursor
Flink
LLM
Spark
LLM Evaluation
OpenAI Agents SDK
DevOps
Vector
Analytics
ETL/ELT
Marketing
HubSpot
Meta Ads
Apply
Report

Integrate

integrate.co
The first program management platform deployed on JWICS. Run complex defense programs across every org, contractor, and clearance level, live.
integrate.co • Seattle • Washington • Government • Project Management • Military • 51-200 employees • Est. 2022
Seattle • Washington • Government • Project Management • Military • 51-200 employees • Est. 2022
Verified live · 4 hours ago 3 months ago

Agentic Operations Engineer

$135k – $175k per year • Middle • 3+ years expIn office (Seattle, United States) • Full-Time • Middle
Go
JavaScript
Python
TypeScript
Databases
PostgreSQL
AI/ML
Edge AI
AI Agents
Model Context Protocol
ChatGPT
Claude
Claude Code
LangChain
LLM
Prompt Engineering
RAG
Anthropic
LLM Evaluation
OpenAI
Function Calling
Frontend
GraphQL
React.js
Tailwind CSS
DevOps
AWS
Azure
GCP
GitHub Actions
Vector
IAM
GitHub
Cybersecurity
Least Privilege
Apply
Report

Userpilot

userpilot.com
Userpilot provides product experience software for onboarding and feature adoption. Teams build in-app flows, surveys and checklists without engineering effort. Its analytics tie guidance directly to product usage outcomes.
userpilot.com • HQ: Austin, United States • Data & Analytics • Product Analytics • Artificial Intelligence • Est. 2018
HQ: Austin, United States • Data & Analytics • Product Analytics • Artificial Intelligence • Est. 2018
$49k – $168k per year (Estimated) • Middle • 3+ years expIn office (Dublin, Ireland) • Visa sponsorship • Full-Time • Middle
Elixir
JavaScript
Python
TypeScript
Python
FastAPI
Pydantic
Databases
Apache Kafka
ClickHouse
AI/ML
AI Agents
Claude
Claude Code
Cursor
LLM
RAG
LangChain
Function Calling
LLM Evaluation
Model Context Protocol
Structured Outputs
Anomaly Detection
AutoGen
Hybrid Search
Langfuse
LangGraph
Time Series Forecasting
Frontend
React.js
DevOps
Vector
WebSockets
OpenTelemetry
Apply
Report

Steerhealth

steerhealth.io
Steer Health deploys 7 specialized AI agents managing the full patient journey across 472+ locations and 109 organizations. Meet the team building the AI healthcare workforce.
steerhealth.io • Dallas • Irving • AI Agents • Artificial Intelligence • Health Care • Est. 2021
Dallas • Irving • AI Agents • Artificial Intelligence • Health Care • Est. 2021
Verified live · 1 day ago 3 months ago

Senior QA Engineer

$20k – $50k per year (Estimated) • Senior • 5+ years exp • Remote (India) • Contractor • Senior
JavaScript
Node JS
TypeScript
AI/ML
Claude
ElevenLabs
LLM
Anthropic
Text-to-Speech
Braintrust
Promptfoo
LLM Evaluation
DevOps
CI/CD
GCP
GitHub Actions
Google Cloud Run
Google GKE
gRPC
WebSockets
Kubernetes
GitHub
Datadog
OpenTelemetry
QA
Playwright
Apply
Report

Future

future.ai
Open 145 daysTop 25% pay 5 months ago

Applied AI Engineer

$215k – $250k per year • Equity • Remote (United States)
Python
Python
Pydantic
AI/ML
AI Agents
Function Calling
LangChain
Langfuse
LangGraph
LLM
Prompt Engineering
LLM Evaluation
Structured Outputs
AWS Bedrock
DevOps
Terraform
AWS
Datadog
OpenTelemetry
Amazon S3
Apply
Report

Path Robotics

path-robotics.com
Path Robotics is a company founded in 2014 in Columbus, Ohio that builds autonomous welding robots for manufacturers facing a shortage of skilled welders. Its systems scan each part with computer vision, plan the weld path themselves and adapt to variation instead of following a pre-programmed trajectory. The technology is aimed at high-mix, low-volume fabrication that conventional robots cannot handle.
path-robotics.com • HQ: Columbus, United States • Est. 2014
HQ: Columbus, United States • Est. 2014
Open 115 daysVerified live · 2 days ago 4 months ago

Senior Software Engineer, Perception Platform

$134k – $242k per year (Estimated) • Senior • 5+ years expIn office (Columbus, United States) • Internship • Senior
C++
Python
C++
Protobuf
PyTorch C++
AI/ML
CUDA
CUDA Toolkit
PyTorch
Computer Vision
LLM Evaluation
DevOps
CI/CD
gRPC
Apply
Report
LLM Evaluation Jobs - Remote & On-site
Frequently asked questions
How many LLM Evaluation jobs are available on Alion?
There are currently 491 active listings from 297 hiring companies on Alion, refreshed daily.
What salary do LLM Evaluation jobs pay?
The average salary is around $230,281 per year based on the current listings.
Are remote options available for LLM Evaluation jobs?
Yes, many of these roles offer remote or hybrid work, and you can filter listings by remote, relocation and visa sponsorship.
How do I apply for LLM Evaluation jobs?
Open any listing on Alion and apply directly with the employer through the provided link - no lengthy sign-up required.