LLM Evaluation Jobs - Remote & On-site

LLM Evaluation jobs hiring now at vetted startups and product companies on Alion, updated daily. Average salary around $230,281/year. Compare remote, hybrid and on-site positions with relocation and visa sponsorship. Browse and apply today.

AI/ML
Data Science
Backend
Frontend
Mobile
DevOps
Web3
Games
Hardware
Robotics
Security
QA
Executive
Networking
Product
Design
Analytics
Support
Enterprise Apps
Quantum

Pretius

pretius.com
Pretius is a software development company that specializes in building custom enterprise applications, low-code systems, and complex IT integrations. The firm focuses heavily on Oracle APEX, custom Java development, and system architecture to help organizations digitize and streamline critical business processes. By delivering tailored software solutions, it enables medium to large enterprises across various industries to optimize performance and modernize legacy IT infrastructure.
pretius.com • HQ: Warsaw, Poland • Information Technology • IT Outsourcing • Software
HQ: Warsaw, Poland • Information Technology • IT Outsourcing • Software
2 months ago

AI Engineer

$64k – $76k per year • Middle • 4+ years expWarsaw • Remote (Poland) • Full-Time • Middle
Python
SQL
C#
C#
ASP.NET Core
AI/ML
Hallucination
Hybrid Search
LangChain
LangGraph
LLM
RAG
Semantic Kernel
Human-in-the-Loop
LLM Evaluation
LLM Guardrails
OpenAI
Structured Outputs
AI Agents
A2A
Model Context Protocol
Frontend
GraphQL
DevOps
Azure
CI/CD
Docker
Git
Vector
Apply
Report

Axabee

axabee.com
Axabee is a Polish software development agency and digital product consultancy headquartered in Wrocław and Opole, Poland. Founded in 2012, the firm specializes in custom software engineering, AI-driven solutions, and technology platforms, with deep expertise in the travel, hospitality, and e-commerce sectors.
axabee.com • Wrocław • Opole • Artificial Intelligence • Mobile App Development • Information Technology
Wrocław • Opole • Artificial Intelligence • Mobile App Development • Information Technology
2 months ago

Senior AI Engineer

$66k – $109k per year (Estimated) • Senior • 5+ years expWrocław • Remote (Poland) • Full-Time • Senior
Java
Node JS
Python
TypeScript
JavaScript
Python
FastAPI
Databases
Neo4j
AI/ML
AI Agents
Hallucination
LangChain
LlamaIndex
LLM
Multimodal AI
NLP
Prompt Engineering
RAG
Semantic Search
Anthropic
Knowledge Graph
LLM Evaluation
OpenAI
Semantic Search
Fine-tuning
Knowledge Distillation
DevOps
AWS
Azure
Docker
GCP
Kubernetes
Rest API
CI/CD
Analytics
A/B Testing
Apply
Report

Fluidstack

fluidstack.io
Fluidstack is an AI cloud platform that designs, builds, and operates high-performance GPU clusters for frontier AI laboratories, enterprises, and governments. The company provides enterprise-grade bare-metal compute infrastructure - scaled across tens of thousands of state-of-the-art NVIDIA GPUs - specifically optimized for training large language models (LLMs) and running high-throughput inference.
fluidstack.io • HQ: New York, United States • Information Technology • Hardware • AI Infrastructure • Data Centers • Cloud Computing • 501-1000 employees
HQ: New York, United States • Information Technology • Hardware • AI Infrastructure • Data Centers • Cloud Computing • 501-1000 employees
Top 25% payVerified live · 1 hour ago 2 months ago

Machine Learning Engineer

$269k – $317k per year • In office (Austin, New York, San Francisco, Seattle, United States) • Full-Time
Fine-tuning
LLM
LLM Evaluation
LLM Guardrails
AI Agents
Model Context Protocol
Apply
Report

Twilio

twilio.com
Twilio is a leading cloud communications platform that enables software developers to programmatically build, scale, and integrate real-time communication capabilities into applications via application programming interfaces (APIs). Headquartered in San Francisco, California, the company provides infrastructure for automated SMS messaging, voice calls, WhatsApp integration, video, and two-factor authentication without requiring traditional telecom hardware.
twilio.com • HQ: San Francisco, United States • Telecommunications • Software • Online Communication • 1001-5000 employees • Est. 2008
HQ: San Francisco, United States • Telecommunications • Software • Online Communication • 1001-5000 employees • Est. 2008
Verified live · 3 hours ago 2 months ago

Senior Technical Program Manager, Services Tools & Diagnostics

$121k – $207k per year (Estimated) • Equity • Senior • 7+ years exp • Remote (United States) • Senior
Node JS
Python
SQL
TypeScript
JavaScript
Databases
ClickHouse
LookML
AI/ML
Claude
Claude Code
Function Calling
Model Context Protocol
Prompt Engineering
RAG
Anthropic
LLM Evaluation
LLM Guardrails
OpenAI Agents SDK
Structured Outputs
Mobile
Twilio
DevOps
Git
Grafana
Platform Engineering
Rest API
Cybersecurity
Least Privilege
Analytics
Tableau
A/B Testing
Management
Airtable
Jira
Slack
Marketing
Salesforce
Zendesk
Apply
Report

Sonatus

sonatus.com
Sonatus is an automotive technology company headquartered in Sunnyvale, California, and was founded in 2018. The firm provides the Fastlane platform, a suite of cloud-to-edge solutions including data collection, edge computing, and AI-driven diagnostics designed to power software-defined vehicles. It operates globally by partnering with major automotive manufacturers and suppliers to implement vehicle architectures that facilitate data management and lifecycle optimization.
sonatus.com • HQ: Sunnyvale, United States • Smart Mobility • Transportation & Logistics • Automotive Manufacturing • Manufacturing • Embedded Software • Software • Edge AI • Artificial Intelligence • 201-500 employees • Est. 2018
HQ: Sunnyvale, United States • Smart Mobility • Transportation & Logistics • Automotive Manufacturing • Manufacturing • Embedded Software • Software • Edge AI • Artificial Intelligence • 201-500 employees • Est. 2018
Verified live · 1 day ago 2 months ago

Senior Manager, Engineering - AI Validation

$112k – $256k per year (Estimated) • Lead • 10+ years expIn office (Sunnyvale, United States) • Bachelor's Degree • Internship • Senior
Python
AI/ML
Embeddings
Hallucination
LangGraph
LLM
RAG
LangChain
LLM Evaluation
LLM Guardrails
AI Agents
DevOps
Jenkins
Vector
QA
Playwright
Pytest
Apply
Report

Rakuda AI

rakuda-ai.com
AIを活用したソフトウェア開発と業務自動化に特化した会社。ファンド業務の自動化から AI活用SaaS の受託開発まで、少数精鋭・高速デリバリーで提供します。.
rakuda-ai.com • Tokyo • Business Process Automation (BPA) • Software • Artificial Intelligence
Tokyo • Business Process Automation (BPA) • Software • Artificial Intelligence
$186k per year • Lead • 8+ years expIn office (Tokyo, Japan) • Full-Time • Principal • Japanese
TypeScript
JavaScript
AI/ML
Claude
Cohere SDK
Cursor
DSPy
Fine-tuning
Knowledge Distillation
LangChain
LangGraph
LLM
Mistral
Perplexity
Prompt Engineering
RAG
Reranking
RLHF
vLLM
Anthropic
DPO
LLM Evaluation
LLM Guardrails
OpenAI
SFT
AI Agents
Function Calling
Frontend
Next.js
React.js
DevOps
AWS
GCP
Vector
Apply
Report

Dialpad

dialpad.com
Dialpad is an AI-native cloud communications and customer experience platform. Headquartered in San Francisco, California, the company provides unified communications and contact center solutions that combine voice calling, business text messaging (SMS/MMS), video conferencing, and contact center operations into a single platform.
dialpad.com • Vancouver • Austin • Denver • Tempe • Kitchener • Customer Relationship Management (CRM) • Conversational AI • Unified Communications • Est. 2011
Vancouver • Austin • Denver • Tempe • Kitchener • Customer Relationship Management (CRM) • Conversational AI • Unified Communications • Est. 2011
Verified live · 27 min ago 2 months ago

Lead Product Manager

$157k – $283k per year (Estimated) • Lead • 2+ years expIn office • Full-Time • Staff
AI Agents
Quantization
Speech Recognition
Edge AI
LLM Evaluation
Text-to-Speech
Apply
Verified live · 27 min ago 3 months ago

AI Engineer, Agentic Voice (TTS)

$96k – $203k per year (Estimated) • Middle • 3+ years expRemote/Hybrid (Kitchener, Canada) • Middle
Python
AI/ML
AI Agents
ElevenLabs
LLM
Prompt Engineering
PyTorch
Speech Recognition
Edge AI
LLM Evaluation
Text-to-Speech
DevOps
GCP
Analytics
A/B Testing
Apply
Verified live · 27 min ago 3 months ago

AI Engineer, Agentic Voice (TTS)

$88k – $186k per year (Estimated) • Middle • 3+ years expRemote/Hybrid (Vancouver, Canada) • Middle
Python
AI/ML
AI Agents
ElevenLabs
LLM
Prompt Engineering
PyTorch
Speech Recognition
Edge AI
LLM Evaluation
Text-to-Speech
DevOps
GCP
Analytics
A/B Testing
Apply
Report

Meesho

meesho.com
Meesho is an Indian e-commerce company headquartered in Bengaluru and founded in 2015. The company operates an online marketplace that enables small businesses and individual entrepreneurs to sell products such as apparel, home goods, and electronics directly to consumers. Originally established as a social commerce platform facilitating sales through networks like WhatsApp and Facebook, it has evolved into a large-scale retail ecosystem serving millions of users across India.
meesho.com • HQ: Bengaluru, India • B2B Commerce • Apparel & Fashion • Consumer Goods • Commerce • Marketplaces • Social Commerce • Est. 2015
HQ: Bengaluru, India • B2B Commerce • Apparel & Fashion • Consumer Goods • Commerce • Marketplaces • Social Commerce • Est. 2015
2 months ago

AI Engineer

$27k – $111k per year (Estimated) • Remote (India)
Python
Databases
pgvector
Pinecone
Qdrant
Weaviate
PostgreSQL
AI/ML
Claude
Claude Code
Copilot
Cursor
Fine-tuning
LangChain
LlamaIndex
LLM
NLP
RAG
LLM Evaluation
Structured Outputs
DevOps
AWS
Azure
GCP
Vector
GitHub
Apply
Report

Humana

humana.com
Humana is a major American health insurance provider and healthcare services company headquartered in Louisville, Kentucky. Founded in 1961, the company is one of the largest managed care organizations in the U.S., specializing primarily in senior-focused care delivery, individual and group Medicare Advantage plans, Medicaid programs, and supplemental insurance products (dental, vision, and pharmacy benefits).
humana.com • HQ: Louisville, United States • Financial Services • Health Care • Insurance • 1001-5000 employees • Est. 1961
HQ: Louisville, United States • Financial Services • Health Care • Insurance • 1001-5000 employees • Est. 1961
Verified live · 1 hour ago 2 months ago

Applied AI Engineer

$142k – $196k per year • Senior • 5+ years expIn office (Park, United States) • Relocation • Bachelor's Degree • Full-Time • Senior
Python
Python
FastAPI
Flask
Pydantic
Databases
Apache Kafka
AI/ML
Claude
Claude Code
Copilot
Cursor
Function Calling
LLM
RAG
AI Agents
LangChain
LangSmith
LlamaIndex
Pydantic AI
LLM Evaluation
Arize Phoenix
DevOps
AWS
Azure
CI/CD
Docker
GCP
Git
Kubernetes
Trunk-Based Development
GitHub
OpenTelemetry
Pulumi
Terraform
Apply
Report

Hostelworld

hostelworld.com
Hostelworld is an Irish travel technology company founded in 1999 and based in Dublin. Its platform books hostel accommodation in thousands of cities. The company is listed on the London Stock Exchange.
hostelworld.com • HQ: Dublin, Ireland • Outdoor Sports • Travel & Tourism • Luggage & Bags • Est. 1999
HQ: Dublin, Ireland • Outdoor Sports • Travel & Tourism • Luggage & Bags • Est. 1999
Verified live · 7 hours ago 2 months ago

Machine Learning Engineer

Remote/Hybrid (Porto, Portugal) • Full-Time
Python
SQL
Databases
Google BigQuery
PostgreSQL
Redis
AI/ML
Embeddings
Fine-tuning
Function Calling
Knowledge Distillation
LangChain
LangGraph
LlamaIndex
LLM
RAG
Synthetic Data
Vertex AI
LLM Evaluation
LLM Guardrails
Structured Outputs
AI Agents
Model Context Protocol
Recommender Systems
DevOps
CI/CD
GCP
Kubernetes
Vector
Analytics
A/B Testing
Apply
Report

ConveGenius

convegenius.com
ConveGenius is an education management company that provides services related to sustainable education for K-10 categories through impact-driven programs, an adaptive learning platform, and data dashboards.
convegenius.com • HQ: Noida, India • Primary Education • Est. 2013
HQ: Noida, India • Primary Education • Est. 2013
Verified live · 7 hours ago 2 months ago

Sr. ML Engineer- Generative AI

$53k – $74k per year • Senior • In office • Full-Time • Senior
Python
AI/ML
Fine-tuning
LLM
RAG
PEFT
LLM Evaluation
AI Agents
DeepSpeed
LangChain
LangGraph
LoRA
ONNX
QLoRA
DPO
FSDP
SFT
Function Calling
Apply
Report

Ollion

ollion.com
ollion is the enterprise tech consultancy that's all in on your future. our global team of ~600 employees around the world is solving the kind of business problems you can actually put a name to, working together to untangle complex challenges on ...
ollion.com • Pune • Seattle • Chicago • Logistics • Clinical Research • Government
Pune • Seattle • Chicago • Logistics • Clinical Research • Government
$33k – $78k per year (Estimated) • Lead • 6+ years expPune • Remote (India) • Contractor • Senior
Go
AI/ML
AI Agents
Function Calling
Gemini
LLM
Prompt Engineering
Vertex AI
Human-in-the-Loop
LLM Evaluation
Google ADK
DevOps
GCP
IAM
Apply
Report

Wooqer

wooqer.com
Wooqer is the AI-powered operations platform for multi-location retail, restaurant and banking chains — run audits, checklists and frontline execution at scale.
wooqer.com • Bengaluru • Retail & Shop Systems • Software • Business Process Automation (BPA) • Est. 2014
Bengaluru • Retail & Shop Systems • Software • Business Process Automation (BPA) • Est. 2014
$30k – $76k per year (Estimated) • Senior • In office (Bengaluru, India) • Senior
Python
SQL
Databases
Redis
AI/ML
Embeddings
Hallucination
LangChain
LangGraph
LLM
Prompt Engineering
RAG
LLM Evaluation
LLM Guardrails
Structured Outputs
AI Agents
DevOps
Docker
Vector
Apply
Report

Shortcut

shortcut.com
Shortcut is project management software built for software teams. Plan, build, and ship products with intuitive tools for Stories, Epics, Sprints, and Roadmaps.
shortcut.com • HQ: New York, United States • AI Agents • Est. 2014
HQ: New York, United States • AI Agents • Est. 2014
Verified live · 2 days ago 2 months ago

Senior Product Engineer

Senior • 5+ years expIn office • Full-Time • Senior
JavaScript
TypeScript
Databases
PostgreSQL
AI/ML
LLM
AI Agents
Anthropic
LLM Evaluation
LLM Guardrails
OpenAI
DevOps
AWS
Apply
Report

Arcana

arcana.io
Arcana enables institutional investors to understand their portfolio risks, decompose single stock & book performance, drill into crowding, and isolate their idiosyncratic differentiation. Built on our proprietary crowding, ownership, factor risk, and performance datasets.
arcana.io • HQ: New York, United States • Financial Services • Risk Management • Asset Management • 501-1000 employees • Est. 2022
HQ: New York, United States • Financial Services • Risk Management • Asset Management • 501-1000 employees • Est. 2022
2 months ago

Backend Engineer - AI

$22k – $95k per year (Estimated)In office (Bengaluru, India)
Go
Java
Python
Databases
Apache Kafka
PostgreSQL
AI/ML
Gemini
LangChain
LlamaIndex
LLM
NLP
Anthropic
LLM Evaluation
OpenAI
Structured Outputs
DevOps
Docker
Apply
Open 169 daysVerified live · 1 day ago 6 months ago

Senior AI Engineer — Inference & Agent Systems

$136k – $265k per year (Estimated) • Senior • In office • Senior
Python
Databases
Apache Kafka
PostgreSQL
AI/ML
AI Agents
Gemini
LangChain
LlamaIndex
LLM
Anthropic
LLM Evaluation
OpenAI
Structured Outputs
DevOps
Docker
Apply
Open 180 daysVerified live · 1 day ago 6 months ago

Senior AI Engineer — Inference & Agent Systems

$31k – $77k per year (Estimated) • Senior • Remote/Hybrid (Bengaluru, India) • Senior
Python
Databases
Apache Kafka
PostgreSQL
AI/ML
AI Agents
Gemini
LangChain
LlamaIndex
LLM
Anthropic
LLM Evaluation
OpenAI
Structured Outputs
DevOps
Docker
Apply
Report

Madhive

madhive.com
Our mission is to empower our clients to create enduring relationships with consumers through maximum-fidelity, TV-first technology that elevates media, embodies relentless innovation, and ensures unrivaled revenue acceleration.
madhive.com • New York • Data Entry & Processing
New York • Data Entry & Processing
Top 25% payVerified live · 1 day ago 2 months ago

AI Code Engineering Lead

$200k – $225k per year • Lead • 10+ years expRemote/Hybrid (New York, United States) • Full-Time • Staff
LLM Evaluation
LLM Guardrails
AI Agents
DevOps
CI/CD
Apply
Report

Pismo

pismo.io
Pismo provides cloud-native core banking and card processing infrastructure. Its platform runs accounts, cards and payments for banks worldwide. The company was acquired by a global payments network.
pismo.io • HQ: São Paulo, Brazil • FinTech • Financial Services • Banking • 1001-5000 employees • Est. 2016
HQ: São Paulo, Brazil • FinTech • Financial Services • Banking • 1001-5000 employees • Est. 2016
Verified live · 19 min ago 2 months ago

Data Engineer

$69k – $173k per year (Estimated) • Middle • 3+ years expRemote/Hybrid (Singapore) • Bachelor's Degree • Full-Time • Middle
SQL
Python
Python
pySpark
AI/ML
Great Expectations
Hallucination
MLFlow
Prompt Engineering
RAG
LLM Evaluation
LLM Guardrails
Hadoop
LangChain
Spark
DevOps
Azure
Azure DevOps
CI/CD
Git
GitHub Actions
Vector
GitHub
Splunk
Analytics
Power BI
Tableau
ETL/ELT
Apply
Report

Deeplab

deeplab.ai
Empower your business with deep intelligenceto make better decisions.. We design and develop scalable ML infrastructure solutions employing cutting-edge technology.
deeplab.ai • Athens • Artificial Intelligence • Machine Learning
Athens • Artificial Intelligence • Machine Learning
Verified live · 2 days ago 2 months ago

R&D Machine Learning Engineer

Middle • 4+ years expAthens • Remote (Greece) • Middle • English
Python
AI/ML
Computer Vision
LLM
PEFT
LLM Evaluation
AI Agents
RAG
Fine-tuning
JAX
LoRA
MLFlow
NLP
PyTorch
TensorFlow
Time Series Forecasting
Weights & Biases
DevOps
CI/CD
Docker
Git
HPC
Apply
Report

Console

console.com
Console is the AI-native platform for IT service management (ITSM), built to help teams automate support and shift from doing the work to managing the system that does the work for them.
console.com • San Francisco • Managed IT Services • IT Management • Information Technology
San Francisco • Managed IT Services • IT Management • Information Technology
Top 25% payVerified live · 2 hours ago 2 months ago

Research Engineer

$200k – $350k per year • In office (San Francisco, United States) • Full-Time
Databricks
AI/ML
AI Agents
Cursor
DSPy
Fine-tuning
Knowledge Distillation
LoRA
QLoRA
Reinforcement Learning
PEFT
DPO
GRPO
LLM Evaluation
OpenAI
Post-training
SFT
DevOps
GitHub
Design
Figma
Management
Slack
Apply
Report

Ambient

ambient.ai
Ambient.ai is transforming physical security with computer vision intelligence, empowering security teams with automated threat detection and visual verification.
ambient.ai • New York • Redwood City • Boston • Los Angeles • Bengaluru • Data Science • Computer Vision • Artificial Intelligence • Est. 2017
New York • Redwood City • Boston • Los Angeles • Bengaluru • Data Science • Computer Vision • Artificial Intelligence • Est. 2017
$168k – $205k per year • Equity • Senior • 4+ years expIn office (Redwood City, United States) • Bachelor's Degree • Full-Time • Senior
Python
AI/ML
Computer Vision
LLM
Multimodal AI
Quantization
RAG
Triton Inference Server
vLLM
Edge AI
LLM Evaluation
AI Agents
CUDA Toolkit
Knowledge Distillation
ONNX
PyTorch
TensorRT
Human-in-the-Loop
NCCL
DevOps
Vector
Cybersecurity
SentinelOne
Management
ServiceNow
Apply
Report
LLM Evaluation Jobs - Remote & On-site
Frequently asked questions
How many LLM Evaluation jobs are available on Alion?
There are currently 491 active listings from 297 hiring companies on Alion, refreshed daily.
What salary do LLM Evaluation jobs pay?
The average salary is around $230,281 per year based on the current listings.
Are remote options available for LLM Evaluation jobs?
Yes, many of these roles offer remote or hybrid work, and you can filter listings by remote, relocation and visa sponsorship.
How do I apply for LLM Evaluation jobs?
Open any listing on Alion and apply directly with the employer through the provided link - no lengthy sign-up required.