Member of Technical Staff (Evals)

Jarmin • Equity 0.2–2% • 4 days ago
Apply now
Save
368,657open jobs
9,442companies
50,883added this week
Browse all
Salary
$130k – $180k per year
Location
In office (San Francisco)
Seniority
Staff · 3+ years exp
Employment
Full-Time
Overview
Company
Impact
Profile match

Jarmin

Hire AI employees for human-level roles — paralegal, financial analyst, ML engineer, and more. Jarmin learns your systems and does the work to production quality, embedded in your team. Backed by Y Combinator.
jarmin.ai • San Francisco • UTC-08:00 • Human Resources • Artificial Intelligence • AI Agents • 11-50 employees

As a Member of Technical Staff on Evals, you'll own how we measure AI Employee performance on the job.

Areas you may work in:

  • Evaluation systems: datasets, offline replay, scorers, and regression alerts
  • Feedback loops for models and agents
  • Autonomous failure analysis
  • Defining what good, degraded, and failed runs look like, and alerting on them

You may be a good fit if you:

  • Have built evaluation or measurement systems, such as AI evals, experimentation, ranking, or search quality
  • Can turn ambiguous quality questions into concrete metrics and decisions
  • Have strong software engineering fundamentals and ship production systems
  • Are comfortable where there are few established patterns
  • Want to work in person in San Francisco

Even better:

  • You've shipped LLM systems to production and learned where they break
  • You have strong data instincts and work well with researchers
  • You contribute to open source projects
Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
368,657 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Similar stack
Same company
San Francisco
$152k – $239k per year • Remote • Full-Time • 8+ years exp
SQL
Databases
Snowflake
AI/ML
LLM
Model Context Protocol
Analytics
A/B Testing
Apply
≈ $69k – $171k per year (Estimated) • Remote/Hybrid • Full-Time • 3+ years exp • Bachelor's Degree • Bogotá
SQL
Databases
Databricks
AI/ML
dbt
LLM
NLP
Context Engineering
Analytics
Power BI
Tableau
Apply
UI/UX Designer 1 day ago
≈ $12k – $55k per year (Estimated) • Remote • Full-Time
AI/ML
Claude
LLM
LLM Guardrails
Lovable
Design
Figma
Apply
≈ $20k – $49k per year (Estimated) • Remote • Full-Time • Bachelor's Degree
Python
Ruby
SQL
Databases
Amazon Neptune
Neo4j
AI/ML
Hallucination
LangChain
LangGraph
LLM
Model Context Protocol
Spark
AI Agents
LLM Guardrails
DevOps
Ansible
AWS
Azure
CI/CD
Docker
GCP
GitHub Actions
GitLab CI
Jenkins
Kubernetes
Rest API
Terraform
GitHub
GitLab
QA
Playwright
Postman
Selenium
Swagger
Apply
≈ $42k – $107k per year (Estimated) • Remote/Hybrid • Full-Time • 8+ years exp • Mexico City
C#
Go
Java
Rust
AI/ML
AI Agents
LLM
Model Context Protocol
Cybersecurity
Delinea
Zero Trust
Apply
$170k – $220k per year • Equity 1–2.8% • In office • Full-Time • 3+ years exp • San Francisco
Python
SQL
Python
Django
AI/ML
AI Agents
Context Engineering
LLM
LLM Evaluation
RAG
Apply
$173k – $314k per year • In office • Full-Time • 12+ years exp • Bachelor's Degree • San Francisco
Apex
JavaScript
Node JS
Python
SQL
TypeScript
Apex
Lightning Web Components
AI/ML
Agentforce
AI Agents
Claude
Claude Code
Copilot
Cursor
LLM
RAG
DevOps
AWS
Azure
CI/CD
Docker
GCP
GitHub
Grafana
gRPC
Kubernetes
New Relic
Prometheus
Splunk
Marketing
Salesforce
QA
Cypress
JMeter
k6
Locust
Playwright
Postman
Rest-Assured
Selenium
Apply
Senior ML Engineer 2 hours ago
$149k – $224k per year • In office • Full-Time • 5+ years exp • Master's Degree • San Francisco • Washington • Palo Alto
Python
Python
pySpark
Databases
Apache Kafka
AI/ML
AI Agents
Agentforce
Airflow
Anomaly Detection
Feature Store
Flink
Ray
Red Teaming
Spark
DevOps
CI/CD
Docker
Kubernetes
Cybersecurity
MITRE ATT&CK
Marketing
Salesforce
Apply
In office • Internship • 1+ year exp • Bachelor's Degree • San Francisco
Go
JavaScript
Ruby
Scala
Apply
$360k – $530k per year • In office • Full-Time • Bachelor's Degree • San Francisco
MATLAB
Python
MATLAB
Simulink
AI/ML
OpenAI
Robotics
Digital Twin
Apply
See all jobs
This is one of many
368,657 more open roles from verified company boards, updated every day.
Browse all jobs Get matched roles