Everyday AI Agent Tester - Remote

YO AI Labs · Bachelor's Degree · 21 days ago

Smart Apply

Similar jobs

2,058,002open jobs

123,297companies

240,284added this week

Browse all

Salary

≈ $100k – $204k per year (Estimated)

Location

In office (New York)

Seniority

Junior · 2+ years exp

Confirmed on the employer's own hiring board on Nov 2, 2026. First seen by Alion on Oct 12, 2026.

Overview

Company

Impact

Profile match

YO AI Labs

YO AI Labs is an artificial intelligence platform specializing in AI model training, evaluation, and data annotation services. Headquartered in San Francisco, California, United States, the company connects domain experts and remote specialists across the globe to review, test, and benchmark generative AI model outputs. Through its decentralized workforce solutions and structured evaluation workflows, it enables frontier technology developers to enhance the accuracy, safety, and reliability of next-generation artificial intelligence systems.

yoailabs.com · HQ: San Francisco, United States · AI Evaluation & Observability · AI Training Data & Annotation · 5000+ employees

Tech stack of YO AI Labs

Role Title: Everyday AI Agent Tester

Role Type: Contractor

Location: United States

We are seeking detail-oriented, U.S.-based professionals to test AI agents performing everyday tasks. You'll evaluate two AI agents side by side, document their performance, and provide feedback to improve user experience and reliability.

Key Responsibilities

Test AI agents on everyday tasks, including bookings, shopping, travel, emails, bill payments, and forms.
Approve or stop agents at key decision points.
Screen-record each testing session.
Evaluate task completion, quality, user control, satisfaction, and time.
Provide clear written feedback on agent performance and usability.

Required Qualifications

Based in the United States with a bachelor's degree.
Regularly use AI tools in daily life.
2-3 years of experience, preferably in engineering, business management, or office operations.
Strong attention to detail and ability to follow instructions.
Excellent written communication and documentation skills.
An active personal AI account (Free, Go, or Plus; not Pro) and a smartphone capable of screen recording.

Preferred: Domain expertise in relevant everyday workflows. No coding or AI development experience is required.

Selection Process

Submit an application and complete screening questions.
Attend a brief interview (15-30 minutes).
Complete profile review and onboarding.

Availability: Onboarding is scheduled before the project kickoff, with tasks expected to begin on Monday, October 12.

Free account

Stop reading job ads. Get the ones that fit.

One free account turns this page into a shortlist built around your stack, your level and your pay.

Match on every job. Stack, seniority, pay and location, scored against your profile.

2,058,002 open roles. Read straight off company career pages, refreshed every day.

Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.

3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.

Create a free account Continue with Google

Free forever. No card. Under a minute.

Your match

How well do you fit this role?

Two answers are enough for a real match. No account needed.

Check my fit

Answers stay in this browser until you create an account.

Recommended for you based on this role

Similar stack

Same company

New York

Junior Software Development Engineer in Test 3 hours ago

In office · Full-Time · 3+ years exp · Bachelor's Degree · Riverton

Python

JavaScript

TypeScript

SQL

.NET

Databases

MS SQL

AI/ML

Copilot

Claude Code

DevOps

Rest API

Azure DevOps

Azure

CI/CD

Git

Selenium

Cypress

Playwright

Apply