{"id":2213449,"url":"https://alion.io/job/micro1-ai-agent-evaluator","title":"AI Agent Evaluator","company":{"id":13196,"name":"Micro1","domain":"micro1.ai","url":"https://alion.io/company/micro1","size_band":"11-50","is_staffing_agency":false,"employer_type":"direct","is_intermediary":true,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Operations","role_family":"Operations","seniority":null,"employment_type":"contractor","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":[],"countries":[],"hiring_countries":["US"],"hiring_countries_total":1,"salary":{"min":25,"max":45,"currency":"USD","period":"hour","gross":null,"usd_annual":90000},"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false}],"status":"live","first_seen_at":"2026-10-10T04:17:05Z","employer_posted_date":null,"last_verified_at":"2026-10-10T04:17:05Z","board_verified":false,"closed_at":null,"days_open":1,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":1},"description":"Role Title: AI Agent EvaluatorRole Type: ContractorLocation: U.S based Opportunity SnapshotPay: $25-45/hour\nTime per task: ~1.5 hours per run (evidence and scoring included)\nProject kickoff: Monday, October 12\nOnboarding: Today (Fri), Saturday, or Sunday, so you’re ready to start Monday October 10th \nYou’ll need: An active personal popular AI account (Free, Go, or Plus; not Pro), an active personal and a smartphone that can screen record\nYour tasks: Run everyday errands with two AI agents, approve key steps, screen record, and score the results\nAbout the roleAI agents are starting to handle real-life errands: paying bills, booking trips, doing the groceries. We want to know how well they actually do it, from the point of view of a real user. That’s you.\nYou’ll test two AI agents side by side, using your own accounts on your phone, and tell us what worked, what didn’t, and how it felt.\nWhat you’ll do\nRun real-life scenarios: bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms\n\nApprove or stop the agent at key steps\n\nScreen record every attempt\n\nScore each run on completion, quality, control issues, satisfaction, and time\n\nShare clear written feedback on the agent experience\n\nWho we’re looking for\n\nUS-based, with a Bachelor’s degree\n\nSomeone who already uses AI in daily life\n\n2-3 years of experience, preferably in engineering, business management, or office/operations\n\nDetail-oriented, great at following instructions and documenting each step\n\nClear written communication\n\nGeneralists welcome, domain experts preferred\n\nNo coding or AI background needed\n\nWhy you’ll like it\nGet paid to use the AI tools you already use\n\nWork entirely from your phone, on your schedule\n\nHave a direct say in how next-generation AI agents get built and improved\n\nHow the selection process works\nApply\n\nComplete the screening questions\n\nInterview: 15-30 minutes max\n\nProfile review: if everything looks good, we present you to the client\n\nOnboarding: our Talent Ops team will contact you to get you started\n\nOriginally posted on Himalayas","description_format":"text","description_chars":2098,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Recruiting & Staffing","Freelance & Talent Marketplaces","AI Training Data & Annotation"],"lifecycle":[{"event":"open","at":"2026-10-10T05:28:30Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":7,"reasons":["seen:0","win:early"],"computed_at":"2026-10-10T05:45:15Z"},"pay":{"stated_usd_annual":90000,"is_top_pay":false},"html_url":"https://alion.io/job/micro1-ai-agent-evaluator","json_url":"https://alion.io/job/micro1-ai-agent-evaluator.json","meta":{"generated_at":"2026-10-11T04:32:08Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler","counted_by":"address","units_charged":1,"used_today":3282,"day_limit":5000,"remaining_today":1718,"minute_limit":60,"resets_at":"2026-10-12T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":13196},"rest":"https://alion.io/mcp/rest/get_company?id=13196"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fmicro1-ai-agent-evaluator"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fmicro1-ai-agent-evaluator"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fmicro1-ai-agent-evaluator"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/micro1-ai-agent-evaluator\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fmicro1-ai-agent-evaluator"}]}