{"id":3592726,"url":"https://alion.io/job/arimlabs-rl-data-qa-engineer","title":"RL Data QA Engineer","company":{"id":4493200,"name":"Arimlabs","domain":"arimlabs.ai","url":"https://alion.io/company/arimlabs","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"QA","role_family":"QA","seniority":null,"employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Ukraine"],"countries":["UA"],"hiring_countries":["UA"],"hiring_countries_total":1,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Bash","optional":false},{"name":"Docker","optional":false},{"name":"Linux","optional":false},{"name":"Python","optional":false},{"name":"Reinforcement Learning","optional":false},{"name":"Claude Code","optional":true},{"name":"Cursor","optional":true},{"name":"LLM","optional":true},{"name":"OpenAI Codex","optional":true},{"name":"Reward Modeling","optional":true}],"status":"live","first_seen_at":"2026-11-01T18:35:16Z","employer_posted_date":null,"last_verified_at":"2026-11-01T18:35:16Z","board_verified":false,"closed_at":null,"days_open":5,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":5},"description":"We're looking for a rigorous, technical RL Data QA Engineer to review the reinforcement-learning tasks built on our platform before they reach frontier AI labs.\n\nEvery task on our platform is a small software environment: instructions, a containerized environment, a reference solution, and tests that turn an AI agent's work into a reward. Before a task reaches you, it has been run many times against frontier coding agents and audited automatically. You are the expert human judgment that decides whether it is good enough to train and evaluate the next generation of AI. Is it correct, fair, challenging and impossible to game?\n\nWhat You'll Do\n\nReview RL tasks end to end: Examine instructions, environments, reference solutions and tests, and decide whether each task meets our quality bar.\n\nBreak the verifier: Look for the ways an agent could pass without really solving the task, such as weak tests, leaked solutions or loopholes in the environment.\n\nRead agent trajectories: Analyze how frontier coding agents attempted each task, and tell a genuine solve from a lucky or illegitimate one.\n\nJudge difficulty with data: Interpret pass rates across models and trials, and decide when a result is solid and when it needs more evidence.\n\nSeparate task faults from platform faults: Make sure contributors are judged on their work, never on infrastructure issues.\n\nWrite feedback that gets tasks fixed: Give contributors precise, actionable comments, so a task improves in one iteration instead of three.\n\nRaise the bar over time: Turn recurring defects into rubric rules, automated checks and better task templates, working with our engineers and researchers.\n\nWhat We're Looking For\n\nStrong engineering skills: Comfortable with Python, Bash, Linux and Docker. You can build, run, break and fix a task environment yourself.\n\nAn adversarial mindset: You enjoy finding the hole: exploits, test gaps, edge cases, reward hacks.\n\nClear, precise writing: Your feedback is specific, actionable and respectful.\n\nStatistical common sense: You understand pass rates, small samples and variance well enough to make sound calls.\n\nOwnership and autonomy: You take responsibility for quality, including the issues nobody else noticed, and you work without constant supervision.\n\nAmbition and drive: A highly motivated, resourceful and results-oriented person who wants to grow quickly and make a tangible impact on frontier AI.\n\nWe value demonstrated achievements, intellectual curiosity and the ability to execute over years of experience or impressive job titles.\n\nNice to Have\n\nExperience with RL environments, LLM evaluations, benchmarks or reward modeling.\n\nDeep technical knowledge in at least one domain: For example QA, software engineering, networking, security, data or ML engineering.\n\nExperience authoring tasks or benchmarks for AI agents.\n\nFluency with AI coding agents (Claude Code, Codex, Cursor or similar) as review tools.\n\nExperience writing review guidelines or calibrating a team of reviewers.\n\nWhy Join ARIMLABS?\n\nShape frontier AI directly: The tasks you approve train and evaluate the next generation of AI coding agents.\n\nExceptional growth opportunities: Join a fast-scaling AI research company at an early stage, with a path to leading the QA function as it grows.\n\nHigh autonomy: Freedom to make expert calls and propose better ways of working, without unnecessary bureaucracy.\n\nWork with exceptional people: Collaborate with highly skilled researchers, engineers and domain experts working on challenging problems at the frontier of artificial intelligence.\n\nExposure to frontier AI: See every day how the most capable AI agents succeed, fail and cheat.\n\nEntrepreneurial environment: Work in a company that values initiative, speed, ownership and ambitious thinking.\n\nCareer flexibility: Grow toward QA leadership, research, or task design as the organization expands.\n\nCompensation & Working Conditions\n\nEngagement: B2B contract.\n\nTime off: 24 days of paid time off annually.","description_format":"text","description_chars":3995,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Growth opportunities"],"hiring_locations":[{"name":"Ukraine","iso":"UA","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-11-01T21:41:45Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.859,"p_room":1,"age_days":5,"expected_fill_days":27,"reasons":["seen:5","win:early"],"computed_at":"2026-11-07T05:47:08Z"},"pay":null,"html_url":"https://alion.io/job/arimlabs-rl-data-qa-engineer","json_url":"https://alion.io/job/arimlabs-rl-data-qa-engineer.json","meta":{"generated_at":"2026-11-07T06:55:18Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":1353,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-08T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4493200},"rest":"https://alion.io/mcp/rest/get_company?id=4493200"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Farimlabs-rl-data-qa-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Farimlabs-rl-data-qa-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Farimlabs-rl-data-qa-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/arimlabs-rl-data-qa-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Farimlabs-rl-data-qa-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3592726}},{"tool":"search_jobs","arguments":{"company":"arimlabs.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3592726},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}