{"id":734757,"url":"https://alion.io/job/idlexperts-problem-design-engineer","title":"Problem Design Engineer","company":{"id":4423928,"name":"idlexperts","domain":"idlexperts.ai","url":"https://alion.io/company/idlexperts","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":null},"role":"Industrial Engineering","role_family":"Industrial Engineering","seniority":"junior","employment_type":"part_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":[],"countries":[],"hiring_countries":["US"],"hiring_countries_total":1,"salary":{"min":40,"max":40,"currency":"USD","period":"hour","gross":null,"usd_annual":80000},"salary_estimate":null,"experience_years_min":1,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[],"status":"live","first_seen_at":"2025-10-28T20:57:00Z","employer_posted_date":"2025-10-28","last_verified_at":"2026-10-22T15:10:06Z","board_verified":true,"closed_at":null,"days_open":358,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":358},"description":"What We’re Building\nWe create high-quality coding problems for training AI models - drawn from real production repositories, not synthetic puzzles. Your role is to ensure these problems are fair, clear, and technically sound.\nThe Role\nPart-Time Code Quality Reviewer | Remote | $40/hr | 0-25 hrs/week\nWe’re looking for engineers who can trace the intent behind a code change, understand what changes were made and why, and express that clearly in natural language.\nThis is judgment work, not mechanical testing. You'll make philosophical decisions about fairness and clarity in technical contexts.\nWhat You'll Actually Do\nReview code changes across unfamiliar repositories and tech stacks\n\nAssess problem fairness: Could a skilled engineer solve this with only the given context?\n\nEvaluate test quality: Do tests properly validate the intended solution?\n\nWrite clear justifications for your assessments (2-3 sentences per decision)\n\nFlag coherence issues: Inconsistent artifacts, unclear requirements, missing context\n\nTime commitment: 1-4 hours per problem, fully async - work whenever fits your schedule.\nWho Should Apply\nIdeal candidates:\nPhD students in CS/computational fields seeking long-term flexible income\n\nOpen source contributors who care deeply about code quality\n\nUndergrad with at least 1 year in CS, ECE, Computational Biology, or related technical programs\n\nProfessional engineers seeking intellectually interesting part-time work\n\nYou'll thrive if you:\nHave strong software engineering fundamentals\n\nCan work independently with minimal supervision\n\nWrite clearly and precisely\n\nEnjoy making judgment calls in ambiguous situations\n\nStay focused during 1-3 hour review sessions\n\nRequirements\n1+ years of software engineering experience (internships, research projects, or professional work)\n\nFluent English; strong writing is a must\n\nResponsive, independent, and attentive to detail\n\n0-25 hours per week, flexible\n\nCompensation & Growth\n$40/hour for all review work\n\nPerformance-based advancement: Top contributors can receive promotions and bonuses\n\nLong-term engagement: We're building a scalable team, not a temporary workforce\n\nApplication Process\n15-minute async video interview - answer 4 questions about your technical background and work style\n\nSample problem review - demonstrate your assessment skills on a real coding problem\n\nOnboarding - join our active problem design engineering community and start reviewing\n\nFAQ\nQ: Is this suitable for PhD students?\nYes - many of our top contributors are PhD students. The work is fully async and fits around research schedules.\nQ: Do I need to be available during specific hours?\nNo. Work whenever you want. Some problem design engineers work weekends, others weekday evenings.\nQ: What if I'm not familiar with the programming language in a problem?\nThat's expected. You're assessing problem fairness and clarity, not implementing solutions. Familiarity with general software engineering principles is what matters.\nQ: How quickly can I start?\nIf you pass the video interview and sample review, you can start within 1-2 weeks.\nQ: Is this a path to full-time employment?\nThis is a long-term contract role, not a pipeline to full-time positions. We value contributors who excel at this specific type of work.\nAbout idlexperts\nidlexperts is the quality assurance division of Idler (YC S25), where we ensure coding problems meet research-grade standards for AI training. Our mission is to build the most reliable QA system for AI coding benchmarks.","description_format":"text","description_chars":3512,"description_truncated":false,"requirements":{"experience_years_min":1,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"phd","optional":false},"security_clearance":false,"languages":[{"language":"English","level":"Advanced (C1)","optional":false}]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-09-11T10:41:54Z"}],"visa":[],"liveness":{"score":4,"band":"cold","label":"Long shot","p_open":1,"p_active":0.128,"p_room":0.28,"age_days":358,"expected_fill_days":24,"reasons":["conf:1","win:tail","crowd:junior"],"computed_at":"2026-10-22T06:10:10Z"},"pay":{"stated_usd_annual":80000,"is_top_pay":false},"html_url":"https://alion.io/job/idlexperts-problem-design-engineer","json_url":"https://alion.io/job/idlexperts-problem-design-engineer.json","meta":{"generated_at":"2026-10-22T17:22:20Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":18289,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-23T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4423928},"rest":"https://alion.io/mcp/rest/get_company?id=4423928"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fidlexperts-problem-design-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fidlexperts-problem-design-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fidlexperts-problem-design-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/idlexperts-problem-design-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fidlexperts-problem-design-engineer"}]}