{"id":2936882,"url":"https://alion.io/job/huzzle-accounting-expert-ai-model-evaluation","title":"Accounting Expert: AI Model Evaluation","company":{"id":720230,"name":"Huzzle","domain":"huzzle.com","url":"https://alion.io/company/huzzle-2","size_band":"1001-5000","is_staffing_agency":true,"employer_type":"agency","is_intermediary":false,"listed_via":null,"ats_vendor":"Workable","truth_index":{"grade":"A","score":95,"open_postings":561,"ghost_share":0,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":125,"computed_at":"2026-10-21T05:46:15Z"}},"role":"Finance","role_family":"Finance","seniority":"senior","employment_type":"contractor","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":[],"countries":[],"hiring_countries":["US","IN","DE","GB"],"hiring_countries_total":4,"salary":null,"salary_estimate":{"min_usd":60000,"max_usd":119000,"period":"year","method":"role_seniority_country_cell","sample_n":98},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[],"status":"live","first_seen_at":"2026-10-20T16:18:17Z","employer_posted_date":"2026-10-20","last_verified_at":"2026-10-21T13:44:06Z","board_verified":true,"closed_at":null,"days_open":1,"trust":{"level":"ok","repost_count":0,"flags":["company_stale"],"days_open":1},"description":"About Huzzle Labs\nHuzzle Labs partners with frontier AI labs to evaluate and improve frontier models through deep human expertise. Contributors directly assess advanced AI systems in their specialist fields, receive competitive compensation, and help define the standards used to measure model performance.\nAbout the Role\nHuzzle Labs is seeking experienced accountants to write realistic accounting tasks that test how well frontier AI systems handle accounting work. You will create prompts that are difficult enough for a frontier model to fail, then write a scoring rubric for each task so reviewers can assess the output consistently. A rubric is a list of objective, checkable criteria that a reviewer marks pass or fail. Your judgement helps define the standards used to measure model performance.\nThis is an expert judgement role focused on realistic task design, careful review of model output and precise, consistent documentation. Training is provided. Prior AI training or data-annotation experience is required.\nResponsibilities\nWrite realistic accounting tasks: Create requests of the kind a client, controller or business owner would send, covering month-end close, bank and credit card reconciliations, AR/AP ageing, accruals and adjusting entries, sales tax, job costing and the clean-up of incomplete or inconsistent books.\nBuild from real material: Review a deal room containing hundreds of files, select the relevant documents and use them to develop prompts and rubrics for tasks a frontier model struggles to complete correctly.\nRequire usable deliverables: Each request should call for a working output, usually a spreadsheet, a journal entry schedule or a short memo.\nTest against a frontier model: Run each request and review the output with the care you would apply when signing off accounting work. Where the model succeeds, increase the difficulty until a genuine gap appears.\nWrite scoring rubrics: Define objective criteria that another reviewer can apply without repeating your work, for example, “the reconciled ending balance equals $X” and “the $X deposit is recorded as a customer deposit liability, not revenue.”\nApply guidelines consistently: Follow the task and rubric guidelines, flag ambiguous cases and help refine the criteria.\nRequirements\nAt least 5 years in accounting, bookkeeping or finance operations.\nPrior AI training or data-annotation experience is required.\nHands-on knowledge of month-end close, reconciliations, AR/AP, accruals and adjusting entries, sales tax and job costing.\nExcellent written English and precision with numbers.\nCareful and detail-oriented: accurate figures and objective, checkable criteria.\nAvailable 20 to 40 hours per week for an initial one-week trial, with potential extension.\nExample Assignment\nFor a client deal room covering a small business with incomplete and inconsistent books, you would:\nReview the files and select the relevant documents, such as bank statements, ledgers, invoices and ageing reports.\nWrite a prompt as the business owner or controller would, asking for a reconciliation and the adjusting entries needed to close the month.\nRun the prompt against a frontier AI model and check the spreadsheet or journal entry schedule line by line.\nIf the model completes the task correctly, increase the difficulty, for example, by adding a customer deposit, a timing difference or a sales tax complication, until it fails.\nWrite the rubric, such as “the reconciled ending balance equals $X” and “the $X deposit is recorded as a customer deposit liability, not revenue,” so another reviewer can grade any answer without repeating the work.\nConfidentiality\nAn NDA must be signed before the engagement begins.\nBenefits\nCompensation: $60 - $90 USD / hour","description_format":"text","description_chars":3743,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[{"language":"English","level":"All levels","optional":false}]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"},{"name":"India","iso":"IN","kind":"country"},{"name":"Germany","iso":"DE","kind":"country"},{"name":"United Kingdom","iso":"GB","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-20T16:18:17Z"}],"visa":[],"liveness":{"score":54,"band":"ok","label":"Likely open","p_open":1,"p_active":0.542,"p_room":1,"age_days":0,"expected_fill_days":125,"reasons":["conf:1","agency","velocity","win:early","comp:brand"],"computed_at":"2026-10-21T05:46:15Z"},"pay":null,"html_url":"https://alion.io/job/huzzle-accounting-expert-ai-model-evaluation","json_url":"https://alion.io/job/huzzle-accounting-expert-ai-model-evaluation.json","meta":{"generated_at":"2026-10-21T18:25:20Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":25969,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-22T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":720230},"rest":"https://alion.io/mcp/rest/get_company?id=720230"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fhuzzle-accounting-expert-ai-model-evaluation"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fhuzzle-accounting-expert-ai-model-evaluation"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fhuzzle-accounting-expert-ai-model-evaluation"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/huzzle-accounting-expert-ai-model-evaluation\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fhuzzle-accounting-expert-ai-model-evaluation"}]}