Data Analyst (AI Evaluation)
First seen by Alion on Sep 29, 2026.
Data Analyst (AI Evaluation)
Contract · Remote · U.S. Only
Annual Compensation Equivalent: $57,600-$115,200/year
Hourly Rate: $30-$60/hr
The annual compensation figure is a full-time equivalent based on 40 hours per week across 48 workable weeks. It is not a guaranteed salary or earnings level.
About the Role
We are looking for experienced Data Analysts to review datasets and task outputs for accuracy, completeness, consistency, and adherence to defined evaluation criteria.
You’ll apply structured rubrics across large volumes of data, identify errors and inconsistencies, handle ambiguous cases using sound judgment, and provide clear documentation explaining your decisions. No prior AI experience is required.
What You’ll Do
Review datasets and task outputs for accuracy, completeness, consistency, and adherence to guidelines.
Identify and flag inconsistencies, errors, and outputs that deviate from defined instructions.
Apply structured rubrics and evaluation criteria consistently across large datasets.
Make well-reasoned decisions in ambiguous situations where guidelines may not fully address a scenario.
Document the rationale behind evaluation decisions clearly and consistently.
Identify recurring patterns or systemic data-quality issues and escalate them for further analysis.
Provide clear, actionable written and verbal feedback, including recommended resolution paths.
Maintain high levels of accuracy and attention to detail across repetitive and high-volume data review work.
What We’re Looking For
3-5+ years of experience in data review, data quality, quality assurance, analytics, or a related field.
Proven experience reviewing, validating, and critiquing data or deliverables against defined criteria.
Ability to follow detailed written instructions and apply complex evaluation rubrics consistently.
Strong judgment and comfort working with ambiguity.
Excellent error-spotting and attention-to-detail skills, particularly across high-volume or repetitive work.
Strong written and verbal communication skills.
Ability to clearly document findings and explain evaluation decisions.
Baseline spreadsheet proficiency, including sorting, filtering, and simple formulas.
Preferred Experience
Experience with SQL or hands-on data querying.
Data annotation, data labeling, or rubric-based evaluation.
Quality assurance or data validation workflows.
Experience identifying recurring data-quality patterns and escalating systemic issues.
Engagement & Compensation
Contractor engagement.
Remote work, U.S. based.
Hourly rate: $30-$60/hr.
Annual compensation equivalent: $57,600-$115,200 based on 40 hours per week across 48 workable weeks.
Actual earnings depend on the amount and scope of work assigned.
No prior AI experience is required.