Incident management / reliability / SRE Evaluator

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an Incident Management / Reliability / SRE Evaluator based in the United States.

This remote, hourly opportunity is designed for experienced professionals in incident management, reliability engineering, and site reliability engineering.

You’ll use your practical expertise to evaluate AI-generated work products for technical accuracy, rigor, and overall quality.

The role provides an opportunity to apply real-world reliability and incident-management judgment to emerging AI systems.

You’ll assess whether generated outputs reflect sound operational practices, appropriate reasoning, and domain-specific standards.

Your feedback will directly contribute to improving how AI handles reliability, incident response, and SRE-related tasks.

The engagement is flexible and fully remote, allowing you to complete project work on your own schedule.

You’ll collaborate asynchronously with a research team while contributing specialized expertise to AI development.

Accountabilities

• Evaluate AI-generated documents, spreadsheets, slide decks, and other work products against established incident management, reliability, and SRE quality standards.
• Assess generated outputs for technical accuracy, operational rigor, sound judgment, and alignment with real-world reliability practices.
• Identify factual, technical, presentation, and reasoning errors that could affect the quality or usefulness of an AI-generated deliverable.
• Apply subject-matter expertise to determine whether incident-management and reliability recommendations are practical, complete, and appropriately structured.
• Provide clear, concise, and structured written feedback explaining errors, gaps, and opportunities for improvement.
• Help establish and apply domain-specific quality criteria and rubrics when evaluating AI outputs.
• Collaborate asynchronously with the research team to refine evaluation approaches and improve AI performance.
• Complete assigned evaluation projects accurately and within agreed timelines while maintaining professional standards.
• Perform work using examples and knowledge that do not require access to confidential or proprietary information from any employer, client, or institution.

Requirements

• 5+ years of relevant professional experience in incident management, reliability engineering, site reliability engineering (SRE), or closely related disciplines.
• Strong practical understanding of incident response, service reliability, operational processes, troubleshooting, and reliability best practices.
• Ability to critically evaluate technical work products and distinguish accurate, actionable recommendations from flawed or incomplete reasoning.
• Native or professional fluency in English, with strong written communication skills.
• Highly proficient with Microsoft Office and Google Workspace, particularly Google Slides and/or PowerPoint.
• Excellent attention to detail and the ability to identify factual, technical, aesthetic, and presentation-related issues.
• Strong analytical and critical-thinking skills, with the ability to explain technical assessments clearly and objectively.
• Ability to work independently and manage deliverables effectively in a fully remote, asynchronous environment.
• Preferred: Advanced degree (Master’s or higher) from a reputable institution or equivalent advanced professional expertise.
• Comfortable adapting established evaluation criteria and applying consistent judgment across a variety of AI-generated outputs.

Benefits

• Compensation: $80-$120 per hour.
• Fully remote opportunity for professionals based in the United States.
• Flexible work that can generally be completed on your own schedule.
• Independent contractor engagement.
• Weekly payments through Stripe or Wise based on services rendered.
• Opportunity to apply your professional expertise to the development and improvement of AI systems.
• Asynchronous collaboration with a research team.
• Projects may be extended, shortened, or concluded early depending on project needs and performance.
• Work does not require access to confidential or proprietary information from any employer, client, or institution.
• Please note: H-1B and STEM OPT candidates cannot be supported for this opportunity.