Agentic Judge - Backoffice role

Job Summary

This hybrid backoffice helpdesk role supports customer experience operations for independent software vendors and high tech clients by handling day to day case management documentation and coordination tasks. The specialist uses strong spoken English skills to resolve inquiries maintain accurate records and collaborate with internal teams helping the company deliver reliable digital services that improve productivity and trust for global customers.

Responsibilities

AI Annotation Judge

Agentic Annotation Tasks

• Data Verification: Review structured and unstructured data outputs from AI models and flag inaccuracies, inconsistencies, or hallucinations.
• Summary Evaluation: Assess AI-generated summaries against source material for accuracy, completeness, and appropriate length.
• Review AI-generated responses
• Provide structured written feedback where a response fails criteria, explaining the specific deficiency.
• Computer Vision & Image Annotation

Review pre-annotated images

• Maintain productivity and quality benchmarks across high-volume image batches.
• Review AI-generated content related to US real estate (property listings, transaction summaries, market analyses, home inspection reports, valuation commentary, etc.).
• Identify factual errors, outdated terminology, or missing context that would mislead a US-based consumer or professional.
• Augment AI responses with corrective commentary explaining why a response is wrong and how it should be improved.
• Evaluate descriptions of residential property features (floor plans, room types, fixtures, construction materials, appliances) for accuracy and completeness.
• Provide culturally nuanced feedback on whether content resonates authentically with a US homeowner or homebuyer audience.

Qualifications

• Full professional English proficiency (written and spoken). All annotation, feedback, and communication is in English.
• Strong reading comprehension and attention to detail; ability to apply written criteria consistently across many samples.
• Comfort working with structured review interfaces, spreadsheets, or annotation platforms.
• Reliable internet connection; ability to work independently and meet deadlines with minimal supervision.
• Ability to maintain throughput targets while upholding quality - both speed and accuracy are measured.
• Prior experience in data annotation, content moderation, QA, transcription, or a related field is a plus.