2,391,362open jobs
149,182companies
289,228added this week
Browse all
Location
Remote (India)
Seniority
Senior · 2+ years exp
Visa
Sponsors visas in United States; no register of this country names it
Employment
Full-Time

Confirmed on the employer's own hiring board on Nov 19, 2026. First seen by Alion on Nov 19, 2026.

Overview
Company
Impact
Profile match
Welocalize is a language services and AI data company headquartered in New York that provides translation, localization, machine translation, multimedia, interpretation and product testing services to global businesses. Founded in 1997, it runs a global network of linguists and AI training experts, operates the Welo Data brand for annotation and model training data, and has offices in places such as Frederick, Gurgaon, Cluj-Napoca, Barcelona, Thessaloniki, London and Tokyo. It hires linguists and translation quality raters in many languages, AI translation trainers, localization project managers and coordinators, generative AI data operations staff and software test engineers.

If you have a Candidate Login already, but have forgotten your password please use the steps to reset your password. If you have forgotten your email login, please contact [email protected] subject Workday Candidate Login

When creating your Workday account and entering personal information like name, address, please do not use ALL CAPS.

Thank you!

NOTICE: For Privacy Policy please review here

Job Responsibilities:

Welo Data is an award-winning localization and AI data company. We run some of the world’s largest AI model relevance rating programs. This will play a crucial role in analyzing and improving AI-driven online advertising platforms.

This role will help establish benchmarks and guidelines for data used to train AI models, particularly in areas lacking defined standards. That means you will be shaping the future of web-based advertising and AI models.

MAIN DUTIES

The following is a non-exhaustive list of responsibilities and areas of ownership of an Expert Rater

  • Rate Client-provided ads following pre-defined guidelines
  • Evaluate advertisements based on specific quality, relevance, and compliance criteria provided by the client.
  • Ensure ratings are objective, consistent, and aligned with the predefined rating standards.
  • Identify any guideline ambiguities and apply critical thinking when assessing borderline cases.
  • Maintain a high level of accuracy and efficiency, meeting daily/weekly rating quotas.
  • Review and refine rating guidelines, identifying areas for improvement and recommending updates to enhance accuracy, consistency, and overall rating quality.
  • Curate high-quality datasets ("golden sets") by selecting and validating top-tier rated ads, which can serve as benchmarks for AI model training and human rater calibration.
  • Collaborate with cross-functional teams (e.g., quality assurance, data analysts, and AI training teams) to ensure rating processes align with evolving industry and client needs.
  • Analyse trends and patterns in ad ratings, providing insights that contribute to guideline optimizations and overall rating effectiveness.
  • Assist in training and mentoring new raters, ensuring they understand guidelines and best practices for accurate evaluations.

REQUIREMENTS

Experience

  • Have experience in AI model relevance, such as data labeling, linguistic annotation, relevance rating, etc.
  • Are analytical and data-driven with a proven track-record of data-based decision making
  • Have a strong interest and understanding of digital marketing and advertising, especially as it relates to artificial intelligence
  • Can effectively summarize complex information, questions, and solutions for various audiences Have experience researching and gathering insights, and understanding complex problems across a variety of topics
  • Can defining, explain, and defend their points of view in group discussions
  • Are comfortable reviewing and discussing a variety of topics - you do not need to have expertise on the topics, but willingness to research and understand to support consensus decisions is vital
  • Are comfortable with ambiguity in guidelines, workflows, and processes, and are motivated to help establish those systems

Requirements

  • Native level fluency in the target language
  • Excellent understanding of both written & spoken English
  • A 4 year university degree and/or or at least 2 years work experience in one or more of the following: AI-based relevance rating, marketing/advertising, linguistics, content creation, writing, tech-related quality assurance, data labeling, or linguistic annotation.

Additional Job Details:

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,391,362 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Operations
Similar stack
Same company
India
$274k – $3.3M per year · In office · 3+ years exp · Bachelor's Degree · Hyderabad
Apply
In office · Noida
SQL
Databases
Microsoft Fabric
Analytics
Power BI
Management
Power Automate
SharePoint
Apply
In office · 5+ years exp · Associate's Degree · Mumbai
DevOps
Linux
Windows
Apply
Remote (Malaysia) · Freelance · 3+ years exp · Kuala Lumpur
AI/ML
Claude
ChatGPT
Gemini
LLM
Machine Learning
Apply
Remote (Vietnam) · Freelance · 5+ years exp · Hanoi
Apply
In office · Full-Time · India
Apply
In office · Full-Time · 8+ years exp · Bachelor's Degree · India
Design
AutoCAD
Apply
In office · Full-Time · India
Apply
Remote (India) · Full-Time · 6+ years exp · India
PowerShell
AI/ML
Copilot
AI Agents
Copilot Studio
DevOps
Azure DevOps
Azure
CI/CD
Git
Windows
Cybersecurity
Microsoft Entra ID
Analytics
Power BI
Management
Power Automate
Apply
≈ $11k – $28k per year (Estimated) · Hybrid · 8+ years exp · India
DevOps
Dynatrace
Management
Agile
Apply
See all jobs
This is one of many
2,391,362 more open roles from verified company boards, updated every day.