Sociology Expert (AI Evaluation)
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
Role Overview
We are looking for experienced Humanities professionals to support the evaluation and improvement of advanced AI systems. The role involves reviewing questions and answers across a broad range of humanities topics and assessing whether AI-generated content is accurate, relevant, coherent, complete, and well-reasoned.
This opportunity is suitable for professionals with backgrounds in areas such as History, Philosophy, Literature, Linguistics, Languages, Cultural Studies, Sociology, Anthropology, Political Science, Religious Studies, and related disciplines.
CONTRACT: Contractor assignment, 4 weeks
COMMITMENT: Hourly, up to 40h/week
LOCATIONS: Remote - Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Pakistan, Turkey, Vietnam
PROCESS: The selection will be made based on an online questionnaire and resume/education
Key Responsibilities
- Review and evaluate questions and answers across humanities disciplines.
- Assess content for factual accuracy, relevance, clarity, completeness, and logical consistency.
- Identify inaccuracies, misleading claims, ambiguities, and gaps in reasoning.
- Apply subject-matter knowledge to nuanced or complex humanities topics.
- Determine whether responses fully address the question and expected requirements.
- Apply established evaluation criteria and project guidelines consistently.
- Provide clear and structured quality assessments.
- Work across different humanities topics while maintaining strong analytical standards.
- Contribute to the quality of datasets used to train and evaluate AI systems.
Requirements
- At least 5 years of professional, academic, research, teaching, or related experience in a humanities field.
- Bachelor’s degree or equivalent expertise in History, Philosophy, Literature, Linguistics, Languages, Cultural Studies, Sociology, Anthropology, Political Science, Religious Studies, Arts & Humanities, or a closely related discipline.