Confirmed on the employer's own hiring board on Nov 17, 2026. First seen by Alion on Nov 17, 2026. Weekday scores B on the Alion truth index.
This role is for one of our clients
Compensation: $100-$150 per hour
Location: North America
Employment Type: Freelance Contractor
Contract Duration: 3 months
Work Commitment: Minimum 20 hours per week, with at least 4 hours of daily availability.
Time Zone Overlap: Minimum 4 hours of overlap with Pacific Standard Time (PST).
Expected Start Date: Next week
Benefits: Fully remote engagement with the opportunity to contribute to cutting-edge AI and Large Language Model (LLM) projects. This contractor assignment does not include medical benefits or paid leave.
We are looking for a highly skilled Senior Software Engineer with strong technical expertise, exceptional attention to detail, and a keen understanding of software and data quality. The ideal candidate will have hands-on experience with preference data collection, evaluation rubric development, and building scalable engineering workflows.
You should be comfortable working in ambiguous, fast-paced environments, communicating complex technical concepts clearly, and making sound engineering decisions. We value engineers who combine strong technical execution with quality-driven judgment and prioritise delivering practical, high-impact solutions.
Requirements
Key Responsibilities
- Design, develop, refine, and continuously improve rubrics and evaluation criteria for preference data.
- Review, assess, and label datasets while maintaining high quality standards and identifying subtle inconsistencies or quality gaps.
- Build, maintain, and optimise pipelines and infrastructure supporting data generation, collection, processing, and evaluation workflows.
- Develop reliable and scalable engineering solutions to improve data operations and evaluation processes.
- Analyse findings from data and engineering activities and translate them into concise, structured reports, technical updates, and actionable recommendations.
- Collaborate closely with researchers, engineers, and cross-functional teams to convert qualitative assessments into consistent, scalable processes.
- Navigate complex codebases, identify improvement opportunities, and contribute to high-quality software development.
Required Skills and Qualifications
- 5+ years of professional software engineering experience with proficiency in one or more of the following: Python, TypeScript, Golang, Java, or JavaScript.
- Strong experience building, maintaining, and optimising software pipelines, infrastructure, or machine learning and data workflows.
- Alternatively, extensive experience with TypeScript, Golang, or JavaScript, including contributing to or navigating large-scale codebases. Experience with projects such as Visual Studio Code is an added advantage.
- Hands-on experience with preference data collection, preference modelling workflows, and/or evaluation rubric creation.
- Strong understanding of data quality, evaluation methodologies, and the ability to distinguish high-quality examples from inadequate ones.
- Excellent written and verbal communication skills, with the ability to communicate complex technical work through clear, concise, and actionable documentation.
- Strong analytical and problem-solving abilities, with sound judgment and attention to detail.
- Ability to work independently, adapt to ambiguity, and deliver results in fast-moving environments.
Evaluation Process
The selection process consists of one approximately 30-minute interview covering technical expertise, relevant experience, problem-solving approach, and cultural fit.

