Research Engineer, QC Automation
About the Role
This is the top hiring priority on a ~15-person engineering team building infrastructure for reinforcement learning environments and post-training AI datasets. As a Research Engineer focused on QC Automation, you'll own the systems that ensure training data quality scales with growing demand - a critical function that sits at the intersection of engineering rigor and research judgment.
What You'll Do
Automate quality control for training data produced by companies using the platform's infrastructure.
Build QC systems grounded in genuine human understanding and judgment, rather than heavy LLM reliance.
Define and enforce quality standards for AI training data end-to-end.
Design experiments and metrics to grade agent outputs across diverse tasks.
Partner with data vendors to debug quality issues, diagnose agent failure modes, and improve their data generation processes.
Translate QC learnings into auditing systems, including sampling strategies and rule-based or model-assisted validation pipelines.
Continuously integrate QC insights into infrastructure tools and the vendor portal to reduce anomalies and edge cases.
What We're Looking For
2-4 years of experience in engineering or research roles, ideally focused on QC automation or data quality.
Proficiency in Python, Docker, and Linux environments.
Proven experience building scalable data validation pipelines and automated QA/QC systems end-to-end without a fully prescribed roadmap.
Experience working on benchmarks and evaluations for RL training data - including defining realistic tasks, reliable rubrics, and useful trajectories.
Demonstrated ability to create QC systems based on human judgment rather than defaulting to LLM-based approaches.
Experience designing experiments and metrics to grade agent outputs, and partnering with data vendors to provide actionable feedback.
Solid knowledge of statistics and comfort designing metrics and QA/QC processes.
Strong written and verbal communication skills for effective cross-timezone collaboration.
Genuine curiosity, intellectual range, and the ability to work autonomously in fast-paced, early-stage environments.
Compensation & Benefits
Salary range: $150,000 - $250,000 USD annually. Visa sponsorship is available for qualifying candidates.
Location
On-site in San Francisco, CA for U.S.-based candidates; on-site in Singapore for Southeast Asia-based candidates. Fully remote independent contractor arrangements are also considered for candidates based elsewhere, particularly in Europe.