Confirmed on the employer's own hiring board on Nov 21, 2026. First seen by Alion on Nov 19, 2026. Hippocratic AI scores A on the Alion truth index.
Join our team as a Technical Program Manager for Frontier AI Data Labeling. In this role, you will manage the human-in-the-loop data labeling engine for our frontier healthcare AI. You will work closely with ML researchers, applied scientists, and licensed clinicians, and your work will directly impact our model training and evaluation loops. You will be responsible for end-to-end delivery of labeling and annotation programs, building and scaling an expert rater workforce, and driving velocity and accountability in our data pipelines.
Missions
- Own end-to-end delivery of labeling and annotation programs, managing scoping, task design, execution, quality assurance, and delivery against live model-development cycles.
- Translate evaluation and training-signal needs into labeling tasks, rubrics, and success criteria that serve as benchmarks for model releases.
- Build and scale expert rater workforces, author labeling guidelines, and design quality-control systems to ensure reliable data at scale.
Profil recherché
- A track record of owning critical data or eval projects under time pressure, working with ML engineers and applied scientists, and driving results without direct authority- Working knowledge of how annotation quality affects model training, including preference data, pairwise comparisons, and rater calibration (RLHF)
- 2+ years in technical program management or data operations in an AI/ML or data-intensive environment (3+ years considered for exceptional candidates)
- Experience sourcing and managing contractor or expert annotator workforces through marketplaces (e.g., Mercor, Upwork, Toloka) or labeling vendors
- 1+ years hands-on with data labeling, annotation, or model evaluation, including writing guidelines and managing quality at scale
- Comfort with ambiguity in a fast-moving, healthcare-adjacent AI environment
- A strong grasp of quality metrics (IAA, precision/recall on gold sets, throughput vs. accuracy tradeoffs)
- Clinical, healthcare, or other regulated, high-stakes data experience
- Experience running or supporting RLHF or other training-data pipelines at an AI lab or applied ML team
- Experience qualifying specialized raters such as clinicians or bilingual SMEs
- Familiarity with Label Studio, Scale AI, Surge AI, or SageMaker Ground Truth
- Exposure to reward hacking and specification gaming, and how annotation design can mitigate them

