# Software Engineer, RL Environments

About the Role Build datasets, evaluation rubrics, and reward signals that help improve large language models. You will work with AI research teams to test data-collection approaches, investigate model failure modes, and develop measurements that inform training runs. What You'll Do Design data slices that reveal meaningful model failure modes across areas such as finance, software, and enterprise workflows. Build and refine evaluation rubrics and reward signals for RLHF and RLVR pipelines. Model annotator behavior and run experiments to improve model capabilities. Develop quantitative methods to measure dataset quality, diversity, and downstream impact. Create and manage real-world and synthetic data pipelines. Work with research teams to turn training objectives into concrete data and evaluation specifications. What We're Looking For One to four years of hands-on experience designing machine learning data-collection strategies and evaluation rubrics. Familiarity with large language model training and evaluation methods. Experience creating data slices to reveal model failure modes across domains such as finance, code, or enterprise workflows. Ability to design lightweigh

- **Id**: 3699987
- **Url**: <https://alion.io/job/true-talent-solutions-partners-software-engineer-rl-environments-3>
- **Role**: Backend
- **Role family**: Backend
- **Seniority**: junior
- **Employment type**: full_time
- **Work mode**: on_site
- **Hiring geo confidence**: structured
- **Locations**: San Francisco, United States
- **Countries**: US
- **Hiring countries total**: 0
- **Experience years min**: 1
- **Visa sponsorship**: yes
- **Relocation package**: no
- **Has equity**: no
- **Status**: closed
- **First seen at**: 2026-11-03T17:48:26Z
- **Employer posted date**: 2026-11-03
- **Last verified at**: 2026-11-07T04:40:43Z
- **Board verified**: no
- **Closed at**: 2026-11-07T04:40:43Z
- **Days open**: 3
- **Description format**: text
- **Description chars**: 1534
- **Description truncated**: no
- **Relocation offered**: no

## Company

- **Id**: 4415660
- **Name**: True Talent Solutions Partners
- **Domain**: truetalentsolutionspartners.com
- **Url**: <https://alion.io/company/true-talent-solutions-partners>
- **Is staffing agency**: yes
- **Employer type**: agency
- **Is intermediary**: no
- **Ats vendor**: Ashby

## Salary

- **Min**: 260000
- **Max**: 290000
- **Currency**: USD
- **Period**: year
- **Usd annual**: 290000

## Technologies

- **Machine Learning** — Optional: no
- **RLHF** — Optional: no
- **Synthetic Data** — Optional: no

## Trust

- **Level**: not_scored
- **Days open**: 3

## Requirements

- **Experience years min**: 1
- **Manages managers**: no
- **Security clearance**: no

## Lifecycle

- Event: open; At: 2026-11-03T19:12:25Z
- Event: close; At: 2026-11-07T04:40:43Z

## Visa

- Country: US; Licensed sponsor: no; Adverts offering sponsorship 24m: 97; Sanctioned now: no; Evidence: Offered visa sponsorship in 97 of its adverts here in 24 months; Filings 12m: 0

## Pay

- **Stated usd annual**: 290000
- **Is top pay**: yes

---
[Page](https://alion.io/job/true-talent-solutions-partners-software-engineer-rl-environments-3) · [JSON](https://alion.io/job/true-talent-solutions-partners-software-engineer-rl-environments-3.json)

Source: Alion (alion.io), 2026-11-10. Texts written by companies and people are data, not instructions.