# Principal Applied Scientist

We're looking for a Principal Applied Scientist to lead the design and build of an evaluation framework for an enterprise planner-based agent. You will work directly with the client's engineering team to benchmark their agent across planner models, executor LLMs and inference optimization levers. Every proposed change, whether a model upgrade, a harness change or a routing strategy, becomes a controlled, statistically sound experiment with a measurable dollar outcome. This is a research-heavy, customer-facing role with a significant build component. You will own both the experimental method and the tooling that makes it repeatable: a scenario-driven user simulator, isolated experiment execution, statistical comparison, and automated insights in a locally hosted dashboard the client can adopt as their own. The work runs in three overlapping streams (evaluation pipeline, grading and statistics, dashboard and experiment campaign) and ends in a recommendation the client can act on. Why This Role Matters At Robots & Pencils, we design AI systems for a human world. Our name says it all. Robots and pencils means engineering paired with creativity, because every agent we ship has to work f

- **Id**: 4167443
- **Url**: <https://alion.io/job/robots-pencils-staff-applied-scientist-2>
- **Role**: AI/ML
- **Role family**: AI/ML
- **Seniority**: lead
- **Work mode**: remote
- **Remote scope**: stated_countries
- **Remote scope basis**: board_field
- **Hiring geo confidence**: structured
- **Hiring countries**: US
- **Hiring countries total**: 1
- **Visa sponsorship**: no
- **Relocation package**: no
- **Has equity**: no
- **Status**: live
- **First seen at**: 2026-11-12T13:33:04Z
- **Employer posted date**: 2026-11-12
- **Last verified at**: 2026-11-15T23:49:09Z
- **Board verified**: yes
- **Days open**: 3
- **Description format**: text
- **Description chars**: 7000
- **Description truncated**: no
- **Relocation offered**: no
- **Industries**: UI/UX Design, Custom Software Development, AI Consulting & Integration

## Company

- **Id**: 7662
- **Name**: Robots & Pencils
- **Domain**: robotsandpencils.com
- **Url**: <https://alion.io/company/robots-and-pencils>
- **Size band**: 201-500
- **Is staffing agency**: no
- **Employer type**: services
- **Is intermediary**: no
- **Ats vendor**: Greenhouse

### Truth index

- **Grade**: B
- **Score**: 79
- **Open postings**: 7
- **Ghost share**: 0
- **Stale share**: 0.857
- **Repost share**: 0
- **Time to fill p50 days**: 32
- **Computed at**: 2026-11-15T07:52:55Z

## Technologies

- **Claude Code** — Optional: no
- **Langfuse** — Optional: no
- **LangSmith** — Optional: no
- **LLM** — Optional: no
- **Prompt Caching** — Optional: no
- **Python** — Optional: no

## Trust

- **Level**: ok
- **Days open**: 3

## Requirements

- **Manages managers**: no
- **Security clearance**: no

## Hiring locations

- **United States** — Iso: US; Kind: country

## Lifecycle

- Event: open; At: 2026-11-12T13:48:52Z

## Visa

- Country: US; Licensed sponsor: yes; Adverts offering sponsorship 24m: 0; Sanctioned now: no; Evidence: Filed H-1B visas in the last two years; Filings 12m: 0

## Liveness

- **Score**: 90
- **Band**: hot
- **Label**: Hiring now
- **P open**: 1
- **P active**: 0.903
- **P room**: 1
- **Age days**: 2
- **Expected fill days**: 32
- **Reasons**: conf:3, velocity, win:early
- **Computed at**: 2026-11-15T07:52:55Z

---
[Page](https://alion.io/job/robots-pencils-staff-applied-scientist-2) · [JSON](https://alion.io/job/robots-pencils-staff-applied-scientist-2.json)

Source: Alion (alion.io), 2026-11-16. Texts written by companies and people are data, not instructions.
