# Staff Applied Scientist

We're looking for a Staff Applied Scientist to lead the design and build of an evaluation framework for an enterprise planner-based agent. You will work directly with the client's EA engineering team to benchmark their agent across planner models, executor LLMs and inference optimization levers. Every proposed change, whether a model upgrade, a harness change or a routing strategy, becomes a controlled, statistically sound experiment with a measurable dollar outcome. This is a research-heavy, customer-facing role with a significant build component. You will own both the experimental method and the tooling that makes it repeatable: a scenario-driven user simulator, isolated experiment execution, statistical comparison, and automated insights in a locally hosted dashboard the client can adopt as their own. The work runs in three overlapping streams (evaluation pipeline, grading and statistics, dashboard and experiment campaign) and ends in a recommendation the client can act on. Why This Role Matters At Robots & Pencils, we design AI systems for a human world. Our name says it all. Robots and pencils means engineering paired with creativity, because every agent we ship has to work fo

- **Id**: 4130847
- **Url**: <https://alion.io/job/robots-pencils-staff-applied-scientist>
- **Role**: AI/ML
- **Role family**: AI/ML
- **Seniority**: staff
- **Work mode**: remote
- **Remote scope**: stated_countries
- **Remote scope basis**: board_field
- **Hiring geo confidence**: structured
- **Hiring countries**: CA
- **Hiring countries total**: 1
- **Visa sponsorship**: no
- **Relocation package**: no
- **Has equity**: no
- **Status**: live
- **First seen at**: 2026-11-05T20:49:13Z
- **Employer posted date**: 2026-11-11
- **Last verified at**: 2026-11-12T02:25:16Z
- **Board verified**: yes
- **Days open**: 6
- **Description format**: text
- **Description chars**: 7010
- **Description truncated**: no
- **Relocation offered**: no
- **Industries**: UI/UX Design, Custom Software Development, AI Consulting & Integration

## Company

- **Id**: 7662
- **Name**: Robots & Pencils
- **Domain**: robotsandpencils.com
- **Url**: <https://alion.io/company/robots-and-pencils>
- **Size band**: 201-500
- **Is staffing agency**: no
- **Employer type**: services
- **Is intermediary**: no
- **Ats vendor**: Greenhouse

### Truth index

- **Grade**: B
- **Score**: 82
- **Open postings**: 7
- **Ghost share**: 0
- **Stale share**: 0.714
- **Repost share**: 0
- **Time to fill p50 days**: 34
- **Computed at**: 2026-11-11T05:49:22Z

## Technologies

- **Claude Code** — Optional: no
- **Langfuse** — Optional: no
- **LangSmith** — Optional: no
- **LLM** — Optional: no
- **Prompt Caching** — Optional: no
- **Python** — Optional: no

## Trust

- **Level**: ok
- **Days open**: 6

## Requirements

- **Manages managers**: no
- **Security clearance**: no

## Hiring locations

- **Canada** — Iso: CA; Kind: country

## Lifecycle

- Event: open; At: 2026-11-12T02:25:16Z

## Liveness

- **Score**: 86
- **Band**: hot
- **Label**: Hiring now
- **P open**: 1
- **P active**: 0.86
- **P room**: 1
- **Age days**: 6
- **Expected fill days**: 32
- **Reasons**: conf:2, win:early
- **Computed at**: 2026-11-12T04:39:02Z

---
[Page](https://alion.io/job/robots-pencils-staff-applied-scientist) · [JSON](https://alion.io/job/robots-pencils-staff-applied-scientist.json)

Source: Alion (alion.io), 2026-11-12. Texts written by companies and people are data, not instructions.