# AI Evaluation Scientist

Overview We are looking for an AI Evaluation Scientist to design and execute evaluation processes that ensure our predictive and generative AI systems are accurate, reliable, safe, and aligned with mission requirements. This role is essential for establishing trust in AI solutions and supporting continuous improvement across the AI lifecycle. The AI Evaluation Scientist will work closely with engineers, data scientists, governance analysts, and product teams to develop evaluation metrics, build test harnesses, analyze model behavior, and support responsible deployment. Contributions Implement evaluation frameworks for AI models, including accuracy, robustness, relevance, bias, hallucination rate, and safety metrics. Build and maintain automated evaluation scripts, tests, and pipelines that assess AI model outputs and detect performance drift over time. Develop benchmark datasets, challenge sets, and scenario-based test cases tailored to mission and user needs. Perform structured error analysis and behavioral audits of LLMs, retrieval-augmented generation (RAG) systems, and predictive models, documenting findings and improvement recommendations. Collaborate with AI Developers, LL

- **Id**: 4263028
- **Url**: <https://alion.io/job/steampunk-ai-evaluation-scientist-2>
- **Role**: AI/ML
- **Role family**: AI/ML
- **Seniority**: senior
- **Work mode**: on_site
- **Hiring geo confidence**: structured
- **Locations**: McLean, United States
- **Countries**: US
- **Hiring countries total**: 0
- **Experience years min**: 8
- **Visa sponsorship**: no
- **Relocation package**: no
- **Has equity**: no
- **Status**: live
- **First seen at**: 2026-11-13T16:02:22Z
- **Employer posted date**: 2026-11-13
- **Last verified at**: 2026-11-13T16:02:22Z
- **Board verified**: yes
- **Days open**: 0
- **Description format**: text
- **Description chars**: 5011
- **Description truncated**: no
- **Relocation offered**: no
- **Industries**: Digital Government, Custom Software Development

## Company

- **Id**: 3854754
- **Name**: Steampunk
- **Domain**: steampunk.com
- **Url**: <https://alion.io/company/steampunk-5>
- **Size band**: 1001-5000
- **Is staffing agency**: no
- **Employer type**: services
- **Is intermediary**: no
- **Ats vendor**: iCIMS

### Truth index

- **Grade**: A
- **Score**: 100
- **Open postings**: 5
- **Ghost share**: 0
- **Stale share**: 0
- **Repost share**: 0
- **Time to fill p50 days**: 18
- **Computed at**: 2026-11-12T10:38:37Z

## Salary

- **Min**: 105000
- **Max**: 145000
- **Currency**: USD
- **Period**: year
- **Usd annual**: 145000

## Technologies

- **DeepEval** — Optional: no
- **Embeddings** — Optional: no
- **Hallucination** — Optional: no
- **Hugging Face** — Optional: no
- **Human-in-the-Loop** — Optional: no
- **LangChain** — Optional: no
- **LLM** — Optional: no
- **LLMOps** — Optional: no
- **Machine Learning** — Optional: no
- **NLP** — Optional: no
- **OpenTelemetry** — Optional: no
- **Python** — Optional: no
- **PyTorch** — Optional: no
- **RAG** — Optional: no
- **Ragas** — Optional: no
- **Scikit-learn** — Optional: no
- **AWS** — Optional: yes
- **Azure** — Optional: yes
- **NIST AI RMF** — Optional: yes
- **OWASP** — Optional: yes

## Trust

- **Level**: ok
- **Days open**: 0

## Requirements

- **Experience years min**: 8
- **Manages managers**: no
- **Security clearance**: no

### Education

- **Level**: bachelor
- **Optional**: yes

## Lifecycle

- Event: open; At: 2026-11-13T16:02:22Z

## Liveness

- **Score**: 86
- **Band**: hot
- **Label**: Hiring now
- **P open**: 1
- **P active**: 0.86
- **P room**: 1
- **Age days**: 0
- **Expected fill days**: 20
- **Reasons**: conf:1, win:early, comp:brand
- **Computed at**: 2026-11-13T17:55:03Z

## Pay

- **Stated usd annual**: 145000
- **Is top pay**: no

---
[Page](https://alion.io/job/steampunk-ai-evaluation-scientist-2) · [JSON](https://alion.io/job/steampunk-ai-evaluation-scientist-2.json)

Source: Alion (alion.io), 2026-11-13. Texts written by companies and people are data, not instructions.