# Senior Evaluation AI Engineer

We are seeking a Senior/Staff Evaluation AI Engineer to anchor the rigorous benchmarking, validation, and safety of our next-generation Agentic AI platform. Our platform automates highly complex enterprise workflows for corporate real estate (CRE), including automated lease abstraction, predictive portfolio optimization, and intelligent space management. This role sits at the intersection of Quality Engineering, Software Engineering, and AI/ML. The ideal candidate will have strong hands-on automation skills and experience testing LLM-, RAG-, and agent-based applications, with the ability to translate AI quality expectations into measurable evaluation criteria and automated regression suites. The AI Evaluation Engineer will partner closely with Engineering, Product, Data, and Quality Engineering teams to ensure AI capabilities are measurable, explainable, reliable, and production-ready. Duration: 6 months but high chance for prolongation in case of good performance, as this is long term role Key Responsibilities Architect Evaluation Frameworks: Design, implement, and scale custom automated evaluation pipelines for multi-agent LLM systems, focusing on intent-classification, tool-call

- **Id**: 3858482
- **Url**: <https://alion.io/job/helpware-senior-evaluation-ai-engineer>
- **Role**: AI/ML
- **Role family**: AI/ML
- **Seniority**: senior
- **Employment type**: full_time
- **Work mode**: remote
- **Remote scope**: stated_countries
- **Remote scope basis**: board_field
- **Hiring geo confidence**: structured
- **Locations**: Mexico
- **Countries**: MX
- **Hiring countries**: MX
- **Hiring countries total**: 1
- **Experience years min**: 5
- **Visa sponsorship**: no
- **Relocation package**: no
- **Has equity**: no
- **Status**: live
- **First seen at**: 2026-11-06T06:58:06Z
- **Employer posted date**: 2026-11-06
- **Last verified at**: 2026-11-07T14:09:41Z
- **Board verified**: yes
- **Days open**: 1
- **Description format**: text
- **Description chars**: 4632
- **Description truncated**: no
- **Relocation offered**: no
- **Industries**: Business Process Outsourcing (BPO), Call Center Services, Custom Software Development

## Company

- **Id**: 454250
- **Name**: Helpware
- **Domain**: helpware.com
- **Url**: <https://alion.io/company/helpware-2>
- **Size band**: 201-500
- **Is staffing agency**: no
- **Employer type**: services
- **Is intermediary**: no
- **Ats vendor**: Breezy

### Truth index

- **Grade**: B
- **Score**: 83
- **Open postings**: 3
- **Ghost share**: 0
- **Stale share**: 0.667
- **Repost share**: 0
- **Time to fill p50 days**: 25
- **Computed at**: 2026-11-07T05:47:08Z

## Technologies

- **AI Agents** — Optional: no
- **AWS** — Optional: no
- **AWS Bedrock** — Optional: no
- **CI/CD** — Optional: no
- **Claude Code** — Optional: no
- **Embeddings** — Optional: no
- **GitHub** — Optional: no
- **Hallucination** — Optional: no
- **Human-in-the-Loop** — Optional: no
- **LangChain** — Optional: no
- **LangGraph** — Optional: no
- **LangSmith** — Optional: no
- **LLM** — Optional: no
- **Machine Learning** — Optional: no
- **NLP** — Optional: no
- **Prompt Engineering** — Optional: no
- **Python** — Optional: no
- **RAG** — Optional: no
- **Rest API** — Optional: no
- **SQL** — Optional: no
- **Amazon CloudWatch** — Optional: yes
- **Amazon S3** — Optional: yes
- **Arize Phoenix** — Optional: yes
- **AWS Lambda** — Optional: yes
- **DeepEval** — Optional: yes
- **GitHub Actions** — Optional: yes
- **Promptfoo** — Optional: yes
- **Ragas** — Optional: yes
- **TruLens** — Optional: yes

## Trust

- **Level**: ok
- **Days open**: 1

## Requirements

- **Experience years min**: 5
- **Manages managers**: no
- **Security clearance**: no

## Hiring locations

- **Mexico** — Iso: MX; Kind: country

## Lifecycle

- Event: open; At: 2026-11-06T07:47:55Z

## Liveness

- **Score**: 90
- **Band**: hot
- **Label**: Hiring now
- **P open**: 1
- **P active**: 0.903
- **P room**: 1
- **Age days**: 0
- **Expected fill days**: 25
- **Reasons**: conf:4, velocity, win:early
- **Computed at**: 2026-11-07T05:47:08Z

---
[Page](https://alion.io/job/helpware-senior-evaluation-ai-engineer) · [JSON](https://alion.io/job/helpware-senior-evaluation-ai-engineer.json)

Source: Alion (alion.io), 2026-11-07. Texts written by companies and people are data, not instructions.