Member of Technical Staff - Evals
About the Role
As a Member of Technical Staff - Evals at Entendre, you will play a key role in ensuring the quality and reliability of our AI-powered features. Your primary responsibilities will include:
Designing and maintaining evaluation frameworks to measure the accuracy, reliability, and regression behavior of our AI capabilities.
Building automated test harnesses that operate on real accounting data, identifying potential failures before they impact our customers.
Defining metrics and benchmarks to provide the team with clear, quantitative insights into model and system performance.
Creating easy-to-use tooling that enables engineers to write, execute, and interpret evaluation tasks as a seamless part of their development workflow.
Collaborating closely with our Applied AI and Agent Engineering teams to guarantee every shipped capability has a well-defined standard of "good enough"-and a robust method to verify it.
What We're Looking For
Experience writing and reviewing evaluation tasks used by AI researchers to assess and improve model performance.
Background in quality assurance across software, client deliverables, or financial analysis.
Exceptional written communication skills, with the ability to clearly articulate steps to achieve desired outcomes.
(Preferred) Familiarity with accounting, financial systems, tax preparation, or payment technologies.