Senior Quality Automation Engineer
Does redefining what Quality looks like in the age of AI - on a brand-new AI product - make you tick?
At Re-Leased our vision is to shape the future of Commercial Real Estate and drive better outcomes for people, property and planet. We're now building am:pm - an agentic AI product for commercial real estate operators.
Property managers run their portfolios on a system of record like Re-Leased, but most of the actual work happens outside it - emails, phone calls, follow-ups, to-do lists scattered across tools, with no system behind them. am:pm is the system of action for that work: it captures the work to be done as jobs and tasks, grounded in the communication channels, memories, and documents that surround the portfolio - so the work itself finally has a home. You can read more at ampm.ai.
am:pm is in early access with customers right now. The squad is small, the stack is new, and we're moving fast.
Your Role
We're looking for a senior quality automation engineer to own the quality system for am:pm - based in Napier or remote from another city in New Zealand. The whole squad owns quality; you build the automation, quality controls, and practices that make that possible. This is an automation-led role: engineers write unit and integration tests, and you shape where automated coverage sits across the test pyramid and steward the end-to-end suite, with AI as a core part of how you work - generating tests, triaging failures, creating test data. We're also moving to continuous delivery, and a core part of this role is making that safe - the right level of automated quality controls so changes flow to production with confidence.
And because am:pm is an AI product, quality means testing agentic behaviour: non-deterministic outputs, tool use, statistical rather than binary correctness. Engineers write the evals - just as they write unit and integration tests - and you bring the quality lens: when evals are the right control, and what they need to catch.
This is a small-team, start-up environment inside an established SaaS business - less process, more agency, and a tight feedback loop with real customers.
Your Responsibilities
- Drive quality automation across the test pyramid - set where coverage sits so tests run at the layer that's cheapest and most reliable - and steward the end-to-end suite: curate what earns a place in it, and set the standards engineers follow when contributing tests of their own
- Support our move to continuous delivery - design the right level of automated quality controls (pipeline gates, smoke checks, release signals) so we can release frequently without slowing the flow of change
- Use AI tooling as a first-class part of your workflow - test generation, bug triage, test data creation - and keep raising the bar on how the team uses it
- Curate AI-generated tests and findings - decide what to keep, reject, or refine, applying your own testing judgment rather than accepting output wholesale
- Shape how we test AI-powered features - advise on when evals are the right quality control and what they should cover, and contribute adversarial and edge-case scenarios; engineers write and own the evals themselves
- Partner with engineers on acceptance criteria and coach them on designing test scenarios from a user's perspective - positive paths, negative paths, edge cases
- Run exploratory and risk-based testing to find what automation misses, and feed production bugs back into automation at the layer that would have caught them
- Analyse quality workflows - triage, release, escape defects - and drive structural improvements, reporting on quality signals so the squad always knows where quality stands
Your Experience
- 5+ years of QA experience in a SaaS product environment, with automation at the centre of how you work
- Track record of designing test automation frameworks from the ground up - architecture, tooling choices, conventions - and building the JavaScript/TypeScript end-to-end suites (e.g. Playwright, Cypress) that run on them
- Comfortable working in a production codebase - Git, branches, PRs, code review - and able to read code well enough to locate and understand the surface under test
- CI/CD fluent - you understand what runs when and where tests fit, with a clear point of view on the quality controls that make continuous delivery safe: pipeline gates, smoke checks, rollback signals
- Actively using AI tools in your workflow today - and opinions on where they're strong and where they aren't
- Genuine interest in testing AI products: you've thought about (or worked on) how to test non-deterministic systems, and evals excite you rather than scare you
- Strong exploratory testing instincts and sound test-pyramid judgment - you find the bugs automation can't, and you know where automation investment pays off and where it doesn't
- Comfortable with ambiguity, change, and making calls with incomplete information - you're drawn to small teams and early-stage product work
- A genuine focus on the customer outcome, not just the test count
Bonus
- Experience with LLM eval frameworks - golden datasets, judge models, statistical thresholds
- Experience with adversarial testing of AI systems - prompt injection, jailbreaks, hostile inputs
- API testing chops (REST), performance or accessibility testing experience
- Prior start-up or zero-to-one product experience
Our Stack & How We Work
am:pm is built on .NET 8, Vue 3, SQL Server and a growing set of AI services on Microsoft Azure. The squad manages its own DevOps - Terraform for IaC, Azure DevOps for pipelines. We've woven AI tooling deeply into how we work; if you're excited by the idea of testing AI-native software and using AI to test it, you'll fit right in.
Our culture is very important to us too, and we strongly believe in living out our core values:
- People First: One team globally; our people and customers always come first.
- Find a Way: Like a river, we find a way around all obstacles
- Win or Learn: Our journey for improvement never ends; enjoy the ride!
- Aim High: We dare to set big goals and love the challenge of achieving them against the odds.
Why you’ll love working here!
Taking care of you: Including comprehensive health insurance, access to a premium EAP service and an annual wellness allowance to spend on what helps you feel good - whether that’s towards a gym membership, new running shoes, or mindfulness apps.
Support for families: Generous paid parental leave and flexible, family-friendly support designed to help growing families balance work and life with confidence.
Recognition for tenure and milestones: We celebrate time, commitment, and contribution with milestone rewards and additional leave as you grow your career with us.
Flexible working, built for a global team: With customers and teammates across New Zealand, Australia, the United Kingdom, and North America, we work in a hybrid, remote-friendly way with collaborative offices and the flexibility of a balanced approach to WFH/i-office work.
Ready to Level Up?
Join us and help shape the future of real estate at one of the most exciting SaaS companies born from New Zealand.