{"id":61235,"url":"https://alion.io/job/starrez-senior-quality-engineer-ai","title":"Senior Quality Engineer - AI","company":{"id":1958,"name":"StarRez","domain":"starrez.com","url":"https://alion.io/company/starrez","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"A","score":93,"open_postings":7,"ghost_share":0,"stale_share":0.286,"repost_share":0,"time_to_fill_p50_days":35,"computed_at":"2026-10-28T05:59:10Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":null,"work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Melbourne, Australia","Australia"],"countries":["AU"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":94000,"max_usd":246000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":2365},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Hallucination","optional":false},{"name":"LLM","optional":false},{"name":"RAG","optional":false},{"name":"Shift-Left Security","optional":false}],"status":"live","first_seen_at":"2026-05-18T08:47:08Z","employer_posted_date":"2026-10-20","last_verified_at":"2026-10-28T14:22:05Z","board_verified":true,"closed_at":null,"days_open":163,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":162},"description":"About StarRez\nStarRez is the global leader in student housing software, providing innovative solutions for on and off-campus housing management, resident wellness and experience, and revenue generation. Trusted by 1,400+ clients across 25+ countries, StarRez supports more than 4 million beds annually with its user-friendly, all-in-one platform, delivering seamless experiences for students and administrators. With offices in the United States, Australia, the UK, and India, StarRez blends the robust capabilities of a global organization with the personalized care and service of a trusted partner.\nThe Role:\nWe are looking for a Senior Quality Engineer - AI to help shape how StarRez evaluates, validates, and improves AI-powered product experiences.\nYou will play a pivotal role in elevating our quality practices for AI-powered product experiences and tooling. You will be a product expert within our engineering organization, deeply understanding workflows and customer outcomes. This role will balance hands-on individual contributor responsibilities with leading, influencing, and coaching your peers.\nYou’re someone who is passionate about seeing systems holistically and is eager to drive the highest standards of accuracy, safety, usefulness, and reliability. You will help define what \"good\" AI output means for StarRez, build repeatable evaluation systems, coach reviewers and subject matter experts, and turn subjective feedback into measurable product improvement.\nYou will work at the intersection of AI engineering, product, and domain knowledge - partnering with engineers, product managers, and SMEs to raise the bar on how StarRez evaluates, monitors, and improves AI tools.\nKey Responsibilities:\nLead or contributed to end-to-end AI quality and evaluation strategy for product experiences involving LLMs, RAG, prompts, tool-use, or agent workflows.\nProvided quality-focused input during ticket grooming, discovery, and feature discussions, with clear guidance on testability, observability, scalability, reliability, product resilience, customer usability, and operational readiness.\nChampioned shift-left quality practices by influencing or mentoring developers and cross-functional partners on evaluation-driven development, clear acceptance criteria, early validation, and fast feedback loops.\nCoached or calibrated human reviewers, subject matter experts, or product teams by running calibration sessions, reviewing patterns, improving rubric clarity, and converting domain knowledge into reliable evaluation criteria.\nContributed to the evolution of development or quality pipelines, including automated and human evaluation workflows, regression monitoring after model upgrades, prompt changes, RAG updates, or agent workflow changes.\nDefine practical AI quality standards, rubrics, and evaluation criteria that help teams assess AI outputs consistently across StarRez product workflows.\nPartner with engineering and product teams to identify AI quality risks early, including risks related to grounding, hallucination, tool-use behaviour, retrieval quality, customer impact, and safe escalation.\nBuild or support repeatable evaluation approaches that help teams compare prompt, model, RAG, data, or workflow changes before release.\nTranslate AI failure patterns into actionable product and engineering improvements, such as prompt changes, retrieval improvements, source-data fixes, tool safeguards, UX changes, or escalation-flow updates.\nUse metrics, reviewer feedback, quality trends, and operational signals to support release readiness and ongoing product improvement.\nRequired Experience:\nQuality Assurance or Quality Engineering experience in a SaaS B2B or B2B2C company.\nProven track record as a senior individual contributor with a strong quality mindset and the ability to influence technical and product direction.\nExcellent technical communication and collaboration skills, with the ability to guide engineering peers, product partners, and non-technical stakeholders through clear quality insights.\nData-driven approach to quality management, with experience using metrics, dashboards, trend analysis, reviewer agreement, or operational reporting to guide decisions.\nHands-on technical comfort with test automation, evaluation tooling, observability, or scripting, and the ability to partner with engineers to build pragmatic evaluation pipelines.\nHighly regarded:\nExperience with LLM evaluation pipelines, automated eval tooling, LLM judges, or agent observability platforms.\nBackground in traditional QA or quality engineering, particularly in environments transitioning to AI-augmented workflows.\nFamiliarity with safety, bias, and compliance evaluation for AI systems.\nExperience in SaaS or enterprise software environments, ideally with exposure to student housing, property management, or similar domainspecific AI products.\nExposure to production monitoring of AI systems, including drift detection and post-release quality tracking.\nReasons to join our Team:\nYou will be part of a vibrant and supportive culture, with weekly team lunches. \nWe are a flexible and hybrid workplace. \nYou will have access to professional development opportunities. \nBe part of a global team, participate in monthly global meetings and partner with team members in different countries. \nZ-Factor: Our most celebrated value, you will work with a team of caring, energetic, high-performing, and passionate people who have fun supporting our vision, innovation & continuous improvement. \nWe are proud of our diverse workforce and are dedicated to creating a safe and welcoming environment for all employees. People from various ethnicities, ages, genders, and abilities are encouraged to apply.\n Please note, StarRez requires all candidates to have full work rights in Australia. \nNotice to external Recruiters and Recruitment Agencies:\nStarRez will not accept unsolicited resumes from recruitment agencies, headhunters, or any other third parties for this role through this website or directly to any employee. StarRez and any of our subsidiaries will not pay fees to any third-party agency or company. In addition, we ask that you do not reach out to any employee with regards to this position, or any other positions, now, or in the future.\nStarRez is an equal opportunity employer.","description_format":"text","description_chars":6311,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Professional development"],"hiring_locations":[{"name":"Australia","iso":"AU","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Property Management","PropTech"],"lifecycle":[{"event":"open","at":"2026-05-18T08:47:08Z"}],"visa":[],"liveness":{"score":14,"band":"cold","label":"Long shot","p_open":1,"p_active":0.493,"p_room":0.28,"age_days":162,"expected_fill_days":35,"reasons":["conf:0","win:tail","crowd:"],"computed_at":"2026-10-28T05:59:10Z"},"pay":null,"html_url":"https://alion.io/job/starrez-senior-quality-engineer-ai","json_url":"https://alion.io/job/starrez-senior-quality-engineer-ai.json","meta":{"generated_at":"2026-10-28T16:47:21Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":22799,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-29T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1958},"rest":"https://alion.io/mcp/rest/get_company?id=1958"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fstarrez-senior-quality-engineer-ai"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fstarrez-senior-quality-engineer-ai"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fstarrez-senior-quality-engineer-ai"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/starrez-senior-quality-engineer-ai\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fstarrez-senior-quality-engineer-ai"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":61235}},{"tool":"search_jobs","arguments":{"company":"starrez.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":61235},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}