{"id":3329098,"url":"https://alion.io/job/aviva-canada-ai-quality-engineering-team-lead","title":"AI Quality Engineering Team Lead","company":{"id":1074183,"name":"Aviva Canada","domain":"aviva.ca","url":"https://alion.io/company/aviva-ca","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"A","score":92,"open_postings":9,"ghost_share":0,"stale_share":0.333,"repost_share":0,"time_to_fill_p50_days":33,"computed_at":"2026-11-03T05:54:40Z"}},"role":"Leadership","role_family":"Leadership","seniority":"lead","employment_type":null,"work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Warsaw, Poland"],"countries":["PL"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":75000,"max_usd":142000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":41},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"CI/CD","optional":false},{"name":"Function Calling","optional":false},{"name":"Hallucination","optional":false},{"name":"LLM Guardrails","optional":false},{"name":"Python","optional":false},{"name":"RAG","optional":false},{"name":"SQL","optional":false},{"name":"Arize Phoenix","optional":true},{"name":"DeepEval","optional":true}],"status":"live","first_seen_at":"2026-10-26T00:00:00Z","employer_posted_date":"2026-10-26","last_verified_at":"2026-11-03T03:35:13Z","board_verified":true,"closed_at":null,"days_open":8,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":8},"description":"Hi, we're glad you're here! We are hiring to our Aviva Services Excellence Centre! Take a look at our job description - maybe it suits you or one of your friends?\nWe are looking for an AI Quality Engineering Team Lead to help establish trustworthy, scalable quality engineering for GenAI and agentic AI solutions at Aviva. You will combine hands-on evaluation with team leadership, helping stakeholders understand whether AI solutions are accurate, safe, reliable and ready for release.\nThis is a role for someone who can connect business risk with datasets, AI behaviour, evaluation metrics and engineering controls. You will shape reusable quality practices, challenge weak evidence and guide engineers in producing clear, auditable findings.\nWhat will you be doing?\nLeading the quality engineering approach for AI solutions, from risk assessment through release and monitoring.\nEvaluating multi-turn, RAG and agentic AI behaviour, including grounding, guardrails and tool use.\nBuilding representative datasets and assessing coverage, balance, leakage, label quality and freshness.\nAssessing correctness, relevance, hallucination, safety, bias, privacy and behavioural drift.\nInvestigating unexpected outcomes across data, prompts, models, retrieval, integration, orchestration and evaluators.\nBuilding reusable evaluation components, automated test pipelines, reporting and evidence-generation patterns for multiple AI products.\nCombining test methods and validating automated evaluators against human judgement.\nUsing baselines, key metrics and risk-based regression tests to support release decisions.\nIntegrating evaluation into CI/CD, observability and engineering workflows with traceability across datasets, prompts, models and metrics.\nProducing concise, auditable findings that explain confidence, uncertainty, customer impact and residual risk.\nPartnering with product, engineering, architecture, data, risk, compliance and quality stakeholders to influence delivery and release decisions.\nCoaching and guiding AI Quality Engineers, planning team work, reviewing outputs and building a collaborative culture of constructive challenge and learning.\nWhat we're looking for:\nSeveral years of relevant professional experience in quality engineering, software testing, software engineering, data science or a related technical field.\nHands-on experience evaluating AI, large language model, RAG or agent-based applications.\nPractical Python skills and experience with APIs, logs, traces, structured datasets and SQL or similar analysis.\nAbility to design repeatable evaluations and automation, not just manual testing or vendor scoring.\nGood understanding of sampling, thresholds, trends and non-deterministic system behaviour.\nExperience with automated testing, version control, CI/CD and maintainable engineering practices.\nAbility to turn technical findings into decision-quality evidence for business, risk and engineering stakeholders.\nExperience coaching or leading engineers in an emerging or ambiguous domain.\nStrong communication and influencing skills, with the confidence to challenge weak evidence.\nNice to have:\nExperience leading projects, coordinating technical workstreams or managing people.\nExperience with AI evaluation or observability tools such as DeepEval, Arize or similar platforms.\nExperience designing shared frameworks, reusable services or platform capabilities.\nKnowledge of responsible AI, model risk, controls, privacy or customer-outcome requirements.\nExperience in financial services, insurance or another regulated industry.\nExposure to cloud AI systems, observability and production support.\nIn Aviva Services Excellence Centre, we provide:\nContract of employment \nPerformance Bonus \nPrivate medical care (ENEL-MED) \nCafeteria system of benefits (e.g. MultiSport card) \nAdditional parent privileges: - baby leave - 6 weeks leave for second parent - shortened working time after parental leave \nEmployer-funded group insurance \nEmployee Shares Plan \nExtra day off for birthday \nPaid time off to volunteer \nAdditional leave acquired over the years in the company (even 5 days more in a year) \nWellbeing Leave - additional two weeks paid leave after 10 years of service \nPsychological support consultations \nAccess to the e-learning platforms \nInternal employee referral program \nHybrid working model from Poland with a few days in Warsaw office per month\nAttractive office location in Warsaw - Gdański Business Center \nAdditional entitlements for people with disabilities - Work Without Barriers \nWork environment that promotes diversity - Diversity Charter \nDo you see yourself in Aviva Services Excellence Centre? Be sure to apply!\nAviva is for everyone!\nWe’ve been ranked the #1 Best Workplace in Poland for a second year in a row and a top choice for Millennials by Great Place to Work®!\nAs a signatory to the Diversity Charter, Aviva applies a policy of equal treatment and creates a work environment that is friendly to people with disabilities. We do not require from candidates the date of birth, marital status and attached photos to your CV. We encourage all individuals regardless of gender to apply.\nMost of our people are smart working - spending also time in our office - combining the benefits of flexibility, with time together with colleagues.\nWe’re inclusive - we welcome applications from people with diverse backgrounds and experiences.\nExcited but not sure you tick every box? Research tells us that women, particularly, feel this way. So, regardless of gender, why not apply and try!\nTo find out more about current job offers at Aviva take a look here.","description_format":"text","description_chars":5615,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Cafeteria","Hybrid work","Parental leave"],"hiring_locations":[{"name":"Poland","iso":"PL","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Property & Casualty Insurance"],"lifecycle":[{"event":"open","at":"2026-10-27T11:25:49Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":8,"expected_fill_days":33,"reasons":["conf:2","velocity","win:early"],"computed_at":"2026-11-03T05:54:40Z"},"pay":null,"html_url":"https://alion.io/job/aviva-canada-ai-quality-engineering-team-lead","json_url":"https://alion.io/job/aviva-canada-ai-quality-engineering-team-lead.json","meta":{"generated_at":"2026-11-03T06:11:59Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":8525,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-04T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1074183},"rest":"https://alion.io/mcp/rest/get_company?id=1074183"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Faviva-canada-ai-quality-engineering-team-lead"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Faviva-canada-ai-quality-engineering-team-lead"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Faviva-canada-ai-quality-engineering-team-lead"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/aviva-canada-ai-quality-engineering-team-lead\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Faviva-canada-ai-quality-engineering-team-lead"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3329098}},{"tool":"search_jobs","arguments":{"company":"aviva.ca","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3329098},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}