{"id":1360495,"url":"https://alion.io/job/pearson-senior-quality-assurance-engineer","title":"Senior Quality Assurance Engineer","company":{"id":34029,"name":"Pearson","domain":"pearson.com","url":"https://alion.io/company/pearson","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Oracle","truth_index":{"grade":"B","score":80,"open_postings":20,"ghost_share":0,"stale_share":0.8,"repost_share":0,"time_to_fill_p50_days":26,"computed_at":"2026-10-17T05:46:00Z"}},"role":"QA","role_family":"QA","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Sri Lanka"],"countries":["LK"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":18000,"max_usd":51000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":1081},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"CI/CD","optional":false},{"name":"Confluence","optional":false},{"name":"Cypress","optional":false},{"name":"Embeddings","optional":false},{"name":"Hallucination","optional":false},{"name":"Human-in-the-Loop","optional":false},{"name":"JavaScript","optional":false},{"name":"Jira","optional":false},{"name":"JMeter","optional":false},{"name":"k6","optional":false},{"name":"LLM","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"Locust","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"NLP","optional":false},{"name":"Playwright","optional":false},{"name":"Python","optional":false},{"name":"RAG","optional":false},{"name":"Rest API","optional":false},{"name":"Selenium","optional":false},{"name":"Semantic Search","optional":false},{"name":"Semantic Search","optional":false},{"name":"State Management","optional":false}],"status":"live","first_seen_at":"2026-07-20T10:35:11Z","employer_posted_date":"2026-07-20","last_verified_at":"2026-10-17T16:50:29Z","board_verified":true,"closed_at":null,"days_open":89,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":89},"description":"Job Title: Senior Quality Assurance Engineer\nDescription: Senior Quality Assurance Engineer role focused on testing and improving an LLM-powered AI Support Assistant, with emphasis on conversational AI quality, RAG validation, integration testing, automation, and scalable quality monitoring across EdTech platforms. This role aligns to industry level titles such as Lead Quality Engineer, Senior Software Test Engineer, Senior AI Test Engineer\nAbout the Role\nWe’re looking for a Senior Quality Engineer to own the quality strategy for our AI Support Assistant, a context-aware, LLM-powered support tool embedded across our digital learning platform, including admin consoles and student-facing dashboards. This assistant uses runtime context (user role, product, browser, error metadata) to deliver accurate, personalized troubleshooting to millions of students and instructors annually. This is a hands-on QE role centered on AI and conversational testing - you’ll spend most of your time evaluating LLM response quality, not traditional UI testing.\nYou’ll be responsible for ensuring the assistant is accurate, reliable, and delivers a seamless experience across a complex integration landscape spanning multiple LMS platforms, e-commerce flows, and entitlement systems.\nWhat You’ll Do\nAI & Conversational Testing\nDesign and execute test strategies for LLM-powered conversational flows, including response accuracy, relevance, assumption detection, and hallucination prevention\nBuild and maintain evaluation datasets to measure assistant performance against target metrics (95% accuracy, 75% CSAT)\nValidate context injection - ensure the assistant correctly receives and uses runtime context (role, product, LMS type, browser, workflow state, structured error/diagnostic codes) to tailor responses\nIntegration & End-to-End Testing\nTest the context-sharing contract between front-end applications and the AI Support Assistant - verifying session storage writes, schema compliance, and read interfaces\nValidate cross-platform behavior across major LMS integrations and multiple product lines across the courseware portfolio\nTest entitlement and enrollment and third-party content-provider provisioning flows\nQuality Infrastructure\nBuild automated regression suites for assistant response quality, context propagation, and UI behavior (chat widget placement, discoverability, mobile responsiveness)\nDevelop performance and load testing strategies for projected scale (millions of interactions annually, growing year over year)\nEstablish monitoring and alerting for production assistant accuracy, escalation rates, and context-pass-through failures\nCollaboration\nPartner with ML/AI engineers to define acceptance criteria for model updates and prompt changes\nWork closely with product management to translate user research findings (e.g., accuracy as the #1 trust driver, step-based answers over article links) into testable requirements\nCoordinate with backend platform and Customer Success teams on dependency validation\nWhat You Bring\nRequired\n5+ years in software quality engineering, with at least 2 years testing AI/ML systems, conversational AI, chatbots, or NLP-driven products\nHands-on experience testing LLM-based applications - prompt evaluation, response grading, factual accuracy measurement, and regression testing for non-deterministic outputs\nHands-on experience validating Retrieval-Augmented Generation (RAG) pipelines, including retrieval accuracy, context grounding, and relevance validation\nExperience testing semantic search and vector-based retrieval systems (embeddings, similarity scoring, ranking relevance)\nExperience with LLM Evaluator frameworks and LLM-as-judge methodologies for automated, scalable scoring of model outputs\nExperience building evaluation harnesses for LLM outputs, including automated scoring and human-in-the-loop review pipelines\nStrong experience with API testing (REST) and integration testing across distributed systems\nProficiency with test automation frameworks (Selenium, Playwright, Cypress, or similar) and CI/CD pipelines\nStrong scripting/programming ability (e.g., Python, JavaScript, or Java) to build custom test tooling, evaluation harnesses, and data pipelines\nExperience testing across multiple browsers, devices, and platforms - including mobile web\nSolid understanding of session/local storage, client-side state management, and front-end data contracts\nFamiliarity with e-commerce or EdTech platforms - checkout flows, entitlements, user provisioning\nPreferred\nExperience with LMS platforms (e.g., Canvas, Blackboard, Moodle) and LTI integrations\nFamiliarity with MCP (Model Context Protocol) or similar AI tool-integration patterns\nKnowledge of accessibility testing (WCAG 2.1 AA) for embedded chat interfaces\nExperience with performance/load testing tools (k6, Locust, JMeter)\nFamiliarity with Jira, Confluence, and Agile workflows\nWhy This Role Matters\nThe AI Support Assistant is Pearson’s first LLM-powered, context-aware support experience - and quality is the make-or-break factor. User research is clear: accuracy is the #1 driver of trust. If the assistant gives wrong answers, users won’t come back. Your work directly determines whether millions of students get instant, reliable help or hit a dead end.\nOur Stack \nJira · Confluence · LMS Platforms (Canvas, Blackboard, Moodle) · E-commerce Platform · LTI 1.3 · REST APIs · Session Storage · LLM/AI Services\nWhat We Offer\nCompetitive salary and performance-based bonus.\nComprehensive health, dental, and vision benefits.\nGenerous PTO, holidays, and flexible working arrangements.\nAnnual learning and development budget.\nAccess to Pearson's full catalog of learning products and certifications.\nOpportunity to shape quality practices across a globally recognized education technology organization.\nAbout Pearson \nPearson is the world's leading learning company, delivering education to over 160 countries. Our digital-first strategy powers millions of learners through a growing portfolio of integrated learning tools, assessments, and LMS-connected experiences. We are committed to building inclusive, equitable, and effective digital learning-and quality engineering is at the heart of that mission.","description_format":"text","description_chars":6228,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Flexible schedule"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Commerce"],"lifecycle":[{"event":"open","at":"2026-09-27T23:45:59Z"}],"visa":[],"liveness":{"score":10,"band":"cold","label":"Long shot","p_open":1,"p_active":0.369,"p_room":0.28,"age_days":88,"expected_fill_days":26,"reasons":["conf:5","stale_co","velocity","win:tail","crowd:brand"],"computed_at":"2026-10-17T05:46:00Z"},"pay":null,"html_url":"https://alion.io/job/pearson-senior-quality-assurance-engineer","json_url":"https://alion.io/job/pearson-senior-quality-assurance-engineer.json","meta":{"generated_at":"2026-10-17T20:21:42Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":4144,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":34029},"rest":"https://alion.io/mcp/rest/get_company?id=34029"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fpearson-senior-quality-assurance-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fpearson-senior-quality-assurance-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fpearson-senior-quality-assurance-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/pearson-senior-quality-assurance-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fpearson-senior-quality-assurance-engineer"}]}