{"id":3785939,"url":"https://alion.io/job/expedia-group-principal-product-manager-agentic-evals","title":"Principal Product Manager, Agentic Evals","company":{"id":6644,"name":"Expedia Group","domain":"expedia.com","url":"https://alion.io/company/expedia","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"A","score":86,"open_postings":19,"ghost_share":0,"stale_share":0.579,"repost_share":0,"time_to_fill_p50_days":42,"computed_at":"2026-11-14T05:58:10Z"}},"role":"Product","role_family":"Product","seniority":"lead","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Washington, United States","San Jose, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":242000,"max":338500,"currency":"USD","period":"year","gross":null,"usd_annual":338500},"salary_estimate":null,"experience_years_min":10,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"LLM","optional":true}],"status":"live","first_seen_at":"2026-11-04T00:00:00Z","employer_posted_date":"2026-11-04","last_verified_at":"2026-11-14T16:16:10Z","board_verified":true,"closed_at":null,"days_open":11,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":11},"description":"At Expedia Group, we help travelers explore the world, one journey at a time. As a global travel company powered by passionate people, trusted partnerships, and leading technology, we connect travelers, partners, and advertisers through our consumer brands, B2B network, and travel advertising business.\nHere, you'll do meaningful work that helps millions of people discover, book, and experience travel with more ease, confidence, and joy. Our five Behaviors-Traveler First, Think Big, Operate with Excellence, Ownership Mindset, and Succeed Together-help foster a supportive environment where people can grow their careers and have the flexibility, benefits, and support to do their best work. Join us and build for travelers everywhere.\nPrincipal Product Manager, Agentic Evals\nIntroduction to The Team:\nThe Product Team creates high-quality end-to-end experiences for travelers, partners, and Expedia Group. Our focus on customer-centric innovation enables us to develop products that build loyalty and repeat business. We partner closely with teams across Expedia Group to drive growth and achieve results for our customers and the company.\nExpedia Group powers travel for everyone, everywhere. As AI becomes core to how travelers plan, book, and manage their trips, our ability to evaluate quality, safety, and performance becomes essential. You will build the frameworks and tools that ensure every AI experience meets a high standard of clarity, accuracy, and trust.\nThe Core AI Experiences team defines the foundational methods for evaluating AI across the company. We create the metrics, tooling, and processes that guide how teams measure accuracy, safety, latency, helpfulness, and long-term traveler outcomes.\nIn this role, you will lead the strategy and build-out of Expedia Group’s AI evaluation framework. You will work closely with engineering, data science, and product teams to drive consistency and support decisions grounded in evidence. Your work will influence every AI-powered experience across Expedia Group and set the bar for quality, safety, and trust company-wide.\nIn this Role You Will:\nDefine Expedia Group’s AI evaluation framework, principles, and adoption strategy\n\nBuild tools and workflows that support reliable, scalable measurement across brands\n\nPartner with engineering and data science on metric design, instrumentation, and model evaluation\n\nEstablish clear, durable standards for accuracy, safety, transparency, and performance\n\nPublish evaluation insights, scorecards, and recommendations that shape roadmaps\n\nInfluence strategic decisions on AI investments and risk management\n\nDrive alignment across product, engineering, data science, and platform teams on evaluation strategy, standards, and prioritization\n\nCreate governance models that ensure consistency and responsible deployment at scale\n\nShape cross-team operating models and evaluation practices that guide how AI products are built and deployed across the company\n\nMinimum Requirements:\nBachelor's degree in a technical, quantitative, or related field, or equivalent practical experience.\n\n10+ years of product management experience, including significant work with AI, ML, data platforms, or evaluation systems\n\nStrong technical depth and experience partnering with DS/ML teams\n\nProven ability to build frameworks, metrics, or tools that scale across large organizations\n\nExperience with model evaluation, instrumentation, experimentation, or system reliability\n\nStrong written and verbal communication that brings clarity to complex technical spaces\n\nTrack record of influencing cross-functional leaders and driving alignment\n\nPreferred Qualifications\nExperience building evaluation platforms, ML tooling, or governance frameworks\n\nFamiliarity with LLM behavior, model evaluation methods, or prompt testing\n\nUnderstanding of trustworthy AI principles, safety evaluation, and failure-mode analysis\n\nThe total cash range for this position in Seattle is $224,000.00 to $313,500.00. Employees in this role have the potential to increase their pay up to $358,500.00, which is the top of the range, based on ongoing, demonstrated, and sustained performance in the role.\nThe total cash range for this position in San Jose is $242,000.00 to $338,500.00. Employees in this role have the potential to increase their pay up to $387,000.00, which is the top of the range, based on ongoing, demonstrated, and sustained performance in the role.Starting pay for this role will vary based on multiple factors, including location, available budget, and an individual’s knowledge, skills, and experience. Pay ranges may be modified in the future.\nBenefits and perks\nExpedia Group offers benefits and perks designed to support employees and their families, including medical, dental, and vision coverage, paid time off, an Employee Assistance Program, wellness and travel reimbursement, travel discounts, and International Airlines Travel Agent Network (IATAN) membership. Learn more about life at Expedia Group at https://careers.expediagroup.com/life.\nAccommodation requests\nExpedia Group is committed to providing an inclusive and accessible recruiting experience. If you need an accommodation or adjustment due to a disability during the application or recruiting process, please submit a request athttps://expedia.service-now.com/askeg?id=job_accommodation.\nAbout Expedia Group\nExpedia Group includes three flagship consumer brands - Expedia, Hotels.com, and Vrbo - along with a leading B2B travel business and travel advertising offerings. Across our brands and business, we help travelers explore the world with confidence and ease.\nImportant notice\nEmployment opportunities and job offers at Expedia Group will always come from Expedia Group's Talent Acquisition and hiring teams. Never share sensitive personal information unless you are confident of the recipient. Expedia Group does not extend job offers via email or messaging tools to individuals with whom we have not made prior contact. Our email domain is @expediagroup.com. The official place to find and apply for roles is https://careers.expediagroup.com/jobs/.\nEqual Opportunity\nExpedia is committed to creating an inclusive work environment with a diverse workforce. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, veteran status, or any other characteristic protected by law. This employer participates in E-Verify. The employer will provide the Social Security Administration (SSA) and, if necessary, the Department of Homeland Security (DHS) with information from each new employee's I-9 to confirm work authorization.","description_format":"text","description_chars":6744,"description_truncated":false,"requirements":{"experience_years_min":10,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Travel & Tourism","Vacation Rentals","Travel Agencies"],"lifecycle":[{"event":"open","at":"2026-11-05T03:35:26Z"}],"visa":[{"country":"US","licensed_sponsor":true,"adverts_offering_sponsorship_24m":0,"sanctioned_now":false,"evidence":"H-1B filings in 12 months: 447 · green card filings: 123","filings_12m":447,"filings_prev_12m":513,"green_card_filings_12m":123,"median_offered_wage_usd":145342,"route":null,"cap_exempt":false,"checked_at":"2026-11-14T07:33:33+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","US Department of Labor: PERM disclosure data (green cards)"],"filings_for_role_12m":89}],"liveness":{"score":63,"band":"ok","label":"Likely open","p_open":1,"p_active":0.632,"p_room":1,"age_days":10,"expected_fill_days":42,"reasons":["conf:4","stale_co","velocity","win:early","comp:brand"],"computed_at":"2026-11-14T05:58:10Z"},"pay":{"stated_usd_annual":338500,"is_top_pay":true},"html_url":"https://alion.io/job/expedia-group-principal-product-manager-agentic-evals","json_url":"https://alion.io/job/expedia-group-principal-product-manager-agentic-evals.json","meta":{"generated_at":"2026-11-15T01:30:04Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2807,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-16T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":6644},"rest":"https://alion.io/mcp/rest/get_company?id=6644"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fexpedia-group-principal-product-manager-agentic-evals"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fexpedia-group-principal-product-manager-agentic-evals"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fexpedia-group-principal-product-manager-agentic-evals"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/expedia-group-principal-product-manager-agentic-evals\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fexpedia-group-principal-product-manager-agentic-evals"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3785939}},{"tool":"search_jobs","arguments":{"company":"expedia.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3785939},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}