{"id":4063660,"url":"https://alion.io/job/aptiv-ai-qa-evaluation-engineer","title":"AI QA Evaluation Engineer","company":{"id":5724,"name":"Aptiv","domain":"aptiv.com","url":"https://alion.io/company/aptiv","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workday","truth_index":{"grade":"B","score":79,"open_postings":202,"ghost_share":0,"stale_share":0.832,"repost_share":0,"time_to_fill_p50_days":44,"computed_at":"2026-11-11T05:49:22Z"}},"role":"QA","role_family":"QA","seniority":null,"employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Dublin, Ireland"],"countries":["IE"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"Agile","optional":false},{"name":"AI Agents","optional":false},{"name":"Kanban","optional":false},{"name":"Azure","optional":true},{"name":"CI/CD","optional":true},{"name":"Copilot Studio","optional":true},{"name":"Hallucination","optional":true}],"status":"live","first_seen_at":"2026-11-09T00:00:00Z","employer_posted_date":"2026-11-09","last_verified_at":"2026-11-11T19:18:15Z","board_verified":true,"closed_at":null,"days_open":3,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":3},"description":"AI QA / Evaluation Engineer (Enterprise AI Agents)\nThe QA / Evaluation Engineer will work directly with business stakeholders and engineering teams to define, execute and continuously improve testing and evaluation practices for enterprise AI agents and agentic workflows.\nReporting to the AI Platform Manager, this role is responsible for ensuring all AI solutions are functional, reliable, compliant and fit for purpose prior to release. The role owns evaluation frameworks and test standards, and is accountable for maintaining development and test environments to support consistent, high-quality delivery.\nThe role will also liaise directly with business users to support user acceptance testing and to validate end-user built agents ahead of publishing, ensuring alignment with enterprise standards, governance requirements and quality thresholds.\nResponsibilities\nDefine and implement testing and evaluation frameworks for AI agents and agentic workflows, covering functional quality, groundedness, safety, reliability, and performance.\nDevelop and execute test plans, test cases, evaluation criteria, test data, datasets, and benchmark scenarios to ensure consistent and repeatable validation.\nPartner with DevOps Agent Engineers and Release / Operations Engineers to validate AI solutions throughout build, integration, pre-production, and deployment phases.\nWork with AI Architects to ensure evaluation approaches align with solution design, risk profiles, enterprise standards, and responsible AI requirements.\nOwn and maintain development and test environments for AI agents, ensuring stability, availability, and alignment with enterprise platform standards.\nSupport business users during user acceptance testing and validate end-user developed agents prior to publishing, ensuring solutions are fit for purpose and meet enterprise quality and governance requirements.\nIdentify, log, and track defects, working closely with engineering teams to support resolution, retesting, and continuous improvement of testing practices, tooling, and automation.\nCollaborate with Cyber Security, Legal, and Data Privacy teams to ensure testing addresses compliance, security, and responsible AI requirements, while maintaining clear documentation of test results, quality metrics, and evaluation outcomes.\nRequired Qualifications\nBachelor’s degree in Computer Science, Software Engineering, Information Systems or a related field.\nExperience in software testing, QA engineering or validation roles within enterprise technology environments.\nStrong understanding of test design, test execution and defect management practices.\nExperience working with development teams in agile or Kanban-based delivery environments.\nAbility to define structured test approaches for complex systems, including integration and end-to-end scenarios.\nExperience working across engineering, architecture and control functions in a governed enterprise environment.\nStrong analytical and problem-solving skills with attention to detail.\nUnderstanding of AI/ML or automation solutions and their unique testing considerations is highly desirable.\nPreferred Qualifications\nExperience with AI evaluation frameworks, including groundedness, safety, hallucination detection and performance testing.\nFamiliarity with Microsoft Copilot Studio, Azure AI Foundry or similar enterprise AI platforms.\nExperience with test automation tools, scripting or CI/CD-integrated testing approaches.\nExposure to data validation, prompt testing or evaluation dataset design.\nExperience working with enterprise governance, responsible AI, security and compliance requirements.\nExperience operating in global, matrixed or regulated enterprise environments.\nWhy join Aptiv\nYou grow at Aptiv.\nAptiv's winning culture is global by design. We bring together diverse perspectives, cultures, and business contexts to solve our customers' toughest challenges. In this environment, every individual can grow, lead, and make an impact, regardless of background, because inclusion fuels how we innovate and win.\nYou make an impact at Aptiv.\nOne of the strengths of humanity is our drive to progress, to improve, to achieve more tomorrow than we did yesterday. People need solutions they can trust when it matters most. At Aptiv, you are building those solutions every day.\nYou have support at Aptiv.\nWe ensure you have the resources and support you need to take care of your family, your physical health, and your mental health with a competitive benefits package.\nYour Benefits at Aptiv:\nBenefits/Perks: Personal holidays, Healthcare, Pension, Tax saver scheme, Free Onsite Breakfast & Lunch, Discounted Corporate Gym Membership.\nMulticultural environment\nLearning, professional growth and development in a world-recognized international environment.\nAccess to internal & external training, coaching & certifications.\nRecognition for innovation and excellence.\nAccess to transportation: Grand Canal Dock is well-connected to public transportation, including DART trains, buses, and bike-sharing services, making it easy to get to and from the area.\nAt Aptiv, we deliver innovative solutions that people trust when it matters most.\nProgress is never static. It is about reaching further, exploring new possibilities, and enabling mobility. Aptiv empowers customers to create solutions people rely on-advancing safety and autonomy in Automotive and Commercial Vehicles, enabling mission-critical performance in Aerospace & Defense, and powering intelligent systems across Energy, Enterprise, and Telecommunications. We also drive precision and efficiency in Industrials & Robotics and support life-critical innovation in Medical technologies.\nGuided by our winning culture and values to Think & Act Like Owners, as One Team, and Play to Win, we turn commitment into results and achieve more together.\nJoin Us to Deliver Innovative Solutions That People Trust!\nPrivacy Notice - Active Candidates: https://www.aptiv.com/privacy-notice-active-candidates\nAptiv is an equal employment opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, gender identity, sexual orientation, disability status, protected veteran status or any other characteristic protected by law.","description_format":"text","description_chars":6298,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Gym membership"],"hiring_locations":[{"name":"Ireland","iso":"IE","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Passive Components & Connectors","Automotive Components","Automotive Software","Electrical Equipment & Cables"],"lifecycle":[{"event":"open","at":"2026-11-11T10:09:53Z"}],"visa":[{"country":"IE","licensed_sponsor":true,"adverts_offering_sponsorship_24m":0,"sanctioned_now":false,"evidence":"Irish employment permits in 12 months: 11","filings_12m":11,"filings_prev_12m":13,"green_card_filings_12m":null,"median_offered_wage_usd":null,"route":"Employment permits","cap_exempt":false,"checked_at":"2026-11-12T07:36:15+00:00","sources":["Department of Enterprise, Trade and Employment: employment permits issued to companies"],"filings_for_role_12m":0}],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":3,"expected_fill_days":45,"reasons":["conf:14","win:early","comp:brand"],"computed_at":"2026-11-12T09:42:26Z"},"pay":null,"html_url":"https://alion.io/job/aptiv-ai-qa-evaluation-engineer","json_url":"https://alion.io/job/aptiv-ai-qa-evaluation-engineer.json","meta":{"generated_at":"2026-11-12T09:42:26Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":19738,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-13T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":5724},"rest":"https://alion.io/mcp/rest/get_company?id=5724"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Faptiv-ai-qa-evaluation-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Faptiv-ai-qa-evaluation-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Faptiv-ai-qa-evaluation-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/aptiv-ai-qa-evaluation-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Faptiv-ai-qa-evaluation-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4063660}},{"tool":"search_jobs","arguments":{"company":"aptiv.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4063660},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}