{"id":2169225,"url":"https://alion.io/job/codurance-sdet-qa-engineer-ai-evaluation","title":"SDET / QA Engineer (+AI evaluation)","company":{"id":3322,"name":"Codurance","domain":"codurance.com","url":"https://alion.io/company/codurance","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workable","truth_index":null},"role":"QA","role_family":"QA","seniority":null,"employment_type":"full_time","work_mode":"remote","remote_scope":"stated_regions","remote_scope_basis":"posting_text","remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["Barcelona, Spain"],"countries":["ES"],"hiring_countries":["DE","FR","AT","BE","BG","HR","CY","CZ","DK","FI","HU","IE","IT","NL","PL","RO","ES","SE","EE","GR","LV","LT","LU","MT","PT","SK","SI"],"hiring_countries_total":27,"salary":{"min":38000,"max":43000,"currency":"EUR","period":"year","gross":null,"usd_annual":48158},"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":true,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Appium","optional":false},{"name":"CI/CD","optional":false},{"name":"Cucumber","optional":false},{"name":"Jenkins","optional":false},{"name":"LLM","optional":false},{"name":"Playwright","optional":false},{"name":"Python","optional":false},{"name":"Selenium","optional":false},{"name":"SonarQube","optional":false},{"name":"Agile","optional":true},{"name":"Anthropic","optional":true},{"name":"LLM Evaluation","optional":true},{"name":"Windows","optional":true}],"status":"live","first_seen_at":"2026-10-09T11:25:35Z","employer_posted_date":"2026-10-09","last_verified_at":"2026-10-12T03:45:20Z","board_verified":true,"closed_at":null,"days_open":2,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":2},"description":"We are looking for a QA Engineer or Software Development Engineer in Test (SDET) who shares the same values of pragmatism, professionalism, and transparency that we do. Working from anywhere in Spain, we support our clients in many ways and you will have the opportunity to work on a wide range of projects.\nYou will evaluate, validate and improve quality across teams, participate in developing and executing exploratory and automated tests to ensure software quality. You might also review system requirements and track quality assurance metrics (e.g. pass rate, defect densities and open defect counts.) as well as liaise with various stakeholders (technical and business) from Codurance and our client’s teams.\nResponsibilities\nTake ownership of a Python LLM evaluation tool, diagnostic suite, and golden datasets, with the goal of raising pass rates on production AI agents.\nRefine scoring criteria and design multi-turn conversational evaluations for AI agents.\nSet up automated regression testing for core product flows using Playwright, covering integrated workflows and third-party channels.\nImplement SonarQube quality gate monitoring and daily deviation reporting\nPartner with senior stakeholders on the QA strategy, standards, and continuous improvement\nEstimate, prioritize, plan, and coordinate testing activities\nStay up-to-date with new testing tools and test strategies\nRequirements\nAt least4 years of experience in a similar role\nFluent in Spanish and English (speaking and writing)\n Consultancy experience and an open mind around what is quality (adapting to clients)\nHands-on experience with Playwright, SonarQube and other testing tools (Cucumber, Selenium, Appium, Jenkins)\nKnowledge of CI/CD tools and processes\nContinuous learning (keep up to date with advancements in the market), curiosity\nStrong programming skills with Python (other stacks are a plus)\nPractical experience in AI evaluation (LLM evals, agent testing, multi-turn chat scoring, golden dataset management).\nIt would be a bonus if you also have:\n QA strategy knowledge (holistic testing, Agile testing Manifesto, ...)\nAnthropic certification\nExperience with performance/load testing for AI agents /call center agents, and multi-channel integration\nFamiliarity with BDD and TDD\nISTQB certificate\nBenefits\nWhat's in it for you?\nThe gross annual salary range for this role is 38,000€ to 43,000 €.\nTransparency - all of our salary bandings and company finances are available to everyone from day one.\nPersonal Learning Budget - Access to Learnerbly, our online learning & development tool, to support your personal development and career progression here.\nTraining - No fixed training budget. So long as the course is relevant for the company and you'll do a lightning talk on the subject, we’re happy to pay for it.\nWorking 100% remote (or hybrid if you want to drop by the office from time to time)\nPrivate medical and dental insurance\nRestaurant, transport and nursery tickets (retribución flexible/ tax deductible)\nGym discounts\nLanguage classes (Spanish, English, French, German)\nYou choose between Linux, Mac or Windows!\nOur People - You’ll be working alongside Craftspeople whether that’s on a client project or contributing to our internal projects.\n\nWe are interested in every qualified candidate who is eligible to work in the EU. However, we are not able to sponsor visas or provide relocation assistance for this role.\n\nEveryone should have the right to bring their whole self to work and be celebrated for who they are. Our people are hired purely on their commitment to these values and their ambition to deliver outstanding results for our clients. Codurance is proud to be an Equal Opportunities Employer and is committed to fostering an inclusive workplace.","description_format":"text","description_chars":3760,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[{"language":"Spanish","level":"Advanced (C1)","optional":false},{"language":"English","level":"Advanced (C1)","optional":false}]},"benefits":["Continuous learning","Dental insurance","Relocation assistance"],"hiring_locations":[{"name":"Germany","iso":"DE","kind":"region"},{"name":"France","iso":"FR","kind":"region"},{"name":"Austria","iso":"AT","kind":"region"},{"name":"Belgium","iso":"BE","kind":"region"},{"name":"Bulgaria","iso":"BG","kind":"region"},{"name":"Croatia","iso":"HR","kind":"region"},{"name":"Cyprus","iso":"CY","kind":"region"},{"name":"Czech Republic","iso":"CZ","kind":"region"},{"name":"Denmark","iso":"DK","kind":"region"},{"name":"Finland","iso":"FI","kind":"region"},{"name":"Hungary","iso":"HU","kind":"region"},{"name":"Ireland","iso":"IE","kind":"region"},{"name":"Italy","iso":"IT","kind":"region"},{"name":"Netherlands","iso":"NL","kind":"region"},{"name":"Poland","iso":"PL","kind":"region"},{"name":"Romania","iso":"RO","kind":"region"},{"name":"Spain","iso":"ES","kind":"region"},{"name":"Sweden","iso":"SE","kind":"region"},{"name":"Estonia","iso":"EE","kind":"region"},{"name":"Greece","iso":"GR","kind":"region"},{"name":"Latvia","iso":"LV","kind":"region"},{"name":"Lithuania","iso":"LT","kind":"region"},{"name":"Luxembourg","iso":"LU","kind":"region"},{"name":"Malta","iso":"MT","kind":"region"},{"name":"Portugal","iso":"PT","kind":"region"},{"name":"Slovakia","iso":"SK","kind":"region"},{"name":"Slovenia","iso":"SI","kind":"region"}],"hiring_excludes":[],"relocation_offered":true,"industries":["Artificial Intelligence","IT Consulting & Digital Transformation","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-10-09T11:25:35Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":0,"expected_fill_days":52,"reasons":["conf:2","velocity","win:early"],"computed_at":"2026-10-10T05:45:15Z"},"pay":{"stated_usd_annual":48158,"is_top_pay":false},"html_url":"https://alion.io/job/codurance-sdet-qa-engineer-ai-evaluation","json_url":"https://alion.io/job/codurance-sdet-qa-engineer-ai-evaluation.json","meta":{"generated_at":"2026-10-12T05:08:22Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":8197,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-13T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3322},"rest":"https://alion.io/mcp/rest/get_company?id=3322"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcodurance-sdet-qa-engineer-ai-evaluation"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcodurance-sdet-qa-engineer-ai-evaluation"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcodurance-sdet-qa-engineer-ai-evaluation"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/codurance-sdet-qa-engineer-ai-evaluation\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcodurance-sdet-qa-engineer-ai-evaluation"}]}