{"id":4523246,"url":"https://alion.io/job/doctolib-staff-applied-ai-software-engineer-python-ai-evaluation-xfm","title":"Staff Applied AI Software Engineer (Python) – AI Evaluation (x/f/m)","company":{"id":52886,"name":"Doctolib","domain":"doctolib.fr","url":"https://alion.io/company/doctolib","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"B","score":84,"open_postings":62,"ghost_share":0.016,"stale_share":0.565,"repost_share":0.065,"time_to_fill_p50_days":42,"computed_at":"2026-11-18T05:48:20Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"staff","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["France"],"countries":["FR"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":6,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"CI/CD","optional":false},{"name":"Embeddings","optional":false},{"name":"LLM","optional":false},{"name":"Machine Learning","optional":false},{"name":"Python","optional":false}],"status":"live","first_seen_at":"2026-11-18T09:22:28Z","employer_posted_date":"2026-11-18","last_verified_at":"2026-11-18T09:27:28Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"What we do\nJoin a team of passionate and hardworking entrepreneurs to transform healthcare!\nWorking in the tech team at Doctolib involves building innovative products and features to improve the daily lives of care teams and patients. We work in feature teams in an agile environment, while collaborating with product, design, and business teams.\nWhat you'll do\nWe are looking for a Staff AI Software Engineer to join the AI Evaluation team (EvalX) at Doctolib and help every AI product team measure, understand and improve the quality, reliability and safety of their systems.\nOur AI products span patient-facing conversational agents, AI-powered phone assistants and clinical assistants supporting healthcare professionals. EvalX builds the platform, tools and methods that let these teams evaluate continuously, and our ambition is to make evaluation a core engineering practice.\nIn this role, you will work at the intersection of platform engineering, applied AI and product teams: you build shared evaluation capabilities, and you work directly with the teams that use them so they actually get adopted.\nIn this role, you will:\nDesign, ship, and operate production-grade evaluation tooling (APIs, services, CI/CD integrations) used by AI product teams across Doctolib\n\nWork hands-on with product teams to assess how they evaluate today, define improvement plans, and help them own their evaluation quality\n\nDesign evaluation strategies for agentic systems, combining deterministic checks, LLM-as-a-judge, simulation-based testing, and human review\n\nBuild intelligent capabilities for production sampling, dataset representativeness monitoring, distribution-shift detection, and test case generation\n\nHelp teams define meaningful quality metrics, representative datasets, and reliable quality gates throughout the development lifecycle\n\nTurn recurring product needs into reusable, self-service platform capabilities, and measure success through adoption, team autonomy, and evaluation coverage\n\nContribute to cross-functional initiatives that shape Doctolib's AI platform and best practices for building and evaluating agentic systems\n\nWho you are\nBefore you read on, if you don't have the exact profile described below, but you feel this job description matches your skill set, we still encourage you to apply! You could be our next team mate if you:\nHave 6+ years of experience as a software engineer, with significant experience using Python; or as a machine learning engineer\n\nHave built or operated evaluation for LLM-powered or agentic systems: datasets, LLM-as-a-judge, human review, production-driven evaluation\n\nUnderstand and promote software development best practices\n\nAre product-oriented and enjoy working directly with other teams, challenging assumptions, and turning complex needs into concrete solutions\n\nCan influence engineering practices and drive change without hierarchical authority: your job is not done until your solution is adopted\n\nAre comfortable speaking and writing in English in your day-to-day with the team\n\nNow, it would be fantastic if you:\nHave experience with applied ML techniques such as embeddings, clustering, statistical sampling, and distribution monitoring\n\nHave experience building developer platforms or internal tooling, or in a Forward Deployed Engineering role\n\nHave experience in the medical domain or another regulated environment\n\nWhat we offer\nFree comprehensive health insurance (basic package) for you and your children\n\n25 days of paid vacation per year, plus up to 14 days of RTT\n\nFree mental health and coaching services through our partner Moka.care\n\nWork from abroad for up to 10 days per year thanks to our flexibility days policy\n\nLunch vouchers (Swile card) worth €8.50 per working day, with €4.50 covered by Doctolib\n\nA subsidy from the work council to refund part of the membership to a sport club or a creative class\n\n50% reimbursement of your public transport subscription\n\nEnrollment in Doctolib's long-term employee value sharing plan called DoctoGrowth\n\nParentCare Program: Enjoy full salary coverage (100%) during your first month of birth leave, and 75% during the second, covered by Doctolib\n\nFor caregivers and workers with disabilities, a package including an adaptation of the remote policy, extra days off for medical reasons, and psychological support\n\nThe interview process\nA first interview with a member of the Talent Acquisition team\n\nA coding interview\n\nAn agentics system design\n\nA behavioural interview\n\nReference check\n\nJob details\nPermanent position\n\nFull Time\n\nWorkplace: Levallois\n\nStart date : As soon as possible\n\nIf you would like to find out more about tech life at Doctolib, feel free to read our latest Medium blog articles!\nAt Doctolib, we believe in improving access to healthcare for everyone - regardless of where you come from, what you look like. This translates into our recruitment process: Doctolib is an equal opportunity employer. We don't just accept diversity at Doctolib, we respect and celebrate it!\nThe more diverse ideas are heard, the more our product will truly improve healthcare for all. You are welcome to apply to Doctolib, regardless of your gender, religion, age, sexual orientation, ethnicity, disability, or place of origin. If you have a disability, let us know if there's any way we can make the interview process smoother for you!\nAll the information transmitted via this form is processed by Doctolib for the purpose of managing applications. For more information on how Doctolib processes your application data, click here.\nIf you wish to exercise your rights or if you have any questions about the processing of your data, you can write to us at .","description_format":"text","description_chars":5708,"description_truncated":false,"requirements":{"experience_years_min":6,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[{"language":"English","level":"All levels","optional":false}]},"benefits":["Health insurance"],"hiring_locations":[{"name":"France","iso":"FR","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Practice Management Software","Patient Access & Booking"],"lifecycle":[{"event":"open","at":"2026-11-18T09:27:28Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":42,"reasons":["conf:4","win:early","comp:brand"],"computed_at":"2026-11-18T14:07:04Z"},"pay":null,"html_url":"https://alion.io/job/doctolib-staff-applied-ai-software-engineer-python-ai-evaluation-xfm","json_url":"https://alion.io/job/doctolib-staff-applied-ai-software-engineer-python-ai-evaluation-xfm.json","meta":{"generated_at":"2026-11-18T14:07:04Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":9722,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-19T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":52886},"rest":"https://alion.io/mcp/rest/get_company?id=52886"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fdoctolib-staff-applied-ai-software-engineer-python-ai-evaluation-xfm"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fdoctolib-staff-applied-ai-software-engineer-python-ai-evaluation-xfm"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fdoctolib-staff-applied-ai-software-engineer-python-ai-evaluation-xfm"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/doctolib-staff-applied-ai-software-engineer-python-ai-evaluation-xfm\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fdoctolib-staff-applied-ai-software-engineer-python-ai-evaluation-xfm"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4523246}},{"tool":"search_jobs","arguments":{"company":"doctolib.fr","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4523246},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}