{"id":3282561,"url":"https://alion.io/job/mercado-libre-senior-software-engineer-evals-for-agents","title":"Senior Software Engineer - Evals for Agents","company":{"id":237,"name":"Mercado Libre","domain":"mercadolibre.com","url":"https://alion.io/company/mercado-libre","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Eightfold","truth_index":{"grade":"A","score":88,"open_postings":64,"ghost_share":0.016,"stale_share":0.422,"repost_share":0.063,"time_to_fill_p50_days":36,"computed_at":"2026-10-29T05:49:05Z"}},"role":"Backend","role_family":"Backend","seniority":"senior","employment_type":null,"work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Buenos Aires, Argentina"],"countries":["AR"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Claude","optional":false},{"name":"Clean Architecture","optional":false},{"name":"Gemini","optional":false},{"name":"Go","optional":false},{"name":"Google BigQuery","optional":false},{"name":"LLM","optional":false},{"name":"Python","optional":false},{"name":"Rest API","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-08-20T18:56:15Z","employer_posted_date":"2026-10-26","last_verified_at":"2026-10-29T13:18:14Z","board_verified":true,"closed_at":null,"days_open":69,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":69},"description":"Como Senior Software Engineer Backend en Mercado Libre, diseñarás y escalarás sistemas innovadores y seguros que resuelven problemas reales y de alto impacto. Trabajarás en un entorno dinámico con tecnología de vanguardia, aplicando buenas prácticas de ingeniería, modelos de IA propios y aprendizaje continuo, con el propósito de democratizar el comercio electrónico y los servicios financieros en América Latina.\nImagináte emprendiendo proyectos desafiantes, dinámicos e innovadores y siendo responsable de:\nEvolucionar y mantener la plataforma de evaluación de agentes (online y offline evals) que usan los equipos de MELI que construyen agentes y desarrollar capabilities para entender la salud de los agentes en producción: métricas de calidad, detección de regresiones, observabilidad y alertas.\nConstruir herramientas de experimentación offline: replay de conversaciones reales, usuarios simulados y comparación de versiones de modelos, prompts y configuraciones.\nCrear el ciclo de mejora continua de agentes para transformar los resultados de las evaluaciones en mejoras accionables para los equipos dueños de los agentes: selección de modelos por calidad y costo, ajustes de prompts, tools y flujos conversacionales.\nHacer que la plataforma sea simple de adoptar: APIs, CLI e integraciones con herramientas de desarrollo asistido por IA (Skills, MCPs).\nColaborar con equipos de producto, negocio y tecnología para traducir necesidades concretas en soluciones de impacto.Mantenerse actualizado/a sobre tendencias, modelos y buenas prácticas del ecosistema de IA generativa y evaluación de agentes, en un contexto de evolución constante.\n\n¿Qué buscamos en vos?\nPoseer conocimiento sobre arquitectura de microservicios y contar con experiencia con sistemas distribuidos y procesamiento asíncrono (colas de mensajes, retry patterns, idempotency).\nPoseer conocimientos sólidos en Go y/o Python, y capacidad de aprender otros, así como tener conocimiento de diseño de APIs, RESTful y buenas prácticas de desarrollo (SOLID, clean architecture, design patterns, etc).\nPoseer conocimiento de bases de datos relacionales y no relacionales.\nEstar familiarizado con LLMs y sus APIs (GPT, Gemini, Claude u otros)\nEs un plus tener experiencia con frameworks de evaluación, LLM-as-judge u observabilidad de sistemas con LLMs, haber trabajado con plataformas de IA y/o desarrollo de agentes y tener conocimientos de análisis de datos y estadística aplicada (SQL, BigQuery, diseño de experimentos).\n¿Te animás a dejar huella en la tecnología de América Latina?\n¡Postulate y sumate a nuestro propósito!\nModalidad de trabajo híbrido. Site Buenos Aires, Argentina","description_format":"text","description_chars":2652,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[{"language":"Spanish","level":"Advanced (C1)","optional":false}]},"benefits":[],"hiring_locations":[{"name":"Argentina","iso":"AR","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Marketplaces","Payment Processing & Gateways","Digital Banks & Neobanks"],"lifecycle":[{"event":"open","at":"2026-10-26T17:13:04Z"}],"visa":[],"liveness":{"score":25,"band":"fade","label":"Fading","p_open":1,"p_active":0.685,"p_room":0.36,"age_days":69,"expected_fill_days":36,"reasons":["conf:1","velocity","win:tail","crowd:brand"],"computed_at":"2026-10-29T05:49:05Z"},"pay":null,"html_url":"https://alion.io/job/mercado-libre-senior-software-engineer-evals-for-agents","json_url":"https://alion.io/job/mercado-libre-senior-software-engineer-evals-for-agents.json","meta":{"generated_at":"2026-10-29T13:59:57Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":21072,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-30T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":237},"rest":"https://alion.io/mcp/rest/get_company?id=237"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fmercado-libre-senior-software-engineer-evals-for-agents"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fmercado-libre-senior-software-engineer-evals-for-agents"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fmercado-libre-senior-software-engineer-evals-for-agents"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/mercado-libre-senior-software-engineer-evals-for-agents\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fmercado-libre-senior-software-engineer-evals-for-agents"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3282561}},{"tool":"search_jobs","arguments":{"company":"mercadolibre.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3282561},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}