{"id":941825,"url":"https://alion.io/job/aircall-machine-learning-engineer-evals-and-voice-models","title":"Machine Learning Engineer (Evals and Voice Models)","company":{"id":52973,"name":"Aircall","domain":"aircall.io","url":"https://alion.io/company/aircall","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Greenhouse","truth_index":{"grade":"A","score":100,"open_postings":7,"ghost_share":0,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":46,"computed_at":"2026-11-08T05:47:12Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Francisco, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":181000,"max":250000,"currency":"USD","period":"year","gross":null,"usd_annual":250000},"salary_estimate":null,"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Fine-tuning","optional":false},{"name":"LLM","optional":false},{"name":"Machine Learning","optional":false},{"name":"Speech Recognition","optional":false},{"name":"Synthetic Data","optional":false},{"name":"Text-to-Speech","optional":false},{"name":"Voice Agents","optional":false},{"name":"WhatsApp","optional":false},{"name":"Model Distillation","optional":true},{"name":"Prompt Engineering","optional":true},{"name":"RAG","optional":true}],"status":"live","first_seen_at":"2026-09-15T16:19:49Z","employer_posted_date":"2026-09-15","last_verified_at":"2026-11-08T17:04:17Z","board_verified":true,"closed_at":null,"days_open":54,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":54},"description":"Aircall is a unicorn, AI-powered customer communications platform used by 22,000+ companies worldwide to drive revenue, resolve issues faster, and scale customer-facing teams. We’re redefining customer communications by bringing voice, SMS, WhatsApp, and AI together into one seamless workspace.\nOur momentum comes from a simple idea: help teams work smarter, not harder. Aircall’s AI Voice Agent automates routine calls, AI Assist streamlines post-call work, and AI Assist Pro delivers real-time guidance so people can do their best work. The result is higher revenue, faster resolutions, and teams that scale with confidence.\nAircall is headquartered in Paris, our European HQ, with a strong North American presence anchored in Seattle, our North American HQ, and teams across Madrid, London, Berlin, San Francisco, New York City, Sydney, and Mexico City. We’ve built a product customers love and a business that’s scaling quickly, backed by world-class investors and driven by rapid AI innovation across multiple product lines.\nAt Aircall, you’ll join a company in motion. We’re ambitious, product-driven, and execution-focused, with visible impact, fast decisions, and real growth.\nHow we work at Aircall: We’re customer-obsessed, data-driven, and focused on delivering meaningful outcomes. We value ownership, continuous learning, and thoughtful speed. If you thrive in a collaborative, fast-moving environment where trust and impact matter, you’ll feel at home here.\nAircall's AI suite includes an AI Voice Agent and AI Messaging Agent that autonomously handle calls, WhatsApp, and SMS, plus AI Assist, which delivers real-time coaching, call summaries, and CRM automation for sales and support teams. We are looking for someone that can build out the evaluation foundation across all of these products and other agentic products. You'll work on voice models, agent capability evals, benchmark design, LLM-as-judge systems, failure analysis, and the infrastructure that ties it together by establishing shared metrics, test sets, and tooling to measure accuracy, resolution quality, and safety consistently across products. You will set up repeatable pipelines for regression testing and benchmarking as models and features evolve so teams can ship confidently without re-inventing evaluation methodology for each product.\nKey Responsibilities\nDesign and document comprehensive evaluation frameworks for Aircall’s AI agents across voice, chat and messaging.\nTrain and fine-tune voice models (TTS, ASR, speech-to-speech) using production and synthetic data, iterating on architecture, data mix, and training strategy to improve accuracy, naturalness, and latency.\nAssess AI generated solutions across training pipelines, experimentation setups, debugging processes, and optimization strategies. \nAnalyze system design decisions and identify strengths, weaknesses, and potential failure points.\nDesign annotation guidelines and workflows for human-labeled evaluation data, and calibrate LLM-as-judge systems against human raters to ensure automated evals stay trustworthy over time.\nBuild and maintain live quality monitoring for deployed AI agents, tracking accuracy, resolution rate, and safety signals in production, and flagging model or data drift before it impacts customers.\nOwn the metric contract for every published AI metrics, including definition, population, grain, rollup, validity window.\nBuild release gates, the offline regression suite each AI surface must pass before a prompt, model, or config change ships, measuring reliability across repeated trials, not just average pass rates.\nBuild voice-specific evaluation: simulated callers across accents, languages, background noise, barge-in, DTMF, and tool failures, with latency and ASR accuracy as first-class quality metrics.\nMinimum Qualifications\nBS in Computer Science, Machine Learning, Statistics, or related field\n3+ years of experience in ML Engineering or Applied ML with 8+ years of overall experience\nStrong experience in evaluating supervised, unsupervised, LLMs and deep learning models.\nHands-on experience in failure analysis and evaluating LLMs\nExperience building automated evaluation systems\nStrong communication skills to articulate complex technical concepts across technical and non-technical audiences\nHands-on experience training or fine-tuning voice/speech models (TTS, ASR, or speech-to-speech), including data pipeline construction and experimentation.\nPreferred Qualifications\nMS / PhD in Computer Science, Machine Learning, Statistics, or related field\nExperience evaluating LLMs or agentic systems (e.g., LLM-as-a-judge, RAG evaluation)\nExperience with synthetic data generation and prompt engineering\nExperience training or fine-tuning voice models at scale, with familiarity in synthetic data generation, model distillation, or low-latency inference optimization for production voice agents.\nBase salary range:\n$181,000—$250,000 USD\nWhy join us?\nKey moment to join Aircall in terms of growth and opportunities\nOur people matter, work-life balance is important at Aircall\nFast-learning environment, entrepreneurial and strong team spirit\n45+ Nationalities: cosmopolite & multi-cultural mindset\nCompetitive salary package & benefits\nDE&I Statement: \nAt Aircall, we believe diversity, equity and inclusion - irrespective of origins, identity, background and orientations - are core to our journey. \nWe pride ourselves on promoting active inclusion within our business to foster a strong sense of belonging for all. We’re working to create a place filled with diverse people who can enrich and learn from one another. We’re committed to ensuring that everyone not only has a seat at the table but is valued and respected at it by providing equal opportunities to develop and thrive. \nWe will constantly challenge ourselves to make sure that we live up to our ambitions around diversity, equity and inclusion, and keep this conversation open. Above all else, we understand and acknowledge that we have work to do and much to learn.\nWant to know more about candidate privacy? Find our Candidate Privacy Notice here.\nWe may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.","description_format":"text","description_chars":6491,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":["Continuous learning","Equity"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Unified Communications","Contact Center Software"],"lifecycle":[{"event":"open","at":"2026-09-15T18:48:24Z"}],"visa":[{"country":"US","licensed_sponsor":true,"adverts_offering_sponsorship_24m":0,"sanctioned_now":false,"evidence":"H-1B filings in 12 months: 20","filings_12m":20,"filings_prev_12m":9,"green_card_filings_12m":0,"median_offered_wage_usd":222500,"route":null,"cap_exempt":false,"checked_at":"2026-11-08T10:45:15+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":8}],"liveness":{"score":52,"band":"ok","label":"Likely open","p_open":1,"p_active":0.952,"p_room":0.55,"age_days":53,"expected_fill_days":46,"reasons":["conf:1","urgency","velocity","win:tail"],"computed_at":"2026-11-08T05:47:12Z"},"pay":{"stated_usd_annual":250000,"is_top_pay":true},"html_url":"https://alion.io/job/aircall-machine-learning-engineer-evals-and-voice-models","json_url":"https://alion.io/job/aircall-machine-learning-engineer-evals-and-voice-models.json","meta":{"generated_at":"2026-11-08T19:45:19Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":41323,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-09T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":52973},"rest":"https://alion.io/mcp/rest/get_company?id=52973"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Faircall-machine-learning-engineer-evals-and-voice-models"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Faircall-machine-learning-engineer-evals-and-voice-models"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Faircall-machine-learning-engineer-evals-and-voice-models"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/aircall-machine-learning-engineer-evals-and-voice-models\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Faircall-machine-learning-engineer-evals-and-voice-models"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":941825}},{"tool":"search_jobs","arguments":{"company":"aircall.io","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":941825},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}