{"id":747054,"url":"https://alion.io/job/leadallies-research-scientist","title":"Research Scientist","company":{"id":678941,"name":"Lead Allies","domain":"leadallies.com","url":"https://alion.io/company/leadallies","size_band":null,"is_staffing_agency":true,"employer_type":"agency","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"B","score":75,"open_postings":7,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-11-04T06:34:25Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["San Francisco, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":168000,"max_usd":373000,"period":"year","method":"role_country_seniority_unknown","sample_n":4425},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Contrastive Learning","optional":false},{"name":"Diffusion Models","optional":false},{"name":"Discord","optional":false},{"name":"DPO","optional":false},{"name":"Fine-tuning","optional":false},{"name":"JAX","optional":false},{"name":"Machine Learning","optional":false},{"name":"NLP","optional":false},{"name":"OpenAI","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"Reinforcement Learning","optional":false},{"name":"Reward Modeling","optional":false},{"name":"RLHF","optional":false},{"name":"TensorFlow","optional":false},{"name":"Transformers","optional":false},{"name":"Vercel","optional":false}],"status":"live","first_seen_at":"2025-06-06T21:38:18Z","employer_posted_date":"2025-06-06","last_verified_at":"2026-11-05T04:26:50Z","board_verified":true,"closed_at":null,"days_open":516,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":516},"description":"Research Scientist / Machine Learning Scientist\nLocation: SF Bay Area/Hybrid / Remote\nType: Full-Time\nAbout the Role:\nThe Client is seeking a variety of Machine Learning Scientist to help advance how we evaluate and understand AI models. You’ll help design and analyze experiments that uncover what makes models useful, trustworthy and capable through human preference signals. Your work will contribute to the scientific foundations of understanding AI at scale.\nThis role is deeply interdisciplinary. You’ll work closely with engineers, product teams, marketing and the broader research community to develop new methods for comparing models, analyzing preference data, and disentangling performance factors like style, reasoning, and robustness. Your work will inform both the public leaderboard and the tools we provide to model developers.\nIf you’re excited by open-ended questions, rigorous evaluation, and research that’s grounded in real-world impact, you’ll find a meaningful home here. We’re looking for:\n\nHands-on experience training large-scale models, including reward models, preference models, and fine-tuning LLMs with methods like RLHF, DPO, and contrastive learning.\n\nStrong foundation in ML and statistics, with a track record of designing novel training objectives, evaluation schemes, or statistical frameworks to improve model reliability and alignment.\n\nFluent in the full experimental stack, from dataset design and large-batch training to rigorous evaluation and ablation, with an eye for what scales to production.\n\nDeeply collaborative mindset, working closely with engineers to productionize research insights and iterating with product teams to align modeling goals with user needs.\nResponsibilities:\n\nDesign and conduct experiments to evaluate AI model behavior across reasoning, style, robustness, and user preference dimensions\n\nDevelop new metrics, methodologies, and evaluation protocols that go beyond traditional benchmarks\n\nAnalyze large-scale human voting and interaction data to uncover insights into model performance and user preferences\n\nCollaborate with engineers to implement and scale research findings into production systems\n\nPrototype and test research ideas rapidly, balancing rigor with iteration speed\n\nAuthor internal reports and external publications that contribute to the broader ML research community\n\nPartner with model providers to shape evaluation questions and support responsible model testing\n\nContribute to the scientific integrity and transparency of the The Client leaderboard and tools\nWho is The Client?\nCreated by researchers fromUC Berkeley’s SkyLab, The Client is an open platform where everyone can easily access, explore and interact with the world’s leading AI models. By comparing them side by side and casting votes for the better response, the community helps shape a public leaderboard, making AI progress more transparent, and grounded in real-world usage.\nWhy Join Us?\nTrusted by organizations like Google, OpenAI, Meta, xAI, and more, The Client is rapidly becoming essential infrastructure for transparent, human-centered AI evaluation at scale. With over one million monthly users and growing developer adoption, our impact is helping guide the next generation of safe, aligned AI systems-grounded in open access and collective feedback.\nOur work is regularly referenced by industry leaders pushing the frontier of safe and reliable AI.Sundar Pichai,Jeff Dean,Elon Musk, andSam Altman.\n\nHigh Impact: Your work will be used daily by the world’s most advanced AI labs.\n\nGlobal Reach: Develop data infrastructure powering millions of real-world evaluations, influencing AI reliability across industries at the top-tier\n\nExceptional Team: We are a small team of top talent from Google, DeepMind, Discord, Vercel, UC Berkeley, and Stanford.\nRequirements:\n\nPhD or equivalent research experience in Machine Learning, Natural Language Processing, Statistics, or a related field\n\nStrong understanding of LLMs and modern deep learning architectures (e.g., Transformers, diffusion models, reinforcement learning with human feedback) Proficiency in Python and ML research libraries such as PyTorch, JAX, or TensorFlow\n\nDemonstrated ability to design and analyze experiments with statistical rigor\n\nExperience publishing research or working on open-source projects in ML, NLP, or AI evaluation\n\nComfortable working with real-world usage data and designing metrics beyond standard benchmarks\n\nAbility to translate research questions into practical systems and collaborate across engineering and product teams\n\nPassion for open science, reproducibility, and community-driven research\nWhat we offer:\n\nThe cash compensation for this position has not yet been finalized. Actual compensation will depend on job-related knowledge, skills, experience, and candidate location.\n\nCompetitive salary and meaningful equity\n\nComprehensive healthcare coverage (medical, dental, vision)\n\nThe opportunity to work on cutting-edge AI with a small, mission-driven team\n\nA culture that values transparency, trust, and community impact\nCome help build the space where anyone can explore and help shape the future of AI.","description_format":"text","description_chars":5165,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"phd","optional":false},"security_clearance":false,"languages":[]},"benefits":["Equity"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-09-11T14:29:20Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"Filed H-1B visas in the last two years","filings_12m":0,"filings_prev_12m":1,"green_card_filings_12m":0,"median_offered_wage_usd":null,"route":null,"cap_exempt":false,"checked_at":"2026-10-29T16:27:21+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":0}],"liveness":{"score":2,"band":"cold","label":"Long shot","p_open":1,"p_active":0.078,"p_room":0.28,"age_days":515,"expected_fill_days":25,"reasons":["conf:1","agency","win:tail","crowd:"],"computed_at":"2026-11-04T06:34:25Z"},"pay":null,"html_url":"https://alion.io/job/leadallies-research-scientist","json_url":"https://alion.io/job/leadallies-research-scientist.json","meta":{"generated_at":"2026-11-05T05:06:17Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":6171,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-06T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":678941},"rest":"https://alion.io/mcp/rest/get_company?id=678941"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fleadallies-research-scientist"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fleadallies-research-scientist"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fleadallies-research-scientist"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/leadallies-research-scientist\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fleadallies-research-scientist"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":747054}},{"tool":"search_jobs","arguments":{"company":"leadallies.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":747054},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}