{"id":3364127,"url":"https://alion.io/job/lightly-ai-research-scientist-scientific-evaluation-mlai","title":"AI Research Scientist – Scientific Evaluation (ML/AI)","company":{"id":679232,"name":"Lightly","domain":"lightly.ai","url":"https://alion.io/company/lightly","size_band":"11-50","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Join","truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"contractor","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":[],"countries":[],"hiring_countries":["CH"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":92000,"max_usd":226000,"period":"year","method":"role_country_seniority_unknown","sample_n":21},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Computer Vision","optional":false},{"name":"Machine Learning","optional":false}],"status":"live","first_seen_at":"2026-10-27T13:36:14Z","employer_posted_date":"2026-10-27","last_verified_at":"2026-11-01T17:50:32Z","board_verified":true,"closed_at":null,"days_open":5,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":5},"description":"Lightly AG is a Zurich-based AI company and ETH/HSG spin-off, backed by Y Combinator and top-tier investors. Our machine learning and computer vision technology is trusted by global leaders in autonomous driving, medical imaging, and visual inspection.\nWe’re looking for experienced ML/AI researchers with a strong publication track record at top-tier conferences (NeurIPS, ICML, ICLR) to support an advanced AI research evaluation project. You'll critically assess scientific papers and research outputs generated by autonomous AI research agents, evaluating their technical correctness, scientific novelty, experimental rigor, and overall research quality against the standards of leading ML/AI conferences.\nThis is a remote, project-based contractor opportunity with flexible working hours.\nTasks\nWhat you'll be doing\nWhat you'll be doing\n\nCritically evaluate AI-generated ML/AI research papers, assessing their scientific contributions, technical correctness, methodology, experimental design, and conclusions\n\nAssess scientific novelty and significance, determining whether proposed approaches represent meaningful advancements over existing state-of-the-art research\n\nEvaluate experimental rigor and validity, including baseline comparisons, benchmark selection, evaluation metrics, statistical evidence, and ablation studies\n\nIdentify methodological flaws, unsupported claims, inconsistencies, or misleading results that undermine the scientific validity of proposed contributions\n\nAssess reproducibility and technical soundness, including whether reported findings are sufficiently supported by experimental evidence and, where available, accompanying code and implementation details\n\nReview and verify relevant academic literature, identifying overlooked prior work, inaccurate citations, and unsupported novelty claims\n\nEvaluate whether AI-generated research meets the scientific standards of top-tier ML/AI conferences, such as NeurIPS, ICML, and ICLR\n\nProvide structured, evidence-based assessments of research quality, including strengths, weaknesses, critical technical concerns, and overall scientific merit\n\nApply detailed evaluation rubrics consistently, providing clear, well-reasoned justifications for your assessments\n\nThe project focuses on evaluating advanced AI-generated scientific research, with an emphasis on technical validity, scientific novelty, experimental rigor, and overall research quality. Your role is to provide expert human judgment on whether the resulting research is scientifically valid, genuinely novel, and of a quality comparable to work accepted at leading ML/AI conferences.\nRequirements\nYou're a strong candidate if you:\nHave a PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical field, or equivalent demonstrated research expertise\n\nHave published at least 3 peer-reviewed research papers at NeurIPS, ICML, and/or ICLR (hard requirement), ideally including first-author or significant research contributions\n\nHave substantial hands-on ML/AI research experience, including developing novel methods, designing experiments, analyzing results, and publishing original scientific work\n\nHave a strong understanding of scientific methodology and experimental rigor, including appropriate baselines, statistical validity, benchmark design, ablation studies, and reproducibility\n\nCan independently assess the novelty, significance, and technical soundness of advanced ML/AI research, identifying both meaningful scientific contributions and fundamental methodological weaknesses\n\nHave experience reviewing research papers for top-tier ML/AI conferences or journals, ideally NeurIPS, ICML, or ICLR - strongly preferred\n\nAre highly familiar with state-of-the-art ML/AI research and can critically assess new methods against existing literature and established benchmarks\n\nAre comfortable verifying scientific claims, including reviewing technical arguments, experimental evidence, citations, and, where relevant, code implementations\n\nHave strong analytical and scientific writing skills and can articulate nuanced technical judgments through concise, evidence-based feedback\n\nCan consistently apply detailed evaluation guidelines and scoring rubrics while exercising independent expert judgment\n\nHave exceptional attention to detail, particularly when identifying subtle methodological flaws, unsupported conclusions, or discrepancies between claims and evidence\n\nBenefits\nFully remote and flexible - work from anywhere\n\nPart-time contractor role with flexible hours\n\nWork on the evaluation of cutting-edge autonomous AI systems capable of conducting end-to-end scientific research\n\nApply your research expertise to help assess and advance the scientific capabilities of frontier AI agents\n\nSend us your CV along with a brief note about your research background, areas of expertise, and publication record. Please include links to at least 3 peer-reviewed publications at NeurIPS, ICML, and/or ICLR, as well as any relevant academic peer-review experience.\nIf applicable, we'd also love to know which ML/AI research areas you're most familiar with and whether you've served as a reviewer, area chair, or in a similar capacity at major research conferences.\nWe look forward to hearing from you!","description_format":"text","description_chars":5293,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"phd","optional":false},"security_clearance":false,"languages":[]},"benefits":["Flexible schedule"],"hiring_locations":[{"name":"Switzerland","iso":"CH","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Computer Vision","AI Development Tools"],"lifecycle":[{"event":"open","at":"2026-10-27T22:21:09Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":4,"expected_fill_days":39,"reasons":["conf:0","win:early"],"computed_at":"2026-11-01T06:10:25Z"},"pay":null,"html_url":"https://alion.io/job/lightly-ai-research-scientist-scientific-evaluation-mlai","json_url":"https://alion.io/job/lightly-ai-research-scientist-scientific-evaluation-mlai.json","meta":{"generated_at":"2026-11-01T19:57:45Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":30737,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-02T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":679232},"rest":"https://alion.io/mcp/rest/get_company?id=679232"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Flightly-ai-research-scientist-scientific-evaluation-mlai"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Flightly-ai-research-scientist-scientific-evaluation-mlai"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Flightly-ai-research-scientist-scientific-evaluation-mlai"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/lightly-ai-research-scientist-scientific-evaluation-mlai\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Flightly-ai-research-scientist-scientific-evaluation-mlai"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3364127}},{"tool":"search_jobs","arguments":{"company":"lightly.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3364127},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}