{"id":711684,"url":"https://alion.io/job/pathway-ai-benchmark-datasets-engineer-researcher","title":"AI Benchmark & Datasets Engineer / Researcher","company":{"id":37529,"name":"Pathway","domain":"pathway.com","url":"https://alion.io/company/pathway-com","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Workable","truth_index":{"grade":"B","score":75,"open_postings":4,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-11-02T06:13:15Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"remote","remote_scope":"stated_regions","remote_scope_basis":"posting_text","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":["Palo Alto, United States"],"countries":["US"],"hiring_countries":["US","FR","GB","CA","PL","DE","AT","BE","BG","HR","CY","CZ","DK","FI","HU","IE","IT","NL","RO","ES","SE","EE","GR","LV","LT","LU","MT","PT","SK","SI"],"hiring_countries_total":30,"salary":null,"salary_estimate":{"min_usd":134000,"max_usd":298000,"period":"year","method":"role_country_seniority_unknown","sample_n":4425},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Anthropic","optional":false},{"name":"ChatGPT","optional":false},{"name":"GitHub","optional":false},{"name":"Hugging Face","optional":false},{"name":"LLM","optional":false},{"name":"Machine Learning","optional":false},{"name":"OpenAI","optional":false},{"name":"Transformers","optional":false}],"status":"live","first_seen_at":"2026-03-19T00:00:00Z","employer_posted_date":"2026-03-19","last_verified_at":"2026-11-02T14:10:05Z","board_verified":true,"closed_at":null,"days_open":228,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":228},"description":"About Pathway\nPathway builds the first post-transformer frontier model that solves AI's fundamental memory problem. While transformers wake up in the same state every time-like Groundhog Day-our architecture enables true continuous learning, infinite context reasoning, and real-time adaptation. We're not optimizing yesterday's technology; we're building what comes after transformers.\nOur breakthrough architecture outperforms Transformer and provides the enterprise with full visibility into how the model works. Combining the foundational model with the fastest data processing engine on the market, Pathway enables enterprises to move beyond incremental optimization and toward truly contextualized, experience-driven intelligence. We are trusted by organizations such as NATO, La Poste, and Formula 1 racing teams.\nPathway is led by co-founder & CEO Zuzanna Stamirowska, a complexity scientist who created a team consisting of AI pioneers, including CTO Jan Chorowski who was the first person to apply Attention to speech and worked with Nobel laureate Geoff Hinton at Google Brain, as well as CSO Adrian Kosowski, a leading computer scientist and quantum physicist who obtained his PhD at the age of 20.\nThe company is backed by leading investors and advisors, including Lukasz Kaiser, co-author of the Transformer (“the T” in ChatGPT) and a key researcher behind OpenAI’s reasoning models. Pathway is headquartered in Palo Alto, California.\nThe Opportunity\nYou will design and execute rigorous benchmarks and define dataset standards. Collaborating closely with our R&D team, you will build the evaluation infrastructure that guides the evolution of Pathway’s post-transformer models.\nYou Will\nProactively identify, prioritize, and curate relevant public and client-driven benchmarks across our target use cases and markets.\nEvaluate candidate benchmarks for clarity, data quality, evaluation methodology, and fit with our model roadmap.\nRun benchmarks with baseline models to validate setup, uncover edge cases, and de-risk R&D runs.\nHand off “benchmark-ready” packages to R&D (specs, data, evaluation scripts, expected metrics, constraints) \nMaintain a shared vocabulary and documentation around benchmarks, datasets, and evaluation formats that GTM and R&D can both use.\nTrack and organize benchmark results, model leaderboards, and “what good looks like” for different customers and scenarios.\nContribute to demos and public-facing proof points based on benchmark outcomes.\nYou will play a key role in defining and driving the benchmarking process for AI model evaluation. Your work will directly influence what we build, how we talk about it, and how customers and the market experience BDH.\nRequirements\nCover letter\nIt's always a pleasure to say hi! If you could leave us 2-3 lines, we'd really appreciate that.\nYou are expected to meet at least one of the following criteria:\nYou have published at least one paper at NeurIPS, ICLR, or ICML - where you were the lead author or made significant conceptual & code contributions. \nYou have significantly contributed to an LLM training effort which became newsworthy (topped a Hugging Face benchmark, best in class model, etc.), preferably using multiple GPU's.\nYou have spent at least 6 months working in a leading Machine Learning research center (e.g. at: Google Brain / Deepmind, Apple, Meta, Anthropic, Nvidia, MILA).\nYou were an ICPC World Finalist, or an IOI, IMO, or IPhO medalist in High School.\nYou\nHave experience with ML/LLM evaluation, data science, or technical product roles, ideally around benchmarks or experimentation.\nAre comfortable reading papers, leaderboards, and Github repos, and turning them into clear, repeatable benchmark specs.\nCan talk comfortably with both engineers and customers, and translate between technical detail and business value.\nCare about high-quality data, reproducible experiments, and crisp documentation\nArerespectful of others\nArefluent in English\nBonus Points\nPublished or open-sourced work on LLM evaluation, benchmarking or data quality.\nExperience designing custom benchmarks or evaluation protocols for novel model capabilities.\nWhy You Should Apply\nJoin an intellectually stimulating work environment.\nBe a pioneer: you get to work with a new type of \"Live AI\" challenges around long sequences and changing data.\nBe part of one of an early-stage AI startup that believes in impactful research and foundational changes.\nBenefits\n Type of contract: Full-time, permanent\n Preferable joining date: Immediate. The positions are open until filled - please apply immediately.\n Compensation: based on profile and location.\n Location: Remote work. Possibility to work or meet with other team members in one of our offices: Palo Alto, CA; Paris, France or Wroclaw, Poland. Candidates based anywhere in the EU, UK, United States, and Canada will be considered.\nIf you meet our broad requirements but are missing some experience, don’t hesitate to reach out to us.","description_format":"text","description_chars":4969,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"phd","optional":false},"security_clearance":false,"languages":[{"language":"English","level":"Advanced (C1)","optional":false}]},"benefits":["Continuous learning"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"},{"name":"France","iso":"FR","kind":"country"},{"name":"United Kingdom","iso":"GB","kind":"country"},{"name":"Canada","iso":"CA","kind":"country"},{"name":"Poland","iso":"PL","kind":"country"},{"name":"Germany","iso":"DE","kind":"region"},{"name":"Austria","iso":"AT","kind":"region"},{"name":"Belgium","iso":"BE","kind":"region"},{"name":"Bulgaria","iso":"BG","kind":"region"},{"name":"Croatia","iso":"HR","kind":"region"},{"name":"Cyprus","iso":"CY","kind":"region"},{"name":"Czech Republic","iso":"CZ","kind":"region"},{"name":"Denmark","iso":"DK","kind":"region"},{"name":"Finland","iso":"FI","kind":"region"},{"name":"Hungary","iso":"HU","kind":"region"},{"name":"Ireland","iso":"IE","kind":"region"},{"name":"Italy","iso":"IT","kind":"region"},{"name":"Netherlands","iso":"NL","kind":"region"},{"name":"Romania","iso":"RO","kind":"region"},{"name":"Spain","iso":"ES","kind":"region"},{"name":"Sweden","iso":"SE","kind":"region"},{"name":"Estonia","iso":"EE","kind":"region"},{"name":"Greece","iso":"GR","kind":"region"},{"name":"Latvia","iso":"LV","kind":"region"},{"name":"Lithuania","iso":"LT","kind":"region"},{"name":"Luxembourg","iso":"LU","kind":"region"},{"name":"Malta","iso":"MT","kind":"region"},{"name":"Portugal","iso":"PT","kind":"region"},{"name":"Slovakia","iso":"SK","kind":"region"},{"name":"Slovenia","iso":"SI","kind":"region"}],"hiring_excludes":[],"relocation_offered":false,"industries":["LLM & Generative AI","Foundation Models","AI Research Labs"],"lifecycle":[{"event":"open","at":"2026-09-11T04:56:45Z"}],"visa":[],"liveness":{"score":5,"band":"cold","label":"Long shot","p_open":1,"p_active":0.165,"p_room":0.28,"age_days":228,"expected_fill_days":27,"reasons":["conf:4","win:tail","crowd:"],"computed_at":"2026-11-02T06:13:15Z"},"pay":null,"html_url":"https://alion.io/job/pathway-ai-benchmark-datasets-engineer-researcher","json_url":"https://alion.io/job/pathway-ai-benchmark-datasets-engineer-researcher.json","meta":{"generated_at":"2026-11-02T14:51:53Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":20545,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-03T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":37529},"rest":"https://alion.io/mcp/rest/get_company?id=37529"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fpathway-ai-benchmark-datasets-engineer-researcher"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fpathway-ai-benchmark-datasets-engineer-researcher"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fpathway-ai-benchmark-datasets-engineer-researcher"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/pathway-ai-benchmark-datasets-engineer-researcher\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fpathway-ai-benchmark-datasets-engineer-researcher"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":711684}},{"tool":"search_jobs","arguments":{"company":"pathway.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":711684},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}