{"id":2712428,"url":"https://alion.io/job/atsnap-gentceshikaifa","title":"agent测试开发","company":{"id":14664,"name":"Patsnap","domain":"patsnap.com","url":"https://alion.io/company/patsnap","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Lever","truth_index":null},"role":"QA","role_family":"QA","seniority":null,"employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Suzhou, China"],"countries":["CN"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":25000,"max":35000,"currency":"CNY","period":"month","gross":null,"usd_annual":62916},"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"CI/CD","optional":false},{"name":"DeepEval","optional":false},{"name":"Java","optional":false},{"name":"LangSmith","optional":false},{"name":"Linux","optional":false},{"name":"LLM","optional":false},{"name":"Promptfoo","optional":false},{"name":"Python","optional":false},{"name":"RAG","optional":false},{"name":"Ragas","optional":false}],"status":"live","first_seen_at":"2026-10-14T06:32:03Z","employer_posted_date":null,"last_verified_at":"2026-10-14T06:32:03Z","board_verified":false,"closed_at":null,"days_open":11,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":11},"description":"工作职责\n\n1.负责产品及项目交付质量保障参与需求与技术方案评审，理解客户业务流程和应用场景，制定测试策略、测试计划及验收标准；开展功能、接口、性能、稳定性等测试，识别交付风险，推动缺陷定位、修复和回归。\n\n2.建立大模型应用评测体系围绕问答、内容生成、检索增强生成（RAG）等场景，定义准确性、完整性、指令遵循、事实一致性和引用可靠性等评测维度，建立指标、评分标准与质量基线，支持模型选型、方案优化及版本效果对比。\n\n3.开展 Agent 专项评测面向多轮交互和多步骤任务，评估任务完成率、工具选择与参数正确性、上下文与记忆管理、异常恢复及终止条件等能力；结合执行轨迹和实际业务结果分析问题，覆盖工具失败、长链路执行、重复操作等复杂场景。\n\n4.建设和维护业务评测集从客户需求、真实业务场景和线上失败案例中提炼评测样本，覆盖典型、边界、长尾及对抗场景；制定标注规范、参考答案或验收条件，建立评测数据分层、版本管理与持续更新机制，兼顾通用能力评测和客户场景验收。\n\n5.设计并验证评测方法根据任务特点组合使用规则校验、可执行结果验证、人工评审、大模型辅助评分（LLM-as-a-Judge）及成对比较等方法；通过人工校准、重复运行和分场景分析，识别评分偏差与结果波动，提高评测结论的可信度。\n\n6.构建自动化测试与评测体系开发评测调度、环境隔离、工具模拟与回放、执行轨迹采集、自动评分、版本对比和报告能力；将评测接入 CI/CD，为模型、Prompt、知识库及 Agent 工作流变更建立自动回归与发布质量门槛，提升多项目交付中的测试复用效率。\n\n7.开展私有化部署环境验证结合客户本地模型、知识库、软硬件及网络条件开展效果、兼容性和集成测试；验证与客户内部系统的对接、权限控制、数据隔离及离线运行能力，支持客户环境问题排查、交付验收和版本升级回归。\n\n8.负责性能、可靠性与安全专项测试评估并发、响应延迟、长任务稳定性、资源占用及 Token 消耗，验证超时重试、限流和降级机制；覆盖提示词注入、越权工具调用、敏感信息泄漏等风险场景，推动受限算力环境下的质量与效率优化。\n\n9.推动质量持续改进联合客户、交付、产品、算法和研发团队，通过运行日志、用户反馈及失败案例分析，定位模型、检索、工具或流程问题；持续补充评测集、验证修复效果，并利用 AI 提升用例设计、数据构造和问题分析效率。\n\n任职资格\n\n1.本科及以上学历，计算机、软件工程等相关专业优先；具备 5 年及以上软件测试或测试开发经验，至少 2 年自动化测试或测试平台开发经验。\n\n2.具备实际的大模型应用或 Agent 评测项目经验，曾独立负责或主导评测方案设计、评测集建设、指标制定及自动化落地；能够结合具体项目说明评测方法、结果可信度及对业务质量的实际改善。\n\n3.具备系统的评测方法论，能够将业务目标转化为可验证的成功标准，合理设计组件评测与端到端评测、离线评测与线上验证，并根据业务风险制定验收阈值。\n\n4.熟悉大模型应用及 Agent 的工作机制，包括 Prompt、RAG、工具调用、多轮对话、上下文管理与工作流编排；能够基于日志和执行轨迹分析失败原因，提出可验证的改进建议。\n\n5.掌握评测集设计、样本抽样、标注一致性检查和评分标准设计；理解模型输出的随机性，能够运用重复试验、置信区间或统计检验等方法判断版本差异，具备识别 LLM-as-a-Judge 偏差并进行人工校准的能力。\n\n6.具备扎实的编码与工程能力，熟练掌握 Python；或精通 Java 等语言并能够使用 Python 开发评测工具。能够独立开发自动化框架、评测脚本或测试服务，熟悉至少一种主流测试框架。\n\n7.具备大模型应用或 Agent 自动化评测的工程实践，能够使用开源框架或自研工具，完成评测集管理、批量执行、自动评分、执行轨迹分析及版本回归，并根据业务需求开发自定义评测指标。熟悉接口自动化、性能与稳定性测试，以及 Linux、容器和 CI/CD 等相关技术。\n\n8.具备良好的业务理解、问题拆解和沟通协调能力，能够与客户、交付、产品及研发团队共同明确质量目标和验收口径，独立推动复杂问题定位与闭环。\n\n加分项\n\n1.有大模型应用或 Agent 私有化部署项目的测试、评测及交付验收经验者优先。能够基于客户真实业务构建评测集、制定验收指标，具备客户内网评测环境搭建、私有模型接入及评测数据本地化管理经验，能够在离线或受限网络、不同算力与模型配置下开展评测。\n\n2.有 Ragas、DeepEval、Promptfoo、LangSmith 等评测工具或平台的实际项目经验，能够结合业务需求完成工具选型、集成及评测能力扩展者优先。\n\n3.有客户现场问题排查、跨环境问题复现、国产软硬件适配或私有化项目版本升级回归经验。\n\n4.有从零搭建大模型或 Agent 评测平台的经验，能够沉淀跨项目复用的评测规范、工具和数据集，并取得可量化的质量或效率提升。\n\n5.有多 Agent 协作、浏览器/计算机操作 Agent、代码 Agent 或长流程业务 Agent 的评测经验。\n\n6.有 RAG 检索与生成分层评测、多模态评测、领域专家评审或 AI 安全对抗评测经验，或参与相关评测工具、数据集及开源项目建设。","description_format":"text","description_chars":2177,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Market Research","Financial Data & Market Intelligence"],"lifecycle":[{"event":"open","at":"2026-10-16T13:21:44Z"}],"visa":[{"country":"CA","licensed_sponsor":true,"evidence":"Positions approved through LMIA in 12 months: 1","filings_12m":1,"filings_prev_12m":0,"green_card_filings_12m":null,"median_offered_wage_usd":null,"route":"High Wage","cap_exempt":false,"checked_at":"2026-10-03T21:10:52+00:00","sources":["ESDC: employers issued a positive LMIA"],"filings_for_role_12m":0},{"country":"GB","licensed_sponsor":true,"evidence":"Licensed UK visa sponsor (Senior or Specialist Worker, Skilled Worker)","filings_12m":null,"filings_prev_12m":null,"green_card_filings_12m":null,"median_offered_wage_usd":null,"route":"Global Business Mobility: Senior or Specialist Worker; Skilled Worker","cap_exempt":false,"checked_at":"2026-10-24T06:24:00+00:00","sources":["UK Home Office: register of licensed sponsors (workers)"],"filings_for_role_12m":0}],"liveness":{"score":72,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.801,"p_room":0.9,"age_days":10,"expected_fill_days":23,"reasons":["seen:10","win:mid"],"computed_at":"2026-10-25T05:49:26Z"},"pay":{"stated_usd_annual":62916,"is_top_pay":false},"html_url":"https://alion.io/job/atsnap-gentceshikaifa","json_url":"https://alion.io/job/atsnap-gentceshikaifa.json","meta":{"generated_at":"2026-10-25T18:44:16Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":26527,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-26T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":14664},"rest":"https://alion.io/mcp/rest/get_company?id=14664"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fatsnap-gentceshikaifa"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fatsnap-gentceshikaifa"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fatsnap-gentceshikaifa"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/atsnap-gentceshikaifa\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fatsnap-gentceshikaifa"}]}