{"id":3791605,"url":"https://alion.io/job/thundersoft-agentceshikaifafuzerena177779a","title":"Agent测试开发负责人(A177779A)","company":{"id":180498,"name":"ThunderSoft","domain":"thundersoft.com","url":"https://alion.io/company/thundersoft","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Feishu","truth_index":{"grade":"D","score":40,"open_postings":88,"ghost_share":1,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-11-09T06:03:00Z"}},"role":"QA","role_family":"QA","seniority":null,"employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Beijing, China"],"countries":["CN"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":35000,"max":45000,"currency":"CNY","period":"month","gross":null,"usd_annual":81084},"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"Appium","optional":false},{"name":"AutoGen","optional":false},{"name":"C++","optional":false},{"name":"CI/CD","optional":false},{"name":"CircleCI","optional":false},{"name":"Claude Code","optional":false},{"name":"Copilot","optional":false},{"name":"Cursor","optional":false},{"name":"DeepEval","optional":false},{"name":"Devin","optional":false},{"name":"Dify","optional":false},{"name":"GitHub Actions","optional":false},{"name":"Java","optional":false},{"name":"LangChain","optional":false},{"name":"LangGraph","optional":false},{"name":"LLM","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"OpenAI Codex","optional":false},{"name":"Playwright","optional":false},{"name":"Prompt Engineering","optional":false},{"name":"Pytest","optional":false},{"name":"Python","optional":false},{"name":"RAG","optional":false},{"name":"Selenium","optional":false},{"name":"TypeScript","optional":false}],"status":"live","first_seen_at":"2026-11-04T11:07:52Z","employer_posted_date":null,"last_verified_at":"2026-11-04T11:07:52Z","board_verified":false,"closed_at":null,"days_open":5,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":5},"description":"职位描述：\n\n一、为什么招这个岗位\n\n我们多条内部研发质量与效率工具(代码 Review、架构设计、Bug 修复、自动化测试等)的核心价值是 \"评测闭环\" + \"质量保障\"。当前产线在「0 → 1 试跑 → 里程碑收口」阶段已规模化运转,评测集已达千级 case 体量。但 谁来保证这些 case 永远对、Skill 升级时不回归、跨平台 smoke 通过率稳定在高位 — 是产线从\"能跑\"到\"能信\"的关键卡口。\n\n二、岗位职责\n\n1、AI Agent 评测体系与平台建设\n\n1）设计评测指标体系:准确率 / 召回率 / 鲁棒性 / 任务完成率 / 工具调用成功率 / 响应时延 / 幻觉率 / 上下文记忆能力 / 跨平台 smoke 通过率。\n\n2）搭建自动化评测平台,支持 Gold 标注 + 回归测试 + 模型版本对比 + A/B 评测。\n\n3）引入 LLM-as-a-Judge、A/B 实验、统计显著性判断等方法。\n\n4）建设 Benchmark 基准测试集，覆盖多轮交互、长短上下文、跨领域场景。\n\n2、内部研发质量线保障\n\n1）代码 Review 产物质量:千级规则 case + 数十流程 case 的 Gold 维护 + 回归测试。\n\n2）架构设计产物质量:百级用例 + WIP + 多平台(mcu / qnx / android)smoke。\n\n3）BugFix 产物质量:评测集 + 错误分类 + 故障归因。\n\n4）AutoTest 产物质量:测试用例 + 自动化覆盖 + 回归 + 跨平台。\n\n3、Skill 升级质量保障\n\n1）每次 Skill 升级前,跑全量回归 + 增量回归 + 灰度验证。\n\n2）维护 Skill 黄金标准集(Skill 评测)。\n\n3）跟踪 Skill MCP 接入的所有 Skill 之间的兼容性。\n\n4、安全与对抗性评测。\n\n1）红队用例:多租户权限隔离、数据沙箱、越权、提示注入\n\n2）实施 Prompt Injection / Jailbreak / 数据泄露测试\n\n3）输出安全审查 / 审计日志完整性测试\n\n5、AI 辅助测试 / 智能测试 Agent 探索\n\n1）探索 LLM 实现自动化测试 + 智能归因\n\n2）推动\"自动化测试\"向\"智能化测试\"演进\n\n职位要求：\n\n必须:\n\n1）本科及以上,计算机/软件工程/AI/数学相关专业优先。\n\n2）3-5 年以上测试开发 / 质量工程 或 AI 评测经验。\n\n3）熟练掌握 Python(必须);Java / Go / TypeScript / C++ 至少一门。\n\n4）熟悉 LLM 应用框架:LangChain / LangGraph / AutoGen / Coze / Dify / Claude Code / ReAct / Plan-and-Solve 至少一个。\n\n5）熟悉 Prompt Engineering / Harness Engineering / RAG / MCP / Skills。\n\n6）熟悉 CI/CD(GitHub Actions / CircleCI / 并行执行 / Sharding)。\n\n加分:\n\n1）搭建过 AI IDE / CLI 工具链 / Agent 平台的评测体系。\n\n2）参与过 SWE-bench / AgentBench / GAIA 类公开 Benchmark 建设或复现。\n\n3）熟悉 Playwright / Selenium / Appium / Pytest / Uiautomator2 / WDA / Midscene / CDP 自动化框架。\n\n4）熟练使用 AI 工具(Cursor / Claude Code / Devin / Codex / Copilot)人机协作开发。\n\n5）了解 RAGAS / DeepEval 等评测框架。\n\n四、加分项(画像)\n\n1）完整跑通过内部产线其中一条的里程碑收口。\n\n2）自己写过 Skill 评测集 + 跑过回归 + 写过 Gold 标注规范。\n\n3）推动\"测试 Agent\"在某个真实产线的落地。","description_format":"text","description_chars":1718,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Embedded Software & RTOS","Operating Systems"],"lifecycle":[{"event":"open","at":"2026-11-05T06:38:45Z"}],"visa":[],"liveness":{"score":63,"band":"ok","label":"Likely open","p_open":1,"p_active":0.632,"p_room":1,"age_days":4,"expected_fill_days":24,"reasons":["seen:4","stale_co","velocity","win:early","comp:brand"],"computed_at":"2026-11-09T06:03:00Z"},"pay":{"stated_usd_annual":81084,"is_top_pay":false},"html_url":"https://alion.io/job/thundersoft-agentceshikaifafuzerena177779a","json_url":"https://alion.io/job/thundersoft-agentceshikaifafuzerena177779a.json","meta":{"generated_at":"2026-11-10T01:32:31Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":3525,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-11T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":180498},"rest":"https://alion.io/mcp/rest/get_company?id=180498"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fthundersoft-agentceshikaifafuzerena177779a"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":2,"usd":0.2},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fthundersoft-agentceshikaifafuzerena177779a"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fthundersoft-agentceshikaifafuzerena177779a"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/thundersoft-agentceshikaifafuzerena177779a\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fthundersoft-agentceshikaifafuzerena177779a"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3791605}},{"tool":"search_jobs","arguments":{"company":"thundersoft.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3791605},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}