@tencentcloudadp/cli 0.1.5 → 0.1.6

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tencentcloudadp/cli",
-  "version": "0.1.5",
+  "version": "0.1.6",
   "description": "CLI for Tencent Cloud Agent Development Platform (ADP)",
   "type": "module",
   "bin": {
@@ -53,6 +53,7 @@
     "chalk": "^5.6.2",
     "chokidar": "^4.0.3",
     "jsonc-parser": "^3.3.1",
+    "qrcode": "^1.5.4",
     "tencentcloud-sdk-nodejs": "^4.1.324",
     "yargs": "^17.7.2",
     "zod": "^3.25.76"
@@ -60,6 +61,7 @@
   "devDependencies": {
     "@eslint/js": "^9.39.1",
     "@types/node": "^22.19.1",
+    "@types/qrcode": "^1.5.6",
     "@types/yargs": "^17.0.33",
     "eslint": "^9.39.1",
     "knip": "^5.73.4",

package/skills/adp/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: adp
-description: 使用 @tencentcloudadp/cli 在终端或 CI 管理腾讯云 ADP 的空间、应用、智能体、草稿、发布、对话、知识库、渠道和工作流。用户明确提到 ADP、adp 命令或腾讯云智能体开发平台时使用；不用于一般 Agent 开发、其他厂商平台或 CLI 源码开发。
+description: 使用 @tencentcloudadp/cli 在终端或 CI 管理腾讯云 ADP 的空间、应用、智能体、草稿、发布、对话、知识库、评测集与评测任务、渠道和工作流。用户明确提到 ADP、adp 命令或腾讯云智能体开发平台时使用；不用于一般 Agent 开发、其他厂商平台或 CLI 源码开发。
 metadata:
   requires:
     bins: ["adp"]
@@ -73,6 +73,7 @@ adp schema --command "channel" --compact --format json
 | 取 AppKey（对话/分享用的应用密钥）、启停和删除 key、分享链接、分享会话、企业共享应用、A2A 地址 | [references/app-keys.md](references/app-keys.md) |
 | 运营看板、用量和计费、审计日志、成员、工作台设置、模型卡片和映射、会话沙箱凭证 | [references/operations.md](references/operations.md) |
 | 调优准则、从评测生成准则、终端用户反馈、同义词、不满意回复 | [references/tuning.md](references/tuning.md) |
+| 撰写、上传、运行评测集，查看评测任务和报告 | [references/evaluation.md](references/evaluation.md) |
 | 写工作流画布、嵌套子工作流、各应用模式下挂载和调用工作流 | [references/workflows.md](references/workflows.md) |
 
 ## 写入边界

package/skills/adp/gotchas/SKILL.md

@@ -48,8 +48,9 @@ Full table: [../references/app-modes.md](../references/app-modes.md).
   - `adp plugin update --from-config` carries `ToolId` over by tool name; a tool without its id is recreated and silently unbinds agents.
 - `adp trigger create` needs a published app (`adp deploy`) plus `--cron` and `--prompt` for the scheduled+instruction path.
 - `adp trigger update` needs the app id (`--app-id` or `adp.jsonc`) and at least one field; only the fields you pass are changed.
-- `adp channel create` needs the credential block for its `--channel-type` in `--spec-json`, e.g. 10016 `{"Lark":{"AppId":"cli_xxx","AppSecret":"xxx"}}`, 10014 `{"WecomRobot":{"Websocket":{"BotId":"...","BotSecret":"..."}}}`. 10015 is C-end only (`--scene 1`) and needs both `--user-id` (any non-empty owner id) and `--agent-id` (the published Claw app's `agent_id` from `adp.jsonc`); pass `--spec-json '{"WechatClawBot":{}}'` and scan the returned `qrcodeUrl` within 120 s. C-end channels are not listed until the QR code is scanned, and `channel get` on an unscanned one returns `pendingAuthorization: true` (ADP answers `ChannelId 0` with only the QR session). ADP keeps returning that session even after `channel delete`, so `channel get` proves neither existence nor deletion; keep the create result and trust `channel list --scene 1`. ADP stores a per-type default name (10016 → 「飞书机器人」). The `--spec-json` help lists every type.
+- `adp channel create` needs the credential block for its `--channel-type` in `--spec-json`, e.g. 10016 `{"Lark":{"AppId":"cli_xxx","AppSecret":"xxx"}}`. ChannelType 10014 is the exception: omit BotId and BotSecret and the CLI opens the WeCom QR (`adp channel create --app-id <id> --channel-type 10014`). `--manual` still requires the spec block `{"WecomRobot":{"Websocket":{"BotId":"...","BotSecret":"..."}}}`. 10015 is C-end only (`--scene 1`) and needs both `--user-id` (any non-empty owner id) and `--agent-id` (the published Claw app's `agent_id` from `adp.jsonc`); pass `--spec-json '{"WechatClawBot":{}}'` and scan the returned `qrcodeUrl` within 120 s. C-end channels are not listed until the QR code is scanned, and `channel get` on an unscanned one returns `pendingAuthorization: true` (ADP answers `ChannelId 0` with only the QR session). ADP keeps returning that session even after `channel delete`, so `channel get` proves neither existence nor deletion; keep the create result and trust `channel list --scene 1`. ADP stores a per-type default name (10016 → 「飞书机器人」). The `--spec-json` help lists every type.
 - Only 10013 DingTalk, 10014 WeCom robot WebSocket and 10016 Lark accept credentials without a provider check. 10002/10009 need the callback URL verified in the WeCom admin console first (`4506004`); LINE, Telegram and WeChat customer service validate real credentials; a rejected credential comes back from ADP as `InternalError`, which the CLI reports as `ChannelCredentialRejected` (validation, not retryable). Do not create those with placeholder values or retry them. 10000 `Wechat` and 10015 `WechatClawBot` authorize by QR code (fields are filled in after the scan; the result carries `qrcodeUrl`): read `adp docs get 132545 --section WechatChannelConfig` before trying.
+- Publish to WeCom with ChannelType **10014** (long connection), not 10002/10009. `adp channel create --channel-type 10014` is the default: it opens the system browser to `https://work.weixin.qq.com/ai/qc/gen?source=tcadp`, waits for the scan, then CreateChannel sends `WecomRobot.Websocket` `{BotId, BotSecret, BindType:1}`. `--no-browser` still prints that page URL; `--output-qrcode <png>` writes the QR; `--manual` skips the scan and requires BotId and BotSecret (`BindType` 2 when omitted). The JSON result has `authorization: "qr"` and never the bot secret or the scan session id. Optional spec siblings the console also sends: `IsDebug`, `ShareGroupConversation`. CreateChannel's `QrcodeUrl` is only for 10000/10015. Do not follow the console 说明文档 (document 216860768951226368): that page is the old URL-callback flow and asks for a 备案域名, Nginx, and a second person in the WeCom admin console. `wecom-cli auth init` scans the same way but its source is `wecom_cli_external` and the bot secret stays encrypted in that CLI, so it cannot be copied into `adp channel create`.
 - `adp channel update` can only change the description and the WeCom robot ids; recreate the channel for anything else.
 - `adp trigger get` returns `cron`, `timezone`, `prompt`, `workflowId`; read it back after `trigger update` to confirm.
 - `adp trigger` is an app trigger (needs a published app); `adp timer` is a workbench timer task in a space (prompt + model, per user). Pick by what the user names.

package/skills/adp/references/evaluation.md

@@ -0,0 +1,103 @@
+# 评测集与评测任务（`adp eval`）
+
+评测不在公开文档的 ADP 接口里。不要用 `adp docs search` 判断有没有评测 API；命令以 `adp schema --command eval` 为准。所有写操作先 `--dry-run`。`--app-id` 不传时用 `adp.jsonc`。
+
+对比评测（`--compare model` 或 `prompt`）只支持标准模式应用。其他模式用 `--score none --compare none`，只记录回复、不打分。`--start` 和 `eval start` 会真实调用应用并消耗 token，必须加 `--yes`。
+
+## 撰写评测集
+
+先看列约定，需要空白模板时再下载：
+
+```bash
+adp eval sample template --format json
+adp eval sample template --output ./eval-set.xlsx
+```
+
+xlsx，最大 20MB。数据行至少 100 条，少了平台会拒绝导入（处理后不足 100 条）。
+
+- 第 1 行是给人看的说明，不是表头。
+- 第 2 行必须是：`session_id`、`request_id`、`query`、`custom_variables`、`reference_output`。
+- 数据从第 3 行开始。
+- `session_id` 必填。相同值是同一段对话，一段最多 100 轮。
+- `query` 必填，是发给应用的用户输入。
+- `request_id` 可选，纯数字。空着会按顺序编号。
+- `custom_variables` 可选，JSON 对象字符串，例如 `{"UserID":"0001"}`。
+- `reference_output` 可选，标准答案。只跑输出、不打分时可以空着。
+- `reference_output` 后面还可以加自定义列，连 `reference_output` 在内最多 10 列。列名以小写字母开头，只能有英文字母和下划线，不超过 40 个字符。
+
+最小的三行例子（第 1 行说明，第 2 行表头，第 3 行起数据）：
+
+| 说明 | 表头 session_id | request_id | query | custom_variables | reference_output |
+| --- | --- | --- | --- | --- | --- |
+| 第 3 行 | 1 | 1 | 今年年假有几天？ |  | 满 1 年不满 10 年是 5 天。 |
+
+## 上传
+
+```bash
+adp eval sample import --file ./eval-set.xlsx --dry-run --format json
+adp eval sample import --file ./eval-set.xlsx --format json
+adp eval sample list --format json
+```
+
+`statusLabel` 为 `ready` 才能建任务。`importing` 再列一次。`failed` 按上面的列约定改文件后重新导入。
+
+```bash
+adp eval sample delete --set-id <id> --yes --format json
+adp eval sample export --set-id <id> --format json
+```
+
+导出只返回 `noticeId`。xlsx 出现在控制台通知中心，不在这条命令的输出里。
+
+## 运行
+
+任务名最长 20 个字符，应用内不能重名。一个任务最多 5 个 `--set-id`。
+
+只记录回复：
+
+```bash
+adp eval create --name "年假评测" --set-id <id> --score none --dry-run --format json
+adp eval create --name "年假评测" --set-id <id> --score none --start --yes --format json
+```
+
+不带 `--start` 只创建、不跑。之后：
+
+```bash
+adp eval start --eval-id <id> --yes --format json
+```
+
+查看进度和结果：
+
+```bash
+adp eval list --format json
+adp eval get --eval-id <id> --format json
+adp eval report --eval-id <id> --format json
+adp eval record list --eval-id <id> --format json
+```
+
+状态：`pending` 待启动，`queued` 排队，`running` 评测中，`failed` 失败，`paused` 已暂停，`waiting_judge` 待标注，`judging` 标注中，`finished` 完成。跑完之前报告可能是空的，过几秒再读 `eval list`。
+
+```bash
+adp eval stop --eval-id <id> --format json
+adp eval delete --eval-id <id> --yes --format json
+adp eval export --eval-id <id> --format json
+```
+
+## 打分
+
+`--score judge` 要 `--judge-json`，`--score rule` 要 `--rule-json`，`--score code` 要 `--code`（明文，CLI 会做 base64）。裁判配置用接口原字段名：
+
+```bash
+adp eval example --set-id <id> --stage baseline --format json
+adp eval create --name "裁判" --set-id <id> --score judge \
+  --judge-json '{"Model":{"ModelName":"DeepSeek-V3"},"Prompt":"..."}' \
+  --dry-run --format json
+```
+
+`--compare model` 或 `--compare prompt` 要 `--compare-json` 数组，并且应用必须是标准模式。对比打分再用 `--compare-score judge` 和 `--compare-score-json`。
+
+人工标注一条记录：
+
+```bash
+adp eval record judge --eval-id <id> --record-id <id> \
+  --answers-json '[{"Sort":1,"Result":1,"Feedback":""}]' --format json
+```

package/skills/adp/references/pitfalls.md

@@ -74,6 +74,14 @@ platform side with `adp docs search "<action>" --format json` when in doubt.
 - **Fix**: use real credentials. For WeCom callback types, configure the receive-message
   URL, Token and EncodingAESKey in the WeCom admin console first. Read a block's fields
   with `adp docs get 132545 --section <Name>ChannelConfig --format json`.
+- Publish to WeCom with 10014 long connection. `adp channel create --channel-type 10014`
+  opens the WeCom QR by default (`--no-browser` prints the page URL, `--output-qrcode` writes a
+  PNG, `--manual` requires BotId and BotSecret in `--spec-json`). `BindType` 1 is the scan;
+  `BindType` 2 is the manual form. CreateChannel does not return that QR (`QrcodeUrl` is only
+  10000 and 10015). The console guide at document 216860768951226368 is the old 10009 callback
+  plus a filed domain and Nginx; use it only when the user asks for URL callback.
+  `wecom-cli auth init` scans the same way, but that bot secret stays inside wecom-cli and is
+  not an ADP channel credential.
 - A second channel of the same type is allowed: a second 10016 Lark channel on the same app
   is created normally. `4506003-企微应用已经被绑定` means that credential is already bound to a
   channel. ADP says 企微应用 for every type (a Lark `AppId` gets the same text); the CLI message