modelfunction-cli 0.5.0 → 0.5.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -5,7 +5,7 @@
5
5
  ## 安装
6
6
 
7
7
  ```bash
8
- npx -y modelfunction-cli@0.4.4 --version
8
+ npx -y modelfunction-cli@0.5.1 --version
9
9
  # 或
10
10
  npm i -D modelfunction-cli
11
11
  npx mf --version
@@ -26,6 +26,7 @@ mf scan --root . --probe # 连 stdio MCP server 拉 tools/list
26
26
  mf coverage --suite mf.suite.json # 看每个工具还缺多少用例
27
27
  mf eval --suite mf.suite.json --models deepseek,qwen --show-cases
28
28
  mf compare before.json after.json # 跨版本比较(内容指纹判定可比性)
29
+ mf audit --suite mf.suite.json --out audit.md # 一次静态体检:lint + 边界 + 覆盖 + 挂载
29
30
  ```
30
31
 
31
32
  被测模型用环境变量提供 Key:
@@ -74,12 +75,13 @@ INTERNAL_LLM_KEY=... mf eval --suite mf.suite.json \
74
75
 
75
76
  | 命令 | 作用 |
76
77
  | --- | --- |
77
- | `mf scan` | 扫描工具定义(function.json / OpenAPI / MCP 配置 / Python `@tool` / JS·TS 声明与 TypeBox·zod schema);`--probe` 连 MCP server(运行时结果覆盖静态推断);`--from` 导入运行时工具数组;草稿按来源分组(`profiles`)并列出未覆盖的 skill |
78
- | `mf eval` | 跑评分卡;`--repeat` 多次取多数并报不稳定用例;`--fail-under` 阈值;`--baseline` + `--fail-on-regression` 回归门禁;`--profile` 按挂载组分别评分 |
78
+ | `mf scan` | 扫描工具定义(function.json / OpenAPI / MCP 配置 / Python `@tool` / JS·TS 声明与 TypeBox·zod schema);`--probe` 连 stdio 或 streamable-HTTP MCP server(运行时结果覆盖静态推断;headers/query 不写入报告);`--from` 导入运行时工具数组;草稿按来源分组(`profiles`)并列出未覆盖的 skill |
79
+ | `mf eval` | 跑评分卡;`--repeat` 多次取多数并报不稳定用例;`--fail-under` 阈值;`--baseline` + `--fail-on-regression` 回归门禁;`--profile` 按挂载组分别评分;`--max-cost` 按累计成本停机(退出 2);`--progress` 逐用例进度(stderr) |
79
80
  | `mf case add` | 写入用例(默认 `source=agent`、`reviewed=false`) |
80
81
  | `mf coverage` | 每工具用例数、参数覆盖、负例比例、疑似混淆对;有 `profiles` 时按组统计 |
82
+ | `mf audit` | 一次静态审计:lint + 混淆族边界 + 用例覆盖 + 挂载检查,输出可转发的 Markdown/JSON;不调用模型 |
81
83
  | `mf lint` | 静态检查工具描述与参数 schema,以及 skills 的 name/description(不调用模型);重复规则自动合并成一条;有 `profiles` 时给出「这些组不要合并挂载」 |
82
- | `mf check` | 安装门禁:一个新能力 × 已装能力(`--manifest` + `--installed`),纯静态、返回 pass/warn、不阻断;也可 `import { check } from "modelfunction-cli"` |
84
+ | `mf check` | 安装门禁:一个或一包新能力 × 已装能力(`--manifest` / `--manifests` + `--installed`),纯静态、返回 pass/warn、不阻断;门禁语境只对混淆对/必填来源/结构性错误报 warn,其余降 info;也可 `import { check, checkMany } from "modelfunction-cli"` |
83
85
  | `mf compare` | 跨版本比较、干扰归因、混淆矩阵;两份报告都有分组时按组对比 |
84
86
  | `mf ablate` | 留一法:去掉某个工具再跑一遍,量化它是否干扰其他工具;`--profile` 限定在某个挂载组内 |
85
87
  | `mf cluster` | 聚类工具,建议挂载分组;参数形状拆散的同一族(如 `sessions_*`)列为「同族候选」;并输出「族内边界清单」(每族会互抢的对 + 每个工具可用的区分词);`--out` 把分组写成套件的 `profiles` |
@@ -87,7 +89,7 @@ INTERNAL_LLM_KEY=... mf eval --suite mf.suite.json \
87
89
  | `mf eval --execute` | 一次跑完选择 + 参数 + 运行断言(`toolInput` / `expect`) |
88
90
  | `mf sandbox check` | 探测本机可用隔离引擎与降级建议 |
89
91
  | `mf config set-key` | 把模型 Key 写入 `~/.modelfunction/credentials.json`(隐藏输入);`remove-key` 删除;`list` 只显示掩码,`path` 打印路径 |
90
- | `mf stats` | 本地使用摘要(命令次数、模型、通过率区间、lint 规则命中);不联网 |
92
+ | `mf stats` | 本地使用摘要(命令次数、本地/CI 区分、模型、通过率区间、lint 规则命中);不联网 |
91
93
  | `mf feedback` | 导出脱敏反馈包(仅计数与比率),发不发由你决定 |
92
94
  | `mf mcp` | stdio MCP server,供 Claude Code / Cursor / Kimi Code 调用 |
93
95
 
@@ -124,19 +126,19 @@ mf eval --suite mf.suite.json --models internal-ds \
124
126
  "mcpServers": {
125
127
  "modelfunction": {
126
128
  "command": "npx",
127
- "args": ["-y", "modelfunction-cli@0.4.4", "mcp"]
129
+ "args": ["-y", "modelfunction-cli@0.5.1", "mcp"]
128
130
  }
129
131
  }
130
132
  }
131
133
  ```
132
134
 
133
- 暴露:`scan_tools`、`list_suite`、`coverage`、`add_cases`、`run_eval`(`run_eval` 支持 `profile` 与 `repeat` 参数:传 `all` 或组名按挂载组评分,`repeat` 取多数结果)。
135
+ 暴露:`scan_tools`、`list_suite`、`coverage`、`audit`、`lint`、`check`、`cluster`、`compare`、`add_cases`、`run_eval`(`run_eval` 支持 `profile`、`repeat`、`maxCost`、`progress`)。`lint`/`check`/`cluster`/`compare` 都是静态操作,不调用模型;`check` 就是安装门禁,Agent 可直接传入 manifest 与 installed。
134
136
 
135
137
  ## 契约与版本
136
138
 
137
139
  对外 JSON 字段、`check()` 返回结构、MCP 参数与退出码见 [SCHEMA.md](SCHEMA.md),自 0.4.3 起冻结为 v1:v1 内只做加法,重命名或删除字段需 major 并提前两个 minor 或 90 天弃用。`check()` 的类型声明随包发布。
138
140
 
139
- 本仓库当前版本 `0.5.0`(凭证与本地反馈渠道,待发布);上文的 `@0.4.4` 是 npm 上已发布的版本,保证示例可复制。发布后把示例里的版本号统一升级即可。
141
+ `0.5.1` 已发布到 npm:含凭证与本地反馈、usage `ci` 标记、eval `--progress`/`--max-cost`、`mf audit`、`scan --probe` streamable HTTP。本仓库 `0.5.2` 待发布:MCP 补齐 `lint`/`check`/`cluster`/`compare`,安装门禁支持 `--manifests` 一包批量检查,门禁 warn 只保留混淆对/必填来源/结构性错误(描述类降 info)。示例统一使用 `@0.5.1`。
140
142
 
141
143
  ## 数据边界
142
144
 
@@ -146,7 +148,7 @@ mf eval --suite mf.suite.json --models internal-ds \
146
148
 
147
149
  ## 本地使用记录与反馈(默认不上传)
148
150
 
149
- 每条命令会在 `~/.modelfunction/usage/YYYY-MM.jsonl` 追加一行**计数记录**:命令名、退出码、耗时、mf/Node/OS 版本、模型 id、用例/工具数量、通过率区间、lint 规则命中数。**不记录**用例原文、工具描述、源码、文件路径或密钥。
151
+ 每条命令会在 `~/.modelfunction/usage/YYYY-MM.jsonl` 追加一行**计数记录**:命令名、退出码、耗时、mf/Node/OS 版本、模型 id、用例/工具数量、通过率区间、lint 规则命中数、是否 CI。**不记录**用例原文、工具描述、源码、文件路径或密钥。`mf stats` 会把本地运行与 CI 运行分开统计。
150
152
 
151
153
  ```bash
152
154
  mf stats # 看本机记录;--json 给脚本
@@ -165,7 +167,7 @@ mf feedback --out out.json # 导出脱敏反馈包(聚合值,不含原始
165
167
 
166
168
  ```bash
167
169
  # 1. 把 mf 挂成 MCP server
168
- claude mcp add modelfunction -- npx -y modelfunction-cli@0.4.4 mcp
170
+ claude mcp add modelfunction -- npx -y modelfunction-cli@0.5.1 mcp
169
171
  ```
170
172
 
171
173
  ```text
package/SCHEMA.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # Schemas v1
2
2
 
3
- 自 `modelfunction-cli@0.4.3` 起冻结为 v1;`0.4.4` 是同一 v1 的补丁;`0.5.0` 增加 §9 本地文件,仍属 v1。这里描述 CLI 与库对外的稳定契约。人类可读文本(表格、提示语)不属于契约;**JSON 字段**属于。
3
+ 自 `modelfunction-cli@0.4.3` 起冻结为 v1;`0.4.4` 是同一 v1 的补丁;`0.5.0` 增加 §9 本地文件、`0.5.1` 增加 §10 审计 JSON,仍属 v1。这里描述 CLI 与库对外的稳定契约。人类可读文本(表格、提示语)不属于契约;**JSON 字段**属于。
4
4
 
5
5
  ## 兼容政策
6
6
 
@@ -110,7 +110,7 @@ skills 只做静态检查,不参与模型评测。
110
110
 
111
111
  `--json` 输出 `{ "report": {...}, "draft": {...} }`;默认写 `mf.suite.json`(draft)与 `mf.scan.json`(report)。
112
112
 
113
- report:`root`、`tools[]`(`name` / `description` / `parameters` / `origin` / `confidence` / `source` / `note?` / `descriptionSource?`)、`skills[]`、`mcpServers[]`(env 只保留 `envKeys`)、`probes[]`、`sources[]`、`warnings[]`。
113
+ report:`root`、`tools[]`(`name` / `description` / `parameters` / `origin` / `confidence` / `source` / `note?` / `descriptionSource?`)、`skills[]`、`mcpServers[]`(env 只保留 `envKeys`,headers 只保留 `headerKeys`;URL 去掉 userinfo / query / hash)、`probes[]`(`server` / `ok` / `toolCount?` / `tools?` / `error?`)、`sources[]`、`warnings[]`。`--probe` 支持 stdio 与 streamable HTTP 两种传输。
114
114
 
115
115
  `scan --json` 在未找到任何工具时退出码为 3。
116
116
 
@@ -174,9 +174,37 @@ import { check } from "modelfunction-cli";
174
174
  ```
175
175
 
176
176
  - `verdict` 只有 `pass` / `warn`,**没有 fail**;质量判定不阻断安装。
177
+ - **严重度策略(门禁语境)**:只有会改变模型行为的规则保持原等级——`desc-missing`、`param-type-missing`(error),`required-unsourced`、`pair-no-boundary`、`skill-pair-no-boundary`(warn);描述、命名、必填数量等质量规则降为 `info`,避免成熟工具集每次安装都告警。`mf lint` 不受影响,仍输出原始严重度。
177
178
  - 唯一抛错场景:`manifest.name` 缺失或为空。
178
179
  - 同步、无网络、无文件系统、无状态。类型声明见 `lib/check.d.ts`。
179
180
 
181
+ 批量(一次安装一包):
182
+
183
+ ```js
184
+ import { checkMany } from "modelfunction-cli";
185
+ const result = checkMany({ manifests: [/* ... */], installed: [/* ... */] });
186
+ ```
187
+
188
+ 返回 `{ verdict, summary, items, results[] }`:
189
+
190
+ ```json
191
+ {
192
+ "verdict": "warn",
193
+ "summary": { "error": 0, "warn": 1, "info": 0 },
194
+ "items": 2,
195
+ "results": [
196
+ { "name": "gitcrawl", "verdict": "warn", "summary": { "error": 0, "warn": 1, "info": 0 },
197
+ "findings": [ { "severity": "warn", "code": "skill-pair-no-boundary", "subject": "gitcrawl",
198
+ "with": "tag-duplicate-prs-issues", "message": "...", "suggestion": "..." } ],
199
+ "family": ["gitcrawl", "tag-duplicate-prs-issues"] },
200
+ { "name": "tag-duplicate-prs-issues", "verdict": "warn", "summary": { "error": 0, "warn": 1, "info": 0 },
201
+ "findings": [ /* 同一对,subject 为自身 */ ], "family": ["gitcrawl", "tag-duplicate-prs-issues"] }
202
+ ]
203
+ }
204
+ ```
205
+
206
+ `summary` 按 subject/with 去重(同一对只计一次);`results[]` 保留与该能力相关的全部发现(pair 会出现在双方)。`manifests` 为空、某项缺 name、名称重复时抛错。CLI:`mf check --manifests pack.json [--installed installed.json] [--json]`。
207
+
180
208
  ## 7. MCP 工具
181
209
 
182
210
  | 工具 | 参数 |
@@ -184,8 +212,13 @@ import { check } from "modelfunction-cli";
184
212
  | `scan_tools` | `root?`、`probe?`、`timeoutMs?`、`writeSuite?` |
185
213
  | `list_suite` | `suite?` |
186
214
  | `coverage` | `suite?` |
215
+ | `audit` | `suite?` |
216
+ | `lint` | `suite?` |
217
+ | `check` | `manifest`(对象)、`installed?`(数组或 `{tools,skills}`) |
218
+ | `cluster` | `suite?`、`threshold?`、`maxSize?` |
219
+ | `compare` | `before`、`after`(评分卡 JSON 路径) |
187
220
  | `add_cases` | `suite?`、`cases[]`、`source?`、`reviewed?`、`dryRun?` |
188
- | `run_eval` | `suite?`、`models?`、`cases?`、`profile?`、`repeat?`、`failUnder?` |
221
+ | `run_eval` | `suite?`、`models?`、`cases?`、`profile?`、`repeat?`、`failUnder?`、`maxCost?`、`progress?` |
189
222
 
190
223
  返回值均为 JSON 文本。
191
224
 
@@ -195,7 +228,7 @@ import { check } from "modelfunction-cli";
195
228
  | --- | --- |
196
229
  | 0 | 成功(`mf check` 的 warn 也是 0) |
197
230
  | 1 | 输入/用法错误;`mf compare` 判定不可直接比较 |
198
- | 2 | `mf eval --fail-under` 未达标,或 `--fail-on-regression` 发现回退 |
231
+ | 2 | `mf eval --fail-under` 未达标、`--fail-on-regression` 发现回退,或 `--max-cost` 超限 |
199
232
  | 3 | `mf exec` 被权限策略拒绝;`mf scan --json` 未找到工具 |
200
233
  | 4 | `mf exec` 超时 |
201
234
 
@@ -226,6 +259,7 @@ import { check } from "modelfunction-cli";
226
259
  "platform": "win32",
227
260
  "arch": "x64",
228
261
  "node": "24.6.0",
262
+ "ci": false,
229
263
  "models": ["mock"],
230
264
  "caseCount": 13,
231
265
  "toolCount": 4,
@@ -241,7 +275,7 @@ import { check } from "modelfunction-cli";
241
275
  }
242
276
  ```
243
277
 
244
- 字段为白名单,只含计数、比率与 id;**不含**用例原文、工具描述、源码、文件路径或密钥。新增字段属于 v1 加法,消费方可忽略未知字段。`MF_NO_USAGE=1` 关闭写入;删除 `usage/` 即清空。
278
+ 字段为白名单,只含计数、比率与 id;**不含**用例原文、工具描述、源码、文件路径或密钥。`ci` 表示检测到 CI 环境(`CI=true/1` 或 `GITHUB_ACTIONS` / `GITLAB_CI` / `CIRCLECI` / `TRAVIS` / `BUILDKITE` / `TF_BUILD` / `JENKINS_URL` / `CNB_BUILD_ID`),用于把机器运行与本地运行分开;`mf stats` 分别统计。新增字段属于 v1 加法,消费方可忽略未知字段。`MF_NO_USAGE=1` 关闭写入;删除 `usage/` 即清空。
245
279
 
246
280
  ### `mf feedback` 包
247
281
 
@@ -260,3 +294,47 @@ import { check } from "modelfunction-cli";
260
294
  ```
261
295
 
262
296
  `usage` 是 `mf stats` 的聚合结果,不含原始事件、路径或密钥。是否发送由用户决定。
297
+
298
+ ## 10. 工具集审计(`mf audit --json`,0.5.1 起)
299
+
300
+ ```json
301
+ {
302
+ "suite": { "name": "…", "path": "…", "tools": 4, "skills": 0, "cases": 13, "reviewed": 13 },
303
+ "lint": {
304
+ "tools": 4,
305
+ "skills": 0,
306
+ "counts": { "error": 0, "warn": 6, "info": 0 },
307
+ "findings": [],
308
+ "profilePairs": []
309
+ },
310
+ "clusters": {
311
+ "threshold": 0.5,
312
+ "total": 4,
313
+ "clusters": [],
314
+ "riskyPairs": [],
315
+ "familyHints": [],
316
+ "boundaries": []
317
+ },
318
+ "coverage": {
319
+ "total": 13,
320
+ "negative": 1,
321
+ "negativeRatio": 0.08,
322
+ "unreviewed": 0,
323
+ "tools": [],
324
+ "pairs": [],
325
+ "profiles": [],
326
+ "warnings": []
327
+ },
328
+ "profiles": { "groups": [], "warnings": [], "unmounted": [] },
329
+ "actions": [
330
+ {
331
+ "kind": "boundary|param-source|cases|args|review|mount",
332
+ "subject": "…",
333
+ "message": "…",
334
+ "detail": "…"
335
+ }
336
+ ]
337
+ }
338
+ ```
339
+
340
+ `profiles` 为 `null` 表示套件未定义分组。`clusters`/`coverage` 的明细与 §5/§4 同源;`actions` 是给人看的优先级列表。`mf audit` 恒退出 0(advisory),不阻断流程。
package/lib/audit.js ADDED
@@ -0,0 +1,149 @@
1
+ // One static pass over a whole tool set: lint + cluster boundaries + case
2
+ // coverage + mount checks. No model calls, no execution, no network.
3
+ // The markdown output is meant to be forwarded as a trial deliverable.
4
+ import { lintSuite, formatLint } from "./lint.js";
5
+ import { clusterTools, formatClusters } from "./cluster.js";
6
+ import { computeCoverage, formatCoverage } from "./cases.js";
7
+ import { normalizeProfiles, profileWarnings, profileCaseCoverage } from "./profile.js";
8
+
9
+ export function auditSuite(suite) {
10
+ const lint = lintSuite(suite);
11
+ const clusters = clusterTools(suite.tools);
12
+ const coverage = computeCoverage(suite);
13
+ const profiles = normalizeProfiles(suite.profiles, suite.tools.map((tool) => tool.name));
14
+ const mounted = profiles ? new Set(profiles.flatMap((profile) => profile.tools)) : null;
15
+ const profile = profiles
16
+ ? {
17
+ groups: profileCaseCoverage(profiles, suite.cases),
18
+ warnings: profileWarnings(profiles, suite),
19
+ unmounted: suite.tools.map((tool) => tool.name).filter((name) => !mounted.has(name))
20
+ }
21
+ : null;
22
+
23
+ return {
24
+ suite: {
25
+ name: suite.name,
26
+ path: suite.path ?? null,
27
+ tools: suite.tools.length,
28
+ skills: suite.skills?.length ?? 0,
29
+ cases: suite.cases.length,
30
+ reviewed: suite.cases.filter((testCase) => testCase.reviewed !== false).length
31
+ },
32
+ lint: {
33
+ tools: lint.tools,
34
+ skills: lint.skills,
35
+ counts: lint.counts,
36
+ findings: lint.findings,
37
+ profilePairs: lint.profilePairs ?? []
38
+ },
39
+ clusters,
40
+ coverage,
41
+ profiles: profile,
42
+ actions: buildActions({ lint, clusters, coverage, profile })
43
+ };
44
+ }
45
+
46
+ function buildActions({ lint, clusters, coverage, profile }) {
47
+ const actions = [];
48
+
49
+ const families = (clusters.boundaries ?? [])
50
+ .filter((family) => family.size > 1)
51
+ .sort((a, b) => b.pairs.length - a.pairs.length)
52
+ .slice(0, 3);
53
+ for (const family of families) {
54
+ actions.push({
55
+ kind: "boundary",
56
+ subject: family.label,
57
+ message: `为「${family.label}」补选择边界(${family.pairs.length} 对互抢)`,
58
+ detail: family.suggestion
59
+ });
60
+ }
61
+
62
+ const unsourced = lint.findings.filter((finding) => finding.code === "required-unsourced");
63
+ for (const finding of unsourced.slice(0, 5)) {
64
+ actions.push({
65
+ kind: "param-source",
66
+ subject: `${finding.tool}.${finding.param}`,
67
+ message: `补参数来源:${finding.tool}.${finding.param}`,
68
+ detail: finding.message
69
+ });
70
+ }
71
+
72
+ const empty = coverage.tools.filter((tool) => tool.total === 0);
73
+ if (empty.length > 0) {
74
+ actions.push({
75
+ kind: "cases",
76
+ subject: `${empty.length} tools`,
77
+ message: `为 ${empty.length} 个工具补用例`,
78
+ detail: empty.map((tool) => tool.name).join(", ")
79
+ });
80
+ }
81
+ const noArgs = coverage.tools.filter((tool) => tool.total > 0 && tool.withArgs === 0);
82
+ if (noArgs.length > 0) {
83
+ actions.push({
84
+ kind: "args",
85
+ subject: `${noArgs.length} tools`,
86
+ message: `为 ${noArgs.length} 个工具补带参数的用例`,
87
+ detail: noArgs.map((tool) => tool.name).join(", ")
88
+ });
89
+ }
90
+ if (coverage.unreviewed > 0) {
91
+ actions.push({
92
+ kind: "review",
93
+ subject: `${coverage.unreviewed} cases`,
94
+ message: `复核 ${coverage.unreviewed} 条未复核用例(复核后才计入正式分数)`,
95
+ detail: "把认可的用例 reviewed 改为 true"
96
+ });
97
+ }
98
+ if (profile && profile.unmounted.length > 0) {
99
+ actions.push({
100
+ kind: "mount",
101
+ subject: `${profile.unmounted.length} tools`,
102
+ message: `${profile.unmounted.length} 个工具不在任何挂载组`,
103
+ detail: profile.unmounted.join(", ")
104
+ });
105
+ }
106
+
107
+ return actions;
108
+ }
109
+
110
+ export function formatAudit(report) {
111
+ const { suite, lint, clusters, coverage, profiles, actions } = report;
112
+ const lines = [
113
+ `# ${suite.name} — 工具集审计`,
114
+ "",
115
+ `- 工具 ${suite.tools} · skills ${suite.skills} · 用例 ${suite.cases}(已复核 ${suite.reviewed})`,
116
+ `- 文件:${suite.path ?? "(内存)"}`,
117
+ `- 纯静态:不调用模型、不执行工具、无网络`,
118
+ "",
119
+ "## 建议动作"
120
+ ];
121
+ if (actions.length === 0) {
122
+ lines.push("暂无必须处理项。");
123
+ } else {
124
+ actions.forEach((action, index) => {
125
+ lines.push(`${index + 1}. ${action.message}`);
126
+ if (action.detail) lines.push(` ${action.detail}`);
127
+ });
128
+ }
129
+
130
+ lines.push("", "## 描述与参数(lint)", "", formatLint(lint));
131
+ lines.push("", "## 混淆与分组(cluster)", "", formatClusters(clusters));
132
+ lines.push("", "## 用例覆盖(coverage)", "", formatCoverage(coverage));
133
+ lines.push("", "## 挂载分组(profiles)", "");
134
+ if (!profiles) {
135
+ lines.push("未定义 profiles(平台可先用 `mf cluster --out <suite>` 生成候选分组)。");
136
+ } else {
137
+ lines.push("group tools cases negatives tools-without-cases");
138
+ lines.push("---------------------- ----- ----- --------- -------------------");
139
+ for (const row of profiles.groups) {
140
+ lines.push(
141
+ `${row.name.padEnd(22)} ${String(row.tools).padStart(5)} ${String(row.cases).padStart(5)} ${String(row.negatives).padStart(9)} ${row.missing.length ? row.missing.join(", ") : "-"}`
142
+ );
143
+ }
144
+ if (profiles.unmounted.length > 0) lines.push("", `未挂载工具:${profiles.unmounted.join(", ")}`);
145
+ for (const warning of profiles.warnings) lines.push(`- ${warning}`);
146
+ }
147
+
148
+ return lines.join("\n");
149
+ }
package/lib/budget.js ADDED
@@ -0,0 +1,30 @@
1
+ // Running cost guard for evaluations. Pricing is known per model, but token
2
+ // usage is only known after a call returns, so the limit is enforced after
3
+ // each call: the run stops as soon as the budget is crossed.
4
+ export class CostLimitError extends Error {
5
+ constructor({ limit, spent, calls }) {
6
+ super(`cost limit reached: spent ¥${spent.toFixed(4)} of ¥${limit} after ${calls} call(s)`);
7
+ this.name = "CostLimitError";
8
+ this.limit = limit;
9
+ this.spent = spent;
10
+ this.calls = calls;
11
+ }
12
+ }
13
+
14
+ export function createBudget(limit = null) {
15
+ let spent = 0;
16
+ let calls = 0;
17
+ return {
18
+ get spent() {
19
+ return spent;
20
+ },
21
+ get calls() {
22
+ return calls;
23
+ },
24
+ spend(cost) {
25
+ calls += 1;
26
+ if (Number.isFinite(cost) && cost > 0) spent += cost;
27
+ if (limit !== null && spent > limit) throw new CostLimitError({ limit, spent, calls });
28
+ }
29
+ };
30
+ }
package/lib/check.d.ts CHANGED
@@ -23,6 +23,11 @@ export interface CheckInput {
23
23
  }
24
24
 
25
25
  export interface CheckFinding {
26
+ /**
27
+ * Install-gate view: only behavior-changing findings keep warn/error
28
+ * (selection conflicts, unsourced required params, schema errors).
29
+ * Description and naming rules are reported as info so mature tools pass.
30
+ */
26
31
  severity: Severity;
27
32
  code: string;
28
33
  subject: string;
@@ -41,6 +46,23 @@ export interface CheckResult {
41
46
  family?: string[];
42
47
  }
43
48
 
49
+ export interface CheckManyInput {
50
+ manifests: Capability[];
51
+ installed?: Capability[] | InstalledCapabilities;
52
+ }
53
+
54
+ export interface CheckItemResult extends CheckResult {
55
+ name: string;
56
+ }
57
+
58
+ export interface CheckManyResult {
59
+ verdict: Verdict;
60
+ /** Deduplicated by subject/with; per-item summaries are in results[]. */
61
+ summary: { error: number; warn: number; info: number };
62
+ items: number;
63
+ results: CheckItemResult[];
64
+ }
65
+
44
66
  /**
45
67
  * Static install gate: judge one new capability against installed ones.
46
68
  * No model call, no network, no file system, no state.
@@ -48,4 +70,13 @@ export interface CheckResult {
48
70
  */
49
71
  export declare function check(input: CheckInput): CheckResult;
50
72
 
73
+ /**
74
+ * Batch install gate: judge a pack of new capabilities against installed ones
75
+ * and against each other.
76
+ * @throws when manifests is empty, an item lacks a name, or names are duplicated.
77
+ */
78
+ export declare function checkMany(input: CheckManyInput): CheckManyResult;
79
+
51
80
  export declare function formatCheck(result: CheckResult): string;
81
+
82
+ export declare function formatCheckMany(result: CheckManyResult): string;
package/lib/check.js CHANGED
@@ -1,4 +1,4 @@
1
- // Install-gate entry point: judge one new capability against what is already
1
+ // Install-gate entry point: judge new capabilities against what is already
2
2
  // installed. Static only — no model call, no network, no file system, no state.
3
3
  // Quality findings are advisory: the verdict is pass/warn, never "block".
4
4
  import { lintSuite } from "./lint.js";
@@ -6,6 +6,41 @@ import { lintSuite } from "./lint.js";
6
6
  export function check({ manifest, installed = [] } = {}) {
7
7
  const target = normalizeEntry(manifest);
8
8
  if (!target) throw new Error('check: manifest 需要非空 "name"');
9
+ const { results } = evaluateTargets([target], installed);
10
+ const [result] = results;
11
+ return {
12
+ verdict: result.verdict,
13
+ summary: result.summary,
14
+ findings: result.findings,
15
+ ...(result.family ? { family: result.family } : {})
16
+ };
17
+ }
18
+
19
+ // Installers usually apply a pack at once: new skills must be checked against
20
+ // installed ones AND against each other.
21
+ export function checkMany({ manifests, installed = [] } = {}) {
22
+ if (!Array.isArray(manifests) || manifests.length === 0) {
23
+ throw new Error("checkMany: manifests 需要非空数组");
24
+ }
25
+ const targets = manifests.map((entry) => normalizeEntry(entry));
26
+ if (targets.some((target) => target === null)) {
27
+ throw new Error('checkMany: 每个 manifest 都需要非空 "name"');
28
+ }
29
+ const names = new Set();
30
+ for (const target of targets) {
31
+ if (names.has(target.name)) throw new Error(`checkMany: duplicate name "${target.name}"`);
32
+ names.add(target.name);
33
+ }
34
+ const { results } = evaluateTargets(targets, installed);
35
+ return {
36
+ verdict: results.some((item) => item.verdict === "warn") ? "warn" : "pass",
37
+ summary: uniqueSummary(results),
38
+ items: results.length,
39
+ results
40
+ };
41
+ }
42
+
43
+ function evaluateTargets(targets, installed) {
9
44
  const pool = [];
10
45
  if (Array.isArray(installed)) pool.push(...installed);
11
46
  else if (installed && typeof installed === "object") {
@@ -16,22 +51,47 @@ export function check({ manifest, installed = [] } = {}) {
16
51
  const tools = [];
17
52
  const skills = [];
18
53
  for (const entry of entries) (isToolLike(entry) ? tools : skills).push(entry);
19
- (isToolLike(target) ? tools : skills).push(target);
54
+ for (const target of targets) (isToolLike(target) ? tools : skills).push(target);
20
55
 
21
56
  const report = lintSuite({ tools, skills });
22
- const findings = report.findings
23
- .filter((finding) => touches(finding, target.name))
24
- .map((finding) => shape(finding, target.name));
25
- const summary = { error: 0, warn: 0, info: 0 };
26
- for (const finding of findings) summary[finding.severity] += 1;
27
- const family = familyOf(report.findings.filter((finding) => finding.pair), target.name);
57
+ const pairFindings = report.findings.filter((finding) => finding.pair);
58
+ const results = targets.map((target) => {
59
+ const findings = report.findings
60
+ .filter((finding) => touches(finding, target.name))
61
+ .map((finding) => shape(finding, target.name));
62
+ const summary = { error: 0, warn: 0, info: 0 };
63
+ for (const finding of findings) summary[finding.severity] += 1;
64
+ const family = familyOf(pairFindings, target.name);
65
+ return {
66
+ name: target.name,
67
+ verdict: summary.error + summary.warn > 0 ? "warn" : "pass",
68
+ summary,
69
+ findings,
70
+ ...(family.length > 1 ? { family } : {})
71
+ };
72
+ });
73
+ return { results };
74
+ }
28
75
 
29
- return {
30
- verdict: summary.error + summary.warn > 0 ? "warn" : "pass",
31
- summary,
32
- findings,
33
- ...(family.length > 1 ? { family } : {})
34
- };
76
+ // A pair finding touches both items, so the batch summary is deduped by
77
+ // subject/with; per-item summaries keep the same finding under each item.
78
+ function uniqueSummary(results) {
79
+ const seen = new Map();
80
+ for (const result of results) {
81
+ for (const finding of result.findings) {
82
+ const key = [
83
+ finding.code,
84
+ finding.severity,
85
+ finding.subject,
86
+ finding.with ?? "",
87
+ finding.param ?? ""
88
+ ].join("|");
89
+ if (!seen.has(key)) seen.set(key, finding);
90
+ }
91
+ }
92
+ const summary = { error: 0, warn: 0, info: 0 };
93
+ for (const finding of seen.values()) summary[finding.severity] += 1;
94
+ return summary;
35
95
  }
36
96
 
37
97
  function normalizeEntry(entry) {
@@ -61,10 +121,22 @@ function touches(finding, name) {
61
121
  return Array.isArray(finding.pair) && finding.pair.includes(name);
62
122
  }
63
123
 
124
+ // An install gate should only warn about things that change model behavior:
125
+ // selection conflicts, unsourceable required params, and hard schema errors.
126
+ // Description and naming rules stay informative — otherwise every mature
127
+ // gateway tool warns on install and the signal becomes noise.
128
+ const GATE_CRITICAL_CODES = new Set([
129
+ "desc-missing",
130
+ "param-type-missing",
131
+ "required-unsourced",
132
+ "pair-no-boundary",
133
+ "skill-pair-no-boundary"
134
+ ]);
135
+
64
136
  function shape(finding, name) {
65
137
  const other = (finding.pair ?? []).find((item) => item !== name);
66
138
  return {
67
- severity: finding.severity,
139
+ severity: GATE_CRITICAL_CODES.has(finding.code) ? finding.severity : "info",
68
140
  code: finding.code,
69
141
  subject: name,
70
142
  ...(finding.param ? { param: finding.param } : {}),
@@ -101,18 +173,41 @@ function familyOf(pairs, name) {
101
173
  return [...seen].sort();
102
174
  }
103
175
 
176
+ function findingLines(finding) {
177
+ const param = finding.param ? `.${finding.param}` : "";
178
+ const withWhat = finding.with ? ` × ${finding.with}` : "";
179
+ const lines = [
180
+ `[${finding.severity}] ${finding.code} ${finding.subject}${param}${withWhat}`,
181
+ ` ${finding.message}`
182
+ ];
183
+ if (finding.suggestion) lines.push(` ${finding.suggestion}`);
184
+ return lines;
185
+ }
186
+
104
187
  export function formatCheck(result) {
105
188
  const lines = [
106
189
  `verdict: ${result.verdict}(error ${result.summary.error} / warn ${result.summary.warn} / info ${result.summary.info})`
107
190
  ];
108
191
  if (result.findings.length === 0) lines.push("没有发现问题");
109
- for (const finding of result.findings) {
110
- const param = finding.param ? `.${finding.param}` : "";
111
- const withWhat = finding.with ? ` × ${finding.with}` : "";
112
- lines.push(`[${finding.severity}] ${finding.code} ${finding.subject}${param}${withWhat}`);
113
- lines.push(` ${finding.message}`);
114
- if (finding.suggestion) lines.push(` ${finding.suggestion}`);
115
- }
192
+ for (const finding of result.findings) lines.push(...findingLines(finding));
116
193
  if (result.family) lines.push("", `族:${result.family.join(", ")}`);
117
194
  return lines.join("\n");
118
195
  }
196
+
197
+ export function formatCheckMany(result) {
198
+ const lines = [
199
+ `verdict: ${result.verdict}(${result.items} 个新能力;去重后 error ${result.summary.error} / warn ${result.summary.warn} / info ${result.summary.info})`
200
+ ];
201
+ result.results.forEach((item, index) => {
202
+ lines.push(
203
+ "",
204
+ `[${index + 1}] ${item.name} ${item.verdict}(error ${item.summary.error} / warn ${item.summary.warn} / info ${item.summary.info})`
205
+ );
206
+ if (item.findings.length === 0) lines.push(" 没有发现问题");
207
+ for (const finding of item.findings) {
208
+ for (const line of findingLines(finding)) lines.push(` ${line}`);
209
+ }
210
+ if (item.family) lines.push(` 族:${item.family.join(", ")}`);
211
+ });
212
+ return lines.join("\n");
213
+ }