modelfunction-cli 0.6.0 → 0.6.2

package/README.md

@@ -5,7 +5,7 @@
 ## 安装
 
 ```bash
-npx -y modelfunction-cli@0.6.0 --version
+npx -y modelfunction-cli@0.6.2 --version
 # 或
 npm i -D modelfunction-cli
 npx mf --version
@@ -80,9 +80,9 @@ INTERNAL_LLM_KEY=... mf eval --suite mf.suite.json \
 | `mf case add` | 写入用例（默认 `source=agent`、`reviewed=false`） |
 | `mf coverage` | 每工具用例数、参数覆盖、负例比例、疑似混淆对；有 `profiles` 时按组统计 |
 | `mf audit` | 一次静态审计：lint + 混淆族边界 + 用例覆盖 + 挂载检查，输出可转发的 Markdown/JSON；不调用模型 |
-| `mf lint` | 静态检查工具描述与参数 schema，以及 skills 的 name/description（不调用模型）；重复规则自动合并成一条；有 `profiles` 时给出「这些组不要合并挂载」 |
+| `mf lint` | 静态检查工具描述与参数 schema，以及 skills 的 name/description（不调用模型）；0.6.1 起含「可选参数祈使句」与「悬空工具名引用」两条 info 规则；重复规则自动合并成一条；有 `profiles` 时给出「这些组不要合并挂载」 |
 | `mf check` | 安装门禁：一个或一包新能力 × 已装能力（`--manifest` / `--manifests` + `--installed`），纯静态、返回 pass/warn、不阻断；门禁语境只对混淆对/必填来源/结构性错误报 warn，其余降 info；也可 `import { check, checkMany } from "modelfunction-cli"` |
-| `mf compare` | 跨版本比较、干扰归因、混淆矩阵；两份报告都有分组时按组对比 |
+| `mf compare` | 跨版本比较、干扰归因、混淆矩阵；两份报告都有分组时按组对比；`--fail-on-regression` 发现回归退出 2；`excludeFromBase` 行不参与回归判定 |
 | `mf ablate` | 留一法：去掉某个工具再跑一遍，量化它是否干扰其他工具；`--profile` 限定在某个挂载组内 |
 | `mf cluster` | 聚类工具，建议挂载分组；参数形状拆散的同一族（如 `sessions_*`）列为「同族候选」；并输出「族内边界清单」（每族会互抢的对 + 每个工具可用的区分词）；`--out` 把分组写成套件的 `profiles` |
 | `mf exec` | 单工具执行 + 指标（Node / Python），支持 `--allow-read/write/net`、超时、sideEffects 准入 |
@@ -126,7 +126,7 @@ mf eval --suite mf.suite.json --models internal-ds \
   "mcpServers": {
     "modelfunction": {
       "command": "npx",
-      "args": ["-y", "modelfunction-cli@0.6.0", "mcp"]
+      "args": ["-y", "modelfunction-cli@0.6.2", "mcp"]
     }
   }
 }
@@ -138,7 +138,7 @@ mf eval --suite mf.suite.json --models internal-ds \
 
 对外 JSON 字段、`check()` 返回结构、MCP 参数与退出码见 [SCHEMA.md](SCHEMA.md)，自 0.4.3 起冻结为 v1：v1 内只做加法，重命名或删除字段需 major 并提前两个 minor 或 90 天弃用。`check()` 的类型声明随包发布。
 
-`0.6.0` 待 npm 发布：token 成为主口径（评分卡新增 `totalTokens`），模型注册表支持 `currency`（CNY/USD，默认 CNY），成本字段改为 `totalCost` / `avgCost` + `costCurrency`（`totalCostCny` / `avgCostCny` 仅在 CNY 时保留），新增 `mf eval --max-tokens <n>` 与 MCP `maxTokens`；`--fail-on-regression` 在基线模型缺失或本轮全部调用失败时改为 exit 2（无法判定 ≠ 无回归），自动选模型时在 stderr 打 note；本地门禁模板改用退出码判定、只在成功 eval 后更新 state，`.cmd` 固定 CRLF。`0.5.5` 含 eval `--retry <n>` / `--min-interval <ms>`、评分卡 `retries`、CI 模板 static 层 `cluster --json`、`templates/local/` 与 `docs/local-gate.md`（内网/非 GitHub 环境的 pre-push 静态门禁、变更门、定时回归）。`0.5.4` 含用例级 `excludeFromBase` / `excludeFromProfiles`、评分卡 `errorKind` / `errorsByKind` / `scoredTotal` / `selectionAccuracyExcludingErrors`、挂载对比错误不计 jammed、audit 分组 `neg%` 与分组 token 用量聚合。`0.5.3` 及更早的能力见下：凭证与本地反馈、usage `ci` 标记、eval `--progress`/`--max-cost`、`mf audit`、`scan --probe` streamable HTTP、MCP 静态工具 `lint`/`check`/`cluster`/`compare`、`checkMany` 一包批量检查、门禁严重度收敛、静态规则性能（真实 100 能力清单 `checkMany` 471ms→40ms）、`installed` 形状校验与 `profiles` 平台形态兼容。示例统一使用 `@0.6.0`。
+`0.6.2` 待 npm 发布：修复 `param-optional-imperative` 的名词短语/工具动作误报与多出现漏报；`mf compare` 遵守 `excludeFromBase`（对比结果带 `excluded`），新增 `mf compare --fail-on-regression`。`0.6.1` 已发布到 npm（latest）：新增 `param-optional-imperative` 与 `desc-dangling-tool` 两条 info lint，并把「具名否定 / 泛化否定 / 可选参数祈使句 / 改名引用 / 话术范围」写进 `skills/mf-cases/SKILL.md`。`0.6.0` 已发布到 npm（latest）：token 成为主口径（评分卡新增 `totalTokens`），模型注册表支持 `currency`（CNY/USD，默认 CNY），成本字段改为 `totalCost` / `avgCost` + `costCurrency`（`totalCostCny` / `avgCostCny` 仅在 CNY 时保留），新增 `mf eval --max-tokens <n>` 与 MCP `maxTokens`；`--fail-on-regression` 在基线模型缺失或本轮全部调用失败时改为 exit 2（无法判定 ≠ 无回归），自动选模型时在 stderr 打 note；本地门禁模板改用退出码判定、只在成功 eval 后更新 state，`.cmd` 固定 CRLF。`0.5.5` 含 eval `--retry <n>` / `--min-interval <ms>`、评分卡 `retries`、CI 模板 static 层 `cluster --json`、`templates/local/` 与 `docs/local-gate.md`（内网/非 GitHub 环境的 pre-push 静态门禁、变更门、定时回归）。`0.5.4` 含用例级 `excludeFromBase` / `excludeFromProfiles`、评分卡 `errorKind` / `errorsByKind` / `scoredTotal` / `selectionAccuracyExcludingErrors`、挂载对比错误不计 jammed、audit 分组 `neg%` 与分组 token 用量聚合。`0.5.3` 及更早的能力见下：凭证与本地反馈、usage `ci` 标记、eval `--progress`/`--max-cost`、`mf audit`、`scan --probe` streamable HTTP、MCP 静态工具 `lint`/`check`/`cluster`/`compare`、`checkMany` 一包批量检查、门禁严重度收敛、静态规则性能（真实 100 能力清单 `checkMany` 471ms→40ms）、`installed` 形状校验与 `profiles` 平台形态兼容。示例统一使用 `@0.6.2`。
 
 ## 数据边界
 
@@ -167,7 +167,7 @@ mf feedback --out out.json  # 导出脱敏反馈包（聚合值，不含原始
 
 ```bash
 # 1. 把 mf 挂成 MCP server
-claude mcp add modelfunction -- npx -y modelfunction-cli@0.6.0 mcp
+claude mcp add modelfunction -- npx -y modelfunction-cli@0.6.2 mcp
 ```
 
 ```text

package/SCHEMA.md

@@ -116,6 +116,7 @@ skills 只做静态检查，不参与模型评测。
 - `--repeat` 时模型对象带 `stability`（翻转用例与多次结果分布）。
 - `--profile` 时每个组含 `profile`、`mounted`、`allInOne`、`delta`、`lifted`、`jammed`、`errors`，以及 `usage`（`inputTokens` / `cachedTokens` / `outputTokens` / `totalTokens` / `cost` / `costCurrency`；CNY 时另带 `costCny`）；错误行不参与挂载对比（不会被记成 jammed），每个组的 `reviewed` 是可评分行数。
 - `fingerprint` 由工具与用例内容计算；`mf compare` 用它判断两份报告是否可比。
+- `mf compare` 与 `--fail-on-regression` 会跳过任一报告中标了 `excludeFromBase` 的行；对比结果里的 `excluded` 给出跳过的行数。这些行既不参与回归判定，也不进入对比指标。
 
 ## 3. 扫描输出（`mf scan`）
 
@@ -141,8 +142,11 @@ report：`root`、`tools[]`（`name` / `description` / `parameters` / `origin` /
 ```
 
 - `kind: "skill"` 时用 `skill` 字段代替 `tool`。
-- `param` 出现在参数级发现上；`pair` 出现在工具对/技能对发现上。
+- `param` 出现在参数级发现上；`pair` 出现在工具对/技能对发现上；`ref` / `similarTo` 出现在悬空工具名引用上。
 - `profilePairs` 只在套件含 `profiles` 时出现。
+- 0.6.1 新增两条 advisory（均为 `info`，不影响 `check()` 的 pass/warn）：
+  - `param-optional-imperative`：可选参数的描述用了祈使句；用户输入没提供该参数时可能抑制调用。0.6.2 起扫描参数名的所有出现位置，英文只认显式“参数/parameter”搭配，名词短语（`Include tool messages`）与工具自身动作（`Send message ... by sessionKey`）不再误报。
+  - `desc-dangling-tool`：描述引用了套件中不存在的工具名；`similarTo` 给出共享领域词的现有工具，通常来自改名后遗留的引用。
 
 ## 5. cluster JSON（`mf cluster --json`）
 
@@ -240,7 +244,7 @@ const result = checkMany({ manifests: [/* ... */], installed: [/* ... */] });
 | --- | --- |
 | 0 | 成功（`mf check` 的 warn 也是 0） |
 | 1 | 输入/用法错误；`mf compare` 判定不可直接比较 |
-| 2 | `mf eval --fail-under` 未达标、`--fail-on-regression` 发现回退或无法判定（基线模型缺失、本轮全部调用失败）、`--max-cost` / `--max-tokens` 超限 |
+| 2 | `mf eval --fail-under` 未达标、`--fail-on-regression` 发现回退或无法判定（基线模型缺失、本轮全部调用失败）、`--max-cost` / `--max-tokens` 超限；`mf compare --fail-on-regression` 发现回归 |
 | 3 | `mf exec` 被权限策略拒绝；`mf scan --json` 未找到工具或 skill |
 | 4 | `mf exec` 超时 |
 

package/lib/compare.js

@@ -33,6 +33,7 @@ export function compareScorecards(before, after) {
     const confusionCounts = new Map();
     const interferenceMap = new Map();
     let reviewed = 0;
+    let excluded = 0;
     let selectionBefore = 0;
     let selectionAfter = 0;
     let passBefore = 0;
@@ -42,6 +43,10 @@ export function compareScorecards(before, after) {
       const rowBefore = rowsBefore.get(id);
       const rowAfter = rowsAfter.get(id);
       if (rowBefore.reviewed === false || rowAfter.reviewed === false) continue;
+      if (rowBefore.excludeFromBase || rowAfter.excludeFromBase) {
+        excluded += 1;
+        continue;
+      }
       reviewed += 1;
       if (rowBefore.selectionCorrect) selectionBefore += 1;
       if (rowAfter.selectionCorrect) selectionAfter += 1;
@@ -109,6 +114,7 @@ export function compareScorecards(before, after) {
       comparable: toolChanges.length === 0 && caseChanges.length === 0 && !executionChanged,
       commonCases: comparableIds.length,
       reviewed,
+      excluded,
       metrics: {
         selection: { before: ratio(selectionBefore, reviewed), after: ratio(selectionAfter, reviewed) },
         pass: { before: ratio(passBefore, reviewed), after: ratio(passAfter, reviewed) }
@@ -246,6 +252,9 @@ export function formatCompare(report) {
     }
     lines.push(`[${model.model}] ${model.comparable ? "可比较" : "不可直接比较（工具定义或用例已变更）"}`);
     lines.push(`  共同且未变更用例: ${model.commonCases}（已复核 ${model.reviewed}）`);
+    if (model.excluded > 0) {
+      lines.push(`  排除出回归判定: ${model.excluded} 条（excludeFromBase）`);
+    }
     lines.push(`  选择准确率: ${percent(model.metrics.selection.before)} → ${percent(model.metrics.selection.after)}`);
     lines.push(`  通过率:     ${percent(model.metrics.pass.before)} → ${percent(model.metrics.pass.after)}`);
     if (model.interference.length > 0) {

package/lib/js-tools.js

@@ -36,6 +36,23 @@ export function extractJsTools(src, { externals, onUnresolvedName } = {}) {
     const parsed = parseArgs(src, match.index + match[0].length - 1);
     if (parsed) push(toolFromArgs(parsed.items, ctx));
   }
+  // Plugin registries (e.g. anything-llm) call `aibitat.function({ name,
+  // description, parameters })` inside a setup hook instead of a builder.
+  for (const match of src.matchAll(/\.function\s*\(/g)) {
+    const parsed = parseArgs(src, match.index + match[0].length - 1);
+    if (!parsed || !parsed.items[0] || parsed.items[0].t !== "obj") continue;
+    const node = parsed.items[0];
+    const preset = {};
+    const nameNode = node.props.get("name");
+    if (!nameNode || nameNode.t !== "str") {
+      // Plugin registries often write `name: this.name` inside the setup hook;
+      // the literal lives on the enclosing plugin object just above it.
+      const before = src.slice(Math.max(0, match.index - 2000), match.index);
+      const names = [...before.matchAll(/name\s*:\s*["']([^"']+)["']/g)];
+      if (names.length > 0) preset.name = names[names.length - 1][1];
+    }
+    push(toolFromObject(node, ctx, preset));
+  }
   for (const match of src.matchAll(/(?<![.\w$])registerTool\s*\(/g)) {
     const parsed = parseArgs(src, match.index + match[0].length - 1);
     if (parsed) push(toolFromArgs(parsed.items, ctx));

package/lib/lint.js

@@ -56,6 +56,93 @@ function idStrength(name) {
 const BOUNDARY_HINT = /(不适用|不要用于|不用于|不能用于|而非|区别于|not for|do not use)/i;
 // Chinese descriptions are compared as bigrams, so mid-0.3s similarity is noise.
 const NO_BOUNDARY_SIMILARITY = 0.5;
+// Two advisory checks from real description experiments: imperatives aimed at
+// optional params can suppress the call entirely, and renamed tools leave stale
+// names in sibling descriptions. Both are info-only; they depend on wording.
+const OPTIONAL_HINT =
+  /(可选|optional|非必填|如有|如果需要|可提供|if [^.\n]{0,30}\b(provided|available|present|omitted)\b|when [^.\n]{0,30}\b(provided|available|present)\b)/i;
+const IMPERATIVE_ZH = /(传入|传|提供|带上|设置|指定|填写|附上|注明)/;
+// `Include tool messages` is a noun-phrase switch description, not an
+// imperative; `include` stays out of the start-of-description list.
+const IMPERATIVE_START_EN = /^(pass|provide|supply|set|specify|add|send|put)\b/i;
+const IMPERATIVE_START_ZH = /^(传入|传|提供|带上|设置|指定|填写|附上|注明)/;
+// English descriptions routinely start with the tool's own action
+// ("Send message to ... by sessionKey"). Require an explicit parameter link
+// ("... via the title parameter") so tool behaviour is not read as guidance.
+const BACKTICK_ID = /`([a-z][a-z0-9_-]{2,})`/g;
+const SNAKE_ID = /\b([a-z][a-z0-9]*(?:_[a-z0-9]+)+)\b/g;
+const NON_TOOL_SUFFIX =
+  /_(id|ids|key|token|path|name|url|uri|type|schema|count|size|format|length|text|json|yaml|yml|md|txt|html|htm|pdf|csv)$/;
+const NON_TOOL_NAMES = new Set([
+  "api_key", "access_token", "base_url", "content_type", "status_code", "error_code",
+  "http_status", "input_schema", "output_schema", "json_schema", "file_path", "file_name",
+  "user_id", "session_id", "tenant_id", "request_id", "trace_id", "tool_name", "mcp_server"
+]);
+const ACTION_TOKENS = new Set([
+  "get", "set", "list", "read", "write", "search", "create", "delete", "update", "send",
+  "fetch", "query", "run", "exec", "execute", "open", "close", "add", "remove", "check",
+  "scan", "parse", "score", "match", "generate", "render", "save", "load", "find", "lookup",
+  "call", "invoke", "publish", "edit", "rewrite", "analyze", "analyse", "summarize",
+  "summarise", "translate", "tool", "tools", "info", "data", "result", "results", "item",
+  "items", "skill", "skills", "mcp"
+]);
+
+function escapeRegExp(value) {
+  return String(value).replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+}
+
+function mentionsImperatively(text, name) {
+  if (!text || !name) return false;
+  let index = text.indexOf(name);
+  while (index !== -1) {
+    const around = text.slice(Math.max(0, index - 80), Math.min(text.length, index + name.length + 80));
+    // "optional charts" earlier in a long description must not mask a real
+    // imperative later. Only a hint next to this occurrence exempts it.
+    const optionalAround = text.slice(
+      Math.max(0, index - 30),
+      Math.min(text.length, index + name.length + 30)
+    );
+    if (!OPTIONAL_HINT.test(optionalAround)) {
+      if (explicitImperativeEnglish(around, name) || IMPERATIVE_ZH.test(around)) return true;
+    }
+    index = text.indexOf(name, index + name.length);
+  }
+  return false;
+}
+
+function explicitImperativeEnglish(text, name) {
+  const escaped = escapeRegExp(name);
+  const pattern = new RegExp(
+    `\\b(pass|provide|supply|set|specify|add|send|put)\\b[^.!?\\n]{0,80}(?:^|[^A-Za-z0-9_])${escaped}(?![A-Za-z0-9_])[^.!?\\n]{0,40}\\b(parameter|param|argument|field|option)\\b`,
+    "i"
+  );
+  return pattern.test(text);
+}
+
+function startsImperatively(text) {
+  const trimmed = String(text ?? "").trim();
+  if (!trimmed || OPTIONAL_HINT.test(trimmed)) return false;
+  return IMPERATIVE_START_EN.test(trimmed) || IMPERATIVE_START_ZH.test(trimmed);
+}
+
+function referencedNames(text) {
+  const found = new Set();
+  for (const match of String(text ?? "").matchAll(BACKTICK_ID)) found.add(match[1]);
+  for (const match of String(text ?? "").matchAll(SNAKE_ID)) found.add(match[1]);
+  return found;
+}
+
+function sharedDomainToken(candidate, toolNames) {
+  const candidateTokens = new Set(nameTokens(candidate).filter((token) => !ACTION_TOKENS.has(token)));
+  if (candidateTokens.size === 0) return null;
+  for (const known of toolNames) {
+    const knownTokens = new Set(nameTokens(known).filter((token) => !ACTION_TOKENS.has(token)));
+    for (const token of candidateTokens) {
+      if (knownTokens.has(token)) return { name: known, token };
+    }
+  }
+  return null;
+}
 
 // `pairScope` (a Set of names) is for incremental gates: only emit pair
 // findings that touch one of the new capabilities. Installed × installed pairs
@@ -63,6 +150,12 @@ const NO_BOUNDARY_SIMILARITY = 0.5;
 export function lintSuite(suite, { pairScope } = {}) {
   const inScope = (pair) => !pairScope || pairScope.has(pair.a) || pairScope.has(pair.b);
   const toolByName = new Map(suite.tools.map((tool) => [tool.name, tool]));
+  const toolNamesList = suite.tools.map((tool) => tool.name);
+  const knownNames = new Set([
+    ...toolNamesList,
+    ...(suite.skills ?? []).map((skill) => skill.name),
+    ...suite.tools.flatMap((tool) => Object.keys(tool.parameters?.properties ?? {}))
+  ]);
   const findings = [];
   const add = (severity, code, tool, message, extra = {}) =>
     findings.push({ severity, code, kind: "tool", tool, message, ...extra });
@@ -207,6 +300,41 @@ export function lintSuite(suite, { pairScope } = {}) {
         add("warn", "param-array-no-items", tool.name, `参数 ${name} 是数组但未定义 items`, { param: name });
       }
     }
+
+    for (const name of names) {
+      if (required.includes(name)) continue;
+      const schema = properties[name] ?? {};
+      if (mentionsImperatively(description, name) || startsImperatively(schema.description)) {
+        add(
+          "info",
+          "param-optional-imperative",
+          tool.name,
+          `可选参数 ${name} 用了祈使句提示；用户输入没提供它时，模型可能因此干脆不调用`,
+          {
+            param: name,
+            suggestion: `改成条件式（例如「如有${name}可传入」），或把 ${name} 改为必填`
+          }
+        );
+      }
+    }
+
+    for (const ref of referencedNames(description)) {
+      if (ref === tool.name || knownNames.has(ref) || NON_TOOL_NAMES.has(ref) || NON_TOOL_SUFFIX.test(ref)) {
+        continue;
+      }
+      const similar = sharedDomainToken(ref, toolNamesList);
+      const backticked = new RegExp(`\`${escapeRegExp(ref)}\``).test(description);
+      if (!similar && !backticked) continue;
+      add(
+        "info",
+        "desc-dangling-tool",
+        tool.name,
+        similar
+          ? `description 提到 ${ref}，但套件里没有这个工具；最接近的是 ${similar.name}（共享 ${similar.token}）。若是改名，请同步描述引用`
+          : `description 用代码格式提到 ${ref}，但套件里没有这个工具；确认不是改名后遗留的悬空引用`,
+        { ref, ...(similar ? { similarTo: similar.name } : {}) }
+      );
+    }
   }
 
   for (const pair of suspectedPairs(suite.tools)) {

package/lib/scan.js

@@ -16,10 +16,21 @@ const SKIP_DIRS = new Set([
   "__pycache__",
   ".idea",
   ".turbo",
-  ".cache"
+  ".cache",
+  // Not part of the agent's tool surface: generated API specs, examples and
+  // test fixtures otherwise look like tools and dominate the report.
+  "examples",
+  "example",
+  "tests",
+  "test",
+  "e2e",
+  "fixtures",
+  "swagger"
 ]);
 
-const MAX_DEPTH = 6;
+// Monorepo tool registries often sit 7–8 levels deep
+// (server/utils/agents/<framework>/plugins/<plugin>/file.js).
+const MAX_DEPTH = 8;
 // 5k was too small: a real agent monorepo (openclaw) has 16k files after the
 // skip list, so the walk stopped a third of the way in and reported 26 of its
 // 53 skills. Silent partial results are worse than a slow scan.
@@ -58,6 +69,7 @@ const CODE_PATTERNS = [
   { label: "registerTool", confidence: "medium", re: /registerTool\s*\(\s*["'`]([\w.-]+)["'`]/ },
   { label: "server.tool", confidence: "medium", re: /\.tool\s*\(\s*["'`]([\w.-]+)["'`]/ },
   { label: "py-tool-decorator", confidence: "low", re: /^\s*@tool\b/ },
+  { label: "py-action-decorator", confidence: "medium", re: /^\s*@(?:self\.)?registry\.action\s*\(/ },
   { label: "vercel-tool-call", confidence: "low", re: /\btool\s*\(\s*\{/ },
   { label: "openai-tools-array", confidence: "low", re: /\btools\s*:\s*\[\s*\{/ },
   { label: "anthropic-input-schema", confidence: "low", re: /\binput_?schema\s*:/ }
@@ -231,13 +243,17 @@ export async function scanProject(root = ".") {
       const text = await readText(file, report);
       if (!text) continue;
       if (name.endsWith(".py") && !TEST_PATH.test(rel)) {
-        const pythonTools = scanPythonTools(text);
+        const pythonTools = [
+          ...scanPythonTools(text),
+          ...scanPythonClassTools(text),
+          ...scanPythonActionDecorators(text)
+        ];
         if (pythonTools.length > 0) {
           addTools(report, pythonTools, {
             source: rel,
             origin: "py-tool",
             confidence: "medium",
-            note: "从 @tool 装饰器与类型注解提取，需人工确认"
+            note: "从 @tool 装饰器、类型注解或 class-based Tool 提取，需人工确认"
           });
         }
       }
@@ -650,6 +666,168 @@ export function scanPythonTools(text) {
   return tools;
 }
 
+// Class-based tools: smolagents-style `class X(Tool)` with `name`,
+// `description` and an `inputs` dict. Static only; never executes the class.
+export function scanPythonClassTools(text) {
+  const lines = text.split(/\r?\n/);
+  const tools = [];
+  for (let i = 0; i < lines.length; i += 1) {
+    const classMatch = lines[i].match(/^(\s*)class\s+\w+\s*\(\s*Tool\s*\)\s*:/);
+    if (!classMatch) continue;
+    const indent = classMatch[1].length;
+    let end = i + 1;
+    for (; end < lines.length; end += 1) {
+      const line = lines[end];
+      if (!line.trim()) continue;
+      if (line.match(/^\s*/)[0].length <= indent) break;
+    }
+    const joined = lines.slice(i + 1, end).join("\n");
+    const nameMatch = joined.match(/^\s*name\s*=\s*["']([^"']+)["']/m);
+    if (!nameMatch) continue;
+    tools.push({
+      name: nameMatch[1],
+      description: pythonAttrString(joined, "description") || pythonDocstring(lines, i),
+      parameters: {
+        type: "object",
+        properties: pythonInputs(joined)
+      }
+    });
+    i = end;
+  }
+  return tools;
+}
+
+// Registry actions: browser-use-style `@registry.action("desc", param_model=…)`
+// above an async function. The model class is usually in another file, so the
+// schema is limited to literal function parameters; name + description are
+// still useful for inventory and description lint.
+export function scanPythonActionDecorators(text) {
+  const lines = text.split(/\r?\n/);
+  const tools = [];
+  for (let i = 0; i < lines.length; i += 1) {
+    const decorator = lines[i].match(/^\s*@(?:self\.)?registry\.action\s*\(/);
+    if (!decorator) continue;
+    const signature = collectSignature(lines, i, decorator[0].length);
+    if (signature === null) continue;
+    const description = firstPythonString(signature.text);
+    let j = signature.endLine + 1;
+    while (j < lines.length && j - signature.endLine <= 6 && !/^\s*(?:async\s+)?def\s+\w+\s*\(/.test(lines[j])) j += 1;
+    const def = lines[j]?.match(/^\s*(?:async\s+)?def\s+(\w+)\s*\(/);
+    if (!def) continue;
+    const args = collectSignature(lines, j, def[0].length);
+    const properties = {};
+    if (args !== null) {
+      for (const part of splitPyParams(args.text)) {
+        const param = parsePyParam(part);
+        if (!param || param.name === "_" || param.name.startsWith("_")) continue;
+        if (["params", "browser_session", "context", "self", "cls"].includes(param.name)) continue;
+        properties[param.name] = param.schema;
+      }
+    }
+    tools.push({
+      name: def[1],
+      description: description || pythonDocstring(lines, j),
+      parameters: { type: "object", properties }
+    });
+    i = args?.endLine ?? signature.endLine;
+  }
+  return tools;
+}
+
+function firstPythonString(text) {
+  const triple = text.match(/(?:[rRuU])?("""|''')([\s\S]*?)\1/);
+  if (triple) {
+    return triple[2].split(/\r?\n/).map((line) => line.trim()).filter(Boolean)[0] ?? "";
+  }
+  const single = text.match(/(?:[rRuU])?["']([^"'\n]*)["']/);
+  return single ? single[1] : "";
+}
+
+function pythonAttrString(text, attr) {
+  const triple = text.match(new RegExp(`^\\s*${attr}\\s*=\\s*(?:[rRuU])?("""|''')([\\s\\S]*?)\\1`, "m"));
+  if (triple) {
+    return triple[2].split(/\r?\n/).map((line) => line.trim()).filter(Boolean)[0] ?? "";
+  }
+  const single = text.match(new RegExp(`^\\s*${attr}\\s*=\\s*["']([^"']*)["']`, "m"));
+  return single ? single[1] : "";
+}
+
+function pythonInputs(text) {
+  const start = text.search(/^\s*inputs\s*=\s*\{/m);
+  if (start === -1) return {};
+  const open = text.indexOf("{", start);
+  const block = balancedBraces(text, open);
+  if (!block) return {};
+  const properties = {};
+  for (const match of block.matchAll(/["']([\w-]+)["']\s*:\s*\{([\s\S]*?)\}/g)) {
+    const schema = {};
+    const type = match[2].match(/["']type["']\s*:\s*["']([^"']+)["']/);
+    const description = match[2].match(/["']description["']\s*:\s*["']([^"']*)["']/);
+    const mapped = type ? mapPythonInputType(type[1]) : null;
+    if (mapped) schema.type = mapped;
+    if (description) schema.description = description[1];
+    properties[match[1]] = schema;
+  }
+  return properties;
+}
+
+function balancedBraces(text, openIndex) {
+  if (openIndex < 0) return null;
+  let depth = 0;
+  let quote = null;
+  let triple = null;
+  for (let i = openIndex; i < text.length; i += 1) {
+    const char = text[i];
+    const rest = text.slice(i, i + 3);
+    if (triple) {
+      if (rest === triple) {
+        i += 2;
+        triple = null;
+      }
+      continue;
+    }
+    if (quote) {
+      if (char === "\\") {
+        i += 1;
+        continue;
+      }
+      if (char === quote) quote = null;
+      continue;
+    }
+    if (rest === '"""' || rest === "'''") {
+      i += 2;
+      triple = rest;
+      continue;
+    }
+    if (char === '"' || char === "'") {
+      quote = char;
+      continue;
+    }
+    if (char === "#") {
+      const newline = text.indexOf("\n", i);
+      if (newline === -1) break;
+      i = newline;
+      continue;
+    }
+    if (char === "{") depth += 1;
+    else if (char === "}") {
+      depth -= 1;
+      if (depth === 0) return text.slice(openIndex + 1, i);
+    }
+  }
+  return null;
+}
+
+function mapPythonInputType(type) {
+  const value = String(type).toLowerCase();
+  if (["string", "str", "text"].includes(value)) return "string";
+  if (["integer", "int", "number", "float"].includes(value)) return "number";
+  if (["boolean", "bool"].includes(value)) return "boolean";
+  if (["array", "list"].includes(value)) return "array";
+  if (["object", "dict"].includes(value)) return "object";
+  return null;
+}
+
 function collectSignature(lines, startLine, offset) {
   let depth = 1;
   let out = "";

package/mf.js

@@ -76,7 +76,7 @@ Usage:
   mf audit [--suite <file>] [--out <file>] [--json]  # 静态审计：lint + 分组边界 + 用例覆盖 + 挂载
   mf lint [--suite <file>] [--json]  # 静态检查 tools 与 skills，不调用模型
   mf check (--manifest <file> | --manifests <file>) [--installed <file>] [--json]  # 安装门禁：新能力 × 已装能力（--manifests 支持一包）
-  mf compare <before.json> <after.json> [--json]
+  mf compare <before.json> <after.json> [--fail-on-regression] [--json]
   mf ablate --tool <name> [--suite <file>] [--profile <group>] [--models <ids>] [--max-cost <amount>] [--max-tokens <n>] [--progress] [--json]
   mf cluster [--suite <file>] [--threshold <0.5>] [--max-size <10>] [--out <file>] [--json]  # 建议挂载分组 + 同族候选
   mf exec --entry <file> (--input <json> | --input-json <json>) [--runtime node|python] [--python <cmd>] [--export run] [--allow-read <paths>] [--allow-write <paths>] [--allow-net <hosts>] [--timeout <ms>] [--warmup <n>] [--json]
@@ -659,6 +659,13 @@ export async function runCompare(options, io = console) {
   const after = JSON.parse(await readFile(resolve(afterPath), "utf8"));
   const report = compareScorecards(before, after);
   io.log(options.json ? JSON.stringify(report, null, 2) : formatCompare(report));
+  const regressions = report.models
+    .filter((model) => model.comparable && model.reviewed > 0)
+    .flatMap((model) => (model.regressions ?? []).map((item) => `${model.model}/${item.id}`));
+  if (options.failOnRegression && regressions.length > 0) {
+    io.error(`regressions: ${regressions.join(", ")}`);
+    return 2;
+  }
   const changed = report.toolChanges.length > 0 || report.caseChanges.length > 0;
   return changed ? 1 : 0;
 }
@@ -1285,12 +1292,16 @@ if (isMain) {
   } else if (argv[0] === "compare") {
     const paths = argv.slice(1).filter((arg) => !arg.startsWith("--"));
     if (paths.length !== 2) {
-      process.stderr.write("usage: mf compare <before.json> <after.json> [--json]\n");
+      process.stderr.write("usage: mf compare <before.json> <after.json> [--fail-on-regression] [--json]\n");
       process.exit(1);
     }
     commandName = "compare";
     try {
-      const code = await runCompare({ paths, json: argv.includes("--json") });
+      const code = await runCompare({
+        paths,
+        json: argv.includes("--json"),
+        failOnRegression: argv.includes("--fail-on-regression")
+      });
       process.exit(code);
     } catch (error) {
       process.stderr.write(`${redactText(error?.message ?? error)}\n`);

package/package.json

@@ -1,16 +1,23 @@
 {
   "name": "modelfunction-cli",
-  "version": "0.6.0",
+  "version": "0.6.2",
   "type": "module",
   "license": "MIT",
-  "description": "mf — function-calling accuracy scorecard for AI agents",
+  "description": "Function-calling quality gate for AI agents: scan MCP tools/skills, lint description boundaries, and score model tool selection. Local-first, MCP-ready.",
   "keywords": [
-    "llm",
     "function-calling",
     "tool-use",
+    "tool-selection",
     "mcp",
-    "eval",
-    "agent"
+    "mcp-server",
+    "ai-agents",
+    "agent-evaluation",
+    "llm-eval",
+    "quality-gate",
+    "developer-tools",
+    "ci",
+    "openai",
+    "claude"
   ],
   "bin": {
     "mf": "mf.js",

package/skills/mf-cases/SKILL.md

@@ -31,6 +31,13 @@ description: 为 ModelFunction 评测套件撰写 function-calling 用例。当
    - `{ "$exists": true|false }`（只关心有没有）
 6. **不要用被测模型生成用例**，也不要让被测模型参与出题。
 
+# 描述边界写法（实测）
+
+- 排除具体竞争者时**点名**：`do NOT call file_read` 有效；泛化否定（`no separate file reads`）几乎压不住任何工具。
+- 可选参数不要用祈使句（“传一个 title”）；用户没给该参数时，模型可能干脆不调用。改成条件式：“如有 title 可传入”。
+- 改名工具后，检查所有 description 是否还引用旧名；悬空引用会继续误导模型。
+- 用例话术必须提交范围（公网 / 本地 / 会话历史）；“上周那个记录”这类歧义无法靠描述修复，只能改话术。
+
 # 输出格式
 
 ```json