dsh-agentic-tool-catalog 0.0.0-stage → 0.3.1

package/CHANGELOG.md

@@ -0,0 +1,33 @@
+# Changelog
+
+## 0.3.1 — 2026-11-11
+
+Documentation-only release: the plugin page was rewritten around the user's questions (what it
+does, what it saves, what to know), the product hero and benchmark charts were added, and the
+in-session screenshot section / implementation jargon were removed. No runtime behavior change.
+
+## 0.3.0 — 2026-11-11
+
+Renamed from `dsh-tool-discovery` / Tool Discovery; public identity is now **Agentic Tool Catalog**.
+
+- **Name/identity**: `@local/dsh-agentic-tool-catalog`, row id `agentic-tool-catalog`,
+  display "Agentic Tool Catalog / Agent 工具目录"; tools renamed `discovery_status` → `catalog_status`;
+  prompt section `catalog:note`; config `agentGroup*` → `catalog*`; internal `policy.js` → `catalog.js`,
+  `agent-groups.js` → `generation.js`.
+- **Persistence**: moved the catalog cache from a self-managed JSON file to official DSH Storage
+  (`ctx.storageDomain`, domain `agentic_tool_catalog`, schema-validated). No second file backend;
+  profiles without a storage service use memory-only caching and report `storage.mode = "memory"`.
+- **Status contract**: `catalog_status` returns fixed sections `catalog` / `session` / `generation` /
+  `storage` / `warnings`.
+- **Migration fix**: when replacing the old `dsh-tool-discovery` bundle, release restrictions left in the
+  legacy global registry so a renamed plugin does not inherit an orphaned deny set.
+- **Empty catalog fix**: with nothing deferred (e.g. every tool resident) the plugin no longer spawns a
+  grouping subagent that can only fail `TOO_FEW_GROUPS`; it emits an empty catalog directly. The group floor
+  now scales down for catalogs with fewer than 6 tools, and validation accepts exactly `0` groups for `0` tools.
+- **Docs**: bilingual README (`README.md` / `README.zh.md`), rewritten product detail pages, MIT LICENSE.
+
+## 0.2.0 — 2026-11-10
+
+- Agent-curated catalog only (all legacy/rules/BM25/heuristic strategies removed).
+- Lazy/pre-warm generation, incremental regeneration, official freshness/origin reporting,
+  last-known-good continuation with explicit `catalog-stale` warnings.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 DeepSeek Harness plugins contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,168 @@
-# Temporary Holding Version
+# Agentic Tool Catalog
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**An agent builds and maintains the tool catalog: schemas stay out of context, tools load on demand.**
+
+![Agentic Tool Catalog](docs/assets/ui-panel.png)
+
+[English](README.md) | [中文](README.zh.md)
+
+DeepSeek Harness normally sends every visible tool schema on every request. With many MCP servers and
+extension plugins that gets expensive: context bloat, slower requests, and degraded tool selection once the
+catalog grows past a few dozen tools.
+
+Agentic Tool Catalog keeps a **directory** of deferred tools instead. A real DSH sub-agent reads the current
+tool inventory, groups tools by intent, self-checks coverage, and returns a catalog. The plugin validates it,
+caches it in **official DSH Storage**, and only ever sends a few lines of directory text — tools are loaded
+by `search_tools` when a task actually needs them.
+
+## What you get
+
+- **Smaller requests**: deferred tool schemas never enter the context until loaded (measured 91 deferred
+  schemas ≈ 109.6 KB ≈ ~27k tokens replaced by a 443-character directory note in this profile).
+- **Agent-curated grouping**: groups are generated by an agent (not by name prefixes, not by a manual list):
+  browser tools split by intent, Google tools split by product, cross-server capabilities merged.
+- **Zero cooperation from other plugins**: no declarations, no metadata, no code changes. Any installed tool
+  is discovered automatically.
+- **Local-first**: the generator runs on your DSH model; no external service is required.
+- **Reliable by design**: strict validation, one retry with feedback, single-flight generation, persistent
+  cache (official `ctx.storageDomain`), incremental regeneration on catalog changes, and a visible
+  "stale / refreshing" state if generation cannot complete. It always keeps working with the last successful
+  catalog or a flat bootstrap directory — and never silently hides the failure.
+- **Observable**: `catalog_status` exposes catalog freshness/origin, groups, loaded tools, generation and
+  storage health.
+
+## How it works
+
+```
+agent needs tools
+  → tool catalog fingerprint (tool names + cache namespace)
+  → hot cache / official Storage hit?  ── yes → use catalog
+  → no: generate with a DSH sub-agent (incremental if a previous catalog exists)
+        → validate (100% coverage, no duplicates, sane groups)
+        → cache in official Storage + apply catalog (restriction releases tools on demand)
+        → failure: retry once with error feedback; still failing → keep last successful grouping
+          (status = stale, explicit warning + backoff retry) or flat bootstrap directory
+```
+
+- **New sessions** start generation immediately (pre-warm); **restored sessions** generate on their first
+  user message/search; **catalog changes** regenerate immediately (incremental first).
+- **Generation is keyed by the tool set, not by time**: one generation per unique tool fingerprint; the result is
+  cached in official Storage and reused across sessions/restarts. There is no periodic refresh — the timestamp
+  only moves when the tool set actually changes (install/remove a plugin or MCP server, preset change, etc.).
+- Tools are hidden from the model with `agent.ctx.tools.restrict({ deny })`; `search_tools` releases them
+  through DSH's native mid-conversation tool-addition path.
+- The catalog cache lives in official DSH Storage (`ctx.storageDomain`, domain `agentic_tool_catalog`).
+  If a profile has no storage service, the plugin falls back to memory-only caching (still functional;
+  one regeneration after restart) and reports `storage.mode = "memory"`.
+
+## Model-facing tools
+
+| Tool | Parameters | Behavior |
+|---|---|---|
+| `search_tools` | `query: string`, `limit?: number` | Search the catalog by intent/keywords (name, description, group id/description). Exact group-name queries load the whole group (up to `groupMaxResults`). Loaded tools become callable for the rest of the session. |
+| `catalog_status` | — | Read-only JSON: `catalog` (source/freshness/origin/total/groups/others), `session` (loaded/pending/restriction), `generation` (runs, cache hits, failures, retries, dedupe, stale uses, incremental runs, last error/ok, instance), `storage` (mode/state), `warnings`. |
+
+A short `catalog:note` system-prompt section lists deferred groups and counts; during bootstrap/stale states it
+adds one explicit line explaining that the catalog is initializing/refreshing.
+
+## Configuration
+
+```yaml
+- id: agentic-tool-catalog
+  name: 'dsh-agentic-tool-catalog'
+  config:
+    maxResults: 6            # tools loaded by one generic search
+    groupMaxResults: 30      # tools loaded by an exact group-name search
+    promptNote: true         # inject the catalog:note system-prompt section
+    resident: []             # override the built-in resident list (tool names/globs)
+    extraKeepResident: []    # append globs that must always stay visible
+    extraDeferred: []        # force globs to stay deferred
+    catalogProvider: spawn   # DSH subagent provider used for generation
+    catalogTimeoutMs: 120000 # per-generation timeout
+    catalogResetNonce: ''    # cache namespace; change it to force regeneration
+```
+
+Most profiles need no configuration. New plugins/MCP servers are picked up automatically on the next catalog
+change; if a tool must always be visible, add it to `extraKeepResident`.
+
+## Install
+
+Local development (this workspace):
+
+```bash
+# via the DSH plugin manager
+dsh plugin --profile web add dsh-agentic-tool-catalog
+# local development (this repository)
+dsh plugin --profile web add /path/to/dsh-agentic-tool-catalog
+# or, from a shell in the DSH checkout
+dsh plugin add /path/to/dsh-agentic-tool-catalog
+```
+
+Published package: `dsh-agentic-tool-catalog` (npm). Requires DSH `0.2.2-rc.1` or later.
+
+```bash
+dsh plugin --profile web add dsh-agentic-tool-catalog
+```
+
+## Performance (A/B: with vs without)
+
+Measured on 5 tasks × 2 independent sessions per arm, same model and tool surface (122 tools incl. Google
+Workspace MCP), DSH 0.2.2-rc.1, `deepseek-flash`:
+
+| Metric | without (all tools resident) | with this plugin | Change |
+|---|---:|---:|---:|
+| Task pass rate | 100% (10/10) | 100% (10/10) | equal |
+| First-request tool schema (payload) | 122 tools / 131,390 chars | 31 tools / 21,877 chars | **−83.3%** |
+| **First-request real prompt tokens** (usage) | **49,504** | **18,536** | **−62.6% / −30,968 per request** |
+| Avg total tokens / session | 452,463 | 268,551 | **−40.6%** |
+| Avg steps | 7.40 | 7.90 | +6.8% |
+| Avg cost / session | $0.0112 | $0.0116 | +3.7% (≈flat) |
+| Avg wall time | 37.0s | 33.3s | −9.9% |
+| Avg `search_tools` calls | 0.10 | 1.30 | +1.2/session |
+
+One-time catalog generation (background, cached in official DSH Storage) is excluded from these task metrics.
+Full methodology and caveats: [`docs/benchmark.md`](docs/benchmark.md) · full data card: [`docs/data-card.md`](docs/data-card.md).
+
+## UI (minimal, read-only)
+
+An icon + a small on-demand-tool count sits in the official composer dock (hover shows the catalog summary and
+uses the same hover treatment as the official context meter). Click it for a read-only panel modeled on the
+official ContextMeter: a segmented bar plus three rows (always available / loaded / on demand), a status dot and
+freshness word, and one meta line (groups · storage · updated). There is no directory dump, no configuration, and
+no long explanatory copy. It uses official slots/theme tokens and follows the light/dark theme.
+
+## Verification
+
+```bash
+node scripts/verify-agentic-tool-catalog.mjs          # module/regression tests
+node scripts/verify-agentic-tool-catalog-schema.mjs   # real dsh-tools/schemastery/storage-domain compile
+node scripts/verify-presentation.mjs                 # presentation files (locale/icon/detail)
+```
+
+Benchmark methodology and data live in [`benchmark/`](../../benchmark/discovery/) and are summarized in the
+plugin page / release data card (compare **with vs without** the plugin on the same tasks).
+
+## Safety & privacy
+
+- Everything runs locally: the catalog is generated by the configured DSH sub-agent; no external service is required.
+- The cache contains only tool names/group descriptions and lives in official DSH Storage (`ctx.storageDomain`). It never
+  contains credentials, prompts, message content or tool results; it is derived data that can be regenerated at any time.
+- The UI status panel is read-only, same-origin, and scoped to the current session; it exposes no secrets.
+- The plugin does not modify other plugins or install anything outside DSH's plugin system.
+
+## Known limits
+
+- The cache is derived data. Losing it costs one regeneration (tens of seconds once per catalog); it is not
+  user data and is not migrated between storage backends.
+- Cold start: the very first run has no catalog yet and starts with a flat bootstrap directory while the agent
+  generates one in the background. Subsequent runs, restarts and catalog changes use the persisted/incremental path.
+- Generation quality depends on the configured model; the plugin validates strictly and retries/falls back
+  with a visible status, but it cannot make a bad grouping "correct" silently.
+- No UI dashboard yet (`catalog_status` is the observable interface); tools only (skills are not part of the
+  catalog). Scale beyond a few hundred tools is not benchmarked yet.
+- Uses the public `tools.definitions/restrict` surface and the official `storageDomain` API; both are
+  DSH-plugin-facing APIs subject to the usual `0.2.x` stability caveats.
+
+## License
+
+MIT.

package/README.zh.md

@@ -0,0 +1,134 @@
+# Agentic Tool Catalog（Agent 工具目录）
+
+**由 Agent 自动生成并维护工具目录：工具 schema 默认不占上下文，按需加载。**
+
+[English](README.md) | [中文](README.zh.md)
+
+DeepSeek Harness 默认把**每个可见工具的完整 schema** 放进**每一次**模型请求。接的 MCP/扩展插件一多，
+上下文就会膨胀、请求变慢，工具超过几十个后模型选择准确率也会下降。
+
+Agentic Tool Catalog 换一种做法：上下文里只保留一份**目录**。插件启动一个**真正的 DSH 子代理**，
+读取当前工具清单、按用户意图分组、自检覆盖率，交回一份目录；插件严格校验后缓存到**官方 DSH Storage**，
+每次请求只发几行目录文字，具体工具由 `search_tools` 在需要时加载。
+
+## 你能得到什么
+
+- **更小的请求**：未加载工具 schema 不进上下文（本机实测：91 个延迟工具 schema 合计约 109.6 KB ≈ 2.7 万 tokens，
+  被 443 字符的目录提示替代）；
+- **Agent 生成的语义分组**：按意图拆组（浏览器拆成操作/观察/判断；Google 按邮件/日历/登录拆；跨 server 能力合并），
+  而不是按名字前缀；
+- **零插件配合**：无需其他插件声明字段、元数据或改代码；装了就会被自动发现；
+- **本地优先**：生成使用本机 DSH 模型，不需要外部服务；
+- **可靠性**：严格校验、失败带反馈重试一次、同指纹单飞、官方 Storage 持久化、目录变化增量重生成；
+  生成没完成时状态会显式标 `stale`（或使用扁平引导目录），**不会静默失败**；
+- **可观测**：`catalog_status` 提供目录新鲜度/来源、分组、已加载、生成与存储健康状态。
+
+## 工作机制
+
+```
+需要工具
+  → 目录指纹（工具名 + 缓存命名空间）
+  → 内存/官方 Storage 命中？ ── 是 → 直接使用
+  → 否：用 DSH 子代理生成（有上次目录时优先增量）
+        → 校验（100% 覆盖、无重复、组合理）
+        → 写入官方 Storage 并应用目录（按需释放工具）
+        → 失败：带错误反馈重试一次；仍失败 → 沿用最近一次成功分组
+          （状态 stale + 显式告警 + 退避重试）或扁平引导目录
+```
+
+- **新会话**创建时立即预热生成；**恢复的旧会话**在第一条用户消息/搜索时才生成；**目录变化**立即重新生成（优先增量）；
+- **生成由工具集决定，不是按时间**：同一份工具指纹只生成一次，结果缓存进官方 Storage，跨会话/重启复用；
+  **没有定时刷新**——只有工具集真的变了（装/卸插件、MCP server 上下线、preset 变化等）才会重新生成。
+- 工具用 `agent.ctx.tools.restrict({ deny })` 从模型请求中隐藏，`search_tools` 命中后通过 DSH 原生
+  tool-addition 机制在会话中途释放；
+- 缓存走官方 `ctx.storageDomain`（domain `agentic_tool_catalog`）；若 profile 没有 storage 服务，
+  降级为纯内存缓存（功能可用、重启后重新生成一次），并在 `storage.mode` 明确报告 `memory`。
+
+## 模型可见工具
+
+| 工具 | 参数 | 行为 |
+|---|---|---|
+| `search_tools` | `query: string`，`limit?: number` | 按意图/关键词搜索目录（名字、描述、组名/组描述）；query 精确等于组名时整组加载（≤`groupMaxResults`）；加载后本会话可直接调用 |
+| `catalog_status` | — | 只读 JSON：`catalog`（来源/新鲜度/来源类型/总数/分组/others）、`session`（已加载/待加载/restriction）、`generation`（次数、缓存命中、失败、重试、去重、stale 次数、增量次数、last error/ok、实例）、`storage`（模式/状态）、`warnings` |
+
+系统提示里有一个 `catalog:note` 段：列出延迟分组和数量；初始化中/刷新中会额外写明一行状态。
+
+## 配置
+
+```yaml
+- id: agentic-tool-catalog
+  name: 'dsh-agentic-tool-catalog'
+  config:
+    maxResults: 6            # 一次普通搜索最多加载的工具数
+    groupMaxResults: 30      # 按组名整组加载的上限
+    promptNote: true         # 是否注入 catalog:note 提示
+    resident: []             # 覆盖内置常驻清单（工具名/glob）
+    extraKeepResident: []    # 追加"必须常驻"的 glob
+    extraDeferred: []        # 强制按需的 glob
+    catalogProvider: spawn   # 生成用子代理 provider
+    catalogTimeoutMs: 120000 # 单次生成超时
+    catalogResetNonce: ''    # 缓存命名空间；改它可强制重新生成
+```
+
+大多数情况下**不需要任何配置**；新装插件/MCP 在目录变化时自动收编；只有"某工具必须永远可见"时才需要
+`extraKeepResident`。
+
+## 安装
+
+本地开发（本工作区）：
+
+```bash
+dsh plugin --profile web add dsh-agentic-tool-catalog
+# 本地开发（本仓库）
+dsh plugin --profile web add /path/to/dsh-agentic-tool-catalog
+# 或在 DSH checkout 里
+dsh plugin add /path/to/dsh-agentic-tool-catalog
+```
+
+发布包：`dsh-agentic-tool-catalog`（npm）；要求 DSH `0.2.2-rc.1` 及以上。
+
+## 性能（A/B：用 vs 不用）
+
+5 个任务 × 每臂 2 轮独立会话，同模型、同工具面（122 tools，含 Google Workspace MCP），DSH 0.2.2-rc.1、`deepseek-flash`：
+
+| 指标 | 不用（全量常驻） | 用本插件 | 变化 |
+|---|---:|---:|---:|
+| 任务通过率 | 100% (10/10) | 100% (10/10) | 持平 |
+| 首请求工具 schema（载荷） | 122 tools / 131,390 chars | 31 tools / 21,877 chars | **−83.3%** |
+| **首请求真实 prompt tokens**（usage） | **49,504** | **18,536** | **−62.6% / 每请求 −30,968** |
+| 平均总 tokens/会话 | 452,463 | 268,551 | **−40.6%** |
+| 平均步数 | 7.40 | 7.90 | +6.8% |
+| 平均成本/会话 | $0.0112 | $0.0116 | +3.7%（≈持平） |
+| 平均墙钟 | 37.0s | 33.3s | −9.9% |
+| 平均 `search_tools` 调用 | 0.10 | 1.30 | +1.2/会话 |
+
+一次性目录生成（后台、缓存到官方 DSH Storage）不计入上表任务指标；完整方法与边界见 [`docs/benchmark.md`](docs/benchmark.md) · 完整数据卡：[`docs/data-card.md`](docs/data-card.md)。
+
+## 最小 UI（只读）
+
+官方输入框下方是 **图标 + 一个极小的按需工具数**（hover 有摘要，hover/展开样式与官方 ContextMeter 一致）。
+点击展开只读面板（照官方 ContextMeter 的尺寸/token 设计）：常驻/已加载/待加载分段条 + 三行图例 +
+状态点/新鲜度 + 一行“分组 · 存储 · 更新时间”。没有目录名堆叠、没有配置项、没有长说明；
+使用官方 slot/主题变量，自动跟随明暗主题。
+
+## 验证
+
+```bash
+node scripts/verify-agentic-tool-catalog.mjs          # 模块/回归测试
+node scripts/verify-agentic-tool-catalog-schema.mjs   # 真实 dsh-tools/schemastery/storage-domain 编译
+node scripts/verify-presentation.mjs                 # 展示文件检查（locale/icon/detail）
+```
+
+Benchmark 方法/数据见 [`benchmark/`](../../benchmark/discovery/)；发布数据卡按**用 vs 不用插件**对照。
+
+## 已知限制
+
+- 缓存是**派生数据**：丢了最多重新生成一次（每个目录一次）；它不是用户数据，不做跨介质迁移；
+- 冷启动首次没有目录时，会先用扁平引导目录，后台生成完成后自动替换；之后重启/目录变化走持久化/增量路径；
+- 生成质量取决于配置的模型：插件会严格校验、重试并在状态里显式标注，但不会"偷偷把坏分组当成正常"；
+- 暂无 UI 仪表盘（`catalog_status` 是观测入口）；目前只处理 tools（不含 skills）；数百个工具以上的规模尚未基准测试；
+- 依赖公开的 `tools.definitions/restrict` 与官方 `storageDomain` API，遵循 DSH `0.2.x` 的稳定性边界。
+
+## License
+
+MIT。

package/cordis.patch.yml

@@ -0,0 +1,29 @@
+# dsh-agentic-tool-catalog —— 按需发现工具（tool search / 延迟加载）
+#
+# 本 bundle 只持有自己的插件行。MCP 能力由各自集成 bundle 持有
+# （谁集成、谁持有自己的组件）：
+#   - Cloudflare API MCP（OAuth）→ plugins/dsh-cloudflare
+#   - Google Workspace MCP → plugins/gws-mcp
+# 之前放在这里的 `mcp-cfapi` / `mcp-cfdocs` 两行已于 2026-11-10 移除
+# （Cloudflare 集成迁入 dsh-cloudflare 插件；cfdocs 已被 API MCP 的 docs 工具覆盖）。
+#
+# 注意：只有 `insert` 会新增行；带 `id` 的顶层条目是"覆盖已有行"语义。
+- insert:
+    - id: agentic-tool-catalog
+      name: 'dsh-agentic-tool-catalog'
+      config:
+        # 分组是插件运行时按策略自动推导的（默认 rules）：
+        #   mcp__<server>__* → mcp-<server>；其余按工具名自动聚类，归不进组的进 others。
+        # discovery 源码里没有任何插件工具名；新增 MCP/插件工具不需要改这里。
+        #
+        # 唯一可能需要改的：某个新工具必须"永远可见、不能按需"时，往这里追加一行
+        # （内置常驻白名单：bash/fs/jobs/skill/ask/todo/web/present/plan/delegation/
+        #  mcp-resources/Agent Teams 等）：
+        # extraKeepResident:
+        #   - some_new_tool
+        #
+        # 一次 search 最多加载 6 个工具（避免一次拉回 21 个 workspace 工具）。
+        maxResults: 6
+        # 生产：唯一策略 = 真 agent 自动分组（目录变化时后台生成一次、按目录指纹缓存、
+        # 跨重启磁盘缓存、last-known-good 降级、无外部模型依赖；旧策略已全部移除）。
+        # 详见 notes/2026-11-10-agent-strategy-production.md；benchmark 见 benchmark/discovery/。

package/docs/benchmark.md

@@ -0,0 +1,54 @@
+# Performance: with vs without Agentic Tool Catalog
+
+Measured 2026-11-11 on DSH `0.2.2-rc.1` (desktop profile), model `deepseek-flash`, Google Workspace MCP online.
+Same 5 tasks × 2 independent sessions per arm; verified independently.
+
+- **baseline (without on-demand catalog)**: all 122 tools resident, every request carries every schema.
+- **treatment (this plugin)**: 31 resident tools + a short catalog note; 92 deferred tools loaded on demand.
+
+| Metric | baseline | treatment | Change |
+|---|---:|---:|---:|
+| Task pass rate | 100% (10/10) | 100% (10/10) | equal |
+| First-request tools | 122 | 31 | −74.6% |
+| First-request tool-schema chars (payload) | 131,390 | 21,877 | **−83.3%** |
+| **First-request real prompt tokens** (usage, not estimate) | **49,504** | **18,536** | **−62.6% (−30,968 tokens/request)** |
+| Max request schema chars | 131,390 | 34,785 | −73.5% |
+| Avg total tokens / session | 452,463 | 268,551 | **−40.6%** |
+| Avg steps | 7.40 | 7.90 | +6.8% |
+| Avg cost / session | $0.0112 | $0.0116 | +3.7% (≈flat) |
+| Avg wall time | 37.0s | 33.3s | −9.9% |
+| Avg `search_tools` calls | 0.10 | 1.30 | +1.2/session |
+
+**Headline**: same task success, **−83.3% tool-schema payload, −62.6% real first-request prompt tokens
+(−30,968 tokens/request)**, and **−41% total session tokens**; the cost is ~1.2 extra catalog searches per session and +7% steps.
+Cost/wall differences are within run-to-run noise.
+
+
+Quality (same runs, from the discovery output):
+
+| Metric | baseline | treatment |
+|---|---:|---:|
+| Target capability hit | 10/10 | 10/10 |
+| missedCapability | 0 | 0 |
+| Searches / session | 0.10 | 1.30 |
+| Setup searches before target call | 0 | 1.20 |
+| Tools loaded / search | — | 9.2 |
+| Loaded tools actually used | — | 1.9 |
+| loadedUsedRatio | — | **0.289** |
+
+Honest read: the catalog trades "see a little more" for "carry much less"; search currently loads ~9 tools and
+uses ~2, so the loaded-used ratio is a tuning target (tighter `maxResults`/ranking), not something hidden here.
+
+Notes:
+- One-time generation (per catalog version, background) is not included in task metrics; it is cached in official
+  DSH Storage and reused across sessions/restarts, with incremental regeneration on catalog changes.
+- This card does **not** claim cost savings: cost is ~flat (+3.7%); it saves context/tokens. Sample is 5 tasks ×
+  2 runs, one model (`deepseek-flash`), one profile — not a cross-model claim.
+- Catalog generation is a one-time background cost and is **not** included in the task metrics; its exact
+  token/cost instrumentation is future work (no precise number is claimed).
+- Tasks: `host-ops-session-list`, `docs-workers-limit`, `terminal-echo`, `goal-set`, `docs-d1-storage`.
+  `workspace-calendar-count` is excluded from the main table because it depends on MCP availability.
+- First-request token figures come from real session usage (`inputTokens + cacheReadTokens`); schema chars are a
+  payload-size metric, **not** a chars/4 token estimate.
+- Full methodology and raw results: `benchmark/discovery/data-card.md` and `benchmark/discovery/results/` in the
+  source repository.

package/docs/data-card.md

@@ -0,0 +1,96 @@
+# 性能数据卡：Agentic Tool Catalog（用 vs 不用）
+
+> 目标：用**同一批任务、同一模型、同一工具面**回答一个问题——
+> **装上 Agentic Tool Catalog，和"所有工具常驻上下文"相比，付多少、得多少？**
+
+## 1. 对照设置
+
+| | baseline（不用插件） | treatment（用插件） |
+|---|---|---|
+| 上下文形态 | 所有工具常驻：每次请求带全部 122 个工具 schema | 31 个常驻工具 + 一小段目录提示；延迟工具 schema 不进请求 |
+| 配置 | 插件 `resident: ['*']`（目录为空，不搜索） | 生产配置（目录由 agent 生成并缓存，`fresh`） |
+| 工具面 | 122 tools（含 Google Workspace MCP 22 个） | deferred 92 tools / 19 组，`catalog_status.storage=ready` |
+| 模型 / 环境 | `deepseek-flash`（provider `deepseek-account`）、DSH `0.2.2-rc.1`、同一桌面 profile | 同左 |
+
+- 任务集（5 个，每任务 2 轮独立会话，共 10+10 run）：
+  `host-ops-session-list`、`docs-workers-limit`、`terminal-echo`、`goal-set`、`docs-d1-storage`；
+- 所有任务由 `benchmark/discovery/run.mjs` 记录，`verify` 独立核验文件/结果；
+- `workspace-calendar-count` 未纳入主表：它依赖 Google Workspace MCP 的可用性，属环境敏感任务；
+- 指标来自 DSH 会话日志：首请求工具 schema 字符（官方 `metrics.firstRequestTools`）、tokens、成本、步数、墙钟、
+  `search_tools` 调用次数。
+
+## 2. 结果（10 + 10 run，全部有效）
+
+| 指标 | baseline（全量常驻） | treatment（按需目录） | 变化 |
+|---|---:|---:|---:|
+| 任务通过率 | **100%** (10/10) | **100%** (10/10) | 持平 |
+| 首请求工具数 | 122 | 31 | **−74.6%** |
+| 首请求工具 schema 字符（纯 schema 载荷） | 131,390 | 21,877 | **−83.3%** |
+| **首请求真实 prompt tokens**（会话日志 usage，非估算） | **49,504** | **18,536** | **−62.6%（−30,968 tokens/请求）** |
+| 最大请求 schema 字符（任务全程） | 131,390 | 34,785 | **−73.5%** |
+| 平均总 tokens / 会话 | 452,463 | 268,551 | **−40.6%** |
+| 平均步数 | 7.40 | 7.90 | +6.8% |
+| 平均成本 / 会话 | $0.0112 | $0.0116 | +3.7%（≈持平） |
+| 平均墙钟 | 37.0s | 33.3s | −9.9% |
+| 平均 `search_tools` 调用 | 0.10 | 1.30 | +1.2 次/会话 |
+
+**一句话**：任务成功率不变，**首请求工具 schema 载荷降 83.3%，首请求真实 prompt tokens 从 49,504 降到 18,536（−62.6%，约 −31k tokens/请求）**，会话总 tokens 降约 41%；
+代价是每会话多约 1.2 次目录搜索、步数 +7%；成本与墙钟在噪声范围内（成本 +3.7%、墙钟 −9.9%）。
+
+
+## 3. 质量指标（同一批结果的 discovery 输出）
+
+| 指标 | baseline（全量常驻） | treatment（按需目录） |
+|---|---:|---:|
+| 目标能力命中（targetToolCalled） | 10/10 | 10/10 |
+| missedCapability | 0 | 0 |
+| 搜索次数 / 会话 | 0.10 | 1.30 |
+| 目标工具调用前的 setup 搜索 | 0 | 1.20 |
+| 每次搜索平均加载工具数 | — | 9.2 |
+| 加载后实际使用工具数 | — | 1.9 |
+| loadedUsedRatio（加载后使用率） | — | **0.289** |
+| 工具调用失败（环境摩擦，非插件失败） | 6 | 2 |
+
+诚实解读：目录让模型"多看少带"——每次搜索会加载约 9 个工具、实际用到约 2 个；
+**加载后使用率 0.289 说明搜索返回偏宽**，这是下一步可优化项（收紧 maxResults/排序或按组加载），
+不是数据卡要掩盖的点。目标能力 10/10 命中、0 missed，说明"该找的工具都找到了"。
+
+### 逐任务首请求 prompt tokens（真实 usage）
+
+| 任务 | baseline | treatment | 变化 |
+|---|---:|---:|---:|
+| host-ops-session-list | 49,511 | 18,543 | −62.5% |
+| docs-workers-limit | 49,516 | 18,548 | −62.5% |
+| terminal-echo | 49,483 | 18,515 | −62.6% |
+| goal-set | 49,477 | 18,509 | −62.6% |
+| docs-d1-storage | 49,533 | 18,565 | −62.5% |
+
+## 4. 一次性成本与持久化（不混入上表）
+
+- 目录生成由 DSH 子代理完成，**每个目录版本一次**（后台，不阻塞任务）；本机 92 工具目录的实测生成约 60–75s、
+  约 6–7 万 tokens 量级（≈ 一次生成），完成后按目录指纹缓存到**官方 DSH Storage**；
+- 之后的会话/重启走缓存（`catalogOrigin=storage`），**零生成**；目录变化时增量重生成；
+- 数据卡上表不含生成成本；生成失败不会静默：状态标 `stale`/`refreshing`，自动带反馈重试并从上次成功分组继续。
+
+## 5. 可靠性与观测（同期验证）
+
+- A/B 的 10 个 treatment run：首请求全部 `31 tools / 21,877 chars`，无 bootstrap/error/stale；
+- 目录与缓存：`catalog_status` 显示 `fresh`、`origin=storage`、`storage.state=ready`；
+- 一次真实故障演练（开发期 Google Workspace MCP 抖动）中，生成校验失败 4 次、按预期走重试/
+  最近一次成功分组（`staleUses=5`），**没有静默隐藏工具**；MCP 恢复后目录继续 `fresh`。
+
+## 6. 边界与复现
+
+- 收益主要来自"延迟工具 schema 不进每次请求"；工具越多、MCP/扩展越多，收益越大；工具很少（<10）时收益有限；
+- 对照依赖同一工具面：需要 Google Workspace MCP 在线（本卡 122 工具面）；MCP 掉线会改变首请求工具数，
+  本卡使用的是工具面一致的 10–4x 批次；
+- 首请求 tokens 来自会话日志真实 usage（非 chars/4 估算）；schema 字符是载荷指标。成本/墙钟受模型与网络影响，应看多轮重复；
+- 复现：`node benchmark/discovery/card.mjs --baseline baseline-resident --treatment agent-catalog --tasks host-ops-session-list,docs-workers-limit,terminal-echo,goal-set,docs-d1-storage --markdown`
+  原始结果在 [`results/`](results/)，方法说明 [`benchmark/discovery/README.md`](README.md)。
+
+## 7. 和竞品怎么比
+
+- 竞品（`dsh-tool-search`、`dsh-tool-activator`、`dsh-capability-catalog`、`dsh-mcp-lens` 等）多数以
+  **搜索/手动激活/人工登记**组织工具；本插件差异是 **agent 自动生成并维护语义目录**，
+  本卡只回答"用 vs 不用"，不做无法公平对齐的跨插件数字对比；定性差异与竞品调研见
+  [`notes/2026-11-11-plugin-naming-interface.md`](../../notes/2026-11-11-plugin-naming-interface.md)。

package/docs/detail-benchmark.svg

@@ -0,0 +1,30 @@
+<svg width="760" height="330" viewBox="0 0 760 330" fill="none" xmlns="http://www.w3.org/2000/svg">
+  <rect width="760" height="330" rx="20" fill="#F5F7FB"/>
+  <text x="32" y="48" font-family="system-ui, -apple-system, sans-serif" font-size="16" font-weight="700" fill="#1F2329">With vs without Agentic Tool Catalog</text>
+  <text x="32" y="70" font-family="system-ui, -apple-system, sans-serif" font-size="11.5" fill="#6B7280">Same tasks · same model · same tool surface · verified independently · 10 runs per arm</text>
+
+  <!-- schema payload -->
+  <text x="32" y="112" font-family="system-ui, -apple-system, sans-serif" font-size="12" fill="#1F2329">First-request tool-schema payload</text>
+  <rect x="290" y="100" width="300" height="14" rx="7" fill="#D8DEE7"/>
+  <text x="600" y="112" font-family="system-ui, -apple-system, sans-serif" font-size="11.5" fill="#6B7280">131,390 chars</text>
+  <rect x="290" y="120" width="50" height="14" rx="7" fill="#145AF3"/>
+  <text x="350" y="132" font-family="system-ui, -apple-system, sans-serif" font-size="11.5" fill="#145AF3">21,877 chars · −83.3%</text>
+
+  <!-- prompt tokens -->
+  <text x="32" y="172" font-family="system-ui, -apple-system, sans-serif" font-size="12" fill="#1F2329">First-request prompt tokens (real usage)</text>
+  <rect x="290" y="160" width="300" height="14" rx="7" fill="#D8DEE7"/>
+  <text x="600" y="172" font-family="system-ui, -apple-system, sans-serif" font-size="11.5" fill="#6B7280">49,504</text>
+  <rect x="290" y="180" width="112" height="14" rx="7" fill="#145AF3"/>
+  <text x="412" y="192" font-family="system-ui, -apple-system, sans-serif" font-size="11.5" fill="#145AF3">18,536 · −62.6%</text>
+
+  <!-- total tokens -->
+  <text x="32" y="232" font-family="system-ui, -apple-system, sans-serif" font-size="12" fill="#1F2329">Total tokens per session</text>
+  <rect x="290" y="220" width="300" height="14" rx="7" fill="#D8DEE7"/>
+  <text x="600" y="232" font-family="system-ui, -apple-system, sans-serif" font-size="11.5" fill="#6B7280">452,463</text>
+  <rect x="290" y="240" width="178" height="14" rx="7" fill="#145AF3"/>
+  <text x="478" y="252" font-family="system-ui, -apple-system, sans-serif" font-size="11.5" fill="#145AF3">268,551 · −40.6%</text>
+
+  <rect x="32" y="280" width="696" height="28" rx="8" fill="#E9EFFB"/>
+  <text x="46" y="295" font-family="system-ui, -apple-system, sans-serif" font-size="11" fill="#33507A">Task success: 100% both arms · cost per session ≈ flat (+3.7%)</text>
+  <text x="46" y="309" font-family="system-ui, -apple-system, sans-serif" font-size="11" fill="#33507A">One-time background catalog generation is not included in these task figures.</text>
+</svg>