dsh-auto-model-routing 0.3.0 → 0.4.1

package/CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## 0.4.1
+
+- Add the on-demand `auto-model-routing-report` skill and read-only `lib/usage-report.js`: joins policy-v2 decisions to actual per-call usage by message ID, grouped by version × model × depth, with coverage gaps, sample thresholds and fixed caveats. Observational only; never writes, spawns or uses the network, and never prints message text, paths or session IDs.
+
+## 0.4.0
+
+- Upgrade AUTO by default: keep the main model fixed, independently judge child capability and reasoning investment, and enforce exact fresh-child bindings with persisted replay. Preserve explicit legacy rollback and MANUAL behavior.
+- Add TypeSafe, Concentrate and custom System One decision services with separate credentials, preset model menus and explicit public-sample connection testing.
+- Expose executable strategy details, candidate distributions, normalized Choice certainty and actual application status; no task-quality or savings claims.
+- Restore per-reply allocation feedback with immutable evaluation snapshots, request-event correlation, timestamped change/cancel events and legacy-record compatibility.
+- Refresh bilingual user documentation; label earlier GIFs as legacy mock-host demonstrations. Fork and real-provider teammate acceptance remain unsupported/unverified as documented.
+
+
+- Accept the DSH 0.2 host's React 18.3.1 as well as React 19 in the runtime peer declaration; client builds continue to use the host's shared React instance.
+
 ## 0.3.0
 
 - Publish as a standalone DSH bundle with the companion `subagent-model-routing` skill included.

package/README.md

@@ -2,12 +2,39 @@
 
 English · [简体中文](README.zh.md)
 
-![AUTO in action: a refactor-and-debug turn is judged complex and runs at high, a follow-up rename is judged simple and drops to low; hovering the AUTO chip shows the decision, clicking it switches to MANUAL](docs/auto-routing.gif)
+![AUTO interaction: toggle AUTO/MANUAL and inspect the decision](docs/auto-routing.gif)
+
+*Interaction recording from the earlier four-tier UI (isolated mock Host); current decisions use independent capability and depth as described below.*
+
+[Repository](https://github.com/prgrmrwy/dsh-auto-model-routing) · [Releases](https://github.com/prgrmrwy/dsh-auto-model-routing/releases) · [npm](https://www.npmjs.com/package/dsh-auto-model-routing)
 
 <!-- problem -->
-You should not have to raise or lower reasoning effort by hand for every message. In AUTO, each user turn is judged by the Jev decision model (via OpenRouter) as `simple` / `standard` / `complex` / `escalated`, and the plugin changes **only the reasoning effort** of the model you already picked. Provider and model are never changed. If Jev fails for any reason, AUTO switches itself off and the chip shows why.
+**Add AUTO mode to DSH: automatic effort for the main agent, automatic model + effort for sub-agents.** Stop adjusting reasoning investment every turn or choosing a model for every delegated task.
+
+- **Main agent: keep your selected model and adjust reasoning effort per user turn.**
+- **Sub-agents: judge capability needs and reasoning investment independently, then select model + effort.** A stronger model does not automatically mean deeper reasoning.
+- **Visible decisions, local feedback:** inspect candidate distributions and actual application status, then rate the allocation below a reply.
+
+### Which DSH scenarios does AUTO cover?
+
+| Scenario | AUTO behavior | Current boundary |
+|---|---|---|
+| Main-agent user turn | Judge effort for each new user turn; keep the main model | Not rejudged at every tool step; model must support the effort |
+| Official fresh `subagent` | Select and bind child model + effort before its first request | Real Magpie Claude/GPT ordinary child calls verified; catalog and allowlist apply |
+| Official fresh `spawn_teammate` | Bind the new teammate's model + effort using the same policy | Mock-host coverage; real-provider teammate acceptance remains unverified |
+| Resume or continue a bound child | Replay its initial allocation without rejudging | Requires the plugin enabled and intact route records |
+| Explicit model / effort | Preserve the specified dimension; judge only the other | Both specified: validate without calling Jev |
+| Fork, third-party or unbound creation paths | No automatic-allocation guarantee | Fork and unbound AUTO child paths are explicitly rejected, not silently replaced |
+
+This describes official integration paths—not acceptance of every DSH entry point, CLI/TUI interface or third-party plugin. The Web UI is the currently verified interface. MANUAL makes no new judgments. `route_subagent` is a preview, not a required step before dispatch.
 
-**Install.** DSH `0.2.0-rc.2`, Node.js `>=22.19`.
+### Local feedback → future learning and adjustment
+
+**Available now:** rate a decision-bound reply as appropriate, too high, too low or unknown; change or clear the rating. Feedback and decision snapshots stay on your machine. Submitting feedback does not upload it to the decision service or immediately change routing; AUTO task judgments themselves still use your selected external service.
+
+**Planned direction:** add learning, evaluation and policy adjustment based on local feedback and execution outcomes, so AUTO can better fit your tasks. **Automatic learning and feedback-driven tuning are not implemented yet.** Today's records support later evaluation; they are not evidence of measured quality gains or cost savings.
+
+**Install.** Tested with DSH `0.2.0-rc.2`, Web UI, Node.js `>=22.19`.
 
 ```sh
 dsh plugin --profile web add dsh-auto-model-routing
@@ -15,40 +42,65 @@ dsh plugin --profile web add dsh-auto-model-routing
 dsh plugin --profile web add prgrmrwy/dsh-auto-model-routing
 ```
 
-Restart DSH, refresh the Web page, then configure your OpenRouter key in **Settings → 自动模型路由**. The companion skill is included; it guides delegation but does not enforce every dispatch. Unlike model-pool routers or local classifiers, this plugin uses external Jev judgments and never switches the main model. The demos use an isolated Host with local mock Jev/model responses.
-
-In [ohmydsh](https://github.com/prgrmrwy/ohmydsh), enable only the local `auto-model-routing` entry and run `dsh build`; no separate skill customization is needed.
+Restart DSH and refresh the Web page. In **Settings → Auto model routing**, choose a decision service, save its API key, and use **Test connection** with a fixed public sample. These are English descriptions of the controls; the plugin's interface labels are currently in Chinese. New chats default to **MANUAL**; click the chip to use AUTO. The companion delegation skill is included—no separate skill installation is needed.
 
 ## Use
 
 | Where | What you do |
 |---|---|
-| **AUTO / MANUAL chip** (left of the model picker) | Click to switch. Hover to see this turn's effort, the latest Jev decision and its distribution. |
-| **Decision hint** | Appears above the chip for 3 s after each judgment, e.g. `complex → high · 0.70s`. |
-| **Gauge button** on a reply | Hover for the judgment; click to record which tier the turn should have used. |
-| **Settings → 自动模型路由** | OpenRouter key, default mode, effort cap, Jev deadline, hint switch and the routing maps (below). |
+| **AUTO / MANUAL chip**, left of the model picker | Click to switch. Hover for the decision, candidate distribution, certainty, model/effort and application status. |
+| **Decision hint** | Shows new decisions briefly; its Settings switch does not hide the hover panel. |
+| **Gauge button** below an eligible reply | Mark the allocation appropriate, too high, too low or unknown; change or clear your feedback later. Only replies bound to an applied main decision qualify. |
+| **Settings → Auto model routing** | Choose service/model, manage its separate key, test the connection, set defaults/cap/deadline, or expand **View decision strategy** to inspect the actual rubric and model slots. |
+
+### Main model stays fixed; children are selected separately
 
-![Settings → Auto model routing: judging service, session defaults, and the Claude / GPT routing maps with main-chat effort and sub-agent recommendation per tier](docs/settings.gif)
+- **Main chat:** `lower / baseline / higher` moves at most one supported effort step below/at/above the model's verified recommended starting point. The cap defaults to `xhigh`; AUTO never selects `max`. It does not change the main provider/model or write global model defaults.
+- **Fresh children:** capability is `bounded / analytical / frontier`; effort is judged separately. Claude slots are Haiku / Sonnet / Opus 5.5; GPT slots are Luna / Sol / Sol. The current catalog, supported efforts and official child allowlist must all permit the route. Magpie uses its exact provider and full family-prefixed model IDs, not a fallback to local transports.
+- **Explicit choices win:** specifying a model or effort leaves only the other dimension to judge. Specifying both skips Jev and validates the route. Missing task scope stops delegation rather than guessing.
+- **Official creation binds the route:** fresh `subagent` and fresh `spawn_teammate` paths bind an exact child identity before its first request. `route_subagent` is an optional preview—not proof of application and not a user constraint to copy into a subsequent call. Fork and unbound creation paths are unsupported; there is no silent substitution.
+- **Resume keeps the initial route.** Switching the root to MANUAL stops new judgments; already bound children retain their initial route. `applied` means the actual request configuration matched, **not** that the task succeeded. Real Magpie Claude/GPT ordinary calls were tested; fresh teammate routing and cold resume have mock-host coverage, not real-provider teammate acceptance.
 
-**Default effort.** Each model starts at its officially recommended effort: `standard` keeps it, `simple` goes one step down, `complex` one up, `escalated` two up. The result is capped (default `xhigh`), and AUTO never picks `max`. Each cell of the routing map can override this: main efforts apply to every model of a provider family, and sub-agent cells are full model + effort pairs.
+### Choose the decision service
 
-**Sub-agents.** The `route_subagent` tool uses the same judgment and returns provider/model/effort (Claude: Haiku → Sonnet → Opus → Opus; GPT: Luna → Sol → Sol → Sol). It only judges when the root chat is AUTO, and otherwise returns `routed: false`. These routes must be in the official sub-agent allowlist, which is fixed when a chat is created.
+| Service | Endpoint | Default model | Credential reference |
+|---|---|---|---|
+| OpenRouter (default, preserves existing installations) | `https://openrouter.ai/api/alpha/decisions` | `typesafe/jev-1.13` | `OPENROUTER_API_KEY` |
+| TypeSafe | `https://api.typesafe.ai/v1/systemone` | `jev-1.13.0` | `TYPESAFE_API_KEY` |
+| Concentrate | `https://api.concentrate.ai/v1/systemone` | `jev-1.13.0` | `CONCENTRATE_API_KEY` |
+| Custom System One | Your full endpoint | Editable | `AUTO_DECISION_CUSTOM_API_KEY` |
+
+Preset model menus expose documented versions, not live availability guarantees. Custom services keep an editable model ID and must support `state + questions → answers`; an OpenAI chat-completions endpoint alone is not compatible. Custom endpoints require HTTPS (loopback HTTP is allowed), reject URL credentials/query/fragment and redirects. Only configure endpoints you trust with the dedicated key. TypeSafe and Concentrate have contract/mock coverage; their live authentication and regional availability were not verified here. No service failover occurs automatically.
+
+**Certainty is not the selected probability.** Choice certainty is `(pmax − 1/n) / (1 − 1/n)`: with three candidates, a 70% top probability gives 55% certainty. The panel shows the service's returned certainty and full candidate distribution separately, including returned zeroes; missing values are marked unavailable. Neither number is task success rate.
 
 ## Good to know
 
-- **New chats default to MANUAL.** You can change this in Settings.
-- **Fail closed.** No key, out of credits, rate limit, region block, timeout or a bad reply: the chat goes to `MANUAL · Jev 停用` with the reason. Click AUTO to retry. There is no local fallback and no retry.
-- **MANUAL only stops changes.** The chat keeps the last effort AUTO set. AUTO also writes it into the official selection, so new chats start from it.
-- **Data sent out.** In AUTO, every user turn sends up to 8000 characters of your message and short parts of the previous turn to OpenRouter / TypeSafe (`typesafe/jev-1.13`). The text is redacted by shape first (keys, tokens, URL credentials, IPs). This is best effort and does not guarantee that every secret is removed.
-- **The key** is stored through the official DSH credentials service, never in the plugin config. A key in the launch environment (`OPENROUTER_API_KEY`) takes precedence and is read-only. To move it to the GUI, remove it from the launch environment, restart once, then save it in Settings.
-- **Files.** Settings live in `$DSH_HOME/plugins/dsh-auto-model-routing/config.json`; per-chat mode and judgments live in `$DSH_HOME/state/auto-model-routing/`. Changes take effect on the next request; no restart is needed.
+- **Failures are visible.** Main judgment failure disables AUTO with a reason; child judgment/route failure stops dispatch. No local classifier, automatic retry, upgrade or cross-service fallback.
+- **MANUAL stops adjustments, not history.** It does not restore an earlier effort. The official current selection remains in control; existing bound children keep their route while the plugin is enabled.
+- **Data leaves the machine.** AUTO sends shape-redacted task text (up to 8000 characters) and short previous-turn excerpts to your chosen service. Redaction is best effort, not a guarantee that all secrets are removed. Test Connection sends only a fixed public sample and can incur a small charge.
+- **Keys use official DSH credentials**, never plugin configuration. Launch-environment keys take precedence and are read-only. Service changes clear key drafts; save the service before managing its separate key. Changing a custom endpoint does not create a new credential namespace—review the key before trusting a new host.
+- **Local state:** configuration is under `$DSH_HOME/plugins/dsh-auto-model-routing/`; mode, child routes and feedback are under `$DSH_HOME/state/auto-model-routing/`. Feedback stores resource metadata, IDs, versions, returned distributions and timestamps—not task/reply text. Request event IDs allow later usage correlation; costs are not fabricated when unavailable. See [feedback and evaluation](docs/feedback-evaluation.md).
+- **No automatic learning.** Human allocation feedback is observational. It does not change routing or prove quality/cost benefits. Old messages without decision bindings are not backfilled; old records with missing evaluation fields remain unknown.
+- **Upgrade:** existing AUTO sessions use the new policy without a version switch or rewriting their stored mode. New chats still default to MANUAL unless configured otherwise. Historical four-tier mappings remain only for explicit `policy: "legacy"` rollback in plugin configuration; they are not the current policy or feedback menu.
 
 ## Architecture
 <!-- section: architecture -->
 
-![auto-model-routing architecture: the chip and RPC endpoints, the agent/pre-step hook asking Jev through OpenRouter, the agent/request hook rewriting only reasoning effort, and the route_subagent tool sharing the same judgment](docs/architecture.dual.svg)
+![AUTO architecture — independent judgments, validated requests and allocation feedback](docs/architecture-v2.en.svg)
+
+Main failures disable AUTO; child failures stop dispatch. Explicit model/effort constraints are preserved. The diagram shows the AUTO paths; MANUAL makes no new judgments and fork is unsupported. Feedback does not automatically alter routing.
+
+Entry: [index.ts](src/index.ts). Enforcement: [policy-host.ts](src/policy-host.ts). Actual rubric: [policy-v2-rubric.json](src/policy-v2-rubric.json). Model slots: [policy-v2.ts](src/policy-v2.ts). UI: [client](src/client/index.ts). Coverage and limitations: [development verification](docs/policy-v2-development.md), [real Magpie acceptance](docs/magpie-live-acceptance.md).
+
+<details>
+<summary>Earlier four-tier UI demos (legacy rollback only)</summary>
+
+These isolated-host recordings use mock Jev/model responses and show the previous UI, not the current default policy.
 
-An `agent/pre-step` hook judges step 1 of each user turn. An `agent/request` hook (prepended) then rewrites only `reasoningEffort`, and syncs it to the official model selection. Entry point: [`src/index.ts`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/src/index.ts). Rubric: [`src/rubric.json`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/src/rubric.json). Effort table: [`src/effort.ts`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/src/effort.ts). UI: [`src/client/`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/src/client/index.ts). Regression cases: [`eval/`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/eval/README.md). Diagram source: [`docs/architecture.json`](docs/architecture.json) (archify).
+![Legacy four-tier routing settings](docs/settings.gif)
+
+</details>
 
 ## Development
 
@@ -58,11 +110,17 @@ npm run typecheck
 npm run build
 ```
 
-Tests use the real AgentLoop with a fake LLM stream and a loopback fake Jev; they never call OpenRouter. `node scripts/label-report.mjs` summarizes your tier labels per rubric version.
+Tests use the official AgentLoop with fake LLM streams and mock/loopback Jev responses, not paid model calls. `scripts/label-report.mjs` reports **legacy tier labels only**. To observe AUTO decisions and their actual token usage, ask the agent to use the `auto-model-routing-report` skill (on demand only; it runs the read-only `lib/usage-report.js`). It is observational, not a savings measurement. Evaluation schema and deduplication rules are in [feedback-evaluation.md](docs/feedback-evaluation.md).
+
+Development stays in the source monorepo; this repository is its release mirror. The package builds and loads independently using its declared dependencies, bundled skill and official DSH services; no monorepo checkout or monorepo instruction file is required. External design references are optional background, not runtime inputs.
 
-Specs and design: the current spec is [`openspec/specs/auto-model-routing`](https://github.com/prgrmrwy/ohmydsh/blob/main/openspec/specs/auto-model-routing/spec.md). The archived changes record the design history: [routing](https://github.com/prgrmrwy/ohmydsh/blob/main/openspec/changes/archive/2026-10-22-auto-model-routing/design.md) (decisions D1–D14) and [settings](https://github.com/prgrmrwy/ohmydsh/blob/main/openspec/changes/archive/2026-10-23-auto-model-routing-settings/design.md). The companion skill is [`skills/subagent-model-routing`](skills/subagent-model-routing/SKILL.md). Before you change the hooks, read [DSH plugin integration pitfalls](https://github.com/prgrmrwy/ohmydsh/blob/main/docs/architecture/dsh-plugin-integration-pitfalls.md).
+The [current specification](https://github.com/prgrmrwy/ohmydsh/blob/main/openspec/specs/auto-model-routing/spec.md) lives in the source monorepo. The bundled [delegation skill](skills/subagent-model-routing/SKILL.md) supplies guidance; enforcement belongs to the official host hooks, not the prompt. No measured task-quality or token-savings claim is made.
 
 ## Removal
 <!-- section: removal -->
 
-Run `dsh plugin --profile web remove dsh-auto-model-routing`, then restart DSH and refresh the page. In ohmydsh, set `enabled: false` and rebuild instead. The plugin-owned skill and tools disappear; settings and logs remain. Chats keep the last effort AUTO set until you pick another one.
+```sh
+dsh plugin --profile web remove dsh-auto-model-routing
+```
+
+Restart DSH and refresh the page. The plugin's skill/tools and request hooks disappear; settings and logs remain. Without the plugin, child replay is no longer enforced and host restoration uses the original creation configuration. In [ohmydsh](https://github.com/prgrmrwy/ohmydsh), disable the local entry and rebuild instead.

package/README.zh.md

@@ -2,12 +2,39 @@
 
 [English](README.md) · 简体中文
 
-![AUTO 实际效果：「重构并调试」被判为 complex，用 high；随后的「改个名」被判为 simple，降到 low；悬停 AUTO 芯片查看判档，点击切到 MANUAL](docs/auto-routing.gif)
+![AUTO 交互：切换 AUTO/MANUAL 并查看决策](docs/auto-routing.gif)
+
+*此交互录屏来自早期四档界面（隔离 mock 宿主）；当前策略按下文独立判断能力与投入。*
+
+[插件仓库](https://github.com/prgrmrwy/dsh-auto-model-routing) · [版本发布](https://github.com/prgrmrwy/dsh-auto-model-routing/releases) · [npm](https://www.npmjs.com/package/dsh-auto-model-routing)
 
 <!-- problem -->
-不必每条消息都手动调高或调低推理强度。AUTO 下，每个用户回合由 Jev 决策模型（经 OpenRouter）判为 `simple` / `standard` / `complex` / `escalated`，插件**只调整你已选模型的推理强度**，不换 provider 和模型。Jev 任何失败都会让 AUTO 自动停用，芯片上显示原因。
+**为 DSH 增加 AUTO 模式：主 Agent 自动调 effort，子 Agent 自动选 model + effort。** 不必每轮手动选择推理投入，也不必为每个子任务手动挑模型。
+
+- **主 Agent：保留你选择的模型，按回合调整推理投入（effort）。**
+- **子 Agent：按任务独立判断能力需求与推理投入，选择模型与 effort。** 不是把“更强模型”和“更深推理”捆成一个档位。
+- **看得见、能反馈：** 展示决策分布与实际生效状态，在回复下方提交本机分配反馈。
+
+### AUTO 覆盖哪些 DSH 场景？
+
+| 场景 | AUTO 的作用 | 当前边界 |
+|---|---|---|
+| 主 Agent 用户回合 | 每个新用户回合判断 effort，主模型不换 | 不逐工具步骤重判；模型必须支持相应 effort |
+| 官方 fresh `subagent` | 首次请求前选择并绑定子 model + effort | 真实 Magpie Claude/GPT 普通子调用已验证；受目录和允许列表约束 |
+| 官方 fresh `spawn_teammate` | 按同一策略绑定新 teammate 的 model + effort | 已有 mock 宿主覆盖，尚未完成真实模型 teammate 验收 |
+| 已绑定子实例恢复或继续执行 | 重用初始路线，不重新分配 | 依赖插件保持启用及路由记录完整 |
+| 显式指定 model / effort | 保留已指定维度，仅判断未指定维度 | 两者都指定则只校验，不调用 Jev |
+| fork、第三方或未绑定创建路径 | 不承诺自动选配 | fork 与未绑定 AUTO 子路径会明确拒绝，不静默替代 |
+
+这里是官方接入路径的覆盖说明，不代表所有 DSH 入口、CLI/TUI 界面或第三方插件都已验收。Web UI 是当前验证入口；MANUAL 不做新判断。`route_subagent` 仅预览，不是必须先调用的派发步骤。
 
-**安装**：DSH `0.2.0-rc.2`，Node.js `>=22.19`。
+### 本机反馈 → 未来的学习调整
+
+**现在：** 在已绑定决策的回复下评价“合适 / 投入偏高 / 投入不足 / 无法判断”，可修改、撤销。反馈与决策快照保存在本机，不会因点击反馈而上传到决策服务，也不会立即改变路由；AUTO 任务判断本身仍按所选服务外发。
+
+**未来方向：** 增加基于本机反馈与执行结果的学习、评估和策略调整，让 AUTO 更贴合你的任务。**尚未实现自动学习或反馈驱动调参**，目前记录用于后续评估，不承诺已证明的质量提升或成本节省。
+
+**安装**：已验证 DSH `0.2.0-rc.2`、Web UI，Node.js `>=22.19`。
 
 ```sh
 dsh plugin --profile web add dsh-auto-model-routing
@@ -15,40 +42,65 @@ dsh plugin --profile web add dsh-auto-model-routing
 dsh plugin --profile web add prgrmrwy/dsh-auto-model-routing
 ```
 
-重启 DSH、刷新页面，再到**设置 → 自动模型路由**配置 OpenRouter key。配套 skill 已包含，提供委派指引，但不强制拦截每次派发。与模型池路由或本地分类器不同，本插件用外部 Jev 判档，始终不换主模型。演示来自隔离 Host，使用本地模拟的 Jev 和模型响应。
-
-在 [ohmydsh](https://github.com/prgrmrwy/ohmydsh) 中只需启用 local `auto-model-routing` 条目并运行 `dsh build`，不再单独启用 skill 条目。
+重启 DSH 并刷新页面，到**设置 → 自动模型路由**选择决策服务、保存对应 API key，点击**测试连接**发送固定公开样例。新会话缺省 **MANUAL**，点击芯片切换 AUTO。配套委派 skill 已包含，无需单独安装。
 
 ## 使用
 
 | 位置 | 操作 |
 |---|---|
-| **AUTO / MANUAL 芯片**（模型选择器左侧） | 点击切换；悬停查看本回合 effort、最近一次 Jev 决策及其分布 |
-| **判档提示** | 每次判档后在芯片上方显示 3 秒，例如 `complex → high · 0.70s` |
-| 回复下方的**仪表按钮** | 悬停看判档，点击标注「这轮应该用哪档」 |
-| **设置 → 自动模型路由** | OpenRouter key、默认模式、强度上限、判档等待上限、提示开关、路由映射（见下图） |
+| **AUTO / MANUAL 芯片**（模型选择器左侧） | 点击切换；悬停查看判断、候选分布、确定度、模型与投入、生效状态 |
+| **判档提示** | 短暂显示新决策；关闭设置中的提示开关不会隐藏悬浮面板 |
+| 回复下方的**仪表按钮** | 评价分配合适、投入偏高、投入不足或无法判断，支持修改和撤销；仅对绑定已应用主决策的回复显示 |
+| **设置 → 自动模型路由** | 选择服务与模型、管理独立 key、测试连接、设置模式/强度上限/等待时间；展开**查看决策策略**查看实际判据与模型槽 |
+
+### 主模型不换，子任务独立选配
 
-![设置 → 自动模型路由：判档服务、会话默认值，以及 Claude / GPT 分页下每档的主会话推理等级与子会话推荐](docs/settings.gif)
+- **主会话**：`lower / baseline / higher` 相对于具体模型已验证的推荐起点，在受支持序列降低一档、保持或提高一档。缺省上限 `xhigh`，自动不选 `max`；不修改主 provider/model，也不改写全局模型默认值。
+- **fresh 子任务**：能力需求为 `bounded / analytical / frontier`，推理投入独立判断。Claude 能力槽为 Haiku / Sonnet / Opus 5.5；GPT 为 Luna / Sol / Sol。实际路由必须同时满足当前目录、支持的 effort 和官方子智能体允许列表。Magpie 保留真实 provider 与家族前缀完整模型 ID，不回退到本地入口。
+- **尊重显式选择**：只指定模型或投入时，仅判断另一维；两者都指定则不请求 Jev，只校验路由。缺少任务目标或范围时停止委派，不猜。
+- **官方创建绑定路由**：fresh `subagent` 和 fresh `spawn_teammate` 在子任务首次请求前绑定精确身份。`route_subagent` 只是可选预览，不代表已应用，也不能把推荐复制成用户约束。fork 和未绑定创建路径暂不支持，不静默替换。
+- **恢复保持初始路线**：根会话改为 MANUAL 后停止新判断，已有绑定子实例维持原路线。`applied` 只证明实际请求配置匹配，**不代表任务成功**。真实 Magpie Claude/GPT ordinary 调用已验证；fresh teammate 路由及冷恢复有 mock 宿主覆盖，不等于真实模型 teammate 已验收。
 
-**缺省强度**：从各模型的官方推荐起点出发。standard 取起点，simple 降一级，complex 升一级，escalated 升两级；受上限约束（缺省 `xhigh`），AUTO 永不选 `max`。路由映射可逐格覆盖：主会话等级作用于同体系所有模型，子会话格为完整的 model + effort。
+### 选择决策服务
 
-**子会话**：`route_subagent` 工具用同一套判档，返回 provider/model/effort（Claude：Haiku → Sonnet → Opus → Opus；GPT：Luna → Sol → Sol → Sol）。只有根会话为 AUTO 时才判档，否则返回 `routed: false`。这些路由须在官方子智能体允许列表中，该列表在会话创建时固定。
+| 服务 | 接口 | 缺省模型 | 凭据引用 |
+|---|---|---|---|
+| OpenRouter（默认，兼容已有安装） | `https://openrouter.ai/api/alpha/decisions` | `typesafe/jev-1.13` | `OPENROUTER_API_KEY` |
+| TypeSafe 官方 | `https://api.typesafe.ai/v1/systemone` | `jev-1.13.0` | `TYPESAFE_API_KEY` |
+| Concentrate | `https://api.concentrate.ai/v1/systemone` | `jev-1.13.0` | `CONCENTRATE_API_KEY` |
+| 自定义 System One | 完整接口地址 | 可编辑 | `AUTO_DECISION_CUSTOM_API_KEY` |
+
+预置模型菜单列出文档确认的版本，不保证实时可用。自定义服务可填写自己的模型 ID，但必须兼容 `state + questions → answers`；只有 OpenAI chat-completions 接口并不兼容。自定义端点限 HTTPS（回环可 HTTP），禁止 URL 凭据、查询参数、片段及重定向。只配置你信任且适合使用该专用 key 的地址。TypeSafe、Concentrate 目前有协议/mock 覆盖，未在本次验证其真实凭据与地区可用性。失败不自动切换服务。
+
+**判定确定度不等于选中概率**：Choice 公式为 `(pmax − 1/n) / (1 − 1/n)`，三个候选中最高概率 70% 对应确定度 55%。面板分别显示服务返回的确定度和完整分布，保留返回的 0%，缺值标未提供。两者都不是任务成功率。
 
 ## 须知
 
-- **新会话缺省 MANUAL**，可在设置中改。
-- **失败即关闭**：无 key、余额不足、限流、地区受限、超时或响应无效时，会话切到 `MANUAL · Jev 停用` 并显示原因，点 AUTO 重试。没有本地降级，也不重试。
-- **MANUAL 只停止调整**：会话保持 AUTO 最后设定的 effort。AUTO 也会把它写入官方模型选择，所以新会话会从这个值开始。
-- **外发数据**：AUTO 下每个用户回合会把你的消息（最多 8000 字符）和上一轮的少量片段发给 OpenRouter / TypeSafe（`typesafe/jev-1.13`）。外发前先按形态脱敏（密钥、token、URL 凭据、IP），这是尽力而为，不保证去掉所有敏感信息。
-- **key** 经官方 DSH 凭据服务保存，不写进插件配置。启动环境里的 `OPENROUTER_API_KEY` 优先且只读；要迁到 GUI，先从启动环境删掉它，重启一次，再在设置页保存。
-- **文件**：设置在 `$DSH_HOME/plugins/dsh-auto-model-routing/config.json`；每个会话的模式与判档记录在 `$DSH_HOME/state/auto-model-routing/`。改动在下一次请求生效，无需重启。
+- **失败可见**：主判断失败会停用 AUTO 并显示原因，子判断或路由失败则停止派发。没有本地分类器、自动重试、自动升级或跨服务降级。
+- **MANUAL 停止调整，不复原历史值**：使用官方当前选择；只要插件启用，已绑定子任务仍保留原路线。
+- **任务会外发**：AUTO 将按形态脱敏的任务文本（最多 8000 字符）及上一轮少量片段发送给所选服务。脱敏尽力而为，不保证去除全部秘密。测试连接只发送固定公开样例，可能产生少量费用。
+- **key 经官方 DSH 凭据服务保存**，不写入插件配置。启动环境 key 优先且只读；切换服务清空 key 草稿，先保存服务再管理新 key。修改自定义端点不会产生新凭据空间，换地址前请确认专用 key 的信任边界。
+- **本地状态**：配置在 `$DSH_HOME/plugins/dsh-auto-model-routing/`，模式、子路由及反馈在 `$DSH_HOME/state/auto-model-routing/`。反馈只保存资源元数据、身份、版本、返回分布及时间，不保存任务或回复正文。请求事件序号供后续关联用量，缺失成本不会编造。见[反馈评估说明](docs/feedback-evaluation.md)。
+- **不自动学习**：人工反馈仅作观察，不改变路由，也不能直接证明质量提升或节省。无决策绑定的旧消息不回填，旧记录缺少的评估字段保持未知。
+- **升级兼容**：已有 AUTO 会话直接使用新策略，无需版本开关，也不重写会话模式。新会话缺省仍为 MANUAL，除非你改过设置。旧四档映射仅供配置中显式 `policy: "legacy"` 回滚，不是当前策略或反馈菜单。
 
 ## 架构
 <!-- section: architecture -->
 
-![auto-model-routing 架构图：芯片与 RPC 端点、agent/pre-step 钩子经 OpenRouter 请求 Jev、agent/request 钩子只改推理强度、route_subagent 工具共用同一判断](docs/architecture.dual.svg)
+![AUTO 架构：独立判断、请求校验与分配反馈](docs/architecture-v2.zh.svg)
+
+主判断失败停用 AUTO，子失败停止派发；保留显式 model/effort。图示为 AUTO 路径，MANUAL 不做新判断，fork 暂不支持。反馈不会自动改变路由。
+
+入口：[index.ts](src/index.ts)。执行接入：[policy-host.ts](src/policy-host.ts)。实际判据：[policy-v2-rubric.json](src/policy-v2-rubric.json)。模型槽：[policy-v2.ts](src/policy-v2.ts)。界面：[client](src/client/index.ts)。覆盖边界：[开发验证](docs/policy-v2-development.md)、[真实 Magpie 验收](docs/magpie-live-acceptance.md)。
+
+<details>
+<summary>旧四档界面演示（仅供 legacy 回滚参考）</summary>
+
+下列隔离宿主录屏使用模拟 Jev/模型响应，展示旧界面，不代表当前默认策略。
 
-`agent/pre-step` 钩子在每个用户回合的第 1 步判档；随后以 prepend 注册的 `agent/request` 钩子只改写 `reasoningEffort`，并同步到官方模型选择。入口 [`src/index.ts`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/src/index.ts)，判档规则 [`src/rubric.json`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/src/rubric.json)，强度表 [`src/effort.ts`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/src/effort.ts)，界面 [`src/client/`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/src/client/index.ts)，回归用例 [`eval/`](https://github.com/prgrmrwy/dsh-auto-model-routing/blob/main/eval/README.zh.md)，图源 [`docs/architecture.json`](docs/architecture.json)（archify）。
+![旧版四档路由设置](docs/settings.gif)
+
+</details>
 
 ## 开发验证
 
@@ -58,11 +110,17 @@ npm run typecheck
 npm run build
 ```
 
-测试使用真实 AgentLoop、假 LLM 流和本地回环假 Jev，不访问 OpenRouter。`node scripts/label-report.mjs` 按规则版本汇总你的档位标注。
+测试使用官方 AgentLoop、假 LLM 流和 mock/回环 Jev，不产生真实模型费用。`scripts/label-report.mjs` **仅汇总旧四档标注**。如需观察 AUTO 决策及实际 token 用量，请明确让 agent 使用 `auto-model-routing-report` skill（仅按需调用，运行只读的 `lib/usage-report.js`）；它是观察统计，不衡量节省；新数据结构与去重口径见[feedback-evaluation.md](docs/feedback-evaluation.md)。
+
+继续在主仓开发，本仓库仅作发布镜像。插件使用自身声明的依赖、包内 skill 和官方 DSH 服务即可独立构建与加载，不要求检出主仓，也不要求安装主仓环境指令。外部设计链接仅供背景阅读，不是运行输入。
 
-规范与设计：当前规范见 [`openspec/specs/auto-model-routing`](https://github.com/prgrmrwy/ohmydsh/blob/main/openspec/specs/auto-model-routing/spec.md)。归档 change 记录了设计演进：[路由](https://github.com/prgrmrwy/ohmydsh/blob/main/openspec/changes/archive/2026-10-22-auto-model-routing/design.md)（决策 D1–D14）与[设置](https://github.com/prgrmrwy/ohmydsh/blob/main/openspec/changes/archive/2026-10-23-auto-model-routing-settings/design.md)。配套 skill 见 [`skills/subagent-model-routing`](skills/subagent-model-routing/SKILL.md)。修改钩子前先读 [DSH 插件集成陷阱](https://github.com/prgrmrwy/ohmydsh/blob/main/docs/architecture/dsh-plugin-integration-pitfalls.md)。
+[当前规范](https://github.com/prgrmrwy/ohmydsh/blob/main/openspec/specs/auto-model-routing/spec.md)在源码 monorepo 中维护。包内[委派 skill](skills/subagent-model-routing/SKILL.md)提供指引，执行保障来自官方宿主钩子，而不是提示词。暂无完整任务质量或 token 节省的量化结论。
 
 ## 移除
 <!-- section: removal -->
 
-运行 `dsh plugin --profile web remove dsh-auto-model-routing`，重启 DSH 并刷新页面；ohmydsh 则设为 `enabled: false` 并构建。插件提供的 skill 与工具一并撤销，设置和日志保留。会话保持 AUTO 最后设定的 effort，直到在模型选择器里另选。
+```sh
+dsh plugin --profile web remove dsh-auto-model-routing
+```
+
+重启 DSH 并刷新页面。插件的 skill、工具和请求钩子撤销，配置与日志保留。关闭插件后不再执行子路由恢复，宿主使用原始创建配置。在 [ohmydsh](https://github.com/prgrmrwy/ohmydsh) 中则禁用 local 条目并重新构建。

package/docs/architecture-v2.en.json

@@ -0,0 +1,26 @@
+{
+ "schema_version":1,"diagram_type":"architecture",
+ "meta":{"title":"AUTO resource allocation","output":".archify/architecture-auto-routing-20261026-014646/auto-routing.html","locale":"en","quality_profile":"showcase","repository":{"url":"git@github.com:prgrmrwy/ohmydsh.git","revision":"79f9eab467047e054a2d66f8bce060834a261a2a","link_mode":"local-only"}},
+ "components":[
+ {"id":"turn","type":"frontend","label":"AUTO user turn","sublabel":"Main model stays fixed","pos":[40,80],"size":[190,68],"sources":[{"path":"packages/auto-model-routing/src/policy-host.ts","line":68,"end_line":88}]},
+ {"id":"judge","type":"cloud","label":"Selected Jev service","sublabel":"Depth / capability independently","pos":[360,80],"size":[220,68],"sources":[{"path":"packages/auto-model-routing/src/decision-service.ts","line":1,"end_line":23},{"path":"packages/auto-model-routing/src/policy-host.ts","line":81,"end_line":88}]},
+ {"id":"route","type":"backend","label":"Validated allocation","sublabel":"Catalog / allowlist / effort cap","pos":[710,80],"size":[220,68],"sources":[{"path":"packages/auto-model-routing/src/policy-host.ts","line":122,"end_line":150}]},
+ {"id":"child","type":"backend","label":"Fresh child binding","sublabel":"Exact child identity; explicit wins","pos":[360,270],"size":[220,68],"sources":[{"path":"packages/auto-model-routing/src/policy-host.ts","line":94,"end_line":132}]},
+ {"id":"request","type":"backend","label":"Official model request","sublabel":"Main effort / child full route","pos":[710,270],"size":[220,68],"sources":[{"path":"packages/auto-model-routing/src/policy-host.ts","line":138,"end_line":160}]},
+ {"id":"persist","type":"database","label":"Persisted child route","sublabel":"Resume keeps initial allocation","pos":[40,270],"size":[220,68],"sources":[{"path":"packages/auto-model-routing/src/policy-host.ts","line":132,"end_line":160}]},
+ {"id":"evidence","type":"backend","label":"Request match evidence","sublabel":"Applied only when config matches","pos":[710,460],"size":[220,68],"sources":[{"path":"packages/auto-model-routing/src/policy-host.ts","line":178,"end_line":201}]},
+ {"id":"feedback","type":"frontend","label":"Decision and reply feedback","sublabel":"Observe allocation; never auto-train","pos":[360,460],"size":[220,68],"sources":[{"path":"packages/auto-model-routing/src/policy-host.ts","line":162,"end_line":174}]},
+ {"id":"snapshot","type":"database","label":"Bound decision snapshot","sublabel":"Metadata / IDs; no task text","pos":[40,460],"size":[220,68],"sources":[{"path":"packages/auto-model-routing/src/policy-host.ts","line":167,"end_line":172}]}
+ ],
+ "connections":[
+ {"id":"main-judge","from":"turn","to":"judge","label":"depth"},
+ {"id":"judge-route","from":"judge","to":"route","label":"decision"},
+ {"id":"child-judge","from":"child","to":"judge","label":"task","fromSide":"top","toSide":"bottom"},
+ {"id":"child-store","from":"child","to":"persist","label":"bind"},
+ {"id":"route-request","from":"route","to":"request","label":"apply"},
+ {"id":"request-match","from":"request","to":"evidence","label":"header"},
+ {"id":"match-feedback","from":"evidence","to":"feedback","label":"matched"},
+ {"id":"feedback-snapshot","from":"feedback","to":"snapshot","label":"bind reply"}
+ ],
+ "cards":[{"dot":"cyan","title":"Decision services","items":["OpenRouter / TypeSafe / Concentrate / custom System One","No automatic service fallback or retry"]},{"dot":"amber","title":"Boundaries","items":["MANUAL: no new judgments; explicit constraints preserved","Fork unsupported; applied is not task success"]}]
+}