dsh-agentic-router 1.0.0 → 1.4.0

package/README.en.md

@@ -4,14 +4,16 @@
 
 DeepSeek Harness (DSH) plugin: a **learning agentic router with a data flywheel**.
 
-It watches every model request, routes it to the most appropriate model tier by task type and complexity, asks four experts in parallel and lets an EXP3 meta-selector decide who to trust. Every turn is scored with **quality-proxy signals** (tool failures, model retries, latency) and the reward feeds back into the strategies — the router gets better as it runs. Every decision and reward is persisted locally: auditable and replayable.
+At the start of every turn (`agent/inbox/claimed`) it routes the request to the most appropriate model tier by task type and complexity, asks four experts in parallel and lets an EXP3 meta-selector decide who to trust. Every turn is scored with **quality-proxy signals** (tool failures, model retries, cost, latency) and the reward feeds back into the strategies — the router gets better as it runs. Every decision and reward is persisted locally: auditable and replayable.
+
+**Real switching since v1.4.0**: v1.3.0 and earlier swapped models by returning a replaced config from the `agent/request` waterfall, but DSH's built-in model-selection listener overrides that value, so the swap never took effect. v1.4.0 instead appends a `request/header` event (reason: `router`) to the session log at turn start, going through DSH's official model-selection precedence chain (in-process selection → latest logged session `request/header` → default model).
 
 ## Architecture
 
 ```
-user input → classify (rules, <1ms) → 4 expert recommendations → EXP3 meta-pick → agent/request swap
-                                           ↑                              │
-              UCB/LinUCB/k-NN/EXP3 updates ← quality-proxy reward ← turn end (tool errors/retries/latency)
+user input → classify (rules, <1ms) → 4 expert recommendations → EXP3 meta-pick → request/header append (real switch)
+                                            ↑                              │
+              UCB/LinUCB/k-NN/EXP3 updates ← quality-proxy reward ← turn end (tool errors/retries/cost/latency)
 ```
 
 | Expert | Algorithm | Status |
@@ -30,7 +32,26 @@ Design rationale and roadmap: [docs/DESIGN.md](./docs/DESIGN.md). The implementa
 dsh plugin --profile web add dsh-agentic-router
 ```
 
-> `--profile` is mandatory. Restart the session (or profile) after installing so the tool schemas join prompt assembly. Default mode is **shadow** (records and learns, never swaps); switch to active once the policy behaves.
+> `--profile` is mandatory. Restart the web profile after installing so the tool schemas join prompt assembly.
+
+### Switch to real routing (active)
+
+Default mode is **shadow** (records and learns, never swaps). To enable real switching, the easiest way is: after restarting, tell the agent in any DSH session:
+
+> Switch the agentic router to active mode
+
+The agent calls `agentic_router_set_mode` (mode=active) and persists it to
+`~/.dsh/storages/dsh-agentic-router/policy.json` — it stays active across restarts.
+
+Alternatively, start active on first load by adding config in the profile's `cordis.patch.yml`:
+
+```yaml
+- id: agentic-router
+  config:
+    mode: active
+```
+
+(Pick one; once a mode has been saved via the tool, `policy.json` takes precedence over `config.mode`.)
 
 ## Tools
 
@@ -40,36 +61,45 @@ dsh plugin --profile web add dsh-agentic-router
 | `agentic_router_set_mode` | shadow (default) / active (real swaps) / off (bypass) |
 | `agentic_router_force` | force a model id (active mode only); clear=true to release |
 | `agentic_router_reset` | wipe flywheel learning state, keep mode |
+| `agentic_router_set_prices` | set/query per-provider per-model (or per-tier) prices |
 
 ## Data flywheel (persistence)
 
 Directory: `${DSH_HOME:-~/.dsh}/storages/dsh-agentic-router/` (override with `AGENTIC_ROUTER_HOME`)
 
 - `decisions.jsonl` — one line per decision: task type/complexity, all four recommendations, meta pick, actual route, swapped?
-- `rewards.jsonl` — one line per reward with breakdown (tool errors/retries/latency) + feature vector; **replayed on startup to restore learning state**
+- `rewards.jsonl` — one line per reward with breakdown (tool errors/retries/latency) + feature vector; **replayed on startup to restore learning state**; human-feedback settlements (`feedback-settle` lines) correct the replayed rewards exactly
+- Human feedback: 45s after a turn closes, the plugin polls `messageFeedback` (the Web UI 👍/👎); a thumbs-down costs −0.5, a thumbs-up adds +0.1, applied as an exact correction to the already-applied reward
 - `policy.json` — mode and forced model (atomic write)
 
-Reward: clean close starts at 1; tool failures −0.2 each (cap 0.6), model retries −0.2 each (cap 0.4), latency >30s −0.1 / >90s −0.3; errored turns score 0.
-
-## Known limits
+Reward: clean close starts at 1; tool failures −0.2 each (cap 0.6), model retries −0.2 each (cap 0.4),
+**cost** min(0.5, 20 × turn cost in CNY), latency >30s −0.1 / >90s −0.3; errored turns score 0.
 
-- Model tiers are recognized by id keywords (flash→fast, pro→strong); real swaps happen only when more than one tier exists
-- Rewards are proxies: a mediocre-but-successful answer still scores full marks, so wire up UI feedback (messageFeedback) for production use
-- LinUCB / k-NN need dozens of turns before they graduate into the meta pool
+**Cost signal (token billing)**: the plugin wraps `llm/stream` and accumulates each model
+call's `usage` chunk (input/output/cache-read/reasoning tokens); providers without usage
+fall back to length-based estimates (flagged `estimated`).
 
-## Development
+**Three-level price resolution** (CNY per million tokens): `provider→model` exact price >
+`provider→tier` price > `*` generic tier > conservative fallback. **Unknown providers
+never inherit DeepSeek pricing** — they use the generic tier and carry a `pricingSource`
+audit field. DeepSeek's official pricing ships as default; add other models at runtime via
+the `agentic_router_set_prices` tool (persisted to policy.json) or at install via
+`config.prices`:
 
-```sh
-npm test        # node --test, zero dependencies
+```js
+{ 'deepseek-official': { 'deepseek-v4-pro': { in: 1, out: 12 } },
+  openai: { 'gpt-x-pro': { in: 15, out: 60 } },
+  '*': { fast: { in: 1, out: 4 }, mid: { in: 2, out: 8 }, strong: { in: 3, out: 16 } } }
 ```
 
-## Publish
+## Known limits
+
+- Model tiers are recognized by id keywords (flash→fast, pro→strong); real swaps happen only when more than one tier exists
+- Feedback settles after a 45s window (configurable via `settleDelayMs`); unsettled pending entries are lost on restart; late feedback may be attributed to a later turn
+- Feedback only works where the deployment's Web UI actually writes it (this plugin only reads)
+- LinUCB / k-NN need dozens of turns before they graduate into the meta pool
 
-```sh
-npm publish                                     # requires npm account
-git push origin main                            # after making the GitHub repo public
-# directory listing: add the dsh-plugin topic, or submit at dshmarketplace.dev/submit
-```
+**Development, testing and release flow (maintainers): see [docs/MAINTAINING.md](./docs/MAINTAINING.md).**
 
 ## License
 

package/README.md

@@ -4,14 +4,22 @@
 
 DeepSeek Harness（DSH）插件：**学习型智能路由（agentic router）+ 数据飞轮**。
 
-监听每次模型请求，按任务类型与复杂度把请求路由到最合适的模型档位；四个专家并行推荐、EXP3 元选择器决定听谁的；每回合按**质量代理信号**（工具失败、模型重试、延迟）计算奖励并回传，路由策略随使用越学越准。全部决策与奖励落盘，可审计、可重放。
+每回合开始时（`agent/inbox/claimed`）按任务类型与复杂度把请求路由到最合适的模型档位；
+四个专家并行推荐、EXP3 元选择器决定听谁的；每回合按**质量代理信号**
+（工具失败、模型重试、成本、延迟）计算奖励并回传，路由策略随使用越学越准。
+全部决策与奖励落盘，可审计、可重放。
+
+**v1.4.0 起切换真实生效**：v1.3.0 及之前通过 `agent/request` 返回替换后的配置切模型，
+但该返回值会被 DSH 内置的模型选择监听器强制改回，实际不生效；v1.4.0 改为在回合开始
+向会话日志追加 `request/header` 事件（reason: `router`），走 DSH 官方模型选择优先级链
+（本进程选择 → 会话日志最新 header → 默认模型）真实切换。
 
 ## 架构
 
 ```
-用户输入 → 分类(规则,<1ms) → 四专家推荐 → EXP3 元选择 → agent/request 切换模型
+用户输入 → 分类(规则,<1ms) → 四专家推荐 → EXP3 元选择 → request/header 追加(真实切换)
                                         ↑                        │
-                 UCB/LinUCB/k-NN/EXP3 更新 ← 质量代理奖励 ← 回合收尾(工具失败/重试/延迟)
+                 UCB/LinUCB/k-NN/EXP3 更新 ← 质量代理奖励 ← 回合收尾(工具失败/重试/成本/延迟)
 ```
 
 | 专家 | 算法 | 状态 |
@@ -22,7 +30,7 @@ DeepSeek Harness（DSH）插件：**学习型智能路由（agentic router）+
 | k-NN | 经验池最近邻（k=5）聚合 | 影子，30 经验毕业 |
 | EXP3 | 元选择器，重要性加权更新 | 常驻，冷启动偏 rule（权重 2.5） |
 
-设计细节与路线图见 [docs/DESIGN.md](./docs/DESIGN.md)。本实现源自真实运行时上的动态原型验证（8 个实测缺陷已修复并固化为回归测试）。
+设计细节与路线图见 [docs/DESIGN.md](./docs/DESIGN.md)。
 
 ## 安装
 
@@ -30,7 +38,43 @@ DeepSeek Harness（DSH）插件：**学习型智能路由（agentic router）+
 dsh plugin --profile web add dsh-agentic-router
 ```
 
-> `--profile` 必填。安装后重启会话（或 profile），工具 schema 才会进入 prompt 组装。默认 **shadow 模式**（只记录与学习，不切换模型），确认策略表现后再切 active。
+> `--profile` 必填。安装后重启 Web，工具 schema 才会进入 prompt 组装。
+
+### 切换到真实切换（active）
+
+默认是 **shadow 模式**（只记录与学习，不切换模型）。切到真实切换，最省事的方式：重启后在任意 DSH 会话里对 agent 说：
+
+> 把 agentic router 切成 active 模式
+
+agent 会调用 `agentic_router_set_mode`（mode=active），并把模式持久化到
+`~/.dsh/storages/dsh-agentic-router/policy.json`——**重启后依然 active**，无需每次设置。
+
+也可以不走会话、首次启动即 active：在 profile 的 `cordis.patch.yml` 里给
+`agentic-router` 条目加配置：
+
+```yaml
+- id: agentic-router
+  config:
+    mode: active
+```
+
+（两种方式选其一；一旦用工具存过模式，`policy.json` 的取值优先于 `config.mode`。）
+
+## 切换机制（v1.4.0）
+
+DSH 的 agent 每步请求最终由模型选择逻辑决定，读取优先级为：
+
+```
+本进程选择 → 会话日志最新 request/header → 默认模型
+```
+
+- 直接改 `agent/request` waterfall 的返回值：会被 `dsh-agent` 自带监听器强制改回「选定模型」——无效（v1.3.0 及之前的缺陷）
+- 改 `llm/stream`：agent-loop 请求到达时已深度冻结，无法改写
+
+v1.4.0 的正确做法：在 `agent/inbox/claimed`（回合开始、prompt 组装之前）向会话日志追加
+`request/header` 事件，下一轮组装自然读到新模型。`agent/request` 只做只读观测
+（记录每步真实出网模型）；插件中途热加载时由 `llm/stream` 在循环自身 header 落盘后补一次应用。
+切到 fast/mid 档会去掉 `reasoningEffort`，让目标模型走自身默认 effort。
 
 ## 工具
 
@@ -40,16 +84,33 @@ dsh plugin --profile web add dsh-agentic-router
 | `agentic_router_set_mode` | 切换 shadow（默认）/ active（真切换）/ off（旁路） |
 | `agentic_router_force` | 强制指定模型 id（active 生效），clear=true 清除 |
 | `agentic_router_reset` | 清空飞轮学习状态，保留模式 |
+| `agentic_router_set_prices` | 配置/查询某 provider 某模型（或档位）的价格 |
 
 ## 数据飞轮（落盘）
 
 目录：`${DSH_HOME:-~/.dsh}/storages/dsh-agentic-router/`（`AGENTIC_ROUTER_HOME` 可覆盖）
 
 - `decisions.jsonl` — 每条决策：任务类型/复杂度/四专家推荐/元层选择/实际路由/是否切换
-- `rewards.jsonl` — 每条奖励：明细（工具失败/重试/延迟）+ 特征向量，**重启后重放恢复学习状态**
+- `rewards.jsonl` — 每条奖励：明细（工具失败/重试/延迟）+ 特征向量，**重启后重放恢复学习状态**；人类反馈结算（`feedback-settle` 行）在重放时精确修正
+- 人类反馈：回合结束 45s 后查询 `messageFeedback`（Web UI 的 👍/👎），点踩 −0.5、点赞 +0.1，修正已应用的奖励（UCB 均值/EXP3 权重精确回退）
 - `policy.json` — 模式与强制模型（原子写）
 
-奖励公式：干净收尾 1 分起，工具失败 −0.2/次（封顶 0.6）、模型重试 −0.2/次（封顶 0.4）、延迟 >30s −0.1 / >90s −0.3；出错回合记 0。
+奖励公式：干净收尾 1 分起，工具失败 −0.2/次（封顶 0.6）、模型重试 −0.2/次（封顶 0.4）、
+**成本** min(0.5, 20×回合成本元)、延迟 >30s −0.1 / >90s −0.3；出错回合记 0。
+
+**成本信号（token 计费）**：透传 `llm/stream`，累加每个模型调用的 `usage` chunk
+（输入/输出/缓存读/推理 token）；无 usage 的 provider 按文本长度估算（标 `estimated`）。
+
+**价格表三级解析**（元/百万 token）：`provider→model` 精确价 > `provider→档位` 价 >
+`*` 通用档位 > 保守兜底。**未知 provider 绝不套用 DeepSeek 价格**，用通用档位并标记
+`pricingSource`（审计字段）。默认含 DeepSeek 官方口径，接入其他模型用
+`agentic_router_set_prices` 工具配价（持久化到 policy.json），或安装时传 `config.prices`：
+
+```js
+{ 'deepseek-official': { 'deepseek-v4-pro': { in: 1, out: 12 } },
+  openai: { 'gpt-x-pro': { in: 15, out: 60 } },
+  '*': { fast: { in: 1, out: 4 }, mid: { in: 2, out: 8 }, strong: { in: 3, out: 16 } } }
+```
 
 ## 分类器（规则先验，<1ms）
 
@@ -59,24 +120,11 @@ dsh plugin --profile web add dsh-agentic-router
 ## 已知边界
 
 - 模型档位靠 id 关键词识别（flash→fast、pro→strong）；可用模型多于一档时路由才真正切换
-- 奖励信号是"代理"：任务成功但质量平庸的情况（尤其模型池内质量差距大时）仍可能被记满分，正式场景建议接入 UI 反馈（messageFeedback）
+- 反馈结算窗口 45s（可配置 `settleDelayMs`），窗口内无反馈按代理奖励结算；重启会丢失未结算的 pending 条目；晚到的反馈可能归属到其后的回合
+- 反馈依赖部署的 Web UI 实际产生数据（本插件只读取，不写入）
 - 毕业策略（LinUCB/k-NN）需要数十个回合样本才会进元层池
 
-## 开发
-
-```sh
-npm test        # node --test，零依赖
-```
-
-回归用例覆盖实测缺陷：qa 祈使句、mid 档回退、UCB 冷启动轮转、agent 键隔离、奖励 JSON 合法性、飞轮重放恢复等。
-
-## 发布
-
-```sh
-npm publish                                     # 需 npm 账号
-git push origin main                            # GitHub 仓库公开后
-# 目录站：给仓库加 dsh-plugin topic 自动收录，或到 dshmarketplace.dev/submit 提交
-```
+**开发、测试与发布流程（维护者）见 [docs/MAINTAINING.md](./docs/MAINTAINING.md)。**
 
 ## License
 

package/docs/DESIGN.md

@@ -2,10 +2,12 @@
 
 > 目标：在 DeepSeek Harness 里做一个**学习型智能路由**插件——监听用户输入，按任务类型与复杂度把请求路由到最合适的模型 / 子 agent / 工具组合；每次路由的**决策与结果全部落盘**，形成数据飞轮，让路由判断随使用越来越准、越来越快。
 
+> ⚠️ **v1.4.0 机制修正**：下文 1.1 的 `agent/request` 返回替换配置方案已废弃——实测该返回值会被 `dsh-agent` 内置的模型选择监听器强制改回「选定模型」，不生效。v1.4.0 改为在 `agent/inbox/claimed` 向会话日志追加 `request/header`（reason: `router`），走官方模型选择优先级链真实切换；`agent/request` 降级为只读观测，`llm/stream` 提供中途热加载的兜底应用。实现见 `dsh/index.js` 顶部注释与 README「切换机制」一节。
+
 ## 0. 一句话架构
 
 ```
-用户输入 → 特征提取(快) → 路由器决策 → 改写请求(agent/request waterfall) / 派发子 agent
+用户输入 → 特征提取(快) → 路由器决策 → request/header 追加(真实切换) / 派发子 agent
                 ↑                                    ↓
         策略(规则表+学习权重) ← 评分与在线学习 ← 结果回传(质量/成本/延迟落盘)
 ```

package/docs/MAINTAINING.md

@@ -0,0 +1,74 @@
+# 维护者手册
+
+> 读者：本仓库的维护者/贡献者。安装使用者请看根目录 README。
+> 原则：**main 同时是 `github:` 安装源与目录站文档源，不要直接推 main；npm 版本不可变、可精确回滚，是正式发布门禁。**
+
+## 一次性设置
+
+### 1. GitHub 分支保护
+
+仓库 Settings → Branches → Add branch protection rule：
+
+- Branch name pattern：`main`
+- ✅ Require a pull request before merging
+- ✅ Require status checks to pass before merging → 选 `ci`
+- ✅ Require linear history（可选，保持历史干净）
+
+### 2. CI 发布令牌
+
+`.github/workflows/release.yml` 在 `v*` tag 上自动测试并 `npm publish`，需要一个 npm 令牌：
+
+1. npmjs.com → Access Tokens → Generate New Token → **Granular Access Token**
+2. 权限选本包 **Read and write**；Token 类型必须选 **Automation**（可绕过 2FA，官方为 CI 设计）
+3. GitHub 仓库 Settings → Secrets and variables → Actions → New repository secret
+   - Name：`NPM_TOKEN`
+   - Value：`npm_xxxxxxxx...`
+
+### 3. 目录站收录
+
+仓库 About → Topics 添加 `dsh-plugin`（目录站周期同步自动收录）；也可主动到
+<https://dshmarketplace.dev/submit> 提交仓库 URL（人工审核）。
+
+## 开发与测试
+
+```sh
+npm test        # node --test，零依赖
+```
+
+回归用例覆盖实测缺陷：qa 祈使句、mid 档回退、UCB 冷启动轮转、agent 键隔离、
+奖励 JSON 合法性、飞轮重放恢复、反馈结算时序、多 provider 价格解析等。
+
+背景：核心逻辑先在真实运行时的动态原型上逐阶段验证（P0 监听落盘 → P1 真实切换 →
+P2 三策略飞轮 → P3 质量/成本/反馈奖励），共发现并修复 8 个实测缺陷后才沉淀为本仓库，
+每个缺陷都对应一条回归测试。改动后请先本地 `npm test` 全绿再开 PR。
+
+## 日常发布流程
+
+```sh
+# 1. 功能分支开发（不要直接在 main 提交）
+git checkout -b feat/xxx
+git push -u origin feat/xxx          # 开 PR，ci 检查绿了再合并
+
+# 2. 合并到 main 后发布
+git checkout main && git pull
+npm version patch                    # bump 版本 + 自动打 vX.Y.Z tag + 提交
+git push origin main --follow-tags   # 推送后 CI 自动跑测试并发布 npm
+
+# 重大变更用 minor（npm version minor），不兼容用 major
+```
+
+## 发布回滚
+
+```sh
+# npm：版本不可变，出问题直接回滚 tag 指向旧版本
+npm dist-tag add dsh-agentic-router@1.2.0 latest
+
+# 源码：直接 revert 再走一遍发布流程
+git revert <坏提交> && npm version patch && git push origin main --follow-tags
+```
+
+## 版本号约定（SemVer）
+
+- **patch**：bug 修复、文档、测试（用户无感知）
+- **minor**：新功能（如新增工具、新奖励信号）
+- **major**：破坏性变更（工具改名、数据格式不兼容）

package/dsh/index.js

@@ -1,21 +1,31 @@
 // dsh-agentic-router —— DeepSeek Harness 插件（Host 侧）
 //
-// 学习型智能路由：监听 agent/pre-step 与 agent/request，按任务类型与复杂度在
-// 模型档位（fast/mid/strong）之间路由；四个专家（规则表 / UCB / LinUCB / k-NN）
-// 并行推荐，EXP3 元选择器决定听谁的；回合结束时按质量代理信号
-// （工具失败 + 模型重试 + 延迟）计算奖励并回传飞轮。
+// 学习型智能路由（v1.4.0 修复版）：按任务类型与复杂度在模型档位（fast/mid/strong）
+// 之间路由；四个专家（规则表 / UCB / LinUCB / k-NN）并行推荐，EXP3 元选择器决定
+// 听谁的；回合结束时按质量代理信号（工具失败 + 模型重试 + 成本 + 延迟）计算奖励
+// 并回传飞轮；人类 👍/👎 反馈在结算窗口后修正奖励。
 //
-// 架构依据 docs/architecture.md：
-//   - "Add a model-facing capability"    → ctx.tools 注册 4 个管理工具
-//   - "Intercept a request"              → agent/request waterfall 替换 LlmCallConfig
-//   - 所有监听经 ctx.on 注册，随插件卸载自动回收
+// ── 切换机制（v1.4.0 修复）────────────────────────────────────────────
+// v1.3.0 及之前通过 `agent/request` waterfall 返回替换后的 LlmCallConfig 来切模型，
+// 但该返回值会被 dsh-agent 内置的模型选择监听器强制改回「选定模型」，实际不生效。
+// 正确做法是走 dsh 官方的模型选择优先级链：
+//
+//     本进程选择 → 会话日志最新 request/header → 默认模型
+//
+// 在 `agent/inbox/claimed`（回合开始、prompt 组装之前）向会话日志追加一条
+// `request/header` 事件（reason: 'router'），下一轮组装就会读到新模型，真正生效。
+// `agent/request` 只做只读观测（记录每步实际出网模型）；插件中途热加载时，
+// `llm/stream` 在循环自身 header 落盘后补一次应用（deferred apply）。
+//
+// 其余架构依据 docs/architecture.md；所有监听经 ctx.on 注册，随插件卸载自动回收。
 // 飞轮数据落盘 ${DSH_HOME:-~/.dsh}/storages/dsh-agentic-router/，重启后重放恢复。
 
 import { classify, featuresOf } from './classify.js'
 import {
   tierOf, tierTarget, resolveTier,
   recommendRule, recommendUcb, recommendLin, recommendKnn,
-  rewardOf, applyReward,
+  rewardOf, applyReward, correctReward,
+  DEFAULT_PRICES, turnCostYuan, resolvePrice,
 } from './strategies.js'
 import { metaPick, DEFAULT_WEIGHTS } from './meta.js'
 import { Store, defaultHome } from './store.js'
@@ -25,6 +35,8 @@ export const inject = ['tools']
 
 const LIN_GRADUATION = 50   // LinUCB 毕业样本线
 const KNN_GRADUATION = 30   // k-NN 毕业经验线
+const SETTLE_DELAY_MS = 45000 // 反馈结算窗口：回合结束 45s 后查询 messageFeedback
+const SWEEP_MS = 15000        // 结算轮询周期
 
 function freshState() {
   return {
@@ -36,6 +48,10 @@ function freshState() {
     modelTiers: null,
     turnDecision: {},
     openTurns: new Map(),
+    pendingFeedback: [],
+    recByTurn: new Map(),
+    headerAppends: 0,
+    lastActual: null,
     ucb: { total: 0, arms: {} },
     lin: {
       total: 0,
@@ -82,6 +98,20 @@ function textOfMessages(messages) {
   return parts.join('\n').trim()
 }
 
+function extractText(message) {
+  if (message == null) return ''
+  if (typeof message.text === 'string') return message.text
+  const blocks = message.content
+  if (Array.isArray(blocks)) {
+    let out = ''
+    for (const b of blocks) {
+      if (b && typeof b === 'object' && b.type === 'text' && typeof b.text === 'string') out += b.text + '\n'
+    }
+    return out
+  }
+  return ''
+}
+
 function isErrorResult(r) {
   if (r == null || typeof r !== 'object') return false
   if (r.ok === false || r.success === false) return true
@@ -105,6 +135,57 @@ function execAgentKey(exec) {
   return null
 }
 
+// 会话句柄：agent 的 session（inbox / request 路径）
+function sessionOf(payload) {
+  try {
+    const a = payload && payload.agent
+    const s = a && a.session
+    if (s && typeof s.append === 'function' && typeof s.requestHeader === 'function' && typeof s.id === 'string') return s
+  } catch {}
+  return null
+}
+
+function headerConfig(session) {
+  try {
+    const h = session.requestHeader()
+    if (h && h.config && typeof h.config === 'object') return h.config
+  } catch {}
+  return null
+}
+
+function currentModel(session) {
+  const c = headerConfig(session)
+  return c && typeof c.model === 'string' ? c.model : null
+}
+
+function currentProvider(session) {
+  const c = headerConfig(session)
+  return c && typeof c.provider === 'string' ? c.provider : ''
+}
+
+// 核心切换：向会话日志追加 request/header（reason: 'router'），
+// 下一轮 prompt 组装即读到新模型。fast/mid 档去掉 reasoningEffort，
+// 让目标模型走自身默认 effort（避免 flash 不支持 max 而整步失败）。
+function appendRouterHeader(session, targetModel) {
+  try {
+    const last = session.requestHeader()
+    const header = {}
+    if (last && typeof last === 'object') {
+      for (const k of Object.keys(last)) header[k] = last[k]
+    }
+    const prevConfig = header.config && typeof header.config === 'object' ? header.config : {}
+    const config = {}
+    for (const k of Object.keys(prevConfig)) config[k] = prevConfig[k]
+    config.model = targetModel
+    if (tierOf(targetModel) !== 'strong') delete config.reasoningEffort
+    header.config = config
+    session.append('request/header', { header, reason: 'router' })
+    return true
+  } catch (e) {
+    return false
+  }
+}
+
 async function refreshModelTiers(llm) {
   if (!llm || typeof llm.listProviders !== 'function') return null
   const providers = await llm.listProviders()
@@ -124,25 +205,197 @@ async function refreshModelTiers(llm) {
 export function apply(ctx, config = {}) {
   const tools = ctx.get('tools')
   const llm = ctx.get('llm')
+  const mf = ctx.get('messageFeedback')
+  const timer = ctx.get('timer')
   const store = new Store(config.home || defaultHome())
+  const settleDelayMs = typeof config.settleDelayMs === 'number' ? config.settleDelayMs : SETTLE_DELAY_MS
   const policy = store.loadPolicy()
+  let prices = (policy.prices && Object.keys(policy.prices).length) ? policy.prices : (config.prices || DEFAULT_PRICES)
+  const costWeight = typeof config.costWeight === 'number' ? config.costWeight : 20
+  const costCap = typeof config.costCap === 'number' ? config.costCap : 0.5
 
   const state = freshState()
   state.mode = policy.mode || config.mode || 'shadow'
   if (typeof policy.forced === 'string') state.forced = policy.forced
 
-  // 飞轮恢复：重放 rewards.jsonl 重建 UCB/LinUCB/k-NN/EXP3
+  // 飞轮恢复：重放 rewards.jsonl 重建 UCB/LinUCB/k-NN/EXP3；
+  // feedback-settle 行在基础奖励重放后按 turn 精确修正
   const rewards = store.loadJsonl('rewards.jsonl')
+  const byTurn = new Map()
+  for (const rec of rewards) {
+    if (rec.type === 'feedback-settle') continue
+    if (typeof rec.reward === 'number' && Array.isArray(rec.f)) {
+      applyReward(state, rec)
+      byTurn.set(rec.turn, rec)
+    }
+  }
   for (const rec of rewards) {
-    if (typeof rec.reward === 'number' && Array.isArray(rec.f)) applyReward(state, rec)
+    if (rec.type !== 'feedback-settle') continue
+    const baseRec = byTurn.get(rec.turn)
+    if (baseRec) correctReward(state, baseRec, rec.newReward)
   }
+  state.recByTurn = byTurn
   state.decisions = store.loadJsonl('decisions.jsonl').slice(-60)
   state.seq = state.decisions.length ? state.decisions[state.decisions.length - 1].seq : 0
 
   const log = (msg) => console.log('[agentic-router]', msg)
   if (rewards.length) log('replayed ' + rewards.length + ' rewards; ucb.total=' + state.ucb.total)
 
+  let mfWarned = false
+  // 反馈结算：对已过结算窗口的回合查询 messageFeedback，按人类 👍/👎 修正奖励
+  async function settleFeedback() {
+    if (!mf || typeof mf.list !== 'function') {
+      if (!mfWarned) { log('messageFeedback service absent — 反馈奖励停用'); mfWarned = true }
+      return
+    }
+    const tNow = now()
+    if (tNow == null) return
+    const remaining = []
+    for (const p of state.pendingFeedback) {
+      if (tNow - p.t0 < settleDelayMs) { remaining.push(p); continue }
+      try {
+        const res = await mf.list({ sessionId: p.sessionId })
+        let neg = 0
+        let pos = 0
+        if (res && res.ok === true && res.value && Array.isArray(res.value.items)) {
+          for (const it of res.value.items) {
+            if (typeof it.createdAt !== 'number' || it.createdAt < p.t0) continue
+            if (it.rating === 'negative') neg += 1
+            else if (it.rating === 'positive') pos += 1
+          }
+        }
+        const newReward = Math.max(0, Math.min(1, Math.round((p.appliedReward - 0.5 * neg + 0.1 * pos) * 100) / 100))
+        if (newReward !== p.appliedReward) {
+          const baseRec = state.recByTurn.get(p.key)
+          if (baseRec) correctReward(state, baseRec, newReward)
+          store.appendJsonl('rewards.jsonl', {
+            type: 'feedback-settle', turn: p.key, sessionId: p.sessionId,
+            negative: neg, positive: pos, applied: p.appliedReward, newReward,
+          })
+          log('feedback settle ' + p.key + ': ' + p.appliedReward + ' -> ' + newReward + ' (neg=' + neg + ' pos=' + pos + ')')
+        }
+      } catch (e) {
+        log('feedback settle failed: ' + String(e))
+      }
+    }
+    state.pendingFeedback = remaining
+  }
+
+  if (timer && typeof timer.interval === 'function') {
+    timer.interval(() => { settleFeedback().catch((e) => log('settle sweep failed: ' + String(e))) }, SWEEP_MS)
+  }
+
+  // ---- 决策流水线（inbox 与 request 兜底共用） ----
+  // 返回 { key, td, record, target }；已决策过本回合则返回 null。
+  // 档位表异步加载；决策前等待就绪（tiers-not-ready 会拿不到 target）。
+  async function decideAndRecord(payload, text, current, provider, source) {
+    if (!state.modelTiers) {
+      try { await tiersReady } catch (e) { /* keep null */ }
+    }
+    const key = fullKey(payload)
+    if (state.turnDecision[key]) return null
+    const cls = classify(text)
+    const f = featuresOf(cls, text)
+    const recs = {
+      rule: recommendRule(cls),
+      ucb: recommendUcb(cls, state.ucb),
+      lin: recommendLin(cls, f, state.lin),
+      knn: recommendKnn(cls, f, state.knn),
+    }
+    if (state.forced) recs.rule = { tier: 'forced', reason: 'forced', conf: 1 }
+    const exp3 = {
+      ...state.exp3,
+      linGraduated: state.lin.total >= LIN_GRADUATION,
+      knnGraduated: state.knn.pool.length >= KNN_GRADUATION,
+    }
+    const meta = metaPick(recs, exp3)
+    const chosenRec = meta.byName[meta.chosen] || recs.rule
+    const resolved = state.forced
+      ? { target: state.forced, used: 'forced', note: 'forced' }
+      : resolveTier(chosenRec.tier, provider, state.modelTiers)
+    const target = resolved.target
+    state.seq += 1
+    const td = {
+      clsType: cls.type, f, recs, meta, chosen: meta.chosen,
+      finalized: false, t0: now(), toolErrors: 0, retries: 0,
+      currentModel: current, provider, costWeight, costCap,
+      estInputTokens: Math.ceil(text.length / 1.5),
+      target, applied: false, needsApply: false, actualModel: current,
+    }
+    const record = {
+      seq: state.seq,
+      turn: key,
+      mode: state.mode,
+      taskType: cls.type,
+      complexity: cls.complexity,
+      inputPreview: text.slice(0, 100),
+      currentModel: current,
+      recs: {
+        rule: recs.rule.tier + '(' + recs.rule.conf + ')',
+        ucb: recs.ucb.tier + '(' + recs.ucb.conf + ')',
+        lin: recs.lin.tier + '(' + recs.lin.conf + ')',
+        knn: recs.knn.tier + '(' + recs.knn.conf + ')',
+      },
+      metaChosen: meta.chosen,
+      metaProbs: meta.ps,
+      metaPool: meta.pool,
+      resolvedTier: resolved.used + (resolved.note ? ' [' + resolved.note + ']' : ''),
+      targetModel: state.mode === 'active' ? target : null,
+      applied: false,
+      appliedAt: source,
+      swapped: false,
+    }
+    td.rec = record
+    state.turnDecision[key] = td
+    state.openTurns.set(agentKey(payload), key)
+    return { key, td, record, target }
+  }
+
+  // 决策记录落盘（在 applyDecision 之后调用，保证 applied/swapped 已是最终值）
+  function commitRecord(record) {
+    store.appendJsonl('decisions.jsonl', record)
+    state.decisions.push(record)
+    if (state.decisions.length > 200) state.decisions.shift()
+  }
+
+  // 应用切换：active 模式且目标与当前不同 → 追加 request/header。
+  // 返回是否真实追加成功。
+  function applyDecision(session, key, td, record, target) {
+    if (state.mode !== 'active' || !target) return false
+    const base = td.actualModel || td.currentModel
+    if (base && target === base) return false
+    const ok = session ? appendRouterHeader(session, target) : false
+    td.applied = ok
+    if (record) { record.applied = ok; record.swapped = ok }
+    if (ok) {
+      state.headerAppends++
+      log('SWAP -> ' + target + ' | ' + td.chosen + ' | ' + (td.recs && td.recs[td.chosen] ? td.recs[td.chosen].reason : ''))
+    }
+    return ok
+  }
+
   // ---- 事件挂载 ----
+  // 回合开始（prompt 组装之前）：捕获用户消息并决策，active 模式立即应用切换。
+  // 这是主路径——零延迟，回合第一步就按目标模型执行。
+  ctx.on('agent/inbox/claimed', async (payload) => {
+    try {
+      const session = sessionOf(payload)
+      const text = extractText(payload && payload.message)
+      if (!text) return
+      const current = session ? currentModel(session) : null
+      const provider = session ? currentProvider(session) : ''
+      const r = await decideAndRecord(payload, text, current, provider, 'inbox')
+      if (!r) return
+      applyDecision(session, r.key, r.td, r.record, r.target)
+      commitRecord(r.record)
+      log('decision #' + state.seq + ' type=' + r.td.clsType + ' c=' + r.td.f[1] +
+        ' meta=' + r.td.chosen + ' tier=' + r.record.resolvedTier +
+        ' applied=' + r.td.applied + ' target=' + (r.target || '-'))
+    } catch (e) {
+      log('inbox decide failed: ' + String(e))
+    }
+  })
+
   ctx.on('agent/pre-step', (payload, next) => {
     try {
       const text = textOfMessages(payload && payload.messages)
@@ -153,76 +406,35 @@ export function apply(ctx, config = {}) {
     return next()
   })
 
+  // 只读观测 + 兜底决策。观测：记录每步真实出网模型（供 stats 验证切换生效）。
+  // 兜底：inbox 未触发（插件中途热加载）时，在首个请求用 pending 文本决策，
+  // 但不再替换配置——改由 llm/stream 在循环 header 落盘后补应用。
   ctx.on('agent/request', async (payload, next) => {
     const cfg = await next()
     if (!cfg || typeof cfg !== 'object') return cfg
     try {
+      if (!timer || typeof timer.interval !== 'function') settleFeedback().catch(() => {})
       const key = fullKey(payload)
-      if (state.turnDecision[key]) return cfg
+      const model = typeof cfg.model === 'string' ? cfg.model : null
+      const td = state.turnDecision[key]
+      if (td) {
+        if (model) {
+          td.actualModel = model
+          state.lastActual = { model, turn: payload.turn, step: payload.step, key }
+        }
+        return cfg
+      }
       const pkey = agentKey(payload) + ':' + payload.turn
       const text = state.pending.get(pkey) || ''
       state.pending.delete(pkey)
-      const cls = classify(text)
-      const f = featuresOf(cls, text)
-      const current = typeof cfg.model === 'string' ? cfg.model : '(unknown)'
       const provider = typeof cfg.provider === 'string' ? cfg.provider : ''
-      const recs = {
-        rule: recommendRule(cls),
-        ucb: recommendUcb(cls, state.ucb),
-        lin: recommendLin(cls, f, state.lin),
-        knn: recommendKnn(cls, f, state.knn),
-      }
-      if (state.forced) recs.rule = { tier: 'forced', reason: 'forced', conf: 1 }
-      const exp3 = {
-        ...state.exp3,
-        linGraduated: state.lin.total >= LIN_GRADUATION,
-        knnGraduated: state.knn.pool.length >= KNN_GRADUATION,
+      const r = await decideAndRecord(payload, text, model, provider, 'request')
+      if (r) {
+        r.td.needsApply = state.mode === 'active' && r.target && r.target !== r.td.currentModel
+        commitRecord(r.record)
+        if (r.td.needsApply) log('decision #' + state.seq + ' -> apply deferred to llm/stream')
+        else log('decision #' + state.seq + ' type=' + r.td.clsType + ' meta=' + r.td.chosen + ' tier=' + r.record.resolvedTier)
       }
-      const meta = metaPick(recs, exp3)
-      const chosenRec = meta.byName[meta.chosen] || recs.rule
-      const resolved = state.forced
-        ? { target: state.forced, used: 'forced', note: 'forced' }
-        : resolveTier(chosenRec.tier, provider, state.modelTiers)
-      const target = resolved.target
-      state.seq += 1
-      const record = {
-        seq: state.seq,
-        turn: key,
-        mode: state.mode,
-        taskType: cls.type,
-        complexity: cls.complexity,
-        inputPreview: text.slice(0, 100),
-        currentModel: current,
-        recs: {
-          rule: recs.rule.tier + '(' + recs.rule.conf + ')',
-          ucb: recs.ucb.tier + '(' + recs.ucb.conf + ')',
-          lin: recs.lin.tier + '(' + recs.lin.conf + ')',
-          knn: recs.knn.tier + '(' + recs.knn.conf + ')',
-        },
-        metaChosen: meta.chosen,
-        metaProbs: meta.ps,
-        metaPool: meta.pool,
-        resolvedTier: resolved.used + (resolved.note ? ' [' + resolved.note + ']' : ''),
-        targetModel: state.mode === 'active' ? target : null,
-        swapped: false,
-      }
-      state.turnDecision[key] = {
-        clsType: cls.type, f, recs, meta, chosen: meta.chosen,
-        finalized: false, t0: now(), toolErrors: 0, retries: 0,
-      }
-      state.openTurns.set(agentKey(payload), key)
-      store.appendJsonl('decisions.jsonl', record)
-      if (state.mode === 'active' && target && target !== current) {
-        record.swapped = true
-        state.decisions.push(record)
-        if (state.decisions.length > 200) state.decisions.shift()
-        log('SWAP -> ' + target + ' | ' + meta.chosen + ' | ' + chosenRec.reason + ' | ' + resolved.note)
-        return { ...cfg, model: target }
-      }
-      state.decisions.push(record)
-      if (state.decisions.length > 200) state.decisions.shift()
-      log('decision #' + state.seq + ' type=' + cls.type + ' c=' + cls.complexity +
-        ' meta=' + meta.chosen + ' tier=' + resolved.used + ' recs=' + JSON.stringify(record.recs))
     } catch (e) {
       log('decide failed: ' + String(e))
     }
@@ -242,6 +454,62 @@ export function apply(ctx, config = {}) {
     return next()
   })
 
+  // 成本信号：透传每次流式模型调用，累加 usage chunk 到当前 open turn；
+  // 无 usage 的 provider 用文本长度估算（estimated=true）。
+  // 同时：request 路径的兜底决策在这里补应用切换（循环自身 header 已落盘，无竞态）。
+  ctx.on('llm/stream', function (options, next) {
+    return (async function* () {
+      let usage = null
+      let outTextLen = 0
+      try {
+        for await (const chunk of next()) {
+          if (chunk && chunk.type === 'usage' && chunk.usage) usage = chunk.usage
+          if (chunk && chunk.type === 'text-delta' && typeof chunk.text === 'string') outTextLen += chunk.text.length
+          yield chunk
+        }
+      } finally {
+        try {
+          const sid = options && (options.sessionId || (options.session && options.session.id))
+          const key = sid ? state.openTurns.get(sid) : null
+          const td = key ? state.turnDecision[key] : null
+          if (td && !td.finalized) {
+            // 兜底应用：request 路径决策未应用时，在 header 落盘后补切换
+            if (td.needsApply && !td.applied) {
+              let session = null
+              try {
+                const sessions = ctx.get('sessions')
+                if (sessions && typeof sessions.get === 'function') session = sessions.get(sid) || null
+              } catch (e) { session = null }
+              if (session) applyDecision(session, key, td, td.rec, td.target)
+            }
+            const u = td.usage || { inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, reasoningTokens: 0, estimated: false }
+            const model = (options && typeof options.model === 'string' && options.model) || td.currentModel || ''
+            const tier = tierOf(model)
+            const provider = (options && typeof options.provider === 'string' && options.provider) || td.provider || ''
+            if (usage && typeof usage.inputTokens === 'number') {
+              u.inputTokens += usage.inputTokens
+              u.outputTokens += usage.outputTokens || 0
+              u.cacheReadTokens += usage.cacheReadTokens || 0
+              u.reasoningTokens += usage.reasoningTokens || 0
+              u.estimated = false
+            } else {
+              u.inputTokens += td.estInputTokens || 0
+              u.outputTokens += Math.round(outTextLen / 2)
+              u.estimated = true
+            }
+            td.usage = u
+            const rp = resolvePrice(provider, model, tier, prices)
+            td.costYuan = turnCostYuan(u, provider, model, tier, prices)
+            td.pricingSource = rp.source
+            log('usage on ' + key + ': in=' + u.inputTokens + ' out=' + u.outputTokens + (u.estimated ? ' (估)' : '') + ' cost=¥' + td.costYuan.toFixed(4) + ' [' + rp.source + ']')
+          }
+        } catch (e) {
+          log('usage hook failed: ' + String(e))
+        }
+      }
+    })()
+  })
+
   ctx.on('tools/result', (exec, result) => {
     try {
       if (!isErrorResult(result)) return
@@ -283,6 +551,15 @@ export function apply(ctx, config = {}) {
       }
       applyReward(state, rec)
       store.appendJsonl('rewards.jsonl', rec)
+      state.recByTurn.set(key, rec)
+      if (state.recByTurn.size > 500) {
+        const first = state.recByTurn.keys().next().value
+        if (first !== undefined) state.recByTurn.delete(first)
+      }
+      if (mf && typeof mf.list === 'function' && td.t0 != null) {
+        state.pendingFeedback.push({ key, sessionId: agentKey(payload), t0: td.t0, appliedReward: rec.reward })
+        if (state.pendingFeedback.length > 200) state.pendingFeedback.shift()
+      }
       const ak = agentKey(payload)
       if (state.openTurns.get(ak) === key) state.openTurns.delete(ak)
       log('reward ' + rec.reward + ' ' + tag + ' on ' + key + ' breakdown=' + JSON.stringify(r.breakdown))
@@ -294,10 +571,12 @@ export function apply(ctx, config = {}) {
   ctx.on('agent/turn-stopping', (payload) => finalize(payload, 'clean-close'))
   ctx.on('agent/error', (payload) => finalize(payload, 'error'))
 
-  refreshModelTiers(llm).then((t) => {
+  // 档位表异步加载；decideAndRecord 在决策前 await 此 promise
+  const tiersReady = refreshModelTiers(llm)
+  tiersReady.then((t) => {
     state.modelTiers = t
     log('model tiers: ' + JSON.stringify(t))
-  })
+  }).catch((e) => log('model tiers refresh failed: ' + String(e)))
 
   // ---- 工具 ----
   const OUT = (label) => ({
@@ -315,7 +594,7 @@ export function apply(ctx, config = {}) {
 
   register({
     name: 'agentic_router_stats',
-    description: '查看 agentic router（UCB/LinUCB/k-NN 三策略 + EXP3 元选择器 + 质量代理奖励）的决策记录、策略权重与飞轮学习状态。',
+    description: '查看 agentic router（UCB/LinUCB/k-NN 三策略 + EXP3 元选择器 + 质量代理奖励）的决策记录、策略权重、真实切换与飞轮学习状态。',
     parameters: { type: 'object', properties: {} },
     output: OUT('agentic router 状态'),
     isConcurrencySafe: () => true,
@@ -325,14 +604,21 @@ export function apply(ctx, config = {}) {
         forcedModel: state.forced,
         totalDecisions: state.seq,
         modelTiers: state.modelTiers,
+        prices,
+        mechanism: 'request/header append (agent/inbox/claimed)',
+        lastActual: state.lastActual,
+        headerAppends: state.headerAppends,
         flywheel: {
           ucb: state.ucb,
           lin: { total: state.lin.total, tierN: state.lin.tierN, tierW: state.lin.tierW },
           knnPoolSize: state.knn.pool.length,
           exp3: state.exp3,
+          pendingFeedback: state.pendingFeedback.length,
+          prices,
           recentRewards: store.loadJsonl('rewards.jsonl').slice(-20),
         },
         recent: state.decisions.slice(-20),
+        note: '切换真实性验证：headerAppends 为本进程追加的 request/header 次数；lastActual 为最近一步真实出网模型；decisions[].applied=true 且 swapped=true 即真实切换成功',
       }
     },
   })
@@ -348,7 +634,7 @@ export function apply(ctx, config = {}) {
     isConcurrencySafe: () => true,
     async execute(args) {
       state.mode = args.mode
-      store.savePolicy({ mode: state.mode, forced: state.forced })
+      store.savePolicy({ mode: state.mode, forced: state.forced, prices })
       return { mode: state.mode }
     },
   })
@@ -367,17 +653,62 @@ export function apply(ctx, config = {}) {
     async execute(args) {
       if (args && args.clear) {
         state.forced = null
-        store.savePolicy({ mode: state.mode, forced: null })
+        store.savePolicy({ mode: state.mode, forced: null, prices })
         return { forced: null }
       }
       if (args && typeof args.model === 'string') {
         state.forced = args.model
-        store.savePolicy({ mode: state.mode, forced: args.model })
+        store.savePolicy({ mode: state.mode, forced: args.model, prices })
+        // 立即对存活会话应用（下一请求生效）
+        if (state.mode === 'active') {
+          try {
+            const agents = ctx.get('agents')
+            if (agents && typeof agents.list === 'function') {
+              for (const agent of agents.list()) {
+                try {
+                  const session = sessionOf({ agent })
+                  if (session && currentModel(session) !== args.model) {
+                    const ok = appendRouterHeader(session, args.model)
+                    if (ok) state.headerAppends++
+                  }
+                } catch (e) { /* per-agent best effort */ }
+              }
+            }
+          } catch (e) { /* best effort */ }
+        }
         return { forced: args.model }
       }
       return { forced: state.forced, error: '需要 model 或 clear=true' }
     },
   })
+  register({
+    name: 'agentic_router_set_prices',
+    description: '配置/查询某 provider 某模型（或档位）的价格（元/百万 tokens，{in, out}）。不带 priceIn/priceOut 时返回当前价格表。接入新模型后用它补价格，未知价格按保守值估算并标记 unknown。',
+    parameters: {
+      type: 'object',
+      properties: {
+        provider: { type: 'string', description: 'provider id，如 openai、anthropic、deepseek-official' },
+        model: { type: 'string', description: '可选：精确模型 id；省略则设置该 provider 的档位价' },
+        tier: { type: 'string', enum: ['fast', 'mid', 'strong'], description: '档位（未给 model 时必填）' },
+        priceIn: { type: 'number', description: '输入价（元/百万 tokens），负数视为查询模式' },
+        priceOut: { type: 'number', description: '输出价（元/百万 tokens），负数视为查询模式' },
+      },
+      required: ['provider'],
+    },
+    output: OUT('agentic router 价格表'),
+    isConcurrencySafe: () => true,
+    async execute(args) {
+      if (typeof args.priceIn !== 'number' || typeof args.priceOut !== 'number' || args.priceIn < 0 || args.priceOut < 0) {
+        return { prices, note: '查看模式：传 priceIn/priceOut 与 model 或 tier 进行设置' }
+      }
+      const byProvider = prices[args.provider] || (prices[args.provider] = {})
+      if (args.model) byProvider[args.model] = { in: args.priceIn, out: args.priceOut }
+      else if (args.tier) byProvider[args.tier] = { in: args.priceIn, out: args.priceOut }
+      else return { error: '需要 model 或 tier', prices }
+      store.savePolicy({ mode: state.mode, forced: state.forced, prices })
+      return { prices, updated: args.model || (args.provider + '/' + args.tier) }
+    },
+  })
   register({
     name: 'agentic_router_reset',
     description: '清空飞轮学习状态（决策与奖励历史），保留当前模式；用于重新开始学习或清理测试数据。',
@@ -396,6 +727,8 @@ export function apply(ctx, config = {}) {
       state.exp3 = { total: 0, weights: { ...DEFAULT_WEIGHTS } }
       state.seq = 0
       state.decisions = []
+      state.headerAppends = 0
+      state.lastActual = null
       return { reset: true, home: store.home }
     },
   })

package/dsh/strategies.js

@@ -105,28 +105,76 @@ export function recommendKnn(cls, f, knnState) {
   return { tier: best.tier, reason: 'knn k=' + k + ' neighbors score=' + Math.round(best.score * 100) / 100, conf: Math.min(0.8, pool.length / 100) }
 }
 
-// 质量代理奖励：干净收尾 1 分起，扣工具失败(0.2/次，封顶 0.6)、模型重试(0.2/次，封顶 0.4)、延迟(>90s −0.3, >30s −0.1)
+// ---- 成本信号（token 计费） ----
+// 价格表（元/百万 tokens）三级解析：
+//   1) prices[provider][model]          精确模型价（最高优先）
+//   2) prices[provider][tier]           该 provider 的档位价
+//   3) prices['*'][tier]                通用档位兜底（未知 provider 不套用 DeepSeek 价）
+//   4) CONSERVATIVE                     保守值 + 标记 unknown，提醒用户补价格
+export const DEFAULT_PRICES = {
+  'deepseek-official': {
+    'deepseek-v4-flash': { in: 0.2, out: 2 },
+    'deepseek-v4-pro': { in: 1, out: 12 },
+  },
+  '*': {
+    fast: { in: 1, out: 4 },
+    mid: { in: 2, out: 8 },
+    strong: { in: 3, out: 16 },
+  },
+}
+
+const CONSERVATIVE = { in: 3, out: 16 }
+
+// 解析价格并返回来源（透明标注：价格到底从哪来的）
+export function resolvePrice(provider, model, tier, prices) {
+  const table = prices || DEFAULT_PRICES
+  const byProvider = table[provider]
+  if (byProvider && byProvider[model]) return { price: byProvider[model], source: 'provider-model' }
+  if (byProvider && byProvider[tier]) return { price: byProvider[tier], source: 'provider-tier' }
+  const generic = table['*']
+  if (generic && generic[tier]) return { price: generic[tier], source: 'generic-tier' }
+  return { price: CONSERVATIVE, source: 'unknown-conservative' }
+}
+
+// 单回合成本（元）：缓存读计入输入，推理 token 计入输出
+export function turnCostYuan(usage, provider, model, tier, prices) {
+  if (!usage) return 0
+  const { price } = resolvePrice(provider, model, tier, prices)
+  const input = (usage.inputTokens || 0) + (usage.cacheReadTokens || 0)
+  const output = (usage.outputTokens || 0) + (usage.reasoningTokens || 0)
+  return (price.in * input + price.out * output) / 1e6
+}
+
+// 质量代理奖励：干净收尾 1 分起，扣工具失败(0.2/次，封顶 0.6)、模型重试(0.2/次，封顶 0.4)、
+// 延迟(>90s −0.3, >30s −0.1)、成本(min(cap, weight × 元)，默认 weight=20 即每 0.01 元 −0.2)
 export function rewardOf(td, tag, dtMs) {
   if (tag !== 'clean-close') {
-    return { reward: 0, breakdown: { base: 0, toolErrors: td.toolErrors || 0, retries: td.retries || 0, dtMs } }
+    return { reward: 0, breakdown: { base: 0, toolErrors: td.toolErrors || 0, retries: td.retries || 0, dtMs, costYuan: td.costYuan || 0, costPenalty: 0, pricingSource: td.pricingSource || 'n/a' } }
   }
   let reward = 1
   const toolPenalty = Math.min(0.6, 0.2 * (td.toolErrors || 0))
   const retryPenalty = Math.min(0.4, 0.2 * (td.retries || 0))
+  const costCap = typeof td.costCap === 'number' ? td.costCap : 0.5
+  const costWeight = typeof td.costWeight === 'number' ? td.costWeight : 20
+  const costYuan = typeof td.costYuan === 'number' ? td.costYuan : 0
+  const costPenalty = Math.min(costCap, costWeight * costYuan)
   reward -= toolPenalty
   reward -= retryPenalty
+  reward -= costPenalty
   if (dtMs != null) {
     if (dtMs > 90000) reward -= 0.3
     else if (dtMs > 30000) reward -= 0.1
   }
   return {
     reward: Math.max(0, Math.round(reward * 100) / 100),
-    breakdown: { base: 1, toolErrors: td.toolErrors || 0, toolPenalty, retries: td.retries || 0, retryPenalty, dtMs },
+    breakdown: { base: 1, toolErrors: td.toolErrors || 0, toolPenalty, retries: td.retries || 0, retryPenalty, costYuan, costPenalty, dtMs, pricingSource: td.pricingSource || 'n/a' },
   }
 }
 
 // 把一条奖励记录应用到飞轮（UCB/LinUCB/k-NN/EXP3），replay 与实时更新共用
 export function applyReward(state, rec) {
+  // 记录本次线性更新的学习率，供反馈结算修正精确回退
+  rec.linLr = 1 / (1 + (state.lin.tierN[rec.linTier] || 0))
   const armKey = rec.clsType + ':' + rec.ucbTier
   const arm = state.ucb.arms[armKey] || (state.ucb.arms[armKey] = { n: 0, mean: 0 })
   arm.n += 1
@@ -151,3 +199,30 @@ export function applyReward(state, rec) {
   }
   state.exp3.total += 1
 }
+
+// 反馈结算修正：把已应用奖励改为 newReward（人类在反馈窗口内点了 👍/👎 之后）
+// UCB 均值修正（n 不变）；LinUCB 用原始学习率做精确梯度回退；k-NN 追加修正经验；
+// EXP3 按乘性更新做精确逆修正。
+export function correctReward(state, rec, newReward) {
+  const delta = newReward - rec.reward
+  if (!delta) return { delta: 0 }
+
+  const arm = state.ucb.arms[rec.clsType + ':' + rec.ucbTier]
+  if (arm && arm.n > 0) arm.mean = arm.mean + delta / arm.n
+
+  const lw = state.lin.tierW[rec.linTier]
+  if (lw && Array.isArray(rec.f)) {
+    const lr = typeof rec.linLr === 'number' ? rec.linLr : 1 / (1 + (state.lin.tierN[rec.linTier] || 0))
+    for (let i = 0; i < rec.f.length; i++) lw[i] = (lw[i] || 0) + lr * delta * rec.f[i]
+  }
+
+  if (state.knn && Array.isArray(rec.f)) state.knn.pool.push({ f: rec.f, tier: rec.ucbTier, r: newReward })
+
+  if (rec.pChosen > 0 && rec.pool && rec.pool.length > 0) {
+    state.exp3.weights[rec.chosen] = (state.exp3.weights[rec.chosen] || 0) *
+      Math.exp(0.3 * delta / (rec.pChosen * rec.pool.length))
+  }
+
+  rec.reward = newReward
+  return { delta }
+}

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "dsh-agentic-router",
-  "version": "1.0.0",
-  "description": "DeepSeek Harness plugin: learning agentic router — rule + UCB + LinUCB + k-NN experts under an EXP3 meta-selector, with a durable quality-reward flywheel",
+  "version": "1.4.0",
+  "description": "DeepSeek Harness plugin: learning agentic router — rule + UCB + LinUCB + k-NN experts under an EXP3 meta-selector, with a durable quality-reward flywheel and REAL model switching via the session request-header precedence chain (v1.4.0)",
   "type": "module",
   "license": "MIT",
   "author": "lhl",
@@ -33,7 +33,7 @@
     "LICENSE"
   ],
   "scripts": {
-    "test": "node --test test/"
+    "test": "node --test"
   },
   "dsh": {
     "bundle": {