dsh-agentic-router 1.0.0 → 1.3.0

package/README.en.md

@@ -40,21 +40,42 @@ dsh plugin --profile web add dsh-agentic-router
 | `agentic_router_set_mode` | shadow (default) / active (real swaps) / off (bypass) |
 | `agentic_router_force` | force a model id (active mode only); clear=true to release |
 | `agentic_router_reset` | wipe flywheel learning state, keep mode |
+| `agentic_router_set_prices` | set/query per-provider per-model (or per-tier) prices |
 
 ## Data flywheel (persistence)
 
 Directory: `${DSH_HOME:-~/.dsh}/storages/dsh-agentic-router/` (override with `AGENTIC_ROUTER_HOME`)
 
 - `decisions.jsonl` — one line per decision: task type/complexity, all four recommendations, meta pick, actual route, swapped?
-- `rewards.jsonl` — one line per reward with breakdown (tool errors/retries/latency) + feature vector; **replayed on startup to restore learning state**
+- `rewards.jsonl` — one line per reward with breakdown (tool errors/retries/latency) + feature vector; **replayed on startup to restore learning state**; human-feedback settlements (`feedback-settle` lines) correct the replayed rewards exactly
+- Human feedback: 45s after a turn closes, the plugin polls `messageFeedback` (the Web UI 👍/👎); a thumbs-down costs −0.5, a thumbs-up adds +0.1, applied as an exact correction to the already-applied reward
 - `policy.json` — mode and forced model (atomic write)
 
-Reward: clean close starts at 1; tool failures −0.2 each (cap 0.6), model retries −0.2 each (cap 0.4), latency >30s −0.1 / >90s −0.3; errored turns score 0.
+Reward: clean close starts at 1; tool failures −0.2 each (cap 0.6), model retries −0.2 each (cap 0.4),
+**cost** min(0.5, 20 × turn cost in CNY), latency >30s −0.1 / >90s −0.3; errored turns score 0.
+
+**Cost signal (token billing)**: the plugin wraps `llm/stream` and accumulates each model
+call's `usage` chunk (input/output/cache-read/reasoning tokens); providers without usage
+fall back to length-based estimates (flagged `estimated`).
+
+**Three-level price resolution** (CNY per million tokens): `provider→model` exact price >
+`provider→tier` price > `*` generic tier > conservative fallback. **Unknown providers
+never inherit DeepSeek pricing** — they use the generic tier and carry a `pricingSource`
+audit field. DeepSeek's official pricing ships as default; add other models at runtime via
+the `agentic_router_set_prices` tool (persisted to policy.json) or at install via
+`config.prices`:
+
+```js
+{ 'deepseek-official': { 'deepseek-v4-pro': { in: 1, out: 12 } },
+  openai: { 'gpt-x-pro': { in: 15, out: 60 } },
+  '*': { fast: { in: 1, out: 4 }, mid: { in: 2, out: 8 }, strong: { in: 3, out: 16 } } }
+```
 
 ## Known limits
 
 - Model tiers are recognized by id keywords (flash→fast, pro→strong); real swaps happen only when more than one tier exists
-- Rewards are proxies: a mediocre-but-successful answer still scores full marks, so wire up UI feedback (messageFeedback) for production use
+- Feedback settles after a 45s window (configurable via `settleDelayMs`); unsettled pending entries are lost on restart; late feedback may be attributed to a later turn
+- Feedback only works where the deployment's Web UI actually writes it (this plugin only reads)
 - LinUCB / k-NN need dozens of turns before they graduate into the meta pool
 
 ## Development

package/README.md

@@ -40,16 +40,33 @@ dsh plugin --profile web add dsh-agentic-router
 | `agentic_router_set_mode` | 切换 shadow（默认）/ active（真切换）/ off（旁路） |
 | `agentic_router_force` | 强制指定模型 id（active 生效），clear=true 清除 |
 | `agentic_router_reset` | 清空飞轮学习状态，保留模式 |
+| `agentic_router_set_prices` | 配置/查询某 provider 某模型（或档位）的价格 |
 
 ## 数据飞轮（落盘）
 
 目录：`${DSH_HOME:-~/.dsh}/storages/dsh-agentic-router/`（`AGENTIC_ROUTER_HOME` 可覆盖）
 
 - `decisions.jsonl` — 每条决策：任务类型/复杂度/四专家推荐/元层选择/实际路由/是否切换
-- `rewards.jsonl` — 每条奖励：明细（工具失败/重试/延迟）+ 特征向量，**重启后重放恢复学习状态**
+- `rewards.jsonl` — 每条奖励：明细（工具失败/重试/延迟）+ 特征向量，**重启后重放恢复学习状态**；人类反馈结算（`feedback-settle` 行）在重放时精确修正
+- 人类反馈：回合结束 45s 后查询 `messageFeedback`（Web UI 的 👍/👎），点踩 −0.5、点赞 +0.1，修正已应用的奖励（UCB 均值/EXP3 权重精确回退）
 - `policy.json` — 模式与强制模型（原子写）
 
-奖励公式：干净收尾 1 分起，工具失败 −0.2/次（封顶 0.6）、模型重试 −0.2/次（封顶 0.4）、延迟 >30s −0.1 / >90s −0.3；出错回合记 0。
+奖励公式：干净收尾 1 分起，工具失败 −0.2/次（封顶 0.6）、模型重试 −0.2/次（封顶 0.4）、
+**成本** min(0.5, 20×回合成本元)、延迟 >30s −0.1 / >90s −0.3；出错回合记 0。
+
+**成本信号（token 计费）**：透传 `llm/stream`，累加每个模型调用的 `usage` chunk
+（输入/输出/缓存读/推理 token）；无 usage 的 provider 按文本长度估算（标 `estimated`）。
+
+**价格表三级解析**（元/百万 token）：`provider→model` 精确价 > `provider→档位` 价 >
+`*` 通用档位 > 保守兜底。**未知 provider 绝不套用 DeepSeek 价格**，用通用档位并标记
+`pricingSource`（审计字段）。默认含 DeepSeek 官方口径，接入其他模型用
+`agentic_router_set_prices` 工具配价（持久化到 policy.json），或安装时传 `config.prices`：
+
+```js
+{ 'deepseek-official': { 'deepseek-v4-pro': { in: 1, out: 12 } },
+  openai: { 'gpt-x-pro': { in: 15, out: 60 } },
+  '*': { fast: { in: 1, out: 4 }, mid: { in: 2, out: 8 }, strong: { in: 3, out: 16 } } }
+```
 
 ## 分类器（规则先验，<1ms）
 
@@ -59,7 +76,8 @@ dsh plugin --profile web add dsh-agentic-router
 ## 已知边界
 
 - 模型档位靠 id 关键词识别（flash→fast、pro→strong）；可用模型多于一档时路由才真正切换
-- 奖励信号是"代理"：任务成功但质量平庸的情况（尤其模型池内质量差距大时）仍可能被记满分，正式场景建议接入 UI 反馈（messageFeedback）
+- 反馈结算窗口 45s（可配置 `settleDelayMs`），窗口内无反馈按代理奖励结算；重启会丢失未结算的 pending 条目；晚到的反馈可能归属到其后的回合
+- 反馈依赖部署的 Web UI 实际产生数据（本插件只读取，不写入）
 - 毕业策略（LinUCB/k-NN）需要数十个回合样本才会进元层池
 
 ## 开发

package/dsh/index.js

@@ -15,7 +15,8 @@ import { classify, featuresOf } from './classify.js'
 import {
   tierOf, tierTarget, resolveTier,
   recommendRule, recommendUcb, recommendLin, recommendKnn,
-  rewardOf, applyReward,
+  rewardOf, applyReward, correctReward,
+  DEFAULT_PRICES, turnCostYuan, resolvePrice,
 } from './strategies.js'
 import { metaPick, DEFAULT_WEIGHTS } from './meta.js'
 import { Store, defaultHome } from './store.js'
@@ -25,6 +26,8 @@ export const inject = ['tools']
 
 const LIN_GRADUATION = 50   // LinUCB 毕业样本线
 const KNN_GRADUATION = 30   // k-NN 毕业经验线
+const SETTLE_DELAY_MS = 45000 // 反馈结算窗口：回合结束 45s 后查询 messageFeedback
+const SWEEP_MS = 15000        // 结算轮询周期
 
 function freshState() {
   return {
@@ -36,6 +39,8 @@ function freshState() {
     modelTiers: null,
     turnDecision: {},
     openTurns: new Map(),
+    pendingFeedback: [],
+    recByTurn: new Map(),
     ucb: { total: 0, arms: {} },
     lin: {
       total: 0,
@@ -124,24 +129,86 @@ async function refreshModelTiers(llm) {
 export function apply(ctx, config = {}) {
   const tools = ctx.get('tools')
   const llm = ctx.get('llm')
+  const mf = ctx.get('messageFeedback')
+  const timer = ctx.get('timer')
   const store = new Store(config.home || defaultHome())
+  const settleDelayMs = typeof config.settleDelayMs === 'number' ? config.settleDelayMs : SETTLE_DELAY_MS
   const policy = store.loadPolicy()
+  let prices = (policy.prices && Object.keys(policy.prices).length) ? policy.prices : (config.prices || DEFAULT_PRICES)
+  const costWeight = typeof config.costWeight === 'number' ? config.costWeight : 20
+  const costCap = typeof config.costCap === 'number' ? config.costCap : 0.5
 
   const state = freshState()
   state.mode = policy.mode || config.mode || 'shadow'
   if (typeof policy.forced === 'string') state.forced = policy.forced
 
-  // 飞轮恢复：重放 rewards.jsonl 重建 UCB/LinUCB/k-NN/EXP3
+  // 飞轮恢复：重放 rewards.jsonl 重建 UCB/LinUCB/k-NN/EXP3；
+  // feedback-settle 行在基础奖励重放后按 turn 精确修正
   const rewards = store.loadJsonl('rewards.jsonl')
+  const byTurn = new Map()
   for (const rec of rewards) {
-    if (typeof rec.reward === 'number' && Array.isArray(rec.f)) applyReward(state, rec)
+    if (rec.type === 'feedback-settle') continue
+    if (typeof rec.reward === 'number' && Array.isArray(rec.f)) {
+      applyReward(state, rec)
+      byTurn.set(rec.turn, rec)
+    }
+  }
+  for (const rec of rewards) {
+    if (rec.type !== 'feedback-settle') continue
+    const baseRec = byTurn.get(rec.turn)
+    if (baseRec) correctReward(state, baseRec, rec.newReward)
   }
+  state.recByTurn = byTurn
   state.decisions = store.loadJsonl('decisions.jsonl').slice(-60)
   state.seq = state.decisions.length ? state.decisions[state.decisions.length - 1].seq : 0
 
   const log = (msg) => console.log('[agentic-router]', msg)
   if (rewards.length) log('replayed ' + rewards.length + ' rewards; ucb.total=' + state.ucb.total)
 
+  let mfWarned = false
+  // 反馈结算：对已过结算窗口的回合查询 messageFeedback，按人类 👍/👎 修正奖励
+  async function settleFeedback() {
+    if (!mf || typeof mf.list !== 'function') {
+      if (!mfWarned) { log('messageFeedback service absent — 反馈奖励停用'); mfWarned = true }
+      return
+    }
+    const tNow = now()
+    if (tNow == null) return
+    const remaining = []
+    for (const p of state.pendingFeedback) {
+      if (tNow - p.t0 < settleDelayMs) { remaining.push(p); continue }
+      try {
+        const res = await mf.list({ sessionId: p.sessionId })
+        let neg = 0
+        let pos = 0
+        if (res && res.ok === true && res.value && Array.isArray(res.value.items)) {
+          for (const it of res.value.items) {
+            if (typeof it.createdAt !== 'number' || it.createdAt < p.t0) continue
+            if (it.rating === 'negative') neg += 1
+            else if (it.rating === 'positive') pos += 1
+          }
+        }
+        const newReward = Math.max(0, Math.min(1, Math.round((p.appliedReward - 0.5 * neg + 0.1 * pos) * 100) / 100))
+        if (newReward !== p.appliedReward) {
+          const baseRec = state.recByTurn.get(p.key)
+          if (baseRec) correctReward(state, baseRec, newReward)
+          store.appendJsonl('rewards.jsonl', {
+            type: 'feedback-settle', turn: p.key, sessionId: p.sessionId,
+            negative: neg, positive: pos, applied: p.appliedReward, newReward,
+          })
+          log('feedback settle ' + p.key + ': ' + p.appliedReward + ' -> ' + newReward + ' (neg=' + neg + ' pos=' + pos + ')')
+        }
+      } catch (e) {
+        log('feedback settle failed: ' + String(e))
+      }
+    }
+    state.pendingFeedback = remaining
+  }
+
+  if (timer && typeof timer.interval === 'function') {
+    timer.interval(() => { settleFeedback().catch((e) => log('settle sweep failed: ' + String(e))) }, SWEEP_MS)
+  }
+
   // ---- 事件挂载 ----
   ctx.on('agent/pre-step', (payload, next) => {
     try {
@@ -157,6 +224,7 @@ export function apply(ctx, config = {}) {
     const cfg = await next()
     if (!cfg || typeof cfg !== 'object') return cfg
     try {
+      if (!timer || typeof timer.interval !== 'function') settleFeedback().catch(() => {})
       const key = fullKey(payload)
       if (state.turnDecision[key]) return cfg
       const pkey = agentKey(payload) + ':' + payload.turn
@@ -209,6 +277,8 @@ export function apply(ctx, config = {}) {
       state.turnDecision[key] = {
         clsType: cls.type, f, recs, meta, chosen: meta.chosen,
         finalized: false, t0: now(), toolErrors: 0, retries: 0,
+        currentModel: current, provider, costWeight, costCap,
+        estInputTokens: Math.ceil(text.length / 1.5),
       }
       state.openTurns.set(agentKey(payload), key)
       store.appendJsonl('decisions.jsonl', record)
@@ -242,6 +312,52 @@ export function apply(ctx, config = {}) {
     return next()
   })
 
+  // 成本信号：透传每次流式模型调用，累加 usage chunk 到当前 open turn；
+  // 无 usage 的 provider 用文本长度估算（estimated=true）
+  ctx.on('llm/stream', function (options, next) {
+    return (async function* () {
+      let usage = null
+      let outTextLen = 0
+      try {
+        for await (const chunk of next()) {
+          if (chunk && chunk.type === 'usage' && chunk.usage) usage = chunk.usage
+          if (chunk && chunk.type === 'text-delta' && typeof chunk.text === 'string') outTextLen += chunk.text.length
+          yield chunk
+        }
+      } finally {
+        try {
+          const sid = options && (options.sessionId || (options.session && options.session.id))
+          const key = sid ? state.openTurns.get(sid) : null
+          const td = key ? state.turnDecision[key] : null
+          if (td && !td.finalized) {
+            const u = td.usage || { inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, reasoningTokens: 0, estimated: false }
+            if (usage && typeof usage.inputTokens === 'number') {
+              u.inputTokens += usage.inputTokens
+              u.outputTokens += usage.outputTokens || 0
+              u.cacheReadTokens += usage.cacheReadTokens || 0
+              u.reasoningTokens += usage.reasoningTokens || 0
+              u.estimated = false
+            } else {
+              u.inputTokens += td.estInputTokens || 0
+              u.outputTokens += Math.round(outTextLen / 2)
+              u.estimated = true
+            }
+            td.usage = u
+            const model = typeof td.currentModel === 'string' ? td.currentModel : ''
+            const tier = tierOf(model)
+            const provider = (options && typeof options.provider === 'string' && options.provider) || td.provider || ''
+            const rp = resolvePrice(provider, model, tier, prices)
+            td.costYuan = turnCostYuan(u, provider, model, tier, prices)
+            td.pricingSource = rp.source
+            log('usage on ' + key + ': in=' + u.inputTokens + ' out=' + u.outputTokens + (u.estimated ? ' (估)' : '') + ' cost=¥' + td.costYuan.toFixed(4) + ' [' + rp.source + ']')
+          }
+        } catch (e) {
+          log('usage hook failed: ' + String(e))
+        }
+      }
+    })()
+  })
+
   ctx.on('tools/result', (exec, result) => {
     try {
       if (!isErrorResult(result)) return
@@ -283,6 +399,15 @@ export function apply(ctx, config = {}) {
       }
       applyReward(state, rec)
       store.appendJsonl('rewards.jsonl', rec)
+      state.recByTurn.set(key, rec)
+      if (state.recByTurn.size > 500) {
+        const first = state.recByTurn.keys().next().value
+        if (first !== undefined) state.recByTurn.delete(first)
+      }
+      if (mf && typeof mf.list === 'function' && td.t0 != null) {
+        state.pendingFeedback.push({ key, sessionId: agentKey(payload), t0: td.t0, appliedReward: rec.reward })
+        if (state.pendingFeedback.length > 200) state.pendingFeedback.shift()
+      }
       const ak = agentKey(payload)
       if (state.openTurns.get(ak) === key) state.openTurns.delete(ak)
       log('reward ' + rec.reward + ' ' + tag + ' on ' + key + ' breakdown=' + JSON.stringify(r.breakdown))
@@ -325,11 +450,14 @@ export function apply(ctx, config = {}) {
         forcedModel: state.forced,
         totalDecisions: state.seq,
         modelTiers: state.modelTiers,
+        prices,
         flywheel: {
           ucb: state.ucb,
           lin: { total: state.lin.total, tierN: state.lin.tierN, tierW: state.lin.tierW },
           knnPoolSize: state.knn.pool.length,
           exp3: state.exp3,
+          pendingFeedback: state.pendingFeedback.length,
+          prices,
           recentRewards: store.loadJsonl('rewards.jsonl').slice(-20),
         },
         recent: state.decisions.slice(-20),
@@ -348,7 +476,7 @@ export function apply(ctx, config = {}) {
     isConcurrencySafe: () => true,
     async execute(args) {
       state.mode = args.mode
-      store.savePolicy({ mode: state.mode, forced: state.forced })
+      store.savePolicy({ mode: state.mode, forced: state.forced, prices })
       return { mode: state.mode }
     },
   })
@@ -367,17 +495,45 @@ export function apply(ctx, config = {}) {
     async execute(args) {
       if (args && args.clear) {
         state.forced = null
-        store.savePolicy({ mode: state.mode, forced: null })
+        store.savePolicy({ mode: state.mode, forced: null, prices })
         return { forced: null }
       }
       if (args && typeof args.model === 'string') {
         state.forced = args.model
-        store.savePolicy({ mode: state.mode, forced: args.model })
+        store.savePolicy({ mode: state.mode, forced: args.model, prices })
         return { forced: args.model }
       }
       return { forced: state.forced, error: '需要 model 或 clear=true' }
     },
   })
+  register({
+    name: 'agentic_router_set_prices',
+    description: '配置/查询某 provider 某模型（或档位）的价格（元/百万 tokens，{in, out}）。不带 priceIn/priceOut 时返回当前价格表。接入新模型后用它补价格，未知价格按保守值估算并标记 unknown。',
+    parameters: {
+      type: 'object',
+      properties: {
+        provider: { type: 'string', description: 'provider id，如 openai、anthropic、deepseek-official' },
+        model: { type: 'string', description: '可选：精确模型 id；省略则设置该 provider 的档位价' },
+        tier: { type: 'string', enum: ['fast', 'mid', 'strong'], description: '档位（未给 model 时必填）' },
+        priceIn: { type: 'number', description: '输入价（元/百万 tokens），负数视为查询模式' },
+        priceOut: { type: 'number', description: '输出价（元/百万 tokens），负数视为查询模式' },
+      },
+      required: ['provider'],
+    },
+    output: OUT('agentic router 价格表'),
+    isConcurrencySafe: () => true,
+    async execute(args) {
+      if (typeof args.priceIn !== 'number' || typeof args.priceOut !== 'number' || args.priceIn < 0 || args.priceOut < 0) {
+        return { prices, note: '查看模式：传 priceIn/priceOut 与 model 或 tier 进行设置' }
+      }
+      const byProvider = prices[args.provider] || (prices[args.provider] = {})
+      if (args.model) byProvider[args.model] = { in: args.priceIn, out: args.priceOut }
+      else if (args.tier) byProvider[args.tier] = { in: args.priceIn, out: args.priceOut }
+      else return { error: '需要 model 或 tier', prices }
+      store.savePolicy({ mode: state.mode, forced: state.forced, prices })
+      return { prices, updated: args.model || (args.provider + '/' + args.tier) }
+    },
+  })
   register({
     name: 'agentic_router_reset',
     description: '清空飞轮学习状态（决策与奖励历史），保留当前模式；用于重新开始学习或清理测试数据。',

package/dsh/strategies.js

@@ -105,28 +105,76 @@ export function recommendKnn(cls, f, knnState) {
   return { tier: best.tier, reason: 'knn k=' + k + ' neighbors score=' + Math.round(best.score * 100) / 100, conf: Math.min(0.8, pool.length / 100) }
 }
 
-// 质量代理奖励：干净收尾 1 分起，扣工具失败(0.2/次，封顶 0.6)、模型重试(0.2/次，封顶 0.4)、延迟(>90s −0.3, >30s −0.1)
+// ---- 成本信号（token 计费） ----
+// 价格表（元/百万 tokens）三级解析：
+//   1) prices[provider][model]          精确模型价（最高优先）
+//   2) prices[provider][tier]           该 provider 的档位价
+//   3) prices['*'][tier]                通用档位兜底（未知 provider 不套用 DeepSeek 价）
+//   4) CONSERVATIVE                     保守值 + 标记 unknown，提醒用户补价格
+export const DEFAULT_PRICES = {
+  'deepseek-official': {
+    'deepseek-v4-flash': { in: 0.2, out: 2 },
+    'deepseek-v4-pro': { in: 1, out: 12 },
+  },
+  '*': {
+    fast: { in: 1, out: 4 },
+    mid: { in: 2, out: 8 },
+    strong: { in: 3, out: 16 },
+  },
+}
+
+const CONSERVATIVE = { in: 3, out: 16 }
+
+// 解析价格并返回来源（透明标注：价格到底从哪来的）
+export function resolvePrice(provider, model, tier, prices) {
+  const table = prices || DEFAULT_PRICES
+  const byProvider = table[provider]
+  if (byProvider && byProvider[model]) return { price: byProvider[model], source: 'provider-model' }
+  if (byProvider && byProvider[tier]) return { price: byProvider[tier], source: 'provider-tier' }
+  const generic = table['*']
+  if (generic && generic[tier]) return { price: generic[tier], source: 'generic-tier' }
+  return { price: CONSERVATIVE, source: 'unknown-conservative' }
+}
+
+// 单回合成本（元）：缓存读计入输入，推理 token 计入输出
+export function turnCostYuan(usage, provider, model, tier, prices) {
+  if (!usage) return 0
+  const { price } = resolvePrice(provider, model, tier, prices)
+  const input = (usage.inputTokens || 0) + (usage.cacheReadTokens || 0)
+  const output = (usage.outputTokens || 0) + (usage.reasoningTokens || 0)
+  return (price.in * input + price.out * output) / 1e6
+}
+
+// 质量代理奖励：干净收尾 1 分起，扣工具失败(0.2/次，封顶 0.6)、模型重试(0.2/次，封顶 0.4)、
+// 延迟(>90s −0.3, >30s −0.1)、成本(min(cap, weight × 元)，默认 weight=20 即每 0.01 元 −0.2)
 export function rewardOf(td, tag, dtMs) {
   if (tag !== 'clean-close') {
-    return { reward: 0, breakdown: { base: 0, toolErrors: td.toolErrors || 0, retries: td.retries || 0, dtMs } }
+    return { reward: 0, breakdown: { base: 0, toolErrors: td.toolErrors || 0, retries: td.retries || 0, dtMs, costYuan: td.costYuan || 0, costPenalty: 0, pricingSource: td.pricingSource || 'n/a' } }
   }
   let reward = 1
   const toolPenalty = Math.min(0.6, 0.2 * (td.toolErrors || 0))
   const retryPenalty = Math.min(0.4, 0.2 * (td.retries || 0))
+  const costCap = typeof td.costCap === 'number' ? td.costCap : 0.5
+  const costWeight = typeof td.costWeight === 'number' ? td.costWeight : 20
+  const costYuan = typeof td.costYuan === 'number' ? td.costYuan : 0
+  const costPenalty = Math.min(costCap, costWeight * costYuan)
   reward -= toolPenalty
   reward -= retryPenalty
+  reward -= costPenalty
   if (dtMs != null) {
     if (dtMs > 90000) reward -= 0.3
     else if (dtMs > 30000) reward -= 0.1
   }
   return {
     reward: Math.max(0, Math.round(reward * 100) / 100),
-    breakdown: { base: 1, toolErrors: td.toolErrors || 0, toolPenalty, retries: td.retries || 0, retryPenalty, dtMs },
+    breakdown: { base: 1, toolErrors: td.toolErrors || 0, toolPenalty, retries: td.retries || 0, retryPenalty, costYuan, costPenalty, dtMs, pricingSource: td.pricingSource || 'n/a' },
   }
 }
 
 // 把一条奖励记录应用到飞轮（UCB/LinUCB/k-NN/EXP3），replay 与实时更新共用
 export function applyReward(state, rec) {
+  // 记录本次线性更新的学习率，供反馈结算修正精确回退
+  rec.linLr = 1 / (1 + (state.lin.tierN[rec.linTier] || 0))
   const armKey = rec.clsType + ':' + rec.ucbTier
   const arm = state.ucb.arms[armKey] || (state.ucb.arms[armKey] = { n: 0, mean: 0 })
   arm.n += 1
@@ -151,3 +199,30 @@ export function applyReward(state, rec) {
   }
   state.exp3.total += 1
 }
+
+// 反馈结算修正：把已应用奖励改为 newReward（人类在反馈窗口内点了 👍/👎 之后）
+// UCB 均值修正（n 不变）；LinUCB 用原始学习率做精确梯度回退；k-NN 追加修正经验；
+// EXP3 按乘性更新做精确逆修正。
+export function correctReward(state, rec, newReward) {
+  const delta = newReward - rec.reward
+  if (!delta) return { delta: 0 }
+
+  const arm = state.ucb.arms[rec.clsType + ':' + rec.ucbTier]
+  if (arm && arm.n > 0) arm.mean = arm.mean + delta / arm.n
+
+  const lw = state.lin.tierW[rec.linTier]
+  if (lw && Array.isArray(rec.f)) {
+    const lr = typeof rec.linLr === 'number' ? rec.linLr : 1 / (1 + (state.lin.tierN[rec.linTier] || 0))
+    for (let i = 0; i < rec.f.length; i++) lw[i] = (lw[i] || 0) + lr * delta * rec.f[i]
+  }
+
+  if (state.knn && Array.isArray(rec.f)) state.knn.pool.push({ f: rec.f, tier: rec.ucbTier, r: newReward })
+
+  if (rec.pChosen > 0 && rec.pool && rec.pool.length > 0) {
+    state.exp3.weights[rec.chosen] = (state.exp3.weights[rec.chosen] || 0) *
+      Math.exp(0.3 * delta / (rec.pChosen * rec.pool.length))
+  }
+
+  rec.reward = newReward
+  return { delta }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-agentic-router",
-  "version": "1.0.0",
+  "version": "1.3.0",
   "description": "DeepSeek Harness plugin: learning agentic router — rule + UCB + LinUCB + k-NN experts under an EXP3 meta-selector, with a durable quality-reward flywheel",
   "type": "module",
   "license": "MIT",
@@ -33,7 +33,7 @@
     "LICENSE"
   ],
   "scripts": {
-    "test": "node --test test/"
+    "test": "node --test"
   },
   "dsh": {
     "bundle": {