dsh-recent-usage-metrics 0.3.6 → 0.4.0

package/CHANGELOG.md

@@ -1,3 +1,23 @@
+## 0.4.0
+
+把「模型切换」与「请求配置变更」分开——它们此前被当成同一件事，而后者远比前者常见。
+
+- 判定改为：窗口内有 `request/header` **且本步模型与上一步不同**才算切换模型；上一步
+  模型未知时以 header 为准。实测一个真实会话：27 个 header 里只有 7 次是真正的模型切换，
+  其余 20 次是工具集变化（启用或禁用插件）。旧规则把这 20 次也标成切换模型，是错的。
+- 新增「配置变更」信号：有 header 但模型未变。它的可见后果就是缓存变冷，所以显示在黄色
+  那一行里。
+- 序号列现在会在高亮行上给出说明：切换模型时写「该行切换模型至：X」；缓存过低时写
+  「该行请求配置变更（工具集 66→65）」加「缓存命中 0.4%，低于 90%」。普通行的序号没有提示。
+- 两个工具数只有都已知时才写成区间，否则只说配置发生变化，不编造数字。
+
+## 0.3.7
+
+- `#` 表头的提示拆成三行：最近记录、三个状态图标、两种行颜色。原来挤成一句，
+  三件事连在一起读起来费劲。
+- 实时行不再显示总用时的工具提示。还在跑的请求尚未结算工具时间，提示只能给出
+  「无工具调用」或一个不完整的数，对一个仍在进行的请求都是错的；行结算后再出现。
+
 ## 0.3.6
 
 - `#` 表头的提示加了一行颜色说明：黄色表示缓存命中过低，蓝色表示这一步切换了模型。

package/README.en.md

@@ -165,6 +165,13 @@ Every drawer column and every header carries a hint:
 When both hold, the text takes the warn colour and the bar keeps the business colour. A
 compaction row never takes the cache highlight.
 
+**A model switch and a configuration change are not the same thing.** A `request/header` only says
+the routed configuration changed — enabling or disabling a plugin rewrites the tool list and writes
+a header without touching the model. On a real session, only 7 of 27 headers were genuine model
+switches. Blue therefore appears only when the model itself differs; a configuration change without
+a model change shows up as a cold cache, coloured yellow, with the tool-list movement named in the
+row-number hint.
+
 **Where the model name comes from:** a `request/header` inside the step's window means a switch
 and supplies the model; otherwise the previous step's model carries over, and a landed reply
 overrides both via `assistant/message.source`. The live row follows the same rule, so the model

package/README.md

@@ -148,6 +148,11 @@ DSH 的极客级请求指标。它在 **设置 → 通用 → 性能与用量**
 
 两者同时成立时文字取警告色、色条保留业务蓝。压缩行不参与缓存高亮。
 
+**「切换模型」与「配置变更」不是一回事。** 一个请求会写出 `request/header`，只说明路由配置
+变了——启用或禁用插件会改变工具集，同样会写 header，但模型没换。实测一个真实会话：27 个
+header 里只有 7 次是真正的模型切换。所以蓝色只在模型确实不同时出现；模型没变但配置变了，
+它的后果就是缓存变冷，显示为黄色，并在序号提示里写明工具集的变化。
+
 **模型名的来源：** 本步窗口内的 `request/header` 判为切换，否则沿用上一步；回复落地后
 由 `assistant/message.source` 覆盖。实时行同样适用，所以生成期间就能看到模型名。
 

package/client/client.js

@@ -114,9 +114,19 @@ window.__ModuleLoader__.load({
       hoursAgo: '{count} 小时前',
       // Column-header hints. `{bolt}` / `{tools}` / `{compact}` are replaced by the glyphs
       // themselves, which is why these are rendered by the tooltip component and not by title.
-      hintIndex:
-        '行号：1–10 为已完成的最近记录，{bolt} 为模型输出中，{tools} 为工具调用中，{compact} 为压缩上下文中',
+      /*
+       * The index hint is three lines: what the column holds, what the glyphs mean, then what
+       * the row colours mean. Splitting them keeps each line short enough to read at a glance
+       * instead of being one long sentence the eye has to parse.
+       */
+      hintIndexRows: '行号：1–10 为已完成的最近记录',
+      hintIndexGlyphs: '{bolt} 为模型输出中，{tools} 为工具调用中，{compact} 为压缩上下文中',
       hintColour: '整行黄色表示缓存命中过低，蓝色表示这一步切换了模型',
+      /* The row-number hint: only a highlighted row carries one, and it says why in numbers. */
+      rowSwitched: '该行切换模型至：{model}',
+      rowLowCache: '缓存命中 {rate}，低于 {threshold}%',
+      rowConfigChanged: '该行请求配置变更（工具集 {from}→{to}）',
+      rowConfigChangedUnknown: '该行请求配置变更，缓存前缀失效',
       hintTime: '请求发出的时刻（本地时间）',
       hintTtft: '从发出到收到第一个 token 的耗时',
       hintSpeed: '模型生成速度 = 输出 token ÷ 解码时长',
@@ -178,9 +188,13 @@ window.__ModuleLoader__.load({
       secondsAgo: '{count}s ago',
       minutesAgo: '{count}m ago',
       hoursAgo: '{count}h ago',
-      hintIndex:
-        'Row number: 1–10 are the most recent finished records; {bolt} is a model reply, {tools} is a tool call, {compact} is a context compaction',
+      hintIndexRows: 'Row number: 1–10 are the most recent finished records',
+      hintIndexGlyphs: '{bolt} is a model reply, {tools} is a tool call, {compact} is a context compaction',
       hintColour: 'A yellow row means a low cache hit; a blue row means this step switched models',
+      rowSwitched: 'This row switched to {model}',
+      rowLowCache: 'Cache hit {rate}, below {threshold}%',
+      rowConfigChanged: 'Request configuration changed on this row (tools {from}→{to})',
+      rowConfigChangedUnknown: 'Request configuration changed on this row; the cached prefix was lost',
       hintTime: 'When the request was issued (local time)',
       hintTtft: 'Time from issuing the request to receiving its first token',
       hintSpeed: 'Generation speed = output tokens ÷ decode time',
@@ -1301,27 +1315,36 @@ window.__ModuleLoader__.load({
     }
 
     /**
-     * Which model a step used, and whether that request changed models.
+     * Which model a step used, whether that request changed models, and whether it changed the
+     * routed configuration at all.
      *
-     * A `request/header` is written only when the routed model or tool set CHANGES, so its
-     * placement is itself the signal:
+     * A `request/header` is written when the routed configuration changes, and that includes far
+     * more than the model: enabling or disabling a plugin changes the tool list, which rewrites
+     * the request prefix and invalidates the provider's cache. Measured on a real session, 27
+     * headers covered only 7 genuine model switches, so "a header exists" is NOT "the model
+     * changed" — treating the two as one flagged every plugin toggle as a model switch.
      *
-     * - inside a step's own seq window (from its `step/start` to the next one) → this request
-     *   changed the routed configuration, and the header names the model it switched to;
-     * - absent → the step reused whatever was already routed, so it inherits the previous
-     *   step's model.
+     * The three signals are therefore separated:
      *
-     * Measured on a real 3202-step session: 22 steps carried a header and all 22 were genuine
-     * switches; 3180 inherited, of which every one with a known predecessor matched the model
-     * the reply later reported. The only gaps are a turn's first step with no header, where
-     * nothing earlier is known and the model stays null.
-     *
-     * A finished reply overrides both, because `assistant/message.source` is the authority.
+     * - `switched` — a header landed in the step's window AND the model differs from the one
+     *   before it. When the previous model is unknown the header is trusted, because there is
+     *   nothing to contradict it.
+     * - `configChanged` — a header landed in the window but the model did NOT change, so the
+     *   change was elsewhere in the configuration (the tool list). Its visible consequence is a
+     *   cold cache, which is why it is reported beside the low-cache highlight.
+     * - `toolCount` / `previousToolCount` — the tool-list sizes the two headers carried, so the
+     *   hint can say what actually moved rather than gesturing at "the configuration".
      */
     function modelPlanFromStats(statsSnapshot, known) {
       const plan = new Map()
       const turns = statsSnapshot?.turns
       if (!Array.isArray(turns)) return plan
+      /*
+       * The plan walks the session's steps in time order across turns, not turn by turn, because
+       * "the previous model" has to compare against the step before it even when that step lives
+       * in an earlier turn — a turn's first step is exactly where a switch is most likely.
+       */
+      const sequence = []
       for (const turn of turns) {
         if (!Array.isArray(turn.steps)) continue
         const headers = []
@@ -1331,62 +1354,98 @@ window.__ModuleLoader__.load({
             const at = numberOrNull(event.time)
             if (at === null) continue
             if (typeof event.model !== 'string' && typeof event.provider !== 'string') continue
-            headers.push({ time: at, provider: event.provider ?? null, model: event.model ?? null })
+            headers.push({
+              time: at,
+              provider: event.provider ?? null,
+              model: event.model ?? null,
+              tools: numberOrNull(event.toolCount),
+            })
           }
           headers.sort((left, right) => left.time - right.time)
         }
-        const starts = []
+        const steps = []
         for (const step of turn.steps) {
           const at = numberOrNull(step.startTime)
           if (at === null || typeof step.step !== 'number') continue
-          starts.push({ at, step: step.step })
+          steps.push({ at, step: step.step })
+        }
+        steps.sort((left, right) => left.at - right.at)
+        for (const step of steps) sequence.push({ turn: turn.turn, step: step.step, at: step.at, headers })
+      }
+      sequence.sort((left, right) => left.at - right.at)
+
+      /** The model of the step before this one, or null when it is not known. */
+      let previousModel = null
+      let previousTools = null
+      let inForce = null
+
+      for (let index = 0; index < sequence.length; index += 1) {
+        const entry = sequence[index]
+        const key = `${entry.turn}:${entry.step}`
+        const nextAt = index + 1 < sequence.length ? sequence[index + 1].at : Number.POSITIVE_INFINITY
+        let inside = null
+        for (const header of entry.headers) {
+          if (header.time < entry.at) continue
+          if (header.time >= nextAt) break
+          inside = header
+        }
+
+        const settled = known === undefined ? undefined : known.get(key)
+        const settledModel =
+          settled !== undefined && settled.model !== null
+            ? { provider: settled.provider, model: settled.model }
+            : null
+
+        if (inside !== null) {
+          const headerModel = { provider: inside.provider, model: inside.model }
+          const changed =
+            previousModel === null ||
+            headerModel.model === null ||
+            previousModel.model !== headerModel.model ||
+            previousModel.provider !== headerModel.provider
+          plan.set(key, {
+            provider: inside.provider,
+            model: inside.model,
+            /*
+             * A header alone proves only that the routed configuration changed. Saying "the model
+             * switched" additionally requires that the model itself differs — and when the
+             * previous model is unknown, the header is the only evidence, so it is trusted.
+             */
+            switched: changed,
+            configChanged: !changed,
+            toolCount: inside.tools,
+            previousToolCount: previousTools,
+            source: 'header',
+          })
+          inForce = headerModel
+          previousModel = headerModel
+          previousTools = inside.tools
+          continue
         }
-        if (starts.length === 0) continue
-        starts.sort((left, right) => left.at - right.at)
 
         /*
-         * The model currently in force, carried forward while the plan is built.
-         *
-         * A step whose header names a switch becomes the new carrier, so the step AFTER it
-         * inherits the switched-to model — not the one before the switch, which `known` alone
-         * would give.
+         * No header: the step reused what was already routed. A finished reply outranks the
+         * carried model, because `assistant/message.source` is the authority for a settled step.
          */
-        let inForce = null
-
-        for (let i = 0; i < starts.length; i += 1) {
-          const key = `${turn.turn}:${starts[i].step}`
-          const from = starts[i].at
-          const until = i + 1 < starts.length ? starts[i + 1].at : Number.POSITIVE_INFINITY
-          let inside = null
-          for (const header of headers) {
-            if (header.time < from) continue
-            if (header.time >= until) break
-            inside = header
-          }
-          if (inside !== null) {
-            // The header was written for THIS request, so the model changed here.
-            inForce = { provider: inside.provider, model: inside.model }
-            plan.set(key, { provider: inside.provider, model: inside.model, switched: true, source: 'header' })
-            continue
-          }
-          /*
-           * No header: inherit. A finished reply outranks the carried model, because
-           * `assistant/message.source` is the authority for a step that has settled.
-           */
-          const settled = known === undefined ? undefined : known.get(key)
-          if (settled !== undefined && settled.model !== null) {
-            inForce = { provider: settled.provider, model: settled.model }
-            plan.set(key, { provider: settled.provider, model: settled.model, switched: false, source: 'reply' })
-            continue
-          }
-          if (inForce !== null) {
-            plan.set(key, { provider: inForce.provider, model: inForce.model, switched: false, source: 'inherit' })
-          }
+        const resolved = settledModel ?? inForce
+        if (resolved !== null) {
+          plan.set(key, {
+            provider: resolved.provider,
+            model: resolved.model,
+            switched: false,
+            configChanged: false,
+            toolCount: null,
+            previousToolCount: null,
+            source: settledModel !== null ? 'reply' : 'inherit',
+          })
+          inForce = resolved
+          previousModel = resolved
         }
       }
       return plan
     }
 
+
     /**
      * Whether any step started after this compaction finished.
      *
@@ -1459,6 +1518,14 @@ window.__ModuleLoader__.load({
           row.model = owner.model
           row.provider = owner.provider
           row.switched = owner.switched === true
+          /*
+           * A configuration change that was not a model switch still rewrote the request prefix,
+           * so it is worth naming when the cache came back cold: that is the difference between
+           * "the account changed" and "the tool list moved".
+           */
+          row.configChanged = owner.configChanged === true
+          row.toolCount = owner.toolCount ?? null
+          row.previousToolCount = owner.previousToolCount ?? null
         }
         rows.push(row)
       }
@@ -1879,6 +1946,47 @@ window.__ModuleLoader__.load({
       )
     }
 
+    /**
+     * The row-number hint, or an empty list when the row is not highlighted.
+     *
+     * A highlighted row is worth explaining, and the number column is the one place that has room
+     * without competing with a figure. Two independent reasons can apply, so the result is a list
+     * of lines rather than a sentence:
+     *
+     * - the model switched, named outright;
+     * - the cache came back cold, with the rate and the threshold it fell below.
+     *
+     * A cold row that ALSO changed its routed configuration names the tool-list movement, because
+     * that is the usual cause of a lost prefix and the difference between "the account rotated"
+     * and "a plugin was enabled". When the two tool counts are not both known, the line says only
+     * that the configuration changed rather than inventing numbers.
+     */
+    function rowHintLines(row, t) {
+      const lines = []
+      if (row.compaction === true) return lines
+      if (row.switched === true) {
+        lines.push(t('rowSwitched', { model: modelHintText(row, t) }))
+      }
+      if (isLowCacheHit(row)) {
+        if (row.configChanged === true) {
+          const from = numberOrNull(row.previousToolCount)
+          const to = numberOrNull(row.toolCount)
+          lines.push(
+            from !== null && to !== null
+              ? t('rowConfigChanged', { from, to })
+              : t('rowConfigChangedUnknown'),
+          )
+        }
+        lines.push(
+          t('rowLowCache', {
+            rate: `${(row.cacheHit * 100).toFixed(1)}%`,
+            threshold: Math.round(LOW_CACHE_HIT * 100),
+          }),
+        )
+      }
+      return lines
+    }
+
     /**
      * The speed column's hint: which model served this row.
      *
@@ -1961,29 +2069,33 @@ window.__ModuleLoader__.load({
     function indexHint(t) {
       const glyph = (Icon) => h(Icon, { className: 'dsh-rum-tip-icon' })
       /*
-       * Split on every placeholder first, then walk the pieces: a string piece is literal text,
-       * a placeholder becomes its glyph. Walking once over a token list is simpler to follow
-       * than nesting one split per placeholder.
+       * Turn one sentence into elements, replacing each glyph placeholder with the glyph itself.
+       * A single pass over a token list is simpler to follow than nesting one split per
+       * placeholder, and it keeps the placeholders in whatever order a translation uses.
        */
-      const tokens = t('hintIndex').split(/(\{bolt\}|\{tools\}|\{compact\})/)
-      const nodes = []
-      for (let i = 0; i < tokens.length; i += 1) {
-        const token = tokens[i]
-        if (token === '') continue
-        if (token === '{bolt}') nodes.push(glyph(BoltIcon))
-        else if (token === '{tools}') nodes.push(glyph(ToolIcon))
-        else if (token === '{compact}') nodes.push(glyph(CompactionIcon))
-        else nodes.push(token)
+      const withGlyphs = (text) => {
+        const tokens = text.split(/(\{bolt\}|\{tools\}|\{compact\})/)
+        const nodes = []
+        for (const token of tokens) {
+          if (token === '') continue
+          if (token === '{bolt}') nodes.push(glyph(BoltIcon))
+          else if (token === '{tools}') nodes.push(glyph(ToolIcon))
+          else if (token === '{compact}') nodes.push(glyph(CompactionIcon))
+          else nodes.push(token)
+        }
+        return nodes
       }
       /*
-       * The colour legend is its own line, with the two highlights drawn as swatches rather than
-       * named. A reader can then match a row to the legend by colour instead of translating a
-       * word, and the swatches use the same tokens the rows do, so the two cannot drift.
+       * Three lines: what the column holds, what the glyphs mean, then what the row colours mean.
+       * The colour line leads with two swatches instead of naming the hues, so a reader matches a
+       * row against the legend by colour rather than by translating a word. The swatches take the
+       * same tokens the rows take, so the two cannot drift apart.
        */
       return h(
         'span',
         { className: 'dsh-rum-tip-body' },
-        h('span', { className: 'dsh-rum-tip-line' }, ...nodes),
+        h('span', { className: 'dsh-rum-tip-line' }, t('hintIndexRows')),
+        h('span', { className: 'dsh-rum-tip-line' }, ...withGlyphs(t('hintIndexGlyphs'))),
         h(
           'span',
           { className: 'dsh-rum-tip-line' },
@@ -3912,15 +4024,17 @@ window.__ModuleLoader__.load({
                               : speedText(live.speed, live.estimated === true, t),
                           ),
                           h('span', { className: 'dsh-rum-col', role: 'cell' }, tokensText(live.tokens, live.estimated === true, t)),
+                          /*
+                           * No hint on the live row's total.
+                           *
+                           * A running request has not reported its tool time yet, so the only
+                           * possible readings are "no tools" or a partial count — both would be
+                           * wrong about a request that is still going. The hint appears when the
+                           * row settles and the fold has the real figures.
+                           */
                           h(
                             'span',
-                            {
-                              className: 'dsh-rum-col',
-                              role: 'cell',
-                              /* Same hint as the numbered rows, so the meaning never changes. */
-                              onMouseEnter: (event) => hint.show(toolHintText(live, t), event),
-                              onMouseLeave: hint.hide,
-                            },
+                            { className: 'dsh-rum-col', role: 'cell' },
                             secondsText(
                               live.compaction !== null && live.compaction !== undefined
                                 ? compactionSpan(live.compaction, now)
@@ -3961,7 +4075,33 @@ window.__ModuleLoader__.load({
                          */
                         h(
                           'span',
-                          { className: 'dsh-rum-index', role: 'cell' },
+                          {
+                            className: 'dsh-rum-index',
+                            role: 'cell',
+                            /*
+                             * Only a highlighted row explains itself here; an ordinary row's number
+                             * needs no gloss. The handler is attached unconditionally but returns
+                             * nothing to show when the list is empty, so hovering a plain row stays
+                             * silent.
+                             */
+                            onMouseEnter: (event) => {
+                              const lines = rowHintLines(entry.row, t)
+                              if (lines.length === 0) return
+                              hint.show(
+                                lines.length === 1
+                                  ? lines[0]
+                                  : h(
+                                      'span',
+                                      { className: 'dsh-rum-tip-body' },
+                                      ...lines.map((line, index) =>
+                                        h('span', { className: 'dsh-rum-tip-line' }, line),
+                                      ),
+                                    ),
+                                event,
+                              )
+                            },
+                            onMouseLeave: hint.hide,
+                          },
                           entry.row.compaction === true
                             ? h(CompactionIcon, {
                                 className:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-recent-usage-metrics",
-  "version": "0.3.6",
+  "version": "0.4.0",
   "description": "Geek level for the Performance & usage setting: a live reading of the newest request (first-token latency, speed, tokens, total, amortised) plus a drawer listing the last ten, all derived from the shipped Statistics view data",
   "repository": {
     "type": "git",