dsh-all-usage 1.1.8 → 1.1.10

package/CHANGELOG.md

@@ -2,6 +2,34 @@
 
 All notable changes to `dsh-all-usage` are documented here.
 
+## [1.1.10] - 2026-09-14
+
+### Fixed
+
+- **The ledger fast path is back.** DSH 0.1.5 renamed `sessionPersistence.listSnapshots()` to `list()`, so the plugin never obtained a per-session revision: `sessionsSkippedByRevision` stayed at 0 and every baseline re-read every session log (431 reads / ~13 minutes for 479 sessions on the maintainer's host). Both spellings are now accepted; against a real 0.1.5 service the plugin reports `persistenceSnapshotsAvailable: true` and skips unchanged sessions again.
+- **A registry change during a baseline is no longer dropped.** `synchronizeWorkspaceRegistry()` returned early while `scan.done` was false, so a workspace added during the (previously very long) scan stayed invisible until the next restart. Such a change is now remembered and replayed as soon as the scan settles.
+
+### Changed
+
+- The workspace registry is additionally polled every 30 seconds, so a missed `domain/changed` event can no longer leave the workspace list stale until a restart.
+- `/api/all-usage/status` exposes `workspaceProbeEvents`, `workspaceSyncRuns`, `workspaceSyncAdded`, `workspaceSyncRemoved` and `workspaceSyncDeferred` (cumulative, not reset by a baseline) so the probe can be verified on a live host.
+
+### Note
+
+- The first start after upgrading still reads every log once: ledger records written before this release carry no `lastRevision`, so the revision comparison has nothing to match against. From the next start on, unchanged sessions are applied from the ledger without re-reading their logs.
+
+## [1.1.9] - 2026-09-12
+
+### Fixed
+
+- Deleting a workspace no longer erases its recorded usage. The registry probe used to call `removeSession()` for every session of a deregistered workspace and drop their ledger rows, and recovery skipped any row whose `sourceCwd` no longer mapped to a registered workspace; together they destroyed history the durable ledger had already recorded. (v1.1.6 fixed a different path — the DSH 0.1.5 `session.events` removal that wrote empty ledger records.)
+
+### Changed
+
+- Usage from every removed workspace is now summed into a single **Deleted** row instead of vanishing, and one row per dead workspace is no longer created. Retiring a workspace re-keys the live aggregate in place through the new `aggregation.retargetSession()`, so usage folded from the live feed that had not reached the ledger yet survives too.
+- Persisted ledger rows keep their original workspace id and cwd: retargeting happens on the in-memory copy only, the bucket is re-derived from the ledger after every restart, and a row with no usage never advertises the bucket.
+- The strict registration boundary is unchanged — sessions in unregistered or deleted directories are still excluded from new statistics — and legacy `unregistered:` rows are still never resurrected. Re-registering a workspace keeps its old history in the Deleted row while new sessions count under the live workspace.
+
 ## [1.1.8] - 2026-09-11
 
 ### Fixed

package/README.md

@@ -47,7 +47,7 @@ DeepSeek Harness 全量用量看板：按模型、供应商、工作区和时间
 - **中断请求**：上游请求被中断时可能只有 `assistant/chunk` 的 usage，没有最终 `assistant/message`；本插件会保留该 chunk 用量。同一 `turn / step` 后续出现最终 message 时，message 会替换 chunk。若上游完全没有 usage 事件，则无法从响应内容精确恢复 Token。
 - **估算成本**：成本是基于 models.dev 价格和 DSH usage 桶的估算，不是供应商账单；目录不可用或模型没有官方匹配时不会猜测价格，而是显示未计价。缓存读取、缓存写入和 reasoning 的口径取决于 DSH 上游事件。
 - **分层价格**：models.dev 的 tiered/context-dependent 价格按本次请求的输入上下文（fresh input + cache read + cache write）选择对应档位；阈值边界遵循目录定义，无法验证的异常 tier 仍显示为 unsupported。
-- **工作区边界**：只有 cwd 能映射到 DSH 已注册工作区的会话才进入统计；未注册 cwd（包括已存在但未在 registry 中登记的目录）会被忽略。工作区注册列表通过 DSH 的 `domain/changed` 探针自动同步：注册表一有改动就重读并只对新增/删除的工作区做增量处理，未变化的已有工作区直接复用已计算账本，不会全量重扫。会话尚未成功 flush 前删除或损坏的日志无法由独立账本恢复。
+- **工作区边界**：只有 cwd 能映射到 DSH 已注册工作区的会话才进入统计；未注册 cwd（包括已存在但未在 registry 中登记的目录）会被忽略。工作区注册列表通过 DSH 的 `domain/changed` 探针自动同步：注册表一有改动就重读并只对新增/删除的工作区做增量处理，未变化的已有工作区直接复用已计算账本，不会全量重扫。会话尚未成功 flush 前删除或损坏的日志无法由独立账本恢复。工作区被删除时历史用量不会丢失：它会被保留并汇总为一行「已删除」。
 
 ### 本地统计与官方账单
 
@@ -56,7 +56,7 @@ DeepSeek Harness 全量用量看板：按模型、供应商、工作区和时间
 - 本地统计读取 DSH 的 `assistant/chunk`、最终 `assistant/message` 和其他会话事件，按同一 `turn / step` 去重和替换；官方账单可能按供应商自己的请求、分词器、舍入、折扣、免费额度和结算周期计算。
 - 失败请求只要留下 usage chunk，就会进入本地统计；供应商是否对该失败请求收费，应以官方账单为准。
 - 价格来自 models.dev 的公开模型目录和本地显式覆盖；目录价格、供应商实际价格、区域费率和账单折扣可能不同。成本字段应理解为估算值。
-- 本地统计只包含已注册工作区；未注册 cwd 与已删除目录的旧 ledger 不进入统计，并可能因日志损坏、清理或上游没有发出 usage 而少于官方账单。
+- 本地统计只包含已注册工作区；未注册 cwd 的旧 ledger 不进入统计。**已删除工作区**的历史用量会保留并汇总为一行「已删除」，删除工作区不会让历史统计变小。统计仍可能因日志损坏、清理或上游没有发出 usage 而少于官方账单。
 
 ### 可复现事件示例
 
@@ -105,24 +105,11 @@ node scripts/replay-fixture.mjs fixtures/usage-events.json
 
 ### 最近更新
 
-**v1.1.5**
+**v1.1.10**
 
-- **工作区注册探针**：自动跟随 DSH 工作区注册表（`domain/changed` 事件）；只对新增/删除的工作区增量重扫，未变化工作区直接复用账本，零全量重扫。移除了手动“刷新工作区”按钮与其 `POST /api/all-usage/workspaces/refresh` 路由。
-- **严格注册边界**：统计只包含 cwd 能映射到已注册工作区的会话；未登记目录（即使存在）与历史 `unregistered:` 账本行不再进入统计，已删除目录的旧账本行在恢复时跳过（`sourceCwd` 现在随每条账本持久化以校验归属）。
-
-**v1.1.4**
-
-- **模型品牌图标**：请求日志、选中调用详情、模型统计表、成本设置匹配表、环形图图例与模型筛选下拉现在显示真实厂商品牌 SVG（DeepSeek、OpenAI、Claude、Gemini、Meta、Zhipu、xAI、Qwen、Doubao、Kimi、MiniMax）；模型命名空间优先于 DSH 供应商名，未知或混牌行保持中性。图标在构建期以 data URI 内嵌并由 scripts/svg-guard.mjs 校验（实体解码、CSS 转义/CDO/命名空间、url()/image-set() 均覆盖，注释与文本不误杀），运行时零网络请求。
-- **构建安全加固**：补齐 CDO/CDC 邻接 at-import、命名空间 style 元素、CSS 转义 url() 与外部 image-set() 四类绕过，并把 CSS 检查收窄到 style 属性与 style 体；对抗用例现在断言拒绝原因。
-- 三张用量仪表盘截图刷新为当前 UI。
-
-**v1.1.3**
-
-- 成本统计支持经验证的 context-tiered 官方费率、可展开费率表和显式价格覆盖。
-- **性能优化**：账本采用稳定分片、dirty flush 合并和 revision 快路径；查询、趋势和固定 53 周热力图改用写入时聚合索引，减少历史扫描、存储写放大和 Dashboard 重渲染。
-- 定价目录、价格同步、模型检索和映射编辑强化确定性与并发刷新边界。
-- 加强非法序列、旧账本、工作区重建、滞后 persistence revision 与复合账本键的恢复保护。
-- 增加 Node 22/24、DSH rc.1/rc.2 runtime smoke、脱敏 fixture replay 和包内容发布门禁。
+- **恢复账本的 revision 快路径**：DSH 0.1.5 把 `sessionPersistence.listSnapshots()` 改名为 `list()`，插件因此拿不到每会话 revision，`sessionsSkippedByRevision` 恒为 0、每次启动都重读全部会话日志（实测 479 个会话读了 431 个、约 13 分钟）。现在两种写法都支持，未变化的会话重新从账本复用。
+- **扫描期间的工作区变更不再被丢弃**：以前基线运行中到达的注册表变更会被直接忽略，导致新增工作区必须重启才出现；现在会被记住并在扫描结束后立即补跑，另有每 30 秒的注册表轮询兜底。
+- 升级后**首次**启动仍会全量读一次（旧账本记录没有 `lastRevision` 字段），此后启动走快路径。
 
 完整版本记录见 [CHANGELOG.md](CHANGELOG.md)。
 
@@ -183,7 +170,7 @@ dsh plugin --profile web add github:ParticleLight/dsh-all-usage
 
 - 使用次数与 Token 来自 DSH 会话日志；`session/flush` 只在存在新的相关事件时重建并将派生账本写入异步队列，同一 session 的 pending record 会合并，插件退出时 drain；插件激活时会回填日志与账本历史，插件卸载/重启后已成功持久化的数据不丢
 - 按日范围统计会保留全部可读取历史会话的有使用记录日期；热力图仅作为最近 53 周的固定视图窗口
-- 会话删除后，已成功 flush 的用量仍从独立账本恢复；会话销毁提示和周期对账只负责触发重建，不会删除账本记录
+- 会话删除后，已成功 flush 的用量仍从独立账本恢复；工作区删除同样不会丢数据——其历史用量汇总为一行「已删除」（含未落账的实时用量）。会话销毁提示和周期对账只负责触发重建，不会删除账本记录
 - 同一会话的同一 `turn / step` 只保留一份最终 usage；重试或替换消息会替换旧贡献，不重复累计
 - 输入 Token 按「未含缓存命中」计（缓存命中 / 写入独立成桶）；全 0 用量的重放事件不会覆盖已记录的真实用量，纯缓存命中的请求仍会计入
 - 轻量状态接口只公开 Host 实例、统计 revision、扫描进度与同步计数，不公开会话 ID、工作区路径、提示词或回复正文；完整快照仅在状态变化或手动刷新时获取
@@ -256,7 +243,7 @@ This plugin reports replayable statistics from local DSH event logs; it is not a
 - Local statistics read DSH `assistant/chunk`, final `assistant/message`, and related session events, then deduplicate and replace samples by logical `turn / step`. Official billing may use a provider tokenizer, rounding rules, discounts, free quotas, and billing periods.
 - A failed request is included locally whenever it leaves a usage chunk; whether the provider charged for that failed request must be checked against the official bill.
 - Prices come from the public models.dev catalog and local explicit overrides. Catalog prices can differ from provider prices, regional rates, and invoice discounts, so the cost field is an estimate.
-- Local statistics include registered workspaces only; unregistered cwds and old ledger rows for deleted directories are excluded, and totals can still be lower than the official bill when logs are damaged, cleaned up, or the upstream emits no usage event.
+- Local statistics include registered workspaces only; old ledger rows for unregistered cwds are excluded. Usage recorded for a **deleted workspace** is kept and summed into one "Deleted" row, so removing a workspace never shrinks historical totals. Totals can still be lower than the official bill when logs are damaged, cleaned up, or the upstream emits no usage event.
 
 ### Reproducible Event Examples
 
@@ -305,24 +292,11 @@ The command loads the real plugin Host, calls its compatible APIs, checks the do
 
 ### Latest Update
 
-**v1.1.5**
-
-- **Workspace registry probe**: all-usage follows DSH's workspace registry automatically through the `domain/changed` event; only added/removed workspaces are reprocessed incrementally while unchanged workspaces reuse their ledger with zero rescan. The manual Refresh workspaces control and its `POST /api/all-usage/workspaces/refresh` route were removed.
-- **Strict registration boundary**: only sessions whose cwd maps to a registered workspace are counted; unregistered directories (even existing ones) and legacy `unregistered:` ledger rows never re-enter statistics, and ledger rows for deleted directories are skipped on recovery (`sourceCwd` is now persisted with every ledger record so ownership can be validated after a restart).
-
-**v1.1.4**
-
-- **Vendor brand icons**: request logs, selected-call details, the model summary table, the cost-settings match table, donut legends, and the model filter dropdown now render real vendor brand SVGs (DeepSeek, OpenAI, Claude, Gemini, Meta, Zhipu, xAI, Qwen, Doubao, Kimi, MiniMax); the model namespace wins over the DSH provider name, and unknown or mixed-brand rows stay neutral. Icons are embedded as data URIs at build time and validated by scripts/svg-guard.mjs (entity decoding, CSS escapes/CDO/namespaced styles, url()/image-set() all covered, comments and text stay inert), with zero runtime network access.
-- **Build hardening**: per the independent audit, closed four bypass classes (CDO/CDC-adjacent at-import, namespaced style elements, CSS-escaped url(), external image-set()) and scoped the CSS checks to style attributes and bodies; adversarial fixtures now assert the rejection reason.
-- Refreshed the three usage-dashboard screenshots to the current UI.
-
-**v1.1.3**
+**v1.1.10**
 
-- Added validated context-tiered official pricing, expandable rate schedules, and explicit price overrides.
-- **Performance optimizations**: the durable ledger uses stable shards, dirty-flush coalescing, and revision reuse; scoped queries, trends, and the fixed 53-week heatmap use ingest-time aggregates to reduce historical scans, storage write amplification, and Dashboard rerenders.
-- Hardened deterministic pricing catalogs, pricing sync, official-model search, and mapping refresh races.
-- Strengthened recovery for invalid sequences, legacy ledgers, recreated workspaces, lagging persistence revisions, and composite ledger keys.
-- Added Node 22/24, DSH rc.1/rc.2 runtime smoke, redacted fixture replay, and package-content release gates.
+- **The ledger fast path is back**: DSH 0.1.5 renamed `sessionPersistence.listSnapshots()` to `list()`, so no per-session revision was ever obtained — `sessionsSkippedByRevision` stayed at 0 and every start re-read every session log (431 reads / ~13 minutes for 479 sessions). Both spellings are accepted now, and unchanged sessions are applied from the ledger again.
+- **A registry change during a baseline is no longer dropped**: a change arriving mid-scan used to be ignored, so a newly added workspace only appeared after a restart. It is now replayed as soon as the scan settles, with a 30-second registry poll as a safety net.
+- The first start after upgrading still reads every log once (older ledger records carry no `lastRevision`); later starts take the fast path.
 
 See [CHANGELOG.md](CHANGELOG.md) for the complete version history.
 
@@ -373,7 +347,7 @@ The profile patch layer hot-reloads; save the file and refresh the page.
 
 ### Data semantics
 
-- Calls and tokens come from DSH session logs; `session/flush` rebuilds and queues the derived ledger only when related events are dirty, coalescing the latest pending record per session and draining on plugin disposal. Readable logs and ledger history are backfilled when the plugin activates, so successfully persisted data survives reloads or session deletion
+- Calls and tokens come from DSH session logs; `session/flush` rebuilds and queues the derived ledger only when related events are dirty, coalescing the latest pending record per session and draining on plugin disposal. Readable logs and ledger history are backfilled when the plugin activates, so successfully persisted data survives reloads or session deletion; deleting a workspace likewise keeps its history, summed into one "Deleted" row together with usage that had not reached the ledger yet
 - Day-level range data retains every readable historical session date with tracked usage; the heatmap is only a fixed latest-53-week view
 - After a session is deleted, successfully flushed usage is restored from the separate ledger; disposal hints and periodic reconciliation trigger rebuilds without deleting ledger rows
 - For each session and logical `turn / step`, only the final usage contribution is kept; retries or replaced messages do not double-count

package/lib/aggregation.js

@@ -1,4 +1,5 @@
 import { createHash } from 'node:crypto'
+import { RETIRED_WORKSPACE_ID } from './ledger.js'
 import { COST_SCHEMA_VERSION, addCostAccumulator, addCostAggregate, calculateCost, createCostAccumulator, decimalSubtract, isCostAccumulator, normalizeCostSnapshot, resolvePricing, serializeCostAggregate, temporalPlanFor } from './pricing.js'
 import { extractUsageEvent, normalizeEventSeq, normalizeUsageValues as usageValues, upsertUsageSample as upsertUsageSampleState, usageStepKey, validEventTime as validUsageEventTime } from './usage-core.js'
 
@@ -97,6 +98,9 @@ export function createAggregation(host) {
     state.byDay.clear()
     state.byDayUtc.clear()
     state.perWorkspace.clear()
+    // The deleted bucket is re-derived from the ledger during the rebuild that
+    // follows, so a stale flag cannot advertise an empty row.
+    state.retiredBucketUsed = false
     state.perModel.clear()
     state.usageByStep.clear()
     state.turnRecords.clear()
@@ -377,6 +381,47 @@ export function createAggregation(host) {
     if (removed > 0) markStatsChanged('data')
     return removed
   }
+  // Re-key every aggregate entry owned by one session to another workspace id.
+  //
+  // Retiring a workspace cannot rely on the ledger alone: usage folded from the
+  // live feed in the last few milliseconds has no persisted row yet, so it would
+  // keep a workspace id the dashboard no longer advertises. Moving the in-memory
+  // records keeps the totals, the per-workspace split and the query buckets
+  // consistent no matter when the workspace disappears.
+  function retargetSession(sid, workspaceId) {
+    if (typeof sid !== 'string' || sid === '' || typeof workspaceId !== 'string' || workspaceId === '') return 0
+    let moved = 0
+    for (const item of state.usageByStep.values()) {
+      if (item.sid !== sid || item.wsId === workspaceId) continue
+      const previousWs = item.wsId
+      const identity = item.identity || item.modelId
+      const dates = { local: item.date, utc: item.dateUtc }
+      // Decrement under the old workspace, then increment under the new one: the
+      // session, model and global totals cancel out exactly.
+      adjustUsage(previousWs, item.time, item.values, identity, -1, sid, dates, item.cost)
+      adjustQueryUsage(state.byDay.get(item.date), item, -1, false)
+      adjustQueryUsage(state.byDayUtc.get(item.dateUtc), item, -1, true)
+      item.wsId = workspaceId
+      adjustUsage(workspaceId, item.time, item.values, identity, 1, sid, dates, item.cost)
+      adjustQueryUsage(state.byDay.get(item.date), item, 1, false)
+      adjustQueryUsage(state.byDayUtc.get(item.dateUtc), item, 1, true)
+      moved += 1
+    }
+    for (const turn of state.turnRecords.values()) {
+      if (turn.sid !== sid || turn.wsId === workspaceId) continue
+      removeTurnRecord(turn)
+      turn.wsId = workspaceId
+      ensureWs(workspaceId).turns += 1
+      state.totals.turns += 1
+      addDayTurn(state.byDay, turn.date, workspaceId, turn.sid, 1)
+      addDayTurn(state.byDayUtc, turn.dateUtc, workspaceId, turn.sid, 1)
+      adjustQueryTurn(state.byDay.get(turn.date), turn, 1, false)
+      adjustQueryTurn(state.byDayUtc.get(turn.dateUtc), turn, 1, true)
+      moved += 1
+    }
+    if (moved > 0) markStatsChanged('data')
+    return moved
+  }
   function upsertUsageSample(target, sample, options = {}) {
     const previous = target.get(sample.key)
     const candidate = sample.cost === undefined
@@ -595,7 +640,10 @@ export function createAggregation(host) {
       usageSchemaVersion: 3,
       costSchemaVersion: COST_SCHEMA_VERSION,
       requestToken: state.requestToken,
-      workspaces: Array.from(state.wsMeta.values(), (w) => ({ id: w.id, title: w.title, path: w.path })),
+      workspaces: [
+        ...Array.from(state.wsMeta.values(), (w) => ({ id: w.id, title: w.title, path: w.path })),
+        ...(state.retiredBucketUsed ? [{ id: RETIRED_WORKSPACE_ID, title: '', path: '', deleted: true, retiredBucket: true }] : []),
+      ],
       aliases: Object.assign({}, state.aliases),
       pricing: pricingSnapshot({ detailed: false }),
       tokenSemantics: {
@@ -983,6 +1031,7 @@ export function createAggregation(host) {
     resolveCurrentPricing,
     costForUsage,
     removeSession,
+    retargetSession,
     resetAggregationState,
     ensureDay,
     ensureWs,