billion-context-dsh 0.2.20 → 0.2.21

package/README.en.md

@@ -3,7 +3,7 @@
 [English](./README.en.md) | [中文](./README.md)
 
 > **⚠️ Beta notice — not for production use**
-> This project (**v0.2.20**) is a work-in-progress beta. The [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) itself is also in **public beta**. **Do not use either in engineering / production environments** — expect breaking changes and rough edges.
+> This project (**v0.2.21**) is a work-in-progress beta. The [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) itself is also in **public beta**. **Do not use either in engineering / production environments** — expect breaking changes and rough edges.
 
 <p align="center">
 <strong>Built with gratitude on top of these projects</strong> — please give them a ⭐:
@@ -98,7 +98,7 @@ This only installs the package into your project/global store; it does **not** t
 **Install from the git source (`github:` spec — the form the plugin store shows).** The prebuilt `dist/` artifacts are committed to this repository, so a git-source install also works out of the box — **no build step needed**, and pnpm 11's default build-script blocking (`allowBuilds`) never applies to this package:
 
 ```bash
-dsh plugin --profile web add github:Tyan66666/billion-context-dsh#v0.2.20
+dsh plugin --profile web add github:Tyan66666/billion-context-dsh#v0.2.21
 ```
 
 Prefer a `#<tag>` ref to get artifacts identical to that npm release; without a ref you get the latest default-branch build. Only building the repo yourself (`npm run build`) requires approving build scripts. Background and trade-offs: [docs/git-source-install-design.md](docs/git-source-install-design.md) (issue #92).
@@ -179,7 +179,7 @@ DSH derives every model request from its append-only session log (the *surface*)
 | `acp_status` | CONTEXT BREAKDOWN (tool/text/summaries shares of the visible total) + compressed-block ledger + nudge decision line + a `Checkpoint seqs` row mapping each ACTIVE block's kernel ref (`bN`) to its checkpoint summary seq — compressing a checkpoint seq distills that block (issue #60); no context-window rows; scope/view/tool/sort/limit drilldown supported |
 | block state | in-memory kernel state + **log-rebuilt ledger** (no sidecar files) |
 | tiered distillation (T2/T3) | re-compressing a block's summary node distills that block (tier 2); distilling a tier-2 block yields tier 3. Tier + kernel block ids are persisted to the log, so kernel state rehydrates from the log after a restart and stays distillable |
-| compression accounting (shadow price) | `shadowedTokenCount` (what the host occupancy display deducts) is priced in the **host token-meter's vocabulary** (`ctx.tokenMeter.measure` preferred; exact mirror in `src/host-tokens.ts` as fallback) — never the plugin's internal CJK-aware estimate (that is display currency; mixing it into the host ledger can drive `messageTokens` negative and brick a CJK-heavy session, issue #54) |
+| compression accounting (shadow price) | `shadowedTokenCount` (what the host occupancy display deducts) is priced with the **host token-meter's fixed-heuristic price** (`ctx.tokenMeter.measure` preferred, reading the `heuristicTokens ?? tokens` fixed-heuristic basis; exact mirror in `src/host-tokens.ts` as fallback) — never the plugin's internal CJK-aware estimate (that is display currency; mixing it into the host ledger can drive `messageTokens` negative and brick a CJK-heavy session, issue #54), and never the route-repriced `node.tokens` either (under 0.1.2+ image route pricing that is request-pressure currency; summing it overstates an image range's claim and folds the same ledger negative, issue #103) |
 
 The load-bearing compression guidance (tools, philosophy, summary rules, tier rules) is registered as a one-time system-prompt section; each nudge carries a condensed version (efficiency note + philosophy + context breakdown + HOW_TO_COMPRESS_RULES + range table + batch tip). There is deliberately **no automatic summarization**: automatic policy only nudges the model (`compactIfNeeded` returns null).
 

package/README.md

@@ -3,7 +3,7 @@
 [中文](./README.md) | [English](./README.en.md)
 
 > **⚠️ 测试版声明——请勿用于生产环境**
-> 本项目（**v0.2.20**）仍处于开发中的测试版。[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 本身也处于**公开测试版**阶段。**请勿将两者用于工程化 / 生产环境**——预期会有破坏性变更与粗糙之处。
+> 本项目（**v0.2.21**）仍处于开发中的测试版。[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 本身也处于**公开测试版**阶段。**请勿将两者用于工程化 / 生产环境**——预期会有破坏性变更与粗糙之处。
 
 <p align="center">
 <strong>衷心感谢以下项目——请给它们一个 ⭐：</strong>
@@ -97,7 +97,7 @@ npm install billion-context-dsh
 **git 源安装（`github:` 规格，插件商店展示的形态）。** 预构建产物 `dist/` 已提交到仓库，从 git 源安装同样开箱即用——**无需任何构建步骤**，pnpm 11 默认拦截构建脚本（`allowBuilds`）的机制对这个包不构成障碍：
 
 ```bash
-dsh plugin --profile web add github:Tyan66666/billion-context-dsh#v0.2.20
+dsh plugin --profile web add github:Tyan66666/billion-context-dsh#v0.2.21
 ```
 
 建议带 `#<tag>` 安装，拿到与对应 npm 版本完全一致的产物；不带 ref 则装默认分支的最新构建。只有 clone 仓库自行从源码构建（`npm run build`）才需要放行构建。背景与方案取舍见 [docs/git-source-install-design.md](docs/git-source-install-design.md)（issue #92）。
@@ -176,7 +176,7 @@ DSH 的每个模型请求都派生自其 append-only 会话日志（*surface*）
 | `acp_status` | CONTEXT BREAKDOWN（tool/text/summaries 占可见总量）+ 压缩块账本 + nudge 决策行 + `Checkpoint seqs` 行（active 块的 `bN → seq` 映射——压缩某个 checkpoint seq 即蒸馏该块，issue #60）；不含上下文窗口；支持 scope/view/tool/sort/limit 钻取 |
 | 块状态 | 内存内核状态 + **日志重建账本**（无旁车文件） |
 | 分层蒸馏（T2/T3） | 再次压缩某块的摘要节点 = 蒸馏该块（tier 2），蒸馏 tier-2 块得 tier 3；tier 与内核块 id 持久化进日志，重启后内核状态从日志再水合、可继续蒸馏 |
-| 压缩记账（影子价格） | `shadowedTokenCount`（宿主占用率据此扣减）**用宿主 token-meter 的词汇计价**（`ctx.tokenMeter.measure` 优先，`src/host-tokens.ts` 精确镜像兜底）——绝不混用插件内部的 CJK 感知估算（那是展示货币，混用会把宿主账本扣成负数、卡死中文会话，issue #54） |
+| 压缩记账（影子价格） | `shadowedTokenCount`（宿主占用率据此扣减）**用宿主 token-meter 的固定启发价计价**（`ctx.tokenMeter.measure` 优先、按 `heuristicTokens ?? tokens` 读固定启发价基准，`src/host-tokens.ts` 精确镜像兜底）——绝不混用插件内部的 CJK 感知估算（那是展示货币，混用会把宿主账本扣成负数、卡死中文会话，issue #54），也不按路由重定价的 `node.tokens` 计价（0.1.2+ 图片路由计价下那是请求压力价，读它会让含图片区间的 claim 虚报视觉价、同样扣穿账本，issue #103） |
 
 承载性的压缩指引（工具、哲学、摘要规则、tier 蒸馏/浓缩规则）注册为一次性系统提示段；每条 nudge 携带精简版（效率提示 + 哲学 + 上下文分解 + 压缩规则 + 范围表 + 批量提示）。刻意**不做自动摘要**：自动策略只 nudge 模型（`compactIfNeeded` 返回 null）。
 

package/dist/host-tokens.d.ts

@@ -11,10 +11,20 @@
  * display currency, NEVER event currency).
  *
  * This module prices claims in the host's vocabulary: it prefers the live
- * meter's own per-node prices (`ctx.tokenMeter.measure(session).nodes` —
- * exact by construction, follows host estimator changes automatically, the
- * same path the host's own `compaction-basic` uses) and falls back to an
+ * meter's own per-node FIXED-HEURISTIC prices (`ctx.tokenMeter.measure(session)`
+ * nodes' `heuristicTokens` — the same basis the projection ledger accumulates
+ * appends with, so the claim is exact by construction) and falls back to an
  * exact mirror of the host's estimator when the meter is unreachable.
+ *
+ * Two vocabularies share the meter's node since DSH 0.1.2: `tokens` carries
+ * the measured route's request pressure (image occurrences re-priced with the
+ * route's declared visual tokens) while `heuristicTokens` keeps the fixed
+ * flat-4 heuristic the ledger prices appends with. The claim MUST read
+ * `heuristicTokens`: a routed `tokens` claim overstates the replaced range
+ * against its own ledger accumulation and folds `messageTokens` negative —
+ * the same session-bricking schema rejection as #54, through the image-route
+ * channel (issue #103). Older hosts (0.1.0/0.1.1 lines) expose a single
+ * `tokens` field that IS the fixed heuristic, so the fallback reads it.
  */
 import type { Session, SessionEvent } from '@deepseek-ai/dsh-session';
 /** The host's model-visible content block union (structural, mirror-side only). */
@@ -61,8 +71,17 @@ export declare function hostPriceEvent(event: SessionEvent): number;
 export declare function shadowedHostTokens(session: Session, seqs: readonly number[]): number;
 /**
  * Claim price for `seqs` in the host's vocabulary. Prefers the live meter's
- * own per-node prices when `ctx.tokenMeter` is reachable and covers every
- * shadowed seq (exact by construction, follows host estimator changes); ANY
+ * own per-node FIXED-HEURISTIC prices when `ctx.tokenMeter` is reachable and
+ * covers every shadowed seq (exact by construction — the ledger's
+ * `foldSurfaceProjection` accumulates appends with the same fixed heuristic,
+ * so the claim and the ledger stay in agreement; follows host estimator
+ * changes automatically). `node.heuristicTokens` is that basis since DSH 0.1.2;
+ * `node.tokens` there is the measured route's REQUEST pressure (image
+ * occurrences carry the route's visual price via `priceSurface`) and MUST NOT
+ * be claimed — reading it overstates the claim and folds the host projection
+ * negative on image-containing ranges (issue #103, the image-route channel of
+ * the #54 brick). Older meters expose a single `tokens` field that IS the
+ * fixed heuristic, so `heuristicTokens ?? tokens` covers both shapes. ANY
  * failure — meter absent, `measure` throwing (e.g. a step-less log), or a seq
  * missing from the measurement — falls back to the exact mirror. Never returns
  * a `defaultCountTokens` price (rule 12).

package/dist/index.js

@@ -2637,13 +2637,7 @@ function makePreview(text, query, len) {
 
 // src/region.ts
 import { randomUUID } from "crypto";
-import {
-  CompactionId,
-  compactCheckpointSource,
-  toolPairingBalancedAfter,
-  toolPairingBalancedBefore
-} from "@deepseek-ai/dsh-compaction";
-import { createAssistantMessage, createUserMessage } from "@deepseek-ai/dsh-llm";
+import { CompactionId, compactCheckpointSource } from "@deepseek-ai/dsh-compaction";
 
 // src/session-events.ts
 function sessionEventsOf(session) {
@@ -2657,6 +2651,78 @@ function eventAtOf(session, seq) {
   return session.events[seq];
 }
 
+// src/tool-pairing.ts
+var balanceCacheBySession = /* @__PURE__ */ new WeakMap();
+function eventDelta(event) {
+  if (event.type === "tool/result") return -1;
+  if (event.type === "assistant/message") {
+    const content = event.data.message?.content;
+    if (!Array.isArray(content)) return 0;
+    let calls = 0;
+    for (const block of content) {
+      if (block !== null && typeof block === "object" && block.type === "tool-call") calls += 1;
+    }
+    return calls;
+  }
+  return 0;
+}
+function eventForSeq(session, seq) {
+  const event = eventAtOf(session, seq);
+  if (event === void 0 || event.seq !== seq) {
+    throw new Error(`tool-pairing balance: surface seq ${seq} has no matching session event (corrupt surface)`);
+  }
+  return event;
+}
+function extendCache(session, cache2, seqs) {
+  const processed = cache2.cutBalanced.length - 1;
+  const tail = seqs.slice(processed);
+  const pendingCuts = [];
+  let inProgressToolCalls = cache2.inProgressToolCalls;
+  for (const seq of tail) {
+    inProgressToolCalls += eventDelta(eventForSeq(session, seq));
+    if (inProgressToolCalls < 0) {
+      throw new Error(`tool-pairing balance: tool/result at surface seq ${seq} has no matching tool-call (corrupt surface)`);
+    }
+    pendingCuts.push(inProgressToolCalls === 0);
+  }
+  tail.forEach((seq, offset) => cache2.indexBySeq.set(seq, processed + offset));
+  cache2.cutBalanced = cache2.cutBalanced.concat(pendingCuts);
+  cache2.inProgressToolCalls = inProgressToolCalls;
+  return cache2;
+}
+function balanceCache(session) {
+  const seqs = session.surface.nodes;
+  const generation = session.surface.replaceGeneration;
+  const cached = balanceCacheBySession.get(session);
+  if (cached === void 0 || cached.generation !== generation || cached.cutBalanced.length - 1 > seqs.length) {
+    const rebuilt = extendCache(session, {
+      generation,
+      cutBalanced: [true],
+      indexBySeq: /* @__PURE__ */ new Map(),
+      inProgressToolCalls: 0
+    }, seqs);
+    balanceCacheBySession.set(session, rebuilt);
+    return rebuilt;
+  }
+  if (cached.cutBalanced.length - 1 < seqs.length) return extendCache(session, cached, seqs);
+  return cached;
+}
+function cutBalance(cache2, seq, offset) {
+  const index = cache2.indexBySeq.get(seq);
+  const balanced = index === void 0 ? void 0 : cache2.cutBalanced[index + offset];
+  if (balanced === void 0) throw new Error(`tool-pairing balance: surface seq ${seq} not found`);
+  return balanced;
+}
+function toolPairingBalancedBefore(session, seq) {
+  return cutBalance(balanceCache(session), seq, 0);
+}
+function toolPairingBalancedAfter(session, seq) {
+  return cutBalance(balanceCache(session), seq, 1);
+}
+
+// src/region.ts
+import { createAssistantMessage, createUserMessage } from "@deepseek-ai/dsh-llm";
+
 // src/messages.ts
 function extractText(content) {
   if (typeof content === "string") return content;
@@ -2847,7 +2913,7 @@ function shadowedTokensViaMeter(session, seqs, ctx) {
   try {
     const meter = ctx?.get?.("tokenMeter");
     if (meter?.measure !== void 0) {
-      const bySeq = new Map(meter.measure(session).nodes.map((node) => [node.seq, node.tokens]));
+      const bySeq = new Map(meter.measure(session).nodes.map((node) => [node.seq, node.heuristicTokens ?? node.tokens]));
       let total = 0;
       let missing = false;
       for (const seq of seqs) {