dsh-engram 0.3.7 → 0.4.0

package/README.md

@@ -238,7 +238,14 @@ curl -s -X POST http://127.0.0.1:3080/api/dsh-engram/import \
   -d "{\"payload\": $(cat engram-backup.json), \"mode\": \"merge\"}"
 ```
 
-## DSH compatibility
+## Compatibility
+
+<!-- dsh-market compatibility block — keep labels bilingual and in sync with
+     docs/RELEASE-NOTES.TEMPLATE.md (market README convention) -->
+
+- **DSH 版本 / DSH versions**: `>=0.1.2-alpha.2 <0.2.0-0` (dsh-engram `0.3.7`; releases `0.2.0`–`0.3.6` support the old generation only — see matrix below)
+- **最后验证日期 / Last verified**: 2026-09-10 — full regression on `dsh-v0.1.5-alpha.1`; real-machine settings GET/PUT/conflict/reset/restart-persistence on `dsh 0.1.5-rc.1` (isolated profile)
+- **安装方式已验证 / Verified install**: `dsh plugin --profile web add dsh-engram`
 
 > **Short version.** Every **published** release (`0.2.0` … `0.3.6`) supports
 > the **old** DSH API graph (`0.1.0-rc.7` / `0.1.1`, peers `^0.1.0-rc.7`)
@@ -278,7 +285,7 @@ viewer), so the card is generation-agnostic on both sides of the
 So: keep DSH on `0.1.1-rc.2` or older while running a release ≤ `0.3.6`, and
 upgrade the plugin to `>=0.3.7` before moving DSH to `0.1.2-alpha.2`.
 
-### Current release — `0.3.7`
+### Current release — `0.4.0`
 
 This release declares support for DSH `>=0.1.2-alpha.2 <0.2.0-0` and was fully
 validated against the `dsh-v0.1.5-alpha.1` source and web runtime from the local
@@ -675,6 +682,32 @@ means that preset was customized (or has no compaction group) and was left untou
 `dsh-compaction-basic unavailable` means the backend is missing and everything
 falls back to the default compressor.
 
+### BoundaryPrune (subtask-boundary pruning, default OFF)
+
+Context GC owns *what a compaction summary says*; BoundaryPrune owns *when a
+deterministic prune fires*: at todo-completion progress and goal terminal
+events, it calls DSH's own `toolResultPruner` (deterministic surface-replace,
+zero LLM) so long-stale over-threshold tool results stop replaying in every
+later request. The two switches are independent.
+
+- **Evidence chain** ([oracle analysis](docs/ORACLE-ANALYSIS.zh.md) +
+  [proposal](docs/PROPOSAL-resultpack.zh.md)): 97 real sessions measured —
+  2.77B–7.53B chars of tool-result replay exposure (~692M–1.88M tokens), 96%
+  of result bytes from bash/read/run_code; the counterfactual simulation says
+  todo boundaries cover 79% of that exposure and a repayment gate lifts
+  cache-aware net savings 12–32% on every threshold config.
+- **Repayment gate**: a prune rewrites the prefix (one KV-cache break), so it
+  fires only while the session is still expected to run at least
+  `boundaryPruneMinRemaining` (default 50) more requests — expectation is a
+  workspace-history prior with ×2 growth projection past it, so long sessions
+  are never starved.
+- **Honest activation surface**: sessions with no goal and no todo progress
+  simply never trigger (no boundary → no prune, zero side effects); the
+  corpus-wide dry run activates on 38% of sessions. If the pruner service is
+  absent the mechanism disables itself with one log line.
+- Default **off**: `boundaryPrune: false`. When on, the log shows
+  `engram boundary-prune (todo @N reqs): pruned …`.
+
 ## Auto-capture policy
 
 Capture is deterministic and offline — it only sees tool *results*, never the
@@ -783,7 +816,7 @@ Defaults are token-conscious; override any key via the profile patch
 ## Development
 
 ```sh
-npm test            # 152 tests: core + web API + GC + usage-observability + Context GC + ESR triggers (node:test)
+npm test            # 279 tests: core + web API + GC + usage-observability + Context GC + ESR triggers + BoundaryPrune (node:test)
 npm run eval        # offline recall + structure benchmark (deterministic)
 npm run build:client
 ```

package/README.zh.md

@@ -151,6 +151,12 @@ curl -s -X POST http://127.0.0.1:3080/api/dsh-engram/import \
 
 ## 兼容性
 
+<!-- dsh-market 兼容信息块 —— 与 docs/RELEASE-NOTES.TEMPLATE.md 的市场 README 公约保持一致 -->
+
+- **DSH 版本**：`>=0.1.2-alpha.2 <0.2.0-0`（dsh-engram `0.3.7`；`0.2.0`–`0.3.6` 仅旧代际 —— 见下文矩阵）
+- **最后验证日期**：2026-09-10 —— `dsh-v0.1.5-alpha.1` 完整回归；`dsh 0.1.5-rc.1` 隔离 profile 真机验证设置 GET/PUT/冲突/复位/跨重启持久化
+- **安装方式已验证**：`dsh plugin --profile web add dsh-engram`
+
 > **一句话结论**：所有**已真正发布**的版本（`0.2.0` … `0.3.6`）**只支持旧版
 > DSH**（`0.1.0-rc.7` / `0.1.1` API 图，peers `^0.1.0-rc.7`）。DSH
 > `0.1.2-alpha.2` 及更新版本与它们**不兼容**——settings/client API 族在该版本
@@ -182,7 +188,7 @@ HTTP 路由（`/api/dsh-engram/settings`，与记忆 viewer 同样的同源 fetc
 所以：跑 ≤ `0.3.6` 的已发布版本时请把 DSH 保持在 `0.1.1-rc.2` 或更旧；升级
 DSH 到 `0.1.2-alpha.2` 之前，先升级插件到 `>=0.3.7`。
 
-### 当前发布版本 —— `0.3.7`
+### 当前发布版本 —— `0.4.0`
 
 当前发布版本声明适配 DSH `>=0.1.2-alpha.2 <0.2.0-0`，并以本地
 `deepseek-harness` 的 `dsh-v0.1.5-alpha.1` 代码和 Web 运行时完成完整验证。
@@ -327,10 +333,18 @@ npm run build:client
 ## 测试与评测
 
 ```sh
-npm test    # 152 项单元测试（含 usage 滚动 / /stats 路由 / Context GC / ESR 触发）
+npm test    # 279 项单元测试（含 usage 滚动 / /stats 路由 / Context GC / ESR 触发 / BoundaryPrune）
 npm run eval  # 离线召回 + 结构基准（确定性语料，跑真实 store/recall 路径）
 ```
 
+另有三个**会话语料分析工具**（零 LLM、确定性，读 `~/.dsh/sessions` 真实日志）：
+
+```sh
+node eval/oracle-analysis.mjs             # 浪费源量化：结果重放暴露 / 融合候选 / 机制激活率
+node eval/simulate-boundary-prune.mjs     # L0 反事实仿真：边界剪枝的预期节省与偿还门扫掠
+node eval/dryrun-boundary-prune.mjs       # BoundaryPrune 全量干跑：真实激活面与节省
+```
+
 `npm run eval` 的检索部分参照 LongMemEval 的问答式评测：受控语料（ASCII + CJK、标签/实体/时间戳已知），
 对真实 `domain.recall()` 逐一校准 **Precision@k / Recall@k / MRR / Hit@1**（probe 覆盖 tag 精确、子串、
 多词、CJK、短语唯一、标签排序、负样本无回）；结构部分借鉴 StructMemEval：精确去重率（同文本存 3 次折叠为 1）、
@@ -487,6 +501,26 @@ Context GC 的装配在**两个平面**——web 现在**全部自动**，零配
 - （若见 `… keeping the existing compaction service` 说明同域已有 provider、未替换成功；`web-provision … skipped (custom)`
   说明某预设是自定义/无 compaction 布局、未被触碰；`dsh-compaction-basic unavailable` 说明环境缺依赖，自动回退默认压缩。）
 
+### BoundaryPrune（子任务边界剪枝，默认关）
+
+Context GC 管「压缩的摘要内容」，BoundaryPrune 管「确定性剪枝的触发时机」——
+在 todo 完成推进 / goal 终态这些子任务边界，主动调用 DSH 自带的
+`toolResultPruner`（确定性 surface-replace，零 LLM），把早已超阈值的旧工具
+结果剪掉头尾保留，省掉它们在后续每个请求里的整段重放。两个开关互相独立。
+
+- **证据链**（[docs/ORACLE-ANALYSIS.zh.md](docs/ORACLE-ANALYSIS.zh.md) +
+  [docs/PROPOSAL-resultpack.zh.md](docs/PROPOSAL-resultpack.zh.md)）：97 个真实
+  会话实测，工具结果重放暴露 2.77B–7.53B 字符（≈692M–1.88M token），96% 的
+  结果字节来自 bash/read/run_code；反事实仿真显示 todo 边界覆盖 79% 的暴露，
+  且偿还门在所有配置下提升净收益 12–32%。
+- **偿还门**：剪枝会重写前缀（KV 缓存击穿一次），只有当会话预计还要跑
+  `boundaryPruneMinRemaining`（默认 50）个以上请求时才 fire——预计长度用
+  workspace 历史先验 + 超过后 ×2 外推（有界乐观），长会话不会被挡死。
+- **诚实激活面**：goal=0 且无 todo 推进的会话天然不触发（无边界不剪，零副作用）；
+  全量干跑 77 会话激活 38%。pruner 服务缺席即自动禁用并打一条日志。
+- 默认**关**：`boundaryPrune: false`。开启后启动日志出现
+  `engram boundary-prune (todo @N reqs): pruned …`。
+
 ## 自动捕获策略
 
 捕获是确定性、离线的——只看到工具*结果*，从不看对话本身。什么会被记录成一条记忆：
@@ -575,6 +609,8 @@ promote: 2 pending todo(s) vs 1 ESR task(s) — esr_task(name="…") #suggest-pr
     gcStableRetentionDays: 120  # 超过此天数的 stable 任务离开 [ESR]
     gcReplacesCompaction: true # Context GC：接管 DSH 自动 compact（false=回退默认 LLM 压缩）
     gcNarrative: true        # 无锚轮次走 scoped LLM 叙事；false=纯机械（零 LLM）
+    boundaryPrune: false     # BoundaryPrune：todo/goal 边界主动剪超阈值旧工具结果（默认关）
+    boundaryPruneMinRemaining: 50  # 偿还门：预计剩余请求数低于此值不剪
     engramIndexOrder: 40    # systemPrompt section 顺序（位于 tools 段之前）
     esrOrder: 41
 ```

package/docs/ORACLE-ANALYSIS.zh.md

@@ -0,0 +1,63 @@
+# Oracle Analysis:真实会话 token 浪费源量化
+
+> 方法论来自 [SoL-Pi](https://nvlabs.github.io/SoL-Pi/)(NVIDIA Labs):**动手造机制之前,先在真实轨迹上量化浪费源**("Oracle Analysis" / M6 "Use issue history and blind failure mining as proposal evidence" / M42 "Choose tasks with mechanism headroom before scaling")。
+>
+> 工具:`node eval/oracle-analysis.mjs`(流式解压 `~/.dsh/sessions`,零 LLM、确定性、约 90 秒跑完全量)。
+
+## 语料
+
+97 个真实会话日志(3 个 workspace 桶),13,900 次模型请求,15,020 次工具调用,65 轮 compaction,1,376 次会话中途 inbox splice。模型混合 local-vllm deepseek-v4-flash 为主,兼 glm-5.3 / qwen3.8-flash 等。
+
+## M1 工具结果重放暴露(ObservationPack headroom,SoL-Pi C11/C23)
+
+每个工具结果产生后,会在后续每个请求里重放(直到 compaction 改写历史)。exposure = Σ 结果字符 × 后续请求数:
+
+| 口径 | 字符数 | ≈token(chars/4) |
+|---|---|---|
+| naive(无 compaction 截断,上界) | 7.53B | 1.88B |
+| compaction 截断(下界) | 2.77B | 692M |
+| 占「重放+固定开销」比重 | 57.6%(naive)– 21.2%(cut) | — |
+
+- **top-10 结果占全会话结果字节的 46.1%(中位数)**——少数巨型输出承载了近一半的重放压力,正是 ObservationPack「归档载荷、留 handle+摘录」的靶子。
+- 结果字节来源:**bash 14.2M + read 10.6M + run_code 6.1M ≈ 96%**。截断/归档机制只需覆盖这三个工具即可命中几乎全部暴露。
+- 重放最重的会话(解析器开发,549–1,399 请求/会话)单会话重放高达 91–383Mc(23–96M token)——长时程会话正是 SoL-Pi 指出效率差异可测的场景。
+
+**固定开销**(每个请求重发的 system+tools schema):全语料 436Mc ≈ 109M token,平均 ~7.8K token/请求;富预设(43 工具)单请求 schema 41.8Kc。**本插件自身 schema 仅 2.9Kc(≈730 token,占该预设 7%)——插件的开销主张成立。**
+
+## M2 Action-Fusion headroom(SoL-Pi "Action Fusion")
+
+编辑类调用(edit/write/str_replace_editor,共 1,993 次)之后**紧接着**由模型发起 bash(中间隔一次模型决策)的候选:607 次。
+
+- 占全部工具调用 **4.0%**;占 bash 调用 **7.5%**;占步间转移 ~4.4%(SoL-Pi 在 Pi 上测得 12.3%,DSH 上 headroom 更小)。
+- 结论:融合编辑+运行为单一工具调用是**上游 harness 的杠杆**,不是插件的;优先级排后。
+
+## M3 engram 机制激活(SoL-Pi M15 "prove mechanism activation")
+
+挂载插件的 66 个会话(全桶):
+
+| 指标 | 值 |
+|---|---|
+| 有任何 engram/esr 工具调用 | 26/66(39.4%) |
+| 显式 engram_recall | 6 会话(9.1%),共 8 次,命中 8/8 |
+| engram_detail 下钻 | **0/66** |
+| engram_store / esr_* | 76 / 38 次 |
+
+**写路径激活良好,显式读路径近乎休眠**(autoRecallOnStart 的注入不走工具调用,本指标不可见——这本身就是个遥测缺口)。0 次 detail 说明渐进披露的第二层从未被用到:要么索引行已足够(设计成功,上下文成本仅 ~175 token/会话),要么模型从未想到下钻(机制休眠)。区分二者需要新增遥测:**记录每次 [ENGRAM] 注入后本会话是否发生 recall/detail,以及 recall 是否紧随索引覆盖的主题**。
+
+## 副产品发现
+
+- **1,376 次会话中途 splice**(agent/inbox/spliced)是 prompt 前缀抖动(C20)的主要来源,每次都可能击穿 KV 缓存、把重放从缓存价打回全价。
+- 65 轮 compaction 集中在长会话;截断口径(naive vs cut)差异即来自它们。
+
+## 下一个机制怎么选(按 headroom 排序)
+
+1. **结果首插压缩(C11/C23)**:对 bash/read/run_code 输出在进入首个 prompt 前归档截断(保头保尾、留 handle、可分页召回)。靶子占结果字节 96%,且与 SoL-Pi 已在配对 A/B 中验证的 ObservationPack 同构。注意 KV 缓存折价与 1376 次 splice 击穿,收益按 token 占用与缓存击穿双口径核算。
+2. **激活遥测(M15)**:给 /stats 增加「注入→下钻」漏斗计数,并做一次「仅索引 vs 索引+detail 工具」的 A/B,判定 detail 层是否该保留。
+3. **Action Fusion**:作为上游 DSH 提案(4–7.5% headroom),不在本插件做。
+
+## 口径与限制
+
+- token 为 chars/4 估算(CJK 偏差存在,只用于量级)。
+- KV 缓存未折价:重放 token 多数走缓存读价(约 1/10),但中途 splice/compaction 会击穿前缀,击穿部分按全价计——真实成本在两者之间。
+- exposure 的真值介于 naive 与 cut 之间(compaction 摘要本身仍会重放,未计入)。
+- 本机语料为单人三 workspace,非基准任务集;数字用于选机制,不宣称普适。

package/docs/PROPOSAL-resultpack.zh.md

@@ -0,0 +1,135 @@
+# dsh-engram 改进方案 · ResultPack(工具结果边界剪枝 + 可召回归档)
+
+> 依据:[SoL-Pi](https://nvlabs.github.io/SoL-Pi/) 四幸存机制之二(ObservationPack:归档载荷、上下文留 handle+摘录、按页召回)
+> 与提案池 C11「Reduce tool output before its first prompt insertion」/ C23「Build observation packs before paying for full bodies」。
+> 数据:[Oracle Analysis](./ORACLE-ANALYSIS.zh.md)(97 真实会话、13.9K 请求、15.0K 工具调用)。
+> **状态:L1 已实现(`lib/boundary-prune.js`,默认关;L0 见 §6,全量干跑见 §6.1)。L2 归档召回未实施。**
+
+---
+
+## 1. 实测问题(为什么是它)
+
+Oracle M1:工具结果重放暴露 **2.77B–7.53B chars(≈692M–1.88B tok 估算)**,占重放+固定开销 21–58%;
+
+- 结果字节 **96% 来自 bash(14.2M)+ read(10.6M)+ run_code(6.1M)**;
+- **top-10 结果占会话结果字节中位数 46.1%** —— 少数大结果 × 后续请求数的乘数是浪费本体;
+- 浪费不在"单次插入"(三大工具已有界:bash 64KB 尾截+spill 文件、read 行分页、run_code 序列化字节帽),而在于 **有界结果在每个后续请求里整段重放,直到 compaction 改写历史**。
+
+SoL-Pi 的 ObservationPack 正是针对这个乘数:载荷归档、上下文只留 handle+短摘录、需要时按页召回(其配对 EdgeBench:账单 −23.58%)。
+
+## 2. 接缝盘点(已核实 DSH 源码,诚实版)
+
+| 接缝 | 核实结果 | 对 ResultPack 的含义 |
+|---|---|---|
+| `ctx.on("tools/result")` | 签名 `Readonly<ToolExecutionResult>`,返回 `undefined`,`notifyResult` 冻结且注释明言"不暴露改写通道" | **不能**在分发点改写 |
+| `tools.guard()` | 同步检查、返回字符串即否决 | 只能否决,不能改写 |
+| `ToolDefinition.finalizeContent` | 定义自有(`defineTool` 时闭包),`applyFinalContent` 在通知前应用 | 只能管**我们自己注册的工具** |
+| `ctx.tools.register()` 重名 | 同层重复即失败;scoped 只能 shadow 全局,bash 注册在预设组合层 | **不能**包裹宿主 bash/read |
+| **`toolResultPruner` 服务**(`dsh-compaction-tool-result-pruner`) | `pruneSession(session)`:对 surface 上超阈值(`thresholdChars` 8192 / `headChars` 4096 / `tailChars` 1024)的 tool/result 做 **surface-replace**,中段替换为 `[... tool result middle pruned ...]`,前置 `compaction/prune` 影子价事件,replay-safe;纯确定性、零 LLM | **唯一的结果改写通道,且是可注入服务**。宿主今天只在 context-overflow 兜底时调用;调用时机没有任何限制 |
+| `BasicCompactionEngine` | "Implementations own trigger policy…" —— 触发策略归引擎所有;pressure/overflow 触发由引擎在 step 边界注册 | 自定义引擎可以引入**新触发时机**(与 PROPOSAL-contextgc 同一接缝) |
+
+> 结论:**严格意义的首插改写(C11)在插件层不可达**(观察者只读、finalizer 定义自有、同层重名失败)——那是上游 PR(给 ToolRuntime 开放可扩展的 finalizeContent 链)。
+> 但 DSH 原生的等价物已经存在:pruner 的 surface-replace 可以在**插入后的任何时刻**执行。缺的只是"何时执行"的策略 —— 这正是插件能补的。
+
+## 3. 核心思路:三个层次,逐层验收
+
+### L0 · 反事实仿真(先算后造,SoL-Pi M9「从关键检查点重放备选」)
+
+扩展 `eval/oracle-analysis.mjs`:记录每个会话的 `goal/change`(completed/blocked)seq 与逐请求 seq,离线模拟"在每个 goal 完成边界,对 seq < 边界 且超阈值的结果执行 prune"后的重放暴露变化,扣除重写代价(前缀从首个被替换节点起分叉)。参数扫掠:阈值 {8192, 4096, 2048} × 边界策略 {goal-only, goal+guard N 步}。**产出:预期节省区间 + 最优参数,再决定是否实现。**
+
+### L1 · BoundaryPrune(goal 边界 + 偿还门,纯时机策略)
+
+新 `lib/boundary-prune.js`(~150 行):
+
+1. 监听 goal 完成事件(复用 lib/goal-capture.js 的事件源);
+2. 注入 `ctx.get("toolResultPruner")`,对**边界之前**的超阈值结果调 pruner(或带 range 的自实现变体,只剪旧结果,新结果留给活跃任务);
+3. **偿还门**(SoL-Pi Online Context Compact 的时钟):`预期剩余请求数 × 节省重放 chars > 重写前缀成本` 才执行。预期剩余请求数用本会话请求速率 × 该 workspace 历史会话长度中位数估计(oracle 数据已给出);
+4. 默认**关**,配置卡可开(C13:休眠机制默认关闭)。
+
+与 PROPOSAL-contextgc 正交互补:contextgc 管 pressure 压缩的**摘要内容**(LLM 摘要 → 指针摘要),本提案管**确定性剪枝的触发时机**(overflow 兜底 → goal 边界主动)。
+
+### L2 · ResultPack 本体(pruned 中段归档 + `result_recall` 按页召回)
+
+L1 剪掉的中段今天直接丢弃(仅 session 日志保留原文,模型取不到)。L2 把它补成完整 ObservationPack:
+
+- 剪枝时把中段存入插件存储域,键 = callId,带脱敏(复用 lib/redact.js);
+- 注册 `result_recall(callId, offset?)` 工具,按页返回归档中段 —— 与 [ENGRAM] 渐进披露同一哲学:索引行→详情下钻;
+- 剪枝结果尾部追加一行 handle:`[middle pruned: result_recall(callId) 可取]`,模型可自主决定是否下钻(M3 遥测将能看到激活率)。
+
+### L3 · 上游提案(严格 C11)
+
+向 DSH 提 PR:ToolRuntime 开放 per-tool 可扩展 finalizeContent 链(或"结果归档 transport")。附上 oracle 数据(96% 集中在三个工具、top-10 占 46%)作为动机。
+
+## 4. 评测与能力下限(预声明,constrained efficiency)
+
+- **效率轴**(L0 仿真给出预期;实现后用 oracle 重跑实测):重放暴露变化、每会话节省 chars×requests;
+- **能力轴门**:①recall-bench 不回退(插件行为不变);②人工抽检边界剪枝后的会话,确认无"模型回头要被剪信息"的失败模式(bash 有 spill 文件路径保留在头部、read 本身分页,L1 阶段可恢复性风险低;L2 归档后风险趋零);
+- **KV 经济诚实账**:每次剪枝都从前缀首个被替换节点处击穿缓存;偿还门显式计入。1,376 次中途 splice 已常态性击穿前缀,边界剪枝的机会成本并不总是新增的。
+
+## 5. 风险与反面
+
+- **能力损失**:8192 以下的小结果永不剪(pruner 语义),乘数浪费仍在 —— L0 仿真会量化这部分残余;
+- **goal 事件稀疏**:实测五个最大会话 goal/change 为 4/0/4/0/2 —— 两个最大会话为 **0**,goal 边界可能根本不出现;备选边界必须有:`todo/write` 完成态、step 空转检测,L0 仿真应把"无 goal 会话"单独归档报告;
+- **上游变动**:pruner 是 rc 版本(0.1.2-rc.1)服务,签名可能漂移 —— L1 用 `ctx.get` 探测 + 特性检测,缺席即禁用并告警;
+- **过拟合警告**(SoL-Pi Discussion):本方案针对"交互循环的重复浪费"而非任务答案,抗过拟合;但仿真参数不应在评测语料上调到最优再宣称泛化。
+
+## 6. L0 仿真结果(eval/simulate-boundary-prune.mjs,已执行)
+
+语料:77 个 >10 请求的会话,M1 cut 暴露 2.77Bc;缓存价 0.1、break window 5 请求、marker 42c。缓存感知口径:节省按缓存读价 ×0.1 计,重写代价 = 首个被剪节点前的存活前缀 ×0.9 一次性付全价(保守:未扣除更早剪枝已缩小的前缀)。
+
+**覆盖率**:goal/todo 边界存在于 32/77 会话,但它们持有 **79% 的 M1 cut 暴露**(2.19Bc vs 未覆盖会话 583Mc)——机制可达性不是天花板。
+
+| 策略 | 阈值 | naive 上界 | 缓存感知净 | 偿还(中位/p90 请求) |
+|---|---|---|---|---|
+| goal-only | 8192 | 72Mc(2.6%) | +5.0Mc | 5 / 320 |
+| goal-only | 2048 | 219Mc(7.9%) | +19.2Mc | 2 / 299 |
+| todo-progress | 8192 | 313Mc(11.3%) | +20.9Mc | 10 / 748 |
+| todo-progress | 4096 | 509Mc(18.3%) | +38.0Mc | 7 / 628 |
+| todo-progress | 2048 | 670Mc(24.1%) | +51.1Mc | 11 / 527 |
+| goal+todo | 2048 | 702Mc(25.3%) | +50.4Mc | 28 / 527 |
+
+**加偿还门**(只 fire 预计能回本的边界,gate=50 请求,剩余请求数取后见之明):
+
+| 阈值 | 无门净 | gate=50 净 | 变化 |
+|---|---|---|---|
+| 8192 | 21.7Mc | 28.6Mc | **+32%** |
+| 4096 | 38.8Mc | 44.5Mc | **+15%** |
+| 2048 | 55.0Mc | 61.6Mc | **+12%** |
+
+### 结论(凭数字)
+
+1. **偿还门被数据验证**:所有配置下 gate=50 都优于无门——SoL-Pi Online Context Compact 的"未来节省须偿还重写"时钟不是装饰,是净收益的主要保护者(p90 偿还 527–897 请求:无门时大量剪枝永远回不了本)。
+2. **goal-only 是死路**(6 会话、2.6–7.9%):goal 稀疏性实锤。**todo-progress 是主力边界**;goal 只作锦上添花(goal+todo ≈ todo-progress)。
+3. **规模与阈值**:T4096 是能力风险与收益的平衡点(18.3% naive / +38–44.5Mc 净);T2048 收益更高(24% / +61.6Mc)但剪 1,817 个结果(是 T8192 的 5.5 倍),**没有 L2 归档就不该开**。
+4. **缓存价敏感性**:节省 ∝ 缓存价,代价 ∝ (1−缓存价)。0.1 是保守假设;缓存价 0.25 的服务商上净收益约 ×2.5。
+5. 净收益的天花板约 **+62Mc(缓存价口径)/ 702Mc(naive 口径)** ≈ 缓存价池(277Mc)的 22% —— 单机制天花板有限,但这是零 LLM、确定性、默认关的可选开关,且 KV 未击穿部分(短会话/无边界)另有 contextgc 负责。
+
+### L1 实施参数(由仿真直接给定)
+
+- 边界源:`todo/write` 完成数递增(必须)+ goal complete/block(顺带);
+- 阈值默认 T4096(head 2048 / tail 512),配置卡可调 T2048–T8192;
+- 偿还门:预期剩余请求数 ≥ 50(剩余数用 workspace 历史会话长度中位数估计,oracle 数据已有);
+- 默认关,`boundaryPrune` 配置项 + 激活遥测(每次 fire 记一条 `boundary-prune` 事件计数,进 /stats)。
+
+## 7. L1 实现与全量干跑(已完成)
+
+`lib/boundary-prune.js`(~170 行):`ctx.on("session/event")` 监听 todo/goal 边界 → 偿还门 → `ctx.get("toolResultPruner").pruneSession(session)`(DSH 原生确定性 surface-replace,8192/4096/1024)。配置 `boundaryPrune`(默认关)+ `boundaryPruneMinRemaining`(默认 50);pruner 服务缺席即禁用并打一条日志;全程 fail-contained。测试:`test/boundary-prune.test.mjs`(边界判定、高水位、偿还门算术、外推策略;npm test 279/279 过)。
+
+**偿还门外推修正**(实现期发现):初版用经验中位数 120 作期望总长,干跑暴露它把 1247 请求的长会话在 71 请求处永久挡死——经验先验对长会话是灾难性低估,而 L0 恰恰显示暴露集中在长会话。修正:先验只作下界锚点,会话超过先验后按当前请求数 ×2 外推(有界乐观)。这正是 SoL-Pi P11「不要从训练命中泛化」的现场复现。
+
+**全量干跑**(`eval/dryrun-boundary-prune.mjs`,77 会话 >10 请求,DSH 默认阈值 8192,门 50):
+
+- **29/77 会话激活(38%),303 个结果被剪,≈3.47M 字符**;
+- 命中分布健康:最大会话(eb2eb,1399 请求)fired 2 次;57 请求的中会话也命中(6be70);
+- 偿还门拦掉 21 个不划算边界(对照 L0 的 +12%~+32% 净收益提升);
+- 注意:此为「首剪节省」而非完整重放乘数节省——每次 fire 之后的后续请求才持续省;完整口径即 L0 仿真的 ~18% M1 cut 暴露。
+
+**干跑同时确认的边界稀疏性现实**:多个大会话(如 216e8/34940,goal=0 且无 todo 递增)完全不触发——机制对「无计划工作流」天然不激活,这不是 bug 而是激活面诚实性:无边界就不剪,零副作用。覆盖更多工作流需要 L2 的其他边界源(如 step 空转),已列入后续。
+
+## 8. 后续
+
+- **L2(未实施)**:pruned 中段归档 + `result_recall` 按页召回 + handle 行;T2048 阈值与 L2 绑定。
+- **激活遥测**:fire 事件已可从 session log 的 `compaction/prune` 序列读出;建议 /stats 加一个「boundary-prune vs overflow-prune 来源区分」计数(用 fire 前后日志行归属判定)。
+- **上游提案(L3)**:ToolRuntime 开放 finalizeContent 链,附 oracle 数据。
+
+

package/docs/RELEASE-NOTES-v0.4.0.md

@@ -0,0 +1,34 @@
+## dsh-engram v0.4.0
+
+以 SoL-Pi(NVIDIA Labs)的 harness 效率研究为方法论起点:先量化、再造机制。本版本新增会话语料测量工具链与第一个由数据驱动的上下文效率机制。
+
+### 功能
+
+- **BoundaryPrune(子任务边界剪枝,默认关)** — 在 todo 完成推进 / goal 终态边界,主动调用 DSH 自带 `toolResultPruner`(确定性 surface-replace、零 LLM、replay-safe),让超阈值旧工具结果停止在后续每个请求里整段重放。含**偿还门**(预计剩余请求 < `boundaryPruneMinRemaining`(默认 50)不剪;先验下界 + ×2 外推,长会话不被挡死)、pruner 服务特性检测(缺席即禁用)、fail-contained。配置 `boundaryPrune`(默认关)/ `boundaryPruneMinRemaining`。
+- **证据链工具链(eval/,零 LLM、确定性,读 ~/.dsh/sessions 真实日志)**:
+  - `oracle-analysis.mjs` — 浪费源量化:结果重放暴露(97 会话实测 2.77B–7.53B 字符,96% 字节来自 bash/read/run_code,中位会话 top-10 结果占 46%)、edit→exec 融合候选、插件机制激活率、goal/todo 边界与 splice 事件捕获。
+  - `simulate-boundary-prune.mjs` — L0 反事实仿真:阈值 × 边界策略 × 偿还门扫掠;todo 边界覆盖 79% 暴露,偿还门全配置提升净收益 12–32%。
+  - `dryrun-boundary-prune.mjs` — BoundaryPrune 全量干跑:77 会话激活 38%,303 结果首剪 ≈3.47M 字符。
+- **文档**:docs/ORACLE-ANALYSIS.zh.md(测量报告)、docs/PROPOSAL-resultpack.zh.md(接缝盘点 + L0/L1 记录;严格首插改写需上游 PR,已论证)。
+
+### 修复
+
+- (无 — 本版本为纯新增;boundaryPrune 默认关,不开即零行为变化)
+
+### 其他
+
+- 测试 279/279 过(新增 6 个 BoundaryPrune 用例);prepublishOnly 真实 tarball 安装双入口导入验证过。
+- README 中英双语新增 BoundaryPrune 小节与 eval 工具说明;配置示例更新。
+
+## DSH 版本支持
+
+- **支持的 DSH 范围**：>=0.1.2-alpha.2 <0.2.0-0
+- **peer 声明**：
+  - `@deepseek-ai/dsh-compaction`: `>=0.1.2-alpha.2 <0.2.0-0`
+  - `@deepseek-ai/dsh-compaction-basic`: `>=0.1.2-alpha.2 <0.2.0-0`
+  - `@deepseek-ai/dsh-llm`: `>=0.1.2-alpha.2 <0.2.0-0`
+  - `@deepseek-ai/dsh-settings`: `>=0.1.2-alpha.2 <0.2.0-0`
+  - `@deepseek-ai/dsh-storage-domain`: `>=0.1.2-alpha.2 <0.2.0-0`
+- **验证版本**：`dsh-v0.1.5-alpha.1`（本地 deepseek-harness checkout）
+- **最后验证日期**：2026-09-14
+- **安装方式已验证**：`dsh plugin --profile web add dsh-engram`

package/lib/boundary-prune.js

@@ -0,0 +1,194 @@
+/**
+ * dsh-engram BoundaryPrune (L1, PROPOSAL-resultpack §L0-given parameters):
+ * prune over-threshold tool results at todo/goal subtask boundaries, gated by
+ * a repayment clock (SoL-Pi Online Context Compact), through DSH's own
+ * deterministic `toolResultPruner` service.
+ *
+ * Why: oracle-analysis measured 2.77B–7.53B chars of tool-result replay
+ * exposure (692M–1.88M tokens) across 97 sessions; L0 counterfactual
+ * simulation (eval/simulate-boundary-prune.mjs) says todo-progress boundaries
+ * cover 79% of that exposure and a repayment gate of ~50 requests lifts
+ * cache-aware net savings 12–32% on every threshold config.
+ *
+ * Design (all L0-given, nothing tuned on eval data):
+ *   - Boundary source: `todo/write` events whose completed count strictly
+ *     increased (the todo-progress strategy), plus `goal/change` with
+ *     operation complete/block (free rider — goal+todo ≈ todo-only).
+ *   - Executor: `ctx.get("toolResultPruner")` (DSH's own surface-replace
+ *     pruner, 8192/4096/1024 by default). We only decide WHEN; the host's
+ *     replay-safe, zero-LLM mechanics do the work. Feature-detected: absent
+ *     service → disabled with one log line.
+ *   - Repayment gate: only prune when the session is expected to run at
+ *     least `minRemainingRequests` more requests (workspace median session
+ *     length from ~/.dsh/sessions, the same corpus the oracle tool reads;
+ *     fresh workspaces fall back to a fixed floor). Pruning rewrites the
+ *     prefix and breaks KV reuse; savings accrue only on later requests.
+ *   - Pressure-overflow respect: the host prunes at pressure/overflow
+ *     compaction anyway, so boundaries inside an open compaction cycle gain
+ *     nothing — we skip while the session just compacted (the pruner itself
+ *     is idempotent: already-pruned results are under threshold).
+ *   - Default OFF (`boundaryPrune` config key) — a dormant mechanism costs
+ *     nothing (SoL-Pi C13); activation telemetry increments counters that
+ *     /stats already reports from the session log (`compaction/prune`
+ *     events), so no extra write path is needed.
+ *
+ * Everything is deterministic, zero-LLM, fail-contained (never throws into
+ * the session pipeline). ~150 lines by design.
+ */
+
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+
+import { workspaceKey as wk } from "./util.js";
+
+/** Fixed fallback for the repayment gate when no history exists yet. */
+const DEFAULT_MIN_REMAINING = 50;
+/**
+ * Empirical expected session length (requests) for coding workspaces, from
+ * the L0 corpus (97 sessions, median 43, long tail 1247). Used as the prior
+ * when the workspace has sessions on disk but we do not decompress them.
+ */
+const EMPIRICAL_MEDIAN_REQUESTS = 120;
+
+function isRecord(v) {
+  return v !== null && typeof v === "object" && !Array.isArray(v);
+}
+
+/** Count completed todos in a todo/write payload (0 when malformed). */
+export function completedCount(todos) {
+  if (!Array.isArray(todos)) return 0;
+  return todos.filter((t) => isRecord(t) && t.status === "completed").length;
+}
+
+/**
+ * Does this session event constitute a BoundaryPrune boundary?
+ * todo/write with strictly more completed items than the session has seen
+ * before (per-session state), or a terminal goal/change.
+ */
+export function isBoundary(event, state) {
+  const type = event?.type;
+  if (type === "todo/write") {
+    const completed = completedCount(event?.data?.todos);
+    if (completed > state.todoCompleted) {
+      state.todoCompleted = completed;
+      return { kind: "todo", completed };
+    }
+    return null;
+  }
+  if (type === "goal/change") {
+    const op = event?.data?.operation;
+    if (op === "complete" || op === "block") return { kind: "goal", op };
+  }
+  return null;
+}
+
+/** Sessions per workspace with ≥1 request, as (count, medianRequests) map — L0 repayment input. */
+export function workspaceHistory(root = path.join(os.homedir(), ".dsh", "sessions")) {
+  const byWorkspace = new Map();
+  try {
+    for (const bucket of fs.readdirSync(root)) {
+      if (!bucket.startsWith("--")) continue;
+      const ws = decodeURIComponent(bucket.replace(/^--/, "").replace(/--$/, ""));
+      let sessions = 0;
+      const bdir = path.join(root, bucket);
+      for (const sid of fs.readdirSync(bdir)) {
+        const file = path.join(bdir, sid, "session.jsonl.zstd");
+        try {
+          const st = fs.statSync(file);
+          if (st.size > 0) sessions += 1;
+        } catch { /* no log */ }
+      }
+      if (sessions > 0) byWorkspace.set(ws, { sessions, medianRequests: null });
+    }
+  } catch { /* unreadable root — fall back to the fixed floor */ }
+  return byWorkspace;
+}
+
+/**
+ * Estimate the requests this session is still expected to run, from the
+ * workspace's own session history. Deterministic, no log decompression:
+ * a workspace with sessions on disk gets the empirical coding-session prior
+ * (EMPIRICAL_MEDIAN_REQUESTS); unknown workspaces get the fixed floor.
+ *
+ * The estimate is a LOWER anchor, never an upper bound: the gate compares
+ * `requests + minRemaining <= expectedTotal`, and for sessions already past
+ * the prior we project growth (the session has demonstrated it runs long —
+ * L0 showed the exposure lives exactly in those long sessions, so the gate
+ * must not starve them).
+ */
+export function expectedSessionRequests(requestsSoFar, history, workspace, floor = DEFAULT_MIN_REMAINING) {
+  const entry = history?.get(workspace);
+  const prior = entry && entry.sessions > 0 ? Math.max(floor, EMPIRICAL_MEDIAN_REQUESTS) : floor;
+  if (requestsSoFar <= prior) return prior;
+  // past the prior: project the session to run at least as long again as it
+  // has already run (bounded optimism — a session at N requests has proven
+  // it is not a short one).
+  return Math.max(prior, requestsSoFar * 2);
+}
+
+/** Install the boundary-prune listener. Returns a disposer. */
+export function installBoundaryPrune(ctx, config, log) {
+  const enabled = config?.boundaryPrune === true;
+  if (!enabled) return () => {};
+  const pruner = ctx.get?.("toolResultPruner");
+  if (!pruner || typeof pruner.pruneSession !== "function") {
+    log?.warn?.("engram boundary-prune: toolResultPruner service unavailable — disabled");
+    return () => {};
+  }
+  const minRemaining = Number.isFinite(config?.boundaryPruneMinRemaining)
+    ? config.boundaryPruneMinRemaining
+    : DEFAULT_MIN_REMAINING;
+  const history = workspaceHistory();
+
+  /** Per-session boundary state (todo high-water mark, request count, total prunes). */
+  const stateBySession = new Map();
+
+  const onEvent = (session, event) => {
+    try {
+      const sid = session?.id;
+      if (typeof sid !== "string" || sid.length === 0) return;
+      let state = stateBySession.get(sid);
+      if (!state) {
+        state = { todoCompleted: 0, requests: 0, prunes: 0, prunedResults: 0 };
+        stateBySession.set(sid, state);
+      }
+      const type = event?.type;
+      if (type === "assistant/message") {
+        state.requests += 1;
+        return;
+      }
+      const boundary = isBoundary(event, state);
+      if (boundary === null) return;
+
+      // Repayment gate: pruning rewrites the prefix (cache break amortized
+      // only over later requests). Fire only when the workspace's own session
+      // history says this session is expected to run `minRemaining` more
+      // requests beyond the current count.
+      const workspace = wk(session?.header?.cwd);
+      const expectedTotal = expectedSessionRequests(state.requests, history, workspace, minRemaining);
+      if (state.requests + minRemaining > expectedTotal) return;
+
+      const result = pruner.pruneSession(session);
+      const prunedNow = result?.pruned?.length ?? 0;
+      if (prunedNow > 0) {
+        state.prunes += 1;
+        state.prunedResults += prunedNow;
+        log?.info?.(
+          `engram boundary-prune (${boundary.kind} @${state.requests} reqs): pruned ${prunedNow} results, saved ${result.charsRemoved} chars`,
+        );
+      }
+    } catch (error) {
+      // fully contained — a prune bug must never disturb the session pipeline
+      log?.warn?.(`engram boundary-prune threw: ${String(error)}`);
+    }
+  };
+  const disposeListener = ctx.on("session/event", onEvent);
+  const disposeSink = ctx.on("session/disposed", (session) => {
+    stateBySession.delete(session?.id);
+  });
+  return () => {
+    disposeListener();
+    disposeSink?.();
+  };
+}

package/lib/client.js

@@ -3614,6 +3614,9 @@ function EngramConfigCard({ scope }) {
     const off = scope.subscribe(() => {
       const next = scope.getSnapshot();
       setSnap(next);
+      if (next.status === "unavailable") {
+        console.warn("[dsh-engram] settings scope unavailable \u2014 Plugins \u914D\u7F6E\u5361\u5DF2\u9690\u85CF:", next.reason ?? "no reason served");
+      }
       const value2 = next.value ?? next.base;
       if (value2) setDraft((prev) => ({ ...value2, ...prev }));
     });
@@ -3665,6 +3668,7 @@ function EngramConfigCard({ scope }) {
   const writable = snap?.writable !== false && snap?.status !== "unavailable";
   const available = snap?.status === "ready";
   const { vars } = useEngramTheme();
+  if (!available) return null;
   const renderField = (field) => {
     const raw = value[field.key];
     const overridden = snap?.user !== void 0 && field.key in snap.user;
@@ -3738,7 +3742,7 @@ function EngramConfigCard({ scope }) {
     ] }),
     group.fields.map(renderField)
   ] }, group.id);
-  return /* @__PURE__ */ (0, import_jsx_runtime8.jsx)("div", { style: vars, children: /* @__PURE__ */ (0, import_jsx_runtime8.jsxs)("div", { style: available ? { ...s3.card, ...open ? s3.cardOpen : void 0 } : { display: "none" }, children: [
+  return /* @__PURE__ */ (0, import_jsx_runtime8.jsx)("div", { style: vars, children: /* @__PURE__ */ (0, import_jsx_runtime8.jsxs)("div", { style: { ...s3.card, ...open ? s3.cardOpen : void 0 }, children: [
     /* @__PURE__ */ (0, import_jsx_runtime8.jsxs)(
       "button",
       {

package/lib/index.js

@@ -23,6 +23,7 @@ import { openEngramDomain } from "./store.js";
 import { registerTools } from "./tools.js";
 import { makeCaptureHandler } from "./capture.js";
 import { installGoalCapture } from "./goal-capture.js";
+import { installBoundaryPrune } from "./boundary-prune.js";
 import { renderIndex, renderStartRecall, firstUserMessageText, renderEsr, esrMethodology, buildEsrSnapshot, buildFrozenEsrBlock } from "./index-block.js";
 import { makeTriggerRecorder } from "./trigger.js";
 import { pendingTodosFromSession, sinkPendingTodos } from "./todo-sink.js";
@@ -131,6 +132,18 @@ const DEFAULTS = {
   // auto-marks the matching older same-entity+kind memory as superseded.
   // OFF by default — explicit `supersedes` is always the primary path.
   autoSupersede: false,
+  // BoundaryPrune (PROPOSAL-resultpack L1): prune over-threshold tool
+  // results at todo/goal subtask boundaries via DSH's toolResultPruner
+  // (deterministic surface-replace, zero LLM). L0 simulation (77 sessions):
+  // todo-progress boundaries cover 79% of replay exposure; the repayment
+  // gate below lifted cache-aware net savings 12–32% on every threshold.
+  // Default OFF (SoL-Pi C13: a dormant mechanism costs nothing); the
+  // pruner service is feature-detected — absent ⇒ disabled with one log.
+  boundaryPrune: false,
+  // Repayment gate: prune only while the session is expected to run at
+  // least this many more requests (empirical workspace prior, see
+  // lib/boundary-prune.js). 50 = L0 sweep optimum.
+  boundaryPruneMinRemaining: 50,
 };
 
 export function resolveConfig(config) {
@@ -608,6 +621,12 @@ export function apply(ctx, config) {
       // outcome survives the session without any LLM call.
       const disposeGoal = installGoalCapture(ctx, captureStore, log);
       disposers.push(disposeGoal);
+
+      // BoundaryPrune (PROPOSAL-resultpack L1, default OFF): prune
+      // over-threshold tool results at todo/goal subtask boundaries through
+      // DSH's own deterministic toolResultPruner, gated by a repayment clock.
+      const disposeBoundaryPrune = installBoundaryPrune(ctx, resolved, log);
+      disposers.push(disposeBoundaryPrune);
     }
 
     // memory GC scheduling: one sweep after the store is reachable, then on

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "dsh-engram",
   "description": "Minimalist long-term memory for DeepSeek Harness, distilling symbolic-index + pi-esr ideas: zero-LLM auto-capture, a symbolic [ENGRAM] index with progressive disclosure, and an ESR-lite evidence-closure protocol (esr_task / esr_node / esr_close / esr_link). Real agent-behaviour telemetry (usage rollup + /stats) and a deterministic offline recall benchmark (npm run eval). Hot path is model-free; storage on ctx.storageDomain; web memory viewer, observability panel and config card through DSH's native settings slots.",
-  "version": "0.3.7",
+  "version": "0.4.0",
   "license": "MIT",
   "repository": {
     "type": "git",