dsh-engram 0.3.7 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,139 @@
1
+ # dsh-engram 改进方案 · ResultPack(工具结果边界剪枝 + 可召回归档)
2
+
3
+ > 依据:[SoL-Pi](https://nvlabs.github.io/SoL-Pi/) 四幸存机制之二(ObservationPack:归档载荷、上下文留 handle+摘录、按页召回)
4
+ > 与提案池 C11「Reduce tool output before its first prompt insertion」/ C23「Build observation packs before paying for full bodies」。
5
+ > 数据:[Oracle Analysis](./ORACLE-ANALYSIS.zh.md)(97 真实会话、13.9K 请求、15.0K 工具调用)。
6
+ > **状态:L1 已实现(`lib/boundary-prune.js`,默认关;L0 见 §6,全量干跑见 §6.1)。L2 已实现(`lib/result-pack.js`,默认关)。**
7
+
8
+ ---
9
+
10
+ ## 1. 实测问题(为什么是它)
11
+
12
+ Oracle M1:工具结果重放暴露 **2.77B–7.53B chars(≈692M–1.88B tok 估算)**,占重放+固定开销 21–58%;
13
+
14
+ - 结果字节 **96% 来自 bash(14.2M)+ read(10.6M)+ run_code(6.1M)**;
15
+ - **top-10 结果占会话结果字节中位数 46.1%** —— 少数大结果 × 后续请求数的乘数是浪费本体;
16
+ - 浪费不在"单次插入"(三大工具已有界:bash 64KB 尾截+spill 文件、read 行分页、run_code 序列化字节帽),而在于 **有界结果在每个后续请求里整段重放,直到 compaction 改写历史**。
17
+
18
+ SoL-Pi 的 ObservationPack 正是针对这个乘数:载荷归档、上下文只留 handle+短摘录、需要时按页召回(其配对 EdgeBench:账单 −23.58%)。
19
+
20
+ ## 2. 接缝盘点(已核实 DSH 源码,诚实版)
21
+
22
+ | 接缝 | 核实结果 | 对 ResultPack 的含义 |
23
+ |---|---|---|
24
+ | `ctx.on("tools/result")` | 签名 `Readonly<ToolExecutionResult>`,返回 `undefined`,`notifyResult` 冻结且注释明言"不暴露改写通道" | **不能**在分发点改写 |
25
+ | `tools.guard()` | 同步检查、返回字符串即否决 | 只能否决,不能改写 |
26
+ | `ToolDefinition.finalizeContent` | 定义自有(`defineTool` 时闭包),`applyFinalContent` 在通知前应用 | 只能管**我们自己注册的工具** |
27
+ | `ctx.tools.register()` 重名 | 同层重复即失败;scoped 只能 shadow 全局,bash 注册在预设组合层 | **不能**包裹宿主 bash/read |
28
+ | **`toolResultPruner` 服务**(`dsh-compaction-tool-result-pruner`) | `pruneSession(session)`:对 surface 上超阈值(`thresholdChars` 8192 / `headChars` 4096 / `tailChars` 1024)的 tool/result 做 **surface-replace**,中段替换为 `[... tool result middle pruned ...]`,前置 `compaction/prune` 影子价事件,replay-safe;纯确定性、零 LLM | **唯一的结果改写通道,且是可注入服务**。宿主今天只在 context-overflow 兜底时调用;调用时机没有任何限制 |
29
+ | `BasicCompactionEngine` | "Implementations own trigger policy…" —— 触发策略归引擎所有;pressure/overflow 触发由引擎在 step 边界注册 | 自定义引擎可以引入**新触发时机**(与 PROPOSAL-contextgc 同一接缝) |
30
+
31
+ > 结论:**严格意义的首插改写(C11)在插件层不可达**(观察者只读、finalizer 定义自有、同层重名失败)——那是上游 PR(给 ToolRuntime 开放可扩展的 finalizeContent 链)。
32
+ > 但 DSH 原生的等价物已经存在:pruner 的 surface-replace 可以在**插入后的任何时刻**执行。缺的只是"何时执行"的策略 —— 这正是插件能补的。
33
+
34
+ ## 3. 核心思路:三个层次,逐层验收
35
+
36
+ ### L0 · 反事实仿真(先算后造,SoL-Pi M9「从关键检查点重放备选」)
37
+
38
+ 扩展 `eval/oracle-analysis.mjs`:记录每个会话的 `goal/change`(completed/blocked)seq 与逐请求 seq,离线模拟"在每个 goal 完成边界,对 seq < 边界 且超阈值的结果执行 prune"后的重放暴露变化,扣除重写代价(前缀从首个被替换节点起分叉)。参数扫掠:阈值 {8192, 4096, 2048} × 边界策略 {goal-only, goal+guard N 步}。**产出:预期节省区间 + 最优参数,再决定是否实现。**
39
+
40
+ ### L1 · BoundaryPrune(goal 边界 + 偿还门,纯时机策略)
41
+
42
+ 新 `lib/boundary-prune.js`(~150 行):
43
+
44
+ 1. 监听 goal 完成事件(复用 lib/goal-capture.js 的事件源);
45
+ 2. 注入 `ctx.get("toolResultPruner")`,对**边界之前**的超阈值结果调 pruner(或带 range 的自实现变体,只剪旧结果,新结果留给活跃任务);
46
+ 3. **偿还门**(SoL-Pi Online Context Compact 的时钟):`预期剩余请求数 × 节省重放 chars > 重写前缀成本` 才执行。预期剩余请求数用本会话请求速率 × 该 workspace 历史会话长度中位数估计(oracle 数据已给出);
47
+ 4. 默认**关**,配置卡可开(C13:休眠机制默认关闭)。
48
+
49
+ 与 PROPOSAL-contextgc 正交互补:contextgc 管 pressure 压缩的**摘要内容**(LLM 摘要 → 指针摘要),本提案管**确定性剪枝的触发时机**(overflow 兜底 → goal 边界主动)。
50
+
51
+ ### L2 · ResultPack 本体(pruned 中段归档 + `result_recall` 按页召回)
52
+
53
+ L1 剪掉的中段今天直接丢弃(仅 session 日志保留原文,模型取不到)。L2 把它补成完整 ObservationPack:
54
+
55
+ - 剪枝时把中段存入插件存储域,键 = callId,带脱敏(复用 lib/redact.js);
56
+ - 注册 `result_recall(callId, offset?)` 工具,按页返回归档中段 —— 与 [ENGRAM] 渐进披露同一哲学:索引行→详情下钻;
57
+ - 剪枝结果尾部追加一行 handle:`[middle pruned: result_recall(callId) 可取]`,模型可自主决定是否下钻(M3 遥测将能看到激活率)。
58
+
59
+ > **已实现(2026-09-23,§8)**:实现细节与设计中两处修正——①归档落 `~/.dsh/engram/resultpack/<callId>.json`(独立文件而非域表,大块内容不撑 domain JSON);②handle 走宿主 surface-replace 协议(`session.append("tool/result", …, surfaceOp replace)`),replay-safe 零 LLM;脱敏先于持久化。17 项单测 + L1→L2 钩子 3 项 + 进程内端到端(归档→分页→缺档)验证通过。
60
+
61
+ ### L3 · 上游提案(严格 C11)
62
+
63
+ 向 DSH 提 PR:ToolRuntime 开放 per-tool 可扩展 finalizeContent 链(或"结果归档 transport")。附上 oracle 数据(96% 集中在三个工具、top-10 占 46%)作为动机。
64
+
65
+ ## 4. 评测与能力下限(预声明,constrained efficiency)
66
+
67
+ - **效率轴**(L0 仿真给出预期;实现后用 oracle 重跑实测):重放暴露变化、每会话节省 chars×requests;
68
+ - **能力轴门**:①recall-bench 不回退(插件行为不变);②人工抽检边界剪枝后的会话,确认无"模型回头要被剪信息"的失败模式(bash 有 spill 文件路径保留在头部、read 本身分页,L1 阶段可恢复性风险低;L2 归档后风险趋零);
69
+ - **KV 经济诚实账**:每次剪枝都从前缀首个被替换节点处击穿缓存;偿还门显式计入。1,376 次中途 splice 已常态性击穿前缀,边界剪枝的机会成本并不总是新增的。
70
+
71
+ ## 5. 风险与反面
72
+
73
+ - **能力损失**:8192 以下的小结果永不剪(pruner 语义),乘数浪费仍在 —— L0 仿真会量化这部分残余;
74
+ - **goal 事件稀疏**:实测五个最大会话 goal/change 为 4/0/4/0/2 —— 两个最大会话为 **0**,goal 边界可能根本不出现;备选边界必须有:`todo/write` 完成态、step 空转检测,L0 仿真应把"无 goal 会话"单独归档报告;
75
+ - **上游变动**:pruner 是 rc 版本(0.1.2-rc.1)服务,签名可能漂移 —— L1 用 `ctx.get` 探测 + 特性检测,缺席即禁用并告警;
76
+ - **过拟合警告**(SoL-Pi Discussion):本方案针对"交互循环的重复浪费"而非任务答案,抗过拟合;但仿真参数不应在评测语料上调到最优再宣称泛化。
77
+
78
+ ## 6. L0 仿真结果(eval/simulate-boundary-prune.mjs,已执行)
79
+
80
+ 语料:77 个 >10 请求的会话,M1 cut 暴露 2.77Bc;缓存价 0.1、break window 5 请求、marker 42c。缓存感知口径:节省按缓存读价 ×0.1 计,重写代价 = 首个被剪节点前的存活前缀 ×0.9 一次性付全价(保守:未扣除更早剪枝已缩小的前缀)。
81
+
82
+ **覆盖率**:goal/todo 边界存在于 32/77 会话,但它们持有 **79% 的 M1 cut 暴露**(2.19Bc vs 未覆盖会话 583Mc)——机制可达性不是天花板。
83
+
84
+ | 策略 | 阈值 | naive 上界 | 缓存感知净 | 偿还(中位/p90 请求) |
85
+ |---|---|---|---|---|
86
+ | goal-only | 8192 | 72Mc(2.6%) | +5.0Mc | 5 / 320 |
87
+ | goal-only | 2048 | 219Mc(7.9%) | +19.2Mc | 2 / 299 |
88
+ | todo-progress | 8192 | 313Mc(11.3%) | +20.9Mc | 10 / 748 |
89
+ | todo-progress | 4096 | 509Mc(18.3%) | +38.0Mc | 7 / 628 |
90
+ | todo-progress | 2048 | 670Mc(24.1%) | +51.1Mc | 11 / 527 |
91
+ | goal+todo | 2048 | 702Mc(25.3%) | +50.4Mc | 28 / 527 |
92
+
93
+ **加偿还门**(只 fire 预计能回本的边界,gate=50 请求,剩余请求数取后见之明):
94
+
95
+ | 阈值 | 无门净 | gate=50 净 | 变化 |
96
+ |---|---|---|---|
97
+ | 8192 | 21.7Mc | 28.6Mc | **+32%** |
98
+ | 4096 | 38.8Mc | 44.5Mc | **+15%** |
99
+ | 2048 | 55.0Mc | 61.6Mc | **+12%** |
100
+
101
+ ### 结论(凭数字)
102
+
103
+ 1. **偿还门被数据验证**:所有配置下 gate=50 都优于无门——SoL-Pi Online Context Compact 的"未来节省须偿还重写"时钟不是装饰,是净收益的主要保护者(p90 偿还 527–897 请求:无门时大量剪枝永远回不了本)。
104
+ 2. **goal-only 是死路**(6 会话、2.6–7.9%):goal 稀疏性实锤。**todo-progress 是主力边界**;goal 只作锦上添花(goal+todo ≈ todo-progress)。
105
+ 3. **规模与阈值**:T4096 是能力风险与收益的平衡点(18.3% naive / +38–44.5Mc 净);T2048 收益更高(24% / +61.6Mc)但剪 1,817 个结果(是 T8192 的 5.5 倍),**没有 L2 归档就不该开**。
106
+ 4. **缓存价敏感性**:节省 ∝ 缓存价,代价 ∝ (1−缓存价)。0.1 是保守假设;缓存价 0.25 的服务商上净收益约 ×2.5。
107
+ 5. 净收益的天花板约 **+62Mc(缓存价口径)/ 702Mc(naive 口径)** ≈ 缓存价池(277Mc)的 22% —— 单机制天花板有限,但这是零 LLM、确定性、默认关的可选开关,且 KV 未击穿部分(短会话/无边界)另有 contextgc 负责。
108
+
109
+ ### L1 实施参数(由仿真直接给定)
110
+
111
+ - 边界源:`todo/write` 完成数递增(必须)+ goal complete/block(顺带);
112
+ - 阈值默认 T4096(head 2048 / tail 512),配置卡可调 T2048–T8192;
113
+ - 偿还门:预期剩余请求数 ≥ 50(剩余数用 workspace 历史会话长度中位数估计,oracle 数据已有);
114
+ - 默认关,`boundaryPrune` 配置项 + 激活遥测(每次 fire 记一条 `boundary-prune` 事件计数,进 /stats)。
115
+
116
+ ## 7. L1 实现与全量干跑(已完成)
117
+
118
+ `lib/boundary-prune.js`(~170 行):`ctx.on("session/event")` 监听 todo/goal 边界 → 偿还门 → `ctx.get("toolResultPruner").pruneSession(session)`(DSH 原生确定性 surface-replace,8192/4096/1024)。配置 `boundaryPrune`(默认关)+ `boundaryPruneMinRemaining`(默认 50);pruner 服务缺席即禁用并打一条日志;全程 fail-contained。测试:`test/boundary-prune.test.mjs`(边界判定、高水位、偿还门算术、外推策略;npm test 279/279 过)。
119
+
120
+ **偿还门外推修正**(实现期发现):初版用经验中位数 120 作期望总长,干跑暴露它把 1247 请求的长会话在 71 请求处永久挡死——经验先验对长会话是灾难性低估,而 L0 恰恰显示暴露集中在长会话。修正:先验只作下界锚点,会话超过先验后按当前请求数 ×2 外推(有界乐观)。这正是 SoL-Pi P11「不要从训练命中泛化」的现场复现。
121
+
122
+ **全量干跑**(`eval/dryrun-boundary-prune.mjs`,77 会话 >10 请求,DSH 默认阈值 8192,门 50):
123
+
124
+ - **29/77 会话激活(38%),303 个结果被剪,≈3.47M 字符**;
125
+ - 命中分布健康:最大会话(eb2eb,1399 请求)fired 2 次;57 请求的中会话也命中(6be70);
126
+ - 偿还门拦掉 21 个不划算边界(对照 L0 的 +12%~+32% 净收益提升);
127
+ - 注意:此为「首剪节省」而非完整重放乘数节省——每次 fire 之后的后续请求才持续省;完整口径即 L0 仿真的 ~18% M1 cut 暴露。
128
+
129
+ **干跑同时确认的边界稀疏性现实**:多个大会话(如 216e8/34940,goal=0 且无 todo 递增)完全不触发——机制对「无计划工作流」天然不激活,这不是 bug 而是激活面诚实性:无边界就不剪,零副作用。覆盖更多工作流需要 L2 的其他边界源(如 step 空转),已列入后续。
130
+
131
+ ## 8. 后续
132
+
133
+ - ~~**L2**~~ **已实现**(2026-09-23, `lib/result-pack.js`):pruned 中段归档 + `result_recall` 按页召回 + handle 行;默认关(`resultPack`,须与 `boundaryPrune` 同开),保留 7 天走 GC;T2048 阈值与 L2 绑定,**现已满足开低阈值的归档前提**。
134
+ - ~~**激活遥测**~~ **已实现**(2026-09-23):`/stats` 与 GUI overview 返回 `activation: { boundaryPrune, recall }` —— BoundaryPrune fire 每次落盘 `~/.dsh/engram/boundary-prune.status.json`(fires/prunedResults/charsRemoved/byKind,与 resultPack 无关),[RECALL] 注入计数落盘 `recall-injections.json`,与 session-log 已统计的 recall 漏斗(查询→命中→detail 下钻)闭合(SoL-Pi M15/C13)。
135
+ - ~~**step 空转边界**~~ **已实现**(2026-09-23):`boundaryPruneIdleCadence`(默认 0=关)——请求计数边界,覆盖无 todo/goal 会话。侦察 105 会话:65 个无边界、含 5.23M over-threshold 暴露(38%);反事实仿真(cadence 50):激活面 36→64 会话、首剪节省 4.23M→7.84M(+85%),仍受偿还门约束。
136
+ - **T2048 落地**(2026-09-23):`boundaryPruneThresholdChars`/`HeadChars`/`TailChars`(默认 0=宿主)——插件以 `Object.create(pruner)` 原型克隆宿主 pruner 实例、仅覆盖 config,宿主不变。真实语料干跑:T2048 H1024/T256 首剪 10.14M vs 宿主默认 4.23M(+2.4x)。
137
+ - **上游提案(L3)**:ToolRuntime 开放 finalizeContent 链,附 oracle 数据。
138
+
139
+
@@ -0,0 +1,34 @@
1
+ ## dsh-engram v0.4.0
2
+
3
+ 以 SoL-Pi(NVIDIA Labs)的 harness 效率研究为方法论起点:先量化、再造机制。本版本新增会话语料测量工具链与第一个由数据驱动的上下文效率机制。
4
+
5
+ ### 功能
6
+
7
+ - **BoundaryPrune(子任务边界剪枝,默认关)** — 在 todo 完成推进 / goal 终态边界,主动调用 DSH 自带 `toolResultPruner`(确定性 surface-replace、零 LLM、replay-safe),让超阈值旧工具结果停止在后续每个请求里整段重放。含**偿还门**(预计剩余请求 < `boundaryPruneMinRemaining`(默认 50)不剪;先验下界 + ×2 外推,长会话不被挡死)、pruner 服务特性检测(缺席即禁用)、fail-contained。配置 `boundaryPrune`(默认关)/ `boundaryPruneMinRemaining`。
8
+ - **证据链工具链(eval/,零 LLM、确定性,读 ~/.dsh/sessions 真实日志)**:
9
+ - `oracle-analysis.mjs` — 浪费源量化:结果重放暴露(97 会话实测 2.77B–7.53B 字符,96% 字节来自 bash/read/run_code,中位会话 top-10 结果占 46%)、edit→exec 融合候选、插件机制激活率、goal/todo 边界与 splice 事件捕获。
10
+ - `simulate-boundary-prune.mjs` — L0 反事实仿真:阈值 × 边界策略 × 偿还门扫掠;todo 边界覆盖 79% 暴露,偿还门全配置提升净收益 12–32%。
11
+ - `dryrun-boundary-prune.mjs` — BoundaryPrune 全量干跑:77 会话激活 38%,303 结果首剪 ≈3.47M 字符。
12
+ - **文档**:docs/ORACLE-ANALYSIS.zh.md(测量报告)、docs/PROPOSAL-resultpack.zh.md(接缝盘点 + L0/L1 记录;严格首插改写需上游 PR,已论证)。
13
+
14
+ ### 修复
15
+
16
+ - (无 — 本版本为纯新增;boundaryPrune 默认关,不开即零行为变化)
17
+
18
+ ### 其他
19
+
20
+ - 测试 279/279 过(新增 6 个 BoundaryPrune 用例);prepublishOnly 真实 tarball 安装双入口导入验证过。
21
+ - README 中英双语新增 BoundaryPrune 小节与 eval 工具说明;配置示例更新。
22
+
23
+ ## DSH 版本支持
24
+
25
+ - **支持的 DSH 范围**:>=0.1.2-alpha.2 <0.2.0-0
26
+ - **peer 声明**:
27
+ - `@deepseek-ai/dsh-compaction`: `>=0.1.2-alpha.2 <0.2.0-0`
28
+ - `@deepseek-ai/dsh-compaction-basic`: `>=0.1.2-alpha.2 <0.2.0-0`
29
+ - `@deepseek-ai/dsh-llm`: `>=0.1.2-alpha.2 <0.2.0-0`
30
+ - `@deepseek-ai/dsh-settings`: `>=0.1.2-alpha.2 <0.2.0-0`
31
+ - `@deepseek-ai/dsh-storage-domain`: `>=0.1.2-alpha.2 <0.2.0-0`
32
+ - **验证版本**:`dsh-v0.1.5-alpha.1`(本地 deepseek-harness checkout)
33
+ - **最后验证日期**:2026-09-14
34
+ - **安装方式已验证**:`dsh plugin --profile web add dsh-engram`
@@ -0,0 +1,58 @@
1
+ ## dsh-engram v0.5.0
2
+
3
+ DSH 0.2.x 兼容修复版:`0.1.7` 起 `@deepseek-ai/dsh-settings` 移除了
4
+ `settings.installSection`,设置卡改为**条目式 volatile Config** 模型;本版本把设置
5
+ 绑定迁移到新模型(entry-id `engram`),并把声明范围修到能覆盖 DSH `0.2.x` 全系,
6
+ 使插件在最新 DSH 上恢复加载与设置卡读写。
7
+
8
+ ### 功能
9
+
10
+ - **DSH ≥0.1.7 / 0.2.x 设置模型支持** — 插件 `Config`(`lib/index.js` 导出的
11
+ `Config`)声明全部 48 个 DEFAULTS 键;27 个 GUI 旋钮为 `.volatile()` 字段,
12
+ 设置表单由 DSH 以 profile 条目 id `engram` 对外提供,卡片编辑经
13
+ `loader/volatile-update` **热提交**到运行配置(新会话即时生效,已冻结块保持
14
+ 冻结——插件既有契约)。其余键普通声明,profile patch 里的内部旋钮不再被校验丢弃。
15
+ - **双代际特性检测** — `installEngramSettings` 自动识别宿主:DSH `≤0.1.5`
16
+ 回退旧的 `settings.installSection` 命名空间(`dsh-engram`),行为与 0.4.0 完全
17
+ 一致;`≥0.1.7` / `0.2.x` 走新模型(无挂载调用)。HTTP wire 命名空间两代际都保持
18
+ `dsh-engram`,**客户端 bundle 零改动**。
19
+
20
+ ### 修复
21
+
22
+ - **版本范围**:`engines.dsh` + 5 个 `@deepseek-ai/dsh-*` peer 从
23
+ `>=0.1.2-alpha.2 <0.2.0-0` 修正为 `>=0.1.2-alpha.2 <0.3.0-0`——原范围把
24
+ DSH `0.2.0-rc.x` 全部排除(semver 预发布窗口),插件在最新 DSH 上被兼容门槛拒绝。
25
+ 新范围已用 DSH 自身的 `includePrerelease` 检查实测:`0.2.0-rc.2` 与整个 `0.2.x`
26
+ 线放行,`0.3.0-0` 拒绝。
27
+ - **宿主设置崩溃**:移除对已不存在的 `settings.installSection` 的无条件调用
28
+ (其在 `dsh-settings 0.1.7-alpha.2` 起被删除),改为特性检测。
29
+ - **`boundaryPruneThresholdChars` 默认 `null` → `0`**:与设置卡长期使用的
30
+ “0=用宿主 8192” 语义一致(`0` 与 `null` 在剪枝判定中等价),并让 describe 的
31
+ 表单值完整包含该键。
32
+
33
+ ### 其他
34
+
35
+ - 测试 329/329 通过(新增 `test/settings-v2.test.mjs` 6 个用例,覆盖 volatile
36
+ Config 校验/解包/热同步与两代际安装路径;`test/api.test.mjs` 随 ns 语义更新)。
37
+ - settings 投影与写路径针对真实 `dsh-settings 0.2.0-rc.2` 工具函数
38
+ (`volatileForm`/`projectForm`/`isVolatilePath`/`plainConfig`)验证;
39
+ 真实 cordis 4.0.4 集成验证 Config 校验与热提交。
40
+ - **真机烟测通过**(隔离 profile,`dsh 0.2.0-rc.2`,`--port 3099`):插件启动无
41
+ 版本拒绝、`/api/dsh-engram/settings` 提供 27 键表单、
42
+ GET → PUT(mutate,revision 0→1)→ unset 往返落盘并读回。
43
+ - `scripts/prepublish-check.mjs` 随新导出面与 peer 范围更新,`prepublishOnly`
44
+ 在 0.2.x peer 集上通过。
45
+
46
+ ## DSH 版本支持
47
+
48
+ - **支持的 DSH 范围**:>=0.1.2-alpha.2 <0.3.0-0
49
+ - **peer 声明**:
50
+ - `@deepseek-ai/dsh-compaction`: `>=0.1.2-alpha.2 <0.3.0-0`
51
+ - `@deepseek-ai/dsh-compaction-basic`: `>=0.1.2-alpha.2 <0.3.0-0`
52
+ - `@deepseek-ai/dsh-llm`: `>=0.1.2-alpha.2 <0.3.0-0`
53
+ - `@deepseek-ai/dsh-settings`: `>=0.1.2-alpha.2 <0.3.0-0`
54
+ - `@deepseek-ai/dsh-storage-domain`: `>=0.1.2-alpha.2 <0.3.0-0`
55
+ - **验证版本**:`dsh-v0.2.0-rc.2`(全局 DSH CLI,并针对真实
56
+ `dsh-settings` / `cordis` / `schemastery` 依赖集)
57
+ - **最后验证日期**:2026-10-15
58
+ - **安装方式已验证**:`dsh plugin --profile web add dsh-engram`
package/lib/api.js CHANGED
@@ -12,7 +12,7 @@
12
12
 
13
13
  import { uuid, shortId } from "./util.js";
14
14
  import { collectToolCounts, collectUsageStats } from "./usage.js";
15
- import { SETTINGS_NS } from "./settings.js";
15
+ import { SETTINGS_NS, WIRE_SETTINGS_NS, LEGACY_SETTINGS_NS } from "./settings.js";
16
16
 
17
17
  /** Route family prefix (kept in sync with the client half). */
18
18
  export const API_PREFIX = "/api/dsh-engram";
@@ -160,8 +160,9 @@ export function makeEngramRoutes(service) {
160
160
  writeJson(res, 200, {
161
161
  ...summary,
162
162
  indexes,
163
- captures: service.captureStats ?? { total: 0, git: 0, file: 0, error: 0 },
163
+ captures: service.captureStats ?? { total: 0, git: 0, file: 0, error: 0, preference: 0 },
164
164
  gc: service.gcStats ?? { lastRun: 0, archivedMemories: 0, archivedTasks: 0, removedLinks: 0 },
165
+ activation: service.packTelemetry ? service.packTelemetry() : { boundaryPrune: null, recall: null },
165
166
  compaction: service.compactionStatus ?? { host: "unknown", web: { action: "unknown", presets: [] } },
166
167
  config: service.config ?? {},
167
168
  anonymous: {
@@ -512,6 +513,10 @@ export function makeEngramRoutes(service) {
512
513
  recallHitsPerQuery: queries > 0 ? +(hitsTotal / queries).toFixed(3) : null,
513
514
  detailFollowRate: queries > 0 ? +(detailFollows / queries).toFixed(4) : null,
514
515
  },
516
+ // SoL-Pi activation telemetry: the durable BoundaryPrune / ResultPack
517
+ // activation record (fires actually landed on this profile) + the
518
+ // [RECALL] injection-funnel leg (blocks actually injected into sessions).
519
+ activation: service.packTelemetry ? service.packTelemetry() : { boundaryPrune: null, recall: null },
515
520
  });
516
521
  };
517
522
 
@@ -561,8 +566,13 @@ export function makeEngramRoutes(service) {
561
566
  // Normalize both onto the HTTP contract {namespaces:[...]} the client uses.
562
567
  const payload = settings.describe();
563
568
  const namespaces = Array.isArray(payload) ? payload : (payload?.namespaces ?? []);
564
- const view = namespaces.find((n) => n && n.ns === SETTINGS_NS);
565
- writeJson(res, 200, view ? { namespaces: [view] } : { namespaces: [] });
569
+ // The seam carries the ACTIVE namespace — entry id `engram` on DSH
570
+ // ≥0.1.7 / 0.2.x, legacy `dsh-engram` on ≤0.1.5. The view is re-labelled
571
+ // onto the stable WIRE_SETTINGS_NS the client card filters on, so the
572
+ // shipped client works unchanged across generations.
573
+ const activeNs = settings.ns ?? SETTINGS_NS;
574
+ const view = namespaces.find((n) => n && (n.ns === activeNs || n.ns === LEGACY_SETTINGS_NS));
575
+ writeJson(res, 200, view ? { namespaces: [{ ...view, ns: WIRE_SETTINGS_NS }] } : { namespaces: [] });
566
576
  } catch (error) {
567
577
  writeError(res, 500, String(error?.message ?? error));
568
578
  }
@@ -586,10 +596,13 @@ export function makeEngramRoutes(service) {
586
596
  return writeError(res, 400, "body must provide ops (array of {op,path[,value]}) or patch (object)");
587
597
  }
588
598
  try {
599
+ // Drive the seam's ACTIVE namespace (entry id `engram` on DSH ≥0.1.7 /
600
+ // 0.2.x, legacy `dsh-engram` on ≤0.1.5), never the wire label.
601
+ const activeNs = settings.ns ?? SETTINGS_NS;
589
602
  if (Array.isArray(ops)) {
590
- await settings.mutate(SETTINGS_NS, ops, expectedRevision);
603
+ await settings.mutate(activeNs, ops, expectedRevision);
591
604
  } else {
592
- await settings.update(SETTINGS_NS, patch, expectedRevision);
605
+ await settings.update(activeNs, patch, expectedRevision);
593
606
  }
594
607
  writeJson(res, 200, { ok: true });
595
608
  } catch (error) {
@@ -0,0 +1,294 @@
1
+ /**
2
+ * dsh-engram BoundaryPrune (L1, PROPOSAL-resultpack §L0-given parameters):
3
+ * prune over-threshold tool results at todo/goal subtask boundaries, gated by
4
+ * a repayment clock (SoL-Pi Online Context Compact), through DSH's own
5
+ * deterministic `toolResultPruner` service.
6
+ *
7
+ * Why: oracle-analysis measured 2.77B–7.53B chars of tool-result replay
8
+ * exposure (692M–1.88M tokens) across 97 sessions; L0 counterfactual
9
+ * simulation (eval/simulate-boundary-prune.mjs) says todo-progress boundaries
10
+ * cover 79% of that exposure and a repayment gate of ~50 requests lifts
11
+ * cache-aware net savings 12–32% on every threshold config.
12
+ *
13
+ * Design (all L0-given, nothing tuned on eval data):
14
+ * - Boundary source: `todo/write` events whose completed count strictly
15
+ * increased (the todo-progress strategy), plus `goal/change` with
16
+ * operation complete/block (free rider — goal+todo ≈ todo-only).
17
+ * - Executor: `ctx.get("toolResultPruner")` (DSH's own surface-replace
18
+ * pruner, 8192/4096/1024 by default). We only decide WHEN; the host's
19
+ * replay-safe, zero-LLM mechanics do the work. Feature-detected: absent
20
+ * service → disabled with one log line.
21
+ * - Repayment gate: only prune when the session is expected to run at
22
+ * least `minRemainingRequests` more requests (workspace median session
23
+ * length from ~/.dsh/sessions, the same corpus the oracle tool reads;
24
+ * fresh workspaces fall back to a fixed floor). Pruning rewrites the
25
+ * prefix and breaks KV reuse; savings accrue only on later requests.
26
+ * - Pressure-overflow respect: the host prunes at pressure/overflow
27
+ * compaction anyway, so boundaries inside an open compaction cycle gain
28
+ * nothing — we skip while the session just compacted (the pruner itself
29
+ * is idempotent: already-pruned results are under threshold).
30
+ * - Default OFF (`boundaryPrune` config key) — a dormant mechanism costs
31
+ * nothing (SoL-Pi C13); activation telemetry increments counters that
32
+ * /stats already reports from the session log (`compaction/prune`
33
+ * events), so no extra write path is needed.
34
+ *
35
+ * Everything is deterministic, zero-LLM, fail-contained (never throws into
36
+ * the session pipeline). ~150 lines by design.
37
+ */
38
+
39
+ import fs from "node:fs";
40
+ import os from "node:os";
41
+ import path from "node:path";
42
+
43
+ import { workspaceKey as wk } from "./util.js";
44
+
45
+ /** Fixed fallback for the repayment gate when no history exists yet. */
46
+ const DEFAULT_MIN_REMAINING = 50;
47
+ /**
48
+ * Empirical expected session length (requests) for coding workspaces, from
49
+ * the L0 corpus (97 sessions, median 43, long tail 1247). Used as the prior
50
+ * when the workspace has sessions on disk but we do not decompress them.
51
+ */
52
+ const EMPIRICAL_MEDIAN_REQUESTS = 120;
53
+
54
+ function isRecord(v) {
55
+ return v !== null && typeof v === "object" && !Array.isArray(v);
56
+ }
57
+
58
+ /** Count completed todos in a todo/write payload (0 when malformed). */
59
+ export function completedCount(todos) {
60
+ if (!Array.isArray(todos)) return 0;
61
+ return todos.filter((t) => isRecord(t) && t.status === "completed").length;
62
+ }
63
+
64
+ /**
65
+ * Does this session event constitute a BoundaryPrune boundary?
66
+ * todo/write with strictly more completed items than the session has seen
67
+ * before (per-session state), or a terminal goal/change.
68
+ */
69
+ export function isBoundary(event, state) {
70
+ const type = event?.type;
71
+ if (type === "todo/write") {
72
+ const completed = completedCount(event?.data?.todos);
73
+ if (completed > state.todoCompleted) {
74
+ state.todoCompleted = completed;
75
+ return { kind: "todo", completed };
76
+ }
77
+ return null;
78
+ }
79
+ if (type === "goal/change") {
80
+ const op = event?.data?.operation;
81
+ if (op === "complete" || op === "block") return { kind: "goal", op };
82
+ }
83
+ return null;
84
+ }
85
+
86
+ /** Sessions per workspace with ≥1 request, as (count, medianRequests) map — L0 repayment input. */
87
+ export function workspaceHistory(root = path.join(os.homedir(), ".dsh", "sessions")) {
88
+ const byWorkspace = new Map();
89
+ try {
90
+ for (const bucket of fs.readdirSync(root)) {
91
+ if (!bucket.startsWith("--")) continue;
92
+ const ws = decodeURIComponent(bucket.replace(/^--/, "").replace(/--$/, ""));
93
+ let sessions = 0;
94
+ const bdir = path.join(root, bucket);
95
+ for (const sid of fs.readdirSync(bdir)) {
96
+ const file = path.join(bdir, sid, "session.jsonl.zstd");
97
+ try {
98
+ const st = fs.statSync(file);
99
+ if (st.size > 0) sessions += 1;
100
+ } catch { /* no log */ }
101
+ }
102
+ if (sessions > 0) byWorkspace.set(ws, { sessions, medianRequests: null });
103
+ }
104
+ } catch { /* unreadable root — fall back to the fixed floor */ }
105
+ return byWorkspace;
106
+ }
107
+
108
+ /**
109
+ * Estimate the requests this session is still expected to run, from the
110
+ * workspace's own session history. Deterministic, no log decompression:
111
+ * a workspace with sessions on disk gets the empirical coding-session prior
112
+ * (EMPIRICAL_MEDIAN_REQUESTS); unknown workspaces get the fixed floor.
113
+ *
114
+ * The estimate is a LOWER anchor, never an upper bound: the gate compares
115
+ * `requests + minRemaining <= expectedTotal`, and for sessions already past
116
+ * the prior we project growth (the session has demonstrated it runs long —
117
+ * L0 showed the exposure lives exactly in those long sessions, so the gate
118
+ * must not starve them).
119
+ */
120
+ export function expectedSessionRequests(requestsSoFar, history, workspace, floor = DEFAULT_MIN_REMAINING) {
121
+ const entry = history?.get(workspace);
122
+ const prior = entry && entry.sessions > 0 ? Math.max(floor, EMPIRICAL_MEDIAN_REQUESTS) : floor;
123
+ if (requestsSoFar <= prior) return prior;
124
+ // past the prior: project the session to run at least as long again as it
125
+ // has already run (bounded optimism — a session at N requests has proven
126
+ // it is not a short one).
127
+ return Math.max(prior, requestsSoFar * 2);
128
+ }
129
+
130
+ /** Install the boundary-prune listener. Returns a disposer. */
131
+ export function installBoundaryPrune(ctx, config, log, hooks = {}, opts = {}) {
132
+ const enabled = config?.boundaryPrune === true;
133
+ if (!enabled) return () => {};
134
+ const pruner = ctx.get?.("toolResultPruner");
135
+ if (!pruner || typeof pruner.pruneSession !== "function") {
136
+ log?.warn?.("engram boundary-prune: toolResultPruner service unavailable — disabled");
137
+ return () => {};
138
+ }
139
+ const minRemaining = Number.isFinite(config?.boundaryPruneMinRemaining)
140
+ ? config.boundaryPruneMinRemaining
141
+ : DEFAULT_MIN_REMAINING;
142
+ // History is injectable for tests; production reads the real session corpus.
143
+ const history = opts?.history ?? workspaceHistory();
144
+
145
+ // Optional threshold override (ResultPack L0, T2048 landing): the host pruner
146
+ // ships a fixed threshold (8192) that the plugin cannot change in place. When
147
+ // `boundaryPruneThresholdChars` is set, we instantiate our own pruner at that
148
+ // threshold — same deterministic head/middle/tail engine, lower budget — via
149
+ // the exported `ToolResultPruner` prototype (bypassing the cordis Service
150
+ // constructor, which needs a full context; the instance only uses its own
151
+ // `config` + `ctx.tokenMeter`). Falls back to the host pruner when the package
152
+ // or the token meter is unavailable. `boundaryPruneHeadChars` /
153
+ // `boundaryPruneTailChars` tune the retained edges (defaults stay at the
154
+ // host's 4096/1024).
155
+ let activePruner = pruner;
156
+ const thresholdOverride = Number.isFinite(config?.boundaryPruneThresholdChars)
157
+ ? Math.trunc(config.boundaryPruneThresholdChars)
158
+ : null;
159
+ if (thresholdOverride !== null && thresholdOverride > 0) {
160
+ activePruner = makeThresholdPruner(pruner, {
161
+ thresholdChars: thresholdOverride,
162
+ headChars: Number.isFinite(config?.boundaryPruneHeadChars) && config.boundaryPruneHeadChars > 0
163
+ ? Math.trunc(config.boundaryPruneHeadChars)
164
+ : 4096,
165
+ tailChars: Number.isFinite(config?.boundaryPruneTailChars) && config.boundaryPruneTailChars > 0
166
+ ? Math.trunc(config.boundaryPruneTailChars)
167
+ : 1024,
168
+ });
169
+ if (activePruner === null) {
170
+ log?.warn?.(
171
+ `engram boundary-prune: threshold override ${thresholdOverride} unavailable (pruner package/tokenMeter) — using host pruner ${pruner.config?.thresholdChars ?? "default"}`,
172
+ );
173
+ activePruner = pruner;
174
+ } else {
175
+ log?.info?.(
176
+ `engram boundary-prune: using threshold ${activePruner.config?.thresholdChars} (head ${activePruner.config?.headChars} / tail ${activePruner.config?.tailChars})`,
177
+ );
178
+ }
179
+ }
180
+
181
+ /** Per-session boundary state (todo high-water mark, request count, total prunes). */
182
+ const stateBySession = new Map();
183
+
184
+ const onEvent = (session, event) => {
185
+ try {
186
+ const sid = session?.id;
187
+ if (typeof sid !== "string" || sid.length === 0) return;
188
+ let state = stateBySession.get(sid);
189
+ if (!state) {
190
+ state = { todoCompleted: 0, requests: 0, prunes: 0, prunedResults: 0, lastFireRequest: void 0 };
191
+ stateBySession.set(sid, state);
192
+ }
193
+ const type = event?.type;
194
+ if (type === "assistant/message") {
195
+ state.requests += 1;
196
+ // Request-count boundary (SoL-Pi "step idle", default OFF): sessions
197
+ // with no todo progress / no goal never fire the event boundaries (L0
198
+ // corpus: ~62% of sessions), leaving their over-threshold replay
199
+ // exposure permanently untouched. When `boundaryPruneIdleCadence` is
200
+ // set, the counter itself is a boundary source: fire at most once per
201
+ // cadence requests, still under the repayment gate (so cadence ≠
202
+ // aggressive — the gate keeps per-session reuse healthy). The implied
203
+ // boundary kind is `step` so telemetry can attribute it.
204
+ const cadence = config?.boundaryPruneIdleCadence;
205
+ if (Number.isFinite(cadence) && cadence > 0 && state.requests % Math.trunc(cadence) === 0) {
206
+ const sinceLastPrune = state.lastFireRequest === void 0 ? state.requests : state.requests - state.lastFireRequest;
207
+ if (sinceLastPrune >= Math.trunc(cadence)) {
208
+ const workspace = wk(session?.header?.cwd);
209
+ const expectedTotal = expectedSessionRequests(state.requests, history, workspace, minRemaining);
210
+ if (state.requests + minRemaining <= expectedTotal) {
211
+ // reuse the same fire path via a synthetic boundary
212
+ return fireBoundary(session, { kind: "step" }, state, expectedTotal);
213
+ }
214
+ }
215
+ }
216
+ return;
217
+ }
218
+ const boundary = isBoundary(event, state);
219
+ if (boundary === null) return;
220
+
221
+ // Repayment gate: pruning rewrites the prefix (cache break amortized
222
+ // only over later requests). Fire only when the workspace's own session
223
+ // history says this session is expected to run `minRemaining` more
224
+ // requests beyond the current count.
225
+ const workspace = wk(session?.header?.cwd);
226
+ const expectedTotal = expectedSessionRequests(state.requests, history, workspace, minRemaining);
227
+ if (state.requests + minRemaining > expectedTotal) return;
228
+ return fireBoundary(session, boundary, state, expectedTotal);
229
+ } catch (error) {
230
+ // fully contained — a prune bug must never disturb the session pipeline
231
+ log?.warn?.(`engram boundary-prune threw: ${String(error)}`);
232
+ }
233
+ };
234
+
235
+ /** Shared fire path for every boundary source (event + step-cadence). */
236
+ const fireBoundary = (session, boundary, state, expectedTotal) => {
237
+ const result = activePruner.pruneSession(session);
238
+ const prunedNow = result?.pruned?.length ?? 0;
239
+ if (prunedNow > 0) {
240
+ state.prunes += 1;
241
+ state.prunedResults += prunedNow;
242
+ state.lastFireRequest = state.requests;
243
+ log?.info?.(
244
+ `engram boundary-prune (${boundary.kind} @${state.requests} reqs): pruned ${prunedNow} results, saved ${result.charsRemoved} chars`,
245
+ );
246
+ // L2 hook (ResultPack): archiving + handle tail is wired by index.js
247
+ // when `resultPack` is enabled. Contained there — never throws here.
248
+ // Attach the live pruner budgets so the archiver extracts the exact
249
+ // removed span (archives must match what the surface now displays).
250
+ try {
251
+ const budgets = {
252
+ headChars: activePruner.config?.headChars ?? 4096,
253
+ tailChars: activePruner.config?.tailChars ?? 1024,
254
+ };
255
+ for (const item of result?.pruned ?? []) {
256
+ if (item && typeof item === "object") Object.assign(item, budgets);
257
+ }
258
+ hooks?.onPruned?.(session, result, boundary);
259
+ } catch (error) {
260
+ log?.warn?.(`engram boundary-prune: L2 hook threw: ${String(error)}`);
261
+ }
262
+ }
263
+ void expectedTotal;
264
+ };
265
+ const disposeListener = ctx.on("session/event", onEvent);
266
+ const disposeSink = ctx.on("session/disposed", (session) => {
267
+ stateBySession.delete(session?.id);
268
+ });
269
+ return () => {
270
+ disposeListener();
271
+ disposeSink?.();
272
+ };
273
+ }
274
+
275
+ /**
276
+ * Build a pruner at an override threshold by prototyping off the injected
277
+ * host service instance.
278
+ *
279
+ * The host `toolResultPruner` ships a fixed budget (8192/4096/1024) that
280
+ * cannot be mutated in place. DSH's pruner is a plain prototype-owning class
281
+ * whose methods only read `this.config` and `this.ctx.tokenMeter`, so we
282
+ * create a child object inheriting the host instance (methods + ctx shared,
283
+ * config overridden on the child). Zero LLM, deterministic, contained — the
284
+ * host instance and its config stay untouched.
285
+ */
286
+ function makeThresholdPruner(pruner, cfg) {
287
+ try {
288
+ const clone = Object.create(pruner);
289
+ clone.config = cfg;
290
+ return clone;
291
+ } catch {
292
+ return null;
293
+ }
294
+ }