@aiwayds/dsh-dcp 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +144 -0
- package/README.md +68 -143
- package/lib/setup.js +95 -0
- package/package.json +9 -2
- package/scripts/compare.mjs +285 -0
- package/scripts/setup.mjs +90 -0
package/README.en.md
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
1
|
+
# dsh-dcp
|
|
2
|
+
|
|
3
|
+
Deterministic context-compaction backend for dsh (DeepSeek Harness): **context
|
|
4
|
+
compaction without an LLM call**, works out of the box.
|
|
5
|
+
|
|
6
|
+
> [简体中文](README.md) · **English**
|
|
7
|
+
|
|
8
|
+
## Why
|
|
9
|
+
|
|
10
|
+
dsh compacts conversation context by default with `compaction-basic`, which
|
|
11
|
+
asks an LLM to re-summarize older messages on every compaction — costly, slow,
|
|
12
|
+
and non-deterministic. dsh-dcp is a pure-code port of the ideas behind
|
|
13
|
+
[opencode-dcp](https://github.com/Opencode-DCP/opencode-dynamic-context-pruning)
|
|
14
|
+
(dedup, error cleanup, "technical summary instead of prose"):
|
|
15
|
+
|
|
16
|
+
- **Zero LLM calls**: compaction itself costs no extra tokens
|
|
17
|
+
- **Deterministic**: identical input always yields identical output
|
|
18
|
+
- **CJK-friendly**: verbatim user text / paths / commands / errors, priced at
|
|
19
|
+
real CJK density
|
|
20
|
+
- **Inherits all official safety**: triggers, retained tail, transaction
|
|
21
|
+
locks, tool-pairing — dsh's own machinery, only the summarizer is replaced
|
|
22
|
+
|
|
23
|
+
## Effects
|
|
24
|
+
|
|
25
|
+
### vs. the official compaction-basic
|
|
26
|
+
|
|
27
|
+
| | compaction-basic | dsh-dcp |
|
|
28
|
+
|---|---|---|
|
|
29
|
+
| Summarization | LLM rewrite per compaction | deterministic code extraction |
|
|
30
|
+
| LLM calls per compaction | 1 | **0** |
|
|
31
|
+
| Determinism | may differ run to run | identical input → identical output |
|
|
32
|
+
| Summary content | semantic | verbatim hard facts (paths/commands/errors/todos/user text) |
|
|
33
|
+
| Chinese | model re-transcribes | kept verbatim + CJK-aware pricing |
|
|
34
|
+
| Triggers/retention/overflow/safety | official | **inherited, identical** |
|
|
35
|
+
| Checkpoint format | official | compatible (mutually mergeable) |
|
|
36
|
+
|
|
37
|
+
It also borrows the dedup / error-purge / `/dcp` / technical-summary ideas from
|
|
38
|
+
[opencode-dcp](https://github.com/Opencode-DCP/opencode-dynamic-context-pruning),
|
|
39
|
+
re-implemented against dsh's compaction seam — that one serves opencode, this
|
|
40
|
+
one serves dsh.
|
|
41
|
+
|
|
42
|
+
### CJK adaptation
|
|
43
|
+
|
|
44
|
+
Content is kept verbatim (no re-transcription into English); tokens are priced
|
|
45
|
+
at real CJK density (~2 chars/token for Chinese/Japanese/Korean/full-width)
|
|
46
|
+
instead of the host's flat 4 chars/token that underestimates Chinese — so CJK
|
|
47
|
+
sessions get a budget that reflects real cost, and checkpoints stay
|
|
48
|
+
information-dense.
|
|
49
|
+
|
|
50
|
+
### Real dsh session
|
|
51
|
+
|
|
52
|
+
~80k tokens of history → ~700-token checkpoint (**~100x**), zero LLM calls;
|
|
53
|
+
cache-hit rate is barely affected (any backend pays one "cold request" right
|
|
54
|
+
after a compaction).
|
|
55
|
+
|
|
56
|
+
A checkpoint produced on a real session (Chinese content kept verbatim):
|
|
57
|
+
|
|
58
|
+
```
|
|
59
|
+
## Primary Request and Intent
|
|
60
|
+
- 帮我把登录页的重定向 bug 修掉
|
|
61
|
+
|
|
62
|
+
## Files and Code
|
|
63
|
+
- /app/src/auth/login.ts — W×1 R×1
|
|
64
|
+
|
|
65
|
+
## Errors and Fixes
|
|
66
|
+
- bash: FAIL src/auth.test.ts
|
|
67
|
+
|
|
68
|
+
## Pending Jobs
|
|
69
|
+
- add regression test
|
|
70
|
+
|
|
71
|
+
## Critical Context
|
|
72
|
+
- dsh-dcp 确定性压缩了 12 条消息 / 8 次工具调用（未调用 LLM 摘要）
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Not in scope
|
|
76
|
+
|
|
77
|
+
- **No semantic summarization**: it preserves facts that appeared, it does not
|
|
78
|
+
"understand" code. Need deep semantic checkpoints? Stick with the official
|
|
79
|
+
`compaction-basic`
|
|
80
|
+
- **Things dsh already does, deliberately not re-implemented**:
|
|
81
|
+
- tool-result pruning (`compaction-tool-result-pruner`, deterministic by size)
|
|
82
|
+
- trigger policy, retained tail, overflow recovery (inherited from official)
|
|
83
|
+
- `/compact` command, UI checkpoint cards (shipped with dsh)
|
|
84
|
+
|
|
85
|
+
## Install
|
|
86
|
+
|
|
87
|
+
**Recommended: pair it with our dsh-tui-pi** (the TUI already depends on
|
|
88
|
+
dsh-dcp):
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
npm i @aiwayds/dsh-tui-pi
|
|
92
|
+
dsh plugin add @aiwayds/dsh-dcp # activates dcp; the bundle auto-mounts
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
**Standalone:**
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
npm i @aiwayds/dsh-dcp
|
|
99
|
+
npx dsh-dcp-setup # safe: date-stamped backup → append-only → idempotent checks
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
> dsh-dcp plugs into dsh's compaction seam and only affects profiles that
|
|
103
|
+
> mount it. The web profile does not bundle the TUI, so it keeps the official
|
|
104
|
+
> backend and is unaffected.
|
|
105
|
+
|
|
106
|
+
## /dcp command
|
|
107
|
+
|
|
108
|
+
| Command | Effect |
|
|
109
|
+
|---|---|
|
|
110
|
+
| `/dcp` | status: config, compaction count, tokens saved |
|
|
111
|
+
| `/dcp compact` | compact now (zero LLM) |
|
|
112
|
+
| `/dcp set <k> <v>` | adjust a knob for this session, with a persist hint |
|
|
113
|
+
|
|
114
|
+
Settable: `dedup`, `purgeErrors`, `maxItems`, `maxItemChars`,
|
|
115
|
+
`maxSummaryTokens`, `language`, `tokenEstimate`, `thresholdRatio`.
|
|
116
|
+
|
|
117
|
+
## Configuration
|
|
118
|
+
|
|
119
|
+
All optional, defaults work out of the box:
|
|
120
|
+
|
|
121
|
+
| Key | Default | Meaning |
|
|
122
|
+
|---|---|---|
|
|
123
|
+
| `thresholdRatio` | 0.7 | compaction trigger; 0.7 recommended for CJK-heavy sessions |
|
|
124
|
+
| `language` | `zh` | summary language; `zh` also enables Chinese error/"待办：" detection |
|
|
125
|
+
| `tokenEstimate` | `cjk` | CJK (zh/ja/ko/full-width) at ~2 chars/token; `ascii` matches the host |
|
|
126
|
+
| `dedup` | `true` | annotate repeated tool calls |
|
|
127
|
+
| `purgeErrors` | `true` | collapse stale errors into one note |
|
|
128
|
+
| `maxItems` / `maxItemChars` | 10 / 200 | summary density |
|
|
129
|
+
| `maxSummaryTokens` | 2048 | summary token budget |
|
|
130
|
+
|
|
131
|
+
## Design reference
|
|
132
|
+
|
|
133
|
+
- [Opencode-DCP/opencode-dynamic-context-pruning](https://github.com/Opencode-DCP/opencode-dynamic-context-pruning)
|
|
134
|
+
- dsh official compaction seam: `docs/subsystems/compaction.md` (deepseek-harness)
|
|
135
|
+
|
|
136
|
+
## Development
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
npm install && npm test # 45 tests: extractor/compaction/command/config/setup
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
## License
|
|
143
|
+
|
|
144
|
+
MIT
|
package/README.md
CHANGED
|
@@ -1,107 +1,47 @@
|
|
|
1
1
|
# dsh-dcp
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
dsh（DeepSeek Harness）的确定性压缩后端：**上下文压缩不调 LLM**，开箱即用。
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
> **简体中文** · [English](README.en.md)
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## 为什么做
|
|
8
8
|
|
|
9
|
-
dsh
|
|
9
|
+
dsh 默认的压缩（`compaction-basic`）每次压缩都要让模型把旧对话**重新总结一遍**——费 token、慢、结果还不稳定。我们参考 opencode 社区的 [opencode-dcp](https://github.com/Opencode-DCP/opencode-dynamic-context-pruning)（去重、清错、"技术摘要代替散文"），做了一个纯代码版本：
|
|
10
10
|
|
|
11
|
-
- **零 LLM
|
|
12
|
-
-
|
|
13
|
-
-
|
|
14
|
-
-
|
|
15
|
-
- **继承一切安全机制**：压力触发、保留尾巴、溢出恢复、事务锁、tool-pairing 边界全部复用官方实现（只 override 官方留的唯一钩子 `summarize()`）
|
|
11
|
+
- **零 LLM 调用**：压缩本身不消耗任何额外 token
|
|
12
|
+
- **输出稳定**：相同对话永远得到相同摘要
|
|
13
|
+
- **中文友好**：用户原话/路径/命令/报错逐字保留，按 CJK 真实密度计价
|
|
14
|
+
- **继承官方全部安全机制**：触发、保留尾巴、事务锁、tool-pairing 边界都复用 dsh 官方实现（只替换"摘要"这一环）
|
|
16
15
|
|
|
17
|
-
##
|
|
16
|
+
## 效果
|
|
18
17
|
|
|
19
|
-
|
|
18
|
+
### 与官方默认压缩的对比
|
|
20
19
|
|
|
21
|
-
|
|
22
|
-
npm i @aiwayds/dsh-tui-pi # 重启 dsh 即可，dcp 开箱即用，无需手动配置
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
**独立使用（非 tui 环境）** —— npm 只把包装进 node_modules，**挂载（写 patch 条目）不在 npm 行为里**，除非宿主 bundle（如 dsh-tui-pi）自带挂载。不装 tui 就得手动写：
|
|
26
|
-
|
|
27
|
-
```bash
|
|
28
|
-
# 1. 安装
|
|
29
|
-
npm i @aiwayds/dsh-dcp
|
|
30
|
-
# 或源码：git clone git@github.com:fan56/dsh-dcp.git && cd dsh-dcp && npm install
|
|
31
|
-
|
|
32
|
-
# 2. 手动挂载：在 ~/.dsh/cordis.patch.yml 追加
|
|
33
|
-
- id: compaction-basic
|
|
34
|
-
disabled: true
|
|
35
|
-
- insert:
|
|
36
|
-
- id: dsh-dcp
|
|
37
|
-
name: '@aiwayds/dsh-dcp' # 包装进 profile 后用裸包名即可（与 tui-pi 一致）
|
|
38
|
-
config:
|
|
39
|
-
thresholdRatio: 0.7
|
|
40
|
-
language: zh
|
|
41
|
-
|
|
42
|
-
# 3. 重启 dsh，输入 /dcp 验证
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
> 挂载名用**裸包名**（`@aiwayds/dsh-dcp`）即可——只要它装进了 profile 的 node_modules（`dsh plugin add` 或 profile package.json 依赖），loader 就能像解析 tui-pi 一样找到它；用**绝对路径**指向源码入口同样可行。
|
|
46
|
-
|
|
47
|
-
`/compact` 命令、自动压力触发、overflow 恢复、UI 的 checkpoint 卡片照常工作——它们只依赖 `ctx.compaction` 接口，与本后端无关。
|
|
48
|
-
|
|
49
|
-
## `/dcp` 命令
|
|
50
|
-
|
|
51
|
-
| 命令 | 作用 |
|
|
52
|
-
|---|---|
|
|
53
|
-
| `/dcp` | 状态：当前配置、压缩次数、shadowed token 数、省掉的 LLM 调用数 |
|
|
54
|
-
| `/dcp compact` | 立即压缩（确定性，无 LLM 调用；等价 `/compact`） |
|
|
55
|
-
| `/dcp set <k> <v>` | 本会话内调整参数，并打印持久化到 `cordis.patch.yml` 的片段 |
|
|
56
|
-
| `/dcp help` | 用法 |
|
|
57
|
-
|
|
58
|
-
可调键：`dedup`、`purgeErrors`、`maxItems`、`maxItemChars`、`maxSummaryTokens`、`language`、`thresholdRatio`。
|
|
59
|
-
|
|
60
|
-
## 配置
|
|
61
|
-
|
|
62
|
-
全部可选，默认即用。写在 `~/.dsh/cordis.patch.yml` 的 `config:` 下：
|
|
63
|
-
|
|
64
|
-
```yaml
|
|
65
|
-
- id: compaction-basic
|
|
66
|
-
name: /Users/<you>/github/dsh-dcp/lib/index.js
|
|
67
|
-
config:
|
|
68
|
-
thresholdRatio: 0.7 # 中文场景建议 0.7（默认 0.8）
|
|
69
|
-
language: zh # 输出语言 en|zh：zh 额外启用中文报错/待办规则
|
|
70
|
-
# tokenEstimate: cjk # 默认即 cjk：CJK 字符按 ~2 字符/token 计价
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
### dsh-dcp 自己的键
|
|
74
|
-
|
|
75
|
-
| 键 | 默认 | 说明 |
|
|
20
|
+
| | 官方 compaction-basic | dsh-dcp |
|
|
76
21
|
|---|---|---|
|
|
77
|
-
|
|
|
78
|
-
|
|
|
79
|
-
|
|
|
80
|
-
|
|
|
81
|
-
|
|
|
82
|
-
|
|
|
83
|
-
|
|
|
84
|
-
| `tokenEstimate` | `cjk` | 摘要预算的 token 计价方式：`cjk`（默认）把 **CJK 字符**（中文汉字、日文假名、韩文谚文、全角标点——CJK 不止中文）按 **~2 字符/token** 计价，ASCII 按 4 字符/token；`ascii` 则与宿主 meter 完全一致，所有字符一律 4 字符/token |
|
|
85
|
-
|
|
86
|
-
### 继承自 compaction-basic 的键
|
|
22
|
+
| 摘要方式 | 每次调 LLM 重写 | 确定性代码抽取 |
|
|
23
|
+
| 每次压缩的模型调用 | 1 次 | **0 次** |
|
|
24
|
+
| 输出稳定性 | 同对话多次可能不同 | 相同输入永远相同 |
|
|
25
|
+
| 摘要内容 | 语义归纳 | 逐字保硬信息（路径/命令/报错/待办/用户原话） |
|
|
26
|
+
| 中文 | 依赖模型转写 | 原样保留 + CJK 计价 |
|
|
27
|
+
| 触发/保留/溢出/安全 | 官方 | **继承官方，完全相同** |
|
|
28
|
+
| 检查点格式 | 官方 | 兼容（可互相合并） |
|
|
87
29
|
|
|
88
|
-
|
|
30
|
+
设计上还吸收了 [opencode-dcp](https://github.com/Opencode-DCP/opencode-dynamic-context-pruning) 的思路（去重、清错、`/dcp` 命令、技术摘要），但按 dsh 的压缩接口重新实现——它服务于 opencode，dsh-dcp 服务于 dsh。
|
|
89
31
|
|
|
90
|
-
|
|
32
|
+
### CJK 适配
|
|
91
33
|
|
|
92
|
-
|
|
34
|
+
内容逐字保留、不做英文转写；token 计价按 CJK 真实密度（中/日/韩/全角约 2 字符/token），不沿用宿主"4 字符/token"对中文的低估——中文会话的摘要预算反映真实成本，不会被饿死，信息更密集。
|
|
93
35
|
|
|
94
|
-
|
|
95
|
-
- **中文规则集（`language: zh`）**：报错识别补充中文关键词（找不到/未找到/不存在/失败/错误/报错/异常/无法/拒绝/超时/崩溃/致命），待办识别补充 `待办：` 标记；`en` 模式只认英文规则
|
|
96
|
-
- **内容逐字保留**：用户原话、文件路径、命令、报错串原样进入摘要，不做英文转写，中文信息零损耗
|
|
36
|
+
### 真实 dsh 会话实测
|
|
97
37
|
|
|
98
|
-
|
|
38
|
+
一段约 8 万 token 的历史压成约 700 token（**~100x**），全程零 LLM 调用；缓存命中率几乎不变（压缩后总会有一个"冷请求"，任何后端都一样）。
|
|
99
39
|
|
|
100
|
-
|
|
40
|
+
真实会话里压缩出的检查点（中文内容逐字保留）：
|
|
101
41
|
|
|
102
42
|
```
|
|
103
43
|
## Primary Request and Intent
|
|
104
|
-
-
|
|
44
|
+
- 帮我把登录页的重定向 bug 修掉
|
|
105
45
|
|
|
106
46
|
## Files and Code
|
|
107
47
|
- /app/src/auth/login.ts — W×1 R×1
|
|
@@ -112,85 +52,70 @@ dsh 的 host meter 对所有字符一律按 **4 字符/token** 计价，这对
|
|
|
112
52
|
## Pending Jobs
|
|
113
53
|
- add regression test
|
|
114
54
|
|
|
115
|
-
## Current Work
|
|
116
|
-
- Fix the login redirect bug and update README.
|
|
117
|
-
|
|
118
|
-
## Next Step
|
|
119
|
-
- add regression test
|
|
120
|
-
|
|
121
55
|
## Critical Context
|
|
122
56
|
- dsh-dcp 确定性压缩了 12 条消息 / 8 次工具调用（未调用 LLM 摘要）
|
|
123
57
|
```
|
|
124
58
|
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
## 工作原理
|
|
128
|
-
|
|
129
|
-
```
|
|
130
|
-
BasicCompactionEngine dsh 官方：阈值/保留/事务/事件锁/tool-pairing
|
|
131
|
-
▲
|
|
132
|
-
│ 仅 override summarize()
|
|
133
|
-
DcpEngine ── register /dcp
|
|
134
|
-
│
|
|
135
|
-
summarizeDeterministically() 纯代码：抽取 → 合并 prior checkpoint → 预算压缩
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
- 压缩区间选择、`compaction/start → summary → end` 事件序、`surfaceOp: replace`、收敛校验（摘要必须小于被压区间）、`ManualCompactionError` 错误分类，全部由父类承担
|
|
139
|
-
- 摘要预算 = `min(maxSummaryTokens, 45% × 被压区间)`，按 `tokenEstimate` 计价（默认 cjk，对中文准确；对纯英文与宿主一致）；超出时逐级降级（砍条数 → 丢空 section → terse 固定格式 → 硬截断）。45% 的余量保证收敛校验（宿主按 4 字符/token 计价）总能通过
|
|
140
|
-
- 挂载方式是 `cordis.patch.yml` 按 `id: compaction-basic` 覆盖 `name`，行 id 不变、仍留在原 isolate 组内，隔离语义不破坏
|
|
59
|
+
## Not in scope
|
|
141
60
|
|
|
142
|
-
|
|
61
|
+
- **不做语义归纳**：不"理解"代码，只保留"出现过的事实"。需要深度语义摘要的场景，请继续用官方 `compaction-basic`
|
|
62
|
+
- **dsh 已经有的我们不重复做**：
|
|
63
|
+
- 工具结果剪枝（`compaction-tool-result-pruner`，确定性按大小剪）
|
|
64
|
+
- 触发策略、保留尾巴、溢出恢复（直接继承官方）
|
|
65
|
+
- `/compact` 命令、UI 检查点卡片（dsh 自带）
|
|
143
66
|
|
|
144
|
-
|
|
145
|
-
|---|---|---|
|
|
146
|
-
| 宿主 | opencode | dsh（DeepSeek Harness） |
|
|
147
|
-
| 接入点 | 插件 + 模型可调的 compress 工具 | 官方 compaction seam 的 `summarize()` 钩子 |
|
|
148
|
-
| 历史 | 不改 session，请求前换占位符 | 走官方 durable 替换事务（`surfaceOp: replace`） |
|
|
149
|
-
| 摘要 | 模型生成技术摘要 | 确定性模板抽取，零 LLM 调用 |
|
|
150
|
-
| 命令 | `/dcp` TUI 面板 + `/dcp-compress` | `/dcp`（status/compact/set） |
|
|
151
|
-
| 配置 | `dcp.jsonc`（~30 键） | `cordis.patch.yml` config（8 个自有键） |
|
|
67
|
+
## 安装
|
|
152
68
|
|
|
153
|
-
|
|
69
|
+
**推荐：配合我们的 dsh-tui-pi 用**（tui 已依赖 dsh-dcp）：
|
|
154
70
|
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
71
|
+
```bash
|
|
72
|
+
npm i @aiwayds/dsh-tui-pi
|
|
73
|
+
dsh plugin add @aiwayds/dsh-dcp # 激活 dcp，bundle 自动挂载
|
|
74
|
+
```
|
|
158
75
|
|
|
159
|
-
|
|
76
|
+
**独立使用**：
|
|
160
77
|
|
|
161
78
|
```bash
|
|
162
|
-
npm
|
|
163
|
-
|
|
79
|
+
npm i @aiwayds/dsh-dcp
|
|
80
|
+
npx dsh-dcp-setup # 安全脚本：带日期备份 → 只追加 → 幂等判断，不碰你已有的配置
|
|
164
81
|
```
|
|
165
82
|
|
|
166
|
-
|
|
83
|
+
> dsh-dcp 挂在 dsh 的压缩接口上，只对挂载了它的 profile 生效。web profile 没挂 tui，继续用官方压缩，不受影响。
|
|
167
84
|
|
|
168
|
-
|
|
85
|
+
## /dcp 命令
|
|
169
86
|
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
87
|
+
| 命令 | 作用 |
|
|
88
|
+
|---|---|
|
|
89
|
+
| `/dcp` | 状态：配置、压缩次数、省下的 token |
|
|
90
|
+
| `/dcp compact` | 立即压缩（零 LLM） |
|
|
91
|
+
| `/dcp set <k> <v>` | 会话内调参，并提示如何持久化 |
|
|
92
|
+
|
|
93
|
+
可调键：`dedup`、`purgeErrors`、`maxItems`、`maxItemChars`、`maxSummaryTokens`、`language`、`tokenEstimate`、`thresholdRatio`。
|
|
94
|
+
|
|
95
|
+
## 配置
|
|
173
96
|
|
|
174
|
-
|
|
97
|
+
全部可选，默认即用：
|
|
175
98
|
|
|
176
|
-
|
|
99
|
+
| 键 | 默认 | 说明 |
|
|
100
|
+
|---|---|---|
|
|
101
|
+
| `thresholdRatio` | 0.7 | 触发阈值；中文场景建议 0.7 |
|
|
102
|
+
| `language` | `zh` | 摘要语言；`zh` 额外识别中文报错和"待办：" |
|
|
103
|
+
| `tokenEstimate` | `cjk` | CJK（中/日/韩/全角）按 ~2 字符/token 计价；`ascii` 与宿主一致 |
|
|
104
|
+
| `dedup` | `true` | 标注重复工具调用 |
|
|
105
|
+
| `purgeErrors` | `true` | 旧报错折叠成一条提示 |
|
|
106
|
+
| `maxItems` / `maxItemChars` | 10 / 200 | 摘要密度 |
|
|
107
|
+
| `maxSummaryTokens` | 2048 | 摘要 token 预算 |
|
|
177
108
|
|
|
178
|
-
|
|
179
|
-
|---|---|---|---|---|---|
|
|
180
|
-
| S1（759 节点，中文主题调试） | 18.2 万 tok | 99.6% / 4152 万 | 99.1% / 2206 万（-47%） | 1 | 81,632→514（**158.8x**） |
|
|
181
|
-
| S2（765 节点） | 18.7 万 tok | 99.1% / 4187 万 | 98.8% / 2102 万（-50%） | 2 | 82,525→577、82,163→821（143x/100x） |
|
|
182
|
-
| S3（128k 未触发） | 12.3 万 tok | 98.6% / 1234 万 | 98.6% / 1234 万（不变） | 0 | — |
|
|
183
|
-
| S3（64k 窗口） | 12.3 万 tok | 98.6% / 1234 万 | 97.2% / 715 万（-42%） | 3 | 42,085→325 等（80~130x） |
|
|
109
|
+
## 设计参考
|
|
184
110
|
|
|
185
|
-
|
|
111
|
+
- [Opencode-DCP/opencode-dynamic-context-pruning](https://github.com/Opencode-DCP/opencode-dynamic-context-pruning)
|
|
112
|
+
- dsh 官方 compaction 接口：`docs/subsystems/compaction.md`（deepseek-harness）
|
|
186
113
|
|
|
187
|
-
|
|
114
|
+
## 开发
|
|
188
115
|
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
- `lib/config.js` — 配置切分与校验
|
|
193
|
-
- 升级 dsh 后：同步 `package.json` 里 `@deepseek-ai/*` 的版本与 `overrides`，`npm install && npm test`
|
|
116
|
+
```bash
|
|
117
|
+
npm install && npm test # 45 个用例：抽取/压缩/命令/配置/安装脚本
|
|
118
|
+
```
|
|
194
119
|
|
|
195
120
|
## License
|
|
196
121
|
|
package/lib/setup.js
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* dsh-dcp setup — safe patching of a user's cordis.patch.yml.
|
|
3
|
+
*
|
|
4
|
+
* The user's file may already carry their own entries (and comments), so this
|
|
5
|
+
* never parses-and-rewrites the whole document: it only APPENDS the dsh-dcp
|
|
6
|
+
* mount block, and only when the mount is not already present. An existing
|
|
7
|
+
* file is backed up (date-stamped) before any change; a missing file is
|
|
8
|
+
* generated fresh.
|
|
9
|
+
*
|
|
10
|
+
* @module dsh-dcp/setup
|
|
11
|
+
*/
|
|
12
|
+
import { readdirSync, readFileSync } from 'node:fs'
|
|
13
|
+
|
|
14
|
+
/** The patch entry that disables the default LLM summarizer. */
|
|
15
|
+
const DISABLE_ENTRY = `- id: compaction-basic
|
|
16
|
+
disabled: true`
|
|
17
|
+
|
|
18
|
+
/** The dsh-dcp mount block, one YAML list item, appended to the file. */
|
|
19
|
+
export function mountBlock({ name, includeDisable }) {
|
|
20
|
+
const parts = []
|
|
21
|
+
if (includeDisable) parts.push(DISABLE_ENTRY)
|
|
22
|
+
parts.push(`- insert:
|
|
23
|
+
- id: dsh-dcp
|
|
24
|
+
name: ${name}
|
|
25
|
+
config:
|
|
26
|
+
thresholdRatio: 0.7
|
|
27
|
+
language: zh`)
|
|
28
|
+
return '\n# dsh-dcp — deterministic compaction backend (added by @aiwayds/dsh-dcp setup).\n' + parts.join('\n') + '\n'
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
/** Whether the given patch text already mounts dsh-dcp (idempotency guard). */
|
|
32
|
+
export function isMounted(text) {
|
|
33
|
+
return /(^|\n)\s*-\s+id:\s*dsh-dcp\b/.test(text)
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
/** Whether the patch text already has an entry with the given id. */
|
|
37
|
+
export function hasEntry(text, id) {
|
|
38
|
+
return new RegExp(`(^|\\n)\\s*-\\s+id:\\s*${id}\\b`).test(text)
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
/**
|
|
42
|
+
* Profiles under `profilesDir` that already bundle the given package (so a
|
|
43
|
+
* home-patch mount of the same package would duplicate its entry id).
|
|
44
|
+
* @param {string} profilesDir - `$DSH_HOME/profiles`.
|
|
45
|
+
* @param {string} pkg - package name, e.g. '@aiwayds/dsh-dcp'.
|
|
46
|
+
* @returns {string[]} profile names that bundle it.
|
|
47
|
+
*/
|
|
48
|
+
export function findBundledProfiles(profilesDir, pkg) {
|
|
49
|
+
if (!profilesDir || !pkg) return []
|
|
50
|
+
let names
|
|
51
|
+
try {
|
|
52
|
+
names = readdirSync(profilesDir)
|
|
53
|
+
} catch {
|
|
54
|
+
return []
|
|
55
|
+
}
|
|
56
|
+
const bundled = []
|
|
57
|
+
for (const name of names) {
|
|
58
|
+
try {
|
|
59
|
+
const manifest = JSON.parse(readFileSync(`${profilesDir}/${name}/package.json`, 'utf8'))
|
|
60
|
+
if (manifest?.dsh?.profile?.bundles?.includes(pkg)) bundled.push(name)
|
|
61
|
+
} catch { /* not a profile dir / unreadable manifest */ }
|
|
62
|
+
}
|
|
63
|
+
return bundled
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
/**
|
|
67
|
+
* Decide what to do for one target patch file.
|
|
68
|
+
*
|
|
69
|
+
* @param {string|undefined} text - existing file content, or undefined when the file is absent.
|
|
70
|
+
* @param {{ name: string }} options - the resolvable dsh-dcp entry specifier to mount.
|
|
71
|
+
* @returns {{ action: 'create'|'patch'|'skip', block?: string, note?: string }}
|
|
72
|
+
*/
|
|
73
|
+
export function planPatch(text, { name }) {
|
|
74
|
+
if (text === undefined || text === null) {
|
|
75
|
+
return { action: 'create', block: mountBlock({ name, includeDisable: true }) }
|
|
76
|
+
}
|
|
77
|
+
if (isMounted(text)) {
|
|
78
|
+
return { action: 'skip' }
|
|
79
|
+
}
|
|
80
|
+
const alreadyHasBasic = hasEntry(text, 'compaction-basic')
|
|
81
|
+
if (alreadyHasBasic) {
|
|
82
|
+
return {
|
|
83
|
+
action: 'patch',
|
|
84
|
+
block: mountBlock({ name, includeDisable: false }),
|
|
85
|
+
note: 'compaction-basic already has an entry in this file; make sure it is disabled, otherwise two backends would both register.',
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
return { action: 'patch', block: mountBlock({ name, includeDisable: true }) }
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
/** Date-stamped backup suffix, e.g. `.bak.20260818-1130`. */
|
|
92
|
+
export function backupStamp(date = new Date()) {
|
|
93
|
+
const p = (n) => String(n).padStart(2, '0')
|
|
94
|
+
return `${date.getFullYear()}${p(date.getMonth() + 1)}${p(date.getDate())}-${p(date.getHours())}${p(date.getMinutes())}`
|
|
95
|
+
}
|
package/package.json
CHANGED
|
@@ -1,24 +1,31 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@aiwayds/dsh-dcp",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "Deterministic context-pruning compaction backend for dsh (DeepSeek Harness) — zero-LLM summaries, /dcp command, works out of the box. Design references Opencode-DCP/opencode-dynamic-context-pruning.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/index.js",
|
|
7
|
+
"bin": {
|
|
8
|
+
"dsh-dcp-setup": "scripts/setup.mjs"
|
|
9
|
+
},
|
|
7
10
|
"exports": {
|
|
8
11
|
".": "./lib/index.js",
|
|
9
12
|
"./summarizer": "./lib/summarizer.js",
|
|
10
13
|
"./config": "./lib/config.js",
|
|
14
|
+
"./setup": "./lib/setup.js",
|
|
11
15
|
"./package.json": "./package.json"
|
|
12
16
|
},
|
|
13
17
|
"files": [
|
|
14
18
|
"lib",
|
|
19
|
+
"scripts",
|
|
15
20
|
"README.md",
|
|
21
|
+
"README.en.md",
|
|
16
22
|
"LICENSE",
|
|
17
23
|
"cordis.patch.example.yml",
|
|
18
24
|
"cordis.patch.yml"
|
|
19
25
|
],
|
|
20
26
|
"scripts": {
|
|
21
|
-
"test": "node --test"
|
|
27
|
+
"test": "node --test",
|
|
28
|
+
"setup": "node scripts/setup.mjs"
|
|
22
29
|
},
|
|
23
30
|
"keywords": [
|
|
24
31
|
"dsh",
|
|
@@ -0,0 +1,285 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Simulate compaction + provider prefix-cache mechanics against a real dsh
|
|
4
|
+
* session log, comparing the dcp backend with the no-compaction baseline.
|
|
5
|
+
*
|
|
6
|
+
* What it measures (static, deterministic — no LLM calls):
|
|
7
|
+
* - per-request input tokens under two pricings: host flat 4 chars/token
|
|
8
|
+
* and dcp's CJK-aware estimate
|
|
9
|
+
* - provider prefix-cache hit/miss: a request is billed as a cache hit for
|
|
10
|
+
* the longest token-prefix it shares with the previous request. Pure
|
|
11
|
+
* append → the previous request is a prefix of the next (≈full hit). A
|
|
12
|
+
* head-anchored compaction swaps the leading span for a checkpoint, so
|
|
13
|
+
* the next request starts cold (≈0 hit) — for EVERY backend, including
|
|
14
|
+
* compaction-basic; the checkpoint text is brand new.
|
|
15
|
+
* - when the dcp engine would compact (host tokens ≥ threshold × window)
|
|
16
|
+
* and what its deterministic summary costs vs the region it shadows.
|
|
17
|
+
*
|
|
18
|
+
* Usage:
|
|
19
|
+
* node scripts/compare.mjs <session.jsonl.zstd|session.jsonl> [contextWindow] [thresholdRatio] [retainRatio] [language]
|
|
20
|
+
*
|
|
21
|
+
* Environment: DCP_* mirrors the plugin config knobs (dedup, purgeErrors,
|
|
22
|
+
* maxItems, maxItemChars, maxSummaryTokens, protectedTools).
|
|
23
|
+
*/
|
|
24
|
+
import fs from 'node:fs'
|
|
25
|
+
import { execFileSync } from 'node:child_process'
|
|
26
|
+
import { isDeepStrictEqual } from 'node:util'
|
|
27
|
+
import { summarizeDeterministically, estimateMessageTokens } from '../lib/summarizer.js'
|
|
28
|
+
|
|
29
|
+
const [, , sessionArg, windowArg, thresholdArg, retainArg, languageArg] = process.argv
|
|
30
|
+
const CONTEXT_WINDOW = Number(windowArg ?? 128000)
|
|
31
|
+
const THRESHOLD = Number(thresholdArg ?? 0.8)
|
|
32
|
+
const RETAIN = Number(retainArg ?? 0.16)
|
|
33
|
+
const LANGUAGE = languageArg ?? 'zh'
|
|
34
|
+
|
|
35
|
+
const DCP = {
|
|
36
|
+
dedup: process.env.DCP_DEDUP !== 'false',
|
|
37
|
+
purgeErrors: process.env.DCP_PURGE_ERRORS !== 'false',
|
|
38
|
+
maxItems: Number(process.env.DCP_MAX_ITEMS ?? 10),
|
|
39
|
+
maxItemChars: Number(process.env.DCP_MAX_ITEM_CHARS ?? 200),
|
|
40
|
+
maxSummaryTokens: Number(process.env.DCP_MAX_SUMMARY_TOKENS ?? 2048),
|
|
41
|
+
language: LANGUAGE,
|
|
42
|
+
tokenEstimate: 'cjk',
|
|
43
|
+
protectedTools: ['write', 'edit', 'apply_patch'],
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
const CHECKPOINT_PREAMBLE = 'This is an automatically generated checkpoint condensing an earlier span of the conversation to free up context. Treat the captured context as established background and build on it without restating it. Continue the task directly from the messages that follow, without acknowledging this checkpoint.'
|
|
47
|
+
|
|
48
|
+
function loadSession(path) {
|
|
49
|
+
const source = path.endsWith('.zstd')
|
|
50
|
+
? execFileSync('zstd', ['-d', '-f', '-c', path], { maxBuffer: 256 * 1024 * 1024 }).toString()
|
|
51
|
+
: fs.readFileSync(path, 'utf8')
|
|
52
|
+
return source.trim().split('\n').filter(Boolean).map((line) => JSON.parse(line))
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
/** Rebuild the message surface (append + replace ops), returning ordered nodes. */
|
|
56
|
+
function rebuildSurface(events) {
|
|
57
|
+
const surface = []
|
|
58
|
+
for (const event of events) {
|
|
59
|
+
let message
|
|
60
|
+
if (event.type === 'user/message') message = event.data
|
|
61
|
+
else if (event.type === 'assistant/message') message = event.data.message
|
|
62
|
+
else if (event.type === 'tool/result') message = event.data.message
|
|
63
|
+
else continue
|
|
64
|
+
const node = { seq: event.seq, type: event.type, message }
|
|
65
|
+
const op = event.surfaceOp
|
|
66
|
+
if (op && op.op === 'replace') {
|
|
67
|
+
const start = surface.findIndex((n) => n.seq === op.start)
|
|
68
|
+
const end = surface.findIndex((n) => n.seq === op.end)
|
|
69
|
+
if (start !== -1 && end !== -1) surface.splice(start, end - start + 1, node)
|
|
70
|
+
} else {
|
|
71
|
+
surface.push(node)
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
return surface
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
/** Host token meter: flat 4 chars/token, mirroring dsh-token-meter minus per-block overhead. */
|
|
78
|
+
function hostTokens(content) {
|
|
79
|
+
return content.reduce((total, block) => {
|
|
80
|
+
if (block.type === 'text') return total + Math.ceil(block.text.length / 4)
|
|
81
|
+
if (block.type === 'tool-call') return total + Math.ceil(block.arguments.length / 4)
|
|
82
|
+
if (block.type === 'tool-result') {
|
|
83
|
+
return total + block.content.reduce((sum, inner) => sum + (inner.type === 'text' ? Math.ceil(inner.text.length / 4) : 0), 0)
|
|
84
|
+
}
|
|
85
|
+
return total
|
|
86
|
+
}, 0)
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
function nodeTokens(node, mode) {
|
|
90
|
+
return mode === 'cjk' ? estimateMessageTokens(node.message, 'cjk') : hostTokens(node.message.content)
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
function surfaceTokens(surface, mode) {
|
|
94
|
+
return surface.reduce((sum, node) => sum + nodeTokens(node, mode), 0)
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
/**
|
|
98
|
+
* Balance (open tool calls) before every cut, mirroring the official
|
|
99
|
+
* tool-pairing eventDelta: only `assistant/message` tool-call blocks open a
|
|
100
|
+
* call, only `tool/result` events close one; every other surface event
|
|
101
|
+
* (user messages, subagent-settled aggregates) is neutral — subagent tool
|
|
102
|
+
* calls live in the child session and never unbalance the parent surface.
|
|
103
|
+
*/
|
|
104
|
+
function prefixBalances(nodes) {
|
|
105
|
+
const balance = [0]
|
|
106
|
+
let open = 0
|
|
107
|
+
for (const node of nodes) {
|
|
108
|
+
if (node.type === 'assistant/message') {
|
|
109
|
+
open += node.message.content.reduce((delta, block) => delta + (block.type === 'tool-call' ? 1 : 0), 0)
|
|
110
|
+
} else if (node.type === 'tool/result') {
|
|
111
|
+
open -= 1
|
|
112
|
+
}
|
|
113
|
+
balance.push(open)
|
|
114
|
+
}
|
|
115
|
+
return balance
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
/**
|
|
119
|
+
* Head-anchored region selection mirroring compaction-basic's
|
|
120
|
+
* `selectCompactableRange`: retain a priced tail, then back up to a
|
|
121
|
+
* tool-pairing-balanced cut. Returns the region nodes or null.
|
|
122
|
+
*/
|
|
123
|
+
function selectRegion(nodes, retainTokens, mode) {
|
|
124
|
+
if (nodes.length === 0) return null
|
|
125
|
+
const balance = prefixBalances(nodes)
|
|
126
|
+
const tokens = nodes.map((node) => nodeTokens(node, mode))
|
|
127
|
+
let accumulated = 0
|
|
128
|
+
let keepFrom = nodes.length
|
|
129
|
+
for (let index = nodes.length - 1; index >= 0; index -= 1) {
|
|
130
|
+
accumulated += tokens[index]
|
|
131
|
+
keepFrom = index
|
|
132
|
+
if (accumulated >= retainTokens) break
|
|
133
|
+
}
|
|
134
|
+
if (keepFrom === 0) return null
|
|
135
|
+
while (keepFrom > 0 && balance[keepFrom] !== 0) keepFrom -= 1
|
|
136
|
+
if (keepFrom === 0) return null
|
|
137
|
+
return nodes.slice(0, keepFrom)
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
function checkpointMessage(summaryText, compactionId) {
|
|
141
|
+
return {
|
|
142
|
+
role: 'user',
|
|
143
|
+
content: [
|
|
144
|
+
{ type: 'text', text: `${CHECKPOINT_PREAMBLE}\n\n<compacted-summary>` },
|
|
145
|
+
{ type: 'text', text: summaryText },
|
|
146
|
+
{ type: 'text', text: '</compacted-summary>' },
|
|
147
|
+
],
|
|
148
|
+
source: { kind: 'plugin', plugin: 'compact', compactionId },
|
|
149
|
+
}
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
/**
|
|
153
|
+
* Longest common token-prefix of two surfaces, node by node (conservative:
|
|
154
|
+
* a partially-shared node counts as zero).
|
|
155
|
+
*/
|
|
156
|
+
function commonPrefixTokens(prev, curr, mode) {
|
|
157
|
+
let tokens = 0
|
|
158
|
+
const length = Math.min(prev.length, curr.length)
|
|
159
|
+
for (let index = 0; index < length; index += 1) {
|
|
160
|
+
if (!isDeepStrictEqual(prev[index].message, curr[index].message)) break
|
|
161
|
+
tokens += nodeTokens(curr[index], mode)
|
|
162
|
+
}
|
|
163
|
+
return tokens
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
function fmt(n) {
|
|
167
|
+
return n.toLocaleString('en-US')
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
/**
|
|
171
|
+
* Replay the session in event order, maintaining a persistent surface. At
|
|
172
|
+
* each step boundary the engine may compact (dcp scenario), then the request
|
|
173
|
+
* is priced and its prefix-cache hit measured against the previous request's
|
|
174
|
+
* surface.
|
|
175
|
+
*/
|
|
176
|
+
function simulate(events, mode, useDcp) {
|
|
177
|
+
const surface = []
|
|
178
|
+
const stats = { requests: 0, inputTokens: 0, hitTokens: 0, compactions: [] }
|
|
179
|
+
let prevSurface = []
|
|
180
|
+
let compactionId = 0
|
|
181
|
+
|
|
182
|
+
for (const event of events) {
|
|
183
|
+
let message
|
|
184
|
+
if (event.type === 'user/message') message = event.data
|
|
185
|
+
else if (event.type === 'assistant/message') message = event.data.message
|
|
186
|
+
else if (event.type === 'tool/result') message = event.data.message
|
|
187
|
+
else if (event.type === 'step/start') {
|
|
188
|
+
if (useDcp) {
|
|
189
|
+
let safety = 0
|
|
190
|
+
while (surfaceTokens(surface, 'host') >= CONTEXT_WINDOW * THRESHOLD && safety < 8) {
|
|
191
|
+
const region = selectRegion(surface, Math.floor(CONTEXT_WINDOW * RETAIN), 'host')
|
|
192
|
+
if (region === null) break
|
|
193
|
+
const regionHost = region.reduce((sum, node) => sum + hostTokens(node.message.content), 0)
|
|
194
|
+
const regionCjk = region.reduce((sum, node) => sum + estimateMessageTokens(node.message, 'cjk'), 0)
|
|
195
|
+
const result = summarizeDeterministically({ messages: region.map((node) => node.message) }, DCP)
|
|
196
|
+
const framed = checkpointMessage(result.summary[0].text, `sim-${compactionId + 1}`)
|
|
197
|
+
const summaryHost = hostTokens(framed.content)
|
|
198
|
+
const summaryCjk = estimateMessageTokens(framed, 'cjk')
|
|
199
|
+
const checkpointNode = { seq: `checkpoint-${compactionId + 1}`, type: 'user/message', message: framed }
|
|
200
|
+
|
|
201
|
+
surface.splice(0, region.length, checkpointNode)
|
|
202
|
+
const totalAfter = surfaceTokens(surface, mode)
|
|
203
|
+
stats.compactions.push({
|
|
204
|
+
regionNodes: region.length,
|
|
205
|
+
regionHost,
|
|
206
|
+
regionCjk,
|
|
207
|
+
summaryHost,
|
|
208
|
+
summaryCjk,
|
|
209
|
+
compression: (regionHost / Math.max(1, summaryHost)).toFixed(1),
|
|
210
|
+
totalAfter,
|
|
211
|
+
preview: result.summary[0].text.split('\n').slice(0, 8).join(' | '),
|
|
212
|
+
})
|
|
213
|
+
compactionId += 1
|
|
214
|
+
safety += 1
|
|
215
|
+
}
|
|
216
|
+
}
|
|
217
|
+
const total = surfaceTokens(surface, mode)
|
|
218
|
+
const hit = commonPrefixTokens(prevSurface, surface, mode)
|
|
219
|
+
stats.requests += 1
|
|
220
|
+
stats.inputTokens += total
|
|
221
|
+
stats.hitTokens += hit
|
|
222
|
+
prevSurface = surface.map((node) => node)
|
|
223
|
+
continue
|
|
224
|
+
} else {
|
|
225
|
+
continue
|
|
226
|
+
}
|
|
227
|
+
const node = { seq: event.seq, type: event.type, message }
|
|
228
|
+
const op = event.surfaceOp
|
|
229
|
+
if (op && op.op === 'replace') {
|
|
230
|
+
const start = surface.findIndex((n) => n.seq === op.start)
|
|
231
|
+
const end = surface.findIndex((n) => n.seq === op.end)
|
|
232
|
+
if (start !== -1 && end !== -1) surface.splice(start, end - start + 1, node)
|
|
233
|
+
} else {
|
|
234
|
+
surface.push(node)
|
|
235
|
+
}
|
|
236
|
+
}
|
|
237
|
+
return stats
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
// -- main ------------------------------------------------------------------
|
|
241
|
+
|
|
242
|
+
const events = loadSession(sessionArg)
|
|
243
|
+
const surface = rebuildSurface(events)
|
|
244
|
+
const totalHost = surfaceTokens(surface, 'host')
|
|
245
|
+
const totalCjk = surfaceTokens(surface, 'cjk')
|
|
246
|
+
|
|
247
|
+
console.log('='.repeat(72))
|
|
248
|
+
console.log('dsh-dcp cache simulation on a real session')
|
|
249
|
+
console.log('='.repeat(72))
|
|
250
|
+
console.log(`session : ${sessionArg}`)
|
|
251
|
+
console.log(`window : ${fmt(CONTEXT_WINDOW)} · threshold ${THRESHOLD} (trigger at ${fmt(CONTEXT_WINDOW * THRESHOLD)} host tokens) · retain ${RETAIN} · language ${LANGUAGE}`)
|
|
252
|
+
console.log(`surface : ${surface.length} message nodes`)
|
|
253
|
+
console.log(`tokens : host ${fmt(totalHost)} · cjk ${fmt(totalCjk)} (ratio ${(totalCjk / totalHost).toFixed(3)})`)
|
|
254
|
+
console.log('')
|
|
255
|
+
|
|
256
|
+
for (const mode of ['host', 'cjk']) {
|
|
257
|
+
const baseline = simulate(events, mode, false)
|
|
258
|
+
const dcp = simulate(events, mode, true)
|
|
259
|
+
const hitRate = (stats) => (stats.inputTokens > 0 ? stats.hitTokens / stats.inputTokens : 0)
|
|
260
|
+
console.log(`── pricing: ${mode === 'host' ? 'host flat 4 chars/token' : 'dcp CJK-aware'} ──`)
|
|
261
|
+
console.log(` baseline (no compaction)`)
|
|
262
|
+
console.log(` ${baseline.requests} requests · input ${fmt(baseline.inputTokens)} tok · cache hit ${(hitRate(baseline) * 100).toFixed(1)}% · miss ${fmt(Math.round(baseline.inputTokens - baseline.hitTokens))} tok`)
|
|
263
|
+
console.log(` dcp backend`)
|
|
264
|
+
console.log(` ${dcp.compactions.length} compaction(s) · ${dcp.requests} requests · input ${fmt(dcp.inputTokens)} tok · cache hit ${(hitRate(dcp) * 100).toFixed(1)}% · miss ${fmt(Math.round(dcp.inputTokens - dcp.hitTokens))} tok`)
|
|
265
|
+
dcp.compactions.forEach((c, index) => {
|
|
266
|
+
console.log(` #${index + 1}: region ${fmt(c.regionHost)} tok (${c.regionNodes} nodes) → summary ${fmt(c.summaryHost)} tok · ${c.compression}x smaller · next request ${fmt(c.totalAfter)} tok (starts cold for every backend)`)
|
|
267
|
+
})
|
|
268
|
+
if (dcp.compactions.length > 0) {
|
|
269
|
+
const callInput = dcp.compactions.reduce((sum, c) => sum + c.regionHost, 0)
|
|
270
|
+
console.log(` LLM summarization calls: dcp 0 · compaction-basic ${dcp.compactions.length} (≈${fmt(Math.round(callInput))} input tokens replayed across them)`)
|
|
271
|
+
}
|
|
272
|
+
console.log('')
|
|
273
|
+
}
|
|
274
|
+
|
|
275
|
+
const last = simulate(events, 'cjk', true)
|
|
276
|
+
if (last.compactions.length > 0) {
|
|
277
|
+
const c = last.compactions[last.compactions.length - 1]
|
|
278
|
+
console.log('─ sample deterministic checkpoint (last compaction) ─')
|
|
279
|
+
console.log(c.preview)
|
|
280
|
+
console.log('')
|
|
281
|
+
console.log('Note: the request right after a compaction starts cold (hit ≈ 0)')
|
|
282
|
+
console.log('for ANY backend — compaction-basic included — because the leading')
|
|
283
|
+
console.log('span is replaced by brand-new checkpoint text. dcp only differs in')
|
|
284
|
+
console.log('summary size (here above) and in paying zero LLM calls per compaction.')
|
|
285
|
+
}
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* dsh-dcp setup: mount dsh-dcp into a cordis.patch.yml safely.
|
|
4
|
+
*
|
|
5
|
+
* - defaults to the home patch (`$DSH_HOME/cordis.patch.yml`, usually
|
|
6
|
+
* `~/.dsh/cordis.patch.yml`); `--profile <name>` targets that profile's
|
|
7
|
+
* patch; a positional argument targets an explicit file path
|
|
8
|
+
* - backs up an existing file to `<file>.bak.<YYYYMMDD-HHMM>` before touching it
|
|
9
|
+
* - appends only the dsh-dcp mount block — the user's own entries and
|
|
10
|
+
* comments are never rewritten
|
|
11
|
+
* - idempotent: skips when the mount is already present; never duplicates a
|
|
12
|
+
* `compaction-basic` entry
|
|
13
|
+
* - refuses to double-mount: if dsh-dcp is already a bundle in any affected
|
|
14
|
+
* profile, it aborts (use `--force` to override)
|
|
15
|
+
* - generates the file fresh when it does not exist
|
|
16
|
+
* - uses the running package's absolute entry path, so the mount resolves
|
|
17
|
+
* regardless of how dsh-dcp was installed
|
|
18
|
+
*
|
|
19
|
+
* Usage:
|
|
20
|
+
* node scripts/setup.mjs # home patch
|
|
21
|
+
* node scripts/setup.mjs --profile tui # tui profile's patch
|
|
22
|
+
* node scripts/setup.mjs /path/to/cordis.patch.yml
|
|
23
|
+
*/
|
|
24
|
+
import fs from 'node:fs'
|
|
25
|
+
import path from 'node:path'
|
|
26
|
+
import { fileURLToPath } from 'node:url'
|
|
27
|
+
import { planPatch, backupStamp, findBundledProfiles } from '../lib/setup.js'
|
|
28
|
+
|
|
29
|
+
const PKG = '@aiwayds/dsh-dcp'
|
|
30
|
+
const pkgRoot = path.dirname(path.dirname(fileURLToPath(import.meta.url)))
|
|
31
|
+
const ENTRY = path.join(pkgRoot, 'lib', 'index.js')
|
|
32
|
+
|
|
33
|
+
function home() {
|
|
34
|
+
return process.env.DSH_HOME || path.join(process.env.HOME ?? '.', '.dsh')
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
function parseArgs(argv) {
|
|
38
|
+
const force = argv.includes('--force')
|
|
39
|
+
if (argv.includes('--profile')) {
|
|
40
|
+
const index = argv.indexOf('--profile')
|
|
41
|
+
const name = argv[index + 1]
|
|
42
|
+
if (!name) throw new Error('setup: --profile requires a profile name')
|
|
43
|
+
return { target: path.join(home(), 'profiles', name, 'cordis.patch.yml'), kind: 'profile', profileName: name, force }
|
|
44
|
+
}
|
|
45
|
+
const explicit = argv.find((a) => !a.startsWith('-'))
|
|
46
|
+
if (explicit) return { target: path.resolve(explicit), kind: 'explicit', force }
|
|
47
|
+
return { target: path.join(home(), 'cordis.patch.yml'), kind: 'home', force }
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
const { target, kind, profileName, force } = parseArgs(process.argv.slice(2))
|
|
51
|
+
const existed = fs.existsSync(target)
|
|
52
|
+
const text = existed ? fs.readFileSync(target, 'utf8') : undefined
|
|
53
|
+
if (!existed) console.log(`no ${target} — will generate a fresh patch file`)
|
|
54
|
+
|
|
55
|
+
const plan = planPatch(text, { name: ENTRY })
|
|
56
|
+
if (plan.action === 'skip') {
|
|
57
|
+
console.log('dsh-dcp is already mounted in this patch file — nothing to do.')
|
|
58
|
+
process.exit(0)
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
// Refuse to double-mount: a home/profile patch applies to the same profile as
|
|
62
|
+
// the bundle mechanism would, and two dsh-dcp entries would crash the loader.
|
|
63
|
+
let affected = []
|
|
64
|
+
if (kind === 'profile') affected = [profileName]
|
|
65
|
+
else if (kind === 'home') {
|
|
66
|
+
try {
|
|
67
|
+
affected = findBundledProfiles(path.join(home(), 'profiles'), PKG)
|
|
68
|
+
} catch { /* profiles dir unreadable — proceed */ }
|
|
69
|
+
}
|
|
70
|
+
if (affected.length > 0 && !force) {
|
|
71
|
+
console.error(`ERROR: dsh-dcp is already a bundle in profile(s): ${affected.join(', ')}.`)
|
|
72
|
+
console.error('Mounting it in the patch file too would duplicate the entry id and crash the loader.')
|
|
73
|
+
console.error(`Remove it from those profiles' bundles (dsh plugin rm ${PKG}), or pass --force to override.`)
|
|
74
|
+
process.exit(1)
|
|
75
|
+
}
|
|
76
|
+
if (affected.length > 0 && force) {
|
|
77
|
+
console.warn(`WARN: overriding — dsh-dcp is already a bundle in: ${affected.join(', ')}. You are on your own if the loader rejects the duplicate.`)
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
if (plan.note) console.warn(`WARN: ${plan.note}`)
|
|
81
|
+
if (existed) {
|
|
82
|
+
// back up the user's file only now that we know we will modify it
|
|
83
|
+
const backup = `${target}.bak.${backupStamp()}`
|
|
84
|
+
fs.copyFileSync(target, backup)
|
|
85
|
+
console.log(`backup: ${backup}`)
|
|
86
|
+
}
|
|
87
|
+
fs.mkdirSync(path.dirname(target), { recursive: true })
|
|
88
|
+
fs.appendFileSync(target, plan.block)
|
|
89
|
+
console.log(`${plan.action === 'create' ? 'created' : 'patched'} ${target}`)
|
|
90
|
+
console.log('restart dsh, then run /dcp to verify.')
|