dsh-live-compact 1.1.5 → 1.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/README.md +21 -9
- package/gate.js +59 -6
- package/index.js +185 -25
- package/package.json +2 -2
- package/plan.js +27 -3
- package/raw.js +27 -7
- package/shadow.js +42 -15
- package/thunder.js +8 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# 版本变更
|
|
2
2
|
|
|
3
|
+
## 1.1.6
|
|
4
|
+
|
|
5
|
+
- **billion-context 共存**:检测 `bili-native` / `billion-context` 的实际 LOADING/ACTIVE 状态;对方接管官方压缩时,Live Compact 保留官方引擎并让位,finishTurn、compact、shadow 捕获、pruner、cursor 和 recovery 路径均不再改会话。支持 `SPINE_STAND_DOWN=1` 强制让位,并补充一次性告警和回归测试。
|
|
6
|
+
- **P0:隔离组接管与边界修复**:接管直接 `agentPresets.register()` 和 Include 加载的 `compaction` 隔离组,覆盖 Deep PTC 的 compaction 与 toolResultPruner 共存;修复 checkpoint、最新回合和 turn boundary 的误判,不把历史完成回合当成当前未完成回合。
|
|
7
|
+
- **P1:原始结果与压缩路径加固**:同一会话重复 `toolCallId` 对应不同原文时闭合失败,原文、元数据、owner 和卡片采用私有临时文件、权限核验、fsync 和原子发布;临时 I/O 错误可重试,永久 ACL/清理错误保持 fail-closed;增加 30 秒 shadow worker 超时、长 read 非法切点回退、准确 `keptTokens` 和 turn baseline 收敛,并保留 Thunder 事实来源与显式决策边界。
|
|
8
|
+
- **P1:model-level provider endpoint(#45)**:按精确的 provider + model catalog 地址判断 cloud/local,不改变宿主实际请求路由;provider-level 地址继续作为回退。
|
|
9
|
+
- 默认 TUI 行为不变;真实 TUI × Live Compact 联合验收、原用户 Windows 长会话复测和 #27/#34 现场复测仍按开放边界跟进。手动指定切点的 feature request(#48)不纳入本版。
|
|
10
|
+
|
|
3
11
|
## 1.1.5
|
|
4
12
|
|
|
5
13
|
- #27:把默认/共享根的容量扫描、轮换删除和 cut/official 发布事务移入按 root 串行的受管 worker,保留完整扫描与原有协调协议;worker 复用并在空闲时回收。补齐失败派发、迟到退出和 worker 重建的生命周期处理。
|
package/README.md
CHANGED
|
@@ -4,6 +4,18 @@
|
|
|
4
4
|
|
|
5
5
|
版本变更见 [CHANGELOG.md](CHANGELOG.md)。
|
|
6
6
|
|
|
7
|
+
## 测评实证
|
|
8
|
+
|
|
9
|
+
完整数据、原始口径和限制见 [docs/benchmarks.md](docs/benchmarks.md),结构化数字见 [docs/benchmark-results.json](docs/benchmark-results.json)。
|
|
10
|
+
|
|
11
|
+
- **1.0.5 失忆对比**:同一真实 DeepSeek 会话,15 道回忆题两边都是 15/15;Live Compact 的日志中段细节两次都是 3/3,原版一次 1/3、一次 3/3。
|
|
12
|
+
- **历史 DSH 长链路**:早期 150 道短题和 20 道 WebUniverse 连续任务都保留了原版/Live Compact 对照表;它们是历史实测,不是 1.1.6 的新验收。
|
|
13
|
+
- **OpenCode 移植参考**:独立项目 [opencode-live-compact](https://github.com/Haoming9527/opencode-live-compact) 把插件移植到 OpenCode,和 OpenCode 自带的 autocompact 做了对照:在 OpenCode 2.2.6 + GPT-6 Luna 上跑 Aider 150 和 Web-Bench React 20,任务分别 150/150、20/20 且最终浏览器测试 59/59;估算费用分别低 14.3% 和 42.1%。这是 OpenCode 项目的结果,用来说明移植后的可复核测评方法,不冒充 DSH 运行结果。
|
|
14
|
+
|
|
15
|
+

|
|
16
|
+
|
|
17
|
+

|
|
18
|
+
|
|
7
19
|
## 云端和本地怎么用
|
|
8
20
|
|
|
9
21
|
- **用 DeepSeek、GLM 这类云端 API**,按下面的命令安装即可。先看答案是否正确,再一起看缓存命中、未命中输入和实际费用。命中率的百分比不是费用本身,输入变少时要连同 token 数一起比较。
|
|
@@ -16,13 +28,13 @@
|
|
|
16
28
|
headless:
|
|
17
29
|
|
|
18
30
|
```sh
|
|
19
|
-
dsh plugin --profile headless add dsh-live-compact@1.1.
|
|
31
|
+
dsh plugin --profile headless add dsh-live-compact@1.1.6
|
|
20
32
|
```
|
|
21
33
|
|
|
22
34
|
网页端:
|
|
23
35
|
|
|
24
36
|
```sh
|
|
25
|
-
dsh plugin --profile web add dsh-live-compact@1.1.
|
|
37
|
+
dsh plugin --profile web add dsh-live-compact@1.1.6
|
|
26
38
|
```
|
|
27
39
|
|
|
28
40
|
想先装到本机看版本:
|
|
@@ -150,7 +162,7 @@ Windows 会重新设置备份访问权限,失败时停止写备份并保留原
|
|
|
150
162
|
|
|
151
163
|
插件小切参数在每次轮后维护开始时取得本引擎的一致快照,保存后下一次维护生效,多个引擎互不覆盖。官方大压缩参数沿用宿主的构造时快照,保存后需要重建引擎或会话。宿主支持的 `retainRatio`、`maxTokens`、摘要模型路由、`modelPolicies`、溢出重试等配置继续由宿主校验并保留;`retainRatio` 与 `retainTokens` 互斥。
|
|
152
164
|
|
|
153
|
-
|
|
165
|
+
接管开关只影响未来加载的预设及 `compaction` 隔离组;直接注册预设和 Include 加载的组也按原位换包。新预设按 **预设引擎显式配置 > 根插件显式配置 > 有效环境变量 > 默认值** 取值;已有预设保留加载时的数值,根设置后续修改需要重新加载预设。原生客户端 slot、资源声明和安装方式沿用现有接口。
|
|
154
166
|
|
|
155
167
|
每个新的 `compact-checkpoint` 只让轮后小切跳过一次,`spine-checkpoint` 不触发这个跳过。
|
|
156
168
|
|
|
@@ -174,8 +186,8 @@ Windows 会重新设置备份访问权限,失败时停止写备份并保留原
|
|
|
174
186
|
从 npm 装。headless 和网页是两个 profile,要各装一次。桌面端是另一个 profile,普通 `dsh plugin --profile desktop` 会被拒绝。
|
|
175
187
|
|
|
176
188
|
```sh
|
|
177
|
-
dsh plugin --profile headless add dsh-live-compact@1.1.
|
|
178
|
-
dsh plugin --profile web add dsh-live-compact@1.1.
|
|
189
|
+
dsh plugin --profile headless add dsh-live-compact@1.1.6
|
|
190
|
+
dsh plugin --profile web add dsh-live-compact@1.1.6
|
|
179
191
|
```
|
|
180
192
|
|
|
181
193
|
想先装到本机看版本:
|
|
@@ -188,15 +200,15 @@ dsh-live-compact --version
|
|
|
188
200
|
上不了 npm 时的镜像安装命令:
|
|
189
201
|
|
|
190
202
|
```sh
|
|
191
|
-
dsh plugin --profile headless add https://download.merkyorlynn.com/downloads/plugins/dsh-live-compact-1.1.
|
|
192
|
-
dsh plugin --profile web add https://download.merkyorlynn.com/downloads/plugins/dsh-live-compact-1.1.
|
|
193
|
-
npm install -g --force "https://download.merkyorlynn.com/downloads/plugins/dsh-live-compact-1.1.
|
|
203
|
+
dsh plugin --profile headless add https://download.merkyorlynn.com/downloads/plugins/dsh-live-compact-1.1.6.tgz
|
|
204
|
+
dsh plugin --profile web add https://download.merkyorlynn.com/downloads/plugins/dsh-live-compact-1.1.6.tgz
|
|
205
|
+
npm install -g --force "https://download.merkyorlynn.com/downloads/plugins/dsh-live-compact-1.1.6.tgz"
|
|
194
206
|
dsh-live-compact --version
|
|
195
207
|
```
|
|
196
208
|
|
|
197
209
|
需要 DSH `>=0.2.0-rc.2 <0.3.0`。装上后跑长任务,日志里出现 `spine cut …ms` 才算这刀生效。页面:https://download.merkyorlynn.com/live-compact.html
|
|
198
210
|
|
|
199
|
-
DSH-TUI
|
|
211
|
+
DSH-TUI 先不要装。本包已补预设压缩隔离组的接管;TUI 仍需适配 `spine-checkpoint` 的局部卡片显示,两仓联合验收后再开放支持。
|
|
200
212
|
|
|
201
213
|
被切掉的原文会写到 `~/.dsh/compaction-spine/shadow`。Unix 权限为目录 0700 / 文件 0600;Windows 新建/修正后的备份允许当前用户、SYSTEM、Administrators,历史文件及缓存限制见上面的 ACL 说明。上限 256MB。快满时删最老的备份。卡片里的序号是会话日志的事件号。要在当前会话的备份里找一行,用 `import { searchShadow } from "dsh-live-compact/shadow-search"`。插件自己不会去搜。
|
|
202
214
|
|
package/gate.js
CHANGED
|
@@ -4,9 +4,55 @@ import { configSnapshot, engineConfig } from "./settings.js";
|
|
|
4
4
|
|
|
5
5
|
const OURS = "dsh-live-compact";
|
|
6
6
|
const PRESET_IDS = new Set(["standard", "ptc", "minimal", "cordis", "preset-standard", "preset-ptc", "preset-minimal", "preset-cordis"]);
|
|
7
|
+
const COMPETING_COMPRESSORS = new Set(["bili-native", "billion-context"]);
|
|
8
|
+
const COMPETING_FIBER_STATES = new Set([1, 2]); // Cordis 4.1: LOADING=1, ACTIVE=2.
|
|
9
|
+
const standDownNotices = new WeakMap();
|
|
7
10
|
export const GATE_SERVICE = "liveCompactPresetGate";
|
|
8
11
|
export const PRESET_ENGINE_CONFIG = Object.freeze(engineConfig());
|
|
9
12
|
|
|
13
|
+
/**
|
|
14
|
+
* Return the exact active compressor runtimes that own the session when they
|
|
15
|
+
* are loading or active. Cordis keeps one runtime per plugin callback and
|
|
16
|
+
* exposes every live fiber through `ctx.registry.values()`; a package on disk
|
|
17
|
+
* or a failed/unloading fiber is not an active compressor.
|
|
18
|
+
*/
|
|
19
|
+
export function activeCompactionPeers(ctx) {
|
|
20
|
+
const peers = [];
|
|
21
|
+
if (process.env.SPINE_STAND_DOWN === "1") peers.push("SPINE_STAND_DOWN");
|
|
22
|
+
const registry = ctx?.registry;
|
|
23
|
+
if (!registry || typeof registry.values !== "function") return peers;
|
|
24
|
+
try {
|
|
25
|
+
for (const runtime of registry.values()) {
|
|
26
|
+
const name = runtime?.name;
|
|
27
|
+
if (!COMPETING_COMPRESSORS.has(name)) continue;
|
|
28
|
+
const active = [...(runtime?.fibers || [])].some((fiber) => COMPETING_FIBER_STATES.has(fiber?.state));
|
|
29
|
+
if (active && !peers.includes(name)) peers.push(name);
|
|
30
|
+
}
|
|
31
|
+
} catch {
|
|
32
|
+
// Cordis 4.1 always exposes an iterable registry. If an older/partial host
|
|
33
|
+
// cannot enumerate it, preserve the single-plugin path and do not guess a
|
|
34
|
+
// competing package from installation state.
|
|
35
|
+
}
|
|
36
|
+
return peers;
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
export function compactionStandDown(ctx) {
|
|
40
|
+
return activeCompactionPeers(ctx).length > 0;
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
export function warnCompactionStandDown(ctx, logger = ctx?.logger) {
|
|
44
|
+
const peers = activeCompactionPeers(ctx);
|
|
45
|
+
if (!peers.length) return peers;
|
|
46
|
+
const root = ctx?.root || ctx;
|
|
47
|
+
const key = peers.join(",");
|
|
48
|
+
if (root && standDownNotices.get(root) === key) return peers;
|
|
49
|
+
if (root) standDownNotices.set(root, key);
|
|
50
|
+
logger?.warn?.(peers[0] === "SPINE_STAND_DOWN"
|
|
51
|
+
? "spine: SPINE_STAND_DOWN=1,已有压缩插件接管,本插件不切、不改预设。"
|
|
52
|
+
: `spine: 已有压缩插件接管(${peers.join(", ")}),本插件不切、不改预设。`);
|
|
53
|
+
return peers;
|
|
54
|
+
}
|
|
55
|
+
|
|
10
56
|
const require = createRequire(import.meta.url);
|
|
11
57
|
|
|
12
58
|
export function packageLoadable(load = (name) => require(name)) {
|
|
@@ -66,8 +112,9 @@ function rewriteNode(node, loadable, rootConfig) {
|
|
|
66
112
|
return next;
|
|
67
113
|
}
|
|
68
114
|
|
|
69
|
-
export function rewritePresetConfig(config, loadable = packageLoadable(), rootConfig = {}) {
|
|
70
|
-
|
|
115
|
+
export function rewritePresetConfig(config, loadable = packageLoadable(), rootConfig = {}, entry) {
|
|
116
|
+
const group = Array.isArray(config) && entry?.group === true && entry.isolate?.compaction === true;
|
|
117
|
+
if (!isPreset(config) && !group) return config;
|
|
71
118
|
return rewriteNode(config, loadable, rootConfig);
|
|
72
119
|
}
|
|
73
120
|
|
|
@@ -75,6 +122,7 @@ function engineEntry(node) {
|
|
|
75
122
|
if (!node || typeof node !== "object") return null;
|
|
76
123
|
if (node.name === OURS) return node;
|
|
77
124
|
const nested = [];
|
|
125
|
+
if (Array.isArray(node)) nested.push(...node);
|
|
78
126
|
if (Array.isArray(node.plugins)) nested.push(...node.plugins);
|
|
79
127
|
if (Array.isArray(node.config)) nested.push(...node.config);
|
|
80
128
|
else if (node.config && typeof node.config === "object") nested.push(node.config);
|
|
@@ -113,13 +161,18 @@ export function installPresetGate(ctx, config = {}, { legacy = false } = {}) {
|
|
|
113
161
|
return typeof value === "boolean" ? value : value.get();
|
|
114
162
|
};
|
|
115
163
|
const seen = new Set();
|
|
116
|
-
gate.disposeListener = ctx.on("internal/config", function (
|
|
164
|
+
gate.disposeListener = ctx.on("internal/config", function (fiber, next) {
|
|
117
165
|
const resolved = next();
|
|
166
|
+
if (compactionStandDown(ctx)) {
|
|
167
|
+
warnCompactionStandDown(ctx);
|
|
168
|
+
return resolved;
|
|
169
|
+
}
|
|
118
170
|
if (!gate.active || !enabled()) return resolved;
|
|
119
171
|
const loadable = packageLoadable();
|
|
120
|
-
const
|
|
121
|
-
const
|
|
122
|
-
|
|
172
|
+
const entry = fiber?.entry?.options;
|
|
173
|
+
const rewritten = rewritePresetConfig(resolved, loadable, config, entry);
|
|
174
|
+
const id = rewritten?.id || rewritten?.config?.id || entry?.id;
|
|
175
|
+
if (rewritten !== resolved && id && !seen.has(String(id))) {
|
|
123
176
|
seen.add(String(id));
|
|
124
177
|
const engine = engineEntry(rewritten);
|
|
125
178
|
if (engine) {
|
package/index.js
CHANGED
|
@@ -39,6 +39,7 @@ import {
|
|
|
39
39
|
} from "./shadow.js";
|
|
40
40
|
import { buildSendLess, decideThunder } from "./thunder.js";
|
|
41
41
|
import { inCooldown, skipLiveOfficialCompaction } from "./cooldown.js";
|
|
42
|
+
import { activeCompactionPeers, warnCompactionStandDown } from "./gate.js";
|
|
42
43
|
|
|
43
44
|
const { BasicCompactionEngine } = await loadBase();
|
|
44
45
|
const { CompactionId, toolPairingBalancedAfter, toolPairingBalancedBefore } = await loadCompaction();
|
|
@@ -56,6 +57,7 @@ const CAPTURE_RESULT_BATCH = 1;
|
|
|
56
57
|
const RECOVER_RAW_BATCH = 1;
|
|
57
58
|
const CAPTURE_RETRY_LIMIT = 8;
|
|
58
59
|
const RECOVERY_ACL_REAUTH_LIMIT = 32;
|
|
60
|
+
const SHADOW_WORKER_TIMEOUT_MS = 30000;
|
|
59
61
|
|
|
60
62
|
const shadowWorkers = new Map();
|
|
61
63
|
|
|
@@ -88,7 +90,10 @@ function shadowWorkerFor(root) {
|
|
|
88
90
|
clearTimeout(entry.idleTimer);
|
|
89
91
|
entry.idleTimer = null;
|
|
90
92
|
}
|
|
91
|
-
for (const { reject } of entry.pending.values())
|
|
93
|
+
for (const { reject, timer } of entry.pending.values()) {
|
|
94
|
+
if (timer) clearTimeout(timer);
|
|
95
|
+
reject(error);
|
|
96
|
+
}
|
|
92
97
|
entry.pending.clear();
|
|
93
98
|
removeEntry();
|
|
94
99
|
worker.terminate().catch(() => {});
|
|
@@ -96,6 +101,7 @@ function shadowWorkerFor(root) {
|
|
|
96
101
|
worker.on("message", (message) => {
|
|
97
102
|
const task = entry.pending.get(message?.id);
|
|
98
103
|
if (!task) return;
|
|
104
|
+
if (task.timer) clearTimeout(task.timer);
|
|
99
105
|
entry.pending.delete(message.id);
|
|
100
106
|
if (message?.ok) {
|
|
101
107
|
task.resolve(message.result);
|
|
@@ -130,10 +136,16 @@ function runShadowWorker(payload) {
|
|
|
130
136
|
}
|
|
131
137
|
const id = entry.nextId++;
|
|
132
138
|
return new Promise((resolve, reject) => {
|
|
133
|
-
|
|
139
|
+
const timer = setTimeout(() => {
|
|
140
|
+
const error = Object.assign(new Error("spine: shadow worker timed out"), { code: "ESHADOWWORKER" });
|
|
141
|
+
failShadowWorker(entry, error);
|
|
142
|
+
}, SHADOW_WORKER_TIMEOUT_MS);
|
|
143
|
+
timer.unref?.();
|
|
144
|
+
entry.pending.set(id, { resolve, reject, timer });
|
|
134
145
|
try {
|
|
135
146
|
entry.worker.postMessage({ id, payload });
|
|
136
147
|
} catch (error) {
|
|
148
|
+
clearTimeout(timer);
|
|
137
149
|
entry.pending.delete(id);
|
|
138
150
|
reject(error);
|
|
139
151
|
entry.scheduleIdleExit();
|
|
@@ -141,6 +153,24 @@ function runShadowWorker(payload) {
|
|
|
141
153
|
});
|
|
142
154
|
}
|
|
143
155
|
|
|
156
|
+
function failShadowWorker(entry, error) {
|
|
157
|
+
if (!entry || entry.retired) return;
|
|
158
|
+
entry.retired = true;
|
|
159
|
+
if (entry.idleTimer) {
|
|
160
|
+
clearTimeout(entry.idleTimer);
|
|
161
|
+
entry.idleTimer = null;
|
|
162
|
+
}
|
|
163
|
+
for (const { reject, timer } of entry.pending.values()) {
|
|
164
|
+
if (timer) clearTimeout(timer);
|
|
165
|
+
reject(error);
|
|
166
|
+
}
|
|
167
|
+
entry.pending.clear();
|
|
168
|
+
for (const [key, value] of shadowWorkers) {
|
|
169
|
+
if (value === entry) shadowWorkers.delete(key);
|
|
170
|
+
}
|
|
171
|
+
entry.worker.terminate().catch(() => {});
|
|
172
|
+
}
|
|
173
|
+
|
|
144
174
|
function recoveryRetryable(record, dir, toolCallId) {
|
|
145
175
|
if (record?.retryable === true || record?.reason === "unwritable") return true;
|
|
146
176
|
// A completely absent backup has never been captured and can be retried.
|
|
@@ -150,6 +180,11 @@ function recoveryRetryable(record, dir, toolCallId) {
|
|
|
150
180
|
return false;
|
|
151
181
|
}
|
|
152
182
|
|
|
183
|
+
function captureErrorRetryable(error) {
|
|
184
|
+
const code = error?.code;
|
|
185
|
+
return code === "ESHADOWBUSY" || ["EAGAIN", "EBUSY", "EINTR", "EIO"].includes(code);
|
|
186
|
+
}
|
|
187
|
+
|
|
153
188
|
function eventContent(session, seq) {
|
|
154
189
|
const event = session.eventAt(seq);
|
|
155
190
|
if (event?.type === "assistant/message") return event.data?.message?.content ?? event.data?.content;
|
|
@@ -170,7 +205,27 @@ function isCheckpoint(session, seq) {
|
|
|
170
205
|
// 自己的卡片,以及已经写过的官方标记,都要停住。新卡片不再用官方标记。
|
|
171
206
|
// Stop on our own cards and on older official markers. New cards do not use the official marker.
|
|
172
207
|
if (kind === SPINE_SOURCE_KIND || kind === "compact-checkpoint") return true;
|
|
173
|
-
|
|
208
|
+
// The legacy text marker is only meaningful on a user/message card and only
|
|
209
|
+
// at the beginning of its text. A tool result or an ordinary sentence that
|
|
210
|
+
// merely mentions the marker must not create a permanent cut boundary.
|
|
211
|
+
if (event?.type !== "user/message") return false;
|
|
212
|
+
const text = eventText(session, seq).trimStart();
|
|
213
|
+
return text === MARK || text.startsWith(`${MARK}\n`) || text.startsWith(`${MARK}\r\n`);
|
|
214
|
+
}
|
|
215
|
+
|
|
216
|
+
function latestTurnCompleted(events) {
|
|
217
|
+
let lastStart = -1;
|
|
218
|
+
let lastEnd = -1;
|
|
219
|
+
let reason;
|
|
220
|
+
for (let index = 0; index < (events || []).length; index += 1) {
|
|
221
|
+
const event = events[index];
|
|
222
|
+
if (event?.type === "turn/start") lastStart = index;
|
|
223
|
+
if (event?.type === "turn/end") {
|
|
224
|
+
lastEnd = index;
|
|
225
|
+
reason = event.data?.reason?.kind;
|
|
226
|
+
}
|
|
227
|
+
}
|
|
228
|
+
return lastEnd > lastStart && reason === "completed";
|
|
174
229
|
}
|
|
175
230
|
|
|
176
231
|
function spineCheckpointSource(compactionId) {
|
|
@@ -367,8 +422,6 @@ function selectNewestChunk(session, measurement, key = DEFAULT_KEY, context = {}
|
|
|
367
422
|
const cutEnd = span ? reservedCutEnd(visible[span.start].index, first) : null;
|
|
368
423
|
if (span && cutEnd === null) return null;
|
|
369
424
|
if (cutEnd !== null) {
|
|
370
|
-
const keptEnd = visible[span.end].index;
|
|
371
|
-
for (let index = visible[span.start].index; index <= keptEnd; index += 1) keptTokens += priced[index].tokens;
|
|
372
425
|
end = cutEnd;
|
|
373
426
|
while (end >= first && classifications[end].isGenuineUserBoundary && !(floorCard >= 0 && end - 1 <= floorCard)) end -= 1;
|
|
374
427
|
if (floorCard >= 0 && end <= floorCard) return null;
|
|
@@ -456,6 +509,9 @@ function selectNewestChunk(session, measurement, key = DEFAULT_KEY, context = {}
|
|
|
456
509
|
};
|
|
457
510
|
let usedLongCut = false;
|
|
458
511
|
let picked = null;
|
|
512
|
+
const fallbackWithoutLongRead = () => context.disableLongRead
|
|
513
|
+
? null
|
|
514
|
+
: selectNewestChunk(session, measurement, key, { ...context, disableLongRead: true });
|
|
459
515
|
if (longSpan) {
|
|
460
516
|
const longPicked = cutLongReadRound(items, longSpan.start, longSpan.end, cutOptions);
|
|
461
517
|
if (longPicked && longPicked.start >= longSpan.start && longPicked.end <= longSpan.end) {
|
|
@@ -481,27 +537,39 @@ function selectNewestChunk(session, measurement, key = DEFAULT_KEY, context = {}
|
|
|
481
537
|
}
|
|
482
538
|
}
|
|
483
539
|
}
|
|
484
|
-
if (pairStart < 0) return null;
|
|
485
|
-
if (usedLongCut && pairStart < longSpan.start) return
|
|
486
|
-
if (!pointerSpanAllowed(items, pairStart, picked.end, context.engine || "cloud")) return null;
|
|
487
|
-
if (!cutOk(toolPairingBalancedBefore, session, nodes[pairStart])) return null;
|
|
540
|
+
if (pairStart < 0) return usedLongCut ? fallbackWithoutLongRead() : null;
|
|
541
|
+
if (usedLongCut && pairStart < longSpan.start) return fallbackWithoutLongRead();
|
|
542
|
+
if (!pointerSpanAllowed(items, pairStart, picked.end, context.engine || "cloud")) return usedLongCut ? fallbackWithoutLongRead() : null;
|
|
543
|
+
if (!cutOk(toolPairingBalancedBefore, session, nodes[pairStart])) return usedLongCut ? fallbackWithoutLongRead() : null;
|
|
488
544
|
picked.start = pairStart;
|
|
489
545
|
}
|
|
490
|
-
if (usedLongCut && (picked.start < longSpan.start || picked.end > longSpan.end)) return
|
|
546
|
+
if (usedLongCut && (picked.start < longSpan.start || picked.end > longSpan.end)) return fallbackWithoutLongRead();
|
|
491
547
|
const start = picked.start;
|
|
492
548
|
end = picked.end;
|
|
493
549
|
const tokens = items.slice(start, end + 1).reduce((sum, item) => sum + item.tokens, 0);
|
|
494
550
|
const pieces = spanPieces(start, end);
|
|
495
|
-
if (picked.pointerOnly && pieces.some((piece) => piece.role === "tool" && isToolFailure(piece.text, piece.isError)))
|
|
551
|
+
if (picked.pointerOnly && pieces.some((piece) => piece.role === "tool" && isToolFailure(piece.text, piece.isError))) {
|
|
552
|
+
return usedLongCut ? fallbackWithoutLongRead() : null;
|
|
553
|
+
}
|
|
496
554
|
let card = picked.pointerOnly
|
|
497
555
|
? { ok: true, text: pointerOnlyText(pieces), hardLines: [], missing: [], needsBackup: false, rawPaths: [] }
|
|
498
556
|
: buildCard(pieces.map((piece) => piece.text).join("\n"), liveText(start, end), pieces, extras);
|
|
499
|
-
if (card.ok !== true) return null;
|
|
557
|
+
if (card.ok !== true) return usedLongCut ? fallbackWithoutLongRead() : null;
|
|
500
558
|
card = attachBackupHint(card, freshShadowFile("cut", key));
|
|
501
559
|
if (!card.text.includes("不是备份文件路径")) card = { ...card, text: `${card.text}\n序号是本会话日志里的事件号,不是备份文件路径。` };
|
|
502
560
|
if (!cutOk(toolPairingBalancedAfter, session, nodes[end])) return null;
|
|
503
561
|
let after = 0;
|
|
504
562
|
for (let index = end + 1; index < priced.length; index += 1) after += priced[index].tokens;
|
|
563
|
+
// `keptTokens` must describe retained rounds to the right of the actual
|
|
564
|
+
// cut. A long-read cut can be newer than the ordinary rounds selected above;
|
|
565
|
+
// counting those older rounds made `tailBeyondKept` negative and disabled
|
|
566
|
+
// the tail guard.
|
|
567
|
+
for (const round of rounds) {
|
|
568
|
+
const roundStart = visible[round.start]?.index;
|
|
569
|
+
const roundEnd = visible[round.end]?.index;
|
|
570
|
+
if (!Number.isInteger(roundStart) || !Number.isInteger(roundEnd) || roundStart <= end) continue;
|
|
571
|
+
for (let index = roundStart; index <= roundEnd; index += 1) keptTokens += priced[index].tokens;
|
|
572
|
+
}
|
|
505
573
|
let prefix = 0;
|
|
506
574
|
for (let index = 0; index < start; index += 1) prefix += priced[index].tokens;
|
|
507
575
|
return { start: nodes[start], end: nodes[end], tokens, after, keptTokens, card, prefix };
|
|
@@ -544,6 +612,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
544
612
|
});
|
|
545
613
|
ctx.on("session/event", (session, event) => {
|
|
546
614
|
try {
|
|
615
|
+
if (this.standDown?.()) return;
|
|
547
616
|
this.noteEvent(session, event);
|
|
548
617
|
if (event?.type !== "tool/result") return;
|
|
549
618
|
const id = event.data?.message?.toolCallId;
|
|
@@ -557,6 +626,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
557
626
|
job = new Promise((resolve) => {
|
|
558
627
|
scheduleImmediate(() => {
|
|
559
628
|
try {
|
|
629
|
+
if (this.standDown?.()) return;
|
|
560
630
|
const current = this.rawSeen.get(key);
|
|
561
631
|
if (!current || current.retryable) {
|
|
562
632
|
const record = this.captureOne(session, event);
|
|
@@ -580,10 +650,11 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
580
650
|
});
|
|
581
651
|
ctx.on("agent/status", ({ agent, status }) => {
|
|
582
652
|
if (status !== "idle") return;
|
|
653
|
+
if (this.standDown?.()) return;
|
|
583
654
|
const state = this.stateFor(agent);
|
|
584
655
|
if (state.warming) return;
|
|
585
656
|
const events = agent.session.snapshotEvents();
|
|
586
|
-
const done = events
|
|
657
|
+
const done = latestTurnCompleted(events);
|
|
587
658
|
if (!done) return;
|
|
588
659
|
state.warming = true;
|
|
589
660
|
const job = this.finishTurn(agent, events, { deferCapture: true })
|
|
@@ -600,6 +671,24 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
600
671
|
}
|
|
601
672
|
}
|
|
602
673
|
|
|
674
|
+
standDown() {
|
|
675
|
+
const peers = activeCompactionPeers(this.ctx);
|
|
676
|
+
if (peers.length) warnCompactionStandDown(this.ctx);
|
|
677
|
+
return peers.length > 0;
|
|
678
|
+
}
|
|
679
|
+
|
|
680
|
+
ownedSignal(signal) {
|
|
681
|
+
const controller = new AbortController();
|
|
682
|
+
const dispose = this.ctx.on("internal/status", () => {
|
|
683
|
+
if (this.standDown?.()) controller.abort();
|
|
684
|
+
}, { global: true });
|
|
685
|
+
if (this.standDown?.()) controller.abort();
|
|
686
|
+
return {
|
|
687
|
+
signal: signal ? AbortSignal.any([signal, controller.signal]) : controller.signal,
|
|
688
|
+
dispose,
|
|
689
|
+
};
|
|
690
|
+
}
|
|
691
|
+
|
|
603
692
|
stateFor(agent) {
|
|
604
693
|
const key = sessionKey(agent);
|
|
605
694
|
let state = this.sessionState.get(key);
|
|
@@ -700,6 +789,10 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
700
789
|
scheduleImmediate(() => {
|
|
701
790
|
task.scheduled = false;
|
|
702
791
|
try {
|
|
792
|
+
if (this.standDown?.()) {
|
|
793
|
+
this.finishCaptureCatchup(task);
|
|
794
|
+
return;
|
|
795
|
+
}
|
|
703
796
|
const latest = task.freezeSnapshot ? null : task.session.snapshotEvents?.();
|
|
704
797
|
const latestTail = Number.isInteger(latest?.at?.(-1)?.seq) ? latest.at(-1).seq : -1;
|
|
705
798
|
const latestSurfaceTail = task.session.surface?.nodes?.at?.(-1) ?? -1;
|
|
@@ -779,18 +872,20 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
779
872
|
this.pendingCatchups.delete(task.key);
|
|
780
873
|
this.pendingCatchupJobs.delete(task.promise);
|
|
781
874
|
task.resolve();
|
|
782
|
-
if (task.retryRequested && task.progressed) this.scheduleMaintenanceRetry(task.agent);
|
|
875
|
+
if (!this.standDown?.() && task.retryRequested && task.progressed) this.scheduleMaintenanceRetry(task.agent);
|
|
783
876
|
}
|
|
784
877
|
|
|
785
878
|
scheduleMaintenanceRetry(agent) {
|
|
879
|
+
if (this.standDown?.()) return;
|
|
786
880
|
const state = this.stateFor(agent);
|
|
787
881
|
if (state.retryScheduled) return;
|
|
788
882
|
state.retryScheduled = true;
|
|
789
883
|
scheduleImmediate(() => {
|
|
790
884
|
state.retryScheduled = false;
|
|
885
|
+
if (this.standDown?.()) return;
|
|
791
886
|
if (state.warming) return;
|
|
792
887
|
const events = agent.session.snapshotEvents();
|
|
793
|
-
const done = events
|
|
888
|
+
const done = latestTurnCompleted(events);
|
|
794
889
|
if (!done) return;
|
|
795
890
|
state.warming = true;
|
|
796
891
|
const job = this.finishTurn(agent, events, { deferCapture: true })
|
|
@@ -804,6 +899,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
804
899
|
}
|
|
805
900
|
|
|
806
901
|
async finishTurn(agent, events, options = {}) {
|
|
902
|
+
if (this.standDown?.()) return null;
|
|
807
903
|
const settings = readSettings(this.liveConfig);
|
|
808
904
|
const state = this.stateFor(agent);
|
|
809
905
|
const signal = AbortSignal.timeout(30000);
|
|
@@ -824,16 +920,28 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
824
920
|
const calls = indexToolCalls(ev);
|
|
825
921
|
try {
|
|
826
922
|
await agent.runMaintenance(async (agentSignal) => {
|
|
923
|
+
if (this.standDown?.()) return;
|
|
827
924
|
const op = AbortSignal.any([agentSignal, signal]);
|
|
828
925
|
let measurement = this.ctx.tokenMeter.measure(agent.session);
|
|
829
|
-
|
|
926
|
+
const settleTurnBaseline = () => {
|
|
927
|
+
state.turnBase = measurement.totalTokens;
|
|
928
|
+
state.maxTurnGrowth = 0;
|
|
929
|
+
};
|
|
930
|
+
if (measurement.totalTokens < settings.minTokens || measurement.totalTokens >= settings.bigGate) {
|
|
931
|
+
settleTurnBaseline();
|
|
932
|
+
return;
|
|
933
|
+
}
|
|
830
934
|
// Restored cards establish a baseline; a live card still skips once,
|
|
831
935
|
// even when maintenance or engine construction is delayed.
|
|
832
936
|
if (skipLiveOfficialCompaction(agent.session)) {
|
|
833
937
|
this.ctx.logger?.info?.("spine cooldown: live official compaction seen, skip this pass");
|
|
938
|
+
settleTurnBaseline();
|
|
939
|
+
return;
|
|
940
|
+
}
|
|
941
|
+
if (inCooldown(agent.session, { windowMs: settings.cooldownMs })) {
|
|
942
|
+
settleTurnBaseline();
|
|
834
943
|
return;
|
|
835
944
|
}
|
|
836
|
-
if (inCooldown(agent.session, { windowMs: settings.cooldownMs })) return;
|
|
837
945
|
this.ensurePruneGuard();
|
|
838
946
|
const resolved = this.engineFor(agent);
|
|
839
947
|
if (state.engine !== resolved.engine || state.engineSource !== resolved.source) {
|
|
@@ -880,7 +988,8 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
880
988
|
state.localOnly = true;
|
|
881
989
|
state.approvedCard = chunk.card;
|
|
882
990
|
await this.flushPendingCaptures(agent.session, this.chunkToolCallIds(agent.session, chunk));
|
|
883
|
-
await this.commitSelectedSpan(agent, chunk, op);
|
|
991
|
+
const committed = await this.commitSelectedSpan(agent, chunk, op);
|
|
992
|
+
if (committed === false) break;
|
|
884
993
|
measurement = this.ctx.tokenMeter.measure(agent.session);
|
|
885
994
|
const after = measurement.totalTokens;
|
|
886
995
|
reduced = after < before;
|
|
@@ -915,6 +1024,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
915
1024
|
}
|
|
916
1025
|
if (drainNotes.length) flushDrainStats(shadowDirectory(), drainNotes, { sid: state.key, drain });
|
|
917
1026
|
state.turnBase = measurement.totalTokens;
|
|
1027
|
+
state.maxTurnGrowth = 0;
|
|
918
1028
|
await this.ctx.sessions?.flush?.(agent.session);
|
|
919
1029
|
this.rememberPrefix(agent);
|
|
920
1030
|
});
|
|
@@ -956,6 +1066,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
956
1066
|
}
|
|
957
1067
|
|
|
958
1068
|
async commitSelectedSpan(agent, chunk, signal) {
|
|
1069
|
+
if (this.standDown?.()) return false;
|
|
959
1070
|
signal?.throwIfAborted();
|
|
960
1071
|
const session = agent.session;
|
|
961
1072
|
if (chunk.card?.ok !== true || !chunk.card.text) throw new Error("spine: approved card missing; leaving the span");
|
|
@@ -1022,6 +1133,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1022
1133
|
}
|
|
1023
1134
|
const summarized = await this.summarize({ messages, liveCompact: { card: chunk.card } }, agent, signal);
|
|
1024
1135
|
signal?.throwIfAborted();
|
|
1136
|
+
if (this.standDown?.()) return false;
|
|
1025
1137
|
assertCurrentSpan(shadowedSeqs);
|
|
1026
1138
|
const compactionId = CompactionId(randomUUID());
|
|
1027
1139
|
const checkpointMessage = createUserMessage({
|
|
@@ -1081,11 +1193,27 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1081
1193
|
}
|
|
1082
1194
|
|
|
1083
1195
|
async compactIfNeeded(agent, trigger, signal) {
|
|
1196
|
+
if (this.standDown?.()) return null;
|
|
1084
1197
|
const state = this.stateFor(agent);
|
|
1085
1198
|
state.liveAgent = agent;
|
|
1086
1199
|
state.localOnly = false;
|
|
1087
1200
|
this.ensurePruneGuard();
|
|
1088
|
-
|
|
1201
|
+
const owned = this.ownedSignal(signal);
|
|
1202
|
+
try {
|
|
1203
|
+
return await super.compactIfNeeded(agent, trigger, owned.signal);
|
|
1204
|
+
} finally {
|
|
1205
|
+
owned.dispose?.();
|
|
1206
|
+
}
|
|
1207
|
+
}
|
|
1208
|
+
|
|
1209
|
+
async compactNow(agent, signal, sourceCommandId) {
|
|
1210
|
+
if (this.standDown?.()) return null;
|
|
1211
|
+
const owned = this.ownedSignal(signal);
|
|
1212
|
+
try {
|
|
1213
|
+
return await super.compactNow(agent, owned.signal, sourceCommandId);
|
|
1214
|
+
} finally {
|
|
1215
|
+
owned.dispose?.();
|
|
1216
|
+
}
|
|
1089
1217
|
}
|
|
1090
1218
|
|
|
1091
1219
|
// Preserve explicit URL precedence. Only the exact account provider gets
|
|
@@ -1094,9 +1222,11 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1094
1222
|
const header = agent?.session?.requestHeader?.()?.config || {};
|
|
1095
1223
|
const context = agent?.session?.requestContext?.() || {};
|
|
1096
1224
|
const provider = header.provider || context.provider || "";
|
|
1225
|
+
const model = header.model || context.model || "";
|
|
1097
1226
|
const listed = this.listedProviders();
|
|
1098
1227
|
const baseURL = resolveProviderBaseURL({
|
|
1099
1228
|
provider,
|
|
1229
|
+
model,
|
|
1100
1230
|
headerURL: header.baseURL || context.baseURL || "",
|
|
1101
1231
|
listed,
|
|
1102
1232
|
layers: this.providerLayers(),
|
|
@@ -1168,10 +1298,21 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1168
1298
|
try {
|
|
1169
1299
|
const loaded = createRequire(anchor)("@earendil-works/pi-ai/providers/all");
|
|
1170
1300
|
const list = typeof loaded.builtinProviders === "function" ? loaded.builtinProviders() : [];
|
|
1171
|
-
return list.map((entry) =>
|
|
1172
|
-
|
|
1173
|
-
|
|
1174
|
-
|
|
1301
|
+
return list.map((entry) => {
|
|
1302
|
+
let models = [];
|
|
1303
|
+
try {
|
|
1304
|
+
const declared = typeof entry?.getModels === "function" ? entry.getModels() : entry?.models || [];
|
|
1305
|
+
models = (Array.isArray(declared) ? declared : []).map((model) => ({
|
|
1306
|
+
id: model?.id || "",
|
|
1307
|
+
baseURL: model?.baseUrl || model?.baseURL || "",
|
|
1308
|
+
})).filter((model) => model.id && model.baseURL);
|
|
1309
|
+
} catch {}
|
|
1310
|
+
return {
|
|
1311
|
+
id: entry?.id || "",
|
|
1312
|
+
baseURL: entry?.baseUrl || entry?.baseURL || "",
|
|
1313
|
+
models,
|
|
1314
|
+
};
|
|
1315
|
+
}).filter((entry) => entry.id && (entry.baseURL || entry.models.length));
|
|
1175
1316
|
} catch {}
|
|
1176
1317
|
}
|
|
1177
1318
|
return [];
|
|
@@ -1201,6 +1342,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1201
1342
|
}
|
|
1202
1343
|
|
|
1203
1344
|
captureOne(session, event, access = {}) {
|
|
1345
|
+
if (this.standDown?.()) return null;
|
|
1204
1346
|
const id = event?.data?.message?.toolCallId;
|
|
1205
1347
|
if (!id) return null;
|
|
1206
1348
|
const key = session?.id || DEFAULT_KEY;
|
|
@@ -1247,7 +1389,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1247
1389
|
}
|
|
1248
1390
|
const message = error instanceof Error ? error.message : String(error);
|
|
1249
1391
|
this.ctx?.logger?.warn?.(`spine: raw backup failed: ${message}`);
|
|
1250
|
-
const record = { ok: false, reason: "unwritable", toolCallId: id, retryable: error
|
|
1392
|
+
const record = { ok: false, reason: "unwritable", toolCallId: id, retryable: captureErrorRetryable(error) };
|
|
1251
1393
|
this.rawSeen.set(mapKey, record);
|
|
1252
1394
|
return record;
|
|
1253
1395
|
}
|
|
@@ -1307,6 +1449,15 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1307
1449
|
}
|
|
1308
1450
|
|
|
1309
1451
|
captureNewBatch(session, events, settings = readSettings(this.liveConfig), limits = {}) {
|
|
1452
|
+
if (this.standDown?.()) {
|
|
1453
|
+
return {
|
|
1454
|
+
seen: new Set(),
|
|
1455
|
+
progressed: false,
|
|
1456
|
+
retryable: false,
|
|
1457
|
+
done: true,
|
|
1458
|
+
nextIndex: Number.isInteger(limits.startIndex) ? limits.startIndex : 0,
|
|
1459
|
+
};
|
|
1460
|
+
}
|
|
1310
1461
|
if (!Array.isArray(events)) events = [];
|
|
1311
1462
|
const key = session?.id || DEFAULT_KEY;
|
|
1312
1463
|
const cursor = this.captureCursorStart(session, events, settings);
|
|
@@ -1371,6 +1522,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1371
1522
|
}
|
|
1372
1523
|
|
|
1373
1524
|
captureNew(session, events, settings = readSettings(this.liveConfig)) {
|
|
1525
|
+
if (this.standDown?.()) return new Set();
|
|
1374
1526
|
if (!Array.isArray(events)) events = [];
|
|
1375
1527
|
const key = session?.id || DEFAULT_KEY;
|
|
1376
1528
|
const cursor = this.captureCursorStart(session, events, settings);
|
|
@@ -1426,6 +1578,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1426
1578
|
}
|
|
1427
1579
|
|
|
1428
1580
|
writeCaptureCursor(key, cursor, access = {}) {
|
|
1581
|
+
if (this.standDown?.()) return;
|
|
1429
1582
|
if (this.persistCapture !== true || !Number.isSafeInteger(cursor) || cursor < 0) return;
|
|
1430
1583
|
let tmp;
|
|
1431
1584
|
let fd;
|
|
@@ -1452,6 +1605,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1452
1605
|
}
|
|
1453
1606
|
|
|
1454
1607
|
recoverVisibleRaw(session, access = {}) {
|
|
1608
|
+
if (this.standDown?.()) return;
|
|
1455
1609
|
const dir = sessionDir(shadowDirectory(), session?.id || DEFAULT_KEY);
|
|
1456
1610
|
const ids = [...new Set(this.surfaceToolIds(session))];
|
|
1457
1611
|
if (ids.length && existsSync(dir)) ensureShadowDir(dir, access);
|
|
@@ -1486,6 +1640,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1486
1640
|
}
|
|
1487
1641
|
|
|
1488
1642
|
recoverVisibleRawBatch(session, access = {}, startIndex = 0, limit = RECOVER_RAW_BATCH, visibleIds = null) {
|
|
1643
|
+
if (this.standDown?.()) return { nextIndex: startIndex, done: true, progressed: false };
|
|
1489
1644
|
const ids = visibleIds || [...new Set(this.surfaceToolIds(session))];
|
|
1490
1645
|
const dir = sessionDir(shadowDirectory(), session?.id || DEFAULT_KEY);
|
|
1491
1646
|
if (ids.length && startIndex === 0 && existsSync(dir)) {
|
|
@@ -1536,6 +1691,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1536
1691
|
}
|
|
1537
1692
|
|
|
1538
1693
|
captureSession(session) {
|
|
1694
|
+
if (this.standDown?.()) return new Set();
|
|
1539
1695
|
const key = session?.id || DEFAULT_KEY;
|
|
1540
1696
|
const events = session?.snapshotEvents?.() || [];
|
|
1541
1697
|
const fingerprint = this.captureRecoveryFingerprint(session, events);
|
|
@@ -1592,6 +1748,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1592
1748
|
}
|
|
1593
1749
|
|
|
1594
1750
|
ensurePruneGuard() {
|
|
1751
|
+
if (this.standDown?.()) return;
|
|
1595
1752
|
if (this.pruneGuarded) return;
|
|
1596
1753
|
let pruner;
|
|
1597
1754
|
try {
|
|
@@ -1609,6 +1766,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1609
1766
|
function guarded(session) {
|
|
1610
1767
|
const engine = pruner.__spineEngine;
|
|
1611
1768
|
try {
|
|
1769
|
+
if (engine.standDown?.()) return;
|
|
1612
1770
|
engine.captureSession(session);
|
|
1613
1771
|
return runPruneAfterCapture(engine.surfaceToolIds(session), engine.rawVerifiedSet(session), () => original(session));
|
|
1614
1772
|
} catch (error) {
|
|
@@ -1651,6 +1809,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1651
1809
|
access = {},
|
|
1652
1810
|
rotationIO = {},
|
|
1653
1811
|
) {
|
|
1812
|
+
if (this.standDown?.()) return { ok: false, reason: "stand-down" };
|
|
1654
1813
|
return saveShadowImpl.call(
|
|
1655
1814
|
this,
|
|
1656
1815
|
messages,
|
|
@@ -1672,6 +1831,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1672
1831
|
access = {},
|
|
1673
1832
|
rotationIO = {},
|
|
1674
1833
|
) {
|
|
1834
|
+
if (this.standDown?.()) return { ok: false, reason: "stand-down" };
|
|
1675
1835
|
if (!access.runner && (access.platform ?? process.platform) === "win32" && shadowAccessFused()) {
|
|
1676
1836
|
const error = new Error("spine: shadow permission already failed; backup refused");
|
|
1677
1837
|
error.code = "ESHADOWACCESS";
|
|
@@ -1767,7 +1927,7 @@ export default class SpineCompactionEngine extends BasicCompactionEngine {
|
|
|
1767
1927
|
role: message?.role ?? message?.message?.role,
|
|
1768
1928
|
text: messageText(message),
|
|
1769
1929
|
})));
|
|
1770
|
-
const facts = [...
|
|
1930
|
+
const facts = [...(trunk.hardFacts || []), ...(trunk.missingFacts || [])];
|
|
1771
1931
|
const slim = buildSendLess({ taskLine, uncovered: trunk.uncovered, facts });
|
|
1772
1932
|
return this.summarizeOfficial({
|
|
1773
1933
|
...input,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-live-compact",
|
|
3
|
-
"version": "1.1.
|
|
3
|
+
"version": "1.1.6",
|
|
4
4
|
"description": "In-place newest-turn compaction for DeepSeek Harness. Compress while the task runs and keep the prefix.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "MerkyorLynn <268568828+MerkyorLynn@users.noreply.github.com>",
|
|
@@ -43,7 +43,7 @@
|
|
|
43
43
|
"compaction"
|
|
44
44
|
],
|
|
45
45
|
"scripts": {
|
|
46
|
-
"test:gate": "node test/p0-gate.mjs && node test/p0-blockers.mjs && node test/excerpt.mjs && node test/shadow-worker.mjs && node test/shadow-worker-pool.mjs && node test/shadow-worker-fuse.mjs && node --test test/failed-diagnostics.test.mjs test/public-exports.test.mjs test/cooldown.test.mjs test/gate-config.test.mjs test/drain-stopline.mjs test/scan-once.mjs test/issue-19.mjs test/issue-20.mjs test/issue-34.mjs test/windows-acl.test.mjs test/issue-27.mjs test/issue-29.mjs test/issue-29-notice.mjs test/client-settings.test.mjs test/settings-integration.test.mjs test/runtime-lifecycle.mjs test/shadow-lifecycle.mjs test/finalize-over-32000.mjs",
|
|
46
|
+
"test:gate": "node test/p0-gate.mjs && node test/p0-blockers.mjs && node test/excerpt.mjs && node test/shadow-worker.mjs && node test/shadow-worker-pool.mjs && node test/shadow-worker-fuse.mjs && node --test test/failed-diagnostics.test.mjs test/public-exports.test.mjs test/cooldown.test.mjs test/gate-config.test.mjs test/stand-down.test.mjs test/drain-stopline.mjs test/scan-once.mjs test/issue-19.mjs test/issue-20.mjs test/issue-34.mjs test/windows-acl.test.mjs test/issue-27.mjs test/issue-29.mjs test/issue-29-notice.mjs test/client-settings.test.mjs test/settings-integration.test.mjs test/runtime-lifecycle.mjs test/shadow-lifecycle.mjs test/finalize-over-32000.mjs test/p1-fable-review.mjs",
|
|
47
47
|
"test:compat41": "node test/compat-cordis41.mjs",
|
|
48
48
|
"test:live-gate": "python3 -B -m unittest discover -s test/live-fill -p test_*.py"
|
|
49
49
|
},
|
package/plan.js
CHANGED
|
@@ -734,11 +734,18 @@ export function providerURLFromPatch(text, provider) {
|
|
|
734
734
|
|
|
735
735
|
// 内置目录只在这家没有自己的地址时使用。调用方传入 {id, baseURL},这里不读包。
|
|
736
736
|
// Use a built-in catalog URL only when this provider has no URL of its own.
|
|
737
|
-
export function catalogProviderURL(provider, catalog) {
|
|
737
|
+
export function catalogProviderURL(provider, catalog, model = "") {
|
|
738
738
|
const name = String(provider || "");
|
|
739
739
|
if (!name || !catalog) return "";
|
|
740
740
|
const list = Array.isArray(catalog) ? catalog : [];
|
|
741
741
|
const match = list.find((entry) => entry?.id === name || entry?.provider === name);
|
|
742
|
+
const modelName = String(model || "");
|
|
743
|
+
if (match && modelName) {
|
|
744
|
+
const models = Array.isArray(match.models) ? match.models : [];
|
|
745
|
+
const modelMatch = models.find((entry) => entry?.id === modelName || entry?.model === modelName);
|
|
746
|
+
const modelURL = modelMatch?.baseURL || modelMatch?.baseUrl || "";
|
|
747
|
+
if (modelURL) return String(modelURL);
|
|
748
|
+
}
|
|
742
749
|
return String(match?.baseURL || match?.baseUrl || "");
|
|
743
750
|
}
|
|
744
751
|
|
|
@@ -748,6 +755,7 @@ export function catalogProviderURL(provider, catalog) {
|
|
|
748
755
|
// Official DeepSeek and the built-in catalog apply only when the profile has no URL. Empty stays unresolved.
|
|
749
756
|
export function resolveProviderBaseURL({
|
|
750
757
|
provider = "",
|
|
758
|
+
model = "",
|
|
751
759
|
headerURL = "",
|
|
752
760
|
listed = [],
|
|
753
761
|
layers = [],
|
|
@@ -775,7 +783,7 @@ export function resolveProviderBaseURL({
|
|
|
775
783
|
if (provider === "deepseek-account" && launch) return launch;
|
|
776
784
|
const fallback = fallbackProviderURL(provider);
|
|
777
785
|
if (fallback) return fallback;
|
|
778
|
-
return catalogProviderURL(provider, catalog);
|
|
786
|
+
return catalogProviderURL(provider, catalog, model);
|
|
779
787
|
}
|
|
780
788
|
|
|
781
789
|
export function resolveEngine({ provider = "", baseURL = "", env = process.env } = {}) {
|
|
@@ -1185,6 +1193,17 @@ function hasProtectedToolFact(line) {
|
|
|
1185
1193
|
return /必须|不要|禁止|口令|方案|[a-z]{2,}-[a-z0-9]+-\d+|\bpinned\s+to\b/i.test(line);
|
|
1186
1194
|
}
|
|
1187
1195
|
|
|
1196
|
+
// A model/version/region token by itself is ordinary prose. Only retain an
|
|
1197
|
+
// assistant line as an unresolved decision when the line contains a decision
|
|
1198
|
+
// verb or the explicit Chinese plan form; this avoids treating `us-east-1`
|
|
1199
|
+
// and `claude-3-5` as commands that block the cut.
|
|
1200
|
+
function unresolvedAssistantDecision(line) {
|
|
1201
|
+
const text = String(line || "");
|
|
1202
|
+
if (/方案\s*[A-Za-z0-9]/.test(text)) return true;
|
|
1203
|
+
if (/(?:决定|采用|选用|选择|固定为|改用|切换到|定为|使用|确定)/.test(text)) return true;
|
|
1204
|
+
return /\b(?:pinned\s+to|decid(?:e|ed|es)|adopt(?:ed|s)?|selected|choose|chosen|will\s+use|must\s+use)\b/i.test(text);
|
|
1205
|
+
}
|
|
1206
|
+
|
|
1188
1207
|
function normalizeSourceLine(role, text) {
|
|
1189
1208
|
return (role === "tool" ? text.replace(/^[ \t]*\d+:[ \t]*/, "") : text).trim();
|
|
1190
1209
|
}
|
|
@@ -1316,7 +1335,7 @@ export function buildCard(
|
|
|
1316
1335
|
if (source.role !== "assistant") continue;
|
|
1317
1336
|
source.text.split(/\r?\n/).forEach((line) => {
|
|
1318
1337
|
if (liveLines.has(line.trim())) return;
|
|
1319
|
-
if (
|
|
1338
|
+
if (unresolvedAssistantDecision(line)) {
|
|
1320
1339
|
coverage.ok = false;
|
|
1321
1340
|
coverage.missing.push({ sourceId: source.id, text: line, reason: "assistant-decision-not-retained" });
|
|
1322
1341
|
}
|
|
@@ -1373,10 +1392,15 @@ export function buildCard(
|
|
|
1373
1392
|
failureDiagnosticsComplete: incompleteFailures.length === 0,
|
|
1374
1393
|
scope: TRUNK_SCOPE,
|
|
1375
1394
|
hardLines: hard.map((item) => item.line),
|
|
1395
|
+
hardFacts: hard.map((item) => ({ role: item.role, line: item.line })),
|
|
1376
1396
|
hardCount: hard.length,
|
|
1377
1397
|
omittedHard,
|
|
1378
1398
|
deferred,
|
|
1379
1399
|
missing,
|
|
1400
|
+
missingFacts: coverage.missing.map((item) => ({
|
|
1401
|
+
role: sourceById.get(item.sourceId)?.role || "unknown",
|
|
1402
|
+
line: item.text,
|
|
1403
|
+
})),
|
|
1380
1404
|
omittedOrdinaryToolLines: omittedOrdinaryToolLines,
|
|
1381
1405
|
uncovered: split.uncovered,
|
|
1382
1406
|
keptChars: used,
|
package/raw.js
CHANGED
|
@@ -20,6 +20,10 @@ import {
|
|
|
20
20
|
|
|
21
21
|
export const PRUNE_MARKER = "[... tool result middle pruned ...]";
|
|
22
22
|
|
|
23
|
+
function transientCaptureError(error) {
|
|
24
|
+
return ["ESHADOWBUSY", "EAGAIN", "EBUSY", "EINTR", "EIO"].includes(error?.code);
|
|
25
|
+
}
|
|
26
|
+
|
|
23
27
|
export function contentHash(text) {
|
|
24
28
|
const hash = createHash("sha256");
|
|
25
29
|
hash.update(typeof text === "string" ? Buffer.from(text, "utf8") : text);
|
|
@@ -122,7 +126,7 @@ export function captureToolResult(options) {
|
|
|
122
126
|
ensureShadowDir(dir, access);
|
|
123
127
|
} catch (error) {
|
|
124
128
|
if (error?.code === "ESHADOWACCESS" || error?.code === "ESHADOWCLEANUP") throw error;
|
|
125
|
-
return { ok: false, reason: "unwritable", toolCallId };
|
|
129
|
+
return { ok: false, reason: "unwritable", toolCallId, retryable: transientCaptureError(error) };
|
|
126
130
|
}
|
|
127
131
|
let entered = false;
|
|
128
132
|
try {
|
|
@@ -131,7 +135,7 @@ export function captureToolResult(options) {
|
|
|
131
135
|
return captureToolResultLocked({ ...options, root });
|
|
132
136
|
});
|
|
133
137
|
} catch (error) {
|
|
134
|
-
if (!entered || error
|
|
138
|
+
if (!entered || transientCaptureError(error)) return { ok: false, reason: "unwritable", toolCallId, retryable: transientCaptureError(error) };
|
|
135
139
|
throw error; // In particular, preserve exceptions from failed raw cleanup.
|
|
136
140
|
}
|
|
137
141
|
}
|
|
@@ -153,6 +157,8 @@ function captureToolResultLocked({
|
|
|
153
157
|
const body = String(text ?? "");
|
|
154
158
|
const file = join(dir, rawName(toolCallId));
|
|
155
159
|
const metaPath = `${file}.meta.json`;
|
|
160
|
+
const bytes = Buffer.from(body, "utf8");
|
|
161
|
+
const hash = contentHash(bytes);
|
|
156
162
|
|
|
157
163
|
if (!existsSync(file) && body.includes(PRUNE_MARKER)) {
|
|
158
164
|
return { ok: false, reason: "already-pruned", toolCallId };
|
|
@@ -163,10 +169,24 @@ function captureToolResultLocked({
|
|
|
163
169
|
let createdRaw = false;
|
|
164
170
|
|
|
165
171
|
try {
|
|
166
|
-
if (existsSync(file))
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
172
|
+
if (existsSync(file)) {
|
|
173
|
+
const existing = readProtectedExisting();
|
|
174
|
+
if (!existing.ok) {
|
|
175
|
+
// A raw without its meta marker is an incomplete publication. When
|
|
176
|
+
// the current event still has the original body, discard the orphan
|
|
177
|
+
// and rebuild the pair; a malformed/valid meta file remains fail-closed.
|
|
178
|
+
if (existing.reason === "missing-meta" && !body.includes(PRUNE_MARKER)) {
|
|
179
|
+
discardShadowFiles([file, metaPath], new Error("incomplete raw/meta pair"));
|
|
180
|
+
} else {
|
|
181
|
+
return existing;
|
|
182
|
+
}
|
|
183
|
+
}
|
|
184
|
+
if (existing.ok && body.includes(PRUNE_MARKER)) return existing;
|
|
185
|
+
if (existing.ok && (existing.sha256 !== hash || existing.bytes !== bytes.length)) {
|
|
186
|
+
return { ok: false, reason: "duplicate-tool-call-id", toolCallId, retryable: false };
|
|
187
|
+
}
|
|
188
|
+
if (existing.ok) return existing;
|
|
189
|
+
}
|
|
170
190
|
|
|
171
191
|
writeShadowFile(file, bytes, access, { exclusive: true, root });
|
|
172
192
|
createdRaw = true;
|
|
@@ -208,7 +228,7 @@ function captureToolResultLocked({
|
|
|
208
228
|
return readProtectedExisting();
|
|
209
229
|
}
|
|
210
230
|
|
|
211
|
-
return { ok: false, reason: "unwritable", toolCallId, retryable: error
|
|
231
|
+
return { ok: false, reason: "unwritable", toolCallId, retryable: transientCaptureError(error) };
|
|
212
232
|
}
|
|
213
233
|
}
|
|
214
234
|
|
package/shadow.js
CHANGED
|
@@ -972,7 +972,24 @@ export function ensureShadowDir(dir, access = {}) {
|
|
|
972
972
|
return dir;
|
|
973
973
|
}
|
|
974
974
|
|
|
975
|
-
export function tightenCreatedShadow(file, fd, access, { remember = true } = {}) {
|
|
975
|
+
export function tightenCreatedShadow(file, fd, access, { remember = true, skipRunner = false } = {}) {
|
|
976
|
+
if (skipRunner) {
|
|
977
|
+
try {
|
|
978
|
+
const entry = accessEntry(file);
|
|
979
|
+
const opened = fstatSync(fd, { bigint: true });
|
|
980
|
+
if (!entry.identity || entry.identity !== accessIdentity(opened) || opened.nlink !== 1n) {
|
|
981
|
+
throw new Error("spine: shadow published file identity changed");
|
|
982
|
+
}
|
|
983
|
+
if (remember) lockedFiles.set(entry.canonical, entry);
|
|
984
|
+
return;
|
|
985
|
+
} catch (cause) {
|
|
986
|
+
tripFuse();
|
|
987
|
+
const error = new Error("spine: shadow permission action failed; backup refused", { cause });
|
|
988
|
+
error.code = "ESHADOWACCESS";
|
|
989
|
+
error.step = "file";
|
|
990
|
+
throw error;
|
|
991
|
+
}
|
|
992
|
+
}
|
|
976
993
|
if ((access.platform ?? process.platform) === "win32" && !access.runner && !shadowAccessFused()) {
|
|
977
994
|
try {
|
|
978
995
|
if (shadowDirLocked(dirname(file))) {
|
|
@@ -1043,12 +1060,13 @@ function writeShadowFileLocked(file, body, access, { exclusive, write, unlink, a
|
|
|
1043
1060
|
const dir = dirname(file);
|
|
1044
1061
|
const parentIdentity = rememberShadowParent(dir);
|
|
1045
1062
|
|
|
1046
|
-
|
|
1047
|
-
|
|
1048
|
-
|
|
1049
|
-
|
|
1050
|
-
|
|
1051
|
-
|
|
1063
|
+
// Always stage the body in a private temporary file. Publishing the final
|
|
1064
|
+
// name only after the bytes are flushed prevents readers from observing a
|
|
1065
|
+
// half-written raw or meta file after a process interruption.
|
|
1066
|
+
const target = join(
|
|
1067
|
+
dir,
|
|
1068
|
+
`.${basename(file)}-${randomBytes(8).toString("hex")}.tmp`,
|
|
1069
|
+
);
|
|
1052
1070
|
|
|
1053
1071
|
let fd = null;
|
|
1054
1072
|
let created = false;
|
|
@@ -1058,25 +1076,34 @@ function writeShadowFileLocked(file, body, access, { exclusive, write, unlink, a
|
|
|
1058
1076
|
fd = openSync(target, "wx", 0o600);
|
|
1059
1077
|
created = true;
|
|
1060
1078
|
|
|
1061
|
-
if (!exclusive) {
|
|
1062
|
-
renameSync(target, file);
|
|
1063
|
-
protectedPath = file;
|
|
1064
|
-
}
|
|
1065
|
-
|
|
1066
|
-
// Keep the caller's spelling for the permission cache/runner.
|
|
1067
|
-
// The opened descriptor and cleanup path remain in the canonical root.
|
|
1068
|
-
tightenCreatedShadow(accessPath, fd, access);
|
|
1069
1079
|
verifyRotationScope();
|
|
1070
1080
|
if (parentIdentity && directoryIdentity(dir) !== parentIdentity) {
|
|
1071
1081
|
retireRotationScope();
|
|
1072
1082
|
throw coordinationError(new Error("shadow session directory identity changed"));
|
|
1073
1083
|
}
|
|
1084
|
+
// Confirm ACL/mode before any secret bytes are written to the staged file.
|
|
1085
|
+
tightenCreatedShadow(target, fd, access);
|
|
1074
1086
|
// Full replacement bytes are a conservative charge, even for overwrites,
|
|
1075
1087
|
// partial writes and failed cleanup. Metadata/owner/stats are not capacity.
|
|
1076
1088
|
if (shadowUsage && isPluginShadowName(basename(file))) {
|
|
1077
1089
|
shadowUsage.bytes += typeof body === "string" ? Buffer.byteLength(body, "utf8") : body.byteLength;
|
|
1078
1090
|
}
|
|
1079
1091
|
write(fd, body);
|
|
1092
|
+
fsyncSync(fd);
|
|
1093
|
+
|
|
1094
|
+
// The rotation/root lock provides the no-overwrite boundary for exclusive
|
|
1095
|
+
// raw files. Refuse a file that appeared after admission; never replace it.
|
|
1096
|
+
if (exclusive && existsSync(file)) {
|
|
1097
|
+
const error = new Error("shadow file already exists");
|
|
1098
|
+
error.code = "EEXIST";
|
|
1099
|
+
throw error;
|
|
1100
|
+
}
|
|
1101
|
+
renameSync(target, file);
|
|
1102
|
+
protectedPath = file;
|
|
1103
|
+
|
|
1104
|
+
// Keep the caller's spelling for the permission cache/runner. The opened
|
|
1105
|
+
// descriptor and the final path now refer to the same published object.
|
|
1106
|
+
tightenCreatedShadow(file, fd, access, { skipRunner: true });
|
|
1080
1107
|
|
|
1081
1108
|
const opened = fd;
|
|
1082
1109
|
fd = null;
|
package/thunder.js
CHANGED
|
@@ -34,7 +34,14 @@ function estimateTokens(text) {
|
|
|
34
34
|
}
|
|
35
35
|
|
|
36
36
|
export function buildSendLess({ taskLine, uncovered, facts, proseTokens = PROSE_TOKEN_CAP }) {
|
|
37
|
-
const kept = (facts || []).filter(Boolean)
|
|
37
|
+
const kept = (facts || []).filter(Boolean).map((fact) => {
|
|
38
|
+
if (typeof fact === "object" && fact !== null) {
|
|
39
|
+
const role = String(fact.role || "unknown");
|
|
40
|
+
const line = String(fact.line || "");
|
|
41
|
+
return line ? `[${role}] ${line}` : "";
|
|
42
|
+
}
|
|
43
|
+
return `[unknown] ${String(fact)}`;
|
|
44
|
+
}).filter(Boolean);
|
|
38
45
|
let prose = uncovered || "";
|
|
39
46
|
if (estimateTokens(prose) > proseTokens) {
|
|
40
47
|
let low = 0;
|