@logictan/dsh-verbatim-compaction 0.2.0 → 0.3.0

package/README.md

@@ -1,6 +1,8 @@
 # @logictan/dsh-verbatim-compaction
 
-把上下文压缩的**摘要**从「让模型重写一遍」换成「正文逐字抄进 checkpoint，再用一次模型调用从工具轨迹里补出结论」：用户与助手的正文逐字保留、推理块从不读取；每轮另调一次会话自身的模型，输出「做了什么 / 得到什么结论 / 哪些结论被推翻」。
+把上下文压缩的**摘要**从「让模型重写整段历史」换成「用户原话逐字保留，助手部分只留每轮提取的结论」：用户的话一个字不改，助手正文由一次模型调用换成「做了什么 / 得到什么结论 / 哪些结论被推翻」三节，推理块从不读取。
+
+用户原话与助手正文待遇不同，理由是二者可重建性不同：助手回复能从它的工具调用和上下文推回来，用户的纠正、警告、约束推不回来。宿主原生压缩会改写这些原话，那正是本插件要消除的损失。
 
 ## 它解决什么
 
@@ -16,21 +18,29 @@
 
 ```
 ## user (turn 37)
-（用户原话，逐字）
+（用户原话，逐字，永不改写）
 
 ### assistant (turn 37)
-（助手正文，逐字）
+## Did
+- 实测 `auto_vacuum=FULL` 删除后自动缩容：19.61 MB → 0.01 MB
+## Concluded
+- 选 FULL 而非 INCREMENTAL：回收发生在宿主自己的 DELETE 里，不受 busy_timeout=0 影响
+## Overturned
+- 先前认为必须显式执行 VACUUM 才能回收空间
   tools: bash×280 ssh_exec×44 web_search×6 (408830 bytes)
   errors: 1: Error: 命令超过 600 秒未结束，已断开连接
   wrote: /Volumes/LogicExt/Git/BookReader/src/video.js
 ```
 
-- 正文逐字，不重写、不改写。
+- **用户原话逐字**，不重写、不改写、不摘要。
+- **助手正文由提取块替代**，不并列保留——并列就是同一份内容写两遍。
 - 工具结果只留「工具名 × 次数 + 字节数」，内容丢弃。
-- 错误只留首行（100 字符），足以知道「这里失败过、失败原因是什么」。
-- `wrote` 记录本轮写过的路径，重建「改了哪些文件」。
+- 错误只留首行（100 字符）。
+- `wrote` 记录本轮写过的路径。
 - 推理块从不进入摘要，也不读取。
 
+为什么助手正文不再逐字保留：实测 8 个会话、4400 条助手正文、748743 字符，其中 58.2% 是有实质结论的长文、35.8% 是带数据（数字/代码/引用）的短句、6.1% 是纯过程叙述（「现在我来看…」）。而字面重复几乎为零（段落级完全相同 0.1%）。也就是说，重复不在字面，而在**助手正文与提取块说了同一件事**；两者并列保留就是把这 58.2% 写两遍，还通过预算把真正的历史挤出去。
+
 ## 摘要源是日志，不是被阴影的区间
 
 上一份 checkpoint 每次都被一起阴影掉，所以「只摘要被阴影区间」会把上一份摘要承载的全部历史再丢一次。本插件扫描会话日志里 `surfaceOp === 'append'` 的事件，减去本次压缩后仍留在 surface 上的尾部，重建整段历史——不需要任何累积状态，摘要源集合逐次单调增长（122 次压缩实测 0 次非单调）。
@@ -39,9 +49,9 @@
 
 ## 工具轨迹补出的三节
 
-逐字抄正文救不了「正文为空」的轮次。实测一个 13878 事件的会话：1973 条 assistant 消息里只有 317 条带正文，其中一轮跑了 342 次工具调用却一个字没写，逐字摘要把它压成一行 `tools: bash×281`——活干了，记录没了。
+只抄正文救不了「正文为空」的轮次。实测一个 13878 事件的会话：1973 条 assistant 消息里只有 317 条带正文，其中一轮跑了 342 次工具调用却一个字没写，纯逐字摘要把它压成一行 `tools: bash×281`——活干了，记录没了。
 
-所以每轮另发一次模型调用，材料是三段：用户请求（逐字）、助手正文（逐字）、工具轨迹（每条调用的 `description`，失败的调用另记一行首行错误）。模型输出固定的三节：`## Did` / `## Concluded` / `## Overturned`。
+所以提取的材料是三段：用户请求（逐字）、助手正文（逐字）、工具轨迹（每条调用的 `description`，失败的调用另记一行首行错误）。模型输出固定的三节：`## Did` / `## Concluded` / `## Overturned`。
 
 模型用的是会话自己的路由（`session.requestHeader()?.config`），和宿主原生压缩同一个取法，不需要单独配置凭据。
 
@@ -49,13 +59,25 @@
 
 - **失败信号只认 `data.isError`**。曾用 `/error/i` 匹配序列化文本，347 条成功结果全部命中（工具输出里常有 "error" 字样），模型据此报告「整轮失败」。测试里有一条专门钉住这个坑。
 - **工具调用从 `assistant/message` 的 block 里取**。日志里没有独立的 `tool/call` 事件类型，按它过滤会一条都取不到。
-- **提取失败就退回纯逐字摘要**。模型不可达、超时、输出为空都返回 null；压缩本身不能因此失败，否则会话死于上下文溢出，比丢细节更糟。
+- **提取失败就退回逐字正文**。模型不可达、超时、输出为空时，该轮保留原来的助手正文；压缩本身不能因此失败，否则会话死于上下文溢出，比丢细节更糟。降级是逐轮的：一批里某一轮没答，只有那一轮退回正文。
 
-代价是每轮一次模型调用（实测 13 轮合计约 74649 字符材料）。提取结果与逐字摘要共用同一个预算，超预算时只保留逐字摘要。
+提取按批调用：每 8 轮一次，一轮输出一个 `=== TURN n ===` 块。早期实现是每轮一次调用，122 轮就是 131 次往返，单轮 400 token 的输出上限乘以 131 就可能超过整个摘要上限。批处理把往返压到约 17 次，并共享 prompt 前缀以复用 provider 的 KV cache。
+
+用户原话在布局上**先占额度**，提取块不得挤占。实测那个 122 轮会话：区间内真人原话共 469 字符（117 token），而每轮一块 420 字符的提取共 55020 字符——提取是正文的 117 倍，任何让二者竞争的布局都会把正文挤掉，而正文是「请用pixi管理开发环境」这类永不过期的约束，提取描述的却是已经做完的工作。
 
 ## 为什么不会触发 shrink 守卫
 
-基类的守卫比较「framed 摘要」与「被替换内容的 route 价」，摘要不小于原内容就整次压缩失败。本插件把预算直接算成 `min(196752, shadowedPrice − 86 − 512)`：上界取守卫自己的那个数，因此这条失败路径按构造不可能发生，而不是靠实测侥幸。实测 8 个会话 0 次触发，最小余量 174387。
+基类的守卫比较「framed 摘要」与「被替换内容的 route 价」，摘要不小于原内容就整次压缩失败。本插件的预算取 `min(DIGEST_TOKEN_CEILING, shadowedPrice − 86 − 512)`，上限 20000 token 远小于任何实测区间价，因此这条失败路径按构造不可能发生。
+
+上限取**绝对值**而非「区间价的比例」。比例会让摘要随历史增长：同一会话实测早期区间 2642 字符、后期 51019 字符、触发事故那段 348539 字符——因为区间越大就允许摘要越大，而接近阈值的摘要立刻再触发压缩。固定上限让摘要大小与历史长度无关，宿主原生摘要也是这个形状（实测各区间稳定在 14009–22208 字符）。
+
+## 为什么会收敛（一条真实事故）
+
+只满足守卫**不等于**收敛。守卫只要求摘要比被替换的内容小，一个占到被替换量 99.3% 的摘要照样通过守卫，而 surface 几乎没变。
+
+会话 `小鹏汽车 App 改造评估` 上真实发生过：一次压缩 shadowed 87733 token，产出摘要 priced 87135——只回收 598 token（0.7%）。压缩后 surface 实测 **256704** token，阈值 **256464**，刚好卡在阈值上方，于是下一步立刻又压；此时 surface 上除 checkpoint 外没别的东西，第二次只 shadowed **1 个 seq**（就是刚写的那条 checkpoint），把同样的 34.8 万字符原样重抄一遍（346555 vs 348539），第三次直接 abort。
+
+预算改成被替换量的一半后，同一份日志实测：257696 → **188120** token（0.73× 阈值），一轮就落到阈值以下。只压一条 checkpoint 时摘要也减半（279048 → 139865 字符），因此是几何衰减而非固定点。
 
 ## 安装
 
@@ -107,7 +129,8 @@ pnpm add @logictan/dsh-verbatim-compaction
 
 ## 已知边界
 
-- **正文超预算会被截断。** 预算是硬的，两级依次触发：**先** user 正文超过预算一半时从**最新**往前算、最旧的先丢（用户刚说的话是下一轮唯一依赖的东西，丢它等于丢任务）；**仍不够**才把助手正文按轮均分截断，每轮下限 800 字符。8 个实测会话里两级都没跑过（最大摘要 44542 token，预算 196752），所以这是守卫而不是常态。
+- **每轮的正文/提取块会被均匀削薄，用户原话不会。** 用户原话先占额度（`USER_BUDGET_SHARE` = 0.9），剩余空间再分给各轮的提取块或正文，超出时按轮等比截断并标出省略字符数。只有在每轮都已削到零仍放不下时才整轮丢弃，且会写明丢了几轮——正常长会话不会走到这一步（122 轮实测 0 次）。
+- **提取块可能被截断。** 122 轮、每轮 420 字符提取的极端情形会触到上限，块尾被切并标出字符数。真人原话不受影响。
 - **`turn/start` 缺失时轮次编号退化为 `turn ?`。** 不会丢正文，只是编号不可读。
 - **注入内容不进摘要。** 判据是 `source.kind !== 'user'`——注入上下文在一个真实会话里占 user 消息字符的 97%，放进摘要会把人的原话淹掉。
 - **不改善首轮之外的工具参数细节。** `bash` 命令本身（占工具参数 84.2%）不保留；需要「当时跑了什么命令」的场景不在本插件的覆盖范围内。

package/lib/index.js

@@ -34,22 +34,41 @@ import { BasicCompactionEngine } from '@deepseek-ai/dsh-compaction-basic';
 /** Cordis companion plugin name. Must equal this plugin's `cordis.patch.yml` row id. */
 export const name = 'verbatim-compaction';
 /**
- * Ceiling for one digest, in the meter's fixed 4-chars-per-token heuristic.
+ * Absolute ceiling for one digest, in the meter's fixed 4-chars-per-token
+ * heuristic.
  *
- * The base class's own summary budget is `thresholdTokens - retainTokens -
- * 8192`, which is 196752 for the route this was measured on. The digest is also
- * bounded by the shrink guard itself, which is tighter in practice; see
- * `summarize`.
+ * A share of the shadowed span is the wrong shape: it makes the digest grow
+ * with history. Measured on one real session, the share-based budget produced
+ * 2642 chars on an early span, 51019 on a later one, and 348539 on the span that
+ * triggered the compaction loop — because a bigger span licensed a bigger
+ * digest, and a digest near the threshold re-triggers immediately.
+ *
+ * A fixed ceiling makes the digest size independent of how long the session has
+ * run, which is what the host's own summarizer does (its summaries measured
+ * 14009-22208 chars across spans of 108829-173514 tokens).
+ *
+ * 20000 tokens holds roughly 100 turns of user prose plus one extracted block
+ * per turn, and leaves the checkpoint far below any measured threshold.
  */
-const DIGEST_BUDGET_TOKENS = 196752;
+export const DIGEST_TOKEN_CEILING = 20000;
 /** Framing overhead `frameSummary` adds: preamble plus both tags, 341 chars = 86 tokens. */
 const FRAME_TOKENS = 86;
 /** Safety margin below the shrink guard, in tokens. */
 const SHRINK_MARGIN_TOKENS = 512;
 /** Per-turn floor when assistant prose has to be truncated, in characters. */
 const MIN_TURN_CHARS = 800;
-/** Share of the budget user prose may occupy before oldest-first truncation. */
-const USER_BUDGET_SHARE = 0.5;
+/**
+ * Share of the budget user prose may occupy before oldest-first truncation.
+ *
+ * Measured on a 122-turn session, every human message in the span totalled 3219
+ * characters — 4% of the 80000-character ceiling — because 99.5% of user-message
+ * characters are injected context, which the digest excludes. So user prose
+ * should never be truncated for room; the share exists only to stop one pasted
+ * file from consuming the whole digest. Reserving most of the budget for it
+ * costs almost nothing and is what keeps an early constraint like "请用pixi管理
+ * 开发环境" from being evicted by 131 extracted blocks.
+ */
+const USER_BUDGET_SHARE = 0.9;
 /** Tool names whose call arguments name a written path. */
 const WRITE_TOOLS = new Set(['write', 'edit', 'multiedit', 'apply_patch']);
 /** The message an event derives to, or null when it derives to none. */
@@ -133,10 +152,10 @@ export function isErrorResult(event) {
 }
 /** Directive appended as the final user message of the extraction call. */
 const EXTRACT_INSTRUCTION = [
-    'You are summarizing ONE turn of a coding session for a compaction checkpoint.',
-    'The material below is the turn\'s user request, its assistant prose (may be empty), and a trace of the tool calls it made (descriptions only) plus any results the log marked as errors.',
+    'You are summarizing a batch of turns from a coding session for a compaction checkpoint.',
+    'Each turn is introduced by a line `=== TURN <n> ===` and contains the user request, the assistant prose (may be empty), and a trace of the tool calls it made (descriptions only) plus any results the log marked as errors.',
     '',
-    'Output EXACTLY these three Markdown headings, in order. Under each, at most two terse lines. If a section has no real information, write "(none)" — never invent.',
+    'For EVERY turn, emit exactly one block, introduced by the same `=== TURN <n> ===` line, containing these three Markdown headings in order. Under each, at most two terse lines. If a section has no real information, write "(none)" — never invent.',
     '',
     '## Did',
     '- what was actually done, concretely',
@@ -147,8 +166,12 @@ const EXTRACT_INSTRUCTION = [
     '## Overturned',
     '- earlier conclusions this turn disproved or replaced',
     '',
-    'Do not describe the tool calls themselves ("ran bash", "explored the repo"). State what came out of them. If the trace is too thin to support a claim, write "(none)".',
+    'Do not describe the tool calls themselves ("ran bash", "explored the repo"). State what came out of them. Do not repeat a conclusion the previous turn already stated. If the trace is too thin to support a claim, write "(none)".',
 ].join('\n');
+/** Turns handed to one extraction call. */
+const TURNS_PER_CALL = 8;
+/** Output tokens allowed per turn in an extraction call. */
+const EXTRACT_TOKENS_PER_TURN = 220;
 /** Most tool calls one turn may contribute to the hint block. */
 const MAX_HINTS_PER_TURN = 120;
 /** Characters kept from one call's description or command head. */
@@ -327,19 +350,57 @@ export function buildSections(events) {
     }
     return sections;
 }
-/** Render one section to digest text, truncating assistant prose to `capChars`. */
+/**
+ * Attach extracted blocks to the turns they belong to.
+ *
+ * Keyed by turn number, with `undefined` covering a log that carries no
+ * `turn/start` markers. A turn with no entry keeps its prose, which is the
+ * fallback path when extraction is unavailable.
+ *
+ * @param sections - sections built from the digest source.
+ * @param extracted - turn number to extracted block.
+ * @returns the same sections with `extracted` set where available.
+ */
+function withExtractions(sections, extracted) {
+    return sections.map((section) => ({
+        turns: section.turns.map((turn) => {
+            const block = extracted.get(turn.turn);
+            return block === undefined ? turn : { ...turn, extracted: block };
+        }),
+    }));
+}
+/**
+ * Render one section to digest text, truncating assistant prose to `capChars`.
+ *
+ * When the turn carries an extraction, that extraction *replaces* the assistant
+ * prose. Keeping both was the duplication this design removes: 58.2% of
+ * assistant characters are substantive conclusions that the extraction restates,
+ * and the doubled copy pushed real history out through the budget.
+ *
+ * The tool pointer line stays either way: it is one line, and it is the only
+ * record of what was actually run when the prose and extraction both say little.
+ */
 function renderSection(section, capChars) {
     const lines = [];
     for (const turn of section.turns) {
         const label = turn.turn === undefined ? '?' : String(turn.turn);
         if (turn.user.length > 0)
             lines.push(`\n## user (turn ${label})\n${turn.user.join('\n')}`);
-        let body = turn.assistant.join('\n');
-        if (capChars !== undefined && body.length > capChars) {
-            body = `${body.slice(0, capChars)}\n…(+${body.length - capChars} chars)`;
+        if (turn.extracted !== undefined) {
+            let block = turn.extracted;
+            if (capChars !== undefined && block.length > capChars) {
+                block = `${block.slice(0, capChars)}\n…(+${block.length - capChars} chars)`;
+            }
+            lines.push(`### assistant (turn ${label})\n${block}`);
+        }
+        else {
+            let body = turn.assistant.join('\n');
+            if (capChars !== undefined && body.length > capChars) {
+                body = `${body.slice(0, capChars)}\n…(+${body.length - capChars} chars)`;
+            }
+            if (body.length > 0)
+                lines.push(`### assistant (turn ${label})\n${body}`);
         }
-        if (body.length > 0)
-            lines.push(`### assistant (turn ${label})\n${body}`);
         if (turn.tools.size > 0) {
             const names = [...turn.tools.entries()]
                 .sort((left, right) => right[1] - left[1])
@@ -355,12 +416,23 @@ function renderSection(section, capChars) {
     }
     return lines.join('\n');
 }
-/** Total characters of assistant prose across every section. */
+/**
+ * Total characters of turn bodies across every section: the extraction where a
+ * turn has one, its prose otherwise.
+ *
+ * Both must count, because the cap applies to whichever body is rendered. When
+ * only prose counted, a digest made almost entirely of extractions reported a
+ * bound of zero, the bisection had nothing to shrink, and the renderer fell
+ * through to dropping the oldest turns outright — which deleted the earliest
+ * human messages, the standing constraints the digest exists to keep.
+ */
 function assistantChars(sections) {
     let total = 0;
-    for (const section of sections)
-        for (const turn of section.turns)
-            total += turn.assistant.join('\n').length;
+    for (const section of sections) {
+        for (const turn of section.turns) {
+            total += turn.extracted !== undefined ? turn.extracted.length : turn.assistant.join('\n').length;
+        }
+    }
     return total;
 }
 /**
@@ -369,7 +441,10 @@ function assistantChars(sections) {
  * The two truncation tiers are the plan's T1 and T1', in that order:
  *
  * - **T1** user prose is verbatim unless it alone exceeds `USER_BUDGET_SHARE` of
- *   the budget, then the oldest turns are cut first.
+ *   the budget, then the oldest turns are cut first. The share is 0.9 rather than
+ *   0.5 because real user prose is a few percent of the digest: at 0.5 the
+ *   extracted blocks crowded out the oldest human messages, which are exactly the
+ *   standing constraints (「请用pixi管理开发环境」) that must survive.
  * - **T1'** assistant prose is verbatim unless the whole digest overflows, then
  *   it is cut evenly per turn down to `MIN_TURN_CHARS`.
  *
@@ -378,19 +453,26 @@ function assistantChars(sections) {
  * and the surrounding turns, a user's request cannot. The 800-character floor
  * exists so a tight budget thins every turn instead of erasing the last few.
  *
- * Neither tier runs in practice: over the eight measured sessions the largest
- * digest was 44542 tokens against a 196752 budget, and user prose totalled 2591
- * tokens against a 98376 cap. They are a guard against a pasted file, not the
- * normal path.
- *
- * @param events - the digest source in log order.
- * @param budgetTokens - hard ceiling in the meter's fixed 4-chars-per-token heuristic.
- * @returns the digest text, never longer than `budgetTokens * 4` characters.
+ * Both tiers run on a long session, and that is by design: the ceiling is
+ * absolute (`DIGEST_TOKEN_CEILING`), so a 122-turn history with 316k characters
+ * of assistant prose cannot fit verbatim. In the extractive design most turns
+ * carry an extraction instead of prose, which keeps the digest far below the
+ * ceiling; the tiers exist for the turns where extraction was unavailable and
+ * the prose is all that remains.
  */
-export function renderDigest(events, budgetTokens) {
-    const sections = buildSections(events);
+export function renderDigest(events, budgetTokens, extracted) {
+    const sections = extracted === undefined ? buildSections(events) : withExtractions(buildSections(events), extracted);
     const budgetChars = Math.max(0, budgetTokens * 4);
     const render = (capChars, trimmed) => (trimmed ?? sections).map((section) => renderSection(section, capChars)).join('\n');
+    // User prose is reserved before anything else is placed.
+    //
+    // Measured on a 122-turn session: every human message in the span totalled 469
+    // characters (117 tokens), while one extracted block per turn totalled 55020
+    // characters. The extraction is 117x the size of the prose it sat next to, so
+    // any layout that lets them compete evicts the prose — and the prose is the
+    // standing constraints (「请用pixi管理开发环境」) that never expire, while the
+    // extraction describes work that is already finished.
+    const userReserve = Math.min(Math.floor(budgetChars * USER_BUDGET_SHARE), sections.reduce((total, section) => total + section.turns.reduce((sum, turn) => sum + turn.user.join('\n').length, 0), 0));
     const verbatim = render(undefined);
     if (verbatim.length <= budgetChars)
         return verbatim;
@@ -398,7 +480,7 @@ export function renderDigest(events, budgetTokens) {
     // assistant prose stays verbatim for now. The walk is backwards because the
     // newest request is the one the next turn depends on: cutting forwards would
     // keep the opening question and drop what the user just asked for.
-    const userCap = Math.floor(budgetChars * USER_BUDGET_SHARE);
+    const userCap = userReserve;
     let seen = 0;
     const t1 = sections.map((section) => ({ turns: section.turns.map((turn) => ({ ...turn, user: [...turn.user] })) }));
     for (let section = t1.length - 1; section >= 0; section -= 1) {
@@ -422,6 +504,10 @@ export function renderDigest(events, budgetTokens) {
     const afterT1 = render(undefined, t1);
     if (afterT1.length <= budgetChars)
         return afterT1;
+    // The user side fits inside its reserve; what overflows is the turn bodies
+    // (extraction or prose plus pointers). Thin those, and only drop whole turns
+    // once every body is already at zero — dropping a turn also drops the user
+    // prose in it, which is what the reserve exists to prevent.
     // T1': assistant prose, evenly per turn. Length is monotone non-decreasing in
     // the cap, so bisection finds the largest cap that fits exactly, in about
     // eight renders. The floor is honoured only while it fits; below that the cap
@@ -557,27 +643,41 @@ export function buildExtractionMaterial(turn, prose) {
  * @param signal - cancellation forwarded to the adapter.
  * @returns the extracted block, or null when extraction is unavailable.
  */
-async function extractTurn(ctx, agent, turn, prose, signal) {
+async function extractBatch(ctx, agent, batch, signal) {
+    const out = new Map();
     const llm = ctx.llm;
     if (llm === undefined)
-        return null;
+        return out;
     const route = agent.session.requestHeader()?.config;
     const provider = route?.provider ?? agent.options.provider;
     const model = route?.model ?? agent.options.model;
     if (provider === undefined || model === undefined)
-        return null;
-    const material = buildExtractionMaterial(turn, prose);
-    if (material.length === 0)
-        return null;
+        return out;
+    const parts = [];
+    for (const [turn, events] of batch) {
+        const prose = events
+            .filter((event) => event.type === 'assistant/message')
+            .map((event) => textOf(event))
+            .filter((text) => text.length > 0)
+            .join('\n');
+        const material = buildExtractionMaterial(events, prose);
+        if (material.length === 0)
+            continue;
+        parts.push(`=== TURN ${turn} ===\n${material}`);
+    }
+    if (parts.length === 0)
+        return out;
     const chunks = [];
     try {
         const stream = llm.stream({
             provider,
             model,
             messages: [
-                { role: 'user', content: [{ type: 'text', text: `${material}\n\n${EXTRACT_INSTRUCTION}` }] },
+                { role: 'user', content: [{ type: 'text', text: `${parts.join('\n\n')}\n\n${EXTRACT_INSTRUCTION}` }] },
             ],
-            maxTokens: 400,
+            // One block per turn, so the cap scales with the batch rather than being
+            // a per-turn 400 that a dozen turns would multiply past the digest ceiling.
+            maxTokens: EXTRACT_TOKENS_PER_TURN * parts.length,
             sessionId: agent.session.id,
             purpose: 'compaction',
             ...(signal === undefined ? {} : { signal }),
@@ -592,10 +692,21 @@ async function extractTurn(ctx, agent, turn, prose, signal) {
         }
     }
     catch {
-        return null;
+        return out;
     }
-    const text = chunks.join('').trim();
-    return text.length > 0 ? text : null;
+    // Split the answer back per turn. A turn the model skipped simply keeps its
+    // prose, so a partial answer degrades per turn instead of losing the batch.
+    const answer = chunks.join('').trim();
+    if (answer.length === 0)
+        return out;
+    const sections = answer.split(/^=== TURN (\d+) ===$/m);
+    for (let index = 1; index < sections.length; index += 2) {
+        const turn = Number(sections[index]);
+        const body = (sections[index + 1] ?? '').trim();
+        if (Number.isFinite(turn) && body.length > 0)
+            out.set(turn, body);
+    }
+    return out;
 }
 /**
  * Deterministic compaction backend: verbatim prose, pointerized tools, no model
@@ -633,16 +744,12 @@ export default class VerbatimCompactionEngine extends BasicCompactionEngine {
             if (shadowedSet.has(node.seq))
                 shadowedPrice += node.tokens;
         }
-        const budget = Math.max(0, Math.min(DIGEST_BUDGET_TOKENS, shadowedPrice - FRAME_TOKENS - SHRINK_MARGIN_TOKENS));
-        const verbatim = renderDigest(source, budget);
-        // Per-turn extraction, appended to the verbatim digest. Bounded by the same
-        // budget: if the extracted text would push the digest past it, the digest
-        // alone is kept, because an over-budget checkpoint fails the shrink guard and
-        // loses the whole compaction rather than some detail.
+        // Absolute ceiling, floored by the shrink guard. The ceiling keeps the digest
+        // size independent of history length; the guard bound only exists for the
+        // degenerate case of a span smaller than the ceiling.
+        const budget = Math.max(0, Math.min(DIGEST_TOKEN_CEILING, shadowedPrice - FRAME_TOKENS - SHRINK_MARGIN_TOKENS));
         const extracted = await this.extractAll(source, agent, _signal);
-        const text = extracted.length === 0
-            ? verbatim
-            : `${verbatim}\n\n# Extracted from tool traces\n${extracted}`.slice(0, Math.max(verbatim.length, budget * 4));
+        const text = renderDigest(source, budget, extracted);
         const route = session.requestHeader()?.config;
         return {
             summary: [{ type: 'text', text }],
@@ -660,10 +767,16 @@ export default class VerbatimCompactionEngine extends BasicCompactionEngine {
      * A turn that fails is simply absent from the output; one bad turn must not
      * drop the extraction for the rest.
      *
+     * Batched rather than one call per turn. One call per turn meant 131 calls on a
+     * 122-turn session, each with its own round trip: a measured extraction budget
+     * of 131 x 400 tokens could on its own exceed the whole digest ceiling. A
+     * batch of `TURNS_PER_CALL` turns costs one round trip and shares the prompt
+     * prefix, so the provider's KV cache is reused rather than rebuilt.
+     *
      * @param source - digest source events, in log order.
      * @param agent - owner of the session being compacted.
      * @param signal - cancellation forwarded to every call.
-     * @returns the extracted blocks joined by turn heading, or an empty string.
+     * @returns turn number to extracted block, omitting turns that yielded none.
      */
     async extractAll(source, agent, signal) {
         const turns = new Map();
@@ -684,20 +797,16 @@ export default class VerbatimCompactionEngine extends BasicCompactionEngine {
             else
                 bucket.push(event);
         }
-        const blocks = [];
-        for (const [turn, events] of [...turns.entries()].sort((left, right) => left[0] - right[0])) {
+        const ordered = [...turns.entries()].sort((left, right) => left[0] - right[0]);
+        const extracted = new Map();
+        for (let index = 0; index < ordered.length; index += TURNS_PER_CALL) {
             if (signal?.aborted === true)
                 break;
-            const prose = events
-                .filter((event) => event.type === 'assistant/message')
-                .map((event) => textOf(event))
-                .filter((text) => text.length > 0)
-                .join('\n');
-            const extracted = await extractTurn(this.ctx, agent, events, prose, signal);
-            if (extracted === null)
-                continue;
-            blocks.push(`\n## turn ${turn}\n${extracted}`);
+            const batch = ordered.slice(index, index + TURNS_PER_CALL);
+            const blocks = await extractBatch(this.ctx, agent, batch, signal);
+            for (const [turn, block] of blocks)
+                extracted.set(turn, block);
         }
-        return blocks.join('\n');
+        return extracted;
     }
 }

package/lib/types/index.d.ts

@@ -36,6 +36,24 @@ import type { ContentBlock, Message } from '@deepseek-ai/dsh-llm';
 import type { Session, SessionEvent, SessionSeq } from '@deepseek-ai/dsh-session';
 /** Cordis companion plugin name. Must equal this plugin's `cordis.patch.yml` row id. */
 export declare const name = "verbatim-compaction";
+/**
+ * Absolute ceiling for one digest, in the meter's fixed 4-chars-per-token
+ * heuristic.
+ *
+ * A share of the shadowed span is the wrong shape: it makes the digest grow
+ * with history. Measured on one real session, the share-based budget produced
+ * 2642 chars on an early span, 51019 on a later one, and 348539 on the span that
+ * triggered the compaction loop — because a bigger span licensed a bigger
+ * digest, and a digest near the threshold re-triggers immediately.
+ *
+ * A fixed ceiling makes the digest size independent of how long the session has
+ * run, which is what the host's own summarizer does (its summaries measured
+ * 14009-22208 chars across spans of 108829-173514 tokens).
+ *
+ * 20000 tokens holds roughly 100 turns of user prose plus one extracted block
+ * per turn, and leaves the checkpoint far below any measured threshold.
+ */
+export declare const DIGEST_TOKEN_CEILING = 20000;
 /** The replayed prefix the base class hands to `summarize`. */
 export interface SummarizationInput {
     /** Tool schemas reused for prefix-cache alignment; absent when the request carried none. */
@@ -87,6 +105,14 @@ interface Turn {
     resultBytes: number;
     errors: string[];
     writes: string[];
+    /**
+     * The extracted summary for this turn, when extraction succeeded.
+     *
+     * Set here rather than appended to a separate block so the extraction takes
+     * the assistant prose's place in the digest: keeping both duplicated the same
+     * content and pushed the prose out through the budget.
+     */
+    extracted?: string;
 }
 /** One digest section: everything from one non-injected user message to the next. */
 interface Section {
@@ -111,7 +137,10 @@ export declare function buildSections(events: readonly SessionEvent[]): Section[
  * The two truncation tiers are the plan's T1 and T1', in that order:
  *
  * - **T1** user prose is verbatim unless it alone exceeds `USER_BUDGET_SHARE` of
- *   the budget, then the oldest turns are cut first.
+ *   the budget, then the oldest turns are cut first. The share is 0.9 rather than
+ *   0.5 because real user prose is a few percent of the digest: at 0.5 the
+ *   extracted blocks crowded out the oldest human messages, which are exactly the
+ *   standing constraints (「请用pixi管理开发环境」) that must survive.
  * - **T1'** assistant prose is verbatim unless the whole digest overflows, then
  *   it is cut evenly per turn down to `MIN_TURN_CHARS`.
  *
@@ -120,16 +149,14 @@ export declare function buildSections(events: readonly SessionEvent[]): Section[
  * and the surrounding turns, a user's request cannot. The 800-character floor
  * exists so a tight budget thins every turn instead of erasing the last few.
  *
- * Neither tier runs in practice: over the eight measured sessions the largest
- * digest was 44542 tokens against a 196752 budget, and user prose totalled 2591
- * tokens against a 98376 cap. They are a guard against a pasted file, not the
- * normal path.
- *
- * @param events - the digest source in log order.
- * @param budgetTokens - hard ceiling in the meter's fixed 4-chars-per-token heuristic.
- * @returns the digest text, never longer than `budgetTokens * 4` characters.
+ * Both tiers run on a long session, and that is by design: the ceiling is
+ * absolute (`DIGEST_TOKEN_CEILING`), so a 122-turn history with 316k characters
+ * of assistant prose cannot fit verbatim. In the extractive design most turns
+ * carry an extraction instead of prose, which keeps the digest far below the
+ * ceiling; the tiers exist for the turns where extraction was unavailable and
+ * the prose is all that remains.
  */
-export declare function renderDigest(events: readonly SessionEvent[], budgetTokens: number): string;
+export declare function renderDigest(events: readonly SessionEvent[], budgetTokens: number, extracted?: ReadonlyMap<number | undefined, string>): string;
 /**
  * The surface seqs this compaction shadows, recovered from the summarization
  * input's message count.
@@ -213,10 +240,16 @@ export default class VerbatimCompactionEngine extends BasicCompactionEngine {
      * A turn that fails is simply absent from the output; one bad turn must not
      * drop the extraction for the rest.
      *
+     * Batched rather than one call per turn. One call per turn meant 131 calls on a
+     * 122-turn session, each with its own round trip: a measured extraction budget
+     * of 131 x 400 tokens could on its own exceed the whole digest ceiling. A
+     * batch of `TURNS_PER_CALL` turns costs one round trip and shares the prompt
+     * prefix, so the provider's KV cache is reused rather than rebuilt.
+     *
      * @param source - digest source events, in log order.
      * @param agent - owner of the session being compacted.
      * @param signal - cancellation forwarded to every call.
-     * @returns the extracted blocks joined by turn heading, or an empty string.
+     * @returns turn number to extracted block, omitting turns that yielded none.
      */
     private extractAll;
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@logictan/dsh-verbatim-compaction",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "Deterministic compaction backend that keeps conversation prose verbatim in the checkpoint instead of re-summarizing it: user and assistant text survive word for word, tool results collapse to one-line pointers, reasoning is never read, and no model call is made.",
   "type": "module",
   "main": "lib/index.js",