pi-condense 2.10.2 → 2.10.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +4 -0
- package/PRUNING.md +12 -3
- package/README.md +1 -1
- package/index.ts +34 -1
- package/package.json +1 -1
- package/src/protected.test.ts +8 -1
- package/src/protected.ts +6 -1
- package/src/pruner.test.ts +74 -1
- package/src/pruner.ts +19 -1
- package/src/reload-rearm.integration.test.ts +518 -0
- package/src/supersede.test.ts +302 -0
- package/src/supersede.ts +117 -0
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,10 @@ Published to npm as [`pi-condense`](https://www.npmjs.com/package/pi-condense) (
|
|
|
7
7
|
Pushing a `vX.Y.Z` tag triggers `.github/workflows/release.yml`, which runs the tests and
|
|
8
8
|
publishes via OIDC trusted publishing. See `.agents/skills/release/SKILL.md`.
|
|
9
9
|
|
|
10
|
+
## [2.10.3] - 2026-09-07
|
|
11
|
+
|
|
12
|
+
- **Protected-path supersession.** Only the newest read of a protected path (`protectedPaths` / `protectedTools` calls with a string `path`) stays verbatim; earlier reads of the same path become a one-line `[Superseded: ...]` stub. Applied at render time (`pruneMessages` phase 1b, `src/supersede.ts`) and only when the pruner is already rewriting at or before that position, or on a cold-cache event (`session_start`, `session_tree`, `model_select`, `session_compact`, `thinking_level_select`) - never as the sole mid-prefix change. No new session entry, index record, or config key; supersession stops exactly when no protected call remains (`protectedPaths: []` with the default `protectedTools: []`); a read protected by tool name alone still participates. Spec: `doc/specs/2026-09-07-protected-path-supersede.md` (partially supersedes the 2026-06-11 protected-paths spec's "verbatim forever" edge case).
|
|
13
|
+
|
|
10
14
|
## [2.10.2] - 2026-09-06
|
|
11
15
|
|
|
12
16
|
- Protect `gauntlet-overrides.md` reads by default alongside skill files, including `.pi/` and `doc/` paths, so per-repo harness contracts survive context pruning. User-supplied `protectedPaths` still replaces the defaults; path matching remains limited to `args.path`.
|
package/PRUNING.md
CHANGED
|
@@ -567,6 +567,8 @@ graph LR
|
|
|
567
567
|
captured batches (from turn_end or session scan)
|
|
568
568
|
│
|
|
569
569
|
├─ 1. Protected-tools/paths filter (capture-time, see below)
|
|
570
|
+
│ newest read per protected path stays verbatim; older reads of the same
|
|
571
|
+
│ path are stubbed at render time once the cache is cold anyway (see § Supersession)
|
|
570
572
|
│ tool calls whose toolName is in protectedTools, OR whose args.path
|
|
571
573
|
│ matches any protectedPaths glob, never enter the batch
|
|
572
574
|
│
|
|
@@ -614,7 +616,7 @@ Implementation: `src/pruner.ts` `pruneMessages(messages, indexer)` returns `{ me
|
|
|
614
616
|
|
|
615
617
|
### Protected tools & paths
|
|
616
618
|
|
|
617
|
-
A tool call is protected if **either** its `toolName` is in `protectedTools` **or** its `args.path` (string) matches any glob in `protectedPaths`. Protected calls are filtered out **at capture time** - they never enter the `pendingBatches` queue, so their raw `ToolResultMessage` stays verbatim in future LLM context.
|
|
619
|
+
A tool call is protected if **either** its `toolName` is in `protectedTools` **or** its `args.path` (string) matches any glob in `protectedPaths`. Protected calls are filtered out **at capture time** - they never enter the `pendingBatches` queue, so their raw `ToolResultMessage` stays verbatim in future LLM context until a newer read of the same path supersedes it (below).
|
|
618
620
|
|
|
619
621
|
**`protectedTools: string[]`** (default `[]`) - allowlist of tool names. Covers tools whose output is a small handle that must be reused byte-for-byte (e.g. a session-id) or planning tools like `todowrite` / `todoread`.
|
|
620
622
|
|
|
@@ -624,6 +626,10 @@ Glob contract: full-path match against the raw `args.path` string with `\` norma
|
|
|
624
626
|
|
|
625
627
|
**Render-time re-check:** stub replacement runs in-flight on every turn (`pruneMessages`). If a tool call's persisted `args` now satisfy `isProtected` (e.g. a pattern was added mid-session), the stub is skipped and the raw result is left verbatim - this repairs already-summarized records in existing sessions with no schema change. Declared limitation: records inside already-compressed chains (`context-prune-chain` entries) are NOT repaired - their `protectedToolCallIds` set is fixed at compression time (forward-only). Dedup-alias edge: an alias resolving to an unprotected original stays stubbed.
|
|
626
628
|
|
|
629
|
+
**Supersession (phase 1b, `src/supersede.ts`):** protected calls are never indexed, so content-hash dedup never sees them. Instead, at render time only the **newest** protected occurrence per normalized `args.path` (backslash -> slash; `offset`/`limit` ignored) stays verbatim; every earlier occurrence with a paired result becomes the one-line stub `[Superseded: <path> was read again later in this conversation - see the newer read. Re-read the file if this earlier content is needed.]`. The stub keeps `toolCallId`/`toolName`/`timestamp`, so pi-ai's orphan repair never fires; the assistant `toolCall` block is untouched. A call without a paired result never participates (an aborted call cannot steal the win). Recovery is "re-read the file" - no `t<N>` ref. Provider tool-call ids repeat across turns and an aborted call has no result, so a result is paired only with the same-id call in the immediately preceding assistant message (the per-turn open-set model orphan-sweep uses) - never by a global per-id cursor.
|
|
630
|
+
|
|
631
|
+
**Cadence (prompt-cache economics):** a superseded read is stubbed only when the pruner is already rewriting at or before its position - `floor` is the earliest result timestamp phase 1 will stub on the next render (indexed batches, plus dedup aliases registered in the pre-flush pass - those are stubbed by phase 1 whatever their batch's outcome; a `skipped-trivial`/`skipped-oversized` batch's own calls set none) or the `startUserTimestamp` of a chain compressed this turn - or on a guaranteed-cold event: `session_start`, `session_tree`, `model_select`, `session_compact`, `thinking_level_select` (`floor = 0`, activate all). Activation is session-sticky (in-memory `occKey` set, cleared on `session_start`/`session_tree`); between those moments a freshly superseded copy stays verbatim on purpose, since a mid-prefix rewrite re-bills the whole tail once. `isProtected` is evaluated live, so a path that stops matching `protectedPaths` drops out of supersession and rejoins the normal pipeline. No config key: supersession is on whenever protection is. Spec: `doc/specs/2026-09-07-protected-path-supersede.md`.
|
|
632
|
+
|
|
627
633
|
Names and patterns that don't match any captured tool call are silently ignored.
|
|
628
634
|
|
|
629
635
|
### Eager single-result spill
|
|
@@ -691,7 +697,7 @@ The last attempted prune boundary is persisted as `context-prune-frontier` so `f
|
|
|
691
697
|
- **Tree browser (`/pruner tree`):** interactive, foldable tree of pruned tool calls grouped under their summaries. `Ctrl-O` on a summary node opens the full markdown summary in a bordered overlay.
|
|
692
698
|
- **Configurable summarizer thinking (`summarizerThinking`):** trade summary cost / latency for quality (`off` / `minimal` / `low` / `medium` / `high` / `xhigh`). `default` omits the option entirely so the provider chooses.
|
|
693
699
|
- **Cumulative stats:** `context-prune-stats` entries track input/output tokens and cost of every summarizer call; full detail surfaces in `/pruner stats`. Cost is also emitted on the `cost:external` pi.events channel for external aggregators (cumulative per session, live only).
|
|
694
|
-
- **Live reclaim ratio:** measured once per `pruneMessages` call via `sizeMessages(messages) = JSON.stringify(messages).length`, comparing the input array before pruning to the result after. Estimated tokens = chars / 4. The measurement covers all
|
|
700
|
+
- **Live reclaim ratio:** measured once per `pruneMessages` call via `sizeMessages(messages) = JSON.stringify(messages).length`, comparing the input array before pruning to the result after. Estimated tokens = chars / 4. The measurement covers all five phases in a single point (stub-replace, supersede, error-purge, chain-range-prune, orphan-sweep); appears on the status line as `│ prune: ON · 92.0k->14.0k (-85%)` once at least one prune has occurred (the leading `│` keeps the segment visually isolated in the shared footer, load-order independent - there is no trailing divider, since the footer's own space-join between segments already provides one).
|
|
695
701
|
- **Live progress for `/pruner now`:** an `aboveEditor` widget shows one row per pending batch with braille spinner, streamed summary-char count, and ✓ / ⚠ status.
|
|
696
702
|
|
|
697
703
|
### Summarizer outage fallback
|
|
@@ -884,6 +890,7 @@ A **closed chain** is a span of messages from one user message - or a non-pruner
|
|
|
884
890
|
raw messages from session
|
|
885
891
|
│
|
|
886
892
|
├─ [1] tool-result stub-replace (per-batch; existing)
|
|
893
|
+
├─ [1b] supersede (older protected reads of a re-read path -> stub; see § Supersession)
|
|
887
894
|
├─ [2] error-purge (phase 2)
|
|
888
895
|
├─ [3] chain-range-prune (runs AFTER stubs)
|
|
889
896
|
│ resolve each entry to a positional index range
|
|
@@ -966,6 +973,8 @@ Chain compression does not delete data from the session JSONL. The original tool
|
|
|
966
973
|
|
|
967
974
|
The protected output is relocated (moved), not copied — the original `ToolResultMessage` is dropped with the rest of the middle turns. The text stays in LLM context because it is embedded in the surviving synthetic block. It is NOT registered in the tool-call index and is NOT recoverable via `context_tree_query`; it does not need to be, because it is present verbatim.
|
|
968
975
|
|
|
976
|
+
Relocation reads the array **after** phase 1b, so a protected read that has been superseded relocates as its one-line stub, not the verbatim body - the verbatim copy is the newer read elsewhere in context.
|
|
977
|
+
|
|
969
978
|
The `context-prune-chain` session entry carries the matching `protectedToolCallIds` array so `session_start` reconstruction can re-embed the outputs on reload.
|
|
970
979
|
|
|
971
980
|
**Rejected alternative:** skip compression for any chain that contains a protected tool. Rejected because `todowrite`/`todoread` recur in most chains for opted-in users, so this strategy would forfeit most chain compression for the people who most need `protectedTools`.
|
|
@@ -1089,7 +1098,7 @@ Error purge replaces those arg bodies with compact stubs after the error has coo
|
|
|
1089
1098
|
**Transform position:** Error purge runs in Phase 2, after stub-replace and before chain range prune.
|
|
1090
1099
|
|
|
1091
1100
|
```
|
|
1092
|
-
[stub-replace] → [error-purge] → [chain-range-prune] → [orphan-sweep]
|
|
1101
|
+
[stub-replace] → [supersede] → [error-purge] → [chain-range-prune] → [orphan-sweep]
|
|
1093
1102
|
```
|
|
1094
1103
|
|
|
1095
1104
|
**Config keys:**
|
package/README.md
CHANGED
|
@@ -169,7 +169,7 @@ Settings live under `contextPrune` in `<agent-dir>/settings.json` (`$PI_CODING_A
|
|
|
169
169
|
| `pruneOn` | `agent-message` | Trigger mode - see Architecture above |
|
|
170
170
|
| `autoBudgetThreshold` | `null` | Fraction (e.g. `0.8`) of the context window that force-flushes everything regardless of `pruneOn`; the trigger point is capped at 300k tokens |
|
|
171
171
|
| `frontierGapThresholdTokens` | `null` | Opt-in absolute-token flush trigger: fires at `turn_end` once the un-pruned tail past the prune frontier reaches N tokens, regardless of window size; recommended starting value `80000` |
|
|
172
|
-
| `protectedTools` / `protectedPaths` | `[]` / `["**/skills/**/*.md", "**/gauntlet-overrides.md"]` | Tool names / path globs that are never
|
|
172
|
+
| `protectedTools` / `protectedPaths` | `[]` / `["**/skills/**/*.md", "**/gauntlet-overrides.md"]` | Tool names / path globs that are never summarized; only the newest read per protected path stays verbatim (older reads of the same path are stubbed once the prompt cache is cold anyway) |
|
|
173
173
|
| `spillThreshold` | `65536` | Chars above which a single oversized result spills straight to a sidecar file |
|
|
174
174
|
|
|
175
175
|
The default also protects reads of [pi-gauntlet](https://github.com/jjuraszek/pi-gauntlet)'s per-repo `gauntlet-overrides.md` so the repo's harness contract stays available for gate decisions after pruning.
|
package/index.ts
CHANGED
|
@@ -45,6 +45,7 @@ import { StatsAccumulator, emitExternalCost } from "./src/stats.js";
|
|
|
45
45
|
import { PruneFrontierTracker } from "./src/frontier.js";
|
|
46
46
|
import { BlockRefIssuer } from "./src/block-refs.js";
|
|
47
47
|
import { compressEligible } from "./src/chain-compressor.js";
|
|
48
|
+
import { createSupersedeState, earliestChainStart, earliestResultTimestamp, lowerFloor } from "./src/supersede.js";
|
|
48
49
|
import { detectChains, withClosingMessage } from "./src/chain-detector.js";
|
|
49
50
|
import { inGraceRecoveryToolCallIds } from "./src/recovery-grace.js";
|
|
50
51
|
import { shouldBudgetFlush, shouldDeltaFlush, shouldFrontierGapFlush, usageFraction } from "./src/budget.js";
|
|
@@ -82,6 +83,10 @@ export default function (pi: ExtensionAPI) {
|
|
|
82
83
|
// (dedup'd across the session's lifetime, not per-render).
|
|
83
84
|
const diagnostics = new DiagnosticSink((type, data) => pi.appendEntry(type, data));
|
|
84
85
|
|
|
86
|
+
// Newest-protected-read-wins state (spec 2026-09-07). In-memory only: on
|
|
87
|
+
// session_start / session_tree the cold floor re-activates everything.
|
|
88
|
+
const supersede = createSupersedeState();
|
|
89
|
+
|
|
85
90
|
// Pending batches — accumulated until the prune trigger fires
|
|
86
91
|
const pendingBatches: CapturedBatch[] = [];
|
|
87
92
|
let isFlushing = false;
|
|
@@ -454,6 +459,12 @@ export default function (pi: ExtensionAPI) {
|
|
|
454
459
|
const dedupedBatches: CapturedBatch[] = [];
|
|
455
460
|
let firstFailureIndex = -1;
|
|
456
461
|
|
|
462
|
+
// Every tool call phase 1 will stub on the next render is a floor
|
|
463
|
+
// source for supersession: dedup aliases regardless of batch outcome,
|
|
464
|
+
// plus the batch's own calls when the batch was actually indexed.
|
|
465
|
+
const floorSources: import("./src/types.js").CapturedToolCall[] = [];
|
|
466
|
+
for (let i = 0; i < batches.length; i++) floorSources.push(...dedupedPerBatch[i].toolCalls);
|
|
467
|
+
|
|
457
468
|
for (let i = 0; i < batches.length; i++) {
|
|
458
469
|
const result = results[i];
|
|
459
470
|
if (result === null) {
|
|
@@ -530,6 +541,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
530
541
|
// Keep the in-memory summary-body registry current so chain compression
|
|
531
542
|
// can build synthetic chain messages without rescanning session entries.
|
|
532
543
|
indexer.registerSummaryBody(batchOccurrenceKeys, summaryText);
|
|
544
|
+
floorSources.push(...batch.toolCalls);
|
|
533
545
|
} else {
|
|
534
546
|
oversizedBatches.push(batch);
|
|
535
547
|
}
|
|
@@ -546,6 +558,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
546
558
|
processedBatches.push(batch);
|
|
547
559
|
}
|
|
548
560
|
|
|
561
|
+
lowerFloor(supersede, earliestResultTimestamp(floorSources));
|
|
562
|
+
|
|
549
563
|
// Restore unprocessed batches (those at and after the first failure)
|
|
550
564
|
if (firstFailureIndex >= 0) {
|
|
551
565
|
restoreBatches(batches.slice(firstFailureIndex));
|
|
@@ -664,6 +678,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
664
678
|
inGrace,
|
|
665
679
|
);
|
|
666
680
|
if (compressedEntries.length > 0) {
|
|
681
|
+
lowerFloor(supersede, earliestChainStart(compressedEntries));
|
|
667
682
|
statsAccum.addChainsCompressed(compressedEntries.length);
|
|
668
683
|
statsAccum.persist(pi);
|
|
669
684
|
emitExternalCost(pi, statsAccum);
|
|
@@ -785,6 +800,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
785
800
|
statsAccum.reconstructFromSession(ctx);
|
|
786
801
|
fallbackController.reset();
|
|
787
802
|
diagnostics.reset();
|
|
803
|
+
supersede.activated.clear();
|
|
804
|
+
supersede.floor = 0;
|
|
788
805
|
|
|
789
806
|
// Rebuild prune frontier from persisted session entries
|
|
790
807
|
frontier.reconstructFromSession(ctx);
|
|
@@ -826,6 +843,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
826
843
|
blockRefs.rebuildFrom(indexer.getChainEntries().map((e) => e.blockId));
|
|
827
844
|
statsAccum.reconstructFromSession(ctx);
|
|
828
845
|
diagnostics.reset();
|
|
846
|
+
supersede.activated.clear();
|
|
847
|
+
supersede.floor = 0;
|
|
829
848
|
frontier.reconstructFromSession(ctx);
|
|
830
849
|
// Pending batches belong to the old branch — discard them
|
|
831
850
|
pendingBatches.length = 0;
|
|
@@ -842,6 +861,18 @@ export default function (pi: ExtensionAPI) {
|
|
|
842
861
|
setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim(), diagnostics.counts());
|
|
843
862
|
});
|
|
844
863
|
|
|
864
|
+
// Cache is a per-model prefix; these three moments are cold regardless, so
|
|
865
|
+
// activating every pending supersession here costs no extra cache miss.
|
|
866
|
+
pi.on("model_select", async () => {
|
|
867
|
+
supersede.floor = 0;
|
|
868
|
+
});
|
|
869
|
+
pi.on("session_compact", async () => {
|
|
870
|
+
supersede.floor = 0;
|
|
871
|
+
});
|
|
872
|
+
pi.on("thinking_level_select", async () => {
|
|
873
|
+
supersede.floor = 0;
|
|
874
|
+
});
|
|
875
|
+
|
|
845
876
|
// ── turn_end: capture batch, flush immediately or queue ──────────────────
|
|
846
877
|
pi.on("turn_end", async (event, ctx) => {
|
|
847
878
|
if (!currentConfig.value.enabled) return;
|
|
@@ -994,7 +1025,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
994
1025
|
|
|
995
1026
|
// pruneMessages is the single source of truth for "is there work to do".
|
|
996
1027
|
// It returns the original array reference (pruned: false) only when none of
|
|
997
|
-
// the
|
|
1028
|
+
// the five phases changed anything; index/registry emptiness alone does not
|
|
998
1029
|
// imply a no-op, since error-purge (phase 2) prunes independently of them.
|
|
999
1030
|
// Calling it unconditionally is safe and avoids a split gate here.
|
|
1000
1031
|
const result = pruneMessages(
|
|
@@ -1005,6 +1036,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
1005
1036
|
currentConfig.value,
|
|
1006
1037
|
currentConfig.value.recoveryGraceTurns,
|
|
1007
1038
|
diagnostics,
|
|
1039
|
+
{ state: supersede, isProtected: protectionPredicate },
|
|
1008
1040
|
);
|
|
1009
1041
|
if (result.pruned) {
|
|
1010
1042
|
messages = result.messages;
|
|
@@ -1046,6 +1078,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
1046
1078
|
inGrace,
|
|
1047
1079
|
);
|
|
1048
1080
|
if (result.compressedEntries.length > 0) {
|
|
1081
|
+
lowerFloor(supersede, earliestChainStart(result.compressedEntries));
|
|
1049
1082
|
statsAccum.addChainsCompressed(result.compressedEntries.length);
|
|
1050
1083
|
statsAccum.persist(pi);
|
|
1051
1084
|
emitExternalCost(pi, statsAccum);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-condense",
|
|
3
|
-
"version": "2.10.
|
|
3
|
+
"version": "2.10.3",
|
|
4
4
|
"description": "Pi coding-agent extension that summarizes completed tool-call batches, replaces raw outputs with short stubs, compresses closed tool-call chains, and recovers any original on demand via context_tree_query.",
|
|
5
5
|
"author": "Jacek Juraszek",
|
|
6
6
|
"license": "MIT",
|
package/src/protected.test.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { describe, expect, test } from "bun:test";
|
|
2
|
-
import { globToRegExp, isProtected } from "./protected.js";
|
|
2
|
+
import { globToRegExp, isProtected, normalizePath } from "./protected.js";
|
|
3
3
|
import { DEFAULT_CONFIG } from "./types.js";
|
|
4
4
|
|
|
5
5
|
describe("globToRegExp", () => {
|
|
@@ -82,3 +82,10 @@ describe("isProtected", () => {
|
|
|
82
82
|
expect(isProtected("read", { path: "skills/a/SKILL.md" }, { protectedTools: [], protectedPaths: [] })).toBe(false);
|
|
83
83
|
});
|
|
84
84
|
});
|
|
85
|
+
|
|
86
|
+
describe("normalizePath", () => {
|
|
87
|
+
test("normalizePath turns backslashes into forward slashes and nothing else", () => {
|
|
88
|
+
expect(normalizePath("h\\skills\\x\\SKILL.md")).toBe("h/skills/x/SKILL.md");
|
|
89
|
+
expect(normalizePath("./a/../b.md")).toBe("./a/../b.md");
|
|
90
|
+
});
|
|
91
|
+
});
|
package/src/protected.ts
CHANGED
|
@@ -41,11 +41,16 @@ export function globToRegExp(pattern: string): RegExp {
|
|
|
41
41
|
return compiled;
|
|
42
42
|
}
|
|
43
43
|
|
|
44
|
+
/** Identity normalization shared by protection matching and supersession: slash direction only, no resolution. */
|
|
45
|
+
export function normalizePath(path: string): string {
|
|
46
|
+
return path.replace(/\\/g, "/");
|
|
47
|
+
}
|
|
48
|
+
|
|
44
49
|
export function isProtected(toolName: string, args: unknown, config: ProtectionConfig): boolean {
|
|
45
50
|
if (config.protectedTools.includes(toolName)) return true;
|
|
46
51
|
if (config.protectedPaths.length === 0) return false;
|
|
47
52
|
const path = (args as Record<string, unknown> | null | undefined)?.path;
|
|
48
53
|
if (typeof path !== "string") return false;
|
|
49
|
-
const normalized = path
|
|
54
|
+
const normalized = normalizePath(path);
|
|
50
55
|
return config.protectedPaths.some((p) => globToRegExp(p).test(normalized));
|
|
51
56
|
}
|
package/src/pruner.test.ts
CHANGED
|
@@ -4,7 +4,9 @@ import { ToolCallIndexer } from "./indexer.js";
|
|
|
4
4
|
import { CUSTOM_TYPE_INDEX } from "./types.js";
|
|
5
5
|
import type { ChainCompressionConfig, ChainCompressionEntry } from "./types.js";
|
|
6
6
|
import { DiagnosticSink } from "./diagnostics.js";
|
|
7
|
-
import { pruneWithZeroSweepAssertion } from "./test-support.js";
|
|
7
|
+
import { pruneWithZeroSweepAssertion, expectNoOrphanToolResults } from "./test-support.js";
|
|
8
|
+
import { createSupersedeState, supersededStub } from "./supersede.js";
|
|
9
|
+
import { isProtected } from "./protected.js";
|
|
8
10
|
|
|
9
11
|
// Minimal mock exposing only the ToolCallIndexer surface that pruneMessages calls.
|
|
10
12
|
// `hasLegacyBareRecord` defaults to the bare `summarized` set: most of the fixture
|
|
@@ -1053,3 +1055,74 @@ describe("G4/C3: orphan-sweep zero-fire proof across pruner fixtures", () => {
|
|
|
1053
1055
|
it(`zero orphan sweeps: ${name}`, run);
|
|
1054
1056
|
}
|
|
1055
1057
|
});
|
|
1058
|
+
|
|
1059
|
+
describe("pruneMessages phase 1b (supersede)", () => {
|
|
1060
|
+
const protection = { protectedTools: [], protectedPaths: ["**/skills/**/*.md"] };
|
|
1061
|
+
const isProt = (n: string, a: unknown) => isProtected(n, a, protection);
|
|
1062
|
+
const SKILL = "/h/skills/x/SKILL.md";
|
|
1063
|
+
|
|
1064
|
+
function twoReads(): any[] {
|
|
1065
|
+
return [
|
|
1066
|
+
{ role: "user", timestamp: 1, content: [{ type: "text", text: "go" }] },
|
|
1067
|
+
{ role: "assistant", timestamp: 2, content: [{ type: "toolCall", id: "r1", name: "read", input: { path: SKILL } }] },
|
|
1068
|
+
{ role: "toolResult", toolCallId: "r1", toolName: "read", content: [{ type: "text", text: "FIRST" }], isError: false, timestamp: 3 },
|
|
1069
|
+
{ role: "assistant", timestamp: 4, content: [{ type: "text", text: "ok" }] },
|
|
1070
|
+
{ role: "user", timestamp: 5, content: [{ type: "text", text: "again" }] },
|
|
1071
|
+
{ role: "assistant", timestamp: 6, content: [{ type: "toolCall", id: "r2", name: "read", input: { path: SKILL } }] },
|
|
1072
|
+
{ role: "toolResult", toolCallId: "r2", toolName: "read", content: [{ type: "text", text: "SECOND" }], isError: false, timestamp: 7 },
|
|
1073
|
+
{ role: "assistant", timestamp: 8, content: [{ type: "text", text: "done" }] },
|
|
1074
|
+
];
|
|
1075
|
+
}
|
|
1076
|
+
|
|
1077
|
+
it("supersede param absent -> output identical to today", () => {
|
|
1078
|
+
const msgs = twoReads();
|
|
1079
|
+
const { messages: out, pruned } = pruneMessages(msgs, makeMockIndexer(), undefined, undefined, protection);
|
|
1080
|
+
expect(pruned).toBe(false);
|
|
1081
|
+
expect(out).toBe(msgs);
|
|
1082
|
+
});
|
|
1083
|
+
|
|
1084
|
+
it("nothing activated -> input reference, pruned false", () => {
|
|
1085
|
+
const msgs = twoReads();
|
|
1086
|
+
const state = createSupersedeState();
|
|
1087
|
+
const { messages: out, pruned } = pruneMessages(msgs, makeMockIndexer(), undefined, undefined, protection, 0, undefined, { state, isProtected: isProt });
|
|
1088
|
+
expect(pruned).toBe(false);
|
|
1089
|
+
expect(out).toBe(msgs);
|
|
1090
|
+
});
|
|
1091
|
+
|
|
1092
|
+
it("one activation -> fresh array, pruned true, input untouched, newest verbatim, metadata kept", () => {
|
|
1093
|
+
const msgs = twoReads();
|
|
1094
|
+
const before = JSON.stringify(msgs);
|
|
1095
|
+
const state = createSupersedeState();
|
|
1096
|
+
state.floor = 0;
|
|
1097
|
+
const { messages: out, pruned } = pruneMessages(msgs, makeMockIndexer(), undefined, undefined, protection, 0, undefined, { state, isProtected: isProt });
|
|
1098
|
+
expect(pruned).toBe(true);
|
|
1099
|
+
expect(out).not.toBe(msgs);
|
|
1100
|
+
expect(JSON.stringify(msgs)).toBe(before);
|
|
1101
|
+
expect(out[2]).toEqual({ ...msgs[2], content: [{ type: "text", text: supersededStub(SKILL) }] });
|
|
1102
|
+
expect(out[6]).toBe(msgs[6]);
|
|
1103
|
+
expectNoOrphanToolResults(out);
|
|
1104
|
+
});
|
|
1105
|
+
|
|
1106
|
+
it("superseded read inside a compressed chain relocates as the stub", () => {
|
|
1107
|
+
const msgs = twoReads();
|
|
1108
|
+
const entry = {
|
|
1109
|
+
blockId: "b1",
|
|
1110
|
+
startUserTimestamp: 1,
|
|
1111
|
+
droppedToolCallIds: ["r1"],
|
|
1112
|
+
protectedToolCallIds: ["r1"],
|
|
1113
|
+
finalAssistantTimestamp: 4,
|
|
1114
|
+
toolRefs: [],
|
|
1115
|
+
compressedAt: 100,
|
|
1116
|
+
} as any;
|
|
1117
|
+
const indexer = makeMockIndexer({ chainEntries: [entry], summaryBodyMap: new Map([["r1", "SUMMARY"]]) });
|
|
1118
|
+
const state = createSupersedeState();
|
|
1119
|
+
state.floor = 0;
|
|
1120
|
+
const { messages: out } = pruneMessages(msgs, indexer, enabledCC, undefined, protection, 0, undefined, { state, isProtected: isProt });
|
|
1121
|
+
const synthetic = out.find((m: any) => typeof m.content?.[0]?.text === "string" && m.content[0].text.startsWith("<compressed-chain"));
|
|
1122
|
+
expect(synthetic.content[0].text).toContain('<protected-output tool="read">');
|
|
1123
|
+
expect(synthetic.content[0].text).toContain(supersededStub(SKILL));
|
|
1124
|
+
expect(synthetic.content[0].text).not.toContain("FIRST");
|
|
1125
|
+
expect(out.find((m: any) => m.role === "toolResult" && m.toolCallId === "r2").content[0].text).toBe("SECOND");
|
|
1126
|
+
expectNoOrphanToolResults(out);
|
|
1127
|
+
});
|
|
1128
|
+
});
|
package/src/pruner.ts
CHANGED
|
@@ -8,6 +8,7 @@ import { inGraceRecoveryToolCallIds } from "./recovery-grace.js";
|
|
|
8
8
|
import { occKey } from "./occurrence-key.js";
|
|
9
9
|
import { sweepOrphanToolResults } from "./orphan-sweep.js";
|
|
10
10
|
import type { DiagnosticSink } from "./diagnostics.js";
|
|
11
|
+
import { applySupersede, type SupersedeState } from "./supersede.js";
|
|
11
12
|
|
|
12
13
|
/**
|
|
13
14
|
* Estimate of a message array's context weight. Serializing the whole array
|
|
@@ -20,7 +21,7 @@ export function sizeMessages(messages: any[]): number {
|
|
|
20
21
|
}
|
|
21
22
|
|
|
22
23
|
/**
|
|
23
|
-
* Transforms the `context` event message array in
|
|
24
|
+
* Transforms the `context` event message array in five phases:
|
|
24
25
|
*
|
|
25
26
|
* Phase 1 — stub-replace: ToolResultMessages for summarized tool calls are
|
|
26
27
|
* replaced with short stubs pointing the model at `context_tree_query`.
|
|
@@ -37,6 +38,13 @@ export function sizeMessages(messages: any[]): number {
|
|
|
37
38
|
* to recovery is present on the toolResult itself, not only in the
|
|
38
39
|
* separate summary message.
|
|
39
40
|
*
|
|
41
|
+
* Phase 1b — supersede: protected reads (never indexed) whose `args.path`
|
|
42
|
+
* is read again later in the same context are replaced with a one-line
|
|
43
|
+
* "superseded" stub, but only once `SupersedeState.floor` says the pruner
|
|
44
|
+
* is rewriting at/before their position anyway (or the cache is cold).
|
|
45
|
+
* See src/supersede.ts. Runs before phase 3 so a superseded read inside a
|
|
46
|
+
* compressed chain relocates as the stub, not the verbatim body.
|
|
47
|
+
*
|
|
40
48
|
* Phase 2 — error purge: replaces failed toolCall arg bodies with stubs after a
|
|
41
49
|
* cooldown, reclaiming context from large `write`/`edit` arguments that will
|
|
42
50
|
* never succeed. The toolResult error message stays visible.
|
|
@@ -78,6 +86,7 @@ export function pruneMessages(
|
|
|
78
86
|
protection?: ProtectionConfig,
|
|
79
87
|
recoveryGraceTurns: number = 0,
|
|
80
88
|
diagnostics?: DiagnosticSink,
|
|
89
|
+
supersede?: { state: SupersedeState; isProtected: (toolName: string, args: unknown) => boolean },
|
|
81
90
|
): { messages: any[]; pruned: boolean; beforeChars: number; afterChars: number } {
|
|
82
91
|
// Phase 1: stub-replace summarized tool results
|
|
83
92
|
let pruned = false;
|
|
@@ -136,6 +145,15 @@ export function pruneMessages(
|
|
|
136
145
|
|
|
137
146
|
let current: any[] = pruned ? next : messages;
|
|
138
147
|
|
|
148
|
+
// Phase 1b: supersede older protected reads of a re-read path
|
|
149
|
+
if (supersede) {
|
|
150
|
+
const afterSupersede = applySupersede(current, supersede.state, supersede.isProtected);
|
|
151
|
+
if (afterSupersede !== current) {
|
|
152
|
+
current = afterSupersede;
|
|
153
|
+
pruned = true;
|
|
154
|
+
}
|
|
155
|
+
}
|
|
156
|
+
|
|
139
157
|
// Phase 2: error purge — replace failed toolCall arg bodies after cooldown
|
|
140
158
|
if (errorPurge?.enabled) {
|
|
141
159
|
const afterPurge = purgeErroredArgs(current, errorPurge);
|