pi-condense 2.11.0 → 2.11.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,21 @@ Published to npm as [`pi-condense`](https://www.npmjs.com/package/pi-condense) (
7
7
  Pushing a `vX.Y.Z` tag triggers `.github/workflows/release.yml`, which runs the tests and
8
8
  publishes via OIDC trusted publishing. See `.agents/skills/release/SKILL.md`.
9
9
 
10
+ ## [2.11.2] - 2026-09-29
11
+
12
+ ### Fixed
13
+
14
+ - Summaries of image-bearing tool results now say an image was returned instead of reporting the read as empty; the summarizer is told never to describe an image it cannot see.
15
+ - `context_tree_query` returns the original image blocks of a pruned image result instead of only its text.
16
+ - Pre-flush dedup no longer aliases different screenshots whose tool text is identical; image data is part of the dedup key.
17
+ - The live reclaim figure prices image blocks at the flat 1,600-token estimate instead of counting base64 as text.
18
+
19
+ ## [2.11.1] - 2026-09-29
20
+
21
+ ### Fixed
22
+
23
+ - The frontier-gap metric prices image blocks at a flat estimate instead of counting base64 as text, so screenshot reads no longer trigger a premature frontier-gap flush that prunes images before the model sees them.
24
+
10
25
  ## [2.11.0] - 2026-09-24
11
26
 
12
27
  ### Added
package/PRUNING.md CHANGED
@@ -312,7 +312,7 @@ The intended recovery flow is:
312
312
  3. The model decides the summary is not enough and wants exact raw output.
313
313
  4. The model calls `context_tree_query({ toolCallIds: ["t1", ...] })`. The tool accepts short refs and full `toolCallId`s interchangeably (`indexer.resolveToolCallId`).
314
314
  5. The tool looks up those IDs in the pruner index.
315
- 6. The tool returns the original stored output back into the current turn.
315
+ 6. The tool returns the original stored output back into the current turn. For each `[image returned: ...]` marker in a recovered record, it appends the original image block from the current session branch whose SHA-256 prefix matches; a record with no marker, or whose image is off the branch, returns text only.
316
316
  7. The model can now inspect that raw result and continue reasoning.
317
317
 
318
318
  ### ASCII: end-to-end "prune, then re-read" flow
@@ -690,10 +690,12 @@ Mechanism:
690
690
  - Keying by occurrence key, not bare id, means a duplicate is only ever aliased to the specific earlier *occurrence* it matches byte-for-byte - a reused id whose later occurrence has different content is not conflated with the stale one.
691
691
  - A `context-prune-dedup-alias` custom entry is persisted so `reconstructFromSession` rebuilds the maps after a restart.
692
692
  4. The duplicate is removed from the batch — no summarizer call, no new index entry.
693
- 5. Later, `pruneMessages` stub-replaces the duplicate's `ToolResultMessage` using the original's short ref, and `context_tree_query` returns the original's record whether the model passes the duplicate's id or the original's.
693
+ 5. Later, `pruneMessages` stub-replaces the duplicate's `ToolResultMessage` using the original's short ref, and `context_tree_query` returns the original's record whether the model passes the duplicate's id or the original's. An aliased image result recovers the original's image blocks (see [How the Model Re-reads Raw Outputs](#how-the-model-re-reads-raw-outputs)).
694
694
 
695
695
  Normalization is conservative: `\r\n` → `\n`, per-line trailing whitespace stripping, final `trim()`. Internal whitespace, tabs, and capitalization are preserved so two genuinely different outputs do **not** collide.
696
696
 
697
+ Image blocks enter `resultText` as leading marker lines `[image returned: <mimeType> sha256:<8hex>]` (`extractToolResultText`, `src/batch-capture.ts`), so the marker is part of the dedup key and two image results alias only when their base64 data matches to the 8-hex prefix. Results captured before the marker existed carry none.
698
+
697
699
  Typical wins: re-reading an unchanged file, repeated `git status` / `ls`, retries of the same command. v1 only matches against records **already in the indexer** (cross-flush dedup); intra-flush dedup is deferred so canonicals that get skipped as oversized / trivial never produce dangling aliases.
698
700
 
699
701
  Edit with `/pruner dedup on|off|status` or the settings overlay.
@@ -715,7 +717,7 @@ The frontier's `lastAttemptedTurnIndex` uses a session-wide numbering domain: it
715
717
  - **Tree browser (`/pruner tree`):** interactive, foldable tree of pruned tool calls grouped under their summaries. `Ctrl-O` on a summary node opens the full markdown summary in a bordered overlay.
716
718
  - **Configurable summarizer thinking (`summarizerThinking`):** trade summary cost / latency for quality (`off` / `minimal` / `low` / `medium` / `high` / `xhigh`). `default` omits the option entirely so the provider chooses.
717
719
  - **Cumulative stats:** `context-prune-stats` entries track input/output tokens and cost of every summarizer call; full detail surfaces in `/pruner stats`. Cost is also emitted on the `cost:external` pi.events channel for external aggregators (cumulative per session, live only).
718
- - **Live reclaim ratio:** measured once per `pruneMessages` call via `sizeMessages(messages) = JSON.stringify(messages).length`, comparing the input array before pruning to the result after. Estimated tokens = chars / 4. The measurement covers all five phases in a single point (stub-replace, supersede, error-purge, chain-range-prune, orphan-sweep); appears on the status line as `│ prune: ON · 92.0k->14.0k (-85%)` once at least one prune has occurred (the leading `│` keeps the segment visually isolated in the shared footer, load-order independent - there is no trailing divider, since the footer's own space-join between segments already provides one).
720
+ - **Live reclaim ratio:** measured once per `pruneMessages` call via `sizeMessages(messages)` = the sum of `charsOf(message)` (`src/context-metrics.ts`): whole-message JSON chars with each image block priced at a flat 1,600 tokens (6,400 chars) instead of its base64 length, the same estimate the frontier-gap trigger uses. This compares the input array before pruning to the result after. Estimated tokens = chars / 4. The measurement covers all five phases in a single point (stub-replace, supersede, error-purge, chain-range-prune, orphan-sweep); appears on the status line as `│ prune: ON · 92.0k->14.0k (-85%)` once at least one prune has occurred (the leading `│` keeps the segment visually isolated in the shared footer, load-order independent - there is no trailing divider, since the footer's own space-join between segments already provides one).
719
721
  - **Live progress for `/pruner now`:** an `aboveEditor` widget shows one row per pending batch with braille spinner, streamed summary-char count, and ✓ / ⚠ status.
720
722
 
721
723
  ### Summarizer outage fallback
package/README.md CHANGED
@@ -82,7 +82,7 @@ Every summarizer cost update is emitted on the shared `pi.events` channel `cost:
82
82
  | Term | Meaning |
83
83
  |---|---|
84
84
  | Stub | The short breadcrumb (`[Summarized in pruner summary, ref \`t1\`...]`) that replaces a pruned tool result in context |
85
- | `context_tree_query` | The tool the model calls to recover a stubbed original by ref (`tN`) or `toolCallId`. A reused id returns every matching occurrence, not just one, including any that were content-deduplicated to an earlier record - see [PRUNING.md § Occurrence Identity](PRUNING.md#occurrence-identity) |
85
+ | `context_tree_query` | The tool the model calls to recover a stubbed original by ref (`tN`) or `toolCallId`. Image results come back with their original image blocks. A reused id returns every matching occurrence, not just one, including any that were content-deduplicated to an earlier record - see [PRUNING.md § Occurrence Identity](PRUNING.md#occurrence-identity) |
86
86
  | Batch vs chain | A batch is one flush's worth of tool calls; a chain is a longer closed sequence eligible for range compression |
87
87
  | Prune frontier | The last attempted prune boundary - advances even on a skip, so nothing is reconsidered twice |
88
88
  | Diagnostics (`diag u/m/o/b`) | A self-hiding status-line segment surfacing prune-time degradations: `u` = unresolved chain range, `m` = detection/render id mismatch (informational, does not change what's dropped), `o` = orphan tool-result sweep, `b` = a zero-coverage chain with nothing left to backfill (genuine span mismatch, see below). Each letter's count is omitted when zero; the whole segment disappears when all four are zero. Backing session entries are `context-prune-diagnostic` - see below |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-condense",
3
- "version": "2.11.0",
3
+ "version": "2.11.2",
4
4
  "description": "Pi coding-agent extension that summarizes completed tool-call batches, replaces raw outputs with short stubs, compresses closed tool-call chains, and recovers any original on demand via context_tree_query.",
5
5
  "author": "Jacek Juraszek",
6
6
  "license": "MIT",
@@ -3,10 +3,14 @@ import {
3
3
  captureBatch,
4
4
  captureUnindexedBatchesFromSession,
5
5
  deriveLiveTurnIndex,
6
+ extractToolResultText,
7
+ imageDigest,
8
+ imageMarkerDigests,
6
9
  projectBranchMessages,
7
10
  serializeBatchForSummarizer,
8
11
  } from "./batch-capture.js";
9
12
  import type { CapturedBatch, CapturedToolCall } from "./types.js";
13
+ import { ToolCallIndexer } from "./indexer.js";
10
14
 
11
15
  function toolCall(overrides: Partial<CapturedToolCall> = {}): CapturedToolCall {
12
16
  return {
@@ -278,3 +282,62 @@ describe("custom-anchor group boundary", () => {
278
282
  expect(batches[0].userTurnGroup).toBe(batches[1].userTurnGroup);
279
283
  });
280
284
  });
285
+
286
+ const img = (data: string, mimeType = "image/png") => ({ type: "image", data, mimeType });
287
+
288
+ describe("image markers", () => {
289
+ test("markers lead the text, in block order, with mime type and 8-hex hash", () => {
290
+ const text = extractToolResultText({
291
+ content: [{ type: "text", text: "Read image file [image/png]" }, img("AAAA"), img("BBBB", "image/jpeg")],
292
+ });
293
+ expect(text.split("\n")).toEqual([
294
+ `[image returned: image/png sha256:${imageDigest("AAAA")}]`,
295
+ `[image returned: image/jpeg sha256:${imageDigest("BBBB")}]`,
296
+ "Read image file [image/png]",
297
+ ]);
298
+ expect(imageDigest("AAAA")).toMatch(/^[0-9a-f]{8}$/);
299
+ });
300
+
301
+ test("same base64 -> same marker, different base64 -> different marker", () => {
302
+ const a1 = extractToolResultText({ content: [img("AAAA")] });
303
+ const a2 = extractToolResultText({ content: [img("AAAA")] });
304
+ const b = extractToolResultText({ content: [img("BBBB")] });
305
+ expect(a1).toBe(a2);
306
+ expect(a1).not.toBe(b);
307
+ });
308
+
309
+ test("text-only content is unchanged", () => {
310
+ expect(extractToolResultText({ content: [{ type: "text", text: "a" }, { type: "text", text: "b" }] })).toBe("a\nb");
311
+ });
312
+
313
+ test("imageMarkerDigests reads the digests back in marker order", () => {
314
+ const text = extractToolResultText({ content: [img("AAAA"), { type: "text", text: "x" }, img("BBBB")] });
315
+ expect(imageMarkerDigests(text)).toEqual([imageDigest("AAAA"), imageDigest("BBBB")]);
316
+ expect(imageMarkerDigests("no markers here")).toEqual([]);
317
+ });
318
+
319
+ test("a result with >2,000 text chars plus an image keeps the marker in the summarizer input", () => {
320
+ const message = { role: "assistant", content: [{ type: "toolCall", id: "r1", name: "read", input: { path: "a.png" } }], timestamp: 1 };
321
+ const results = [
322
+ { role: "toolResult", toolCallId: "r1", toolName: "read", content: [{ type: "text", text: "x".repeat(5000) }, img("AAAA")], isError: false, timestamp: 2 },
323
+ ];
324
+ const out = serializeBatchForSummarizer(captureBatch(message, results, 0, 9999));
325
+ expect(out).toContain(`[image returned: image/png sha256:${imageDigest("AAAA")}]`);
326
+ });
327
+
328
+ test("identical text with different data misses lookupByContent; identical data hits", () => {
329
+ const captureRead = (id: string, data: string, ts: number) =>
330
+ captureBatch(
331
+ { role: "assistant", content: [{ type: "toolCall", id, name: "read", input: { path: "s.png" } }], timestamp: ts },
332
+ [{ role: "toolResult", toolCallId: id, toolName: "read", content: [{ type: "text", text: "Read image file [image/png]" }, img(data)], isError: false, timestamp: ts + 1 }],
333
+ 0,
334
+ ts
335
+ );
336
+ const idx = new ToolCallIndexer();
337
+ idx.addBatch(captureRead("r1", "AAAA", 100), () => {});
338
+ const other = captureRead("r2", "BBBB", 200).toolCalls[0];
339
+ expect(idx.lookupByContent(other.toolName, other.resultText)).toBeUndefined();
340
+ const same = captureRead("r3", "AAAA", 300).toolCalls[0];
341
+ expect(idx.lookupByContent(same.toolName, same.resultText)).toBe("r1@101");
342
+ });
343
+ });
@@ -1,3 +1,4 @@
1
+ import { createHash } from "node:crypto";
1
2
  import type { SessionEntry } from "@earendil-works/pi-coding-agent";
2
3
  import type { CapturedBatch, CapturedToolCall, BatchingMode } from "./types.js";
3
4
  import { occKey, resultTimestampOf } from "./occurrence-key.js";
@@ -44,13 +45,28 @@ function projectCustomMessageEntry(e: any): any {
44
45
  return { role: "custom", customType: e.customType, content: e.content, display: e.display, details: e.details, timestamp: new Date(e.timestamp).getTime() };
45
46
  }
46
47
 
47
- /** Joins the text blocks of a ToolResultMessage into a single string. */
48
+ export function imageDigest(data: string): string {
49
+ return createHash("sha256").update(data).digest("hex").slice(0, 8);
50
+ }
51
+
52
+ const IMAGE_MARKER_DIGEST = /\[image returned: [^\]\n]* sha256:([0-9a-f]{8})\]/g;
53
+
54
+ export function imageMarkerDigests(text: string): string[] {
55
+ return [...text.matchAll(IMAGE_MARKER_DIGEST)].map((m) => m[1]);
56
+ }
57
+
58
+ /**
59
+ * Joins a ToolResultMessage's content into one string: one marker line per
60
+ * image block, then the text blocks. Markers lead because the summarizer
61
+ * input keeps only the first 2,000 result chars.
62
+ */
48
63
  export function extractToolResultText(msg: any): string {
49
64
  const content: any[] = Array.isArray(msg?.content) ? msg.content : [];
50
- return content
51
- .filter((c: any) => c.type === "text")
52
- .map((c: any) => c.text)
53
- .join("\n");
65
+ const markers = content
66
+ .filter((c: any) => c.type === "image")
67
+ .map((c: any) => `[image returned: ${c.mimeType} sha256:${imageDigest(c.data)}]`);
68
+ const texts = content.filter((c: any) => c.type === "text").map((c: any) => c.text);
69
+ return [...markers, ...texts].join("\n");
54
70
  }
55
71
 
56
72
  /**
@@ -260,6 +260,22 @@ describe("computeContextMetrics", () => {
260
260
  expect(result.frontierGapTokens).toBe(expected);
261
261
  });
262
262
 
263
+ test("frontierGapTokens: image blocks use a flat estimate instead of base64 length", () => {
264
+ const msgs = [userMsg(100), assistantWithTools(200, ["tc1", "tc2", "tc3"])];
265
+ for (let i = 0; i < 3; i++) {
266
+ const result = toolResult(300 + i, `tc${i + 1}`, "read", "screenshot");
267
+ result.content.push({ type: "image", data: "A".repeat(137000), mimeType: "image/png" });
268
+ msgs.push(result);
269
+ }
270
+ const expected = msgs.slice(2).reduce((sum, msg) => {
271
+ const stub = { ...msg, content: msg.content.map((block: any) => block.type === "image" ? { type: "image" } : block) };
272
+ return sum + Math.round((JSON.stringify(stub).length + 1600 * 4) / 4);
273
+ }, 0);
274
+ const result = computeContextMetrics(msgs, null, noSummarized, noProtected);
275
+ expect(result.frontierGapTokens).toBe(expected);
276
+ expect(result.frontierGapTokens).toBeLessThan(20000);
277
+ });
278
+
263
279
  test("frontierGapTokens: boundary mid-turn split excludes at-or-before calls, includes later calls in same turn", () => {
264
280
  const msgs = [userMsg(100), assistantWithTools(200, ["tc1", "tc2"]), toolResult(300, "tc1"), toolResult(310, "tc2")];
265
281
  const frontier = fullFrontier({ lastAttemptedToolCallId: "tc1", lastAttemptedTurnIndex: 0 });
@@ -2,8 +2,17 @@ import { detectChains, isChainAnchorCustom } from "./chain-detector.js";
2
2
  import { occKey, resultTimestampOf } from "./occurrence-key.js";
3
3
  import type { ContextMetricsSnapshot, PruneFrontier } from "./types.js";
4
4
 
5
- function charsOf(msg: any): number {
6
- return JSON.stringify(msg).length;
5
+ const IMAGE_TOKEN_ESTIMATE = 1600;
6
+
7
+ export function charsOf(msg: any): number {
8
+ if (!Array.isArray(msg.content)) return JSON.stringify(msg).length;
9
+ let imageCount = 0;
10
+ const content = msg.content.map((block: any) => {
11
+ if (block?.type !== "image") return block;
12
+ imageCount++;
13
+ return { type: "image" };
14
+ });
15
+ return JSON.stringify({ ...msg, content }).length + imageCount * IMAGE_TOKEN_ESTIMATE * 4;
7
16
  }
8
17
 
9
18
  function tokensOf(msg: any): number {
@@ -1,5 +1,6 @@
1
1
  import { describe, expect, it } from "bun:test";
2
2
  import { pruneMessages, sizeMessages } from "./pruner.js";
3
+ import { charsOf } from "./context-metrics.js";
3
4
  import { ToolCallIndexer } from "./indexer.js";
4
5
  import { CUSTOM_TYPE_INDEX } from "./types.js";
5
6
  import type { ChainCompressionConfig, ChainCompressionEntry } from "./types.js";
@@ -899,6 +900,19 @@ describe("occurrence-keyed coverage via mock indexer (spill / protection / grace
899
900
  });
900
901
 
901
902
  describe("sizeMessages", () => {
903
+ it("prices an image block with the flat charsOf estimate, not its base64 length", () => {
904
+ const msg = {
905
+ role: "toolResult",
906
+ toolCallId: "r1",
907
+ toolName: "read",
908
+ content: [{ type: "text", text: "Read image file [image/png]" }, { type: "image", data: "A".repeat(137000), mimeType: "image/png" }],
909
+ isError: false,
910
+ timestamp: 1,
911
+ };
912
+ expect(sizeMessages([msg])).toBe(charsOf(msg));
913
+ expect(sizeMessages([msg])).toBeLessThan(JSON.stringify(msg).length / 10);
914
+ });
915
+
902
916
  it("counts hidden fields (thinking blocks), not just visible text", () => {
903
917
  // Two messages with identical visible .text but different hidden content.
904
918
  // sizeMessages must count the full serialized weight so all reclaim
package/src/pruner.ts CHANGED
@@ -1,4 +1,5 @@
1
1
  import { createHash } from "node:crypto";
2
+ import { charsOf } from "./context-metrics.js";
2
3
  import type { ToolCallIndexer } from "./indexer.js";
3
4
  import type { ChainCompressionConfig, ErrorPurgeConfig } from "./types.js";
4
5
  import { isProtected, type ProtectionConfig } from "./protected.js";
@@ -11,13 +12,13 @@ import type { DiagnosticSink } from "./diagnostics.js";
11
12
  import { applySupersede, type SupersedeState } from "./supersede.js";
12
13
 
13
14
  /**
14
- * Estimate of a message array's context weight. Serializing the whole array
15
- * (not just visible text) is deliberate: it counts tool-call argument bodies
16
- * (error-purge) and tool-result arrays (stub-replace / chain-range) so all
17
- * reclaim mechanisms register.
15
+ * Estimate of a message array's context weight: whole-message JSON chars
16
+ * (tool-call arguments, tool-result arrays, hidden blocks) so every reclaim
17
+ * mechanism registers, with image blocks at the flat estimate `charsOf`
18
+ * shares with the frontier-gap trigger.
18
19
  */
19
20
  export function sizeMessages(messages: any[]): number {
20
- return JSON.stringify(messages).length;
21
+ return messages.reduce((sum, message) => sum + charsOf(message), 0);
21
22
  }
22
23
 
23
24
  /**
@@ -1,5 +1,6 @@
1
1
  import { describe, expect, test } from "bun:test";
2
2
  import { registerQueryTool } from "./query-tool.js";
3
+ import { imageDigest } from "./batch-capture.js";
3
4
  import { ToolCallIndexer } from "./indexer.js";
4
5
  import type { CapturedBatch } from "./types.js";
5
6
 
@@ -23,14 +24,72 @@ const captureLegacy = (idx: ToolCallIndexer, id: string, timestamp: number, text
23
24
  idx.addBatch(batch, () => {});
24
25
  };
25
26
 
26
- // execute returns { content: [{ type: "text", text }], details } (src/query-tool.ts:73-76)
27
- const runTool = async (indexer: ToolCallIndexer, toolCallIds: string[]): Promise<string> => {
27
+ const runToolResult = async (indexer: ToolCallIndexer, toolCallIds: string[], branch: any[] = []): Promise<any> => {
28
28
  let registered: any;
29
29
  registerQueryTool({ registerTool: (def: any) => (registered = def) } as any, indexer);
30
- const result = await registered.execute("call-1", { toolCallIds }, undefined, undefined, undefined);
31
- return result.content[0].text as string;
30
+ return registered.execute("call-1", { toolCallIds }, undefined, undefined, { sessionManager: { getBranch: () => branch } });
32
31
  };
33
32
 
33
+ const runTool = async (indexer: ToolCallIndexer, toolCallIds: string[], branch: any[] = []): Promise<string> =>
34
+ (await runToolResult(indexer, toolCallIds, branch)).content[0].text as string;
35
+
36
+ const png = (data: string) => ({ type: "image", data, mimeType: "image/png" });
37
+ const branchWith = (...images: any[]) => [
38
+ { type: "message", message: { role: "toolResult", toolCallId: "x", content: [{ type: "text", text: "Read image file [image/png]" }, ...images] } },
39
+ ];
40
+ const markerText = (data: string) => `[image returned: image/png sha256:${imageDigest(data)}]\nRead image file [image/png]`;
41
+
42
+ describe("context_tree_query image recovery", () => {
43
+ test("short ref returns only the image named by its marker", async () => {
44
+ const idx = new ToolCallIndexer();
45
+ capture(idx, "read_1", 1150, markerText("AAAA"), 0);
46
+ idx.registerSummaryRefs([{ shortId: "t1", toolCallId: "read_1", resultTimestamp: 1150 }]);
47
+ const result = await runToolResult(idx, ["t1"], branchWith(png("AAAA"), png("BBBB")));
48
+ expect(result.content[0].type).toBe("text");
49
+ expect(result.content.slice(1)).toEqual([png("AAAA")]);
50
+ });
51
+
52
+ test("explicit occurrence key returns the image", async () => {
53
+ const idx = new ToolCallIndexer();
54
+ capture(idx, "read_1", 1150, markerText("AAAA"), 0);
55
+ const result = await runToolResult(idx, ["read_1@1150"], branchWith(png("AAAA")));
56
+ expect(result.content.slice(1)).toEqual([png("AAAA")]);
57
+ });
58
+
59
+ test("bare alias id returns its original's image", async () => {
60
+ const idx = new ToolCallIndexer();
61
+ capture(idx, "read_1", 1150, markerText("AAAA"), 0);
62
+ idx.registerDuplicate("read_2@2150", "read_1@1150", () => {});
63
+ const result = await runToolResult(idx, ["read_2"], branchWith(png("AAAA")));
64
+ expect(result.content.slice(1)).toEqual([png("AAAA")]);
65
+ });
66
+
67
+ test("legacy record without marker returns text only", async () => {
68
+ const idx = new ToolCallIndexer();
69
+ captureLegacy(idx, "read_1", 1000, "Read image file [image/png]", 0);
70
+ expect((await runToolResult(idx, ["read_1"], branchWith(png("AAAA")))).content).toHaveLength(1);
71
+ });
72
+
73
+ test("image off the current branch returns text only", async () => {
74
+ const idx = new ToolCallIndexer();
75
+ capture(idx, "read_1", 1150, markerText("AAAA"), 0);
76
+ expect((await runToolResult(idx, ["read_1"], branchWith(png("BBBB")))).content).toHaveLength(1);
77
+ });
78
+
79
+ test("pre-change alias between different images returns text only", async () => {
80
+ const idx = new ToolCallIndexer();
81
+ capture(idx, "read_1", 1150, "Read image file [image/png]", 0);
82
+ idx.registerDuplicate("read_2@2150", "read_1@1150", () => {});
83
+ expect((await runToolResult(idx, ["read_2"], branchWith(png("AAAA"), png("BBBB")))).content).toHaveLength(1);
84
+ });
85
+
86
+ test("description mentions image blocks", () => {
87
+ let registered: any;
88
+ registerQueryTool({ registerTool: (def: any) => (registered = def) } as any, new ToolCallIndexer());
89
+ expect(registered.description).toContain("image blocks");
90
+ });
91
+ });
92
+
34
93
  describe("context_tree_query occurrence handling", () => {
35
94
  test("a bare id with two occurrences returns both blocks, chronologically", async () => {
36
95
  const idx = new ToolCallIndexer();
package/src/query-tool.ts CHANGED
@@ -2,6 +2,8 @@ import { readFile } from "node:fs/promises";
2
2
  import { Type } from "@sinclair/typebox";
3
3
  import { truncateHead, DEFAULT_MAX_BYTES, DEFAULT_MAX_LINES } from "@earendil-works/pi-coding-agent";
4
4
  import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
5
+ import type { ImageContent } from "@earendil-works/pi-ai";
6
+ import { imageDigest, imageMarkerDigests } from "./batch-capture.js";
5
7
  import type { ToolCallIndexer } from "./indexer.js";
6
8
  import { QUERY_TOOL_NAME } from "./types.js";
7
9
 
@@ -19,7 +21,7 @@ export function registerQueryTool(pi: ExtensionAPI, indexer: ToolCallIndexer): v
19
21
  name: QUERY_TOOL_NAME,
20
22
  label: "Query Original Tool History",
21
23
  description:
22
- "Retrieve original tool call results that have been pruned from active context. Pass the short refs listed in a pruner-summary message, e.g. context_tree_query({ toolCallIds: [\"t12\", \"t3\"] }), to get back the full original outputs. A raw id that was reused returns every occurrence, each labelled id@timestamp.",
24
+ "Retrieve original tool call results that have been pruned from active context. Pass the short refs listed in a pruner-summary message, e.g. context_tree_query({ toolCallIds: [\"t12\", \"t3\"] }), to get back the full original outputs. A raw id that was reused returns every occurrence, each labelled id@timestamp. Results include the original image blocks when the pruned output carried images.",
23
25
  promptSnippet: "Retrieve original pruned tool outputs by short ref",
24
26
  promptGuidelines: [
25
27
  "When you need the full output of a tool call that was summarized and pruned from context, use context_tree_query with the short refs listed in the relevant pruner-summary message.",
@@ -30,7 +32,15 @@ export function registerQueryTool(pi: ExtensionAPI, indexer: ToolCallIndexer): v
30
32
  }),
31
33
  }),
32
34
 
33
- async execute(_toolCallId, params, _signal, _onUpdate, _ctx) {
35
+ async execute(_toolCallId, params, _signal, _onUpdate, ctx) {
36
+ const branchImages = new Map<string, ImageContent>();
37
+ for (const entry of ctx.sessionManager.getBranch()) {
38
+ if (entry.type !== "message" || entry.message.role !== "toolResult") continue;
39
+ for (const block of entry.message.content) {
40
+ if (block.type === "image") branchImages.set(imageDigest(block.data), block);
41
+ }
42
+ }
43
+ const images: ImageContent[] = [];
34
44
  const foundRecords: Record<string, any> = {};
35
45
  const blocks: string[] = [];
36
46
 
@@ -70,6 +80,11 @@ export function registerQueryTool(pi: ExtensionAPI, indexer: ToolCallIndexer): v
70
80
  }
71
81
  }
72
82
 
83
+ for (const digest of imageMarkerDigests(raw)) {
84
+ const image = branchImages.get(digest);
85
+ if (image) images.push(image);
86
+ }
87
+
73
88
  const t = truncateHead(raw, {
74
89
  maxLines: DEFAULT_MAX_LINES,
75
90
  maxBytes: DEFAULT_MAX_BYTES,
@@ -87,7 +102,7 @@ export function registerQueryTool(pi: ExtensionAPI, indexer: ToolCallIndexer): v
87
102
  const combined = blocks.join("\n\n---\n\n");
88
103
 
89
104
  return {
90
- content: [{ type: "text", text: combined }],
105
+ content: [{ type: "text", text: combined }, ...images],
91
106
  details: { results: foundRecords },
92
107
  };
93
108
  },
package/src/spill.test.ts CHANGED
@@ -152,7 +152,7 @@ describe("G4/C4: legacy bare-id sidecar recovery", () => {
152
152
 
153
153
  let registered: any;
154
154
  registerQueryTool({ registerTool: (def: any) => (registered = def) } as any, indexer);
155
- const result = await registered.execute("call-1", { toolCallIds: ["bash_7"] }, undefined, undefined, undefined);
155
+ const result = await registered.execute("call-1", { toolCallIds: ["bash_7"] }, undefined, undefined, { sessionManager: { getBranch: () => [] } });
156
156
  const text = result.content[0].text as string;
157
157
 
158
158
  expect(text).toContain("OLD SPILLED BODY");
@@ -1,5 +1,26 @@
1
- import { describe, it, expect } from "bun:test";
2
- import { isUsableSummary } from "./summarizer.js";
1
+ import { describe, it, expect, mock } from "bun:test";
2
+ import * as actualCompat from "@earendil-works/pi-ai/compat";
3
+
4
+ let seenInput: any;
5
+ mock.module("@earendil-works/pi-ai/compat", () => ({
6
+ ...actualCompat,
7
+ stream: (_model: any, input: any) => {
8
+ seenInput = input;
9
+ return {
10
+ async *[Symbol.asyncIterator]() {},
11
+ async result() {
12
+ return {
13
+ stopReason: "stop",
14
+ content: [{ type: "text", text: "- summary" }],
15
+ usage: { input: 1, output: 1, cacheRead: 0, cacheWrite: 0, totalTokens: 2, cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 } },
16
+ };
17
+ },
18
+ };
19
+ },
20
+ }));
21
+
22
+ const { isUsableSummary, summarizeBatch } = await import("./summarizer.js");
23
+ const { DEFAULT_CONFIG } = await import("./types.js");
3
24
 
4
25
  describe("isUsableSummary", () => {
5
26
  it("accepts non-empty text that stopped normally", () => {
@@ -15,3 +36,26 @@ describe("isUsableSummary", () => {
15
36
  expect(isUsableSummary("- partial", "length")).toBe(false);
16
37
  });
17
38
  });
39
+
40
+ describe("summarizer prompt", () => {
41
+ it("tells the model that an image marker is an image it cannot see", async () => {
42
+ const model = { id: "m", provider: "p", name: "M" };
43
+ const ctx = {
44
+ model,
45
+ modelRegistry: {
46
+ find: () => model,
47
+ getApiKeyAndHeaders: async () => ({ ok: true, apiKey: "k", headers: {} }),
48
+ getProviderAuth: async () => undefined,
49
+ },
50
+ ui: { notify() {} },
51
+ } as any;
52
+ const batch = {
53
+ turnIndex: 0,
54
+ timestamp: 0,
55
+ assistantText: "",
56
+ toolCalls: [{ toolCallId: "a", toolName: "read", args: {}, resultText: "[image returned: image/png sha256:3f9a2c1e]\nRead image file [image/png]", isError: false }],
57
+ } as any;
58
+ await summarizeBatch(batch, DEFAULT_CONFIG, ctx);
59
+ expect(JSON.stringify(seenInput)).toContain("means the tool returned an image you cannot see");
60
+ });
61
+ });
package/src/summarizer.ts CHANGED
@@ -20,6 +20,8 @@ For each tool call provide:
20
20
 
21
21
  Keep each tool call to 1-3 bullet points. Skip calls that succeeded with nothing reusable to record. Be concise.
22
22
 
23
+ A result line of the form [image returned: <type> sha256:<hash>] means the tool returned an image you cannot see: state that an image was returned, and never describe or guess its content.
24
+
23
25
  Begin the first bullet of each tool call with that tool's [[N:toolname]] label, copied verbatim (both the number and the name) from its line in the input, as the plain, first thing on the line - no bold, backticks, or list numbering around it. Do not renumber, rename, or invent labels; if you skip a tool, skip its label too.`;
24
26
 
25
27
  const RANGE_SYSTEM_PROMPT = `You are fusing several per-step summaries of one CLOSED sub-task from an AI coding assistant's history into a SINGLE cohesive summary.