pi-condense 2.4.3 → 2.6.0

package/CHANGELOG.md

@@ -7,6 +7,20 @@ Published to npm as [`pi-condense`](https://www.npmjs.com/package/pi-condense) (
 Pushing a `vX.Y.Z` tag triggers `.github/workflows/release.yml`, which runs the tests and
 publishes via OIDC trusted publishing. See `.agents/skills/release/SKILL.md`.
 
+## [2.6.0] - 2026-08-12
+
+- **Fix: reused provider tool-call ids could delete a live turn and produce a rejected request ([#8](https://github.com/jjuraszek/pi-condense/issues/8)).** Provider `toolCallId`s (e.g. `bash_23`) are unique only within one response - some providers restart a `${tool}_${n}` counter, so the same bare id recurs across a session denoting different tool calls. `applyChainCompressions` treated the id as session-durable identity: it unioned every persisted chain entry's `droppedToolCallIds` into one session-wide set and dropped any message matching it anywhere in the array, deleting a live assistant turn that happened to reuse a compressed chain's id and orphaning its tool result - rejected outright by Anthropic (`unexpected tool_use_id found in tool_result blocks`) and Kimi K3 (unresolvable tool name), unrecoverable without hand-editing the session JSONL. The same bare ids also mis-keyed the indexer, dedup, `isSummarized`, and batch capture.
+  - **Positional chain ranges.** `applyChainCompressions` now resolves each persisted `ChainCompressionEntry` to an index range via `resolveRange` (`src/chain-range-prune.ts`): exactly one `user`-role message at `startUserTimestamp`, exactly one `assistant`-role message at `finalAssistantTimestamp`, `start < end` - any other outcome (including `finalAssistantTimestamp === null`) drops nothing and inserts no synthetic, fail-closed. Drops are role-restricted to `assistant` / `toolResult` / per-batch-summary messages inside the range, so `user`-role messages - including the user-role `<compressed-chain>` synthetic, which keeps re-application idempotent - and third-party `custom_message` entries survive. Per-batch summary suppression stays coverage-based (`toolCallRefs` overlap), not index-membership, because under `batchingMode: "agent-message"` the summary lands after the range. `droppedToolCallIds` is retained only as a diagnostic cross-check against the range's actual contents; the range always wins.
+  - **Occurrence-keyed identity.** Records are now keyed `id@resultTimestamp` (`src/occurrence-key.ts`), the discriminant being the `ToolResultMessage`'s own timestamp (not `ToolCallRecord.timestamp`, which is a batch-level timestamp computed inconsistently between the live-capture and session-rescan paths). Persisted shapes gained optional `resultTimestamp` (index records, summary refs), `newResultTimestamp` / `originalResultTimestamp` (dedup aliases), and `droppedOccurrenceKeys` (chain entries) - all optional, so pre-upgrade entries keep bare-id keying and a session spanning the upgrade retains today's behavior for its pre-upgrade half. `isSummarized` is now a strict occurrence-key lookup; the sole sanctioned bare-id fallback is `hasLegacyBareRecord`, true only for a bare id with no occurrence-keyed siblings (a mixed id fails closed on both). Spill sidecars are named from the occurrence key going forward; recovery still reads the persisted `spillPath`, so pre-upgrade blobs keep resolving.
+  - **Orphan sweep.** `pruneMessages` now ends with a structural post-condition, `sweepOrphanToolResults` (`src/orphan-sweep.ts`): one forward pass with **per-turn** open-call tracking (each assistant message replaces the open set rather than accumulating into it, so an id used validly early cannot license a later genuine orphan under the same reused id) removes any `toolResult` no surviving assistant opened. Provider-agnostic and structural rather than keyed to any provider's error string; reference-preserving when nothing is swept, so the no-op / prompt-cache-prefix invariant (`doc/specs/2026-08-04-pruner-noop-serialization.md`) holds.
+  - **Diagnostics.** New `context-prune-diagnostic` session entry (`{ kind, detail }`, kinds `unresolved-range` / `range-id-mismatch` / `orphan-sweep`) surfaces these degradations without ever entering LLM context - zero tokens, zero cache-prefix change. Deduped per `(kind, dedupKey)`, reset on `session_start` / `session_tree`. The `/pruner` status line grows a self-hiding ` · diag u<N>/m<N>/o<N>` segment (u/m/o map to the three kinds above in that order); each zero counter is omitted and the whole segment disappears when all three are zero.
+  - **`context_tree_query` multi-occurrence recovery.** A raw provider id that was reused now returns every matching occurrence (not just one), each labelled `id@timestamp`, chronologically; a legacy record with no `resultTimestamp` is labelled with the bare id plus an explicit `Occurrence: legacy (no resultTimestamp)` line. Short refs (`tN`) are unaffected - still 1:1 and the primary recovery path.
+  - **Accepted limitation:** a session that spans the upgrade keeps bare-id keying for everything captured before it landed; only the post-upgrade half of such a session gets occurrence-keyed identity. No migration runs on load. Concretely, within such a session a live tool result whose provider id collides with a pre-upgrade summarized record can still be stub-replaced with that record's stale content (`hasLegacyBareRecord`'s bare-id fallback in `src/pruner.ts` cannot distinguish the two once both denote the same bare id with no occurrence-keyed siblings); new records are unaffected, and the exposure disappears once a session contains no legacy records.
+
+## [2.5.0] - 2026-08-05
+
+- **Removed the main-loop thinking strip (breaking: `contextPrune.thinkingStrip.*` no longer read).** The feature assumed thinking blocks we send are thinking blocks we are billed for. Verified against the live Anthropic API (`count_tokens` and real billed usage agree to the token), the actual rule is: thinking in a *closed* cycle bills 0 input tokens, and thinking in an *open* cycle survives only as an unbroken run starting at the cycle's first assistant turn — any gap discards everything after it. `keepLastTurns` kept the **last** K, so firing it punched a gap at the front and the API dropped all of it: the model received no thinking either way, and the no-gap → gap transition cost a full cache invalidation (measured +33% in a controlled 24-tool-call A/B, $0.6430 → $0.8531, with turn 17 rewriting 63113 tokens at 0% reuse). Across 58 sessions / 1233 open cycles the strip fired 253 times and reached its ~130-turn break-even twice. It was also redundant: chain compression's synthetic block is a `role: "user"` text message, which closes the cycle and frees all prior thinking server-side at zero cache cost. `pruneMessages` drops to three phases (`stub-replace → error-purge → chain-range-prune`); `src/thinking-strip.ts`, `ThinkingStripConfig`, `KEEP_LAST_TURNS_PRESETS`, the two `/pruner` settings entries, and `PruneFrontier.thinkingStripBoundaryTimestamp` are gone. A leftover `thinkingStrip` block in `settings.json` is inert and round-trips untouched (`normalize()` never filters unknown keys); a persisted `context-prune-frontier` entry carrying the old boundary field loads and is ignored. Rationale and measurements: `doc/specs/2026-08-05-remove-thinking-strip.md`.
+
 ## [2.4.3] - 2026-08-04
 
 - **Flush-gated, timestamp-keyed thinking strip ([#3](https://github.com/jjuraszek/pi-condense/issues/3)).** Phase 4 (`stripOldThinking`, `src/thinking-strip.ts`) recomputed its keep-window from the *live assistant count on every `context` render*, so each turn the `(count - keepLastTurns)`-th assistant slid forward and its thinking was stripped **deep in history**. pi-ai sets prompt-cache breakpoints only at `tools + system + last message` (verified in `@earendil-works/pi-ai` `api/anthropic-messages.js` `convertMessages` - no in-history breakpoint), so every deep mutation busted the cached suffix - roughly every render inside a tool loop. The strip boundary is now a **flush-computed, persisted assistant-message timestamp** (`thinkingStripBoundaryTimestamp`, added to the `PruneFrontier` snapshot on the existing `context-prune-frontier` entry): fixed between flushes so consecutive renders are byte-stable in their historical prefix (the cache survives a whole tool loop), advancing only on a non-empty flush (piggybacking summarization's own cache bust - zero marginal busts), monotonically clamped (never re-adds thinking to an already-stripped message, even when `keepLastTurns` is increased mid-session), and keyed by timestamp rather than array index so it is robust to phase-3 chain-range middle drops. An absent boundary (pre-feature sessions, pre-first-flush) falls back to the original live-count window verbatim. Additive optional field, fully backward-compatible; **no new config key**, `keepLastTurns` presets unchanged. `PRUNING.md` cache-impact model corrected. Turns the old k-busts-per-render into ~1-per-request.

package/PRUNING.md

@@ -27,13 +27,15 @@
    - [Budget-delta flush](#budget-delta-flush)
 10. [Chain Compression](#chain-compression)
     - [Protected-output relocation](#protected-output-relocation)
-11. [Error Purge](#error-purge)
-12. [Main-loop Thinking Strip](#main-loop-thinking-strip)
-13. [Why Summarization Works: Research Evidence](#why-summarization-works-research-evidence)
+11. [Occurrence Identity](#occurrence-identity)
+12. [Error Purge](#error-purge)
+13. [Orphan Sweep](#orphan-sweep)
+14. [Diagnostics](#diagnostics)
+15. [Why Summarization Works: Research Evidence](#why-summarization-works-research-evidence)
     - [SUPO — Summarization augmented Policy Optimization](#supo--summarization-augmented-policy-optimization)
     - [ReSum — Recursive Summarization for Long-Horizon Agents](#resum--recursive-summarization-for-long-horizon-agents)
     - [ACON — Agent Context Optimization](#acon--agent-context-optimization)
-14. [Summary](#summary)
+16. [Summary](#summary)
 
 ---
 
@@ -198,7 +200,7 @@ Pruning does **not** delete data. It moves raw tool results out of the hot path
 There are two separate things happening during pruning:
 
 1. **Context filtering:** future requests stop including the old `toolResult` messages.
-2. **Index preservation:** the extension stores each summarized tool call in the pruner index, keyed by `toolCallId`.
+2. **Index preservation:** the extension stores each summarized tool call in the pruner index, keyed by its occurrence key (`id@resultTimestamp`, or the bare id for legacy records - see [Occurrence Identity](#occurrence-identity)).
 
 That distinction is the core idea:
 
@@ -311,7 +313,7 @@ batch gets summarized
         ├─► summary message added to context
         │      └─► includes short refs (`t1`, `t2`, …)
         │
-        ├─► tool results indexed by toolCallId
+        ├─► tool results indexed by occurrence key (id@resultTimestamp)
         │      └─► full raw resultText stored in index/session
         │
         └─► old toolResult messages removed from future context
@@ -606,7 +608,7 @@ Properties:
 - `role: "toolResult"` and the original `toolCallId` / `toolName` / `timestamp` are preserved — role alternation is intact; no synthetic-result injection.
 - `isError: false`, so the model does not interpret the stub as a tool failure.
 - The stub references the **short ref** (`t1`, `t2`, …) the indexer assigned at summary time. Legacy entries from before short-refs landed fall back to the raw `toolCallId`.
-- Deterministic per `toolCallId` — the stub text never changes across calls, so the prefix cache continues to hit on the pruned range.
+- Deterministic per occurrence key — the stub text never changes across renders of the same `toolResult`, so the prefix cache continues to hit on the pruned range. A reused `toolCallId` is a *different* occurrence (different `resultTimestamp`) and gets its own stub and its own short ref.
 
 Implementation: `src/pruner.ts` `pruneMessages(messages, indexer)` returns `{ messages, pruned }`. When `pruned === false`, the original array reference is returned and the `context` handler skips reconstruction entirely.
 
@@ -660,9 +662,10 @@ Mechanism:
 
 1. When a batch enters `flushPending`, each tool call is hashed by `SHA-1(toolName + "\0" + normalize(resultText))`.
 2. The indexer's `contentHashToOriginal` map (populated by every earlier `addBatch` / `reconstructFromSession`) is consulted.
-3. A hit means an earlier prune already covered identical content. The duplicate is registered as an alias of the original via `indexer.registerDuplicate(newId, originalId, appendEntry)`:
-   - `dedupAliasToOriginal[newId] = originalId` (so `isSummarized(newId) === true` and `resolveToolCallId(newId) === originalId`).
-   - `toolCallIdToAlias[newId] = toolCallIdToAlias[originalId]` (so `getShortRefForToolCallId(newId)` returns the **same** `tN` as the original).
+3. A hit means an earlier prune already covered identical content. The duplicate is registered as an alias of the original via `indexer.registerDuplicate(newKey, originalKey, appendEntry)`, where both are occurrence keys (or legacy bare ids):
+   - `dedupAliasToOriginal[newKey] = originalKey` (so `isSummarized(newKey) === true` and `resolveToolCallId(newKey) === originalKey`).
+   - `toolCallIdToAlias[newKey] = toolCallIdToAlias[originalKey]` (so `getShortRefForToolCallId(newKey)` returns the **same** `tN` as the original).
+   - Keying by occurrence key, not bare id, means a duplicate is only ever aliased to the specific earlier *occurrence* it matches byte-for-byte - a reused id whose later occurrence has different content is not conflated with the stale one.
    - A `context-prune-dedup-alias` custom entry is persisted so `reconstructFromSession` rebuilds the maps after a restart.
 4. The duplicate is removed from the batch — no summarizer call, no new index entry.
 5. Later, `pruneMessages` stub-replaces the duplicate's `ToolResultMessage` using the original's short ref, and `context_tree_query` returns the original's record whether the model passes the duplicate's id or the original's.
@@ -688,7 +691,7 @@ The last attempted prune boundary is persisted as `context-prune-frontier` so `f
 - **Tree browser (`/pruner tree`):** interactive, foldable tree of pruned tool calls grouped under their summaries. `Ctrl-O` on a summary node opens the full markdown summary in a bordered overlay.
 - **Configurable summarizer thinking (`summarizerThinking`):** trade summary cost / latency for quality (`off` / `minimal` / `low` / `medium` / `high` / `xhigh`). `default` omits the option entirely so the provider chooses.
 - **Cumulative stats:** `context-prune-stats` entries track input/output tokens and cost of every summarizer call; full detail surfaces in `/pruner stats`. Cost is also emitted on the `cost:external` pi.events channel for external aggregators (cumulative per session, live only).
-- **Live reclaim ratio:** measured once per `pruneMessages` call via `sizeMessages(messages) = JSON.stringify(messages).length`, comparing the input array before pruning to the result after. Estimated tokens = chars / 4. The measurement covers all four reclaim mechanisms in a single point (stub-replace, error-purge, chain-range-prune, thinking-strip); appears on the status line as `│ prune: ON · 92k->14k (-85%) │` once at least one prune has occurred (the `│ … │` wrapper keeps the segment visually isolated in the shared footer, load-order independent).
+- **Live reclaim ratio:** measured once per `pruneMessages` call via `sizeMessages(messages) = JSON.stringify(messages).length`, comparing the input array before pruning to the result after. Estimated tokens = chars / 4. The measurement covers all four phases in a single point (stub-replace, error-purge, chain-range-prune, orphan-sweep); appears on the status line as `│ prune: ON · 92.0k->14.0k (-85%)` once at least one prune has occurred (the leading `│` keeps the segment visually isolated in the shared footer, load-order independent - there is no trailing divider, since the footer's own space-join between segments already provides one).
 - **Live progress for `/pruner now`:** an `aboveEditor` widget shows one row per pending batch with braille spinner, streamed summary-char count, and ✓ / ⚠ status.
 
 ### Summarizer outage fallback
@@ -863,6 +866,8 @@ A **closed chain** is a span of messages from one user message through any numbe
 | Final text-only assistant | **Kept**, thinking blocks stripped (safe — no following tool cycle depends on the signature) |
 | Synthetic `<compressed-chain>` user message | **Injected** immediately after the start user message |
 
+**Why there is no separate thinking strip.** The synthetic `<compressed-chain>` block is injected as a `role: "user"` text message, which Anthropic reads as a genuine user turn. That closes the assistant cycle, and the API drops every prior thinking block from the context window server-side - verified against the live API: the same 8-turn loop bills 3152 thinking tokens with a full `toolResult` history and 0 once a user text block precedes the final assistant. So chain compression already reclaims thinking mass for free, as a side effect of the range drop that was rewriting that region anyway. Stub replacement does not have this effect: it preserves `role: "toolResult"`, so the cycle stays open. A dedicated strip phase shipped through v2.4.3; removed in v2.5.0 - see `doc/specs/2026-08-05-remove-thinking-strip.md`.
+
 ### Transform composition order
 
 ```
@@ -871,23 +876,30 @@ raw messages from session
   ├─ [1] tool-result stub-replace   (per-batch; existing)
   ├─ [2] error-purge                (phase 2)
   ├─ [3] chain-range-prune          (runs AFTER stubs)
-  │        for each compressed chain:
-  │          drop middle assistants (by toolCallId overlap)
-  │          drop middle toolResults (by toolCallId)
-  │          suppress per-batch summaries (by toolCallRefs overlap)
-  │          inject <compressed-chain> after start user
-  │          strip thinking from final assistant
-  └─ [4] thinking-strip             (keep thinking on last K assistant turns)
+  │        resolve each entry to a positional index range
+  │        drop assistant / toolResult / per-batch-summary inside the range
+  │        inject <compressed-chain> after start user
+  │        strip thinking from final assistant
+  └─ [4] orphan sweep               (structural post-condition, see below)
 ```
 
-### Identification model
+### Identification model: positional ranges, not ids
+
+Pi-ai's `Message` union (`UserMessage | AssistantMessage | ToolResultMessage`) has no `.id` field, and provider `toolCallId`s are not session-durable (see [Occurrence Identity](#occurrence-identity) below). Chain compression therefore decides **what to drop** by message position, not by id membership.
+
+`resolveRange` (`src/chain-range-prune.ts`) maps a persisted `ChainCompressionEntry` to an index range in the current message array:
 
-Pi-ai's `Message` union (`UserMessage | AssistantMessage | ToolResultMessage`) has no `.id` field. Chain compression uses:
+- exactly one `user`-role message at `entry.startUserTimestamp`
+- exactly one `assistant`-role message at `entry.finalAssistantTimestamp`
+- `startIndex < endIndex`
 
-- `timestamp: number` to identify user / final-assistant boundary messages
-- `toolCallId` sets to identify middle assistant turns and their tool results
+Any other outcome - zero or multiple matches on either boundary, `finalAssistantTimestamp === null`, or a non-ordered pair - resolves to `null` and the entry drops **nothing** and inserts **no synthetic** for that range. This is fail-closed on purpose: an id-set or timestamp-window fallback (accepting a match even when it's ambiguous) risks deleting a live assistant turn that reuses a compressed chain's provider id, orphaning its tool result and producing a rejected request. A resolution failure is invisible in context - it is recorded as an `unresolved-range` diagnostic (see [Diagnostics](#diagnostics) below) and the affected range simply stays raw in context instead of being silently mis-compressed.
 
-This is why the persisted `ChainCompressionEntry` stores `startUserTimestamp` + `droppedToolCallIds` rather than message IDs.
+**Drops are role-restricted**, not index-membership-restricted: inside an accepted range, only `assistant`, `toolResult`, and per-batch-summary (`custom`, `context-prune-summary`) messages are removed. `user`-role messages inside the range - including the already-inserted `<compressed-chain>` synthetic, which is itself `role: "user"` - and any third-party `custom_message` entries survive untouched. Preserving the synthetic is what makes re-application idempotent: calling `applyChainCompressions` a second time with the same entries sees its own synthetic already present (matched by `blockId`) and skips re-inserting it.
+
+**Per-batch summary suppression is coverage-based, not index-membership-based.** Under `batchingMode: "agent-message"` a per-batch summary is appended *after* `finalAssistantTimestamp`, i.e. outside the resolved range. Suppressing it by index membership would miss it entirely, so suppression instead checks whether the summary's own `toolCallRefs` overlap the set of ids actually dropped in this pass (`perBatchSummaryOverlapsDropped`).
+
+**Diagnostic cross-check.** Each accepted entry also carries `droppedToolCallIds` from detection time. At render time the ids actually inside the resolved range are compared against that recorded set purely as a health check - a mismatch never changes what gets dropped (the range always wins) but emits a `range-id-mismatch` diagnostic, surfacing drift between detection-time and render-time state without ever acting on it.
 
 ### Rolling window
 
@@ -955,6 +967,48 @@ The `context-prune-chain` session entry carries the matching `protectedToolCallI
 
 ---
 
+## Occurrence Identity
+
+Every lookup keyed on a tool call - the indexer's record map, dedup aliases, `isSummarized`, batch capture - needs a string that uniquely names one *occurrence* of a tool call for the lifetime of a session.
+
+### Why the provider `toolCallId` cannot be that string
+
+A provider's `toolCallId` (e.g. `bash_23`) is unique only within the single response that produced it. Some providers restart a `${tool}_${n}` counter, so the same bare id recurs across turns and across a session, denoting a different tool call each time. The indexer cannot treat it as session-durable identity by itself - see [Identification model](#identification-model-positional-ranges-not-ids) for how chain compression sidesteps the same problem by keying on position instead of id.
+
+### Why `ToolCallRecord.timestamp` cannot serve as the discriminant either
+
+`ToolCallRecord.timestamp` is the *batch's* timestamp, computed once per captured batch (`src/batch-capture.ts`) and shared by every tool call inside that batch - it cannot discriminate between two calls captured together, let alone across a reused id. It is also not computed the way this discriminant would need: the live `turn_end` path stamps it unconditionally with `Date.now()` (`index.ts`'s `turn_end` handler), not from the assistant message's own timestamp. This is exactly why a separate per-tool-call field was needed - `resultTimestamp`, sourced from the `ToolResultMessage`'s own `timestamp` instead of the batch. On reconstruction, `reconstructFromSession` replays whatever value was persisted on the record verbatim; it does not recompute it.
+
+### The key: `id@resultTimestamp`
+
+The one value read identically at capture time and at every later render or rescan is the `ToolResultMessage`'s own `timestamp` field, stamped once by the message itself. `src/occurrence-key.ts` combines it with the bare id:
+
+```
+occKey(toolCallId, resultTimestamp) = `${toolCallId}@${resultTimestamp}`
+```
+
+A record with no `resultTimestamp` is keyed by its bare id - the shape for any record that predates this field, not a separate code path.
+
+**Where the key lives:** `ToolCallRecord.resultTimestamp` and `SummaryToolCallRef.resultTimestamp` (both optional), the dedup-alias pair `newResultTimestamp` / `originalResultTimestamp`, and `ChainCompressionEntry.droppedOccurrenceKeys`. Bare-id and occurrence keys coexist in the same maps: `getRecord` / `isSummarized` compare keys as opaque strings, with no casting between the two shapes.
+
+**`isSummarized` is strict.** It does an exact lookup on the occurrence key (or dedup-alias map) and never falls back to the bare id on a miss. The sole sanctioned bare-id path is `hasLegacyBareRecord(toolCallId)`, true only when the bare id is indexed **and** has no occurrence-keyed siblings (`bareIdToKeys`). A bare id with mixed legacy and occurrence-keyed entries fails closed on both checks - a permissive fallback would stub or recover the wrong occurrence whenever an id was reused.
+
+**Spill sidecars** are named from the occurrence key, so a reused id spills to distinct files per occurrence. Recovery reads the `spillPath` persisted on the record rather than re-deriving it from the id, so bare-id-named sidecars still resolve.
+
+**Render-time (`src/pruner.ts` stub-replace) gates the bare-id fallback on `hasLegacyBareRecord`, not on "message has no timestamp".** The first lookup key is `occKey(msg.toolCallId, msg.timestamp)` when the message carries a `timestamp` (a live `ToolResultMessage` almost always does) or the bare id otherwise. If that first key misses `isSummarized`, the bare id is tried as a second rung - but **only** when `indexer.hasLegacyBareRecord(msg.toolCallId)` is true (bare id indexed, no occurrence-keyed siblings), regardless of whether the message itself carried a timestamp. Gating that second rung on "the message has no timestamp" instead would mean a pure-legacy session (every record captured before `resultTimestamp` existed, so every occurrence-key lookup misses) never stubs at all, since a timestamped message would never even attempt the bare-id rung. A mixed bare+occurrence id still fails closed on both lookups.
+
+### No migration, and the limitation that leaves
+
+A record persisted before `resultTimestamp` existed is bare-keyed and stays that way - `reconstructFromSession` does not scan the branch to retroactively assign it one. No migration runs on load; `hasLegacyBareRecord` is the only accommodation, and it is a derivation (single-key check on `bareIdToKeys`), not a rewrite.
+
+This leaves one accepted, documented exposure: within a session that spans the upgrade, a live tool result whose provider id collides with a pre-upgrade summarized record can be stub-replaced with that record's stale content - the render-time bare-id fallback in `src/pruner.ts` cannot tell the two apart once both denote the same bare `toolCallId` with no occurrence-keyed siblings. New records (both sides occurrence-keyed) are unaffected; the exposure is confined to the pre-upgrade half of a spanning session and disappears entirely once a session contains no legacy records.
+
+### Multi-occurrence recovery via `context_tree_query`
+
+Short refs (`tN`) always resolve 1:1 - one ref, one record - and are the primary, recommended recovery path. A raw provider `toolCallId` passed to `context_tree_query` is looked up against every record that shares it via `getRecordsForId` (`src/indexer.ts`): if the id was reused, the tool returns **every** matching occurrence instead of silently picking one, one block each, chronologically, each labelled `id@timestamp`. This includes occurrences that were content-deduplicated to an earlier record: `getRecordsForId` also scans the dedup-alias map for aliases sharing the queried bare id and returns each aliased occurrence too, labelled with **its own** occurrence timestamp rather than the original's, so a reader can tell the two apart. A record with no `resultTimestamp` caught up in such a set is labelled with the bare id plus an explicit `Occurrence: legacy (no resultTimestamp)` line.
+
+---
+
 ## Error Purge
 
 Failed tool calls often embed large argument bodies in the assistant message — a `write` call with a 30 KB file body, an `edit` call with a multiline diff. The error result is small (e.g. `"Error: file not found"`), but the original `arguments` stay in the assistant turn indefinitely.
@@ -981,7 +1035,7 @@ Error purge replaces those arg bodies with compact stubs after the error has coo
 **Transform position:** Error purge runs in Phase 2, after stub-replace and before chain range prune.
 
 ```
-[stub-replace] → [error-purge] → [chain-range-prune] → [thinking-strip]
+[stub-replace] → [error-purge] → [chain-range-prune] → [orphan-sweep]
 ```
 
 **Config keys:**
@@ -994,49 +1048,37 @@ Error purge replaces those arg bodies with compact stubs after the error has coo
 
 ---
 
-## Main-loop Thinking Strip
-
-Chain compression and the summarizer target *tool* mass. But in long single-agent sessions the dominant cost is often **assistant `thinking` blocks**: on Opus 4.5+/Sonnet 4.6+ the API retains every prior-turn thinking block by default, and pi-ai replays them all (with signatures) on every request. One autonomous ops session held ~405 K tokens (~80% of a 500 K window) in thinking alone, untouched by every other strategy — chain compression only fires on *closed* spans, and that session was one long open span.
-
-Thinking strip is a deterministic, zero-LLM transform (Phase 4) that keeps `thinking` blocks only on the last `keepLastTurns` **assistant turns** and strips them from older assistant messages, leaving each message's `text` and `toolCall` blocks intact.
-
-### Turn unit
+## Orphan Sweep
 
-`keepLastTurns` counts **assistant messages**, not user-bounded spans. The target failure mode is a single long open chain (zero subagents, near-zero user turns) where a span-based window would keep everything. Counting assistant turns directly bounds thinking accumulation regardless of whether any chain closes.
+`pruneMessages` ends with an unconditional structural pass, `sweepOrphanToolResults` (`src/orphan-sweep.ts`): a `toolResult` message whose id was not opened by the immediately preceding assistant turn is removed. It runs after every other phase, over whatever the previous three produced, as a final post-condition rather than a targeted fix for one code path.
 
-### Provider safety
+**Why it exists.** pi-ai's auto-repair (`insertSyntheticToolResults`) only fills in a missing `toolResult` for an orphaned `toolCall` - it has no equivalent repair for the opposite shape, an orphaned `toolResult` with no matching `toolCall`. Providers reject that shape outright (Anthropic: `unexpected tool_use_id found in tool_result blocks`; Kimi K3: a tool message needs a resolvable preceding tool_call). An orphaned `toolResult` can appear from any combination of id reuse, an unresolved chain range, or a bug in an upstream phase; the sweep is the last line of defense regardless of cause, and it is intentionally not keyed to any provider's specific error string - the fix is structural ("does this id have an open call"), not reactive to how one provider happens to phrase rejection.
 
-Anthropic's extended-thinking contract during tool use:
+**Per-turn, not cumulative, open-call tracking.** The sweep does one forward pass over the message array. Each `assistant` message **replaces** the current "open" id set with its own `toolCall` ids; each `toolResult` message either consumes (removes) a matching id from that set or, if its id is not open, is swept. A cumulative seen-set across the whole array would be wrong here: it would let an id used validly by an early turn license a *later* genuine orphan under the same reused id - exactly the collision scenario this exists to catch. Per-turn tracking means only the immediately preceding assistant turn can vouch for a `toolResult`'s id.
 
-- Only the **last assistant turn's** thinking is required; "you can omit thinking blocks from prior assistant role turns" and the API auto-filters them.
-- A message's thinking blocks must be dropped **all-or-nothing** ("the entire sequence of consecutive thinking blocks must match the outputs … you can't rearrange or modify the sequence"). The strip reuses `withoutThinkingBlocks`, which removes every thinking block (and its signature) from a message.
+**Provider-agnostic and reference-preserving.** The sweep has no knowledge of which provider is in play; it operates purely on message shape. When nothing is swept it returns the **same array reference** it was given, so a clean render is a true no-op and the no-op / prompt-cache-prefix invariant of `doc/specs/2026-08-04-pruner-noop-serialization.md` holds - the sweep never forces a cache bust on a session where nothing is actually orphaned.
 
-`keepLastTurns` is clamped to `>= 1`, so the most-recent assistant turn — the one that may be awaiting tool results — always keeps its thinking. This is the minimum safe window; the default of 16 is far above the floor and preserves recent reasoning continuity.
+Every swept id is reported once (hashed batch, not per-id) as an `orphan-sweep` diagnostic.
 
-### Transform position
-
-Thinking strip runs **last**, after chain-range-prune, at render time:
-
-```
-[stub-replace] → [error-purge] → [chain-range-prune] → [thinking-strip]
-```
+---
 
-The keep-window is a flush-computed assistant-message timestamp (the `keepLastTurns`-back boundary over the **raw** session branch — see the **Cache impact** note below), not a render-time recount over the post-phase-3 survivors. So when chain compression drops closed middle turns inside the window, fewer than `keepLastTurns` *surviving* turns may retain thinking — deliberate, and it only ever strips more, never re-adds. In a session with no closed chains, Phases 1–3 may be no-ops and thinking strip does all the work. Where chain compression *does* fire, the two cooperate: chain compression drops whole old middle turns (including their thinking); thinking strip mops up thinking in the surviving recent / in-flight turns beyond the boundary.
+## Diagnostics
 
-### Cache impact
+Prune-time degradations - an unresolvable chain range, a detection/render id mismatch, a swept orphan - are recorded on an out-of-band channel instead of surfacing in the model's context. `DiagnosticSink` (`src/diagnostics.ts`) writes a `context-prune-diagnostic` session entry (`{ kind, detail }`) for each of three kinds:
 
-pi-ai serializes prompt-cache breakpoints only at `tools`, `system`, and the last conversation message (verified in `@earendil-works/pi-ai` `api/anthropic-messages.js` `convertMessages`) - there is **no in-history breakpoint**. So the strip boundary is flush-gated, not per-render: it is a persisted assistant-message timestamp on the `context-prune-frontier` entry that stays fixed between flushes and advances only on a non-empty flush. Between flushes every render is byte-stable in its historical prefix, so the cache holds through a whole tool loop; the boundary moves at most once per flush (~once per request), turning the old k-busts-per-render into ~1-per-request. That residual bust is not always free: for a request shorter than `keepLastTurns` turns the boundary (tail-K) sits deeper than the current request's just-summarized tool results, so thinking-strip is the dominant invalidator, ~1 deep reprocess per request; for a request longer than `keepLastTurns` turns summarization's stub-replace reaches deeper and subsumes it (~0 marginal). Retained thinking is bounded to `keepLastTurns` raw-session turns, drifting up to `keepLastTurns + turns-since-flush` between flushes (deliberate: the frozen boundary is what buys cache stability). Note `error-purge` still mutates old history off a live per-render count, an independent cache-bust source not addressed here.
+| Kind | Emitted from | Meaning |
+|---|---|---|
+| `unresolved-range` | `applyChainCompressions` | A persisted chain entry's boundaries didn't resolve to a unique range (or the range was rejected as nested/duplicate) - the entry compressed nothing |
+| `range-id-mismatch` | `applyChainCompressions` | The ids actually inside a resolved range don't match the entry's recorded `droppedToolCallIds` - informational only, the range still wins |
+| `orphan-sweep` | `pruneMessages` (Phase 4) | One or more `toolResult` messages were removed for having no open matching `toolCall` |
 
-### Recovery
+**Never in LLM context.** These are session entries only - zero tokens added, zero cache-prefix change, never read back into the message array the model sees.
 
-Stripped thinking is **not** recoverable via `context_tree_query` — unlike tool outputs, thinking blocks are not indexed. The raw thinking remains in the session JSONL on disk (the `context` hook never mutates storage); reloading the session without the extension, or reading the file directly, shows the original blocks. Thinking is transient model-internal reasoning, so drop-without-recovery is intentional.
+**Deduped per `(kind, dedupKey)`**, so a permanently degraded condition (e.g. the same chain entry failing to resolve on every render) writes one entry, not one per render. The dedup key is entry-specific: a chain `blockId` for `range-id-mismatch` and for a genuinely unresolvable `unresolved-range`; `overlap:<blockId>` for the same `unresolved-range` kind when the entry was instead skipped as nested inside or duplicating another range's start (a distinct dedup-key prefix keeps the two cases greppable under one kind); and a short hash of the sorted swept-id list for `orphan-sweep` (hashing rather than joining the raw list avoids `DiagnosticSink`'s dedup set retaining an ever-longer key per additional orphan over a session's lifetime). A write is only marked seen after the `appendEntry` call succeeds, so a failed write is retried on the next render instead of being silently dropped.
 
-### Config keys
+**Reset on `session_start` and `session_tree`**, matching every other in-memory, non-persisted piece of prune state.
 
-| Key | Default | Description |
-|---|---|---|
-| `thinkingStrip.enabled` | `true` | Master toggle (gated behind the top-level `enabled`) |
-| `thinkingStrip.keepLastTurns` | `16` | Keep thinking on the last N assistant turns; strip older. Clamped to `>= 1` |
+**Surfaced on the status line.** The footer status widget (`setPruneStatusWidget`, gated by `showPruneStatusLine`) appends a self-hiding ` · diag u<N>/m<N>/o<N>` segment (u = `unresolved-range`, m = `range-id-mismatch`, o = `orphan-sweep`) built from the sink's live counters via `pruneStatusText`. Each letter is omitted when its counter is zero, and the whole segment is absent when all three are zero. `/pruner status` (the slash command) prints a separate settings/stats block and does not include this segment.
 
 ---
 

package/README.md

@@ -80,12 +80,17 @@ Every summarizer cost update is emitted on the shared `pi.events` channel `cost:
 | Term | Meaning |
 |---|---|
 | Stub | The short breadcrumb (`[Summarized in pruner summary, ref \`t1\`...]`) that replaces a pruned tool result in context |
-| `context_tree_query` | The tool the model calls to recover a stubbed original by ref |
+| `context_tree_query` | The tool the model calls to recover a stubbed original by ref (`tN`) or `toolCallId`. A reused id returns every matching occurrence, not just one, including any that were content-deduplicated to an earlier record - see [PRUNING.md § Occurrence Identity](PRUNING.md#occurrence-identity) |
 | Batch vs chain | A batch is one flush's worth of tool calls; a chain is a longer closed sequence eligible for range compression |
 | Prune frontier | The last attempted prune boundary - advances even on a skip, so nothing is reconsidered twice |
+| Diagnostics (`diag u/m/o`) | A self-hiding status-line segment surfacing prune-time degradations: `u` = unresolved chain range, `m` = detection/render id mismatch (informational, does not change what's dropped), `o` = orphan tool-result sweep. Each letter's count is omitted when zero; the whole segment disappears when all three are zero. Backing session entries are `context-prune-diagnostic` - see below |
 | Prompt-cache interaction | Why batching (not per-turn pruning) is the default - see [PRUNING.md](PRUNING.md#how-prefix-caching-works) |
 | `cost:external` | The shared cost-reporting channel pi-condense emits on (see above) |
 
+### Diagnostic entries (`context-prune-diagnostic`)
+
+The status-line `diag u<N>/m<N>/o<N>` segment above is backed by `context-prune-diagnostic` session entries - session-log-only, never added to what the model sees. Full mechanics: [PRUNING.md § Diagnostics](PRUNING.md#diagnostics).
+
 ## When to use / when NOT to use
 
 **Use it for:** long coding or research sessions where tool output dominates the prompt; setups deliberately running a smaller/cheaper driver model; pi-cohort fan-outs or pi-gauntlet runs where cost compounds across many turns or many children.

package/index.ts

@@ -5,7 +5,7 @@
  *   config       — load/save <agent-dir>/settings.json `contextPrune` namespace (honors PI_CODING_AGENT_DIR)
  *   batch-capture — serialize turn_end event into CapturedBatch
  *   summarizer   — call LLM to summarize a CapturedBatch
- *   indexer      — maintain Map<toolCallId, ToolCallRecord> + session persistence
+ *   indexer      — maintain Map<occurrenceKey, ToolCallRecord> + session persistence
  *   pruner       — filter context event messages
  *   query-tool   — register context_tree_query tool
  *   commands     — register /pruner command + message renderer
@@ -36,10 +36,11 @@ import { PruneFrontierTracker } from "./src/frontier.js";
 import { BlockRefIssuer } from "./src/block-refs.js";
 import { compressEligible } from "./src/chain-compressor.js";
 import { detectChains, withClosingMessage } from "./src/chain-detector.js";
-import { computeThinkingBoundary } from "./src/thinking-strip.js";
 import { inGraceRecoveryToolCallIds } from "./src/recovery-grace.js";
 import { shouldBudgetFlush, shouldDeltaFlush, usageFraction } from "./src/budget.js";
 import { spillOversizedBatch } from "./src/spill.js";
+import { occKey } from "./src/occurrence-key.js";
+import { DiagnosticSink } from "./src/diagnostics.js";
 
 export default function (pi: ExtensionAPI) {
   // Shared mutable config reference — updated by /pruner commands
@@ -65,6 +66,10 @@ export default function (pi: ExtensionAPI) {
   // rebuilt from session on session_start / session_tree
   const blockRefs = new BlockRefIssuer();
 
+  // Session-scoped diagnostic sink — tracks recovery-path anomaly counters
+  // (dedup'd across the session's lifetime, not per-render).
+  const diagnostics = new DiagnosticSink((type, data) => pi.appendEntry(type, data));
+
   // Pending batches — accumulated until the prune trigger fires
   const pendingBatches: CapturedBatch[] = [];
   let isFlushing = false;
@@ -104,7 +109,7 @@ export default function (pi: ExtensionAPI) {
     let toolCalls = batch.toolCalls;
 
     // The indexer tells us what was successfully summarized earlier.
-    toolCalls = toolCalls.filter((tc) => !indexer.isSummarized(tc.toolCallId));
+    toolCalls = toolCalls.filter((tc) => !indexer.isSummarized(occKey(tc.toolCallId, tc.resultTimestamp)));
     if (toolCalls.length === 0) return null;
 
     // The frontier tells us the last attempted boundary even when the attempt did
@@ -234,8 +239,9 @@ export default function (pi: ExtensionAPI) {
           const remaining: typeof batch.toolCalls = [];
           for (const tc of batch.toolCalls) {
             const originalId = indexer.lookupByContent(tc.toolName, tc.resultText);
-            if (originalId && originalId !== tc.toolCallId) {
-              indexer.registerDuplicate(tc.toolCallId, originalId, persistAlias);
+            const key = occKey(tc.toolCallId, tc.resultTimestamp);
+            if (originalId && originalId !== key) {
+              indexer.registerDuplicate(key, originalId, persistAlias);
               dedupedPerBatch[i].toolCalls.push(tc);
               dedupedPerBatch[i].rawChars += tc.resultText.length;
             } else {
@@ -414,7 +420,7 @@ export default function (pi: ExtensionAPI) {
             // `display: false` keeps the summary in future LLM context (convertToLlm
             // ignores `display`) while suppressing the full markdown block from Pi's
             // main window; rebuild keys on customType, not display.
-            const batchToolCallIds = batch.toolCalls.map((tc) => tc.toolCallId);
+            const batchOccurrenceKeys = batch.toolCalls.map((tc) => occKey(tc.toolCallId, tc.resultTimestamp));
             if (delivery === "runtime") {
               pi.sendMessage(
                 { customType: CUSTOM_TYPE_SUMMARY, content: summaryText, display: false, details: batchDetails },
@@ -429,7 +435,7 @@ export default function (pi: ExtensionAPI) {
             }
             // Keep the in-memory summary-body registry current so chain compression
             // can build synthetic chain messages without rescanning session entries.
-            indexer.registerSummaryBody(batchToolCallIds, summaryText);
+            indexer.registerSummaryBody(batchOccurrenceKeys, summaryText);
           } else {
             oversizedBatches.push(batch);
           }
@@ -453,7 +459,7 @@ export default function (pi: ExtensionAPI) {
 
       if (processedBatches.length === 0) {
         // Nothing was persisted (all calls failed or first call failed)
-        setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim());
+        setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim(), diagnostics.counts());
         return { ok: false, reason: "summarizer-failed" };
       }
 
@@ -487,34 +493,15 @@ export default function (pi: ExtensionAPI) {
               ? "skipped-deduped"
               : "skipped-trivial";
 
-      // Raw session branch, unwrapped once and shared by the thinking-strip boundary
-      // computation and the chain-compression block below - both walk it, so avoid a
-      // second O(session-size) pass on every flush. Only materialized when at least
-      // one consumer is enabled.
+      // Raw session branch, unwrapped once for the chain-compression block below.
+      // Only materialized when chain compression is enabled.
       let branchMessages: any[] | undefined;
-      if (currentConfig.value.thinkingStrip.enabled || currentConfig.value.chainCompression.enabled) {
+      if (currentConfig.value.chainCompression.enabled) {
         branchMessages = ctx.sessionManager.getBranch()
           .filter((e: any) => e.type === "message" && e.message)
           .map((e: any) => e.message);
       }
 
-      // Flush-gated thinking-strip boundary: recompute the (count - keepLastTurns)-th
-      // assistant timestamp over the RAW branch (+ the not-yet-persisted closing
-      // assistant), monotonically clamped. Stays on the frontier snapshot so renders
-      // between flushes read a fixed value and keep the cache prefix. Carries prev
-      // through when disabled. Must run regardless of chainCompression.enabled.
-      let thinkingBoundary = frontier.get()?.thinkingStripBoundaryTimestamp;
-      if (currentConfig.value.thinkingStrip.enabled) {
-        const assistantTimestamps = withClosingMessage(branchMessages!, options.closingMessage)
-          .filter((m: any) => m?.role === "assistant" && typeof m.timestamp === "number")
-          .map((m: any) => m.timestamp);
-        thinkingBoundary = computeThinkingBoundary(
-          assistantTimestamps,
-          currentConfig.value.thinkingStrip.keepLastTurns,
-          thinkingBoundary,
-        );
-      }
-
       const frontierSnapshot: PruneFrontier = {
         lastAttemptedToolCallId: lastTC.toolCallId,
         lastAttemptedToolName: lastTC.toolName,
@@ -525,7 +512,6 @@ export default function (pi: ExtensionAPI) {
         rawCharCount: totalRawCharCount,
         summaryCharCount: totalSummaryCharCount,
         outcome: flushOutcome,
-        thinkingStripBoundaryTimestamp: thinkingBoundary,
       };
 
       try {
@@ -546,7 +532,7 @@ export default function (pi: ExtensionAPI) {
         return { ok: false, reason: isStaleContextError(err) ? "stale-context" : "failed", error: errorMessage(err) };
       }
 
-      setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim());
+      setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim(), diagnostics.counts());
       emitExternalCost(pi, statsAccum);
 
       // Chain compression — compress closed chains beyond the rolling window.
@@ -556,7 +542,7 @@ export default function (pi: ExtensionAPI) {
         try {
           // message_end fires before pi persists the closing assistant, so thread it
           // in here; otherwise the newest chain reads as open and K over-retains by 1.
-          // branchMessages was unwrapped once above (shared with the boundary block).
+          // branchMessages was unwrapped once above, gated on chainCompression.enabled.
           const chains = detectChains(withClosingMessage(branchMessages!, options.closingMessage), protectionPredicate);
           const inGrace = inGraceRecoveryToolCallIds(branchMessages!, currentConfig.value.recoveryGraceTurns);
           const { compressedEntries } = await compressEligible(
@@ -657,7 +643,7 @@ export default function (pi: ExtensionAPI) {
       // When the abort signal fired, summarizeBatch rethrows rather than
       // swallowing the error.  Don't show a UI error — the user intended this.
       if (options.signal?.aborted) {
-        setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim());
+        setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim(), diagnostics.counts());
         return { ok: false, reason: "aborted" };
       }
       if (isStaleContextError(err)) {
@@ -684,6 +670,7 @@ export default function (pi: ExtensionAPI) {
     // Rebuild stats accumulator from persisted session entries
     statsAccum.reconstructFromSession(ctx);
     fallbackController.reset();
+    diagnostics.reset();
 
     // Rebuild prune frontier from persisted session entries
     frontier.reconstructFromSession(ctx);
@@ -693,7 +680,7 @@ export default function (pi: ExtensionAPI) {
     previousFraction = null;
 
     // Update footer status
-    setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim());
+    setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim(), diagnostics.counts());
 
     ctx.ui.setWidget(
       "pruner-boot",
@@ -716,6 +703,7 @@ export default function (pi: ExtensionAPI) {
     indexer.reconstructFromSession(ctx);
     blockRefs.rebuildFrom(indexer.getChainEntries().map((e) => e.blockId));
     statsAccum.reconstructFromSession(ctx);
+    diagnostics.reset();
     frontier.reconstructFromSession(ctx);
     // Pending batches belong to the old branch — discard them
     pendingBatches.length = 0;
@@ -847,25 +835,23 @@ export default function (pi: ExtensionAPI) {
     // pruneMessages is the single source of truth for "is there work to do".
     // It returns the original array reference (pruned: false) only when none of
     // the four phases changed anything; index/registry emptiness alone does not
-    // imply a no-op, since error-purge (phase 2) and thinking-strip (phase 4)
-    // prune independently of them. Calling it unconditionally is safe and avoids
-    // a split gate here.
+    // imply a no-op, since error-purge (phase 2) prunes independently of them.
+    // Calling it unconditionally is safe and avoids a split gate here.
     const result = pruneMessages(
       messages,
       indexer,
       currentConfig.value.chainCompression,
       currentConfig.value.purgeErrors,
-      currentConfig.value.thinkingStrip,
       currentConfig.value,
       currentConfig.value.recoveryGraceTurns,
-      frontier.get()?.thinkingStripBoundaryTimestamp,
+      diagnostics,
     );
     if (result.pruned) {
       messages = result.messages;
       changed = true;
       statsAccum.setLiveReclaim(result.beforeChars, result.afterChars);
     }
-    setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim());
+    setPruneStatusWidget(ctx, currentConfig.value, statsAccum.getLiveReclaim(), diagnostics.counts());
 
     if (!changed) return undefined;
     return { messages };
@@ -902,5 +888,5 @@ export default function (pi: ExtensionAPI) {
     return { compressedEntries: result.compressedEntries, skipped: result.skipped.filter((s) => s.reason === "no-summary").length };
   };
 
-  registerCommands(pi, currentConfig, flushPending, capturePendingBatches, () => statsAccum.getStats(), () => statsAccum.getLiveReclaim(), indexer, compactChains);
+  registerCommands(pi, currentConfig, flushPending, capturePendingBatches, () => statsAccum.getStats(), () => statsAccum.getLiveReclaim(), indexer, compactChains, () => diagnostics.counts());
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-condense",
-  "version": "2.4.3",
+  "version": "2.6.0",
   "description": "Pi coding-agent extension that summarizes completed tool-call batches, replaces raw outputs with short stubs, compresses closed tool-call chains, and recovers any original on demand via context_tree_query.",
   "author": "Jacek Juraszek",
   "license": "MIT",