pi-context-guard-jev 0.1.0 → 0.3.1

package/docs/DESIGN.md

@@ -9,26 +9,35 @@ evidence behind the thresholds are in [JEV.md](JEV.md). Cache behaviour is in [C
 |---|---|
 | `src/index.ts` | The only module that uses the Pi API: hooks, `recall` tool, `/guard` command, status bar, classifier lookup |
 | `src/config.ts` | `DEFAULT_CONFIG`, validation (`normalizeConfig`), load/save of `context-guard.json`, `provider/id` parsing |
-| `src/run.ts` | `collectRun`: finds the run that just finished, its question, answer, notes and candidate tool results |
+| `src/run.ts` | `collectRun`: finds the run that just finished, its question, answer, notes and candidate tool results; the shared span and candidate rules |
+| `src/checkpoint.ts` | Mid-run checkpoints: eligible outputs (`collectCheckpoint`), superseded detection, the memo and pending edits from the branch, the read point, the break-even rule |
 | `src/chunk.ts` | `chunkOutput` (grep-aware chunking) and `segmentChunks` (grouping chunks into Jev requests) |
-| `src/decide.ts` | `buildRequest` (the Jev request) and `interpret` (answers → keep set); `citedFiles` |
+| `src/decide.ts` | `buildRequest` (the run-end Jev request), `buildCheckpointRequest` (the mid-run one) and `interpret` (answers → keep set); `citedFiles` |
 | `src/distill.ts` | `distillRun`: chunk, run Jev requests in parallel within a time budget, decide per result, build `context_edit` drafts |
+| `src/items.ts` | Whole items: small outputs (`collectSmall`) and old exchanges (`collectExchanges`), their Jev requests, stubs and edits, and `judgeItems` (batched, parallel, time budget) |
+| `src/process.ts` | `processItems`: large outputs, small outputs and old exchanges of one pass, judged in parallel and merged into one list of edits and records; `addImages` adds the image removals |
+| `src/images.ts` | Old images: `collectImages` (tool results with images at least `imageKeepTurns` old), their stub and replacement |
+| `src/size.ts` | Context size in text characters: images by pixels (width × height ÷ 750 tokens × 4), thinking signatures at half, image headers (PNG/JPEG/GIF/WebP) |
 | `src/render.ts` | The replacement text: header, verbatim chunks, omission lines; the `[context-guard]` marker |
-| `src/cache-pin.ts` | Anthropic breakpoint at the previous user question |
+| `src/cache-pin.ts` | Anthropic breakpoints: the previous user question, and the read point after edits (`placeGuardBreakpoints`) |
 | `src/stats.ts` | Savings and Jev usage computed from the session; status-bar text |
-| `src/recall.ts` | The text returned by `recall` |
+| `src/recall.ts` | What `recall` returns (a tool output with its images, or the transcript of an omitted exchange with its user images) |
 | `src/types.ts` | Loose structural types for the Pi objects used, so the pure modules can be tested without Pi |
 
 Everything except `index.ts` is pure. `distillRun` gets the classifier as an injected function.
 
-## The boundary: `agent_before_settle`
+## The boundaries: `agent_before_settle` and `turn_end`
 
-The work happens in `agent_before_settle`, and only when `event.outcome === "completed"` and the guard is on.
+Run-end distillation happens in `agent_before_settle`, and only when `event.outcome === "completed"` and
+the guard is on. Long runs also get mid-run checkpoints at `turn_end` (see
+[Mid-run checkpoints](#mid-run-checkpoints)); everything below about drafts applies to both.
 
 Why there:
 
 - The final answer exists, so relevance can be judged against it. Pruning during a run would take raw
   data away from the model while it still needs it, and every edit would break the prompt cache again.
+  Mid-run checkpoints accept those costs only for outputs that are several turns old, in rare batches,
+  and only when the break-even rule says they pay off.
 - A boundary handler can return session entry drafts. A `context_edit` draft is append-only: it changes
   only what the model sees from then on. The raw entry stays in the session file, the UI and exports, and
   `ctx.sessionManager.getEntry(id)` still returns it. That is what `recall` reads.
@@ -45,18 +54,24 @@ Hooks used:
 | Hook | Use |
 |---|---|
 | `session_start` | Load settings, check the classifier model and credential, draw the status |
-| `agent_before_settle` | Distill the finished run |
-| `before_provider_request` | Pin the Anthropic cache breakpoint (only for `anthropic-messages` models) |
-| `session_tree`, `session_compact`, `agent_settled` | Redraw the status |
+| `turn_end` | Mid-run checkpoint, when a batch is due |
+| `agent_before_settle` | Distill the finished run (skipping outputs judged mid-run) |
+| `before_provider_request` | Anthropic breakpoints (only for `anthropic-messages` models) |
+| `session_tree`, `session_compact` | Redraw the status (checkpoint state is read from the branch anyway) |
+| `agent_settled` | Redraw the status |
 
 ## Algorithm
 
 ### 1. Run span
 
-`collectRun` walks back through `event.context.contextEntries` (Pi's projected entries) to the last entry
-that has a user message, with or without text. Its text is the **question**; a prompt with only an image
-gets the placeholder `[the user sent only an image]`, so an older question is never reused. Everything
-after it is the run. Assistant
+`collectRun` walks back through `event.context.contextEntries` (Pi's projected entries) to the user
+message that started the run (`findRunStart`), with or without text. **Steering messages** (user messages
+sent while the agent was working: the message before them, custom notices aside, is a tool result or an
+assistant message with tool calls) do not start a run; the walk goes on to the prompt before them. The
+**question** is that prompt's text followed by every steering message of the run, each prefixed with
+`[The user added during the run]`; a message with only an image gets the placeholder
+`[the user sent only an image]`, so an older question is never reused. Everything after the prompt is the
+run. Assistant
 text in the run gives the **answer** (the last text) and the **notes** (all earlier text). Tool calls are
 indexed by id so each result can be matched with its tool name and arguments.
 
@@ -66,7 +81,7 @@ default 3; `0` turns all of it off):
 - `summary`: the text (`summary` field) of the latest `compactionSummary` or `branchSummary` message, at
   most 2,000 characters.
 - `exchanges`: the last N earlier exchanges, oldest first. Every user message starts one (steering messages
-  included; an image-only prompt shows as `[the user sent only an image]`). Each holds the prompt (at most
+  of earlier runs included; an image-only prompt shows as `[the user sent only an image]`). Each holds the prompt (at most
   800 characters) and the **last** assistant text before the next user message (at most 1,200). Tool calls
   and tool results are left out.
 - `firstRequest`: the first user prompt in the projection (at most 1,000 characters), only when it is not
@@ -186,6 +201,184 @@ In this order:
   usually already cleared at this boundary, so Esc does not cancel it; the time budget is the limit.
 - An exception from the whole pass shows a warning and leaves the run untouched.
 
+## Items: what one pass judges
+
+Every pass (run end, or a mid-run checkpoint) collects three kinds of item and judges them in parallel
+(`processItems`); their edits and results go into one record.
+
+| Kind | What | How it is judged | Dropped item becomes |
+|---|---|---|---|
+| `large` | Tool results of the current run of at least `minResultChars` (4,000), under the candidate rules above | Chunked, one request per segment (above) | The kept chunks verbatim with omission lines |
+| `small` | Tool results of the current run from `smallResultMinChars` (400) up to `minResultChars`, under the same rules | Whole: one bool per item, many items per request | A one-line stub: `[context-guard] Omitted the output of bash \`…\` (1.2k chars): judged no longer needed. Full output: recall({"entryId":"…"}).` |
+| `exchange` | Old exchanges (below) | Whole: one bool per exchange, several per request | A stub on the prompt, every other entry omitted |
+
+**Small outputs** (`collectSmall`). An output whose stub would not be shorter is skipped. Each item carries
+the tool label, `status: failed` for error results, `superseded` when a later call made it out of date
+(`supersededBy`, as for large outputs mid-run), and the output (at most 4,000 characters). The question is
+"will the agent still need the output in item_N?" mid-run, or "does item_N contain anything that
+final_answer relies on, or that a likely follow-up would need?" at run end; both say that an output marked
+superseded is out of date. An item is kept when P ≥ `smallKeepThreshold` (0.45; probe in
+[JEV.md](JEV.md#small-outputs)). Mid-run, small outputs follow the same age rule and memo as large ones; at
+run end every small output without the marker is judged, including those kept at a checkpoint.
+
+**Old exchanges** (`collectExchanges`). An exchange is a user prompt and every entry up to the next one:
+assistant messages, tool calls and results, custom messages such as subagent notices. Eligible are
+exchanges before the current run's user message, without the last `keepRecentExchanges` (2) before it,
+that completed (their last assistant message has no tool calls; a steering message belongs to the exchange
+it was sent in), that are not already omitted (their prompt shows the stub), that were not judged during the
+current run (`exchangeMemo`: exchange results in records after the run's prompt, except deferred ones), and that can be recovered and are worth it: no entry whose model-visible
+content differs from the raw entry for any reason other than our own marker (images are fine: `recall` returns
+them) (another extension's edit,
+or an omission by someone else, which Pi keeps in the projection as an entry with no messages), at least
+2,000 characters of editable content, and a stub shorter than that content. A kept exchange is
+judged again in a later run, against that run's work. Compaction and branch summaries are never part of an
+exchange; entries Pi cannot edit (system messages, `!` shell executions) stay in place.
+
+Jev sees the current work (`user_question`, `earlier_conversation`, and `agent_progress` mid-run or
+`final_answer` at run end) and, per exchange, the prompt (at most 600 characters), the last assistant text
+(800), the tool calls as `toolLabel()`s (at most 20) and the size. The question: "Is exchange_N still
+relevant to the current work? Relevant means the agent may need its details: the same files, task,
+decisions or facts." An exchange is omitted only when P < `exchangeOmitThreshold` (0.2; probe in
+[JEV.md](JEV.md#old-exchanges)). Then the prompt entry gets the stub
+`[context-guard] Omitted an earlier exchange judged unrelated to the current work: "<first 120 characters of the prompt>" (N messages, ~Mk tokens). Full exchange: recall({"entryId":"<prompt entry id>"}).`
+and every other editable entry gets `replacement: null`, which omits it from model context. The record's
+result for the exchange lists exactly those entry ids (`omitted`), and `recall` on the stub shows exactly
+the prompt and those entries. Tool calls and
+their results are omitted together. The resulting shapes (a stub user message right before the next user
+message; spans without assistant and tool-result messages) are accepted by Anthropic (with
+`pi-claude-auth`) and OpenAI Codex, checked live with `test/e2e/omit-probe.ts` and the `topics` scenario.
+
+**Run-end break-even for exchanges** (`exchangeBreakEven`, on). After the pass is judged, its exchange
+omissions are checked: their marginal cache cost is the rewrite with them minus the rewrite the pass's
+other edits cause anyway (`rewriteCharsFor`, with the trusted read point and floor rules of
+[CACHE.md](CACHE.md); on OpenAI Codex any edit costs the whole context, so exchanges ride along for free
+when the pass edits anything else). If that cost is above zero, the omissions must save at least
+`exchangeMinSavingChars` (8,000) and pass `paysOff` with as many later requests as the session has had so
+far (model-visible assistant messages). Otherwise they are deferred: their edits are dropped, their results
+say `kept` / `deferred`, and they are judged again at later run ends against the work of that run, so
+deferred exchanges accumulate until their total pays off. `exchangeBreakEven: false` always omits
+unrelated exchanges.
+
+**Old images** (`images.ts → collectImages`, `imageKeepTurns`, 3). Every tool result in the projection,
+earlier runs included, whose model-visible content has images and that at least `imageKeepTurns` assistant
+messages follow. Skipped: user messages (the images you send are usually the spec), results another extension
+changed (text or image count differs from the raw entry), and results whose images are already gone. No Jev
+request: Jev reads text only, and a model reacts to an image in its next message, whose text stays. The
+replacement is a text-only content: the stub
+`[context-guard] Removed 1 image from this output (1280×960; ~1.6k tokens), 4 turns old. recall({"entryId":"…"}) shows it again.`
+followed by the result's own text blocks. Because the stub leads, the result counts as ours (`editedByOthers`
+is false, stats count it), and as already distilled (its text is not judged again). Its size before and after
+uses `size.ts` (an image is width × height ÷ 750 tokens, 4 characters per token; header unreadable: 1,600
+tokens; at most 6,700). Records have `kind: "image"`, `requests: 0`, reason `N turns old`. When Jev is
+unavailable, a pass still removes images. At run end, images from before the run's prompt go through the
+same marginal-cost gate as exchanges (`exchangeBreakEven`, with no minimum saving); deferred ones are just
+collected again next time. Mid-run, images count towards the batch; the tiers are: everything; without
+exchanges; without exchanges and images from before the last user message.
+
+**Batching.** Whole items are grouped into requests of at most `maxSegmentChars` characters of item text and
+`maxChunksPerSegment` (40) items, under the same `timeoutMs` budget; the three kinds run at the same time,
+and one shared limiter keeps at most `concurrency` (6) Jev requests in flight across all of them. An item without an
+answer (error, timeout) is kept and asked again next time.
+
+## Mid-run checkpoints
+
+A 30–120 minute autonomous run would otherwise carry every tool output until it ends; Pi compacts only
+near the window minus 16k tokens. Checkpoints judge older outputs while the run is still going.
+
+**Hook (verified on Pi 1.0.3).** `turn_end` is an actionable boundary (`TurnEndEvent extends
+BoundaryState`). `agent-session.js` dispatches it from `agent.finishTurn`, commits the returned drafts
+(`_commitBoundaryDrafts` → `_refreshFinalizedContext`), and only then runs `prepareNextTurn`, which rebuilds
+the messages from the session projection and runs the turn-end compaction check
+(`_compactBeforeNextAssistantResponse`). So a `context_edit` returned at `turn_end` reaches the very next
+provider request, before any compaction decision; the e2e runs confirm it (the next request's context
+drops by the removed amount). The runner replaces the draft list as for `agent_before_settle`, so the
+handler returns `[...event.entries, ...edits, record]`, and it never asks for `continue`.
+
+**When.** At the end of a turn whose assistant message has tool calls (the final turn of a run is left to
+run-end distillation), with `outcome: "completed"` and `midRun` on:
+
+1. **Eligible items:** large outputs (`collectCheckpoint`) and small outputs (`collectSmall`) of the current
+   run whose call was made at least `midRunMinAgeTurns` (4) turns before the current turn (turn = assistant
+   message index in the run) and that are not in the memo, and old exchanges not judged during this run.
+2. **Batch:** their total must reach `midRunBatchChars` (60,000; `midRunBatchCharsOpenAI` for
+   `openai-codex` models), so edits come in rare batches.
+3. **Break-even** (`midRunBreakEven`): the checkpoint must be likely to pay for its one-time cache
+   rewrite: `pending chars × turns so far ≥ factor × rewrite chars`, with factor 13 and the whole context
+   for OpenAI Codex; factor 16 for Anthropic, where the rewrite is the context after the read point the
+   next request would get (the latest trusted entry before the first eligible output, `trustedReadPoint`)
+   or, without one, after the question (the floor; with `pinAnthropicCache` off, the whole context); and
+   factor 16 and the context from the first eligible output on for other providers
+   (derivation in `checkpoint.ts` and [CACHE.md](CACHE.md#mid-run-checkpoints)). "Turns so far" stands in
+   for the turns still to come. The rewrite starts at the earliest edited entry: an old exchange comes
+   before the question, so with exchanges in the batch the Anthropic rewrite is everything after the latest
+   trusted entry before that exchange, or the whole context. If the whole batch does not pay off, the rule
+   is tried for the tool outputs alone, and the exchanges wait for run end.
+
+Then all eligible outputs go to `distillRun` with the checkpoint request builder and
+`chunkKeepThreshold = midRunChunkKeepThreshold` (0.6), under the same `timeoutMs` budget, abort signal and
+keep-on-error rules, with the busy status and working message shown.
+
+**The checkpoint request** (`buildCheckpointRequest`; separate from the run-end request): there is no final
+answer yet. The state holds `situation` (the agent is still working; this is an earlier call's output),
+`earlier_conversation`, `user_question`, `agent_progress` (`latest_note`: the latest assistant text, at
+most 2,000 characters; `earlier_notes`: earlier text of the run, clipped from the start to 2,000),
+`later_tool_calls` (labels of the calls made after this output, oldest first; above 30 the first and last
+15 with a gap note), `superseded` when it applies, and then tool, status, arguments, size and chunks as at
+run end. The questions ask whether the agent will still need the output (or chunk) to finish
+`user_question`; out-of-date lines and lines it has already acted on and moved past are not needed. The
+probe behind the wording and the threshold is in [JEV.md](JEV.md#checkpoint-wording).
+
+**Superseded** (`supersededBy`), judged from the later tool calls: a `read` whose file was later changed by
+`edit`/`write` ("this file was changed after this read (edit src/a.ts, 3 turns later)") or read again with
+the same offset and limit; a command that ran again (whitespace and a trailing `2>&1` normalized); any other
+tool called again with identical arguments.
+
+**State from the branch.** The memo and the pending edits are derived, at that moment, from the entries
+persisted on the active branch (`ctx.sessionManager.getBranch()`). A boundary's drafts are not final: a
+later handler can replace the list, and one invalid draft makes Pi discard all of them; state taken from
+the returned drafts would then skip those outputs forever. Reading the branch also makes `/tree`,
+compaction and reloads correct without any rebuild step. The only in-memory state is the log of cache
+entries this process wrote (below), which only ever adds a read point.
+
+**Memo.** Each output is judged mid-run at most once. The memo is the set of output results in the mid-run
+records on the branch, except those Jev never answered for (`timeout`, `error`, `aborted`), which the next
+checkpoint asks again. A "keep" is therefore not re-asked at every checkpoint. Old exchanges have their own
+memo per run (see Items).
+
+**Run end judges kept outputs again.** An output kept at a checkpoint (whole, or `not-worth`) is a
+candidate again at run end, this time with the final answer: files the agent was still editing are usually not
+needed once the task is done. Distilled outputs carry the marker and are skipped by the candidate rules.
+
+**Anthropic breakpoints** (`placeGuardBreakpoints`). Every Anthropic request gets the previous-question
+pin (the floor; with `pi-claude-auth` that block also carries the relocated system prompt). On the first
+request after **pending edits** (`context_edit` entries on the branch after the last answered assistant
+message, whether from a checkpoint, from run end on the next prompt, or from another extension; a failed or
+aborted response does not count, since its retry repeats the request), `cacheAnchors` adds up to two more:
+
+- **Read point**, only at a cache entry this process saw itself write. After placing its breakpoints, the
+  extension records every `tool_result` block that carries a breakpoint in the payload (Pi's rolling one
+  and its own): entry id, model, time, and the last branch entry at that moment. An entry is trusted only
+  if the same model wrote it, or read through it, within the TTL (5 minutes), the entry and that last
+  branch entry are still on the branch, and since then no `context_edit` touched anything at or before it
+  and no compaction or branch summary was appended. The latest trusted entry before the first edited entry
+  is the read point. The log is cleared on `session_start` (and so on reload); anything uncertain falls
+  back to the question pin.
+- **Refresh on read-through** (`message_end`, `refreshOnReadThrough`): reading a longer prefix keeps the
+  entries on its path alive (CACHE.md), so a response moves a logged entry's time to its request's time
+  when the request went to the same model, the entry's prefix is unchanged, and `cacheRead` reaches past
+  the entry with a margin: at least the run's baseline (the full input of its first request: system prompt,
+  tools, question) plus the projected characters from the question to the entry divided by 1.5. Code and
+  logs run at about 2.3 characters per token and prose at about 4, so this overestimates the tokens up to
+  the entry. Without a baseline, or with any doubt, nothing is refreshed.
+- **Write anchor:** the end of the tool-result batch holding the first edited entry, so the edited prefix
+  gets an entry that a later checkpoint can read. It counts as written only once it was seen in a request
+  the extension sent (the same log).
+
+Priority under the 4-breakpoint budget: after dropping Pi's redundant identity and tools breakpoints, the
+question comes first, then the read point, then the write anchor. Cache-warming replays add no assistant
+message, so they get the same breakpoints. Measured effect and the TTL findings:
+[CACHE.md](CACHE.md#mid-run-checkpoints).
+
 ## Replacement format (`render.ts`)
 
 Kept chunks are copied verbatim, in order. Each run of removed chunks becomes one omission line. For
@@ -208,21 +401,27 @@ The draft is `{ type: "context_edit", targetId, replacement: { content: [{ type:
 
 ## The `context-guard` record
 
-Each run that made at least one Jev request also appends a custom entry (`customType: "context-guard"`):
+Each run (and each mid-run checkpoint) that made at least one Jev request also appends a custom entry
+(`customType: "context-guard"`):
 
 | Field | Meaning |
 |---|---|
 | `v` | Format version, `1` |
+| `phase` | `"mid-run"` for checkpoints, `"run-end"` otherwise (a record without it counts as run end) |
 | `model` | Classifier used |
 | `savedChars` | Characters removed by this run's edits |
 | `requests`, `inputTokens`, `costUsd`, `ms` | Jev usage for the run |
 | `timedOut` | The time budget ran out |
-| `results[]` | One per candidate: `entryId`, `tool`, `label`, `outcome` (`distilled`, `removed`, `kept`), `reason`, `beforeChars`, `afterChars`, `jev` |
+| `results[]` | One per item: `kind` (`large`, `small`, `exchange`; absent means large), `entryId` (for an exchange, its prompt entry), `tool`, `label`, `outcome` (`distilled`, `removed`, `kept`), `reason`, `beforeChars`, `afterChars`, `jev` |
 
 `reason` is one of `chunks`, `none-needed`, `not-worth`, `all-chunks`, `whole-needed`, `whole-chosen`,
-`no-answer`, `error` or `timeout` (several are joined with commas). `jev` holds one trace per segment, such as
+`no-answer`, `error` or `timeout` (several are joined with commas) for large outputs, `needed` or
+`not-needed` for small outputs, and `relevant`, `unrelated` or `deferred` for exchanges (an omitted exchange's
+result also has `omitted`, the entry ids that got `replacement: null`). Whole items have the trace
+`p.06` (P of the bool). `jev` holds one trace per segment, such as
 `kw.15 focus=chunk_4:.52 keep 9/29` or `kw.82 focus=whole:.71 keep all (whole-needed)`. That is
-P(`keep_whole`), the focus choice and its probability, and the result. `/guard` prints these for the last run.
+P(`keep_whole`), the focus choice and its probability, and the result. `/guard` prints these for the last
+record (run or checkpoint) and counts the checkpoints.
 
 The extension does not write Jev's usage into Pi's own usage entries. It is recorded only here.
 
@@ -231,48 +430,73 @@ The extension does not write Jev's usage into Pi's own usage entries. It is reco
 Nothing is kept in memory. Every redraw recomputes from the session:
 
 - **Savings:** Pi's session projection (`buildSessionProjection()`) is checked for tool results whose
-  model-visible text starts with `[context-guard]` while the raw entry does not. Each one adds raw length
-  minus visible length. Tokens are estimated as characters ÷ 4.
-- **Jev usage:** the `context-guard` records on the active branch (`getBranch()`) are summed.
+  model-visible text starts with `[context-guard]` while the raw entry does not; each one adds raw size
+  minus visible size (`size.ts`: text length, images by pixels). Results that lost only their images count
+  as image results. A prompt that shows the exchange stub adds the raw size of its exchange (the
+  prompt and the entries up to the next prompt that the model no longer sees: Pi keeps an omitted
+  entry in the projection with no messages) minus the stub.
+  Tokens are estimated as characters ÷ 4.
+- **Lifetime:** the `savedChars` of every `context-guard` record on the active branch, and whether a
+  compaction came after a record that saved something (`compacted`).
+- **Jev usage:** the same records are summed.
 
 So the numbers follow branches, `/tree`, reloads and compaction.
 
-The ready-state footer is `🛡 −4.2k · 1`: ≈ tokens kept out of context · distilled results (the first part
-in the theme's success colour, the rest dimmed). Before anything is distilled it shows `🛡 0 saved`.
+The ready-state footer is `🛡 −4.2k · 1`: ≈ tokens kept out of context · distilled outputs, outputs without
+their images and omitted exchanges (the first part in the theme's success colour, the rest dimmed). Before
+anything is distilled it shows `🛡 0 saved`. After a compaction dropped earlier edits it adds the lifetime
+total: `🛡 0 · Σ−79k`, or `🛡 −4.2k · 3 · Σ−83k` once new edits exist (only when Σ is larger).
 
 ## Recall
 
 `recall` reads the raw entry with `ctx.sessionManager.getEntry(entryId)`. `context_edit` never changes that
-entry. The tool returns the whole text, or only lines that match `pattern` (with line numbers), or an
+entry. For a tool result it returns the output and its images (image blocks after the text, at most 6, only
+for a plain recall: with `pattern` or `offset` a note says how many there are); for a user prompt (the stub of an omitted exchange) it
+returns a transcript of the raw exchange from the branch: `## user`, `## assistant` with the text and
+`[tool call] name {arguments}` lines, `## tool result (tool)` and `## custom message (type)` sections, for
+exactly the entries the latest record lists as `omitted` (without such a record, up to the next prompt).
+Images of the exchange's user messages are attached (at most 6); a tool result's images are named with a
+`recall({"entryId":…})` line for that result. The tool returns the whole text, or only lines that match `pattern` (with line numbers), or an
 `offset`/`limit` window. Output is capped at 2,000 lines and 50 KB, with a continuation note; a single
 line longer than 50 KB is cut with a `[line N cut at 50KB]` note. `pattern` is a JavaScript regex of at most
 500 characters, matched against the first 4,000 characters of each line inside a `node:vm` sandbox with a
 1-second timeout, so a catastrophic pattern returns an error instead of freezing Pi. An id that is
-not a tool result gives an error that tells the model to use the id from a `[context-guard]` header.
+neither a tool result nor a user prompt on the branch gives an error that tells the model to use the id
+from a `[context-guard]` note.
 
 ## Pitfalls
 
-- **Never use `replacement: null` on a tool result.** The matching `tool_use` would lose its result, and
-  providers reject the request. A fully removed result is always replaced with a one-line stub.
-- **Never edit assistant messages.** A replacement turns the content into one text block, which drops
-  thinking, signatures and tool calls.
-- **Steering messages move the span.** A user message sent during a run starts a new span. Tool results
-  from before it are never candidates, now or later. That is the safe direction.
+- **Never advance state on returned drafts.** A later handler may replace them, or Pi may reject the list;
+  read state back from the branch.
+- **Never omit half of a tool call.** `replacement: null` on a tool result alone would leave its `tool_use`
+  without a result, and providers reject the request. A single output that is not needed is replaced with
+  a one-line stub; `null` is only used for whole exchanges, which omit the assistant message with the call
+  and its results together.
+- **Never replace an assistant message's content.** A replacement turns the content into one text block,
+  which drops thinking, signatures and tool calls. Assistant messages are only omitted, as part of a whole
+  exchange.
+- **Steering messages continue the run.** Until 0.2.0 a user message sent during a run started a new span,
+  and tool results from before it were never candidates again. In real sessions that left 0.1–0.4M
+  characters per steered run in context for good (they also blocked omitting the exchange, which contains
+  images). Now the span goes back to the prompt and the steering messages join the question. The
+  Anthropic question pin sits on the last user message with text (the steering message), so an edit before
+  it has no readable pin: `anthropicRewriteChars` counts the whole context unless a trusted read point
+  applies.
+- **Recall outputs are not judged mid-run.** The model just asked for them. Judging them at the next
+  checkpoint made models recall the same entry again (seen in real sessions). At run end they are judged
+  like other outputs.
 - **`keepCitedFiles` is off by default.** Answers also name files to rule them out. In a live run, Claude's
   answer said `test/fixtures/entities.json` was *not* a usage, and the option kept 40 lines of noise from
   that file. See [JEV.md](JEV.md).
-- **Only the run that just finished is edited.** Editing older runs would break the prompt cache from that
-  point on, for every later turn.
+- **Earlier runs are only edited as whole exchanges.** Editing single outputs of older runs would break the
+  prompt cache from that point on for little gain; an old exchange is omitted as a whole, once, when it is
+  unrelated to the current work, and only its first later request pays the rewrite.
 - **Don't distill from a `context` hook.** Output that varies between requests makes every request miss the
   cache.
 
 ## Open questions
 
-1. Do Pi's cache-warming replays (`cache-warmer.js`, `streamSimple`) go through `before_provider_request`?
-   If not, warm requests do not carry the question breakpoint. Not checked.
-2. Should the pin go on the last unedited tool result instead of the question? That would keep more of the
-   run cached on the next prompt. Not tried.
-3. Other classifier entries (e.g. `typesafe/jev-latest` with a 64k context) and non-code outputs have
+1. Other classifier entries (e.g. `typesafe/jev-latest` with a 64k context) and non-code outputs have
    not been tested.
-4. The OpenAI Codex WebSocket transport loses its delta continuation after an edit
+2. The OpenAI Codex WebSocket transport loses its delta continuation after an edit
    ([CACHE.md](CACHE.md#openai)). It is not clear whether anything can be done about that from an extension.

package/docs/JEV.md

@@ -53,7 +53,7 @@ question per chunk separated them clearly: cited files 0.90–0.93, noise 0.14
 A 30k-character grep with 28 chunks and 30 questions: **13,155 input tokens, 745 ms, $0.00055**. Code is
 about 2.3 characters per token.
 
-**Worst-case budget probe** (after earlier conversation was added to the state): every history part at its
+**Worst-case budget probe** (with earlier conversation in the state): every history part at its
 clip limit (2,000-character summary, 1,000-character first request, 800 + 1,200 characters per exchange),
 a 4,000-character question, a 6,000-character answer, 2,000 characters of notes, and one 40-chunk segment of
 31,848 characters of dense `rg` output over Pi's `dist/core`:
@@ -64,8 +64,8 @@ a 4,000-character question, a 6,000-character answer, 2,000 characters of notes,
 | 10 (maximum) | 83,214 | **23,619** | 799 ms | $0.00099 |
 
 Both fit Jev's 32k-token context with room to spare (the target was ≤ 28k), so `maxSegmentChars` is
-**32,000** (lowered from 40,000 to keep the worst case well below 32k now that history is included) and
-`maxChunksPerSegment` stays 40. Nested objects in the state (`earlier_conversation`) are accepted.
+**32,000**, which keeps the worst case well below 32k with history included, and `maxChunksPerSegment`
+is 40. Nested objects in the state (`earlier_conversation`) are accepted.
 
 In the live end-to-end runs, one qualifying run took 2 Jev requests: 550 ms and $0.00083 (GPT-6 Luna run), and
 1.18 s and $0.00084 (Claude run).
@@ -98,14 +98,14 @@ noise. It is off by default.
 ### Small reads are not worth it
 
 `read` results of about 2.3k characters always ended as `not-worth`: Jev kept 2 of 3 chunks. So
-`minResultChars` was raised to 4,000.
+`minResultChars` is 4,000.
 
 ### Earlier conversation (`historyExchanges`)
 
 Without history, Jev only sees the run's own question and answer. A broad search in a session whose goal was
 stated earlier ("I'm about to migrate every API client to a shared retry helper") then loses exactly the
 lines the ongoing work needs, because this run's question ("where is StatusBadge used?") does not mention
-them. So the state now carries `earlier_conversation`: the latest compaction/branch summary, the first
+them. So the state carries `earlier_conversation`: the latest compaction/branch summary, the first
 prompt and the last 3 exchanges (prompt + last assistant text), clipped (docs/DESIGN.md).
 
 **Wording.** Offline A/B on the real turn-2 run of the e2e `history` scenario (Claude's `rg -n -i status`,
@@ -126,28 +126,93 @@ still borderline (the relevant chunks sit around the threshold), so history make
 what the ongoing work needs, not reliably all of it; `recall` remains the safety net.
 
 **Live A/B** (`node test/e2e/run-e2e.mjs --scenario history`, with and without `--config
-'{"historyExchanges":0}'`). Turn 1 states the retry-migration goal, turn 2 is the broad StatusBadge search,
-turn 3 asks which clients retry on 429 without a new search (9 of the 10 clients do). Round 1 used the
-first wording above, round 2 the final one; one run per cell:
-
-| Model, history | Round | src/api retry lines kept in turn 2 (of 80; with 429) | Turn-2 output, chars before → after | Jev | Turn 3 used `recall` | Turn 3 answer |
-|---|---|---|---|---|---|---|
-| GPT-6 Luna, on | 1 | 0 | 36,113 → 1,772 | 518 ms | yes | correct |
-| GPT-6 Luna, off | 1 | 0 | 36,113 → 2,100 | 594 ms | no | wrong ("can't determine") |
-| Claude Sonnet 5.5, on | 1 | 0 | 36,113 → 2,612 | 685 ms | yes | correct |
-| Claude Sonnet 5.5, off | 1 | 8 (3) | 36,113 → 4,439 | 588 ms | yes | correct list; headline says "10" |
-| GPT-6 Luna, on | 2 | **72 (21)** | 36,113 → 12,402 | 568 ms | yes | correct |
-| GPT-6 Luna, off | 2 | 0 | 36,113 → 1,753 | 564 ms | yes | correct |
-| Claude Sonnet 5.5, on | 2 | 5 (1) | 22,641 → 2,103 (Claude ran `rg … \| head -200`) | 555 ms | yes | missed `ticketStatus` (cut by Claude's own `head -200`) |
-| Claude Sonnet 5.5, off | 2 | 0 | 36,113 → 2,638 | 547 ms | yes | correct list; headline says "eight" |
-
-Honest reading: with the first wording, history made no difference to what was kept (0 retry lines with
-history; the one run that kept 8 had history off). With the final wording, GPT round 2 kept 72 of the 80
-retry lines (9 of the 10 clients), at the cost of keeping 12.4k instead of ~1.8k characters; Claude round 2
-ran a different, truncated search (`| head -200`) and kept 5. It did not change turn-3 behaviour: the main models used `recall` in 7 of 8
-runs either way, and the turn-3 answers were about as good with and without history. Where history pays
-off is the context the model already has after turn 2, which avoids the recall round trip only when enough
-is kept. Single runs per cell; the variance between rounds is large.
+'{"historyExchanges":0}'`), with the wording `src/decide.ts` sends. Turn 1 states the retry-migration goal,
+turn 2 is the broad StatusBadge search, turn 3 asks which clients retry on 429 without a new search (9 of
+the 10 clients do); one run per cell:
+
+| Model, history | src/api retry lines kept in turn 2 (of 80; with 429) | Turn-2 output, chars before → after | Jev | Turn 3 used `recall` | Turn 3 answer |
+|---|---|---|---|---|---|
+| GPT-6 Luna, on | **72 (21)** | 36,113 → 12,402 | 568 ms | yes | correct |
+| GPT-6 Luna, off | 0 | 36,113 → 1,753 | 564 ms | yes | correct |
+| Claude Sonnet 5.5, on | 5 (1) | 22,641 → 2,103 (Claude ran `rg … \| head -200`) | 555 ms | yes | missed `ticketStatus` (cut by Claude's own `head -200`) |
+| Claude Sonnet 5.5, off | 0 | 36,113 → 2,638 | 547 ms | yes | correct list; headline says "eight" |
+
+With history, GPT kept 72 of the 80 retry lines (9 of the 10 clients), at the cost of keeping 12.4k instead
+of ~1.8k characters; Claude ran a truncated search and kept 5. A round with the second wording of the
+table above kept no retry lines with history. History did not change turn-3 behaviour: the models used
+`recall` either way, and the answers were about as good. Where it pays off is the context the model already
+has after turn 2. Single runs per cell; the variance between runs is large.
+
+### Checkpoint wording
+
+Mid-run checkpoints ask a different question: there is no final answer, so Jev judges whether the agent
+will still need an output to finish the task (`buildCheckpointRequest`; state and wording in
+[DESIGN.md](DESIGN.md#mid-run-checkpoints)). Probed offline on real mid-run states from two no-guard runs
+of the e2e `long` scenario (`fix all failing tests`), two repetitions per variant, reporting the share of
+the output's characters that would be kept at chunk thresholds 0.5 / 0.55 / 0.6:
+
+| Output (state) | What should happen | base wording | "acted" wording |
+|---|---|---|---|
+| GPT, 6 old `npm test` logs, each rerun later (`superseded`) | mostly go | 0.5: 15–69%, 0.6: **8–36%** | 0.5: 15–45%, 0.6: 15–30% |
+| Claude, first `npm test` log (still the only full failure list; later runs were filtered) | keep the unresolved failures | 0.5: 73–80%, 0.6: 35–41% | 0.5: 80–87%, 0.6: 33–46% |
+| Claude, `cat` of 5 source files, **all fixed since** | go | 0.5: 49–62%, 0.6: **11%** | 11% at every threshold |
+| Claude, the same `cat` one turn after it ran, **files not fixed yet** | stay | 0.5: 100%, 0.6: **78%** | 0.5: 90%, 0.6: **10%** |
+
+- *base*: "To finish user_question, will the agent still need the lines in chunk_N? Lines that are out of
+  date or that the agent has already acted on and moved past are not needed."
+- *acted*: "Will the agent need to look at chunk_N again before it finishes user_question? Say no if
+  later_tool_calls show the agent already acted on it, or if superseded says a newer version exists."
+  It dropped the in-use files (10%) and was rejected.
+- A stronger `superseded` sentence ("the agent still sees that newer output…"), with the acted wording, did not help (15–65% at 0.5).
+
+Mid-run probabilities are compressed (per-chunk 0.14–0.76, against 0.14–0.93 at run end). At 0.5,
+superseded logs and finished files kept 50–70%, which `maxKeepRatio` (0.6) would then turn into "not
+worth it", so most checkpoints would edit nothing. **Base wording at 0.6** separated the cases: in-use
+files 78%, finished files 11%, superseded logs 8–36%. So `midRunChunkKeepThreshold` defaults to 0.6, the
+same as run end, not lower: the brief proposed starting at 0.5 to keep more, and the probe showed that
+0.5 keeps nearly everything.
+
+### Old exchanges
+
+`buildExchangeRequest` asks one bool per exchange: "Is exchange_N still relevant to the current work
+(user_question and what the agent is doing for it)? Relevant means the agent may need its details: the
+same files, task, decisions or facts." Probed with `node test/e2e/probe-items.mjs` (2 repetitions):
+
+- three hand-written sessions (`test/e2e/probe-data/exchanges.json`) modeled on the shape of real long
+  main-thread sessions (a prompt, a final answer, tool-call labels, a size; 5 old exchanges each, mixed
+  relevant and unrelated to the current task);
+- the eligible exchanges at prompts 4 and 5 of two no-guard runs of the `topics` scenario (GPT and Claude,
+  public test repo): the first exchange (API retries) is unrelated to prompt 4 (`StatusBadge`) and relevant
+  to prompt 5 (its follow-up).
+
+| Label | P(relevant), 42 answers |
+|---|---|
+| relevant (16) | 0.47–0.96 (the weakest: "add a disk-usage alert for the backup volume" for a failing-backup task, 0.47–0.49; "how is contextTokens estimated with edits" for a compaction-settings task, 0.68) |
+| unrelated (26) | 0.03–0.10 |
+
+| Omit when P < | Accuracy | Relevant exchanges omitted | Unrelated exchanges omitted |
+|---|---|---|---|
+| 0.1 – 0.4 | 42/42 | 0/16 | 26/26 |
+| 0.5 | 41/42 | 1/16 | 26/26 |
+
+`exchangeOmitThreshold` is 0.2: omit only when Jev is clearly sure, with a wide margin to the weakest
+relevant exchange (0.47). The sessions are synthetic or generated, so real sessions may be less clear-cut;
+the margin is meant to absorb that. Repetitions agreed within 0.03.
+
+### Small outputs
+
+`buildSmallRequest` asks one bool per output ("will the agent still need the output in item_N?" mid-run;
+"does item_N contain anything that final_answer relies on, or that a likely follow-up would need?" at run
+end), with `superseded` shown when a later call made the output out of date.
+
+| Probe | Needed | Not needed |
+|---|---|---|
+| Two hand-written mid-run states on the `long` fixture's code (`test/e2e/probe-data/small.json`), 24 answers | 0.59–0.74 | 0.06–0.32 |
+| A `long` run of GPT-6 Luna (public fixture), mid-run at request 12 and at run end: reads of source files | not superseded: 0.48–0.68 | superseded (the file was edited afterwards): 0.19–0.41 |
+
+At 0.6 (the chunk threshold) 2 of 10 needed outputs would be dropped; at 0.45 none, and every superseded or
+unneeded output goes. `smallKeepThreshold` is 0.45. Without the `superseded` hint, Jev kept reads of files
+the agent had already edited (0.49–0.86), so the hint is part of the request.
 
 ## The final question set
 
@@ -175,5 +240,8 @@ answer did not quote but that a natural next question would need.
 | `noneThreshold` | 0.5 | Remove a whole output only if `none` wins and no chunk passes its own question |
 | `maxKeepRatio` | 0.6 | If most of the output is kept anyway, the saving does not justify the cache miss and the lost context |
 | `minResultChars` | 4,000 | Reads around 2.3k characters were never worth an edit |
+| `smallKeepThreshold` | 0.45 | Small-output probe above: needed outputs ≥ 0.48, superseded or unneeded ≤ 0.41 |
+| `exchangeOmitThreshold` | 0.2 | Exchange probe above: unrelated ≤ 0.10, relevant ≥ 0.47; omit only when Jev is clearly sure |
+| `midRunChunkKeepThreshold` | 0.6 | Checkpoint probe above: 0.5 kept 50–100% of everything; 0.6 kept in-use files and dropped finished ones |
 
 When an answer is missing, the request fails or the budget runs out, the extension keeps everything.