pi-context-guard-jev 0.1.0 → 0.3.1

package/docs/REFERENCE.md

@@ -0,0 +1,164 @@
+# Reference
+
+Everything you can see, call and configure. How it works inside: [DESIGN.md](DESIGN.md).
+
+## Requirements
+
+- Pi ≥ 1.0.3
+- An OpenRouter credential (`OPENROUTER_API_KEY` or `/login`) for the default classifier,
+  `openrouter/~typesafe/jev-latest`. Any classifier model in Pi's catalog can be set with the `model`
+  setting, but only the default has been tested.
+
+If the model or credential is missing, the extension does nothing and the status bar says so.
+
+## Status bar
+
+| Text | Meaning |
+|---|---|
+| `🛡 0 saved` | On and ready; nothing is distilled on this branch yet |
+| `🛡 0 · Σ−79k` | A compaction dropped the earlier edits from context: nothing is kept out right now, about 79k tokens were saved on this branch in total |
+| `🛡 distilling…` | Jev is judging the outputs of the run that just finished, or of a mid-run checkpoint |
+| `🛡 −4.2k · 1` | ≈ tokens kept out of context · items: about 4.2k tokens are currently kept out of context, by 1 distilled or omitted tool output, output without its images, or old exchange |
+| `🛡 −4.2k · 1 · Σ−83k` | The same, after a compaction: Σ is the total saved on this branch, including edits the compaction dropped |
+| `🛡 guard off` | Turned off with `/guard off` or `"enabled": false` |
+| `🛡 no openrouter key` | No credential for the classifier's provider (the provider name is filled in) |
+| `🛡 Jev model not found` | The configured `model` is not in Pi's classifier catalog |
+
+The token figure is Pi's usual estimate, characters ÷ 4. It counts what is kept out of the model's context
+**on the current branch right now**: original size minus replacement size, for every distilled output,
+removed image and omitted exchange that is still in the context (an image counts as about width × height ÷ 750
+tokens). It goes down after `/tree` to a point before an edit, and after compaction removes distilled
+results; then Σ shows the total of every pass on the branch so far. It is not a measured token count.
+
+## `/guard` command
+
+| Command | Effect |
+|---|---|
+| `/guard` or `/guard status` | Shows whether the guard is on, the model, the savings on this branch (distilled outputs and omitted exchanges), Jev runs and mid-run checkpoints, requests and cost, whether checkpoints and exchange pruning are on, and the last run's or checkpoint's result for each item (large, small, exchange), including Jev traces such as `kw.15 focus=chunk_4:.52 keep 9/29` or `p.06` |
+| `/guard on` / `/guard off` | Turns the guard on or off and saves `enabled` to the settings file |
+| `/guard reload` | Re-reads the settings file and shows any warnings again |
+
+Settings are read at session start and on `/guard reload`.
+
+## `recall` tool
+
+Every distilled output and omitted exchange starts with `[context-guard]` and ends with
+`recall({"entryId":"…"})`. For an omitted exchange, recall returns the whole exchange as a transcript
+(prompt, replies, tool calls and results, custom messages). The model can call:
+
+| Parameter | Meaning |
+|---|---|
+| `entryId` (required) | The entry id from the header |
+| `pattern` | JavaScript regex (≤ 500 chars, 1 s limit); returns only matching lines, prefixed with their line numbers |
+| `offset` | First line to return, 1-based. When `pattern` is set, this counts matching lines. |
+| `limit` | Most lines to return |
+
+Like Pi's `read` tool, the output is capped at 2,000 lines and 50 KB, and a note says how to continue.
+
+## Configuration
+
+The settings file is `~/.pi/agent/context-guard.json` (inside Pi's agent directory), or the path in
+`PI_CONTEXT_GUARD_CONFIG`. Every key is optional. Values that are missing, the wrong type or out of range
+fall back to the default.
+
+| Key | Default | Meaning |
+|---|---|---|
+| `enabled` | `true` | Master switch (`/guard on` and `/guard off` write it) |
+| `model` | `"openrouter/~typesafe/jev-latest"` | Classifier as `provider/id`; the first `/` separates the provider |
+| `minResultChars` | `4000` | Tool results from this size are chunked; shorter ones are judged whole (see `smallResultMinChars`) |
+| `minRunChars` | `8000` | Distill a run's large outputs only if they add up to at least this (small outputs and exchanges are judged regardless) |
+| `keepWholeThreshold` | `0.7` | Keep a segment untouched if P(`keep_whole`) is at least this |
+| `focusWholeThreshold` | `0.6` | Keep a segment untouched if `focus` picks "whole" with at least this probability |
+| `chunkKeepThreshold` | `0.6` | Keep a chunk if P(chunk needed) is at least this |
+| `noneThreshold` | `0.5` | Remove a segment entirely if `focus` picks "none" with at least this probability and no chunk passes |
+| `maxKeepRatio` | `0.6` | Skip the edit if the kept chunks are more than this fraction of the original |
+| `keepCitedFiles` | `false` | Also keep grep chunks for files that the final answer names (see DESIGN.md for why it is off) |
+| `timeoutMs` | `8000` | Total time budget for all Jev requests of one run; unfinished parts are kept |
+| `concurrency` | `6` | Jev requests in parallel |
+| `maxSegmentChars` | `32000` | Most characters of one output per Jev request (keeps the worst-case request under Jev's 32k-token context, see [JEV.md](JEV.md)) |
+| `maxChunksPerSegment` | `40` | Most chunks per Jev request |
+| `excludeTools` | `["edit", "write"]` | Tools whose results are never distilled |
+| `distillErrors` | `true` | Also distill error results, such as the log of a failing test or build; Jev is told the call failed, and that its details are usually no longer needed once the agent got past it |
+| `historyExchanges` | `3` | Earlier exchanges (prompt + last assistant text) sent to Jev so it judges relevance against the ongoing work; `0` sends no earlier conversation at all (integer 0–10) |
+| `pinAnthropicCache` | `true` | Anthropic cache breakpoints: at the previous user question on every request, and on the first request after context edits a read point (at a cache entry this process wrote) and a write anchor ([CACHE.md](CACHE.md)) |
+| `midRun` | `true` | Mid-run checkpoints during long runs ([RESULTS.md](RESULTS.md#long-autonomous-runs)) |
+| `midRunMinAgeTurns` | `4` | Judge an output mid-run only once it is at least this many turns old (integer 1–100) |
+| `midRunBatchChars` | `60000` | Run a checkpoint only when the not-yet-judged eligible outputs add up to this many characters |
+| `midRunBatchCharsOpenAI` | `60000` | The same, for `openai-codex` models |
+| `midRunChunkKeepThreshold` | `0.6` | Keep a chunk at a checkpoint if P(still needed) is at least this (probe in [JEV.md](JEV.md)) |
+| `midRunBreakEven` | `true` | Skip a checkpoint unless its one-time prompt-cache rewrite is likely to pay off; `false` checkpoints purely for context room |
+| `smallResultMinChars` | `400` | Tool results from this size up to `minResultChars` are judged as whole items; `0` turns this off |
+| `smallKeepThreshold` | `0.45` | Keep a small output if P(still needed) is at least this (probe in [JEV.md](JEV.md)) |
+| `pruneExchanges` | `true` | Judge old exchanges and omit the unrelated ones |
+| `keepRecentExchanges` | `2` | The last this many exchanges before the current prompt are never omitted |
+| `exchangeOmitThreshold` | `0.2` | Omit an exchange only if P(still relevant) is below this (probe in [JEV.md](JEV.md)) |
+| `exchangeBreakEven` | `true` | At run end, omit unrelated exchanges only when the cache rewrite they cause pays off; deferred ones are judged again at later run ends and omitted once their total pays off. `false` always omits them |
+| `exchangeMinSavingChars` | `8000` | A run end omits exchanges only if they save at least this many characters together (when they cost a cache rewrite) |
+| `imageKeepTurns` | `3` | Remove images from tool results once this many assistant messages have followed them (`recall` shows them again); images in user messages are kept; `0` turns this off |
+
+Example:
+
+```json
+{ "minResultChars": 6000, "excludeTools": ["edit", "write", "bash"] }
+```
+
+## Data sent to Jev
+
+**This extension sends data to a third party, and it is on by default.** After every run that qualifies,
+each Jev request sends the following to OpenRouter, which forwards it to TypeSafe:
+
+- the user's question (up to 4,000 characters),
+- the final answer (up to 6,000 characters),
+- the assistant's notes written during the run (up to 2,000 characters),
+- excerpts of the earlier conversation in this session: the latest compaction or branch summary (up to
+  2,000 characters), the first prompt (up to 1,000), and the last 3 earlier exchanges, each one prompt
+  (up to 800) and the last assistant text after it (up to 1,200). Tool calls and tool outputs of earlier runs
+  are not included in this part. Set `historyExchanges` to `0` to send none of it,
+- the tool name and arguments,
+- the full text of the large tool output being judged, and of small outputs (from 400 characters; up to
+  4,000 characters each). This can include source code, logs or anything else the tools returned,
+- excerpts of old exchanges being judged: each prompt (up to 600 characters), the last assistant text (up to
+  800), the tool calls as short labels (tool name and the command or path, up to 80 characters each) and the
+  size. Set `pruneExchanges` to `false` to send none of it.
+
+Never sent: tool outputs below `smallResultMinChars` (400 characters), results of excluded tools (`edit`,
+`write`), images (never sent: they are removed by age), the text of outputs with images, old exchanges with entries another extension changed,
+exchanges below 2,000 characters, and the last `keepRecentExchanges` (2) exchanges before the current
+prompt. Large outputs are sent only when the run's large outputs add up to `minRunChars` (8,000); small
+outputs and old exchanges are sent at every run end and every mid-run checkpoint that has any.
+
+`/guard off` turns the extension off and saves that choice in the settings file, so it stays off in later
+sessions. See OpenRouter's data policy:
+<https://openrouter.ai/docs/guides/privacy/data-collection>.
+
+## What it never touches
+
+- messages of the current run other than tool results, and the last `keepRecentExchanges` exchanges before
+  the current prompt
+- compaction and branch summaries, exchanges that did not finish (the last reply has tool calls), and
+  messages Pi cannot edit (system messages, `!` shell executions), which stay in place inside an omitted
+  exchange
+- images in your own messages; images in tool results younger than `imageKeepTurns` (3) turns
+- the text of tool results that contain images (only the images are removed, by age)
+- error results, if `distillErrors` is `false`
+- results of excluded tools (`edit`, `write` by default)
+- results shorter than `smallResultMinChars` (400), and large results of runs below `minRunChars`
+- aborted or failed runs
+- results that are already distilled, or that another extension already edited
+- single lines too long for one Jev request (kept as they are)
+- tool outputs of earlier runs one by one: earlier runs are only touched as whole exchanges (and their old
+  images)
+
+## Limitations
+
+- **Lossy by design.** Relevance is judged against the run's question and answer and the earlier
+  conversation (the ongoing task), so a follow-up that goes somewhere new may need something that was
+  removed. The header and omission lines say what is missing, and `recall` returns it. An earlier task
+  that comes back after several unrelated ones may have been omitted as a whole exchange.
+- **Latency.** Settling waits for Jev. This adds about 0.5–1.2 s after each qualifying run (measured), and
+  never more than `timeoutMs` (8 s). A mid-run checkpoint pauses the run the same way.
+- **Cost.** Each qualifying run costs a few Jev requests, measured at under $0.001 per run. The first prompt
+  after an edit also has a one-time partial prompt-cache miss ([CACHE.md](CACHE.md)).
+- **Steering.** A message sent during a run joins the run's question. Outputs from before it can be distilled,
+  but the first edit before it rewrites the whole prompt cache once (the question pin moves to the steering
+  message).

package/docs/RESULTS.md

@@ -0,0 +1,146 @@
+# Measured results
+
+Single runs on Pi 1.0.3, not a benchmark: the main model takes a different path each time. Costs are what Pi
+reports at list price. How to reproduce them: [TESTING.md](TESTING.md).
+
+## Examples
+
+Real runs on public code: each example ran once with the guard on and once with it off (`--no-guard`), from
+the same prompts. Reproduce with `node test/e2e/examples.mjs`.
+Examples A–C judge large outputs only (their numbers match `smallResultMinChars: 0`); example D shows the
+default configuration and `exchangeBreakEven: false`.
+
+**A. Searching a real codebase.** In a copy of the published Pi package (`dist/` without source maps and
+the minified bundle, plus `docs/`), the agent was asked where Pi decides to auto-compact, starting with
+`rg -n -i compact dist docs`. Pi's bash tool returned the last 50 KB of that search. Jev kept the matches
+in the compaction module, the settings and the decision points in `agent-session.js`, and dropped matches
+in the grep tool, HTML export templates, the SDK and so on. Then the agent was asked "which of those
+settings can a user change, and what are their defaults?".
+- Claude Sonnet 5.5: 50,413 → 16,210 chars (141 of 431 lines kept). The first turn-2 request carried
+  **15.5k tokens instead of 28.2k**. The answer was correct (`enabled` true, `reserveTokens` 16384,
+  `keepRecentTokens` 20000), without `recall`.
+- GPT-6 Luna: 51,216 → 13,023 chars (107 of 385 lines), and a 5.4k read of `compaction.js` → 1.8k; two
+  smaller outputs were left whole. Turn 2 carried **11.5k tokens instead of 34.3k** and was correct, without
+  `recall`.
+
+**C. Reading a 164 KB source file.** The agent read `dist/core/agent-session.js` (3,535 lines) in
+seven 500-line pieces and explained how a prompt reaches the model. Jev kept 45–164 lines of each of the first
+four pieces and removed the last three completely (each became a one-line stub). The follow-up asked for a
+detail the explanation never mentioned: the exact error thrown when a prompt arrives during compaction.
+That line was among the kept ones, so the agent quoted it exactly without `recall`. The turn-2 request
+carried **9.2k tokens instead of 35.3k**.
+
+**B. A failing test run.** In a copy of this repository with one injected off-by-one bug, the agent ran
+the unit tests (a 15.8k-character log, 3 failures among 116 tests) and reported the cause. Jev kept the
+3 failures with their assertion messages and the failing-test summary, and dropped the passing tests:
+15.8k → 8.6k chars. Turn 2 ("now fix it") carried **5.7k tokens instead of 7.5k**, and both runs fixed
+the bug. Failing commands are distilled because `distillErrors` is on by default.
+
+What the model sees after distilling (example A, Claude; the first lines of the real replacement):
+
+```text
+[context-guard] Distilled the output of bash `rg -n -i compact dist docs`: kept 141 of 431 lines (15.1k of 50.4k chars); the rest was judged not needed for the answer. Full output: recall({"entryId":"1704787f"}).
+[… 79 lines omitted: matches in dist/core/tools/grep.js, dist/core/session-manager.js, dist/core/export-html/template.js, dist/core/export-html/template.css, dist/core/sdk.js and 5 more …]
+dist/core/compaction/compaction.d.ts:25:    /** Extension-specific data (e.g., ArtifactIndex, version markers for structured compaction) */
+dist/core/compaction/compaction.d.ts:28:export interface CompactionSettings {
+dist/core/compaction/compaction.d.ts:33:export declare const DEFAULT_COMPACTION_SETTINGS: CompactionSettings;
+dist/core/compaction/compaction.d.ts:54:/** Estimate projected context without trusting usage captured before a later edit or compaction. */
+dist/core/compaction/compaction.d.ts:57: * Check if compaction should trigger based on context usage.
+dist/core/compaction/compaction.d.ts:59:export declare function shouldCompact(contextTokens: number, contextWindow: number, settings: CompactionSettings): boolean;
+[… 44 lines omitted: matches in dist/core/compaction/compaction.d.ts, dist/core/compaction/branch-summarization.d.ts, dist/core/compaction/index.d.ts, dist/core/session-manager.d.ts, dist/core/cache-stats.js and 2 more …]
+```
+
+| Example | Model | Tool output before → after | Jev | Next request's context, on / off | Main-model cost of turn 2, on / off | Follow-up |
+|---|---|---|---|---|---|---|
+| A. code search | Claude Sonnet 5.5 | 50.4k → 16.2k chars | 3 requests, 0.8 s, $0.0012 | 15.5k / 28.2k tokens | $0.033 / $0.016 | correct, no `recall` |
+| A. code search | GPT-6 Luna | 56.6k → 14.8k chars (2 outputs; 2 more, 15k, left whole) | 5 requests, 0.6 s, $0.0015 | 11.5k / 34.3k tokens | $0.0012 / $0.0005 | correct, no `recall` |
+| C. 164 KB file | GPT-6 Luna | 163k → 26k chars (7 reads) | 7 requests, 0.9 s, $0.0027 | 9.2k / 35.3k tokens | $0.0007 / $0.0005 | correct, no `recall` |
+| B. failing test log | GPT-6 Luna | 15.8k → 8.6k chars | 1 request, 0.8 s, $0.0004 | 5.7k / 7.5k tokens | $0.0022 / $0.0016 | fixed in both runs |
+| D. five topics | Claude Sonnet 5.5, `exchangeBreakEven: false` | 1 old exchange omitted (8.8k chars) | 2 requests, $0.0003 | 8.0k / 12.8k tokens (prompt 5) | $0.017 / $0.006 (prompt 5's first request) | correct, searched again |
+
+**D. Several topics in one session.** In the generated test repo, five prompts in a row: which API
+clients retry on 503; a README paragraph for `StatusBar`; a count of HTTP 500 lines in a log; where
+`StatusBadge` is used; and then "back to the clients that retry on 503: which of them also retry on 502?"
+(`node test/e2e/run-e2e.mjs --scenario topics --turns 5`). The README exchange is under 2,000 characters
+and is never judged; the last 2 exchanges before each prompt are never judged either.
+
+| Model, configuration | First exchange at the end of prompt 4 | At the end of prompt 5 | Prompt 5 | Context of prompt 5's first request | Main-model cost | Jev |
+|---|---|---|---|---|---|---|
+| GPT-6 Luna, default | judged unrelated (P = 0.07), **deferred**: too small to pay for the cache rewrite in a 4-prompt session | relevant (P = 0.95), kept | correct, no `recall` | 7.5k, 6,656 read from cache | $0.0018 | $0.0005 |
+| Claude Sonnet 5.5, default | unrelated (P = 0.07), deferred | relevant (P = 0.97), kept | correct, no `recall` | 11.5k, 11,049 read from cache | $0.073 | $0.0003 |
+| Claude Sonnet 5.5, `exchangeBreakEven: false` | unrelated (P = 0.08), **omitted** (8.8k characters, 5 entries) | – | correct, searched again | 8.0k: 1,760 read, 6,280 written | $0.087 | $0.0003 |
+| Claude Sonnet 5.5, guard off | – | – | – | 12.8k | $0.083 | – |
+
+In this short session the gate kept the first exchange, which the follow-up then needed. With the gate off,
+the omission made the next request rewrite the cache from the omitted exchange on (it comes before the
+question pin) and the model searched again; the smaller context pays off only in longer sessions.
+
+**When it doesn't help.**
+
+- **The first request after an edit costs more, not less.** The prompt cache only matches up to the
+  first changed message, so the next request writes the shorter history again instead of reading the long
+  one from cache. In A that made Claude's turn-2 request cost $0.033 instead of $0.016, and GPT's $0.0012
+  instead of $0.0005; OpenAI read nothing from cache on that request (in the fixture runs it read 2,560
+  tokens instead of 13,824: a partial miss).
+  At the per-token prices in these runs, the smaller context earns that back after about 7 more requests
+  (Claude, A), about 3 (GPT, A) or about 1 (GPT, C). Before that, the benefit is a smaller context window,
+  not a lower bill. Anthropic requests also get a cache breakpoint at the previous question; in a run with
+  11 sequential tool calls it raised the first request's cache read from 1,693 to 5,810 tokens compared
+  with `pinAnthropicCache: false` ([CACHE.md](CACHE.md)).
+- **Small or fully needed outputs are left whole.** In A, GPT's 9k-character read of `agent-session.js` was
+  `not-worth` (too much of it was needed) and a 6k narrowed search was judged needed whole.
+
+Numbers on the generated StatusBadge test repo (96% removed, `recall` of a removed line) are in
+[TESTING.md](TESTING.md#results-pi-103).
+
+## Long autonomous runs
+
+**Measured** (Pi 1.0.3, e2e scenario `long`: "fix all failing tests" in a generated project with 10 bugs,
+the full suite after every fix; single runs, not a benchmark; costs are what Pi reports at list price):
+
+| Main model | Configuration | Requests | Peak context | Main-model cost | Checkpoints (Jev) | Suite green |
+|---|---|---|---|---|---|---|
+| GPT-6 Luna | guard off | 32 / 37 | 68.4k / 87.7k | $0.020 / $0.030 | – | yes / yes |
+| GPT-6 Luna | `midRun: false` (run end only) | 26 | 86.3k | $0.022 | – (run end $0.006) | yes |
+| GPT-6 Luna | default | 29 | **45.0k** | $0.021 | 2 at requests 16 and 24, 0.8 s each, removed 62k and 60k chars ($0.006 with run end) | yes |
+| GPT-6 Luna | `smallResultMinChars: 0` (large outputs only) | 31 | 72.4k | $0.024 | 1 at request 19, 1.0 s, removed 154k chars ($0.007 with run end) | yes |
+| Claude Sonnet 5.5 | guard off | 14 | 89.8k | $0.41 | – | yes |
+| Claude Sonnet 5.5 | `smallResultMinChars: 0` | 8 | 65.7k | $0.24 | none: too short to pay off (run end $0.003) | yes |
+| Claude Sonnet 5.5 | forced: `midRunBreakEven: false`, `midRunMinAgeTurns: 2`, `midRunBatchChars: 20000`, `smallResultMinChars: 0` | 12 | 43.3k | $0.34 | 3 ($0.0032 with run end) | yes |
+
+Context per request (every 5th request), GPT-6 Luna:
+
+| Request | 1 | 6 | 11 | 16 | 21 | 26 | 31 | 36 |
+|---|---|---|---|---|---|---|---|---|
+| guard off (37 requests) | 3.0k | 14.6k | 29.3k | 41.4k | 52.1k | 62.5k | 73.3k | 83.4k |
+| default (29 requests) | 3.2k | 17.0k | 35.3k | 41.0k | 39.3k | 28.9k | – | – |
+
+At the default run's first checkpoint, Jev distilled four superseded test logs (for example 30,164 → 8,163)
+and replaced 4 of 7 small reads of files the agent had already edited with one-line notes (for example
+3,386 → 145); the next request carried 20.1k instead of 41.0k tokens. No run needed `recall`.
+
+**Trade-offs:**
+- **OpenAI Codex reads nothing from cache after any edit** (2,560 or 0 tokens on the next request, after
+  every checkpoint measured, wherever the edit was). Each checkpoint re-sends the context uncached once and
+  pays back after about 8–10 later requests. In these 26–37-request runs the main-model cost came out
+  about the same with and without checkpoints, plus about $0.005 of Jev per run; the gain is a smaller
+  context.
+- **On Anthropic a checkpoint rewrites the cache after the latest entry the next request can read** (at worst after the question; writing costs 12.5× the read price).
+  In the forced run, the requests after the three checkpoints read 5.8k / 16.7k / 16.7k tokens and wrote
+  10.9k / 22.8k / 20.0k. After the first checkpoint of a run only the system prompt and question can be
+  read, because the run's first output is usually the first one edited. Claude solved the task in 8–14
+  requests, too few for a checkpoint to pay off, so with the default break-even rule it never
+  checkpointed; forced checkpoints in such short runs cost more than they saved. Checkpoints are meant
+  for runs that go on for dozens of requests.
+- **Latency mid-run:** a checkpoint pauses the run for the Jev requests (0.5–2.2 s measured, at most
+  `timeoutMs`).
+- Details: [CACHE.md](CACHE.md#mid-run-checkpoints), [DESIGN.md](DESIGN.md#mid-run-checkpoints),
+  wording probe in [JEV.md](JEV.md#checkpoint-wording).
+
+## Old images
+
+Claude Sonnet 5.5 with `pi-claude-auth`, 2026-10-08: six 1200×800 PNGs, read one per message, then a
+follow-up about the second one. The run end removed the four images that were 3–6 turns old, with no Jev
+request (19.9k chars ≈ 5k tokens). The next request carried 2 images instead of 6: 9.4k input tokens
+instead of about 14.1k. Told not to read the file again, the model called `recall`, got the image back and
+read the code number in it correctly.

package/docs/TESTING.md

@@ -13,15 +13,19 @@ These need Node ≥ 22.18, which runs the `.ts` files directly. There is no netw
 
 | File | Covers |
 |---|---|
-| `run.test.ts` | Run span (including image-only prompts), question/answer/notes, candidate rules (errors, images, excluded tools, short, already-distilled and other-extension-edited results), steering messages; history (last N exchanges, first request, compaction/branch summaries, image-only and steering prompts, clipping, `historyExchanges` 0) |
+| `checkpoint.test.ts` | Mid-run eligibility and age, the memo and candidate rules, later calls, notes, superseded detection (edits, re-reads, reruns), cache anchors (read point only from logged entries of the same model within the TTL, invalidated by later edits and compaction, parallel-call batches never split, early return without pending edits), refresh on read-through (same model, cacheRead clearly past the entry, unchanged prefix; trusted after 5 minutes once refreshed), the run baseline, the Anthropic rewrite estimate (from the trusted read point, else from the question), pending edits, the memo from records, the break-even rule, context size with summaries |
+| `items.test.ts` | Small outputs (size window, candidate rules, age, memo, stub length, superseded), exchange spans (custom messages, completed exchanges only, the current run and the recent exchanges never touched, already omitted, memo; images, foreign edits and foreign omissions skipped, our own markers allowed; system, `!` and custom messages; a compaction that cut an exchange; minimum size), stubs and edits, the small and exchange requests in both phases, batching, the shared limiter, `judgeItems` (parallel limit, failures keep, time budget), `processItems` (stubs, omissions, records with kinds, one `concurrency` across all kinds) |
+| `run.test.ts` | Run span (including image-only prompts), question/answer/notes, candidate rules (errors, images, excluded tools, short, already-distilled and other-extension-edited results), steering messages (continue the run, join the question); history (last N exchanges, first request, compaction/branch summaries, image-only and steering prompts, clipping, `historyExchanges` 0) |
 | `chunk.test.ts` | grep detection, grouping by file, context lines and `--`, `-digits-` file names, split/merge rules, plain-text windows, segment limits |
-| `decide.test.ts` | Request shape, with and without `earlier_conversation` (state order, omitted parts, history wording vs. the unchanged single-run wording); the decision order (error, missing answers, `keep_whole`, strong/weak `whole`, chunks, `none`); `citedFiles` |
+| `decide.test.ts` | Checkpoint request shape and wording; run-end request shape, with and without `earlier_conversation` (state order, omitted parts, history wording vs. the unchanged single-run wording); the decision order (error, missing answers, `keep_whole`, strong/weak `whole`, chunks, `none`); `citedFiles` |
 | `distill.test.ts` | Skips, edits, `removed`, `not-worth`, failures, time budget (including a classifier that never answers and late answers), parent abort, concurrency limit, multi-segment outputs, oversize segments never sent, `keepCitedFiles`, usage sums |
 | `render.test.ts` | Labels, verbatim chunks with omission lines, `read` offsets, file lists, full removal |
-| `cache-pin.test.ts` | API-key and OAuth payloads, trailing system messages, TTL selection and `ttl-conflict`, string content, tool-result-only messages, over-budget restore |
-| `stats.test.ts` | Savings from the projection, run records, status texts and colours, cost format |
+| `cache-pin.test.ts` | Question pin: API-key and OAuth payloads, trailing system messages, TTL selection and `ttl-conflict`, string content, tool-result-only messages, over-budget restore. Anchors: question > read > write priority, read anchor inside a batch of tool_result blocks, breakpoint budget (identity and tools breakpoints dropped, then write, then read; the question survives a foreign breakpoint), not found / last message / already marked, normalized tool ids, TTL order, `markedToolResults` |
+| `stats.test.ts` | Savings from the projection (images by pixels), run records (checkpoints counted), lifetime total and `compacted`, status texts (`🛡 0 · Σ−79k`) and colours, cost format |
+| `images.test.ts` | Old images: age rule, keep-turns 0/1/3, stub text and sizes, user images / already removed / foreign-edited results skipped |
+| `size.test.ts` | Image sizes from PNG/JPEG/GIF/WebP headers, image tokens by pixels (not base64 length), thinking and tool-call sizes, `contextChars` with screenshots |
 | `recall.test.ts` | Original output, pattern (length cap, 4,000-char match window, catastrophic-regex timeout), offset/limit, line and byte caps, oversized lines |
-| `config.test.ts` | Defaults, validation and ranges (incl. `historyExchanges` 0–10 integer), `provider/id` parsing, load/save round trip |
+| `config.test.ts` | Defaults, validation and ranges (incl. `historyExchanges` 0–10 integer and the `midRun*` settings), `provider/id` parsing, load/save round trip |
 | `extension.test.ts` | The real entry point (see below) |
 
 `extension.test.ts` loads `src/index.ts` through **Pi's own extension loader** (`loadExtensions` from the
@@ -29,7 +33,31 @@ These need Node ≥ 22.18, which runs the `.ts` files directly. There is no netw
 `PI_CODING_AGENT_DIR` and `PI_CONTEXT_GUARD_CONFIG` to a temporary directory, so `/guard on|off` never touches the real
 `~/.pi/agent/context-guard.json`. It checks that the handlers, tool and command are registered, that drafts
 are appended after other extensions' drafts, that unfinished runs and a missing model are ignored, that the
-pin applies only to `anthropic-messages`, that `recall` works, and that `/guard off|on` persists.
+pin applies only to `anthropic-messages`, that `recall` works, and that `/guard off|on` persists. For
+`turn_end` it checks that the final turn is skipped, that a batch below `midRunBatchChars` and a checkpoint
+that would not break even are skipped, that a due checkpoint appends edits and a `mid-run` record after other
+drafts (never `continue`), that nothing is judged twice once the record is committed, that drafts a later
+handler dropped (or Pi rejected) are judged again, that an unanswered output (error) is asked again, that the
+memo follows `/tree`, and that run end judges outputs kept mid-run again. On a `pi-claude-auth`-shaped
+payload (system prompt moved into the first user message) it checks the question pin on every request; on
+the first request after edits a write anchor, and a read point only at an entry this process sent earlier
+(also on a warming replay); a read point after run-end edits; and no read point for another model, after
+the TTL (mocked clock), after a reload, for an anchor dropped for the budget, or for history made with
+pinning off. Through `message_end` it checks that a response that read through a logged entry keeps it
+usable past 5 minutes, and that a short read or another model's response does not. For whole items it
+checks that run end omits an unrelated old exchange (stub on the prompt, `null` for the rest) and stubs a
+small output, that the next run judges the next eligible exchange, that `recall` on the stub returns the
+whole exchange, that `pruneExchanges: false` and `smallResultMinChars: 0` turn them off, and that the
+break-even rule drops exchanges from a checkpoint batch that would not pay off with them. The run-end
+exchange gate: a small omission is deferred and the accumulated batch is omitted at a later run end; a big
+early exchange in a short session is deferred, and omitted with `exchangeBreakEven: false`; on OpenAI Codex
+an exchange rides along when the pass edits something else and is deferred when it does not. Old images: run
+end removes images 3+ turns old with no Jev request (also with the model missing), `recall` returns them as
+image blocks (a pattern search only names them), nothing is removed twice; a checkpoint fills its batch with
+old images alone and still obeys break-even, `imageKeepTurns: 0` turns it off; images of an earlier run are
+deferred by the run-end gate and removed with `exchangeBreakEven: false`; `recall` of an omitted exchange
+attaches user images and names tool-result images. The fake session uses Pi's projection shape (an omitted
+entry stays with no messages).
 
 Test through Pi's loader (or `pi -e`), not a plain Node import of the extension. Plain Node imports of
 `@earendil-works/pi-coding-agent` can fail with `ERR_PACKAGE_PATH_NOT_EXPORTED`, even though Pi loads the
@@ -41,7 +69,7 @@ This uses a real Pi, a real main model and real Jev requests. **It costs money o
 main-model requests per turn, plus a few Jev requests (under $0.001 per distilled run).
 
 ```bash
-node test/e2e/run-e2e.mjs --model <provider/id> [--ext <path>]... [--scenario badge|long|history] \
+node test/e2e/run-e2e.mjs --model <provider/id> [--ext <path>]... [--scenario badge|sequential|history|long|topics] \
   [--config '<JSON>'] [--no-guard] [--turns N] [--thinking level]
 ```
 
@@ -49,7 +77,7 @@ node test/e2e/run-e2e.mjs --model <provider/id> [--ext <path>]... [--scenario ba
 |---|---|---|
 | `--model` | (required) | Main model, e.g. `openai-codex/gpt-6-luna` or `anthropic/claude-sonnet-5-5` |
 | `--ext` | none | Extra extension to load; can be repeated (e.g. an auth extension) |
-| `--scenario` | `badge` | `badge`: 3 turns about `StatusBadge` usages. `long`: 2 turns with 10+ sequential tool calls, for the cache pin. `history`: 3 turns for the earlier-conversation feature (below). |
+| `--scenario` | `badge` | `badge`: 3 turns about `StatusBadge` usages. `sequential`: 2 turns with 10+ sequential tool calls, for the question pin. `history`: 3 turns for the earlier-conversation feature. `long`: one autonomous "fix all failing tests" run, for mid-run checkpoints. `topics`: 5 prompts on different topics, the last a follow-up on the first, for old exchanges (both below; use `--turns 5`). |
 | `--config` | `{}` | context-guard settings for this run, e.g. `'{"pinAnthropicCache":false}'` |
 | `--no-guard` | off | Control run without context-guard |
 | `--turns` | `3` | Turns to run (capped by the scenario) |
@@ -80,7 +108,10 @@ How it works:
 | turn 2 first request reads cache | `cacheRead` > 0 on the first request after the edit |
 | turn 3 quotes the removed line exactly | The model quoted line 3 of `src/api/jobStatus.ts` from the "first search" |
 | turn 3 used recall (line was distilled away) | Only when that line is missing from turn 3's payload: the model must have called `recall` |
-| long: turn 1 produced context edits / 10+ tool calls / 10+ model requests | The long scenario really was long and sequential |
+| sequential: turn 1 produced context edits / 10+ tool calls / 10+ model requests | The sequential scenario really was long and sequential |
+| long: the suite is green afterwards | `npm test` passes in the fixture after the run |
+| long: at least one mid-run checkpoint (forced) | Only with `midRunBreakEven: false` in `--config`: a `mid-run` record was written (a default run may rightly make none) |
+| long: request after the checkpoint at request N reads the cached prefix | Anthropic: the first request after each checkpoint reads at least what the run's first request read and wrote (system prompt and question) |
 | history: turn 2 produced context edits | The broad search in turn 2 was distilled |
 | history: turn 3 names exactly the clients that retry on 429 | The answer names the 9 clients whose fixture file has `=== 429`; a client named only to say it does *not* retry is allowed |
 | anthropic requests stay within 4 breakpoints | For `anthropic/…` models: the pin never exceeds the limit |
@@ -93,6 +124,59 @@ summary has a `history` block: `keptApiRetryLines` / `kept429Lines` (src/api ret
 distilled text), the turn-2 Jev results and ms, turn 3's tool calls and whether it used `recall`, and the
 clients turn 3 named. Results are in [JEV.md](JEV.md#earlier-conversation-historyexchanges).
 
+**`long` scenario.** `test/e2e/make-long-fixture.mjs` generates a small Node project (`ledgerly`: money
+in integer cents, business-day calendars, CSV, pricing, a ledger report; 182 tests). Ten bugs sit in
+different modules, in up to three layers: an import-time `ReferenceError` hides a whole test file, and two
+tests fail first because of one bug and then, once it is fixed, because of another. A full `npm test` prints
+12k characters when green and 25–30k with the failures, so every rerun is a large, quickly superseded tool
+output. The prompt asks the agent to fix one bug at a time and run the plain suite after every fix, without
+asking anything. (`--fixed` writes the bug-free version, `--bugs 1,4,…` a subset.) GPT-6 Luna needs 26–37
+requests; Claude Sonnet 5.5 needs 8–14, because it batches several fixes per command and pipes test output
+into `grep`/`head`. Load `--ext test/e2e/plain-test-runs.ts` for Claude to strip a pipe after `npm test`
+(it logs each rewrite to stderr), so its suite runs return full logs like GPT's.
+
+The summary gets a `long` block: per-request contexts (input + cacheRead + cacheWrite) and a sparkline,
+totals of input/cacheRead/cacheWrite/output and the main-model cost Pi reports, every `context-guard`
+record with its phase, saved characters, Jev ms and cost, and the request before and after it (context,
+cacheRead, cacheWrite), whether the suite is green, `recall` calls, and repeated non-suite tool calls (lost
+outputs searched again). Results are in [RESULTS.md](RESULTS.md#long-autonomous-runs) and in
+[CACHE.md](CACHE.md#mid-run-checkpoints).
+
+**`topics` scenario.** Five prompts in the generated StatusBadge repo: which API clients retry on 503; a
+README paragraph for `StatusBar`; a count of HTTP 500 log lines; where `StatusBadge` is used; and "back to
+the API clients that retry on 503: which of them also retry on 502?". The summary gets a `topics` block:
+the context of each prompt's first request and its usage, the exchange results (phase, prompt, outcome,
+P), the small-output results, the last prompt's tool calls and whether it used `recall`, and whether its
+answer names exactly the clients whose file has both `=== 503` and `=== 502` (a crude check: it fails on
+answers that also list clients to say they do not qualify). Results are in [RESULTS.md](RESULTS.md#examples) (example D) and in
+[CACHE.md](CACHE.md#old-exchanges).
+
+**API-shape probe.** `CG_OMIT_AT_TURN=2 node test/e2e/run-e2e.mjs --model … --no-guard --ext
+test/e2e/omit-probe.ts` omits the session's first exchange after turn 2 the way the guard does, so turn 3
+sends a stub user message right before the next user message, without the omitted assistant and
+tool-result messages. Both GPT-6 Luna and Claude Sonnet 5.5 (`pi-claude-auth`) accepted it and answered.
+
+**Old images, live** (2026-10-08, Claude Sonnet 5.5 with `pi-claude-auth`, ad-hoc): six 1200×800 PNGs read
+one per message, then a follow-up about the second one. Run end removed the four images 3–6 turns old with
+no Jev request (record `requests: 0`, 19.9k chars ≈ 5k tokens); the next request carried 2 images instead
+of 6 (input 14.1k → 9.5k tokens). Told not to read the file again, the model called
+`recall({"entryId":…})`, got the image back and read the code in it correctly.
+
+**Session replay.** `node test/e2e/replay-session.ts <session.jsonl> [--from N] [--all] [--summary]` rebuilds Pi's
+projection at every turn of a real session file and prints what the mid-run gates decide (run start, eligible
+large and small outputs, old exchanges, batch size, break-even) and the run-end candidates. No model calls,
+nothing leaves the machine. Without the live cache log it uses the question pin, so its break-even is the
+conservative case.
+
+**Whole-item probe.** `node test/e2e/probe-items.mjs [--reps 2] [--sessions <topics no-guard run dir>]…`
+sends the hand-written exchanges and small outputs in `test/e2e/probe-data/` (and the `topics` exchanges)
+to Jev and prints P per item and the accuracy per threshold (results in [JEV.md](JEV.md#old-exchanges)).
+
+**Anthropic TTL experiment.** `node test/e2e/ttl-probe.mjs [--arms refresh,control,ttl1h,idle,idle1h]`
+runs `test/e2e/ttl-probe.ts` with Claude: does a cache entry stay alive while later requests read a longer
+prefix, and does it expire when nothing reads it for 6 minutes? About 9 minutes and roughly $0.9 at list price for all five arms.
+Results: [CACHE.md](CACHE.md#do-cache-entries-on-the-read-path-stay-alive-measured).
+
 With the guard on, the badge scenario has 9 checks, plus 1 for Anthropic models. Most guard checks are
 skipped with `--no-guard`.
 
@@ -118,30 +202,37 @@ the runner disables installed extensions:
 
 ```bash
 node test/e2e/run-e2e.mjs --model anthropic/claude-sonnet-5-5 --ext ~/.pi/agent/npm/node_modules/pi-claude-auth
-node test/e2e/run-e2e.mjs --model anthropic/claude-sonnet-5-5 --ext ~/.pi/agent/npm/node_modules/pi-claude-auth --scenario long
-node test/e2e/run-e2e.mjs --model anthropic/claude-sonnet-5-5 --ext ~/.pi/agent/npm/node_modules/pi-claude-auth --scenario long --config '{"pinAnthropicCache":false}'
+node test/e2e/run-e2e.mjs --model anthropic/claude-sonnet-5-5 --ext ~/.pi/agent/npm/node_modules/pi-claude-auth --scenario sequential
+node test/e2e/run-e2e.mjs --model anthropic/claude-sonnet-5-5 --ext ~/.pi/agent/npm/node_modules/pi-claude-auth --scenario sequential --config '{"pinAnthropicCache":false}'
 node test/e2e/run-e2e.mjs --model openai-codex/gpt-6-luna
 node test/e2e/run-e2e.mjs --model openai-codex/gpt-6-luna --no-guard
 node test/e2e/run-e2e.mjs --model openai-codex/gpt-6-luna --scenario history
 node test/e2e/run-e2e.mjs --model openai-codex/gpt-6-luna --scenario history --config '{"historyExchanges":0}'
+# mid-run checkpoints (up to 4 in parallel)
+node test/e2e/run-e2e.mjs --model openai-codex/gpt-6-luna --scenario long --no-guard
+node test/e2e/run-e2e.mjs --model openai-codex/gpt-6-luna --scenario long --config '{"midRun":false}'
+node test/e2e/run-e2e.mjs --model openai-codex/gpt-6-luna --scenario long
+node test/e2e/run-e2e.mjs --model anthropic/claude-sonnet-5-5 --ext ~/.pi/agent/npm/node_modules/pi-claude-auth --ext test/e2e/plain-test-runs.ts --scenario long
 ```
 
-### Results so far (Pi 1.0.3)
+### Results (Pi 1.0.3)
 
 | Run | Result |
 |---|---|
 | GPT-6 Luna, badge | 36,113 → 1,501 chars, 2 Jev requests, 550 ms, $0.00083; turn 3 used `recall` with a regex; 9/9 checks pass |
 | Claude Sonnet 5.5, badge | 36,113 → 3,066 chars, 2 Jev requests, 1.18 s, $0.00084; turn 2 cacheRead 5,777 / cacheWrite 2,036; turn 3 used `recall`; 10/10 checks pass |
-| Claude Sonnet 5.5, long, with / without pin | first request after the run: cacheRead 5,810 / 1,693, cacheWrite 8,913 / 13,068 |
+| Claude Sonnet 5.5, sequential, with / without pin | first request after the run: cacheRead 5,810 / 1,693, cacheWrite 8,913 / 13,068 |
 | GPT-6 Luna, badge, with / without guard | turn 2 cacheRead 2,560 / 13,824 |
-| GPT-6 Luna, badge, after the history feature | 36,113 → ~2.3k chars; 9/9 checks pass (an earlier run in parallel with 4 others had one OpenAI cache miss on turn 2: 8/9) |
+| GPT-6 Luna, badge, with history (default) | 36,113 → ~2.3k chars; 9/9 checks pass (a run in parallel with 4 others had one OpenAI cache miss on turn 2: 8/9) |
 | GPT-6 Luna / Claude Sonnet 5.5, history on / off | See [JEV.md](JEV.md#earlier-conversation-historyexchanges): 2 rounds × 4 runs |
+| GPT-6 Luna / Claude Sonnet 5.5, long (mid-run checkpoints) | See [RESULTS.md](RESULTS.md#long-autonomous-runs); all suites green, no `recall` |
+| GPT-6 Luna / Claude Sonnet 5.5, topics, guard on / off | See [RESULTS.md](RESULTS.md#examples), example D |
 
 See [CACHE.md](CACHE.md) for what the cache numbers mean.
 
 ## Real-world examples (`examples.mjs`)
 
-The README's "Examples (measured)" section comes from `test/e2e/examples.mjs`. It runs the same prompts on
+The examples in [RESULTS.md](RESULTS.md#examples) come from `test/e2e/examples.mjs`. It runs the same prompts on
 real public code twice, with context-guard on and with it off (`--no-guard`), and compares the first
 request of the follow-up turn.
 
@@ -159,7 +250,7 @@ node test/e2e/examples.mjs --jobs 2
 
 Each run gets its own workspace, session id and `PI_CONTEXT_GUARD_CONFIG` file (defaults). The Pi package is
 public (npm); nothing private is sent to OpenRouter. Example B also records the test log's size
-(`testLogChars`, which must be ≥ 8,000 to be a fair example; it was 14.4k).
+(`testLogChars`, which must be ≥ 8,000 to be a fair example; it was 14.4–15.8k).
 
 Output: `test/e2e/out/examples-<stamp>/` with `results.md` (the table), `results.json`, and one directory per
 run (`A-claude-on`, …) holding `summary.json` (per turn: tool calls, tool-result sizes, edits with kept/total
@@ -171,17 +262,15 @@ model saw.
 Pi from list prices (most of it Claude in example A; GPT runs used a subscription). About 5 minutes with 4
 parallel jobs.
 
-**Findings of the first full run** (details in the README):
-
-- Next-request context, on / off: A Claude 15.5k / 28.2k tokens, A GPT 11.5k / 34.3k, C GPT 9.2k / 35.3k;
-  all follow-ups correct, none needed `recall`.
-- The first request after an edit cost more with the guard on (cache rewrite): A Claude $0.033 / $0.016.
-- B was not distilled at all: the failing test command is an error result, which context-guard then never
-  edited. Fixed by `distillErrors` (default on); a rerun distilled the log 15.8k → 8.6k chars, keeping the
-  3 failures with their assertion messages, and turn 2 still fixed the bug.
-- In C, `read` outputs carry Pi's two-line "[Showing lines …]" note, so omission ranges at the end of a piece
-  named two line numbers past the file lines (e.g. "lines 466–502" of a 1–500 read). Fixed: ranges are now
-  clamped to the file lines and a removed note gets no omission line.
+**Findings** (details in [RESULTS.md](RESULTS.md#examples)):
+
+- Next-request context, on / off: A Claude 15.5k / 28.2k tokens, A GPT 11.5k / 34.3k, C GPT 9.2k / 35.3k,
+  B GPT 5.7k / 7.5k; all follow-ups correct, none needed `recall`.
+- The first request after an edit costs more with the guard on (cache rewrite): A Claude $0.033 / $0.016.
+- B: the failing test log (an error result, distilled because `distillErrors` is on) went 15.8k → 8.6k
+  chars, keeping the 3 failures with their assertion messages.
+- C: `read` outputs end with Pi's "[Showing lines …]" note; omission ranges are clamped to the file lines
+  and a removed note gets no omission line.
 
 ## Manual TUI check (tmux)
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "pi-context-guard-jev",
-  "version": "0.1.0",
-  "description": "Keep big tool outputs from bloating your Pi context: after each run, the Jev classifier keeps only the lines your work still needs, verbatim and recallable, with savings in the status bar. Claude + OpenAI, cache-aware.",
+  "version": "0.3.1",
+  "description": "Pi extension that removes tool output the agent no longer needs from its context. The Jev classifier keeps the needed lines verbatim; stale small outputs, unrelated old exchanges and old screenshots become one-line notes. Everything can be recalled.",
   "type": "module",
   "license": "MIT",
   "author": "alandarev",