pi-context-guard-jev 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 alandarev
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,294 @@
-# Temporary Holding Version
+# pi-context-guard-jev
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**Your agent ran a 50 KB search to answer one question. Every later request still carries all 50 KB.
+This Pi extension keeps the lines that mattered.**
+
+![A broad 496-line search is distilled after the run; the footer shows about 10.1k tokens kept out of context](https://raw.githubusercontent.com/alandarev/pi-context-guard-jev/master/assets/screenshot.png)
+
+When a run finishes, the extension asks the Jev classifier (by TypeSafe) which parts of each large tool
+output the work still needs: a broad `rg`, a full-file `read`, a failing test log. It then uses Pi's
+append-only `context_edit` entries to replace each output with only those parts, copied verbatim. Later
+turns carry the answer and its evidence instead of everything that was searched. The raw output stays in
+the session file, and the model can fetch it again with the `recall` tool.
+
+```bash
+pi install npm:pi-context-guard-jev     # needs an OpenRouter key: OPENROUTER_API_KEY or /login
+```
+
+## Why use it
+
+- **Most of a session's context is tool output that stopped mattering.** Pi keeps every output for the
+  rest of the session, so every later request re-sends it, the context window fills, and compaction (a
+  lossy summary of everything) comes sooner. In the [measured examples](#examples-measured), the next
+  request carried **9.2k instead of 35.3k tokens** after reading a 164 KB file, **11.5k instead of 34.3k**
+  after a code search, and **5.7k instead of 7.5k** after a failing test run. Every follow-up was still
+  answered correctly.
+- **Verbatim, never rewritten.** Jev is a classifier: it answers yes/no and multiple-choice questions and
+  does not write text. Kept lines are copied byte for byte, and each removed stretch becomes one line that
+  says what was there (`[… 79 lines omitted: matches in dist/core/tools/grep.js, …]`). Unlike a summary,
+  nothing the model sees afterwards is paraphrased or invented.
+- **Nothing is lost.** The original stays in the session file. The model can `recall` it (whole, by
+  regex, or by line range), and `/tree` back to before the edit restores it.
+- **Judged against the work, not by size.** Truncation and size limits can't tell a needed line from
+  noise. Jev sees the question, the final answer and the earlier conversation, and decides chunk by chunk,
+  so lines the ongoing task needs survive even when this run's question was about something else. A
+  failed command the agent already got past is removed outright.
+- **Cheap and quick.** $0.0004–0.003 and 0.6–1 s per qualifying run (measured). It runs once, after the
+  run has finished, never while the agent works.
+- **Claude and OpenAI, cache-aware.** Tested with Claude Sonnet 5.5 and GPT-6 Luna. On Anthropic it adds a
+  cache breakpoint at the previous question, so the request after an edit still reads the conversation
+  up to there from cache.
+- **You can see what it does.** The footer shows `🛡 −10.1k · 1`. `/guard` lists every decision with Jev's
+  probabilities, and `/guard off` turns it off for good.
+
+**Trade-offs:**
+- **It sends data to a third party.** Tool output goes to OpenRouter and TypeSafe ([Privacy](#privacy)).
+- **It can cost a little more in short sessions.** The first request after an edit rewrites the prompt
+  cache, so the first gain is context room; the bill shrinks only in longer sessions
+  ([Examples](#examples-measured)).
+- **Follow-ups may need `recall`.** A question that goes somewhere new can need something that was
+  removed.
+
+## How it works
+
+1. **A run finishes.** When Pi is about to settle a completed run (`agent_before_settle`), the extension
+   looks at that run only: everything after the last user message.
+2. **It picks candidates.** These are text-only tool results of at least 4,000 characters, including the
+   output of failed commands. The run is
+   skipped if the candidates add up to less than 8,000 characters.
+3. **It chunks each output.** grep-style output is grouped by file. Other output is cut into windows of
+   roughly 0.8–4k characters, with breaks at blank lines where possible.
+4. **It sends one Jev request per segment** (up to 32,000 characters and 40 chunks). Each request asks three
+   kinds of question about the user's question and the final answer, and, when there is one, the earlier
+   conversation of the session (so output that the ongoing work still needs is kept even if this run's
+   question was about something else):
+   - `keep_whole`: is all of this output still needed?
+   - `focus`: does the answer use most of it, none of it, or a few chunks (and if so, which one matters most)?
+   - one yes/no question per chunk: is this chunk needed?
+5. **It writes a `context_edit` per result.** The edit keeps the needed chunks verbatim, replaces each run of
+   removed chunks with an omission line, and starts with a header that tells the model how to
+   `recall` the full output. If the kept part would still be more than 60% of the original, the result is
+   left as it is.
+
+Errors, timeouts and unclear answers always mean *keep*. For the full algorithm, see
+[docs/DESIGN.md](docs/DESIGN.md).
+
+## Install
+
+```bash
+pi install npm:pi-context-guard-jev
+# or from GitHub
+pi install git:github.com/alandarev/pi-context-guard-jev
+# or, for development, load it for one session from a checkout
+pi -e ./src/index.ts
+```
+
+### Requirements
+
+- Pi ≥ 1.0.3
+- An OpenRouter credential (`OPENROUTER_API_KEY` or `/login`) for the default classifier,
+  `openrouter/~typesafe/jev-latest`. Any classifier model in Pi's catalog can be set with the `model`
+  setting, but only the default has been tested.
+
+If the model or credential is missing, the extension does nothing and the status bar says so.
+
+## Privacy
+
+**This extension sends data to a third party, and it is on by default.** After every run that qualifies,
+each Jev request sends the following to OpenRouter, which forwards it to TypeSafe:
+
+- the user's question (up to 4,000 characters),
+- the final answer (up to 6,000 characters),
+- the assistant's notes written during the run (up to 2,000 characters),
+- excerpts of the earlier conversation in this session: the latest compaction or branch summary (up to
+  2,000 characters), the first prompt (up to 1,000), and the last 3 earlier exchanges, each one prompt (up to
+  800) and the last assistant text after it (up to 1,200). Tool calls and tool outputs of earlier runs are not
+  included. Set `historyExchanges` to `0` to send none of it,
+- the tool name and arguments,
+- the full text of the large tool output being judged. This can include source code, logs or anything else
+  the tools returned.
+
+Small results, edit/write results and runs below the size limits are never sent.
+
+`/guard off` turns the extension off and saves that choice in the settings file, so it stays off in later
+sessions. See OpenRouter's data policy:
+<https://openrouter.ai/docs/guides/privacy/data-collection>.
+
+## Status bar
+
+| Text | Meaning |
+|---|---|
+| `🛡 0 saved` | On and ready; nothing is distilled on this branch yet |
+| `🛡 distilling…` | Jev is judging the outputs of the run that just finished |
+| `🛡 −4.2k · 1` | ≈ tokens kept out of context · distilled results: about 4.2k tokens are currently kept out of context, by 1 distilled tool result |
+| `🛡 guard off` | Turned off with `/guard off` or `"enabled": false` |
+| `🛡 no openrouter key` | No credential for the classifier's provider (the provider name is filled in) |
+| `🛡 Jev model not found` | The configured `model` is not in Pi's classifier catalog |
+
+The token figure is Pi's usual estimate, characters ÷ 4. It counts what is kept out of the model's context
+**on the current branch right now**: original length minus replacement length, for every distilled result
+that is still in the context. It goes down after `/tree` to a point before an edit, and after compaction
+removes distilled results. It is not a measured token count and not a cumulative total.
+
+## `/guard` command
+
+| Command | Effect |
+|---|---|
+| `/guard` or `/guard status` | Shows whether the guard is on, the model, the savings on this branch, Jev runs, requests and cost, and the last run's result for each tool output, including Jev traces such as `kw.15 focus=chunk_4:.52 keep 9/29` |
+| `/guard on` / `/guard off` | Turns the guard on or off and saves `enabled` to the settings file |
+| `/guard reload` | Re-reads the settings file and shows any warnings again |
+
+Settings are read at session start and on `/guard reload`.
+
+## `recall` tool
+
+Every distilled result starts with `[context-guard]` and ends its header with
+`Full output: recall({"entryId":"…"}).` The model can call:
+
+| Parameter | Meaning |
+|---|---|
+| `entryId` (required) | The entry id from the header |
+| `pattern` | JavaScript regex (≤ 500 chars, 1 s limit); returns only matching lines, prefixed with their line numbers |
+| `offset` | First line to return, 1-based. When `pattern` is set, this counts matching lines. |
+| `limit` | Most lines to return |
+
+Like Pi's `read` tool, the output is capped at 2,000 lines and 50 KB, and a note says how to continue.
+
+## Configuration
+
+The settings file is `~/.pi/agent/context-guard.json` (inside Pi's agent directory), or the path in
+`PI_CONTEXT_GUARD_CONFIG`. Every key is optional. Values that are missing, the wrong type or out of range
+fall back to the default.
+
+| Key | Default | Meaning |
+|---|---|---|
+| `enabled` | `true` | Master switch (`/guard on` and `/guard off` write it) |
+| `model` | `"openrouter/~typesafe/jev-latest"` | Classifier as `provider/id`; the first `/` separates the provider |
+| `minResultChars` | `4000` | Ignore tool results shorter than this many characters |
+| `minRunChars` | `8000` | Skip the run if its candidates add up to less than this |
+| `keepWholeThreshold` | `0.7` | Keep a segment untouched if P(`keep_whole`) is at least this |
+| `focusWholeThreshold` | `0.6` | Keep a segment untouched if `focus` picks "whole" with at least this probability |
+| `chunkKeepThreshold` | `0.6` | Keep a chunk if P(chunk needed) is at least this |
+| `noneThreshold` | `0.5` | Remove a segment entirely if `focus` picks "none" with at least this probability and no chunk passes |
+| `maxKeepRatio` | `0.6` | Skip the edit if the kept chunks are more than this fraction of the original |
+| `keepCitedFiles` | `false` | Also keep grep chunks for files that the final answer names (see DESIGN.md for why it is off) |
+| `timeoutMs` | `8000` | Total time budget for all Jev requests of one run; unfinished parts are kept |
+| `concurrency` | `6` | Jev requests in parallel |
+| `maxSegmentChars` | `32000` | Most characters of one output per Jev request (keeps the worst-case request under Jev's 32k-token context, see docs/JEV.md) |
+| `maxChunksPerSegment` | `40` | Most chunks per Jev request |
+| `excludeTools` | `["edit", "write"]` | Tools whose results are never distilled |
+| `distillErrors` | `true` | Also distill error results, such as the log of a failing test or build; Jev is told the call failed, and that its details are usually no longer needed once the agent got past it |
+| `historyExchanges` | `3` | Earlier exchanges (prompt + last assistant text) sent to Jev so it judges relevance against the ongoing work; `0` sends no earlier conversation at all (integer 0–10) |
+| `pinAnthropicCache` | `true` | Add an Anthropic cache breakpoint at the previous user question ([docs/CACHE.md](docs/CACHE.md)) |
+
+Example:
+
+```json
+{ "minResultChars": 6000, "excludeTools": ["edit", "write", "bash"] }
+```
+
+## What it does not touch
+
+- user and assistant messages, custom messages
+- tool results that contain images
+- error results, if `distillErrors` is `false`
+- results of excluded tools (`edit`, `write` by default)
+- results shorter than `minResultChars`, and runs below `minRunChars`
+- aborted or failed runs
+- results that are already distilled, or that another extension already edited
+- single lines too long for one Jev request (kept as they are)
+- earlier runs: only the run that just finished is edited
+
+## Limitations
+
+- **Lossy by design.** Relevance is judged against the run's question and answer and the earlier
+  conversation (the ongoing task), so a follow-up that goes somewhere new may need something that was
+  removed. The header and omission lines say what is missing, and `recall` returns it.
+- **Latency.** Settling waits for Jev. This adds about 0.5–1.2 s after each qualifying run (measured), and
+  never more than `timeoutMs` (8 s).
+- **Cost.** Each qualifying run costs a few Jev requests, measured at under $0.001 per run. The first prompt
+  after an edit also has a one-time partial prompt-cache miss ([docs/CACHE.md](docs/CACHE.md)).
+- **Steering.** A message sent during a run starts a new span. Tool results from before it are never
+  distilled.
+
+## Examples (measured)
+
+Real runs on public code with Pi 1.0.3: each example ran once with the guard on and once with it off
+(`--no-guard`), from the same prompts. These are single runs, not a benchmark: the main model takes a
+different path each time. Reproduce with `node test/e2e/examples.mjs` ([docs/TESTING.md](docs/TESTING.md)).
+
+**A. Searching a real codebase.** In a copy of the published Pi package (`dist/` without source maps and
+the minified bundle, plus `docs/`), the agent was asked where Pi decides to auto-compact, starting with
+`rg -n -i compact dist docs`. Pi's bash tool returned the last 50 KB of that search. Jev kept the matches
+in the compaction module, the settings and the decision points in `agent-session.js`, and dropped matches
+in the grep tool, HTML export templates, the SDK and so on. Then the agent was asked "which of those
+settings can a user change, and what are their defaults?".
+- Claude Sonnet 5.5: 50,413 → 16,210 chars (141 of 431 lines kept). The first turn-2 request carried
+  **15.5k tokens instead of 28.2k**. The answer was correct (`enabled` true, `reserveTokens` 16384,
+  `keepRecentTokens` 20000), without `recall`.
+- GPT-6 Luna: 51,216 → 13,023 chars (107 of 385 lines), and a 5.4k read of `compaction.js` → 1.8k; two
+  smaller outputs were left whole. Turn 2 carried **11.5k tokens instead of 34.3k** and was correct, without
+  `recall`.
+
+**C. Reading a 164 KB source file.** The agent read `dist/core/agent-session.js` (3,535 lines) in
+seven 500-line pieces and explained how a prompt reaches the model. Jev kept 45–164 lines of each of the first
+four pieces and removed the last three completely (each became a one-line stub). The follow-up asked for a
+detail the explanation never mentioned: the exact error thrown when a prompt arrives during compaction.
+That line was among the kept ones, so the agent quoted it exactly without `recall`. The turn-2 request
+carried **9.2k tokens instead of 35.3k**.
+
+**B. A failing test run.** In a copy of this repository with one injected off-by-one bug, the agent ran
+the unit tests (a 15.8k-character log, 3 failures among 116 tests) and reported the cause. Jev kept the
+3 failures with their assertion messages and the failing-test summary, and dropped the passing tests:
+15.8k → 8.6k chars. Turn 2 ("now fix it") carried **5.7k tokens instead of 7.5k**, and both runs fixed
+the bug. (In the first run of this example, failing commands were never distilled; `distillErrors` was
+added because of it.)
+
+What the model sees after distilling (example A, Claude; the first lines of the real replacement):
+
+```text
+[context-guard] Distilled the output of bash `rg -n -i compact dist docs`: kept 141 of 431 lines (15.1k of 50.4k chars); the rest was judged not needed for the answer. Full output: recall({"entryId":"1704787f"}).
+[… 79 lines omitted: matches in dist/core/tools/grep.js, dist/core/session-manager.js, dist/core/export-html/template.js, dist/core/export-html/template.css, dist/core/sdk.js and 5 more …]
+dist/core/compaction/compaction.d.ts:25:    /** Extension-specific data (e.g., ArtifactIndex, version markers for structured compaction) */
+dist/core/compaction/compaction.d.ts:28:export interface CompactionSettings {
+dist/core/compaction/compaction.d.ts:33:export declare const DEFAULT_COMPACTION_SETTINGS: CompactionSettings;
+dist/core/compaction/compaction.d.ts:54:/** Estimate projected context without trusting usage captured before a later edit or compaction. */
+dist/core/compaction/compaction.d.ts:57: * Check if compaction should trigger based on context usage.
+dist/core/compaction/compaction.d.ts:59:export declare function shouldCompact(contextTokens: number, contextWindow: number, settings: CompactionSettings): boolean;
+[… 44 lines omitted: matches in dist/core/compaction/compaction.d.ts, dist/core/compaction/branch-summarization.d.ts, dist/core/compaction/index.d.ts, dist/core/session-manager.d.ts, dist/core/cache-stats.js and 2 more …]
+```
+
+| Example | Model | Tool output before → after | Jev | Next request's context, on / off | Main-model cost of turn 2, on / off | Follow-up |
+|---|---|---|---|---|---|---|
+| A. code search | Claude Sonnet 5.5 | 50.4k → 16.2k chars | 3 requests, 0.8 s, $0.0012 | 15.5k / 28.2k tokens | $0.033 / $0.016 | correct, no `recall` |
+| A. code search | GPT-6 Luna | 56.6k → 14.8k chars (2 outputs; 2 more, 15k, left whole) | 5 requests, 0.6 s, $0.0015 | 11.5k / 34.3k tokens | $0.0012 / $0.0005 | correct, no `recall` |
+| C. 164 KB file | GPT-6 Luna | 163k → 26k chars (7 reads) | 7 requests, 0.9 s, $0.0027 | 9.2k / 35.3k tokens | $0.0007 / $0.0005 | correct, no `recall` |
+| B. failing test log | GPT-6 Luna | 15.8k → 8.6k chars | 1 request, 0.8 s, $0.0004 | 5.7k / 7.5k tokens | $0.0022 / $0.0016 | fixed in both runs |
+
+**When it doesn't help.**
+
+- **The first request after an edit costs more, not less.** The prompt cache only matches up to the
+  first changed message, so the next request writes the shorter history again instead of reading the long
+  one from cache. In A that made Claude's turn-2 request cost $0.033 instead of $0.016, and GPT's $0.0012
+  instead of $0.0005; OpenAI read nothing from cache on that request (in the fixture runs it read 2,560
+  tokens instead of 13,824: a partial miss).
+  At the per-token prices in these runs, the smaller context earns that back after about 7 more requests
+  (Claude, A), about 3 (GPT, A) or about 1 (GPT, C). Before that, the benefit is a smaller context window,
+  not a lower bill. Anthropic requests also get a cache breakpoint at the previous question, which raised
+  the first request's cache read from 1,693 to 5,810 tokens in a long run ([docs/CACHE.md](docs/CACHE.md)).
+- **Small or fully needed outputs are left whole.** In A, GPT's 9k-character read of `agent-session.js` was
+  `not-worth` (too much of it was needed) and a 6k narrowed search was judged needed whole.
+
+Earlier numbers on the generated StatusBadge test repo (96% removed, `recall` of a removed line) are in
+[docs/TESTING.md](docs/TESTING.md#results-so-far-pi-103).
+
+## Documentation
+
+- [docs/DESIGN.md](docs/DESIGN.md): architecture and algorithm
+- [docs/CACHE.md](docs/CACHE.md): prompt-cache impact (Anthropic, OpenAI) and the cost model
+- [docs/JEV.md](docs/JEV.md): the Jev questions, probe results and threshold choices
+- [docs/TESTING.md](docs/TESTING.md): unit tests, live end-to-end tests, manual TUI check
+
+## License
+
+MIT

package/docs/CACHE.md

@@ -0,0 +1,149 @@
+# Prompt-cache impact
+
+**Summary.** An edit changes the context from the first distilled tool result onwards. The next prompt
+therefore has a one-time partial cache miss: everything before the edited result can still be read from the
+cache, and everything after it is written again. After that, each request carries the smaller, distilled
+history. On Anthropic models, a long run can push the reusable cache entry out of reach. The extension adds
+one cache breakpoint to prevent that (measured below). On OpenAI, caching is automatic. Pi's Codex transport
+also loses its delta continuation once after an edit.
+
+Pi facts were checked in the Pi 1.0.3 source (`@earendil-works/pi-ai/dist/api/anthropic-messages.js`,
+`openai-codex-responses.js`). Provider facts come from the primary docs, read on 2026-10-06 (see
+[Sources](#sources)). Numbers marked *measured* come from the live tests in [TESTING.md](TESTING.md). Single
+runs, not averages.
+
+## Anthropic
+
+### What Anthropic does (documented)
+
+- At most **4** cache breakpoints per request.
+- For each breakpoint, the lookup goes back **at most 20 positions** to find an earlier cache entry.
+  Consecutive `tool_use` blocks count as one position, and so do consecutive `tool_result` blocks.
+- Entries with a longer TTL must come before entries with a shorter TTL.
+- Reading an entry refreshes its TTL. The docs do **not** say that reading a longer prefix refreshes a
+  shorter entry that was written separately.
+- Pricing relative to base input: 5-minute write 1.25×, 1-hour write 2×, read 0.1× (0.05× on Opus 5.5).
+- The minimum cacheable prompt is 512 tokens on Sonnet/Opus 5.5.
+
+### Where Pi puts breakpoints
+
+| Breakpoint | Where |
+|---|---|
+| tools | the last tool definition |
+| system | the system prompt block |
+| messages | rolling: the last user or system message only |
+
+With an API key that makes 3 breakpoints. With Anthropic OAuth, Pi also adds a `"You are Claude Code…"`
+identity block as `system[0]`, with its own breakpoint, which makes 4. With `PI_CACHE_RETENTION=long`, all
+breakpoints get `ttl: "1h"` where the model supports it.
+
+The `pi-claude-auth` extension, used for the Claude subscription tests, rewrites the payload differently.
+It moves the system prompt into the first user message and adds a billing block that is not cached. The
+measured requests never had more than 4 breakpoints.
+
+### The problem
+
+The first request of a run has the question as its last message, so it writes a cache entry ending at the
+question. After the run is distilled, the next prompt's prefix matches up to the first edited result, which
+includes the question. But that entry was written by the run's first request, and the rolling breakpoint is
+now at the end of the conversation. If the run had more than about 20 positions between the question and the
+end (a dozen sequential tool calls is enough), the lookback never reaches the question's entry. Only tools and
+system are read, and the whole history is written again.
+
+### The fix, as implemented (`src/cache-pin.ts`)
+
+In `before_provider_request`, for models with `api === "anthropic-messages"` (when `enabled` and
+`pinAnthropicCache` are on):
+
+1. Find Pi's rolling `cache_control` (the last one in `messages`). Without one, caching is off: do nothing.
+2. Skip trailing mid-conversation `system` messages (effort or tool changes). Then find the most recent user
+   message with text that is **not** the last message. Tool-result-only user messages do not count. This is
+   the run's question during a run, and the previous question on the first request of a new prompt.
+3. Put a `cache_control` on that message's last text block (string content is converted to one text block
+   first). If it already has one, leave it. The TTL must keep Anthropic's order (every breakpoint before it
+   at least as long, every one after it no longer): the rolling breakpoint's TTL is used when that holds,
+   otherwise the other TTL; if neither fits, nothing is pinned (`ttl-conflict`). Pi itself always uses one
+   TTL for every breakpoint, so in practice the pin copies it.
+4. If that makes more than 4 breakpoints, remove the breakpoint from `system[0]` (the OAuth identity block).
+   The `system[1]` breakpoint covers that prefix too.
+5. If it is still more than 4, undo everything and leave the payload as Pi built it.
+
+During a run, every request reads the question's entry directly, so the entry stays refreshed and the lookback
+limit does not matter. Moving a breakpoint does not change the cached content, so existing entries stay
+valid. The pin is added on every Anthropic request, whether or not anything was distilled.
+
+### Measured
+
+Claude Sonnet 5.5 via `pi-claude-auth`, Pi 1.0.3:
+
+| Case | First request after the distilled run |
+|---|---|
+| Short run (badge scenario), with pin | cacheRead 5,777, cacheWrite 2,036 |
+| Long run (11 sequential tool calls, 36,113 → ~3.3k chars), **with pin** | cacheRead **5,810**, cacheWrite **8,913** |
+| Same long run, **without pin** (`pinAnthropicCache: false`) | cacheRead **1,693** (tools + system only), cacheWrite **13,068** |
+
+Without the pin, the 20-position lookback missed the question's entry. With the pin, it was hit. No request
+had more than 4 breakpoints.
+
+## OpenAI
+
+### What OpenAI does (documented)
+
+- Caching is automatic, with no breakpoints. The whole rendered prefix must match. A change prevents reuse
+  from that point on.
+- GPT-5.6 and later: 1,024-token minimum, about 30 minutes retention, reads cost about 0.1×.
+
+### Pi's Codex transport
+
+Over WebSocket, Pi's `openai-codex` transport sends only the new input items with `previous_response_id`, as
+long as the input only grows. After an edit, the input is no longer an extension of the previous one, so
+Pi sends the full input. The server-side continuation is lost for that request, and the prompt cache is reused
+only up to the first edited item. This happens once, on the first request after the edit.
+
+### Measured
+
+GPT-6 Luna (openai-codex), badge scenario, turn 2 first request:
+
+| Case | cacheRead |
+|---|---|
+| Without the guard (control) | 13,824 |
+| With the guard | 2,560 (only the stable prefix before the first edited item) |
+| With the guard, one earlier run | 0 |
+
+The pin does not apply to OpenAI.
+
+## Cost model
+
+All units are input-token equivalents, with a 5-minute TTL (write 1.25×, read 0.1×).
+
+**Measured, long Claude run, first request after the run:**
+
+| | Read | Write | Cost |
+|---|---|---|---|
+| With pin | 5,810 × 0.1 | 8,913 × 1.25 | **≈ 11.7k** |
+| Without pin | 1,693 × 0.1 | 13,068 × 1.25 | **≈ 16.5k** |
+
+The prompt was the same size in both cases, about 14.7k tokens. The pin moved about 4.1k tokens from write to
+read and made that request about 29% cheaper.
+
+**Assumed sizes, to show the trade-off.** System prompt and tools 15k tokens, earlier history 30k, the run's
+tool output 60k distilled to 5k:
+
+| Next prompt | Cost |
+|---|---|
+| Not distilled (full cache hit) | read 105k × 0.1 → **≈ 10.5k** |
+| Distilled, question entry hit (pin) | read 45k × 0.1 + write 5k × 1.25 → **≈ 10.8k**; every later request then saves about 5.5k |
+| Distilled, only tools and system hit (no pin, long run) | read 15k × 0.1 + write 35k × 1.25 → **≈ 45k** one time; pays back after about 6 requests |
+
+Notes:
+
+- If the user waits longer than the cache TTL before the next prompt, everything is cold anyway, and the
+  edit costs nothing extra.
+- Jev requests do not touch the main model's cache. They cost about $0.0008 per qualifying run (measured).
+- On OpenAI, the one-time cost is the difference between 13,824 and 2,560 cached tokens on one request
+  (measured above), against a smaller context on every later request.
+
+## Sources
+
+- Anthropic, prompt caching: <https://docs.claude.com/en/docs/build-with-claude/prompt-caching>
+- OpenAI, prompt caching: <https://developers.openai.com/api/docs/guides/prompt-caching>