pi-tuck 1.0.3 → 1.0.5

package/CHANGELOG.md

@@ -3,6 +3,40 @@
 All notable changes to pi-tuck. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
 [Semantic Versioning](https://semver.org/).
 
+## [1.0.5] - 2026-10-31
+
+### Fixed
+
+- One oversize tool result no longer leaves a session unusable. A fresh tool round larger than the input room is still refused, since it must
+  reach the model whole; but after that run ended (its reply aborted, or hidden by Pi when it recovered from an overflow), the result was
+  still treated as not yet seen and kept whole, so
+  every later prompt was refused, and a manual `/compact` refused too. Once the session moves on (a new prompt, a manual `/compact`, or a
+  run a custom message starts) it is now ordinary history, and the next cut summarizes it; `tuck_recall` still reads it in full.
+- A cut that is needed when Pi has nothing to compact (no history older than its `keepRecentTokens` since the last cut) now refuses that
+  request only. Before, Pi's "Nothing to compact" or "Already compacted" stayed as a fault, and every later prompt of the session failed on it.
+- No failure outlives its attempt: after any refusal or fault, the next prompt, a manual cut or the end of the run plans afresh from the
+  session as it is. Before, a fault other than a refusal stayed until the session restarted. The one exception is a cut pi-tuck supplied that
+  Pi then failed to write: the session may hold a cut its file lacks, so that fault still stays until the session restarts.
+- A refusal no longer turns semantic recall off. Background indexing read any compaction failure as the host revoking access, so it disabled
+  semantic recall and deleted its index for the rest of the session.
+
+## [1.0.4] - 2026-10-31
+
+### Changed
+
+- When pi-tuck refuses a request because it cannot fit, and the provider has counted far more input than pi-tuck estimates (1.5 times or
+  more), the notice now says so and suggests that a different upstream may count less. Behind a router such as OpenRouter, one upstream can
+  spend about twice the context of others on tool schemas, which is what made pi-tuck refuse at a 32k window with a 16k reserve.
+
+### Fixed
+
+- An omission notice lists each omitted message's locator once: when several text blocks of one message were shortened to nothing, repeats
+  could crowd the list and drop its "Locator list incomplete" flag.
+- A refusal no longer outlives the attempt it was for. When pi-tuck refused a request (a host view or a fresh tool round larger than the
+  input room, or a cut it could not make), a later run that no prompt started, such as a custom message that triggers a turn, was aborted
+  before its first request until the session restarted, even once there was room, and it reported the old refusal. A refusal that was not for
+  want of room was held the same way and also aborted every later prompt. The next run, whatever starts it, now decides afresh.
+
 ## [1.0.3] - 2026-10-31
 
 First npm release since 1.0.1: it also carries everything listed under 1.0.2, which was tagged but not published to npm. Releases are
@@ -59,9 +93,10 @@ First release on npm. 1.0.0 was tagged but never published; its notes below desc
 
 ### Fixed
 
-- A long session whose dialogue no longer fit the summary stalled for tens of seconds at every cut (88 s at 3,900 entries), because
-  the ranked view re-rendered the whole candidate summary for each block it considered. It now keeps the summary's size up to date
-  as blocks join, so a cut takes well under a second and grows linearly with the session. Summaries are byte for byte the same.
+- A long session whose dialogue no longer fit the summary stalled for tens of seconds at every cut, because the ranked view re-rendered
+  the whole candidate summary for each block it considered. It now keeps the summary's size up to date as blocks join, so a cut takes
+  well under a second and grows linearly with the session: 72 s before and 174 ms after at 3,888 entries, measured on one machine with the
+  same synthetic session. Summaries are byte for byte the same.
 
 ## [1.0.0] - 2026-10-30
 
@@ -103,6 +138,7 @@ Tagged, not published.
 Source preview on GitHub, not published to npm. Deterministic compaction with `pi_tuck_recall` and `pi_tuck_search`; semantic search
 opt-in, with a runtime the user had to supply.
 
+[1.0.4]: https://github.com/galyavin/pi-tuck/releases/tag/v1.0.4
 [1.0.3]: https://github.com/galyavin/pi-tuck/releases/tag/v1.0.3
 [1.0.2]: https://github.com/galyavin/pi-tuck/tree/v1.0.2
 [1.0.1]: https://github.com/galyavin/pi-tuck/releases/tag/v1.0.1

package/README.md

@@ -1,74 +1,169 @@
 # pi-tuck
 
-Deterministic compaction for [Pi](https://pi.dev). When a session's context fills up, pi-tuck replaces older turns with a summary built
-from the session itself, not written by a model: no extra model call, no cost, no invented history, and the same input always gives the
-same summary. Nothing is lost: the full history stays in the session file, and the agent gets two tools to read it back, by keyword and
-by meaning.
-
-## Install
+**Compaction for [Pi](https://pi.dev) with no model call, and a history the agent can search by keyword or by meaning, all on your machine.** In our live run, a cut took a median 0.09 s. It costs no tokens. The same session always gives the same summary. And nothing is lost: the agent can read back anything earlier, word for word, and on Apple silicon it can find it by meaning.
 
 ```sh
-pi install npm:pi-tuck@1.0.3
+pi install npm:pi-tuck
+```
+
+## pi-tuck against Pi's built-in compaction
+
+Pi's built-in compaction asks a model to write a summary. pi-tuck builds the summary from the session itself. The two time and context rows compare pi-tuck 1.0.1 with Pi 1.4.0's built-in compaction, where a local model writes the summary. Each measured result in this README names the pi-tuck build or version it was measured on; the release you install may be newer.
+
+| | Pi's built-in (local model writes the summary) | pi-tuck |
+|---|---|---|
+| Model call per cut | One, to a model you pay for or wait on | None |
+| Time per cut | **269 s** (median) | **0.09 s** (median) |
+| Context carried after a cut | 29,200 tokens | 27,800 tokens (about the same) |
+| Same session, same summary | Not guaranteed, since a model writes it | Yes |
+| Can the agent get dropped detail back | No recall tool in Pi 1.4.0. The full history stays in the session file, but the agent has no tool made to search it | Yes. `tuck_recall` reads any original exactly |
+| Search history by meaning | No | Yes, with `tuck_search`, on macOS with Apple silicon. It runs locally, and no text leaves your machine. Elsewhere it searches by keyword |
+| Can the summary invent history | It can, since a model writes it | No, it holds only recorded text |
+| What the summary leaves out | Whatever the model dropped | Tool output only; the agent reads it back exactly |
+| Billed input at the default 60% cut point (simulated) | Not measured | 10–18% less than pi-tuck cutting at the full room. This is not a comparison with the built-in |
+| Answer quality against Pi's own compaction on a local model (`mac-studio/main`) | | **Being measured. Not claimed yet.** |
+| Answer quality against Pi's compaction prompts on a frontier model | | **Untested.** |
+
+The time and context rows are live runs with a 131k window: 27 cuts per arm over 5 chains, interim. Every number comes from a file in this repository. Conditions, sample sizes and links are in [Benchmarks](#benchmarks). The last row is the one that matters most, and it is not finished: see [What is proven, and what is not](#what-is-proven-and-what-is-not).
+
+## Ask your history
+
+A summary cannot hold everything. So pi-tuck lets the agent look things up instead of guess. Say you fixed a login bug two hours ago, and the summary no longer holds the details. You ask:
+
+> "What was the reason we dropped the retry on the token refresh?"
+
+The original message says "removing the backoff loop from the auth renewal". The words differ, so a keyword search finds nothing. The agent calls `tuck_search`, which also searches by meaning:
+
+```json
+{ "query": "why did we drop the retry on token refresh" }
 ```
 
-or straight from GitHub:
+It gets back the original passage, with a pointer such as `#212`. It reads the whole message, exactly as it was written:
+
+```json
+{ "index": 212 }
+```
+
+That is `tuck_recall`. The agent answers from the original text, not from a model's memory of it.
+
+*This example is an illustration. It is not a recorded session.* The call shapes are real. **Semantic search has no measured quality numbers yet.** A test of it is pre-registered in [`benchmark/PREREG-RECALL.md`](benchmark/PREREG-RECALL.md); it has no result yet. The benchmarks below measure cut time, cost and answer retention after cuts, not how well `tuck_search` finds a passage. It runs only on macOS with Apple silicon. Elsewhere, `tuck_search` uses keywords, and the example above would not find its match.
+
+## Install
 
 ```sh
-pi install git:github.com/galyavin/pi-tuck@v1.0.3
+pi install npm:pi-tuck
 ```
 
-Requires Pi 1.1 or later (tested on 1.1.0 and 1.4.0) on macOS or Linux. Semantic search (`tuck_search` by meaning) runs only on macOS
-on Apple silicon; elsewhere `tuck_search` uses keywords and everything else works.
-Start a new Pi session after installing.
+Then start a new Pi session. That is all. pi-tuck takes over Pi's compaction, both automatic and `/compact`. You change no setting.
+
+- Needs Pi 1.1 or later (tested on 1.1.0 and 1.4.0), on macOS or Linux.
+- On macOS with Apple silicon, the first session also downloads a local search model in the background (about 390 MB, once). Until it is ready, search works by keyword. See [Semantic search](#semantic-search).
+- Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.5`.
+- To pin a version, add it: `pi install npm:pi-tuck@1.0.5`.
 
-## What it does
+## How it works
 
-**Compaction.** pi-tuck takes over Pi's compaction, both automatic and `/compact`. A summary keeps:
+When context fills up, pi-tuck replaces the older turns with one summary. The summary keeps:
 
-- the dialogue: what you asked and what the agent answered, in order;
-- the files the agent read, wrote or edited, newest first (up to 60), with pointers to read them back;
-- the agent's other tool calls, such as shell commands, newest first (up to 30), without their output but with pointers to it;
-- your first prompt in the session, word for word (up to 12,000 characters).
+- **The dialogue.** What you asked and what the agent answered, in order, word for word. If it is too long for the room, it shrinks: the oldest assistant text is clipped first, then older prompts. Clipped text says how to read it in full.
+- **The files.** Files the agent read, wrote or edited, newest first (up to 60), each with a pointer to read it back.
+- **The action trail.** The agent's other tool calls, such as shell commands, newest first (up to 30). The output is left out. A pointer leads to it.
+- **Your first prompt**, word for word (up to 12,000 characters).
 
-Tool output is left out of the summary; it stays in the session file. If the dialogue itself is too long for the room left, it is
-shortened, oldest assistant text first, instead of being replaced by a ranked selection. Each summary ends with a short instruction to
-the agent: re-read a file before editing it, and look things up rather than guess. The summary is plain text, one line per record
-(`#12 user: ...`, where 12 is the index `tuck_recall` takes); summaries written by earlier versions are still read.
+Tool output stays in the session file, not in the summary. Each summary ends with a short instruction to the agent: re-read a file before you edit it, look things up, and treat anything not shown as unknown.
 
-**`tuck_recall`** finds earlier text by keyword and reads any original message or tool result exactly, by the index shown in a summary.
-It can also write an original back out to a file.
+The summary is plain text, one line per record, such as `#12 user @2026-10-30T14:00:00Z: text`. The number is the index that `tuck_recall` takes. Summaries from earlier versions are still read.
 
-**`tuck_search`** finds earlier history related to a question even when the wording differs. It combines keyword hits with local
-semantic search (see below) and returns the original text with pointers for `tuck_recall`.
+The agent gets two tools to get anything back:
 
-**Commands**
+- **`tuck_recall`** finds earlier text by keyword. It reads any original message or tool result exactly, by the index shown in a summary. It can also write an original out to a file.
+- **`tuck_search`** finds earlier history related to a question, even when the wording differs. It combines keyword hits with local semantic search and returns the original text with pointers for `tuck_recall`.
+
+Both tools cost about 1,050 tokens of tool description per request.
 
 | Command | Does |
 |---|---|
 | `/tuck-semantic on\|off\|status` | Turns semantic search on or off for this session, or shows its state. |
 | `/tuck-incident on\|next\|off\|status` | Records this session's compaction decisions to a file you can attach to a bug report. |
 
-`/compact` works as usual; pi-tuck does not take custom summary instructions. Pi's own setting `"compaction": { "enabled": false }`
-turns compaction off.
+`/compact` works as usual, but pi-tuck does not take custom summary instructions. Pi's own setting `"compaction": { "enabled": false }` turns compaction off.
+
+## What is proven, and what is not
+
+**Measured so far:** in the live run, a pi-tuck cut took a median 0.09 s, where Pi's built-in compaction with a local model writing the summary took a median 269 s. A cut calls no model, so it costs no tokens. After a cut, pi-tuck carried about as many tokens as the built-in did, not fewer. In simulation, cutting at 60% of the room instead of 100% saved billed input for pi-tuck itself, and answers held in replay.
+
+**Not proven yet: that the summary is as good as a model-written one.** The claim is that pi-tuck compacts as well as an LLM compactor, in far less time and at no cost. The comparator is Pi's own summarize prompts, run on a local model (`mac-studio/main`). The frontier comparison is untested: the model first named for it (`openai/gpt-6-astra`) was retired on 2026-10-31, and its recordings are a labelled partial view that feeds no test. A replacement, GPT-6.1 Terra, is pre-registered in [`benchmark/PREREG-TERRA.md`](benchmark/PREREG-TERRA.md). See [`benchmark/PINS.md`](benchmark/PINS.md). That claim has a pre-registered test. The test is running now. It was written down before any result came in, and a failed test will be reported as failed. When the result is final, it goes in [`benchmark/`](benchmark) and in this README. Until then, this README does not claim that pi-tuck is as good as, or better than, any LLM compactor.
+
+The test has three parts:
+
+1. **A live run** at a 128k window on a local model, with pi-tuck 1.0.1. Ten chains of 30 SWE-bench Verified tasks per arm, with Pi's built-in compaction in one arm and pi-tuck in the other. Official grading.
+2. **A paired replay with probe questions.** The same cuts, made by pi-tuck and by Pi's summarize prompts on the local model, then questions about facts that fell before the cut. Pre-registered rule: the claim holds only if pi-tuck is no more than 3 points worse than the better compactor, with a 95% bound. The comparators are named above and in `benchmark/`.
+3. **A 32k suite**, where tasks are bigger than the room, so cuts must summarise work in progress.
 
-**Small context windows.** pi-tuck keeps room for the model's reply: the larger of the model's output limit and Pi's
-`compaction.reserveTokens` (default 16,384). It refuses a request that cannot fit rather than let the provider cut it off. For a model
-with a window of 32k tokens or less, lower the reserve for that model in Pi's `settings.json`:
+The plan: [`benchmark/PREREG.md`](benchmark/PREREG.md), [`benchmark/PREREG-32K.md`](benchmark/PREREG-32K.md), [`benchmark/PREREG-TERRA.md`](benchmark/PREREG-TERRA.md), and every change after the freeze in [`benchmark/DEVIATIONS.md`](benchmark/DEVIATIONS.md).
+
+Known limits today:
+
+- pi-tuck's summary does not explain why things happened. It holds the words that were said, not a model's reading of them. The agent must look things up. Each look-up costs a tool call.
+- Small windows: an upstream that spends far more context on tool schemas than others (some OpenRouter upstreams do) can leave a small input room too little space, and pi-tuck then refuses to cut rather than overflow. This is what made 1.0.1 refuse at a 32k window with Pi's default 16,384 reserve. Since 1.0.4 the notice says when the provider is counting far above pi-tuck's estimate. In 1.0.4, after an oversize tool result or a "Nothing to compact" error from Pi, a session could keep refusing later prompts. 1.0.5 fixes both. Pin a different upstream, lower the reserve, or use a larger window; see [Small windows](#small-windows).
+- Semantic search runs only on macOS with Apple silicon.
+
+## Benchmarks
+
+Each row says what was measured, under which conditions, and where the data is. "Interim" means the run is not finished.
+
+| What | Result | Conditions | Source |
+|---|---|---|---|
+| Cut time, live | pi-tuck median 0.087 s (90th percentile 0.205 s); Pi's built-in median 268.8 s (90th percentile 421.6 s). Context carried after a cut, median: 27,751 against 29,241 tokens | **Interim.** pi-tuck 1.0.1, Pi 1.4.0, SWE-bench Verified chains of 30 tasks, 131,072-token window, reply reserve 32,768, local model `mac-studio/main` writing Pi's summaries. 27 cuts per arm, 5 chains. The built-in time depends on that model and on server load. pi-tuck's time is an upper bound read from the session file | [`benchmark/RESULTS-INTERIM.md`](benchmark/RESULTS-INTERIM.md) section 1 |
+| Cut time, paired replay | pi-tuck median 0.062 s (range 0.005 to 0.48 s); local-model summarizer 193.4 s (77.3 to 413.3 s); frontier summarizer 96.1 s (39.1 to 267.8 s; measured on the retired model `openai/gpt-6-astra`; feeds no test). Context carried after a cut, median: 28,006 (pi-tuck), 26,424 (local), 25,555 (frontier) tokens | **Interim.** The same recorded sessions cut at the same positions. pi-tuck: 10 recordings, 52 cuts. Summarizers: 6 recordings, 28 and 29 cuts. pi-tuck's time leaves out the background search index | [`benchmark/RESULTS-INTERIM.md`](benchmark/RESULTS-INTERIM.md) section 2 |
+| Billed input, cutting at 60% of the room instead of 100% | −17.9% at 128k, −14.0% at 32k with automatic prompt caching; −17.0% and −9.6% with explicit cache breakpoints. Cuts go from 6 to 12 (128k) and from 84 to 157 (32k); none frees under a tenth of the room | Simulated billing, not provider bills. Prompt caching priced. Build `1bda64b` (the default shipped in 1.0.3). 5 recorded live sessions. 128k = 131,072 window, reserve 32,768. 32k = 32,768 window, reserve 16,384 | [`rnd/replay32/RESULT-lean.md`](rnd/replay32/RESULT-lean.md) |
+| Answers after cutting at 60% instead of 100% | No more losses than the limit on all 4 sets. Losses / wins over n questions: 128k shared 3 / 7 of 99; 128k asymmetric 3 / 1 of 67; 32k shared 5 / 3 of 77; 32k asymmetric 9 / 5 of 343. In the 32k asymmetric set the losses lean against 60% (test_result 4 / 2, error_hit 3 / 1); it is the set to watch | Paired replay, same 5 sessions, DeepSeek flash as agent and judge. Limit, set in advance: losses minus wins at most max(2, 5% of n). Asymmetric = evidence the base keeps native and the variant has summarised | [`rnd/replay32/RESULT-lean.md`](rnd/replay32/RESULT-lean.md) |
+| 1.0.2 changes (plain-text summary, shrinking dialogue view, time on user turns only) against 1.0.1 | No more losses than the limit on all 4 sets. Losses / wins: 128k 1 / 2 of 128; 128k time questions 0 / 0 of 48; 32k 4 / 2 of 127; 32k asymmetric 0 / 2 of 56 | Paired replay, 5 recorded live sessions, DeepSeek flash as agent and judge. 11 of 146 answer sessions hit a 15-minute wall limit and drop out of the pairing | [`rnd/replay32/RESULT-lean.md`](rnd/replay32/RESULT-lean.md), [`CHANGELOG.md`](CHANGELOG.md) |
+| Summary size, plain-text format against the 1.0.1 JSON format | About 30% shorter where the agent writes little, about 14% shorter where it writes a lot | Same dialogue | [`CHANGELOG.md`](CHANGELOG.md) 1.0.2 |
+| Small-window fix | A session that writes little prose cut 174 times in 300 tool rounds at 32k with a 16k reserve (1.0.1 code). With the fix, a test pins that no cut frees under a tenth of the room | Real planner on synthetic sessions | [`test/cut-yield.test.ts`](test/cut-yield.test.ts), which pins that every cut frees at least a tenth of the room |
+| Stall fix: a long session's cut | A cut at 3,888 entries took 72 s in 1.0.0 and takes 174 ms since 1.0.1. Since 1.0.1 the cost grows linearly with the session | One machine, the same synthetic session for both numbers (little prose, 128k window, reserve 16,384, turn 644) | [`CHANGELOG.md`](CHANGELOG.md) 1.0.1 |
+
+Notes:
+
+- The billing figures come from a simulation that replays real sessions through the real cut trigger and prices every request. They are not bills.
+- The answer replays use one cheap hosted model (DeepSeek flash) as agent and judge. They compare pi-tuck with other pi-tuck settings and versions. They do not compare pi-tuck with an LLM compactor.
+- The live and replay time rows measure pi-tuck 1.0.1. The newer versions are listed in the [changelog](CHANGELOG.md). A cut still calls no model, but the eval has not yet re-timed 1.0.3.
+- Everything here is from SWE-bench-style tasks on a small set of models. It may not carry to your work.
+
+## Settings
+
+pi-tuck needs none. Two are optional. Put them in `pi-tuck.json` in Pi's agent folder (`~/.pi/agent`, or the folder `PI_CODING_AGENT_DIR` names):
+
+```json
+{ "cutAt": 0.6, "semanticRecall": true }
+```
+
+- **`cutAt`** (above 0, at most 1; default 0.6) is the part of the input room a request may fill before pi-tuck cuts. At 0.6, each cut leaves room to grow, so fewer tokens are carried and billed per request. `{ "cutAt": 1 }` cuts at the full room. A lower value cuts earlier and smaller. The whole room still decides which request pi-tuck refuses.
+- **`semanticRecall`** (`false` turns semantic search off). Nothing is then downloaded, and `tuck_search` uses keywords only.
+
+A bad file is reported at session start, and pi-tuck uses the defaults.
+
+### Small windows
+
+pi-tuck keeps room for the model's reply: the larger of the model's output limit and Pi's `compaction.reserveTokens` (default 16,384). It refuses a request that cannot fit, rather than let the provider cut it off. It estimates the request's size, and it scales that estimate by the worst gap it has seen between its count and the provider's. In a small window, an upstream that spends far more context on tool schemas (some OpenRouter upstreams do) can leave too little room. pi-tuck then refuses to cut rather than overflow. Lower the reserve for that model in Pi's `settings.json`, or use a larger window. For example, to lower it to 8,192:
 
 ```json
-{ "compaction": { "modelOverrides": { "my-provider/my-model": { "reserveTokens": 4096 } } } }
+{ "compaction": { "modelOverrides": { "my-provider/my-model": { "reserveTokens": 8192 } } } }
 ```
 
-**Cut point.** pi-tuck cuts once a request passes 60% of the input room, so each cut leaves room to grow and fewer tokens are carried (and billed) per request; measured against cutting at the full room, answers held and billed input fell 10-18%. Set `"cutAt"` in `pi-tuck.json` in Pi's agent folder (`~/.pi/agent`) to change it, a number above 0 and at most 1: `{ "cutAt": 1 }` restores cutting at the full room, a lower value cuts earlier and smaller. The guard is unaffected, because the whole room still decides which request is refused.
+The reserve is the larger of `reserveTokens` and the model's `maxTokens`, so lowering `reserveTokens` alone does nothing when `models.json` sets a larger `maxTokens` for that model. Lower both, and keep `maxTokens` above your largest expected reply.
+
+If you reach the model through OpenRouter, you can also pin one upstream, so that every request is served by the same one. Put this in `models.json` in Pi's agent folder, and name the upstream you want:
+
+```json
+{ "providers": { "openrouter": { "compat": { "openRouterRouting": { "only": ["your-upstream"], "allow_fallbacks": false } } } } }
+```
 
 ## Semantic search
 
-Semantic search is on by default and runs entirely on your Mac: an embedding model,
-[EmbeddingGemma-2](https://huggingface.co/google/embeddinggemma-2), served by [llama.cpp](https://github.com/ggml-org/llama.cpp) on the
-CPU. No text leaves your machine.
+Semantic search is on by default on macOS with Apple silicon. It runs entirely on your Mac: an embedding model, [EmbeddingGemma-2](https://huggingface.co/google/embeddinggemma-2), served by [llama.cpp](https://github.com/ggml-org/llama.cpp) on the CPU. No text leaves your machine. Elsewhere, `tuck_search` uses keywords, and everything else works.
 
-The first time a session starts, pi-tuck downloads what it needs, once, about 390 MB in the background, and keeps working with keyword
-search meanwhile:
+The first time a session starts, pi-tuck downloads what it needs, once, about 390 MB, in the background. Keyword search works meanwhile:
 
 | File | From |
 |---|---|
@@ -76,21 +171,9 @@ search meanwhile:
 | llama.cpp release b11476 for macOS arm64 (12 MB) | github.com/ggml-org/llama.cpp |
 | CPython 3.13 (25 MB) and 16 Python packages for the tokenizer (9 MB) | github.com/astral-sh/python-build-standalone, pypi.org |
 
-Every file is pinned to a SHA-256 in [src/assets.ts](src/assets.ts) and checked before use. Everything goes to `~/.pi/agent/pi-tuck`;
-to remove it, delete that folder. Setup needs about 6.5 GB of free disk space while it runs and uses 460 MB after. If it fails, for
-example offline, pi-tuck says so and tries again at the next session start. One-shot runs (`pi -p`) don't start the setup.
+Every file is pinned to a SHA-256 in [src/assets.ts](src/assets.ts) and checked before use. Everything goes to `~/.pi/agent/pi-tuck`; to remove it, delete that folder. Setup needs about 6.5 GB of free disk space while it runs, and uses 460 MB after. If it fails, for example offline, pi-tuck says so and tries again at the next session start. One-shot runs (`pi -p`) do not start the setup.
 
-Indexing starts in the background the first time a session uses `tuck_search`, and catches up after each compaction. Semantic search is off when Pi
-runs in a folder that contains Pi's agent folder, such as your home folder, because its own files live there; open Pi in a project folder.
-
-**Turn it off** by creating `~/.pi/agent/pi-tuck.json`:
-
-```json
-{ "semanticRecall": false }
-```
-
-Nothing is then downloaded, and `tuck_search` uses keywords only. If you set `PI_CODING_AGENT_DIR`, the file and the download folder are
-in that folder instead.
+Indexing starts in the background the first time a session uses `tuck_search`, and catches up after each compaction. Semantic search is off when Pi runs in a folder that contains Pi's agent folder, such as your home folder, because its own files live there. Open Pi in a project folder.
 
 ## Use from code
 
@@ -105,8 +188,7 @@ const handle = await installCompaction(pi, { semanticRecall });
 await handle.setSemanticEnabled(true, ctx); // e.g. in a session_start handler
 ```
 
-`installCompaction` options (reply reserve, host-injected messages, recall policy and others) are documented in
-[src/compaction.ts](src/compaction.ts). `handle.close()` stops the semantic worker; call it before disposing a session.
+`installCompaction` options (reply reserve, host-injected messages, recall policy, `cutAt` and others) are documented in [src/compaction.ts](src/compaction.ts). `handle.close()` stops the semantic worker; call it before disposing a session.
 
 ## Development
 
@@ -117,12 +199,11 @@ npm test        # unit tests (Vitest) and the Python runtime tests
 npm run build   # dist/ for the library export
 ```
 
+Changes are listed in [CHANGELOG.md](CHANGELOG.md). Releases are published from GitHub Actions by npm trusted publishing, with provenance.
+
 ## Credits
 
-pi-tuck grew out of [pi-vcc](https://github.com/sting8k/pi-vcc) by sting8k and includes code from
-[pi-blackhole](https://github.com/k0valik/pi-blackhole) by k0valik and from [Pi](https://github.com/earendil-works/pi) by Mario
-Zechner, all MIT; their notices are in [LICENSE](LICENSE). The downloaded model is Apache-2.0, llama.cpp is MIT, and CPython is under the
-PSF license.
+pi-tuck grew out of [pi-vcc](https://github.com/sting8k/pi-vcc) by sting8k and includes code from [pi-blackhole](https://github.com/k0valik/pi-blackhole) by k0valik and from [Pi](https://github.com/earendil-works/pi) by Mario Zechner, all MIT; their notices are in [LICENSE](LICENSE). The downloaded model is Apache-2.0, llama.cpp is MIT, and CPython is under the PSF license.
 
 ## License
 

package/dist/src/build-id.d.ts

@@ -1 +1 @@
-export declare const BUILD_ID = "72d28833b7180debd3ea6a7ca0308b000f8d27671b42c4d2b73dfaaa7bbdad59";
+export declare const BUILD_ID = "6c391c76284110aae5bb375c04b2147d2fab17de3cb427448e8a869f5ad1e659";

package/dist/src/build-id.js

@@ -1 +1 @@
-export const BUILD_ID = "72d28833b7180debd3ea6a7ca0308b000f8d27671b42c4d2b73dfaaa7bbdad59";
+export const BUILD_ID = "6c391c76284110aae5bb375c04b2147d2fab17de3cb427448e8a869f5ad1e659";

package/dist/src/compaction.d.ts

@@ -3,6 +3,9 @@ import { type RecorderOptions } from "./diagnostics.js";
 import { type AgentMessage } from "./budget.js";
 import { SettingsManager, type ExtensionAPI, type ExtensionContext, type SessionEntry } from "@earendil-works/pi-coding-agent";
 import type { RecallPolicy } from "./recall.js";
+/** What a capacity refusal tells the operator. When the provider has counted far more input than pi-tuck's estimate, the refusal most
+ * likely comes from that counting (an upstream behind a router can spend much more context on tool schemas), so the notice says so. */
+export declare const refusalNotice: (message: string, factor: number) => string;
 /** The most a seed or an overflow raise may set the calibration factor to. A paired reply's own ratio is not bounded by it. */
 export declare const CALIBRATION_CEILING = 1.5;
 /** How many recent paired replies (and overflow bounds, and the seed while it is among them) the calibration factor is the maximum of. */

package/dist/src/compaction.js

@@ -36,8 +36,21 @@ const RecallInput = Type.Union([
 ]);
 const SearchInput = Type.Object({ query, priorResults: Type.Optional(Type.Array(Type.Object({ entryId, sourceSha: Type.String({ minLength: 64, maxLength: 64 }), start: Type.Integer({ minimum: 0 }), end: Type.Integer({ minimum: 0 }) }, { additionalProperties: false }), { maxItems: 20 })) }, { additionalProperties: false });
 const reply = (value) => ({ content: [{ type: "text", text: JSON.stringify(value) }], details: {} });
-class CapacityError extends Error {
+/** The compactor's verdict that this attempt cannot be made: the cut it needs cannot be made, or the request cannot fit. Like every failure but
+ * a `CommitFault`, it ends with its attempt (`forgetFailure`), and the next attempt plans afresh from the session as it is. */
+class Refusal extends Error {
 }
+/** A cut this compactor supplied that Pi then failed to commit. Pi appends a cut to the session in memory before it writes the file, so the
+ * session may hold a cut its file lacks; what is written after would build on it. The one failure that stays until the session restarts. */
+class CommitFault extends Error {
+}
+/** A refusal for want of room, which the user can remedy: it carries advice. */
+class CapacityError extends Refusal {
+}
+/** What a capacity refusal tells the operator. When the provider has counted far more input than pi-tuck's estimate, the refusal most
+ * likely comes from that counting (an upstream behind a router can spend much more context on tool schemas), so the notice says so. */
+export const refusalNotice = (message, factor) => message + "; shorten the input, use a model with a larger window, or lower Pi's compaction.reserveTokens, then retry." +
+    (factor >= 1.5 ? ` This provider has counted up to ${factor.toFixed(1)}x the input pi-tuck estimates; through a router such as OpenRouter, a different upstream may count less.` : "");
 /** The most a seed or an overflow raise may set the calibration factor to. A paired reply's own ratio is not bounded by it. */
 export const CALIBRATION_CEILING = 1.5;
 /** How many recent paired replies (and overflow bounds, and the seed while it is among them) the calibration factor is the maximum of. */
@@ -85,10 +98,17 @@ export async function installCompaction(pi, options = {}) {
     if (!validCutAt(cutAt))
         throw new Error("Invalid cut point: cutAt must be a number above 0 and at most 1");
     const semantic = options.semanticRecall ?? createSemanticRecall();
+    const hostAccess = () => {
+        if (!active())
+            throw new Error("Compaction host is inactive");
+        options.assertAccess?.();
+    };
     let semanticTransition = Promise.resolve();
     let semanticRequest = 0;
     const semanticPending = new Set();
-    const semanticProvider = (ctx) => () => memoizedEligibleSources(ctx, options.recallPolicy, access);
+    // Semantic indexing outlives any one attempt, so it checks only that the host still grants access: a compactor failure, which ends with its
+    // attempt, must not read as a revocation (that disables semantic recall and deletes its index for the rest of the session).
+    const semanticProvider = (ctx) => () => memoizedEligibleSources(ctx, options.recallPolicy, hostAccess);
     const recorder = new IncidentRecorder(options.diagnostics);
     let proposalId;
     const observe = (ctx) => recorder.measureInline(() => {
@@ -249,7 +269,7 @@ export async function installCompaction(pi, options = {}) {
             learn(ratio);
     });
     pi.on("agent_start", () => { admittedRun = true; pending = undefined; phase = "idle"; });
-    pi.on("agent_settled", () => { admittedRun = false; pending = undefined; phase = "idle"; });
+    pi.on("agent_settled", () => { admittedRun = false; pending = undefined; phase = "idle"; forgetFailure(); });
     const active = () => options.active?.() ?? true;
     const settingsFor = () => options.settings ?? SettingsManager.inMemory(pi.getSettings());
     // The standalone default reserves the model's catalogue maximum output: a transport may omit the output cap, and then only that
@@ -298,7 +318,9 @@ export async function installCompaction(pi, options = {}) {
     const fail = (error, ctx) => {
         recorder.stop("native-attempt-failed");
         const notify = !failure && error instanceof CapacityError;
-        failure = error instanceof Error ? error : new Error(String(error));
+        // A CommitFault stays whatever fails after it.
+        if (!(failure instanceof CommitFault))
+            failure = error instanceof Error ? error : new Error(String(error));
         pendingBoundaryCommit = false;
         pending = undefined;
         try {
@@ -307,19 +329,21 @@ export async function installCompaction(pi, options = {}) {
         finally {
             try {
                 if (notify)
-                    ctx.ui.notify(failure.message + "; shorten the input, use a model with a larger window, or lower Pi's compaction.reserveTokens, then retry.", "error");
+                    ctx.ui.notify(refusalNotice(failure.message, factor), "error");
             }
             finally {
                 ctx.abort();
             }
         }
     };
+    // The one place a failure is dropped: when its attempt has settled, when a prompt arrives, and when the operator asks for a cut. A run that no
+    // input started (a custom message) has only the first of these. No failure but a CommitFault outlives its attempt.
+    const forgetFailure = () => { if (!(failure instanceof CommitFault))
+        failure = undefined; };
     const access = () => {
         if (failure)
             throw failure;
-        if (!active())
-            throw new Error("Compaction host is inactive");
-        options.assertAccess?.();
+        hostAccess();
     };
     // Abort alone is insufficient: Pi starts a fresh abort controller after
     // before_agent_start. Keep this attempt blocked, then recheck on new idle input.
@@ -329,8 +353,7 @@ export async function installCompaction(pi, options = {}) {
     pi.on("input", async (event, ctx) => {
         if (!ctx.isIdle() || phase === "starting")
             return; // a steering or follow-up message joins the running request
-        if (failure instanceof CapacityError)
-            failure = undefined;
+        forgetFailure();
         phase = "decided";
         try {
             await preflight(ctx, incomingTokens(event.text, event.images), ctx.getSystemPrompt(), pi.getActiveTools());
@@ -404,8 +427,8 @@ export async function installCompaction(pi, options = {}) {
             // Pi also labels the awaited idle incoming preflight "manual".
             const incoming = pendingCut;
             const operatorManual = event.reason === "manual" && !(incoming && !admittedRun);
-            if (operatorManual && !admittedRun && failure instanceof CapacityError)
-                failure = undefined;
+            if (operatorManual && !admittedRun)
+                forgetFailure();
             access();
             if (event.customInstructions?.trim()) {
                 recorder.stop("no-cut-custom-instructions");
@@ -440,7 +463,9 @@ export async function installCompaction(pi, options = {}) {
                     : "Compaction did not reduce context within the estimated request budget",
                 plan: (target) => {
                     const reserve = planReserve(target, room, mandatory, ctx);
-                    const draft = tuckDraft(tuck, event, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, retain, operatorManual);
+                    // Every cut asked while no run is admitted (a prompt's preflight, an operator's cut, Pi's own cut before a run a custom message
+                    // starts) is planned for a run that has not begun: the session has moved on from any run that ended. A retry is inside its run.
+                    const draft = tuckDraft(tuck, event, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, retain, operatorManual, Boolean(incoming) || operatorManual || !admittedRun);
                     return draft.compaction ? { draft: true, after: draft.after, mode: draft.mode, compaction: draft.compaction, reserve }
                         : { draft: false, capacityRejected: Boolean(draft.capacityRejected), reason: draft.unavailable };
                 },
@@ -456,7 +481,7 @@ export async function installCompaction(pi, options = {}) {
             recorder.stop("no-cut");
             ctx.ui.notify(decision.message, "warning");
             if (!operatorManual)
-                fail(decision.capacity ? new CapacityError(decision.message) : new Error(decision.message), ctx);
+                fail(new (decision.capacity ? CapacityError : Refusal)(decision.message), ctx);
             return { cancel: true };
         }
         catch (error) {
@@ -513,10 +538,14 @@ export async function installCompaction(pi, options = {}) {
                     // An opportunistic cut is not needed to send the request, which is below the room: the guard decides. However it ends (the
                     // planner declined it, Pi had nothing to compact, an earlier handler cancelled it) it is no fault, unless a failure is latched
                     // or the compactor proposed a cut that then failed to commit (the session may hold what the file lacks).
+                    // A needed cut that Pi has nothing to make (no history older than its keepRecentTokens since the last cut) is a refusal of
+                    // this request, which ends with it: as a fault it stayed, and every later prompt of the session failed on it.
                     if (cut.opportunistic && !failure && !cut.proposed)
                         resolve();
+                    else if (!cut.proposed && isNoWork(error.message))
+                        reject(new CapacityError(`The request needs a cut and Pi has none to make (${error.message})`));
                     else
-                        reject(error.message === "Compaction cancelled" && failure instanceof CapacityError ? failure : error);
+                        reject(error.message === "Compaction cancelled" && failure instanceof Refusal ? failure : cut.proposed ? new CommitFault(error.message) : error);
                 } }));
         }
         finally {
@@ -526,7 +555,7 @@ export async function installCompaction(pi, options = {}) {
         // The cut awaited: take the snapshot again. An opportunistic cut never throws; the request is not over the room, and the guard decides.
         const next = snapshot(), after = next.before + next.mandatory, afterFits = fits(after, limits(ctx).room, ctx);
         if (!cut.opportunistic && (after >= total || !afterFits))
-            throw new (afterFits ? Error : CapacityError)("Incoming Pi preflight compaction did not reduce context within the estimated request budget");
+            throw new (afterFits ? Refusal : CapacityError)("Incoming Pi preflight compaction did not reduce context within the estimated request budget");
     };
     pi.on("before_agent_start", async (event, ctx) => {
         const prompted = phase === "decided";
@@ -583,7 +612,7 @@ export async function installCompaction(pi, options = {}) {
                 },
             });
             if (decision.kind === "fault")
-                throw new (decision.capacity ? CapacityError : Error)(decision.message);
+                throw new (decision.capacity ? CapacityError : Refusal)(decision.message);
             if (decision.kind !== "cut")
                 return;
             const compaction = captureDraft(decision.plan.compaction, ctx, decision.plan.reserve, "tool-loop", before, true, false, previousSummary, { response, host, mandatory, target: decision.target, margin: room - band });
@@ -632,13 +661,13 @@ export async function installCompaction(pi, options = {}) {
         if (event.reason === "manual" && !event.fromExtension && !event.aborted && isNoWork(event.errorMessage))
             return;
         if (active() && !event.aborted)
-            fail(new Error(event.errorMessage ?? "Compaction failed"), ctx);
+            fail(new (event.fromExtension ? CommitFault : Error)(event.errorMessage ?? "Compaction failed"), ctx);
     });
     pi.on("agent_before_settle", (event, ctx) => {
         if (!pendingBoundaryCommit)
             return;
         if (event.outcome === "error")
-            fail(new Error("Native tool-loop boundary draft remained unconfirmed at error settlement"), ctx);
+            fail(new CommitFault("Native tool-loop boundary draft remained unconfirmed at error settlement"), ctx);
         else
             pendingBoundaryCommit = false;
     });

package/dist/src/diagnostics.d.ts

@@ -1,6 +1,6 @@
 import { type RecallPolicy } from './recall.js';
 import type { SessionEntry } from '@earendil-works/pi-coding-agent';
-export declare const DIAGNOSTIC_BUILD = "72d28833b7180debd3ea6a7ca0308b000f8d27671b42c4d2b73dfaaa7bbdad59";
+export declare const DIAGNOSTIC_BUILD = "6c391c76284110aae5bb375c04b2147d2fab17de3cb427448e8a869f5ad1e659";
 export interface Receipt {
     id: string;
     build: string;

package/dist/src/tuck/core/brief.js

@@ -108,14 +108,16 @@ const analyze = (text) => {
 const excerptRecords = (text, { passages, costs }, role, sourceIndex, headLimit, tailLimit, readFrom) => {
     const charLimit = 1024;
     const identity = { role, ...(sourceIndex !== undefined ? { sourceIndex } : {}) };
-    if (escapedLength(text) <= charLimit && costs.reduce((sum, cost) => sum + cost, 0) <= headLimit + tailLimit)
+    // A line carries at least as many characters as the text, so a long text is over the limit without counting its escapes.
+    const fits = (from, to) => to - from <= charLimit && escapedLength(text.slice(from, to)) <= charLimit;
+    if (fits(0, text.length) && costs.reduce((sum, cost) => sum + cost, 0) <= headLimit + tailLimit)
         return [{ kind: "excerpt", ...identity, text }];
     let head = 0, headCost = 0, tail = passages.length, tailCost = 0;
-    while (head < tail && headCost + costs[head] <= headLimit && escapedLength(text.slice(0, passages[head].end)) <= charLimit) {
+    while (head < tail && headCost + costs[head] <= headLimit && fits(0, passages[head].end)) {
         headCost += costs[head++];
     }
     if (tailLimit > 0) {
-        while (tail > head && tailCost + costs[tail - 1] <= tailLimit && escapedLength(text.slice(passages[tail - 1].start)) <= charLimit) {
+        while (tail > head && tailCost + costs[tail - 1] <= tailLimit && fits(passages[tail - 1].start, text.length)) {
             tailCost += costs[--tail];
         }
     }

package/dist/src/tuck/core/filter-noise.js

@@ -1,6 +1,7 @@
 import { isSourceExport, isLookupTool, isRecallTool } from "./recall-operation.js";
+import { isDialogue } from "./brief.js";
 /** Suppress lookup echoes, retaining recorded export attempts and exactly paired outcomes. */
 export const filterNoise = (blocks) => blocks.filter(b => !(b.kind === "tool_call" && isLookupTool(b.name) && !(isRecallTool(b.name) && isSourceExport(b.args))) &&
     !(b.kind === "tool_result" && isLookupTool(b.name) &&
         !(isRecallTool(b.name) && b.recallExport)) &&
-    !((b.kind === "user" || b.kind === "assistant" || b.kind === "notice") && !b.text.trim()));
+    !(isDialogue(b) && !b.text.trim()));

package/dist/src/tuck/core/normalize.js

@@ -18,14 +18,10 @@ const normalizeOne = (msg, msgIndex) => {
         }
         return blocks.length > 0 ? blocks : [{ kind: "user", text: "", sourceIndex: msgIndex }];
     }
-    if (msg.role === "bashExecution") {
-        const cmd = msg.command ?? "";
-        const out = msg.output ?? "";
-        const exit = msg.exitCode;
-        return [{ kind: "bash", command: cmd, output: out, exitCode: exit, cancelled: msg.cancelled, truncated: msg.truncated, sourceIndex: msgIndex }];
-    }
+    if (msg.role === "bashExecution")
+        return [{ kind: "bash", sourceIndex: msgIndex }];
     if (msg.role === "toolResult")
-        return [{ kind: "tool_result", name: msg.toolName, text: sanitize(textOf(msg.content)), sourceIndex: msgIndex }];
+        return [{ kind: "tool_result", name: msg.toolName, sourceIndex: msgIndex }];
     if (msg.role === "assistant") {
         if (!msg.content)
             return [];
@@ -46,7 +42,6 @@ const normalizeOne = (msg, msgIndex) => {
                     kind: "tool_call",
                     name: part.name,
                     args: part.arguments,
-                    callId: part.id,
                     sourceIndex: msgIndex,
                 });
             }

package/dist/src/tuck/core/shrink.js

@@ -82,18 +82,16 @@ const createLane = (role, units, mandatory, decorate) => {
             const empty = emptied[tier], counts = emptyCounts[tier];
             count += counts[end] - counts[drop];
             if (counts[end] > counts[drop]) {
-                const extra = [];
-                for (let i = drop; i < end && extra.length < 9; i++)
+                // Text blocks of one message share a locator: collect distinct ones, enough to tell whether the shown list is complete.
+                const extra = new Set();
+                for (let i = drop; i < end && (extra.size < 9 || !unlocated); i++)
                     if (empty[i]) {
                         const source = units[i].sourceIndex;
                         if (source === undefined)
                             unlocated = true;
-                        else
-                            extra.push(source);
+                        else if (extra.size < 9)
+                            extra.add(source);
                     }
-                for (let i = drop; i < end && !unlocated; i++)
-                    if (empty[i] && units[i].sourceIndex === undefined)
-                        unlocated = true;
                 sources = [...new Set([...sources, ...extra])].sort((a, b) => a - b).slice(0, 9);
             }
         }
@@ -119,10 +117,9 @@ const createDialogue = (input) => {
     const dialogue = input.blocks.filter(isDialogue);
     const times = dialogue.find(block => block.sourceTimes)?.sourceTimes;
     const decorate = (record) => decorateRecord(record, times);
-    const roleOf = (block) => block.kind;
-    const lanes = Object.fromEntries(ROLES.map(role => [role, createLane(role, dialogue.filter(block => roleOf(block) === role), input.mandatorySourceIndices, decorate)]));
+    const lanes = Object.fromEntries(ROLES.map(role => [role, createLane(role, dialogue.filter(block => block.kind === role), input.mandatorySourceIndices, decorate)]));
     const seen = { user: 0, assistant: 0, notice: 0 };
-    const order = dialogue.map(block => ({ role: roleOf(block), i: seen[roleOf(block)]++ }));
+    const order = dialogue.map(block => ({ role: block.kind, i: seen[block.kind]++ }));
     const tools = input.blocks.length > dialogue.length ? rendered([TOOLS_OMITTED]) : undefined;
     const overhead = TRANSCRIPT_HEADER.length + DIALOGUE_SUFFIX.length;
     /** The length of the summary text of a configuration: its header, one line per record, and the closing section; 0 when it has no record. */
@@ -201,7 +198,7 @@ export const shrinkDialogue = (input, limit) => {
             }
         }
     for (let d = 0; d <= recent.drop; d++)
-        if (hit = found("B", { user: WHOLE, assistant: { ...recent, drop: d, end: Math.max(recent.end, recent.drop) }, notice: clipped }))
+        if (hit = found("B", { user: WHOLE, assistant: { ...recent, drop: d }, notice: clipped }))
             return hit;
     // From here users give way, and the assistant keeps only that recent third.
     for (const t of tiers)
@@ -215,7 +212,7 @@ export const shrinkDialogue = (input, limit) => {
         if (hit = found("E1", { user: gone(user), assistant: recent, notice: { drop: d, tier: 1, end: notice.n } }))
             return hit;
     for (let d = recent.drop; d <= assistant.n; d++)
-        if (hit = found("E2", { user: gone(user), assistant: { drop: d, tier: recent.tier, end: Math.max(recent.end, d) }, notice: gone(notice) }))
+        if (hit = found("E2", { user: gone(user), assistant: { ...recent, drop: d }, notice: gone(notice) }))
             return hit;
     return { text: "", step: "none", size: 0 };
 };

package/dist/src/tuck/core/summarize.js

@@ -1,12 +1,20 @@
 import { normalize } from "./normalize.js";
 import { filterNoise } from "./filter-noise.js";
 import { shrinkDialogue, wholeDialogue } from "./shrink.js";
-const dialogueInput = (input) => ({
-    blocks: filterNoise(normalize(input.messages, input.sourceIndices, input.receiptContext, input.sourceEntryIds, input.sourceTimes)),
-    mandatorySourceIndices: input.ranking?.mandatorySourceIndices,
-    sourceIndices: input.sourceIndices ?? input.messages.map((_, index) => index),
-    sourceEntryIds: input.sourceEntryIds,
-});
+// A cut offers the whole dialogue and then, when it does not fit, the shrunk one, from the same input: normalize it once. The blocks are
+// never mutated, and keeping the same objects also keeps their cached passage analyses.
+const inputs = new WeakMap();
+const dialogueInput = (input) => {
+    let prepared = inputs.get(input);
+    if (!prepared)
+        inputs.set(input, prepared = {
+            blocks: filterNoise(normalize(input.messages, input.sourceIndices, input.receiptContext, input.sourceEntryIds, input.sourceTimes)),
+            mandatorySourceIndices: input.ranking?.mandatorySourceIndices,
+            sourceIndices: input.sourceIndices ?? input.messages.map((_, index) => index),
+            sourceEntryIds: input.sourceEntryIds,
+        });
+    return prepared;
+};
 /** The whole pre-cut dialogue: user, assistant and notice records; tool actions and observations are omitted.
  * This disposable view grants neither present authority nor native tool execution. */
 export const compileDialogue = (input) => wholeDialogue(dialogueInput(input));

package/dist/src/tuck/core/transcript.js

@@ -81,7 +81,15 @@ const LEGACY_RECEIPT_RANGE_NOTICE = RECEIPT_RANGE_NOTICE.replace("tuck_recall",
  */
 const escapeText = (text) => text.replace(/[\\\n\r]/g, c => c === "\\" ? "\\\\" : c === "\n" ? "\\n" : "\\r");
 /** The length of `text` as a record line carries it: one character more for each "\", line feed and carriage return. */
-export const escapedLength = (text) => text.length + (text.match(/[\\\n\r]/g)?.length ?? 0);
+export const escapedLength = (text) => {
+    let length = text.length;
+    for (let i = 0; i < text.length; i++) {
+        const c = text.charCodeAt(i);
+        if (c === 92 || c === 10 || c === 13)
+            length++;
+    }
+    return length;
+};
 const TITLE = /^[A-Z][A-Za-z0-9_-]*$/;
 const refuse = (record) => { throw new Error(`Cannot write a ${record.kind} record as ${TRANSCRIPT_HEADER}`); };
 /** One record as a line of TUCK-TRANSCRIPT/2 (without its newline). The records this build emits only: a record whose fields the grammar

package/dist/src/tuck/types.d.ts

@@ -42,20 +42,13 @@ export type NormalizedBlock = {
     kind: "tool_call";
     name: string;
     args: Record<string, unknown>;
-    callId?: string;
     sourceIndex?: number;
 } | {
     kind: "tool_result";
     name: string;
-    text: string;
     sourceIndex?: number;
     recallExport?: true;
 } | {
     kind: "bash";
-    command: string;
-    output: string;
-    exitCode: number | undefined;
-    cancelled?: boolean;
-    truncated?: boolean;
     sourceIndex?: number;
 });

package/dist/src/tuck.d.ts

@@ -81,13 +81,16 @@ export declare const tuck: Tuck;
  * initiating prompt, file ledger) are already charged in base. */
 export declare function historicalAllowance(window: number, reserve: number, base: number, before: number): number;
 /** Uses pi-tuck's own cut, calibrated maximum tail and dialogue compiler; Pi owns persistence and continuation. */
-export declare function tuckDraft(tuck: Tuck, event: SessionBeforeCompactEvent, ctx: ExtensionContext, reserveTokens: number, measure: Measure, policy?: RecallPolicy, retainCurrentInput?: boolean, operatorManual?: boolean): Draft;
+export declare function tuckDraft(tuck: Tuck, event: SessionBeforeCompactEvent, ctx: ExtensionContext, reserveTokens: number, measure: Measure, policy?: RecallPolicy, retainCurrentInput?: boolean, operatorManual?: boolean, newPrompt?: boolean): Draft;
 /** Shared compiler observes canonical branch and current token estimate; no native preparation utility. */
+/** `newPrompt`: the cut is planned for a run that has not begun (a prompt Pi has not appended yet, an operator's manual cut, Pi's own cut before
+ * a run a custom message starts); the session has moved on from an ended run. */
 export declare function tuckObservedDraft(tuck: Tuck, input: {
     branchEntries: SessionEntry[];
     tokensBefore: number;
     previousSummary?: string;
     operatorManual?: boolean;
+    newPrompt?: boolean;
 }, ctx: ExtensionContext, reserveTokens: number, measure: Measure, policy?: RecallPolicy, retainCurrentInput?: boolean): Draft;
 /** Numeric references use the original session index, then intersect with readable current-branch entries. */
 export declare function tuckRecall(tuck: Tuck, entries: readonly SessionEntry[], branch: readonly SessionEntry[], input: RecallOptions & {

package/dist/src/tuck.js

@@ -26,10 +26,12 @@ export function historicalAllowance(window, reserve, base, before) {
     return Math.max(0, Math.min(Math.floor(free / 4), before - base - 1)) * 4;
 }
 /** Uses pi-tuck's own cut, calibrated maximum tail and dialogue compiler; Pi owns persistence and continuation. */
-export function tuckDraft(tuck, event, ctx, reserveTokens, measure, policy = {}, retainCurrentInput = true, operatorManual = false) {
-    return tuckObservedDraft(tuck, { branchEntries: event.branchEntries, tokensBefore: event.preparation.tokensBefore, previousSummary: event.preparation.previousSummary, operatorManual }, ctx, reserveTokens, measure, policy, retainCurrentInput);
+export function tuckDraft(tuck, event, ctx, reserveTokens, measure, policy = {}, retainCurrentInput = true, operatorManual = false, newPrompt = false) {
+    return tuckObservedDraft(tuck, { branchEntries: event.branchEntries, tokensBefore: event.preparation.tokensBefore, previousSummary: event.preparation.previousSummary, operatorManual, newPrompt }, ctx, reserveTokens, measure, policy, retainCurrentInput);
 }
 /** Shared compiler observes canonical branch and current token estimate; no native preparation utility. */
+/** `newPrompt`: the cut is planned for a run that has not begun (a prompt Pi has not appended yet, an operator's manual cut, Pi's own cut before
+ * a run a custom message starts); the session has moved on from an ended run. */
 export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, policy = {}, retainCurrentInput = true) {
     const { isHistorySource, readable } = createRecall(policy);
     const projected = ctx.sessionManager.buildSessionProjection();
@@ -73,10 +75,14 @@ export function tuckObservedDraft(tuck, input, ctx, reserveTokens, measure, poli
         const calls = new Set(response.message.content.flatMap(block => block.type === "toolCall" ? [block.id] : []));
         const hasFreshReceipt = entries.slice(lastResponse + 1).some(entry => entry.type === "message" &&
             entry.message.role === "toolResult" && calls.has(entry.message.toolCallId));
-        freshRound = hasFreshReceipt;
+        // Fresh means the receipts are the answer the next request of this run carries. Once the session has moved on (the cut is planned for a
+        // run that has not begun, or a prompt has arrived since them) they are ordinary history, whatever ended their run: a reply discarded
+        // or one Pi hid when it recovered. Kept native, a receipt that never fits would refuse every later prompt. A retry is inside its run.
+        const superseded = input.newPrompt || entries.slice(lastResponse + 1).some(entry => entry.type === "message" && entry.message.role === "user");
+        freshRound = hasFreshReceipt && !superseded;
         const keptId = cut.firstKeptEntryId;
         const chosenBoundary = cut.compactAll ? entries.length : entries.findIndex(entry => entry.id === keptId);
-        if (hasFreshReceipt && lastResponse < chosenBoundary) {
+        if (freshRound && lastResponse < chosenBoundary) {
             const prefix = tuck.buildOwnCut(entries.slice(0, lastResponse), 0);
             if (!prefix.ok)
                 return { unavailable: "No reducible history before the fresh tool round" };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-tuck",
-  "version": "1.0.3",
+  "version": "1.0.5",
   "description": "Deterministic compaction for Pi: compact summaries that keep the dialogue and the files touched, plus history recall by keyword and by meaning, all local.",
   "keywords": [
     "pi-package",

package/src/build-id.ts

@@ -1 +1 @@
-export const BUILD_ID = "72d28833b7180debd3ea6a7ca0308b000f8d27671b42c4d2b73dfaaa7bbdad59";
+export const BUILD_ID = "6c391c76284110aae5bb375c04b2147d2fab17de3cb427448e8a869f5ad1e659";

package/src/compaction.ts

@@ -40,7 +40,19 @@ const RecallInput = Type.Union([
 ]);
 const SearchInput = Type.Object({query,priorResults:Type.Optional(Type.Array(Type.Object({entryId,sourceSha:Type.String({minLength:64,maxLength:64}),start:Type.Integer({minimum:0}),end:Type.Integer({minimum:0})},{additionalProperties:false}),{maxItems:20}))},{additionalProperties:false});
 const reply = (value: unknown) => ({ content: [{ type: "text" as const, text: JSON.stringify(value) }], details: {} });
-class CapacityError extends Error {}
+/** The compactor's verdict that this attempt cannot be made: the cut it needs cannot be made, or the request cannot fit. Like every failure but
+ * a `CommitFault`, it ends with its attempt (`forgetFailure`), and the next attempt plans afresh from the session as it is. */
+class Refusal extends Error {}
+/** A cut this compactor supplied that Pi then failed to commit. Pi appends a cut to the session in memory before it writes the file, so the
+ * session may hold a cut its file lacks; what is written after would build on it. The one failure that stays until the session restarts. */
+class CommitFault extends Error {}
+/** A refusal for want of room, which the user can remedy: it carries advice. */
+class CapacityError extends Refusal {}
+/** What a capacity refusal tells the operator. When the provider has counted far more input than pi-tuck's estimate, the refusal most
+ * likely comes from that counting (an upstream behind a router can spend much more context on tool schemas), so the notice says so. */
+export const refusalNotice = (message: string, factor: number): string =>
+    message + "; shorten the input, use a model with a larger window, or lower Pi's compaction.reserveTokens, then retry." +
+    (factor >= 1.5 ? ` This provider has counted up to ${factor.toFixed(1)}x the input pi-tuck estimates; through a router such as OpenRouter, a different upstream may count less.` : "");
 /** The most a seed or an overflow raise may set the calibration factor to. A paired reply's own ratio is not bounded by it. */
 export const CALIBRATION_CEILING = 1.5;
 /** How many recent paired replies (and overflow bounds, and the seed while it is among them) the calibration factor is the maximum of. */
@@ -108,10 +120,16 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
     const cutAt = options.cutAt ?? DEFAULT_CUT_AT;
     if (!validCutAt(cutAt)) throw new Error("Invalid cut point: cutAt must be a number above 0 and at most 1");
     const semantic = options.semanticRecall ?? createSemanticRecall();
+    const hostAccess = () => {
+        if (!active()) throw new Error("Compaction host is inactive");
+        options.assertAccess?.();
+    };
     let semanticTransition: Promise<void> = Promise.resolve();
     let semanticRequest = 0;
     const semanticPending = new Set<Promise<unknown>>();
-    const semanticProvider = (ctx: ExtensionContext) => () => memoizedEligibleSources(ctx, options.recallPolicy, access);
+    // Semantic indexing outlives any one attempt, so it checks only that the host still grants access: a compactor failure, which ends with its
+    // attempt, must not read as a revocation (that disables semantic recall and deletes its index for the rest of the session).
+    const semanticProvider = (ctx: ExtensionContext) => () => memoizedEligibleSources(ctx, options.recallPolicy, hostAccess);
     const recorder = new IncidentRecorder(options.diagnostics);
     let proposalId: string | undefined;
     const observe = (ctx: ExtensionContext) => recorder.measureInline(() => {
@@ -220,7 +238,7 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
         if (input > 0 && Number.isSafeInteger(input) && Number.isFinite(ratio)) learn(ratio);
     });
     pi.on("agent_start", () => { admittedRun = true; pending = undefined; phase = "idle"; });
-    pi.on("agent_settled", () => { admittedRun = false; pending = undefined; phase = "idle"; });
+    pi.on("agent_settled", () => { admittedRun = false; pending = undefined; phase = "idle"; forgetFailure(); });
 	const active = () => options.active?.() ?? true;
 	const settingsFor = () => options.settings ?? SettingsManager.inMemory(pi.getSettings());
 	// The standalone default reserves the model's catalogue maximum output: a transport may omit the output cap, and then only that
@@ -270,18 +288,21 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
 	const fail = (error: unknown, ctx: ExtensionContext) => {
 		recorder.stop("native-attempt-failed");
         const notify = !failure && error instanceof CapacityError;
-		failure = error instanceof Error ? error : new Error(String(error));
+		// A CommitFault stays whatever fails after it.
+		if (!(failure instanceof CommitFault)) failure = error instanceof Error ? error : new Error(String(error));
         pendingBoundaryCommit = false;
         pending = undefined;
 		try { options.fail?.(error); } finally {
-            try { if (notify) ctx.ui.notify(failure.message + "; shorten the input, use a model with a larger window, or lower Pi's compaction.reserveTokens, then retry.", "error"); }
+            try { if (notify) ctx.ui.notify(refusalNotice(failure.message, factor), "error"); }
             finally { ctx.abort(); }
         }
 	};
+	// The one place a failure is dropped: when its attempt has settled, when a prompt arrives, and when the operator asks for a cut. A run that no
+	// input started (a custom message) has only the first of these. No failure but a CommitFault outlives its attempt.
+	const forgetFailure = () => { if (!(failure instanceof CommitFault)) failure = undefined; };
 	const access = () => {
 		if (failure) throw failure;
-		if (!active()) throw new Error("Compaction host is inactive");
-		options.assertAccess?.();
+		hostAccess();
 	};
     // Abort alone is insufficient: Pi starts a fresh abort controller after
     // before_agent_start. Keep this attempt blocked, then recheck on new idle input.
@@ -290,7 +311,7 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
     let phase: "idle" | "decided" | "starting" = "idle";
     pi.on("input", async (event, ctx) => {
         if (!ctx.isIdle() || phase === "starting") return; // a steering or follow-up message joins the running request
-        if (failure instanceof CapacityError) failure = undefined;
+        forgetFailure();
         phase = "decided";
         try { await preflight(ctx, incomingTokens(event.text, event.images), ctx.getSystemPrompt(), pi.getActiveTools()); }
         catch (error) { fail(error, ctx); }
@@ -346,7 +367,7 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
             // Pi also labels the awaited idle incoming preflight "manual".
             const incoming = pendingCut;
             const operatorManual = event.reason === "manual" && !(incoming && !admittedRun);
-            if (operatorManual && !admittedRun && failure instanceof CapacityError) failure = undefined;
+            if (operatorManual && !admittedRun) forgetFailure();
 			access();
 			if (event.customInstructions?.trim()) {
 				recorder.stop("no-cut-custom-instructions");
@@ -380,7 +401,9 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
                     : "Compaction did not reduce context within the estimated request budget",
                 plan: (target): Plan<{ compaction: NonNullable<ReturnType<typeof tuckDraft>["compaction"]>; reserve: number }> => {
                     const reserve = planReserve(target, room, mandatory, ctx);
-                    const draft = tuckDraft(tuck, event, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, retain, operatorManual);
+                    // Every cut asked while no run is admitted (a prompt's preflight, an operator's cut, Pi's own cut before a run a custom message
+                    // starts) is planned for a run that has not begun: the session has moved on from any run that ended. A retry is inside its run.
+                    const draft = tuckDraft(tuck, event, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, retain, operatorManual, Boolean(incoming) || operatorManual || !admittedRun);
                     return draft.compaction ? { draft: true, after: draft.after, mode: draft.mode, compaction: draft.compaction, reserve }
                         : { draft: false, capacityRejected: Boolean(draft.capacityRejected), reason: draft.unavailable };
                 },
@@ -391,7 +414,7 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
             if (decision.kind === "none") return { cancel: true };
 			recorder.stop("no-cut");
             ctx.ui.notify(decision.message, "warning");
-			if (!operatorManual) fail(decision.capacity ? new CapacityError(decision.message) : new Error(decision.message), ctx);
+			if (!operatorManual) fail(new (decision.capacity ? CapacityError : Refusal)(decision.message), ctx);
 			return { cancel: true };
 		} catch (error) {
 			try { fail(error, ctx); } finally { return { cancel: true }; }
@@ -437,15 +460,18 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
                     // An opportunistic cut is not needed to send the request, which is below the room: the guard decides. However it ends (the
                     // planner declined it, Pi had nothing to compact, an earlier handler cancelled it) it is no fault, unless a failure is latched
                     // or the compactor proposed a cut that then failed to commit (the session may hold what the file lacks).
+                    // A needed cut that Pi has nothing to make (no history older than its keepRecentTokens since the last cut) is a refusal of
+                    // this request, which ends with it: as a fault it stayed, and every later prompt of the session failed on it.
                     if (cut.opportunistic && !failure && !cut.proposed) resolve();
-                    else reject(error.message === "Compaction cancelled" && failure instanceof CapacityError ? failure : error);
+                    else if (!cut.proposed && isNoWork(error.message)) reject(new CapacityError(`The request needs a cut and Pi has none to make (${error.message})`));
+                    else reject(error.message === "Compaction cancelled" && failure instanceof Refusal ? failure : cut.proposed ? new CommitFault(error.message) : error);
                 } }));
         } finally { pendingCut = undefined; }
         access();
         // The cut awaited: take the snapshot again. An opportunistic cut never throws; the request is not over the room, and the guard decides.
         const next = snapshot(), after = next.before + next.mandatory, afterFits = fits(after, limits(ctx).room, ctx);
         if (!cut.opportunistic && (after >= total || !afterFits))
-            throw new (afterFits ? Error : CapacityError)("Incoming Pi preflight compaction did not reduce context within the estimated request budget");
+            throw new (afterFits ? Refusal : CapacityError)("Incoming Pi preflight compaction did not reduce context within the estimated request budget");
     };
     pi.on("before_agent_start", async (event, ctx) => {
         const prompted = phase === "decided";
@@ -492,7 +518,7 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
                     return { draft: true, after: measure(messages), mode: draft.mode, compaction: draft.compaction, reserve };
                 },
             });
-            if (decision.kind === "fault") throw new (decision.capacity ? CapacityError : Error)(decision.message);
+            if (decision.kind === "fault") throw new (decision.capacity ? CapacityError : Refusal)(decision.message);
             if (decision.kind !== "cut") return;
             const compaction = captureDraft(decision.plan.compaction, ctx, decision.plan.reserve, "tool-loop", before, true, false, previousSummary,
                 { response, host, mandatory, target: decision.target, margin: room - band });
@@ -523,11 +549,11 @@ export async function installCompaction(pi: ExtensionAPI, options: CompactionOpt
 		if (exhausted) return;
 		// Pi reports manual no-work as failures; an accepted pi-tuck draft cannot be a no-op.
 		if (event.reason === "manual" && !event.fromExtension && !event.aborted && isNoWork(event.errorMessage)) return;
-		if (active() && !event.aborted) fail(new Error(event.errorMessage ?? "Compaction failed"), ctx);
+		if (active() && !event.aborted) fail(new (event.fromExtension ? CommitFault : Error)(event.errorMessage ?? "Compaction failed"), ctx);
 	});
     pi.on("agent_before_settle", (event, ctx) => {
         if (!pendingBoundaryCommit) return;
-        if (event.outcome === "error") fail(new Error("Native tool-loop boundary draft remained unconfirmed at error settlement"), ctx);
+        if (event.outcome === "error") fail(new CommitFault("Native tool-loop boundary draft remained unconfirmed at error settlement"), ctx);
         else pendingBoundaryCommit = false;
     });
 	pi.on("context_with_system", (event, ctx) => {

package/src/tuck/core/brief.ts

@@ -98,12 +98,14 @@ const excerptRecords = (text: string, { passages, costs }: Analysis, role: "user
   headLimit: number, tailLimit: number, readFrom: (offset: number) => string): TranscriptRecord[] => {
   const charLimit = 1024;
   const identity = { role, ...(sourceIndex !== undefined ? { sourceIndex } : {}) };
-  if (escapedLength(text) <= charLimit && costs.reduce((sum, cost) => sum + cost, 0) <= headLimit + tailLimit)
+  // A line carries at least as many characters as the text, so a long text is over the limit without counting its escapes.
+  const fits = (from: number, to: number) => to - from <= charLimit && escapedLength(text.slice(from, to)) <= charLimit;
+  if (fits(0, text.length) && costs.reduce((sum, cost) => sum + cost, 0) <= headLimit + tailLimit)
     return [{ kind: "excerpt", ...identity, text }];
   let head = 0, headCost = 0, tail = passages.length, tailCost = 0;
-  while (head < tail && headCost + costs[head]! <= headLimit && escapedLength(text.slice(0, passages[head]!.end)) <= charLimit) { headCost += costs[head++]!; }
+  while (head < tail && headCost + costs[head]! <= headLimit && fits(0, passages[head]!.end)) { headCost += costs[head++]!; }
   if (tailLimit > 0) {
-    while (tail > head && tailCost + costs[tail - 1]! <= tailLimit && escapedLength(text.slice(passages[tail - 1]!.start)) <= charLimit) { tailCost += costs[--tail]!; }
+    while (tail > head && tailCost + costs[tail - 1]! <= tailLimit && fits(passages[tail - 1]!.start, text.length)) { tailCost += costs[--tail]!; }
   }
   const headEnd = head ? passages[head - 1]!.end : 0;
   const tailStart = tail < passages.length ? passages[tail]!.start : text.length;

package/src/tuck/core/filter-noise.ts

@@ -1,9 +1,10 @@
 import type { NormalizedBlock } from "../types";
 import { isSourceExport, isLookupTool, isRecallTool } from "./recall-operation";
+import { isDialogue } from "./brief";
 
 /** Suppress lookup echoes, retaining recorded export attempts and exactly paired outcomes. */
 export const filterNoise = (blocks: NormalizedBlock[]): NormalizedBlock[] => blocks.filter(b =>
   !(b.kind === "tool_call" && isLookupTool(b.name) && !(isRecallTool(b.name) && isSourceExport(b.args))) &&
   !(b.kind === "tool_result" && isLookupTool(b.name) &&
     !(isRecallTool(b.name) && b.recallExport)) &&
-  !((b.kind === "user" || b.kind === "assistant" || b.kind === "notice") && !b.text.trim()));
+  !(isDialogue(b) && !b.text.trim()));

package/src/tuck/core/normalize.ts

@@ -19,14 +19,9 @@ const normalizeOne = (msg: CompilerMessage, msgIndex: number | undefined): Norma
     return blocks.length > 0 ? blocks : [{ kind: "user", text: "", sourceIndex: msgIndex }];
   }
 
-  if (msg.role === "bashExecution") {
-    const cmd = msg.command ?? "";
-    const out = msg.output ?? "";
-    const exit = msg.exitCode;
-    return [{ kind: "bash", command: cmd, output: out, exitCode: exit, cancelled: msg.cancelled, truncated: msg.truncated, sourceIndex: msgIndex }];
-  }
+  if (msg.role === "bashExecution") return [{ kind: "bash", sourceIndex: msgIndex }];
 
-  if (msg.role === "toolResult") return [{ kind: "tool_result", name: msg.toolName, text: sanitize(textOf(msg.content)), sourceIndex: msgIndex }];
+  if (msg.role === "toolResult") return [{ kind: "tool_result", name: msg.toolName, sourceIndex: msgIndex }];
 
   if (msg.role === "assistant") {
     if (!msg.content) return [];
@@ -47,7 +42,6 @@ const normalizeOne = (msg: CompilerMessage, msgIndex: number | undefined): Norma
           kind: "tool_call",
           name: part.name,
           args: part.arguments,
-          callId: part.id,
           sourceIndex: msgIndex,
         });
       }

package/src/tuck/core/shrink.ts

@@ -102,12 +102,12 @@ const createLane = (role: Role, units: DialogueBlock[], mandatory: ReadonlySet<n
       const empty = emptied[tier]!, counts = emptyCounts[tier]!;
       count += counts[end]! - counts[drop]!;
       if (counts[end]! > counts[drop]!) {
-        const extra: number[] = [];
-        for (let i = drop; i < end && extra.length < 9; i++) if (empty[i]) {
+        // Text blocks of one message share a locator: collect distinct ones, enough to tell whether the shown list is complete.
+        const extra = new Set<number>();
+        for (let i = drop; i < end && (extra.size < 9 || !unlocated); i++) if (empty[i]) {
           const source = units[i]!.sourceIndex;
-          if (source === undefined) unlocated = true; else extra.push(source);
+          if (source === undefined) unlocated = true; else if (extra.size < 9) extra.add(source);
         }
-        for (let i = drop; i < end && !unlocated; i++) if (empty[i] && units[i]!.sourceIndex === undefined) unlocated = true;
         sources = [...new Set([...sources, ...extra])].sort((a, b) => a - b).slice(0, 9);
       }
     }
@@ -132,11 +132,10 @@ const createDialogue = (input: DialogueInput) => {
   const dialogue = input.blocks.filter(isDialogue);
   const times = dialogue.find(block => block.sourceTimes)?.sourceTimes;
   const decorate = (record: TranscriptRecord) => decorateRecord(record, times);
-  const roleOf = (block: DialogueBlock): Role => block.kind;
-  const lanes = Object.fromEntries(ROLES.map(role => [role, createLane(role, dialogue.filter(block => roleOf(block) === role), input.mandatorySourceIndices, decorate)])) as
+  const lanes = Object.fromEntries(ROLES.map(role => [role, createLane(role, dialogue.filter(block => block.kind === role), input.mandatorySourceIndices, decorate)])) as
     Record<Role, ReturnType<typeof createLane>>;
   const seen: Record<Role, number> = { user: 0, assistant: 0, notice: 0 };
-  const order = dialogue.map(block => ({ role: roleOf(block), i: seen[roleOf(block)]++ }));
+  const order = dialogue.map(block => ({ role: block.kind, i: seen[block.kind]++ }));
   const tools = input.blocks.length > dialogue.length ? rendered([TOOLS_OMITTED]) : undefined;
   const overhead = TRANSCRIPT_HEADER.length + DIALOGUE_SUFFIX.length;
 
@@ -204,7 +203,7 @@ export const shrinkDialogue = (input: DialogueInput, limit: number): Shrunk => {
     if (cost > 0 && cost <= limit / 3) { recent = { drop: assistant.n - 1, tier: t, end: assistant.n }; break; }
   }
   for (let d = 0; d <= recent.drop; d++)
-    if (hit = found("B", { user: WHOLE, assistant: { ...recent, drop: d, end: Math.max(recent.end, recent.drop) }, notice: clipped })) return hit;
+    if (hit = found("B", { user: WHOLE, assistant: { ...recent, drop: d }, notice: clipped })) return hit;
   // From here users give way, and the assistant keeps only that recent third.
   for (const t of tiers) for (let u = 0; u <= user.n; u++)
     if (hit = found(`C${t}`, { user: { drop: 0, tier: t, end: u }, assistant: recent, notice: clipped })) return hit;
@@ -213,6 +212,6 @@ export const shrinkDialogue = (input: DialogueInput, limit: number): Shrunk => {
   for (let d = 0; d <= notice.n; d++)
     if (hit = found("E1", { user: gone(user), assistant: recent, notice: { drop: d, tier: 1, end: notice.n } })) return hit;
   for (let d = recent.drop; d <= assistant.n; d++)
-    if (hit = found("E2", { user: gone(user), assistant: { drop: d, tier: recent.tier, end: Math.max(recent.end, d) }, notice: gone(notice) })) return hit;
+    if (hit = found("E2", { user: gone(user), assistant: { ...recent, drop: d }, notice: gone(notice) })) return hit;
   return { text: "", step: "none", size: 0 };
 };

package/src/tuck/core/summarize.ts

@@ -20,12 +20,19 @@ export interface CompileInput {
   };
 }
 
-const dialogueInput = (input: CompileInput): DialogueInput => ({
-  blocks: filterNoise(normalize(input.messages, input.sourceIndices, input.receiptContext, input.sourceEntryIds, input.sourceTimes)),
-  mandatorySourceIndices: input.ranking?.mandatorySourceIndices,
-  sourceIndices: input.sourceIndices ?? input.messages.map((_, index) => index),
-  sourceEntryIds: input.sourceEntryIds,
-});
+// A cut offers the whole dialogue and then, when it does not fit, the shrunk one, from the same input: normalize it once. The blocks are
+// never mutated, and keeping the same objects also keeps their cached passage analyses.
+const inputs = new WeakMap<CompileInput, DialogueInput>();
+const dialogueInput = (input: CompileInput): DialogueInput => {
+  let prepared = inputs.get(input);
+  if (!prepared) inputs.set(input, prepared = {
+    blocks: filterNoise(normalize(input.messages, input.sourceIndices, input.receiptContext, input.sourceEntryIds, input.sourceTimes)),
+    mandatorySourceIndices: input.ranking?.mandatorySourceIndices,
+    sourceIndices: input.sourceIndices ?? input.messages.map((_, index) => index),
+    sourceEntryIds: input.sourceEntryIds,
+  });
+  return prepared;
+};
 
 /** The whole pre-cut dialogue: user, assistant and notice records; tool actions and observations are omitted.
  * This disposable view grants neither present authority nor native tool execution. */

package/src/tuck/core/transcript.ts

@@ -106,7 +106,11 @@ const LEGACY_RECEIPT_RANGE_NOTICE = RECEIPT_RANGE_NOTICE.replace("tuck_recall",
  */
 const escapeText = (text: string): string => text.replace(/[\\\n\r]/g, c => c === "\\" ? "\\\\" : c === "\n" ? "\\n" : "\\r");
 /** The length of `text` as a record line carries it: one character more for each "\", line feed and carriage return. */
-export const escapedLength = (text: string): number => text.length + (text.match(/[\\\n\r]/g)?.length ?? 0);
+export const escapedLength = (text: string): number => {
+  let length = text.length;
+  for (let i = 0; i < text.length; i++) { const c = text.charCodeAt(i); if (c === 92 || c === 10 || c === 13) length++; }
+  return length;
+};
 const TITLE = /^[A-Z][A-Za-z0-9_-]*$/;
 const refuse = (record: TranscriptRecord): never => { throw new Error(`Cannot write a ${record.kind} record as ${TRANSCRIPT_HEADER}`); };
 

package/src/tuck/types.ts

@@ -18,6 +18,7 @@ export type NormalizedBlock = { sourceTimes?: Record<number, import("./core/sour
   | { kind: "user"; text: string; originalText?: string; sourceIndex?: number; startsSource?: true }
   | { kind: "assistant"; text: string; originalText?: string; sourceIndex?: number; startsSource?: true }
   | { kind: "notice"; text: string; originalText?: string; sourceIndex?: number; sourceEntryId?: string; startsSource?: true }
-  | { kind: "tool_call"; name: string; args: Record<string, unknown>; callId?: string; sourceIndex?: number }
-  | { kind: "tool_result"; name: string; text: string; sourceIndex?: number; recallExport?: true }
-  | { kind: "bash"; command: string; output: string; exitCode: number | undefined; cancelled?: boolean; truncated?: boolean; sourceIndex?: number });
+  // Tool history renders only as one omission notice, so its blocks keep what lookup-echo filtering reads.
+  | { kind: "tool_call"; name: string; args: Record<string, unknown>; sourceIndex?: number }
+  | { kind: "tool_result"; name: string; sourceIndex?: number; recallExport?: true }
+  | { kind: "bash"; sourceIndex?: number });

package/src/tuck.ts

@@ -58,11 +58,13 @@ export function historicalAllowance(window: number, reserve: number, base: numbe
     return Math.max(0, Math.min(Math.floor(free / 4), before - base - 1)) * 4;
 }
 /** Uses pi-tuck's own cut, calibrated maximum tail and dialogue compiler; Pi owns persistence and continuation. */
-export function tuckDraft(tuck: Tuck, event: SessionBeforeCompactEvent, ctx: ExtensionContext, reserveTokens: number, measure: Measure, policy: RecallPolicy = {}, retainCurrentInput = true, operatorManual = false) {
- return tuckObservedDraft(tuck, { branchEntries: event.branchEntries, tokensBefore: event.preparation.tokensBefore, previousSummary: event.preparation.previousSummary, operatorManual }, ctx, reserveTokens, measure, policy, retainCurrentInput);
+export function tuckDraft(tuck: Tuck, event: SessionBeforeCompactEvent, ctx: ExtensionContext, reserveTokens: number, measure: Measure, policy: RecallPolicy = {}, retainCurrentInput = true, operatorManual = false, newPrompt = false) {
+ return tuckObservedDraft(tuck, { branchEntries: event.branchEntries, tokensBefore: event.preparation.tokensBefore, previousSummary: event.preparation.previousSummary, operatorManual, newPrompt }, ctx, reserveTokens, measure, policy, retainCurrentInput);
 }
 /** Shared compiler observes canonical branch and current token estimate; no native preparation utility. */
-export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEntry[]; tokensBefore: number; previousSummary?: string; operatorManual?: boolean }, ctx: ExtensionContext,
+/** `newPrompt`: the cut is planned for a run that has not begun (a prompt Pi has not appended yet, an operator's manual cut, Pi's own cut before
+ * a run a custom message starts); the session has moved on from an ended run. */
+export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEntry[]; tokensBefore: number; previousSummary?: string; operatorManual?: boolean; newPrompt?: boolean }, ctx: ExtensionContext,
  reserveTokens: number, measure: Measure, policy: RecallPolicy = {}, retainCurrentInput = true): Draft {
  const { isHistorySource, readable } = createRecall(policy);
 	const projected = ctx.sessionManager.buildSessionProjection();
@@ -102,10 +104,14 @@ export function tuckObservedDraft(tuck: Tuck, input: { branchEntries: SessionEnt
         const calls = new Set(response.message.content.flatMap(block => block.type === "toolCall" ? [block.id] : []));
         const hasFreshReceipt = entries.slice(lastResponse + 1).some(entry => entry.type === "message" &&
             entry.message.role === "toolResult" && calls.has(entry.message.toolCallId));
-        freshRound = hasFreshReceipt;
+        // Fresh means the receipts are the answer the next request of this run carries. Once the session has moved on (the cut is planned for a
+        // run that has not begun, or a prompt has arrived since them) they are ordinary history, whatever ended their run: a reply discarded
+        // or one Pi hid when it recovered. Kept native, a receipt that never fits would refuse every later prompt. A retry is inside its run.
+        const superseded = input.newPrompt || entries.slice(lastResponse + 1).some(entry => entry.type === "message" && entry.message.role === "user");
+        freshRound = hasFreshReceipt && !superseded;
         const keptId = cut.firstKeptEntryId;
         const chosenBoundary = cut.compactAll ? entries.length : entries.findIndex(entry => entry.id === keptId);
-        if (hasFreshReceipt && lastResponse < chosenBoundary) {
+        if (freshRound && lastResponse < chosenBoundary) {
             const prefix = tuck.buildOwnCut(entries.slice(0, lastResponse), 0);
             if (!prefix.ok) return { unavailable: "No reducible history before the fresh tool round" };
             cut = { ...cut, messages: prefix.messages, selectedIds: prefix.selectedIds,