pi-tuck 1.0.16 → 1.0.18

package/CHANGELOG.md

@@ -3,6 +3,81 @@
 All notable changes to pi-tuck. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
 [Semantic Versioning](https://semver.org/).
 
+## [1.0.18] - 2026-11-08
+
+### Fixed
+
+- **A summary could drop the task a session was working on (1.0.13 to 1.0.17).** Separate rules decided what a summary quoted and what the kept tail
+  held. In a session with many turns, the prompt that set the current task, or the first prompt, could lose its opening after a cut. The settings stress
+  test found it 40 times in 32 of its cells on 1.0.16 and 1.0.17. A cut now fills its room by one ranked list, across the summary and the kept tail
+  together:
+  1. the current prompt, whole;
+  2. the openings of the first prompt and the newest answered prompts, sized to the room;
+  3. the newest part of the conversation, kept as it is;
+  4. the file and action lists;
+  5. the rest of the conversation.
+
+  The stress test now finds no lost opening in 287 cells. Dropped turns and tool results are counted with index ranges you can read back with
+  `tuck_recall`.
+
+### Changed
+
+- **One number sets how much a cut carries, and the decision to cut reads only amounts.** The quote budget and its fallback steps are gone, and so are
+  the rules that compared two versions of a summary. The kept tail is at most half of the 25,000-token tail cap, the size the cut point already assumed.
+- **pi-tuck's tools appear only after its first cut.** Before that, there is nothing out of view to recall. Until then a request is byte-identical to one
+  without pi-tuck: 254 of 254 requests on replay. Every cut plans room for the tools the requests after it carry.
+- **A prompt's late cut is planned when the prompt joins the session,** after other extensions have changed the system prompt and before Pi fixes the
+  request's tools. It replaces 1.0.17's reduction in the final guard and keeps its effect: a host section added in any load order is cut, not refused.
+- **Cuts compute faster:** about half the time of 1.0.17 on the regression sessions, and 76 ms against 159 ms at 3,888 entries.
+- An answered prompt with an image keeps its text opening, and a current prompt of only spaces is kept.
+
+### Correction to 1.0.16
+
+- 1.0.16's notes said its cut rate and cost were unchanged. That held on replay of single-task sessions, but not in live 30-task chains at 81,920
+  against 1.0.13. There it took 1.48 times the turns (95% interval 1.34 to 1.64), 74% more cuts and 1.66 times the tokens, with resolved 56 against 55
+  ([benchmark/RESULTS-V1016.md](benchmark/RESULTS-V1016.md)). Most of the extra cuts follow from the extra turns. Replay cannot show this cost:
+  replay puts 1.0.13 and 1.0.17 within 0.1% of each other.
+
+### Known issues
+
+- On replay of 17 recordings, estimated request time against 1.0.13 is 2.2% higher quiet and 4.2% loaded at 81,920, and 1.1% and 1.9% lower at
+  131,072. Billed input moves the same way. At 81,920 each cut carries about 600 more tokens.
+- Two stress findings from earlier versions remain: a cut that frees little with a 40,000-token setup at 65,536, and a fresh assistant reply larger than
+  the free room.
+- Sessions that are open keep the build they started with until they are restarted.
+
+### Release gates
+
+- **Shipped before its live gate, as a fix for lost work.** The gate runs right after release; if it fails, npm and this version are rolled back to
+  1.0.17.
+- Replay on 29 recordings: 0 lost tasks, 0 dead tasks, 0 faults, every clip read back.
+- Tests, CI, the settings stress test (287 cells, no cut over its target), and the shape checks (21 cells at 32,768, 81,920 and 131,072).
+
+## [1.0.17] - 2026-11-08
+
+### Fixed
+
+- **A host that patches a large section into the system prompt could get a refusal a cut would fit (1.0.0 to 1.0.16).** pi-tuck plans a prompt's cut when
+  the prompt arrives, before other extensions change the system prompt. A host engine that adds a section of thousands of tokens for one turn, as an
+  orchestrator does for its stand-off, could push that turn's first request over the room, and the final guard refused it. The guard now reduces first:
+  when the run's first request is over the room and no cut is planned, it plans the prompt's cut there, against the system prompt as finally sent, and
+  refuses only if that cut does not fit either. It works whatever order the extensions load in.
+
+### Added
+
+- Stress cells for a host that patches an 8k then 24k system section, at 65,536 and 81,920, registered before and after pi-tuck, with the section arriving
+  on a stand-off or on the first step turn after one and removed at the next stand-off. 1.0.16 refuses in the four cells at 81,920; 1.0.17 in none.
+
+### Changed
+
+- When queued input alone exceeds the room, the refusal names it as incoming input that cannot fit, instead of complete context. It still fails closed
+  with no cut.
+
+### Release gates
+
+- Replay matches 1.0.16: 0 of 13,946 requests differ on 29 recordings, 0 lost or dead tasks. Shipped without a live gate as a replay-matching patch.
+  Tests; CI; the stress test, with no cell worse than 1.0.16.
+
 ## [1.0.16] - 2026-11-08
 
 ### Fixed

package/README.md

@@ -64,8 +64,8 @@ Then start a new Pi session. That is all. pi-tuck takes over Pi's compaction, bo
 
 - Needs Pi 1.2.0 or later (tested on 1.2.0 and 1.4.0), on macOS or Linux. On an older Pi, pi-tuck stays off, Pi's own compaction runs, and pi-tuck says so once. Full features on macOS with Apple silicon; elsewhere, compaction and word recall.
 - On macOS with Apple silicon, the first session also downloads a local search model in the background (about 390 MB, once). Until it is ready, search works by keyword. See [Semantic search](#semantic-search).
-- Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.15`.
-- To pin a version, add it: `pi install npm:pi-tuck@1.0.15`.
+- Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.17`.
+- To pin a version, add it: `pi install npm:pi-tuck@1.0.17`.
 
 ## How it works
 
@@ -74,7 +74,7 @@ When context fills up, pi-tuck replaces the older turns with one summary. The su
 - **The dialogue.** What you asked and what the agent answered, in order, word for word. If it is too long for the room, it shrinks: the oldest assistant text is clipped first, then older prompts. Clipped text says how to read it in full.
 - **The files.** Files the agent read, wrote or edited, newest first (up to 60), each with a pointer to read it back.
 - **The action trail.** The agent's other tool calls, such as shell commands, newest first (up to 30). The output is left out. A pointer leads to it.
-- **Your first prompt**, word for word (up to 12,000 characters).
+- **Your prompts, quoted from both ends** (1.0.16 and later): first the opening of every recent answered turn, then the first prompt's opening (up to 2,000 characters), then more of the newest turns, then the rest of the first prompt. A turn that does not fit keeps its opening and a locator to the rest. The budget is 15% of the room and never less than 12,000 characters. When room is tight, the quotes shrink step by step: the extensions first, then the openings, then the first prompt alone. Before 1.0.16 a summary quoted only the first prompt (up to 12,000 characters) and the newest turn; see Known limits.
 
 Tool output stays in the session file, not in the summary. Each summary ends with a short instruction to the agent: re-read a file before you edit it, look things up, and treat anything not shown as unknown.
 
@@ -114,8 +114,11 @@ Known limits today:
 - Small windows: an upstream that spends far more context on tool schemas than others (some OpenRouter upstreams do) can leave a small input room too little space. pi-tuck then refuses to cut rather than overflow. This made 1.0.1 refuse at a 32k window with Pi's default 16,384 reserve. Version 1.0.4 added a notice that says when the provider counts far above pi-tuck's estimate. In 1.0.4, after an oversize tool result or a "Nothing to compact" error from Pi, a session could keep refusing later prompts; 1.0.5 fixed both. Version 1.0.8 clips a fresh tool result that is too large for the room (its head and tail, with a `tuck_recall` locator to the rest) instead of refusing the run. If a small window still refuses, pin a different upstream, lower the reserve, or use a larger window; see [Small windows](#small-windows).
 - Small window, after a stopped run (1.0.8 and 1.0.9 only; fixed in 1.0.10): Pi's default `compaction.keepRecentTokens` (20,000) can exceed pi-tuck's input room, which is the window less the larger of `compaction.reserveTokens` and the model's maximum output (16,384 at a 32k window with the default reserve). After a run that is stopped (Esc, or a time limit) with the context above the room, every later prompt could be refused with "The request needs a cut and Pi has none to make (Nothing to compact (session too small))", and the session stayed stuck. In one check at these defaults (pi-tuck 1.0.8, 32k window, one session of 30 tasks, one local model), 28 tasks were lost, tasks 3 to 30 ([`benchmark/RESULTS-DEFAULTS.md`](benchmark/RESULTS-DEFAULTS.md)). If you are on 1.0.8 or 1.0.9, upgrade to 1.0.10. In 1.0.10 the cut a prompt needs is pi-tuck's own, so `keepRecentTokens` no longer leaves it with nothing to cut. A rerun of the same check on the 1.0.10 candidate had no refusal and no stuck session. It had two caveats: two of its tasks had a first request the local server never answered, and 8 of its 30 tasks began without their prompt, which the refusal counter cannot see (fixed in 1.0.12; see Benchmarks and [`benchmark/RESULTS-DEFAULTS-1010.md`](benchmark/RESULTS-DEFAULTS-1010.md)). Windows of 128k and up were not affected at the defaults.
 - A prompt could be lost, with no notice (1.0.10 and 1.0.11; fixed in 1.0.12). When a new prompt arrived over the cut point and pi-tuck's cut kept nothing of the conversation before it, which happens when everything since the last cut is one long turn, the prompt was dropped from the request and from the saved session. The model never saw it and usually went on with the previous task, so work you asked for could be lost, and could look answered. The next prompt ran normally. If you used 1.0.10 or 1.0.11 and a reply ignored what you had just asked, this is the likely cause. Upgrade to 1.0.12 ([`CHANGELOG.md`](CHANGELOG.md) 1.0.12).
+- A host that patches a large section into the system prompt could get a refusal that a cut would have fit (1.0.0 to 1.0.16; fixed in 1.0.17). pi-tuck plans a prompt's cut when the prompt arrives, before other extensions change the system prompt. A host engine that adds a section of thousands of tokens for one turn could push that turn's first request over the room, and the final guard refused it. Now the guard plans the cut against the system prompt as finally sent, and refuses only if that cut does not fit either ([`CHANGELOG.md`](CHANGELOG.md) 1.0.17).
+- A long session could lose the task it was working on (1.0.0 to 1.0.15; fixed in 1.0.16). A summary quoted only the session's first user turn, up to 12,000 characters, and the newest one. Every other user turn competed in the shrunk dialogue, which in a long session clips user turns to a few words and drops the oldest. In a session that runs task after task, the current task is an older turn followed by short steps, so it was clipped or dropped while a finished first task stayed whole, and agents acted on a days-old task. Short sessions, and the SWE-bench chains behind our benchmarks, keep each task's prompt in the kept tail and never hit it. Upgrade to 1.0.16 ([`CHANGELOG.md`](CHANGELOG.md) 1.0.16).
+- At very small free room (a few thousand tokens or less), the quotes give way and an older prompt can still lose its opening, in 1.0.16: 40 findings in 32 stress cells, against 447 in 168 cells on 1.0.15 ([`CHANGELOG.md`](CHANGELOG.md) 1.0.16).
 - The cut point adapts by itself since 1.0.13; before it, pi-tuck cut too often at small windows or with a large setup (1.0.11 and 1.0.12). The trade, from the changelog: on windows that cut too often, each request now carries more context. Cuts, re-processing and request time fall, and billed tokens rise on paid models. On replay of real sessions at an 81,920 window, cuts went from 22.3 to 9.3 per 100 requests, request time was about 16 to 20% lower, and billed tokens were 16 to 19% higher; at a 65,536 window billed tokens rose by as much as 35%. A very large setup at a small window, such as 40k at 65,536, still leaves too little free room for a good rhythm: cuts stay frequent and free little ([`CHANGELOG.md`](CHANGELOG.md) 1.0.13).
-- Still applying (1.0.11 to 1.0.13): a fresh assistant message larger than the free room is refused, and a short next prompt recovers (in an unattended `pi -p` run, a refusal ends the run). A reply that reports more than the window reads to Pi as a silent overflow; when nothing can be cut, pi-tuck refuses the next prompt, and the prompt after that runs. If a provider keeps rejecting requests in a wording Pi does not class as a context overflow, or with a bare HTTP 413, pi-tuck does not recover on its own: the session stays intact, and every prompt can fail until the request shrinks. Try `/compact` ([`CHANGELOG.md`](CHANGELOG.md) 1.0.13).
+- Still applying (1.0.11 to 1.0.16): a fresh assistant message larger than the free room is refused, and a short next prompt recovers (in an unattended `pi -p` run, a refusal ends the run). A reply that reports more than the window reads to Pi as a silent overflow; when nothing can be cut, pi-tuck refuses the next prompt, and the prompt after that runs. If a provider keeps rejecting requests in a wording Pi does not class as a context overflow, or with a bare HTTP 413, pi-tuck does not recover on its own: the session stays intact, and every prompt can fail until the request shrinks. Try `/compact` ([`CHANGELOG.md`](CHANGELOG.md) 1.0.13).
 - Other extensions (1.0.10): a prompt's cut no longer goes through Pi's `session_before_compact`, so another extension's veto there no longer stops it. It never stopped a cut between tool rounds. Pi's own threshold and overflow cuts, and `/compact`, still ask other extensions as before.
 - Semantic search (`tuck_search`) needs a Mac with Metal (Apple silicon). Elsewhere, compaction and `tuck_recall`'s word search work, there is no `tuck_search`, and nothing is downloaded.
 

package/dist/src/budget.d.ts

@@ -25,11 +25,19 @@ export declare const validCutAt: (value: unknown) => value is number;
 /** The initial default cut point: 60% of the room before a usable previous cut. Measured against 1 with paired replays (no answer loss) and billed
  * input under automatic and explicit prompt caching (about 17% less at 128k, 10-14% at 32k). */
 export declare const DEFAULT_CUT_AT = 0.6;
+/** Preferred native carry, in the planner's estimate: never more than the
+ * keep-recent cut offers or the tail amount the adaptive band assumes. */
+export declare const carriedTailTokens: (offeredTokens: number) => number;
 export declare const bandOf: (room: number, cutAt?: number, adaptive?: {
     system: number;
     tokensAfter: number;
     factor: number;
 }) => number;
+/** The cut's one carried-context grant, in the planner's estimate. Combines
+ * the existing native/fixed base with the quarter-free historical amount;
+ * neither component remains an independent spending limit in the fill.
+ * The grant depends only on quantities. */
+export declare function carriedTarget(window: number, reserve: number, base: number, before: number): number;
 /**
  * Replayed reasoning that Pi's estimator does not see. On the OpenAI Responses APIs an assistant turn's reasoning item rides in its thinking
  * block's `thinkingSignature` (encrypted) and is sent back with later requests (pi-ai openai-responses-shared; transform-messages skips
@@ -61,11 +69,11 @@ export declare const measurableConversationTokens: (messages: Messages, ctx: Ext
 /** Raw reserve for the cut planner: the window less the raw tokens that fit the target room, plus the input no cut can remove. */
 export declare const planningReserve: (window: number, targetRoom: number, factor: number, mandatory: number) => number;
 export type Target = "band" | "room";
-/** A planned cut: the conversation it leaves, and the summary view that produced it (whole dialogue or shrunk). */
+/** A planned cut: the conversation it leaves. The optional mode is diagnostic data, not a decision input. */
 export type Plan<D extends object = {}> = ({
     draft: true;
     after: number;
-    mode: string;
+    mode?: string;
 } & D) | {
     draft: false;
     capacityRejected: boolean;
@@ -110,21 +118,10 @@ export interface CutInput<D extends object = {}> {
 export type Verdict<D extends object = {}> = Exclude<Decision<D>, {
     kind: "defer";
 }>;
-/**
- * Rule 4. The exact room is planned first. When its plan already lands below the band it is taken as it is (target "room"): it keeps the
- * room plan's content exactly, because every allowance of the planner is derived from the reserve and the band's is smaller. Only
- * otherwise is the band planned. Inside the band a cut is opportunistic, so a failed plan is never a fault and never falls back to the
- * room (a cut that still lands in the band would trigger again every round); the exact room would cut nothing there, so a band cut is
- * taken only when the room can plan a cut too and with the same mode (a lossy view the room could not even plan is a quality change with
- * no fault to prevent). At or above the room, where a cut is due, a band plan of another mode gives way to the room plan (today's
- * view), and a band plan stands when the room has none (a cut instead of a fault).
- * A lean band, one the cut point binds (`band < bandOf(room)`), is planned first and the room plan landing below it is not taken for that:
- * the room plan keeps a native tail of up to half the room, so it can land just under a band that far below the room, and the next round
- * would cut again, freeing little. The room is planned only where the rules above use it: to compare the mode, or when the band has no plan
- * and a cut is due. An adaptive band can also reach the margin, in which case the room plan is tried first.
- * The margin adds no new decision fault. Exceptions thrown by `plan` are not caught here: they are faults, and they now surface at
- * boundaries inside the band too, where no planner ran before the margin.
- */
+/** Plan the band first at every cut point. An opportunistic cut must land
+ * below it and free the existing minimum. A due cut takes a band draft, or
+ * falls back to the room only when the band has no draft. Final reduction and
+ * room-fit checks own faults. Planner exceptions propagate. */
 export declare function chooseCut<D extends object = {}>(input: CutInput<D> & {
     opportunistic: boolean;
 }): Verdict<D>;

package/dist/src/budget.js

@@ -14,11 +14,24 @@ export const validCutAt = (value) => typeof value === "number" && value > 0 && v
 /** The initial default cut point: 60% of the room before a usable previous cut. Measured against 1 with paired replays (no answer loss) and billed
  * input under automatic and explicit prompt caching (about 17% less at 128k, 10-14% at 32k). */
 export const DEFAULT_CUT_AT = 0.6;
+/** Preferred native carry, in the planner's estimate: never more than the
+ * keep-recent cut offers or the tail amount the adaptive band assumes. */
+export const carriedTailTokens = (offeredTokens) => Math.min(offeredTokens, MAX_TAIL_TOKENS / 2);
 export const bandOf = (room, cutAt = 1, adaptive) => {
     const positive = Math.max(0, room);
     const base = Math.floor(positive * cutAt);
-    return Math.min(room - Math.ceil(positive * TRIGGER_MARGIN_FRACTION), adaptive ? Math.max(base, Math.ceil((adaptive.system + adaptive.tokensAfter + MAX_TAIL_TOKENS / 2) * adaptive.factor)) : base);
+    return Math.min(room - Math.ceil(positive * TRIGGER_MARGIN_FRACTION), adaptive ? Math.max(base, Math.ceil((adaptive.system + adaptive.tokensAfter + carriedTailTokens(MAX_TAIL_TOKENS)) * adaptive.factor)) : base);
 };
+/** The cut's one carried-context grant, in the planner's estimate. Combines
+ * the existing native/fixed base with the quarter-free historical amount;
+ * neither component remains an independent spending limit in the fill.
+ * The grant depends only on quantities. */
+export function carriedTarget(window, reserve, base, before) {
+    const room = Math.max(0, window - reserve - 1);
+    const free = Math.max(0, window - reserve - base - 1);
+    const historical = Math.max(0, Math.min(Math.floor(free / 4), before - base - 1));
+    return Math.min(room, before - 1, base + historical);
+}
 /** The APIs whose request converter (pi-ai openai-responses-shared) sends each signed thinking block back as an input reasoning item. */
 const REPLAYING_APIS = new Set(["openai-responses", "azure-openai-responses", "openai-codex-responses"]);
 /**
@@ -70,45 +83,25 @@ export const projectedConversationTokens = (messages, ctx, policy) => conversati
 export const measurableConversationTokens = (messages, ctx, policy) => conversationTokens(measurableMessages(messages, ctx, policy));
 /** Raw reserve for the cut planner: the window less the raw tokens that fit the target room, plus the input no cut can remove. */
 export const planningReserve = (window, targetRoom, factor, mandatory) => window - Math.floor(targetRoom / factor) + mandatory;
-/**
- * Rule 4. The exact room is planned first. When its plan already lands below the band it is taken as it is (target "room"): it keeps the
- * room plan's content exactly, because every allowance of the planner is derived from the reserve and the band's is smaller. Only
- * otherwise is the band planned. Inside the band a cut is opportunistic, so a failed plan is never a fault and never falls back to the
- * room (a cut that still lands in the band would trigger again every round); the exact room would cut nothing there, so a band cut is
- * taken only when the room can plan a cut too and with the same mode (a lossy view the room could not even plan is a quality change with
- * no fault to prevent). At or above the room, where a cut is due, a band plan of another mode gives way to the room plan (today's
- * view), and a band plan stands when the room has none (a cut instead of a fault).
- * A lean band, one the cut point binds (`band < bandOf(room)`), is planned first and the room plan landing below it is not taken for that:
- * the room plan keeps a native tail of up to half the room, so it can land just under a band that far below the room, and the next round
- * would cut again, freeing little. The room is planned only where the rules above use it: to compare the mode, or when the band has no plan
- * and a cut is due. An adaptive band can also reach the margin, in which case the room plan is tried first.
- * The margin adds no new decision fault. Exceptions thrown by `plan` are not caught here: they are faults, and they now surface at
- * boundaries inside the band too, where no planner ran before the margin.
- */
+/** Plan the band first at every cut point. An opportunistic cut must land
+ * below it and free the existing minimum. A due cut takes a band draft, or
+ * falls back to the room only when the band has no draft. Final reduction and
+ * room-fit checks own faults. Planner exceptions propagate. */
 export function chooseCut(input) {
     const { before, mandatory, room, band, fits, plan, unreduced } = input;
     const lands = (p) => p.draft && p.after < before && fits(p.after + mandatory, band);
     // A cut no one needs yet must be worth its summary: one that frees under a tenth of the room would trigger again within a round or two.
-    // Without this, a band at the cap takes the room plan on every round where free room is small. A due cut is not asked: it has to happen.
+    // A due cut is not asked: it has to happen.
     const worthwhile = (p) => !input.usefulOnly || !input.opportunistic || !fits(10 * (before - p.after), room);
-    let roomPlan;
-    const exact = () => roomPlan ??= plan("room");
-    if (band >= bandOf(room)) {
-        const e = exact();
-        if (lands(e) && worthwhile(e))
-            return { kind: "cut", target: "room", plan: e };
-    }
     const preferred = plan("band");
     if (input.opportunistic) {
         if (!lands(preferred) || !worthwhile(preferred))
             return { kind: "none" };
-        const e = exact();
-        return e.draft && e.mode === preferred.mode ? { kind: "cut", target: "band", plan: preferred } : { kind: "none" };
+        return { kind: "cut", target: "band", plan: preferred };
     }
-    const e = exact();
     let chosen = preferred, target = "band";
-    if (!preferred.draft || (e.draft && e.mode !== preferred.mode)) {
-        chosen = e;
+    if (!preferred.draft) {
+        chosen = plan("room");
         target = "room";
     }
     if (!chosen.draft)

package/dist/src/build-id.d.ts

@@ -1 +1 @@
-export declare const BUILD_ID = "e695eee2b3e13753eb135cb61b69783a79d72ac4c1ce08f28304b98cca6b5a49";
+export declare const BUILD_ID = "c53f4e367c1f45ac6b3865cbf25bf99992da4b7e961557e5510369dca81fb087";

package/dist/src/build-id.js

@@ -1 +1 @@
-export const BUILD_ID = "e695eee2b3e13753eb135cb61b69783a79d72ac4c1ce08f28304b98cca6b5a49";
+export const BUILD_ID = "c53f4e367c1f45ac6b3865cbf25bf99992da4b7e961557e5510369dca81fb087";

package/dist/src/compaction.js

@@ -6,15 +6,16 @@ import { isDeepStrictEqual } from "node:util";
 import { assertNativeToolRounds } from "./protocol.js";
 import { projectedMessages } from "./omissions.js";
 import { applyClips, clipBudgets, clipFor } from "./clip.js";
-import { DEFAULT_CUT_AT, bandOf, chooseCut, conversationTokens, decideCut, gateCut, incomingEventTokens, incomingTokens, measurableConversationTokens, planningReserve, projectedConversationTokens, requestTokens, systemPart, validCutAt } from "./budget.js";
-import { isContextOverflow } from "@earendil-works/pi-ai";
-import { convertToLlm, SessionManager, SettingsManager } from "@earendil-works/pi-coding-agent";
+import { DEFAULT_CUT_AT, bandOf, chooseCut, conversationTokens, gateCut, incomingEventTokens, incomingTokens, measurableConversationTokens, planningReserve, projectedConversationTokens, systemPart, validCutAt } from "./budget.js";
+import { getCurrentSystemMessage, isContextOverflow } from "@earendil-works/pi-ai";
+import { calculateContextTokens, convertToLlm, estimateTokens, shouldCompact, SessionManager, SettingsManager } from "@earendil-works/pi-coding-agent";
 import { Type } from "typebox";
 import { Check } from "typebox/value";
 import { MAX_RECALL_CHARACTERS, MAX_RECALL_ENTRY_ID_CHARACTERS, MAX_RECALL_QUERY_CHARACTERS } from "./recall.js";
 import { tuck, tuckDraft, tuckObservedDraft, tuckRecall } from "./tuck.js";
 import { ImageRecallInput, historyImage, historyAround } from "./history-image.js";
 import { eligibilityKey, eligibilityKeyMoved, memoizedEligibleSources } from "./eligibility-cache.js";
+import { applyHistoryTools, hasTuckCut, historyToolDeclarations, historyToolNames, stageHistoryTools } from "./history-tools.js";
 const query = Type.String({ minLength: 1, maxLength: MAX_RECALL_QUERY_CHARACTERS });
 const entryId = Type.String({ minLength: 1, maxLength: MAX_RECALL_ENTRY_ID_CHARACTERS });
 const paging = {
@@ -319,12 +320,20 @@ export async function installCompaction(pi, options = {}) {
     const measureFor = (ctx) => (messages) => projectedConversationTokens(applyClips(messages, clips), ctx, options.recallPolicy);
     // The planner counts with the same function, except that it does not fail on a summary in another format: a cut can replace one
     // (an operator's /compact repairs a session whose saved summary is foreign). Identical to the strict one for this compactor's summaries.
-    const plannerMeasureFor = (ctx) => (messages) => measurableConversationTokens(applyClips(messages, clips), ctx, options.recallPolicy);
+    const plannerMeasureFor = (ctx) => Object.assign((messages) => measurableConversationTokens(applyClips(messages, clips), ctx, options.recallPolicy), { messageTokens: (message) => conversationTokens(applyClips([message], clips)) });
     const hostTokens = (ctx) => options.hostMessages ? conversationTokens(options.hostMessages(ctx)) : 0;
     const toolShapes = (names) => {
         const selected = new Set(names);
         return pi.getAllTools().filter(tool => selected.has(tool.name)).map(tool => ({ name: tool.name, description: tool.description, parameters: tool.parameters }));
     };
+    // Every decision prices the declaration set a cut must carry, even before the first cut enables our tools on the wire.
+    const requestSystemPart = (observedPrompt, names, saved, guardPrompt) => {
+        const shapes = toolShapes(historyToolNames(pi, names, true));
+        // Keep the guard's observed schemas (Pi may enrich a host tool's schema), and add our future declarations to that same view.
+        // Taking max(current guard, future raw shapes) alone could lose the enrichment when the future shapes are larger.
+        const future = saved.some(message => message.role === "system") ? historyToolDeclarations(saved, shapes, true) : saved;
+        return systemPart(observedPrompt, shapes, future, guardPrompt);
+    };
     // Pi reports a manual cut with nothing to do as a failed compaction; the thrown error lacks the event's "Compaction failed: " prefix.
     const isNoWork = (message) => ["Already compacted", "Nothing to compact (session too small)"].includes(message?.replace(/^Compaction failed: /, "") ?? "");
     // A missing model is configuration, not capacity.
@@ -385,7 +394,9 @@ export async function installCompaction(pi, options = {}) {
         forgetFailure();
         phase = "decided";
         try {
+            syncHistoryTools(ctx);
             await preflight(ctx, incomingTokens(event.text, event.images), ctx.getSystemPrompt(), pi.getActiveTools());
+            syncHistoryTools(ctx);
         }
         catch (error) {
             fail(error, ctx);
@@ -477,7 +488,7 @@ export async function installCompaction(pi, options = {}) {
             // cut: the system part and the host input. A prompt's cut is pi-tuck's own and never comes here.
             const saved = ctx.sessionManager.buildSessionProjection().messages;
             const host = hostTokens(ctx);
-            const system = systemPart(ctx.getSystemPrompt(), toolShapes(pi.getActiveTools()), saved, ctx.getSystemPrompt());
+            const system = requestSystemPart(ctx.getSystemPrompt(), pi.getActiveTools(), saved, ctx.getSystemPrompt());
             const mandatory = system + host, band = decisionBand(ctx, room, system);
             const before = plannerMeasureFor(ctx)(saved);
             // Pi's own usage threshold (reason "threshold") counts the last reply's output tokens, reasoning included, plus the messages after
@@ -528,6 +539,15 @@ export async function installCompaction(pi, options = {}) {
     });
     pi.on("session_before_tree", (event) => event.preparation.userWantsSummary ? { cancel: true } : undefined);
     pi.on("session_start", () => { clips.clear(); promptCut = undefined; recorder.stop("session-transition"); proposalId = undefined; failure = undefined; admittedRun = false; pendingBoundaryCommit = false; identity = ""; factor = 1; pending = undefined; phase = "idle"; });
+    const historyToolsEnabled = (ctx) => Boolean(promptCut) || hasTuckCut(ctx.sessionManager.getBranch());
+    const syncHistoryTools = (ctx) => { if (active())
+        applyHistoryTools(pi, historyToolsEnabled(ctx)); };
+    pi.on("session_start", (_event, ctx) => syncHistoryTools(ctx));
+    pi.on("session_tree", (_event, ctx) => syncHistoryTools(ctx));
+    pi.on("session_compact", (_event, ctx) => { if (!admittedRun)
+        syncHistoryTools(ctx); });
+    pi.on("turn_start", (_event, ctx) => syncHistoryTools(ctx));
+    pi.on("agent_settled", (_event, ctx) => syncHistoryTools(ctx));
     // The session as it will be once a cut that is not saved yet is saved, and its conversation as the next request carries it.
     const cutSession = (ctx, cut) => {
         const header = ctx.sessionManager.getHeader();
@@ -542,7 +562,7 @@ export async function installCompaction(pi, options = {}) {
     // custom message starts). The cut is pi-tuck's own, planned as at a tool boundary. It is not Pi's: Pi's cut finder keeps the last
     // keepRecentTokens, and where those are the whole conversation it has nothing to cut, so every later prompt was refused for good (t06 at
     // Pi's defaults on a 32k window). Each request of the run carries the cut, and the run's first boundary saves it.
-    const preflight = async (ctx, incomingPart, observedPrompt, tools) => {
+    const preflight = (ctx, incomingPart, observedPrompt, tools, hostSnapshot) => {
         promptCut = undefined;
         promptCutAt = ctx.sessionManager.getBranch().length;
         if (!active())
@@ -555,9 +575,9 @@ export async function installCompaction(pi, options = {}) {
         const { response, room } = limits(ctx);
         // Public early view only: later handlers, image normalization and tool-loop growth remain the complete-context guard's responsibility.
         const saved = ctx.sessionManager.buildSessionProjection().messages;
-        const system = systemPart(observedPrompt, toolShapes(tools), saved, ctx.getSystemPrompt());
+        const system = requestSystemPart(observedPrompt, tools, saved, ctx.getSystemPrompt());
         const band = decisionBand(ctx, room, system);
-        const host = hostTokens(ctx), measure = measureFor(ctx), before = measure(saved);
+        const host = hostSnapshot ?? hostTokens(ctx), measure = measureFor(ctx), before = measure(saved);
         const incoming = incomingPart + system, mandatory = incoming + host;
         if (!fits(incoming, room, ctx))
             throw new CapacityError("Incoming Pi prompt and response reserve cannot fit the estimated request budget");
@@ -594,10 +614,43 @@ export async function installCompaction(pi, options = {}) {
         const prompted = phase === "decided";
         phase = "starting";
         try {
+            syncHistoryTools(ctx);
             if (prompted)
                 access(); // decided on input; a failure latched there still stops the run
             else
-                await preflight(ctx, incomingEventTokens(event), event.systemPrompt, event.systemPromptOptions.selectedTools);
+                await preflight(ctx, incomingEventTokens(event), event.systemPrompt, historyToolNames(pi, event.systemPromptOptions.selectedTools, historyToolsEnabled(ctx)));
+            // Pi declares tool changes before turn_start, then snapshots executable tools in prepareRequest. Introduce our tools in
+            // the request projection: recording an activation message here would shift every later session-global recall index.
+            if (historyToolsEnabled(ctx))
+                stageHistoryTools(pi, ctx.sessionManager.buildSessionProjection().messages);
+            else
+                applyHistoryTools(pi, false);
+            const staged = pi.getActiveTools();
+            event.systemPromptOptions.selectedTools = historyToolNames(pi, event.systemPromptOptions.selectedTools, true).filter(name => name !== "tuck_recall" && name !== "tuck_search" || staged.includes(name));
+        }
+        catch (error) {
+            fail(error, ctx);
+        }
+    });
+    // The incoming user message is about to be appended (including steering inside a run), and later before_agent_start/turn_start
+    // handlers have run. Pi's prepareRequest has not yet snapshotted executable tools. A first cut discovered in context_with_system
+    // would be too late for that snapshot, even if its declarations were repaired there, so make the late prompt decision here.
+    pi.on("message_end", (event, ctx) => {
+        if (event.message.role !== "user" || !active() || !admittedRun || promptCut)
+            return;
+        try {
+            access();
+            if (!ctx.model)
+                throw missingModel();
+            const message = event.message;
+            const incoming = incomingTokens(typeof message.content === "string" ? message.content : message.content.flatMap(b => b.type === "text" ? [b.text] : []).join("\n"), typeof message.content === "string" ? [] : message.content.filter((b) => b.type === "image"));
+            const saved = ctx.sessionManager.buildSessionProjection().messages;
+            const host = hostTokens(ctx);
+            const system = requestSystemPart(ctx.getSystemPrompt(), pi.getActiveTools(), saved, ctx.getSystemPrompt());
+            if (fits(conversationTokens(saved) + system + incoming + host, limits(ctx).room, ctx))
+                return;
+            preflight(ctx, incoming, ctx.getSystemPrompt(), pi.getActiveTools(), host);
+            syncHistoryTools(ctx);
         }
         catch (error) {
             fail(error, ctx);
@@ -620,11 +673,28 @@ export async function installCompaction(pi, options = {}) {
         // One snapshot per decision. Messages queued for the next request (steering, follow-ups) are in that request but not in the
         // saved conversation. Pi's peek returns the steering queue, else the follow-up queue; a follow-up is not in the very next
         // request while tools still run, so it can be counted early: conservative, and nothing else would cut at that later boundary.
-        const system = systemPart(ctx.getSystemPrompt(), toolShapes(pi.getActiveTools()), saved, ctx.getSystemPrompt());
+        const system = requestSystemPart(ctx.getSystemPrompt(), pi.getActiveTools(), saved, ctx.getSystemPrompt());
         const band = decisionBand(ctx, room, system);
         const host = hostTokens(ctx);
         const queued = event.context.pendingMessages.length ? conversationTokens(event.context.pendingMessages) : 0;
         const mandatory = system + host + queued;
+        // Without usable provider usage, Pi's own threshold uses its unwrapped message estimate as the compiler's density input.
+        // Our declarations are request-only, so Pi can miss that threshold. Preserve the same density input at settlement; every
+        // budget decision below still uses `system` and the complete conversation measure, including the summary envelope.
+        let densityTokens;
+        const lastAssistant = saved.findLast(message => message.role === "assistant");
+        const lastCut = ctx.sessionManager.getBranch().findLast(entry => entry.type === "compaction");
+        if (reason === "run-end" && lastAssistant && (!lastCut || lastAssistant.timestamp > Date.parse(lastCut.timestamp)) &&
+            !saved.some(message => message.role === "assistant" &&
+                !["error", "aborted"].includes(message.stopReason) && message.usage && calculateContextTokens(message.usage) > 0)) {
+            const forecast = historyToolDeclarations(saved, toolShapes(historyToolNames(pi, pi.getActiveTools(), true)), true);
+            if (forecast !== saved) {
+                const head = getCurrentSystemMessage(convertToLlm(forecast));
+                const native = (head ? estimateTokens(head) : 0) + saved.reduce((sum, message) => sum + (message.role === "system" ? 0 : estimateTokens(message)), 0);
+                if (shouldCompact(native, ctx.model.contextWindow, settingsFor().getCompactionSettings(ctx.model)))
+                    densityTokens = native;
+            }
+        }
         const previousSummary = saved.find(message => message.role === "compactionSummary")?.summary;
         // The cut previews this decision has built, in the order it built them. The clip retry below measures them for what a cut leaves the
         // request besides the fresh results. Reset per decision, so a retry sizes itself against the cut it is retrying, not an earlier attempt's.
@@ -634,18 +704,22 @@ export async function installCompaction(pi, options = {}) {
             previews = [];
             floorKept = undefined;
             const before = measure(saved);
-            return decideCut({
-                total: before + mandatory, mandatory, before, room, band, fits: (tokens, limit) => fits(tokens, limit, ctx), usefulOnly: options.cutAt === undefined,
+            const gate = gateCut({ total: before + mandatory, mandatory, room, band, fits: (tokens, limit) => fits(tokens, limit, ctx) });
+            if (gate.kind !== "plan")
+                return gate;
+            return chooseCut({
+                opportunistic: gate.opportunistic, mandatory, before, room, band, fits: (tokens, limit) => fits(tokens, limit, ctx), usefulOnly: options.cutAt === undefined,
                 unreduced: "Tool-loop preflight compaction did not reduce context within the estimated request budget",
                 plan: (target) => {
                     const reserve = planReserve(target, room, band, mandatory, ctx);
-                    const draft = tuckObservedDraft(tuck, { branchEntries: ctx.sessionManager.getBranch(), tokensBefore: before, previousSummary }, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, true);
+                    const draft = tuckObservedDraft(tuck, { branchEntries: ctx.sessionManager.getBranch(), tokensBefore: densityTokens ?? before, previousSummary }, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, true);
                     if (!draft.compaction) {
                         // The planner's own floor, so a clip's second budget is measured against what the smallest cut really keeps.
                         if (draft.floorMessages && event.toolResults) {
                             const fresh = new Set(event.toolResults.map(result => result.toolCallId));
                             const kept = measure(draft.floorMessages.filter(message => !(message.role === "toolResult" && fresh.has(message.toolCallId))));
-                            floorKept = Math.min(floorKept ?? kept, kept);
+                            // Both targets must leave their protected openings beside the clip.
+                            floorKept = Math.max(floorKept ?? kept, kept);
                         }
                         return { draft: false, capacityRejected: Boolean(draft.capacityRejected), reason: draft.unavailable };
                     }
@@ -671,7 +745,7 @@ export async function installCompaction(pi, options = {}) {
             const branch = ctx.sessionManager.getBranch();
             const entryIdOf = (toolCallId) => branch.findLast(entry => entry.type === "message" && entry.message.role === "toolResult" && entry.message.toolCallId === toolCallId)?.id;
             const fresh = new Set(event.toolResults.map(result => result.toolCallId));
-            const kept = previews.length ? Math.min(...previews.map(preview => measure(preview.filter(message => !(message.role === "toolResult" && fresh.has(message.toolCallId)))))) : floorKept;
+            const kept = previews.length ? Math.max(...previews.map(preview => measure(preview.filter(message => !(message.role === "toolResult" && fresh.has(message.toolCallId)))))) : floorKept;
             for (const budget of clipBudgets(room, factor, mandatory, kept)) {
                 const round = clipFor(event.toolResults, entryIdOf, budget);
                 if (!round.length)
@@ -720,13 +794,24 @@ export async function installCompaction(pi, options = {}) {
             if (!promptCut && !decide)
                 return;
             const result = atBoundary(event, ctx, "tool-loop", decide);
-            if (result)
+            if (result) {
                 pendingBoundaryCommit = true;
+            }
             return result;
         }
         catch (error) {
             fail(error, ctx);
         }
+        finally {
+            // Keep our part of Pi's next declaration snapshot unchanged; turn_start restores the branch-derived executable set before
+            // prepareRequest. This avoids an extra persisted system message at the first cut and on later rounds/runs.
+            if (active()) {
+                if (historyToolsEnabled(ctx))
+                    stageHistoryTools(pi, ctx.sessionManager.buildSessionProjection().messages);
+                else
+                    applyHistoryTools(pi, false);
+            }
+        }
     });
     // A request of a run whose prompt's cut is not saved yet: the saved conversation in it, which other handlers may surround, is replaced by
     // the conversation with the cut applied. Pi sends the saved system message apart from the request's messages, so neither side carries it
@@ -743,6 +828,9 @@ export async function installCompaction(pi, options = {}) {
         try {
             if (!active())
                 return;
+            // Before a cut, Pi owns the whole request, including its object identities and system-message serialization.
+            if (!historyToolsEnabled(ctx))
+                return clips.size ? { messages: applyClips(event.messages, clips) } : undefined;
             access();
             return { messages: applyClips(projectedMessages(promptCut ? cutRequest(event.messages, ctx, promptCut) : event.messages, ctx, options.recallPolicy), clips) };
         }
@@ -751,7 +839,7 @@ export async function installCompaction(pi, options = {}) {
                 fail(error, ctx);
             }
             finally {
-                return { messages: [] };
+                return historyToolsEnabled(ctx) ? { messages: [] } : undefined;
             }
         }
     });
@@ -807,22 +895,27 @@ export async function installCompaction(pi, options = {}) {
                 throw missingModel();
             // The complete request, counted after Pi's conversion (summary/custom/bash envelopes), with the effective system once.
             // Pi applies a forced prompt after this hook; the public prompt already observes it.
-            const tokens = requestTokens(event.messages, ctx.getSystemPrompt());
+            syncHistoryTools(ctx);
+            const messages = historyToolsEnabled(ctx) ? historyToolDeclarations(event.messages, toolShapes(pi.getActiveTools()), true) : event.messages;
+            const system = requestSystemPart(ctx.getSystemPrompt(), pi.getActiveTools(), messages, ctx.getSystemPrompt());
+            const tokens = system + conversationTokens(messages);
             if (!fits(tokens, limits(ctx).room, ctx))
                 throw new CapacityError("Complete Pi context and response reserve exceed the estimated request budget");
             if (recorder.enabled && !recorder.waitingForCut)
                 recorder.measureInline(() => {
                     recorder.request();
                     // No system prompt or hidden assistant blocks; no provider payload/headers.
-                    const permitted = event.messages.filter(m => m.role !== "system" &&
+                    const permitted = messages.filter(m => m.role !== "system" &&
                         !(m.role === "assistant" && m.content.some(b => b.type === "thinking")) &&
                         !(m.role === "custom" && options.recallPolicy?.privateCustomTypes?.includes(m.customType)));
                     recorder.capture("prepared-request", "public-context-with-system-hook-before-later-transforms-not-wire", { messages: permitted, estimatedInputTokens: tokens }, permitted.length !== event.messages.length);
                 });
             if (tokens > 0 && Number.isSafeInteger(tokens))
                 pending = {
-                    identity: identityFor(ctx), tokens, messages: event.messages, snapshot: structuredClone(convertToLlm(event.messages)),
+                    identity: identityFor(ctx), tokens, messages, snapshot: structuredClone(convertToLlm(messages)),
                 };
+            if (messages !== event.messages)
+                return { messages };
         }
         catch (error) {
             fail(error, ctx);

package/dist/src/diagnostics.d.ts

@@ -1,6 +1,6 @@
 import { type RecallPolicy } from './recall.js';
 import type { SessionEntry } from '@earendil-works/pi-coding-agent';
-export declare const DIAGNOSTIC_BUILD = "e695eee2b3e13753eb135cb61b69783a79d72ac4c1ce08f28304b98cca6b5a49";
+export declare const DIAGNOSTIC_BUILD = "c53f4e367c1f45ac6b3865cbf25bf99992da4b7e961557e5510369dca81fb087";
 export interface Receipt {
     id: string;
     build: string;

package/dist/src/history-tools.d.ts

@@ -0,0 +1,16 @@
+import { type ExtensionAPI } from "@earendil-works/pi-coding-agent";
+import type { Messages, ToolShape } from "./budget.js";
+/** Native Pi compactions do not make pi-tuck history available. No session-wide activation state. */
+export declare const hasTuckCut: (branch: readonly {
+    type: string;
+    details?: unknown;
+}[]) => boolean;
+/** Preserve the host's loadout and its order; only our registered tools may change. */
+export declare function historyToolNames(pi: ExtensionAPI, names: readonly string[], enabled: boolean): string[];
+export declare function applyHistoryTools(pi: ExtensionAPI, enabled: boolean): void;
+/** Match only our part of Pi's saved declaration snapshot, so its loop records no activation/removal message. Legacy sessions can
+ * already declare our tools; new sessions declare them only in the request projection. Executable tools are restored at turn_start. */
+export declare function stageHistoryTools(pi: ExtensionAPI, messages: Messages): string[];
+/** Pi has already declared the request's tools before queued user messages are appended. A prompt cut planned on such a message needs its declarations
+ * in that very request. Change only our declarations, leaving all other system content and tools in their original places. */
+export declare function historyToolDeclarations(messages: Messages, tools: readonly ToolShape[], enabled: boolean): Messages;