pi-tuck 1.0.24 → 1.0.26

package/CHANGELOG.md

@@ -3,6 +3,55 @@
 All notable changes to pi-tuck. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
 [Semantic Versioning](https://semver.org/).
 
+## [1.0.26] - 2026-11-10
+
+### Fixed
+
+- **1.0.25 failed to load in an installed Pi** ("Cannot find module '@earendil-works/pi-ai/utils/estimate'"). Pi's extension loader resolves
+  only a few pi-ai entry points for an installed package, and 1.0.25 imported another one by name.
+- **Since 1.0.23, the check of Pi's own reply cap never ran in an installed Pi**, for the same reason. It loaded only in tests and the eval
+  harness, where pi-ai resolves by name; the installed package silently fell back without it. The 4,096-token margin in the hard line held,
+  because it fell back to the same value.
+  - pi-tuck now loads Pi's reply clamp and estimator from the running Pi's own install. If it can't find them (a host embedding Pi), it
+    keeps the fallback as before.
+
+### Release gates
+
+- **New gate:** the packed tarball was loaded in the real installed Pi before publishing. It loaded, and Pi's clamp resolved from Pi's own
+  install.
+- Requests are unchanged from 1.0.25 wherever pi-ai resolves by name, which covers replay, stress and tests. Tests and CI pass.
+
+## [1.0.25] - 2026-11-10
+
+### Fixed
+
+- **A request above the room could be admitted and then refused, leaving the session stuck where native Pi would have sent.** pi-tuck checked
+  its line in its own corrected token count. Pi's reply cap uses Pi's own estimate, which is higher when the provider counts fewer tokens
+  than pi-tuck's raw measure.
+  - Now each point that sends above the room checks Pi's own cap on the request it would send.
+  - When that leaves less than the reply floor, it cuts deeper: up to three re-plans, then the deepest cut that still carries the whole
+    prompt. It refuses only if that leaves less than the floor too.
+  - In the stress test, 7 refusals now answer, 3 of which had stranded the session.
+
+### Added
+
+- **One retry after a reply that stops for length with thinking only.** pi-tuck retries the turn once, leaving the stopped reply out of the
+  request. It uses the deepest cut that still carries the whole prompt when that gives the reply more room; otherwise it retries uncut.
+  - After such a stop, the reply floor is the largest reply on the branch, the stopped one included, instead of the whole reserve. The full
+    reserve would refuse every request when the setup alone is above the room.
+  - A retried reply that stops again ends the run.
+  - `/tuck` counts retries.
+- **A notice when a run ends on a reply cut at the output limit:** "the last reply stopped at the model's output limit and may be cut off".
+  It goes to stderr when there's no UI and never enters the model's context.
+- **Stress runs strict in CI:** any finding fails it. A refusal is exempt only when an independent check allows it. One such check is Pi's
+  own reply cap on the smallest request that answers the prompt.
+
+### Release gates
+
+- **Shipped without a live gate, because its replay matches 1.0.24 exactly:** 13,946 requests on 29 recordings, 0 different, 0 lost and
+  0 dead tasks, clips 17 → 17.
+- **Stress, 310 cells, strict:** 0 findings (1.0.24: 25 lines, 14 of them this defect). Tests (731) and CI.
+
 ## [1.0.24] - 2026-11-10
 
 ### Fixed

package/README.md

@@ -64,8 +64,8 @@ Then start a new Pi session. That is all. pi-tuck takes over Pi's compaction, bo
 
 - Needs Pi 1.2.0 or later (tested on 1.2.0 and 1.4.0), on macOS or Linux. On an older Pi, pi-tuck stays off, Pi's own compaction runs, and pi-tuck says so once. Full features on macOS with Apple silicon; elsewhere, compaction and word recall.
 - On macOS with Apple silicon, the first session also downloads a local search model in the background (about 390 MB, once), and says once when it is set up. Until it is ready, search works by keyword. See [Semantic search](#semantic-search).
-- Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.23`.
-- To pin a version, add it: `pi install npm:pi-tuck@1.0.23`.
+- Prefer GitHub? `pi install git:github.com/galyavin/pi-tuck@v1.0.24`.
+- To pin a version, add it: `pi install npm:pi-tuck@1.0.24`.
 
 ## Uninstall
 
@@ -133,6 +133,7 @@ Known limits today:
 - pi-tuck refused requests that fit the window, and could strand a session (1.0.0 to 1.0.21; fixed in 1.0.22). It refused at the room, the window less the full reply reserve, where native Pi still sends. With Pi's defaults and a large setup, a session could be refused at every prompt. The final refusal now sits at the window less a reply floor ([`CHANGELOG.md`](CHANGELOG.md) 1.0.22).
 - Near the 1.0.22 refusal line, replies were cut to a few hundred tokens (1.0.22; fixed in 1.0.23). 1.0.22 left only its reply floor and did not count Pi's 4,096-token margin on the reply cap ([`CHANGELOG.md`](CHANGELOG.md) 1.0.23).
 - Cost of the 1.0.22 change: between the room and the window, the reply reserve no longer cushions a wrong count. At a cold start an undercount can reach the provider. A provider that errors is caught and learned from; a server that truncates silently truncates, as it does for native Pi. With a setup above the room, cuts are forced and frequent, each freeing little, which beats a stranded session ([`CHANGELOG.md`](CHANGELOG.md) 1.0.22).
+- A prompt's cut could reach a run it was not planned for (fixed in 1.0.24). pi-tuck plans a prompt's cut when the prompt arrives. When another extension handled that prompt, no run started, and the plan waited for the next run. A run started by a custom message carried a cut sized for the prompt that never ran, deeper than needed. After a move in the session tree (`/tree`), the plan was applied on the new branch, where its kept entry was not present, so the request lost the custom message that started the run, even on a branch that needed no cut ([`CHANGELOG.md`](CHANGELOG.md) 1.0.24).
 - Four `tuck_search` faults, fixed in 1.0.23: it said semantic recall was still indexing when the history was fully indexed; in the turn after a prompt's cut it ranked nothing ("nothing is out of view"); a keyword-search timeout on a very long history failed the whole search, where now the semantic results still answer; and the timeout message named a pattern instead of the history's size ([`CHANGELOG.md`](CHANGELOG.md) 1.0.23).
 - pi-tuck cut earlier than native Pi when a model's reply cap is above Pi's reserve (1.0.0 to 1.0.19; fixed in 1.0.20). It reserved the larger of Pi's `compaction.reserveTokens` and the model's `maxTokens`: with `maxTokens` 32,768 and a reserve of 16,384 on an 81,920 window, its room fell to 49,152 tokens, where native Pi compacts at 65,536 ([`CHANGELOG.md`](CHANGELOG.md) 1.0.20).
 - A summary could drop the task a session was working on (1.0.0 to 1.0.17; fixed in 1.0.18). Until 1.0.15 a summary quoted only the session's first user turn, up to 12,000 characters, and the newest one, so in a long session the current task, an older turn followed by short steps, was clipped or dropped while a finished first task stayed whole, and agents acted on a days-old task. 1.0.16 added more quotes. From 1.0.13 to 1.0.17, separate rules decided what a summary quoted and what the kept tail held, and the prompt that set the current task, or the first prompt, could lose its opening after a cut: the stress test found it 40 times in 32 cells on 1.0.16 and 1.0.17. Since 1.0.18 one ranked list fills a cut's room, and the stress test finds no lost opening in 287 cells. Short sessions, and the SWE-bench chains behind our benchmarks, keep each task's prompt in the kept tail and never hit it. Upgrade to 1.0.18 ([`CHANGELOG.md`](CHANGELOG.md) 1.0.18).

package/dist/src/build-id.d.ts

@@ -1 +1 @@
-export declare const BUILD_ID = "c3eefa3bb76e52635b1e8394a025d3ffd6a056fc5636ed9b35f89dfddc1a322a";
+export declare const BUILD_ID = "e95edde95c614d01e1cbf6489134676fac0c1bbf6aa57ef639ffac1e7e105041";

package/dist/src/build-id.js

@@ -1 +1 @@
-export const BUILD_ID = "c3eefa3bb76e52635b1e8394a025d3ffd6a056fc5636ed9b35f89dfddc1a322a";
+export const BUILD_ID = "e95edde95c614d01e1cbf6489134676fac0c1bbf6aa57ef639ffac1e7e105041";

package/dist/src/compaction.d.ts

@@ -51,7 +51,8 @@ export interface CompactionOptions {
     cutAt?: number;
     /** A host's response reserve: without `responseExact: true`, the catalogue maximum output stays its floor; with it, `response` is exact.
      * Standalone cuts against Pi's reserveTokens, then admits up to the window less Pi's safety margin and max(3,584, largest branch output), capped at that reserve.
-     * After a length stop or context overflow it holds the full reserve, also raised to the catalogue maximum.
+     * After a length stop with text/tool output or context overflow it holds the full reserve, also raised to the catalogue maximum.
+     * Thinking-only exhaustion keeps Pi's reserve and requires the branch's largest observed reply instead.
      * Resume, fork and /tree follow that branch's history; a new session has no inherited fallback. */
     requestReserve?: (ctx: ExtensionContext) => {
         response: number;

package/dist/src/compaction.js

@@ -1,12 +1,12 @@
 import { textOf } from "./tuck/core/content.js";
-import { requestReplyFloor, carriesPrompt, piContextSafetyMargin } from "./request-capacity.js";
+import { requestReplyFloor, carriesPrompt, needsFullReplyReserve, thinkingOnlyLength, piContextSafetyMargin, loadPiRequestMath } from "./request-capacity.js";
 import { BUILD_ID } from "./build-id.js";
 import { createSemanticRecall, budgetSemantic } from "./semantic-recall.js";
 import { DIAGNOSTIC_BUILD, IncidentRecorder, attachReceipt, permittedEntries, receiptId, findReceiptEntry } from "./diagnostics.js";
 import { createHash } from "node:crypto";
 import { exportSource } from "./source-export.js";
 import { isDeepStrictEqual } from "node:util";
-import { assertNativeToolRounds } from "./protocol.js";
+import { assertNativeToolRounds, isDiscardedResponse } from "./protocol.js";
 import { projectedMessages } from "./omissions.js";
 import { applyClips, clipBudgets, clipFor } from "./clip.js";
 import { DEFAULT_CUT_AT, bandOf, chooseCut, conversationTokens, gateCut, incomingEventTokens, incomingTokens, measurableConversationTokens, planningReserve, projectedConversationTokens, systemPart, validCutAt } from "./budget.js";
@@ -104,9 +104,9 @@ export function seedWindow(branch, key) {
     return initialWindow(key);
 }
 export async function installCompaction(pi, options = {}) {
-    const piMargin = options.requestReserve ? 0 : await piContextSafetyMargin();
-    const piClamp = options.requestReserve ? undefined : await import('@earendil-works/pi-ai/api/simple-options')
-        .then(module => module.clampMaxTokensToContext).catch(() => undefined);
+    const piMath = await loadPiRequestMath();
+    const piMargin = options.requestReserve ? 0 : await piContextSafetyMargin(piMath);
+    const piClamp = piMath?.clampMaxTokensToContext;
     const cutAt = options.cutAt ?? DEFAULT_CUT_AT;
     if (!validCutAt(cutAt))
         throw new Error("Invalid cut point: cutAt must be a number above 0 and at most 1");
@@ -224,6 +224,7 @@ export async function installCompaction(pi, options = {}) {
     // Pi's native start/settled events bracket the run and its post-turn cuts.
     let admittedRun = false;
     let runPrompt;
+    let thinkingRetryPending = false;
     const protectedPrompt = (ctx) => runPrompt ? ctx.sessionManager.getBranch().findLast(entry => entry.type === "message" && entry.message.role === "user" && textOf(entry.message.content).includes(runPrompt))?.id : undefined;
     const promptRefusal = () => new Refusal("Prompt check failed: the whole starting prompt cannot fit after cutting to the band and room, shrinking the summary and clipping fresh results. Shorten the prompt or choose a model with a larger context window, then retry");
     let estimatedTokens = 0;
@@ -301,21 +302,21 @@ export async function installCompaction(pi, options = {}) {
         if (input > 0 && input <= ctx.model.contextWindow && Number.isSafeInteger(input) && Number.isFinite(ratio))
             learn(ratio);
     });
-    pi.on("agent_start", () => { admittedRun = true; pending = undefined; phase = "idle"; });
-    pi.on("agent_settled", () => { admittedRun = false; runPrompt = undefined; pending = undefined; phase = "idle"; forgetFailure(); clips.clear(); promptCut = undefined; });
+    pi.on("agent_start", () => { admittedRun = true; thinkingRetryPending = false; pending = undefined; phase = "idle"; });
+    pi.on("agent_settled", () => { admittedRun = false; runPrompt = undefined; thinkingRetryPending = false; pending = undefined; phase = "idle"; forgetFailure(); clips.clear(); promptCut = undefined; });
     const active = () => options.active?.() ?? true;
     const settingsFor = () => options.settings ?? SettingsManager.inMemory(pi.getSettings());
     // The one reserve decision; every consumer takes its room from limits. Standalone starts at Pi's native cut point. Once a reply did
-    // not fit (a length stop, or a refusal pi-ai classes as a context overflow), hold the catalogue maximum too. Other errors, such as a 503,
-    // keep native's room. Read the whole branch, including history before compactions, each time: no flag to persist or reset on resume,
+    // not fit (a length stop with text/tools, or a refusal pi-ai classes as a context overflow), hold the catalogue maximum too. A thinking-only
+    // length stop uses branchReplyFloor's observed output instead; reserving the catalogue maximum would strand setups above the room.
+    // Other errors, such as a 503, keep native's room. Read the whole branch, including history before compactions, each time: no flag to persist or reset on resume,
     // fork, /tree or a new session.
     const requestReserve = (ctx) => {
         const host = options.requestReserve?.(ctx);
         const budget = host ?? { response: settingsFor().getCompactionSettings(ctx.model).reserveTokens };
         if (!Number.isSafeInteger(budget.response) || budget.response < 0)
             throw new Error("Invalid host request reserve");
-        const floor = host ? host.responseExact !== true : ctx.sessionManager.getBranch().some(entry => entry.type === "message" && entry.message.role === "assistant" &&
-            (entry.message.stopReason === "length" || (entry.message.stopReason === "error" && isContextOverflow(entry.message))));
+        const floor = host ? host.responseExact !== true : ctx.sessionManager.getBranch().some(entry => entry.type === "message" && needsFullReplyReserve(entry.message));
         return { response: floor ? Math.max(budget.response, ctx.model?.maxTokens ?? 0) : budget.response };
     };
     const inputRoom = (response, ctx) => {
@@ -345,7 +346,7 @@ export async function installCompaction(pi, options = {}) {
     // The planner counts with the same function, except that it does not fail on a summary in another format: a cut can replace one
     // (an operator's /compact repairs a session whose saved summary is foreign). Identical to the strict one for this compactor's summaries.
     const plannerMeasureFor = (ctx) => Object.assign((messages) => measurableConversationTokens(applyClips(messages, clips), ctx, options.recallPolicy), { messageTokens: (message) => conversationTokens(applyClips([message], clips)) });
-    const hostTokens = (ctx) => options.hostMessages ? conversationTokens(options.hostMessages(ctx)) : 0;
+    const hostMessages = (ctx) => options.hostMessages?.(ctx) ?? [];
     const toolShapes = (names) => {
         const selected = new Set(names);
         return pi.getAllTools().filter(tool => selected.has(tool.name)).map(tool => ({ name: tool.name, description: tool.description, parameters: tool.parameters }));
@@ -358,6 +359,53 @@ export async function installCompaction(pi, options = {}) {
         const future = saved.some(message => message.role === "system") ? historyToolDeclarations(saved, shapes, true) : saved;
         return systemPart(observedPrompt, shapes, future, guardPrompt);
     };
+    const clampMessage = "Pi's request clamp leaves less than the reply floor";
+    // Both the guard and the early projections ask Pi about the very same converted context. Pi's cap uses its own estimator,
+    // whereas our hard line is in corrected units; fitting that line alone need not leave the branch's reply floor.
+    const requestClamp = (messages, ctx) => {
+        const context = normalizeContext({ messages: convertToLlm(messages) });
+        const cap = piClamp?.(ctx.model, context, ctx.model.maxTokens);
+        return { cap, leavesFloor: cap === undefined || cap >= limits(ctx).replyFloor,
+            estimate: piMath ? piMath.estimateContextTokens(context).tokens : 0 };
+    };
+    const projectedRequest = (messages, ctx, host, incoming, prompt, tools, cut) => {
+        const enabled = cut || historyToolsEnabled(ctx), shapes = toolShapes(historyToolNames(pi, tools, enabled));
+        const saved = messages.some(message => message.role === "system") ? messages :
+            [{ role: "system", content: prompt, toolsAdded: shapes, timestamp: 0 }, ...messages];
+        const declared = enabled ? historyToolDeclarations(saved, shapes, true) : saved;
+        return applyClips([...host, ...(enabled ? projectedMessages(declared, ctx, options.recallPolicy) : declared), ...incoming], clips);
+    };
+    const clampAboveRoom = (decision, ctx, room, target, total, project, replan, deepest) => {
+        if (options.requestReserve || !piClamp)
+            return decision;
+        const lastAttempt = () => {
+            const previousClips = new Map(clips);
+            const candidate = deepest();
+            if (candidate.kind === "cut" && fits(total(candidate), limits(ctx).hardLine, ctx) && requestClamp(project(candidate), ctx).leavesFloor)
+                return candidate;
+            clips.clear();
+            for (const [id, clip] of previousClips)
+                clips.set(id, clip);
+            return { kind: "fault", capacity: true, message: clampMessage };
+        };
+        for (let retry = 0;; retry++) {
+            if (decision.kind !== "cut" && decision.kind !== "none")
+                return decision;
+            if (corrected(total(decision), ctx) <= room)
+                return decision;
+            const check = requestClamp(project(decision), ctx);
+            if (check.leavesFloor)
+                return decision;
+            if (retry === 3)
+                return lastAttempt();
+            // Shrink in the planner's units by Pi's measured shortfall, at most three times. Once we reach the original room,
+            // use its existing band/room policy; admission below the room is deliberately unchanged.
+            target = Math.max(room, Math.floor(target * (ctx.model.contextWindow - piMargin - limits(ctx).replyFloor) / check.estimate));
+            decision = replan(target);
+            if (decision.kind === "defer" || decision.kind === "fault")
+                return lastAttempt();
+        }
+    };
     // Pi reports a manual cut with nothing to do as a failed compaction; the thrown error lacks the event's "Compaction failed: " prefix.
     const isNoWork = (message) => ["Already compacted", "Nothing to compact (session too small)"].includes(message?.replace(/^Compaction failed: /, "") ?? "");
     // A missing model is configuration, not capacity.
@@ -433,7 +481,7 @@ export async function installCompaction(pi, options = {}) {
         phase = "decided";
         try {
             syncHistoryTools(ctx);
-            await preflight(ctx, incomingTokens(event.text, event.images), ctx.getSystemPrompt(), pi.getActiveTools());
+            await preflight(ctx, incomingTokens(event.text, event.images), [{ role: "user", content: [{ type: "text", text: event.text }, ...(event.images ?? [])], timestamp: 0 }], ctx.getSystemPrompt(), pi.getActiveTools());
             syncHistoryTools(ctx);
         }
         catch (error) {
@@ -535,7 +583,7 @@ export async function installCompaction(pi, options = {}) {
             // A cut Pi asks for (an operator's /compact, Pi's threshold or overflow recovery) charges what the request must carry whatever is
             // cut: the system part and the host input. A prompt's cut is pi-tuck's own and never comes here.
             const saved = ctx.sessionManager.buildSessionProjection().messages;
-            const host = hostTokens(ctx);
+            const hostInput = hostMessages(ctx), host = conversationTokens(hostInput);
             const system = requestSystemPart(ctx.getSystemPrompt(), pi.getActiveTools(), saved, ctx.getSystemPrompt());
             const mandatory = system + host, band = decisionBand(ctx, room, system);
             const before = plannerMeasureFor(ctx)(saved);
@@ -561,7 +609,8 @@ export async function installCompaction(pi, options = {}) {
                     // Every cut asked while no run is admitted (an operator's cut) is planned for a run that has not begun: the session has moved on
                     // from any run that ended. A retry is inside its run.
                     const draft = tuckDraft(tuck, event, ctx, reserve, plannerMeasureFor(ctx), options.recallPolicy, retain, operatorManual, operatorManual || !admittedRun, admittedRun ? protectedPrompt(ctx) : undefined);
-                    return draft.compaction ? { draft: true, after: draft.after, mode: draft.mode, compaction: draft.compaction, reserve }
+                    return draft.compaction ? { draft: true, after: draft.after, mode: draft.mode, compaction: draft.compaction, reserve,
+                        messages: previewCut(ctx, { type: "compaction", ...draft.compaction, tokensBefore: before }).buildSessionProjection().messages }
                         : { draft: false, capacityRejected: Boolean(draft.capacityRejected), reason: draft.unavailable };
                 },
             });
@@ -569,15 +618,14 @@ export async function installCompaction(pi, options = {}) {
             let decision = decide();
             const { hardLine } = limits(ctx);
             if (!operatorManual && hardLine > room && decision.kind === "fault") {
-                if (fits(before + mandatory, hardLine, ctx))
-                    return { cancel: true };
-                decision = decide(hardLine, hardLine);
+                decision = fits(before + mandatory, hardLine, ctx) ? { kind: "none" } : decide(hardLine, hardLine);
+                decision = clampAboveRoom(decision, ctx, room, hardLine, candidate => (candidate.kind === "cut" ? candidate.plan.after : before) + mandatory, candidate => projectedRequest(candidate.kind === "cut" ? candidate.plan.messages : saved, ctx, hostInput, [], ctx.getSystemPrompt(), pi.getActiveTools(), candidate.kind === "cut"), target => target <= room ? decide() : decide(target, target), () => deepestDecision(ctx, operatorManual || !admittedRun, retain));
             }
             if (decision.kind === "cut") {
                 return { compaction: captureDraft(decision.plan.compaction, ctx, decision.plan.reserve, event.reason, event.preparation.tokensBefore, retain, operatorManual, event.preparation.previousSummary, { response, host, mandatory, target: decision.target, margin: room - band }) };
             }
             // An opportunistic cut inside the band that cannot reach it is no fault: the request goes to the guard.
-            if (decision.kind === "none")
+            if (decision.kind === "none" || decision.kind === "defer")
                 return { cancel: true };
             recorder.stop("no-cut");
             ctx.ui.notify(decision.message, "warning");
@@ -609,21 +657,78 @@ export async function installCompaction(pi, options = {}) {
         syncHistoryTools(ctx); });
     pi.on("turn_start", (_event, ctx) => syncHistoryTools(ctx));
     pi.on("agent_settled", (_event, ctx) => syncHistoryTools(ctx));
+    // A reply that stopped at the output limit may be cut off, and Pi says nothing. Tell the operator once per run, on the operator's side only:
+    // the notice never enters the model's context. Decided at the end of the run, so a later reply in the same run (a retry or a continuation)
+    // that ends normally clears it.
+    let lengthStop = false;
+    pi.on("agent_start", () => { lengthStop = false; });
+    pi.on("message_end", event => { if (event.message.role === "assistant")
+        lengthStop = event.message.stopReason === "length"; });
+    pi.on("agent_settled", (_event, ctx) => {
+        if (!lengthStop || !active())
+            return;
+        lengthStop = false;
+        const message = "pi-tuck: the last reply stopped at the model's output limit and may be cut off. Ask the agent to continue.";
+        try {
+            if (ctx.hasUI === false)
+                process.stderr.write(message + "\n");
+            else
+                ctx.ui.notify(message, "warning");
+        }
+        catch { /* ended */ } // with no UI (pi -p), stderr is the operator's
+    });
     // The session as it will be once a cut that is not saved yet is saved, and its conversation as the next request carries it.
-    const cutSession = (ctx, cut) => {
+    const previewCut = (ctx, cut) => {
         const header = ctx.sessionManager.getHeader();
         if (!header)
             throw new Error("Missing native session header for the prompt's cut");
         const preview = SessionManager.inMemory(ctx.cwd, undefined, [header, ...ctx.sessionManager.getBranch()]);
-        preview.appendCompaction(cut.summary, keptFrom(cut, ctx), 0, cut.details, true);
+        preview.appendCompaction(cut.summary, cut.firstKeptEntryId, 0, cut.details, true);
         return preview;
     };
+    const cutSession = (ctx, cut) => previewCut(ctx, { ...cut, firstKeptEntryId: keptFrom(cut, ctx) });
     const withCut = (ctx, cut) => cutSession(ctx, cut).buildSessionProjection().messages;
+    // The shared final planner: preserve the compiler's prompt/protocol floor and spend nothing on optional history. Fresh text results
+    // get the smallest existing head/tail + recall-locator representation; their originals remain in the session, as for ordinary clips.
+    const deepestDecision = (ctx, newPrompt, retain) => {
+        const saved = ctx.sessionManager.buildSessionProjection().messages;
+        if (!newPrompt) {
+            const at = saved.findLastIndex(message => message.role === "assistant" && !isDiscardedResponse(message));
+            const response = saved[at];
+            const calls = new Set(response?.role === "assistant" ? response.content.flatMap(block => block.type === "toolCall" ? [block.id] : []) : []);
+            const results = saved.slice(at + 1).filter((message) => message.role === "toolResult" && calls.has(message.toolCallId));
+            const entryIdOf = (id) => ctx.sessionManager.getBranch().findLast(entry => entry.type === "message" && entry.message.role === "toolResult" && entry.message.toolCallId === id)?.id;
+            for (const result of results) {
+                let low = 0, high = conversationTokens([result]) - 1;
+                if (!clipFor([result], entryIdOf, high).length)
+                    continue;
+                while (low < high) {
+                    const middle = Math.floor((low + high) / 2);
+                    if (clipFor([result], entryIdOf, middle).length)
+                        high = middle;
+                    else
+                        low = middle + 1;
+                }
+                for (const clip of clipFor([result], entryIdOf, low))
+                    clips.set(clip.toolCallId, clip);
+            }
+        }
+        const before = measureFor(ctx)(saved);
+        const draft = tuckObservedDraft(tuck, { branchEntries: ctx.sessionManager.getBranch(), tokensBefore: before,
+            previousSummary: saved.find(message => message.role === "compactionSummary")?.summary, newPrompt, reRenderSummary: true,
+            protectedPromptEntryId: newPrompt ? undefined : protectedPrompt(ctx), deepest: true }, ctx, 0, plannerMeasureFor(ctx), options.recallPolicy, retain);
+        if (!draft.compaction)
+            return { kind: "fault", capacity: Boolean(draft.capacityRejected), message: draft.unavailable };
+        const messages = previewCut(ctx, { type: "compaction", ...draft.compaction }).buildSessionProjection().messages;
+        assertNativeToolRounds(messages);
+        return { kind: "cut", target: "room", plan: { draft: true, after: measureFor(ctx)(messages), mode: draft.mode,
+                compaction: draft.compaction, reserve: 0, messages } };
+    };
     // The cut before a run, decided on `input` while the session is idle; before_agent_start decides only for a run no input started (one a
     // custom message starts). The cut is pi-tuck's own, planned as at a tool boundary. It is not Pi's: Pi's cut finder keeps the last
     // keepRecentTokens, and where those are the whole conversation it has nothing to cut, so every later prompt was refused for good (t06 at
     // Pi's defaults on a 32k window). Each request of the run carries the cut, and the run's first boundary saves it.
-    const preflight = (ctx, incomingPart, observedPrompt, tools, hostSnapshot) => {
+    const preflight = (ctx, incomingPart, incomingMessages, observedPrompt, tools, hostSnapshot) => {
         promptCut = undefined;
         promptCutAt = ctx.sessionManager.getBranch().length;
         if (!active())
@@ -638,7 +743,7 @@ export async function installCompaction(pi, options = {}) {
         const saved = ctx.sessionManager.buildSessionProjection().messages;
         const system = requestSystemPart(observedPrompt, tools, saved, ctx.getSystemPrompt());
         const band = decisionBand(ctx, room, system);
-        const host = hostSnapshot ?? hostTokens(ctx), measure = measureFor(ctx), before = measure(saved);
+        const hostInput = hostSnapshot ?? hostMessages(ctx), host = conversationTokens(hostInput), measure = measureFor(ctx), before = measure(saved);
         const incoming = incomingPart + system, mandatory = incoming + host;
         observeBudget(before + mandatory, ctx);
         estimatedTokens = corrected(before + mandatory, ctx);
@@ -662,15 +767,15 @@ export async function installCompaction(pi, options = {}) {
                     return { draft: false, capacityRejected: Boolean(draft.capacityRejected), reason: draft.unavailable };
                 const messages = withCut(ctx, { type: "compaction", ...draft.compaction, tokensBefore: before });
                 assertNativeToolRounds(messages);
-                return { draft: true, after: measure(messages), mode: draft.mode, compaction: draft.compaction, reserve };
+                return { draft: true, after: measure(messages), mode: draft.mode, compaction: draft.compaction, reserve, messages };
             },
         });
         let decision = gate.kind === "plan" ? decide(room, band, gate.opportunistic) : { kind: "defer" };
         const { hardLine } = limits(ctx);
         if (hardLine > room && (decision.kind === "defer" || decision.kind === "fault")) {
-            if (fits(before + mandatory, hardLine, ctx))
-                return;
-            decision = fits(mandatory, hardLine, ctx) ? decide(hardLine, hardLine, false) : { kind: "defer" };
+            decision = fits(before + mandatory, hardLine, ctx) ? { kind: "none" } :
+                fits(mandatory, hardLine, ctx) ? decide(hardLine, hardLine, false) : { kind: "defer" };
+            decision = clampAboveRoom(decision, ctx, room, hardLine, candidate => (candidate.kind === "cut" ? candidate.plan.after : before) + mandatory, candidate => projectedRequest(candidate.kind === "cut" ? candidate.plan.messages : saved, ctx, hostInput, incomingMessages, observedPrompt, tools, candidate.kind === "cut"), target => decide(target, target <= room ? band : target, false), () => deepestDecision(ctx, true, false));
         }
         if (decision.kind === "defer")
             return;
@@ -693,7 +798,8 @@ export async function installCompaction(pi, options = {}) {
             if (prompted)
                 access(); // decided on input; a failure latched there still stops the run
             else
-                await preflight(ctx, incomingEventTokens(event), event.systemPrompt, historyToolNames(pi, event.systemPromptOptions.selectedTools, historyToolsEnabled(ctx)));
+                await preflight(ctx, incomingEventTokens(event), [{ role: "user", timestamp: 0,
+                        content: event.messageType === "custom" ? event.message.content : [{ type: "text", text: event.prompt }, ...(event.images ?? [])] }], event.systemPrompt, historyToolNames(pi, event.systemPromptOptions.selectedTools, historyToolsEnabled(ctx)));
             // Pi declares tool changes before turn_start, then snapshots executable tools in prepareRequest. Introduce our tools in
             // the request projection: recording an activation message here would shift every later session-global recall index.
             if (historyToolsEnabled(ctx))
@@ -720,11 +826,11 @@ export async function installCompaction(pi, options = {}) {
             const message = event.message;
             const incoming = incomingTokens(typeof message.content === "string" ? message.content : message.content.flatMap(b => b.type === "text" ? [b.text] : []).join("\n"), typeof message.content === "string" ? [] : message.content.filter((b) => b.type === "image"));
             const saved = ctx.sessionManager.buildSessionProjection().messages;
-            const host = hostTokens(ctx);
+            const hostInput = hostMessages(ctx), host = conversationTokens(hostInput);
             const system = requestSystemPart(ctx.getSystemPrompt(), pi.getActiveTools(), saved, ctx.getSystemPrompt());
             if (fits(conversationTokens(saved) + system + incoming + host, limits(ctx).room, ctx))
                 return;
-            preflight(ctx, incoming, ctx.getSystemPrompt(), pi.getActiveTools(), host);
+            preflight(ctx, incoming, [message], ctx.getSystemPrompt(), pi.getActiveTools(), hostInput);
             syncHistoryTools(ctx);
         }
         catch (error) {
@@ -750,7 +856,7 @@ export async function installCompaction(pi, options = {}) {
         // request while tools still run, so it can be counted early: conservative, and nothing else would cut at that later boundary.
         const system = requestSystemPart(ctx.getSystemPrompt(), pi.getActiveTools(), saved, ctx.getSystemPrompt());
         const band = decisionBand(ctx, room, system);
-        const host = hostTokens(ctx);
+        const hostInput = hostMessages(ctx), host = conversationTokens(hostInput);
         const queued = event.context.pendingMessages.length ? conversationTokens(event.context.pendingMessages) : 0;
         const mandatory = system + host + queued;
         // Without usable provider usage, Pi's own threshold uses its unwrapped message estimate as the compiler's density input.
@@ -777,12 +883,13 @@ export async function installCompaction(pi, options = {}) {
         let floorKept;
         let wholePromptFloor;
         const protectedId = reason === "tool-loop" ? protectedPrompt(ctx) : undefined;
-        const decide = (targetRoom = room, targetBand = band) => {
+        const decide = (targetRoom = room, targetBand = band, required = false) => {
             previews = [];
             floorKept = undefined;
             wholePromptFloor = undefined;
             const before = measure(saved);
-            const gate = gateCut({ total: before + mandatory, mandatory, room: targetRoom, band: targetBand, fits: (tokens, limit) => fits(tokens, limit, ctx) });
+            const gate = required && fits(mandatory, targetRoom, ctx) ? { kind: "plan", opportunistic: false } :
+                gateCut({ total: before + mandatory, mandatory, room: targetRoom, band: targetBand, fits: (tokens, limit) => fits(tokens, limit, ctx) });
             if (gate.kind !== "plan")
                 return gate;
             return chooseCut({
@@ -811,7 +918,7 @@ export async function installCompaction(pi, options = {}) {
                     const messages = preview.buildSessionProjection().messages;
                     assertNativeToolRounds(messages);
                     previews.push(messages);
-                    return { draft: true, after: measure(messages), mode: draft.mode, compaction: draft.compaction, reserve };
+                    return { draft: true, after: measure(messages), mode: draft.mode, compaction: draft.compaction, reserve, messages };
                 },
             });
         };
@@ -821,7 +928,7 @@ export async function installCompaction(pi, options = {}) {
         // clipBudgets states the budgets tried and why they are counted in the estimate's units; `kept` is measured from the previews this decision
         // built, so the second budget is the cut's own floor and not a new formula. When nothing is clipped or the decision still faults, the
         // refusal stands.
-        const tryClips = (clipRoom, targetBand) => {
+        const tryClips = (clipRoom, targetBand, required = false) => {
             if (decision.kind !== "fault" || !event.toolResults)
                 return;
             const branch = ctx.sessionManager.getBranch();
@@ -834,7 +941,7 @@ export async function installCompaction(pi, options = {}) {
                     continue;
                 for (const clip of round)
                     clips.set(clip.toolCallId, clip);
-                const clippedDecision = decide(clipRoom, targetBand);
+                const clippedDecision = decide(clipRoom, targetBand, required);
                 if (clippedDecision.kind === "fault")
                     for (const clip of round)
                         clips.delete(clip.toolCallId);
@@ -851,9 +958,17 @@ export async function installCompaction(pi, options = {}) {
         const { hardLine } = limits(ctx);
         if (hardLine > room && (decision.kind === "defer" || decision.kind === "fault")) {
             if (fits(before + mandatory, hardLine, ctx))
-                return;
-            decision = decide(hardLine, hardLine);
-            tryClips(hardLine, hardLine);
+                decision = { kind: "none" };
+            else {
+                decision = decide(hardLine, hardLine);
+                tryClips(hardLine, hardLine);
+            }
+            decision = clampAboveRoom(decision, ctx, room, hardLine, candidate => (candidate.kind === "cut" ? candidate.plan.after : before) + mandatory, candidate => projectedRequest(candidate.kind === "cut" ? candidate.plan.messages : saved, ctx, hostInput, event.context.pendingMessages, ctx.getSystemPrompt(), pi.getActiveTools(), candidate.kind === "cut"), target => {
+                const targetBand = target <= room ? band : target;
+                decision = decide(target, targetBand, target > room);
+                tryClips(target, targetBand, target > room);
+                return decision;
+            }, () => deepestDecision(ctx, false, true));
         }
         if (decision.kind === "fault" && reason === "tool-loop") {
             if (wholePromptFloor !== undefined && !fits(wholePromptFloor, hardLine, ctx))
@@ -880,10 +995,71 @@ export async function installCompaction(pi, options = {}) {
         setImmediate(() => { semantic.indexAtCut(semanticProvider(ctx)); });
         return { entries: [...event.entries, saved, ...(more?.entries ?? [])] };
     };
+    const retryThinking = (event, ctx) => {
+        // Pi's supported boundary drafts commit before finishTurn honors `continue`. Omitting the selected failed response also prevents
+        // Pi's later native length recovery from retrying it independently. The stored message and its output usage stay on the branch.
+        const entries = [...(atBoundary(event, ctx, "run-end", false)?.entries ?? event.entries),
+            { type: "context_edit", targetId: event.messageEntryId, replacement: null }];
+        if (thinkingRetryPending)
+            return { entries, continue: false };
+        const header = ctx.sessionManager.getHeader();
+        if (!header)
+            throw new Error("Missing native session header for the thinking-only retry");
+        const preview = SessionManager.inMemory(ctx.cwd, undefined, [header, ...ctx.sessionManager.getBranch()]);
+        for (const entry of entries) {
+            if (entry.type === "context_edit")
+                preview.appendContextEdit(entry.targetId, entry.replacement);
+            else if (entry.type === "compaction")
+                preview.appendCompaction(entry.summary, entry.firstKeptEntryId, 0, entry.details, true, entry.usage);
+            else if (entry.type === "custom")
+                preview.appendCustomEntry(entry.customType, entry.data);
+            else
+                preview.appendCustomMessageEntry(entry.customType, entry.content, entry.display, entry.details);
+        }
+        const view = Object.create(ctx, { sessionManager: { value: preview } });
+        const hostInput = hostMessages(ctx), host = conversationTokens(hostInput);
+        // The deepest cut only when it buys reply room: where Pi's cap on the request as it stands (the stopped reply left out) is already
+        // as large as the deepest cut's, the turn is retried uncut, and no history is given up for nothing.
+        const asIs = projectedRequest(preview.buildSessionProjection().messages, view, hostInput, event.context.pendingMessages, ctx.getSystemPrompt(), pi.getActiveTools(), false);
+        const asIsCheck = requestClamp(asIs, view);
+        const previousClips = new Map(clips);
+        const decision = deepestDecision(view, false, true);
+        if (decision.kind === "cut" && asIsCheck.cap !== undefined && asIsCheck.leavesFloor &&
+            requestClamp(projectedRequest(decision.plan.messages, view, hostInput, event.context.pendingMessages, ctx.getSystemPrompt(), pi.getActiveTools(), true), view).cap <= asIsCheck.cap) {
+            clips.clear();
+            for (const [id, clip] of previousClips)
+                clips.set(id, clip);
+            const { room, hardLine } = limits(view);
+            thinkingRetryPending = true;
+            return { entries: [...entries, { type: "custom", customType: "pi-tuck-capacity", data: { kind: "retry", stoppedReplyEntryId: event.messageEntryId,
+                            stoppedOutputTokens: event.message.role === "assistant" ? event.message.usage?.output ?? 0 : 0, cut: false,
+                            piEstimateTokens: asIsCheck.estimate, piCap: asIsCheck.cap, room, hardLine, buildId: BUILD_ID } }], continue: true };
+        }
+        if (decision.kind !== "cut")
+            throw new CapacityError(clampMessage);
+        const messages = projectedRequest(decision.plan.messages, view, hostInput, event.context.pendingMessages, ctx.getSystemPrompt(), pi.getActiveTools(), true);
+        if (runPrompt !== undefined && !carriesPrompt(messages, runPrompt))
+            throw promptRefusal();
+        const system = requestSystemPart(ctx.getSystemPrompt(), pi.getActiveTools(), messages, ctx.getSystemPrompt());
+        const tokens = system + conversationTokens(messages), check = requestClamp(messages, view);
+        const { response, room, hardLine } = limits(view);
+        if (!fits(tokens, hardLine, view) || check.cap === undefined || !check.leavesFloor)
+            throw new CapacityError(clampMessage);
+        const compaction = captureDraft(decision.plan.compaction, view, decision.plan.reserve, "length-retry", tokens, true, false, undefined, { response, host, mandatory: system + host + conversationTokens(event.context.pendingMessages), target: decision.target, margin: room - decisionBand(view, room, system) });
+        thinkingRetryPending = true;
+        pendingBoundaryCommit = true;
+        return { entries: [...entries, { type: "compaction", summary: compaction.summary, firstKeptEntryId: compaction.firstKeptEntryId || null, details: compaction.details },
+                { type: "custom", customType: "pi-tuck-capacity", data: { kind: "retry", stoppedReplyEntryId: event.messageEntryId,
+                        stoppedOutputTokens: event.message.role === "assistant" ? event.message.usage?.output ?? 0 : 0, cut: true,
+                        estimatedTokens: corrected(tokens, view), piEstimateTokens: check.estimate, piCap: check.cap, room, hardLine, buildId: BUILD_ID } }], continue: true };
+    };
     pi.on("turn_end", (event, ctx) => {
         try {
             if (!active() || !admittedRun)
                 return;
+            if (thinkingOnlyLength(event.message) && !failure && !ctx.signal?.aborted && settingsFor().getCompactionSettings(ctx.model).enabled)
+                return retryThinking(event, ctx);
+            thinkingRetryPending = false;
             const decide = event.outcome === "completed" && !ctx.signal?.aborted && event.context.canContinue && event.toolResults.length > 0 &&
                 settingsFor().getCompactionSettings(ctx.model).enabled;
             if (!promptCut && !decide)
@@ -996,16 +1172,15 @@ export async function installCompaction(pi, options = {}) {
             const tokens = system + conversationTokens(messages);
             observeBudget(tokens, ctx);
             estimatedTokens = corrected(tokens, ctx);
-            const { room, hardLine, replyFloor } = limits(ctx);
+            const { room, hardLine } = limits(ctx);
             if (runPrompt !== undefined && !carriesPrompt(messages, runPrompt))
                 throw new Refusal("Prompt check failed: the request is missing the whole text of the prompt that started this run. Shorten the prompt or choose a model with a larger context window, then retry");
             if (!fits(tokens, hardLine, ctx))
                 throw new CapacityError("Complete Pi context and response reserve exceed the estimated request budget");
             // Pi's request estimator and its compaction estimator need not agree. The hard line is
             // necessary; above the room, also require the real request clamp to preserve the floor.
-            if (!options.requestReserve && estimatedTokens > room && piClamp &&
-                piClamp(ctx.model, normalizeContext({ messages: convertToLlm(messages) }), ctx.model.maxTokens) < replyFloor)
-                throw new CapacityError("Pi's request clamp leaves less than the reply floor");
+            if (!options.requestReserve && estimatedTokens > room && piClamp && !requestClamp(messages, ctx).leavesFloor)
+                throw new CapacityError(clampMessage);
             if (estimatedTokens > room)
                 receipt("above-room", "Request exceeds the room and fits the hard line", ctx);
             if (recorder.enabled && !recorder.waitingForCut)

package/dist/src/diagnostics.d.ts

@@ -1,6 +1,6 @@
 import { type RecallPolicy } from './recall.js';
 import type { SessionEntry } from '@earendil-works/pi-coding-agent';
-export declare const DIAGNOSTIC_BUILD = "c3eefa3bb76e52635b1e8394a025d3ffd6a056fc5636ed9b35f89dfddc1a322a";
+export declare const DIAGNOSTIC_BUILD = "e95edde95c614d01e1cbf6489134676fac0c1bbf6aa57ef639ffac1e7e105041";
 export interface Receipt {
     id: string;
     build: string;

package/dist/src/request-capacity.d.ts

@@ -1,14 +1,31 @@
 import type { SessionEntry } from '@earendil-works/pi-coding-agent';
-import type { Messages } from './budget.js';
+import type { AgentMessage, Messages } from './budget.js';
+type SimpleOptions = typeof import('@earendil-works/pi-ai/api/simple-options');
+type Estimate = typeof import('@earendil-works/pi-ai/utils/estimate');
+export type PiRequestMath = {
+    clampMaxTokensToContext: SimpleOptions['clampMaxTokensToContext'];
+    estimateContextTokens: Estimate['estimateContextTokens'];
+};
+/** Pi's own request arithmetic: its reply clamp and the estimator behind it, as the running Pi has them.
+ * An installed extension cannot import these subpaths by name: Pi's extension loader aliases only pi-ai's root, compat, models, oauth and
+ * providers/all, and an installed package has no node_modules of its own. So resolve them first by name (tests, the eval harness, a dev
+ * checkout), then from the running Pi's own install, beside its executable. If neither works (an SDK host), there is none, and callers
+ * keep their fallbacks. */
+export declare function loadPiRequestMath(): Promise<PiRequestMath | undefined>;
 /** Read Pi's private safety allowance through its public clamp, rather than copying it.
- * Older/unavailable subpaths fall back to Pi 1.2–1.5's documented 4,096-token allowance.
+ * Without Pi's request arithmetic, fall back to Pi 1.2–1.5's documented 4,096-token allowance.
  */
-export declare function piContextSafetyMargin(): Promise<number>;
+export declare function piContextSafetyMargin(math?: PiRequestMath): Promise<number>;
 /** The smallest reply room a request above the room must leave after Pi's own clamp: above the user's p99 reply (3,270 tokens). */
 export declare const REPLY_FLOOR_MIN = 3584;
+/** A spent thinking budget that produced neither a visible answer nor work. Empty/text/tool replies keep the existing length recovery. */
+export declare const thinkingOnlyLength: (message: AgentMessage) => boolean;
+/** Thinking-only exhaustion uses the observed largest reply, not the full reserve. Every other failed length/overflow keeps its override. */
+export declare const needsFullReplyReserve: (message: AgentMessage) => boolean;
 /** Read the whole current branch, including replies before cuts. No session-local fallback flag. */
 export declare function branchReplyFloor(branch: readonly SessionEntry[], reserve: number): number;
 /** Admission never promises more output than a model with a positive reply cap can produce. */
 export declare function requestReplyFloor(branch: readonly SessionEntry[], reserve: number, maxTokens?: number): number;
 /** Check decoded text, including a whole verbatim user quote in our summary; never JSON-escaped substring matching. */
 export declare function carriesPrompt(messages: Messages, prompt: string): boolean;
+export {};