pi-codex-compaction 0.1.1 → 0.1.3

package/CHANGELOG.md

@@ -6,6 +6,25 @@ This project follows the spirit of [Keep a Changelog](https://keepachangelog.com
 
 ## [Unreleased]
 
+## [0.1.3] - 2026-09-11
+
+### Fixed
+
+- Estimate compaction tokens separately from request bytes so long Codex sessions do not fall back merely because UTF-8 bytes exceed the token budget.
+- Respect request-transform field removals when sizing, restrict warnings to TUI mode, and distinguish preparation failures from transport failures.
+- Preserve the independent 16 MiB request ceiling, count the complete transformed envelope, and reject unavailable context limits.
+- Record safe fallback reasons and numeric size diagnostics in local custom session entries; notify when UI is available without exposing request or provider content.
+
+## [0.1.2] - 2026-09-11
+
+### Fixed
+
+- Preserve pi-codex-tools grammar metadata and custom-tool history during remote compaction.
+- Honor pi-fast on direct compaction requests through a public event-bus contract.
+- Include cache writes and standard Pi compaction usage in session totals.
+- Restrict endpoint paths and ports; merge beta features and honor null auth headers.
+- Avoid I/O after pre-cancellation, bound total request time, and close completed SSE streams promptly.
+
 ## [0.1.1] - 2026-08-06
 
 ### Fixed

package/CONTRIBUTING.md

@@ -29,6 +29,33 @@ Before opening a pull request:
 - Preserve current-model compaction, context bounds, cancellation, HTTPS, and standard Pi fallback behavior.
 - Add a regression test when changing wire parsing, request construction, checkpoint rehydration, or model-switch fallback behavior.
 
+## Token-budget regression and smoke checks
+
+`tests/integration.test.mjs` uses the real Pi compaction hook, serializer, and
+session tree with fake credentials and mocked Responses transport. A synthetic
+long transcript must produce a remote checkpoint even when its UTF-8 bytes
+exceed the numeric token budget. Oversized input must still invoke the standard
+compactor and record a safe fallback reason. A later model request must not
+contain the diagnostic entry. No private session fixtures or live requests are
+needed for these tests.
+
+The token conversion follows Codex's ordinary JSON-item heuristic:
+[byte-to-token estimate](https://github.com/openai/codex/blob/654b0a77d0d2f81aa21f61caf7af4be88fe550bb/codex-rs/utils/string/src/truncate.rs)
+and [history sizing](https://github.com/openai/codex/blob/654b0a77d0d2f81aa21f61caf7af4be88fe550bb/codex-rs/core/src/context_manager/history.rs).
+This package keeps all serialized opaque/image bytes in its estimate instead of
+copying Codex's modality-specific discounts. Never equate bytes and tokens or
+remove the independent wire-size ceiling.
+
+For an offline replay, reconstruct a compaction's discarded history in memory,
+use fake authentication, replace `fetch` with a fixture checkpoint response,
+and call `createRemoteCompaction`. Print only sizes and the success/fallback
+code. Do not save the transcript, request, auth headers, or opaque content.
+
+For an approved live smoke test, follow the procedure in [README.md](./README.md#reference-and-smoke-test).
+Check for `fromHook: true` and `details.kind: "pi-codex-compaction"` on success.
+On fallback, inspect only the reason and counters in the custom diagnostic
+entry. Never inspect or print `encryptedContent`.
+
 ## Code of conduct
 
 This project follows the Contributor Covenant Code of Conduct.

package/README.md

@@ -9,11 +9,11 @@ Keep long Pi sessions usable on OpenAI Codex models by replacing Pi's local summ
 - Retains the normal Codex Responses request envelope, including system instructions, active tool schemas, reasoning settings, prompt-cache fields, and routing fields.
 - Persists Codex's opaque encrypted checkpoint and rehydrates it only for supported Codex requests.
 - Reuses checkpoints only for the same model, trusted endpoint, Codex account, and authentication mode.
-- Bounds input with a UTF-8-aware token estimate, trims tool output when necessary, retries transient failures, and honors cancellation.
+- Bounds input with a Codex-style UTF-8 token estimate and a separate hard byte limit, trims tool output when necessary, retries transient failures, and honors cancellation.
 - Falls back to standard Pi compaction on failure or when custom compaction instructions are requested.
 - Keeps a bounded readable transcript excerpt so switching models or providers remains usable.
 
-The current Codex catalog includes models such as `gpt-5.6-sol`, `gpt-5.6-terra`, and `gpt-5.6-luna`. Capability detection follows the provider/API contract (`openai-codex` + `openai-codex-responses`) rather than a brittle model-name list.
+The current Codex catalog includes `gpt-6-astra`, `gpt-5.6-sol`, `gpt-5.6-terra`, and `gpt-5.6-luna`. Capability detection follows the provider/API contract (`openai-codex` + `openai-codex-responses`) rather than a brittle model-name list. Use Pi 0.85.1 or later.
 
 ## Installation
 
@@ -39,10 +39,84 @@ When Pi starts compaction on a supported Codex model, the extension sends a stre
 
 Compaction uses the model active when Pi triggers it. If a session switches from a larger to a smaller model, the remote request is bounded against the new model's context window and tool outputs are reduced before sending. A previous opaque checkpoint is treated as incompatible after a model, endpoint, account, or authentication-mode switch; Pi's readable previous summary is sent instead. If the full request still cannot fit, the extension leaves compaction to Pi's normal implementation.
 
-If the remote request fails, is cancelled, or returns an unexpected response, Pi's standard compaction path runs. Custom compaction instructions also use Pi's standard path because RemoteCompactionV2 has no documented custom-instructions field. The direct checkpoint request is restricted to `https://chatgpt.com`, rejects redirects, limits request/response size, and never decodes or logs `encrypted_content`. No configuration is required.
+If the remote request fails or returns an unexpected response, Pi's standard compaction path runs. Cancellation remains cancelled. Custom compaction instructions also use Pi's standard path because RemoteCompactionV2 has no documented custom-instructions field. The direct checkpoint request is restricted to `https://chatgpt.com`, rejects redirects, limits request/response size, and never decodes or logs `encrypted_content`. No configuration is required.
+
+### Size limits
+
+Input is estimated as `ceil(UTF-8 request bytes / 4)`, with 8,192 tokens reserved
+from the active model's context window. This uses Codex's ordinary-item heuristic,
+not an exact tokenizer. The complete transformed request is counted, including
+system instructions, tool definitions, and routing fields. Opaque checkpoints
+and image data remain counted at their serialized size; they are not decoded or
+discounted. Non-ASCII text uses UTF-8 bytes, not JavaScript string length.
+
+The uncompressed request also has an independent **16 MiB hard limit**. Tool
+outputs are reduced only when one of these limits is exceeded. User messages,
+tool calls, and opaque checkpoints are not removed. If the remaining request
+still cannot fit, or the model's context limit is unknown, standard Pi compaction
+runs. The estimate can differ from the server's token count; a server rejection
+still uses the existing fallback.
+
+### Fallback diagnostics
+
+A fallback on a supported model records a local custom session entry with type
+`pi-codex-compaction:fallback:v1`. It contains `version: 1` and a reason:
+`custom-instructions`, `auth-unavailable`, `request-unavailable`,
+`context-window-unavailable`, `context-limit`, `request-size-limit`, or
+`remote-failed`. Size failures also include estimated tokens, token budget,
+request bytes, byte limit, and the number of tool outputs reduced.
+Unexpected preparation failures use `request-unavailable`; `remote-failed`
+is reserved for failures from the transport call.
+
+No prompt, tool content, encrypted checkpoint, account identifier, credential, or
+raw provider error is included. These entries are not sent to the model. Pi
+shows a warning only in TUI mode when notifications are available; print, JSON,
+and RPC modes get no extra notifications or console output. Unsupported models
+and cancelled attempts do not create fallback diagnostics. Diagnostic storage
+or notification failure does not stop the standard compactor.
 
 ## Development
 
+### Other Codex extensions
+
+With `pi-fast` installed, direct compaction requests use the current Fast toggle.
+With `pi-codex-tools` installed, `apply_patch` keeps its raw grammar definition,
+custom-tool calls, and custom-tool results during compaction. Neither package is
+required. No package reads a private Pi tool registry.
+With `pi-openai-reasoning` installed, verified Astra requests keep the original
+request effort and receive the current effort as a configuration update.
+Failed compaction does not change saved reasoning state.
+
+Pi 0.85.1 does not expose grammar metadata in `getAllTools()`. Two synchronous,
+versioned `pi.events` contracts let cooperating extensions supply it:
+
+- `pi-codex-compaction:tools:v1`: `{ model, tools }`, before provider serialization.
+  A tool owner can attach its own `constrainedSampling` metadata.
+- `pi-codex-compaction:request:v1`: `{ ctx, messages, payload }`, after input
+  assembly and before size checks. A listener can replace `payload`. This event
+  is not the general `before_provider_request` chain and does not carry auth.
+  Size checks use the transformed envelope, including field removals.
+
+Other extensions' private request changes are not applied automatically.
+Unknown third-party grammar metadata needs cooperation through the tools event.
+The checkpoint entry includes standard Pi `usage`, including cache reads and
+cache writes, as well as the original bounded token counters in `details`.
+Costs follow Pi's catalog estimates. They are not a ChatGPT subscription bill.
+
+### Reference and smoke test
+
+Behavior was checked against `openai/codex` commit
+`654b0a77d0d2f81aa21f61caf7af4be88fe550bb` (2026-09-11), notably
+`core/src/compact_remote_v2{,_attempt}.rs`. No Codex code was copied.
+RemoteCompactionV2 is a changing Codex protocol, not the public `/responses/compact`
+API. Async tools and mid-turn steering require upstream Pi support.
+
+For a small live test, load this package and select `openai-codex/gpt-6-astra`.
+Send two short messages, run `/compact`, then ask about the first message.
+Repeat with `/fast on` and `pi-codex-tools` loaded. Check that compaction succeeds,
+the continuation retains context, and session usage includes compaction tokens.
+Use only a temporary file if you test `apply_patch`. Do not generate images.
+
 ```bash
 npm install
 npm run -w packages/pi-codex-compaction check

package/SECURITY.md

@@ -25,4 +25,28 @@ The extension bounds request, compaction input, and response size, validates the
 
 The extension never logs prompts, conversation contents, credentials, authorization headers, or raw provider responses. Install/update telemetry is best-effort and sends only package/version/runtime metadata; it can be disabled with `PI_OFFLINE=1`, `PI_TELEMETRY=0`, or Pi's `enableInstallTelemetry: false` setting.
 
+Direct requests allow only the official `/backend-api/codex/responses` endpoint
+on the default HTTPS port, without query strings or fragments. Total request
+time is limited to five minutes, including retries. Completed streams are closed
+without waiting for a server disconnect. A pre-aborted request does no network I/O.
+Null auth headers remove matching model headers; beta features are merged.
+
+Cooperating local extensions can inspect and transform compaction inputs through
+the documented event bus before size checks. These events contain no credentials.
+They have the same trust level as other installed Pi extensions.
+
+Context sizing uses `ceil(UTF-8 serialized request bytes / 4)` with an 8,192-token
+reserve from the active model's context window. It is an estimate, not a strict
+tokenizer bound. The complete transformed envelope is counted, including opaque
+content at its serialized size. A separate 16 MiB uncompressed request limit is
+enforced before network I/O. The HTTPS, redirect, response-size, checkpoint
+compatibility, and cancellation checks remain independent of token estimation.
+
+Fallback diagnostics store only a fixed reason code and finite non-negative
+size counters in `pi-codex-compaction:fallback:v1` custom session entries.
+These local records never include credentials, account/model identifiers,
+request content, encrypted checkpoints, or raw errors, and do not enter model
+context. No external diagnostic telemetry is added. They use the existing
+session's permissions and retention policy.
+
 See [CONTRIBUTING.md](./CONTRIBUTING.md) for development and validation instructions.

package/extensions/index.ts

@@ -7,6 +7,8 @@ import { BETA_FEATURE, getCodexAccountFingerprint } from "../src/codex-wire.js";
 import { reportInstallTelemetry } from "../src/install-telemetry.js";
 import {
   applyRemoteCompactionMarker,
+  COMPACTION_FALLBACK_ENTRY,
+  type CompactionFallback,
   createRemoteCompaction,
   findActiveRemoteCompaction,
   getCodexAuthKind,
@@ -14,24 +16,69 @@ import {
   supportsRemoteCompaction,
 } from "../src/remote-compaction.js";
 
+const FALLBACK_MESSAGES: Record<CompactionFallback["reason"], string> = {
+  "custom-instructions": "custom compaction instructions require the standard compactor",
+  "auth-unavailable": "Codex authentication is unavailable",
+  "request-unavailable": "the Codex request could not be prepared",
+  "context-window-unavailable": "the active model's context limit is unavailable",
+  "context-limit": "the estimated input exceeds the active model's token budget",
+  "request-size-limit": "the request exceeds the 16 MiB byte limit",
+  "remote-failed": "the remote request failed",
+};
+
 export default function piCodexCompaction(pi: ExtensionAPI): void {
   reportInstallTelemetry();
 
   const onBeforeCompact: ExtensionHandler<SessionBeforeCompactEvent, { compaction?: NonNullable<Awaited<ReturnType<typeof createRemoteCompaction>>> }> = async (event, ctx) => {
     if (!supportsRemoteCompaction(ctx.model)) return undefined;
 
+    let reported = false;
+    const reportFallback = (diagnostic: CompactionFallback) => {
+      if (reported || event.signal.aborted) return;
+      reported = true;
+      // No text, model/account identifiers, headers, or provider errors belong
+      // in diagnostics. Custom entries do not enter the model's context.
+      const data: Record<string, unknown> = { version: 1, reason: diagnostic.reason };
+      for (const key of ["estimatedTokens", "tokenBudget", "requestBytes", "byteLimit", "trimmedToolOutputs"] as const) {
+        const value = diagnostic[key];
+        if (typeof value === "number" && Number.isFinite(value) && value >= 0) data[key] = value;
+      }
+      try {
+        pi.appendEntry(COMPACTION_FALLBACK_ENTRY, data);
+      } catch {
+        // Diagnostics must not prevent the standard compactor from running.
+      }
+      if (ctx.mode === "tui" && ctx.hasUI) {
+        try {
+          ctx.ui.notify(`Codex remote compaction skipped: ${FALLBACK_MESSAGES[diagnostic.reason]}. Using standard Pi compaction.`, "warning");
+        } catch {
+          // Notification failure must not change compaction behavior either.
+        }
+      }
+    };
     try {
       const compaction = await createRemoteCompaction(event, ctx, () => {
         const active = new Set(pi.getActiveTools());
-        return pi.getAllTools()
-          .filter((tool) => active.has(tool.name))
-          .map(({ name, description, parameters }) => ({ name, description, parameters }));
-      }, pi.getThinkingLevel?.());
+        const data = {
+          model: ctx.model,
+          tools: pi.getAllTools().filter((tool) => active.has(tool.name)),
+        };
+        // Pi's public ToolInfo omits constrainedSampling. Tool owners can supply
+        // it here without exposing executors or inspecting private registries.
+        pi.events?.emit("pi-codex-compaction:tools:v1", data);
+        return data.tools;
+      }, pi.getThinkingLevel?.(), (payload, messages) => {
+        const data = { payload, messages, ctx };
+        // Synchronous bus contract: apply pure request transforms before bounds
+        // and before network I/O. Never include auth in this event.
+        pi.events?.emit("pi-codex-compaction:request:v1", data);
+        return data.payload;
+      }, reportFallback);
       return compaction ? { compaction } : undefined;
     } catch {
-      if (!event.signal.aborted && ctx.hasUI) {
-        ctx.ui.notify("Codex remote compaction failed; using standard Pi compaction.", "warning");
-      }
+      // Transport failures already have a reason; other exceptions mean the
+      // compaction request could not be prepared or processed.
+      reportFallback({ reason: "request-unavailable" });
       return undefined;
     }
   };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-codex-compaction",
-  "version": "0.1.1",
+  "version": "0.1.3",
   "description": "Use OpenAI Codex RemoteCompactionV2 for supported Pi sessions.",
   "type": "module",
   "license": "MIT",
@@ -52,10 +52,10 @@
     "@earendil-works/pi-coding-agent": "*"
   },
   "devDependencies": {
-    "@earendil-works/pi-ai": "^0.82.1",
-    "@earendil-works/pi-coding-agent": "^0.82.1",
-    "@types/node": "^26.1.2",
-    "tsx": "^4.23.1",
+    "@earendil-works/pi-ai": "^0.85.1",
+    "@earendil-works/pi-coding-agent": "^0.85.1",
+    "@types/node": "^26.2.0",
+    "tsx": "^4.23.5",
     "typescript": "^7.0.2"
   },
   "publishConfig": {

package/src/codex-wire.ts

@@ -1,10 +1,11 @@
 import { createHash } from "node:crypto";
 
-const MAX_REQUEST_BYTES = 16 * 1024 * 1024;
+export const MAX_COMPACTION_REQUEST_BYTES = 16 * 1024 * 1024;
 const MAX_RESPONSE_BYTES = 4 * 1024 * 1024;
 const MAX_ENCRYPTED_CONTENT_CHARS = 2_000_000;
 const REQUEST_HEADER_TIMEOUT_MS = 30_000;
 const REQUEST_IDLE_TIMEOUT_MS = 120_000;
+const REQUEST_TOTAL_TIMEOUT_MS = 300_000;
 const MAX_RETRIES = 2;
 const RETRY_BASE_DELAY_MS = 200;
 const MAX_RETRY_DELAY_MS = 30_000;
@@ -16,10 +17,10 @@ export interface CodexCompactionRequest {
   model: {
     id: string;
     baseUrl: string;
-    headers?: Record<string, string>;
+    headers?: Record<string, string | null>;
   };
   apiKey: string;
-  authHeaders?: Record<string, string>;
+  authHeaders?: Record<string, string | null>;
   sessionId?: string;
   body: Record<string, unknown>;
   signal?: AbortSignal;
@@ -30,12 +31,14 @@ export interface CodexCompactionUsage {
   outputTokens?: number;
   totalTokens?: number;
   cachedInputTokens?: number;
+  cacheWriteInputTokens?: number;
   reasoningTokens?: number;
 }
 
 export interface CodexCompactionResult {
   encryptedContent: string;
   usage?: CodexCompactionUsage;
+  serviceTier?: "default" | "priority" | "flex";
 }
 
 /**
@@ -49,17 +52,19 @@ export async function requestRemoteCompaction(request: CodexCompactionRequest):
 export async function requestRemoteCompactionWithUsage(
   request: CodexCompactionRequest,
 ): Promise<CodexCompactionResult> {
+  request.signal?.throwIfAborted();
   const endpoint = resolveCodexResponsesUrl(request.model.baseUrl);
   const accountId = extractAccountId(request.apiKey);
   const headers = buildHeaders(request, accountId);
   const body = JSON.stringify(request.body);
-  if (new TextEncoder().encode(body).byteLength > MAX_REQUEST_BYTES) {
+  if (Buffer.byteLength(body, "utf8") > MAX_COMPACTION_REQUEST_BYTES) {
     throw new Error("Codex compaction request exceeded the size limit");
   }
 
   const requestState = requestSignal(request.signal);
   try {
     for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
+      requestState.signal.throwIfAborted();
       requestState.startAttempt();
 
       try {
@@ -116,8 +121,14 @@ export function resolveCodexResponsesUrl(baseUrl: string): string {
   if (url.username || url.password) {
     throw new Error("Codex compaction endpoint must not contain URL credentials");
   }
+  if (url.port || url.search || url.hash) {
+    throw new Error("Codex compaction endpoint must use the default port without query or fragment");
+  }
 
   const path = url.pathname.replace(/\/+$/, "");
+  if (!["/backend-api", "/backend-api/codex", "/backend-api/codex/responses"].includes(path)) {
+    throw new Error("Codex compaction endpoint must use the Codex Responses path");
+  }
   if (path.endsWith("/codex/responses")) {
     return url.toString();
   }
@@ -140,14 +151,20 @@ export function parseCompactionSseResult(sse: string): CodexCompactionResult {
 
 function buildHeaders(request: CodexCompactionRequest, accountId: string): Headers {
   const headers = new Headers();
-  for (const [name, value] of Object.entries(request.model.headers ?? {})) headers.set(name, value);
-  for (const [name, value] of Object.entries(request.authHeaders ?? {})) headers.set(name, value);
+  for (const [name, value] of Object.entries(request.model.headers ?? {})) {
+    if (value !== null) headers.set(name, value);
+  }
+  for (const [name, value] of Object.entries(request.authHeaders ?? {})) {
+    if (value !== null) headers.set(name, value);
+    else headers.delete(name);
+  }
 
   headers.set("Authorization", `Bearer ${request.apiKey}`);
   headers.set("chatgpt-account-id", accountId);
   headers.set("originator", "pi");
   headers.set("OpenAI-Beta", "responses=experimental");
-  headers.set("x-codex-beta-features", BETA_FEATURE);
+  const features = headers.get("x-codex-beta-features")?.split(",").map((part) => part.trim()).filter(Boolean) ?? [];
+  headers.set("x-codex-beta-features", [...new Set([...features, BETA_FEATURE])].join(","));
   headers.set("accept", "text/event-stream");
   headers.set("content-type", "application/json");
 
@@ -208,6 +225,7 @@ async function readCompactionSse(
         throw new CodexCompactionError("Codex compaction response exceeded the size limit", false);
       }
       parser.push(decoder.decode(value, { stream: true }));
+      if (parser.isComplete) return parser.finish();
     }
     parser.push(decoder.decode());
     return parser.finish();
@@ -227,6 +245,11 @@ class CompactionSseParser {
   private status: string | undefined;
   private compactionItems: Array<string | undefined> = [];
   private usage: CodexCompactionUsage | undefined;
+  private serviceTier: CodexCompactionResult["serviceTier"];
+
+  get isComplete(): boolean {
+    return this.completed;
+  }
 
   push(chunk: string): void {
     this.buffer += chunk;
@@ -262,6 +285,7 @@ class CompactionSseParser {
     return {
       encryptedContent,
       ...(this.usage ? { usage: this.usage } : {}),
+      ...(this.serviceTier ? { serviceTier: this.serviceTier } : {}),
     };
   }
 
@@ -294,6 +318,8 @@ class CompactionSseParser {
       const response = isRecord(event.response) ? event.response : undefined;
       this.status = typeof response?.status === "string" ? response.status : undefined;
       this.usage = parseUsage(response?.usage ?? event.usage);
+      const tier = response?.service_tier;
+      this.serviceTier = tier === "default" || tier === "priority" || tier === "flex" ? tier : undefined;
 
       // Codex emits output_item.done before response.completed. Retain this
       // fallback for compact Responses implementations that only return the
@@ -331,12 +357,14 @@ function parseUsage(value: unknown): CodexCompactionUsage | undefined {
   const inputDetails = isRecord(value.input_tokens_details) ? value.input_tokens_details : undefined;
   const outputDetails = isRecord(value.output_tokens_details) ? value.output_tokens_details : undefined;
   const cachedInputTokens = finiteNonNegativeNumber(inputDetails?.cached_tokens);
+  const cacheWriteInputTokens = finiteNonNegativeNumber(inputDetails?.cache_write_tokens);
   const reasoningTokens = finiteNonNegativeNumber(outputDetails?.reasoning_tokens);
   if (
     inputTokens === undefined &&
     outputTokens === undefined &&
     totalTokens === undefined &&
     cachedInputTokens === undefined &&
+    cacheWriteInputTokens === undefined &&
     reasoningTokens === undefined
   ) {
     return undefined;
@@ -346,6 +374,7 @@ function parseUsage(value: unknown): CodexCompactionUsage | undefined {
     ...(outputTokens !== undefined ? { outputTokens } : {}),
     ...(totalTokens !== undefined ? { totalTokens } : {}),
     ...(cachedInputTokens !== undefined ? { cachedInputTokens } : {}),
+    ...(cacheWriteInputTokens !== undefined ? { cacheWriteInputTokens } : {}),
     ...(reasoningTokens !== undefined ? { reasoningTokens } : {}),
   };
 }
@@ -428,6 +457,10 @@ interface RequestSignal {
 
 function requestSignal(signal: AbortSignal | undefined): RequestSignal {
   const controller = new AbortController();
+  const totalTimeout = setTimeout(
+    () => controller.abort(new Error("Codex compaction request exceeded the total time limit")),
+    REQUEST_TOTAL_TIMEOUT_MS,
+  );
   let headerTimeout: ReturnType<typeof setTimeout> | undefined;
   let idleTimeout: ReturnType<typeof setTimeout> | undefined;
 
@@ -468,6 +501,7 @@ function requestSignal(signal: AbortSignal | undefined): RequestSignal {
     },
     touch,
     cleanup: () => {
+      clearTimeout(totalTimeout);
       clearTimers();
       signal?.removeEventListener("abort", onAbort);
     },

package/src/remote-compaction.ts

@@ -1,10 +1,11 @@
 import { stream as captureProviderPayload } from "@earendil-works/pi-ai/compat";
-import type { Model, Tool } from "@earendil-works/pi-ai";
+import { calculateCost, type Model, type Tool, type Usage } from "@earendil-works/pi-ai";
 import type { ExtensionContext, SessionBeforeCompactEvent } from "@earendil-works/pi-coding-agent";
 import { convertToLlm, serializeConversation, sessionEntryToContextMessages } from "@earendil-works/pi-coding-agent";
 import type { SessionEntry } from "@earendil-works/pi-coding-agent";
 import {
   getCodexAccountFingerprint,
+  MAX_COMPACTION_REQUEST_BYTES,
   requestRemoteCompactionWithUsage,
   resolveCodexResponsesUrl,
   type CodexCompactionUsage,
@@ -12,10 +13,24 @@ import {
 
 export const REMOTE_SUMMARY_MARKER = "[pi-codex-compaction:v1]";
 export const REMOTE_COMPACTION_KIND = "pi-codex-compaction";
+export const COMPACTION_FALLBACK_ENTRY = "pi-codex-compaction:fallback:v1";
+
+export interface CompactionFallback {
+  reason: "custom-instructions" | "auth-unavailable" | "request-unavailable" |
+    "context-window-unavailable" | "context-limit" | "request-size-limit" | "remote-failed";
+  estimatedTokens?: number;
+  tokenBudget?: number;
+  requestBytes?: number;
+  byteLimit?: number;
+  trimmedToolOutputs?: number;
+}
 
 const FALLBACK_SUMMARY_MAX_CHARS = 12_000;
 const MAX_ENCRYPTED_CONTENT_CHARS = 2_000_000;
 const COMPACTION_RESPONSE_RESERVE_TOKENS = 8_192;
+// Codex's ordinary-item estimate uses ceil(UTF-8 bytes / 4), not bytes as
+// tokens. This is a heuristic, not a tokenizer or a context-fit guarantee.
+const APPROX_BYTES_PER_TOKEN = 4;
 const TRUNCATED_TOOL_OUTPUT = "[Tool output omitted from the Codex compaction request to fit the active model context window.]";
 const PI_COMPACTION_SUMMARY_PREFIX = "The conversation history before this point was compacted into the following summary:";
 
@@ -95,23 +110,31 @@ export async function createRemoteCompaction(
   ctx: ExtensionContext,
   getTools: () => readonly ToolInfoLike[],
   thinkingLevel?: string,
+  prepareRequest?: (payload: Record<string, unknown>, messages: readonly AgentMessage[]) => Record<string, unknown>,
+  onFallback?: (diagnostic: CompactionFallback) => void,
 ): Promise<{
   summary: string;
   firstKeptEntryId: string;
   tokensBefore: number;
   details: RemoteCompactionDetails;
+  usage?: Usage;
 } | undefined> {
+  if (event.signal.aborted) return undefined;
+  const skip = (reason: CompactionFallback["reason"]) => {
+    onFallback?.({ reason });
+    return undefined;
+  };
   // Pi's custom focus is part of the standard summarizer contract. The
   // Responses compaction envelope has no documented equivalent, so do not
   // silently discard it.
-  if (event.customInstructions?.trim()) return undefined;
+  if (event.customInstructions?.trim()) return skip("custom-instructions");
 
   const model = ctx.model as CodexModel | undefined;
   if (!model || !supportsRemoteCompaction(model)) return undefined;
 
   const endpoint = resolveCodexResponsesUrl(model.baseUrl);
   const auth = await ctx.modelRegistry.getApiKeyAndHeaders(model as Model<any>);
-  if (!auth.ok || !auth.apiKey) return undefined;
+  if (!auth.ok || !auth.apiKey) return skip("auth-unavailable");
   const accountFingerprint = getCodexAccountFingerprint(auth.apiKey);
   const authKind = getCodexAuthKind(ctx.modelRegistry, model as Model<any>);
 
@@ -133,24 +156,29 @@ export async function createRemoteCompaction(
     event.signal,
     thinkingLevel,
   );
-  if (!providerPayload) return undefined;
+  if (!providerPayload) return skip("request-unavailable");
   const providerInput = providerPayload.input;
-  if (!Array.isArray(providerInput)) return undefined;
+  if (!Array.isArray(providerInput)) return skip("request-unavailable");
   const providerTools = Array.isArray(providerPayload.tools) ? providerPayload.tools : [];
 
   const input = appendCompactionItems(providerInput, event.preparation, compatiblePrevious);
-  const requestBody = {
+  const requestBody = prepareRequest?.({
     ...providerPayload,
+    input,
+  }, messages) ?? { ...providerPayload, input };
+  Object.assign(requestBody, {
     model: model.id,
     store: false,
     stream: true,
-  };
+  });
+  if (!Array.isArray(requestBody.input)) return skip("request-unavailable");
   const boundedInput = boundCompactionInput(
-    input,
+    requestBody.input,
     instructions,
-    providerTools,
+    Array.isArray(requestBody.tools) ? requestBody.tools : providerTools,
     model.contextWindow,
     requestBody,
+    onFallback,
   );
   if (!boundedInput) return undefined;
 
@@ -164,6 +192,11 @@ export async function createRemoteCompaction(
       input: boundedInput,
     },
     signal: event.signal,
+  }).catch((error: unknown) => {
+    // Only failures from the transport path receive this reason. Preparation
+    // failures are classified by the extension's outer fallback handler.
+    if (!event.signal.aborted) onFallback?.({ reason: "remote-failed" });
+    throw error;
   });
 
   const fallback = buildFallbackSummary(event.preparation, messages);
@@ -171,6 +204,11 @@ export async function createRemoteCompaction(
     summary: `${REMOTE_SUMMARY_MARKER}\n\n${fallback}`,
     firstKeptEntryId: event.preparation.firstKeptEntryId,
     tokensBefore: event.preparation.tokensBefore,
+    ...(result.usage ? { usage: toPiUsage(
+      result.usage,
+      ctx.model!,
+      result.serviceTier && result.serviceTier !== "default" ? result.serviceTier : requestBody.service_tier,
+    ) } : {}),
     details: {
       kind: REMOTE_COMPACTION_KIND,
       version: 2,
@@ -200,16 +238,25 @@ export function boundCompactionInput(
   tools: readonly unknown[],
   contextWindow: number,
   requestPayload?: Record<string, unknown>,
+  onLimit?: (diagnostic: CompactionFallback) => void,
 ): unknown[] | undefined {
-  const budget = Math.max(1, Math.floor(contextWindow - COMPACTION_RESPONSE_RESERVE_TOKENS));
-  // A byte-level bound is conservative when the active model's tokenizer is unavailable:
-  // a token can be represented by a single UTF-8 byte, but not fewer.
-  const budgetBytes = budget;
+  if (!Number.isFinite(contextWindow) || contextWindow <= 0) {
+    onLimit?.({ reason: "context-window-unavailable" });
+    return undefined;
+  }
+  const budgetTokens = Math.max(0, Math.floor(contextWindow) - COMPACTION_RESPONSE_RESERVE_TOKENS);
   const bounded = input.map((item) => item);
-  let requestBytes = estimateConservativeBytes(
+  let requestBytes = serializedBytes(
     buildCompactionRequest(requestPayload, bounded, instructions, tools),
   );
-  if (requestBytes <= budgetBytes) return bounded;
+  if (!Number.isFinite(requestBytes)) {
+    onLimit?.({ reason: "request-unavailable" });
+    return undefined;
+  }
+  const fits = () => requestBytes <= MAX_COMPACTION_REQUEST_BYTES &&
+    Math.ceil(requestBytes / APPROX_BYTES_PER_TOKEN) <= budgetTokens;
+  if (fits()) return bounded;
+  let trimmedToolOutputs = 0;
 
   // ponytail: trim tool outputs first; if structural content still exceeds the active model window,
   // let Pi's standard compaction path handle the request instead of inventing a lossy transcript rewrite.
@@ -218,11 +265,24 @@ export function boundCompactionInput(
     const replacement = trimToolOutput(item);
     if (!replacement) continue;
 
-    requestBytes += estimateConservativeBytes(replacement) - estimateConservativeBytes(item);
+    const removedBytes = serializedBytes(item) - serializedBytes(replacement);
+    // A short output can be smaller than the omission notice. Never make a
+    // request larger while trying to fit either limit.
+    if (removedBytes <= 0) continue;
+    requestBytes -= removedBytes;
     bounded[index] = replacement;
-    if (requestBytes <= budgetBytes) return bounded;
+    trimmedToolOutputs++;
+    if (fits()) return bounded;
   }
 
+  onLimit?.({
+    reason: requestBytes > MAX_COMPACTION_REQUEST_BYTES ? "request-size-limit" : "context-limit",
+    estimatedTokens: Math.ceil(requestBytes / APPROX_BYTES_PER_TOKEN),
+    tokenBudget: budgetTokens,
+    requestBytes,
+    byteLimit: MAX_COMPACTION_REQUEST_BYTES,
+    trimmedToolOutputs,
+  });
   return undefined;
 }
 
@@ -247,7 +307,7 @@ async function captureCodexPayload(
   messages: ReturnType<typeof convertToLlm>,
   tools: readonly ToolInfoLike[],
   apiKey: string,
-  authHeaders: Record<string, string> | undefined,
+  authHeaders: Record<string, string | null> | undefined,
   sessionId: string,
   signal: AbortSignal,
   thinkingLevel: string | undefined,
@@ -475,10 +535,12 @@ function buildCompactionRequest(
   instructions: string,
   tools: readonly unknown[],
 ): Record<string, unknown> {
-  const payload: Record<string, unknown> = requestPayload ? { ...requestPayload } : {};
-  payload.instructions = instructions;
+  // A supplied envelope is authoritative, including fields a listener removed.
+  // Separate defaults apply only to callers without a complete payload.
+  const payload: Record<string, unknown> = requestPayload
+    ? { ...requestPayload }
+    : { instructions, ...(tools.length > 0 ? { tools } : {}) };
   payload.input = input;
-  if (tools.length > 0 || requestPayload && "tools" in requestPayload) payload.tools = tools;
   return payload;
 }
 
@@ -499,10 +561,10 @@ function trimToolOutput(value: unknown): unknown | undefined {
   return undefined;
 }
 
-function estimateConservativeBytes(value: unknown): number {
+function serializedBytes(value: unknown): number {
   try {
     const json = JSON.stringify(value) ?? "";
-    return new TextEncoder().encode(json).byteLength;
+    return Buffer.byteLength(json, "utf8");
   } catch {
     return Number.POSITIVE_INFINITY;
   }
@@ -534,7 +596,27 @@ function limitText(text: string, maxChars: number): string {
   return `${text.slice(0, head)}${omission}${text.slice(text.length - (available - head))}`;
 }
 
-type ToolInfoLike = Pick<Tool, "name" | "description" | "parameters">;
+type ToolInfoLike = Tool;
+
+function toPiUsage(raw: CodexCompactionUsage, model: Model<any>, tier: unknown): Usage {
+  const cacheRead = raw.cachedInputTokens ?? 0;
+  const cacheWrite = raw.cacheWriteInputTokens ?? 0;
+  const output = raw.outputTokens ?? 0;
+  const usage: Usage = {
+    input: Math.max(0, (raw.inputTokens ?? 0) - cacheRead - cacheWrite),
+    output,
+    cacheRead,
+    cacheWrite,
+    ...(raw.reasoningTokens !== undefined ? { reasoning: raw.reasoningTokens } : {}),
+    totalTokens: raw.totalTokens ?? (raw.inputTokens ?? 0) + output,
+    cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
+  };
+  calculateCost(model, usage);
+  // Match Pi 0.85.1's catalog estimates, not subscription credit accounting.
+  const multiplier = tier === "priority" ? (model.id === "gpt-5.5" ? 2.5 : 2) : tier === "flex" ? 0.5 : 1;
+  for (const key of ["input", "output", "cacheRead", "cacheWrite", "total"] as const) usage.cost[key] *= multiplier;
+  return usage;
+}
 
 function isRecord(value: unknown): value is Record<string, unknown> {
   return typeof value === "object" && value !== null && !Array.isArray(value);