pi-openai-codex-compat 0.0.13 → 0.0.15

package/CHANGELOG.md

@@ -2,6 +2,57 @@
 
 ## Unreleased
 
+## 0.0.15 - 2026-09-23
+
+### Fixed
+
+- Preserve earlier message omissions and replacements during overflow-recovery
+  compaction. Recover only the latest attempt Pi omitted for recovery, keeping
+  its committed progress without reviving unrelated content. Stored native
+  responses no longer override later assistant-message replacements.
+- Rebuild the compaction request's tool declarations when a tool's parameter
+  schema changes under an unchanged name and description, instead of reusing a
+  stale schema from the previous turn.
+
+## 0.0.14 - 2026-09-22
+
+### Changed
+
+- Require Pi `>=0.87.0 <0.88.0` and use Pi 0.87.0 as the development and
+  packaged-CLI integration baseline. The load-time host check now also
+  requires Pi 0.87's session projection API.
+- Honour Pi 0.87 `context_edit` entries when building request history from
+  the branch, so omitted or replaced messages match what Pi's own adapters
+  send.
+- Preserve mid-conversation prompt updates when filtering compaction summaries.
+- Skip percentage compaction when nothing has been sampled yet. Pi 0.87
+  estimates the prompt and tool declarations before the first response, so a
+  low threshold could trigger an empty compaction request.
+- Send Pi's leading system message as the prompt and later system messages as
+  inline developer items, exactly as Pi AI's Codex adapter does. Previously
+  every system message was replayed into one `instructions` string, so any
+  prompt-section change rewrote the stable prefix. Models without
+  mid-conversation system-message support still receive one collapsed prompt.
+- Keep every developer and system message in the native compaction checkpoint,
+  in place and outside the 64k retained-context budget, so prompt updates
+  survive compaction. Only user messages are charged to the budget.
+
+### Fixed
+
+- Keep the conversation in overflow-recovery compaction on Pi 0.87. Pi now
+  records the failed attempt's omission as a session entry whose parent is
+  that attempt; removing the attempt from the branch copy broke the entry
+  chain and the compaction request contained only the recovered prefix.
+- Hold a system message that lands between a tool call and its results until
+  after those results in replayed history, matching Pi AI.
+- Send the branch's leading system message as compaction `instructions`
+  instead of Pi's fully replayed prompt, so Pi-driven compaction shares the
+  turn requests' prompt prefix.
+- Reuse the last turn's exact tool declarations for `/compact` and threshold
+  compaction while they still describe the active tools. Rebuilding them from
+  the tool registry lost Pi's strict schemas for built-in tools, so every
+  Pi-driven compaction request missed the turn requests' prompt-cache prefix.
+
 ## 0.0.13 - 2026-09-21
 
 ### Fixed

package/README.md

@@ -22,7 +22,7 @@ Pi provides the Codex OAuth flow and model catalog. At session start, this packa
 ## Requirements
 
 - Node.js 22.19 or newer
-- Pi `>=0.86.0 <0.87.0`
+- Pi `>=0.87.0 <0.88.0`
 - An OpenAI Codex login in Pi
 
 Restart Pi after upgrading its runtime. `/reload` reloads extensions but cannot
@@ -67,7 +67,7 @@ The compatibility baseline is official Codex CLI `0.149.1`, released August 24,
 | Session storage             | Pi remains the canonical session owner. Opaque Codex compaction checkpoints are stored in Pi `compaction` entries, and otherwise lossy Responses output is stored in sparse custom native-response entries. Official Codex owns a rollout/thread store directly.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
 | Branching                   | Checkpoints and native response overrides follow Pi's active session branch. Official Codex uses its own thread, turn, rollback, fork, and context-window lineage.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
 | Model switching             | This package rejects model switches while the active Pi branch contains a native Codex checkpoint because the checkpoint is model-specific.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
-| System instructions         | Replay Pi's system-message sections into the complete current prompt. Responses Lite models prepend it as developer input after `additional_tools`; other models send it through Responses `instructions`. Do not replay older Pi system-prompt text alongside that current prompt. `/reload`, forced prompts, compaction snapshots, and resumed sessions use the current state without rewriting old checkpoints.                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+| System instructions         | Send the leading Pi system message as the prompt, as Pi AI's Codex adapter does: Responses Lite models prepend it as developer input after `additional_tools`; other models send it through Responses `instructions`. Later Pi system messages (`/reload`, section changes) travel inline as developer items on models that accept mid-conversation system messages and collapse into the leading prompt otherwise. A forced prompt replaces the leading message. Compaction snapshots and resumed sessions lead with Pi's replayed prompt without rewriting old checkpoints.                                                                                                                                                                                                                                                                                                                    |
 | Dynamic tool declarations   | Send the complete current tool set, replayed from Pi's `toolsAdded` and `toolsRemoved` system messages, in the top-level `tools` field on every request, as the official Codex client does. Tool declarations never travel inside `input`: inline `additional_tools` items and synthetic tool-search pairs made the Codex backend unreliable about which tools exist. A mid-session tool change costs one prompt-cache miss, and native checkpoints need no declaration rebase.                                                                                                                                                                                                                                                                                                                                                                                                                  |
 | Turn metadata               | Requests send a persisted installation id plus Pi-derived session, thread, context-window, turn, source, sandbox, request-kind, and nested compaction-operation metadata in `client_metadata` and compatible headers. The in-memory context-window number advances after successful compaction. One turn id is reused throughout a Pi agent run, while prewarm has its own id. First-party requests also carry Codex's model-and-tier routing hint. The provider captures the server-issued `x-codex-turn-state` once per agent run, replays it on WebSocket retries, SSE requests, and WebSocket-to-SSE fallback, and records all identity values in transport diagnostics. Pi does not reconstruct prior window number after extension reload/session resume or reproduce workspace Git/parent/subagent/Code Mode metadata. Each marked Pi tree branch receives its own persisted thread UUID. |
 | Cache preparation           | Before the first cache-enabled WebSocket turn, the package prewarms only the stable instruction/tool prefix: ordinary Responses uses empty `input`, while Responses Lite uses `additional_tools` plus the developer instructions. The first generated request then contributes only dynamic conversation input to the continuation. No explicit prompt-cache breakpoints are added.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
@@ -76,7 +76,7 @@ The compatibility baseline is official Codex CLI `0.149.1`, released August 24,
 | Transport recovery          | WebSocket failures before model-visible output receive five fresh-connection retries before sticky SSE fallback; SSE transport failures receive five retries. Official Codex `0.149.1` separately retries sampling connection-establishment failures indefinitely with delays capped at 60 seconds. The package remains bounded because Fetch does not expose Reqwest's narrower connection-error category, and a phase-only approximation could indefinitely repeat requests that reached the server.                                                                                                                                                                                                                                                                                                                                                                                           |
 | Compaction lifecycle events | Pre-turn percentage compaction writes through Pi's mutable session manager and cannot emit Pi's internal `session_compact` event through the public extension API. Mid-response percentage boundaries and manual, threshold, or overflow compactions initiated by Pi emit the normal lifecycle.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
 | Header hooks                | An internal percentage-compaction request reuses the already transformed provider headers. It cannot independently rerun Pi's `before_provider_headers` hook.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
-| Native retained context     | Deliberately differs from current Codex. The package retains recent user/developer/system messages under a text-only 64k budget before the opaque compaction item. Current Codex applies a second installed-history filter that drops developer/system wrappers and non-real-user messages, can retain eligible structured agent commentary, and trims oversized function outputs before compaction. Codex `0.149.1` also adds a default-disabled mode that charges retained images to the budget and keeps each image with adjacent harness labels as an atomic boundary unit. Pi keeps its existing checkpoint shape by design.                                                                                                                                                                                                                                                                |
+| Native retained context     | Deliberately differs from current Codex. The package retains recent user messages under a text-only 64k budget before the opaque compaction item, and keeps every developer and system message in place outside that budget so prompt updates are never compacted away. Current Codex applies a second installed-history filter that drops developer/system wrappers and non-real-user messages, can retain eligible structured agent commentary, and trims oversized function outputs before compaction. Codex `0.149.1` also adds a default-disabled mode that charges retained images to the budget and keeps each image with adjacent harness labels as an atomic boundary unit. Pi keeps its existing checkpoint shape by design.                                                                                                                                                           |
 | Tool namespaces             | Responses Lite groups Pi's ordinary function/custom declarations into upstream's canonical `functions` namespace and maps that default namespace back to bare Pi names. Pi registers dotted names such as `web.run` as exact flat identifiers, so the provider converts only the fixed extension-owned allowlist into non-default Responses namespace/member identities and rejects unknown or ambiguously flat namespaced calls.                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
 | Capability gating           | Tool activation is based on the selected `openai-codex` provider plus package settings. It does not reproduce every official model-metadata, plan, feature-stage, executor, mode, or account gate.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
 | Sandbox and approvals       | Pi extensions run with full process permissions. Command tools, `apply_patch`, local image reads, generated-image writes, and sibling Codex endpoints do not use Codex's sandbox, permission-profile, environment, or approval lifecycle.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
@@ -327,9 +327,22 @@ The extension handles native compaction for `openai-codex`. It follows the Codex
 
 1. Send normal Responses history followed by `{ "type": "compaction_trigger" }`.
 2. Validate the returned opaque `compaction` item.
-3. Retain approximately 64,000 tokens of recent user, developer, and system context.
+3. Retain approximately 64,000 tokens of recent user messages, plus every developer and
+   system message outside that budget. Repeated prompt updates (for example several
+   `/reload` runs or prompt-section changes) each remain in place as their own inline
+   developer item; they accumulate across checkpoints and are never dropped, merged, or
+   deduplicated, because each one is a distinct instruction the model already saw.
 4. Persist the opaque checkpoint in the Pi session and replay it on later requests.
 
+Overflow and length recovery follow Pi 0.87's `context_edit` semantics. Every request
+reads Pi's session projection, so earlier omissions and replacements (an elided tool
+output, an omitted reply) apply exactly as they do for Pi's own adapters. Before an
+overflow-recovery compaction, Pi omits the latest failed or truncated attempt and its tool
+results; the checkpoint request projects only those entries as they were before that
+omission, so a truncated response and its tool results survive as committed progress while
+every unrelated earlier edit stays in force. A failed attempt contributes only the native
+output committed before the overflowing subrequest.
+
 Ordinary responses and compaction use the same extension-managed SSE/WebSocket transport. Before the first WebSocket turn for a session and model, the provider performs a best-effort v2 `generate: false` prewarm of the static instruction/tool prefix, then generates the dynamic conversation input from its continuation. With `responsesLite: false`, supported models use the ordinary Responses envelope and receive their own ordinary-prefix prewarm.
 
 Healthy session WebSockets remain available until the server closes them or Pi tears down the session. Retryable WebSocket failures before model-visible output receive up to five fresh-connection retries before the session switches to sticky SSE. Retryable SSE HTTP failures and dropped streams receive up to five same-request resampling attempts before model-visible output. Both transports use Codex-style exponential backoff with ±10% jitter and preserve the same prompt-cache, session, account, installation, and window identities. Server metadata is not considered model-visible output, so a routing-state-only response can still be retried safely. Transport failures after model-visible output fail closed rather than risk duplicate text or tool calls. Explicit retryable `response.failed`/`response.incomplete` protocol terminals instead return to the provider-owned sampling loop, which preserves completed output items as the next request's history.
@@ -575,6 +588,15 @@ mise run check
 npm run pack:dry
 ```
 
+`mise run check` runs the linters, formatters, type checks, and the offline
+test suite; `mise run test` runs only the offline suite. Both tasks bind
+`PI_PACKAGE_DIR` to `node_modules/@earendil-works/pi-coding-agent`, so tests
+that load Pi in-process read this repository's Pi 0.87.0 resources even when a
+global `PI_PACKAGE_DIR` points at another installation. Ordinary `pi` launches
+outside these tasks are unaffected. Offline tests also exercise
+`scripts/release.ts` with every child process mocked: they never run Git, npm,
+Mise, or provider calls.
+
 Run the credentialed Pi/Codex integration tests separately. They load the real
 extension into headless Pi sessions, use the real WebSocket service, and ask
 the model to report all prior history markers after each text and tool
@@ -587,11 +609,26 @@ mise run test:live:codex
 The task obtains the local Codex bearer token and runs the tests with
 `gpt-5.6-luna` at medium reasoning effort.
 It also packs the extension and loads that archive through the shipped Pi
-0.86.1 CLI. Ordinary Responses and Responses Lite tests exercise tool calls,
+0.87.0 CLI. Ordinary Responses and Responses Lite tests exercise tool calls,
 prompt reload, native compaction, and persisted-session resume against Codex.
 Test credentials stay in memory and child-process environments. Test sessions
 and configuration are isolated from the user's agent directory.
 
+The packaged-CLI test selects its archive and executable as follows:
+
+- Without `PI_CODEX_PACKAGE_ARCHIVE`, it packs the current worktree into a
+  temporary directory. With `PI_CODEX_PACKAGE_ARCHIVE`, it loads exactly that
+  archive: the value resolves against the working directory, must be an
+  existing regular file that `tar` can list, and an empty, missing, directory,
+  or invalid value fails the test instead of falling back to a fresh pack.
+- `PI_CODEX_CLI_PATH` selects another Pi `cli.js`; the default is this
+  repository's Pi 0.87.0 dependency. The test asserts that the selected
+  executable reports version 0.87.0, the only Pi version the packaged CLI test
+  is run against. The supported range is `>=0.87.0 <0.88.0`.
+- Each Pi child process receives `PI_PACKAGE_DIR` bound to the selected
+  executable's package directory, so the runtime under test reads its own
+  metadata.
+
 The public package entrypoint is `extensions/index.ts`; implementation modules remain under
 `extensions/openai-codex-compat/`. The focused Pi AI serializer copy lives under
 `extensions/openai-codex-compat/vendor/pi-ai/`. The custom Codex provider transport and stream
@@ -605,10 +642,74 @@ search, and compaction continuation.
 1. Run `npm run release -- X.Y.Z` from a clean, synchronized `main`.
 2. The command builds the exact package locally and runs live CLI tests against that archive. It also runs the existing live SDK tests against checkout source. Both suites must pass before it records the archive's SHA-256 in an SSH-signed release commit, proves a clean rebuild is reproducible, and creates a lightweight tag. Missing credentials or failing live tests stop the release.
 3. Inspect the result, then push atomically with `git push --atomic origin main vX.Y.Z`.
-4. A read-only GitHub Actions job validates and packs the package. After approval in the tag-restricted `npm-publish` environment, a separate GitHub-owned job verifies the signature and signed digest before attesting and staging that exact archive through npm trusted publishing.
-5. A final job creates the immutable GitHub release for the tag from the same verified archive, its
-   checksum, and the version's `CHANGELOG.md` section (`Unreleased` for prereleases).
-6. Approve the staged package on npmjs.com, or with `npm stage approve <stage-id>`.
+4. A read-only GitHub Actions job validates and packs the package. After approval
+   in the tag-restricted `npm-publish` environment, a separate GitHub-owned job
+   verifies the signature and signed digest before attesting and staging that
+   exact archive through npm trusted publishing.
+5. A final job creates the immutable GitHub release from the verified archive,
+   its checksum, and the version's changelog section (`Unreleased` for prereleases).
+6. Approve the staged package on npmjs.com or with `npm stage approve <stage-id>`.
+
+The release command runs these steps in order:
+
+1. Refuse to start unless the branch is `main`, `git status --porcelain` is
+   empty, `CHANGELOG.md` has a section for the version, `HEAD` equals
+   `origin/main` after a fetch, and tag `vX.Y.Z` does not exist. Nothing has
+   changed when one of these checks fails.
+2. Write the version into `package.json` and `package-lock.json` and stage
+   only those two files.
+3. Check out the staged index into a temporary directory, run
+   `npm ci --ignore-scripts` and `npm pack` there, validate the package
+   identity, file list, and manifest, then run `mise run test:live:codex` in
+   the repository with `PI_CODEX_PACKAGE_ARCHIVE` set to that archive. This is
+   the exact archive release gate: the live CLI tests load that archive and
+   nothing else. A live process that cannot start, a nonzero exit, or a failed
+   package validation stops the release here.
+4. Create the SSH-signed `release: vX.Y.Z` commit with the archive's SHA-256
+   as its `Npm-Artifact-SHA256` trailer and verify the signature and trailer.
+5. Rebuild the package from the committed tree without repeating the live
+   tests and require the same digest.
+6. Create the lightweight tag and verify that it points at the release commit.
+
+### Recovering from a failed release command
+
+Do not run any blanket `git restore`, `git reset`, `git checkout --`, or
+`git clean`. Inspect first, then undo only what the failed command produced.
+
+If the command fails during the version update, version changes can remain
+in the worktree. Once the update and `git add` succeed, package, live-test,
+and signing failures leave those changes staged. No release commit or tag
+was created by that attempt. Inspect the state:
+
+```sh
+git status --short
+git diff -- package.json package-lock.json
+git diff --cached -- package.json package-lock.json
+```
+
+Undo only this attempt's version edits in the worktree and index. Preserve
+concurrent changes, including edits in those same files. Do not stage whole
+files containing unrelated edits. Fix the cause and confirm that `main` is
+clean and synchronized before rerunning the release command.
+
+If the command fails after the release commit (steps 4 to 6), a local signed
+`release: vX.Y.Z` commit exists on `main`, and a local tag may exist. Do not rerun
+the release command: it would refuse the unsynchronized `HEAD`, and a second
+run on top of the existing commit would produce a duplicate release commit.
+Do not push the commit or tag until signature verification, the reproducibility
+check, and tag verification are complete. Inspect the commit and the digest:
+
+```sh
+git log -1 --format='%H%n%s%n%(trailers:key=Npm-Artifact-SHA256,valueonly)' HEAD
+git diff origin/main..HEAD -- package.json package-lock.json
+git tag --list 'vX.Y.Z'
+```
+
+Removing the local release commit or tag changes local refs. Confirm that
+they were never pushed, record their hashes and a recovery path, and obtain
+explicit approval before changing them. Never replace a published tag.
+Fix the underlying cause before starting a new release from clean,
+synchronized `main`.
 
 Stable releases use `latest`; prereleases derive their npm dist-tag from the first prerelease identifier.
 

package/extensions/openai-codex-compat/codex-protocol.ts

@@ -276,17 +276,23 @@ function shortenMessage(
     : undefined;
 }
 
-function retainedRole(item: ResponsesInputItem): boolean {
-  return (
-    Value.Check(RESPONSES_MESSAGE_ITEM_SCHEMA, item) &&
-    (item.role === "user" || item.role === "developer" || item.role === "system")
-  );
+function messageRole(item: ResponsesInputItem): string | undefined {
+  return Value.Check(RESPONSES_MESSAGE_ITEM_SCHEMA, item) ? item.role : undefined;
+}
+
+/** Developer and system messages carry prompt state, not conversation history. */
+function isInstructionMessage(item: ResponsesInputItem): boolean {
+  const role = messageRole(item);
+  return role === "developer" || role === "system";
 }
 
 /**
- * Reproduce Codex remote-compaction-v2 history installation: select recent
- * user/developer/system messages from newest to oldest, then restore their
- * chronological order. The oldest selected message may be truncated.
+ * Reproduce Codex remote-compaction-v2 history installation for user messages:
+ * select recent user messages from newest to oldest under the budget, then
+ * restore chronological order. The oldest selected user message may be
+ * truncated. Developer and system messages are prompt updates rather than
+ * conversation history: every one of them is retained in place, outside the
+ * budget, and never truncated.
  */
 export function selectRetainedContext(
   history: readonly ResponsesInputItem[],
@@ -295,10 +301,14 @@ export function selectRetainedContext(
   let remaining = budget;
   const newestFirst: ResponsesInputItem[] = [];
 
-  for (let index = history.length - 1; index >= 0 && remaining > 0; index--) {
+  for (let index = history.length - 1; index >= 0; index--) {
     const item = history[index];
     if (item === undefined) continue;
-    if (!retainedRole(item)) continue;
+    if (isInstructionMessage(item)) {
+      newestFirst.push(structuredClone(item));
+      continue;
+    }
+    if (remaining <= 0 || messageRole(item) !== "user") continue;
 
     const tokens = Math.max(1, messageTextTokens(item));
     if (tokens <= remaining) {

package/extensions/openai-codex-compat/codex-provider/codex-provider-contracts.ts

@@ -47,6 +47,8 @@ export type RuntimeScope = {
 export type RequestTemplate = {
   modelId: string;
   payload: JsonRecord;
+  /** Fingerprint of the raw tool definitions that produced `payload.tools`. */
+  toolFingerprint: string;
   grammarToolInputProperties: GrammarToolInputProperties;
   requestOptions: OpenAICodexResponsesOptions;
 };

package/extensions/openai-codex-compat/codex-provider/codex-provider-history.ts

@@ -1,4 +1,4 @@
-import type { SessionEntry, ToolInfo } from "@earendil-works/pi-coding-agent";
+import type { SessionEntry } from "@earendil-works/pi-coding-agent";
 import type { Api, Model } from "@earendil-works/pi-ai";
 import { providerHistory, type GrammarToolInputProperties } from "../compaction-checkpoint.ts";
 import type { ImageDetail } from "../config.ts";
@@ -37,7 +37,6 @@ export function splitUnsampledUserInput(options: {
   branch: readonly SessionEntry[];
   history: readonly ResponsesInputItem[];
   model: Model<Api>;
-  allTools: readonly ToolInfo[];
   grammarToolInputProperties: GrammarToolInputProperties;
   imageDetail: ImageDetail;
 }):
@@ -56,7 +55,6 @@ export function splitUnsampledUserInput(options: {
   const encoded = providerHistory({
     branch: options.branch.slice(unsampledIndex),
     wireModel: options.model,
-    allTools: options.allTools,
     grammarToolInputProperties: options.grammarToolInputProperties,
     imageDetail: options.imageDetail,
   });

package/extensions/openai-codex-compat/codex-provider/codex-provider-runtime.ts

@@ -5,10 +5,8 @@ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
 import {
   clampThinkingLevel,
   createAssistantMessageEventStream,
-  getCurrentSystemPrompt,
   getCurrentTools,
   getDeclaredTools,
-  normalizeContext,
   type Api,
   type AssistantMessage,
   type AssistantMessageEventStream,
@@ -30,8 +28,11 @@ import {
   checkpointData,
   providerHistory,
   remoteCompactionMarkerSummary,
+  requestInstructions,
+  resolveRequestTranscript,
   responsesCompatibility,
   searchCheckpoint,
+  toolDefinitionFingerprint,
   type CheckpointData,
   type CompactionDecision,
   type GrammarToolInputProperties,
@@ -502,8 +503,7 @@ export class CodexProviderRuntime {
       const nativeItems = new Map<string, ResponsesOutputItem[]>();
       const serializationOptions: NonNullable<Parameters<typeof convertResponsesMessages>[3]> = {
         includeSystemPrompt: false,
-        includeSystemUpdates: false,
-        supportsMidConvoSystemMessages: true,
+        supportsMidConvoSystemMessages: compat.supportsMidConvoSystemMessages ?? false,
         supportsAdditionalTools: false,
         supportsToolSearch: false,
         grammarToolInputProperties,
@@ -524,7 +524,6 @@ export class CodexProviderRuntime {
     return providerHistory({
       branch: scope.manager.getBranch(),
       wireModel: model,
-      allTools: this.pi.getAllTools(),
       grammarToolInputProperties,
       imageDetail: scope.config.imageDetail,
     });
@@ -547,11 +546,14 @@ export class CodexProviderRuntime {
     // checkpoint drops such history anyway. A mid-session tool change costs one
     // cache miss.
     const requestTools = getCurrentTools(context.messages);
+    // The leading system message is the prompt; later system messages travel
+    // inline in `input` (or collapsed into the leading message by
+    // resolveRequestTranscript), as Pi AI's Codex adapter does.
     let body: JsonRecord = {
       model: model.id,
       store: false,
       stream: true,
-      instructions: getCurrentSystemPrompt(context.messages) || "You are a helpful assistant.",
+      instructions: requestInstructions(context) || "You are a helpful assistant.",
       input: this.wireHistory(model, context, grammarToolInputProperties, runtimeSessionId),
       text: { verbosity: requestOptions.textVerbosity ?? "low" },
       include: ["reasoning.encrypted_content"],
@@ -735,7 +737,6 @@ export class CodexProviderRuntime {
       branch,
       history,
       model,
-      allTools: this.pi.getAllTools(),
       grammarToolInputProperties,
       imageDetail: scope.config.imageDetail,
     });
@@ -746,6 +747,10 @@ export class CodexProviderRuntime {
       );
       return body;
     }
+    // Pi 0.87 estimates the prompt and tool declarations before any response
+    // usage exists, so a low threshold can be reached before anything was
+    // sampled. There is nothing to compact until the first response.
+    if (split.history.length === 0) return body;
 
     const compacted = await this.performCompaction({
       model,
@@ -755,7 +760,7 @@ export class CodexProviderRuntime {
       template: withoutConversationInput(body),
       instructions: isString(body.instructions)
         ? body.instructions
-        : getCurrentSystemPrompt(context.messages) || "You are a helpful assistant.",
+        : requestInstructions(context) || "You are a helpful assistant.",
       grammarToolInputProperties,
       priority: scope.config.fastMode,
       compactionMetadata: responsesCompactionV2Metadata("auto", "context_limit", "pre_turn"),
@@ -797,7 +802,7 @@ export class CodexProviderRuntime {
     input: Context,
     options?: OpenAICodexResponsesOptions,
   ): AssistantMessageEventStream {
-    const context = normalizeContext(input);
+    const context = resolveRequestTranscript(model, input);
     const stream = createAssistantMessageEventStream();
     const requestOptions = transportOptions(options);
     const output: AssistantMessage = {
@@ -867,6 +872,7 @@ export class CodexProviderRuntime {
         this.templates.set(runtimeSessionId, {
           modelId: model.id,
           payload: replacementValues ? {} : withoutConversationInput(body),
+          toolFingerprint: toolDefinitionFingerprint(getCurrentTools(context.messages)),
           grammarToolInputProperties,
           requestOptions: { ...requestOptions },
         });

package/extensions/openai-codex-compat/compaction-checkpoint.ts

@@ -1,13 +1,26 @@
 import { isBoolean, isString } from "./value-contracts.ts";
-import { randomUUID } from "node:crypto";
+import { createHash, randomUUID } from "node:crypto";
 import {
   buildSessionContext,
+  buildSessionProjection,
   convertToLlm,
   sessionEntryToContextMessages,
+  type ContextEditEntry,
   type SessionEntry,
   type ToolInfo,
 } from "@earendil-works/pi-coding-agent";
-import type { Api, Message, Model, SystemMessage } from "@earendil-works/pi-ai";
+import {
+  getInitialSystemMessage,
+  getSystemMessageText,
+  normalizeContext,
+  resolveTranscript,
+  type Api,
+  type Context,
+  type Message,
+  type Model,
+  type SystemMessage,
+  type TranscriptContext,
+} from "@earendil-works/pi-ai";
 import { Value } from "typebox/value";
 import { APPLY_PATCH_LARK_GRAMMAR, APPLY_PATCH_TOOL_NAME } from "./apply-patch.ts";
 import { CODEX_TOOL_CALL_PROVIDERS } from "./codex-identifiers.ts";
@@ -19,6 +32,7 @@ import {
   type JsonRecord,
 } from "./codex-protocol.ts";
 import { nativeCommittedPrefixBeforeOverflow, nativeResponseOverrides } from "./native-history.ts";
+import { stableResponsesJson } from "./responses-replay.ts";
 import {
   CODEX_NAMESPACED_TOOL_NAMES,
   CODEX_TEXT_CONTENT_ITEM_TOOL_RESULT_NAMES,
@@ -34,6 +48,8 @@ import {
 import type { ResponsesToolDefinition } from "./responses-tool-schema.ts";
 import { convertResponsesMessages } from "./vendor/pi-ai/openai-responses-serialization.ts";
 
+type AgentMessages = ReturnType<typeof sessionEntryToContextMessages>;
+
 export const CHECKPOINT_ENTRY_TYPE = "openai-codex-compat-remote-compaction";
 export const CHECKPOINT_FORMAT_VERSION = 1;
 
@@ -58,17 +74,16 @@ export type CheckpointSearch =
 export type GrammarToolInputProperties = ReadonlyMap<string, string>;
 
 export interface ResponsesCompatibility {
-  supportsAdditionalTools?: boolean;
+  supportsMidConvoSystemMessages?: boolean;
   supportsOpenAIGrammarTools?: boolean;
   supportsStrictMode?: boolean;
-  supportsToolSearch?: boolean;
 }
 
 export function responsesCompatibility(value: unknown): ResponsesCompatibility {
   const compatibility: ResponsesCompatibility = {};
   if (!isObject(value)) return compatibility;
-  if (isBoolean(value["supportsAdditionalTools"])) {
-    compatibility.supportsAdditionalTools = value["supportsAdditionalTools"];
+  if (isBoolean(value["supportsMidConvoSystemMessages"])) {
+    compatibility.supportsMidConvoSystemMessages = value["supportsMidConvoSystemMessages"];
   }
   if (isBoolean(value["supportsOpenAIGrammarTools"])) {
     compatibility.supportsOpenAIGrammarTools = value["supportsOpenAIGrammarTools"];
@@ -76,12 +91,37 @@ export function responsesCompatibility(value: unknown): ResponsesCompatibility {
   if (isBoolean(value["supportsStrictMode"])) {
     compatibility.supportsStrictMode = value["supportsStrictMode"];
   }
-  if (isBoolean(value["supportsToolSearch"])) {
-    compatibility.supportsToolSearch = value["supportsToolSearch"];
-  }
   return compatibility;
 }
 
+/**
+ * Resolve Pi's transcript the way Pi AI's Codex adapter does before building a
+ * request: models that accept mid-conversation system messages keep later
+ * system messages in place, and every other model collapses them into the
+ * leading system message.
+ */
+export function resolveRequestTranscript(model: Model<Api>, context: Context): TranscriptContext {
+  return resolveTranscript(
+    normalizeContext(context),
+    responsesCompatibility(model.compat).supportsMidConvoSystemMessages ?? false,
+  );
+}
+
+/** The leading system message of a resolved transcript, rendered as Responses `instructions`. */
+export function requestInstructions(context: TranscriptContext): string {
+  const initial = getInitialSystemMessage(context.messages);
+  return initial ? getSystemMessageText(initial) : "";
+}
+
+/** `instructions` for a request built from a session branch rather than Pi's live transcript. */
+export function branchInstructions(model: Model<Api>, branch: readonly SessionEntry[]): string {
+  return requestInstructions(
+    resolveRequestTranscript(model, {
+      messages: convertToLlm(buildSessionContext([...branch]).messages),
+    }),
+  );
+}
+
 function responsesToolParameters(tool: ToolInfo): JsonRecord {
   if (!isObject(tool.parameters)) {
     throw new Error(`Tool ${tool.name} must have JSON object parameters.`);
@@ -143,6 +183,81 @@ export function activeResponsesTools(
     : undefined;
 }
 
+/** The raw tool definition fields that both Pi AI's `Tool` and Pi's `ToolInfo` carry. */
+export type ToolDeclarationSource = Pick<ToolInfo, "name" | "description" | "parameters">;
+
+/**
+ * Fingerprint raw tool definitions by name, description, and parameter schema,
+ * independent of declaration order. Both the transcript tools a turn request
+ * declared and the registry's active `ToolInfo` list produce the same value for
+ * the same definitions, so a cached turn template can be compared with the
+ * current tools without reversing the strict schema conversion applied on the
+ * wire.
+ */
+export function toolDefinitionFingerprint(tools: readonly ToolDeclarationSource[]): string {
+  const declarations = tools
+    .map((tool) => {
+      // The JSON round-trip drops typebox symbol keys and undefined fields, as
+      // Pi AI's own declaration comparison does.
+      const parameters: unknown = JSON.parse(JSON.stringify(tool.parameters));
+      return { name: tool.name, description: tool.description, parameters };
+    })
+    .sort((left, right) => left.name.localeCompare(right.name));
+  return createHash("sha256").update(stableResponsesJson(declarations)).digest("hex");
+}
+
+/** Fingerprint of the registry tools that are currently active. */
+export function activeToolFingerprint(
+  allTools: readonly ToolInfo[],
+  activeNames: readonly string[],
+): string {
+  const enabled = new Set(activeNames);
+  return toolDefinitionFingerprint(allTools.filter((tool) => enabled.has(tool.name)));
+}
+
+/**
+ * Whether a cached turn template still declares exactly the active tools.
+ *
+ * The serialized `tools` list must name the active set (namespace members
+ * count as `namespace.member`), and the raw definitions that produced it must
+ * fingerprint identically to the active registry tools. Comparing names and
+ * descriptions alone missed parameter-schema changes under an unchanged name.
+ *
+ * A cached turn template carries the exact declarations the turn requests sent,
+ * including strict schema conversion that `ToolInfo` cannot reproduce. Reusing
+ * it keeps the compaction request on the turn requests' prompt-cache prefix.
+ */
+export function declaresActiveTools(
+  serialized: readonly JsonRecord[],
+  cachedToolFingerprint: string,
+  allTools: readonly ToolInfo[],
+  activeNames: readonly string[],
+): boolean {
+  if (cachedToolFingerprint !== activeToolFingerprint(allTools, activeNames)) return false;
+  const declared = new Map<string, string | undefined>();
+  for (const item of serialized) {
+    if (item.type === "namespace" && isString(item.name) && Array.isArray(item.tools)) {
+      for (const member of item.tools) {
+        if (!isObject(member) || !isString(member.name)) return false;
+        declared.set(
+          `${item.name}.${member.name}`,
+          isString(member["description"]) ? member["description"] : undefined,
+        );
+      }
+    } else if ((item.type === "function" || item.type === "custom") && isString(item.name)) {
+      declared.set(item.name, isString(item["description"]) ? item["description"] : undefined);
+    } else {
+      return false;
+    }
+  }
+  const enabled = new Set(activeNames);
+  const active = allTools.filter((tool) => enabled.has(tool.name));
+  return (
+    declared.size === active.length &&
+    active.every((tool) => declared.has(tool.name) && declared.get(tool.name) === tool.description)
+  );
+}
+
 export function remoteCompactionMarkerSummary(): string {
   return `OpenAI Codex remote compaction checkpoint (${randomUUID()}).`;
 }
@@ -150,7 +265,6 @@ export function remoteCompactionMarkerSummary(): string {
 type EncodeMessagesOptions = {
   model: Model<Api>;
   messages: Message[];
-  allTools: readonly ToolInfo[];
   grammarToolInputProperties: GrammarToolInputProperties;
   imageDetail: ImageDetail;
   nativeAssistantItems?: ReadonlyMap<string, readonly ResponsesOutputItem[]>;
@@ -159,16 +273,22 @@ type EncodeMessagesOptions = {
 export type EncodeSessionEntriesOptions = {
   model: Model<Api>;
   entries: readonly SessionEntry[];
-  allTools: readonly ToolInfo[];
   grammarToolInputProperties?: GrammarToolInputProperties;
   imageDetail?: ImageDetail;
   nativeAssistantItems?: ReadonlyMap<string, readonly ResponsesOutputItem[]>;
   initialSystemMessage?: SystemMessage;
+  /** Model-visible messages per entry. Defaults to the entry's raw messages. */
+  entryMessages?: (entry: SessionEntry) => AgentMessages;
 };
 
 /**
  * Encode Pi's canonical messages using Pi AI's OpenAI Responses serializer.
  *
+ * The leading system message travels as `instructions`, so it is excluded here.
+ * Later system messages become inline developer items on models that accept
+ * mid-conversation system messages and collapse into the leading message
+ * otherwise, exactly as Pi AI's Codex adapter does.
+ *
  * Tool declarations never enter the input: every request carries the complete
  * current tool set at the top level, as the official Codex client does. Inline
  * `additional_tools` items and synthetic tool-search pairs made the Codex
@@ -180,8 +300,7 @@ function encodeMessages(options: EncodeMessagesOptions): ResponsesInputItem[] {
   const compat = responsesCompatibility(model.compat);
   const serializationOptions: NonNullable<Parameters<typeof convertResponsesMessages>[3]> = {
     includeSystemPrompt: false,
-    includeSystemUpdates: false,
-    supportsMidConvoSystemMessages: true,
+    supportsMidConvoSystemMessages: compat.supportsMidConvoSystemMessages ?? false,
     supportsAdditionalTools: false,
     supportsToolSearch: false,
     grammarToolInputProperties,
@@ -206,21 +325,21 @@ export function encodeSessionEntries(options: EncodeSessionEntriesOptions): Resp
   const {
     model,
     entries,
-    allTools,
     grammarToolInputProperties = new Map(),
     imageDetail = "auto",
     nativeAssistantItems,
+    entryMessages = sessionEntryToContextMessages,
   } = options;
-  const messages = entries.flatMap((entry) => sessionEntryToContextMessages(entry));
+  const messages = entries.flatMap((entry) => entryMessages(entry));
   const encodeOptions: EncodeMessagesOptions = {
     model,
     // A partial tail can start with a system update. Seed its prior state so
-    // the serializer does not mistake that update for the leading declaration.
+    // the serializer renders that update inline instead of dropping it as the
+    // leading system message that `instructions` already carries.
     messages: [
       options.initialSystemMessage ?? { role: "system", content: "", timestamp: 0 },
       ...convertToLlm(messages),
     ],
-    allTools,
     grammarToolInputProperties,
     imageDetail,
   };
@@ -336,14 +455,73 @@ export function checkpointData(
   return checkpoint;
 }
 
+/**
+ * The entries Pi 0.87 omits for overflow or length recovery: the latest
+ * assistant attempt and the tool results of that turn, each targeted by a
+ * `context_edit` omission appended after the attempt. `leafId` is the parent of
+ * the first such omission, so a projection built up to it reflects every edit
+ * that existed before Pi omitted the attempt.
+ *
+ * Pi only selects a recovery attempt while it is still projected, so an
+ * omission of the latest assistant found here is Pi's recovery omission and
+ * not an unrelated edit.
+ */
+function recoveryOmissions(
+  branch: readonly SessionEntry[],
+): { entryIds: ReadonlySet<string>; leafId: string } | undefined {
+  const attemptIndex = branch.findLastIndex(
+    (entry) => entry.type === "message" && entry.message.role === "assistant",
+  );
+  const attempt = branch[attemptIndex];
+  if (attempt?.type !== "message" || attempt.message.role !== "assistant") return undefined;
+
+  const toolCallIds = new Set(
+    attempt.message.content.flatMap((block) => (block.type === "toolCall" ? [block.id] : [])),
+  );
+  const candidates = new Set([attempt.id]);
+  for (const entry of branch.slice(attemptIndex + 1)) {
+    if (entry.type !== "message") continue;
+    if (entry.message.role !== "toolResult" || !toolCallIds.has(entry.message.toolCallId)) break;
+    candidates.add(entry.id);
+  }
+
+  const latestEdits = new Map<string, ContextEditEntry>();
+  for (const entry of branch.slice(attemptIndex + 1)) {
+    if (entry.type === "context_edit" && candidates.has(entry.targetId)) {
+      latestEdits.set(entry.targetId, entry);
+    }
+  }
+  if (latestEdits.get(attempt.id)?.replacement !== null) return undefined;
+  const omissions = new Set(
+    [...latestEdits.values()].filter((entry) => entry.replacement === null),
+  );
+  const first = branch.find((entry) => entry.type === "context_edit" && omissions.has(entry));
+  if (first?.type !== "context_edit" || first.parentId === null) return undefined;
+  return {
+    entryIds: new Set([...omissions].map((entry) => entry.targetId)),
+    leafId: first.parentId,
+  };
+}
+
 /**
  * Materialize provider history for the active branch. Checkpoint history
  * replaces everything before its entry; later branch entries form the tail.
+ *
+ * Every request reads the session projection, so `context_edit` omissions and
+ * replacements apply exactly as they do for Pi's own adapters. Recovery
+ * compaction differs only for the latest attempt: Pi 0.87 omits the failed or
+ * truncated assistant and its tool results through `context_edit` entries
+ * before it runs `session_before_compact`. Those entries are projected as they
+ * were before Pi's omission, so a truncated boundary stays as committed
+ * progress while every earlier omission or replacement remains in force. The
+ * serializer skips error and aborted assistants; a failed attempt contributes
+ * only the native prefix committed before the overflowing subrequest. Entries
+ * are never removed from the branch copy because the omission entry's parent
+ * chain would break and the history would vanish.
  */
 export function providerHistory(options: {
   branch: readonly SessionEntry[];
   wireModel: Model<Api>;
-  allTools: readonly ToolInfo[];
   grammarToolInputProperties?: GrammarToolInputProperties;
   imageDetail?: ImageDetail;
   recoverLatestOverflowPrefix?: boolean;
@@ -351,29 +529,51 @@ export function providerHistory(options: {
   const branch = [...options.branch];
   let recoveredPrefix: ResponsesInputItem[] = [];
   if (options.recoverLatestOverflowPrefix) {
-    const index = branch.findLastIndex(
-      (entry) => entry.type === "message" && entry.message.role === "assistant",
+    const entry = branch.findLast(
+      (candidate) => candidate.type === "message" && candidate.message.role === "assistant",
     );
-    const entry = index >= 0 ? branch[index] : undefined;
     if (
       entry?.type === "message" &&
       entry.message.role === "assistant" &&
-      (entry.message.stopReason === "error" || entry.message.stopReason === "aborted")
+      entry.message.stopReason === "error" &&
+      entry.message.responseId
     ) {
-      if (entry.message.stopReason === "error" && entry.message.responseId) {
-        recoveredPrefix =
-          nativeCommittedPrefixBeforeOverflow(
-            branch,
-            options.wireModel.id,
-            entry.message.responseId,
-          ) ?? [];
+      recoveredPrefix =
+        nativeCommittedPrefixBeforeOverflow(
+          branch,
+          options.wireModel.id,
+          entry.message.responseId,
+        ) ?? [];
+    }
+  }
+
+  const projection = buildSessionProjection(branch);
+  const projected = new Map<string, AgentMessages>();
+  for (const entry of projection.entries) projected.set(entry.sourceEntry.id, entry.messages);
+  const recovery = options.recoverLatestOverflowPrefix ? recoveryOmissions(branch) : undefined;
+  if (recovery) {
+    for (const entry of buildSessionProjection(branch, recovery.leafId).entries) {
+      if (recovery.entryIds.has(entry.sourceEntry.id)) {
+        projected.set(entry.sourceEntry.id, entry.messages);
       }
-      branch.splice(index, 1);
     }
   }
+  const entryMessages = (entry: SessionEntry): AgentMessages => projected.get(entry.id) ?? [];
 
   const checkpoint = searchCheckpoint(branch);
-  const nativeAssistantItems = nativeResponseOverrides(branch, options.wireModel.id);
+  const nativeAssistantItems = new Map(nativeResponseOverrides(branch, options.wireModel.id));
+  for (const entry of branch) {
+    if (
+      entry.type === "message" &&
+      entry.message.role === "assistant" &&
+      entry.message.responseId &&
+      entryMessages(entry).some((message) => message !== entry.message)
+    ) {
+      // A replacement keeps the response ID but changes the model-visible content.
+      // Its original native output must not override the canonical replacement.
+      nativeAssistantItems.delete(entry.message.responseId);
+    }
+  }
   if (checkpoint.kind === "corrupt") {
     throw new Error("The latest Codex compaction checkpoint is corrupt.");
   }
@@ -391,22 +591,22 @@ export function providerHistory(options: {
       ...encodeSessionEntries({
         model: options.wireModel,
         entries: branch.slice(checkpoint.entryIndex + 1),
-        allTools: options.allTools,
         grammarToolInputProperties: options.grammarToolInputProperties ?? new Map(),
         imageDetail: options.imageDetail ?? "auto",
         nativeAssistantItems,
+        entryMessages,
         ...(initialSystemMessage ? { initialSystemMessage } : {}),
       }),
       ...recoveredPrefix,
     ];
   }
 
-  const context = buildSessionContext(branch);
   return [
     ...encodeMessages({
       model: options.wireModel,
-      messages: convertToLlm(context.messages),
-      allTools: options.allTools,
+      messages: convertToLlm(
+        projection.entries.flatMap((entry) => entryMessages(entry.sourceEntry)),
+      ),
       grammarToolInputProperties: options.grammarToolInputProperties ?? new Map(),
       imageDetail: options.imageDetail ?? "auto",
       nativeAssistantItems,

package/extensions/openai-codex-compat/image-generation.ts

@@ -306,7 +306,7 @@ export default function registerImageGeneration(
         },
       });
       const config = resolveConfig(ctx);
-      const history = codexToolHistory(pi, ctx, model, config.imageDetail);
+      const history = codexToolHistory(ctx, model, config.imageDetail);
       const request = await imageRequest(params, history);
       const authentication = await codexToolAuthentication(ctx, model);
       const callId = toolCallId.split("|")[0] || toolCallId;

package/extensions/openai-codex-compat/index.ts

@@ -1,5 +1,6 @@
 import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
 import * as piAi from "@earendil-works/pi-ai";
+import * as piCodingAgent from "@earendil-works/pi-coding-agent";
 import { requirePiTranscriptRuntime } from "./pi-runtime.ts";
 import { DEFAULT_CONFIG, loadConfig, type CodexCompatConfig } from "./config.ts";
 import type { ConfigContext } from "./config-context.ts";
@@ -23,7 +24,7 @@ export {
 } from "./codex-transport.ts";
 
 export default function registerOpenAICodexCompat(pi: ExtensionAPI): void {
-  requirePiTranscriptRuntime(piAi);
+  requirePiTranscriptRuntime(piAi, piCodingAgent);
   let activeConfig: CodexCompatConfig | undefined;
   const resolveConfig = (ctx: ConfigContext): CodexCompatConfig => {
     activeConfig ??= loadConfig(ctx.cwd, ctx.isProjectTrusted());

package/extensions/openai-codex-compat/pi-runtime.ts

@@ -1,19 +1,45 @@
-const TRANSCRIPT_APIS = [
-  "normalizeContext",
-  "getCurrentSystemPrompt",
+/**
+ * Every Pi 0.86 transcript API this extension imports as a value from
+ * `@earendil-works/pi-ai`. Pi's extension loader turns a missing named import
+ * into `undefined` rather than a load error, so an older host fails at first use
+ * unless the surface is checked up front. Keep this list equal to the value
+ * imports in `codex-provider/codex-provider-runtime.ts`, `compaction-checkpoint.ts`,
+ * and `vendor/pi-ai/openai-responses-serialization.ts`.
+ */
+export const TRANSCRIPT_APIS = [
   "getCurrentTools",
   "getDeclaredTools",
   "getInitialSystemMessage",
+  "getSystemMessageText",
+  "normalizeContext",
+  "renderSystemMessageUpdate",
+  "resolveTranscript",
+  "resolveTranscriptTools",
 ] as const;
 
+/**
+ * Every Pi 0.87 session API this extension imports as a value from
+ * `@earendil-works/pi-coding-agent`. Keep this list equal to the value imports
+ * in `compaction-checkpoint.ts`.
+ */
+export const SESSION_APIS = ["buildSessionProjection"] as const;
+
+export const REQUIRED_PI = "Pi 0.87.x";
+
 /** Check the loaded host, not package metadata that PI_PACKAGE_DIR can override. */
-export function requirePiTranscriptRuntime(api: Record<string, unknown>): void {
-  const missing = TRANSCRIPT_APIS.filter((name) => typeof api[name] !== "function");
+export function requirePiTranscriptRuntime(
+  transcriptApi: Record<string, unknown>,
+  sessionApi: Record<string, unknown>,
+): void {
+  const missing = [
+    ...TRANSCRIPT_APIS.filter((name) => typeof transcriptApi[name] !== "function"),
+    ...SESSION_APIS.filter((name) => typeof sessionApi[name] !== "function"),
+  ];
   if (missing.length > 0) {
     throw new Error(
-      "pi-openai-codex-compat requires a running Pi 0.86.x runtime. " +
+      `pi-openai-codex-compat requires a running ${REQUIRED_PI} runtime. ` +
         `Missing host APIs: ${missing.join(", ")}. ` +
-        "Exit Pi and start Pi 0.86.x after updating. /reload cannot upgrade the running " +
+        `Exit Pi and start ${REQUIRED_PI} after updating. /reload cannot upgrade the running ` +
         "runtime, and PI_PACKAGE_DIR can make an older executable report a newer version.",
     );
   }

package/extensions/openai-codex-compat/remote-compaction.ts

@@ -2,7 +2,7 @@ import { isString } from "./value-contracts.ts";
 import type {
   BeforeProviderHeadersEvent,
   CompactionResult,
-  ContextEvent,
+  ContextWithSystemEvent,
   ExtensionAPI,
   ExtensionContext,
   SessionBeforeCompactEvent,
@@ -18,6 +18,8 @@ import { addRemoteCompactionFeature, isObject, type JsonRecord } from "./codex-p
 import { CODEX_API, CODEX_PROVIDER } from "./codex-identifiers.ts";
 import {
   activeResponsesTools,
+  branchInstructions,
+  declaresActiveTools,
   providerHistory,
   remoteCompactionMarkerSummary,
   responsesCompatibility,
@@ -50,9 +52,9 @@ export type RemoteCompactionLifecycleHandler = (
 ) => void;
 
 export type RemoteCompactionContextHandler = (
-  event: Pick<ContextEvent, "messages">,
+  event: Pick<ContextWithSystemEvent, "messages">,
   ctx: RemoteCompactionContext,
-) => { messages: ContextEvent["messages"] } | undefined;
+) => { messages: ContextWithSystemEvent["messages"] } | undefined;
 
 export type RemoteCompactionHeadersHandler = (
   event: Pick<BeforeProviderHeadersEvent, "headers">,
@@ -114,7 +116,12 @@ export function remoteCompactionApi(pi: ExtensionAPI): RemoteCompactionApi {
     getAllTools: () => pi.getAllTools(),
     onBeforeProviderHeaders: (handler) =>
       pi.on("before_provider_headers", (event, ctx) => handler(event, context(ctx))),
-    onContext: (handler) => pi.on("context", (event, ctx) => handler(event, context(ctx))),
+    // The full-transcript event returns messages verbatim. A changed `context`
+    // result would fold every later system message into the leading one, so
+    // `instructions` would no longer be the branch's leading prompt and the
+    // inline developer items in `input` would repeat the folded updates.
+    onContext: (handler) =>
+      pi.on("context_with_system", (event, ctx) => handler(event, context(ctx))),
     onSessionBeforeCompact: (handler) =>
       pi.on("session_before_compact", (event, ctx) => handler(event, context(ctx))),
     onSessionShutdown: (handler) =>
@@ -245,15 +252,26 @@ export default function registerRemoteCompaction(
       const history = providerHistory({
         branch: event.branchEntries,
         wireModel: ctx.model,
-        allTools,
         grammarToolInputProperties,
         imageDetail: config.imageDetail,
         recoverLatestOverflowPrefix: event.reason === "overflow" && event.willRetry,
       });
-      // Compaction rebases history, so use the complete current tool set.
+      // Reuse the last turn's exact declarations when they still describe the
+      // active tools, so the compaction request shares the turn requests'
+      // prompt-cache prefix. Rebuild from the registry when the active set or
+      // any definition (including a parameter schema under an unchanged name)
+      // changed since that turn, or when no turn template exists.
       const template: JsonRecord = { ...matching?.payload };
+      const activeNames = pi.getActiveTools();
+      const cachedTools = Array.isArray(matching?.payload.tools)
+        ? matching.payload.tools.filter(isObject)
+        : undefined;
       template.tools =
-        activeResponsesTools(allTools, pi.getActiveTools(), grammarToolInputProperties) ?? [];
+        cachedTools &&
+        matching &&
+        declaresActiveTools(cachedTools, matching.toolFingerprint, allTools, activeNames)
+          ? cachedTools
+          : (activeResponsesTools(allTools, activeNames, grammarToolInputProperties) ?? []);
       const requestOptions: OpenAICodexResponsesOptions = {
         ...matching?.requestOptions,
         apiKey: authentication.apiKey,
@@ -265,7 +283,14 @@ export default function registerRemoteCompaction(
         model: ctx.model,
         requestOptions,
         history,
-        instructions: instructionsForCompaction(ctx.getSystemPrompt(), event.customInstructions),
+        // Turn requests send the branch's leading system message as `instructions`
+        // and carry later system messages inline in the history. A branch without
+        // a leading system message (a pre-0.86 session) falls back to Pi's
+        // current prompt so the compaction still sees instructions.
+        instructions: instructionsForCompaction(
+          branchInstructions(ctx.model, event.branchEntries) || ctx.getSystemPrompt(),
+          event.customInstructions,
+        ),
         grammarToolInputProperties:
           matching?.grammarToolInputProperties ??
           requestGrammarToolInputProperties(template, allTools),

package/extensions/openai-codex-compat/tool-runtime.ts

@@ -1,4 +1,4 @@
-import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
+import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
 import type { Api, Model, ProviderHeaders } from "@earendil-works/pi-ai";
 import { providerHistory } from "./compaction-checkpoint.ts";
 import type { ImageDetail } from "./config.ts";
@@ -12,7 +12,6 @@ export interface CodexToolAuthentication {
 export type ToolAuthenticationContext = {
   modelRegistry: Pick<ExtensionContext["modelRegistry"], "getApiKeyAndHeaders">;
 };
-export type ToolHistoryApi = Pick<ExtensionAPI, "getAllTools">;
 export type ToolHistoryContext = {
   sessionManager: Pick<ExtensionContext["sessionManager"], "getBranch">;
 };
@@ -32,7 +31,6 @@ export async function codexToolAuthentication(
 }
 
 export function codexToolHistory(
-  pi: ToolHistoryApi,
   ctx: ToolHistoryContext,
   model: Model<Api>,
   imageDetail: ImageDetail = "auto",
@@ -40,7 +38,6 @@ export function codexToolHistory(
   return providerHistory({
     branch: ctx.sessionManager.getBranch(),
     wireModel: model,
-    allTools: pi.getAllTools(),
     imageDetail,
   });
 }

package/extensions/openai-codex-compat/vendor/pi-ai/README.md

@@ -8,14 +8,15 @@ Pi AI's transcript helpers replay system sections and tool declarations.
 `supportsAdditionalTools` and `supportsToolSearch` select the dynamic tool
 history representation for additive `toolsAdded` system messages:
 message-anchored `additional_tools` or a `tool_search_call`/`tool_search_output`
-pair. Tool removal, redefinition, or a model without either capability requires
-the complete current top-level tool list.
-
-Compat uses the extension-owned `includeSystemUpdates: false` option alongside
-`includeSystemPrompt: false`. It sends the complete current prompt separately
-and preserves only supported tool additions in system-message positions.
-This keeps old prompt text from overriding current instructions after
-compaction, forced-prompt projection, reload, or resume.
+pair. The copy keeps both options so the wire-equivalence tests hold, but Compat
+always passes `false`: it declares every tool at the top level.
+
+System messages follow Pi AI exactly. Compat passes `includeSystemPrompt: false`
+because the leading system message travels as `instructions`, and forwards the
+model's `supportsMidConvoSystemMessages` flag so later system messages render
+inline through `renderSystemMessageUpdate` or collapse into the leading message.
+A system message that lands between a tool call and its results is held until
+after those results, as upstream `transformMessages` does.
 
 The optional `namespacedToolNames`, `textContentItemToolResultNames`, and `toolResultImageDetail` paths are extension-owned additions. Without those options, serialization must continue to match Pi AI. `namespacedToolNames` groups only the fixed Codex allowlist into native Responses namespaces and replays namespace/member call identities. `textContentItemToolResultNames` preserves Codex tools such as `web.run` whose successful text output is transported as an `input_text` content-item array instead of Pi's usual plain string. `toolResultImageDetail` overrides the otherwise canonical `auto` detail used for image tool-result content.
 

package/extensions/openai-codex-compat/vendor/pi-ai/openai-responses-serialization.ts

@@ -53,8 +53,6 @@ type ToolResultOutput =
 
 type ConvertResponsesMessagesOptions = {
   includeSystemPrompt?: boolean;
-  /** Compat sends the complete current prompt outside replay history. */
-  includeSystemUpdates?: boolean;
   grammarToolInputProperties?: ReadonlyMap<string, string>;
   supportsMidConvoSystemMessages?: boolean;
   supportsAdditionalTools?: boolean;
@@ -551,25 +549,33 @@ function transformMessages(
   const result: Message[] = [];
   let pendingToolCalls: ToolCall[] = [];
   let existingToolResultIds = new Set<string>();
-  const insertSyntheticToolResults = () => {
-    for (const toolCall of pendingToolCalls) {
-      if (existingToolResultIds.has(toolCall.id)) continue;
-      result.push({
-        role: "toolResult",
-        toolCallId: toolCall.id,
-        toolName: toolCall.name,
-        content: [{ type: "text", text: "No result provided" }],
-        isError: true,
-        timestamp: Date.now(),
-      } satisfies ToolResultMessage);
+  // System messages are transparent to tool-call accounting: one that lands between a
+  // tool call and its results is held back and emitted after the results (synthetic
+  // ones included), so it never causes a duplicate result for a call answered later.
+  const heldSystemMessages: Message[] = [];
+  const closePendingToolCalls = () => {
+    if (pendingToolCalls.length > 0) {
+      for (const toolCall of pendingToolCalls) {
+        if (existingToolResultIds.has(toolCall.id)) continue;
+        result.push({
+          role: "toolResult",
+          toolCallId: toolCall.id,
+          toolName: toolCall.name,
+          content: [{ type: "text", text: "No result provided" }],
+          isError: true,
+          timestamp: Date.now(),
+        } satisfies ToolResultMessage);
+      }
+      pendingToolCalls = [];
+      existingToolResultIds = new Set();
     }
-    pendingToolCalls = [];
-    existingToolResultIds = new Set();
+    result.push(...heldSystemMessages);
+    heldSystemMessages.length = 0;
   };
 
   for (const message of transformed) {
     if (message.role === "assistant") {
-      insertSyntheticToolResults();
+      closePendingToolCalls();
       const assistantMessage = message;
       if (assistantMessage.stopReason === "error" || assistantMessage.stopReason === "aborted") {
         continue;
@@ -585,14 +591,17 @@ function transformMessages(
     } else if (message.role === "toolResult") {
       existingToolResultIds.add(message.toolCallId);
       result.push(message);
+    } else if (message.role === "system") {
+      if (pendingToolCalls.length > 0) heldSystemMessages.push(message);
+      else result.push(message);
     } else if (message.role === "user") {
-      insertSyntheticToolResults();
+      closePendingToolCalls();
       result.push(message);
     } else {
       result.push(message);
     }
   }
-  insertSyntheticToolResults();
+  closePendingToolCalls();
   return result;
 }
 
@@ -737,7 +746,7 @@ export function convertResponsesMessages(
           tools: convertResponsesTools(addedTools, { ...toolOptions, deferLoading: true }),
         });
       }
-      if (isLeadingSystemMessage ? includeSystemPrompt : options?.includeSystemUpdates !== false) {
+      if (!isLeadingSystemMessage || includeSystemPrompt) {
         const text = isLeadingSystemMessage
           ? getSystemMessageText(message)
           : renderSystemMessageUpdate(message);

package/extensions/openai-codex-compat/web-run.ts

@@ -176,7 +176,7 @@ export default function registerWebRun(
       }
       const authentication = await codexToolAuthentication(ctx, model);
       const config = resolveConfig(ctx);
-      const history = codexToolHistory(pi, ctx, model, config.imageDetail);
+      const history = codexToolHistory(ctx, model, config.imageDetail);
       const input = recentSearchInput(history);
       const body: JsonRecord = {
         id: ctx.sessionManager.getSessionId(),

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-openai-codex-compat",
-  "version": "0.0.13",
+  "version": "0.0.15",
   "description": "OpenAI Codex compatibility for Pi with native compaction, fast mode, and Codex-optimized capabilities",
   "keywords": [
     "pi-package"
@@ -51,9 +51,9 @@
   "devDependencies": {
     "@2h2d/oxlint-config": "0.1.1",
     "@2h2d/ts-config": "0.0.1",
-    "@earendil-works/pi-ai": "0.86.1",
-    "@earendil-works/pi-coding-agent": "0.86.1",
-    "@earendil-works/pi-tui": "0.86.1",
+    "@earendil-works/pi-ai": "0.87.0",
+    "@earendil-works/pi-coding-agent": "0.87.0",
+    "@earendil-works/pi-tui": "0.87.0",
     "@types/node": "22.20.1",
     "oxfmt": "0.64.0",
     "oxlint": "1.79.0",
@@ -62,9 +62,9 @@
     "typescript": "7.0.2"
   },
   "peerDependencies": {
-    "@earendil-works/pi-ai": ">=0.86.0 <0.87.0",
-    "@earendil-works/pi-coding-agent": ">=0.86.0 <0.87.0",
-    "@earendil-works/pi-tui": ">=0.86.0 <0.87.0",
+    "@earendil-works/pi-ai": ">=0.87.0 <0.88.0",
+    "@earendil-works/pi-coding-agent": ">=0.87.0 <0.88.0",
+    "@earendil-works/pi-tui": ">=0.87.0 <0.88.0",
     "typebox": "*"
   },
   "engines": {