pi-openai-codex-compat 0.0.13 → 0.0.14

package/CHANGELOG.md

@@ -2,6 +2,45 @@
 
 ## Unreleased
 
+## 0.0.14 - 2026-09-22
+
+### Changed
+
+- Require Pi `>=0.87.0 <0.88.0` and use Pi 0.87.0 as the development and
+  packaged-CLI integration baseline. The load-time host check now also
+  requires Pi 0.87's session projection API.
+- Honour Pi 0.87 `context_edit` entries when building request history from
+  the branch, so omitted or replaced messages match what Pi's own adapters
+  send.
+- Preserve mid-conversation prompt updates when filtering compaction summaries.
+- Skip percentage compaction when nothing has been sampled yet. Pi 0.87
+  estimates the prompt and tool declarations before the first response, so a
+  low threshold could trigger an empty compaction request.
+- Send Pi's leading system message as the prompt and later system messages as
+  inline developer items, exactly as Pi AI's Codex adapter does. Previously
+  every system message was replayed into one `instructions` string, so any
+  prompt-section change rewrote the stable prefix. Models without
+  mid-conversation system-message support still receive one collapsed prompt.
+- Keep every developer and system message in the native compaction checkpoint,
+  in place and outside the 64k retained-context budget, so prompt updates
+  survive compaction. Only user messages are charged to the budget.
+
+### Fixed
+
+- Keep the conversation in overflow-recovery compaction on Pi 0.87. Pi now
+  records the failed attempt's omission as a session entry whose parent is
+  that attempt; removing the attempt from the branch copy broke the entry
+  chain and the compaction request contained only the recovered prefix.
+- Hold a system message that lands between a tool call and its results until
+  after those results in replayed history, matching Pi AI.
+- Send the branch's leading system message as compaction `instructions`
+  instead of Pi's fully replayed prompt, so Pi-driven compaction shares the
+  turn requests' prompt prefix.
+- Reuse the last turn's exact tool declarations for `/compact` and threshold
+  compaction while they still describe the active tools. Rebuilding them from
+  the tool registry lost Pi's strict schemas for built-in tools, so every
+  Pi-driven compaction request missed the turn requests' prompt-cache prefix.
+
 ## 0.0.13 - 2026-09-21
 
 ### Fixed

package/README.md

@@ -22,7 +22,7 @@ Pi provides the Codex OAuth flow and model catalog. At session start, this packa
 ## Requirements
 
 - Node.js 22.19 or newer
-- Pi `>=0.86.0 <0.87.0`
+- Pi `>=0.87.0 <0.88.0`
 - An OpenAI Codex login in Pi
 
 Restart Pi after upgrading its runtime. `/reload` reloads extensions but cannot
@@ -67,7 +67,7 @@ The compatibility baseline is official Codex CLI `0.149.1`, released August 24,
 | Session storage             | Pi remains the canonical session owner. Opaque Codex compaction checkpoints are stored in Pi `compaction` entries, and otherwise lossy Responses output is stored in sparse custom native-response entries. Official Codex owns a rollout/thread store directly.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
 | Branching                   | Checkpoints and native response overrides follow Pi's active session branch. Official Codex uses its own thread, turn, rollback, fork, and context-window lineage.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
 | Model switching             | This package rejects model switches while the active Pi branch contains a native Codex checkpoint because the checkpoint is model-specific.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
-| System instructions         | Replay Pi's system-message sections into the complete current prompt. Responses Lite models prepend it as developer input after `additional_tools`; other models send it through Responses `instructions`. Do not replay older Pi system-prompt text alongside that current prompt. `/reload`, forced prompts, compaction snapshots, and resumed sessions use the current state without rewriting old checkpoints.                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+| System instructions         | Send the leading Pi system message as the prompt, as Pi AI's Codex adapter does: Responses Lite models prepend it as developer input after `additional_tools`; other models send it through Responses `instructions`. Later Pi system messages (`/reload`, section changes) travel inline as developer items on models that accept mid-conversation system messages and collapse into the leading prompt otherwise. A forced prompt replaces the leading message. Compaction snapshots and resumed sessions lead with Pi's replayed prompt without rewriting old checkpoints.                                                                                                                                                                                                                                                                                                                    |
 | Dynamic tool declarations   | Send the complete current tool set, replayed from Pi's `toolsAdded` and `toolsRemoved` system messages, in the top-level `tools` field on every request, as the official Codex client does. Tool declarations never travel inside `input`: inline `additional_tools` items and synthetic tool-search pairs made the Codex backend unreliable about which tools exist. A mid-session tool change costs one prompt-cache miss, and native checkpoints need no declaration rebase.                                                                                                                                                                                                                                                                                                                                                                                                                  |
 | Turn metadata               | Requests send a persisted installation id plus Pi-derived session, thread, context-window, turn, source, sandbox, request-kind, and nested compaction-operation metadata in `client_metadata` and compatible headers. The in-memory context-window number advances after successful compaction. One turn id is reused throughout a Pi agent run, while prewarm has its own id. First-party requests also carry Codex's model-and-tier routing hint. The provider captures the server-issued `x-codex-turn-state` once per agent run, replays it on WebSocket retries, SSE requests, and WebSocket-to-SSE fallback, and records all identity values in transport diagnostics. Pi does not reconstruct prior window number after extension reload/session resume or reproduce workspace Git/parent/subagent/Code Mode metadata. Each marked Pi tree branch receives its own persisted thread UUID. |
 | Cache preparation           | Before the first cache-enabled WebSocket turn, the package prewarms only the stable instruction/tool prefix: ordinary Responses uses empty `input`, while Responses Lite uses `additional_tools` plus the developer instructions. The first generated request then contributes only dynamic conversation input to the continuation. No explicit prompt-cache breakpoints are added.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
@@ -76,7 +76,7 @@ The compatibility baseline is official Codex CLI `0.149.1`, released August 24,
 | Transport recovery          | WebSocket failures before model-visible output receive five fresh-connection retries before sticky SSE fallback; SSE transport failures receive five retries. Official Codex `0.149.1` separately retries sampling connection-establishment failures indefinitely with delays capped at 60 seconds. The package remains bounded because Fetch does not expose Reqwest's narrower connection-error category, and a phase-only approximation could indefinitely repeat requests that reached the server.                                                                                                                                                                                                                                                                                                                                                                                           |
 | Compaction lifecycle events | Pre-turn percentage compaction writes through Pi's mutable session manager and cannot emit Pi's internal `session_compact` event through the public extension API. Mid-response percentage boundaries and manual, threshold, or overflow compactions initiated by Pi emit the normal lifecycle.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
 | Header hooks                | An internal percentage-compaction request reuses the already transformed provider headers. It cannot independently rerun Pi's `before_provider_headers` hook.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
-| Native retained context     | Deliberately differs from current Codex. The package retains recent user/developer/system messages under a text-only 64k budget before the opaque compaction item. Current Codex applies a second installed-history filter that drops developer/system wrappers and non-real-user messages, can retain eligible structured agent commentary, and trims oversized function outputs before compaction. Codex `0.149.1` also adds a default-disabled mode that charges retained images to the budget and keeps each image with adjacent harness labels as an atomic boundary unit. Pi keeps its existing checkpoint shape by design.                                                                                                                                                                                                                                                                |
+| Native retained context     | Deliberately differs from current Codex. The package retains recent user messages under a text-only 64k budget before the opaque compaction item, and keeps every developer and system message in place outside that budget so prompt updates are never compacted away. Current Codex applies a second installed-history filter that drops developer/system wrappers and non-real-user messages, can retain eligible structured agent commentary, and trims oversized function outputs before compaction. Codex `0.149.1` also adds a default-disabled mode that charges retained images to the budget and keeps each image with adjacent harness labels as an atomic boundary unit. Pi keeps its existing checkpoint shape by design.                                                                                                                                                           |
 | Tool namespaces             | Responses Lite groups Pi's ordinary function/custom declarations into upstream's canonical `functions` namespace and maps that default namespace back to bare Pi names. Pi registers dotted names such as `web.run` as exact flat identifiers, so the provider converts only the fixed extension-owned allowlist into non-default Responses namespace/member identities and rejects unknown or ambiguously flat namespaced calls.                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
 | Capability gating           | Tool activation is based on the selected `openai-codex` provider plus package settings. It does not reproduce every official model-metadata, plan, feature-stage, executor, mode, or account gate.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
 | Sandbox and approvals       | Pi extensions run with full process permissions. Command tools, `apply_patch`, local image reads, generated-image writes, and sibling Codex endpoints do not use Codex's sandbox, permission-profile, environment, or approval lifecycle.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
@@ -327,7 +327,8 @@ The extension handles native compaction for `openai-codex`. It follows the Codex
 
 1. Send normal Responses history followed by `{ "type": "compaction_trigger" }`.
 2. Validate the returned opaque `compaction` item.
-3. Retain approximately 64,000 tokens of recent user, developer, and system context.
+3. Retain approximately 64,000 tokens of recent user messages, plus every developer and
+   system message outside that budget.
 4. Persist the opaque checkpoint in the Pi session and replay it on later requests.
 
 Ordinary responses and compaction use the same extension-managed SSE/WebSocket transport. Before the first WebSocket turn for a session and model, the provider performs a best-effort v2 `generate: false` prewarm of the static instruction/tool prefix, then generates the dynamic conversation input from its continuation. With `responsesLite: false`, supported models use the ordinary Responses envelope and receive their own ordinary-prefix prewarm.
@@ -587,7 +588,7 @@ mise run test:live:codex
 The task obtains the local Codex bearer token and runs the tests with
 `gpt-5.6-luna` at medium reasoning effort.
 It also packs the extension and loads that archive through the shipped Pi
-0.86.1 CLI. Ordinary Responses and Responses Lite tests exercise tool calls,
+0.87.0 CLI. Ordinary Responses and Responses Lite tests exercise tool calls,
 prompt reload, native compaction, and persisted-session resume against Codex.
 Test credentials stay in memory and child-process environments. Test sessions
 and configuration are isolated from the user's agent directory.

package/extensions/openai-codex-compat/codex-protocol.ts

@@ -276,17 +276,23 @@ function shortenMessage(
     : undefined;
 }
 
-function retainedRole(item: ResponsesInputItem): boolean {
-  return (
-    Value.Check(RESPONSES_MESSAGE_ITEM_SCHEMA, item) &&
-    (item.role === "user" || item.role === "developer" || item.role === "system")
-  );
+function messageRole(item: ResponsesInputItem): string | undefined {
+  return Value.Check(RESPONSES_MESSAGE_ITEM_SCHEMA, item) ? item.role : undefined;
+}
+
+/** Developer and system messages carry prompt state, not conversation history. */
+function isInstructionMessage(item: ResponsesInputItem): boolean {
+  const role = messageRole(item);
+  return role === "developer" || role === "system";
 }
 
 /**
- * Reproduce Codex remote-compaction-v2 history installation: select recent
- * user/developer/system messages from newest to oldest, then restore their
- * chronological order. The oldest selected message may be truncated.
+ * Reproduce Codex remote-compaction-v2 history installation for user messages:
+ * select recent user messages from newest to oldest under the budget, then
+ * restore chronological order. The oldest selected user message may be
+ * truncated. Developer and system messages are prompt updates rather than
+ * conversation history: every one of them is retained in place, outside the
+ * budget, and never truncated.
  */
 export function selectRetainedContext(
   history: readonly ResponsesInputItem[],
@@ -295,10 +301,14 @@ export function selectRetainedContext(
   let remaining = budget;
   const newestFirst: ResponsesInputItem[] = [];
 
-  for (let index = history.length - 1; index >= 0 && remaining > 0; index--) {
+  for (let index = history.length - 1; index >= 0; index--) {
     const item = history[index];
     if (item === undefined) continue;
-    if (!retainedRole(item)) continue;
+    if (isInstructionMessage(item)) {
+      newestFirst.push(structuredClone(item));
+      continue;
+    }
+    if (remaining <= 0 || messageRole(item) !== "user") continue;
 
     const tokens = Math.max(1, messageTextTokens(item));
     if (tokens <= remaining) {

package/extensions/openai-codex-compat/codex-provider/codex-provider-history.ts

@@ -1,4 +1,4 @@
-import type { SessionEntry, ToolInfo } from "@earendil-works/pi-coding-agent";
+import type { SessionEntry } from "@earendil-works/pi-coding-agent";
 import type { Api, Model } from "@earendil-works/pi-ai";
 import { providerHistory, type GrammarToolInputProperties } from "../compaction-checkpoint.ts";
 import type { ImageDetail } from "../config.ts";
@@ -37,7 +37,6 @@ export function splitUnsampledUserInput(options: {
   branch: readonly SessionEntry[];
   history: readonly ResponsesInputItem[];
   model: Model<Api>;
-  allTools: readonly ToolInfo[];
   grammarToolInputProperties: GrammarToolInputProperties;
   imageDetail: ImageDetail;
 }):
@@ -56,7 +55,6 @@ export function splitUnsampledUserInput(options: {
   const encoded = providerHistory({
     branch: options.branch.slice(unsampledIndex),
     wireModel: options.model,
-    allTools: options.allTools,
     grammarToolInputProperties: options.grammarToolInputProperties,
     imageDetail: options.imageDetail,
   });

package/extensions/openai-codex-compat/codex-provider/codex-provider-runtime.ts

@@ -5,10 +5,8 @@ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
 import {
   clampThinkingLevel,
   createAssistantMessageEventStream,
-  getCurrentSystemPrompt,
   getCurrentTools,
   getDeclaredTools,
-  normalizeContext,
   type Api,
   type AssistantMessage,
   type AssistantMessageEventStream,
@@ -30,6 +28,8 @@ import {
   checkpointData,
   providerHistory,
   remoteCompactionMarkerSummary,
+  requestInstructions,
+  resolveRequestTranscript,
   responsesCompatibility,
   searchCheckpoint,
   type CheckpointData,
@@ -502,8 +502,7 @@ export class CodexProviderRuntime {
       const nativeItems = new Map<string, ResponsesOutputItem[]>();
       const serializationOptions: NonNullable<Parameters<typeof convertResponsesMessages>[3]> = {
         includeSystemPrompt: false,
-        includeSystemUpdates: false,
-        supportsMidConvoSystemMessages: true,
+        supportsMidConvoSystemMessages: compat.supportsMidConvoSystemMessages ?? false,
         supportsAdditionalTools: false,
         supportsToolSearch: false,
         grammarToolInputProperties,
@@ -524,7 +523,6 @@ export class CodexProviderRuntime {
     return providerHistory({
       branch: scope.manager.getBranch(),
       wireModel: model,
-      allTools: this.pi.getAllTools(),
       grammarToolInputProperties,
       imageDetail: scope.config.imageDetail,
     });
@@ -547,11 +545,14 @@ export class CodexProviderRuntime {
     // checkpoint drops such history anyway. A mid-session tool change costs one
     // cache miss.
     const requestTools = getCurrentTools(context.messages);
+    // The leading system message is the prompt; later system messages travel
+    // inline in `input` (or collapsed into the leading message by
+    // resolveRequestTranscript), as Pi AI's Codex adapter does.
     let body: JsonRecord = {
       model: model.id,
       store: false,
       stream: true,
-      instructions: getCurrentSystemPrompt(context.messages) || "You are a helpful assistant.",
+      instructions: requestInstructions(context) || "You are a helpful assistant.",
       input: this.wireHistory(model, context, grammarToolInputProperties, runtimeSessionId),
       text: { verbosity: requestOptions.textVerbosity ?? "low" },
       include: ["reasoning.encrypted_content"],
@@ -735,7 +736,6 @@ export class CodexProviderRuntime {
       branch,
       history,
       model,
-      allTools: this.pi.getAllTools(),
       grammarToolInputProperties,
       imageDetail: scope.config.imageDetail,
     });
@@ -746,6 +746,10 @@ export class CodexProviderRuntime {
       );
       return body;
     }
+    // Pi 0.87 estimates the prompt and tool declarations before any response
+    // usage exists, so a low threshold can be reached before anything was
+    // sampled. There is nothing to compact until the first response.
+    if (split.history.length === 0) return body;
 
     const compacted = await this.performCompaction({
       model,
@@ -755,7 +759,7 @@ export class CodexProviderRuntime {
       template: withoutConversationInput(body),
       instructions: isString(body.instructions)
         ? body.instructions
-        : getCurrentSystemPrompt(context.messages) || "You are a helpful assistant.",
+        : requestInstructions(context) || "You are a helpful assistant.",
       grammarToolInputProperties,
       priority: scope.config.fastMode,
       compactionMetadata: responsesCompactionV2Metadata("auto", "context_limit", "pre_turn"),
@@ -797,7 +801,7 @@ export class CodexProviderRuntime {
     input: Context,
     options?: OpenAICodexResponsesOptions,
   ): AssistantMessageEventStream {
-    const context = normalizeContext(input);
+    const context = resolveRequestTranscript(model, input);
     const stream = createAssistantMessageEventStream();
     const requestOptions = transportOptions(options);
     const output: AssistantMessage = {

package/extensions/openai-codex-compat/compaction-checkpoint.ts

@@ -2,12 +2,24 @@ import { isBoolean, isString } from "./value-contracts.ts";
 import { randomUUID } from "node:crypto";
 import {
   buildSessionContext,
+  buildSessionProjection,
   convertToLlm,
   sessionEntryToContextMessages,
   type SessionEntry,
   type ToolInfo,
 } from "@earendil-works/pi-coding-agent";
-import type { Api, Message, Model, SystemMessage } from "@earendil-works/pi-ai";
+import {
+  getInitialSystemMessage,
+  getSystemMessageText,
+  normalizeContext,
+  resolveTranscript,
+  type Api,
+  type Context,
+  type Message,
+  type Model,
+  type SystemMessage,
+  type TranscriptContext,
+} from "@earendil-works/pi-ai";
 import { Value } from "typebox/value";
 import { APPLY_PATCH_LARK_GRAMMAR, APPLY_PATCH_TOOL_NAME } from "./apply-patch.ts";
 import { CODEX_TOOL_CALL_PROVIDERS } from "./codex-identifiers.ts";
@@ -34,6 +46,8 @@ import {
 import type { ResponsesToolDefinition } from "./responses-tool-schema.ts";
 import { convertResponsesMessages } from "./vendor/pi-ai/openai-responses-serialization.ts";
 
+type AgentMessages = ReturnType<typeof sessionEntryToContextMessages>;
+
 export const CHECKPOINT_ENTRY_TYPE = "openai-codex-compat-remote-compaction";
 export const CHECKPOINT_FORMAT_VERSION = 1;
 
@@ -58,17 +72,16 @@ export type CheckpointSearch =
 export type GrammarToolInputProperties = ReadonlyMap<string, string>;
 
 export interface ResponsesCompatibility {
-  supportsAdditionalTools?: boolean;
+  supportsMidConvoSystemMessages?: boolean;
   supportsOpenAIGrammarTools?: boolean;
   supportsStrictMode?: boolean;
-  supportsToolSearch?: boolean;
 }
 
 export function responsesCompatibility(value: unknown): ResponsesCompatibility {
   const compatibility: ResponsesCompatibility = {};
   if (!isObject(value)) return compatibility;
-  if (isBoolean(value["supportsAdditionalTools"])) {
-    compatibility.supportsAdditionalTools = value["supportsAdditionalTools"];
+  if (isBoolean(value["supportsMidConvoSystemMessages"])) {
+    compatibility.supportsMidConvoSystemMessages = value["supportsMidConvoSystemMessages"];
   }
   if (isBoolean(value["supportsOpenAIGrammarTools"])) {
     compatibility.supportsOpenAIGrammarTools = value["supportsOpenAIGrammarTools"];
@@ -76,12 +89,37 @@ export function responsesCompatibility(value: unknown): ResponsesCompatibility {
   if (isBoolean(value["supportsStrictMode"])) {
     compatibility.supportsStrictMode = value["supportsStrictMode"];
   }
-  if (isBoolean(value["supportsToolSearch"])) {
-    compatibility.supportsToolSearch = value["supportsToolSearch"];
-  }
   return compatibility;
 }
 
+/**
+ * Resolve Pi's transcript the way Pi AI's Codex adapter does before building a
+ * request: models that accept mid-conversation system messages keep later
+ * system messages in place, and every other model collapses them into the
+ * leading system message.
+ */
+export function resolveRequestTranscript(model: Model<Api>, context: Context): TranscriptContext {
+  return resolveTranscript(
+    normalizeContext(context),
+    responsesCompatibility(model.compat).supportsMidConvoSystemMessages ?? false,
+  );
+}
+
+/** The leading system message of a resolved transcript, rendered as Responses `instructions`. */
+export function requestInstructions(context: TranscriptContext): string {
+  const initial = getInitialSystemMessage(context.messages);
+  return initial ? getSystemMessageText(initial) : "";
+}
+
+/** `instructions` for a request built from a session branch rather than Pi's live transcript. */
+export function branchInstructions(model: Model<Api>, branch: readonly SessionEntry[]): string {
+  return requestInstructions(
+    resolveRequestTranscript(model, {
+      messages: convertToLlm(buildSessionContext([...branch]).messages),
+    }),
+  );
+}
+
 function responsesToolParameters(tool: ToolInfo): JsonRecord {
   if (!isObject(tool.parameters)) {
     throw new Error(`Tool ${tool.name} must have JSON object parameters.`);
@@ -143,6 +181,43 @@ export function activeResponsesTools(
     : undefined;
 }
 
+/**
+ * Whether a serialized Responses `tools` list declares exactly the active tools,
+ * by name and description. Namespace members count as `namespace.member`.
+ *
+ * A cached turn template carries the exact declarations the turn requests sent,
+ * including strict schema conversion that `ToolInfo` cannot reproduce. Reusing
+ * it keeps the compaction request on the turn requests' prompt-cache prefix.
+ */
+export function declaresActiveTools(
+  serialized: readonly JsonRecord[],
+  allTools: readonly ToolInfo[],
+  activeNames: readonly string[],
+): boolean {
+  const declared = new Map<string, string | undefined>();
+  for (const item of serialized) {
+    if (item.type === "namespace" && isString(item.name) && Array.isArray(item.tools)) {
+      for (const member of item.tools) {
+        if (!isObject(member) || !isString(member.name)) return false;
+        declared.set(
+          `${item.name}.${member.name}`,
+          isString(member["description"]) ? member["description"] : undefined,
+        );
+      }
+    } else if ((item.type === "function" || item.type === "custom") && isString(item.name)) {
+      declared.set(item.name, isString(item["description"]) ? item["description"] : undefined);
+    } else {
+      return false;
+    }
+  }
+  const enabled = new Set(activeNames);
+  const active = allTools.filter((tool) => enabled.has(tool.name));
+  return (
+    declared.size === active.length &&
+    active.every((tool) => declared.has(tool.name) && declared.get(tool.name) === tool.description)
+  );
+}
+
 export function remoteCompactionMarkerSummary(): string {
   return `OpenAI Codex remote compaction checkpoint (${randomUUID()}).`;
 }
@@ -150,7 +225,6 @@ export function remoteCompactionMarkerSummary(): string {
 type EncodeMessagesOptions = {
   model: Model<Api>;
   messages: Message[];
-  allTools: readonly ToolInfo[];
   grammarToolInputProperties: GrammarToolInputProperties;
   imageDetail: ImageDetail;
   nativeAssistantItems?: ReadonlyMap<string, readonly ResponsesOutputItem[]>;
@@ -159,16 +233,22 @@ type EncodeMessagesOptions = {
 export type EncodeSessionEntriesOptions = {
   model: Model<Api>;
   entries: readonly SessionEntry[];
-  allTools: readonly ToolInfo[];
   grammarToolInputProperties?: GrammarToolInputProperties;
   imageDetail?: ImageDetail;
   nativeAssistantItems?: ReadonlyMap<string, readonly ResponsesOutputItem[]>;
   initialSystemMessage?: SystemMessage;
+  /** Model-visible messages per entry. Defaults to the entry's raw messages. */
+  entryMessages?: (entry: SessionEntry) => AgentMessages;
 };
 
 /**
  * Encode Pi's canonical messages using Pi AI's OpenAI Responses serializer.
  *
+ * The leading system message travels as `instructions`, so it is excluded here.
+ * Later system messages become inline developer items on models that accept
+ * mid-conversation system messages and collapse into the leading message
+ * otherwise, exactly as Pi AI's Codex adapter does.
+ *
  * Tool declarations never enter the input: every request carries the complete
  * current tool set at the top level, as the official Codex client does. Inline
  * `additional_tools` items and synthetic tool-search pairs made the Codex
@@ -180,8 +260,7 @@ function encodeMessages(options: EncodeMessagesOptions): ResponsesInputItem[] {
   const compat = responsesCompatibility(model.compat);
   const serializationOptions: NonNullable<Parameters<typeof convertResponsesMessages>[3]> = {
     includeSystemPrompt: false,
-    includeSystemUpdates: false,
-    supportsMidConvoSystemMessages: true,
+    supportsMidConvoSystemMessages: compat.supportsMidConvoSystemMessages ?? false,
     supportsAdditionalTools: false,
     supportsToolSearch: false,
     grammarToolInputProperties,
@@ -206,21 +285,21 @@ export function encodeSessionEntries(options: EncodeSessionEntriesOptions): Resp
   const {
     model,
     entries,
-    allTools,
     grammarToolInputProperties = new Map(),
     imageDetail = "auto",
     nativeAssistantItems,
+    entryMessages = sessionEntryToContextMessages,
   } = options;
-  const messages = entries.flatMap((entry) => sessionEntryToContextMessages(entry));
+  const messages = entries.flatMap((entry) => entryMessages(entry));
   const encodeOptions: EncodeMessagesOptions = {
     model,
     // A partial tail can start with a system update. Seed its prior state so
-    // the serializer does not mistake that update for the leading declaration.
+    // the serializer renders that update inline instead of dropping it as the
+    // leading system message that `instructions` already carries.
     messages: [
       options.initialSystemMessage ?? { role: "system", content: "", timestamp: 0 },
       ...convertToLlm(messages),
     ],
-    allTools,
     grammarToolInputProperties,
     imageDetail,
   };
@@ -339,11 +418,19 @@ export function checkpointData(
 /**
  * Materialize provider history for the active branch. Checkpoint history
  * replaces everything before its entry; later branch entries form the tail.
+ *
+ * Ordinary requests read the session projection, so `context_edit` omissions
+ * and replacements apply exactly as they do for Pi's own adapters. Recovery
+ * compaction reads the raw entries instead: Pi 0.87 omits the failed or
+ * truncated attempt through a `context_edit` child of that entry before it
+ * runs `session_before_compact`. The serializer already skips error and
+ * aborted assistants, and a truncated boundary is committed progress that the
+ * checkpoint must keep. Entries are never removed from the branch copy because
+ * the omission entry's parent chain would break and the history would vanish.
  */
 export function providerHistory(options: {
   branch: readonly SessionEntry[];
   wireModel: Model<Api>;
-  allTools: readonly ToolInfo[];
   grammarToolInputProperties?: GrammarToolInputProperties;
   imageDetail?: ImageDetail;
   recoverLatestOverflowPrefix?: boolean;
@@ -351,27 +438,32 @@ export function providerHistory(options: {
   const branch = [...options.branch];
   let recoveredPrefix: ResponsesInputItem[] = [];
   if (options.recoverLatestOverflowPrefix) {
-    const index = branch.findLastIndex(
-      (entry) => entry.type === "message" && entry.message.role === "assistant",
+    const entry = branch.findLast(
+      (candidate) => candidate.type === "message" && candidate.message.role === "assistant",
     );
-    const entry = index >= 0 ? branch[index] : undefined;
     if (
       entry?.type === "message" &&
       entry.message.role === "assistant" &&
-      (entry.message.stopReason === "error" || entry.message.stopReason === "aborted")
+      entry.message.stopReason === "error" &&
+      entry.message.responseId
     ) {
-      if (entry.message.stopReason === "error" && entry.message.responseId) {
-        recoveredPrefix =
-          nativeCommittedPrefixBeforeOverflow(
-            branch,
-            options.wireModel.id,
-            entry.message.responseId,
-          ) ?? [];
-      }
-      branch.splice(index, 1);
+      recoveredPrefix =
+        nativeCommittedPrefixBeforeOverflow(
+          branch,
+          options.wireModel.id,
+          entry.message.responseId,
+        ) ?? [];
     }
   }
 
+  const projection = buildSessionProjection(branch);
+  const projected = new Map<string, AgentMessages>();
+  for (const entry of projection.entries) projected.set(entry.sourceEntry.id, entry.messages);
+  const entryMessages = (entry: SessionEntry): AgentMessages =>
+    options.recoverLatestOverflowPrefix
+      ? sessionEntryToContextMessages(entry)
+      : (projected.get(entry.id) ?? []);
+
   const checkpoint = searchCheckpoint(branch);
   const nativeAssistantItems = nativeResponseOverrides(branch, options.wireModel.id);
   if (checkpoint.kind === "corrupt") {
@@ -391,22 +483,22 @@ export function providerHistory(options: {
       ...encodeSessionEntries({
         model: options.wireModel,
         entries: branch.slice(checkpoint.entryIndex + 1),
-        allTools: options.allTools,
         grammarToolInputProperties: options.grammarToolInputProperties ?? new Map(),
         imageDetail: options.imageDetail ?? "auto",
         nativeAssistantItems,
+        entryMessages,
         ...(initialSystemMessage ? { initialSystemMessage } : {}),
       }),
       ...recoveredPrefix,
     ];
   }
 
-  const context = buildSessionContext(branch);
   return [
     ...encodeMessages({
       model: options.wireModel,
-      messages: convertToLlm(context.messages),
-      allTools: options.allTools,
+      messages: convertToLlm(
+        projection.entries.flatMap((entry) => entryMessages(entry.sourceEntry)),
+      ),
       grammarToolInputProperties: options.grammarToolInputProperties ?? new Map(),
       imageDetail: options.imageDetail ?? "auto",
       nativeAssistantItems,

package/extensions/openai-codex-compat/image-generation.ts

@@ -306,7 +306,7 @@ export default function registerImageGeneration(
         },
       });
       const config = resolveConfig(ctx);
-      const history = codexToolHistory(pi, ctx, model, config.imageDetail);
+      const history = codexToolHistory(ctx, model, config.imageDetail);
       const request = await imageRequest(params, history);
       const authentication = await codexToolAuthentication(ctx, model);
       const callId = toolCallId.split("|")[0] || toolCallId;

package/extensions/openai-codex-compat/index.ts

@@ -1,5 +1,6 @@
 import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
 import * as piAi from "@earendil-works/pi-ai";
+import * as piCodingAgent from "@earendil-works/pi-coding-agent";
 import { requirePiTranscriptRuntime } from "./pi-runtime.ts";
 import { DEFAULT_CONFIG, loadConfig, type CodexCompatConfig } from "./config.ts";
 import type { ConfigContext } from "./config-context.ts";
@@ -23,7 +24,7 @@ export {
 } from "./codex-transport.ts";
 
 export default function registerOpenAICodexCompat(pi: ExtensionAPI): void {
-  requirePiTranscriptRuntime(piAi);
+  requirePiTranscriptRuntime(piAi, piCodingAgent);
   let activeConfig: CodexCompatConfig | undefined;
   const resolveConfig = (ctx: ConfigContext): CodexCompatConfig => {
     activeConfig ??= loadConfig(ctx.cwd, ctx.isProjectTrusted());

package/extensions/openai-codex-compat/pi-runtime.ts

@@ -1,19 +1,45 @@
-const TRANSCRIPT_APIS = [
-  "normalizeContext",
-  "getCurrentSystemPrompt",
+/**
+ * Every Pi 0.86 transcript API this extension imports as a value from
+ * `@earendil-works/pi-ai`. Pi's extension loader turns a missing named import
+ * into `undefined` rather than a load error, so an older host fails at first use
+ * unless the surface is checked up front. Keep this list equal to the value
+ * imports in `codex-provider/codex-provider-runtime.ts`, `compaction-checkpoint.ts`,
+ * and `vendor/pi-ai/openai-responses-serialization.ts`.
+ */
+export const TRANSCRIPT_APIS = [
   "getCurrentTools",
   "getDeclaredTools",
   "getInitialSystemMessage",
+  "getSystemMessageText",
+  "normalizeContext",
+  "renderSystemMessageUpdate",
+  "resolveTranscript",
+  "resolveTranscriptTools",
 ] as const;
 
+/**
+ * Every Pi 0.87 session API this extension imports as a value from
+ * `@earendil-works/pi-coding-agent`. Keep this list equal to the value imports
+ * in `compaction-checkpoint.ts`.
+ */
+export const SESSION_APIS = ["buildSessionProjection"] as const;
+
+export const REQUIRED_PI = "Pi 0.87.x";
+
 /** Check the loaded host, not package metadata that PI_PACKAGE_DIR can override. */
-export function requirePiTranscriptRuntime(api: Record<string, unknown>): void {
-  const missing = TRANSCRIPT_APIS.filter((name) => typeof api[name] !== "function");
+export function requirePiTranscriptRuntime(
+  transcriptApi: Record<string, unknown>,
+  sessionApi: Record<string, unknown>,
+): void {
+  const missing = [
+    ...TRANSCRIPT_APIS.filter((name) => typeof transcriptApi[name] !== "function"),
+    ...SESSION_APIS.filter((name) => typeof sessionApi[name] !== "function"),
+  ];
   if (missing.length > 0) {
     throw new Error(
-      "pi-openai-codex-compat requires a running Pi 0.86.x runtime. " +
+      `pi-openai-codex-compat requires a running ${REQUIRED_PI} runtime. ` +
         `Missing host APIs: ${missing.join(", ")}. ` +
-        "Exit Pi and start Pi 0.86.x after updating. /reload cannot upgrade the running " +
+        `Exit Pi and start ${REQUIRED_PI} after updating. /reload cannot upgrade the running ` +
         "runtime, and PI_PACKAGE_DIR can make an older executable report a newer version.",
     );
   }

package/extensions/openai-codex-compat/remote-compaction.ts

@@ -2,7 +2,7 @@ import { isString } from "./value-contracts.ts";
 import type {
   BeforeProviderHeadersEvent,
   CompactionResult,
-  ContextEvent,
+  ContextWithSystemEvent,
   ExtensionAPI,
   ExtensionContext,
   SessionBeforeCompactEvent,
@@ -18,6 +18,8 @@ import { addRemoteCompactionFeature, isObject, type JsonRecord } from "./codex-p
 import { CODEX_API, CODEX_PROVIDER } from "./codex-identifiers.ts";
 import {
   activeResponsesTools,
+  branchInstructions,
+  declaresActiveTools,
   providerHistory,
   remoteCompactionMarkerSummary,
   responsesCompatibility,
@@ -50,9 +52,9 @@ export type RemoteCompactionLifecycleHandler = (
 ) => void;
 
 export type RemoteCompactionContextHandler = (
-  event: Pick<ContextEvent, "messages">,
+  event: Pick<ContextWithSystemEvent, "messages">,
   ctx: RemoteCompactionContext,
-) => { messages: ContextEvent["messages"] } | undefined;
+) => { messages: ContextWithSystemEvent["messages"] } | undefined;
 
 export type RemoteCompactionHeadersHandler = (
   event: Pick<BeforeProviderHeadersEvent, "headers">,
@@ -114,7 +116,12 @@ export function remoteCompactionApi(pi: ExtensionAPI): RemoteCompactionApi {
     getAllTools: () => pi.getAllTools(),
     onBeforeProviderHeaders: (handler) =>
       pi.on("before_provider_headers", (event, ctx) => handler(event, context(ctx))),
-    onContext: (handler) => pi.on("context", (event, ctx) => handler(event, context(ctx))),
+    // The full-transcript event returns messages verbatim. A changed `context`
+    // result would fold every later system message into the leading one, so
+    // `instructions` would no longer be the branch's leading prompt and the
+    // inline developer items in `input` would repeat the folded updates.
+    onContext: (handler) =>
+      pi.on("context_with_system", (event, ctx) => handler(event, context(ctx))),
     onSessionBeforeCompact: (handler) =>
       pi.on("session_before_compact", (event, ctx) => handler(event, context(ctx))),
     onSessionShutdown: (handler) =>
@@ -245,15 +252,23 @@ export default function registerRemoteCompaction(
       const history = providerHistory({
         branch: event.branchEntries,
         wireModel: ctx.model,
-        allTools,
         grammarToolInputProperties,
         imageDetail: config.imageDetail,
         recoverLatestOverflowPrefix: event.reason === "overflow" && event.willRetry,
       });
-      // Compaction rebases history, so use the complete current tool set.
+      // Reuse the last turn's exact declarations when they still describe the
+      // active tools, so the compaction request shares the turn requests'
+      // prompt-cache prefix. Rebuild from the registry only when the active
+      // set changed since that turn or no turn template exists.
       const template: JsonRecord = { ...matching?.payload };
+      const activeNames = pi.getActiveTools();
+      const cachedTools = Array.isArray(matching?.payload.tools)
+        ? matching.payload.tools.filter(isObject)
+        : undefined;
       template.tools =
-        activeResponsesTools(allTools, pi.getActiveTools(), grammarToolInputProperties) ?? [];
+        cachedTools && declaresActiveTools(cachedTools, allTools, activeNames)
+          ? cachedTools
+          : (activeResponsesTools(allTools, activeNames, grammarToolInputProperties) ?? []);
       const requestOptions: OpenAICodexResponsesOptions = {
         ...matching?.requestOptions,
         apiKey: authentication.apiKey,
@@ -265,7 +280,14 @@ export default function registerRemoteCompaction(
         model: ctx.model,
         requestOptions,
         history,
-        instructions: instructionsForCompaction(ctx.getSystemPrompt(), event.customInstructions),
+        // Turn requests send the branch's leading system message as `instructions`
+        // and carry later system messages inline in the history. A branch without
+        // a leading system message (a pre-0.86 session) falls back to Pi's
+        // current prompt so the compaction still sees instructions.
+        instructions: instructionsForCompaction(
+          branchInstructions(ctx.model, event.branchEntries) || ctx.getSystemPrompt(),
+          event.customInstructions,
+        ),
         grammarToolInputProperties:
           matching?.grammarToolInputProperties ??
           requestGrammarToolInputProperties(template, allTools),

package/extensions/openai-codex-compat/tool-runtime.ts

@@ -1,4 +1,4 @@
-import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
+import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
 import type { Api, Model, ProviderHeaders } from "@earendil-works/pi-ai";
 import { providerHistory } from "./compaction-checkpoint.ts";
 import type { ImageDetail } from "./config.ts";
@@ -12,7 +12,6 @@ export interface CodexToolAuthentication {
 export type ToolAuthenticationContext = {
   modelRegistry: Pick<ExtensionContext["modelRegistry"], "getApiKeyAndHeaders">;
 };
-export type ToolHistoryApi = Pick<ExtensionAPI, "getAllTools">;
 export type ToolHistoryContext = {
   sessionManager: Pick<ExtensionContext["sessionManager"], "getBranch">;
 };
@@ -32,7 +31,6 @@ export async function codexToolAuthentication(
 }
 
 export function codexToolHistory(
-  pi: ToolHistoryApi,
   ctx: ToolHistoryContext,
   model: Model<Api>,
   imageDetail: ImageDetail = "auto",
@@ -40,7 +38,6 @@ export function codexToolHistory(
   return providerHistory({
     branch: ctx.sessionManager.getBranch(),
     wireModel: model,
-    allTools: pi.getAllTools(),
     imageDetail,
   });
 }

package/extensions/openai-codex-compat/vendor/pi-ai/README.md

@@ -8,14 +8,15 @@ Pi AI's transcript helpers replay system sections and tool declarations.
 `supportsAdditionalTools` and `supportsToolSearch` select the dynamic tool
 history representation for additive `toolsAdded` system messages:
 message-anchored `additional_tools` or a `tool_search_call`/`tool_search_output`
-pair. Tool removal, redefinition, or a model without either capability requires
-the complete current top-level tool list.
-
-Compat uses the extension-owned `includeSystemUpdates: false` option alongside
-`includeSystemPrompt: false`. It sends the complete current prompt separately
-and preserves only supported tool additions in system-message positions.
-This keeps old prompt text from overriding current instructions after
-compaction, forced-prompt projection, reload, or resume.
+pair. The copy keeps both options so the wire-equivalence tests hold, but Compat
+always passes `false`: it declares every tool at the top level.
+
+System messages follow Pi AI exactly. Compat passes `includeSystemPrompt: false`
+because the leading system message travels as `instructions`, and forwards the
+model's `supportsMidConvoSystemMessages` flag so later system messages render
+inline through `renderSystemMessageUpdate` or collapse into the leading message.
+A system message that lands between a tool call and its results is held until
+after those results, as upstream `transformMessages` does.
 
 The optional `namespacedToolNames`, `textContentItemToolResultNames`, and `toolResultImageDetail` paths are extension-owned additions. Without those options, serialization must continue to match Pi AI. `namespacedToolNames` groups only the fixed Codex allowlist into native Responses namespaces and replays namespace/member call identities. `textContentItemToolResultNames` preserves Codex tools such as `web.run` whose successful text output is transported as an `input_text` content-item array instead of Pi's usual plain string. `toolResultImageDetail` overrides the otherwise canonical `auto` detail used for image tool-result content.
 

package/extensions/openai-codex-compat/vendor/pi-ai/openai-responses-serialization.ts

@@ -53,8 +53,6 @@ type ToolResultOutput =
 
 type ConvertResponsesMessagesOptions = {
   includeSystemPrompt?: boolean;
-  /** Compat sends the complete current prompt outside replay history. */
-  includeSystemUpdates?: boolean;
   grammarToolInputProperties?: ReadonlyMap<string, string>;
   supportsMidConvoSystemMessages?: boolean;
   supportsAdditionalTools?: boolean;
@@ -551,25 +549,33 @@ function transformMessages(
   const result: Message[] = [];
   let pendingToolCalls: ToolCall[] = [];
   let existingToolResultIds = new Set<string>();
-  const insertSyntheticToolResults = () => {
-    for (const toolCall of pendingToolCalls) {
-      if (existingToolResultIds.has(toolCall.id)) continue;
-      result.push({
-        role: "toolResult",
-        toolCallId: toolCall.id,
-        toolName: toolCall.name,
-        content: [{ type: "text", text: "No result provided" }],
-        isError: true,
-        timestamp: Date.now(),
-      } satisfies ToolResultMessage);
+  // System messages are transparent to tool-call accounting: one that lands between a
+  // tool call and its results is held back and emitted after the results (synthetic
+  // ones included), so it never causes a duplicate result for a call answered later.
+  const heldSystemMessages: Message[] = [];
+  const closePendingToolCalls = () => {
+    if (pendingToolCalls.length > 0) {
+      for (const toolCall of pendingToolCalls) {
+        if (existingToolResultIds.has(toolCall.id)) continue;
+        result.push({
+          role: "toolResult",
+          toolCallId: toolCall.id,
+          toolName: toolCall.name,
+          content: [{ type: "text", text: "No result provided" }],
+          isError: true,
+          timestamp: Date.now(),
+        } satisfies ToolResultMessage);
+      }
+      pendingToolCalls = [];
+      existingToolResultIds = new Set();
     }
-    pendingToolCalls = [];
-    existingToolResultIds = new Set();
+    result.push(...heldSystemMessages);
+    heldSystemMessages.length = 0;
   };
 
   for (const message of transformed) {
     if (message.role === "assistant") {
-      insertSyntheticToolResults();
+      closePendingToolCalls();
       const assistantMessage = message;
       if (assistantMessage.stopReason === "error" || assistantMessage.stopReason === "aborted") {
         continue;
@@ -585,14 +591,17 @@ function transformMessages(
     } else if (message.role === "toolResult") {
       existingToolResultIds.add(message.toolCallId);
       result.push(message);
+    } else if (message.role === "system") {
+      if (pendingToolCalls.length > 0) heldSystemMessages.push(message);
+      else result.push(message);
     } else if (message.role === "user") {
-      insertSyntheticToolResults();
+      closePendingToolCalls();
       result.push(message);
     } else {
       result.push(message);
     }
   }
-  insertSyntheticToolResults();
+  closePendingToolCalls();
   return result;
 }
 
@@ -737,7 +746,7 @@ export function convertResponsesMessages(
           tools: convertResponsesTools(addedTools, { ...toolOptions, deferLoading: true }),
         });
       }
-      if (isLeadingSystemMessage ? includeSystemPrompt : options?.includeSystemUpdates !== false) {
+      if (!isLeadingSystemMessage || includeSystemPrompt) {
         const text = isLeadingSystemMessage
           ? getSystemMessageText(message)
           : renderSystemMessageUpdate(message);

package/extensions/openai-codex-compat/web-run.ts

@@ -176,7 +176,7 @@ export default function registerWebRun(
       }
       const authentication = await codexToolAuthentication(ctx, model);
       const config = resolveConfig(ctx);
-      const history = codexToolHistory(pi, ctx, model, config.imageDetail);
+      const history = codexToolHistory(ctx, model, config.imageDetail);
       const input = recentSearchInput(history);
       const body: JsonRecord = {
         id: ctx.sessionManager.getSessionId(),

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-openai-codex-compat",
-  "version": "0.0.13",
+  "version": "0.0.14",
   "description": "OpenAI Codex compatibility for Pi with native compaction, fast mode, and Codex-optimized capabilities",
   "keywords": [
     "pi-package"
@@ -51,9 +51,9 @@
   "devDependencies": {
     "@2h2d/oxlint-config": "0.1.1",
     "@2h2d/ts-config": "0.0.1",
-    "@earendil-works/pi-ai": "0.86.1",
-    "@earendil-works/pi-coding-agent": "0.86.1",
-    "@earendil-works/pi-tui": "0.86.1",
+    "@earendil-works/pi-ai": "0.87.0",
+    "@earendil-works/pi-coding-agent": "0.87.0",
+    "@earendil-works/pi-tui": "0.87.0",
     "@types/node": "22.20.1",
     "oxfmt": "0.64.0",
     "oxlint": "1.79.0",
@@ -62,9 +62,9 @@
     "typescript": "7.0.2"
   },
   "peerDependencies": {
-    "@earendil-works/pi-ai": ">=0.86.0 <0.87.0",
-    "@earendil-works/pi-coding-agent": ">=0.86.0 <0.87.0",
-    "@earendil-works/pi-tui": ">=0.86.0 <0.87.0",
+    "@earendil-works/pi-ai": ">=0.87.0 <0.88.0",
+    "@earendil-works/pi-coding-agent": ">=0.87.0 <0.88.0",
+    "@earendil-works/pi-tui": ">=0.87.0 <0.88.0",
     "typebox": "*"
   },
   "engines": {