pi-openai-toolkit 0.14.12 → 0.15.0

package/README.md

@@ -11,11 +11,11 @@ Add Codex context windows, Responses compaction, hosted tools, and reviewed tool
 
 | Feature | Use it to |
 | --- | --- |
-| Codex Remote Context | Start a new context window and retrieve earlier windows with `history`. |
+| Codex Remote Context | Use the adapted Codex window protocol to start a new context window and retrieve earlier windows with `history`. |
 | Remote Compaction v2 | Continue an eligible Responses session with an encrypted server checkpoint. |
-| Web Search routes | Choose local `pi-web-access`, hosted Responses `web_search`, or CPA standalone `web.run` per exact model route. |
-| Image generation | Generate images or edit explicitly supplied local reference images. |
-| Tool-call review | Ask a reviewer model whether selected tool calls may run. |
+| Web Search routes | Choose local `pi-web-access`, hosted Responses `web_search`, or experimental CPA standalone `web_run` per exact model. |
+| Image generation | Call the hosted Responses image tool to generate images or edit explicitly supplied local references. |
+| Tool-call review | Use Toolkit's approval gate to ask a reviewer model whether selected tool calls may run. |
 
 The package uses Pi's existing model, authentication, and session configuration. It does not add a provider or model.
 
@@ -23,36 +23,33 @@ The package uses Pi's existing model, authentication, and session configuration.
 
 Requires Pi 0.85.1 or newer and Node.js 22.19.0 or newer.
 
-Install the extension:
-
 ```bash
 pi install npm:pi-openai-toolkit
 ```
 
-Use `--local` to install it in the current project.
+Use `--local` to install the extension in the current project. Toolkit policy is still global; there are no project or environment policy overlays.
 
-Installing the package alone does not enable every feature. With no extension config, compaction is enabled but Remote Context is off, Web Search has no toolkit-selected route, image generation is disabled, and Auto Mode has no allowed models or reviewer.
+Without a config file, Remote Compaction v2 is enabled for eligible models, Remote Context windows are off, search is unmanaged, image generation is disabled, and Auto Mode is unavailable. These defaults do not verify backend support.
 
-The extension config file is:
+The only Toolkit config file is:
 
 `~/.pi/agent/extensions/pi-openai-toolkit/config.json`
 
-All JSON configuration examples below, except the `models.json` example, go in this file. If it does not exist, create the file and its parent directory. If it already exists, merge fields into the matching objects and keep the other settings.
+All JSON examples below except the Pi model-registration example use **configuration schema v2**. Create the file and parent directory if absent. Merge examples into an existing v2 document, keeping its other settings. An unversioned legacy file remains supported: do not add v2 fields to it or overwrite it with an example. First use `/toolkit-config migration-preview`; the [configuration reference](docs/configuration.md) explains the review and installed-version checks. Loading config and previewing migration never rewrite the source file.
 
 ## Quick start: enable Remote Context
 
-This section is for users who want Codex-style context windows. If you only want Web Search, image generation, or tool-call review, skip to [Common tasks](#common-tasks).
+This section is for Codex-style context windows. For search, images, or tool-call review only, skip to [Common tasks](#common-tasks).
 
 ### Use Pi's built-in Codex provider
 
-You must already be signed in to Pi's built-in `openai-codex` provider.
-
-Create or merge this extension config:
+You must already be signed in to Pi's built-in `openai-codex` provider. Create or merge this v2 Toolkit config:
 
 ```json
 {
-  "compaction": {
-    "contextManagement": "remote"
+  "schemaVersion": 2,
+  "defaults": {
+    "context": { "mode": "remote-windows" }
   }
 }
 ```
@@ -63,13 +60,13 @@ Start Pi with a model from your existing Codex catalog:
 pi --model openai-codex/<model-id>
 ```
 
-Replace `<model-id>` with the model ID shown by your Pi setup. The session is activated when `new_context`, `get_context_remaining`, `history`, and `notes` appear as available tools.
+Replace `<model-id>` with the ID shown by your Pi setup. The session is activated when `new_context`, `get_context_remaining`, `history`, and `notes` appear. This verifies activation, not a completed backend round trip.
 
 ### Use a compatible gateway
 
-This route requires the `openai-responses` API and a gateway that preserves the Codex protocol fields used by Remote Context. A successful ordinary chat request does not prove Remote Context compatibility.
+This route requires `openai-responses` and a gateway that preserves Remote Context's Codex protocol fields. A successful ordinary chat request does not prove Remote Context compatibility.
 
-If `~/.pi/agent/models.json` already contains a gateway model that meets these conditions, skip model configuration and set the extension allowlist directly. Otherwise, add or merge the provider entry below. Replace `my-gateway`, the URL, the environment variable name, and the model values with values from your setup. The numeric values shown are examples, not project defaults; they must match the actual model and gateway.
+If `~/.pi/agent/models.json` already registers a compatible model, skip registration and set Toolkit's exact model override below. Otherwise, add or merge this Pi provider entry. Replace the provider name, URL, environment variable, and model values with your setup. The numeric limits are examples, not project defaults.
 
 ```json
 {
@@ -103,136 +100,149 @@ In a POSIX shell:
 export MY_GATEWAY_KEY="replace-with-your-gateway-key"
 ```
 
-Use the same terminal session to start Pi. Create or merge the extension config, and make the allowlist entry exactly match the provider and model ID:
+Use the same terminal to start Pi. In the Toolkit config, make the key exactly match the registered provider and model ID:
 
 ```json
 {
-  "compaction": {
-    "contextManagement": "remote",
-    "gatewayContextModels": ["my-gateway/gpt-5.6-luna"]
+  "schemaVersion": 2,
+  "models": {
+    "my-gateway/gpt-5.6-luna": {
+      "context": { "mode": "remote-windows" },
+      "compatibility": { "transport": "codex-gateway" }
+    }
   }
 }
 ```
 
-Start Pi with the same model specification:
-
 ```bash
 pi --model my-gateway/gpt-5.6-luna
 ```
 
-The enablement check is the same: the session should expose `new_context`, `get_context_remaining`, `history`, and `notes`. If they do not appear, read the toolkit notification and check the exact provider/model string, API, key, base URL, and allowlist entry.
+Check for `new_context`, `get_context_remaining`, `history`, and `notes`. If absent, read the notification and check `/toolkit-config`, the exact model key, Pi API, credentials, and base URL. The transport setting opts into a protocol profile; it does not register a model or prove the backend supports it.
 
-Earlier windows remain available through `history`, but they are not all automatically added to the current context.
+Earlier windows remain retrievable through `history`, but are not all automatically inserted into the current context.
 
 ## Common tasks
 
 ### Continue a session with server-side compaction
 
-Leave Remote Context off when you want the Responses compaction path instead. Remote Compaction v2 stores and replays an encrypted checkpoint for eligible Responses models. Set `compaction.remoteCompactModel` only when the compaction request should use a separate model.
+Use `context.mode: "remote-compaction"` for the Responses compaction path (the default), or `"pi"` to relinquish Toolkit context management. Set `context.remoteCompaction.model` only when a separate model should produce the checkpoint. These fields belong under `defaults` or an exact `models` override.
 
-When `compaction.remoteV2ContextSource` is omitted, Remote V2 keeps the original `"legacy"` input chain: first compaction uses Pi's current session context (or the supplied preparation as a last resort), and recursive compaction uses the raw branch tail. This preserves existing behavior, but can diverge from provider-visible context when other extensions rewrite messages.
+`context.remoteCompaction.inputSource` defaults to `"legacy"`: first compaction uses Pi's current session context, with event preparation as a last resort; recursion uses the raw branch tail. This preserves existing behavior but can diverge from provider-visible context when other extensions rewrite messages.
 
-Set `compaction.remoteV2ContextSource` to `"pi-context-hook"` to opt into the Pi 0.85.1 runtime bridge. The toolkit then uses the same ordered `context` hook chain as the live provider request. If Pi cannot expose that public hook path, Remote V2 cancels instead of sending an unprojected history. Checkpoint replay is accepted only when its source marker matches the current setting. Switching between the two modes requires a new Remote V2 checkpoint; disable Remote V2 or use Pi's native compaction when custom-only state must survive inside the opaque checkpoint.
+Opt into `"pi-context-hook"` to use the ordered Pi context-hook projection. If that bridge is unavailable, compaction cancels rather than sending unprojected history. Checkpoints are source-specific; switching input sources requires a new checkpoint. See [protocol details](docs/internals.md#remote-compaction-v2-wire-contract).
 
 ### Choose a Web Search route
 
-Web Search has three mutually exclusive routes. Configure one global default and, when needed, exact `provider/model-id` overrides:
+Configure a global default and exact model overrides:
 
 ```json
 {
-  "webSearch": {
-    "enabled": true,
-    "defaultRoute": "hosted",
-    "routes": {
-      "uwoacrimson/gpt-6-astra": "standalone-alpha",
-      "my-gateway/gpt-5.6-luna": "local"
+  "schemaVersion": 2,
+  "defaults": {
+    "webSearch": { "route": "local" }
+  },
+  "models": {
+    "my-gateway/gpt-5.6-luna": {
+      "webSearch": { "route": "hosted" }
     }
   }
 }
 ```
 
-Route selection is exact and deterministic: `routes[provider/model-id]` wins over `defaultRoute`; `defaultRoute` wins over the legacy `models` list. There is no wildcard, fuzzy model matching, capability guessing, fallback, or retry between routes. Invalid route values/keys are ignored with warnings rather than guessed, and overlapping new/legacy fields produce migration warnings. `enabled: false` releases toolkit ownership and disables all three toolkit-selected paths.
+- `unmanaged` releases Toolkit ownership. It does not disable third-party search or all Internet access.
+- `local` preserves the original local `pi-web-access` tool state and removes conflicting hosted/standalone tools from provider payloads. It never activates a local tool that was inactive.
+- `hosted` replaces the local `web_search` function with the native Responses search tool and source annotations.
+- `standalone-alpha` deliberately opts into experimental CPA/Codex gateway search. It exposes sequential `web_run` and sends one isolated request per call to the provider-relative `/alpha/search`, using the current Pi model and credentials. The gateway must actually support that endpoint and capability; Toolkit does not probe or enable it.
 
-- **`local`** keeps the already-installed `pi-web-access` `web_search` tool. The toolkit does not activate it when it was not active, and it does not add a hosted provider tool or `web.run`.
-- **`hosted`** removes the local function named `web_search` and injects the native Responses `{ "type": "web_search" }` tool plus source annotations. The legacy configuration remains valid:
-
-  ```json
-  {
-    "webSearch": {
-      "models": ["my-gateway/gpt-5.6-luna"]
-    }
-  }
-  ```
-
-  With no new route fields, only models in this exact list and the existing Responses-family APIs (`openai-responses` or `openai-codex-responses`) use hosted search.
-- **`standalone-alpha`** exposes one sequential `web.run` tool and sends one isolated `POST` request to the provider-relative `/alpha/search` endpoint. A base URL such as `https://gateway.example/v1` therefore receives `https://gateway.example/v1/alpha/search`, not a Responses endpoint. Supported command families are `search_query`, `image_query`, `open`, `click`, `find`, `screenshot`, `finance`, `weather`, `sports`, and `time`; `response_length` controls the requested result size.
-
-The standalone route is an experimental CPA/Codex gateway protocol, not a stable public OpenAI Responses endpoint. The gateway/provider must expose `/alpha/search`, enable its `alpha-search` capability, and support standalone web search (Codex providers commonly expose this as `supports_standalone_web_search = true`). The request reuses Pi's current model, authentication, provider headers, session identity, and Codex/gateway affinity; it does not change the active model. The MVP sends the bounded command envelope rather than the full conversation transcript, relies on provider session/reference handling for `ref_id` follow-ups, makes at most one request per tool call, and fails closed on invalid configuration, unavailable routes, cancellation, timeout, non-2xx, malformed, or oversized responses.
+Exact overrides win over defaults. There are no patterns, inferred capabilities, or fallback between routes. Invalid selected policy blocks the affected operation instead of choosing another route. The legacy hosted model list remains readable, but its permissive failure behavior is not identical to explicit v2 `hosted`; migration preview flags that difference.
 
 ### Generate an image
 
-Image generation requires a Responses session and may incur provider charges. Enable it with:
+Image generation requires a Responses session and may incur provider charges. Enable it globally:
 
 ```json
 {
-  "imageGeneration": {
-    "enabled": true,
-    "models": ["gpt-image-2.5", "grok-imagine-image-2.0"]
+  "schemaVersion": 2,
+  "defaults": {
+    "imageGeneration": {
+      "enabled": true,
+      "defaultModel": "gpt-image-2.5",
+      "allowedModels": ["gpt-image-2.5", "grok-imagine-image-2.0"]
+    }
   }
 }
 ```
 
-`models` contains the bare model IDs used by the nested Responses `image_generation` tool. The first entry is the default; `openai_generate_image` also accepts an optional `model` argument for a one-call override, but it must match a configured entry exactly. If `models` is omitted, the default is `gpt-image-2.5`. Empty or invalid lists are ignored with a warning and fall back to that default; set `enabled` to `false` to disable the tool. The provider or gateway must support the configured image model.
+These are bare output-model IDs for the nested Responses `image_generation` tool. List order does not choose the default. `defaultModel` must belong to the nonempty `allowedModels` list; an optional one-call `model` must also be allowed. Invalid policy or a disallowed choice fails before authentication, reference upload, and paid dispatch. The provider must support the chosen model.
 
-The `openai_generate_image` tool supports text-to-image requests and edits using explicitly supplied local reference images.
+`openai_generate_image` accepts text-to-image requests and edits using explicitly supplied local references. Image policy cannot be overridden per session model.
 
 ### Review tool calls automatically
 
-Allow a model and reviewer in `autoMode`:
+Make Auto Mode available to an exact model and choose its reviewer:
 
 ```json
 {
-  "autoMode": {
-    "models": ["my-gateway/gpt-5.6-luna"],
-    "reviewerModel": "my-gateway/gpt-5.6-luna"
+  "schemaVersion": 2,
+  "models": {
+    "my-gateway/gpt-5.6-luna": {
+      "autoMode": {
+        "available": true,
+        "reviewerModel": "my-gateway/gpt-5.6-luna"
+      }
+    }
   }
 }
 ```
 
-Use `/auto on` in the session, or start Pi with `--auto`. The TUI shows an activation notice and temporarily changes the working indicator to `Auto mode: reviewing <tool>` while a gated call is being reviewed; the footer keeps the active gate visible. On Pi versions that expose the compatible tool renderer, each tool block in Auto Mode also gets a bottom line: reviewed calls show states such as `allowed by reviewer · low risk · authorization medium` or `denied · <reason>`, while calls outside the configured gate show `not reviewed · outside the configured gate`. If that renderer seam is unavailable, the extension warns once and keeps the footer-only status. The default `side-effect` gate reviews `bash`, `write`, `edit`, and configured extra tools. Set `gate` to `"all"` when every tool call needs review. A reviewer timeout does not approve a call.
+`available` permits engagement; use `/auto on` or `--auto` to engage. `/auto off` explicitly disengages it. The default `side-effect` gate covers `bash`, `write`, `edit`, and configured extra tools; `gate: "all"` reviews every tool call. A reviewer timeout does not approve a call. Invalid configuration while engaged keeps a blocking gate until corrected or explicitly turned off.
+
+The TUI shows activation, review activity, and a footer status. Compatible Pi tool renderers also show per-call allowed, denied, blocked, or unreviewed states. If that renderer seam is unavailable, Toolkit warns once and retains the footer display. Reviewer, classifier, and circuit-breaker controls are in the [configuration reference](docs/configuration.md).
 
 ## Common configuration
 
-The config file is `~/.pi/agent/extensions/pi-openai-toolkit/config.json`. Unknown keys are ignored with a warning. Most model lists use exact `provider/model-id` strings, not globs; `imageGeneration.models` is an exception and contains bare nested image-generation model IDs.
+The global file uses built-ins, then `defaults`, then exact `models["provider/model-id"]` overrides. Missing fields inherit; arrays replace; `false` and `0` are retained. Supported optional model references accept `null` to clear inheritance. Defaults are not master switches: an exact override can enable a feature whose default is disabled.
 
-| Key | Default | Use |
+| Field | Default | Use |
 | --- | --- | --- |
-| `compaction.enabled` | `true` | Master switch for compaction. |
-| `compaction.contextManagement` | `"off"` | Enables Codex Remote Context when set to `"remote"`. |
-| `compaction.gatewayContextModels` | `[]` | Gateway models allowed to use Remote Context. |
-| `compaction.remoteCompactModel` | unset | Optional model used only for a v2 compaction request. |
-| `compaction.remoteV2ContextSource` | `"legacy"` | Preserve the original raw session/branch input path. Set `"pi-context-hook"` to opt into Pi's ordered context-hook projection. Checkpoints are mode-specific. |
-| `compaction.contextReminderThresholdPercent` | `5` | Remaining budget percentage for the once-per-window reminder. `0` disables the reminder and exhausted-window fallback. |
-| `webSearch.enabled` | `true` | Total switch for the toolkit's Web Search route selection. `false` selects no toolkit route. |
-| `webSearch.defaultRoute` | unset | Default route: `local`, `hosted`, or `standalone-alpha`. Unset preserves legacy behavior. |
-| `webSearch.routes` | unset | Exact `provider/model-id` to route overrides. Exact entries take precedence over `defaultRoute`. |
-| `webSearch.models` | `[]` | Legacy exact allowlist; without the new route fields, listed Responses-family models receive hosted Web Search. |
-| `imageGeneration.enabled` | `false` | Enables `openai_generate_image`. |
-| `imageGeneration.models` | `["gpt-image-2.5"]` | Bare image-generation model IDs; the first entry is the default. |
-| `autoMode.models` | `[]` | Models allowed to use Auto Mode. |
-| `autoMode.reviewerModel` | unset | Model that reviews Auto Mode calls. |
-| `autoMode.gate` | `"side-effect"` | Use `"all"` to review every tool call. |
-| `autoMode.timeoutMs` | `30000` | Review timeout in milliseconds. |
+| `defaults.context.mode` | `"remote-compaction"` | `"pi"`, `"remote-compaction"`, or `"remote-windows"`. |
+| `models[exact].compatibility.transport` | `"standard"` | Exact gateway opt-in with `"codex-gateway"`. |
+| `defaults.context.remoteCompaction.model` | `null` | Optional checkpoint producer. |
+| `defaults.context.remoteCompaction.inputSource` | `"legacy"` | Source-specific checkpoint input policy. |
+| `defaults.context.remoteWindows.reminderThresholdPercent` | `5` | `0` disables reminders and exhausted-window fallback. |
+| `defaults.webSearch.route` | `"unmanaged"` | Toolkit search ownership policy. |
+| `defaults.imageGeneration.enabled` | `false` | Global image-generation switch. |
+| `defaults.imageGeneration.defaultModel` | `"gpt-image-2.5"` | Explicit default output model. |
+| `defaults.imageGeneration.allowedModels` | `["gpt-image-2.5"]` | Allowed output models, global only. |
+| `defaults.autoMode.available` | `false` | Permission to engage tool review. |
+| `defaults.autoMode.gate` | `"side-effect"` | Selected side-effect tools or `"all"`. |
+| `defaults.autoMode.timeoutMs` | `30000` | Reviewer timeout in milliseconds. |
+| `diagnostics.level` | `"info"` | `"debug"` enables debug artifacts. |
+| `diagnostics.captureRequests` / `captureResponses` | `false` | Independent request/compact-response capture opt-ins. |
+
+Use `/toolkit-config` for effective values and origins, `/toolkit-config validate` for document issues, and `/toolkit-config migration-preview` for a read-only legacy candidate. These commands require a UI and perform no network/auth lookup, tool activation, or file writes. Configuration selection does not verify backend support.
+
+Unknown v2 policy keys and malformed values produce visible scoped errors, independent of debug mode. Each public callback or tool execution uses one immutable snapshot across its awaited helpers; later operations reread the file. Separate Pi events are not one atomic transaction. See the [complete configuration reference](docs/configuration.md) and [editor schema](config.schema.json).
 
 ## Development
 
 From the repository root, after dependencies are installed:
 
 ```bash
-npm run typecheck    # Type-check
-bun test             # Run tests
-npm run test:pi      # Run the Pi smoke test
-npm pack --dry-run   # Inspect the package contents
+npm run typecheck
+```
+
+```bash
+bun test
+```
+
+```bash
+npm run test:pi
+```
+
+```bash
+npm pack --dry-run
 ```
 
 ## License