pi-codex-tools 0.3.0 → 0.4.0

package/CHANGELOG.md

@@ -6,6 +6,23 @@ This project follows the spirit of [Keep a Changelog](https://keepachangelog.com
 
 ## [Unreleased]
 
+## [0.4.0] - 2026-10-09
+
+### Fixed
+
+- Clear theme-colored patch preview caches on invalidation so existing calls repaint after theme changes without another execution.
+
+### Changed
+
+- Add `codex_files` namespace metadata and accurate mutation/destruction hints without changing grammar, activation, model-only exposure or native file tools.
+- Remove the obsolete direct-compaction grammar event-bus adapter. Public server-side compaction preserves the ordinary request's grammar declarations.
+- Update the shared Pi development and contract-test baseline to 1.1.0; require Node.js >=22.19.0 to match the host runtime. Pi remains a host-supplied peer dependency.
+- Register `apply_patch` with model-only exposure, excluding nested calls through codemode.
+- Hide selected native `edit` and `write` declarations through Pi's public loadout hook while keeping their implementations, activation, and exposure intact for nested calls.
+- Preserve tool selection across reloads and model switches, including explicit activation and deactivation. Excluded tools remain excluded; older Pi runtimes retain the legacy selection policy.
+- Preserve approval wrappers regardless of extension load order or registration time, and leave native file tools intact when another extension overrides `apply_patch`.
+- On legacy Pi runtimes, hide reactivated file tools again on each supported-model switch while retaining them for restoration on unsupported models.
+
 ## [0.3.0] - 2026-09-18
 
 ### Changed

package/CONTRIBUTING.md

@@ -33,3 +33,9 @@ Before opening a pull request:
 ## Code of conduct
 
 This project follows the Contributor Covenant Code of Conduct.
+## Pi 1.1.0 terminal audit
+
+From the monorepo root, run `node --import tsx --test tests/tui-contracts.test.mjs`.
+The suite uses actual regular/fullscreen tool shells, theme invalidation, resize,
+Unicode and host-owned output padding. See `tests/TUI_AUDIT.md` for remaining
+physical-terminal and export acceptance checks.

package/README.md

@@ -5,10 +5,11 @@ Give grammar-capable OpenAI/Codex models the Codex `apply_patch` tool in Pi with
 ## What it adds
 
 - **Raw `apply_patch`** — sends Codex's Lark grammar as an OpenAI custom tool, so patches are not JSON-wrapped.
+- **Model-only exposure** — `apply_patch` is available directly to the model, never through codemode or other nested tool calls. Requires Pi 0.99.1 or newer.
 - **Capability-based activation** — requires `openai-codex-responses` or `openai-responses` plus `model.compat.supportsOpenAIGrammarTools === true`; model names alone are never enough.
 - **Pi-style filesystem access** — accepts relative or absolute paths and follows symlinked files and directories, including macOS `/tmp`. Uses Node filesystem APIs without a native binding or platform gate.
 - **Validated patches** — limits patches to 1 MiB and target-file reads to 64 MiB, preflights all hunks, and serializes writes with Pi's mutation queue.
-- **Model switching** — supported models replace Pi's `edit` and `write` tools with `apply_patch`; other active tools are preserved. Switching back restores only the file tools that were active before the switch.
+- **Native editing through codemode** — supported models see `apply_patch` instead of native `edit` and `write` declarations. Selected native tools remain active and callable through codemode, without replacing their implementations. Model switches and reloads preserve tool selection and approval wrappers. Keep your usual `defaultTools` selection; no separate exposure extension is needed.
 - **Sequential patch calls** — the extension marks patch execution sequential while leaving provider-side parallel tool calls enabled.
 - **Streaming progress** — while a patch is generated, the TUI shows a live, color-coded glimpse of the content being written (new-file content, or `+`/`-` lines for updates) plus a running `+added -removed` tally and a per-file roster for multi-file patches. It reuses Pi's shared diff rendering and mirrors the built-in `write`/`edit` previews; patch execution is unchanged.
 
@@ -26,7 +27,7 @@ pi -e /path/to/pi-mono/packages/pi-codex-tools
 
 ## Scope decisions
 
-The current Codex source does not define separate `read_file` or `write_file` tools: file inspection is normally done through shell commands and file mutation through `apply_patch`. This package keeps Pi's bounded `read` and `bash` tools, and uses `apply_patch` in place of Pi's `edit` and `write` tools for supported models. Filesystem access uses the local user's permissions, like native Pi tools; it is not a sandbox. `apply_patch` requires a Pi model runtime that advertises `compat.supportsOpenAIGrammarTools`; older runtimes leave the tool inactive.
+The current Codex source does not define separate `read_file` or `write_file` tools: file inspection is normally done through shell commands and file mutation through `apply_patch`. This package keeps Pi's bounded `read` and `bash` tools, and presents `apply_patch` in place of native `edit` and `write` declarations for supported models. Filesystem access uses the local user's permissions, like native Pi tools; it is not a sandbox. `apply_patch` requires a Pi model runtime that advertises `compat.supportsOpenAIGrammarTools`; older runtimes leave the tool inactive.
 
 | Codex surface | Decision |
 | --- | --- |
@@ -41,15 +42,58 @@ These choices are based on the Codex tool specifications in `codex-rs/core/src/t
 
 ## Compatibility notes
 
+The Pi 1.1.0 metadata contract groups `apply_patch` under `codex_files`, without
+renaming it or changing exposure. Its advisory hints are mutating, destructive,
+non-idempotent, and local-only. There is no public output schema: it intentionally
+remains model-only, not discoverable/callable from scripts. Metadata does not
+replace approvals or change native file-tool implementations.
+
+While this package's `apply_patch` is active on Pi 0.99.1 or newer, its public
+`prepareLoadout` hook hides selected native `edit` and `write` declarations from
+model requests. The tools keep their original implementations and `direct`
+exposure, and remain in `pi.getActiveTools()`. Activate `codemode` to call them
+from scripts; `describeTool("edit")` and `describeTool("write")` provide their
+schemas. The `apply_patch` description points to these helpers when codemode is
+active, including in codemode's `on` mode where direct tools are not listed
+inline.
+
+Explicitly activating a native file tool makes it callable but does not reveal
+its declaration while `apply_patch` is active. Deactivating it removes nested
+access too. Switching to an unsupported model stops hiding native declarations;
+other loadout hooks, such as codemode's `only` mode, still apply. Reloads preserve
+the active selection without a package-owned snapshot.
+
+Tools omitted from `defaultTools` or an explicit `--tools` selection are not
+introduced into codemode. Other extensions' file-tool implementations, including
+approval wrappers registered during or after `session_start`, are not replaced
+or hidden by this package. If another extension replaces `apply_patch`, this
+package's loadout hook does not apply to that replacement.
+
+Older Pi runtimes without exposure metadata retain the legacy behavior:
+supported models replace active file tools with `apply_patch`, without
+registering codemode overrides. They do not offer this package's nested native
+editing route; upgrade Pi for that capability. The fallback restores its saved
+file-tool selection when leaving supported models. Older Pi cannot distinguish
+an explicit deactivation of an already-hidden tool from leaving it unchanged.
+To disable such a tool, switch to an unsupported model before changing the
+selection, or upgrade Pi to preserve explicit deactivation while `apply_patch`
+is active.
+
 ### GPT-6 Astra
 
+A Pi 1.1.0 virtual-model selection does not advertise the physical request's
+grammar capability to this extension. `apply_patch` stays unavailable even when
+the router chooses Astra; selected native `edit` and `write` declarations remain
+available, subject to other loadout hooks. The extension does not infer support
+from a virtual model name or reactivate excluded tools.
+
 Pi 0.85.1's model catalog advertises grammar-tool support for `gpt-6-astra` on both `openai-responses` and `openai-codex-responses`. The extension uses that capability directly, with no model-name allowlist or JSON wrapper. Tests cover the pinned Pi transports, streamed raw calls, execution, and result replay using mocked HTTP responses; they do not certify live account access.
 
 The [GPT-6 guide](https://developers.openai.com/api/docs/guides/latest-model?model=gpt-6-astra) also describes async tool calls, mid-turn steering, and reasoning updates. Those belong to the provider/session runtime and are not enabled by this extension. Patch execution remains sequential; provider-side parallel tool calling remains enabled.
 
-With an updated `pi-codex-compaction` installed, the package supplies its owned
-grammar metadata through Pi's public event bus. This keeps raw `apply_patch`
-calls and results intact in direct Codex compaction requests, including Astra.
+With `pi-codex-compaction` installed, public server-side compaction uses Pi's
+ordinary effective request, including this tool's grammar, calls and results.
+The former direct-compaction grammar event-bus adapter has been removed.
 No private Pi registry is patched.
 
 ### Filesystem behavior
@@ -75,7 +119,17 @@ Like native Pi tools, normal path-based I/O does not protect against another pro
 
 These behaviors intentionally match Codex `apply_patch`.
 
-The provider contract is runtime-specific: use Pi 0.83.0 or newer for OpenAI grammar-tool support. For a manual smoke test, start Pi with this extension and a model that advertises `supportsOpenAIGrammarTools`, then ask it to create and update a disposable file through a symlinked directory (on macOS, `/tmp` is suitable). Verify that changes appear as raw `apply_patch` calls, the referent changes, and the symlink remains. Switch to an unsupported model and verify that the original file tools return.
+The provider contract is runtime-specific: use Pi 0.99.1 or newer for model-only tool exposure and the loadout hook; the metadata contract and development/integration tests use Pi 1.1.0.
+
+For a manual smoke test:
+
+1. Start Pi with this extension, the normal file tools, `codemode`, and a model that advertises `supportsOpenAIGrammarTools`.
+2. Ask it to create and update a disposable file through a symlinked directory (on macOS, `/tmp` is suitable). Verify raw `apply_patch` calls, changed referent content, and an intact symlink.
+3. Verify that `describeTool("edit")` and `describeTool("write")` work inside codemode and that scripts can edit a disposable file. `apply_patch` must not be callable inside codemode.
+4. Run `/reload` and repeat the nested-editing check.
+5. Switch to an unsupported model and verify that the selected native file tools remain usable. With codemode in `on` mode, their direct declarations return.
+
+Automated contract tests cover provider declarations, codemode's `on` and `only` modes, reloads, exclusions, and approval wrappers registered in either extension load order.
 
 ## Development
 

package/SECURITY.md

@@ -19,6 +19,19 @@ Report privately through [GitHub Security Advisories](https://github.com/jvm/pi-
 
 Pi extensions execute with the same permissions as the local user running Pi. Review installed extensions and only install packages from sources you trust.
 
+On modern Pi, this package hides selected native `edit` and `write` declarations
+with the public loadout hook. It does not replace their implementations or alter
+their activation or exposure. Approval wrappers remain effective even when
+registered by a later-loaded extension during or after session startup. Nested
+calls use Pi's normal tool validation and `tool_call`/`tool_result` hooks.
+
+Hiding a declaration is presentation, not an execution or permission boundary:
+active tools remain executable. Tools excluded from the active selection are
+not introduced into codemode by this package. Older Pi runtimes without exposure
+metadata retain the legacy tool-selection policy without nested-editing
+overrides. Tool selections are not an OS-level sandbox; `apply_patch` itself can
+modify files when enabled.
+
 `apply_patch` does not access the network or credential APIs. Like Pi's native `edit` and `write` tools, it accepts relative or absolute paths, follows symlinks for reads/writes, and can modify files outside the current working directory with the local user's permissions. It can read credential-containing files when a patch targets them. Deleting a symlink removes the link, not its referent; moving a symlink source copies its referent's updated content and removes the source link.
 
 The tool uses Node filesystem APIs without a platform-specific native binding. This deliberately replaces the previous no-follow policy with Pi-style filesystem access. Path canonicalization is used for preflight identity and queue keys, not as a security boundary. There is no workspace confinement or protection against another process swapping path components between resolution and I/O. Use an OS-level sandbox or restricted user account when filesystem isolation is required.

package/extensions/index.ts

@@ -12,6 +12,11 @@ class ApplyPatchCallComponent extends Text {
   constructor() {
     super("", 0, 0);
   }
+
+  override invalidate(): void {
+    this.cache = undefined;
+    super.invalidate();
+  }
 }
 
 function readPatchArg(args: unknown): string {
@@ -19,10 +24,9 @@ function readPatchArg(args: unknown): string {
 }
 
 const APPLY_PATCH = "apply_patch";
-const EDIT = "edit";
-const WRITE = "write";
-const REPLACED_TOOLS = [EDIT, WRITE] as const;
-type ReplacedTool = (typeof REPLACED_TOOLS)[number];
+const FILE_TOOLS = ["edit", "write"] as const;
+type FileTool = (typeof FILE_TOOLS)[number];
+const APPLY_PATCH_DESCRIPTION = "Apply a Codex patch to files. This is a FREEFORM tool: send the patch text directly, never as JSON.";
 
 const APPLY_PATCH_PARAMETERS = createFreeformInputSchema(
   "patch",
@@ -38,8 +42,15 @@ export default function piCodexTools(pi: ExtensionAPI): void {
 
   registerGrammarTool({
     name: APPLY_PATCH,
+    namespace: {
+      name: "codex_files",
+      description: "Direct model-only grammar-based file patches.",
+      instructions: "apply_patch stays model-only and requires a grammar-capable OpenAI model. Send raw patch text directly; never call it from codemode or replace native file tools to alter visibility. File changes may overwrite or delete local data and remain subject to approval hooks.",
+    },
+    annotations: { readOnlyHint: false, destructiveHint: true, idempotentHint: false, openWorldHint: false },
+    exposure: "model-only",
     label: APPLY_PATCH,
-    description: "Apply a Codex patch to files. This is a FREEFORM tool: send the patch text directly, never as JSON.",
+    description: APPLY_PATCH_DESCRIPTION,
     promptSnippet: "Apply Codex-format file patches without JSON wrapping",
     promptGuidelines: [
       "Use apply_patch for file changes when it is available.",
@@ -50,6 +61,26 @@ export default function piCodexTools(pi: ExtensionAPI): void {
     parameters: APPLY_PATCH_PARAMETERS,
     constrainedSampling: createOpenAILarkSampling(APPLY_PATCH_GRAMMAR),
     executionMode: "sequential",
+    prepareLoadout(loadout) {
+      const tools = pi.getAllTools();
+      // Hide declarations, not implementations or activation. Registering native
+      // replacements would outrank approval tools registered by later extensions.
+      const hiddenDeclarations = FILE_TOOLS.filter((name) =>
+        loadout.declared.some((tool) => tool.name === name)
+        && tools.some((tool) => tool.name === name && tool.sourceInfo?.path === `builtin:${name}`),
+      );
+      if (hiddenDeclarations.length === 0) return;
+      return {
+        hiddenDeclarations,
+        // Codemode's "on" mode does not list direct tools in its own description.
+        // Keep the hidden tools discoverable without overriding codemode's hook.
+        descriptions: loadout.declared.some((tool) => tool.name === "codemode") ? {
+          [APPLY_PATCH]: `${APPLY_PATCH_DESCRIPTION}\n\nNative ${hiddenDeclarations.join(" and ")} remain callable through codemode. Use ${
+            hiddenDeclarations.map((name) => `describeTool("${name}")`).join(" and ")
+          } for their schemas and usage guidance.`,
+        } : undefined,
+      };
+    },
     renderCall(args, theme, context) {
       const component =
         context.lastComponent instanceof ApplyPatchCallComponent ? context.lastComponent : new ApplyPatchCallComponent();
@@ -89,43 +120,31 @@ export default function piCodexTools(pi: ExtensionAPI): void {
     },
   });
 
-  let replacedToolsWasActive: Record<ReplacedTool, boolean> | undefined;
-
-  pi.events?.on("pi-codex-compaction:tools:v1", (value) => {
-    const data = value as {
-      model?: ExtensionContext["model"];
-      tools?: Array<{ name: string; parameters: unknown; constrainedSampling?: OpenAIGrammarSampling }>;
-    } | undefined;
-    if (!data || !supportsOpenAIGrammarTools(data.model) || !Array.isArray(data.tools)) return;
-    for (const tool of data.tools) {
-      // Do not attach our grammar to another extension's apply_patch override.
-      if (tool.name === APPLY_PATCH && tool.parameters === APPLY_PATCH_PARAMETERS) {
-        tool.constrainedSampling = createOpenAILarkSampling(APPLY_PATCH_GRAMMAR);
-      }
-    }
-  });
+  let legacyReplacedTools: Set<FileTool> | undefined;
 
   function synchronizeTools(ctx: ExtensionContext): void {
     if (typeof pi.getActiveTools !== "function" || typeof pi.setActiveTools !== "function") return;
 
     const active = new Set(pi.getActiveTools());
+    const tools = typeof pi.getAllTools === "function" ? pi.getAllTools() : [];
+    // Exposure metadata and prepareLoadout arrived together. Do not infer runtime
+    // support or this extension's ownership from an overridable apply_patch tool.
+    const supportsLoadout = tools.some((tool) => tool.exposure !== undefined);
     if (supportsOpenAIGrammarTools(ctx.model)) {
-      if (replacedToolsWasActive === undefined) {
-        replacedToolsWasActive = {
-          edit: active.has(EDIT),
-          write: active.has(WRITE),
-        };
+      if (!supportsLoadout) {
+        // Older runtimes cannot hide declarations independently of activation.
+        // Re-hide later activations on every supported-model switch, retaining
+        // them for restoration when leaving the supported models.
+        legacyReplacedTools ??= new Set();
+        for (const name of FILE_TOOLS) {
+          if (active.delete(name)) legacyReplacedTools.add(name);
+        }
       }
-      for (const tool of REPLACED_TOOLS) active.delete(tool);
       active.add(APPLY_PATCH);
     } else {
       active.delete(APPLY_PATCH);
-      if (replacedToolsWasActive) {
-        for (const tool of REPLACED_TOOLS) {
-          if (replacedToolsWasActive[tool]) active.add(tool);
-        }
-      }
-      replacedToolsWasActive = undefined;
+      for (const name of legacyReplacedTools ?? []) active.add(name);
+      legacyReplacedTools = undefined;
     }
     pi.setActiveTools([...active]);
   }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-codex-tools",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "description": "Codex-compatible apply_patch tooling for Pi's grammar-capable OpenAI models.",
   "type": "module",
   "license": "Apache-2.0",
@@ -56,19 +56,19 @@
     "typebox": "*"
   },
   "devDependencies": {
-    "@earendil-works/pi-ai": "^0.85.1",
-    "@earendil-works/pi-coding-agent": "^0.85.1",
-    "@earendil-works/pi-tui": "^0.85.1",
-    "@types/node": "^26.2.0",
-    "tsx": "^4.23.5",
-    "typebox": "^1.3.10",
+    "@earendil-works/pi-ai": "1.1.0",
+    "@earendil-works/pi-coding-agent": "1.1.0",
+    "@earendil-works/pi-tui": "1.1.0",
+    "@types/node": "^26.6.3",
+    "tsx": "^4.23.15",
+    "typebox": "^1.3.34",
     "typescript": "^7.0.2"
   },
   "publishConfig": {
     "access": "public"
   },
   "engines": {
-    "node": ">=20.6.0"
+    "node": ">=22.19.0"
   },
   "dependencies": {
     "@mocito/install-telemetry": "0.1.1"