pi-grok-agent 0.1.0

package/docs/first-class-model.md

@@ -0,0 +1,121 @@
+# Grok Build as a Pi model: design
+
+Version 0.1.0. This page describes the implemented design. For use and settings, see the [README](../README.md) and [usage.md](usage.md).
+
+Goal: `pi --model grok/<id>`, and any Pi caller that selects a model by ID, uses Grok Build as the model.
+Grok runs each turn on its own harness. Its native tools, permission rules, subagents, compaction, and history stay on the Grok side.
+Pi drives the turns, streams the output, keeps the transcript, gates Grok's tools, and can lend extra tools.
+
+Reason: a Grok Build agent session includes Grok's tool harness. A plain completion API would replace that harness with Pi's tools. This provider keeps the harness.
+
+## Layers
+
+| Layer | Owner | Contract | Source |
+| --- | --- | --- | --- |
+| Model registration | Pi | `pi.registerProvider("grok", { streamSimple })`, API `grok-acp` | `src/model.ts`, Pi `docs/custom-provider.md` |
+| Transcript to prompt | Provider | Pi system prompt as `_meta.rules`. New user messages and unmatched tool results as `session/prompt`. For a new Grok session, the earlier Pi transcript as text (last 60,000 characters). | `src/model/provider.ts` |
+| Transport | Provider | One WebSocket to the gateway, many Grok sessions, routed by `sessionId` | `src/model/connection.ts`, `src/client.ts` |
+| Backend | Gateway | One supervised `grok agent leader`. One `grok agent --leader stdio` bridge for each WebSocket. The gateway binds its port first and acquires a leader second, so a launch that loses its port owns nothing to stop. | `scripts/server.ts`, `test/gateway.test.ts` |
+| Tool execution | Grok | Native tools run inside Grok. `tool_call` updates become Pi thinking text `[grok <tool>] …`. | `src/model/session.ts` `onUpdate` |
+| File edit scheme | Grok configuration | `[toolset] file_toolset = "hashline"` in `~/.grok/config.toml` selects `hashline_read`, `hashline_edit`, `hashline_grep`. Otherwise Grok uses its default file tools. | Grok Build configuration, not this repository |
+| Permissions | Grok asks, Pi answers | `session/request_permission` becomes a Pi selection dialog. Headless Pi uses `headlessPermissions`. | `src/model/permissions.ts` |
+| Questions | Grok asks, Pi answers | `_x.ai/ask_user_question` becomes one Pi dialog for each question. | `src/model/questions.ts` |
+| Lent Pi tools | Optional | `mcpServers: [{ type: "http", url: "<gateway>/mcp/<token>" }]`. The gateway relays each MCP message to the owning Pi socket as `_x.ai/mcp/sdk_call`. The call waits until Pi returns the result. | `scripts/server.ts` `handleMcpHttp`, `src/model/session.ts` `onMcp` |
+| Output | Provider | Agent chunks become `text_delta`. Thought chunks become `thinking_delta`. | `src/model/provider.ts` |
+| Gate | Pi hook | `pre_tool_use`: `denyGrokTools` first, then `allowGrokTools`, then the capability mirror classified by Grok's `x.ai/tool` stamp, with `mcpReadOnlyServers` for MCP tools. `/grok perms` sets the mirrored capabilities (`yolo` adds edit, write, and shell), so a deny entry wins in every mode. Grok's own `grokMode` and permission prompts are separate. | `src/model/hooks.ts` `capabilityGate`, `src/model/session.ts` |
+| Guard | Gateway | `ReverseRequestGuard`: one guarded lifetime for each hook, permission prompt, and question. Tiered fail-closed answers when Pi is slow or gone, driven by `pi/gate-ack`; one answer per request, late Pi answers dropped; `ask` mode moves a hook to the dialog deadline. | `scripts/server.ts`, `resolveGuard` in `src/config.ts`, `test/gateway.test.ts` |
+| Enrich | Pi hook | `post_tool_use` after an edit: syntax check or `postEditCheck`. A failure returns as `additionalContext`. | `postEditContext` in `src/model/hooks.ts` |
+| Hold | Pi hook | `stop`: a failed `stopCheck` blocks the end of turn with the output as the reason. | `stopGate` in `src/model/hooks.ts` |
+| Transcript | Pi session | One `grok-tool` custom entry for each native call. Rendered by the extension. Not in model context. | `src/model.ts` |
+| Steering | Pi input event | Mid-turn Enter goes to `_x.ai/interject` and is recorded as `grok-steer`. Alt+Enter passes through as a follow-up. | `src/model/steer.ts` |
+| Media | Pi hook and message | Media tool results are copied to `mediaDir` and shown after the turn as a display-only `grok-media` message. | `src/model/session.ts` `copyMedia`, `src/model.ts` `flushMedia` |
+
+## Turn mapping
+
+1. Pi calls `streamSimple(model, context)`.
+2. First call for a Pi session: `session/new`. Later calls on the same connection reuse the attached session with no request. `session/load` of the stored Grok session occurs only on attach to a connection that has not attached it yet, for example in a new Pi process or after a reconnect.
+3. If a lent Pi tool call is waiting, the newest tool-result message answers it and the Grok turn continues.
+4. Otherwise the new user messages go out as `session/prompt`.
+5. Text, thoughts, and native tool activity stream as Pi events.
+6. A lent tool call ends the Pi assistant message with `toolUse`. Pi runs the tool. The next `streamSimple` resumes the same Grok turn.
+7. Prompt completion ends with `stop`, or `length` for `max_tokens`. Abort sends `session/cancel` and returns an aborted result.
+
+`/grok goal` and `/grok compact` use the same `startPrompt` lifetime as step 4, with a text collector in place of the Pi stream. A command timeout cancels on Grok and frees the session, and the cancelled prompt's late completion is dropped (`test/model.test.ts`).
+
+Grok returns `stopReason: cancelled` both for a Pi cancel and for a rejected permission prompt. The provider maps it to `aborted` only when Pi aborted. Otherwise it maps it to `stop`.
+
+## Tool result visibility
+
+Grok keeps the full result of each native tool in its own context. Pi keeps shortened copies:
+
+| Place | Limit | Source |
+| --- | --- | --- |
+| Thinking text, tool input and output | 400 characters each | `compact` in `src/model/session.ts` |
+| `grok-tool` entry `output` | 8000 characters | `resultText` in `src/model/session.ts` |
+| Rendered `grok-tool` line | 100 characters of input. Expanded: 600 characters of output. | `src/model.ts` |
+
+Results of lent Pi tools go to Grok complete, text and image blocks included (`resolveToolResults`).
+
+## Lent tools policy
+
+`piTools` selects what Pi offers: `extensions` (default), `none`, `all`, or a list of names.
+The default excludes Pi core tools, so Grok's own file and shell tools have no duplicates.
+Grok chooses between its own tools and lent ones.
+
+## Leader routing and the HTTP relay
+
+The design first tried Grok's in-process `sdk_call` channel through the leader. In Grok Build 1.0.41, the leader did not route those calls, because the call parameters carry no session ID. A local Grok fork (branch `pi/sdk-call-session-id`, commit `2201fa2e`) added the session ID. That fork is not published and is not required.
+
+The current design does not use that channel through the leader. Pi registers the lent tools as an HTTP MCP server at the gateway. Grok's own MCP client sends `initialize`, `tools/list`, and `tools/call` as POST requests to `/mcp/<token>`. The gateway learns `token -> Pi socket` from the `session/new` or `session/load` that it forwards. It relays each POST as `_x.ai/mcp/sdk_call` and returns Pi's reply as the HTTP response.
+
+GET gets 405, so Grok's client does not open an event stream. An unknown token gets 404. A request without a JSON-RPC method gets 400, not 401, so Grok treats the server as reachable without an auth challenge. The stock binary runs everything. `PI_GROK_BINARY` selects another build.
+
+## Permission prompts
+
+Grok sends `session/request_permission` for some native calls, for example a shell redirect that writes a file.
+Interactive Pi shows the dialog. Headless Pi answers by `headlessPermissions` (`dialog`, `deny`, `reads`, `allow`). The default `dialog` cancels without a UI.
+`grokMode` (`default`, `auto`, `yolo`) goes to Grok as `_meta.autoMode` or `_meta.yoloMode` on `session/new` and `session/load`. Grok decides which prompts it sends in each mode. `/grok perms yolo` acts only in Pi's hook and does not answer these prompts.
+
+## Steering
+
+Pi's `input` event gives `streamingBehavior`: `undefined` when idle, `steer` for Enter during a turn, `followUp` for Alt+Enter. The steer handler takes only the `steer` case when a Grok session exists. It sends the text to `_x.ai/interject`, records a `grok-steer` entry, and returns `{ action: "handled" }`, so the text does not enter Pi's queue. Slash commands and empty input pass through. If the interject request fails, Pi queues the text normally and shows a notice.
+
+Unit tests cover this with a mocked Grok (`test/steer.test.ts`). The live effect of an interjection on the running Grok turn has no saved evidence in this repository. See the development record below.
+
+The registered `input` handler acts only while the active model's provider is `grok`. After a switch to another model in the same Pi session, mid-turn Enter goes to that model; the stored Grok session is used again on a switch back. `test/extension.test.ts` drives the registered handler across Grok, another provider, no model, and Grok again. An earlier version checked only for a stored Grok session ID and would have taken the steer after the switch.
+
+## Development record
+
+These observations come from development runs of the scripts in `scripts/`. Their JSON results were written to `evidence/`, which is not in this repository. They are not current proof. Run the probes again for current results ([usage.md](usage.md#live-probes)). Observations about Grok internals come from reading Grok Build source, which is not in this repository.
+
+| Area | Observation at the time | Script |
+| --- | --- | --- |
+| Native harness | `pi -p --model grok/grok-4.7` read a token and wrote a file with Grok's native tools. Pi executed zero tools. With `piTools: all` and `headlessPermissions: allow`, Grok still used native tools. | `scripts/model-live.sh gateway` |
+| Hashline | With the hashline toolset set, a one-line change used `hashline_grep` then `hashline_edit`. Other lines were unchanged. This run used the local Grok fork. | Manual run |
+| Lent tools | Grok found and called a Pi-only tool through `/mcp/<token>`, waited 5 s for the held result, and answered with the token that Pi held. | `scripts/model-probe.ts` |
+| Guard | A Pi that never acked was denied at the `ackMs` tier (about 5 s, and about 3 s with a setting of 3000). A Pi that dropped its socket during a permission prompt: the file was not created. A dialog answered after an ack was used. | `scripts/gateway-guard-probe.ts` |
+| MCP gate | In a read-only session, an MCP tool with `_meta.readOnlyHint` was allowed. An unmarked tool on another server was denied before it reached the server. | `scripts/mcp-gate-probe.ts` |
+| MCP metadata | `_x.ai/mcp/list` returned each tool's `_meta` without MCP `annotations`. | `scripts/mcp-list-probe.ts` |
+| Hooks | A read-only Pi session denied Grok's edit and the file stayed unchanged. A broken edit was repaired in the same turn after the syntax-check context. A `stopCheck` held the turn until `done.txt` existed. | `scripts/hooks-live.sh` |
+| Timing | Hook and permission round trips took less than 1 ms on loopback. Turn totals did not change measurably with hooks on. | `scripts/perm-timing.ts` |
+| Images | `image_gen` returned `{ type: "ImageGen", path, filename, session_folder }` with a JPEG under `~/.grok/sessions/`. An ACP image block was accepted but not seen by the model (`promptCapabilities.image: false`). A file path worked. | `scripts/image-probe.ts` |
+| Reconnect | The gateway restarted between two turns. Turn 2 reconnected and kept context. In two manual leader stops, the gateway respawned the leader once and adopted a bridge-started leader once. The session continued. The probe targets the default gateway only (port 2419 and a PID file); it is not usable for isolated runs. | `scripts/reconnect-probe.ts`, manual `SIGTERM` to the leader |
+| Queue | A second `session/prompt` during a turn was queued by Grok and ran as its own turn afterwards. | `scripts/queue-probe.ts` |
+| Interject | `_x.ai/interject` was accepted at once. In a raw ACP run, the running turn ended early, and the interjection's answer was not in that turn's stream. A later note recorded that no second assistant turn followed. These two notes do not settle whether the running turn uses the interjected text. Treat the live steering effect as unverified. | `scripts/queue-probe.ts interject` |
+
+## Open items
+
+- Steering: verify the live effect of an interjection on the running turn, and check the active model before steering.
+- Pi skills under the Grok model: Pi expands `/skill:x` into the user message, and Grok receives it as plain text. Grok must translate Pi tool names such as `edit` and `bash` to its own tools. Not yet checked live. Earlier live checks ran with `--no-skills`.
+- Media: probe `image_edit` and the video result types. Video shows as a path only.
+- Gateway: add a way to restart the gateway on failure or login, for example a user service. Only the leader is supervised now.
+- Status line: surface `_x.ai/session/setup` and MCP server status.
+- Tasks: Grok's task tools (`spawn_subagent`, `monitor`, and others) are model tools only. A `/grok tasks` command needs a listing method from Grok.
+- Pi stream hooks: call `options.onPayload` and `options.onResponse`, as Pi's custom provider guide asks.
+
+## Limits
+
+- When the prompt response carries no usage report, usage and cost are zero.
+- The offered Pi tool list is read once for each Grok session.
+- Pi's transcript holds Grok tool activity as thinking text and custom entries, not as structured tool calls.
+- Pi compaction does not change Grok's history. `/grok compact` compacts Grok's history.

package/docs/usage.md

@@ -0,0 +1,380 @@
+# pi-grok-agent reference
+
+This page is the reference for settings, commands, and operation. Start with the [README](../README.md) for requirements and the quick start. Design notes are in [first-class-model.md](first-class-model.md).
+
+## Contents
+
+- [Components](#components)
+- [Install options](#install-options)
+- [Models and Pi controls](#models-and-pi-controls)
+- [Slash command /grok](#slash-command-grok)
+- [Grok permission prompts](#grok-permission-prompts)
+- [Lent Pi tools](#lent-pi-tools)
+- [Hooks around Grok tools](#hooks-around-grok-tools)
+- [Gateway guard](#gateway-guard)
+- [Generated media and attached images](#generated-media-and-attached-images)
+- [Settings](#settings)
+- [Environment variables](#environment-variables)
+- [Session lifecycle](#session-lifecycle)
+- [Files and network](#files-and-network)
+- [Run a second, isolated gateway](#run-a-second-isolated-gateway)
+- [Checks](#checks)
+- [Live probes](#live-probes)
+- [Troubleshooting](#troubleshooting)
+
+## Components
+
+| Component | Source | Function |
+| --- | --- | --- |
+| Pi extension | `src/model.ts` | Registers provider `grok`, the `/grok` command, the steer handler, and the renderers for `grok-tool`, `grok-media`, `grok-steer`, and `grok-command`. |
+| Stream adapter | `src/model/provider.ts` | Turns a Pi turn into an ACP `session/prompt` and turns ACP updates into Pi stream events. |
+| Connection | `src/model/connection.ts` | One WebSocket to the gateway. Creates or loads Grok sessions and routes reverse requests. |
+| Session state | `src/model/session.ts` | Turn state, hook answers, lent tool calls, media copies, usage totals. |
+| Hooks | `src/model/hooks.ts` | Tool classification, capability gate, post-edit check, stop check. |
+| Permissions | `src/model/permissions.ts` | Pi dialogs and headless answers for Grok permission prompts. |
+| Questions | `src/model/questions.ts` | Pi dialogs for Grok's `ask_user_question`. |
+| Steering | `src/model/steer.ts` | Sends mid-turn Enter to Grok's `_x.ai/interject`. |
+| Gateway | `scripts/server.ts` | Supervises one `grok agent leader`, bridges each WebSocket to a stdio leader client, relays lent-tool MCP calls, and guards reverse requests. |
+| Settings | `src/config.ts` | Reads `grok-ws.json`, the secret, and environment overrides. Validates guard tiers. |
+
+## Install options
+
+The README uses one command: `pi install npm:pi-grok-agent`. The package contains the gateway. When a Grok turn finds nothing listening on a loopback `ws://` endpoint, the extension starts that gateway (see [Gateway auto-start](#gateway-auto-start)). The gateway runs compiled JavaScript from `dist/`, because Node does not strip TypeScript types under `node_modules`. The extension stays TypeScript: Pi loads it with its own loader.
+
+To run the gateway yourself instead, for example under a service manager, install the command with `npm install -g pi-grok-agent` and run `pi-grok-gateway`. Pi does not put a package's `bin` on `PATH`, so this needs its own install. An extension that finds your gateway running does not start another one.
+
+With a clone, `npm run server` and the extension (`pi -e .` or `pi install .`) come from the same checkout.
+
+Pi does not install dependencies for a local path. It loads the directory in place. Run `npm install --omit=dev` in the clone before the first start. Pi runs the same `npm install --omit=dev` when it installs a git source.
+
+A clone also auto-starts the gateway: it runs `scripts/server.ts` from the checkout. A git install (`pi install git:github.com/JangMan-J/pi-grok-agent`) takes the same path; its auto-start is not yet tested live. All install paths are recorded in [launch-verification.md](launch-verification.md).
+
+### Gateway auto-start
+
+- Trigger: a Grok turn opens the connection, and nothing accepts TCP on the configured loopback `ws://` endpoint. A `wss://` endpoint is never started.
+- Process: the gateway of the installed version runs as a detached process with its own session, working directory `~`, and Pi's environment. It writes to `<agent dir>/grok-ws.log`. The first turn waits until the port accepts connections, about 5 seconds with a cold leader.
+- Lifetime: the gateway keeps running after Pi exits. Every Pi process on the machine shares it. Stop it with `pkill -INT -f 'pi-grok-agent/(dist/)?scripts/server'`; it stops its leader.
+- Two Pi processes that start at the same time are safe. The gateway binds its port before it starts or adopts a leader, so the second one exits with `EADDRINUSE` and both connect to the first.
+- `/grok debug` shows `auto-start on` or `off`, and the pid when this Pi process started the gateway.
+- Turn it off with `"autoStartGateway": false` in `grok-ws.json` or `PI_GROK_AUTOSTART=0`. Then start the gateway yourself before the first Grok turn.
+
+## Models and Pi controls
+
+| Model ID | Name | Pi thinking levels mapped to Grok reasoning effort |
+| --- | --- | --- |
+| `grok/grok-4.7` | Grok 4.7 | low, medium, high, xhigh |
+| `grok/grok-4.7-build-fast` | Grok 4.7 Build Fast | low, medium, high, xhigh |
+| `grok/grok-4.6` | Grok 4.6 | low, medium, high, xhigh |
+| `grok/grok-4.5` | Grok 4.5 | low, medium, high |
+
+Each model has a 500,000-token context window and a 32,000-token output limit in Pi's metadata. Per-token cost is zero in the metadata. The turn cost comes from Grok's `turn_completed` report, converted at 1e9 ticks per US dollar. That ratio is inferred from Grok's rates. It is not documented by Grok.
+
+Pi controls work as usual:
+
+| Pi control | Effect on Grok |
+| --- | --- |
+| Thinking level (Shift+Tab or `/thinking`) | Sets Grok's `reasoning_effort` session option when the level changes. |
+| Escape | Sends `session/cancel`. The next message starts a new Grok prompt. |
+| Enter during a Grok turn | Sends the text to `_x.ai/interject` and records a `grok-steer` entry. The text does not enter Pi's queue. Slash commands are not steered. Grok accepts the request, but its effect on the running turn is not verified live. A `grok-steer` entry is not proof that Grok used the text. |
+| Alt+Enter during a Grok turn | Queues a Pi follow-up. It becomes the next Grok prompt. |
+| `/new` | New Pi session and new Grok session. |
+| `/compact` | Compacts Pi's transcript only. Use `/grok compact` for Grok's history. |
+
+Other Pi extensions that select a model by ID can use the same IDs, for example `grok/grok-4.7`.
+
+The steer handler acts only while the active model is `grok/*`. After a switch to another model in the same Pi session, mid-turn Enter goes to that model. The stored Grok session is used again when you switch back (`test/extension.test.ts`).
+
+## Slash command /grok
+
+`/grok` covers Grok features that Pi has no control for. `/grok` without a subcommand does nothing. The completion menu shows the subcommands and their arguments.
+
+| Command | Effect |
+| --- | --- |
+| `/grok login` | Runs `grok login --device-auth` in the background and shows the URL and code as an entry and a notice. Grok may open the page itself, in your default browser. Approve it there; Pi reports when the login finished. Works before any Grok session exists. A running gateway picks up the new login on the next turn, without a restart. |
+| `/grok debug` | Shows the gateway URL and connection state, the Grok session ID, Grok mode, Pi permission mode, Grok context size, usage and cost totals, lent tools, and hook decision counts. |
+| `/grok perms` | Shows the Pi permission mode. |
+| `/grok perms auto` | Default. Mirrors the Pi session's tools onto Grok's tools. |
+| `/grok perms read-only` | Denies Grok's edit and shell tools, whatever tools the Pi session has. |
+| `/grok perms ask` | Mirrors, then shows a Pi confirm dialog for each Grok edit or shell call. Without a UI it acts as `read-only`. |
+| `/grok perms yolo` | Treats the Pi session as if it had `read`, `edit`, `write`, and `bash`, so the capability mirror allows Grok's edit and shell tools with no Pi confirm dialog. `denyGrokTools` still denies. Grok's own permission prompts still arrive. For autonomous work in a workspace you can lose. |
+| `/grok plan on`, `/grok plan off` | Sets Grok's session mode to `plan` or `default` with `session/set_mode`. |
+| `/grok goal <objective>`, `goal status`, `goal pause`, `goal resume`, `goal clear` | Sends Grok's `/goal` command. This is a Grok turn outside Pi's model loop. |
+| `/grok compact [note]` | Sends Grok's `/compact` on Grok's own history. This is a Grok turn. |
+
+`/grok goal` and `/grok compact` run through the same prompt lifetime as a normal turn. While one runs, the session is busy: a second command reports `Grok is busy with a turn`, and a normal message waits. A command times out after 10 minutes. The timeout cancels the prompt on Grok (`session/cancel`) and frees the session, and a late reply from the cancelled prompt is dropped.
+
+The `perms` mode applies to Grok tool calls through the `pre_tool_use` hook, before Grok's own permission rules. It lasts for the Pi process. It does not change `grokMode` or `headlessPermissions`.
+
+## Grok permission prompts
+
+Three separate settings affect Grok's tool calls:
+
+| Setting | Where it acts | Values |
+| --- | --- | --- |
+| `/grok perms` | Pi's `pre_tool_use` hook, before Grok runs the tool | `auto` (default), `read-only`, `ask`, `yolo` |
+| `grokMode` | Grok's own permission mode, sent with `session/new` and `session/load` | `default` (default), `auto` (sets Grok's `autoMode`), `yolo` (sets Grok's `yoloMode`) |
+| `headlessPermissions` | Pi's answer to a Grok permission prompt when Pi has no UI | `dialog` (default), `deny`, `reads`, `allow` |
+
+A call must pass Pi's hook first. `/grok perms yolo` does not answer Grok's permission prompts, and `denyGrokTools` wins in every mode. `grokMode` decides which prompts Grok sends. Grok, not this package, defines what `auto` and `yolo` skip.
+
+Grok asks for permission for some native tool calls, for example a shell command that writes a file with a redirect. Interactive Pi shows a selection dialog with Grok's options. If you dismiss the dialog, Grok gets `cancelled` and ends the turn.
+
+Headless Pi (`pi -p`, RPC, or another agent process without a UI) uses `headlessPermissions`:
+
+| Value | Headless answer |
+| --- | --- |
+| `dialog` (default) | Cancel the prompt. Grok reports the tool as cancelled and ends the turn. |
+| `deny` | Reject once. |
+| `reads` | Allow once for prompts of kind read, search, fetch, or think. Reject the rest. |
+| `allow` | Allow once. |
+
+A rejected or cancelled prompt ends the Grok turn. Pi records a completed message, not an abort. Pi records an abort only when Pi itself cancelled the turn.
+
+When Grok calls `ask_user_question`, Pi shows one dialog for each question. The dialog has Grok's options and `Other` for free text. In plan mode it also has `Chat about this` and `Skip interview`. Headless Pi answers `cancelled`.
+
+## Lent Pi tools
+
+Pi can offer its own tools to Grok in addition to Grok's tools. `piTools` in `grok-ws.json`, or `PI_GROK_PI_TOOLS`, selects them:
+
+| Value | Tools offered to Grok |
+| --- | --- |
+| `extensions` (default) | Pi tools other than `read`, `bash`, `edit`, `write`, `grep`, `find`, `ls` |
+| `none` | No Pi tools |
+| `all` | Every Pi tool |
+| `a,b,c` | The named tools |
+
+Grok sees an offered tool as `pi__<name>`. When Grok calls it, the Pi assistant message ends with a tool call, Pi executes the tool through its own loop and permission gates, and the same Grok turn continues with the result. Pi passes the complete tool result to Grok.
+
+The gateway serves the lent tools as an HTTP MCP server at `http://127.0.0.1:2419/mcp/<token>`. It relays each MCP message to the Pi connection that registered the token. Grok connects to that server with its own MCP client, so the stock `grok` binary works.
+
+Grok reads the tool list once for each Grok session. If Grok has an equivalent native tool, it usually uses its own tool.
+
+## Hooks around Grok tools
+
+The provider registers Grok client hooks in every Grok session it creates. Grok still executes each tool. Pi decides and adds context around it.
+
+| Hook | What Pi does |
+| --- | --- |
+| `pre_tool_use` | Applies the `/grok perms` mode and the capability mirror. A Pi session without `edit` or `write` denies Grok's edit tools. A session without `bash` denies Grok's shell. The deny reason goes to Grok. |
+| `post_tool_use` | After a Grok edit, runs a syntax check on the file or the configured `postEditCheck`. A failure goes to Grok as additional context in the same turn. Records a `grok-tool` entry and copies media. |
+| `post_tool_use_failure` | Records a failed `grok-tool` entry. |
+| `stop` | Runs the configured `stopCheck` at the end of a turn. A non-zero exit blocks the end of the turn and gives Grok the output. Grok limits the number of continuations. |
+
+Classification uses Grok's `x.ai/tool` stamp (kind and `read_only`) on each tool call. A table of tool names is the fallback. In a read-only Pi session, a stamped tool that Grok marks as mutating is denied when the table does not know it.
+
+MCP and plugin tools arrive as `server__tool` through Grok's `use_tool` dispatcher. In a read-only Pi session they are denied, unless the tool's `_meta` has `readOnlyHint: true` or the server is in `mcpReadOnlyServers`. Grok 1.0.41 does not forward MCP `annotations`, so a server must put `readOnlyHint` in `_meta`. Lent Pi read tools do this.
+
+Built-in post-edit checks:
+
+| File | Check |
+| --- | --- |
+| `.ts`, `.mts`, `.cts` | Node type stripping, then a module parse. The code does not run. |
+| `.js`, `.mjs`, `.cjs` | `node --check` |
+| `.py` | `python3 -m py_compile` |
+| `.json` | `JSON.parse` |
+| `.rs` | `rustfmt --check --edition 2021` |
+
+Precedence: `denyGrokTools`, then `allowGrokTools`, then the capability mirror, where a tool's `_meta` read-only marker and `mcpReadOnlyServers` apply to MCP tools. `/grok perms` changes the capabilities that the mirror uses, so `denyGrokTools` wins in every mode. In `ask` mode, an allowed edit or shell call then gets a Pi confirm dialog. Hook errors fail open, as Grok's own hooks do.
+
+Each Grok tool call becomes a `grok-tool` session entry with the tool, input, status, output (up to 8000 characters), and duration. The transcript shows one line for each call. The expanded view shows up to 600 characters of output. No model receives these entries.
+
+## Gateway guard
+
+When a Grok client hook times out, Grok continues as if the hook allowed the call. Grok waits with no limit for a permission prompt. So the gateway answers for Pi when Pi cannot answer:
+
+| Tier | Condition | Answer |
+| --- | --- | --- |
+| 0 | Pi answers | Pi's answer |
+| 1 | Pi sent `pi/gate-ack` with `dialog: true` | Wait `dialogMs`, then reject |
+| 1 | Pi sent `pi/gate-ack` with `check: true` | Wait `checkBudgetMs`, then continue |
+| 1 | Pi sent `pi/gate-ack` without flags | Wait `policyMs`, then deny or reject |
+| 2 | No ack in `ackMs`, or the Pi socket closed | Deny (`pre_tool_use`), continue (`post_tool_use`, `stop`), reject (permission), cancel (question) |
+
+Each request has one guarded lifetime on the gateway. Pi's answer settles it and is forwarded. Once the gateway has answered for Pi, a later answer from Pi is dropped, so Grok gets exactly one response per request. When `/grok perms ask` opens a confirm dialog for a `pre_tool_use` hook, Pi sends a second ack with `dialog: true`, and the request moves from the policy deadline to the dialog deadline. `test/gateway.test.ts` checks these paths on the real wire with a fake Grok binary.
+
+Defaults: `ackMs` 5000, `policyMs` 15000, `checkBudgetMs` 590000, `dialogMs` 600000. The settings loader refuses `ackMs` or `policyMs` at or above 30000 (the `pre_tool_use` hook timeout that this package registers), `checkBudgetMs` at or above 600000 (Grok's hook limit), and `ackMs` above `policyMs`.
+
+## Generated media and attached images
+
+| Direction | Behavior |
+| --- | --- |
+| Grok to Pi | Tool results of type `ImageGen`, `ImageEdit`, `ImageToVideo`, `ReferenceToVideo`, or `VideoGen` are copied to `mediaDir` (default `.pi/grok-images/` under the Pi working directory, with a `.gitignore` that ignores all files). The `grok-tool` entry shows `saved <path>`. After the turn, a `grok-media` message shows the path and, for PNG, JPEG, WebP, and GIF, the image inline where the terminal supports images. |
+| Pi to Grok | Attached images are written to `pi-grok-images/` in the system temp directory with mode 0600. The prompt refers to the file path. Grok reads the file with its own tools. |
+
+PNG shows directly. JPEG, WebP, and GIF are converted to PNG with `magick` for display and cached in the temp directory. Without `magick`, the message shows only the path for those formats. Video shows as a path only. Set `mediaDir` to an empty string to keep only Grok's original path.
+
+The `grok-media` message is for display only. The provider removes it from the prompt, so Grok does not receive its own image back.
+
+Development runs probed `image_gen` only. `scripts/image-probe.ts` repeats that check. Image edit and the video types are recognized by name in code and are not yet probed.
+
+## Settings
+
+Optional settings file: `~/.pi/agent/grok-ws.json`. If `PI_CODING_AGENT_DIR` is set, the file is in that directory.
+
+```json
+{
+  "url": "ws://127.0.0.1:2419/ws",
+  "secretFile": "~/.pi/agent/grok-ws.secret",
+  "piTools": "extensions",
+  "headlessPermissions": "dialog",
+  "mediaDir": ".pi/grok-images",
+  "grokMode": "default",
+  "hooks": {
+    "denyGrokTools": [],
+    "allowGrokTools": [],
+    "mcpReadOnlyServers": [],
+    "postEditCheck": "",
+    "stopCheck": ""
+  },
+  "guard": { "ackMs": 5000, "policyMs": 15000, "checkBudgetMs": 590000, "dialogMs": 600000 }
+}
+```
+
+| Key | Meaning |
+| --- | --- |
+| `url` | Gateway WebSocket URL. A non-loopback URL must use `wss://`. The gateway itself accepts only a loopback `ws://` URL that ends in `/ws`. |
+| `secretFile` | Absolute path or a path that starts with `~/`. |
+| `autoStartGateway` | `true` (default) or `false`. See [Gateway auto-start](#gateway-auto-start). |
+| `grokMode` | Grok's own permission mode: `default`, `auto`, or `yolo`. Sent each time Pi attaches a Grok session. Separate from `/grok perms`. See [Grok permission prompts](#grok-permission-prompts). |
+| `hooks.denyGrokTools`, `hooks.allowGrokTools` | Regular expressions that match the whole Grok tool name. |
+| `hooks.mcpReadOnlyServers` | MCP server names whose tools count as read-only. |
+| `hooks.postEditCheck` | Command after a Grok edit. `{file}` is the edited file. Replaces the built-in checks. |
+| `hooks.stopCheck` | Command at the end of a turn. A non-zero exit holds the turn. |
+
+Example with checks:
+
+```json
+{
+  "hooks": {
+    "denyGrokTools": ["web_search", "image_.*"],
+    "postEditCheck": "npx tsc --noEmit -p .",
+    "stopCheck": "npm test"
+  }
+}
+```
+
+## Environment variables
+
+| Variable | Overrides or sets |
+| --- | --- |
+| `GROK_ACP_URL` | `url` |
+| `GROK_AGENT_SECRET` | The secret. The gateway then does not create a secret file. |
+| `PI_CODING_AGENT_DIR` | Pi's agent directory, which holds `grok-ws.json` and the secret |
+| `PI_GROK_PI_TOOLS` | `piTools` |
+| `PI_GROK_HEADLESS_PERMISSIONS` | `headlessPermissions` |
+| `PI_GROK_MEDIA_DIR` | `mediaDir` |
+| `PI_GROK_GROK_MODE` | `grokMode` |
+| `PI_GROK_AUTOSTART` | `autoStartGateway`. `0`, `false`, `no`, or `off` turn it off. |
+| `PI_GROK_DENY_TOOLS` | `hooks.denyGrokTools`, comma-separated |
+| `PI_GROK_POST_EDIT_CHECK` | `hooks.postEditCheck` |
+| `PI_GROK_STOP_CHECK` | `hooks.stopCheck` |
+| `PI_GROK_ACK_MS`, `PI_GROK_POLICY_MS`, `PI_GROK_CHECK_BUDGET_MS`, `PI_GROK_DIALOG_MS` | `guard` values |
+| `PI_GROK_LEADER_SOCKET` | Gateway only. Leader socket path. Default `~/.grok/pi/leader.sock`. |
+| `PI_GROK_BINARY` | Gateway only. Grok executable. Default `grok`. |
+
+## Session lifecycle
+
+- The first turn in a Pi session sends `session/new`. Pi stores the Grok session ID in a `grok-model-session` entry, outside model context.
+- Ordinary later turns reuse the same connection and the attached Grok session. They send no `session/load`.
+- `session/load` with the stored ID occurs only when Pi attaches a stored session that this connection has not attached yet: for example, when a new Pi process resumes the session, and after a reconnect.
+- A Pi fork or tree navigation starts a new Grok session.
+- Pi sends its system prompt as `_meta.rules` and only the new user messages or tool results as the prompt.
+- If the gateway restarts during a session, the next turn reconnects and loads the same Grok session. The turn in progress at the drop is lost.
+- The gateway supervises the leader. If the leader exits, the gateway closes the bridges and starts a new leader or adopts one that a bridge started.
+- After you change files under `src/model/`, start a new `pi` process. `/reload` can keep the provider module that Pi already imported.
+
+## Files and network
+
+| Item | Created by | Content |
+| --- | --- | --- |
+| `~/.pi/agent/` | Gateway | Directory, mode 0700, if missing |
+| `~/.pi/agent/grok-ws.secret` | Gateway, first start | Random bearer secret, mode 0600 |
+| `~/.pi/agent/grok-ws.json` | You | Optional settings |
+| `~/.pi/agent/grok-ws.log` | Auto-started gateway | Gateway output, appended, mode 0600 |
+| `~/.grok/pi/leader.sock` and `leader.lock` | Grok leader | Leader socket. It is outside Grok's `leader-*.sock` discovery pattern, so the Grok TUI does not attach to it. |
+| `.pi/grok-images/` in the Pi working directory | Extension | Copies of Grok media, with a `.gitignore` |
+| `pi-grok-images/` in the system temp directory | Extension | Attached images for Grok and PNG copies for display |
+| Pi session file | Pi | `grok-model-session`, `grok-tool`, `grok-steer`, and `grok-command` entries |
+| Grok session data | Grok | Stored by Grok under `~/.grok/` |
+
+| Connection | Direction | Authentication |
+| --- | --- | --- |
+| `ws://127.0.0.1:2419/ws` | Pi to gateway | `Authorization: Bearer <secret>` header |
+| `http://127.0.0.1:2419/mcp/<token>` | Grok to gateway | Token in the path. Unknown tokens get 404. |
+| Leader socket | Gateway bridges to Grok leader | Local Unix socket |
+| Grok's own network use | Grok | Managed by Grok Build, not by this package |
+
+## Run a second, isolated gateway
+
+Use this for a demo or a test next to a gateway that is in use. A second gateway with the default settings exits with `EADDRINUSE` before it touches any leader, so the running gateway is unaffected. It still needs its own port, leader socket, and agent directory to run.
+
+Set the same variables in the gateway terminal and in the Pi terminal:
+
+```sh
+export PI_CODING_AGENT_DIR="$HOME/.pi-grok-isolated/agent"
+export GROK_ACP_URL=ws://127.0.0.1:2429/ws
+export PI_GROK_LEADER_SOCKET="$HOME/.grok/pi/isolated-leader.sock"
+```
+
+Then run `npm run server` in the clone, and in the other terminal run `pi -e <path-to-clone> --model grok/grok-4.7`. The gateway creates the leader directory `~/.grok/pi/` only, so keep the socket in that directory. `PI_CODING_AGENT_DIR` also gives Pi a separate agent directory with its own settings and sessions.
+
+To clean up, press Ctrl+C in the gateway terminal. This stops its bridges and its leader. Then remove `~/.pi-grok-isolated`.
+
+Do not run `scripts/reconnect-probe.ts` for isolated validation. These variables do not isolate it. See [Live probes](#live-probes).
+
+## Checks
+
+```sh
+npm install          # includes TypeScript and type packages
+npm run check        # tsc --noEmit
+npm test             # node --test test/*.test.ts, no Grok calls
+```
+
+`test/gateway.test.ts` runs the real `scripts/server.ts` with `test/fixtures/fake-grok.ts` as the Grok binary (`PI_GROK_BINARY`) in a scratch `HOME`. It covers a launch that loses its port, leader ownership through shutdown, the guard tiers, the one-answer rule, disconnect, and `ask` mode's dialog deadline.
+
+The unit tests cover the turn split around a lent tool call, abort and resend, prompt tail selection, display-only messages, usage mapping, tool classification and gates, `/grok perms` modes, guard tier validation, question dialogs, steering with a mocked Grok, the steer handler across a model switch, `/grok` command timeout and completion through the shared prompt lifetime, and media copies.
+
+## Live probes
+
+The scripts in `scripts/` run against a running gateway and your Grok login. They cost Grok usage. Several of them allow Grok's permission prompts inside temporary directories. They write JSON results to `evidence/` in the repository root. That directory is not in the repository, and it is not ignored by git. Create it first:
+
+```sh
+mkdir -p evidence
+npm run test:live    # model-live.sh gateway, then hooks-live.sh
+```
+
+| Script | Checks | Writes |
+| --- | --- | --- |
+| `scripts/model-live.sh gateway` | A real `pi -p --model grok/grok-4.7` reads and writes files. Records whether Pi or Grok executed the tools. | `evidence/model-live-gateway-<policy>.json` |
+| `scripts/hooks-live.sh` | Read-only session denies an edit. A broken edit is repaired after the check context. The stop check holds the turn. | `evidence/hooks-live.json` |
+| `scripts/model-probe.ts` | Grok finds and calls a Pi-only tool through the MCP relay and waits for a held result. | `evidence/model-probe.json` |
+| `scripts/gateway-guard-probe.ts` | A hung Pi is denied at the ack tier. A closed Pi socket rejects. A late dialog answer after an ack is used. | `evidence/gateway-guard-probe.json` |
+| `scripts/mcp-gate-probe.ts` | A marked MCP tool is allowed and an unmarked one is denied in a read-only session. | `evidence/mcp-gate-probe.json` |
+| `scripts/question-probe.ts` | `ask_user_question` round trip and `/grok` command paths. | `evidence/question-probe.json` |
+| `scripts/image-probe.ts` | `image_gen` result shape and file location. Inbound image by path. | `evidence/image-probe.json` |
+| `scripts/queue-probe.ts [interject]` | A second prompt or an interject while a turn runs. Prints a timeline. | Standard output only |
+| `scripts/reconnect-probe.ts` | Restarts the gateway, or stops the leader, between two turns. Turn 2 must keep context. | `evidence/reconnect-probe.json` or `evidence/reconnect-probe-leader.json` |
+| `scripts/hooks-probe.ts` | Raw ACP hook frames around a native tool call. | `evidence/hooks-probe.json` |
+| `scripts/perm-timing.ts`, `scripts/usage-probe.ts` | Permission round-trip timing and usage frames. | Standard output only |
+
+`scripts/reconnect-probe.ts` is not isolated. It hardcodes port 2419 and reads a gateway PID from `~/.pi/agent/grok-ws.pid`, a file that `npm run server` does not write. It sends SIGTERM to that PID (gateway mode) or to its `agent leader` child (leader mode). `GROK_ACP_URL`, `PI_CODING_AGENT_DIR`, and the other variables do not change these targets. Run it only when the default gateway on 2419 is disposable and the PID file names it. Its check for other clients looks only at port 2419.
+
+Sanitized results from the run recorded in [launch-verification.md](launch-verification.md) are in `evidence/`. Treat the probes as reproducible checks, and run them again for current results. Probe output and `evidence/` files can contain private paths, session IDs, and tokens. Review and redact them before you share them.
+
+## Troubleshooting
+
+| Symptom | Cause and action |
+| --- | --- |
+| `Grok gateway secret not found at …` on a Grok turn | The gateway never ran with this agent directory. Start `pi-grok-gateway` (or `npm run server` in a clone) once; it creates the file. Then send the message again. Pi does not need a restart. |
+| Pi reports that the extension failed to load, with `ENOENT` for `grok-ws.secret` | A version before the lazy secret read. The gateway never ran with this agent directory. Run `npm run server` once, or set `GROK_AGENT_SECRET` in both terminals. |
+| Pi does not know the model `grok/grok-4.7` | The extension did not load. Use `pi -e <path-to-clone>` or `pi install <path-to-clone>`, and check the load error at startup. |
+| `Grok WebSocket handshake failed. Check the endpoint, server, and secret.` | The gateway is not running, the URL is different, or the two terminals use different secrets. Compare `GROK_ACP_URL` and `PI_CODING_AGENT_DIR` in both terminals. |
+| Gateway exits with `EADDRINUSE` | The port is in use. The gateway binds its port before it starts or adopts a leader, so the other gateway keeps running. See [Run a second, isolated gateway](#run-a-second-isolated-gateway). |
+| `The local launcher requires a loopback ws:// endpoint ending in /ws.` | The gateway URL is not local. The gateway serves loopback only. |
+| `Grok leader socket startup timed out.` or `Grok leader exited before startup.` | Check `grok --version`, `grok login`, and `PI_GROK_BINARY`. |
+| `Grok Build is not signed in. Run /grok login, ...` | Grok has no stored login. Run `/grok login` and approve the code, then send the message again. Pi's `/login` xAI entry does not sign in Grok Build: Grok authenticates agent sessions only with its own stored login, and `XAI_API_KEY` does not replace it. |
+| `Grok connection dropped mid-turn (…)` | The gateway or leader restarted. Send the message again. |
+| `No Grok session yet. Send a message first.` | `/grok plan`, `goal`, and `compact` need a Grok session. Send one prompt first. |
+| Headless Pi ends the turn after a permission prompt | The default `headlessPermissions` is `dialog`, which cancels. Set `deny`, `reads`, or `allow`. |
+| No inline image | The terminal has no image support, or `magick` is missing for a JPEG, WebP, or GIF. The path is still shown. |
+| No hashline tools | `~/.grok/config.toml` does not set `[toolset] file_toolset = "hashline"`. This is expected. |

package/package.json

@@ -0,0 +1,70 @@
+{
+  "name": "pi-grok-agent",
+  "version": "0.1.0",
+  "license": "Apache-2.0",
+  "type": "module",
+  "description": "Pi model provider for Grok Build: Grok keeps its native harness and tools while Pi drives the session over a local WebSocket ACP gateway",
+  "keywords": [
+    "pi-package",
+    "pi",
+    "pi-coding-agent",
+    "pi-extension",
+    "model-provider",
+    "grok",
+    "grok-build",
+    "xai",
+    "acp",
+    "agent-client-protocol",
+    "coding-agent"
+  ],
+  "homepage": "https://github.com/JangMan-J/pi-grok-agent#readme",
+  "bugs": {
+    "url": "https://github.com/JangMan-J/pi-grok-agent/issues"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/JangMan-J/pi-grok-agent.git"
+  },
+  "engines": {
+    "node": ">=22.19.0"
+  },
+  "bin": {
+    "pi-grok-gateway": "dist/scripts/server.js"
+  },
+  "files": [
+    "src",
+    "dist",
+    "docs/usage.md",
+    "docs/first-class-model.md",
+    "README.md",
+    "LICENSE"
+  ],
+  "pi": {
+    "extensions": [
+      "./src/model.ts"
+    ]
+  },
+  "scripts": {
+    "test": "node --test test/*.test.ts",
+    "check": "tsc --noEmit",
+    "build": "tsc -p tsconfig.build.json",
+    "prepack": "npm run build",
+    "server": "node scripts/server.ts",
+    "test:live": "scripts/model-live.sh gateway && scripts/hooks-live.sh"
+  },
+  "dependencies": {
+    "@agentclientprotocol/sdk": "1.5.0",
+    "ws": "8.22.0",
+    "zod": "4.6.5"
+  },
+  "peerDependencies": {
+    "@earendil-works/pi-ai": "*",
+    "@earendil-works/pi-coding-agent": "*",
+    "@earendil-works/pi-tui": "*"
+  },
+  "devDependencies": {
+    "@types/node": "^26.0.0",
+    "@types/ws": "^8.18.1",
+    "typescript": "^6.0.0"
+  }
+}