@earendil-works/pi-coding-agent 0.87.0 → 0.87.1

package/docs/compaction.md

@@ -1,6 +1,6 @@
-# Compaction & Branch Summarization
+# Compaction Reference
 
-LLMs have limited context windows. When conversations grow too long, Pi uses compaction to summarize older content while preserving recent work. This page covers both auto-compaction and branch summarization.
+This reference describes automatic compaction, branch summarization, persisted entries, and extension hooks. For the user workflow, see [Sessions and Context](sessions.md#manage-conversation-context).
 
 **Source files** ([pi](https://github.com/earendil-works/pi)):
 - [`packages/coding-agent/src/core/compaction/compaction.ts`](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/src/core/compaction/compaction.ts) - Auto-compaction logic
@@ -20,7 +20,7 @@ Pi has two summarization mechanisms:
 | Compaction | Context exceeds threshold, or `/compact` | Summarize old messages to free up context |
 | Branch summarization | `/tree` navigation | Preserve context when switching branches |
 
-Both use the same structured summary format and track file operations cumulatively. Compaction and branch-summary requests use fresh routing session IDs and, where supported by the provider, disable prompt-cache writes because these one-off prompts are unlikely to be reused.
+Both use closely related structured formats and track file operations cumulatively. Summarization requests disable prompt-cache writes because these one-off prompts are unlikely to be reused.
 
 ## Compaction
 
@@ -97,14 +97,14 @@ persist final assistant response
 
 If recovery compaction fails or is cancelled, Pi keeps the omission edits, appends no compaction, and schedules no internal retry. Existing queued work remains governed by ordinary steering and follow-up rules. `agent_before_settle` sees the repaired projection after recovery processing. Raw transcript history, exports, billing totals, and history-search extensions can still inspect the omitted attempt.
 
-### Split Turns
+### Split user-message spans
 
-A "turn" starts with a user message and includes all assistant responses and tool calls until the next user message. Normally, compaction cuts at turn boundaries.
+A user-message span starts with a user message and includes all turns until the next user message. Normally, compaction cuts at user-message boundaries.
 
-When a single turn exceeds `keepRecentTokens`, the cut point lands mid-turn at an assistant message. This is a "split turn":
+When one user-message span exceeds `keepRecentTokens`, the cut point lands within that span at an assistant message. This is a split user-message span:
 
 ```
-Split turn (one huge turn exceeds budget):
+Split user-message span (one span exceeds budget):
 
   entry:  0     1     2      3     4      5      6     7      8
         ┌─────┬─────┬─────┬──────┬─────┬──────┬──────┬─────┬──────┐
@@ -117,13 +117,13 @@ Split turn (one huge turn exceeds budget):
                                                       └── kept (7-8)
 
   isSplitTurn = true
-  messagesToSummarize = []  (no complete turns before)
+  messagesToSummarize = []  (no earlier user-message spans)
   turnPrefixMessages = [usr, ass, tool, ass, tool, tool]
 ```
 
-For split turns, Pi generates two summaries and merges them:
+For split user-message spans, Pi generates two summaries and merges them:
 1. **History summary**: Previous context (if any)
-2. **Turn prefix summary**: The early part of the split turn
+2. **User-message-span prefix summary**: The early part of the split user-message span
 
 ### Cut Point Rules
 
@@ -145,8 +145,8 @@ Defined in [`session-manager.ts`](https://github.com/earendil-works/pi/blob/main
 interface CompactionEntry<T = unknown> {
   type: "compaction";
   id: string;
-  parentId: string;
-  timestamp: number;
+  parentId: string | null;
+  timestamp: string;
   summary: string;
   firstKeptEntryId: string;
   tokensBefore: number;
@@ -199,11 +199,9 @@ After navigation with summary:
 
 ### Cumulative File Tracking
 
-Both compaction and branch summarization track files cumulatively. When generating a summary, pi extracts file operations from:
-- Tool calls in the messages being summarized
-- Previous compaction or branch summary `details` (if any)
+Default compaction and branch summarization track files cumulatively. Both extract file operations from tool calls in the messages being summarized. Compaction also carries file lists from the previous Pi-generated compaction. Branch summarization carries file lists from Pi-generated branch summaries in the entries it summarizes.
 
-This means file tracking accumulates across multiple compactions or nested branch summaries, preserving the full history of read and modified files.
+File tracking therefore accumulates across default compactions and nested default branch summaries. Pi does not automatically carry file lists from extension-generated summaries whose `fromHook` field is `true`; extensions manage their own `details` format.
 
 ### BranchSummaryEntry Structure
 
@@ -213,8 +211,8 @@ Defined in [`session-manager.ts`](https://github.com/earendil-works/pi/blob/main
 interface BranchSummaryEntry<T = unknown> {
   type: "branch_summary";
   id: string;
-  parentId: string;
-  timestamp: number;
+  parentId: string | null;
+  timestamp: string;
   summary: string;
   fromId: string;      // Entry we navigated from
   usage?: Usage;       // LLM usage that generated the summary
@@ -235,7 +233,9 @@ See [`collectEntriesForBranchSummary()`](https://github.com/earendil-works/pi/bl
 
 ## Summary Format
 
-Both compaction and branch summarization use the same structured format:
+Both formats include Goal, Constraints & Preferences, Progress, Key Decisions, and Next Steps. Compaction summaries also include Critical Context. Branch summaries stop after Next Steps. Pi appends file lists to either format when relevant.
+
+Compaction summaries use this format:
 
 ```markdown
 ## Goal
@@ -302,7 +302,7 @@ pi.on("session_before_compact", async (event, ctx) => {
   const { preparation, branchEntries, customInstructions, reason, willRetry, signal } = event;
 
   // preparation.messagesToSummarize - messages to summarize
-  // preparation.turnPrefixMessages - split turn prefix (if isSplitTurn)
+  // preparation.turnPrefixMessages - user-message-span prefix (if isSplitTurn)
   // preparation.previousSummary - previous compaction summary
   // preparation.fileOps - extracted file operations
   // preparation.tokensBefore - context tokens before compaction
@@ -376,7 +376,7 @@ pi.on("session_compact_failed", async (event, ctx) => {
   const { reason, errorMessage, aborted, willRetry, fromExtension } = event;
   // reason - "manual" (/compact), "threshold", or "overflow"
   // errorMessage - present for non-abort failures
-  // aborted - true for cancelled/aborted compactions
+  // aborted - true for canceled/aborted compactions
   // willRetry - whether the aborted turn would have retried after compaction
   // fromExtension - whether extension-provided compaction content was being used
 });
@@ -460,4 +460,4 @@ Keys are exact, case-sensitive `provider/modelId` values, including any slashes
 
 These resolved values are used for manual compaction, all automatic threshold checks, overflow recovery, and extension-visible `preparation.settings`. Model switches affect subsequent checks and compactions without changing ordinary settings. Compaction already in progress uses the model and settings captured for that operation. Branch summarization settings are unaffected.
 
-Overrides work in both global and project settings. The files merge recursively before lookup, so a global model-specific value beats a project-wide fallback; a project must override that model entry to change it. See [settings.md](settings.md#per-model-compaction-overrides) for details.
+Overrides work in both global and project settings. The files merge recursively before lookup, so a global model-specific value beats a project-wide fallback; a project must override that model entry to change it. See [Settings](settings.md#per-model-compaction-overrides) for details.

package/docs/configuration.md

@@ -0,0 +1,45 @@
+# Configuration
+
+Pi supports user-level and project configuration. User-level configuration lives in the agent directory, which defaults to `~/.pi/agent`. Project configuration lives in `.pi` under the working directory and loads after [project trust](security.md#understand-project-trust) is granted. The only exception is `sessionDir`, which Pi reads before resolving trust so it can locate sessions.
+
+In interactive mode, use `/settings` to change common preferences. For other options, ask Pi to update the configuration or edit the relevant files directly. Run `/reload` after manually changing settings, keybindings, instructions, or resources.
+
+## Agent directory
+
+The agent directory is shown as `<agent-dir>` below. Set its location with the `PI_CODING_AGENT_DIR` environment variable or the SDK's [`agentDir`](sdk.md) option.
+
+| Path | Responsibility |
+|---|---|
+| `<agent-dir>/settings.json` | User-level [settings](settings.md), including preferences, defaults, resource paths, and Pi package declarations. |
+| `<agent-dir>/keybindings.json` | Custom terminal UI and application [keybindings](keybindings.md). |
+| `<agent-dir>/models.json` | [Compatible endpoints, models, and model overrides](models.md#configure-a-compatible-endpoint). |
+| `<agent-dir>/auth.json` | Saved API keys and OAuth credentials. |
+| `<agent-dir>/AGENTS.override.md`, `AGENTS.md`, `AGENTS.MD`, `CLAUDE.md`, or `CLAUDE.MD` | User instructions applied across working directories. |
+| `<agent-dir>/SYSTEM.md` | Replaces Pi’s default system prompt. |
+| `<agent-dir>/APPEND_SYSTEM.md` | Adds instructions to Pi’s system prompt. |
+| `<agent-dir>/extensions/` | User [extensions](extensions.md). |
+| `<agent-dir>/skills/` | User [skills](skills.md) and supporting files. |
+| `<agent-dir>/prompts/` | User [prompt templates](prompt-templates.md) exposed as slash commands. |
+| `<agent-dir>/themes/` | User [theme](themes.md) files. |
+
+## Project `.pi` directory
+
+| Path | Responsibility |
+|---|---|
+| `.pi/settings.json` | Project-level [settings](settings.md), resource paths, and Pi package declarations. |
+| `.pi/SYSTEM.md` | Replaces the system prompt for the project. |
+| `.pi/APPEND_SYSTEM.md` | Adds project-specific instructions to the system prompt. |
+| `.pi/extensions/` | Project extensions. |
+| `.pi/skills/` | Project skills and supporting files. |
+| `.pi/prompts/` | Project prompt templates exposed as slash commands. |
+| `.pi/themes/` | Project theme files. |
+
+For `SYSTEM.md` and `APPEND_SYSTEM.md`, the trusted project file takes precedence over the corresponding agent-directory file. Files with the same name are not combined.
+
+## Context files
+
+Context files are separate from project `.pi` configuration. Pi loads them from the agent directory, the working directory, and its parent directories. A context file applies whenever Pi runs in its directory or anywhere below it.
+
+An `AGENTS.override.md` replaces `AGENTS.md` or `CLAUDE.md` only in the same directory. It does not suppress context files from the agent directory or other directories.
+
+Context-file discovery does not require project trust.

package/docs/containerization.md

@@ -1,53 +1,39 @@
-# Containerization
+# Run Pi in an isolated environment
 
-Pi runs with all permissions by default, but in some cases, you will want to have more control over what directories Pi can write to and which accesses it has.
+Use an isolated environment to limit the files, credentials, processes, and network services that generated commands can access or affect.
 
-There are two general options. You can either
-1. run the whole `pi` process inside an isolated environment, or
-2. run `pi` on the host and route tool execution into an isolated environment.
+You can isolate the complete Pi process or keep Pi on the host and route selected tools into an isolated environment.
 
-## Choose a pattern
+## Choose an isolation method
 
-| Pattern | What is isolated | Best for | Notes |
-| --- | --- | --- | --- |
-| Gondolin extension | Built-in tools and `!` commands | Local micro-VM isolation while keeping auth on host | See [`examples/extensions/gondolin/`](../examples/extensions/gondolin/). |
-| Plain Docker | Whole `pi` process in a local container | Simple local isolation | Provider API keys enter the container. |
-| OpenShell | Whole `pi` process in a policy-controlled sandbox | Local or remote managed sandbox | Requires an OpenShell gateway |
-| Docker Sandboxes | Whole `pi` process in a managed sandbox | Local isolation with provider keys kept on the host | Requires Docker Sandboxes (`sbx`). |
+| Method | Where Pi runs | What is isolated | Credential handling | Best for |
+|---|---|---|---|---|
+| Plain Docker | Container | Pi, built-in tools, `!` commands, and extensions | Credentials passed into the container | A straightforward local container boundary |
+| Docker Sandboxes | Managed sandbox | Pi, built-in tools, `!` commands, and extensions | Provider credentials remain on the host and are substituted by the proxy | Managed local isolation without exposing the real provider key |
+| OpenShell | Local or remote sandbox | Pi, built-in tools, `!` commands, and extensions | Policy-controlled credentials and inference routing | Filesystem, process, network, and credential policies |
+| Gondolin extension | Host | Built-in tools and `!` commands | Stored Pi credentials remain on the host, but commands inherit host environment variables | A local micro-VM for tool execution while retaining the host interface |
 
-Extensions run wherever the `pi` process runs. If you run host `pi` with a tool-routing extension, other custom extension tools still run on the host unless they also delegate their operations.
+The method changes where extensions run. When the complete Pi process runs inside an isolated environment, its extensions run there too. When host Pi delegates built-in tools through Gondolin, other extension tools still run on the host unless they also delegate their work.
 
-## Gondolin
+## Decide what Pi can access
 
-[Gondolin](https://github.com/earendil-works/gondolin) is a local Linux micro-VM.
-Use the [example extension](../examples/extensions/gondolin) when you want `pi` on the host but all built-in tools routed into the VM.
+An isolated process can still affect resources you expose to it:
 
-Setup:
+- A read-write host mount lets Pi modify those host files.
+- Mounting `~/.pi/agent` exposes your Pi credentials, settings, extensions, and sessions.
+- Environment variables passed into a container are available to processes inside it.
+- Network access may allow code or tool output to leave the environment.
+- Tool-only isolation does not constrain the host Pi process or extension tools that do not use the isolated backend.
 
-```bash
-cp -R packages/coding-agent/examples/extensions/gondolin ~/.pi/agent/extensions/gondolin
-cd ~/.pi/agent/extensions/gondolin
-npm install --ignore-scripts
-```
-
-Run from the project you want mounted:
-
-```bash
-cd /path/to/project
-pi -e ~/.pi/agent/extensions/gondolin
-```
+Expose only the working folder, credentials, and network destinations needed for the task. Use read-only mounts or copy files into and out of the environment when you do not want writes to affect the host.
 
-The extension mounts the host cwd at `/workspace` in the VM and overrides `read`, `write`, `edit`, `bash`, `grep`, `find`, and `ls`.
-User `!` commands are routed into the VM, as well.
-File changes under `/workspace` write through to the host.
+## Run Pi in plain Docker
 
-Requirements: Node.js >= 23.6.0 for `@earendil-works/gondolin`, plus QEMU (requires installation through your package manager).
+Plain Docker provides the simplest whole-process container boundary.
 
-## Plain Docker
+### Build the image
 
-Run the whole `pi` process in Docker when you want the simplest local container boundary.
-
-`Dockerfile.pi`:
+Create `Dockerfile.pi`:
 
 ```dockerfile
 FROM node:24-bookworm-slim
@@ -61,11 +47,17 @@ WORKDIR /workspace
 ENTRYPOINT ["pi"]
 ```
 
-Build and run:
+Build it from the directory containing the file:
 
 ```bash
 docker build -t pi-sandbox -f Dockerfile.pi .
+```
 
+### Start Pi
+
+From the working folder you want Pi to access, run:
+
+```bash
 docker run --rm -it \
   -e ANTHROPIC_API_KEY \
   -v "$PWD:/workspace" \
@@ -73,84 +65,119 @@ docker run --rm -it \
   pi-sandbox
 ```
 
-The `-v "$PWD:/workspace"` mounts your current directory into the container at /workspace such that reads and writes in `/workspace` inside Docker directly affect your host files, like in the Gondolin example.
+Replace `ANTHROPIC_API_KEY` with the credential required by your provider. The named `pi-agent-home` volume keeps container-local settings, credentials, and sessions between runs.
+
+Do not mount the host's `~/.pi/agent` unless the container should have access to your host Pi configuration and credentials.
+
+### Verify the workspace
+
+Inside Pi, run:
+
+```text
+!pwd
+```
+
+The command should report `/workspace`. Changes under `/workspace` write through to the mounted host folder. Remove the bind mount or use a read-only mount when that is not acceptable.
+
+## Run Pi with Docker Sandboxes
 
-Use a named volume for `/root/.pi/agent` if you want container-local settings and sessions. Mounting your host `~/.pi/agent` exposes host auth and session files to the container.
+[Docker Sandboxes](https://docs.docker.com/ai/sandboxes/) runs the complete Pi process inside a managed sandbox. Its proxy can keep the real provider credential on the host and substitute it when requests leave the sandbox.
 
-## OpenShell
+Configure credentials before creating the sandbox. Do not run `/login` inside the sandbox because that writes a real credential into it.
 
-Use [NVIDIA OpenShell](https://docs.nvidia.com/openshell/about/overview) when you want a policy-controlled sandbox with filesystem, process, network, credential, and inference controls.
-OpenShell can run sandboxes through a local gateway backed by Docker, Podman, or a VM runtime, or through a remote Kubernetes gateway.
+### Use a Claude Pro or Max token
 
-Every sandbox requires an active gateway.
-Register and select one before creating a sandbox:
+Generate the token with `claude setup-token` on a machine with Claude Code. If an `anthropic` secret is already configured, remove it first so the proxy does not add an API-key header alongside the bearer token:
 
 ```bash
-openshell gateway add <gateway-url> --name <name>
-openshell gateway select <name>
+sbx secret rm anthropic
+
+sbx secret set-custom \
+  --host api.anthropic.com \
+  --env ANTHROPIC_OAUTH_TOKEN \
+  --placeholder 'sk-ant-oat01-{rand}'
 ```
 
-Launch `pi` inside an OpenShell sandbox:
+`sbx secret set-custom` reads the real token from standard input. The sandbox receives an OAuth-shaped placeholder, which the proxy replaces only for requests to the configured host.
+
+For an Anthropic API key, use `sbx secret set anthropic` instead.
+
+### Start Pi
+
+Run this from the working folder you want mounted:
 
 ```bash
-openshell sandbox create --name pi-sandbox --from pi -- pi
+sbx run --kit "docker.io/sbx/pi-kit:latest" pi
 ```
 
-In this pattern, the whole `pi` process runs inside the sandbox.
-Built-in tools, `!` commands, and extension tools execute inside the OpenShell boundary.
-
-If the gateway is remote, project files are not bind-mounted from the host, meaning writes in the sandbox are not reflected on your machine.
-Clone the repository inside the sandbox or use OpenShell file transfer commands:
+For an existing sandbox, run Pi non-interactively with:
 
 ```bash
-openshell sandbox upload pi-sandbox ./repo /workspace
-openshell sandbox download pi-sandbox /workspace/repo ./repo-out
+sbx exec <sandbox-name> -- pi -p "list the failing tests"
 ```
 
-OpenShell providers can keep raw model API keys outside the sandbox.
-When inference routing is configured, code inside the sandbox can call `https://inference.local`, and the gateway injects the configured provider credentials upstream.
-Configure Pi to use the corresponding OpenAI-compatible or Anthropic-compatible endpoint if you want model traffic to use this route.
+See the [Pi kit documentation](https://github.com/docker/sbx-kits-contrib/tree/main/pi) for other providers, troubleshooting, and image pinning.
 
-## Docker Sandboxes
+## Run Pi with OpenShell
 
-[Docker Sandboxes](https://docs.docker.com/ai/sandboxes/) is a managed sandbox runtime from Docker that runs the whole `pi` process inside a sandbox.
-It is one of the container boundaries [No Built-in Sandbox](security.md#no-built-in-sandbox) points to.
+[NVIDIA OpenShell](https://docs.nvidia.com/openshell/about/overview) provides local or remote sandboxes with filesystem, process, network, credential, and inference policies.
 
-Unlike the Plain Docker pattern above, the provider credential is not passed into the container.
-The sandbox receives a sentinel value instead, and the `sbx` proxy substitutes the real credential on egress to `api.anthropic.com`.
-Credentials are wired at creation time, so store yours on the host before you create the sandbox.
+### Select a gateway
 
-For a Claude Pro/Max subscription, run `claude setup-token` on a machine with Claude Code, then store the result on the host.
-If an `anthropic` secret is already bound, remove it first: otherwise the proxy adds an `x-api-key` header alongside the Bearer token and Anthropic rejects the request.
-`sbx secret set-custom` reads the token from stdin, so it stays out of shell history.
+Every sandbox requires an active gateway:
 
 ```bash
-sbx secret rm anthropic
+openshell gateway add <gateway-url> --name <name>
+openshell gateway select <name>
+```
 
-sbx secret set-custom \
-  --host api.anthropic.com \
-  --env ANTHROPIC_OAUTH_TOKEN \
-  --placeholder 'sk-ant-oat01-{rand}'
+### Create the sandbox
+
+```bash
+openshell sandbox create --name pi-sandbox --from pi -- pi
 ```
 
-The sandbox gets an OAuth-shaped placeholder, not the real token, and the proxy swaps it on egress to that host; `ANTHROPIC_OAUTH_TOKEN` is a variable pi already reads and prefers over an API key, so no extra pi configuration is needed.
+Pi, its built-in tools, `!` commands, and extension tools run inside the OpenShell boundary.
 
-For an API key, store it with `sbx secret set anthropic` instead. The kit wires it the same way, as a sentinel the proxy substitutes on egress.
+### Transfer files to a remote sandbox
 
-With the credential stored, launch `pi` from the project you want mounted:
+A remote gateway does not bind-mount your host working folder. Clone the repository inside the sandbox or transfer files explicitly:
 
 ```bash
-sbx run --kit "docker.io/sbx/pi-kit:latest" pi
+openshell sandbox upload pi-sandbox ./working-folder /workspace
+openshell sandbox download pi-sandbox /workspace/working-folder ./working-folder-out
 ```
 
-The kit pre-bakes `pi` into its image, so the sandbox starts without installing anything, and the current directory is the sandbox workspace.
+OpenShell inference routing can keep raw model credentials outside the sandbox. When configured, point Pi at the corresponding OpenAI-compatible or Anthropic-compatible endpoint exposed by the gateway.
+
+## Route tools through Gondolin
+
+[Gondolin](https://github.com/earendil-works/gondolin) is a local Linux micro-VM. Its example extension keeps the Pi process and file-based provider credentials on the host while routing the built-in tools and user `!` commands into the VM.
+
+Commands inside the VM inherit the host process environment. Provider keys supplied through environment variables can therefore be visible inside the VM. Do not use this pattern as a credential boundary unless you remove sensitive variables or change the extension's environment handling.
 
-Do not authenticate from inside the sandbox: `/login` there writes a real token into the container and defeats the proxy model.
+Gondolin requires Node.js 23.6 or newer and QEMU installed through your operating-system package manager.
 
-Scripted use works the same way:
+### Install the extension
+
+From a Pi source checkout:
 
 ```bash
-sbx exec <sandbox-name> -- pi -p "list the failing tests"
+mkdir -p ~/.pi/agent/extensions
+cp -R packages/coding-agent/examples/extensions/gondolin ~/.pi/agent/extensions/gondolin
+cd ~/.pi/agent/extensions/gondolin
+npm install --ignore-scripts
+```
+
+### Start Pi
+
+Run Pi from the working folder you want mounted:
+
+```bash
+cd /path/to/working-folder
+pi -e ~/.pi/agent/extensions/gondolin
 ```
 
-See the [kit documentation](https://github.com/docker/sbx-kits-contrib/tree/main/pi) for the full credential matrix, troubleshooting, and pinning.
+The extension mounts the host working folder at `/workspace` in the VM and overrides `read`, `write`, `edit`, `bash`, `grep`, `find`, and `ls`. File changes under `/workspace` write through to the host.
+
+Other extension tools still run on the host unless they explicitly delegate their operations. Review the [Gondolin example](../examples/extensions/gondolin/) before adding tools that could bypass the VM boundary.