@zihanw/pi-forge 0.4.0-beta.1 → 0.4.0

package/docs/design/subagents/interface-design.md

@@ -0,0 +1,274 @@
+# Subagent request/response design
+
+Status: historical design and implementation record for the completed 0.4 path. Execution ownership (backend registry, preflight binding, plan sealing, conversation/execution fingerprints, lifecycle, and the fresh-process `pi-subprocess-readonly`/`pi-rpc-readonly` backends) lives in `@zihanw/pi-subagent-runtime`; the former in-package registry and `pi-sdk-isolated` compatibility backend were removed in that migration. The broader Pi SDK spike is recorded in the [SDK spike findings](sdk-spike-findings.md) (harness since removed), and the concrete exported semantics are documented in the [subagent adapter contract](../../reference/subagent-adapter.md).
+
+## Goals
+
+- Make pi-forge profiles usable by native, subprocess, package-provided, or remote backends without requiring every backend to reproduce the parent Pi runtime.
+- Start every subagent with a clean conversational context while allowing explicit, bounded context seeding.
+- Keep prompt stacks as the profile-level source of prompt layout, visible-tool policy, and model-visible skills.
+- Keep access, limits, cancellation, trace storage, and result reporting outside reusable profiles.
+- Return a compact parent-visible tool result while retaining normalized execution history for authorized inspection.
+- Export the demonstrated request/resolution/preflight/plan/response boundary while keeping a full owned runner, orchestration, and automatic delegation out of the package surface.
+
+## Execution Flow
+
+The portable caller request is not itself executable. Execution has five stages:
+
+```text
+AgentRequest
+    -> host profile/stack/dependency resolution
+    -> backend discovery and preflight
+    -> backend-assisted host plan preparation
+    -> backend execution
+    -> AgentResponse
+```
+
+### 1. AgentRequest
+
+The parent supplies intent: profile selection, task/media, explicit selected context, access requirements, limits, and parent-depth provenance. It does not contain parent runtime model objects, loaded files, credentials, raw session entries, or a compiled prompt.
+
+### 2. Host resolution
+
+pi-forge validates profile syntax, resolves the referenced prompt stack, identifies required custom macro/slot registrations, resolves selected parent-context references, and creates immutable declarative provenance. This stage does not claim that the target backend has the model, credentials, tools, isolation, or limit support needed to execute it.
+
+Current parent-runtime resolution used by `/profile use` remains valid for applying profiles to the parent. Subagent host resolution must be a separate operation rather than reusing parent model/auth diagnostics as backend truth.
+
+### 3. Backend discovery and preflight
+
+Before prompt compilation, the selected backend reports:
+
+- Model availability, authentication, thinking-level support, and media support.
+- A stable tool catalog with backend tool IDs, policy-facing names, prompt snippets, and optional adapter mappings.
+- Prompt-runtime inputs needed for Pi-compatible base prompts, tool guidance, skills, and context resources.
+- Workspace/mount materialization support and granular access enforcement.
+- Supported hard limits, cancellation behavior, trace support, and artifact behavior.
+- Remote data-egress requirements.
+
+The host applies the prompt stack's tool policy to the backend catalog. An allow pattern that matches nothing remains a warning unless separate dependency metadata marks that tool or capability as required.
+
+### 4. Backend-assisted plan preparation and AgentExecutionPlan
+
+The host combines the resolved profile, backend preflight receipt, selected context, and prompt-stack compiler into an immutable execution plan. The plan contains prepared system text/messages, exact model and thinking level, effective backend tool IDs, materialized workspace handles, enforced limits, diagnostics, provenance, and a stable execution fingerprint.
+
+Some backends cannot expose exact prompt-runtime inputs during passive preflight. Pi SDK 0.80.6, for example, exposes its exact base prompt, tool snippets, skills, and context options through `before_agent_start`, immediately before provider execution. Such an adapter may supply those inputs to a trusted host preparation callback after accepting the prompt, provided it blocks provider transport until host compilation, protected-task validation, limit checks, and plan finalization succeed. A partial dry preflight must declare its prompt-runtime fidelity and cannot masquerade as the exact execution plan.
+
+Custom macros and slots execute during plan preparation in the trusted host. Backends receive their rendered result and dependency receipt, not executable registration code. A raw `PromptStack` remains snapshot provenance and is not treated as an executable backend artifact.
+
+For a null stack or append/prepend stack mode, the backend preflight must provide the base-prompt inputs required by the selected adapter. If the host cannot reproduce the intended base prompt, plan preparation fails.
+
+### 5. AgentResponse
+
+The backend returns normalized terminal status, output, enforcement receipts, effective tools/limits, artifacts, usage, and a routed trace handle. A separate projection determines what enters the parent model's context.
+
+## Responsibility Boundaries
+
+Effective permissions are the intersection of:
+
+1. Profile prompt-stack tool selection.
+2. Request access and limit requirements.
+3. Backend-advertised capabilities and enforcement.
+
+A backend must reject a required policy it cannot enforce. Prompt tool filtering is not a filesystem, process, or network sandbox.
+
+pi-forge owns profile/stack resolution, trusted prompt compilation, context selection, plan creation, stable fingerprints, diagnostics, and parent-visible response projection. Backends own target-runtime preflight, execution, cancellation, backend-namespace canonicalization, enforcement receipts, usage collection, artifacts, and normalized traces.
+
+## Context Model and Task Preservation
+
+Subagents inherit no parent conversation automatically. Their prepared context contains:
+
+1. System instructions compiled from the resolved profile and backend prompt inputs.
+2. Optional parent-selected context rendered as quoted background evidence with provenance.
+3. A protected final user task containing all request text and media.
+
+Selected context may contain a provenance-bearing summary, visible user/assistant excerpts, quoted tool-result excerpts, and resource/artifact references. It never automatically contains the parent system prompt, full chat history, hidden reasoning, raw provider payloads, secrets, or environment variables.
+
+Tool-result excerpts are background content, not native tool-result messages, because they do not have matching tool calls in the clean subagent history. Delimiters communicate instruction priority but are not a prompt-injection security boundary.
+
+The delegated task is appended as a protected final user message after unrestricted prompt-stack message layout has been compiled. A backend adapter may combine adjacent user messages for provider compatibility, but it must preserve the delegated-context/task boundary and all structured media parts. Subagent preparation fails if the final normalized plan does not contain the complete task.
+
+Context budgeting uses a required character/byte ceiling in v1. Any optional token estimate records the estimator/tokenizer name and version. Truncation removes optional context before required context and never truncates the final task or required media.
+
+## Contract Artifacts
+
+The TypeScript shapes and pure validators for the following artifacts are exported. Backend registration, explicit backend selection, and execution lifecycle are owned by `@zihanw/pi-subagent-runtime`; the extension composition root registers local `forge_subagent_profiles` discovery and drives the runtime's `pi-subprocess-readonly` (default) and `pi-rpc-readonly` backends for `forge_subagent` and `/forge-agent`.
+
+### AgentRequest
+
+- Schema version and host-generated request ID.
+- Profile ID plus an optional expected source fingerprint for optimistic consistency.
+- Structured text/media input.
+- Optional selected-context summary/items with provenance and a character budget.
+- Access requirements expressed through opaque workspace/resource handles, not backend-specific absolute paths.
+- Hard execution limits and a separate parent-result projection limit.
+- Parent run/session provenance and bounded delegation depth.
+- Explicit remote-egress consent when a backend would transmit local project data.
+
+### AgentProfileSnapshot
+
+- Normalized profile fields and canonical source-profile digest.
+- Normalized prompt-stack definition and canonical stack digest, or null.
+- Required custom macro/slot dependency identities when available.
+- No model-registry object, credential/auth result, loaded file path, diagnostics, session state, or secret.
+
+This snapshot is immutable provenance. It is not sent to a backend as a promise that the profile is executable there.
+
+### BackendPreflightResult
+
+- Accepted/rejected state with structured diagnostics.
+- Resolved target model and thinking level.
+- Stable tool catalog and adapter mappings.
+- Prompt-runtime/base-prompt inputs.
+- Workspace materialization/mount mappings in the backend namespace.
+- Granular enforcement capabilities and accepted hard limits.
+- Media, cancellation, trace, artifact, and remote-egress behavior.
+
+### AgentExecutionPlan
+
+- Host-generated run ID, request ID, and backend ID.
+- Prepared system prompt and normalized initial messages.
+- Exact model, thinking level, effective backend tool IDs, and unmatched policy diagnostics.
+- Materialized workspace handles, relative working directory, network policy, and enforcement requirements.
+- Enforced execution limits and separate result-projection limits.
+- Profile/stack provenance, dependency receipt, and backend preflight receipt.
+- Canonical execution fingerprint covering all behavior-affecting prepared inputs.
+
+The run ID exists before backend execution begins so cancellation and trace correlation do not depend on a remote backend first returning its own ID.
+
+### AgentResponse
+
+Response status is a discriminated union:
+
+- `completed`: no error; output may be empty.
+- `failed`: structured error required; optional partial output must be marked partial.
+- `cancelled`: cancellation reason required; output is absent or explicitly partial.
+- `timed-out`: timeout reason and enforced timeout required; output is absent or explicitly partial.
+- `limit-reached`: the reached hard limit is required; output is absent or explicitly partial.
+
+Every response includes request/run/backend correlation, model/profile/execution fingerprints, backend-produced enforcement receipt, effective tools and limits, duration, and artifact/trace routing metadata. `effectiveAccess` is never a request echo.
+
+Token usage records tokenizer/model provenance when known. Cost uses an amount plus ISO currency code rather than an unqualified number. Artifact/change paths live in a named workspace namespace and include authorization, lifetime, and cleanup metadata.
+
+Opaque backend trace IDs are registered behind host trace handles. Inspection authorizes the caller and routes through the backend/host trace registry. Hidden provider reasoning remains excluded by default.
+
+Hard execution limits are distinct from response projection truncation. A backend that cannot enforce a required timeout, turn, token, output, filesystem, or network constraint rejects preflight.
+
+## Access and Data-Egress Model
+
+Requests identify host resources with opaque workspace/resource handles. Backend preflight maps them into backend-visible mounts or uploaded resources. The execution plan uses mount IDs plus normalized relative paths rather than assuming host absolute paths exist remotely.
+
+- `none` permits no filesystem mounts and no filesystem working directory.
+- `read-only` permits only read-only mounts.
+- `workspace-write` permits explicitly identified read-write mounts; all other mounts remain read-only or absent.
+- An access receipt distinguishes an `isolated` execution boundary from an explicitly unsafe `shared-user` process. At the shared-user boundary, access levels describe the model-visible tool policy rather than the child process's operating-system permissions.
+- A working directory, when present, must be contained within an accepted mount.
+- The backend canonicalizes and checks paths in its own namespace immediately before access and must enforce containment against symlink races.
+- Model/provider transport is distinct from agent-accessible network tools.
+- Sending project content to a remote backend requires an explicit egress decision; project trust alone is insufficient consent.
+
+Capabilities are granular claims such as read-only mount isolation, read-write mount isolation, symlink-safe containment, agent-network isolation, timeout enforcement, turn enforcement, token-budget enforcement, output enforcement, cancellation, media transport, artifact retention, and trace inspection. Broad booleans are insufficient.
+
+## Fingerprints and Compatibility
+
+New portable fingerprints use canonical serialization and a named algorithm/version such as `sha256:v1:<digest>`.
+
+- Source-profile fingerprint: normalized declarative profile.
+- Prompt-stack fingerprint: normalized declarative stack.
+- Execution fingerprint: compiled system/messages, exact model/thinking, effective tool IDs/mappings, relevant prompt-runtime inputs, selected-context/resource digests, dependency receipt, materialized access policy, enforced limits, and adapter/preflight version.
+
+The existing `agentProfileFingerprint()` JSON string remains unchanged for stored branch provenance and drift compatibility. Portable snapshot/execution digests are new fields with separate semantics.
+
+## Parent-Visible Result
+
+The complete response is control-plane data. The main agent receives a bounded tool-result projection containing run ID, terminal status, output, and compact errors/warnings when applicable.
+
+The human can expand the same tool result to inspect the plan summary, approval receipt, diagnostics, usage, normalized response, and complete captured subprocess transcript/tool events. Those details stay out of the normal model-visible projection. The full prepared prompt is shown only on demand before approval and is not persisted in result details by default. `/tree` changes the active conversation branch; it cannot undo provider egress, billing, or external effects. Contract trace inspection, artifacts, and resumable sessions remain separate and deferred.
+
+## Revised Implementation Plan
+
+### Integration checkpoint: require a user-visible walking skeleton
+
+Fake-backend conformance proves contract and registry behavior, while the standalone SDK spike proved broader Pi provider behavior. The shipped walking skeleton sends a human-requested task through the complete profile-resolution, runtime-preflight, prompt-preparation, plan-validation, provider-execution, and response-projection path.
+
+The first concrete integration was a pi-forge-owned `pi-sdk-isolated` adapter plus a human-operated plan/run command. The current integration drives the runtime package's foreground `pi-subprocess-readonly` (default) and `pi-rpc-readonly` backends for the command and model-callable tool. It exposes only stack-filtered read/list/search tools, declares a shared-user boundary rather than filesystem isolation, requires explicit approval of an immutable plan bound to runtime-issued fingerprints, and returns bounded output plus expandable reports.
+
+This integration also exposed a contract seam hidden by the fake backend: backend-assisted preparation cannot require the caller to fabricate the prompt runtime that the backend is responsible for discovering. The backend must supply complete, canonical compiler inputs; the host must compile from them; and the registry must return the exact runtime/preparation pair used for execution-plan construction.
+
+### Iteration 1: Shared profile service
+
+- Extract shared typed profile repository operations for load, capture, save/update, and delete.
+- Extract profile application and rollback from command rendering.
+- Extract typed resolution preview and runtime/provenance drift status.
+- Refactor commands to use the service without changing current CLI behavior.
+- Add service-level tests so profile UI and subagent preparation share one implementation.
+
+### Iteration 2: Internal real-backend spike (completed)
+
+- Prototype one in-memory Pi SDK backend using a fresh `SessionManager`, explicit model/thinking, a controlled tool baseline, cancellation, and compact output.
+- Exercise a real profile, restrictive prompt stack, custom macro/slot dependency, media input, and current local provider.
+- Record actual SDK/runtime requirements and failure points.
+- Keep all spike types internal and expose no run tool.
+
+Implemented as the opt-in `scripts/subagent-sdk-spike.ts` (removed in the 0.4 cleanup after its findings were productized in the runtime's shared preparation gate); results and limitations are recorded in `SUBAGENT_SDK_SPIKE_FINDINGS.md`.
+
+### Iteration 3: Resolve/preflight/plan boundary (completed)
+
+- Split parent application resolution from backend-independent host resolution.
+- Define internal request, backend descriptor/preflight, backend-assisted preparation callback, execution-plan, response, enforcement-receipt, and diagnostic types based on the spike.
+- Implement protected task/context preparation and tool negotiation.
+- Validate null/replace/append/prepend stacks and missing custom dependencies.
+
+Implemented in `src/subagent-host.ts` and focused modules under `src/subagent/`, with `src/subagent-contract.ts` retained as a compatibility barrel. The SDK spike consumes the shared protected-task and host-resolution helpers.
+
+### Iteration 4: Stable validation and fingerprints (completed)
+
+- Add canonical snapshot/stack/execution digests without changing legacy provenance fingerprints.
+- Add pure validators and the full status/error/access/limit matrices.
+- Add deterministic context budgeting and artifact/trace namespace validation.
+- Export only the portions demonstrated by the real spike.
+
+Implemented with canonical `sha256:v1` fingerprints, deterministic UTF-8 context budgeting, access/limit/status matrices, artifact/trace validation, and package-root exports. Legacy profile provenance fingerprints are unchanged.
+
+### Iteration 5: Backend registration and conformance (completed)
+
+- Added an optional backend registry/dispatcher with validated descriptor discovery and granular capability negotiation.
+- Added a deterministic fake backend and reusable conformance fixtures shaped by the real spike.
+- Normalized cancellation races, host-abort timeouts, provider failures, malformed responses, enforcement receipts, and authorization-scoped trace routing.
+- pi-forge remains fully functional with no registered backend; the registry starts empty.
+
+### Iteration 6: Concrete Pi SDK walking skeleton (completed)
+
+- Correct backend-assisted prompt-runtime ownership and add complete compiler-input validation/fingerprints.
+- Extract an access-none, no-tool, in-memory Pi SDK backend from the completed spike.
+- Add human-operated backend discovery, dry plan, and explicit real-run commands through the registry.
+- Cover pre-transport refusal, provider failure, host timeout, cancellation, response normalization, and cleanup without live provider calls in ordinary tests.
+
+Implemented as the exported experimental `PiSdkIsolatedBackend` and `/forge-agent backends|plan|run` command path. Ordinary tests use Pi's faux provider through a real in-memory `AgentSession`, proving provider gating and final context without network traffic.
+
+### Iteration 7: Foreground parent integration (completed)
+
+- Added no-egress `forge_subagent_profiles` discovery so the main agent can select only from profile IDs explicitly enabled by trusted-project delegation policy rather than guessing.
+- Added the sequential `forge_subagent` tool on the shared command/runtime path rather than creating a second runner.
+- Added project-only `subagents.profiles.<id>` policy so ordinary profiles remain portable and independently usable while delegation eligibility, backend, and timeout are resolved per profile; global config remains limited to general defaults until global profile and stack storage have explicit scope semantics.
+- Resolve the profile and prepare the exact immutable plan before asking for interactive approval or permitting provider transport.
+- Show a compact default review, with full prompt inspection on demand, and bind approval to the execution fingerprint.
+- Insert only bounded response text into parent context while retaining complete transcript/tool-event details for expanded human inspection.
+- Keep execution text-only, foreground, clean-context, read-only, and shared-user; cancellation, rejection, provider failure, and missing-UI cases fail closed.
+
+### Iteration 8: Sandbox and staged-write decision
+
+- Evaluate an optional bubblewrap-style backend that can honestly enforce allowed roots, network policy, and subprocess restrictions without changing the portable contract.
+- Design writes as a separately approved staged patch/change set with clear inspection and undo semantics; do not simply add `write`, `edit`, or shell to the shared-user child.
+- Continue deferring resumable agents, retries, queues, chains, pipelines, background work, and concurrency orchestration until concrete demand exists.
+
+Profile UI can proceed independently on the completed Iteration 1 service; it does not need to block parent integration or a concrete adapter decision. Each iteration remains independently reviewable and revertible.
+
+## Deferred Decisions
+
+- Whether the experimental Pi SDK backend remains built in, becomes separately configurable, or moves to an optional integration after 0.4 feedback.
+- Trace/artifact storage location, retention defaults, redaction, and cleanup implementation.
+- Resumable sessions and continuation references.
+- Automatic retries, fallbacks, priorities, queues, and concurrency.
+- Secret/environment injection.
+- Structured JSON-schema output.
+- Automatic parent-context selection beyond explicit summaries and references.

package/docs/design/subagents/sdk-spike-findings.md

@@ -0,0 +1,117 @@
+# Historical Pi SDK subagent spike findings
+
+Status: completed internal spike against `@earendil-works/pi-coding-agent` 0.80.6 on 2026-07-12. Its contract findings are productized in `@zihanw/pi-subagent-runtime`'s shared SDK preparation gate, which both fresh-process backends use. The spike harness (`scripts/subagent-sdk-spike*.ts`) was removed in the 0.4 cleanup; with it went the only live media-transport and trusted-extension preparation diagnostic, to be re-established when delegated media tasks are productized. The 0.80.6 observations and terminology below remain a historical record.
+
+## Deliverable
+
+The spike was available through `npm run spike:subagent -- --cwd /path/to/project --profile PROFILE_ID` before its removal in the 0.4 cleanup.
+
+Dry-run is the default. A provider call requires `--execute`; that explicit flag is also the spike's consent to transmit the supplied task/media to the selected provider. The other spike-only flags are:
+
+- `--task TEXT`
+- `--timeout MILLISECONDS`
+- `--access none|read-only|workspace-write`
+- `--image PATH` (repeatable)
+- `--load-forge-extensions` to execute trusted global/project pi-forge registration code
+
+The command emits a JSON preflight/execution report. It creates an in-memory `SessionManager`, uses an isolated SDK resource directory and working directory for `access=none`, loads only the inline compiler bridge, does not write a Pi session file, and disposes the session and trusted registrations when finished.
+
+Execution currently rejects `read-only` and `workspace-write`. The SDK can filter tools, but this adapter cannot produce an allowed-root, symlink-safe filesystem/process/network isolation receipt. `access=none` is enforceable by setting the active tool set to empty.
+
+## Validated Behavior
+
+### Real profile, model, auth, and stack
+
+The dry and execute paths resolved `/home/bruhw/programming`'s `default` profile to:
+
+- Model: `opencode-go/glm-5.2`
+- Thinking: `high`
+- Stack: `qiqi-assistant`
+- Auth: configured through Pi's real `AuthStorage` and `ModelRegistry`
+
+The SDK session used those exact values, remained in memory, and exposed no session file. A real no-tool turn completed with `SPIKE_OK`.
+
+### Tool discovery and policy
+
+The isolated Pi 0.80.6 SDK session advertised seven built-ins: `read`, `bash`, `edit`, `write`, `grep`, `find`, and `ls`. This confirms that adapter discovery must be dynamic; neither pi-forge nor the contract should hard-code the interactive UI's historical four-tool baseline.
+
+The spike computes:
+
+```text
+backend catalog ∩ prompt-stack policy ∩ request access
+```
+
+It records the backend catalog, stack-selected names, effective names, unmatched allow patterns, and whether the requested access can actually be enforced. Tool selection worked for dry preflight, and `access=none` produced an empty effective set in real execution.
+
+Pi's public `ToolInfo` includes names, descriptions, parameter schemas, prompt guidelines, and source metadata, but not the one-line `toolSnippets` used by the base prompt. Those snippets are available in `before_agent_start.systemPromptOptions`. A dry plan built only from the public catalog is therefore intentionally marked `partial-dry-run`.
+
+### Exact prompt preparation and protected task
+
+The inline SDK extension receives the exact base system prompt and `BuildSystemPromptOptions` in `before_agent_start`, then calls the existing pi-forge compiler. On the first `context` event it:
+
+1. Removes the delegated task from unrestricted history compilation.
+2. Compiles optional history and synthetic prompt-stack messages.
+3. Appends a structured clone of the delegated text and media as the protected final user message.
+
+This preserved the task through `qiqi-assistant` and `image-viewer`, including layouts that add synthetic user messages. Offline tests also cover a `chat-history` slot configured to omit the latest user message and prove that it cannot remove the protected task.
+
+The exact `BuildSystemPromptOptions` are not publicly obtainable during passive SDK preflight. For a Pi SDK adapter, exact plan preparation must therefore be backend-assisted inside the accepted prompt lifecycle, after capability preflight but before the provider request. The adapter must fail the turn before transport if compilation or task-preservation validation fails.
+
+### Trusted custom macros and slots
+
+`--load-forge-extensions` explicitly loads trusted global/project pi-forge registration modules in the host. A dry fixture using `examples/custom-system-status-extension` validated that:
+
+- The extension module was discovered and registered.
+- Prompt stacks were reloaded and revalidated after registration.
+- The `cpuLoad` macro and `machine-status` slot compiled without stale unsupported-slot diagnostics.
+- Registrations were unloaded during cleanup.
+
+This supports keeping executable dependencies in the trusted host. It also confirms that host resolution must load required registrations before final stack validation. Dependency identity and missing-dependency receipts remain future work.
+
+### Media
+
+A real `opencode-go/qwen3.7-plus` turn used the `image-viewer` profile and a 1.6 MB PNG. The model received the image, completed successfully, and identified the visible Pi symbol. The final structured task and image were preserved by the prompt-stack layout.
+
+Preflight rejects image input when the selected model does not advertise `image` support. File paths are converted to SDK `ImageContent` only for explicitly supplied `--image` references.
+
+### Timeout, cancellation, response, and trace
+
+A real `glm-5.2` turn with a 1 ms host deadline called `session.abort()`, settled in roughly 70 ms, reported `timed-out`, and recorded zero provider tokens/cost. A normal run reported completion, output, duration, session statistics, and a compact normalized lifecycle trace.
+
+The SDK does not enforce hard turn, token, or output limits through `createAgentSession`. Its statistics report numeric cost without a currency code. Future response normalization must mark those limits unsupported and treat cost currency as unknown unless the adapter can supply it.
+
+The spike trace includes lifecycle and tool start/end metadata only. It excludes streaming content, provider payloads, credentials, and hidden reasoning.
+
+## Contract Revisions Implemented
+
+1. Split backend work into discovery/preflight, backend-assisted plan preparation, and execution. Preparation may occur inside an adapter-controlled pre-provider hook when the backend cannot expose exact prompt-runtime inputs earlier.
+2. Keep the host responsible for profile/stack resolution, trusted compiler execution, protected-task assembly, diagnostics, and plan validation, even when the adapter supplies runtime inputs through a callback.
+3. Do not require a passive dry preflight to reproduce the exact prompt when the backend cannot expose all inputs. Record prompt-runtime fidelity and reject execution if exact preparation cannot be completed before provider transport.
+4. Make tool discovery dynamic and include a separate prompt-runtime receipt; `ToolInfo` alone is insufficient to recreate Pi's prompt.
+5. Treat tool filtering and access isolation as separate receipts. The current Pi SDK adapter supports only `access=none` honestly.
+6. Distinguish backend-native limits from host best-effort controls. The observed timeout uses host abort and is not proof of process-level hard isolation.
+7. Load trusted macro/slot registrations before final stack validation and later add explicit dependency identities/fingerprints.
+8. Normalize assistant error/aborted stop reasons because `session.prompt()` may settle with a terminal assistant message rather than throw.
+
+## Remaining Concrete-Adapter Gaps
+
+- No allowed-root filesystem, subprocess, or agent-network isolation.
+- No hard turn, token, or output-byte enforcement.
+- The retained SDK adapter exposes an empty tool catalog. The default subprocess adapter adds only stack-filtered `read`, `grep`, `find`, and `ls`, under an explicit shared-user rather than sandboxed boundary.
+- The registry, model-callable foreground tool, and text-only subprocess backend now exist, but there is still no artifact store, contract trace storage, media resolution in the shipped path, sandbox enforcement, or staged write workflow.
+- Custom dependency scanning records macro/slot names and registration sources, but it cannot fingerprint executable registration code.
+- The command/tool is foreground-only and does not expose a separate cancel command; Pi/host abort signals and host timeouts terminate the active SDK session or subprocess on a best-effort basis.
+
+These gaps do not block the approval-gated read-only foreground workflow. They do require the UI and receipts to keep saying shared-user, and they rule out adding direct write/shell tools until stronger sandboxing or a separately approved staged-change design exists.
+
+## Verification Evidence
+
+- Offline suite covers the contract matrix, fake-backend conformance, command consent/dry-plan behavior, and concrete SDK execution through Pi's faux provider and a real in-memory `AgentSession` without network traffic.
+- TypeScript typecheck: passed.
+- Real profile dry preflight: passed.
+- Real `glm-5.2` no-tool completion: passed.
+- Real host-timeout/abort case: passed.
+- Real `qwen3.7-plus` PNG input: passed.
+- Trusted custom macro/slot dry fixture: passed.
+
+The generated reports were inspected during the spike and were not committed because they contain machine-specific paths, run IDs, timing, and provider usage.

package/docs/development/complexity-review.md

@@ -0,0 +1,86 @@
+# Complexity review
+
+> **Historical snapshot:** This review records the repository structure at the end of the 0.4 implementation cycle. File counts and hotspot rankings may age; use it as refactoring context, not current user guidance.
+
+Date: 2026-07-18
+
+## Executive Summary
+
+The structural refactor recommended by the original 2026-07-12 review is complete:
+
+- The browser editor is authored as typed modules and bundled only for distribution.
+- The subagent contract is split into focused modules behind compatibility barrels.
+- `src/index.ts` is a small composition root over dedicated runtime services.
+- Package contents and stable, experimental, and compatibility exports are explicitly classified.
+
+pi-forge is still a reasonable size for its feature set. The main engineering risk has moved from a few monolithic files to the safety and lifecycle seams of the experimental subagent backend. The narrow parent-agent integration is now complete; the next iteration should harden the shared-user boundary and evaluate an optional sandbox without adding orchestration breadth.
+
+## Current Size
+
+Measured from the current `feature/0.4-subagents-prep` working tree:
+
+| Area | Files | Lines of TypeScript |
+|---|---:|---:|
+| Production source (`src`) | 72 | 16,355 |
+| Core tests and helpers (`tests`) | 23 | 7,271 |
+| Browser tests (`tests-browser`) | 1 | 139 |
+| Internal TypeScript scripts (`scripts`) | 2 | 628 |
+
+Additional observations:
+
+- `dist/` is generated output and is about 2.1 MB; verification rejects stale generated files.
+- Runtime dependency count remains low.
+- The implementation checkpoint passes 184 core tests, one browser test, TypeScript typecheck, generated-client and `dist` consistency, and npm package-layout checks.
+- The npm package resolves both entry points to compiled output and contains no physical `src/` files.
+
+## Completed Structural Work
+
+### Typed browser client
+
+The previous 2,253-line embedded browser script is now a generated one-line compatibility wrapper. Typed source modules under `src/web-editor/client/` own API transport, DOM access, policy editing, regex editing, inspection, and orchestration. `src/web-editor/client/main.ts` is still large, but it is type-checked and can be split incrementally as profile UI introduces clear component boundaries.
+
+### Modular subagent contract
+
+The previous 1,373-line `src/subagent-contract.ts` implementation is now a seven-line compatibility barrel. Types, canonicalization, request validation, preflight, tools, context, plans, response handling, and validation live in focused `src/subagent/` modules with import-surface tests preserving the experimental API.
+
+### Small composition root
+
+`src/index.ts` remains a small composition root and primarily wires prompt-stack, profile, tool-policy, web-editor, lifecycle, subagent runtime, command, and tool registration. Operational behavior lives in dedicated modules.
+
+### Package and API boundaries
+
+The [public API policy](../reference/public-api.md) classifies stable, experimental, and compatibility surfaces. `@zihanw/pi-forge/subagent` is the preferred experimental integration entry point, while legacy `src/*` aliases resolve to compiled modules without publishing source files.
+
+## Current Complexity Hotspots
+
+| File | Lines | Current assessment |
+|---|---:|---|
+| `src/web-editor/client/main.ts` | 1,491 | Largest remaining orchestration module; split along profile UI boundaries when that work begins |
+| `src/web-editor/styles.ts` | 995 | Large but mostly presentation logic |
+| `src/compiler.ts` | 578 | Mature core compiler; change cautiously |
+| `src/subagent/backend-registry.ts` | 575 | Highest semantic risk: dispatch, cancellation, timeout, binding, and trace routing |
+| `src/subagent/pi-subprocess-backend.ts` | 773 | Foreground child lifecycle, exact bridge inputs, sanitized report transport, cancellation, and cleanup |
+| `src/subagent-tool.ts` | 453 | Approval UX, bounded model projection, and expandable human report rendering |
+| `src/loader.ts` | 518 | Broad input normalization and validation surface |
+| `src/regex.ts` | 500 | Mature transformation engine with focused tests |
+| `src/subagent/pi-sdk-backend.ts` | 450 | Experimental provider/session lifecycle boundary |
+
+The registry, subprocess backend, and approval tool are not concerning solely by line count, but they carry the strongest correctness and egress guarantees. Prefer focused helpers and end-to-end lifecycle tests over broad rewrites.
+
+## Near-Term Priorities
+
+1. Keep provider egress fail-closed in interactive, non-UI, cancelled, and timed-out paths.
+2. Dogfood the model-callable foreground path without creating a second runner or allowing a tool call to grant its own egress consent.
+3. Preserve read-only defaults, bounded result projection, no automatic parent-history export, and honest shared-user receipts while evaluating bubblewrap-style isolation.
+4. Add profile UI by reusing the existing profile repository/application services and splitting browser orchestration only where the new UI creates stable boundaries.
+5. Re-run the complete verification suite for every release candidate and keep generated output synchronized.
+
+## Guardrails
+
+Do not remove strict validation, immutable preparation binding, task preservation, access/limit receipts, rollback, provenance, or drift reporting merely to reduce line count. These mechanisms make the experimental runtime's behavior inspectable and enforceable.
+
+Avoid building retries, queues, chains, background execution, direct write/shell access, media routing, or artifact/trace storage into the 0.4 foreground path. Evaluate sandbox and external orchestration packages behind the existing backend contract before expanding pi-forge into a general runner.
+
+## Final Assessment
+
+The original concentration problems and narrow parent-agent integration are addressed. Future complexity control should be incremental: isolate lifecycle-sensitive backend logic, split the browser client along real product boundaries, and resist turning pi-forge into a general orchestration framework without demonstrated need.

package/docs/development/release.md

@@ -0,0 +1,31 @@
+# Release process
+
+[Documentation](../README.md)
+
+## Before release
+
+1. Confirm the changelog and user documentation describe the intended version and experimental surfaces accurately.
+2. Publish and smoke-test any required `@zihanw/pi-subagent-runtime` version first.
+3. Install dependencies from the lockfile and run `npm run verify`.
+4. Test a packed installation against the documented minimum and current Pi versions.
+5. Exercise ordinary stack/profile use independently of delegation.
+6. Exercise both configured foreground backends and confirm unsupported host capabilities fail closed before provider transport.
+7. Inspect `npm pack --dry-run` for package size and unexpected or missing files.
+
+## Dependency policy
+
+Published manifests use wildcard peer dependencies for Pi-host-provided SDK packages. Exact versions belong in development dependencies and the lockfile so tests are reproducible without restricting compatible host releases.
+
+`pi-subagent-runtime` remains a normal exact dependency until its compatibility policy says otherwise. Its own host-facing Pi dependencies must follow the same host-provided peer model.
+
+## Package contents
+
+The tarball must include compiled `dist/`, examples, the English and Chinese landing pages, changelog, license, and user/reference documentation. It must not include physical `src/` files. Both the default extension entry and experimental subagent entry must resolve to compiled output.
+
+The root `PUBLIC_API.md` and `SUBAGENT_ADAPTER_CONTRACT.md` files are compatibility pointers; authoritative content lives under `docs/reference/`.
+
+## Publish and verify
+
+Publish the intended version/tag, then install it through Pi in a clean project. Verify `/preset`, `/profile`, `/preset ui`, and—when deliberately enabled—delegation. Restart Pi after installation to avoid testing a stale extension instance.
+
+For a stable release, ensure npm `latest` points to the new version and any prerelease channel no longer leaves users on an incompatible older build.

package/docs/development/roadmap.md

@@ -0,0 +1,42 @@
+# Roadmap
+
+[Documentation](../README.md)
+
+This file contains forward-looking product work only. Completed capability belongs in the [feature inventory](../reference/features.md), release history in the root [changelog](../../CHANGELOG.md), and completed investigation in the [design archive](../design/README.md).
+
+## 0.4 release
+
+The profile UI, foreground-delegation dogfooding, dependency compatibility work, and documentation reorganization are complete. Runtime beta.2 is published and the packed extension passes the full verification surface against the documented Pi versions. The 0.4.0 release commit is ready; remaining release work is intentionally small:
+
+- merge the release commit to `main`;
+- publish 0.4.0 with delegation explicitly labelled experimental;
+- install the published package through Pi and complete the post-publish smoke check.
+
+Stable 0.4 does not imply that the subagent adapter or runtime has become stable. Prompt stacks and ordinary profile use remain independent of delegation.
+
+## Near-term candidates
+
+### Scoped global profiles and stacks
+
+If cross-project presets are added, introduce global profiles and stacks together. Preserve source scope, let same-ID project resources shadow global resources, and apply delegation policy only within the matching scope. Define discovery, auto-activation, reference, and editor behavior before implementation.
+
+### Sandbox and staged writes
+
+Evaluate an optional backend that can honestly enforce roots, process behavior, symlink containment, and requested network policy. Keep shared-user as an explicit compatibility boundary. Design writes as a separately approved staged patch/change set; do not add write/edit/shell tools directly to the shared-user child.
+
+### Import, history, and prompt diagnostics
+
+Add SillyTavern fidelity only from real unsupported fixtures. Candidate history controls need concrete use cases and dangling tool-pair tests. Provider-payload rewriting and display-only streaming regex remain deferred until a stable, previewable lifecycle hook exists.
+
+## Product guardrails
+
+- Prompt-stack JSON stays declarative; executable customization stays in trusted extensions/packages.
+- Profiles remain one-shot presets, not continuous runtime owners.
+- Tool and skill policy stays in prompt stacks, not profiles.
+- Skill filtering is model-visible prompt filtering, not an invocation or security boundary.
+- Delegation remains opt-in, foreground, clean-context, and fail-closed on missing capabilities.
+- Do not report shared-user read-only policy as an OS sandbox.
+- New editor workflows require real-browser coverage.
+- Run the full verification and package checks before release.
+
+The detailed completed 0.4 plan is retained in the [historical roadmap](../design/roadmap-0.4-archive.md).

package/docs/development/setup.md

@@ -0,0 +1,75 @@
+# Development setup
+
+[Documentation](../README.md)
+
+## Build from source
+
+```bash
+git clone https://github.com/MacroSony/pi-forge.git
+cd pi-forge
+npm install
+npm run build
+pi
+```
+
+Trust the project in Pi and use `/reload` when needed.
+
+The npm package loads compiled `dist/index.js` and intentionally omits physical `src/` files. Clone or fork the repository to inspect or modify source; do not edit generated files in `node_modules`.
+
+## Load the extension
+
+For release-like local testing, register the cloned package directory in Pi settings:
+
+```json
+{
+  "packages": ["../pi-forge"]
+}
+```
+
+For live source development, remove that package entry and load the TypeScript extension directly:
+
+```json
+{
+  "extensions": ["../pi-forge/src/index.ts"]
+}
+```
+
+You can also run `pi -e ../pi-forge/src/index.ts` for a one-off smoke test. Never load both package and source entries simultaneously; pi-forge would initialize twice.
+
+Browser-client source changes require `npm run build:client` because the editor serves a generated embedded bundle.
+
+## Verification
+
+```bash
+npm test                 # core Node test suite
+npm run test:browser     # real-browser editor characterization
+npm run typecheck        # TypeScript and Vue
+npm run build            # client bundle and dist output
+npm run verify           # all tests plus generated/package consistency
+npm pack --dry-run       # inspect the publishable tarball
+```
+
+Set `CHROME_PATH` when Chrome/Chromium is outside a standard location. CI runs the full verification. When source changes generated output, commit matching generated client and `dist` files.
+
+## Pi compatibility
+
+Published pi-forge treats Pi-owned SDK packages (`pi-agent-core`, `pi-ai`, `pi-coding-agent`, `pi-tui`, and `typebox`) as host-provided wildcard peers. The running Pi host supplies one coherent SDK instance, avoiding duplicate packages and avoiding an install-time lock to Pi's frequent release cadence.
+
+The repository keeps exact SDK versions as development/test fixtures for reproducibility. Exact fixtures do not constrain which Pi version may load the published extension.
+
+Release validation should test:
+
+- the documented minimum supported Pi version;
+- the current Pi version at release time;
+- an automated or scheduled probe of npm `latest`.
+
+Document the tested range separately from peer constraints. Pi-coupled experimental subagent capabilities must preflight against the actual host and fail closed with a precise compatibility diagnostic when required APIs are unavailable; ordinary stacks and profiles should remain usable.
+
+## Package boundaries
+
+- `@zihanw/pi-forge` is the Pi extension and stable macro/slot registration surface.
+- `@zihanw/pi-forge/subagent` is the preferred experimental host-contract entry point.
+- `@zihanw/pi-subagent-runtime` owns execution lifecycle and fresh-process backends.
+- Legacy `@zihanw/pi-forge/src/*` aliases resolve to compiled compatibility modules and are not a promise that internals are public.
+
+See the [public API policy](../reference/public-api.md).