@zihanw/pi-forge 0.4.1 → 0.5.1

package/docs/design/archive/0.5-full-proposal/template-language-spike.md

@@ -0,0 +1,156 @@
+# Template-language and compiler spike
+
+[Design index](README.md) · [0.5 architecture plan](architecture-0.5.md) · [0.5 migration inventory](0.5-inventory.md)
+
+Status: completed Phase 0 spike; architecture decision pending
+
+Date: 2026-08-17
+
+## Question
+
+Which template language can replace 0.4 macros while keeping prompt compilation
+deterministic over immutable inputs, statically analyzable, migrationable, and
+free of arbitrary evaluation or template loading?
+
+## Characterized 0.4 surface
+
+The current macro language mixes four separate concerns:
+
+| Concern | 0.4 behavior | 0.5 direction |
+|---|---|---|
+| Immutable runtime values | Latest user message, date/time, cwd, selected tools, active model | Keep as explicit values in `PromptEnvironment` |
+| Static reusable values | `stack.variables`, referenced as `{{name}}` | Keep as JSON-compatible immutable `parameters` |
+| Text transforms | `trim`, `upper`, `lower`, `json`, `xml`, including nesting | Retain only as a finite, pure filter set if accepted |
+| Control flow | `ifvar`, `ifeq`, `iftools`, `ifslot`, with lazy branches | Replace only with a parsed, side-effect-free conditional form if accepted |
+| Mutable state | Turn/session variables plus set/get/clear macros and `variables` slot | Remove; emit migration diagnostics |
+| Executable extension code | Trusted custom macro and slot renderers receive live runtime objects | Replace only through a separately accepted, immutable template/slot port |
+
+The current examples demonstrate simple parameter and runtime interpolation;
+the image-reader and SillyTavern examples also depend on nested filters,
+conditionals, and mutable variables. SillyTavern is already removed from 0.5
+scope. Any remaining non-mechanical construct must be diagnosed by migration,
+not silently approximated.
+
+## Requirements
+
+The v2 engine must provide one `parse → analyze → render` contract and meet all
+of the following:
+
+- parsing has no filesystem, network, process, clock, environment, or Pi access;
+- rendering receives only a frozen normalized environment and parsed template;
+- undefined paths, invalid filters, invalid conditions, parse errors, recursion,
+  and output-limit breaches are typed errors, never silent mutation or ambient
+  fallback;
+- analysis returns source-spanned dependencies on environment paths, filters,
+  control predicates, and registered template/slot capabilities;
+- no includes, extends, imports, macros, function calls, loops, arbitrary
+  property traversal, or general expression evaluation;
+- whitespace is preserved unless a deliberately documented syntax construct
+  changes it; rendering has a bounded output size;
+- preview, parent runtime, and subagent preparation invoke the same engine and
+  receive recorded environment/provenance values.
+
+## Candidate comparison
+
+| Candidate | Parse/analyze/render fit | Restriction and determinism | Migration fit | Result |
+|---|---|---|---|---|
+| General Jinja-like engine (Nunjucks) | Has parsing/rendering and strict-undefined option | Fails the trust requirement: its own documentation says it does not sandbox user-defined templates; it supports loaders, includes, extensions, async behavior, and a rich expression language | Familiar delimiters, but an unsafe/surplus language would need a fragile denylist | Reject |
+| Restricted LiquidJS wrapper | Provides parsed templates and documented static analysis | Better parser option, but its Liquid tags, includes, filters, and plugin model would still require a security-sensitive allowlist/wrapper; syntax is not Jinja-compatible | Requires a new Liquid migration plus wrapper-specific dependency semantics | Do not adopt for 0.5 |
+| Small Forge AST grammar | Contract is designed exactly for parse/analyze/render and the allowed syntax is closed | No ambient capabilities or loaders by construction; all values and filters are explicit | Breaking but small, predictable, and mechanically migrates common interpolation | Recommended candidate |
+
+Nunjucks documents both its lack of sandboxing and its extensible loaders,
+includes, custom tags, and asynchronous behavior. [Nunjucks API](https://mozilla.github.io/nunjucks/api.html)
+LiquidJS documents parsed-template reuse and static analysis, but is a broader
+Liquid language with tags and plugins rather than the deliberately closed v2
+language. [LiquidJS static analysis](https://liquidjs.com/tutorials/static-analysis.html)
+
+## Recommended decision candidate: `forge-v1`
+
+Adopt a small parsed Forge template grammar. It resembles Jinja only at the
+delimiter level; it must not claim Jinja compatibility.
+
+### Closed grammar
+
+```text
+text        := any text outside delimiters
+output      := "{{" path filters? "}}"
+filters     := ("|" filterName)*
+if-block    := "{% if" predicate "%}" template
+               ("{% else %}" template)? "{% endif %}"
+path        := identifier ("." identifier)*
+predicate   := path | path "==" json-string | path "!=" json-string
+```
+
+The only initially proposed paths are `parameters.*` and documented
+`runtime.*` fields. No bracket access, method invocation, object construction,
+numeric arithmetic, loops, user-defined functions, imports, includes, extends,
+macro definitions, or arbitrary expressions are in the grammar.
+
+The initially proposed filters are `trim`, `upper`, `lower`, `json`, and `xml`.
+They are pure, unary, versioned built-ins; unknown filters are errors. Tool and
+slot conditions, if retained, must be represented as documented boolean values
+in the environment rather than callable helpers. The exact v2 environment field
+names remain part of the architecture decision.
+
+### Engine contract
+
+```ts
+interface TemplateEngine {
+  readonly id: "forge-v1";
+  readonly version: 1;
+  parse(source: string): TemplateParseResult;
+  analyze(ast: TemplateAst): readonly TemplateDependency[];
+  render(ast: TemplateAst, environment: PromptEnvironment): TemplateRenderResult;
+}
+```
+
+`PromptEnvironment` is a frozen, JSON-compatible snapshot. It must carry an
+explicit timestamp and timezone/formatting policy, cwd, model identity, selected
+tool facts, latest-user-message data, static parameters, and the Pi-derived
+inputs required by slots as separate structured values. The compiler—not the
+engine—owns history placement, structured slots, tool/skill policy, base-system
+prompt handling, message assembly, and regex stages.
+
+The render receipt must identify engine ID/version, template dependencies,
+environment/provenance fingerprint, diagnostics, and output size. Dependencies
+are authoritative for preview and subagent preparation; no consumer may
+re-parse syntax independently.
+
+### Extension boundary implication
+
+The current `registerMacro()` API cannot cross this boundary unchanged because
+it receives live runtime objects and can mutate or observe ambient state. The
+template decision does not yet authorize a replacement API. If custom template
+values, filters, or slots remain in 0.5, a separate accepted port must give them
+only frozen input, declared dependency/capability metadata, bounded output, and
+a pure-rendering contract. Trusted extension code is not a security boundary;
+the core engine nevertheless must remain deterministic when no such extension
+port participates.
+
+## Migration disposition
+
+| 0.4 construct | Candidate v2 migration | Result when not mechanical |
+|---|---|---|
+| `{{name}}` for a known static value | `{{ parameters.name }}` | Warn when name could be a custom macro or ambiguous |
+| `{{lastUserMessage}}`, date/time, cwd, model, selected tools | Corresponding documented `runtime.*` path | Mechanical only after field names are accepted |
+| Nested value filters | Equivalent output plus pipeline where semantics match | Diagnose when nested arguments or behavior differ |
+| `ifvar` / `ifeq` | Parsed `if` predicate where semantics match | Diagnose `iftools`/`ifslot` until boolean environment fields are accepted |
+| Custom macros and slots | New port only after its decision | Diagnose; no compatibility execution shim |
+| Set/get/clear variable macros and `variables` slot | None | Error-level migration diagnostic; removed behavior |
+| Unknown macros / unresolved-policy behavior | None | Error-level diagnostic in v2; no keep/warn rendering policy |
+
+The migration tool must preserve the original source, produce a field/item-level
+report, and refuse a partly transformed file unless the user explicitly accepts
+diagnostics. It must not rewrite SillyTavern inputs in 0.5.
+
+## Required acceptance and implementation evidence
+
+Before Phase 3 implementation, accept the grammar, exact environment schema,
+filter/condition set, undefined/error semantics, output limits, engine ID in
+stack schema v2, extension-port disposition, and migration behavior.
+
+Implementation then needs characterization and conformance coverage for parser
+errors, source spans, strict undefined values, filters, nested/else conditionals,
+whitespace, output limits, deterministic repeated render, dependency receipts,
+preview/runtime identity, and migration diagnostics for every removed mutable or
+custom-macro construct.

package/docs/design/context-diff-plan.md

@@ -0,0 +1,78 @@
+# Context Diff — design plan (post-0.5.0)
+
+Status: folded into `docs/design/0.5.1-plan.md` as Lane 2 — that file is now the owner of truth. Kept for discussion history.
+
+Implemented follow-up: the dock now provides git-style line diffs and correlates
+captured requests with Pi's assistant usage. The structural cache-prefix figure
+remains a chars/4 estimate; provider-reported cache hit rate is shown separately
+and is unavailable until cache reporting is observed for that provider/model.
+
+Original status: planned. Target release: 0.5.1 (first feature release after 0.5.0).
+
+## Goal
+
+Per-turn observability for prompt changes so users can optimize KV-cache reuse:
+
+- Mark which blocks of the prompt changed after each turn.
+- Show how many tokens were added/removed/modified (delta vs the previous turn).
+- Mark where the KV-cache prefix survives ("cache boundary").
+
+Secondary goal: merge with live preview — while editing a stack, debounce-compile and diff against the previous compile, so edits show their prompt/token impact immediately.
+
+## Existing building blocks
+
+- `src/payload-capture.ts` — captures the real provider request payload (secret-redacted), with `approxTokens` (chars/4) estimation. `SAFE_TOKEN_METADATA_KEYS` already whitelists `cached_tokens` etc., so real cache-hit numbers can be surfaced if response usage ever becomes available.
+- `src/preview.ts` — edit-time compile output split into sections (system + per-message), each with chars/approxTokens.
+- Legacy editor already polls payload state every 2s.
+
+## Two data sources
+
+| Scenario | Source | Question answered |
+|---|---|---|
+| Edit time (live preview merge point) | debounced compile vs previous compile | "what does this edit change, how many tokens" |
+| Run time (KV-cache truth) | diff of consecutive real payloads | "where did the cache break, how many extra tokens this turn" |
+
+## Core engine: `src/context-diff.ts` (host-neutral pure functions)
+
+```
+TurnSnapshot  { turnId, capturedAt, stackId, blocks: Block[] }
+Block         { key, role, text, chars, approxTokens, hash }
+TurnDiff      { blocks: DiffBlock[], prefixTokens, prefixRatio, deltaTokens, summary }
+DiffBlock     { status: same|added|removed|modified, before?, after?, tokenDelta }
+```
+
+Cache-boundary algorithm: KV-cache hits depend on the longest common prefix of the serialized request. Walk the block arrays in order while hashes match; at the first mismatch, trim a char-level common prefix inside that block and convert to tokens. Render a boundary marker: "cache valid up to ~18,204 tokens (63% of prompt)". No Myers diff needed — prefix + block classification suffices.
+
+Honesty note: tokens are chars/4 estimates. Relative deltas are accurate; absolute boundary claims must be labeled "approx" in UI and in promo material.
+
+## UI
+
+Promote the Preview modal into a dockable right-side panel with two tabs:
+
+- **Compiled** — current preview content, auto-refresh with 500ms debounce while editing (this is the live preview).
+- **Diff** — top summary strip (`Turn N · +412 tokens vs previous · cache boundary ~63% · 2 blocks changed`), block list with green/red/yellow gutters and per-block token chips, scissor-line boundary marker.
+
+Run-time mode: payload poll captures a new payload → store in rolling history (last 20 turns) → auto-compute diff.
+Edit mode: diff current edited compile vs the active on-disk version.
+
+Implementation constraint: write the panel as a self-contained Vue component bridged in via the vue-host mechanism. Do NOT add more imperative code to `legacy-editor.ts`. Do NOT refactor the legacy editor in the same lane.
+
+## Phases / estimate
+
+| Phase | Content | Effort |
+|---|---|---|
+| 0 | Design freeze + golden fixtures (turn payload sets) | 0.5d |
+| 1 | `context-diff.ts` engine + unit tests (prefix/add/remove/modify/token rollups) | 1d |
+| 2 | Server endpoint + rolling snapshot state on the web host | 0.5d |
+| 3 | Web UI: preview dock + Diff view (self-contained Vue, bridged) | 1.5d |
+| 4 | Browser tests + docs + changelog | 0.5–1d |
+
+Total ~4–5 working days; ~1 calendar week with review. Descope option (~3d): run-time diff only, live preview reduced to plain auto-refresh without edit-time diffing.
+
+## Release narrative
+
+Do not fold into 0.5.0 — 0.5.0 ships as the architecture-split release; context diff headlines 0.5.1 as "observability". The promo video leads with this feature (money shot: edit one system-prompt line, watch the cache-boundary marker jump to the top).
+
+## Promo video pipeline (reference: AIGC/VIDEO_PRODUCTION ep02)
+
+Reuse the ep02 pipeline: `PLAN.md` → script draft → `narration.json` (production source of truth) → roughcut → review. Terminology follows the ep02 rule: say it in plain words first, name the formal term once ("KV cache" on first mention, then "prefix cache / cache reuse region").

package/docs/design/decision-template.md

@@ -0,0 +1,33 @@
+# Architecture decision: title
+
+[Design index](README.md)
+
+Status: proposed
+
+Date: YYYY-MM-DD
+
+Owners: maintainer names or roles
+
+## Context
+
+Describe the concrete problem, current constraints, and evidence that a decision is needed.
+
+## Decision
+
+State the chosen behavior and ownership boundary precisely. Include a small interface or diagram when it makes the boundary easier to verify.
+
+## Consequences
+
+Describe what becomes easier, what becomes harder or impossible, and which compatibility or operational costs are accepted.
+
+## Migration
+
+List the implementation order, compatibility period if any, user migration, and removal condition for temporary shims.
+
+## Alternatives considered
+
+Record the serious alternatives and why they were rejected. Do not use this section to restate obviously inferior options.
+
+## Verification
+
+Define the tests, dependency checks, documentation, and release evidence that prove the decision was implemented.

package/docs/design/roadmap-0.4-archive.md

@@ -10,7 +10,7 @@ This file was the forward-looking product roadmap during 0.4 development. Comple
 - [Feature inventory](../reference/features.md): implemented profile, prompt-stack, web-editor, payload, extension, packaging, and experimental subagent behavior.
 - [Changelog](../../CHANGELOG.md): release and unreleased change history.
 - [Subagent interface design](subagents/interface-design.md): accepted request/response architecture, context model, responsibility boundaries, and implementation history.
-- [Subagent adapter contract](../reference/subagent-adapter.md): current exported contract, validation, enforcement receipts, and adapter responsibilities.
+- Subagent adapter contract: the 0.4 exported contract documented the execution surface that moved to `@zihanw/pi-forge-subagents` in 0.5; see the [subagent host port contract](../reference/subagent-host-port.md) for the current boundary.
 - [SDK spike findings](subagents/sdk-spike-findings.md): historical real Pi SDK findings for model/auth resolution, exact prompt preparation, dynamic tools, media, timeout, and cleanup.
 - [Subagent design review](subagents/design-review.md): issues that drove resolution, preflight, enforcement, task-preservation, fingerprint, and response revisions.
 - [Public API policy](../reference/public-api.md): stable, experimental, and internal compatibility surfaces.

package/docs/design/subagents/interface-design.md

@@ -1,6 +1,6 @@
 # Subagent request/response design
 
-Status: historical design and implementation record for the completed 0.4 path. Execution ownership (backend registry, preflight binding, plan sealing, conversation/execution fingerprints, lifecycle, and the fresh-process `pi-subprocess-readonly`/`pi-rpc-readonly` backends) lives in `@zihanw/pi-subagent-runtime`; the former in-package registry and `pi-sdk-isolated` compatibility backend were removed in that migration. The broader Pi SDK spike is recorded in the [SDK spike findings](sdk-spike-findings.md) (harness since removed), and the concrete exported semantics are documented in the [subagent adapter contract](../../reference/subagent-adapter.md).
+Status: historical design and implementation record for the completed 0.4 path. Execution ownership (backend registry, preflight binding, plan sealing, conversation/execution fingerprints, lifecycle, and the fresh-process `pi-subprocess-readonly`/`pi-rpc-readonly` backends) lives in `@zihanw/pi-subagent-runtime`; the former in-package registry and `pi-sdk-isolated` compatibility backend were removed in that migration. The broader Pi SDK spike is recorded in the [SDK spike findings](sdk-spike-findings.md) (harness since removed), and the 0.5 boundary is documented in the [subagent host port contract](../../reference/subagent-host-port.md).
 
 ## Goals
 

package/docs/development/architecture-rules.md

@@ -0,0 +1,148 @@
+# Architecture and development rules
+
+[Documentation](../README.md) · [0.5 architecture plan](../design/architecture-0.5.md)
+
+These rules keep implementation throughput from outrunning architectural understanding. They apply to humans and coding agents. The intent is not to slow local implementation; it is to make boundary changes scarce, explicit, and reviewable.
+
+## Decision authority
+
+Implementation may move quickly inside an accepted boundary. A change to a boundary, persistent model, public contract, or product concept requires an explicit design decision before implementation.
+
+Agents may draft decisions, diagrams, interfaces, migration plans, and implementation slices. A human maintainer accepts the decision and its tradeoffs.
+
+## Development mode for 0.5
+
+Until the 0.5 release gates are complete:
+
+- Feature development is frozen unless the feature is explicitly added to the 0.5 plan.
+- Only one boundary-changing initiative should be active at a time.
+- Simplification and removal are preferred over compatibility layers without a demonstrated consumer.
+- A completed slice must leave the repository in a coherent, documented, verified state; partial architecture migrations must not silently become permanent.
+
+## Required dependency direction
+
+```mermaid
+flowchart LR
+    Adapters["Adapters<br/>Pi lifecycle · commands · HTTP · browser"]
+    Application["Application<br/>ForgeWorkspace · stack/profile services"]
+    Domain["Domain/core<br/>schemas · catalogs · compiler · templates"]
+    Ports["Ports<br/>repositories · runtime controller · host capabilities"]
+    Infrastructure["Infrastructure<br/>filesystem · Pi SDK · process registry"]
+
+    Adapters --> Application
+    Application --> Domain
+    Application --> Ports
+    Infrastructure --> Ports
+```
+
+Infrastructure implements inward-facing ports. Domain/core code must not import adapters or infrastructure implementations.
+
+Until the target directory/package structure exists, apply the rule to logical ownership rather than relying on current file locations.
+
+## Component rules
+
+### Domain and compiler
+
+- Domain types contain no command, HTTP, browser, TUI, or filesystem behavior.
+- Prompt compilation consumes a normalized, immutable prompt environment rather than a live Pi context.
+- Compilation returns prompt output, diagnostics, dependencies, and any explicit state transition. Renderers must not silently mutate shared state.
+- Template dependency analysis is part of the template-engine contract; subagents and previews must not reverse-engineer template syntax independently.
+- Tool policy remains stack-owned. Profiles may reference a stack but must not duplicate the policy.
+
+### Application services
+
+- `ForgeWorkspace` owns a coherent workspace snapshot and coordinates resource reloads.
+- `PromptStackService` owns stack resolution, mutation, activation, validation, and preview/compile entry points.
+- `AgentProfileService` owns profile resolution, mutation, preflight, transactional application, provenance, and drift.
+- Services expose typed results. Adapters translate those results into command messages, HTTP statuses, and UI view models.
+
+### Persistence
+
+- All profile and stack reads/writes/deletes go through repositories.
+- Repositories use codecs as the single source of parsing, normalization, validation, and serialization.
+- Mutations validate scope and containment. No adapter may call `writeFileSync` or `unlinkSync` for a domain resource.
+- **Lean 0.5.0 interim:** repositories and codecs are introduced in minimal form. Expected-fingerprint writes, fingerprinting in codecs, and guaranteed atomic file replacement are 0.5.x work. Current replacement semantics remain characterized by tests until then.
+
+### State
+
+- Every mutable value has one named owner.
+- Debug payload capture, browser presentation state, prompt resource state, and session provenance are separate state slices.
+- Reload produces a complete snapshot before publishing it. Consumers must not observe a half-reloaded stack/profile graph.
+- New session-persisted state requires a version, restoration semantics, migration behavior, and an architecture decision.
+
+### Adapters and optional packages
+
+- Commands and HTTP handlers parse input, invoke an application service, and render a result; they do not implement domain workflows.
+- Browser view models must not leak into core runtime state.
+- Optional extensions use versioned host ports and fail clearly when the required host version is unavailable.
+- `pi-forge-subagents` must not load an independent conflicting copy of the active Forge workspace or reach into pi-forge internals.
+
+### Public APIs
+
+- Public API is an allowlist of package entry points, not a side effect of file placement.
+- The 0.5 release removes the broad `src/*` compatibility surface.
+- A public type should describe a stable domain or port, not an internal runtime object.
+- Breaking changes belong in the changelog and the 0.5 migration guide.
+
+## Change classification
+
+### Local change
+
+A local change stays within one accepted component and preserves schemas, state ownership, dependency direction, and public behavior. It needs focused tests and ordinary review.
+
+Examples include fixing a renderer, improving a view mapper, or extracting a private helper within one service.
+
+### Boundary-affecting change
+
+A boundary-affecting change moves responsibility, adds a dependency across components, changes an application port, or introduces a new state owner. It requires an accepted architecture decision or an amendment to the 0.5 plan.
+
+### Product-affecting change
+
+A product-affecting change adds/removes a feature, changes a schema, alters persistence or trust semantics, or changes a public API/package. It requires an accepted decision, migration notes, documentation updates, and explicit human review.
+
+## Architecture-decision triggers
+
+An architecture decision is required when changing any of the following:
+
+- package or extension boundaries;
+- JSON schemas or persisted session entries;
+- public exports or compatibility promises;
+- `ForgeWorkspace` or another state owner;
+- compiler/template semantics;
+- filesystem repository guarantees;
+- trust, approval, tool-policy, or provider-egress boundaries;
+- cross-extension discovery;
+- removal or addition of a product-level feature.
+
+Use the [architecture decision template](../design/decision-template.md). Small decisions can amend the active 0.5 plan instead of creating a separate document if the alternatives and consequences remain clear.
+
+## Pull-request requirements
+
+During lean 0.5.0, every pull request states summary, breaking impact, and verification, as in the repository template. Boundary- or product-affecting changes must link to the accepted decision in the [lean 0.5 plan](../design/architecture-0.5.md); a separate decision document is not required for decisions already accepted there.
+
+For full-target 0.5.x work, pull requests additionally state change classification, affected components, dependency-direction/state/schema/public-API impact, and the linked decision.
+
+## Definition of done
+
+A slice is complete when:
+
+1. Ownership and dependency direction match the accepted architecture.
+2. Behavior is covered at the narrowest useful level; cross-adapter behavior has integration coverage.
+3. No domain persistence remains in command, HTTP, or browser-host adapters.
+4. Public and persisted changes include migration notes.
+5. Current and target documentation reflect any moved boundary.
+6. Generated artifacts are synchronized.
+7. `npm run verify` passes for release-sized or cross-cutting work.
+
+## Review stop conditions
+
+Stop and request architectural review when an implementation needs to:
+
+- import an adapter from domain/application code;
+- access `PiForgeRuntimeState` from a new component;
+- add mutable template variables or render-time side effects;
+- write a resource outside a repository;
+- add a wildcard export;
+- duplicate profile/stack loading in another extension;
+- retain a compatibility layer with no named consumer or removal date;
+- introduce a second boundary-changing initiative before the current one is complete.

package/docs/development/release.md

@@ -6,12 +6,18 @@
 
 1. Confirm the changelog and user documentation describe the intended version and experimental surfaces accurately.
 2. Publish and smoke-test any required `@zihanw/pi-subagent-runtime` version first.
-3. Install dependencies from the lockfile and run `npm run verify`.
+3. Install dependencies from the lockfile and run `npm run verify`; require the
+   Ubuntu, macOS, and Windows GitHub Actions jobs to pass for the release commit.
 4. Test a packed installation against the documented minimum and current Pi versions.
 5. Exercise ordinary stack/profile use independently of delegation.
 6. Exercise both configured foreground backends and confirm unsupported host capabilities fail closed before provider transport.
 7. Inspect `npm pack --dry-run` for package size and unexpected or missing files.
 
+The macOS and Windows jobs run the same complete verification surface as Linux,
+including the real-browser editor suite against the hosted runner's system
+Chrome. Compatibility-version and scheduled latest-Pi probes remain Linux-only;
+they test dependency drift rather than operating-system behavior.
+
 ## Dependency policy
 
 Published manifests use wildcard peer dependencies for Pi-host-provided SDK packages. Exact versions belong in development dependencies and the lockfile so tests are reproducible without restricting compatible host releases.

package/docs/development/roadmap.md

@@ -4,6 +4,37 @@
 
 This file contains forward-looking product work only. Completed capability belongs in the [feature inventory](../reference/features.md), release history in the root [changelog](../../CHANGELOG.md), and completed investigation in the [design archive](../design/README.md).
 
+## 0.5.0 breaking cleanup (lean)
+
+0.5.0 is a deliberately breaking cleanup release plus the minimum foundation for 0.5.x. Net-new feature work is frozen.
+
+The authoritative scope, accepted decisions, implementation lanes, and release gates are in the [lean 0.5 architecture plan](../design/architecture-0.5.md). The original six-phase architecture target remains the long-term goal and is archived in the [full proposal](../design/archive/0.5-full-proposal/README.md).
+
+Implementation order:
+
+0. Documentation convergence: archive the full proposal and make the lean plan active.
+1a. Removals only: remove SillyTavern, mutable variable state/macros/slot, variable session entries, and regex `display`/`both`.
+1b. Compiler, schema v2, and extension contract: implement `forge-v1`, frozen `PromptEnvironment`, immutable `parameters`, and the redesigned trusted extension port; retain `finalize` under lifecycle-adapter ownership.
+1c. Migration and documentation: v1-to-v2 utility, example migration, and English/Chinese breaking notes.
+1d. Compiler/extension conformance: pure slot contract symmetry, slot dependency resolution, shared dependency analysis/compilation context, lazy extension evaluation, nested conditionals, parser fixes, and focused conformance tests.
+2a. Minimal repositories and codecs: make repositories/codecs the only stack/profile read/write path; defer fingerprint and atomic writes.
+2b. ForgeWorkspace and host port v1: minimal snapshot owner plus `/subagent` discovery, profile listing/snapshot, and prepare with mandatory lifecycle semantics.
+3. Subagent extraction: move subagent code into `pi-forge-subagents`; remove the main-package hard dependency and delegation UI; optional package owns dedicated `subagents.json` files.
+3.5. Host-neutrality hardening: host-neutral prompt compiler, `ForgeWorkspace` as the single resource-state owner, optional-package docs/config alignment, and portable packed smoke.
+4. Public surface and release: (4a) Forge-native host contract — main drops the runtime dependency and the synthetic preparation model, and the 0.4 execution contract moves to the optional package; (4b) public-surface cut — root default, root named extension API, and `/subagent` only, `src/*` aliases removed, package checks flipped to the allowlist; (4c) optional-package packed smoke over a loopback transport; (4d) migration notes and English/Chinese docs; (4e) packed-install verification and the 0.5.0 release.
+
+Only one lane is active at a time. Sandbox, staged writes, new prompt features, richer imports, and the remaining full-plan architecture work remain deferred until after this sequence.
+
+## 0.5.x continuation
+
+After 0.5.0, continue toward the archived full target in small increments:
+
+- expected-fingerprint writes, codec fingerprinting, and atomic persistence;
+- full `PromptStackService` / `AgentProfileService` / `ForgeWorkspace` ownership;
+- physical `pi-forge-core` boundaries and dependency-direction checking;
+- complete host RPC catalogue, progress events, and optional subagent UI;
+- public-surface classification and rolling Pi compatibility matrix.
+
 ## 0.4 baseline
 
 The profile UI, foreground-delegation dogfooding, dependency compatibility work, and documentation reorganization are complete and merged to `main`. Runtime beta.2 is published, and the packed 0.4.0 extension passes the full verification surface against the documented Pi versions.
@@ -12,25 +43,26 @@ The 0.4.1 release adds scoped global profiles and prompt stacks with project-ove
 
 Stable 0.4 does not imply that the subagent adapter or runtime has become stable. Prompt stacks and ordinary profile use remain independent of delegation.
 
-## Near-term candidates
+## Deferred candidates
 
 ### Sandbox and staged writes
 
 Evaluate an optional backend that can honestly enforce roots, process behavior, symlink containment, and requested network policy. Keep shared-user as an explicit compatibility boundary. Design writes as a separately approved staged patch/change set; do not add write/edit/shell tools directly to the shared-user child.
 
-### Import, history, and prompt diagnostics
+### History and prompt diagnostics
 
-Add SillyTavern fidelity only from real unsupported fixtures. Candidate history controls need concrete use cases and dangling tool-pair tests. Provider-payload rewriting and display-only streaming regex remain deferred until a stable, previewable lifecycle hook exists.
+Candidate history controls need concrete use cases and dangling tool-pair tests. Provider-payload rewriting and display-only streaming regex remain deferred until a stable, previewable lifecycle hook exists. SillyTavern fidelity is no longer a core roadmap goal; 0.5 removes it.
 
 ## Product guardrails
 
 - Prompt-stack JSON stays declarative; executable customization stays in trusted extensions/packages.
+- Prompt rendering moves toward immutable inputs and explicit outputs; do not add mutable variable behavior during 0.5.
 - Profiles remain one-shot presets, not continuous runtime owners.
 - Tool and skill policy stays in prompt stacks, not profiles.
 - Skill filtering is model-visible prompt filtering, not an invocation or security boundary.
-- Delegation remains opt-in, foreground, clean-context, and fail-closed on missing capabilities.
+- Delegation remains opt-in, foreground, clean-context, and fail-closed on missing capabilities, and lives in the optional package.
 - Do not report shared-user read-only policy as an OS sandbox.
-- New editor workflows require real-browser coverage.
+- New editor product workflows are frozen; migration changes retain real-browser coverage.
 - Run the full verification and package checks before release.
 
 The detailed completed 0.4 plan is retained in the [historical roadmap](../design/roadmap-0.4-archive.md).

package/docs/development/scoped-global-profiles-stacks.md

@@ -2,7 +2,7 @@
 
 [Documentation](../README.md) · [Roadmap](roadmap.md)
 
-Status: accepted design; implementation complete for 0.4.1 (identity, catalogs, scoped stacks/profiles, persistence, auto-activation, delegation, zh-CN docs, and web-editor global create/fork/import with explicit scope).
+Status: accepted design; implementation complete for 0.4.1 (identity, catalogs, scoped stacks/profiles, persistence, auto-activation, delegation, zh-CN docs, and web-editor global create/fork/import with explicit scope). Historical record: the delegation-policy references to `.pi/forge/config.json` `subagents.*` below are superseded in 0.5 — delegation configuration moved to the optional package's dedicated `subagents.json` files (see the [0.5 migration guide](../guides/migrating-to-0.5.md)).
 
 ## Confirmed design decisions
 

package/docs/development/setup.md

@@ -14,6 +14,8 @@ pi
 
 Trust the project in Pi and use `/reload` when needed.
 
+Before making structural changes, read the [architecture and development rules](architecture-rules.md) and the active [0.5 architecture plan](../design/architecture-0.5.md). Boundary, schema, persistence, public API, and product changes require an accepted architecture decision before implementation.
+
 The npm package loads compiled `dist/index.js` and intentionally omits physical `src/` files. Clone or fork the repository to inspect or modify source; do not edit generated files in `node_modules`.
 
 ## Load the extension
@@ -67,9 +69,9 @@ Document the tested range separately from peer constraints. Pi-coupled experimen
 
 ## Package boundaries
 
-- `@zihanw/pi-forge` is the Pi extension and stable macro/slot registration surface.
-- `@zihanw/pi-forge/subagent` is the preferred experimental host-contract entry point.
-- `@zihanw/pi-subagent-runtime` owns execution lifecycle and fresh-process backends.
-- Legacy `@zihanw/pi-forge/src/*` aliases resolve to compiled compatibility modules and are not a promise that internals are public.
+- `@zihanw/pi-forge` is the Pi extension (default export) and the stable macro/slot registration surface; no other root exports or `src/*` aliases exist.
+- `@zihanw/pi-forge/subagent` is the experimental versioned host-port entry point (wire DTOs, validators, transport, client/host lifecycle, canonical fingerprints).
+- `@zihanw/pi-forge-subagents` is the optional package owning subagent execution, configuration, and the 0.4 execution contract.
+- `@zihanw/pi-subagent-runtime` owns execution lifecycle and fresh-process backends (a dependency of the optional package, not of the main package).
 
 See the [public API policy](../reference/public-api.md).

package/docs/getting-started.md

@@ -70,11 +70,12 @@ Profiles live in `.pi/forge/agent-profiles/*.json` by default; `/profile save gl
 |---|---|
 | `.pi/forge/prompt-stacks/` | Project prompt stacks |
 | `.pi/forge/agent-profiles/` | Project agent profiles |
-| `.pi/forge/config.json` | Trusted project configuration and `project:<id>` delegation authorization |
+| `.pi/forge/config.json` | Trusted project configuration (`webEditor.*`) |
 | `.pi/forge/extensions/` | Trusted project macro/slot registration code |
 | `~/.pi/forge/prompt-stacks/` | User-global prompt stacks |
 | `~/.pi/forge/agent-profiles/` | User-global agent profiles |
-| `~/.pi/forge/config.json` | User defaults and `global:<id>` delegation authorization |
+| `~/.pi/forge/config.json` | User defaults (`webEditor.*`) |
+| `.pi/forge/subagents.json` / `~/.pi/forge/subagents.json` | Delegation authorization owned by the optional `@zihanw/pi-forge-subagents` package |
 | `~/.pi/forge/extensions/` | Trusted user macro/slot registration code |
 
 Legacy `.pi/prompt-stacks/*.json` files remain readable. Command-created stacks go to `.pi/forge/prompt-stacks`; use the web editor's `global` scope selector to create stacks in `~/.pi/forge/prompt-stacks`. Same-named new-location files shadow legacy ones. Migrate safely with:
@@ -89,7 +90,7 @@ Add `--overwrite` only when target replacement is intended. Add `--delete-legacy
 ## Where to go next
 
 - Learn the [prompt-stack model](concepts/prompt-stacks.md).
-- Fork a [focused reviewer](../examples/reviewer-prompt-stack.json) or [DM writer](../examples/sillytavern-dm-writer-prompt-stack.json).
-- Use the [SillyTavern importer](guides/sillytavern-import.md).
+- Fork a [focused reviewer](../examples/reviewer-prompt-stack.json).
 - Learn [macros and slots](reference/macros-and-slots.md).
+- Read the [0.5 migration guide](guides/migrating-to-0.5.md).
 - Inspect the [complete commands](reference/commands.md) and [stack schema](reference/stack-schema.md).

package/docs/guides/custom-macros-and-slots.md

@@ -15,8 +15,9 @@ Modules receive the pi-forge API directly, so they do not need to import the pac
 export default function register(api) {
   api.registerMacro({
     name: "ticketId",
-    description: "Current ticket id from session variables.",
-    render: (ctx) => ctx.variables.toMacroText(ctx.variables.get("ticket.id")),
+    description: "Current ticket id from static parameters.",
+    dependencies: ["parameters.ticket.id"],
+    render: ({ env, helpers }) => String(env.parameters["ticket.id"]),
   });
 
   api.registerSlot({
@@ -25,10 +26,10 @@ export default function register(api) {
     options: {
       heading: { type: "string", default: "Ticket context" },
     },
-    render: (ctx) => [
-      String(ctx.options.heading ?? "Ticket context") + ":",
-      "- Ticket: " + ctx.variables.toMacroText(ctx.variables.get("ticket.id")),
-      "- Project: " + ctx.helpers.normalizePath(ctx.runtime.options.cwd),
+    render: ({ item, options, env, helpers }) => [
+      String(options.heading ?? "Ticket context") + ":",
+      "- Ticket: " + String(env.parameters["ticket.id"]),
+      "- Project: " + helpers.normalizePath(String(env.runtime.cwd)),
     ].join("\n"),
   });
 }
@@ -57,7 +58,7 @@ Use the slot declaratively:
 - Missing custom slots are validation warnings until their module is loaded.
 - Registration ownership is disposed when the runtime shuts down.
 
-The API provides `cwd`, `forgeDir`, `extensionPath`, helpers, registration functions, and `getRegisteredMacros()` / `getRegisteredSlots()`. Global `forgeDir` is `~/.pi/forge`; project `forgeDir` is `<project>/.pi/forge`.
+The API provides `forgeDir`, `extensionPath`, helpers, registration functions, and `getRegisteredMacros()` / `getRegisteredSlots()`. Custom macro renderers receive a frozen `{ env, helpers }` and dependencies from the declaration. Global `forgeDir` is `~/.pi/forge`; project `forgeDir` is `<project>/.pi/forge`.
 
 Reusable Pi packages may import `registerMacro` and `registerSlot` from `@zihanw/pi-forge`. The directory loaders are intended for small trusted customizations without package boilerplate.
 

package/docs/guides/debugging.md

@@ -30,7 +30,7 @@ Saved payloads can contain prompt and conversation content. Keep them out of ver
 
 ## Common checks
 
-- Unexpected duplicate task: set the `chat-history` slot's `includeLastUserMessage` to `false` when a later block uses `{{lastUserMessage}}`.
+- Unexpected duplicate task: set the `chat-history` slot's `includeLastUserMessage` to `false` when a later block uses `{{ runtime.lastUserMessage }}`.
 - Missing tool: inspect stack `allow`/`deny` policy and `/preset status`; the tool-call guard enforces the selected stack even if another extension modifies active tools.
 - Skill still visible: use `replace` mode when Pi's base prompt must not include its own skill listing.
 - Missing custom slot: trust the project, check `.pi/forge/extensions`, reload, and inspect diagnostics.