pi-fabric 0.22.2 → 0.22.3

package/docs/interface.md

@@ -0,0 +1,87 @@
+# Interface & commands
+
+Fabric's UI is built on the public `pi-code-previews` cooperative shell and a general-purpose, theme-aware activity surface. Users do not need to install `pi-code-previews` separately.
+
+## Code previews
+
+`fabric_exec` inherits the user's border/background mode, collapsed-result behavior, error styling, and tool-call timing without taking ownership of Pi's built-in tool renderers. Its renderer adds a numbered TypeScript preview, live phase/call activity, and compact phase/nested-call summaries. Fabric refreshes these settings at session start and applies the project-local layer only when Pi marks the project as trusted.
+
+Nested `pi.read`/`pi.bash`/`pi.grep`/`pi.find`/`pi.ls`/`pi.write`/`pi.edit` calls use the same preview settings and presentation vocabulary as `pi-code-previews`: offset-aware read gutters, syntax and secret warnings, grouped/highlighted grep matches, iconized path trees, bash warnings and output metadata, and proposed/applied edit or overwrite diffs with compact context, line gutters, full-row backgrounds, and word-level emphasis. Large inputs fall back to escaped plain text with an explicit notice. The same layer renders in-memory Activity call details. Rich call data is retained only in current-session renderer/activity state; durable traces keep their existing bounded confidentiality projection.
+
+Fabric keeps its orchestration-specific behavior around those previews. Multi-call rows remain compact, live write composition and phase/call progress remain visible, and agent/actor audits can expose their current child tools directly in the parent card with owner, runner, and run metadata. One execution-wide debounce coalesces parallel nested-call updates before they reach Pi. Long ANSI rows are bounded without clearing the enclosing tool background, and the widget lease plus result-to-source row transfer continue stabilizing completion height. The highlighter initializes lazily and falls back to plain text until ready. Collapsed previews use the configured expand keybinding (for example `Ctrl-O`).
+
+## Activity surface
+
+Fabric owns a general-purpose, theme-aware activity surface for any agent setup:
+
+- A compact widget above the chat (like `pi-supervisor`) follows the current phase in its header and lists active/completed one-shot agents plus active actor workers. Their recent or running child tools are nested beneath the owner with compact edit/write change lines and owner metadata. Extensions, custom items, and shared tasks/state remain summarized by the header or available in the dashboard instead of occupying widget rows. The widget retains completed agent rows and a per-run high-water height so tool finalization does not pull the chat upward, resets that lease when a newer run starts, and keeps the latest completed summary visible until that newer run replaces it. When a collapsed Fabric result becomes shorter at completion, the card reveals a bounded number of otherwise-hidden TypeScript source lines to replace those rows without blank padding; any residual deficit shrinks naturally.
+- `/fabric` (or `/fabric dashboard`) opens a responsive interactive overlay with two top-level views: **Activity** and **Topology**. Topology contains **Run** and **Project mesh** child views. `1` opens Activity, `2` opens Topology on Run, `3` opens Topology on Project mesh, and `r` toggles Activity and Topology · Run.
+  - **Activity** places the workflow phase sidebar on the left and the selected phase's activity on the right. The user-facing Pi agent is always present as **Main**, including when no Fabric child exists and regardless of the status filter, so it remains an actionable queue target. The right pane orders entities under type headings for agents, actors, tools, extensions, tasks, custom items, and shared state, with a blank row between groups. Selectable and rendered group order are identical, while one-shot agents remain stable in creation order; attention-priority work keeps default focus instead of Main stealing it.
+  - **Topology · Run** is rooted at **Main**, even before a retained run exists, followed by the full-width, phase-grouped one-shot agent hierarchy for the selected run. It connects recursive children to their parents and keeps completed recursive leaves from a bounded in-memory status snapshot after nested run directories are cleaned, until the parent is cleaned up or the Fabric session shuts down. Large runs stay centered on the selected agent and replace omitted regions with directional summaries of hidden, active, blocked, and failed nodes while preserving phase and ancestor context.
+  - **Topology · Project mesh** is independent of retained run selection. It uses selectable **Main** (with the exact session identity used for routing) as the live project root, then maps persistent actors and their host-event/delivery configuration, transient agents observed in mesh traffic, project topics and subscriber edges, shared-state ownership, and normalized recent event routes. Route rows aggregate repeated traffic by source, target, topic, and kind; this is a bounded live window, not an audit-history replacement. The two-row recent mesh-event feed remains reserved above the footer, including when it has fewer events, so switching views does not collapse the dashboard. Global actor templates are intentionally absent because they are not live project nodes. Topics and routes are selectable and have read-only detail views; actors retain their normal controls.
+  - **Inspection and control** are shared across views. Agent detail includes Markdown-rendered task/results, highlighted YAML values, model, current tool, usage, worktree, and attach metadata. Tool-call details show highlighted command/file/edit inputs and Markdown or YAML outputs; persisted bash calls retain their command text. Space peeks at an agent or actor transcript and `t` toggles summary/transcript detail. The initial bounded tail is paged directly from Pi RPC, Pi session, and Claude stream-JSON logs; `k`/Up at the loaded top fetches one older page, `g` loads and jumps to the true beginning, and `G` jumps to the growing tail and resumes follow mode. Assistant text uses native Markdown, while tool calls retain structured arguments/results for the same read/bash/search/edit/write previews used by the parent card. Credential/token redaction applies before rendering. Main and active one-shot agents can be steered or queued a follow-up; one-shot agents can also be safely stopped. Persistent actors accept `s` mailbox messages and expose model (`m`), thinking (`e`), host events (`v`), instructions (`i`), mailbox clearing (`c`), and export (`x`) where configured. Observed remote mesh agents accept best-effort `s`/`u` routing by exact identity. Global templates remain available from Activity for import (`p`), instruction editing (`i`), and deletion (`d`).
+- `/fabric settings` opens an inline settings view that mirrors Pi core's `/settings` (top and bottom borders, fuzzy search, section submenus) and writes changes to `fabric.json`. Trusted projects write to `<project>/.pi/fabric.json`; untrusted sessions write to the global `~/.pi/agent/fabric.json`. Full code mode, capture, executor, approvals, and UI changes apply immediately; mesh, subagent, and MCP changes persist and take effect on the next `/fabric reload`. The UI section includes a default-on **Nested tool calls** toggle and a global **Nested tool debounce** (`0`/Off through `1000ms`, default `100ms`); both apply immediately. The Subagents section selects the default runner and keeps independent Pi and runtime-enumerated Claude model pickers. List editors for `subagents.defaultTools` and `capture.keepVisible` toggle known tools on and off; `keepVisible` candidates include `fabric_exec` plus every captured extension tool.
+
+### Keybindings
+
+- `1` opens Activity; `2` and `3` enter the Topology parent on Run and Project mesh respectively. `r` toggles Activity and Topology · Run. Inside Topology, `←`/`→`, `h`/`l`, or Tab switches its Run/Project mesh child view; in Activity those keys switch panes. `↑`/`↓` or `j`/`k` select, `g`/`G` jump to the first/last selectable node, Enter focuses or inspects, `f` cycles status filters, `[`/`]` move through retained runs in Activity or Run topology, and `?` opens contextual help.
+- On Main, `s` sends a user-authored message/steer and `u` queues a user-authored follow-up through Pi's host queue.
+- On agents and actors, Space peeks at the transcript and `t` toggles transcript/summary detail; inside a transcript, `g` loads/jumps to the true top and `G` jumps to the bottom and follows new output. On active one-shot agents, `s` opens a steer editor, `u` queues a follow-up, and pressing `x` twice stops the run. Observed remote mesh agents expose best-effort `s`/`u` delivery.
+- On actors, `s` queues a serial mailbox message, `m` changes the model, `e` thinking, `v` host events, and `i` instructions; `x` exports to global, `p` imports a global template, `d` deletes one, and Esc backs out or closes.
+
+## Data-driven activity
+
+The surface is data-driven. Fabric automatically instruments nested provider calls, subagents, persistent actors, and task-shaped mesh entries. A workflow can add domain-specific labels and arbitrary progress without adding extension UI code:
+
+```ts
+await workflow.configure({ name: "Release train", description: "Build, verify, and publish" });
+await phase("Build", { total: packages.length });
+await workflow.item({
+  id: "docs",
+  label: "Documentation",
+  status: "running",
+  completed: 2,
+  total: 5,
+});
+await workflow.event({ message: "Canary passed", level: "success" });
+```
+
+External Fabric providers can emit structured `context.activity()` updates for an entity, progress message, or metrics. This keeps the TUI generic while allowing a virtual provider to expose richer live state.
+
+```ts
+async invoke(actionName, args, context) {
+  context.activity?.({ type: "entity", id: job.id, kind: "custom", name: job.name });
+  context.activity?.({ type: "progress", message: "Indexing package 3/12" });
+  context.activity?.({ type: "metrics", tokens: 4200, toolCalls: 9 });
+  return job.result;
+}
+```
+
+## Commands
+
+```text
+/fabric status
+/fabric dashboard
+/fabric settings
+/fabric reload
+/fabric providers
+/fabric captured [query]
+/fabric agents
+/fabric actors
+/fabric messages <actor-id>
+/fabric attach <subagent-id>
+/fabric stop <actor-or-subagent-id>
+```
+
+Actor slash commands mirror the [global template API](agents.md#global-actor-templates): `/fabric global` lists templates, `/fabric import <name> [as <new>]` stamps one into the project, and `/fabric export <id> [--overwrite]` promotes a project actor. `/fabric log <id>` previews an actor or run transcript; `/fabric export-log <id> [path]` writes the raw `session.jsonl` plus retained `runs/` to disk.
+
+## Headless focused agents
+
+Pi already runs one-shot, non-interactive agents with `pi -p` (`--print`), and it reads piped stdin as part of the prompt — so a focused agent composes with pipes, cron, git hooks, and CI like a Unix program, with no wrapper needed:
+
+```bash
+git diff | pi -p --no-session -t read,grep "Review this diff for concrete defects."
+pi -p --no-session --mode json -e <path-to-pi-fabric> "Map the persistence layer."
+```
+
+`--no-session` keeps the run ephemeral, `-t` restricts the tool allowlist, `--mode json` emits a structured event stream for scripting (`| jq`), and `-e <pi-fabric>` loads Fabric so the agent can use `fabric_exec`. The process exits non-zero on failure. See `pi --help` for the full flag list.

package/docs/memory-recall.md

@@ -0,0 +1,347 @@
+# Memory & Recall
+
+Pi Fabric's `memory` provider searches Pi session JSONL files. Session JSONL is
+the source of truth; the memory index is derived, disposable state.
+
+The index uses structural extraction only. It does not classify goals,
+preferences, errors, or other prose concepts with regexes. Roles, tool names,
+timestamps, entry IDs, operation addresses, tool errors, and tool argument
+paths come from typed session fields.
+
+## Active branches
+
+`memory.recall`, `memory.sessions`, and `memory.expand` accept
+`branches: "active" | "all"`. The default is `"active"` for every scope,
+including `project`, `global`, and explicit `session:<id-or-path>` scopes.
+Only records carried on each session's active parent-linked path contribute to
+hot text, cold vocabulary, structural addresses, segments, and entry counts.
+An abandoned sibling therefore cannot match by default. `branches: "all"` is
+an explicit request to include every branch; responses, segments, cold
+pointers, and session rows identify their branch mode.
+
+For the current live session, Fabric calls the extension context's live
+`SessionManager.getBranch()` and `getLeafId()` getters for every memory action.
+This observes `/tree` navigation even when no new record has been appended. For
+another persisted session, Fabric matches Pi 0.80.6's persisted semantics: the
+last persisted non-header entry is the leaf, duplicate IDs resolve to the last
+record in the ID map, and `parentId` links are followed from that leaf to a
+root. Append order is never treated as one transcript. A parent cycle is
+stopped defensively and makes coverage incomplete with
+`invalid_parent_graph`.
+
+Each derived record carries `branches` and a SHA-256 `lineageFingerprint` over
+the selected branch mode, leaf, and active path IDs. Active and all caches have
+separate filenames. Cache policy includes branch mode, lineage fingerprint,
+and privacy settings, so navigation, append, or policy changes rebuild the
+relevant derived record without contaminating the other mode. `sourceHash`
+continues to cover the complete JSONL source, including off-lineage records.
+
+## Cache V5
+
+Cache records use `cacheVersion: 5`. Older or malformed records are removed and
+rebuilt from source; they are never migrated or interpreted as V5. Refresh also
+removes orphan records, records whose encoded cache path does not match their
+source identity, and records for deleted source sessions.
+
+Every cache record contains the exact session file path, branch mode,
+lineage fingerprint, privacy policy, and a SHA-256 `sourceHash`, in addition to
+source mtime and size. A same-size rewrite with a preserved mtime therefore
+invalidates the record. Cache directories and files are created with `0700`
+and `0600` permissions on a best-effort basis.
+
+A hot shard contains bounded normalized entry text plus `indexCoverage`. A cold
+digest contains:
+
+```ts
+{
+  cacheVersion: 5,
+  kind: "digest",
+  sessionId, file, cwd,
+  mtime, size, sourceHash,
+  branches, lineageFingerprint, policy,
+  firstTs, lastTs, entryCount,
+  filesTouched, toolHistogram, errorCount,
+  vocabulary,   // sorted unique canonical strings; no posting lists
+  addresses,    // structural entry/operation identities, stored separately
+  indexCoverage,
+  cacheBytes, cacheSourceRatio
+}
+```
+
+Cold vocabulary maps an exact lexical term only to the containing session. It
+does not retain a per-term list of entry indices. Consequently a cold result is
+a session pointer with exact `sessionFile` and `sourceHash`, never an inferred
+entry range. Exact entry IDs, indices, and Fabric operation addresses are
+returned after explicit hydration.
+
+`maxColdVocabularyBytes` bounds vocabulary construction for each session and
+`maxColdCacheBytes` is a hard per-session persisted-cache bound. If either cap
+is reached, `indexCoverage.complete` is false and contains an explicit reason.
+Structural addresses or vocabulary may be retained only as exact prefixes when
+the cache-size cap requires it; this is always reported as
+`max_cold_cache_bytes`, never silently treated as complete. `cacheSourceRatio`
+reports persisted cache bytes divided by source bytes.
+
+## Exact lexical queries
+
+`queryMode` is explicit:
+
+- `"literal"` is the default;
+- `"regex"` is opt-in.
+
+Literal mode never inspects punctuation to guess whether input looks like a
+regular expression and never compiles the input with `RegExp`. A path such as
+`src/foo.ts` is therefore literal input.
+
+`tokenize.ts` is the single canonical tokenizer used for literal queries, hot
+BM25 scoring, and cold vocabulary creation. It applies Unicode NFKC
+normalization, extracts Unicode letters, numbers, and underscores, then
+lowercases. Literal terms use exact canonical-token equality. Matching is
+lexical OR across the unique query terms; there is no stemming, synonym
+expansion, phrase inference, or semantic regex classification.
+
+For a cold session whose coverage is complete, every unique canonical token in
+normalized source text occurs exactly once in the sorted vocabulary. Rare terms
+remain exactly discoverable as long as the configured vocabulary and cache
+bounds are not exceeded. If a bound is exceeded, an empty result is explicitly
+non-authoritative.
+
+Hot text is limited by Unicode scalar count, not raw UTF-16 code units. The cut
+therefore cannot split a surrogate pair and always remains valid UTF-8. Because
+hot shards do not retain a separate complete tail vocabulary, truncating any
+normalized entry sets shard `indexCoverage.complete: false` with reason
+`max_entry_chars`. A token occurring only after the cut cannot produce an
+authoritative no-match; recall says `No indexed matches` and includes that
+reason. Expansion still re-reads the complete source record.
+
+## Bounded regular expressions
+
+Regex mode runs JavaScript regex only in a disposable worker thread. The host
+never evaluates an untrusted pattern. The worker is forcibly terminated at the
+hard timeout, so catastrophic backtracking cannot continue on the host thread.
+Regex execution is bounded by:
+
+- UTF-8 pattern bytes;
+- haystack item count;
+- aggregate UTF-8 haystack bytes;
+- wall-clock worker timeout.
+
+Hot haystacks are normalized entry text. Cold haystacks are individual bounded
+canonical vocabulary terms, not transcript prose. Invalid patterns, oversized
+patterns, haystack truncation, worker failures, and timeouts return structured
+query coverage. A timeout, for example, returns
+`coverage.complete: false`, reason `regex_timeout`, and a structured
+`coverage.error`. No incomplete regex result is presented as an authoritative
+no-match.
+
+## Tiers, refresh, and work budgets
+
+The `memory.hotSessions` most recently modified sessions are hot. Older sessions
+are cold. A session crossing the boundary loses its old derived tier record
+after the replacement is built. Explicit hydration re-reads source without
+promoting a cold session.
+
+Cache synchronization is bounded by session count and aggregate source bytes.
+Cache cleanup is bounded by inspected cache files and aggregate cache bytes
+(the byte budget is shared with `maxSyncSourceBytes`). Reaching a work budget stops
+additional indexing and sets `coverage.complete: false`; all eligible sessions
+remain counted. There is no unbounded background job or database dependency.
+
+Recall discovers all eligible sessions for query and no-query browse modes.
+`memory.maxSessions` limits session listing only. Search materialization has
+explicit deterministic per-call budgets: 50,000 filtered hot entry candidates,
+10,000 cold digest candidates, and 10,000 grouped result items. Reaching one
+sets coverage incomplete with `candidate_entry_budget`,
+`candidate_digest_budget`, or `candidate_item_budget`. Totals then describe the
+retained deterministic candidate set, not unknown omitted candidates. Coverage
+reports:
+
+```ts
+coverage: {
+  complete: boolean,
+  indexedSessions: number,
+  eligibleSessions: number,
+  staleSessions: number,
+  incompleteSessions: number,
+  reasons: string[],
+  error?: { code: string, message: string }
+}
+```
+
+`No matches` is authoritative only when both cache/index coverage and query
+execution coverage are complete. Otherwise the response says
+`No indexed matches` and reports reasons such as source unavailability,
+`max_entry_chars`, duplicate identities, vocabulary/cache caps, candidate or
+synchronization budgets, or regex limits.
+
+## Scopes
+
+| Scope | Meaning |
+| --- | --- |
+| `session` | Current session, or newest session for the current cwd. |
+| `project` | Sessions in the current cwd's Pi session directory. |
+| `global` | Sessions under the agent directory. This is explicit, never the default. |
+| `session:<id-or-path>` | One source session, explicitly hydrated without promotion. |
+
+Duplicate session IDs are ambiguous. `session:<id>` and `memory.expand` refuse
+an ambiguous ID with `ambiguous_session` and list candidate paths. Use the exact
+session file path from the cold pointer. Duplicate normalized entry IDs and
+operation addresses also make index coverage incomplete (`duplicate_entry_id`
+or `duplicate_operation_address`). Stable-address expansion requires exactly
+one record: zero matches return `address_not_found`, and multiple matches return
+`ambiguous_address`; neither case returns source records.
+
+## Pointers, hydration, and expansion
+
+A cold result has session identity only:
+
+```ts
+{
+  tier: "cold",
+  sessionId,
+  sessionFile,
+  sourceHash,
+  branches,
+  lineageFingerprint,
+  matchedTerms
+}
+```
+
+It does not collapse disjoint term occurrences into a misleading inclusive
+range and does not truncate a hidden list of exact matches. Hydrate the exact
+path and pass the pointer hash:
+
+```ts
+memory.recall({
+  scope: `session:${pointer.sessionFile}`,
+  branches: pointer.branches,
+  expectedSourceHash: pointer.sourceHash,
+  expectedLineageFingerprint: pointer.lineageFingerprint,
+  query: "rare_token"
+})
+```
+
+Hydrated/hot segments include `exactMatches` with exact normalized entry index,
+entry ID, and operation address. Recall first groups all retained hot matches
+into entry segments, combines those segments with cold pointers, globally ranks
+the combined item stream, and only then applies `page`/`pageSize`. Thus several
+matches in one segment consume one item, not several pages. Responses expose
+stable `totalItems`, `totalMatches`, and `hasNext` for that retained stream.
+No-query browse follows the same pagination path and has no earlier 25-entry or
+`maxSessions` cap. An optional inclusive `entryRange` may bound
+hydration, but both endpoints must be valid session indices. Out-of-range or
+negative addresses return structured `index_out_of_bounds` errors rather than
+being clamped or silently dropped.
+
+`memory.expand` re-reads full, untruncated source text and accepts indices,
+stable entry IDs, operation addresses, or an inclusive range:
+
+```ts
+memory.expand({
+  session: pointer.sessionFile,
+  branches: pointer.branches,
+  expectedSourceHash: pointer.sourceHash,
+  expectedLineageFingerprint: pointer.lineageFingerprint,
+  indices: [12, 14]
+})
+memory.expand({ session: pointer.sessionFile, entryIds: ["entry-uuid"] })
+memory.expand({ session: pointer.sessionFile, operationAddresses: ["entry-uuid/7"] })
+```
+
+Hydration and expansion compare `expectedSourceHash` with the current source
+and `expectedLineageFingerprint` with the selected live or persisted lineage.
+A rewrite, append, or active-leaf navigation returns structured
+`stale_pointer` and no source content when it changes an expected binding.
+Results include source hash, branch mode, and lineage fingerprint so callers
+can retain pointer integrity. Under active mode an off-lineage stable address
+returns `address_not_found`; the same address can be expanded only by an
+explicit `branches: "all"` request.
+
+A valid `FabricExecutionTraceV1` on an outer `fabric_exec` result emits one child
+record per operation immediately after the outer normalized entry. Each child
+keeps `parentEntryId`, `operationAddress`, exact `toolName`, `ref`, `provider`,
+`action`, typed `filesTouched`, `outcome`, and a bounded structured `operation`
+object. Expansion re-reads and re-normalizes source rather than reconstructing
+operations from output prose.
+
+Valid `FabricBranchSummaryDetailsV1` envelopes emit typed child records for
+user, phase, and operation facts. Children preserve the original fact address,
+ref/provider/action/tool/outcome/arguments and structurally derived paths, plus
+`carrierEntryId`, `carrierParentId`, and `carrierFromId`. Operation facts expand
+by their original operation address; user and phase facts use that address as
+their stable entry ID. Repeated nested summaries are deduplicated by exact fact
+address in source order, so the earliest carrier is deterministic. Addresses
+must be unique within each consumed details envelope; a duplicate rejects that
+envelope and marks coverage incomplete. Unknown or malformed details and all
+branch-summary prose remain non-semantic.
+
+## Local-cache privacy and deletion
+
+The memory index is local derived state, not an encryption or semantic-privacy
+boundary. Depending on configuration and tier, cache JSON can retain plaintext
+lexical vocabulary, cwd and file paths, source pointers and hashes, structural
+tool metadata, selected user/assistant/custom-message content, tool arguments,
+and (with the default `indexToolOutput: true`) selected tool output. Cold
+vocabulary alone can reveal exact words even though it has no posting lists.
+Thinking text is excluded by default but is plaintext when explicitly enabled.
+No secret scanning is attempted; privacy is structural inclusion or exclusion,
+not regex classification.
+
+Fabric requests `0700` for cache directories and `0600` for cache files, but
+this is best effort and inherits the host filesystem, account, backup, and
+administrative trust model. The cache is not encrypted. Project configuration
+is read only for a project Pi has marked trusted; otherwise only global Fabric
+configuration applies. A `global` memory search is an explicit scope and is
+not the default, but global indexing still creates local derived records for
+sessions selected by that explicit call.
+
+Deleting a Pi session removes the source of truth. A later memory refresh
+best-effort removes its orphaned cache records; delete the configured
+`memory.indexDir` as well when immediate cache removal is required. Removing
+that directory is safe because all records are disposable and rebuilt from
+remaining session JSONL. Deleting caches does not delete source sessions,
+filesystem backups, or copies outside the index directory.
+
+## Configuration
+
+```json
+{
+  "memory": {
+    "enabled": true,
+    "indexDir": "~/.pi/agent/fabric/memory-index",
+    "maxSessions": 500,
+    "maxEntryChars": 2000,
+    "indexThinking": false,
+    "indexToolOutput": true,
+    "hotSessions": 50,
+    "maxColdVocabularyBytes": 524288,
+    "maxColdCacheBytes": 1048576,
+    "maxSyncSessions": 10000,
+    "maxSyncSourceBytes": 536870912,
+    "maxCacheCleanupFiles": 100000,
+    "regexMaxPatternBytes": 1024,
+    "regexMaxHaystackTerms": 20000,
+    "regexMaxHaystackBytes": 2097152,
+    "regexTimeoutMs": 250
+  }
+}
+```
+
+- `maxSessions`: session-list discovery budget only; recall uses candidate and
+  indexing budgets instead.
+- `maxEntryChars`: persisted hot entry-text Unicode-scalar limit; any cut marks
+  lexical coverage incomplete, while expand re-reads full source.
+- `indexThinking`: assistant thinking blocks enter normalized text and lexical
+  vocabulary only when true. Default: false.
+- `indexToolOutput`: tool-result bodies, bash output, and typed Fabric operation
+  results enter derived text only when true. Default: true, deliberately chosen
+  for coding recall. When false, typed tool name/ref/action, error/outcome, and
+  structurally extracted path metadata remain searchable; output bodies do not.
+- `hotSessions`: globally newest sessions retaining hot shards.
+- `maxColdVocabularyBytes`: per-session canonical vocabulary bound.
+- `maxColdCacheBytes`: hard per-session cold cache-file bound.
+- `maxSyncSessions` / `maxSyncSourceBytes`: synchronous indexing work budgets.
+- `maxCacheCleanupFiles`: synchronous cache-file count budget; cleanup bytes use
+  `maxSyncSourceBytes`.
+- `regexMaxPatternBytes`, `regexMaxHaystackTerms`,
+  `regexMaxHaystackBytes`, `regexTimeoutMs`: isolated regex execution bounds.

package/docs/programmatic-compaction.md

@@ -0,0 +1,208 @@
+# Programmatic compaction
+
+Programmatic compaction lets the model (or a skill, or a peer) **ask** the host
+to compact its own context — or a running subagent's context — at a safe
+boundary, instead of compaction being only a token-threshold reflex. It is the
+distillation of two proven ideas into first-principles primitives native to
+pi-fabric:
+
+- **Harness enforcement** motivates a single gated channel from thought to
+  action, with the host — not the model — deciding when it is safe to act.
+- **Deterministic compaction** avoids adding another model call to a context
+  transition and makes repeated results testable.
+
+Both point at the same primitive: compaction should be a **deliberate,
+labeled transition** of the agent's own context, requested by the model
+(**advisory**) and committed by the host only at a safe boundary
+(**committed**).
+
+## Why advisory-then-committed
+
+The model runs inside the context it would compact. If it could compact the
+running context directly, it would race with its own in-flight turn: tool calls
+mid-execution, partial plans, and unresolved steering. Pi Fabric avoids that
+race with a typed, validated write path and an open status path.
+
+Pi Fabric therefore exposes compaction as **two separable acts**:
+
+1. **Advisory** — `compact.request` (host) or `agents.compact` (child) only
+   *records an intent*. It never touches the context. It is a write-risk,
+   schema-validated declaration: "I think this context should be
+   compacted, with these instructions, for this reason."
+
+2. **Committed** — the host, at a boundary it knows to be safe, forwards the
+   intent to `ExtensionContext.compact()` (host) or to the child pi's `compact`
+   RPC frame (child). For the host, that boundary is `agent_settled`: the
+   handler awaits Pi's callback completion or error before Pi publishes its
+   public settled event. For a child, the worker waits for the child's own
+   `agent_settled`, then sends a correlated compact request and keeps the RPC
+   channel open until it observes both the response and `compaction_end`. It
+   never sends compact while the child turn is active.
+
+There is exactly one write path from intent to action. The model cannot
+compact the running context directly; it can only ask. The ask is a single
+replaceable slot — a new request replaces the pending one, keeping the latest
+instructions.
+
+## First principles, mapped
+
+| Principle | How programmatic compaction realizes it |
+| --- | --- |
+| The context is a cache, not the store. | Compaction is an explicit, labeled transition of the cache, not a silent eviction. The intent and the last commit are recorded outside the context (`status()`) and survive it. |
+| Derived views are pure functions of the log. | `CompactStatus` is a pure snapshot of the controller's recorded intents and commits — never of the compacted context itself. |
+| Enforcement in the harness beats discipline in the prompt. | The model only *requests*; the host *commits* at `agent_settled`. The gate is in the harness (`maybeCommit` + the `agent_settled` handler), not in the prompt. |
+| Compaction is a deliberate, advisory-then-committed act at a model-chosen task boundary. | The model chooses *when to ask* and *with what instructions*; the host chooses *when to commit* (a safe boundary). A token-threshold reflex still exists in pi core; this adds a deliberate path on top of it. |
+
+## API surface
+
+### Host session — the `compact` provider
+
+Always available (no config guard). Exposed through `fabric_exec` as
+`compact.request`, `compact.status`, `compact.cancel`.
+
+```ts
+// Record an advisory intent. Replaces any pending one. Returns immediately;
+// the host commits it at the next agent_settled boundary.
+await compact.request({
+  reason: "the file map and the failing test are the only live state",
+  instructions: "Keep the failing test name and the file map; drop the rest.",
+  preserve: ["Auth regression is still open", "tests/auth.test.ts"], // optional
+  requestedBy: "model", // optional; default "model"
+});
+
+// Read the pending intent and the last committed/failed compaction info.
+const status = await compact.status();
+// { pending?: { reason?, instructions?, preserve?, requestedBy, requestedAt },
+//   last?:   { at, requestedBy, status: "committed"|"failed",
+//             summary?, tokensBefore?, estimatedTokensAfter?, error? } }
+
+// Clear a pending intent before the host commits it.
+await compact.cancel();
+```
+
+Risk classes: `request` is `write` (it mutates host session state); `status`
+and `cancel` are `read`.
+
+`instructions` alone is forwarded as ordinary Pi `customInstructions`, so manual
+`/compact` text and programmatic requests have the same Fabric rendering. When
+`preserve` is present, the controller encodes `{version: 1, instructions?,
+preserve}` behind an exact versioned prefix plus JSON. The compaction and branch
+hooks strictly decode that shape and render valid bounded values under
+`[Compaction Request]`. On tree navigation, Pi's explicit
+`replaceInstructions: true` mode is delegated to Pi/default summarization:
+Fabric cannot execute an arbitrary replacement summarizer prompt and therefore
+produces no typed Fabric branch details in that mode.
+
+The prefix is reserved: malformed JSON/scalars, duplicate decoded protocol
+keys (including escaped aliases), unknown fields or versions, invalid types,
+unpaired UTF-16 surrogates, excessive structure, or exceeded limits return a
+structured decode error and cancel instead of falling back to prose. A bounded
+structural parser performs these checks before canonicalization; it does not
+recover protocol data with regex. The rejected payload is never rendered. A
+context with UI/RPC notification support receives a bounded error. The exact
+`__pi_vcc__` value retains compaction-routing precedence and has no special
+effect on the tree hook.
+
+`compact.request` uses a bounded TypeBox schema before argument mapping.
+Instructions are limited to 8192 characters and 8192 UTF-8 bytes; `preserve` to
+16 items; each item to 2048 characters and 2048 UTF-8 bytes; and the complete
+encoded prefix-plus-JSON request to 16 KiB. The decoder checks aggregate source
+bytes before structural parsing, validates duplicate keys/scalars/surrogate
+pairing while parsing, and checks preserve count before iterating or
+canonicalizing items. Ordinary manual/Pi instructions remain bounded explicit
+text rather than typed protocol input.
+
+#### Commit semantics
+
+- `maybeCommit(context)` is awaited from the host's `agent_settled` handler —
+  never mid-turn, never while a turn is in flight. The returned Promise settles
+  on `onComplete`, `onError`, a synchronous startup throw, or a pre-start abort.
+- It is a no-op when nothing is pending. Reentrant calls share the in-flight
+  Promise rather than starting a second compaction.
+- A new `request()` while a commit is in flight is allowed: it replaces the
+  pending intent. The in-flight commit proceeds with the intent it captured; on
+  completion it clears *that* intent (by identity), leaving any newer intent for
+  the next settled boundary.
+- On pi's `onComplete`: the intent is cleared and `last` records
+  `status: "committed"` with the summary and token counts.
+- On pi's `onError` with `"Compaction cancelled"` or `"Already compacted"`:
+  the intent is cleared **quietly** — nothing to compact, no failure recorded.
+- On any other error: the intent is cleared and `last` records
+  `status: "failed"` with the message. If `compact()` itself throws
+  synchronously, the same failure path applies.
+
+### Subagent compaction — `agents.compact`
+
+```ts
+const handle = await agents.spawn({
+  task: "Audit auth flows.",
+  tools: ["read", "grep", "find", "ls"],
+});
+
+// Advisory: the worker queues this until the child is fully settled.
+await agents.compact({ id: handle.id, instructions: "Keep the finding list." });
+
+return await agents.wait({ id: handle.id });
+```
+
+- Appended to the same `<runDir>/steer.jsonl` channel as `agents.steer` — the
+  orchestrator (or any peer via the mesh relay) can request a child compaction
+  without stopping or respawning it, preserving the child's accumulated
+  context.
+- The worker tails `steer.jsonl` but does not forward compact during an active
+  child turn. It waits for `agent_settled`, then sends
+  `{"id":"...","type":"compact","customInstructions":"..."}` (the
+  instructions field is omitted when absent). See pi's [RPC `compact`](https://github.com/earendil-works/pi-coding-agent/blob/main/docs/rpc.md).
+- The worker correlates the compact response by `id`, observes the matching
+  compaction lifecycle through `compaction_end`, records queued/in-flight/
+  completed/failed status, and only then closes stdin for one-shot shutdown.
+  A rejected response, aborted compaction, or `compaction_end.errorMessage` is
+  recorded as failed without aborting the child turn.
+- Multiple requests waiting before the boundary coalesce to one request with
+  the latest instructions. Requests arriving during an in-flight compaction
+  coalesce into one deterministic follow-on request before shutdown.
+- **Claude-runner children are rejected** with a clear error: the official
+  Claude Code CLI exposes no compact RPC, so a fresh run is the only way to
+  reset a Claude child's context. Compaction is a Pi-runner primitive.
+- Risk class: `agent`.
+
+## Audit and observability
+
+- **Activity surface**: `compact.request` and `agents.compact` emit
+  `context.activity` updates (entity + progress) inside the `fabric_exec` call
+  that issued them, following the existing provider pattern. Host commits and
+  child enqueues are therefore visible in the dashboard and widget.
+- **Mesh**: when the mesh is enabled, the host controller publishes best-effort
+  events to the durable `fabric.compact` topic on each transition:
+  `kind: "requested"` when an intent is recorded, and
+  `kind: "committed" | "failed"` when the host commits. Pi's benign
+  `"Compaction cancelled"` / `"Already compacted"` outcomes clear quietly and
+  do not publish a commit event. Other
+  Fabric participants (persistent actors, peer sessions) can subscribe to
+  observe compaction transitions. Activity-only sessions (mesh disabled)
+  silently skip this.
+- **Status query**: `compact.status()` is the context-independent, in-memory
+  record of the pending intent and the last commit for the current initialized
+  extension session. It survives compaction itself, but not extension reload,
+  session replacement, process restart, or shutdown.
+
+## Configuration
+
+None required. Programmatic compaction is a first-principles primitive and is
+always available. There is intentionally no `compact` config block: the model
+decides when and how to ask; the host decides when to commit; neither needs
+configuration to be safe.
+
+## Files
+
+| File | Role |
+| --- | --- |
+| `src/core/compact-controller.ts` | Pending-intent controller: `request`, `cancel`, `status`, `maybeCommit`. Single replaceable slot; typed preserve encoding; in-flight guard; quiet-clear on cancelled/already-compacted. |
+| `src/providers/compact-provider.ts` | Fabric provider exposing bounded TypeBox-validated `request` (write, including optional `preserve: string[]`), `status` (read), `cancel` (read). Always registered; activity audit. |
+| `src/fabric-state.ts` | Constructs the controller with mesh-publish hooks; registers the provider; resets on re-init/shutdown. |
+| `src/index.ts` | Invokes `state.compact.maybeCommit(context)` in the existing `agent_settled` handler. |
+| `src/subagents/types.ts` | `SubagentSteerEntry["type"]` extended with `"compact"`; optional `instructions` field. |
+| `src/subagents/manager.ts` | `compact(id, instructions?)` appends a compact entry through the steer channel; rejects Claude-runner children. |
+| `src/worker.ts` | Feeds compact controls into the child boundary coordinator and observes Pi RPC lifecycle events. |
+| `src/subagents/compact-control.ts` | Coalesces child requests, waits for `agent_settled`, correlates compact response + `compaction_end`, records outcome, and gates one-shot stdin close. |
+| `src/providers/agents-provider.ts` | `agents.compact({id, instructions?})` action (risk: agent) with activity audit. |