@xingular/harness 0.1.7 → 1.1.1

package/docs/architecture/04-prompt.md

@@ -0,0 +1,122 @@
+# 04 Prompt
+
+## Purpose
+
+One compiler builds every model instruction the harness gives. It merges the pinned variant's
+sections, the harness's own sections and the application's sections into an ordered,
+budgeted pack, then splits the pack into a **stable system prompt** (cached by the provider) and
+a **context update** (changing state appended to native history). Keeping the two apart is what
+keeps provider caching effective across turns.
+
+## Components
+
+| Type / function | Role |
+|---|---|
+| `compile(env, caller, launch, variant)` → `ContextPack` | Builds the ordered, budgeted sections, including the configured data-root path in native file guidance. |
+| `render(env, pack)` → `string` | One string for a session that takes a single prompt. |
+| `sessionLaunch(env, caller, launch, historyFile, variant)` → `{ system, update, pack }` | SDK sessions: stable prefix plus an update with only the dynamic sections that changed since the last delivered snapshot. |
+| `deliveredSnapshot(sessionFile)` | Reads the last delivered snapshot on the active history branch; a compaction forces a fresh one. |
+| `saveForSession`, `copyToRun` | Terminal sessions keep the pack they started with and copy it to each run. |
+| `record`, `load`, `packPath` | The pack each run was given, at `runs/<id>/context.json`. |
+| `toolGroups`, `fit`, `clip` | The tool manifest by group; cutting text to a budget at a paragraph boundary. |
+
+All in `prompt/compile.ts`. A `ContextPack` holds `sections: Section[]` (each with `id`, `title`,
+`body`, `sources`, `budget`, `size`, `cut`), `toolGroups`, `harness` (scope and fingerprint),
+`profile`, `compiledAt` and `delivery` (`systemSectionIds`, `updatedSectionIds`).
+
+## Section order
+
+`SECTION_ORDER`: `harnessIdentity` 5, `system` 25, `guardrails` 40, `memory` 45, `references`
+50, `subagents` 64, `scripts` 66, `wiki` 80, `tools` 90, `reporting` 95. Application sections
+sit among them by their `order`; plan mode uses `plan` at 25. Ties keep insertion order.
+When a run may spawn, `subagents` lists every agent profile it can start (built-in, global,
+project and the variant's c5 names) and replaces the variant's own list (PROP-0029). The `tools`
+section has a Progress paragraph whenever `checklist_write` is available: it asks for a checklist
+early and kept current, because the person follows work from it (PROP-0006).
+
+The fixed `harness-identity` section says `You are an AI agent running in <productName>.` and uses
+`env.productName` (default `Xingular Agent`). Host-provided identity and role sections remain
+separate, so an embedding application can still describe its agent and task.
+
+Budgets (characters): variant sections 30,000 each (already capped at save, so never cut here);
+references 12,000; Wiki pointer 2,500; tools 8,000; reporting 2,000; application sections their
+own `budget`.
+
+## Reporting
+
+Every pack ends with the harness's `reporting` section ("Reporting your work",
+`REPORTING_GUIDANCE` in `prompt/compile.ts`). It is fixed text that shapes the final reply: lead
+with the answer or outcome, then use only headings or structure that help explain it; no fixed list
+of report headings is required. When a concrete verification requires a person because the agent
+could not perform or observe it, the optional **Need human inspection** heading says what remains
+unverified, why, and the specific check the person should make. Report only checks actually run,
+with their results. Markdown tables are used where they help compare options or present repeated
+fields or check results; any format the person, application or current mode asks for takes
+precedence. The section names no application, is never dynamic, and is never cut. See
+[PROP-0001](proposals/PROP-0001-reporting-section.md),
+[PROP-0004](proposals/PROP-0004-flexible-reporting-guidance.md), and
+[ADR-0008](../decisions/ADR-0008-flexible-reporting-guidance.md).
+
+## Delivery
+
+```text
+compile ──► pack ──► stable sections ──► system prompt (+ "Context updates" guidance)
+                └──► dynamic sections ──► compare with delivered snapshot ──► custom message
+                                                                              type xingular-context
+```
+
+- Dynamic sections are the built-in `references` and `wiki`, plus every application section with
+  `dynamic: true`.
+- An update lists only the dynamic sections that changed, and announces a withdrawn one.
+  `references` is always resent when present, because it belongs to the current request.
+- The session appends the update as a hidden custom message and saves its snapshot in the
+  message details (`xingularContext`); recording a pack does not mark it delivered.
+- Terminal sessions use `compile` + `render` once, pass it as `--append-system-prompt`, and keep
+  that pack for the life of the session.
+- Read-only native file guidance states that absolute and `~` paths work when the operating system
+  permits access, names the configured data root, and describes the working folder as a default
+  location. Edits and commands remain subject to the gate.
+- A side-question launch attributes its fixed no-tool instruction to the `side-questions` plugin.
+  Its launch suppresses tool guidance entirely. The main run's effective prompt and current context
+  update are included in the separately bounded, redacted context message (PROP-0019).
+
+## Invariants
+
+- **INV-PRM-1** The prompt is assembled only in `src/prompt/compile.ts`. Nothing else builds model
+  instructions. SDK sessions use `sessionLaunch`; terminal sessions use `compile` + `render`.
+- **INV-PRM-2** The system prompt holds only stable text: identity, role, the variant's sections, the
+  harness's reporting guidance, tool guidance and the tool manifest. Changing state never enters it.
+- **INV-PRM-3** `references`, `wiki`, and every section the application marks `dynamic` are
+  delivered as context updates in native history, never in the system prompt.
+- **INV-PRM-4** An update carries only changed dynamic sections, so an unchanged turn appends
+  nothing. A compaction on the active branch forces the full snapshot again.
+- **INV-PRM-5** The pack only describes tools the caller can call (the frozen capability set).
+  `ask_user` is not part of the compiler's tool-group manifest; its mode-specific availability is
+  resolved with the run's callable tools (page 02).
+- **INV-PRM-6** Each run's pack is recorded beside the run. Failing to record it never fails the
+  session.
+- **INV-PRM-7** The compiler is synchronous (`INV-BND-4`).
+- **INV-PRM-8** Whether a prompt or callable tool-list change keeps the provider's cache is measured
+  with real requests from the application, only with the account owner's permission. Tests do not
+  stand in for it.
+- **INV-PRM-9** Native file guidance describes the existing OS-permission boundary accurately:
+  absolute and `~` paths are valid for read-only file tools; the working folder is not a sandbox.
+  Side-question runs with no callable tools omit tool guidance.
+
+## Relation to DESIGN-0001
+
+[`../system-prompt-design.md`](../system-prompt-design.md) is a draft design (not accepted). Only
+what this page states is binding. Adopting parts of that draft is an architecture change.
+
+## Source map
+
+`src/prompt/compile.ts`, `src/sessions/sessions.ts` (calls `sessionLaunch`),
+`src/terminal/terminals.ts` (calls `compile` + `render`), `src/variant/materialise.ts` (variant
+sections), `src/sessions/planMode.ts` (`plan` section).
+
+## Session naming guidance (PROP-0010)
+
+The Session tool group describes `session_name` when available: name meaningful work early
+with an imperative sentence of at most six words, and revise when the objective changes.
+This is fixed foundation guidance. The current name stays in native session metadata and
+the terminal snapshot; it is never interpolated into the stable system prompt.

package/docs/architecture/05-refinement.md

@@ -0,0 +1,107 @@
+# 05 Refinement
+
+## Purpose
+
+Refinement improves a variant from evidence of how it ran, with model weights fixed
+(Harness²). On one task it runs the current variant, lets improver sessions edit it, probes each
+candidate, composes the best behaviours, and picks one final output without ever seeing a score.
+A person adopts the result. Adopted variants and their lessons go to an experience library that the
+next task's improvers read.
+
+## Components
+
+| Type / function | Module | Role |
+|---|---|---|
+| `Refinements` | `refine/service.ts` | `start`, `wait`, `get`, `list`, `cancel`, `candidates`, `adopt`, `library`, `libraryEntry`, `forget`. |
+| `runRefinement` | `refine/runner.ts` | One refinement end to end; writes `summary.json`. |
+| `runMatrix` | `refine/suite.ts` | Evaluates variant versions on a task set without refining: one probe execution per version × task, recorded as `matrix.json` with comparability keys. Reads with `readMatrix`. |
+| recursion | `refine/recursion.ts` | Parallel (`agg`) and sequential (`seq`) modes; stages `INV`, `DOM`, `AGG`, `COMPOSE`. |
+| `Ledger` | `refine/ledger.ts` | Budget identity: R = 4k rollouts, P = 3k improver sessions. |
+| evidence | `refine/evidence.ts` | Builds the evidence folder and guards it against grader output. |
+| selection | `refine/selection.ts` | Blind choice among anonymised outputs, then reflection. |
+| `Library` | `refine/library.ts` | The experience library per scope and area. |
+| `PiExecutor`, `PiImprover` | `refine/pi.ts` | Executions and improvers as ordinary harness runs, triggers `probe` and `improver`. |
+| `RefineHost` | `refine/host.ts` | The application's part (page 01, surface 7). |
+| prompts | `refine/prompts.ts` | Improver missions. |
+
+## Method
+
+```text
+base variant ──► k base executions (probe runs, isolated workspaces)
+          ──► evidence folder (task, source, trajectories, deliverables, variant, library, pool)
+          ──► improvers: k INV (how the agent works) + k DOM (what a practitioner knows)
+          ──► probe each candidate ──► k compositions from behavioural contrasts ──► k finals
+          ──► blind selection among anonymised outputs ──► reflection ──► candidate for adoption
+```
+
+Parallel mode explores wider; sequential mode refines deeper. Both cost R = 4k executions and
+P = 3k improver sessions. An invalid proposal gets one retry with the validator's message
+(`STAGE_ATTEMPTS = 2`), then inherits the current variant.
+
+An **evaluation matrix** is the measurement half on its own: `runMatrix` runs every
+`{variant version × task}` pair through the same `Executor` contract — no improver, no selection,
+no adoption, and an empty or failed execution is an honest row. Every row records
+deliverability (`ok`, `delivered`) and cost, plus its comparability keys (task-prompt hash,
+executor name, package version, variant fingerprint), so matrices taken across variant versions or
+package commits are diffable directly. An evaluation never moves a head, and it inherits the
+answer-free rule: a variant or record carrying grader output is refused before any execution
+(`INV-REF-1`).
+
+## Contracts
+
+- `start({ task: { id, scope, prompt, ... }, k?, mode?: 'parallel' | 'sequential', baseVersion?, improver?, executor?, retainWorkspaces? })`
+  returns `{ id }` at once; progress arrives as `refine.changed` and `refine.event` notices.
+- With `retainWorkspaces: true` (default `false`), each execution's workspace — base, candidate
+  and final — is copied whole to `rollouts/<label>/workspace/` after its run settles and before
+  the host's `cleanup`. Binary and hidden files are kept, and symbolic links are copied as links.
+  A failed copy is recorded as `workspaceError` in `rollout.json` and never fails the execution.
+  `candidates(id)` reports each execution's `workspace` path, or `null` (PROP-0002).
+- `executor: { model?, effort?, accountId?, poolId? }` names how executions run. For each value
+  the host's `session(...)` wins, then `executor`, then the account default. `summary.json`
+  records `settings: { executor, improver }` as started; each probe run's record holds what it
+  resolved (PROP-0003).
+- `adopt(id, { label? })` commits the chosen (or named) candidate as the scope's head with source
+  `refinement`. It refuses while any plugin code in it is untrusted.
+- Executions run as `sessions.run` with `trigger: 'probe'`, `automatic: true`, in the host's
+  workspace. Improvers run with `trigger: 'improver'`, `automatic: true`, file tools only, no
+  providers, no connected apps, no harness tools, no variant, in a temporary copy of the evidence.
+
+## Invariants
+
+- **INV-REF-1** Refinement is answer-free. No rubric, score, grader output or reviewer verdict
+  reaches an improver, the selector or the library. `refine/evidence.ts` guards every evidence
+  folder (`GRADER_VERDICT_MARKERS` as a tripwire) and `assertTreeHasNoGraderVerdicts` guards the
+  library. Retained workspaces are output for measurement after the fact; evidence, selection,
+  reflection and the library never read them.
+- **INV-REF-2** The selector sees only the request, the source and the outputs, shuffled under
+  arbitrary ids. It never sees a harness, a trajectory or a verdict.
+- **INV-REF-3** The budget identity R = 4k, P = 3k holds in both modes (`Ledger.assertIdentity`).
+  A change to the method changes both modes and their tests together.
+- **INV-REF-4** Refinement never acts in the world. Executions are unattended `probe` runs in a
+  folder the `RefineHost` prepared; anything that needs a person is refused at the gate, never
+  queued.
+- **INV-REF-5** Improvers have file tools only and work on a copy of the evidence.
+- **INV-REF-6** Refinement never moves a head by itself. Only `adopt`, a person's action, does, and
+  only with all plugin code trusted (`INV-VAR-5`).
+- **INV-REF-7** The library carries whole adopted variants with an evidence-backed lesson, never a
+  summary in place of the variant.
+- **INV-REF-8** Per-observation truncation of trajectories is never below `MIN_OBS_TRUNC`
+  (10,000 characters); lower values fabricate defects.
+
+## Data
+
+```text
+<data root>/refine/<id>/      summary.json, task.json, variants/<label>/, rollouts/<label>/,
+                              sessions/<label>/, context/, selection/, reflection/, improver/
+<data root>/refine/<id>/rollouts/<label>/workspace/    only with retainWorkspaces
+<data root>/refine/<id>/      matrix.json, suite.json   (evaluation matrices; see above)
+<data root>/library/<scope>/<area>/INDEX.md
+<data root>/library/<scope>/<area>/variants/<task>/{variant/, EXPERIENCE.md}
+```
+
+## Source map
+
+`src/refine/*.ts`. Method origin: Harness² (`harness2/core/{recursion,ledger,selection,library,context}.py`,
+Apache-2.0). The evaluation matrix is this harness's addition (`refine/suite.ts`), shaped by the
+same executor contract and demo executor. Try it with no model: `npm run build && node dist/cli.js refine demo --k 2`,
+or evaluate a matrix with `node dist/cli.js refine matrix --demo [--data <root>] [--out <file.json>]`.

package/docs/architecture/06-plugins-and-capabilities.md

@@ -0,0 +1,129 @@
+# 06 Plugins and capabilities
+
+## Purpose
+
+A *plugin* is a bundle with a title, a description and contributions: instructions, skills,
+commands, tools, connections, code (a Pi extension) and themes. A Pi package is a plugin as it is;
+so are the harness's own parts, the application's parts, a person's registry, the folder every
+account shares, and what a working folder brings. This part decides what a session loads, who has
+what switched on, which code may run, and what reaches one caller. A *capability set* is the
+callable result for one run.
+
+## Components
+
+| Type / function | Module | Role |
+|---|---|---|
+| `Plugin`, `Contribution`, `PluginDescriptor`, `CONTRIBUTION_KINDS`, `PLUGIN_SOURCES` | `plugins/types.ts` | The model of a plugin and what it brings. |
+| `Plugins` (`loading`, `loaded`) | `plugins/plugins.ts` | Chooses the paths a session or terminal loads; records what each extension registered. |
+| discovery | `plugins/discover.ts` | Finds items with Pi's own resolver, which runs no extension code. |
+| attachments (`load`, `save`, `attached`, `decided`, `set`) | `plugins/attachments.ts` | The one store of what is switched on, per scope. |
+| `BuiltinPluginRegistry`, `BuiltinPluginModule` | `plugins/builtinRegistry.ts`, `plugins/builtin.ts` | Registers built-in modules and their descriptors, world parts, server handlers and terminal command metadata. |
+| `HARNESS_PLUGINS` | `plugins/builtin.ts` | Descriptor view of the registered Harness parts and feature plugins. |
+| `FIRST_PARTY_PLUGINS` | `plugins/firstParty.ts` | `output-trim`, `budget-guard`: inline extensions shipped switched off. `lessons`: switched off; its contribution turns on reflection after a run (page 08, PROP-0008) and loads no Pi code. |
+| `Hooks` | `plugins/hooks.ts` | Runs a package's hooks file after a write. |
+| install / trust | `plugins/install.ts`, `plugins/trust.ts` | Install through Pi's package manager with scripts off; trust code by hash. |
+| inventory / api | `plugins/inventory.ts`, `plugins/api.ts` | `plugins.list`, `plugins.inventory`, attach, trust. |
+| capability api | `capabilities/api.ts`, `capabilities/types.ts` | `list`, `resolve`, `toolIntersection`, `inventory`, `attach`. |
+| run snapshot | `capabilities/runSnapshot.ts` | `freezeRun`, `loadRun`, `capabilitySet`. |
+
+## Contracts
+
+- **Loading.** `Plugins.loading({ cwd, agentDir, caller, contributions, install? })` returns
+  `extensionPaths`, `skillPaths`, `promptPaths`, `themePaths`, `builtins` (terminal only), `hooks`,
+  `found`, `detached`, `projectCode`, `packageCode`. A session opens Pi with its own discovery off
+  and only these paths; a terminal receives them as `--extension`, `--skill`,
+  `--prompt-template` and `--theme` after `--no-extensions --no-skills --no-prompt-templates
+  --no-themes`. Pi still reads `AGENTS.md` and `SYSTEM.md` itself.
+- **Attachments.** `config/plugin-attachments.json` holds `global`, `agents`, `projects` (each
+  `{ on, off }`) and `settings` per plugin. The narrowest scope that says something wins: agent,
+  then project, then everywhere. With no entry an item keeps its default.
+- **Hooks.** A package declares `"harness": { "extends": [...], "hooks": "./hooks.mjs" }`. Events:
+  `wiki.page.saved`, `history.run.ended`. A hook has 30 seconds; its errors are kept in
+  `plugin-hooks.json`. `ctx.call(connection, tool, args)` reaches only a connected app that is on
+  for everyone, and only a tool marked to run without asking.
+- **Capability set** (`CapabilitySet`): `{ version: 1, tools, connectors, contributions }`.
+  `capabilities.resolve` returns the set available to a caller; `capabilities.intersect` /
+  `toolIntersection` narrows it for a child. The required `ask-user` plugin remains attached, but its
+  `ask_user` tool is included only for an attended plan run or a continuation pinned with that
+  capability. Automatic or refinement-triggered continuations with it are rejected rather than
+  weakening the frozen set. The `side-questions` descriptor owns the fixed instruction for the
+  restricted side-question run; it has no tools, providers, connectors, subagents or variant code
+  (PROP-0019).
+
+## Invariants
+
+- **INV-PLG-1** The harness is the only loader. A session or terminal loads exactly what
+  `Plugins.loading` chose, with Pi's discovery switched off. Never pass a session a loader that
+  discovers on its own.
+- **INV-PLG-2** Code waits for trust. A working folder's `.pi/extensions` and the packages its
+  settings name are trusted together; each installed package is trusted on its own. Trust is a
+  hash of the files; a changed file asks again. A loose file a person put in the shared
+  `extensions/` folder needs no trust. Text contributions always load.
+- **INV-PLG-3** Installing runs no code (`npm_config_ignore_scripts`). A package arrives switched
+  off and, if it has code, untrusted. Installing grants no attachment, trust, credential or
+  permission.
+- **INV-PLG-4** The approval gate is the last extension to see a tool call (page 07, `INV-GATE-2`).
+  A plugin that changes a call's input does so before the gate sees it.
+- **INV-PLG-5** Everything model-visible belongs to a plugin. A new prompt section, tool group or
+  tool is declared by a `PluginDescriptor` (`plugins/builtin.ts` for the harness's own), or
+  `plugins.inventory` reports it as `unattributed`.
+- **INV-PLG-6** One store says what is switched on: `plugins/attachments.ts`. A new kind of
+  switchable thing gets a default and is read with `attached`; no second enablement file.
+- **INV-PLG-7** A hook is told, never asked. Hooks run after the write, in the background; a write
+  never waits for a hook and a hook's failure never reaches it. A new write point that should be
+  heard calls `noteChange` (`core/changes.ts`). No hook may stop or change a write.
+- **INV-PLG-8** A hook runs only while its plugin is on for everyone and trusted, checked before
+  each delivery. A working folder's package cannot bring a hook.
+- **INV-PLG-9** Unattended runs load no extension found on disk (`INV-RUN-5`). The harness's own
+  parts are required and refuse to be switched off; switching something on grants no permission.
+  A side-question run also has no variant plugin code or tool guidance (PROP-0019).
+- **INV-PLG-10** Every Harness-managed built-in feature is registered by `BuiltinPluginRegistry`.
+  Its descriptor, world tools, server handlers and terminal command metadata share one owner. A
+  module's world part releases resources through the existing session and Harness lifecycle. This
+  registry does not load third-party Pi code or bypass its trust checks.
+- **INV-PLG-11** Built-in plugin categories are a documentation ownership taxonomy. They are not a
+  `PluginDescriptor` field, a public response field or a second attachment scope.
+- **INV-CAP-1** A capability set only narrows. `intersect` never adds access; a run's set is
+  frozen when it starts (`INV-RUN-2`). An agent profile's `plugins` list keeps only the parent's
+  contributions of those plugins (required plugins always stay) and drops the tools they own; it
+  never adds one (PROP-0029).
+- **INV-CAP-2** The capability contract is generic. It names no application task, role or UI
+  concept.
+- **INV-CAP-3** A capability set says what may be called, not whether a call may run. The gate
+  still decides each call.
+
+## Data
+
+```text
+<data root>/config/plugin-attachments.json     (registry-enabled.json read until the first change)
+<data root>/plugins/<name>/                     packages kept with the data; removal moves to trash/plugins/
+<state dir>/plugin-trust.json                   trusted code, by hash
+<state dir>/plugin-registered.json              what each extension registered last time
+<state dir>/plugin-hooks.json                   hooks and their last run
+<state dir>/accounts/_shared/pi/                settings and packages shared by every account
+```
+
+## Source map
+
+`src/plugins/*.ts`, `src/capabilities/*.ts`, `src/core/changes.ts`.
+
+## Built-in ownership taxonomy (PROP-0020)
+
+These categories make ownership visible in docs and source layout. They do not change plugin
+metadata or availability:
+
+| Category | Features | Foundation boundary |
+|---|---|---|
+| Foundation | Accounts, sessions and run recording, prompt and variant validation, gate, plugin loading/trust/attachments, resource browser, server and terminal frameworks | Release-owned infrastructure and security contracts |
+| World | Web search/read tools, browser tools, page annotations | The isolated browser engine and network policy remain Foundation services |
+| Memory | Wiki, history and trajectory | Storage and run records remain Foundation-owned services |
+| Automation | Subagents, ask-user, checklist, lessons and refinement | Run lifecycle, capability narrowing and approval gate remain Foundation-owned |
+| Integrations | Connected apps/MCP and Hub | Connector credentials and plugin installation remain behind Foundation services |
+| Operations | Doctor and worktrees | Diagnostics and repository operations remain Harness-managed services |
+
+## Terminal resource inspection
+
+The unified browser joins saved plugin/contribution metadata with actual session skills and tool
+registration. A saved on/off state does not prove a resource loaded; origins and owners distinguish
+same-name resources. Protected install/remove, settings, attachment and trust actions call the
+existing services. No second discovery or enablement mechanism is introduced (PROP-0013).

package/docs/architecture/07-tools-and-gate.md

@@ -0,0 +1,103 @@
+# 07 Tools and the gate
+
+## Purpose
+
+A run's tools come from four places: the harness's own tools, the connected-apps broker,
+subagent tools, and the application's tool providers. Every call, whatever its source, passes one
+per-call gate that asks the application's policy, then standing approvals, then the person.
+
+## Components
+
+| Type / function | Module | Role |
+|---|---|---|
+| `HarnessToolDefinition`, `RECALL_TOOLS`, `TRAJECTORY_TOOLS`, `WIKI_READ_TOOLS`, `WIKI_WRITE_TOOLS`, `SKILL_TOOLS`, `DEFAULT_TOOL_NAMES` | `tools/definitions.ts` | The harness's own tools, defined once. |
+| `dispatch` | `tools/dispatch.ts` | One entry point for every History, Wiki and skill call, from a Pi tool, the server or a terminal. |
+| `harnessTools` | `tools/harnessTools.ts` | The definitions as `ToolSpec`s for one session. |
+| `askUserTool` (`ask_user`) | `tools/askUser.ts` | Stops an attended plan run and asks up to five related questions with choices and concise 1–2 word labels; absent from newly resolved normal runs. |
+| `checklistTool` (`checklist_write`) | `tools/checklist.ts` | Writes the run's whole checklist as a `checklist.updated` event (PROP-0006). Read-only for the gate, so plan mode allows it. Terminal sessions register the same tool; it reaches the run through the `checklist` telemetry call. The prompt's Progress group asks the model to keep it current because the user follows work from it. |
+| `subagentTools` | `tools/spawnSubagent.ts` | `spawn_subagent` (requires a short `title`, PROP-0030), `wait_subagents`. |
+| `annotationsPart` (`page_notes_list`, `page_notes_read`, `page_notes_mark_fixed`) | `annotations/part.ts` | World tools over the page notes a person shared with the session or agent (PROP-0017). `page_notes_mark_fixed` records the agent's claim that it fixed a note it was sent; it never resolves a note or changes grants (PROP-0018). |
+| `webTools` | `web/tools.ts` | `web_search` (Brave, key from the secret store), `web_fetch` (checked public reads). An application provides them as a provider. |
+| gate | `sessions/sessions.ts::gate`, `sessions/runner.ts` | `ToolGate { ask, check }`, loaded as the last extension (`xagent-approvals`). |
+| `Approvals`, `decide`, `grant`, `approvalKey` | `sessions/approvals.ts` | Standing approvals. |
+
+Harness tools: `history_search`, `history_read`, `trajectory_search`, `trajectory_read`,
+`wiki_search`, `wiki_read`, `wiki_list`, `wiki_write` (not in the default set), `skill_load`,
+`ask_user` (attended plan runs and pinned continuations only), `checklist_write`. Broker:
+`mcp_list_tools`, `mcp_call_tool`. Native Pi tools: `read`, `grep`, `find`,
+`ls`, `edit`, `write`, `bash`.
+
+## Native file paths
+
+The read-only `read`, `grep`, `find` and `ls` tools accept paths relative to the working folder,
+absolute paths, and `~` paths. The working folder is a default location, not a repository boundary;
+the operating system's access for the Harness process decides what can be read. The compiled file
+guidance names the configured data root. `edit`, `write` and `bash` continue through the existing
+gate and policy. A side-question run has an empty capability set and no tools (PROP-0019).
+
+## The gate, per call
+
+```text
+child run and tool outside its snapshot ─► refuse
+plan mode and tool not research-safe    ─► refuse (before approvals)
+policy.scope(call)
+  not allowed and (automatic or no approval possible) ─► refuse with reason
+policy.authorize(call) throws           ─► refuse
+allowed, not humanOnly, (automatic or not gated) ─► run
+trigger probe or improver               ─► refuse (nobody to ask)
+run.requestApproval(...)                ─► standing approvals may answer, else the person
+  denied                                ─► refuse
+policy.authorize(call) again            ─► run or refuse
+```
+
+Standing approvals (`approvals.json`): an always-approve switch per key (`chat`, or the run's
+`harness`), tools allowed across the state directory per key, and targets remembered per
+session. `humanOnly` calls are never answered by remembered tools, and by the switch only when
+`allowGlobalApproval`. `ApprovalFacts.unattended` disables standing answers; `readOnly` disables
+the switch and global tools.
+
+## Invariants
+
+- **INV-GATE-1** Every tool call of every run passes the gate. No tool source bypasses it,
+  including extension-registered tools (which ask before every call) and the initial Bash command.
+- **INV-GATE-2** The gate is the last extension loaded, after disk, variant and inline extensions, so
+  it sees the final input of every call.
+- **INV-GATE-3** Order is fixed: capability and plan-mode refusals, then the policy's scope and
+  authorize, then standing approvals, then the person, then authorize again.
+- **INV-GATE-4** Refinement runs (`probe`, `improver`) never ask a person; what would need one is
+  refused, never queued (`INV-REF-4`).
+- **INV-GATE-5** SDK tools are registered only if their names are in the run's frozen capability
+  set (`INV-CAP-1`). The terminal may retain question-tool definitions in its private Pi inventory,
+  but normal mode deactivates them; a plan request activates only registered questions under its
+  plan-mode gate.
+- **INV-GATE-6** Tool attachment bytes never enter the run-event stream; only references are
+  recorded.
+- **INV-GATE-7** Tool descriptions name the product with `{product}` (filled by `describeTool`),
+  never an application's name (`INV-BND-2`).
+- **INV-GATE-8** Native read-only file access is bounded by process and operating-system
+  permissions, not by the repository root. Absolute and `~` paths are valid inputs. This does not
+  change the gate or permissions for writes and commands.
+
+## Data
+
+`<state dir>/approvals.json`.
+
+## Source map
+
+`src/tools/*.ts`, `src/web/*.ts`, `src/sessions/sessions.ts`, `src/sessions/runner.ts`,
+`src/sessions/approvals.ts`, `src/sessions/policy.ts`.
+
+## Session naming (PROP-0010)
+
+`session_name` (`tools/sessionName.ts`) binds to the current native session. It maintains
+only Pi session metadata and is explicitly plan-safe, while still subject to capability
+selection and the host policy. The terminal registers the same definition through its Pi
+bridge; standalone policy permits naming without approval. See page 02 for validation and
+persistence. It is a foundation tool, not mutable variant data.
+
+## Diagnostic probes
+
+`/doctor` is a user command, never a model tool. Live diagnostics create separate real runs,
+intersect current capabilities, and execute tool calls through the existing gate. Calls that
+require an additional per-call approval report blocked; batch diagnostic consent does not grant
+standing permission. No raw tool inputs, results, or screenshots enter diagnostic findings.

package/docs/architecture/08-memory-and-history.md

@@ -0,0 +1,94 @@
+# 08 Memory and history
+
+## Purpose
+
+Two kinds of memory sit outside the variant. The **Wiki** holds project notes: linked
+Markdown that people and agents read and write as reference material, never as instructions.
+**History** is the record of earlier runs: recall over what was said, and bounded reads of the
+recorded trajectory. Both are served through harness tools (page 07) and kept searchable in a
+rebuildable SQLite index.
+
+## Components
+
+| Type / function | Module | Role |
+|---|---|---|
+| `paths`, `list`, `read`, `write`, `remove`, `versions`, `version`, `search`, `graph`, `ensureLayout`, `moveLegacy` | `wiki/notes.ts` | The Wiki. Logical module (functions), not a class. |
+| `noteChange` | `core/changes.ts` | Announces a completed write to listeners (hooks). |
+| `indexRun`, `sync`, `search`, `read` | `history/recall.ts` | Recall passages from runs, scoped to the caller. |
+| trajectory search and read | `history/trajectory.ts` | Paged, redacted reads of canonical run events. |
+| `Lessons`, `keptLessons`, `lessonMission` | `history/lessons.ts`, `history/lessonPrompt.ts` | Reflection on finished runs when the `lessons` plugin is on; keeps general lessons as Wiki notes (PROP-0008). |
+| `withIndex`, `migrate`, `MIGRATIONS` | `core/index.ts` | The shared `index.sqlite` connection and its migrations. |
+
+## Contracts
+
+- **Note paths.** A note's path is its file path under `wiki/` without `.md`. Links are
+  `[[path]]`, `[[path|label]]` or a relative Markdown link. Notes live in `home.md`, `library/`
+  (no project) and `projects/<slug>/` (a project the application named in `projects.json`).
+- **Writes.** `write(root, path, body, { expectedUpdatedAt?, actor?, reason? })` refuses text over
+  400,000 characters, text that looks like a credential (`secret_refused`), an unknown project
+  slug, and a stale `expectedUpdatedAt` (`conflict`). The previous text is kept in `wiki-history/`.
+- **Recall scope.** A `Code` caller (a session in a project) reads only session runs of its project,
+  or its own session. A `Chat` caller reads its agent's conversations. `recallExcludeAgents` hides
+  named agents' conversations. A caller never reads a run outside its scope.
+- **Trajectory reads** return pages of canonical execution events, including active runs,
+  bounded and marked when truncated. They omit account identifiers, credential fields,
+  attachment bytes, native adapter payloads and provider reasoning text. They report evidence,
+  never a score.
+- **Lessons** (PROP-0008, [ADR-0010](../decisions/ADR-0010-lessons-in-the-wiki.md)). When the
+  first-party `lessons` plugin is on for a run's caller (its contribution is in the run's
+  `capabilities.json`), a **completed, top-level** run that made at least `minToolCalls` tool calls
+  is queued after it ends. Subagent runs and runs with trigger `probe`, `improver` or `reflection`
+  are never queued. One reflection runs at a time, in the background, and the queue is not kept
+  across a restart.
+  - The reflection is a session with trigger `reflection` on the run's own account and model, file
+    tools `read`, `grep`, `find`, `ls` only, no providers, connections, harness tools or subagents,
+    `automatic: true`; anything needing a person is refused at the gate. Its evidence is a temporary
+    `context/` with `run.md` (the request and the rendered trajectory, observations kept to at least
+    10,000 characters) and `lessons.md` (the kept lessons' paths and cues), checked with the grader
+    tripwire.
+  - It replies `NO_ENTRY`, `SAME <path>`, or one `<experience>` block (`cue`, `lesson`, `evidence`,
+    `confidence`): refinement's lesson shape, which never carries the run's own facts.
+  - A new lesson is written with `notes.write` to `library/lessons/<slug of cue>` with actor
+    `reflection`, frontmatter `title`, `cue`, `confidence`, `sources: [runId]`. `SAME` adds the run
+    id to that note's `sources`. Nothing is written when the lesson is longer than `maxChars`,
+    looks like a credential, carries a grader marker, or the Wiki already keeps `maxLessons`
+    lessons; nothing is evicted.
+  - The `wiki` pointer section says how many lessons there are when there are any.
+- **Index.** Every finished run is indexed when it ends; `open` and `rebuildIndex()` sync from
+  `runs/`.
+
+## Invariants
+
+- **INV-MEM-1** The Wiki is reference material, not instructions. Only the variant shapes an agent
+  (`INV-VAR-1`); the Wiki reaches a prompt only as the dynamic `wiki` pointer and selected
+  `references` (`INV-PRM-3`).
+- **INV-MEM-2** The harness is the only writer of the Wiki and of run records. An application
+  reads them and writes notes only through harness methods (`wiki.*`). Inside the harness, the
+  Wiki is written by `wiki_write`, by `wiki.*` methods, and by `Lessons` (actor `reflection`, only
+  under `library/lessons/`, only while the `lessons` plugin is on for the run's caller).
+- **INV-MEM-3** Every write point calls `noteChange` after the file is in place, and never waits
+  for listeners (`INV-PLG-7`). (Removal does not yet: drift D-7.)
+- **INV-MEM-4** A note write keeps the previous text as a version. Nothing deletes history except
+  an explicit person's action.
+- **INV-MEM-5** Files are the source of truth. `index.sqlite` is derived and can be deleted and
+  rebuilt (`harness.rebuildIndex()`).
+- **INV-MEM-6** Index migrations are append-only once released. Each runs in one transaction
+  after the file is backed up; a failure restores the previous index.
+- **INV-MEM-7** Recall and trajectory reads respect the caller's scope and redact secrets. They
+  never return another scope's runs.
+- **INV-MEM-8** Wiki code is synchronous (`INV-BND-4`).
+
+## Data
+
+```text
+<data root>/wiki/            home.md, library/, projects/<slug>/
+<data root>/wiki-history/    earlier versions of notes
+<data root>/legacy/          an earlier Wiki's memory blocks, facts, revisions, Inbox (moved once)
+<data root>/index.sqlite     derived: pages, links, runs, recall passages (FTS5)
+<data root>/projects.json    read only: the application owns it
+```
+
+## Source map
+
+`src/wiki/notes.ts`, `src/history/recall.ts`, `src/history/trajectory.ts`, `src/history/lessons.ts`, `src/core/index.ts`,
+`src/core/changes.ts`, `src/core/projects.ts`, `src/core/secrets.ts`.