@xingular/harness 2.0.0 → 2.0.1-unstable

package/docs/architecture/04-prompt.md

@@ -0,0 +1,277 @@
+# 04 Prompt
+
+## Purpose
+
+One compiler builds every model instruction the harness gives. A terminal compiles its pack again
+only when the person switches the conversation to another variant version, at a turn boundary
+(ADR-0065); that costs the provider cache once. It merges the pinned variant's
+sections, the harness's own sections and the application's sections into an ordered,
+budgeted pack, then splits the pack into a **stable system prompt** (cached by the provider) and
+a **context update** (changing state appended to native history). Keeping the two apart is what
+keeps provider caching effective across turns.
+
+## Components
+
+| Type / function | Role |
+|---|---|
+| `compile(env, caller, launch, variant)` → `ContextPack` | Builds the ordered, budgeted sections, including the configured data-root path in native file guidance. |
+| `render(env, pack)` → `string` | One string for a session that takes a single prompt. |
+| `renderSized(env, pack)` → `{ text, sizes, framing }` | The same string, plus the characters each section occupied and the framing around them. |
+| `sessionLaunch(env, caller, launch, historyFile, variant)` → `{ system, update, pack }` | SDK sessions: stable prefix plus an update with only the dynamic sections that changed since the last delivered snapshot. |
+| `deliveredSnapshot(sessionFile)` | Reads the last delivered snapshot on the active history branch; a compaction forces a fresh one. |
+| `renderCompactionCheckpoint` | Compiles a bounded current-request, todo list and plan-state message for post-compaction model requests. |
+| `saveForSession`, `copyToRun` | Terminal sessions keep the pack they started with and copy it to each run. |
+| `record`, `load`, `packPath` | The pack each run was given, at `runs/<id>/context.json`. |
+| `toolGroups`, `fit`, `clip` | The tool manifest by group; cutting text to a budget at a paragraph boundary. |
+
+Assembly is in `prompt/compile.ts`; shared standard-section data is in `prompt/standard.ts`.
+A `ContextPack` holds `sections: Section[]` (each with `id`, `title`,
+`body`, `sources`, `budget`, `size`, `cut`), `toolGroups`, `harness` (scope and fingerprint),
+`profile`, `compiledAt` and `delivery` (`systemSectionIds`, `updatedSectionIds`).
+
+`delivery.sizes` records what each delivered section occupied when it was rendered — `system` and
+`update` maps by section id, plus the framing characters that belong to no section (the product
+heading, the section headings, the context-update guidance). It is what lets a surface attribute the
+context window without re-parsing a prompt whose bodies may contain their own headings. A pack
+recorded before the compiler measured them has no `sizes`, and a reader then falls back to
+`Section.size` and says the split is approximate.
+
+## Section order
+
+`SECTION_ORDER`: `harnessIdentity` 5, `system` 25, `sessionState` 30, `guardrails` 40, `memory` 45, `references` 50, `requiredSkills` 52, `subagents` 64, `scripts` 66, `wiki` 80, `tools` 90, `reporting` 95. Application sections
+sit among them by their `order`; plan mode uses `plan` at 25. Ties keep insertion order.
+When a run may spawn, the dynamic `subagents` section lists every profile it can start (built-in,
+global, project and variant c5 names) and replaces the variant's own list (PROP-0029). Normal-mode
+guidance in that section explains when delegation is worth its overhead, profile selection,
+self-contained briefs, independent parallel work, and parent ownership of integration. Plan mode
+lists only effective profiles eligible for read-only research and requires a named profile; keeping
+this section dynamic leaves the stable system prompt unchanged across modes. The `tools`
+section has a Progress paragraph whenever `todo_write` is available: it asks for `todo_write` before
+the first other tool call when work has several steps, asks for the list to stay current, then
+requests a final update before the reply. Only fully completed work is
+marked completed; unfinished or blocked steps remain accurate (PROP-0006).
+
+`session-state` is a built-in dynamic section at order 30 when a top-level run supplies it. It
+names the current session or says “unnamed”, shows this run's todo (empty at the start) and a
+previous run's list as context, and says whether this is the first turn. It describes only tools
+in the frozen set: the name line appears only with `session_name`, and the todo line only with
+`todo_write`. It never enters the stable prefix. Structured sessions deliver it through
+`sessionLaunch` and the changed-section rule; the terminal appends the same text as a request-time
+custom message.
+
+`required-skills` is a built-in dynamic section at order 52 when `Launch.requiredSkills` names any
+skill. Named in a message as `$name`, a skill is required for that request: the compiler reads its
+`SKILL.md` from the pinned variant (the built-in `plugin-authoring` skill fills in when the variant
+has none) through `readSkillFile`, and the section carries the skill's text and the directive to
+follow it. It is bounded like `references`, names an unavailable skill honestly, and changes no tool
+or permission. Structured sessions deliver it through `sessionLaunch`; the terminal builds the same
+text with `requiredSkillsText` from Pi's loaded skills and appends it as a hidden request-time custom
+message. The section belongs to the request, so an identical one is delivered again.
+
+The fixed `harness-identity` section says `You are an AI agent running in <productName>[, based on
+<model>]. You and the user share one workspace, and your job is to collaborate with them until their
+intended goal is completely handled.` It uses `env.productName` (default `Xingular Agent`) and
+`Launch.model` when the host knows the model. `Launch.standard: { cwd, project?: { name } | null }` requests shared role, personality and working-folder
+sections from `prompt/standard.ts`; the compiler supplies common connections, file and annotation
+guidance for the effective tools. Terminal preparation defaults to standard sections (explicit
+`standard: null` opts out); the host interactive path requests the same sections. Hosts may add
+attributed add-on and task context, not a duplicate standard role or connection policy (PROP-0040).
+
+Budgets (characters): variant sections 30,000 each (already capped at save, so never cut here);
+references 12,000; Wiki pointer 2,500; tools 8,000; reporting 12,000 (ADR-0076); the
+standard session's sections: role 1,400, autonomy 2,600, personality 1,200, permission 1,800,
+steering 2,000, writing 1,900 (ADR-0072); application sections their own `budget`.
+
+## Reporting
+
+Every pack ends with the harness's `reporting` section ("Reporting your work",
+`REPORTING_GUIDANCE` in `prompt/reporting.ts`, imported by `prompt/compile.ts`). It is fixed text
+that shapes the final reply. ADR-0076 adopts Codex's final-answer guidance (`gpt-6.1-sol`), amended
+by ADR-0079: the reply is self-contained, outcome-first, explains the work and its evidence and
+limits, and gives the reader enough context to pick it up cold. It explicitly reports checks that
+were not run or failed; a person-owned next step appears only when one is needed. For substantial
+replies with distinct parts, it often uses content-named Markdown `#` and `##` headings; there is no
+fixed heading vocabulary or required template, and short replies may stay in prose. Code fences name
+the language; commands the person must run use a `bash` fence, while short references stay inline.
+Tables are used for mappings and comparisons. Length follows complexity; prose comes first, with
+lists when the information is genuinely parallel, sequential or easier to compare. References and formatting
+follow rules supplied by the active session or host. Progress updates stay in the standard `writing`
+section, not `reporting`. A subagent's reply gives its caller the concise work
+summary and key findings, with useful absolute paths and evidence; snippets appear only when exact
+text is load-bearing, and code merely read is not recapped. The section ships two examples (a
+headed screen change and a heading-free blocked publish), the shape-by-work table and a four-line
+pre-send list; the blocked-publish example puts its login command in a `bash` fence. A person,
+application or mode format takes precedence. The section names no application, is never dynamic, and is never cut; its
+budget is 12,000 characters, with 5,081 currently delivered (ADR-0079, amended for language-labeled
+code fences).
+
+## When a reply may end
+
+A reply ends when the request is done, when the compaction guard ends it (INV-RUN-13), when a person
+interrupts, or when the work cannot continue without the person. An open todo item, a passing check,
+a partial result or a long task is not such a reason: the words below ask the model to carry the
+request to completion, and they are the only layer (ADR-0075). The harness does not send a reply
+back and does not mark it unfinished from the todo list; a reply the model ends settles `completed`.
+**No model judges an ordinary reply**; a goal's completion claim is the one thing a model checks
+(INV-RUN-16). A goal that ends blocked settles `needs_person`; a goal that ends on a limit, a pause
+or a stall settles `unfinished`.
+
+The words are what asks the model to carry on, and none of them may read as permission to stop with
+work left. A person's own session (`launch.standard`, `prompt/standard.ts`)
+reads six stable sections adapted from OpenAI Codex's gpt-6.1-sol instructions (Apache-2.0, see
+`NOTICE.md`), the same for every model so the terminal and a host read the same text (INV-PAR-1):
+
+- `role` — the assistant's job, and the extra work it does without asking (the obvious adjacent
+  step, what the work exposes, the check it needs) inside the working folder and away from what a
+  person must decide (a deploy, a secret, remote data, money). Extra work and departures from the
+  plan are allowed on those terms and reported at the end.
+- `autonomy` — bias to action; persist until the intended goal is complete; "can you...", "help
+  me..." are instructions; no partial or "helpful enough" solution; an open todo item, a passing
+  check, a partial result or a long task is not a reason to end the turn; the todo list shows
+  progress and the request is the scope; which decisions are the agent's own; a document the request
+  points to is the request; a question the request or files already answer is not a question for the
+  person; the only reasons to stop, one of them a fix to a test or error that does not come from
+  this session's work (and then say exactly what is needed).
+- `personality` — curious, thoughtful and candid; keeps its own judgment, disagrees when it has
+  reason and reconsiders when the evidence warrants it, without flattery or forced enthusiasm.
+- `permission` — authorisation persists; finish the authorised work first so that approval is the
+  last step; no permission for reversible, read-only or already authorised work; a test or error
+  that does not come from this session's work is reported, not fixed, unless the person asks (a
+  failure is this session's work when the agent's changes caused it), and is kept off the todo
+  list; no messages to other people unless asked; when the agent must ask, it says why and where
+  the requirement comes from.
+- `steering` — a new message steers the active task; a correction means "fix it"; compaction does
+  not end the task, and work across compactions is one chain.
+- `writing` — plain words, the outcome first, evidence and limits, routine verification summarised,
+  no AI-slop phrases, no "X, not Y" contrast, no invented labels, no needless apology.
+
+Other fixed text that shapes the end of a reply:
+
+- `TODO_GUIDANCE` (`tools/todo.ts`): the list is progress, not scope; the request, and any document
+  it points to, is the scope; add steps as the work shows them; list a handoff's steps instead of one
+  "remaining work" item; before ending, compare the request, not the list, with what is done and
+  checked. An open step is work to do in this reply, not something to report.
+- `reporting` (`REPORTING_GUIDANCE`): the final answer is self-contained, outcome-first and gives
+  the reader enough context to pick up cold. It reports checks accurately, including checks not run
+  or failed; names a person's next step only when one is needed; and often uses content-named `#` / `##`
+  headings for substantial multi-part replies without a fixed heading set. A subagent gives its
+  caller a concise report of the work and findings. Length and structure follow the content, and
+  references follow the active session's format; a login the work needs is named with its exact
+  command.
+- `ask_user` (`tools/askUser.ts`): for a decision the request and files do not answer, never for
+  permission to do work inside the request; no answer means continue with best judgment (Codex's
+  Default-mode rule).
+- In a person's own session the `Progress` tool group says that a reply with open items is sent back.
+  There is no tool to stop "on purpose": ADR-0069 removed `checkpoint_write` and ADR-0071 removed
+  `finish`. The only tool that ends work is `update_goal` in a goal the person started (INV-TOOL-9).
+
+All of these are fixed text in the stable prefix, so changing them changes the cached prefix once.
+A live terminal session keeps the pack, and the extension code, it started with; a harness update
+reaches it only when the session restarts.
+
+A goal's text (`sessions/goalPrompts.ts`, adapted from Codex's `ext/goal/templates`) is not in the
+prefix: the continuation, round-cap and edit prompts ride as hidden user messages, with the
+objective escaped and wrapped as data (INV-PRM-2).
+
+## Delivery
+
+```text
+compile ──► pack ──► stable sections ──► system prompt (+ "Context updates" guidance)
+                └──► dynamic sections ──► compare with delivered snapshot ──► custom message
+                                                                              type xingular-context
+```
+
+- Dynamic sections are the built-in `references`, `required-skills` and `wiki`, plus every application section with
+  `dynamic: true`.
+- The rendered sizes of both halves are recorded with the pack (`delivery.sizes`), so the context a run
+  was given can be attributed after the fact without replaying the compiler.
+- An update lists only the dynamic sections that changed, and announces a withdrawn one.
+  `references` and `required-skills` are always resent when present, because they belong to the current request.
+- After a compaction during the active run, the foundation checkpoint extension detects the new
+  summary and uses `renderCompactionCheckpoint` once to build one hidden, ephemeral context message.
+  It inserts that same message directly after the summary in each later model request while the run
+  continues, when a todo list or applicable plan instructions existed at the compaction. A moving
+  message would make each request differ from the previous one before its end and defeat provider
+  caching (ADR-0067). A run that starts from already-compacted history gets no checkpoint.
+  It contains only the current request and available state. It is not written to native history, and its state is
+  discarded when the run settles or the branch changes.
+- The session appends the update as a hidden custom message and saves its snapshot in the
+  message details (`xingularContext`); recording a pack does not mark it delivered.
+- Terminal sessions use `compile` + `render` once, pass it as `--append-system-prompt`, and keep
+  that pack for the life of the session.
+- Native file guidance states that absolute and `~` paths work for reads when the operating
+  system permits access, names the configured data root, and describes the working folder as a
+  default location. Native writes and Bash filesystem effects default to the run's pinned project
+  or worktree root; an outside file target is allowed, and the Harness pauses that one call until
+  the person allows or denies it with a control. The tools section opens with the approval rule:
+  call the tool, never ask for permission in a message, and never end the turn to wait for an
+  answer. Bash is sandboxed unless the eligible user explicitly enables Full Permission. Edits and
+  commands remain subject to the gate.
+- A side-question launch attributes its fixed no-tool instruction to the `side-questions` plugin.
+  Its launch suppresses tool guidance entirely. The main run's effective prompt and current context
+  update are included in the separately bounded, redacted context message (PROP-0019).
+
+## Launch parity status (2026-11-19)
+
+Shared section generation is present in source; `surfaceParity.test.ts` checks compiler equality.
+That does not verify actual prepared launches, registered schemas, credentials, variant pins or
+provider-cache reuse. PROP-0040 also requires one Harness-owned interactive default for roots,
+variant scope and credential backend. Adoption and runtime acceptance remain in progress; explicit
+roots/scopes and continuation pins take precedence. Matching `--data` alone cannot reconcile a
+file SecretStore with a host Keychain. See [parity](../parity.md).
+
+## Invariants
+
+- **INV-PRM-1** The prompt is assembled only in `src/prompt/compile.ts`. Nothing else builds model
+  instructions. SDK sessions use `sessionLaunch`; terminal sessions use `compile` + `render`.
+- **INV-PRM-2** The system prompt holds only stable text: identity, role, the variant's sections, the
+  harness's reporting guidance, tool guidance and the tool manifest. Changing state never enters it.
+- **INV-PRM-3** `session-state`, `references`, `required-skills`, `wiki`, and every section the application marks
+  `dynamic` are delivered as context updates in native history, never in the system prompt.
+- **INV-PRM-4** An update carries only changed dynamic sections, so an unchanged turn appends
+  nothing. A compaction on the active branch forces the full snapshot again.
+- **INV-PRM-5** The pack only describes tools the caller can call (the frozen capability set).
+  `ask_user` is not part of the compiler's tool-group manifest; its mode-specific availability is
+  resolved with the run's callable tools (page 02).
+- **INV-PRM-6** Each run's pack is recorded beside the run. Failing to record it never fails the
+  session.
+- **INV-PRM-7** The compiler is synchronous (`INV-BND-4`).
+- **INV-PRM-8** Whether a prompt or callable tool-list change keeps the provider's cache is measured
+  with real requests from the application, only with the account owner's permission. Tests do not
+  stand in for it.
+- **INV-PRM-9** Native file guidance describes the boundary accurately: absolute and `~` paths
+  are valid for read-only file tools, which remain bounded by OS permissions rather than the
+  workspace root. Native file writes and Bash filesystem effects are workspace-scoped by default;
+  the working folder itself is not the boundary. Side-question runs with no callable tools omit
+  tool guidance.
+- **INV-PRM-10** Compaction checkpoints are assembled by this compiler, bounded, and delivered only
+  as ephemeral context while their active run continues, at a fixed place right after the compaction
+  summary and unchanged until the next compaction. They do not change the stable system prompt
+  or native history, and do not carry hidden reasoning (PROP-0033, ADR-0032, ADR-0067). A work-slice handoff is
+  the same kind of bounded projection: the `checkpoint` event is rendered into the continuation
+  section by `continuationBody`, never into the system prompt (PROP-0041).
+- **INV-PRM-11** The Progress group names `checkpoint_write` when it is in the frozen capability
+  set, with the same budget as the other tool guidance, and says only that a bounded reply records a
+  handoff when its budget is reached. It does not describe the limits themselves, which the guard
+  states at the boundary.
+
+## Relation to DESIGN-0001
+
+[`../system-prompt-design.md`](../system-prompt-design.md) is a draft design (not accepted). Only
+what this page states is binding. Adopting parts of that draft is an architecture change.
+
+## Source map
+
+`src/prompt/compile.ts`, `src/sessions/sessions.ts` (calls `sessionLaunch`),
+`src/terminal/terminals.ts` (calls `compile` + `render`), `src/sessions/sessionState.ts`
+(the native name reader and state shape), `src/variant/materialise.ts` (variant sections),
+`src/sessions/planMode.ts` (`plan` section).
+
+## Session naming guidance (PROP-0010)
+
+The Session tool group describes `session_name` when available: call it once before the first
+other tool call when a request needs tools or changes state, with an imperative sentence of at
+most six words, and revise only when the objective changes.
+This is fixed foundation guidance. The current name stays in native session metadata and
+the terminal snapshot; it may appear in the request-scoped `session-state` update, but it is never
+interpolated into the stable system prompt.

package/docs/architecture/05-refinement.md

@@ -0,0 +1,134 @@
+# 05 Refinement
+
+## Purpose
+
+Refinement improves a variant from evidence of how it ran, with model weights fixed
+(Harness²). On one task it runs the current variant, lets improver sessions edit it, probes each
+candidate, composes the best behaviours, and picks one final output without ever seeing a score.
+A person adopts the result. Adopted variants and their lessons go to an experience library that the
+next task's improvers read.
+
+## Components
+
+| Type / function | Module | Role |
+|---|---|---|
+| `Refinements` | `refine/service.ts` | `start`, `taskFromRun`, `wait`, `get`, `list`, `cancel`, `candidates`, `candidateDiff`, `candidateOutput`, `adopt`, `library`, `libraryEntry`, `forget`. |
+| `runRefinement` | `refine/runner.ts` | One refinement end to end; writes `summary.json`. |
+| `runMatrix` | `refine/suite.ts` | Evaluates variant versions on a task set without refining: one probe execution per version × task, recorded as `matrix.json` with comparability keys. Reads with `readMatrix`. |
+| recursion | `refine/recursion.ts` | Parallel (`agg`) and sequential (`seq`) modes; stages `INV`, `DOM`, `AGG`, `COMPOSE`. |
+| `Ledger` | `refine/ledger.ts` | Budget identity: R = 4k rollouts, P = 3k improver sessions. |
+| evidence | `refine/evidence.ts` | Builds the evidence folder and guards it against grader output. |
+| selection | `refine/selection.ts` | Blind choice among anonymised outputs, then reflection. |
+| `Library` | `refine/library.ts` | The experience library per scope and area. |
+| `PiExecutor`, `PiImprover` | `refine/pi.ts` | Executions and improvers as ordinary harness runs, triggers `probe` and `improver`. |
+| `RefineHost` | `refine/host.ts` | The application's part (page 01, surface 7). The default host replays a git snapshot in a shared clone (`prepareSnapshotWorkspace`). |
+| `snapshotWorkingTree` | `worktrees/snapshot.ts` | The working tree as a typed turn found it, as a commit (PROP-0052). |
+| prompts | `refine/prompts.ts` | Improver missions. |
+
+## Method
+
+```text
+base variant ──► k base executions (probe runs, isolated workspaces)
+          ──► evidence folder (task, source, trajectories, deliverables, variant, library, pool)
+          ──► improvers: k INV (how the agent works) + k DOM (what a practitioner knows)
+          ──► probe each candidate ──► k compositions from behavioural contrasts ──► k finals
+          ──► blind selection among anonymised outputs ──► reflection ──► candidate for adoption
+```
+
+Parallel mode explores wider; sequential mode refines deeper. Both cost R = 4k executions and
+P = 3k improver sessions. An invalid proposal gets one retry with the validator's message
+(`STAGE_ATTEMPTS = 2`), then inherits the current variant.
+
+An **evaluation matrix** is the measurement half on its own: `runMatrix` runs every
+`{variant version × task}` pair through the same `Executor` contract — no improver, no selection,
+no adoption, and an empty or failed execution is an honest row. Every row records
+deliverability (`ok`, `delivered`) and cost, plus its comparability keys (task-prompt hash,
+executor name, package version, variant fingerprint), so matrices taken across variant versions or
+package commits are diffable directly. An evaluation never moves a head, and it inherits the
+answer-free rule: a variant or record carrying grader output is refused before any execution
+(`INV-REF-1`).
+
+## Contracts
+
+- `start({ task: { id, scope, prompt, ... }, k?, mode?: 'parallel' | 'sequential', baseVersion?, improver?, executor?, retainWorkspaces? })`
+  returns `{ id }` at once; progress arrives as `refine.changed` and `refine.event` notices.
+- With `retainWorkspaces: true` (default `false`), each execution's workspace — base, candidate
+  and final — is copied whole to `rollouts/<label>/workspace/` after its run settles and before
+  the host's `cleanup`. Binary and hidden files are kept, and symbolic links are copied as links.
+  A failed copy is recorded as `workspaceError` in `rollout.json` and never fails the execution.
+  `candidates(id)` reports each execution's `workspace` path, or `null` (PROP-0002).
+- `executor: { model?, effort?, accountId?, poolId? }` names how executions run. For each value
+  the host's `session(...)` wins, then `executor`, then the account default. `summary.json`
+  records `settings: { executor, improver }` as started; each probe run's record holds what it
+  resolved (PROP-0003).
+- `adopt(id, { label?, tag? })` commits the chosen (or named) candidate as the scope's head with
+  source `refinement`, records `tunedFor` from the model its execution ran on, and tags it when
+  asked. It refuses while any plugin code in it is untrusted, and checks the tag before committing.
+- `taskFromRun({ runId? | sessionId?, nativeSessionId?, guidance? })` builds a task that replays
+  one turn a person typed: its request text, the scope its variant came from, and its
+  `workspaceSnapshot` as `task.workspace`. With a session it takes the latest typed turn; with a
+  native session id it also finds turns recorded before the terminal restarted on another version.
+  It refuses a run something else started, a run with no variant, and a run with no snapshot, and
+  lists what the replay leaves out (attachments, untracked files the snapshot omitted).
+- `task.workspace: { kind: 'git-snapshot', repo, commit }`: the default host prepares each
+  execution as a `--shared` clone of `repo` checked out at `commit`, with HEAD and the index at the
+  commit's parent, so tracked edits, deletions and new files look as they did. Ignored files named
+  by `.worktreeinclude` are copied from the repository as it is now. The person's repository gains
+  no worktree, ref or file.
+- `task.guidance`: where the person asked improvers to look, at most 2,000 characters, refused when
+  it names measurement (`checkGuidance`). It is added to the INV, DOM, aggregate and compose
+  missions as a focus block that directs attention and keeps `KEEP_BASE` valid, and recorded on the
+  summary. It never reaches the selector or the reflector.
+- `candidateDiff(id, label)` and `candidateOutput(id, label)` return one candidate's file changes
+  from the base and what its execution produced, for review before adopting.
+- Executions run as `sessions.run` with `trigger: 'probe'`, `automatic: true`, in the host's
+  workspace. Improvers run with `trigger: 'improver'`, `automatic: true`, file tools only, no
+  providers, no connected apps, no harness tools, no variant, in a temporary copy of the evidence.
+
+## Invariants
+
+- **INV-REF-1** Refinement is answer-free. No rubric, score, grader output or reviewer verdict
+  reaches an improver, the selector or the library. `refine/evidence.ts` guards every evidence
+  folder (`GRADER_VERDICT_MARKERS` as a tripwire) and `assertTreeHasNoGraderVerdicts` guards the
+  library. Retained workspaces are output for measurement after the fact; evidence, selection,
+  reflection and the library never read them.
+- **INV-REF-2** The selector sees only the request, the source and the outputs, shuffled under
+  arbitrary ids. It never sees a harness, a trajectory, a verdict or the person's guidance.
+- **INV-REF-3** The budget identity R = 4k, P = 3k holds in both modes (`Ledger.assertIdentity`).
+  A change to the method changes both modes and their tests together.
+- **INV-REF-4** Refinement never acts in the world. Executions are unattended `probe` runs in a
+  folder the `RefineHost` prepared; anything that needs a person is refused at the gate, never
+  queued.
+- **INV-REF-5** Improvers have file tools only and work on a copy of the evidence.
+- **INV-REF-6** Refinement never moves a head by itself. Only `adopt`, a person's action, does, and
+  only with all plugin code trusted (`INV-VAR-5`).
+- **INV-REF-7** The library carries whole adopted variants with an evidence-backed lesson, never a
+  summary in place of the variant.
+- **INV-REF-8** Per-observation truncation of trajectories is never below `MIN_OBS_TRUNC`
+  (10,000 characters); lower values fabricate defects.
+
+## Data
+
+```text
+<data root>/refine/<id>/      summary.json, task.json, variants/<label>/, rollouts/<label>/,
+                              sessions/<label>/, context/, selection/, reflection/, improver/
+<data root>/refine/<id>/rollouts/<label>/workspace/    only with retainWorkspaces
+<data root>/refine/<id>/      matrix.json, suite.json   (evaluation matrices; see above)
+<data root>/library/<scope>/<area>/INDEX.md
+<data root>/library/<scope>/<area>/variants/<task>/{variant/, EXPERIENCE.md}
+```
+
+## Source map
+
+In the terminal, `/refine [--k N] [--mode parallel|sequential] [--model id] [--executor-model id]
+[guidance]` replays the latest request typed in this conversation. It shows the request, its
+snapshot, the focus, the run count (4k executions, 3k improver sessions, selection and reflection)
+and the models, asks, then runs in the background with a status item. Executions use this chat's
+model unless `--executor-model` names another; `--model` sets the improver's. `/refine status`,
+`/refine stop` and `/refine review` follow it; review marks the selector's pick, shows each
+candidate's changes and output, and adopts any candidate, then switches the conversation to it.
+
+`src/refine/*.ts`. Method origin: Harness² (`harness2/core/{recursion,ledger,selection,library,context}.py`,
+Apache-2.0). The evaluation matrix is this harness's addition (`refine/suite.ts`), shaped by the
+same executor contract and demo executor. Try it with no model: `npm run build && node dist/cli.js refine demo --k 2`,
+or evaluate a matrix with `node dist/cli.js refine matrix --demo [--data <root>] [--out <file.json>]`.

package/docs/architecture/06-plugins-and-capabilities.md

@@ -0,0 +1,208 @@
+# 06 Plugins and capabilities
+
+## Purpose
+
+A *plugin* is a bundle with a title, a description and contributions: instructions, skills,
+commands, subagent profiles (`agent`), tools, connections, code (a Pi extension) and themes. A Pi package is a plugin as it is;
+so are the harness's own parts, the application's parts, a person's registry, the folder every
+account shares, and what a working folder brings. This part decides what a session loads, who has
+what switched on, which code may run, and what reaches one caller. A *capability set* is the
+callable result for one run.
+
+## Components
+
+| Type / function | Module | Role |
+|---|---|---|
+| `Plugin`, `Contribution`, `PluginDescriptor`, `CONTRIBUTION_KINDS`, `PLUGIN_SOURCES` | `plugins/types.ts` | The model of a plugin and what it brings. |
+| `Plugins` (`loading`, `loaded`) | `plugins/plugins.ts` | Chooses the paths a session or terminal loads; records what each extension registered. |
+| discovery | `plugins/discover.ts` | Finds items with Pi's own resolver, which runs no extension code. |
+| attachments (`load`, `save`, `attached`, `decided`, `set`) | `plugins/attachments.ts` | The one store of what is switched on, per scope. |
+| `BuiltinPluginRegistry`, `BuiltinPluginModule` | `plugins/builtinRegistry.ts`, `plugins/builtin.ts` | Registers built-in modules and their descriptors, world parts, server handlers and plugin-owned terminal commands. Harness core commands live in `commands/registry.ts`, shared by both surfaces. |
+| `HARNESS_PLUGINS` | `plugins/builtin.ts` | Descriptor view of the registered Harness parts and feature plugins. |
+| `FIRST_PARTY_PLUGINS` | `plugins/firstParty.ts` | `output-trim`, `budget-guard`: inline extensions shipped switched off. `lessons`: switched off; its contribution turns on reflection after a run (page 08, PROP-0008) and loads no Pi code. |
+| `Hooks` | `plugins/hooks.ts` | Runs a package's hooks file after a write. |
+| install / trust | `plugins/install.ts`, `plugins/trust.ts` | Install through Pi's package manager with scripts off; trust code by hash. |
+| proposals | `plugins/proposals.ts` | Validate and store inert proposals; preview, reject or accept a hash-bound review. |
+| inventory / api | `plugins/inventory.ts`, `plugins/api.ts` | `plugins.list`, `plugins.inventory`, attach, trust. |
+| capability api | `capabilities/api.ts`, `capabilities/types.ts` | `list`, `resolve`, `toolIntersection`, `inventory`, `attach`. |
+| run snapshot | `capabilities/runSnapshot.ts` | `freezeRun`, `loadRun`, `capabilitySet`. |
+
+## Contracts
+
+- **Loading.** `Plugins.loading({ cwd, agentDir, caller, contributions, install? })` returns
+  `extensionPaths`, `skillPaths`, `promptPaths`, `themePaths`, `profilePaths`, `builtins` (legacy shape; no Pi CLI built-ins selected), `hooks`,
+  `found`, `detached`, `projectCode`, `packageCode`. `profilePaths` are read by the harness for
+  subagent profiles (page 02); Pi is never given them. A session opens Pi with its own discovery off
+  and only these paths; a terminal receives them as `--extension`, `--skill`,
+  `--prompt-template` and `--theme` after `--no-extensions --no-skills --no-prompt-templates
+  --no-themes`. Pi still reads `AGENTS.md` and `SYSTEM.md` itself.
+- **Surface metadata (PROP-0040).** `PluginDescriptor.surfaces` and `Plugin.surfaces` optionally
+  carry `{ preferred: 'host', terminal: 'fallback' | 'unavailable', note: string }`.
+  `plugins/surfaces.ts` formats the note and badge. Inspection explains what the host adds and
+  what the terminal does instead; this is not a category, attachment switch or permission.
+- **Shared resources (PROP-0040).** Both surfaces load Harness-selected resources and enabled
+  first-party hooks, including output trim and budget guard. Terminal preparation exposes these
+  through its protected bridge. Pi CLI-only llama.cpp, codemode, tool-search and MCP built-ins are
+  not selected or advertised as extra terminal capabilities. This does not remove Harness MCP.
+- **Attachments.** `config/plugin-attachments.json` holds `global`, `agents`, `projects` (each
+  `{ on, off }`) and `settings` per plugin. The narrowest scope that says something wins: agent,
+  then project, then everywhere. With no entry an item keeps its default.
+- **Package metadata.** A Pi package may declare `"harness": { "extends": [...], "hooks": "./hooks.mjs", "commandPrefix": "my-tools", "defaultPrompt": "review", "agents": "./agents" }`.
+  `defaultPrompt` names one of that package's prompt templates as the target of `/<owner-prefix>:`;
+  it is not a path or an arbitrary command. Without a valid, unambiguous owned target, it must not
+  route to another owner's template or guess a default (ADR-0061).
+  A package without a `package.json` name is named, described and versioned by the first of
+  `plugin.json`, `.claude-plugin/plugin.json`, `.codex-plugin/plugin.json`, so an agent plugin
+  written for other harnesses installs as it is. The first of those manifests that declares
+  `skills` (a path or a list of paths) also brings those skill folders as package skill
+  contributions, so a plugin laid out for another harness contributes the skills it declares. A
+  declared path must stay inside the package, and a symlinked skill folder is skipped. Pi's own
+  `skills/` folder and `package.json` manifest take precedence, and a package whose settings entry
+  names skill patterns is left to that filter.
+- **Skill installation routes.** A skill reaches a run as text from the working folder's `.pi/skills`
+  or `.agents/skills` (walking up to the repository root), the OS user's `~/.agents/skills`, the
+  state's `accounts/_shared/pi/skills`, a package's `skills` or a foreign manifest's declared paths,
+  or the pinned variant's `skills`. A model may write the first two; `~/.agents` is outside the
+  workspace, so a write there needs the person's approval. Variant, library, Hub and package
+  installs change committed state and are person acts (`/skills`, `/components`, `/hub`,
+  `/plugins`). A variant holds `SKILL.md` and flat `scripts/` files only. `plugins.install` takes a
+  package address: a GitHub or GitLab `tree`/`blob` page address is refused with the repository
+  address and these routes. The install guidance lives in the release-owned `plugin-authoring`
+  skill, not the always-on prompt ([STUDY-0005](../study/STUDY-0005-skill-installation-guidance.md)).
+- **Package subagent profiles.** An installed package's agents folder (`agents/` unless
+  `harness.agents` names another) brings each Markdown file whose frontmatter has a `name` and a
+  `description` as an `agent` contribution (at most 50). It is text: it needs no trust and follows
+  its switch like a skill. The profile is named by its frontmatter, not its file. Fields only other
+  harnesses understand are dropped (`agents/profiles.ts`, `parsePackageProfile`): their tool names
+  are translated (`Read`→`read`, `Glob`→`find`, …), a model without a provider is left to the
+  parent, `plugins` and `variant` are never taken, and a tool list of which nothing translates
+  keeps the child read-only. A package profile never replaces a built-in, person's, project's or
+  variant's profile of the same name; the listing reports it as a problem instead.
+  `commandPrefix` is optional; in the terminal, commands otherwise use a normalized package name as
+  their owner prefix. Prefixes use lowercase letters, digits and hyphens, starting with a letter;
+  `plugin` and `skill` are reserved. Package-owned prompt templates share the package's command
+  namespace with its extension commands; prompts discovered by a loose extension use that
+  extension's namespace. Neither is exposed as a bare plugin alias. Standalone prompt templates
+  retain their existing form. See page 11 for routing and collision rules.
+- **Plugin authoring.** `plugin_propose` creates a pending proposal, not an installation. The
+  release-owned `plugin-authoring` skill is available on demand in structured and terminal sessions;
+  a pinned variant's skill of the same name wins. It adds no trust or permission
+  ([ADR-0045](../decisions/ADR-0045-default-plugin-authoring-skill.md)).
+- **Proposal review.** `plugins/proposals.ts` validates a lowercase package name, nonempty description,
+  and 1–100 UTF-8 text files (256 KiB each, 1 MiB total). It accepts `README.md`, extensions
+  (`.ts`, `.js`, `.mjs`), Markdown skills, prompts and agents, and JSON themes in their respective
+  directories; paths must be safe and cannot contain `.git` or `node_modules`. A
+  `skills/<name>/SKILL.md` needs matching name and nonempty description frontmatter. The Harness
+  generates `package.json`; it is not supplied by the proposal. Preview revalidates the stored data
+  and computes SHA-256 over name, description, generated manifest and sorted file entries. Accept
+  and reject require the current preview hash and pending status; stale reviews fail.
+  Acceptance rechecks after adoption and listing, refuses an existing package name or already-trusted
+  identical package code, stages and publishes under `<data root>/plugins/<name>`, clears old
+  attachment decisions for that package and explicitly switches it off globally. It grants no code
+  trust or permission; reject retains the proposal as rejected. These are trusted caller review
+  operations, not model tools. See [ADR-0046](../decisions/ADR-0046-hash-bound-plugin-proposals.md).
+- **Hooks.** A package declares `"harness": { "extends": [...], "hooks": "./hooks.mjs" }`. Events:
+  `wiki.page.saved`, `history.run.ended`. A hook has 30 seconds; its errors are kept in
+  `plugin-hooks.json`. `ctx.call(connection, tool, args)` reaches only a connected app that is on
+  for everyone, and only a tool marked to run without asking.
+- **Capability set** (`CapabilitySet`): `{ version: 1, tools, connectors, contributions }`.
+  `capabilities.resolve` returns the set available to a caller; `capabilities.intersect` /
+  `toolIntersection` narrows it for a child. The required `ask-user` plugin remains attached, but its
+  `ask_user` tool is included only for an attended plan run or a continuation pinned with that
+  capability. Automatic or refinement-triggered continuations with it are rejected rather than
+  weakening the frozen set. The `side-questions` descriptor owns the fixed instruction for the
+  restricted side-question run; it has no tools, providers, connectors, subagents or variant code
+  (PROP-0019). In plan mode, delegation is limited to research children: the resolved profile must
+  have explicit tool and plugin allowlists confined to the research set, and the child's frozen
+  capabilities are intersected with the parent and profile. A profile name or its instructions alone
+  never grants eligibility (INV-RUN-8).
+
+## Invariants
+
+- **INV-PLG-1** The harness is the only loader. A session or terminal loads exactly what
+  `Plugins.loading` chose, with Pi's discovery switched off. Never pass a session a loader that
+  discovers on its own.
+- **INV-PLG-2** Code waits for trust. A working folder's `.pi/extensions` and the packages its
+  settings name are trusted together; each installed package is trusted on its own. Trust is a
+  hash of the files; a changed file asks again. A loose file a person put in the shared
+  `extensions/` folder needs no trust. Text contributions always load.
+- **INV-PLG-3** Installing runs no code (`npm_config_ignore_scripts`). A package arrives switched
+  off and, if it has code, untrusted. Installing grants no attachment, trust, credential or
+  permission.
+- **INV-PLG-4** The approval gate is the last extension to see a tool call (page 07, `INV-GATE-2`).
+  A plugin that changes a call's input does so before the gate sees it.
+- **INV-PLG-5** Everything model-visible belongs to a plugin. A new prompt section, tool group or
+  tool is declared by a `PluginDescriptor` (`plugins/builtin.ts` for the harness's own), or
+  `plugins.inventory` reports it as `unattributed`.
+- **INV-PLG-6** One store says what is switched on: `plugins/attachments.ts`. A new kind of
+  switchable thing gets a default and is read with `attached`; no second enablement file.
+- **INV-PLG-7** A hook is told, never asked. Hooks run after the write, in the background; a write
+  never waits for a hook and a hook's failure never reaches it. A new write point that should be
+  heard calls `noteChange` (`core/changes.ts`). No hook may stop or change a write.
+- **INV-PLG-8** A hook runs only while its plugin is on for everyone and trusted, checked before
+  each delivery. A working folder's package cannot bring a hook.
+- **INV-PLG-9** Unattended runs load no extension found on disk (`INV-RUN-5`). The harness's own
+  parts are required and refuse to be switched off; switching something on grants no permission.
+  A side-question run also has no variant plugin code or tool guidance (PROP-0019).
+- **INV-PLG-10** Every Harness-managed built-in feature is registered by `BuiltinPluginRegistry`.
+  Its descriptor, world tools, server handlers and any plugin-owned terminal commands share one
+  owner. First-class Harness terminal commands are registered by the terminal command registry and
+  do not depend on a feature plugin's command metadata or attachment state. A module's world part
+  releases resources through the existing session and Harness lifecycle. This registry does not
+  load third-party Pi code or bypass its trust checks.
+- **INV-PLG-11** Built-in plugin categories are a documentation ownership taxonomy. They are not a
+  `PluginDescriptor` field, a public response field or a second attachment scope.
+- **INV-PLG-12** Plugin command ownership includes plugin-owned prompt templates. An owner prefix
+  identifies exactly one owner; prefix collisions between owners and duplicate routes within an
+  owner fail closed with owner diagnostics, including collisions between a template and an extension
+  command. Package `harness.defaultPrompt` can select only its own unambiguous template for the
+  namespace root. Ownership does not change attachment, code trust or permission rules.
+- **INV-CAP-1** A capability set only narrows. `intersect` never adds access; a run's set is
+  frozen when it starts (`INV-RUN-2`). An agent profile's `plugins` list keeps only the parent's
+  contributions of those plugins (required plugins always stay) and drops the tools they own; it
+  never adds one (PROP-0029).
+- **INV-CAP-2** The capability contract is generic. It names no application task, role or UI
+  concept.
+- **INV-CAP-3** A capability set says what may be called, not whether a call may run. The gate
+  still decides each call.
+
+## Data
+
+```text
+<data root>/config/plugin-attachments.json     (registry-enabled.json read until the first change)
+<data root>/plugins/<name>/                     packages kept with the data; removal moves to trash/plugins/
+<data root>/plugin-proposals/<id>.json          proposed text, hash and review status
+<state dir>/plugin-trust.json                   trusted code, by hash
+<state dir>/plugin-registered.json              what each extension registered last time
+<state dir>/plugin-hooks.json                   hooks and their last run
+<state dir>/accounts/_shared/pi/                settings and packages shared by every account
+```
+
+## Source map
+
+`src/plugins/*.ts`, `src/capabilities/*.ts`, `src/core/changes.ts`.
+
+## Built-in ownership taxonomy (PROP-0020)
+
+These categories make ownership visible in docs and source layout. They do not change plugin
+metadata or availability:
+
+| Category | Features | Foundation boundary |
+|---|---|---|
+| Foundation | Accounts, sessions and run recording, prompt and variant validation, gate, plugin loading/trust/attachments, resource browser, server and terminal frameworks | Release-owned infrastructure and security contracts |
+| World | Web search/read tools, browser tools, page annotations | The isolated browser engine and network policy remain Foundation services |
+| Memory | Wiki, history and trajectory | Storage and run records remain Foundation-owned services |
+| Automation | Subagents, ask-user, todo, lessons and refinement | Run lifecycle, capability narrowing and approval gate remain Foundation-owned |
+| Integrations | Connected apps/MCP and Hub | Connector credentials and plugin installation remain behind Foundation services |
+| Operations | Doctor, worktrees and usage | Diagnostics, repository operations and usage analytics remain Harness-managed services |
+
+## Terminal plugin and resource inspection
+
+The broad `/harness` browser joins saved plugin/contribution metadata with actual session skills and
+tool registration. A saved on/off state does not prove a resource loaded; origins and owners
+distinguish same-name resources. `/plugins` is a focused local manager with Plugins, Trust and
+Packages tabs. It uses the existing plugin services for switching, trust, settings, package install,
+update and removal. `/hub` is a distinct Community catalog surface; it does not load code or add a
+second loader. Hub-owned package removal through `plugins.remove` delegates to Hub so the exact
+receipt and its resources stay consistent. No second discovery or enablement mechanism is
+introduced (PROP-0013, PROP-0047).