@kontextmind/kxm 0.7.94 → 0.7.96

package/docs/concepts/architecture.md

@@ -0,0 +1,262 @@
+# Architecture
+
+KXM connects coding agents in different harnesses through a durable [hub](../glossary.md#hub) and runs reviewed workflows on your machine through the [Runtime](../glossary.md#runtime). This page explains what each component does, how the pieces fit together, and what the system does not guarantee. Read it before you deploy KXM for a team or build on its interfaces.
+
+## The system at a glance
+
+The diagram shows the three client surfaces, the two local services, and what crosses the loopback boundary.
+
+```mermaid
+flowchart LR
+  subgraph Clients["Agents and operator"]
+    CC["Claude Code<br/>plugin: MCP stdio, channel, SessionStart hook"]
+    PI["Pi<br/>extension and skills"]
+    CLI["kxm CLI<br/>operator"]
+  end
+  subgraph Machine["Your machine: loopback by default"]
+    HUB[("KXM hub<br/>HTTP and SSE, SQLite kxm.db")]
+    RT["Runtime supervisor<br/>runs, event store, outbox"]
+  end
+  GIT[["Git repository<br/>.kxm/*.yaml"]]
+  EXT["Webhooks and CI<br/>signed starts and callbacks"]
+  CC -->|"MCP tools, SSE"| HUB
+  PI -->|"HTTP, SSE"| HUB
+  CLI -->|"admin and project APIs"| HUB
+  EXT -->|"HMAC-signed"| HUB
+  CLI -->|"kxm run, kxm runs"| RT
+  RT -->|"sync events, outbound only"| HUB
+  GIT -.->|"reviewed config"| CLI
+  GIT -.->|"pinned per run"| RT
+```
+
+KXM has two planes:
+
+- The **hub coordinates.** It authenticates agents, stores their messages, runs signed webhook workflows, keeps the journal and context, grants leases, and receives Runtime sync events. It never runs a model.
+- The **Runtime executes.** It owns event-sourced runs of `kxm.workflow.v1` workflows on one machine, drives their steps through harnesses, and works with or without a hub.
+
+## What KXM is and is not
+
+KXM is:
+
+- a durable request/reply layer between agents in Pi, Claude Code, and other harnesses, with discovery by name and declared purpose;
+- a workflow engine with typed steps, gates, budgets, and receipts. The Runtime schedules each run's steps and queues runs behind a per-project limit (`limits.maxConcurrentRuns`, default 1);
+- a provenance and learning layer: hub-verified peer evidence, a structured journal, role-aware context, and governed memory and skills.
+
+KXM is not a sandbox, a distributed system, an exactly-once executor, a planner that splits tasks or picks peers for you, or a shared model context. Agents exchange bounded messages, never conversation histories. [Limits and trade-offs](#limits-and-trade-offs) covers the boundaries that matter in production.
+
+## The hub
+
+The hub is one Node.js process that serves HTTP and server-sent events (SSE) and writes everything to one SQLite database, `.kxm/state/kxm.db` by default. It binds `127.0.0.1:7331` unless you set `KXM_HOST` and `KXM_PORT`, and it refuses to bind beyond loopback without an admin token. SQLite is the source of truth after a restart; in-memory maps are only the live working set.
+
+| Subsystem | What it does | Read more |
+|---|---|---|
+| Projects | The authentication and discovery namespace; agents see only peers in their own project | [Trust model](trust-model.md#project-isolation) |
+| Agents and presence | Registration returns a rotating agent key; presence (`online`, `stale`, `offline`) comes from the hub's clock | [Peer messaging](../guides/peer-messaging.md) |
+| Messages | Durable request/reply with TTL, cancellation, idempotency keys, and fanout to one to three peers | [Message lifecycle](#message-lifecycle) |
+| Webhook workflows | Signed webhooks start runs with ordered stages, evidence checkpoints, waits, and signed callbacks | [Webhook workflows](../guides/webhook-workflows.md) |
+| Journal | Ten categories of run knowledge, from `plan` to `skill-candidate`; a retrospective for every finished run | [Continuous improvement](../guides/continuous-improvement.md) |
+| Context OS | Context items, temporal state, recall, episodes, a compiled wiki, and role-aware packets | [Context and memory](../guides/context-and-memory.md) |
+| Leases | Fenced, project-scoped leases timed by the hub's clock | [HTTP API](../reference/http-api.md) |
+| Sync ingest | Accepts Runtime presence and sync events, each `{project, run, sequence}` exactly once | [Runtime sync](../operations/runtime-sync.md) |
+| Operations stream | Admin-only, metadata-only snapshot and SSE for `kxm dash`, plus Prometheus `/metrics` | [Monitoring](../operations/monitoring.md) |
+
+A lease holder receives a fencing token that increments only when another holder takes over an expired lease, so a stale holder can be refused before it commits a shared change. The lease routes are available to any agent in the project; the Runtime does not take leases on its own yet.
+
+### Message lifecycle
+
+A message is a durable request with one recipient and at most one reply; the diagram shows every state the hub sets.
+
+```mermaid
+stateDiagram-v2
+  state "error (reserved)" as err
+  [*] --> queued: sender posts
+  queued --> delivered: recipient acknowledges
+  queued --> replied: recipient replies
+  delivered --> replied: recipient replies
+  queued --> cancelled: sender cancels
+  delivered --> cancelled: sender cancels
+  queued --> expired: TTL elapses
+  delivered --> expired: TTL elapses
+  replied --> [*]
+  cancelled --> [*]
+  expired --> [*]
+  note right of queued
+    Stored in SQLite. Pushed again on every
+    reconnect until the recipient acknowledges it.
+  end note
+  classDef reserved stroke-dasharray:5 5
+  class err reserved
+```
+
+- The sender gets the message ID at once. The recipient may reply straight from `queued` without acknowledging first.
+- `error` is declared in the protocol, but the hub never sets it. Treat it as reserved.
+- Open messages survive a hub restart. When an agent reconnects under the same project and name, the hub rotates its key and pushes every `queued` message again with its original ID. A `delivered` message is not pushed again; it stays open until a reply, cancellation, or expiry. Clients suppress a second turn for an ID they already handle.
+- An idempotency key deduplicates an exact retry by the same sender. It does not stop the recipient from repeating a side effect, so handlers must be safe to repeat.
+- The default TTL is 24 hours (at most 7 days). Terminal messages are purged after the retention window, 7 days by default.
+- If a workflow coordinator's prompt expires before its run finishes, the run fails.
+
+### Hub workflow run lifecycle
+
+A signed webhook creates a run and its coordinator message in one request; the diagram shows how the run moves between its four states.
+
+```mermaid
+stateDiagram-v2
+  [*] --> running: signed webhook
+  running --> running: warning or failed retries, passed opens next stage
+  running --> waiting: coordinator waits, or audit escalation
+  waiting --> running: signed signal, more work needed
+  running --> completed: last stage passes
+  waiting --> completed: signed signal passes last stage
+  running --> failed: maxAttempts or budget used up, early reply, prompt expired
+  waiting --> failed: deadline passes, or signal exhausts attempts
+  completed --> [*]
+  failed --> [*]
+```
+
+- Stages run in order. A `warning` or `failed` checkpoint retries the same stage until its `maxAttempts` is used up. Declared `on` outcomes add typed transitions, including back-edges, each bounded by per-edge, per-stage, and global `maxTransitions` budgets.
+- A coordinator reply while the run is `running` fails the run, because required work was abandoned.
+- A wait releases the coordinator: its reply while `waiting` is expected. A signed callback resumes the stage and sends a fresh message when more work remains. The deadline is 1 second to 30 days (default 24 hours); when it passes, the run fails and the coordinator is told.
+- After `autoResumeLimit` failed attempts, a stage escalates to an `audit_escalation` wait for an operator ruling.
+- Every run records a secret-free `definitionHash`, and every finished run exports a retrospective.
+
+Peer-reply evidence, quorum, and degradation have their own guide: [Provenance gates](../guides/provenance-gates.md).
+
+## The Runtime
+
+The Runtime is one supervisor process per OS user and machine, and it can host many projects. `kxm run` and `kxm runs` start it on demand; `kxm runtime start` starts it explicitly. It listens on an ephemeral `127.0.0.1` port, and every call except its health check needs the bearer token from a `0600` file in the user state root.
+
+A run moves through five phases:
+
+1. `kxm run <workflow> [prompt]` validates the project, records the run as `created`, and pins the configuration, memory, executor-policy, and tool-policy revisions. No hub is involved.
+2. `kxm runs drive <runId>` pins the compiled run plan (`preparing`) and then executes steps in order (`running`). With `--simulated`, a model-free producer answers the agent steps while gate steps still run their commands; without it, the drive is live.
+3. Every change is an event, appended to the project's event store in one transaction with an outbox row. State is always a replay of those events, and a stored projection that disagrees with the log is refused.
+4. When a drive stops, it writes a drive receipt: terminal, handoff, or unsettled. `kxm runs status` and `kxm runs receipt` show it and whether it verifies.
+5. The supervisor pushes outbox rows to the bound hub. Without a hub, rows wait locally until one is bound.
+
+Steps are `agent`, `moa`, `approval`, `wait`, or `gate`. The first four go to a producer through an assignment and an attempt; each attempt gets a capability secret that is stored only as a hash. `approval` and `wait` steps go to the first agent in the step's `assignments.allowedAgents`, else to the agent whose ID is literally `coordinator`, not to the workflow's `coordinator:` value. Gate steps run entries from `.kxm/gates.yaml` as an argument list without a shell, and keep only hashes and sizes of their output.
+
+A live drive runs each agent's harness as a one-shot, read-only process. When a step needs something this build does not execute, such as write access in a live drive, the drive stops with a handoff (`run_handoff_required`) instead of guessing. The [configuration reference](../reference/config-reference.md#steps-the-runtime-does-not-execute-yet) lists every such case.
+
+A dispatched agent also receives the project's committed memory and hash-verified promoted skills, but only when those files are committed, clean, and match the run's pinned memory revision. Otherwise dispatch continues without them and records a gap. Dispatch reads nothing from the hub.
+
+### Runtime run lifecycle
+
+The diagram shows the run states the event fold accepts in this build.
+
+```mermaid
+stateDiagram-v2
+  [*] --> created: kxm run
+  created --> preparing: first drive pins the plan
+  preparing --> running: steps start
+  running --> completed: terminal transition
+  running --> failed: failure outcome or budget exhausted
+  running --> cancelled: workflow ends as cancelled
+  running --> cancelling: cancel or run-duration budget
+  cancelling --> cancelled: attempts drained
+  running --> blocked_uncertain: gate outcome cannot be proven
+  blocked_uncertain --> running: retry or unblock
+  blocked_uncertain --> failed: fail
+  blocked_uncertain --> cancelling: cancel
+  completed --> [*]
+  failed --> [*]
+  cancelled --> [*]
+```
+
+A run can also be cancelled or fail before it reaches `running`, and the fold permits `cancelling → failed`. There is no `waiting` state: the contracts specify one, but the fold refuses it, so a Runtime run never parks the way a hub workflow run does. You resolve `blocked_uncertain` with `kxm gate signal <runId> … --recovery-action retry|unblock|fail|cancel`.
+
+Inside a run, each object has its own linear lifecycle:
+
+| Object | States in this build |
+|---|---|
+| Step | `pending` → `preparing` → `running` → `passed`, `failed`, or `cancelled` |
+| Assignment | `created` → `accepted` → `dispatched` → `executing` → `result_recorded` → `terminal` |
+| Attempt | `created` → `starting` → `executing` → `settling` → `terminal`, with result class `outcome`, `outcome_unknown`, `producer_rejected`, or `cancelled` |
+
+> [!NOTE]
+> [Durable lifecycles](../contracts/lifecycles.md) specifies the target lifecycle, including `waiting`, `skipped`, `retry_pending`, `reattaching`, and `connection_lost`. This build does not reach those states yet.
+
+## The clients
+
+### The kxm CLI
+
+The CLI is the operator's tool. It starts and binds the hub (`kxm hub`), creates and drives runs (`kxm run`, `kxm runs`), manages the supervisor (`kxm runtime`), sets up and reviews projects (`kxm init`, `kxm trust`), sends peer requests (`kxm peer`), operates workflows and gates (`kxm workflow`, `kxm gate`), and backs up and restores SQLite stores (`kxm backup`, `kxm restore`). The [CLI reference](../reference/cli-reference.md) covers every command.
+
+### The Pi extension
+
+The extension loads into Pi from the KXM package. It registers the 19 KXM tools, the `/kxm` and `/workflow` commands, and status widgets, and it turns inbound requests into Pi turns. It can start a hub in the background (`hub.autoStart: background`). `kxm agent worker` runs Pi as a long-lived supervised worker, optionally with one Pi session per workflow run. See [Pi workers](../guides/pi-workers.md).
+
+### The Claude Code plugin
+
+The plugin has four parts:
+
+- an **MCP stdio server** that exposes the same 19 tools;
+- an optional **channel** that pushes peer requests into the session, with `kxm_inbox` and `kxm_reply` as the pull alternative;
+- a **SessionStart hook**, a bundled read-only Node script that prints a short KXM status and the project memory brief, and needs no `kxm` on `PATH`;
+- the bundled **skills**.
+
+The plugin never falls back to the admin token. See the [plugin README](../../plugins/kxm/README.md) and [Quick start with Claude Code](../start/quickstart-claude-code.md).
+
+### Agent Skills
+
+The package ships a directory of `SKILL.md` suites that Pi and Claude Code both load: the `kxm-*` skills that teach each command group, browser-automation skills, and skills for the separate KontextMind knowledge plane. See [Agent Skills](../guides/agent-skills.md).
+
+### Dash and Studio
+
+`kxm dash` draws live terminal screens (agents, tasks, workflows, plans, inbox, processes, and spend, which is always empty today) from the admin-only operations stream, a presence-only fallback, and a read-only snapshot of `kxm.db`. Treat it as an observer: its action keys `a`, `r`, `s` and `c` post to hub routes that do not exist, so they change nothing even though the status line reports success, and `d` creates a git branch and worktree.
+
+`kxm studio layout` renders a workflow as DAG, stepper, and swimlane JSON, and `kxm studio serve` hosts a local viewer on `127.0.0.1:4242`. The Studio mutation endpoint is not wired to commands: it acknowledges requests without running them.
+
+## Configuration is reviewed Git YAML
+
+Project behavior lives in Git under `.kxm/`: `project.yaml`, `agents/`, `models/`, `workflows/`, `gates.yaml`, `roles/`, `routes.yaml`, `memory/`, and `skills/`. The project bundle (`project.yaml`, `agents/`, `models/`, `workflows/`, and `gates.yaml`) is restricted YAML checked against a JSON Schema. `routes.yaml`, `roles/`, and the front matter in `memory/` and `skills/` are parsed as ordinary YAML with their own checks, and a live drive re-reads an agent file the same way to resolve its route. Git review is the activation boundary:
+
+- Every run pins revision hashes of the project bundle, memory, executor policy, and tool policy. An edit affects only later runs, and a run whose pinned revisions drift is refused.
+- `kxm trust diff` prints a structured permission diff against a base revision (default `HEAD`). `kxm trust check` exits non-zero when the change expands permissions. Neither covers `routes.yaml`, `roles/`, `roster.yaml`, or `prices.yaml`, so admitting a route is not flagged.
+- Legacy `.kxm/config/*.json` files are refused, never converted.
+
+Webhook workflow definitions are separate JSON that the hub loads at start, with secrets named by environment variable. Personal settings merge built-in defaults, then `~/.config/kxm/config.yaml`, then the project's `.kxm/config.yaml`. See the [configuration file reference](../reference/config-reference.md) and [Configuration](../reference/configuration.md).
+
+## Harness routing
+
+A route is a harness plus a model. In a live drive, the Runtime resolves each agent's harness (its `harness`, else the project's `defaultHarness`) and model, refuses any `provider/model` that `.kxm/routes.yaml` does not admit, and checks the role roster when one exists. It then probes the harness's login and starts a one-shot process. A refusal never falls back to another route.
+
+The rule is to run a vendor's model in that vendor's native harness, never through Pi. The Pi native-vendor brake enforces it in two places: the probe each producer runs before it starts a harness, and `kxm agent worker`, which checks the primary model and every fallback before Pi starts. It refuses a native vendor's model whether the id names the vendor directly, through that vendor's own Pi provider, or behind an aggregator.
+
+Admission is the second layer. Some cases are still operator policy, among them reseller ids that name no vendor and whether Google work runs through `agy` or the `antigravity` Pi provider; see [where the code is looser than the rules](../reference/harness-routing.md#where-the-code-is-looser-than-the-rules). [Harness routing](../reference/harness-routing.md) explains how to choose a route and confirm which one ran.
+
+## Packaging
+
+One npm package, `@kontextmind/kxm`, carries every surface. It needs Node.js 22.19 or later on the 22 line, or 24 and later. It has no native dependencies: SQLite comes from Node's built-in `node:sqlite`, or from `bun:sqlite` inside Pi. The plugin does not install the CLI, so install both when you want Claude Code and the operator commands on one machine.
+
+| Surface | What ships | Install guide |
+|---|---|---|
+| npm package | The `kxm` CLI, prebuilt hub, Runtime supervisor, MCP server and hook bundles, wrapper scripts, schemas, docs, and examples | [Install](../start/install.md) |
+| Pi package | The same package: its `pi` manifest loads the extension and the skills directory | [Quick start with Pi](../start/quickstart-pi.md) |
+| Claude Code plugin | `plugins/kxm` from the `kxm` marketplace: MCP server, channel, SessionStart hook, and skills | [Quick start with Claude Code](../start/quickstart-claude-code.md) |
+
+## Why KXM is built this way
+
+- **Execution stays local.** A hub outage never stops a run, and repositories and provider credentials never collect in one service. The original decision is [ADR-001](../contracts/architecture.md).
+- **SQLite for every store.** The workload is many small writes owned by one process. See [ADR-0003](../adr/ADR-0003-sqlite-only-store.md).
+- **Git review activates behavior.** Runs pin revisions, and learned content cannot activate itself; memory and skills reach agents only through reviewed files and governed promotion.
+- **Browser identity stays at the edge.** A hosted hub sits behind a proxy and a portal and never interprets browser identity. See [ADR-0004](../adr/ADR-0004-edge-identity-authentik.md).
+- **Stores refuse old schemas.** An older or newer store is refused, never migrated in place. See [Data and storage](data-and-storage.md#schema-versions-refuse-do-not-migrate).
+
+## Limits and trade-offs
+
+- **Single node.** One hub process owns one SQLite file, and one Runtime owns each machine's runs. Do not load-balance hubs.
+- **At-least-once.** A crash after a side effect but before the reply or settlement can repeat work. Use idempotency keys and delivery IDs, and design every handler to be safe to repeat.
+- **Not a sandbox.** Tool allowlists, roles, session isolation, and project tokens prevent mistakes, not a hostile process under the same OS user. See the [trust model](trust-model.md#kxm-is-not-a-sandbox).
+- **Quorum proves routing, not truth.** A verified peer reply proves which registered identity answered in which run, stage, and attempt. It does not prove the answer is correct, independent, or approved by a person.
+- **The Runtime does not run everything yet.** Live drives are read-only, runs have no `waiting` state, and unsupported steps hand off. A drive receipt's `logHash` covers event IDs and sequence numbers, not payloads, and receipts are not signed.
+- **Retention is short on the hub.** Finished hub runs and their journal are purged after 7 days, which also ends webhook deduplication for them. Runtime event stores keep everything.
+- **Some reads are local.** `kxm workflow list` and `get`, `kxm session brief`, the SessionStart hook, and part of `kxm dash` read `kxm.db` directly, so they see hub runs only on the hub's machine. The session brief and the hook also read this machine's Runtime runs, and report what they found as `source`: `legacy`, `runtime`, or `both`.
+
+## Related
+
+- [Trust model](trust-model.md)
+- [Data and storage](data-and-storage.md)
+- [Architecture decision records](../adr/README.md)
+- [Durable lifecycles](../contracts/lifecycles.md)
+- [Deploy KXM](../operations/deploy.md)
+- [Glossary](../glossary.md)

package/docs/concepts/data-and-storage.md

@@ -0,0 +1,194 @@
+# Data and storage
+
+KXM keeps its state in a few SQLite databases and a handful of files, split between your project checkout and a per-user state root. This page explains what each store holds, where it lives, what is kept as a hash instead of as content, how long data stays, and why KXM refuses old stores instead of migrating them. Read it before you back up, wipe, or audit a KXM installation.
+
+## Where data lives
+
+The diagram shows which process owns each store and which root it lives under.
+
+```mermaid
+flowchart LR
+  subgraph Project["Project checkout"]
+    CFG[".kxm/*.yaml<br/>reviewed config in Git"]
+    HDB[(".kxm/state/kxm.db<br/>hub store")]
+    LOGS[".kxm/logs<br/>hub and worker logs"]
+    RETRO[".kxm/assets/retrospectives"]
+  end
+  subgraph StateRoot["User state root"]
+    ENV["hub-env.json<br/>hub credentials"]
+    REG[("runtime/registry.db")]
+    EVT[("run-events.db per project<br/>+ run-prompts.json")]
+  end
+  HUB["KXM hub"] -->|"writes"| HDB
+  HUB -->|"writes"| LOGS
+  HUB -->|"exports"| RETRO
+  ENV -.->|"credentials"| HUB
+  RT["Runtime supervisor"] -->|"writes"| REG
+  RT -->|"writes"| EVT
+  CFG -.->|"read at run start"| RT
+  RT -->|"sync events"| HUB
+```
+
+KXM uses three locations:
+
+- **The project workspace**, `.kxm/` at the project root. Reviewed configuration is tracked in Git. `state/`, `logs/`, and `assets/` hold what the hub and workers write. `KXM_WORKSPACE_DIR`, `KXM_STATE_DIR`, `KXM_LOGS_DIR`, `KXM_ASSETS_DIR`, and `KXM_DATA_PATH` move those parts; configuration always stays under `<project root>/.kxm/`.
+- **The user state root**, for machine-level state that must never be committed.
+- **The user configuration directory**, `KXM_USER_CONFIG_DIR` (default `~/.config/kxm`), for personal `config.yaml`, global roles and workflows, and the local session token.
+
+The user state root is `KXM_STATE_HOME` when set; it must be an absolute path. Otherwise it depends on the operating system:
+
+| Operating system | User state root |
+|---|---|
+| macOS | `~/Library/Application Support/KXM` |
+| Linux | `$XDG_STATE_HOME/kxm`, default `~/.local/state/kxm` |
+| Windows | `%LOCALAPPDATA%\KXM` |
+
+The hub writes its database under the workspace of the directory it starts in. Start it from the project root so that `kxm.db` lands in that project's `.kxm/state/`.
+
+## The hub store
+
+The hub store is `.kxm/state/kxm.db` (or `KXM_DATA_PATH`). Like every KXM store, it records its schema version and refuses any other; see [Schema versions: refuse, do not migrate](#schema-versions-refuse-do-not-migrate). It has ten `STRICT` tables:
+
+| Table | What it holds | Sensitive content |
+|---|---|---|
+| `agents` | Agent ID, name, purpose, project, model, host label, timestamps, and the current agent key | Agent keys in plain text |
+| `messages` | Each request and reply with routing metadata and any workflow context | Request and reply bodies, as sent |
+| `consumer_cursors`, `agent_sequences` | Each agent's delivery position and next message sequence | None |
+| `workflow_runs` | Hub workflow runs: stages, evidence, waits, signal receipts, transitions, verified peer snapshots, hashes | Evidence strings |
+| `workflow_journal` | Journal entries with category, area, stage, and attempt | Summaries, details, and evidence |
+| `context_items` | Context items and state proposals with provenance and validity | Summaries, pattern-redacted |
+| `leases` | Fenced leases with holder and fencing token | None |
+| `sync_events` | Runtime events the hub accepted, once per project, run, and sequence | Sync-safe summaries |
+| `runtime_presence` | Runtime heartbeats per project | Host label |
+
+A coordinator message holds the rendered workflow prompt, which can include fields from the webhook payload. The raw webhook body is not stored; the run keeps only its SHA-256.
+
+Some readers open `kxm.db` directly and read-only: `kxm workflow list` and `get`, `kxm session brief`, the Claude Code SessionStart hook, and parts of `kxm dash`. They see hub runs only on the hub's machine. The session brief and the hook also open this machine's Runtime stores read-only, and report what they found as `source`: `runtime` or `both` when a Runtime store exists, otherwise `legacy`.
+
+## The Runtime stores
+
+The Runtime keeps a registry for the machine and one event store for each project, all under `runtime/` in the user state root.
+
+The **registry** is `runtime/registry.db`. Its `supervisor` table holds one row: the Runtime ID, process ID, port, a hash of the supervisor token, heartbeat, and state. Its `projects` table records each project's ID, canonical root path, project key, and home Runtime.
+
+Each **event store** is `runtime/projects/<key>/run-events.db`. The key is the first 24 hex characters of the SHA-256 of the canonical project root. The store has 14 `STRICT` tables:
+
+| Tables | What they hold |
+|---|---|
+| `runs`, `events`, `commands` | Each run with pinned revisions and a prompt hash; the append-only event log; idempotent command results |
+| `run_plans`, `run_state` | The pinned compiled plan; the materialized projection |
+| `attempt_capabilities` | Attempt bindings and the hash of each capability secret |
+| `gate_attempts`, `gate_observations`, `gate_evidence` | Gate identity, process outcome with output hashes and sizes, and settled evidence |
+| `drive_receipts` | The final receipt of each drive |
+| `coordinators`, `intake_messages`, `project_controls` | Intake bindings, idempotent intake, and the project pause switch |
+| `outbox` | One sync row per event, with its acknowledgement or refusal |
+
+Gate rows are checked again every time the Runtime folds a run: their content hashes must recompute and match the events that name them, or the run is refused with a `gate_evidence_*` code such as `gate_evidence_hash_mismatch` or `gate_evidence_orphan`. A gate command that never starts fails its run with the reason `gate_start_failed`.
+
+Next to each event store, `run-events.db.run-prompts.json` (`0600`) holds the **full prompt text** of every run. The `runs` table keeps only its hash, so the sidecar is the one place a prompt survives. Event payloads, command results, and run plans can also contain instructions, summaries, and evidence.
+
+Every event commits in the same transaction as its outbox row. The supervisor pushes those rows to the bound hub, and a row the hub durably refuses stays parked until you run `kxm runtime sync-retry`. See [Runtime sync](../operations/runtime-sync.md).
+
+## Other files
+
+| File | What it holds | Notes |
+|---|---|---|
+| `hub-env.json` (state root) | The raw admin token and project-token map | Written `0600` through a temporary file; the hub's primary credential file |
+| `hub-binding.json` (state root) | This machine's hub URL and when it was bound | No credential |
+| `runtime/supervisor.token` (state root) | The raw supervisor token | `0600`; the registry stores only its hash |
+| `runtime/logs/kxm-runtime.jsonl` (state root) | The supervisor's structured log, including sync state changes | Rotates at 2 MiB |
+| `session.token` (user configuration directory) | An unsigned local session token with a tool policy | `0600`; 24-hour default lifetime |
+| `.kxm/state/hub.pid`, `hub.stop` | Hub process claim and stop request | Process control only; let `kxm hub stop` manage them |
+| `.kxm/state/pi-sessions/<worker>/` | Pi conversation history for a supervised worker | At most `KXM_WORKER_MAX_RUN_SESSIONS` (default 128) run histories per worker |
+| `.kxm/logs/kxm-hub.jsonl` | The hub's structured log, without message bodies | Rotates at 10 MiB and keeps three rotated files |
+| `.kxm/logs/pi-agent-<worker>.log` | Raw Pi standard output and error | Can contain model and tool output verbatim |
+| `.kxm/logs/telemetry.jsonl` | Result envelopes from gate commands | Read by `kxm improve` |
+| `.kxm/assets/retrospectives/<runId>.json`, `.md` | Exported retrospectives of finished hub runs | `0600`; no message bodies |
+
+Every SQLite database also has `-wal` and `-shm` sidecars while in use. The write-ahead log can hold copies of any recent row, so treat it as sensitively as the database.
+
+## What is stored, hashed, or never kept
+
+**Stored as sent**, without encryption and without general redaction:
+
+- message request and reply bodies, and rendered workflow prompts;
+- workflow evidence and journal entries;
+- agent keys, and the admin and project tokens in `hub-env.json`;
+- Runtime prompts, event payloads, command results, and run plans;
+- Pi session histories and raw Pi logs.
+
+**Kept only as a hash:**
+
+- the raw webhook body (`payloadHash`);
+- the request and reply behind each verified peer snapshot, which therefore survives message purging;
+- the Runtime prompt, in the event store's `runs` table (the sidecar keeps the text);
+- attempt capability secrets and, in the registry, the supervisor token;
+- gate command standard output and error, with their sizes;
+- the workflow definition (`definitionHash`, computed without its secrets), and the reproduction oracle and plan hash of a hub run.
+
+**Never persisted:** webhook signatures, which are checked and discarded, and workflow secrets, which stay in the environment variables that `secretEnv` and `signalSecretEnv` name.
+
+## Redaction
+
+KXM redacts with known credential patterns, such as API keys, bearer tokens, and named token variables. It applies them to:
+
+- the hub's structured log, which also redacts by key name;
+- context item summaries and source references;
+- retrospectives, ranked improvement signals, and some diagnostics;
+- every Runtime event before it enters the outbox, together with the allowlist transform described in the [trust model](trust-model.md#redaction-and-what-stays-local).
+
+Nothing redacts messages, hub workflow runs, the journal, or the Runtime's own stores at write time. Pattern redaction also misses unfamiliar secret formats, and its rule for 64-character hex strings masks ordinary SHA-256 digests too. Keep credentials out of prompts and evidence.
+
+## Retention
+
+| Data | How long it stays |
+|---|---|
+| Terminal messages (`replied`, `cancelled`, `expired`) | `KXM_MESSAGE_RETENTION_MS` after they end; default 7 days |
+| Finished hub workflow runs and their journal | 7 days after their last update; not configurable |
+| Journal entries whose run is gone | 7 days |
+| Superseded, rejected, or expired context items | 7 days after they end |
+| Expired leases | 7 days past their deadline, so a fencing token never restarts |
+| Agents, cursors, sequences, sync events, Runtime presence | Kept |
+| Runtime registry and event stores | Kept; the event log is append-only |
+| Retrospectives | Kept; purging a run does not remove them |
+
+Purging a hub run also ends webhook delivery deduplication for it and removes the episodes derived from its journal. Purging superseded state limits `--as-of` queries to roughly the last 7 days.
+
+## Schema versions: refuse, do not migrate
+
+Each database records its schema version in SQLite's `user_version`. When a process opens a store:
+
+- a newer store fails with `runtime_schema_newer`;
+- an older store fails with `runtime_schema_outdated`;
+- a store with tables but no version fails with `runtime_schema_shape_invalid`.
+
+KXM never upgrades a store in place, because the code would otherwise have to keep working against schema shapes it no longer tests. To cross a store version, stop the owner, back up if you need the history, delete the store with its `-wal` and `-shm` files, and let the owner recreate it: `kxm hub start` for `kxm.db`, and the Runtime for the registry and event stores. `kxm init` rebuilds no database. Legacy `.kxm/config/*.json` files are refused the same way.
+
+Every store opens in WAL mode with a 5-second busy timeout, `synchronous=NORMAL`, and foreign keys on. A database or sidecar that is a symbolic link is refused. [ADR-0003](../adr/ADR-0003-sqlite-only-store.md) records why SQLite is the only store.
+
+## Reset a store
+
+> [!CAUTION]
+> Deleting a store erases its history for good. Back it up first if you might need it. `kxm backup` copies the hub store, but it does not find the Runtime stores in the user state root, so copy those with the Runtime stopped.
+
+Stop every writer first, then delete each database together with its sidecars:
+
+```bash
+kxm hub stop
+kxm runtime stop
+```
+
+- Hub messages, workflow runs, and context: `.kxm/state/kxm.db`, `kxm.db-wal`, and `kxm.db-shm`.
+- Runtime runs for one project: `runtime/projects/<key>/` in the user state root, including `run-events.db`, its sidecars, and `run-events.db.run-prompts.json`.
+- Runtime registration: `runtime/registry.db` and its sidecars. Deleting it alone leaves event stores that nothing points to.
+- Hub credentials: `hub-env.json`, only when you intend to rotate them. The next `kxm hub start` generates a new admin token, and every client that used the old one fails until you reconfigure it.
+
+Never delete a live database, and never delete only the main file while its `-wal` remains. [Back up and restore](../operations/backup-and-restore.md) covers `kxm backup` and `kxm restore`, and [Upgrade KXM](../operations/upgrade.md) covers version changes.
+
+## Related
+
+- [Trust model](trust-model.md)
+- [Architecture](architecture.md)
+- [Back up and restore](../operations/backup-and-restore.md)
+- [Configuration file reference](../reference/config-reference.md#workspace-layout-tracked-ignored-and-state)
+- [ADR-0003: SQLite as the only store](../adr/ADR-0003-sqlite-only-store.md)

package/docs/concepts/trust-model.md

@@ -0,0 +1,152 @@
+# Trust model
+
+KXM decides who may do what with a small set of credentials, each scoped to one job. This page explains which credential proves what, who should hold it, and where each boundary stops, including the things KXM deliberately does not protect against. It is for operators who deploy a hub and for anyone reviewing KXM's security.
+
+## Who can reach what
+
+The diagram shows each credential holder and the only endpoints that credential opens.
+
+```mermaid
+flowchart LR
+  subgraph Holders
+    OP["Operator<br/>admin token"]
+    AG["Agents<br/>project token + agent key"]
+    RTS["Runtime supervisor<br/>project token"]
+    WH["Webhook sender<br/>start secret"]
+    CI["CI reporter<br/>signal secret"]
+    CLI["Local CLI<br/>supervisor token"]
+  end
+  subgraph Hub["KXM hub"]
+    ADMIN["Admin routes<br/>metrics, ops, degrade, promote"]
+    PROJ["Project routes<br/>register, presence, sync"]
+    AGENT["Agent routes<br/>messages, workflows, context, leases"]
+    START["Webhook start"]
+    SIGNAL["Run signal"]
+  end
+  RAPI["Runtime API<br/>127.0.0.1 only"]
+  OP -->|"bearer"| ADMIN
+  OP -.->|"only projects without a token"| PROJ
+  AG -->|"bearer"| PROJ
+  AG -->|"bearer + agent key"| AGENT
+  RTS -->|"bearer"| PROJ
+  WH -->|"HMAC-SHA256"| START
+  CI -->|"HMAC-SHA256"| SIGNAL
+  CLI -->|"bearer"| RAPI
+```
+
+`/health` and `/ready` need no credential and are not rate limited. Everything else on the hub needs one of the credentials below.
+
+## Credentials
+
+| Credential | Holder | Where it lives | What it opens |
+|---|---|---|---|
+| Admin token (`KXM_AUTH_TOKEN`) | The hub and a trusted operator terminal | The environment, else `hub-env.json` (`0600`) in the user state root | Admin routes; agent routes only for projects that have no project token |
+| Project token (`KXM_PROJECT_TOKENS` entry) | Agents and the Runtime for that project | The hub's environment or `hub-env.json`; clients pass it as `KXM_AUTH_TOKEN` | Registration, Runtime presence, and sync events in that one project |
+| Agent key | The hub and one registered client | Issued at registration, rotated at every reconnect, stored in `kxm.db` | Identity routes: messages, events, workflows, context, leases |
+| Workflow start secret | The hub and the webhook sender | The variable named by the definition's `secretEnv` | Starting runs of that one definition |
+| Workflow signal secret | The hub and the callback sender | The variable named by `signalSecretEnv`; falls back to the start secret | Signals for runs of that definition |
+| Runtime supervisor token | The Runtime and local CLI processes | `runtime/supervisor.token` (`0600`); the registry keeps only its hash | The Runtime API on `127.0.0.1` |
+| Session token | Local KXM tools | `session.token` (`0600`) in the user configuration directory, or `KXM_SESSION_TOKEN` | Local tool-policy checks and Studio mutation requests; never sent to the hub |
+| Attempt capability | The Runtime, for one attempt | Only its SHA-256 hash is stored | Settling that one attempt; revoked on cancel |
+
+`kxm hub start` generates an admin token (`kxm_admin_…`) when none exists and saves it, so a hub started that way always requires authentication. Give agents only their project token. Never give the admin token to an agent, a callback sender, or a dashboard that a project token can serve.
+
+### How clients choose a token
+
+- **Claude Code plugin.** The MCP server uses the plugin's `auth_token` setting, else the project token the hub saved for this project. It never falls back to the saved admin token; without a project token, its tools report that and do nothing.
+- **SessionStart hook.** It reads local state read-only. It mints no token, writes no file, and never prints a token.
+- **Pi extension.** It uses `KXM_AUTH_TOKEN`, else the token from a hub it auto-started, else the saved admin token. When your project has its own project token, set `KXM_AUTH_TOKEN` to it for Pi, because the hub rejects the admin token for that project.
+- **CLI and Runtime supervisor.** Operator tools use `KXM_AUTH_TOKEN`, else the saved project token, else the saved admin token.
+
+## Project isolation
+
+A project is the unit of isolation on a hub:
+
+- Every agent route checks the caller's project token and agent key together.
+- A message can target only an agent in the sender's project.
+- A context request outside the agent's project fails with `context_isolation_violation`.
+- A webhook run belongs to the project named in its definition, not to the caller.
+- The Runtime keeps a separate event store for every project.
+
+A project listed in `KXM_PROJECT_TOKENS` accepts only its own token on agent routes. A project missing from the map falls back to the admin token, so the admin token can register agents under any unlisted project name. List every project you run so the admin token stops working on agent routes.
+
+Isolation is logical. All projects on one hub share one process, one SQLite file, one log, and one in-memory rate limiter. Run a separate hub for teams that must not trust each other.
+
+## What provenance proves
+
+In a hub workflow, a coordinator can ask peers for evidence. The hub stamps each request with the run, stage, requirement, and attempt. At a checkpoint it re-reads its own records, verifies every cited message, and counts unique producer identities toward the quorum. The coordinator never counts as a producer.
+
+A verified quorum proves that the hub observed durable replies from the configured producer identities for this exact run, stage, requirement, and attempt. It does not prove:
+
+- that any reply is correct or true;
+- which model produced it, or that producers reasoned independently;
+- that producers did not collude;
+- that a person approved the result.
+
+Every holder of one project token is inside the same provenance domain, because a project-token holder can register a new agent or reclaim an offline agent's name and durable ID. Issue a separate project token per trust domain. Degrading a quorum needs the admin token, must be allowed by the stage's policy, applies to one attempt, and is journaled. See [Provenance gates](../guides/provenance-gates.md).
+
+## Context authority
+
+Context carries an authority level, and each origin can grant at most a fixed ceiling. The hub enforces the ceiling when it parses an item, and deriving new content from an item never raises its authority.
+
+| Origin | Highest authority it can grant |
+|---|---|
+| `human`, `workflow` | `policy` |
+| `git` | `instruction` |
+| `peer`, `tool`, `external`, `derived` | `evidence` |
+
+- A state proposal made with an agent key is **peer** origin, so it is capped at `evidence`.
+- A **human**-origin proposal needs a configured admin token, with no loopback exception.
+- `proposedBy` must name the authenticated caller, or the hub refuses the proposal.
+- Promoting a proposal to current state needs a configured admin token and a promoter other than the author.
+- Context items cannot carry control-plane fields such as `permissions`, `tools`, or `token`.
+- A Runtime-dispatched agent receives memory and promoted skills only when they are committed, clean, and match the run's pinned memory revision.
+
+See [Context and memory](../guides/context-and-memory.md).
+
+## Tool policy is a guardrail
+
+Every call to one of the 19 KXM agent tools checks a tool policy first, whether it comes from Claude Code, Pi, or a matching CLI command such as `kxm peer send`. The check reads `KXM_ATTEMPT_TOKEN`, then `KXM_SESSION_TOKEN`, then the session token on disk. `kxm session brief` and `kxm auth token --issue` write that disk token with an operator preset and a 24-hour lifetime.
+
+These tokens are unsigned JSON, so any local process can write one. Tool policy keeps a well-behaved agent inside its role; it does not stop a hostile one. An expired or malformed disk token denies every KXM agent tool until you remove it with `kxm session token --clear`. `kxm session token --status` reports no active token even while that expired file still blocks the tools.
+
+## Redaction and what stays local
+
+- The hub's structured log omits prompt and reply bodies and redacts values by key name and by known credential patterns.
+- The operations stream and `kxm dash` carry metadata only, never bodies.
+- The Runtime-to-hub sync builds a new event from an allowlist. Raw prompts, raw logs, diffs, environment values, and absolute paths never leave the machine that way. Registered secrets and credential shapes are replaced, and unknown fields are dropped.
+- Provider credentials never reach the hub, because the hub never runs an agent.
+
+Redaction is pattern-based and best effort. Message bodies, workflow evidence, the journal, and Runtime prompts and event payloads are stored as sent. [Data and storage](data-and-storage.md) lists what each store keeps.
+
+## Network exposure
+
+- The hub binds `127.0.0.1` by default and refuses to bind beyond loopback without an admin token. Started directly on loopback without a token, it prints `auth=none` and trusts every local caller.
+- The Runtime supervisor binds `127.0.0.1` only. Its health check answers a nonce with a keyed proof, so a client can tell the real supervisor from a process that took over a stale port.
+- `kxm studio serve` binds `127.0.0.1` by default and allows any browser origin. Keep it on loopback.
+- The hub's rate limit is per agent ID or remote address and lives in memory. It is a courtesy limit, not abuse protection.
+- Webhook signatures are HMAC-SHA256 over the raw body with no timestamp window. A replayed delivery is deduplicated by its delivery ID, not rejected, until its run is purged after 7 days.
+- Binding a machine to a remote hub (`kxm hub bind`) requires a credential for that hub.
+
+For a hosted hub, KXM uses one tenant per box. Browsers authenticate at an HTTPS proxy with Authentik, a portal backend calls the loopback hub and Runtime with machine credentials, and no hub port is public. The hub never reads browser identity headers. See [Deploy KXM](../operations/deploy.md) and [ADR-0004](../adr/ADR-0004-edge-identity-authentik.md).
+
+## KXM is not a sandbox
+
+> [!WARNING]
+> KXM does not isolate processes that run as the same OS user. A shell-capable agent can read any file that account can read, including hub state, tokens, and Pi session files.
+
+- `KXM_WORKER_TOOLS` limits which tools a Pi worker may call, not which paths it may touch.
+- Role descriptions and prompt instructions document intent; they do not remove shell, edit, or write tools.
+- Workflow session isolation keeps model contexts apart; it is not a security boundary.
+- Peer output stays untrusted even when the peer is authenticated.
+
+When an agent is outside your trust boundary, run it under a separate OS account or container, give it a read-only worktree, and withhold shell and write tools.
+
+## Related
+
+- [Architecture](architecture.md)
+- [Data and storage](data-and-storage.md)
+- [Security policy](../../SECURITY.md)
+- [Provenance gates](../guides/provenance-gates.md)
+- [Deploy KXM](../operations/deploy.md)
+- [ADR-0004: Edge identity with Authentik](../adr/ADR-0004-edge-identity-authentik.md)