@kontextmind/kxm 0.7.96 → 0.7.98

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (68) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/.kxm/workflows/default.yaml +2 -0
  3. package/CHANGELOG.md +61 -2
  4. package/docs/concepts/architecture.md +1 -1
  5. package/docs/concepts/data-and-storage.md +1 -1
  6. package/docs/concepts/trust-model.md +3 -2
  7. package/docs/contracts/routing.md +1 -1
  8. package/docs/contributing/test-matrix.md +6 -5
  9. package/docs/glossary.md +1 -1
  10. package/docs/guides/peer-messaging.md +2 -2
  11. package/docs/guides/pi-workers.md +1 -1
  12. package/docs/guides/webhook-workflows.md +53 -18
  13. package/docs/operations/backup-and-restore.md +43 -24
  14. package/docs/operations/deploy.md +2 -2
  15. package/docs/operations/troubleshooting.md +3 -2
  16. package/docs/reference/cli-reference.md +65 -30
  17. package/docs/reference/config-reference.md +24 -13
  18. package/docs/reference/configuration.md +2 -2
  19. package/docs/reference/harness-routing.md +3 -3
  20. package/docs/reference/http-api.md +7 -7
  21. package/docs/reference/tools.md +1 -1
  22. package/docs/reference/workflow-definitions.md +2 -2
  23. package/docs/start/first-workflow.md +4 -4
  24. package/docs/start/quickstart-claude-code.md +2 -2
  25. package/docs/start/quickstart-pi.md +1 -1
  26. package/examples/workflow-signal.ts +4 -5
  27. package/package.json +1 -1
  28. package/plugins/kxm/.claude-plugin/plugin.json +1 -1
  29. package/plugins/kxm/dist/claude-hook.js +11 -1
  30. package/plugins/kxm/dist/cli.js +552 -228
  31. package/plugins/kxm/dist/client.js +3 -1
  32. package/plugins/kxm/dist/core.js +16 -3
  33. package/plugins/kxm/dist/extension.js +45 -13
  34. package/plugins/kxm/dist/mcp-server.js +20 -4
  35. package/plugins/kxm/dist/runtime-supervisor.js +232 -51
  36. package/plugins/kxm/dist/runtime.js +432 -95
  37. package/plugins/kxm/dist/server.js +122 -20
  38. package/plugins/kxm/package.json +1 -1
  39. package/plugins/kxm/skills/kxm-hub-ops/SKILL.md +4 -3
  40. package/plugins/kxm/skills/kxm-project-setup/SKILL.md +3 -2
  41. package/plugins/kxm/skills/kxm-routing-improve/SKILL.md +3 -0
  42. package/plugins/kxm/skills/kxm-runs/SKILL.md +8 -7
  43. package/plugins/kxm/src/cli/project.ts +22 -13
  44. package/plugins/kxm/src/cli/system.ts +22 -1
  45. package/plugins/kxm/src/cli/workflows.ts +12 -7
  46. package/plugins/kxm/src/cli.ts +30 -8
  47. package/plugins/kxm/src/client.ts +4 -0
  48. package/plugins/kxm/src/commands.ts +23 -1
  49. package/plugins/kxm/src/database.ts +210 -36
  50. package/plugins/kxm/src/engine.ts +117 -2
  51. package/plugins/kxm/src/extension.ts +20 -14
  52. package/plugins/kxm/src/github-watch.ts +8 -5
  53. package/plugins/kxm/src/harness.ts +29 -0
  54. package/plugins/kxm/src/hub-env.ts +19 -1
  55. package/plugins/kxm/src/hub.ts +105 -21
  56. package/plugins/kxm/src/improve-sources.ts +2 -7
  57. package/plugins/kxm/src/init-guide-setup.ts +43 -28
  58. package/plugins/kxm/src/mcp-server.ts +9 -2
  59. package/plugins/kxm/src/oneshot-producer.ts +16 -8
  60. package/plugins/kxm/src/prices.ts +33 -2
  61. package/plugins/kxm/src/routing.ts +13 -7
  62. package/plugins/kxm/src/runtime-store.ts +23 -0
  63. package/plugins/kxm/src/studio-layout.ts +5 -4
  64. package/plugins/kxm/src/template.ts +31 -0
  65. package/plugins/kxm/src/workflow.ts +70 -1
  66. package/plugins/kxm/src/worktree-witness.ts +71 -0
  67. package/schemas/backup-manifest.schema.json +33 -0
  68. package/scripts/smoke-multi-pi.mjs +5 -1
@@ -11,7 +11,7 @@
11
11
  "name": "kxm",
12
12
  "source": "./plugins/kxm",
13
13
  "description": "Durable workflows, peer agents, and kxm tui",
14
- "version": "0.7.96",
14
+ "version": "0.7.98",
15
15
  "category": "development",
16
16
  "tags": ["kxm", "multi-agent", "workflows", "mcp"]
17
17
  }
@@ -7,6 +7,8 @@ steps:
7
7
  - id: implement
8
8
  kind: agent
9
9
  agent: implementer
10
+ repositories:
11
+ control: write
10
12
  maxAttempts: 3
11
13
  on:
12
14
  passed: review-arch
package/CHANGELOG.md CHANGED
@@ -69,6 +69,18 @@ All notable user-facing changes are documented here. The project follows [Semant
69
69
 
70
70
  ### Changed
71
71
 
72
+ - **The rotation surface drive uses is the one setup writes.** Guide setup appends only
73
+ reviewed selectors to `.kxm/routes.yaml` and routes Google through Pi `antigravity`.
74
+
75
+ - **CLI and Studio stop reporting work they did not do.** Drive help no longer
76
+ describes the default as a model-free simulation. A Studio mutation with no handler returns 501 and `mappedToCli: false`.
77
+
78
+ - **Missing cost stays unknown.** `kxm prices acknowledge` restamps the local catalog
79
+ as today without fetching vendor rates, which is what an estimate will accept.
80
+ Routing totals are null when any attempt has no cost, and those rows sort after
81
+ complete-cost rows. `kxm improve` stays proposal-only. Wiki compile writes a file
82
+ only with `--out` and does not ingest.
83
+
72
84
  - **The workflow loader refuses a gate step that can never settle
73
85
  (`gate_outcome_impossible`).** A gate step settles only on `passed` or
74
86
  `implementation-failure` when `expect` is `pass`, and only on `passed` or `repro-missing`
@@ -90,8 +102,8 @@ All notable user-facing changes are documented here. The project follows [Semant
90
102
  execute until the run engine lands. The JSON result and its
91
103
  `phase` are unchanged. A `run_handoff_required` refusal from `kxm runs drive` now ends
92
104
  with `(handoff reason …; field …; detail …)`, each part capped at 200 characters; a run of
93
- the `default` workflow that `kxm init` writes, for example, reports `limit_unsupported`
94
- on `limits.maxAgentTimeMs`. Top-level help names the product KXM instead of KontextMind,
105
+ a workflow that declares `limits.maxAgentTimeMs`, for example, reports `limit_unsupported`
106
+ on that field. Top-level help names the product KXM instead of KontextMind,
95
107
  and `kxm init` text output lists each validation issue as `file: code: message`.
96
108
  - **`kxm suggest` recommends only KXM command skills.** Suggested skills come from the
97
109
  command skills shipped in `plugins/kxm/skills` (such as `kxm-workflow`, `kxm-runs`,
@@ -307,6 +319,53 @@ All notable user-facing changes are documented here. The project follows [Semant
307
319
 
308
320
  ### Fixed
309
321
 
322
+ - **Live `kxm runs drive` can author on an audited writer profile.** A write-repository
323
+ step on pi (`-a`, with extensions, skills, and the session off) or grok
324
+ (`--always-approve`, with subagents and web search off) runs against the checkout.
325
+ A live write that leaves the tree unchanged settles `failed` with `authored: false`.
326
+ A read-only step that changes the tree cannot settle `passed`. Harnesses without a
327
+ writer profile still hand off. Simulated drive does not require a diff. The witness
328
+ fingerprints the one checkout, so a live write step must be a single assignment
329
+ (`assignments.maximum: 1`) in a project whose `limits.maxConcurrentRuns` is 1; a
330
+ panel of writers or a project that admits concurrent runs hands off with
331
+ `step_unsupported` instead of crediting one writer's change to another.
332
+
333
+ - **Fresh `kxm init` can be driven.** The current template drops `limits.maxAgentTimeMs`,
334
+ names coordinator `claude` / `anthropic/fable` and implementer `grok` / `xai/grok-4.6`,
335
+ and admits those two routes. One-shot production no longer falls through to an
336
+ unadmitted `claude-3-7-sonnet`.
337
+
338
+ - **`kxm backup` includes the Runtime stores under the user-state root.** Discovery
339
+ copies `$S/runtime/registry.db` and `$S/runtime/projects/<projectKey>/run-events.db`,
340
+ plus each `run-events.db.run-prompts.json` sidecar. A copy that misses a discovered
341
+ store is `complete: false`: `kxm backup` exits 1 with `ok: false`, and restore
342
+ refuses that manifest. Cross-box remap of absolute `$S` paths is still the file recipe.
343
+ The Runtime stores are machine-wide, so a backup holds every project's run store and a
344
+ restore rolls all of them back. `kxm backup --help` no longer says Runtime stores are
345
+ left out.
346
+
347
+ - **Signed webhooks cannot be replayed.** KXM's own webhook senders now sign the
348
+ timestamp, delivery ID, definition, run and signal key along with the body
349
+ (`x-kxm-signature`, `x-kxm-timestamp`, `x-kxm-delivery-id`; see
350
+ `docs/guides/webhook-workflows.md#kxm-sender-contract`), and the hub refuses a signature
351
+ more than 300 seconds old. Every `generic` workflow start and every signal callback
352
+ must use this contract; a body-only `X-Hub-Signature-256` there is refused. Jira and
353
+ GitHub deliveries keep their provider signature, and a signed body starts at most one
354
+ run (`webhook_payload_replayed`). A reused delivery ID with a different body is
355
+ refused with 409 `webhook_delivery_conflict`, and a duplicate start returns only
356
+ `duplicate`, `runId` and `status`. Update any custom sender to the new contract.
357
+ - **Agents never borrow the hub admin token.** With `KXM_AUTH_TOKEN` unset, the Pi
358
+ extension and the `kxm peer` / `kxm workflow` agent commands used the persisted admin
359
+ token. They now use only this project's saved project token, as the Claude MCP server
360
+ does, and otherwise stop with a message naming the fix (`project_token_missing`, exit 2,
361
+ on the CLI).
362
+ - **The hop limit bounds agent forwarding chains.** `kxm_send` and `kxm_fanout` from Pi
363
+ or the Claude MCP server send one hop past the inbound request being handled, so a
364
+ chain of agents forwarding to each other stops at `hop_limit_reached`.
365
+ - **Workflow prompts no longer point agents at `.kxm/config`**, a path KXM refuses.
366
+ - **`kxm gate signal` and `kxm workflow wait` inside a KXM project reach the hub for hub
367
+ runs.** They go to the local Runtime only for a run its store holds.
368
+
310
369
  - **The Claude plugin's SessionStart hook is one bundled, read-only, project-scoped
311
370
  script.** The two shell hooks it replaces (`kxm session brief --status` and
312
371
  `kxm memory brief`) exited 127 without `kxm` on `PATH`, ran whichever `kxm` was on
@@ -204,7 +204,7 @@ The package ships a directory of `SKILL.md` suites that Pi and Claude Code both
204
204
 
205
205
  `kxm dash` draws live terminal screens (agents, tasks, workflows, plans, inbox, processes, and spend, which is always empty today) from the admin-only operations stream, a presence-only fallback, and a read-only snapshot of `kxm.db`. Treat it as an observer: its action keys `a`, `r`, `s` and `c` post to hub routes that do not exist, so they change nothing even though the status line reports success, and `d` creates a git branch and worktree.
206
206
 
207
- `kxm studio layout` renders a workflow as DAG, stepper, and swimlane JSON, and `kxm studio serve` hosts a local viewer on `127.0.0.1:4242`. The Studio mutation endpoint is not wired to commands: it acknowledges requests without running them.
207
+ `kxm studio layout` renders a workflow as DAG, stepper, and swimlane JSON, and `kxm studio serve` hosts a local viewer on `127.0.0.1:4242`. The Studio mutation endpoint is not wired to commands: it answers every allowed request with HTTP 501 `mutation_handler_missing` and runs nothing.
208
208
 
209
209
  ## Configuration is reviewed Git YAML
210
210
 
@@ -169,7 +169,7 @@ Every store opens in WAL mode with a 5-second busy timeout, `synchronous=NORMAL`
169
169
  ## Reset a store
170
170
 
171
171
  > [!CAUTION]
172
- > Deleting a store erases its history for good. Back it up first if you might need it. `kxm backup` copies the hub store, but it does not find the Runtime stores in the user state root, so copy those with the Runtime stopped.
172
+ > Deleting a store erases its history for good. Back it up first if you might need it. `kxm backup` copies the hub store and the Runtime stores in the user state root, for every project on the machine.
173
173
 
174
174
  Stop every writer first, then delete each database together with its sidecars:
175
175
 
@@ -55,8 +55,9 @@ flowchart LR
55
55
 
56
56
  - **Claude Code plugin.** The MCP server uses the plugin's `auth_token` setting, else the project token the hub saved for this project. It never falls back to the saved admin token; without a project token, its tools report that and do nothing.
57
57
  - **SessionStart hook.** It reads local state read-only. It mints no token, writes no file, and never prints a token.
58
- - **Pi extension.** It uses `KXM_AUTH_TOKEN`, else the token from a hub it auto-started, else the saved admin token. When your project has its own project token, set `KXM_AUTH_TOKEN` to it for Pi, because the hub rejects the admin token for that project.
59
- - **CLI and Runtime supervisor.** Operator tools use `KXM_AUTH_TOKEN`, else the saved project token, else the saved admin token.
58
+ - **Pi extension.** It uses `KXM_AUTH_TOKEN`, else the project token the hub saved for this project. It never uses the saved admin token, nor the one a hub it auto-started generated; without a project token it reports the fix and stays offline.
59
+ - **`kxm peer` and `kxm workflow` agent commands.** They act as a peer agent, so they choose a token as the Pi extension does, and exit 2 with `project_token_missing` when none resolves.
60
+ - **Dashboard and Runtime supervisor.** Operator tools use `KXM_AUTH_TOKEN`, else the saved project token, else the saved admin token.
60
61
 
61
62
  ## Project isolation
62
63
 
@@ -227,7 +227,7 @@ backfilled: they still resolve an outcome, but they group per run.
227
227
  - **Price catalog:** `.kxm/prices.yaml` (`kxm.prices.v1`, dated and hashed) defines input, output, cache-read, cache-write rates, and context tiers for active models. Missing rows or uncataloged models evaluate to `costBasis: "unknown"`.
228
228
  - **Ranked report:** `kxm routing report` (`plugins/kxm/src/routing.ts`, CLI command `kxm routing report`) groups records by `(harness, model, thinking, role)`.
229
229
  - **Ranking order:** Quality first (`verifyPassRate` descending, then `reworkRate` ascending where rework measures back-edge re-entries `transitions > 0`), followed by `costPerAcceptedUsd` ascending.
230
- - **Underquote prevention:** Routes with unknown cost are flagged (`*`). Any unknown-cost attempt makes a route's cost a lower bound, so the route ranks after every route of equal quality that has no unknown-cost attempt. Displayed values are unchanged: a route that mixes unmetered and unknown-cost attempts still shows `$0` per accepted attempt, so read the flag and the population counts before comparing cost.
230
+ - **Underquote prevention:** Routes with unknown cost are flagged (`*`). Any unknown-cost attempt makes a route's cost unknown: `costPerAcceptedUsd` is `null` (displayed `-`), never a partial sum, and the route ranks after every route of equal quality that has no unknown-cost attempt. Per-configuration comparisons likewise report `totalCostUsd: null` with a `missingCostRuns` count when any record lacks a cost.
231
231
  - **Population separation:** Reports metered cost, unmetered attempt counts, unknown-cost attempt counts, and quota-exhausted attempt counts as separate metrics rather than a single misleading total.
232
232
  - **List prices flag:** Supports `--equivalent-list-cost` / `--list-prices` to display estimated list rates for comparison alongside actual recorded spend.
233
233
  - **Rework column:** reads `transitions`, which Runtime records never set, so Runtime rework shows up only as a resolved `reworked` outcome, which the report does not count as a pass.
@@ -133,16 +133,17 @@ The context suites in detail:
133
133
  | Metadata-only dashboard: ops mode, presence-only fallback, observer filtering, keys, body-free local projection | `tui.test.ts`, `hub-api.test.ts` |
134
134
  | Restricted loader, deterministic bundle hash, init classification, atomic creation, three-way repair, crash resumption | `project-config.test.ts`, `cli.test.ts`, `package-install.test.ts` |
135
135
  | Legacy `.kxm/config` JSON is refused with `legacy_state_unsupported`; `kxm migrate` is unknown; older stores are refused | `project-config.test.ts`, `cli.test.ts`, `e6-backup-restore-migrations.test.ts` |
136
- | `kxm backup` and `kxm restore` round-trip the hub store and Runtime stores seeded under `.kxm/runtime/`; tampered or newer backups are refused | `e6-backup-restore-migrations.test.ts` |
136
+ | `kxm backup` and `kxm restore` round-trip the hub store and Runtime stores seeded under `.kxm/runtime/`; Runtime stores and prompt sidecars under `KXM_STATE_HOME` are discovered; tampered, newer or incomplete (`complete: false`) backups are refused | `e6-backup-restore-migrations.test.ts` |
137
137
  | Permission-diff trust: authority lattice, prose neutrality, Git base shadowing, CLI diff and check | `permission.test.ts`, `cli.test.ts`, `contracts.test.ts`, `package-install.test.ts` |
138
138
  | KXM schemas, restricted YAML fixtures, cross-resource semantics and sync-safe rejection | `contracts.test.ts`, `restricted-yaml.test.ts` |
139
139
  | Harness detection, auth and dispatch for the built-in catalog, including Windows launch rules | `harness.test.ts` |
140
140
  | `kxm update`: `update.yaml` validation, GitHub or npm version checks, install-kind detection, `kxm-<v>.tgz` asset selection | `kxm-update.test.ts`, `kxm-update-cli.test.ts`, `kxm-install-kind.test.ts` |
141
141
 
142
- The backup round trip seeds its Runtime stores in a project-local
143
- `.kxm/runtime/` layout that the Runtime never writes. Production Runtime stores
144
- live under the user state root, which `kxm backup` does not discover, so no test
145
- backs up the real Runtime stores.
142
+ The round trip seeds its Runtime stores in a project-local `.kxm/runtime/`
143
+ layout that the Runtime never writes. A separate test seeds a registry, an event
144
+ store and its prompt sidecar under a temporary `KXM_STATE_HOME` and checks that
145
+ `kxm backup` finds them and that a partial manifest is refused. No test backs up
146
+ stores a running supervisor wrote.
146
147
 
147
148
  ## Packaging, release and repository gates
148
149
 
package/docs/glossary.md CHANGED
@@ -184,7 +184,7 @@ The coordinator's recorded result for the active [stage](#stage): `passed`, `war
184
184
 
185
185
  ### Delivery ID
186
186
 
187
- The identifier a webhook sender puts in `X-GitHub-Delivery`, `X-Atlassian-Webhook-Identifier` or `x-kxm-delivery-id`. The hub requires one and deduplicates on it: a repeated start delivery returns the existing run.
187
+ The identifier a webhook sender puts in `x-kxm-delivery-id`, which the KXM sender contract signs, or for a Jira or GitHub delivery in `X-Atlassian-Webhook-Identifier` or `X-GitHub-Delivery`. The hub requires one and deduplicates on it: a repeated start with the same body returns the existing run ID, and the same ID with a different body is refused.
188
188
 
189
189
  ### Journal
190
190
 
@@ -246,7 +246,7 @@ Neither field proves where a reply came from. Workflow evidence uses `workflowCo
246
246
 
247
247
  When a request expires, both the sender and the recipient receive an `expired` event.
248
248
 
249
- The hop limit refuses a request whose `hops` count has reached `maxHops` (`hop_limit_reached`). Only a client that forwards a request through the [Hub HTTP API reference](../reference/http-api.md) and carries its hop count is affected. `kxm_send`, `kxm_fanout`, and `kxm peer send` always start at hop 0, so the limit does not stop a chain of agents that each send a new request.
249
+ The hop limit refuses a request whose `hops` count has reached `maxHops` (`hop_limit_reached`). In Pi and in Claude Code, `kxm_send` and `kxm_fanout` send one hop past the inbound request the session is handling, under that request's `maxHops`, so a chain of agents forwarding work to each other stops at the limit. The Claude Code MCP server counts from the furthest request in its open inbox. `kxm peer send` from the CLI handles no inbound request, so it always starts a new chain at hop 0.
250
250
 
251
251
  ## Queue work for an offline agent
252
252
 
@@ -272,7 +272,7 @@ Tools and the CLI report the message text; the HTTP response also carries the co
272
272
  | `online target not found: <name>` | `target_not_found` | The target is offline or unknown. Check `kxm_list`; use `allowOffline` for a registered agent. |
273
273
  | `cannot send a request to yourself` | `self_target` | Send to a different agent. |
274
274
  | `idempotency key was already used for another request` | `idempotency_conflict` | A different request reused the key. Repeat the original exactly, or use a new key for new work. |
275
- | `hop limit reached (<hops>/<max>)` | `hop_limit_reached` | A forwarding client reached `maxHops`. Stop the chain. |
275
+ | `hop limit reached (<hops>/<max>): …` | `hop_limit_reached` | Forwarding would extend the chain past `maxHops`. Answer the inbound request directly instead of forwarding it. |
276
276
  | `ttlMs must be an integer between 1000 and 604800000` | `protocol_error` | Use a TTL from 1 second to 7 days. |
277
277
  | `message is not visible to this agent` | `message_forbidden` | Run as the sender or recipient. |
278
278
  | `message already has a reply` | `duplicate_reply` | The request is already answered. |
@@ -60,7 +60,7 @@ would start worker
60
60
  ```
61
61
 
62
62
  > [!IMPORTANT]
63
- > Set `KXM_AUTH_TOKEN` to the project token in the worker's environment. When it is unset, the Pi extension falls back to the hub credential persisted on this machine, which is the admin token for a hub that `kxm hub start` or Pi auto-start created.
63
+ > Set `KXM_AUTH_TOKEN` to the project token in the worker's environment. When it is unset, the Pi extension uses only the project token saved for this project on this machine. It never falls back to the saved admin token, so without a project token the worker's Pi session stays offline and says so.
64
64
 
65
65
  | Flag | Environment variable | Effect |
66
66
  |---|---|---|
@@ -47,7 +47,7 @@ sequenceDiagram
47
47
  Hub-->>Coord: completed = true
48
48
  ```
49
49
 
50
- The hub verifies the signature over the raw body, deduplicates retries by delivery ID, and records the run before it answers. It keeps a SHA-256 hash of the payload and the rendered prompt, not the raw body, so keep prompt templates narrow.
50
+ The hub verifies the delivery's signature, refuses replays, deduplicates retries by delivery ID, and records the run before it answers. It keeps a SHA-256 hash of the payload and the rendered prompt, not the raw body, so keep prompt templates narrow.
51
51
 
52
52
  ## Copy the example definition
53
53
 
@@ -102,7 +102,7 @@ A file holds a JSON array of definitions. The hub checks every limit below at st
102
102
  | Field | Meaning |
103
103
  |---|---|
104
104
  | `id` | Up to 64 characters; the last segment of the webhook URL. |
105
- | `source` | `jira`, `github`, or `generic` (the default). A label on the run. |
105
+ | `source` | `jira`, `github`, or `generic` (the default). A label on the run, and it decides which signatures the hub accepts: see [Signatures and delivery IDs](#signatures-and-delivery-ids). |
106
106
  | `project`, `target` | Hub project, and the coordinator's agent name or ID. |
107
107
  | `secretEnv` | Variable holding the start secret. `secret` takes a literal instead; prefer the variable. |
108
108
  | `signalSecretEnv` | Variable holding the callback secret. Without it, callbacks use the start secret. |
@@ -180,7 +180,7 @@ kxm agent worker --name coordinator --project product --model <pi-model> \
180
180
 
181
181
  ## Send a test delivery
182
182
 
183
- `kxm workflow start` signs a payload and posts it to the hub at `KXM_SERVER_URL`, as a provider would. It signs with `KXM_WORKFLOW_SECRET`, or with the definition's own start secret when `KXM_WEBHOOK_WORKFLOWS_FILE` is set in the same shell.
183
+ `kxm workflow start` signs a payload under the [KXM sender contract](#kxm-sender-contract) and posts it to the hub at `KXM_SERVER_URL`, as a provider would. It signs with `KXM_WORKFLOW_SECRET`, or with the definition's own start secret when `KXM_WEBHOOK_WORKFLOWS_FILE` is set in the same shell.
184
184
 
185
185
  ```bash
186
186
  export KXM_SERVER_URL=http://127.0.0.1:7331
@@ -195,7 +195,7 @@ Expected output:
195
195
  started workflow run_7c187d2a0fde408ea408f0d8c53201c8
196
196
  ```
197
197
 
198
- Repeating the command with the same delivery ID returns the same run. Inspect runs from the hub's workspace on the hub host; these commands read its local SQLite store:
198
+ Repeating the command with the same delivery ID and payload returns the same run ID; the same delivery ID with a different payload is refused with `409`. Inspect runs from the hub's workspace on the hub host; these commands read its local SQLite store:
199
199
 
200
200
  ```bash
201
201
  kxm workflow list
@@ -204,26 +204,60 @@ kxm workflow get run_7c187d2a0fde408ea408f0d8c53201c8 --json
204
204
 
205
205
  ## Connect Jira
206
206
 
207
- In Jira, create a webhook for the `jira:issue_updated` event that posts to `https://<kxm-host>/v1/webhooks/jira-development`, and set the same start secret. The hub accepts these headers:
207
+ In Jira, create a webhook for the `jira:issue_updated` event that posts to `https://<kxm-host>/v1/webhooks/jira-development`, and set the same start secret.
208
208
 
209
- | Header | Sent by | Purpose |
209
+ ### Signatures and delivery IDs
210
+
211
+ The hub accepts two ways to sign a start:
212
+
213
+ | Headers | Sent by | Accepted by |
210
214
  |---|---|---|
211
- | `X-Hub-Signature-256` or `X-Hub-Signature` | GitHub, Jira, `kxm` | `sha256=<hex>` HMAC of the exact body. Other algorithms are refused. |
212
- | `X-Atlassian-Webhook-Identifier` | Jira | Delivery ID. Checked first. |
213
- | `X-GitHub-Delivery` | GitHub | Delivery ID. Checked second. |
214
- | `X-KXM-Delivery-ID` | `kxm` and your own senders | Delivery ID. Checked last. |
215
+ | `x-kxm-signature`, `x-kxm-timestamp`, `x-kxm-delivery-id` | `kxm` and your own senders | Every definition. See [KXM sender contract](#kxm-sender-contract). |
216
+ | `X-Hub-Signature-256` or `X-Hub-Signature`, with `X-Atlassian-Webhook-Identifier` | Jira | A `jira` definition only. |
217
+ | `X-Hub-Signature-256` with `X-GitHub-Delivery` | GitHub | A `github` definition only. |
218
+
219
+ Jira and GitHub sign only the body, as `sha256=<hex>`; other algorithms are refused. Because the delivery ID is not signed, the body is the delivery's replay identity: a signed body starts at most one run, under one delivery ID.
215
220
 
216
221
  | Response | Meaning |
217
222
  |---|---|
218
223
  | `202` | Run created and coordinator prompt queued. |
219
- | `200` with `"duplicate": true` | The delivery ID was seen before; the existing run is returned. The body is not compared. |
224
+ | `200` with `"duplicate": true` | The delivery ID and body were seen before. Only `runId` and `status` come back, never the run. |
220
225
  | `204` | The event or filter did not match. Nothing was stored. |
221
- | `401` | Missing, unsupported, or wrong signature. |
226
+ | `401` | Missing, unsupported, or wrong signature, or a KXM signature outside its 300-second window. |
222
227
  | `404` | No definition with that ID. |
223
- | `409` | The coordinator has never registered. |
228
+ | `409` `workflow_target_unavailable` | The coordinator has never registered. |
229
+ | `409` `webhook_delivery_conflict` | The delivery ID was already used with a different body. |
230
+ | `409` `webhook_payload_replayed` | A Jira or GitHub body already started a run under another delivery ID. |
224
231
 
225
232
  Webhook authentication authorizes only workflow creation. The `report` stage updates Jira through the coordinator's own authorized Jira tool; never put Jira credentials in a definition or prompt.
226
233
 
234
+ ## KXM sender contract
235
+
236
+ A body-only signature cannot tell a retry from a replay, so KXM's own senders sign more: `kxm workflow start`, `kxm gate signal`, `kxm gate github watch`, and [`examples/workflow-signal.ts`](../../examples/workflow-signal.ts). A start of a `generic` definition and every signal must use this contract; a body-only signature there is refused with `401 webhook_signature_missing`.
237
+
238
+ | Header | Value |
239
+ |---|---|
240
+ | `x-kxm-delivery-id` | Stable retry identifier, at most 128 characters |
241
+ | `x-kxm-timestamp` | Unix seconds at send time |
242
+ | `x-kxm-signature` | `sha256=` and the hex HMAC-SHA256 of the signed material, under the start or callback secret |
243
+
244
+ The signed material is seven newline-terminated fields followed by the exact body bytes. No field may contain a line break. `workflowWebhookHeaders` in `plugins/kxm/src/workflow.ts` builds all three headers.
245
+
246
+ ```text
247
+ kxm-webhook-v1
248
+ start | signal
249
+ <x-kxm-timestamp>
250
+ <x-kxm-delivery-id>
251
+ <definition ID>
252
+ <run ID; empty for a start>
253
+ <signal key; empty for a start>
254
+ <body>
255
+ ```
256
+
257
+ - A signature over any other timestamp, delivery ID, definition, run, signal key, or body is refused with `401 webhook_signature_invalid`, so a captured request cannot be replayed under a new delivery ID or against another run.
258
+ - An authentic signature whose timestamp is more than 300 seconds from the hub clock is refused with `401 webhook_timestamp_expired`. Sign at send time: a retry re-signs with a fresh timestamp and keeps the same delivery ID and body, which the hub answers as a duplicate.
259
+ - The signed kind keeps a start signature from ever verifying as a signal, even when both use the start secret.
260
+
227
261
  ## Follow the coordinator procedure
228
262
 
229
263
  For every stage, the coordinator:
@@ -276,7 +310,7 @@ Tool call (`kxm_workflow_wait`):
276
310
 
277
311
  Store the run ID and signal key where the external system can find them, such as pull-request metadata; they are not secrets.
278
312
 
279
- The external system reports back with a signed signal to `POST /v1/webhooks/<definition-id>/runs/<run-id>/signals/<signal-key>`. The body is `{"status": "passed" | "warning" | "failed", "summary": "...", "evidence": {...}}`, signed with the callback secret in `X-Hub-Signature-256`, with a stable `X-KXM-Delivery-ID`.
313
+ The external system reports back with a signed signal to `POST /v1/webhooks/<definition-id>/runs/<run-id>/signals/<signal-key>`. The body is `{"status": "passed" | "warning" | "failed", "summary": "...", "evidence": {...}}`, signed with the callback secret under the [KXM sender contract](#kxm-sender-contract), bound to the route's run ID and signal key, with a stable delivery ID.
280
314
 
281
315
  - `passed` applies the evidence rule to the saved and new evidence together, then advances or completes the run.
282
316
  - `warning` or `failed` uses up an attempt, records an error, and sends the coordinator a correction prompt while attempts remain.
@@ -303,8 +337,8 @@ Expected output:
303
337
  posted signed signal
304
338
  ```
305
339
 
306
- > [!WARNING]
307
- > Run `kxm gate signal` and `kxm workflow wait` outside any KXM project directory. Inside a Git repository with `.kxm/project.yaml`, they treat a `run_<32-hex>` ID as a Runtime run and send it to the Runtime supervisor, not the hub. Check with `--dry-run`: the hub path prints `would post signed signal`, the Runtime path `would post signal to KXM run`.
340
+ > [!NOTE]
341
+ > Hub and Runtime run IDs share the `run_<32-hex>` shape. Inside a KXM project, `kxm gate signal` and `kxm workflow wait` send a run to the Runtime supervisor only when the project's Runtime store holds that run; a hub run goes to the hub. Check with `--dry-run`: the hub path prints `would post signed signal`, the Runtime path `would post signal to KXM run`.
308
342
 
309
343
  For your own adapters, [`examples/workflow-signal.ts`](../../examples/workflow-signal.ts) shows the same signed request in about 60 lines.
310
344
 
@@ -339,7 +373,7 @@ If a stage's peer policy declares a lower `degradation.minProducers`, an operato
339
373
  ## Security notes
340
374
 
341
375
  - The start secret authorizes creating runs; the callback secret authorizes only checkpointing a waiting run with a matching signal key. Keep them separate with `signalSecretEnv`.
342
- - The signature covers the body, not the delivery-ID header, and there is no timestamp window. Anyone who captures a signed request can replay it under a new delivery ID, so terminate TLS and restrict ingress.
376
+ - A KXM signature covers the timestamp, delivery ID, definition, run, and signal key, and expires after 300 seconds, so a captured request cannot be replayed under a new delivery ID or against another run. Jira and GitHub sign only the body: a captured provider delivery re-sent under its own delivery ID only returns the duplicate, and a body starts at most one run. Terminate TLS and restrict ingress anyway.
343
377
  - Repository rules, human approvals, and harness permissions stay in charge of pushing, merging, and changing Jira.
344
378
 
345
379
  ## Troubleshooting
@@ -353,7 +387,8 @@ If a stage's peer policy declares a lower `degradation.minProducers`, an operato
353
387
  | `stage <id> is missing required evidence: <key>` | A required key is missing or empty. | Supply every key from `kxm_workflow_get`. |
354
388
  | `stage <id> is not currently active` or `workflow is waiting` | Wrong stage, or the stage is paused for a signal. | Use `currentStage`; send the signal instead of a checkpoint. |
355
389
  | The run fails right after the coordinator replies | It settled before the last checkpoint. | Checkpoint every stage, or wait, before replying. |
356
- | `kxm gate signal` prints `signed signal failed` | Wrong signal key, run not waiting, or a delivery-ID conflict. | Compare the run's `waiting.signalKey`; use a new delivery ID for a new result. |
390
+ | `kxm gate signal` prints `signed signal failed` | Wrong signal key, run not waiting, a delivery-ID conflict, or a clock more than 300 seconds off. | Compare the run's `waiting.signalKey`; use a new delivery ID for a new result; check the sender's clock. |
391
+ | A custom sender gets `401 webhook_signature_missing` | It sends only `X-Hub-Signature-256` to a `generic` definition or a signal. | Sign with the [KXM sender contract](#kxm-sender-contract). |
357
392
 
358
393
  ## Next steps
359
394
 
@@ -59,17 +59,17 @@ Record every override with the backup. A restore that lands where the running se
59
59
 
60
60
  ## Know what each backup covers
61
61
 
62
- `kxm backup` protects SQLite stores it can find from the current directory. Everything else needs the stopped-state copy described below.
62
+ `kxm backup` copies the SQLite stores it can find: the hub store under the current directory, and the Runtime stores and their prompt sidecars under the user state root. Everything else needs the stopped-state copy described below.
63
63
 
64
64
  > [!WARNING]
65
- > `kxm backup` does not back up the Runtime. It looks for Runtime stores under `.kxm/runtime/` in the checkout, but the Runtime writes them under the user state root (`$S/runtime/registry.db` and `$S/runtime/projects/<key>/run-events.db`). In practice a backup holds only the hub store. Back up the Runtime stores and their prompt sidecars by hand, stopped, as shown in [Back up everything else](#back-up-everything-else).
65
+ > The Runtime stores are shared by every project on the machine. `kxm backup` copies `$S/runtime/registry.db` and the `run-events.db` of every project under `$S/runtime/projects/`, not only the project you run it from, and `kxm restore` writes all of them back. Restoring one project's backup returns every project's runs to the moment of that backup.
66
66
 
67
67
  | Path | Holds | In `kxm backup` |
68
68
  |---|---|---|
69
69
  | `$W/kxm.db` | Hub store: agents, messages, workflow runs, journals, context items, leases, synced run facts | Yes, at `<current directory>/.kxm/state/kxm.db` only |
70
- | `$S/runtime/registry.db` | Runtime registry: projects, their roots, the supervisor identity and claim | No |
71
- | `$S/runtime/projects/<key>/run-events.db` | Event-sourced runs, drive receipts, gate evidence, the sync outbox | No |
72
- | `$S/runtime/projects/<key>/run-events.db.run-prompts.json` | Run prompt text; restoring a store without it loses every prompt | No |
70
+ | `$S/runtime/registry.db` | Runtime registry: projects, their roots, the supervisor identity and claim | Yes (`registry`) |
71
+ | `$S/runtime/projects/<key>/run-events.db` | Event-sourced runs, drive receipts, gate evidence, the sync outbox | Yes, for every project (`events:<key>`) |
72
+ | `$S/runtime/projects/<key>/run-events.db.run-prompts.json` | Run prompt text; restoring a store without it loses every prompt | Yes, as a plain file (`events:<key>:run-prompts`) |
73
73
  | `$S/projects/<hash>/repository-bindings.json`, `$S/update.yaml` | Member repository paths; updater settings | No |
74
74
  | `$S/hub-env.json`, `$S/hub-binding.json`, `$C/session.token` | Credentials and the machine's hub binding | No; prefer regenerating secrets to copying them |
75
75
  | `$R/.kxm/` definition files and durable records | Project, roles, routes, prices, roster, memory, skills, goals, tasks, candidates | No; commit them to Git or copy the checkout |
@@ -82,9 +82,9 @@ A restore without `roster.yaml`, `routes.yaml` or `prices.yaml` comes back healt
82
82
 
83
83
  These files are disposable and need no backup: `hub.pid`, `hub.stop`, `worker-*.pid`, `session-brief.json`, `update-check.json`, `runtime/supervisor.token`, `runtime/supervisor.error`.
84
84
 
85
- ## Back up the hub store with `kxm backup`
85
+ ## Back up the SQLite stores with `kxm backup`
86
86
 
87
- Run it from the checkout root. It resolves stores from the current directory and ignores `--workspace`, `KXM_WORKDIR`, `KXM_STATE_DIR` and `KXM_DATA_PATH`, so a relocated hub database is not found.
87
+ Run it from the checkout root. It finds the hub store from the current directory and ignores `--workspace`, `KXM_WORKDIR`, `KXM_STATE_DIR` and `KXM_DATA_PATH`, so a relocated hub database is not found. It finds the Runtime stores under the user state root, including one moved with `KXM_STATE_HOME`.
88
88
 
89
89
  Preview first. A dry run lists what it would write, opens no store and writes nothing:
90
90
 
@@ -96,31 +96,40 @@ kxm backup --dry-run
96
96
  Expected output:
97
97
 
98
98
  ```text
99
- dry run: back up 1 store(s) to /srv/kxm/product/.kxm/backups/backup-2026-09-23T18-29-09-107Z (sources are not opened, so their WAL is not checkpointed)
99
+ dry run: back up 3 store(s) and 1 file(s) to /srv/kxm/product/.kxm/backups/backup-2026-09-23T18-29-09-107Z (sources are not opened, so their WAL is not checkpointed)
100
100
  would write /srv/kxm/product/.kxm/backups/backup-2026-09-23T18-29-09-107Z/kxm.db
101
+ would write /srv/kxm/product/.kxm/backups/backup-2026-09-23T18-29-09-107Z/registry.db
102
+ would write /srv/kxm/product/.kxm/backups/backup-2026-09-23T18-29-09-107Z/run-events.db
103
+ would write /srv/kxm/product/.kxm/backups/backup-2026-09-23T18-29-09-107Z/run-events.db.run-prompts.json
101
104
  would write /srv/kxm/product/.kxm/backups/backup-2026-09-23T18-29-09-107Z/manifest.json
102
105
  ```
103
106
 
104
107
  Then write the backup outside the checkout:
105
108
 
106
109
  ```bash
107
- kxm backup --out /backups/kxm/2026-09-23/hub
110
+ kxm backup --out /backups/kxm/2026-09-23/sqlite
108
111
  ```
109
112
 
110
113
  Expected output:
111
114
 
112
115
  ```text
113
- Created SQLite backup with 1 store(s):
116
+ Created SQLite backup with 3 store(s):
114
117
  - hub-store: /srv/kxm/product/.kxm/state/kxm.db -> kxm.db (schema v5, 110592 bytes, sha256 sha256:86549...)
115
- Manifest: /backups/kxm/2026-09-23/hub/manifest.json
118
+ - registry: /home/kxm/.local/state/kxm/runtime/registry.db -> registry.db (schema v1, 20480 bytes, sha256 sha256:a5bde...)
119
+ - events:38ed26cb8eeaa297f3b0b452: /home/kxm/.local/state/kxm/runtime/projects/38ed26cb8eeaa297f3b0b452/run-events.db -> run-events.db (schema v7, 348160 bytes, sha256 sha256:27c13...)
120
+ Manifest: /backups/kxm/2026-09-23/sqlite/manifest.json
116
121
  ```
117
122
 
118
- For each store, `kxm backup` checkpoints the write-ahead log, runs `PRAGMA integrity_check`, copies the database with `VACUUM INTO`, checks the copy's integrity, sets mode `0600`, and records its schema version and SHA-256 in `manifest.json` (`kxm.backup-manifest.v1`). `VACUUM INTO` reads one consistent snapshot, so the hub can keep running during this step.
123
+ The summary lists stores only; the prompt sidecar is in the manifest's `files`. When two projects' event stores share a file name, the second copy is prefixed with its store id.
124
+
125
+ For each store, `kxm backup` checkpoints the write-ahead log, runs `PRAGMA integrity_check`, copies the database with `VACUUM INTO`, checks the copy's integrity, sets mode `0600`, and records its schema version and SHA-256 in `manifest.json` (`kxm.backup-manifest.v1`). `VACUUM INTO` reads one consistent snapshot, so the hub and the Runtime can keep running during this step. Each prompt sidecar is copied as a regular file with mode `0600` and its SHA-256 recorded.
126
+
127
+ A backup is complete only when it copied everything it found. If a store or sidecar cannot be copied, or one appears while the backup runs, the manifest lists it under `omitted` and records `complete: false`, the command prints `Backup is incomplete (<n> omitted); not ok:` and exits 1, and `kxm restore` refuses that manifest.
119
128
 
120
129
  > [!NOTE]
121
130
  > Without `--out`, backups go to `.kxm/backups/` inside the checkout. Keep that directory out of Git, or always pass `--out`.
122
131
 
123
- It fails with `backup_no_stores` when `.kxm/state/kxm.db` does not exist under the current directory, and with `database_corrupted` when an integrity check fails.
132
+ It fails with `backup_no_stores` when it finds no store at all, neither `.kxm/state/kxm.db` under the current directory nor a Runtime store under the user state root, and with `database_corrupted` when an integrity check fails.
124
133
 
125
134
  ## Back up everything else
126
135
 
@@ -142,11 +151,11 @@ Copy the remaining state with both services stopped. SQLite runs in write-ahead-
142
151
  S="${KXM_STATE_HOME:-$HOME/.local/state/kxm}" # macOS: "$HOME/Library/Application Support/KXM"
143
152
  B=/backups/kxm/2026-09-23
144
153
  mkdir -p "$B"
145
- kxm backup --out "$B/hub"
154
+ kxm backup --out "$B/sqlite"
146
155
  tar -C "$S" --exclude 'supervisor.token' --exclude 'hub-env.json' -czf "$B/user-state.tgz" .
147
156
  ```
148
157
 
149
- The archive holds `registry.db`, every project's `run-events.db` with any `-wal` and `-shm` files and its `.run-prompts.json` sidecar, the repository bindings, the hub binding and `update.yaml`. It leaves out the credential file; keep tokens in your secret store, or include `hub-env.json` and protect the archive as a secret.
158
+ `kxm backup` already holds the Runtime stores and sidecars. The archive holds them again, as files, together with what `kxm backup` does not copy: the repository bindings, the hub binding and `update.yaml`. It leaves out the credential file; keep tokens in your secret store, or include `hub-env.json` and protect the archive as a secret.
150
159
 
151
160
  If `KXM_STATE_DIR` or `KXM_DATA_PATH` moved the hub database, `kxm backup` fails with `backup_no_stores`. With both services stopped, copy that database file and any `-wal` and `-shm` files instead.
152
161
  4. Copy the other roots your recovery needs: the checkout's untracked `.kxm/` records, `$D/assets/`, `$W/worker-*.json` (and `$W/pi-sessions/` only if your policy keeps model history), and `$C`.
@@ -157,40 +166,45 @@ Keep at least one previous backup, and bound retention: run events and prompt si
157
166
 
158
167
  ## Restore with `kxm restore`
159
168
 
160
- `kxm restore` overwrites the live database and deletes its `-wal` and `-shm` files, and it does not check whether the hub is running. Stop the hub and the Runtime first, and move the current state aside rather than deleting it.
169
+ `kxm restore` overwrites every live database in the manifest, including the Runtime stores of every project on the machine, and deletes their `-wal` and `-shm` files. It does not check whether the hub or the Runtime is running. Stop the hub and the Runtime first, and move the current state aside rather than deleting it.
161
170
 
162
171
  Preview the restore. The dry run performs every check below and lists what it would overwrite:
163
172
 
164
173
  ```bash
165
174
  cd /srv/kxm/product
166
- kxm restore /backups/kxm/2026-09-23/hub/manifest.json --dry-run
175
+ kxm restore /backups/kxm/2026-09-23/sqlite/manifest.json --dry-run
167
176
  ```
168
177
 
169
178
  Expected output:
170
179
 
171
180
  ```text
172
- dry run: restore 1 store(s) from /backups/kxm/2026-09-23/hub/manifest.json; digests verified against the manifest
181
+ dry run: restore 3 store(s) and 1 file(s) from /backups/kxm/2026-09-23/sqlite/manifest.json; digests verified against the manifest
173
182
  would write /srv/kxm/product/.kxm/state/kxm.db
174
183
  would delete /srv/kxm/product/.kxm/state/kxm.db-wal
175
184
  would delete /srv/kxm/product/.kxm/state/kxm.db-shm
185
+ would write /home/kxm/.local/state/kxm/runtime/registry.db
186
+ would write /home/kxm/.local/state/kxm/runtime/projects/38ed26cb8eeaa297f3b0b452/run-events.db
187
+ would write /home/kxm/.local/state/kxm/runtime/projects/38ed26cb8eeaa297f3b0b452/run-events.db.run-prompts.json
176
188
  ```
177
189
 
178
190
  Then run it without `--dry-run`:
179
191
 
180
192
  ```bash
181
- kxm restore /backups/kxm/2026-09-23/hub/manifest.json
193
+ kxm restore /backups/kxm/2026-09-23/sqlite/manifest.json
182
194
  ```
183
195
 
184
196
  Expected output:
185
197
 
186
198
  ```text
187
- Restored 1 SQLite store(s) from /backups/kxm/2026-09-23/hub/manifest.json:
199
+ Restored 3 SQLite store(s) from /backups/kxm/2026-09-23/sqlite/manifest.json:
188
200
  - hub-store: -> /srv/kxm/product/.kxm/state/kxm.db (schema v5, integrity ok)
201
+ - registry: -> /home/kxm/.local/state/kxm/runtime/registry.db (schema v1, integrity ok)
202
+ - events:38ed26cb8eeaa297f3b0b452: -> /home/kxm/.local/state/kxm/runtime/projects/38ed26cb8eeaa297f3b0b452/run-events.db (schema v7, integrity ok)
189
203
  ```
190
204
 
191
- Before it overwrites anything, `kxm restore` checks that the manifest is a `kxm.backup-manifest.v1` document, that every listed file exists, that each file's SHA-256 matches the manifest, and that no store is newer than this build supports. The manifest's own `manifestSha256` is not checked.
205
+ Before it overwrites anything, `kxm restore` checks that the manifest is a `kxm.backup-manifest.v1` document that does not record `complete: false` (`restore_incomplete`), that every listed file exists, that each file's SHA-256 matches the manifest, and that no store is newer than this build supports. The manifest's own `manifestSha256` is not checked. A manifest from an older build that has no `complete` field still restores.
192
206
 
193
- It then checks each backup's integrity and schema version again, copies it into place with mode `0600`, and checks the result. Each store returns to its recorded path, rebased onto the current directory when the manifest came from another checkout.
207
+ It then checks each backup's integrity and schema version again, copies it into place with mode `0600`, checks the result, and then copies the prompt sidecars back. A store under the checkout is rebased onto the current directory when the manifest came from another checkout. The Runtime stores and sidecars return to their recorded absolute paths under the user state root; restore does not move them to a different machine's or a different `KXM_STATE_HOME`.
194
208
 
195
209
  ### Restore ceilings
196
210
 
@@ -207,6 +221,8 @@ The ceilings come from `KXM_BACKUP_CEILINGS` in `plugins/kxm/src/database.ts` an
207
221
 
208
222
  ### Restore the Runtime stores by hand
209
223
 
224
+ `kxm restore` puts the Runtime stores back at the paths they came from. Use the archive instead when the user state root moved, or when you also need the bindings and `update.yaml`.
225
+
210
226
  1. Stop the Runtime and the hub, as in the backup procedure.
211
227
  2. Move `$S/runtime/registry.db` and `$S/runtime/projects/` aside.
212
228
  3. Extract the archive into the user state root:
@@ -232,8 +248,11 @@ Test a full restore on a spare machine before you rely on it, and repeat the tes
232
248
 
233
249
  | Symptom | Cause | Fix |
234
250
  |---|---|---|
235
- | `backup_no_stores` | No `.kxm/state/kxm.db` under the current directory, or it was moved with `KXM_STATE_DIR` or `KXM_DATA_PATH` | Run from the checkout root; copy a relocated database with the stopped-state procedure |
236
- | Runs are missing after a restore | `kxm backup` never contained the Runtime stores | Restore `$S/runtime/` from the stopped-state archive |
251
+ | `backup_no_stores` | No `.kxm/state/kxm.db` under the current directory (or it was moved with `KXM_STATE_DIR` or `KXM_DATA_PATH`) and no Runtime store under the user state root | Run from the checkout root; copy a relocated database with the stopped-state procedure |
252
+ | `Backup is incomplete (<n> omitted); not ok:`, exit 1 | A store or sidecar could not be copied, or appeared while the backup ran | Fix the source named after `omitted`, then run `kxm backup` again |
253
+ | `restore_incomplete` | The manifest records `complete: false` | Restore a complete backup; the partial one is not restorable |
254
+ | Runs are missing after a restore | The checkout moved to another absolute path, so the Runtime derives a different store key, or the user state root changed | Keep the checkout at its original path and restore under the same `KXM_STATE_HOME` |
255
+ | Another project's recent runs disappeared after a restore | `kxm restore` rolled back every project's Runtime store to the backup's moment | Restore that project's store from a newer backup or from the moved-aside copy |
237
256
  | `restore_manifest_digest_mismatch` | A backup file changed after the manifest was written | Use another backup; do not edit files in a backup set |
238
257
  | `restore_file_missing` | A file listed in the manifest is not beside it | Copy the whole backup directory, not only `manifest.json` |
239
258
  | `runtime_schema_mismatch` | A backup file's schema version differs from the one its manifest records | Use another backup set; never mix files between sets |
@@ -68,7 +68,7 @@ Expected output on first start:
68
68
  kxm hub: using newly generated KXM_AUTH_TOKEN from /home/kxm/.local/state/kxm/hub-env.json
69
69
  ```
70
70
 
71
- One-shot clients on the same machine (`kxm dash`, `kxm peer`, the Runtime) read the same file. They use `KXM_AUTH_TOKEN` from their environment first, then the saved project token for their project, then the saved admin token. The Claude Code MCP server never falls back to the admin token.
71
+ Operator clients on the same machine (`kxm dash`, the Runtime) read the same file. They use `KXM_AUTH_TOKEN` from their environment first, then the saved project token for their project, then the saved admin token. Agent clients (the Pi extension, the Claude Code MCP server, and the `kxm peer` and `kxm workflow` agent commands) never fall back to the admin token.
72
72
 
73
73
  To rotate the admin token, restart the hub with a new `KXM_AUTH_TOKEN`; the hub saves it in place of the old one and keeps the saved project tokens. Rotate a project token by restarting with an updated full `KXM_PROJECT_TOKENS` map. Then restart every client that held the old value. Deleting `hub-env.json` also works, but it drops the saved project tokens too.
74
74
 
@@ -237,7 +237,7 @@ flowchart LR
237
237
  Hold these invariants on every tenant box:
238
238
 
239
239
  1. **A service account, not root.** Session isolation routes model context; it is not a sandbox against a hostile process under the same account. Give untrusted workers separate accounts or containers.
240
- 2. **Explicit, stable paths.** Set `KXM_WORKSPACE_DIR`, `KXM_STATE_DIR`, `KXM_DATA_PATH`, `KXM_LOG_PATH` and `KXM_STATE_HOME` instead of inheriting a home directory, and pin `KXM_HOST=127.0.0.1`. `kxm backup` ignores `KXM_STATE_DIR` and `KXM_DATA_PATH`, so on such a box it finds no hub store and fails with `backup_no_stores`; use the [stopped-state backup](backup-and-restore.md#back-up-everything-else) instead.
240
+ 2. **Explicit, stable paths.** Set `KXM_WORKSPACE_DIR`, `KXM_STATE_DIR`, `KXM_DATA_PATH`, `KXM_LOG_PATH` and `KXM_STATE_HOME` instead of inheriting a home directory, and pin `KXM_HOST=127.0.0.1`. `kxm backup` ignores `KXM_STATE_DIR` and `KXM_DATA_PATH`, so on such a box it finds no hub store: it copies only the Runtime stores under `KXM_STATE_HOME`, or fails with `backup_no_stores` when there are none. Copy the hub database with the [stopped-state backup](backup-and-restore.md#back-up-everything-else).
241
241
  3. **Loopback listeners only.** Neither the hub nor the supervisor has a public port, and nothing is load-balanced across hubs.
242
242
  4. **Edge authentication.** The proxy authenticates browsers (for example with Authentik forward auth) on the portal's routes only. The hub never interprets browser identity; see [ADR-0004](../adr/ADR-0004-edge-identity-authentik.md).
243
243
  5. **Server-side machine credentials.** The portal backend keeps the hub credentials and calls the hub itself. `kxm tenant status`, which uses the admin token, gives it one labelled view of hub metadata and Runtime runs.
@@ -183,14 +183,15 @@ The hub does not poll GitHub. Run `kxm gate github watch` with the run's `--run-
183
183
 
184
184
  | Response | Meaning |
185
185
  |---|---|
186
- | 401 | The HMAC-SHA256 signature is missing or does not match the raw body; a callback must use `signalSecretEnv` when the definition sets it |
186
+ | 401 | The signature is missing, does not match, or is older than 300 seconds (`webhook_timestamp_expired`: check the sender's clock). Signals and `generic` starts need the [KXM sender contract](../guides/webhook-workflows.md#kxm-sender-contract); a callback must use `signalSecretEnv` when the definition sets it |
187
187
  | 400 | The delivery ID or JSON body is missing; `workflow_evidence_incomplete` names `missingRequirements`; `invalid_workflow_evidence` means non-string or duplicate keys |
188
188
  | 404 | The definition or run ID does not match this hub |
189
189
  | 409 `workflow_target_unavailable` | The configured coordinator has never registered; start it once with the matching project and name |
190
190
  | 409 `workflow_not_waiting` | No wait is active: `kxm_workflow_wait` was not called, the deadline failed the run, or a prior signal advanced it |
191
191
  | 409 `workflow_signal_mismatch`, `workflow_signal_context_mismatch` | Use the run's exact `waiting.signalKey`; fix or omit `workflow.run`, `workflow.stage` and `workflow.signal` evidence |
192
192
  | 204 | The event or filter did not match, so no run was intended |
193
- | 200 with `duplicate: true` | A retry of the same delivery ID was deduplicated |
193
+ | 409 `webhook_delivery_conflict`, `webhook_payload_replayed` | The delivery ID was reused with a different body, or a Jira or GitHub body already started a run under another delivery ID |
194
+ | 200 with `duplicate: true` | A retry of the same delivery ID and body was deduplicated |
194
195
 
195
196
  A failed or timed-out callback consumes that wait. Re-enter the wait and start a new `github watch` or `kxm gate signal`; keep an explicit `--delivery-id` only for retries of one unchanged body. See [Run webhook workflows](../guides/webhook-workflows.md).
196
197