@kontextmind/kxm 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (227) hide show
  1. package/.claude-plugin/marketplace.json +19 -0
  2. package/.kxm/README.md +14 -0
  3. package/.kxm/assets/README.md +5 -0
  4. package/.kxm/assets/retrospectives/README.md +5 -0
  5. package/.kxm/config/README.md +5 -0
  6. package/.kxm/config/agents.json +43 -0
  7. package/.kxm/config/env.example +56 -0
  8. package/.kxm/config/update.example.yaml +9 -0
  9. package/.kxm/config/workflows/fix.json +160 -0
  10. package/.kxm/config/workflows/jira-development.json +116 -0
  11. package/.kxm/config/workflows/provenance-quorum.json +150 -0
  12. package/.kxm/config/workflows/v04-dogfood.json +72 -0
  13. package/CHANGELOG.md +465 -0
  14. package/LICENSE +21 -0
  15. package/README.md +306 -0
  16. package/SECURITY.md +72 -0
  17. package/docs/README.md +48 -0
  18. package/docs/agent-communication-envelopes-and-gates.md +553 -0
  19. package/docs/architecture.md +242 -0
  20. package/docs/assignment-runner.md +241 -0
  21. package/docs/configuration.md +361 -0
  22. package/docs/continuous-improvement.md +114 -0
  23. package/docs/getting-started.md +253 -0
  24. package/docs/kxm-handbook.md +1090 -0
  25. package/docs/operations.md +205 -0
  26. package/docs/provenance-gates.md +291 -0
  27. package/docs/skills.md +45 -0
  28. package/docs/templates/README.md +95 -0
  29. package/docs/templates/adr.md +88 -0
  30. package/docs/templates/architecture.md +120 -0
  31. package/docs/templates/bug-fix.md +109 -0
  32. package/docs/templates/feature.md +108 -0
  33. package/docs/templates/handoff.md +72 -0
  34. package/docs/templates/postmortem.md +77 -0
  35. package/docs/templates/research.md +100 -0
  36. package/docs/templates/review.md +85 -0
  37. package/docs/templates/runbook.md +73 -0
  38. package/docs/templates/test-plan.md +87 -0
  39. package/docs/templates/test-report.md +72 -0
  40. package/docs/test-matrix.md +121 -0
  41. package/docs/troubleshooting.md +249 -0
  42. package/docs/vnext/README.md +62 -0
  43. package/docs/vnext/architecture.md +185 -0
  44. package/docs/vnext/effects-and-recovery.md +172 -0
  45. package/docs/vnext/lifecycles.md +235 -0
  46. package/docs/vnext/migration.md +220 -0
  47. package/docs/vnext/routing.md +184 -0
  48. package/docs/vnext/synchronization.md +172 -0
  49. package/docs/vnext/terminology.md +240 -0
  50. package/docs/vnext/validation.md +335 -0
  51. package/docs/webhook-workflows.md +240 -0
  52. package/docs/workflow-guide.md +1150 -0
  53. package/examples/README.md +102 -0
  54. package/examples/provenance-workflow.json +40 -0
  55. package/examples/requester.ts +30 -0
  56. package/examples/reviewer-agent.ts +29 -0
  57. package/examples/roundtrip.ts +46 -0
  58. package/examples/vnext/.kxm/agents/coordinator.yaml +16 -0
  59. package/examples/vnext/.kxm/agents/critic-1.yaml +16 -0
  60. package/examples/vnext/.kxm/agents/critic-2.yaml +15 -0
  61. package/examples/vnext/.kxm/agents/critic-3.yaml +15 -0
  62. package/examples/vnext/.kxm/agents/implementer.yaml +15 -0
  63. package/examples/vnext/.kxm/agents/planner.yaml +13 -0
  64. package/examples/vnext/.kxm/agents/reproducer.yaml +15 -0
  65. package/examples/vnext/.kxm/agents/reviewer.yaml +15 -0
  66. package/examples/vnext/.kxm/gates.yaml +8 -0
  67. package/examples/vnext/.kxm/models/critic-claude.yaml +11 -0
  68. package/examples/vnext/.kxm/models/critic-gemini.yaml +11 -0
  69. package/examples/vnext/.kxm/models/critic-grok.yaml +12 -0
  70. package/examples/vnext/.kxm/models/implementation.yaml +14 -0
  71. package/examples/vnext/.kxm/models/primary.yaml +17 -0
  72. package/examples/vnext/.kxm/prices.yaml +111 -0
  73. package/examples/vnext/.kxm/project/env.yaml +7 -0
  74. package/examples/vnext/.kxm/project.yaml +32 -0
  75. package/examples/vnext/.kxm/repo/repo.yaml +8 -0
  76. package/examples/vnext/.kxm/workflows/default.yaml +92 -0
  77. package/examples/vnext/.kxm/workflows/fix.yaml +376 -0
  78. package/examples/vnext/.kxm/workflows/improve.yaml +57 -0
  79. package/examples/vnext/README.md +53 -0
  80. package/examples/vnext/records/assignment-result-recorded.json +63 -0
  81. package/examples/vnext/records/assignment-result.json +46 -0
  82. package/examples/vnext/records/context-candidate.json +42 -0
  83. package/examples/vnext/records/delivery-manifest.json +66 -0
  84. package/examples/vnext/records/effect-uncertainty-resolved-sync.json +67 -0
  85. package/examples/vnext/records/effect-uncertainty-resolved.json +62 -0
  86. package/examples/vnext/records/run-created.json +54 -0
  87. package/examples/vnext/records/sync-event.json +65 -0
  88. package/examples/vnext/repositories/api/.kxm/repo/env.yaml +7 -0
  89. package/examples/vnext/repositories/api/.kxm/repo/repo.yaml +8 -0
  90. package/examples/vnext/repositories/web/.kxm/repo/repo.yaml +8 -0
  91. package/examples/workflow-signal.ts +63 -0
  92. package/package.json +129 -0
  93. package/plugins/kxm/.claude-plugin/plugin.json +73 -0
  94. package/plugins/kxm/.mcp.json +19 -0
  95. package/plugins/kxm/README.md +93 -0
  96. package/plugins/kxm/dist/cli.js +42853 -0
  97. package/plugins/kxm/dist/client.js +416 -0
  98. package/plugins/kxm/dist/core.js +1823 -0
  99. package/plugins/kxm/dist/extension.js +3797 -0
  100. package/plugins/kxm/dist/mcp-server.js +17104 -0
  101. package/plugins/kxm/dist/runtime.js +23361 -0
  102. package/plugins/kxm/dist/server.js +13640 -0
  103. package/plugins/kxm/dist/vnext-runtime-supervisor.js +21109 -0
  104. package/plugins/kxm/package.json +12 -0
  105. package/plugins/kxm/skills/kxm/SKILL.md +97 -0
  106. package/plugins/kxm/skills/kxm/references/protocol.md +103 -0
  107. package/plugins/kxm/skills/kxm-session/SKILL.md +53 -0
  108. package/plugins/kxm/src/arbiter.ts +355 -0
  109. package/plugins/kxm/src/artifacts-exist.ts +62 -0
  110. package/plugins/kxm/src/autocomplete.ts +236 -0
  111. package/plugins/kxm/src/cli.ts +3707 -0
  112. package/plugins/kxm/src/client.ts +614 -0
  113. package/plugins/kxm/src/commands.ts +1063 -0
  114. package/plugins/kxm/src/config.ts +290 -0
  115. package/plugins/kxm/src/context/providers.ts +101 -0
  116. package/plugins/kxm/src/context-packet.ts +332 -0
  117. package/plugins/kxm/src/context.ts +499 -0
  118. package/plugins/kxm/src/core.ts +6 -0
  119. package/plugins/kxm/src/database.ts +563 -0
  120. package/plugins/kxm/src/diagnostics.ts +184 -0
  121. package/plugins/kxm/src/envelope.ts +118 -0
  122. package/plugins/kxm/src/extension.ts +895 -0
  123. package/plugins/kxm/src/external-effects.ts +299 -0
  124. package/plugins/kxm/src/github-watch.ts +255 -0
  125. package/plugins/kxm/src/hub-binding.ts +160 -0
  126. package/plugins/kxm/src/hub.ts +2502 -0
  127. package/plugins/kxm/src/improve.ts +383 -0
  128. package/plugins/kxm/src/inbox.ts +10 -0
  129. package/plugins/kxm/src/kxm-install-kind.ts +113 -0
  130. package/plugins/kxm/src/kxm-update-config.ts +39 -0
  131. package/plugins/kxm/src/kxm-update.ts +238 -0
  132. package/plugins/kxm/src/local-snapshot.ts +406 -0
  133. package/plugins/kxm/src/logger.ts +198 -0
  134. package/plugins/kxm/src/mcp-server.ts +143 -0
  135. package/plugins/kxm/src/memory.ts +385 -0
  136. package/plugins/kxm/src/nous-pi.ts +287 -0
  137. package/plugins/kxm/src/nous-provider.ts +729 -0
  138. package/plugins/kxm/src/price-calc.ts +87 -0
  139. package/plugins/kxm/src/prices.ts +121 -0
  140. package/plugins/kxm/src/protocol.ts +172 -0
  141. package/plugins/kxm/src/recovery.ts +211 -0
  142. package/plugins/kxm/src/redact.ts +26 -0
  143. package/plugins/kxm/src/retrospective.ts +400 -0
  144. package/plugins/kxm/src/routing.ts +830 -0
  145. package/plugins/kxm/src/runtime.ts +9 -0
  146. package/plugins/kxm/src/server.ts +117 -0
  147. package/plugins/kxm/src/session-work.ts +571 -0
  148. package/plugins/kxm/src/session.ts +184 -0
  149. package/plugins/kxm/src/skills.ts +535 -0
  150. package/plugins/kxm/src/state.ts +326 -0
  151. package/plugins/kxm/src/store.ts +637 -0
  152. package/plugins/kxm/src/studio-layout.ts +268 -0
  153. package/plugins/kxm/src/suggest.ts +162 -0
  154. package/plugins/kxm/src/task-manager.ts +244 -0
  155. package/plugins/kxm/src/telemetry.ts +116 -0
  156. package/plugins/kxm/src/tui.ts +1046 -0
  157. package/plugins/kxm/src/vnext-bindings.ts +403 -0
  158. package/plugins/kxm/src/vnext-config.ts +1646 -0
  159. package/plugins/kxm/src/vnext-engine-artifacts.ts +86 -0
  160. package/plugins/kxm/src/vnext-engine-command.ts +533 -0
  161. package/plugins/kxm/src/vnext-engine-compile.ts +722 -0
  162. package/plugins/kxm/src/vnext-engine-evidence.ts +273 -0
  163. package/plugins/kxm/src/vnext-engine-fold.ts +1400 -0
  164. package/plugins/kxm/src/vnext-engine-gate-records.ts +583 -0
  165. package/plugins/kxm/src/vnext-engine-plan.ts +717 -0
  166. package/plugins/kxm/src/vnext-engine.ts +2458 -0
  167. package/plugins/kxm/src/vnext-gate-hash.ts +10 -0
  168. package/plugins/kxm/src/vnext-harness.ts +1142 -0
  169. package/plugins/kxm/src/vnext-init.ts +430 -0
  170. package/plugins/kxm/src/vnext-migrate.ts +1848 -0
  171. package/plugins/kxm/src/vnext-oneshot-producer.ts +424 -0
  172. package/plugins/kxm/src/vnext-permission.ts +936 -0
  173. package/plugins/kxm/src/vnext-pi-producer.ts +628 -0
  174. package/plugins/kxm/src/vnext-repair.ts +1094 -0
  175. package/plugins/kxm/src/vnext-runtime-owner.ts +320 -0
  176. package/plugins/kxm/src/vnext-runtime-store.ts +1560 -0
  177. package/plugins/kxm/src/vnext-runtime-supervisor.ts +586 -0
  178. package/plugins/kxm/src/vnext-runtime.ts +663 -0
  179. package/plugins/kxm/src/vnext-template.ts +247 -0
  180. package/plugins/kxm/src/wiki.ts +313 -0
  181. package/plugins/kxm/src/workflow.ts +1548 -0
  182. package/schemas/vnext/README.md +46 -0
  183. package/schemas/vnext/agent.schema.json +40 -0
  184. package/schemas/vnext/assignment-result.schema.json +66 -0
  185. package/schemas/vnext/backup-manifest.schema.json +89 -0
  186. package/schemas/vnext/candidate.schema.json +109 -0
  187. package/schemas/vnext/common.schema.json +422 -0
  188. package/schemas/vnext/context-candidate.schema.json +76 -0
  189. package/schemas/vnext/context-packet.schema.json +192 -0
  190. package/schemas/vnext/delivery-manifest.schema.json +159 -0
  191. package/schemas/vnext/environment.schema.json +66 -0
  192. package/schemas/vnext/gate-registry.schema.json +109 -0
  193. package/schemas/vnext/handoff-manifest.schema.json +146 -0
  194. package/schemas/vnext/init-operation.schema.json +61 -0
  195. package/schemas/vnext/local-repository-bindings.schema.json +30 -0
  196. package/schemas/vnext/memory-record.schema.json +45 -0
  197. package/schemas/vnext/migration-decision.schema.json +26 -0
  198. package/schemas/vnext/migration-plan.schema.json +123 -0
  199. package/schemas/vnext/migration-receipt.schema.json +52 -0
  200. package/schemas/vnext/model.schema.json +42 -0
  201. package/schemas/vnext/permission-diff.schema.json +57 -0
  202. package/schemas/vnext/prices.schema.json +115 -0
  203. package/schemas/vnext/project.schema.json +85 -0
  204. package/schemas/vnext/repository.schema.json +24 -0
  205. package/schemas/vnext/run-event.schema.json +460 -0
  206. package/schemas/vnext/session-brief.schema.json +153 -0
  207. package/schemas/vnext/sync-event.schema.json +234 -0
  208. package/schemas/vnext/template-provenance.schema.json +38 -0
  209. package/schemas/vnext/workflow.schema.json +248 -0
  210. package/scripts/assignment-run.d.mts +354 -0
  211. package/scripts/assignment-run.mjs +4451 -0
  212. package/scripts/build-runtime.mjs +56 -0
  213. package/scripts/check-generated.mjs +77 -0
  214. package/scripts/check-versions.mjs +34 -0
  215. package/scripts/emit-codex-artifacts.d.mts +9 -0
  216. package/scripts/emit-codex-artifacts.mjs +91 -0
  217. package/scripts/harness-run.d.mts +83 -0
  218. package/scripts/harness-run.mjs +2095 -0
  219. package/scripts/kxm-hub.mjs +105 -0
  220. package/scripts/kxm-publish-npm.mjs +327 -0
  221. package/scripts/kxm-release-github.mjs +472 -0
  222. package/scripts/kxm-runtime-supervisor.mjs +7 -0
  223. package/scripts/kxm-worker.mjs +1127 -0
  224. package/scripts/kxm.mjs +27 -0
  225. package/scripts/roster-policy.d.mts +20 -0
  226. package/scripts/roster-policy.mjs +161 -0
  227. package/scripts/smoke-multi-pi.mjs +479 -0
@@ -0,0 +1,249 @@
1
+ # Troubleshooting
2
+
3
+ Start with the smallest boundary: hub health, authentication, registration, peer discovery, then message delivery.
4
+
5
+ ## Quick diagnostic sequence
6
+
7
+ 1. Confirm the hub terminal still shows `kxm hub listening`.
8
+ 2. Request `/health`, then `/ready` to confirm storage access.
9
+ 3. Compare the hub URL, token, and project on both agents.
10
+ 4. Confirm every agent has a unique name.
11
+ 5. Run `/kxm hub` in Pi or call `kxm_list` in Claude.
12
+ 6. Inspect hub logs for registration, stale-agent, or server-error events.
13
+ 7. If a workflow tool returns `workflow_forbidden`, read `operation`, `assignedCoordinatorName`, and `nextAction`. Do not retry as a peer.
14
+
15
+ ## Common problems
16
+
17
+ ### A continued Pi session rejects every turn
18
+
19
+ If a worker was stopped during `kxm_await`, `--continue` may leave a `tool_use` without `tool_result`. The worker retries once without `--continue` and writes a project-and-agent identity-keyed recovery envelope under `.kxm/state`. Do not paste agent logs into the journal. Keep the same project and agent name so the hub identity and recovery key resume.
20
+
21
+ ### A model quota or provider error settles the agent
22
+
23
+ KXM waits until Pi has exhausted its own automatic retries. It then keeps the inbound message in `delivered` state, records an allowlisted `quota` or `provider_error` diagnostic without the provider body, and restarts the RPC child. Configure `KXM_WORKER_FALLBACK_MODELS` (or `--fallback-models`) to rotate immediately; otherwise the worker retries after `KXM_WORKER_PROVIDER_RETRY_MS`. Keep continuation enabled so finished peer calls and tool results survive the model switch. Use `--fresh-start`, not `--no-continue`, when only the first launch must avoid old session state.
24
+
25
+ ### A worker heartbeat is healthy but one tool never finishes
26
+
27
+ Set `KXM_WORKER_TOOL_TIMEOUT_MS` above the longest legitimate tool call. Its 31-minute default intentionally gives a 30-minute `kxm_await` or `kxm_fanout` time to return durable pending handles before supervision intervenes. When that bound is exceeded, the structured worker log records `worker_tool_timeout` with only the allowlisted tool name and diagnostic class, the delivered hub request stays recoverable, and the RPC process is restarted. If the stuck worker was supposed to be read-only, also set `KXM_WORKER_TOOLS=read,grep,find,ls`; prompt wording alone does not remove shell or write capabilities.
28
+
29
+ ### A hub or worker PID claim is stale
30
+
31
+ Version 0.4.3 prevents a second wrapper from replacing a live hub or worker claim. `kxm hub stop` ignores an invalid, non-running, or ownership-mismatched record rather than guessing. If a crash or pre-0.4.3 process left one behind, inspect the exact `.pid` JSON and verify that its recorded PID is no longer running; for a hub, also verify the configured port has no listener. Then remove only that exact `.pid` and its recorded `.stop` control file before relaunching once. Worker filenames include a project/agent identity digest and their records include the exact names and generation, so do not substitute a similarly sanitized filename. Never delete the `.kxm/state` directory or SQLite database to clear a claim.
32
+
33
+ ### GitHub checks passed but the workflow is still waiting
34
+
35
+ The hub does not poll GitHub. Run `kxm gate github watch` with the same `runId`, `stageId`, and `signalKey`. A watcher timeout posts the exact signed `failed` signal, retains bounded check evidence, and exits `4`; it never invents `passed`.
36
+
37
+ ### The hub refuses to start
38
+
39
+ **`KXM_PORT must be an integer between 0 and 65535`**
40
+
41
+ Set `KXM_PORT` to a valid integer. Remove the variable to use `7331`.
42
+
43
+ **`KXM_AUTH_TOKEN is required when binding beyond localhost`**
44
+
45
+ Either restore `KXM_HOST=127.0.0.1` or configure a token before using a non-loopback interface.
46
+
47
+ #### Database schema is newer than this runtime supports
48
+
49
+ Do not delete or rewrite the database. Start the package version that created it, or upgrade this runtime. Restore the pre-upgrade backup when rolling back.
50
+
51
+ #### Address already in use
52
+
53
+ Another process owns the port. Stop that process or choose another port, then update every agent's `KXM_SERVER_URL`.
54
+
55
+ ### Pi shows `hub:off`
56
+
57
+ - Confirm the hub is reachable from the Pi terminal.
58
+ - Verify `KXM_AUTH_TOKEN` exactly matches the hub token.
59
+ - Check whether a live agent already uses the same name in the same project.
60
+ - Restart Pi after changing environment variables.
61
+ - For an exact development load, use `pi --no-extensions -e ./plugins/kxm/src/extension.ts`. Add every required provider extension with another `-e`; otherwise Pi discovery is intentionally disabled.
62
+ - For long-lived workers, set the reviewed `KXM_WORKER_EXTENSION_PATHS` and `KXM_WORKER_SKILL_PATHS` described in [Configuration](configuration.md#long-lived-worker-settings). Invalid paths fail before supervision instead of entering a restart loop.
63
+
64
+ ### Pi update fails looking for `refs/heads/master`
65
+
66
+ The KXM default branch is `main`. An older Pi git checkout still tracking
67
+ `master` fails with `couldn't find remote ref refs/heads/master`. Remove the
68
+ package and reinstall with an explicit ref:
69
+
70
+ ```text
71
+ pi remove git:github.com/kontextmind/kxm
72
+ pi install git:github.com/kontextmind/kxm@main
73
+ ```
74
+
75
+ ### `kxm --help` prints a former flat command list
76
+
77
+ If the installed `kxm --help` prints `validate | status | hub | worker | stop | …`
78
+ instead of the current Commander groups, the committed `plugins/kxm/dist/cli.js`
79
+ is stale. Run `npm run build` and commit the generated `dist` so the operator
80
+ CLI matches source.
81
+
82
+ ### `kxm` is not recognized
83
+
84
+ `pi install git:github.com/kontextmind/kxm@main` installs the Pi extension
85
+ and Agent Skill, not a global operator command. Install the versioned `.tgz`
86
+ release asset through the authenticated `gh release download` flow in
87
+ [Getting started](getting-started.md#install-the-operator-command), or run
88
+ `node scripts/kxm.mjs` from a clone after `npm ci`. `npx kxm` and a
89
+ global `git+https` npm install are not supported installation paths.
90
+
91
+ ### An expected peer is missing
92
+
93
+ The two agents usually have different `KXM_PROJECT` values or one stopped sending heartbeats. Compare settings and check for an `agent_stale` event. Names and projects are case-sensitive for display; live-name uniqueness is case-insensitive.
94
+
95
+ ### A request stays `queued`
96
+
97
+ The recipient registered but has no active SSE stream. Confirm its process is running and connected. Proxies must disable response buffering for `/v1/events` and allow long-lived connections.
98
+
99
+ ### A request stays `delivered`
100
+
101
+ The recipient acknowledged it but has not replied. It may still be working, waiting for approval, or blocked. Avoid sending the same request repeatedly. Check the recipient session directly if the wait is unexpected.
102
+
103
+ If the work is obsolete, the sender can call `kxm_cancel`. This changes hub state only; it cannot reverse file changes or external effects already performed by the peer.
104
+
105
+ ### `kxm_await` times out
106
+
107
+ The default timeout is 30 minutes. Use `kxm_get` to inspect the state. `cancelled`, `expired`, and `error` are terminal outcomes. Resend only when the task is safe to repeat, and use an idempotency key when retrying after an uncertain network result.
108
+
109
+ ### A message disappears after completion
110
+
111
+ Terminal records are removed after seven days by default. Increase `KXM_MESSAGE_RETENTION_MS` if operators need a longer diagnostic window. Durable artifacts should live in Git or another system of record.
112
+
113
+ ### Claude tools do not appear
114
+
115
+ 1. Confirm the marketplace and plugin are installed.
116
+ 2. Run `/reload-plugins` or restart Claude Code.
117
+ 3. Inspect `/mcp` and verify the `kxm` server connected.
118
+ 4. Confirm Node.js 22.19 or newer on the 22.x line, or Node.js 24 or newer, is on the `PATH` used by Claude Code.
119
+ 5. Reinstall or update the marketplace if the cached plugin predates the `dist/mcp-server.js` bundle.
120
+
121
+ ### Claude does not receive pushed requests
122
+
123
+ Ordinary MCP tools and channel delivery are separate. During the research preview, start the community channel explicitly:
124
+
125
+ ```text
126
+ claude --dangerously-load-development-channels plugin:kxm@kxm
127
+ ```
128
+
129
+ Accept the trust prompt and check the channel startup notice. Organization policy can still block channels. If pushed delivery remains unavailable, use `kxm_inbox` and `kxm_reply`.
130
+
131
+ ### Jira webhook is rejected
132
+
133
+ - HTTP 401 means the SHA-256 signature is missing, uses another algorithm, or does not match the raw UTF-8 body. Confirm Jira and `secretEnv` resolve the same secret.
134
+ - HTTP 400 usually means the delivery identifier or JSON body is missing.
135
+ - HTTP 409 means the configured coordinator has never registered. Start it once with the matching project and name; Jira retries 409 responses.
136
+ - HTTP 204 means the event or JSON-path filter did not match, so no workflow was intended.
137
+ - HTTP 200 with `duplicate: true` means a provider retry was safely deduplicated.
138
+
139
+ ### Long-lived worker keeps restarting
140
+
141
+ Inspect the structured `worker_process_error` and `worker_exited` events. Confirm Pi is installed on the service account's `PATH`, the working directory exists, model credentials are available, the package is enabled, and non-interactive project trust was configured intentionally. Set `KXM_PI_COMMAND` to an explicit executable path when service-manager environments have a reduced `PATH`.
142
+
143
+ ### A workflow message stays queued while the worker restarts once
144
+
145
+ This is normally the safe session-routing handshake. With `--session-isolation workflow`, a message for a different run is deliberately not acknowledged in the current Pi context. Look for `worker_session_routed`; the old child must close before one replacement starts with the run-specific `--session-dir`, after which the same message ID replays and advances to `delivered`.
146
+
147
+ If it repeats, inspect `worker_session_request_rejected` and verify:
148
+
149
+ - the worker was started through `kxm agent worker` with a valid state directory;
150
+ - `KXM_WORKER_SESSION_SCOPE` was not manually set (the supervisor owns it);
151
+ - the state directory is writable by only the service account;
152
+ - the hub and worker are from the same release; and
153
+ - the message has a canonical hub-owned `workflowRunId`, not only a correlation ID.
154
+
155
+ Do not manually acknowledge the message, edit the route request, copy a run JSONL into `default`, or launch a second worker with the same identity. Those actions defeat context isolation.
156
+
157
+ ### `worker_session_state_recovered` appears
158
+
159
+ The binding manifest did not match its bounded schema or exact worker owner. The supervisor renamed it to `worker-session-binding-<workerKey>.json.corrupt-<timestamp>` and started the stable default binding rather than guessing a workflow. Read `kxm_workflow_get` for unfinished stages and inspect queued/delivered message IDs. Preserve the quarantined manifest for diagnosis, then re-drive unfinished work from the hub. Repeated corruption suggests disk, antivirus, concurrent-service, or permission problems; confirm only one supervisor owns the exact project/agent PID claim.
160
+
161
+ ### A workflow seems to remember another run
162
+
163
+ Confirm the worker log says `"sessionIsolation":"workflow"` and the Pi child has a `runs/<exact-runId>` session directory. Isolation is opt-in for upgrade compatibility, and both the CLI and raw supervisor default to `off`. Restart cleanly with `kxm agent worker ... --session-isolation workflow`. The first isolated start intentionally uses fresh scoped storage because KXM cannot safely infer which session in the former shared Pi directory belonged to this worker. Existing content created in a formerly shared Pi session cannot be automatically separated retroactively; treat authoritative workflow journal/assets as the recovery source and start a fresh run-specific history.
164
+
165
+ ### Fanout returns pending before a model replies
166
+
167
+ `kxm_fanout.timeoutMs` is a local wait, not the message lifetime. A pending result includes the durable `messageId`, current message status, expiry, and whether the wait timed out or was aborted. Use `kxm_get` to inspect that ID, or repeat the exact fanout with the same correlation ID, idempotency prefix, targets, and content. Do not send a replacement with a new prefix while the original remains pending. Normally omit `ttlMs` for model work so time spent queued behind another request does not prematurely expire it. A pending peer has not contributed review or planning evidence and must not be counted toward a workflow checkpoint.
168
+
169
+ ### Workflow cannot advance
170
+
171
+ Call `kxm_workflow_get` and use only `currentStage`. A passing checkpoint needs
172
+ a keyed, non-empty value for every declared `requiredEvidence` identity; extra
173
+ or unrelated keys do not count. Warnings and failures remain active until
174
+ corrected, and their evidence is journaled but does not satisfy a later passing
175
+ attempt. If attempts are exhausted or the coordinator settles early, the run
176
+ becomes failed and its journal records the reason; start a new provider delivery
177
+ only after deciding whether repeating external effects is safe.
178
+
179
+ For a requirement with `kind: peer-reply`, inspect
180
+ `resolvedEvidencePolicies`, `verifiedEvidence`, and the current attempt. An
181
+ ordinary evidence string cannot satisfy it. Every eligible agent must have
182
+ registered in the workflow project before the run starts, and a passing
183
+ checkpoint must cite durable replied message IDs in `evidenceRefs` before those
184
+ source messages reach terminal retention.
185
+
186
+ Common provenance failures are:
187
+
188
+ - `workflow_context_forbidden`: the sender is not the run's assigned coordinator;
189
+ - `workflow_context_inactive`: the run or stage is not currently running;
190
+ - `workflow_context_attempt_mismatch`: use `stage.attempts + 1` and send fresh work after a retry;
191
+ - `workflow_evidence_producer_forbidden`: the target is not in the run's snapshotted eligible set;
192
+ - `workflow_evidence_policy_missing` or `workflow_evidence_policy_unresolved`: the requirement has no usable resolved peer policy;
193
+ - `workflow_provenance_invalid`: a cited message is missing, pending, ineligible, wrong-direction, or bound to another project, run, stage, requirement, or attempt;
194
+ - `workflow_evidence_incomplete`: there are fewer unique verified producers than the effective minimum.
195
+
196
+ Multiple replied messages from one peer count once. Correlation IDs and
197
+ idempotency prefixes are retry controls, not provenance. Do not replace a
198
+ rejected reference with an unscoped send.
199
+
200
+ If policy declares a lower `degradation.minProducers`, an operator can inspect
201
+ and approve it with `kxm gate --dry-run --json degrade ...` followed by
202
+ the same command without `--dry-run`, using the administrative token. Approval
203
+ must target the current stage and attempt and does not advance the workflow;
204
+ the coordinator must still checkpoint with enough verified references. A
205
+ callback, project token, or peer cannot approve degradation.
206
+
207
+ ### `kxm gate degrade` returns HTTP 503 `admin_auth_not_configured`
208
+
209
+ The hub started without a non-empty `KXM_AUTH_TOKEN`, so no administrative
210
+ credential exists for the degradation route. Project tokens deliberately cannot
211
+ substitute for it, even when the operator holds every project credential. The
212
+ route fails closed and does not create an approval.
213
+
214
+ Stop the hub gracefully, set a new high-entropy `KXM_AUTH_TOKEN` in the hub
215
+ service, retain the explicit `KXM_PROJECT_TOKENS` mapping for workers, and
216
+ restart against the same `.kxm/state/kxm.db`. Give the administrative token
217
+ only to the operator terminal, never to agents or callbacks. Read the run again
218
+ because the current attempt may have changed, run the exact degradation command
219
+ with `--dry-run --json`, and then approve the current stage, requirement, and
220
+ attempt without `--dry-run`. A restart does not make an earlier-attempt approval
221
+ valid for the new attempt.
222
+
223
+ ### External workflow callback is rejected or does not resume
224
+
225
+ - HTTP 401 means the callback signature does not match the exact raw body. Use `signalSecretEnv` when configured; the workflow-start secret will not work in that case.
226
+ - HTTP 404 means the workflow definition or run ID does not match this hub.
227
+ - HTTP 409 with `workflow_not_waiting` means the coordinator did not successfully call `kxm_workflow_wait`, the deadline already failed the run, or a prior signal advanced it.
228
+ - HTTP 409 with `workflow_signal_mismatch` means the URL's signal key differs from the active wait. Read the run and use its exact `waiting.signalKey`.
229
+ - HTTP 409 with `workflow_signal_context_mismatch` means a supplied `workflow.run`, `workflow.stage`, or `workflow.signal` evidence value disagrees with the route or active wait. Correct it or omit optional context evidence.
230
+ - HTTP 400 with `workflow_evidence_incomplete` means a passing callback omitted one or more named requirements. Read `missingRequirements`; extra checks and context fields cannot substitute for them.
231
+ - HTTP 400 with `invalid_workflow_evidence` means evidence was not a keyed string object or contained duplicate keys after case/whitespace normalization.
232
+ - HTTP 200 with `duplicate: true` is expected after retrying the same provider delivery ID. Do not generate a new ID for the same callback attempt.
233
+ - A failed or timed-out callback consumes that wait attempt. Re-enter the wait and start a new `github watch` or `signal` command so its default delivery generation is new; reserve an explicit `--delivery-id` for retries of one unchanged callback body.
234
+
235
+ Inspect `workflow_wait_started`, `workflow_signal_received`, and `workflow_wait_timed_out` logs without copying secrets or full callback bodies. If a run timed out, review whether the external action completed before starting a replacement workflow.
236
+
237
+ ## Collecting a useful bug report
238
+
239
+ Include:
240
+
241
+ - operating system and Node.js version;
242
+ - Pi or Claude Code version;
243
+ - package version or Git commit;
244
+ - whether the hub is local or behind a proxy;
245
+ - redacted environment values, excluding the token;
246
+ - the relevant structured hub events;
247
+ - exact reproduction steps and expected behavior.
248
+
249
+ Never attach authentication tokens, private prompts, credentials, or unrelated repository contents.
@@ -0,0 +1,62 @@
1
+ # KXM vNext contract package
2
+
3
+ > **Status: planned normative contract.** This directory describes the target
4
+ > architecture accepted for KXM vNext. Not all commands are implemented.
5
+ > Phase 1 (init/migrate/trust) and Phase 2 (Runtime create/recover) have landed
6
+ > slices. Phase 3 has D3 S1–S4 and D4 U2a-2 implemented (unreleased); it is not
7
+ > only an agent-only simulated loop, and the default/fix driver gate remains
8
+ > open. Operator tracking for the KXM rename, `kxm dash`, hub CLI, and
9
+ > harness YAML lives in the
10
+ > [implementation plan](../../plans/implementation-plan.md#tracking-working-tree-not-a-release).
11
+ > For current hub execution behavior, use [Architecture](../architecture.md) and
12
+ > [Configuration](../configuration.md).
13
+
14
+ KXM vNext is a convention-over-configuration, local-first orchestration and
15
+ context platform. One local Runtime owns execution; an optional multi-project
16
+ hub coordinates requests, synchronized facts, and aggregate views.
17
+
18
+ The words **MUST**, **MUST NOT**, **SHOULD**, and **MAY** are normative.
19
+
20
+ ## Package contents
21
+
22
+ | Contract | Purpose |
23
+ |---|---|
24
+ | [Architecture decision](architecture.md) | Authority, component boundaries, locality, and rejected alternatives |
25
+ | [Terminology](terminology.md) | Canonical names and identity hierarchy |
26
+ | [Lifecycles](lifecycles.md) | Run, step, assignment, attempt, effect, delivery, and synchronization states |
27
+ | [Effects and recovery](effects-and-recovery.md) | Retry, reconciliation, reattachment, and `blocked_uncertain` rules |
28
+ | [Synchronization](synchronization.md) | Sync-safe allowlist and pre-outbox redaction (Phase 8 implementation) |
29
+ | [Routing](routing.md) | Shipped v1 parser/report vs helper telemetry vs planned v2/catalog |
30
+ | [Validation](validation.md) | Parse, schema, reference, semantic, permission, and snapshot validation |
31
+ | [Migration](migration.md) | Compatibility from the current environment/JSON/SQLite surfaces |
32
+ | [Implementation plan](../../plans/implementation-plan.md) | Ordered implementation and release gates |
33
+ | [Examples](../../examples/vnext/README.md) | Complete project and workflow fixture |
34
+
35
+ Machine-readable schemas live under [`schemas/vnext`](../../schemas/vnext).
36
+ JSON Schema validates the data model after a YAML document has been parsed with
37
+ custom tags disabled and bounded aliases, depth, scalar size, and document size.
38
+
39
+ ## Non-negotiable invariants
40
+
41
+ 1. A project has exactly one authoritative Git-tracked project root.
42
+ 2. A run has one immutable `homeRuntimeId`.
43
+ 3. A run pins exact configuration, memory, model, and repository revisions.
44
+ 4. Hub unavailability does not prevent local or uniquely namespaced work.
45
+ 5. Shared mutable actions require an online lease.
46
+ 6. An `assignmentId` identifies logical work; an `attemptId` identifies one execution.
47
+ 7. Unknown side effects are never replayed automatically.
48
+ 8. Agent session history never crosses runs or a narrowing disclosure scope.
49
+ 9. Provider credentials and repository contents remain Runtime-local by default.
50
+ 10. Only schema-allowlisted, pre-redacted events enter the hub outbox.
51
+ 11. Memory and learned content cannot grant tools, secrets, approvals, or policy.
52
+ 12. Learned executable behavior activates only through a reviewed Git change.
53
+
54
+ ## Compatibility rule
55
+
56
+ The current v0.5 contracts remain authoritative until a release explicitly
57
+ activates a vNext schema. Implementations MUST NOT infer vNext behavior merely
58
+ because these documents or examples are present.
59
+
60
+ Every persisted vNext resource carries an exact schema identity. Additive
61
+ changes require a new compatible schema revision; a semantic breaking change
62
+ requires a new major schema identity and an explicit migration.
@@ -0,0 +1,185 @@
1
+ # ADR-001: Local Runtime, project authority, and aggregate hub
2
+
3
+ - **Status:** accepted target
4
+ - **Scope:** KXM vNext
5
+ - **Supersedes:** no current contract until migration activation
6
+
7
+ ## Context
8
+
9
+ The current implementation combines durable peer transport, workflow state,
10
+ and context in one hub-oriented process. KXM vNext must support multiple
11
+ projects and repositories while continuing useful work when a shared hub is
12
+ unavailable. It must also prevent a hub, model, or learned record from silently
13
+ becoming an execution or policy authority.
14
+
15
+ ## Decision
16
+
17
+ One auto-started **KXM Runtime** per OS user and machine owns execution. A
18
+ Runtime may host many trusted local projects. The optional **KXM hub** accepts
19
+ run requests, coordinates shared-operation leases, synchronizes safe events and
20
+ promoted factual memory, and powers aggregate views. The hub does not execute
21
+ agents and does not receive provider credentials or repository files.
22
+
23
+ ```text
24
+ Git-tracked project root machine-local authority
25
+ ┌─────────────────────────┐ ┌─────────────────────────┐
26
+ │ .kxm/project.yaml │ │ KXM Runtime │
27
+ │ .kxm/agents/*.yaml │──snapshot──▶│ workflow engine │
28
+ │ .kxm/models/*.yaml │ │ Pi/SSH executors │
29
+ │ .kxm/workflows/*.yaml │ │ worktrees + secrets │
30
+ │ project/repo skills, KB │ │ per-project event stores│
31
+ └─────────────────────────┘ └───────────┬─────────────┘
32
+ │ sync-safe events
33
+ ▼
34
+ ┌─────────────────────────┐
35
+ │ multi-project hub │
36
+ │ requests, leases, memory│
37
+ │ aggregate read models │
38
+ └─────────────────────────┘
39
+ ```
40
+
41
+ ## Authority matrix
42
+
43
+ | Subject | Authority | Replicas or projections |
44
+ |---|---|---|
45
+ | Executable project behavior | Reviewed Git project root | Runtime configuration snapshot; hub immutable snapshot |
46
+ | Repository input | Base commit plus content snapshot hash | Worktree; optional executor bundle |
47
+ | Run execution | Immutable home Runtime event log | Hub synchronized event log and read models |
48
+ | Harness/model capability | Runtime inventory | Bounded hub capability advertisement |
49
+ | Provider credentials | Harness or host secret store | Resolution status only |
50
+ | Secret values | Host secret store | Never replicated |
51
+ | Current factual project state | Promoted temporal state | Pinned local replica by memory revision |
52
+ | Learned executable guidance | Reviewed Git skill/configuration | Runtime snapshot; hub metadata |
53
+ | Human-readable dashboard state | Projection | Rebuildable from events and snapshots |
54
+
55
+ A projection is never an authority merely because it is easier to query.
56
+
57
+ ## Project and repository boundary
58
+
59
+ A project MUST have exactly one control Git root containing `.kxm/project.yaml`.
60
+ A project MAY bind multiple member repositories. Logical repository IDs and
61
+ portable remote identities are Git-tracked; absolute machine paths are
62
+ Runtime-local bindings.
63
+
64
+ Project resources and member-repository resources have explicit path scope.
65
+ When the control root is also a member repository, project and repository
66
+ resources still occupy different directories.
67
+
68
+ ## Run ownership
69
+
70
+ A run is created only after one Runtime accepts it. The accepted run records an
71
+ immutable `homeRuntimeId`. A hub request is not itself a run and cannot append
72
+ home-owned run events.
73
+
74
+ A run pins:
75
+
76
+ - project and workflow identity;
77
+ - configuration revision;
78
+ - promoted memory revision;
79
+ - exact model selections before their first dispatch;
80
+ - repository base commits and dirty snapshot hashes;
81
+ - executor and tool-policy revisions.
82
+
83
+ An offline-created run receives the same globally unique identity and ownership
84
+ fields as an online-created run. Offline runs may read their pinned memory
85
+ revision and create candidates, but MUST NOT promote hub memory.
86
+
87
+ ## Workflow execution
88
+
89
+ Top-level workflow steps are ordered. Transitions are typed, declared, and
90
+ bounded. Dynamic assignments exist only inside the active step and cannot
91
+ expand its agent, model, repository, tool, secret, time, cost, or parallelism
92
+ ceilings.
93
+
94
+ One isolated coordinator Pi session is created per run. Agent sessions are
95
+ identified by `{runId, agentId, instanceNo, scopeEpoch}` and are never reused
96
+ across runs. A narrowing or incompatible repository, secret, tool, model,
97
+ executor, or disclosure scope increments `scopeEpoch` and starts a clean
98
+ physical session.
99
+
100
+ ## Offline operation classes
101
+
102
+ | Class | Offline behavior | Examples |
103
+ |---|---|---|
104
+ | Local | Allowed | Planning, worktree edits, tests, local commits, candidates |
105
+ | Uniquely namespaced external | Allowed with deterministic key and receipt | Unique run branch, run artifact |
106
+ | Shared mutable | Requires online hub lease | Merge, deploy, shared tag, promotion, shared issue state |
107
+
108
+ The Runtime classifies the operation. A coordinator or model cannot downgrade
109
+ an operation to avoid a lease.
110
+
111
+ ## Storage topology
112
+
113
+ Each project has a local event store owned by its home Runtime. The Runtime has
114
+ a separate registry for host bindings, capabilities, enrollment, and process
115
+ state. The hub stores a registry plus isolated project stores. Raw logs,
116
+ repository contents, full prompts, full results, and secret values remain local
117
+ unless a reviewed policy explicitly transfers them.
118
+
119
+ ## Interface strategy
120
+
121
+ The CLI, standard TUI, Pi extension, and future web dashboard use the same
122
+ Runtime/hub command and read-model APIs. The TUI is first; the web dashboard is
123
+ not a separate workflow implementation.
124
+
125
+ The primary user path is:
126
+
127
+ ```text
128
+ kxm init
129
+ kxm run <workflow> [prompt]
130
+ ```
131
+
132
+ The Runtime auto-starts when needed. Normal projects rely on built-in
133
+ environment, harness, executor, retention, and synchronization defaults.
134
+
135
+ ## Security scope
136
+
137
+ The first release is trusted-local: logical project isolation, scoped tools,
138
+ isolated worktrees, explicit secret grants, protected local state, and audited
139
+ permission changes. It is not a sandbox against a hostile process running as
140
+ the same OS user. Container/OS isolation and hostile multi-tenant operation are
141
+ later phases.
142
+
143
+ ## Consequences
144
+
145
+ Benefits:
146
+
147
+ - local work survives hub outages;
148
+ - one hub can aggregate many projects without receiving repositories or provider credentials;
149
+ - Git review remains the activation boundary;
150
+ - recovery decisions are made where process and filesystem evidence exists;
151
+ - TUI and web interfaces share durable contracts.
152
+
153
+ Costs:
154
+
155
+ - events, projections, and synchronization require explicit versioning;
156
+ - shared external actions need leases and receipts;
157
+ - local bindings and Git configuration must be reconciled;
158
+ - run takeover cannot be added safely without an explicit fencing protocol.
159
+
160
+ ## Rejected alternatives
161
+
162
+ ### Hub-owned execution
163
+
164
+ Rejected because a hub outage would stop local work and would centralize
165
+ repository and provider credential access.
166
+
167
+ ### Git as the run event log
168
+
169
+ Rejected because high-frequency lifecycle events, locks, and crash recovery are
170
+ poor Git workloads. Git remains authoritative for reviewed behavior.
171
+
172
+ ### Mutable active-run configuration
173
+
174
+ Rejected because it destroys reproducibility and can expand permissions during
175
+ execution. Configuration edits affect future runs.
176
+
177
+ ### Automatic learned-policy activation
178
+
179
+ Rejected because evidence and summaries cannot grant authority. Activation is
180
+ a reviewed Git change.
181
+
182
+ ### Blind retry after a lost process
183
+
184
+ Rejected because at-least-once replay can duplicate irreversible effects.
185
+ Unknown outcomes enter `blocked_uncertain`.
@@ -0,0 +1,172 @@
1
+ # Effects, idempotency, and recovery
2
+
3
+ KXM provides at-least-once command delivery with effect-aware recovery. It does
4
+ not claim exactly-once external execution.
5
+
6
+ ## Effect declaration
7
+
8
+ Every Runtime-managed tool or adapter declares an effect class in a trusted,
9
+ versioned adapter/tool-preset registry. A workflow references the adapter or
10
+ preset; it does not declare its own effective class. Model output and workflow
11
+ YAML cannot downgrade the registry classification.
12
+
13
+ A resolved policy stored in an event or delivery manifest has this shape:
14
+
15
+ ```yaml
16
+ effect:
17
+ class: external-idempotent
18
+ idempotencyKey: assignment
19
+ receiptQuery: github-pull-request-by-head
20
+ sharedMutable: false
21
+ ```
22
+
23
+ Unknown tools and arbitrary shell commands default to `unknown`. A wrapper may
24
+ provide a stronger declaration only when it owns both dispatch and deterministic
25
+ reconciliation. Registry changes are reviewed permission changes and receive a
26
+ pinned `toolPolicyRevision` or `executorPolicyRevision`.
27
+
28
+ ## Classes and automatic behavior
29
+
30
+ | Class | Examples | Recovery |
31
+ |---|---|---|
32
+ | `read-only` | Read file, Git status, query API | Retry with bounded policy |
33
+ | `workspace-mutation` | Edit isolated worktree, formatter, generated fixture | Inspect and reconcile workspace; do not blindly reapply |
34
+ | `external-idempotent` | Put by stable key, push exact commit to unique ref | Query stable key/ref; retry only after confirmed absence |
35
+ | `receipt-queryable` | PR by unique head branch, named CI job | Query external system; record found receipt or execute after confirmed absence |
36
+ | `unknown` | Arbitrary shell/API/deploy script | Enter `blocked_uncertain` after a lost acknowledgement |
37
+ | `external-non-idempotent` | Unkeyed payment, irreversible deployment | Enter `blocked_uncertain`; require evidence or human resolution |
38
+
39
+ A local filesystem mutation outside the assigned isolated workspace is
40
+ `unknown`, even if the command was expected to edit a file.
41
+
42
+ ## Dispatch fence
43
+
44
+ The Runtime uses this order:
45
+
46
+ 1. Resolve and validate effective permissions.
47
+ 2. Allocate `assignmentId`, `attemptId`, and effect ID.
48
+ 3. Compute any deterministic idempotency key.
49
+ 4. Append the dispatch-intent event.
50
+ 5. Commit the event transaction.
51
+ 6. Start the process or external call.
52
+ 7. Observe completion.
53
+ 8. Append the result and receipt.
54
+ 9. Advance the assignment/step only after the result transaction commits.
55
+
56
+ Crashing before step 6 is safe to restart. Crashing between steps 6 and 8 is
57
+ handled according to effect class.
58
+
59
+ ## Crash matrix
60
+
61
+ | Last durable evidence | Recovery decision |
62
+ |---|---|
63
+ | Intent recorded; process provably never started | Start same planned attempt or mint a new attempt according to executor contract |
64
+ | Exact process/session still alive | Reattach same attempt from its cursor |
65
+ | Process ended; workspace state inspectable | Reconcile expected filesystem/Git postcondition |
66
+ | Stable external receipt found | Record receipt; do not repeat action |
67
+ | Receipt query proves absence | Retry using the same deterministic key where supported |
68
+ | Query unavailable, ambiguous, or non-authoritative | `blocked_uncertain` |
69
+ | Replacement process already exists | Do not reattach old attempt; reconcile or block |
70
+
71
+ Provider inference is not evidence that a tool effect did or did not happen.
72
+
73
+ ## `blocked_uncertain`
74
+
75
+ The state records:
76
+
77
+ - run, step, assignment, attempt, and effect identities;
78
+ - effect classification;
79
+ - last durable transition and cursor;
80
+ - expected postcondition and receipt query;
81
+ - reconciliation attempts and bounded diagnostics;
82
+ - permitted resolution actions.
83
+
84
+ Automatic execution stops only where dependent work could compound the unknown
85
+ effect. Independent read-only inspection MAY continue in a separate assignment
86
+ if the workflow permits it.
87
+
88
+ Resolution options:
89
+
90
+ | Resolution | Required evidence |
91
+ |---|---|
92
+ | Mark completed | Authoritative receipt or verified postcondition |
93
+ | Confirm absent and retry | Authoritative negative query plus new `attemptId` when a process is restarted |
94
+ | Compensate then retry | Compensation receipt and workflow-authorized new attempt |
95
+ | Mark failed | Audited deterministic or human decision |
96
+ | Cancel | Proof no unresolved owned effect remains, otherwise uncertainty remains visible |
97
+
98
+ A human resolution is an audited control-plane action, not retroactive proof.
99
+ The event records actor, reason, supplied references, and resulting action.
100
+
101
+ ## Workspace reconciliation
102
+
103
+ For isolated workspaces, recovery compares:
104
+
105
+ - pinned baseline snapshot;
106
+ - current manifest and Git index/worktree state;
107
+ - expected changed paths or postcondition;
108
+ - child process identity and open file/process handles where available;
109
+ - produced artifacts and hashes.
110
+
111
+ KXM applies no patch a second time merely because the prior tool result is
112
+ missing. If intent cannot be compared deterministically with state, the effect
113
+ is uncertain.
114
+
115
+ ## Git operations
116
+
117
+ Safe conventions:
118
+
119
+ - commits include run/assignment provenance in metadata or notes;
120
+ - run branches are globally unique;
121
+ - pushes target an exact expected object ID;
122
+ - receipt queries compare the remote ref to the expected object ID;
123
+ - shared branches, tags, merges, and releases require an online lease;
124
+ - a force update is never inferred safe from branch uniqueness.
125
+
126
+ Creating a pull request is queryable only when the adapter uses a deterministic
127
+ head branch and can authoritatively find an existing matching request.
128
+
129
+ ## Remote execution
130
+
131
+ The SSH helper reports a helper generation, process identity, attempt identity,
132
+ and monotonic event cursor. Connection loss alone does not create a new attempt.
133
+
134
+ Reattach only when the same remote helper and process can prove continuity. If
135
+ the remote host or helper has been recreated, reconcile workspace and external
136
+ receipts before starting a new attempt. An absent remote process does not prove
137
+ that an external side effect failed.
138
+
139
+ ## Cancellation
140
+
141
+ Cancellation is a request followed by supervised settlement:
142
+
143
+ 1. append cancellation intent;
144
+ 2. stop new dispatches;
145
+ 3. request graceful child cancellation;
146
+ 4. wait a bounded drain interval;
147
+ 5. terminate the owned process group where safe;
148
+ 6. reconcile in-flight effects;
149
+ 7. mark cancelled only when no unresolved effect remains.
150
+
151
+ A run with an unresolved effect remains `blocked_uncertain` rather than being
152
+ made cosmetically cancelled.
153
+
154
+ ## Lease failure
155
+
156
+ A shared mutable operation records lease identity and fencing token with its
157
+ dispatch intent. If the lease expires before dispatch, the operation is not
158
+ started. If it expires after dispatch, recovery queries the receipt; KXM never
159
+ assumes expiry rolled the external action back.
160
+
161
+ ## User interface
162
+
163
+ The TUI and Pi menu show uncertainty as an attention state with:
164
+
165
+ - last known operation;
166
+ - why automatic recovery is unsafe;
167
+ - available reconciliation adapter;
168
+ - supplied receipts;
169
+ - actions requiring human authorization.
170
+
171
+ Normal read, workspace, and queryable operations recover automatically. Manual
172
+ intervention is reserved for genuinely unprovable effects.