@staix/agent-hub 0.12.19 → 0.12.21

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (87) hide show
  1. package/CHANGELOG.md +38 -0
  2. package/README.md +7 -6
  3. package/docs/agent-notes/adapters.md +10 -2
  4. package/docs/agent-notes/bus.md +6 -2
  5. package/docs/agent-notes/daemon.md +10 -3
  6. package/docs/agent-notes/local-worker.md +3 -1
  7. package/docs/agent-notes/models.md +3 -0
  8. package/docs/agent-notes/tasks.md +6 -3
  9. package/docs/agent-notes/tests.md +3 -2
  10. package/docs/bench.md +143 -0
  11. package/docs/events.md +6 -2
  12. package/docs/operations.md +655 -34
  13. package/docs/quickstart.md +12 -10
  14. package/docs/research.md +92 -0
  15. package/docs/security.md +59 -17
  16. package/docs/smoke.md +137 -0
  17. package/docs/specs/2026-09-19-agent-hub-design.md +586 -16
  18. package/docs/specs/2026-09-20-pi-local-models-design.md +2 -0
  19. package/docs/specs/2026-09-20-upgrade-recovery-design.md +345 -0
  20. package/docs/specs/2026-10-04-switchyard-source-port-design.md +10 -0
  21. package/docs/verified.json +29 -18
  22. package/package.json +1 -1
  23. package/plugins/agent-hub/.claude-plugin/plugin.json +1 -1
  24. package/plugins/agent-hub/server.js +65 -26
  25. package/src/adapters/acp.ts +67 -6
  26. package/src/adapters/claude-channel.ts +61 -20
  27. package/src/adapters/codex-appserver.ts +70 -2
  28. package/src/adapters/local-worker.ts +8 -3
  29. package/src/adapters/pi.ts +58 -4
  30. package/src/cli/bench.ts +287 -0
  31. package/src/cli/console-state.ts +531 -110
  32. package/src/cli/console.ts +59 -26
  33. package/src/cli/facts-hook.ts +50 -8
  34. package/src/cli/help.ts +130 -0
  35. package/src/cli/identity.ts +11 -5
  36. package/src/cli/launch.ts +98 -17
  37. package/src/cli/main.ts +325 -88
  38. package/src/cli/preview.ts +14 -3
  39. package/src/cli/reset.ts +194 -0
  40. package/src/cli/status-lines.ts +10 -5
  41. package/src/cli/terminal-recovery.ts +196 -40
  42. package/src/cli/upgrade-runtime.ts +327 -60
  43. package/src/cli/upgrade.ts +442 -45
  44. package/src/hub/ask.ts +1 -1
  45. package/src/hub/bench.ts +211 -0
  46. package/src/hub/board.ts +15 -7
  47. package/src/hub/bus.ts +79 -8
  48. package/src/hub/child-process.ts +1 -0
  49. package/src/hub/conductor.ts +54 -21
  50. package/src/hub/config-trust.ts +1 -0
  51. package/src/hub/control-client.ts +9 -4
  52. package/src/hub/daemon.ts +401 -48
  53. package/src/hub/delivery-journal.ts +6 -6
  54. package/src/hub/events.ts +9 -2
  55. package/src/hub/execution-budget.ts +15 -6
  56. package/src/hub/hub-tools.ts +4 -4
  57. package/src/hub/inference.ts +136 -1
  58. package/src/hub/lifecycle.ts +13 -12
  59. package/src/hub/manager.ts +16 -14
  60. package/src/hub/peers.ts +2 -0
  61. package/src/hub/permission-mode.ts +44 -0
  62. package/src/hub/recovery-store.ts +68 -8
  63. package/src/hub/registry.ts +36 -22
  64. package/src/hub/report.ts +43 -1
  65. package/src/hub/research.ts +255 -0
  66. package/src/hub/restart.ts +25 -0
  67. package/src/hub/routing.ts +48 -16
  68. package/src/hub/snapshots.ts +3 -2
  69. package/src/hub/tasks.ts +295 -115
  70. package/src/hub/tool-refusal.ts +7 -0
  71. package/src/hub/ui.ts +4 -1
  72. package/src/local/deny.ts +2 -1
  73. package/src/local/tools.ts +14 -10
  74. package/src/models/mlx.ts +11 -8
  75. package/src/models/ollama.ts +2 -2
  76. package/src/models/relay.ts +190 -42
  77. package/src/models/route/relay-selector.ts +43 -31
  78. package/src/models/route/runtime.ts +17 -7
  79. package/src/models/route/signals.ts +12 -1
  80. package/src/models/route/stage.ts +57 -3
  81. package/src/omniroute/client.ts +1 -1
  82. package/src/pi/process-signature.ts +23 -0
  83. package/src/pi/tool-receipts.ts +7 -1
  84. package/src/ui/index.html +84 -6
  85. package/src/ui/task-progress.ts +33 -0
  86. package/templates/AGENTS.block.md +1 -1
  87. package/templates/routing.toml +8 -0
@@ -1,8 +1,25 @@
1
1
  # Operations guide
2
2
 
3
- This guide describes ahub 0.12.19 and control protocol 15. Live verification
3
+ This guide describes ahub 0.12.21 and control protocol 16. Live verification
4
4
  results and remaining prerequisites are recorded separately in [the smoke ledger](smoke.md).
5
5
 
6
+ ## Command help
7
+
8
+ `ahub help`, `ahub --help` and `ahub -h` print the commands grouped by section;
9
+ `ahub help <command>` prints only that command's entries, and so does
10
+ `ahub <command> -h` (or `--help`) when the flag is the only argument; anywhere else
11
+ it is an ordinary argument, and `ahub claude` and `ahub codex` pass it on to the
12
+ agent. An unknown command
13
+ prints `unknown command "<name>"; run ahub help` and exits non-zero. Usages start
14
+ at two spaces and descriptions at one fixed column, wrapped at word boundaries to
15
+ the terminal width, read as at least 80 and at most 100 columns.
16
+
17
+ Help uses the console palette: section headings bold cyan, `ahub <command>` words
18
+ cyan, arguments and descriptions the terminal's default foreground. It has no
19
+ `--color` flag: color is on only when standard output is a terminal, `TERM` is
20
+ not `dumb` and `NO_COLOR` is empty or absent. Pipes and redirects get plain text
21
+ without escape sequences, laid out for 80 columns.
22
+
6
23
  ## Operator console and panels
7
24
 
8
25
  `ahub console` combines the existing tail stream with peer, quota, approval-age
@@ -23,15 +40,88 @@ asking, under the same path guard and sandbox; a Pi restart or replacement, or a
23
40
  hub restart, drops it. The hub log records each granted call by tool name only.
24
41
  The dashboard can only deny Pi's requests.
25
42
 
26
- Tab toggles stream and panels; `ahub console --panels` starts in panels. Peers,
27
- Approvals, Tasks, Queue and Events support arrow keys or j/k, Enter for detail,
28
- Escape to return and `?` for help. `:` enters a command. Assignment, delivery
29
- resolution and allow decisions require confirmation; delivery resolution requires
30
- a reason. Tasks use the same public redaction as the board. Panels need at least
31
- 80 columns by 24 rows; smaller terminals stay in stream mode. Task and queue
32
- polling runs only while the corresponding panel is visible. Leaving restores
43
+ Tab toggles stream and panels; `ahub console --panels` starts in panels. A panel
44
+ screen has a header (the project's directory name, then the tabs), the panel and
45
+ a three-line footer, separated by ASCII `-` rules; the stream has one rule above
46
+ its footer. The active tab is bracketed, as in `[2 Approvals]`, so it reads
47
+ without color. The Approvals tab counts pending requests and the Queue tab
48
+ counts `needs_review` deliveries, in the attention and failure tones.
49
+
50
+ Peers, Approvals, Tasks and Queue are tables with a header row. A column is as
51
+ wide as its widest cell, up to a third of the terminal; the last column takes
52
+ the rest and is cut with `...`. In Tasks, TITLE takes the rest and the STAGE
53
+ meter keeps a fixed column at the right edge.
54
+
55
+ | Panel | Columns |
56
+ |---|---|
57
+ | Peers | PEER STATE MODE LINK Q ! REVIEW PAUSE QUOTA MODEL |
58
+ | Approvals | ID PEER LEFT TITLE |
59
+ | Tasks | ID STATE OWNER REVIEWER CLASS AGE TITLE STAGE |
60
+ | Queue | ID PEER STATE REV AGE |
61
+
62
+ The MODE column appears only when at least one peer has a non-`ask` mode.
63
+ Zero counters and unknown values read `-`. Durations read `45s`, `12m`, `3h05m`
64
+ or `2d03h`; quota resets and budget pauses read `in 2h13m`, and a pause by the
65
+ console user reads `user`. When a peer's status carries #205's optional
66
+ `toolsOnly` field (a Claude session attached without channel pushes), LINK reads
67
+ `tools-only`. Below the Approvals table, a rule and the selected
68
+ request: its title, its allow options numbered as `a` offers them, and `d deny`.
69
+ A title too long for the panel is marked `(more)` in the label column; Enter
70
+ shows it whole. `d deny` always shows; options that do not fit are counted on a
71
+ `(more)` line. The first request is selected when it arrives; `[` and `]` (or
72
+ j/k in the panel) select another. `a`, `d` and `v` act on the selected request
73
+ only: when it closes, the selection clears with a notice instead of moving to
74
+ another request, and a request that arrives later waits for `[` or `]`. Moving
75
+ the selection cancels a pending allow choice or confirmation; the choice prompt
76
+ names its request. Agent-written text below a field, and option names in
77
+ prompts, are quoted unless they are a single word starting with a letter, so
78
+ punctuation, a newline or a bare number in them cannot imitate the console's own
79
+ structure. Under the `?` key table only `?`, Escape and `q` act.
80
+
81
+ Arrow keys or j/k move, Enter opens a detail of labeled fields (relative times,
82
+ never JSON), Escape returns, and `?` shows the keys grouped by panel; in the
83
+ stream, `?` prints that table and `v` prints the selected request as its push
84
+ showed it. The footer's last line lists only the keys that act in the current
85
+ mode, panel and state. A notice shows on the footer's second line until the next
86
+ key or for about ten seconds: errors and refusals (a refused command, a request
87
+ error, the size refusal) in the failure tone, notices that only inform (a closed
88
+ or missing selection) in the attention tone. With a detail open, `p`, `r`, `m` and `a`
89
+ act on the item the detail shows. `:` enters a command; its output lines
90
+ start at column 4, as message bodies do. Assignment, delivery resolution, allow
91
+ decisions and a never-ask permission mode require confirmation; delivery resolution requires a reason. Tasks use the same public redaction as the board. Panels need at least
92
+ 80 columns by 24 rows; smaller terminals stay in stream mode. Full task and queue reads run only in panels; stream progress uses the existing
93
+ status state counts and never fetches the board. Leaving restores
33
94
  the terminal and returning from panels replays the bounded stream buffer.
34
95
 
96
+ The Tasks panel summarizes the whole public board with an approved/all fraction,
97
+ a 20-cell ASCII bar and state counts. Proposed tasks with unapproved or missing
98
+ `after` dependencies count as waiting; PII stubs count normally. Each task has a
99
+ four-cell stage meter (proposed, in progress, review, approved). Changes requested
100
+ returns to in progress and uses `!` plus the failure tone. Narrow panels retain
101
+ the fraction/bar before optional counts, which appear in fixed importance order:
102
+ changes requested, review, waiting, in progress, proposed, then unknown when nonzero. Narrow widths drop
103
+ only the suffix. Bar cells and percentages round down so open tasks never imply
104
+ completion. Stream `tasks N/M approved` lives on the approvals footer line, leaving the
105
+ peer/queue footer line intact. Operator notices or an already-full approvals line
106
+ take precedence over this optional count. A brief notice hides it only until the
107
+ notice's ten-second TTL expires, then the retained known count returns on redraw; the scroll
108
+ region starts at row 1 to preserve terminal scrollback. All existing status state
109
+ counts contribute to its total, including an unfamiliar state. Until a successful
110
+ status read it says loading; later failures preserve the prior count with a notice.
111
+ Panels retain their existing full-board refresh and loading/error handling.
112
+
113
+ The dashboard Task board shows the same shared public-board progress model,
114
+ including state counts, waiting, stacked segments and a labeled four-stage track
115
+ per row. Filtering rows does not filter overall progress. The existing signature guard
116
+ updates the live region only when progress changes. The stacked SVG fills the
117
+ section width; its text legend carries every state count. Theme offers System,
118
+ Light and Dark; it is applied before first paint and stored in a host-scoped loopback cookie
119
+ (`Path=/`, `SameSite=Strict`, one-year lifetime), shared across dashboard ports.
120
+ Blocked cookie access defaults to System on reload and never prevents operation. System follows
121
+ OS color preference through CSS. Both inline scripts and the style retain CSP
122
+ hashes; there are no external chart/theme assets. Live theme/readability inspection
123
+ is tracked in `docs/smoke.md` and remains unverified until performed.
124
+
35
125
  Console color policy is `--color=auto|always|never`, with `auto` as the default.
36
126
  Auto enables color only when both input and output are TTYs, `TERM` is not
37
127
  `dumb`, and `NO_COLOR` is empty or absent. Explicit `always` overrides these
@@ -45,12 +135,31 @@ management or the final reset. Invalid values fail before connecting.
45
135
  | Success and availability | Green | Idle peer, approved task |
46
136
  | Waiting and attention | Yellow | Busy/paused peer, pending approval, confirmation, review/ready task, important priority |
47
137
  | Failure and intervention | Red | Failed/check-failed task, undeliverable/overflow, `needs_review` queue, denial request or expired/cancelled approval |
48
- | Metadata | Bright black | Ages and remaining approval time |
138
+ | Metadata | Bright black | Ages, remaining approval time and rules |
139
+ | Stream peer names | Claude bright blue (94), Codex bright cyan (96); all others default | Fixed map, no hashing or themes |
140
+ | Stream `task` keyword | Magenta | Task event keyword |
141
+ | Stream numbers | Bold default foreground (1) | Percentages, durations and measured times |
142
+ | Stream issue references | Underlined default foreground (4) | `#232` |
143
+ | Stream task references | Magenta underlined (4;35) | `#1` following `task`/`Task`, `[task #1]`, `[review #1]` |
49
144
 
50
145
  Offline peers, primary titles, action details and message body lines keep the
51
146
  terminal's default foreground. Labels and prompts remain readable without
52
- color. Stream styling applies only to the header, using structured event data;
53
- message text cannot choose a color. A local denial is shown as requested, not
147
+ color. Stream styling applies only to meaningful tokens in the first line: peer names,
148
+ `task`, numbers and references. Ordinary words keep the default foreground.
149
+ Task prefixes and bracketed `[review #N]` identify task references
150
+ case-insensitively; prose `review #N` and other `#N` tokens are issue references. Legacy assign/which notices lack typed reference
151
+ ranges and use the issue tone until structured spans identify them. The dated
152
+ #239 palette reserves green/yellow/red for states, cyan for navigation and
153
+ blue/cyan for Claude/Codex. Tokens use plain magenta, underlining and bold default
154
+ foreground; no token SGR equals a state or peer SGR. The `task` keyword is never
155
+ bold. Only hub-written peer slots are colored, not title words or decline reasons.
156
+ State words and structural markers retain their event tone. Numeric-looking
157
+ identifiers stay plain. Wrapped headers normalize tabs before tokenization; plain headers preserve
158
+ their original tabs with color enabled or disabled.
159
+ A projection miss on the first physical header keeps only structural tones.
160
+ A missed continuation and every following continuation are fully plain; their
161
+ text cannot supply structural tones. Projection carries its span index forward
162
+ instead of rescanning earlier spans for every line. Bodies and command output stay plain. A local denial is shown as requested, not
54
163
  as a confirmed receipt; a remote answered closure has no option-kind metadata
55
164
  and is not guessed to be a denial. A fixed palette is applied after terminal
56
165
  control sanitization. Width, clipping, wrapping and cursor placement use plain
@@ -70,8 +179,8 @@ recorded separately in the smoke ledger; fake-terminal tests establish policy,
70
179
  geometry, sanitization and restoration only.
71
180
 
72
181
  The command input accepts existing status, board, task, review, say, pause,
73
- resume, budget, queue, permit, ask, remember, route, turns, undo, check-path and
74
- report operations. It executes an argument vector with closed stdin. Lifecycle,
182
+ resume, budget, queue, permit, permission, ask, remember, route, turns, undo,
183
+ check-path and report operations. It executes an argument vector with closed stdin. Lifecycle,
75
184
  launch, nested console, setup, UI, logs and tail commands are refused.
76
185
 
77
186
  ## Conducting a team from Claude Code or Codex
@@ -118,9 +227,11 @@ off withdraws pending feed notices.
118
227
 
119
228
  For a Claude conductor, `ahub claude` also observes native session and turn
120
229
  boundaries when facts injection and task-idle sweeps are off. Keep the managed
121
- hooks enabled to measure completion and supervision usage. Passing your own
122
- `--settings` takes precedence and produces a warning when it replaces that
123
- observation. The launcher records its private session identity in ordinary
230
+ hooks enabled to measure completion and supervision usage. Your own
231
+ `--settings` (given once, a JSON object or file) keeps that observation: the
232
+ launcher adds the hub's hooks to its hooks and warns only that the status line
233
+ tee is off; one that sets `disableAllHooks` is refused. The launcher records its
234
+ private session identity in ordinary
124
235
  terminals too; this does not grant terminal-recovery authority.
125
236
  Ordinary Claude sessions with turn-free facts or task-idle sweeps enabled get the
126
237
  same session/start observation. Other ordinary launches remain non-opt-in.
@@ -157,7 +268,8 @@ may request an approval.
157
268
 
158
269
  `.agenthub/config.json` can be committed and shared. The fields that choose
159
270
  what the hub runs, which files it sends as credentials, where task text goes,
160
- or how far the local worker's sandbox reaches (`kimi_cmd`, `codex_bin`,
271
+ how far the local worker's sandbox reaches, or how often a peer asks before it
272
+ acts (`kimi_cmd`, `permission_modes`, `codex_bin`,
161
273
  `pi.cmd`, `checks`, `mlx.bin`, `mlx.runtimeDir`, `mlx.modelPath`, `omniroute.urls`,
162
274
  `omniroute.access_hosts`, the `omniroute` key files, `memory.worker_url`,
163
275
  `local.read_allow`, `local.bash_network`, `local.network_allow`) are machine-local:
@@ -305,7 +417,10 @@ the routing configuration.
305
417
  A review request carries a checklist: map the changed signatures and call sites
306
418
  to the task's plan (without one, to its detail, which the request then
307
419
  includes), read the check result, and list what is
308
- unmet (`hub_review` takes `unmet`, `ahub review ... --unmet <item>`). The hub
420
+ unmet (`hub_review` takes `unmet`, `ahub review ... --unmet <item>`). A reviewer
421
+ that missed the request, and the owner, read the done summary and check result
422
+ with `hub_task_show {id}`: the task's public view with its history, as the
423
+ conductor sees it, and only a stub for a PII task. Other peers are refused. The hub
309
424
  records how each review turned out, per implementer, reviewer and class:
310
425
  approved; caught (changes were requested and the owner's redo was approved);
311
426
  contradicted (within a week, work on the same file or symbol failed its check
@@ -327,7 +442,8 @@ one line with its next message, the plan included, at no turn of its own. When a
327
442
  task is done (after its check passes, when one is configured), the owners of
328
443
  open tasks on the same paths or symbols get a message with the changed files,
329
444
  the plan's signatures and the first line of the summary, each left out when it
330
- matches a PII pattern; nobody else does. PII
445
+ matches a PII pattern (the summary also when `signals.pii_screen = "local"` did
446
+ not clear it); nobody else does. PII
331
447
  tasks are left out on both sides.
332
448
 
333
449
  A task can wait for others: `hub_task_propose` takes `after: [ids]` (`ahub task
@@ -336,12 +452,43 @@ offered to nobody, cannot be claimed, accepted or marked done, and `ahub route
336
452
  explain <id>` says what it waits for. When the last one is approved, the task
337
453
  goes through assignment like a new one; if the hub stopped before it got that
338
454
  far, the task is offered within a minute of a peer that can take it attaching
339
- to the next run.
455
+ to the next run. An offer is used up only by an assignment, after an approval
456
+ and on the timer alike. If no peer can take it or a ready/assignment write fails,
457
+ the task stays eligible; the timer adds no history or notices while nobody can
458
+ take it.
340
459
  `ahub board --ready` and
341
460
  `hub_task_list {ready: true}` list the proposed tasks with nothing left to wait
342
461
  for. Dependencies are fixed when a task is proposed and can only name tasks that
343
- already exist, so they cannot form a cycle. A waiting task cannot name an owner;
344
- use `ahub task assign` once it is ready. An owner offline longer than `tasks.release_after_min`
462
+ already exist, so they cannot form a cycle.
463
+
464
+ An owner named for a task that waits (`owner` with `after`, `--owner` with
465
+ `--after`) is its reserved owner: the task still waits, and when it is ready
466
+ routing offers it to that peer first, not to the first idle peer of the class.
467
+ A reserved owner that is offline, paused, failing, not attached or excluded is
468
+ passed over: routing proceeds as usual, the console and hub.log say which
469
+ reservation was passed over and why, and the assignment's history note keeps
470
+ it, unless it is no news (the reserved peer is the owner the task moves away
471
+ from, or it already refused the task). A peer the task was declined for (by
472
+ itself or by the console) or escalated away from after repeated
473
+ changes_requested or by hand stays excluded on every later reroute; an owner
474
+ released as gone, or moved by the hub after a failed delivery or inference, is
475
+ excluded only from that move and may get the task again from its reservation
476
+ later. Your own `ahub task assign` is not blocked by those exclusions, and it
477
+ drops the agent's reservation. The PII constraint and capability limits
478
+ still apply, so a PII task goes to `local` or nobody whoever was reserved.
479
+ `ahub task assign` on a task that waits changes its reserved owner instead of
480
+ handing it over. `ahub route explain <id>` names the reserved owner; once the
481
+ task is ready it also says whether routing would honor the reservation now, and
482
+ why not.
483
+
484
+ The peer that proposed a task may redirect it with `hub_task_assign` while it
485
+ is `proposed` and nobody ever accepted it, without the conductor role (handing
486
+ it to another peer needs `assign` when the proposer has a capabilities list).
487
+ Once it was accepted, even if a decline or release later put it back in
488
+ `proposed`, or after a person assigned or reserved it from the console (also
489
+ once the hub carried that out), only the conductor and the console move it.
490
+
491
+ An owner offline longer than `tasks.release_after_min`
345
492
  (default 30, `0` turns it off) in `.agenthub/config.json` loses its open tasks
346
493
  to a peer routing can give them to; with nobody to take them they stay, and a
347
494
  paused peer or a hub in a recovery operation is left alone.
@@ -393,6 +540,13 @@ ahub report --by task # tokens, turns and wall time per task and class
393
540
  ahub export --since 24h # the raw events as JSON lines, for your own analysis
394
541
  ```
395
542
 
543
+ Per peer and model route, `ahub report` also counts model changes between consecutive
544
+ decisions, how many of them landed inside tool loops and, with `stay_switch` on, sessions
545
+ and the planner's switches (#197). With `stay_switch = "off"` session boundaries are not
546
+ recorded; `shadow` routes exactly as `off` does, so it is the baseline to compare `enforce` with.
547
+ A route event records the decision, so a load-moved request that its MLX fallback served counts
548
+ as a change to `dgx/fast`; the relay's request journal records which backend served it.
549
+
396
550
  `ahub report` counts the same overlap warnings as `scripts/overlaps.ts`, from the
397
551
  structured events instead of log lines. `--by task` uses the task each usage and
398
552
  token record was attributed to when it was written: the delivery that started the
@@ -400,6 +554,37 @@ turn, otherwise the peer's only `in_progress` task. Everything else is reported
400
554
  unattributed, and records from before 0.12.18 as a separate bucket; neither is
401
555
  redistributed (rules: [events](events.md#per-task-usage-reports)).
402
556
 
557
+ ## Research records
558
+
559
+ With `"research": { "enabled": true }` in `.agenthub/config.json` (off by default), the hub keeps one record per
560
+ approved task in `~/.agenthub/research/`, shared by every project that opts in: outcome, review rounds, rework, failed
561
+ checks, tokens and time per peer and model, never task or message text ([schema](research.md)).
562
+
563
+ ```bash
564
+ ahub research --since 30d # success, first-pass, rework and check-failure rates; tokens and wall time per task
565
+ ahub research --all --json # every opted-in project on this machine
566
+ ahub research export --format csv # the records, for your own analysis
567
+ ahub research backfill # build this project's records once from events.jsonl
568
+ ahub task label 12 reverted # a later verdict (ok, regressed, reverted, incomplete, wrong, abandoned)
569
+ ```
570
+
571
+ ## Benchmarks
572
+
573
+ In a git repository kept for benchmarks (`"bench": { "enabled": true }` in its `.agenthub/config.json`, a clean tree),
574
+ `ahub bench` runs a suite of tasks against the peers attached to its hub and compares configurations: pass rate, first
575
+ pass, tokens and wall time per attempt, with bootstrap intervals between arms. Each attempt resets the tree to the
576
+ task's commit, so never enable it in a project you work in. The project must be the root of its repository with
577
+ nothing tracked under `.agenthub/`, and the suite file lives outside its tree ([suite format and store](bench.md)).
578
+
579
+ ```bash
580
+ ahub bench run ../suites/suite.json --arm baseline --repeat 5 # a person's: resets the tree, proposes the tasks
581
+ ahub bench status # the run in progress
582
+ ahub bench compare baseline ripwire-on # arms side by side, inconclusive below 5 attempts each
583
+ ahub bench export --format csv
584
+ ```
585
+
586
+ The dashboard's Benchmarks section shows the latest runs and, per suite with two or more arms, the comparison.
587
+
403
588
  ## Turns and undo
404
589
 
405
590
  In a git work tree the hub snapshots the project's tracked and unignored files (the
@@ -509,8 +694,9 @@ stays the default until an evaluation says otherwise (`docs/cooperbench.md`).
509
694
  held.
510
695
  - Verified context path. A hub-launched Claude session (`ahub claude`) gets the
511
696
  hub's hook before and after every tool call and at the end of each turn, in
512
- its `--settings` next to the status line tee (a `--settings` of your own turns
513
- them off). Codex gets context by steer into its running turn. Until a fact or a
697
+ its `--settings` next to the status line tee (a `--settings` of your own keeps
698
+ them, added to its own hooks, and loses only the tee). Codex gets context by
699
+ steer into its running turn. Until a fact or a
514
700
  one-line probe has been read back, the peer counts as unverified: for Claude
515
701
  the row Claude Code writes in its transcript for the hook's additional
516
702
  context, matched by tool use id and the offer's id; for Codex the steered
@@ -655,6 +841,83 @@ stays the default until an evaluation says otherwise (`docs/cooperbench.md`).
655
841
  turns the stale-notice drop off, for a controlled comparison only (the #106
656
842
  ablation in `docs/cooperbench.md`); the hub logs it at start.
657
843
 
844
+ ## Permission modes for running peers
845
+
846
+ Use a plain terminal to inspect or change a running peer:
847
+
848
+ ```bash
849
+ ahub permission
850
+ ahub permission local ask-when-needed
851
+ ahub permission kimi never-ask --yes
852
+ ahub permission kimi ask
853
+ ```
854
+
855
+ In the console Peers panel, `m` opens the three modes. Never-ask requires `y`,
856
+ even for a typed command carrying --yes. Its confirmation says whether the peer
857
+ uses its own tools outside the hub sandbox (Kimi), the hub sandbox/path guard/
858
+ denylist (Pi and local), or native vendor boundaries (Claude and Codex).
859
+
860
+ | Peer | ask-when-needed | never-ask | ask |
861
+ | --- | --- | --- | --- |
862
+ | Kimi Code CLI | ACP yolo | ACP auto | ACP default |
863
+ | Codex | on-request for every outgoing turn/start | never for every outgoing turn/start | Restore the known native policy once, then stop overriding |
864
+ | Claude | Allow resolving project file tools outside .agenthub/.git/.claude/.codex/.qwen/.kimi/.pi/.mcp.json | Hook allow for every tool | No hook decision |
865
+ | Pi and local | Grant write/edit outside native agent configuration paths; read never asks; shell/git writes ask | Grant each tool once within the existing sandbox/guard | Normal approval handling |
866
+
867
+ Kimi's map applies only to the actual Kimi Code CLI identity, not another ACP
868
+ agent configured in kimi_cmd. Qwen and other ACP agents start fresh/resumed
869
+ unmanaged without mode changes or mode-based startup refusals; runtime changes
870
+ are refused until their own map exists.
871
+ A set_mode timeout leaves the peer offline with mode unknown; inspect and
872
+ restart before using it. Missing ids and refused startup changes name the mode.
873
+ The benign Kimi live check verifies file edits and harmless shells in both
874
+ modes without requests; risky-action behavior remains unverified.
875
+
876
+ Claude's hook is installed in every managed launch. File grants use canonical
877
+ real paths under the authenticated project root. Missing targets, symlink
878
+ escapes, metadata directories and uncertain wildcard targets leave the native
879
+ rules in charge. Grep/Glob path-bearing filters cannot bypass the exclusions.
880
+ Other tools retain native decisions, which may already allow a harmless shell.
881
+ Permission-only PreToolUse never marks Claude busy; former observation hooks
882
+ retain their native-turn role. Settings are passed as a 0600 state file retained
883
+ until that native launch exits. Live/unknown previous launchers keep their files;
884
+ verified dead wrapper and native identities permit crash cleanup. Without
885
+ separate native identity proof, old settings remain for manual inspection. Settings carry
886
+ no inline caller values in argv. Status reports an unverified Claude hook as
887
+ unverified. Unattended native sessions cannot take a runtime mode; restart
888
+ without that native flag. Codex must be behind the hub proxy, and returning to
889
+ ask requires a known native approval baseline. An offline or absent peer accepts
890
+ `ahub permission <peer> ask` to clear its hub choice. Codex --unattended launch
891
+ is refused under a non-ask choice; clear it with `ahub permission codex ask`
892
+ first. A pending never-ask default cannot be confirmed for an unattended launch.
893
+
894
+ Project defaults use permission_modes in .agenthub/config.json or
895
+ .agenthub/config.local.json. Keys merge per peer; a local ask overrides the
896
+ same peer's earlier default, while an empty local block preserves it. Every
897
+ non-ask default start logs its mode and contributing filename. Automatic
898
+ config effects stop at ask-when-needed. Git-tracked defaults are ignored and never
899
+ offered for confirmation. Never-ask from an applied untracked file starts in ask
900
+ and appears as a pending source-labelled console question at hub start.
901
+ Only its separate console y enables that default; there is no boot --yes
902
+ bypass. Only explicit n declines and keeps ask for that hub. Esc defers without declining
903
+ and leaves approval keys usable; a refused y is not reoffered until its state changes. A successful runtime change removes that pending default. Runtime choices expire when the hub
904
+ stops, and a later restart asks again for a never-ask config default.
905
+
906
+ The CLI marker and console-role checks are operating policy, not a hostile-agent
907
+ boundary. An agent able to run unrestricted shells may clear its markers,
908
+ use --yes, or read the control token and act as the console. Hub tools,
909
+ conductor operations and ordinary agent messages have no direct mode-changing
910
+ or startup-confirmation operation, but those checks do not contain such a
911
+ shell. Review the boundaries in [security](security.md) before choosing a mode.
912
+ Modes never alter native sandboxes, hub path guards or denylists.
913
+
914
+ Permission mode starts reconcile the current operator choice after native startup before bus delivery. Each non-ask start logs the mode and its config or runtime source. Dropped git-tracked defaults never appear as startup confirmations. Non-Kimi ACP peers run unmanaged, preserving their native fresh/resumed mode; permission commands refuse changes until a vendor map exists.
915
+
916
+ Status shows non-ask modes and pending defaults, and each runtime change emits
917
+ permission_mode (peer/from/to) with no tool arguments under events schema 1.
918
+ The optional control requests preserve PROTOCOL; an older hub answers unknown
919
+ and the CLI advises upgrading.
920
+
658
921
  ## Approvals and pauses
659
922
 
660
923
  Inspect permission requests in the terminal:
@@ -673,8 +936,8 @@ ahub permit <request-id> allow
673
936
 
674
937
  Use the exact option shown by `ahub tail`; do not approve an unresolved or
675
938
  unexpected request. On macOS a waiting request also raises a desktop
676
- notification that names the peer and, for Kimi, the tool, never what it would
677
- run. An unanswered request is cancelled after `approvals.timeout_s` (default
939
+ notification that names the peer and, for Kimi and the local worker, the tool,
940
+ never what it would run. An unanswered request is cancelled after `approvals.timeout_s` (default
678
941
  120, 30 to 3600) in `.agenthub/config.json`, and the console and log say so.
679
942
  Set `approvals.notify` to `false` to turn notifications off; a hub started
680
943
  without a project config file raises none. While Kimi waits for an answer its
@@ -734,12 +997,39 @@ has only `ahub status`, `ahub tail` and the task board.
734
997
 
735
998
  Resolution requires the current observed revision. A retry closes the old
736
999
  record and creates one linked attempt. Stale or conflicting resolutions fail.
1000
+ A refused or failed resolution puts back only that peer's queue and preface
1001
+ and keeps every pause, including budget and conductor holds. A storage failure
1002
+ stops the bus; a validation refusal leaves it running under the existing holds.
737
1003
  If dispatching or accepted work has an uncertain outcome after a crash,
738
1004
  the system marks it `needs_review`. Establish evidence before choosing
739
1005
  `completed`, `retry`, or `discard`. Timeouts,
740
1006
  disconnects, cancellation, and partial effects are uncertain; do not replay
741
1007
  them automatically.
742
1008
 
1009
+ ### Claude sessions without channel pushes
1010
+
1011
+ Only `ahub claude` starts Claude Code with the development-channel flag. A
1012
+ Claude session started another way (plain `claude`, an IDE) keeps the hub tools
1013
+ but cannot show pushes, so it attaches tools-only (protocol 16, issue #205):
1014
+
1015
+ - Its messages stay queued. `ahub status` shows `queued N` and `tools-only:
1016
+ messages wait for hub_inbox; for pushes restart Claude with ahub claude`; the
1017
+ console footer shows `tools-only: ahub claude` and the dashboard shows the
1018
+ status line.
1019
+ - The session reads them with `hub_inbox`, at most ten messages at a time plus
1020
+ the hub's recall note when one waits (it says how many still wait). Each read is one `completed` delivery (`read through
1021
+ hub_inbox` in `ahub queue list`); nothing is ever `accepted`, and the session
1022
+ has no `hub_delivery_done`. A read the model never saw (the tool call was
1023
+ cancelled, the plugin died) cannot be retried: `ahub queue show <id>` of the
1024
+ `completed` row prints the messages to send again.
1025
+ - Whatever holds pushes holds `hub_inbox` too (`needs_review`, recovery,
1026
+ `ahub pause`, a budget pause, a conductor hold): it reads nothing and says
1027
+ which hold applies.
1028
+ - Restarting Claude with `ahub claude` takes the peer over and pushes what still
1029
+ waits; the plain session stands by. The other way round, a plain session
1030
+ started while an `ahub claude` session holds the peer stands by and attaches
1031
+ only after that session leaves.
1032
+
743
1033
  ## Graceful shutdown
744
1034
 
745
1035
  Before stopping, let active turns and approvals settle when possible:
@@ -764,6 +1054,95 @@ so a restarted hub never launches beside a survivor. A hub whose project root
764
1054
  or state directory was deleted stops itself within about 10 seconds; the
765
1055
  dashboard manager does the same when its home directory vanishes.
766
1056
 
1057
+ ## Resetting a project's hub
1058
+
1059
+ `ahub reset` returns a project's hub to a clean state. Without `--yes` it is a
1060
+ dry run: it lists what it would settle, clear, archive and keep, by delivery
1061
+ id, peer name and count, never task or message text, and changes nothing.
1062
+ With `--yes` it first stops a running hub the way `ahub kill` does, then acts.
1063
+
1064
+ ```bash
1065
+ ahub reset # dry run, runtime scope
1066
+ ahub reset --yes # runtime reset
1067
+ ahub reset --all # dry run, full reset
1068
+ ahub reset --all --yes # full reset
1069
+ ```
1070
+
1071
+ The runtime reset (the default):
1072
+
1073
+ - discards every `queued` and `needs_review` delivery with reason `reset`
1074
+ through the delivery journal, one `resolution_history` entry each. Work
1075
+ that was dispatching or accepted when the hub stopped is held as
1076
+ `needs_review` first, so it is discarded too;
1077
+ - clears manual peer holds (`ahub pause`), budget pauses and conductor holds;
1078
+ - drops the agent session resume pointers `sessions.json`,
1079
+ `claude-session.json` and `claude-context.json`, so the next launch starts
1080
+ new agent sessions. Pi transcripts under `pi-sessions/` stay;
1081
+ - removes the manifest (`status.json`, `control-token`, and `hub.pid` from a
1082
+ hub of 0.12.20 or older) a hub left when it did not stop cleanly, once no
1083
+ process behind it is alive (a pid that now belongs to another process counts
1084
+ as gone when the manifest carries the hub's signature, from 0.12.21);
1085
+ - keeps the board (tasks, reviews, outcomes, turns, touches), `hub.log`,
1086
+ `events.jsonl` and `cli-audit/`, recovery records, execution budgets,
1087
+ configuration and files the hub does not own.
1088
+
1089
+ The full reset (`--all`) moves the whole state directory, unchanged, to
1090
+ `.agenthub/archive/state-<UTC time>/` (mode 0700) and starts an empty state
1091
+ directory holding only `project.json`, so the project keeps its id and
1092
+ registration; the next `ahub up` starts with an empty board and queue.
1093
+ `.agenthub/config.json`, `config.local.json` and `routing.toml` live outside
1094
+ the state directory and are not touched. The archive keeps the task text,
1095
+ PII included, so it stays in the project: the reset writes
1096
+ `.agenthub/archive/.gitignore` (`*`) so git ignores it, and the local
1097
+ worker's and Pi's denylist (file tools, sandbox, memory capture) covers
1098
+ `.agenthub/archive` as it covers `.agenthub/state`. Turn snapshots leave the
1099
+ archive out as they leave the state out. Nothing deletes or prunes
1100
+ archives; removing one is a manual act.
1101
+
1102
+ To restore an archive, stop the hub, move the current state directory into
1103
+ the archive too (where git and the local worker's denylist cover it), then move
1104
+ the archive back:
1105
+
1106
+ ```bash
1107
+ ahub kill
1108
+ mv .agenthub/state .agenthub/archive/state-$(date -u +%Y%m%dT%H%M%SZ)
1109
+ mv .agenthub/archive/state-<UTC time> .agenthub/state
1110
+ ahub up
1111
+ ```
1112
+
1113
+ Only a person runs `ahub reset`: an agent shell is refused before anything
1114
+ happens, as for `kill`, `restart` and `upgrade`. It is refused, with nothing
1115
+ changed:
1116
+
1117
+ - while an upgrade or recovery operation is open (the machine's recovery lock,
1118
+ or a running hub that reports an unreleased recovery);
1119
+ - when no registration matches this project and state directory. A running
1120
+ hub must then be stopped with its matching CLI; a stopped project is
1121
+ registered again by `ahub up` (then `ahub reset --yes` stops it first). When
1122
+ the project is registered with another state directory, the error names
1123
+ `ahub --project <id> reset` instead;
1124
+ - when the hub's ownership cannot be verified, as `ahub kill` refuses it;
1125
+ - for `--all`, when the state directory is not `<root>/.agenthub/state`
1126
+ (`AGENTHUB_STATE_DIR` or a symlink elsewhere): archive such a directory by
1127
+ hand after `ahub kill`;
1128
+ - for `--all`, when `.agenthub/archive` is a symlink or not your own
1129
+ directory, or its `.gitignore` is not a regular file: anything with write
1130
+ access to the project could otherwise send `hub.db` out of it. Inspect and
1131
+ remove what is there, then run the reset again.
1132
+
1133
+ After stopping the hub the reset holds the project's registry claim, the one a
1134
+ daemon takes to run, until it is done, so no hub starts under it; a hub that
1135
+ started in between makes it stop with nothing reset; a crashed hub's leftover
1136
+ manifest does not count as one. If a step fails after the stop, the error says
1137
+ how far it got: rerun `ahub reset --yes` to finish a runtime reset (every step
1138
+ can run again), unless the error says the state cannot be read (a damaged
1139
+ `hub.db` or journal fails the same way every time): `ahub reset --all --yes`
1140
+ archives such a state directory as it is. A full reset names the archive once
1141
+ the state directory has moved.
1142
+ It never touches claude-mem: notes saved with `hub_remember` are shared memory,
1143
+ not hub state. Claude Code sessions attached to the hub lose their hub session;
1144
+ relaunch them with `ahub claude`.
1145
+
767
1146
  ## Orphaned daemons
768
1147
 
769
1148
  Daemons from before those guards, or daemons whose project directory was
@@ -792,21 +1171,21 @@ Rows without a live process are stale registrations; forget them with
792
1171
 
793
1172
  Upgrade running projects with the target release's own coordinator. It accepts
794
1173
  a running source on control protocol 9 (0.6.x), 10 (0.7.0 through 0.12.0),
795
- 11 (0.12.1 and 0.12.2), 12 (0.12.3), 13 (0.12.4 through 0.12.15) 14 (0.12.16) or 15 (0.12.17 through 0.12.19), and only
1174
+ 11 (0.12.1 and 0.12.2), 12 (0.12.3), 13 (0.12.4 through 0.12.15), 14 (0.12.16), 15 (0.12.17 through 0.12.19) or 16 (0.12.20 and 0.12.21), and only
796
1175
  a target on its own protocol, so the target's coordinator fits every supported
797
1176
  source and carries every recovery fix released up to it. Protocol 8 and older
798
1177
  (0.5.x and earlier) are refused as `manual-bootstrap-required`. Run from the
799
1178
  project directory, without replacing the global CLI first:
800
1179
 
801
1180
  ```bash
802
- bunx --package @staix/agent-hub@0.12.19 ahub upgrade --to 0.12.19 --dry-run
803
- bunx --package @staix/agent-hub@0.12.19 ahub upgrade --to 0.12.19 --yes
1181
+ bunx --package @staix/agent-hub@0.12.21 ahub upgrade --to 0.12.21 --dry-run
1182
+ bunx --package @staix/agent-hub@0.12.21 ahub upgrade --to 0.12.21 --yes
804
1183
  ```
805
1184
 
806
1185
  | Running now | Coordinator to use |
807
1186
  | --- | --- |
808
1187
  | 0.6.x (protocol 9) | the target's, through `bunx` as above |
809
- | 0.7.0 through 0.12.0 (protocol 10), 0.12.1 and 0.12.2 (protocol 11), 0.12.3 (protocol 12), 0.12.4 through 0.12.15 (protocol 13), 0.12.16 (protocol 14), 0.12.17 through 0.12.19 (protocol 15) | the target's, through `bunx` as above |
1188
+ | 0.7.0 through 0.12.0 (protocol 10), 0.12.1 and 0.12.2 (protocol 11), 0.12.3 (protocol 12), 0.12.4 through 0.12.15 (protocol 13), 0.12.16 (protocol 14), 0.12.17 through 0.12.19 (protocol 15), 0.12.20 and 0.12.21 (protocol 16) | the target's, through `bunx` as above |
810
1189
  | any supported source, with the installed CLI already at the target | `ahub upgrade` below, which is the same coordinator |
811
1190
  | 0.5.x or earlier (protocol 8 and older) | not supported: bootstrap by hand with the matching CLI |
812
1191
 
@@ -838,17 +1217,19 @@ projects first:
838
1217
 
839
1218
  ```bash
840
1219
  ahub restart --dry-run
841
- ahub upgrade --to 0.12.19 --dry-run
1220
+ ahub upgrade --to 0.12.21 --dry-run
842
1221
  ```
843
1222
 
844
1223
  Apply only after reviewing the plan:
845
1224
 
846
1225
  ```bash
847
1226
  ahub restart --yes
848
- ahub upgrade --to 0.12.19 --yes
1227
+ ahub upgrade --to 0.12.21 --yes
849
1228
  ahub recovery status <operation-id>
850
1229
  ahub recovery resume <operation-id>
851
1230
  ahub recovery abort <operation-id>
1231
+ ahub recovery dispose <operation-id> --fresh-session <peer> --reason <text>
1232
+ ahub recovery dispose <operation-id> --stop-and-archive --reason <text>
852
1233
  ```
853
1234
 
854
1235
  The coordinator commits only once the source is quiet: no turn running, no
@@ -904,6 +1285,20 @@ Kimi and local sessions may start fresh with preserved routing and task context.
904
1285
  A Claude session that never persisted a transcript (zero turns) also starts fresh:
905
1286
  there is nothing to resume, so the upgrade accepts the new session id; a session
906
1287
  with a transcript must come back with its original id.
1288
+ A Claude session that `ahub claude` did not launch in a recorded Orca terminal
1289
+ (a plain `claude`, or one whose launcher has ended) is planned as
1290
+ reconnect-only: the plan lists it under `reconnectOnly`, and the command prints
1291
+ `<project>: claude is reconnect-only` on stderr. Nothing in its terminal is
1292
+ closed or relaunched, a `claude-session.json` left by an earlier launch is never
1293
+ used as its target, and the plan does not show that record's id. Its plugin
1294
+ reconnects to the new hub by itself (the coordinator waits up to 90 seconds) and
1295
+ keeps the plugin version it started with until that Claude session restarts.
1296
+ The target release must read recovery waivers; staging refuses one that does
1297
+ not, before any runtime changes. Across a control-protocol change it cannot
1298
+ reconnect; the plan then names a
1299
+ blocker: end the session, or relaunch it with `ahub claude` in Orca, and make a
1300
+ new plan. A refused `--yes`, and every dry-run, prints each blocker on its own
1301
+ `ahub: blocker:` line before the final one.
907
1302
  Previously stopped projects stay stopped; only the reviewed running projects
908
1303
  are upgraded.
909
1304
 
@@ -911,6 +1306,170 @@ Do not run an upgrade with an incompatible active protocol, an unverified
911
1306
  terminal binding, or an unresolved operation lock. Dry-run performs no package,
912
1307
  plugin, daemon, or terminal mutation.
913
1308
 
1309
+ ### A partial operation
1310
+
1311
+ `ahub recovery status <operation-id>` reads the receipt without changing it:
1312
+ its phase, step and error; `runner`, which says whether a runner process holds
1313
+ the operation now (`running` with its pid, or `none`; `stale` marks a receipt
1314
+ that says running with no runner behind it); each project's phase and effect
1315
+ receipts (`closed:<peer>`, `restored:<peer>` as `done`, `pending` or `failed`);
1316
+ whether the shared plugin and the global CLI were installed; and `next`, the
1317
+ commands that apply now, read from every source the operation has not stopped
1318
+ yet (every error of `resume` ends with the same list). `resume` is left out
1319
+ when it can never get past: a source replaced by another instance, or a
1320
+ prepared source that stopped before any commit request. It shows no task or
1321
+ message text. `resume` does not
1322
+ start a second runner while one is alive. Every `next` entry and error names the
1323
+ operation's own coordinator, `bun <preserved source>/src/cli/main.js recovery
1324
+ ...`: in the middle of an upgrade the global `ahub` may still be the older
1325
+ release, whose `recovery` lacks these commands. Copy that command line. When
1326
+ an older coordinator started the operation, `dispose` is named from the running
1327
+ release, which has it. What
1328
+ each receipt allows, given what is live, is tabulated in the recovery spec
1329
+ (`docs/specs/2026-09-20-upgrade-recovery-design.md`, "Receipts, evidence and
1330
+ next actions").
1331
+
1332
+ - The source hold lapses after 10 minutes. A later `resume` prepares the same
1333
+ source again and checks its peers again, keeping the receipts, so no terminal
1334
+ is closed twice. It refuses, naming the next step, when the source daemon was
1335
+ replaced, when another operation holds it, or when a peer changed: a peer whose
1336
+ terminal the operation closed must stay closed, a session that joined since
1337
+ must end, every other one must keep its conversation. With nothing closed yet,
1338
+ `abort` cancels the operation (also once its hold has lapsed, or when another
1339
+ daemon or operation holds the source, which it leaves alone) so a new plan
1340
+ can be made, also after a source crashed before any commit was requested
1341
+ (`resume` then stops, as there is nothing to start from, and names abort, or
1342
+ stop-and-archive once anything was done); it
1343
+ refuses while a prepared source cannot be read (its hold may still stand),
1344
+ once a commit request may have been sent (or, for an operation from
1345
+ an older coordinator, which records no such thing, while a prepared source is
1346
+ not running), and once any project has effects the way out is
1347
+ stop-and-archive. `next` offers abort only where abort would succeed, and
1348
+ `resume` (and `--fresh-session`, which resume carries out) only where it can
1349
+ get past what is live: not past a replaced or missing source, a source that
1350
+ crashed before any commit request, a refusal at staging (the target release
1351
+ or the preserved source can never change back), a hub of another control
1352
+ protocol, a target that stopped after it started when the operation's
1353
+ coordinator is older than 0.12.21, or a stopped hub without the operation's
1354
+ restart snapshot.
1355
+ A second `resume` while its own hold still stands checks the peers again
1356
+ before it closes anything.
1357
+ - When a target hub dies after it started (a crash or a reboot), `resume`
1358
+ starts it again from the operation's snapshot while the project's state
1359
+ directory still holds it: it records the restart (status shows `restarts`),
1360
+ closes the terminals the dead hub's peers were restored into (or finds them
1361
+ gone), and relaunches those peers against the new hub, a session you had
1362
+ accepted instead of the original included. A launcher still running from
1363
+ the dead hub, or a session attached without an id, stops it until you end
1364
+ that session. A target that dies again in the same `resume` stops it with the
1365
+ restart count; look at its `hub.log` before resuming again. Without the
1366
+ snapshot, or for an operation started by a coordinator older than 0.12.21,
1367
+ the way out is stop-and-archive.
1368
+ - A Codex conversation comes back only when a rollout file naming its thread
1369
+ exists under `sessions/` of the store the restored terminal uses (the
1370
+ recorded `CODEX_HOME`, else `~/.codex`). Codex writes that file with the
1371
+ first message, so a thread the hub saw Codex start, with no turn on it since,
1372
+ is listed under `freshStart` and restarts as a new session, with nothing to
1373
+ lose; the command says so. A thread with turns, or one whose start the hub's
1374
+ log does not show (it was resumed, or the hub is older), blocks the plan when
1375
+ it has no rollout, and so does a store that cannot be read (unknown, not
1376
+ missing). The coordinator checks again before closing the terminal and before
1377
+ creating the new one. To continue without such a conversation, end that Codex
1378
+ session and close its Orca terminal, then `resume`; it stops at restoring
1379
+ Codex, where `--fresh-session codex` below is the way on.
1380
+ - A restored terminal runs its launch in a login shell, so the coordinator
1381
+ watches the launcher, not the terminal: once `ahub codex`, `ahub claude` or
1382
+ `ahub pi` has recorded itself there and its process is gone, or Orca no longer
1383
+ lists that terminal at all, the restoration failed (found within one 5-second
1384
+ wait). A launcher whose process cannot be read counts as running, never as
1385
+ gone. One that dies before recording itself leaves no record to read, and
1386
+ what happens next depends on whether Orca reports the bare login shell it
1387
+ returned to as TUI-idle, which is not verified. If it does, the coordinator
1388
+ stops at once with "original session restoration needs manual verification"
1389
+ (the terminal cannot be mapped to the session). If it does not, it stops with
1390
+ the same error after the 10-minute wait. Either way the receipt stays
1391
+ `pending`; close that terminal in Orca (or wait, if the session may still
1392
+ attach), then `resume`: with no live launcher and no attached session it
1393
+ records `restored:<peer>` `failed` and lists the failed-restoration choices.
1394
+ A `failed` receipt never counts as restored.
1395
+ `resume` settles a `pending` or `failed` receipt by what is live: the planned
1396
+ session attached is the restoration, a launcher still running is waited for,
1397
+ and no second terminal is opened while one may run. A target hub that cannot
1398
+ be read counts as unknown, never as "nothing attached", and so does a launcher
1399
+ record file that cannot be read: `resume` then stops without changing any
1400
+ receipt; when the record file is what cannot be read, inspect it and move it
1401
+ aside, then `resume`. A record file that parses but holds a row that cannot
1402
+ be evaluated (a hand edit, a damaged or a future layout) is read the same way
1403
+ for the peer that row may belong to, and `ahub codex`, `ahub claude` and
1404
+ `ahub pi --mode tui` write such rows back unchanged: fix or remove that row,
1405
+ then `resume`. A launcher recorded by 0.12.12 or older is read with its own
1406
+ unpinned signature too, so it counts as running when this command runs in
1407
+ the same time zone and locale as the launcher did. `resume` is refused while a stop-and-archive is partway. A lifecycle command refused by the lock names
1408
+ the operation's own `status` command, which lists what to do next.
1409
+ - `ahub recovery dispose <operation-id> --fresh-session <peer> --reason <text>`
1410
+ applies to a Codex or Claude peer whose restoration failed (Pi is refused: a
1411
+ restored hub resumes Pi's recorded session). It records the lost session or
1412
+ thread id (status shows it under `lostContinuity`) and resumes: the peer
1413
+ starts without its old conversation, and the rest is verified as usual.
1414
+ - `ahub recovery dispose <operation-id> --stop-and-archive --reason <text>`
1415
+ abandons the operation. It checks every project first and refuses while a
1416
+ hub is unreachable, starting, stopping or of another control protocol; `next`
1417
+ then says `wait until <project>'s hub settles, then ... recovery status
1418
+ <id>` instead of offering it. A hub that never answers (its pid is alive,
1419
+ nothing replies): take the `pid` from that project's
1420
+ `.agenthub/state/status.json`, check with `ps -p <pid> -o command=` that it
1421
+ is that project's hub daemon, end it by hand, then run `status` again. From
1422
+ 0.12.21 a hub's manifest and registry claim and a runner's record carry the
1423
+ owner's process signature, so a pid that a reboot handed to another process
1424
+ reads as gone by itself. Records written by 0.12.20 or older carry none and
1425
+ are judged by their pid alone: if that pid now belongs to another process,
1426
+ move the stale `status.json` and `hub.pid` aside together (with only one of
1427
+ them gone the other still reads as an owner), then run `status` again;
1428
+ likewise when `next` says `wait: runner <pid>` and that pid is not an `ahub
1429
+ recovery-run` process, move the operation's
1430
+ `~/.agenthub/recovery/<id>.json.runner.db` aside, then run `status` again.
1431
+ An unsigned registry claim of such a hub, whose pid now belongs to another
1432
+ process, still reads as starting, and no command clears it: after `ps -p
1433
+ <pid> -o command=` shows that the process is not an `ahub ... daemon` of that
1434
+ project, clear it with `sqlite3 ~/.agenthub/registry.db "UPDATE projects SET
1435
+ instance_id = NULL, pid = NULL, claimed_at = NULL WHERE id = '<project id>'
1436
+ AND pid = <pid>"` (`ahub projects --json` shows the id). A
1437
+ project whose directory is
1438
+ gone is only recorded (`ahub doctor --orphans` lists a hub left running
1439
+ there). It records its decision before it acts. It releases a source hold of
1440
+ its own (that hub keeps running), stops a target hub it started and has not
1441
+ released (one an older coordinator started is stopped at that release's
1442
+ control protocol), moves that operation's
1443
+ `restart.json` aside as `restart.abandoned.<hash>.json` and leaves any other
1444
+ hub running; only then is the operation recorded `cancelled` and the lock
1445
+ released. If it fails partway it keeps the lock, records what it did, and
1446
+ `resume` refuses until you run it again. Once a stop-and-archive is recorded,
1447
+ only the command `next` names continues it. On an operation whose
1448
+ coordinator predates these commands it also marks the receipt `schema: 2`,
1449
+ which every older release refuses before it acts: an older global `ahub
1450
+ recovery resume` still prints "scheduled", but its runner exits without
1451
+ touching the operation, and an older `ahub recovery abort` refuses with
1452
+ "unsupported operation receipt". Queued messages are not delivered
1453
+ from the archived file; a 0.7.0 or later hub reloads the queues it kept in
1454
+ `hub.db` when it starts again. Terminals it closed stay closed: start those
1455
+ sessions again by hand. Replacement terminals it opened stay open but have
1456
+ no hub: Claude's plugin reconnects once a hub runs, Codex needs `ahub codex`
1457
+ again.
1458
+ Start a project whose target ran with that release's CLI (`bunx --package
1459
+ @staix/agent-hub@<version> ahub up`), since it may have changed the task
1460
+ database. The global CLI was not promoted.
1461
+ - Both choices are for a person in a terminal; an agent shell is refused, and
1462
+ the reason is kept in the operation's audit. Run each with the command
1463
+ `next` or the error prints: the operation's own coordinator, or the release
1464
+ you are running when that coordinator predates these commands (its `resume`
1465
+ still runs the old runner, and `next` says what that runner cannot do).
1466
+ `--fresh-session` never applies to such an operation: its runner never
1467
+ records a restoration as `failed`.
1468
+ `--fresh-session` is not offered when the target release cannot read recovery
1469
+ waivers, since it could never release a new session; `status` says so in
1470
+ `freshSession`. A runner record that cannot be read shows as `unknown`, and
1471
+ `resume`, `abort` and `dispose` are refused until it can be read.
1472
+
914
1473
  ### After an unplanned stop
915
1474
 
916
1475
  While it runs, the hub keeps each attached peer's session identity in
@@ -1037,9 +1596,46 @@ PII calls require a positively confirmed campus gateway immediately before trans
1037
1596
  PII turns never enter shared history, memory capture, or progress observation.
1038
1597
  The `route`, `advisor`, `progress` and `stuck` events contain identifiers and aggregates only.
1039
1598
 
1599
+ Session-aware stay/switch (#197) covers `hub/auto` and `stage` routes. The top-level
1600
+ `routing.toml` key `stay_switch` is `off`, `shadow` (default: route events record what the
1601
+ planner would do, routing unchanged) or `enforce`. Enforced, a hard override (compaction,
1602
+ critical failure, repeated failure) escalates at once; a tool-result turn keeps the
1603
+ session's tier; other changes wait for the next user turn, and a de-escalation whose
1604
+ conversation is larger than `max_switch_prefill_tokens` (default 32000) stays; a PII route
1605
+ has no such bound, so its size never shows in a decision. A capable
1606
+ hold then lasts at least until the next user turn; `hold_turns` can extend it past that
1607
+ turn. A tier whose backend
1608
+ cannot hold the conversation is never chosen; no route summarizes or trims the history.
1609
+
1040
1610
  Pi exposes `hub/auto` for stage routing when available. Fixed `dgx/coding`, `dgx/fast`
1041
1611
  and `mlx/fast` aliases still pin the backend. Automatic MLX selection admits the complete
1042
1612
  input, tool schemas and requested output within the configured context window.
1613
+ A `hub/auto` request bound for MLX has two candidates, MLX and its `dgx/fast` fallback; load
1614
+ and cooldowns only change their order, never remove one (#199). A fixed alias (`mlx/fast`,
1615
+ `pi_backend = "mlx"`) keeps its backend first as before. `dgx/fast` goes first when MLX is
1616
+ cooling down (route event `source: "cooldown"`). Load moves are opt-in until they are
1617
+ measured: with `[pi] efficient_wait_ms` set (0 to 119999; 0 tries once; read when the hub
1618
+ first starts Pi, like `dgx_max_context_tokens`, so a change takes a hub restart), a request
1619
+ whose MLX slot is still busy after that wait goes to `dgx/fast` first (`source: "load"`), and
1620
+ so does an enforced tool loop pinned to it (`source: "pin"`); without it nothing moves for load. `dgx/fast`
1621
+ never goes first while it is cooling down, or while its own last dispatch failed in any way,
1622
+ an error status or a failed stream included, until it succeeds or 30 s pass (`ahub status`:
1623
+ `last dispatch failed, no load moves until ...`). No load move or pin happens while the request
1624
+ runs under an execution budget, and enforced, a load move happens only at a user turn. A moved
1625
+ attempt gets 15 s to its response headers after the gateway lookup (OmniRoute's own probe,
1626
+ up to two 4 s rounds when no gateway is cached) and is then abandoned for MLX, which serves
1627
+ with the usual slot wait. The bound ends with the headers: a stream that stalls or fails
1628
+ after them is not retried on MLX, and a failed one marks `dgx/fast` failing (a client that
1629
+ disconnects does not). Each backend is tried once per request, so with moves on, a moved attempt
1630
+ cut at 15 s followed by an MLX slot that stays busy past 120 s fails the request (502), where
1631
+ without moves the request would have waited for MLX and then had `dgx/fast` with its full
1632
+ deadline; this is part of what the opt-in accepts until #199 AC5 measures it. After three
1633
+ consecutive transport or startup failures outside a cooldown (a failure more than 10 min after
1634
+ the last counted one starts the count over), a relay alias cools down for 30 s, doubling up to
1635
+ 5 min. A busy MLX slot and timeouts cut short by an execution budget never count; any HTTP
1636
+ answer, a success or an error status, proves the transport works and ends the streak and the
1637
+ cooldown. `ahub status` shows `cooling down until ...` on the backend line and `events.jsonl`
1638
+ records `cooldown` events.
1043
1639
  Progress judgements suggest reassignment; they never change task ownership.
1044
1640
 
1045
1641
  ## Disable local MLX while keeping remote auto routing
@@ -1112,8 +1708,8 @@ The hooks return no facts in advisory mode. A native Stop establishes an idle
1112
1708
  boundary; a subsequent PreToolUse marks activity. A delivery acknowledgement or
1113
1709
  task approval does not establish native idle. Restart the daemon and relaunch
1114
1710
  Claude after enabling the sweep so its session receives the hooks. A caller's
1115
- `--settings` still wins: the launcher warns that native idle observation is off,
1116
- and the sweep cannot verify that Claude session between turns.
1711
+ `--settings` keeps them: the launcher adds the hub's hooks to the caller's and
1712
+ warns only that the status line tee is off.
1117
1713
 
1118
1714
  `auto_reassign: true` explicitly allows an available alternative owner selected
1119
1715
  under the existing routing, role and PII constraints to receive the task at step
@@ -1238,3 +1834,28 @@ Quota pause and task handoff are separate; a context checkpoint neither pauses
1238
1834
  nor hands work over. Continue normally or deliberately restart into a fresh
1239
1835
  session with your chosen checkpoint as preface. No automatic restart or native
1240
1836
  compaction override is performed.
1837
+
1838
+ ## Pi tool refusals, exit causes and idle automatic restart
1839
+
1840
+ Pi hub-task/conductor refusals proven before any task or board effect return
1841
+ their original `error:` text and are recorded as completed tool receipts.
1842
+ PII screening and its telemetry record may precede the refusal. An uncertain
1843
+ post-effect or storage failure remains pending: inspect it before repeating an
1844
+ action. Old pending receipts carry no proof of a past refusal and stay pending
1845
+ across startup. Conductor assignment and escalation entry checks explicitly opt
1846
+ into refusal typing; internal escalation after a saved review remains generic.
1847
+
1848
+ Pi exit records show code or signal, turn/tool activity, whether teardown was
1849
+ expected, startup status, and the last tool's name without arguments. An
1850
+ unobserved native-terminal exit records unknown OS status. The console notice
1851
+ gives the next action, usually `ahub pi`, or says why automatic start is held.
1852
+ A requested replacement says the new owner is starting; internal failure
1853
+ teardown directs session inspection rather than claiming a person requested it.
1854
+
1855
+ With `pi.auto_start`, an unexpectedly idle headless Pi is resumed on its
1856
+ verified persisted session once. The 60-second retry window rearms when the
1857
+ attempt settles; a second exit inside that window stays offline. Missing or
1858
+ invalid session history has no fresh fallback; inspect it before running
1859
+ `ahub pi`. Active turns/tools, failed startup, a superseded owner, native
1860
+ terminal exits, requested shutdown and recovery operations suppress this
1861
+ restart. Crash recovery's #66 recorded-session/fresh-start choices are unchanged.