@staix/agent-hub 0.12.18 → 0.12.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (66) hide show
  1. package/CHANGELOG.md +21 -0
  2. package/README.md +3 -3
  3. package/docs/agent-notes/adapters.md +2 -0
  4. package/docs/agent-notes/bus.md +1 -0
  5. package/docs/agent-notes/daemon.md +2 -1
  6. package/docs/agent-notes/models.md +3 -0
  7. package/docs/agent-notes/tasks.md +3 -2
  8. package/docs/events.md +4 -1
  9. package/docs/operations.md +419 -17
  10. package/docs/quickstart.md +1 -1
  11. package/docs/security.md +18 -11
  12. package/docs/smoke.md +91 -0
  13. package/docs/specs/2026-09-19-agent-hub-design.md +412 -3
  14. package/docs/specs/2026-09-20-pi-local-models-design.md +2 -0
  15. package/docs/specs/2026-09-20-upgrade-recovery-design.md +287 -0
  16. package/docs/specs/2026-10-04-switchyard-source-port-design.md +10 -0
  17. package/docs/verified.json +13 -13
  18. package/package.json +1 -1
  19. package/plugins/agent-hub/.claude-plugin/plugin.json +1 -1
  20. package/plugins/agent-hub/server.js +63 -25
  21. package/src/adapters/claude-channel.ts +61 -20
  22. package/src/adapters/codex-appserver.ts +3 -0
  23. package/src/adapters/local-worker.ts +4 -0
  24. package/src/cli/console-state.ts +325 -102
  25. package/src/cli/console.ts +44 -23
  26. package/src/cli/help.ts +120 -0
  27. package/src/cli/identity.ts +3 -3
  28. package/src/cli/launch.ts +2 -0
  29. package/src/cli/main.ts +171 -75
  30. package/src/cli/preview.ts +1 -0
  31. package/src/cli/reset.ts +194 -0
  32. package/src/cli/status-lines.ts +9 -5
  33. package/src/cli/terminal-recovery.ts +140 -36
  34. package/src/cli/upgrade-runtime.ts +274 -56
  35. package/src/cli/upgrade.ts +366 -41
  36. package/src/hub/ask.ts +1 -1
  37. package/src/hub/board.ts +15 -7
  38. package/src/hub/bus.ts +60 -4
  39. package/src/hub/child-process.ts +1 -0
  40. package/src/hub/conductor.ts +38 -6
  41. package/src/hub/control-client.ts +5 -3
  42. package/src/hub/daemon.ts +70 -12
  43. package/src/hub/events.ts +7 -1
  44. package/src/hub/execution-budget.ts +15 -6
  45. package/src/hub/hub-tools.ts +4 -4
  46. package/src/hub/inference.ts +136 -1
  47. package/src/hub/lifecycle.ts +7 -1
  48. package/src/hub/peers.ts +2 -0
  49. package/src/hub/recovery-store.ts +51 -3
  50. package/src/hub/report.ts +43 -1
  51. package/src/hub/restart.ts +25 -0
  52. package/src/hub/routing.ts +48 -16
  53. package/src/hub/snapshots.ts +3 -2
  54. package/src/hub/tasks.ts +236 -83
  55. package/src/local/deny.ts +2 -1
  56. package/src/models/mlx.ts +11 -8
  57. package/src/models/ollama.ts +2 -2
  58. package/src/models/relay.ts +190 -42
  59. package/src/models/route/relay-selector.ts +43 -31
  60. package/src/models/route/runtime.ts +17 -7
  61. package/src/models/route/signals.ts +12 -1
  62. package/src/models/route/stage.ts +57 -3
  63. package/src/omniroute/client.ts +1 -1
  64. package/src/ui/index.html +1 -0
  65. package/templates/AGENTS.block.md +1 -1
  66. package/templates/routing.toml +8 -0
@@ -1,8 +1,25 @@
1
1
  # Operations guide
2
2
 
3
- This guide describes ahub 0.12.18 and control protocol 15. Live verification
3
+ This guide describes ahub 0.12.20 and control protocol 16. Live verification
4
4
  results and remaining prerequisites are recorded separately in [the smoke ledger](smoke.md).
5
5
 
6
+ ## Command help
7
+
8
+ `ahub help`, `ahub --help` and `ahub -h` print the commands grouped by section;
9
+ `ahub help <command>` prints only that command's entries, and so does
10
+ `ahub <command> -h` (or `--help`) when the flag is the only argument; anywhere else
11
+ it is an ordinary argument, and `ahub claude` and `ahub codex` pass it on to the
12
+ agent. An unknown command
13
+ prints `unknown command "<name>"; run ahub help` and exits non-zero. Usages start
14
+ at two spaces and descriptions at one fixed column, wrapped at word boundaries to
15
+ the terminal width, read as at least 80 and at most 100 columns.
16
+
17
+ Help uses the console palette: section headings bold cyan, `ahub <command>` words
18
+ cyan, arguments and descriptions the terminal's default foreground. It has no
19
+ `--color` flag: color is on only when standard output is a terminal, `TERM` is
20
+ not `dumb` and `NO_COLOR` is empty or absent. Pipes and redirects get plain text
21
+ without escape sequences, laid out for 80 columns.
22
+
6
23
  ## Operator console and panels
7
24
 
8
25
  `ahub console` combines the existing tail stream with peer, quota, approval-age
@@ -17,11 +34,59 @@ approval titles are terminal-only; expiry and answers from another console remov
17
34
  the pending item. Approval audit records contain id, peer, option kind, response
18
35
  time and answering surface, without the title.
19
36
 
20
- Tab toggles stream and panels; `ahub console --panels` starts in panels. Peers,
21
- Approvals, Tasks, Queue and Events support arrow keys or j/k, Enter for detail,
22
- Escape to return and `?` for help. `:` enters a command. Assignment, delivery
23
- resolution and allow decisions require confirmation; delivery resolution requires
24
- a reason. Tasks use the same public redaction as the board. Panels need at least
37
+ Pi's write, edit, bash and git write requests also offer `Always allow <tool>
38
+ until Pi restarts`. It allows later calls of that tool by the running Pi without
39
+ asking, under the same path guard and sandbox; a Pi restart or replacement, or a
40
+ hub restart, drops it. The hub log records each granted call by tool name only.
41
+ The dashboard can only deny Pi's requests.
42
+
43
+ Tab toggles stream and panels; `ahub console --panels` starts in panels. A panel
44
+ screen has a header (the project's directory name, then the tabs), the panel and
45
+ a three-line footer, separated by ASCII `-` rules; the stream has one rule above
46
+ its footer. The active tab is bracketed, as in `[2 Approvals]`, so it reads
47
+ without color. The Approvals tab counts pending requests and the Queue tab
48
+ counts `needs_review` deliveries, in the attention and failure tones.
49
+
50
+ Peers, Approvals, Tasks and Queue are tables with a header row. A column is as
51
+ wide as its widest cell, up to a third of the terminal; the last column takes
52
+ the rest and is cut with `...`.
53
+
54
+ | Panel | Columns |
55
+ |---|---|
56
+ | Peers | PEER STATE LINK Q ! REVIEW PAUSE QUOTA MODEL |
57
+ | Approvals | ID PEER LEFT TITLE |
58
+ | Tasks | ID STATE OWNER REVIEWER CLASS AGE TITLE |
59
+ | Queue | ID PEER STATE REV AGE |
60
+
61
+ Zero counters and unknown values read `-`. Durations read `45s`, `12m`, `3h05m`
62
+ or `2d03h`; quota resets and budget pauses read `in 2h13m`, and a pause by the
63
+ console user reads `user`. When a peer's status carries #205's optional
64
+ `toolsOnly` field (a Claude session attached without channel pushes), LINK reads
65
+ `tools-only`. Below the Approvals table, a rule and the selected
66
+ request: its title, its allow options numbered as `a` offers them, and `d deny`.
67
+ A title too long for the panel is marked `(more)` in the label column; Enter
68
+ shows it whole. `d deny` always shows; options that do not fit are counted on a
69
+ `(more)` line. The first request is selected when it arrives; `[` and `]` (or
70
+ j/k in the panel) select another. `a`, `d` and `v` act on the selected request
71
+ only: when it closes, the selection clears with a notice instead of moving to
72
+ another request, and a request that arrives later waits for `[` or `]`. Moving
73
+ the selection cancels a pending allow choice or confirmation; the choice prompt
74
+ names its request. Agent-written text below a field, and option names in
75
+ prompts, are quoted unless they are a single word starting with a letter, so
76
+ punctuation, a newline or a bare number in them cannot imitate the console's own
77
+ structure. Under the `?` key table only `?`, Escape and `q` act.
78
+
79
+ Arrow keys or j/k move, Enter opens a detail of labeled fields (relative times,
80
+ never JSON), Escape returns, and `?` shows the keys grouped by panel; in the
81
+ stream, `?` prints that table and `v` prints the selected request as its push
82
+ showed it. The footer's last line lists only the keys that act in the current
83
+ mode, panel and state. A notice shows on the footer's second line until the next
84
+ key or for about ten seconds: errors and refusals (a refused command, a request
85
+ error, the size refusal) in the failure tone, notices that only inform (a closed
86
+ or missing selection) in the attention tone. With a detail open, `p`, `r` and `a`
87
+ act on the item the detail shows. `:` enters a command; its output lines
88
+ start at column 4, as message bodies do. Assignment, delivery resolution and
89
+ allow decisions require confirmation; delivery resolution requires a reason. Tasks use the same public redaction as the board. Panels need at least
25
90
  80 columns by 24 rows; smaller terminals stay in stream mode. Task and queue
26
91
  polling runs only while the corresponding panel is visible. Leaving restores
27
92
  the terminal and returning from panels replays the bounded stream buffer.
@@ -39,7 +104,7 @@ management or the final reset. Invalid values fail before connecting.
39
104
  | Success and availability | Green | Idle peer, approved task |
40
105
  | Waiting and attention | Yellow | Busy/paused peer, pending approval, confirmation, review/ready task, important priority |
41
106
  | Failure and intervention | Red | Failed/check-failed task, undeliverable/overflow, `needs_review` queue, denial request or expired/cancelled approval |
42
- | Metadata | Bright black | Ages and remaining approval time |
107
+ | Metadata | Bright black | Ages, remaining approval time and rules |
43
108
 
44
109
  Offline peers, primary titles, action details and message body lines keep the
45
110
  terminal's default foreground. Labels and prompts remain readable without
@@ -299,7 +364,10 @@ the routing configuration.
299
364
  A review request carries a checklist: map the changed signatures and call sites
300
365
  to the task's plan (without one, to its detail, which the request then
301
366
  includes), read the check result, and list what is
302
- unmet (`hub_review` takes `unmet`, `ahub review ... --unmet <item>`). The hub
367
+ unmet (`hub_review` takes `unmet`, `ahub review ... --unmet <item>`). A reviewer
368
+ that missed the request, and the owner, read the done summary and check result
369
+ with `hub_task_show {id}`: the task's public view with its history, as the
370
+ conductor sees it, and only a stub for a PII task. Other peers are refused. The hub
303
371
  records how each review turned out, per implementer, reviewer and class:
304
372
  approved; caught (changes were requested and the owner's redo was approved);
305
373
  contradicted (within a week, work on the same file or symbol failed its check
@@ -321,7 +389,8 @@ one line with its next message, the plan included, at no turn of its own. When a
321
389
  task is done (after its check passes, when one is configured), the owners of
322
390
  open tasks on the same paths or symbols get a message with the changed files,
323
391
  the plan's signatures and the first line of the summary, each left out when it
324
- matches a PII pattern; nobody else does. PII
392
+ matches a PII pattern (the summary also when `signals.pii_screen = "local"` did
393
+ not clear it); nobody else does. PII
325
394
  tasks are left out on both sides.
326
395
 
327
396
  A task can wait for others: `hub_task_propose` takes `after: [ids]` (`ahub task
@@ -334,8 +403,36 @@ to the next run.
334
403
  `ahub board --ready` and
335
404
  `hub_task_list {ready: true}` list the proposed tasks with nothing left to wait
336
405
  for. Dependencies are fixed when a task is proposed and can only name tasks that
337
- already exist, so they cannot form a cycle. A waiting task cannot name an owner;
338
- use `ahub task assign` once it is ready. An owner offline longer than `tasks.release_after_min`
406
+ already exist, so they cannot form a cycle.
407
+
408
+ An owner named for a task that waits (`owner` with `after`, `--owner` with
409
+ `--after`) is its reserved owner: the task still waits, and when it is ready
410
+ routing offers it to that peer first, not to the first idle peer of the class.
411
+ A reserved owner that is offline, paused, failing, not attached or excluded is
412
+ passed over: routing proceeds as usual, the console and hub.log say which
413
+ reservation was passed over and why, and the assignment's history note keeps
414
+ it, unless it is no news (the reserved peer is the owner the task moves away
415
+ from, or it already refused the task). A peer the task was declined for (by
416
+ itself or by the console) or escalated away from after repeated
417
+ changes_requested or by hand stays excluded on every later reroute; an owner
418
+ released as gone, or moved by the hub after a failed delivery or inference, is
419
+ excluded only from that move and may get the task again from its reservation
420
+ later. Your own `ahub task assign` is not blocked by those exclusions, and it
421
+ drops the agent's reservation. The PII constraint and capability limits
422
+ still apply, so a PII task goes to `local` or nobody whoever was reserved.
423
+ `ahub task assign` on a task that waits changes its reserved owner instead of
424
+ handing it over. `ahub route explain <id>` names the reserved owner; once the
425
+ task is ready it also says whether routing would honor the reservation now, and
426
+ why not.
427
+
428
+ The peer that proposed a task may redirect it with `hub_task_assign` while it
429
+ is `proposed` and nobody ever accepted it, without the conductor role (handing
430
+ it to another peer needs `assign` when the proposer has a capabilities list).
431
+ Once it was accepted, even if a decline or release later put it back in
432
+ `proposed`, or after a person assigned or reserved it from the console (also
433
+ once the hub carried that out), only the conductor and the console move it.
434
+
435
+ An owner offline longer than `tasks.release_after_min`
339
436
  (default 30, `0` turns it off) in `.agenthub/config.json` loses its open tasks
340
437
  to a peer routing can give them to; with nobody to take them they stay, and a
341
438
  paused peer or a hub in a recovery operation is left alone.
@@ -387,6 +484,13 @@ ahub report --by task # tokens, turns and wall time per task and class
387
484
  ahub export --since 24h # the raw events as JSON lines, for your own analysis
388
485
  ```
389
486
 
487
+ Per peer and model route, `ahub report` also counts model changes between consecutive
488
+ decisions, how many of them landed inside tool loops and, with `stay_switch` on, sessions
489
+ and the planner's switches (#197). With `stay_switch = "off"` session boundaries are not
490
+ recorded; `shadow` routes exactly as `off` does, so it is the baseline to compare `enforce` with.
491
+ A route event records the decision, so a load-moved request that its MLX fallback served counts
492
+ as a change to `dgx/fast`; the relay's request journal records which backend served it.
493
+
390
494
  `ahub report` counts the same overlap warnings as `scripts/overlaps.ts`, from the
391
495
  structured events instead of log lines. `--by task` uses the task each usage and
392
496
  token record was attributed to when it was written: the delivery that started the
@@ -734,6 +838,30 @@ the system marks it `needs_review`. Establish evidence before choosing
734
838
  disconnects, cancellation, and partial effects are uncertain; do not replay
735
839
  them automatically.
736
840
 
841
+ ### Claude sessions without channel pushes
842
+
843
+ Only `ahub claude` starts Claude Code with the development-channel flag. A
844
+ Claude session started another way (plain `claude`, an IDE) keeps the hub tools
845
+ but cannot show pushes, so it attaches tools-only (protocol 16, issue #205):
846
+
847
+ - Its messages stay queued. `ahub status` shows `queued N` and `tools-only:
848
+ messages wait for hub_inbox; for pushes restart Claude with ahub claude`; the
849
+ console footer shows `tools-only: ahub claude` and the dashboard shows the
850
+ status line.
851
+ - The session reads them with `hub_inbox`, at most ten messages at a time plus
852
+ the hub's recall note when one waits (it says how many still wait). Each read is one `completed` delivery (`read through
853
+ hub_inbox` in `ahub queue list`); nothing is ever `accepted`, and the session
854
+ has no `hub_delivery_done`. A read the model never saw (the tool call was
855
+ cancelled, the plugin died) cannot be retried: `ahub queue show <id>` of the
856
+ `completed` row prints the messages to send again.
857
+ - Whatever holds pushes holds `hub_inbox` too (`needs_review`, recovery,
858
+ `ahub pause`, a budget pause, a conductor hold): it reads nothing and says
859
+ which hold applies.
860
+ - Restarting Claude with `ahub claude` takes the peer over and pushes what still
861
+ waits; the plain session stands by. The other way round, a plain session
862
+ started while an `ahub claude` session holds the peer stands by and attaches
863
+ only after that session leaves.
864
+
737
865
  ## Graceful shutdown
738
866
 
739
867
  Before stopping, let active turns and approvals settle when possible:
@@ -758,6 +886,93 @@ so a restarted hub never launches beside a survivor. A hub whose project root
758
886
  or state directory was deleted stops itself within about 10 seconds; the
759
887
  dashboard manager does the same when its home directory vanishes.
760
888
 
889
+ ## Resetting a project's hub
890
+
891
+ `ahub reset` returns a project's hub to a clean state. Without `--yes` it is a
892
+ dry run: it lists what it would settle, clear, archive and keep, by delivery
893
+ id, peer name and count, never task or message text, and changes nothing.
894
+ With `--yes` it first stops a running hub the way `ahub kill` does, then acts.
895
+
896
+ ```bash
897
+ ahub reset # dry run, runtime scope
898
+ ahub reset --yes # runtime reset
899
+ ahub reset --all # dry run, full reset
900
+ ahub reset --all --yes # full reset
901
+ ```
902
+
903
+ The runtime reset (the default):
904
+
905
+ - discards every `queued` and `needs_review` delivery with reason `reset`
906
+ through the delivery journal, one `resolution_history` entry each. Work
907
+ that was dispatching or accepted when the hub stopped is held as
908
+ `needs_review` first, so it is discarded too;
909
+ - clears manual peer holds (`ahub pause`), budget pauses and conductor holds;
910
+ - drops the agent session resume pointers `sessions.json`,
911
+ `claude-session.json` and `claude-context.json`, so the next launch starts
912
+ new agent sessions. Pi transcripts under `pi-sessions/` stay;
913
+ - removes the manifest (`status.json`, `control-token`, `hub.pid`) a hub left
914
+ when it did not stop cleanly, once no process behind it is alive;
915
+ - keeps the board (tasks, reviews, outcomes, turns, touches), `hub.log`,
916
+ `events.jsonl` and `cli-audit/`, recovery records, execution budgets,
917
+ configuration and files the hub does not own.
918
+
919
+ The full reset (`--all`) moves the whole state directory, unchanged, to
920
+ `.agenthub/archive/state-<UTC time>/` (mode 0700) and starts an empty state
921
+ directory holding only `project.json`, so the project keeps its id and
922
+ registration; the next `ahub up` starts with an empty board and queue.
923
+ `.agenthub/config.json`, `config.local.json` and `routing.toml` live outside
924
+ the state directory and are not touched. The archive keeps the task text,
925
+ PII included, so it stays in the project: the reset writes
926
+ `.agenthub/archive/.gitignore` (`*`) so git ignores it, and the local
927
+ worker's and Pi's denylist (file tools, sandbox, memory capture) covers
928
+ `.agenthub/archive` as it covers `.agenthub/state`. Turn snapshots leave the
929
+ archive out as they leave the state out. Nothing deletes or prunes
930
+ archives; removing one is a manual act.
931
+
932
+ To restore an archive, stop the hub, move the current state directory into
933
+ the archive too (where git and the local worker's denylist cover it), then move
934
+ the archive back:
935
+
936
+ ```bash
937
+ ahub kill
938
+ mv .agenthub/state .agenthub/archive/state-$(date -u +%Y%m%dT%H%M%SZ)
939
+ mv .agenthub/archive/state-<UTC time> .agenthub/state
940
+ ahub up
941
+ ```
942
+
943
+ Only a person runs `ahub reset`: an agent shell is refused before anything
944
+ happens, as for `kill`, `restart` and `upgrade`. It is refused, with nothing
945
+ changed:
946
+
947
+ - while an upgrade or recovery operation is open (the machine's recovery lock,
948
+ or a running hub that reports an unreleased recovery);
949
+ - when no registration matches this project and state directory. A running
950
+ hub must then be stopped with its matching CLI; a stopped project is
951
+ registered again by `ahub up` (then `ahub reset --yes` stops it first). When
952
+ the project is registered with another state directory, the error names
953
+ `ahub --project <id> reset` instead;
954
+ - when the hub's ownership cannot be verified, as `ahub kill` refuses it;
955
+ - for `--all`, when the state directory is not `<root>/.agenthub/state`
956
+ (`AGENTHUB_STATE_DIR` or a symlink elsewhere): archive such a directory by
957
+ hand after `ahub kill`;
958
+ - for `--all`, when `.agenthub/archive` is a symlink or not your own
959
+ directory, or its `.gitignore` is not a regular file: anything with write
960
+ access to the project could otherwise send `hub.db` out of it. Inspect and
961
+ remove what is there, then run the reset again.
962
+
963
+ After stopping the hub the reset holds the project's registry claim, the one a
964
+ daemon takes to run, until it is done, so no hub starts under it; a hub that
965
+ started in between makes it stop with nothing reset; a crashed hub's leftover
966
+ manifest does not count as one. If a step fails after the stop, the error says
967
+ how far it got: rerun `ahub reset --yes` to finish a runtime reset (every step
968
+ can run again), unless the error says the state cannot be read (a damaged
969
+ `hub.db` or journal fails the same way every time): `ahub reset --all --yes`
970
+ archives such a state directory as it is. A full reset names the archive once
971
+ the state directory has moved.
972
+ It never touches claude-mem: notes saved with `hub_remember` are shared memory,
973
+ not hub state. Claude Code sessions attached to the hub lose their hub session;
974
+ relaunch them with `ahub claude`.
975
+
761
976
  ## Orphaned daemons
762
977
 
763
978
  Daemons from before those guards, or daemons whose project directory was
@@ -786,21 +1001,21 @@ Rows without a live process are stale registrations; forget them with
786
1001
 
787
1002
  Upgrade running projects with the target release's own coordinator. It accepts
788
1003
  a running source on control protocol 9 (0.6.x), 10 (0.7.0 through 0.12.0),
789
- 11 (0.12.1 and 0.12.2), 12 (0.12.3), 13 (0.12.4 through 0.12.15) 14 (0.12.16) or 15 (0.12.17 and 0.12.18), and only
1004
+ 11 (0.12.1 and 0.12.2), 12 (0.12.3), 13 (0.12.4 through 0.12.15), 14 (0.12.16), 15 (0.12.17 through 0.12.19) or 16 (0.12.20), and only
790
1005
  a target on its own protocol, so the target's coordinator fits every supported
791
1006
  source and carries every recovery fix released up to it. Protocol 8 and older
792
1007
  (0.5.x and earlier) are refused as `manual-bootstrap-required`. Run from the
793
1008
  project directory, without replacing the global CLI first:
794
1009
 
795
1010
  ```bash
796
- bunx --package @staix/agent-hub@0.12.18 ahub upgrade --to 0.12.18 --dry-run
797
- bunx --package @staix/agent-hub@0.12.18 ahub upgrade --to 0.12.18 --yes
1011
+ bunx --package @staix/agent-hub@0.12.20 ahub upgrade --to 0.12.20 --dry-run
1012
+ bunx --package @staix/agent-hub@0.12.20 ahub upgrade --to 0.12.20 --yes
798
1013
  ```
799
1014
 
800
1015
  | Running now | Coordinator to use |
801
1016
  | --- | --- |
802
1017
  | 0.6.x (protocol 9) | the target's, through `bunx` as above |
803
- | 0.7.0 through 0.12.0 (protocol 10), 0.12.1 and 0.12.2 (protocol 11), 0.12.3 (protocol 12), 0.12.4 through 0.12.15 (protocol 13), 0.12.16 (protocol 14), 0.12.17 and 0.12.18 (protocol 15) | the target's, through `bunx` as above |
1018
+ | 0.7.0 through 0.12.0 (protocol 10), 0.12.1 and 0.12.2 (protocol 11), 0.12.3 (protocol 12), 0.12.4 through 0.12.15 (protocol 13), 0.12.16 (protocol 14), 0.12.17 through 0.12.19 (protocol 15), 0.12.20 (protocol 16) | the target's, through `bunx` as above |
804
1019
  | any supported source, with the installed CLI already at the target | `ahub upgrade` below, which is the same coordinator |
805
1020
  | 0.5.x or earlier (protocol 8 and older) | not supported: bootstrap by hand with the matching CLI |
806
1021
 
@@ -832,17 +1047,19 @@ projects first:
832
1047
 
833
1048
  ```bash
834
1049
  ahub restart --dry-run
835
- ahub upgrade --to 0.12.18 --dry-run
1050
+ ahub upgrade --to 0.12.20 --dry-run
836
1051
  ```
837
1052
 
838
1053
  Apply only after reviewing the plan:
839
1054
 
840
1055
  ```bash
841
1056
  ahub restart --yes
842
- ahub upgrade --to 0.12.18 --yes
1057
+ ahub upgrade --to 0.12.20 --yes
843
1058
  ahub recovery status <operation-id>
844
1059
  ahub recovery resume <operation-id>
845
1060
  ahub recovery abort <operation-id>
1061
+ ahub recovery dispose <operation-id> --fresh-session <peer> --reason <text>
1062
+ ahub recovery dispose <operation-id> --stop-and-archive --reason <text>
846
1063
  ```
847
1064
 
848
1065
  The coordinator commits only once the source is quiet: no turn running, no
@@ -898,6 +1115,20 @@ Kimi and local sessions may start fresh with preserved routing and task context.
898
1115
  A Claude session that never persisted a transcript (zero turns) also starts fresh:
899
1116
  there is nothing to resume, so the upgrade accepts the new session id; a session
900
1117
  with a transcript must come back with its original id.
1118
+ A Claude session that `ahub claude` did not launch in a recorded Orca terminal
1119
+ (a plain `claude`, or one whose launcher has ended) is planned as
1120
+ reconnect-only: the plan lists it under `reconnectOnly`, and the command prints
1121
+ `<project>: claude is reconnect-only` on stderr. Nothing in its terminal is
1122
+ closed or relaunched, a `claude-session.json` left by an earlier launch is never
1123
+ used as its target, and the plan does not show that record's id. Its plugin
1124
+ reconnects to the new hub by itself (the coordinator waits up to 90 seconds) and
1125
+ keeps the plugin version it started with until that Claude session restarts.
1126
+ The target release must read recovery waivers; staging refuses one that does
1127
+ not, before any runtime changes. Across a control-protocol change it cannot
1128
+ reconnect; the plan then names a
1129
+ blocker: end the session, or relaunch it with `ahub claude` in Orca, and make a
1130
+ new plan. A refused `--yes`, and every dry-run, prints each blocker on its own
1131
+ `ahub: blocker:` line before the final one.
901
1132
  Previously stopped projects stay stopped; only the reviewed running projects
902
1133
  are upgraded.
903
1134
 
@@ -905,6 +1136,140 @@ Do not run an upgrade with an incompatible active protocol, an unverified
905
1136
  terminal binding, or an unresolved operation lock. Dry-run performs no package,
906
1137
  plugin, daemon, or terminal mutation.
907
1138
 
1139
+ ### A partial operation
1140
+
1141
+ `ahub recovery status <operation-id>` reads the receipt without changing it:
1142
+ its phase, step and error; `runner`, which says whether a runner process holds
1143
+ the operation now (`running` with its pid, or `none`; `stale` marks a receipt
1144
+ that says running with no runner behind it); each project's phase and effect
1145
+ receipts (`closed:<peer>`, `restored:<peer>` as `done`, `pending` or `failed`);
1146
+ whether the shared plugin and the global CLI were installed; and `next`, the
1147
+ commands that apply now, read from every source the operation has not stopped
1148
+ yet (every error of `resume` ends with the same list). `resume` is left out
1149
+ when it can never get past: a source replaced by another instance, or a
1150
+ prepared source that stopped before any commit request. It shows no task or
1151
+ message text. `resume` does not
1152
+ start a second runner while one is alive. Every `next` entry and error names the
1153
+ operation's own coordinator, `bun <preserved source>/src/cli/main.js recovery
1154
+ ...`: in the middle of an upgrade the global `ahub` may still be the older
1155
+ release, whose `recovery` lacks these commands. Copy that command line. When
1156
+ an older coordinator started the operation, `dispose` is named from the running
1157
+ release, which has it. What
1158
+ each receipt allows, given what is live, is tabulated in the recovery spec
1159
+ (`docs/specs/2026-09-20-upgrade-recovery-design.md`, "Receipts, evidence and
1160
+ next actions").
1161
+
1162
+ - The source hold lapses after 10 minutes. A later `resume` prepares the same
1163
+ source again and checks its peers again, keeping the receipts, so no terminal
1164
+ is closed twice. It refuses, naming the next step, when the source daemon was
1165
+ replaced, when another operation holds it, or when a peer changed: a peer whose
1166
+ terminal the operation closed must stay closed, a session that joined since
1167
+ must end, every other one must keep its conversation. With nothing closed yet,
1168
+ `abort` cancels the operation (also once its hold has lapsed, or when another
1169
+ daemon or operation holds the source, which it leaves alone) so a new plan
1170
+ can be made, also after a source crashed before any commit was requested
1171
+ (`resume` then stops, as there is nothing to start from, and names abort, or
1172
+ stop-and-archive once anything was done); it
1173
+ refuses while a prepared source cannot be read (its hold may still stand),
1174
+ once a commit request may have been sent (or, for an operation from
1175
+ an older coordinator, which records no such thing, while a prepared source is
1176
+ not running), and once any project has effects the way out is
1177
+ stop-and-archive. `next` offers abort only where abort would succeed, and
1178
+ `resume` (and `--fresh-session`, which resume carries out) only where it can
1179
+ get past what is live: not past a replaced or missing source, a source that
1180
+ crashed before any commit request, a refusal at staging (the target release
1181
+ or the preserved source can never change back), a hub of another control
1182
+ protocol, a target that stopped after it started, or a stopped hub without
1183
+ the operation's restart snapshot.
1184
+ A second `resume` while its own hold still stands checks the peers again
1185
+ before it closes anything.
1186
+ - When a target hub dies after it started (a crash or a reboot), restarting it
1187
+ from its snapshot is not supported in this release (its peers would have to
1188
+ be relaunched against the new hub): `resume` is not offered and the way out
1189
+ is stop-and-archive.
1190
+ - A Codex conversation comes back only when a rollout file naming its thread
1191
+ exists under `sessions/` of the store the restored terminal uses (the
1192
+ recorded `CODEX_HOME`, else `~/.codex`). Codex writes that file with the
1193
+ first message, so a thread the hub saw Codex start, with no turn on it since,
1194
+ is listed under `freshStart` and restarts as a new session, with nothing to
1195
+ lose; the command says so. A thread with turns, or one whose start the hub's
1196
+ log does not show (it was resumed, or the hub is older), blocks the plan when
1197
+ it has no rollout, and so does a store that cannot be read (unknown, not
1198
+ missing). The coordinator checks again before closing the terminal and before
1199
+ creating the new one. To continue without such a conversation, end that Codex
1200
+ session and close its Orca terminal, then `resume`; it stops at restoring
1201
+ Codex, where `--fresh-session codex` below is the way on.
1202
+ - A restored terminal runs its launch in a login shell, so the coordinator
1203
+ watches the launcher, not the terminal: once `ahub codex`, `ahub claude` or
1204
+ `ahub pi` has recorded itself there and its process is gone, or Orca no longer
1205
+ lists that terminal at all, the restoration failed (found within one 5-second
1206
+ wait). A launcher whose process cannot be read counts as running, never as
1207
+ gone. One that dies before recording itself leaves no record to read, and
1208
+ what happens next depends on whether Orca reports the bare login shell it
1209
+ returned to as TUI-idle, which is not verified. If it does, the coordinator
1210
+ stops at once with "original session restoration needs manual verification"
1211
+ (the terminal cannot be mapped to the session). If it does not, it stops with
1212
+ the same error after the 10-minute wait. Either way the receipt stays
1213
+ `pending`; close that terminal in Orca (or wait, if the session may still
1214
+ attach), then `resume`: with no live launcher and no attached session it
1215
+ records `restored:<peer>` `failed` and lists the failed-restoration choices.
1216
+ A `failed` receipt never counts as restored.
1217
+ `resume` settles a `pending` or `failed` receipt by what is live: the planned
1218
+ session attached is the restoration, a launcher still running is waited for,
1219
+ and no second terminal is opened while one may run. A target hub that cannot
1220
+ be read counts as unknown, never as "nothing attached", and so does a launcher
1221
+ record file that cannot be read: `resume` then stops without changing any
1222
+ receipt; when the record file is what cannot be read, inspect it and move it
1223
+ aside, then `resume`. `resume` is refused while a stop-and-archive is partway. A lifecycle command refused by the lock names
1224
+ the operation's own `status` command, which lists what to do next.
1225
+ - `ahub recovery dispose <operation-id> --fresh-session <peer> --reason <text>`
1226
+ applies to a Codex or Claude peer whose restoration failed (Pi is refused: a
1227
+ restored hub resumes Pi's recorded session). It records the lost session or
1228
+ thread id (status shows it under `lostContinuity`) and resumes: the peer
1229
+ starts without its old conversation, and the rest is verified as usual.
1230
+ - `ahub recovery dispose <operation-id> --stop-and-archive --reason <text>`
1231
+ abandons the operation. It checks every project first and refuses while a
1232
+ hub is unreachable, starting, stopping or of another control protocol; `next`
1233
+ then says `wait until <project>'s hub settles, then ... recovery status
1234
+ <id>` instead of offering it. A hub that never answers (its pid is alive,
1235
+ nothing replies): take the `pid` from that project's
1236
+ `.agenthub/state/status.json`, check with `ps -p <pid> -o command=` that it
1237
+ is that project's hub daemon, end it by hand, then run `status` again. If
1238
+ that pid now belongs to another process (reused after a reboot), move the
1239
+ stale `status.json` aside instead, then run `status` again; likewise when
1240
+ `next` says `wait: runner <pid>` and that pid is not an `ahub recovery-run`
1241
+ process, move the operation's `~/.agenthub/recovery/<id>.json.runner.db`
1242
+ aside, then run `status` again. A
1243
+ project whose directory is
1244
+ gone is only recorded (`ahub doctor --orphans` lists a hub left running
1245
+ there). It records its decision before it acts. It releases a source hold of
1246
+ its own (that hub keeps running), stops a target hub it started and has not
1247
+ released (one an older coordinator started is stopped at that release's
1248
+ control protocol), moves that operation's
1249
+ `restart.json` aside as `restart.abandoned.<hash>.json` and leaves any other
1250
+ hub running; only then is the operation recorded `cancelled` and the lock
1251
+ released. If it fails partway it keeps the lock, records what it did, and
1252
+ `resume` refuses until you run it again. Queued messages are not delivered
1253
+ from the archived file; a 0.7.0 or later hub reloads the queues it kept in
1254
+ `hub.db` when it starts again. Terminals it closed stay closed: start those
1255
+ sessions again by hand. Replacement terminals it opened stay open but have
1256
+ no hub: Claude's plugin reconnects once a hub runs, Codex needs `ahub codex`
1257
+ again.
1258
+ Start a project whose target ran with that release's CLI (`bunx --package
1259
+ @staix/agent-hub@<version> ahub up`), since it may have changed the task
1260
+ database. The global CLI was not promoted.
1261
+ - Both choices are for a person in a terminal; an agent shell is refused, and
1262
+ the reason is kept in the operation's audit. Run each with the command
1263
+ `next` or the error prints: the operation's own coordinator, or the release
1264
+ you are running when that coordinator predates these commands (its `resume`
1265
+ still runs the old runner, and `next` says what that runner cannot do).
1266
+ `--fresh-session` never applies to such an operation: its runner never
1267
+ records a restoration as `failed`.
1268
+ `--fresh-session` is not offered when the target release cannot read recovery
1269
+ waivers, since it could never release a new session; `status` says so in
1270
+ `freshSession`. A runner record that cannot be read shows as `unknown`, and
1271
+ `resume`, `abort` and `dispose` are refused until it can be read.
1272
+
908
1273
  ### After an unplanned stop
909
1274
 
910
1275
  While it runs, the hub keeps each attached peer's session identity in
@@ -1031,9 +1396,46 @@ PII calls require a positively confirmed campus gateway immediately before trans
1031
1396
  PII turns never enter shared history, memory capture, or progress observation.
1032
1397
  The `route`, `advisor`, `progress` and `stuck` events contain identifiers and aggregates only.
1033
1398
 
1399
+ Session-aware stay/switch (#197) covers `hub/auto` and `stage` routes. The top-level
1400
+ `routing.toml` key `stay_switch` is `off`, `shadow` (default: route events record what the
1401
+ planner would do, routing unchanged) or `enforce`. Enforced, a hard override (compaction,
1402
+ critical failure, repeated failure) escalates at once; a tool-result turn keeps the
1403
+ session's tier; other changes wait for the next user turn, and a de-escalation whose
1404
+ conversation is larger than `max_switch_prefill_tokens` (default 32000) stays; a PII route
1405
+ has no such bound, so its size never shows in a decision. A capable
1406
+ hold then lasts at least until the next user turn; `hold_turns` can extend it past that
1407
+ turn. A tier whose backend
1408
+ cannot hold the conversation is never chosen; no route summarizes or trims the history.
1409
+
1034
1410
  Pi exposes `hub/auto` for stage routing when available. Fixed `dgx/coding`, `dgx/fast`
1035
1411
  and `mlx/fast` aliases still pin the backend. Automatic MLX selection admits the complete
1036
1412
  input, tool schemas and requested output within the configured context window.
1413
+ A `hub/auto` request bound for MLX has two candidates, MLX and its `dgx/fast` fallback; load
1414
+ and cooldowns only change their order, never remove one (#199). A fixed alias (`mlx/fast`,
1415
+ `pi_backend = "mlx"`) keeps its backend first as before. `dgx/fast` goes first when MLX is
1416
+ cooling down (route event `source: "cooldown"`). Load moves are opt-in until they are
1417
+ measured: with `[pi] efficient_wait_ms` set (0 to 119999; 0 tries once; read when the hub
1418
+ first starts Pi, like `dgx_max_context_tokens`, so a change takes a hub restart), a request
1419
+ whose MLX slot is still busy after that wait goes to `dgx/fast` first (`source: "load"`), and
1420
+ so does an enforced tool loop pinned to it (`source: "pin"`); without it nothing moves for load. `dgx/fast`
1421
+ never goes first while it is cooling down, or while its own last dispatch failed in any way,
1422
+ an error status or a failed stream included, until it succeeds or 30 s pass (`ahub status`:
1423
+ `last dispatch failed, no load moves until ...`). No load move or pin happens while the request
1424
+ runs under an execution budget, and enforced, a load move happens only at a user turn. A moved
1425
+ attempt gets 15 s to its response headers after the gateway lookup (OmniRoute's own probe,
1426
+ up to two 4 s rounds when no gateway is cached) and is then abandoned for MLX, which serves
1427
+ with the usual slot wait. The bound ends with the headers: a stream that stalls or fails
1428
+ after them is not retried on MLX, and a failed one marks `dgx/fast` failing (a client that
1429
+ disconnects does not). Each backend is tried once per request, so with moves on, a moved attempt
1430
+ cut at 15 s followed by an MLX slot that stays busy past 120 s fails the request (502), where
1431
+ without moves the request would have waited for MLX and then had `dgx/fast` with its full
1432
+ deadline; this is part of what the opt-in accepts until #199 AC5 measures it. After three
1433
+ consecutive transport or startup failures outside a cooldown (a failure more than 10 min after
1434
+ the last counted one starts the count over), a relay alias cools down for 30 s, doubling up to
1435
+ 5 min. A busy MLX slot and timeouts cut short by an execution budget never count; any HTTP
1436
+ answer, a success or an error status, proves the transport works and ends the streak and the
1437
+ cooldown. `ahub status` shows `cooling down until ...` on the backend line and `events.jsonl`
1438
+ records `cooldown` events.
1037
1439
  Progress judgements suggest reassignment; they never change task ownership.
1038
1440
 
1039
1441
  ## Disable local MLX while keeping remote auto routing
@@ -17,7 +17,7 @@ ahub setup # installs the Claude Code channel plu
17
17
  Or use the matching GitHub release:
18
18
 
19
19
  ```bash
20
- bun add -g github:STAIxBWLB/agent-hub#v0.12.18
20
+ bun add -g github:STAIxBWLB/agent-hub#v0.12.20
21
21
  ahub setup
22
22
  ```
23
23