@staix/agent-hub 0.12.3 → 0.12.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/docs/cooperbench.md +35 -2
- package/docs/events.md +9 -0
- package/docs/operations.md +158 -5
- package/docs/specs/2026-09-19-agent-hub-design.md +30 -1
- package/docs/verification/2026-10-02-0.12.4.md +82 -0
- package/package.json +1 -1
- package/plugins/agent-hub/.claude-plugin/plugin.json +1 -1
- package/plugins/agent-hub/server.js +5 -4
- package/src/adapters/claude-channel.ts +2 -1
- package/src/adapters/codex-appserver.ts +38 -0
- package/src/cli/facts-hook.ts +40 -0
- package/src/cli/launch.ts +27 -3
- package/src/cli/main.ts +31 -2
- package/src/cli/upgrade-runtime.ts +1 -1
- package/src/hub/bus.ts +75 -3
- package/src/hub/cohorts.ts +308 -0
- package/src/hub/control-client.ts +3 -3
- package/src/hub/daemon.ts +348 -14
- package/src/hub/events.ts +9 -0
- package/src/hub/facts.ts +754 -0
- package/src/hub/report.ts +3 -0
- package/src/hub/routing.ts +85 -0
- package/src/hub/tasks.ts +380 -19
- package/templates/AGENTS.block.md +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
Issue and pull request numbers in the entries for 0.7.7 and earlier refer to the previous repository, archived on 2026-09-30 when this repository's history was rewritten; the one exception is the open smoke-check issue, formerly #12, which moved here as #1. Numbers in newer entries refer to this repository.
|
|
4
4
|
|
|
5
|
+
## 0.12.4
|
|
6
|
+
|
|
7
|
+
- A completed-change or edit-conflict notice is checked again for each recipient right before it is handed over, after condensation: a copy whose task has closed, changed owner or is gone is dropped instead of starting a turn, recorded as discarded and as a `stale` event. Other recipients and unrecorded envelopes are delivered as before (#106).
|
|
8
|
+
- Opt-in `coordination: "turn-free"` (advisory stays the default): owners of overlapping tasks form a cohort, silent only when every owner's context path is verified and no PII task is open. Messages between members are held back for those members only, with an explicit sender result and no charge against the sender's limits when nobody got them, until the sender has stopped after its task closed; that settlement is never undone, so a member's later turns are new work. A PII task opening lifts silent cohorts for good and says so. The member that finishes last (a console done counts as an intent) is asked to integrate, and its next done counts only for the same owner, cohort revision and files once the others have stopped; after three requests the outcome is recorded as unresolved. After a restart each peer's open overlapping tasks are told that overlaps are settled by message again, with the completed-change notices the silence held (#107).
|
|
9
|
+
- Turn-free facts at tool boundaries: what changed in an owner's files since it acknowledged them, with attribution only on effect evidence, offered through Claude's hooks (`ahub claude` adds them before and after every tool call and at Stop) or by steer into a running Codex turn, and acknowledged by a readback in the native session (facts sent with an integration request: by the next done). A Claude Read counts as seen only when it returned the whole file; a diff that matches a PII pattern is not shown. Contained paths, small regular files, tracking reset after PII. Control protocol 13; 0.12.3 (protocol 12) remains an upgrade source (#108).
|
|
10
|
+
- The split rule is a shadow prediction only: routing never changes; an overlap that forms or changes a cohort records a `split` event for the task, routed or named, and `ahub route explain` shows the trace, unknown unless its assumptions hold (an overlapping task its owner has started is other work) (#109).
|
|
11
|
+
- CooperBench manifest v2 adds a turn-free arm with hooks and status lines equal across arms and a planned-attempt check; a separate manifest is the #106 ablation (`experiments.stale_notices: "deliver"`); the grader grades completed and timed-out attempts and refuses a turn-free attempt whose tasks did not stay in one silent cohort, or any attempt whose records show a hook that is not the hub's; repeats rotate the arm order; run records carry their hook and MCP conditions; `scripts/benchmarks/ledger.py` reports every measure with its unit and coverage over one or several run directories, including in-task and whole-attempt usage per agent with tokens, completion and native settlement times, facts, hook timings and held-back messages, the grader's validity gates, and identifier and fragment contributions (#110).
|
|
12
|
+
|
|
5
13
|
## 0.12.3
|
|
6
14
|
|
|
7
15
|
- Fix live Claude delivery settlement holds, add generation-bound explicit completion and pending-settlement status (#100).
|
package/docs/cooperbench.md
CHANGED
|
@@ -28,7 +28,7 @@ The manifest's archive and prompt hashes are checked before a fixture is accepte
|
|
|
28
28
|
|
|
29
29
|
Stage `case-00.json` through `case-09.json` outside the repository. Each JSON file supplies the two exact feature prompts in feature order and the official evaluator's private case data. Pass each prior native Claude/Codex transcript as a separate `--protect <file>` argument. Do not expose those files, upstream evaluator/data tree, or prior run artifacts to the agents.
|
|
30
30
|
|
|
31
|
-
Prepare a new private run directory, then execute a fixed cohort. `--cases 0` runs
|
|
31
|
+
Prepare a new private run directory, then execute a fixed cohort. `--cases 0` runs every predeclared arm of the manifest for case zero (three in v1, four in v2). It never selects individual arms or retries only a scored subset.
|
|
32
32
|
|
|
33
33
|
```sh
|
|
34
34
|
python3 scripts/benchmarks/runner.py prepare \
|
|
@@ -51,7 +51,40 @@ bun scripts/benchmarks/native.ts \
|
|
|
51
51
|
|
|
52
52
|
The runner registers each exact fixture root with Orca, uses the project CLI to start and stop each daemon, and requires exact worktree/cwd readback for native sessions. Claude starts through the canonical `ahub claude` guard and loads this checkout's candidate bundle through an exact session-only `--mcp-config` server (`server:agent-hub`); it does not promote or mutate the globally installed plugin. Codex uses the native app-server adapter and `workspace-write` sandbox. Each agent must execute a setup-only `head -c 1` probe against the exact protected file and produce only the denied marker before scored tasks begin. There is one native sandbox layer per agent.
|
|
53
53
|
|
|
54
|
-
Every run record, diff, PTY/provider trace and evaluation artifact stays in the mode-0700 private run directory. Setup errors, structured provider quota errors, hub budget pauses, unsettled deliveries and interruptions stay separate and unscored. A quota snapshot or numeric `429` alone is not a provider error. The runner restores exact input/read-lock modes and the scoped Claude trust flag (removing its own fresh project entry) and sibling artifact modes on exit. It stops only its recorded Claude terminal, hub project and hub terminal handles. Orca currently has no repo removal command, so inactive exact-path fixture repos remain registered after a run; one case
|
|
54
|
+
Every run record, diff, PTY/provider trace and evaluation artifact stays in the mode-0700 private run directory. Setup errors, structured provider quota errors, hub budget pauses, unsettled deliveries and interruptions stay separate and unscored. A quota snapshot or numeric `429` alone is not a provider error. The runner restores exact input/read-lock modes and the scoped Claude trust flag (removing its own fresh project entry) and sibling artifact modes on exit. It stops only its recorded Claude terminal, hub project and hub terminal handles. Orca currently has no repo removal command, so inactive exact-path fixture repos remain registered after a run; one case leaves one record per arm.
|
|
55
|
+
|
|
56
|
+
## Manifest v2: the turn-free arm
|
|
57
|
+
|
|
58
|
+
`scripts/benchmarks/manifest-v2.json` keeps v1's cases, models and limits and adds a fourth arm, `hub-turnfree-codex-claude` (issue #110): the same two agents and assignment rotation as `hub-codex-claude`, with `coordination: "turn-free"` in the fixture's hub config and fixture instructions that tell the owners not to message each other. A v1 manifest still validates and prepares. Its cohorts are run, graded and reported with the release and runner sources they were prepared with: `native.ts` refuses a manifest whose hub version is not the checkout's (v1 pins 0.12.3), and `runner.py` refuses to grade a cohort whose runner sources differ from its own.
|
|
59
|
+
|
|
60
|
+
Hooks are equal across arms: no arm runs the user's or a plugin's hooks, and no arm runs a status line (`disableAllHooks` turns it off in the other arms, so the turn-free arm leaves it out; Claude's quota reaches the hub in no arm). Claude starts with `--setting-sources project` and `--strict-mcp-config`; the solo and advisory arms set `disableAllHooks`, and the turn-free arm's session settings carry the hub's own hooks (before and after every tool call, and at Stop) and nothing else, because they are the treatment. Every Codex thread starts with `features.hooks` off: the turn-free arm's Codex boundary is the adapter's steer into the running turn, whose readback is the steered input coming back as a user message item. MCP servers are isolated too: Claude has only the hub's (`--strict-mcp-config`), and every Codex runs through a wrapper written into each arm's fixture that turns off the user's plugins, apps, sub-agents and turn-end notifier and disables each MCP server the user's config defines, by name, so it starts only the hub's; the user's config is not changed. Instructions: Claude reads the fixture's `AGENTS.md` through `--append-system-prompt-file` (Claude Code reads `CLAUDE.md`, not `AGENTS.md`), Codex as the project's `AGENTS.md`; Codex also reads the user's global `AGENTS.md`, which only a separate Codex home with its own login would leave out (the run records say so), and the user's Codex skills stay available to it (the wrapper leaves them on); both are the same in every arm. Each run record carries these `conditions`; its `events` carry the hub's `capability` readbacks.
|
|
61
|
+
|
|
62
|
+
Validity, decided by the grader and applied by the ledger alike: an attempt whose records show a hook or an MCP server that is not the hub's, or whose Claude transcript cannot be read, is unavailable. A turn-free attempt is valid only with its context paths working: both verified before its tasks, and none lost, no cohort lifted and none formed open while the agents worked; teardown comes after that and does not count. Whether the agents' plans overlapped, so that a cohort formed at all, is their doing after assignment and is not a condition: every turn-free attempt without a capability failure counts for the arm, and the ledger reports the treatment received and a median over treated attempts beside it. A capability failure while the agents work is the one exclusion after assignment, because #110 forbids reporting it as a turn-free run (AC3); such attempts are listed with their reasons, never dropped silently. Each attempt records its transcript's length and hash when it ends; validity and every Claude measure are read from that prefix (Claude Code may append rows after it exits: a response still being written at teardown is not counted, and that agent's settlement is then unknown), and a prefix that changed counts as unreadable.
|
|
63
|
+
|
|
64
|
+
`scripts/benchmarks/manifest-v2-ablation-106.json` is the #106 ablation: the advisory arm against the same arm with `experiments.stale_notices: "deliver"`, which turns stale-notice dropping off and changes nothing else, on the same release and conditions. Its attempts are counted apart from the four-arm plan.
|
|
65
|
+
|
|
66
|
+
Pass `--repeat <n>` for the n-th repeat of a case (0 for the first): the arm order is row (case index + repeat) of a Williams design (0, 1, n-1, 2, n-2, ... shifted by the row), so over n consecutive rows every arm runs right before every other one once and repeats of one pair change which arm runs last. The manifest's `plan` names the release pilot (case 0, three repeats, 12 attempts, an active-time ceiling of one hour) and the study (ten cases, two repeats, 80 attempts, 6 hours 40 minutes at 300 s each, setup, grading and teardown excluded); `runner.py` refuses a plan whose attempts or ceiling do not follow from its arms, cases and repeats.
|
|
67
|
+
|
|
68
|
+
## Coordination ledger
|
|
69
|
+
|
|
70
|
+
```sh
|
|
71
|
+
python3 scripts/benchmarks/ledger.py --run /private/tmp/ahub-0124-r1 [--run /private/tmp/ahub-0124-r2 ...]
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
It reads each run record, the Claude transcript it names and the fixture's git history, and writes `ledger.json` into the first run directory with a `units` table that names the unit and coverage of every measure. Give `--run` once per directory to pool the repeats of one plan; an attempt a directory's `cohort.json` planned that wrote no record is listed as missing. Per attempt: completion (the runner's end reason and its detail, such as `wall-timeout`, and whether every task has a done); setup and active time; first candidate, completion intents, integration and check times, and each agent's settlement read from its own record (the end of its last native turn after the last done, turns started by late messages included; the last done itself when it did not work after it), and how long the agents could still write after the active time ended (`stopped_s`); usage per agent in task (to its last done on the board) and over the whole attempt: Codex turns, assistant messages, token-usage updates whose running total grew past the total before the window, and the token growth; Claude assistant messages (unique message ids), turns and the tokens their usage records; Claude's main-loop requests by request id (side requests and retries are not in its transcript), and Codex's provider requests unknown (app-server 0.159 does not send its response ids); Codex turns after its done and what started them; late replies, steered or at the next turn; the hub's fact offers, acknowledgements, bytes offered and acknowledged, build times, hook start-up times and steer round trips by path, in the task window; capability readbacks; validity by the grader's gates, and for turn-free the treatment received; held-back messages (`quiet` events) apart from [FYI] messages (which include the final [FYI] the instructions ask for); stale notices; shadow split predictions with their traces; the hooks each agent's records show, by a label that keeps paths and arguments out, with Claude's hook durations and the hub's own timing of every facts hook call; and contributions. Summaries give medians over valid completed attempts and over the (case, repeat) pairs every arm completed validly, totals over the attempts whose tasks were handed out and to which the measure applies (a Codex measure in a Claude-only arm is not counted as unknown), with the number of attempts each was unknown for, and the reasons for the rest. A repeat given twice is refused, and `--plan pilot` (or `study`) lists every planned attempt that wrote no record, whole repeats included. `ledger.json` holds code fragments from the agents' writes and local paths: keep it with the private run data and never commit it; the summary is what a verification record quotes. In-task windows end at different points by arm (a turn-free integration step comes before the done, an advisory completed-change notice turn after it), so arms are compared on whole-attempt usage.
|
|
75
|
+
|
|
76
|
+
Contributions are a heuristic for possible loss, never a certificate: per agent and file, the identifiers and changed fragments its applied writes introduced (only a Claude tool call with a successful result, or a completed Codex patch, counts; a Write replaces the agent's earlier contribution and is not credited with what other agents wrote; a delete removes it; when a file is moved, every agent's contributions to it are checked at its new path) that the final tree lacks, a fragment counting as present anywhere in the file. A same-name overwrite shows as a lost fragment. Shell commands run during the task are not attributed and are counted under `coverage`, with a missing transcript or an unreadable file. Correctness comes from the official grader, for every arm.
|
|
77
|
+
|
|
78
|
+
## Preregistration (v2)
|
|
79
|
+
|
|
80
|
+
Fixed before outcomes are collected (issue #110):
|
|
81
|
+
|
|
82
|
+
- Order of ablations: the stale-notice change first (#106: `manifest-v2-ablation-106.json`, the advisory arm with and without stale-notice dropping on the same release; comparing with 0.12.3's runs would confound it with that release's Codex hooks), then cohort silence with acknowledged facts and the revision-fenced integration (#107 and #108, the turn-free arm). The split rule (#109) is recorded as shadow predictions only; task allocation stays fixed by case index in both joint arms.
|
|
83
|
+
- Primary outcome: the final submitted artifact of each attempt, graded by the official tests with the same controls for every arm. Completed and timed-out attempts are graded (a timeout's final tree is its submission, collected once the agents were stopped, `stopped_s` after the limit); interrupted, infrastructure-failed and invalid attempts are reported unavailable with their reasons. Time to both done is reported only with completion, failures, timeouts and unavailable attempts beside it, and timing comparisons use attempts that completed in every compared arm.
|
|
84
|
+
- Resource outcomes per provider over the whole attempt, in the units the ledger names: Codex turns, usage growth and tokens; Claude assistant messages, main-loop requests and tokens (amended on 2026-10-02, before the study: side requests are not in Claude's transcript); hook and fact bytes and latencies.
|
|
85
|
+
- The turn-free arm is analysed over every attempt without a capability failure (the only exclusion after assignment, required by AC3, and listed), with the treated attempts' median as a secondary result. A 35% time and 30% usage reduction are hypotheses, not targets met; counters of different providers are not added together.
|
|
86
|
+
- Every preregistered attempt is retained, including failed and infrastructure-unavailable ones; no repeat is selected after inspecting results.
|
|
87
|
+
- The release pilot (case 0, three repeats of four arms) is feasibility evidence only. Changing the default from advisory needs paired quality and resource criteria fixed in advance, held-out repository and task instances, and comparison with both solo controls and the current advisory arm; any fact loss, missed integration, stale-generation injection or held-back workflow event is a safety result that blocks it.
|
|
55
88
|
|
|
56
89
|
## Grade and report
|
|
57
90
|
|
package/docs/events.md
CHANGED
|
@@ -14,6 +14,15 @@ marked `private: true`, and PII tasks `pii: true`.
|
|
|
14
14
|
|---|---|
|
|
15
15
|
| `envelope` | `id`, `from`, `to` (absent for broadcast), `priority`, `hop`, `kind`, `task` (task id from refs), `bytes` (UTF-8 size of the body; absent on a private envelope, whose size would say something about the PII text), `private`, `dropped` (`hop` or `fyi` when not delivered) |
|
|
16
16
|
| `overflow`, `undeliverable` | `id`, `from`, `peer` |
|
|
17
|
+
| `stale` | `id`, `from`, `peer`, `task`: a notice about the recipient's open task, dropped unsent because, right before it would have been handed over, that task was closed, had another owner or was gone (issue #106) |
|
|
18
|
+
| `quiet` | `id`, `from`, `peers`: an agent message held back from these members of a silent turn-free cohort; its other recipients got it (issue #107) |
|
|
19
|
+
| `fact` | `peer`, `id` (the offer), `files` (files whose diff it carried), `plans`, `unknown` (files whose change it showed with attribution unknown), `bytes` (the injected text), `via` (`hook` for Claude, `steer` for Codex, `done` with an integration request), `ms` (the hub's time to build it), `hookMs` (the hook process's own start-up and connect time), `accepted` (whether app-server took the steer), `unanswered` (app-server did not answer it within 10 s: it may have gone in), `rttMs` (an accepted steer: from sending it to app-server's answer), `probe` (a context check), `coverage` (it named files earlier changes are not covered for): one fact offer (issue #108) |
|
|
20
|
+
| `fact_ack` | `peer`, `id`, `via` (`hook` and `steer`: a readback found the offer in the native session; `done`: the next `hub_task_done`), `ms` (from the offer): an acknowledged offer, the only thing that moves a peer's view |
|
|
21
|
+
| `capability` | `peer`, `state` (`verified` or `lost`), `via`: a peer's context path for facts |
|
|
22
|
+
| `native_turn_end` | `peer`: Claude's Stop hook, the end of its turn (Codex's is its `turn_end`); quiescence evidence for an integration |
|
|
23
|
+
| `hook_stats` | `peer`, `n` (facts hook calls in the turn, its Stop included), `startupMs` and `maxStartupMs` (the hook processes' start-up and connect time, summed and the largest), `hubMs` (the hub's own time for them): at Claude's Stop (issue #108) |
|
|
24
|
+
| `cohort` | `id`, `event` (`formed`, `joined`, `lifted`), `silent`, `tasks`, `owners`: owners of overlapping tasks formed a cohort, it changed membership, or it stopped being silent (issue #107). Recorded in every regime; only a turn-free project's cohorts can be silent |
|
|
25
|
+
| `split` | `task`, `verdict` (`split`, `single`, `unknown`), `single` (the peer that would finish both units alone soonest), `splitS`, `singleS`, `reason` (for `unknown`), `trace` (the inputs and steps: peer names and numbers only): a shadow split prediction where an overlap forms or changes a cohort, routed or named; it never changes the assignment (issue #109) |
|
|
17
26
|
| `state` | `peer`, `state` |
|
|
18
27
|
| `turn_start` | `peer`, `turn` (`<peer>#<hub run>.<n>`, unique across restarts). A turn follows the adapter: pausing a busy peer does not end it |
|
|
19
28
|
| `turn_end` | `peer`, `turn`, `ms`, `tokens` (when the adapter reported any during the turn), `files` and `snapshotMs` (when snapshots are on: how many files the turn changed, and the time both snapshots took) |
|
package/docs/operations.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Operations guide
|
|
2
2
|
|
|
3
|
-
This guide describes ahub 0.12.
|
|
3
|
+
This guide describes ahub 0.12.4 and control protocol 13. Live verification
|
|
4
4
|
results and remaining prerequisites are recorded separately in [the smoke ledger](smoke.md).
|
|
5
5
|
|
|
6
6
|
## Install and start
|
|
@@ -344,6 +344,159 @@ are quoted and marked as other agents' text. The hook runs `ahub`, so it has to
|
|
|
344
344
|
be on the PATH Claude Code's hooks see; otherwise every edit shows a hook error
|
|
345
345
|
(it never blocks). `ahub check-path <file>` prints the same list in a terminal.
|
|
346
346
|
|
|
347
|
+
## Turn-free coordination
|
|
348
|
+
|
|
349
|
+
A notice about an open task of its recipient (the completed-change notice and the
|
|
350
|
+
edit-conflict messages above) is checked again for each recipient right before it
|
|
351
|
+
is handed over, after any digest condensation: if that recipient's task has
|
|
352
|
+
closed, changed owner or is gone, its copy is dropped instead of starting a turn,
|
|
353
|
+
the journal records it as `discarded`, `hub.log` has a `STALE` line and
|
|
354
|
+
`events.jsonl` a `stale` event. Other recipients keep their copies. Approvals,
|
|
355
|
+
check results, review requests, assignments and budget, permission and recovery
|
|
356
|
+
messages are never dropped this way, and a dropped notice never resolves another
|
|
357
|
+
delivery. The condition lives in memory: after a restart, or for the oldest of
|
|
358
|
+
more than 1024 notices, the notice is delivered as before.
|
|
359
|
+
|
|
360
|
+
Set `"coordination": "turn-free"` in `.agenthub/config.json` to let owners of
|
|
361
|
+
overlapping open tasks (the overlap rules above) work without messaging each
|
|
362
|
+
other. The default, `"advisory"`, keeps the behaviour described so far, and it
|
|
363
|
+
stays the default until an evaluation says otherwise (`docs/cooperbench.md`).
|
|
364
|
+
|
|
365
|
+
- Cohorts. Owners of overlapping tasks form a cohort when the overlap is found.
|
|
366
|
+
It is silent only if, at that moment, turn-free is on, no PII task is open and
|
|
367
|
+
every owner's context path is verified (below). It never turns silent later;
|
|
368
|
+
it stops being silent, for good, when an owner without a verified path joins,
|
|
369
|
+
a path is lost or a PII task opens, and every member still at work is told at
|
|
370
|
+
once that it may message again, with the completed-change notices the silence
|
|
371
|
+
held.
|
|
372
|
+
- Verified context path. A hub-launched Claude session (`ahub claude`) gets the
|
|
373
|
+
hub's hook before and after every tool call and at the end of each turn, in
|
|
374
|
+
its `--settings` next to the status line tee (a `--settings` of your own turns
|
|
375
|
+
them off). Codex gets context by steer into its running turn. Until a fact or a
|
|
376
|
+
one-line probe has been read back, the peer counts as unverified: for Claude
|
|
377
|
+
the row Claude Code writes in its transcript for the hook's additional
|
|
378
|
+
context, matched by tool use id and the offer's id; for Codex the steered
|
|
379
|
+
input coming back as a user message item of the turn. A new native session,
|
|
380
|
+
the peer going offline, or three offers past a minute without a readback
|
|
381
|
+
(checked at its boundaries and every 30 seconds; facts sent with an
|
|
382
|
+
integration request wait for the next done and do not count, a refused steer
|
|
383
|
+
is no offer, and an unanswered one may still be read back), makes it
|
|
384
|
+
unverified again, and a Claude session whose
|
|
385
|
+
transcript the hub cannot find gets no facts at all and loses a verified
|
|
386
|
+
path. A session whose hooks stop altogether leaves no offer unread, so it
|
|
387
|
+
stays verified; a peer without a verified path that left three
|
|
388
|
+
offers unread gets none until a readback arrives. Kimi, Pi and the local
|
|
389
|
+
worker have no path yet, so a cohort with one of them is never silent.
|
|
390
|
+
- Silence. While a cohort is silent, an agent message from a member to another
|
|
391
|
+
member is held back for that recipient only: other recipients and the console
|
|
392
|
+
get it unchanged, and `events.jsonl` records a `quiet` event. `hub_send`
|
|
393
|
+
answers `not delivered to <peer>: ...` (or `sent to: ...; not delivered to
|
|
394
|
+
...` when some recipients got it); a native turn answer's sender hears it on
|
|
395
|
+
its next delivery. Workflow messages from the hub are never held back. A
|
|
396
|
+
member's messages stay held until its native turn has ended after its task
|
|
397
|
+
closed (Codex's turn completes, or Claude's Stop hook runs; a member already
|
|
398
|
+
between turns when its task closes has stopped; a paused peer may still be in
|
|
399
|
+
its turn, and Claude's channel going offline says nothing about its session),
|
|
400
|
+
so a late answer is still the cohort's. That settlement is recorded when it happens and
|
|
401
|
+
never undone: the member's next turn is new work, and once every member has
|
|
402
|
+
settled the cohort is over (a task reopened after that is outside it). Only a tool call starting counts as activity after
|
|
403
|
+
a turn end. A message held back from all its recipients does not count
|
|
404
|
+
against the sender's limits.
|
|
405
|
+
- Facts. At each tool call (Claude) or completed tool item (Codex) of a cohort
|
|
406
|
+
member with an open task, the hub offers what changed since it last
|
|
407
|
+
acknowledged them in the files every member's task names and in the files it
|
|
408
|
+
touched, with the other members' new plans; the last member still at work
|
|
409
|
+
keeps the others' files after they finish. A directory a task names stands for
|
|
410
|
+
git's changed (staged or not) files against HEAD, new and deleted files under
|
|
411
|
+
it (200 at most; the fact says when more were cut, until it is read back and
|
|
412
|
+
again in a new session). A file the peer has seen there stays covered after git
|
|
413
|
+
stops listing it (put back to HEAD's bytes, or the directory moved away), so
|
|
414
|
+
the way back is shown (200 at most, the newest versions first; the rest are
|
|
415
|
+
named once as no longer tracked). A file there that the peer has neither seen
|
|
416
|
+
nor touched appears once someone changed or created it: it is named without a
|
|
417
|
+
diff (what happened before is never shown) until the peer reads the fact back,
|
|
418
|
+
and until then it counts as a change the peer has not been shown. `.git` directories
|
|
419
|
+
at any depth and what the denylist keeps from every agent
|
|
420
|
+
(`src/local/deny.ts` and `local.deny`) are never read or shown. A file a peer touched before it had a view of it (a
|
|
421
|
+
partial read, say) is compared with what it was then, so a change landing in
|
|
422
|
+
between is shown. A change is credited to an agent
|
|
423
|
+
only with effect evidence: a Claude Edit, MultiEdit or Write whose result is
|
|
424
|
+
exactly its input applied to the file as observed before it, or a Codex patch
|
|
425
|
+
whose diff is exactly what changed. Shell commands, concurrent writers and
|
|
426
|
+
unreported changes are shown with their attribution unknown, never credited by
|
|
427
|
+
elimination; an agent's own verified writes are not shown back to it. A Codex
|
|
428
|
+
read action never counts as having seen a file (it may be partial); a Claude
|
|
429
|
+
Read does when it returns the whole file: no offset or limit, at most 2000
|
|
430
|
+
lines and no line over 2000 characters. A diff that matches a PII pattern is
|
|
431
|
+
not shown (the file is named, to be read), and a changed file whose name
|
|
432
|
+
matches one is counted, not named. Only an acknowledgement (a readback, or the next
|
|
433
|
+
`hub_task_done` for facts sent with an integration request) moves the peer's
|
|
434
|
+
view, so a fact that does not arrive is
|
|
435
|
+
offered again at a later boundary; no turn is ever started for one. An
|
|
436
|
+
acknowledgement says the context reached the native session, not that the
|
|
437
|
+
model read it. 60 changed lines are shown at most, the cut files named; a
|
|
438
|
+
history longer than the hub keeps is shown with its attribution unknown, and a
|
|
439
|
+
file that falls out of the 64 a peer touched is named once. Only regular files
|
|
440
|
+
of 256 KB or less inside the project are read, re-resolved at every read and
|
|
441
|
+
opened without following links; larger ones are named without a diff. Facts never go through the bus or the delivery journal;
|
|
442
|
+
`events.jsonl` records `fact`, `fact_ack` and `capability` events with bytes
|
|
443
|
+
and latencies.
|
|
444
|
+
- Integration. A member's `hub_task_done` is a completion intent. The member
|
|
445
|
+
whose intent completes the set is asked, as its done result, to check its work
|
|
446
|
+
against the others' (their files, signatures and summaries, plus its own facts)
|
|
447
|
+
and to call `hub_task_done` again; nothing is recorded as done yet. The next
|
|
448
|
+
call counts only for the same target: the same owner, cohort revision and
|
|
449
|
+
files (the named ones, and those each member wrote with an edit tool the hub
|
|
450
|
+
saw, Claude's Edit, MultiEdit and Write or a Codex patch, between being handed
|
|
451
|
+
its task and settling, so a symbol-only overlap counts and
|
|
452
|
+
a member settling keeps its files in; reading a file never moves it, a settled
|
|
453
|
+
member's later work does not count, a shell command's writes outside the named
|
|
454
|
+
paths are not seen, and a file the hub reads whole whose bytes equal HEAD's
|
|
455
|
+
is no change, whatever git's stat data says), with every other owner's native turn
|
|
456
|
+
ended after its done (or that owner idle when it finished); the integrating
|
|
457
|
+
owner's own other tasks in the cohort never count as still running. A done of a member by the console counts as its
|
|
458
|
+
intent too. Edits in between, a new member, an owner change, a failed check or
|
|
459
|
+
a reopened review ask again; a done within two seconds of a request is taken as a retry and gets the
|
|
460
|
+
same request again; after three requests the done is recorded with
|
|
461
|
+
`integration unresolved`, never as integrated, and that revision asks nothing
|
|
462
|
+
more (a configured check then counts as usual). A configured check of the integrating member
|
|
463
|
+
counts only for the target it confirmed; when another member reopens its task,
|
|
464
|
+
that member integrates instead and the earlier one's check counts as usual.
|
|
465
|
+
Inside a silent cohort a member's completed-change notice is held, not sent;
|
|
466
|
+
where no integration step runs for the others (the silence was lifted, a PII
|
|
467
|
+
task opened), the held notices go with the lift notice or the done result, and
|
|
468
|
+
for a task the console finishes, to the console. Open tasks outside the cohort
|
|
469
|
+
get their notices as usual. Facts sent with an integration request are
|
|
470
|
+
offered again at the next boundary until the next done acknowledges them.
|
|
471
|
+
Cohorts live in memory: after a hub restart an open
|
|
472
|
+
request is recorded as unresolved, and when a peer first attaches, each of its
|
|
473
|
+
open tasks that overlaps other work hears that overlaps are settled by message
|
|
474
|
+
again, with the completed-change notices of overlapping tasks finished since
|
|
475
|
+
it was handed the task (a notice may come twice; none is lost).
|
|
476
|
+
- While any PII task is open the project behaves as advisory: no facts, no
|
|
477
|
+
silence and no integration step, and the cohorts that were silent stay lifted.
|
|
478
|
+
When it closes, the hub forgets what it had observed, so nothing changed
|
|
479
|
+
meanwhile is shown as a diff; each member is told which of its files to read
|
|
480
|
+
again.
|
|
481
|
+
- `ahub check-path` asks the hub whether the owner of the claimed path shares a
|
|
482
|
+
silent cohort with the caller, and only then leaves out the request to settle
|
|
483
|
+
by message. `templates/claude-hooks.json` holds only the check-path hook; the
|
|
484
|
+
facts hooks need a hub-launched session.
|
|
485
|
+
- Routing does not change. When a task's overlap with another owner's open task
|
|
486
|
+
forms or changes a cohort, routed or named, the hub records a shadow split
|
|
487
|
+
prediction (`split` event; `ahub route explain <id>` shows its trace as it would
|
|
488
|
+
be now): whether splitting two equal units between the two peers
|
|
489
|
+
(`o_s + u_s < o_f + 2u_f`) would finish sooner than the faster one alone, from
|
|
490
|
+
this hub run's recorded task stages. It is unknown unless the units are equal
|
|
491
|
+
and known, both peers are available (the other owner idle; the task's own peer
|
|
492
|
+
idle or busy taking it) with no other open work (an overlapping task its owner
|
|
493
|
+
has started counts), and each has five measured tasks with no more than 30%
|
|
494
|
+
failures and comparable work times. The work stage of a task ends at its first
|
|
495
|
+
`hub_task_done`, and the task itself is never one of its own observations.
|
|
496
|
+
- `"experiments": {"stale_notices": "deliver"}` in `.agenthub/config.json`
|
|
497
|
+
turns the stale-notice drop off, for a controlled comparison only (the #106
|
|
498
|
+
ablation in `docs/cooperbench.md`); the hub logs it at start.
|
|
499
|
+
|
|
347
500
|
## Approvals and pauses
|
|
348
501
|
|
|
349
502
|
Inspect permission requests in the terminal:
|
|
@@ -474,21 +627,21 @@ Rows without a live process are stale registrations; forget them with
|
|
|
474
627
|
|
|
475
628
|
Upgrade running projects with the target release's own coordinator. It accepts
|
|
476
629
|
a running source on control protocol 9 (0.6.x), 10 (0.7.0 through 0.12.0),
|
|
477
|
-
|
|
630
|
+
11 (0.12.1 and 0.12.2) or 12 (0.12.3) and only
|
|
478
631
|
a target on its own protocol, so the target's coordinator fits every supported
|
|
479
632
|
source and carries every recovery fix released up to it. Protocol 8 and older
|
|
480
633
|
(0.5.x and earlier) are refused as `manual-bootstrap-required`. Run from the
|
|
481
634
|
project directory, without replacing the global CLI first:
|
|
482
635
|
|
|
483
636
|
```bash
|
|
484
|
-
bunx --package @staix/agent-hub@0.12.
|
|
485
|
-
bunx --package @staix/agent-hub@0.12.
|
|
637
|
+
bunx --package @staix/agent-hub@0.12.4 ahub upgrade --to 0.12.4 --dry-run
|
|
638
|
+
bunx --package @staix/agent-hub@0.12.4 ahub upgrade --to 0.12.4 --yes
|
|
486
639
|
```
|
|
487
640
|
|
|
488
641
|
| Running now | Coordinator to use |
|
|
489
642
|
| --- | --- |
|
|
490
643
|
| 0.6.x (protocol 9) | the target's, through `bunx` as above |
|
|
491
|
-
| 0.7.0 through 0.12.0 (protocol 10) | the target's, through `bunx` as above |
|
|
644
|
+
| 0.7.0 through 0.12.0 (protocol 10), 0.12.1 and 0.12.2 (protocol 11), 0.12.3 (protocol 12) | the target's, through `bunx` as above |
|
|
492
645
|
| any supported source, with the installed CLI already at the target | `ahub upgrade` below, which is the same coordinator |
|
|
493
646
|
| 0.5.x or earlier (protocol 8 and older) | not supported: bootstrap by hand with the matching CLI |
|
|
494
647
|
|
|
@@ -1187,4 +1187,33 @@ of a native turn boundary. Failed notifications, disconnects, replaced sessions
|
|
|
1187
1187
|
and interrupted daemon instances retain `needs_review`, with no automatic replay.
|
|
1188
1188
|
A person inspects and resolves uncertain work through the existing revision-fenced
|
|
1189
1189
|
`ahub queue` commands. Older source protocols 9, 10 and 11 remain authenticated
|
|
1190
|
-
upgrade sources; ordinary clients must use protocol 12.
|
|
1190
|
+
upgrade sources; ordinary clients must use protocol 12 (13 from 0.12.4, with 12 a source).
|
|
1191
|
+
|
|
1192
|
+
## Amendment: stale overlap notices (issue #106)
|
|
1193
|
+
|
|
1194
|
+
- The completed-change notice (#31) and the edit-conflict notices (#32, #91) are about an open task of their recipient. `Tasks.whileOpen` publishes each to one recipient and remembers the condition (the newest 1024). The bus asks `Tasks.relevant(peer, env)` per recipient when it builds a delivery and again right before it hands the delivery over, after condensation; it is false only for a recorded notice to that peer whose task is gone, has another owner, or left the states it was about: `proposed`, `in_progress`, `changes_requested`, and for conflict notices also `in_review` (#91 reports conflicts with work under review). A dropped copy is a journal row `discarded` with `stale: task #N is no longer open for <peer>`, a `STALE` log line and a `stale` event; other recipients and other deliveries are untouched.
|
|
1195
|
+
- An envelope without a record (another kind, a restart, an evicted record) is delivered as before: its purpose is never guessed from its kind. Assignments, approvals, check results, review requests and budget, permission and recovery messages are never conditional.
|
|
1196
|
+
- The ablation is the advisory arm with and without stale-notice dropping on the same release (`manifest-v2-ablation-106.json`, #110); comparing with 0.12.3's advisory runs would confound it with that release's Codex hooks.
|
|
1197
|
+
|
|
1198
|
+
## Amendment: turn-free cohorts (issue #107)
|
|
1199
|
+
|
|
1200
|
+
- `coordination` in the project config: `"advisory"` (default) or `"turn-free"`; anything else is advisory with a log line. While a PII task is open the project is advisory.
|
|
1201
|
+
- `src/hub/cohorts.ts`: owners of overlapping open tasks (#31) form a cohort when the overlap is found (assignment, an accept with a plan). Members are tasks with their owner and owner generation (ownership events); every change of membership or owner, a withdrawn intent and a lift bump the cohort's revision. A cohort is silent if, when formed, every owner's context path is verified (#108); it never becomes silent later, and it is lifted for good (members still at work told at once, with the held notices) when an unverified owner joins, a path is lost or a PII task opens. Cohorts live in memory: after a restart, each peer's first attach tells its open overlapping tasks that overlaps are settled by message again and replays the completed-change notices of overlapping tasks finished since they were handed over.
|
|
1202
|
+
- Silence is a per-recipient bus policy (`BusOptions.silence`), applied once the audience is final (implicit replies and `digest` resolved): an agent's chat from a member that has not settled to another member is not queued for that member; the other recipients get the envelope unchanged, so correlation, priority caps, dedupe and hops are untouched. A `quiet` bus event goes to the log and `events.jsonl`; `hub_send` answers with the held-back peers (an error when nobody got it); a native turn answer's sender hears it as a note. A member settles once its task left the open states and its native turn has ended after that (Codex's adapter leaving its turn, Claude's Stop hook; a member already between turns when its task closes settles at once; a paused peer counts by its adapter's state, and Claude's channel going offline says nothing about its session). Settlement is recorded as it happens (`Cohorts.turnEnded`, `Cohorts.closed`) and never undone: a settled member's next turn is new work, and a cohort whose members all settled is over. A message held back from all its recipients is not admitted against the sender's limits.
|
|
1203
|
+
- Texts: in a silent cohort, overlap results and notices, task envelopes and the conflict notices name the plans and say not to message; `hub_task_accept` returns the overlapping plans with every accept; `ahub check-path` asks the hub (`silenced` control request). A member's completed-change notice to another member is held (`Cohort.held`), never dropped: it goes with the lift notice when silence is lifted, with the done result of a member for whom no integration step runs (turn-free off, a PII task open), or to the console when the console finishes the task (its done still counts as that member's intent). Notices to open tasks outside the cohort go as usual. Otherwise the advisory texts stay.
|
|
1204
|
+
- Integration: a member's `hub_task_done` records an intent. The member whose intent completes the set is selected in one synchronous step and gets an integration request as its done result (the others' files, signatures and summaries, plus its facts); the request is recorded as `integration requested`, not done. Its next done is accepted (`integrated`, then the usual done) only for the same owner generation, cohort revision and file hash (named files, for a directory git's changed, new and deleted files under it against HEAD, and the files each member wrote with an edit tool the hub saw (Claude's Edit, MultiEdit and Write; Codex's patches) between being handed its task and settling; reads never count, nor shell writes outside the named paths, and a file the hub reads whole whose bytes equal HEAD's is no change there), with its facts acknowledged (the done first acknowledges the facts sent with the request) and every other owner stopped after its intent (a native turn end after it, or idle when it was recorded); the integrating owner's own other tasks never count as running. A cohort whose members have all settled is over, whoever asks first, and is never lifted or reopened after that. A done within two seconds of a request is a retry and gets the same request, with nothing acknowledged or counted. Otherwise it is asked again; past three requests the done is recorded with `integration unresolved`, and that revision asks nothing more (the member's check counts as usual). A failed check or changes requested withdraws that member's intent; a check of the integrating member that passes after its target moved within the same revision counts as finished late; once the revision moved (another member reopened), the former integrating member is an earlier finisher and its check counts. After a restart an open request is recorded as unresolved.
|
|
1205
|
+
|
|
1206
|
+
## Amendment: turn-free facts at tool boundaries (issue #108)
|
|
1207
|
+
|
|
1208
|
+
- `src/hub/facts.ts` keeps three things apart per peer: what the hub observed in the tree (latest version and transitions per file), what it offered (bounded offers with their file versions and plans), and what the peer acknowledged (its view). Only an acknowledgement moves a view; a later boundary offers everything since the view again. Files are the paths every task of each live cohort in which the peer has an open task names (a directory expands to git's changed files against HEAD, staged or not, and its new and deleted files, 200 at most, with a note when more were cut, told until it is read back and again in a new session), contained in the project as real paths and never `.git` or anything the denylist (`src/local/deny.ts`, `local.deny`) keeps from agents, plus the last 64 it touched; a file that falls out of those is named once. A file the peer touched before it had a view of it is compared with its version at that touch, so a change in between is shown, not absorbed into a first look; a file it saw under a named directory stays covered after git stops listing it (put back to HEAD's bytes, or the directory moved away; 200 at most, the rest named once as no longer tracked). A directory file it has neither seen nor touched is named without a diff when it appears (diffing it against HEAD would show what changed while tracking was off, a PII window, and credit unseen changes by elimination), and counts as not yet shown until that fact is read back. Only regular files of 256 KB or less are read, re-resolved at every read and opened without following a final link or blocking. A history longer than the 64 kept transitions makes the change's attribution unknown.
|
|
1209
|
+
- Attribution: a transition is credited only with effect evidence: a Claude Edit, MultiEdit or Write whose result equals its input applied to the file as observed at PreToolUse with no observation in between, or a Codex `fileChange` whose changed lines equal the observed diff. Everything else is shown with its attribution unknown; an agent's own verified writes advance its view and are not shown back. A Claude Read advances its view only when it returned the whole file (no offset or limit, at most 2000 lines, none over 2000 characters); a Codex read action never does (it may be partial). A diff that matches a PII pattern is replaced by a note naming the file, and a changed file whose name matches one is counted without its name.
|
|
1210
|
+
- Acknowledgement and capability: Claude's hook runs before and after every tool and at Stop (`ahub claude` injects it in a turn-free project); the pre answer's text becomes `additionalContext`, and the row Claude Code writes in its transcript for it (matched by tool use id and offer id) is the readback. Codex's offer goes in with `steerText` (bound to the running turn), and the steered input coming back as a `userMessage` item is the readback. A readback marks the peer's path verified; a new session or thread, the peer going offline, or three offers past a minute without one (checked at boundaries and every 30 seconds; facts sent with an integration request wait for the next done and never count, a refused steer is dropped, not unread, and an unanswered one stays pending, since it may have gone in) unverifies it. Until verified, a peer gets a one-line probe at most three times per session, and no offers at all once three are unread; a Claude session whose transcript the hub cannot find gets none at all and loses a verified path.
|
|
1211
|
+
- The control contract is protocol 13: `facts` carries the phase, tool, input, tool use id, session id and transcript path; `silenced` serves check-path; `send` results name held-back recipients. Protocol 12 stays an authenticated recovery source.
|
|
1212
|
+
- While a PII task is open nothing is observed or offered; every board change re-checks this, so tracking switches at once. When tracking resumes everything observed is dropped and each peer is told which files earlier changes are not covered for. Turn ends and tool starts are recorded in every regime.
|
|
1213
|
+
- `events.jsonl`: `fact` (bytes, build time, the hook's own time, the steer round trip), `fact_ack` (latency), `capability`, `native_turn_end`, `hook_stats` (every facts hook call of a Claude turn, at its Stop), `cohort`.
|
|
1214
|
+
|
|
1215
|
+
## Amendment: shadow split prediction (issue #109)
|
|
1216
|
+
|
|
1217
|
+
- `predictSplit(input)` is a pure function next to `assign()`, which no longer takes a split option: assignment never changes. For the pair formed by the peer routing picks and the owner of an overlapping open task, it compares a split (the later of each peer's orientation plus one unit) with the best peer alone (orientation plus two units). It is unknown unless the units are equal and known (one task of the same class is one unit), both peers are available (the other owner idle; the task's own peer idle or busy taking it) with no other open work (an overlapping task its owner has started counts), and each has five measured tasks in this hub run with at most 30% failures and work times whose IQR does not exceed their median; a difference under a tenth of the single time is inconclusive.
|
|
1218
|
+
- `Tasks.splitObservations` reads only tasks handed out in this hub run (one version and hook profile), never the routed task itself, leaves out claims, types failures (failed check, changes requested, escalation, release, decline, unresolved integration), ends the work stage at the first done call (checks and integration are not work) and leaves the stages unknown for an accept recorded by the done itself. The `split` event carries the trace.
|
|
1219
|
+
- An overlap that forms or changes a cohort records a `split` event for the task, routed or named (a benchmark names every owner); `ahub route explain <id>` appends the trace as it would be now. Rerouting needs held-out evidence through #110 first.
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# Agent-hub 0.12.4 verification
|
|
2
|
+
|
|
3
|
+
Implementation and candidate verification for issues #106 through #110, performed on 2026-10-02. The release pilot ran on runtime and runner source `ae8f8f5`; one more four-arm repeat ran on `3555ed5`, after the later review fixes. The released source differs from `3555ed5` in the facts for files under a named directory (no live task named a directory), a behaviour-preserving refactor in `tasks.ts` and `native.ts` (the latter changes the runner hash, so that repeat cannot be re-graded by the released runner), the fact event type, the ledger, tests and docs. Production application is verified separately after publication.
|
|
4
|
+
|
|
5
|
+
## Runtime checks
|
|
6
|
+
|
|
7
|
+
- Recovery: a real published 0.12.3 package (protocol 12) was recovered into the candidate (protocol 13) with `scripts/smoke-recovery-09-10.ts` at `ae8f8f5`, its queued envelope ids preserved (operation `975034f1-b2c0-4aba-9ba9-611ef1ce5daa`). The control protocol did not change after it.
|
|
8
|
+
- Context paths: in every turn-free attempt and in the setup-only calibration, both native readbacks verified before the first task. Claude Code 2.1.287 wrote the hook's additional context as a transcript row matched by tool use id and offer id; Codex 0.159.3 returned the steered fact as a user message item of the running turn.
|
|
9
|
+
- Facts reached the owners and were read back: 17 offers in the three pilot attempts, 37,936 bytes, all acknowledged; 5 more (all acknowledged) in the repeat on `3555ed5`. No attempt lost a context path while the agents worked.
|
|
10
|
+
- Isolation: no attempt ran a Codex hook or a Claude hook other than the hub's facts hook. Every Codex started only the hub's MCP server: its wrapper turns off the user's plugins, apps, sub-agents and turn-end notifier and disables each MCP server of the user's configuration by name. In the repeat on `3555ed5` Claude's transcript shows the fixture instructions it received through `--append-system-prompt-file`.
|
|
11
|
+
|
|
12
|
+
## Native CooperBench release pilot
|
|
13
|
+
|
|
14
|
+
Feasibility only (#110, T9): one pair (`pallets_click_task:2068`, features 1 and 6), three repeats of the four v2 arms and three repeats of the #106 ablation, graded by the official tests. No adoption claim is made; advisory stays the default.
|
|
15
|
+
|
|
16
|
+
- Official source commit: `63b9d44d9f39a02fccf5bf0052db48a917a011fd`.
|
|
17
|
+
- Codex CLI 0.159.3, `gpt-6.1-sol`, medium effort; Claude Code 2.1.287, `claude-opus-5-5`, medium effort.
|
|
18
|
+
- Each feature-work arm has a 300-second wall limit; setup and denied hidden-file probes are unscored. The arm order is a Williams design, a different row per repeat.
|
|
19
|
+
- The official evaluator runs on the recorded image digest, with empty-base-fail and combined-oracle-pass controls; every control passed.
|
|
20
|
+
|
|
21
|
+
Seconds to both tasks done (active time in parentheses), and the official grade:
|
|
22
|
+
|
|
23
|
+
| Repeat | Solo Codex | Solo Claude | Advisory | Turn-free | Grading |
|
|
24
|
+
|---|---|---|---|---|---|
|
|
25
|
+
| 0 (`ae8f8f5`) | 174.6 (183.9) | 43.6 (49.6) | 176.3 (189.3) | 161.3 (172.2) | 4/4 passed both features |
|
|
26
|
+
| 1 (`ae8f8f5`) | 189.1 (198.3) | 54.3 (59.9) | 153.2 (165.9) | 168.8 (178.7) | 4/4 passed |
|
|
27
|
+
| 2 (`ae8f8f5`) | 170.6 (179.3) | 39.9 (45.1) | 173.3 (186.9) | 151.8 (161.1) | 3/4 passed; Solo Claude unavailable |
|
|
28
|
+
| 3 (`3555ed5`) | 154.9 (164.1) | 39.2 (45.1) | 153.6 (167.4) | 147.2 (156.5) | 4/4 passed |
|
|
29
|
+
|
|
30
|
+
Repeat 2's Solo Claude attempt completed, but its grade is unavailable: Claude Code wrote its last answer and a turn-duration row about 0.1 s after the attempt's transcript hash was taken, so the integrity check reported "transcript changed since the attempt". Attempts now record the transcript's length with its hash and read that prefix only (`3555ed5`; repeat 3 graded under it). The teardown starts five seconds after the hub sees both tasks done and every peer idle, which a final answer still being written can outlast; that answer is then not counted, and the agent's settlement is unknown. The pilot's prepared runner hashes pin the earlier check, so repeat 2 is not re-graded, and its prepared hashes were not rewritten.
|
|
31
|
+
|
|
32
|
+
Coordination over the three pilot repeats (ledger medians over valid attempts; whole-attempt usage per agent; counters of different providers are never added):
|
|
33
|
+
|
|
34
|
+
| Measure | Advisory | Turn-free |
|
|
35
|
+
|---|---|---|
|
|
36
|
+
| Both done, median s | 173.3 | 161.3 |
|
|
37
|
+
| Settlement (all agents stopped), median s | 181.7 | 167.0 |
|
|
38
|
+
| Codex turns / token-usage growth / tokens, median | 2 / 21 / 835,130 | 1 / 13 / 494,408 |
|
|
39
|
+
| Claude assistant messages / tokens, median | 8 / 213,333 | 8 / 211,666 |
|
|
40
|
+
| Codex `hub_send` calls, total | 10 | 0 |
|
|
41
|
+
| Codex turns after its done, total | 3 | 0 |
|
|
42
|
+
| Claude messages reaching Codex after its done, total | 6 | 0 |
|
|
43
|
+
| Stale notices dropped (#106), total | 3 | 0 |
|
|
44
|
+
| Integration requests / unresolved, total | 0 / 0 | 3 / 0 |
|
|
45
|
+
| Turn-free treatment received (silent cohort) | - | 3 of 3 |
|
|
46
|
+
|
|
47
|
+
Every turn-free attempt took one integration request and confirmed it; none was unresolved. Each advisory post-done turn was started by a late reply from Claude. Repeat 3, on `3555ed5`, matches the pattern: turn-free one integration request and no post-done turn; advisory one post-done turn after two late replies.
|
|
48
|
+
|
|
49
|
+
#106 ablation (advisory with and without stale-notice dropping, three repeats each, all graded and passed both features):
|
|
50
|
+
|
|
51
|
+
| Measure | Dropping on | Dropping off |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| Both done, median s | 153.1 | 158.6 |
|
|
54
|
+
| Stale notices dropped, total | 3 | 0 |
|
|
55
|
+
| Codex turns after its done, total | 3 | 3 |
|
|
56
|
+
| What started them | a late reply | a late reply with the stale notice |
|
|
57
|
+
| Codex tokens, median | 737,127 | 864,458 |
|
|
58
|
+
|
|
59
|
+
The drop removed the stale notice from every post-done turn, but in this pair the turn still happened because a late reply arrived with it. In the turn-free arm neither occurred in the pilot (no late reply, no post-done turn).
|
|
60
|
+
|
|
61
|
+
Contribution heuristic: three attempts (one turn-free, one advisory, one ablation) flagged one changed fragment each as absent from the final tree; all three passed the official grading. The heuristic cannot tell a rewrite from a loss, and a passing grade shows only that the features' tests pass. No identifier was flagged. Shell commands' writes are not attributed (each attempt lists them under coverage).
|
|
62
|
+
|
|
63
|
+
This is a one-pair shared-workspace sample with three repeats per arm: time differences of this size are within its noise, and no performance, resource or causal claim is made. Native session counters include setup probes only where the ledger says so; prompts, conversations, transcripts, private inputs, gold solutions and `ledger.json` (which holds code fragments and local paths) stay outside version control.
|
|
64
|
+
|
|
65
|
+
## Automated gate and review
|
|
66
|
+
|
|
67
|
+
`scripts/check.sh` on the released source: 689 tests passed, 0 failed, 3,506 expectations across 65 files; typecheck, bundle freshness, npm package contents and process ownership passed; `check: OK`. Linux and macOS CI passed on the final head. Five earlier heads' macOS runs failed, all on timing: the end-to-end test's steer window at `ae8f8f5` (its fake now leaves a wider one), a large-directory test's 5 s timeout at `3555ed5` (given a longer one), and two known flaky tests, a permission test whose 200 ms permission timeout a slow runner outlasts (`03122f9`, `c822f5c`, `12071c4`) and a sidecar test's 5 s timeout (`c822f5c`).
|
|
68
|
+
|
|
69
|
+
Independent host Claude Code OCR delegation reviews (read-only reviewers with the OCR rule groups, `REVIEW.md` and the `AGENTS.md` invariants) covered the runtime and benchmark changes in seven rounds, each on the exact head, and this document as well. Every Critical and Important finding was fixed and re-reviewed; the last code round found none. Changes between `ae8f8f5` and `3555ed5` were checked by tests and the repeat on `3555ed5`; later runtime changes by tests only, not by a second full pilot.
|
|
70
|
+
|
|
71
|
+
What the rounds changed, in brief:
|
|
72
|
+
|
|
73
|
+
- Settlement is recorded when it happens and never undone by a later turn (only reopening the task clears it); a paused or channel-offline peer is not taken as stopped; a finished cohort is never lifted at teardown.
|
|
74
|
+
- The integration target covers the named paths and what each member wrote with an edit tool the hub saw (Claude's Edit, MultiEdit and Write; Codex's patches) between its hand-over and its settlement (never what it only read, and a shell command's writes only under the named paths); an unresolved outcome closes its revision.
|
|
75
|
+
- Facts never read `.git` or denylisted paths; a diff matching a PII pattern is not shown; integration-request facts and refused steers never count as unread; a file under a named directory that the peer never saw is named without a diff, never compared with HEAD (one round's HEAD comparison could have shown, after a PII window, what was written during it, and was reverted).
|
|
76
|
+
- Benchmark validity gates are shared by the grader and the ledger: capability failures while the agents work exclude an attempt, teardown does not, and Codex and Claude run with hook and MCP isolation; a wrapper in each fixture turns off Codex's plugins, apps, sub-agents and turn-end notifier and disables the MCP servers of the user's configuration (Codex still reads the user's global `AGENTS.md` and can use the user's Codex skills, the same in every arm).
|
|
77
|
+
|
|
78
|
+
Known limits of 0.12.4 (minor findings of the last code round, accepted, to be filed as a follow-up):
|
|
79
|
+
|
|
80
|
+
- Seen files under a named directory beyond the 200 kept are named under the "no longer tracked" notice, whose stated reason is the 64-file touched limit; that notice is spent when an offer is built, not when it is read back, and a new session does not hear it again.
|
|
81
|
+
- A file under a named directory that is rewritten with identical bytes is named as changed to a member that never saw it until git refreshes its index; during an integration this can cost one more request, or, after the third request, record the outcome as unresolved.
|
|
82
|
+
- An offer that only names directory files reports zero files in its fact event.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-hub",
|
|
3
|
-
"version": "0.12.
|
|
3
|
+
"version": "0.12.4",
|
|
4
4
|
"description": "Channel between Claude Code and the agent-hub daemon: peer messages from Codex, Kimi and the local worker arrive as channel events; hub_send replies.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Young Joon Lee",
|
|
@@ -15615,8 +15615,8 @@ function projectContext(cwd, env = process.env) {
|
|
|
15615
15615
|
function stateDirFor(cwd) {
|
|
15616
15616
|
return projectContext(cwd).stateDir;
|
|
15617
15617
|
}
|
|
15618
|
-
var PROTOCOL =
|
|
15619
|
-
var RECOVERY_SOURCE_PROTOCOLS = [9, 10, 11, PROTOCOL];
|
|
15618
|
+
var PROTOCOL = 13;
|
|
15619
|
+
var RECOVERY_SOURCE_PROTOCOLS = [9, 10, 11, 12, PROTOCOL];
|
|
15620
15620
|
function readControl(stateDir) {
|
|
15621
15621
|
try {
|
|
15622
15622
|
const status = JSON.parse(readFileSync2(join2(stateDir, "status.json"), "utf8"));
|
|
@@ -15727,7 +15727,7 @@ class ControlClient {
|
|
|
15727
15727
|
// package.json
|
|
15728
15728
|
var package_default = {
|
|
15729
15729
|
name: "@staix/agent-hub",
|
|
15730
|
-
version: "0.12.
|
|
15730
|
+
version: "0.12.4",
|
|
15731
15731
|
description: "Native multi-agent hub: Claude Code, Codex, Kimi Code, Pi and local inference as peers in one project",
|
|
15732
15732
|
license: "MIT",
|
|
15733
15733
|
type: "module",
|
|
@@ -16039,7 +16039,8 @@ server.setRequestHandler(CallToolRequestSchema, async (req) => {
|
|
|
16039
16039
|
return text(`not sent: ${res.error}`);
|
|
16040
16040
|
if (res.recorded)
|
|
16041
16041
|
return text("recorded only ([FYI]): it is on the hub console and log, and no peer spent a turn on it");
|
|
16042
|
-
|
|
16042
|
+
const sent = `sent to: ${res.targets.join(", ") || "(no other peers attached)"}`;
|
|
16043
|
+
return text(typeof res.notice === "string" ? `${sent}; ${res.notice}` : sent);
|
|
16043
16044
|
}
|
|
16044
16045
|
if (TASK_TOOL_NAMES.has(name)) {
|
|
16045
16046
|
if (!hub)
|
|
@@ -222,7 +222,8 @@ server.setRequestHandler(CallToolRequestSchema, async (req) => {
|
|
|
222
222
|
const res = await hub.request({ t: "send", body, to, reply_to });
|
|
223
223
|
if (!res.ok) return text(`not sent: ${res.error}`);
|
|
224
224
|
if (res.recorded) return text("recorded only ([FYI]): it is on the hub console and log, and no peer spent a turn on it");
|
|
225
|
-
|
|
225
|
+
const sent = `sent to: ${res.targets.join(", ") || "(no other peers attached)"}`;
|
|
226
|
+
return text(typeof res.notice === "string" ? `${sent}; ${res.notice}` : sent);
|
|
226
227
|
}
|
|
227
228
|
if (TASK_TOOL_NAMES.has(name)) {
|
|
228
229
|
if (!hub) return text(offline());
|
|
@@ -22,8 +22,17 @@ export interface CodexOptions {
|
|
|
22
22
|
onTokens?: (added: number) => void;
|
|
23
23
|
/** The native id of each turn as it starts, after the peer turned busy (issue #33: `ahub undo --context`). */
|
|
24
24
|
onTurn?: (turnId: string) => void;
|
|
25
|
+
/**
|
|
26
|
+
* Each completed `fileChange`, `commandExecution`, `mcpToolCall` and `userMessage` item, for turn-free facts (issue
|
|
27
|
+
* #108): the first three are boundaries, a `userMessage` is the readback of a steered fact. Codex emits
|
|
28
|
+
* `item/started` about when a command has finished, so only completions are reported. Untyped JSON (hence `any`):
|
|
29
|
+
* the receiver checks every field it reads.
|
|
30
|
+
*/
|
|
31
|
+
onItem?: (item: any) => void;
|
|
25
32
|
/** How often to ask app-server for the rate limits while a TUI is attached. */
|
|
26
33
|
usagePollMs?: number;
|
|
34
|
+
/** How long a fact's steer waits for app-server's answer (tests shorten it). */
|
|
35
|
+
steerTimeoutMs?: number;
|
|
27
36
|
cwd: string;
|
|
28
37
|
/** Optional launch environment; recovery authority is always removed before spawn. */
|
|
29
38
|
env?: NodeJS.ProcessEnv;
|
|
@@ -88,6 +97,11 @@ export class CodexPeer extends BasePeer {
|
|
|
88
97
|
};
|
|
89
98
|
}
|
|
90
99
|
|
|
100
|
+
/** The app-server thread the hub drives; a new one is a new native session. */
|
|
101
|
+
get thread(): string {
|
|
102
|
+
return this.threadId;
|
|
103
|
+
}
|
|
104
|
+
|
|
91
105
|
get proxyUrl(): string {
|
|
92
106
|
return `ws://127.0.0.1:${this.server?.port ?? this.opts.proxyPort}`;
|
|
93
107
|
}
|
|
@@ -220,6 +234,24 @@ export class CodexPeer extends BasePeer {
|
|
|
220
234
|
});
|
|
221
235
|
}
|
|
222
236
|
|
|
237
|
+
/**
|
|
238
|
+
* A turn-free fact for the running turn (issue #108). Not a message: no envelope, no delivery record, nothing the
|
|
239
|
+
* turn's answer is addressed to. `refused` when there is no running turn or app-server says no: it never went in.
|
|
240
|
+
* `unanswered` when app-server did not answer in time: it may have gone in, and its readback can still come.
|
|
241
|
+
*/
|
|
242
|
+
steerText(text: string): Promise<"accepted" | "refused" | "unanswered"> {
|
|
243
|
+
const link = this.link;
|
|
244
|
+
const expectedTurnId = [...this.activeTurns].reverse().find((t) => !t.startsWith("unknown:"));
|
|
245
|
+
if (!link || link.up.readyState !== WebSocket.OPEN || !expectedTurnId) return Promise.resolve("refused");
|
|
246
|
+
const id = this.nextId--;
|
|
247
|
+
return new Promise((resolve) => {
|
|
248
|
+
const timer = setTimeout(() => this.pending.delete(id) && resolve("unanswered"), this.opts.steerTimeoutMs ?? 10_000);
|
|
249
|
+
timer.unref?.();
|
|
250
|
+
this.pending.set(id, { resolve: () => (clearTimeout(timer), resolve("accepted")), reject: () => (clearTimeout(timer), resolve("refused")) });
|
|
251
|
+
link.up.send(JSON.stringify({ method: "turn/steer", id, params: { threadId: this.threadId, expectedTurnId, input: [{ type: "text", text }] } }));
|
|
252
|
+
});
|
|
253
|
+
}
|
|
254
|
+
|
|
223
255
|
/** A silent turn is interrupted before the peer is declared idle, or the next turn/start would land inside it. */
|
|
224
256
|
protected override onWatchdog(): void {
|
|
225
257
|
for (const turnId of this.activeTurns) {
|
|
@@ -417,6 +449,12 @@ export class CodexPeer extends BasePeer {
|
|
|
417
449
|
const buf = this.deltas.get(params.itemId) ?? [];
|
|
418
450
|
buf.push(params.delta);
|
|
419
451
|
this.deltas.set(params.itemId, buf);
|
|
452
|
+
} else if (method === "item/completed" && ["fileChange", "commandExecution", "mcpToolCall", "userMessage"].includes(params.item?.type)) {
|
|
453
|
+
try {
|
|
454
|
+
this.opts.onItem?.(params.item);
|
|
455
|
+
} catch (error) {
|
|
456
|
+
this.opts.log?.(`[${this.id}] item handler failed: ${(error as Error).message}`); // the proxy keeps going
|
|
457
|
+
}
|
|
420
458
|
} else if (method === "item/completed" && params.item?.type === "agentMessage") {
|
|
421
459
|
const item = params.item;
|
|
422
460
|
const text: string =
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
#!/usr/bin/env bun
|
|
2
|
+
// Claude Code PreToolUse, PostToolUse and Stop hook for turn-free facts (issue #108). `ahub claude` adds it to the
|
|
3
|
+
// session's settings in a turn-free project. It never blocks or fails a tool call: a hub that is down, an advisory
|
|
4
|
+
// project or any error prints nothing. A fact comes back as `additionalContext`, which Claude Code adds with the tool
|
|
5
|
+
// call; the hub confirms it from the transcript row Claude Code writes for it, by tool use id, before it counts as seen.
|
|
6
|
+
import { ControlClient } from "../hub/control-client.ts";
|
|
7
|
+
|
|
8
|
+
/** The hook's stdout for one Claude Code hook input, or undefined for none. */
|
|
9
|
+
export async function factsHook(stdin: string, stateDir: string, peer: string, timeoutMs = 2000): Promise<string | undefined> {
|
|
10
|
+
const input = JSON.parse(stdin) as { hook_event_name?: unknown; tool_name?: unknown; tool_input?: unknown; tool_use_id?: unknown; session_id?: unknown; transcript_path?: unknown };
|
|
11
|
+
const phase = ({ PostToolUse: "post", Stop: "stop" } as Record<string, string>)[String(input.hook_event_name)] ?? "pre";
|
|
12
|
+
const hub = await ControlClient.connect(stateDir, { role: "tools", peer }, timeoutMs);
|
|
13
|
+
try {
|
|
14
|
+
const res = await hub.request({
|
|
15
|
+
t: "facts",
|
|
16
|
+
phase,
|
|
17
|
+
tool: typeof input.tool_name === "string" ? input.tool_name : "",
|
|
18
|
+
input: input.tool_input ?? {},
|
|
19
|
+
...(typeof input.tool_use_id === "string" ? { toolUseId: input.tool_use_id } : {}),
|
|
20
|
+
...(typeof input.session_id === "string" ? { sessionId: input.session_id } : {}),
|
|
21
|
+
...(typeof input.transcript_path === "string" ? { transcriptPath: input.transcript_path } : {}),
|
|
22
|
+
startedMs: performance.now(), // this process's own start-up and connect time, for the latency record
|
|
23
|
+
}, timeoutMs);
|
|
24
|
+
if (phase !== "pre" || !res?.ok || typeof res.text !== "string" || !res.text) return undefined;
|
|
25
|
+
return JSON.stringify({ hookSpecificOutput: { hookEventName: "PreToolUse", additionalContext: res.text } });
|
|
26
|
+
} finally {
|
|
27
|
+
hub.close();
|
|
28
|
+
}
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
if (import.meta.main) {
|
|
32
|
+
try {
|
|
33
|
+
const stateDir = process.env.AGENTHUB_STATE_DIR;
|
|
34
|
+
const out = stateDir ? await factsHook(await Bun.stdin.text(), stateDir, process.env.AGENTHUB_PEER_ID ?? "claude") : undefined;
|
|
35
|
+
if (out) console.log(out);
|
|
36
|
+
} catch {
|
|
37
|
+
// a hook that fails must not get in the way of the tool call
|
|
38
|
+
}
|
|
39
|
+
process.exit(0);
|
|
40
|
+
}
|