pi-legion 3.1.0 → 4.0.0

package/CHANGELOG.md

@@ -11,6 +11,216 @@ summary so you can see **2.x vs 0.16** without leaving the tree.
 
 ## [Unreleased]
 
+## [4.0.0] — 2026-10-24
+
+A major line — but not only the pi-only collapse. Relative to 3.1.0 this release
+removes every non-`pi` agent kind and the archived Herdr backend behind it, along
+with the settings and error codes that only its transport read. It also carries
+everything else that landed on `master` while that removal was held back: a
+launch admission queue that queues instead of refusing, one liveness verdict
+shared by every display surface, a status panel built on pid / rss /
+context-token data, and ten measured performance fixes.
+
+### Breaking
+
+- **Kinds collapse to `pi`.** `AGENT_KINDS` is now `["pi"]`; the
+  `cursor` / `claude` / `codex` / `gemini` and other third-party CLI kinds are
+  gone, and with them the `cursorModel` / `nativeModelFor` / `planKindStart`
+  model translation and the "model dropped, runs on its own default" notice
+  (a `pi` child expresses every model, so the case no longer exists). `kind` is
+  still a legal frontmatter key, but `pi` is the only accepted value: a non-`pi`
+  value is ignored — one note on stderr per role (deduplicated, because the
+  catalog is re-read on several UI paths) naming the role and the original
+  value — and the role launches as `kind: pi`. The same applies to
+  `subagents.presets.<role>.kind` and `subagents.agentOverrides.<role>.kind`,
+  the three places a role's `kind` can come from. No legacy prefix aliases.
+- **Herdr backend removed.** `src/backends/legacy/` and `test/legacy/` are
+  deleted, together with the deprecated `pi-legion/api` re-exports that only
+  served them (`createHerdrClient`, `parseHerdrResponse`, `mapHerdrErrorCode`,
+  `readPaneDiagnostic`, `createCommandRunner`, `resolveHerdrBin`, `HerdrClient`
+  and their types). `pi-legion@0.16.x` stays the pin for the Herdr-era
+  execution model; the `spawnMode: rpc` / `inprocess` backends are unaffected —
+  those are two ways of running **Pi**, not third-party transports.
+- **Dead `herdr.*` settings removed.** `subagents.herdr.*` is no longer parsed
+  (a leftover block is silently ignored, as it has been ignored since 2.0.0),
+  the seven `herdr` constants are gone from `DEFAULTS`, and
+  `ErrorCodes.HERDR_UNAVAILABLE` / `ErrorCodes.PANE_BUSY` are removed.
+  `RunRecord.herdr` and `StatusEntry.herdrStatus` stay: they carry live
+  `{supervisor, node_id}` state, not backend selection.
+
+### Added
+
+- **Launch admission queue (#266).** A launch that exceeds a *concurrency* gate
+  (`maxChildrenPerNode`, `maxActive`, ledger `max_children_per_node` /
+  `max_active_nodes`) no longer fails with `BUDGET_EXCEEDED`. It is enqueued,
+  the caller gets `▶ scout-7 (scout) supervisor=inprocess queued(3)` and treats
+  the work as dispatched, and the child starts automatically when a slot frees.
+  Previously a 12-step fan-out against `maxChildrenPerNode: 8` ran 8 children and
+  reported 4 errors — and nothing ever retried them, because the parent was
+  already inside `wait` and the freed slot had no subscriber.
+  - New settings: `subagents.inprocess.maxQueuedLaunches` (default **32**) and
+    `subagents.inprocess.launchQueueTimeoutMs` (default **900000**). Both are
+    shared by the in-process and RPC paths — one scheduler, not two.
+  - Depth, spawn-per-session, memory, name-collision and config errors are still
+    refused synchronously, byte-identically to before: only gates that a slot
+    release can satisfy are queueable. `maxChildrenPerNode: 0` is a hard refusal
+    (0 means "no children", so waiting can never satisfy it).
+  - A queued launch reserves nothing: no ledger row, no child run directory, no
+    worktree, no session file. `timeout`/`overflow`/`stop` are reported by name;
+    `stop` dequeues without ever spawning, and a parent-turn abort flushes the
+    queue and says how many launches it dropped.
+  - `subagent({action:"status"})` shows `state=queued`, `collect`/`wait` block
+    until admission instead of reading the placeholder session path, and
+    `async: false` reports `↳ <name> waited Ns in the launch queue`.
+  - Two new process-only events (`launch_queued`, `launch_admitted`) and a
+    `legion-queue: N queued` status pill.
+
+
+- **One liveness verdict for every display surface (#liveness).** The list, the
+  text view and the railway chart each decided "is this agent still working?"
+  their own way, and the railway one compared two clocks that no single writer
+  owned: `eventOverrides` vetoed an event against `node.updatedAt`, but the
+  event row is stamped by this process's event bus while `updatedAt` is stamped
+  by the child's heartbeat — and only `isAlive` children are heartbeaten, so a
+  dead child's `updatedAt` freezes. The comparison ranked two clocks and the
+  losing side was the truthful one: a finished node was repainted `running`,
+  forever, because the frozen row never moved again. `src/extension/liveness.ts`
+  replaces all three with one `liveness(facts)` in TOTAL ORDER — `live` /
+  `dead` / `unknown` — with no clock, no fs and no timer of its own. Unknown or
+  unrecognized states now render `unknown` instead of being guessed into
+  `retired`, and `queued` is a first-class display state.
+  - Contract: `docs/contracts/display-liveness-v1.md` (frozen first, then
+    implemented; D1-a and D5-a record two amendments found during the work).
+
+- **A status panel on the `/legion` board (#241, #245).** A fifth pane carries
+  the `/legion status` table onto the full-screen viewer — name · role · state ·
+  supervisor · ctx · rss · turns — graded down by absolute column count
+  (72/52/36/20) so a narrow board keeps name · state instead of a column of
+  stubs. A portrait layout gets a zero-area rect and Tab skips it rather than
+  painting a fifth pane that cannot fit. Only an `rpc` child owns a process, so
+  only it shows rss; an `inprocess` child shows `—` rather than a share of the
+  host figure it does not own, because a fabricated number would get used.
+  - `pid`, `spawnMode`, `contextTokens` and `rss` are optional on `VizNode` and
+    projected conditionally: a missing mode renders `unknown`, never a guessed
+    `rpc`. No new `legion.db` column and no schema-version bump — pid is
+    host-scoped and never persisted, so a rebuild from a `ChildRecord` invents
+    none.
+  - Context tokens are absolute only. Session JSONL carries no context window,
+    so no percentage is computed and none is displayed.
+  - `/legion status` also stopped dropping `spawnMode` and now supplies `owner`.
+
+- **`/legion stop --force` (#179).** The TUI slash command can now hard-retire a
+  child, equivalent to the tool's `force: true`; `--force` / `-f` is listed in
+  the doctor help and the README. This closes the "optional follow-up" that the
+  docs had been carrying.
+
+- **`vexillatio` accepts `tasks[]` on its own.** `task` was required, so a
+  `{tasks: [...]}` call was rejected by the host's typebox pre-validation before
+  Legion ever saw it, and callers had to send a dummy `task` and retry. It is
+  optional now; a `tasks[]`-only launch is legal.
+
+### Changed
+
+- **WAL is now an enforced startup precondition (#236).** `openLegionDb` used to
+  discard the value returned by `PRAGMA journal_mode = WAL`. That PRAGMA is a
+  *query*: the returned mode is the ground truth, and SQLite is entitled to
+  answer with the mode it settled on rather than to fail — which is how a
+  connection ends up on a rollback journal while every reader of the code
+  believes WAL is live. The ledger's durability reasoning (single-writer
+  serialisation, readers that never block the writer, the shutdown checkpoint)
+  rests entirely on WAL, so the observed mode is now captured and verified: a
+  file database that does not report `wal` throws at startup, with the observed
+  mode in the message, instead of surfacing later as data loss. `:memory:` is
+  exempt deliberately, keyed on `PRAGMA database_list` reporting an empty
+  `file` — not on a database name, and not by accident.
+  - The M2-(g) concurrency acceptance now runs as `npm run test:wal-stress`:
+    60 supervisor processes against one ledger, 9004 attempts, 9004 commits, 0
+    busy, p50 1.03 ms / p95 1.99 ms / p99 4.92 ms.
+
+- **The worker role's turn budget is 45 minutes, not 30.** The worker does the
+  heaviest per-dispatch work — several files, its own verification run, and a
+  report — yet it shared centurion's budget while scout gets 10 minutes and
+  reviewer 15. Two dispatches in this repository were cut off mid-work, one
+  with nothing on disk and one that ran out while verifying. 45 rather than 60
+  keeps a genuinely stuck child from holding a concurrency slot for an hour.
+  Only `timeoutMs` changed.
+
+### Fixed
+
+- **A poison event row no longer freezes the cross-process event poller
+  (#233).** `node_abandoned` was missing from the event-type closed set, so a
+  snapshot row of that type could not be decoded; the 250 ms poller then threw
+  on the same row forever, its cursor never advanced, and no later event was
+  ever delivered. The closed set now contains `node_abandoned` (15 types), and
+  the poller advances past a single poison row instead of freezing: the row is
+  reported once through `onError`, the cursor steps over it, and the remaining
+  events in that batch are still delivered.
+
+- **A finished supervisor's pid is no longer read as its successor's (#250).**
+  `rss` probes re-read `/proc/<pid>` after the child had exited, and Linux
+  reuses pids; the panel could attribute a stranger's memory to a retired node.
+  The probe now closes the race it was silently losing.
+
+- **The log view stays inside its rect on narrow panes (#251).** `drawLog` wrote
+  past its own `maxX` when the pane was squeezed, which corrupted the frame
+  beside it.
+
+- **The live overlay no longer scans the whole tree per token (#261).** A
+  per-token `getTreeNodes()` call made a long child turn quadratic in the number
+  of nodes; the scan is gone and re-opening the node that is already open is
+  guarded.
+
+- **Repository subdirectories are no longer mistaken for child worktrees
+  (#237 P10).** The probe treated any git-visible path below the repo root as an
+  existing worktree, so a dispatch from `src/` could reuse the wrong directory.
+  All git probes are now bounded: `GIT_PROBE_TIMEOUT_MS = 5_000` and
+  `GIT_WORKTREE_TIMEOUT_MS = 120_000`, so a hung `git` can no longer wedge a
+  launch forever.
+
+- **`SubagentError` is exported explicitly again.** It had reached
+  `pi-legion/api` only as a side effect of the legacy client re-exporting it, so
+  removing the Herdr backend silently dropped it from the public surface. It is
+  a deliberate export now, pinned by a test.
+
+### Performance
+
+Ten fixes measured against the baseline in `docs/performance-2026-10-19.md`,
+plus the harness that measured them (#239). Numbers below are the before/after
+from each fix's own commit.
+
+- **`collect()` no longer waits out a fixed settle-quiet window (#232).** An
+  already-settled turn cost a flat ~2.5 s of dead time; that wait is dropped when
+  the turn is settled (2509 ms → 503 ms).
+- **The settle polls back off instead of re-parsing every 500 ms (#232).** The
+  transient-connection wait re-parsed the whole session file on every tick. It
+  now keeps the last parse, re-parses only when the artifact size moved, and
+  backs off 500 ms → 4 s. On an unchanged file in a 900 s window: 1800
+  iterations / 1804 parses → 228 iterations / 5 parses. The blip tolerance is
+  unchanged — sleeps are clamped to the remaining budget and the loop still ends
+  exactly on the deadline.
+- **The `/legion` log view reads the session incrementally (#234).** It seeks
+  past the cached byte offset and projects only the new JSONL lines, instead of
+  re-parsing from the top on every refresh.
+- **`attachUsageTotals` is one batched query (#235/#248).** It accounted for
+  ~72% of the cost of a UI snapshot; it is now a chunked `IN` query instead of a
+  per-node round trip.
+- **Heartbeats are one transaction (#236 P1).** N children meant N commits;
+  eight children now cost one (3.83 ms → 0.99 ms, 3.9×).
+- **Prepared statements are cached per connection (#236 P2).** `getNode` ×400
+  drops 15.7 ms → 4.8 ms.
+- **Session parsing projects its result instead of `structuredClone` (#237 P7).**
+- **The DB event source seeds its cursor at attach instead of replaying the log
+  (#238).** Attaching used to block ~1027 ms on replay.
+- **Widget events are coalesced into one snapshot and one paint per tick
+  (#238).** `legion-tools` emits N `mail_sent` events in one synchronous loop, so
+  every burst cost 2N scans; a burst now costs a constant 2 snapshots — 2575 ms
+  → 4.4 ms for 1000 events over a 120-node ledger. `applyTreeEvent` itself still
+  runs synchronously, per event, in arrival order: it is an incremental state
+  machine, not a render, so deferring or reordering it would change the state it
+  computes.
+- **The messages rate-limit `COUNT` is covered by an index (#237 P9).** 50k rows
+  go from a `SCAN` to ~6.6 µs.
+
 ## [3.1.0] — 2026-10-19
 
 ### Added
@@ -436,12 +646,15 @@ First **Pi Legion** line on headless Pi RPC. Relative to the final Herdr-era
 
 Last line that runs children in Herdr panes. Not developed further on this
 branch — pin `pi-legion@0.16.x` if you still need that model. See
-[Herdr transition](README.md#herdr-transition-and-role-migration) in the README.
+[Herdr transition](README.md#supported-kinds) in the README.
 
 Earlier 0.11–0.15 history (including the `pi-herdr-swarm` /
 `@zzjcool/pi-herdr-subagents` renames) lives in git tags and older Releases.
 
-[Unreleased]: https://github.com/zzjcool/pi-legion/compare/v3.0.2...HEAD
+[Unreleased]: https://github.com/zzjcool/pi-legion/compare/v4.0.0...HEAD
+[4.0.0]: https://github.com/zzjcool/pi-legion/compare/v3.1.0...v4.0.0
+[3.1.0]: https://github.com/zzjcool/pi-legion/compare/v3.0.3...v3.1.0
+[3.0.3]: https://github.com/zzjcool/pi-legion/compare/v3.0.2...v3.0.3
 [3.0.2]: https://github.com/zzjcool/pi-legion/compare/v3.0.1...v3.0.2
 [3.0.1]: https://github.com/zzjcool/pi-legion/compare/v3.0.0...v3.0.1
 [3.0.0]: https://github.com/zzjcool/pi-legion/compare/v2.0.4...v3.0.0

package/CONTRIBUTING.md

@@ -107,10 +107,10 @@ the top of that file). Do not fail CI on it.
 Prefer **TDD** for behaviour changes: a failing test in `test/unit` or
 `test/integration` first, then the production code. Integration tests should
 drive the orchestrator through `FakeSupervisor` (`test/helpers/fake-supervisor.ts`)
-so nesting / launch / collect stay covered without a live `pi`. The herdr-era
-helpers `createHerdrClient` / `createFakeRunner` remain as **deprecated legacy**
-exports (see `src/api.ts` and `test/legacy/`); use them only when touching the
-archived Herdr backend.
+so nesting / launch / collect stay covered without a live `pi`. The old
+herdr-era `createHerdrClient` / `createFakeRunner` helpers are **gone** as of
+v4.0.0, together with `src/backends/legacy/` and `test/legacy/`; everything
+runs as `kind: pi`, in-process or over `pi --mode rpc`.
 
 ## Mock LLM
 
@@ -161,19 +161,6 @@ Or all at once (RPC mainline **1–11** only):
 npm run test:docker
 ```
 
-**Legacy Herdr / Space / kind checks** live under `test/legacy/docker/` (need
-pi-legion **0.16.x** Herdr-era setup). They are **not** part of
-`npm run test:docker`:
-
-```bash
-bash test/legacy/docker/verify-12-workspace.sh  # Herdr Space pin (legacy)
-bash test/legacy/docker/verify-13-kinds.sh      # kind/model matrix; live CodeBuddy when mounted
-```
-
-`verify-13-kinds.sh` also bind-mounts host `codebuddy` / `cursor-agent` and
-(unless `LIVE_KIND_PROBES=0`) calls live CodeBuddy. The numbered 1–11 scripts
-under `test/docker/` stay offline/mock.
-
 Override paths with `HOST_PI_AGENT`, `DOCKER_PI_HOME`, `PI_DOCKER_IMAGE`.
 
 Each `inside-N-*.sh` is what actually runs **in** the container. Keep those

package/README.md

@@ -30,7 +30,7 @@ shared SQLite ledger records the whole tree.
 - [Team 配置](#team-配置)
 - [Parent/child context conventions](#parentchild-context-conventions)
 - [Delivery and resume](#delivery-and-resume)
-- [Herdr transition and role migration](#herdr-transition-and-role-migration)
+- [Supported kinds](#supported-kinds)
 - [Changelog](CHANGELOG.md)
 - [Development](#development)
 
@@ -84,9 +84,8 @@ trades process isolation for spawn latency.
 **Automatic fallback to `rpc`:** when `spawnMode` is omitted *and* the resolved
 spawn cannot be in-process, Legion launches it over `pi --mode rpc` instead.
 That happens when the role needs an OS worktree (`worktree: true` — only the
-`centurion` role may hold this; see Isolation below). A `kind !== "pi"` role is
-rejected earlier with `INVALID_PARAMS` (only `kind: pi` children are supported
-at all), so it never reaches this fallback.
+`centurion` role may hold this; see
+[Delivery and resume](#delivery-and-resume)).
 
 **Explicit is strict:** `spawnMode: "inprocess"` with `worktree: true` is
 refused with `INVALID_PARAMS` (message prefers `set worktree: false`) rather
@@ -167,16 +166,17 @@ controls run graceful `stop` (a stopped parent retires its own subtree on
 shutdown); process exit / session shutdown also retires:
 
 - **`/legion stop`** stops every live subagent; **`/legion stop <name>`** stops
-  one (`all` is an alias for no name). Names complete from the live children;
+  one (`all` is an alias for no name). Add **`--force`** / **`-f`** for hard
+  retire (same as tool `force: true`). Names complete from the live children;
   nested agents complete as `parent/name` and are reported as owned by their
   parent (stop the parent to retire them).
 - **`x`** on the selected live agent in the `/legion` view or its live
-  overlay asks `y` to confirm in place, then stops it.
+  overlay asks `y` to confirm in place, then stops it (graceful; use
+  `/legion stop --force <name>` for hard retire).
 - **Tool**: `subagent({ action: "stop", name })` is graceful; pass
   `force: true` for hard retire (SIGTERM → SIGKILL + descendant sweep per
   `docs/retire-hardening-contract.md`). Example:
   `subagent({ action: "stop", name: "worker-0", force: true })`.
-  TUI `/legion stop --force` is an optional follow-up (same `force: true`).
 
 **`wait` ≠ stop.** A `wait` timeout (or still-running result) does **not**
 retire children. The tool result includes live child names, elapsed ms, and
@@ -184,8 +184,8 @@ suggested decisions (`extend` / `steer_again` / `stop` / `force_stop`); the
 parent chooses the next tool call. See `docs/contracts/wait-not-stop.md`.
 
 While subagents are live, the railway header can show a dim
-`/legion stop [name|all]` hint (shortened on narrow terminals; idle gating /
-tree-strip coverage is a TUI follow-up).
+`/legion stop [--force] [name|all]` hint (shortened on narrow terminals; idle
+gating / tree-strip coverage is a TUI follow-up).
 
 ### Status (`/legion status`)
 
@@ -505,17 +505,13 @@ process ends, its child processes do not become detached workers; resuming the
 parent restores the squad, while already-settled children are not restarted.
 Keep the run records and session files while the tree may need to resume.
 
-## Herdr transition and role migration
+## Supported kinds
 
-Version **0.16.x is the final Herdr-era line**. Pi Legion 2.x is RPC-only:
-child execution uses headless `pi --mode rpc` processes, and the legacy backend
-is archived under `src/backends/legacy/`, not offered as a v2 runtime option.
-Pin to `pi-legion` 0.16.x if a workflow must stay on the former Herdr-based
-execution model. A short breaking-change summary lives in
-[CHANGELOG.md](CHANGELOG.md#200---2026-10-14).
-
-Pi Legion 2.x supports `kind: pi` only. Migrate a custom role that uses another
-CLI kind by changing its frontmatter, for example:
+Pi Legion runs `kind: pi` roles only. `kind` is still a legal frontmatter key,
+but `pi` is the only accepted value: a non-`pi` value is ignored (one note on
+stderr) and the role launches as `kind: pi`. The same holds for
+`subagents.presets.<role>.kind` and `subagents.agentOverrides.<role>.kind` —
+those three spots are the only places a role's `kind` can come from.
 
 ```yaml
 ---
@@ -524,8 +520,16 @@ kind: pi
 ---
 ```
 
-A non-Pi kind is rejected by v2. If that role must continue to run through
-another CLI, keep it on the 0.16.x line or replace it with a Pi role.
+Child execution is Pi either way: in-process by default, with an automatic
+`rpc` fallback when the resolved spawn needs an OS worktree (see
+[Spawn mode](#spawn-mode)).
+
+Version **0.16.x was the final Herdr-era line** — it launched children in Herdr
+panes instead of Pi processes. That backend was archived under
+`src/backends/legacy/` for 2.x and 3.x, and is gone in v4. Pin to
+`pi-legion@0.16.x` if a workflow must stay on that former execution model; a
+short breaking-change summary lives in
+[CHANGELOG.md](CHANGELOG.md#200---2026-10-14).
 
 ## Development
 

package/agents/worker.md

@@ -6,7 +6,7 @@ systemPromptMode: replace
 inheritProjectContext: true
 inheritSkills: false
 async: true
-timeoutMs: 1800000
+timeoutMs: 2700000
 allowNestedSubagents: true
 acceptance:
   level: verified