pi-legion 3.0.3 → 4.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +263 -2
- package/CONTRIBUTING.md +4 -17
- package/README.md +29 -22
- package/agents/worker.md +1 -1
- package/docs/contracts/display-liveness-v1.md +373 -0
- package/docs/contracts/drop-non-pi-kinds-v4.md +198 -0
- package/docs/contracts/esc-abort-v1.md +4 -4
- package/docs/contracts/esc-no-retire-v3.md +2 -2
- package/docs/contracts/launch-admission-queue-v1.md +839 -0
- package/docs/contracts/status-panel-v1.md +174 -0
- package/docs/inprocess-supervisor.md +37 -4
- package/docs/legion-v2-contract.md +18 -2
- package/docs/performance-2026-10-19.md +199 -0
- package/docs/spawn-mode-visibility-v1.md +206 -0
- package/docs/vexillatio-contract.md +3 -1
- package/index.ts +633 -84
- package/package.json +4 -2
- package/skills/pi-legion/SKILL.md +2 -3
- package/src/agents/agents.ts +17 -10
- package/src/agents/overrides.ts +0 -1
- package/src/agents/presets.ts +14 -40
- package/src/agents/settings.ts +26 -77
- package/src/agents/step-model.ts +7 -22
- package/src/api.ts +1 -35
- package/src/extension/event-bus.ts +80 -8
- package/src/extension/legion-commands.ts +43 -17
- package/src/extension/legion-doctor.ts +1 -1
- package/src/extension/legion-stop.ts +39 -13
- package/src/extension/legion-tools.ts +21 -4
- package/src/extension/liveness.ts +200 -0
- package/src/extension/notify.ts +6 -6
- package/src/extension/playbook.ts +6 -6
- package/src/extension/recycle.ts +6 -2
- package/src/extension/runtime.ts +422 -18
- package/src/extension/slash.ts +0 -21
- package/src/extension/spawn-render.ts +183 -0
- package/src/extension/summary.ts +1 -1
- package/src/legion/db.ts +165 -1
- package/src/legion/events.ts +81 -16
- package/src/legion/nodes.ts +13 -2
- package/src/legion/rebuild.ts +7 -1
- package/src/legion/usage.ts +58 -4
- package/src/runs/orchestrator.ts +1165 -62
- package/src/runs/worktree.ts +190 -21
- package/src/shared/ignored-kind.ts +57 -0
- package/src/shared/progress.ts +11 -0
- package/src/shared/session.ts +199 -19
- package/src/shared/types.ts +50 -66
- package/src/supervisor/admission-queue.ts +516 -0
- package/src/supervisor/inprocess-supervisor.ts +108 -33
- package/src/supervisor/proc-memory.ts +67 -0
- package/src/supervisor/routing-supervisor.ts +58 -1
- package/src/tui/live-overlay.ts +14 -0
- package/src/tui/session-messages-file.ts +100 -0
- package/src/tui/status.ts +103 -21
- package/src/tui/tree-view.ts +36 -3
- package/src/tui/viz/layout.ts +54 -3
- package/src/tui/viz/model.ts +67 -14
- package/src/tui/viz/palette.ts +11 -0
- package/src/tui/viz/panels.ts +222 -8
- package/src/tui/viz/railway.ts +28 -25
- package/src/tui/viz/view.ts +110 -23
- package/src/backends/legacy/client.ts +0 -638
- package/src/backends/legacy/cursor-chat.ts +0 -348
- package/src/backends/legacy/runner.ts +0 -101
- package/src/backends/legacy/types.ts +0 -138
- package/src/runs/kind.ts +0 -260
|
@@ -0,0 +1,174 @@
|
|
|
1
|
+
# Status panel v1 — `/legion` 大盘的 status 面板 + status 字段缺口补齐
|
|
2
|
+
|
|
3
|
+
> 冻结契约。合并权人裁定,worker 不得自行解释矛盾,遇矛盾向父会话报告。
|
|
4
|
+
> 基线:`3071dee` (3.1.0),`npm run typecheck` = 0、`npm test` = 0。
|
|
5
|
+
|
|
6
|
+
## 0. 目标
|
|
7
|
+
|
|
8
|
+
1. 在 `/legion` 全屏大盘(`src/tui/viz/`)新增 `status` 面板,把目前只存在于聊天文本里的 `/legion status` 数据搬到大盘上。
|
|
9
|
+
2. 补齐 `/legion status` 的已知缺口:`supervisor=unknown`、`owner` 缺失、无内存、无 context tokens、`usage` 未格式化。
|
|
10
|
+
|
|
11
|
+
## 1. 不可违反的既有约束
|
|
12
|
+
|
|
13
|
+
| 约束 | 锚点 |
|
|
14
|
+
| --- | --- |
|
|
15
|
+
| **禁止给 `legion.db` 加列、禁止升 `LEGION_DB_USER_VERSION`** | `docs/spawn-mode-visibility-v1.md:143`、`test/unit/legion-schema-sentinel.test.ts` |
|
|
16
|
+
| 运行时零外部依赖(Node stdlib only) | `CONTRIBUTING.md:215` |
|
|
17
|
+
| 每行 `visibleWidth === w`(严格相等,非 ≤) | `test/unit/legion-viz.test.ts:417-434,628,911,1464,1630` |
|
|
18
|
+
| 平台降级不得抛错:非 Linux 返回空值 | 先例 `src/supervisor/proc-tree.ts:44` |
|
|
19
|
+
| `spawnMode` 缺失必须渲染 `unknown`,**绝不猜 `rpc`** | `src/extension/runtime.ts:84-89`、`test/unit/runtime.test.ts:1490-1501` |
|
|
20
|
+
|
|
21
|
+
## 2. 设计决策
|
|
22
|
+
|
|
23
|
+
### D1 — 新增第 5 个 pane,但 portrait 不渲染
|
|
24
|
+
|
|
25
|
+
- `wide` / `stacked` 新增 `status` rect;`portrait` 的 `layout.status` 返回**零面积 rect** `{x,y,w:0,h:0}`。
|
|
26
|
+
- 理由:portrait `46×24` 实测 `galaxy=5 chart=10 timing=3 log=3`,`bodyH=21`,4 个 pane 最小高之和 `3+10+3+3=19`,仅剩 2 行 —— 物理上装不下第 5 个 pane(最小 3 行)。`timing`/`log` 在 `h=3` 时内框 `h=1`,`drawTiming`/`drawLog` 本就提前 return,**portrait 极矮时这两个 pane 已是空壳**。
|
|
27
|
+
- `frame()` 在 `h<3` 已早退且不画任何东西,所以零面积 rect 天然安全,无需特判绘制。
|
|
28
|
+
- **Tab 必须跳过零面积 pane**:新增 `availablePanes(layout)`,返回 `h>0` 的 pane 列表;`handleViewKey` 增加可选第 4 参 `panes`,**缺省 = 全部 pane**(保证既有测试不变)。
|
|
29
|
+
- 被否决:(a) 全布局都加 → portrait 溢出 footer;(b) status 复用 timing 做可切换视图 → 隐藏态发现性差,且 `drawTiming` 宽度算术已很紧;(c) 把 status 塞进 log → 语义冲突。
|
|
30
|
+
|
|
31
|
+
### D2 — inprocess 内存口径:只显示 host rss,禁止按子 agent 摊派
|
|
32
|
+
|
|
33
|
+
- `rpc` 子 agent:有独立 pid → 显示 per-child `rss`。
|
|
34
|
+
- `inprocess` 子 agent:**没有独立进程**(`inprocess-supervisor.ts:751-754` 只返回 name+sessionFile)。面板与 status 文本中 **inprocess 行的 rss 列留空/`—`**,另起一行显示 `host rss <N> MB · shared by N inprocess`。
|
|
35
|
+
- **严禁**按 token 比例把 host rss 摊到子 agent 头上 —— 那是编造数据,用户会拿它做决策。
|
|
36
|
+
- 理由:诚实性优先于信息密度。
|
|
37
|
+
|
|
38
|
+
### D3 — pid 穿透(不加 DB 列)
|
|
39
|
+
|
|
40
|
+
链路:`ChildHandle.pid` → `Handle.pid` → `TrackedJobInput.pid` → `StatusEntry.pid` → `VizNode.pid`。
|
|
41
|
+
|
|
42
|
+
- `src/shared/types.ts:482` `Handle` **当前无 pid**,必须加 `pid?: number`。
|
|
43
|
+
- `src/runs/orchestrator.ts:800` 附近(`spawnMode: mode` 同处)写入 `...(result.pid ? { pid: result.pid } : {})`。
|
|
44
|
+
- `TrackedJobInput`(`src/extension/runtime.ts:73-107`)加 `pid?: number`;`index.ts:2675` 的 `runtime.track({...})` 补传。
|
|
45
|
+
- pid 不落盘是既有事实,**不新增持久化**——viz 只在当前宿主进程可见,跨进程显示为未知。
|
|
46
|
+
|
|
47
|
+
### D4 — context tokens:扩 `ParsedSession`,不做百分比
|
|
48
|
+
|
|
49
|
+
- `accumulateUsage`(`src/shared/session.ts:124-136`)在累加的同时,把**最后一条 assistant message** 的 `input + cacheRead + cacheWrite` 记到 `ParsedSession.lastContextTokens`。
|
|
50
|
+
- `LiveProgress`(`src/shared/progress.ts:9-28`)加 `contextTokens?: number`;`progressFromSession` 投影;`mergeProgress` 用「later wins」语义。
|
|
51
|
+
- **不显示百分比**:`contextWindow` 在 JSONL 中不存在,全仓零命中。要百分比就得引模型注册表,超出本契约范围。显示绝对值 `ctx 24.3K tok`。
|
|
52
|
+
- 被否决:直读 session JSONL 最后一条 assistant usage → 与既有 `parseSessionFile` 重复解析,且绕过 `accumulateUsage` 单一事实源。
|
|
53
|
+
|
|
54
|
+
### D5 — 修 `childStatus` 漏传(现存 bug)
|
|
55
|
+
|
|
56
|
+
`index.ts:926-946` 的 `childStatus` 回调补 `spawnMode` 与 `owner`:
|
|
57
|
+
|
|
58
|
+
- `spawnMode: child.spawnMode`(run.json 已有值,`src/shared/types.ts:365-373`)
|
|
59
|
+
- `owner:` 用既有 `runOwnerStatus(runDir)`(`src/runs/owner.ts:60` 返回 `live|dead|unknown`)
|
|
60
|
+
- 正确范例:`index.ts:1417-1428`、`index.ts:2887-2899`(这两处**已传** spawnMode)。
|
|
61
|
+
- 这是纯 bug fix,与新增面板解耦,可独立验收。
|
|
62
|
+
|
|
63
|
+
### D6 — 所有新字段一律 optional
|
|
64
|
+
|
|
65
|
+
`VizNode` 新增字段必须 optional,否则 3 处字面量构造编译失败:`test/unit/legion-viz.test.ts:18-20`、`test/unit/legion-stop.test.ts:92`、`src/tui/viz/panels.ts:103-105`。
|
|
66
|
+
|
|
67
|
+
投影沿用仓库惯例 `...(cond ? {k:v} : {})`(`model.ts:72-79`),**不写 `undefined` 字面量**。
|
|
68
|
+
|
|
69
|
+
### D7 — status 面板字段集与宽度降级档位
|
|
70
|
+
|
|
71
|
+
表头 1 行 + 可选页脚 1 行;列按**绝对列数**分层(沿用 `drawTiming` 惯例):
|
|
72
|
+
|
|
73
|
+
| 内框宽 | 列 |
|
|
74
|
+
| --- | --- |
|
|
75
|
+
| `w >= 72` | `name · role · state · supervisor · ctx · rss · turns` |
|
|
76
|
+
| `w >= 52` | `name · state · supervisor · ctx` |
|
|
77
|
+
| `w >= 36` | `name · state · ctx` |
|
|
78
|
+
| `w >= 20` | `name · state` |
|
|
79
|
+
| `w < 20` | 不画(return) |
|
|
80
|
+
|
|
81
|
+
- 守卫:`if (r.w < 20 || r.h < 2) return;`(`drawTiming` 用 `w<8,h<2`;本面板列多,下限提到 20)
|
|
82
|
+
- `footerH = rows.length > 0 && r.h >= 4 ? 1 : 0`;`rowsAvail = max(0, r.h - 1 - footerH)`
|
|
83
|
+
- **所有 `cv.text` 必须带 `maxX` 或先 `ellipsize`** —— 禁止复制 `drawLog`(`panels.ts:530-532`)的越界 bug。
|
|
84
|
+
- `rss` 列:rpc → `152M`;inprocess → `—`;未知 → `—`。**不得写大写单位**(`legion-viz.test.ts:927` 的字符类只含小写 a-z),用 `M` 会红 → 改用 `m`;或把该断言一并更新(见 §5)。
|
|
85
|
+
|
|
86
|
+
### D8 — Tab 顺序:status 追加到末尾
|
|
87
|
+
|
|
88
|
+
`PANES = ["galaxy","chart","timing","log","status"]`(`src/tui/viz/view.ts:36`)。
|
|
89
|
+
|
|
90
|
+
- 追加在末尾使 `legion-viz.test.ts:442`(chart)、`:444`(timing)、`:446`(log)**全部继续通过**,只需新增一条 status 断言。
|
|
91
|
+
- 被否决:插入 timing 与 log 之间 → 直接打红 3 条既有断言,且 `:1588-1593` 再红一条。
|
|
92
|
+
|
|
93
|
+
## 3. 文件所有权(严格边界,禁止越界)
|
|
94
|
+
|
|
95
|
+
### Worker A — 数据管道 + status 文本(先做)
|
|
96
|
+
|
|
97
|
+
| 文件 | 性质 |
|
|
98
|
+
| --- | --- |
|
|
99
|
+
| `src/supervisor/proc-memory.ts` | **新增**:`readProcRss(pid)`,读 `/proc/<pid>/status` 的 `VmRSS`/`VmHWM`,非 Linux / 读失败 / pid 已退 → `undefined`,绝不抛 |
|
|
100
|
+
| `src/shared/types.ts` | 改:`Handle.pid?`、`ParsedSession.lastContextTokens?` |
|
|
101
|
+
| `src/shared/session.ts` | 改:`accumulateUsage` 记录最后一条 assistant 的 context tokens |
|
|
102
|
+
| `src/shared/progress.ts` | 改:`LiveProgress.contextTokens?` + `progressFromSession` + `mergeProgress` |
|
|
103
|
+
| `src/runs/orchestrator.ts` | 改:`Handle` 写入 `pid` |
|
|
104
|
+
| `src/extension/runtime.ts` | 改:`TrackedJobInput.pid?`、`statusEntryFromJob`/`statusEntryFromNode` 写 `spawnMode/pid/contextTokens/rss`、`renderLiveChildStatus` 增 memory/ctx 行 |
|
|
105
|
+
| `src/tui/status.ts` | 改:`StatusEntry` 加 `spawnMode?/pid?/contextTokens?/rss?`(**不动 `formatWidgetLines`**) |
|
|
106
|
+
| `index.ts` | 改:`childStatus` 补 `spawnMode`/`owner`;`runtime.track` 补 `pid` |
|
|
107
|
+
| `test/unit/proc-memory.test.ts` | **新增** |
|
|
108
|
+
| `test/unit/legion-status.test.ts` | 改:新增字段断言(`:110-147`、`:192-254` 现有断言**不得放宽**,只追加) |
|
|
109
|
+
| `test/unit/runtime.test.ts` | 改:`renderLiveChildStatus` 新行断言 |
|
|
110
|
+
|
|
111
|
+
**Worker A 严禁触碰**:`src/tui/viz/**`、`src/tui/tree-view.ts`、`src/tui/live-overlay.ts`、`docs/**`。
|
|
112
|
+
|
|
113
|
+
### Worker B — viz 面板(等 A 合入后)
|
|
114
|
+
|
|
115
|
+
| 文件 | 性质 |
|
|
116
|
+
| --- | --- |
|
|
117
|
+
| `src/tui/viz/layout.ts` | 改:`LegionLayout.status: Rect` + 三批分支 + `availablePanes` |
|
|
118
|
+
| `src/tui/viz/panels.ts` | 改:`Pane` 联合加 `"status"`;**新增 `drawStatus`**;**不修改** 既有 4 个 draw* |
|
|
119
|
+
| `src/tui/viz/view.ts` | 改:`PANES` 末尾追加;`frame`+`drawStatus` 调用;`clampList` 支持 `"status"`;`handleViewKey` 第 4 参 `panes` |
|
|
120
|
+
| `src/tui/viz/model.ts` | 改:`VizNode` 加 optional 字段 + `nodesFromEntries` 投影 |
|
|
121
|
+
| `test/unit/legion-viz.test.ts` | 改:新增 status 面板断言(§5 清单) |
|
|
122
|
+
|
|
123
|
+
**Worker B 严禁触碰**:`src/extension/**`、`src/shared/**`、`src/runs/**`、`src/tui/status.ts`、`index.ts`。
|
|
124
|
+
|
|
125
|
+
## 4. 分阶段与验收
|
|
126
|
+
|
|
127
|
+
| 阶段 | 内容 | 验收 |
|
|
128
|
+
| --- | --- | --- |
|
|
129
|
+
| P1 | Worker A 全量 | `npm run typecheck` && `npm test` |
|
|
130
|
+
| P2 | Worker B 全量(依赖 P1 已合入) | `npm run typecheck` && `npm test` |
|
|
131
|
+
|
|
132
|
+
每个阶段必须 **typecheck 0 / test 0**。首个非零退出即判 rejected。
|
|
133
|
+
|
|
134
|
+
## 5. 必须同步更新的测试(逐条)
|
|
135
|
+
|
|
136
|
+
| 测试 | 原因 | 期望 |
|
|
137
|
+
| --- | --- | --- |
|
|
138
|
+
| `legion-viz.test.ts:417-434` | 新增 status 面板标题 | 追加 `plain.some(l => l.includes("status"))`;**不得放宽** `visibleWidth === w` |
|
|
139
|
+
| `legion-viz.test.ts:437-459` | Tab 顺序末尾新增 | 在 `:446` 后追加 `handleViewKey("\t")` → `assert.equal(state.focus, "status")`;既有 3 条保留 |
|
|
140
|
+
| `legion-viz.test.ts:160-183` | layout 新字段 | `for (const r of [galaxy,chart,timing,log,status])` 加入界内断言;**portrait 跳过 status**(零面积) |
|
|
141
|
+
| `legion-viz.test.ts:872-901` | 几何锁 | portrait 段保持四段相接断言不变;wide 段新增「右列三段相接」断言 |
|
|
142
|
+
| `legion-viz.test.ts:927` | header 字符类 | 若 header 未变则不动;**status 面板内容不写入 header** |
|
|
143
|
+
| `legion-viz.test.ts` 新增块 | `drawStatus` 单测 | 覆盖 D7 四档宽度 + `w<20` 退化 + inprocess `rss` 为 `—` |
|
|
144
|
+
| `legion-status.test.ts:110-147` | 新增 memory/ctx 行 | 既有首行/`alive`/`execution snapshot` 断言保留,追加新行断言 |
|
|
145
|
+
| `legion-status.test.ts:192-254` | `formatTree` 未变 | **不动**(`formatTree` 不在本契约改动范围) |
|
|
146
|
+
|
|
147
|
+
## 6. 风险与裁决
|
|
148
|
+
|
|
149
|
+
| 风险 | 裁决 |
|
|
150
|
+
| --- | --- |
|
|
151
|
+
| `drawLog` 内框 `w<=14` 写出 rect 右边界(`panels.ts:530-532`) | **顺手修**:给该 `cv.text` 补 `maxX: r.x + r.w`。属 P2 范围,单独 commit,附回归断言。理由:Worker B 正在改同一文件,且新面板极易复制此模式。**勘误**:`Canvas.text` 的 `maxX` 是**排他上界**(`canvas.ts:39` `if (col >= limit) break`),而 `frame()` 返回的是内框、其右边框列即 `r.x + r.w`,所以「保住最后一个合法内格且不碰边框」的值是 `r.x + r.w`(与 `frame()` / `drawStatus` 一致)。本文早期写的 `r.x + r.w - 1` 会少吃一格,已按 #251 评审改为 `r.x + r.w`。 |
|
|
152
|
+
| view 层 `layout.timing.h - 3` 与 draw 层 `innerH-1-footerH` off-by-one(`view.ts:90`) | **本契约不修**。改动会影响 `timingVisible` 语义与既有断言,独立提 issue。新面板**必须**用精确公式,不照抄 `-3`。 |
|
|
153
|
+
| portrait 零面积 status 被 Tab 跳过 | 由 `availablePanes` 保证;`handleViewKey` 缺省参数保证既有测试不变 |
|
|
154
|
+
| pid 跨进程不可见 | 接受。viz 显示 `—`,不猜。 |
|
|
155
|
+
| `readProcRss` 的 pid 复用 race | **实现为「不打印」**:`!alive` 时直接不读 `/proc`,返回 `undefined`。v1 无 last-known 缓存设施(`memoryLine` 无状态、`TrackedJob` 无 rss 字段),新增缓存需额外的失效规则,比不打印更易说谎。`(last)` 分支**明确不在 v1 范围**。读取必须 try/catch。 |
|
|
156
|
+
|
|
157
|
+
## 7. 契约修订点
|
|
158
|
+
|
|
159
|
+
- 本文件即新增契约,无需改动 `docs/legion-v2-contract.md` §10.1(三层 UI 定义不变,status 面板属第 2 层内部枚举新增)。
|
|
160
|
+
- `docs/tui-viewer-contract.md:202` 的禁改清单**不包含** `src/tui/viz/**`,Worker B 无需先修订该文档。
|
|
161
|
+
- `README.md` 的「Three-layer visualization」段落可在 P2 后追加一句 status 面板说明(可选,非门禁)。
|
|
162
|
+
|
|
163
|
+
## 8. 契约勘误(P1 实施中记录)
|
|
164
|
+
|
|
165
|
+
1. **§5 必改测试表漏列 `test/unit/progress.test.ts`**。D4 的 `progressFromSession` 投影让该文件两处完整对象 `deepEqual` 机械性必红。实际处理:加 `contextTokens: 10` 两行,属**加强断言**(新字段被钉死),零删除。
|
|
166
|
+
2. **`runtime.test.ts` 既有 D2 测试与 §6 冲突**。该测试原本用 `alive: false` + pid 断言「有 memory 行」,与 pid 复用防护直接矛盾。实际处理:把前置条件改为 `alive: true`,断言文本/正则/非 Linux 分支全部原样保留,强度不变。
|
|
167
|
+
3. **`(last)` 展示未实现**(见 §6 修订)。
|
|
168
|
+
|
|
169
|
+
## 9. 修订历史
|
|
170
|
+
|
|
171
|
+
| 日期 | 版本 | 说明 |
|
|
172
|
+
| --- | --- | --- |
|
|
173
|
+
| 2026-10-19 | v1 | 初版。父会话直接裁定(planner 卡死于「声称写文档但从不落盘」,已止损)。 |
|
|
174
|
+
| 2026-10-19 | v1.1 | P1 交付后补:pid 复用裁决改为「不打印」;记录 3 条勘误。 |
|
|
@@ -11,7 +11,7 @@ with `INVALID_PARAMS` — the fallback only covers the implicit default.
|
|
|
11
11
|
| Mode | Good for | Avoid when |
|
|
12
12
|
|---|---|---|
|
|
13
13
|
| `inprocess` (default; auto-fallback below) | Short, trusted fan-out; lower spawn latency; simpler approval / UI fan-in | Need OS worktree, separate credentials, crash isolation |
|
|
14
|
-
| `rpc` (fallback: worktree
|
|
14
|
+
| `rpc` (fallback: worktree; or explicit `spawnMode: rpc`) | Long runs, untrusted tasks, need crash / env / cwd isolation, **worktree** | Cold-start cost dominates |
|
|
15
15
|
|
|
16
16
|
There is no single global default in the config sense: omitting `spawnMode`
|
|
17
17
|
means "inprocess if this role can run in-process, otherwise rpc". Roles that
|
|
@@ -62,10 +62,46 @@ worktree: false # explicit inprocess + worktree: true is still refused
|
|
|
62
62
|
| Concurrent alive | `legion.maxActiveNodes` | `inprocess.maxActive` (default **30**; **8** when the block is present without `maxActive`) |
|
|
63
63
|
| Children / node | `legion.maxChildrenPerNode` | `inprocess.maxChildrenPerNode` (default **8**) |
|
|
64
64
|
| Wait timeout | agent `timeoutMs` / wait default | `inprocess.waitTimeoutMs` (no default) |
|
|
65
|
+
| Queue depth | `inprocess.maxQueuedLaunches` (shared by both paths) | `inprocess.maxQueuedLaunches` (default **32**) |
|
|
66
|
+
| Queue wait | `inprocess.launchQueueTimeoutMs` (shared by both paths) | `inprocess.launchQueueTimeoutMs` (default **900000** = 15 min) |
|
|
65
67
|
|
|
66
68
|
Never fall back across columns: in-process limits are read from
|
|
67
69
|
`subagents.inprocess.*` only, never from the `legion.*` keys.
|
|
68
70
|
|
|
71
|
+
### Launch admission queue (#266)
|
|
72
|
+
|
|
73
|
+
A launch that hits a **queueable** gate waits in an in-memory queue instead of
|
|
74
|
+
throwing `BUDGET_EXCEEDED`, and is admitted automatically when a slot frees. The
|
|
75
|
+
caller gets `▶ scout-7 (scout) supervisor=inprocess queued(3)` back immediately
|
|
76
|
+
and treats the work as dispatched.
|
|
77
|
+
|
|
78
|
+
| Gate | Queued? | Why |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| `inprocess.maxChildrenPerNode` | ✅ | pure concurrency: the slot always frees |
|
|
81
|
+
| `inprocess.maxActive` | ✅ | same |
|
|
82
|
+
| `legion.maxChildrenPerNode` (`max_children_per_node`) | ✅ | same, RPC path |
|
|
83
|
+
| `legion.maxActiveNodes` (`max_active_nodes`) | ✅ | same, RPC path |
|
|
84
|
+
| `maxDepth` / `maxSpawnsPerSession` / ledger `max_depth` | ❌ | waiting cannot satisfy them; a queue would disguise a permanent failure as a 15-minute delay |
|
|
85
|
+
| `maxChildrenPerNode: 0` | ❌ | 0 means "no children at all" — impossible to satisfy |
|
|
86
|
+
| `parent_not_found`, memory gate, `NAME_TAKEN`, unknown/disabled role, `inprocess`+`worktree`, model scope | ❌ | not a concurrency gate |
|
|
87
|
+
|
|
88
|
+
The two queue bounds are **shared** by both backends on purpose: one scheduler,
|
|
89
|
+
not one per spawn mode. Overflow is an immediate hard refusal
|
|
90
|
+
(`launch queue overflow (32/32)`), a timeout is reported as
|
|
91
|
+
`launch queue timed out after Ns ('<name>')`, and **neither ever creates a
|
|
92
|
+
ledger row, a run directory, a worktree or a session file** — a queued launch has
|
|
93
|
+
reserved nothing yet.
|
|
94
|
+
|
|
95
|
+
Admission is event-driven (a child reaching `retired`/`exited` frees its slot)
|
|
96
|
+
with one `unref`'d fallback recheck, because the ledger can also be freed by
|
|
97
|
+
another process. An `idle` child (turn ended, not yet collected) **still holds
|
|
98
|
+
its slot**, exactly as before.
|
|
99
|
+
|
|
100
|
+
Known limit: the queue is in memory, so a process restart drops it. The leftover
|
|
101
|
+
`queued` records in `run.json` are inert — `collect`/`steer` on one says
|
|
102
|
+
`never started: the launch queue was lost …` instead of `unknown child`, and
|
|
103
|
+
`stop` cleans it up.
|
|
104
|
+
|
|
69
105
|
Omitting an `inprocess.*` key is **not** unlimited. The documented defaults
|
|
70
106
|
mirror `legion.*`, so a deployment with no `subagents.inprocess` block at all
|
|
71
107
|
still runs under `maxDepth: 4` / `maxChildrenPerNode: 8` / `maxActive: 30`.
|
|
@@ -187,9 +223,6 @@ a decode rate.
|
|
|
187
223
|
|
|
188
224
|
## Refusals
|
|
189
225
|
|
|
190
|
-
- `kind !== "pi"` → `INVALID_PARAMS` regardless of `spawnMode`: only `pi` roles
|
|
191
|
-
are supported, so a non-`pi` role is refused up front and never reaches the
|
|
192
|
-
worktree fallback
|
|
193
226
|
- `worktree: true` + explicit `inprocess` → `INVALID_PARAMS` (an omitted
|
|
194
227
|
`spawnMode` falls back to `rpc` instead)
|
|
195
228
|
- A non-centurion requesting `worktree: true` is **not** a refusal: it is
|
|
@@ -101,7 +101,9 @@ interface LegionSupervisor {
|
|
|
101
101
|
| `pane close` 杀进程 | stdin close 优雅退出;超时 SIGTERM;仍超时 SIGKILL;后代进程按进程组清场(REV-16) |
|
|
102
102
|
| `run.json` herdr 字段 | 载体改为 `{supervisor: "rpc", node_id}`,`paneId/tabId` 删除 |
|
|
103
103
|
|
|
104
|
-
### 3.3 判定链不变:成败仍以 session jsonl 为唯一权威(F26–F33)。collect 复用 `
|
|
104
|
+
### 3.3 判定链不变:成败仍以 session jsonl 为唯一权威(F26–F33)。collect 复用 `parseSessionFile` + `deriveOutcome`(+ `waitSettled` 的 settle 事件)收割,收割后顺手写 `node_settled` 事件 + 更新 nodes 行。
|
|
105
|
+
|
|
106
|
+
> 修订(#232):原文此处写作「collect 复用 `waitForQuiet` + ...」。`waitForQuiet`(2.5s 安静窗口 stat 轮询)已由 #232 / PR #244 删除——`waitSettled` 本身就是事件驱动的,`settled` 之后再对 append-only 文件轮询 size 得不到任何新信息。判定链的输入与权威性未变。
|
|
105
107
|
|
|
106
108
|
### 3.4 每节点身份:`ownerToken` 语义改为「树根 session id + node id」;多会话同 repo 隔离靠 node id 命名空间(替代 `parentPaneId` 过滤)。
|
|
107
109
|
|
|
@@ -116,6 +118,7 @@ interface LegionSupervisor {
|
|
|
116
118
|
- 根会话**首次 launch 时**创建:`<rootRunDir>/legion.db`(`.pi-subagents/runs/<runId>/` 下)
|
|
117
119
|
- 绝对路径经 env **`PI_LEGION_DB`** 随 `lineageEnv()` 逐层下发——任何深度的 supervisor 直连同一个库,不经父节点转发(RPC spawn 的 env 由宿主直接控制,F35 类丢失坑不复存在)
|
|
118
120
|
- 连接约定:每 supervisor 进程一连接;`PRAGMA journal_mode=WAL; PRAGMA busy_timeout=5000;`;写入均为短事务 prepared statement
|
|
121
|
+
- **WAL 是前置条件,不是请求**:`PRAGMA journal_mode=WAL` 是**查询**,返回值才是实际生效的 journal mode,它可能在**不报错**的情况下降级(网络/FUSE 挂载无 shared-memory 支持、只读介质、争用锁)。`openLegionDb()` 因此**读取返回值并在其为文件库时校验**:非 `wal` 直接抛错,在启动期炸掉,而不是带着「已开 WAL」的假设继续跑、把问题拖成后发性数据事故。`:memory:` 是唯一的刻意豁免——SQLite 对内存库不实现 WAL、永远报 `memory`,而该库本来就不跨进程存活(由 `PRAGMA database_list` 的空 `file` 判定,不是靠路径字符串匹配)。测试见 `test/unit/legion-db.test.ts`。
|
|
119
122
|
|
|
120
123
|
### 4.2 Schema(`PRAGMA user_version = 1`)
|
|
121
124
|
|
|
@@ -177,6 +180,7 @@ CREATE TABLE messages ( -- 邮件(§8)
|
|
|
177
180
|
created_at INTEGER NOT NULL
|
|
178
181
|
);
|
|
179
182
|
CREATE INDEX idx_messages_inbox ON messages(to_node, delivered, id);
|
|
183
|
+
CREATE INDEX idx_messages_rate ON messages(from_node, created_at); -- from_node 维度的限速 COUNT(§8,REV-19)
|
|
180
184
|
```
|
|
181
185
|
|
|
182
186
|
### 4.3 事件类型枚举(闭集,新增需修订契约)
|
|
@@ -187,6 +191,14 @@ CREATE INDEX idx_messages_inbox ON messages(to_node, delivered, id);
|
|
|
187
191
|
|
|
188
192
|
### 4.4 与 run.json 的关系(D20):run.json 保留为 per-run 权威记录(resume 凭据、原子写、写锁、坏文件隔离原样不动)。legion.db 是树级索引+账本,**可重建**:`legion-rebuild` 命令遍历 nodes 各 run_id 的 run.json + session jsonl 重建 nodes/events(best effort,验收项 M2-h)。
|
|
189
193
|
|
|
194
|
+
**「可重建」的确切边界(实测,勿过度引用)**:重建**不是**无损复原,以下三类事实在当前 `rebuildLegionDb()` 实现中**无来源、不可恢复**(「无来源」= 该函数不读取/不映射这些字段,不等于磁盘上没有;见每条的括注)——
|
|
195
|
+
|
|
196
|
+
1. **节点字段**:`phase` / `phase_since` / `rework_count` / `contract_path` / `task_summary` 一律回落为 `null` / `0`。根因是 `NodeSource`(`src/legion/rebuild.ts:37`)根本没有这几个字段,`run.json` 也不携带 phase(`RunRecord` 无该字段),而 `insertNode` 对缺省值填 `?? null` / `?? 0`。
|
|
197
|
+
2. **events 塌缩为合成事件**:重建每个节点只产出一条 `node_launched`(或按终态 `node_settled` / `node_failed` / `node_retired`),payload 为 `{ recovered: true, fromSession }`。`phase_change` / `budget_refused` / `mail_sent` / `tree_finalized` / `orphan_detected` / `phase_timeout` / `node_blocked` / `node_resumed` 等**全部无幸存者**——`ChildRecord` 只留终态,而**当前**的 rebuild 不解析 session jsonl 里的工具调用流水(§4.3 把树级语义事件与工具级明细分在两处)。这是对**当前实现**的陈述,不是「jsonl 里没有」:`legion_mail` 的 subject/body(`legion-tools.ts:412-413`)与 `legion_phase` 的 `to`(:458)确实以工具调用记录的形式存在于**发送方**会话 jsonl 中,未来说的 rebuild 完全可以(且应该)把它们解析回来。
|
|
198
|
+
3. **messages 永不重建**:`INSERT INTO messages` 只存在于 `mail.ts`(`sendMail` :209 与 bounce 通知 :294);`rebuild.ts:79` 只有条件 `DELETE`(`preserveMessages === false` 时)而**无对应 INSERT**。`ChildRecord` 不含 subject/body,`mail_sent` 的 payload 只有 `{messageId, toNode}`。故 **`messages` 表一旦丢失即不可再生**(同样限于当前实现:正文可从发送方 jsonl 的工具调用参数中恢复;但**未投递**的消息若发送方会话文件也已轮转/删除,正文即永久消失)。
|
|
199
|
+
|
|
200
|
+
因此 §4.4 的「可重建」只能支撑**索引/聚合层可重算**这一层结论,**不能**用来论证「丢写无代价」——phase 状态机断点、预算拒绝、邮件正文都不在重建范围内。任何以「反正能重建」为由放松持久性的论证都必须先与本节对账。实测方法:构造含 5 类真实事件的账本 + 一封带 subject/body 的邮件 + 一个填满 phase 字段的 centurion 节点,跑真实 `rebuildLegionDb()`(`test/unit/legion-rebuild.test.ts` 之外的一次性探针),行数自 6→2、上述字段全清空、邮件行因默认 `preserveMessages` 而**幸存但不可再生**。
|
|
201
|
+
|
|
190
202
|
### 4.5 写入权责:节点行由**其父的 supervisor** 写(launch 插入、settled/failed 更新、心跳刷新);phase 字段仅由状态机引擎(§7)迁移;usage 由各 supervisor 轮询 upsert(默认 30s,仅 running 节点)。LLM 永不直连库——唯一入口是扩展实现的工具(§8.4),child-guard 禁止 bash 触碰 legion.db(§13)。
|
|
191
203
|
|
|
192
204
|
## 5. 节点标识与命名
|
|
@@ -363,6 +375,7 @@ interface LegionEventBus {
|
|
|
363
375
|
| `maxDepth`(全树深度) | 4 | 硬顶 = `MAX_NESTED_PATH_ENTRIES`(血缘路径在 4 处饱和;settings/env 超 4 截断——告警在 M1-integration 的 launch 路径接入,M0 阶段截断静默可接受)。每进程生效上限 C:根 `C = min(env 运营者覆盖, settings.legion.maxDepth, 4)`;深度 d、上限 C 的父 launch 角色 R:先过闸门 `d+1 > C` → BUDGET_EXCEEDED;再经 `PI_SUBAGENT_MAX_DEPTH` 下发 `C子 = min(C, d + R, 4)`(R 缺省=不收紧;非法值按叶子 R=1 处理;语义 = 子树代数含自身,位置无关,只收紧不放宽)。不变量:C子 ≤ C父,且每进程 depth ≤ maxDepth |
|
|
364
376
|
| `maxChildrenPerNode`(单节点扇出) | 8 | 父 supervisor 查自己 **live** 子节点数(`starting`/`running`/`blocked`;`settled`/`failed`/`retired` 不占槽,session 可保留 resume) |
|
|
365
377
|
| `maxActiveNodes`(全树并发 running+starting) | 30 | **全树真闸门**(替代只当 hint 的 maxConcurrentAgents):launch 前一条 SQL,超限 BUDGET_EXCEEDED + `budget_refused` 事件 |
|
|
378
|
+
| `maxQueuedLaunches`(准入队列上限,#266) | 32 | 只有扇出/并发两类闸门可排队(`max_children_per_node`、`max_active_nodes`,以及 in-process 的 `maxChildrenPerNode`/`maxActive`);队列满 → 立即硬拒 `launch queue overflow (N/32)`,不留账本行。`maxDepth`/`max_active_nodes` 之外的深度与 spawn 预算**永不排队**。键名 `subagents.inprocess.maxQueuedLaunches`,两个后端共用同一个值 |
|
|
366
379
|
|
|
367
380
|
### 11.2 心跳与孤儿:postman 循环每次刷新直接子节点 `updated_at`。根 supervisor 每 60s 扫描:非终态 + `updated_at` 超 90s = `orphan_detected` 事件 + mail 通知 CEO。恢复 = resume 链(CEO 裁决,auto-resume 默认关)。孤儿判定可判定化:**非终态 ∧ 无活跃进程 ∧ 心跳超时**。
|
|
368
381
|
|
|
@@ -417,7 +430,7 @@ interface LegionEventBus {
|
|
|
417
430
|
### M2 军团树(组织层:db + centurion + 状态机 + 邮件)
|
|
418
431
|
|
|
419
432
|
- **归属**:新增 `src/legion/{db,nodes,events,usage,mail,state-machine,budget,rebuild}.ts`、`agents/centurion.md`、`src/extension/legion-tools.ts`;修改 `orchestrator.ts`(launch 落库/树命名/预算查库/子树 worktree——经 REV-17 收为 centurion 专属,非 centurion 请求 warns 一次后降级)、`src/runs/worktree.ts`(subtree 模式)、`src/runs/args.ts`(env 增 `PI_LEGION_DB`/`PI_LEGION_NODE_ID`)、`src/agents/agents.ts`、`index.ts`(队级 resume、collect 落库)
|
|
420
|
-
- **验收**:(a) 三层树 e2e:CEO→centurion→workers×2 全阶段走通(契约落盘→并行实施→review verdict→verify 实跑→done 收队 + squash MR);(b) rework 循环:review major→回 implementing→复审通过;(c) rework≥2 → failed 上抛;(d) kill centurion 进程 → 孤儿报警 → resume 恢复整队、settled 组员不复活、运行中组员上下文延续;(e) `maxActiveNodes` 超限拒派 + 事件;(f) 邮件:同级/父子/父←子通、跨子树拒发并提示路由、retired 收件人唤醒投递、终态收件人退信;(g) WAL 压测:60 模拟 supervisor 并发写 5 分钟,busy 超时率 <0.1
|
|
433
|
+
- **验收**:(a) 三层树 e2e:CEO→centurion→workers×2 全阶段走通(契约落盘→并行实施→review verdict→verify 实跑→done 收队 + squash MR);(b) rework 循环:review major→回 implementing→复审通过;(c) rework≥2 → failed 上抛;(d) kill centurion 进程 → 孤儿报警 → resume 恢复整队、settled 组员不复活、运行中组员上下文延续;(e) `maxActiveNodes` 超限拒派 + 事件;(f) 邮件:同级/父子/父←子通、跨子树拒发并提示路由、retired 收件人唤醒投递、终态收件人退信;(g) WAL 压测:60 模拟 supervisor 并发写 5 分钟,busy 超时率 <0.1%(**实测 2026-10-22 通过**:60 个 `fork` 子进程、各持一连接、共享一库,postman 节奏每 ~2s 一短事务,300s 内 9004 次尝试 / 9004 提交 / 0 次 busy = 0.000%;成功事务 p50 1.03ms / p95 1.99ms / p99 4.92ms。复跑:`npm run test:wal-stress`;CI 快速守卫 `test/unit/wal-concurrency.test.ts` 只钉住 busy_timeout 生效、WAL/超时前置条件与零 busy,**不证明** 5 分钟跨进程结论);(h) `legion-rebuild` 从 run.json+jsonl 重建 nodes/events;(i) 非 pi kind 明确拒绝
|
|
421
434
|
|
|
422
435
|
### M3 可视化
|
|
423
436
|
|
|
@@ -462,3 +475,6 @@ interface LegionEventBus {
|
|
|
462
475
|
| 2026-10-11 | REV-15 | §6.1 centurion 角色卡的 `team: be-value` 改为可选省略:be-value 是示例值,仓库/用户 settings 均未定义该队,不覆盖时子进程收警告并退化全 roster。team 字段语义 = 可选的默认队绑定(须真实存在),省略即全 roster。触发:reviewer-6 对 PR #13 的范围外观察 |
|
|
463
476
|
| 2026-10-12 | REV-16 | §3.2 retire 链加固:SIGTERM 超时后升级 SIGKILL(新 option `killTimeoutMs`,默认 1s),并在 retire 时枚举子 pi 的后代进程(/proc 祖先链),按进程组 SIGTERM→SIGKILL 清场(排除 supervisor 自身进程组与子 pi 自身 pid)。触发:2026-10-12 孤儿 `node --test` 进程泄露事故(两个 PPID=1 进程各烧 4h20m CPU,SIGTERM 无效,人工 SIGKILL 收场)。子契约:`docs/retire-hardening-contract.md`(FROZEN-1) |
|
|
464
477
|
| 2026-10-19 | REV-17 | §9 worktree 收为 centurion 专属(权限变更,按 §15-4 走修订):① 判定以角色名 `centurion` 为准,centurion 卡一律拿子树 worktree、嵌套 centurion 拿 nested worktree;② 其余角色一律不拿——删除「`acceptance.role: writer` 自动补 `worktree: true`」的隐式默认,`agents/worker.md` 从 `worktree: true` 改为显式 `worktree: false`(`agents/centurion.md` 的 `true` 与 `agents/prototype.md` 的 `false` 不变);③ 违规一律**降级为 `false` + `console.warn` 一次**,**不抛错**(抛错会阻断 launch);`subagent({ worktree: true })` 显式传参同样降级。不改 `resolveLaunchWorktree` 的优先级链(不变式 I5),不新增事件类型,不改 `spawnMode` 仲裁逻辑。原因:issue #214(worktree 收为 centurion 专属,与 §9 已冻结的「组员一律 `worktree: false`」对齐;实现向契约收敛,隔离边界继续由「每棵子树」承载) |
|
|
478
|
+
| 2026-10-19 | REV-18 | v4 收紧为仅 kind: pi;AGENT_KINDS 收敛为 ["pi"];非 pi 值忽略并按 pi 启动。字段不删(值域收窄);`spawnMode` 的 `rpc`/`inprocess` 不受影响。契约:`docs/contracts/drop-non-pi-kinds-v4.md`(D1,v4.0.0) |
|
|
479
|
+
| 2026-10-21 | REV-19 | §4.2 新增索引 `idx_messages_rate ON messages(from_node, created_at)`(schema 变更,按 §15-4 走修订)。原因:`sendMail` 的限速 COUNT(`src/legion/mail.ts`,`WHERE from_node = ? AND created_at > ?`)在**写事务内**执行且 `messages` 无任何保留策略(`DELETE FROM messages` 仅见于 `rebuild.ts` 且只在 `preserveMessages === false`),实测 50k 行时由 `SCAN messages` 变为覆盖索引 `SEARCH`,成本随运行时长单调劣化。零行为变更:COUNT 结果不变,仅查询计划改变。**不改 `user_version`**(仍为 1),因此**既有 legion.db 不会获得该索引**——`migrate()` 在 `user_version === 1` 时提前返回,`SCHEMA_V1` 只在版本 0 执行。**且 `legion-rebuild` 不能修复它**:`rebuildLegionDb()` 拿的是已打开的 connection,事务内只发 `DELETE FROM usage/events/nodes`(以及 `preserveMessages === false` 时的 `DELETE FROM messages`),**没有 DROP/CREATE TABLE,也不重跑 `SCHEMA_V1`**;行删光后索引集合原封不动。实测:旧账本跑完 `rebuildLegionDb()` 与 close+reopen 后均仍为 `["idx_messages_inbox"]`,只有**删掉 legion.db 文件后重新打开**(走版本 0 建库)才会得到 `idx_messages_rate`。故补救方式**仅限重建账本文件本身**(legion.db 是可丢弃的树级索引,§4.4:从 run.json + session jsonl 重新生成),不是跑 rebuild 命令。接受此代价:每棵树各自新建 legion.db,长跑账本才需要人工删库重启。代价度量(50k 行表,200 发送者):限速 COUNT 由 `SCAN` 的 ~3327 µs/call 降至 ~6.6 µs/call(**~500x**,该量级稳定复现);写侧索引维护在已开事务内的单行 INSERT 上是**零点几 µs/行量级(低双位数百分比)**,方向确定但**具体倍数机器相关且不可移植**(同一代码在不同环境实测 +32% 到 +47% 不等,勿当常数引用);一次**含 COMMIT 的**单行写入被 fsync 主导(数百 µs、高方差),索引那零点几 µs 在该噪声内不可分辨。净效应每封信仍是 ms 量级的节省,且随行数增长而扩大。不新增「打开时补建索引」后门(会让契约漂移)。不引入 `messages` 保留策略(另立产品决策:归属、是否借用 `eventRetentionDays`、bounce 风暴是否单独限流)。触发:issue #237 P9 |
|
|
480
|
+
| 2026-10-22 | REV-20 | ① §4.1 显式写下 WAL **前置条件**:`PRAGMA journal_mode=WAL` 是查询、返回值才是实际生效模式,可在不报错的情况下降级,故 `openLegionDb()` 校验返回值、文件库非 `wal` 即抛错(`:memory:` 刻意豁免,由 `PRAGMA database_list` 的空 `file` 判定)。② §4.4 补「可重建」的**实测边界**:phase/phase_since/rework_count/contract_path/task_summary 全部不可恢复(`NodeSource` 无这些字段);events 塌缩为每节点一条合成事件(phase_change/mail_sent/budget_refused/tree_finalized 等零幸存);`messages` 无任何重建路径(INSERT 仅存在于 `mail.ts`,`rebuild.ts` 只有条件 DELETE)。按 §15-4 走修订的理由:②是**订正一条会误导持久性推理的既有断言**(原文「可重建」被当成放松丢写的前提),①是把一句被隐含的实现假设升为契约明文。非 schema/事件类型/权限变更,`user_version` 不变;**未触碰** `synchronous` / `journal_mode` 的取值 / `temp_store` / `cache_size` / `mmap_size`(`synchronous=NORMAL` 提案已评估**否决**:生产写速率 0.53–5.47 tx/s,`FULL` 占空比 0.027–0.273%,而 `NORMAL` 会把掉电窗口从单事务放大到上个 checkpoint 以来的全部事务、且长活读者下无界)。触发:issue #236 **注(对 REV-19 措辞的澄清)**:REV-19 原文写「legion.db 是可丢弃的树级索引」,该断言的语境**仅是**「旧账本拿不到新索引、补救只能删库重开」这一 schema 索引场景,并非「账本可随意丢弃」。本修订不追改 REV-19 的历史正文(修订日志按 §15-1 追加、保留当时的实际决定),而在此澄清其适用范围:可丢弃的**只是索引层**,`messages` 与 phase 状态断点不在重建覆盖内(见 §4.4)。 |
|
|
@@ -0,0 +1,199 @@
|
|
|
1
|
+
# Performance measurements (2026-10-19)
|
|
2
|
+
|
|
3
|
+
Measured on `v3.1.0` (`3071dee`), Node v24.13.1, one container. **Absolute
|
|
4
|
+
milliseconds drift with the machine; the ratios and the call chains are the
|
|
5
|
+
findings.** Reproduce with `npm run test:perf`.
|
|
6
|
+
|
|
7
|
+
These numbers back the perf issues (#236–#241). The harness lives in
|
|
8
|
+
`test/perf/` so a fix can turn each one around. Regression tests that pin an
|
|
9
|
+
open defect say so in their name and in a comment next to the assertion.
|
|
10
|
+
|
|
11
|
+
## Tier 1 — user-visible latency and correctness
|
|
12
|
+
|
|
13
|
+
### P1 · `collect()` burns ~2.5s per child in a fixed sleep
|
|
14
|
+
|
|
15
|
+
`Orchestrator.collect()` receives `wait === "settled"` from the supervisor (the
|
|
16
|
+
child's turn ended), then still awaits `waitForQuiet`, which polls
|
|
17
|
+
`statSync(sessionFile)` until the size has been stable for
|
|
18
|
+
`DEFAULTS.settleQuietMs` (2500ms).
|
|
19
|
+
|
|
20
|
+
Measured on the real `Orchestrator` + `FakeSupervisor` + a real session JSONL,
|
|
21
|
+
changing only the quiet window:
|
|
22
|
+
|
|
23
|
+
```text
|
|
24
|
+
collect(), production settings 2509 ms
|
|
25
|
+
collect(), settleQuietMs = 0 503 ms
|
|
26
|
+
=> ~2007 ms of dead time per successful collect
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Guarding nothing: when `collect()` enters that branch the session's last
|
|
30
|
+
assistant message is already terminal **and on disk** (`stopReason: "stop"`,
|
|
31
|
+
`lastTurnMissing: false`). For an append-only file, a terminal message cannot
|
|
32
|
+
grow afterwards.
|
|
33
|
+
|
|
34
|
+
Root cause: `LegionSupervisor.onEvent` exists (`src/supervisor/types.ts`) and
|
|
35
|
+
both backends already publish settle events, but `src/runs/orchestrator.ts`
|
|
36
|
+
never subscribes — every post-`waitSettled` wait degrades to stat polling. The
|
|
37
|
+
same root cause drives `waitOutTransientConnection` (worst case 1800 full
|
|
38
|
+
`parseSessionFile` calls, ~500ms `POLL_INTERVAL_MS` cadence) and
|
|
39
|
+
`collectArtifactGrowing`.
|
|
40
|
+
|
|
41
|
+
> **Status after #232** (kept for the measurement record; the numbers above are
|
|
42
|
+
> the pre-fix baseline): `waitForQuiet` is gone (PR #244). The two remaining poll
|
|
43
|
+
> loops keep their channel but had their *cost* cut:
|
|
44
|
+
> `waitOutTransientConnection` re-parses only when the artifact size changed and
|
|
45
|
+
> backs off 500ms → 4s (default 900s deadline: 1800 full parses → 1 parse, 228
|
|
46
|
+
> cheap polls); `collectArtifactGrowing` samples the grace window at 50ms so it
|
|
47
|
+
> returns as soon as the artifact grows instead of always one full 500ms later.
|
|
48
|
+
> Fixing the *channel* (subscribing `onEvent`) stays open — see #232.
|
|
49
|
+
> Pinned by `test/unit/collect-polling.test.ts`.
|
|
50
|
+
|
|
51
|
+
Test: `test/perf/collect-latency.test.ts` (virtual clock: the injected
|
|
52
|
+
`now`/`sleep` deps measure the window without sleeping).
|
|
53
|
+
|
|
54
|
+
### P2 · one `node_abandoned` row permanently stalls the event bus
|
|
55
|
+
|
|
56
|
+
`src/runs/hygiene.ts` writes `type = 'node_abandoned'`, but
|
|
57
|
+
`LEGION_EVENT_TYPES` does not contain it and `mapEvent` throws on an unknown
|
|
58
|
+
type. `createDbEventSource` records the error and leaves the cursor
|
|
59
|
+
unadvanced, so every 250ms poll re-reads the same poisoned batch and throws again:
|
|
60
|
+
|
|
61
|
+
```text
|
|
62
|
+
valid ledger: delivered=3 cursor=3 errors=0
|
|
63
|
+
after one poison row: delivered=0 cursor=0 errors=4
|
|
64
|
+
repeat poll: delivered=0 cursor=0 errors=4
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
The cross-process view is dead from that point on, and the poll spins at 4Hz.
|
|
68
|
+
Reachable whenever hygiene marks a stale `starting/running/blocked` node and
|
|
69
|
+
any process later attaches to that ledger.
|
|
70
|
+
|
|
71
|
+
Test: `test/perf/event-bus.test.ts`, the `REGRESSION (open defect)` case.
|
|
72
|
+
|
|
73
|
+
## Tier 2 — costs that scale with load
|
|
74
|
+
|
|
75
|
+
### P3 · live-overlay re-reads the whole session file on every append
|
|
76
|
+
|
|
77
|
+
`readSessionDelta` compares `statSync().size` against `cursor.fileSize`, then —
|
|
78
|
+
on any change — does `readFileSync(sessionFile, "utf8")` **from byte 0** and
|
|
79
|
+
re-parses every line. The byte offset is already stored in the cursor; it is
|
|
80
|
+
used only as a change flag, never as a read offset.
|
|
81
|
+
|
|
82
|
+
```text
|
|
83
|
+
4.9MB session, poll with no append: 31.9 ms (full re-read)
|
|
84
|
+
4.9MB session, tail-read prototype: 0.05 ms (stat only) => ~600x
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Cost grows with session length while the trigger rate grows with token rate, so
|
|
88
|
+
the viewer gets slower the longer the turn runs.
|
|
89
|
+
|
|
90
|
+
### P4 · `attachUsageTotals` is an N+1 (72% of the UI snapshot)
|
|
91
|
+
|
|
92
|
+
`src/legion/usage.ts` calls `getUsage` once per node inside a `map`. Measured at
|
|
93
|
+
400 nodes: `getSubtree` 2.62ms + `attachUsageTotals` 6.90ms = 9.64ms per
|
|
94
|
+
snapshot. A single `WHERE node_id IN (...)` batch measured **8.95ms -> 0.90ms**
|
|
95
|
+
(9.9x) with byte-identical output.
|
|
96
|
+
|
|
97
|
+
Runs twice per second from the 500ms refresh, plus once per bus event
|
|
98
|
+
(`src/extension/runtime.ts` subscribes to every event type except `text_delta`).
|
|
99
|
+
|
|
100
|
+
Test: `test/perf/ledger.test.ts` (asserts the N+1 exceeds the CTE).
|
|
101
|
+
|
|
102
|
+
### P5 · heartbeats pay one fsync per child
|
|
103
|
+
|
|
104
|
+
`refreshHeartbeats` loops `updateNode` per child; each is an implicit
|
|
105
|
+
transaction under the default `synchronous=FULL` (only `busy_timeout`,
|
|
106
|
+
`foreign_keys`, `journal_mode` are set in `src/legion/db.ts`).
|
|
107
|
+
|
|
108
|
+
```text
|
|
109
|
+
8 children, per-child autocommit 3.83 ms (479 us/child)
|
|
110
|
+
8 children, one transaction 0.99 ms (124 us/child) => 3.9x
|
|
111
|
+
60 children 31.6 ms -> 8.2 ms
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
### P6 · no statement cache: every `prepare()` recompiles
|
|
115
|
+
|
|
116
|
+
45 `prepare()` call sites; the data layer exposes only `exec/prepare/close`.
|
|
117
|
+
|
|
118
|
+
```text
|
|
119
|
+
prepare-per-call 30.1 us/op
|
|
120
|
+
reused statement 9.2 us/op
|
|
121
|
+
getNode x400 15.7 ms -> 4.8 ms (3.2x)
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
> **Status after #236 P2** (kept for the measurement record; the numbers above
|
|
125
|
+
> are the pre-fix baseline): `prepare()` is now served from a bounded
|
|
126
|
+
> per-connection LRU (cap 64) installed *inside* the data layer, so all 46 call
|
|
127
|
+
> sites reuse compiled statements without being touched. Re-measured on the
|
|
128
|
+
> same harness: prepare-per-call 30.96/28.43/31.75 → 8.86/9.61/11.89 us/op,
|
|
129
|
+
> `getNode` x400 17.36/17.34/16.99 → 4.64/4.63/6.49 ms (3.6x). The `getSubtree`
|
|
130
|
+
> snapshot barely moves (0.64 → 0.47 ms) because it was already few statements.
|
|
131
|
+
> A schema-mutating `exec()` (`ALTER`/`CREATE`/`DROP`/`REINDEX`/`VACUUM`)
|
|
132
|
+
> invalidates the cache, since a cached `SELECT *` keeps the column set it
|
|
133
|
+
> compiled against. Pinned by `test/unit/legion-statement-cache.test.ts`
|
|
134
|
+
> (asserts statement identity, compile counts and invalidation — not timing).
|
|
135
|
+
> `synchronous` / `temp_store` / `cache_size` / `mmap_size` stay untouched: P3
|
|
136
|
+
> is a maintainer口径 decision, and the recon here measured those at near-zero
|
|
137
|
+
> benefit (0.33 MB db, 85 pages; default `cache_size` already holds 512 pages).
|
|
138
|
+
|
|
139
|
+
### P7 · `structuredClone` spends the incremental-parse win
|
|
140
|
+
|
|
141
|
+
`parseSessionFile` keeps a dev/ino/offset fingerprint cache (#36) but then
|
|
142
|
+
returns `structuredClone(state.parsed)`:
|
|
143
|
+
|
|
144
|
+
```text
|
|
145
|
+
6.9MB session: cold 73.3ms, warm 3.24ms
|
|
146
|
+
structuredClone(parsed) 3.95ms (~106% of the warm call)
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
Called once per job and once per node per 500ms refresh. Note the honest
|
|
150
|
+
bound: a **warm** pass over 60 distinct sessions is only 12.3ms (~2% of the
|
|
151
|
+
tick). The cliff is the 64-entry LRU cap — past that every tick is cold
|
|
152
|
+
(measured 312ms for the same pass).
|
|
153
|
+
|
|
154
|
+
### P8 · event replay: the cursor starts at 0
|
|
155
|
+
|
|
156
|
+
`createDbEventSource` defaults `cursor = 0`, so attaching to a ledger with
|
|
157
|
+
history replays it from the oldest row, and each replayed event triggers a
|
|
158
|
+
snapshot: 300 events x 3.42ms = **1027ms** of blocking work on attach. Seeding
|
|
159
|
+
from `MAX(id)` makes it one query.
|
|
160
|
+
|
|
161
|
+
## Tier 3 — smaller, still measured
|
|
162
|
+
|
|
163
|
+
### P9 · `messages` rate-limit is a full table SCAN
|
|
164
|
+
|
|
165
|
+
`SELECT COUNT(*) FROM messages WHERE from_node = ? AND created_at > ?` has no
|
|
166
|
+
covering index. At 50,000 rows: **5.04ms** per `sendMail`, `EXPLAIN QUERY PLAN`
|
|
167
|
+
says `SCAN messages`. The table also has no retention policy.
|
|
168
|
+
|
|
169
|
+
> Measurement note: a first attempt reported 0.04ms because the default
|
|
170
|
+
> `ratePer5Min` limit kept the table at zero rows. Re-measured by inserting rows
|
|
171
|
+
> directly — the SCAN is real.
|
|
172
|
+
|
|
173
|
+
### P10 · six synchronous git processes per first launch
|
|
174
|
+
|
|
175
|
+
`orchestrator.ts` calls `isInsideChildWorktree(this.cwd)` twice in the same
|
|
176
|
+
branch (lines 608 and 615) with the same argument:
|
|
177
|
+
|
|
178
|
+
```text
|
|
179
|
+
isInsideChildWorktree x1 9.4 ms (2 execFileSync)
|
|
180
|
+
isInsideChildWorktree x2 16.0 ms <- 4 processes, 2 of them redundant
|
|
181
|
+
isGitRepo x1 3.6 ms
|
|
182
|
+
=> 6 synchronous git processes per first centurion launch
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
All `execFileSync`, so they block the parent event loop; provision adds one
|
|
186
|
+
`bash -lc` (login shell, default 600s timeout).
|
|
187
|
+
|
|
188
|
+
## Deliberately not measured / open decisions
|
|
189
|
+
|
|
190
|
+
- `PRAGMA synchronous = NORMAL`: would cut every autocommit write a further
|
|
191
|
+
4–5x. WAL-safe against corruption but can lose the last transactions on power
|
|
192
|
+
loss. Needs an explicit decision; the contract only fixes WAL + `busy_timeout`.
|
|
193
|
+
- The contract asks for a 60-supervisor concurrent-WAL stress run
|
|
194
|
+
(`docs/legion-v2-contract.md`, "busy timeout rate < 0.1%"). **Implemented
|
|
195
|
+
2026-10-22** as `npm run test:wal-stress` (`test/stress/wal-stress.ts` +
|
|
196
|
+
`wal-worker.ts`): 60 forked processes x 1 connection each x postman cadence
|
|
197
|
+
(1 txn / 2 s) x 300 s -> 9004 attempts, 0 busy (0.000%), p50 1.03 ms /
|
|
198
|
+
p95 1.99 ms / p99 4.92 ms. That measurement does not revisit the
|
|
199
|
+
`synchronous` question above: it ran on the shipped `FULL`.
|