@mu-crew/mu 3.13.1 → 3.13.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/extension/mu-pi.js +1 -1
- package/docs/VOCABULARY.md +13 -0
- package/docs/specs/2026-10-21-fast-and-slow-gates.md +229 -0
- package/package.json +1 -1
- package/skills/mu/SKILL.md +8 -5
- package/skills/mu/recipes/adversarial-review.md +14 -10
- package/skills/mu/recipes/brief.md +7 -2
- package/skills/mu/recipes/orchestrator-loop.md +31 -15
- package/skills/mu/recipes/plan.md +6 -2
- package/skills/mu/recipes/remote-workers.md +12 -8
- package/skills/mu/recipes/review-panel.md +4 -3
- package/skills/mu/recipes/test-health.md +92 -0
- package/skills/mu/recipes/waves.md +3 -3
- package/skills/mu/recipes/worker.md +5 -1
package/dist/extension/mu-pi.js
CHANGED
|
@@ -1362,7 +1362,7 @@ function registerRefuteNudge(pi, run = defaultRunner(), nested = notNested, hold
|
|
|
1362
1362
|
}
|
|
1363
1363
|
|
|
1364
1364
|
// extension/mu-pi.ts
|
|
1365
|
-
var EXT_VERSION = true ? "3.13.
|
|
1365
|
+
var EXT_VERSION = true ? "3.13.3" : void 0;
|
|
1366
1366
|
var FRESH_COMMAND = "mu-fresh";
|
|
1367
1367
|
var FRESH_TIMEOUT_MS = 3e4;
|
|
1368
1368
|
function commandName(name) {
|
package/docs/VOCABULARY.md
CHANGED
|
@@ -83,6 +83,19 @@ agent names, and file paths are in [reference/naming.md](reference/naming.md).
|
|
|
83
83
|
| **anonymous claim** | A `--self` claim by an actor that is not a worker. `owner` stays NULL; `ops.actor` records who. | "unowned claim" |
|
|
84
84
|
| **release** | Clear `tasks.owner`. | "unclaim", "unassign" |
|
|
85
85
|
|
|
86
|
+
### Recipe gates
|
|
87
|
+
|
|
88
|
+
| Term | Meaning | Don't use |
|
|
89
|
+
| --- | --- | --- |
|
|
90
|
+
| **fast gate** | Repo-defined iteration checks: format, lint, unit tests and cheap integration tests. Cost is measured, not guaranteed seconds. | "gate command" (alone) |
|
|
91
|
+
| **slow gate** | The remaining merge checks: real-environment e2e, release and packaging checks. | "gate command" (alone) |
|
|
92
|
+
| **gate budget** | The slow gate's target wall time; crossing it prioritises test upkeep. | "timeout" |
|
|
93
|
+
| **gate record** | A note identifying commit/tree, exact command/scope, result, duration, environment/toolchain and skips; reuse needs equivalent conditions. | "tests pass" (alone) |
|
|
94
|
+
| **test upkeep** | Ordinary DAG work to profile, speed up, move down, merge or remove tests while preserving coverage; see [test-health](../skills/mu/recipes/test-health.md). | "coverage reduction" |
|
|
95
|
+
| **test level** | Unit < integration < e2e; use the lowest level that catches the guarded failure. | |
|
|
96
|
+
| **review gate** | A review task blocked by a work task, deciding whether its commits count. | "gate command" |
|
|
97
|
+
| **merge batch** | Accepted commits combined into one pinned tree for full merge verification, without a crew-wide barrier. | "wave" (when only verification is batched) |
|
|
98
|
+
|
|
86
99
|
### Workspaces and housekeeping
|
|
87
100
|
|
|
88
101
|
| Term | Meaning | Don't use |
|
|
@@ -0,0 +1,229 @@
|
|
|
1
|
+
# Fast and slow gates, and test upkeep as routine work
|
|
2
|
+
|
|
3
|
+
Status: implemented in mu `10d758c`. Swaywardlock pilot commits `90c3f1c`
|
|
4
|
+
and `36273ff` are merged locally: gate instrumentation and one targeted wait
|
|
5
|
+
improvement. Full-suite upkeep and the slow-gate budget remain open.
|
|
6
|
+
|
|
7
|
+
**Measurement correction (implementation check):** the initial figures below
|
|
8
|
+
came from exploratory regex scripts, not an execution classifier. The
|
|
9
|
+
museum tool's drill-down showed that many counted "full gate runs" were
|
|
10
|
+
commands writing commit messages or task notes that merely mentioned the
|
|
11
|
+
gate. The claimed six full runs per task and daily run totals are not
|
|
12
|
+
verified and must not be used as a baseline. Command duration also includes
|
|
13
|
+
other work in a compound command, and background work appears in later
|
|
14
|
+
polls. Preserve the initial investigation below for provenance, but establish
|
|
15
|
+
a fresh, inspected baseline with `museum-search calls --tool bash` and step
|
|
16
|
+
timing before judging the pilot. Unit-test runtime and source-size figures
|
|
17
|
+
were read directly from logs and git history.
|
|
18
|
+
Date: 2026-10-21
|
|
19
|
+
|
|
20
|
+
## Problem
|
|
21
|
+
|
|
22
|
+
Crews slow down as a project grows, and the gate is where the time goes.
|
|
23
|
+
swaywardlock (repo `swayward-wm/swaywardlock`, started 2026-10-19) is the
|
|
24
|
+
clearest case. From 225 museum sessions over its first three days:
|
|
25
|
+
|
|
26
|
+
| | Oct 19 | Oct 20 | Oct 21 |
|
|
27
|
+
|---|---|---|---|
|
|
28
|
+
| `contrib/e2e-sway` lines | 273 → 980 | → 2,437 | → 3,587 |
|
|
29
|
+
| unit tests (`#[test]`) | 0 → 69 | → 165 | → 224 |
|
|
30
|
+
| test + harness share of the day's diff | 12% | 43% | 36% |
|
|
31
|
+
| median full `contrib/gate` run | 1–3 min | 2.3 → 5.9 min | 5.5 → 7.9 min |
|
|
32
|
+
| full gate runs | 88 | 152 | 92 (by 15:00) |
|
|
33
|
+
| hours spent in gate commands | 3.3 | 8.4 | 7.0 |
|
|
34
|
+
|
|
35
|
+
- A task got about 6 full gate runs (13 at most): about 4–5 by the
|
|
36
|
+
worker and 2–3 by its reviewer, then a merge-time run by the
|
|
37
|
+
orchestrator.
|
|
38
|
+
- A worker's median tool calls per session went from 45 to 95, and the
|
|
39
|
+
gate's share of its wall time from 10–20% to 22–49%.
|
|
40
|
+
- Up to 4 gates ran at once on the same machine, which slows each one
|
|
41
|
+
down.
|
|
42
|
+
|
|
43
|
+
**Tests are not the cost.** A representative gate log
|
|
44
|
+
(`scratch/review-e2e_scaffold_2/gate.log`) shows:
|
|
45
|
+
- the 214 unit tests take **0.96 s**
|
|
46
|
+
- a cold compile of 583 crates for dev, test and release takes about
|
|
47
|
+
3 min, because every review worktree starts with an empty `target/`
|
|
48
|
+
- the rest is `contrib/e2e-sway`: 64 cases, 406 checks, and 178 fixed
|
|
49
|
+
`sleep`s in the script
|
|
50
|
+
|
|
51
|
+
What kills progress is the slow, unprofiled part of the gate, run
|
|
52
|
+
redundantly and with nobody whose job is to look at it.
|
|
53
|
+
|
|
54
|
+
Our own rules caused part of this:
|
|
55
|
+
- `recipes/adversarial-review.md` lists "an edge case without a test" as
|
|
56
|
+
a finding.
|
|
57
|
+
- The swaywardlock briefs told each worker to "re-run the gate" after
|
|
58
|
+
every rebase, and each reviewer to re-run the full `contrib/gate`,
|
|
59
|
+
then build the parent in a second worktree to prove every new test
|
|
60
|
+
fails there.
|
|
61
|
+
- Nothing in any recipe ever removes, merges, moves or speeds up a test.
|
|
62
|
+
|
|
63
|
+
## Goal
|
|
64
|
+
|
|
65
|
+
Testing stays encouraged. What changes is where tests live, how often
|
|
66
|
+
the expensive ones run, and that looking after the test suite becomes
|
|
67
|
+
normal project work instead of something that never happens.
|
|
68
|
+
|
|
69
|
+
## Concepts
|
|
70
|
+
|
|
71
|
+
- **Fast gate.** The checks anyone runs at any time, in seconds: format,
|
|
72
|
+
lint, unit tests, and cheap integration tests. A worker runs it every
|
|
73
|
+
time it iterates.
|
|
74
|
+
- **Slow gate.** Everything else the merge needs: e2e against a real
|
|
75
|
+
environment (headless compositor, real PAM), release builds, packaging
|
|
76
|
+
checks. Run once per tree, by whoever has to.
|
|
77
|
+
- **Gate budget.** The slow gate's target wall time, written next to the
|
|
78
|
+
gate commands. The default is 5 min, and the project can change it.
|
|
79
|
+
- **Gate record.** A task note names commit/tree, exact command and scope,
|
|
80
|
+
duration, toolchain/environment and skips. Reuse requires matching tree,
|
|
81
|
+
scope and equivalent execution conditions. A targeted run or privileged
|
|
82
|
+
check that skipped is not a full slow-gate pass.
|
|
83
|
+
- **Test upkeep.** Profile the gate, then speed up, move down a level,
|
|
84
|
+
merge or delete tests, so the gate stays inside its budget. It is
|
|
85
|
+
ordinary work in the DAG, not a cleanup that happens when someone
|
|
86
|
+
remembers.
|
|
87
|
+
- **Test level.** Unit < integration < e2e. A test belongs at the lowest
|
|
88
|
+
level that can catch the failure it guards against.
|
|
89
|
+
|
|
90
|
+
## Decisions
|
|
91
|
+
|
|
92
|
+
1. **The gates are defined in the repo.** The project defines its gates
|
|
93
|
+
in AGENTS.md (or the README), in a `## Gates` section: the fast
|
|
94
|
+
command, the slow command, the budget, and how to run one slow test,
|
|
95
|
+
e.g. `TESTS=preview contrib/e2e-sway`. Workstream plan notes point to
|
|
96
|
+
it. Recipes say "the fast gate" and "the slow gate" instead of "the
|
|
97
|
+
gate command".
|
|
98
|
+
*Why:* gates outlive workstreams. A plan note copy goes out of date.
|
|
99
|
+
2. **Who runs which gate:**
|
|
100
|
+
- **Worker:** the fast gate while iterating. Before closing, the
|
|
101
|
+
slow tests for its own area once, then the fast gate on the final
|
|
102
|
+
commit, and records both.
|
|
103
|
+
- **Reviewer:** reuses the worker's gate record when tree, scope and
|
|
104
|
+
execution conditions match. It runs only the tests that cover the diff, and reproduces
|
|
105
|
+
a finding when one is claimed. It no longer runs the full gate or
|
|
106
|
+
builds the parent to show a test fails there.
|
|
107
|
+
- **Orchestrator:** the full slow gate once per merge batch, on the
|
|
108
|
+
merged tree. When it fails, it bisects the batch.
|
|
109
|
+
|
|
110
|
+
*Why:* avoid repeating equivalent full runs across roles. The merged-tree
|
|
111
|
+
run checks interactions not covered by individual worker runs. Measure
|
|
112
|
+
batch failure rate during the pilot; the exploratory counts do not
|
|
113
|
+
establish how often bisecting will be necessary.
|
|
114
|
+
3. **Red-green only for bug fixes.** A fix shows the test failing on the
|
|
115
|
+
base and passing on the fix, and the worker shows it, not the
|
|
116
|
+
reviewer. Feature work does not have to show a test failing first.
|
|
117
|
+
4. **Rules for adding a test:**
|
|
118
|
+
- it goes at the lowest level that can catch the failure
|
|
119
|
+
- it covers behaviour no existing test covers
|
|
120
|
+
- a new slow-gate case states what only it can see, in a comment or
|
|
121
|
+
the commit message
|
|
122
|
+
- poll readiness/completion; retain documented temporal sampling,
|
|
123
|
+
negative-observation windows and deliberately sleeping process fixtures
|
|
124
|
+
5. **Reviewer findings about tests** change from "an edge case without a
|
|
125
|
+
test" to:
|
|
126
|
+
- a risky behaviour that is untested, or broken
|
|
127
|
+
- a test at the wrong level
|
|
128
|
+
- a test that duplicates test X
|
|
129
|
+
- an unnecessary readiness sleep, not a documented temporal assertion
|
|
130
|
+
- a gate that got noticeably slower in this diff
|
|
131
|
+
6. **Build cache across workspaces.** Prefer private targets with shared
|
|
132
|
+
dependency downloads and, when available, a compiler cache such as
|
|
133
|
+
sccache. Keep normal-auth, test-auth and release artifacts distinct and
|
|
134
|
+
pass explicit paths to consumers. Cargo's lock does not protect a binary
|
|
135
|
+
after a build exits: a shared writable target may replace it under tests.
|
|
136
|
+
Optional warm-up uses a quiescent snapshot into private storage, never
|
|
137
|
+
hardlinks or a live copy. This is recipe guidance, not a mu feature.
|
|
138
|
+
*Why:* inspected logs show substantial build time, but reuse and
|
|
139
|
+
contention must be measured rather than assuming a cache removes it.
|
|
140
|
+
7. **When test upkeep happens.** A test upkeep task is queued:
|
|
141
|
+
- when a merged slow gate goes over its budget (it comes before any
|
|
142
|
+
new feature work)
|
|
143
|
+
- at the end of every phase or milestone
|
|
144
|
+
8. **Gate timing is visible.** The gate prints how long each step takes
|
|
145
|
+
and names its slowest tests. The orchestrator adds the slow gate's
|
|
146
|
+
duration to its merge note, so the trend is visible in `mu log`
|
|
147
|
+
without any new tooling.
|
|
148
|
+
9. **No `mu gate` command yet.** Gate records are note text. Add a
|
|
149
|
+
command only if agents keep re-running gates despite the records.
|
|
150
|
+
|
|
151
|
+
## The `test-health` recipe (new)
|
|
152
|
+
|
|
153
|
+
`skills/mu/recipes/test-health.md`. It runs when a decision 7 trigger
|
|
154
|
+
fires, or when the user asks.
|
|
155
|
+
|
|
156
|
+
1. **Profile.**
|
|
157
|
+
- Time every gate step.
|
|
158
|
+
- List the 20 slowest tests at each level.
|
|
159
|
+
- Inspect waits: distinguish avoidable readiness padding from temporal
|
|
160
|
+
assertions, negative observations and sleeping fixtures.
|
|
161
|
+
- Find build work that runs more than once: cold caches, duplicate
|
|
162
|
+
profiles, the same build in dev and release.
|
|
163
|
+
- Record the gate's wall time and test counts as a BEFORE note.
|
|
164
|
+
2. **Classify.** One finder per area (parallel delegate calls). Each
|
|
165
|
+
slow or overlapping test gets one verdict:
|
|
166
|
+
- KEEP
|
|
167
|
+
- SPEED-UP: poll readiness instead of padding; share setup only if test
|
|
168
|
+
isolation is preserved
|
|
169
|
+
- MOVE-DOWN: rewrite at a lower level
|
|
170
|
+
- MERGE: fold into another test, which it names
|
|
171
|
+
- DELETE: covered by the test it names
|
|
172
|
+
Every verdict comes with its evidence.
|
|
173
|
+
3. **Refute.** Every MERGE and DELETE goes to a refuter: "which failure
|
|
174
|
+
would no longer be caught?" A refuted verdict becomes KEEP.
|
|
175
|
+
4. **Do.** The surviving verdicts run as normal tasks. They are
|
|
176
|
+
test-only diffs, and their review checks only that the coverage the
|
|
177
|
+
refuter accepted is still there.
|
|
178
|
+
5. **Measure.** Run the slow gate again and record an AFTER note. A pass
|
|
179
|
+
that leaves the gate neither faster nor smaller did not do its job:
|
|
180
|
+
say so, and say why.
|
|
181
|
+
|
|
182
|
+
Done when the AFTER note is recorded, and the slow gate is within its
|
|
183
|
+
budget or the note says what still keeps it over.
|
|
184
|
+
|
|
185
|
+
## Changes
|
|
186
|
+
|
|
187
|
+
| File | Change |
|
|
188
|
+
|---|---|
|
|
189
|
+
| `skills/mu/SKILL.md` | Vocabulary: replace the single gate command with fast gate / slow gate / gate budget / gate record / test upkeep. Orchestrator rules: one slow gate per merge batch; queue test-health when a trigger fires. |
|
|
190
|
+
| `skills/mu/recipes/test-health.md` | New, as above. |
|
|
191
|
+
| `skills/mu/recipes/adversarial-review.md` | Reviewer brief: reuse gate records; run only the tests for the diff; the new test findings (decision 5). |
|
|
192
|
+
| `skills/mu/recipes/worker.md`, `recipes/brief.md` | Who runs which gate (decision 2); the gate record line in the DONE note; red-green only for fixes; rules for adding a test. |
|
|
193
|
+
| `skills/mu/recipes/orchestrator-loop.md` § Merging | Batch merge, then one slow gate, then bisect when it fails. Add the slow gate's duration to the merge note. Section on sharing the build cache. |
|
|
194
|
+
| `skills/mu/recipes/waves.md`, `review-panel.md`, `remote-workers.md` | "Gate command" → "slow gate" where they mean the merge gate. |
|
|
195
|
+
| `skills/mu/recipes/plan.md` | A new project's plan defines `## Gates` in the repo first. Every phase ends with a test-health task. |
|
|
196
|
+
|
|
197
|
+
Out of scope: a `mu gate` command, caching gate results in the mu DB,
|
|
198
|
+
and test-coverage tooling.
|
|
199
|
+
|
|
200
|
+
## Rollout
|
|
201
|
+
|
|
202
|
+
1. **Battle-test on swaywardlock** before editing any recipe:
|
|
203
|
+
- after Phase 3 owners release the repo, add `## Gates` to its README
|
|
204
|
+
- split and instrument `contrib/gate`, preserving default full execution
|
|
205
|
+
and all checks; route CI through the canonical inventory
|
|
206
|
+
- full slow mode clears/rejects inherited `TESTS`; targeted runs remain
|
|
207
|
+
explicit, and standalone slow mode builds its own test-auth executable
|
|
208
|
+
- preserve default-feature auth tests and all three release hook scans;
|
|
209
|
+
report privilege/scdoc skips explicitly
|
|
210
|
+
- time every gate step and e2e case including setup/teardown; establish
|
|
211
|
+
an inspected baseline before moving or removing coverage
|
|
212
|
+
- retain private build artifacts; investigate safe cache reuse separately
|
|
213
|
+
- make one low-risk readiness-wait change, then measure before/after
|
|
214
|
+
- run `test-health` by hand, from this spec
|
|
215
|
+
- write down where the spec was wrong
|
|
216
|
+
2. Write the recipe changes from what we learned.
|
|
217
|
+
3. **Measure** with `museum-search stats` (museum spec
|
|
218
|
+
`2026-10-21-session-stats.md`): full gate runs per task, gate share
|
|
219
|
+
of worker time, slow gate duration. Compare swaywardlock Phase 3 with
|
|
220
|
+
the next phase.
|
|
221
|
+
|
|
222
|
+
## Open questions
|
|
223
|
+
|
|
224
|
+
- Where the line between the fast and slow gate falls when a project has
|
|
225
|
+
a cheap integration test that needs a daemon. Settle it on
|
|
226
|
+
swaywardlock, where `contrib/test-real-pam` and the converter tests
|
|
227
|
+
are candidates for either side.
|
|
228
|
+
- Whether the gate record should carry the commit sha as well as the
|
|
229
|
+
tree hash, for readability. It costs nothing, so probably yes.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mu-crew/mu",
|
|
3
|
-
"version": "3.13.
|
|
3
|
+
"version": "3.13.3",
|
|
4
4
|
"description": "A persistent, observable crew of pi agents running in one tmux or herdr session per workstream, coordinated through a built-in task DAG.",
|
|
5
5
|
"publishConfig": {
|
|
6
6
|
"access": "public"
|
package/skills/mu/SKILL.md
CHANGED
|
@@ -56,9 +56,9 @@ Recipe words, the same in every recipe:
|
|
|
56
56
|
finding and round; holds the brief, criteria and stop rule; closes
|
|
57
57
|
with `mu task close <umbrella> --if-ready`.
|
|
58
58
|
- **unit** — one piece of the job, one task (a file, a call site, a fix).
|
|
59
|
-
- **gate
|
|
60
|
-
|
|
61
|
-
blocked by `x`)
|
|
59
|
+
- **fast gate / slow gate / gate budget / gate record / test upkeep** —
|
|
60
|
+
repo-defined verification and its upkeep ([test-health](recipes/test-health.md#gate-contract));
|
|
61
|
+
a **review gate** (`review_x` blocked by `x`) decides whether `x` counts.
|
|
62
62
|
- **wave** — the units dispatched from one main revision; refresh
|
|
63
63
|
workspaces before the next.
|
|
64
64
|
- **finding** — one reported problem, recorded as an `OPEN/triage` task.
|
|
@@ -183,9 +183,11 @@ rules hold even when you skip it:
|
|
|
183
183
|
|
|
184
184
|
- **Claim before sending**, even one-shot reviewers. Ownership is
|
|
185
185
|
durable and waitable; agent state is not.
|
|
186
|
-
- **Pipeline; don't barrier.** Merge
|
|
186
|
+
- **Pipeline; don't barrier.** Merge accepted tasks as they close; verify
|
|
187
|
+
one pinned merge batch without pausing independent workers.
|
|
187
188
|
- **Verify the merge, not the worker's rerun.** Only `CLOSED/done` ships.
|
|
188
|
-
- **Push
|
|
189
|
+
- **Push after merged-tree gates pass.** Run full slow once per batch, plus
|
|
190
|
+
fast if needed; queue [test upkeep](recipes/test-health.md#upkeep-loop) at milestones or budget crossings.
|
|
189
191
|
- **Define done before a long run**: the completion criterion goes in
|
|
190
192
|
the task note first ([long-run](recipes/long-run.md)).
|
|
191
193
|
- **New work to pi is `mu agent send --fresh`.** Never chain `/new` and
|
|
@@ -281,6 +283,7 @@ risky job that needs several of them, start with
|
|
|
281
283
|
| [brief](recipes/brief.md) | you write a task note or prompt a worker will read |
|
|
282
284
|
| [plan](recipes/plan.md) | you turn a spec or requirements into a task DAG |
|
|
283
285
|
| [worker](recipes/worker.md) | a task was claimed for you |
|
|
286
|
+
| [test-health](recipes/test-health.md) | define gates, choose verification, add tests, or upkeep at milestones/budget crossings |
|
|
284
287
|
| [recovery](recipes/recovery.md) | undo, rebuild, teardown, sync setup, `doctor` cleanup |
|
|
285
288
|
| [remote-workers](recipes/remote-workers.md) | an agent runs on another machine |
|
|
286
289
|
| [waves](recipes/waves.md) | more than one worker edits the same repo at once |
|
|
@@ -17,9 +17,9 @@ PR or diff from several angles, use [review-panel](review-panel.md).
|
|
|
17
17
|
## Steps
|
|
18
18
|
|
|
19
19
|
1. **Write the criteria first.** Before dispatching the work, note on
|
|
20
|
-
the work task what "accepted" means:
|
|
21
|
-
|
|
22
|
-
|
|
20
|
+
the work task what "accepted" means: requirements, risky behaviours
|
|
21
|
+
needing coverage, and files that must not change. The reviewer judges
|
|
22
|
+
against this note, not against its own taste.
|
|
23
23
|
2. **Add the review task, blocked by the work:**
|
|
24
24
|
|
|
25
25
|
```bash
|
|
@@ -53,20 +53,24 @@ PR or diff from several angles, use [review-panel](review-panel.md).
|
|
|
53
53
|
ask the human. A third round of the same argument rarely converges.
|
|
54
54
|
|
|
55
55
|
Done when the latest review task is `CLOSED/done` (ACCEPT), and the
|
|
56
|
-
accepted commits are merged and the
|
|
56
|
+
accepted commits are merged and the batch passes the merged-tree gates
|
|
57
|
+
([orchestrator-loop § Merging](orchestrator-loop.md#merging)).
|
|
57
58
|
|
|
58
59
|
## Reviewer brief
|
|
59
60
|
|
|
60
|
-
Send this with the task id,
|
|
61
|
+
Send this with the task id, workspace path, commit range, gate records and
|
|
62
|
+
[test-health](test-health.md) pointer:
|
|
61
63
|
|
|
62
64
|
```text
|
|
63
65
|
You are reviewing task <x>. You did not write it. Your job is to find
|
|
64
66
|
where it fails the acceptance criteria in `mu task notes <x>`.
|
|
65
67
|
|
|
66
|
-
- Read the diff (<range>)
|
|
67
|
-
-
|
|
68
|
-
|
|
69
|
-
|
|
68
|
+
- Read the diff (<range>), criteria and worker gate records. Follow
|
|
69
|
+
test-health.md for record equivalence, targeted runs and test findings.
|
|
70
|
+
Reproduce claimed failures; a full rerun or parent build is not routine.
|
|
71
|
+
- Report findings against the criteria: a requirement not met, risky
|
|
72
|
+
untested/broken behaviour, a change outside scope, unsupported evidence.
|
|
73
|
+
Not style, not preferences.
|
|
70
74
|
- Record each as a finding, per findings.md § Record:
|
|
71
75
|
mu task add -w <ws> --triage -t "<severity>: <title>" -i <n> -e <days> \
|
|
72
76
|
--note 'FINDING: <severity> <file:line> <what fails> EVIDENCE: <command + output>'
|
|
@@ -92,7 +96,7 @@ by orchestrator`.
|
|
|
92
96
|
## Traps
|
|
93
97
|
|
|
94
98
|
- **A reviewer that sees the author's reasoning agrees with it.** Send
|
|
95
|
-
the diff and
|
|
99
|
+
the diff, criteria and gate records, not the worker's reasoning transcript.
|
|
96
100
|
- **"Looks good" is not a verdict.** Require a close that matches the
|
|
97
101
|
verdict (`done` or `--as rejected`) and at least one command the
|
|
98
102
|
reviewer ran. A review with no evidence is a
|
|
@@ -14,7 +14,7 @@ not in the brief, or one pointer away, does not exist for it.
|
|
|
14
14
|
`npm run typecheck`", not "update the call sites". A vague bound
|
|
15
15
|
invites the worker to stop at the first plausible point.
|
|
16
16
|
2. **Inline what every path needs; point to the rest.** Files, the
|
|
17
|
-
verify command, scope, and the final action go in the brief. Shared
|
|
17
|
+
targeted verify command, scope, and the final action go in the brief. Shared
|
|
18
18
|
context goes in one place (an umbrella's note) and the brief points
|
|
19
19
|
at it: `Read: mu task notes sweep -w audit`. Repeating it per task
|
|
20
20
|
copies it into every worker's context and lets the copies drift.
|
|
@@ -43,13 +43,18 @@ Read: mu task notes <id> -w <ws> (and the umbrella's, if any)
|
|
|
43
43
|
GOAL: <one sentence>
|
|
44
44
|
FILES: <paths it may change>
|
|
45
45
|
DONE WHEN: <checkable, exhaustive criterion>
|
|
46
|
-
VERIFY: <exact command>
|
|
46
|
+
VERIFY: <repo Gates pointer + exact targeted command/scope; test-health.md by role>
|
|
47
|
+
NOTE: <applicable gate records before close; test-health.md § Gate contract>
|
|
47
48
|
|
|
48
49
|
⚠️ FINAL ACTION
|
|
49
50
|
git commit -am '<msg>' THEN
|
|
50
51
|
mu task close <id> -w <ws> --evidence "$v" # command + result; see Quoting
|
|
51
52
|
```
|
|
52
53
|
|
|
54
|
+
Use [test-health](test-health.md) for worker/reviewer gate allocation, gate
|
|
55
|
+
records, test levels and cheap bug red-green. Point to it rather than asking
|
|
56
|
+
each role for a full gate or parent build.
|
|
57
|
+
|
|
53
58
|
## Quoting
|
|
54
59
|
|
|
55
60
|
Single quotes suit short text with no `'`. Prose (an apostrophe ends
|
|
@@ -19,9 +19,9 @@ the loop and the reasons behind them.
|
|
|
19
19
|
`mu agent send <w> --fresh '...'` (new session + prompt in one step;
|
|
20
20
|
refuses while busy). The brief ends with its final-action block.
|
|
21
21
|
5. `mu task wait ... --first --on-stall exit --json`.
|
|
22
|
-
6. Cherry-pick the
|
|
23
|
-
then continue the loop in the same turn. Do not barrier or
|
|
24
|
-
shell.
|
|
22
|
+
6. Cherry-pick the accepted worker's **new** commit(s) into a merge batch
|
|
23
|
+
(below), then continue the loop in the same turn. Do not barrier or
|
|
24
|
+
loop in shell.
|
|
25
25
|
Only `CLOSED/done` ships; other closes: read the reason note.
|
|
26
26
|
7. Repeat from `mu state`.
|
|
27
27
|
|
|
@@ -42,9 +42,9 @@ keeps it.
|
|
|
42
42
|
ends with its `VERDICT: <id>` block, so a batch still maps back. Read
|
|
43
43
|
every verdict in the batch before deciding any.
|
|
44
44
|
- **Workers with workspaces** build and test: about one per CPU core,
|
|
45
|
-
fewer when
|
|
46
|
-
session cap ([remote-workers](remote-workers.md)).
|
|
47
|
-
- **Stream
|
|
45
|
+
fewer when targeted slow checks contend for resources. A remote host has
|
|
46
|
+
its own session cap ([remote-workers](remote-workers.md)).
|
|
47
|
+
- **Stream dispatch.** Keep the pool full: dispatch the next unit
|
|
48
48
|
when one closes, rather than waiting for a whole wave. With the
|
|
49
49
|
`codemode` tool, a [driver](codemode-driver.md) can dispatch a batch
|
|
50
50
|
of workers in one script.
|
|
@@ -72,15 +72,21 @@ per turn ([remote-workers](remote-workers.md#step-5-poll-once-per-turn)).
|
|
|
72
72
|
|
|
73
73
|
## Merging
|
|
74
74
|
|
|
75
|
-
- **Pipeline; don't barrier.**
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
- **
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
75
|
+
- **Pipeline; don't barrier.** Cherry-pick accepted new commits as tasks
|
|
76
|
+
close and dispatch the next unit. Choose a bounded merge batch; an umbrella
|
|
77
|
+
wait hides progress and stale branches can restore reverted code.
|
|
78
|
+
- **Pin the batch tree.** Run full slow once, plus fast if slow excludes it,
|
|
79
|
+
under [the gate contract](test-health.md#gate-contract). Keep the tested
|
|
80
|
+
checkout unchanged during the run; independent workers continue in their
|
|
81
|
+
workspaces. Later merges form the next batch, not an extension of this pass.
|
|
82
|
+
Remote hosts: [run the merged gate there](remote-workers.md#run-the-merged-gate-on-the-host).
|
|
83
|
+
- **Bisect a failed batch.** Isolate the offending commit/interaction, fix it,
|
|
84
|
+
and gate the corrected merged tree before shipping.
|
|
85
|
+
- **Push only the verified tree.** Make `git push` the last line under
|
|
86
|
+
`set -e`; required integration and safety checks must actually run.
|
|
87
|
+
Record the gate result and slow duration with the merged tree in the merge
|
|
88
|
+
note/`mu log`. Queue [upkeep](test-health.md#upkeep-loop) on a budget crossing
|
|
89
|
+
or milestone, rather than stopping all independent work.
|
|
84
90
|
- **Accept evidence, not close notes.** Re-run the key measurement from
|
|
85
91
|
a clean checkout; close notes have claimed unpushed commits. A
|
|
86
92
|
worker's "done" counts only once the diff shows the change. Your own
|
|
@@ -90,6 +96,16 @@ per turn ([remote-workers](remote-workers.md#step-5-poll-once-per-turn)).
|
|
|
90
96
|
agreeing with itself, a fixture you tuned on. A nonzero diff is a
|
|
91
97
|
failure even when the command exited 0.
|
|
92
98
|
|
|
99
|
+
## Build cache
|
|
100
|
+
|
|
101
|
+
Keep build targets private per workspace and auth/profile lane. Share dependency
|
|
102
|
+
downloads and use an existing compiler cache when available; measure cold/warm
|
|
103
|
+
cost and contention. Keep normal-auth, test-auth and release artifacts distinct,
|
|
104
|
+
with explicit executable paths for consumers. A shared writable Cargo target
|
|
105
|
+
can replace a binary after its build lock is released, while tests still use it.
|
|
106
|
+
Warm-up may copy a quiescent snapshot into private storage, never hardlinks or
|
|
107
|
+
a live target. This is build guidance, not a new mu cache or dependency.
|
|
108
|
+
|
|
93
109
|
## Sending
|
|
94
110
|
|
|
95
111
|
A running worker sees what you send, never a new task note: deliver
|
|
@@ -7,8 +7,10 @@ human asks for a file.
|
|
|
7
7
|
|
|
8
8
|
## Steps
|
|
9
9
|
|
|
10
|
-
1. **Map
|
|
11
|
-
|
|
10
|
+
1. **Map files and gates.** List what gets created or changed and each
|
|
11
|
+
file's responsibility. For a new project, define repo `## Gates` first
|
|
12
|
+
([test-health](test-health.md#gate-contract)); point plan notes there.
|
|
13
|
+
Existing gates remain until their replacement is approved.
|
|
12
14
|
2. **Size the tasks.** A task is the smallest unit that carries its own
|
|
13
15
|
verification and is worth a review. Split only where a reviewer
|
|
14
16
|
could reject one task and accept its neighbour. Fold setup, config,
|
|
@@ -59,5 +61,7 @@ lists the tasks you meant to run in parallel as Ready together. Then dispatch pe
|
|
|
59
61
|
its step count. Guess honestly rather than defaulting to 50 and 1.
|
|
60
62
|
- Review gates follow [adversarial-review](adversarial-review.md): a
|
|
61
63
|
review task blocked by the work.
|
|
64
|
+
- End each phase/milestone with a [test-health task](test-health.md#upkeep-loop);
|
|
65
|
+
connect real dependencies, not a barrier across independent workers.
|
|
62
66
|
- When reality disagrees with the plan, append a note saying so. Do not
|
|
63
67
|
rewrite earlier notes.
|
|
@@ -53,7 +53,7 @@ git fetch "ssh://dev/~/ws/worker-1" HEAD && git cherry-pick FETCH_HEAD
|
|
|
53
53
|
```
|
|
54
54
|
|
|
55
55
|
Done when the worker's commits are cherry-picked locally, the merged
|
|
56
|
-
tree passes the
|
|
56
|
+
tree passes the merged-tree gates (host and local checks below), and the pane is
|
|
57
57
|
closed.
|
|
58
58
|
|
|
59
59
|
### Step 2: the note is load-bearing
|
|
@@ -117,9 +117,9 @@ while the worker asks a question, do not poll: read the pane
|
|
|
117
117
|
|
|
118
118
|
### Run the merged gate on the host
|
|
119
119
|
|
|
120
|
-
|
|
121
|
-
Cherry-pick locally, push
|
|
122
|
-
there with
|
|
120
|
+
Reuse only equivalent [gate records](test-health.md#gate-contract).
|
|
121
|
+
Cherry-pick a merge batch locally, push its pinned head to the host, and run
|
|
122
|
+
full slow there with warm dependencies, plus fast if slow excludes it:
|
|
123
123
|
|
|
124
124
|
```bash
|
|
125
125
|
git cherry-pick <sha>
|
|
@@ -131,12 +131,16 @@ mule run --cwd ~/hacking/<gate-checkout> --wait \
|
|
|
131
131
|
`<gate-checkout>` is a scratch checkout or worktree used only for
|
|
132
132
|
gates, never one holding work. Push to a temporary ref, never `main`;
|
|
133
133
|
delete it afterwards (`git push origin :gate-tmp`). Push to `main` only
|
|
134
|
-
after the
|
|
134
|
+
after required gates pass. Replace `npm run check` with the repo's canonical
|
|
135
|
+
merged-tree command(s), preserving full scope and recording host conditions.
|
|
136
|
+
Keep the gate checkout pinned while other remote workers continue in theirs;
|
|
137
|
+
use private artifacts ([build cache](orchestrator-loop.md#build-cache)).
|
|
135
138
|
|
|
136
139
|
`--wait` exits with the suite's code. A bare ssh would hold the capped
|
|
137
|
-
channel for minutes.
|
|
138
|
-
Linux `ps -e` appends the environment, and a real bug lived
|
|
139
|
-
the final gate before the push that matters.
|
|
140
|
+
channel for minutes. Retain local platform-sensitive checks (macOS `ps`
|
|
141
|
+
returns argv, Linux `ps -e` appends the environment, and a real bug lived
|
|
142
|
+
there), and the repo-required final local gate before the push that matters.
|
|
143
|
+
A remote pass does not substitute for non-equivalent local conditions.
|
|
140
144
|
|
|
141
145
|
## Traps
|
|
142
146
|
|
|
@@ -28,11 +28,12 @@ panel's findings will be fixed by agents. Both modes are in
|
|
|
28
28
|
| correctness | logic errors, edge cases, error paths, races |
|
|
29
29
|
| security | injection, auth and permission checks, secrets, unsafe input |
|
|
30
30
|
| simplicity | duplication, dead code, needless abstraction, an existing helper not reused |
|
|
31
|
-
| tests |
|
|
31
|
+
| tests | risky coverage gaps, ineffective tests, level/duplication/wait/cost findings ([test-health](test-health.md#adding-or-changing-tests)) |
|
|
32
32
|
| rules | the repo's written rules (see [rules-audit](rules-audit.md)) |
|
|
33
33
|
|
|
34
34
|
Each reviewer gets the range, its angle, and the [brief](brief.md)
|
|
35
|
-
rules
|
|
35
|
+
rules and available gate records ([test-health](test-health.md#gate-contract));
|
|
36
|
+
coordinate targeted checks across angles. It edits nothing. A clean result (`FINDINGS: none` plus the
|
|
36
37
|
commands it ran) is a valid result.
|
|
37
38
|
3. **Record findings** by mode:
|
|
38
39
|
- delegate: each reviewer ends its answer with `FINDING:` lines;
|
|
@@ -60,5 +61,5 @@ triage and the umbrella closes with `--if-ready` (workstream mode).
|
|
|
60
61
|
author's tier or higher ([models](models.md#checkers)); one model's
|
|
61
62
|
blind spot repeated five times is still one blind spot. Pass `model`
|
|
62
63
|
to `mu_delegate`.
|
|
63
|
-
- **The panel is not the gate
|
|
64
|
+
- **The panel is not the slow gate.** Merge verification still runs on the
|
|
64
65
|
merged tree ([orchestrator-loop](orchestrator-loop.md#merging)).
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# Test health
|
|
2
|
+
|
|
3
|
+
Read when defining gates, choosing worker/reviewer verification, adding tests,
|
|
4
|
+
or doing upkeep at a milestone, a budget crossing, or the user's request.
|
|
5
|
+
Testing stays encouraged; upkeep protects coverage and the iteration loop.
|
|
6
|
+
|
|
7
|
+
## Gate contract
|
|
8
|
+
|
|
9
|
+
Use the repo's `## Gates` in AGENTS.md or README: fast and slow commands,
|
|
10
|
+
slow-gate budget (default 5 min), and a targeted slow-test command. Plan notes
|
|
11
|
+
point there. Until a split is approved, keep the existing repo gate intact;
|
|
12
|
+
recipe guidance does not waive project checks. Fast means iteration checks,
|
|
13
|
+
not a universal seconds promise: measure cold and warm costs.
|
|
14
|
+
|
|
15
|
+
- **Worker:** run the fast gate while iterating; before closing, run slow
|
|
16
|
+
tests for the changed area once, then fast on the final commit. Record
|
|
17
|
+
both, or why a scope is inapplicable/unavailable. Changes invalidate records.
|
|
18
|
+
- **Reviewer:** inspect the diff and reuse equivalent worker records. Run
|
|
19
|
+
uncovered tests for the diff and reproduce claimed failures, rather than
|
|
20
|
+
repeating the full suite or building the parent for every new test.
|
|
21
|
+
- **Orchestrator:** verify one pinned merged tree per batch with the full
|
|
22
|
+
slow gate, plus fast if slow does not include it. All required integration,
|
|
23
|
+
auth, release and packaging checks still pass before push. Bisect a failed
|
|
24
|
+
batch; record the corrected tree's pass. See [merging](orchestrator-loop.md#merging).
|
|
25
|
+
|
|
26
|
+
Gate records are task-note text, not a mu command or stored result cache:
|
|
27
|
+
|
|
28
|
+
```text
|
|
29
|
+
GATE: commit=<sha> tree=<hash>; command=<exact command + filters>; scope=<full/area>
|
|
30
|
+
RESULT: exit=<code>; duration=<wall time>; counts=<tests/assertions>; skips=<none or reasons>
|
|
31
|
+
CONDITIONS: <toolchain, host/platform, privileges, relevant env/features, cache state>
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Reuse requires the same tree, command/scope and equivalent execution
|
|
35
|
+
conditions. Record uncommitted inputs too; a dirty tree hash alone is not
|
|
36
|
+
an identity. A targeted pass, zero selected cases, or a skipped privileged
|
|
37
|
+
check is not a full pass. Clear/reject inherited selectors for full runs;
|
|
38
|
+
report skips explicitly and resolve required missing checks before push.
|
|
39
|
+
|
|
40
|
+
## Adding or changing tests
|
|
41
|
+
|
|
42
|
+
Use the lowest level that catches the failure, covering behaviour not
|
|
43
|
+
already guarded. A new slow case states what only it can see in a comment
|
|
44
|
+
or commit message. For a bug with a cheap, obvious local target, the worker
|
|
45
|
+
shows red on the base and green on the fix; feature work and integration-heavy
|
|
46
|
+
paths need no mandatory red-green ceremony. Use direct executable evidence
|
|
47
|
+
when stronger, recording the reason.
|
|
48
|
+
|
|
49
|
+
Poll readiness/completion with a bound and useful failure output. Retain
|
|
50
|
+
necessary temporal sampling, negative-observation windows and deliberately
|
|
51
|
+
sleeping process fixtures; explain their timing contract.
|
|
52
|
+
|
|
53
|
+
Reviewer findings identify risky untested/broken behaviour, a wrong test
|
|
54
|
+
level, duplication of a named test, avoidable readiness padding, or measured
|
|
55
|
+
gate slowdown—not every conceivable edge case without a test.
|
|
56
|
+
|
|
57
|
+
## Upkeep loop
|
|
58
|
+
|
|
59
|
+
Queue ordinary upkeep work at each phase/milestone end or when a merged slow
|
|
60
|
+
gate exceeds budget. Prioritise it before new features; continue independent
|
|
61
|
+
work. A budget is a scheduling trigger, not an indefinite crew-wide stop.
|
|
62
|
+
|
|
63
|
+
1. **Profile.** Time each gate step and test, including setup/teardown.
|
|
64
|
+
List up to 20 slowest tests per level; inspect waits and repeated builds,
|
|
65
|
+
cold caches and profiles. BEFORE records wall time, counts, skips and
|
|
66
|
+
conditions. Use inspected execution logs, not commands merely mentioning
|
|
67
|
+
a gate or compound-command durations attributed entirely to tests.
|
|
68
|
+
2. **Classify.** Parallel [delegate calls](tasks-or-calls.md#delegate-call),
|
|
69
|
+
one per area, give slow/overlapping tests an evidenced verdict:
|
|
70
|
+
KEEP; SPEED-UP (poll or share setup with isolation); MOVE-DOWN;
|
|
71
|
+
MERGE into a named test; DELETE as covered by a named test.
|
|
72
|
+
3. **Refute.** Send every MERGE/DELETE verdict to a refuter: which failure
|
|
73
|
+
would no longer be caught? A refuted or unresolved verdict becomes KEEP.
|
|
74
|
+
4. **Do.** Dispatch surviving changes as normal test-only tasks. Review
|
|
75
|
+
coverage preservation, isolation and correctness of the harness changes.
|
|
76
|
+
Preserve auth lanes, privilege checks and release scans; measure safe
|
|
77
|
+
[build reuse](orchestrator-loop.md#build-cache) separately.
|
|
78
|
+
5. **Measure.** Run the merged gates and record AFTER under comparable
|
|
79
|
+
conditions, including coverage/count changes. If neither faster nor
|
|
80
|
+
smaller, say so and why; smaller alone does not prove preserved coverage.
|
|
81
|
+
|
|
82
|
+
6. **Report.** Before teardown, commit a dated report to the repo
|
|
83
|
+
(`docs/test-health/<YYYY-MM-DD>.md`): conditions, BEFORE/AFTER, each
|
|
84
|
+
finding with its evidence, every refuted or rejected verdict with the
|
|
85
|
+
reason, and the remaining bottleneck. Task notes die with the
|
|
86
|
+
workstream; the next pass reads the latest report first and still
|
|
87
|
+
re-measures. Add a file per pass; never edit an old one.
|
|
88
|
+
|
|
89
|
+
Done when AFTER records the outcome, the report is committed, and the slow
|
|
90
|
+
gate is within budget or the report names the remaining bottleneck and
|
|
91
|
+
prioritised follow-up. Keep unmet required checks distinct from a
|
|
92
|
+
time-budget overrun.
|
|
@@ -10,12 +10,12 @@ runs in its own `--workspace`; this recipe decides what each one gets.
|
|
|
10
10
|
the shared files. A blanket freeze idled four of five workers for a
|
|
11
11
|
day.
|
|
12
12
|
3. **Pipeline the merge.** Cherry-pick each task as it closes and
|
|
13
|
-
verify
|
|
14
|
-
|
|
13
|
+
verify pinned merge batches ([orchestrator-loop § Merging](orchestrator-loop.md#merging));
|
|
14
|
+
batching gates does not mean waiting for the whole wave.
|
|
15
15
|
4. **Refresh before each dispatch.** `mu workspace refresh <agent>`
|
|
16
16
|
rebases onto the tracked main (git: `origin/HEAD`, so unpushed local
|
|
17
17
|
merges are missed; pass `--from <local-branch>` to include them) and
|
|
18
18
|
keeps the agent's context; run it before every `--fresh` send.
|
|
19
19
|
|
|
20
20
|
Done when every task in the wave is `CLOSED/done`, merged, and the
|
|
21
|
-
merged tree passes the gate
|
|
21
|
+
merged tree passes the slow gate and fast if needed, per the repo contract.
|
|
@@ -17,11 +17,15 @@ mu task note <id> "FILES: ...\nCOMMANDS: ... (exit codes)\nDECISION: ...\nVERIFI
|
|
|
17
17
|
mu task close <id> --evidence "tests pass: ..." # LAST action
|
|
18
18
|
```
|
|
19
19
|
|
|
20
|
+
Follow [test-health](test-health.md) for fast iteration checks, targeted slow
|
|
21
|
+
checks before close, gate records, and cheap bug red-green. Include the gate
|
|
22
|
+
records in your final note; adding tests uses its test-level and timing rules.
|
|
23
|
+
|
|
20
24
|
The note follows the task note contract in SKILL.md. Keep each tool
|
|
21
25
|
call to minutes (short batches, waits with a timeout): steering reaches
|
|
22
26
|
you only between tool calls. Commit before you close: the orchestrator cherry-picks your commits, not your tree. A
|
|
23
27
|
review or audit task records each problem per
|
|
24
28
|
[findings § Record](findings.md#record).
|
|
25
29
|
|
|
26
|
-
Done when your changes are committed in the workspace, the task note holds FILES, COMMANDS (with exit codes) and
|
|
30
|
+
Done when your changes are committed in the workspace, the task note holds FILES, COMMANDS (with exit codes), VERIFIED and applicable gate records, and `mu task close` succeeded as your last action. Skipping close makes the orchestrator's
|
|
27
31
|
wait hang. Won't do it: `close --as wontfix --why "..."`.
|