opencode-thrum-pro 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/assets/commands/thrum-_compact-runtime-commands.md +61 -0
- package/assets/commands/thrum-_snapshot-protocol.md +17 -6
- package/assets/commands/thrum-_stamp-protocol.md +9 -2
- package/assets/commands/thrum-compact-extended.md +352 -0
- package/assets/commands/thrum-compact.md +300 -0
- package/assets/commands/thrum-inbox.md +1 -1
- package/assets/commands/thrum-restart-extended.md +84 -11
- package/assets/commands/thrum-restart.md +105 -13
- package/assets/commands/thrum-send.md +4 -1
- package/assets/commands/thrum-sleep-extended.md +67 -46
- package/assets/commands/thrum-sleep.md +67 -58
- package/assets/skills/adversarial-critique/SKILL.md +5 -8
- package/assets/skills/agent-messaging-protocol/SKILL.md +35 -0
- package/assets/skills/brainstorm-steward/SKILL.md +65 -11
- package/assets/skills/brainstormer-staying-current/SKILL.md +113 -0
- package/assets/skills/choosing-subagent-models/SKILL.md +14 -0
- package/assets/skills/conversation-only-context/SKILL.md +11 -11
- package/assets/skills/conversation-only-context/resources/driver-subagent-pattern.md +1 -1
- package/assets/skills/conversation-only-context/resources/remote-environment-scenarios.md +1 -1
- package/assets/skills/coordinator-assessing-agent-completion/SKILL.md +98 -6
- package/assets/skills/coordinator-context-monitoring/SKILL.md +29 -19
- package/assets/skills/coordinator-deploying-a-node/SKILL.md +124 -0
- package/assets/skills/coordinator-dispatching-work/SKILL.md +24 -11
- package/assets/skills/coordinator-hotpath-merge-gate/SKILL.md +62 -20
- package/assets/skills/coordinator-managing-state-and-lifecycle/SKILL.md +6 -6
- package/assets/skills/coordinator-merging-code/SKILL.md +25 -11
- package/assets/skills/coordinator-philosophy-merge-gate/SKILL.md +28 -7
- package/assets/skills/coordinator-plan-reconcile-gate/SKILL.md +3 -3
- package/assets/skills/coordinator-rolling-out-to-nodes/SKILL.md +92 -0
- package/assets/skills/coordinator-running-autonomous-feature-development/SKILL.md +3 -3
- package/assets/skills/coordinator-running-brainstorm-cycles/SKILL.md +58 -71
- package/assets/skills/coordinator-running-review-cycles/SKILL.md +15 -4
- package/assets/skills/diagnosing-fleet-and-daemon/SKILL.md +0 -2
- package/assets/skills/efficient-multi-agent-research/SKILL.md +6 -7
- package/assets/skills/implementer-orchestrating-execution/SKILL.md +11 -14
- package/assets/skills/orchestrate/SKILL.md +47 -35
- package/assets/skills/persistent-watcher-archetype/SKILL.md +188 -37
- package/assets/skills/persistent-watcher-archetype/resources/capture_archive.py +618 -0
- package/assets/skills/persistent-watcher-archetype/resources/jev_watcher_filter.py +837 -0
- package/assets/skills/persistent-watcher-archetype/resources/roster-watch.sh +1140 -0
- package/assets/skills/persistent-watcher-archetype/resources/roster-watch_test.sh +291 -0
- package/assets/skills/persistent-watcher-archetype/resources/thrum-capture-fallback.sh +2 -2
- package/assets/skills/persistent-watcher-archetype/resources/thrum-watch-pane-capture.sh +20 -8
- package/assets/skills/prime-agent/SKILL.md +10 -0
- package/assets/skills/project-setup/SKILL.md +22 -16
- package/assets/skills/project-setup/resources/implementation-agent.md +56 -35
- package/assets/skills/researcher-investigating/SKILL.md +2 -5
- package/assets/skills/state-migration/SKILL.md +1 -1
- package/assets/skills/thrum/SKILL.md +14 -7
- package/assets/skills/thrum/resources/ANTI_PATTERNS.md +16 -18
- package/assets/skills/thrum/resources/CLI_REFERENCE.md +17 -11
- package/assets/skills/thrum/resources/LISTENER_PATTERN.md +9 -3
- package/assets/skills/thrum/resources/MESSAGING.md +18 -11
- package/assets/skills/thrum/resources/WORKTREES.md +1 -1
- package/assets/skills/thrum-memory/SKILL.md +1 -1
- package/assets/skills/trace-spec-to-plan/resources/sweep-agent-prompt.md +1 -1
- package/assets/skills/update-queue/SKILL.md +6 -8
- package/assets/skills/using-the-dashboard/SKILL.md +30 -0
- package/assets/skills/using-the-queue/SKILL.md +76 -5
- package/assets/skills/using-thrum-state/SKILL.md +84 -1
- package/assets/skills/verify-against-plan/SKILL.md +120 -5
- package/assets/skills/watcher-archetype/SKILL.md +18 -0
- package/assets/skills/writing-durable-markdown/SKILL.md +5 -0
- package/dist/installer.js +4 -4
- package/dist/permission-fixture.integration.test.js +4 -4
- package/package.json +1 -1
- package/assets/skills/coordinator-deploying-the-fleet/SKILL.md +0 -396
- package/dist/allowlist-sync.test.d.ts +0 -1
- package/dist/allowlist-sync.test.js +0 -44
|
@@ -117,23 +117,35 @@ rediscovering these the hard way:
|
|
|
117
117
|
carrying dupes instead of the real content.
|
|
118
118
|
6. Run full-package `-race`, not targeted `-run` — a targeted race run misses
|
|
119
119
|
cross-test races.
|
|
120
|
-
7.
|
|
120
|
+
7. **`internal/daemon/rpc` must NEVER be run monolithically at gate time, in
|
|
121
|
+
either lane** — a bare package-level `go test [-race] ./internal/daemon/rpc`
|
|
122
|
+
starves and hits the configured `-timeout` wall in both the race and
|
|
123
|
+
non-race lane. The gate's OUTPUT must show the sharded runner actually
|
|
124
|
+
executed for RPC (`race-shard` / `package-shard --mode norace`, or
|
|
125
|
+
`scripts/test-run.sh race|norace ./internal/daemon/rpc`, or `make gate`
|
|
126
|
+
which shards both lanes automatically) — not just that some RPC test ran.
|
|
127
|
+
A bound-timeout with ZERO test failures is a FALSE RED, not a finding:
|
|
128
|
+
distinguish it from a genuine hang/deadlock via
|
|
129
|
+
`go run ./cmd/testgate classify-timeout --log <file> --shard-config <file>
|
|
130
|
+
--shard <name>` (fails closed — only BOUND-TIMEOUT exits 0; HANG and
|
|
131
|
+
UNCLASSIFIED both exit nonzero) before reporting it as a regression.
|
|
132
|
+
8. Build+test the MERGED-tree result as a SEPARATE condition — neither gate
|
|
121
133
|
currently runs a build of the actual post-merge tree; a clean pre-merge
|
|
122
134
|
build/test does not prove the merged result compiles or passes.
|
|
123
|
-
|
|
124
|
-
`--force` as the default (see item
|
|
135
|
+
9. Use `rm -r`, NEVER `rm -rf`; use plain `git worktree remove` to drop a worktree, never
|
|
136
|
+
`--force` as the default (see item 13 below). A broad `ask` rule on `rm -rf *` outranks any narrow `/tmp` allow,
|
|
125
137
|
so `rm -rf` raises a human permission prompt EVERY time regardless of path —
|
|
126
138
|
it stalls gate sub-agents mid-run waiting on a keystroke. `rm -r` runs free. Do NOT widen the ask rule
|
|
127
139
|
to work around this; that entry is the only deletion protection on the box.
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
140
|
+
10. Check the discriminator before picking a scratch path, don't assume from
|
|
141
|
+
the platform: `[ -L /tmp ]`. macOS (symlink) — create throwaway worktrees
|
|
142
|
+
under `/private/tmp`; a worktree under `/tmp` false-FAILs worktree-ancestor
|
|
143
|
+
tests via the symlink, producing a confident wrong gate result. Linux (real
|
|
144
|
+
dir) — `/tmp` is correct; `/private/tmp` may not exist there and must not
|
|
145
|
+
be created.
|
|
146
|
+
11. Your report reaches the coordinator ONLY as your final returned text.
|
|
135
147
|
Side-channel output is discarded. Put the whole verdict in the return value.
|
|
136
|
-
|
|
148
|
+
12. **NEVER run `git stash`, `git checkout`, `git reset`, or any working-tree
|
|
137
149
|
mutation in the SHARED repo.** Do read-only inspection there (`git show
|
|
138
150
|
<sha>:<path>`, `git log`, `git diff`) and do every build/test in a throwaway
|
|
139
151
|
detached worktree. `git stash` is a SINGLE SHARED STACK across every
|
|
@@ -142,7 +154,7 @@ rediscovering these the hard way:
|
|
|
142
154
|
hold uncommitted `State.md` they are actively re-authoring.
|
|
143
155
|
If you believe you must mutate the shared tree, STOP and report instead —
|
|
144
156
|
that is always a finding, never a step.
|
|
145
|
-
|
|
157
|
+
13. **TEAR DOWN YOUR THROWAWAY WORKTREE WHEN THE GATE ENDS — after two checks,
|
|
146
158
|
in this order.** A required final step, not cleanup etiquette. Each
|
|
147
159
|
abandoned worktree pins its HEAD commit against `gc` and adds a
|
|
148
160
|
`.git/worktrees` admin entry, so the object store grows monotonically.
|
|
@@ -155,7 +167,7 @@ rediscovering these the hard way:
|
|
|
155
167
|
it. Usually fine (a gate merge is reproducible by redoing it) but say so
|
|
156
168
|
in your report rather than doing it silently.
|
|
157
169
|
Then `git worktree remove <wt>` — **plain, no `--force`.** Its rc=128 refusal backstops
|
|
158
|
-
ONLY untracked/modified content (see rule
|
|
170
|
+
ONLY untracked/modified content (see rule 9 on `rm -r`) — it does NOT cover gitignored
|
|
159
171
|
content, which git removes silently with no refusal; a clean rc=0 is not proof nothing of
|
|
160
172
|
value was in there (see coordinator-assessing-agent-completion's reap procedure). `--force`
|
|
161
173
|
is a justified override only, never the default (see coordinator-merging-code §7a).
|
|
@@ -169,6 +181,21 @@ config entry (see `lens_scope_semantics` in the config), which still runs on
|
|
|
169
181
|
every diff regardless of trigger directories. If the config has no
|
|
170
182
|
`trigger_directories`, fail with guidance to run `project-hotpath-gate`.
|
|
171
183
|
|
|
184
|
+
## Invariant-ratchet lenses
|
|
185
|
+
|
|
186
|
+
A lens may declare `mandatory_test_command` (see `census_classification_coverage`
|
|
187
|
+
and `write_owner_bypass_ratchet` in `.thrum/hotpath-gate.json`) instead of
|
|
188
|
+
(or alongside) a pattern-match `question`/`required_evidence` pair. When
|
|
189
|
+
present, PASS/FAIL is the command's own exit code — never a judgment call
|
|
190
|
+
from reading the diff. This is the shape for any invariant that already has
|
|
191
|
+
a mechanical detector: wire the detector as an `always_run` lens with a
|
|
192
|
+
monotonic budget (a count that may only decrease, or an explicit
|
|
193
|
+
waiver-tagged exception), so a diff cannot silently re-widen the invariant
|
|
194
|
+
in the same change that appears to narrow it. `write_owner_bypass_ratchet`
|
|
195
|
+
is the reference instance: it fails a merge that bumps a
|
|
196
|
+
ceiling/aggregate in the ledger file without a `RATCHET-WAIVER:` tag, even
|
|
197
|
+
when the ledger-equality test alone would pass.
|
|
198
|
+
|
|
172
199
|
## Delta re-gate continuity — walk the whole branch, read the whole function
|
|
173
200
|
|
|
174
201
|
**Why:** A commit introducing a lock can be an ancestor of a delta-re-gate's
|
|
@@ -432,7 +459,7 @@ hoc test target nothing routes to. A merge that adds a `state.NewState(...)`
|
|
|
432
459
|
call site in an `internal/daemon/rpc/*_test.go` file (test-fixture setup, not a
|
|
433
460
|
handler) can leave it unclassified and desync the frozen census snapshot with
|
|
434
461
|
no gate catching it — trunk went RED post-merge on 573mo v87 for exactly this
|
|
435
|
-
reason
|
|
462
|
+
reason.
|
|
436
463
|
|
|
437
464
|
**Relationship to Lenses 1–10:** every other lens in this file is an
|
|
438
465
|
LLM-judgment pattern check — grep a pattern, read the surrounding code, apply
|
|
@@ -476,7 +503,7 @@ diff on `TestCensusRegeneration`'s brittleness with zero added
|
|
|
476
503
|
defect-prevention (`ClassificationCoverage` alone already catches the defect
|
|
477
504
|
this lens exists for). `TestCensusRegeneration`'s own staleness is tracked
|
|
478
505
|
separately (nd2j1/P1 owns the classification data; the brittle count-snapshot
|
|
479
|
-
assertion itself is
|
|
506
|
+
assertion itself is a distinct open question — "make advisory or drop the
|
|
480
507
|
raw-count assertion"), not by this lens.
|
|
481
508
|
|
|
482
509
|
**Severity:** Trigger fires and `mandatory_test_command` was not run before
|
|
@@ -520,10 +547,24 @@ otherwise leave them.
|
|
|
520
547
|
|
|
521
548
|
## If it fails
|
|
522
549
|
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
550
|
+
For a genuine hot-path/perf defect, a production-behaviour change, or a design
|
|
551
|
+
judgment call: send it back as a normal review finding — same channel as
|
|
552
|
+
dual-review output, folded into the same numbered list if dual review is still
|
|
553
|
+
open, or as its own short reply if dual review already passed. Cite the
|
|
554
|
+
offending function/file, which lens it failed, and the config value that
|
|
555
|
+
triggered the finding.
|
|
556
|
+
|
|
557
|
+
For mechanical bookkeeping only (census/shard regen via the tool, line-number
|
|
558
|
+
or key refreshes, ledger totals/ceilings where the arithmetic is derivable, a
|
|
559
|
+
per-entry waiver or citation comment recording a classification already
|
|
560
|
+
established elsewhere (never a new safety judgment of your own), a nosec
|
|
561
|
+
baseline hash for an already-reviewed pattern, comment wording, stale test
|
|
562
|
+
expectations that merely follow a landed change): fix it yourself as a
|
|
563
|
+
`GATE-AUTHORED` commit on your
|
|
564
|
+
`gate/` branch, re-run ONLY the lens(es) the fix affects, and report
|
|
565
|
+
`CLEAN-WITH-GATE-FIXES` listing each fix instead of bouncing it to the owner.
|
|
566
|
+
When unsure which bucket a finding falls in, ask the coordinator with the exact
|
|
567
|
+
diff — don't bounce reflexively.
|
|
527
568
|
|
|
528
569
|
## Efficacy tracking (v1 acceptance requirement)
|
|
529
570
|
|
|
@@ -563,6 +604,7 @@ rows.
|
|
|
563
604
|
| `FALSE_POSITIVE` | Gate flagged something correct — record which lens and why |
|
|
564
605
|
| `SLIPPED` | A wedge occurred AFTER merge that the gate did not catch |
|
|
565
606
|
| `SKIPPED_WITH_EVIDENCE` | Gate deliberately skipped (no trigger dirs), evidence cited |
|
|
607
|
+
| `CLEAN-WITH-GATE-FIXES` | Gate found only mechanical bookkeeping gaps, fixed them itself as a GATE-AUTHORED commit, re-ran only the affected checks, merge proceeded |
|
|
566
608
|
|
|
567
609
|
**Retrospective reclassification:** When triaging a new wedge or incident, grep
|
|
568
610
|
`dev-docs/hotpath-gate-efficacy.md` for the merged branch/date. If the gate
|
|
@@ -591,4 +633,4 @@ gaps and lens improvements for the next version.
|
|
|
591
633
|
first gate run, appended after each run)
|
|
592
634
|
|
|
593
635
|
<!-- THRUM-GATE: stage=skill next=review -->
|
|
594
|
-
<!-- THRUM-REVIEW: stage=skill verdict=Ready:Yes cycle=2
|
|
636
|
+
<!-- THRUM-REVIEW: stage=skill verdict=Ready:Yes cycle=2 -->
|
|
@@ -57,8 +57,11 @@ created from origin, unpushed commits can be lost or inaccessible.
|
|
|
57
57
|
|
|
58
58
|
**How to apply:** Session close protocol — push the coordination branch every
|
|
59
59
|
session end. **Read the branch name, never recall it:**
|
|
60
|
-
`
|
|
61
|
-
|
|
60
|
+
`CONFIG=.thrum/config.json; [ -f .thrum/redirect ] && CONFIG="$(cat .thrum/redirect)/config.json"; jq -r '.orchestration.merge_target' "$CONFIG"`
|
|
61
|
+
(a worktree's `.thrum/` holds only a `redirect` file pointing at the main
|
|
62
|
+
repo's `.thrum` dir, so reading `.thrum/config.json` directly there returns
|
|
63
|
+
empty) — or your project's CLAUDE.md § Branching Strategy. Push every
|
|
64
|
+
`feature/*` and `fix/*` branch too.
|
|
62
65
|
A docs/site branch pushes only when the site is ready to deploy; `main` and
|
|
63
66
|
`release/*` go through the release flow.
|
|
64
67
|
|
|
@@ -77,10 +80,7 @@ batch — the visual tree exposes inversion immediately. For bulk epic creation
|
|
|
77
80
|
(many tasks at once), spawn parallel sub-agents to issue the `bd create` calls;
|
|
78
81
|
bd is fully concurrent-safe.
|
|
79
82
|
|
|
80
|
-
> **Model tiers:** pass an explicit `model:` on every dispatch
|
|
81
|
-
> (low effort) mechanical, `sonnet` (medium effort) judgment, Opus only on
|
|
82
|
-
> operator-ask or a skill step that names it. See the
|
|
83
|
-
> `choosing-subagent-models` skill for the full policy.
|
|
83
|
+
> **Model tiers:** pass an explicit `model:` on every dispatch; choose model and effort per the `choosing-subagent-models` skill (read `runtime.role_models` at the moment of use). Never let a sub-agent inherit your own model.
|
|
84
84
|
|
|
85
85
|
## Use `bd close --suggest-next` to surface unblocked work
|
|
86
86
|
|
|
@@ -90,9 +90,11 @@ merge target before merging.
|
|
|
90
90
|
|
|
91
91
|
## 3. PASS-3 GATES — DISPATCH BOTH TO THE GATE RUNNER
|
|
92
92
|
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
93
|
+
1. Select from your team of established long-running gate agents.
|
|
94
|
+
2. Choose an available pair for hot-path and philosophy.
|
|
95
|
+
3. Tell those agents where sensitive tests must execute.
|
|
96
|
+
4. Launch a new gate agent only if the persistent set is genuinely unavailable —
|
|
97
|
+
never merely because an old candidate-specific session is stale.
|
|
96
98
|
|
|
97
99
|
Record the dispatch on your own queue (`thrum queue assign <bundle>
|
|
98
100
|
<gate-runner>`) per `coordinator-dispatching-work`'s queue step before
|
|
@@ -157,7 +159,9 @@ worktree; its sub-agents get READ-ONLY git. `checkout`/`reset`/`restore`/`stash`
|
|
|
157
159
|
`clean`/`rebase` forbidden in **every** directory for both. Give an escape hatch
|
|
158
160
|
for instrument-and-undo (a `cp` copy, or edit-then-revert) or the careful agent
|
|
159
161
|
breaches the fence doing good work. **Use `rm -r`, NEVER `rm -rf`.**
|
|
160
|
-
**Pasting a fence is not enforcing it.**
|
|
162
|
+
**Pasting a fence is not enforcing it.** This same fenced commit capability is
|
|
163
|
+
what the sanctioned `GATE-AUTHORED` mechanical-bookkeeping-fix commit uses — it
|
|
164
|
+
is not a new permission grant.
|
|
161
165
|
|
|
162
166
|
## 4. THE EXECUTED VERIFY — SCOPED TO THE CHANGE
|
|
163
167
|
|
|
@@ -175,7 +179,13 @@ executed half is **scoped to the change**:
|
|
|
175
179
|
- The `merge-base --is-ancestor` fast-forward check (§6).
|
|
176
180
|
- Any `*_tripwire_test.go` the change touches must stay green or the diff must
|
|
177
181
|
state why its premise changed; a tripwire edited to accommodate the change is a
|
|
178
|
-
finding, not a fix
|
|
182
|
+
finding, not a fix — **except** a mechanical follow-up (updating a stale
|
|
183
|
+
expectation to merely track an already-landed, already-reviewed change: a
|
|
184
|
+
renumbered line, a regenerated census key, and the like), which the gate may
|
|
185
|
+
fix itself as a `GATE-AUTHORED` commit and re-run. A tripwire edit that changes
|
|
186
|
+
what the tripwire actually *enforces* — any design or behavior judgment call —
|
|
187
|
+
stays a finding, never a self-administered fix. Keep this distinction sharp;
|
|
188
|
+
it is the highest-risk edit in this whole sequence.
|
|
179
189
|
- 🔴 **The CLASS-firing governance gates, on the merged tree — they are BLIND to
|
|
180
190
|
changed-package scoping, exactly like the `//go:embed` asset in §5.**
|
|
181
191
|
`TestFixtureDriftGuard_NoUnlistedSchemaMirrors` (`internal/schema`) and
|
|
@@ -186,18 +196,18 @@ executed half is **scoped to the change**:
|
|
|
186
196
|
it every merge: `git diff <base> <tip> | grep -iE 'CREATE TABLE|ALTER TABLE|durablelane|MirrorWriter|sanctionedMirrorWriters'`
|
|
187
197
|
— ANY hit ⇒ also run `go test ./internal/schema/ -run TestFixtureDriftGuard` and
|
|
188
198
|
`go test ./tests/gate/ -run TestEveryMirrorWriterIsSanctioned` on the MERGED tree.
|
|
189
|
-
Measured
|
|
199
|
+
Measured previously: kgi7t (`TestEveryMirrorWriterIsSanctioned`) and xo58f
|
|
190
200
|
(`TestFixtureDriftGuard`) BOTH escaped per-merge Pass-3 this way and surfaced only
|
|
191
201
|
in the daily gate as trunk-reds. The changed-package rule below is necessary, not
|
|
192
202
|
sufficient — a class gate is a property of the TREE, not the neighbourhood.
|
|
193
203
|
🔴 **Also always-run, same class-gate reasoning:**
|
|
194
204
|
`TestNoMainStateDBWriteOwnerBypasses` (`internal/testgate`; corpus 5.1
|
|
195
205
|
owner-bypass-check) — a scanner ratchet over every raw main-State-DB write
|
|
196
|
-
site, blind to changed-package scoping the same way. Measured
|
|
197
|
-
|
|
206
|
+
site, blind to changed-package scoping the same way. Measured previously:
|
|
207
|
+
a merged branch introduced a raw `r.db.ExecContext` outside
|
|
198
208
|
`safedb.WriteCoordinator` in a durable-lane cold-boot restore step; the
|
|
199
209
|
ratchet went 213→214 and gate_thrum's Pass-3 on that merge was CLEAN because
|
|
200
|
-
this test was not yet in the always-run set
|
|
210
|
+
this test was not yet in the always-run set. Run it on the
|
|
201
211
|
MERGED tree every merge: `go test ./internal/testgate/... -run
|
|
202
212
|
TestNoMainStateDBWriteOwnerBypasses`.
|
|
203
213
|
|
|
@@ -272,12 +282,16 @@ agreeing (`rev-parse origin/<branch>` and `ls-remote`). **Assert the branch TIP*
|
|
|
272
282
|
1. **The verdict artifact.** The runner saves verdicts to
|
|
273
283
|
`dev-docs/gate-reports/<ISO-week>/<date>/<gate-slug>/` and **commits them
|
|
274
284
|
BEFORE removing any worktree.** Save → commit → then destroy. A verdict that
|
|
275
|
-
exists only in a message cannot be audited later.
|
|
285
|
+
exists only in a message cannot be audited later. The artifact must be able to
|
|
286
|
+
carry a `CLEAN-WITH-GATE-FIXES` outcome, and when one applies it cites the
|
|
287
|
+
`GATE-AUTHORED` commit SHA.
|
|
276
288
|
2. **The efficacy row**, in `dev-docs/hotpath-gate-efficacy.md` — required for
|
|
277
289
|
**every** trigger-dir merge, including `SKIPPED_WITH_EVIDENCE` rows and
|
|
278
290
|
orchestrator-run merges. The log measures gate **coverage**, not just outcomes.
|
|
279
291
|
Record `FALSE_POSITIVE=0` explicitly on a clean run so the zero is a
|
|
280
|
-
measurement rather than an absence.
|
|
292
|
+
measurement rather than an absence. Same rule applies here: an
|
|
293
|
+
`outcome=CLEAN-WITH-GATE-FIXES` row cites the `GATE-AUTHORED` commit SHA when
|
|
294
|
+
one exists.
|
|
281
295
|
3. **Commit accumulated agent state** under `.thrum/agents/` along with the merge.
|
|
282
296
|
Uncommitted agent state exists in exactly one working tree, and it IS the
|
|
283
297
|
agent's memory across restarts. **Never `git add -f`.**
|
|
@@ -90,6 +90,14 @@ dispatch time — hand this list verbatim:
|
|
|
90
90
|
carrying dupes instead of the real content.
|
|
91
91
|
6. Run full-package `-race`, not targeted `-run` — a targeted race run
|
|
92
92
|
misses cross-test races that only a full-package run surfaces.
|
|
93
|
+
**Exception: `internal/daemon/rpc` must NEVER be run monolithically** — a
|
|
94
|
+
bare `go test [-race] ./internal/daemon/rpc` cannot finish inside any
|
|
95
|
+
gate-length timeout. Run it through the sharded runner (`race-shard`,
|
|
96
|
+
`scripts/test-run.sh race ./internal/daemon/rpc`, or `make gate`) and confirm
|
|
97
|
+
the gate's output shows it executed. A bound-timeout with ZERO test failures
|
|
98
|
+
is a false red: classify it with `go run ./cmd/testgate classify-timeout
|
|
99
|
+
--log <file> --shard-config <file> --shard <name>` (only BOUND-TIMEOUT exits
|
|
100
|
+
0) before reporting a regression. See item 7 of `coordinator-hotpath-merge-gate`.
|
|
93
101
|
7. Build+test the MERGED-tree result as a SEPARATE condition — neither gate
|
|
94
102
|
currently runs a build of the actual post-merge tree; a clean pre-merge
|
|
95
103
|
build/test does not prove the merged result compiles or passes.
|
|
@@ -414,13 +422,26 @@ Two cautions, so this rule does not cause its own incident:
|
|
|
414
422
|
|
|
415
423
|
## If it fails
|
|
416
424
|
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
425
|
+
For a genuine anti-pattern, Lens 10 finding, production-behaviour issue, or
|
|
426
|
+
design judgment call: send it back as a normal review finding — same channel as
|
|
427
|
+
dual-review output, folded into the same numbered list if dual review is still
|
|
428
|
+
open, or as its own short reply if dual review already passed. Cite the
|
|
429
|
+
offending function/file and which anti-pattern (`ph<N>`), meta-check, or Lens
|
|
430
|
+
10 sub-probe it failed. Do NOT block on Lens 10d findings that are genuinely
|
|
431
|
+
disclaimed substrate issues (see Lens 10 scope discipline above); DO block on
|
|
432
|
+
any Lens 10b or 10d finding on NEW work.
|
|
433
|
+
|
|
434
|
+
For mechanical bookkeeping only (census/shard regen via the tool, line-number
|
|
435
|
+
or key refreshes, ledger totals/ceilings where the arithmetic is derivable, a
|
|
436
|
+
per-entry waiver or citation comment recording a classification already
|
|
437
|
+
established elsewhere (never a new safety judgment of your own), a nosec
|
|
438
|
+
baseline hash for an already-reviewed pattern, comment wording, stale test
|
|
439
|
+
expectations that merely follow a landed change): fix it yourself as a
|
|
440
|
+
`GATE-AUTHORED` commit on your
|
|
441
|
+
`gate/` branch, re-run ONLY the check(s) the fix affects, and report
|
|
442
|
+
`CLEAN-WITH-GATE-FIXES` listing each fix instead of bouncing it to the owner.
|
|
443
|
+
When unsure which bucket a finding falls in, ask the coordinator with the exact
|
|
444
|
+
diff — don't bounce reflexively.
|
|
424
445
|
|
|
425
446
|
## See also
|
|
426
447
|
|
|
@@ -67,11 +67,11 @@ Spawn **one** top-level agent, `model: "fable"`.
|
|
|
67
67
|
|
|
68
68
|
This is not a self-granted exception; it is the single deliberate use of
|
|
69
69
|
Fable anywhere in the system, kept as an independent check-and-balance
|
|
70
|
-
against everything else, which runs on the
|
|
70
|
+
against everything else, which runs on the standard tier ceiling (see `choosing-subagent-models`).
|
|
71
71
|
Fable — a distinct, more-capable model — is the right instrument specifically
|
|
72
72
|
because a holistic cross-plan reconcile (does seam X mean the same thing in
|
|
73
|
-
plan ① as in plan ②?) benefits from a different model than the
|
|
74
|
-
per-plan reviews, a form of cross-document judgment a
|
|
73
|
+
plan ① as in plan ②?) benefits from a different model than the standard-tier
|
|
74
|
+
per-plan reviews, a form of cross-document judgment a standard-tier pass has
|
|
75
75
|
been shown to miss.
|
|
76
76
|
|
|
77
77
|
This is the **one named, cited exception** to the standing "never SELECT
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: coordinator-rolling-out-to-nodes
|
|
3
|
+
description: "Use when rolling a build across all nodes — fleet deploy, rolling the nodes, deploy everywhere, ship to every node, canary then the rest, promote the canary, staged rollout, deploying after a merge batch. Orchestrates the staged order and the promotion gate between stages, and delegates each node to the coordinator-deploying-a-node runbook. Load this BEFORE dispatching any node."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Coordinator: Rolling Out a Build Across All Nodes — staged order, canary, promotion gate
|
|
7
|
+
|
|
8
|
+
> This skill is the ORCHESTRATION layer. It does not replace
|
|
9
|
+
> `coordinator-deploying-a-node` — it calls it, once per node.
|
|
10
|
+
|
|
11
|
+
## 0. Universal invariants (read first — these outrank everything below)
|
|
12
|
+
|
|
13
|
+
These hold for every deployment regardless of project or topology. The adapter supplies *how*, these define *what must be true*.
|
|
14
|
+
|
|
15
|
+
1. **Build once; distribute identical bytes.** One pinned immutable artifact (SHA/version/checksum triple) is built once at one SHA on the build host; every node installs from that same artifact. No node re-derives it. Pin exact SHA, never tip. Adapter: `fleet.deployment.thrum.artifact` or `fleet.deployment.products.<id>.artifact`.
|
|
16
|
+
2. **Target-local mutation.** The node-local coordinator performs the mutation (install/migrate/restart) locally on its own host. Never drive a node's build/install/restart from another host. The orchestrator dispatches and gates; the node executes.
|
|
17
|
+
3. **Immutable identifier + checksum.** Artifact is identified by SHA + version + checksum; verify BEFORE install that the delivered bundle is the one the orchestrator built. Adapter: `artifact.checksum`, `artifact.version_file`, `artifact.commit_file`.
|
|
18
|
+
4. **Canary evidence does not transfer.** A canary pass proves the *build* is sound; it does not prove any other node's *state* is sound. Every node takes its own BEFORE readings.
|
|
19
|
+
5. **Verify by observed effect, never exit code.** Install/restart success is proven by post-mutation observation (version equals pin, NEW PID, service status reports pin, migration moved), not by installer exit 0. Adapter: `install.verify_by_effect`, `health.probes`.
|
|
20
|
+
6. **Capture before/after.** For every node, capture BEFORE state (current version/state, health counters) before mutation and AFTER state after; compare.
|
|
21
|
+
7. **Account for every target.** Roster completeness: enumerate topology from config, disposition each node (rolled on pin / deferred with owner sign-off + notify local coordinator), and never normalize unmeasured rows.
|
|
22
|
+
|
|
23
|
+
Additional parameterized invariants (skill-owned, adapter-valued):
|
|
24
|
+
|
|
25
|
+
- Pull-current first (ff-only) before reasoning about state. Adapter: `preflight.pull_command`.
|
|
26
|
+
- Backup before touching anything; sizing via adapter formula. Adapter: `backup.command`, `backup.sizing_formula`.
|
|
27
|
+
- Never bypass the storage engine's file ownership. Adapter: `storage_access.via`, `storage_access.forbidden`.
|
|
28
|
+
- Install-before-restart ordering; re-render between install and restart. Adapter: `templates.order`.
|
|
29
|
+
- Convergence wait from mechanism, not impatience. Adapter: `health.convergence_wait`.
|
|
30
|
+
|
|
31
|
+
## 1. Enumerate topology (from adapter — never hard-code)
|
|
32
|
+
|
|
33
|
+
Read the project's deployment adapter inside `.thrum/config.json` (`fleet.deployment`; human-readable reference at the deployment-adapter reference). Never hard-code topology or commands in the dispatch or in this skill — read them from that single source.
|
|
34
|
+
|
|
35
|
+
- For Thrum fleet plus connected daemons: `fleet.deployment.thrum.topology` (`canary`, `waves`, `excluded`, `connected_daemons`, `identity_resolution`).
|
|
36
|
+
- For a customer product `<id>`: `fleet.deployment.products.<id>.topology` (same shape, per-product `canary`/`nodes`/`waves`).
|
|
37
|
+
|
|
38
|
+
Resolve each node's `daemon_id` / `ssh_alias` / `coordinator` via the adapter's `identity_resolution` method (daemon_id join, not name). If topology is empty, STOP — do not invent nodes.
|
|
39
|
+
|
|
40
|
+
**Why every node carries an `ssh_alias`:** so the rolling coordinator can reach the node directly when the messaging or pane-mirror path fails. The main case is an agent stuck on a permission prompt or dialog. Read the pane with `ssh <alias> tmux capture-pane -p -t <session>`, then answer a prompt you recognise as safe with `tmux send-keys`. Never approve blindly, and never pick a "don't ask again" option. Re-capture afterwards to confirm the pane moved on. A node flagged `blocked-on-human` gets unblocked this way in the same turn; don't wait on it or reorder the roll around it. The alias is for pane interaction and artifact distribution only. The node still builds, installs and restarts itself, through its own coordinator.
|
|
41
|
+
|
|
42
|
+
> Example — Thrum project: topology canary and waves are defined in `topology.canary` / `topology.waves` (seven boxes, one excluded), connected daemons per `topology.connected_daemons`. Build host and build command are adapter values, not skill literals.
|
|
43
|
+
|
|
44
|
+
## 2. Pin the artifact (build once at SHA/version/checksum)
|
|
45
|
+
|
|
46
|
+
Derive the immutable pin on the build host (adapter `artifact.build_host`): current `HEAD` SHA, version, and checksum. When multiple related artifacts (e.g., core plus connected-repo bundles) are involved, pin each immutably and distribute the same set to every relevant node. Announce the pin in the dispatch so every node can verify it.
|
|
47
|
+
|
|
48
|
+
Artifact identity lives in `artifact.output`, `artifact.version_file`, `artifact.commit_file`, `artifact.checksum`. Never use a floating tag like tip; always the pinned SHA.
|
|
49
|
+
|
|
50
|
+
## 3. Preflight (control-plane current, forward-only, backup sizing via adapter)
|
|
51
|
+
|
|
52
|
+
- Verify control-plane checkout is current: `preflight.pull_command` (must be ff-only; if not fast-forwardable, STOP and surface).
|
|
53
|
+
- Forward-only check per node: `preflight.forward_only_check` — prove the pin is an ancestor of the node's current SHA (plus a control that must FAIL, e.g., an invented SHA, to prove the check discriminates).
|
|
54
|
+
- Backup sizing via adapter: `backup.sizing_formula` — compute `free_at_completion` per the formula before running `backup.command`; if the projected peak exceeds the safe threshold, STOP and escalate.
|
|
55
|
+
|
|
56
|
+
## 4. Build once → distribute identical artifact (mechanism via adapter)
|
|
57
|
+
|
|
58
|
+
Build once per artifact on its `artifact.build_host` via `artifact.build_command` → verify the produced bundle's version/checksum/commit sidecar → distribute the identical bytes to every node's inbound drop path (`artifact.distribution.drop_path`) via the adapter's `artifact.distribution.mechanisms` (e.g., drop-folder or copy). No node re-derives or rebuilds. Verify BEFORE unpack that the delivered bundle matches the pin.
|
|
59
|
+
|
|
60
|
+
> Example — Thrum project: build once via the adapter's `artifact.build_command` on the adapter's `artifact.build_host`, distribute via the adapter's `artifact.distribution.mechanisms` to `artifact.distribution.drop_path`, bundle carries sidecars verified per adapter.
|
|
61
|
+
|
|
62
|
+
## 5. The staged order — canary, waves, prepare-and-hold, promotion gate
|
|
63
|
+
|
|
64
|
+
- **Canary first, alone.** One node (adapter `topology.canary`) rolls first. Its health must be observed green before any wave proceeds.
|
|
65
|
+
- **Waves with coordinated dependency/order/rollback.** Subsequent waves follow `topology.waves`. When related daemons share ordering (e.g., connected-repo daemons that must restart together), respect the adapter-defined dependency and restart order; a failure in one dependent daemon triggers coordinated rollback per adapter `rollback` / `backup.restore_command`.
|
|
66
|
+
- **Prepare-and-hold + release BY NAME.** If the mechanism supports it, prepare all nodes in a wave (acquire + verify artifact, take BEFORE snapshot, but do not restart) then release by explicit node name, not by broadcast.
|
|
67
|
+
- **Promotion gate.** After each stage, verify by observed effect on that node (see node skill) before promoting. On no-go, halt the rollout and roll back the affected node(s) per adapter.
|
|
68
|
+
|
|
69
|
+
## 6. Dispatching a node — what the message must carry
|
|
70
|
+
|
|
71
|
+
Dispatch each node to its own local coordinator (resolved via `identity_resolution`), never via remote shell for the mutation itself. The dispatch message must carry:
|
|
72
|
+
|
|
73
|
+
- Pinned artifact identifier (SHA/version/checksum) and drop-path location.
|
|
74
|
+
- Which adapter section to use (`fleet.deployment.thrum` vs `fleet.deployment.products.<id>`).
|
|
75
|
+
- Expected pin for verification and the BEFORE-snapshot request.
|
|
76
|
+
- Hold/release semantics if using prepare-and-hold.
|
|
77
|
+
|
|
78
|
+
The node's local runbook is `coordinator-deploying-a-node` — reference that skill by name in the dispatch.
|
|
79
|
+
|
|
80
|
+
## 7. After the last node — roster completeness, phase reconciliation, test gate
|
|
81
|
+
|
|
82
|
+
- Enumerate the topology again from the same adapter source and disposition every entry (rolled on pin / deferred with owner sign-off + notify local coordinator). Missing or unmeasured rows are not normalized.
|
|
83
|
+
- Reconcile registry/phase: verify every daemon reports the pin (adapter `health.probes` + `install.verify_by_effect`), and that agent phases have converged.
|
|
84
|
+
- Run the project's gate that is appropriate for the change (e.g., `make gate` or equivalent) before declaring success.
|
|
85
|
+
|
|
86
|
+
## 8. Holds and red flags
|
|
87
|
+
|
|
88
|
+
- If `fleet.deployment` is absent or `fleet.deployment.products` and `fleet.deployment.thrum` are both empty, STOP — no topology is defined.
|
|
89
|
+
- If the control-plane pull is not fast-forwardable, STOP.
|
|
90
|
+
- If `fleet.deployment.thrum.products` exists (vestigial), STOP — products live only at `fleet.deployment.products`.
|
|
91
|
+
- If a node reports success by exit code alone without observed-effect verification, treat as not-verified.
|
|
92
|
+
- Never wrap a service restart in a blind timeout on the node (see node skill `service.never_timeout` and `migration.signal_rules`).
|
|
@@ -106,8 +106,8 @@ outlive its author and later be enforced as if it were the human's mandate.
|
|
|
106
106
|
through a standing orchestrator.**
|
|
107
107
|
|
|
108
108
|
The reason, in one line: orchestrators own the implementer lifecycle and the
|
|
109
|
-
`--model
|
|
110
|
-
|
|
109
|
+
`--model` pin (from `runtime.role_models`, per `choosing-subagent-models`); standing one up yourself risks the implementer inheriting
|
|
110
|
+
your own model. Your lane is dispatch-decision + gate + merge. Full dispatch mechanics
|
|
111
111
|
live in **`coordinator-dispatching-work`**.
|
|
112
112
|
|
|
113
113
|
**Recovery:** if an implementer was ever stood up directly, **tear it down
|
|
@@ -117,7 +117,7 @@ coordinator-spawned implementer — it may carry the wrong model.
|
|
|
117
117
|
| Rationalization | Reality |
|
|
118
118
|
| -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
119
119
|
| "I'll just stand up the implementer myself, it's faster." | **WRONG** — route through a standing orchestrator. |
|
|
120
|
-
| "The orchestrator can just adopt the implementer I already spawned." | **WRONG** — it may carry
|
|
120
|
+
| "The orchestrator can just adopt the implementer I already spawned." | **WRONG** — it may carry a costly model. Tear it down first; let the orchestrator spawn its own with the pinned model. |
|
|
121
121
|
|
|
122
122
|
## Coordinator-run Q&A vs escalate
|
|
123
123
|
|