loki-mode 9.50.2 → 9.50.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SKILL.md +2 -2
- package/VERSION +1 -1
- package/autonomy/lib/proof-generator.py +7 -5
- package/autonomy/run.sh +264 -6
- package/dashboard/__init__.py +1 -1
- package/docs/COMPETITIVE-INTEL-2026-09.md +444 -0
- package/docs/INSTALLATION.md +1 -1
- package/docs/ONE-RUN-AUDIT.md +298 -0
- package/loki-ts/dist/loki.js +4 -4
- package/mcp/__init__.py +1 -1
- package/package.json +1 -1
- package/plugins/loki-mode/.claude-plugin/plugin.json +1 -1
- package/skills/00-index.md +15 -0
- package/skills/factory-operations.md +508 -0
- package/skills/release-cadence.md +382 -0
|
@@ -0,0 +1,508 @@
|
|
|
1
|
+
# Factory Operations (skill module)
|
|
2
|
+
|
|
3
|
+
**Purpose:** the operating system for running multiple agent teams against this
|
|
4
|
+
repo at once without losing work, without false greens, and without the teams
|
|
5
|
+
blocking each other.
|
|
6
|
+
|
|
7
|
+
**Scope:** this module governs COORDINATION (who decides what, who owns which
|
|
8
|
+
file, how many things run at once, what a blocked team does). It does not
|
|
9
|
+
restate the release procedure, which lives in `skills/release-cadence.md` and,
|
|
10
|
+
authoritatively, in the CLAUDE.md "Release Workflow" section.
|
|
11
|
+
|
|
12
|
+
**Relationship to `skills/sdlc-fleet.md`:** that module defines the six roles
|
|
13
|
+
for ONE change. This module defines what happens when several such fleets run
|
|
14
|
+
concurrently. Read sdlc-fleet.md for the roles; read this one for the traffic
|
|
15
|
+
rules between them.
|
|
16
|
+
|
|
17
|
+
**REGISTRATION REQUIRED (blocking).** `tests/test-skill-doc-accuracy.sh:126-133`
|
|
18
|
+
loops every `skills/*.md` and FAILS any file that `skills/00-index.md` cannot
|
|
19
|
+
route to. Measured on creation: the suite went from 57 passed / 0 failed to 57
|
|
20
|
+
passed / 2 failed, both failures being this file and `release-cadence.md`.
|
|
21
|
+
Until a routing row is added to `00-index.md`, the gate is red. This is the
|
|
22
|
+
cross-cutting-registration failure class this repo already records: a new
|
|
23
|
+
artifact needs ALL its registrations, not just its own file.
|
|
24
|
+
|
|
25
|
+
**The spine of this module, read section 7 first.** Every rule below exists to
|
|
26
|
+
stop ONE failure shape: **a measurement that cannot see the thing it claims to
|
|
27
|
+
measure.** It is worse than no measurement, because it emits a log line that
|
|
28
|
+
looks exactly like evidence. SIX instances were observed in a single session on
|
|
29
|
+
2026-09-14, in six different tools: a staleness guard that read the wrong
|
|
30
|
+
registry row, a verification loop whose comparison errored on every file while
|
|
31
|
+
printing OK, a `grep` over a built artifact that silently found nothing because
|
|
32
|
+
the file contains NUL bytes, a 0/3 benchmark that never reached the tool, a
|
|
33
|
+
`git push` that reported exit 0 while origin never moved, and a shell glob
|
|
34
|
+
failure that returned rc=0 without ever running the search.
|
|
35
|
+
Section 7 carries the evidence table and the three rules that fall out. The
|
|
36
|
+
coordination rules in sections 1 through 6 are what keep a factory of parallel
|
|
37
|
+
agents from manufacturing these at scale.
|
|
38
|
+
|
|
39
|
+
A note on the evidence in this file: every mechanism claim carries a
|
|
40
|
+
`file:line` or a measured number with its date. Line numbers drift, so each
|
|
41
|
+
citation also names the label, function, or string to grep for. Where a number
|
|
42
|
+
is quoted it is the number already measured and recorded in this repo; nothing
|
|
43
|
+
here is estimated.
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## 1. Roles: what each one DECIDES vs what it ADVISES
|
|
48
|
+
|
|
49
|
+
The distinction matters because a role that only advises must never be able to
|
|
50
|
+
stop work, and a role that decides must never be overridden by consensus. Most
|
|
51
|
+
coordination failure in this repo has been a role acting outside its column.
|
|
52
|
+
|
|
53
|
+
| Role | DECIDES (binding) | ADVISES (non-binding) |
|
|
54
|
+
|---|---|---|
|
|
55
|
+
| **Chief of Staff** (the integrator session) | Work partition and file ownership; WIP admission; when to escalate to the founder; integration of parallel outputs | Implementation approach inside a team's slice |
|
|
56
|
+
| **PM** (per workstream) | Scope and acceptance criteria for its workstream; accepts or rejects the delivered slice against those criteria | Sequencing relative to other workstreams (the CoS decides that) |
|
|
57
|
+
| **Dev Fleet** (3-5 engineers, parallel) | Implementation detail within an owned file set | Scope changes; anything touching a file it does not own |
|
|
58
|
+
| **SDET** | Whether a test actually exercises the acceptance criterion | Whether the feature ships (that is the council's call) |
|
|
59
|
+
| **Council Reviewers** (3, parallel) | SHIP / NO-SHIP. Unanimous APPROVE required | Style preferences, which are findings, not vetoes |
|
|
60
|
+
| **Release Captain** | When the VERSION bump is pushed, and sole ownership of main during a release window | Content of the release (the PM and council settled that already) |
|
|
61
|
+
|
|
62
|
+
Three rules that follow, each of which this repo has broken and paid for:
|
|
63
|
+
|
|
64
|
+
**The Chief of Staff does not write code in a file an agent owns.** Not a
|
|
65
|
+
one-line fix, not while the agent looks idle. An agent holds the file's prior
|
|
66
|
+
content and writes whole regions from its own buffer; an edit made underneath
|
|
67
|
+
it vanishes with no conflict and no error. Measured on 2026-08-08: a
|
|
68
|
+
`--allow-unanchored` flag added by hand to `autonomy/loki` while an agent was
|
|
69
|
+
running was silently discarded, and `grep -c` returned 0 for code that had
|
|
70
|
+
just been verified with `bash -n`. `Edit` warns in the reverse direction only;
|
|
71
|
+
nothing protects the lead. If a requirement has not landed after two asks, the
|
|
72
|
+
fix is a third message carrying a measured number and the exact distinction
|
|
73
|
+
wanted, not an edit.
|
|
74
|
+
|
|
75
|
+
**The council decides ship, and 2-of-3 is never acceptable.** This is
|
|
76
|
+
inherited verbatim from `skills/sdlc-fleet.md:83` and CLAUDE.md's standing
|
|
77
|
+
fleet pattern. A CONCERN or REJECT sends the integrator to the source to
|
|
78
|
+
validate it; a valid concern means fix and RE-RUN the entire council, not
|
|
79
|
+
patch and count votes again.
|
|
80
|
+
|
|
81
|
+
**The Release Captain owns main during a release window.** See section 5.
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## 2. Partition: how two teams never edit the same file
|
|
86
|
+
|
|
87
|
+
### 2.1 Per-file ownership is the primary mechanism
|
|
88
|
+
|
|
89
|
+
Every spawn brief states two things explicitly:
|
|
90
|
+
|
|
91
|
+
1. the exact file set this agent OWNS and may write, and
|
|
92
|
+
2. the file sets the OTHER concurrent agents own.
|
|
93
|
+
|
|
94
|
+
Point 2 is not courtesy. In the 2026-08-08 session the briefs carried per-file
|
|
95
|
+
ownership and the only collision that occurred was the one the lead created by
|
|
96
|
+
editing an owned file directly.
|
|
97
|
+
|
|
98
|
+
An agent that needs a change in a file it does not own does not make the
|
|
99
|
+
change. It reports the required change to the Chief of Staff, who routes it to
|
|
100
|
+
the owner. A cross-cutting change with no single owner is a signal the
|
|
101
|
+
partition is wrong; re-partition rather than letting two agents negotiate.
|
|
102
|
+
|
|
103
|
+
### 2.2 Worktree isolation is the EXCEPTION, not the default
|
|
104
|
+
|
|
105
|
+
The generic advice is "give each team a worktree." In this repo that is wrong
|
|
106
|
+
most of the time, and confirmed wrong twice on 2026-07-15.
|
|
107
|
+
|
|
108
|
+
Worktree agents cut their tree from the repo's DEFAULT branch (main),
|
|
109
|
+
regardless of your committed feature HEAD. With v8 work committed to
|
|
110
|
+
`feature/v8-agent-sdk` at HEAD `d3cd833c` (VERSION 8.0.0), every worktree agent
|
|
111
|
+
still got a tree at `08a09b19` (main, VERSION 7.129.3). Files that existed only
|
|
112
|
+
on the feature branch "did not exist" to those agents, and their diffs were
|
|
113
|
+
against main's version of shared files.
|
|
114
|
+
|
|
115
|
+
The rule:
|
|
116
|
+
|
|
117
|
+
- Feature-branch work that is ahead of main: do NOT use worktree isolation.
|
|
118
|
+
Run agents without isolation against the live tree, partitioned by file
|
|
119
|
+
ownership.
|
|
120
|
+
- Agents writing DISJOINT NEW files: no isolation needed; there is nothing to
|
|
121
|
+
collide.
|
|
122
|
+
- Same-file parallel mutation, AND your base == main: worktree isolation is
|
|
123
|
+
correct and is what it is for.
|
|
124
|
+
|
|
125
|
+
After any worktree reconcile, re-run parse checks (`bash -n`, `ast.parse`,
|
|
126
|
+
typecheck) on every reconciled file and confirm your own earlier edits to those
|
|
127
|
+
files survived the patch. A patch to a nearby hunk can silently coexist with or
|
|
128
|
+
clobber them.
|
|
129
|
+
|
|
130
|
+
### 2.3 A red test during a review window may be someone else's mutation
|
|
131
|
+
|
|
132
|
+
Council and mutation-testing agents work by copy -> mutate -> assert red ->
|
|
133
|
+
restore. While they run, the tree transiently contains deliberate breakage.
|
|
134
|
+
A suite that goes red mid-cycle on a file an agent is reviewing is that agent's
|
|
135
|
+
proof-of-catch, not a regression. Do not act on it, and do not run those tests,
|
|
136
|
+
until the agents complete. A `<system-reminder>` reporting such a file as
|
|
137
|
+
"modified ... intentional" has been observed reporting an agent's transient
|
|
138
|
+
mutation that was reverted seconds later.
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## 3. WIP limit
|
|
143
|
+
|
|
144
|
+
**The limit: one verification gate at a time, and at most 3 concurrent
|
|
145
|
+
implementation streams.**
|
|
146
|
+
|
|
147
|
+
### 3.1 Why one gate, not more
|
|
148
|
+
|
|
149
|
+
This is the non-obvious half, and it is not about agent capacity. The gate
|
|
150
|
+
refuses to run concurrently by design. `scripts/local-ci.sh:69-80` (grep
|
|
151
|
+
`SINGLE INSTANCE`) records four suites measured on 2026-07-30 that FAILED under
|
|
152
|
+
concurrent load and passed cleanly in isolation:
|
|
153
|
+
|
|
154
|
+
| Suite | Under load | Alone |
|
|
155
|
+
|---|---|---|
|
|
156
|
+
| `tests/cli/test-alias-forwarding.sh` | FAIL | 213 passed, 0 failed |
|
|
157
|
+
| `tests/test-plan-command.sh` | FAIL | 27 passed, 0 failed |
|
|
158
|
+
| `tests/test-proven-pr-receipt.sh` | FAIL | 14 passed, 0 failed |
|
|
159
|
+
| `tests/test-heal-assess-readiness.sh` | FAIL | 8 passed, 0 failed |
|
|
160
|
+
|
|
161
|
+
The script's own reasoning is the load-bearing part: "Hours were spent treating
|
|
162
|
+
those as defects. Worse, a phantom failure trains the reader to distrust the
|
|
163
|
+
gate, which is exactly how a REAL failure gets waved through. Refusing to start
|
|
164
|
+
is cheaper than a verdict nobody believes."
|
|
165
|
+
|
|
166
|
+
So WIP is bounded by VERIFICATION SERIALIZATION, not by how many agents can be
|
|
167
|
+
spawned. Throughput above that bound is not throughput; it is a queue of
|
|
168
|
+
unverifiable work plus a gate nobody believes.
|
|
169
|
+
|
|
170
|
+
The same effect has been seen at the suite level: a provider-backed review
|
|
171
|
+
suite once sat 28 minutes at 0.06s CPU under overlap, then passed 8/8 in
|
|
172
|
+
isolation (`scripts/local-ci.sh:1138-1140`, the `LOCAL_CI_SERIAL` fallback
|
|
173
|
+
comment).
|
|
174
|
+
|
|
175
|
+
### 3.2 Why 3 implementation streams
|
|
176
|
+
|
|
177
|
+
`LOKI_MAX_PARALLEL_SESSIONS` defaults to 3 (`skills/parallel-workflows.md:180-181`,
|
|
178
|
+
grep `caps parallel Claude sessions`).
|
|
179
|
+
The honest ceiling documented there is "dozens, not thousands", for a reason
|
|
180
|
+
that is a coordination fact rather than a hardware one: "The 3-reviewer council
|
|
181
|
+
is a serialization point: every non-trivial change funnels through blind review
|
|
182
|
+
before merge, so review throughput, not spawn count, sets the end-to-end pace"
|
|
183
|
+
(`skills/parallel-workflows.md:216-218`).
|
|
184
|
+
|
|
185
|
+
Spawning a fourth stream does not produce a fourth shipped change. It produces
|
|
186
|
+
a third change that waits longer, because the council and the gate are both
|
|
187
|
+
serial. Raise the stream count only after measuring that review throughput has
|
|
188
|
+
risen, never in the hope that it will.
|
|
189
|
+
|
|
190
|
+
Opt-in dynamic concurrency (`LOKI_DYNAMIC_CONCURRENCY=1`) only ever reduces the
|
|
191
|
+
cap under CPU or memory pressure; it never raises it above the configured
|
|
192
|
+
ceiling.
|
|
193
|
+
|
|
194
|
+
### 3.3 The admission rule
|
|
195
|
+
|
|
196
|
+
A new stream is admitted only when all of these hold:
|
|
197
|
+
|
|
198
|
+
- fewer than 3 implementation streams are active,
|
|
199
|
+
- no gate run is in flight,
|
|
200
|
+
- the new stream's file set is disjoint from every active stream's file set,
|
|
201
|
+
- main is not inside a release window (section 5).
|
|
202
|
+
|
|
203
|
+
If any fails, the work is QUEUED, not started. A queued item with a written
|
|
204
|
+
owner and file set is cheap. A started item that collides is expensive twice:
|
|
205
|
+
once to discover, once to reconstruct.
|
|
206
|
+
|
|
207
|
+
---
|
|
208
|
+
|
|
209
|
+
## 4. Escalation: what a blocked team does instead of waiting
|
|
210
|
+
|
|
211
|
+
Waiting is the failure mode. A blocked agent that idles burns the WIP slot that
|
|
212
|
+
bounds the whole factory. The rule is: **every block converts into either a
|
|
213
|
+
reroute or a written escalation within one turn.**
|
|
214
|
+
|
|
215
|
+
Route by the KIND of block:
|
|
216
|
+
|
|
217
|
+
| Block | Action | Never |
|
|
218
|
+
|---|---|---|
|
|
219
|
+
| Needs an edit in a file another agent owns | Report the required change to the Chief of Staff with the exact diff intent; continue on the rest of the owned slice | Edit it yourself |
|
|
220
|
+
| Ambiguous scope | Escalate to the PM, who uses `AskUserQuestion` to lock it. Per `skills/sdlc-fleet.md:40`, NEVER guess at scope | Pick an interpretation and build |
|
|
221
|
+
| Gate is red | Read the actual assertion and the whole guard above it before bisecting; the assertion type names the failing line | Call it environmental |
|
|
222
|
+
| Gate red and unreproducible | Re-run on the IDENTICAL SHA first. A rerun proves flake AND diff-innocence for free | Edit the test to make it pass |
|
|
223
|
+
| Blocked on a gate run held by another stream | Queue behind it and do useful non-gate work (plan the next item, write the test, draft the brief) | Start a second concurrent gate run |
|
|
224
|
+
| Dependency on an unfinished stream | Stub the seam, write the test against the seam, hand back | Block until the other stream lands |
|
|
225
|
+
| Red that nobody in the repo can clear (external state, credentials) | Escalate to make the guard ADVISORY: report and exit 0 on the push path, hard exit behind a scheduled-job env var. See section 7.1 | Leave a permanent red on every push |
|
|
226
|
+
| A check that passed but you cannot show how it could fail | Treat as UNVERIFIED and add a positive control before reporting. See section 7 | Report the green |
|
|
227
|
+
| An operation that stalls (push, build, gate, agent) | MEASURE whether it is alive first: process table, output mtime, downstream state. See section 4.1 | Retry it, or abandon it |
|
|
228
|
+
|
|
229
|
+
### 4.1 A stall is a measurement problem, not a patience problem
|
|
230
|
+
|
|
231
|
+
**When an operation stalls, the first action is to measure whether it is ALIVE,
|
|
232
|
+
before retrying or abandoning.** A retry on a wedged operation produces a second
|
|
233
|
+
wedged operation and destroys the evidence of the first. Abandoning it discards
|
|
234
|
+
the same evidence more quietly.
|
|
235
|
+
|
|
236
|
+
Three cheap measurements, in order:
|
|
237
|
+
|
|
238
|
+
1. **Process table.** `ps -eo pid,etime,command | grep <thing>`. This answers
|
|
239
|
+
alive-or-dead, and `etime` answers how long, which is the number that decides
|
|
240
|
+
whether it is slow or wedged. Check ELAPSED before killing anything: a
|
|
241
|
+
name-based match has already killed a 33-hour-old MCP server in this repo
|
|
242
|
+
when the intended target was a 3-minute test.
|
|
243
|
+
2. **Output mtime.** A process that is working usually writes. A log whose mtime
|
|
244
|
+
is advancing is alive; one frozen for minutes while the process lives is
|
|
245
|
+
blocked, which is a different fault with a different fix.
|
|
246
|
+
3. **Downstream state.** Ask the destination, not your local view. This is the
|
|
247
|
+
only one of the three that can confirm an outward-facing action; see
|
|
248
|
+
section 4.2. For a push,
|
|
249
|
+
`git ls-remote origin main` asks the server; `git rev-parse origin/main` only
|
|
250
|
+
reads a local remote-tracking ref that updates on fetch, so it reports the
|
|
251
|
+
old commit indefinitely and looks exactly like a failed push.
|
|
252
|
+
|
|
253
|
+
Worked example, measured live on 2026-09-14. A release push to origin/main
|
|
254
|
+
appeared to hang for several minutes with origin still at the old commit. Two
|
|
255
|
+
documented causes were suspected: the shared `.git/config` identity placeholder,
|
|
256
|
+
and `core.bare` spontaneously flipping true. Measurement ruled out BOTH in three
|
|
257
|
+
commands:
|
|
258
|
+
|
|
259
|
+
```
|
|
260
|
+
git config --get core.bare -> false (not flipped)
|
|
261
|
+
git config --get user.email -> lokeshmure@live.com (what the hook demands)
|
|
262
|
+
ps -eo pid,etime,command | grep -E "git push|pytest" -> no match
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
No process was alive. The push was not stalled; it was not running. The stale
|
|
266
|
+
`origin/main` was a local tracking ref, not the server's state. Had the first
|
|
267
|
+
action been a retry, it would have produced a second push against an undiagnosed
|
|
268
|
+
repo and erased the evidence that the first one had already exited.
|
|
269
|
+
|
|
270
|
+
Know the real timings before calling something hung, because the documented
|
|
271
|
+
stall has a specific shape and this was not it. `.githooks/pre-push:37-42`
|
|
272
|
+
checks identity and fails FAST with a `FAIL:` line. The expensive part is the
|
|
273
|
+
pytest run, and `.githooks/pre-push:90-93` scopes it to files changed since
|
|
274
|
+
`merge-base HEAD origin/main` and skips it entirely when nothing relevant
|
|
275
|
+
changed. The historical 220s "hang" happened because the hook ran the full
|
|
276
|
+
3160-test suite BEFORE printing its identity verdict. `git ls-remote` returning
|
|
277
|
+
instantly while a push hangs is the tell that it is the hook, not the network.
|
|
278
|
+
|
|
279
|
+
Elsewhere: macOS Bun CI jobs legitimately take about 22 minutes, so compare
|
|
280
|
+
against a prior successful run's `startedAt -> completedAt` before calling one
|
|
281
|
+
hung.
|
|
282
|
+
|
|
283
|
+
### 4.2 An exit code reports a PROCESS, never the WORLD
|
|
284
|
+
|
|
285
|
+
**Never report an outward-facing action as done based on an exit code.** Verify
|
|
286
|
+
that the remote state actually changed:
|
|
287
|
+
|
|
288
|
+
| Action | Do NOT trust | Assert instead |
|
|
289
|
+
|---|---|---|
|
|
290
|
+
| `git push` | the command's exit code | `git ls-remote origin <branch>`, compare to local HEAD |
|
|
291
|
+
| npm publish | a green publish job | `npm view <pkg> version`, then `gitHead` ancestry (section 5 of `release-cadence.md`) |
|
|
292
|
+
| a release | the workflow's conclusion | the artifact exists and is downloadable |
|
|
293
|
+
| any registry write | the publisher's exit code | query the registry for the value you wrote |
|
|
294
|
+
|
|
295
|
+
An exit code answers "did this process end cleanly". Whether the world changed
|
|
296
|
+
is a different question, and only the destination can answer it. Every
|
|
297
|
+
outward-facing step must assert the second one.
|
|
298
|
+
|
|
299
|
+
Measured 2026-09-14, and this is instance 5 of the section 7 table. A release
|
|
300
|
+
push reported `[exited with code 0]` and the release was nearly announced on
|
|
301
|
+
that basis. Origin had not moved. The push was wrapped in `timeout 180`, the
|
|
302
|
+
repo's pre-push hook runs pytest, and the timeout killed the hook mid-run while
|
|
303
|
+
the WRAPPER reported success. The only thing that caught it was fetching origin
|
|
304
|
+
and comparing SHAs.
|
|
305
|
+
|
|
306
|
+
Confirmed on this checkout minutes later: `git ls-remote origin main` returned
|
|
307
|
+
`bb73addb` while local HEAD was `d1e4e6b7`. The server is the authority; a
|
|
308
|
+
`git rev-parse origin/main` would have agreed with the stale value indefinitely,
|
|
309
|
+
because a remote-tracking ref only moves on fetch.
|
|
310
|
+
|
|
311
|
+
The same shape has been recorded here before on the publish side: a publisher
|
|
312
|
+
that exits 0 on an authentication failure, so a green step meant nothing. Two
|
|
313
|
+
different tools, same hour, same lesson.
|
|
314
|
+
|
|
315
|
+
**Corollary, the timeout rule: never wrap an operation in a timeout shorter than
|
|
316
|
+
its own internal gate.** The 180s wrapper sat in front of a hook whose pytest
|
|
317
|
+
run alone is ~128s, plus transfer. That converted a working safety gate into a
|
|
318
|
+
silent failure, which is strictly worse than having no wrapper: the gate no
|
|
319
|
+
longer protects, and its failure is now invisible. Either allow the gate its
|
|
320
|
+
full budget, or do not wrap it at all. Before choosing a timeout, measure what
|
|
321
|
+
the operation's own gates cost.
|
|
322
|
+
|
|
323
|
+
Three hard constraints on escalation content:
|
|
324
|
+
|
|
325
|
+
**Never escalate a zero as a finding.** A 0/N result is the one outcome fully
|
|
326
|
+
explained by an environment mistake; every other outcome at least proves the
|
|
327
|
+
harness reached the tool. On 2026-08-08 a competitor CLI measured 0/3, and a
|
|
328
|
+
known-good CLI with 5 prior successful trials ALSO measured 0/3 in the same
|
|
329
|
+
harness. Both zeros were harness bugs (a missing
|
|
330
|
+
`--skip-git-repo-check` and a missing `</dev/null`). A zero is reportable only
|
|
331
|
+
alongside a positive control that scored non-zero in the SAME harness on the
|
|
332
|
+
SAME task, and the control travels with the report.
|
|
333
|
+
|
|
334
|
+
**Never escalate an absence as evidence.** An empty result is an absent
|
|
335
|
+
measurement, not a clean one. A substring search over an empty haystack reports
|
|
336
|
+
nothing missing. Assert the haystack is plausibly sized before believing
|
|
337
|
+
anything about its contents. `release.yml:120-130` implements exactly this as a
|
|
338
|
+
VACUITY GUARD: an empty workflow-runs API result never satisfies the wait loop,
|
|
339
|
+
because "no failing runs" over zero runs is not "everything passed."
|
|
340
|
+
|
|
341
|
+
**Never escalate a red you cannot clear as a blocker.** Drift in external state
|
|
342
|
+
that only a credential holder can fix is a REPORT, not a gate. Escalate it as a
|
|
343
|
+
request to make the guard advisory, per section 7.1. The measured cost of the
|
|
344
|
+
other choice: the MCP registry guard's first version exited 1 from
|
|
345
|
+
`run-all-tests.sh`, which every push runs in every shard, so a publishing gap
|
|
346
|
+
nobody in the repo could close turned Tests red on every push (run 93989130,
|
|
347
|
+
shard 1). A red nobody can clear is how the team learns to ignore reds.
|
|
348
|
+
|
|
349
|
+
---
|
|
350
|
+
|
|
351
|
+
## 5. Release windows: main is WIP=1
|
|
352
|
+
|
|
353
|
+
When a VERSION bump is pushed, main enters a release window that closes only
|
|
354
|
+
when npm shows the new version. Inside that window the Release Captain owns
|
|
355
|
+
main and NOBODY else pushes to it, including docs.
|
|
356
|
+
|
|
357
|
+
This is not caution. `release.yml`'s `required-ci` job waits for Tests, Bun
|
|
358
|
+
Parity and Security Audit to be green AT THE EXACT RELEASE SHA and fails
|
|
359
|
+
closed, treating cancelled, timed_out, failure and skipped as non-passes
|
|
360
|
+
(`release.yml:95-171`, grep `cancelled, timed_out, failure, skipped: none is a
|
|
361
|
+
pass`). Any later push to the branch moves HEAD and GitHub's concurrency rules
|
|
362
|
+
CANCEL the older run at the release SHA. `required-ci` then correctly reports
|
|
363
|
+
failure although nothing was broken.
|
|
364
|
+
|
|
365
|
+
Measured twice. On 2026-08-08, v9.18.0 failed with `FAIL: a required workflow
|
|
366
|
+
did not succeed at 8892f477` while Bun Parity and Security Audit were both
|
|
367
|
+
green and Tests read `completed/cancelled`. Re-broken on 2026-09-10 by a
|
|
368
|
+
docs-only commit pushed during v9.27.3's release, which cancelled its Tests and
|
|
369
|
+
stopped the publish. "It is only docs" is not an exemption: concurrency
|
|
370
|
+
cancellation keys on the BRANCH, not the diff.
|
|
371
|
+
|
|
372
|
+
Queue follow-up commits locally. See `skills/release-cadence.md` for the window
|
|
373
|
+
procedure and its recovery paths.
|
|
374
|
+
|
|
375
|
+
---
|
|
376
|
+
|
|
377
|
+
## 6. Contended main: verify ancestry, never force
|
|
378
|
+
|
|
379
|
+
Several agent processes run against this checkout at once. On 2026-08-07 a
|
|
380
|
+
parallel process force-reset main to a different lineage and five commits
|
|
381
|
+
vanished from the remote, including one already reported to the founder as
|
|
382
|
+
landed with CI running against it.
|
|
383
|
+
|
|
384
|
+
- Before claiming a push landed, check ANCESTRY, not equality:
|
|
385
|
+
`git fetch origin main && git merge-base --is-ancestor <sha> FETCH_HEAD`.
|
|
386
|
+
A point-in-time `git ls-remote` match is true at that instant and can be
|
|
387
|
+
false minutes later.
|
|
388
|
+
- Never `git push --force*` to main here. `--force-with-lease` compares against
|
|
389
|
+
the lease you name; on a contended branch the lease can match while the
|
|
390
|
+
remote's real lineage has already moved, so the force SUCCEEDS and overwrites
|
|
391
|
+
another agent's work.
|
|
392
|
+
- `git push ... | tail -N` MASKS git's exit code; a rejected push reports rc=0.
|
|
393
|
+
Redirect to a file and read `$?` directly.
|
|
394
|
+
|
|
395
|
+
---
|
|
396
|
+
|
|
397
|
+
## 7. The void measurement: the failure shape that costs the most
|
|
398
|
+
|
|
399
|
+
**A measurement that cannot see the thing it claims to measure is worse than no
|
|
400
|
+
measurement, because it produces a log line that looks exactly like evidence.**
|
|
401
|
+
|
|
402
|
+
This is the organizing rule of this module. SIX instances were observed in a
|
|
403
|
+
SINGLE session on 2026-09-14, in six different tools, by different people, none
|
|
404
|
+
of whom was being careless. One of them is instance 6 below, committed by the
|
|
405
|
+
author of this document while writing this section. That rate is why it is
|
|
406
|
+
section 7 and not a footnote: it is not a mistake anyone stops making by trying
|
|
407
|
+
harder.
|
|
408
|
+
|
|
409
|
+
| # | The check | What it reported | What was true | Why it was blind |
|
|
410
|
+
|---|---|---|---|---|
|
|
411
|
+
| 1 | MCP registry staleness guard | drift, 7.34.1 vs 9.50.1 | registry was already current | iterated `servers[]`, took the FIRST name match; the registry keeps every published version as an `active` row |
|
|
412
|
+
| 2 | "any stale version string left after the bump" | OK on every file | measured nothing at all | shell arithmetic comparison errored on every file; the error text printed alongside the OK lines |
|
|
413
|
+
| 3 | `grep -c '9.50.1' dashboard/static/index.html` | no match, rc=1 | the string class IS present | file contains NUL bytes, so grep treats it as binary and silently reports nothing |
|
|
414
|
+
| 4 | competitor benchmark | 0/3 | harness never reached the tool | missing `--skip-git-repo-check` and `</dev/null` (2026-08-08) |
|
|
415
|
+
| 5 | `git push` wrapped in `timeout 180` | `[exited with code 0]` | origin never moved | the timeout killed the pre-push hook mid-pytest; the WRAPPER's exit code replaced the operation's |
|
|
416
|
+
| 6 | `grep -rn ... --include=*.sh` for a precedent | no output, `rc=0` | the search never ran | zsh expanded `--include=*.sh` as a glob, found no match, and aborted the command before grep started |
|
|
417
|
+
|
|
418
|
+
Instance 5 is the most dangerous of the set: it nearly shipped a false release
|
|
419
|
+
announcement to the founder. It was caught only because the operator fetched
|
|
420
|
+
origin and compared SHAs instead of trusting the exit code. See section 4.2.
|
|
421
|
+
|
|
422
|
+
Instance 6 is instructive because it is the cheapest possible version of the
|
|
423
|
+
mistake and it still produced a confident wrong answer. A shell glob failure
|
|
424
|
+
printed `no matches found` and returned `rc=0`, so a "no results" reading looked
|
|
425
|
+
like a verified absence. The fix is the same as everywhere else in this table:
|
|
426
|
+
run a positive control. `grep -rl "mcp-publisher"` returned 0 files while the
|
|
427
|
+
control `grep -rl "loki-mode"` returned 448 in the same command, which is what
|
|
428
|
+
made the zero trustworthy.
|
|
429
|
+
|
|
430
|
+
Instances 1 and 3 are reproducible right now. For 1, the fix is at
|
|
431
|
+
`tests/test-mcp-registry-not-stale.sh:135-143`, which selects on
|
|
432
|
+
`official.get("isLatest") is True` and comments the reason: "Taking the first
|
|
433
|
+
name match reports whichever row the API happens to order first, which is how
|
|
434
|
+
this guard read 7.34.1 as live for hours AFTER 9.50.1 was published and flagged
|
|
435
|
+
latest. Select on isLatest, never order."
|
|
436
|
+
|
|
437
|
+
For 3, measured on this checkout, three tools give three answers for the same
|
|
438
|
+
question on the same file:
|
|
439
|
+
|
|
440
|
+
```
|
|
441
|
+
grep -c 'Loki' dashboard/static/index.html -> (nothing), rc=1
|
|
442
|
+
grep -ac 'Loki' dashboard/static/index.html -> 27, rc=0
|
|
443
|
+
python3 ... .read().count('Loki') -> 91
|
|
444
|
+
```
|
|
445
|
+
|
|
446
|
+
The default tool was silently wrong. A dist check built on a plain `grep` would
|
|
447
|
+
have reported clean every single time, which matters here more than anywhere
|
|
448
|
+
else: this repo's worst recorded release defect is a stale `loki-ts/dist/loki.js`
|
|
449
|
+
shipping the wrong version for 27 releases.
|
|
450
|
+
|
|
451
|
+
### 7.1 The three rules that follow
|
|
452
|
+
|
|
453
|
+
**A CHECK THAT CANNOT FAIL IS NOT A CHECK.** Every verification needs a positive
|
|
454
|
+
control proving it can produce a non-passing result. This repo already applies
|
|
455
|
+
that rule to zeros; extend it to GREENS, because a green from a broken
|
|
456
|
+
comparison is indistinguishable from a real green in the log. Instance 2 is the
|
|
457
|
+
proof: every line printed OK while the comparison errored on every file.
|
|
458
|
+
|
|
459
|
+
**NEVER GREP A BUILT ARTIFACT WITHOUT A POSITIVE CONTROL.** Bundles, minified
|
|
460
|
+
JS and dist HTML routinely contain bytes that make text tools bail silently. A
|
|
461
|
+
grep over dist that finds nothing has not proven absence; it has usually proven
|
|
462
|
+
the tool could not read the file. Use a control string known to be present, or
|
|
463
|
+
a reader that cannot silently bail (`grep -a`, or Python with an explicit
|
|
464
|
+
encoding).
|
|
465
|
+
|
|
466
|
+
**A GUARD THAT CANNOT SEE SUCCESS IS WORSE THAN NO GUARD.** Instance 1 reported
|
|
467
|
+
red against a correct state. A red nobody can clear trains the team to ignore
|
|
468
|
+
reds, which is how a REAL red gets waved through. This is the same reasoning
|
|
469
|
+
`scripts/local-ci.sh:69-80` gives for refusing concurrent gate runs: "a phantom
|
|
470
|
+
failure trains the reader to distrust the gate."
|
|
471
|
+
|
|
472
|
+
Pair that last one with the repo's existing ADVISORY-GUARD rule, because they
|
|
473
|
+
are the two halves of one design. When drift is closable only by a credential
|
|
474
|
+
holder, the guard must REPORT and exit 0 on the push path, with the hard exit
|
|
475
|
+
behind an env var only a scheduled job sets
|
|
476
|
+
(`LOKI_MCP_REGISTRY_STRICT=1`). Precedent:
|
|
477
|
+
`tests/test-model-catalog-staleness.sh:97-102` asserts staleness must not change
|
|
478
|
+
the exit code, calling it "THE INVARIANT: advisory only." The cost of getting
|
|
479
|
+
this wrong is measured: the registry guard's first version exited 1, was
|
|
480
|
+
registered in `run-all-tests.sh`, and turned a publishing gap nobody in the repo
|
|
481
|
+
could fix into a failing Tests run on every push (run 93989130, shard 1).
|
|
482
|
+
|
|
483
|
+
Verify that the advisory green did NOT come from silencing the guard: the log
|
|
484
|
+
must still print the FAIL line.
|
|
485
|
+
|
|
486
|
+
### 7.2 Why this is a COORDINATION rule, not a testing tip
|
|
487
|
+
|
|
488
|
+
Void measurements are what make every other rule in this module load-bearing:
|
|
489
|
+
|
|
490
|
+
- They are why **WIP is bounded by verification** (section 3). More streams
|
|
491
|
+
produce more checks, and an unverified check is a confident false green.
|
|
492
|
+
- They are why **file ownership is exclusive** (section 2). A silently
|
|
493
|
+
discarded edit is a void measurement of your own work: `bash -n` passed, and
|
|
494
|
+
the code was gone.
|
|
495
|
+
- They are why **escalations carry controls** (section 4). A finding without a
|
|
496
|
+
control is a claim, and claims do not survive a council.
|
|
497
|
+
|
|
498
|
+
### 7.3 If this module is ever trimmed
|
|
499
|
+
|
|
500
|
+
1. **A file belongs to exactly one writer at a time, and the lead is not
|
|
501
|
+
exempt.** Concurrent writes here do not conflict; they silently discard.
|
|
502
|
+
2. **WIP is bounded by what can be VERIFIED serially, not by what can be
|
|
503
|
+
spawned.** The gate refuses concurrency on purpose, and the council is a
|
|
504
|
+
serialization point.
|
|
505
|
+
3. **Every measurement needs a control that proves it can fail.** Zeros need
|
|
506
|
+
positive controls; greens need them too; empty haystacks need size
|
|
507
|
+
assertions; built artifacts need a reader that cannot silently bail; and
|
|
508
|
+
cancelled is not passed.
|