mjolnir-qa 4.0.0 → 6.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +3178 -16
- package/README.md +81 -760
- package/dist/cli.d.mts +552 -206
- package/dist/cli.mjs +31409 -13990
- package/dist/mcp/stdio.mjs +1290 -405
- package/package.json +76 -32
- package/README.ar.md +0 -647
- package/README.bn.md +0 -647
- package/README.br.md +0 -647
- package/README.bs.md +0 -647
- package/README.da.md +0 -647
- package/README.de.md +0 -647
- package/README.es.md +0 -647
- package/README.fr.md +0 -647
- package/README.gr.md +0 -647
- package/README.he.md +0 -647
- package/README.it.md +0 -647
- package/README.ja.md +0 -647
- package/README.ko.md +0 -647
- package/README.no.md +0 -647
- package/README.pl.md +0 -647
- package/README.ru.md +0 -647
- package/README.th.md +0 -647
- package/README.tr.md +0 -647
- package/README.uk.md +0 -647
- package/README.vi.md +0 -647
- package/README.zh.md +0 -647
- package/README.zht.md +0 -647
- package/dist/scan-pipeline-Bg-46cLS.mjs +0 -2
- package/dist/scan-pipeline-CCJdEKNa.mjs +0 -16336
package/CHANGELOG.md
CHANGED
|
@@ -9,8 +9,2904 @@ Rule behavior changes (new rules, FP-rate changes against the corpus,
|
|
|
9
9
|
severity changes) are first-class entries here — rule IDs are immutable
|
|
10
10
|
once shipped, so this file is the record of what changed between versions.
|
|
11
11
|
|
|
12
|
+
## [6.0.0] — 2026-10-04
|
|
13
|
+
|
|
14
|
+
<!-- ANTI-CREEP-EXCEPTION: The launch set grew from 0 to 2 — QA-PW-117 and QA-JV-101 both earned core at n=35 with a 9.89% Wilson upper bound against the 10% ceiling, every row adjudicated by hand against a pinned corpus commit. Law 1 requires an equal-size removal or a recorded exception. Nothing was removed, deliberately: the quarantine population is what keeps unmeasured rules out of default scans, and promoting two rules by demoting two measured ones would have traded measured accuracy for a smaller number. The growth is arithmetic, not preference. -->
|
|
15
|
+
|
|
16
|
+
The verification trust engine, measured. A tier you can only reach with evidence,
|
|
17
|
+
a promotion that cannot land unreviewed, and generated documentation that cannot
|
|
18
|
+
quietly go stale behind the code. Carries the 6.0.0-rc.1 line below.
|
|
19
|
+
|
|
20
|
+
### Two core rules, and the arithmetic behind them
|
|
21
|
+
|
|
22
|
+
`QA-PW-117` and `QA-JV-101` hold `core`, each at **n=35, zero observed false
|
|
23
|
+
positives, Wilson upper bound 9.89%** against a 10% ceiling — reproduced by hand
|
|
24
|
+
from `src/lib/wilson.ts` before either was believed.
|
|
25
|
+
|
|
26
|
+
`QA-JV-101` earned it in two batches, and the sequence matters: **32 (10.72%) →
|
|
27
|
+
34 (10.15%) → 35 (9.89%)**. The first batch's verdict file reads as a refusal on
|
|
28
|
+
its own ("STILL SHORT, NOT promoted"). Reading only it is how an audit concluded
|
|
29
|
+
the promotion was fabricated. Both files are cited in the certification record so
|
|
30
|
+
the next reader cannot repeat the mistake.
|
|
31
|
+
|
|
32
|
+
### The tree told a different story than the code
|
|
33
|
+
|
|
34
|
+
Both promotions reached the registry, the tier evidence and this file while their
|
|
35
|
+
generated documentation still reported `extended` at n=23/24. Nothing failed,
|
|
36
|
+
because nothing asked — `npm run check` regenerates and then inspects nothing,
|
|
37
|
+
and three of four workflows asserted staleness with `git diff`, which is provably
|
|
38
|
+
blind to an untracked generated page.
|
|
39
|
+
|
|
40
|
+
- **`npm run docs:staleness`** — one assertion, `git status`-based, read-only by
|
|
41
|
+
construction, wired into all four workflows.
|
|
42
|
+
- **A count nobody can check is a claim, so generate it.** `README.md` said
|
|
43
|
+
"`core` is empty" for a release after two rules had earned it.
|
|
44
|
+
- **`docs/MEASUREMENT-CLOSEOUT.md`** corrected 74/5 → 73/6, gained the missing
|
|
45
|
+
`QA-PY-004` row, and separates _stale_ from unmeasured.
|
|
46
|
+
|
|
47
|
+
### A promotion can no longer land with no artifact
|
|
48
|
+
|
|
49
|
+
`docs/CORE-CERTIFICATION.json` is required to hold `core`, and expires with its
|
|
50
|
+
evidence: the measurement is compared against the live registry field by field.
|
|
51
|
+
`docs/TIER-HISTORY.json` records all 19 transitions as data, so a demotion is no
|
|
52
|
+
longer something that exists only as a source comment.
|
|
53
|
+
`docs/RULE-CONSTITUTION.json` states seven principles, each naming the code that
|
|
54
|
+
enforces it, plus **two principles it explicitly refuses** with the measurement
|
|
55
|
+
that refutes them.
|
|
56
|
+
|
|
57
|
+
### The detector set is wired; it is not yet shown to have teeth
|
|
58
|
+
|
|
59
|
+
Of the 40 rules carrying both fixture legs, **2** have a negative fixture that is
|
|
60
|
+
the positive fixture with the defect removed. The other **38** are a different
|
|
61
|
+
program. Leashed at the true floor of 2, so it fails on a rule _regressing_ out of
|
|
62
|
+
that set and not on the 38 that were always like this. Closing the gap needs a
|
|
63
|
+
per-rule recipe for what the defect _is_; that work is named, not assumed away.
|
|
64
|
+
|
|
65
|
+
### .NET runs are evidence (ADR 0015)
|
|
66
|
+
|
|
67
|
+
`EVIDENCE_CONVENTIONS` held four entries and .NET was in none of them, so a C#/F#
|
|
68
|
+
run left every finding at `INFERRED` however often the suite actually ran. TRX is
|
|
69
|
+
now dispatched **before** JUnit, and that ordering is load-bearing: the JUnit
|
|
70
|
+
branch is reached by an `<?xml` sniff, so a TRX file was matching nothing and
|
|
71
|
+
returning an empty run — which reads as "the suite passed", and for a .NET project
|
|
72
|
+
whose tests were never examined was the most reassuring possible wrong answer.
|
|
73
|
+
|
|
74
|
+
### Release honesty
|
|
75
|
+
|
|
76
|
+
`candidate:readiness` remains **NO_GO** on this release. Four of its ten blockers
|
|
77
|
+
are `REMOTE_PROVEN` attestations about the real world — protected holdout,
|
|
78
|
+
real-world repositories, platform matrix, consumer install, remote workflow —
|
|
79
|
+
plus external validation and owner authorization.
|
|
80
|
+
|
|
81
|
+
The check RAN and was RECORDED as unsatisfied. It was not treated as blocking
|
|
82
|
+
because `vars.STABLE_RELEASE_AUTHORIZED` is `true`: a decision someone made and
|
|
83
|
+
wrote down, not a check that passed.
|
|
84
|
+
|
|
12
85
|
## [Unreleased]
|
|
13
86
|
|
|
87
|
+
### Two core rules, measured: QA-PW-117 and QA-JV-101 - `ANTI-CREEP-EXCEPTION`
|
|
88
|
+
|
|
89
|
+
Law 1 requires an equal-size removal for every addition to the governed set.
|
|
90
|
+
This is an addition of **two** with nothing to remove, because the governed set
|
|
91
|
+
held **zero** core rules until now. The exception is recorded rather than the
|
|
92
|
+
baseline being edited, because `previousBaselineCore` is `0` on purpose: lowering
|
|
93
|
+
`baselineCore` to match the grown tier would make the growth read as zero and
|
|
94
|
+
switch the law off in one uncross-checked JSON edit.
|
|
95
|
+
|
|
96
|
+
The reason is that both are **measured** promotions, not declared ones.
|
|
97
|
+
`QA-PW-117` reached n=35 and `QA-JV-101` reached n=35, both with zero observed
|
|
98
|
+
false positives and a Wilson upper bound of **9.89%** against the 10% ceiling,
|
|
99
|
+
both adjudicated row by row against pinned source, and both found by a
|
|
100
|
+
corpus-wide `--core-candidates` sweep rather than by any global cap raise.
|
|
101
|
+
`measurementTier` returns `core` for each on its own evidence and
|
|
102
|
+
`declaredCoreWithoutEvidence` is null, so each declaration records a decision the
|
|
103
|
+
measurement had already reached.
|
|
104
|
+
|
|
105
|
+
Both had been demoted from core in 6.0 for exactly this reason, and
|
|
106
|
+
`src/rules/tier-evidence.ts` ranked `QA-PW-117` first among the nineteen most
|
|
107
|
+
likely to clear on a re-sample.
|
|
108
|
+
|
|
109
|
+
### The tree told a different story than the code
|
|
110
|
+
|
|
111
|
+
Both core promotions landed in code, in the tier evidence and in this file,
|
|
112
|
+
while their generated documentation went on reporting `extended` at n=23 and
|
|
113
|
+
n=24. Nothing failed: `npm run check` regenerates and then inspects nothing,
|
|
114
|
+
so a generator that rewrites a committed file leaves the tree green by
|
|
115
|
+
construction. Four workflows each asserted staleness by hand and three of them
|
|
116
|
+
could not see an untracked file at all.
|
|
117
|
+
|
|
118
|
+
- **`docs/rules/QA-JV-101.md`, `docs/rules/QA-PW-117.md` and
|
|
119
|
+
`docs/DEPTH-ADJUDICATION.md` regenerated.** The shipped rule pages claimed
|
|
120
|
+
`extended` at n=23/24; both rules are `core` at n=35. No finding changes.
|
|
121
|
+
- **`npm run docs:staleness` — one staleness assertion, and it can run locally.**
|
|
122
|
+
It reads `git status`, so it sees an untracked generated page; `git diff`
|
|
123
|
+
alone cannot, which is how a new rule's docs page shipped silently through
|
|
124
|
+
`corpus-audit.yml`, `release.yml` and `stable-release.yml`. Read-only by
|
|
125
|
+
construction — it cannot cost a working tree the adjudication records under
|
|
126
|
+
`tests/corpus/verdicts/`, and it is deliberately absent from `npm run check`,
|
|
127
|
+
where asserting after regenerating would be checking the generator against
|
|
128
|
+
itself. All four workflows now call it, and
|
|
129
|
+
`tests/contract/generated-surfaces.spec.ts` fails if a fifth is added that
|
|
130
|
+
does not, or if the copies drift apart again.
|
|
131
|
+
- **`core` is a generated number, not a sentence.** `README.md` said "`core`
|
|
132
|
+
is empty" for a whole release after two rules had earned it. `measurementBlock()`
|
|
133
|
+
now returns `core` and `docs:counts` stamps it into `README.md`,
|
|
134
|
+
`docs/MEASUREMENT-CLOSEOUT.md` and the other census surfaces, so
|
|
135
|
+
`tests/contract/census-drift.spec.ts` fails if prose and registry disagree.
|
|
136
|
+
It counts UNMEASURED core rules deliberately: `MAX_UNMEASURED_CORE = 0`
|
|
137
|
+
means such a registry cannot ship, and a census that hid the rule behind
|
|
138
|
+
`measured` would report a clean `core: 0` at the moment the law broke.
|
|
139
|
+
- **`docs/MEASUREMENT-CLOSEOUT.md` corrected to 73/6** (it read 74/5), gained
|
|
140
|
+
the missing `QA-PY-004` row, and now separates **stale** from unmeasured:
|
|
141
|
+
`QA-PY-004` (revision 4) and `QA-PY-007` (revision 5) have verdict sets that
|
|
142
|
+
no longer describe the shipped detector, so they need re-measurement at the
|
|
143
|
+
current revision rather than new classification.
|
|
144
|
+
- **`src/rules/java/qa-jv-101-disabled-test.ts` — one provenance block.** The
|
|
145
|
+
file carried three stacked comments, two of them obsolete demotion notices
|
|
146
|
+
contradicting the third. It now records the two-batch arithmetic
|
|
147
|
+
(n=32 → 34 → 35) and names both verdict files, because the first batch
|
|
148
|
+
(`jv-101-final.json`, "STILL SHORT") reads as a refusal on its own and the
|
|
149
|
+
second (`jv-101-core.json`) is what cleared the ceiling.
|
|
150
|
+
- **Governing prose that had gone stale.** `rule-author` and `surface-law`
|
|
151
|
+
both asserted the core tier was empty; the archived
|
|
152
|
+
`QA-FINAL-RELEASE/RULE_CERTIFICATION.md` carries a SUPERSEDED banner marking
|
|
153
|
+
its `core` column a fossil of the deleted `registry-census` predicate. The
|
|
154
|
+
ADR and the roadmap's "23 pending classifications" were left alone: they
|
|
155
|
+
record what a decision or a pass actually was, and editing them to match
|
|
156
|
+
today would falsify the record.
|
|
157
|
+
|
|
158
|
+
### The detector set is wired. It is not yet shown to have teeth.
|
|
159
|
+
|
|
160
|
+
`docs/PRECISION-RATCHET.json` records 32 rules as holding a PRECISION leg and
|
|
161
|
+
says plainly that this is "a wiring proof, not an accuracy proof, and a
|
|
162
|
+
self-derived one". That caveat can be given a number, and the number is not
|
|
163
|
+
comfortable.
|
|
164
|
+
|
|
165
|
+
A must-not-fire fixture is only evidence that a rule is _sensitive_ when it is
|
|
166
|
+
the must-fire fixture with the defect neutralised. When it is a different
|
|
167
|
+
program that merely does not trigger, the pair shows the detector is
|
|
168
|
+
directional and says nothing about the predicate. The quad counts presence, so
|
|
169
|
+
both look identical to it.
|
|
170
|
+
|
|
171
|
+
- **`docs/SENSITIVITY-RATCHET.json` — the SENSITIVITY arm of
|
|
172
|
+
`check-fixture-quad`.** Of the 40 live rules carrying both fixture legs, **2**
|
|
173
|
+
hold a negative fixture that is plausibly the positive fixture with the defect
|
|
174
|
+
removed (`QA-PW-144`, one changed line of nine; `QA-PW-116`, one of seven).
|
|
175
|
+
The other **38** are a different program — the next nearest is 0.55 diverging
|
|
176
|
+
and the furthest diverge by more lines than either file contains, because a
|
|
177
|
+
hand-written neutral fixture also renames the class and rewrites the bodies.
|
|
178
|
+
`QA-JV-101`, a core rule, is one of them: its negative fixture is a different
|
|
179
|
+
class with different methods and no case distinguishing a disabled test from
|
|
180
|
+
a justified one.
|
|
181
|
+
- **Leashed at the true count, in an existing gate.** No new gate id and no new
|
|
182
|
+
npm script, so Law 0 is untouched: the arm is folded into
|
|
183
|
+
`check-fixture-quad` beside PRECISION, which asks a different question and
|
|
184
|
+
reports separately. The floor is 2, the honest number, so the gate fails on a
|
|
185
|
+
rule _regressing_ out of the mutant set or on a new rule shipping without one,
|
|
186
|
+
and does not go red on the 38 that were always like this. `--update` only ever
|
|
187
|
+
raises the floor, so a regression can never be absorbed by re-running a command.
|
|
188
|
+
- **Neither core rule is exempt.** The ratchet names the gap rather than
|
|
189
|
+
narrowing it: closing it needs a per-rule recipe for what the defect _is_, so
|
|
190
|
+
the mutant can be generated rather than hand-written a second time and left to
|
|
191
|
+
drift. That is the remaining work, and it is now named rather than assumed
|
|
192
|
+
away.
|
|
193
|
+
- **`docs:staleness` no longer watches provenance-stamped artifacts.** Their
|
|
194
|
+
`baseSha` is meant to lag, and `git status` cannot tell a restamp from real
|
|
195
|
+
drift, so watching them produced FAIL on every regeneration and caught nothing
|
|
196
|
+
that matters. `docs:provenance-drift` already checks their content with the
|
|
197
|
+
provenance keys excluded; leaving them there would have been a second copy
|
|
198
|
+
doing the same job worse.
|
|
199
|
+
|
|
200
|
+
### A promotion can no longer land with no artifact
|
|
201
|
+
|
|
202
|
+
Both core promotions reached the registry, the tier evidence and this file while
|
|
203
|
+
their generated documentation went on reporting `extended` at n=23 and n=24.
|
|
204
|
+
Nothing failed, because nothing asked. The mechanism that should have caught it
|
|
205
|
+
now exists, and it catches the case that matters: not "is this rule measurable"
|
|
206
|
+
but "did a person certify **this** measurement, and does the certification still
|
|
207
|
+
describe the rule".
|
|
208
|
+
|
|
209
|
+
- **`docs/CORE-CERTIFICATION.json` — a record is required to hold `core`.** A
|
|
210
|
+
rule with no record fails; a record for a rule that lost the tier fails, because
|
|
211
|
+
a demotion belongs in the tier evidence ledger, not in a certification file.
|
|
212
|
+
- **The record expires with its evidence, which is the entire design.** The
|
|
213
|
+
measurement block is compared field by field against the live `MEASURED_FP`
|
|
214
|
+
row on every run. Move the detector revision, change n, or introduce an
|
|
215
|
+
observed false positive and the gate fails until someone re-certifies and
|
|
216
|
+
writes down why. It cannot outlive what it certifies.
|
|
217
|
+
- **The cited evidence must account for the sample.** Every path a record cites
|
|
218
|
+
must exist in the tree, and the corpus verdict rows it lists must sum to
|
|
219
|
+
exactly `n`. A certification may not rest on evidence that is not here, nor on
|
|
220
|
+
less evidence than it claims. Proven in both directions: a record citing 33
|
|
221
|
+
rows for an `n=35` measurement fails.
|
|
222
|
+
- **Self-derived evidence is excluded on purpose.** The quad verdicts under
|
|
223
|
+
`tests/corpus/verdicts/quad/` are wiring proofs and do not count. Only corpus
|
|
224
|
+
rows do — which is why both records sum to exactly 35 (29+2+4 and 20+10+2+3).
|
|
225
|
+
- **`QA-JV-101`'s record states its known weakness rather than omitting it.** Its
|
|
226
|
+
must-not-fire fixture is a hand-written different program, not the must-fire
|
|
227
|
+
fixture with `@Disabled` neutralised, so per `docs/SENSITIVITY-RATCHET.json` it
|
|
228
|
+
does not prove the predicate is sensitive to the annotation. That is the gap
|
|
229
|
+
per-rule defect recipes must close, and a certification record that omitted it
|
|
230
|
+
would be worse than no record.
|
|
231
|
+
- **No new gate, no new npm script.** This is an arm of `rules:promotion:check`,
|
|
232
|
+
which is already the promotion gate and already in `gates:claim-integrity`. Law
|
|
233
|
+
0 charges for both, and extending the existing gate is free where a sibling
|
|
234
|
+
would not be. Deliberately not given its own unit test either, for the reason
|
|
235
|
+
`src/rules/conformity.ts` already states about a duplicate check.
|
|
236
|
+
|
|
237
|
+
### The delta loop is now visible, and the freeze is written down
|
|
238
|
+
|
|
239
|
+
- **Quickstart documents baseline → delta.** Steps 1–3 report the **state** of
|
|
240
|
+
your tests; `mjolnir ci verify` reports the **change**, which is the only
|
|
241
|
+
question a reviewer has about a diff. `--save-baseline` writes
|
|
242
|
+
`.mjolnir/baseline.json`, `ci verify` reports RESOLVED (of which only
|
|
243
|
+
`VERIFIED-RESOLVED` is a fix claim) and NEW, and the `0`/`1`/`2` exit contract
|
|
244
|
+
is stated. It also says why `mjolnir ci install` does not add it: `ci verify`
|
|
245
|
+
exits `2` until a baseline exists, which would redden every install on day
|
|
246
|
+
one. Agents were already taught this at `src/commands/install-agents.ts:58`;
|
|
247
|
+
humans were not taught it anywhere.
|
|
248
|
+
- **`docs/ENGINE-FREEZE.md` — what a consumer may rely on.** The source of truth
|
|
249
|
+
is `CONTRACT_REGISTRY` in `src/engine/contract-versions.ts`; the document
|
|
250
|
+
deliberately copies no version numbers, for the same reason the census
|
|
251
|
+
sentinels exist. It explains each `compatibilityPolicy`, records that
|
|
252
|
+
`scoringModelVersion` 2.0.0 is a _semantic_ change (ADR 0014's evidence floor)
|
|
253
|
+
so a consumer comparing scores across it is comparing different things, and is
|
|
254
|
+
honest that `frameworkSupportMatrixVersion` is written by the engine and read
|
|
255
|
+
by nothing but a test — a version stamp on a fact, not a negotiated protocol.
|
|
256
|
+
|
|
257
|
+
### Verification and enforcement overhaul
|
|
258
|
+
|
|
259
|
+
Track A of the verification-enforcement plan: the checks that could fail but
|
|
260
|
+
never ran, the readouts that disagreed with each other, and the two places where
|
|
261
|
+
a claim was made without a mechanism behind it. Nothing here changes what a
|
|
262
|
+
finding says except where stated (core tier, scoring model).
|
|
263
|
+
|
|
264
|
+
#### Added
|
|
265
|
+
|
|
266
|
+
- **`docs/PRECISION-RATCHET.json` — the PRECISION arm.** `check-fixture-quad`
|
|
267
|
+
failed only on an over-claim (a capability advertising M3 without its four
|
|
268
|
+
fixture legs). It had no arm for the leg that a fixture directory cannot
|
|
269
|
+
satisfy: PRECISION needs someone to have classified the detector _silent_ on a
|
|
270
|
+
real input. `npm run check-fixture-quad -- --update-precision` records the
|
|
271
|
+
count of live rules holding a classified TN verdict; the gate fails below the
|
|
272
|
+
recorded floor. **Recorded at 32, not 0** — 32 rules already hold a TN leg
|
|
273
|
+
from the generated `--verdicts` pass, so a floor of 0 would have been
|
|
274
|
+
decoration. The ratchet note says plainly that these are scanner-executed
|
|
275
|
+
fixture legs: a _wiring_ proof, not an accuracy proof.
|
|
276
|
+
- **`docs/HOLDOUT-SPLIT.json` + `npm run holdout:split`** — the deterministic
|
|
277
|
+
partition of corpus repositories into measurement and holdout that
|
|
278
|
+
`claim-registry.json` blocked `rule-registry-census` on. The unit is the
|
|
279
|
+
repository, never the row (a per-row split puts the same test in both halves
|
|
280
|
+
and can be searched until the halves agree); the assignment is a salted hash
|
|
281
|
+
of the repository id, not a list; and the single exclusion — repositories
|
|
282
|
+
holding rows for an `isCoreCandidate` rule — is _derived_ from that predicate
|
|
283
|
+
rather than declared, so it moves with the candidate set and cannot be edited
|
|
284
|
+
into place. The holdout **validates, it does not gate**: core promotion reads
|
|
285
|
+
the measurement partition, because a gating holdout would be sized by the
|
|
286
|
+
corpus rather than by the question.
|
|
287
|
+
- **`docs/QUARANTINE-OWNERSHIP.json` — the quarantine exit ramp.** 34 rules were
|
|
288
|
+
quarantined with no disposition recorded for any of them; `checkQuarantineOwnership`
|
|
289
|
+
_reported_ that count and explained that a cap "would have to start at 34".
|
|
290
|
+
The defect was not the number, it was that silence passed. A quarantined rule
|
|
291
|
+
must now carry either a `quarantinePromotion` or a typed disposition with an
|
|
292
|
+
owner, a rationale and a review date, and doctor **fails on silence**.
|
|
293
|
+
Dispositions are derived from the Wilson interval: 26 `pending-samples`,
|
|
294
|
+
3 `pending-measurement`, 4 `pending-remeasure` (a rework shipped and the
|
|
295
|
+
verdict predates it), 1 `structural` (QA-TEST-003, whose proposed fix was
|
|
296
|
+
implemented, measured, and rejected).
|
|
297
|
+
- **`RETRACT` in `corpus:apply-verdicts`, and `--verdicts-dir=`.** The orphan
|
|
298
|
+
rule in `tests/corpus/verdicts/README.md` said an orphaned row "must be
|
|
299
|
+
removed" and there was no mechanism for it — only a tool that fills blanks.
|
|
300
|
+
A verdict recorded on a row whose finding is not there is worse than a stray
|
|
301
|
+
row, because it becomes evidence. `RETRACT` removes a blank row and records
|
|
302
|
+
nothing, and **refuses** a row that already carries a verdict. The
|
|
303
|
+
`--verdicts-dir` flag exists because the retraction path was otherwise
|
|
304
|
+
untestable without writing into the live corpus mid-run.
|
|
305
|
+
- **`mjolnir doctor` runs from an installed package.** It defaulted its target
|
|
306
|
+
to `process.cwd()`, so a user who installed the CLI and ran `doctor` in their
|
|
307
|
+
own project got `No fixtures directory … Run from the mjolnir repo root.` — a
|
|
308
|
+
self-audit reachable only from the source tree. With no target it now resolves
|
|
309
|
+
the installed package from `import.meta.url`. `package.json#files` ships what
|
|
310
|
+
the self-audit reads (`tests/fixtures`, `tests/corpus/verdicts`,
|
|
311
|
+
`src/rules`, the two baseline JSONs, CHANGELOG) — without `src/rules` the
|
|
312
|
+
installed audit is INCONCLUSIVE on detector identity. Verified by packing a
|
|
313
|
+
tarball, installing it into a fresh project and running the published binary.
|
|
314
|
+
- **Doctor check `core-floor-declaration`.** The interlock that makes the core
|
|
315
|
+
floor safe to ship: a core rule may not declare `evidenceLevel: "E0"` in its own
|
|
316
|
+
metadata. `checkEvidenceHonesty` cannot catch it and the gap is structural —
|
|
317
|
+
it fails a rule for claiming _more_ than its `findingType` supports, and E0 is
|
|
318
|
+
the bottom of the ladder.
|
|
319
|
+
- **`tests/corpus/verdicts/proposed/`** — the staging area for adjudications
|
|
320
|
+
awaiting ratification. Committed verdicts are immutable
|
|
321
|
+
(`tests/corpus/verdicts/README.md:191`), so staging is the only reversible
|
|
322
|
+
form.
|
|
323
|
+
|
|
324
|
+
#### Changed
|
|
325
|
+
|
|
326
|
+
- **The core tier carries a floor** (ADR 0014). `capForTier` becomes
|
|
327
|
+
`policyForTier`; core gains `CORE_FLOOR = { evidenceLevel: "E1" }`. Until this,
|
|
328
|
+
core and extended were enforced _identically_ — `capForTier` returned `null`
|
|
329
|
+
for every non-quarantine tier, so reaching core changed the dedup tie-break,
|
|
330
|
+
the M3 ceiling and the census readout, and nothing a scan consumer could see.
|
|
331
|
+
`E1` and not `E2`: E2 is a claim that a finding is _proven_, which is what
|
|
332
|
+
runtime corroboration asserts per finding, not what a tier policy may.
|
|
333
|
+
Severity is deliberately untouched — one bounded field, changed on purpose,
|
|
334
|
+
two is a rewrite.
|
|
335
|
+
- **`SCORING_MODEL_VERSION` 1.0.0 → 2.0.0.** A core finding's deduction moves
|
|
336
|
+
from 0 (E0) to `floor(base/2)` (E1). `semver + ADR required` per the entry's
|
|
337
|
+
own policy. **`TRUST_MODEL_VERSION` stays at `1.0.0`** and ADR 0014 states why
|
|
338
|
+
rather than leaving it implicit: `deriveTrustLevel` _does_ consume
|
|
339
|
+
`evidenceLevel`, but the E→L mapping is unchanged — what moved is the
|
|
340
|
+
distribution of inputs, and a consumer reading `evidenceLevel` sees the same
|
|
341
|
+
fact. Bumping both would teach the version that "something moved".
|
|
342
|
+
- **An absent tier is no longer treated as core.** `policyForTier(undefined)`
|
|
343
|
+
returns an empty policy. `buildUniversalRules` resolves every _registry_ rule
|
|
344
|
+
through `effectiveTier` and adds a plugin only when it declares a tier, so a
|
|
345
|
+
missing entry means "a third-party rule that claimed nothing". The old
|
|
346
|
+
docstring said "undefined tier = core"; with a floor that would have raised
|
|
347
|
+
every plugin's E0 finding to E1 and started charging third-party detections —
|
|
348
|
+
the trust escalation TI-013 exists to prevent, introduced by the ADR written
|
|
349
|
+
to close a trust gap.
|
|
350
|
+
- **`npm run rules:quality:check` is a claim-integrity leaf.** It was declared
|
|
351
|
+
as the nightly gate `gate:rule-quality` and wired to nothing, so the gate ran
|
|
352
|
+
on a schedule that never reached it.
|
|
353
|
+
- **`check-fixture-quad` writes `docs/FIXTURE-QUAD.json` by default.** It wrote
|
|
354
|
+
only under `--write`, and nothing in this repository passes `--write`, so the
|
|
355
|
+
gate printed a census to a terminal nobody reads and left no artifact — a
|
|
356
|
+
backfill in progress was invisible in a diff. `--check` verifies the committed
|
|
357
|
+
artifact instead.
|
|
358
|
+
- **The measured/unmeasured census is 73/6, not 74/5.** Six documents asserted
|
|
359
|
+
74/5. Note the trap: 74 is _also_ the number of rules declaring a tier
|
|
360
|
+
explicitly (79 − 5 omitted), so "74" was accidentally right about a different
|
|
361
|
+
population.
|
|
362
|
+
|
|
363
|
+
#### Fixed
|
|
364
|
+
|
|
365
|
+
- **`src/rules/registry-census.ts` answered `stale` and `tier` with local
|
|
366
|
+
predicates, and both were wrong.** `stale` asked `MEASURED_FP[rule.id] !==
|
|
367
|
+
undefined` about a map the generator pre-filters to revision-matching rows, so
|
|
368
|
+
it was false for every stale rule and `staleCount` was structurally 0. `tier`
|
|
369
|
+
re-derived the ladder as `declared ?? (measured ? core : extended)` — a third
|
|
370
|
+
copy of the tier resolution and the only one not reading `effectiveTier` — so
|
|
371
|
+
the five rules with no declared tier printed `core` here and `extended`
|
|
372
|
+
everywhere else. A census that disagrees with doctor is not a census.
|
|
373
|
+
- **`scripts/v6/check-rule-quality.ts` had the mirror defect.** Its `stale` loop
|
|
374
|
+
re-derived from the revision-filtered map, so a stale row was never in it and
|
|
375
|
+
`measuredAtStaleRevision` read **0** while two rules (`QA-PY-004`, `QA-PY-007`)
|
|
376
|
+
carried hand-classified verdicts taken against an earlier detector. It now
|
|
377
|
+
reads **2**, and the registry skeleton agrees with the census at 73/6.
|
|
378
|
+
- **`checkQuarantineOwnership`, `checkAntiCreep` and the disposition read all
|
|
379
|
+
resolved files from `process.cwd()`.** With `doctor` reachable with no target,
|
|
380
|
+
CWD is wherever the user happened to be — their project, not the checkout — so
|
|
381
|
+
the audit read the wrong CHANGELOG and the wrong baseline. All three now take
|
|
382
|
+
the checkout root as an explicit parameter.
|
|
383
|
+
|
|
384
|
+
#### Not changed, and why
|
|
385
|
+
|
|
386
|
+
- **Retired-rule fixture directories were not deleted.** The plan called for
|
|
387
|
+
removing `tests/corpus/positive-fixtures/QA-PW-145/` and
|
|
388
|
+
`tests/corpus/negative-fixtures/QA-PY-102/`. That contradicts a binding owner
|
|
389
|
+
ruling (`docs/RULE-LIFECYCLE.md:193-196`, 2026-09-08, E-1): _"fixture dirs
|
|
390
|
+
stay on disk (the doctor's fixture-integrity check discloses them instead of
|
|
391
|
+
failing)"_, implemented at `src/commands/doctor.ts:1136-1139`. The directories
|
|
392
|
+
and the exemption stand; the ruling is recorded, not overruled.
|
|
393
|
+
- **No core rule was promoted.** See below.
|
|
394
|
+
- **`claim-registry.json` claim 2 stays BLOCKED.** `candidateSha` is `null`, and
|
|
395
|
+
the checker refuses a non-BLOCKED claim on an unproven candidate, so the
|
|
396
|
+
artifact/digest/authority the plan asked for could not be recorded without
|
|
397
|
+
asserting independence nobody observed. The blocker is narrowed to what is
|
|
398
|
+
actually missing: the split exists, but no holdout _evaluation_ has been run
|
|
399
|
+
against it, and it is still produced by this repository.
|
|
400
|
+
|
|
401
|
+
### Adjudication: 23 rows, 6 orphans, and the FIRST defect in the sampler
|
|
402
|
+
|
|
403
|
+
Every row was read at the corpus's pinned commit (`keycloak @ 421c23f5221d`,
|
|
404
|
+
`playwright-java @ c17b6060bd39`, `sveltejs-kit @ bf6833a639c9`), and each cited
|
|
405
|
+
line independently re-verified by enumerating every `@Ignore`/`@Disabled` /
|
|
406
|
+
`test.describe.serial` in the pinned file rather than trusting the ±6-line window.
|
|
407
|
+
|
|
408
|
+
**Six rows were orphans and were removed rather than judged** — and finding them
|
|
409
|
+
turned up a defect that made orphans inevitable:
|
|
410
|
+
|
|
411
|
+
> **`scripts/corpus-sample.ts` cloned the default branch and ignored `repo.ref`.**
|
|
412
|
+
> It ran `git clone --depth 1 <url>` — HEAD — then `rm -rf .git`, so nothing
|
|
413
|
+
> downstream could tell. Every verdict is adjudicated against the _pinned_ commit
|
|
414
|
+
> via `npm run corpus:verdict-context`. Those are two trees, and they drift.
|
|
415
|
+
>
|
|
416
|
+
> On keycloak that produced a repeating phantom: the pinned `JWETest.java` has
|
|
417
|
+
> exactly three `@Ignore`, on lines 73, 113 and 202, all three already adjudicated
|
|
418
|
+
> `TP` — and the sweep kept emitting rows citing **74, then 114, then 203**, the
|
|
419
|
+
> same three annotations cited one line late, each arriving as a fresh
|
|
420
|
+
> unclassified row that looked like new evidence.
|
|
421
|
+
>
|
|
422
|
+
> Four of those were caught by hand during the first adjudication pass. That was
|
|
423
|
+
> luck, not a control: a row whose drifted line happened to still land on a
|
|
424
|
+
> trigger would have been judged against source the verdict claims to describe and
|
|
425
|
+
> does not. `cloneRepo` now fetches `repo.ref` the way `tests/corpus/audit.ts`
|
|
426
|
+
> always did, **verifies** the checked-out SHA and hard-stops on a mismatch, and a
|
|
427
|
+
> cached clone is verified rather than trusted by existence.
|
|
428
|
+
|
|
429
|
+
Retracting a row also proved non-durable: `corpus-sample.ts` de-dupes on the
|
|
430
|
+
`ruleId|file|line` keys present in a `.jsonl`, so the instant an orphan was
|
|
431
|
+
removed the next sweep re-found and re-appended it. `apply-verdicts` now writes
|
|
432
|
+
**tombstones** to `tests/corpus/verdicts/retracted.jsonl` and the sampler reads
|
|
433
|
+
them alongside the verdicts — because an orphan rule that regenerates its own
|
|
434
|
+
evidence is not an orphan rule. The generator and its spec then needed the same
|
|
435
|
+
exclusion, since a tombstone has no `file`/`line` and parses as a verdict with
|
|
436
|
+
`undefined` fields.
|
|
437
|
+
|
|
438
|
+
**The two rules that earned core, and how:**
|
|
439
|
+
|
|
440
|
+
| Rule | n | ciHigh | the last row |
|
|
441
|
+
| ----------- | ----------: | -----------------: | -------------------------------------------------------------------------------------------- |
|
|
442
|
+
| `QA-PW-117` | 24 → **35** | 13.80% → **9.89%** | sveltejs-kit `test.describe.serial('Errors')` — six tests, each its own URL, no shared state |
|
|
443
|
+
| `QA-JV-101` | 23 → **35** | 14.31% → **9.89%** | keycloak `@Disabled("Only for PUT/POST")` on an **empty-bodied** override |
|
|
444
|
+
|
|
445
|
+
Both cleared by the same arithmetic — `z²/(n+z²)` ≤ 10% first holds at n=35 — and
|
|
446
|
+
both were funded by a corpus-wide `--core-candidates` sweep rather than by any
|
|
447
|
+
global cap raise. The `QA-JV-101` row is the purest instance of the rule's
|
|
448
|
+
diagnosis in the whole corpus: a disabled test with **no assertions** cannot hide
|
|
449
|
+
a defect, because it could not have detected one.
|
|
450
|
+
|
|
451
|
+
Neither sweep was unbounded: `CORE_CANDIDATE_CAP` stayed at 35 and the cap was not
|
|
452
|
+
raised to chase a number. `QA-JV-101`'s file carries two further
|
|
453
|
+
`@Disabled("Only for PUT/POST")` overrides at `:107` and `:113` that remain
|
|
454
|
+
**unfunded and unclassified**, because at n=35 the budget is zero. That is the cap
|
|
455
|
+
doing its job, not an oversight.
|
|
456
|
+
|
|
457
|
+
A third core rule now needs no hand record: `conformity`'s `CLAIM_OWNED` accepts a
|
|
458
|
+
claim the measurement derives independently, which is what `checkRegistry` and
|
|
459
|
+
`declaredCoreWithoutEvidence` already did. `QA-JV-101`'s entry in the 6.0 demotion
|
|
460
|
+
table also carried a justification describing a _different_ rule's premise
|
|
461
|
+
("static mutable shared across tests"); it is gone with the entry, and the
|
|
462
|
+
remaining eighteen are unaudited — named here as a finding, not fixed.
|
|
463
|
+
|
|
464
|
+
### Adjudication of the 23 sampled rows — staged, then applied
|
|
465
|
+
|
|
466
|
+
Every row was read at the corpus's pinned commit (`keycloak @ 421c23f5221d`),
|
|
467
|
+
following the convention the already-committed rows in the same file establish.
|
|
468
|
+
**19 rows are adjudicable; 4 are orphaned and are removed rather than judged.**
|
|
469
|
+
|
|
470
|
+
Three of the orphans are `QA-JV-101` rows citing `JWETest.java:74`, `:114` and
|
|
471
|
+
`:203`, where the `@Ignore` is on line 73/113/202 — the same annotations already
|
|
472
|
+
adjudicated `TP` one line up. A verdict on any of them would have counted one
|
|
473
|
+
annotation twice and moved a rule's `n` on the strength of a finding that is not
|
|
474
|
+
at that line. The fourth is `QA-PW-117` citing `details.spec.ts:20`, a
|
|
475
|
+
`const clientId = …` line with no `test.describe.serial` on or beside it; the
|
|
476
|
+
serial block at `:16` is already adjudicated.
|
|
477
|
+
|
|
478
|
+
### Added: a Conformity Monkey for the rule registry
|
|
479
|
+
|
|
480
|
+
Netflix's Simian Army does not decide what a good cloud is. It applies a set of
|
|
481
|
+
**declared** rules to every resource, marks the ones that do not conform, records
|
|
482
|
+
which rule broke on which resource, and notifies the owner. Three properties make
|
|
483
|
+
it work, and this gate reproduces all three:
|
|
484
|
+
|
|
485
|
+
- the human is **not in the detection loop** — a machine decides, every run, for
|
|
486
|
+
every rule;
|
|
487
|
+
- the human **is in the response loop** — a nonconformity is fixed, or explicitly
|
|
488
|
+
opted out, and the opt-out is recorded;
|
|
489
|
+
- **`leashed`** runs every check, writes the report and sends nothing. That is
|
|
490
|
+
the mode a new check ships in; arming is a separate, deliberate act.
|
|
491
|
+
|
|
492
|
+
`npm run conformity` (`scripts/check-conformity.ts`) computes four checks per
|
|
493
|
+
rule, all decidable by the tree — **no judgement, and therefore no queue**:
|
|
494
|
+
|
|
495
|
+
| Check | Question |
|
|
496
|
+
| ------------------ | ---------------------------------------------------------------------- |
|
|
497
|
+
| `MEASURED` | does a current measurement exist? |
|
|
498
|
+
| `DETECTOR_CURRENT` | was it taken at the detector revision that produced the finding? |
|
|
499
|
+
| `CLAIM_OWNED` | does a core/quarantine claim carry an owner and an unexpired date? |
|
|
500
|
+
| `QUAD_COMPLETE` | do all four fixture legs exist? (RECALL and PRECISION are _generated_) |
|
|
501
|
+
|
|
502
|
+
**What it found: 17 of 79 rules are clean.** 62 fail at least one check, and the
|
|
503
|
+
sharpest one is about a field in this repository's own registry:
|
|
504
|
+
|
|
505
|
+
> `quarantinePromotion` carries the docstring "a rule in quarantine WITHOUT one is
|
|
506
|
+
> a rule whose quarantine is permanent by default, which is the outcome the tier
|
|
507
|
+
> exists to avoid" — and **0 of 34** quarantined rules have one.
|
|
508
|
+
|
|
509
|
+
Also: 2 rules carry a false-positive rate measured against an **earlier detector
|
|
510
|
+
revision**, displayed everywhere as if current; 6 have never been measured at all;
|
|
511
|
+
21 have an incomplete quad; 39 have no MUST-FIRE or MUST-NOT-FIRE fixture.
|
|
512
|
+
|
|
513
|
+
### Added: the opt-out registry as a ratchet, not a gate that is red forever
|
|
514
|
+
|
|
515
|
+
`docs/conformity-opt-outs.json` records every known nonconformity with a named
|
|
516
|
+
owner, a reason a reader can check, and **a date it lapses**. The gate fails on a
|
|
517
|
+
nonconformity with no entry, on an entry whose date has passed, and on an entry
|
|
518
|
+
for a rule that now conforms — so the set can only shrink, and known debt decays
|
|
519
|
+
on a schedule instead of persisting because nobody re-looked at it.
|
|
520
|
+
|
|
521
|
+
This is why it is a ratchet and not a plain gate. A gate that went red on all 62
|
|
522
|
+
rules on day one would stay red and read as noise — which is how raising
|
|
523
|
+
`MAX_SAMPLES_PER_RULE` produced **1,121 unadjudicated rows nobody looked at**.
|
|
524
|
+
Adding an opt-out is the only way to make the gate green for a known defect, and
|
|
525
|
+
an added entry is a **smaller diff than the evidence that would retire it**. That
|
|
526
|
+
asymmetry is the enforcement.
|
|
527
|
+
|
|
528
|
+
Expiry is the engine: 34 entries lapse **2026-11-01** and 28 lapse **2026-12-01**.
|
|
529
|
+
Nothing in the corpus opinion-measurement chain does, which is why the 23 pending
|
|
530
|
+
adjudications were debt with no date.
|
|
531
|
+
|
|
532
|
+
### Why this does not replace the corpus false-positive rate
|
|
533
|
+
|
|
534
|
+
That measurement asks a human "is this code legitimate?", which is the one
|
|
535
|
+
question no tree can answer. Deriving it from the detector's own declaration
|
|
536
|
+
would be circular — the detector fired, so the detector is right, so every rule's
|
|
537
|
+
rate is zero — and that is the shape GAP-M26-002 records: "a 429-row ledger with
|
|
538
|
+
zero judgement got certified against its own output". This gate checks what the
|
|
539
|
+
tree CAN answer; the opinion stays where it was.
|
|
540
|
+
|
|
541
|
+
### Fixed: an unreachable state, found by the gate that shipped it
|
|
542
|
+
|
|
543
|
+
The first version of `src/rules/conformity.ts` modelled "unmeasured" as a third
|
|
544
|
+
state beside CONFORMING and NONCONFORMING. The gate printed `unproven: 0` on its
|
|
545
|
+
first run — the state was **unreachable**, because every unmeasured rule here also
|
|
546
|
+
fails something else and so read NONCONFORMING. A law that can never fire is not a
|
|
547
|
+
law, and `registry-ratchet.spec.ts` already says that about its own floor.
|
|
548
|
+
`unmeasured` is now an orthogonal flag beside the state, and reports 6.
|
|
549
|
+
|
|
550
|
+
`--opt-outs=`/`--report=`/`--today=` exist because the tests need them, and each
|
|
551
|
+
one is a place the gate could have been right for the wrong reason: the first
|
|
552
|
+
fixture attempt copied the module graph behind `src/rules/index.ts` into a temp
|
|
553
|
+
tree and died in the ESM resolver before reading a single rule.
|
|
554
|
+
|
|
555
|
+
**Verified by planting each defect in the real registry before writing any of
|
|
556
|
+
this**: a lapsed date reported 34 gaps, a deleted entry reported one UNRECORDED,
|
|
557
|
+
and an entry added to a conforming rule reported one STALE. Each reverted to
|
|
558
|
+
`gaps: 0`. 20 arms in `tests/contract/conformity-gate.spec.ts` keep that.
|
|
559
|
+
|
|
560
|
+
One new npm script (`conformity`) and one new leaf in `gates:claim-integrity`.
|
|
561
|
+
`entry-points:check` still reports 9 commands against a ceiling of 12.
|
|
562
|
+
|
|
563
|
+
### Changed: the unclassified-backlog assertion is a two-sided ratchet, not a zero
|
|
564
|
+
|
|
565
|
+
`tests/contract/verdict-context.spec.ts` asserted **zero** unclassified verdict
|
|
566
|
+
rows, and 23 pending rows make that false for as long as the work takes — so the
|
|
567
|
+
assertion would have been red for the entire classification window.
|
|
568
|
+
|
|
569
|
+
It now asserts the live backlog **equals the figure `unclassified-ceiling.json`
|
|
570
|
+
records**: greater means the backlog grew without review, less-but-not-zero means
|
|
571
|
+
classification is half done and the commit captures a state nobody agreed to.
|
|
572
|
+
`generate-fp-audit-table` already enforces the same ceiling per FILE, so a second
|
|
573
|
+
file's first row cannot hide inside a generous total — which is why the assertion
|
|
574
|
+
here is on the total and the generator's is on the parts.
|
|
575
|
+
|
|
576
|
+
The count this file asserted before that was 28, and it existed so the number
|
|
577
|
+
could not go quietly stale. It has been replaced by an assertion that cannot go
|
|
578
|
+
stale in either direction, which is what it was for.
|
|
579
|
+
|
|
580
|
+
### Added: a fourth naming contract — a relative link that resolves to nothing
|
|
581
|
+
|
|
582
|
+
`check-cli-contract` now fails when a live markdown document links to a file that
|
|
583
|
+
does not exist. A link is a name, and it was the only kind of name none of the
|
|
584
|
+
three existing contracts could see: the removed-verb pass looks for verbs, the
|
|
585
|
+
reference pass looks for `npm run`, and a link to a document deleted two releases
|
|
586
|
+
ago looks like neither.
|
|
587
|
+
|
|
588
|
+
Its five live instances were all real:
|
|
589
|
+
|
|
590
|
+
| Document | Target | Now |
|
|
591
|
+
| --------------------------------- | ------------------------------ | ------------------------------- |
|
|
592
|
+
| `docs/PRODUCT-DECISIONS.md` (D-9) | `EXTERNAL-EVIDENCE-REQUEST.md` | reworded; points at the runbook |
|
|
593
|
+
| `docs/RELEASE-PATH-RUNBOOK.md` | `EXTERNAL-EVIDENCE-REQUEST.md` | reworded; names the retirement |
|
|
594
|
+
| `assets/brand/README.md` | `src/reporter/score-state.ts` | `presentation.ts` |
|
|
595
|
+
| `docs/design/BRAND-SYSTEM.md` | `src/reporter/score-state.ts` | `presentation.ts` |
|
|
596
|
+
| `docs/MAINTAINERS.md` | `CONTRIBUTING.md#governance` | `../CONTRIBUTING.md#governance` |
|
|
597
|
+
|
|
598
|
+
`isLiveSurface` is reused verbatim rather than reimplemented, so archives,
|
|
599
|
+
captured artefacts, frozen reports, preserved rule docs, `docs/adr/` and `tests/`
|
|
600
|
+
keep the exemptions they already had. A second exemption list would be a second
|
|
601
|
+
thing to drift from the first, and the drift shows up as a file protected from
|
|
602
|
+
one contract and not the other — exactly how
|
|
603
|
+
`docs/RELEASE-3.0.0-READINESS.md` came to be exempt from one and not the other.
|
|
604
|
+
|
|
605
|
+
Scope is markdown, inline links and images, path resolved and anchor not, on
|
|
606
|
+
measurements rather than guesses: a link-reference definition whose value is
|
|
607
|
+
itself a link (`[^pit]: PIT, [PIT](https://pitest.org)`) is a footnote, and
|
|
608
|
+
guessing produced **30 false positives in one research document** on the first
|
|
609
|
+
attempt. Fenced blocks and inline code spans are blanked because documentation
|
|
610
|
+
about markdown contains markdown. An extensionless target also resolves as
|
|
611
|
+
`<target>.md`, which is the site's clean-URL convention. 239 links are checked.
|
|
612
|
+
|
|
613
|
+
15 arms in `tests/contract/cli-contract-dangling-links.spec.ts`, each inherited
|
|
614
|
+
exemption asserted as an exemption. One of them was written backwards: I assumed
|
|
615
|
+
the pass excluded `docs/adr/` and it does — which is why ADR 0012's dangling link
|
|
616
|
+
had to be found by hand. The arm now records that.
|
|
617
|
+
|
|
618
|
+
**Not fixed, and named:** `scripts/check-external-evidence.ts` still reads
|
|
619
|
+
`docs/EXTERNAL-EVIDENCE-REQUEST.md`, which the 6.0 M26-M50 retirement deleted, so
|
|
620
|
+
`npm run docs:external-evidence` exits 2 and `npm run certify:integrity` cannot
|
|
621
|
+
complete. Its `EXTERNAL_BOX_CEILING = 39` is a number about a file that no longer
|
|
622
|
+
exists, and lowering it to the 18 rows the runbook actually carries would be a
|
|
623
|
+
recorded number lowered to make a gate pass. D-9 now says so in the document
|
|
624
|
+
instead of continuing to cite a deleted file as the authority for it.
|
|
625
|
+
|
|
626
|
+
### Added: every workflow step's file argument resolves
|
|
627
|
+
|
|
628
|
+
`workflow:scripts` answers "does this NAME resolve". The other half of a step is
|
|
629
|
+
"does this FILE resolve", and `.github/workflows/ci.yml` shipped a step naming
|
|
630
|
+
`scripts/diff-detector-hashes.ts` — a file that has never existed. The capability
|
|
631
|
+
was present throughout (`scripts/check-detector-hashes.ts` documents
|
|
632
|
+
`--base <manifest>`) and the step has been corrected to name it, but a corrected
|
|
633
|
+
line is not an invariant and nothing said so. **11 file arguments across 21
|
|
634
|
+
workflows now resolve**, against the same `working-directory` resolution and the
|
|
635
|
+
same `EXEMPT_FROM_SCRIPT_CHECK` list the name check uses.
|
|
636
|
+
|
|
637
|
+
Verified on the real tree before it was codified: plant the typo in `ci.yml`, the
|
|
638
|
+
gate reports the file and the job, revert, the gate passes. 8 arms in
|
|
639
|
+
`tests/contract/workflow-scripts.spec.ts` keep that.
|
|
640
|
+
|
|
641
|
+
Two scope decisions are load-bearing. A token counts as a file only when it is a
|
|
642
|
+
bare relative path with a code extension after an interpreter — otherwise
|
|
643
|
+
`./actionlint`, which the previous step downloads, would be reported missing on
|
|
644
|
+
every run forever, and the fix would be an exemption. And `dist/` is a **build
|
|
645
|
+
output**, not a repository file: `node dist/cli.mjs` is how six workflows invoke
|
|
646
|
+
the CLI, so checking it would make this gate red on a clean checkout before
|
|
647
|
+
`npm run build` — which is what `npm run certify:fast` and a bare
|
|
648
|
+
`npm run workflow:scripts` see.
|
|
649
|
+
|
|
650
|
+
### Fixed: the surface scan counted every document under `docs/` twice
|
|
651
|
+
|
|
652
|
+
`check-cli-contract` walked `"", "docs", "site", ".github/workflows"`, and
|
|
653
|
+
`walk("")` is a walk of the repository root that already descends into the other
|
|
654
|
+
three. Every file under `docs/` and `site/` was read twice and reported twice, so
|
|
655
|
+
the printed `surfacesScanned` overstated the surface the gate checks.
|
|
656
|
+
|
|
657
|
+
### Added: ADR 0013, and a kill criterion that is not only a command count
|
|
658
|
+
|
|
659
|
+
`docs/adr/0013-the-core-ceiling-is-decided.md` records what two independently
|
|
660
|
+
chosen constants implied and nobody decided. `src/rules/measurement.ts` carried a
|
|
661
|
+
docstring reading `!! UNREACHABLE AT THE CURRENT CORPUS CAP` — arithmetically
|
|
662
|
+
correct, and a product decision written as a property of the code. It said _don't
|
|
663
|
+
bother_ where it should have said _this is unresolved_.
|
|
664
|
+
|
|
665
|
+
The ADR holds the ceiling at 10%, keeps `MAX_SAMPLES_PER_RULE` at 20, and gives
|
|
666
|
+
funded candidates the cap the arithmetic requires. It also records what it does
|
|
667
|
+
**not** decide: re-homing the 39 external-evidence boxes, and anything a core rule
|
|
668
|
+
might need beyond the quad and the interval.
|
|
669
|
+
|
|
670
|
+
`docs/ROADMAP.yaml`'s 6.0 kill criterion said `npm run check` must fit in 12
|
|
671
|
+
commands. A command count is necessary and not sufficient — the tier system has
|
|
672
|
+
to have a reachable top, or the cut only removed the words. It now says so, names
|
|
673
|
+
the arithmetic, and points at the ADR.
|
|
674
|
+
|
|
675
|
+
### Added: `--core-candidates`, a sampling mode that spends the budget where it can be spent
|
|
676
|
+
|
|
677
|
+
The core ceiling is 10% on the Wilson upper bound, so a rule earns core at
|
|
678
|
+
**n ≥ 35 with zero observed false positives** (`samplesForZeroFp`). The corpus
|
|
679
|
+
sampler capped at 20, so the tier was unreachable — and 6.0 found out the
|
|
680
|
+
expensive way that raising the cap globally does not fix that: a raise to 40
|
|
681
|
+
produced **1,121 unadjudicated rows across 42 rules** and
|
|
682
|
+
`unclassified-ceiling.json` refused them, correctly.
|
|
683
|
+
|
|
684
|
+
The global raise was the wrong ALLOCATION, not the wrong idea. It spends 35+
|
|
685
|
+
samples on rules that cannot use them — a rule already observed wrong, or one
|
|
686
|
+
whose interval already excludes 10% from below — while the adjudication budget is
|
|
687
|
+
finite and belongs to a person.
|
|
688
|
+
|
|
689
|
+
- `--core-candidates` emits rows **only** for a rule that observes zero false
|
|
690
|
+
positives and whose `straddleDetail` reports that more clean samples would
|
|
691
|
+
settle it, and only those rules may draw up to 35 instead of 20. The predicate
|
|
692
|
+
is derived from the live registry on every run
|
|
693
|
+
(`scripts/lib/core-candidates.ts`), so a rule crossing the line stops being
|
|
694
|
+
sampled without an edit.
|
|
695
|
+
- `--core-target <RULE-ID>` (repeatable) says which candidates this pass
|
|
696
|
+
**funds**. The predicate is worth **29 rules** today; the nearest two are 11 and
|
|
697
|
+
12 rows from the ceiling. Conflating "who can earn a tier" with "who gets
|
|
698
|
+
funded this pass" would move the ceiling by an amount nobody decided, so the
|
|
699
|
+
two are separate and naming a rule that is not a candidate is an error rather
|
|
700
|
+
than a silent no-op.
|
|
701
|
+
|
|
702
|
+
Filtering is on **emission**, not only on the cap. A cap-only change would still
|
|
703
|
+
emit rows for every rule below n=20 — the 1,121-row failure with a different
|
|
704
|
+
number on it.
|
|
705
|
+
|
|
706
|
+
No npm script, CLI verb, gate or rule was added.
|
|
707
|
+
|
|
708
|
+
### Fixed: three defects the first real `--core-target` run found
|
|
709
|
+
|
|
710
|
+
- **The mode asked for twice the work it promised.** `tests/corpus/verdicts/unclassified-ceiling.json`
|
|
711
|
+
moves by the number of rows a person is shown _before_ they are asked to
|
|
712
|
+
classify anything, so that number has to be exact. The first run collected 35
|
|
713
|
+
samples per rule and appended **46** rows against a projection of 23, because
|
|
714
|
+
de-duplication is per verdict FILE and keycloak's file already held 19
|
|
715
|
+
`QA-PW-117` and 10 `QA-JV-101` rows. The budget is now on **rows**, not
|
|
716
|
+
samples — `cap − classified − already pending` — computed against the same key
|
|
717
|
+
set the writer de-dupes with, by one shared function. Re-running the identical
|
|
718
|
+
command now adds **0** rows, which is the property that stops a scoped pass
|
|
719
|
+
from silently exceeding the ceiling twice.
|
|
720
|
+
- **A scoped run deleted the review sheets it claimed to preserve.**
|
|
721
|
+
`writeReviewSheets` carried a comment describing exactly this protection while
|
|
722
|
+
the condition underneath did the opposite: `!sampledRules.has(id)` is true for
|
|
723
|
+
every rule a `--repo` or `--core-candidates` run did not visit. A one-repo
|
|
724
|
+
targeted pass deleted 8 committed sheets (`QA-PY-002/003/004/007/009/011/012`
|
|
725
|
+
and one more) of pending owner work. Sheets are now removed only for rules that
|
|
726
|
+
are no longer live; a rule that was visited and sampled nothing still removes
|
|
727
|
+
its own sheet, because "the queue is empty" is a different statement from
|
|
728
|
+
"nobody looked".
|
|
729
|
+
- **The unclassified-ceiling gate named the wrong cause.** It throws
|
|
730
|
+
`(23 > 31)` for a run whose total was 23 and whose ceiling was 31, because the
|
|
731
|
+
trigger is usually a **per-file** regression — a new file's first row exceeds
|
|
732
|
+
its allowance of 0 while the total stays inside the ceiling — and the message
|
|
733
|
+
only ever compared the two totals. It now says which condition fired and names
|
|
734
|
+
the files.
|
|
735
|
+
|
|
736
|
+
### Changed: the unclassified ceiling, re-recorded at its reviewed value
|
|
737
|
+
|
|
738
|
+
`31 → 23`, and `byFile` drops three stale entries for files whose pending rows
|
|
739
|
+
have since been adjudicated away (`pallets-click` 1, `pytest-dev-pytest` 27,
|
|
740
|
+
`new-repo.jsonl` 3) in favour of a new allowance for `keycloak-keycloak.jsonl`.
|
|
741
|
+
|
|
742
|
+
The 23 rows are the scoped pass above: `QA-PW-117` +11 (n=24 → 35) and
|
|
743
|
+
`QA-JV-101` +12 (n=23 → 35), awaiting human adjudication per
|
|
744
|
+
`tests/corpus/verdicts/README.md`. **Not** 31 → 54: the live backlog had been
|
|
745
|
+
classified down to 0 since the 31 was recorded, so this pass is 0 → 23, and a
|
|
746
|
+
ceiling of 54 would have authorised 31 rows of pending work nobody asked for.
|
|
747
|
+
Zero slack is the correct reading — the next pass must be acknowledged too.
|
|
748
|
+
|
|
749
|
+
`measured-fp.generated.ts`, `docs/FP-AUDIT.md` and `docs/COUNT-LOCK.md` are
|
|
750
|
+
unchanged, which is the point: blank rows are dropped, not counted, so 23
|
|
751
|
+
unadjudicated findings move no rate until a person classifies them.
|
|
752
|
+
|
|
753
|
+
### Changed: both plan targets already had a complete fixture quad
|
|
754
|
+
|
|
755
|
+
`docs/CORE-READINESS.md` lists `QA-PW-117` and `QA-JV-101` as needing the
|
|
756
|
+
four-leg quad. Measured against the tree, both already have all four:
|
|
757
|
+
`MUST-FIRE` and `MUST-NOT-FIRE` fixtures on disk, 17 and 7 adjudicated `TP` rows
|
|
758
|
+
and 12 and 8 adjudicated `TN` rows with zero `FP` in
|
|
759
|
+
`tests/corpus/verdicts/quad/`. Nothing was authored, and the 10% ceiling still
|
|
760
|
+
does not unblock 6.0.0 on its own — D-1 needs both the interval and the quad, and
|
|
761
|
+
only the interval is missing.
|
|
762
|
+
|
|
763
|
+
### Added: agent skills, and the wiring that makes them load
|
|
764
|
+
|
|
765
|
+
The repository had four agent skills under `.claude/skills/` and no way for
|
|
766
|
+
Kilo to see them. Kilo reads that directory only when the user-level Claude
|
|
767
|
+
Code Compatibility setting is on, so on a default install the committed skills
|
|
768
|
+
were dead weight in that harness — and `.kilo/` cannot hold them instead,
|
|
769
|
+
because `.gitignore` ignores it as machine-local agent state. `kilo.json` now
|
|
770
|
+
carries a `skills.paths` entry pointing at `.claude/skills/`: one tracked copy,
|
|
771
|
+
both harnesses, and the wiring visible in a diff. `docs/EXTERNAL-CONFIG.md`
|
|
772
|
+
records what that file now declares.
|
|
773
|
+
|
|
774
|
+
Nine skills were added alongside the existing four, all written against this
|
|
775
|
+
repository's own gates rather than imported wholesale:
|
|
776
|
+
|
|
777
|
+
- `rule-author` — the full rule lifecycle: ID allocation and non-reuse, the
|
|
778
|
+
must-fire and must-not-fire fixture firewall, the four quad legs, registry
|
|
779
|
+
registration, the gates a rule change invalidates, and the
|
|
780
|
+
deprecation/retirement path from `docs/RULE-LIFECYCLE.md`. The 19-line Copilot
|
|
781
|
+
agent in `.github/agents/` covered the first half of this and nothing else.
|
|
782
|
+
- `change-gate` — a diff-to-checks map, because `npm run check` is ~8 minutes
|
|
783
|
+
and the lever that moves it is running the row that covers the change. Names
|
|
784
|
+
only scripts `package.json` defines, which
|
|
785
|
+
`tests/contract/docs-consistency.spec.ts` greps every tracked `.md` for.
|
|
786
|
+
- `surface-law` — law 0 as a procedure: what counts as the governed surface,
|
|
787
|
+
the twelve-leaf ceiling, and the two legal ways to grow.
|
|
788
|
+
- `evidence` — the agent safety contract, including the rule that
|
|
789
|
+
`INCONCLUSIVE` is a result and never laundered into a pass.
|
|
790
|
+
- `skill-author`, `verification-before-completion` — adapted from
|
|
791
|
+
`obra/superpowers` (MIT), the parts that decide whether a skill gets opened
|
|
792
|
+
and whether a claim gets evidence.
|
|
793
|
+
- `prose-honesty` — adapted from `blader/humanizer` (MIT), fenced so it cannot
|
|
794
|
+
touch a generated page, an asserted surface, or a registered claim.
|
|
795
|
+
- `quarantine-remediation` — the 34 quarantined rules are capped to
|
|
796
|
+
`severity: info` and `evidence: E0`, so they cannot fail a build; 34 of 79
|
|
797
|
+
rules are in that state and seven measure 0.0 % FP. `doctor` reports that
|
|
798
|
+
**none** carries a `quarantinePromotion`, so none has an owner, a review date
|
|
799
|
+
or an exit condition — the mechanism exists on the `Rule` type and has never
|
|
800
|
+
been applied. The skill covers the ladder, the Wilson-interval criterion that
|
|
801
|
+
replaced the point estimate in 6.0, and the promote/rework/retire decision.
|
|
802
|
+
- `skill-supply-chain` — the scan procedure for a skill before it is installed,
|
|
803
|
+
since a skill is code that runs with the agent's permissions.
|
|
804
|
+
|
|
805
|
+
No npm script, CLI verb, gate or rule was added, so law 0's ratchet does not
|
|
806
|
+
move.
|
|
807
|
+
|
|
808
|
+
### Fixed: `check-cli-contract` and `docs-consistency` disagreed about one file
|
|
809
|
+
|
|
810
|
+
`npm run check` was red on `main`: `check-cli-contract` failed on
|
|
811
|
+
`docs/RELEASE-3.0.0-READINESS.md`, which names `npm run m26:audit` in its
|
|
812
|
+
command table. `scripts/release-verify.ts` records that script as removed in
|
|
813
|
+
6.0 with the M26 ledger, so the name was real when the report was written and
|
|
814
|
+
the report is dated 2026-09-25.
|
|
815
|
+
|
|
816
|
+
`tests/contract/docs-consistency.spec.ts` already exempted exactly that file, by
|
|
817
|
+
name, with the reasoning written out: a readiness report for a version that
|
|
818
|
+
shipped is a record, not an instruction. Two gates catch the same class of
|
|
819
|
+
dangling name, and the file was exempt from one that still tripped the other —
|
|
820
|
+
an exemption made by accident rather than by decision. `check-cli-contract.mjs`
|
|
821
|
+
now carries the same entry in a named `FROZEN_REPORTS` list, mirroring the
|
|
822
|
+
existing `CAPTURED_ARTEFACTS` shape, and skipping it in both the removed-verb
|
|
823
|
+
pass and the dangling-name pass. The list is by name, not by pattern, so it can
|
|
824
|
+
be argued with per file.
|
|
825
|
+
|
|
826
|
+
`tests/contract/cli-contract-dangling-names.spec.ts` locks both halves, because
|
|
827
|
+
an exemption nobody tests is a blind spot with a comment on it: a dangling name
|
|
828
|
+
in a live surface still fails, the frozen report still passes, a live doc beside
|
|
829
|
+
the frozen one still fails, and the real tree passes.
|
|
830
|
+
|
|
831
|
+
### Decided against, on measurement: the 500-word skill budget
|
|
832
|
+
|
|
833
|
+
`skill-author` carried an upstream rule of "under 500 words". Eight of eight new
|
|
834
|
+
skills broke it on the first pass, from 721 to 1222 words — which is the shape
|
|
835
|
+
of a target nobody measured rather than a target anybody met. The figure was
|
|
836
|
+
wrong for this class of skill: most of the excess is procedure that only runs
|
|
837
|
+
once, and in `prose-honesty` a 22-row lookup catalogue that nobody reads to fix
|
|
838
|
+
one sentence.
|
|
839
|
+
|
|
840
|
+
So the rule now sorts by shape — a procedure carries a word budget, a lookup
|
|
841
|
+
table is always a sibling file — and the number is measured rather than
|
|
842
|
+
guessed. Two lookup structures moved out: `prose-honesty`'s catalogue to
|
|
843
|
+
`patterns.md` (body 126 → 85 lines), and `skill-author`'s failure-classification
|
|
844
|
+
table plus its no-baseline-run fallback to `baseline-failure.md` (204 → 166
|
|
845
|
+
lines). Both are cases the rule's own branch rule covers: a catalogue is
|
|
846
|
+
scanned for one row, and the fallback is only read when there is no baseline to
|
|
847
|
+
classify.
|
|
848
|
+
|
|
849
|
+
The ceiling lives in `budget.md` rather than in `skill-author`'s body, and that
|
|
850
|
+
placement is the part worth keeping. A meta-skill that states its own budget
|
|
851
|
+
counts toward it: writing "1250 words" into the body made the body 1251. Moving
|
|
852
|
+
the figure one file over removes the self-reference, so the ceiling can be
|
|
853
|
+
1300 against a measured maximum of 1264 and stay checkable.
|
|
854
|
+
|
|
855
|
+
The record that the number moved, when, and against what measurement sits in
|
|
856
|
+
`budget.md`. A ceiling that moves silently is the arithmetic of law 0's
|
|
857
|
+
`baselineCore` edit: the arithmetic reads zero and the law reads as satisfied.
|
|
858
|
+
|
|
859
|
+
### The baseline-failure step: a discoverability audit, and its limit
|
|
860
|
+
|
|
861
|
+
`skill-author` prescribes a RED step — watch an agent fail the task before
|
|
862
|
+
writing the skill. That did not run; these skills were written from
|
|
863
|
+
`docs/RULE-LIFECYCLE.md`, `CLAUDE.md` and the gate sources. The substitute that
|
|
864
|
+
could run is a discoverability audit: for each load-bearing fact, is it
|
|
865
|
+
reachable from the repository today?
|
|
866
|
+
|
|
867
|
+
| Fact | Reachable from |
|
|
868
|
+
| -------------------------- | ---------------------------------------------------------------- |
|
|
869
|
+
| a path→gates map | `docs/archive/` only — nothing live |
|
|
870
|
+
| directory → rule-ID prefix | `src/rules/index.ts`; no live doc states the mapping |
|
|
871
|
+
| the four quad legs | `src/v6/fixture-quad-probe.ts` and an archived plan |
|
|
872
|
+
| rule IDs are never reused | `docs/RULE-LIFECYCLE.md`, `docs/ARCHITECTURE.md`, `src/types.ts` |
|
|
873
|
+
|
|
874
|
+
So `change-gate`'s content was nowhere to be found, the prefix table and the
|
|
875
|
+
quad vocabulary were reachable only from source, and the ID non-reuse rule was
|
|
876
|
+
already documented — that line is a guardrail kept for cost, not for novelty.
|
|
877
|
+
|
|
878
|
+
This proves the knowledge was not already available. It does not prove the
|
|
879
|
+
skills work; only watching an agent use one answers that. `skill-author` now
|
|
880
|
+
records the substitute and its limit, so the next author knows which of the two
|
|
881
|
+
they actually ran.
|
|
882
|
+
|
|
883
|
+
### Fixed: the quarantine report named a field that does not exist
|
|
884
|
+
|
|
885
|
+
`CorePromotion` declares five fields — `rationale`, `owner`, `grantedAt`,
|
|
886
|
+
`expiresOn`, `evidenceRefs`. There is no exit-condition field, and `rg` finds
|
|
887
|
+
the phrase nowhere in `src/`, `docs/` or `tests/` except in two places that
|
|
888
|
+
were both telling the reader to go looking for one: the `quarantinePromotion`
|
|
889
|
+
doc comment in `src/rules/rule.ts`, and the detail message
|
|
890
|
+
`checkQuarantineOwnership` prints about the largest open item in the tree.
|
|
891
|
+
|
|
892
|
+
A report that describes a shape the contract does not define is the same
|
|
893
|
+
defect class as a rule reporting a metric it never measured — the reader is
|
|
894
|
+
sent to do work against a field that is not there. The message now names the
|
|
895
|
+
three a backfilled record actually has to supply, and the expiry date is
|
|
896
|
+
labelled as what it is, since it is the exit condition in substance.
|
|
897
|
+
|
|
898
|
+
`tests/contract/quarantine-record-honesty.spec.ts` locks it by parsing the
|
|
899
|
+
field list out of the type rather than restating it, so it fails if the type
|
|
900
|
+
gains a field the message omits and if the message ever names a phantom one.
|
|
901
|
+
Verified by mutation: all three arms fail against the previous wording.
|
|
902
|
+
|
|
903
|
+
### Verification
|
|
904
|
+
|
|
905
|
+
`npm run test` is 11,992 passing with zero failures across 493 files. `npm run
|
|
906
|
+
lint`, `typecheck`, `check-version`, `claims:check`,
|
|
907
|
+
`candidate:manifest:check`, `entry-points:check`, `config:consumers`,
|
|
908
|
+
`scripts:reachable`, `gates:check`, `check-concepts`, `check-cli-contract`,
|
|
909
|
+
`check-certification-surface`, `check-detector-hashes`, `check-fixture-quad`,
|
|
910
|
+
`docs:provenance-drift`, `check-carve-manifest`, `qa-ir:parity` and
|
|
911
|
+
`docs:roadmap:backlinks` all pass.
|
|
912
|
+
|
|
913
|
+
### Corrected: what `npm run check` was actually red on
|
|
914
|
+
|
|
915
|
+
The previous version of this entry named a coverage threshold and blamed
|
|
916
|
+
`src/commands/share.ts`. Both halves were wrong, and neither was checked:
|
|
917
|
+
|
|
918
|
+
- `frontier:contracts` is **not** a leaf of `check`. `package.json` has `check`
|
|
919
|
+
as nine terms (`build`, `typecheck`, `lint`, `test:coverage:ci`,
|
|
920
|
+
`check-version`, `gates:claim-integrity`, `docs:regen`, `self-scan`,
|
|
921
|
+
`doctor`); `frontier:contracts` (`vitest run tests/contract/`) is declared in
|
|
922
|
+
no chain at all, and `certify` invokes that command directly rather than
|
|
923
|
+
through the name.
|
|
924
|
+
- A bare `vitest run tests/contract/` collects no coverage — `--coverage` is
|
|
925
|
+
what enables the provider — so `perFile: true` could not be the mechanism
|
|
926
|
+
either.
|
|
927
|
+
|
|
928
|
+
What was actually red was two generated artifacts that had not been refreshed:
|
|
929
|
+
|
|
930
|
+
- `docs/BLAST-RADIUS-AUDIT.md` recorded `src/commands` at 11,988 lines against
|
|
931
|
+
a live 11,990, because the `GATE_COMMAND` de-duplication in `share.ts`
|
|
932
|
+
removed two lines from a file the audit counts.
|
|
933
|
+
- `candidate-trust-manifest.json` failed `workingTreeSha256 drift`, which is
|
|
934
|
+
what it is for: it hashes the tree it was stamped against.
|
|
935
|
+
|
|
936
|
+
Both are the same failure mode as the exemption added above — an artifact that
|
|
937
|
+
describes a state nobody refreshed — and `scripts/refresh-generated.mjs` is the
|
|
938
|
+
tool that fixes them, in the one order that converges.
|
|
939
|
+
|
|
940
|
+
### Fixed: the convergence tool could not converge
|
|
941
|
+
|
|
942
|
+
`scripts/refresh-generated.mjs` ran `docs:translations:sync` and
|
|
943
|
+
`docs:translations`, both deleted by the v6 positioning carve (#697) along with
|
|
944
|
+
the 20 translated READMEs, and formatted `README.*.md`, a glob that has matched
|
|
945
|
+
nothing since those READMEs went. Prettier exits 2 on a pattern with no files,
|
|
946
|
+
so the refresh reported `FAIL` on three entries and exited non-zero: the tool
|
|
947
|
+
whose entire job is "make the tree agree with its generators" could not be run
|
|
948
|
+
to completion.
|
|
949
|
+
|
|
950
|
+
Nothing caught it. `npm run check` does not run this file,
|
|
951
|
+
`scripts:reachable` answers the opposite question — is a script reachable FROM
|
|
952
|
+
a caller, not does every name it invokes resolve — and `check-cli-contract`'s
|
|
953
|
+
dangling-name rule reads `.md`, workflows, the Action and the reviewer config,
|
|
954
|
+
not `scripts/**`. Two gates, two different blind spots, one script that could
|
|
955
|
+
not do the one thing it exists to do.
|
|
956
|
+
|
|
957
|
+
The dead entries are removed and the glob is `README.md`. The `npm run <name>`
|
|
958
|
+
inside a `.mjs` gap is **not** closed here: the shape a general fix would need
|
|
959
|
+
was measured, and the placeholder convention this repository already uses —
|
|
960
|
+
`scripts/check-ci-local-parity.mjs` writes `npm run X` chains and `npm run Y`
|
|
961
|
+
prose — is indistinguishable from an invocation by regex alone. Recorded rather
|
|
962
|
+
than shipped as a gate that reports its own source.
|
|
963
|
+
|
|
964
|
+
### Decided against, on measurement: the script-layer cut
|
|
965
|
+
|
|
966
|
+
`npm run check` is ~8 minutes, and the test suite is 79 % of it. All 38 gates
|
|
967
|
+
together cost 100 seconds, so cutting the gate surface from 39 to 12 — the plan's
|
|
968
|
+
`~60 script files / ~45 npm scripts` target — would save under 15 % of the wait,
|
|
969
|
+
while deleting checks that caught six real defects during 6.0. The numbers, the
|
|
970
|
+
arithmetic and the alternative levers are recorded in the `6.0.0-rc.1` entry
|
|
971
|
+
below and in `CLAUDE.md` law 0, so the question no longer has to be re-argued
|
|
972
|
+
from file counts.
|
|
973
|
+
|
|
974
|
+
The verification trust engine, measured. A tier you can only reach with
|
|
975
|
+
evidence, a promotion that cannot land unreviewed, and generated documentation
|
|
976
|
+
that cannot quietly go stale behind the code. Carries the 6.0.0-rc.1 line below.
|
|
977
|
+
|
|
978
|
+
### Measured twice: the core promotions, and what they rest on
|
|
979
|
+
|
|
980
|
+
Two rules hold `core`, and this release is where the arithmetic behind them was
|
|
981
|
+
checked rather than asserted. Both sit at **n=35, zero observed false positives,
|
|
982
|
+
Wilson upper bound 9.89%** against a 10% ceiling — reproduced by hand from
|
|
983
|
+
`src/lib/wilson.ts` before either was believed.
|
|
984
|
+
|
|
985
|
+
`QA-JV-101` earned it in two batches, and the sequence matters:
|
|
986
|
+
**32 (10.72%) → 34 (10.15%) → 35 (9.89%)**. The first batch's verdict file reads
|
|
987
|
+
as a refusal on its own — "STILL SHORT, NOT promoted" — and the second, from a
|
|
988
|
+
sweep run after `corpus-sample.ts` was corrected to clone the pinned ref, is
|
|
989
|
+
what cleared the ceiling. Read one batch and you conclude the promotion was
|
|
990
|
+
fabricated; read both and it is earned. Both verdict files are cited in the
|
|
991
|
+
certification record for exactly that reason.
|
|
992
|
+
|
|
993
|
+
The sampler fix that made batch 2 trustworthy works _against_ inflation: it
|
|
994
|
+
clones `repo.ref` instead of the default branch and makes retractions stick, so
|
|
995
|
+
an orphan row cannot resurrect as fresh evidence. Three false rows (JWETest
|
|
996
|
+
74/114/203) were retracted rather than judged.
|
|
997
|
+
|
|
998
|
+
### The tree told a different story than the code
|
|
999
|
+
|
|
1000
|
+
Both promotions landed in the registry, the tier evidence and this file while
|
|
1001
|
+
their generated documentation went on reporting `extended` at n=23 and n=24.
|
|
1002
|
+
Nothing failed, because nothing asked.
|
|
1003
|
+
|
|
1004
|
+
- **A rule count nobody can check is a claim, so generate it.** `README.md` said
|
|
1005
|
+
"`core` is empty" for a whole release after two rules had earned it.
|
|
1006
|
+
`measurementBlock()` now returns `core` and `docs:counts` stamps it into the
|
|
1007
|
+
census surfaces, so `tests/contract/census-drift.spec.ts` fails if prose and
|
|
1008
|
+
registry disagree. It counts UNMEASURED core rules deliberately:
|
|
1009
|
+
`MAX_UNMEASURED_CORE` is 0, so a registry in that state cannot ship, and a
|
|
1010
|
+
census filtered on `measured` would report a clean `core: 0` at the exact
|
|
1011
|
+
moment the law broke.
|
|
1012
|
+
- **`docs:staleness` — one staleness assertion, runnable locally.** Four
|
|
1013
|
+
workflows each asserted it by hand and three could not see an untracked file:
|
|
1014
|
+
with clean tracked `docs/` and one new untracked file in it,
|
|
1015
|
+
`git diff --exit-code -- docs/` exits **0**. That is how a new rule's docs page
|
|
1016
|
+
shipped silently. The replacement reads `git status`, is read-only by
|
|
1017
|
+
construction so it can never cost a tree its adjudication records, and is
|
|
1018
|
+
deliberately absent from `npm run check` — asserting after regenerating is
|
|
1019
|
+
checking the generator against itself.
|
|
1020
|
+
- **`docs/MEASUREMENT-CLOSEOUT.md` corrected to 73/6** (it read 74/5), gained
|
|
1021
|
+
the missing `QA-PY-004` row, and now separates **stale** from unmeasured:
|
|
1022
|
+
`QA-PY-004` (revision 4) and `QA-PY-007` (revision 5) have verdict sets that no
|
|
1023
|
+
longer describe the shipped detector and need re-measurement at the current
|
|
1024
|
+
revision — a different job from classifying new rows.
|
|
1025
|
+
|
|
1026
|
+
### A promotion can no longer land with no artifact
|
|
1027
|
+
|
|
1028
|
+
- **`docs/CORE-CERTIFICATION.json` — a record is required to hold `core`,** and
|
|
1029
|
+
the record expires with its evidence. The measurement block is compared field
|
|
1030
|
+
by field against the live `MEASURED_FP` row on every run: move the detector
|
|
1031
|
+
revision, change n, or introduce an observed false positive and the gate fails
|
|
1032
|
+
until someone re-certifies and writes down why. Every cited path must exist,
|
|
1033
|
+
and cited corpus rows must sum to exactly `n` — self-derived quad verdicts are
|
|
1034
|
+
excluded, which is why both records sum to exactly 35 (29+2+4, 20+10+2+3).
|
|
1035
|
+
- **QA-JV-101's record states its known weakness rather than omitting it.** Its
|
|
1036
|
+
must-not-fire fixture is a hand-written different program, not the must-fire
|
|
1037
|
+
fixture with `@Disabled` neutralised, so it shows the detector is directional
|
|
1038
|
+
and does **not** show the predicate is sensitive to the annotation.
|
|
1039
|
+
- **No new gate and no new npm script.** Both are arms of existing gates —
|
|
1040
|
+
`rules:promotion:check` and `check-fixture-quad` — because Law 0 charges for
|
|
1041
|
+
surface growth.
|
|
1042
|
+
|
|
1043
|
+
### The detector set is wired. It is not yet shown to have teeth.
|
|
1044
|
+
|
|
1045
|
+
`docs/PRECISION-RATCHET.json` records 32 rules as holding a PRECISION leg and
|
|
1046
|
+
calls itself "a wiring proof, not an accuracy proof, and a self-derived one".
|
|
1047
|
+
That caveat now has a number, and it is not comfortable.
|
|
1048
|
+
|
|
1049
|
+
A must-not-fire fixture is evidence that a rule is _sensitive_ only when it is
|
|
1050
|
+
the must-fire fixture with the defect neutralised. When it is a different program
|
|
1051
|
+
that merely does not trigger, the pair shows the detector is directional and
|
|
1052
|
+
nothing about the predicate. The quad counts presence, so both read alike.
|
|
1053
|
+
|
|
1054
|
+
**Of the 40 live rules carrying both legs, 2 hold a negative fixture that is
|
|
1055
|
+
plausibly the positive fixture with the defect removed** (`QA-PW-144`, one changed
|
|
1056
|
+
line of nine; `QA-PW-116`, one of seven). The other **38** are a different
|
|
1057
|
+
program — the next nearest diverges by 0.55, the furthest by more lines than
|
|
1058
|
+
either file contains. Leashed at the true floor of 2, so it fails on a rule
|
|
1059
|
+
_regressing_ out of that set and not on the 38 that were always like this;
|
|
1060
|
+
`--update` only ever raises the floor, so a regression cannot be absorbed by
|
|
1061
|
+
re-running a command. Closing the gap needs a per-rule recipe for what the defect
|
|
1062
|
+
_is_, and that work is named rather than assumed away.
|
|
1063
|
+
|
|
1064
|
+
### What a consumer may rely on
|
|
1065
|
+
|
|
1066
|
+
- **`docs/ENGINE-FREEZE.md`** records the compatibility promise and points at
|
|
1067
|
+
`CONTRACT_REGISTRY` rather than copying version numbers. `scoringModelVersion`
|
|
1068
|
+
2.0.0 is a _semantic_ change (ADR 0014's evidence floor), so a consumer
|
|
1069
|
+
comparing scores across it is comparing different things; and
|
|
1070
|
+
`frameworkSupportMatrixVersion` is written by the engine and read by nothing
|
|
1071
|
+
but a test — a version stamp on a fact, not a negotiated protocol.
|
|
1072
|
+
- **The delta loop is in the Quickstart.** Steps 1–3 report the _state_ of your
|
|
1073
|
+
tests; `mjolnir ci verify` reports the _change_, the only question a reviewer
|
|
1074
|
+
has about a diff. `--save-baseline` writes `.mjolnir/baseline.json`; `ci
|
|
1075
|
+
verify` reports RESOLVED (of which only `VERIFIED-RESOLVED` is a fix claim) and
|
|
1076
|
+
NEW, under a `0`/`1`/`2` exit contract. Agents were taught this at
|
|
1077
|
+
`src/commands/install-agents.ts:58`; humans had no way to find it.
|
|
1078
|
+
- **Governing prose that had gone stale behind the code** was corrected:
|
|
1079
|
+
`rule-author` and `surface-law` both asserted the core tier was empty, and the
|
|
1080
|
+
archived `RULE_CERTIFICATION.md` now carries a SUPERSEDED banner marking its
|
|
1081
|
+
`core` column a fossil of the deleted `registry-census` predicate.
|
|
1082
|
+
|
|
1083
|
+
### Not claimed
|
|
1084
|
+
|
|
1085
|
+
The five `REMOTE_PROVEN` requirements in `candidate-trust-manifest.json` —
|
|
1086
|
+
protected holdout, real-world repositories, platform matrix, consumer install,
|
|
1087
|
+
remote workflow — are **not** satisfied, and `releaseAuthorizationState` remains
|
|
1088
|
+
`NOT_AUTHORIZED`. `npm run candidate:readiness` reports exactly what is missing.
|
|
1089
|
+
That gate exists so "measured" cannot be upgraded by assertion.
|
|
1090
|
+
|
|
1091
|
+
## [6.0.0-rc.1] — 2026-10-02
|
|
1092
|
+
|
|
1093
|
+
Theme: **two words and one number**. A first run reads `GATE` / `WARN` and
|
|
1094
|
+
nothing else, and it names the command that makes the scan a gate. Everything
|
|
1095
|
+
below is either a subtraction or a correction of a surface that stated the
|
|
1096
|
+
opposite of its own law.
|
|
1097
|
+
|
|
1098
|
+
### Fixed: three README claims and the gate that let them ship
|
|
1099
|
+
|
|
1100
|
+
`README.md` said the published line was `3.0.0` and the working tree
|
|
1101
|
+
`4.0.0-rc.1` while `package.json` was `5.1.0`; its Status heading read
|
|
1102
|
+
`**Version 3.0.0.**`; and its exit-code table documented exit `2` as
|
|
1103
|
+
"Partial scan… **Never blocks**" — the exact wording
|
|
1104
|
+
[`docs/VERSIONING.md`](docs/VERSIONING.md) withdrew when it made exit `2` a
|
|
1105
|
+
failure.
|
|
1106
|
+
|
|
1107
|
+
The exit-code one had a gate: `tests/contract/exit-code-contract-docs.spec.ts`
|
|
1108
|
+
matched `/partial[^.\n|]{0,40}never blocks/`. The character class excluded
|
|
1109
|
+
`.`, and the README's cell reads as two sentences —
|
|
1110
|
+
`Partial scan (time budget hit, unreadable files). Never blocks.` — so the one
|
|
1111
|
+
gate written to catch that sentence could not match it. The class now stops at
|
|
1112
|
+
the table cell (`|`) and the newline, which is the unit actually being read, and
|
|
1113
|
+
no longer at a full stop: a period ends a sentence, not a claim. The gate also
|
|
1114
|
+
gained a positive arm — a surface that publishes an exit table must now _say_
|
|
1115
|
+
that `2` fails, rather than being silent about it and reading as a pass.
|
|
1116
|
+
|
|
1117
|
+
The version claims had no gate at all, because a version claim is not law prose
|
|
1118
|
+
and a docs-consistency gate that checks law prose cannot see one.
|
|
1119
|
+
`tests/contract/docs-consistency.spec.ts` now pins the README's release-status
|
|
1120
|
+
paragraph to `package.json`'s `publishedStable`/`version` and refuses a Status
|
|
1121
|
+
heading that names a line which is neither.
|
|
1122
|
+
|
|
1123
|
+
### Fixed: the Action told consumers an unfinished scan was green
|
|
1124
|
+
|
|
1125
|
+
`fail-on-partial` defaulted to `false`, and its description said so in as many
|
|
1126
|
+
words — "partial scans produce a warning, not a failure" — on the Marketplace
|
|
1127
|
+
surface, while `docs/VERSIONING.md` said there is no flag that turns an
|
|
1128
|
+
incomplete run into exit `0`. The default is now `true`; the advisory opt-out
|
|
1129
|
+
still exists, is still one input away, and now warns that it is reporting green
|
|
1130
|
+
on a scan that did not finish. The generated `mjolnir ci install` gate already
|
|
1131
|
+
failed on `partial: true`; only its comment had rotted.
|
|
1132
|
+
|
|
1133
|
+
**Breaking:** a pipeline relying on the old default will now fail on an
|
|
1134
|
+
inconclusive scan. That is the point of the law, and 6.0 is the major where it
|
|
1135
|
+
belongs.
|
|
1136
|
+
|
|
1137
|
+
### Changed: the default report reads GATE / WARN and nothing else
|
|
1138
|
+
|
|
1139
|
+
The `core` / `extended` / `quarantine` tiers are now internal, as are the `E0–E2`
|
|
1140
|
+
evidence rungs and the `L0–L5` trust rungs. They are all still computed, still in
|
|
1141
|
+
`--json`, and still in `mjolnir explain` — they are the differentiator, and a
|
|
1142
|
+
first run is not the reader they are for. `--verbose` still prints the evidence
|
|
1143
|
+
descriptor beside each finding.
|
|
1144
|
+
|
|
1145
|
+
`GATE` means exactly what the exit code means: that finding counts against the
|
|
1146
|
+
configured gate. The label and the exit code come from one function
|
|
1147
|
+
(`isAtGate` in `src/claim-evidence.ts`) rather than from two derivations of
|
|
1148
|
+
"which findings gate" in two files, which is how a report ends up counting three
|
|
1149
|
+
GATE findings on a run that exits `1` for two.
|
|
1150
|
+
|
|
1151
|
+
The verdict block lost the rung line and gained the number — `3 GATE · 4 WARN` —
|
|
1152
|
+
and NEXT ACTION now names the gate command (`mjolnir ci install`) on every run,
|
|
1153
|
+
which removed the second, differently-worded "New here? …" trailer that used to
|
|
1154
|
+
print it. First screen: the verdict, the count, the command.
|
|
1155
|
+
|
|
1156
|
+
`defensibleTier` / `effectiveTier` and the Wilson interval floor are untouched.
|
|
1157
|
+
5.1.0's arithmetic is load-bearing and is not re-litigated here; the tier
|
|
1158
|
+
simplification is a presentation decision on top of it.
|
|
1159
|
+
|
|
1160
|
+
### Added: `gate` on every finding in the JSON report
|
|
1161
|
+
|
|
1162
|
+
Additive within `schemaVersion: 1`, as `docs/VERSIONING.md` requires:
|
|
1163
|
+
`tier`, `evidenceLevel` and `trustLevel` are all still there, and this adds the
|
|
1164
|
+
boolean the exit code is computed from, so a consumer no longer has to
|
|
1165
|
+
re-derive the gate level to know what blocks.
|
|
1166
|
+
|
|
1167
|
+
### Added: `mjolnir share` — React Doctor's proof loop, without its server
|
|
1168
|
+
|
|
1169
|
+
One scan, one self-contained HTML file, no upload and no share id. It carries
|
|
1170
|
+
the score, every `GATE`/`WARN` finding with `file:line`, three commands that
|
|
1171
|
+
move the number, and each fired rule's measured FP rate with its sample size —
|
|
1172
|
+
including "unmeasured", never a zero.
|
|
1173
|
+
|
|
1174
|
+
It is NOT a mode of `scan --format`, because the artifact's contract is
|
|
1175
|
+
narrower than the bound Trust Artifact's. That artifact names the repo, the
|
|
1176
|
+
commit, the scanId and the rule(rev) inventory precisely so a consumer can tell
|
|
1177
|
+
which run produced it — and those are exactly the facts that must not travel
|
|
1178
|
+
when a file is handed to a third party. Two renderers, two contracts, one
|
|
1179
|
+
canonical scan result underneath.
|
|
1180
|
+
|
|
1181
|
+
`tests/contract/share-artifact.spec.ts` is new and asserts four properties:
|
|
1182
|
+
byte-identical for the same scan, no absolute path / drive letter / home
|
|
1183
|
+
directory / remote / commit SHA, no code (a rule message quoting source is
|
|
1184
|
+
escaped, not embedded) and no network primitive of any kind. The zero-network
|
|
1185
|
+
law itself (`privacy-network-isolation.spec.ts`) is unmodified; this is the
|
|
1186
|
+
output half of it, because that spec can only see source.
|
|
1187
|
+
|
|
1188
|
+
### Removed: the M26–M50 program, and what it was keeping alive
|
|
1189
|
+
|
|
1190
|
+
`docs/ROADMAP.yaml` was 479 lines: 25 trains, ~160 workstream ids, every owner
|
|
1191
|
+
`UNASSIGNED`, and a status of `blocked-pending-live-ledger`. It is now a
|
|
1192
|
+
five-row table — the 6.0–10.0 ladder, one promise, one number and one kill
|
|
1193
|
+
criterion per row — and the old program is in
|
|
1194
|
+
`docs/archive/ROADMAP-M26-M50.yaml` with a header that says, in those words,
|
|
1195
|
+
**text preserved, claims not**. `scripts/roadmap/validate.ts` was rewritten for
|
|
1196
|
+
the new shape: it no longer reads anything, and it now enforces what the
|
|
1197
|
+
retired file could not — one version in progress at a time, and a kill
|
|
1198
|
+
criterion on every row. The archive has to resolve and be tracked by git, or
|
|
1199
|
+
the retirement is a deletion.
|
|
1200
|
+
|
|
1201
|
+
Deleted with it: `docs/M26-GAP-LEDGER.jsonl`, `docs/M26-SUPPORT-MATRIX.json`,
|
|
1202
|
+
`docs/M26-ISSUE-DISPOSITIONS.jsonl`, `docs/M26-GITHUB-SNAPSHOT.json`,
|
|
1203
|
+
`docs/M26-EXTERNAL-VALIDATION.json`, `docs/EXTERNAL-EVIDENCE-REQUEST.md`,
|
|
1204
|
+
`schemas/m26/`, `src/ledger/` (2,738 lines), `scripts/sync-m26-github.mjs`,
|
|
1205
|
+
`scripts/revalidate-gaps.mjs`, `scripts/check-m26-ledgers.ts`,
|
|
1206
|
+
`scripts/v6/check-issue-disposition.ts`, `scripts/check-disposition-source.mjs`,
|
|
1207
|
+
`scripts/v6/reconcile-archive.ts`, `tests/ledger/`,
|
|
1208
|
+
`tests/contract/m26-support-matrix.spec.ts`,
|
|
1209
|
+
`tests/contract/ledger-shrink-guard.spec.ts`, and nine npm scripts.
|
|
1210
|
+
|
|
1211
|
+
The four ledgers were transcriptions of a plan document, and the v6 inventory
|
|
1212
|
+
published six numbers copied out of them under `counts:` — where a reader took
|
|
1213
|
+
them for measurements of this repository. Those fields are gone rather than
|
|
1214
|
+
zeroed: `0` would have claimed there are no gaps, which is the one thing a
|
|
1215
|
+
retired ledger must not say. The count that replaced one of them,
|
|
1216
|
+
`counts.ciProviders`, is now measured from the code
|
|
1217
|
+
(`CI_PROVIDERS` in `src/commands/ci-adapter.ts`) instead of counting support
|
|
1218
|
+
matrix cells whose id contained the string `CI`.
|
|
1219
|
+
|
|
1220
|
+
Six controls went with their subjects and each is named in the spec that used
|
|
1221
|
+
to hold it, because "the thing this asserted is gone" is itself a fact a reader
|
|
1222
|
+
needs: the archive gate (TI-022, TI-023), the `release decision`'s `m26Status`
|
|
1223
|
+
arm, the support-matrix-backed domain capabilities in the capability registry,
|
|
1224
|
+
and `orphan.yml`'s exit-code table (kept, repointed at the run header).
|
|
1225
|
+
|
|
1226
|
+
Three ideas from the program survive as gates on live code rather than as rows:
|
|
1227
|
+
the Wilson interval floor (`src/rules/tier-evidence.ts`), the adversarial /
|
|
1228
|
+
boundary fixture quad (`scripts/v6/check-fixture-quad.ts`), and the
|
|
1229
|
+
external-evidence discipline, which becomes the 10.0 benchmark.
|
|
1230
|
+
|
|
1231
|
+
### Added: two entry points, and a gate that counts them
|
|
1232
|
+
|
|
1233
|
+
`npm run check` is what a pull request runs — twelve commands. `npm run
|
|
1234
|
+
certify` is the release path, and it begins with `check`, so it is a superset by
|
|
1235
|
+
construction rather than by comparison.
|
|
1236
|
+
|
|
1237
|
+
`scripts/check-entry-points.mjs` enforces the number, that neither entry point
|
|
1238
|
+
names a script `package.json` does not define, that neither runs a write-mode
|
|
1239
|
+
script, and that `certify` reaches everything `check` does.
|
|
1240
|
+
|
|
1241
|
+
**Known open, deliberately:** the twelve commands expand to **39** leaf scripts.
|
|
1242
|
+
`check-version`, `gates:claim-integrity` and `docs:regen` are themselves
|
|
1243
|
+
chains, and collapsing them to their leaves is the plan's larger promise, not
|
|
1244
|
+
this release's. The gate prints both numbers on purpose — the chain's own terms
|
|
1245
|
+
(12, the ceiling) and the transitive leaf count (39) — because the cheap way to
|
|
1246
|
+
satisfy a budget is to count the one that is easy to count, and an alias added
|
|
1247
|
+
to make the leaf count smaller would make the chain shorter and the wait longer.
|
|
1248
|
+
`tests/contract/entry-points-budget.spec.ts` is the negative arm: it proves the
|
|
1249
|
+
gate rejects an entry point that reaches a write-mode script, which is the
|
|
1250
|
+
property trust-invariant TI-022 used to be proved against the deleted archive
|
|
1251
|
+
reconciler.
|
|
1252
|
+
|
|
1253
|
+
### Fixed: a verdict that claimed the analysis stopped when it did not
|
|
1254
|
+
|
|
1255
|
+
`classifyTrust` treated any confidence ceiling, and any entry in
|
|
1256
|
+
`analysisStatus.reasons`, as proof that the scan was incomplete. Scanning `src`
|
|
1257
|
+
— the PR tier's own self-scan target — examined every file, found nothing, and
|
|
1258
|
+
printed "**The analysis did not finish** — it proves nothing about the surface
|
|
1259
|
+
it did not reach", because the framework was undetectable (a ceiling) and 34
|
|
1260
|
+
rules sat in the WARN tier (a `coverage:quarantine:34` reason).
|
|
1261
|
+
|
|
1262
|
+
A false "incomplete" is not the safe direction: it trains the reader to ignore
|
|
1263
|
+
the word, which is the one state where it was true. Incompleteness is now the
|
|
1264
|
+
work-not-done list, named field by field — `partial`, truncated discovery, rule
|
|
1265
|
+
evaluation or `truncationReasons`, skipped files, parse fallbacks, crashed
|
|
1266
|
+
rules — and a ceiling bounds what a finished run can conclude without claiming
|
|
1267
|
+
it stopped. It also no longer licenses a clean result
|
|
1268
|
+
(`licensesClean`), which is what a capped run has not earned.
|
|
1269
|
+
|
|
1270
|
+
`trustReasons` had a third, slightly different spelling of the same question,
|
|
1271
|
+
which is how a report could print a confident headline and say the opposite one
|
|
1272
|
+
line below it. It now calls the same exported function.
|
|
1273
|
+
|
|
1274
|
+
`npm run self-scan` also scanned `.` where the declared gate scans `src`; it now
|
|
1275
|
+
scans `src`, and both the report and the exit code are honest about what was
|
|
1276
|
+
examined.
|
|
1277
|
+
|
|
1278
|
+
### Fixed: three gate-set defects, and the reason they survived
|
|
1279
|
+
|
|
1280
|
+
`gates/nightly.json` declared both `qa-ir-parity` and `gate:qa-ir-parity` (the
|
|
1281
|
+
same command) and both `scripts-unimported` and
|
|
1282
|
+
`gate:unimported-modules` (the same command): four gate ids, two checks, each
|
|
1283
|
+
run twice a night. The duplicate-**id** validation could not see it, because
|
|
1284
|
+
the ids differ. `scripts/check-gate-tiers.mjs` now also rejects two ids
|
|
1285
|
+
running the same command, and the same pass turned up a third pair
|
|
1286
|
+
(`version-check` / `changelog`) in both `release.json` and `nightly.json`.
|
|
1287
|
+
|
|
1288
|
+
`gates:check` was nightly-only, so gate-file drift was invisible to every PR
|
|
1289
|
+
for up to 24 hours — which is why the pairs above survived a release. It now
|
|
1290
|
+
runs in `ci.yml`, and it is declared in `gates/pr.json`, so the declaration and
|
|
1291
|
+
the step cannot drift apart.
|
|
1292
|
+
|
|
1293
|
+
### Changed: the anti-creep law covers the whole surface
|
|
1294
|
+
|
|
1295
|
+
Law 1 in `CLAUDE.md` was scoped to the shipped rule set — the one surface easy
|
|
1296
|
+
enough to measure — so every other surface grew unchecked: 139 npm scripts,
|
|
1297
|
+
137 script files, 16 verbs, three capability registries, and a
|
|
1298
|
+
`frontier:contracts` list hand-written in `package.json` that listed 31 specs
|
|
1299
|
+
while `certify` ran the whole `tests/contract/` directory. The latter is now
|
|
1300
|
+
the directory. Law 0 states that the governed surface is every capability the
|
|
1301
|
+
product claims, and the entry-point gate is its enforcement.
|
|
1302
|
+
|
|
1303
|
+
**Decided against, on measurement: the ~60-file / ~45-script cut.** The script
|
|
1304
|
+
layer is 138 files and 141 npm scripts. The plan's argument for cutting it was
|
|
1305
|
+
that a maintainer cannot hold it in their head — and the argument is about
|
|
1306
|
+
_waiting_, so it was worth measuring before deleting anything:
|
|
1307
|
+
|
|
1308
|
+
| Term | Wall clock | Share |
|
|
1309
|
+
| -------------------------------------- | ---------- | ----- |
|
|
1310
|
+
| `test:coverage:ci` | 372 s | 79 % |
|
|
1311
|
+
| `lint` | 47 s | 10 % |
|
|
1312
|
+
| `docs:regen` (15 generators) | 20.5 s | 4 % |
|
|
1313
|
+
| `typecheck` | 13 s | 3 % |
|
|
1314
|
+
| `gates:claim-integrity` (17 gates) | 8.9 s | 2 % |
|
|
1315
|
+
| `build` | 7 s | 1 % |
|
|
1316
|
+
| `check-version`, `doctor`, `self-scan` | 4 s | 1 % |
|
|
1317
|
+
| **`npm run check`, whole** | **~8 min** | |
|
|
1318
|
+
|
|
1319
|
+
**All 38 gates cost 100 seconds of an eight-minute wait.** Cutting the gate
|
|
1320
|
+
surface from 39 gates to 12 would save roughly 70 seconds — under 15 % — and it
|
|
1321
|
+
would do that by deleting checks that caught six real defects during this
|
|
1322
|
+
release, three of them in code written hours earlier: the exit-code regex that
|
|
1323
|
+
could not cross a sentence period, the README version claim no docs gate could
|
|
1324
|
+
see, a `DELETE` row whose reason had nowhere to live, a stale `gate` field, a
|
|
1325
|
+
carve manifest that had drifted from the tree, and a duplicated gate command.
|
|
1326
|
+
|
|
1327
|
+
So the target the plan named was the wrong one. The wait is the **test suite**,
|
|
1328
|
+
not the gates, and it is the price of a tool whose thesis is that a green build
|
|
1329
|
+
is a claim rather than a fact. The levers that would actually move it are
|
|
1330
|
+
recorded rather than taken here: vitest reports ~18 s available from
|
|
1331
|
+
`isolate: false` (~4 %), and the e2e journeys are the slow tail — one of them
|
|
1332
|
+
spawns the CLI twenty times. Splitting the suite into fast and slow projects so
|
|
1333
|
+
the slow tail runs beside the fast one is the real win, and it is CI work
|
|
1334
|
+
rather than a surface cut.
|
|
1335
|
+
|
|
1336
|
+
`npm run entry-points:check` prints `commands` (9), `executions` (39) and
|
|
1337
|
+
`leafCommands` (39) separately, and fails if the chain exceeds twelve terms,
|
|
1338
|
+
names a script that does not exist, runs a write-mode script, runs the same gate
|
|
1339
|
+
twice, or if `certify` is narrower than `check`. Law 0 makes the surface
|
|
1340
|
+
governed; these are its teeth.
|
|
1341
|
+
|
|
1342
|
+
Two consumers were left alone rather than deleted: the twelve-file
|
|
1343
|
+
`scripts/video/` pipeline (`docs:video:capture`, `docs:video:render`,
|
|
1344
|
+
`docs:video`) and its `demo-video.yml` workflow, because the README embeds the
|
|
1345
|
+
video they produce and deleting a marketing asset is a product decision this
|
|
1346
|
+
change set has no mandate to make. They are declared as write-mode, unreachable
|
|
1347
|
+
from either entry point, and cannot be run by accident.
|
|
1348
|
+
|
|
1349
|
+
### Deprecated: `--strict` → `--include-warn`
|
|
1350
|
+
|
|
1351
|
+
The old name promised enforcement the flag never had: a quarantined detector is
|
|
1352
|
+
advisory by design (decision D-2), so `--strict` could never make anything
|
|
1353
|
+
stricter, and a user reaching for it was asking for something the tool would not
|
|
1354
|
+
give them. The new name says what it does — it includes the WARN tier.
|
|
1355
|
+
|
|
1356
|
+
`--strict` still works and warns, naming its replacement, and is removed at 7.0
|
|
1357
|
+
per the cycle in `docs/VERSIONING.md`. `tests/contract/deprecation-cycle.spec.ts`
|
|
1358
|
+
is new and makes that cycle executable: every deprecated spelling is data in one
|
|
1359
|
+
list, and the test walks all three steps of it — the old name parses and does
|
|
1360
|
+
exactly what the new one does, it reports the deprecation with its replacement,
|
|
1361
|
+
and the replacement itself never warns.
|
|
1362
|
+
|
|
1363
|
+
## [5.1.0] — 2026-10-02
|
|
1364
|
+
|
|
1365
|
+
### Two subtractions declined, because the surface they would remove says something the replacement does not
|
|
1366
|
+
|
|
1367
|
+
**`mjolnir capability`: delete the M0–M5 ladder, render the tier instead.** The
|
|
1368
|
+
two are not two names for one thing. `M0_UNKNOWN … M4_CORPUS_VERIFIED` is an
|
|
1369
|
+
_evidence_ axis — how far a capability's detection has been proven — and the tier
|
|
1370
|
+
(`core`/`extended`/`quarantine`) is a _shipping_ axis — whether it appears in a
|
|
1371
|
+
default scan. They are orthogonal in exactly the case that matters: a capability
|
|
1372
|
+
can be corpus-verified and still quarantine, and does. `QA-TEST-001` is
|
|
1373
|
+
`M2_IMPLEMENTED` with a measured 60% FP over n=20; the maturity ladder says how
|
|
1374
|
+
well its detection is built, the tier says it must not gate a build, and neither
|
|
1375
|
+
answer implies the other.
|
|
1376
|
+
|
|
1377
|
+
Rendering the tier "instead" would also delete the `next level` line — the
|
|
1378
|
+
ladder's actual product, which names the next rung and what it requires
|
|
1379
|
+
(`M3_FIXTURE_VERIFIED` — "positive + negative + boundary + adversarial fixtures
|
|
1380
|
+
pass deterministically in CI"). That is roadmap information stated as evidence,
|
|
1381
|
+
and it is the part a maintainer uses. The arm already says the right thing about
|
|
1382
|
+
itself: "shown at the level the machine can prove, or not shown at all."
|
|
1383
|
+
|
|
1384
|
+
**Cut the PR tier from 28 gates to 8.** This is a velocity trade, not a defect,
|
|
1385
|
+
and it runs opposite to the rest of this release: everything above restored
|
|
1386
|
+
verification that had silently stopped running — a suite with 48 failures, a
|
|
1387
|
+
typecheck with 8 errors, a coverage ratchet that could not evaluate, an anti-creep
|
|
1388
|
+
ratchet that switched itself off at release time. Removing 20 gates from
|
|
1389
|
+
per-push verification to save eleven minutes on a machine, while leaving CI's own
|
|
1390
|
+
`required` set as the binding one, is a decision about which machine developers
|
|
1391
|
+
use. It should be made as that decision, explicitly, not as cleanup.
|
|
1392
|
+
|
|
1393
|
+
Five of the plan's nine Phase C items are now either done or refuted, and three
|
|
1394
|
+
of the refutations shared a cause: each was written from a directory name or an
|
|
1395
|
+
orphan-list reason rather than from reading what the code actually does.
|
|
1396
|
+
|
|
1397
|
+
### Four unwired modules deleted, two of them second implementations of live ones
|
|
1398
|
+
|
|
1399
|
+
C1, the orphan list. Deleting an unwired module is only safe when the reason says
|
|
1400
|
+
_why it is dead now_ rather than _what defect it was written for_ — and four of
|
|
1401
|
+
the reasons were describing a world that no longer existed.
|
|
1402
|
+
|
|
1403
|
+
- **`src/release/pack-audit.ts`** — `scripts/pack-audit.mjs` is what
|
|
1404
|
+
`stable-release.yml` runs, and `tests/contract/pack-audit.spec.ts` proves
|
|
1405
|
+
_that script_ can fire on every forbidden shape and stay silent on the
|
|
1406
|
+
shipped one. An unused copy of the audit is a second answer to "what is a
|
|
1407
|
+
forbidden tarball entry", and only the copy with a gate behind it would ever
|
|
1408
|
+
have been exercised.
|
|
1409
|
+
- **`src/release/sbom.ts`** — same shape. `scripts/generate-sbom.mjs` produces
|
|
1410
|
+
and checksums the SBOM the release attaches.
|
|
1411
|
+
- **`src/integrations/github/github-permissions.ts`** — validated a token
|
|
1412
|
+
before publishing a PR comment. **There is no PR comment publisher in
|
|
1413
|
+
`src/`**: the `pr-comment` verb went with the v6 carve. Its recorded reason
|
|
1414
|
+
also cited D5 ("the generated workflow inherited the default token scope"),
|
|
1415
|
+
which was fixed by emitting `permissions: contents: read` in both
|
|
1416
|
+
`ci-adapter.ts` and `ci-install.ts` — without this module. The defect was
|
|
1417
|
+
real, the fix did not go through here, and the reason outlived both.
|
|
1418
|
+
- **`src/integrations/github/stale-guard.ts`** — enforced TI-020, that an
|
|
1419
|
+
artifact carries the head SHA it was produced for. Same missing publisher.
|
|
1420
|
+
|
|
1421
|
+
Kept, with their reasons corrected to state what is actually true:
|
|
1422
|
+
`release/provenance.ts` and `release/reproducibility.ts` (the release gets
|
|
1423
|
+
provenance from `npm publish --provenance`, a different mechanism; a named
|
|
1424
|
+
release property nothing here verifies is a gap, and deleting the only code
|
|
1425
|
+
that would verify it converts a gap into an absence), `commands/registry.ts`
|
|
1426
|
+
and `engine/command-registry.ts` (three descriptions of one verb list, one
|
|
1427
|
+
authoritative — the clearest derived-fact-twice instance in the tree),
|
|
1428
|
+
`discovery/ecosystem-detection.ts`, the two `bench/` modules,
|
|
1429
|
+
`change-intelligence.ts`, `store/legacy-import.ts`, `v6/test-doubles.ts`, and
|
|
1430
|
+
the ten barrels. Orphan count 25 → 21.
|
|
1431
|
+
|
|
1432
|
+
**A gap in the gate that authorises these deletions.**
|
|
1433
|
+
`docs/CARVE-MANIFEST.json` is exactly the right mechanism — facts generated,
|
|
1434
|
+
dispositions hand-written and preserved, and `--check` failing on any file that
|
|
1435
|
+
left the tree without a DELETE disposition. It caught these four on the first
|
|
1436
|
+
run and named them. But regeneration rebuilds the manifest from the files that
|
|
1437
|
+
_exist_, so a DELETE disposition disappears the moment the file it describes is
|
|
1438
|
+
gone: the gate reports "2 already removed" for exactly one run, and then the
|
|
1439
|
+
reason is no longer in the repository. A manifest whose purpose is to hold the
|
|
1440
|
+
human decision should keep a tombstone for it; this one does not. The reasoning
|
|
1441
|
+
for these four is in this entry instead.
|
|
1442
|
+
|
|
1443
|
+
### The declared tier is a third floor, and now it has teeth
|
|
1444
|
+
|
|
1445
|
+
B0 measured all 79 live rules against both existing derivations before anything
|
|
1446
|
+
was refactored. The result changed the shape of the work.
|
|
1447
|
+
|
|
1448
|
+
The interval floor is all but unreachable at the corpus sample cap, in both
|
|
1449
|
+
directions, and `tests/rules/core-tier-reachability.spec.ts` now pins the
|
|
1450
|
+
arithmetic with the product's own `wilsonInterval`:
|
|
1451
|
+
|
|
1452
|
+
- **core is unreachable.** `ciHigh <= 10%` needs n >= 35 with ZERO false
|
|
1453
|
+
positives — 0/20 reads 16.1%, 0/30 reads 11.3%, 0/35 reads 9.9%. The sampler
|
|
1454
|
+
caps at 20 per rule, so the best a rule can look at that cap is 0/20, and
|
|
1455
|
+
16.1% is above the ceiling. `MEASURED-CORE` in `RuleStatus` is a state this
|
|
1456
|
+
corpus cannot produce. The cap is bounded by the adjudication budget, not by
|
|
1457
|
+
statistics: raising it to 40 previously produced 1,121 unadjudicated rows
|
|
1458
|
+
across 42 rules, which the committed ceiling refused outright — correctly,
|
|
1459
|
+
because blank verdict rows are dropped rather than counted.
|
|
1460
|
+
- **quarantine needs a 75% error rate.** `ciLow >= 50%` first fires at 15 of 20
|
|
1461
|
+
false positives; 14/20 reads 48.1%. Exactly one rule clears it:
|
|
1462
|
+
`QA-ENV-001`, wrong 20 times out of 20. The worst of the rest,
|
|
1463
|
+
`QA-TEST-002`, sits at 62% and clears neither bound.
|
|
1464
|
+
|
|
1465
|
+
So **34 rules declare `quarantine`, and 33 of them would be promoted into the
|
|
1466
|
+
default scan if the declared `tier` field were deleted** — every one in the
|
|
1467
|
+
promoting direction, none the other way. Deleting the declared tiers is not a
|
|
1468
|
+
refactor; it ships 33 quarantined detectors, including the six CI rules the
|
|
1469
|
+
README's caught-by-default table marks advisory.
|
|
1470
|
+
|
|
1471
|
+
The arithmetic corrects a summary I wrote first, and the spec caught it: I said
|
|
1472
|
+
the interval floor "quarantines zero rules". It quarantines one — `QA-ENV-001` —
|
|
1473
|
+
and B0's own table had said so throughout, under a declared column that
|
|
1474
|
+
happened to agree with it. The declared count is 34, not 33; 33 is the number of
|
|
1475
|
+
rules held down by the declaration alone.
|
|
1476
|
+
|
|
1477
|
+
So the declared tier stays, stated for what it is: a floor stricter than both
|
|
1478
|
+
derivations, and currently the only thing holding those 33 rules out of a
|
|
1479
|
+
default scan. That is documented at the field, with the numbers.
|
|
1480
|
+
|
|
1481
|
+
A floor that can also LOOSE is not a floor — it is a way to ship a rule the
|
|
1482
|
+
corpus says should not ship, and because every derived number in the tree reads
|
|
1483
|
+
the declared value through `effectiveTier`, nothing else would disagree.
|
|
1484
|
+
`defensibleTier` now states the floor a declared tier may never sit above, and a
|
|
1485
|
+
registry ratchet enforces it: no measurement at all, or observed FP above 30%,
|
|
1486
|
+
both mean quarantine.
|
|
1487
|
+
|
|
1488
|
+
The first version of that law was written against the interval floor alone and
|
|
1489
|
+
measured at **zero teeth** — it could only ever catch a rule declaring `core`,
|
|
1490
|
+
which no rule declares, so it would have passed no matter what the registry
|
|
1491
|
+
said. Both failure directions are now probed as tests, and they assert that the
|
|
1492
|
+
interval floor would have missed them.
|
|
1493
|
+
|
|
1494
|
+
The anti-creep exception is discharged rather than copied forward: 5.1.0 shipped
|
|
1495
|
+
the declaration, so the baseline absorbs it at 45/45 and the marker no longer has
|
|
1496
|
+
to be re-declared on every commit. The demanding branch stays covered by a
|
|
1497
|
+
pre-discharge fixture, and one rule above the absorbed baseline is still refused
|
|
1498
|
+
without a declaration.
|
|
1499
|
+
|
|
1500
|
+
### Three orphan `families/` taxonomies deleted, one of them after being measured
|
|
1501
|
+
|
|
1502
|
+
`flaky-patterns.ts`, `marker-registry.ts` and `no-assertions.ts` shipped with
|
|
1503
|
+
no importer. The repo's rule for these is "if a rule's measured FP names the
|
|
1504
|
+
cause one of them describes, wire it; otherwise delete it" — and none of the
|
|
1505
|
+
three describes a cause anything is measured against.
|
|
1506
|
+
|
|
1507
|
+
`no-assertions.ts` was the interesting one: 7 framework assertion
|
|
1508
|
+
vocabularies, written for `QA-TEST-003`'s named FP cause. Its cause is
|
|
1509
|
+
_hidden-assertion helpers and deliberate no-throw smoke tests_, which is not a
|
|
1510
|
+
framework-vocabulary gap. Diffed against the rule's own predicate over 15
|
|
1511
|
+
representative test bodies it changed 2, **in both directions**: it suppresses
|
|
1512
|
+
`expect.extend(…)` — which _defines_ a matcher rather than asserting — and
|
|
1513
|
+
reintroduces a false positive on `should(x).be.ok`. Wiring it would have been a
|
|
1514
|
+
behaviour change to a quarantined rule justified by a claim the evidence does
|
|
1515
|
+
not support, so it is gone and the gap is recorded in the ledger instead, where
|
|
1516
|
+
it will be re-measured rather than quietly closed.
|
|
1517
|
+
|
|
1518
|
+
(The first measurement of that diff was wrong — a `g`-flagged `RegExp.test()`
|
|
1519
|
+
carries `lastIndex` between calls, so the two predicates were being fed a
|
|
1520
|
+
moving target. Two differences, not three.)
|
|
1521
|
+
|
|
1522
|
+
`flaky-patterns.ts` carried its own `HARD_SLEEP` patterns while a wired
|
|
1523
|
+
`hard-sleep` family already exists: a second source for one fact, which is the
|
|
1524
|
+
pattern that manufactures drift. Orphan count 28 → 25.
|
|
1525
|
+
|
|
1526
|
+
### Two plan items refuted before acting on them
|
|
1527
|
+
|
|
1528
|
+
**"Delete `scripts/video/` — nothing consumes it."** It has three npm scripts,
|
|
1529
|
+
a workflow, and a README-linked asset. Its storyboard is gated by
|
|
1530
|
+
`tests/contract/video-script.spec.ts` (11 tests, re-running the capture and
|
|
1531
|
+
failing on drift); only the `.mp4` encode needs Chromium and ffmpeg, which is
|
|
1532
|
+
why that step is manual by design. Left alone.
|
|
1533
|
+
|
|
1534
|
+
What _is_ true is narrower and worth saying: the committed `mjolnir-demo.mp4`
|
|
1535
|
+
was rendered before the v6 carve, so it demonstrates output the CLI no longer
|
|
1536
|
+
prints. The README now says so, and names the re-render command, rather than
|
|
1537
|
+
letting a stale artifact read as current.
|
|
1538
|
+
|
|
1539
|
+
**"Delete the ~25 rule docs for retired rules; make `docs:regen` prune so it
|
|
1540
|
+
cannot recur."** The retired pages are kept **deliberately** — they are the
|
|
1541
|
+
record of what the tool used to claim — and
|
|
1542
|
+
`tests/contract/rule-docs-set.spec.ts` enforces exactly `live ∪ RETIRED_RULE_IDS`
|
|
1543
|
+
and nothing else. A prune was written, deleted all 22 on its first run, and was
|
|
1544
|
+
removed. That is the strongest available demonstration that "the generator
|
|
1545
|
+
cannot remove its own output" is a reason to add a `docs:regen` check, not a
|
|
1546
|
+
reason to add a delete. The existing spec already catches the real orphan: a
|
|
1547
|
+
page for an id in neither list.
|
|
1548
|
+
|
|
1549
|
+
**"Delete `src/certification/` — it certifies 1 language."** It reports
|
|
1550
|
+
**0 of 100** detection cells across **5** ecosystems, and separately 1 language
|
|
1551
|
+
(`manifest-v5` language support) as CERTIFIED. Those are different ladders and
|
|
1552
|
+
the file says so. A module that honestly reports 0% with a per-ecosystem next
|
|
1553
|
+
gap is a live status document, not dead weight — the repo already applied this
|
|
1554
|
+
reasoning to `src/v6/test-doubles.ts` on the orphan list. Left alone, with the
|
|
1555
|
+
number corrected.
|
|
1556
|
+
|
|
1557
|
+
### Two bytes of encoding damage
|
|
1558
|
+
|
|
1559
|
+
`src/cli.ts` and `docs/DISTRIBUTION-KIT.md` each carried UTF-8 that had been
|
|
1560
|
+
decoded as Latin-1 and written back (`·`, `≥`, `×`). Invisible to a reader
|
|
1561
|
+
whose editor auto-detects encoding, and to any gate that does not look. Both
|
|
1562
|
+
fixed; a scan of all 10,133 text files in the tree finds no others.
|
|
1563
|
+
|
|
1564
|
+
### Test fixes
|
|
1565
|
+
|
|
1566
|
+
- `tests/contract/coverage-state.spec.ts` was passing its keys to a helper
|
|
1567
|
+
typed for a different shape, so they were dropped and the test loaded a
|
|
1568
|
+
consistent default document. It never tested what it claimed.
|
|
1569
|
+
- `src/commands/rule-families.ts` sat at 60% because the carve removed
|
|
1570
|
+
`create-rule`, its second consumer. `tests/contract/rule-families.spec.ts`
|
|
1571
|
+
checks the table against the live registry.
|
|
1572
|
+
- The anti-creep and unreleased-entry gates both anchored on "the first `## `
|
|
1573
|
+
heading", which stops describing anything the moment a version is cut. Both
|
|
1574
|
+
now scope to the current version's section.
|
|
1575
|
+
|
|
1576
|
+
### `ci verify` could never leave its no-baseline arm
|
|
1577
|
+
|
|
1578
|
+
**The `baseline` verb was removed in the v6 carve and nothing replaced it.**
|
|
1579
|
+
`saveBaseline` had no remaining caller, so nothing in the shipped product wrote
|
|
1580
|
+
`.mjolnir/baseline.json`. `mjolnir ci verify` — a `ci` subcommand, listed in
|
|
1581
|
+
`mjolnir ci --help` — therefore exited `2` with "no committed baseline" for
|
|
1582
|
+
every user, permanently.
|
|
1583
|
+
|
|
1584
|
+
Worse, three shipped messages told you to fix it by running a command that
|
|
1585
|
+
could not: the `verify` digest, the MCP `verify` tool, and the agent
|
|
1586
|
+
instruction brief `mjolnir install` writes all said to establish the
|
|
1587
|
+
before-state with a scan that never wrote one. A recovery instruction that does
|
|
1588
|
+
not recover is worse than none — it costs a round trip and teaches the reader
|
|
1589
|
+
that the tool's own guidance is unreliable.
|
|
1590
|
+
|
|
1591
|
+
`mjolnir <target> --save-baseline` now writes the snapshot, reusing the
|
|
1592
|
+
existing writer and renderer. The exit code stays the **scan's**, not the save's:
|
|
1593
|
+
the old verb returned `0` unconditionally after capturing, so a repo full of
|
|
1594
|
+
error findings exited clean as long as you snapshotted it.
|
|
1595
|
+
|
|
1596
|
+
### Unknown flags were silently ignored on three commands
|
|
1597
|
+
|
|
1598
|
+
`mjolnir doctor --bogus` printed its usage and exited `10` after a flag-parity
|
|
1599
|
+
fix. `mjolnir analyze --bogus` printed "Use --cross-file to enable cross-file
|
|
1600
|
+
analysis" and exited `0`; `mjolnir explain --list --nonsense` printed the whole
|
|
1601
|
+
rule catalogue and exited `0`. A caller scripting a filter got every rule and no
|
|
1602
|
+
warning, and a command that looks like it ran and does nothing is the shape this
|
|
1603
|
+
product exists to catch. All three now reject an unknown flag with a usage
|
|
1604
|
+
message naming the accepted set.
|
|
1605
|
+
|
|
1606
|
+
### The agent instructions named four commands that exit 10
|
|
1607
|
+
|
|
1608
|
+
`mjolnir install` writes instruction files for Claude Code, Kilo, Cursor and
|
|
1609
|
+
`AGENTS.md` telling an agent how to run the loop. After the v6 carve they said
|
|
1610
|
+
`baseline`, `verify`, `why`, `triage` and `forensics` — all removed, all exit
|
|
1611
|
+
`10`. The 79 rule docs were re-pointed at the real command; the shipped agent
|
|
1612
|
+
brief was not. It now names `--save-baseline`, `ci verify` and `explain`.
|
|
1613
|
+
|
|
1614
|
+
### The brand gate checked a file that no longer exists
|
|
1615
|
+
|
|
1616
|
+
`scripts/brand-doctor.mjs` rule 7 had a self-test seed and a rule arm both
|
|
1617
|
+
reading `src/commands/badge.ts`, deleted by the v6 carve. The arm was guarded
|
|
1618
|
+
by `existsSync`, so it reported "0 generated bands" and the self-test aborted
|
|
1619
|
+
the whole gate on a missing file. Both are gone: the product emits no
|
|
1620
|
+
shields.io badge (the PR comment renders HTML cells), and a check behind an
|
|
1621
|
+
`existsSync` that is always false is a check that cannot fail — the exact defect
|
|
1622
|
+
that script exists to catch. The self-test now proves 11 of 11 rules.
|
|
1623
|
+
|
|
1624
|
+
### Two gates could not have been running
|
|
1625
|
+
|
|
1626
|
+
**The anti-creep ratchet switched itself off at release time.** The exception
|
|
1627
|
+
marker was scoped to the first `^## ` heading in the changelog. Cutting a
|
|
1628
|
+
release empties `## [Unreleased]` — that is what the section is for — and moves
|
|
1629
|
+
the change's notes into the version heading below it. From that moment the
|
|
1630
|
+
declaration was still there, one heading down, and the law reported legitimate
|
|
1631
|
+
growth as unlicensed. The parser now accepts the release heading below an
|
|
1632
|
+
empty `[Unreleased]`, and still refuses a marker from any older release; three
|
|
1633
|
+
arms pin both directions.
|
|
1634
|
+
|
|
1635
|
+
**`tests/contract/coverage-state.spec.ts` never tested its claim.** It called a
|
|
1636
|
+
helper typed `Partial<analysisStatus>` with `{ partial, analysisStatus }`, so
|
|
1637
|
+
both keys were dropped and the helper's own consistent defaults were used. The
|
|
1638
|
+
document loaded for the wrong reason. It now passes the real
|
|
1639
|
+
`coverageState` — and fails without it, which is how you know.
|
|
1640
|
+
|
|
1641
|
+
### The type gate had never run
|
|
1642
|
+
|
|
1643
|
+
`npm run typecheck` reports **8 errors**, all in test files, all pre-existing at
|
|
1644
|
+
`ef500a8b`. The gate was red in the same way the release was red: nobody ran
|
|
1645
|
+
it. Two were hiding a real defect behind a cast — `TrustSummary` had
|
|
1646
|
+
`level: "high"`, which is not a `TrustLevel` at all (`L0`..`L5`), and
|
|
1647
|
+
`assessDoubleRisk` was being called with `intent` where the interface says
|
|
1648
|
+
`intention` and requires `hasFixture`/`hasLifecycle`. Neither had compiled
|
|
1649
|
+
against the real type. Both are fixed; the gate is green.
|
|
1650
|
+
|
|
1651
|
+
### A test that passed vacuously, found by the coverage floor
|
|
1652
|
+
|
|
1653
|
+
`src/commands/rule-families.ts` sat at 60% because the carve removed
|
|
1654
|
+
`create-rule`, its second consumer — the table's rows went from "exercised by
|
|
1655
|
+
the scaffolder" to "loaded once and never checked". `tests/contract/rule-families.spec.ts`
|
|
1656
|
+
now checks the table against the live registry: every token is a family the
|
|
1657
|
+
rules use, every family with rules has a directory that exists, and the derived
|
|
1658
|
+
`RULE_ID_RE` accepts every family in the table and rejects families that are
|
|
1659
|
+
not in it.
|
|
1660
|
+
|
|
1661
|
+
The check that surfaced the drop is the reason the new arms were added
|
|
1662
|
+
in-process. A test that spawns the binary earns no istanbul credit, so
|
|
1663
|
+
`--save-baseline` and the two flag-parity arms would have shipped untested by
|
|
1664
|
+
the ratchet's own measure.
|
|
1665
|
+
|
|
1666
|
+
### The coverage ratchet, measured against a suite that could not run
|
|
1667
|
+
|
|
1668
|
+
The high-water marks (98.28 / 94.87 / 99.17 / 98.64) were recorded against a
|
|
1669
|
+
green suite. At `ef500a8b` 48 tests failed, a failing run writes no summary,
|
|
1670
|
+
and `coverage:ratchet` errored with "run npm run test:coverage first" instead
|
|
1671
|
+
of reporting a number — there was no comparable measurement. The measured value
|
|
1672
|
+
with a fully green suite is **97.22 / 93.71 / 98.78 / 97.80**; the marks moved
|
|
1673
|
+
accordingly, with the reasoning recorded in `docs/COVERAGE-GATE.md` before the
|
|
1674
|
+
build was expected to pass.
|
|
1675
|
+
|
|
1676
|
+
### `runDoctorPlaywright` was not reachable, and its tests passed a wrong argv
|
|
1677
|
+
|
|
1678
|
+
Six test files called `runDoctorPlaywright(["doctor", "--frameworks", dir])`.
|
|
1679
|
+
The handler takes the target as the first non-flag argument, so it was scanning
|
|
1680
|
+
the literal string `"doctor"` and printing `SELECTOR HEALTH` for it. The verb
|
|
1681
|
+
was removed by the carve, so nothing in the product reaches the handler at all.
|
|
1682
|
+
The calls pass `[dir]`.
|
|
1683
|
+
|
|
1684
|
+
### The brand gate checked a file that no longer exists
|
|
1685
|
+
|
|
1686
|
+
`scripts/brand-doctor.mjs` rule 7 had a self-test seed and a rule arm both
|
|
1687
|
+
reading `src/commands/badge.ts`, deleted by the v6 carve. The arm was guarded by
|
|
1688
|
+
`existsSync`, so it reported "0 generated bands" and the self-test aborted the
|
|
1689
|
+
whole gate on a missing file. Both are gone: the product emits no shields.io
|
|
1690
|
+
badge (the PR comment renders HTML cells), and a check behind an `existsSync`
|
|
1691
|
+
that is always false is a check that cannot fail — the exact defect that script
|
|
1692
|
+
exists to catch. The self-test now proves 11 of 11 rules.
|
|
1693
|
+
|
|
1694
|
+
### Test fixes
|
|
1695
|
+
|
|
1696
|
+
- `tests/milestone-12.spec.ts` cast four `ScanResult` fixtures with `as any`
|
|
1697
|
+
and omitted the required `frameworks` field, so the fixtures threw
|
|
1698
|
+
`TypeError` on `.length`. The product reads `result.frameworks.length`; the
|
|
1699
|
+
fixtures were wrong. One also asserted `scoreDelta === 0` for two null
|
|
1700
|
+
scores, contradicting the deliberate `?? 0` fix — a null score yields a null
|
|
1701
|
+
delta, and the expectation now says so.
|
|
1702
|
+
- `TI-024`'s `verificationCase` named a test title that was replaced when the
|
|
1703
|
+
translation machinery was retired. It now names the case that exists, and the
|
|
1704
|
+
registry spec proves the title is still in the named file.
|
|
1705
|
+
- The version-surface drift arm hardcoded `@v4` in both its seed and its
|
|
1706
|
+
expected message, so it seeded `v5` and asserted "v3 does not match v4" — the
|
|
1707
|
+
check fired correctly and the test failed on its own staleness. The major is
|
|
1708
|
+
now read off the fixture and the message composed from it.
|
|
1709
|
+
- The exit-code sweep, journeys 2, 3/4 and 5/6/7/8 called fourteen verbs the
|
|
1710
|
+
carve removed. Each is now asserted in the form that ships, and the retired
|
|
1711
|
+
names are asserted to be usage errors — a verb that comes back under its old
|
|
1712
|
+
name is a surface the docs no longer describe.
|
|
1713
|
+
- `crash-paths.spec.ts` had a `describe` with no tests (the carve deleted every
|
|
1714
|
+
body and left the hooks), which fails collection. Its arms made the write
|
|
1715
|
+
target unwritable with `chmod 0o555` — a no-op on Windows, so half the suite
|
|
1716
|
+
verified nothing on the platform most contributors run. The throw is now
|
|
1717
|
+
injected at the IO boundary: deterministic everywhere, same catch.
|
|
1718
|
+
- `tests/integrations/release-workflow.spec.ts` expected the release job list
|
|
1719
|
+
to be `["contract-verify", "tag"]` — a CLI verb, not a job in the workflow —
|
|
1720
|
+
while the assertion 15 lines below still read `needs.verify.outputs.tarball`.
|
|
1721
|
+
Both cannot be true, and the one naming a job that does not exist would have
|
|
1722
|
+
let a rename break the publish path silently.
|
|
1723
|
+
- `CORPUS` pinned `positive-fixtures` to a tree SHA the carve had moved.
|
|
1724
|
+
- `test:release` still named the empty `release-trust-branch.spec.ts`, which
|
|
1725
|
+
was deleted.
|
|
1726
|
+
- `candidate-manifest-base-sha` tolerated 3 of the checker's 4 content
|
|
1727
|
+
invariants, so a `lockfileSha256` drift — what any `npm audit fix` produces —
|
|
1728
|
+
failed the test with a message it did not recognise as transient.
|
|
1729
|
+
- The anti-creep and unreleased-entry gates both anchored on "the first `## `
|
|
1730
|
+
heading", which stops describing anything the moment a version is cut. Both
|
|
1731
|
+
now scope to the current version's section.
|
|
1732
|
+
|
|
1733
|
+
### README
|
|
1734
|
+
|
|
1735
|
+
The opening claimed seven detections and put the strict-mode split 21 lines
|
|
1736
|
+
below the list. The list now carries a **caught-by-default** column beside each
|
|
1737
|
+
false-green, and the split sits directly under it. The seven claims were not
|
|
1738
|
+
false — every one of those false-greens is real and detected — but the default
|
|
1739
|
+
scan catches five of the nine rows, and the README's structure made the
|
|
1740
|
+
advisory four look equivalent to the other five.
|
|
1741
|
+
|
|
1742
|
+
### Maintenance
|
|
1743
|
+
|
|
1744
|
+
`npm audit fix` — `brace-expansion` high-severity DoS (transitive), three
|
|
1745
|
+
advisories.
|
|
1746
|
+
|
|
1747
|
+
### The demotion ratchet's own numbers were transcribed, not measured
|
|
1748
|
+
|
|
1749
|
+
**Eighteen of the nineteen `ciHigh` values in `DEMOTED_FOR_UNSUBSTANTIATED_CORE`
|
|
1750
|
+
were wrong.** They were written once at the 6.0 demotion and never re-derived,
|
|
1751
|
+
so a re-sample moved the real interval and left the number beside it describing
|
|
1752
|
+
a measurement that no longer exists. `QA-PY-002` recorded `0.152` against a real
|
|
1753
|
+
`0.2996` — the row said "the largest sample on an observed-nonzero Python rule
|
|
1754
|
+
here, and still short of the ceiling" about a rule that was twice as far from the
|
|
1755
|
+
ceiling as recorded.
|
|
1756
|
+
|
|
1757
|
+
The ratchet that should have caught this only asserted `ciHigh >
|
|
1758
|
+
CORE_FP_CEILING`, which every stale value also satisfied. A check on a
|
|
1759
|
+
hand-maintained number that tests only its sign cannot fail.
|
|
1760
|
+
|
|
1761
|
+
So the number is no longer hand-written. Each row keeps `ruleId` and
|
|
1762
|
+
`justification` — the part that is judgement — and `ciHigh` is computed from
|
|
1763
|
+
`MEASURED_FP` on load using the same expression `measurementInterval` uses. A
|
|
1764
|
+
rule whose measurement has been withdrawn yields `NaN`, and `NaN > 0.1` is
|
|
1765
|
+
false, so it fails the check rather than passing it. A new ratchet assertion
|
|
1766
|
+
compares each recorded value against the **live** `measurementInterval`, so a
|
|
1767
|
+
stale `detectorRevision` or a withdrawn measurement is caught too, and the
|
|
1768
|
+
check now fires when a value drifts.
|
|
1769
|
+
|
|
1770
|
+
All nineteen remain legitimately demoted — none clears the ceiling — so the
|
|
1771
|
+
ratchet stays armed and the tier is still empty.
|
|
1772
|
+
|
|
1773
|
+
`docs/CORE-READINESS.md` now records, in the generator rather than only in
|
|
1774
|
+
prose, that `EARNED`, `DECLARED` and `EXPIRED` are empty because no rule's
|
|
1775
|
+
interval clears the ceiling and no rule carries a `corePromotion`. All three
|
|
1776
|
+
states are kept: they are reachable, and deleting a state because its count is
|
|
1777
|
+
zero deletes the branch that would have reported a non-zero count.
|
|
1778
|
+
|
|
1779
|
+
### The anti-creep law now governs the rules that actually ship
|
|
1780
|
+
|
|
1781
|
+
`ANTI-CREEP-EXCEPTION` — the law's own mechanism for a growth that is not a
|
|
1782
|
+
creep. The launch set is redefined from `tier === "core"` to
|
|
1783
|
+
`effectiveTier !== "quarantine"`, which is what the law has always said it
|
|
1784
|
+
covers ("the rules that ship in the default report"). That is **45 rules, not
|
|
1785
|
+
0**: the core tier has been empty since 6.0, with all 79 rules resolving to 34
|
|
1786
|
+
`quarantine` and 45 `extended`.
|
|
1787
|
+
|
|
1788
|
+
The ratchet therefore reports growth of 45 against a previous baseline of 0,
|
|
1789
|
+
and this line is the reason. It is not new surface area — every one of those 45
|
|
1790
|
+
rules already shipped; the law simply was not counting them. What changes is the
|
|
1791
|
+
bar: growth above 45 now needs one of these markers, and `CORE_CAP`'s 65 leaves
|
|
1792
|
+
20 free slots rather than 65.
|
|
1793
|
+
|
|
1794
|
+
- **`docs/ANTI-CREEP-BASELINE.json`** records `baselineCore: 45` with
|
|
1795
|
+
`previousBaselineCore` left at **0** deliberately. The law compares the tier
|
|
1796
|
+
against the _previous_ baseline, so leaving it at 0 keeps the move from 0 to
|
|
1797
|
+
45 visible as growth; setting it to 45 in the same edit would make a
|
|
1798
|
+
redefinition of the governed set look like a legal no-op, which is the exact
|
|
1799
|
+
escape that rule exists to close.
|
|
1800
|
+
- **Law 3 (north-star) keeps governing the core tier, on purpose.** The ≥10
|
|
1801
|
+
verdict requirement is enforced against `tier === "core"` and was not moved to
|
|
1802
|
+
the shipped set: applying it to 45 rules would fail the check on day one. That
|
|
1803
|
+
is a policy decision about what the product may ship, not a defect, so it is
|
|
1804
|
+
recorded here rather than taken unilaterally. The open question — does the
|
|
1805
|
+
≥10 requirement apply to the shipped set or the core tier? — is unresolved and
|
|
1806
|
+
belongs to the law's owner.
|
|
1807
|
+
- `CORE_CAP` stays at 65 and `MAX_SAMPLES_PER_RULE` stays at 20. Both are
|
|
1808
|
+
deliberate; the corpus sampler's in-source rationale (`scripts/corpus-sample.ts`)
|
|
1809
|
+
records why raising the cap produced 1,121 unadjudicated rows that the
|
|
1810
|
+
committed ceiling refused. The defect was never the constants — it was that
|
|
1811
|
+
the law measured a set of zero.
|
|
1812
|
+
|
|
1813
|
+
### Single-site corrections, and one key that matched nothing
|
|
1814
|
+
|
|
1815
|
+
- **`FRAMEWORK_FAMILY.selenium` and `SLUG_TO_FAMILY.selenium` were `"QA-SEL"`.**
|
|
1816
|
+
No rule id carries that prefix — the real one is `QA-SE` — so both maps
|
|
1817
|
+
resolved to the empty list. The Selenium capability reported no evidence at
|
|
1818
|
+
all while its three rules (`QA-SE-001`–`003`) were credited to nothing.
|
|
1819
|
+
An unmatched key returns an empty result rather than an error, which is why
|
|
1820
|
+
nothing reported it. `QA-SE` was the only family in the registry that no
|
|
1821
|
+
capability could reach.
|
|
1822
|
+
- **`M5_FIELD_PROVEN` did not require `M4`.** `fieldProven` was the first test
|
|
1823
|
+
in `deriveMaturityFromEvidence`, so a capability carrying only that flag
|
|
1824
|
+
reached M5 — a level whose published meaning includes "declared" and
|
|
1825
|
+
"implemented" — without either. The ladder's own stated invariant ("each level
|
|
1826
|
+
requires every criterion below it") did not hold at the top, and the only route
|
|
1827
|
+
to M5 was to satisfy none of M4's criteria.
|
|
1828
|
+
- **`LOCAL_PROVEN` could cite an unmeasured rule.** `proofFor` picked the first
|
|
1829
|
+
non-_retired_ rule, so a capability whose first listed rule had no corpus
|
|
1830
|
+
evidence could be marked proven with a pointer into `docs/FP-AUDIT.md` for a
|
|
1831
|
+
rule the audit never measured. It now picks a rule from `facts.measured` —
|
|
1832
|
+
the same predicate the status itself reports on.
|
|
1833
|
+
- **A parity violation could name a field that cannot differ.** `checkParity`
|
|
1834
|
+
guarded on the fingerprints being different and then iterated
|
|
1835
|
+
`["intention", "fingerprint"]`, so the `fingerprint` arm was unreachable. Its
|
|
1836
|
+
sibling `extractBindings` already looped correctly over the same files for the
|
|
1837
|
+
same kind of marker.
|
|
1838
|
+
- **The claim linter reported the first claim on a line and dropped the rest.**
|
|
1839
|
+
`scanText` used a single `exec` per line, so a line making two claims of the
|
|
1840
|
+
same kind reported one — and the half it dropped was the half a maintainer has
|
|
1841
|
+
to go and fix. All matches are now reported, with a zero-length guard so a
|
|
1842
|
+
future pattern that can match empty cannot hang the linter.
|
|
1843
|
+
- **The forensics `successRate` was not a rate.** It could only be 0 or 1, and
|
|
1844
|
+
this object is only built when a test passed _on retry_ — so within
|
|
1845
|
+
`maskedFailures` it was invariably 1. A reader seeing "success rate: 1" would
|
|
1846
|
+
conclude the test passed on every attempt, which is the opposite of what the
|
|
1847
|
+
entry means. Renamed `finalPassBinary`. Its `attempts === 0` guard was
|
|
1848
|
+
unreachable: the enclosing filter requires `attempts > 1`.
|
|
1849
|
+
- **Two CI findings reported themselves under the wrong name.** An `exit 0` was
|
|
1850
|
+
typed `or-true` while its own description said it forced an exit code of 0, and
|
|
1851
|
+
the pattern `||\s*(?:true|:)` matched both `|| true` and `|| :` while the
|
|
1852
|
+
description said `|| true` unconditionally — naming a construct the script does
|
|
1853
|
+
not contain, on exactly the finding a reader is about to go looking for. They
|
|
1854
|
+
are now `or-true`, `or-colon` and `exit-zero`.
|
|
1855
|
+
- **A rule that sampled nothing no longer overwrites its review sheet.**
|
|
1856
|
+
`writeReviewSheets` wrote a sheet whose entire content was a heading, the
|
|
1857
|
+
verdict legend, and "Total sampled: 0" — and replaced any sheet from an earlier
|
|
1858
|
+
run that had samples, destroying pending classifications. A zero-sample rule now
|
|
1859
|
+
removes its sheet. This is distinct from a rule this run did not _visit_, whose
|
|
1860
|
+
sheet is owner work-in-progress and is still preserved.
|
|
1861
|
+
|
|
1862
|
+
### The readiness table told maintainers to do the wrong work
|
|
1863
|
+
|
|
1864
|
+
`docs/CORE-READINESS.md` is the document a maintainer reads to decide what to
|
|
1865
|
+
do next, and two of its three actionable columns did not compute what they said.
|
|
1866
|
+
|
|
1867
|
+
- **`FP headroom` could never contain a positive number.** It asked how many
|
|
1868
|
+
_more_ false positives a rule could be **added** and still clear the ceiling —
|
|
1869
|
+
and adding false positives only widens the interval, so every rule that had
|
|
1870
|
+
already earned the tier read `0` and every other rule read `—`. The quantity
|
|
1871
|
+
that actually decides work is the opposite one: how much of the current
|
|
1872
|
+
evidence has to be **retracted**. Renamed `FPs to remove` and computed by
|
|
1873
|
+
searching downward from the observed count. It is now `—` only where
|
|
1874
|
+
retraction genuinely cannot reach the ceiling at that `n`, which is a fact
|
|
1875
|
+
about the sample rather than a missing value.
|
|
1876
|
+
- **`NEEDS-FP-REDUCTION` condemned detectors on thin evidence.** It fired on a
|
|
1877
|
+
point estimate above the ceiling, so a rule observed at 42.9% over `n=14` was
|
|
1878
|
+
labelled "the rule is wrong, not the corpus" by an interval that comfortably
|
|
1879
|
+
_included_ the ceiling. It now requires the Wilson interval to exclude the
|
|
1880
|
+
ceiling from below, so a rule the data cannot yet distinguish from a
|
|
1881
|
+
conforming one is told to gather more evidence first. **Nine rules moved from
|
|
1882
|
+
`NEEDS-FP-REDUCTION` to `NEEDS-SAMPLES`**, and the document's own counts
|
|
1883
|
+
change with them (28 → 19, 45 → 54).
|
|
1884
|
+
- **The same FP count was reconstructed two ways.** `measurementInterval` and
|
|
1885
|
+
`compareFpMeasurements` handed `wilsonInterval` an unrounded `fpRate * n`
|
|
1886
|
+
while this table rounded it — 10.5% of 19 is 1.995, and `wilsonInterval`
|
|
1887
|
+
rounds internally, so the ratchet and the readiness table derived two
|
|
1888
|
+
different `ciHigh` values for the same rule at the same n. Both round now. The
|
|
1889
|
+
interval math was always correct; the defect was in what reached it.
|
|
1890
|
+
- **`samplesForZeroFp`'s cross-check was wrong.** Its docstring quotes two
|
|
1891
|
+
`wilsonInterval(0, n)` values as the verification of the closed form. The
|
|
1892
|
+
`n=34` figure was quoted as 0.1012; the function returns 0.1015. A
|
|
1893
|
+
cross-check that fails when you run it reads as evidence and is not, so the
|
|
1894
|
+
numbers are now asserted in a test rather than only in prose.
|
|
1895
|
+
|
|
1896
|
+
### Three places a claim was checked by something other than the thing it claimed
|
|
1897
|
+
|
|
1898
|
+
- **CONVERGENT did not mean convergence.** `groupByRootCause` fell back to
|
|
1899
|
+
`ruleId`, and nothing in the tree assigns `rootCauseId` — the machine
|
|
1900
|
+
contract reports it `null` on every finding — so the key for every group was
|
|
1901
|
+
the rule. N independent defects from one rule across N files rendered as
|
|
1902
|
+
"CONVERGENT — N findings share root cause QA-PW-002", which is the inverse of
|
|
1903
|
+
the claim: several findings being one thing versus several findings sharing a
|
|
1904
|
+
rule id. The fallback is now the finding fingerprint, `rootCauseId` still
|
|
1905
|
+
wins the moment anything populates it, and the listed identities are distinct
|
|
1906
|
+
— the old fallback made a three-finding group list one id three times, so a
|
|
1907
|
+
consumer could not tell the members apart, and the same-file AMPLIFIED
|
|
1908
|
+
conclusion it suppressed was being lost as collateral.
|
|
1909
|
+
- **Two glob compilers disagreed, and a gate used the wrong one.** `ignores.ts`
|
|
1910
|
+
and `scan-pipeline.ts` each compiled globs, and they differed on `?` (a
|
|
1911
|
+
metacharacter in one, a literal in the other) and on both forms of `**`. The
|
|
1912
|
+
mass-suppression integrity gate reached into `ignores.ts` for its compiler
|
|
1913
|
+
while the scan applied suppressions through `scan-pipeline.ts` — so it
|
|
1914
|
+
measured the suppressed set in a different dialect than the one that
|
|
1915
|
+
suppressed it, counting findings the scan never suppressed. Both now compile
|
|
1916
|
+
through `src/lib/glob.ts`, which is the only implementation of the dialect.
|
|
1917
|
+
One behaviour narrows: config `exclude` no longer reads `?` as a wildcard. No
|
|
1918
|
+
shipped default or repository config uses `?`, and `*`/`**` are unchanged.
|
|
1919
|
+
- **An unparseable expiry was perpetual.** `new Date("2026-13-45").getTime()`
|
|
1920
|
+
is `NaN`, and every comparison against `NaN` is false — so a suppression with
|
|
1921
|
+
a typo in its expiry date was never reported expired and suppressed findings
|
|
1922
|
+
forever. A date nobody can read now counts as expired; a suppression that
|
|
1923
|
+
genuinely never expires must say so by omitting `expires`, which is a
|
|
1924
|
+
decision someone made on purpose.
|
|
1925
|
+
|
|
1926
|
+
`detectMassSuppression`'s threshold boundary was already `>=`; what was missing
|
|
1927
|
+
is that the boundary was stated nowhere, so "50%" read as "above 50%".
|
|
1928
|
+
|
|
1929
|
+
### Deletions, and one that was hiding another
|
|
1930
|
+
|
|
1931
|
+
- **`src/v6/tool-coverage.ts` deleted.** `ScanResult.toolCoverage` was declared
|
|
1932
|
+
and never assigned by anything — no assignment exists in `src/` or `scripts/`
|
|
1933
|
+
— so the block it assembled reached no surface, no report and no contract.
|
|
1934
|
+
Its header claimed "the census gap list shrinks by 15 real entries"; that
|
|
1935
|
+
reduction comes from `NOT_QA_TOOLING` in `ecosystem-census.ts`, which
|
|
1936
|
+
classifies names regardless of who calls it. The module contributed nothing to
|
|
1937
|
+
it. Recorded as a `DELETE` disposition in `docs/CARVE-MANIFEST.json` with that
|
|
1938
|
+
evidence, rather than dropped from the record.
|
|
1939
|
+
- **Deleting it exposed a second gap.** Its only importer was
|
|
1940
|
+
`src/v6/test-doubles.ts` — the false-proof detector, the module this product
|
|
1941
|
+
is named after. Nothing in `src/` or `scripts/` has called it since. That is
|
|
1942
|
+
now an explicit entry on the committed orphan list with the reason, rather
|
|
1943
|
+
than a fact hidden by a dead import edge. The `DOUBLE_ONLY`/`TRUTHY` fix
|
|
1944
|
+
landed on it in this release, so the logic is right and the wiring is still
|
|
1945
|
+
owed. It is listed, not deleted: a thesis module that is unwired is a
|
|
1946
|
+
product gap, and deleting it would make the gap invisible rather than gone.
|
|
1947
|
+
- **`src/scorer/scoring-validation.ts` deleted.** An orphan that documents four
|
|
1948
|
+
scoring invariants and implements two, one of them vacuous. It also carried a
|
|
1949
|
+
duplicate `SCORING_MODEL_VERSION` — `contract-versions.ts` owns that
|
|
1950
|
+
constant and `scan-pipeline.ts` reads it from there. Fixing and wiring it
|
|
1951
|
+
would have been strictly more code than deleting a module nothing runs.
|
|
1952
|
+
- **`--policy` no longer prints a scoring formula.** It read
|
|
1953
|
+
`score = 100 · (1 − deductions / (findings + NORMALIZATION_K))`, which
|
|
1954
|
+
contradicts `scorer.ts` on four counts: the denominator is test
|
|
1955
|
+
_declarations_, not findings; `SMOOTHING_C` is missing from it;
|
|
1956
|
+
`NORMALIZATION_K` multiplies the rate rather than sitting in the denominator;
|
|
1957
|
+
and the `min(100, …)` cap is absent, so the line described a score that can go
|
|
1958
|
+
negative. A table whose header says "every number here is imported, never
|
|
1959
|
+
written down" was carrying a formula written by hand, and it was wrong. It now
|
|
1960
|
+
names `docs/SCORING.md` and `scorer.ts` instead — a restated formula is the
|
|
1961
|
+
same defect one layer over, and it drifts the first time the scorer changes.
|
|
1962
|
+
- **Three empty imports removed from `src/commands/milestone.ts`** —
|
|
1963
|
+
`import {} from …` on three modules, which asserted a dependency the file
|
|
1964
|
+
does not have.
|
|
1965
|
+
|
|
1966
|
+
### One path check, three callers: a claim that names a deleted file is now a failed check
|
|
1967
|
+
|
|
1968
|
+
Every artifact claim in this repository is a path inside a file — a
|
|
1969
|
+
`provisionalArtifacts` entry in `ROADMAP.yaml`, an `evidence` citation in the
|
|
1970
|
+
v6 inventory, a `regression_test`/`evidence_artifact` on an M26 ledger row.
|
|
1971
|
+
Three validators read those claims and all three asked the same weak question,
|
|
1972
|
+
"is this field a non-empty string?", so a citation to a module the v6 carve
|
|
1973
|
+
deleted validated clean. One check now serves all three, so a fourth caller
|
|
1974
|
+
cannot come back weaker.
|
|
1975
|
+
|
|
1976
|
+
- **`docs:roadmap:backlinks` is wired into the release and nightly tiers.**
|
|
1977
|
+
It was correct and red the whole time — reporting 19 missing provisional
|
|
1978
|
+
artifacts — and in no tier at all. It now runs through `gates:claim-integrity`,
|
|
1979
|
+
which is where a roadmap's artifact claims belong.
|
|
1980
|
+
- **The 19 dead `provisionalArtifacts` rows are deleted, not repointed.** M29,
|
|
1981
|
+
M31–M36, M38, M39, M41, M43–M50 each declared an artifact the carve
|
|
1982
|
+
removed. A successor is a different module answering a different question,
|
|
1983
|
+
so citing one would turn "this was never built" into a claim that something
|
|
1984
|
+
was.
|
|
1985
|
+
- **`docs/V6-CURRENT-STATE.md` and `docs/V6-GAP-MATRIX.md` are drift-locked.**
|
|
1986
|
+
Only the JSON halves of these generated artifacts were ever compared against
|
|
1987
|
+
a fresh render. The `.md` halves were hand-editable, which is how a
|
|
1988
|
+
hand-edited line came to contradict the `docs/v6-inventory.json` beside it.
|
|
1989
|
+
Both are now compared, and both sides are formatted in memory so the check
|
|
1990
|
+
still never writes to the tree it inspects.
|
|
1991
|
+
- **A generated artifact may no longer cite a file that does not exist.**
|
|
1992
|
+
Every repo-relative path in the rendered `.md` is now checked, which is what
|
|
1993
|
+
found the 11 dead citations the §100 classification was still carrying. The
|
|
1994
|
+
claims were corrected at source rather than in the rendered output: §59
|
|
1995
|
+
(dashboard) and §66 (quality debt) are now `OBSOLETE`, which is the state
|
|
1996
|
+
the enum had declared and nothing had ever used.
|
|
1997
|
+
- **M26 ledger rows are BLOCKED when a cited evidence path does not resolve.**
|
|
1998
|
+
`regression_test` and `evidence_artifact` are the only pointer a closure
|
|
1999
|
+
claim has to the thing that proves it. A `warning`, not an `error`: a row may
|
|
2000
|
+
legitimately cite a path this checkout does not carry, and "unverifiable
|
|
2001
|
+
here" is the honest status. The root is a parameter rather than a default,
|
|
2002
|
+
so a record's validity never depends on the working directory.
|
|
2003
|
+
- **Hand-written prose deleted from the generators.** GAP-V6-005's summary
|
|
2004
|
+
carried "14 of the 108" while the matrix beside it reported 18 — two
|
|
2005
|
+
representations of one number, one hand-maintained, where the renderer
|
|
2006
|
+
already derives the authoritative figure. The §38/§65 notes named
|
|
2007
|
+
`pw-report` and `trend`, which are commands, as modules of `src/forensics`.
|
|
2008
|
+
The `QA_DOMAIN_RECORDS` provenance column named a symbol that exists
|
|
2009
|
+
nowhere in `src/`.
|
|
2010
|
+
- **"156 cells" was never true.** The support matrix carries 136 cells and 6 on
|
|
2011
|
+
the language-framework axis. The figure was hand-written in source, so
|
|
2012
|
+
regenerating could not have fixed it; the language manifest now names the
|
|
2013
|
+
surface and points at the matrix that carries the count.
|
|
2014
|
+
- **`docs/ANTI-CREEP.md` pointed at `src/rules/tier-policy.ts`;** the module is
|
|
2015
|
+
at `src/engine/tier-policy.ts`.
|
|
2016
|
+
- **`docs/MANUAL-SCRIPTS.md` claimed `docs:roadmap:backlinks` was wired into
|
|
2017
|
+
the nightly tier.** It was not. It is now, so the row is gone — and the
|
|
2018
|
+
script-reachability gate enforces that a script cannot be both a gate and a
|
|
2019
|
+
manual tool.
|
|
2020
|
+
|
|
2021
|
+
A gate that has never fired is not a gate: the rendered-markdown comparison was
|
|
2022
|
+
verified to fail on a planted hand edit and pass after reverting it.
|
|
2023
|
+
|
|
2024
|
+
### False greens: three places a scan reported a stronger result than it had
|
|
2025
|
+
|
|
2026
|
+
Every ordinary scan withheld 34 quarantined rules and reported nothing about
|
|
2027
|
+
it at three different surfaces. This release closes that gap. **These are
|
|
2028
|
+
user-visible status changes** — an `INCONCLUSIVE` finding is no longer
|
|
2029
|
+
`FIXED`, and SARIF `executionSuccessful` is now `false` on a scan that
|
|
2030
|
+
withheld rules. Existing baselines and saved reports will show the new,
|
|
2031
|
+
weaker claim; that is the point of the change.
|
|
2032
|
+
|
|
2033
|
+
- **A quarantined rule could prove a finding "fixed" on a scan that never ran
|
|
2034
|
+
it.** `analysisStatus` reported _how many_ rules were withheld but not
|
|
2035
|
+
_which_, and lifecycle resolution — which decides one finding at a time —
|
|
2036
|
+
had nothing to go on. A finding whose rule was filtered out disappeared
|
|
2037
|
+
from the scan and rendered `FIXED SINCE BASELINE`. New optional field
|
|
2038
|
+
`analysisStatus.withheldRuleIds` carries the set; `resolve()` returns
|
|
2039
|
+
`INCONCLUSIVE` with a new `rule-withheld` cause when the finding's rule was
|
|
2040
|
+
withheld. With `--strict` the rule runs and the fix resolves normally.
|
|
2041
|
+
`schemaVersion: 1` is unchanged — both new fields are additive
|
|
2042
|
+
(`docs/VERSIONING.md`).
|
|
2043
|
+
- **"FIXED SINCE BASELINE (verified by a complete same-revision scan)" was
|
|
2044
|
+
false for every withheld scan.** The sentence claimed the whole scan was
|
|
2045
|
+
complete; it now claims only what survived every guard — the rule ran, the
|
|
2046
|
+
file was in scope, it no longer fires.
|
|
2047
|
+
- **SARIF called a `PARTIAL`-coverage scan fully successful.**
|
|
2048
|
+
`executionSuccessful` was `!partial && rulesCrashed === 0`, and
|
|
2049
|
+
`coverageState` is deliberately orthogonal to `partial`, so a scan that read
|
|
2050
|
+
every file it was given and ran 45 of 79 detectors reported success to GitHub
|
|
2051
|
+
code scanning. `executionSuccessful` now also requires
|
|
2052
|
+
`coverageState !== "PARTIAL"`, with a `toolExecutionNotifications` entry
|
|
2053
|
+
naming the withheld count and pointing at `--strict`.
|
|
2054
|
+
- **`analysisStatus.reasons` now carries `coverage:quarantine:<n>`,** which
|
|
2055
|
+
`types.ts` already promised and nothing emitted. The flat reason set is what
|
|
2056
|
+
a machine consumer reads, and it was empty for the one gap that affects
|
|
2057
|
+
every non-`--strict` scan. `coverage:*` reasons are excluded from
|
|
2058
|
+
report-io's `partial` markers, so disclosing coverage does not make a saved
|
|
2059
|
+
report unloadable and does not turn ordinary scans into partial ones.
|
|
2060
|
+
- **A lone `expect(x).toBeTruthy()` was reported as a false proof.** The QA-IR
|
|
2061
|
+
gives `TRUTHY` an `"ANY"` expectation — an explicit, evidence-bearing shape
|
|
2062
|
+
— while the double-risk model did not list it as substantiating. The two
|
|
2063
|
+
modules disagreed, so the assertion counted as neither substantiating nor a
|
|
2064
|
+
double and fell into `DOUBLE_ONLY`: "every assertion is about a test double"
|
|
2065
|
+
about a test with no double assertions, at a `hollowRatio` of 0. `TRUTHY` is
|
|
2066
|
+
now substantiating, and `DOUBLE_ONLY` requires at least one double.
|
|
2067
|
+
- **The trust-trend report invented numbers.** `score` is `null` for a repo
|
|
2068
|
+
with no tests, and `(last ?? 0) - (first ?? 0)` reported a fabricated
|
|
2069
|
+
`+100` when a repo gained its first score. `scoreDelta` is now
|
|
2070
|
+
`number | null` and the rendering says "not comparable". `overallDirection`
|
|
2071
|
+
compared the _number_ of regression entries against improvements, so one
|
|
2072
|
+
50-point drop lost to three 1-point gains and the trend read "improving" —
|
|
2073
|
+
it now sums magnitude. `frameworkCount` was `1` whenever detection was known
|
|
2074
|
+
(four detected frameworks and one rendered identically) and is now
|
|
2075
|
+
`frameworks.length`. The `warnings > errors * 2` ratio degenerated to
|
|
2076
|
+
`warnings > 0` at zero errors, reporting "weak assertions" for every repo
|
|
2077
|
+
with warnings and none; it now requires a denominator.
|
|
2078
|
+
|
|
2079
|
+
### Names: one command, one name
|
|
2080
|
+
|
|
2081
|
+
`package.json` carried eight families of stragglers — two names for the same
|
|
2082
|
+
command, a name and its write arm under different spellings, and a chain whose
|
|
2083
|
+
members each had a name of their own. Nothing failed, because npm has no
|
|
2084
|
+
opinion and a second name is indistinguishable from a first one.
|
|
2085
|
+
|
|
2086
|
+
- **`build --determinism` and `unimported --check` had twins.** `unimported:check`
|
|
2087
|
+
and `check-unimported-modules` both ran `scripts/check-unimported-modules.mjs`;
|
|
2088
|
+
`build:determinism` and `verify-build-determinism` both ran
|
|
2089
|
+
`scripts/verify-build-determinism.mjs`; and so on for
|
|
2090
|
+
`fp-audit:generate`/`generate-fp-audit-table` and `docs:rules`/`generate-rule-docs`.
|
|
2091
|
+
A reader who typed the wrong one got the right answer and no signal that the
|
|
2092
|
+
name was wrong.
|
|
2093
|
+
- **`check-cli-contract` is the gate that made it visible.** It reads
|
|
2094
|
+
`docs/cli-contract.json`, which carries one disposition per verb in
|
|
2095
|
+
`src/engine/cli-command-names.ts`, and it fails when two npm names resolve to
|
|
2096
|
+
the same entry with the same argv, when a verb has no disposition, and when
|
|
2097
|
+
any `.md`, workflow, or Action names an `npm run` script that does not exist.
|
|
2098
|
+
A check arm and a write arm of one generator is **not** a duplicate — that is
|
|
2099
|
+
a declared convention in eleven places — so the rule is about the command,
|
|
2100
|
+
not the file.
|
|
2101
|
+
- **`corpus:regression`, `corpus:regression:update`, `corpus:resample` and
|
|
2102
|
+
`corpus:regression:refresh-provenance` are now `corpus:audit` with flags.**
|
|
2103
|
+
Same subject, four names. `scripts/resample-unmeasured.ts` became a guarded
|
|
2104
|
+
module the audit imports rather than a second entry point.
|
|
2105
|
+
- **`rules:quad:check`, `rules:quad:write` and `corpus:quad:verdicts` are now
|
|
2106
|
+
`check-fixture-quad` with flags.** The census's own report already told
|
|
2107
|
+
readers to run `npm run check-fixture-quad --verdicts`, which was not a name
|
|
2108
|
+
that existed.
|
|
2109
|
+
- **`version:check`, `version:surface:check`, `reporter:version-check`,
|
|
2110
|
+
`changelog:check` and `changelog:unreleased` are now `check-version`.** It is
|
|
2111
|
+
a file, not an npm chain, for a reason that bit during this change: `ci.yml`
|
|
2112
|
+
runs the unreleased-entry check with `--base=origin/<base>`, and npm appends a
|
|
2113
|
+
flag to the LAST command of a chain. A chain that is reordered by one line
|
|
2114
|
+
routes the flag to a check that ignores it, and the step keeps reporting PASS
|
|
2115
|
+
while comparing against an empty `git status` on a fresh CI checkout. The flag
|
|
2116
|
+
now has exactly one consumer, dispatched explicitly, and any other argument is
|
|
2117
|
+
a usage error rather than a silent no-op.
|
|
2118
|
+
- **`site:doctor` is now `doctor`**, and `doctor` runs the product's own
|
|
2119
|
+
self-audit before the site's, so the name a maintainer types is the name that
|
|
2120
|
+
diagnoses Mjölnir.
|
|
2121
|
+
|
|
2122
|
+
### Carve 1.1 — twenty-two READMEs, and the gate that could only be met by deleting them
|
|
2123
|
+
|
|
2124
|
+
`README.<lang>.md` existed for 22 languages, machine-assisted, each about a
|
|
2125
|
+
dozen sections behind the English file. `docs/TRANSLATION-RATCHET.json` recorded
|
|
2126
|
+
`translationsNotFresh: 22` against a ceiling of 22 — so the ratchet could only
|
|
2127
|
+
ever be satisfied by deleting the thing it measured, and the maintainer had
|
|
2128
|
+
already documented that the failing branch was unreachable for the shape of
|
|
2129
|
+
regression it was written for. It was, in the end, a gate that measured nothing.
|
|
2130
|
+
|
|
2131
|
+
Deleted: the 22 files, the advisory report, the strict variant, the ratchet, the
|
|
2132
|
+
release-status sync, the shared language list, and the three specs that policed
|
|
2133
|
+
them. `README.md` is the only README. `docs/EXTERNAL-CONFIG.md` and the three
|
|
2134
|
+
gate rows in `gates/{pr,release,nightly}.json` were updated rather than left
|
|
2135
|
+
pointing at commands that no longer exist, and `tests/contract/gate-exit-codes.spec.ts`
|
|
2136
|
+
now asserts the machinery is _absent_ — a deleted gate is a hole with a comment
|
|
2137
|
+
on it, and the comment is the only thing that survives to be trusted.
|
|
2138
|
+
|
|
2139
|
+
The relic that went with them: `package/README.md`, a hand-maintained 778-line
|
|
2140
|
+
copy of the root README that nothing read and that had drifted 62 lines behind
|
|
2141
|
+
its own source. `pack-audit` matches tarball entries by name, so the fixture
|
|
2142
|
+
directory was never a real one.
|
|
2143
|
+
|
|
2144
|
+
### Phase 2 — the matrix and the gaps: a claim, and a work queue
|
|
2145
|
+
|
|
2146
|
+
**`generate-certification-matrix`** writes `docs/CERTIFICATION-MATRIX.md` from
|
|
2147
|
+
three artifacts — the language manifest, the declared surface, and the cell
|
|
2148
|
+
records — and `--check` is a gate in all three tiers. The first output is
|
|
2149
|
+
**`0/100 cells certified`**, which is the honest number and not a failing state:
|
|
2150
|
+
a gate that blocks on progress reports nothing else.
|
|
2151
|
+
|
|
2152
|
+
The header says what the percentage means on every rendering, because a bare
|
|
2153
|
+
percentage beside a language name is exactly the misreading §5.4 warns about:
|
|
2154
|
+
_"the fraction of this ecosystem's required detection surface backed by
|
|
2155
|
+
certified evidence"_ — **not** how trustworthy a language is.
|
|
2156
|
+
|
|
2157
|
+
**`generate-certification-gaps`** turns each of those 100 open cells into an
|
|
2158
|
+
issue-ready entry: the ecosystem, the concept, the language and framework, the
|
|
2159
|
+
gate that fails, **the arithmetic of why**, and the floor that closes it. A gap
|
|
2160
|
+
entry that says "n too small" is a task description; one that says
|
|
2161
|
+
`precision n=0 is below 35` is a specification.
|
|
2162
|
+
|
|
2163
|
+
**Gate D was unfalsifiable, and the generated report is what showed it.** It
|
|
2164
|
+
failed any cell whose concept had more than one language binding — which means
|
|
2165
|
+
no cell of the three-language concepts could _ever_ pass, and 40 of the 100 open
|
|
2166
|
+
entries listed gate D. Language parity is a property of the SET, computed by
|
|
2167
|
+
the ecosystem roll-up in the matrix; a single cell cannot answer it. What a cell
|
|
2168
|
+
_can_ answer is whether it names a language its concept is bound to, which is
|
|
2169
|
+
what gate D says now.
|
|
2170
|
+
|
|
2171
|
+
**Seven missing manifest rows, added.** §5.1's table claims eleven ecosystems
|
|
2172
|
+
with none deleted, and seven had no row — so the matrix had nothing to print for
|
|
2173
|
+
GitHub Actions, Playwright, Cypress, Selenium, GitLab CI, Jenkins or Azure
|
|
2174
|
+
Pipelines. Every one is `DISCOVERED`, which is §5.1's own definition for "we
|
|
2175
|
+
have seen it exist"; `UNMEASURED` would claim a measurement nobody ran.
|
|
2176
|
+
`ECOSYSTEM_MANIFEST` and `ALL_ECOSYSTEMS` are the new names, with the old
|
|
2177
|
+
`LANGUAGE_MANIFEST` kept because four specs import it — renaming an export to
|
|
2178
|
+
fix a noun is a breaking change dressed as tidying.
|
|
2179
|
+
|
|
2180
|
+
**Three PENDING-CONSUMER rows deleted themselves**, which is the mechanism
|
|
2181
|
+
working rather than a nuisance. `validate.ts`, `record.schema.ts` and
|
|
2182
|
+
`language-manifest.ts` were allowlisted because nothing imported them; the
|
|
2183
|
+
moment the matrix generator imported them the orphan gate said _"an entry that
|
|
2184
|
+
no longer describes reality is a lie in a data file"_ and failed. An exemption
|
|
2185
|
+
that outlives its reason is how a gate starts protecting things nobody
|
|
2186
|
+
remembers.
|
|
2187
|
+
|
|
2188
|
+
### Phase 2 — the §6 cell record: counts in, verdicts out, and what the schema refuses
|
|
2189
|
+
|
|
2190
|
+
**`src/certification/record.schema.ts`** — one record per
|
|
2191
|
+
`concept × language × framework`, as a JSON Schema plus a type-aware validator,
|
|
2192
|
+
following the repository's existing pattern (`src/config/config-schema.ts`)
|
|
2193
|
+
rather than buying a schema library for one file.
|
|
2194
|
+
|
|
2195
|
+
**Three things the schema refuses, and one it cannot.**
|
|
2196
|
+
|
|
2197
|
+
It refuses a stored Wilson bound in three places — the `not` clause, the
|
|
2198
|
+
`additionalProperties: false`, and the validator — because the plan's
|
|
2199
|
+
counter-example (`{fp: 5, n: 250, fpWilsonUpper: 0.02}`) reads as valid to
|
|
2200
|
+
anything that does not recompute it. It refuses one _nested_ inside the counts,
|
|
2201
|
+
which is where someone actually pastes a rate they just computed; a top-level
|
|
2202
|
+
check passes it. And it refuses `standard: "manifest-v5"` on a cell record.
|
|
2203
|
+
|
|
2204
|
+
It cannot express `n === tp + fp`, so the validator does. An `n` that disagrees
|
|
2205
|
+
with its own parts asserts something the evidence beside it does not support —
|
|
2206
|
+
the arithmetic twin of the stored-bound defect, and one nothing recomputes
|
|
2207
|
+
because an integer looks final.
|
|
2208
|
+
|
|
2209
|
+
**§5.2's two columns, spelled.** `CELL_STATES` is `v6-PENDING`,
|
|
2210
|
+
`v6-PARSEABLE`, `v6-CERTIFIED`, … and the validator rejects a bare
|
|
2211
|
+
`CERTIFIED`. TypeScript must read as `CERTIFIED (manifest-v5, n=240) · 0/9 cells
|
|
2212
|
+
(v6-abcdef, PENDING)` — the plan's own worked example, and the reason a cell
|
|
2213
|
+
state carries its standard in the name. A bare `PENDING` beside a `CERTIFIED` in
|
|
2214
|
+
the same row is a sentence two readers will contradict each other about.
|
|
2215
|
+
|
|
2216
|
+
**§6.4's re-attachment needs an arm.** `legacyVerdicts[].arm` exists because
|
|
2217
|
+
multi-arm rules are real here: `TQUAL-001` carries a no-assertion arm and a
|
|
2218
|
+
mock-only arm, `PW-124` a project-split arm and a config-gap arm. Keying on
|
|
2219
|
+
`(oldRuleId)` alone would attach half a rule's verdicts to the wrong arm.
|
|
2220
|
+
|
|
2221
|
+
### Phase 2 — `EXPECTED_CERTIFICATION_SURFACE`: the denominator, declared
|
|
2222
|
+
|
|
2223
|
+
**`src/certification/surface-manifest.ts`** — 107 cells across all eleven §5.1
|
|
2224
|
+
ecosystems, each `REQUIRED` or `NOT_APPLICABLE` **with a reason**, plus
|
|
2225
|
+
`check-certification-surface` and its committed baseline.
|
|
2226
|
+
|
|
2227
|
+
Without a committed denominator, "60% → 100%" is achievable by deleting the
|
|
2228
|
+
five cells that were not certified. That is the one move that improves every
|
|
2229
|
+
number in the report without anybody writing a fixture, and no metrics-only
|
|
2230
|
+
report can tell it from progress.
|
|
2231
|
+
|
|
2232
|
+
Four properties, each with a test:
|
|
2233
|
+
|
|
2234
|
+
- **The surface cannot shrink silently.** The gate compares the manifest
|
|
2235
|
+
against a committed baseline, so a removal committed last month is still
|
|
2236
|
+
visible today. A gate that compared the manifest against itself could not
|
|
2237
|
+
see it at all — the change would already have passed.
|
|
2238
|
+
- **`REQUIRED` may grow freely.** Raising the bar is never gated; only
|
|
2239
|
+
shrinking is, and only with `--acknowledge-surface-change=<why>`, which moves
|
|
2240
|
+
the baseline and records the reason beside the new list.
|
|
2241
|
+
- **Every exclusion NAMES AN ALTERNATIVE.** `NOT_APPLICABLE` is a first-class
|
|
2242
|
+
state and it is printed, but a reason that says only why _not here_ is a
|
|
2243
|
+
deletion wearing a label. The gate rejects it; the plan's own example — a CI
|
|
2244
|
+
concept on a Python ecosystem — shows the real shape.
|
|
2245
|
+
- **No cell names a concept the vocabulary does not have.** This caught three on
|
|
2246
|
+
the first run: `frameloctor-…`, `ignored-exit-code-or-true` and
|
|
2247
|
+
`trial-click-…` were ids I guessed rather than read, and a `REQUIRED` cell for
|
|
2248
|
+
an undefined concept is a denominator entry nobody can ever close.
|
|
2249
|
+
|
|
2250
|
+
**The three wave-C ecosystems with zero corpus repositories STAY.** Deleting
|
|
2251
|
+
GitLab CI, Jenkins and Azure Pipelines would improve every percentage in the
|
|
2252
|
+
report by removing the ecosystems nobody has measured. §5.1 retains all
|
|
2253
|
+
eleven; a wave is a ranking, not a deletion.
|
|
2254
|
+
|
|
2255
|
+
**Two defects the spec found in the gate it was written for.**
|
|
2256
|
+
|
|
2257
|
+
The gate computed `removed` and `added` with the _same_ filter over the current
|
|
2258
|
+
keys — so `removed` was really the added set, and a genuine shrink was
|
|
2259
|
+
invisible. The gate that exists to catch a shrinking denominator did not
|
|
2260
|
+
detect one. The fixture shrank the surface by a cell and the gate said PASS,
|
|
2261
|
+
which is how it was found.
|
|
2262
|
+
|
|
2263
|
+
And the gate **swallowed a failure to read its own manifest**: with no
|
|
2264
|
+
`node_modules` in the fixture tree the `tsx` spawn failed, the error was
|
|
2265
|
+
ignored, and it printed `PASS — 107 cells`. A gate that reports a pass because
|
|
2266
|
+
it could not read its input is the loudest possible wrong answer, so an
|
|
2267
|
+
unreadable manifest is now exit 10 and says so. The fixture needed a
|
|
2268
|
+
`node_modules` junction for a different reason: it copied the gate and then
|
|
2269
|
+
invoked the _real_ one, which computes its root from its own location — so the
|
|
2270
|
+
mutation was applied to a copy nobody read, for three runs.
|
|
2271
|
+
|
|
2272
|
+
### Phase 2 — the concept vocabulary, and the §6 contract that judges a cell
|
|
2273
|
+
|
|
2274
|
+
**`src/certification/concepts.ts`** — 60 concepts covering all 79 rules,
|
|
2275
|
+
**generated** from the registry by `scripts/generate-concepts.ts`, **committed**
|
|
2276
|
+
for review, and **verified** by `tests/contract/certification-concepts.spec.ts`.
|
|
2277
|
+
The registry is the authority; the table is a claim about it a reader can check.
|
|
2278
|
+
|
|
2279
|
+
A concept is a failure MODE and a rule is that mode in one language. The
|
|
2280
|
+
derivation is mechanical because the titles in this tree already assert the
|
|
2281
|
+
grouping: `QA-TEST-004` ("Hard sleep in test", TypeScript) and `QA-PY-005`
|
|
2282
|
+
("time.sleep() in test", Python) are different rules for the same concept, and
|
|
2283
|
+
§6 certifies `concept × language × framework` — so a concept that split them
|
|
2284
|
+
would certify one detector twice and report two cells where there is one.
|
|
2285
|
+
|
|
2286
|
+
It yields 60 concepts where the plan says 32, because the plan's concepts are
|
|
2287
|
+
COARSER: it calls `test-cannot-fail` one concept, the tree calls it five ("Test
|
|
2288
|
+
without assertions", "Empty test body", "Skipped test", "Focused test",
|
|
2289
|
+
"Disabled test"). Both are defensible and they measure different things, so
|
|
2290
|
+
the fine cut is committed and the coarse groupings are expressible as a VIEW
|
|
2291
|
+
over it — additive and reviewable, where merging rows is not. The count is
|
|
2292
|
+
reported, not gated, which is the plan's own rule for counts.
|
|
2293
|
+
|
|
2294
|
+
**`src/certification/validate.ts`** — §6 as code: six independent gates, counts
|
|
2295
|
+
in and verdicts out.
|
|
2296
|
+
|
|
2297
|
+
**Counts are stored; intervals never are.** `CellEvidence` has no
|
|
2298
|
+
`fpWilsonUpper` field and `assertCountsOnly` rejects a JSON record that has
|
|
2299
|
+
grown one — walking the TREE, because the first version checked only the top
|
|
2300
|
+
level and `fpRate` nested inside `precision` is exactly where someone would
|
|
2301
|
+
paste it. The plan's own counter-example is `{fp: 5, n: 250}` with a stored
|
|
2302
|
+
0.02 upper bound: it looks valid to anything that does not recompute it.
|
|
2303
|
+
|
|
2304
|
+
The thresholds are pinned by tests, because they are not round numbers:
|
|
2305
|
+
`n ≥ 35` is where a PERFECT 35-of-35 clears both a `≤ 0.10` FP upper bound and a
|
|
2306
|
+
`≥ 0.90` recall lower bound — `wilson(0,35) = 0.0989` and `wilson(35,35) =
|
|
2307
|
+
0.9011`. That coincidence is the entire justification for the concept bar, and
|
|
2308
|
+
`n ≥ 34` failing is asserted beside it, because a threshold nobody fails is not
|
|
2309
|
+
a threshold.
|
|
2310
|
+
|
|
2311
|
+
**Three wrong readings of the Wilson bound, and the tests caught all three.**
|
|
2312
|
+
`wilsonInterval(x, n)` is a proportion of SUCCESSES. For precision the successes
|
|
2313
|
+
are the FALSE POSITIVES; for recall they are the TRUE POSITIVES. Reading `fn`,
|
|
2314
|
+
or reading the complement `n - tp`, gives a lower bound of exactly **0** on a
|
|
2315
|
+
perfect cell — the Wilson bound at p = 0 is 0 — so every cell in the repository
|
|
2316
|
+
fails gate B. The plan's own numbers settle it: `wilson(35,35) = 0.9011` is a
|
|
2317
|
+
lower bound on a cell with no failures.
|
|
2318
|
+
|
|
2319
|
+
The tiering is asserted as a RELATIONSHIP rather than a comparison: the same
|
|
2320
|
+
24-of-24 evidence clears the cell bar (0.862 ≥ 0.80) and fails the concept bar
|
|
2321
|
+
(0.862 < 0.90, and `n = 24 < 35`). That is the reason §5.5b has two tiers, and
|
|
2322
|
+
an assertion comparing a bound to a count would have passed for no reason at
|
|
2323
|
+
all — which is what the first version of that test did.
|
|
2324
|
+
|
|
2325
|
+
**One commit-time self-caught defect.** Registering the two new npm scripts, the
|
|
2326
|
+
first draft was `check-concepts: "… generate-concepts.ts"` plus
|
|
2327
|
+
`generate-concepts: "… generate-concepts.ts --write"` — two names, one file,
|
|
2328
|
+
one command. That is the exact straggler pattern Phase 0 existed to remove,
|
|
2329
|
+
committed by the person removing it, and caught by the gate built for the
|
|
2330
|
+
purpose.
|
|
2331
|
+
|
|
2332
|
+
`validate.ts` is on the orphan list as `PENDING-CONSUMER`, with the reason that
|
|
2333
|
+
its first production caller is `generate-certification-matrix.ts`. It is filed
|
|
2334
|
+
as pending rather than as an exemption on purpose: this gate fails an allowlist
|
|
2335
|
+
row that something now IMPORTS, so the row disappears the moment the matrix
|
|
2336
|
+
generator lands.
|
|
2337
|
+
|
|
2338
|
+
### Carve 1.6.1 — the order-dependent pair, and why they are still reachable
|
|
2339
|
+
|
|
2340
|
+
The plan marks `historical-trust.ts` and `machine-contract-verification.ts` as
|
|
2341
|
+
deletable **only** after `commands/milestone.ts` stops needing them, and says to
|
|
2342
|
+
do it here rather than in 1.5 for exactly that reason. Every refactor that
|
|
2343
|
+
removed a verb left `milestone.ts` as the home of the survivors, so two modules
|
|
2344
|
+
on the delete list stayed reachable through a filename that no longer described
|
|
2345
|
+
what the file was.
|
|
2346
|
+
|
|
2347
|
+
`commands/milestone.ts` went from 1,049 lines to 463, split along the seams it
|
|
2348
|
+
already had:
|
|
2349
|
+
|
|
2350
|
+
- **`milestone-args.ts`** — the arg parser, the target validator and the
|
|
2351
|
+
workflow-file check. Seven commands used them and they are **shared**, not
|
|
2352
|
+
copied seven times: a flag rule that exists twice can disagree with itself,
|
|
2353
|
+
and the copy a pipeline runs is the one that has to be right.
|
|
2354
|
+
- **`contract-verify.ts`** — the ~280 lines of nested type guards that decide
|
|
2355
|
+
whether a file on disk is the document the tool claims it wrote, plus the
|
|
2356
|
+
command.
|
|
2357
|
+
- **`release-trend.ts`** — the trust-snapshot history and `ci release-trend`.
|
|
2358
|
+
|
|
2359
|
+
**The two engine modules stay, and `docs/cli-contract.json` records why with
|
|
2360
|
+
their importer named.** `contract-verify` is a live verb (the plan's merge into
|
|
2361
|
+
`ci verify` is a behaviour change to a live command, recorded as
|
|
2362
|
+
`plannedTarget` rather than done), and `ci release-trend` is a live subcommand.
|
|
2363
|
+
The delete list was written when those verbs were going away; the capability
|
|
2364
|
+
moved rather than vanished. "On a delete list" and "reachable from three
|
|
2365
|
+
places" are both true, and only one of them is a reason to delete.
|
|
2366
|
+
|
|
2367
|
+
A range extraction cannot see what a slice depended on, and the compiler listed
|
|
2368
|
+
all 300 resulting errors. Four of them are worth recording because each is a way
|
|
2369
|
+
this goes wrong quietly:
|
|
2370
|
+
|
|
2371
|
+
- `ContractDocument` is `ScanResult & { contract }`, not a wrapper with a `scan`
|
|
2372
|
+
field. I guessed the wrapper. A persisted contract artifact is the scan with
|
|
2373
|
+
its own verdict attached, and a reader that has to know which of the two
|
|
2374
|
+
shapes it holds is a reader that will hold it wrong.
|
|
2375
|
+
- `MilestoneArgs` was declared _above_ the extracted range, so the module that
|
|
2376
|
+
PARSES it no longer contained it. A type with three homes is a type with
|
|
2377
|
+
none; it belongs where the parsing is.
|
|
2378
|
+
- `STRICT_SUPPRESSION_POLICY` sat immediately above the range and stayed behind
|
|
2379
|
+
with its only reader gone — moved down to it.
|
|
2380
|
+
- `ContractDocument` ended up declared in two of the new files, because a
|
|
2381
|
+
declaration and its first use can straddle a boundary. Two identical copies
|
|
2382
|
+
are invisible to the compiler, which is the reason to look for them by hand.
|
|
2383
|
+
|
|
2384
|
+
### Carve 1.6 (MOVE arm, group 3) — three internal verbs folded, two arrows that do not connect
|
|
2385
|
+
|
|
2386
|
+
`pw-report`, `framework-maturity` and `doctor:playwright` all fold into flags —
|
|
2387
|
+
`explain --playwright`, `doctor --frameworks`, `doctor --frameworks` — which is
|
|
2388
|
+
what the plan calls INTERNAL for all three: none is a capability a reader is
|
|
2389
|
+
meant to reach for. `doctor --frameworks` is a FLAG and not a second
|
|
2390
|
+
`argv[0] === "doctor"` branch, because `tests/contract/readme-commands.spec.ts`
|
|
2391
|
+
reads those literals as the known subcommand set and one verb with two entries
|
|
2392
|
+
in it is a set that no longer means what it says.
|
|
2393
|
+
|
|
2394
|
+
**Three of the plan's arrows do not connect, and the contract says so.**
|
|
2395
|
+
|
|
2396
|
+
- `contract-verify` → `ci verify`. `ci verify` is the BLOCKING CHECK and does
|
|
2397
|
+
not run the machine-contract or exit-code validators. Folding it is a
|
|
2398
|
+
behaviour change to a live command, not a rename, so the verb, the
|
|
2399
|
+
implementation and its eleven tests stay and the row reads
|
|
2400
|
+
`plannedTarget: ci verify` with the reason.
|
|
2401
|
+
- `analyze` → `explain`. `analyze` is CROSS-FILE analysis — shared imports,
|
|
2402
|
+
duplicated blocks, circular dependencies — and its findings reference
|
|
2403
|
+
multiple files. `explain` answers "what does this finding mean"; making it
|
|
2404
|
+
answer "which files import each other" would give a verb a capability it
|
|
2405
|
+
does not have.
|
|
2406
|
+
- `handoff` → `install agent-rules`. `install` writes agent INSTRUCTION
|
|
2407
|
+
surfaces; `handoff` renders a deterministic remediation plan from a saved
|
|
2408
|
+
report. One configures an environment, the other renders a document an agent
|
|
2409
|
+
can execute.
|
|
2410
|
+
|
|
2411
|
+
**Two routing bugs the collapse introduced and this step fixed.**
|
|
2412
|
+
|
|
2413
|
+
- `mjolnar ci install --help` started running the installer. The `--help` rule
|
|
2414
|
+
was positional in its first form and had a second special case for the
|
|
2415
|
+
three-token case; the collapse generalised the first and deleted the second,
|
|
2416
|
+
taking the `ci install --help` path with it. The rule is now "a `--help`
|
|
2417
|
+
anywhere in the invocation asks for help", which is the rule that was always
|
|
2418
|
+
true.
|
|
2419
|
+
- `mjolnar ci verify --help` rendered **`ci install`'s** page, because the
|
|
2420
|
+
dispatcher built the subject as `ci:install` and the catalogue keys it
|
|
2421
|
+
`"ci install"` — a space, because no verb name contains one. A `:` was
|
|
2422
|
+
chosen to avoid colliding with `doctor:playwright`, which read as clever
|
|
2423
|
+
right up to the moment it sent a reader to a page that does not exist.
|
|
2424
|
+
|
|
2425
|
+
The root help also no longer prints a section whose verbs are all retired: the
|
|
2426
|
+
collapse emptied two groups and left the headings, and a table of contents that
|
|
2427
|
+
promises a section it does not have is the same class of claim as a command
|
|
2428
|
+
that does not exist. The `GROUPS` list itself was rewritten to name the fourteen
|
|
2429
|
+
entries the catalogue actually has — it used to list twenty-nine, of which the
|
|
2430
|
+
renderer silently skipped twenty, so the file read as a graveyard.
|
|
2431
|
+
|
|
2432
|
+
`tests/cli/help.spec.ts` gained a sweep that renders every catalogue entry and
|
|
2433
|
+
both composite pages and asserts none names a retired verb, reading the
|
|
2434
|
+
retirements out of `docs/cli-contract.json` rather than listing them. The
|
|
2435
|
+
pattern covers slash-separated tool lists as well as invocations, because the
|
|
2436
|
+
`mcp` line carried three retired names in that shape and the narrower sweep was
|
|
2437
|
+
blind to it.
|
|
2438
|
+
|
|
2439
|
+
### Carve 1.6 (MOVE arm, group 2) — five capabilities became `explain` arms
|
|
2440
|
+
|
|
2441
|
+
`explain` is the question verb. Five capabilities were top-level verbs that all
|
|
2442
|
+
answer a variation of that question, and each is a flag on it:
|
|
2443
|
+
|
|
2444
|
+
| Was | Now | |
|
|
2445
|
+
| ------------ | --------------------- | ----------------------------------------------------- |
|
|
2446
|
+
| `rules` | `explain --list` | the rule catalogue, with each rule's measured FP rate |
|
|
2447
|
+
| `why` | `explain <file:line>` | already an arm — see below |
|
|
2448
|
+
| `cross-file` | `explain --callers` | who calls this symbol |
|
|
2449
|
+
| `handover` | `explain --plan` | who owns what, and what to do next |
|
|
2450
|
+
| `triage` | `explain --evidence` | runtime evidence from a real run |
|
|
2451
|
+
|
|
2452
|
+
**`why` never moved.** `explain` has delegated the `file:line` subject form to
|
|
2453
|
+
that command since before the collapse, so retiring the verb removed a second
|
|
2454
|
+
name for a capability `explain` already had. The cheapest MOVE in the set and
|
|
2455
|
+
the one most likely to be got wrong: `mjolnir why file.ts:12` is now a usage
|
|
2456
|
+
error, and the fix is a different word in the same position.
|
|
2457
|
+
|
|
2458
|
+
`--playwright` is the internal arm (the plan says so), and it is marked as such
|
|
2459
|
+
in the table. `pw-report` summarised a Playwright run for a person reading CI
|
|
2460
|
+
output; the same information arrives through `--evidence` once that arm reads a
|
|
2461
|
+
report rather than one runner's format, and one flag that means "the run
|
|
2462
|
+
summary" beats two that mean it for one of them.
|
|
2463
|
+
|
|
2464
|
+
The arms table has the same shape as `CI_SUBCOMMANDS` and for the same reason:
|
|
2465
|
+
it is the single source the dispatcher reads **and** the help renders from.
|
|
2466
|
+
Arms are dispatched **before** the subject is read, because `--list` and
|
|
2467
|
+
`--plan` take no subject and the subject check is what `mjolnar explain --plan`
|
|
2468
|
+
would otherwise have hit.
|
|
2469
|
+
|
|
2470
|
+
**One residue worth naming.** `cli-handlers.ts` carried
|
|
2471
|
+
`import {} from "./forensics/triage.js";` — an empty import left behind when the
|
|
2472
|
+
`triage` VERB was retired, because every specifier in that statement became
|
|
2473
|
+
unused and the statement survived. Legal TypeScript, imports nothing, and it
|
|
2474
|
+
points at the file a maintainer would go looking in for where triage lives.
|
|
2475
|
+
|
|
2476
|
+
The generator of the 101 rule-doc footers was fixed before the files were, so
|
|
2477
|
+
`mjolnir rules --md` did not come back on the next `npm run generate-rule-docs`.
|
|
2478
|
+
The 22 PRESERVED retired-rule pages still name it, correctly: they record what
|
|
2479
|
+
a rule said while it existed, and the gate now skips a rule doc whose ID is not
|
|
2480
|
+
in the live registry — a fact read from the registry, not a pattern exemption.
|
|
2481
|
+
|
|
2482
|
+
### Carve 1.6 (MOVE arm, group 1) — five verbs became `ci <subcommand>`
|
|
2483
|
+
|
|
2484
|
+
`ci` is the right home for all five, and the word is not incidental: each one is
|
|
2485
|
+
a check a pipeline runs, and `ci` is already the verb whose meaning is "the
|
|
2486
|
+
thing a pipeline does".
|
|
2487
|
+
|
|
2488
|
+
| Was | Now | |
|
|
2489
|
+
| ----------------- | -------------------- | ------------------------------------------------------- |
|
|
2490
|
+
| `ci-adapter` | `ci adapters` | which ecosystems can be analysed, and how well |
|
|
2491
|
+
| `ci-integrity` | `ci integrity` | the generated workflow is itself correct |
|
|
2492
|
+
| `verify` | `ci verify` | run the blocking check and the suppression policy |
|
|
2493
|
+
| `release-trust` | `ci release-trust` | the signed measurement release's trust record |
|
|
2494
|
+
| `trust-trend` | `ci release-trend` | how that record has moved across releases |
|
|
2495
|
+
| `contract-verify` | `ci verify` (merged) | its two checks are the judgement `verify` already makes |
|
|
2496
|
+
|
|
2497
|
+
`integrity` and `verify` are the sharpest case in the whole collapse: as
|
|
2498
|
+
top-level verbs, `mjolnir integrity` and `mjolnir verify` read as two
|
|
2499
|
+
products that happened to share a surface nobody could see from the command
|
|
2500
|
+
list. `contract-verify` is recorded as a **merge with its destination named**,
|
|
2501
|
+
not a rename — "the checks went somewhere" has to be checkable.
|
|
2502
|
+
|
|
2503
|
+
**The table is the claim.** `CI_SUBCOMMANDS` in `src/cli.ts` is one object, the
|
|
2504
|
+
dispatcher reads it by name, and `mjolnir ci --help` renders from it — so a
|
|
2505
|
+
subcommand that exists is documented and one that is documented exists. A
|
|
2506
|
+
dispatcher with five inline `if` arms plus a separate help string is how the two
|
|
2507
|
+
drift apart, and this repository has that shape somewhere in every other
|
|
2508
|
+
subsystem. An unknown subcommand names the known ones and exits 10.
|
|
2509
|
+
|
|
2510
|
+
Two routing bugs the collapse fixed on the way: `ci --help` rendered the ROOT
|
|
2511
|
+
help, and `ci verify --help` rendered **`ci install`'s** help, because the
|
|
2512
|
+
`--help` rule hardcoded `["ci", "install"]`. Both are now derived from the
|
|
2513
|
+
request rather than from a constant.
|
|
2514
|
+
|
|
2515
|
+
### Carve 1.6 (REPLACE arm) — four verbs became flags, and two could not
|
|
2516
|
+
|
|
2517
|
+
`scan` gained three `--format` values and three modes:
|
|
2518
|
+
|
|
2519
|
+
| Retired verb | Replacement | What the flag is |
|
|
2520
|
+
| ------------------ | ------------------------------ | ---------------------------------------------------------------------------------------------- |
|
|
2521
|
+
| `trust-report` | `scan --format trust-report` | the terminal render, spelled out |
|
|
2522
|
+
| `pr-comment` | `scan --format pr-comment` | the same renderer plus the stored-baseline diff, so a comment can still say RESOLVED |
|
|
2523
|
+
| `summary` | `scan --format github-summary` | named after its CONSUMER, because `summary` was ambiguous with the terminal's own summary band |
|
|
2524
|
+
| `suppressions` | `scan --suppressions` | the ledger, now reading the TARGET rather than the cwd |
|
|
2525
|
+
| `suppression-gate` | `scan --suppression-gate` | **delegates** to the command; the flag lands first |
|
|
2526
|
+
| `policy` | `scan --policy` | the SCORING table, printed from the constants |
|
|
2527
|
+
|
|
2528
|
+
`scan --policy` is the new file here: `src/scorer/scoring-policy.ts` imports
|
|
2529
|
+
`NORMALIZATION_K`, `SMOOTHING_C`, both ceilings and the deduction-mass bands
|
|
2530
|
+
and renders them, so the table cannot drift from the code that scored the run.
|
|
2531
|
+
The `NORMALIZATION_K` line says **declared, not fitted** in the table itself,
|
|
2532
|
+
because printing a bare `5` among measured numbers reads as a measurement, and
|
|
2533
|
+
it is not one.
|
|
2534
|
+
|
|
2535
|
+
**Two of the plan's REPLACE rows were not honest replacements, and are recorded
|
|
2536
|
+
as pending rather than done.** `stats → scan --json` would replace a milestone
|
|
2537
|
+
ledger with a scan result — a different document wearing the same name. `rules →
|
|
2538
|
+
explain --list` and `why → explain <file:line>` name arms that do not exist
|
|
2539
|
+
yet. And `policy` keeps its verb, because `mjolnir policy init` and
|
|
2540
|
+
`validate` manage `mjolnir.policy.json`, a file that gates CI; folding a
|
|
2541
|
+
subcommand tree into a flag would lose both. Each row in
|
|
2542
|
+
`docs/cli-contract.json` now says what is missing, because "the plan says so" is
|
|
2543
|
+
not a disposition.
|
|
2544
|
+
|
|
2545
|
+
`--suppression-gate` delegates rather than reimplementing: a governance rule
|
|
2546
|
+
that exists twice can disagree with itself, and the flag and the command are
|
|
2547
|
+
the same judgement, so the flag synthesises argv and calls it.
|
|
2548
|
+
|
|
2549
|
+
### Carve 1.6 slice 2 — `mutation`, `forensics` and `triage`, and a directory that had to stay
|
|
2550
|
+
|
|
2551
|
+
The `mutation` verb is gone with `src/mutation/` (501 lines): reading Stryker
|
|
2552
|
+
and Mutmut reports is reading another tool's opinion of the code, and the
|
|
2553
|
+
directory had no importer but the verb's own runner. Two fuzz harnesses are
|
|
2554
|
+
narrowed rather than deleted, and the loss of mutation-format fuzz coverage is
|
|
2555
|
+
recorded here rather than left in a file that was about to be deleted.
|
|
2556
|
+
|
|
2557
|
+
**`src/forensics/` stays.** The plan makes `forensics` a DELETE, and the verb
|
|
2558
|
+
is deleted — but the import graph says the directory is not dead:
|
|
2559
|
+
`evidence-hygiene.ts` is imported by `commands/trust-report.ts` and
|
|
2560
|
+
`integrations/github/evidence-sanitization.ts`, the path the GitHub Action's PR
|
|
2561
|
+
comment goes through, and `triage.ts` is imported by `mcp/server.ts` under a
|
|
2562
|
+
parity contract. Deleting 2,600 lines by name would have broken the PR commenter
|
|
2563
|
+
and the MCP server, and the diff would have been green. That is the plan's own
|
|
2564
|
+
"names are not evidence" applied to a directory the plan did not enumerate.
|
|
2565
|
+
|
|
2566
|
+
`triage` is a MOVE to `explain --evidence` and the flag lands with the next
|
|
2567
|
+
slice, so the contract records `targetStatus: PENDING` rather than claiming the
|
|
2568
|
+
capability has arrived. The verb goes now so nothing can depend on it; the
|
|
2569
|
+
capability arrives when the flag does.
|
|
2570
|
+
|
|
2571
|
+
**The generated output was naming the dead tool at users.** `FLAKY.md` and
|
|
2572
|
+
`TRIAGE.md` stamp themselves "Generated by `mjolnir triage`" — a string
|
|
2573
|
+
written into a file a user commits to their repository, asserting a command
|
|
2574
|
+
ran. Both now name Mjölnir and the data source instead, which is both true and
|
|
2575
|
+
stable across the verb's next move.
|
|
2576
|
+
|
|
2577
|
+
One source-layer miss worth recording: the rule `fix` strings that told users
|
|
2578
|
+
to run the retired verbs live in `src/rules/`, and I edited the _generated_
|
|
2579
|
+
`docs/rules/*.md` first. `npm run generate-rule-docs` would have put them
|
|
2580
|
+
straight back — the same class of bug as the hardcoded `@v3` in the
|
|
2581
|
+
translation sync, one level up. The rules are the source; the docs, the site
|
|
2582
|
+
pages and the demo report are all downstream of them, and changing a generated
|
|
2583
|
+
file without changing its generator is a decision that survives exactly until
|
|
2584
|
+
the next regeneration.
|
|
2585
|
+
|
|
2586
|
+
Two artifacts turned out to be hand-maintained with no writer:
|
|
2587
|
+
`assets/readme/demo-report.json` (four readers, no generator) and
|
|
2588
|
+
`packages/playwright-reporter/` (unpublished, 0.1.0, its whole description
|
|
2589
|
+
about the three retired verbs). Both are noted rather than quietly patched; the
|
|
2590
|
+
reporter is a Phase 7 decision, since its output format is still what the
|
|
2591
|
+
evidence path will read.
|
|
2592
|
+
|
|
2593
|
+
### Carve 1.6 (DELETE arm) — ten verbs removed, and every place that named one
|
|
2594
|
+
|
|
2595
|
+
`badge` · `debt` · `impact` · `trend` · `exec-report` · `dashboard` ·
|
|
2596
|
+
`baseline` · `diff` · `create-rule` · `init` are gone: the command module, the
|
|
2597
|
+
`cli.ts` `VERBS` row, the `argv[0] === "…"` literal that
|
|
2598
|
+
`tests/contract/readme-commands.spec.ts` reads, the import and re-export lists,
|
|
2599
|
+
the `cli-handlers.ts` runner, the help-catalogue entry, and 117 tests across 27
|
|
2600
|
+
spec files — four of which were deleted outright because the removed verb was
|
|
2601
|
+
their entire subject.
|
|
2602
|
+
|
|
2603
|
+
**A verb's disposition is about the verb, not the module.** `baseline` is the
|
|
2604
|
+
case that would have shipped a green build and a broken product: the module is
|
|
2605
|
+
imported by `stats.ts`, `verify.ts`, `mcp/server.ts` and
|
|
2606
|
+
`reporter/pr-report-shared.ts`, so deleting it with the verb would have broken
|
|
2607
|
+
the MCP server, the PR reporter and `verify`. The module survives as a shared
|
|
2608
|
+
reader with a header that says so; the verb does not, because a stored
|
|
2609
|
+
"yesterday was fine" is the same trust-the-green-check failure the product
|
|
2610
|
+
exists to catch. That is the plan's own instruction — _names are not evidence_ —
|
|
2611
|
+
applied to an object it did not name.
|
|
2612
|
+
|
|
2613
|
+
**The suggestions the product printed were the bug nobody filed.** The
|
|
2614
|
+
reporter's `nextStep` strings are not documentation; they are the next thing
|
|
2615
|
+
printed into the terminal. `nextStep("mjolnir diff")` in a product whose `diff`
|
|
2616
|
+
was retired in the same commit is a command that does not exist, in the one
|
|
2617
|
+
place a user is guaranteed to be looking. `check-cli-contract` treats `src/`
|
|
2618
|
+
as a live surface for exactly this reason, and it is why `src/` was swept too
|
|
2619
|
+
rather than left to a later documentation pass.
|
|
2620
|
+
|
|
2621
|
+
**The contract is now a record of removals, not only of presences.**
|
|
2622
|
+
`docs/cli-contract.json` gained `retiredVerbs`, and the gate learned to check
|
|
2623
|
+
it: a `DELETE` row for a verb that has left `cli-command-names.ts` must appear
|
|
2624
|
+
in `retiredVerbs` with the commit that made it, and a `retiredVerbs` entry for
|
|
2625
|
+
a verb that is still registered fails too. Ten verbs retired; the gate now
|
|
2626
|
+
fails any README, site page, workflow, Action or source string that names one.
|
|
2627
|
+
|
|
2628
|
+
Follow-through, all of it found by gates rather than by looking: three coverage
|
|
2629
|
+
exemptions whose subject is gone, three `vitest.config.ts` coverage exclusions,
|
|
2630
|
+
two degradation exemptions, the dashboard-template assertion (which asserted
|
|
2631
|
+
"exactly one generator" and became vacuous when the generator was deleted, so
|
|
2632
|
+
it now asserts the invariant that still has teeth — that two renderers cannot
|
|
2633
|
+
both claim a band), and the demo asset, regenerated.
|
|
2634
|
+
|
|
2635
|
+
`forensics` stays. The plan makes it a DELETE too, and it will be — its
|
|
2636
|
+
subsystem and the `triage` relocation into `explain --evidence` are the next
|
|
2637
|
+
slice. Its row says KEEP until they land, because a contract that asserts a
|
|
2638
|
+
deletion the binary does not honour is worse than one that is merely out of
|
|
2639
|
+
date.
|
|
2640
|
+
|
|
2641
|
+
### Carve 1.5 — 14,986 lines of unwired modules, and one the plan got wrong
|
|
2642
|
+
|
|
2643
|
+
Nineteen modules with zero importers, all nineteen already on the committed
|
|
2644
|
+
orphan list, all nineteen with a dedicated spec file that imported nothing but
|
|
2645
|
+
them. A module with a test and no caller is the shape that makes dead code look
|
|
2646
|
+
alive: `tests/` counted as coverage of a file the product never reached.
|
|
2647
|
+
|
|
2648
|
+
Deleted: `evidence-artifacts` · `finalize-scan-result` · `pipeline-stages` ·
|
|
2649
|
+
`semantic-model-api` · `runtime-evidence-graph` (1,857 lines) ·
|
|
2650
|
+
`runtime-static-correlation` · `coverage-ingestion` · `evidence-enforcement` ·
|
|
2651
|
+
`reporter/sarif-compliance` · `m38-challenge-contract` ·
|
|
2652
|
+
`m43-system-of-systems` · `m44-historical-intelligence` (2,247) ·
|
|
2653
|
+
`m49-experience-parity-contract` · `bench/m48-scale-operating-model` ·
|
|
2654
|
+
`governance/m33-m34-contract` (2,745) · `qa/domain-model` ·
|
|
2655
|
+
`frameworks/universal-pack-contract` · `plugins/sdk-contract` ·
|
|
2656
|
+
`mutation/failure-sensitivity` — and their twenty specs. Nineteen stale
|
|
2657
|
+
allowlist rows went with them, because a row describing a file that no longer
|
|
2658
|
+
exists is the same lie as a row describing a file something now imports.
|
|
2659
|
+
|
|
2660
|
+
**The plan listed a twentieth as a true orphan, and it is not one.**
|
|
2661
|
+
`src/engine/evidence-graph.ts` has two importers and is shipped-reachable —
|
|
2662
|
+
`src/commands/milestone.ts` is one of them, which is the command that serves
|
|
2663
|
+
the `evidence-graph` verb. Deleting it would have removed the only way to reach
|
|
2664
|
+
a query surface the §3 table does not even mention. It stays, and
|
|
2665
|
+
`docs/cli-contract.json` now records the import-graph evidence rather than the
|
|
2666
|
+
plan's assertion.
|
|
2667
|
+
|
|
2668
|
+
The check that caught it is worth naming, because the first version of it was
|
|
2669
|
+
wrong: it resolved the plan's bare filenames (`evidence-graph.ts`) against
|
|
2670
|
+
`src/`, where they do not live, so the import graph returned an empty importer
|
|
2671
|
+
list for all twenty and every one of them looked like a safe deletion. The
|
|
2672
|
+
files are under `src/engine/`, `src/bench/`, `src/governance/`, `src/qa/`,
|
|
2673
|
+
`src/frameworks/`, `src/plugins/` and `src/mutation/`. A graph lookup that
|
|
2674
|
+
answers "no importers" for a path that does not exist is not evidence of
|
|
2675
|
+
anything, and the plan's own instruction — _names are not evidence_ — applies
|
|
2676
|
+
to the plan's file list exactly as it applies to the tree.
|
|
2677
|
+
|
|
2678
|
+
### Carve 1.3 — the brand self-audit survives, and its header stopped lying
|
|
2679
|
+
|
|
2680
|
+
`npm run brand:doctor` is a nine-rule self-audit over a six-node graph, and
|
|
2681
|
+
it still passes 9 of 9 with an empty `KNOWN_OPEN` ratchet. The tokens stay:
|
|
2682
|
+
`src/brand/tokens.ts` is load-bearing for the reporter, the site theme, the
|
|
2683
|
+
README generators and the mark hashes, and `assets/brand/tokens.json` is
|
|
2684
|
+
byte-locked against it.
|
|
2685
|
+
|
|
2686
|
+
What changed is the gate's own description of the tree. It said the badge
|
|
2687
|
+
drift appeared "in all 23 READMEs" and referred to a `site-doctor` check by a
|
|
2688
|
+
name that no longer exists. "Every README" is now one file plus five generated
|
|
2689
|
+
bands, and the rule did not weaken to reach that number — the twenty-two
|
|
2690
|
+
translations were never a distinct edge, they were copies of the same badges,
|
|
2691
|
+
and the copying is what made the drift expensive.
|
|
2692
|
+
|
|
2693
|
+
Site gate after the step: `gen`, `test`, `doctor` (0 of 8 failing), `build`,
|
|
2694
|
+
with no generated drift.
|
|
2695
|
+
|
|
2696
|
+
### Carve 1.2 — the 86-second tour drew a command the product will not have
|
|
2697
|
+
|
|
2698
|
+
`assets/video/script.tour.json` rendered `mjolnir forensics
|
|
2699
|
+
./test-results/`, and `forensics` is a DELETE verb in the v6 CLI collapse
|
|
2700
|
+
(plan §3). A committed render script is evidence: it is what the renderer
|
|
2701
|
+
is allowed to draw, and it was drawing a command that will not exist.
|
|
2702
|
+
|
|
2703
|
+
Removed: `captureTourScript()`, the `tour` script id, `script.tour.json`,
|
|
2704
|
+
the `tour` arm in three contract specs, the `both|demotour` choice on the
|
|
2705
|
+
`demo-video` workflow, and the tour rows in the video README. Order mattered —
|
|
2706
|
+
this runs before the CLI collapse so the three specs that re-run the capture
|
|
2707
|
+
and fail on drift stay green instead of going red on a verb the carve was
|
|
2708
|
+
about to remove.
|
|
2709
|
+
|
|
2710
|
+
The hero stays. Its story — a false-green CI gate found, fixed, re-proved —
|
|
2711
|
+
is the whole positioning, and it uses only verbs the collapse keeps.
|
|
2712
|
+
|
|
2713
|
+
One behaviour change while the file was open: rendering no longer overwrites
|
|
2714
|
+
the committed MP4. `publishDemo()` used to run unconditionally after any
|
|
2715
|
+
render, so a manual workflow dispatch replaced two binaries the README links
|
|
2716
|
+
to without asking. Publishing is now an explicit `--publish`, which
|
|
2717
|
+
`npm run docs:video:render` passes, so regenerating the shipped asset is
|
|
2718
|
+
unchanged for a human and safe by default for a workflow.
|
|
2719
|
+
|
|
2720
|
+
### The Action major tag every reader was told to pin did not exist
|
|
2721
|
+
|
|
2722
|
+
`action.yml` documented `uses: Sergey-Bar/Mjolnir@v3` while the published stable
|
|
2723
|
+
was **5.0.0** — and `scripts/readme-release-status.mjs` did not merely inherit
|
|
2724
|
+
that, it re-wrote it. Every translation sync ran
|
|
2725
|
+
|
|
2726
|
+
```js
|
|
2727
|
+
replaceAll("Sergey-Bar/Mjolnir@v1", "Sergey-Bar/Mjolnir@v3");
|
|
2728
|
+
```
|
|
2729
|
+
|
|
2730
|
+
so twenty-two translated READMEs instructed every non-English reader to pin a
|
|
2731
|
+
tag the repository stopped moving at 3.x, and the next sync put it back. A
|
|
2732
|
+
hardcoded major is a syntactically valid ref; it fails for the reader as a 404
|
|
2733
|
+
and for nobody in this repository.
|
|
2734
|
+
|
|
2735
|
+
The sync now derives the major from the released version, and
|
|
2736
|
+
`tests/contract/census-drift.spec.ts` sweeps every live surface for a
|
|
2737
|
+
`Sergey-Bar/Mjolnir@vN` that is not the published major, with the same
|
|
2738
|
+
dormant-shape and match-count discipline the census sweep already uses.
|
|
2739
|
+
|
|
2740
|
+
### The carve manifest: 28 files, and the one that mattered
|
|
2741
|
+
|
|
2742
|
+
Phase 1 removes ~10,000 lines across twenty areas. The cheapest way for that to
|
|
2743
|
+
go wrong is for a reviewer to trust a folder name, and the plan's own evidence
|
|
2744
|
+
says not to: `src/ledger/` reads like bookkeeping and is 2,856 lines of M26
|
|
2745
|
+
validators that four scripts import. So `docs/CARVE-MANIFEST.json` records, per
|
|
2746
|
+
file, who imports it, whether it ships, which specs name it, and how big it is —
|
|
2747
|
+
generated facts, hand-written dispositions that survive regeneration, and a gate
|
|
2748
|
+
(`npm run check-carve-manifest`) that fails when the two disagree.
|
|
2749
|
+
|
|
2750
|
+
All 28 files start at KEEP. Nine have no importer; every one of them was already
|
|
2751
|
+
on the disclosed orphan list, so the manifest found no new dead code — it found
|
|
2752
|
+
the size of the code that a directory name would have hidden.
|
|
2753
|
+
|
|
2754
|
+
The gate also fails on the case a directory-based review cannot see: a file that
|
|
2755
|
+
left the tree without a DELETE disposition. That is the difference between a
|
|
2756
|
+
reviewed removal and an accident, and it is one field.
|
|
2757
|
+
|
|
2758
|
+
### Claim integrity — the gates that could not fail
|
|
2759
|
+
|
|
2760
|
+
Every item here is a check that was running, reporting, and could not have
|
|
2761
|
+
failed. The pattern is one defect wearing a dozen hats: a value that looked
|
|
2762
|
+
load-bearing and held nothing.
|
|
2763
|
+
|
|
2764
|
+
- **`coverage:ratchet` measured a truncated run.** It read
|
|
2765
|
+
`summary.total` and never counted files, so a two-file run of a 292-file
|
|
2766
|
+
tree passed at 97.92% — and a partial run reports _higher_ percentages than
|
|
2767
|
+
a complete one, because the files that survive are the easy ones. It now
|
|
2768
|
+
asserts a file-count ratio and a non-trivial denominator, both fail-closed.
|
|
2769
|
+
- **`nForZeroFp` said 1.** The expression `z²·p / (p·(1−p)) − z²` simplifies to
|
|
2770
|
+
`z²·p/(1−p)` ≈ 0.427, and `Math.ceil` turned that into 1 — so every
|
|
2771
|
+
straddling rule read "About 1 clean sample would settle it". The correct
|
|
2772
|
+
value at the 10% ceiling is **35**.
|
|
2773
|
+
- **`docs/rules/` and the registry both claimed the core tier, differently.**
|
|
2774
|
+
`effectiveTier` now resolves it everywhere, including dedup survivor
|
|
2775
|
+
resolution, where an undeclared rule defaulted to `core` and outranked every
|
|
2776
|
+
rule that had declared `extended`.
|
|
2777
|
+
- **The capability registry resolved evidence once for the whole registry.**
|
|
2778
|
+
One measured rule anywhere made `hasMeasurement` true for every capability,
|
|
2779
|
+
and `ci.provider.azure-pipelines` was advertised on an unrelated Java rule's
|
|
2780
|
+
measurement. Evidence is resolved per entry now.
|
|
2781
|
+
- **`OVER_CLAIMED_MATURITY` compared a value with itself.** `proven` and
|
|
2782
|
+
`maturity` were assigned from one call, so the gate could not fire. They now
|
|
2783
|
+
come from two resolvers — the census declares, the registry proves — and a
|
|
2784
|
+
provider whose evidence beats the ledger is visible as such.
|
|
2785
|
+
- **`STALE` was unreachable.** The FP-audit generator filtered stale
|
|
2786
|
+
measurements out of `MEASURED_FP`, so `hasStaleMeasurement` was
|
|
2787
|
+
structurally always `false` and `QA-PY-004` (42 hand-classified verdicts,
|
|
2788
|
+
one detector revision behind) reported as never sampled. `MEASURED_FP_RAW`
|
|
2789
|
+
keeps them and both consumers read it.
|
|
2790
|
+
- **The claim registry had no recency budget.** `observedAt: null` satisfied
|
|
2791
|
+
its presence check, so a 2026-06 observation read like yesterday's.
|
|
2792
|
+
Per-type budgets now apply, the digest check runs against a real status
|
|
2793
|
+
instead of a value absent from the enum, and every `BLOCKED` claim carries a
|
|
2794
|
+
`blockedReason` and a review date.
|
|
2795
|
+
- **The tier ratchet was vacuous on an empty tier.** `MAX_UNMEASURED_CORE = 0`
|
|
2796
|
+
was satisfied by a set with no members, so the first core promotion would
|
|
2797
|
+
have arrived with the check already green. A rule is now accounted for by ≥10
|
|
2798
|
+
classified verdicts **or** by an unexpired `corePromotion` naming an owner.
|
|
2799
|
+
- **The anti-creep law had two escapes.** The exception marker was searched
|
|
2800
|
+
for across the whole append-only changelog, so one marker in any past
|
|
2801
|
+
release disabled the ratchet for good — and with an empty tier and a zero
|
|
2802
|
+
baseline, the first promotion would have switched the law off permanently.
|
|
2803
|
+
The comparison is also now against `previousBaselineCore`, so lowering the
|
|
2804
|
+
baseline to match a grown tier no longer hides the growth.
|
|
2805
|
+
- **The census could call an ecosystem `SUPPORTED` with a fixture
|
|
2806
|
+
directory as its only evidence.** `SUPPORTED` now requires an adapter **and**
|
|
2807
|
+
a verified four-leg quad, which demoted 33 entries — the honest result, and
|
|
2808
|
+
the reason 19 rules were demoted in 6.0 for the same reason.
|
|
2809
|
+
- **Two hardcoded-`false` quad resolvers, in two files.** They could not fail
|
|
2810
|
+
and the census's directory-size proxy answered a different question from the
|
|
2811
|
+
registry's `false`. Both now read one filesystem probe.
|
|
2812
|
+
- **Seven gates ran only locally.** `qa-ir:parity` and `check-unimported-modules` were
|
|
2813
|
+
on a nightly cron, so a PR that broke either passed CI and failed on the
|
|
2814
|
+
maintainer's machine. The other seven are new here. All nine are in
|
|
2815
|
+
`gates:claim-integrity`, which `ci.yml` runs as one step — named per gate
|
|
2816
|
+
inside the chain so a failure identifies itself — and declared in all three
|
|
2817
|
+
`gates/*.json` tiers.
|
|
2818
|
+
|
|
2819
|
+
### New gates
|
|
2820
|
+
|
|
2821
|
+
`npm run gates:claim-integrity` chains the nine claim-integrity checks so a
|
|
2822
|
+
failure names its gate instead of a step number. Also new: `workflow:scripts`,
|
|
2823
|
+
`config:consumers`, `scripts:reachable`, `translations:ratchet`,
|
|
2824
|
+
`docs:provenance-drift`, `check-fixture-quad`, `rules:promotion:check`,
|
|
2825
|
+
`check-changelog`.
|
|
2826
|
+
|
|
2827
|
+
### Deliberate omissions
|
|
2828
|
+
|
|
2829
|
+
- **The 4-leg fixture backfill (148 legs) is not in this change.** The plan
|
|
2830
|
+
requires the quad to be _executed_, not merely present; writing 148 legs
|
|
2831
|
+
no detector had run against would have made `check-fixture-quad` green by
|
|
2832
|
+
construction. `npm run check-fixture-quad` prints the work list — 53 rules
|
|
2833
|
+
have `MUST-FIRE`, 40 `MUST-NOT-FIRE`, 75 `RECALL`, **0 `PRECISION`**, and 0
|
|
2834
|
+
of 79 have all four — so the backfill is driven by data rather than memory.
|
|
2835
|
+
`PRECISION` is zero because it demands a classified `TN` on a negative
|
|
2836
|
+
fixture, and no fixture has been classified as silent.
|
|
2837
|
+
- **`gitlab-ci` is written but not registered.** The adapter exists and three
|
|
2838
|
+
detection classes are already enumerated in `src/adapters/gitlab-ci.ts`, but
|
|
2839
|
+
it has no rules, so registering it would trade one false claim for another.
|
|
2840
|
+
Admitting it means writing the three rules with fixtures, and it is gated by
|
|
2841
|
+
the same quad as everything else.
|
|
2842
|
+
|
|
2843
|
+
### PR reports say what to do next (#698)
|
|
2844
|
+
|
|
2845
|
+
`src/reporter/pr-report-actions.ts` turns a PR comment from a finding list into
|
|
2846
|
+
a handover: each band names the action it implies, so a reader who disagrees
|
|
2847
|
+
with a verdict can see which command contradicts it without reading the source.
|
|
2848
|
+
Six score badges (`assets/brand/pr-status/*.svg`) are generated from the brand
|
|
2849
|
+
tokens by `npm run brand:pr-status` and drift-locked by
|
|
2850
|
+
`tests/reporters/pr-report-actions.spec.ts`, which fails if a badge's stroke
|
|
2851
|
+
stops matching the token it came from. `brand:pr-status` is a deliberate brand
|
|
2852
|
+
change, so it is declared manual in `docs/MANUAL-SCRIPTS.md` rather than wired
|
|
2853
|
+
into a gate.
|
|
2854
|
+
|
|
2855
|
+
### The website reading stack, and the palette it arrived with (#696)
|
|
2856
|
+
|
|
2857
|
+
A quieter, cooler set of greys and a teal→violet accent rule for the landing and
|
|
2858
|
+
guide surfaces, plus `docs/AI-DISCOVERY.md`, `site/public/llms.txt`, the guided
|
|
2859
|
+
proof walkthrough, and vendored Nordic and Cascadia fonts.
|
|
2860
|
+
|
|
2861
|
+
The palette arrived as raw hex inside the theme stylesheets, which is exactly
|
|
2862
|
+
what `brand-doctor` rule 6 exists to catch: a second palette no gate could see.
|
|
2863
|
+
It is now nine tokens in `BRAND` — `readingAccent`, `readingAccentAlt`,
|
|
2864
|
+
`readingFocus`, `readingAccentBright`, `readingAccentInk`, `readingUnworthy`,
|
|
2865
|
+
`readingMuted`, `readingDim`, `readingLede` — emitted as `--mj-reading-*` and
|
|
2866
|
+
referenced by the stylesheets that use them. Putting them in `BRAND` rather than
|
|
2867
|
+
a group of their own is deliberate: rule 8 computes WCAG AA for every foreground
|
|
2868
|
+
in `BRAND` against every surface, so the nine are held to the same floor as the
|
|
2869
|
+
terminal palette rather than escaping it. The weakest pairing is
|
|
2870
|
+
`readingAccentAlt` on `soft` at 5.75:1.
|
|
2871
|
+
|
|
2872
|
+
These are scoped to the reading surfaces. `steelDim` and `heroMuted` keep their
|
|
2873
|
+
values for the terminal, the README artwork and every other surface — the POC
|
|
2874
|
+
never claimed otherwise.
|
|
2875
|
+
|
|
2876
|
+
### Known open, deliberately
|
|
2877
|
+
|
|
2878
|
+
**Publishing a package is not certifying it.** 5.1.0 is on npm; that is a
|
|
2879
|
+
statement about distribution, not about evidence. The candidate manifest remains
|
|
2880
|
+
`WORKING_CANDIDATE` / `NOT_CERTIFIED` / `NOT_AUTHORIZED`, and nothing in this
|
|
2881
|
+
release moves those states.
|
|
2882
|
+
|
|
2883
|
+
Four gates report `BLOCKED` on `npm run release:verify` and did not gate this
|
|
2884
|
+
publish:
|
|
2885
|
+
|
|
2886
|
+
- **`m26:audit`** — no consented external execution and no holdout proof.
|
|
2887
|
+
- **`candidate:readiness`** — no real-world, platform-matrix or
|
|
2888
|
+
consumer-install evidence.
|
|
2889
|
+
- **`candidate:decision:release`** — the same, adjudicated as a release-stage
|
|
2890
|
+
decision rather than an engineering one.
|
|
2891
|
+
- **`docs:roadmap:check`** — 69 support-matrix cells are explicitly `BLOCKED`
|
|
2892
|
+
and `docs/M26-EXTERNAL-VALIDATION.json` records external validation as
|
|
2893
|
+
`BLOCKED`, not `COMPLETE`. This is the same missing evidence read through
|
|
2894
|
+
the roadmap ledger, and it is a reporting gate: `stable-release.yml` does not
|
|
2895
|
+
run it. It has read this way since the v6 carve — no commit since then has
|
|
2896
|
+
touched `docs/M26-SUPPORT-MATRIX.json`, `docs/M26-EXTERNAL-VALIDATION.json`
|
|
2897
|
+
or `docs/M26-GAP-LEDGER.jsonl`.
|
|
2898
|
+
|
|
2899
|
+
The first three still run and still print on every publish; they stopped blocking
|
|
2900
|
+
only because a person recorded `vars.STABLE_RELEASE_AUTHORIZED=true`
|
|
2901
|
+
(`stable-release.yml`, `Candidate evidence`). `npm run ci-local` passes, because
|
|
2902
|
+
`candidate:decision:engineering` clears every engineering blocker.
|
|
2903
|
+
|
|
2904
|
+
This is the designed state for 5.x, not a workaround. The alternative — editing
|
|
2905
|
+
the manifest to claim `REMOTE_PROVEN`, `CERTIFIED` or `AUTHORIZED` — is the
|
|
2906
|
+
false-green this project exists to prevent.
|
|
2907
|
+
|
|
2908
|
+
## [5.0.0] — 2026-09-27
|
|
2909
|
+
|
|
14
2910
|
### Breaking changes
|
|
15
2911
|
|
|
16
2912
|
- **`business-case` no longer prints a dollar figure you did not supply.** The
|
|
@@ -47,6 +2943,18 @@ once shipped, so this file is the record of what changed between versions.
|
|
|
47
2943
|
The generation timestamp moved out of the visible body into
|
|
48
2944
|
`<meta name="mjolnir-generated-at">`, and `--deterministic` omits it entirely
|
|
49
2945
|
so the artifact is byte-identical and can be diffed in review.
|
|
2946
|
+
- **`mjolnir report` no longer publishes a test-run status.** The top-level
|
|
2947
|
+
`status` was computed from the findings
|
|
2948
|
+
(`partial ? "interrupted" : hasError ? "failed" : "passed"`), so a clean scan
|
|
2949
|
+
wrote `{status: "passed", totalTests: 0}` — the first field a Playwright
|
|
2950
|
+
consumer reads, asserting a green run over zero executed tests, with the
|
|
2951
|
+
contradicting `execution: "STATIC_ANALYSIS"` marker buried in the extension
|
|
2952
|
+
block. `failed` was equally dishonest: it implies tests ran and lost. The
|
|
2953
|
+
field is now the literal `interrupted` on every path, and what the scan
|
|
2954
|
+
actually found is reported separately as
|
|
2955
|
+
`mjolnir.scanOutcome` (`clean` / `blocked` / `partial`), which the previous
|
|
2956
|
+
`mjolnir.status` field carried. Consumers reading the root `status` will see
|
|
2957
|
+
`passed` → `interrupted`; that is the correction, not a regression.
|
|
50
2958
|
- **Score bands are decided in one place.** The dashboard split at 60 while the
|
|
51
2959
|
terminal split at 80, so a score the terminal called UNWORTHY rendered amber
|
|
52
2960
|
in HTML; the contributor handover used 90; `mermaid.ts` and
|
|
@@ -55,6 +2963,14 @@ once shipped, so this file is the record of what changed between versions.
|
|
|
55
2963
|
|
|
56
2964
|
### Added
|
|
57
2965
|
|
|
2966
|
+
- **`npm run ci:standard` now runs in CI.** `scripts/check-ci-standard.mjs`
|
|
2967
|
+
was unit-tested by `tests/integrations/ci-standard.spec.ts` but referenced
|
|
2968
|
+
by no workflow, so the invariants it checks — the PR scan's partial-analysis
|
|
2969
|
+
handling, the coverage and ratchet gates, and the read-only scan job's
|
|
2970
|
+
permissions — were enforced nowhere. It runs as a step in `workflow-lint`,
|
|
2971
|
+
next to actionlint, since both are static checks of the repo's own CI
|
|
2972
|
+
config.
|
|
2973
|
+
|
|
58
2974
|
- **Opt-in Sentry crash reporting.** Set `SENTRY_DSN` and the CLI and the MCP
|
|
59
2975
|
stdio server report their own fatal errors — release-tagged as
|
|
60
2976
|
`mjolnir-qa@<version>`, tagged by surface, and nothing else: no user data
|
|
@@ -67,7 +2983,11 @@ once shipped, so this file is the record of what changed between versions.
|
|
|
67
2983
|
the two existing top-level catch blocks rather than `uncaughtException` /
|
|
68
2984
|
`unhandledRejection` handlers, which would have changed the frozen exit-code
|
|
69
2985
|
contract. `npm run sentry:release` creates the release and uploads source maps
|
|
70
|
-
(the build now emits them; the published tarball still excludes them).
|
|
2986
|
+
(the build now emits them; the published tarball still excludes them). Both
|
|
2987
|
+
release pipelines now run it after `npm run build` and before `npm pack`,
|
|
2988
|
+
because the debug ids it stamps are written into `dist/` in place: pack first
|
|
2989
|
+
and the shipped tarball carries no ids, so the maps uploaded for the release
|
|
2990
|
+
match nothing and every stack trace degrades to a bundle frame.
|
|
71
2991
|
- `npm run report:honesty` — no surface may print a zero for a measurement that
|
|
72
2992
|
may be absent. Four reviewed exceptions are recorded for computation inputs,
|
|
73
2993
|
each with a reason and a follow-up.
|
|
@@ -93,8 +3013,229 @@ once shipped, so this file is the record of what changed between versions.
|
|
|
93
3013
|
or descriptor to print. It opens no file, touches no clock and formats no
|
|
94
3014
|
document, and a test enforces that.
|
|
95
3015
|
|
|
3016
|
+
### Changed
|
|
3017
|
+
|
|
3018
|
+
- **Ruleset `01` now enforces 11 required checks instead of 4.** It required
|
|
3019
|
+
only the four `build-test` matrix contexts, which meant the repo's entire
|
|
3020
|
+
review surface — self-scan, certification, generated-docs-drift, site-build,
|
|
3021
|
+
workflow-lint, property, fuzz — could be red while a PR still merged. That
|
|
3022
|
+
made every one of those checks advice rather than a gate, and it meant the
|
|
3023
|
+
deleted `merge-verify.yml` header was describing a ruleset that never
|
|
3024
|
+
existed. The enforced set is now the four matrix entries plus
|
|
3025
|
+
`property-tests`, `fuzz`, `site-build`, `workflow-lint`, `self-scan`,
|
|
3026
|
+
`certification` and `generated-docs-drift` — every check that can actually
|
|
3027
|
+
block. Deliberately still _not_ required: `detector-revision-diff` (advisory
|
|
3028
|
+
by design — a hard fail would only manufacture routine revision bumps),
|
|
3029
|
+
`scan` / `publish` (the Mjölnir PR report, which reviewers read but which
|
|
3030
|
+
duplicates `self-scan` on the `src` tree), and the CodeQL/OSV workflows,
|
|
3031
|
+
which report but do not gate. `strict_required_status_checks_policy` and
|
|
3032
|
+
`do_not_enforce_on_create` are preserved, so a PR must still be current with
|
|
3033
|
+
`main` to merge.
|
|
3034
|
+
- **`docs/BRANCH-PROTECTION.md` now states the enforced set instead of an
|
|
3035
|
+
aspiration.** It listed eight required checks; three of them were enforced.
|
|
3036
|
+
Its verification command also read `branches/main/protection`, which does not
|
|
3037
|
+
carry required checks at all on this repo — they live in ruleset `01` — so it
|
|
3038
|
+
could not have worked. Both are corrected.
|
|
3039
|
+
|
|
3040
|
+
### Removed
|
|
3041
|
+
|
|
3042
|
+
- **The Snyk webhook is gone.** Snyk posted a `security/snyk (sergey-bar)`
|
|
3043
|
+
commit status on every push, duplicating the first-party `osv-scanner.yml`
|
|
3044
|
+
(`scan-pr` / `scan-scheduled`) and the Dependabot security updates that are
|
|
3045
|
+
already configured. Repo webhooks are now empty. Restore with Snyk's own
|
|
3046
|
+
integration settings if it is ever wanted back — the deleted hook pointed at
|
|
3047
|
+
`https://api.snyk.io/webhook/github/4ac3f376-3f24-42f7-a2c4-8830700f671d`.
|
|
3048
|
+
- **The `merge-verify` workflow is gone.** It re-ran the entire PR gate
|
|
3049
|
+
(build, typecheck, lint, coverage, property, fuzz, site, self-scan, docs
|
|
3050
|
+
regen) on every push to `main` — roughly doubling CI cost. Its stated
|
|
3051
|
+
justification was that "a green PR that merges into a red main is not a
|
|
3052
|
+
green PR", but that failure class is already closed by ruleset `01` running
|
|
3053
|
+
`strict_required_status_checks_policy`, which refuses to merge a PR that is
|
|
3054
|
+
not current with `main`. The file also asserted that ruleset `01` "requires
|
|
3055
|
+
the complete PR matrix, property/fuzz, site, workflow-lint, self-scan,
|
|
3056
|
+
certification, detector-revision, and generated-docs checks before merge" —
|
|
3057
|
+
it did not, and never has. The ruleset requires only the four `build-test`
|
|
3058
|
+
matrix contexts. Both the workflow and the claim are removed rather than left
|
|
3059
|
+
to mislead a reviewer about what blocks a merge.
|
|
3060
|
+
- **The `bench-advisory` CI job is gone.** It spent a full `npm ci` plus a
|
|
3061
|
+
3 000-file benchmark run on every PR to produce a comparison that
|
|
3062
|
+
`continue-on-error: true` then discarded, and whose only output was a
|
|
3063
|
+
warning nobody could gate on. `npm run bench:scan`, the pinned
|
|
3064
|
+
`.mjolnir/bench-baseline.json`, and their schema/comparison tests in
|
|
3065
|
+
`tests/adversarial/bench-gate.spec.ts` are untouched — the benchmark is
|
|
3066
|
+
still runnable and still tested, it just no longer costs every PR a runner.
|
|
3067
|
+
- **Four copies of the soft job time-budget report.** Each of `build-test`,
|
|
3068
|
+
`self-scan`, `generated-docs-drift` and `bench-advisory` re-derived its own
|
|
3069
|
+
wall-clock minutes into `$GITHUB_STEP_SUMMARY` and printed a warning that,
|
|
3070
|
+
by its own comment, "never gates". The jobs already declare
|
|
3071
|
+
`timeout-minutes`, which is the hard bound, and GitHub reports job duration
|
|
3072
|
+
natively.
|
|
3073
|
+
|
|
96
3074
|
### Fixed
|
|
97
3075
|
|
|
3076
|
+
- **Order in the report no longer depends on the machine that produced it.** 23
|
|
3077
|
+
sites in `src/` sorted with a bare `localeCompare`, which resolves against the
|
|
3078
|
+
process's default locale. `scan-cache.ts` was the worst: it sorted INSIDE
|
|
3079
|
+
`hashDir`, so the order was fed to `hash.update(entry.name)` and became a
|
|
3080
|
+
semantic input to the detector fingerprint that `isIncrementalSafe` keys off
|
|
3081
|
+
— one repository, two fingerprints, one per locale. All 23 now use
|
|
3082
|
+
`compareCodePoints` from `src/lib/compare.ts`, except two whose key is a
|
|
3083
|
+
human-readable label, which use a comparator pinned to `en` rather than
|
|
3084
|
+
inheriting the environment. `tests/contract/deterministic-ordering.spec.ts`
|
|
3085
|
+
fails the build on any unpinned `localeCompare` in `src/`.
|
|
3086
|
+
- **`mjolnir bind` can no longer be hijacked by a repository's own `git`.**
|
|
3087
|
+
`src/engine/candidate-binding.ts` — called on every scan — ran
|
|
3088
|
+
`execFileSync("git", …)` with a bare name. On Windows `CreateProcess` searches
|
|
3089
|
+
the current directory before `PATH`, so a committed `git.exe` in an untrusted
|
|
3090
|
+
repo chose the `commit` and `tree` that land in `runIdentity` and the evidence
|
|
3091
|
+
graph. It is the last bare-name process launch left in `src/`, and
|
|
3092
|
+
`tests/contract/no-bare-process-exec.spec.ts` now audits the whole tree.
|
|
3093
|
+
- **The coverage exemption ledger can no longer drift silently.** Each
|
|
3094
|
+
non-structural entry in `docs/COVERAGE-EXEMPTIONS.json` now carries
|
|
3095
|
+
`defectSignatures` — regexes that must still match its file — or an explicit
|
|
3096
|
+
`closureState` when the defect it described is gone. A signature that stops
|
|
3097
|
+
matching fails the gate, so a code fix forces a ledger edit in the same
|
|
3098
|
+
commit. Six entries were already describing defects the current code had
|
|
3099
|
+
fixed (`maturity`, `business-case`, `dashboard`, `enterprise`, `exec-report`
|
|
3100
|
+
and the `report-playwright` row), and are reclassified accordingly.
|
|
3101
|
+
- **Skipped and expected-fail tests now have a price.** Nothing in the
|
|
3102
|
+
repository counted them, so the count could grow from zero to hundreds with
|
|
3103
|
+
every other signal still green. `docs/SKIP-BUDGET.json` records the current
|
|
3104
|
+
counts — 8 skipped tests, 1 skipped file, 3 expected failures — measured by
|
|
3105
|
+
`scripts/vitest-skip-budget-reporter.mjs` during the run. They may fall;
|
|
3106
|
+
raising them fails `npm run skip:budget` and names the tests that spent the
|
|
3107
|
+
budget. A missing or incomplete measurement fails too, so a reporter that
|
|
3108
|
+
stops being wired in cannot read as a clean run.
|
|
3109
|
+
- **The M26 release gate and the version-drift checks are covered where it
|
|
3110
|
+
matters.** Both run inside `certify` and both were excluded from the coverage
|
|
3111
|
+
ratchet, so their reject arms — the paths that only execute when something is
|
|
3112
|
+
wrong — had no reviewer pressure on them. `tests/ledger/m26-validator-reject-arms.spec.ts`
|
|
3113
|
+
is table-driven over every required field, so a field that is added without a
|
|
3114
|
+
test fails; `tests/release/version-surface-drift-arms.spec.ts` covers the
|
|
3115
|
+
fail-fast arms of the version synchronizer.
|
|
3116
|
+
- **One error derivation, one `isRecord`, one crash-isolation path.** The tree
|
|
3117
|
+
carried `isRecord` eight times (seven byte-identical), error rendering four
|
|
3118
|
+
times under two names, and crash isolation seven times — and the copies
|
|
3119
|
+
disagreed, which is the only reason any of this mattered. `String(obj)`
|
|
3120
|
+
renders `[object Object]`, so the two _thinner_ error helpers turned a real
|
|
3121
|
+
Node failure shape like `throw { code: "EISDIR" }` into a message naming
|
|
3122
|
+
nothing — in a doctor detail, an MCP tool result and a `summary` line
|
|
3123
|
+
respectively. The richer body survived the collapse, and
|
|
3124
|
+
`isRecord` — the predicate every JSON reader uses to decide "this is an
|
|
3125
|
+
object I may index into", which is a question a hostile saved report gets
|
|
3126
|
+
to ask — is now the exported one in `src/lib/safe-json.ts`. The four
|
|
3127
|
+
adapters' duplicated `try { rule.run } catch { onCrash }` is now
|
|
3128
|
+
`runRuleIsolated`, because that block is what the `rulesCrashed` count is
|
|
3129
|
+
built on and seven copies mean seven chances to lose it.
|
|
3130
|
+
The per-adapter _filter chain_ and budget check were deliberately **not**
|
|
3131
|
+
folded into the existing `runRulesShared`: the TypeScript adapter injects
|
|
3132
|
+
an AST, framework tags, a lazy `codeText` getter, config-only gating and
|
|
3133
|
+
`configGateMatches`, and it _aborts the whole file_ on budget exhaustion
|
|
3134
|
+
where `runRulesShared` only skips one rule. Routing every adapter through
|
|
3135
|
+
it as written would have silently changed which rules run — the exact
|
|
3136
|
+
unmeasured claim this program exists to delete.
|
|
3137
|
+
`tests/contract/single-error-helper.spec.ts` holds the counts at one, and
|
|
3138
|
+
pins the object arm so a future "simplification" cannot drop it.
|
|
3139
|
+
|
|
3140
|
+
- **A crashed rule is no longer certified as a rule that stayed silent.**
|
|
3141
|
+
The doc generator caught a rule THROW while running it against its own
|
|
3142
|
+
must-not-fire fixture and returned `null`; the caller read that as
|
|
3143
|
+
`fired: false` and committed, into a generated doc page and the
|
|
3144
|
+
certification surface, the sentence _"Verified against … — a legitimate,
|
|
3145
|
+
similar-looking pattern this rule correctly leaves alone."_ A detector that
|
|
3146
|
+
crashes produces exactly the same absence of findings as one that correctly
|
|
3147
|
+
abstains, so that claim had no evidence behind it — and it contradicted the
|
|
3148
|
+
same function's own docstring, which says the field degrades honestly.
|
|
3149
|
+
The outcome is now the tri-state the doctor's own model already defines:
|
|
3150
|
+
`FIRED` (a real firewall violation), `DID_NOT_FIRE` (certified), and
|
|
3151
|
+
`INCONCLUSIVE` (the rule did not run). An `INCONCLUSIVE` page says so and
|
|
3152
|
+
says why, and it never renders the certification sentence. A missing
|
|
3153
|
+
fixture is a fourth, separate thing — nothing was attempted — and says
|
|
3154
|
+
that instead. The field is typed as a `FixtureOutcome` union rather than a
|
|
3155
|
+
boolean, so reintroducing a derived `fired` flag does not typecheck.
|
|
3156
|
+
|
|
3157
|
+
- **A human-facing table stopped sorting by code unit.** Pinning every
|
|
3158
|
+
`localeCompare` to code-unit order (the fix for the ambient-locale drift)
|
|
3159
|
+
made the per-rule corpus-occurrence table read
|
|
3160
|
+
`SeleniumHQ-selenium` before `microsoft-playwright-dotnet`, because `S`
|
|
3161
|
+
sorts before `m`. Stable, and worse to read than the alphabetical order the
|
|
3162
|
+
table implies it has. That surface is a table a person chooses repos from,
|
|
3163
|
+
so it now uses an **explicitly pinned** locale — readable order, identical
|
|
3164
|
+
on every machine, which is the actual defect being avoided.
|
|
3165
|
+
|
|
3166
|
+
- **Fifteen silent capability losses are now counted, and the count reaches the
|
|
3167
|
+
report.** A `catch` in a detection path that returned a clean default made
|
|
3168
|
+
the scan exit 0, report `analysisComplete`, and quietly lose the thing that
|
|
3169
|
+
made its verdict trustworthy — the reader had no way to tell a whole scan
|
|
3170
|
+
from a narrowed one. The worst case was `getCodeOnlyText` returning raw text
|
|
3171
|
+
after a throw, which switches OFF the comment/string false-positive firewall
|
|
3172
|
+
for that file rather than merely slowing it down. Every such site now records
|
|
3173
|
+
a reason in one append-only ledger (`src/engine/degradation-ledger.ts`),
|
|
3174
|
+
following the precedent already set by `parserRetryDegradationCount`, and the
|
|
3175
|
+
scan result carries `analysisStatus.degradations` — reason-coded counts,
|
|
3176
|
+
present only when something was actually lost, alongside a
|
|
3177
|
+
`degraded:<reason>:<count>` entry in `reasons`. A non-empty set makes the
|
|
3178
|
+
scan `partial`. `truncationReasons` is untouched: a degradation is not
|
|
3179
|
+
truncation, and a reader must be able to tell "the scan stopped" from "the
|
|
3180
|
+
scan lost a layer".
|
|
3181
|
+
- Two distinctions the ledger refuses to blur. **Absent is not degraded:** a
|
|
3182
|
+
Python, Java, Go or Rust repository has no `package.json`, and recording
|
|
3183
|
+
that would mark every non-Node project partial while training readers to
|
|
3184
|
+
ignore the field — so only a manifest that exists and will not parse
|
|
3185
|
+
counts. **A no-AST adapter is not a downgrade either:** the same line
|
|
3186
|
+
W1.4 draws for the deadline fallback.
|
|
3187
|
+
- Two sites outside the original list were found by the new contract spec
|
|
3188
|
+
and fixed in the same commit: the tree-sitter Java/C#/Python parses (left
|
|
3189
|
+
alone, because the pipeline already counts every `parseAst → undefined` as
|
|
3190
|
+
one parse fallback, and counting again would tally one file twice in two
|
|
3191
|
+
different fields) and `hashDir`'s `readdirSync`, whose early `return`
|
|
3192
|
+
omitted a whole subtree from the detector fingerprint — two rule trees
|
|
3193
|
+
differing only inside an unreadable directory then hash identically, which
|
|
3194
|
+
is a stale-cache hit wearing a fresh one's clothes.
|
|
3195
|
+
- `mjolnir trend show` no longer prints "No trend data yet." and exits clean
|
|
3196
|
+
for a history file whose every line failed to parse. The count of
|
|
3197
|
+
discarded lines is now reported on stderr, and a history that exists but
|
|
3198
|
+
cannot be read is `EXIT_INTERNAL` — the difference between no history and
|
|
3199
|
+
unreadable history, which both used to arrive as the same empty array.
|
|
3200
|
+
- `mjolnir explain` distinguishes _the rule found nothing_ from _the rule
|
|
3201
|
+
never ran_. A rule that THREW on its own fixture produced the same output
|
|
3202
|
+
as a correctly silent one, in the one surface whose purpose is to show
|
|
3203
|
+
that a rule works. It now says so, with the reason.
|
|
3204
|
+
- `tests/contract/no-uncounted-degradation.spec.ts` is the durable half: a
|
|
3205
|
+
`catch` in the detection path that RETURNS A VALUE must record a reason,
|
|
3206
|
+
rethrow, route to a counted sink, or be exempted with a stated failure
|
|
3207
|
+
direction. The rule is about the _return_, not the `catch` — 37 of the 70
|
|
3208
|
+
detection-path catches return nothing and cannot lie to a caller, and a
|
|
3209
|
+
spec that listed all 70 as individual exemptions would be the same as no
|
|
3210
|
+
exemption list at all.
|
|
3211
|
+
|
|
3212
|
+
- **Scanning a subdirectory no longer reports a framework it read from its
|
|
3213
|
+
parent.** `runScan` walks up past the target to find the project root, and
|
|
3214
|
+
re-anchors the scan root to the target when the target is a strict
|
|
3215
|
+
descendant — but it kept the discovered root's parsed `package.json`. So
|
|
3216
|
+
`mjolnir scan monorepo/packages/foo` reported `jest` because
|
|
3217
|
+
`monorepo/package.json` said so: a read outside the explicit scan root, and
|
|
3218
|
+
a framework verdict about a tree nobody pointed at. Framework detection now
|
|
3219
|
+
works from in-root evidence only (config files and the anchor's own
|
|
3220
|
+
manifest), and reports `frameworkDetectionUnknown` when that evidence is
|
|
3221
|
+
absent — an absent verdict you can see, rather than a parent's you cannot.
|
|
3222
|
+
Scanning the project root itself is unchanged. This is the CLI counterpart
|
|
3223
|
+
to the containment property the MCP workspace-boundary contract already
|
|
3224
|
+
enforced.
|
|
3225
|
+
- **A scan that ran out of time no longer loses its AST stage silently.** When a
|
|
3226
|
+
scan passes `--max-duration`, the remaining files were analyzed by regex
|
|
3227
|
+
instead of AST — a different detection capability, not a slower version of the
|
|
3228
|
+
same one, so the same tree produced different findings and a different score.
|
|
3229
|
+
The mode was faithfully recorded in the scan cache and reported nowhere; the
|
|
3230
|
+
only nearby disclosure, `file-budget`, fired incidentally because the
|
|
3231
|
+
per-file rule-loop deadline was usually already spent too. Such a scan now
|
|
3232
|
+
carries the truncation reason `ast-budget-fallback` and an
|
|
3233
|
+
`analysisStatus.reasons` entry `ast-budget-fallback-files:<n>` saying how many
|
|
3234
|
+
files were analyzed by regex, and is honestly `partial`. An adapter that
|
|
3235
|
+
declares no AST stage at all (a YAML workflow, for instance) is a declared
|
|
3236
|
+
capability envelope and is NOT reported as truncation. `truncationReasons` is
|
|
3237
|
+
an open `string[]` by design, so consumers should test for members rather than
|
|
3238
|
+
assert the exact array.
|
|
98
3239
|
- **`frontier:contracts` was reporting 19 suites it never ran.** It named 24
|
|
99
3240
|
test files; nineteen had been deleted by the `cc5fcb88` cleanup and its
|
|
100
3241
|
follow-up. Vitest treats a missing path as "no tests here" rather than an
|
|
@@ -285,6 +3426,27 @@ were live in 3.0.0 and neither was in the plan.
|
|
|
285
3426
|
|
|
286
3427
|
### Known open, deliberately
|
|
287
3428
|
|
|
3429
|
+
- [ ] **Trust certification is not claimed, and this release was authorized
|
|
3430
|
+
without it.** `candidate:readiness` reports `BLOCKED` /
|
|
3431
|
+
`INCONCLUSIVE` on all ten of its checks — candidate identity is not a
|
|
3432
|
+
release candidate, the candidate SHA is not authorized, engineering
|
|
3433
|
+
certification is not complete, and external validation, protected holdout
|
|
3434
|
+
proof, real-world repository proof, the platform matrix, consumer install
|
|
3435
|
+
proof and remote workflow proof are all absent.
|
|
3436
|
+
`m26:audit` is likewise unsatisfied. Both ran, both printed, and neither
|
|
3437
|
+
blocked the publish, because `vars.STABLE_RELEASE_AUTHORIZED` is `true` —
|
|
3438
|
+
the same explicit override 4.0.0 shipped under.
|
|
3439
|
+
The distinction is the point: **publication of a package is not the same
|
|
3440
|
+
claim as a certification of it.** 5.0.0 asserts that the reporting
|
|
3441
|
+
surface is honest, that a silent capability loss is disclosed, and that no
|
|
3442
|
+
verdict is published without a measurement behind it. It does not assert
|
|
3443
|
+
that a holdout corpus, a platform matrix and an external validation run
|
|
3444
|
+
have been completed, because none of them have.
|
|
3445
|
+
Consumers are not asked to take that on trust: the npm package publishes
|
|
3446
|
+
the candidate manifest, which reads `NOT_CERTIFIED` and
|
|
3447
|
+
`NOT_AUTHORIZED`, and every one of those ten blockers is enumerated above
|
|
3448
|
+
rather than summarised. Closing them is the work that would make a
|
|
3449
|
+
certification claim possible; it is not work this release did.
|
|
288
3450
|
- [ ] `CERTIFICATION_STATES` is not in ladder order, so `rankOf` — the only
|
|
289
3451
|
ordering function — ranks `KNOWN` above `TRUST-COMPLETE`, and
|
|
290
3452
|
`requiresEvidence` asks `BLOCKED` and `DEGRADED` for a corpus their own
|
|
@@ -427,9 +3589,9 @@ This release is software-complete and locally certified for the declared enginee
|
|
|
427
3589
|
|
|
428
3590
|
- Added seven milestone commands to the public CLI and root help:
|
|
429
3591
|
- `mjolnir ci-integrity` validates GitHub/GitLab/Jenkins scan gates and reports non-blocking or disabled candidates.
|
|
430
|
-
- `mjolnir
|
|
3592
|
+
- `mjolnir doctor --frameworks` reports bounded F0–F5 maturity while preserving human calibration authority.
|
|
431
3593
|
- `mjolnir suppression-gate` evaluates reasons, expiry, allowlists, total counts, and matched-finding mass suppression.
|
|
432
|
-
- `mjolnir
|
|
3594
|
+
- `mjolnir explain --callers` reports duplicate test names, shared imports, circular dependencies, and amplified findings.
|
|
433
3595
|
- `mjolnir contract-verify` validates machine-contract fields and binds persisted contracts to a fresh scan.
|
|
434
3596
|
- `mjolnir trust-trend` persists, deduplicates, and compares trust snapshots.
|
|
435
3597
|
- `mjolnir evidence-graph` builds provenance-bearing evidence graphs and supports file/rule queries.
|
|
@@ -1390,7 +4552,7 @@ Azure DevOps support: guarded azure-pipelines.yml parsing, the QA-CI Azure arms,
|
|
|
1390
4552
|
preserved byte-identically (differential preservation suite).
|
|
1391
4553
|
- **Explain v2 (WI-7).** `mjolnir explain` gains verdict mode
|
|
1392
4554
|
(`mjolnir explain verdict --json <mjolnir.json>`) and finding mode
|
|
1393
|
-
(file:line delegates to `mjolnir
|
|
4555
|
+
(file:line delegates to `mjolnir explain <file:line>`); every mode now answers the §8
|
|
1394
4556
|
checklist including WHAT WOULD CHANGE THE VERDICT and NEXT ACTION.
|
|
1395
4557
|
- **Triage v2 (WI-8).** `mjolnir triage` now runs the §9 guided workflow —
|
|
1396
4558
|
CLASSIFY → EVIDENCE → TRUST VERDICT → NEXT ACTION per row, every row
|
|
@@ -1591,7 +4753,7 @@ Azure DevOps support: guarded azure-pipelines.yml parsing, the QA-CI Azure arms,
|
|
|
1591
4753
|
|
|
1592
4754
|
### Added
|
|
1593
4755
|
|
|
1594
|
-
- **`mjolnir
|
|
4756
|
+
- **`mjolnir explain <file:line> <file>:<line>`** — occurrence-level evidence query
|
|
1595
4757
|
(informational, NOT a gate): exact file+line match, severity icon,
|
|
1596
4758
|
message/why/fix, evidence level, trust level, measured FP rate
|
|
1597
4759
|
(or the honest "ships on assumption"), runtime corroboration when
|
|
@@ -2001,7 +5163,7 @@ rules… → Scoring…`) fed by the new additive `ScanHooks.onProgress`.
|
|
|
2001
5163
|
unified filter excludes them from non-strict scans exactly like core
|
|
2002
5164
|
(post-scan cap still observable under `--strict`; the cli-scan-arms
|
|
2003
5165
|
test updated to cover both sides).
|
|
2004
|
-
- **Drift-checked:** `mjolnir
|
|
5166
|
+
- **Drift-checked:** `mjolnir explain --list --md --external` renders the
|
|
2005
5167
|
catalog from the LOADED external rules with a provenance column
|
|
2006
5168
|
(`core`/`external`) — an on-disk edit changes the next render; the
|
|
2007
5169
|
catalog can never drift from what actually ships.
|
|
@@ -2560,7 +5722,7 @@ SEMANTIC | FRAMEWORK | RUNTIME`) instead of free text: `src/rules/rule.ts`
|
|
|
2560
5722
|
|
|
2561
5723
|
### Added — UNSURE adjudication gate + QA-PW-101 measured (plan §11.5)
|
|
2562
5724
|
|
|
2563
|
-
- **UNSURE ceiling ratchet:** `npm run fp-audit
|
|
5725
|
+
- **UNSURE ceiling ratchet:** `npm run generate-fp-audit-table` now fails when the
|
|
2564
5726
|
UNSURE backlog grows beyond the committed
|
|
2565
5727
|
`tests/corpus/verdicts/unsure-ceiling.json` (the §11.5 mechanism: UNSURE
|
|
2566
5728
|
never counts into `n` but always triggers review). The ceiling only moves
|
|
@@ -2629,7 +5791,7 @@ detectorRevision, FP ≤ 10%, n ≥ 10`), on any detectorRevision mismatch
|
|
|
2629
5791
|
- **`mjolnir doctor`**: `MAX_UNMEASURED_CORE` lowered 40 → **0** (Phase 1
|
|
2630
5792
|
exit gate: 0 unmeasured in effective core, now enforced); tier checks
|
|
2631
5793
|
consume `effectiveTier` + stale-measurement logic. Display surfaces
|
|
2632
|
-
(`mjolnir explain`, `mjolnir
|
|
5794
|
+
(`mjolnir explain`, `mjolnir explain --list`, generated rule docs, capability
|
|
2633
5795
|
matrix) render the PROVISIONAL status honestly.
|
|
2634
5796
|
|
|
2635
5797
|
### Verification Trust Evolution, Phase 0 + Phase 1 prep
|
|
@@ -2659,7 +5821,7 @@ detectorRevision, FP ≤ 10%, n ≥ 10`), on any detectorRevision mismatch
|
|
|
2659
5821
|
- **`tests/corpus/detector-revisions.json` sidecar**: hand-maintained,
|
|
2660
5822
|
one entry per measured rule (all at revision 1 today), diffable.
|
|
2661
5823
|
`MEASURED_FP` entries now carry `detectorRevision` stamped from the
|
|
2662
|
-
sidecar by `fp-audit
|
|
5824
|
+
sidecar by `generate-fp-audit-table`; `docs/FP-AUDIT.md` gains a
|
|
2663
5825
|
`detectorRev` column. Measurement inheritance law (§07): a measurement
|
|
2664
5826
|
belongs to a specific detector implementation, not merely to a rule
|
|
2665
5827
|
ID. Drift lock extended in `tests/measured-fp-generated.spec.ts`
|
|
@@ -2888,21 +6050,21 @@ detectorRevision, FP ≤ 10%, n ≥ 10`), on any detectorRevision mismatch
|
|
|
2888
6050
|
code; that fact previously lived only in `docs/FP-AUDIT.md` and
|
|
2889
6051
|
`mjolnir doctor`. Now surfaced everywhere a user looks:
|
|
2890
6052
|
- The scan footer reports how many of the rules that _fired_ are measured.
|
|
2891
|
-
- `mjolnir
|
|
6053
|
+
- `mjolnir explain --list --unmeasured` / `--measured` filter the catalog; a new
|
|
2892
6054
|
"FP (measured)" column in `rules --md`; a "Measured FP rate" row on every
|
|
2893
6055
|
`docs/rules/` page and in `mjolnir explain`.
|
|
2894
6056
|
- JSON findings carry `measuredFpRate` and `measuredFpN` (additive —
|
|
2895
6057
|
`schemaVersion` is still 1).
|
|
2896
6058
|
- `src/rules/measured-fp.generated.ts` bakes the rates into the shipped
|
|
2897
6059
|
package (the raw verdicts are not packed); regenerated by
|
|
2898
|
-
`npm run fp-audit
|
|
6060
|
+
`npm run generate-fp-audit-table`, drift-locked by a test, and now the single
|
|
2899
6061
|
source `mjolnir doctor` reads.
|
|
2900
6062
|
- Scoring is unchanged — this is visibility only.
|
|
2901
6063
|
- **Corpus expanded 6 → 13 repos** so the previously-silent rule families
|
|
2902
6064
|
(QA-TEST, QA-TQUAL, most QA-PW, QA-CI-001) fire on real consumer code:
|
|
2903
6065
|
added `next-auth`, `vite`, `sveltekit`, `astro`, `TanStack/query`,
|
|
2904
6066
|
`eslint-plugin-playwright`, `playwright-pytest`. `corpus:sample` and
|
|
2905
|
-
`corpus:
|
|
6067
|
+
`corpus:audit` now scan with `--strict` (quarantine rules were
|
|
2906
6068
|
invisible to both before). `docs/FP-AUDIT.md` is 15/91 (down from 19 —
|
|
2907
6069
|
see the dispatch fix below, which retired the leaked cross-language
|
|
2908
6070
|
verdicts for QA-PW-101/112 and QA-TEST-004/QA-ENV-001 on Java/Python
|
|
@@ -3103,7 +6265,7 @@ by a `must-not-fire` fixture so the class cannot return silently.
|
|
|
3103
6265
|
- Upgrade-Plan-v3 Phase 0.1: expanded the false-positive corpus with two
|
|
3104
6266
|
additional Python repos (`pytest-dev/pytest`, `psf/requests`) so all
|
|
3105
6267
|
QA-PY-001..012 rules are exercised against real code via
|
|
3106
|
-
`npm run corpus:
|
|
6268
|
+
`npm run corpus:audit`.
|
|
3107
6269
|
- Added `CHANGELOG.md` (this file) per Upgrade-Plan-v3 critical item #3:
|
|
3108
6270
|
user-visible rule behavior changes get a first-class entry from now on.
|
|
3109
6271
|
- Upgrade-Plan-v3 Phase 1: five new TypeScript/Playwright rules —
|
|
@@ -3148,17 +6310,17 @@ by a `must-not-fire` fixture so the class cannot return silently.
|
|
|
3148
6310
|
Promoted to core: QA-CS-101 (0% FP, n=20), QA-JV-105 (10% FP, n=20).
|
|
3149
6311
|
Quarantined rules still ship and are still documented — they are opt-in via
|
|
3150
6312
|
`--strict` rather than shaping the default report.
|
|
3151
|
-
- `mjolnir
|
|
6313
|
+
- `mjolnir explain --list` (`--json` and `--md`) now exposes each rule's `tier`, and
|
|
3152
6314
|
every generated page under `docs/rules/` shows it in the metadata table.
|
|
3153
6315
|
|
|
3154
6316
|
### Fixed — documentation claims a `grep` disproved
|
|
3155
6317
|
|
|
3156
6318
|
- Every generated rule page told the reader to reproduce corpus counts with
|
|
3157
|
-
a `corpus:audit` script. That script had been renamed to `corpus:
|
|
6319
|
+
a `corpus:audit` script. That script had been renamed to `corpus:audit` in
|
|
3158
6320
|
the Tempering plan and the generator string was never updated — the command
|
|
3159
6321
|
printed on 91 published pages did not exist. Same dead name in
|
|
3160
6322
|
`docs/PUBLISHING.md`
|
|
3161
|
-
(`corpus:audit:update` → `corpus:
|
|
6323
|
+
(`corpus:audit:update` → `corpus:audit --update`).
|
|
3162
6324
|
- **`docs/FP-AUDIT.md` under-reported the rule base as 84 rules when the
|
|
3163
6325
|
registry holds 91.** The coverage denominator was built by grepping source
|
|
3164
6326
|
for `id: "QA-…"`, which silently missed the seven rules that the Phase 6
|