mjolnir-qa 0.4.0 → 0.5.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -9,29 +9,1034 @@ Rule behavior changes (new rules, FP-rate changes against the corpus,
9
9
  severity changes) are first-class entries here — rule IDs are immutable
10
10
  once shipped, so this file is the record of what changed between versions.
11
11
 
12
- ## [0.4.0] — 2026-08-27
12
+ ## [0.5.2] — 2026-09-05
13
13
 
14
- ### Changed
14
+ ### npm 12 pack-shape repair of the release pipeline
15
15
 
16
- - **BREAKING: Rebranded from QA Doctor to Mjölnir.** Package name is now
17
- `mjolnir-qa` (bin: `mjolnir`). Config file: `mjolnir.config.json`.
18
- Data directory: `.mjolnir/`. Badge: `mjolnir-badge.json`.
19
- - Score label: "SCORE" → "WORTHINESS".
20
- - Verdicts: "HEALTHY" → "WORTHY", "CRITICAL" → "UNWORTHY".
21
- - Environment variable: `QA_DOCTOR_ASCII` → `MJOLNIR_ASCII`.
22
- - SARIF tool.driver.name: "Mjölnir".
23
- - Repository: `github.com/Sergey-Bar/Mjolnir`.
24
- - CLI: all help text, error messages, usage strings reference `mjolnir`.
25
- - Generated workflows: `mjolnir.yml`, `npx mjolnir-qa@latest`.
16
+ ### Fixed
26
17
 
27
- ### Added
18
+ - The fresh-install gate (`tests/integrations/registry-install.spec.ts`),
19
+ `tests/integrations/package-smoke.spec.ts`, and
20
+ `tests/e2e/journey-1-first-run.spec.ts` parse `npm pack --json` through
21
+ a shared shape-tolerant helper
22
+ (`tests/helpers/npm-pack-json.ts`): npm 12 changed the output from an
23
+ array to an object keyed by package name, which stopped the v0.5.1
24
+ publish at the gate — tag cut, nothing shipped, by design.
25
+ - `release.yml` upgrades to `npm@11` (the proven line) instead of
26
+ `npm@latest`: toolchain majors must be deliberate, verified changes,
27
+ never implicit drift on the publish path.
28
+ - `registry-install.spec.ts` cleanup no longer cascades a second error
29
+ when `beforeAll` fails early (the cascade buried the real diagnosis).
28
30
 
29
- - `--format mermaid` test-architecture diagram (Sprint 9).
30
- - `--tone blunt` — opt-in blunter messages (Sprint 9).
31
- - Milestones first flawless scan / first debt reduction announced once.
32
- - New MJÖLNIR ASCII art logo (minimal Nordic hammer).
31
+ ## [0.5.1]2026-09-05
32
+
33
+ ### Verification Trust Evolution, Phase 8 Local Extensibility (plan §18)
34
+
35
+ ### Added — folder-based external rules, zero network
36
+
37
+ - **`mjolnir-rules/` contract** (`src/plugins/local-rules.ts`): a
38
+ workspace directory loaded from the scan target root alongside npm
39
+ plugins. Two file kinds: **JSON rule manifests** (declarative regex
40
+ patterns — NO code executed; id/title/severity/category/appliesTo/
41
+ patterns/message/why/fix/languages/frameworks) and **JS modules**
42
+ (`rules: QADoctorRule[]`, full-Node trust, same posture as npm
43
+ plugins). Missing directory → no-op.
44
+ - **Same trust contract as core/npm plugins:** reserved core prefixes
45
+ rejected (case-insensitive spoofing guard); bad metadata/regexes
46
+ degrade to warning entries (QA-PLUGIN-000), never a crash; external
47
+ rules carry the full trust metadata shape and are born
48
+ quarantine/unmeasured. **Core-tier clamp**: an external rule
49
+ declaring `tier: "core"` is clamped to `extended` with a load
50
+ warning — core requires a measured FP rate from the committed corpus
51
+ sidecar, which external rules cannot have.
52
+ - **Tier caps obeyed, filter unified:** the quarantine exclusion filter
53
+ in `buildUniversalRules` now consults the tier map that includes
54
+ plugin/external tiers — plugin-declared quarantine rules were
55
+ previously excluded only when the core registry knew the ID; the
56
+ unified filter excludes them from non-strict scans exactly like core
57
+ (post-scan cap still observable under `--strict`; the cli-scan-arms
58
+ test updated to cover both sides).
59
+ - **Drift-checked:** `mjolnir rules --md --external` renders the
60
+ catalog from the LOADED external rules with a provenance column
61
+ (`core`/`external`) — an on-disk edit changes the next render; the
62
+ catalog can never drift from what actually ships.
63
+ `scripts/generate-capability-matrix.ts --external <root>` writes a
64
+ workspace-local `MJOLNIR-RULES-MATRIX.md` with provenance "external"
65
+ (unmeasured by definition — outside the corpus sidecar). The
66
+ committed matrix stays core-registry-only and byte-stable.
67
+ - **S-8 disclosure:** external rule surfaces appear in the scan's
68
+ plugin disclosure block.
69
+
70
+ Registry/marketplace explicitly deferred (plan §18).
71
+
72
+ ### Verification Trust Evolution, Phase 7 — Agentic QA Trust (plan §17)
73
+
74
+ ### Added — Agentic Trust Profile (plan §17.2, §17.4)
75
+
76
+ - **Provenance detection** (`src/engine/provenance.ts`): per-file static
77
+ markers, EXTRACTED not invented — the industry-wide generated-file
78
+ header convention ("auto-generated"/"generated by"/"do not edit") and
79
+ the Playwright codegen recorder fingerprint (its default test title
80
+ is exactly `'test'`). Honest boundary, documented in the module:
81
+ healed and MCP-edited files WITHOUT markers are statically
82
+ indistinguishable from hand-written ones — the classifier returns
83
+ `unmarked`, never a guess.
84
+ - **`ScanResult.agenticProfile`** (additive): share of test files with
85
+ detected generative markers + the findings split across those
86
+ surfaces, with the boundary in the profile's own `note`. Surfaced in
87
+ the terminal report only when something was actually detected
88
+ (silence over noise); JSON carries it on every scan.
89
+ - **§17.4 — same evidence standard regardless of author:** the profile
90
+ is metadata only. It never changes scoring, evidence levels, tier
91
+ caps, or rule behavior. AI tests earn trust through exactly the same
92
+ evidence as human ones.
93
+
94
+ ### Added — locator.normalize() framework-standards rules (plan §17.3)
95
+
96
+ Two new frozen `QA-PW-*` rules, BORN QUARANTINE (§17 exit gate:
97
+ measured before leaving provisional), fixtures both directions:
98
+
99
+ - **QA-PW-146 — CSS/XPath string selector instead of a normalized
100
+ locator** (warning, heuristic): `css=`/`xpath=` engine prefixes, bare
101
+ id/class/attr CSS, and `nth-child` chains inside
102
+ `.locator()`/`waitForSelector()`/`page.$` — aligned with Playwright's
103
+ locator standard. Fills the TS gap (the brittle-selectors family
104
+ covers Java/C#/Python; the retired QA-PW-112 was style police, this
105
+ is markup-coupling detection). Not auto-fixable: the normalized
106
+ getter depends on app semantics only a human knows — the fix field
107
+ carries the concrete `getByRole`/`getByTestId`/`getByText`
108
+ suggestion. Both fixtures + all three cached OSS corpora (0 fires —
109
+ real suites use normalized locators) recorded.
110
+ - **QA-PW-147 — codegen default test title** (info, observation): a
111
+ committed spec still titled `'test'`/`'test N'` is an unreviewed
112
+ recording artifact — the §17.1 provenance marker as a finding.
113
+
114
+ Registry: 99 rules (73 measured). Docs pages + capability matrix
115
+ regenerated.
116
+
117
+ ### Verification Trust Evolution, Phase 6 — Runtime Evidence (plan §16)
118
+
119
+ ### Added — runtime corroboration + the honest L0–L5 trust ladder
120
+
121
+ Built on the existing forensics ingestion (no greenfield):
122
+ `packages/playwright-reporter` → `mjolnir.report.json` → the
123
+ `ForensicsReport` pipeline that `forensics`/`triage`/`pw-report`
124
+ already consume. Findings gain two additive, optional fields
125
+ (schemaVersion 1 unchanged):
126
+
127
+ - **`runtimeCorroboration`** — what a real run report vouches for:
128
+ `level: "file" | "test" | "defect"`, the report source, executed-test
129
+ count, and (test/defect level) the containing test's full verdict
130
+ (attempts, final status, passed-on-retry, ever-failed, skipped).
131
+ - **`trustLevel`** — the L0–L5 ladder, derived deterministically:
132
+ L0 (E0 observation) / L1 (E1 heuristic) / L2 (E2 deterministic) are
133
+ the static-only ceiling; L3 (file executed) / L4 (the containing
134
+ test executed — matched via the report's spec declaration lines) /
135
+ L5 (the run verdict directly corroborates the defect class — a
136
+ FLAKY-RISK finding whose test actually flaked, retried, or timed
137
+ out) exist ONLY when runtime corroboration is present. The
138
+ no-static-only-L4/L5 invariant is structurally enforced in
139
+ `deriveTrustLevel` and locked by tests across the full
140
+ findingType×confidence×evidenceLevel matrix.
141
+
142
+ - **Matching is honest by construction:** Playwright JSON reports now
143
+ carry the spec declaration line (additive `line` on
144
+ `TestRecord`/`TestVerdict`); a finding is tied to a specific test
145
+ only when the report's declaration spans place it there — otherwise
146
+ corroboration stays at file level, and files the report never ran
147
+ get NOTHING (no fabricated evidence). JUnit XML has no locations:
148
+ file-level only.
149
+ - **Scan wiring:** `runScan` auto-discovers a run report next to the
150
+ scan target (`mjolnir.report.json` — the reporter package's default
151
+ output — or a `test-results/` directory), runs the existing
152
+ forensics ingestion, and stamps findings. No report → findings
153
+ unchanged (honest "runtime evidence: not available"). A hostile
154
+ report degrades the scan, never fails it.
155
+ - **Reporters split verified vs assumed (plan §16):** the terminal
156
+ footer reports `Runtime evidence: N/M findings corroborated by a
157
+ real run report (trust L3–L5)` or the explicit not-available line;
158
+ finding cards show `trust L4 · runtime: test executed`. SARIF
159
+ results carry `trustLevel` + `runtimeCorroboration` in properties.
160
+
161
+ ### Verification Trust Evolution, Phase 5 — Framework Expansion (plan §15, D7 closed)
162
+
163
+ ### Added — FrameworkDimension enforced (plan §15.1, defect D7 closed)
164
+
165
+ - **Real dependency parsing per build system:** `package.json` (JSON),
166
+ `pom.xml` (Maven `<dependency>` blocks), `build.gradle(.kts)`
167
+ (dependency-statement coordinates), and EVERY `.csproj` at the root
168
+ (`<PackageReference Include="…">` attributes — the old "first
169
+ `.csproj` only" defect is closed), plus requirements*.txt for Python.
170
+ - **Per-file framework tags** (`ParsedFile.frameworkTags`): derived from
171
+ the file's OWN imports/usings/import-lines — `@playwright/test` →
172
+ "playwright", `cypress` → "cypress", `org.junit.*` → "junit",
173
+ `NUnit` → "nunit", `import selenium` → "selenium", etc. AST-truth for
174
+ Java/C#/TS; import-line scan for Python (no AST seam by design).
175
+ - **`rule.frameworks ∩ file.frameworkTags` filtering** with
176
+ open-when-unknown: a rule that declares `frameworks` runs on a file
177
+ only when the file's tags intersect it; files without tags and rules
178
+ without `frameworks` are always analyzed — the dimension narrows, it
179
+ never silently drops evidence. Shared in
180
+ `src/engine/adapter.ts` (`frameworkFilterApplies`), enforced in the
181
+ TS/Java/C#/Python adapters' `runRules`.
182
+ - **Generalized config-gating (§15.2):** `configRule: true` +
183
+ `configFiles: string[]` (regex sources) replaces the hard-coded
184
+ `playwright.config.*` regex that lived in the TS adapter and
185
+ duplicated inside all five config rules (QA-PW-121/122/141/143/144
186
+ migrated). Discovery knows the config filename conventions
187
+ (`cypress.config.*` added alongside `playwright.config.*`).
188
+
189
+ ### Added — Cypress integration (first framework per the §15 order)
190
+
191
+ Three rules in the new frozen `QA-CYP-*` namespace, BORN QUARANTINE
192
+ (§15.5) with fixtures both directions (`tests/fixtures/QA-CYP-*/`):
193
+
194
+ - **QA-CYP-001 — fixed `cy.wait(n)`** (warning): numeric-literal
195
+ `cy.wait(3000)` — the Cypress hard-sleep idiom. Alias waits
196
+ (`cy.wait('@route')`) are the legitimate form and never fire.
197
+ File gate: framework tag, `.cy.*` extension, or `cy.*` API usage in
198
+ the file (real Cypress suites rarely import cypress — surfaced by the
199
+ first measurement against cypress-example-kitchensink). MEASURED on
200
+ the kitchensink corpus: 12 TP (viewport-switch fixed waits) / 3 FP
201
+ (doc-example artifacts, the suite intentionally demonstrates the API
202
+ — QA-PY-003 precedent) → 20% FP at n=15, 95% Wilson
203
+ [7.1%, 45.2%]; tier extended (band-consistent), corpus
204
+ cypress-io-kitchensink added with baselines.
205
+ cypress-realworld-app scanned as precision evidence: 0 fires (the
206
+ suite uses alias waits exclusively), baseline recorded.
207
+ - **QA-CYP-002 — focused test (`.only`)** (error): committed
208
+ `it.only`/`describe.only`/`context.only` de-schedules the rest of the
209
+ suite. Quarantine, unmeasured (fixtures only) pending a measured
210
+ Cypress corpus with the pattern.
211
+ - **QA-CYP-003 — `chromeWebSecurity: false`** (error): deterministic
212
+ config defect via the generalized `configFiles` gate. Quarantine,
213
+ unmeasured pending corpus.
214
+
215
+ ### Added — Selenium cross-language reach (plan §15.3, JV/CS/Py reuse)
216
+
217
+ Three rules in the new frozen `QA-SE-*` namespace sharing one sequence
218
+ detector (hard sleep followed by an element lookup within 3 lines —
219
+ the sleep standing in for an explicit `WebDriverWait`), all BORN
220
+ QUARANTINE:
221
+
222
+ - **QA-SE-001** (Java): `Thread.sleep` → `findElement`/`click`/
223
+ `sendKeys` within 3 lines.
224
+ - **QA-SE-002** (C#): `Thread.Sleep`/`Task.Delay` →
225
+ `FindElement`/`Click`/`SendKeys` within 3 lines. MEASURED on the
226
+ SeleniumHQ/selenium .NET webdriver suite: 3 TP + 1 FP (a
227
+ sleep-inside-polling-loop cadence — loop-body containment is the
228
+ documented residue) → 25% FP at n=4; below the n≥10 measurement bar,
229
+ stays quarantine/unmeasured with verdicts recorded
230
+ (tests/corpus/verdicts/SeleniumHQ-selenium.jsonl, baseline added).
231
+ - **QA-SE-003** (Python): `time.sleep` → `find_element`/`click`/
232
+ `send_keys` within 3 lines.
233
+
234
+ ### Registry
235
+
236
+ - 97 rules (was 91); registry/doctor/scaffolder ID validators widened
237
+ for the new frozen namespaces (QA-CYP-_, QA-SE-_, QA-WDIO-_,
238
+ QA-PPTR-_, QA-APM-* reserved). Measured coverage 73/97 (75%); the
239
+ §20.1(a) unmeasured-count ratchet now tracks the PRE-EXISTING set
240
+ mechanically (`introduced` ≤ 0.5.0) — new waves onboard under §20.1(b)
241
+ - the per-framework exit gate, not the global count freeze.
242
+ - Defect ledger: **D7 closed** (target phase 5). Capability matrix
243
+ regenerated (97 rules); docs pages regenerated; corpus baselines
244
+ added for the three Phase 5 measurement repos.
245
+
246
+ ### Verification Trust Evolution, Phase 4 — Common QA Semantic Model (plan §14, behavior-neutral)
247
+
248
+ ### Added — `src/engine/qa-model.ts`: the normalized QA concept IR (extract-only, no scan wiring)
249
+
250
+ The plan-§14 vocabulary — Test, TestBoundary, Setup/Teardown, Fixture,
251
+ Action, Locator, Wait, Assertion, Mock, NetworkInteraction, Retry,
252
+ Navigation, Interaction, Lifecycle — now exists as a typed model
253
+ (`QaNode`/`QaSemanticModel`, all 14 concepts in `EXTRACTOR_COVERAGE`)
254
+ with per-language extractors over the ALREADY-existing parse stage:
255
+ ts-morph for TS/JS (tests/hooks via the scorer's it/test vocabulary,
256
+ calls via the measured rule vocabularies of qa-pw-002/004/005/101–145,
257
+ qa-test-001/003/004/006, jest/vi mock+retry), tree-sitter for Java/C#
258
+ (test boundaries REUSE `javaTestMethods`/`csharpTestMethods` verbatim —
259
+ the model cannot drift from the rules' scoping; call/hook/retry
260
+ vocabularies copied from the QA-JV-_/QA-CS-_ rules), regex boundaries
261
+ for Python (def test_, @pytest.fixture, time.sleep, assert).
262
+
263
+ - **Extracted, not invented:** every classification table cites the
264
+ rule whose measured vocabulary it copies (headers in qa-model.ts);
265
+ the shared `isHelperIdiom` helper moved to `jv-cs-ast.ts` so the
266
+ QA-CS-103 rule and the model share ONE implementation.
267
+ - **Adoption is additive:** NOTHING in the scan pipeline imports the
268
+ module — BEHAVIOR-NEUTRAL by construction, golden/corpus locks
269
+ untouched (proven: locks byte-identical with the model present).
270
+ - **Coverage is honest:** `EXTRACTOR_COVERAGE` documents per-language
271
+ gaps (e.g. Java has no fixture/interaction/lifecycle extractor;
272
+ Python extracts only test/fixture/assertion/wait) — visible gaps,
273
+ never silent claims (No False Proof).
274
+ - **Equivalence proven:** `tests/qa-model.spec.ts` re-expresses the
275
+ QA-JV-103 / QA-CS-103 / QA-CS-102 oracles over the model
276
+ (`testVerifies`, ancestor-chain containment via the
277
+ `firstAncestorCallNamed` generalization now carried as
278
+ `node.ancestors`) and asserts FINDING-IDENTICAL results against the
279
+ rules on the committed fixture corpora + synthetic edge shapes —
280
+ the model can carry these rules without changing any output.
281
+ Awaitedness (qa-pw-002's consumption oracle) rides on TS nodes as
282
+ `node.awaited`.
283
+
284
+ ### Verification Trust Evolution, Phase 3 — Java/C# semantic upgrade (plan §13)
285
+
286
+ ### Changed — three JV/CS rules migrated to L2 tree-sitter analysis (EVIDENCE-BACKED, detectorRevision 2)
287
+
288
+ The Phase 0.5 parse stage's tree-sitter trees (`parseJavaAst`/`parseCSharpAst`,
289
+ delivered via `ParsedFile.ast`) are now consumed by rules: a new L2
290
+ structural-analysis layer (`src/engine/jv-cs-ast.ts`) provides test-method
291
+ scoping by annotation/attribute, invocation structure, and call/argument
292
+ containment; the rule contract gains an optional `astQuery` hook whose
293
+ regex path is a MANDATORY fallback (no AST ⇒ regex, never a second
294
+ detector, plan §13.2). No type/symbol semantics are promised for JV/CS
295
+ (no Roslyn, no classpath) — semantic depth is L2 per plan §13.4.
296
+
297
+ - **QA-JV-103 (test without assertions), 50% FP (n=20) → 25.9% FP
298
+ (n=58), quarantine → extended.** Test boundaries now come from real
299
+ `method_declaration` nodes (any `@Test`/`@org.junit.Test`, argumented
300
+ TestNG forms included). The assertion oracle adds the two measured
301
+ rev-1 FP classes: Playwright's THROWING waits (`waitFor*` except
302
+ `waitForTimeout` — they throw on timeout, so the wait IS the
303
+ verification; 6 rev-1 FPs) and `verify*/check*/assert*` helper
304
+ calls (4 rev-1 FPs). Remaining FP class, documented as the L2
305
+ boundary: assertions behind arbitrarily-named helpers (keycloak's
306
+ `testValidationValid`, playwright-java's `testEnterKey`,
307
+ appsmith's `check`).
308
+ - **QA-CS-103 (test without assertions), 95% FP (n=20) → 0% FP (n=9),
309
+ quarantine → core.** The rev-1 oracle missed the whole Shouldly
310
+ extension family — 17 of the 19 FPs were `ShouldBeOfType<...>(
311
+ ...).Message.ShouldBe(...)`-style assertion-rich tests (spectre-console,
312
+ 310 count-lock fires → 0). The rev-2 oracle counts Shouldly chains,
313
+ `Assert`/`Should` receivers, the Verify snapshot framework,
314
+ PascalCase `verify*/check*/assert*` helpers (Humanizer's
315
+ `VerifyAnalyzerAsync` — the lowercase-only form was a Java camelCase
316
+ inheritance bug), C# throwing waits (`WaitFor*Async` except
317
+ `WaitForTimeoutAsync`), and conditional `throw new
318
+ *Assertion*Exception` (playwright-dotnet's ConventionTests). Grammar
319
+ error-node guard: a truncated parse never produces a finding.
320
+ - **QA-CS-102 (`Thread.Sleep`/`Task.Delay` hard sleep), 65% FP (n=20) →
321
+ 8.3% FP (n=24), quarantine → core.** Tree-sitter invocation scoping
322
+ excludes the measured environment-simulation classes: delays inside
323
+ route/expose/server delegates (`Route*Async`, `SetRoute`,
324
+ `ExposeFunction*`, 10 rev-1 FPs + delta), deliberate infinite/negative
325
+ blocks (`Task.Delay(-1)`, `int.MaxValue`, `Timeout.Infinite*`),
326
+ `Task.WhenAny` timeout races, and runner payload fixtures
327
+ (`RunAndWaitFor{Request,RequestFinished,Response}Async(() => Task.Delay…)`,
328
+ `UnrouteAllAsync` timing-window sleeps stay flagged — their delay
329
+ creates the timing window the assertion measures). Documented
330
+ trade-off: one rev-1 TP (a sub-second artificial-timing delay inside a
331
+ route delegate) is no longer flagged; the structural boundary cannot
332
+ read that intent.
333
+
334
+ All three: verdict corpus reconciled per the §07 loop (41 superseded rows
335
+ removed, 49 fresh rows adjudicated from source with per-row notes, class-B
336
+ positive fixtures for QA-CS-102/QA-JV-103 and class-C negative fixtures
337
+ for all three), count-lock baselines refreshed (microsoft-playwright-java
338
+ QA-JV-103 97→43, microsoft-playwright-dotnet QA-CS-102 52→21,
339
+ spectre-console QA-CS-103 310→0, Humanizer 34→2; keycloak +602 is a
340
+ recall gain — the rev-1 regex missed `void x() throws Exception`
341
+ signatures, tree-sitter scoping does not), and
342
+ `detector-revisions.json` bumped (sidecar entries stay; measurements
343
+ stamped rev 2). QA-JV-102 stays LEXICAL rev 1 — no migration for
344
+ symmetry (plan §12.4).
345
+
346
+ ### Verification Trust Evolution, Phase 2 — quarantine-cluster triage (plan §12.2)
347
+
348
+ ### Deprecated — 21 rules retired per docs/RULE-LIFECYCLE.md (measured 100% FP, premise wrong)
349
+
350
+ Every rule below measured 100% FP (zero TPs at n ≥ 10, `docs/FP-AUDIT.md`)
351
+ on real-world code. Per the lifecycle policy the severity is downgraded to
352
+ `info` (non-blocking everywhere) and `falsePositiveRisk` is set to `high`;
353
+ the code and fixtures stay in the repo, the frozen ID is never reused, and
354
+ any salvageable detection idea ships under a NEW rule ID. If you gated CI
355
+ on these findings, they no longer block at `info` severity; add an explicit
356
+ `severityOverrides` entry in `mjolnir.config.json` to restore blocking.
357
+
358
+ - **QA-PW-005** (business logic in `page.evaluate()`): in test files,
359
+ branching inside evaluate is the only way to reach browser state — every
360
+ measured use was browser-only test instrumentation or the API under test,
361
+ never leaked app logic. No successor.
362
+ - **QA-PW-103** (missing timeout on navigation): bare `goto()` to the app
363
+ under test is the universal navigation idiom; per-call budgets are
364
+ config territory. No successor.
365
+ - **QA-PW-105** (`expect.poll` without timeout): the default poll timeout
366
+ is a hard bound that raises — the claimed masking harm cannot occur.
367
+ No successor.
368
+ - **QA-PW-107** (`toBeVisible` on toast/banner/modal): presence +
369
+ auto-retry semantics is what suites assert; viewport visibility is a
370
+ different question the suites are not asking. No successor.
371
+ - **QA-PW-108** (`toHaveText` coupling): asserting self-owned markup's
372
+ exact text is a legitimate strong assertion; "whose markup is this" is
373
+ not statically decidable. No successor.
374
+ - **QA-PW-112** (testid naming convention): hardcoding kebab-case as _the_
375
+ convention is the defect — conventions are repo-local. Successor idea
376
+ (mixed-conventions-within-one-repo check) requires a NEW ID.
377
+ - **QA-PW-114** (legacy element handles): the auto-wait harm needs a
378
+ timing window; every measured use was immediate reads or deliberate
379
+ existence checks. No successor.
380
+ - **QA-PW-118** (`networkidle` waits): the flake source is environmental
381
+ background traffic, absent by construction where the rule fired; harm is
382
+ not code-detectable. No successor.
383
+ - **QA-PW-119** (module-level state order dependence, was `error`): FPs
384
+ scatter across ≥5 legitimate infrastructure idioms (per-test teardown
385
+ harnesses, counters, vi.hoisted fixtures, memoized shared infra) — an
386
+ error-severity rule at 0 TP / 24 is actively misleading. Successor idea
387
+ (cross-test write→read dataflow analysis) requires a NEW ID.
388
+ - **QA-PW-120** (missing environment guard): file-level keyword
389
+ co-occurrence does not imply engine dependence; even the corpus's e2e
390
+ specs are engine-agnostic. No successor.
391
+ - **QA-PW-145** (no a11y assertions): absence of optional coverage is not
392
+ a defect finding; the heuristic fires on every UI spec by construction.
393
+ Successor idea (a11y-coverage reporting) requires a NEW ID.
394
+ - **QA-TQUAL-001** (mock-only verification, 0 TP / 26): spies observe the
395
+ real unit's output — the mock call IS the observable contract;
396
+ stand-in vs observer is not statically decidable. No successor.
397
+ - **QA-PY-006** (empty test body `pass`): 18/20 FPs were pytester
398
+ test-data scripts; genuinely collected empty tests don't occur in real
399
+ code. No successor.
400
+ - **QA-PY-008** (mock-only verification): boundary mocking and real-output
401
+ spies are contract testing, not mock theater. No successor.
402
+ - **QA-PY-010** (random/time without freeze): wall-clock reads are the
403
+ measured subject in timing/throttle tests; freezing would defeat them.
404
+ No successor.
405
+ - **QA-JV-108 / QA-CS-108** (hardcoded environment URL): FPs split across
406
+ HAR-replay fixtures, route-mocked origins, and proxy-failure tests — no
407
+ mechanically discriminable shape (the M-06 header concedes a fake-TLD
408
+ lookahead fixes zero measured FPs). No successor.
409
+ - **QA-JV-110 / QA-CS-110** (no a11y assertions): same absence-heuristic
410
+ premise failure as QA-PW-145. Successor idea requires a NEW ID.
411
+ - **QA-JV-111 / QA-CS-111** (blanket route mock): route-API self-tests and
412
+ fixture setup; no provable exclusion, and framework gating cannot help.
413
+ Revival on an application-repo corpus would need a NEW ID.
414
+
415
+ ### Changed — Phase 2 retunes (EVIDENCE-BACKED, detectorRevision 2 per plan §07)
416
+
417
+ Each retune targets the single fixable root cause its measured FP cohort
418
+ shares (full evidence table: docs/RULE-LIFECYCLE.md "Phase 2
419
+ quarantine-cluster triage"). Detection-logic changes bump the rules'
420
+ `detectorRevision` to 2, invalidating the revision-1 measurements (stale →
421
+ provisional → re-measure, plan §07); must-not-fire fixtures now encode the
422
+ measured FP shapes so the retunes are regression-locked in both directions.
423
+ Tier stays quarantine until re-measurement says otherwise.
424
+
425
+ - **QA-PW-102** (load-wait instead of assertion): the wait now fires only
426
+ when it is the TERMINAL wait (no `expect`/assert/`expect.poll` follows)
427
+ and is skipped when consumed by `expect(...).rejects` — the "instead of
428
+ an assertion" premise, now actually checked. Clears all 20 measured FPs
429
+ (vite HMR synchronization waits); keeps the must-fire no-assertion shape.
430
+ - **QA-ENV-001** (environment coupling): the fixed-port sub-pattern no
431
+ longer matches loopback endpoints (`localhost`/`127.x`) — they are the
432
+ suite's own fixture containers (Azurite / DynamoDB Local / Mongo), the
433
+ entire measured FP cohort. Dotted hostnames and non-loopback IPv4
434
+ literals with ports still fire; OS-path, locale, and local-time-getter
435
+ sub-patterns are unchanged.
436
+ - **QA-PY-003** (no-assertion test): the verification vocabulary gains
437
+ `pytest.warns` / `pytest.deprecated_call` / `pytest.fail`, and a
438
+ `test_*` function referenced by name elsewhere in its file (pytester-
439
+ style test data) is skipped — the collected assertion lives in the
440
+ parent test.
441
+ - **QA-PY-004** (bare truthiness assert): boolean-predicate calls are
442
+ skipped — `assert isinstance(x, T)` type guards and
443
+ `assert s.startswith(...)`-style content predicates are real checks
444
+ (the measured FP clusters). Bare identifier/attribute asserts still
445
+ fire. This rule was measured at 45% FP (n=20), not 100%; with the
446
+ clusters removed the re-measurement is expected to approach the
447
+ extended band (≤30%), pending delta classification.
448
+ - **QA-PY-007** (raises without match): `pytest.raises(X) as exc_info`
449
+ followed by an assert/expect on `exc_info.value` is skipped — the
450
+ message IS verified without `match=` (8 of 13 measured FPs). The
451
+ genuinely vague raises blocks still fire; remaining FP residue
452
+ (single-possible-exception blocks) is not statically decidable.
453
+ - **QA-PY-105** (Playwright-Python test without assertions): a called
454
+ helper whose name asserts (`assert_*`/`expect_*`/`verify_*`/`check_*`)
455
+ or waits (`wait_for_*`) counts as verification — assertions delegated
456
+ to imported helpers, the entire measured FP cohort (streamlit e2e).
457
+ - **QA-JV-106 / QA-CS-106 / QA-PY-104** (brittle selectors): no pattern
458
+ change — the measured 100% FP rows predate Bug Map M-06's removal of
459
+ the querySelector/QuerySelectorAsync/query_selector patterns, so the
460
+ measurement described a detector that no longer ships. The §07 fix is
461
+ the detectorRevision bump to 2 (stale → provisional → re-measure);
462
+ the surviving xpath=/nth-child/absolute-path patterns stay
463
+ quarantine-tier until re-measured.
464
+
465
+ ### Changed — §07 loop closed: rev-3 delta re-measurement (corpus rescans + verdict reconciliation)
466
+
467
+ The revision-2/3 detectors were re-run over the corpus repos that
468
+ produced the retuned rules' original verdicts, and the committed verdict
469
+ corpus was reconciled to the current detectors (the §07 loop the
470
+ capability matrix calls "re-measure"):
471
+
472
+ - **Delta method:** every committed row whose finding no longer fires
473
+ was removed (superseded — it described a pre-retune detector); every
474
+ NEW finding was adjudicated from source context per
475
+ `tests/corpus/verdicts/README.md` criteria, with a quota sample of 20
476
+ per rule on the largest surfaces (QA-PY-004/007) and full coverage on
477
+ the small ones. Net: 157 rows retired, 71 adjudicated rows appended,
478
+ 1423 → 1337 classified verdicts, measured coverage 78 → **72/91**
479
+ (the six rules whose remaining classified counts fell below n=10 —
480
+ QA-PW-102, QA-PY-104, QA-PY-105, QA-ENV-001, QA-JV-106, QA-CS-106 —
481
+ are now unmeasured at their current revisions; their sidecar entries
482
+ were removed accordingly and they stay quarantine/extended-tier
483
+ pending re-measurement).
484
+ - **Re-measured envelopes (rev 3):** QA-PY-003 82% FP (n=17; remaining
485
+ FPs are pytest doc examples and pytester collection fixtures —
486
+ accepted residue), QA-PY-004 76% (n=21; remaining FPs are
487
+ pytest's own predicate-call idiom families: fnmatch/samefile/
488
+ isimportable membership checks and deliberate `assert False`
489
+ fail-marker DATA), QA-PY-007 79% (n=34; remaining FPs are
490
+ single-possible-exception API-error contracts where the raised type
491
+ IS the assertion — not statically decidable). All three stay
492
+ quarantine-tier with their detectorRevision 3 stamped in the sidecar
493
+ and FP-AUDIT; their accepted-residue FP causes are documented in
494
+ docs/RULE-LIFECYCLE.md's triage table.
495
+ - **QA-PW-102 / QA-PY-105: fully cleared** — 0 findings on the
496
+ previously-20/20-FP corpus slices at revision 2.
497
+ - **QA-ENV-001 (rev 3):** the wave-2 delta (20/20 FP on the rev-2
498
+ detector) showed no host shape is statically decidable — the
499
+ fixed-port sub-pattern is dropped entirely; OS-path, locale, and
500
+ local-time-getter sub-patterns stay (their remaining 6 findings on
501
+ vite are path-literal test DATA, adjudicated FP and recorded).
502
+ - **§07 comparability guard:** the FP regression governor
503
+ (`checkFpRegression`) no longer compares measurements across
504
+ detectorRevision boundaries — a revision mismatch means the old rate
505
+ does not describe the current detector, so it is neither flagged nor
506
+ blessed.
507
+
508
+ ### Verification Trust Evolution, Phase 2 — detectionStrategy enum (D6 closed, scan-behavior-neutral)
509
+
510
+ ### Changed — D6 enum migration (plan §12.1; metadata-only)
511
+
512
+ - **`detectionStrategy` is now the enforced §09.6 enum** (`LEXICAL | AST |
513
+ SEMANTIC | FRAMEWORK | RUNTIME`) instead of free text: `src/rules/rule.ts`
514
+ types `RuleMeta.detectionStrategy` as the union, and a registry ratchet
515
+ (tests/rules.registry.spec.ts) fails CI when any rule omits it or carries
516
+ a non-enum value — the "free text" drift class cannot reintroduce itself.
517
+ All 91 registry rules were migrated in place. This is a metadata migration,
518
+ not a detection change: no `run()` body, pattern, or scoping was touched,
519
+ so scan findings, golden fixtures, and corpus baselines are byte-identical
520
+ (the Phase 0 classification of this work as BEHAVIOR-NEUTRAL for scan
521
+ findings, plan §06). Per §11.3 the enum conversion is metadata naming only
522
+ and does NOT bump any `detectorRevision` — every measurement stays valid.
523
+ - **Legacy nuance preserved, not deleted:** the richer free-text
524
+ declarations ("regex pattern + inside-string oracle", "parsed YAML +
525
+ test-command gate", …) moved verbatim into a new optional
526
+ `detectionNotes` field rendered alongside the enum in the rule docs
527
+ pages. The capability matrix now renders the declared enum directly in a
528
+ single "Detection strategy (enum)" column (the provisional
529
+ "Enum (proposed)" guess column is gone — the declared value IS the
530
+ enum); `UNCLASSIFIED` renders only for an undeclared value.
531
+ - Mapping applied: regex-over-text/absence-sweep detectors → `LEXICAL`
532
+ (incl. QA-TQUAL-002's AST-stripped text pattern, which is a text pass
533
+ over code-only text); ts-morph node-walk detectors (QA-PW-002,
534
+ QA-PW-005) → `AST`; GitHub-Actions workflow-structure detectors
535
+ (QA-CI-001/009/010) → `FRAMEWORK`. `SEMANTIC` and `RUNTIME` remain
536
+ reserved (no rule ships either yet).
537
+
538
+ ### Verification Trust Evolution, Phase 1 exit — dedicated corpora + wave-5 measurement (plan §11.5/§08)
539
+
540
+ ### Added — dedicated corpora (§11.5) and the wave-5 measurement
541
+
542
+ - **CORPUS 19 → 34 repos** (`tests/corpus/audit.ts`): 14 new real-world
543
+ repos chosen by evaluating each candidate's unmeasured-rule fire count
544
+ at HEAD before committing (`vitest-dev-vitest`, `streamlit-streamlit`,
545
+ `apache-airflow`, `iluwatar-java-design-patterns`,
546
+ `spectreconsole-spectre-console`, `Humanizr-Humanizer`,
547
+ `cypress-realworld-app`, `keycloak-keycloak`, `appsmithorg-appsmith`,
548
+ `getsentry-sentry`, `github-docs`, `vercel-next-js`, `hashicorp-vault`,
549
+ `nocodb-nocodb`). This is the plan's dedicated-Corpora requirement for
550
+ §11.5: CI-workflow density for the starved QA-CI-* rules and JV/CS
551
+ application repos so the QA-JV/QA-CS rules measure on consumer code,
552
+ not just the Playwright bindings themselves (D5). Candidates whose scan
553
+ truncated against the budget (`n8n`, `posthog`, `vscode`) were rejected —
554
+ a partial scan can never be count-locked.
555
+ - **Committed class-B/C fixture corpora (§08):**
556
+ `tests/corpus/positive-fixtures/` (realistic anti-pattern exhibits that
557
+ MUST fire — class-B recall evidence, every fire classifies TP) and
558
+ `tests/corpus/negative-fixtures/` (realistic legitimate code that must
559
+ NOT fire — class-C precision evidence, any fire is a recorded FP).
560
+ These give rules whose patterns are rare in the wild a measurement-grade
561
+ verdict surface, versioned with the verdicts that classify them via
562
+ `local:` corpus URLs. Excluded from self-scan, vitest, eslint, prettier
563
+ and the test tsconfig — they are DATA.
564
+ - **`corpus-sample.ts --unmeasured-only`:** the verdict-harvesting loop
565
+ can sample only rules without a valid measurement, so classification
566
+ effort goes to the exit gate instead of re-sampling measured rules.
567
+ - **`audit.ts --only=<name>,<name>`:** re-check or re-record a corpus
568
+ subset (used to re-verify a repo after a transient truncation without
569
+ rescanning everything); the completeness threshold now applies to the
570
+ filtered set. Clones use `git -c core.longpaths=true` (scoped, not a
571
+ global config change) and the per-repo scan budget is 60s → 120s; the
572
+ audit job gets `NODE_OPTIONS=--max-old-space-size=8192` (sentry's
573
+ repo-scale parse OOMs the default heap) and a 60-minute timeout.
574
+ - **Measured coverage 43 → 78 of 91** (1423 classified verdicts, 0 blank,
575
+ 0 UNSURE): 35 rules newly measured at n ≥ 10 — QA-TEST-001, QA-TEST-006,
576
+ QA-TEST-010, QA-TQUAL-002, QA-TQUAL-009, QA-TQUAL-011, QA-CI-001,
577
+ QA-CI-002, QA-CI-005, QA-CI-008, QA-PW-003, QA-PW-004, QA-PW-104,
578
+ QA-PW-113, QA-PW-115, QA-PW-117, QA-PW-121, QA-PW-123, QA-PW-140,
579
+ QA-PW-141, QA-PW-142, QA-PW-144, QA-PY-001, QA-PY-009, QA-PY-011,
580
+ QA-PY-012, QA-PY-103, QA-PY-105, QA-JV-101, QA-JV-102, QA-JV-109,
581
+ QA-CS-103, QA-CS-107 (plus the four previously counted). **Phase 1
582
+ exit gate MET: unmeasured 48 → 13 (≤ 20).**
583
+ - **Explicit tier declarations for all 35 newly measured rules, set from
584
+ the measured FP band (§11.2: ≤10% core, ≤30% extended, >30%
585
+ quarantine):** core — QA-PW-003, QA-PW-104, QA-PW-113, QA-PW-117,
586
+ QA-PW-121, QA-PW-140, QA-PY-001, QA-PY-009, QA-PY-011, QA-PY-103,
587
+ QA-JV-101, QA-JV-109; extended — QA-TQUAL-011, QA-CI-002, QA-CI-007,
588
+ QA-PW-141, QA-PW-142, QA-PW-144; quarantine — QA-TEST-001, QA-TEST-006,
589
+ QA-TEST-010, QA-TQUAL-002, QA-TQUAL-009, QA-PW-004, QA-PW-115,
590
+ QA-PW-123, QA-PY-012, QA-PY-105, QA-CI-001, QA-CI-005, QA-CI-008,
591
+ QA-CI-010, QA-CS-103. Every measured entry carries `detectorRevision`
592
+ in the sidecar. **Evidence-backed, not silent:** the quarantine
593
+ demotions remove error-severity deductions from default scans, which is
594
+ the tier policy working as designed — the failing specs were updated
595
+ with their reasoning inline (gate tests re-anchored on QA-CI-009, the
596
+ demo repo's CI grew real CI-009 exhibits, the demo/hero assets and
597
+ `fix` command scan with `--strict`).
598
+ - **Recall-floor + corpus baselines:** 34 count-locked baselines recorded
599
+ (non-partial scans only); the §20.6 recall floor extends to the
600
+ expanded registry.
601
+
602
+ ### Changed — scan-behavior fallout of the measured demotions (explained)
603
+
604
+ - `mjolnir fix` now scans with `--strict`: an auto-fixable rule that is
605
+ measured into quarantine (QA-TEST-001's `.only` fix) must still be
606
+ fixable — hiding it would make `fix` a no-op on its own target debt.
607
+ - The demo repo's CI workflow gained genuine CI-009 exhibits (piped and
608
+ `;`-sequenced test commands) so the demo keeps demonstrating the
609
+ NEEDS-WORK band now that its QA-TEST-001/QA-CI-001/PW-004 debt is
610
+ quarantine-capped to info (non-deducting); `docs:demo`/`docs:hero` scan
611
+ with `--strict` so the committed assets and their drift locks stay in
612
+ sync with the precision-contract spec.
613
+
614
+ ### Verification Trust Evolution, Phase 1 — measurement infrastructure
615
+
616
+ ### Added — UNSURE adjudication gate + QA-PW-101 measured (plan §11.5)
617
+
618
+ - **UNSURE ceiling ratchet:** `npm run fp-audit:generate` now fails when the
619
+ UNSURE backlog grows beyond the committed
620
+ `tests/corpus/verdicts/unsure-ceiling.json` (the §11.5 mechanism: UNSURE
621
+ never counts into `n` but always triggers review). The ceiling only moves
622
+ DOWN via documented adjudication; upward movement requires an explicit
623
+ `--update` whose diff names every rule that grew. Criteria live in
624
+ `tests/corpus/verdicts/README.md`.
625
+ - **QA-PW-101 is measured (the D5 "parked on 20 UNSURE" defect resolved):**
626
+ all 20 UNSURE verdicts were adjudicated by reading the cited sources at
627
+ repo HEAD — **20 TP, 0 FP, n=20** (next-auth's session-sync sleeps before
628
+ reading session state; sveltejs/kit's 100 ms request-observation windows
629
+ that false-pass when a stray refresh starts after the window). The rule
630
+ now declares `tier: "core"` (0% FP ≤ 10%, n ≥ 10, revision 1, recall
631
+ floor satisfied: fires in 4 corpus baselines) — the measurement-dependent
632
+ default would resolve it to core anyway, but measured rules declare
633
+ their tier explicitly (the D3 Step 2 invariant).
634
+ - **QA-TQUAL-009 UNSURE row resolved → FP** (tanstack-query angular
635
+ inject-query:529 — a deliberate `void` fire-and-forget whose assertions
636
+ still fail the run via vitest's unhandled-rejection handling; the
637
+ FALSE-GREEN diagnosis does not hold). n=2 — below the measurement bar,
638
+ still unmeasured/PROVISIONAL.
639
+ - **Coverage:** 42/91 → **43/91** measured rules; UNSURE backlog 21 → **0**.
640
+
641
+ ### Changed — D3 two-step tier fix (plan §11.2; scan-behavior-neutral)
642
+
643
+ - **Step 1 (explicit tiers):** every rule now declares its `tier`
644
+ explicitly — the 42 formerly implicit-core rules received an explicit
645
+ `tier: "core"` matching their effective tier. Byte-identical scan
646
+ behavior (the declared value equaled the omitted-tier default);
647
+ generated docs updated with an explained diff (the matrix's
648
+ "explicit tier declarations" line moved 49/91 → 91/91).
649
+ - **Step 2 (measurement-dependent default):** the omitted-tier default is
650
+ no longer unconditionally core. A rule that omits `tier` resolves via
651
+ `effectiveTier` (new `src/rules/measurement.ts`): **core** only with a
652
+ valid corpus measurement (n ≥ 10 at a matching `detectorRevision`),
653
+ otherwise **extended**, displayed with the new **PROVISIONAL** status.
654
+ PROVISIONAL is a display status (`tier extended/unmeasured`), not a
655
+ tier value — no schema churn. Scan findings are byte-identical: the
656
+ pipeline enforces only `quarantine` (severity/info + E0 caps, --strict
657
+ filter), and every quarantine rule declares its tier explicitly;
658
+ overlap-dedup consumes declared tiers only, and no demoted rule
659
+ participates in an `overlapWith` tier comparison that could change a
660
+ survivor set (all overlap groups are single-declarer/single-target).
661
+ - **The D3 demotion (38 rules drop from effective core to
662
+ extended/PROVISIONAL):** QA-TEST-001, QA-TEST-006, QA-TEST-010,
663
+ QA-TQUAL-002, QA-TQUAL-009, QA-TQUAL-011, QA-PW-003, QA-PW-004,
664
+ QA-PW-101, QA-PW-104, QA-PW-113, QA-PW-115, QA-PW-116, QA-PW-117,
665
+ QA-PW-121, QA-PW-123, QA-PW-124, QA-PW-125, QA-PW-140, QA-PW-141,
666
+ QA-PW-142, QA-PW-144, QA-CI-001, QA-CI-002, QA-CI-005, QA-CI-007,
667
+ QA-CI-008, QA-CI-009, QA-CI-010, QA-PY-001, QA-PY-009, QA-PY-011,
668
+ QA-PY-012, QA-PY-101, QA-PY-103, QA-PY-105, QA-PY-106, QA-PY-107.
669
+ The 4 measured implicit-core rules (QA-PW-002, QA-PY-002, QA-JV-105,
670
+ QA-CS-101) keep core. Unmeasured-effective-core count: 38 → **0**.
671
+ **Suite-invalidating callout (plan §11.2):** QA-TEST-001 (focused test
672
+ committed, `suiteInvalidating: true`) is among the demoted rules — its
673
+ findings still void the suite's pass claim and still gate CI at
674
+ severity=error; only its tier/statistics status changed. QA-PY-001 is
675
+ the suite-invalidating Python sibling (same class).
676
+ - **Registry ratchet (plan §20.3, enforced in code):** new
677
+ `tests/registry-ratchet.spec.ts` fails CI on ANY unmeasured rule in
678
+ effective core (`tier core ⇒ valid MEASURED_FP with matching
679
+ detectorRevision, FP ≤ 10%, n ≥ 10`), on any detectorRevision mismatch
680
+ (§20.5: stale ⇒ provisional), and — recall floor (§20.6) — on a core
681
+ rule that fires nowhere in the corpus baselines. §20.1 evidence-state
682
+ monotonicity: the measured ratio may only improve without a
683
+ machine-detectable `MEASUREMENT-EXCEPTION` marker in this CHANGELOG.
684
+ - **`mjolnir doctor`**: `MAX_UNMEASURED_CORE` lowered 40 → **0** (Phase 1
685
+ exit gate: 0 unmeasured in effective core, now enforced); tier checks
686
+ consume `effectiveTier` + stale-measurement logic. Display surfaces
687
+ (`mjolnir explain`, `mjolnir rules`, generated rule docs, capability
688
+ matrix) render the PROVISIONAL status honestly.
33
689
 
34
- ## [Unreleased]
690
+ ### Verification Trust Evolution, Phase 0 + Phase 1 prep
691
+
692
+ ### Added — Rule Capability Matrix (Phase 0)
693
+
694
+ - **Capability Matrix v0** (`docs/RULE-CAPABILITY-MATRIX.md` +
695
+ `docs/RULE-CAPABILITY-MATRIX.json`): generated per-rule inventory of all
696
+ 91 rules from the registry + `MEASURED_FP` + the verdict corpus —
697
+ category, languages, frameworks, declared detection strategy with a
698
+ provisional enum mapping (Phase 0 contract proposal), semantic depth,
699
+ measured flag, FP rate with sample size, corpus size and diversity.
700
+ Unknown fields render as `UNCLASSIFIED` — visible gaps are the
701
+ deliverable. Regenerated with `npm run docs:capability`; drift-locked by
702
+ `tests/capability-matrix.spec.ts` and the generated-docs-drift CI job.
703
+ - **Declared-vs-measured cross-check report** (ledger class D9): the
704
+ matrix names every measured rule whose declared tier violates the FP
705
+ ceilings (core > 10%, extended > 30%) and produces the D3 demotion list
706
+ (38 unmeasured rules currently in effective core — Phase 1 input).
707
+ Current run: **0 D9 mismatches** (all measured > 30% FP rules already
708
+ declare `quarantine`).
709
+ - **Defect ledger recorded** into the matrix metadata (plan §02, D1–D8
710
+ with owning phase per defect).
711
+
712
+ ### Added — detectorRevision scaffold (Phase 1 prep)
713
+
714
+ - **`tests/corpus/detector-revisions.json` sidecar**: hand-maintained,
715
+ one entry per measured rule (all at revision 1 today), diffable.
716
+ `MEASURED_FP` entries now carry `detectorRevision` stamped from the
717
+ sidecar by `fp-audit:generate`; `docs/FP-AUDIT.md` gains a
718
+ `detectorRev` column. Measurement inheritance law (§07): a measurement
719
+ belongs to a specific detector implementation, not merely to a rule
720
+ ID. Drift lock extended in `tests/measured-fp-generated.spec.ts`
721
+ (sidecar covers exactly the measured set; revisions are positive
722
+ integers matching the sidecar).
723
+
724
+ ### Fixed — packaging: offline grammar loading (Phase 0.5 spike, D2)
725
+
726
+ - **`tree-sitter-wasms` and `web-tree-sitter` moved to `dependencies`**
727
+ (web-tree-sitter keeps its exact `0.25.6` pin — 0.26.x cannot load the
728
+ prebuilt grammar files). The published CLI's dependency tree now
729
+ carries the tree-sitter Java/C# grammars, so `npm install mjolnir-qa`
730
+ can load them offline once the Phase 0.5 parse-stage wiring consumes
731
+ them. Removed the misleading `!dist/**/*.wasm` files exclusion (the
732
+ grammars ship via the dependency, not the bundle).
733
+ - **Pack-smoke regression test**: `tests/package-smoke.spec.ts` asserts
734
+ the packed package declares both as runtime dependencies and that the
735
+ java/c_sharp grammars resolve inside the installed dependency tree.
736
+
737
+ ### Fixed — adapter header claims (D4)
738
+
739
+ - `src/adapters/java.ts` no longer claims to be a "Second tree-sitter
740
+ consumer" — it is a regex-layer adapter; the tree-sitter-java grammar
741
+ and the async `parseJavaAst` seam exist but are not wired into the
742
+ synchronous scan (D1). Same honest correction for
743
+ `src/adapters/csharp.ts` and `src/engine/adapter.ts` (whose header
744
+ still claimed tree-sitter "arrives in R2 with Python").
745
+
746
+ ### Verification Trust Evolution, Phase 0.5 — async parse stage (D1)
747
+
748
+ ### Changed — parse stage wired into the scan pipeline (D1 closed, BEHAVIOR-NEUTRAL)
749
+
750
+ - **Async parse stage between discovery and rule execution** (plan §10.1):
751
+ `runScan`'s per-file loop now awaits an optional `LanguageAdapter.parseAst`
752
+ hook before running rules. `runRules` and every rule stay synchronous and
753
+ consume the tree via `ParsedFile.ast` — the engine is NOT async end-to-end,
754
+ only the one inherently-async seam (WASM grammar load) is. `main()` and the
755
+ script entry points await the returned promise.
756
+ - **Java and C# adapters implement `parseAst`** backed by the previously-dead
757
+ `parseJavaAst`/`parseCSharpAst` (defect D1: tree-sitter AST was built and
758
+ tested in Sprint 8 but never consumed). Parse failure or a missing grammar
759
+ resolves `undefined` and rules fall back to the regex path — never fatal.
760
+ No rule consumes the AST yet (Phase 3 wires specific JV/CS rules), so scan
761
+ findings are byte-identical: golden lock, corpus baselines, and generated
762
+ assets all unchanged (BEHAVIOR-NEUTRAL mode, plan §06).
763
+ - **Parser lifecycle management** (plan §10.3): one memoized `Parser` per
764
+ grammar, bounded by a fixed-size parse-slot semaphore
765
+ (`MAX_CONCURRENT_PARSES`); every per-file tree is released via
766
+ `ParsedAst.dispose()` (`tree.delete()`) in a `finally`-equivalent position
767
+ that runs on normal completion, rule crash, per-file budget expiry, and
768
+ adapter throw; `releaseTreeSitterResources()` tears down the memoized
769
+ parsers after each scan (library-consumer hygiene). No leak path depends
770
+ on rules completing successfully.
771
+ - **Call-graph consequences**: `computeImpact` (and therefore
772
+ `runImpactCommand`) is async; `scripts/corpus-sample.ts`,
773
+ `scripts/generate-readme-demo.ts`, and `scripts/generate-readme-hero.ts`
774
+ await `runScan`; the full test suite (~40 call sites across 25 spec files)
775
+ awaits the now-async commands. No behavior change anywhere.
776
+ - **Verification evidence** (exit gate §10): golden lock byte-identical
777
+ (3/3), corpus count-lock unaffected (no JV/CS rule reads the AST),
778
+ capability-matrix + FP-AUDIT generated docs byte-identical, pack smoke
779
+ re-proven manually: `npm pack` → clean `npm install` → offline grammar
780
+ load + parse of real Java and C# source → offline `mjolnir` scan of a
781
+ Java fixture detecting QA-JV-102. `web-tree-sitter` stays pinned to
782
+ exactly `0.25.6` (§10.5, documented 0.26.x breakage).
783
+
784
+ ### Added — score instrument redesign (hammer states)
785
+
786
+ - **ScoreState model** (`src/reporter/score-state.ts`): one pure source of
787
+ truth for band / verdict / color / headline per score — critical 0–49,
788
+ warning 50–79, trusted 80–99, forged 100. `verdictFor`, the terminal
789
+ gauge and the badge all delegate to it.
790
+ - **The hammer is now the score instrument** (terminal): a state-colored
791
+ hammer block renders above the WORTHINESS line — cracked (0–49),
792
+ strained with partial runes (50–79), charged with energy arcs (80–99),
793
+ halo + lightning at 100. A plain-text caption (`[CRACKED]` /
794
+ `[STRAINED]` / `[CHARGED]` / `[FORGED]`) carries the state without
795
+ color; ASCII fallback included.
796
+ - **Trusted is aurora-cyan, forged is white-gold** on every surface
797
+ (terminal palette, web tokens, brand README). Green is no longer a
798
+ score color — it survives for non-score success contexts only.
799
+ - **Findings render as cards** (terminal): Problem → Impact → Fix →
800
+ Verify with the evidence tag and measured FP rate beside the title;
801
+ rules with >3 findings collapse under one "same fix applies" header;
802
+ non-verbose shows 10 cards with an overflow line, `--verbose` shows
803
+ everything.
804
+ - **FORGED block at 100** replaces the bare FLAWLESS VICTORY line in
805
+ unicode mode (trophy retained inside; the `*** FLAWLESS VICTORY ***`
806
+ ASCII contract string is preserved).
807
+ - **PR comments show score drift** (`Score: 72/100 (+5 since baseline
808
+ <sha>)`) using the new additive `score` field in the baseline JSON, and
809
+ carry per-finding evidence tags.
810
+ - **Badge thresholds aligned** with the reporter: ≥80 / ≥50 / 100
811
+ (was ≥90 / ≥75 / ≥50), colors `red` / `yellow` / `important` /
812
+ `success`; the message at 100 reads `100/100 · forged`.
813
+
814
+ ### Fixed — security & detection-regression audit (`.planning/AUDIT-2026-08-30-QA.md`)
815
+
816
+ - **QA-1 (P0, detection regression):** QA-TEST-003's M0-#4 header rewrite
817
+ required a space after `async` — `async()=>{` bodies silently stopped
818
+ matching. `\s*` restores them without reopening the bare-`return` exemption.
819
+ - **QA-2 (P0, detection regression):** single-walk discovery applied the
820
+ UNION of every adapter's `dirSkips`, so Python's `env` (= virtualenv)
821
+ and Java's `build` (= Gradle output) hid directories from every other
822
+ language — real TS test dirs (e.g. withastro/astro's `test/units/env/`)
823
+ silently vanished from scans. `dirSkips` are now applied per owning
824
+ language at file level.
825
+ - **QA-3 (P1):** `isInsideEmbeddedCode` truncated the masked run at the
826
+ first space inside a literal, so embedded test-data
827
+ (`'test(" foo", function () {})'`) was classified as live code and
828
+ fired 6 FP findings on eslint-plugin-playwright's ruleTester tables.
829
+ - **QA-4 (P1):** config `exclude` was never validated —
830
+ `exclude: [1, {}, null]` crashed the scan (exit 20). Non-string entries
831
+ are now a fixable usage error (exit 10), with a defense-in-depth filter
832
+ in pattern compilation.
833
+ - **QA-5 (P1):** an unparseable suppression `expires` value silently
834
+ degraded to "expired" via NaN comparisons; it is now a fixable
835
+ validation error at load time.
836
+ - **QA-6 (P1, documented-behavior fix):** the README's 90-day suppression
837
+ policy was only applied at write time by the `ignore` command —
838
+ hand-written entries without `expires` stayed active forever. The
839
+ default is now enforced at enforcement time, anchored at the config
840
+ file's mtime; `mjolnir suppressions` labels the default explicitly.
841
+ - **QA-7 (P1):** plugin reserved-prefix spoof rejection was
842
+ case-sensitive — `"qa-test-001"` walked straight past it. Matching is
843
+ now case-insensitive.
844
+ - **QA-8 (P2):** suppression `files` globs written with Windows
845
+ backslashes could never match the normalized finding paths; both sides
846
+ are normalized before matching.
847
+ - **QA-9 (P2, defense in depth):** option-shaped `--base` values are
848
+ refused at the git layer even when a programmatic caller bypasses
849
+ `parseArgs` (option-injection → `--upload-pack=` command execution).
850
+ - **QA-10 (P2):** finding metadata (hostile filenames, plugin messages)
851
+ reached the terminal and markdown PR comments unsanitized — ANSI
852
+ escapes and control characters are now stripped, and the PR comment
853
+ escapes markdown-significant characters.
854
+ - **QA-11 (P2):** the JUnit XML scan was quadratic on unclosed
855
+ `<testcase` floods (minutes of CPU for a 20 MB hostile report); the
856
+ scanner is now linear with a bounded-time regression test.
857
+ - **QA-12 (P2):** `loadBaseline`/`loadStats` are now total over arbitrary
858
+ JSON — `null` finding entries used to crash `diff`, and hostile stats
859
+ shapes leaked string junk into totals.
860
+ - **QA-13 (P3, defense in depth):** SARIF artifact URIs normalize
861
+ backslashes — `encodeURI` leaves `\` literal, which is not a valid
862
+ RFC 3986 uri-reference character.
863
+ - **QA-15 (P1):** the code-text mask iterated code points instead of
864
+ UTF-16 code units — one emoji made the mask shorter than the text and
865
+ silently disabled masking for the whole file.
866
+ - **QA-16 (P2):** files larger than the 1 MiB discovery cap were dropped
867
+ silently; the skip is now counted and flagged (`file-size`), keeping
868
+ the scan honest about what it did not read.
869
+
870
+ ### Changed — corpus baseline
871
+
872
+ - Regenerated after review: M0-#2's QA-TEST-004 duplicate-count fix
873
+ (−5 on withastro-astro — every removed finding was a same-position
874
+ duplicate), M0-#4's function-body/return-narrowing detections (net
875
+ +1/+1/+8/+6 across vite/svelte-kit/tanstack-query/eslint-plugin-playwright,
876
+ spot-reviewed on real code), QA-3/QA-6 masking suppressions
877
+ (embedded-code test data no longer fires), and QA-15's mask-alignment
878
+ fix (QA-PW-105/108 match against `codeText`, whose offsets were
879
+ misaligned after any astral character — +27/+22/+1 restored matches on
880
+ vite/astro/tanstack-query).
881
+
882
+ ### Added — Open-Beta E2E test plan (Tier 1–5, ~1,700 new test assertions)
883
+
884
+ - **Coverage ratchet:** statements/branches/functions/lines at literal
885
+ 100% per file (`perFile: true`). Every branch arm, catch path, and
886
+ fallback in `src/**` is exercised or provably dead — ~30 dead guards
887
+ (unreachable `?? ""` fallbacks, unreachable-`if` arms behind
888
+ `noUncheckedIndexedAccess`, a dead proof-abort block in `fix`) were
889
+ removed or exported as testable pure helpers with the golden lock,
890
+ self-scan and determinism tripwires green throughout.
891
+ - **E2E journeys** (`tests/e2e/`): nine spec files run the built
892
+ `dist/cli.mjs` as a real child process — tarball first-run (the path
893
+ that historically caught macOS-only CLI breakage), CI PR flow with
894
+ changed-scope attribution against real git fixtures, the baseline →
895
+ diff → stats loop, the fix flow, the forensics flow, explain/rules,
896
+ create-rule onboarding, the config journey (gate/severityOverrides/
897
+ ignore/expiry honored end-to-end), and a full exit-code contract sweep
898
+ across every documented command.
899
+ - **Precision & accuracy:** every scorer number verified against the
900
+ documented benchmark (deductions 8/3/1, E2/E1/E0, honesty cap 99,
901
+ error ceiling 95, suite-invalidating ceiling 49, smoothing) with
902
+ fast-check property invariants (≥1,000 iterations: score bounds,
903
+ monotonicity, order-symmetry), Selector Health exact score vectors,
904
+ hand-computed forensics math, terminal-footer/JSON deduction
905
+ consistency, Mermaid well-formedness, and a three-verdict-band proof
906
+ (WORTHY / NEEDS WORK / UNWORTHY each reached for its stated reason).
907
+ - **Regression & integration:** adapter→reporter matrix (one finding
908
+ asserted on terminal, JSON, SARIF, and Mermaid), plugin flow
909
+ integration (valid plugin + reserved-prefix rejection), cross-file
910
+ analysis, monorepo containment (a workspace scan never reports
911
+ siblings), a mutation guard (each sampled rule's finding drops to zero
912
+ when the offending line is removed), and baseline forward-compat.
913
+ - **Nightly stress workflow** (`.github/workflows/stress.yml`, never
914
+ PR-blocking): 10k-file mixed-language synthetic repo under a 120 s
915
+ budget with no partial degradation, pathological trees (200-deep
916
+ nesting, 10k-char lines, unicode filenames, LF/CRLF/BOM mixes,
917
+ junctions, malformed specs), a 20-run byte-identical + flat-RSS soak,
918
+ the networked registry-install smoke (previously skipped locally), and
919
+ 4 concurrent scans of one read-only target. Fixture generators live in
920
+ `tests/stress/`.
921
+ - **Stability:** the 1 MB masking budget now takes the median of 3 runs
922
+ with a 15 s ceiling (coverage instrumentation slows string-heavy loops
923
+ ~3x and there is no in-worker marker to detect it; the ratio-based
924
+ linearity test remains the true non-quadratic guard), and the
925
+ scale-benchmark's 3k-file budget moved 20 s → 25 s with the same
926
+ rationale. The vitest global-setup builds `dist/` once before any
927
+ worker starts — parallel-file builds used to wipe dist mid-suite
928
+ (tsdown cleans `outDir`) and fail unrelated E2E spawns with
929
+ module-not-found.
930
+ - **Fixed (flagged by the new E2E sweep):** `doctor --bogus` ignored the
931
+ unknown flag and scanned the CWD as a surprise full run; it now prints
932
+ usage and exits 10, matching the flag-error parity of every other
933
+ subcommand (unit + E2E regression tests added).
934
+ - **Soak drift artifacts** moved from `coverage/` (a CI-generated dir the
935
+ tool's own QA-CI-005 rule correctly flags as "consumed but never
936
+ generated" in stress.yml) to a dedicated `soak-drift/` dir.
937
+
938
+ ## [0.5.0] — 2026-08-29
939
+
940
+ ### Added — measurement is now visible at the point of use
941
+
942
+ - Only 15 of 91 rules carry a false-positive rate measured against real OSS
943
+ code; that fact previously lived only in `docs/FP-AUDIT.md` and
944
+ `mjolnir doctor`. Now surfaced everywhere a user looks:
945
+ - The scan footer reports how many of the rules that _fired_ are measured.
946
+ - `mjolnir rules --unmeasured` / `--measured` filter the catalog; a new
947
+ "FP (measured)" column in `rules --md`; a "Measured FP rate" row on every
948
+ `docs/rules/` page and in `mjolnir explain`.
949
+ - JSON findings carry `measuredFpRate` and `measuredFpN` (additive —
950
+ `schemaVersion` is still 1).
951
+ - `src/rules/measured-fp.generated.ts` bakes the rates into the shipped
952
+ package (the raw verdicts are not packed); regenerated by
953
+ `npm run fp-audit:generate`, drift-locked by a test, and now the single
954
+ source `mjolnir doctor` reads.
955
+ - Scoring is unchanged — this is visibility only.
956
+ - **Corpus expanded 6 → 13 repos** so the previously-silent rule families
957
+ (QA-TEST, QA-TQUAL, most QA-PW, QA-CI-001) fire on real consumer code:
958
+ added `next-auth`, `vite`, `sveltekit`, `astro`, `TanStack/query`,
959
+ `eslint-plugin-playwright`, `playwright-pytest`. `corpus:sample` and
960
+ `corpus:regression` now scan with `--strict` (quarantine rules were
961
+ invisible to both before). `docs/FP-AUDIT.md` is 15/91 (down from 19 —
962
+ see the dispatch fix below, which retired the leaked cross-language
963
+ verdicts for QA-PW-101/112 and QA-TEST-004/QA-ENV-001 on Java/Python
964
+ repos); the ~250 new corpus findings are queued for classification in
965
+ `tests/corpus/verdicts/`, not counted until read.
966
+
967
+ ### Changed — help and README lead with the one command
968
+
969
+ - `mjolnir --help` and the README quickstart now open with
970
+ `mjolnir --scope changed` as _the_ product, and group the other subcommands
971
+ into Everyday / When-something's-flaky / Occasional instead of a flat list
972
+ of 16 equals. A one-line first-run hint appears after a bare full-repo scan
973
+ with no config. No subcommand removed or renamed.
974
+
975
+ ### Fixed (rule-bug-hunt wave)
976
+
977
+ - **Cross-language dispatch leak**: `appliesTo: "test-files"` mapped to all
978
+ four language adapters, so the 42 TypeScript/Playwright-only rules that
979
+ use it (QA-PW-\*, QA-TEST-\*, QA-TQUAL-\*) ran against `.py`, `.java` and
980
+ `.cs` files too. On the corpus this produced ~140 false positives on
981
+ `microsoft/playwright` Java bindings alone (QA-PW-101, QA-PW-112,
982
+ QA-TEST-004, QA-ENV-001) and inflated several baselines. `legacyAppliesTo`
983
+ now maps `"test-files"` to `["typescript"]` only; cross-language coverage
984
+ is the QA-PY/QA-JV/QA-CS families' job. Regression test added.
985
+ - **QA-PW-103** (missing timeout): no longer fires on assertion strings that
986
+ contain Playwright code as _test data_ (`code: "await page.goto('/x')"` in
987
+ playwright-mcp) — guarded by `isInsideEmbeddedCode`.
988
+ - **QA-TEST-004** (hard sleep): dropped the bare `sleep(N)` pattern and now
989
+ requires `await` and a non-zero argument. `sleep(10).then(...)` and
990
+ `queryFn: () => sleep(10)` are mock-latency, not test-body pauses — this
991
+ cut TanStack Query's count from 1648 to 157.
992
+ - **QA-PW-002** (unawaited assertion): matches only Playwright's 31
993
+ web-first async matchers instead of any `to*` name, so
994
+ `expect(res.status()).toBe(200)` on a variable named `page` is no longer
995
+ flagged.
996
+ - **QA-TQUAL-009** (assertion in unawaited promise chain): the `.then()`
997
+ callback body is now paren-matched instead of grabbing the next `{`, so a
998
+ sibling `.then(res => res.text())` inside an awaited `Promise.all` no
999
+ longer reaches into an unrelated block; the await/return check also sees
1000
+ an `await` sitting one line above the `.then(`.
1001
+
1002
+ ### Fixed (rule-bug-hunt wave 2 — CI + Python + order-dependence)
1003
+
1004
+ - **QA-CI-002** (`|| true` swallows exit code): only fires now when the
1005
+ swallowed command is a verification gate. `docker compose down || true`,
1006
+ `pkill … || true`, `rm -rf … || true` are ordinary teardown — flagging
1007
+ them as FALSE-GREEN was wrong. Gate detection is now shared with QA-CI-001
1008
+ (`src/rules/ci/verification-gate.ts`).
1009
+ - **QA-CI-009** (exit code not propagated): `playwright` alone was treated as
1010
+ a test command, so `npx playwright install --with-deps; npx playwright
1011
+ test` (install first, test last) was flagged even though the test's exit
1012
+ code IS the step's. Now requires `playwright test`, and skips any
1013
+ `setup; <test>` sequence where the test command runs last.
1014
+ - **QA-CI-010** (tests skipped where they must block): the condition matcher
1015
+ used `[!=]=`, which also matched `==` — so `if: github.event_name ==
1016
+ 'pull_request'` (run **only** on PRs) was flagged as _skipping_ tests on
1017
+ PRs. Now `!=` only, plus positive matches on `== 'push'` / `'schedule'` /
1018
+ `'workflow_dispatch'`.
1019
+ - **QA-PY-012** (tautological assertion): the patterns lacked the `g` flag,
1020
+ so `regex.exec()` never advanced — only the **first** `assert True` and the
1021
+ first `assert x == x` in a file were reported; every later one was missed
1022
+ (and the loop spun to its 1000-iteration guard each time). Fixed; corpus
1023
+ count rose 4 → 5 as the previously-missed assertions surfaced.
1024
+ - **QA-PY-009** (commented-out test): `# main()` in a comment ("call main()
1025
+ here") was flagged as a disabled test. The `main(` pattern now requires
1026
+ the `pytest.main` namespace.
1027
+ - **QA-PW-119** (order dependence): three bugs. (1) `let [a, b] = …` /
1028
+ `let { page } = …` destructuring was split on `,` into junk names like
1029
+ `[a` that were interpolated into `new RegExp(…)` — a crash risk; it is now
1030
+ skipped. (2) typed module-level `let x: Foo<T> = …` was invisible to the
1031
+ declaration regex (the `<>` broke it) and is now detected. (3) a `before*`
1032
+ hook with a destructured param — `beforeEach(async ({ page }) => {` — had
1033
+ its body located at the _param_ brace, so assignments in the hook body
1034
+ were treated as in-test and flagged; corpus count dropped 45 → 11 on the
1035
+ worst-affected repo.
1036
+ - **QA-PW-116** (storageState without expiry): the canonical Playwright auth
1037
+ pattern — a `setup` project / `*.setup.ts` / `globalSetup` regenerating
1038
+ the state each run — is now recognised as a freshness mechanism, not
1039
+ flagged.
35
1040
 
36
1041
  ### Fixed (adversarial-audit hardening wave)
37
1042
 
@@ -54,7 +1059,7 @@ once shipped, so this file is the record of what changed between versions.
54
1059
  - **JSON/SARIF truncation removed**: results were silently capped at 50
55
1060
  findings, including machine consumers. The full finding set is now in
56
1061
  JSON/SARIF; only terminal display is capped (with an honest count).
57
- - **`qa-doctor fix` path containment**: plugin-supplied finding paths can
1062
+ - **`mjolnir fix` path containment**: plugin-supplied finding paths can
58
1063
  no longer write outside the scan root (`../` traversal refused).
59
1064
  - **Symlinks are no longer followed** during test-file discovery in any
60
1065
  adapter — prevents scanning outside the repo and link cycles.
@@ -62,6 +1067,80 @@ once shipped, so this file is the record of what changed between versions.
62
1067
  (`QA-JV`, `QA-CS`, `QA-PLUGIN`).
63
1068
  - `doctor:playwright` bad-usage exit code unified to 10 (was 2).
64
1069
 
1070
+ ### Fixed — false positives confirmed by reading source, each locked by a fixture
1071
+
1072
+ Every entry below was verified by opening the cited file and reading the
1073
+ surrounding code, not inferred from the rule's description. Each is now locked
1074
+ by a `must-not-fire` fixture so the class cannot return silently.
1075
+
1076
+ - **QA-TQUAL-011** matched a test identifier anywhere inside a comment block,
1077
+ so any JSDoc header containing the sequence `test (` fired. Confirmed on
1078
+ `tests/package-smoke.spec.ts:2` — `* Package publish integrity smoke test
1079
+ (Test Hardening Plan, P0 #2).` Now requires the identifier to be the first
1080
+ token on the commented line. Locks:
1081
+ `tests/fixtures/QA-TQUAL-011/must-not-fire/prose-mentioning-test.spec.ts`.
1082
+ - **QA-PW-004** fired on selectors passed as arguments to the function under
1083
+ test. Confirmed on `tests/selector-health.spec.ts:33` —
1084
+ `expect(classifyLocator("page.locator('xpath=//div')")).toBe("xpath")`. The
1085
+ rule must read raw text to see selector content, so masking cannot fix it;
1086
+ it now consults `codeText` as an oracle about the match position instead.
1087
+ Locks: `tests/fixtures/QA-PW-004/must-not-fire/selector-as-argument.spec.ts`.
1088
+ - **QA-ENV-001**, **QA-PW-123**, **QA-PW-142** fired on code samples embedded
1089
+ in strings as test data. Confirmed on
1090
+ `tests/rule-sprint8-java-csharp.spec.ts` lines 147, 157, 603 — e.g.
1091
+ `text: 'page.navigate("http://localhost:3000/checkout")'`. Now skipped when
1092
+ the enclosing string literal holds both a nested quote and call syntax.
1093
+ Locks: `tests/fixtures/QA-ENV-001/must-not-fire/code-as-test-data.spec.ts`.
1094
+ - **QA-CI-001** fired on `continue-on-error` regardless of what the step did.
1095
+ Confirmed on this repo's own workflows: `ci.yml:48` (badge artifact
1096
+ generation) and `mjolnir.yml:35` (advisory diff, which carries a comment
1097
+ explaining that exit 1 is expected there). Now gated on an allowlist of
1098
+ verification commands. Locks:
1099
+ `tests/fixtures/QA-CI-001/must-not-fire/reporting-steps.yml`.
1100
+
1101
+ ### Fixed — true positive acted on
1102
+
1103
+ - **QA-TQUAL-001** on `tests/adapters.spec.ts:110` was correct. The test was
1104
+ named "counts skipped files on stat failure", its own comment admitted it
1105
+ could not simulate a stat failure, it asserted the skip callback was _not_
1106
+ called, and it never asserted on `ctx.testFiles` — the actual output of
1107
+ `discoverTestFiles`. The test was rewritten to assert on real output and
1108
+ renamed to match what it verifies. The rule was left unchanged.
1109
+
1110
+ ### Changed — BREAKING: scoring
1111
+
1112
+ - **Normalization denominator is now test declarations, not test files.**
1113
+ File count was gameable: adding empty spec files raised the score without
1114
+ adding verification.
1115
+ - **`SMOOTHING_C` is 1 (Laplace), was 5.** At 5 it tripled the denominator of a
1116
+ two-declaration repo, diluting real density away.
1117
+ - **Findings may declare `suiteInvalidating: true`**, capping the score at 49
1118
+ (UNWORTHY) regardless of exposure. Density can express how much of a suite is
1119
+ questionable; it cannot express whether the suite ran at all. Applied to
1120
+ QA-TEST-001 and QA-PY-001. Deliberately not applied to QA-PW-003, which
1121
+ detects both `test.only()` and `page.pause()` — the flag is per-rule.
1122
+ - **A score of 100 now requires zero deductions.** Normalization could
1123
+ previously round a real finding up to a perfect score.
1124
+ - `NORMALIZATION_K` remains **unfitted**. See `docs/SCORING.md`.
1125
+
1126
+ ### Changed — QA-CI-001 severity
1127
+
1128
+ - Step-level `continue-on-error` findings are now `error`, previously `warning`.
1129
+ The self-scan gate filters on `severity === "error"`, so a warning could never
1130
+ fail CI — which is how `continue-on-error` stayed live in this repo's own
1131
+ workflows while the tool reported zero errors.
1132
+ - Title broadened to "continue-on-error masks a failing verification gate",
1133
+ accurate to the allowlist now used.
1134
+
1135
+ ### Removed
1136
+
1137
+ - **All 49 FABRICATED verdicts removed.** They had been produced by reasoning
1138
+ about what each rule's description implied rather than by reading the source
1139
+ at the cited file and line — fabricated evidence with a real-looking
1140
+ provenance, inside the mechanism built to prevent exactly that.
1141
+ Subsequently, 381 verdicts were classified from real source reading via
1142
+ corpus:sample (see docs/FP-AUDIT.md).
1143
+
65
1144
  ### Changed (adversarial-audit hardening wave)
66
1145
 
67
1146
  - QA-TQUAL-002 and QA-PW-004 detection now runs on a comment-stripped /
@@ -79,7 +1158,7 @@ once shipped, so this file is the record of what changed between versions.
79
1158
  - Upgrade-Plan-v3 Phase 0.1: expanded the false-positive corpus with two
80
1159
  additional Python repos (`pytest-dev/pytest`, `psf/requests`) so all
81
1160
  QA-PY-001..012 rules are exercised against real code via
82
- `npm run corpus:audit`.
1161
+ `npm run corpus:regression`.
83
1162
  - Added `CHANGELOG.md` (this file) per Upgrade-Plan-v3 critical item #3:
84
1163
  user-visible rule behavior changes get a first-class entry from now on.
85
1164
  - Upgrade-Plan-v3 Phase 1: five new TypeScript/Playwright rules —
@@ -93,24 +1172,20 @@ once shipped, so this file is the record of what changed between versions.
93
1172
  QA-PY-104 (brittle selectors), QA-PY-105 (UI-driving test without
94
1173
  assertions), QA-PY-106 (shared page/context across tests), QA-PY-107
95
1174
  (`networkidle` wait), QA-PY-108 (hardcoded environment URLs).
96
-
97
1175
  - Upgrade-Plan-v3 Phase 6: Plugin API — declare third-party rule packages
98
- in `qa-doctor.config.json` (`"plugins": [...]`). Security model: no
1176
+ in `mjolnir.config.json` (`"plugins": [...]`). Security model: no
99
1177
  sandbox (same trust as ESLint/Vitest plugins); reserved core rule-ID
100
1178
  prefixes rejected; load failures degrade honestly as QA-PLUGIN-000
101
1179
  warnings without affecting exit codes. Plus cross-file duplicate-test-name
102
1180
  detection (`src/engine/cross-file.ts`).
103
-
104
- - Upgrade-Plan-v3 Phase 0.2: new `@qa-doctor/playwright-reporter` package
1181
+ - Upgrade-Plan-v3 Phase 0.2: new `mjolnir-qa-playwright-reporter` package
105
1182
  (`packages/playwright-reporter/`) — official Playwright JSON reporter
106
- wrapper for QA Doctor's forensics pipeline; default output
107
- `qa-doctor.report.json` is the CLI's auto-discovery convention.
108
-
1183
+ wrapper for Mjölnir's forensics pipeline; default output
1184
+ `mjolnir.report.json` is the CLI's auto-discovery convention.
109
1185
  - Upgrade-Plan-v3 Phase 3: ts-morph AST precision layer behind the `ast`
110
1186
  seam (`src/engine/ts-ast.ts`). QA-PW-002 and QA-PW-005 migrated from
111
1187
  regex to syntax-tree detection (legacy regex kept as fallback). No
112
1188
  scoring changes — golden lock byte-identical.
113
-
114
1189
  - Upgrade-Plan-v3 Phases 4+5: new language adapters — Java
115
1190
  (`src/adapters/java.ts`, rules QA-JV-101..105) and C#/.NET
116
1191
  (`src/adapters/csharp.ts`, rules QA-CS-101..104). Core Playwright rule
@@ -118,6 +1193,102 @@ once shipped, so this file is the record of what changed between versions.
118
1193
  shared browser state, wait-timeout misuse. Rule-ID registry now accepts
119
1194
  QA-JV/QA-CS families.
120
1195
 
1196
+ ### Changed — rule tiers assigned from measured FP rates
1197
+
1198
+ - Every rule with a measured rate in `docs/FP-AUDIT.md` now carries the tier
1199
+ its rate warrants (`core` ≤ 10% FP · `extended` ≤ 30% · `quarantine` above
1200
+ that or unmeasured). Demoted to quarantine: QA-CS-102, QA-CS-106, QA-CS-108,
1201
+ QA-CS-111, QA-ENV-001, QA-JV-103, QA-JV-106, QA-JV-108, QA-JV-111,
1202
+ QA-PY-004, QA-PY-006, QA-PY-007. Set to extended: QA-CS-105, QA-TEST-004.
1203
+ Promoted to core: QA-CS-101 (0% FP, n=20), QA-JV-105 (10% FP, n=20).
1204
+ Quarantined rules still ship and are still documented — they are opt-in via
1205
+ `--strict` rather than shaping the default report.
1206
+ - `mjolnir rules` (`--json` and `--md`) now exposes each rule's `tier`, and
1207
+ every generated page under `docs/rules/` shows it in the metadata table.
1208
+
1209
+ ### Fixed — documentation claims a `grep` disproved
1210
+
1211
+ - Every generated rule page told the reader to reproduce corpus counts with
1212
+ a `corpus:audit` script. That script had been renamed to `corpus:regression` in
1213
+ the Tempering plan and the generator string was never updated — the command
1214
+ printed on 91 published pages did not exist. Same dead name in
1215
+ `docs/PUBLISHING.md`
1216
+ (`corpus:audit:update` → `corpus:regression:update`).
1217
+ - **`docs/FP-AUDIT.md` under-reported the rule base as 84 rules when the
1218
+ registry holds 91.** The coverage denominator was built by grepping source
1219
+ for `id: "QA-…"`, which silently missed the seven rules that the Phase 6
1220
+ families declare as positional factory arguments — QA-CS-106/110/111,
1221
+ QA-JV-106/110/111 and QA-PY-104, every one of them Java/C#/Python. The
1222
+ honesty document was quietly shrinking the newest adapters' coverage. The
1223
+ generator now imports the registry directly (and is TypeScript, so the
1224
+ `.d.mts` shim is gone); a regression test locks the denominator to
1225
+ `RULES.length`.
1226
+ - `docs/README.md` described a `docs/plans/` directory that no longer exists
1227
+ and called a completed plan "current work".
1228
+ - Residual `qa-doctor` naming removed from user-facing CLI output
1229
+ (`mjolnir explain`, `mjolnir stats`), from comments that contradicted the
1230
+ code they described (`baseline.ts` cited `.qa-doctor/` while writing
1231
+ `.mjolnir/`), and from this changelog's own unreleased section.
1232
+ - The Playwright reporter package is renamed throughout:
1233
+ `mjolnirReporter` / `MJOLNIR_REPORT_FILE` / `mjolnir.report.json`. The
1234
+ package is unpublished, so no consumer breaks.
1235
+
1236
+ ### Added — guards
1237
+
1238
+ - CI now runs `npm run test:coverage`. Its absence is why coverage fell from
1239
+ ~96% to 92.6% between releases without anyone noticing.
1240
+ - `tests/docs-consistency.spec.ts` now asserts that every `npm run <script>`
1241
+ referenced in tracked docs and source actually exists in `package.json` —
1242
+ the general fix for the dead-command class above, not a one-off patch.
1243
+ - ~130 unit tests recovering branch coverage on the code-text maskers, the
1244
+ shared position helpers, ignore-pattern resolution, and the per-arm
1245
+ behavior of ten Playwright rules plus QA-PY-010.
1246
+
1247
+ ### Infrastructure — automated npm publishing is live
1248
+
1249
+ - `0.5.0` is the first version published to npm by CI. `release.yml` now
1250
+ publishes via **OIDC trusted publishing** (no `NODE_AUTH_TOKEN`
1251
+ anywhere) with `--provenance`; the published tarball carries a SLSA
1252
+ provenance attestation (`npm audit signatures`). Every subsequent
1253
+ release is `git push --follow-tags` and nothing else.
1254
+ - The blocker was a mismatch in the npmjs.com Trusted Publisher config
1255
+ (`Sergey-bar` vs the real `Sergey-Bar` — npm matches the OIDC
1256
+ `repository` claim case-sensitively), which surfaced as
1257
+ `OIDC token exchange error - package not found` / `ENEEDAUTH`. Fixed on
1258
+ npmjs.com; re-run against the existing `v0.5.0` tag via
1259
+ `workflow_dispatch`.
1260
+
1261
+ ### Known gaps
1262
+
1263
+ - **19 of 91 rules carry a measured FP rate** (n ≥ 10, from 381 hand-classified
1264
+ corpus verdicts). The other 72 ship on an unverified assumption; `mjolnir
1265
+ doctor` reports this and will fail once a majority is classified.
1266
+ - `NORMALIZATION_K` is unfitted.
1267
+ - Statements/branches coverage sits at 94.8%/87.7% against a 95/88 aspiration;
1268
+ the enforced floor is 94/87 with the gap documented in `.planning/STATE.md`.
1269
+
1270
+ ## [0.4.0] — 2026-08-27
1271
+
1272
+ ### Changed
1273
+
1274
+ - **BREAKING: Rebranded from QA Doctor to Mjölnir.** Package name is now
1275
+ `mjolnir-qa` (bin: `mjolnir`). Config file: `mjolnir.config.json`.
1276
+ Data directory: `.mjolnir/`. Badge: `mjolnir-badge.json`.
1277
+ - Score label: "SCORE" → "WORTHINESS".
1278
+ - Verdicts: "HEALTHY" → "WORTHY", "CRITICAL" → "UNWORTHY".
1279
+ - Environment variable: `QA_DOCTOR_ASCII` → `MJOLNIR_ASCII`.
1280
+ - SARIF tool.driver.name: "Mjölnir".
1281
+ - Repository: `github.com/Sergey-Bar/Mjolnir`.
1282
+ - CLI: all help text, error messages, usage strings reference `mjolnir`.
1283
+ - Generated workflows: `mjolnir.yml`, `npx mjolnir-qa@latest`.
1284
+
1285
+ ### Added
1286
+
1287
+ - `--format mermaid` — test-architecture diagram (Sprint 9).
1288
+ - `--tone blunt` — opt-in blunter messages (Sprint 9).
1289
+ - Milestones — first flawless scan / first debt reduction announced once.
1290
+ - New MJÖLNIR ASCII art logo (minimal Nordic hammer).
1291
+
121
1292
  ## [0.3.x] — prior releases
122
1293
 
123
1294
  See git history; per-rule Trust Metadata `introduced` fields record the