mjolnir-qa 0.4.0 → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +1199 -28
- package/README.ar.md +667 -0
- package/README.bn.md +678 -0
- package/README.br.md +706 -0
- package/README.bs.md +690 -0
- package/README.da.md +698 -0
- package/README.de.md +711 -0
- package/README.es.md +709 -0
- package/README.fr.md +714 -0
- package/README.gr.md +705 -0
- package/README.he.md +664 -0
- package/README.it.md +709 -0
- package/README.ja.md +692 -0
- package/README.ko.md +681 -0
- package/README.md +425 -152
- package/README.no.md +696 -0
- package/README.pl.md +699 -0
- package/README.ru.md +701 -0
- package/README.th.md +672 -0
- package/README.tr.md +699 -0
- package/README.uk.md +693 -0
- package/README.vi.md +680 -0
- package/README.zh.md +652 -0
- package/README.zht.md +652 -0
- package/dist/cli.d.mts +469 -22
- package/dist/cli.mjs +9780 -5230
- package/package.json +25 -7
package/CHANGELOG.md
CHANGED
|
@@ -9,29 +9,1034 @@ Rule behavior changes (new rules, FP-rate changes against the corpus,
|
|
|
9
9
|
severity changes) are first-class entries here — rule IDs are immutable
|
|
10
10
|
once shipped, so this file is the record of what changed between versions.
|
|
11
11
|
|
|
12
|
-
## [0.
|
|
12
|
+
## [0.5.2] — 2026-09-05
|
|
13
13
|
|
|
14
|
-
###
|
|
14
|
+
### npm 12 pack-shape repair of the release pipeline
|
|
15
15
|
|
|
16
|
-
|
|
17
|
-
`mjolnir-qa` (bin: `mjolnir`). Config file: `mjolnir.config.json`.
|
|
18
|
-
Data directory: `.mjolnir/`. Badge: `mjolnir-badge.json`.
|
|
19
|
-
- Score label: "SCORE" → "WORTHINESS".
|
|
20
|
-
- Verdicts: "HEALTHY" → "WORTHY", "CRITICAL" → "UNWORTHY".
|
|
21
|
-
- Environment variable: `QA_DOCTOR_ASCII` → `MJOLNIR_ASCII`.
|
|
22
|
-
- SARIF tool.driver.name: "Mjölnir".
|
|
23
|
-
- Repository: `github.com/Sergey-Bar/Mjolnir`.
|
|
24
|
-
- CLI: all help text, error messages, usage strings reference `mjolnir`.
|
|
25
|
-
- Generated workflows: `mjolnir.yml`, `npx mjolnir-qa@latest`.
|
|
16
|
+
### Fixed
|
|
26
17
|
|
|
27
|
-
|
|
18
|
+
- The fresh-install gate (`tests/integrations/registry-install.spec.ts`),
|
|
19
|
+
`tests/integrations/package-smoke.spec.ts`, and
|
|
20
|
+
`tests/e2e/journey-1-first-run.spec.ts` parse `npm pack --json` through
|
|
21
|
+
a shared shape-tolerant helper
|
|
22
|
+
(`tests/helpers/npm-pack-json.ts`): npm 12 changed the output from an
|
|
23
|
+
array to an object keyed by package name, which stopped the v0.5.1
|
|
24
|
+
publish at the gate — tag cut, nothing shipped, by design.
|
|
25
|
+
- `release.yml` upgrades to `npm@11` (the proven line) instead of
|
|
26
|
+
`npm@latest`: toolchain majors must be deliberate, verified changes,
|
|
27
|
+
never implicit drift on the publish path.
|
|
28
|
+
- `registry-install.spec.ts` cleanup no longer cascades a second error
|
|
29
|
+
when `beforeAll` fails early (the cascade buried the real diagnosis).
|
|
28
30
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
31
|
+
## [0.5.1] — 2026-09-05
|
|
32
|
+
|
|
33
|
+
### Verification Trust Evolution, Phase 8 — Local Extensibility (plan §18)
|
|
34
|
+
|
|
35
|
+
### Added — folder-based external rules, zero network
|
|
36
|
+
|
|
37
|
+
- **`mjolnir-rules/` contract** (`src/plugins/local-rules.ts`): a
|
|
38
|
+
workspace directory loaded from the scan target root alongside npm
|
|
39
|
+
plugins. Two file kinds: **JSON rule manifests** (declarative regex
|
|
40
|
+
patterns — NO code executed; id/title/severity/category/appliesTo/
|
|
41
|
+
patterns/message/why/fix/languages/frameworks) and **JS modules**
|
|
42
|
+
(`rules: QADoctorRule[]`, full-Node trust, same posture as npm
|
|
43
|
+
plugins). Missing directory → no-op.
|
|
44
|
+
- **Same trust contract as core/npm plugins:** reserved core prefixes
|
|
45
|
+
rejected (case-insensitive spoofing guard); bad metadata/regexes
|
|
46
|
+
degrade to warning entries (QA-PLUGIN-000), never a crash; external
|
|
47
|
+
rules carry the full trust metadata shape and are born
|
|
48
|
+
quarantine/unmeasured. **Core-tier clamp**: an external rule
|
|
49
|
+
declaring `tier: "core"` is clamped to `extended` with a load
|
|
50
|
+
warning — core requires a measured FP rate from the committed corpus
|
|
51
|
+
sidecar, which external rules cannot have.
|
|
52
|
+
- **Tier caps obeyed, filter unified:** the quarantine exclusion filter
|
|
53
|
+
in `buildUniversalRules` now consults the tier map that includes
|
|
54
|
+
plugin/external tiers — plugin-declared quarantine rules were
|
|
55
|
+
previously excluded only when the core registry knew the ID; the
|
|
56
|
+
unified filter excludes them from non-strict scans exactly like core
|
|
57
|
+
(post-scan cap still observable under `--strict`; the cli-scan-arms
|
|
58
|
+
test updated to cover both sides).
|
|
59
|
+
- **Drift-checked:** `mjolnir rules --md --external` renders the
|
|
60
|
+
catalog from the LOADED external rules with a provenance column
|
|
61
|
+
(`core`/`external`) — an on-disk edit changes the next render; the
|
|
62
|
+
catalog can never drift from what actually ships.
|
|
63
|
+
`scripts/generate-capability-matrix.ts --external <root>` writes a
|
|
64
|
+
workspace-local `MJOLNIR-RULES-MATRIX.md` with provenance "external"
|
|
65
|
+
(unmeasured by definition — outside the corpus sidecar). The
|
|
66
|
+
committed matrix stays core-registry-only and byte-stable.
|
|
67
|
+
- **S-8 disclosure:** external rule surfaces appear in the scan's
|
|
68
|
+
plugin disclosure block.
|
|
69
|
+
|
|
70
|
+
Registry/marketplace explicitly deferred (plan §18).
|
|
71
|
+
|
|
72
|
+
### Verification Trust Evolution, Phase 7 — Agentic QA Trust (plan §17)
|
|
73
|
+
|
|
74
|
+
### Added — Agentic Trust Profile (plan §17.2, §17.4)
|
|
75
|
+
|
|
76
|
+
- **Provenance detection** (`src/engine/provenance.ts`): per-file static
|
|
77
|
+
markers, EXTRACTED not invented — the industry-wide generated-file
|
|
78
|
+
header convention ("auto-generated"/"generated by"/"do not edit") and
|
|
79
|
+
the Playwright codegen recorder fingerprint (its default test title
|
|
80
|
+
is exactly `'test'`). Honest boundary, documented in the module:
|
|
81
|
+
healed and MCP-edited files WITHOUT markers are statically
|
|
82
|
+
indistinguishable from hand-written ones — the classifier returns
|
|
83
|
+
`unmarked`, never a guess.
|
|
84
|
+
- **`ScanResult.agenticProfile`** (additive): share of test files with
|
|
85
|
+
detected generative markers + the findings split across those
|
|
86
|
+
surfaces, with the boundary in the profile's own `note`. Surfaced in
|
|
87
|
+
the terminal report only when something was actually detected
|
|
88
|
+
(silence over noise); JSON carries it on every scan.
|
|
89
|
+
- **§17.4 — same evidence standard regardless of author:** the profile
|
|
90
|
+
is metadata only. It never changes scoring, evidence levels, tier
|
|
91
|
+
caps, or rule behavior. AI tests earn trust through exactly the same
|
|
92
|
+
evidence as human ones.
|
|
93
|
+
|
|
94
|
+
### Added — locator.normalize() framework-standards rules (plan §17.3)
|
|
95
|
+
|
|
96
|
+
Two new frozen `QA-PW-*` rules, BORN QUARANTINE (§17 exit gate:
|
|
97
|
+
measured before leaving provisional), fixtures both directions:
|
|
98
|
+
|
|
99
|
+
- **QA-PW-146 — CSS/XPath string selector instead of a normalized
|
|
100
|
+
locator** (warning, heuristic): `css=`/`xpath=` engine prefixes, bare
|
|
101
|
+
id/class/attr CSS, and `nth-child` chains inside
|
|
102
|
+
`.locator()`/`waitForSelector()`/`page.$` — aligned with Playwright's
|
|
103
|
+
locator standard. Fills the TS gap (the brittle-selectors family
|
|
104
|
+
covers Java/C#/Python; the retired QA-PW-112 was style police, this
|
|
105
|
+
is markup-coupling detection). Not auto-fixable: the normalized
|
|
106
|
+
getter depends on app semantics only a human knows — the fix field
|
|
107
|
+
carries the concrete `getByRole`/`getByTestId`/`getByText`
|
|
108
|
+
suggestion. Both fixtures + all three cached OSS corpora (0 fires —
|
|
109
|
+
real suites use normalized locators) recorded.
|
|
110
|
+
- **QA-PW-147 — codegen default test title** (info, observation): a
|
|
111
|
+
committed spec still titled `'test'`/`'test N'` is an unreviewed
|
|
112
|
+
recording artifact — the §17.1 provenance marker as a finding.
|
|
113
|
+
|
|
114
|
+
Registry: 99 rules (73 measured). Docs pages + capability matrix
|
|
115
|
+
regenerated.
|
|
116
|
+
|
|
117
|
+
### Verification Trust Evolution, Phase 6 — Runtime Evidence (plan §16)
|
|
118
|
+
|
|
119
|
+
### Added — runtime corroboration + the honest L0–L5 trust ladder
|
|
120
|
+
|
|
121
|
+
Built on the existing forensics ingestion (no greenfield):
|
|
122
|
+
`packages/playwright-reporter` → `mjolnir.report.json` → the
|
|
123
|
+
`ForensicsReport` pipeline that `forensics`/`triage`/`pw-report`
|
|
124
|
+
already consume. Findings gain two additive, optional fields
|
|
125
|
+
(schemaVersion 1 unchanged):
|
|
126
|
+
|
|
127
|
+
- **`runtimeCorroboration`** — what a real run report vouches for:
|
|
128
|
+
`level: "file" | "test" | "defect"`, the report source, executed-test
|
|
129
|
+
count, and (test/defect level) the containing test's full verdict
|
|
130
|
+
(attempts, final status, passed-on-retry, ever-failed, skipped).
|
|
131
|
+
- **`trustLevel`** — the L0–L5 ladder, derived deterministically:
|
|
132
|
+
L0 (E0 observation) / L1 (E1 heuristic) / L2 (E2 deterministic) are
|
|
133
|
+
the static-only ceiling; L3 (file executed) / L4 (the containing
|
|
134
|
+
test executed — matched via the report's spec declaration lines) /
|
|
135
|
+
L5 (the run verdict directly corroborates the defect class — a
|
|
136
|
+
FLAKY-RISK finding whose test actually flaked, retried, or timed
|
|
137
|
+
out) exist ONLY when runtime corroboration is present. The
|
|
138
|
+
no-static-only-L4/L5 invariant is structurally enforced in
|
|
139
|
+
`deriveTrustLevel` and locked by tests across the full
|
|
140
|
+
findingType×confidence×evidenceLevel matrix.
|
|
141
|
+
|
|
142
|
+
- **Matching is honest by construction:** Playwright JSON reports now
|
|
143
|
+
carry the spec declaration line (additive `line` on
|
|
144
|
+
`TestRecord`/`TestVerdict`); a finding is tied to a specific test
|
|
145
|
+
only when the report's declaration spans place it there — otherwise
|
|
146
|
+
corroboration stays at file level, and files the report never ran
|
|
147
|
+
get NOTHING (no fabricated evidence). JUnit XML has no locations:
|
|
148
|
+
file-level only.
|
|
149
|
+
- **Scan wiring:** `runScan` auto-discovers a run report next to the
|
|
150
|
+
scan target (`mjolnir.report.json` — the reporter package's default
|
|
151
|
+
output — or a `test-results/` directory), runs the existing
|
|
152
|
+
forensics ingestion, and stamps findings. No report → findings
|
|
153
|
+
unchanged (honest "runtime evidence: not available"). A hostile
|
|
154
|
+
report degrades the scan, never fails it.
|
|
155
|
+
- **Reporters split verified vs assumed (plan §16):** the terminal
|
|
156
|
+
footer reports `Runtime evidence: N/M findings corroborated by a
|
|
157
|
+
real run report (trust L3–L5)` or the explicit not-available line;
|
|
158
|
+
finding cards show `trust L4 · runtime: test executed`. SARIF
|
|
159
|
+
results carry `trustLevel` + `runtimeCorroboration` in properties.
|
|
160
|
+
|
|
161
|
+
### Verification Trust Evolution, Phase 5 — Framework Expansion (plan §15, D7 closed)
|
|
162
|
+
|
|
163
|
+
### Added — FrameworkDimension enforced (plan §15.1, defect D7 closed)
|
|
164
|
+
|
|
165
|
+
- **Real dependency parsing per build system:** `package.json` (JSON),
|
|
166
|
+
`pom.xml` (Maven `<dependency>` blocks), `build.gradle(.kts)`
|
|
167
|
+
(dependency-statement coordinates), and EVERY `.csproj` at the root
|
|
168
|
+
(`<PackageReference Include="…">` attributes — the old "first
|
|
169
|
+
`.csproj` only" defect is closed), plus requirements*.txt for Python.
|
|
170
|
+
- **Per-file framework tags** (`ParsedFile.frameworkTags`): derived from
|
|
171
|
+
the file's OWN imports/usings/import-lines — `@playwright/test` →
|
|
172
|
+
"playwright", `cypress` → "cypress", `org.junit.*` → "junit",
|
|
173
|
+
`NUnit` → "nunit", `import selenium` → "selenium", etc. AST-truth for
|
|
174
|
+
Java/C#/TS; import-line scan for Python (no AST seam by design).
|
|
175
|
+
- **`rule.frameworks ∩ file.frameworkTags` filtering** with
|
|
176
|
+
open-when-unknown: a rule that declares `frameworks` runs on a file
|
|
177
|
+
only when the file's tags intersect it; files without tags and rules
|
|
178
|
+
without `frameworks` are always analyzed — the dimension narrows, it
|
|
179
|
+
never silently drops evidence. Shared in
|
|
180
|
+
`src/engine/adapter.ts` (`frameworkFilterApplies`), enforced in the
|
|
181
|
+
TS/Java/C#/Python adapters' `runRules`.
|
|
182
|
+
- **Generalized config-gating (§15.2):** `configRule: true` +
|
|
183
|
+
`configFiles: string[]` (regex sources) replaces the hard-coded
|
|
184
|
+
`playwright.config.*` regex that lived in the TS adapter and
|
|
185
|
+
duplicated inside all five config rules (QA-PW-121/122/141/143/144
|
|
186
|
+
migrated). Discovery knows the config filename conventions
|
|
187
|
+
(`cypress.config.*` added alongside `playwright.config.*`).
|
|
188
|
+
|
|
189
|
+
### Added — Cypress integration (first framework per the §15 order)
|
|
190
|
+
|
|
191
|
+
Three rules in the new frozen `QA-CYP-*` namespace, BORN QUARANTINE
|
|
192
|
+
(§15.5) with fixtures both directions (`tests/fixtures/QA-CYP-*/`):
|
|
193
|
+
|
|
194
|
+
- **QA-CYP-001 — fixed `cy.wait(n)`** (warning): numeric-literal
|
|
195
|
+
`cy.wait(3000)` — the Cypress hard-sleep idiom. Alias waits
|
|
196
|
+
(`cy.wait('@route')`) are the legitimate form and never fire.
|
|
197
|
+
File gate: framework tag, `.cy.*` extension, or `cy.*` API usage in
|
|
198
|
+
the file (real Cypress suites rarely import cypress — surfaced by the
|
|
199
|
+
first measurement against cypress-example-kitchensink). MEASURED on
|
|
200
|
+
the kitchensink corpus: 12 TP (viewport-switch fixed waits) / 3 FP
|
|
201
|
+
(doc-example artifacts, the suite intentionally demonstrates the API
|
|
202
|
+
— QA-PY-003 precedent) → 20% FP at n=15, 95% Wilson
|
|
203
|
+
[7.1%, 45.2%]; tier extended (band-consistent), corpus
|
|
204
|
+
cypress-io-kitchensink added with baselines.
|
|
205
|
+
cypress-realworld-app scanned as precision evidence: 0 fires (the
|
|
206
|
+
suite uses alias waits exclusively), baseline recorded.
|
|
207
|
+
- **QA-CYP-002 — focused test (`.only`)** (error): committed
|
|
208
|
+
`it.only`/`describe.only`/`context.only` de-schedules the rest of the
|
|
209
|
+
suite. Quarantine, unmeasured (fixtures only) pending a measured
|
|
210
|
+
Cypress corpus with the pattern.
|
|
211
|
+
- **QA-CYP-003 — `chromeWebSecurity: false`** (error): deterministic
|
|
212
|
+
config defect via the generalized `configFiles` gate. Quarantine,
|
|
213
|
+
unmeasured pending corpus.
|
|
214
|
+
|
|
215
|
+
### Added — Selenium cross-language reach (plan §15.3, JV/CS/Py reuse)
|
|
216
|
+
|
|
217
|
+
Three rules in the new frozen `QA-SE-*` namespace sharing one sequence
|
|
218
|
+
detector (hard sleep followed by an element lookup within 3 lines —
|
|
219
|
+
the sleep standing in for an explicit `WebDriverWait`), all BORN
|
|
220
|
+
QUARANTINE:
|
|
221
|
+
|
|
222
|
+
- **QA-SE-001** (Java): `Thread.sleep` → `findElement`/`click`/
|
|
223
|
+
`sendKeys` within 3 lines.
|
|
224
|
+
- **QA-SE-002** (C#): `Thread.Sleep`/`Task.Delay` →
|
|
225
|
+
`FindElement`/`Click`/`SendKeys` within 3 lines. MEASURED on the
|
|
226
|
+
SeleniumHQ/selenium .NET webdriver suite: 3 TP + 1 FP (a
|
|
227
|
+
sleep-inside-polling-loop cadence — loop-body containment is the
|
|
228
|
+
documented residue) → 25% FP at n=4; below the n≥10 measurement bar,
|
|
229
|
+
stays quarantine/unmeasured with verdicts recorded
|
|
230
|
+
(tests/corpus/verdicts/SeleniumHQ-selenium.jsonl, baseline added).
|
|
231
|
+
- **QA-SE-003** (Python): `time.sleep` → `find_element`/`click`/
|
|
232
|
+
`send_keys` within 3 lines.
|
|
233
|
+
|
|
234
|
+
### Registry
|
|
235
|
+
|
|
236
|
+
- 97 rules (was 91); registry/doctor/scaffolder ID validators widened
|
|
237
|
+
for the new frozen namespaces (QA-CYP-_, QA-SE-_, QA-WDIO-_,
|
|
238
|
+
QA-PPTR-_, QA-APM-* reserved). Measured coverage 73/97 (75%); the
|
|
239
|
+
§20.1(a) unmeasured-count ratchet now tracks the PRE-EXISTING set
|
|
240
|
+
mechanically (`introduced` ≤ 0.5.0) — new waves onboard under §20.1(b)
|
|
241
|
+
- the per-framework exit gate, not the global count freeze.
|
|
242
|
+
- Defect ledger: **D7 closed** (target phase 5). Capability matrix
|
|
243
|
+
regenerated (97 rules); docs pages regenerated; corpus baselines
|
|
244
|
+
added for the three Phase 5 measurement repos.
|
|
245
|
+
|
|
246
|
+
### Verification Trust Evolution, Phase 4 — Common QA Semantic Model (plan §14, behavior-neutral)
|
|
247
|
+
|
|
248
|
+
### Added — `src/engine/qa-model.ts`: the normalized QA concept IR (extract-only, no scan wiring)
|
|
249
|
+
|
|
250
|
+
The plan-§14 vocabulary — Test, TestBoundary, Setup/Teardown, Fixture,
|
|
251
|
+
Action, Locator, Wait, Assertion, Mock, NetworkInteraction, Retry,
|
|
252
|
+
Navigation, Interaction, Lifecycle — now exists as a typed model
|
|
253
|
+
(`QaNode`/`QaSemanticModel`, all 14 concepts in `EXTRACTOR_COVERAGE`)
|
|
254
|
+
with per-language extractors over the ALREADY-existing parse stage:
|
|
255
|
+
ts-morph for TS/JS (tests/hooks via the scorer's it/test vocabulary,
|
|
256
|
+
calls via the measured rule vocabularies of qa-pw-002/004/005/101–145,
|
|
257
|
+
qa-test-001/003/004/006, jest/vi mock+retry), tree-sitter for Java/C#
|
|
258
|
+
(test boundaries REUSE `javaTestMethods`/`csharpTestMethods` verbatim —
|
|
259
|
+
the model cannot drift from the rules' scoping; call/hook/retry
|
|
260
|
+
vocabularies copied from the QA-JV-_/QA-CS-_ rules), regex boundaries
|
|
261
|
+
for Python (def test_, @pytest.fixture, time.sleep, assert).
|
|
262
|
+
|
|
263
|
+
- **Extracted, not invented:** every classification table cites the
|
|
264
|
+
rule whose measured vocabulary it copies (headers in qa-model.ts);
|
|
265
|
+
the shared `isHelperIdiom` helper moved to `jv-cs-ast.ts` so the
|
|
266
|
+
QA-CS-103 rule and the model share ONE implementation.
|
|
267
|
+
- **Adoption is additive:** NOTHING in the scan pipeline imports the
|
|
268
|
+
module — BEHAVIOR-NEUTRAL by construction, golden/corpus locks
|
|
269
|
+
untouched (proven: locks byte-identical with the model present).
|
|
270
|
+
- **Coverage is honest:** `EXTRACTOR_COVERAGE` documents per-language
|
|
271
|
+
gaps (e.g. Java has no fixture/interaction/lifecycle extractor;
|
|
272
|
+
Python extracts only test/fixture/assertion/wait) — visible gaps,
|
|
273
|
+
never silent claims (No False Proof).
|
|
274
|
+
- **Equivalence proven:** `tests/qa-model.spec.ts` re-expresses the
|
|
275
|
+
QA-JV-103 / QA-CS-103 / QA-CS-102 oracles over the model
|
|
276
|
+
(`testVerifies`, ancestor-chain containment via the
|
|
277
|
+
`firstAncestorCallNamed` generalization now carried as
|
|
278
|
+
`node.ancestors`) and asserts FINDING-IDENTICAL results against the
|
|
279
|
+
rules on the committed fixture corpora + synthetic edge shapes —
|
|
280
|
+
the model can carry these rules without changing any output.
|
|
281
|
+
Awaitedness (qa-pw-002's consumption oracle) rides on TS nodes as
|
|
282
|
+
`node.awaited`.
|
|
283
|
+
|
|
284
|
+
### Verification Trust Evolution, Phase 3 — Java/C# semantic upgrade (plan §13)
|
|
285
|
+
|
|
286
|
+
### Changed — three JV/CS rules migrated to L2 tree-sitter analysis (EVIDENCE-BACKED, detectorRevision 2)
|
|
287
|
+
|
|
288
|
+
The Phase 0.5 parse stage's tree-sitter trees (`parseJavaAst`/`parseCSharpAst`,
|
|
289
|
+
delivered via `ParsedFile.ast`) are now consumed by rules: a new L2
|
|
290
|
+
structural-analysis layer (`src/engine/jv-cs-ast.ts`) provides test-method
|
|
291
|
+
scoping by annotation/attribute, invocation structure, and call/argument
|
|
292
|
+
containment; the rule contract gains an optional `astQuery` hook whose
|
|
293
|
+
regex path is a MANDATORY fallback (no AST ⇒ regex, never a second
|
|
294
|
+
detector, plan §13.2). No type/symbol semantics are promised for JV/CS
|
|
295
|
+
(no Roslyn, no classpath) — semantic depth is L2 per plan §13.4.
|
|
296
|
+
|
|
297
|
+
- **QA-JV-103 (test without assertions), 50% FP (n=20) → 25.9% FP
|
|
298
|
+
(n=58), quarantine → extended.** Test boundaries now come from real
|
|
299
|
+
`method_declaration` nodes (any `@Test`/`@org.junit.Test`, argumented
|
|
300
|
+
TestNG forms included). The assertion oracle adds the two measured
|
|
301
|
+
rev-1 FP classes: Playwright's THROWING waits (`waitFor*` except
|
|
302
|
+
`waitForTimeout` — they throw on timeout, so the wait IS the
|
|
303
|
+
verification; 6 rev-1 FPs) and `verify*/check*/assert*` helper
|
|
304
|
+
calls (4 rev-1 FPs). Remaining FP class, documented as the L2
|
|
305
|
+
boundary: assertions behind arbitrarily-named helpers (keycloak's
|
|
306
|
+
`testValidationValid`, playwright-java's `testEnterKey`,
|
|
307
|
+
appsmith's `check`).
|
|
308
|
+
- **QA-CS-103 (test without assertions), 95% FP (n=20) → 0% FP (n=9),
|
|
309
|
+
quarantine → core.** The rev-1 oracle missed the whole Shouldly
|
|
310
|
+
extension family — 17 of the 19 FPs were `ShouldBeOfType<...>(
|
|
311
|
+
...).Message.ShouldBe(...)`-style assertion-rich tests (spectre-console,
|
|
312
|
+
310 count-lock fires → 0). The rev-2 oracle counts Shouldly chains,
|
|
313
|
+
`Assert`/`Should` receivers, the Verify snapshot framework,
|
|
314
|
+
PascalCase `verify*/check*/assert*` helpers (Humanizer's
|
|
315
|
+
`VerifyAnalyzerAsync` — the lowercase-only form was a Java camelCase
|
|
316
|
+
inheritance bug), C# throwing waits (`WaitFor*Async` except
|
|
317
|
+
`WaitForTimeoutAsync`), and conditional `throw new
|
|
318
|
+
*Assertion*Exception` (playwright-dotnet's ConventionTests). Grammar
|
|
319
|
+
error-node guard: a truncated parse never produces a finding.
|
|
320
|
+
- **QA-CS-102 (`Thread.Sleep`/`Task.Delay` hard sleep), 65% FP (n=20) →
|
|
321
|
+
8.3% FP (n=24), quarantine → core.** Tree-sitter invocation scoping
|
|
322
|
+
excludes the measured environment-simulation classes: delays inside
|
|
323
|
+
route/expose/server delegates (`Route*Async`, `SetRoute`,
|
|
324
|
+
`ExposeFunction*`, 10 rev-1 FPs + delta), deliberate infinite/negative
|
|
325
|
+
blocks (`Task.Delay(-1)`, `int.MaxValue`, `Timeout.Infinite*`),
|
|
326
|
+
`Task.WhenAny` timeout races, and runner payload fixtures
|
|
327
|
+
(`RunAndWaitFor{Request,RequestFinished,Response}Async(() => Task.Delay…)`,
|
|
328
|
+
`UnrouteAllAsync` timing-window sleeps stay flagged — their delay
|
|
329
|
+
creates the timing window the assertion measures). Documented
|
|
330
|
+
trade-off: one rev-1 TP (a sub-second artificial-timing delay inside a
|
|
331
|
+
route delegate) is no longer flagged; the structural boundary cannot
|
|
332
|
+
read that intent.
|
|
333
|
+
|
|
334
|
+
All three: verdict corpus reconciled per the §07 loop (41 superseded rows
|
|
335
|
+
removed, 49 fresh rows adjudicated from source with per-row notes, class-B
|
|
336
|
+
positive fixtures for QA-CS-102/QA-JV-103 and class-C negative fixtures
|
|
337
|
+
for all three), count-lock baselines refreshed (microsoft-playwright-java
|
|
338
|
+
QA-JV-103 97→43, microsoft-playwright-dotnet QA-CS-102 52→21,
|
|
339
|
+
spectre-console QA-CS-103 310→0, Humanizer 34→2; keycloak +602 is a
|
|
340
|
+
recall gain — the rev-1 regex missed `void x() throws Exception`
|
|
341
|
+
signatures, tree-sitter scoping does not), and
|
|
342
|
+
`detector-revisions.json` bumped (sidecar entries stay; measurements
|
|
343
|
+
stamped rev 2). QA-JV-102 stays LEXICAL rev 1 — no migration for
|
|
344
|
+
symmetry (plan §12.4).
|
|
345
|
+
|
|
346
|
+
### Verification Trust Evolution, Phase 2 — quarantine-cluster triage (plan §12.2)
|
|
347
|
+
|
|
348
|
+
### Deprecated — 21 rules retired per docs/RULE-LIFECYCLE.md (measured 100% FP, premise wrong)
|
|
349
|
+
|
|
350
|
+
Every rule below measured 100% FP (zero TPs at n ≥ 10, `docs/FP-AUDIT.md`)
|
|
351
|
+
on real-world code. Per the lifecycle policy the severity is downgraded to
|
|
352
|
+
`info` (non-blocking everywhere) and `falsePositiveRisk` is set to `high`;
|
|
353
|
+
the code and fixtures stay in the repo, the frozen ID is never reused, and
|
|
354
|
+
any salvageable detection idea ships under a NEW rule ID. If you gated CI
|
|
355
|
+
on these findings, they no longer block at `info` severity; add an explicit
|
|
356
|
+
`severityOverrides` entry in `mjolnir.config.json` to restore blocking.
|
|
357
|
+
|
|
358
|
+
- **QA-PW-005** (business logic in `page.evaluate()`): in test files,
|
|
359
|
+
branching inside evaluate is the only way to reach browser state — every
|
|
360
|
+
measured use was browser-only test instrumentation or the API under test,
|
|
361
|
+
never leaked app logic. No successor.
|
|
362
|
+
- **QA-PW-103** (missing timeout on navigation): bare `goto()` to the app
|
|
363
|
+
under test is the universal navigation idiom; per-call budgets are
|
|
364
|
+
config territory. No successor.
|
|
365
|
+
- **QA-PW-105** (`expect.poll` without timeout): the default poll timeout
|
|
366
|
+
is a hard bound that raises — the claimed masking harm cannot occur.
|
|
367
|
+
No successor.
|
|
368
|
+
- **QA-PW-107** (`toBeVisible` on toast/banner/modal): presence +
|
|
369
|
+
auto-retry semantics is what suites assert; viewport visibility is a
|
|
370
|
+
different question the suites are not asking. No successor.
|
|
371
|
+
- **QA-PW-108** (`toHaveText` coupling): asserting self-owned markup's
|
|
372
|
+
exact text is a legitimate strong assertion; "whose markup is this" is
|
|
373
|
+
not statically decidable. No successor.
|
|
374
|
+
- **QA-PW-112** (testid naming convention): hardcoding kebab-case as _the_
|
|
375
|
+
convention is the defect — conventions are repo-local. Successor idea
|
|
376
|
+
(mixed-conventions-within-one-repo check) requires a NEW ID.
|
|
377
|
+
- **QA-PW-114** (legacy element handles): the auto-wait harm needs a
|
|
378
|
+
timing window; every measured use was immediate reads or deliberate
|
|
379
|
+
existence checks. No successor.
|
|
380
|
+
- **QA-PW-118** (`networkidle` waits): the flake source is environmental
|
|
381
|
+
background traffic, absent by construction where the rule fired; harm is
|
|
382
|
+
not code-detectable. No successor.
|
|
383
|
+
- **QA-PW-119** (module-level state order dependence, was `error`): FPs
|
|
384
|
+
scatter across ≥5 legitimate infrastructure idioms (per-test teardown
|
|
385
|
+
harnesses, counters, vi.hoisted fixtures, memoized shared infra) — an
|
|
386
|
+
error-severity rule at 0 TP / 24 is actively misleading. Successor idea
|
|
387
|
+
(cross-test write→read dataflow analysis) requires a NEW ID.
|
|
388
|
+
- **QA-PW-120** (missing environment guard): file-level keyword
|
|
389
|
+
co-occurrence does not imply engine dependence; even the corpus's e2e
|
|
390
|
+
specs are engine-agnostic. No successor.
|
|
391
|
+
- **QA-PW-145** (no a11y assertions): absence of optional coverage is not
|
|
392
|
+
a defect finding; the heuristic fires on every UI spec by construction.
|
|
393
|
+
Successor idea (a11y-coverage reporting) requires a NEW ID.
|
|
394
|
+
- **QA-TQUAL-001** (mock-only verification, 0 TP / 26): spies observe the
|
|
395
|
+
real unit's output — the mock call IS the observable contract;
|
|
396
|
+
stand-in vs observer is not statically decidable. No successor.
|
|
397
|
+
- **QA-PY-006** (empty test body `pass`): 18/20 FPs were pytester
|
|
398
|
+
test-data scripts; genuinely collected empty tests don't occur in real
|
|
399
|
+
code. No successor.
|
|
400
|
+
- **QA-PY-008** (mock-only verification): boundary mocking and real-output
|
|
401
|
+
spies are contract testing, not mock theater. No successor.
|
|
402
|
+
- **QA-PY-010** (random/time without freeze): wall-clock reads are the
|
|
403
|
+
measured subject in timing/throttle tests; freezing would defeat them.
|
|
404
|
+
No successor.
|
|
405
|
+
- **QA-JV-108 / QA-CS-108** (hardcoded environment URL): FPs split across
|
|
406
|
+
HAR-replay fixtures, route-mocked origins, and proxy-failure tests — no
|
|
407
|
+
mechanically discriminable shape (the M-06 header concedes a fake-TLD
|
|
408
|
+
lookahead fixes zero measured FPs). No successor.
|
|
409
|
+
- **QA-JV-110 / QA-CS-110** (no a11y assertions): same absence-heuristic
|
|
410
|
+
premise failure as QA-PW-145. Successor idea requires a NEW ID.
|
|
411
|
+
- **QA-JV-111 / QA-CS-111** (blanket route mock): route-API self-tests and
|
|
412
|
+
fixture setup; no provable exclusion, and framework gating cannot help.
|
|
413
|
+
Revival on an application-repo corpus would need a NEW ID.
|
|
414
|
+
|
|
415
|
+
### Changed — Phase 2 retunes (EVIDENCE-BACKED, detectorRevision 2 per plan §07)
|
|
416
|
+
|
|
417
|
+
Each retune targets the single fixable root cause its measured FP cohort
|
|
418
|
+
shares (full evidence table: docs/RULE-LIFECYCLE.md "Phase 2
|
|
419
|
+
quarantine-cluster triage"). Detection-logic changes bump the rules'
|
|
420
|
+
`detectorRevision` to 2, invalidating the revision-1 measurements (stale →
|
|
421
|
+
provisional → re-measure, plan §07); must-not-fire fixtures now encode the
|
|
422
|
+
measured FP shapes so the retunes are regression-locked in both directions.
|
|
423
|
+
Tier stays quarantine until re-measurement says otherwise.
|
|
424
|
+
|
|
425
|
+
- **QA-PW-102** (load-wait instead of assertion): the wait now fires only
|
|
426
|
+
when it is the TERMINAL wait (no `expect`/assert/`expect.poll` follows)
|
|
427
|
+
and is skipped when consumed by `expect(...).rejects` — the "instead of
|
|
428
|
+
an assertion" premise, now actually checked. Clears all 20 measured FPs
|
|
429
|
+
(vite HMR synchronization waits); keeps the must-fire no-assertion shape.
|
|
430
|
+
- **QA-ENV-001** (environment coupling): the fixed-port sub-pattern no
|
|
431
|
+
longer matches loopback endpoints (`localhost`/`127.x`) — they are the
|
|
432
|
+
suite's own fixture containers (Azurite / DynamoDB Local / Mongo), the
|
|
433
|
+
entire measured FP cohort. Dotted hostnames and non-loopback IPv4
|
|
434
|
+
literals with ports still fire; OS-path, locale, and local-time-getter
|
|
435
|
+
sub-patterns are unchanged.
|
|
436
|
+
- **QA-PY-003** (no-assertion test): the verification vocabulary gains
|
|
437
|
+
`pytest.warns` / `pytest.deprecated_call` / `pytest.fail`, and a
|
|
438
|
+
`test_*` function referenced by name elsewhere in its file (pytester-
|
|
439
|
+
style test data) is skipped — the collected assertion lives in the
|
|
440
|
+
parent test.
|
|
441
|
+
- **QA-PY-004** (bare truthiness assert): boolean-predicate calls are
|
|
442
|
+
skipped — `assert isinstance(x, T)` type guards and
|
|
443
|
+
`assert s.startswith(...)`-style content predicates are real checks
|
|
444
|
+
(the measured FP clusters). Bare identifier/attribute asserts still
|
|
445
|
+
fire. This rule was measured at 45% FP (n=20), not 100%; with the
|
|
446
|
+
clusters removed the re-measurement is expected to approach the
|
|
447
|
+
extended band (≤30%), pending delta classification.
|
|
448
|
+
- **QA-PY-007** (raises without match): `pytest.raises(X) as exc_info`
|
|
449
|
+
followed by an assert/expect on `exc_info.value` is skipped — the
|
|
450
|
+
message IS verified without `match=` (8 of 13 measured FPs). The
|
|
451
|
+
genuinely vague raises blocks still fire; remaining FP residue
|
|
452
|
+
(single-possible-exception blocks) is not statically decidable.
|
|
453
|
+
- **QA-PY-105** (Playwright-Python test without assertions): a called
|
|
454
|
+
helper whose name asserts (`assert_*`/`expect_*`/`verify_*`/`check_*`)
|
|
455
|
+
or waits (`wait_for_*`) counts as verification — assertions delegated
|
|
456
|
+
to imported helpers, the entire measured FP cohort (streamlit e2e).
|
|
457
|
+
- **QA-JV-106 / QA-CS-106 / QA-PY-104** (brittle selectors): no pattern
|
|
458
|
+
change — the measured 100% FP rows predate Bug Map M-06's removal of
|
|
459
|
+
the querySelector/QuerySelectorAsync/query_selector patterns, so the
|
|
460
|
+
measurement described a detector that no longer ships. The §07 fix is
|
|
461
|
+
the detectorRevision bump to 2 (stale → provisional → re-measure);
|
|
462
|
+
the surviving xpath=/nth-child/absolute-path patterns stay
|
|
463
|
+
quarantine-tier until re-measured.
|
|
464
|
+
|
|
465
|
+
### Changed — §07 loop closed: rev-3 delta re-measurement (corpus rescans + verdict reconciliation)
|
|
466
|
+
|
|
467
|
+
The revision-2/3 detectors were re-run over the corpus repos that
|
|
468
|
+
produced the retuned rules' original verdicts, and the committed verdict
|
|
469
|
+
corpus was reconciled to the current detectors (the §07 loop the
|
|
470
|
+
capability matrix calls "re-measure"):
|
|
471
|
+
|
|
472
|
+
- **Delta method:** every committed row whose finding no longer fires
|
|
473
|
+
was removed (superseded — it described a pre-retune detector); every
|
|
474
|
+
NEW finding was adjudicated from source context per
|
|
475
|
+
`tests/corpus/verdicts/README.md` criteria, with a quota sample of 20
|
|
476
|
+
per rule on the largest surfaces (QA-PY-004/007) and full coverage on
|
|
477
|
+
the small ones. Net: 157 rows retired, 71 adjudicated rows appended,
|
|
478
|
+
1423 → 1337 classified verdicts, measured coverage 78 → **72/91**
|
|
479
|
+
(the six rules whose remaining classified counts fell below n=10 —
|
|
480
|
+
QA-PW-102, QA-PY-104, QA-PY-105, QA-ENV-001, QA-JV-106, QA-CS-106 —
|
|
481
|
+
are now unmeasured at their current revisions; their sidecar entries
|
|
482
|
+
were removed accordingly and they stay quarantine/extended-tier
|
|
483
|
+
pending re-measurement).
|
|
484
|
+
- **Re-measured envelopes (rev 3):** QA-PY-003 82% FP (n=17; remaining
|
|
485
|
+
FPs are pytest doc examples and pytester collection fixtures —
|
|
486
|
+
accepted residue), QA-PY-004 76% (n=21; remaining FPs are
|
|
487
|
+
pytest's own predicate-call idiom families: fnmatch/samefile/
|
|
488
|
+
isimportable membership checks and deliberate `assert False`
|
|
489
|
+
fail-marker DATA), QA-PY-007 79% (n=34; remaining FPs are
|
|
490
|
+
single-possible-exception API-error contracts where the raised type
|
|
491
|
+
IS the assertion — not statically decidable). All three stay
|
|
492
|
+
quarantine-tier with their detectorRevision 3 stamped in the sidecar
|
|
493
|
+
and FP-AUDIT; their accepted-residue FP causes are documented in
|
|
494
|
+
docs/RULE-LIFECYCLE.md's triage table.
|
|
495
|
+
- **QA-PW-102 / QA-PY-105: fully cleared** — 0 findings on the
|
|
496
|
+
previously-20/20-FP corpus slices at revision 2.
|
|
497
|
+
- **QA-ENV-001 (rev 3):** the wave-2 delta (20/20 FP on the rev-2
|
|
498
|
+
detector) showed no host shape is statically decidable — the
|
|
499
|
+
fixed-port sub-pattern is dropped entirely; OS-path, locale, and
|
|
500
|
+
local-time-getter sub-patterns stay (their remaining 6 findings on
|
|
501
|
+
vite are path-literal test DATA, adjudicated FP and recorded).
|
|
502
|
+
- **§07 comparability guard:** the FP regression governor
|
|
503
|
+
(`checkFpRegression`) no longer compares measurements across
|
|
504
|
+
detectorRevision boundaries — a revision mismatch means the old rate
|
|
505
|
+
does not describe the current detector, so it is neither flagged nor
|
|
506
|
+
blessed.
|
|
507
|
+
|
|
508
|
+
### Verification Trust Evolution, Phase 2 — detectionStrategy enum (D6 closed, scan-behavior-neutral)
|
|
509
|
+
|
|
510
|
+
### Changed — D6 enum migration (plan §12.1; metadata-only)
|
|
511
|
+
|
|
512
|
+
- **`detectionStrategy` is now the enforced §09.6 enum** (`LEXICAL | AST |
|
|
513
|
+
SEMANTIC | FRAMEWORK | RUNTIME`) instead of free text: `src/rules/rule.ts`
|
|
514
|
+
types `RuleMeta.detectionStrategy` as the union, and a registry ratchet
|
|
515
|
+
(tests/rules.registry.spec.ts) fails CI when any rule omits it or carries
|
|
516
|
+
a non-enum value — the "free text" drift class cannot reintroduce itself.
|
|
517
|
+
All 91 registry rules were migrated in place. This is a metadata migration,
|
|
518
|
+
not a detection change: no `run()` body, pattern, or scoping was touched,
|
|
519
|
+
so scan findings, golden fixtures, and corpus baselines are byte-identical
|
|
520
|
+
(the Phase 0 classification of this work as BEHAVIOR-NEUTRAL for scan
|
|
521
|
+
findings, plan §06). Per §11.3 the enum conversion is metadata naming only
|
|
522
|
+
and does NOT bump any `detectorRevision` — every measurement stays valid.
|
|
523
|
+
- **Legacy nuance preserved, not deleted:** the richer free-text
|
|
524
|
+
declarations ("regex pattern + inside-string oracle", "parsed YAML +
|
|
525
|
+
test-command gate", …) moved verbatim into a new optional
|
|
526
|
+
`detectionNotes` field rendered alongside the enum in the rule docs
|
|
527
|
+
pages. The capability matrix now renders the declared enum directly in a
|
|
528
|
+
single "Detection strategy (enum)" column (the provisional
|
|
529
|
+
"Enum (proposed)" guess column is gone — the declared value IS the
|
|
530
|
+
enum); `UNCLASSIFIED` renders only for an undeclared value.
|
|
531
|
+
- Mapping applied: regex-over-text/absence-sweep detectors → `LEXICAL`
|
|
532
|
+
(incl. QA-TQUAL-002's AST-stripped text pattern, which is a text pass
|
|
533
|
+
over code-only text); ts-morph node-walk detectors (QA-PW-002,
|
|
534
|
+
QA-PW-005) → `AST`; GitHub-Actions workflow-structure detectors
|
|
535
|
+
(QA-CI-001/009/010) → `FRAMEWORK`. `SEMANTIC` and `RUNTIME` remain
|
|
536
|
+
reserved (no rule ships either yet).
|
|
537
|
+
|
|
538
|
+
### Verification Trust Evolution, Phase 1 exit — dedicated corpora + wave-5 measurement (plan §11.5/§08)
|
|
539
|
+
|
|
540
|
+
### Added — dedicated corpora (§11.5) and the wave-5 measurement
|
|
541
|
+
|
|
542
|
+
- **CORPUS 19 → 34 repos** (`tests/corpus/audit.ts`): 14 new real-world
|
|
543
|
+
repos chosen by evaluating each candidate's unmeasured-rule fire count
|
|
544
|
+
at HEAD before committing (`vitest-dev-vitest`, `streamlit-streamlit`,
|
|
545
|
+
`apache-airflow`, `iluwatar-java-design-patterns`,
|
|
546
|
+
`spectreconsole-spectre-console`, `Humanizr-Humanizer`,
|
|
547
|
+
`cypress-realworld-app`, `keycloak-keycloak`, `appsmithorg-appsmith`,
|
|
548
|
+
`getsentry-sentry`, `github-docs`, `vercel-next-js`, `hashicorp-vault`,
|
|
549
|
+
`nocodb-nocodb`). This is the plan's dedicated-Corpora requirement for
|
|
550
|
+
§11.5: CI-workflow density for the starved QA-CI-* rules and JV/CS
|
|
551
|
+
application repos so the QA-JV/QA-CS rules measure on consumer code,
|
|
552
|
+
not just the Playwright bindings themselves (D5). Candidates whose scan
|
|
553
|
+
truncated against the budget (`n8n`, `posthog`, `vscode`) were rejected —
|
|
554
|
+
a partial scan can never be count-locked.
|
|
555
|
+
- **Committed class-B/C fixture corpora (§08):**
|
|
556
|
+
`tests/corpus/positive-fixtures/` (realistic anti-pattern exhibits that
|
|
557
|
+
MUST fire — class-B recall evidence, every fire classifies TP) and
|
|
558
|
+
`tests/corpus/negative-fixtures/` (realistic legitimate code that must
|
|
559
|
+
NOT fire — class-C precision evidence, any fire is a recorded FP).
|
|
560
|
+
These give rules whose patterns are rare in the wild a measurement-grade
|
|
561
|
+
verdict surface, versioned with the verdicts that classify them via
|
|
562
|
+
`local:` corpus URLs. Excluded from self-scan, vitest, eslint, prettier
|
|
563
|
+
and the test tsconfig — they are DATA.
|
|
564
|
+
- **`corpus-sample.ts --unmeasured-only`:** the verdict-harvesting loop
|
|
565
|
+
can sample only rules without a valid measurement, so classification
|
|
566
|
+
effort goes to the exit gate instead of re-sampling measured rules.
|
|
567
|
+
- **`audit.ts --only=<name>,<name>`:** re-check or re-record a corpus
|
|
568
|
+
subset (used to re-verify a repo after a transient truncation without
|
|
569
|
+
rescanning everything); the completeness threshold now applies to the
|
|
570
|
+
filtered set. Clones use `git -c core.longpaths=true` (scoped, not a
|
|
571
|
+
global config change) and the per-repo scan budget is 60s → 120s; the
|
|
572
|
+
audit job gets `NODE_OPTIONS=--max-old-space-size=8192` (sentry's
|
|
573
|
+
repo-scale parse OOMs the default heap) and a 60-minute timeout.
|
|
574
|
+
- **Measured coverage 43 → 78 of 91** (1423 classified verdicts, 0 blank,
|
|
575
|
+
0 UNSURE): 35 rules newly measured at n ≥ 10 — QA-TEST-001, QA-TEST-006,
|
|
576
|
+
QA-TEST-010, QA-TQUAL-002, QA-TQUAL-009, QA-TQUAL-011, QA-CI-001,
|
|
577
|
+
QA-CI-002, QA-CI-005, QA-CI-008, QA-PW-003, QA-PW-004, QA-PW-104,
|
|
578
|
+
QA-PW-113, QA-PW-115, QA-PW-117, QA-PW-121, QA-PW-123, QA-PW-140,
|
|
579
|
+
QA-PW-141, QA-PW-142, QA-PW-144, QA-PY-001, QA-PY-009, QA-PY-011,
|
|
580
|
+
QA-PY-012, QA-PY-103, QA-PY-105, QA-JV-101, QA-JV-102, QA-JV-109,
|
|
581
|
+
QA-CS-103, QA-CS-107 (plus the four previously counted). **Phase 1
|
|
582
|
+
exit gate MET: unmeasured 48 → 13 (≤ 20).**
|
|
583
|
+
- **Explicit tier declarations for all 35 newly measured rules, set from
|
|
584
|
+
the measured FP band (§11.2: ≤10% core, ≤30% extended, >30%
|
|
585
|
+
quarantine):** core — QA-PW-003, QA-PW-104, QA-PW-113, QA-PW-117,
|
|
586
|
+
QA-PW-121, QA-PW-140, QA-PY-001, QA-PY-009, QA-PY-011, QA-PY-103,
|
|
587
|
+
QA-JV-101, QA-JV-109; extended — QA-TQUAL-011, QA-CI-002, QA-CI-007,
|
|
588
|
+
QA-PW-141, QA-PW-142, QA-PW-144; quarantine — QA-TEST-001, QA-TEST-006,
|
|
589
|
+
QA-TEST-010, QA-TQUAL-002, QA-TQUAL-009, QA-PW-004, QA-PW-115,
|
|
590
|
+
QA-PW-123, QA-PY-012, QA-PY-105, QA-CI-001, QA-CI-005, QA-CI-008,
|
|
591
|
+
QA-CI-010, QA-CS-103. Every measured entry carries `detectorRevision`
|
|
592
|
+
in the sidecar. **Evidence-backed, not silent:** the quarantine
|
|
593
|
+
demotions remove error-severity deductions from default scans, which is
|
|
594
|
+
the tier policy working as designed — the failing specs were updated
|
|
595
|
+
with their reasoning inline (gate tests re-anchored on QA-CI-009, the
|
|
596
|
+
demo repo's CI grew real CI-009 exhibits, the demo/hero assets and
|
|
597
|
+
`fix` command scan with `--strict`).
|
|
598
|
+
- **Recall-floor + corpus baselines:** 34 count-locked baselines recorded
|
|
599
|
+
(non-partial scans only); the §20.6 recall floor extends to the
|
|
600
|
+
expanded registry.
|
|
601
|
+
|
|
602
|
+
### Changed — scan-behavior fallout of the measured demotions (explained)
|
|
603
|
+
|
|
604
|
+
- `mjolnir fix` now scans with `--strict`: an auto-fixable rule that is
|
|
605
|
+
measured into quarantine (QA-TEST-001's `.only` fix) must still be
|
|
606
|
+
fixable — hiding it would make `fix` a no-op on its own target debt.
|
|
607
|
+
- The demo repo's CI workflow gained genuine CI-009 exhibits (piped and
|
|
608
|
+
`;`-sequenced test commands) so the demo keeps demonstrating the
|
|
609
|
+
NEEDS-WORK band now that its QA-TEST-001/QA-CI-001/PW-004 debt is
|
|
610
|
+
quarantine-capped to info (non-deducting); `docs:demo`/`docs:hero` scan
|
|
611
|
+
with `--strict` so the committed assets and their drift locks stay in
|
|
612
|
+
sync with the precision-contract spec.
|
|
613
|
+
|
|
614
|
+
### Verification Trust Evolution, Phase 1 — measurement infrastructure
|
|
615
|
+
|
|
616
|
+
### Added — UNSURE adjudication gate + QA-PW-101 measured (plan §11.5)
|
|
617
|
+
|
|
618
|
+
- **UNSURE ceiling ratchet:** `npm run fp-audit:generate` now fails when the
|
|
619
|
+
UNSURE backlog grows beyond the committed
|
|
620
|
+
`tests/corpus/verdicts/unsure-ceiling.json` (the §11.5 mechanism: UNSURE
|
|
621
|
+
never counts into `n` but always triggers review). The ceiling only moves
|
|
622
|
+
DOWN via documented adjudication; upward movement requires an explicit
|
|
623
|
+
`--update` whose diff names every rule that grew. Criteria live in
|
|
624
|
+
`tests/corpus/verdicts/README.md`.
|
|
625
|
+
- **QA-PW-101 is measured (the D5 "parked on 20 UNSURE" defect resolved):**
|
|
626
|
+
all 20 UNSURE verdicts were adjudicated by reading the cited sources at
|
|
627
|
+
repo HEAD — **20 TP, 0 FP, n=20** (next-auth's session-sync sleeps before
|
|
628
|
+
reading session state; sveltejs/kit's 100 ms request-observation windows
|
|
629
|
+
that false-pass when a stray refresh starts after the window). The rule
|
|
630
|
+
now declares `tier: "core"` (0% FP ≤ 10%, n ≥ 10, revision 1, recall
|
|
631
|
+
floor satisfied: fires in 4 corpus baselines) — the measurement-dependent
|
|
632
|
+
default would resolve it to core anyway, but measured rules declare
|
|
633
|
+
their tier explicitly (the D3 Step 2 invariant).
|
|
634
|
+
- **QA-TQUAL-009 UNSURE row resolved → FP** (tanstack-query angular
|
|
635
|
+
inject-query:529 — a deliberate `void` fire-and-forget whose assertions
|
|
636
|
+
still fail the run via vitest's unhandled-rejection handling; the
|
|
637
|
+
FALSE-GREEN diagnosis does not hold). n=2 — below the measurement bar,
|
|
638
|
+
still unmeasured/PROVISIONAL.
|
|
639
|
+
- **Coverage:** 42/91 → **43/91** measured rules; UNSURE backlog 21 → **0**.
|
|
640
|
+
|
|
641
|
+
### Changed — D3 two-step tier fix (plan §11.2; scan-behavior-neutral)
|
|
642
|
+
|
|
643
|
+
- **Step 1 (explicit tiers):** every rule now declares its `tier`
|
|
644
|
+
explicitly — the 42 formerly implicit-core rules received an explicit
|
|
645
|
+
`tier: "core"` matching their effective tier. Byte-identical scan
|
|
646
|
+
behavior (the declared value equaled the omitted-tier default);
|
|
647
|
+
generated docs updated with an explained diff (the matrix's
|
|
648
|
+
"explicit tier declarations" line moved 49/91 → 91/91).
|
|
649
|
+
- **Step 2 (measurement-dependent default):** the omitted-tier default is
|
|
650
|
+
no longer unconditionally core. A rule that omits `tier` resolves via
|
|
651
|
+
`effectiveTier` (new `src/rules/measurement.ts`): **core** only with a
|
|
652
|
+
valid corpus measurement (n ≥ 10 at a matching `detectorRevision`),
|
|
653
|
+
otherwise **extended**, displayed with the new **PROVISIONAL** status.
|
|
654
|
+
PROVISIONAL is a display status (`tier extended/unmeasured`), not a
|
|
655
|
+
tier value — no schema churn. Scan findings are byte-identical: the
|
|
656
|
+
pipeline enforces only `quarantine` (severity/info + E0 caps, --strict
|
|
657
|
+
filter), and every quarantine rule declares its tier explicitly;
|
|
658
|
+
overlap-dedup consumes declared tiers only, and no demoted rule
|
|
659
|
+
participates in an `overlapWith` tier comparison that could change a
|
|
660
|
+
survivor set (all overlap groups are single-declarer/single-target).
|
|
661
|
+
- **The D3 demotion (38 rules drop from effective core to
|
|
662
|
+
extended/PROVISIONAL):** QA-TEST-001, QA-TEST-006, QA-TEST-010,
|
|
663
|
+
QA-TQUAL-002, QA-TQUAL-009, QA-TQUAL-011, QA-PW-003, QA-PW-004,
|
|
664
|
+
QA-PW-101, QA-PW-104, QA-PW-113, QA-PW-115, QA-PW-116, QA-PW-117,
|
|
665
|
+
QA-PW-121, QA-PW-123, QA-PW-124, QA-PW-125, QA-PW-140, QA-PW-141,
|
|
666
|
+
QA-PW-142, QA-PW-144, QA-CI-001, QA-CI-002, QA-CI-005, QA-CI-007,
|
|
667
|
+
QA-CI-008, QA-CI-009, QA-CI-010, QA-PY-001, QA-PY-009, QA-PY-011,
|
|
668
|
+
QA-PY-012, QA-PY-101, QA-PY-103, QA-PY-105, QA-PY-106, QA-PY-107.
|
|
669
|
+
The 4 measured implicit-core rules (QA-PW-002, QA-PY-002, QA-JV-105,
|
|
670
|
+
QA-CS-101) keep core. Unmeasured-effective-core count: 38 → **0**.
|
|
671
|
+
**Suite-invalidating callout (plan §11.2):** QA-TEST-001 (focused test
|
|
672
|
+
committed, `suiteInvalidating: true`) is among the demoted rules — its
|
|
673
|
+
findings still void the suite's pass claim and still gate CI at
|
|
674
|
+
severity=error; only its tier/statistics status changed. QA-PY-001 is
|
|
675
|
+
the suite-invalidating Python sibling (same class).
|
|
676
|
+
- **Registry ratchet (plan §20.3, enforced in code):** new
|
|
677
|
+
`tests/registry-ratchet.spec.ts` fails CI on ANY unmeasured rule in
|
|
678
|
+
effective core (`tier core ⇒ valid MEASURED_FP with matching
|
|
679
|
+
detectorRevision, FP ≤ 10%, n ≥ 10`), on any detectorRevision mismatch
|
|
680
|
+
(§20.5: stale ⇒ provisional), and — recall floor (§20.6) — on a core
|
|
681
|
+
rule that fires nowhere in the corpus baselines. §20.1 evidence-state
|
|
682
|
+
monotonicity: the measured ratio may only improve without a
|
|
683
|
+
machine-detectable `MEASUREMENT-EXCEPTION` marker in this CHANGELOG.
|
|
684
|
+
- **`mjolnir doctor`**: `MAX_UNMEASURED_CORE` lowered 40 → **0** (Phase 1
|
|
685
|
+
exit gate: 0 unmeasured in effective core, now enforced); tier checks
|
|
686
|
+
consume `effectiveTier` + stale-measurement logic. Display surfaces
|
|
687
|
+
(`mjolnir explain`, `mjolnir rules`, generated rule docs, capability
|
|
688
|
+
matrix) render the PROVISIONAL status honestly.
|
|
33
689
|
|
|
34
|
-
|
|
690
|
+
### Verification Trust Evolution, Phase 0 + Phase 1 prep
|
|
691
|
+
|
|
692
|
+
### Added — Rule Capability Matrix (Phase 0)
|
|
693
|
+
|
|
694
|
+
- **Capability Matrix v0** (`docs/RULE-CAPABILITY-MATRIX.md` +
|
|
695
|
+
`docs/RULE-CAPABILITY-MATRIX.json`): generated per-rule inventory of all
|
|
696
|
+
91 rules from the registry + `MEASURED_FP` + the verdict corpus —
|
|
697
|
+
category, languages, frameworks, declared detection strategy with a
|
|
698
|
+
provisional enum mapping (Phase 0 contract proposal), semantic depth,
|
|
699
|
+
measured flag, FP rate with sample size, corpus size and diversity.
|
|
700
|
+
Unknown fields render as `UNCLASSIFIED` — visible gaps are the
|
|
701
|
+
deliverable. Regenerated with `npm run docs:capability`; drift-locked by
|
|
702
|
+
`tests/capability-matrix.spec.ts` and the generated-docs-drift CI job.
|
|
703
|
+
- **Declared-vs-measured cross-check report** (ledger class D9): the
|
|
704
|
+
matrix names every measured rule whose declared tier violates the FP
|
|
705
|
+
ceilings (core > 10%, extended > 30%) and produces the D3 demotion list
|
|
706
|
+
(38 unmeasured rules currently in effective core — Phase 1 input).
|
|
707
|
+
Current run: **0 D9 mismatches** (all measured > 30% FP rules already
|
|
708
|
+
declare `quarantine`).
|
|
709
|
+
- **Defect ledger recorded** into the matrix metadata (plan §02, D1–D8
|
|
710
|
+
with owning phase per defect).
|
|
711
|
+
|
|
712
|
+
### Added — detectorRevision scaffold (Phase 1 prep)
|
|
713
|
+
|
|
714
|
+
- **`tests/corpus/detector-revisions.json` sidecar**: hand-maintained,
|
|
715
|
+
one entry per measured rule (all at revision 1 today), diffable.
|
|
716
|
+
`MEASURED_FP` entries now carry `detectorRevision` stamped from the
|
|
717
|
+
sidecar by `fp-audit:generate`; `docs/FP-AUDIT.md` gains a
|
|
718
|
+
`detectorRev` column. Measurement inheritance law (§07): a measurement
|
|
719
|
+
belongs to a specific detector implementation, not merely to a rule
|
|
720
|
+
ID. Drift lock extended in `tests/measured-fp-generated.spec.ts`
|
|
721
|
+
(sidecar covers exactly the measured set; revisions are positive
|
|
722
|
+
integers matching the sidecar).
|
|
723
|
+
|
|
724
|
+
### Fixed — packaging: offline grammar loading (Phase 0.5 spike, D2)
|
|
725
|
+
|
|
726
|
+
- **`tree-sitter-wasms` and `web-tree-sitter` moved to `dependencies`**
|
|
727
|
+
(web-tree-sitter keeps its exact `0.25.6` pin — 0.26.x cannot load the
|
|
728
|
+
prebuilt grammar files). The published CLI's dependency tree now
|
|
729
|
+
carries the tree-sitter Java/C# grammars, so `npm install mjolnir-qa`
|
|
730
|
+
can load them offline once the Phase 0.5 parse-stage wiring consumes
|
|
731
|
+
them. Removed the misleading `!dist/**/*.wasm` files exclusion (the
|
|
732
|
+
grammars ship via the dependency, not the bundle).
|
|
733
|
+
- **Pack-smoke regression test**: `tests/package-smoke.spec.ts` asserts
|
|
734
|
+
the packed package declares both as runtime dependencies and that the
|
|
735
|
+
java/c_sharp grammars resolve inside the installed dependency tree.
|
|
736
|
+
|
|
737
|
+
### Fixed — adapter header claims (D4)
|
|
738
|
+
|
|
739
|
+
- `src/adapters/java.ts` no longer claims to be a "Second tree-sitter
|
|
740
|
+
consumer" — it is a regex-layer adapter; the tree-sitter-java grammar
|
|
741
|
+
and the async `parseJavaAst` seam exist but are not wired into the
|
|
742
|
+
synchronous scan (D1). Same honest correction for
|
|
743
|
+
`src/adapters/csharp.ts` and `src/engine/adapter.ts` (whose header
|
|
744
|
+
still claimed tree-sitter "arrives in R2 with Python").
|
|
745
|
+
|
|
746
|
+
### Verification Trust Evolution, Phase 0.5 — async parse stage (D1)
|
|
747
|
+
|
|
748
|
+
### Changed — parse stage wired into the scan pipeline (D1 closed, BEHAVIOR-NEUTRAL)
|
|
749
|
+
|
|
750
|
+
- **Async parse stage between discovery and rule execution** (plan §10.1):
|
|
751
|
+
`runScan`'s per-file loop now awaits an optional `LanguageAdapter.parseAst`
|
|
752
|
+
hook before running rules. `runRules` and every rule stay synchronous and
|
|
753
|
+
consume the tree via `ParsedFile.ast` — the engine is NOT async end-to-end,
|
|
754
|
+
only the one inherently-async seam (WASM grammar load) is. `main()` and the
|
|
755
|
+
script entry points await the returned promise.
|
|
756
|
+
- **Java and C# adapters implement `parseAst`** backed by the previously-dead
|
|
757
|
+
`parseJavaAst`/`parseCSharpAst` (defect D1: tree-sitter AST was built and
|
|
758
|
+
tested in Sprint 8 but never consumed). Parse failure or a missing grammar
|
|
759
|
+
resolves `undefined` and rules fall back to the regex path — never fatal.
|
|
760
|
+
No rule consumes the AST yet (Phase 3 wires specific JV/CS rules), so scan
|
|
761
|
+
findings are byte-identical: golden lock, corpus baselines, and generated
|
|
762
|
+
assets all unchanged (BEHAVIOR-NEUTRAL mode, plan §06).
|
|
763
|
+
- **Parser lifecycle management** (plan §10.3): one memoized `Parser` per
|
|
764
|
+
grammar, bounded by a fixed-size parse-slot semaphore
|
|
765
|
+
(`MAX_CONCURRENT_PARSES`); every per-file tree is released via
|
|
766
|
+
`ParsedAst.dispose()` (`tree.delete()`) in a `finally`-equivalent position
|
|
767
|
+
that runs on normal completion, rule crash, per-file budget expiry, and
|
|
768
|
+
adapter throw; `releaseTreeSitterResources()` tears down the memoized
|
|
769
|
+
parsers after each scan (library-consumer hygiene). No leak path depends
|
|
770
|
+
on rules completing successfully.
|
|
771
|
+
- **Call-graph consequences**: `computeImpact` (and therefore
|
|
772
|
+
`runImpactCommand`) is async; `scripts/corpus-sample.ts`,
|
|
773
|
+
`scripts/generate-readme-demo.ts`, and `scripts/generate-readme-hero.ts`
|
|
774
|
+
await `runScan`; the full test suite (~40 call sites across 25 spec files)
|
|
775
|
+
awaits the now-async commands. No behavior change anywhere.
|
|
776
|
+
- **Verification evidence** (exit gate §10): golden lock byte-identical
|
|
777
|
+
(3/3), corpus count-lock unaffected (no JV/CS rule reads the AST),
|
|
778
|
+
capability-matrix + FP-AUDIT generated docs byte-identical, pack smoke
|
|
779
|
+
re-proven manually: `npm pack` → clean `npm install` → offline grammar
|
|
780
|
+
load + parse of real Java and C# source → offline `mjolnir` scan of a
|
|
781
|
+
Java fixture detecting QA-JV-102. `web-tree-sitter` stays pinned to
|
|
782
|
+
exactly `0.25.6` (§10.5, documented 0.26.x breakage).
|
|
783
|
+
|
|
784
|
+
### Added — score instrument redesign (hammer states)
|
|
785
|
+
|
|
786
|
+
- **ScoreState model** (`src/reporter/score-state.ts`): one pure source of
|
|
787
|
+
truth for band / verdict / color / headline per score — critical 0–49,
|
|
788
|
+
warning 50–79, trusted 80–99, forged 100. `verdictFor`, the terminal
|
|
789
|
+
gauge and the badge all delegate to it.
|
|
790
|
+
- **The hammer is now the score instrument** (terminal): a state-colored
|
|
791
|
+
hammer block renders above the WORTHINESS line — cracked (0–49),
|
|
792
|
+
strained with partial runes (50–79), charged with energy arcs (80–99),
|
|
793
|
+
halo + lightning at 100. A plain-text caption (`[CRACKED]` /
|
|
794
|
+
`[STRAINED]` / `[CHARGED]` / `[FORGED]`) carries the state without
|
|
795
|
+
color; ASCII fallback included.
|
|
796
|
+
- **Trusted is aurora-cyan, forged is white-gold** on every surface
|
|
797
|
+
(terminal palette, web tokens, brand README). Green is no longer a
|
|
798
|
+
score color — it survives for non-score success contexts only.
|
|
799
|
+
- **Findings render as cards** (terminal): Problem → Impact → Fix →
|
|
800
|
+
Verify with the evidence tag and measured FP rate beside the title;
|
|
801
|
+
rules with >3 findings collapse under one "same fix applies" header;
|
|
802
|
+
non-verbose shows 10 cards with an overflow line, `--verbose` shows
|
|
803
|
+
everything.
|
|
804
|
+
- **FORGED block at 100** replaces the bare FLAWLESS VICTORY line in
|
|
805
|
+
unicode mode (trophy retained inside; the `*** FLAWLESS VICTORY ***`
|
|
806
|
+
ASCII contract string is preserved).
|
|
807
|
+
- **PR comments show score drift** (`Score: 72/100 (+5 since baseline
|
|
808
|
+
<sha>)`) using the new additive `score` field in the baseline JSON, and
|
|
809
|
+
carry per-finding evidence tags.
|
|
810
|
+
- **Badge thresholds aligned** with the reporter: ≥80 / ≥50 / 100
|
|
811
|
+
(was ≥90 / ≥75 / ≥50), colors `red` / `yellow` / `important` /
|
|
812
|
+
`success`; the message at 100 reads `100/100 · forged`.
|
|
813
|
+
|
|
814
|
+
### Fixed — security & detection-regression audit (`.planning/AUDIT-2026-08-30-QA.md`)
|
|
815
|
+
|
|
816
|
+
- **QA-1 (P0, detection regression):** QA-TEST-003's M0-#4 header rewrite
|
|
817
|
+
required a space after `async` — `async()=>{` bodies silently stopped
|
|
818
|
+
matching. `\s*` restores them without reopening the bare-`return` exemption.
|
|
819
|
+
- **QA-2 (P0, detection regression):** single-walk discovery applied the
|
|
820
|
+
UNION of every adapter's `dirSkips`, so Python's `env` (= virtualenv)
|
|
821
|
+
and Java's `build` (= Gradle output) hid directories from every other
|
|
822
|
+
language — real TS test dirs (e.g. withastro/astro's `test/units/env/`)
|
|
823
|
+
silently vanished from scans. `dirSkips` are now applied per owning
|
|
824
|
+
language at file level.
|
|
825
|
+
- **QA-3 (P1):** `isInsideEmbeddedCode` truncated the masked run at the
|
|
826
|
+
first space inside a literal, so embedded test-data
|
|
827
|
+
(`'test(" foo", function () {})'`) was classified as live code and
|
|
828
|
+
fired 6 FP findings on eslint-plugin-playwright's ruleTester tables.
|
|
829
|
+
- **QA-4 (P1):** config `exclude` was never validated —
|
|
830
|
+
`exclude: [1, {}, null]` crashed the scan (exit 20). Non-string entries
|
|
831
|
+
are now a fixable usage error (exit 10), with a defense-in-depth filter
|
|
832
|
+
in pattern compilation.
|
|
833
|
+
- **QA-5 (P1):** an unparseable suppression `expires` value silently
|
|
834
|
+
degraded to "expired" via NaN comparisons; it is now a fixable
|
|
835
|
+
validation error at load time.
|
|
836
|
+
- **QA-6 (P1, documented-behavior fix):** the README's 90-day suppression
|
|
837
|
+
policy was only applied at write time by the `ignore` command —
|
|
838
|
+
hand-written entries without `expires` stayed active forever. The
|
|
839
|
+
default is now enforced at enforcement time, anchored at the config
|
|
840
|
+
file's mtime; `mjolnir suppressions` labels the default explicitly.
|
|
841
|
+
- **QA-7 (P1):** plugin reserved-prefix spoof rejection was
|
|
842
|
+
case-sensitive — `"qa-test-001"` walked straight past it. Matching is
|
|
843
|
+
now case-insensitive.
|
|
844
|
+
- **QA-8 (P2):** suppression `files` globs written with Windows
|
|
845
|
+
backslashes could never match the normalized finding paths; both sides
|
|
846
|
+
are normalized before matching.
|
|
847
|
+
- **QA-9 (P2, defense in depth):** option-shaped `--base` values are
|
|
848
|
+
refused at the git layer even when a programmatic caller bypasses
|
|
849
|
+
`parseArgs` (option-injection → `--upload-pack=` command execution).
|
|
850
|
+
- **QA-10 (P2):** finding metadata (hostile filenames, plugin messages)
|
|
851
|
+
reached the terminal and markdown PR comments unsanitized — ANSI
|
|
852
|
+
escapes and control characters are now stripped, and the PR comment
|
|
853
|
+
escapes markdown-significant characters.
|
|
854
|
+
- **QA-11 (P2):** the JUnit XML scan was quadratic on unclosed
|
|
855
|
+
`<testcase` floods (minutes of CPU for a 20 MB hostile report); the
|
|
856
|
+
scanner is now linear with a bounded-time regression test.
|
|
857
|
+
- **QA-12 (P2):** `loadBaseline`/`loadStats` are now total over arbitrary
|
|
858
|
+
JSON — `null` finding entries used to crash `diff`, and hostile stats
|
|
859
|
+
shapes leaked string junk into totals.
|
|
860
|
+
- **QA-13 (P3, defense in depth):** SARIF artifact URIs normalize
|
|
861
|
+
backslashes — `encodeURI` leaves `\` literal, which is not a valid
|
|
862
|
+
RFC 3986 uri-reference character.
|
|
863
|
+
- **QA-15 (P1):** the code-text mask iterated code points instead of
|
|
864
|
+
UTF-16 code units — one emoji made the mask shorter than the text and
|
|
865
|
+
silently disabled masking for the whole file.
|
|
866
|
+
- **QA-16 (P2):** files larger than the 1 MiB discovery cap were dropped
|
|
867
|
+
silently; the skip is now counted and flagged (`file-size`), keeping
|
|
868
|
+
the scan honest about what it did not read.
|
|
869
|
+
|
|
870
|
+
### Changed — corpus baseline
|
|
871
|
+
|
|
872
|
+
- Regenerated after review: M0-#2's QA-TEST-004 duplicate-count fix
|
|
873
|
+
(−5 on withastro-astro — every removed finding was a same-position
|
|
874
|
+
duplicate), M0-#4's function-body/return-narrowing detections (net
|
|
875
|
+
+1/+1/+8/+6 across vite/svelte-kit/tanstack-query/eslint-plugin-playwright,
|
|
876
|
+
spot-reviewed on real code), QA-3/QA-6 masking suppressions
|
|
877
|
+
(embedded-code test data no longer fires), and QA-15's mask-alignment
|
|
878
|
+
fix (QA-PW-105/108 match against `codeText`, whose offsets were
|
|
879
|
+
misaligned after any astral character — +27/+22/+1 restored matches on
|
|
880
|
+
vite/astro/tanstack-query).
|
|
881
|
+
|
|
882
|
+
### Added — Open-Beta E2E test plan (Tier 1–5, ~1,700 new test assertions)
|
|
883
|
+
|
|
884
|
+
- **Coverage ratchet:** statements/branches/functions/lines at literal
|
|
885
|
+
100% per file (`perFile: true`). Every branch arm, catch path, and
|
|
886
|
+
fallback in `src/**` is exercised or provably dead — ~30 dead guards
|
|
887
|
+
(unreachable `?? ""` fallbacks, unreachable-`if` arms behind
|
|
888
|
+
`noUncheckedIndexedAccess`, a dead proof-abort block in `fix`) were
|
|
889
|
+
removed or exported as testable pure helpers with the golden lock,
|
|
890
|
+
self-scan and determinism tripwires green throughout.
|
|
891
|
+
- **E2E journeys** (`tests/e2e/`): nine spec files run the built
|
|
892
|
+
`dist/cli.mjs` as a real child process — tarball first-run (the path
|
|
893
|
+
that historically caught macOS-only CLI breakage), CI PR flow with
|
|
894
|
+
changed-scope attribution against real git fixtures, the baseline →
|
|
895
|
+
diff → stats loop, the fix flow, the forensics flow, explain/rules,
|
|
896
|
+
create-rule onboarding, the config journey (gate/severityOverrides/
|
|
897
|
+
ignore/expiry honored end-to-end), and a full exit-code contract sweep
|
|
898
|
+
across every documented command.
|
|
899
|
+
- **Precision & accuracy:** every scorer number verified against the
|
|
900
|
+
documented benchmark (deductions 8/3/1, E2/E1/E0, honesty cap 99,
|
|
901
|
+
error ceiling 95, suite-invalidating ceiling 49, smoothing) with
|
|
902
|
+
fast-check property invariants (≥1,000 iterations: score bounds,
|
|
903
|
+
monotonicity, order-symmetry), Selector Health exact score vectors,
|
|
904
|
+
hand-computed forensics math, terminal-footer/JSON deduction
|
|
905
|
+
consistency, Mermaid well-formedness, and a three-verdict-band proof
|
|
906
|
+
(WORTHY / NEEDS WORK / UNWORTHY each reached for its stated reason).
|
|
907
|
+
- **Regression & integration:** adapter→reporter matrix (one finding
|
|
908
|
+
asserted on terminal, JSON, SARIF, and Mermaid), plugin flow
|
|
909
|
+
integration (valid plugin + reserved-prefix rejection), cross-file
|
|
910
|
+
analysis, monorepo containment (a workspace scan never reports
|
|
911
|
+
siblings), a mutation guard (each sampled rule's finding drops to zero
|
|
912
|
+
when the offending line is removed), and baseline forward-compat.
|
|
913
|
+
- **Nightly stress workflow** (`.github/workflows/stress.yml`, never
|
|
914
|
+
PR-blocking): 10k-file mixed-language synthetic repo under a 120 s
|
|
915
|
+
budget with no partial degradation, pathological trees (200-deep
|
|
916
|
+
nesting, 10k-char lines, unicode filenames, LF/CRLF/BOM mixes,
|
|
917
|
+
junctions, malformed specs), a 20-run byte-identical + flat-RSS soak,
|
|
918
|
+
the networked registry-install smoke (previously skipped locally), and
|
|
919
|
+
4 concurrent scans of one read-only target. Fixture generators live in
|
|
920
|
+
`tests/stress/`.
|
|
921
|
+
- **Stability:** the 1 MB masking budget now takes the median of 3 runs
|
|
922
|
+
with a 15 s ceiling (coverage instrumentation slows string-heavy loops
|
|
923
|
+
~3x and there is no in-worker marker to detect it; the ratio-based
|
|
924
|
+
linearity test remains the true non-quadratic guard), and the
|
|
925
|
+
scale-benchmark's 3k-file budget moved 20 s → 25 s with the same
|
|
926
|
+
rationale. The vitest global-setup builds `dist/` once before any
|
|
927
|
+
worker starts — parallel-file builds used to wipe dist mid-suite
|
|
928
|
+
(tsdown cleans `outDir`) and fail unrelated E2E spawns with
|
|
929
|
+
module-not-found.
|
|
930
|
+
- **Fixed (flagged by the new E2E sweep):** `doctor --bogus` ignored the
|
|
931
|
+
unknown flag and scanned the CWD as a surprise full run; it now prints
|
|
932
|
+
usage and exits 10, matching the flag-error parity of every other
|
|
933
|
+
subcommand (unit + E2E regression tests added).
|
|
934
|
+
- **Soak drift artifacts** moved from `coverage/` (a CI-generated dir the
|
|
935
|
+
tool's own QA-CI-005 rule correctly flags as "consumed but never
|
|
936
|
+
generated" in stress.yml) to a dedicated `soak-drift/` dir.
|
|
937
|
+
|
|
938
|
+
## [0.5.0] — 2026-08-29
|
|
939
|
+
|
|
940
|
+
### Added — measurement is now visible at the point of use
|
|
941
|
+
|
|
942
|
+
- Only 15 of 91 rules carry a false-positive rate measured against real OSS
|
|
943
|
+
code; that fact previously lived only in `docs/FP-AUDIT.md` and
|
|
944
|
+
`mjolnir doctor`. Now surfaced everywhere a user looks:
|
|
945
|
+
- The scan footer reports how many of the rules that _fired_ are measured.
|
|
946
|
+
- `mjolnir rules --unmeasured` / `--measured` filter the catalog; a new
|
|
947
|
+
"FP (measured)" column in `rules --md`; a "Measured FP rate" row on every
|
|
948
|
+
`docs/rules/` page and in `mjolnir explain`.
|
|
949
|
+
- JSON findings carry `measuredFpRate` and `measuredFpN` (additive —
|
|
950
|
+
`schemaVersion` is still 1).
|
|
951
|
+
- `src/rules/measured-fp.generated.ts` bakes the rates into the shipped
|
|
952
|
+
package (the raw verdicts are not packed); regenerated by
|
|
953
|
+
`npm run fp-audit:generate`, drift-locked by a test, and now the single
|
|
954
|
+
source `mjolnir doctor` reads.
|
|
955
|
+
- Scoring is unchanged — this is visibility only.
|
|
956
|
+
- **Corpus expanded 6 → 13 repos** so the previously-silent rule families
|
|
957
|
+
(QA-TEST, QA-TQUAL, most QA-PW, QA-CI-001) fire on real consumer code:
|
|
958
|
+
added `next-auth`, `vite`, `sveltekit`, `astro`, `TanStack/query`,
|
|
959
|
+
`eslint-plugin-playwright`, `playwright-pytest`. `corpus:sample` and
|
|
960
|
+
`corpus:regression` now scan with `--strict` (quarantine rules were
|
|
961
|
+
invisible to both before). `docs/FP-AUDIT.md` is 15/91 (down from 19 —
|
|
962
|
+
see the dispatch fix below, which retired the leaked cross-language
|
|
963
|
+
verdicts for QA-PW-101/112 and QA-TEST-004/QA-ENV-001 on Java/Python
|
|
964
|
+
repos); the ~250 new corpus findings are queued for classification in
|
|
965
|
+
`tests/corpus/verdicts/`, not counted until read.
|
|
966
|
+
|
|
967
|
+
### Changed — help and README lead with the one command
|
|
968
|
+
|
|
969
|
+
- `mjolnir --help` and the README quickstart now open with
|
|
970
|
+
`mjolnir --scope changed` as _the_ product, and group the other subcommands
|
|
971
|
+
into Everyday / When-something's-flaky / Occasional instead of a flat list
|
|
972
|
+
of 16 equals. A one-line first-run hint appears after a bare full-repo scan
|
|
973
|
+
with no config. No subcommand removed or renamed.
|
|
974
|
+
|
|
975
|
+
### Fixed (rule-bug-hunt wave)
|
|
976
|
+
|
|
977
|
+
- **Cross-language dispatch leak**: `appliesTo: "test-files"` mapped to all
|
|
978
|
+
four language adapters, so the 42 TypeScript/Playwright-only rules that
|
|
979
|
+
use it (QA-PW-\*, QA-TEST-\*, QA-TQUAL-\*) ran against `.py`, `.java` and
|
|
980
|
+
`.cs` files too. On the corpus this produced ~140 false positives on
|
|
981
|
+
`microsoft/playwright` Java bindings alone (QA-PW-101, QA-PW-112,
|
|
982
|
+
QA-TEST-004, QA-ENV-001) and inflated several baselines. `legacyAppliesTo`
|
|
983
|
+
now maps `"test-files"` to `["typescript"]` only; cross-language coverage
|
|
984
|
+
is the QA-PY/QA-JV/QA-CS families' job. Regression test added.
|
|
985
|
+
- **QA-PW-103** (missing timeout): no longer fires on assertion strings that
|
|
986
|
+
contain Playwright code as _test data_ (`code: "await page.goto('/x')"` in
|
|
987
|
+
playwright-mcp) — guarded by `isInsideEmbeddedCode`.
|
|
988
|
+
- **QA-TEST-004** (hard sleep): dropped the bare `sleep(N)` pattern and now
|
|
989
|
+
requires `await` and a non-zero argument. `sleep(10).then(...)` and
|
|
990
|
+
`queryFn: () => sleep(10)` are mock-latency, not test-body pauses — this
|
|
991
|
+
cut TanStack Query's count from 1648 to 157.
|
|
992
|
+
- **QA-PW-002** (unawaited assertion): matches only Playwright's 31
|
|
993
|
+
web-first async matchers instead of any `to*` name, so
|
|
994
|
+
`expect(res.status()).toBe(200)` on a variable named `page` is no longer
|
|
995
|
+
flagged.
|
|
996
|
+
- **QA-TQUAL-009** (assertion in unawaited promise chain): the `.then()`
|
|
997
|
+
callback body is now paren-matched instead of grabbing the next `{`, so a
|
|
998
|
+
sibling `.then(res => res.text())` inside an awaited `Promise.all` no
|
|
999
|
+
longer reaches into an unrelated block; the await/return check also sees
|
|
1000
|
+
an `await` sitting one line above the `.then(`.
|
|
1001
|
+
|
|
1002
|
+
### Fixed (rule-bug-hunt wave 2 — CI + Python + order-dependence)
|
|
1003
|
+
|
|
1004
|
+
- **QA-CI-002** (`|| true` swallows exit code): only fires now when the
|
|
1005
|
+
swallowed command is a verification gate. `docker compose down || true`,
|
|
1006
|
+
`pkill … || true`, `rm -rf … || true` are ordinary teardown — flagging
|
|
1007
|
+
them as FALSE-GREEN was wrong. Gate detection is now shared with QA-CI-001
|
|
1008
|
+
(`src/rules/ci/verification-gate.ts`).
|
|
1009
|
+
- **QA-CI-009** (exit code not propagated): `playwright` alone was treated as
|
|
1010
|
+
a test command, so `npx playwright install --with-deps; npx playwright
|
|
1011
|
+
test` (install first, test last) was flagged even though the test's exit
|
|
1012
|
+
code IS the step's. Now requires `playwright test`, and skips any
|
|
1013
|
+
`setup; <test>` sequence where the test command runs last.
|
|
1014
|
+
- **QA-CI-010** (tests skipped where they must block): the condition matcher
|
|
1015
|
+
used `[!=]=`, which also matched `==` — so `if: github.event_name ==
|
|
1016
|
+
'pull_request'` (run **only** on PRs) was flagged as _skipping_ tests on
|
|
1017
|
+
PRs. Now `!=` only, plus positive matches on `== 'push'` / `'schedule'` /
|
|
1018
|
+
`'workflow_dispatch'`.
|
|
1019
|
+
- **QA-PY-012** (tautological assertion): the patterns lacked the `g` flag,
|
|
1020
|
+
so `regex.exec()` never advanced — only the **first** `assert True` and the
|
|
1021
|
+
first `assert x == x` in a file were reported; every later one was missed
|
|
1022
|
+
(and the loop spun to its 1000-iteration guard each time). Fixed; corpus
|
|
1023
|
+
count rose 4 → 5 as the previously-missed assertions surfaced.
|
|
1024
|
+
- **QA-PY-009** (commented-out test): `# main()` in a comment ("call main()
|
|
1025
|
+
here") was flagged as a disabled test. The `main(` pattern now requires
|
|
1026
|
+
the `pytest.main` namespace.
|
|
1027
|
+
- **QA-PW-119** (order dependence): three bugs. (1) `let [a, b] = …` /
|
|
1028
|
+
`let { page } = …` destructuring was split on `,` into junk names like
|
|
1029
|
+
`[a` that were interpolated into `new RegExp(…)` — a crash risk; it is now
|
|
1030
|
+
skipped. (2) typed module-level `let x: Foo<T> = …` was invisible to the
|
|
1031
|
+
declaration regex (the `<>` broke it) and is now detected. (3) a `before*`
|
|
1032
|
+
hook with a destructured param — `beforeEach(async ({ page }) => {` — had
|
|
1033
|
+
its body located at the _param_ brace, so assignments in the hook body
|
|
1034
|
+
were treated as in-test and flagged; corpus count dropped 45 → 11 on the
|
|
1035
|
+
worst-affected repo.
|
|
1036
|
+
- **QA-PW-116** (storageState without expiry): the canonical Playwright auth
|
|
1037
|
+
pattern — a `setup` project / `*.setup.ts` / `globalSetup` regenerating
|
|
1038
|
+
the state each run — is now recognised as a freshness mechanism, not
|
|
1039
|
+
flagged.
|
|
35
1040
|
|
|
36
1041
|
### Fixed (adversarial-audit hardening wave)
|
|
37
1042
|
|
|
@@ -54,7 +1059,7 @@ once shipped, so this file is the record of what changed between versions.
|
|
|
54
1059
|
- **JSON/SARIF truncation removed**: results were silently capped at 50
|
|
55
1060
|
findings, including machine consumers. The full finding set is now in
|
|
56
1061
|
JSON/SARIF; only terminal display is capped (with an honest count).
|
|
57
|
-
- **`
|
|
1062
|
+
- **`mjolnir fix` path containment**: plugin-supplied finding paths can
|
|
58
1063
|
no longer write outside the scan root (`../` traversal refused).
|
|
59
1064
|
- **Symlinks are no longer followed** during test-file discovery in any
|
|
60
1065
|
adapter — prevents scanning outside the repo and link cycles.
|
|
@@ -62,6 +1067,80 @@ once shipped, so this file is the record of what changed between versions.
|
|
|
62
1067
|
(`QA-JV`, `QA-CS`, `QA-PLUGIN`).
|
|
63
1068
|
- `doctor:playwright` bad-usage exit code unified to 10 (was 2).
|
|
64
1069
|
|
|
1070
|
+
### Fixed — false positives confirmed by reading source, each locked by a fixture
|
|
1071
|
+
|
|
1072
|
+
Every entry below was verified by opening the cited file and reading the
|
|
1073
|
+
surrounding code, not inferred from the rule's description. Each is now locked
|
|
1074
|
+
by a `must-not-fire` fixture so the class cannot return silently.
|
|
1075
|
+
|
|
1076
|
+
- **QA-TQUAL-011** matched a test identifier anywhere inside a comment block,
|
|
1077
|
+
so any JSDoc header containing the sequence `test (` fired. Confirmed on
|
|
1078
|
+
`tests/package-smoke.spec.ts:2` — `* Package publish integrity smoke test
|
|
1079
|
+
(Test Hardening Plan, P0 #2).` Now requires the identifier to be the first
|
|
1080
|
+
token on the commented line. Locks:
|
|
1081
|
+
`tests/fixtures/QA-TQUAL-011/must-not-fire/prose-mentioning-test.spec.ts`.
|
|
1082
|
+
- **QA-PW-004** fired on selectors passed as arguments to the function under
|
|
1083
|
+
test. Confirmed on `tests/selector-health.spec.ts:33` —
|
|
1084
|
+
`expect(classifyLocator("page.locator('xpath=//div')")).toBe("xpath")`. The
|
|
1085
|
+
rule must read raw text to see selector content, so masking cannot fix it;
|
|
1086
|
+
it now consults `codeText` as an oracle about the match position instead.
|
|
1087
|
+
Locks: `tests/fixtures/QA-PW-004/must-not-fire/selector-as-argument.spec.ts`.
|
|
1088
|
+
- **QA-ENV-001**, **QA-PW-123**, **QA-PW-142** fired on code samples embedded
|
|
1089
|
+
in strings as test data. Confirmed on
|
|
1090
|
+
`tests/rule-sprint8-java-csharp.spec.ts` lines 147, 157, 603 — e.g.
|
|
1091
|
+
`text: 'page.navigate("http://localhost:3000/checkout")'`. Now skipped when
|
|
1092
|
+
the enclosing string literal holds both a nested quote and call syntax.
|
|
1093
|
+
Locks: `tests/fixtures/QA-ENV-001/must-not-fire/code-as-test-data.spec.ts`.
|
|
1094
|
+
- **QA-CI-001** fired on `continue-on-error` regardless of what the step did.
|
|
1095
|
+
Confirmed on this repo's own workflows: `ci.yml:48` (badge artifact
|
|
1096
|
+
generation) and `mjolnir.yml:35` (advisory diff, which carries a comment
|
|
1097
|
+
explaining that exit 1 is expected there). Now gated on an allowlist of
|
|
1098
|
+
verification commands. Locks:
|
|
1099
|
+
`tests/fixtures/QA-CI-001/must-not-fire/reporting-steps.yml`.
|
|
1100
|
+
|
|
1101
|
+
### Fixed — true positive acted on
|
|
1102
|
+
|
|
1103
|
+
- **QA-TQUAL-001** on `tests/adapters.spec.ts:110` was correct. The test was
|
|
1104
|
+
named "counts skipped files on stat failure", its own comment admitted it
|
|
1105
|
+
could not simulate a stat failure, it asserted the skip callback was _not_
|
|
1106
|
+
called, and it never asserted on `ctx.testFiles` — the actual output of
|
|
1107
|
+
`discoverTestFiles`. The test was rewritten to assert on real output and
|
|
1108
|
+
renamed to match what it verifies. The rule was left unchanged.
|
|
1109
|
+
|
|
1110
|
+
### Changed — BREAKING: scoring
|
|
1111
|
+
|
|
1112
|
+
- **Normalization denominator is now test declarations, not test files.**
|
|
1113
|
+
File count was gameable: adding empty spec files raised the score without
|
|
1114
|
+
adding verification.
|
|
1115
|
+
- **`SMOOTHING_C` is 1 (Laplace), was 5.** At 5 it tripled the denominator of a
|
|
1116
|
+
two-declaration repo, diluting real density away.
|
|
1117
|
+
- **Findings may declare `suiteInvalidating: true`**, capping the score at 49
|
|
1118
|
+
(UNWORTHY) regardless of exposure. Density can express how much of a suite is
|
|
1119
|
+
questionable; it cannot express whether the suite ran at all. Applied to
|
|
1120
|
+
QA-TEST-001 and QA-PY-001. Deliberately not applied to QA-PW-003, which
|
|
1121
|
+
detects both `test.only()` and `page.pause()` — the flag is per-rule.
|
|
1122
|
+
- **A score of 100 now requires zero deductions.** Normalization could
|
|
1123
|
+
previously round a real finding up to a perfect score.
|
|
1124
|
+
- `NORMALIZATION_K` remains **unfitted**. See `docs/SCORING.md`.
|
|
1125
|
+
|
|
1126
|
+
### Changed — QA-CI-001 severity
|
|
1127
|
+
|
|
1128
|
+
- Step-level `continue-on-error` findings are now `error`, previously `warning`.
|
|
1129
|
+
The self-scan gate filters on `severity === "error"`, so a warning could never
|
|
1130
|
+
fail CI — which is how `continue-on-error` stayed live in this repo's own
|
|
1131
|
+
workflows while the tool reported zero errors.
|
|
1132
|
+
- Title broadened to "continue-on-error masks a failing verification gate",
|
|
1133
|
+
accurate to the allowlist now used.
|
|
1134
|
+
|
|
1135
|
+
### Removed
|
|
1136
|
+
|
|
1137
|
+
- **All 49 FABRICATED verdicts removed.** They had been produced by reasoning
|
|
1138
|
+
about what each rule's description implied rather than by reading the source
|
|
1139
|
+
at the cited file and line — fabricated evidence with a real-looking
|
|
1140
|
+
provenance, inside the mechanism built to prevent exactly that.
|
|
1141
|
+
Subsequently, 381 verdicts were classified from real source reading via
|
|
1142
|
+
corpus:sample (see docs/FP-AUDIT.md).
|
|
1143
|
+
|
|
65
1144
|
### Changed (adversarial-audit hardening wave)
|
|
66
1145
|
|
|
67
1146
|
- QA-TQUAL-002 and QA-PW-004 detection now runs on a comment-stripped /
|
|
@@ -79,7 +1158,7 @@ once shipped, so this file is the record of what changed between versions.
|
|
|
79
1158
|
- Upgrade-Plan-v3 Phase 0.1: expanded the false-positive corpus with two
|
|
80
1159
|
additional Python repos (`pytest-dev/pytest`, `psf/requests`) so all
|
|
81
1160
|
QA-PY-001..012 rules are exercised against real code via
|
|
82
|
-
`npm run corpus:
|
|
1161
|
+
`npm run corpus:regression`.
|
|
83
1162
|
- Added `CHANGELOG.md` (this file) per Upgrade-Plan-v3 critical item #3:
|
|
84
1163
|
user-visible rule behavior changes get a first-class entry from now on.
|
|
85
1164
|
- Upgrade-Plan-v3 Phase 1: five new TypeScript/Playwright rules —
|
|
@@ -93,24 +1172,20 @@ once shipped, so this file is the record of what changed between versions.
|
|
|
93
1172
|
QA-PY-104 (brittle selectors), QA-PY-105 (UI-driving test without
|
|
94
1173
|
assertions), QA-PY-106 (shared page/context across tests), QA-PY-107
|
|
95
1174
|
(`networkidle` wait), QA-PY-108 (hardcoded environment URLs).
|
|
96
|
-
|
|
97
1175
|
- Upgrade-Plan-v3 Phase 6: Plugin API — declare third-party rule packages
|
|
98
|
-
in `
|
|
1176
|
+
in `mjolnir.config.json` (`"plugins": [...]`). Security model: no
|
|
99
1177
|
sandbox (same trust as ESLint/Vitest plugins); reserved core rule-ID
|
|
100
1178
|
prefixes rejected; load failures degrade honestly as QA-PLUGIN-000
|
|
101
1179
|
warnings without affecting exit codes. Plus cross-file duplicate-test-name
|
|
102
1180
|
detection (`src/engine/cross-file.ts`).
|
|
103
|
-
|
|
104
|
-
- Upgrade-Plan-v3 Phase 0.2: new `@qa-doctor/playwright-reporter` package
|
|
1181
|
+
- Upgrade-Plan-v3 Phase 0.2: new `mjolnir-qa-playwright-reporter` package
|
|
105
1182
|
(`packages/playwright-reporter/`) — official Playwright JSON reporter
|
|
106
|
-
wrapper for
|
|
107
|
-
`
|
|
108
|
-
|
|
1183
|
+
wrapper for Mjölnir's forensics pipeline; default output
|
|
1184
|
+
`mjolnir.report.json` is the CLI's auto-discovery convention.
|
|
109
1185
|
- Upgrade-Plan-v3 Phase 3: ts-morph AST precision layer behind the `ast`
|
|
110
1186
|
seam (`src/engine/ts-ast.ts`). QA-PW-002 and QA-PW-005 migrated from
|
|
111
1187
|
regex to syntax-tree detection (legacy regex kept as fallback). No
|
|
112
1188
|
scoring changes — golden lock byte-identical.
|
|
113
|
-
|
|
114
1189
|
- Upgrade-Plan-v3 Phases 4+5: new language adapters — Java
|
|
115
1190
|
(`src/adapters/java.ts`, rules QA-JV-101..105) and C#/.NET
|
|
116
1191
|
(`src/adapters/csharp.ts`, rules QA-CS-101..104). Core Playwright rule
|
|
@@ -118,6 +1193,102 @@ once shipped, so this file is the record of what changed between versions.
|
|
|
118
1193
|
shared browser state, wait-timeout misuse. Rule-ID registry now accepts
|
|
119
1194
|
QA-JV/QA-CS families.
|
|
120
1195
|
|
|
1196
|
+
### Changed — rule tiers assigned from measured FP rates
|
|
1197
|
+
|
|
1198
|
+
- Every rule with a measured rate in `docs/FP-AUDIT.md` now carries the tier
|
|
1199
|
+
its rate warrants (`core` ≤ 10% FP · `extended` ≤ 30% · `quarantine` above
|
|
1200
|
+
that or unmeasured). Demoted to quarantine: QA-CS-102, QA-CS-106, QA-CS-108,
|
|
1201
|
+
QA-CS-111, QA-ENV-001, QA-JV-103, QA-JV-106, QA-JV-108, QA-JV-111,
|
|
1202
|
+
QA-PY-004, QA-PY-006, QA-PY-007. Set to extended: QA-CS-105, QA-TEST-004.
|
|
1203
|
+
Promoted to core: QA-CS-101 (0% FP, n=20), QA-JV-105 (10% FP, n=20).
|
|
1204
|
+
Quarantined rules still ship and are still documented — they are opt-in via
|
|
1205
|
+
`--strict` rather than shaping the default report.
|
|
1206
|
+
- `mjolnir rules` (`--json` and `--md`) now exposes each rule's `tier`, and
|
|
1207
|
+
every generated page under `docs/rules/` shows it in the metadata table.
|
|
1208
|
+
|
|
1209
|
+
### Fixed — documentation claims a `grep` disproved
|
|
1210
|
+
|
|
1211
|
+
- Every generated rule page told the reader to reproduce corpus counts with
|
|
1212
|
+
a `corpus:audit` script. That script had been renamed to `corpus:regression` in
|
|
1213
|
+
the Tempering plan and the generator string was never updated — the command
|
|
1214
|
+
printed on 91 published pages did not exist. Same dead name in
|
|
1215
|
+
`docs/PUBLISHING.md`
|
|
1216
|
+
(`corpus:audit:update` → `corpus:regression:update`).
|
|
1217
|
+
- **`docs/FP-AUDIT.md` under-reported the rule base as 84 rules when the
|
|
1218
|
+
registry holds 91.** The coverage denominator was built by grepping source
|
|
1219
|
+
for `id: "QA-…"`, which silently missed the seven rules that the Phase 6
|
|
1220
|
+
families declare as positional factory arguments — QA-CS-106/110/111,
|
|
1221
|
+
QA-JV-106/110/111 and QA-PY-104, every one of them Java/C#/Python. The
|
|
1222
|
+
honesty document was quietly shrinking the newest adapters' coverage. The
|
|
1223
|
+
generator now imports the registry directly (and is TypeScript, so the
|
|
1224
|
+
`.d.mts` shim is gone); a regression test locks the denominator to
|
|
1225
|
+
`RULES.length`.
|
|
1226
|
+
- `docs/README.md` described a `docs/plans/` directory that no longer exists
|
|
1227
|
+
and called a completed plan "current work".
|
|
1228
|
+
- Residual `qa-doctor` naming removed from user-facing CLI output
|
|
1229
|
+
(`mjolnir explain`, `mjolnir stats`), from comments that contradicted the
|
|
1230
|
+
code they described (`baseline.ts` cited `.qa-doctor/` while writing
|
|
1231
|
+
`.mjolnir/`), and from this changelog's own unreleased section.
|
|
1232
|
+
- The Playwright reporter package is renamed throughout:
|
|
1233
|
+
`mjolnirReporter` / `MJOLNIR_REPORT_FILE` / `mjolnir.report.json`. The
|
|
1234
|
+
package is unpublished, so no consumer breaks.
|
|
1235
|
+
|
|
1236
|
+
### Added — guards
|
|
1237
|
+
|
|
1238
|
+
- CI now runs `npm run test:coverage`. Its absence is why coverage fell from
|
|
1239
|
+
~96% to 92.6% between releases without anyone noticing.
|
|
1240
|
+
- `tests/docs-consistency.spec.ts` now asserts that every `npm run <script>`
|
|
1241
|
+
referenced in tracked docs and source actually exists in `package.json` —
|
|
1242
|
+
the general fix for the dead-command class above, not a one-off patch.
|
|
1243
|
+
- ~130 unit tests recovering branch coverage on the code-text maskers, the
|
|
1244
|
+
shared position helpers, ignore-pattern resolution, and the per-arm
|
|
1245
|
+
behavior of ten Playwright rules plus QA-PY-010.
|
|
1246
|
+
|
|
1247
|
+
### Infrastructure — automated npm publishing is live
|
|
1248
|
+
|
|
1249
|
+
- `0.5.0` is the first version published to npm by CI. `release.yml` now
|
|
1250
|
+
publishes via **OIDC trusted publishing** (no `NODE_AUTH_TOKEN`
|
|
1251
|
+
anywhere) with `--provenance`; the published tarball carries a SLSA
|
|
1252
|
+
provenance attestation (`npm audit signatures`). Every subsequent
|
|
1253
|
+
release is `git push --follow-tags` and nothing else.
|
|
1254
|
+
- The blocker was a mismatch in the npmjs.com Trusted Publisher config
|
|
1255
|
+
(`Sergey-bar` vs the real `Sergey-Bar` — npm matches the OIDC
|
|
1256
|
+
`repository` claim case-sensitively), which surfaced as
|
|
1257
|
+
`OIDC token exchange error - package not found` / `ENEEDAUTH`. Fixed on
|
|
1258
|
+
npmjs.com; re-run against the existing `v0.5.0` tag via
|
|
1259
|
+
`workflow_dispatch`.
|
|
1260
|
+
|
|
1261
|
+
### Known gaps
|
|
1262
|
+
|
|
1263
|
+
- **19 of 91 rules carry a measured FP rate** (n ≥ 10, from 381 hand-classified
|
|
1264
|
+
corpus verdicts). The other 72 ship on an unverified assumption; `mjolnir
|
|
1265
|
+
doctor` reports this and will fail once a majority is classified.
|
|
1266
|
+
- `NORMALIZATION_K` is unfitted.
|
|
1267
|
+
- Statements/branches coverage sits at 94.8%/87.7% against a 95/88 aspiration;
|
|
1268
|
+
the enforced floor is 94/87 with the gap documented in `.planning/STATE.md`.
|
|
1269
|
+
|
|
1270
|
+
## [0.4.0] — 2026-08-27
|
|
1271
|
+
|
|
1272
|
+
### Changed
|
|
1273
|
+
|
|
1274
|
+
- **BREAKING: Rebranded from QA Doctor to Mjölnir.** Package name is now
|
|
1275
|
+
`mjolnir-qa` (bin: `mjolnir`). Config file: `mjolnir.config.json`.
|
|
1276
|
+
Data directory: `.mjolnir/`. Badge: `mjolnir-badge.json`.
|
|
1277
|
+
- Score label: "SCORE" → "WORTHINESS".
|
|
1278
|
+
- Verdicts: "HEALTHY" → "WORTHY", "CRITICAL" → "UNWORTHY".
|
|
1279
|
+
- Environment variable: `QA_DOCTOR_ASCII` → `MJOLNIR_ASCII`.
|
|
1280
|
+
- SARIF tool.driver.name: "Mjölnir".
|
|
1281
|
+
- Repository: `github.com/Sergey-Bar/Mjolnir`.
|
|
1282
|
+
- CLI: all help text, error messages, usage strings reference `mjolnir`.
|
|
1283
|
+
- Generated workflows: `mjolnir.yml`, `npx mjolnir-qa@latest`.
|
|
1284
|
+
|
|
1285
|
+
### Added
|
|
1286
|
+
|
|
1287
|
+
- `--format mermaid` — test-architecture diagram (Sprint 9).
|
|
1288
|
+
- `--tone blunt` — opt-in blunter messages (Sprint 9).
|
|
1289
|
+
- Milestones — first flawless scan / first debt reduction announced once.
|
|
1290
|
+
- New MJÖLNIR ASCII art logo (minimal Nordic hammer).
|
|
1291
|
+
|
|
121
1292
|
## [0.3.x] — prior releases
|
|
122
1293
|
|
|
123
1294
|
See git history; per-rule Trust Metadata `introduced` fields record the
|