testguard-cli 0.12.0 → 0.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,103 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.14.0] - 2026-09-22
11
+
12
+ Automated weekly release — everything merged since `v0.13.0`.
13
+
14
+ ### Changed
15
+
16
+ - fix: three defects the arid measurement found, presentational JSX set aside, and the claimspec signal bump (#164)
17
+ - docs: carry the 0.12 and 0.13 results into the brief and the prior-art record (#162)
18
+ - chore(homebrew): sha256 for v0.13.0 (#161)
19
+
20
+
21
+ ### Added
22
+
23
+ - **claimspec v1 evidence admits `persistence-payload-unasserted`** to
24
+ `defenders.signals`, the enum shared with every claimspec consumer. The
25
+ signal had lived only in the sweep document, so `probe` could not surface
26
+ why a save-path survivor was missed. Now a `survived` record whose fault
27
+ sits on the write path carries it for each resolved defender that mocks the
28
+ persistence layer, with the mocked `specifiers` and the file's assertion
29
+ `counts` (exact, partial, argument-free). The validator refuses it on any
30
+ other verdict, on a file that is not a resolved defender, or without its
31
+ reason — the same rule the sweep document already enforced on its
32
+ findings, and the sweep now repeats what the record says rather than
33
+ re-deriving it. Conformance carries a valid example and three invalid
34
+ cases. No consumer outside this repository validates claimspec evidence
35
+ today; DocGuard's adoption (docguard#420) is notified rather than broken.
36
+ - `sweep` sets aside a purely presentational JSX element — an icon, or a
37
+ static wrapper with no expression and no handler — instead of spending a
38
+ probe run on it, and says so: `selection.presentational` in the document, a
39
+ line in the report, and the remainder arithmetic now proves
40
+ proposed = probed + deferred + set aside. Measured on a 14-file UI diff:
41
+ 150 of 510 proposals, 29 of 30 probe slots and 7 of 10 survivors were of
42
+ this shape — Google's definition of an arid node, in a syntax Google never
43
+ mutated. Because the ordering learns from survival, those survivors were
44
+ also teaching it to rank the class higher; set-aside proposals never enter
45
+ the evidence, so they no longer can. Google's own five arid categories were
46
+ measured at under 1% of proposals and are not suppressed.
47
+
48
+ ### Fixed
49
+
50
+ - Python defender discovery lost the last name on an import line that ends in
51
+ a comment: `from pkg import cli, demo # noqa: E402` found only `cli`, and
52
+ `import pkg.demo # noqa` found nothing. On a real suite where `# noqa` is
53
+ the idiom of every src-layout test, 12 of 30 probed faults were reported
54
+ NOCOVER for modules whose tests import them — a gating verdict in the
55
+ forbidden direction. Comments are now stripped before names are read.
56
+ - File walking descended into `.claude/`, so a nested agent worktree under
57
+ `.claude/worktrees/` contributed another branch's tests as defenders of this
58
+ tree — files the scratch worktree cannot even contain, because the directory
59
+ is gitignored. `.claude` joins the tool-directory skip list, and the claim
60
+ that promised this now names it.
61
+ - The Python `statement-deleted` producer proposed on lines inside a bracket
62
+ opened earlier — `help="…")` inside an `add_argument(` — and none of those
63
+ faults compiled: 81 of 600 proposals on a real CLI (13.5%). The producer now
64
+ tracks bracket depth across lines (triple-quoted strings included, so a
65
+ docstring's parentheses cannot leave the rest of the file "inside") and
66
+ refuses the `pass` shapes there; dict entries and call arguments keep their
67
+ own producers. Re-measured: 7 of 526 (1.3%).
68
+
69
+ ### Changed
70
+
71
+ - The self-probe cost budget is 1650 s, from 1440. Six new claims (85 s on
72
+ the Node 22 PR leg, worst 22 s, all under the 100 s per-claim ceiling) took
73
+ that leg from 1367 s to 1457 s, over by 17 s. The v0.13.0 release job
74
+ measured 1113 s from scratch, but the PR leg runs the same claims 25-30%
75
+ slower and is where the ceiling is checked, so 1650 restores ~13% headroom
76
+ over the leg that binds. The per-claim ceiling is unchanged.
77
+ - The technical brief carries the 0.12 and 0.13 results it was missing: the
78
+ whole-surface provability measurement (32 of 37 files that write are defended
79
+ only by tests that mock the database) replaces the sample-level number in
80
+ Fig. 7; the ladder's second rung is now a shipped verb with a real finding
81
+ (a five-line concern found an admin route whose owner/manager check can be
82
+ forced to `false` with no test failing) rather than a description of intent;
83
+ and the first rung says its ordering learns from the project's own runs.
84
+ Re-rendered; still eight sheets from eight pages.
85
+ - `docs-canonical/PRIOR-ART.md` records the arid-node measurement instead of
86
+ a promise: every proposal from four sweeps over two of the maintainer's
87
+ repositories was classified by hand against Google's five arid categories,
88
+ and they account for under 1% of what the producers propose. No suppressor
89
+ is built for them. The same pass found what IS material — presentational
90
+ JSX taking 29 of 30 probe slots on a UI diff, and 13.5% of Python proposals
91
+ that cannot compile — and lists both as open, with numbers.
92
+ - `docs-canonical/PRIOR-ART.md` no longer lists the feedback loop and concerns
93
+ as unbuilt. The two gaps that remain are restated precisely: the loop learns
94
+ from verdicts rather than explicit developer clicks, and concerns have no
95
+ model-backed producer.
96
+
97
+ ## [0.13.0] - 2026-09-22
98
+
99
+ Automated weekly release — everything merged since `v0.12.0`.
100
+
101
+ ### Changed
102
+
103
+ - chore(homebrew): sha256 for v0.12.0 (#159)
104
+ - feat: persistence provability, and an ordering that learns from this project's own verdicts (#157)
105
+
106
+
10
107
  ## [0.12.0] - 2026-09-22
11
108
 
12
109
  Automated weekly release — everything merged since `v0.11.0`.
@@ -18,6 +115,53 @@ Automated weekly release — everything merged since `v0.11.0`.
18
115
  - chore(homebrew): sha256 for v0.11.0 (#153)
19
116
 
20
117
 
118
+ ### Added
119
+
120
+ - **Concerns** — one sentence a human writes that generates faults across a
121
+ whole surface, the rung between writing nothing and writing a claim per
122
+ behaviour. A concern says *where* to look and *which shapes of break* are
123
+ relevant: `"every write persists the data it was given"` targets the write
124
+ sites and applies the payload producers. `testguard concerns` lists them,
125
+ `sweep --concern <ID>` aims a sweep by one, and `--save-paths` is now sugar
126
+ for the built-in `SAVE-PERSISTS`.
127
+ - Two built-ins ship, and only two: an authorization concern would need a
128
+ heuristic for "which files check permissions", and a guess there is the
129
+ non-actionable noise that gets a check switched off. A project that knows
130
+ its own auth layer says so with a `glob` target in three lines.
131
+ - A project concern with the same id **replaces** a built-in, so a team can
132
+ retune `SAVE-PERSISTS` for an ORM this tool does not recognise without
133
+ forking anything. The replacement is reported, never silent.
134
+ - `urn:claimspec:v1:concerns`, with rules that refuse a glob target naming no
135
+ globs (it would match nothing and report clean) and an empty fault-class
136
+ list (a typo for "every class", every time).
137
+ - **No model is required, deliberately.** A concern's useful core is a named
138
+ scope plus a producer selection, which the mechanical producers already
139
+ satisfy — so every verifying command stays offline and NFR-01 is unchanged.
140
+ An LLM can later propose faults a line-oriented producer cannot see, as an
141
+ additional source for a concern that already works.
142
+
143
+ ### Added
144
+
145
+ - A save-path sweep now reports **what its defenders could prove**, before it
146
+ reports what they did: per file, whether every defender mocks the persistence
147
+ layer (call shape only), at least one does not (a real write is possible), or
148
+ there is no defender at all. When nothing is unmocked the report says so
149
+ outright — a clean probe there is not evidence of persistence, it is evidence
150
+ that nothing could have measured it. Measured on a real AI-authored
151
+ application: of 37 files that write to storage, 32 were defended only by
152
+ mocking tests.
153
+ - `scope.provability` on the sweep document, with a validator rule that every
154
+ target is classified exactly once.
155
+ - The candidate ordering now **learns from this project's own probed verdicts**
156
+ instead of only the shipped productivity prior. Every evidence record already
157
+ carries a `faultClass` and a `verdict`, so "how often does a fault of this
158
+ class survive *here*" is a tally rather than a new thing to capture. A sweep
159
+ persists its own evidence to `.testguard/sweep-evidence.json` — beside the
160
+ canonical one, never replacing it — so the next run learns from
161
+ machine-proposed faults, the same distribution the ranker orders. The sweep
162
+ document records `ordering.observed` and `ordering.sources`, because an
163
+ ordering nobody can trace is a number nobody should trust.
164
+
21
165
  ### Added
22
166
 
23
167
  - `sweep --save-paths` — sweep every file that writes to storage instead of the
package/README.md CHANGED
@@ -127,9 +127,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
127
127
  | npm | `npm i -D testguard-cli` then `npx testguard probe` |
128
128
  | pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
129
129
  | Homebrew | `brew tap raccioly/tap && brew install testguard` |
130
- | GitHub Action | `uses: raccioly/testguard@v0.12.0` — see [`action.yml`](./action.yml) |
130
+ | GitHub Action | `uses: raccioly/testguard@v0.14.0` — see [`action.yml`](./action.yml) |
131
131
  | pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-probe` |
132
- | GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.12.0/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
132
+ | GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.14.0/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
133
133
 
134
134
  Projects that set `min-release-age` in `.npmrc` cannot see a version published
135
135
  less than that many days ago (`ENOVERSIONS`); install that one with
@@ -234,6 +234,10 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
234
234
  static signal `mocked-never-asserted` — the cheapest blind-spot signal
235
235
  there is, and the exact signature of one escaped bug in the field
236
236
  reports. Silence it, visibly, with `// unasserted: <why>` above the mock.
237
+ Its sibling `persistence-payload-unasserted` is attached to a **survivor
238
+ on a write path** whose defender mocks the database: the test can prove
239
+ the call shape and never that the row landed, and the signal names the
240
+ mocked layer and the file's assertion mix (exact, partial, argument-free).
237
241
 
238
242
  Runners: **vitest**, **jest**, **Playwright** and **Python** (`--runner
239
243
  auto` picks the first of vitest, jest, python that resolves). A JavaScript
@@ -423,8 +427,9 @@ alarming: something was deliberately broken and not one test failed.
423
427
 
424
428
  ```
425
429
  swept 13 of 14 unclaimed changed files against origin/main.
426
- proposed 510 faults, probed 10 (cap 10), deferred 500.
427
- The 500 deferred are not a verdict: raise --cap, or sweep a smaller change.
430
+ proposed 510 faults, probed 10 (cap 10), deferred 350.
431
+ The 350 deferred are not a verdict: raise --cap, or sweep a smaller change.
432
+ 150 presentational elements (icons, static wrappers) set aside: nobody writes a test for those, and a survivor there would teach the ordering the wrong lesson.
428
433
 
429
434
  9 findings — a deliberate break that no test noticed:
430
435
 
@@ -451,6 +456,14 @@ Three rules make it safe to run on a repository that has never seen this tool:
451
456
  is reported and never gates — a tool that fails because its own guess was bad
452
457
  is a tool people switch off.
453
458
 
459
+ The ordering **learns from your own runs**. Every evidence record already
460
+ carries a fault class and a verdict, so "how often does a fault of this class
461
+ survive here" is a tally, not a new thing to collect — a sweep persists its
462
+ evidence and the next one reads it. A project with no evidence falls back to
463
+ the shipped prior and says so; one with its own records overrides it in
464
+ proportion to how many it has. The report names which, because an ordering
465
+ nobody can trace is a number nobody should trust.
466
+
454
467
  The cap and the ordering are Google's: their mutation service surfaces at most
455
468
  7 × |files| mutants per change and orders candidates on the measured
456
469
  productivity of their operator in similar context, which took their productive
@@ -466,6 +479,60 @@ A sweep is weaker evidence than a probe, and says so. Nobody stated that the
466
479
  behaviour mattered. It is stronger than nothing, which is what a repository
467
480
  with no claims has — and the survivors worth defending become the first claims.
468
481
 
482
+ #### Concerns: one sentence, two hundred screens
483
+
484
+ A claim names one promise precisely, which is why a repository with two hundred
485
+ screens never finishes writing them. A **concern** names a *kind* of promise and
486
+ says where to look for it:
487
+
488
+ ```bash
489
+ npx testguard-cli concerns # what this project can be swept by
490
+ npx testguard-cli sweep --concern ADMIN-GUARDS # aim a sweep at one
491
+ ```
492
+
493
+ ```json
494
+ {
495
+ "schemaVersion": 1,
496
+ "concerns": [
497
+ {
498
+ "id": "ADMIN-GUARDS",
499
+ "statement": "Every admin route refuses a caller without the admin role.",
500
+ "severity": "critical",
501
+ "targets": { "kind": "glob", "globs": ["src/app/api/admin/**"] },
502
+ "faultClasses": ["guard-removed", "condition-forced", "return-altered"]
503
+ }
504
+ ]
505
+ }
506
+ ```
507
+
508
+ Five lines, and the sweep reports:
509
+
510
+ ```
511
+ concern ADMIN-GUARDS matches 24 files. Swept 23.
512
+ 158 further proposals were outside this concern's fault classes.
513
+
514
+ SURVIVED src/app/api/admin/claims/[id]/route.ts
515
+ [line 41] Guard never triggers: `if (!membership || (membership.role !== 'owner'
516
+ && membership.role !== 'manager'))` becomes `if (false)`.
517
+ ```
518
+
519
+ The idea is Meta's: ACH has an engineer describe an area of concern in plain
520
+ text and gates every generated step by execution. What is taken here is the
521
+ **unit** — the concern as the thing a human writes — not their generator, and
522
+ **no model is required**. A concern's useful core is a named scope plus a
523
+ producer selection, which the mechanical producers already satisfy, so every
524
+ verifying command stays offline.
525
+
526
+ Two concerns ship built in, and only two. An authorization concern would need a
527
+ heuristic for *"which files check permissions"*, and a guess there is exactly
528
+ the noise that gets a check switched off — so `SAVE-PERSISTS` and
529
+ `CHANGED-CODE` are the defaults, and a project that knows its own auth layer
530
+ says so in three lines. Declaring a concern with a built-in's id replaces it,
531
+ and the replacement is reported rather than silent.
532
+
533
+ A concern is **not** a claim and never becomes one by itself. It says where to
534
+ look; the probe says what it found; a human states the sentence worth defending.
535
+
469
536
  #### Sweeping the save surface instead of the diff
470
537
 
471
538
  A diff cannot answer *"when the user clicks save, does it actually save?"* — it
@@ -494,10 +561,26 @@ denominator comes first and the document carries `writeSites` and
494
561
  AI-authored application: **12 of 26 probed persistence faults survived a fully
495
562
  green suite, and all 12 were a dropped payload field.**
496
563
 
497
- This is a *sample* of the surface, and says so. It does not yet prove a write
498
- reached storage — only that a test would notice if the payload changed.
499
- Read-back oracles are the next rung
500
- ([#154](https://github.com/raccioly/testguard/issues/154)).
564
+ It also states, before any verdict, **what the defenders could prove at all**:
565
+
566
+ ```
567
+ of those 37 files: 32 defended only by tests that mock the persistence layer,
568
+ 1 with an unmocked defender, 4 with no defender at all.
569
+ ```
570
+
571
+ That line is the read-back answer. A test that replaced the database can prove
572
+ the *call shape* and nothing beyond it — a `where` that matches no rows, a
573
+ rolled-back transaction and a rejected constraint all pass against a mock that
574
+ recorded the arguments and returned a plausible object. So rather than pretend
575
+ to measure persistence, TestGuard states the limit, the same way `nocover` says
576
+ "no test imports this" instead of guessing.
577
+
578
+ When every defender mocks the layer, the report says so outright: *nothing in
579
+ this suite can prove a write reached storage*. A clean probe there is not
580
+ evidence of persistence — it is evidence that nothing could have measured it.
581
+
582
+ `unmocked` is deliberately weaker than "proves persistence": a test that does
583
+ not replace the database may simply never reach it. Possible is the honest word.
501
584
 
502
585
  ### Every change needs a claim
503
586
 
@@ -580,12 +663,12 @@ detected base that does not resolve is a warning for `status` and `brief`
580
663
  # GitHub Actions
581
664
  - uses: actions/checkout@v4
582
665
  with: { fetch-depth: 0 }
583
- - uses: raccioly/testguard@v0.12.0
666
+ - uses: raccioly/testguard@v0.14.0
584
667
  with: { command: gate }
585
668
 
586
669
  # GitLab CI — the component-shaped template: gate + probe, brief as an artifact and, opted in, as a merge-request note
587
670
  include:
588
- - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.12.0/packaging/gitlab/testguard.gitlab-ci.yml'
671
+ - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.14.0/packaging/gitlab/testguard.gitlab-ci.yml'
589
672
  inputs: { dir: backend, post_note: true } # post_note needs TESTGUARD_GITLAB_TOKEN (api scope); one note, updated in place
590
673
  ```
591
674
 
@@ -903,7 +986,8 @@ parameter-derived argument swapped for `None`, a key dropped from a payload
903
986
  `dict` or an allow-list. Two differences are deliberate. A statement is
904
987
  removed by replacing it with `pass`, never by deleting the line, because a
905
988
  block whose only statement is gone is an `IndentationError`; and a line that
906
- leaves a bracket open (`COLOURS = {`) is never removed at all. Both exist
989
+ leaves a bracket open (`COLOURS = {`), or sits inside one opened on an earlier
990
+ line (`help="…"` in an `add_argument(`), is never removed at all. Both exist
907
991
  because a fault that cannot compile is a `fault-invalid` verdict — a probe run
908
992
  spent saying nothing about the tests. The two JSX shapes (`element-removed`,
909
993
  `handler-dropped`) have no Python meaning and are absent rather than faked.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testguard-cli",
3
- "version": "0.12.0",
3
+ "version": "0.14.0",
4
4
  "description": "Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -343,6 +343,14 @@ the suppress-up-to-count semantics are identical.
343
343
  tool then records `unasserted-annotated` with the reason — silenced, never
344
344
  hidden. A file that mocks the subject is never a discovered defender: a
345
345
  mock cannot detect a fault in what it replaced.
346
+ `persistence-payload-unasserted`: a resolved defender mocks the persistence
347
+ layer, so it can prove the call shape and never that a row landed; the
348
+ reason names the layer and the file's assertion mix. It is attached only to
349
+ a **survived** record whose fault sits on the write path — a field of a
350
+ persisted payload or the write call itself — because there it says why the
351
+ survivor was missed, and anywhere else it says nothing. A conforming
352
+ document never carries it on another verdict, on a file that is not a
353
+ resolved defender, or without its reason.
346
354
 
347
355
  ## Claimed source surface
348
356
 
@@ -7,7 +7,7 @@ import { reproduces, decimals, MAX_PLACES } from './wilson.mjs';
7
7
 
8
8
  const schemaDir = join(dirname(fileURLToPath(import.meta.url)), '..', 'schemas');
9
9
 
10
- export const KINDS = Object.freeze(['claims', 'evidence', 'baseline', 'ignore', 'calibration', 'brief', 'status', 'gate', 'replay', 'sweep']);
10
+ export const KINDS = Object.freeze(['claims', 'evidence', 'baseline', 'ignore', 'calibration', 'brief', 'status', 'gate', 'replay', 'sweep', 'concerns']);
11
11
  /**
12
12
  * How a document says it tried to falsify its claims. Absent means
13
13
  * `fault-injection`, so every document written before the field existed is
@@ -144,6 +144,16 @@ const semantic = {
144
144
  if (s.signal === 'target-attribute-patched') {
145
145
  if (!s.reason) errors.push({ path: `${p}/defenders/signals`, message: `target-attribute-patched on ${s.file} requires the patched attributes as its reason` });
146
146
  if (!r.defenders.resolved.includes(s.file)) errors.push({ path: `${p}/defenders/signals`, message: `${s.file} patches attributes of the subject but is not a resolved defender` });
147
+ } else if (s.signal === 'persistence-payload-unasserted') {
148
+ // The same rule the sweep document enforces on its findings: the
149
+ // signal explains a SURVIVOR on the write path. A record does not
150
+ // carry the fault's line, so the write-path half is the producer's
151
+ // to keep (`persistenceSignalsFor`); the verdict half is checked here.
152
+ // The file mocks the persistence layer, not the subject, so it is a
153
+ // resolved defender and never a mocking one.
154
+ if (!s.reason) errors.push({ path: `${p}/defenders/signals`, message: `persistence-payload-unasserted on ${s.file} requires the mocked layer and assertion mix as its reason` });
155
+ if (r.verdict !== 'survived') errors.push({ path: `${p}/defenders/signals`, message: `persistence-payload-unasserted explains a survivor; on a ${r.verdict} record it explains nothing` });
156
+ if (!r.defenders.resolved.includes(s.file)) errors.push({ path: `${p}/defenders/signals`, message: `${s.file} mocks the persistence layer but is not a resolved defender` });
147
157
  } else if (!(r.defenders.mocking ?? []).includes(s.file)) {
148
158
  errors.push({ path: `${p}/defenders/signals`, message: `${s.file} carries a mock signal but is not listed in defenders.mocking` });
149
159
  }
@@ -305,12 +315,36 @@ const semantic = {
305
315
  return errors;
306
316
  },
307
317
 
318
+ concerns(doc) {
319
+ const errors = [];
320
+ const seen = new Set();
321
+ doc.concerns.forEach((c, i) => {
322
+ if (seen.has(c.id)) errors.push({ path: `/concerns/${i}/id`, message: `duplicate concern id "${c.id}"` });
323
+ seen.add(c.id);
324
+ // A glob target with no globs matches nothing, which reads as "this
325
+ // concern found no problems" rather than "this concern was never aimed".
326
+ if (c.targets?.kind === 'glob' && !(c.targets.globs?.length > 0)) {
327
+ errors.push({ path: `/concerns/${i}/targets/globs`, message: `concern "${c.id}" targets a glob but names none; it would match nothing and report clean` });
328
+ }
329
+ if (c.targets?.kind !== 'glob' && c.targets?.globs) {
330
+ errors.push({ path: `/concerns/${i}/targets/globs`, message: `concern "${c.id}" names globs but does not target them` });
331
+ }
332
+ // An empty list is not "every class"; it is a concern that can never
333
+ // propose anything, which is a typo for `null` every time.
334
+ if (Array.isArray(c.faultClasses) && c.faultClasses.length === 0) {
335
+ errors.push({ path: `/concerns/${i}/faultClasses`, message: `concern "${c.id}" allows no fault class; omit the field for every class` });
336
+ }
337
+ });
338
+ return errors;
339
+ },
340
+
308
341
  sweep(doc) {
309
342
  const errors = [];
310
343
  const s = doc.selection;
311
344
  // A cap that hides its remainder is a coverage claim nobody made, so the
312
345
  // arithmetic that proves nothing was dropped is checked, not trusted.
313
- if (s.proposed !== s.selected + s.deferred) errors.push({ path: '/selection/proposed', message: `proposed (${s.proposed}) must equal selected (${s.selected}) + deferred (${s.deferred})` });
346
+ const setAside = s.presentational ?? 0;
347
+ if (s.proposed !== s.selected + s.deferred + setAside) errors.push({ path: '/selection/proposed', message: `proposed (${s.proposed}) must equal selected (${s.selected}) + deferred (${s.deferred})${setAside ? ` + presentational (${setAside})` : ''}` });
314
348
  if (s.selected > s.cap) errors.push({ path: '/selection/selected', message: `selected (${s.selected}) exceeds the cap (${s.cap})` });
315
349
  if (doc.scope.swept > doc.scope.targets) errors.push({ path: '/scope/swept', message: `swept (${doc.scope.swept}) cannot exceed targets (${doc.scope.targets})` });
316
350
  // The counters that only mean something in one mode must not appear in the
@@ -318,10 +352,17 @@ const semantic = {
318
352
  // measured, and its absence on a save-paths sweep hides the denominator
319
353
  // entirely — which is the whole reason that mode exists.
320
354
  const saves = doc.scope.mode === 'save-paths';
321
- for (const k of ['writeSites', 'payloadFields']) {
355
+ for (const k of ['writeSites', 'payloadFields', 'provability']) {
322
356
  if (saves && doc.scope[k] === undefined) errors.push({ path: `/scope/${k}`, message: `a save-paths sweep must report ${k}: the surface is the denominator its findings are read against` });
323
357
  if (!saves && doc.scope[k] !== undefined) errors.push({ path: `/scope/${k}`, message: `${k} belongs to a save-paths sweep; a changed sweep did not measure the write surface` });
324
358
  }
359
+ if (saves && doc.scope.provability) {
360
+ const pv = doc.scope.provability;
361
+ const total = pv.mocked + pv.unmocked + pv.none;
362
+ // Every target is classified exactly once. A surface whose provability
363
+ // does not account for all of it has a bucket nobody looked in.
364
+ if (total !== doc.scope.targets) errors.push({ path: '/scope/provability', message: `provability accounts for ${total} files; targets is ${doc.scope.targets}` });
365
+ }
325
366
  if (saves && doc.scope.writeSites !== undefined && doc.scope.targets > doc.scope.writeSites) {
326
367
  errors.push({ path: '/scope/targets', message: `targets (${doc.scope.targets}) exceeds writeSites (${doc.scope.writeSites}); every target is a file with at least one write` });
327
368
  }
@@ -343,6 +384,12 @@ const semantic = {
343
384
  const notKilled = probed - (doc.counts.killed ?? 0);
344
385
  if (doc.findings.length !== notKilled) errors.push({ path: '/findings', message: `${doc.findings.length} findings for ${notKilled} records that were not killed; findings are every non-killed record` });
345
386
 
387
+ // An ordering that claims to have learned from something must name it.
388
+ if (doc.ordering) {
389
+ if (doc.ordering.observed > 0 && doc.ordering.sources.length === 0) errors.push({ path: '/ordering/sources', message: 'an ordering learned from evidence must name the documents it was learned from' });
390
+ if (doc.ordering.observed === 0 && doc.ordering.sources.length > 0) errors.push({ path: '/ordering/observed', message: 'sources are named but nothing was observed; the ordering used the shipped prior' });
391
+ }
392
+
346
393
  const gating = doc.findings.filter((f) => f.verdict === 'survived' || f.verdict === 'nocover').length;
347
394
  if (gating > 0 && doc.exitCode !== 1) errors.push({ path: '/exitCode', message: 'a survived or nocover finding must exit 1; a sweep never passes over a deliberate break nothing noticed' });
348
395
  if (gating === 0 && doc.exitCode !== 0) errors.push({ path: '/exitCode', message: 'exit 1 requires a survived or nocover finding; a sweep does not fail on its own unanchorable proposal' });
@@ -0,0 +1,41 @@
1
+ {
2
+ "$id": "urn:claimspec:v1:concerns",
3
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
4
+ "title": "claimspec v1 — concerns",
5
+ "description": "A concern is one sentence a human writes that generates faults across a whole surface: it says WHERE to look and WHICH shapes of break are relevant. It is the rung between writing nothing and writing a claim per behaviour. A concern is never a claim and never becomes one by itself — it says where to look, the probe says what it found, and a human states the sentence worth defending. Nothing in this document is a promise about the code; only a claim is that.",
6
+ "type": "object",
7
+ "required": ["schemaVersion", "concerns"],
8
+ "additionalProperties": false,
9
+ "properties": {
10
+ "$schema": { "type": "string" },
11
+ "schemaVersion": { "const": 1 },
12
+ "concerns": {
13
+ "type": "array",
14
+ "items": {
15
+ "type": "object",
16
+ "required": ["id", "statement"],
17
+ "additionalProperties": false,
18
+ "properties": {
19
+ "id": { "$ref": "urn:claimspec:v1:common#/$defs/identifier", "description": "Declaring an id that matches a built-in REPLACES it, which is how a project retunes a default for an ORM or a layout this tool does not recognise. The replacement is reported, never silent." },
20
+ "statement": { "type": "string", "minLength": 1, "description": "The sentence, in plain language. It describes a KIND of promise, not one behaviour: \"every write persists the data it was given\", not \"the register action stores password_hash\". The second is a claim." },
21
+ "severity": { "$ref": "urn:claimspec:v1:common#/$defs/severity" },
22
+ "targets": {
23
+ "type": "object",
24
+ "required": ["kind"],
25
+ "additionalProperties": false,
26
+ "description": "Where this concern applies. `write-sites` asks the write enumerator. `glob` matches paths. `changed` defers to whatever diff the caller is already measuring — a concern does not re-derive what the gate computed.",
27
+ "properties": {
28
+ "kind": { "enum": ["write-sites", "glob", "changed"] },
29
+ "globs": { "type": "array", "items": { "type": "string", "minLength": 1 } }
30
+ }
31
+ },
32
+ "faultClasses": {
33
+ "type": ["array", "null"],
34
+ "items": { "$ref": "urn:claimspec:v1:common#/$defs/faultClass" },
35
+ "description": "Which producers are relevant. `null` or absent means every class, the honest default for a concern with no shape to narrow by. Narrowing is what makes one sentence useful across two hundred files: a persistence concern that also reported every altered return value would bury the payload findings it exists to surface."
36
+ }
37
+ }
38
+ }
39
+ }
40
+ }
41
+ }
@@ -547,7 +547,7 @@
547
547
  }
548
548
  },
549
549
  "signals": {
550
- "description": "Static, annotation-grade signals about the defenders. They never change a verdict. `mocked-never-asserted`: the file mocks the subject and never asserts on anything imported from it. `unasserted-annotated`: the same, but the author annotated the mock with a reason (`// unasserted: <why>`), recorded here \u2014 silenced, never hidden. `target-attribute-patched`: a Python defender that patches individual attributes of the subject (`patch(\"pkg.mod.fn\")`) rather than the module. Unlike a JavaScript module mock it does NOT disqualify the file \u2014 a fault anywhere else in the module is still detectable \u2014 so the file stays in `resolved`; `reason` names the patched attributes, which are where that test is least likely to notice anything.",
550
+ "description": "Static, annotation-grade signals about the defenders. They never change a verdict. `mocked-never-asserted`: the file mocks the subject and never asserts on anything imported from it. `unasserted-annotated`: the same, but the author annotated the mock with a reason (`// unasserted: <why>`), recorded here \u2014 silenced, never hidden. `target-attribute-patched`: a Python defender that patches individual attributes of the subject (`patch(\"pkg.mod.fn\")`) rather than the module. Unlike a JavaScript module mock it does NOT disqualify the file \u2014 a fault anywhere else in the module is still detectable \u2014 so the file stays in `resolved`; `reason` names the patched attributes, which are where that test is least likely to notice anything. `persistence-payload-unasserted`: a resolved defender that mocks the persistence layer (`vi.mock('@/lib/prisma')`) and so can prove the call shape but never that a row landed; `reason` names the mocked layer and the file's assertion mix (exact, partial, argument-free), `counts` carries the mix as numbers and `specifiers` the mocked module paths. It belongs only to a `survived` record whose fault sits on the write path \u2014 a field of a persisted payload, or the write call itself \u2014 because there it explains why the survivor was missed and anywhere else it explains nothing. Measured on a real application: 12 of 12 survivors on the save path were a dropped payload field, and every one of their defenders asserted with a partial matcher.",
551
551
  "type": "array",
552
552
  "items": {
553
553
  "type": "object",
@@ -563,13 +563,46 @@
563
563
  "enum": [
564
564
  "mocked-never-asserted",
565
565
  "unasserted-annotated",
566
- "target-attribute-patched"
566
+ "target-attribute-patched",
567
+ "persistence-payload-unasserted"
567
568
  ]
568
569
  },
569
570
  "reason": {
570
571
  "type": "string",
571
572
  "minLength": 1,
572
573
  "maxLength": 512
574
+ },
575
+ "specifiers": {
576
+ "description": "persistence-payload-unasserted only: the persistence-layer module specifiers the file mocks.",
577
+ "type": "array",
578
+ "items": {
579
+ "type": "string",
580
+ "minLength": 1
581
+ }
582
+ },
583
+ "counts": {
584
+ "description": "persistence-payload-unasserted only: the file's call-shape assertions by kind. `exact` is the only kind a dropped field can fail.",
585
+ "type": "object",
586
+ "required": [
587
+ "exact",
588
+ "partial",
589
+ "argless"
590
+ ],
591
+ "additionalProperties": false,
592
+ "properties": {
593
+ "exact": {
594
+ "type": "integer",
595
+ "minimum": 0
596
+ },
597
+ "partial": {
598
+ "type": "integer",
599
+ "minimum": 0
600
+ },
601
+ "argless": {
602
+ "type": "integer",
603
+ "minimum": 0
604
+ }
605
+ }
573
606
  }
574
607
  },
575
608
  "additionalProperties": false
@@ -62,6 +62,10 @@
62
62
  ],
63
63
  "description": "What the sweep was pointed at. `changed` proposes for the changed source files that carry no claim. `save-paths` proposes for every file that writes to storage, because a diff cannot answer how many save paths exist — and a clean report over an unstated denominator is a sample of unknown size, not a guarantee."
64
64
  },
65
+ "concern": {
66
+ "$ref": "urn:claimspec:v1:common#/$defs/identifier",
67
+ "description": "The concern this sweep was aimed by, when one was. A concern narrows both the targets and the fault classes, so the same surface swept under two concerns is two different questions."
68
+ },
65
69
  "changed": {
66
70
  "type": "integer",
67
71
  "minimum": 0,
@@ -107,6 +111,30 @@
107
111
  "type": "integer",
108
112
  "minimum": 0,
109
113
  "description": "save-paths only: payload keys sitting on their own line, which is what a field-dropped fault can anchor to."
114
+ },
115
+ "provability": {
116
+ "type": "object",
117
+ "required": [
118
+ "mocked",
119
+ "unmocked",
120
+ "none"
121
+ ],
122
+ "additionalProperties": false,
123
+ "description": "save-paths only: what the defenders of this surface are CAPABLE of proving, before anything is probed. `mocked` — every defender replaces the persistence layer, so it can prove the call shape and never that the row landed. `unmocked` — at least one does not, so a real write is at least possible (not a promise that one happens). `none` — no defender at all. On a surface that is entirely `mocked`, a clean probe is not evidence of persistence; it is evidence that nothing could have measured it.",
124
+ "properties": {
125
+ "mocked": {
126
+ "type": "integer",
127
+ "minimum": 0
128
+ },
129
+ "unmocked": {
130
+ "type": "integer",
131
+ "minimum": 0
132
+ },
133
+ "none": {
134
+ "type": "integer",
135
+ "minimum": 0
136
+ }
137
+ }
110
138
  }
111
139
  }
112
140
  },
@@ -165,6 +193,16 @@
165
193
  }
166
194
  }
167
195
  }
196
+ },
197
+ "outOfScope": {
198
+ "type": "integer",
199
+ "minimum": 0,
200
+ "description": "Proposals the concern's fault classes excluded. Reported so a narrow concern cannot look like a quiet one: the producers found these, and this concern is not about them."
201
+ },
202
+ "presentational": {
203
+ "type": "integer",
204
+ "minimum": 0,
205
+ "description": "Proposals set aside because they remove a purely presentational JSX element — an icon, or a static wrapper with no expression and no handler. Nobody writes a test for those, and a survivor on one would teach the survival-learned ordering to prefer the class. Counted in `proposed` and in `byClass`; never probed, never a finding. Absent when zero."
168
206
  }
169
207
  }
170
208
  },
@@ -305,6 +343,29 @@
305
343
  "items": {
306
344
  "type": "object"
307
345
  }
346
+ },
347
+ "ordering": {
348
+ "type": "object",
349
+ "required": [
350
+ "observed",
351
+ "sources"
352
+ ],
353
+ "additionalProperties": false,
354
+ "description": "Where the candidate ordering came from. `observed` is how many of this project's OWN probed faults informed it; 0 means the shipped productivity prior was used unchanged. An ordering nobody can trace is a number nobody should trust.",
355
+ "properties": {
356
+ "observed": {
357
+ "type": "integer",
358
+ "minimum": 0
359
+ },
360
+ "sources": {
361
+ "type": "array",
362
+ "items": {
363
+ "type": "string",
364
+ "minLength": 1
365
+ },
366
+ "description": "The evidence documents the counts were read from."
367
+ }
368
+ }
308
369
  }
309
370
  }
310
371
  }