testguard-cli 0.9.4 → 0.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.10.1] - 2026-09-19
11
+
12
+ Automated weekly release — everything merged since `v0.10.0`.
13
+
14
+ ### Changed
15
+
16
+ - fix(probe): preserve repeated claim selections (#139)
17
+ - chore(homebrew): sha256 for v0.10.0 (#138)
18
+
19
+
20
+ ### Fixed
21
+
22
+ - Repeated `probe --claim` flags no longer collapse silently to the last value.
23
+ Repeated and comma-separated ids are flattened in first-seen order,
24
+ duplicates are collapsed, empty selections fail, and both human and JSON
25
+ partial-run output show requested versus probed ids and counts. Probe JSON is
26
+ now validated against the published status schema before it is emitted. The
27
+ singular `scaffold` and `admit` forms reject ambiguous repeated values.
28
+
29
+ ## [0.10.0] - 2026-09-19
30
+
31
+ Automated weekly release — everything merged since `v0.9.4`.
32
+
33
+ ### Changed
34
+
35
+ - feat(status): expose claimed source surface (#135)
36
+ - chore(homebrew): sha256 for v0.9.4 (#136)
37
+
38
+
39
+ ### Added
40
+
41
+ - `status` now makes the claim denominator visible: claimed versus current
42
+ source modules, coverage of the 20 highest-churn modules over a bounded
43
+ 200-commit window, and specific unclaimed modules ranked by churn with
44
+ explicit path-risk signals. A clean project whose majority of modules are
45
+ still unclaimed is directed to scaffold the highest-churn gap. The existing
46
+ co-authorship caveat is also printed by both `status` and `probe`, rather
47
+ than remaining available only in JSON and the agent brief.
48
+
10
49
  ## [0.9.4] - 2026-09-19
11
50
 
12
51
  Automated weekly release — everything merged since `v0.9.3`.
package/README.md CHANGED
@@ -127,9 +127,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
127
127
  | npm | `npm i -D testguard-cli` then `npx testguard probe` |
128
128
  | pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
129
129
  | Homebrew | `brew tap raccioly/tap && brew install testguard` |
130
- | GitHub Action | `uses: raccioly/testguard@v0.9.4` — see [`action.yml`](./action.yml) |
130
+ | GitHub Action | `uses: raccioly/testguard@v0.10.1` — see [`action.yml`](./action.yml) |
131
131
  | pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-probe` |
132
- | GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.9.4/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
132
+ | GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.10.1/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
133
133
 
134
134
  Projects that set `min-release-age` in `.npmrc` cannot see a version published
135
135
  less than that many days ago (`ENOVERSIONS`); install that one with
@@ -279,8 +279,12 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
279
279
  because it fails when the claim is false.
280
280
 
281
281
  Practical loop: first pass `--no-escalate` (escalation re-runs the whole
282
- suite N times per survivor); iterate on one claim with `--claim <ID>` and
283
- either `--include-dirty` or `--in-place` (only fault target files must be
282
+ suite N times per survivor); iterate on one claim with `--claim <ID>`, or
283
+ select a batch with commas or repeated flags (`--claim A,B --claim C`).
284
+ Every occurrence is retained in first-seen order, duplicates are collapsed,
285
+ and partial human/JSON output names both the requested and probed counts, so
286
+ a green subset cannot masquerade as the selection the operator asked for.
287
+ Use either `--include-dirty` or `--in-place` (only fault target files must be
284
288
  clean there; test files may be dirty), optionally `--confirm 1` for a fast
285
289
  **provisional** signal — verdicts print with a `?`, evidence goes to
286
290
  `evidence-provisional.json`, and `baseline` refuses it; final pass with
@@ -469,12 +473,12 @@ detected base that does not resolve is a warning for `status` and `brief`
469
473
  # GitHub Actions
470
474
  - uses: actions/checkout@v4
471
475
  with: { fetch-depth: 0 }
472
- - uses: raccioly/testguard@v0.9.4
476
+ - uses: raccioly/testguard@v0.10.1
473
477
  with: { command: gate }
474
478
 
475
479
  # GitLab CI — the component-shaped template: gate + probe, brief as an artifact and, opted in, as a merge-request note
476
480
  include:
477
- - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.4/packaging/gitlab/testguard.gitlab-ci.yml'
481
+ - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.10.1/packaging/gitlab/testguard.gitlab-ci.yml'
478
482
  inputs: { dir: backend, post_note: true } # post_note needs TESTGUARD_GITLAB_TOKEN (api scope); one note, updated in place
479
483
  ```
480
484
 
@@ -633,9 +637,17 @@ things make that safe:
633
637
  *should* ship with its regression test — but a repository where every kill
634
638
  is co-authored has no independent verification, whatever its claim
635
639
  verification rate says. It is a **signal**: ranking reads it, verdicts never
636
- do, and the brief says it in one line. Nothing else in this category
640
+ do, and `probe`, `status` and the brief all say it in one line. Nothing else in this category
637
641
  measures it, and agents saturate the tests they can see
638
642
  ([SpecBench](https://arxiv.org/abs/2605.21384)).
643
+ - **Claimed surface is visible.** A clean probe is only about claims that
644
+ exist. `status` therefore reports claimed source modules over the current
645
+ module denominator, how many of the 20 highest-churn modules are claimed,
646
+ and the concrete unclaimed modules worth examining next over a bounded
647
+ 200-commit window. Churn and path-risk signals stay separate; there is no
648
+ synthetic health score. When most modules remain unclaimed and the existing
649
+ evidence is clean, the next action expands the denominator instead of
650
+ congratulating the same small set of claims again.
639
651
  - **Gaming is visible.** The cheapest way to make a survivor disappear is to
640
652
  weaken its fault, not to write a test. Evidence records every fault's
641
653
  content hash; `status` lists any fault edited after it survived, with its
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testguard-cli",
3
- "version": "0.9.4",
3
+ "version": "0.10.1",
4
4
  "description": "Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -344,6 +344,27 @@ the suppress-up-to-count semantics are identical.
344
344
  hidden. A file that mocks the subject is never a discovered defender: a
345
345
  mock cannot detect a fault in what it replaced.
346
346
 
347
+ ## Claimed source surface
348
+
349
+ `status` reports the denominator that `probe` cannot: current, non-test source
350
+ modules with a supported extension and outside the change gate's documented
351
+ default exclusions. A module is claimed when at least one fault anchors to it.
352
+ The document carries the claimed, unclaimed and total module counts.
353
+
354
+ The report also names concrete unclaimed modules. It counts touches over at
355
+ most the latest 200 commits, reports how many of the 20 highest-churn modules
356
+ are claimed, and lists up to ten unclaimed modules in descending churn order.
357
+ Security- and money-shaped path names are explicit tie-breaking signals, never
358
+ semantic claims about a file. Churn, path risk and claim coverage remain raw
359
+ facts: they are never collapsed into one health score.
360
+
361
+ When existing evidence is clean but more source modules are unclaimed than
362
+ claimed, the next action is to scaffold the highest-churn unclaimed module.
363
+ Earlier correctness states still win: invalid anchors, stale evidence and
364
+ unproven faults are repaired before expanding the denominator. Without git
365
+ history the counts and concrete files remain available, while the document
366
+ says history is unavailable and reports zero commits read.
367
+
347
368
  ## Claim coverage of a change
348
369
 
349
370
  `probe` asks whether the tests defend the claims that exist. It says nothing
@@ -415,6 +436,16 @@ new anchor is a semantic edit to the claim and must be reviewed and re-probed.
415
436
  `unclaimed-changes`, when a change reference is known). Its next action is
416
437
  `repair-fault`, naming the first fault and `testguard claims --check-anchors`.
417
438
 
439
+ ## Partial probe scope
440
+
441
+ Every `--claim` occurrence contributes to the selection. Comma-separated ids
442
+ inside each occurrence are flattened in first-seen order and duplicates are
443
+ collapsed. An explicitly empty selection is a usage error, never a successful
444
+ zero-record probe. Partial human output and the `probe --json` run wrapper name
445
+ the requested and actually probed claim ids and counts. Commands whose
446
+ `--claim` is singular (`scaffold` and `admit`) reject repeated or comma-separated
447
+ values instead of silently choosing one.
448
+
418
449
  ## Severity floor
419
450
 
420
451
  `--severity <level>` gates only findings whose claim severity is at or above
@@ -244,6 +244,19 @@ const semantic = {
244
244
  if (doc.invalidFaults?.length > 0 && doc.state !== 'invalid-anchors') errors.push({ path: '/state', message: 'invalid fault anchors are hidden behind a later state; invalid-anchors precedes every evidence state' });
245
245
  if (doc.evidenceSource && !doc.evidenceHead) errors.push({ path: '/evidenceHead', message: 'evidence was read, so the commit it describes must be recorded' });
246
246
  if (doc.changes && doc.changes.uncovered.length > 0 && !['no-claims', 'unclaimed-changes'].includes(doc.state)) errors.push({ path: '/state', message: 'uncovered changed files are hidden behind a later state; unclaimed-changes precedes every evidence state' });
247
+ if (doc.surface) {
248
+ if (doc.surface.claimedModules + doc.surface.unclaimedModules !== doc.surface.sourceModules) errors.push({ path: '/surface', message: 'claimedModules + unclaimedModules must equal sourceModules' });
249
+ if (doc.surface.highChurn.claimed > doc.surface.highChurn.modules) errors.push({ path: '/surface/highChurn/claimed', message: 'claimed high-churn modules cannot exceed the high-churn window' });
250
+ if (!doc.surface.history.available && doc.surface.history.commitsRead !== 0) errors.push({ path: '/surface/history/commitsRead', message: 'unavailable history must report zero commits read' });
251
+ if (new Set(doc.surface.rankedUnclaimed.map((item) => item.file)).size !== doc.surface.rankedUnclaimed.length) errors.push({ path: '/surface/rankedUnclaimed', message: 'ranked unclaimed modules must be unique' });
252
+ }
253
+ if (doc.run?.scope) {
254
+ const scope = doc.run.scope;
255
+ if (scope.requestedCount !== scope.requestedClaims.length) errors.push({ path: '/run/scope/requestedCount', message: 'requestedCount must equal requestedClaims.length' });
256
+ if (scope.probedCount !== scope.probedClaims.length) errors.push({ path: '/run/scope/probedCount', message: 'probedCount must equal probedClaims.length' });
257
+ const requested = new Set(scope.requestedClaims);
258
+ if (scope.probedClaims.some((id) => !requested.has(id))) errors.push({ path: '/run/scope/probedClaims', message: 'every probed claim must have been requested' });
259
+ }
247
260
  return errors;
248
261
  },
249
262
 
@@ -102,6 +102,130 @@
102
102
  "provisional": {
103
103
  "type": "boolean"
104
104
  },
105
+ "surface": {
106
+ "description": "The current claim denominator. A source module is a non-test file with a supported source extension that is not covered by the documented default exclusions. Churn is bounded recent git history and remains separate from path-risk signals; neither is collapsed into a score.",
107
+ "type": "object",
108
+ "required": ["sourceModules", "claimedModules", "unclaimedModules", "history", "highChurn", "rankedUnclaimed"],
109
+ "properties": {
110
+ "sourceModules": { "type": "integer", "minimum": 0 },
111
+ "claimedModules": { "type": "integer", "minimum": 0 },
112
+ "unclaimedModules": { "type": "integer", "minimum": 0 },
113
+ "history": {
114
+ "type": "object",
115
+ "required": ["available", "maxCommits", "commitsRead"],
116
+ "properties": {
117
+ "available": { "type": "boolean" },
118
+ "maxCommits": { "type": "integer", "minimum": 1 },
119
+ "commitsRead": { "type": "integer", "minimum": 0 }
120
+ },
121
+ "additionalProperties": false
122
+ },
123
+ "highChurn": {
124
+ "description": "How many of the up-to-20 most changed source modules carry a claim.",
125
+ "type": "object",
126
+ "required": ["modules", "claimed"],
127
+ "properties": {
128
+ "modules": { "type": "integer", "minimum": 0, "maximum": 20 },
129
+ "claimed": { "type": "integer", "minimum": 0, "maximum": 20 }
130
+ },
131
+ "additionalProperties": false
132
+ },
133
+ "rankedUnclaimed": {
134
+ "description": "Up to ten concrete unclaimed modules, ordered by changes in the bounded history window; path-risk signals break ties and are reported explicitly.",
135
+ "type": "array",
136
+ "maxItems": 10,
137
+ "items": {
138
+ "type": "object",
139
+ "required": ["file", "changes", "riskSignals"],
140
+ "properties": {
141
+ "file": { "$ref": "urn:claimspec:v1:common#/$defs/repoPath" },
142
+ "changes": { "type": "integer", "minimum": 0 },
143
+ "riskSignals": {
144
+ "type": "array",
145
+ "uniqueItems": true,
146
+ "items": { "enum": ["security", "money"] }
147
+ }
148
+ },
149
+ "additionalProperties": false
150
+ }
151
+ }
152
+ },
153
+ "additionalProperties": false
154
+ },
155
+ "run": {
156
+ "description": "Present on probe --json: the validated status document plus the completed invocation's result. scope is present only for a partial --claim run.",
157
+ "type": "object",
158
+ "required": ["id", "evidence", "provisional", "records", "newSinceBaseline", "exitCode"],
159
+ "properties": {
160
+ "id": { "type": "string", "minLength": 1, "maxLength": 128 },
161
+ "evidence": { "type": "string", "minLength": 1, "maxLength": 4096 },
162
+ "provisional": { "type": "boolean" },
163
+ "records": { "type": "integer", "minimum": 0 },
164
+ "newSinceBaseline": { "type": "integer", "minimum": 0 },
165
+ "exitCode": { "enum": [0, 1] },
166
+ "scope": {
167
+ "type": "object",
168
+ "required": ["requestedClaims", "requestedCount", "probedClaims", "probedCount"],
169
+ "properties": {
170
+ "requestedClaims": {
171
+ "type": "array",
172
+ "minItems": 1,
173
+ "uniqueItems": true,
174
+ "items": { "$ref": "urn:claimspec:v1:common#/$defs/identifier" }
175
+ },
176
+ "requestedCount": { "type": "integer", "minimum": 1 },
177
+ "probedClaims": {
178
+ "type": "array",
179
+ "uniqueItems": true,
180
+ "items": { "$ref": "urn:claimspec:v1:common#/$defs/identifier" }
181
+ },
182
+ "probedCount": { "type": "integer", "minimum": 0 }
183
+ },
184
+ "additionalProperties": false
185
+ }
186
+ },
187
+ "additionalProperties": false
188
+ },
189
+ "cost": {
190
+ "description": "Present when probe --json --cost is requested; derived only from the runs in the emitted evidence.",
191
+ "type": "object",
192
+ "required": ["totalMs", "totalRuns", "reusedRecords", "records", "claims", "defenders", "sharedDefenders"],
193
+ "properties": {
194
+ "totalMs": { "type": "number", "minimum": 0 },
195
+ "totalRuns": { "type": "integer", "minimum": 0 },
196
+ "reusedRecords": { "type": "integer", "minimum": 0 },
197
+ "records": { "type": "integer", "minimum": 0 },
198
+ "claims": {
199
+ "type": "array",
200
+ "items": {
201
+ "type": "object",
202
+ "required": ["claimId", "ms", "runs", "faults", "reused", "defenders"],
203
+ "properties": {
204
+ "claimId": { "$ref": "urn:claimspec:v1:common#/$defs/identifier" },
205
+ "ms": { "type": "number", "minimum": 0 },
206
+ "runs": { "type": "integer", "minimum": 0 },
207
+ "faults": { "type": "integer", "minimum": 0 },
208
+ "reused": { "type": "integer", "minimum": 0 },
209
+ "defenders": {
210
+ "type": "array",
211
+ "uniqueItems": true,
212
+ "items": { "$ref": "urn:claimspec:v1:common#/$defs/repoPath" }
213
+ }
214
+ },
215
+ "additionalProperties": false
216
+ }
217
+ },
218
+ "defenders": {
219
+ "type": "array",
220
+ "items": { "$ref": "#/$defs/costDefender" }
221
+ },
222
+ "sharedDefenders": {
223
+ "type": "array",
224
+ "items": { "$ref": "#/$defs/costDefender" }
225
+ }
226
+ },
227
+ "additionalProperties": false
228
+ },
105
229
  "counts": {
106
230
  "type": "object",
107
231
  "required": [
@@ -349,5 +473,23 @@
349
473
  "$ref": "urn:claimspec:v1:common#/$defs/gitSha"
350
474
  }
351
475
  },
352
- "additionalProperties": false
476
+ "additionalProperties": false,
477
+ "$defs": {
478
+ "costDefender": {
479
+ "type": "object",
480
+ "required": ["file", "ms", "runs", "claims"],
481
+ "properties": {
482
+ "file": { "$ref": "urn:claimspec:v1:common#/$defs/repoPath" },
483
+ "ms": { "type": "number", "minimum": 0 },
484
+ "runs": { "type": "integer", "minimum": 0 },
485
+ "claims": {
486
+ "type": "array",
487
+ "minItems": 1,
488
+ "uniqueItems": true,
489
+ "items": { "$ref": "urn:claimspec:v1:common#/$defs/identifier" }
490
+ }
491
+ },
492
+ "additionalProperties": false
493
+ }
494
+ }
353
495
  }
@@ -1,5 +1,5 @@
1
1
  import { gate } from '../baseline/baseline.mjs';
2
- import { summarize, formatVerdict } from '../render.mjs';
2
+ import { coAuthorshipWarning, summarize, formatVerdict } from '../render.mjs';
3
3
 
4
4
  export const HEADING = '## TEST BLINDSPOT CONTEXT';
5
5
  /** First line of every markdown brief, so a poster can find and update its own note instead of adding another. */
@@ -98,7 +98,7 @@ export function renderBriefText(brief, { hasBaseline, total, independence }) {
98
98
  }
99
99
  // L3: a kill written in the same change as the code it guards is not
100
100
  // independent evidence. One line, no per-item noise.
101
- if (independence?.coAuthored) lines.push('', `${independence.coAuthored} of ${independence.kills} kills are co-authored with the code they defend — the test and the code were written in the same change, so those kills are not independent evidence.`);
101
+ if (independence?.coAuthored) lines.push('', coAuthorshipWarning(independence.coAuthored, independence.kills));
102
102
  if (brief.next) lines.push('', `NEXT [${brief.next.action}]: ${brief.next.command}`, ` why: ${brief.next.why}`);
103
103
  if (brief.items.length === 0) {
104
104
  lines.push('', 'Every probed claim is defended. Keep it that way: new claims need a fault and a test that fails on it.');
package/src/cli.mjs CHANGED
@@ -46,7 +46,7 @@ probe
46
46
  --ref <commit> probe this commit in the scratch worktree (default: HEAD). An explicit --ref is honoured even when
47
47
  defender/target files are dirty: a warning names them, the evidence records them (repo.ignoredDirty)
48
48
  --ignore-dirty probe HEAD as committed although defender/target files are dirty (same warning and record)
49
- --claim <ID,ID> probe only these claims; writes .testguard/evidence-partial.json
49
+ --claim <ID,ID> probe only these claims; repeatable, order-preserving, and writes .testguard/evidence-partial.json
50
50
  --include-dirty probe the working tree (a snapshot commit) instead of HEAD; uncommitted tests count
51
51
  (without it, a dirty defender/target with the implicit HEAD is refused: the silent-mismatch trap)
52
52
  --verbose also print each killed fault (default: only unproven ones, plus a count)
@@ -137,7 +137,7 @@ export async function main(argv, io = { out: (s) => process.stdout.write(s + '\n
137
137
  ref: { type: 'string' },
138
138
  'ignore-dirty': { type: 'boolean', default: false },
139
139
  serial: { type: 'boolean', default: false },
140
- claim: { type: 'string' },
140
+ claim: { type: 'string', multiple: true },
141
141
  fault: { type: 'string' },
142
142
  'include-dirty': { type: 'boolean', default: false },
143
143
  changed: { type: 'string' },
@@ -29,10 +29,19 @@ function findProjectDir(testAbs, claimsFlag) {
29
29
  * decision here can never disagree with the gate.
30
30
  */
31
31
  export async function admitCommand({ file, values, version }, io) {
32
- if (!file || !values.claim) {
32
+ if (!file || !values.claim?.length) {
33
33
  io.err('usage: testguard admit <test-file> --claim <ID> [--fault <FID>] [--confirm <n>] [--json]');
34
34
  return 3;
35
35
  }
36
+ if (values.claim.length > 1 || values.claim[0].includes(',')) {
37
+ io.err('admit accepts exactly one --claim <ID>');
38
+ return 3;
39
+ }
40
+ const claimId = values.claim[0].trim();
41
+ if (!claimId) {
42
+ io.err('--claim must name one claim id');
43
+ return 3;
44
+ }
36
45
  const confirmRuns = Number(values.confirm);
37
46
  const budgetMs = Number(values.budget);
38
47
  if (!Number.isInteger(confirmRuns) || confirmRuns < 1 || !Number.isInteger(budgetMs) || budgetMs < 1000) {
@@ -46,7 +55,6 @@ export async function admitCommand({ file, values, version }, io) {
46
55
  if (testRel.startsWith('..')) throw new PreconditionError(`${file} is outside the project directory ${projectDir}`);
47
56
 
48
57
  const all = loadClaims(values.claims ? resolve(values.claims) : defaultClaimsPath(projectDir));
49
- const claimId = values.claim.trim();
50
58
  const claim = all.claims.find((c) => c.id === claimId);
51
59
  if (!claim) throw new PreconditionError(`--claim: unknown claim id ${claimId}`);
52
60
  const faults = values.fault ? claim.faults.filter((f) => f.id === values.fault) : claim.faults;
@@ -9,11 +9,30 @@ import { computeStatus } from '../status/status.mjs';
9
9
  import { resolveChangedRef, withChangedRef } from '../gate/changed.mjs';
10
10
  import { costReport, renderCost } from '../probe/cost.mjs';
11
11
  import { progressMode, stageReporter, clearStageLine, recordEvent, isProgressMode, progressStream } from '../probe/progress.mjs';
12
+ import { validate } from '../../spec/lib/validate.mjs';
12
13
  export const provisionalEvidencePath = (projectDir) => join(projectDir, '.testguard', 'evidence-provisional.json');
13
14
 
14
15
  export const evidencePath = (projectDir) => join(projectDir, '.testguard', 'evidence.json');
15
16
  export const baselinePath = (projectDir) => join(projectDir, '.testguard', 'baseline.json');
16
17
 
18
+ /** Every repeated/comma-separated --claim value, in first-seen order. */
19
+ export function claimSelection(values) {
20
+ if (values === undefined) return undefined;
21
+ return [...new Set(values.flatMap((value) => value.split(',')).map((id) => id.trim()).filter(Boolean))];
22
+ }
23
+
24
+ /** Scope is invocation metadata, kept outside the evidence contract. */
25
+ export function partialScope(only, records) {
26
+ if (!only) return undefined;
27
+ const probedClaims = [...new Set(records.map((record) => record.claim.id))];
28
+ return {
29
+ requestedClaims: only,
30
+ requestedCount: only.length,
31
+ probedClaims,
32
+ probedCount: probedClaims.length,
33
+ };
34
+ }
35
+
17
36
  export async function probeCommand({ projectDir, values, version }, io) {
18
37
  const confirmRuns = Number(values.confirm);
19
38
  const budgetMs = Number(values.budget);
@@ -34,7 +53,11 @@ export async function probeCommand({ projectDir, values, version }, io) {
34
53
  io.err('claims file declares no claims; nothing to verify');
35
54
  return 2;
36
55
  }
37
- const only = values.claim ? values.claim.split(',').map((s) => s.trim()).filter(Boolean) : undefined;
56
+ const only = claimSelection(values.claim);
57
+ if (only?.length === 0) {
58
+ io.err('--claim must name at least one claim id');
59
+ return 3;
60
+ }
38
61
  // A --claim run is partial evidence; keep it away from the canonical file unless --out says otherwise.
39
62
  const provisional = confirmRuns < 3;
40
63
  // Provisional and partial runs never overwrite the canonical evidence: only confirmed, complete runs may feed a baseline.
@@ -81,9 +104,13 @@ export async function probeCommand({ projectDir, values, version }, io) {
81
104
  writeSpecDoc('evidence', outPath, evidence);
82
105
 
83
106
  const g = gate(evidence.records, baseline, { severityFloor: values.severity });
107
+ const scope = partialScope(only, evidence.records);
84
108
  if (values.json) {
85
109
  const status = withChangedRef(resolveChangedRef({ explicit: values.changed }), (changedRef) => computeStatus({ projectDir, toolVersion: version, changedRef, includeDirty: values['include-dirty'], evidence: outPath }), io.err);
86
- io.out(JSON.stringify({ ...status, run: { id: evidence.run.id, evidence: outPath, provisional, records: evidence.records.length, newSinceBaseline: g.new.length, exitCode: g.new.length > 0 ? 1 : 0 }, ...(values.cost ? { cost: costReport(evidence.records) } : {}) }, null, 2));
110
+ const doc = { ...status, run: { id: evidence.run.id, evidence: outPath, provisional, records: evidence.records.length, newSinceBaseline: g.new.length, exitCode: g.new.length > 0 ? 1 : 0, ...(scope ? { scope } : {}) }, ...(values.cost ? { cost: costReport(evidence.records) } : {}) };
111
+ const result = validate('status', doc);
112
+ if (!result.ok) throw new Error(`probe JSON document does not conform: ${result.errors.map((e) => `${e.path}: ${e.message}`).join('; ')}`);
113
+ io.out(JSON.stringify(doc, null, 2));
87
114
  return g.new.length > 0 ? 1 : 0;
88
115
  }
89
116
  if (!values.quiet && baseline) {
@@ -102,7 +129,7 @@ export async function probeCommand({ projectDir, values, version }, io) {
102
129
  io.out(renderCost(costReport(evidence.records)));
103
130
  io.out('');
104
131
  }
105
- io.out(`evidence: ${outPath}${only ? ` (partial: --claim ${only.join(',')}; not the canonical evidence file)` : provisional ? ' (provisional; not the canonical evidence file)' : ''}`);
132
+ io.out(`evidence: ${outPath}${scope ? ` (partial: ${scope.requestedCount} claim${scope.requestedCount === 1 ? '' : 's'} requested, ${scope.probedCount} probed: --claim ${scope.requestedClaims.join(',')}; not the canonical evidence file)` : provisional ? ' (provisional; not the canonical evidence file)' : ''}`);
106
133
  if (provisional && !values.quiet) io.err(PROVISIONAL_WARNING(confirmRuns));
107
134
  return g.new.length > 0 ? 1 : 0;
108
135
  }
@@ -10,6 +10,15 @@ export async function scaffoldCommand({ projectDir, file, values, version }, io)
10
10
  io.err('usage: testguard scaffold <source-file> [--claim <ID>] [--out <path>] [--json]');
11
11
  return 3;
12
12
  }
13
+ if (values.claim?.length > 1 || values.claim?.[0]?.includes(',')) {
14
+ io.err('scaffold accepts exactly one --claim <ID>');
15
+ return 3;
16
+ }
17
+ const claimId = values.claim?.[0]?.trim();
18
+ if (values.claim && !claimId) {
19
+ io.err('--claim must name one claim id');
20
+ return 3;
21
+ }
13
22
  const abs = resolve(file);
14
23
  if (!existsSync(abs)) throw new PreconditionError(`no such file: ${file}`);
15
24
  const rel = relative(projectDir, abs);
@@ -17,7 +26,7 @@ export async function scaffoldCommand({ projectDir, file, values, version }, io)
17
26
 
18
27
  const claimsPath = values.claims ? resolve(values.claims) : defaultClaimsPath(projectDir);
19
28
  const existingClaims = existsSync(claimsPath) ? loadClaims(claimsPath) : undefined;
20
- const { doc, stats } = scaffoldFile({ projectDir, file: rel, claimId: values.claim, existingClaims, toolVersion: version });
29
+ const { doc, stats } = scaffoldFile({ projectDir, file: rel, claimId, existingClaims, toolVersion: version });
21
30
 
22
31
  if (values.json) {
23
32
  io.out(JSON.stringify(doc, null, 2));
@@ -28,7 +28,7 @@ import { globToRegExp } from '../util/glob.mjs';
28
28
  export const SOURCE_EXT = new Set(['.js', '.mjs', '.cjs', '.ts', '.mts', '.cts', '.jsx', '.tsx', '.py']);
29
29
  const TEST_RE = /\.(test|spec)\.[cm]?[jt]sx?$/;
30
30
  // Python names tests by file, not by a dotted suffix: `test_foo.py`, `foo_test.py`.
31
- const isTestFile = (f) => TEST_RE.test(f) || IS_PY_TEST.test(f);
31
+ export const isTestFile = (f) => TEST_RE.test(f) || IS_PY_TEST.test(f);
32
32
 
33
33
  /** Files that are source by extension but never carry claims. Printed by --explain. */
34
34
  export const DEFAULT_EXCLUDES = Object.freeze([
@@ -42,6 +42,10 @@ export const DEFAULT_EXCLUDES = Object.freeze([
42
42
  '**/__snapshots__/**',
43
43
  '.testguard/**',
44
44
  ]);
45
+ const DEFAULT_EXCLUDE_RES = DEFAULT_EXCLUDES.map(globToRegExp);
46
+
47
+ /** Whether a source-shaped path is outside TestGuard's claimable module surface. */
48
+ export const isDefaultExcluded = (file) => DEFAULT_EXCLUDE_RES.some((re) => re.test(file));
45
49
 
46
50
  export const defaultIgnorePath = (projectDir) => join(projectDir, 'testguard.ignore.json');
47
51
 
@@ -158,7 +162,7 @@ export function computeChangedGate({ projectDir, ref, includeDirty = false, excl
158
162
 
159
163
  const pathEntries = ignore.entries.filter((e) => e.kind === 'path').map((e) => ({ entry: e, re: globToRegExp(e.pattern), expired: isExpired(e, now), files: [] }));
160
164
  const userExcludes = exclude.map((g) => ({ glob: g, re: globToRegExp(g) }));
161
- const defaultExcludes = DEFAULT_EXCLUDES.map((g) => ({ glob: g, re: globToRegExp(g) }));
165
+ const defaultExcludes = DEFAULT_EXCLUDES.map((g, i) => ({ glob: g, re: DEFAULT_EXCLUDE_RES[i] }));
162
166
 
163
167
  const excluded = [];
164
168
  const covered = [];
@@ -40,6 +40,16 @@ function readRunnerVersion(projectDir, name = 'vitest') {
40
40
  }
41
41
  }
42
42
 
43
+ /** Validate and order a partial claim selection without losing caller order. */
44
+ export function selectClaims(claims, only) {
45
+ if (!only) return claims;
46
+ if (only.length === 0) throw new PreconditionError('--claim must name at least one claim id');
47
+ const byId = new Map(claims.map((claim) => [claim.id, claim]));
48
+ const unknown = only.filter((id) => !byId.has(id));
49
+ if (unknown.length) throw new PreconditionError(`--claim: unknown claim id(s) ${unknown.join(', ')}`);
50
+ return only.map((id) => byId.get(id));
51
+ }
52
+
43
53
  /**
44
54
  * Probe every fault of every claim. Returns a spec-conformant evidence
45
55
  * document; writing it is the caller's job.
@@ -76,18 +86,13 @@ export async function probe({
76
86
  const head = headSha(root, ref);
77
87
  if (!head) throw new PreconditionError(ref === 'HEAD' ? 'repository has no commits; every verdict is tied to a commit' : `ref ${ref} does not resolve to a commit`);
78
88
 
79
- const targets = [...new Set(claims.claims.flatMap((c) => c.faults.map((f) => relative(root, join(projectDir, f.file)))))];
89
+ const scopedClaims = selectClaims(claims.claims, only);
90
+ const targets = [...new Set(scopedClaims.flatMap((c) => c.faults.map((f) => relative(root, join(projectDir, f.file)))))];
80
91
  if (mode === 'in-place' && isDirty(root, targets)) {
81
92
  throw new PreconditionError(`uncommitted changes in fault target files (${targets.join(', ')}); commit or stash them, or drop --in-place. Only the files faults are applied to must be clean — test files may be dirty, which is what makes --in-place usable while writing tests.`);
82
93
  }
83
94
  if (includeDirty && mode !== 'worktree') throw new PreconditionError('--include-dirty applies to worktree mode; drop --in-place');
84
95
  if (includeDirty && ref !== 'HEAD') throw new PreconditionError('--include-dirty snapshots the working tree; it cannot be combined with --ref');
85
- const selected = only ? new Set(only) : null;
86
- if (selected) {
87
- const known = new Set(claims.claims.map((c) => c.id));
88
- const unknown = [...selected].filter((id) => !known.has(id));
89
- if (unknown.length) throw new PreconditionError(`--claim: unknown claim id(s) ${unknown.join(', ')}`);
90
- }
91
96
 
92
97
  // Worktree mode probes a commit, not the working tree. Uncommitted defender
93
98
  // or target edits would be silently absent — the same survivors, no hint why.
@@ -103,8 +108,7 @@ export async function probe({
103
108
  if (isDirty(root)) snapshot = snapshotWorkingTree(root);
104
109
  } else {
105
110
  const watched = new Set(targets);
106
- for (const claim of claims.claims) {
107
- if (selected && !selected.has(claim.id)) continue;
111
+ for (const claim of scopedClaims) {
108
112
  const declared = claim.defendedBy?.length ? resolveDefenders(projectDir, claim.defendedBy) : claim.faults.flatMap((f) => discoverDefenders(projectDir, f.file));
109
113
  for (const d of declared) watched.add(relative(root, join(projectDir, d)));
110
114
  for (const g of claim.defendedBy ?? []) if (!g.includes('*')) watched.add(relative(root, join(projectDir, g)));
@@ -193,8 +197,7 @@ export async function probe({
193
197
  return Object.fromEntries([...groups].map(([r, group]) => [labelOf(r), group]));
194
198
  };
195
199
 
196
- for (const claim of claims.claims) {
197
- if (selected && !selected.has(claim.id)) continue;
200
+ for (const claim of scopedClaims) {
198
201
  const declared = claim.defendedBy?.length ? resolveDefenders(iso.projectDir, claim.defendedBy) : null;
199
202
  for (const fault of claim.faults) {
200
203
  // Mock-awareness: a discovered file that mocks the target is not a
package/src/render.mjs CHANGED
@@ -5,6 +5,8 @@ export const formatVerdict = (v, provisional = false) => (v === 'killed' ? 'kill
5
5
 
6
6
  export const PROVISIONAL_WARNING = (n) => `PROVISIONAL — confirmRuns ${n} (< 3): nothing below is confirmed. Verdicts carry a "?"; this evidence cannot be frozen into a baseline. Re-run with --confirm 3 before trusting it.`;
7
7
 
8
+ export const coAuthorshipWarning = (coAuthored, kills) => `${coAuthored} of ${kills} kills are co-authored with the code they defend — the test and the code were written in the same change, so those kills are not independent evidence.`;
9
+
8
10
  export function renderRecord(r, { provisional = false, showDiscovered = true } = {}) {
9
11
  const head = `${formatVerdict(r.verdict, provisional).padEnd(15)} ${r.claim.id}/${r.subject.id}`.padEnd(38);
10
12
  let why = '';
@@ -32,7 +34,10 @@ export function renderSummary(records, run) {
32
34
  const unproven = records.filter((r) => r.verdict !== 'killed');
33
35
  const claims = new Set(unproven.map((r) => r.claim.id)).size;
34
36
  const where = run ? ` Probed ${run.repo.snapshot ? `working tree (snapshot ${run.repo.snapshot.slice(0, 7)} of ${run.repo.head.slice(0, 7)})` : run.mode === 'in-place' ? `in place at ${run.repo.head.slice(0, 7)}${run.repo.dirty ? ' (dirty)' : ''}` : run.repo.head.slice(0, 7)}.` : '';
35
- return `${run?.provisional ? 'PROVISIONAL: ' : ''}${records.length} faults probed: ${parts.join(', ')}. ${unproven.length} unproven fault${unproven.length === 1 ? '' : 's'} across ${claims} claim${claims === 1 ? '' : 's'}.${where}`;
37
+ const kills = records.filter((r) => r.verdict === 'killed');
38
+ const coAuthored = kills.filter((r) => r.detail.independence?.class === 'co-authored').length;
39
+ const quality = coAuthored ? `\n${coAuthorshipWarning(coAuthored, kills.length)}` : '';
40
+ return `${run?.provisional ? 'PROVISIONAL: ' : ''}${records.length} faults probed: ${parts.join(', ')}. ${unproven.length} unproven fault${unproven.length === 1 ? '' : 's'} across ${claims} claim${claims === 1 ? '' : 's'}.${where}${quality}`;
36
41
  }
37
42
 
38
43
  /** Survivors first, then by rank score; killed last. */
@@ -6,13 +6,18 @@ import { gate } from '../baseline/baseline.mjs';
6
6
  import { hashFile, sha256 } from '../util/hash.mjs';
7
7
  import { resolveDefenders } from '../probe/runners/shared.mjs';
8
8
  import { discoverDefenders } from '../probe/discover.mjs';
9
- import { sortForReport } from '../render.mjs';
9
+ import { coAuthorshipWarning, sortForReport } from '../render.mjs';
10
10
  import { computeChangedGate, defaultIgnorePath } from '../gate/changed.mjs';
11
11
  import { headSha, isAncestor } from '../git.mjs';
12
12
  import { checkAnchorLocations } from '../claims/anchors.mjs';
13
+ import { computeClaimedSurface, needsClaimExpansion } from './surface.mjs';
13
14
 
14
15
  export const faultContentHash = (fault) => sha256(`${fault.find}\n${fault.replace}`);
15
16
 
17
+ const candidateReason = (surface, candidate) => surface.history.available
18
+ ? `${candidate.file}, the highest-churn unclaimed module in the last ${surface.history.commitsRead} commit${surface.history.commitsRead === 1 ? '' : 's'}`
19
+ : `${candidate.file}, the first unclaimed module by path because git history is unavailable`;
20
+
16
21
  const P = (projectDir) => ({
17
22
  claims: defaultClaimsPath(projectDir),
18
23
  evidence: join(projectDir, '.testguard', 'evidence.json'),
@@ -68,6 +73,11 @@ export function computeStatus({ projectDir, toolVersion = '0.0.0', generatedAt =
68
73
  };
69
74
 
70
75
  if (!existsSync(paths.claims)) {
76
+ doc.surface = computeClaimedSurface({ projectDir });
77
+ const candidate = doc.surface.rankedUnclaimed[0];
78
+ if (candidate && !changes?.uncovered.length) {
79
+ doc.next = { action: 'scaffold', command: `testguard scaffold ${candidate.file}`, why: `${doc.surface.sourceModules} source module${doc.surface.sourceModules === 1 ? '' : 's'} exist and none carries a claim. Start with ${candidateReason(doc.surface, candidate)}.`, file: candidate.file };
80
+ }
71
81
  if (changes?.uncovered.length) {
72
82
  const u = changes.uncovered[0];
73
83
  doc.next = { action: 'scaffold', command: `testguard scaffold ${u.file}`, why: `No testguard.claims.json in this project, and ${unclaimedWhy()}`, file: u.file };
@@ -79,6 +89,7 @@ export function computeStatus({ projectDir, toolVersion = '0.0.0', generatedAt =
79
89
  const claims = loadClaims(paths.claims);
80
90
  doc.counts.claims = claims.claims.length;
81
91
  doc.counts.faults = claims.claims.reduce((n, c) => n + c.faults.length, 0);
92
+ doc.surface = computeClaimedSurface({ projectDir, claims });
82
93
  const faultIndex = new Map();
83
94
  for (const c of claims.claims) for (const f of c.faults) faultIndex.set(`${c.id}/${f.id}`, { claim: c, fault: f });
84
95
 
@@ -236,6 +247,15 @@ export function computeStatus({ projectDir, toolVersion = '0.0.0', generatedAt =
236
247
  } else {
237
248
  doc.next = { action: 'none', command: 'testguard probe', why: unprovenTotal ? `${unprovenTotal} baselined finding${unprovenTotal === 1 ? '' : 's'} remain; nothing new. Re-probe after changing code, tests or claims.` : 'Every claim is defended. Re-probe after changing code, tests or claims; add a claim for every new invariant.' };
238
249
  }
250
+ if (needsClaimExpansion(doc.surface)) {
251
+ const candidate = doc.surface.rankedUnclaimed[0];
252
+ doc.next = {
253
+ action: 'claim',
254
+ command: `testguard scaffold ${candidate.file}`,
255
+ why: `The existing claims are defended, but only ${doc.surface.claimedModules} of ${doc.surface.sourceModules} source modules carry a claim. Expand the denominator at ${candidateReason(doc.surface, candidate)}.`,
256
+ file: candidate.file,
257
+ };
258
+ }
239
259
  return doc;
240
260
  }
241
261
 
@@ -246,6 +266,15 @@ export function renderStatus(doc) {
246
266
  }
247
267
  lines.push(`state: ${doc.state}${doc.provisional ? ' (provisional)' : ''} — ${doc.counts.claims} claims / ${doc.counts.faults} faults` + (doc.counts.byVerdict ? `; ${Object.entries(doc.counts.byVerdict).map(([k, v]) => `${v} ${k}`).join(', ')}; ${doc.counts.new ?? 0} new, ${doc.counts.baselined ?? 0} baselined` : '')
248
268
  );
269
+ if (doc.surface) {
270
+ const s = doc.surface;
271
+ const history = s.history.available ? `last ${s.history.commitsRead} commit${s.history.commitsRead === 1 ? '' : 's'}` : 'git history unavailable';
272
+ lines.push(`surface: ${s.claimedModules} of ${s.sourceModules} source modules carry a claim; ${s.highChurn.claimed} of ${s.highChurn.modules} highest-churn modules claimed (${history})`);
273
+ for (const item of s.rankedUnclaimed.slice(0, 3)) lines.push(`UNCLAIMED ${item.file} — ${item.changes} change${item.changes === 1 ? '' : 's'} in history window${item.riskSignals.length ? `; path risk: ${item.riskSignals.join(', ')}` : ''}`);
274
+ }
275
+ const coAuthored = doc.counts.killedCoAuthored ?? 0;
276
+ const kills = doc.counts.byVerdict?.killed ?? 0;
277
+ if (coAuthored) lines.push(`evidence: ${coAuthorshipWarning(coAuthored, kills)}`);
249
278
  if (doc.changes) {
250
279
  const c = doc.changes;
251
280
  lines.push(`changes: ${c.changed} file${c.changed === 1 ? '' : 's'} since ${c.ref}; ${c.evaluated} evaluated, ${c.excluded} excluded, ${c.uncovered.length} unclaimed`);
@@ -0,0 +1,72 @@
1
+ import { existsSync } from 'node:fs';
2
+ import { extname, join } from 'node:path';
3
+ import { git } from '../git.mjs';
4
+ import { SOURCE_EXT, isDefaultExcluded, isTestFile } from '../gate/changed.mjs';
5
+ import { walk } from '../util/glob.mjs';
6
+
7
+ export const HISTORY_COMMITS = 200;
8
+ export const HIGH_CHURN_MODULES = 20;
9
+ export const RANKED_UNCLAIMED = 10;
10
+
11
+ const RISK_SIGNALS = Object.freeze([
12
+ ['security', /(^|[._/-])(auth|authn|authz|authorization|credential|crypto|login|password|permission|rbac|secret|security|session|token)([._/-]|$)/i],
13
+ ['money', /(^|[._/-])(billing|checkout|invoice|money|payment|payout|price|pricing|refund)([._/-]|$)/i],
14
+ ]);
15
+
16
+ const pathRiskSignals = (file) => RISK_SIGNALS.filter(([, re]) => re.test(file)).map(([name]) => name);
17
+
18
+ function listedFiles(projectDir) {
19
+ try {
20
+ return git(['ls-files', '-co', '--exclude-standard', '-z', '--', '.'], projectDir).split('\0').filter(Boolean);
21
+ } catch {
22
+ return walk(projectDir);
23
+ }
24
+ }
25
+
26
+ function claimableModules(projectDir) {
27
+ return listedFiles(projectDir).filter((file) =>
28
+ existsSync(join(projectDir, file)) &&
29
+ SOURCE_EXT.has(extname(file)) &&
30
+ !isTestFile(file) &&
31
+ !isDefaultExcluded(file));
32
+ }
33
+
34
+ function recentChanges(projectDir, sourceModules) {
35
+ const counts = new Map();
36
+ try {
37
+ const commitsRead = Number(git(['rev-list', '--count', `--max-count=${HISTORY_COMMITS}`, 'HEAD', '--', '.'], projectDir));
38
+ const paths = git(['log', '--relative', '-n', String(HISTORY_COMMITS), '--pretty=format:', '--name-only', '-z', '--', '.'], projectDir).split('\0').filter(Boolean);
39
+ for (const file of paths) if (sourceModules.has(file)) counts.set(file, (counts.get(file) ?? 0) + 1);
40
+ return { available: true, commitsRead, counts };
41
+ } catch {
42
+ return { available: false, commitsRead: 0, counts };
43
+ }
44
+ }
45
+
46
+ /**
47
+ * The denominator `probe` cannot provide: current source modules carrying at
48
+ * least one fault, plus concrete unclaimed modules ranked by bounded git
49
+ * history. Churn and path-risk remain separate facts; there is no score.
50
+ */
51
+ export function computeClaimedSurface({ projectDir, claims }) {
52
+ const modules = new Set(claimableModules(projectDir));
53
+ const claimedFiles = new Set((claims?.claims ?? []).flatMap((claim) => claim.faults.map((fault) => fault.file)));
54
+ const claimed = new Set([...modules].filter((file) => claimedFiles.has(file)));
55
+ const history = recentChanges(projectDir, modules);
56
+ const describe = (file) => ({ file, changes: history.counts.get(file) ?? 0, riskSignals: pathRiskSignals(file) });
57
+ const ranked = [...modules].map(describe).sort((a, b) =>
58
+ b.changes - a.changes || b.riskSignals.length - a.riskSignals.length || a.file.localeCompare(b.file));
59
+ const highChurn = ranked.slice(0, Math.min(HIGH_CHURN_MODULES, ranked.length));
60
+ const rankedUnclaimed = ranked.filter((item) => !claimed.has(item.file)).slice(0, RANKED_UNCLAIMED);
61
+ return {
62
+ sourceModules: modules.size,
63
+ claimedModules: claimed.size,
64
+ unclaimedModules: modules.size - claimed.size,
65
+ history: { available: history.available, maxCommits: HISTORY_COMMITS, commitsRead: history.commitsRead },
66
+ highChurn: { modules: highChurn.length, claimed: highChurn.filter((item) => claimed.has(item.file)).length },
67
+ rankedUnclaimed,
68
+ };
69
+ }
70
+
71
+ /** Expand the claim surface only when most current source modules are unseen. */
72
+ export const needsClaimExpansion = (surface) => surface.unclaimedModules > surface.claimedModules && surface.rankedUnclaimed.length > 0;