testguard-cli 0.16.0 → 0.17.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (62) hide show
  1. package/CHANGELOG.md +51 -0
  2. package/README.md +152 -13
  3. package/package.json +1 -1
  4. package/spec/GATE-SEMANTICS.md +295 -0
  5. package/spec/README.md +27 -2
  6. package/spec/lib/authoring-input.mjs +40 -0
  7. package/spec/lib/origins.mjs +56 -0
  8. package/spec/lib/validate.mjs +120 -1
  9. package/spec/schemas/annotations.schema.json +65 -0
  10. package/spec/schemas/authoring-input.schema.json +67 -0
  11. package/spec/schemas/brief.schema.json +2 -0
  12. package/spec/schemas/claims.schema.json +2 -8
  13. package/spec/schemas/common.schema.json +73 -0
  14. package/spec/schemas/evidence.schema.json +37 -8
  15. package/spec/schemas/status.schema.json +6 -2
  16. package/src/brief/brief.mjs +11 -4
  17. package/src/claims/annotation-placement.mjs +357 -0
  18. package/src/claims/annotations.mjs +61 -2
  19. package/src/cli.mjs +40 -5
  20. package/src/commands/admit.mjs +13 -0
  21. package/src/commands/annotations.mjs +67 -0
  22. package/src/commands/claims.mjs +8 -4
  23. package/src/commands/probe.mjs +50 -7
  24. package/src/commands/replay.mjs +7 -0
  25. package/src/commands/scaffold.mjs +49 -2
  26. package/src/commands/sweep.mjs +7 -0
  27. package/src/evidence/writer.mjs +12 -4
  28. package/src/git.mjs +4 -4
  29. package/src/init/templates/SKILL.md +35 -0
  30. package/src/probe/attribution.mjs +10 -0
  31. package/src/probe/contention.mjs +20 -2
  32. package/src/probe/cost.mjs +6 -1
  33. package/src/probe/inject.mjs +2 -6
  34. package/src/probe/origin-policy.mjs +69 -0
  35. package/src/probe/policy-freshness.mjs +74 -0
  36. package/src/probe/probe.mjs +48 -23
  37. package/src/probe/runners/discovery.mjs +2 -0
  38. package/src/probe/runners/jest.mjs +2 -2
  39. package/src/probe/runners/lifecycle.mjs +58 -0
  40. package/src/probe/runners/playwright.mjs +6 -2
  41. package/src/probe/runners/python.mjs +10 -4
  42. package/src/probe/runners/shared.mjs +12 -6
  43. package/src/probe/runners/vitest.mjs +3 -3
  44. package/src/probe/universe.mjs +47 -0
  45. package/src/probe/worktree.mjs +3 -1
  46. package/src/render.mjs +17 -1
  47. package/src/replay/replay.mjs +8 -4
  48. package/src/scaffold/admission.mjs +117 -0
  49. package/src/scaffold/append.mjs +41 -0
  50. package/src/scaffold/apply-append.mjs +71 -0
  51. package/src/scaffold/fix-input.mjs +119 -0
  52. package/src/scaffold/fix-paths.mjs +90 -0
  53. package/src/scaffold/intent.mjs +2 -0
  54. package/src/scaffold/open-append.mjs +44 -0
  55. package/src/scaffold/plan-append.mjs +37 -0
  56. package/src/scaffold/plan-intent-input.mjs +38 -0
  57. package/src/scaffold/prepare-append.mjs +46 -0
  58. package/src/scaffold/request.mjs +30 -0
  59. package/src/scaffold/scaffold.mjs +23 -10
  60. package/src/status/status.mjs +34 -9
  61. package/src/sweep/pool.mjs +33 -0
  62. package/src/sweep/sweep.mjs +9 -4
package/CHANGELOG.md CHANGED
@@ -7,6 +7,57 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.17.0] - 2026-10-01
11
+
12
+ Automated weekly release — everything merged since `v0.16.0`.
13
+
14
+ ### Changed
15
+
16
+ - feat: integrate adoption guidance and machine-safe probing (#189)
17
+ - chore(deps-dev): bump vitest from 5.0.1 to 5.0.2 in the all-npm group (#187)
18
+ - chore(homebrew): sha256 for v0.16.0 (#186)
19
+
20
+
21
+ ### Added
22
+
23
+ - Unannotated scaffold defaults use unfinished `TODO-CLAIM-N` identifiers rather than file/function-shaped claim names. Allocation avoids supplied IDs; explicit IDs/annotations and existing evidence remain unchanged, while sweep pooling preserves distinct candidates.
24
+
25
+ - Read-only `scaffold --from-document <local-text-path>` and `--from-fix <full-commit-ID>` inspection, backed by the closed `authoring-input` report contract. Validation retains historical partition counts and rejects unsafe/private paths, inconsistent identities and optimistic verification labels. Stdout-only output emits no document text or claims; input inspection requires independent intent and is not verification.
26
+
27
+ - Explicit `scaffold <source...> --into <existing-draft.json> --claim <ID>` appends mechanically proposed faults while preserving supplied intent and existing faults. `--json` previews without writes; ordinary scaffold remains compatible. Draft updates use bounded admission, owned locking and private recovery, and refuse stale/unsafe inputs or unconfirmed writes rather than claiming verified coverage.
28
+
29
+ - Sweep keeps equal-basename and repeated-annotation drafts distinct before selection, assigning collision-free pooled claim IDs without changing fault content, metadata or defenders.
30
+
31
+ - New mechanical scaffold drafts declare `inferred` origin and ask for intended observable behavior from a requirement, ADR, bug or incident. CLI/status/skill guidance explains the source handoff without authenticating references or promoting annotations/tests into independent intent. Existing supplied scaffold metadata is preserved.
32
+
33
+ - Explicit `probe --require-origin <kinds>` checks declared origins on complete confirmed current-project probes, rechecks native universe and file bindings, and reports failed/unavailable policy without suppressing survivors through baselines or severity floors. Kinds do not authenticate independence; custom/static universes are unavailable. Offline status audits current declarations without certifying freshness; brief retains labeled recorded-run policy before caps and filtering.
34
+
35
+ - Candidate evidence/status/brief schemas accept a closed optional declared-origin policy result, with unsuppressed fault identities, refusal reasons, consistent states and probe exit codes. This is an audit contract, not authenticated independence.
36
+
37
+ - Claim sources share a closed origin vocabulary across claims and evidence, adding `doc`, `bug`, `incident`, `review` and `inferred`. Status reports current declared origins; evidence and briefs summarize all recorded origins before filtering/caps, separating distinct claims from records and reporting conflicting declarations. Human output labels these as unauthenticated declarations. Labels do not change default gates; origin policy requires explicit selection.
38
+
39
+ - Claims/status missing-source-link advisories identify distinct claim IDs and suggest read-only annotation preview without changing verification state, next action or exit codes; annotation-sourced claims retain their existing drift error.
40
+
41
+ - Explicit `claims --annotate` read-only file-header preview and `--apply` authoring for JavaScript/TypeScript and Python, with claim selection, bounded all-target admission, owned locking, private recoverable originals and truthful partial-failure reporting. Authoring JSON uses a closed `annotations` contract; placement is not verification or authentication of intent.
42
+
43
+ ### Fixed
44
+
45
+ - Clarify that cold probes and changed discovery/configuration can require full verification; unchanged target and defender files alone do not guarantee a cheap CI run.
46
+
47
+ - Status marks evidence stale when a recorded automatic-discovery dependency (including a negative candidate, barrel or resolver config) changes or becomes unavailable. Previously unchanged defender lists could hide this drift. Native test-universe policy freshness remains pending.
48
+
49
+ - Annotation scanning and placement ID admission avoid excessive regex backtracking on long alphanumeric tokens without a hyphen; status advice uses bounded, failure-aware scanning so unrelated source failures cannot prevent its verification decision.
50
+
51
+ - Claim statement, severity, source and producer edits now invalidate cached probe evidence and appear as stale in status. Old kills cannot silently carry an edited requirement or provenance declaration.
52
+
53
+ ### Changed
54
+
55
+ - Reduce repeated self-probe setup with focused defenders for 40 existing fault checks, retaining the complete native integration regressions, all 327 fault recipes, three-run confirmation and existing cost budgets.
56
+ - Bound built-in runner concurrency to one worker by default, with `--workers` overrides, and cap outer regression workers at two (one on a two-core machine).
57
+ - Drain noisy runner stdout, bound diagnostics and report reads, and terminate owned runners before restoring mutations and scratch worktrees on cancellation.
58
+ - Run the project-pinned Playwright CLI directly instead of spawning npm for each confirmation, and reset Python interpreter caches at measurement boundaries.
59
+ - Record actual probe elapsed time and unique fresh runner invocations separately from historical attributed costs; preserve all confirmation and cost gates, and invalidate reuse when native worker policy changes.
60
+
10
61
  ## [0.16.0] - 2026-09-30
11
62
 
12
63
  Automated weekly release — everything merged since `v0.15.1`.
package/README.md CHANGED
@@ -127,9 +127,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
127
127
  | npm | `npm i -D testguard-cli` then `npx testguard probe` |
128
128
  | pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
129
129
  | Homebrew | `brew tap raccioly/tap && brew install testguard` |
130
- | GitHub Action | `uses: raccioly/testguard@v0.16.0` — see [`action.yml`](./action.yml) |
130
+ | GitHub Action | `uses: raccioly/testguard@v0.17.0` — see [`action.yml`](./action.yml) |
131
131
  | pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-probe` |
132
- | GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.16.0/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
132
+ | GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.17.0/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
133
133
 
134
134
  Projects that set `min-release-age` in `.npmrc` cannot see a version published
135
135
  less than that many days ago (`ENOVERSIONS`); install that one with
@@ -142,6 +142,7 @@ npx testguard-cli init # install the agent layer at the git
142
142
  npx testguard-cli status --json # where the project is, and the ONE next action
143
143
  npx testguard-cli claims # what does this project claim, and is each claim probeable?
144
144
  npx testguard-cli claims --check-anchors # fail fast if an exact anchor moved or a replacement no longer parses
145
+ npx testguard-cli claims --annotate --claim AUTH-ADMIN --json # read-only file-header placement preview
145
146
  npx testguard-cli scaffold src/auth.ts # propose faults for a file, as a draft to keep or drop
146
147
  npx testguard-cli probe # try to falsify each claim; report what the tests missed
147
148
  npx testguard-cli admit test/auth.test.ts --claim AUTH-ADMIN # does this test satisfy the two-gate rule?
@@ -158,6 +159,54 @@ unclaimed changes or invalid fault anchors, `2` a precondition failed and
158
159
  nothing was probed, `3` usage.
159
160
  Every command accepts `--json`.
160
161
 
162
+ ### Intent-first drafts
163
+
164
+ Scaffold drafts propose mechanical faults, not intended behavior. New claims
165
+ declare `inferred` origin even with an annotation. Supply the intended observable
166
+ behavior from a requirement, ADR, bug or incident independently of the code:
167
+ inputs, expected outcome and forbidden outcome. Record the supplied source
168
+ kind/ref; when unavailable keep inferred. References are not authenticated or
169
+ fetched, and passing tests do not establish independent intent. Review proposals
170
+ against the supplied intent before adopting and probing; existing `--claim`
171
+ metadata is preserved.
172
+
173
+ ### Optional annotation placement
174
+
175
+ Ordinary `claims` output and status notes identify missing scanned source links
176
+ as advisories. Counts are distinct claim IDs, not faults. They do not change
177
+ verification state, its next action or exit codes; annotation-sourced missing
178
+ IDs still produce the existing claims drift error. Scanned text is a lexical
179
+ link, not authenticated intent or source ownership.
180
+ Status advice uses a bounded scan (10,000 entries, 1,000 files, 64 directory
181
+ levels, 256 KiB per file, 2 MiB total); incomplete scans report unavailable
182
+ advice without concluding an ID is absent. Keep the original `--claims`
183
+ argument for preview when inspecting a custom claims selection.
184
+
185
+ `testguard claims --annotate` previews JavaScript/TypeScript or Python
186
+ file-header links for declared claims; repeat `--claim` or use comma-separated
187
+ IDs to select claims. Add `--apply` only after reviewing the selection to
188
+ explicitly write annotations. Each invocation recomputes its plan: an earlier
189
+ preview is not a saved approval token. No `--out` or unrelated inspection
190
+ switches are accepted with authoring. JSON conforms to the `annotations`
191
+ authoring schema; it contains no source/proposed contents or probe verdicts.
192
+
193
+ This is file-level placement, not symbol ownership or proof of correctness.
194
+ Missing annotations do not disable exact fault-anchor validation, and placement
195
+ does not change claim origins, evidence or baselines. Unsupported, ambiguous or
196
+ unsafe targets block the complete selection. Original/proposed files are capped
197
+ at 2 MiB and the aggregate at 16 MiB. Preserve BOM, shebang, Python encoding
198
+ cookie, newline style and permissions. A preview exits 0 when applicable, 2 when
199
+ refused; apply exits 0 only after verified completion, 2 for refusal/partial
200
+ failure; malformed options exit 3.
201
+
202
+ Apply uses an exclusive owned lock and retains fsynced originals in an owner-only
203
+ recovery directory outside the checkout (the result reports its location).
204
+ External claims files can be previewed but cannot authorize writes. It never
205
+ reclaims an existing lock or automatically rolls back over editor changes.
206
+ This is recoverable, not a cross-file atomic transaction: interruption or
207
+ arbitrary concurrent edits can leave partial content. Compare retained originals
208
+ manually before recovery; inspect ownership before removing a leftover lock.
209
+
161
210
  ## How it works
162
211
 
163
212
  ```bash
@@ -308,9 +357,11 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
308
357
  `--runner-cmd "<cmd> {files} … {out}"`; if the scratch worktree cannot
309
358
  see your `node_modules`, pass `--node-modules <dir>`.
310
359
  3. **Baseline** freezes every non-passing fingerprint. Later probes suppress
311
- what was already known and exit non-zero only on what is new. Claims whose
312
- source and defenders are unchanged reuse their prior verdict, so a probe
313
- in CI costs only what changed. A baseline frozen from `--include-dirty`
360
+ what was already known and exit non-zero only on what is new. Prior verdicts
361
+ can be reused when their recorded inputs and runner policy still match.
362
+ Cold runs, changed discovery/configuration or `--no-reuse` can require a
363
+ full probe; unchanged target and defender files alone do not guarantee reuse.
364
+ A baseline frozen from `--include-dirty`
314
365
  evidence records the snapshot and points at the *parent* of the commit
315
366
  that will carry your tests; after you commit, a clean `probe` plus
316
367
  `baseline --restamp` moves it to that commit — only when the fingerprints
@@ -494,6 +545,11 @@ proposed 510 faults, probed 10 (cap 10), deferred 350.
494
545
 
495
546
  Three rules make it safe to run on a repository that has never seen this tool:
496
547
 
548
+ Pooled draft IDs are unique even when source files share basenames/functions or
549
+ annotation IDs. Sweep reserves original IDs and suffixes collisions before
550
+ selection, keeping every fault, its metadata and defenders attached to its own
551
+ draft. These are proposal identities, not authenticated claims.
552
+
497
553
  - **It never writes `testguard.claims.json`**, and never will. A claim is a
498
554
  sentence someone is willing to stand behind; a sentence nobody wrote is not
499
555
  one. Its drafts land beside the evidence to keep or drop.
@@ -713,12 +769,12 @@ detected base that does not resolve is a warning for `status` and `brief`
713
769
  # GitHub Actions
714
770
  - uses: actions/checkout@v4
715
771
  with: { fetch-depth: 0 }
716
- - uses: raccioly/testguard@v0.16.0
772
+ - uses: raccioly/testguard@v0.17.0
717
773
  with: { command: gate }
718
774
 
719
775
  # GitLab CI — the component-shaped template: gate + probe, brief as an artifact and, opted in, as a merge-request note
720
776
  include:
721
- - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.16.0/packaging/gitlab/testguard.gitlab-ci.yml'
777
+ - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.17.0/packaging/gitlab/testguard.gitlab-ci.yml'
722
778
  inputs: { dir: backend, post_note: true } # post_note needs TESTGUARD_GITLAB_TOKEN (api scope); one note, updated in place
723
779
  ```
724
780
 
@@ -968,16 +1024,20 @@ said what they want.
968
1024
 
969
1025
  ### What the probe costs, and why a gate gets slow
970
1026
 
971
- A probe's wall clock is not a function of how many claims you have. Per claim
972
- it is roughly
1027
+ A fresh probe grows with the selected faults, confirmation count and the cost
1028
+ of their defenders. Shared baselines and valid verdict reuse avoid some repeated
1029
+ work. Roughly, the fresh work is
973
1030
 
974
1031
  ```
975
- (baseline runs + faults × --confirm) × the cost of that claim's defender SET
1032
+ fresh baseline runs × defender-set cost
1033
+ + Σ (fault confirmations × that fault's defender-set cost)
1034
+ + survivor diagnostics and setup/discovery work
976
1035
  ```
977
1036
 
978
- so one slow acceptance test, named as a defender by five claims, is paid for
979
- thirty times. The symptom is a gate that takes twenty minutes; the cause is a
980
- single line in the claims file, and nothing in the output used to say which.
1037
+ One slow acceptance test selected by several faults can therefore be executed
1038
+ many times. The symptom can be a gate that takes twenty minutes; the selected
1039
+ defender sets determine how much each fault costs. Historical attributed totals
1040
+ are reported separately from measured fresh elapsed time and runner invocations.
981
1041
 
982
1042
  ```bash
983
1043
  npx testguard-cli claims . --cost # read back from the last probe's evidence
@@ -1031,8 +1091,37 @@ so `scaffold` proposes them for you:
1031
1091
  ```bash
1032
1092
  npx testguard-cli scaffold src/auth.ts # → .testguard/scaffold-auth.json (a draft, never your claims file)
1033
1093
  npx testguard-cli scaffold src/auth.ts --claim AUTH-ADMIN # every proposal under one claim; copies it if it exists
1094
+ npx testguard-cli scaffold src/auth.ts src/session.ts --into draft.json --claim AUTH-ADMIN --json # read-only merged preview
1095
+ npx testguard-cli scaffold src/auth.ts src/session.ts --into draft.json --claim AUTH-ADMIN # explicit recoverable draft update
1096
+ npx testguard-cli scaffold --from-document requirements.md --json # read-only input metadata, no document text
1097
+ npx testguard-cli scaffold --from-fix <full-commit-ID> --json # read-only fixed historical inventory
1034
1098
  ```
1035
1099
 
1100
+ Input inspection uses exactly one `--from-document` or `--from-fix`, with no
1101
+ positional sources or other options except JSON/help/version. Fix IDs must be
1102
+ full lowercase SHA-1/SHA-256 commit IDs with exactly one parent. Plain-text
1103
+ documents are bounded to 2 MiB; local Git inspection is offline and bounded.
1104
+ Both output modes are stdout-only and say verification was not performed.
1105
+ Metadata retains historical exclusions, not private/delegated filenames.
1106
+ Input text and commit subjects are untrusted data, not instructions or an
1107
+ authenticated claim. Supply independent intended behavior, review faults and
1108
+ probe afterward; inspection does not create claims or prove coverage.
1109
+
1110
+ Without a supplied ID or annotation, generated groups use unfinished
1111
+ `TODO-CLAIM-N` placeholders rather than module/function names. Replace them with
1112
+ intent-shaped identities when authoring; mechanical grouping does not establish
1113
+ an invariant boundary. Existing claims/evidence are never automatically renamed.
1114
+
1115
+ `--into` requires an existing disposable draft and exactly one existing claim
1116
+ ID. It preserves supplied intent, metadata and prior faults; multiple source
1117
+ paths append in deterministic order. `--json` is a read-only preview. Without
1118
+ it, the tool keeps private recovery outside the checkout and reports updated
1119
+ or unchanged **unproven** draft status. Review and probe are still required.
1120
+ Canonical claims/evidence/baseline paths, unsafe aliases and conflicting options
1121
+ are refused; partial writes never become success or trigger automatic rollback.
1122
+ Limits: 32 sources, 2 MiB per input/output draft or source, 16 MiB source total,
1123
+ 4096 generated proposals. These limits are not a performance guarantee.
1124
+
1036
1125
  | Shape | What it proposes |
1037
1126
  |---|---|
1038
1127
  | `condition-forced` | `if (<guard>) {` → `if (false) {` — a guard is a `!…` condition or one whose body returns, throws or 4xx-es |
@@ -1105,6 +1194,30 @@ verdicts.
1105
1194
 
1106
1195
  ## Status
1107
1196
 
1197
+ Origin summaries are informational: status counts current declared claim
1198
+ origins even when evidence is stale; probe evidence and briefs count recorded
1199
+ origins across all records before filtering or truncation. Distinct claims and
1200
+ records have separate totals; conflicting recorded kinds or references count
1201
+ as mixed claims. Labels such as `spec`, `bug` or `incident` do not authenticate
1202
+ independence and never waive a failed measurement or change default gates.
1203
+ References remain opaque and are not echoed in these summaries. Older strict
1204
+ readers reject these additive fields; use the matching candidate reader for
1205
+ unreleased formats. The candidate `probe --require-origin spec,adr,bug,incident`
1206
+ option selects declared kinds for a complete confirmed current-project probe.
1207
+ Repeat the option or use comma-separated kinds; partial, provisional, explicit
1208
+ historical and ignore-dirty combinations are refused. Failed policy exits 1
1209
+ even when baseline debt or a severity floor hides ordinary findings; unavailable
1210
+ current bindings exit 2. The command rechecks native runner/configuration
1211
+ membership, current claims, targets, defenders and discovery dependencies before
1212
+ publishing. Custom/static runner universes cannot certify this boundary. This is
1213
+ a declaration check, not authenticated independence or an atomic disk snapshot.
1214
+ Offline status recomputes a current-declaration audit using the recorded eligible
1215
+ selection, always marking native freshness unavailable without launching a
1216
+ runner. It does not implicitly activate a new gate or replace status's next
1217
+ action. Briefs preserve the recorded-run policy before caps and baseline
1218
+ filtering, explicitly not current freshness. A zero-item cap never turns
1219
+ survivors into an all-defended message.
1220
+
1108
1221
  **v0.5.** Eleven commands (`status`, `init`, `claims`, `probe`, `admit`,
1109
1222
  `replay`, `baseline`, `brief`, `gate`, `scaffold`, `sweep`, `mcp`), vitest, jest,
1110
1223
  Playwright and Python (pytest / stdlib unittest) runners, hand-authored faults
@@ -1120,6 +1233,32 @@ the transfer of a calibration between repositories — `replay` measures it now;
1120
1233
  whether it carries to a repository with no history is unproven. Each is
1121
1234
  designed for; none is claimed.
1122
1235
 
1236
+ ### Keeping developer machines responsive
1237
+
1238
+ Built-in probe, sweep, replay and admission runners default to one worker. Use
1239
+ `--workers 2` for a larger Vitest/Jest/Playwright worker ceiling; `--serial` always
1240
+ uses one. Python remains serial. Faults and mixed-runner groups run sequentially.
1241
+ Custom `--runner-cmd` commands, browser subprocesses, native library threads and
1242
+ test-created processes are not an OS resource sandbox. The regression suite also
1243
+ caps outer workers at two, or one when fewer than three CPUs are available
1244
+ in an interactive run; dedicated CI uses at most two because its acceptance tests start nested runners.
1245
+
1246
+ `--budget` bounds each runner invocation; `--command-budget` bounds a complete
1247
+ command cooperatively and refuses partial result publication. Cancellation kills
1248
+ owned runners before restoring mutations and removing scratch worktrees. Already
1249
+ reparented daemons remain outside the portable cleanup guarantee. No runner output
1250
+ limit can count as an assertion failure or detection.
1251
+
1252
+ New probe evidence includes actual elapsed time, unique fresh runner time and
1253
+ setup/discovery/other overhead. `probe --cost` and `claims --cost` display this
1254
+ separately from historical attributed record costs. Existing cost ceilings are
1255
+ unchanged; old evidence does not supply a measured breakdown.
1256
+
1257
+ Playwright confirmations run the same project-resolved CLI used for discovery,
1258
+ avoiding an npm subprocess per run. Python projects whose configuration requires
1259
+ pytest-xdist must support serial execution; an incompatible configuration remains
1260
+ unproven rather than bypassing the worker policy.
1261
+
1123
1262
  ## Licence
1124
1263
 
1125
1264
  MIT.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testguard-cli",
3
- "version": "0.16.0",
3
+ "version": "0.17.0",
4
4
  "description": "Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -1,5 +1,108 @@
1
1
  # Gate semantics
2
2
 
3
+ ## Declared claim origins
4
+
5
+ Status rechecks every recorded automatic-discovery dependency hash, including
6
+ negative candidate tests, barrels and resolver configuration. Edited, deleted,
7
+ unreadable or non-regular/escaping inputs make the evidence stale without
8
+ rewriting its verdicts. Legacy omission remains readable but does not establish
9
+ a complete origin-policy freshness binding. Rechecking recorded dependencies
10
+ alone does not detect a newly added test/configuration: native test-universe
11
+ recomputation remains required before origin-policy activation.
12
+
13
+ Canonical and recorded claim sources share the closed kinds `spec`, `adr`,
14
+ `annotation`, `comment`, `manual`, `doc`, `bug`, `incident`, `review`, `inferred`.
15
+ These are declarations, not authentication of independence, chronology,
16
+ production occurrence or correctness. The optional reference remains opaque
17
+ metadata (at most 512 characters); verification does not fetch it. Producer
18
+ provenance and measured test independence are separate signals.
19
+ Adding kinds does not change verdicts, baseline/fingerprint identity or default
20
+ gates. Existing ranking weights stay unchanged; new kinds use the existing 0.9
21
+ fallback, not an invented strength hierarchy. Existing documents remain valid
22
+ under the candidate reader; older closed-schema readers reject new kinds.
23
+ Optional `origins` summaries on evidence, status and brief have closed kind
24
+ buckets and safe nonnegative integer counts. Distinct claim buckets plus
25
+ `mixed` equal the claim total; record buckets equal the record total. Status
26
+ uses `basis: declared`, omits records, has zero mixed claims and binds its
27
+ total to current `counts.claims`, even when evidence is stale. Evidence and
28
+ brief use `basis: recorded` and require record counts. Evidence validation
29
+ recomputes from every actual record regardless of verdict: each claim ID
30
+ counts once, with conflicting source kind OR reference counted as one mixed
31
+ claim. Object-key order is not a conflict; absent and empty references differ.
32
+ Brief totals bind to the complete summary, not its capped/filtered items.
33
+ Status and brief omit the source projections needed to independently validate
34
+ their kind split; arithmetic validity is not source authentication. Legacy
35
+ documents without summaries remain valid; old closed readers reject them.
36
+ Status supplies current declared origins once validated claims are available,
37
+ including empty, unprobed, invalid-anchor and stale-evidence states. Probe
38
+ evidence and evidence-backed briefs supply recorded origins from all records,
39
+ including killed and non-killed records, before caps or baseline/floor filtering.
40
+ Human status/probe/brief text and Markdown label the basis and print "declared
41
+ origins, not authenticated independence". Evidence-free briefs and projects
42
+ without a claims file omit the summary rather than inventing declarations.
43
+ Probe JSON retains status's current declared basis; saved evidence has its own
44
+ recorded basis. These informational fields do not change states, next actions,
45
+ exit codes, fingerprints or default gates. Opt-in origin policy remains pending.
46
+
47
+ ## Explicit annotation authoring
48
+
49
+ Ordinary `claims` reports a separate `annotationAdvisory` with distinct sorted
50
+ `missingIds` and informational `notes`; status uses its existing informational
51
+ `notes`. Missing links reduce discoverability and annotation reconciliation,
52
+ not exact-anchor validation. Advice supplies `testguard claims --annotate` for
53
+ read-only preview; it never changes state, next verification action, verdicts,
54
+ source kinds or exit codes. Annotation-sourced missing IDs still retain the
55
+ existing claims drift error. Scanned occurrences are lexical links, not proof
56
+ of intent, source ownership or authenticated provenance.
57
+ Status advisory scanning is bounded to 10,000 directory entries, 1,000 files,
58
+ 64 directory levels, 256 KiB per file and 2 MiB total read bytes. Read failures,
59
+ unsafe file identities or limits produce unavailable advice, never a claim
60
+ that an ID is absent from an incomplete scan. Verification state/next still
61
+ comes from its existing inputs. For custom claims, preserve the original
62
+ `--claims` argument when following the preview suggestion.
63
+ Scanner and placement ID prefixes use mandatory separators between repeated
64
+ alphanumeric groups, avoiding overlapping quantifiers on tokens lacking a
65
+ hyphen without changing the supported identifier vocabulary.
66
+
67
+ `claims --annotate` previews file-header placement without writes. Add `--apply`
68
+ to explicitly apply a newly computed plan; a previous invocation's preview is
69
+ not a saved approval token. Repeated/comma-separated `--claim` selects IDs;
70
+ unknown or empty IDs are usage errors, never an implicit all-claims selection.
71
+ Authoring flags are valid only on `claims`; `--apply` requires `--annotate`.
72
+ Authoring rejects unrelated inspection/output switches and writes results only
73
+ to stdout. It never updates claims metadata, evidence or baselines.
74
+
75
+ The closed `annotations` document describes authoring, not verification. It
76
+ contains selected IDs, admitted target/action summaries, refusals and, for
77
+ apply, touched/verified/unchanged files and explicit failure/recovery details.
78
+ It excludes source/proposed contents, inode identities, hashes and absolute
79
+ source/root paths. Preview exits 0 when applicable, 2 when refused. Apply exits
80
+ 0 only for a fully verified apply (including idempotent no-op), 2 for refusal or
81
+ partial failure. Invalid invocation exits 3. No authoring result changes probe
82
+ verdicts, rank, coverage, source-kind declarations or authentication of intent.
83
+ Failure reasons are stage-specific public summaries: raw syscall details can
84
+ contain absolute input paths and remain private, never passed through to JSON.
85
+
86
+ Placement supports JavaScript/TypeScript and Python file-level line-comment
87
+ headers, not symbol ownership. Exact anchors remain checked without annotations.
88
+ Every selected target must pass conservative path, regular-file, UTF-8/newline,
89
+ anchor and size admission: 2 MiB per original/proposed file and 16 MiB aggregate
90
+ charged before reads plus proposal growth. Any refusal blocks all writes.
91
+ Preserve BOM, shebang, Python encoding cookies, newline style, inode and mode.
92
+ Only leading line-comment headers are placement witnesses, not string contents.
93
+
94
+ Apply reloads actual claims under an exclusive owned lock, replays all targets,
95
+ and fsyncs owner-only originals/manifest outside the canonical project before
96
+ writing. Recheck claims and targets immediately before each non-truncating
97
+ descriptor write; verify output, distinguish potentially touched from verified
98
+ files, and retain private recovery on partial failure. Reject the actual claims
99
+ file as a target, root/descendant symlinks and recovery inside the checkout.
100
+ Resolve parent aliases to bind the actual root. Never reclaim another writer's
101
+ lock or automatically roll back over an editor's changes. Stage and fsync the
102
+ final recovery journal before final naming. This is recoverable, not atomic:
103
+ arbitrary same-inode editors and process/power failure can race or interrupt
104
+ writes; directory durability across power loss is not guaranteed.
105
+
3
106
  ## Fault-specific defender selection
4
107
 
5
108
  A fault's `defendedBy`, when present, overrides its claim's declaration.
@@ -108,6 +211,17 @@ Rules that follow from the table:
108
211
  the fault is probed again: the cost of re-measuring is a run, the cost of
109
212
  the other direction is a verdict nobody measured.
110
213
 
214
+ **Claim metadata is also an input to reuse.** The recorded claim id,
215
+ statement, severity, source (including its reference), and optional
216
+ producer must match the current claim. Object-key order is not a change;
217
+ field values and producer presence are. Metadata edits make `status`
218
+ stale and require a fresh probe, even when the fault and defender inputs
219
+ are identical. A reused record must never be silently relabelled as a
220
+ different requirement or origin. Legacy records remain readable; missing
221
+ metadata cannot establish equality with a current declaration. This
222
+ does not change verdicts or baseline fingerprints, and a source label
223
+ still does not authenticate independent intent.
224
+
111
225
  9. **The fault must be the code that ran.** A green baseline proves the
112
226
  harness is not reporting everything as broken. It proves nothing about the
113
227
  opposite direction: a fault the interpreter never executes leaves the
@@ -467,6 +581,69 @@ Security- and money-shaped path names are explicit tie-breaking signals, never
467
581
  semantic claims about a file. Churn, path risk and claim coverage remain raw
468
582
  facts: they are never collapsed into one health score.
469
583
 
584
+ ### Intent-first scaffold authoring
585
+
586
+ Unannotated generated draft groups use unfinished `TODO-CLAIM-N` identifiers,
587
+ not file/function-shaped names. Allocation is deterministic in scan order and
588
+ reserves explicit annotations and supplied existing IDs before choosing ordinals.
589
+ It never copies unrelated existing metadata into a generated placeholder. Explicit
590
+ claim IDs/annotations retain their existing behavior; neither annotations nor
591
+ placeholder IDs authenticate intent. Grouping remains mechanical candidate
592
+ organization, not invariant ownership. Sweep's pooled allocator keeps repeated
593
+ per-file placeholders distinct without changing fault content. Existing claims
594
+ and recorded evidence are not renamed. The claims schema/validator is unchanged:
595
+ these are valid draft IDs and still require supplied intent and review/probing.
596
+
597
+ Explicit `scaffold <source...> --into <existing-draft.json> --claim <ID>` appends
598
+ mechanically derived faults only to that existing selected claim. Preview JSON
599
+ is a conforming claims document (the existing claims schema and validator), not
600
+ verification evidence. `--json` creates no files, locks or recovery. Without it,
601
+ only the explicitly selected disposable draft is updated; unchanged repeats
602
+ preserve exact bytes/mtime. Supplied statements/origins/metadata and existing
603
+ faults/sibling claims remain intact. Review and probe are still required.
604
+
605
+ `--into` is scaffold-only, singular and requires one nonempty existing claim ID
606
+ and at least one source. Repeated/comma selections, `--out`, `--claims` and any
607
+ explicit unrelated option refuse with usage exit 3 before reading/writing.
608
+ Help/version retain their normal early-exit behavior. Ordinary scaffold remains
609
+ one-file; annotation `--apply` retains its separate claims-only meaning.
610
+
611
+ Refuse canonical `testguard.claims.json` names and built-in evidence/baseline
612
+ destinations, unsafe aliases/symlinks/hardlinks/nested projects and nonregular
613
+ inputs. Admission limits are 32 sources, 2 MiB per source/draft, 16 MiB aggregate
614
+ source bytes, 4096 generated proposals and 2 MiB serialized output. Caps refuse,
615
+ never truncate proposals. These are not whole-command performance guarantees.
616
+
617
+ Changed drafts require a cooperative owned exclusive lock, current input replay,
618
+ external private original/started/result recovery journals, nofollow descriptor
619
+ publication through `writeSpecDoc`, short-write handling, fsync and exact
620
+ byte/schema/path/source verification. Refusal, stale inputs, partial writes,
621
+ verification or cleanup/journal failure exit 2, report recovery when available,
622
+ and never print updated success. No automatic rollback, stale-lock reclamation,
623
+ atomic multi-file snapshot or power-loss transaction is promised. Exit 0 means
624
+ authoring completed, not defended coverage. Private recovery is not evidence.
625
+
626
+ Sweep pools per-file drafts before selection. Repeated claim IDs from equal
627
+ basenames/functions or annotations do not merge unrelated claims: reserve all
628
+ original IDs, then allocate unused numeric suffixes to collisions in full
629
+ target-path/original-ID order, within the existing 128-character limit. Metadata,
630
+ fault IDs, injection anchors and defenders are preserved. Drafts, selected probe
631
+ claims and finding lookup share these identities. This affects fresh collision
632
+ cases only; existing documents, verdicts, caps and exit rules remain compatible.
633
+
634
+ Scaffold's mechanically generated new claims declare `source.kind: inferred`,
635
+ including proposals grouped by an annotation. Their TODO asks for intended
636
+ observable behavior from a requirement, ADR, bug or incident: inputs, expected
637
+ outcome and forbidden outcome. Existing supplied claim metadata retains its
638
+ statement and declared origin under the existing `--claim` rules. Derived fault
639
+ provenance remains derived. No automatic source promotion or canonical write.
640
+
641
+ Human CLI, status authoring actions and the installed skill hand off independent
642
+ intent authoring. Record a supplied source kind/ref; when unavailable keep
643
+ inferred. References and annotations do not authenticate intent and are never
644
+ fetched. Passing tests do not establish independent intent. State, next action,
645
+ verdict and exit-code semantics are unchanged; this is authoring guidance.
646
+
470
647
  When existing evidence is clean but more source modules are unclaimed than
471
648
  claimed, the next action is to scaffold the highest-churn unclaimed module.
472
649
  Earlier correctness states still win: invalid anchors, stale evidence and
@@ -580,6 +757,48 @@ evidence; they simply do not turn CI red.
580
757
 
581
758
  ## Exit codes
582
759
 
760
+ ### Explicit declared-origin policy result contract
761
+
762
+ Optional `originPolicy` on evidence, status and brief (and status's probe `run`)
763
+ records an explicitly selected eligible-kind set, not authenticated independence.
764
+ Legacy absence is not a pass. `probe --require-origin <kinds>` selects a
765
+ repeatable comma-separated nonempty set from the closed source vocabulary.
766
+ It requires a complete current-project run with confirm >= 3 and refuses
767
+ `--claim`, `--allow-empty`, explicit `--ref` and `--ignore-dirty` as usage errors
768
+ before loading or writing. No policy is inferred when the option is absent.
769
+ After measurement, the command reloads current claims and collects the current
770
+ native runner universe under the same command budget, then binds targets,
771
+ resolved defenders and current discovery dependencies. Missing bindings or
772
+ discovery failure yields unavailable; custom/static runner universes cannot
773
+ certify this boundary. Expired command budgets still write no partial result.
774
+ This is point-in-time admission, not an atomic filesystem snapshot. Stored
775
+ results are not freshness authority. Offline status reuses only the recorded
776
+ eligible-kind selection, recomputes current declarations/complete identities,
777
+ and marks freshness unavailable without executing native discovery. This is
778
+ an audit projection, not implicit activation of a new gate: the existing
779
+ status state, next action and exit remain unchanged. Briefs preserve the
780
+ recorded policy before baseline filtering or item caps, labeled recorded-run
781
+ only and not current freshness. No absent policy is synthesized. A zero-item
782
+ cap cannot turn unproven faults into an all-defended message.
783
+
784
+ The closed result carries `state` (`passed`, `failed`, `unavailable`), sorted
785
+ unique `eligibleKinds`, current `claims`/`faults` denominators, sorted unique
786
+ `ineligibleClaims`, unique `nonKilledFaults` (`claimId`, `faultId` only), and
787
+ sorted unique `unavailableReasons`. References and paths are excluded.
788
+ Refusal reasons require unavailable; otherwise any ineligible claim or
789
+ non-killed fault requires failed. Only a nonempty failure-free universe can
790
+ pass. Baseline debt and severity floors cannot suppress policy failures.
791
+
792
+ Evidence validation binds actual non-killed identities, and, for an available
793
+ policy, unique recorded denominators, recorded eligibility, unmixed origins and
794
+ confirmed injection. Validation cannot authenticate current metadata or disk
795
+ freshness: evaluation must separately bind the complete current fault universe
796
+ and existing input freshness. Projected status/brief bind denominators, not
797
+ hidden source declarations. Brief policy describes its recorded run only.
798
+ Completed probe JSON with unavailable policy exits 2; failed policy or ordinary
799
+ new findings exits 1; passed policy without ordinary findings exits 0.
800
+ Partial/provisional invocations cannot carry an origin policy.
801
+
583
802
  | Code | Meaning |
584
803
  |---|---|
585
804
  | `0` | No new gating findings at or above the severity floor. |
@@ -598,3 +817,79 @@ output is the existing `no-claims` status document, without run metadata;
598
817
  it never reports a clean measurement. Missing or invalid claims still fail,
599
818
  and selected claims cannot be silently skipped. Run `gate` separately to
600
819
  enforce coverage of changed files during adoption. Integrations must opt in.
820
+
821
+ ## Read-only authoring input (candidate)
822
+
823
+ `authoring-input` is metadata-only inspection, never claims or evidence. Its
824
+ closed purpose/verification/next fields explicitly state that verification was
825
+ not performed and independent intent remains to be supplied. Documents expose
826
+ only bounded project-relative file/hash/byte metadata, not text or absolute
827
+ roots. A digest or schema-valid report does not authenticate intent or freshness.
828
+
829
+ Fix inputs retain exact same-width commit/parent/path identities, untrusted
830
+ single-line subjects, selected root/nested scope and full partition accounting.
831
+ Visible rows exclude private/delegated names; their count stays in the total.
832
+ Deleted and unsupported rows cannot become supported source. Validators reject
833
+ duplicate/unsafe/private paths, inconsistent status/mode/zero-ID relations,
834
+ disposition/count mismatches, unsafe subjects and resource overruns. Empty
835
+ documents/inventories are admitted input metadata, not successful verification.
836
+ Readers cannot infer actual history completeness or current input admission
837
+ from shape alone; the runtime must use admitted local inputs without promotion.
838
+
839
+ `scaffold --from-document <local-text-path>` or `--from-fix <full-commit-ID>`
840
+ selects exactly one read-only authoring input. Both token dispatch and direct
841
+ handler entry reject repetition, mixed modes, positional sources and unrelated
842
+ explicit options (only JSON/help/version may accompany selection). Normal
843
+ help/version early exits remain unchanged. Usage refusal exits 3; input admission
844
+ or validated planning failure exits 2 with no success stdout. Success exits 0
845
+ for inspection only, never coverage. Both JSON and human modes are stdout-only.
846
+ JSON uses the validated metadata-only report; human rendering omits text,
847
+ subjects, absolute roots and historical filenames while retaining full fix
848
+ partition counts. Both name verification as not performed and require supplied
849
+ independent intent before reviewing/probing faults. One elapsed budget covers
850
+ input reads and validated serialization; scope resolution never retries with
851
+ fresh time. Existing ordinary scaffold/append/probe behavior is unchanged.
852
+
853
+ ## Resource controls and measurement
854
+
855
+ Built-in runners default to one worker. `--workers N` raises the worker ceiling
856
+ for Vitest, Jest and Playwright; `--serial` overrides it to one. Python remains
857
+ serial and disables pytest-xdist without installing or enabling any plugin.
858
+ Faults and mixed-runner partitions remain sequential. This is a worker ceiling,
859
+ not OS CPU/memory containment; test-created threads, native libraries and custom
860
+ `--runner-cmd` commands manage their own concurrency. Native discovery is bounded
861
+ and uses one worker where applicable.
862
+
863
+ Runner stdout is drained without retention; stderr retains only its last 65,536
864
+ characters. Complete JSON reports are read under the existing 16 MiB discovery
865
+ report admission limit. An oversized, unreadable or incomplete report is an error,
866
+ never an assertion kill. No confirmation count or verdict rule changes.
867
+
868
+ Cancellation terminates only registered children and their attributable process
869
+ trees, waits up to one second for close, restores live mutations and removes the
870
+ registered scratch worktrees. It exits 130 without publishing partial evidence.
871
+ Reparented daemons remain outside the portable containment guarantee. Git
872
+ subprocesses have a 30-second hard deadline; the whole-command budget remains
873
+ cooperative around synchronous setup, and bounded OS cleanup can outlive it.
874
+
875
+ Optional `run.measurements` records monotonic elapsed time, fresh runner execution
876
+ time, invocation count and remaining setup/discovery/other overhead. Each real
877
+ runner invocation is charged once, including negative controls and mixed-runner
878
+ partitions; reused records and cached baselines create no invocation charge.
879
+ `elapsedMs = runnerMs + overheadMs`. Optional `run.workers` records the built-in
880
+ worker ceiling and must be one for serial execution; custom commands omit it.
881
+ Native evidence reuse requires the same recorded worker ceiling and serial policy.
882
+ Older native evidence with no worker ceiling is remeasured; native and opaque
883
+ custom-command policies are not interchangeable.
884
+ `status.cost.measurements` copies these measurements when available. Historical
885
+ `totalMs` and cost gates retain their existing record attribution, including
886
+ reused durations and repeated shared baseline durations. Older evidence has no
887
+ measured breakdown; consumers must not infer it from record totals.
888
+
889
+ The existing Python interpreter cache is reset at probe and replay entry and
890
+ reused within that invocation. Uncached adapter resolution consumes the same
891
+ live per-run deadline as test execution.
892
+
893
+ The regression suite caps outer concurrency at two workers in CI. Interactive
894
+ runs reserve one available CPU and use at most two workers; a two-core machine
895
+ therefore uses one. Nested built-in runners retain their one-worker default.