testguard-cli 0.15.1 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +82 -0
- package/README.md +196 -17
- package/package.json +1 -1
- package/spec/GATE-SEMANTICS.md +316 -0
- package/spec/README.md +27 -2
- package/spec/lib/authoring-input.mjs +40 -0
- package/spec/lib/origins.mjs +56 -0
- package/spec/lib/validate.mjs +129 -1
- package/spec/schemas/annotations.schema.json +65 -0
- package/spec/schemas/authoring-input.schema.json +67 -0
- package/spec/schemas/brief.schema.json +2 -0
- package/spec/schemas/claims.schema.json +8 -8
- package/spec/schemas/common.schema.json +73 -0
- package/spec/schemas/evidence.schema.json +41 -8
- package/spec/schemas/status.schema.json +31 -2
- package/src/brief/brief.mjs +11 -4
- package/src/claims/annotation-placement.mjs +357 -0
- package/src/claims/annotations.mjs +61 -2
- package/src/cli.mjs +40 -5
- package/src/commands/admit.mjs +21 -3
- package/src/commands/annotations.mjs +67 -0
- package/src/commands/claims.mjs +46 -6
- package/src/commands/probe.mjs +50 -7
- package/src/commands/replay.mjs +7 -0
- package/src/commands/scaffold.mjs +49 -2
- package/src/commands/sweep.mjs +7 -0
- package/src/evidence/writer.mjs +12 -4
- package/src/gate/changed.mjs +5 -1
- package/src/git.mjs +4 -4
- package/src/init/init.mjs +6 -0
- package/src/init/templates/SKILL.md +78 -0
- package/src/probe/attribution.mjs +19 -0
- package/src/probe/contention.mjs +20 -2
- package/src/probe/cost.mjs +22 -9
- package/src/probe/inject.mjs +2 -6
- package/src/probe/origin-policy.mjs +69 -0
- package/src/probe/policy-freshness.mjs +74 -0
- package/src/probe/probe.mjs +74 -45
- package/src/probe/runners/discovery.mjs +2 -0
- package/src/probe/runners/jest.mjs +2 -2
- package/src/probe/runners/lifecycle.mjs +58 -0
- package/src/probe/runners/playwright.mjs +6 -2
- package/src/probe/runners/python.mjs +10 -4
- package/src/probe/runners/shared.mjs +12 -6
- package/src/probe/runners/vitest.mjs +3 -3
- package/src/probe/universe.mjs +47 -0
- package/src/probe/worktree.mjs +3 -1
- package/src/render.mjs +17 -1
- package/src/replay/replay.mjs +8 -4
- package/src/scaffold/admission.mjs +117 -0
- package/src/scaffold/append.mjs +41 -0
- package/src/scaffold/apply-append.mjs +71 -0
- package/src/scaffold/fix-input.mjs +119 -0
- package/src/scaffold/fix-paths.mjs +90 -0
- package/src/scaffold/intent.mjs +2 -0
- package/src/scaffold/open-append.mjs +44 -0
- package/src/scaffold/plan-append.mjs +37 -0
- package/src/scaffold/plan-intent-input.mjs +38 -0
- package/src/scaffold/prepare-append.mjs +46 -0
- package/src/scaffold/request.mjs +30 -0
- package/src/scaffold/scaffold.mjs +23 -10
- package/src/status/status.mjs +42 -10
- package/src/sweep/pool.mjs +33 -0
- package/src/sweep/sweep.mjs +9 -4
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,88 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.17.0] - 2026-10-01
|
|
11
|
+
|
|
12
|
+
Automated weekly release — everything merged since `v0.16.0`.
|
|
13
|
+
|
|
14
|
+
### Changed
|
|
15
|
+
|
|
16
|
+
- feat: integrate adoption guidance and machine-safe probing (#189)
|
|
17
|
+
- chore(deps-dev): bump vitest from 5.0.1 to 5.0.2 in the all-npm group (#187)
|
|
18
|
+
- chore(homebrew): sha256 for v0.16.0 (#186)
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
|
|
23
|
+
- Unannotated scaffold defaults use unfinished `TODO-CLAIM-N` identifiers rather than file/function-shaped claim names. Allocation avoids supplied IDs; explicit IDs/annotations and existing evidence remain unchanged, while sweep pooling preserves distinct candidates.
|
|
24
|
+
|
|
25
|
+
- Read-only `scaffold --from-document <local-text-path>` and `--from-fix <full-commit-ID>` inspection, backed by the closed `authoring-input` report contract. Validation retains historical partition counts and rejects unsafe/private paths, inconsistent identities and optimistic verification labels. Stdout-only output emits no document text or claims; input inspection requires independent intent and is not verification.
|
|
26
|
+
|
|
27
|
+
- Explicit `scaffold <source...> --into <existing-draft.json> --claim <ID>` appends mechanically proposed faults while preserving supplied intent and existing faults. `--json` previews without writes; ordinary scaffold remains compatible. Draft updates use bounded admission, owned locking and private recovery, and refuse stale/unsafe inputs or unconfirmed writes rather than claiming verified coverage.
|
|
28
|
+
|
|
29
|
+
- Sweep keeps equal-basename and repeated-annotation drafts distinct before selection, assigning collision-free pooled claim IDs without changing fault content, metadata or defenders.
|
|
30
|
+
|
|
31
|
+
- New mechanical scaffold drafts declare `inferred` origin and ask for intended observable behavior from a requirement, ADR, bug or incident. CLI/status/skill guidance explains the source handoff without authenticating references or promoting annotations/tests into independent intent. Existing supplied scaffold metadata is preserved.
|
|
32
|
+
|
|
33
|
+
- Explicit `probe --require-origin <kinds>` checks declared origins on complete confirmed current-project probes, rechecks native universe and file bindings, and reports failed/unavailable policy without suppressing survivors through baselines or severity floors. Kinds do not authenticate independence; custom/static universes are unavailable. Offline status audits current declarations without certifying freshness; brief retains labeled recorded-run policy before caps and filtering.
|
|
34
|
+
|
|
35
|
+
- Candidate evidence/status/brief schemas accept a closed optional declared-origin policy result, with unsuppressed fault identities, refusal reasons, consistent states and probe exit codes. This is an audit contract, not authenticated independence.
|
|
36
|
+
|
|
37
|
+
- Claim sources share a closed origin vocabulary across claims and evidence, adding `doc`, `bug`, `incident`, `review` and `inferred`. Status reports current declared origins; evidence and briefs summarize all recorded origins before filtering/caps, separating distinct claims from records and reporting conflicting declarations. Human output labels these as unauthenticated declarations. Labels do not change default gates; origin policy requires explicit selection.
|
|
38
|
+
|
|
39
|
+
- Claims/status missing-source-link advisories identify distinct claim IDs and suggest read-only annotation preview without changing verification state, next action or exit codes; annotation-sourced claims retain their existing drift error.
|
|
40
|
+
|
|
41
|
+
- Explicit `claims --annotate` read-only file-header preview and `--apply` authoring for JavaScript/TypeScript and Python, with claim selection, bounded all-target admission, owned locking, private recoverable originals and truthful partial-failure reporting. Authoring JSON uses a closed `annotations` contract; placement is not verification or authentication of intent.
|
|
42
|
+
|
|
43
|
+
### Fixed
|
|
44
|
+
|
|
45
|
+
- Clarify that cold probes and changed discovery/configuration can require full verification; unchanged target and defender files alone do not guarantee a cheap CI run.
|
|
46
|
+
|
|
47
|
+
- Status marks evidence stale when a recorded automatic-discovery dependency (including a negative candidate, barrel or resolver config) changes or becomes unavailable. Previously unchanged defender lists could hide this drift. Native test-universe policy freshness remains pending.
|
|
48
|
+
|
|
49
|
+
- Annotation scanning and placement ID admission avoid excessive regex backtracking on long alphanumeric tokens without a hyphen; status advice uses bounded, failure-aware scanning so unrelated source failures cannot prevent its verification decision.
|
|
50
|
+
|
|
51
|
+
- Claim statement, severity, source and producer edits now invalidate cached probe evidence and appear as stale in status. Old kills cannot silently carry an edited requirement or provenance declaration.
|
|
52
|
+
|
|
53
|
+
### Changed
|
|
54
|
+
|
|
55
|
+
- Reduce repeated self-probe setup with focused defenders for 40 existing fault checks, retaining the complete native integration regressions, all 327 fault recipes, three-run confirmation and existing cost budgets.
|
|
56
|
+
- Bound built-in runner concurrency to one worker by default, with `--workers` overrides, and cap outer regression workers at two (one on a two-core machine).
|
|
57
|
+
- Drain noisy runner stdout, bound diagnostics and report reads, and terminate owned runners before restoring mutations and scratch worktrees on cancellation.
|
|
58
|
+
- Run the project-pinned Playwright CLI directly instead of spawning npm for each confirmation, and reset Python interpreter caches at measurement boundaries.
|
|
59
|
+
- Record actual probe elapsed time and unique fresh runner invocations separately from historical attributed costs; preserve all confirmation and cost gates, and invalidate reuse when native worker policy changes.
|
|
60
|
+
|
|
61
|
+
## [0.16.0] - 2026-09-30
|
|
62
|
+
|
|
63
|
+
Automated weekly release — everything merged since `v0.15.1`.
|
|
64
|
+
|
|
65
|
+
### Changed
|
|
66
|
+
|
|
67
|
+
- feat: fault-specific defenders, provenance and cost reporting (#184)
|
|
68
|
+
- Add advisory AI update guidance and stabilize cleanup regression (#183)
|
|
69
|
+
- chore(homebrew): sha256 for v0.15.1 (#182)
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+
### Added
|
|
73
|
+
|
|
74
|
+
- Fault-specific `defendedBy` overrides, with explicit empty-list discovery,
|
|
75
|
+
evidence selection provenance, per-fault cost reporting and prior-kill
|
|
76
|
+
narrowing warnings (#90). Selection changes invalidate reuse and status;
|
|
77
|
+
admission and changed-file coverage use the actual selected defenders.
|
|
78
|
+
|
|
79
|
+
### Changed
|
|
80
|
+
|
|
81
|
+
- New AI integrations include permission-aware, once-per-session update advice
|
|
82
|
+
that identifies the actual CLI and requests approval before any upgrade.
|
|
83
|
+
CLI commands and session-start hooks remain offline. Existing customized
|
|
84
|
+
instructions remain untouched; safe manual refresh is documented.
|
|
85
|
+
|
|
86
|
+
### Fixed
|
|
87
|
+
|
|
88
|
+
- The daemon-cleanup regression test now requires explicit startup and
|
|
89
|
+
post-cleanup release signals instead of assuming a detached process starts
|
|
90
|
+
and writes within fixed sleeps. Timeout and cleanup semantics are unchanged.
|
|
91
|
+
|
|
10
92
|
## [0.15.1] - 2026-09-30
|
|
11
93
|
|
|
12
94
|
Automated weekly release — everything merged since `v0.15.0`.
|
package/README.md
CHANGED
|
@@ -127,9 +127,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
|
|
|
127
127
|
| npm | `npm i -D testguard-cli` then `npx testguard probe` |
|
|
128
128
|
| pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
|
|
129
129
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
130
|
-
| GitHub Action | `uses: raccioly/testguard@v0.
|
|
130
|
+
| GitHub Action | `uses: raccioly/testguard@v0.17.0` — see [`action.yml`](./action.yml) |
|
|
131
131
|
| pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-probe` |
|
|
132
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
132
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.17.0/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
|
|
133
133
|
|
|
134
134
|
Projects that set `min-release-age` in `.npmrc` cannot see a version published
|
|
135
135
|
less than that many days ago (`ENOVERSIONS`); install that one with
|
|
@@ -142,6 +142,7 @@ npx testguard-cli init # install the agent layer at the git
|
|
|
142
142
|
npx testguard-cli status --json # where the project is, and the ONE next action
|
|
143
143
|
npx testguard-cli claims # what does this project claim, and is each claim probeable?
|
|
144
144
|
npx testguard-cli claims --check-anchors # fail fast if an exact anchor moved or a replacement no longer parses
|
|
145
|
+
npx testguard-cli claims --annotate --claim AUTH-ADMIN --json # read-only file-header placement preview
|
|
145
146
|
npx testguard-cli scaffold src/auth.ts # propose faults for a file, as a draft to keep or drop
|
|
146
147
|
npx testguard-cli probe # try to falsify each claim; report what the tests missed
|
|
147
148
|
npx testguard-cli admit test/auth.test.ts --claim AUTH-ADMIN # does this test satisfy the two-gate rule?
|
|
@@ -158,6 +159,54 @@ unclaimed changes or invalid fault anchors, `2` a precondition failed and
|
|
|
158
159
|
nothing was probed, `3` usage.
|
|
159
160
|
Every command accepts `--json`.
|
|
160
161
|
|
|
162
|
+
### Intent-first drafts
|
|
163
|
+
|
|
164
|
+
Scaffold drafts propose mechanical faults, not intended behavior. New claims
|
|
165
|
+
declare `inferred` origin even with an annotation. Supply the intended observable
|
|
166
|
+
behavior from a requirement, ADR, bug or incident independently of the code:
|
|
167
|
+
inputs, expected outcome and forbidden outcome. Record the supplied source
|
|
168
|
+
kind/ref; when unavailable keep inferred. References are not authenticated or
|
|
169
|
+
fetched, and passing tests do not establish independent intent. Review proposals
|
|
170
|
+
against the supplied intent before adopting and probing; existing `--claim`
|
|
171
|
+
metadata is preserved.
|
|
172
|
+
|
|
173
|
+
### Optional annotation placement
|
|
174
|
+
|
|
175
|
+
Ordinary `claims` output and status notes identify missing scanned source links
|
|
176
|
+
as advisories. Counts are distinct claim IDs, not faults. They do not change
|
|
177
|
+
verification state, its next action or exit codes; annotation-sourced missing
|
|
178
|
+
IDs still produce the existing claims drift error. Scanned text is a lexical
|
|
179
|
+
link, not authenticated intent or source ownership.
|
|
180
|
+
Status advice uses a bounded scan (10,000 entries, 1,000 files, 64 directory
|
|
181
|
+
levels, 256 KiB per file, 2 MiB total); incomplete scans report unavailable
|
|
182
|
+
advice without concluding an ID is absent. Keep the original `--claims`
|
|
183
|
+
argument for preview when inspecting a custom claims selection.
|
|
184
|
+
|
|
185
|
+
`testguard claims --annotate` previews JavaScript/TypeScript or Python
|
|
186
|
+
file-header links for declared claims; repeat `--claim` or use comma-separated
|
|
187
|
+
IDs to select claims. Add `--apply` only after reviewing the selection to
|
|
188
|
+
explicitly write annotations. Each invocation recomputes its plan: an earlier
|
|
189
|
+
preview is not a saved approval token. No `--out` or unrelated inspection
|
|
190
|
+
switches are accepted with authoring. JSON conforms to the `annotations`
|
|
191
|
+
authoring schema; it contains no source/proposed contents or probe verdicts.
|
|
192
|
+
|
|
193
|
+
This is file-level placement, not symbol ownership or proof of correctness.
|
|
194
|
+
Missing annotations do not disable exact fault-anchor validation, and placement
|
|
195
|
+
does not change claim origins, evidence or baselines. Unsupported, ambiguous or
|
|
196
|
+
unsafe targets block the complete selection. Original/proposed files are capped
|
|
197
|
+
at 2 MiB and the aggregate at 16 MiB. Preserve BOM, shebang, Python encoding
|
|
198
|
+
cookie, newline style and permissions. A preview exits 0 when applicable, 2 when
|
|
199
|
+
refused; apply exits 0 only after verified completion, 2 for refusal/partial
|
|
200
|
+
failure; malformed options exit 3.
|
|
201
|
+
|
|
202
|
+
Apply uses an exclusive owned lock and retains fsynced originals in an owner-only
|
|
203
|
+
recovery directory outside the checkout (the result reports its location).
|
|
204
|
+
External claims files can be previewed but cannot authorize writes. It never
|
|
205
|
+
reclaims an existing lock or automatically rolls back over editor changes.
|
|
206
|
+
This is recoverable, not a cross-file atomic transaction: interruption or
|
|
207
|
+
arbitrary concurrent edits can leave partial content. Compare retained originals
|
|
208
|
+
manually before recovery; inspect ownership before removing a leftover lock.
|
|
209
|
+
|
|
161
210
|
## How it works
|
|
162
211
|
|
|
163
212
|
```bash
|
|
@@ -221,6 +270,14 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
|
|
|
221
270
|
the evidence records them as `repo.ignoredDirty`, so the run says what it
|
|
222
271
|
did not look at. Every summary names the commit probed.
|
|
223
272
|
|
|
273
|
+
A fault may declare its own `defendedBy` to override the claim's set.
|
|
274
|
+
Omit the fault field to inherit; set it to `[]` to request discovery rather
|
|
275
|
+
than inheritance. Evidence names the selection origin and selected files.
|
|
276
|
+
Changing either invalidates reuse and makes status stale. A narrowed fault
|
|
277
|
+
set after a prior kill is warned about by `claims`; re-probe before trusting
|
|
278
|
+
it. For example, a unit-level fault can select `["test/unit.test.mjs"]`
|
|
279
|
+
while an integration fault inherits the claim's broader set.
|
|
280
|
+
|
|
224
281
|
A claim with no `defendedBy` has its defenders **discovered**: the test
|
|
225
282
|
files that import the fault's target, by relative path or resolved alias
|
|
226
283
|
(tsconfig `paths`, through `extends` and `references`, vite/vitest
|
|
@@ -300,9 +357,11 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
|
|
|
300
357
|
`--runner-cmd "<cmd> {files} … {out}"`; if the scratch worktree cannot
|
|
301
358
|
see your `node_modules`, pass `--node-modules <dir>`.
|
|
302
359
|
3. **Baseline** freezes every non-passing fingerprint. Later probes suppress
|
|
303
|
-
what was already known and exit non-zero only on what is new.
|
|
304
|
-
|
|
305
|
-
|
|
360
|
+
what was already known and exit non-zero only on what is new. Prior verdicts
|
|
361
|
+
can be reused when their recorded inputs and runner policy still match.
|
|
362
|
+
Cold runs, changed discovery/configuration or `--no-reuse` can require a
|
|
363
|
+
full probe; unchanged target and defender files alone do not guarantee reuse.
|
|
364
|
+
A baseline frozen from `--include-dirty`
|
|
306
365
|
evidence records the snapshot and points at the *parent* of the commit
|
|
307
366
|
that will carry your tests; after you commit, a clean `probe` plus
|
|
308
367
|
`baseline --restamp` moves it to that commit — only when the fingerprints
|
|
@@ -328,6 +387,34 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
|
|
|
328
387
|
offline. The brief's first line says which install answered
|
|
329
388
|
(`local install` or `global`), so a stale one is visible.
|
|
330
389
|
|
|
390
|
+
### Advisory updates for AI sessions
|
|
391
|
+
|
|
392
|
+
New integrations created by `testguard init` tell the AI to identify the actual
|
|
393
|
+
CLI (project-local before global) and, once per session when network policy
|
|
394
|
+
permits, read the registry's stable version:
|
|
395
|
+
|
|
396
|
+
```bash
|
|
397
|
+
npm view testguard-cli dist-tags.latest --json --fetch-retries=0 --fetch-timeout=5000
|
|
398
|
+
```
|
|
399
|
+
|
|
400
|
+
This is an optional check performed by the AI, not by TestGuard's CLI or hook.
|
|
401
|
+
A newer stable version produces a suggestion and a request for approval, never
|
|
402
|
+
an automatic install. Project pins and the original package manager still
|
|
403
|
+
apply; prereleases and development checkouts are not replaced. Offline or
|
|
404
|
+
failed checks do not change verdicts, evidence or exit codes, and do not prove
|
|
405
|
+
that an installation is current. AI instructions are guidance, not a guarantee
|
|
406
|
+
that every harness performs the check.
|
|
407
|
+
|
|
408
|
+
Upgrading the package does **not** refresh an existing copied skill or an
|
|
409
|
+
existing managed section already listing the project. Compare the upgraded
|
|
410
|
+
package's `src/init/templates/SKILL.md` with
|
|
411
|
+
`.claude/skills/testguard/SKILL.md` and merge approved changes, preserving local
|
|
412
|
+
customizations. For an existing `AGENTS.md` section, add the same advisory
|
|
413
|
+
policy after reviewing it. `testguard init --force` replaces the whole skill
|
|
414
|
+
but does not refresh an already-listed managed section; use it only with
|
|
415
|
+
explicit overwrite approval and review the diff. No new automatic refresh
|
|
416
|
+
mechanism is introduced.
|
|
417
|
+
|
|
331
418
|
### Would this suite have caught the bugs that already escaped?
|
|
332
419
|
|
|
333
420
|
An injected fault is a fault somebody thought of. A bug that actually shipped
|
|
@@ -458,6 +545,11 @@ proposed 510 faults, probed 10 (cap 10), deferred 350.
|
|
|
458
545
|
|
|
459
546
|
Three rules make it safe to run on a repository that has never seen this tool:
|
|
460
547
|
|
|
548
|
+
Pooled draft IDs are unique even when source files share basenames/functions or
|
|
549
|
+
annotation IDs. Sweep reserves original IDs and suffixes collisions before
|
|
550
|
+
selection, keeping every fault, its metadata and defenders attached to its own
|
|
551
|
+
draft. These are proposal identities, not authenticated claims.
|
|
552
|
+
|
|
461
553
|
- **It never writes `testguard.claims.json`**, and never will. A claim is a
|
|
462
554
|
sentence someone is willing to stand behind; a sentence nobody wrote is not
|
|
463
555
|
one. Its drafts land beside the evidence to keep or drop.
|
|
@@ -677,12 +769,12 @@ detected base that does not resolve is a warning for `status` and `brief`
|
|
|
677
769
|
# GitHub Actions
|
|
678
770
|
- uses: actions/checkout@v4
|
|
679
771
|
with: { fetch-depth: 0 }
|
|
680
|
-
- uses: raccioly/testguard@v0.
|
|
772
|
+
- uses: raccioly/testguard@v0.17.0
|
|
681
773
|
with: { command: gate }
|
|
682
774
|
|
|
683
775
|
# GitLab CI — the component-shaped template: gate + probe, brief as an artifact and, opted in, as a merge-request note
|
|
684
776
|
include:
|
|
685
|
-
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
777
|
+
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.17.0/packaging/gitlab/testguard.gitlab-ci.yml'
|
|
686
778
|
inputs: { dir: backend, post_note: true } # post_note needs TESTGUARD_GITLAB_TOKEN (api scope); one note, updated in place
|
|
687
779
|
```
|
|
688
780
|
|
|
@@ -932,16 +1024,20 @@ said what they want.
|
|
|
932
1024
|
|
|
933
1025
|
### What the probe costs, and why a gate gets slow
|
|
934
1026
|
|
|
935
|
-
A probe
|
|
936
|
-
|
|
1027
|
+
A fresh probe grows with the selected faults, confirmation count and the cost
|
|
1028
|
+
of their defenders. Shared baselines and valid verdict reuse avoid some repeated
|
|
1029
|
+
work. Roughly, the fresh work is
|
|
937
1030
|
|
|
938
1031
|
```
|
|
939
|
-
|
|
1032
|
+
fresh baseline runs × defender-set cost
|
|
1033
|
+
+ Σ (fault confirmations × that fault's defender-set cost)
|
|
1034
|
+
+ survivor diagnostics and setup/discovery work
|
|
940
1035
|
```
|
|
941
1036
|
|
|
942
|
-
|
|
943
|
-
|
|
944
|
-
|
|
1037
|
+
One slow acceptance test selected by several faults can therefore be executed
|
|
1038
|
+
many times. The symptom can be a gate that takes twenty minutes; the selected
|
|
1039
|
+
defender sets determine how much each fault costs. Historical attributed totals
|
|
1040
|
+
are reported separately from measured fresh elapsed time and runner invocations.
|
|
945
1041
|
|
|
946
1042
|
```bash
|
|
947
1043
|
npx testguard-cli claims . --cost # read back from the last probe's evidence
|
|
@@ -955,19 +1051,23 @@ most expensive claims
|
|
|
955
1051
|
4m 58s TG-FAULT-EDIT-VISIBLE 1 fault · 6 runs
|
|
956
1052
|
|
|
957
1053
|
defender files, by the time of the runs that included them
|
|
958
|
-
(a
|
|
1054
|
+
(a fault runs its selected defender set at once, so these overlap and do not sum to the total)
|
|
959
1055
|
24m 3s test/probe.fixture.test.mjs 5 claims
|
|
960
1056
|
|
|
961
|
-
shared defenders — each
|
|
1057
|
+
shared defenders — each fault selects the file; durations include recorded baseline sharing
|
|
962
1058
|
24m 3s test/probe.fixture.test.mjs
|
|
963
|
-
named by TG-ESCALATION-N-RUNS, TG-FAULT-EDIT-VISIBLE, …
|
|
1059
|
+
named by TG-ESCALATION-N-RUNS/F1, TG-FAULT-EDIT-VISIBLE/F1, …
|
|
964
1060
|
```
|
|
965
1061
|
|
|
966
1062
|
Every number is read back out of the `durationMs` the probe already records.
|
|
967
1063
|
`--cost` re-measures nothing, spawns nothing, and costs the price of reading
|
|
968
1064
|
one JSON file.
|
|
969
1065
|
|
|
970
|
-
|
|
1066
|
+
Cost output also lists individual faults, their selected defenders and selection
|
|
1067
|
+
origin. Sharing is counted across distinct fault identities, including siblings
|
|
1068
|
+
within one claim; repeated confirmation runs alone are not sharing.
|
|
1069
|
+
|
|
1070
|
+
**The per-file figures overlap.** A run executes a fault's whole defender set
|
|
971
1071
|
at once, so the runner never says how much of it belonged to which file. A
|
|
972
1072
|
file's `ms` is the time of every run that *included* it: an upper bound on
|
|
973
1073
|
what removing it could save, which is the decision you are making. Only
|
|
@@ -991,8 +1091,37 @@ so `scaffold` proposes them for you:
|
|
|
991
1091
|
```bash
|
|
992
1092
|
npx testguard-cli scaffold src/auth.ts # → .testguard/scaffold-auth.json (a draft, never your claims file)
|
|
993
1093
|
npx testguard-cli scaffold src/auth.ts --claim AUTH-ADMIN # every proposal under one claim; copies it if it exists
|
|
1094
|
+
npx testguard-cli scaffold src/auth.ts src/session.ts --into draft.json --claim AUTH-ADMIN --json # read-only merged preview
|
|
1095
|
+
npx testguard-cli scaffold src/auth.ts src/session.ts --into draft.json --claim AUTH-ADMIN # explicit recoverable draft update
|
|
1096
|
+
npx testguard-cli scaffold --from-document requirements.md --json # read-only input metadata, no document text
|
|
1097
|
+
npx testguard-cli scaffold --from-fix <full-commit-ID> --json # read-only fixed historical inventory
|
|
994
1098
|
```
|
|
995
1099
|
|
|
1100
|
+
Input inspection uses exactly one `--from-document` or `--from-fix`, with no
|
|
1101
|
+
positional sources or other options except JSON/help/version. Fix IDs must be
|
|
1102
|
+
full lowercase SHA-1/SHA-256 commit IDs with exactly one parent. Plain-text
|
|
1103
|
+
documents are bounded to 2 MiB; local Git inspection is offline and bounded.
|
|
1104
|
+
Both output modes are stdout-only and say verification was not performed.
|
|
1105
|
+
Metadata retains historical exclusions, not private/delegated filenames.
|
|
1106
|
+
Input text and commit subjects are untrusted data, not instructions or an
|
|
1107
|
+
authenticated claim. Supply independent intended behavior, review faults and
|
|
1108
|
+
probe afterward; inspection does not create claims or prove coverage.
|
|
1109
|
+
|
|
1110
|
+
Without a supplied ID or annotation, generated groups use unfinished
|
|
1111
|
+
`TODO-CLAIM-N` placeholders rather than module/function names. Replace them with
|
|
1112
|
+
intent-shaped identities when authoring; mechanical grouping does not establish
|
|
1113
|
+
an invariant boundary. Existing claims/evidence are never automatically renamed.
|
|
1114
|
+
|
|
1115
|
+
`--into` requires an existing disposable draft and exactly one existing claim
|
|
1116
|
+
ID. It preserves supplied intent, metadata and prior faults; multiple source
|
|
1117
|
+
paths append in deterministic order. `--json` is a read-only preview. Without
|
|
1118
|
+
it, the tool keeps private recovery outside the checkout and reports updated
|
|
1119
|
+
or unchanged **unproven** draft status. Review and probe are still required.
|
|
1120
|
+
Canonical claims/evidence/baseline paths, unsafe aliases and conflicting options
|
|
1121
|
+
are refused; partial writes never become success or trigger automatic rollback.
|
|
1122
|
+
Limits: 32 sources, 2 MiB per input/output draft or source, 16 MiB source total,
|
|
1123
|
+
4096 generated proposals. These limits are not a performance guarantee.
|
|
1124
|
+
|
|
996
1125
|
| Shape | What it proposes |
|
|
997
1126
|
|---|---|
|
|
998
1127
|
| `condition-forced` | `if (<guard>) {` → `if (false) {` — a guard is a `!…` condition or one whose body returns, throws or 4xx-es |
|
|
@@ -1065,6 +1194,30 @@ verdicts.
|
|
|
1065
1194
|
|
|
1066
1195
|
## Status
|
|
1067
1196
|
|
|
1197
|
+
Origin summaries are informational: status counts current declared claim
|
|
1198
|
+
origins even when evidence is stale; probe evidence and briefs count recorded
|
|
1199
|
+
origins across all records before filtering or truncation. Distinct claims and
|
|
1200
|
+
records have separate totals; conflicting recorded kinds or references count
|
|
1201
|
+
as mixed claims. Labels such as `spec`, `bug` or `incident` do not authenticate
|
|
1202
|
+
independence and never waive a failed measurement or change default gates.
|
|
1203
|
+
References remain opaque and are not echoed in these summaries. Older strict
|
|
1204
|
+
readers reject these additive fields; use the matching candidate reader for
|
|
1205
|
+
unreleased formats. The candidate `probe --require-origin spec,adr,bug,incident`
|
|
1206
|
+
option selects declared kinds for a complete confirmed current-project probe.
|
|
1207
|
+
Repeat the option or use comma-separated kinds; partial, provisional, explicit
|
|
1208
|
+
historical and ignore-dirty combinations are refused. Failed policy exits 1
|
|
1209
|
+
even when baseline debt or a severity floor hides ordinary findings; unavailable
|
|
1210
|
+
current bindings exit 2. The command rechecks native runner/configuration
|
|
1211
|
+
membership, current claims, targets, defenders and discovery dependencies before
|
|
1212
|
+
publishing. Custom/static runner universes cannot certify this boundary. This is
|
|
1213
|
+
a declaration check, not authenticated independence or an atomic disk snapshot.
|
|
1214
|
+
Offline status recomputes a current-declaration audit using the recorded eligible
|
|
1215
|
+
selection, always marking native freshness unavailable without launching a
|
|
1216
|
+
runner. It does not implicitly activate a new gate or replace status's next
|
|
1217
|
+
action. Briefs preserve the recorded-run policy before caps and baseline
|
|
1218
|
+
filtering, explicitly not current freshness. A zero-item cap never turns
|
|
1219
|
+
survivors into an all-defended message.
|
|
1220
|
+
|
|
1068
1221
|
**v0.5.** Eleven commands (`status`, `init`, `claims`, `probe`, `admit`,
|
|
1069
1222
|
`replay`, `baseline`, `brief`, `gate`, `scaffold`, `sweep`, `mcp`), vitest, jest,
|
|
1070
1223
|
Playwright and Python (pytest / stdlib unittest) runners, hand-authored faults
|
|
@@ -1080,6 +1233,32 @@ the transfer of a calibration between repositories — `replay` measures it now;
|
|
|
1080
1233
|
whether it carries to a repository with no history is unproven. Each is
|
|
1081
1234
|
designed for; none is claimed.
|
|
1082
1235
|
|
|
1236
|
+
### Keeping developer machines responsive
|
|
1237
|
+
|
|
1238
|
+
Built-in probe, sweep, replay and admission runners default to one worker. Use
|
|
1239
|
+
`--workers 2` for a larger Vitest/Jest/Playwright worker ceiling; `--serial` always
|
|
1240
|
+
uses one. Python remains serial. Faults and mixed-runner groups run sequentially.
|
|
1241
|
+
Custom `--runner-cmd` commands, browser subprocesses, native library threads and
|
|
1242
|
+
test-created processes are not an OS resource sandbox. The regression suite also
|
|
1243
|
+
caps outer workers at two, or one when fewer than three CPUs are available
|
|
1244
|
+
in an interactive run; dedicated CI uses at most two because its acceptance tests start nested runners.
|
|
1245
|
+
|
|
1246
|
+
`--budget` bounds each runner invocation; `--command-budget` bounds a complete
|
|
1247
|
+
command cooperatively and refuses partial result publication. Cancellation kills
|
|
1248
|
+
owned runners before restoring mutations and removing scratch worktrees. Already
|
|
1249
|
+
reparented daemons remain outside the portable cleanup guarantee. No runner output
|
|
1250
|
+
limit can count as an assertion failure or detection.
|
|
1251
|
+
|
|
1252
|
+
New probe evidence includes actual elapsed time, unique fresh runner time and
|
|
1253
|
+
setup/discovery/other overhead. `probe --cost` and `claims --cost` display this
|
|
1254
|
+
separately from historical attributed record costs. Existing cost ceilings are
|
|
1255
|
+
unchanged; old evidence does not supply a measured breakdown.
|
|
1256
|
+
|
|
1257
|
+
Playwright confirmations run the same project-resolved CLI used for discovery,
|
|
1258
|
+
avoiding an npm subprocess per run. Python projects whose configuration requires
|
|
1259
|
+
pytest-xdist must support serial execution; an incompatible configuration remains
|
|
1260
|
+
unproven rather than bypassing the worker policy.
|
|
1261
|
+
|
|
1083
1262
|
## Licence
|
|
1084
1263
|
|
|
1085
1264
|
MIT.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "testguard-cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.17.0",
|
|
4
4
|
"description": "Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|