testguard-cli 0.6.0 → 0.7.0

package/README.md

@@ -1,5 +1,8 @@
 # TestGuard
 
+<!-- docguard:quality negation-load off — this README explains a tool defined by what must not happen; the negations are the product. -->
+<!-- docguard:quality passive-voice off — verdicts and artifacts are the subjects throughout ("a fault is applied", "evidence is written"); naming an actor would misdescribe a tool nobody operates interactively. -->
+
 [![CI](https://github.com/raccioly/testguard/actions/workflows/ci.yml/badge.svg)](https://github.com/raccioly/testguard/actions/workflows/ci.yml)
 [![npm](https://img.shields.io/npm/v/testguard-cli.svg)](https://www.npmjs.com/package/testguard-cli)
 [![PyPI](https://img.shields.io/pypi/v/testguard-cli.svg)](https://pypi.org/project/testguard-cli/)
@@ -7,9 +10,8 @@
 [![license](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
 [![deps](https://img.shields.io/badge/runtime%20deps-1%20pinned-brightgreen.svg)](./package.json)
 
-> Proves that a test suite actually defends the claims a project makes — by
-> injecting the faults those claims say cannot happen, and reporting every
-> fault the tests fail to detect.
+> Breaks your code on purpose and reports every promise your tests did not
+> notice breaking.
 
 **Not a test generator. A claim verifier.** Test generation is what happens
 after a claim turns out to be unfalsifiable.
@@ -22,6 +24,65 @@ surface ↔ code). All three run one loop:
 > declare what must be true → try mechanically to falsify it → freeze a
 > baseline → gate only the delta → brief the agent before it writes code.
 
+## In one minute
+
+**Coverage tells you a line ran. It never tells you anyone checked the
+result.** So a codebase can be fully covered and completely undefended, and
+nothing in CI will say a word.
+
+Here is a real test, from this repository's own fixture:
+
+```js
+expect(store.writeAudit).toHaveBeenCalledWith(
+  expect.objectContaining({ action: 'MASK', scope: 'g1', ruleCount: 1 }),
+);  // `content` is never named — so nothing checks it
+```
+
+`objectContaining` ignores keys it does not list. Swap the redacted text for
+the **raw secret** and this test still passes. Coverage of that line: 100%.
+The audit log now leaks the very thing it exists to protect.
+
+**The method: break the code deliberately, then watch what the tests do.**
+
+- **killed** — you broke it, a test failed. Good. That behaviour is genuinely
+  defended.
+- **SURVIVED** — you broke it, everything stayed green. A blind spot.
+
+The word trips everyone once: it is the *fault* that survived, not the test.
+**SURVIVED is the bad news.** A healthy report is full of `killed`.
+
+But *"did the tests fail?"* is a sloppy question — a red suite is not proof of
+detection. So there are seven verdicts, and each means something different:
+
+| Verdict | What it means |
+|---|---|
+| `killed` | A test body ran and rejected the behaviour. The only good outcome. |
+| `SURVIVED` | Everything passed. A real blind spot in the tests. |
+| `NOCOVER` | No test even looks at this code. Not "weak tests" — *no* tests. |
+| `UNVERIFIABLE` | The fault could not be applied: its anchor moved, or matches twice. Nothing was learned. |
+| `FAULT-INVALID` | The break itself was broken — it did not compile. Our fault, not yours. |
+| `TIMEOUT` | The suite hung. A hang is not a detection. |
+| `FLAKY-DEFENDER` | The tests are not reliable enough on untouched code to be asked the question. |
+
+Every ambiguity rounds toward *unproven*: three runs rather than one, a green
+baseline required before any fault is injected, and a timeout, a load failure
+or a mixed result is never a kill. The reason is the constraint the whole
+design follows from — **a tool that reports everything as caught is worse than
+no tool at all, because nobody questions good news.**
+
+**What is not new:** breaking code to test your tests is *mutation testing*,
+and it dates to the 1970s. Stryker, PIT, mutmut and Cosmic Ray all do it.
+
+**What is different is the question.** Classic mutation testing mutates
+everything mechanically and hands you *"mutation score: 73%"* — a number that
+is not actionable, not auditable, and cannot tell you which promise is at
+risk. TestGuard binds every fault to a **stated claim**, so the output is not
+a score but a finding: *"Your project says a missing scope fails closed.
+Nothing checks that."* That is the difference between a metric and an audit.
+
+📄 **[Read the three-page technical brief (PDF)](docs/testguard-explained.pdf)**
+— the idea on page one, mechanics and prior art after it.
+
 ## Why
 
 Coverage cannot tell a test that pins *correct* behaviour from one that pins
@@ -65,14 +126,34 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
 | npm | `npm i -D testguard-cli` then `npx testguard probe` |
 | pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
 | Homebrew | `brew tap raccioly/tap && brew install testguard` |
-| GitHub Action | `uses: raccioly/testguard@v0.6.0` — see [`action.yml`](./action.yml) |
+| GitHub Action | `uses: raccioly/testguard@v0.7.0` — see [`action.yml`](./action.yml) |
 | pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-probe` |
-| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.5.0/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
+| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.7.0/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
 
 Projects that set `min-release-age` in `.npmrc` cannot see a version published
 less than that many days ago (`ENOVERSIONS`); install that one with
 `npm i -D testguard-cli --min-release-age=0`.
 
+## Usage
+
+```bash
+npx testguard-cli init                      # install the agent layer at the git root
+npx testguard-cli status --json             # where the project is, and the ONE next action
+npx testguard-cli claims                    # what does this project claim, and is each claim probeable?
+npx testguard-cli scaffold src/auth.ts      # propose faults for a file, as a draft to keep or drop
+npx testguard-cli probe                     # try to falsify each claim; report what the tests missed
+npx testguard-cli admit test/auth.test.ts --claim AUTH-ADMIN   # does this test satisfy the two-gate rule?
+npx testguard-cli baseline                  # freeze today's unproven findings; only new ones gate
+npx testguard-cli gate --changed origin/main # fail when a changed source file carries no claim
+npx testguard-cli brief --text              # tell the agent where the suite is blind, before it writes
+npx testguard-cli replay --since v1.0..HEAD # would this suite have caught the bugs that escaped?
+npx testguard-cli mcp                       # serve the read-only loop over MCP, on stdio
+```
+
+Exit codes are the contract: `0` nothing new to prove, `1` unproven claims or
+unclaimed changes, `2` a precondition failed and nothing was probed, `3` usage.
+Every command accepts `--json`.
+
 ## How it works
 
 ```bash
@@ -146,14 +227,39 @@ npx testguard-cli admit test/x.test.ts --claim X   # is this test green on HEAD
    there is, and the exact signature of one escaped bug in the field
    reports. Silence it, visibly, with `// unasserted: <why>` above the mock.
 
-   Runners: **vitest** and **jest** (`--runner auto` picks the first that
-   resolves; both read the same jest-compatible JSON report). A runner is
-   resolved from the **project's own package** first — its pinned version,
-   its own bin script — and only then from an executable on PATH, which the
-   evidence records as `runner.source: "path"`; `npx` is never asked,
-   because its cache answers for packages a project does not have. Anything
-   else goes through `--runner-cmd`. Each runner is proven against its own
-   copy of the known-answer fixture.
+   Runners: **vitest**, **jest**, **Playwright** and **Python** (`--runner
+   auto` picks the first of vitest, jest, python that resolves). A JavaScript
+   runner is resolved from the **project's own package** first — its pinned
+   version, its own bin script — and only then from an executable on PATH,
+   which the evidence records as `runner.source: "path"`; `npx` is never
+   asked, because its cache answers for packages a project does not have.
+   Anything else goes through `--runner-cmd`. Each runner is proven against
+   its own copy of the known-answer fixture.
+
+   **Python** runs under `pytest` when the project's interpreter can import
+   it and stdlib `unittest` when it cannot, and the evidence records the
+   engine that actually ran — `--runner pytest` or `--runner unittest` pins
+   the choice and fails rather than falling back. Nothing is installed into
+   the project: the reporters ship inside TestGuard and reach the interpreter
+   through `PYTHONPATH`, so a codebase whose test dependencies are the
+   standard library stays that way. The interpreter is the project's
+   `.venv`/`venv`, the active `VIRTUAL_ENV`, or `--python <path>`. A `.py`
+   defender runs under Python whatever the project runner is, so one claim
+   can be defended by a vitest test and a pytest test at once.
+
+   Two things Python forces that JavaScript does not. **`patch("pkg.mod.fn")`
+   is not `vi.mock`**: it replaces one attribute, so the file is still a
+   defender of every other fault in that module — it keeps its place and
+   carries a `target-attribute-patched` signal naming the attributes. And
+   **the fault must be the code that ran**: a strict editable install puts an
+   import hook ahead of `sys.path`, so Python can load the original file while
+   TestGuard faults the copy — a green baseline, every claim `SURVIVED`, and a
+   report that reads as a devastating finding while being entirely false. With
+   the fault applied, TestGuard asks which file was actually imported and
+   refuses the run when it is outside the tree being probed. When no defender
+   imported the subject at all, the run continues and the record says so
+   (`detail.targetNotImported`), because that is a fact about the defenders'
+   reach and not about their assertions.
 
    The two-gate rule, as one verb: `testguard admit <test-file> --claim <ID>`
    runs the claim's faults against your uncommitted test and answers
@@ -318,12 +424,12 @@ detected base that does not resolve is a warning for `status` and `brief`
 # GitHub Actions
 - uses: actions/checkout@v4
   with: { fetch-depth: 0 }
-- uses: raccioly/testguard@v0.6.0
+- uses: raccioly/testguard@v0.7.0
   with: { command: gate }
 
 # GitLab CI — the component-shaped template: gate + probe, brief as an artifact and, opted in, as a merge-request note
 include:
-  - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.5.0/packaging/gitlab/testguard.gitlab-ci.yml'
+  - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.7.0/packaging/gitlab/testguard.gitlab-ci.yml'
     inputs: { dir: backend, post_note: true }   # post_note needs TESTGUARD_GITLAB_TOKEN (api scope); one note, updated in place
 ```
 
@@ -384,6 +490,45 @@ is the same one as everywhere else: a `claim` entry in
 Removal is allowed — claims can be wrong, superseded or split — and it is
 never silent.
 
+### Probing a Python project
+
+Nothing is installed into your project. TestGuard carries its own reporters and
+puts them on the interpreter's `PYTHONPATH`, so a codebase whose test
+dependencies are the standard library keeps none.
+
+```bash
+npx testguard-cli probe .                        # auto: pytest if importable, else stdlib unittest
+npx testguard-cli probe . --runner unittest      # pin the stdlib engine
+npx testguard-cli probe . --python .venv/bin/python
+```
+
+The evidence records the engine that actually ran, and `--runner pytest` fails
+rather than quietly using `unittest` — which engine ran changes what the
+evidence means.
+
+Three things behave differently from JavaScript, and each one is a place a
+naive port would have produced a confident wrong answer:
+
+- **Defenders are matched by module name**, the way Python itself matches, so
+  a test that reaches your package through `sys.path`, a `conftest.py` or an
+  installed distribution is found either way.
+- **`patch("pkg.mod.fn")` is not `vi.mock`.** It replaces one attribute, so the
+  file still detects a fault anywhere else in that module: it stays a defender
+  and carries a `target-attribute-patched` signal naming the attributes. Only a
+  patch of the module *itself* removes a defender.
+- **The fault has to be the code that ran.** A strict editable install puts an
+  import hook ahead of `sys.path`, so Python can load your original file while
+  TestGuard faults its copy — green baseline, every claim `SURVIVED`, and a
+  report that reads as a catastrophe while being entirely false. With the fault
+  applied TestGuard asks which file was really imported and **refuses the run**
+  if it came from outside the tree being probed. If no defender imported the
+  subject at all, the run continues and the record says so
+  (`detail.targetNotImported`): that is a fact about the tests' reach, not
+  about their assertions, and the two must not read alike.
+
+`testguard scaffold path/to/module.py` proposes Python faults the same way it
+does for JavaScript.
+
 ### Run it from any harness
 
 The operating loop above lives in a Claude Code skill and a session-start
@@ -452,6 +597,92 @@ things make that safe:
   previous verdict, and makes reviewing that edit the next action. Editing a
   claim is allowed — claims can be wrong — but it is never invisible.
 
+### Watching a probe that is still running
+
+A probe runs the defenders N times clean and N times per fault, which takes
+minutes. It used to print its stage line only when `stderr` was a terminal —
+so CI, a redirected log and an agent harness saw **nothing at all** for the
+whole run, which is indistinguishable from a hang. A gate you cannot tell
+from a hang is a gate people start killing.
+
+The rewriting is a rendering choice, not a reason to withhold the
+information:
+
+| `--progress` | What you get |
+|---|---|
+| `auto` (default) | `tty` at a terminal, `plain` everywhere else |
+| `tty` | one line, rewritten in place — for a human watching |
+| `plain` | one append-only line per stage — a log file, CI, an agent |
+| `ndjson` | one JSON object per line, stages **and** verdicts as they land |
+| `none` | silence |
+
+```bash
+npx testguard-cli probe . > probe.log 2>&1          # now reports; used to be silent
+npx testguard-cli probe . --progress ndjson 2>events.ndjson
+```
+
+```
+{"event":"stage","claim":"TG-COST-NAMES-SHARED-DEFENDERS","fault":"F1","stage":"baseline","run":1,"of":3,"at":"…"}
+{"event":"verdict","claim":"TG-COST-NAMES-SHARED-DEFENDERS","fault":"F1","verdict":"killed","severity":"medium","file":"src/probe/cost.mjs","at":"…"}
+```
+
+**Progress always goes to stderr**, so `--json` leaves stdout as one document
+and nothing else. `--quiet` and `--json` imply `none`, and an explicit
+`--progress` overrides both: an operator who asks for a stream of events has
+said what they want.
+
+### What the probe costs, and why a gate gets slow
+
+A probe's wall clock is not a function of how many claims you have. Per claim
+it is roughly
+
+```
+(baseline runs + faults × --confirm) × the cost of that claim's defender SET
+```
+
+so one slow acceptance test, named as a defender by five claims, is paid for
+thirty times. The symptom is a gate that takes twenty minutes; the cause is a
+single line in the claims file, and nothing in the output used to say which.
+
+```bash
+npx testguard-cli claims . --cost     # read back from the last probe's evidence
+npx testguard-cli probe . --cost      # what the run you just made spent
+```
+
+```
+63 fault records cost 24m across 378 defender runs.
+
+most expensive claims
+    4m 58s  TG-FAULT-EDIT-VISIBLE              1 fault  · 6 runs
+
+defender files, by the time of the runs that included them
+  (a claim runs its whole defender set at once, so these overlap and do not sum to the total)
+    24m 3s  test/probe.fixture.test.mjs                  5 claims
+
+shared defenders — each claim pays the file's full cost again
+    24m 3s  test/probe.fixture.test.mjs
+            named by TG-ESCALATION-N-RUNS, TG-FAULT-EDIT-VISIBLE, …
+```
+
+Every number is read back out of the `durationMs` the probe already records.
+`--cost` re-measures nothing, spawns nothing, and costs the price of reading
+one JSON file.
+
+**The per-file figures overlap.** A run executes a claim's whole defender set
+at once, so the runner never says how much of it belonged to which file. A
+file's `ms` is the time of every run that *included* it: an upper bound on
+what removing it could save, which is the decision you are making. Only
+`totalMs` is additive.
+
+**What to do about a shared defender.** Extract the decision it proves into a
+pure function, unit-test that, and point the claim at the unit test. This is
+not a testing trick — it is the same reason `classify()` in this codebase is
+pure. A rule you can state in three lines should not need a fifty-second
+acceptance test to falsify it. When the extraction is right the claim still
+fails on the fault; when it is wrong the claim SURVIVES, and the probe tells
+you so before you have shipped a weaker gate. Do not narrow `defendedBy` and
+assume — re-probe the claim and read the verdict.
+
 ### Authoring faults mechanically
 
 Writing faults by hand means reading the code to find exact anchors. Two
@@ -475,6 +706,18 @@ npx testguard-cli scaffold src/auth.ts --claim AUTH-ADMIN   # every proposal und
 | `element-removed` | a one-line JSX element — self-closing (`<Toggle … />`) or paired (`<button …>Save</button>`) — removed; the UI shape behind "the toggle is invisible", killable by a browser-layer defender |
 | `handler-dropped` | an `on<Event>={…}` prop removed, whether it is its own line or inline in the tag; the control renders and does nothing |
 
+The table above is JavaScript. `scaffold` reads Python too, and proposes the
+same fault classes in Python syntax: `if <guard>:` → `if False:`,
+`return <check>` → `return True`, `verify=True` → `verify=False`, a
+parameter-derived argument swapped for `None`, a key dropped from a payload
+`dict` or an allow-list. Two differences are deliberate. A statement is
+removed by replacing it with `pass`, never by deleting the line, because a
+block whose only statement is gone is an `IndentationError`; and a line that
+leaves a bracket open (`COLOURS = {`) is never removed at all. Both exist
+because a fault that cannot compile is a `fault-invalid` verdict — a probe run
+spent saying nothing about the tests. The two JSX shapes (`element-removed`,
+`handler-dropped`) have no Python meaning and are absent rather than faked.
+
 Every proposal's `find` is the exact line with `expectHits`/`occurrence`
 computed from the file, so it is verifiable by construction; provenance is
 `producer: derived`; `defendedBy` is prefilled from the tests that import
@@ -507,21 +750,28 @@ redacted text for the raw input and the test stays green. `probe` reports it
 as `SURVIVED`; the fixture's [README](fixtures/known-answer/README.md) walks
 through every verdict.
 
-## Status
+The same blind spot, in Python, is
+[`fixtures/known-answer-python/`](fixtures/known-answer-python/README.md) —
+run under **both** stdlib `unittest` and `pytest`, which have different report
+shapes and different notions of failure and must still agree on all sixteen
+verdicts.
 
-**v0.5.** Eleven commands (`status`, `init`, `claims`, `probe`, `admit`, `replay`, `baseline`, `brief`, `gate`, `scaffold`, `mcp`), vitest and jest runners, hand-authored faults plus
-**v0.5.** Nine commands (`status`, `init`, `claims`, `probe`, `admit`, `baseline`, `brief`, `gate`, `scaffold`), vitest, jest and Playwright runners, hand-authored faults plus
-a mechanical scaffold, an agent operating layer (`status`, `init`) and a
-change gate (`gate`). The contract
-spine — eight JSON Schemas shared with the other Guard tools — is under
-[`spec/`](spec/). One exact-pinned runtime dependency (`ajv`, for schema validation); Node ≥ 20.
+## Status
 
-Not yet: test generation (the acceptance half, `admit`, exists; the generating half stays the agent's), runners beyond
-vitest and jest, AST-aware producers, and the transfer of a calibration between repositories — `replay` measures it
-now; whether it carries to a repository with no history is unproven. Each is
+**v0.5.** Eleven commands (`status`, `init`, `claims`, `probe`, `admit`,
+`replay`, `baseline`, `brief`, `gate`, `scaffold`, `mcp`), vitest, jest,
+Playwright and Python (pytest / stdlib unittest) runners, hand-authored faults
+plus a mechanical scaffold for JavaScript and Python, an agent operating layer
+(`status`, `init`) and a change gate (`gate`). The contract spine — eight JSON
+Schemas shared with the other Guard tools — is under [`spec/`](spec/). One
+exact-pinned runtime dependency (`ajv`, for schema validation); Node ≥ 20, and
+Python ≥ 3.8 only when probing Python.
+
+Not yet: test generation (the acceptance half, `admit`, exists; the generating
+half stays the agent's), runners beyond those four, AST-aware producers, and
+the transfer of a calibration between repositories — `replay` measures it now;
+whether it carries to a repository with no history is unproven. Each is
 designed for; none is claimed.
-vitest, jest and Playwright, AST-aware producers, and calibration of fault classes
-against real escaped bugs. Each is designed for; none is claimed.
 
 ## Licence
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "testguard-cli",
-  "version": "0.6.0",
-  "description": "Proves a test suite defends the claims a project makes: injects the faults those claims forbid and reports every one the tests fail to detect.",
+  "version": "0.7.0",
+  "description": "Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest.",
   "type": "module",
   "license": "MIT",
   "engines": {

package/spec/GATE-SEMANTICS.md

@@ -12,7 +12,7 @@ The verdict set is closed. Only one value is a pass.
 | `killed` | Fault applied; every one of N probe runs failed with a genuine assertion failure, on a defender set that was green N/N unmodified. | no |
 | `survived` | Fault applied; every one of N probe runs passed. The claim is **unproven**. | **yes** |
 | `nocover` | No defending test exists: the declared globs resolve to nothing, or no test imports the subject. Worse than `survived` — nothing was even tried. | **yes** |
-| `unverifiable` | The claim could not be probed: the fault's anchor is missing or ambiguous, or its defenders failed to load (`defenders-failed-to-load`). Carries a `reason`. A loud, gating verdict — a claim that cannot be probed is not "skipped", it is undefended until someone fixes the fault or the defenders. Never confused with `flaky-defender`, which requires tests that *ran*. | **yes** |
+| `unverifiable` | The claim could not be probed: the fault's anchor is missing or ambiguous, its defenders failed to load (`defenders-failed-to-load`), probing it threw (`probe-error`), or the defenders do not execute the subject at all (`subject-not-executed`). Carries a `reason`. A loud, gating verdict — a claim that cannot be probed is not "skipped", it is undefended until someone fixes the fault or the defenders. Never confused with `flaky-defender`, which requires tests that *ran*. | **yes** |
 | `timeout` | Probe run exceeded its budget. Not counted as a kill; the pessimistic reading is the safe one because flakiness biases the metric optimistically. | **yes** |
 | `fault-invalid` | The replacement does not load or compile. A bad fault, not a detection. | **yes** |
 | `flaky-defender` | Defenders were not green N/N on unmodified source (`defenders-not-green`), or the N probe runs disagreed with each other (`inconsistent-probe`). Either way no verdict about the fault can be trusted; fix the defenders first. | **yes** |
@@ -36,10 +36,16 @@ Rules that follow from the table:
    that failed and then passed on retry (Playwright's `flaky`) is a
    **non-green run** even when the runner exits 0, and a test the runner
    reports as timed out (`timedOut`) is a timeout, never an assertion
-   failure. When one claim's defenders run under several runners, their
-   runs merge pessimistically — any load error is an error, any timeout is
-   a timeout, any failure is a failure — and the evidence names which file
-   ran under which runner.
+   failure. Python is read the same way: an `AssertionError` and any other
+   exception raised by the test body both count, because both are the suite
+   rejecting the behaviour; a collection error — a module that will not
+   import, which is what a replacement that does not parse looks like from
+   Python — is a load failure and never a kill; and a test the suite expected
+   to fail that passed (`xpass`) makes the run non-green without ever being an
+   assertion failure, exactly as Playwright's `flaky` does. When one claim's
+   defenders run under several runners, their runs merge pessimistically —
+   any load error is an error, any timeout is a timeout, any failure is a
+   failure — and the evidence names which file ran under which runner.
 4. **A mixed result is not a kill.** If some of the N probe runs fail and
    others pass, the defender's response to the fault is nondeterministic.
    Reporting `killed` would be the optimistic bias flakiness introduces;
@@ -68,6 +74,107 @@ Rules that follow from the table:
    content hash; the status document lists every fault whose content
    changed since it was probed, with its previous verdict. The change is
    allowed; it is never invisible.
+   The same hash governs **reuse**: a tool that carries a prior verdict
+   forward to save a run may only do so for the fault that produced it. A
+   verdict is an answer about one exact `find`/`replace`, so once those
+   change the old answer is not an answer to the new question — a weakened
+   fault would keep the verdict it earned before, and a repaired anchor would
+   keep `unverifiable`. Where the prior record has no content hash to compare,
+   the fault is probed again: the cost of re-measuring is a run, the cost of
+   the other direction is a verdict nobody measured.
+
+9. **The fault must be the code that ran.** A green baseline proves the
+   harness is not reporting everything as broken. It proves nothing about the
+   opposite direction: a fault the interpreter never executes leaves the
+   baseline green, every verdict `survived`, and a report that reads as a
+   devastating finding while being entirely false. A tool that can tell which
+   file was actually loaded must check it while the fault is applied, and must
+   **refuse the run** when the subject was loaded from outside the tree being
+   probed — in Python an editable install (PEP 660) registers an import hook
+   consulted ahead of every path entry, and a plain install leaves a copy in
+   `site-packages`, so this is the normal case, not an exotic one. When no
+   defender imported the subject at all, the run continues: an import inside a
+   branch the fault does not reach is legitimate. It is recorded as
+   `detail.targetNotImported`, because a `survived` there is a statement about
+   the defenders' reach and not about their assertions, and the two must not
+   be read as the same finding.
+10. **A signal describes a file, and says which kind of blindness it is.**
+   Replacing a whole module (`vi.mock`, `jest.mock`, a Python `patch` of the
+   module itself) removes a file from the defenders: it cannot detect anything
+   in what it replaced. Replacing **one attribute** of a module —
+   `patch("pkg.mod.fn")`, which is what Python code almost always does — does
+   not: a fault anywhere else in that module is still fully detectable, and
+   dropping the file would report `nocover` for a claim that is well defended.
+   The file stays a defender and carries `target-attribute-patched`, naming
+   the attributes. Signals never change a verdict.
+
+11. **One fault's failure costs that fault, not the run.** A fault whose probe
+   throws is recorded as `unverifiable` with reason `probe-error` and a
+   `detail.message` naming what threw; the run continues and still writes its
+   evidence. Losing the document would throw away every verdict already
+   decided and leave a caller with an exit code this document does not define,
+   which is strictly worse than one loud unverifiable claim. A `probe-error`
+   record is never reused by a later run: it says something about the run, not
+   about the code. The one exception is a **precondition** failure — the
+   interpreter loading the source from outside the probed tree, a runner that
+   does not resolve — which is a statement about every verdict in the run and
+   still refuses it outright.
+   Restoring the source is held to the same reading. The restore is
+   belt-and-braces in worktree mode, where the mutation only ever existed
+   inside a scratch worktree that is discarded anyway, so a target that has
+   vanished is already restored and is recorded as
+   `detail.restoreSkipped: "target-missing"` rather than raised — a reader can
+   still see that the tree moved underneath the run. Under `--in-place` the
+   same condition is the user's own file gone and must be raised. Any other
+   failure to write the original back is raised in both modes: it can mean a
+   mutated file left on disk.
+
+12. **A survivor must prove the defenders execute the subject.** A green
+   baseline proves the defenders can *pass*. It does not prove they can fail
+   *because of this file* — and if the fault is applied to code the test
+   process never executes, the baseline is green, every probe run is green,
+   and every claim is reported `survived`. That output reads as a devastating
+   audit finding and is entirely false, and no individual check in the run
+   contradicts it.
+   So a would-be `survived` is charged one more run: the subject is replaced
+   with content its loader cannot parse, and the defenders run once. Going red
+   proves they execute it (`detail.negativeControl: "reached"`), and the
+   `survived` stands. Staying green proves they do not, and the verdict is
+   `unverifiable` with reason `subject-not-executed` — the pessimistic reading,
+   and the only honest one, because nothing was measured about their
+   assertions.
+   Charged **only** on a would-be survivor, and cached per subject and defender
+   set. A kill already proves the defenders reached the code, so paying there
+   would be waste. A runner that cannot say what "cannot compile" means for its
+   language does not guess: no control is run, no signal is recorded, and the
+   verdict is unchanged. `detail.targetNotImported` (reported by runners that
+   can name the file the interpreter loaded) answers the same question more
+   precisely for one language; where both are present they must agree.
+
+13. **Fault injection is one method, not the definition.** A claim declares
+   what must be true; a probe declares how a tool would try to make it false,
+   and `method` says which way. `fault-injection` — mutate the source, run the
+   claim's defenders, confirm over N runs — is the only method specified in
+   v1, and it is the default when the field is absent, so every document
+   written before the field existed is held to exactly the rules it was
+   written against.
+   Everything in this document above this rule is `fault-injection`'s
+   semantics. `confirmRuns`, `mode`, `defenders`, `inputs` and the baseline
+   and probe runs are what *it* means by showing its work; a tool that reads
+   code or scans a surface has none of them, and requiring them would force it
+   to emit empty arrays to conform — which is lying, and the one thing this
+   format exists to make impossible. The schema therefore requires them only
+   under `fault-injection`, and the validator enforces them there in full: the
+   relaxation is for a tool that injects nothing, never a licence for an
+   injecting tool to stop showing its work.
+   `assertion` and `scan` are **reserved**. Their required fields are
+   deliberately unspecified and will be defined by their first real consumer,
+   with its own conformance examples written from the shape that tool actually
+   has. Designing them for an absent tool is precisely the mistake that made
+   this rule necessary: the spec was drafted against a single consumer and
+   hard-coded its assumptions as requirements.
+   The verdict set stays closed and shared. What each method may legitimately
+   report is part of defining it.
 
 ## Replay reports; it never gates
 

package/spec/README.md

@@ -1,16 +1,25 @@
-# Guard spec — the contract spine
+# claimspec — the contract spine
 
-Eight formats that tools following the Guard pattern share. They share
-*formats*, not code: [`docguard-cli`](https://www.npmjs.com/package/docguard-cli)
-is Node ESM, [`websec-validator`](https://pypi.org/project/websec-validator/)
-is Python, and porting one runtime into the other is not worth it. A tool
-conforms by emitting these shapes; it keeps its own language, CLI and UX.
+Nine formats for declaring what must be true about a codebase, recording the
+evidence that tried to falsify it, and gating only what is new.
 
-The pattern all such tools run:
+They are shared as *formats*, not as code. The first tools to adopt them are
+[`docguard-cli`](https://www.npmjs.com/package/docguard-cli) (Node ESM) and
+[`websec-validator`](https://pypi.org/project/websec-validator/) (Python) —
+porting one runtime into the other was never worth it, and never the point. A
+tool conforms by emitting these shapes; it keeps its own language, CLI and UX.
+
+The method they have in common:
 
 > declare what must be true → try mechanically to falsify it → freeze a
 > baseline → gate only the delta → brief the agent before it writes code.
 
+**Falsifying by fault injection is one method, not the definition.** A tool
+that verifies a claim by reading code, by scanning a surface, or by replaying
+history is as much a consumer as one that mutates source. The formats that
+carry *how* a claim was tested say which method produced them; the formats
+that carry claims, baselines, scoping and briefs do not care.
+
 ## Formats
 
 | Kind | Schema | Purpose |
@@ -30,7 +39,7 @@ Shared definitions (verdicts, fault classes, provenance, annotations) live in
 which verdicts turn CI red, baseline and delta rules, exit codes — is in
 [`GATE-SEMANTICS.md`](GATE-SEMANTICS.md).
 
-Schemas are JSON Schema 2020-12 and identified as `urn:guard-spec:v1:<kind>`.
+Schemas are JSON Schema 2020-12 and identified as `urn:claimspec:v1:<kind>`.
 
 ## Conformance
 
@@ -53,5 +62,14 @@ npm run test:spec
 
 The spec is drafted here and validated against `testguard` as its first
 consumer. It moves to its own repository only when a second tool adopts it.
-Adoption by an existing tool is one additive output writer — no rewrite, no
-behaviour change, and optional until that tool wants it.
+
+Adoption is additive and optional: an existing tool gains an output writer, and
+nothing it already detects or already writes changes. That holds today for
+`baseline`, `ignore`, `brief` and `status`. It does **not** yet hold for
+`claims` and `evidence`, whose required fields assume fault injection
+(`find`/`replace` anchors, `confirmRuns`, `baselineRuns`/`probeRuns`,
+`defenders`) — a tool that scans or reads would have to emit empty arrays to
+satisfy the schema, which is conforming by lying. Making those two kinds
+method-agnostic is tracked as
+[#81](https://github.com/raccioly/testguard/issues/81) and blocks adoption of
+either.