testguard-cli 0.5.0 → 0.7.0

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "testguard-cli",
-  "version": "0.5.0",
-  "description": "Proves a test suite defends the claims a project makes: injects the faults those claims forbid and reports every one the tests fail to detect.",
+  "version": "0.7.0",
+  "description": "Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest.",
   "type": "module",
   "license": "MIT",
   "engines": {

package/spec/GATE-SEMANTICS.md

@@ -12,7 +12,7 @@ The verdict set is closed. Only one value is a pass.
 | `killed` | Fault applied; every one of N probe runs failed with a genuine assertion failure, on a defender set that was green N/N unmodified. | no |
 | `survived` | Fault applied; every one of N probe runs passed. The claim is **unproven**. | **yes** |
 | `nocover` | No defending test exists: the declared globs resolve to nothing, or no test imports the subject. Worse than `survived` — nothing was even tried. | **yes** |
-| `unverifiable` | The claim could not be probed: the fault's anchor is missing or ambiguous, or its defenders failed to load (`defenders-failed-to-load`). Carries a `reason`. A loud, gating verdict — a claim that cannot be probed is not "skipped", it is undefended until someone fixes the fault or the defenders. Never confused with `flaky-defender`, which requires tests that *ran*. | **yes** |
+| `unverifiable` | The claim could not be probed: the fault's anchor is missing or ambiguous, its defenders failed to load (`defenders-failed-to-load`), probing it threw (`probe-error`), or the defenders do not execute the subject at all (`subject-not-executed`). Carries a `reason`. A loud, gating verdict — a claim that cannot be probed is not "skipped", it is undefended until someone fixes the fault or the defenders. Never confused with `flaky-defender`, which requires tests that *ran*. | **yes** |
 | `timeout` | Probe run exceeded its budget. Not counted as a kill; the pessimistic reading is the safe one because flakiness biases the metric optimistically. | **yes** |
 | `fault-invalid` | The replacement does not load or compile. A bad fault, not a detection. | **yes** |
 | `flaky-defender` | Defenders were not green N/N on unmodified source (`defenders-not-green`), or the N probe runs disagreed with each other (`inconsistent-probe`). Either way no verdict about the fault can be trusted; fix the defenders first. | **yes** |
@@ -32,7 +32,20 @@ Rules that follow from the table:
    timeout does not, and a suite that fails to load does not: neither is
    evidence that the suite defends the claim. Tools parse the runner's
    structured report, never its exit code, because the exit code cannot tell
-   these apart.
+   these apart. A runner's own retry mechanism does not change this: a test
+   that failed and then passed on retry (Playwright's `flaky`) is a
+   **non-green run** even when the runner exits 0, and a test the runner
+   reports as timed out (`timedOut`) is a timeout, never an assertion
+   failure. Python is read the same way: an `AssertionError` and any other
+   exception raised by the test body both count, because both are the suite
+   rejecting the behaviour; a collection error — a module that will not
+   import, which is what a replacement that does not parse looks like from
+   Python — is a load failure and never a kill; and a test the suite expected
+   to fail that passed (`xpass`) makes the run non-green without ever being an
+   assertion failure, exactly as Playwright's `flaky` does. When one claim's
+   defenders run under several runners, their runs merge pessimistically —
+   any load error is an error, any timeout is a timeout, any failure is a
+   failure — and the evidence names which file ran under which runner.
 4. **A mixed result is not a kill.** If some of the N probe runs fail and
    others pass, the defender's response to the fault is nondeterministic.
    Reporting `killed` would be the optimistic bias flakiness introduces;
@@ -43,6 +56,11 @@ Rules that follow from the table:
    and `detail.undeclaredKillers` names the tests, so the author can fix
    `defendedBy`. The claim's stated evidence chain is broken even though the
    suite is not blind. It gates, and ranks below a true survivor.
+   Attribution runs **up to N times**, and stops as soon as no test has failed
+   in every run so far: from there no further run can name a killer. The
+   evidence records the early stop (`detail.escalationStoppedEarly`). Stopping
+   can only fail to *name* a killer, never upgrade a verdict, which is the
+   pessimistic side.
 6. **A verdict names the commit it is about.** Evidence records `repo.head`;
    when the working tree was probed instead, `repo.snapshot` holds the
    throwaway commit that captured it. A tool must refuse to probe a commit
@@ -56,6 +74,136 @@ Rules that follow from the table:
    content hash; the status document lists every fault whose content
    changed since it was probed, with its previous verdict. The change is
    allowed; it is never invisible.
+   The same hash governs **reuse**: a tool that carries a prior verdict
+   forward to save a run may only do so for the fault that produced it. A
+   verdict is an answer about one exact `find`/`replace`, so once those
+   change the old answer is not an answer to the new question — a weakened
+   fault would keep the verdict it earned before, and a repaired anchor would
+   keep `unverifiable`. Where the prior record has no content hash to compare,
+   the fault is probed again: the cost of re-measuring is a run, the cost of
+   the other direction is a verdict nobody measured.
+
+9. **The fault must be the code that ran.** A green baseline proves the
+   harness is not reporting everything as broken. It proves nothing about the
+   opposite direction: a fault the interpreter never executes leaves the
+   baseline green, every verdict `survived`, and a report that reads as a
+   devastating finding while being entirely false. A tool that can tell which
+   file was actually loaded must check it while the fault is applied, and must
+   **refuse the run** when the subject was loaded from outside the tree being
+   probed — in Python an editable install (PEP 660) registers an import hook
+   consulted ahead of every path entry, and a plain install leaves a copy in
+   `site-packages`, so this is the normal case, not an exotic one. When no
+   defender imported the subject at all, the run continues: an import inside a
+   branch the fault does not reach is legitimate. It is recorded as
+   `detail.targetNotImported`, because a `survived` there is a statement about
+   the defenders' reach and not about their assertions, and the two must not
+   be read as the same finding.
+10. **A signal describes a file, and says which kind of blindness it is.**
+   Replacing a whole module (`vi.mock`, `jest.mock`, a Python `patch` of the
+   module itself) removes a file from the defenders: it cannot detect anything
+   in what it replaced. Replacing **one attribute** of a module —
+   `patch("pkg.mod.fn")`, which is what Python code almost always does — does
+   not: a fault anywhere else in that module is still fully detectable, and
+   dropping the file would report `nocover` for a claim that is well defended.
+   The file stays a defender and carries `target-attribute-patched`, naming
+   the attributes. Signals never change a verdict.
+
+11. **One fault's failure costs that fault, not the run.** A fault whose probe
+   throws is recorded as `unverifiable` with reason `probe-error` and a
+   `detail.message` naming what threw; the run continues and still writes its
+   evidence. Losing the document would throw away every verdict already
+   decided and leave a caller with an exit code this document does not define,
+   which is strictly worse than one loud unverifiable claim. A `probe-error`
+   record is never reused by a later run: it says something about the run, not
+   about the code. The one exception is a **precondition** failure — the
+   interpreter loading the source from outside the probed tree, a runner that
+   does not resolve — which is a statement about every verdict in the run and
+   still refuses it outright.
+   Restoring the source is held to the same reading. The restore is
+   belt-and-braces in worktree mode, where the mutation only ever existed
+   inside a scratch worktree that is discarded anyway, so a target that has
+   vanished is already restored and is recorded as
+   `detail.restoreSkipped: "target-missing"` rather than raised — a reader can
+   still see that the tree moved underneath the run. Under `--in-place` the
+   same condition is the user's own file gone and must be raised. Any other
+   failure to write the original back is raised in both modes: it can mean a
+   mutated file left on disk.
+
+12. **A survivor must prove the defenders execute the subject.** A green
+   baseline proves the defenders can *pass*. It does not prove they can fail
+   *because of this file* — and if the fault is applied to code the test
+   process never executes, the baseline is green, every probe run is green,
+   and every claim is reported `survived`. That output reads as a devastating
+   audit finding and is entirely false, and no individual check in the run
+   contradicts it.
+   So a would-be `survived` is charged one more run: the subject is replaced
+   with content its loader cannot parse, and the defenders run once. Going red
+   proves they execute it (`detail.negativeControl: "reached"`), and the
+   `survived` stands. Staying green proves they do not, and the verdict is
+   `unverifiable` with reason `subject-not-executed` — the pessimistic reading,
+   and the only honest one, because nothing was measured about their
+   assertions.
+   Charged **only** on a would-be survivor, and cached per subject and defender
+   set. A kill already proves the defenders reached the code, so paying there
+   would be waste. A runner that cannot say what "cannot compile" means for its
+   language does not guess: no control is run, no signal is recorded, and the
+   verdict is unchanged. `detail.targetNotImported` (reported by runners that
+   can name the file the interpreter loaded) answers the same question more
+   precisely for one language; where both are present they must agree.
+
+13. **Fault injection is one method, not the definition.** A claim declares
+   what must be true; a probe declares how a tool would try to make it false,
+   and `method` says which way. `fault-injection` — mutate the source, run the
+   claim's defenders, confirm over N runs — is the only method specified in
+   v1, and it is the default when the field is absent, so every document
+   written before the field existed is held to exactly the rules it was
+   written against.
+   Everything in this document above this rule is `fault-injection`'s
+   semantics. `confirmRuns`, `mode`, `defenders`, `inputs` and the baseline
+   and probe runs are what *it* means by showing its work; a tool that reads
+   code or scans a surface has none of them, and requiring them would force it
+   to emit empty arrays to conform — which is lying, and the one thing this
+   format exists to make impossible. The schema therefore requires them only
+   under `fault-injection`, and the validator enforces them there in full: the
+   relaxation is for a tool that injects nothing, never a licence for an
+   injecting tool to stop showing its work.
+   `assertion` and `scan` are **reserved**. Their required fields are
+   deliberately unspecified and will be defined by their first real consumer,
+   with its own conformance examples written from the shape that tool actually
+   has. Designing them for an absent tool is precisely the mistake that made
+   this rule necessary: the spec was drafted against a single consumer and
+   hard-coded its assumptions as requirements.
+   The verdict set stays closed and shared. What each method may legitimately
+   report is part of defining it.
+
+## Replay reports; it never gates
+
+A replayed bug is history. It escaped, by definition, which means the suite
+missed it; a gate on that is a gate nobody can pass on a first run, and it
+would only teach teams to stop looking. `replay` therefore always exits `0`
+unless it could not run at all.
+
+Its verdicts mirror the probe's, for the same reasons:
+
+| Verdict | Meaning |
+|---|---|
+| `caught` | a remaining test failed **by assertion** on the reverted source, every run. The suite knew. |
+| `blind` | the suite stayed green on known-broken code. |
+| `nocover` | no test imports the reverted files. Worse than `blind`: nothing was even tried. |
+| `unverifiable` | the revert did not apply, the suite failed to load, or it timed out. Carries a `reason`. |
+| `flaky` | the runs disagreed. A flaky failure reads as "the suite caught it", so flakiness biases this metric **optimistically** — mixed runs are never `caught`. |
+
+Two rules that follow:
+
+1. **One patch, one row.** De-duplicate by `git patch-id`; a dual-branch
+   topology carries the same fix under two or three shas, and counting it
+   twice corrupts the corpus a calibration is computed from.
+2. **The fix's own test is removed before the run.** It proves nothing about
+   what the suite knew before the fix existed.
+
+Only `caught` and `blind` carry information. `nocover`, `flaky` and
+`unverifiable` are excluded from both sides of a calibration ratio, because a
+number computed over them would mean nothing.
 
 ## Baseline and delta
 
@@ -72,6 +220,15 @@ derived from identities and outcome, never from the fault's `find`/`replace`
 text, so repairing a rotted anchor does not churn the baseline — while a
 change of verdict on the same claim+subject does surface as new.
 
+A baseline records the commit its evidence was taken at. When that evidence
+came from a working-tree snapshot, the baseline says so (`snapshot`) and its
+`head` is the **parent** of the commit that will carry the tests. A tool may
+offer to re-stamp such a baseline onto a later commit, but only when a clean
+probe of that commit reproduced exactly the same fingerprints; otherwise the
+user re-probes and freezes a new baseline. A frozen contract is never
+silently rewritten, and a status document notes a baseline that predates
+HEAD without turning that into a state.
+
 Adopting tools may reconcile an existing baseline format (for example a
 `{version, fingerprints:{hash:count}}` file) by mapping it onto this shape;
 the suppress-up-to-count semantics are identical.
@@ -84,6 +241,13 @@ the suppress-up-to-count semantics are identical.
   for reasonless `path` entries.
 - An annotation is **strictly additive**. It never changes, suppresses, or
   drops a finding. Ranking may read annotations; verdicts never do.
+- A **signal** is a static, annotation-grade fact about a defender, recorded
+  on the evidence and never a verdict. `mocked-never-asserted`: a test file
+  mocks the subject's module and never asserts on anything imported from it.
+  An author may silence it with `unasserted: <reason>` above the mock; the
+  tool then records `unasserted-annotated` with the reason — silenced, never
+  hidden. A file that mocks the subject is never a discovered defender: a
+  mock cannot detect a fault in what it replaced.
 
 ## Claim coverage of a change
 

package/spec/README.md

@@ -1,16 +1,25 @@
-# Guard spec — the contract spine
+# claimspec — the contract spine
 
-Eight formats that tools following the Guard pattern share. They share
-*formats*, not code: [`docguard-cli`](https://www.npmjs.com/package/docguard-cli)
-is Node ESM, [`websec-validator`](https://pypi.org/project/websec-validator/)
-is Python, and porting one runtime into the other is not worth it. A tool
-conforms by emitting these shapes; it keeps its own language, CLI and UX.
+Nine formats for declaring what must be true about a codebase, recording the
+evidence that tried to falsify it, and gating only what is new.
 
-The pattern all such tools run:
+They are shared as *formats*, not as code. The first tools to adopt them are
+[`docguard-cli`](https://www.npmjs.com/package/docguard-cli) (Node ESM) and
+[`websec-validator`](https://pypi.org/project/websec-validator/) (Python) —
+porting one runtime into the other was never worth it, and never the point. A
+tool conforms by emitting these shapes; it keeps its own language, CLI and UX.
+
+The method they have in common:
 
 > declare what must be true → try mechanically to falsify it → freeze a
 > baseline → gate only the delta → brief the agent before it writes code.
 
+**Falsifying by fault injection is one method, not the definition.** A tool
+that verifies a claim by reading code, by scanning a surface, or by replaying
+history is as much a consumer as one that mutates source. The formats that
+carry *how* a claim was tested say which method produced them; the formats
+that carry claims, baselines, scoping and briefs do not care.
+
 ## Formats
 
 | Kind | Schema | Purpose |
@@ -22,6 +31,7 @@ The pattern all such tools run:
 | `calibration` | [`schemas/calibration.schema.json`](schemas/calibration.schema.json) | P(finding is real) per bucket, with a Wilson interval and the sample size behind it. |
 | `brief` | [`schemas/brief.schema.json`](schemas/brief.schema.json) | What to tell an agent before it writes code — ranked, capped, never a single score, and carrying the one next action. |
 | `status` | [`schemas/status.schema.json`](schemas/status.schema.json) | Where the project is and what happens next — the single machine-readable truth every human rendering derives from. Surfaces faults whose content changed since they were probed, and changed files that carry no claim. |
+| `replay` | [`schemas/replay.schema.json`](schemas/replay.schema.json) | Would this suite have caught the bugs that already escaped? Replays real fix commits: revert the source, remove the test the fix shipped, run what remains. A replayed bug is ground truth — a human already confirmed it was a defect — which is what a fault model is calibrated against. |
 | `gate` | [`schemas/gate.schema.json`](schemas/gate.schema.json) | Claim coverage of one change: which changed files carry a claim, which are excused (and by which ignore entry), which are unclaimed. The delta gate for code that has no claim yet. |
 
 Shared definitions (verdicts, fault classes, provenance, annotations) live in
@@ -29,7 +39,7 @@ Shared definitions (verdicts, fault classes, provenance, annotations) live in
 which verdicts turn CI red, baseline and delta rules, exit codes — is in
 [`GATE-SEMANTICS.md`](GATE-SEMANTICS.md).
 
-Schemas are JSON Schema 2020-12 and identified as `urn:guard-spec:v1:<kind>`.
+Schemas are JSON Schema 2020-12 and identified as `urn:claimspec:v1:<kind>`.
 
 ## Conformance
 
@@ -52,5 +62,14 @@ npm run test:spec
 
 The spec is drafted here and validated against `testguard` as its first
 consumer. It moves to its own repository only when a second tool adopts it.
-Adoption by an existing tool is one additive output writer — no rewrite, no
-behaviour change, and optional until that tool wants it.
+
+Adoption is additive and optional: an existing tool gains an output writer, and
+nothing it already detects or already writes changes. That holds today for
+`baseline`, `ignore`, `brief` and `status`. It does **not** yet hold for
+`claims` and `evidence`, whose required fields assume fault injection
+(`find`/`replace` anchors, `confirmRuns`, `baselineRuns`/`probeRuns`,
+`defenders`) — a tool that scans or reads would have to emit empty arrays to
+satisfy the schema, which is conforming by lying. Making those two kinds
+method-agnostic is tracked as
+[#81](https://github.com/raccioly/testguard/issues/81) and blocks adoption of
+either.

package/spec/lib/validate.mjs

@@ -6,7 +6,14 @@ import { fingerprint } from './fingerprint.mjs';
 
 const schemaDir = join(dirname(fileURLToPath(import.meta.url)), '..', 'schemas');
 
-export const KINDS = Object.freeze(['claims', 'evidence', 'baseline', 'ignore', 'calibration', 'brief', 'status', 'gate']);
+export const KINDS = Object.freeze(['claims', 'evidence', 'baseline', 'ignore', 'calibration', 'brief', 'status', 'gate', 'replay']);
+/**
+ * How a document says it tried to falsify its claims. Absent means
+ * `fault-injection`, so every document written before the field existed is
+ * held to exactly the rules it was written against.
+ */
+export const methodOf = (x) => x?.method ?? 'fault-injection';
+const isInjection = (x) => methodOf(x) === 'fault-injection';
 export const PASSING_VERDICTS = Object.freeze(new Set(['killed']));
 
 const ajv = new Ajv2020({ strict: true, allErrors: true });
@@ -15,7 +22,7 @@ for (const f of readdirSync(schemaDir).filter((n) => n.endsWith('.schema.json'))
 }
 
 const schemaFor = (kind) => {
-  const v = ajv.getSchema(`urn:guard-spec:v1:${kind}`);
+  const v = ajv.getSchema(`urn:claimspec:v1:${kind}`);
   if (!v) throw new RangeError(`unknown Guard-spec kind: ${kind}`);
   return v;
 };
@@ -33,10 +40,15 @@ const semantic = {
         const p = `/claims/${ci}/faults/${fi}`;
         if (seenFault.has(f.id)) errors.push({ path: `${p}/id`, message: `duplicate fault id "${f.id}" in claim "${c.id}"` });
         seenFault.add(f.id);
-        const occ = f.occurrence ?? 1;
-        const hits = f.expectHits ?? 1;
-        if (occ > hits) errors.push({ path: `${p}/occurrence`, message: `occurrence ${occ} exceeds expectHits ${hits}` });
-        if (f.find === f.replace) errors.push({ path: `${p}/replace`, message: 'replace is identical to find; the fault is a no-op' });
+        // Anchor arithmetic is a property of fault injection. A probe that does
+        // not substitute text has no `find` to compare, and `undefined ===
+        // undefined` would report every one of them as a no-op.
+        if (isInjection(f)) {
+          const occ = f.occurrence ?? 1;
+          const hits = f.expectHits ?? 1;
+          if (occ > hits) errors.push({ path: `${p}/occurrence`, message: `occurrence ${occ} exceeds expectHits ${hits}` });
+          if (f.find === f.replace) errors.push({ path: `${p}/replace`, message: 'replace is identical to find; the fault is a no-op' });
+        }
       });
     });
     return errors;
@@ -44,14 +56,39 @@ const semantic = {
 
   evidence(doc) {
     const errors = [];
+    const injection = isInjection(doc.run);
     const n = doc.run.confirmRuns;
-    if (n < 3 && doc.run.provisional !== true) errors.push({ path: '/run/provisional', message: `confirmRuns ${n} is below 3; the run must declare provisional: true` });
-    if (n >= 3 && doc.run.provisional === true) errors.push({ path: '/run/provisional', message: `confirmRuns ${n} is confirmed; provisional must be absent or false` });
+    // N-run agreement is what fault injection means by "confirmed". A method
+    // that does not re-run anything has no N, and a rule about one would be a
+    // rule about a number that is not there.
+    if (injection) {
+      if (n < 3 && doc.run.provisional !== true) errors.push({ path: '/run/provisional', message: `confirmRuns ${n} is below 3; the run must declare provisional: true` });
+      if (n >= 3 && doc.run.provisional === true) errors.push({ path: '/run/provisional', message: `confirmRuns ${n} is confirmed; provisional must be absent or false` });
+    }
+    if (doc.run.runners && doc.run.runner && !doc.run.runners.some((x) => x.name === doc.run.runner.name)) errors.push({ path: '/run/runners', message: 'runners must include the project runner named in run.runner' });
+    if (doc.run.repo.ignoredDirty && doc.run.repo.snapshot) errors.push({ path: '/run/repo/ignoredDirty', message: 'a working-tree snapshot has no ignored dirty files: the tree was probed as it is' });
     doc.records.forEach((r, i) => {
       const p = `/records/${i}`;
       const expected = fingerprint({ claimId: r.claim.id, subjectId: r.subject.id, file: r.subject.file ?? '', verdict: r.verdict });
       if (r.fingerprint !== expected) errors.push({ path: `${p}/fingerprint`, message: `fingerprint does not match spec derivation (expected ${expected})` });
 
+      // What the schema no longer requires, the validator still does — for the
+      // one method that means it. Fault injection without defenders, inputs or
+      // runs is not a leaner document, it is a verdict with nothing behind it;
+      // relaxing the schema was to let a scanner conform, never to let an
+      // injecting tool stop showing its work.
+      if (injection) {
+        for (const k of ['defenders', 'inputs']) {
+          if (!r[k]) errors.push({ path: `${p}/${k}`, message: `fault-injection requires ${k} on every record` });
+        }
+        for (const k of ['baselineRuns', 'probeRuns']) {
+          if (!Array.isArray(r.detail[k])) errors.push({ path: `${p}/detail/${k}`, message: `fault-injection requires detail.${k} on every record` });
+        }
+      }
+      // Every rule below reads those fields, so a record that is already
+      // malformed must not also throw on the way to being reported.
+      if (!injection || !r.defenders || !r.inputs || !Array.isArray(r.detail.baselineRuns) || !Array.isArray(r.detail.probeRuns)) return;
+
       const { baselineRuns, probeRuns } = r.detail;
       if (r.verdict === 'killed' || r.verdict === 'survived') {
         if (baselineRuns.length !== n) errors.push({ path: `${p}/detail/baselineRuns`, message: `${r.verdict} requires exactly confirmRuns (${n}) baseline runs, got ${baselineRuns.length}` });
@@ -74,12 +111,75 @@ const semantic = {
       if (r.verdict === 'nocover' && !r.defenders.nocover) {
         errors.push({ path: `${p}/defenders/nocover`, message: 'nocover verdict requires defenders.nocover = true' });
       }
+      if (r.detail.flakeRate) {
+        const { runs, failures } = r.detail.flakeRate;
+        if (failures > runs) errors.push({ path: `${p}/detail/flakeRate`, message: `flakeRate failures (${failures}) exceed runs (${runs})` });
+        if (failures > 0 && r.verdict !== 'flaky-defender') errors.push({ path: `${p}/detail/flakeRate`, message: 'a defender that failed on unmodified source makes the verdict flaky-defender' });
+      }
+      if (r.detail.independence) {
+        if (r.verdict !== 'killed') errors.push({ path: `${p}/detail/independence`, message: 'independence is a property of a kill; only killed records carry it' });
+        const { class: cls, defenderCommit, targetCommit } = r.detail.independence;
+        if (cls !== 'unknown' && !(defenderCommit && targetCommit)) errors.push({ path: `${p}/detail/independence`, message: `independence class ${cls} requires both defenderCommit and targetCommit` });
+      }
       if (r.detail.undeclaredKillers && r.detail.reason !== 'killed-by-undeclared-tests') {
         errors.push({ path: `${p}/detail/undeclaredKillers`, message: 'undeclaredKillers is only meaningful with reason killed-by-undeclared-tests' });
       }
+      if (r.defenders.byRunner) {
+        const resolved = new Set(r.defenders.resolved);
+        for (const [runner, files] of Object.entries(r.defenders.byRunner)) for (const f of files) if (!resolved.has(f)) errors.push({ path: `${p}/defenders/byRunner/${runner}`, message: `${f} ran under ${runner} but is not a resolved defender` });
+        if (Object.keys(r.defenders.byRunner).length < 2) errors.push({ path: `${p}/defenders/byRunner`, message: 'byRunner is only meaningful when more than one runner ran the defenders' });
+      }
+      if (r.defenders.mocking && r.defenders.discovered) {
+        for (const f of r.defenders.mocking) if (r.defenders.resolved.includes(f)) errors.push({ path: `${p}/defenders/mocking`, message: `${f} mocks the subject and was discovered; it cannot also be a resolved defender` });
+      }
+      for (const s of r.defenders.signals ?? []) {
+        if (s.signal === 'unasserted-annotated' && !s.reason) errors.push({ path: `${p}/defenders/signals`, message: `unasserted-annotated on ${s.file} requires the annotation's reason` });
+        // An attribute patch (Python) leaves the file a defender, so it belongs
+        // to `resolved` and not to `mocking`; every other signal is about a file
+        // that replaced the module and must be listed as mocking it.
+        if (s.signal === 'target-attribute-patched') {
+          if (!s.reason) errors.push({ path: `${p}/defenders/signals`, message: `target-attribute-patched on ${s.file} requires the patched attributes as its reason` });
+          if (!r.defenders.resolved.includes(s.file)) errors.push({ path: `${p}/defenders/signals`, message: `${s.file} patches attributes of the subject but is not a resolved defender` });
+        } else if (!(r.defenders.mocking ?? []).includes(s.file)) {
+          errors.push({ path: `${p}/defenders/signals`, message: `${s.file} carries a mock signal but is not listed in defenders.mocking` });
+        }
+      }
+      // A module the defenders never imported cannot have been killed by them.
+      if (r.detail.targetNotImported && r.verdict === 'killed') {
+        errors.push({ path: `${p}/detail/targetNotImported`, message: 'the subject was never imported, so the defenders cannot have killed the fault' });
+      }
+      // The control is only ever charged on a would-be survivor, so a `reached`
+      // record must be one, and `not-reached` must have become unverifiable.
+      if (r.detail.negativeControl === 'reached' && r.verdict !== 'survived') {
+        errors.push({ path: `${p}/detail/negativeControl`, message: `negativeControl "reached" belongs to a survivor, not to ${r.verdict}` });
+      }
+      if (r.detail.negativeControl === 'not-reached' && !(r.verdict === 'unverifiable' && r.detail.reason === 'subject-not-executed')) {
+        errors.push({ path: `${p}/detail/negativeControl`, message: 'negativeControl "not-reached" requires verdict unverifiable with reason subject-not-executed' });
+      }
+      if (r.detail.reason === 'subject-not-executed' && r.detail.negativeControl !== 'not-reached') {
+        errors.push({ path: `${p}/detail/reason`, message: 'subject-not-executed requires the negativeControl that established it' });
+      }
+      // Python's import provenance and the general control answer the same
+      // question. A document where they disagree describes a run that cannot
+      // have happened.
+      if (r.detail.targetNotImported && r.detail.negativeControl === 'reached') {
+        errors.push({ path: `${p}/detail/negativeControl`, message: 'the subject was never imported, so the negative control cannot have reached it' });
+      }
       if (r.verdict === 'unverifiable' && !r.detail.reason) {
         errors.push({ path: `${p}/detail/reason`, message: 'unverifiable requires a reason (e.g. anchor-missing, anchor-ambiguous)' });
       }
+      // `probe-error` is the one reason that names nothing on its own: it says
+      // the run threw, not what threw. Without the message the record is a
+      // dead end for anyone reading the document instead of rerunning it.
+      if (r.detail.reason === 'probe-error' && !r.detail.message) {
+        errors.push({ path: `${p}/detail/message`, message: 'probe-error requires a message naming what threw' });
+      }
+      // In-place is the mode where a vanished target is the user's own file
+      // gone; it raises rather than records. A document claiming otherwise
+      // describes a run that cannot have happened.
+      if (r.detail.restoreSkipped && doc.run.mode === 'in-place') {
+        errors.push({ path: `${p}/detail/restoreSkipped`, message: 'restoreSkipped cannot occur in in-place mode, where a missing target is raised' });
+      }
     });
     return errors;
   },
@@ -104,10 +204,42 @@ const semantic = {
     if (doc.state === 'clean' && (doc.counts.new ?? 0) > 0) errors.push({ path: '/state', message: 'clean with new findings' });
     if (doc.state === 'unclaimed-changes' && !(doc.changes && doc.changes.uncovered.length > 0)) errors.push({ path: '/changes/uncovered', message: 'unclaimed-changes requires at least one uncovered changed file' });
     if (doc.state === 'unclaimed-changes' && doc.next.action !== 'claim') errors.push({ path: '/next/action', message: 'unclaimed-changes requires next.action = claim' });
+    if (doc.evidenceSource && !doc.evidenceHead) errors.push({ path: '/evidenceHead', message: 'evidence was read, so the commit it describes must be recorded' });
     if (doc.changes && doc.changes.uncovered.length > 0 && !['no-claims', 'unclaimed-changes'].includes(doc.state)) errors.push({ path: '/state', message: 'uncovered changed files are hidden behind a later state; unclaimed-changes precedes every evidence state' });
     return errors;
   },
 
+  replay(doc) {
+    const errors = [];
+    const n = doc.run.confirmRuns;
+    const seen = new Set();
+    doc.records.forEach((r, i) => {
+      const p = `/records/${i}`;
+      // One patch, one row: a dual-branch topology carries the same fix under
+      // two or three shas, and counting it twice corrupts the corpus the
+      // calibration is computed from.
+      if (seen.has(r.patchId)) errors.push({ path: `${p}/patchId`, message: `duplicate patch-id ${r.patchId.slice(0, 12)}: the same fix counted twice` });
+      seen.add(r.patchId);
+      if (['caught', 'blind'].includes(r.verdict) && r.runs.length !== n) {
+        errors.push({ path: `${p}/runs`, message: `${r.verdict} requires exactly confirmRuns (${n}) runs, got ${r.runs.length}` });
+      }
+      // Only a test body rejecting the reverted source counts as caught —
+      // the same rule as `killed`, for the same reason.
+      if (r.verdict === 'caught' && !r.runs.every((x) => x.outcome === 'fail' && (x.assertionFailures ?? 0) > 0)) {
+        errors.push({ path: `${p}/runs`, message: 'caught requires every run to fail with at least one assertion failure' });
+      }
+      if (r.verdict === 'blind' && !r.runs.every((x) => x.outcome === 'pass')) {
+        errors.push({ path: `${p}/runs`, message: 'blind requires every run to pass: the suite stayed green on known-broken code' });
+      }
+      if (r.verdict === 'flaky' && !(r.runs.some((x) => x.outcome === 'pass') && r.runs.some((x) => x.outcome === 'fail'))) {
+        errors.push({ path: `${p}/runs`, message: 'flaky requires runs that disagree' });
+      }
+      if (r.verdict === 'nocover' && r.ranTests) errors.push({ path: `${p}/ranTests`, message: 'nocover means no test exercises the reverted files; it cannot have run tests' });
+      if (r.verdict === 'unverifiable' && !r.reason) errors.push({ path: `${p}/reason`, message: 'unverifiable requires a reason (e.g. revert-did-not-apply, suite-failed-to-load)' });
+    });
+    return errors;
+  },
+
   gate(doc) {
     const errors = [];
     if (doc.evaluated !== doc.covered.length + doc.uncovered.length) errors.push({ path: '/evaluated', message: `evaluated (${doc.evaluated}) must equal covered (${doc.covered.length}) + uncovered (${doc.uncovered.length})` });

package/spec/schemas/baseline.schema.json

@@ -1,20 +1,28 @@
 {
   "$schema": "https://json-schema.org/draft/2020-12/schema",
-  "$id": "urn:guard-spec:v1:baseline",
-  "title": "Guard spec v1 — baseline",
+  "$id": "urn:claimspec:v1:baseline",
+  "title": "claimspec v1 — baseline",
   "description": "Frozen fingerprints of findings that existed when the baseline was taken. A finding whose fingerprint appears here is suppressed up to `count` occurrences; everything else is new and gates. Only non-passing verdicts are ever fingerprinted.",
   "type": "object",
   "required": ["schemaVersion", "tool", "createdAt", "fingerprints"],
   "properties": {
     "$schema": { "type": "string" },
     "schemaVersion": { "const": 1 },
-    "tool": { "$ref": "urn:guard-spec:v1:common#/$defs/tool" },
-    "createdAt": { "$ref": "urn:guard-spec:v1:common#/$defs/isoDateTime" },
-    "head": { "$ref": "urn:guard-spec:v1:common#/$defs/gitSha" },
+    "tool": { "$ref": "urn:claimspec:v1:common#/$defs/tool" },
+    "createdAt": { "$ref": "urn:claimspec:v1:common#/$defs/isoDateTime" },
+    "head": { "$ref": "urn:claimspec:v1:common#/$defs/gitSha" },
     "dirty": {
       "description": "Whether the working tree had uncommitted changes when the evidence behind this baseline was taken.",
       "type": "boolean"
     },
+    "snapshot": {
+      "description": "When the evidence behind this baseline was a working-tree snapshot: that throwaway commit. `head` is then the parent of the commit that will carry the tests; `baseline --restamp` moves `head` once a clean probe reproduces the same fingerprints.",
+      "$ref": "urn:claimspec:v1:common#/$defs/gitSha"
+    },
+    "restampedAt": {
+      "description": "Set when `head` was moved by `baseline --restamp` (same fingerprints, clean tree). `createdAt` keeps the original freeze time.",
+      "$ref": "urn:claimspec:v1:common#/$defs/isoDateTime"
+    },
     "fingerprints": {
       "type": "object",
       "propertyNames": { "pattern": "^[a-f0-9]{64}$" },

package/spec/schemas/brief.schema.json

@@ -1,16 +1,16 @@
 {
   "$schema": "https://json-schema.org/draft/2020-12/schema",
-  "$id": "urn:guard-spec:v1:brief",
-  "title": "Guard spec v1 — agent brief",
+  "$id": "urn:claimspec:v1:brief",
+  "title": "claimspec v1 — agent brief",
   "description": "What to tell an agent before it writes code. `text` is the rendered block for an additionalContext payload; `items` is the same content structured so other tools can compose briefs. Ranked, capped, and never a single score.",
   "type": "object",
   "required": ["schemaVersion", "tool", "generatedAt", "heading", "summary", "items", "text"],
   "properties": {
     "$schema": { "type": "string" },
     "schemaVersion": { "const": 1 },
-    "tool": { "$ref": "urn:guard-spec:v1:common#/$defs/tool" },
-    "generatedAt": { "$ref": "urn:guard-spec:v1:common#/$defs/isoDateTime" },
-    "head": { "$ref": "urn:guard-spec:v1:common#/$defs/gitSha" },
+    "tool": { "$ref": "urn:claimspec:v1:common#/$defs/tool" },
+    "generatedAt": { "$ref": "urn:claimspec:v1:common#/$defs/isoDateTime" },
+    "head": { "$ref": "urn:claimspec:v1:common#/$defs/gitSha" },
     "heading": {
       "description": "The markdown heading that opens `text`, e.g. `## TEST BLINDSPOT CONTEXT`.",
       "type": "string",
@@ -51,7 +51,7 @@
       "required": ["ref", "files"],
       "properties": {
         "ref": { "type": "string", "minLength": 1, "maxLength": 256 },
-        "files": { "type": "array", "minItems": 1, "items": { "$ref": "urn:guard-spec:v1:common#/$defs/uncoveredFile" } }
+        "files": { "type": "array", "minItems": 1, "items": { "$ref": "urn:claimspec:v1:common#/$defs/uncoveredFile" } }
       },
       "additionalProperties": false
     },
@@ -62,13 +62,13 @@
         "type": "object",
         "required": ["fingerprint", "claimId", "statement", "verdict", "severity", "isNew"],
         "properties": {
-          "fingerprint": { "$ref": "urn:guard-spec:v1:common#/$defs/sha256" },
-          "claimId": { "$ref": "urn:guard-spec:v1:common#/$defs/identifier" },
-          "subjectId": { "$ref": "urn:guard-spec:v1:common#/$defs/identifier" },
+          "fingerprint": { "$ref": "urn:claimspec:v1:common#/$defs/sha256" },
+          "claimId": { "$ref": "urn:claimspec:v1:common#/$defs/identifier" },
+          "subjectId": { "$ref": "urn:claimspec:v1:common#/$defs/identifier" },
           "statement": { "type": "string", "minLength": 1, "maxLength": 2000 },
-          "verdict": { "$ref": "urn:guard-spec:v1:common#/$defs/verdict" },
-          "severity": { "$ref": "urn:guard-spec:v1:common#/$defs/severity" },
-          "file": { "$ref": "urn:guard-spec:v1:common#/$defs/repoPath" },
+          "verdict": { "$ref": "urn:claimspec:v1:common#/$defs/verdict" },
+          "severity": { "$ref": "urn:claimspec:v1:common#/$defs/severity" },
+          "file": { "$ref": "urn:claimspec:v1:common#/$defs/repoPath" },
           "rank": { "type": "number" },
           "hint": {
             "description": "One line the agent can act on, e.g. which assertion shape left the gap.",

package/spec/schemas/calibration.schema.json

@@ -1,15 +1,15 @@
 {
   "$schema": "https://json-schema.org/draft/2020-12/schema",
-  "$id": "urn:guard-spec:v1:calibration",
-  "title": "Guard spec v1 — calibration",
+  "$id": "urn:claimspec:v1:calibration",
+  "title": "claimspec v1 — calibration",
   "description": "P(finding is real) per bucket, with a confidence interval and the sample size behind it. A label without n is a vibe; this is the format that stops it being one.",
   "type": "object",
   "required": ["schemaVersion", "tool", "computedAt", "method", "confidence", "bucketBy", "source", "buckets"],
   "properties": {
     "$schema": { "type": "string" },
     "schemaVersion": { "const": 1 },
-    "tool": { "$ref": "urn:guard-spec:v1:common#/$defs/tool" },
-    "computedAt": { "$ref": "urn:guard-spec:v1:common#/$defs/isoDateTime" },
+    "tool": { "$ref": "urn:claimspec:v1:common#/$defs/tool" },
+    "computedAt": { "$ref": "urn:claimspec:v1:common#/$defs/isoDateTime" },
     "method": {
       "description": "Interval method. `wilson` is the only method defined in v1.",
       "enum": ["wilson"]
@@ -17,7 +17,7 @@
     "confidence": { "type": "number", "exclusiveMinimum": 0, "exclusiveMaximum": 1 },
     "bucketBy": {
       "description": "The field findings are bucketed on, e.g. `faultClass` or `severity`.",
-      "$ref": "urn:guard-spec:v1:common#/$defs/identifier"
+      "$ref": "urn:claimspec:v1:common#/$defs/identifier"
     },
     "source": {
       "description": "Where the ground-truth labels came from.",