proof-harness 0.1.0-next.3 → 0.1.0-next.4

package/COMPATIBILITY.md

@@ -11,7 +11,7 @@ Consumers must pin an exact prerelease version, for example:
 ```json
 {
   "devDependencies": {
-    "proof-harness": "0.1.0-next.3"
+    "proof-harness": "0.1.0-next.4"
   }
 }
 ```
@@ -24,3 +24,18 @@ tighten validation or packaging contracts even when its base version remains
 The trace, mission, and health protocol versions are independent from the npm
 package version. Consumers should reject unsupported artifact or protocol
 versions rather than guessing how to interpret them.
+
+## Prerelease migration notes
+
+### 0.1.0-next.4
+
+No entry-point, artifact schema, or protocol changes. The `mutate` command
+tightens validation: each mutation's subject is now read back after apply and
+again after its proof, so a defect that never took effect fails as
+`NOT PLANTED` and one that was externally undone mid-run fails as
+`UN-PLANTED` — both distinct from a missed proof. A catalog mutation whose
+`apply` intentionally leaves its subject unchanged (it plants a side object
+that `cleanup` removes) must now declare `applyDoesNotChangeSubject: true`;
+runs with such undeclared mutations fail until the flag is added. The shared
+dev server is also started before the first defect is planted rather than
+inside the first mutation's window.

package/PROOF_SDK_CONTRACT.md

@@ -941,6 +941,18 @@ operation }` assertion has neither a matching mutation nor a valid entry in
   `.proof/mutation-policy.json`, pinned to the current spec hash and a passing
   positive control. A changed/deleted claim, changed spec, missing control, or
   newly added mutation makes the acceptance stale and fails inventory.
+- **A mutation that is not demonstrably live cannot judge a proof.** The
+  mutation runner never (re)applies migrations; it starts (or adopts) the
+  shared dev server before the first defect is planted, then reads each
+  mutation's subject back after apply and again after the proof. An apply that
+  left the subject unchanged fails as NOT PLANTED (a `REVOKE` no-op, or a
+  privilege still reachable via `PUBLIC` or an inherited role); a subject that
+  reverted mid-run — for example consumer tooling re-applying a wholesale
+  `REVOKE ALL; GRANT …` migration — fails as UN-PLANTED. Neither is reported
+  as a missed proof, because the proof never saw the defect. A mutation whose
+  apply intentionally leaves its subject untouched (it plants a side object
+  that `cleanup` removes) must declare `applyDoesNotChangeSubject: true` in
+  the catalog to opt out of the two subject comparisons explicitly.
 
 `pnpm proof:mutate` is the regression test for these guarantees: it re-opens known
 vulnerabilities in the local database one at a time and fails if the corresponding

package/README.md

@@ -17,7 +17,7 @@ Public prereleases are published under the npm `next` tag. Pin an exact
 prerelease version while the API is stabilizing:
 
 ```sh
-pnpm add proof-harness@0.1.0-next.3
+pnpm add proof-harness@0.1.0-next.4
 ```
 
 The package exposes environment-specific entry points:

package/cli/engines/proof_mutation_check.mjs

@@ -12,7 +12,10 @@
 //
 // Mutations are applied to the DATABASE, not the source tree: they are precise,
 // instant and reversible, where patching SQL files would require a full
-// migration cycle per mutation.
+// migration cycle per mutation. Migrations are never (re)applied by this
+// script; each planted defect is read back after apply and again after the
+// proof, so a plant that never took effect — or was externally undone
+// mid-run — fails as its own finding instead of being blamed on the proof.
 //
 // The mutation list is also the honest answer to "what does the baseline suite
 // actually detect?" — each entry names a finding, the hole it re-opens, and the
@@ -39,6 +42,10 @@ import {
   evaluateMutationClaimCoverage,
   mutationCoversClaim,
 } from "./proof_mutation_inventory.mjs";
+import {
+  assessPlantedSubject,
+  assessSubjectAfterProof,
+} from "./proof_plant_verification.mjs";
 import {
   applySourceMutation,
   restoreSourceMutation,
@@ -913,6 +920,15 @@ Exits non-zero if any proof failed to notice its vulnerability.`);
   let devServer = null;
 
   try {
+    // Start (or adopt) the shared server BEFORE the first defect is planted.
+    // Consumer `pnpm dev` bootstrap hooks — migration re-application, seeding,
+    // schema reconciliation — must run against the healthy database, not
+    // inside the first mutation's window, where a wholesale re-GRANT would
+    // silently un-plant it.
+    if (selected.some((mutation) => !isSourceMutation(mutation))) {
+      devServer = await sharedDevServer();
+    }
+
     for (const m of selected) {
       console.log(`── ${m.id} ──────────────────────────────────────────`);
       console.log(`   planting: ${m.breaks}`);
@@ -933,6 +949,19 @@ Exits non-zero if any proof failed to notice its vulnerability.`);
         applyMutation(container, m, snapshot);
         applied = true;
 
+        // A defect that is not demonstrably live proves nothing: running the
+        // proof anyway would blame the proof ("MISSED") for a plant that never
+        // happened. Read the subject back and refuse to continue on mismatch.
+        const plantedSnapshot = handler.snapshot(container, m.subject);
+        const plantProblem = assessPlantedSubject({
+          id: m.id,
+          description: handler.describe(m.subject),
+          before: snapshot,
+          afterApply: plantedSnapshot,
+          applyDoesNotChangeSubject: m.applyDoesNotChangeSubject === true,
+        });
+        if (plantProblem) throw new Error(plantProblem);
+
         if (isSourceMutation(m)) {
           if (devServer) {
             await devServer.stop();
@@ -947,6 +976,22 @@ Exits non-zero if any proof failed to notice its vulnerability.`);
         const { code, output } = await runProof(m.spec, m.id);
         const playwrightFailed = code !== 0;
 
+        // The defect must still be live now that the proof has finished. If
+        // the subject reverted mid-run (issue #7: a re-applied migration
+        // re-granting a revoked privilege), the proof's verdict — green or
+        // red — says nothing about the proof, and archiving it as MISSED
+        // would misdirect the operator toward a working proof.
+        const afterProofSnapshot = handler.snapshot(container, m.subject);
+        const liveProblem = assessSubjectAfterProof({
+          id: m.id,
+          description: handler.describe(m.subject),
+          before: snapshot,
+          afterApply: plantedSnapshot,
+          afterProof: afterProofSnapshot,
+          applyDoesNotChangeSubject: m.applyDoesNotChangeSubject === true,
+        });
+        if (liveProblem) throw new Error(liveProblem);
+
         // A proof can fail for the wrong reason (a crash, a timeout, a missing
         // browser). Where the reason matters — as with the deny-all mutation that
         // only a positive control can catch — require it in the output.

package/cli/engines/proof_plant_verification.mjs

@@ -0,0 +1,88 @@
+// ---------------------------------------------------------------------------
+// proof_plant_verification.mjs
+//
+// A mutation run is only evidence if the planted defect was actually live for
+// the entire proof. These checks compare the subject's snapshot at three
+// moments — before apply, after apply, and after the proof — and turn every
+// silent mismatch into a structured, correctly-attributed failure instead of
+// letting it masquerade as "MISSED — the proof cannot fail":
+//
+//   - apply changed nothing       → the defect never went live (NOT PLANTED)
+//   - subject reverted mid-proof  → something un-planted it   (UN-PLANTED)
+//   - subject drifted mid-proof   → the run cannot be attributed
+//
+// Without them, an ineffective REVOKE (the privilege still reachable via
+// PUBLIC or an inherited role, a grantor no-op, or tooling that re-applies
+// migrations mid-run) leaves the proof green against a healthy database, and
+// the summary blames the proof for a defect that was never live.
+//
+// A mutation whose apply intentionally leaves its subject untouched — it
+// plants a side object that `cleanup` removes, and the subject only anchors
+// restore — must declare `applyDoesNotChangeSubject: true` in the catalog.
+// Such mutations keep the historical blind spot by explicit, reviewable
+// choice rather than by accident.
+// ---------------------------------------------------------------------------
+
+/**
+ * Judge the subject immediately after `apply`. Returns an error message when
+ * the run must stop, or null when the plant is coherent.
+ */
+export function assessPlantedSubject({
+  id,
+  description,
+  before,
+  afterApply,
+  applyDoesNotChangeSubject = false,
+}) {
+  if (applyDoesNotChangeSubject) {
+    if (afterApply === before) return null;
+    return (
+      `mutation ${id} declares applyDoesNotChangeSubject, but applying it changed ` +
+      `${description}. The declaration is wrong; remove it so the subject is verified.\n` +
+      `  before apply: ${before}\n  after apply:  ${afterApply}`
+    );
+  }
+  if (afterApply !== before) return null;
+  return (
+    `mutation ${id} is NOT PLANTED: ${description} reads exactly the same before and after ` +
+    `apply, so the defect is not live and its proof would run against a healthy database. ` +
+    `Common causes: the privilege is still held via PUBLIC or an inherited role, the ` +
+    `GRANT/REVOKE was a silent no-op, or concurrent tooling immediately restored the subject. ` +
+    `If the apply intentionally leaves the subject untouched (it plants a side object that ` +
+    `cleanup removes), declare applyDoesNotChangeSubject: true on the mutation.\n` +
+    `  subject snapshot: ${before}`
+  );
+}
+
+/**
+ * Judge the subject after the proof ran, before restore. Returns an error
+ * message when the defect was no longer (or differently) planted, or null
+ * when the proof demonstrably ran against the planted defect.
+ */
+export function assessSubjectAfterProof({
+  id,
+  description,
+  before,
+  afterApply,
+  afterProof,
+  applyDoesNotChangeSubject = false,
+}) {
+  const expected = applyDoesNotChangeSubject ? before : afterApply;
+  if (afterProof === expected) return null;
+  if (afterProof === before) {
+    return (
+      `mutation ${id} was UN-PLANTED while its proof ran: ${description} returned to its ` +
+      `pre-mutation state before the proof finished, so the proof never had a chance to detect ` +
+      `the defect and its verdict is meaningless. Something outside the harness restored the ` +
+      `subject mid-run — typically dev-server bootstrap or migration tooling re-applying grants ` +
+      `(for example a wholesale "REVOKE ALL; GRANT ..." migration). This is a ` +
+      `harness/environment failure, not a missed proof.`
+    );
+  }
+  return (
+    `mutation ${id} lost its planted state while its proof ran: ${description} changed to a ` +
+    `state matching neither the planted defect nor the original, so the run cannot be ` +
+    `attributed to the mutation.\n` +
+    `  before apply: ${before}\n  after apply:  ${afterApply}\n  after proof:  ${afterProof}`
+  );
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "proof-harness",
-  "version": "0.1.0-next.3",
+  "version": "0.1.0-next.4",
   "description": "Verification and conformance helpers for proof traces, missions, fixtures, and Playwright probes.",
   "license": "Apache-2.0",
   "author": {