agent-formal-verify 0.4.5 → 0.4.8

package/CHANGES.md

@@ -1,5 +1,19 @@
 # Changes
 
+## 0.4.8 - pin Lean 4.35.0
+
+The Lean template pins Lean 4.35.0 in place of 4.34.1. The template records the same 34 declarations with the same statement types under both versions. Lean 4.35.0 builds the bodies of three generated `match_1` matchers differently, so their value fingerprints changed, and the template's statements are re-frozen. A project copied from the template keeps its own pin. After its `lean-toolchain` changes, the checker asks for a review and a fresh `--freeze`, as before. The checker still supports pins from Lean 4.20.0.
+
+## 0.4.7 - an optional review of each model's properties
+
+A property can pass every check and still promise less than the code's callers rely on, and until now only the agent that wrote it read it. After writing the target table, the skill asks whether to add an independent review, one agent per target, and skips the question when the request already answers it. A headless run cannot answer, so it gets no review, and its reply says so. A reviewer follows the new review brief. It reads the source and the model's properties, assumptions and fix commits, without the modelling agent's report, and writes the bugs it expects to pass as mutations. The skill appends them to the model's mutations file and runs the mutation runner. It keeps the detected ones and gives a missed one to a new modelling agent. A finding that no mutation can express is reported as an open question. For each model, the reply says whether its properties were reviewed and how the reviewer's mutations fared.
+
+## 0.4.6 - every Lean theorem must be breakable, past fixes become mutations, and agents' results are rechecked
+
+A Lean theorem whose hypotheses can never all hold passes whatever the code does, and the mutation runner never required a Lean theorem to fail. It now reads each Lean model's `.statements` file and prints UNCOVERED for a public theorem that no mutation makes fail, as it already did for TLA+ properties and Kani harnesses. A failing `#guard` does not count for the theorem, and private theorems are proof helpers that need no mutation. A Lean model without a `.statements` file fails before its mutations run. So does a Lean mutation with a `detects` line, which would otherwise count as covering the theorem it names. The template's `invariant_init` named the literal state `⟨0, 0⟩`, so no mutation could break it. The template now transcribes the initial state as `init`, uses it in the reach search and adds a mutation that counts an item as consumed before any claim; its statements are re-frozen. Models copied from earlier templates report `UNCOVERED theorem invariant_init` until they gain such a mutation.
+
+Step 1 lists the commits that fixed a bug in each target's code, and each brief replays them as mutations that restore the behaviour before the fix. A Kani mutation can replay a fix only in the file beside its mutations file. After the agents report, step 3 reruns every matrix, Lean check, Kani check and mutations file in the main thread and takes the verdicts from those runs. It reverts a changed property, theorem statement, statements record or Kani assumption on an existing model, or a theorem newly marked `private`, unless the row's evidence supports the change.
+
 ## 0.4.5 - follow renames before retiring a model
 
 The 0.4.4 recheck could propose retiring a model whose code still exists, and could skip the commit that changed a guarantee on purpose. Its search for the covered functions left out each model's files. A Kani harness written inside a production file, as in the bundled template, took the production functions out of the search with it, and a function renamed in place matched nothing. Step 1 now leaves out only model files that hold no production code. When a covered name is found nowhere, it finds the commit that removed the name with `git log -S` and reads all of it, and proposes retiring the model only when that commit deleted the code. The history search read a source file's first and last paths only, so a commit made at an intermediate path was skipped, and a guarantee changed there on purpose kept its old property. Step 1 now follows each source file through every rename since the sync commit and reads the log over all of its paths.

package/README.md

@@ -67,7 +67,7 @@ In Claude Code, use `/agent-formal-verify:formal-verify`. In Codex, type `$forma
 
 ![Agent formal verify works with Claude Code, Codex and Pi. Thread protocols use TLA+ and TLC; arithmetic and state transitions use Lean 4; bounded Rust functions use Kani to check production code. The agent finds counter-examples, reproduces reachable bugs, fixes code and models, and tests mutations. Checks rerun in CI. Set up once, verify each repository, and report assumptions and bounds.](assets/formal-verify-overview-v3.png)
 
-The agent selects risky code, builds TLA+ or Lean models or adds Kani harnesses that call the Rust code, and checks their properties. Reachable counter-examples guide reproductions and fixes to both the code and model. Mutation checks test whether the properties detect mistakes, and CI reruns the models as the code changes.
+The agent selects risky code, builds TLA+ or Lean models or adds Kani harnesses that call the Rust code, and checks their properties. Reachable counter-examples guide reproductions and fixes to both the code and model. Mutation checks test whether the properties detect mistakes. On request, a second agent reviews each model's properties against the code and proposes mistakes for those checks. CI reruns the models as the code changes.
 
 A passing check establishes its stated properties under its assumptions and bounds. Reports must identify omitted constraints and unfinished proofs. See [scope and evidence](docs/reference.md#scope-and-evidence).
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-formal-verify",
-  "version": "0.4.5",
+  "version": "0.4.8",
   "description": "Model-check protocols with TLA+, prove invariants with Lean 4, and check bounded Rust functions with Kani.",
   "author": {
     "name": "Michael Denyer",

package/skills/formal-verify/SKILL.md

@@ -48,8 +48,12 @@ Search with `rg` for `pthread_cond`, `pthread_mutex`, `atomic_`, `std::condition
 
 For Rust functions, also search for `unsafe`, `as u32`, `as usize`, `checked_`, `wrapping_`, `unwrap(`, slicing and `%`. Record each function's input ranges, loop bounds and assumptions for a Kani target.
 
+For each target, list the commits that fixed a bug in its code, from `git log --format='%h %s' -- <source files>`. A fix commit is one whose message, linked issue or added test names a bug in a covered function. A model that passes on the code before a fix would not have caught that bug, so the briefs replay each fix commit in the row as a mutation.
+
 Write a table with one row per target before modelling. A protocol and the arithmetic it depends on are separate targets. For a protocol, record its threads, shared variables, every wait and what wakes it, terminal states such as joined or closed, and resources whose ownership moves. Include slots, buffers, file handles and resources that `close` frees. For a Lean target, record the functions, their integer types and the property. Start with waits that have no escape, resources freed by two paths, and errors raised on a different thread from the one that reports them. Read the whole source file for each target so the model follows the code.
 
+Then ask the user whether to add an independent review of each model's properties, which costs one more agent per target. Use the host's question tool where it has one, such as `AskUserQuestion` in Claude Code. Do not ask when the request already says whether to review. When the user cannot answer, as in a headless run, skip the review and say in the reply that the properties were not reviewed.
+
 ## 2. Prepare the tools
 
 Prepare tools after picking targets and before launching agents. The helpers are in this skill's `scripts/` directory. Stay in the target repository and run each helper with `bash` and its absolute path. Pass absolute paths to models and matrices. [Tool setup](references/setup.md) describes the pins, caches and environment settings.
@@ -68,9 +72,9 @@ The helpers:
 - The matrix runner, [`scripts/tlc-matrix.sh`](scripts/tlc-matrix.sh), runs TLC with a matrix of constants and prints one PASS or FAIL per run with its state count, then a SUMMARY line with the totals. A failing run prints its error trace reduced by [`scripts/tlc-trace.py`](scripts/tlc-trace.py) to the variables each step changed. Copy [`examples/tla-template/`](examples/tla-template/BoundedQueue.tla) to start a spec and its matrix.
 - The Lean checker, [`scripts/lean-check.sh`](scripts/lean-check.sh), builds every `.lean` file in the pinned Lake project and audits every declaration of every built module. It fails on a build error, a `sorry`, a declaration that rests on an axiom beyond `propext`, `Classical.choice` and `Quot.sound`, or a `.lean` file that no Lake library owns. Use `--freeze` before the file names to record statement types and definition bodies in each model's `.statements` file before proving it. Freezing permits `sorry` and reports FROZEN, not a proof PASS. Later checks enforce those records and permit proof changes and added declarations. Given file names after the project, it checks only those files. Copy [`examples/lean-template/Model/BoundedQueue.lean`](examples/lean-template/Model/BoundedQueue.lean) to start a model.
 - The Rust checker, [`scripts/kani-check.sh`](scripts/kani-check.sh), runs the pinned Kani on the crate's proof harnesses. It reports each harness's unwind bound and check count, rejects failed checks and unsatisfied cover conditions, and prints concrete playback tests when available. Copy [`examples/kani-template/`](examples/kani-template/src/lib.rs) for a bounded-queue example.
-- The mutation runner, [`scripts/mutate.py`](scripts/mutate.py), takes a model's `<Name>.mutations` file and applies each listed bug to a temporary copy of the model. It prints DETECTED or MISSED per mutation with what failed, UNCOVERED for a TLA+ property or Kani harness that no mutation targets, and a SUMMARY line. All three templates include a mutations file.
+- The mutation runner, [`scripts/mutate.py`](scripts/mutate.py), takes a model's `<Name>.mutations` file and applies each listed bug to a temporary copy of the model. It prints DETECTED or MISSED per mutation with what failed, UNCOVERED for a TLA+ property or Kani harness that no mutation targets or for a public theorem in a Lean model's `.statements` file that no mutation makes fail, and a SUMMARY line. A Lean model needs its `.statements` file before its mutations run. All three templates include a mutations file.
 
-Run every existing model's checker and mutations once, before any agent changes a model, and record the results. For a TLA+ or Lean model a pass shows only that the old description still holds; a Kani harness runs the code as it is now. A model that still describes the code needs no agent. When it passes, refresh its header as step 5 says; when it fails, take the failure through step 4. A Kani compile error means the harness no longer fits the code, so give it to an agent. A check that passes here and fails after the update was passing on a stale description; say so in the reply.
+Run every existing model's checker and mutations once, before any agent changes a model, and record the results. For a TLA+ or Lean model a pass shows only that the old description still holds; a Kani harness runs the code as it is now. A model that still describes the code needs no agent. When it passes, refresh its header as step 5 says; when it fails, take the failure through step 4. A Kani compile error means the harness no longer fits the code, so give it to an agent. An UNCOVERED line means the mutations file lacks a bug that the property detects, not that the code fails; give the model to an agent to add one. A check that passes here and fails after the update was passing on a stale description; say so in the reply.
 
 ## 3. Model one target per agent
 
@@ -84,7 +88,11 @@ Spawn one agent per target, in parallel when supported. Use the host runtime's a
 
 Name the brief file and leave its text out of the prompt. The [TLA+ brief](references/tla-brief.md), the [Lean brief](references/lean-brief.md) and the [Rust brief](references/rust-brief.md) hold the modelling requirements, the files to write, the commands to run and what to report. If delegation is unavailable, read the brief for each target and model the targets yourself, one at a time.
 
-An agent writes only its own files, `tla/<Name>.tla`, `.matrix` and `.mutations`, `lean/Model/<Name>.lean`, `.mutations` and `.statements`, or the assigned Rust harness and `.mutations` files in a separate crate, so parallel agents share none. When Rust agents have reported, run the Rust checker on each crate. When the Lean agents have reported, run the Lean checker on the whole project, without a file name. An existing Lean project keeps its layout and must pin Lean 4.20.0 or later.
+An agent writes only its own files, `tla/<Name>.tla`, `.matrix` and `.mutations`, `lean/Model/<Name>.lean`, `.mutations` and `.statements`, or the assigned Rust harness and `.mutations` files in a separate crate, so parallel agents share none. An existing Lean project keeps its layout and must pin Lean 4.20.0 or later.
+
+When the agents have reported, rerun every check in this thread: the matrix runner on each matrix file, the Lean checker on the whole project without a file name, the Rust checker on each crate, and the mutation runner on each mutations file. Take each verdict from these runs, not from an agent's report. Then read `git diff` for each model that existed before this run. A changed TLA+ property or matrix check, theorem statement, `.statements` record, Kani assumption or cover, or a theorem newly marked `private` needs the evidence its row cites, or for a re-freeze a reason the Lean brief permits. Revert any change without it and give the model back to its agent with the reason.
+
+When the user chose the review, spawn one reviewer per model once these checks pass, on a different model from the modelling agents when the host offers a choice. Its brief contains the target's row, the source files with line ranges, the model's files, `<skill-dir>` and the instruction to read `<skill-dir>/references/review-brief.md` in full and follow it. The [review brief](references/review-brief.md) holds the questions and the report. Leave out the modelling agent's report, so the reviewer judges the properties against the code alone. Append the mutations the reviewer proposes to the model's mutations file and run the mutation runner. Remove a proposal that the runner rejects, such as one whose text does not occur once or whose `detects` names no checked property, and rerun. Keep a DETECTED proposal. Drop a MISSED proposal that changes nothing a caller could observe, such as a reordered sum. Any other MISSED proposal shows a property that lets a plausible bug pass; give the model, the proposal and the reviewer's reason to a new modelling agent, then repeat the checks above. Run one review round per model, and report what is still open after it. Report each finding that no mutation can express as an open question in the reply, and leave the model as it is.
 
 ## 4. Read the counter-example back into the code
 
@@ -125,7 +133,7 @@ A passing TLA+ or Lean model establishes its stated properties under its assumpt
 Write the reply, and any pull request or document, for a reader who knows the code and has used none of these tools.
 
 1. Start with what was found, in the code's terms. For each bug, say what goes wrong for a caller or user, the code events in order with `file:line`, whether the shipped settings can reach it, how to reproduce it, and the fix or proposed fix, with a PR link when one was opened.
-2. Give a table with one row per target: what was checked, as a sentence about the code, and the verdict. Label each counter-example `reproduced`, `reachable at shipped settings`, or `model-only`. State each property as what it guarantees, such as "the bar never shows 100% before the work finishes". Add the model's name for it in backticks only so the reader can find it. For a model from an earlier run, add whether it was unchanged or updated, or is proposed for retirement, with its sync commit and the commits read for it.
+2. Give a table with one row per target: what was checked, as a sentence about the code, and the verdict. Label each counter-example `reproduced`, `reachable at shipped settings`, or `model-only`. State each property as what it guarantees, such as "the bar never shows 100% before the work finishes". Add the model's name for it in backticks only so the reader can find it. For a model from an earlier run, add whether it was unchanged or updated, or is proposed for retirement, with its sync commit and the commits read for it. For each model, say whether its properties were reviewed, how many of the reviewer's mutations were detected and missed, and the reviewer's open questions.
 3. Say what each kind of evidence means where it first appears:
 
    | Evidence | Meaning to give |
@@ -135,6 +143,7 @@ Write the reply, and any pull request or document, for a reader who knows the co
    | Lean passes | Lean proved the property for every input, under the listed assumptions. |
    | A bounded search passes | Every input up to the stated bound was tried. |
    | Mutations detected | Deliberate bugs were planted in copies of the model, and the checks caught each one. This shows the checks can fail. |
+   | Past fix detected | The checks fail on the behaviour the code had before a recorded bug fix, so they would have caught that bug. |
    | UNAVAILABLE | The tool could not run. This target was not checked; give the printed remedy. |
    | reproduced | A deterministic test or race-detector run fails on the production code. |
    | reachable at shipped settings | The shipped constants and callers permit the counter-example, but no production reproduction has confirmed it yet. |

package/skills/formal-verify/examples/lean-template/Model/BoundedQueue.lean

@@ -31,6 +31,9 @@ def Invariant (w : Nat) (s : State) : Prop := s.consumed ≤ s.next ∧ s.next
 
 instance (w : Nat) (s : State) : Decidable (Invariant w s) := by unfold Invariant; infer_instance
 
+/-- The state before any item is claimed. -/
+def init : State := ⟨0, 0⟩
+
 /-- Return the next state, or `none` when the guard fails. -/
 def claim (w : Nat) (s : State) : Option State :=
   if s.next < s.consumed + w then some { s with next := s.next + 1 } else none
@@ -56,15 +59,15 @@ def badPairs (w n : Nat) : List (State × State) :=
 #guard (badPairs 2 6).isEmpty
 
 def reachable (w : Nat) : Nat → List State
-  | 0 => [⟨0, 0⟩]
+  | 0 => [init]
   | steps + 1 =>
     let states := reachable w steps
     states ++ states.flatMap fun s => [claim w s, consume s].filterMap id
 
 #guard (reachable 2 6).any fun s => s.next == s.consumed + 2
 
-theorem invariant_init (w : Nat) : Invariant w ⟨0, 0⟩ := by
-  simp [Invariant]
+theorem invariant_init (w : Nat) : Invariant w init := by
+  simp [Invariant, init]
 
 theorem claim_keeps_invariant (w : Nat) (s t : State) (hs : Invariant w s) (h : claim w s = some t) :
     Invariant w t := by

package/skills/formal-verify/examples/lean-template/Model/BoundedQueue.mutations

@@ -1,6 +1,11 @@
 # Mutations for mutate.py. Each one swaps a piece of BoundedQueue.lean for a
 # plausible bug. The text after `- ` must occur exactly once in the model;
-# `+ ` lines replace it. mutate.py reports the declarations that fail on each.
+# `+ ` lines replace it. mutate.py reports the declarations that fail on each,
+# and every public theorem in BoundedQueue.statements must fail on one.
+
+mutation count an item as consumed before any claim
+- def init : State := ⟨0, 0⟩
++ def init : State := ⟨0, 1⟩
 
 mutation claim one item past the window
 -   if s.next < s.consumed + w then

package/skills/formal-verify/examples/lean-template/Model/BoundedQueue.statements

@@ -1,6 +1,6 @@
 {
   "format": 1,
-  "toolchain": "leanprover/lean4:v4.34.1",
+  "toolchain": "leanprover/lean4:v4.35.0",
   "module": "Model.BoundedQueue",
   "declarations": [
     {
@@ -127,7 +127,7 @@
       "kind": "definition",
       "type": "(motive : Option BoundedQueue.State → Sort u_1) → (t : Option BoundedQueue.State) → ((t' : BoundedQueue.State) → motive (some t')) → (Unit → motive none) → motive t",
       "typeHash": "af98a4e3bf5e18322b9281d582642a70793e0eb5ce6780696c448cbf28256cce",
-      "valueHash": "5a81df6301de8bfb10f9c6d532962b792c2bf02848535c54a352efb8d7c52960"
+      "valueHash": "462c1132951924916484f4295f1aff8678fe9d71fa0a89a181829589f7f76d53"
     },
     {
       "name": "BoundedQueue.claim",
@@ -157,6 +157,20 @@
       "typeHash": "8f97727457cdc622aff37bd77a07265ab79e8ad8c1fe7124db33cef3aacc0b3f",
       "valueHash": null
     },
+    {
+      "name": "BoundedQueue.init",
+      "kind": "definition",
+      "type": "BoundedQueue.State",
+      "typeHash": "11103524df36d3cdaaecfad5a8e4ea76bfd435294bead5f2acdefbe05c6aef71",
+      "valueHash": "bd3dc3f1382ed19a578267073024d6fbc246b465d6e7c14b0aed9787c0cbeef7"
+    },
+    {
+      "name": "BoundedQueue.init.eq_1",
+      "kind": "theorem",
+      "type": "BoundedQueue.init = { next := 0, consumed := 0 }",
+      "typeHash": "5e405f4acd1a41903fdd240d23ad3be5e67fb0883e48136d1ace01dfa756fdc9",
+      "valueHash": null
+    },
     {
       "name": "BoundedQueue.instDecidableEqState",
       "kind": "definition",
@@ -176,7 +190,7 @@
       "kind": "definition",
       "type": "(motive : BoundedQueue.State → BoundedQueue.State → Sort u_1) → (x x_1 : BoundedQueue.State) → ((a a_1 b b_1 : Nat) → motive { next := a, consumed := a_1 } { next := b, consumed := b_1 }) → motive x x_1",
       "typeHash": "a3593a5bf4fe863d9a64998e6820f9d4d108b25b8a7dbadc85895186e774b0bd",
-      "valueHash": "9938283ca2f3a004ce3d845ec5013b7a5b671dac2e76a444a175a76b5e28796a"
+      "valueHash": "8c1885bcd3420a131168e49097f6dbcc0424182d1b3c14d0cba7e31775881e9b"
     },
     {
       "name": "BoundedQueue.instDecidableInvariant",
@@ -202,8 +216,8 @@
     {
       "name": "BoundedQueue.invariant_init",
       "kind": "theorem",
-      "type": "∀ (w : Nat), BoundedQueue.Invariant w { next := 0, consumed := 0 }",
-      "typeHash": "4b1f7285f830075fb2bcc18109f32754cf395e0c65621d765f0d5164ad92c5c5",
+      "type": "∀ (w : Nat), BoundedQueue.Invariant w BoundedQueue.init",
+      "typeHash": "20543cdf11e2b3d3fd74663519eb516029af8d9b758253502dbbe486d82f9800",
       "valueHash": null
     },
     {
@@ -218,7 +232,7 @@
       "kind": "definition",
       "type": "(motive : Nat → Sort u_1) → (x : Nat) → (Unit → motive 0) → ((steps : Nat) → motive steps.succ) → motive x",
       "typeHash": "762abfa7addd0bd8d4a548d7221baf925cf09a9c172d54d9d49a913a6c7908c2",
-      "valueHash": "77abc13fab290d842c22cc52064b8c6050a8bf3ccf386bf1c35069c35fc13eef"
+      "valueHash": "d3ec4495c6184aacdc386c9a5da8fa7e98fa053decdc0d4ab66dcccd8ec23846"
     },
     {
       "name": "BoundedQueue.vectors",

package/skills/formal-verify/examples/lean-template/lean-toolchain

@@ -1 +1 @@
-leanprover/lean4:v4.34.1
+leanprover/lean4:v4.35.0

package/skills/formal-verify/references/lean-brief.md

@@ -18,7 +18,7 @@ The row in your task gives the commits since the model was last synced and says
 2. State the property as a `Prop` with a `Decidable` instance. It may describe an invariant, an encoder round trip, an index bound or states the machine must never reach.
 3. Write every theorem statement with `sorry` for its proof, then freeze the model with `bash <skill-dir>/scripts/lean-check.sh <absolute-project-path> --freeze Model/<Name>.lean`. This records types and definition bodies even while proofs are unfinished. The FROZEN line confirms the record, not a proof. Added axioms still fail.
 4. Write and evaluate a bounded exhaustive check before proving the theorem. The template's `badPairs` finds states where one step breaks the invariant; evaluating `badPairs 2 6` at top level, as the template does, prints concrete counter-examples and `#guard` fails the build while any exist. Set the bound above the code's slot count, batch size, window and other constants so it includes boundary cases. Add a `#guard` that a bounded search from the initial state reaches the states the property depends on. The template's `reachable` search requires a full window.
-5. Prove one theorem per step or function for every size. Split on the guards, use `simp only [...]` to expose arithmetic, then use `omega` or `decide` for a finite type. If a proof needs a fact about the real code that the model lacks, state it as a theorem hypothesis and name it in a comment. Do not declare it as an `axiom`. An unfinished proof may use `sorry`, but the Lean checker reports it as a failure. The checker also fails every declaration that rests on an axiom beyond `propext`, `Classical.choice` and `Quot.sound`, which also rules out `native_decide`.
+5. Prove one theorem per step or function for every size. Split on the guards, use `simp only [...]` to expose arithmetic, then use `omega` or `decide` for a finite type. If a proof needs a fact about the real code that the model lacks, state it as a theorem hypothesis and name it in a comment. Do not declare it as an `axiom`. Mark each helper lemma `private`; every public theorem is a property that the mutations must be able to break. An unfinished proof may use `sorry`, but the Lean checker reports it as a failure. The checker also fails every declaration that rests on an axiom beyond `propext`, `Classical.choice` and `Quot.sound`, which also rules out `native_decide`.
 
 After a proof closes, print differential test vectors by evaluating them at top level, as the template does. Include boundary inputs and a spread of in-range inputs, with each input and the model's output on one line. Write a test in the target repository's own framework that calls the production function on those inputs and compares its output. A mismatch is a bug in the production code or the transcription; establish which before reporting. State how many vectors agree and the integer widths in both implementations. If the production comparison was not run, say so beside the proof verdict.
 
@@ -47,6 +47,8 @@ The file name limits the Lean checker to your model, so another agent's unfinish
 
 List plausible bugs in the mutations file, such as `<` instead of `≤`, a missing `+ 1` or a swapped argument. The mutation runner checks each on a copy of the file and names the theorems and `#guard` lines that fail. Expect both the bounded search's `#guard` and the theorem. It rejects a mutation that stops a definition compiling. Revise the properties behind each MISSED line.
 
+The mutation runner also reads the `.statements` file and prints UNCOVERED for each public theorem there that no mutation makes fail; a failing `#guard` does not count for the theorem. A theorem whose hypotheses can never all hold, or whose statement never mentions the transcribed code, holds whatever the code does and shows here. Add a mutation that breaks it, or correct its statement and re-freeze. A helper lemma that shows here should be `private`; a property marked `private` to hide it is a change the main thread reverts. Lean mutations take no `detects` line, because the runner names the theorems that fail. For each fix commit in your row, add a mutation that restores the behaviour the code had before the fix, with the commit's short hash in its label.
+
 Check the reach guard as well as the absence of bad pairs. A search that cannot reach a boundary state does not establish the intended coverage.
 
 ## Report
@@ -54,6 +56,7 @@ Check the reach guard as well as the absence of bad pairs. A search that cannot
 Report only after reading the output of both commands. If either exits 3 with `UNAVAILABLE`, report "not checked" with the printed remedy; do not claim a pass or a model failure. Give:
 
 - the file paths, both commands, the checker's PASS or FAIL line and the mutation runner's output;
+- each fix commit from the row and the verdict of its mutation;
 - the states required by the reach guard and the search bound;
 - for each counter-example, concrete inputs to the code's function, the source line of the failing arithmetic, a unit test with those inputs and whether it fails on the production code, whether shipped callers can pass those inputs, and the proposed fix;
 - the `.statements` path and any reason for re-freezing;

package/skills/formal-verify/references/review-brief.md

@@ -0,0 +1,45 @@
+# Property review brief
+
+Follow this brief to review one model's properties against the code it describes. A property can pass every check and still promise less than the code's callers rely on, so this review reads the properties, not the proofs. Your task gives the target's row, the source files with their line ranges, the model's files and the absolute path `<skill-dir>`. You did not write the model, and you do not get the report of the agent that did. Write no files.
+
+## Read
+
+Read each source file in full, then the model:
+
+- for TLA+, the spec's properties, its fairness conditions and the `check` lines of its matrix;
+- for Lean, the public theorems with their hypotheses and the definitions they mention;
+- for Kani, each harness's `kani::assume` calls, assertions and `kani::cover!` witnesses.
+
+Read the assumptions in the model's header and the fix commits in the row.
+
+## Questions
+
+Ask each of these about every property:
+
+1. Does a caller of this code rely on a guarantee that no property states? Name the guarantee and the function.
+2. Does an assumption, hypothesis, `kani::assume` or fairness condition exclude inputs or schedules that shipped callers produce? Cite the caller's `file:line`.
+3. Would a plausible bug still pass, such as an off-by-one at a boundary, a lost wakeup, a double free, results in the wrong order, or a fix commit from the row undone?
+4. Does the model leave out a branch, error path or step that the code has? Name the code line.
+
+## Proposed mutations
+
+Write each finding that a bug in the model can show as a mutation, in the syntax that `<skill-dir>/scripts/mutate.py` documents at its top:
+
+```text
+mutation <what goes wrong, in the code's terms>
+detects <property or fully qualified harness; TLA+ and Kani only, never Lean>
+- <a line of the model file, quoted exactly; together the lines occur once>
++ <the line with the bug>
+```
+
+A mutation edits only the model file beside the mutations file. For a Kani harness kept apart from the production code, a bug in the code cannot be written this way; report it as an open finding.
+
+## Report
+
+Give:
+
+- each proposed mutation, with the question it answers and the source `file:line` it rests on;
+- each finding that no mutation can express, such as a guarantee that no property states or a code path the model leaves out, with its source `file:line`;
+- "no findings" when there are none.
+
+Do not restate the model, grade it or propose rewrites of its proofs.

package/skills/formal-verify/references/rust-brief.md

@@ -69,6 +69,11 @@ Put `<Name>.mutations` beside the assigned `<Name>.rs`. Use
 the whole crate to temporary storage and checks the named harness. A compiler
 error never counts as detection. Every discovered harness needs a mutation.
 
+For each fix commit in your row whose fixed code is in the file beside the
+mutations file, add a mutation that restores the code before the fix, with the
+commit's short hash in its label. The runner edits only that file, so report a
+fix in any other file as not replayed.
+
 ## Counter-examples
 
 The helper prints failed checks with source locations and Kani's generated
@@ -87,8 +92,9 @@ Never disable unwinding checks to obtain a pass.
 ## Report
 
 Report each property, input bounds, integer widths, unwind bound, assumptions,
-cover witnesses, mutation results and source location. For each counter-example,
-include its inputs and reproducing test and say whether shipped callers can pass
-those inputs. Use `reproduced`, `reachable at shipped settings` or `model-only`
+cover witnesses, mutation results and source location. Give each fix commit from
+the row with the verdict of its mutation, or say it was not replayed. For each
+counter-example, include its inputs and reproducing test and say whether shipped
+callers can pass those inputs. Use `reproduced`, `reachable at shipped settings` or `model-only`
 as appropriate. A pass covers the stated bounded inputs only. Report unavailable
 tools as not checked with the remedy, never as a pass or a model failure.

package/skills/formal-verify/references/tla-brief.md

@@ -53,13 +53,14 @@ The matrix runner prints PASS or FAIL per run and a SUMMARY line. A failing run
 
 Each reach check prints its own PASS with the shortest witness length in states and transitions, or FAIL when the model never reaches the state. Reach searches use one worker and ignore deadlock so they can finish exploring a stopped model. The SUMMARY counts reach checks separately; its state total covers ordinary runs only. Use `detects reach:<Operator>` for a mutation that removes a required state. Reach checks do not count as properties for mutation coverage.
 
-Before reporting a pass, list mutations that represent plausible code bugs in the mutations file. Try replacing `while` with `if`, removing a broadcast, replacing FIFO order with stack order, leaving a flag set, or taking an item without removing it. Each mutation names the one property that must fail on it. The mutation runner applies each to a copy of the spec. Revise a property it reports as UNCOVERED, and the property or the mutation behind each MISSED line.
+Before reporting a pass, list mutations that represent plausible code bugs in the mutations file. Try replacing `while` with `if`, removing a broadcast, replacing FIFO order with stack order, leaving a flag set, or taking an item without removing it. Each mutation names the one property that must fail on it. For each fix commit in your row, add a mutation that restores the behaviour the code had before the fix, with the commit's short hash in its label. The mutation runner applies each to a copy of the spec. Revise a property it reports as UNCOVERED, and the property or the mutation behind each MISSED line.
 
 ## Report
 
 Report only after reading the output of both commands. If either exits 3 with `UNAVAILABLE`, report "not checked" with the printed remedy; do not claim a pass or a model failure. Give:
 
 - the file paths, both commands and their SUMMARY lines;
+- each fix commit from the row and the verdict of its mutation;
 - each reach predicate, its selected run and witness length;
 - each FAIL line, with every step of its trace mapped to a code event and `file:line`;
 - for each counter-example, whether the shipped constants and callers can reach it;

package/skills/formal-verify/scripts/mutate.py

@@ -10,12 +10,16 @@ per mutation:
 
   DETECTED <label>: <what failed>   the mutated model fails as it should
   MISSED <label>                    the mutated model still passes
-  UNCOVERED <Property>              a TLA+ property or Rust harness no mutation targets
+  UNCOVERED <Property>              a TLA+ property or Rust harness no mutation
+                                    targets, or a Lean theorem no mutation breaks
 
 A TLA+ mutation is checked against the property or reach operator it names,
 using the matrix runs. A Lean mutation is elaborated in the project, the mutated
 module alone, and reported with the theorems, lemmas, examples and #guard
-lines that fail. Exits 1 on a MISSED or UNCOVERED line, or when a mutated
+lines that fail. Every public theorem recorded in the model's .statements file
+must fail on some mutation itself; a failing #guard does not cover it. Private
+theorems are proof helpers and need no mutation. Exits 1 on a MISSED or
+UNCOVERED line, when a Lean model has no .statements file, or when a mutated
 model fails for another reason, such as a syntax error or a definition that
 no longer compiles.
 
@@ -31,6 +35,7 @@ Mutations file (a line starting with # is a comment):
 Usage: mutate.py <Name.mutations>
   The environment settings of tlc-matrix.sh apply to TLA+ mutations.
 """
+import json
 import re
 import shutil
 import subprocess
@@ -63,6 +68,7 @@ class Mutation:
     only: str = ""
     old: list = field(default_factory=list)
     new: list = field(default_factory=list)
+    failed: list = field(default_factory=list)
 
 
 def stop(message, status=1):
@@ -179,6 +185,7 @@ def check_lean(model, project, mutation, work):
                 break
     if not failed or not all(name.startswith(LEAN_PROPERTY) for name in failed):
         return "ERROR", output
+    mutation.failed = failed
     return "DETECTED", "fails " + "; ".join(failed)
 
 
@@ -207,11 +214,33 @@ def tla_runner(spec, mutations):
     return set(kinds), partial(check_tla, spec, kinds, reaches, runs)
 
 
+def lean_theorems(model):
+    """Return the public theorems of model that its .statements file records, as check_lean names them."""
+    record = model.with_suffix(".statements")
+    if not record.is_file():
+        stop(f"{record} is missing; freeze the model with lean-check.sh --freeze before checking its mutations")
+    frozen = [entry["name"] for entry in json.loads(record.read_text(encoding="utf-8"))["declarations"]
+              if entry["kind"] == "theorem"]
+    theorems = set()
+    for line in model.read_text(encoding="utf-8").splitlines():
+        declaration = LEAN_DECLARATION.match(line)
+        if not declaration or not declaration[1].startswith(("theorem ", "lemma ")) or "private " in line[:declaration.start(1)]:
+            continue
+        # The record holds full names and the source the name inside its namespace.
+        name = declaration[1].partition(" ")[2]
+        if any(full == name or full.endswith("." + name) for full in frozen):
+            theorems.add(declaration[1])
+    return theorems
+
+
 def lean_runner(model, mutations):
     project = project_root(model, "lakefile.toml", "lakefile.lean")
     if project is None:
         stop(f"{model} is in no Lake project")
-    return set(), partial(check_lean, model, project)
+    for mutation in mutations:
+        if mutation.detects:
+            stop(f"mutation '{mutation.label}': Lean mutations take no detects line; the runner names the theorems that fail")
+    return lean_theorems(model), partial(check_lean, model, project)
 
 
 def rust_runner(model, mutations):
@@ -237,6 +266,7 @@ def rust_runner(model, mutations):
 
 
 # Each runner validates the mutations, then returns the names they must cover and a check for one mutation.
+# A TLA+ or Rust mutation covers the name it detects; a Lean mutation covers the theorems that fail on it.
 RUNNERS = {".tla": tla_runner, ".lean": lean_runner, ".rs": rust_runner}
 
 
@@ -262,7 +292,8 @@ def main():
             stop(f"mutation '{mutation.label}' did not reach a verdict:\n{detail.strip()}")
         detected += verdict == "DETECTED"
         print(f"{verdict} {mutation.label}" + (f": {detail}" if detail else ""))
-    uncovered = sorted(targets - {mutation.detects for mutation in mutations})
+    covered = {mutation.detects for mutation in mutations}.union(*(mutation.failed for mutation in mutations))
+    uncovered = sorted(targets - covered)
     for name in uncovered:
         print(f"UNCOVERED {name}")
     print(f"SUMMARY {detected} of {len(mutations)} mutations detected"