physverify 0.7.0__tar.gz → 0.8.0__tar.gz

physverify-0.8.0/PKG-INFO

@@ -0,0 +1,132 @@
+Metadata-Version: 2.4
+Name: physverify
+Version: 0.8.0
+Summary: Verification checks for physics simulation code: manufactured solutions, order-of-accuracy studies, conserved quantities, limiting cases and exact patch tests.
+Author: Ram Chandra Giri
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/rcgiri-physics/physverify
+Project-URL: Documentation, https://rcgiri-physics.github.io/physverify/
+Project-URL: Issues, https://github.com/rcgiri-physics/physverify/issues
+Project-URL: Changelog, https://github.com/rcgiri-physics/physverify/blob/main/CHANGELOG.md
+Keywords: verification,method of manufactured solutions,numerical methods,physics,simulation,testing
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Physics
+Classifier: Framework :: Pytest
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Education
+Classifier: Topic :: Software Development :: Testing
+Classifier: Development Status :: 3 - Alpha
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24
+Requires-Dist: tomli>=1.1; python_version < "3.11"
+Provides-Extra: symbolic
+Requires-Dist: sympy>=1.12; extra == "symbolic"
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: pytest-cov>=6.1; extra == "dev"
+Requires-Dist: hypothesis>=6; extra == "dev"
+Requires-Dist: sympy>=1.12; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: mkdocs<2,>=1.6; extra == "docs"
+Requires-Dist: mkdocs-material<10,>=9.5; extra == "docs"
+Requires-Dist: mkdocstrings[python]<2,>=1.0.8; extra == "docs"
+Dynamic: license-file
+
+# physverify: confirm physics claims by running the code
+
+[![tests](https://github.com/rcgiri-physics/physverify/actions/workflows/ci.yml/badge.svg)](https://github.com/rcgiri-physics/physverify/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/physverify)](https://pypi.org/project/physverify/) [![Python](https://img.shields.io/pypi/pyversions/physverify)](https://pypi.org/project/physverify/) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.23534512.svg)](https://doi.org/10.5281/zenodo.23534512) [![Docs](https://img.shields.io/badge/docs-site-blue)](https://rcgiri-physics.github.io/physverify/)
+
+**A suspected bug in simulation code is a claim. physverify turns it into a run against a reference written from the physics, and gives a verdict with the numbers.**
+
+The claim can come from an AI code review, a colleague, a static check or your own doubt. Reading the code again rarely settles it; a run against an exact solution, a conservation law or a convergence order does. physverify supplies those checks for numerical ODE and PDE code (order of convergence, manufactured solutions, conservation, Runge-Kutta tables, exact references, a few Monte Carlo statistics), a sandbox for running code you do not trust, and plain-language verdicts. It needs only NumPy. Current release: 0.8.0 (v0.8.0); actively developed, contributions welcome.
+
+![A claim from a review becomes a run against a reference from the physics, and the run gives a verdict: confirmed, minor, rejected or open](https://raw.githubusercontent.com/rcgiri-physics/physverify/main/docs/images/claim-to-verdict.svg)
+
+## What confirming claims found on 100 repositories
+
+Five AI reviewers (three Claude models, Qwen 3.8 Flash and GPT 5.6 Terra) reviewed 100 public physics repositories with one fixed prompt. Fifty had a defect reported earlier; fifty had been tested and found clean. On the clean fifty, 119 review claims were drawn and each was given a check run in the sandbox:
+
+![Of 119 claims checked on repositories tested clean: 40 changed a result, 51 were real but minor, 10 were wrong, 18 stayed open](https://raw.githubusercontent.com/rcgiri-physics/physverify/main/docs/images/claim-outcomes.svg)
+
+- **40 claims were real defects that change a result:** 33 distinct defects in 19 of the 50 "clean" repositories. 32 have been reported to their maintainers.
+- **51 were real but minor** (invalid input, display, docstrings, code paths the repository never uses), **10 were wrong** and **18 could not be settled.**
+- Nine of the ten wrong claims came from one model. Where a reading was recorded before the run (20 claims), the run changed 3 verdicts.
+- Against the 67 defects reported earlier, the reviewers named 36 to 49; `physverify scan`, which reads code without running it, named 11.
+
+So AI review finds many real defects, including ones earlier testing missed, but a claim is not a finding until something runs. The counts hold for this sample only and are not a bug rate for physics code. The labels come from two automated checkers following written rules, not from people. Tables: `scripts/study100.py` regenerates them from the per-claim file, released with the preprint ([doi:10.5281/zenodo.24224433](https://doi.org/10.5281/zenodo.24224433)).
+
+## Confirmed by maintainers
+
+Some defects confirmed or fixed by the projects' own maintainers after a report:
+
+| Project | What was found | Outcome |
+|---|---|---|
+| [SymPy](https://github.com/sympy/sympy/issues/30810) | three units differ from their exact definitions | confirmed against primary sources |
+| [Astropy](https://github.com/astropy/astropy/issues/20681) | "surface tension" registered as J m² instead of J/m² | confirmed; fix in progress |
+| [PyGMT](https://github.com/GenericMappingTools/pygmt/issues/4960) | strings with a space treated as datetimes | fixed |
+| [EinsteinPy](https://github.com/einsteinpy/einsteinpy/issues/705) | de Sitter scale factor on a spatial coordinate | fixed |
+| [TESPy](https://github.com/oemof/tespy/issues/1127) | entropy pressure term with the wrong units | fixed |
+| [PyAstronomy](https://github.com/sczesla/PyAstronomy/issues/71) | two constants modules defaulted to different unit systems (CGS and SI), so `planck` and other functions were wrong | fixed |
+| [qmat](https://github.com/Parallel-in-Time/qmat/issues/23) | Runge-Kutta schemes labelled one order too high | fixed; now uses `physverify.butcher` in CI |
+
+[The full list](https://rcgiri-physics.github.io/physverify/confirmed/) has more projects and the details.
+
+## Try it
+
+```bash
+pip install physverify
+physverify demo
+```
+
+The demo checks a correct solver (all checks pass), then the same solver with the damping 10% too large (all fail):
+
+![Error against step size: the correct solver follows the slope-2 line, the broken one levels off](https://raw.githubusercontent.com/rcgiri-physics/physverify/main/docs/images/convergence.svg)
+
+New to this? [Start here](https://rcgiri-physics.github.io/physverify/start-here/) takes about ten minutes and includes a prompt to paste if you use an AI coding assistant.
+
+## Use it on your own code
+
+- **Suspected bugs from a review (yours or an AI's):** write each claim down, pick a reference that does not come from the code, and run the smallest test that tells the two answers apart. [Checking an AI code review](https://rcgiri-physics.github.io/physverify/ai-review/) walks through it with examples.
+- **An ODE solver:** `physverify init verify_solver.py --solver mysolver --order 4` writes a spec; fill in the equations (from the physics, not from your code) and run `physverify run verify_solver.py`.
+- **Errors you already have:** `check_order(hs, errors, expected=2)` gives PASS, FAIL or INCONCLUSIVE with a reason.
+- **A Runge-Kutta table:** `check_tableau(A, b, c, claimed=3)` from `physverify.butcher` checks order, stage order and stability from the coefficients alone.
+- **Code you have not read yet:** `physverify scan PATH` reads the source, runs nothing, and flags a fixed set of patterns (Runge-Kutta tables below their stated order, grid spacings that do not match their grid, near-miss constants, removed NumPy/SciPy features, undefined names, unit tables with the wrong dimension). A clean scan means only that those patterns were not found. `scan` is kept working but no longer gets new rules; the work now goes into confirming claims.
+
+Code from other people runs only in the [sandbox](https://github.com/rcgiri-physics/physverify/tree/main/sandbox) (no network, read-only repository).
+
+## Run it in CI
+
+A failed check fails the job, like a broken unit test ([CI guide](https://rcgiri-physics.github.io/physverify/ci/)):
+
+```yaml
+      - uses: rcgiri-physics/physverify@v0.8.0
+        with:
+          specs: tests/verify_*.py
+```
+
+[Getting started](https://rcgiri-physics.github.io/physverify/getting-started/) · [Recipes](https://rcgiri-physics.github.io/physverify/recipes/) · [What a PASS means](https://rcgiri-physics.github.io/physverify/assumptions/) · [Troubleshooting](https://rcgiri-physics.github.io/physverify/troubleshooting/) · [API](https://rcgiri-physics.github.io/physverify/api/)
+
+## Other evidence
+
+- **Seeded bugs:** of 37 bugs added one at a time to five small solvers, the checks flagged 35 (plain "compare with one answer" tests: 27), with no false alarms. The same person wrote the bugs and the checks, so this shows the tool works as designed, not how it does on other code. `python scripts/reproduce_all.py` regenerates the tables.
+- **Earlier corpus testing (October 2026):** of the first 74 defects reported to authors, 14 were found by a physverify check as such, 11 by running the code at all, and the rest by the reference in each repository's test script or by reading; the [ledger](https://github.com/rcgiri-physics/physverify/blob/main/results/corpus/LEDGER.md) lists them by ID. Details: [results](https://rcgiri-physics.github.io/physverify/results/).
+
+## Limits
+
+Works best for ODEs; PDEs only in 1D so far. Your code needs a way to add a source term, or a small wrapper that can itself be wrong. A PASS means these checks found nothing wrong; it does not prove the simulation correct, and it is only as independent as the reference you wrote: see [What a PASS means](https://rcgiri-physics.github.io/physverify/assumptions/).
+
+## Contributing, citing, licence
+
+Help is welcome: a trial report on your own solver, seeded bugs written by someone other than the author, adapters, documentation. See [CONTRIBUTING.md](https://github.com/rcgiri-physics/physverify/blob/main/CONTRIBUTING.md), the [code of conduct](https://github.com/rcgiri-physics/physverify/blob/main/CODE_OF_CONDUCT.md), [SUPPORT.md](https://github.com/rcgiri-physics/physverify/blob/main/SUPPORT.md), [SECURITY.md](https://github.com/rcgiri-physics/physverify/blob/main/SECURITY.md) on running untrusted code, and [AGENTS.md](https://github.com/rcgiri-physics/physverify/blob/main/AGENTS.md) for AI assistants. To cite, use [doi:10.5281/zenodo.23534512](https://doi.org/10.5281/zenodo.23534512) or [CITATION.cff](https://github.com/rcgiri-physics/physverify/blob/main/CITATION.cff). Changes: [CHANGELOG.md](https://github.com/rcgiri-physics/physverify/blob/main/CHANGELOG.md); plans: [ROADMAP.md](https://github.com/rcgiri-physics/physverify/blob/main/ROADMAP.md). The methods are standard and other tools do parts of this, some better (NodePy for Runge-Kutta analysis); see [related work and credits](https://rcgiri-physics.github.io/physverify/related-work/). MIT licence, copyright (c) 2026 Ram Chandra Giri.
+
+<!-- mcp-name: io.github.rcgiri-physics/physverify -->

physverify-0.8.0/README.md

@@ -0,0 +1,88 @@
+# physverify: confirm physics claims by running the code
+
+[![tests](https://github.com/rcgiri-physics/physverify/actions/workflows/ci.yml/badge.svg)](https://github.com/rcgiri-physics/physverify/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/physverify)](https://pypi.org/project/physverify/) [![Python](https://img.shields.io/pypi/pyversions/physverify)](https://pypi.org/project/physverify/) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.23534512.svg)](https://doi.org/10.5281/zenodo.23534512) [![Docs](https://img.shields.io/badge/docs-site-blue)](https://rcgiri-physics.github.io/physverify/)
+
+**A suspected bug in simulation code is a claim. physverify turns it into a run against a reference written from the physics, and gives a verdict with the numbers.**
+
+The claim can come from an AI code review, a colleague, a static check or your own doubt. Reading the code again rarely settles it; a run against an exact solution, a conservation law or a convergence order does. physverify supplies those checks for numerical ODE and PDE code (order of convergence, manufactured solutions, conservation, Runge-Kutta tables, exact references, a few Monte Carlo statistics), a sandbox for running code you do not trust, and plain-language verdicts. It needs only NumPy. Current release: 0.8.0 (v0.8.0); actively developed, contributions welcome.
+
+![A claim from a review becomes a run against a reference from the physics, and the run gives a verdict: confirmed, minor, rejected or open](https://raw.githubusercontent.com/rcgiri-physics/physverify/main/docs/images/claim-to-verdict.svg)
+
+## What confirming claims found on 100 repositories
+
+Five AI reviewers (three Claude models, Qwen 3.8 Flash and GPT 5.6 Terra) reviewed 100 public physics repositories with one fixed prompt. Fifty had a defect reported earlier; fifty had been tested and found clean. On the clean fifty, 119 review claims were drawn and each was given a check run in the sandbox:
+
+![Of 119 claims checked on repositories tested clean: 40 changed a result, 51 were real but minor, 10 were wrong, 18 stayed open](https://raw.githubusercontent.com/rcgiri-physics/physverify/main/docs/images/claim-outcomes.svg)
+
+- **40 claims were real defects that change a result:** 33 distinct defects in 19 of the 50 "clean" repositories. 32 have been reported to their maintainers.
+- **51 were real but minor** (invalid input, display, docstrings, code paths the repository never uses), **10 were wrong** and **18 could not be settled.**
+- Nine of the ten wrong claims came from one model. Where a reading was recorded before the run (20 claims), the run changed 3 verdicts.
+- Against the 67 defects reported earlier, the reviewers named 36 to 49; `physverify scan`, which reads code without running it, named 11.
+
+So AI review finds many real defects, including ones earlier testing missed, but a claim is not a finding until something runs. The counts hold for this sample only and are not a bug rate for physics code. The labels come from two automated checkers following written rules, not from people. Tables: `scripts/study100.py` regenerates them from the per-claim file, released with the preprint ([doi:10.5281/zenodo.24224433](https://doi.org/10.5281/zenodo.24224433)).
+
+## Confirmed by maintainers
+
+Some defects confirmed or fixed by the projects' own maintainers after a report:
+
+| Project | What was found | Outcome |
+|---|---|---|
+| [SymPy](https://github.com/sympy/sympy/issues/30810) | three units differ from their exact definitions | confirmed against primary sources |
+| [Astropy](https://github.com/astropy/astropy/issues/20681) | "surface tension" registered as J m² instead of J/m² | confirmed; fix in progress |
+| [PyGMT](https://github.com/GenericMappingTools/pygmt/issues/4960) | strings with a space treated as datetimes | fixed |
+| [EinsteinPy](https://github.com/einsteinpy/einsteinpy/issues/705) | de Sitter scale factor on a spatial coordinate | fixed |
+| [TESPy](https://github.com/oemof/tespy/issues/1127) | entropy pressure term with the wrong units | fixed |
+| [PyAstronomy](https://github.com/sczesla/PyAstronomy/issues/71) | two constants modules defaulted to different unit systems (CGS and SI), so `planck` and other functions were wrong | fixed |
+| [qmat](https://github.com/Parallel-in-Time/qmat/issues/23) | Runge-Kutta schemes labelled one order too high | fixed; now uses `physverify.butcher` in CI |
+
+[The full list](https://rcgiri-physics.github.io/physverify/confirmed/) has more projects and the details.
+
+## Try it
+
+```bash
+pip install physverify
+physverify demo
+```
+
+The demo checks a correct solver (all checks pass), then the same solver with the damping 10% too large (all fail):
+
+![Error against step size: the correct solver follows the slope-2 line, the broken one levels off](https://raw.githubusercontent.com/rcgiri-physics/physverify/main/docs/images/convergence.svg)
+
+New to this? [Start here](https://rcgiri-physics.github.io/physverify/start-here/) takes about ten minutes and includes a prompt to paste if you use an AI coding assistant.
+
+## Use it on your own code
+
+- **Suspected bugs from a review (yours or an AI's):** write each claim down, pick a reference that does not come from the code, and run the smallest test that tells the two answers apart. [Checking an AI code review](https://rcgiri-physics.github.io/physverify/ai-review/) walks through it with examples.
+- **An ODE solver:** `physverify init verify_solver.py --solver mysolver --order 4` writes a spec; fill in the equations (from the physics, not from your code) and run `physverify run verify_solver.py`.
+- **Errors you already have:** `check_order(hs, errors, expected=2)` gives PASS, FAIL or INCONCLUSIVE with a reason.
+- **A Runge-Kutta table:** `check_tableau(A, b, c, claimed=3)` from `physverify.butcher` checks order, stage order and stability from the coefficients alone.
+- **Code you have not read yet:** `physverify scan PATH` reads the source, runs nothing, and flags a fixed set of patterns (Runge-Kutta tables below their stated order, grid spacings that do not match their grid, near-miss constants, removed NumPy/SciPy features, undefined names, unit tables with the wrong dimension). A clean scan means only that those patterns were not found. `scan` is kept working but no longer gets new rules; the work now goes into confirming claims.
+
+Code from other people runs only in the [sandbox](https://github.com/rcgiri-physics/physverify/tree/main/sandbox) (no network, read-only repository).
+
+## Run it in CI
+
+A failed check fails the job, like a broken unit test ([CI guide](https://rcgiri-physics.github.io/physverify/ci/)):
+
+```yaml
+      - uses: rcgiri-physics/physverify@v0.8.0
+        with:
+          specs: tests/verify_*.py
+```
+
+[Getting started](https://rcgiri-physics.github.io/physverify/getting-started/) · [Recipes](https://rcgiri-physics.github.io/physverify/recipes/) · [What a PASS means](https://rcgiri-physics.github.io/physverify/assumptions/) · [Troubleshooting](https://rcgiri-physics.github.io/physverify/troubleshooting/) · [API](https://rcgiri-physics.github.io/physverify/api/)
+
+## Other evidence
+
+- **Seeded bugs:** of 37 bugs added one at a time to five small solvers, the checks flagged 35 (plain "compare with one answer" tests: 27), with no false alarms. The same person wrote the bugs and the checks, so this shows the tool works as designed, not how it does on other code. `python scripts/reproduce_all.py` regenerates the tables.
+- **Earlier corpus testing (October 2026):** of the first 74 defects reported to authors, 14 were found by a physverify check as such, 11 by running the code at all, and the rest by the reference in each repository's test script or by reading; the [ledger](https://github.com/rcgiri-physics/physverify/blob/main/results/corpus/LEDGER.md) lists them by ID. Details: [results](https://rcgiri-physics.github.io/physverify/results/).
+
+## Limits
+
+Works best for ODEs; PDEs only in 1D so far. Your code needs a way to add a source term, or a small wrapper that can itself be wrong. A PASS means these checks found nothing wrong; it does not prove the simulation correct, and it is only as independent as the reference you wrote: see [What a PASS means](https://rcgiri-physics.github.io/physverify/assumptions/).
+
+## Contributing, citing, licence
+
+Help is welcome: a trial report on your own solver, seeded bugs written by someone other than the author, adapters, documentation. See [CONTRIBUTING.md](https://github.com/rcgiri-physics/physverify/blob/main/CONTRIBUTING.md), the [code of conduct](https://github.com/rcgiri-physics/physverify/blob/main/CODE_OF_CONDUCT.md), [SUPPORT.md](https://github.com/rcgiri-physics/physverify/blob/main/SUPPORT.md), [SECURITY.md](https://github.com/rcgiri-physics/physverify/blob/main/SECURITY.md) on running untrusted code, and [AGENTS.md](https://github.com/rcgiri-physics/physverify/blob/main/AGENTS.md) for AI assistants. To cite, use [doi:10.5281/zenodo.23534512](https://doi.org/10.5281/zenodo.23534512) or [CITATION.cff](https://github.com/rcgiri-physics/physverify/blob/main/CITATION.cff). Changes: [CHANGELOG.md](https://github.com/rcgiri-physics/physverify/blob/main/CHANGELOG.md); plans: [ROADMAP.md](https://github.com/rcgiri-physics/physverify/blob/main/ROADMAP.md). The methods are standard and other tools do parts of this, some better (NodePy for Runge-Kutta analysis); see [related work and credits](https://rcgiri-physics.github.io/physverify/related-work/). MIT licence, copyright (c) 2026 Ram Chandra Giri.
+
+<!-- mcp-name: io.github.rcgiri-physics/physverify -->

{physverify-0.7.0 → physverify-0.8.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "physverify"
-version = "0.7.0"
-description = "Verification checks for physics simulation code: manufactured solutions, order-of-accuracy studies and a seeded-bug benchmark."
+version = "0.8.0"
+description = "Verification checks for physics simulation code: manufactured solutions, order-of-accuracy studies, conserved quantities, limiting cases and exact patch tests."
 readme = "README.md"
 requires-python = ">=3.10"
 license = "MIT"
@@ -38,8 +38,8 @@ Changelog = "https://github.com/rcgiri-physics/physverify/blob/main/CHANGELOG.md
 
 [project.optional-dependencies]
 symbolic = ["sympy>=1.12"]
-dev = ["pytest>=8", "pytest-cov>=6.1", "hypothesis>=6", "sympy>=1.12"]
-docs = ["mkdocs>=1.6,<2", "mkdocs-material>=9.5,<10"]
+dev = ["pytest>=8", "pytest-cov>=6.1", "hypothesis>=6", "sympy>=1.12", "ruff>=0.6"]
+docs = ["mkdocs>=1.6,<2", "mkdocs-material>=9.5,<10", "mkdocstrings[python]>=1.0.8,<2"]
 
 [project.scripts]
 physverify = "physverify.cli:main"
@@ -53,3 +53,22 @@ where = ["src"]
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 pythonpath = ["src", "benchmark"]
+
+[tool.ruff]
+extend-exclude = ["build"]
+
+[tool.ruff.lint]
+select = ["F", "B", "UP", "I"]
+# zip() truncating to the shorter input is relied on in places; strict= is a separate review
+ignore = ["B905"]
+
+[tool.ruff.lint.isort]
+known-first-party = ["physverify", "pvbench"]
+
+[tool.ruff.lint.per-file-ignores]
+# results/ cites line numbers in the seeded-bug problems, so their imports stay where they are
+"benchmark/pvbench/problems/*" = ["F401", "I001"]
+# the import itself is what this script checks
+"sandbox/probe.py" = ["F401", "I001"]
+# short teaching examples; the closure is called inside its loop
+"docs/examples/*" = ["B023"]

{physverify-0.7.0 → physverify-0.8.0}/src/physverify/__init__.py

@@ -2,11 +2,10 @@
 
 from .checks import Check, OrderResult, Result, SkipCheck, requires
 from .mms import ManufacturedODE, mms_order_study, zero_source
-from .order import check_order, fit_order, pairwise_orders
-from .order import check_exact
+from .order import check_exact, check_order, fit_order, pairwise_orders
 from .studies import invariant_study, ode_checks, patch_study, reference_study
 
-__version__ = "0.7.0"
+__version__ = "0.8.0"
 
 __all__ = [
     "Check",

{physverify-0.7.0 → physverify-0.8.0}/src/physverify/agent.py

@@ -10,8 +10,11 @@ Hook (`physverify hook`)
     assistant. Otherwise it exits 0 and prints nothing. See docs/agent.md.
 
 MCP server (`physverify mcp`)
-    Model Context Protocol over stdio (newline-delimited JSON-RPC 2.0), with one
-    tool, `run_checks`, that takes spec paths and returns the JSON report. It is
+    Model Context Protocol over stdio (newline-delimited JSON-RPC 2.0), with two
+    tools: `run_checks`, that takes spec paths and returns the JSON report, and
+    `confirm_claim`, that judges one claim file (docs/confirm.md). Both run the code
+    in a fresh Python process with a time limit (physverify.isolate), so an edit to
+    the solver between two calls is always seen. It is
     written with the standard library only, so it has no extra dependency; it
     implements the parts of the protocol a tool server needs, not the whole
     specification. It serves both protocol eras on the same process:
@@ -32,6 +35,7 @@ import json
 import sys
 from pathlib import Path
 
+RUN_TIMEOUT = 600.0  # seconds for one run_checks call in its own process
 SKIP_DIRS = {"venv", ".venv", "env", "node_modules", "build", "dist", "__pycache__", "site-packages"}
 MAX_REPORTED = 10
 
@@ -155,7 +159,7 @@ UNSUPPORTED_PROTOCOL_VERSION = -32022
 CACHE_HINTS = {"ttlMs": 300_000, "cacheScope": "public"}  # the tool list does not change while the server runs
 INSTRUCTIONS = (
     "Run physverify checks on physics simulation code after writing or editing it. "
-    "The tool executes the project's verify_*.py spec files (Python), so use it only on projects you trust."
+    "The tools execute the project's verify_*.py spec files and claim files (Python), so use them only on projects you trust."
 )
 
 TOOLS = [
@@ -173,9 +177,31 @@ TOOLS = [
             "properties": {
                 "specs": {"type": "array", "items": {"type": "string"}, "description": "spec file paths"},
                 "project": {"type": "string", "description": "project folder to search for verify_*.py"},
+                "timeout": {"type": "number", "description": f"seconds for the whole run (default {RUN_TIMEOUT:g})"},
             },
         },
-    }
+    },
+    {
+        "name": "confirm_claim",
+        "description": (
+            "Judge one suspected defect with physverify confirm. Takes a claim file that defines CLAIM, "
+            "control() and checks() with a reference written from the physics (docs/confirm.md), runs it "
+            "in a fresh process and returns the verdict: confirmed, rejected or not decidable, with the "
+            "reason and each check's status and message. Appends the result to a ledger file. Runs the "
+            "code, so it needs trusted=true or the sandbox (PHYSVERIFY_REPO set)."
+        ),
+        "inputSchema": {
+            "type": "object",
+            "properties": {
+                "claim": {"type": "string", "description": "path of the claim file"},
+                "trusted": {"type": "boolean", "description": "the code under test is trusted (required outside the sandbox)"},
+                "ledger": {"type": "string", "description": "ledger file (default confirm_ledger.jsonl)"},
+                "timeout": {"type": "number", "description": "seconds for the whole run (default 300)"},
+                "allow_dependent_reference": {"type": "string", "description": "reason, recorded, to run although the reference seems to use the code under test"},
+            },
+            "required": ["claim"],
+        },
+    },
 ]
 
 
@@ -193,12 +219,68 @@ def _run_checks_tool(arguments: dict) -> dict:
         specs = find_specs(Path(arguments.get("project") or "."))
     if not specs:
         return {"content": [{"type": "text", "text": "No spec files found (verify_*.py defining checks())."}], "isError": True}
-    report = run_specs(specs)
+    from .confirm import parse_timeout
+
+    try:
+        timeout = parse_timeout(RUN_TIMEOUT if arguments.get("timeout") is None else arguments["timeout"])
+    except ValueError as exc:
+        return {"content": [{"type": "text", "text": f"run_checks: {exc}"}], "isError": True}
+    report = run_specs_isolated(specs, timeout)
     failed = bool(report["summary"]["not_ok"] or report["load_errors"])
     text = (failure_summary(report) + "\n\n" if failed else "") + json.dumps(report, indent=2)
     return {"content": [{"type": "text", "text": text}], "isError": False}
 
 
+def run_specs_isolated(specs, timeout: float | None = RUN_TIMEOUT) -> dict:
+    """run_specs in a fresh Python process (physverify.isolate): the solver is imported as it is on disk now,
+    and a run longer than `timeout` seconds is stopped and reported as a load error of every spec."""
+    from .isolate import run_isolated
+    from .report import to_json
+
+    specs = [Path(s).resolve() for s in specs]
+    run = run_isolated("specs", specs, timeout)
+    if run["status"] == "done":
+        return run["result"]
+    why = (f"timed out after {timeout:g} s; the run was stopped" if run["status"] == "timeout" else run["error"])
+    if run["stderr_tail"]:
+        why += "; last output: " + run["stderr_tail"][-500:]
+    report = json.loads(to_json([], ", ".join(str(s) for s in specs)))
+    report["load_errors"] = [{"spec": str(s), "error": why} for s in specs]
+    return report
+
+
+def _confirm_claim_tool(arguments: dict) -> dict:
+    """confirm_claim: the same runner and verdict rules as `physverify confirm`."""
+    from .confirm import (
+        DEFAULT_LEDGER,
+        DEFAULT_TIMEOUT,
+        Refused,
+        confirm_claim,
+        format_verdict,
+        parse_timeout,
+    )
+
+    claim = arguments.get("claim")
+    if not isinstance(claim, str) or not claim:
+        return {"content": [{"type": "text", "text": "confirm_claim needs `claim`, the path of a claim file."}], "isError": True}
+    try:
+        timeout = parse_timeout(DEFAULT_TIMEOUT if arguments.get("timeout") is None else arguments["timeout"])
+    except ValueError as exc:
+        return {"content": [{"type": "text", "text": f"confirm_claim: {exc}"}], "isError": True}
+    try:
+        record = confirm_claim(claim, trusted=arguments.get("trusted") is True,
+                               ledger=arguments.get("ledger") or DEFAULT_LEDGER,
+                               timeout=timeout,
+                               allow_dependent_reference=arguments.get("allow_dependent_reference") or None)
+    except Refused as exc:
+        return {"content": [{"type": "text", "text": str(exc)}], "isError": True}
+    return {"content": [{"type": "text", "text": format_verdict(record) + "\n\n" + json.dumps(record, indent=2)}],
+            "isError": False}
+
+
+TOOL_HANDLERS = {"run_checks": _run_checks_tool, "confirm_claim": _confirm_claim_tool}
+
+
 def handle(message: dict):
     """Answer one JSON-RPC message; returns the response dict, or None for a notification."""
     from . import __version__
@@ -242,9 +324,10 @@ def handle(message: dict):
         if method == "tools/list":
             return modern({"tools": TOOLS}, cacheable=True)
         if method == "tools/call":
-            if params.get("name") != "run_checks":
+            tool = TOOL_HANDLERS.get(params.get("name"))
+            if tool is None:
                 return _error(msg_id, INVALID_PARAMS, f"unknown tool {params.get('name')!r}")
-            return modern(_run_checks_tool(params.get("arguments") or {}))
+            return modern(tool(params.get("arguments") or {}))
         return _error(msg_id, METHOD_NOT_FOUND, f"method not found: {method}")
 
     # Legacy era, after the handshake: results carry no resultType or cache hints.
@@ -253,9 +336,10 @@ def handle(message: dict):
     if method == "tools/list":
         return ok({"tools": TOOLS})
     if method == "tools/call":
-        if params.get("name") != "run_checks":
+        tool = TOOL_HANDLERS.get(params.get("name"))
+        if tool is None:
             return _error(msg_id, INVALID_PARAMS, f"unknown tool {params.get('name')!r}")
-        return ok(_run_checks_tool(params.get("arguments") or {}))
+        return ok(tool(params.get("arguments") or {}))
     return _error(msg_id, METHOD_NOT_FOUND, f"method not found: {method}")
 
 

{physverify-0.7.0 → physverify-0.8.0}/src/physverify/butcher.py

@@ -13,14 +13,14 @@ were slower to adjudicate than this check.
 
 from __future__ import annotations
 
-from functools import lru_cache
+from functools import cache
 
 import numpy as np
 
 from .checks import FAIL, PASS, Result
 
 
-@lru_cache(maxsize=None)
+@cache
 def trees(order: int) -> tuple:
     """All rooted trees with `order` vertices, each a sorted tuple of child subtrees."""
     if order == 1:

{physverify-0.7.0 → physverify-0.8.0}/src/physverify/checks.py

@@ -6,8 +6,8 @@ import contextvars
 import importlib
 import threading
 import time
+from collections.abc import Callable
 from dataclasses import dataclass
-from typing import Callable
 
 import numpy as np
 

{physverify-0.7.0 → physverify-0.8.0}/src/physverify/cli.py

@@ -5,6 +5,10 @@
     physverify init [SPEC.py] [--solver MODULE] [--function NAME] [--dim N] [--order P]
     physverify scan PATH   (reads the source, runs nothing: constants, RK tables, grid spacing, inverse pairs,
                            undefined names, removed APIs, typo-like literals, unit tables)
+    physverify confirm CLAIM.py [--ledger FILE] [--timeout S] [--json FILE|-] [--trusted]
+                           (judge one suspected defect; runs the code; exit 0 rejected, 1 confirmed,
+                           2 not decidable; see docs/confirm.md)
+    physverify confirm init CLAIM.py --text "..." --file F --line N
     physverify hook        (Claude Code PostToolUse hook; see docs/agent.md)
     physverify mcp         (MCP server on stdio; see docs/agent.md)
 
@@ -43,7 +47,15 @@ def expand_specs(patterns) -> list[str]:
 
 
 def _cmd_run(args) -> int:
-    from .report import NOTE, load_spec, run_checks, to_html, to_json, to_markdown, to_text
+    from .report import (
+        NOTE,
+        load_spec,
+        run_checks,
+        to_html,
+        to_json,
+        to_markdown,
+        to_text,
+    )
 
     try:
         args.specs = expand_specs(args.specs)
@@ -129,7 +141,7 @@ def _cmd_scan(args) -> int:
     import json
 
     from .probes import DEFAULT_PROBES, PROBES, probe
-    from .scan import STAGES, WARN, scan, to_dict
+    from .scan import STAGES, WARN, python_files, scan, to_dict
 
     known = STAGES + PROBES if args.run else STAGES
     stages = args.only.split(",") if args.only else (STAGES + DEFAULT_PROBES if args.run else STAGES)
@@ -142,12 +154,13 @@ def _cmd_scan(args) -> int:
         print(f"physverify: {args.path} does not exist", file=sys.stderr)
         return 2
     findings, unparsed = scan(args.path, stages=[s for s in stages if s in STAGES])
+    count = len(python_files(args.path))
     probes = [s for s in stages if s in PROBES]
     if probes:
         print("physverify: --run calls the code; use it only on code you trust or inside the sandbox", file=sys.stderr)
         findings += probe(args.path, probes=probes, timeout=args.timeout)
     if args.json:
-        text = json.dumps(to_dict(findings, unparsed), indent=1)
+        text = json.dumps({**to_dict(findings, unparsed), "files_scanned": count}, indent=1)
         if args.json == "-":
             print(text)
         else:
@@ -161,11 +174,65 @@ def _cmd_scan(args) -> int:
         skip = sum(f.level == "SKIP" for f in findings)
         ran = (f", {skip} skipped (not passed)" if probes else "")
         how = "the probes called the code with small inputs" if probes else "the scan reads the code and runs nothing"
-        print(f"{warn} warnings, {len(findings) - warn - skip} notes{ran}, {len(unparsed)} files not parsed"
+        broken = len(unparsed) + sum(f.stage == "names" and "does not parse" in f.message for f in findings)
+        if not count:
+            print(f"physverify: no Python files found in {args.path}; nothing was scanned")
+            return 0
+        print(f"{count} files scanned: {warn} warnings, {len(findings) - warn - skip} notes{ran}, {broken} files not parsed"
               f" ({how}; a clean scan means only that these patterns were not found)")
     return 1 if any(f.level == WARN for f in findings) else 0
 
 
+def _seconds(text) -> float:
+    """argparse type for a time limit: a finite number of seconds above 0."""
+    from .confirm import parse_timeout
+
+    try:
+        return parse_timeout(text)
+    except ValueError as exc:
+        raise argparse.ArgumentTypeError(str(exc)) from None
+
+
+def _cmd_confirm(args) -> int:
+    import json
+
+    from .confirm import (
+        EXIT_CODES,
+        Refused,
+        confirm_claim,
+        format_verdict,
+        write_template,
+    )
+
+    if args.claim == "init":
+        if not args.init_path or args.text is None or args.file is None or args.line is None:
+            print("physverify: usage: physverify confirm init CLAIM.py --text \"...\" --file F --line N", file=sys.stderr)
+            return 2
+        try:
+            path = write_template(args.init_path, args.text, args.file, args.line)
+        except (ValueError, OSError) as exc:
+            print(f"physverify: {exc}", file=sys.stderr)
+            return 2
+        print(f"wrote {path}; fill in source, reference and impact, and write control() and checks() from the physics")
+        return 0
+    if args.init_path:
+        print(f"physverify: unexpected argument {args.init_path!r} (one claim file per run)", file=sys.stderr)
+        return 2
+    try:
+        record = confirm_claim(args.claim, trusted=args.trusted, ledger=args.ledger, timeout=args.timeout,
+                               allow_dependent_reference=args.allow_dependent_reference)
+    except Refused as exc:
+        print(f"physverify: {exc}", file=sys.stderr)
+        return 2
+    if args.json == "-":
+        print(json.dumps(record, indent=2, ensure_ascii=False))
+    else:
+        if args.json:
+            Path(args.json).write_text(json.dumps(record, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
+        print(format_verdict(record))
+    return EXIT_CODES[record["verdict"]]
+
+
 def _cmd_references(args) -> int:
     from . import references
 
@@ -223,10 +290,25 @@ def build_parser() -> argparse.ArgumentParser:
     sc.add_argument("--timeout", type=float, default=60.0, help="with --run: seconds per script or file of functions (default 60)")
     sc.set_defaults(func=_cmd_scan)
 
+    cf = sub.add_parser("confirm", help="judge one suspected defect against a reference from the physics and record the verdict"
+                                        " (runs the code: --trusted or the sandbox only); see docs/confirm.md")
+    cf.add_argument("claim", help="the claim file (a .py file defining CLAIM, control() and checks()), or 'init'")
+    cf.add_argument("init_path", nargs="?", metavar="CLAIM.py", help="with init: the claim file to write")
+    cf.add_argument("--ledger", default="confirm_ledger.jsonl", help="JSON-lines file each run is appended to (default confirm_ledger.jsonl)")
+    cf.add_argument("--timeout", type=_seconds, default=300.0, help="seconds for the whole run before it is stopped (default 300)")
+    cf.add_argument("--json", metavar="FILE", help="write the ledger line as JSON to FILE, or to stdout with - (instead of the text verdict)")
+    cf.add_argument("--trusted", action="store_true", help="the code under test is trusted, so it may run outside the sandbox")
+    cf.add_argument("--allow-dependent-reference", metavar="REASON",
+                    help="run even though the reference seems to use the code under test; REASON is recorded")
+    cf.add_argument("--text", help="with init: the claim as stated")
+    cf.add_argument("--file", help="with init: the file the claim is about, relative to the repository root")
+    cf.add_argument("--line", type=int, help="with init: the line in that file")
+    cf.set_defaults(func=_cmd_confirm)
+
     r = sub.add_parser("references", help="list the exact reference solutions in physverify.references")
     r.set_defaults(func=_cmd_references)
 
-    s = sub.add_parser("mcp", help="run a minimal MCP server on stdio with a run_checks tool")
+    s = sub.add_parser("mcp", help="run a minimal MCP server on stdio with the tools run_checks and confirm_claim")
     s.set_defaults(func=_cmd_mcp)
     return parser