physverify 0.1.0__tar.gz → 0.2.0__tar.gz

{physverify-0.1.0/src/physverify.egg-info → physverify-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: physverify
-Version: 0.1.0
+Version: 0.2.0
 Summary: Verification checks for physics simulation code: manufactured solutions, order-of-accuracy studies and a seeded-bug benchmark.
 Author: Ram Chandra Giri
 License-Expression: MIT
@@ -39,7 +39,7 @@ Dynamic: license-file
 
 **A tool that checks whether a physics simulation is solving the right equations correctly, not just running without errors.**
 
-Status: early version (0.1.0). Nobody besides the author has used it yet, so feedback is very welcome.
+Status: early version (0.2.0). It has been tried on 20 public physics repositories (see below), but nobody besides the author has used it yet, so feedback is very welcome.
 
 ## The problem it addresses
 
@@ -54,6 +54,7 @@ physverify uses a few standard ideas from numerical analysis:
 3. **Check conserved quantities.** Energy, mass or momentum should stay constant (or drift in a predictable way).
 4. **Check known special cases.** For example, a projectile with no air resistance has a range you can write down by hand.
 5. **Exact "patch" tests.** Some solutions should be reproduced to computer precision. This catches small bugs that the other checks let through.
+6. **Algebraic checks on the method itself.** For a Runge-Kutta method, whether the coefficients really have the order they claim can be settled exactly, with no test problem (the "order conditions").
 
 Each check reports **PASS**, **FAIL**, **EXACT** or **INCONCLUSIVE**.
 
@@ -67,10 +68,21 @@ To test it, 37 bugs were added one at a time to five small solvers (projectile,
 - **0 false alarms** on the bug-free versions.
 - The 2 missed bugs are explained in the [technical details](https://github.com/rcgiri-physics/physverify/blob/main/docs/technical.md).
 
-For the Monte Carlo checks, all 20 seeded bugs were caught with no false alarms.
+For the Monte Carlo checks, all 21 seeded bugs were caught with no false alarms.
 
 The bugs and the checks were written by the same person, so these numbers show the tool works as designed. They are not a promise about bugs in other people's code.
 
+## Trying it on other people's code
+
+To see what it does on code it was not written for, 40 public Python physics repositories were drawn at random from GitHub searches (October 2026) and run in a locked-down container: no network, the repository read-only, nothing from it run on the author's computer.
+
+- **20 of the 40 could be tested.** Most of the others were not numerical solvers (neural networks, apps and games, analysis tools), one needed a recent NVIDIA GPU, and a few need more set-up.
+- **9 confirmed bugs in 6 of the 20.** For example: a measured energy that counts the magnetic-field term twice; backward time-stepping that is only first-order accurate; Runge-Kutta coefficient tables labelled with a higher order than they have; error-estimate weights that do not sum to zero; a spectral derivative that crashes for an odd number of grid points; a crash on NumPy 2.
+- **The other 14 had no confirmed bug.** Most passed every check that applied, often to 1e-13 or better; one raised a question for its authors rather than a bug report.
+- **Our own adapters were wrong about 15 times** before checking (a wrong constant, too-coarse steps, and so on). Every FAIL was checked by hand before being called a bug, and these mistakes are recorded, not hidden.
+
+Each lesson became a new check or a clearer message in physverify. The testing and write-ups were done with help from an AI assistant; every number comes from running the code. Authors are told about a finding before their project is named, so the repositories are listed by ID: [results/corpus/SUMMARY.md](https://github.com/rcgiri-physics/physverify/blob/main/results/corpus/SUMMARY.md).
+
 ## Try it
 
 You need Python 3.10 or newer.
@@ -108,6 +120,8 @@ pytest my_spec.py                              # each check becomes a pytest tes
 
 ## More
 
+- [results/corpus/SUMMARY.md](https://github.com/rcgiri-physics/physverify/blob/main/results/corpus/SUMMARY.md): the 40 public repositories, what was tested and what was found.
+- [sandbox/README.md](https://github.com/rcgiri-physics/physverify/blob/main/sandbox/README.md): the locked-down container used to run other people's code.
 - [results/own_work/SUMMARY.md](https://github.com/rcgiri-physics/physverify/blob/main/results/own_work/SUMMARY.md): results on four of the author's own public physics projects (two real bugs found and fixed).
 - [docs/technical.md](https://github.com/rcgiri-physics/physverify/blob/main/docs/technical.md): full details, benchmark tables, limits and related work.
 - [docs/patch.md](https://github.com/rcgiri-physics/physverify/blob/main/docs/patch.md): what the exact "patch" checks assume.

{physverify-0.1.0 → physverify-0.2.0}/README.md

@@ -2,7 +2,7 @@
 
 **A tool that checks whether a physics simulation is solving the right equations correctly, not just running without errors.**
 
-Status: early version (0.1.0). Nobody besides the author has used it yet, so feedback is very welcome.
+Status: early version (0.2.0). It has been tried on 20 public physics repositories (see below), but nobody besides the author has used it yet, so feedback is very welcome.
 
 ## The problem it addresses
 
@@ -17,6 +17,7 @@ physverify uses a few standard ideas from numerical analysis:
 3. **Check conserved quantities.** Energy, mass or momentum should stay constant (or drift in a predictable way).
 4. **Check known special cases.** For example, a projectile with no air resistance has a range you can write down by hand.
 5. **Exact "patch" tests.** Some solutions should be reproduced to computer precision. This catches small bugs that the other checks let through.
+6. **Algebraic checks on the method itself.** For a Runge-Kutta method, whether the coefficients really have the order they claim can be settled exactly, with no test problem (the "order conditions").
 
 Each check reports **PASS**, **FAIL**, **EXACT** or **INCONCLUSIVE**.
 
@@ -30,10 +31,21 @@ To test it, 37 bugs were added one at a time to five small solvers (projectile,
 - **0 false alarms** on the bug-free versions.
 - The 2 missed bugs are explained in the [technical details](https://github.com/rcgiri-physics/physverify/blob/main/docs/technical.md).
 
-For the Monte Carlo checks, all 20 seeded bugs were caught with no false alarms.
+For the Monte Carlo checks, all 21 seeded bugs were caught with no false alarms.
 
 The bugs and the checks were written by the same person, so these numbers show the tool works as designed. They are not a promise about bugs in other people's code.
 
+## Trying it on other people's code
+
+To see what it does on code it was not written for, 40 public Python physics repositories were drawn at random from GitHub searches (October 2026) and run in a locked-down container: no network, the repository read-only, nothing from it run on the author's computer.
+
+- **20 of the 40 could be tested.** Most of the others were not numerical solvers (neural networks, apps and games, analysis tools), one needed a recent NVIDIA GPU, and a few need more set-up.
+- **9 confirmed bugs in 6 of the 20.** For example: a measured energy that counts the magnetic-field term twice; backward time-stepping that is only first-order accurate; Runge-Kutta coefficient tables labelled with a higher order than they have; error-estimate weights that do not sum to zero; a spectral derivative that crashes for an odd number of grid points; a crash on NumPy 2.
+- **The other 14 had no confirmed bug.** Most passed every check that applied, often to 1e-13 or better; one raised a question for its authors rather than a bug report.
+- **Our own adapters were wrong about 15 times** before checking (a wrong constant, too-coarse steps, and so on). Every FAIL was checked by hand before being called a bug, and these mistakes are recorded, not hidden.
+
+Each lesson became a new check or a clearer message in physverify. The testing and write-ups were done with help from an AI assistant; every number comes from running the code. Authors are told about a finding before their project is named, so the repositories are listed by ID: [results/corpus/SUMMARY.md](https://github.com/rcgiri-physics/physverify/blob/main/results/corpus/SUMMARY.md).
+
 ## Try it
 
 You need Python 3.10 or newer.
@@ -71,6 +83,8 @@ pytest my_spec.py                              # each check becomes a pytest tes
 
 ## More
 
+- [results/corpus/SUMMARY.md](https://github.com/rcgiri-physics/physverify/blob/main/results/corpus/SUMMARY.md): the 40 public repositories, what was tested and what was found.
+- [sandbox/README.md](https://github.com/rcgiri-physics/physverify/blob/main/sandbox/README.md): the locked-down container used to run other people's code.
 - [results/own_work/SUMMARY.md](https://github.com/rcgiri-physics/physverify/blob/main/results/own_work/SUMMARY.md): results on four of the author's own public physics projects (two real bugs found and fixed).
 - [docs/technical.md](https://github.com/rcgiri-physics/physverify/blob/main/docs/technical.md): full details, benchmark tables, limits and related work.
 - [docs/patch.md](https://github.com/rcgiri-physics/physverify/blob/main/docs/patch.md): what the exact "patch" checks assume.

{physverify-0.1.0 → physverify-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "physverify"
-version = "0.1.0"
+version = "0.2.0"
 description = "Verification checks for physics simulation code: manufactured solutions, order-of-accuracy studies and a seeded-bug benchmark."
 readme = "README.md"
 requires-python = ">=3.10"

{physverify-0.1.0 → physverify-0.2.0}/src/physverify/__init__.py

@@ -6,7 +6,7 @@ from .order import check_order, fit_order, pairwise_orders
 from .order import check_exact
 from .studies import invariant_study, ode_checks, patch_study, reference_study
 
-__version__ = "0.1.0"
+__version__ = "0.2.0"
 
 __all__ = [
     "Check",

physverify-0.2.0/src/physverify/butcher.py

@@ -0,0 +1,111 @@
+"""Order of a Runge-Kutta method from its Butcher table, by the order conditions.
+
+A Runge-Kutta method (A, b, c) has order p exactly when b . Phi(t) = 1 / gamma(t) for every rooted
+tree t with at most p vertices (Butcher). This is algebra on the coefficients: it needs no test
+problem, cannot be fooled by a lucky cancellation or a superconvergent test equation, and works the
+same for explicit and implicit methods.
+
+Lessons from testing other people's code: one package's `order` labels were wrong for 3 of its
+tables (one claimed 4 where the table satisfies only the order-3 conditions), and another's embedded
+weights for Tsitouras 5(4) did not even sum to 1. Order studies on test equations found both, but
+were slower to adjudicate than this check.
+"""
+
+from __future__ import annotations
+
+from functools import lru_cache
+
+import numpy as np
+
+from .checks import FAIL, PASS, Result
+
+
+@lru_cache(maxsize=None)
+def trees(order: int) -> tuple:
+    """All rooted trees with `order` vertices, each a sorted tuple of child subtrees."""
+    if order == 1:
+        return ((),)
+    out = set()
+
+    def forests(n, max_child):
+        # multisets of trees with n vertices in total, children in non-increasing canonical order
+        if n == 0:
+            yield ()
+            return
+        for k in range(n, 0, -1):
+            for t in trees(k):
+                if max_child is not None and (k, t) > max_child:
+                    continue
+                for rest in forests(n - k, (k, t)):
+                    yield ((k, t),) + rest
+
+    for forest in forests(order - 1, None):
+        out.add(tuple(sorted((t for _, t in forest), key=repr)))
+    return tuple(sorted(out, key=repr))
+
+
+def _size(t) -> int:
+    return 1 + sum(_size(s) for s in t)
+
+
+def _gamma(t) -> int:
+    g = _size(t)
+    for s in t:
+        g *= _gamma(s)
+    return g
+
+
+def _phi(t, A, e):
+    v = e.copy()
+    for s in t:
+        v = v * (A @ _phi(s, A, e))
+    return v
+
+
+def order_residuals(A, b, max_order: int = 6) -> dict:
+    """{order: largest |b . Phi(t) - 1/gamma(t)| over the trees of that order}."""
+    A = np.asarray(A, dtype=float)
+    b = np.asarray(b, dtype=float)
+    e = np.ones(len(b))
+    return {p: max(abs(float(b @ _phi(t, A, e)) - 1.0 / _gamma(t)) for t in trees(p)) for p in range(1, max_order + 1)}
+
+
+def tableau_order(A, b, *, max_order: int = 6, tol: float = 1e-8) -> int:
+    """Highest p <= max_order such that every condition up to order p holds within tol."""
+    p = 0
+    for q, r in order_residuals(A, b, max_order).items():
+        if r > tol:
+            break
+        p = q
+    return p
+
+
+def check_tableau(A, b, c=None, *, claimed: int, label: str = "", tol: float = 1e-8, max_order: int | None = None) -> Result:
+    """PASS when the table has at least the claimed order; FAIL when it has less.
+
+    The default tolerance 1e-8 accepts coefficients typed with 9 or 10 digits (a real package had
+    residuals of 2e-10 for that reason), while a missing order shows residuals of 1e-4 or more.
+    The message gives the residuals, so a table whose conditions hold only to about 1e-10 is visible, and a claim that is lower than the true
+    order is noted rather than failed. If `c` is given, the row sums of A are compared with it.
+    """
+    top = max_order or claimed + 1
+    res = order_residuals(A, b, top)
+    p = tableau_order(A, b, max_order=top, tol=tol)
+    shown = ", ".join(f"{q}: {r:.1e}" for q, r in res.items())
+    prefix = f"{label}: " if label else ""
+    notes = []
+    if c is not None:
+        rs = float(np.max(np.abs(np.asarray(A, float).sum(axis=1) - np.asarray(c, float))))
+        if rs > tol:
+            notes.append(f"row sums of A differ from c by {rs:.1e}")
+    worst_kept = max((r for q, r in res.items() if q <= max(p, 1)), default=0.0)
+    if p >= 1 and worst_kept > 1e-13:
+        notes.append(f"conditions hold only to {worst_kept:.0e} (coefficients with limited digits?)")
+    tail = ("; " + "; ".join(notes)) if notes else ""
+    if p < claimed:
+        return Result(FAIL, f"{prefix}claims order {claimed} but satisfies the order conditions only up to {p} "
+                            f"(largest residual per order: {shown}){tail}")
+    if p > claimed:
+        return Result(PASS, f"{prefix}claims order {claimed}; the conditions hold up to at least {p}, so the claim is "
+                            f"conservative ({shown}){tail}")
+    return Result(PASS, f"{prefix}order {claimed} confirmed by the order conditions ({shown}){tail}")

physverify-0.2.0/src/physverify/constants.py

@@ -0,0 +1,121 @@
+"""Find numeric literals in source code that are close to, but not equal to, a physical constant.
+
+Lessons from testing other people's code: a gravitational constant in au^3 / (Msun yr^2) of
+39.4769 (Gaussian, 365.25-day year) next to an orbital speed of 2 pi (which assumes 4 pi^2 = 39.4784),
+g = 9 in a projectile game, pi typed as 3.14. None of these is necessarily a bug, but each is worth
+one look, and an adapter's reference must use the code's value, not the textbook one.
+
+`scan(path)` returns notes, never failures: the code may be right to round.
+Several CODATA editions are accepted. When writing a reference, use the edition the code says it uses:
+one adapter failed at 1.35e-9 by taking the 2022 proton mass where the code stated 2018.
+"""
+
+from __future__ import annotations
+
+import ast
+import math
+from dataclasses import dataclass
+from pathlib import Path
+
+# name: (accepted values, unit, comment)
+KNOWN = {
+    "pi": ((math.pi,), "", "use math.pi or numpy.pi"),
+    "e (Euler)": ((math.e,), "", "use math.e or numpy.e"),
+    "g": ((9.80665, 9.81, 9.8), "m/s^2", "standard gravity 9.80665"),
+    "G (SI)": ((6.67430e-11, 6.674e-11, 6.67e-11), "m^3 kg^-1 s^-2", "CODATA 2018: 6.67430e-11"),
+    "G M_sun / 4 pi^2 (au, yr)": ((4 * math.pi**2,), "au^3 Msun^-1 yr^-2",
+                                  "4 pi^2 is exact for a 1 au, 1 yr circular orbit; 39.4769 is the Gaussian value for a 365.25-day year"),
+    "c": ((299792458.0, 3e8, 2.998e8), "m/s", "exact: 299792458"),
+    "h (Planck)": ((6.62607015e-34,), "J s", "exact: 6.62607015e-34"),
+    "hbar": ((1.054571817e-34,), "J s", "1.054571817e-34"),
+    "k_B": ((1.380649e-23,), "J/K", "exact: 1.380649e-23"),
+    "e (charge)": ((1.602176634e-19,), "C", "exact: 1.602176634e-19"),
+    "m_e": ((9.1093837015e-31, 9.1093837139e-31), "kg", "CODATA 2018: 9.1093837015e-31, 2022: 9.1093837139e-31"),
+    "m_p": ((1.67262192369e-27, 1.67262192595e-27), "kg", "CODATA 2018: 1.67262192369e-27, 2022: 1.67262192595e-27"),
+    "epsilon_0": ((8.8541878128e-12,), "F/m", "8.8541878128e-12"),
+    "T_c (2D Ising)": ((2 / math.log(1 + math.sqrt(2)),), "J/k_B", "2 / ln(1 + sqrt 2) = 2.269185314"),
+}
+
+
+@dataclass
+class Note:
+    line: int
+    literal: str
+    constant: str
+    nearest: float
+    relative: float
+    comment: str
+
+    def __str__(self):
+        return (f"line {self.line}: {self.literal} is {self.relative:+.2g} relative to {self.constant} "
+                f"({self.nearest:.10g}); {self.comment}")
+
+
+def _digits(x: float) -> int:
+    """Significant digits of a literal as written (3.14 -> 3)."""
+    s = repr(x).lower().split("e")[0].replace("-", "").replace(".", "").lstrip("0")
+    return max(len(s.rstrip("0")), 1)
+
+
+def check_value(value: float, *, max_rel: float = 0.1):
+    """(constant, nearest accepted value, relative difference) for a near miss, else None.
+
+    A value equal to an accepted value, or equal to it to the precision written with
+    at least 4 digits, is not a near miss: 3.1416 for pi and 9.81 for g are fine;
+    3.14 and 9 are noted.
+    """
+    if value == 0 or not math.isfinite(value):
+        return None
+    best = None
+    for name, (accepted, _unit, comment) in KNOWN.items():
+        for ref in accepted:
+            rel = value / ref - 1
+            if abs(rel) > max_rel:
+                continue
+            if value == ref:
+                return None
+            written = 0.5 * 10 ** (1 - _digits(value))  # half a unit in the last written digit, relative
+            if _digits(value) >= 4 and abs(rel) <= written * 1.0000001:
+                return None  # matches to the precision written, with at least 4 digits
+            if best is None or abs(rel) < abs(best[2]):
+                best = (name, ref, rel, comment)
+    return best
+
+
+# names under which a short literal (an integer, or fewer than 3 digits) is still taken to mean the constant
+NAMES = {"g": {"g", "grav", "gravity", "g_acc", "gacc"}, "G (SI)": {"G"}, "pi": {"pi", "PI"}, "c": {"c"}}
+
+
+def _assigned_names(tree):
+    """id(value node) -> names it is assigned to, for simple `name = literal` statements."""
+    out = {}
+    for node in ast.walk(tree):
+        if isinstance(node, ast.Assign):
+            out[id(node.value)] = {t.id for t in node.targets if isinstance(t, ast.Name)}
+        elif isinstance(node, ast.AnnAssign) and node.value is not None and isinstance(node.target, ast.Name):
+            out[id(node.value)] = {node.target.id}
+    return out
+
+
+def scan(path, *, max_rel: float = 0.1) -> list[Note]:
+    """Near-miss constants among the numeric literals of a Python file.
+
+    Literals with 3 or more significant digits are compared with every constant.
+    Integers and shorter literals (9, 3.1, 10) are so common that they are noted
+    only when assigned to a name that suggests the constant (g = 9).
+    """
+    path = Path(path)
+    tree = ast.parse(path.read_text(encoding="utf-8"), str(path))
+    names = _assigned_names(tree)
+    notes = []
+    for node in ast.walk(tree):
+        if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)) and not isinstance(node.value, bool):
+            v = float(node.value)
+            hit = check_value(abs(v), max_rel=max_rel)
+            if hit and (isinstance(node.value, int) or _digits(v) < 3):
+                if not names.get(id(node), set()) & NAMES.get(hit[0], set()):
+                    hit = None
+            if hit:
+                name, ref, rel, comment = hit
+                notes.append(Note(node.lineno, ast.unparse(node), name, ref, rel, comment))
+    return sorted(notes, key=lambda n: n.line)

physverify-0.2.0/src/physverify/loaders.py

@@ -0,0 +1,94 @@
+"""Import a module from a path without touching its folder.
+
+Used by the adapters that test other people's code. Nothing is written next to
+the module (no __pycache__), and optional plotting imports can be stubbed so a
+simulation file can be imported on a machine without matplotlib.
+"""
+
+from __future__ import annotations
+
+import ast
+import importlib
+import importlib.util
+import sys
+import types
+from pathlib import Path
+
+
+def _stub_module(name: str) -> types.ModuleType:
+    return types.ModuleType(name)
+
+
+def load_module_from_path(name: str, path, stubs=()):
+    path = Path(path)
+    if not path.is_file():
+        raise FileNotFoundError(path)
+
+    added = []
+    for stub in stubs:
+        try:
+            importlib.import_module(stub)
+        except ImportError:
+            sys.modules[stub] = _stub_module(stub)
+            added.append(stub)
+
+    old_flag = sys.dont_write_bytecode
+    sys.dont_write_bytecode = True
+    try:
+        spec = importlib.util.spec_from_file_location(name, path)
+        module = importlib.util.module_from_spec(spec)
+        sys.modules[name] = module
+        try:
+            spec.loader.exec_module(module)
+        except BaseException:
+            sys.modules.pop(name, None)  # do not leave a half-initialised module behind
+            raise
+    finally:
+        sys.dont_write_bytecode = old_flag
+        for stub in added:
+            sys.modules.pop(stub, None)
+    return module
+
+
+def load_definitions_from_path(name: str, path, *, keep=(), inner=(), namespace=None, stubs=()):
+    """Load only the definitions of a script whose top level runs a simulation.
+
+    Many research scripts parse argv, open a window or run a long simulation at
+    import. This executes only the top-level imports, functions and classes, and
+    the top-level assignments to the names in `keep` (constants the functions
+    use). `inner` names functions defined inside other functions (for example
+    marimo or Jupyter-exported cells); they are lifted to module level, and
+    their imports must then come from `namespace`.
+
+    Nothing else in the file runs, so a function that relies on a global set by
+    the skipped code fails with NameError; set it on the returned module.
+
+    `stubs` names modules to replace with empty ones when they are not installed
+    (pygame, IPython, orbax, ...), for imports the tested functions do not use.
+    """
+    path = Path(path)
+    tree = ast.parse(path.read_text(encoding="utf-8"), str(path))
+    body = []
+    for node in tree.body:
+        if isinstance(node, (ast.Import, ast.ImportFrom, ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
+            body.append(node)
+        elif isinstance(node, (ast.Assign, ast.AnnAssign)):
+            targets = node.targets if isinstance(node, ast.Assign) else [node.target]
+            if any(isinstance(t, ast.Name) and t.id in keep for t in targets):
+                body.append(node)
+    if inner:
+        wanted = set(inner)
+        body += [n for n in ast.walk(tree) if isinstance(n, ast.FunctionDef) and n.name in wanted and n not in tree.body]
+    for stub in stubs:
+        try:
+            importlib.import_module(stub)
+        except ImportError:
+            parent, _, child = stub.rpartition(".")
+            sys.modules[stub] = _stub_module(stub)
+            if parent in sys.modules:
+                setattr(sys.modules[parent], child, sys.modules[stub])
+    module = types.ModuleType(name)
+    module.__file__ = str(path)
+    module.__dict__.update(namespace or {})
+    exec(compile(ast.Module(body=body, type_ignores=[]), str(path), "exec"), module.__dict__)
+    return module

physverify-0.2.0/src/physverify/method_bugs.py

@@ -0,0 +1,151 @@
+"""Seeded bugs in numerical methods themselves, modelled on bugs found in public code.
+
+The problem benchmark (physverify.problems) seeds bugs into five solvers. Some bugs found in real
+repositories are not bugs of a solver for one problem but of a method or a helper: a Butcher table with
+the wrong order label, error-estimate weights that do not sum to zero, backward time-stepping with
+stage times on the wrong side, a time loop that stops one step late, a spectral derivative that only
+works for even grids. This module seeds each of them, with a bug-free control, and names the check
+meant to catch it. `run_all()` returns one row per case; tests/test_method_bugs.py requires every
+prediction to come true.
+
+Cases and the real codes they come from are listed in results/corpus/SUMMARY.md (by repository ID).
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Callable
+
+import numpy as np
+
+from .butcher import check_tableau
+from .checks import FAIL, PASS, Result, run_check, Check
+from .mms import ManufacturedODE
+from .studies import backward_mms_study, size_sweep_check, time_grid_check
+
+# --- Butcher tables ---------------------------------------------------------------------------------
+
+HEUN_A = np.array([[0.0, 0.0], [1.0, 0.0]])
+HEUN_B = np.array([0.5, 0.5])
+RK4_A = np.array([[0, 0, 0, 0], [0.5, 0, 0, 0], [0, 0.5, 0, 0], [0, 0, 1, 0]], dtype=float)
+RK4_B = np.array([1, 2, 2, 1]) / 6.0
+# Bogacki-Shampine 3(2): main weights order 3, embedded order 2 (b_hat), error weights e = b - b_hat
+BS_A = np.array([[0, 0, 0, 0], [0.5, 0, 0, 0], [0, 0.75, 0, 0], [2 / 9, 1 / 3, 4 / 9, 0]])
+BS_B = np.array([2 / 9, 1 / 3, 4 / 9, 0])
+BS_BHAT = np.array([7 / 24, 1 / 4, 1 / 3, 1 / 8])
+
+
+def _tableau_case(A, b, claimed):
+    return lambda: check_tableau(A, b, claimed=claimed)
+
+
+# --- backward time-stepping -------------------------------------------------------------------------
+
+MMS = ManufacturedODE(lambda t: np.array([np.exp(np.sin(t))]), lambda t, y: np.array([y[0] * np.cos(t)]))
+
+
+def _rk4_back(stage_times_forward: bool):
+    """RK4 from t_end down to 0. With the bug the stages are evaluated at t + h/2, t + h (the forward side)."""
+    def integrate_back(source, y_end, t_end, n):
+        h = t_end / n
+        y, t = np.array(y_end, dtype=float), t_end
+
+        def f(s, u):
+            return MMS.rhs_ref(s, u) + source(s)
+
+        for _ in range(n):
+            sgn = 1.0 if stage_times_forward else -1.0
+            k1 = -f(t, y)
+            k2 = -f(t + sgn * h / 2, y + h / 2 * k1)
+            k3 = -f(t + sgn * h / 2, y + h / 2 * k2)
+            k4 = -f(t + sgn * h, y + h * k3)
+            y = y + h / 6 * (k1 + 2 * k2 + 2 * k3 + k4)
+            t -= h
+        return y
+    return integrate_back
+
+
+# --- time loops -------------------------------------------------------------------------------------
+
+def _while_loop_times(n, t0=0.0, t1=2.0):
+    h = (t1 - t0) / n
+    t, times = t0, [t0]
+    while t <= t1:
+        t += h
+        times.append(t)
+    return times
+
+
+def _counted_times(n, t0=0.0, t1=2.0):
+    h = (t1 - t0) / n
+    return [t0 + i * h for i in range(n + 1)]
+
+
+# --- spectral derivative ----------------------------------------------------------------------------
+
+def _spectral_derivative(halves_bug: bool):
+    def check_at(n):
+        L = 2 * np.pi
+        x = np.arange(n) * L / n
+        if halves_bug:  # wavenumbers built by splitting the array in two halves: only works for even n
+            dx = L / n
+            kmax = np.pi / dx
+            k = np.arange(float(n))
+            k[: int(n / 2)] = k[: int(n / 2)] * (kmax / (n / 2))
+            k[int(n / 2):] = k[: int(n / 2)] - kmax
+        else:
+            k = 2 * np.pi * np.fft.fftfreq(n, L / n)
+        d = np.real(np.fft.ifft(1j * k * np.fft.fft(np.sin(x))))
+        return bool(np.max(np.abs(d - np.cos(x))) < 1e-10)
+    return check_at
+
+
+@dataclass(frozen=True)
+class MethodCase:
+    name: str
+    buggy: bool
+    description: str
+    real_case: str  # repository ID in results/corpus/SUMMARY.md
+    check: str
+    run: Callable[[], Result]
+    missed_by: str = ""  # a check that cannot see it, when that is the point
+
+
+CASES = [
+    MethodCase("tableau_ok", False, "RK4 table, label 4", "-", "butcher.check_tableau", _tableau_case(RK4_A, RK4_B, 4)),
+    MethodCase("tableau_label_too_high", True, "Heun table (order 2) labelled order 3", "F09",
+               "butcher.check_tableau", _tableau_case(HEUN_A, HEUN_B, 3)),
+    MethodCase("embedded_ok", False, "Bogacki-Shampine embedded weights b_hat, label 2", "-",
+               "butcher.check_tableau", _tableau_case(BS_A, BS_BHAT, 2)),
+    MethodCase("error_weights_as_embedded", True,
+               "published error weights e = b - b_hat subtracted from b again, so the embedded weights become e "
+               "(they sum to 0)", "E09", "butcher.check_tableau", _tableau_case(BS_A, BS_B - BS_BHAT, 2)),
+    MethodCase("backward_ok", False, "RK4 stepping backwards with stages at t - h/2, t - h", "-",
+               "studies.backward_mms_study",
+               lambda: backward_mms_study(_rk4_back(False), MMS, 2.0, (10, 20, 40, 80), 4)),
+    MethodCase("backward_stage_times_forward", True, "RK4 stepping backwards with stages at t + h/2, t + h", "D08",
+               "studies.backward_mms_study",
+               lambda: backward_mms_study(_rk4_back(True), MMS, 2.0, (10, 20, 40, 80), 4),
+               missed_by="any forward-in-time check, and backward runs of autonomous equations"),
+    MethodCase("time_loop_ok", False, "loop over a counted number of steps", "-", "studies.time_grid_check",
+               lambda: time_grid_check(_counted_times, 0.0, 2.0, (4, 8, 20))),
+    MethodCase("while_loop_overshoot", True, "`while t <= t_end: t += h` takes one step too many when t lands on t_end",
+               "D08", "studies.time_grid_check", lambda: time_grid_check(_while_loop_times, 0.0, 2.0, (4, 8, 20)),
+               missed_by="order studies that read the state nearest t_end"),
+    MethodCase("spectral_ok", False, "wavenumbers from numpy.fft.fftfreq", "-", "studies.size_sweep_check",
+               lambda: size_sweep_check(_spectral_derivative(False))),
+    MethodCase("spectral_even_only", True, "wavenumbers built from two halves of the array (fails for odd n)", "G10",
+               "studies.size_sweep_check", lambda: size_sweep_check(_spectral_derivative(True)),
+               missed_by="any study on power-of-two grids"),
+]
+
+
+def run_all() -> list[tuple[MethodCase, Result]]:
+    return [(c, run_check(Check("method", c.name, c.run))) for c in CASES]
+
+
+def to_markdown(rows) -> str:
+    lines = ["| Case | Seeded bug | From | Check | Result | Missed by |", "|---|---|---|---|---|---|"]
+    for c, r in rows:
+        lines.append(f"| {c.name} | {'yes' if c.buggy else 'no (control)'} | {c.real_case} | `{c.check}` | {r.status} | {c.missed_by} |")
+    return "\n".join(lines) + "\n"

{physverify-0.1.0 → physverify-0.2.0}/src/physverify/order.py

@@ -110,18 +110,27 @@ def check_order(
         return OrderResult(INCONCLUSIVE, "fewer than 2 resolutions above the noise floor", **base)
 
     if e_m[-2] / e_m[-1] < 1.2:
-        return OrderResult(
-            FAIL,
-            f"error does not decrease with resolution (about {e_m[-1]:.3g} at the finest level)",
-            fitted=0.0,
-            **base,
-        )
+        msg = f"error does not decrease with resolution (about {e_m[-1]:.3g} at the finest level)"
+        if len(e_m) >= 3 and e_m.max() / e_m.min() < 1.1:
+            # seen in a real MD code: one pair crossing an unshifted cutoff adds V(rc) to the energy, whatever dt is
+            msg += ("; the error is the same at every resolution, which points to something independent of the step"
+                    " (a jump in the model such as a potential cutoff crossing, a wrong constant, or a reference for a"
+                    " different problem) rather than to the integrator")
+        return OrderResult(FAIL, msg, fitted=0.0, **base)
 
     h_f, e_f = h_m[-fit_points:], e_m[-fit_points:]
     p = fit_order(h_f, e_f)
     if p < expected - tol_low:
         msg = f"observed order {p:.2f} is below the expected {expected:g}"
+        pairs = pairwise_orders(h_f, e_f)
+        if len(pairs) >= 2 and np.all(np.diff(pairs) > 0) and pairs[-1] >= expected - tol_low:
+            msg += (f"; the pairwise orders ({', '.join(f'{q:.2f}' for q in pairs)}) are still rising and the last one"
+                    " reaches it, so the steps may be too coarse (pre-asymptotic): repeat with finer steps before"
+                    " calling this a bug")
         status = FAIL
+        if np.any(np.diff(e_f) >= 0):
+            msg += ("; the errors are not monotone over the fitted resolutions, which happens when the leading error"
+                    " term changes sign or round-off is reached: try another test problem before calling this a bug")
     elif p > expected + tol_high:
         msg = f"observed order {p:.2f} is above the expected {expected:g}; check the expected order"
         status = FAIL

{physverify-0.1.0 → physverify-0.2.0}/src/physverify/stochastic/ising.py

@@ -112,6 +112,11 @@ BUGS = {
         _f("bookkeeping", "scripted_rng", "moments", "acceptance"),
         "invisible at h = 0; in a field m has the wrong sign",
     ),
+    "field_counted_twice": IsingBug(
+        "metropolis", "measured energy uses -h * (sum of the 4 neighbours) / 2 per site, i.e. -2 h M (dynamics right)",
+        _f("bookkeeping", "moments"),
+        "invisible at h = 0; in a field <e> and <e^2> are wrong while m is right (found in a real thesis code)",
+    ),
     "spins_zero_one": IsingBug(
         "metropolis", "spins stored as {0, 1} and used in the +-1 formulas",
         _f("bookkeeping", "scripted_rng", "exact_distribution", "moments", "acceptance", "symmetry"),

{physverify-0.1.0 → physverify-0.2.0}/src/physverify/stochastic/sampler.py

@@ -10,7 +10,7 @@ black box
         the code's error bar for the mean of one chain's series (optional)
 
 white box (optional; checks that need a method the code lacks are skipped)
-    init(L, n_chains, start, rng) -> state      state.spins (n_chains, L, L), state.E, state.M
+    init(L, n_chains, start, rng, h=0.0) -> state   state.spins (n_chains, L, L), state.E, state.M
     sweep(state, T, h, rng)                      one sweep, updating state.E and state.M incrementally
     step(state, sites, u, T, h)                  one single-site update per chain at the given
                                                  sites (flat index) with the given uniforms
@@ -116,7 +116,8 @@ class IsingSampler:
                 right[:, :, -1] = 0.0
             bonds = (s * (right + np.roll(s, -1, axis=1))).sum(axis=(1, 2))
         M = s.sum(axis=(1, 2))
-        return -self.J * bonds - h * M, M
+        field = 2 * h if "field_counted_twice" in self.bugs else h
+        return -self.J * bonds - field * M, M
 
     # --- set-up ---
 

{physverify-0.1.0 → physverify-0.2.0}/src/physverify/studies.py

@@ -144,3 +144,67 @@ def ode_checks(
         p_y0, p_slope = patch
         checks.append(Check("patch", label, lambda: patch_study(integrate, rhs_ref, p_y0, p_slope, t_end, ns)))
     return checks
+
+
+def time_grid_check(times_of_run: Callable, t_start: float, t_stop: float, ns, *, tol: float = 1e-9) -> Result:
+    """The run stops at t_stop after exactly n steps, for step sizes that divide the interval.
+
+    `times_of_run(n)` runs the code with step (t_stop - t_start) / n and returns
+    the times it reports. A loop such as `while t <= t_end: t += h` takes one step
+    too many whenever t lands exactly on t_end in floating point (h = 0.5, 0.25)
+    and the right number otherwise (h = 0.1), so include step sizes that are
+    exact in binary. An order study that compares at the time nearest t_stop
+    cannot see this; this check can.
+    """
+    bad = []
+    for n in ns:
+        t = np.asarray(times_of_run(n), dtype=float)
+        h = abs(t_stop - t_start) / n
+        if len(t) != n + 1 or abs(t[-1] - t_stop) > tol * max(1.0, abs(t_stop)):
+            bad.append(f"h={h:g}: {len(t) - 1} steps ending at t={t[-1]:.6g} (expected {n} ending at {t_stop:g})")
+    if bad:
+        return Result(FAIL, "; ".join(bad))
+    return Result(PASS, f"ends at t={t_stop:g} with the expected number of steps for every h tested")
+
+
+def backward_mms_study(integrate_back: Callable, mms: ManufacturedODE, t_end: float, ns, expected_order, **kw) -> OrderResult:
+    """MMS order study integrating from t_end back to 0.
+
+    `integrate_back(source, y_end, t_end, n)` integrates y' = f(t, y) + source(t)
+    from t_end down to 0 in n steps and returns y(0). Use a manufactured solution
+    whose forcing depends on t: a code that evaluates its stages at t + h/2 while
+    stepping backwards still passes on autonomous problems but drops to first
+    order here.
+    """
+    y_end = mms.value(t_end)
+    y_0 = mms.value(0.0)
+    kw.setdefault("scale", float(np.max(np.abs(y_0))))
+    hs, errs = [], []
+    for n in ns:
+        hs.append(t_end / n)
+        errs.append(max_abs_error(integrate_back(mms.forcing, y_end, t_end, n), y_0))
+    return check_order(hs, errs, expected_order, **kw)
+
+
+def size_sweep_check(run_at_size: Callable, sizes=(15, 16, 31, 32, 63, 64), *, label: str = "") -> Result:
+    """Call `run_at_size(n)` for odd and even sizes; FAIL if it raises for some of them.
+
+    Spectral and FFT-based codes often build their wavenumber arrays by splitting
+    the grid in two halves, which only works for even n. A real code raised a
+    ValueError for every odd grid size while being exact for even ones; an order
+    study on 16, 32, 64, 128 points cannot see that. `run_at_size` may also
+    return False to mark a wrong result at that size.
+    """
+    bad = []
+    for n in sizes:
+        try:
+            ok = run_at_size(n)
+        except Exception as exc:  # the code under test may crash; that is the finding
+            bad.append(f"n={n}: {type(exc).__name__}: {exc}")
+            continue
+        if ok is False:
+            bad.append(f"n={n}: wrong result")
+    prefix = f"{label}: " if label else ""
+    if bad:
+        return Result(FAIL, prefix + "; ".join(bad))
+    return Result(PASS, f"{prefix}runs for every size tested ({', '.join(map(str, sizes))})")

{physverify-0.1.0 → physverify-0.2.0/src/physverify.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: physverify
-Version: 0.1.0
+Version: 0.2.0
 Summary: Verification checks for physics simulation code: manufactured solutions, order-of-accuracy studies and a seeded-bug benchmark.
 Author: Ram Chandra Giri
 License-Expression: MIT
@@ -39,7 +39,7 @@ Dynamic: license-file
 
 **A tool that checks whether a physics simulation is solving the right equations correctly, not just running without errors.**
 
-Status: early version (0.1.0). Nobody besides the author has used it yet, so feedback is very welcome.
+Status: early version (0.2.0). It has been tried on 20 public physics repositories (see below), but nobody besides the author has used it yet, so feedback is very welcome.
 
 ## The problem it addresses
 
@@ -54,6 +54,7 @@ physverify uses a few standard ideas from numerical analysis:
 3. **Check conserved quantities.** Energy, mass or momentum should stay constant (or drift in a predictable way).
 4. **Check known special cases.** For example, a projectile with no air resistance has a range you can write down by hand.
 5. **Exact "patch" tests.** Some solutions should be reproduced to computer precision. This catches small bugs that the other checks let through.
+6. **Algebraic checks on the method itself.** For a Runge-Kutta method, whether the coefficients really have the order they claim can be settled exactly, with no test problem (the "order conditions").
 
 Each check reports **PASS**, **FAIL**, **EXACT** or **INCONCLUSIVE**.
 
@@ -67,10 +68,21 @@ To test it, 37 bugs were added one at a time to five small solvers (projectile,
 - **0 false alarms** on the bug-free versions.
 - The 2 missed bugs are explained in the [technical details](https://github.com/rcgiri-physics/physverify/blob/main/docs/technical.md).
 
-For the Monte Carlo checks, all 20 seeded bugs were caught with no false alarms.
+For the Monte Carlo checks, all 21 seeded bugs were caught with no false alarms.
 
 The bugs and the checks were written by the same person, so these numbers show the tool works as designed. They are not a promise about bugs in other people's code.
 
+## Trying it on other people's code
+
+To see what it does on code it was not written for, 40 public Python physics repositories were drawn at random from GitHub searches (October 2026) and run in a locked-down container: no network, the repository read-only, nothing from it run on the author's computer.
+
+- **20 of the 40 could be tested.** Most of the others were not numerical solvers (neural networks, apps and games, analysis tools), one needed a recent NVIDIA GPU, and a few need more set-up.
+- **9 confirmed bugs in 6 of the 20.** For example: a measured energy that counts the magnetic-field term twice; backward time-stepping that is only first-order accurate; Runge-Kutta coefficient tables labelled with a higher order than they have; error-estimate weights that do not sum to zero; a spectral derivative that crashes for an odd number of grid points; a crash on NumPy 2.
+- **The other 14 had no confirmed bug.** Most passed every check that applied, often to 1e-13 or better; one raised a question for its authors rather than a bug report.
+- **Our own adapters were wrong about 15 times** before checking (a wrong constant, too-coarse steps, and so on). Every FAIL was checked by hand before being called a bug, and these mistakes are recorded, not hidden.
+
+Each lesson became a new check or a clearer message in physverify. The testing and write-ups were done with help from an AI assistant; every number comes from running the code. Authors are told about a finding before their project is named, so the repositories are listed by ID: [results/corpus/SUMMARY.md](https://github.com/rcgiri-physics/physverify/blob/main/results/corpus/SUMMARY.md).
+
 ## Try it
 
 You need Python 3.10 or newer.
@@ -108,6 +120,8 @@ pytest my_spec.py                              # each check becomes a pytest tes
 
 ## More
 
+- [results/corpus/SUMMARY.md](https://github.com/rcgiri-physics/physverify/blob/main/results/corpus/SUMMARY.md): the 40 public repositories, what was tested and what was found.
+- [sandbox/README.md](https://github.com/rcgiri-physics/physverify/blob/main/sandbox/README.md): the locked-down container used to run other people's code.
 - [results/own_work/SUMMARY.md](https://github.com/rcgiri-physics/physverify/blob/main/results/own_work/SUMMARY.md): results on four of the author's own public physics projects (two real bugs found and fixed).
 - [docs/technical.md](https://github.com/rcgiri-physics/physverify/blob/main/docs/technical.md): full details, benchmark tables, limits and related work.
 - [docs/patch.md](https://github.com/rcgiri-physics/physverify/blob/main/docs/patch.md): what the exact "patch" checks assume.

{physverify-0.1.0 → physverify-0.2.0}/src/physverify.egg-info/SOURCES.txt

@@ -4,11 +4,14 @@ pyproject.toml
 src/physverify/__init__.py
 src/physverify/__main__.py
 src/physverify/agent.py
+src/physverify/butcher.py
 src/physverify/checks.py
 src/physverify/cli.py
 src/physverify/compare.py
+src/physverify/constants.py
 src/physverify/loaders.py
 src/physverify/matrix.py
+src/physverify/method_bugs.py
 src/physverify/mms.py
 src/physverify/order.py
 src/physverify/pytest_plugin.py
@@ -46,7 +49,9 @@ src/physverify/stochastic/sampler.py
 src/physverify/stochastic/stats.py
 tests/test_agent.py
 tests/test_cli.py
+tests/test_corpus_lessons.py
 tests/test_ising.py
+tests/test_method_bugs.py
 tests/test_mms.py
 tests/test_new_problems.py
 tests/test_order.py

physverify-0.2.0/tests/test_corpus_lessons.py

@@ -0,0 +1,145 @@
+"""Checks added after testing other people's repositories (sandbox/README.md).
+
+Each test reproduces, in a few lines, the pattern a real code had, and shows the new check sees it.
+"""
+
+import numpy as np
+
+from physverify.checks import FAIL, PASS
+from physverify.constants import check_value, scan
+from physverify.loaders import load_definitions_from_path
+from physverify.mms import ManufacturedODE
+from physverify.order import check_order
+from physverify.steppers import rk4
+from physverify.studies import backward_mms_study, time_grid_check
+
+
+def while_loop_times(t0, t1, n):
+    """`while t <= t_end: t += h`, as in a real RK4 library."""
+    h = (t1 - t0) / n
+    t, times = t0, [t0]
+    while t <= t1:
+        t += h
+        times.append(t)
+    return times
+
+
+def counted_times(t0, t1, n):
+    h = (t1 - t0) / n
+    return [t0 + i * h for i in range(n + 1)]
+
+
+def test_time_grid_check_catches_the_extra_step_only_for_exact_binary_steps():
+    bad = time_grid_check(lambda n: while_loop_times(0.0, 2.0, n), 0.0, 2.0, (4, 8, 20))
+    assert bad.status == FAIL
+    assert "h=0.5" in bad.message and "h=0.25" in bad.message and "h=0.1:" not in bad.message
+    assert time_grid_check(lambda n: counted_times(0.0, 2.0, n), 0.0, 2.0, (4, 8, 20)).status == PASS
+
+
+MMS = ManufacturedODE(lambda t: np.array([np.exp(np.sin(t))]), lambda t, y: np.array([y[0] * np.cos(t)]))
+
+
+def backward(stage_time_sign):
+    """RK4 from t_end down to 0; stage_time_sign = +1 reproduces the bug (stages at t + h/2 while stepping back)."""
+    def integrate_back(source, y_end, t_end, n):
+        h = t_end / n
+        y, t = np.array(y_end, dtype=float), t_end
+        g = lambda s, u: -(MMS.rhs_ref(s, u) + source(s))  # dy/d(-t)
+        for _ in range(n):
+            if stage_time_sign > 0:
+                y = rk4(g, t, y, h)
+            else:
+                y = rk4(lambda s, u: g(-s, u), -t, y, h)
+            t -= h
+        return y
+    return integrate_back
+
+
+def test_backward_study_separates_wrong_stage_times():
+    ns = (10, 20, 40, 80)
+    assert backward_mms_study(backward(-1), MMS, 2.0, ns, 4).status == PASS
+    bad = backward_mms_study(backward(+1), MMS, 2.0, ns, 4)
+    assert bad.status == FAIL and bad.fitted < 1.5
+
+
+def test_order_check_says_when_the_steps_are_too_coarse():
+    hs = np.array([0.04, 0.02, 0.01, 0.005])
+    errs = hs**4 / (1 + 40 * hs)  # pre-asymptotic: pairwise orders 3.47, 3.64, 3.78, climbing toward 4
+    r = check_order(hs, errs, 4.0)
+    assert r.status == FAIL and "pre-asymptotic" in r.message
+    assert "pre-asymptotic" not in check_order(hs, hs**2, 4.0).message
+
+
+def test_constant_near_misses(tmp_path):
+    assert check_value(3.14)[0] == "pi" and check_value(3.1416) is None
+    assert check_value(39.47692640889762)[0].startswith("G M_sun") and check_value(4 * np.pi**2) is None
+    assert check_value(9.81) is None and check_value(6.674e-11) is None
+    src = tmp_path / "game.py"
+    src.write_text("g = 9\nn = 10\nU = 110\nG = 39.47692640889762\nx = 3.14 * 2\n")
+    names = [n.constant for n in scan(src)]
+    assert names == ["g", "G M_sun / 4 pi^2 (au, yr)", "pi"]
+
+
+def test_load_definitions_skips_the_script_body(tmp_path):
+    src = tmp_path / "sim.py"
+    src.write_text(
+        "import math\nimport argparse\nargs = argparse.ArgumentParser().parse_args(['--bogus'])\n"
+        "J = 2.0\nraise SystemExit('the simulation ran')\n"
+        "def energy(s):\n    return -J * s * math.cos(0)\n"
+        "def cell():\n    def inner(x):\n        return x + 1\n    return inner\n"
+    )
+    mod = load_definitions_from_path("sim_defs", src, keep=("J",), inner=("inner",))
+    assert mod.energy(3.0) == -6.0 and mod.inner(1) == 2
+
+
+def test_butcher_order_conditions():
+    from physverify.butcher import check_tableau, tableau_order, trees
+
+    assert [len(trees(n)) for n in range(1, 8)] == [1, 1, 2, 4, 9, 20, 48]
+    A = np.array([[0, 0, 0, 0], [0.5, 0, 0, 0], [0, 0.5, 0, 0], [0, 0, 1, 0]])
+    b = np.array([1 / 6, 1 / 3, 1 / 3, 1 / 6])
+    assert tableau_order(A, b) == 4
+    assert check_tableau(A, b, [0, 0.5, 0.5, 1], claimed=4).status == PASS
+    # a real case: an order-3 table labelled as order 4
+    bad = check_tableau(A, b, claimed=5)
+    assert bad.status == FAIL and "only up to 4" in bad.message
+    s = np.sqrt(15)
+    gauss3 = np.array([[5 / 36, 2 / 9 - s / 15, 5 / 36 - s / 30], [5 / 36 + s / 24, 2 / 9, 5 / 36 - s / 24],
+                       [5 / 36 + s / 30, 2 / 9 + s / 15, 5 / 36]])
+    assert tableau_order(gauss3, [5 / 18, 4 / 9, 5 / 18], max_order=7) == 6  # implicit tables work too
+    # coefficients typed with few digits are noted
+    rough = check_tableau(np.round(gauss3, 8), [5 / 18, 4 / 9, 5 / 18], claimed=6, tol=1e-6)
+    assert rough.status == PASS and "limited digits" in rough.message
+
+
+def test_order_check_flags_non_monotone_errors():
+    hs = np.array([0.1, 0.05, 0.025, 0.0125])
+    errs = np.array([1e-6, 1e-8, 2e-8, 1e-8])  # leading term changes sign between the second and third
+    r = check_order(hs, errs, 4.0, floor=1e-14)
+    assert r.status == FAIL and "not monotone" in r.message
+
+
+def test_order_check_explains_a_step_independent_error():
+    hs = np.array([0.004, 0.002, 0.001, 0.0005])
+    r = check_order(hs, np.array([0.0164, 0.0164, 0.0163, 0.0163]), 2.0)  # the MD cutoff-crossing case
+    assert r.status == FAIL and "independent of the step" in r.message
+
+
+def test_size_sweep_finds_a_crash_at_odd_sizes():
+    from physverify.studies import size_sweep_check
+
+    def halves(n):  # the pattern of a real spectral code
+        k = np.arange(float(n))
+        k[int(n / 2):] = k[: int(n / 2)] - 1.0
+        return True
+
+    r = size_sweep_check(halves)
+    assert r.status == FAIL and "n=15" in r.message and "n=16" not in r.message
+    assert size_sweep_check(lambda n: np.fft.fftfreq(n).size == n).status == PASS
+
+
+def test_load_definitions_stubs_missing_imports(tmp_path):
+    src = tmp_path / "sim.py"
+    src.write_text("import not_installed_plotting_pkg\nimport math\ndef f(x):\n    return math.sqrt(x)\n")
+    mod = load_definitions_from_path("sim_stub", src, stubs=("not_installed_plotting_pkg",))
+    assert mod.f(4.0) == 2.0

physverify-0.2.0/tests/test_method_bugs.py

@@ -0,0 +1,21 @@
+"""Every seeded method bug is caught by the check named for it, and every control passes."""
+
+from physverify.checks import FAIL
+from physverify.method_bugs import run_all
+
+
+def test_every_method_bug_is_caught_and_every_control_passes():
+    for case, result in run_all():
+        if case.buggy:
+            assert result.status == FAIL, (case.name, result.message)
+        else:
+            assert result.status in ("PASS", "EXACT"), (case.name, result.message)
+
+
+def test_committed_method_bug_table_matches_a_fresh_run():
+    from pathlib import Path
+
+    from physverify.method_bugs import to_markdown
+
+    committed = (Path(__file__).resolve().parents[1] / "results" / "method_bugs.md").read_text(encoding="utf-8")
+    assert to_markdown(run_all()) in committed

physverify-0.1.0/src/physverify/loaders.py

@@ -1,49 +0,0 @@
-"""Import a module from a path without touching its folder.
-
-Used by the adapters that test other people's code. Nothing is written next to
-the module (no __pycache__), and optional plotting imports can be stubbed so a
-simulation file can be imported on a machine without matplotlib.
-"""
-
-from __future__ import annotations
-
-import importlib
-import importlib.util
-import sys
-import types
-from pathlib import Path
-
-
-def _stub_module(name: str) -> types.ModuleType:
-    return types.ModuleType(name)
-
-
-def load_module_from_path(name: str, path, stubs=()):
-    path = Path(path)
-    if not path.is_file():
-        raise FileNotFoundError(path)
-
-    added = []
-    for stub in stubs:
-        try:
-            importlib.import_module(stub)
-        except ImportError:
-            sys.modules[stub] = _stub_module(stub)
-            added.append(stub)
-
-    old_flag = sys.dont_write_bytecode
-    sys.dont_write_bytecode = True
-    try:
-        spec = importlib.util.spec_from_file_location(name, path)
-        module = importlib.util.module_from_spec(spec)
-        sys.modules[name] = module
-        try:
-            spec.loader.exec_module(module)
-        except BaseException:
-            sys.modules.pop(name, None)  # do not leave a half-initialised module behind
-            raise
-    finally:
-        sys.dont_write_bytecode = old_flag
-        for stub in added:
-            sys.modules.pop(stub, None)
-    return module