simaudit 0.1.0a1__tar.gz

simaudit-0.1.0a1/CHANGELOG.md

@@ -0,0 +1,15 @@
+# Changelog
+
+## 0.1.0a1 - 2026-11-14
+
+First public alpha.
+
+- Checks: parameter-assignment consistency, integrator-order convergence,
+  release comparison across interpreters, and observation consistency.
+- Six explicit statuses (PASS, FAIL, ABSTAIN, SKIPPED, PARTIAL, BLOWUP) and
+  strict JSON reports with stated limitations.
+- CLI, pytest fixture, Gymnasium CartPole/Reacher adapters, MuJoCo order adapter.
+- Fixed before publication: `compare_releases` exposed the caller's whole
+  site-packages to its worker interpreters when pip-installed; workers now see
+  only the simaudit package.
+- Validation includes MuJoCo 3.16.0, 3.17.0 and 4.0.1 (see VALIDATION.md).

simaudit-0.1.0a1/CITATION.cff

@@ -0,0 +1,37 @@
+cff-version: 1.2.0
+message: "Please cite the two method papers below when using simaudit. Paper authorship is recorded separately from software authorship."
+title: "simaudit: scoped diagnostics for simulators"
+type: software
+version: 0.1.0a1
+date-released: 2026-11-14
+repository-code: "https://github.com/TesfayZ/simaudit"
+keywords:
+  - reinforcement learning
+  - simulator validation
+  - MuJoCo
+  - Gymnasium
+  - numerical verification
+abstract: "Scoped, reproducible diagnostics for user-supplied simulators: parameter-assignment consistency, integrator-order convergence, release comparison and observation consistency, with explicit statements of what was not assessed."
+license: MIT
+authors:
+  - family-names: Gebrekidan
+    given-names: Tesfay Zemuy
+references:
+  - type: article
+    title: "Diagnosing Model and Discretisation Errors in Reinforcement Learning Simulators Using Invariants"
+    authors:
+      - family-names: Gebrekidan
+        given-names: Tesfay Zemuy
+    year: 2026
+    doi: 10.5281/zenodo.24114609
+    url: "https://zenodo.org/records/24114609"
+  - type: article
+    title: "Diagnosing Faults in Reinforcement Learning Simulators with Canonical Polynomial Invariants"
+    authors:
+      - family-names: Gebrekidan
+        given-names: Tesfay Zemuy
+      - family-names: Gebrerufael
+        given-names: Hadush Hailu
+    year: 2026
+    doi: 10.48550/arXiv.2609.13194
+    url: "https://arxiv.org/abs/2609.13194v2"

simaudit-0.1.0a1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tesfay Zemuy Gebrekidan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

simaudit-0.1.0a1/MANIFEST.in

@@ -0,0 +1,6 @@
+include LICENSE CITATION.cff references.bib THIRD_PARTY.md VALIDATION.md CHANGELOG.md
+recursive-include docs *.md
+recursive-include examples *.py *.json
+recursive-include validation *.py *.md
+recursive-include tests *.py
+prune evidence

simaudit-0.1.0a1/PKG-INFO

@@ -0,0 +1,230 @@
+Metadata-Version: 2.4
+Name: simaudit
+Version: 0.1.0a1
+Summary: Scoped, reproducible diagnostics for user-supplied simulators
+Author: Tesfay Zemuy Gebrekidan
+License-Expression: MIT
+Project-URL: Repository, https://github.com/TesfayZ/simaudit
+Project-URL: Issues, https://github.com/TesfayZ/simaudit/issues
+Project-URL: Documentation, https://github.com/TesfayZ/simaudit#readme
+Keywords: reinforcement-learning,simulator,verification,mujoco,gymnasium,testing
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering
+Classifier: Framework :: Pytest
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24
+Provides-Extra: gymnasium
+Requires-Dist: gymnasium>=1.0; extra == "gymnasium"
+Provides-Extra: mujoco
+Requires-Dist: mujoco>=3.2; extra == "mujoco"
+Requires-Dist: gymnasium>=1.0; extra == "mujoco"
+Provides-Extra: pytest
+Requires-Dist: pytest>=8; extra == "pytest"
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: build>=1; extra == "dev"
+Requires-Dist: gymnasium>=1.0; extra == "dev"
+Dynamic: license-file
+
+# simaudit
+
+Installable diagnostics for **your own simulator**, with explicit limits on what
+was assessed. Supply a small Python adapter; NumPy is the only core dependency.
+Gymnasium and MuJoCo are optional examples. Version **0.1.0a1** is an alpha,
+not a generally calibrated simulator-certification service.
+
+## Install
+
+```bash
+# Pre-release: pip skips alpha versions unless you ask for them.
+python -m pip install --pre simaudit
+# Optional adapters and pytest integration:
+python -m pip install --pre 'simaudit[gymnasium,mujoco,pytest]'
+# Or from a source checkout:
+python -m pip install .
+```
+
+Python 3.11 or newer (CI matrix 3.11-3.14; local validation on 3.14.6 and 3.12.3). Nothing downloads, runs experiments, or imports an optional
+simulator on package import. For development: `python -m pip install '.[dev]'`.
+
+## Test your own simulator
+
+```python
+import numpy as np
+from simaudit import Adapter, Feature, Report, check_order
+
+# Wrap your simulator so each call restores every dependency of its next step.
+def step(state, action, dt):
+    return state + dt * (state + action)
+
+def native_reference(state, action, dt):
+    return np.asarray([(1 + dt) * state[0] + dt * action])
+
+adapter = Adapter(
+    name="my simulator", step_fn=step, native_step_fn=native_reference,
+    features=[Feature("dynamics", "tested", "Smooth scalar ODE, no hidden state", True)],
+    smooth=True, declared_order=1, dt=0.2,
+)
+result = check_order(adapter, [[0.5], [1.0], [1.5]], [0, 0, 0])
+report = Report([result])
+print(report.summary())
+report.write("audit.json")
+```
+
+The native reference should call the original simulator without the adapter
+transformation being tested. A user-provided reference and inventory are trusted
+inputs, not automatically verified specifications. See [the adapter guide](docs/ADAPTERS.md)
+for mutable simulators, noise, arbitrary state metrics, assignment factories,
+observation maps and capability requirements.
+
+The CLI loads a trusted local `module:function` returning an adapter. From this
+directory, run the included NumPy-only example:
+
+```bash
+PYTHONPATH=examples simaudit order --factory custom_simulator:factory \
+  --probes examples/probes.json --strict --json audit.json
+```
+
+Python factories execute code with your Python process's privileges. Probe files
+contain only `states` and matching `actions`. The same interface works with your
+own importable adapter module. `python -m simaudit` is equivalent to `simaudit`.
+
+## Four independent checks
+
+| Check | Python API | Needs | What it reports |
+|---|---|---|---|
+| Parameter assignment | `check_assignment` | Replay, native reference, assignment factory | Baseline/direct/fresh comparison; no-op, stale, consistent or unresolved |
+| Numerical order | `check_order` | Replay, native reference, timestep, declared order, smoothness, distance | Stable refinement slopes, below-declaration order, or inability to diagnose |
+| Release/variant comparison | `compare_releases` | Factory and two supplied Python interpreters | First-step differences on identical inputs; later differences recorded separately |
+| Observation consistency | `check_observation` | Replay, native reference, observed and reference observation maps | Differences above the precision/noise floor |
+
+The package doesn't implement the papers' energy-fingerprint classifier,
+algebraic localisation, parameter identification or automatic equation diagnosis.
+A numerical-order pass cannot establish that equations are correct.
+
+## Results and limitations
+
+Every result contains **PASS, FAIL, ABSTAIN, SKIPPED, PARTIAL or BLOWUP**, a reason,
+scope, limitations, mandatory feature coverage, modifications, versions and raw
+probe measurements. Limitations and untested features are printed beside results.
+
+- Float32 or a declared noise floor blocks a healthy consistency verdict in
+  checks A, B and D. A and D can still flag discrepancies above their floors.
+- Missing capabilities return `SKIPPED`; unsupported smoothness/features give
+  `PARTIAL`; unresolved or unstable slopes give `ABSTAIN`.
+- Nonfinite trajectories or detected engine resets give `BLOWUP`.
+- A failed adapter replay/native-step self-test blocks downstream diagnosis.
+- `PASS` covers the supplied probes under the declared conditions only. A wrong
+  reference, shared implementation error or unreported hidden state can escape it.
+
+By default `FAIL` exits 1; other nonpasses print a warning and exit 0. `--strict`
+exits 1 on **any** nonpass. Configuration errors exit 2. Empty reports are never
+all-clear. [Report contract](docs/REPORTS.md).
+
+## Optional adapters and releases
+
+```bash
+simaudit assignment --factory simaudit.adapters.cartpole:cartpole \
+  --probes examples/cartpole-probes.json --parameter masscart --value 2 --json cartpole.json
+```
+
+This deliberately detects the paper's assignment discrepancy and exits 1. The
+fresh CartPole reference explicitly rebuilds dependent constants; it does not
+assert that Gymnasium promises arbitrary attribute mutation.
+
+`mujoco_xml(xml)` supplies native replay for smooth autonomous rigid models and
+rejects unsupported features. `reacher()` supplies a cache-consistency comparison
+against the shipped observation map after `mj_forward`. Both live in
+`simaudit.adapters.mujoco`; imports are lazy. Custom adapters are not subject to
+these built-in models' limitations but must declare their own.
+
+For a release pair, create two separate environments yourself, with NumPy and
+the simulator versions you want, then supply their Python paths:
+
+```bash
+simaudit release --factory simaudit.adapters.cartpole:cartpole \
+  --probes examples/cartpole-probes.json --python-a /path/a/bin/python \
+  --python-b /path/b/bin/python --strict --json releases.json
+```
+
+The worker runs the current package source in each interpreter. The package does
+not install or upgrade either environment. Your custom factory module must also
+be importable there. Missing versions remain untested. A changed first step means
+compatibility changed, not necessarily that either release is wrong.
+
+## pytest and verification
+
+Install the `pytest` extra to use the auto-discovered `simaudit_check` fixture:
+
+```python
+def test_simulator(simaudit_check):
+    simaudit_check(check_order(adapter, [[1.0]], [0]))
+```
+
+`FAIL` fails the test; other nonpasses skip with their original status and full
+reason. `pytest --simaudit-strict --simaudit-json results.json` fails the session
+on any nonpass and preserves the separate audit counts. Reporting is serial;
+pytest-xdist aggregation is not supported.
+
+Run software tests with `python -m pytest`. Build with
+`python -m build`. See [validation results](VALIDATION.md),
+[prospective paper-case protocol](validation/PROTOCOL.md), and
+[requirements and acceptance criteria](docs/REQUIREMENTS.md). These are small package tests
+and selected reproductions, not a general false-alarm guarantee.
+
+## Cite the two method papers
+
+1. **Tesfay Zemuy Gebrekidan** (2026). *Diagnosing Model and Discretisation Errors
+   in Reinforcement Learning Simulators Using Invariants*.
+   [Zenodo](https://zenodo.org/records/24114609).
+2. **Tesfay Zemuy Gebrekidan and Hadush Hailu Gebrerufael** (2026).
+   *Diagnosing Faults in Reinforcement Learning Simulators with Canonical
+   Polynomial Invariants*. [arXiv:2609.13194v2](https://arxiv.org/abs/2609.13194v2).
+
+[CITATION.cff](CITATION.cff), [BibTeX](references.bib), and
+[reuse and licence notices](THIRD_PARTY.md) preserve the attribution. MIT licensed.
+
+## What it has found so far
+
+From [VALIDATION.md](VALIDATION.md); each row is scoped to the supplied probes.
+
+| Case | Result |
+|---|---|
+| Gymnasium CartPole `masscart`/`masspole`/`length` assignment | FAIL (no-op / stale / partly stale) vs a freshly constructed reference; `gravity` PASS |
+| MuJoCo 3.16.0 and 3.17.0, RK4 heavy top on a ball joint | FAIL: 23/24 probes below declared order 4 (1/24 ABSTAIN), median slope 1.96 |
+| MuJoCo **4.0.1**, same model and probes | PASS 24/24, median slope 3.92 |
+| Same model with three hinges (control) | PASS in every tested engine |
+| Reacher post-step observation vs refreshed map | FAIL, largest discrepancy 2.4e-6 (3.16.0, 3.17.0 and 4.0.1) |
+
+MuJoCo 4.0.0 itself was not installed or tested; no result is inferred for it.
+
+## Roadmap
+
+Not implemented in 0.1.0a1, and not claimed anywhere in this package:
+
+- **0.2:** energy-refinement fingerprint (model-versus-discretisation label), a
+  decision table combining checks, and a one-parameter fit, each only with its
+  negative-control suite passing.
+- **0.3:** algebraic invariant screening behind an optional extra.
+- Other classic-control assignment adapters, vector environments, and shipped
+  Ant/Humanoid with contacts and actuation.
+
+## Documentation
+
+| File | Purpose |
+|---|---|
+| [docs/ADAPTERS.md](docs/ADAPTERS.md) | Writing an adapter and declaring its features |
+| [docs/REPORTS.md](docs/REPORTS.md) | Report schema, statuses and JSON round trip |
+| [docs/REQUIREMENTS.md](docs/REQUIREMENTS.md) | Acceptance criteria recorded before validation |
+| [VALIDATION.md](VALIDATION.md) | What was actually run, and the limits of those runs |
+| [validation/PROTOCOL.md](validation/PROTOCOL.md) | Paper-case protocol and scripts |
+| [evidence/README.md](evidence/README.md) | Archived result files and the hash receipt |
+| [CHANGELOG.md](CHANGELOG.md) | Release history |

simaudit-0.1.0a1/README.md

@@ -0,0 +1,194 @@
+# simaudit
+
+Installable diagnostics for **your own simulator**, with explicit limits on what
+was assessed. Supply a small Python adapter; NumPy is the only core dependency.
+Gymnasium and MuJoCo are optional examples. Version **0.1.0a1** is an alpha,
+not a generally calibrated simulator-certification service.
+
+## Install
+
+```bash
+# Pre-release: pip skips alpha versions unless you ask for them.
+python -m pip install --pre simaudit
+# Optional adapters and pytest integration:
+python -m pip install --pre 'simaudit[gymnasium,mujoco,pytest]'
+# Or from a source checkout:
+python -m pip install .
+```
+
+Python 3.11 or newer (CI matrix 3.11-3.14; local validation on 3.14.6 and 3.12.3). Nothing downloads, runs experiments, or imports an optional
+simulator on package import. For development: `python -m pip install '.[dev]'`.
+
+## Test your own simulator
+
+```python
+import numpy as np
+from simaudit import Adapter, Feature, Report, check_order
+
+# Wrap your simulator so each call restores every dependency of its next step.
+def step(state, action, dt):
+    return state + dt * (state + action)
+
+def native_reference(state, action, dt):
+    return np.asarray([(1 + dt) * state[0] + dt * action])
+
+adapter = Adapter(
+    name="my simulator", step_fn=step, native_step_fn=native_reference,
+    features=[Feature("dynamics", "tested", "Smooth scalar ODE, no hidden state", True)],
+    smooth=True, declared_order=1, dt=0.2,
+)
+result = check_order(adapter, [[0.5], [1.0], [1.5]], [0, 0, 0])
+report = Report([result])
+print(report.summary())
+report.write("audit.json")
+```
+
+The native reference should call the original simulator without the adapter
+transformation being tested. A user-provided reference and inventory are trusted
+inputs, not automatically verified specifications. See [the adapter guide](docs/ADAPTERS.md)
+for mutable simulators, noise, arbitrary state metrics, assignment factories,
+observation maps and capability requirements.
+
+The CLI loads a trusted local `module:function` returning an adapter. From this
+directory, run the included NumPy-only example:
+
+```bash
+PYTHONPATH=examples simaudit order --factory custom_simulator:factory \
+  --probes examples/probes.json --strict --json audit.json
+```
+
+Python factories execute code with your Python process's privileges. Probe files
+contain only `states` and matching `actions`. The same interface works with your
+own importable adapter module. `python -m simaudit` is equivalent to `simaudit`.
+
+## Four independent checks
+
+| Check | Python API | Needs | What it reports |
+|---|---|---|---|
+| Parameter assignment | `check_assignment` | Replay, native reference, assignment factory | Baseline/direct/fresh comparison; no-op, stale, consistent or unresolved |
+| Numerical order | `check_order` | Replay, native reference, timestep, declared order, smoothness, distance | Stable refinement slopes, below-declaration order, or inability to diagnose |
+| Release/variant comparison | `compare_releases` | Factory and two supplied Python interpreters | First-step differences on identical inputs; later differences recorded separately |
+| Observation consistency | `check_observation` | Replay, native reference, observed and reference observation maps | Differences above the precision/noise floor |
+
+The package doesn't implement the papers' energy-fingerprint classifier,
+algebraic localisation, parameter identification or automatic equation diagnosis.
+A numerical-order pass cannot establish that equations are correct.
+
+## Results and limitations
+
+Every result contains **PASS, FAIL, ABSTAIN, SKIPPED, PARTIAL or BLOWUP**, a reason,
+scope, limitations, mandatory feature coverage, modifications, versions and raw
+probe measurements. Limitations and untested features are printed beside results.
+
+- Float32 or a declared noise floor blocks a healthy consistency verdict in
+  checks A, B and D. A and D can still flag discrepancies above their floors.
+- Missing capabilities return `SKIPPED`; unsupported smoothness/features give
+  `PARTIAL`; unresolved or unstable slopes give `ABSTAIN`.
+- Nonfinite trajectories or detected engine resets give `BLOWUP`.
+- A failed adapter replay/native-step self-test blocks downstream diagnosis.
+- `PASS` covers the supplied probes under the declared conditions only. A wrong
+  reference, shared implementation error or unreported hidden state can escape it.
+
+By default `FAIL` exits 1; other nonpasses print a warning and exit 0. `--strict`
+exits 1 on **any** nonpass. Configuration errors exit 2. Empty reports are never
+all-clear. [Report contract](docs/REPORTS.md).
+
+## Optional adapters and releases
+
+```bash
+simaudit assignment --factory simaudit.adapters.cartpole:cartpole \
+  --probes examples/cartpole-probes.json --parameter masscart --value 2 --json cartpole.json
+```
+
+This deliberately detects the paper's assignment discrepancy and exits 1. The
+fresh CartPole reference explicitly rebuilds dependent constants; it does not
+assert that Gymnasium promises arbitrary attribute mutation.
+
+`mujoco_xml(xml)` supplies native replay for smooth autonomous rigid models and
+rejects unsupported features. `reacher()` supplies a cache-consistency comparison
+against the shipped observation map after `mj_forward`. Both live in
+`simaudit.adapters.mujoco`; imports are lazy. Custom adapters are not subject to
+these built-in models' limitations but must declare their own.
+
+For a release pair, create two separate environments yourself, with NumPy and
+the simulator versions you want, then supply their Python paths:
+
+```bash
+simaudit release --factory simaudit.adapters.cartpole:cartpole \
+  --probes examples/cartpole-probes.json --python-a /path/a/bin/python \
+  --python-b /path/b/bin/python --strict --json releases.json
+```
+
+The worker runs the current package source in each interpreter. The package does
+not install or upgrade either environment. Your custom factory module must also
+be importable there. Missing versions remain untested. A changed first step means
+compatibility changed, not necessarily that either release is wrong.
+
+## pytest and verification
+
+Install the `pytest` extra to use the auto-discovered `simaudit_check` fixture:
+
+```python
+def test_simulator(simaudit_check):
+    simaudit_check(check_order(adapter, [[1.0]], [0]))
+```
+
+`FAIL` fails the test; other nonpasses skip with their original status and full
+reason. `pytest --simaudit-strict --simaudit-json results.json` fails the session
+on any nonpass and preserves the separate audit counts. Reporting is serial;
+pytest-xdist aggregation is not supported.
+
+Run software tests with `python -m pytest`. Build with
+`python -m build`. See [validation results](VALIDATION.md),
+[prospective paper-case protocol](validation/PROTOCOL.md), and
+[requirements and acceptance criteria](docs/REQUIREMENTS.md). These are small package tests
+and selected reproductions, not a general false-alarm guarantee.
+
+## Cite the two method papers
+
+1. **Tesfay Zemuy Gebrekidan** (2026). *Diagnosing Model and Discretisation Errors
+   in Reinforcement Learning Simulators Using Invariants*.
+   [Zenodo](https://zenodo.org/records/24114609).
+2. **Tesfay Zemuy Gebrekidan and Hadush Hailu Gebrerufael** (2026).
+   *Diagnosing Faults in Reinforcement Learning Simulators with Canonical
+   Polynomial Invariants*. [arXiv:2609.13194v2](https://arxiv.org/abs/2609.13194v2).
+
+[CITATION.cff](CITATION.cff), [BibTeX](references.bib), and
+[reuse and licence notices](THIRD_PARTY.md) preserve the attribution. MIT licensed.
+
+## What it has found so far
+
+From [VALIDATION.md](VALIDATION.md); each row is scoped to the supplied probes.
+
+| Case | Result |
+|---|---|
+| Gymnasium CartPole `masscart`/`masspole`/`length` assignment | FAIL (no-op / stale / partly stale) vs a freshly constructed reference; `gravity` PASS |
+| MuJoCo 3.16.0 and 3.17.0, RK4 heavy top on a ball joint | FAIL: 23/24 probes below declared order 4 (1/24 ABSTAIN), median slope 1.96 |
+| MuJoCo **4.0.1**, same model and probes | PASS 24/24, median slope 3.92 |
+| Same model with three hinges (control) | PASS in every tested engine |
+| Reacher post-step observation vs refreshed map | FAIL, largest discrepancy 2.4e-6 (3.16.0, 3.17.0 and 4.0.1) |
+
+MuJoCo 4.0.0 itself was not installed or tested; no result is inferred for it.
+
+## Roadmap
+
+Not implemented in 0.1.0a1, and not claimed anywhere in this package:
+
+- **0.2:** energy-refinement fingerprint (model-versus-discretisation label), a
+  decision table combining checks, and a one-parameter fit, each only with its
+  negative-control suite passing.
+- **0.3:** algebraic invariant screening behind an optional extra.
+- Other classic-control assignment adapters, vector environments, and shipped
+  Ant/Humanoid with contacts and actuation.
+
+## Documentation
+
+| File | Purpose |
+|---|---|
+| [docs/ADAPTERS.md](docs/ADAPTERS.md) | Writing an adapter and declaring its features |
+| [docs/REPORTS.md](docs/REPORTS.md) | Report schema, statuses and JSON round trip |
+| [docs/REQUIREMENTS.md](docs/REQUIREMENTS.md) | Acceptance criteria recorded before validation |
+| [VALIDATION.md](VALIDATION.md) | What was actually run, and the limits of those runs |
+| [validation/PROTOCOL.md](validation/PROTOCOL.md) | Paper-case protocol and scripts |
+| [evidence/README.md](evidence/README.md) | Archived result files and the hash receipt |
+| [CHANGELOG.md](CHANGELOG.md) | Release history |

simaudit-0.1.0a1/THIRD_PARTY.md

@@ -0,0 +1,32 @@
+# Method provenance and reuse
+
+The package uses two source papers, with their full authorship preserved:
+
+1. **Tesfay Zemuy Gebrekidan.** *Diagnosing Model and Discretisation Errors in
+   Reinforcement Learning Simulators Using Invariants* (2026).
+   [Zenodo record](https://zenodo.org/records/24114609).
+   Source repository: [`dafd`](https://github.com/TesfayZ/dafd). Assignment consistency, refinement diagnostics,
+   native integrator controls and precision/coverage limitations inform this package.
+2. **Tesfay Zemuy Gebrekidan and Hadush Hailu Gebrerufael.** *Diagnosing Faults in
+   Reinforcement Learning Simulators with Canonical Polynomial Invariants* (2026).
+   [arXiv:2609.13194v2](https://arxiv.org/abs/2609.13194v2).
+   Source repository: [`algebraicRLtest`](https://github.com/TesfayZ/algebraicRLtest). Paired release comparisons, positive
+   controls and observation consistency inform this package. Its algebraic
+   screening/localisation method is not implemented in this alpha.
+
+Both source repositories have MIT licences, copyright 2026 Tesfay Zemuy Gebrekidan.
+That software licence notice does not replace the papers' scholarly authorship.
+The MIT notice is retained in LICENSE. `validation/known_cases.py` adapts the
+minimal heavy-top geometry from `dafd/benchmark_quaternion_mechanism.py`; its
+collision masks are explicitly changed for the conservative test. The small
+checks are new adapter-based implementations of the documented methods. No
+research runner, paper prose, dataset, model or complete source module is vendored.
+Source file hashes inspected during development appear in evidence/reuse_sources.json.
+
+Runtime dependencies are imported, not bundled: NumPy (BSD-3-Clause), Gymnasium
+(MIT, optional), MuJoCo (Apache-2.0, optional), pytest (MIT, optional). A stable
+release still needs a full resolved dependency licence inventory, including
+transitive dependencies. This list does not claim that audit has been completed.
+
+Official interfaces consulted: [MuJoCo API](https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html)
+and [pytest plugin documentation](https://docs.pytest.org/en/stable/how-to/writing_plugins.html).

simaudit-0.1.0a1/VALIDATION.md

@@ -0,0 +1,118 @@
+# Validation of 0.1.0a1
+
+Local measurements, 11 November 2026. Protocols were written before package
+measurements: docs/REQUIREMENTS.md (software controls) and validation/PROTOCOL.md
+(paper cases). Full results are in evidence/*.json. These are new package runs,
+not imports of the papers' result tables and not an independent scientific
+replication of their complete studies.
+
+## Paper cases
+
+Both installed combinations were exercised: **Gymnasium 1.4.0 / MuJoCo 3.16.0**
+and **Gymnasium 1.3.0 / MuJoCo 3.17.0**, under Python 3.14.6.
+
+| Case | Result in both combinations | Scope |
+|---|---|---|
+| CartPole masscart x2 | FAIL: no-op relative to fresh reference | 100 supplied starts, alternate actions |
+| CartPole masspole x2 | FAIL: stale constants | Same 100 starts |
+| CartPole length x2 | FAIL: partly stale update | Mechanism annotation comes from adapter source inspection |
+| CartPole gravity x2 | PASS: consistent | 100 starts, no observed discrepancies |
+| RK4 heavy top, ball joint | FAIL: 23/24 probes below declared order; 1/24 ABSTAIN | Median fine-pair slope 1.95918; conservative geometry, h=.05 |
+| RK4 heavy top, three hinges | PASS: 24/24 probes | Median fine-pair slope 3.86154; healthy geometry control |
+| Reacher post-step observation | FAIL: discrepancy from refreshed map | 24 starts; largest discrepancy 2.39071e-6 |
+
+No 4.0.0 engine was installed or tested. No result is inferred for it, for other
+release versions, or for shipped Ant/Humanoid with contacts and actuation.
+
+### Addendum, 14 November 2026: MuJoCo 4.0.1
+
+The same protocol and seed were run unchanged with Gymnasium 1.4.0 / MuJoCo 4.0.1
+under Python 3.12.3 (evidence/paper_cases_mj401.json). CartPole results are
+identical. The heavy-top **ball-joint case now PASSes on 24/24 probes** (median
+fine-pair slope 3.915; it was FAIL with 1.959 on 3.16.0), and the hinge control
+still passes (3.862). The Reacher observation check still FAILs (largest
+discrepancy 2.39071e-6). This is a measurement on the conservative heavy-top
+model only; it does not establish where the change occurred between 3.17.0 and
+4.0.1, and 4.0.0 remains untested.
+
+### Addendum: release-worker isolation bug (fixed before first publication)
+
+When simaudit was pip-installed, `compare_releases` put the caller's whole
+site-packages directory on the workers' PYTHONPATH, which could make both workers
+import the caller's NumPy/simulator instead of each interpreter's own versions.
+The validation runs above used a source tree, where only the package directory
+was exposed, so they were not affected. Workers now receive a temporary directory
+containing only the simaudit package, with a regression test. The Acrobot release
+controls were not rerun after this change.
+The Reacher reference shares the observation formula and only checks its cache
+consistency. CartPole's fresh reference rebuilds constants under an explicitly
+supplied parameter contract; direct attribute mutation need not be an upstream
+supported API. A FAIL here is scoped to that comparison.
+
+The two runs reuse the same probe design and are not 48 independent starts for
+an engine-wide population claim. The ball-joint abstention is retained. No
+threshold was relaxed to remove it.
+
+## Release comparison and controls from the joint paper
+
+Using isolated workers, Gymnasium 1.3.0 versus 1.4.0 Acrobot agreed exactly on
+all recorded first steps. On the same 1.3.0 version, the book-versus-nips control
+was detected (maximum first-step difference 0.00147277) and the mass2=1.3 control
+was detected (0.0609411). Each used 24 paired starts, 5 recorded steps, and the
+first-step threshold 1e-13. This validates detection, not the joint paper's
+algebraic screening, constraint localisation or parameter attribution, which
+are deferred features. The full historical 350-pair matrix was not rerun.
+
+## General-purpose software controls
+
+Euler and RK4 scalar ODE controls each had 0 failures and 0 abstentions across
+100 starts. The two-sided 95% Wilson false-alarm interval per control is
+approximately [0, 0.0370]. This is conditional on those chosen scalar controls,
+not calibration across arbitrary simulators. Float32 order and observation
+controls abstain; declared-noise order controls abstain. Float64 observation
+controls pass. Tests also cover missing capabilities, bad restoration, wrong
+native references, nonfinite trajectories, unknown features, unlogged assignment
+modifications, schema rejection, CLI, subprocess comparison and pytest statuses.
+
+Implementation testing exposed two issues which were corrected before final
+measurements: independent float32 arithmetic needed a dtype-aware replay
+comparison floor, and returned mutable buffers needed copying. The frozen
+scientific diagnostic thresholds were not changed. A zero-torque constant-speed
+hinge test was corrected to use a gravity-loaded axis; exact flow correctly
+abstains because there is no truncation error to estimate.
+
+## Packaging and software test result
+
+The final suite has 34 passing tests (33 at the 11 November receipt, plus the isolation regression test; 34 pass on Python 3.12.3 with MuJoCo 4.0.1), including the optional MuJoCo tests. A wheel
+and source distribution were built. The wheel was installed without network
+access into a new virtual environment using existing system dependencies, and its
+CLI ran a copied custom-simulator example from /tmp, outside the source tree.
+The installed package imported without loading Gymnasium or MuJoCo, and pytest
+discovered its plugin entry point. This is package isolation, not a clean-room
+reinstallation of all third-party dependencies.
+
+Run `python validation/verify_receipt.py` from the repository to check retained
+hashes. The repository receipt is excluded from distributions to avoid a
+self-referential archive hash.
+
+## Reproduce
+
+From this package directory, after installing the package:
+
+```bash
+python -m pytest
+python -m build
+PYTHONPATH=validation python validation/run_controls.py \
+  --report /tmp/package_controls.json --summary /tmp/false_alarm_controls.json
+PYTHONPATH=src:validation /path/to/simulator/python validation/run_known_cases.py /tmp/paper_cases.json
+PYTHONPATH=src:validation python validation/run_release_cases.py \
+  --python-a /path/to/version-a/python --python-b /path/to/version-b/python \
+  --out /tmp/release_controls.json
+```
+
+`evidence/validation_receipt.json` records final execution checks and hashes.
+A GitHub Actions workflow is included (.github/workflows/tests.yml); hosted CI had
+not been run when this receipt was written. Stable publication still requires
+external usability tests, a larger held-out control set, a complete dependency
+licence inventory and additional version coverage. The name `simaudit` was
+unregistered on PyPI on 14 November 2026.