runboth 0.1.0__tar.gz

runboth-0.1.0/LICENSE.md

@@ -0,0 +1,93 @@
+# Functional Source License, Version 1.1, Apache 2.0 Future License
+
+## Abbreviation
+
+FSL-1.1-Apache-2.0
+
+## Notice
+
+Copyright 2026 Kyle Clouthier, Clouthier Simulation Labs
+
+## Terms and Conditions
+
+### Licensor ("We")
+
+The party offering the Software under these Terms and Conditions.
+
+### The Software
+
+The "Software" is each version of the software that we make available under these Terms and
+Conditions, as indicated by our inclusion of these Terms and Conditions with the Software.
+
+### License Grant
+
+Subject to your compliance with this License Grant and the Patents, Redistribution and Trademark
+clauses below, we hereby grant you the right to use, copy, modify, create derivative works,
+publicly perform, publicly display and redistribute the Software for any Permitted Purpose
+identified below.
+
+### Permitted Purpose
+
+A Permitted Purpose is any purpose other than a Competing Use. A Competing Use means making the
+Software available to others in a commercial product or service that:
+
+1. substitutes for the Software;
+2. substitutes for any other product or service we offer using the Software that exists as of the
+   date we make the Software available; or
+3. offers the same or substantially similar functionality as the Software.
+
+Permitted Purposes specifically include using the Software:
+
+1. for your internal use and access;
+2. for non-commercial education;
+3. for non-commercial research; and
+4. in connection with professional services that you provide to a licensee using the Software in
+   accordance with these Terms and Conditions.
+
+### Patents
+
+To the extent your use for a Permitted Purpose would necessarily infringe our patents, the license
+grant above includes a license under our patents. If you make a claim against any party that the
+Software infringes or contributes to the infringement of any patent, then your patent license to
+the Software ends immediately.
+
+### Redistribution
+
+The Terms and Conditions apply to all copies, modifications and derivatives of the Software.
+
+If you redistribute any copies, modifications or derivatives of the Software, you must include a
+copy of or a link to these Terms and Conditions and not remove any copyright notices provided in
+or with the Software.
+
+### Disclaimer
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT
+NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT.
+
+IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE SOFTWARE,
+INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES, EVEN IF WE HAVE BEEN INFORMED OF
+THEIR POSSIBILITY IN ADVANCE.
+
+### Trademarks
+
+Except for displaying the License Details and identifying us as the origin of the Software, you
+have no right under these Terms and Conditions to use our trademarks, trade names, service marks
+or product names.
+
+## Grant of Future License
+
+We hereby irrevocably grant you an additional license to use the Software under the Apache License,
+Version 2.0 that is effective on the second anniversary of the date we make the Software available.
+On or after that date, you may use the Software under the Apache License, Version 2.0, in which
+case the following will apply:
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this Software
+except in compliance with the License.
+
+You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License
+is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+or implied. See the License for the specific language governing permissions and limitations under
+the License.

runboth-0.1.0/PKG-INFO

@@ -0,0 +1,163 @@
+Metadata-Version: 2.4
+Name: runboth
+Version: 0.1.0
+Summary: An AI changed your code. RunBoth runs both versions and tells you what actually behaves differently, including the functions nobody touched.
+Author-email: Kyle Clouthier <kyle@simgen.dev>
+License: FSL-1.1-Apache-2.0
+Project-URL: Homepage, https://runboth.dev
+Keywords: differential testing,behaviour,refactoring,AI agents,code review
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Dynamic: license-file
+
+<p align="center">
+  <img src="brand/mark-128.png" width="72" alt="">
+</p>
+
+<h1 align="center">RunBoth</h1>
+
+<p align="center">
+  <b>An AI changed your code. RunBoth runs both versions and tells you what actually behaves
+  differently, including the functions nobody touched.</b>
+</p>
+
+<p align="center">
+  <a href="https://runboth.dev">runboth.dev</a> ·
+  <a href="HOW_IT_WORKS.md">How it works</a> ·
+  <a href="RED_TEAM_2026-09-12.md">Red team results</a>
+</p>
+
+---
+
+```
+$ git commit -m "refactor: tidy up the rates module"
+
+  BLOCKED: the behaviour changed and your message does not say so.
+
+    rate(100)
+      used to:  return 0.1
+      now:      return 0.0
+
+  AND 1 function you did NOT touch now behaves differently,
+  because it calls what you changed:
+
+    total(2.5, 100)          pkg/invoice.py
+      used to:  return 225.0
+      now:      return 250.0
+```
+
+## What it does
+
+It checks out both versions of your code, generates inputs for every changed function from its
+signature and from the constants mined out of its own bytecode, runs both versions in separate
+sandboxed subprocesses, and compares seven observation channels. When they disagree it hands you
+the exact input that separates them.
+
+No test suite required. No network calls. No AI model. No dependencies.
+
+## Install
+
+```bash
+pip install runboth                                  # once the first release is on PyPI
+pip install git+https://github.com/runboth/runboth   # works today
+runboth install-hook          # a commit-msg gate, silent unless behaviour moved
+```
+
+As a GitHub Action, running on your own runners:
+
+```yaml
+- uses: runboth/runboth@v0.1.0
+  with:
+    budget: 60
+```
+
+## Three verdicts, never two
+
+| verdict | meaning |
+|---|---|
+| `changed` | with a witness: the arguments, the old result, the new result |
+| `no_change at budget N` | N generated inputs found no difference across seven channels |
+| `abstained` | it could not be checked, and here is the reason |
+
+"Cannot tell" and "no difference" are different claims, and collapsing them into a green check is
+how tools end up lying. **RunBoth never says safe.**
+
+## The seven channels
+
+Return value · exception raised · warnings · stdout · stderr · argument mutation · object state.
+
+A narrow definition of behaviour does not under-report, it lies, because whatever sits outside the
+definition comes back as `no_change`.
+
+## What it is not
+
+**Not a model checker.** Kani and CBMC translate code into logic, let inputs be unconstrained
+symbols, and ask a solver whether a bad state is reachable within a bound. They return a proof.
+RunBoth executes real code on concrete values. It finds differences and reproduces them; it
+cannot prove absence, and never claims to.
+
+**Not mutation testing.** Mutation testing damages your code to score your test suite. RunBoth
+damages nothing; both versions come from your git history, and no test suite is needed.
+
+## Measured
+
+Red-teamed against eight public repositories it had never been tuned on, with an automated oracle
+built to catch the tool lying. **2,548 functions, zero false positives.** Full method and numbers
+in [RED_TEAM_2026-09-12.md](RED_TEAM_2026-09-12.md).
+
+| repo | layout | functions | abstained | median/commit |
+|---|---|---|---|---|
+| boltons | flat | 268 | 0.0% | 7.9s |
+| sqlparse | flat | 230 | 0.9% | 14.3s |
+| arrow | flat | 283 | 1.4% | 118s |
+| cachetools | src/ | 325 | 1.8% | 60.9s |
+| more-itertools | flat | 843 | 4.4% | 102.5s |
+| packaging | src/ | 78 | 5.1% | 0.2s |
+| pluggy | src/ | 157 | 6.4% | 33.5s |
+| tenacity | async | 364 | 14.3% | 192.8s |
+
+An adversarial corpus of 22 functions written specifically to induce false positives (object
+addresses in default `repr`, `datetime.now`, unseeded `random`, `uuid4`, `os.getpid`, set
+iteration order, mutable defaults, generators, `__file__` paths) produced none.
+
+## Environment variables
+
+| variable | effect |
+|---|---|
+| `RUNBOTH_SKIP=1` | let a commit through without checking it |
+| `RUNBOTH_BUDGET` | generated inputs per function (gate default 80) |
+| `RUNBOTH_WORKERS` | parallel adjudications, default 4 |
+| `RUNBOTH_ALL_PATHS=1` | also check tests, benchmarks, docs and task runners |
+| `RUNBOTH_ENGINE` | engine directory, if the hook cannot resolve it |
+
+`git commit --no-verify` also bypasses the gate, and the gate says so itself when it blocks.
+
+## Honest limits
+
+- Function-level checking is **Python only**. Changed files in other languages are named
+  explicitly rather than passed over quietly.
+- Sampling finds differences; it cannot prove their absence.
+- Nondeterministic, too-slow, or unconstructible functions abstain **with a reason**, and are
+  never counted as passing.
+- The sandbox contains accidents: resource limits, network blocked, filesystem writes blocked. It
+  is **not** a security boundary against hostile code, and no pure-Python sandbox is.
+
+## Development
+
+```bash
+pip install -e .
+pytest tests/ -q
+runboth selftest          # the control suites, half of which must fail
+```
+
+## Licence
+
+[FSL-1.1-Apache-2.0](LICENSE.md). Free for every use except building a competing product, and it
+converts to plain Apache 2.0 two years after each release.
+
+Built by Kyle Clouthier at Clouthier Simulation Labs.

runboth-0.1.0/README.md

@@ -0,0 +1,146 @@
+<p align="center">
+  <img src="brand/mark-128.png" width="72" alt="">
+</p>
+
+<h1 align="center">RunBoth</h1>
+
+<p align="center">
+  <b>An AI changed your code. RunBoth runs both versions and tells you what actually behaves
+  differently, including the functions nobody touched.</b>
+</p>
+
+<p align="center">
+  <a href="https://runboth.dev">runboth.dev</a> ·
+  <a href="HOW_IT_WORKS.md">How it works</a> ·
+  <a href="RED_TEAM_2026-09-12.md">Red team results</a>
+</p>
+
+---
+
+```
+$ git commit -m "refactor: tidy up the rates module"
+
+  BLOCKED: the behaviour changed and your message does not say so.
+
+    rate(100)
+      used to:  return 0.1
+      now:      return 0.0
+
+  AND 1 function you did NOT touch now behaves differently,
+  because it calls what you changed:
+
+    total(2.5, 100)          pkg/invoice.py
+      used to:  return 225.0
+      now:      return 250.0
+```
+
+## What it does
+
+It checks out both versions of your code, generates inputs for every changed function from its
+signature and from the constants mined out of its own bytecode, runs both versions in separate
+sandboxed subprocesses, and compares seven observation channels. When they disagree it hands you
+the exact input that separates them.
+
+No test suite required. No network calls. No AI model. No dependencies.
+
+## Install
+
+```bash
+pip install runboth                                  # once the first release is on PyPI
+pip install git+https://github.com/runboth/runboth   # works today
+runboth install-hook          # a commit-msg gate, silent unless behaviour moved
+```
+
+As a GitHub Action, running on your own runners:
+
+```yaml
+- uses: runboth/runboth@v0.1.0
+  with:
+    budget: 60
+```
+
+## Three verdicts, never two
+
+| verdict | meaning |
+|---|---|
+| `changed` | with a witness: the arguments, the old result, the new result |
+| `no_change at budget N` | N generated inputs found no difference across seven channels |
+| `abstained` | it could not be checked, and here is the reason |
+
+"Cannot tell" and "no difference" are different claims, and collapsing them into a green check is
+how tools end up lying. **RunBoth never says safe.**
+
+## The seven channels
+
+Return value · exception raised · warnings · stdout · stderr · argument mutation · object state.
+
+A narrow definition of behaviour does not under-report, it lies, because whatever sits outside the
+definition comes back as `no_change`.
+
+## What it is not
+
+**Not a model checker.** Kani and CBMC translate code into logic, let inputs be unconstrained
+symbols, and ask a solver whether a bad state is reachable within a bound. They return a proof.
+RunBoth executes real code on concrete values. It finds differences and reproduces them; it
+cannot prove absence, and never claims to.
+
+**Not mutation testing.** Mutation testing damages your code to score your test suite. RunBoth
+damages nothing; both versions come from your git history, and no test suite is needed.
+
+## Measured
+
+Red-teamed against eight public repositories it had never been tuned on, with an automated oracle
+built to catch the tool lying. **2,548 functions, zero false positives.** Full method and numbers
+in [RED_TEAM_2026-09-12.md](RED_TEAM_2026-09-12.md).
+
+| repo | layout | functions | abstained | median/commit |
+|---|---|---|---|---|
+| boltons | flat | 268 | 0.0% | 7.9s |
+| sqlparse | flat | 230 | 0.9% | 14.3s |
+| arrow | flat | 283 | 1.4% | 118s |
+| cachetools | src/ | 325 | 1.8% | 60.9s |
+| more-itertools | flat | 843 | 4.4% | 102.5s |
+| packaging | src/ | 78 | 5.1% | 0.2s |
+| pluggy | src/ | 157 | 6.4% | 33.5s |
+| tenacity | async | 364 | 14.3% | 192.8s |
+
+An adversarial corpus of 22 functions written specifically to induce false positives (object
+addresses in default `repr`, `datetime.now`, unseeded `random`, `uuid4`, `os.getpid`, set
+iteration order, mutable defaults, generators, `__file__` paths) produced none.
+
+## Environment variables
+
+| variable | effect |
+|---|---|
+| `RUNBOTH_SKIP=1` | let a commit through without checking it |
+| `RUNBOTH_BUDGET` | generated inputs per function (gate default 80) |
+| `RUNBOTH_WORKERS` | parallel adjudications, default 4 |
+| `RUNBOTH_ALL_PATHS=1` | also check tests, benchmarks, docs and task runners |
+| `RUNBOTH_ENGINE` | engine directory, if the hook cannot resolve it |
+
+`git commit --no-verify` also bypasses the gate, and the gate says so itself when it blocks.
+
+## Honest limits
+
+- Function-level checking is **Python only**. Changed files in other languages are named
+  explicitly rather than passed over quietly.
+- Sampling finds differences; it cannot prove their absence.
+- Nondeterministic, too-slow, or unconstructible functions abstain **with a reason**, and are
+  never counted as passing.
+- The sandbox contains accidents: resource limits, network blocked, filesystem writes blocked. It
+  is **not** a security boundary against hostile code, and no pure-Python sandbox is.
+
+## Development
+
+```bash
+pip install -e .
+pytest tests/ -q
+runboth selftest          # the control suites, half of which must fail
+```
+
+## Licence
+
+[FSL-1.1-Apache-2.0](LICENSE.md). Free for every use except building a competing product, and it
+converts to plain Apache 2.0 two years after each release.
+
+Built by Kyle Clouthier at Clouthier Simulation Labs.

runboth-0.1.0/pyproject.toml

@@ -0,0 +1,42 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "runboth"
+version = "0.1.0"
+description = "An AI changed your code. RunBoth runs both versions and tells you what actually behaves differently, including the functions nobody touched."
+readme = "README.md"
+requires-python = ">=3.10"
+# FSL-1.1-Apache-2.0: source-available, free for every use except building a competing
+# product, and it converts to plain Apache 2.0 automatically two years after each release.
+# Chosen 2026-09-12 over the MIT OR Apache-2.0 that was declared here before, because nothing
+# is published yet so the change costs nothing, and because a permissive licence on a product
+# whose whole value is one narrow mechanism invites exactly one outcome. Reversible: relicense
+# is the author's to make right up until the first public release.
+license = { text = "FSL-1.1-Apache-2.0" }
+authors = [{ name = "Kyle Clouthier", email = "kyle@simgen.dev" }]
+keywords = ["differential testing", "behaviour", "refactoring", "AI agents", "code review"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Developers",
+  "Programming Language :: Python :: 3",
+  "Topic :: Software Development :: Testing",
+]
+
+[project.urls]
+Homepage = "https://runboth.dev"
+
+[project.scripts]
+runboth = "runboth.cli:main"
+
+# The engine is a flat directory of modules that import each other by bare name
+# (`import sandbox`), and every entry point already puts its own directory on sys.path.
+# Installing that directory AS the `runboth` package, with an __init__ that does the same
+# sys.path insert, keeps those imports working unchanged whether the tree is run from a
+# checkout or from site-packages.
+[tool.setuptools]
+packages = ["runboth"]
+
+[tool.setuptools.package-dir]
+runboth = "runboth"

runboth-0.1.0/runboth/__init__.py

@@ -0,0 +1,30 @@
+"""runboth: behaviour adjudication for AI-written code.
+
+This directory is a flat set of modules that import one another by bare name. When it is
+installed as the `runboth` package, this file puts the package directory itself on sys.path so
+those imports resolve to the installed copies. It is the same thing `cli.py` already does when
+run from a checkout, made unconditional so `from runboth.cli import main` works too.
+"""
+import sys as _sys
+from pathlib import Path as _Path
+
+_here = str(_Path(__file__).resolve().parent)
+if _here not in _sys.path:
+    _sys.path.insert(0, _here)
+
+__all__ = ["main", "engine_dir"]
+
+
+def engine_dir():
+    """Where the engine modules live, wherever this package was installed or checked out.
+
+    The git hook uses this to locate `precommit.py` at RUN time instead of having an absolute
+    path baked into it at INSTALL time. Moving the project used to silently break every hook
+    already installed, which is exactly the class of failure this tool exists to catch.
+    """
+    return _here
+
+
+def main():
+    from cli import main as _main
+    return _main()