plainproof 0.1.0__tar.gz → 0.2.0__tar.gz

{plainproof-0.1.0 → plainproof-0.2.0}/.gitignore

@@ -8,3 +8,4 @@ dist/
 build/
 *.egg-info/
 .venv/
+src/plainproof/payload/

{plainproof-0.1.0 → plainproof-0.2.0}/CITATION.cff

@@ -5,3 +5,10 @@ authors:
   - family-names: Vettrivel
     given-names: Vishnu
 license: MIT
+version: 0.2.0
+doi: 10.5281/zenodo.23647649
+identifiers:
+  - type: doi
+    value: 10.5281/zenodo.23647650
+    description: version 0.1.0
+repository-code: https://github.com/cloudronin/plainproof

plainproof-0.2.0/PKG-INFO

@@ -0,0 +1,102 @@
+Metadata-Version: 2.4
+Name: plainproof
+Version: 0.2.0
+Summary: Reproduction tool for the paper 'How many follow-up questions distinguish genuine understanding from apparent mastery?'. Not a general library. Its sealed checks can stop a run on purpose.
+Author: Vishnu Vettrivel
+License-Expression: MIT
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=2.0
+Requires-Dist: scipy>=1.13
+Provides-Extra: run
+Requires-Dist: scikit-learn>=1.5; extra == "run"
+Requires-Dist: torch>=2.4; extra == "run"
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == "test"
+Requires-Dist: pytest-xdist>=3.5; extra == "test"
+Dynamic: license-file
+
+# plainproof
+
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.23647650.svg)](https://doi.org/10.5281/zenodo.23647650)
+
+Archived on Zenodo: version 0.1.0 is DOI [10.5281/zenodo.23647650](https://doi.org/10.5281/zenodo.23647650); all versions are DOI [10.5281/zenodo.23647649](https://doi.org/10.5281/zenodo.23647649). Also on PyPI: `pip install plainproof`.
+
+Reproduction package for **"How many follow-up questions distinguish genuine understanding from apparent mastery?"** (EDM 2027 submission, simulation study). It asks how many adaptive follow-up questions a classification test needs to tell genuine understanding from apparent mastery at a calibrated confidence target, and how that changes as a learner outsources a share *d* of the follow-ups to an AI. A second, sealed study (E0b) tests whether planted hard probe items separate outsourcing learners from honest ones.
+
+Everything here is CPU-only and needs no paid service. Code: MIT (`LICENSE`). Docs, figures and tables: CC BY 4.0 (`LICENSE-docs.md`).
+
+## From install to the figure (about 10 minutes)
+
+```sh
+pip install 'plainproof[test,run]'
+plainproof init ~/plainproof        # writes the working directory: sealed scripts, freezes, receipts, docs, tests
+cd ~/plainproof
+plainproof test                     # 43 tests: SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, probes
+plainproof verify                   # every sealed file against both freezes (undoes the documented substitutions, then sha256)
+plainproof fetch heldout            # the sealed held-out outputs, hash-checked
+plainproof figures                  # tables and figures from the held-out summaries (needs R 4.x on PATH; base graphics only)
+```
+
+The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`, byte-identical to the published copies. `plainproof init` writes the files in the exact layout the sealed scripts expect; nothing sealed changes. Every other command runs inside that directory, or takes `--root DIR`. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; the `scripts/rscript.sh` wrapper, which `plainproof figures` calls, is the one to use.
+
+## Full rerun (about 4 hours of CPU)
+
+Only if you want to regenerate everything, including the held-out outputs.
+
+```sh
+plainproof fetch bobcat     # BOBCAT (MIT) at its pinned commit, with the preprocessed ASSISTments 2009-2010 file
+plainproof fit              # 2PL item fit on the development half of learners; checked against the receipt's hash
+plainproof retrain          # BOBCAT checkpoint, about 2 hours (author defaults, fold 1, seed 221), compared with the original run
+plainproof run e0           # E0 held-out run and summary; refuses to overwrite an existing cell
+plainproof run e0b          # E0b held-out run and summary
+```
+
+The ASSISTments terms of use forbid passing the data on, so this package never contains it: `fetch bobcat` downloads it from its source, and you accept those terms yourself. Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop `fit`; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical stops `run e0`. In either case, use the published held-out outputs (`plainproof fetch heldout`). The development stages (`scripts/edm_v2_e0/study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
+
+## Fingerprints (sha256)
+
+| What | sha256 |
+|---|---|
+| E0 sealed freeze `reports/edm-v2-e0/freeze.json` | `a0f8ce404f2b5f7e671e2503a33b4554040e80714f6053553507e0972acdbcdc` |
+| E0b sealed freeze `reports/edm-v2-e0b/freeze.json` | `3fa0b8d457d813f59abd5255f9303e44deb2119ee79d6a9b92439a6c1cbf58ed` |
+| E0 held-out summary (`e0-heldout.tar` → `heldout/summary.json`) | `aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90` |
+| E0b held-out summary (`e0b-heldout.tar` → `heldout/summary.json`) | `617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f` |
+| `e0-heldout.tar` (release asset) | `bd06a11eb7163da102cad6ea0df12ea79e52ff94aab1dc35d28ec3910af2be22` |
+| `e0b-heldout.tar` (release asset) | `68219f228c41da972fe4c8f92c218e01f85f08eed864185f237cecda05314bbf` |
+| ASSISTments 2009 file at BOBCAT e6b6245e | `c07ca9fa0f08207b06488cec527725938fe96e81e3c1762839e1ad0b84f974cc` |
+| 2PL item fit (regenerated in step b) | `12a3b167cd9c9b78b3c3c7faf71b2e464b0943a454f6418faf0f99b1114cb7fc` |
+| catIrt 0.5.1 CRAN binary used for the fixture | `f5418f447e91d184791e668bd2a135ea5da40c981d4a76826018295b6ea9f079` |
+
+## What is in here
+
+- `src/plainproof/`: item model, SPRT arm, exact stopping time, simulator (uniform and uncertainty-triggered outsourcing), item fit, signatures, planted probes (with the give-up model), matched-coverage scoring.
+- `src/plainproof/cli.py`: the `plainproof` command. `setup.py` packs the working directory into the wheel for `plainproof init`.
+- `scripts/edm_v2_e0/`, `scripts/edm_v2_e0b/`: the runners, report builders and R plots. `scripts/published_cat_reference_v1/`: the BOBCAT loading code the runners call.
+- `reports/`: both sealed freezes, receipts, figures and tables. `docs/`: the E0 protocol and the pre-registration log (verbatim extract).
+- `substitutions.json`: the only differences between this code and the sealed originals (package name, file locations, data paths). `scripts/verify_frozen.py` undoes them and checks every hash. `scripts/edm_v2_e0/r_env.sh` is kept verbatim as the record of the original R runtime; use `scripts/rscript.sh` instead.
+
+## Recorded departures
+
+All are in `docs/PREREG_EDM_V2.md`: the SPRT tuning rule was revised after its first development pass (labelled post hoc); SPRT's calibration map was refitted once on full paths before the freeze; the E0 held-out run exceeded its 2 GiB memory cap (no result depends on it; the runner now enforces the cap); E0b was sealed before its pilot table had been reviewed (no result affected).
+
+## Appendix: the raw script calls
+
+What each `plainproof` command runs, from the working directory (`REPRO_DATA` defaults to `./data`):
+
+```sh
+python -m pytest -n auto -q tests                                       # plainproof test
+python scripts/verify_frozen.py                                         # plainproof verify
+ZENODO_URL=https://github.com/cloudronin/plainproof/releases/download/v0.1.0 bash scripts/fetch_heldout.sh   # plainproof fetch heldout
+python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json  # plainproof figures (E0 tables)
+bash scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png <E0 summary sha256>
+python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
+bash scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png <E0b summary sha256>
+bash scripts/bobcat/fetch_bobcat.sh                                     # plainproof fetch bobcat
+python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit      # plainproof fit
+python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py   # plainproof retrain
+python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout   # plainproof run e0
+python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout       # plainproof run e0b
+scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt   # regenerate the catIrt fixture (R with catIrt 0.5.1)
+```

plainproof-0.2.0/README.md

@@ -0,0 +1,83 @@
+# plainproof
+
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.23647650.svg)](https://doi.org/10.5281/zenodo.23647650)
+
+Archived on Zenodo: version 0.1.0 is DOI [10.5281/zenodo.23647650](https://doi.org/10.5281/zenodo.23647650); all versions are DOI [10.5281/zenodo.23647649](https://doi.org/10.5281/zenodo.23647649). Also on PyPI: `pip install plainproof`.
+
+Reproduction package for **"How many follow-up questions distinguish genuine understanding from apparent mastery?"** (EDM 2027 submission, simulation study). It asks how many adaptive follow-up questions a classification test needs to tell genuine understanding from apparent mastery at a calibrated confidence target, and how that changes as a learner outsources a share *d* of the follow-ups to an AI. A second, sealed study (E0b) tests whether planted hard probe items separate outsourcing learners from honest ones.
+
+Everything here is CPU-only and needs no paid service. Code: MIT (`LICENSE`). Docs, figures and tables: CC BY 4.0 (`LICENSE-docs.md`).
+
+## From install to the figure (about 10 minutes)
+
+```sh
+pip install 'plainproof[test,run]'
+plainproof init ~/plainproof        # writes the working directory: sealed scripts, freezes, receipts, docs, tests
+cd ~/plainproof
+plainproof test                     # 43 tests: SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, probes
+plainproof verify                   # every sealed file against both freezes (undoes the documented substitutions, then sha256)
+plainproof fetch heldout            # the sealed held-out outputs, hash-checked
+plainproof figures                  # tables and figures from the held-out summaries (needs R 4.x on PATH; base graphics only)
+```
+
+The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`, byte-identical to the published copies. `plainproof init` writes the files in the exact layout the sealed scripts expect; nothing sealed changes. Every other command runs inside that directory, or takes `--root DIR`. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; the `scripts/rscript.sh` wrapper, which `plainproof figures` calls, is the one to use.
+
+## Full rerun (about 4 hours of CPU)
+
+Only if you want to regenerate everything, including the held-out outputs.
+
+```sh
+plainproof fetch bobcat     # BOBCAT (MIT) at its pinned commit, with the preprocessed ASSISTments 2009-2010 file
+plainproof fit              # 2PL item fit on the development half of learners; checked against the receipt's hash
+plainproof retrain          # BOBCAT checkpoint, about 2 hours (author defaults, fold 1, seed 221), compared with the original run
+plainproof run e0           # E0 held-out run and summary; refuses to overwrite an existing cell
+plainproof run e0b          # E0b held-out run and summary
+```
+
+The ASSISTments terms of use forbid passing the data on, so this package never contains it: `fetch bobcat` downloads it from its source, and you accept those terms yourself. Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop `fit`; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical stops `run e0`. In either case, use the published held-out outputs (`plainproof fetch heldout`). The development stages (`scripts/edm_v2_e0/study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
+
+## Fingerprints (sha256)
+
+| What | sha256 |
+|---|---|
+| E0 sealed freeze `reports/edm-v2-e0/freeze.json` | `a0f8ce404f2b5f7e671e2503a33b4554040e80714f6053553507e0972acdbcdc` |
+| E0b sealed freeze `reports/edm-v2-e0b/freeze.json` | `3fa0b8d457d813f59abd5255f9303e44deb2119ee79d6a9b92439a6c1cbf58ed` |
+| E0 held-out summary (`e0-heldout.tar` → `heldout/summary.json`) | `aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90` |
+| E0b held-out summary (`e0b-heldout.tar` → `heldout/summary.json`) | `617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f` |
+| `e0-heldout.tar` (release asset) | `bd06a11eb7163da102cad6ea0df12ea79e52ff94aab1dc35d28ec3910af2be22` |
+| `e0b-heldout.tar` (release asset) | `68219f228c41da972fe4c8f92c218e01f85f08eed864185f237cecda05314bbf` |
+| ASSISTments 2009 file at BOBCAT e6b6245e | `c07ca9fa0f08207b06488cec527725938fe96e81e3c1762839e1ad0b84f974cc` |
+| 2PL item fit (regenerated in step b) | `12a3b167cd9c9b78b3c3c7faf71b2e464b0943a454f6418faf0f99b1114cb7fc` |
+| catIrt 0.5.1 CRAN binary used for the fixture | `f5418f447e91d184791e668bd2a135ea5da40c981d4a76826018295b6ea9f079` |
+
+## What is in here
+
+- `src/plainproof/`: item model, SPRT arm, exact stopping time, simulator (uniform and uncertainty-triggered outsourcing), item fit, signatures, planted probes (with the give-up model), matched-coverage scoring.
+- `src/plainproof/cli.py`: the `plainproof` command. `setup.py` packs the working directory into the wheel for `plainproof init`.
+- `scripts/edm_v2_e0/`, `scripts/edm_v2_e0b/`: the runners, report builders and R plots. `scripts/published_cat_reference_v1/`: the BOBCAT loading code the runners call.
+- `reports/`: both sealed freezes, receipts, figures and tables. `docs/`: the E0 protocol and the pre-registration log (verbatim extract).
+- `substitutions.json`: the only differences between this code and the sealed originals (package name, file locations, data paths). `scripts/verify_frozen.py` undoes them and checks every hash. `scripts/edm_v2_e0/r_env.sh` is kept verbatim as the record of the original R runtime; use `scripts/rscript.sh` instead.
+
+## Recorded departures
+
+All are in `docs/PREREG_EDM_V2.md`: the SPRT tuning rule was revised after its first development pass (labelled post hoc); SPRT's calibration map was refitted once on full paths before the freeze; the E0 held-out run exceeded its 2 GiB memory cap (no result depends on it; the runner now enforces the cap); E0b was sealed before its pilot table had been reviewed (no result affected).
+
+## Appendix: the raw script calls
+
+What each `plainproof` command runs, from the working directory (`REPRO_DATA` defaults to `./data`):
+
+```sh
+python -m pytest -n auto -q tests                                       # plainproof test
+python scripts/verify_frozen.py                                         # plainproof verify
+ZENODO_URL=https://github.com/cloudronin/plainproof/releases/download/v0.1.0 bash scripts/fetch_heldout.sh   # plainproof fetch heldout
+python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json  # plainproof figures (E0 tables)
+bash scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png <E0 summary sha256>
+python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
+bash scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png <E0b summary sha256>
+bash scripts/bobcat/fetch_bobcat.sh                                     # plainproof fetch bobcat
+python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit      # plainproof fit
+python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py   # plainproof retrain
+python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout   # plainproof run e0
+python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout       # plainproof run e0b
+scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt   # regenerate the catIrt fixture (R with catIrt 0.5.1)
+```

{plainproof-0.1.0 → plainproof-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "plainproof"
-version = "0.1.0"
+version = "0.2.0"
 description = "Reproduction tool for the paper 'How many follow-up questions distinguish genuine understanding from apparent mastery?'. Not a general library. Its sealed checks can stop a run on purpose."
 readme = "README.md"
 license = "MIT"

{plainproof-0.1.0 → plainproof-0.2.0}/scripts/fetch_heldout.sh

@@ -1,7 +1,8 @@
 #!/bin/bash
-# Download the sealed held-out outputs from Zenodo and check their sha256 before use.
+# Download the sealed held-out outputs and check their sha256 before use. Default source: the v0.1.0 GitHub release assets
+# (cited by DOI 10.5281/zenodo.23647650); set ZENODO_URL (or any mirror base URL) to fetch from elsewhere.
 set -euo pipefail
-DOI_URL=${ZENODO_URL:?set ZENODO_URL to the Zenodo record's file base URL (DOI: to be assigned)}
+DOI_URL=${ZENODO_URL:-https://github.com/cloudronin/plainproof/releases/download/v0.1.0}
 D=${REPRO_DATA:-data}; mkdir -p "$D/edm-v2-e0" "$D/edm-v2-e0b"
 for f in e0-heldout.tar e0b-heldout.tar; do curl -fsSL -o "$D/$f" "$DOI_URL/$f"; done
 cd "$D" && printf '%s  %s\n%s  %s\n' bd06a11eb7163da102cad6ea0df12ea79e52ff94aab1dc35d28ec3910af2be22 e0-heldout.tar 68219f228c41da972fe4c8f92c218e01f85f08eed864185f237cecda05314bbf e0b-heldout.tar | shasum -a 256 -c

plainproof-0.2.0/setup.py

@@ -0,0 +1,31 @@
+"""Build hook: copy the reproduction working directory into the package as data, so `plainproof init` can write it
+out after `pip install plainproof` alone. The copied files are the repository's own files, unchanged."""
+import shutil
+from pathlib import Path
+
+from setuptools import setup
+from setuptools.command.build_py import build_py
+
+PAYLOAD = ['scripts', 'reports', 'docs', 'tests', 'data/edm-v2-e0/study', 'substitutions.json', 'README.md', 'LICENSE',
+           'LICENSE-docs.md', 'CITATION.cff', 'requirements.txt']
+
+
+class with_payload(build_py):
+    def run(self):
+        super().run()
+        here = Path(__file__).resolve().parent
+        dest = Path(self.build_lib) / 'plainproof' / 'payload'
+        if dest.exists():
+            shutil.rmtree(dest)
+        for item in PAYLOAD:
+            src = here / item
+            if not src.exists():
+                raise SystemExit(f'payload item missing from the source tree: {item}')
+            if src.is_dir():
+                shutil.copytree(src, dest / item, ignore=shutil.ignore_patterns('__pycache__', '*.pyc'))
+            else:
+                (dest / item).parent.mkdir(parents=True, exist_ok=True)
+                shutil.copy2(src, dest / item)
+
+
+setup(cmdclass={'build_py': with_payload})

plainproof-0.2.0/src/plainproof/cli.py

@@ -0,0 +1,121 @@
+"""plainproof: the command-line front end of this reproduction package.
+
+`plainproof init DIR` writes a working directory from the package (sealed scripts, both freezes, receipts,
+substitutions.json, R scripts, docs, tests) in the exact layout the sealed scripts expect; nothing sealed changes.
+Every other command runs inside that directory, or takes --root DIR.
+
+  plainproof init DIR                     write the working directory (refuses a non-empty DIR)
+  plainproof test                         the 43 offline tests
+  plainproof verify                       check every sealed file against both freezes
+  plainproof fetch heldout|bobcat|all     the sealed held-out outputs, and/or BOBCAT + the ASSISTments file (hash-checked)
+  plainproof figures                      rebuild tables and figures from the held-out summaries (needs R for the plots)
+  plainproof fit                          refit the 2PL item parameters on ASSISTments development learners
+  plainproof retrain                      retrain the BOBCAT checkpoint, then compare it with the original run
+  plainproof run e0|e0b                   held-out run and summary (refuses to overwrite an existing cell)
+"""
+import argparse
+import hashlib
+import os
+import shutil
+import subprocess
+import sys
+from pathlib import Path
+
+# The sealed held-out outputs. Their sha256 values are checked by scripts/fetch_heldout.sh before use.
+HELDOUT_BASE_URL = 'https://github.com/cloudronin/plainproof/releases/download/v0.1.0'
+PAYLOAD = Path(__file__).resolve().parent / 'payload'
+
+
+def find_root(given):
+    root = Path(given or os.getcwd()).resolve()
+    if not (root / 'substitutions.json').exists() or not (root / 'reports' / 'edm-v2-e0' / 'freeze.json').exists():
+        raise SystemExit(f'{root} is not a plainproof working directory; run `plainproof init DIR` and cd there, or pass --root')
+    return root
+
+
+def sh(root, *cmd, env_extra=None):
+    env = dict(os.environ)
+    env.setdefault('REPRO_DATA', str(root / 'data'))
+    env.update(env_extra or {})
+    print('+', ' '.join(str(c) for c in cmd), flush=True)
+    r = subprocess.run([str(c) for c in cmd], cwd=root, env=env)
+    if r.returncode:
+        raise SystemExit(r.returncode)
+
+
+def sha(p):
+    return hashlib.sha256(Path(p).read_bytes()).hexdigest()
+
+
+def init(target):
+    target = Path(target).expanduser().resolve()
+    if target.exists() and any(target.iterdir()):
+        raise SystemExit(f'{target} is not empty; choose a new directory')
+    if not PAYLOAD.exists():
+        raise SystemExit('this installation has no payload; reinstall plainproof from the wheel or sdist')
+    shutil.copytree(PAYLOAD, target, dirs_exist_ok=True)
+    # The sealed scripts import the package from <root>/src and hash its modules there, so the package is written
+    # out too, byte for byte from this installation.
+    pkg = target / 'src' / 'plainproof'
+    pkg.mkdir(parents=True)
+    for f in Path(__file__).resolve().parent.glob('*.py'):
+        shutil.copy2(f, pkg / f.name)
+    for d in ('data/edm-v2-e0', 'data/edm-v2-e0b'):
+        (target / d).mkdir(parents=True, exist_ok=True)
+    for f in target.rglob('*.sh'):
+        f.chmod(0o755)
+    print(f'wrote {target}\nnext: cd {target} && plainproof test && plainproof verify && plainproof fetch heldout && plainproof figures')
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(prog='plainproof', description=__doc__.split('\n\n')[0])
+    ap.add_argument('--root', help='working directory (default: current directory)')
+    sub = ap.add_subparsers(dest='cmd', required=True)
+    sub.add_parser('init').add_argument('dir')
+    sub.add_parser('test')
+    sub.add_parser('verify')
+    sub.add_parser('fetch').add_argument('what', choices=('heldout', 'bobcat', 'all'))
+    sub.add_parser('figures')
+    sub.add_parser('fit')
+    sub.add_parser('retrain')
+    sub.add_parser('run').add_argument('study', choices=('e0', 'e0b'))
+    a = ap.parse_args(argv)
+    if a.cmd == 'init':
+        return init(a.dir)
+    root = find_root(a.root)
+    py = sys.executable
+    data = Path(os.environ.get('REPRO_DATA', root / 'data'))
+    if a.cmd == 'test':
+        sh(root, py, '-m', 'pytest', '-n', 'auto', '-q', '-p', 'no:cacheprovider', 'tests')
+    elif a.cmd == 'verify':
+        sh(root, py, 'scripts/verify_frozen.py')
+    elif a.cmd == 'fetch':
+        if a.what in ('heldout', 'all'):
+            sh(root, 'bash', 'scripts/fetch_heldout.sh', env_extra={'ZENODO_URL': os.environ.get('ZENODO_URL', HELDOUT_BASE_URL)})
+        if a.what in ('bobcat', 'all'):
+            sh(root, 'bash', 'scripts/bobcat/fetch_bobcat.sh')
+    elif a.cmd == 'figures':
+        s0, s1 = data / 'edm-v2-e0' / 'heldout' / 'summary.json', data / 'edm-v2-e0b' / 'heldout' / 'summary.json'
+        for s in (s0, s1):
+            if not s.exists():
+                raise SystemExit(f'{s} missing; run `plainproof fetch heldout` first')
+        sh(root, py, 'scripts/edm_v2_e0/report.py', s0)
+        sh(root, 'bash', 'scripts/rscript.sh', 'scripts/edm_v2_e0/plot_error_vs_d.R', 'reports/edm-v2-e0/error-vs-d.csv',
+           'reports/edm-v2-e0/figures/error-vs-d.png', sha(s0))
+        sh(root, py, 'scripts/edm_v2_e0b/report.py', s1)
+        sh(root, 'bash', 'scripts/rscript.sh', 'scripts/edm_v2_e0b/plot_probe_auc.R', 'reports/edm-v2-e0b/probe-auc-vs-d.csv',
+           'reports/edm-v2-e0b/figures/probe-auc-vs-d.png', sha(s1))
+    elif a.cmd == 'fit':
+        sh(root, py, 'scripts/edm_v2_e0/fit_assist2009.py', data / 'edm-v2-e0' / 'item-fit')
+    elif a.cmd == 'retrain':
+        sh(root, py, 'scripts/bobcat/retrain.py')
+        sh(root, py, 'scripts/bobcat/check_checkpoint.py')
+    elif a.cmd == 'run':
+        out = data / a.study.replace('e0', 'edm-v2-e0', 1) / 'heldout'
+        script = 'scripts/edm_v2_e0/heldout.py' if a.study == 'e0' else 'scripts/edm_v2_e0b/run.py'
+        sh(root, py, script, 'run', out)
+        sh(root, py, script, 'summarize', out)
+
+
+if __name__ == '__main__':
+    main()

plainproof-0.2.0/src/plainproof.egg-info/PKG-INFO

@@ -0,0 +1,102 @@
+Metadata-Version: 2.4
+Name: plainproof
+Version: 0.2.0
+Summary: Reproduction tool for the paper 'How many follow-up questions distinguish genuine understanding from apparent mastery?'. Not a general library. Its sealed checks can stop a run on purpose.
+Author: Vishnu Vettrivel
+License-Expression: MIT
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=2.0
+Requires-Dist: scipy>=1.13
+Provides-Extra: run
+Requires-Dist: scikit-learn>=1.5; extra == "run"
+Requires-Dist: torch>=2.4; extra == "run"
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == "test"
+Requires-Dist: pytest-xdist>=3.5; extra == "test"
+Dynamic: license-file
+
+# plainproof
+
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.23647650.svg)](https://doi.org/10.5281/zenodo.23647650)
+
+Archived on Zenodo: version 0.1.0 is DOI [10.5281/zenodo.23647650](https://doi.org/10.5281/zenodo.23647650); all versions are DOI [10.5281/zenodo.23647649](https://doi.org/10.5281/zenodo.23647649). Also on PyPI: `pip install plainproof`.
+
+Reproduction package for **"How many follow-up questions distinguish genuine understanding from apparent mastery?"** (EDM 2027 submission, simulation study). It asks how many adaptive follow-up questions a classification test needs to tell genuine understanding from apparent mastery at a calibrated confidence target, and how that changes as a learner outsources a share *d* of the follow-ups to an AI. A second, sealed study (E0b) tests whether planted hard probe items separate outsourcing learners from honest ones.
+
+Everything here is CPU-only and needs no paid service. Code: MIT (`LICENSE`). Docs, figures and tables: CC BY 4.0 (`LICENSE-docs.md`).
+
+## From install to the figure (about 10 minutes)
+
+```sh
+pip install 'plainproof[test,run]'
+plainproof init ~/plainproof        # writes the working directory: sealed scripts, freezes, receipts, docs, tests
+cd ~/plainproof
+plainproof test                     # 43 tests: SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, probes
+plainproof verify                   # every sealed file against both freezes (undoes the documented substitutions, then sha256)
+plainproof fetch heldout            # the sealed held-out outputs, hash-checked
+plainproof figures                  # tables and figures from the held-out summaries (needs R 4.x on PATH; base graphics only)
+```
+
+The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`, byte-identical to the published copies. `plainproof init` writes the files in the exact layout the sealed scripts expect; nothing sealed changes. Every other command runs inside that directory, or takes `--root DIR`. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; the `scripts/rscript.sh` wrapper, which `plainproof figures` calls, is the one to use.
+
+## Full rerun (about 4 hours of CPU)
+
+Only if you want to regenerate everything, including the held-out outputs.
+
+```sh
+plainproof fetch bobcat     # BOBCAT (MIT) at its pinned commit, with the preprocessed ASSISTments 2009-2010 file
+plainproof fit              # 2PL item fit on the development half of learners; checked against the receipt's hash
+plainproof retrain          # BOBCAT checkpoint, about 2 hours (author defaults, fold 1, seed 221), compared with the original run
+plainproof run e0           # E0 held-out run and summary; refuses to overwrite an existing cell
+plainproof run e0b          # E0b held-out run and summary
+```
+
+The ASSISTments terms of use forbid passing the data on, so this package never contains it: `fetch bobcat` downloads it from its source, and you accept those terms yourself. Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop `fit`; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical stops `run e0`. In either case, use the published held-out outputs (`plainproof fetch heldout`). The development stages (`scripts/edm_v2_e0/study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
+
+## Fingerprints (sha256)
+
+| What | sha256 |
+|---|---|
+| E0 sealed freeze `reports/edm-v2-e0/freeze.json` | `a0f8ce404f2b5f7e671e2503a33b4554040e80714f6053553507e0972acdbcdc` |
+| E0b sealed freeze `reports/edm-v2-e0b/freeze.json` | `3fa0b8d457d813f59abd5255f9303e44deb2119ee79d6a9b92439a6c1cbf58ed` |
+| E0 held-out summary (`e0-heldout.tar` → `heldout/summary.json`) | `aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90` |
+| E0b held-out summary (`e0b-heldout.tar` → `heldout/summary.json`) | `617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f` |
+| `e0-heldout.tar` (release asset) | `bd06a11eb7163da102cad6ea0df12ea79e52ff94aab1dc35d28ec3910af2be22` |
+| `e0b-heldout.tar` (release asset) | `68219f228c41da972fe4c8f92c218e01f85f08eed864185f237cecda05314bbf` |
+| ASSISTments 2009 file at BOBCAT e6b6245e | `c07ca9fa0f08207b06488cec527725938fe96e81e3c1762839e1ad0b84f974cc` |
+| 2PL item fit (regenerated in step b) | `12a3b167cd9c9b78b3c3c7faf71b2e464b0943a454f6418faf0f99b1114cb7fc` |
+| catIrt 0.5.1 CRAN binary used for the fixture | `f5418f447e91d184791e668bd2a135ea5da40c981d4a76826018295b6ea9f079` |
+
+## What is in here
+
+- `src/plainproof/`: item model, SPRT arm, exact stopping time, simulator (uniform and uncertainty-triggered outsourcing), item fit, signatures, planted probes (with the give-up model), matched-coverage scoring.
+- `src/plainproof/cli.py`: the `plainproof` command. `setup.py` packs the working directory into the wheel for `plainproof init`.
+- `scripts/edm_v2_e0/`, `scripts/edm_v2_e0b/`: the runners, report builders and R plots. `scripts/published_cat_reference_v1/`: the BOBCAT loading code the runners call.
+- `reports/`: both sealed freezes, receipts, figures and tables. `docs/`: the E0 protocol and the pre-registration log (verbatim extract).
+- `substitutions.json`: the only differences between this code and the sealed originals (package name, file locations, data paths). `scripts/verify_frozen.py` undoes them and checks every hash. `scripts/edm_v2_e0/r_env.sh` is kept verbatim as the record of the original R runtime; use `scripts/rscript.sh` instead.
+
+## Recorded departures
+
+All are in `docs/PREREG_EDM_V2.md`: the SPRT tuning rule was revised after its first development pass (labelled post hoc); SPRT's calibration map was refitted once on full paths before the freeze; the E0 held-out run exceeded its 2 GiB memory cap (no result depends on it; the runner now enforces the cap); E0b was sealed before its pilot table had been reviewed (no result affected).
+
+## Appendix: the raw script calls
+
+What each `plainproof` command runs, from the working directory (`REPRO_DATA` defaults to `./data`):
+
+```sh
+python -m pytest -n auto -q tests                                       # plainproof test
+python scripts/verify_frozen.py                                         # plainproof verify
+ZENODO_URL=https://github.com/cloudronin/plainproof/releases/download/v0.1.0 bash scripts/fetch_heldout.sh   # plainproof fetch heldout
+python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json  # plainproof figures (E0 tables)
+bash scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png <E0 summary sha256>
+python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
+bash scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png <E0b summary sha256>
+bash scripts/bobcat/fetch_bobcat.sh                                     # plainproof fetch bobcat
+python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit      # plainproof fit
+python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py   # plainproof retrain
+python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout   # plainproof run e0
+python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout       # plainproof run e0b
+scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt   # regenerate the catIrt fixture (R with catIrt 0.5.1)
+```

{plainproof-0.1.0 → plainproof-0.2.0}/src/plainproof.egg-info/SOURCES.txt

@@ -6,6 +6,7 @@ MANIFEST.in
 README.md
 pyproject.toml
 requirements.txt
+setup.py
 substitutions.json
 data/edm-v2-e0/study/calibration-v2.json
 data/edm-v2-e0/study/calibration-v2.receipt.json

plainproof-0.1.0/PKG-INFO

@@ -1,95 +0,0 @@
-Metadata-Version: 2.4
-Name: plainproof
-Version: 0.1.0
-Summary: Reproduction tool for the paper 'How many follow-up questions distinguish genuine understanding from apparent mastery?'. Not a general library. Its sealed checks can stop a run on purpose.
-Author: Vishnu Vettrivel
-License-Expression: MIT
-Requires-Python: >=3.11
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: numpy>=2.0
-Requires-Dist: scipy>=1.13
-Provides-Extra: run
-Requires-Dist: scikit-learn>=1.5; extra == "run"
-Requires-Dist: torch>=2.4; extra == "run"
-Provides-Extra: test
-Requires-Dist: pytest>=8; extra == "test"
-Requires-Dist: pytest-xdist>=3.5; extra == "test"
-Dynamic: license-file
-
-# plainproof
-
-Reproduction package for **"How many follow-up questions distinguish genuine understanding from apparent mastery?"** (EDM 2027 submission, simulation study). It asks how many adaptive follow-up questions a classification test needs to tell genuine understanding from apparent mastery at a calibrated confidence target, and how that changes as a learner outsources a share *d* of the follow-ups to an AI. A second, sealed study (E0b) tests whether planted hard probe items separate outsourcing learners from honest ones.
-
-Everything here is CPU-only and needs no paid service. Code: MIT (`LICENSE`). Docs, figures and tables: CC BY 4.0 (`LICENSE-docs.md`).
-
-## From clone to the figure (about 10 minutes)
-
-The `plainproof` command runs each step (`plainproof --help`); the raw script calls below are what it runs. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; use the `scripts/rscript.sh` wrapper, which `plainproof figures` calls.
-
-```sh
-git clone <this repository> && cd plainproof
-python3.11 -m venv .venv && . .venv/bin/activate && pip install -e '.[test,run]'   # or: pip install -r requirements.txt
-
-# 1. Tests (43): SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, planted probes
-python -m pytest -n auto -q tests
-
-# 2. Check that the code here is the sealed code (undoes the documented path substitutions, then compares sha256)
-plainproof verify
-
-# 3. Fetch the sealed held-out outputs from Zenodo (DOI: to be assigned) and check their hashes
-ZENODO_URL=<record file URL> plainproof fetch heldout
-
-# 4. Rebuild the tables and figures from the held-out summaries (needs R 4.x for the plots; base graphics only)
-python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90
-python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png 617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f
-```
-
-The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`; the committed copies are the same files.
-
-## Full rerun (about 4 hours of CPU)
-
-Only if you want to regenerate everything, including the held-out outputs.
-
-```sh
-# a. BOBCAT (MIT) at the pinned commit, with the preprocessed ASSISTments 2009-2010 file; the data's own terms forbid
-#    passing it on, so you fetch it from its source and accept them yourself
-scripts/bobcat/fetch_bobcat.sh
-# b. Item fit on the development half of ASSISTments learners (the output is checked against the receipt's hash)
-python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit
-# c. Retrain the BOBCAT checkpoint (about 2 hours; author defaults, fold 1, seed 221), then compare it with the original
-python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py
-# d. catIrt fixture, development stages, held-out runs (each refuses to overwrite an existing cell)
-scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt
-python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout
-python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout
-```
-
-Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop step b; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical can stop step d. In either case, use the Zenodo outputs. The development stages (`study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
-
-## Fingerprints (sha256)
-
-| What | sha256 |
-|---|---|
-| E0 sealed freeze `reports/edm-v2-e0/freeze.json` | `a0f8ce404f2b5f7e671e2503a33b4554040e80714f6053553507e0972acdbcdc` |
-| E0b sealed freeze `reports/edm-v2-e0b/freeze.json` | `3fa0b8d457d813f59abd5255f9303e44deb2119ee79d6a9b92439a6c1cbf58ed` |
-| E0 held-out summary (Zenodo, `e0-heldout.tar` → `heldout/summary.json`) | `aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90` |
-| E0b held-out summary (Zenodo, `e0b-heldout.tar` → `heldout/summary.json`) | `617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f` |
-| Zenodo `e0-heldout.tar` | `bd06a11eb7163da102cad6ea0df12ea79e52ff94aab1dc35d28ec3910af2be22` |
-| Zenodo `e0b-heldout.tar` | `68219f228c41da972fe4c8f92c218e01f85f08eed864185f237cecda05314bbf` |
-| ASSISTments 2009 file at BOBCAT e6b6245e | `c07ca9fa0f08207b06488cec527725938fe96e81e3c1762839e1ad0b84f974cc` |
-| 2PL item fit (regenerated in step b) | `12a3b167cd9c9b78b3c3c7faf71b2e464b0943a454f6418faf0f99b1114cb7fc` |
-| catIrt 0.5.1 CRAN binary used for the fixture | `f5418f447e91d184791e668bd2a135ea5da40c981d4a76826018295b6ea9f079` |
-
-## What is in here
-
-- `src/plainproof/`: item model, SPRT arm, exact stopping time, simulator (uniform and uncertainty-triggered outsourcing), item fit, signatures, planted probes (with the give-up model), matched-coverage scoring.
-- `scripts/edm_v2_e0/`, `scripts/edm_v2_e0b/`: the runners, report builders and R plots. `scripts/published_cat_reference_v1/`: the BOBCAT loading code the runners call.
-- `reports/`: both sealed freezes, receipts, figures and tables. `docs/`: the E0 protocol and the pre-registration log (verbatim extract).
-- `substitutions.json`: the only differences between this code and the sealed originals (package name, file locations, data paths). `scripts/verify_frozen.py` undoes them and checks every hash. `scripts/edm_v2_e0/r_env.sh` is kept verbatim as the record of the original R runtime; use `scripts/rscript.sh` instead.
-
-## Recorded departures
-
-All are in `docs/PREREG_EDM_V2.md`: the SPRT tuning rule was revised after its first development pass (labelled post hoc); SPRT's calibration map was refitted once on full paths before the freeze; the E0 held-out run exceeded its 2 GiB memory cap (no result depends on it; the runner now enforces the cap); E0b was sealed before its pilot table had been reviewed (no result affected).

plainproof-0.1.0/README.md

@@ -1,76 +0,0 @@
-# plainproof
-
-Reproduction package for **"How many follow-up questions distinguish genuine understanding from apparent mastery?"** (EDM 2027 submission, simulation study). It asks how many adaptive follow-up questions a classification test needs to tell genuine understanding from apparent mastery at a calibrated confidence target, and how that changes as a learner outsources a share *d* of the follow-ups to an AI. A second, sealed study (E0b) tests whether planted hard probe items separate outsourcing learners from honest ones.
-
-Everything here is CPU-only and needs no paid service. Code: MIT (`LICENSE`). Docs, figures and tables: CC BY 4.0 (`LICENSE-docs.md`).
-
-## From clone to the figure (about 10 minutes)
-
-The `plainproof` command runs each step (`plainproof --help`); the raw script calls below are what it runs. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; use the `scripts/rscript.sh` wrapper, which `plainproof figures` calls.
-
-```sh
-git clone <this repository> && cd plainproof
-python3.11 -m venv .venv && . .venv/bin/activate && pip install -e '.[test,run]'   # or: pip install -r requirements.txt
-
-# 1. Tests (43): SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, planted probes
-python -m pytest -n auto -q tests
-
-# 2. Check that the code here is the sealed code (undoes the documented path substitutions, then compares sha256)
-plainproof verify
-
-# 3. Fetch the sealed held-out outputs from Zenodo (DOI: to be assigned) and check their hashes
-ZENODO_URL=<record file URL> plainproof fetch heldout
-
-# 4. Rebuild the tables and figures from the held-out summaries (needs R 4.x for the plots; base graphics only)
-python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90
-python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png 617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f
-```
-
-The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`; the committed copies are the same files.
-
-## Full rerun (about 4 hours of CPU)
-
-Only if you want to regenerate everything, including the held-out outputs.
-
-```sh
-# a. BOBCAT (MIT) at the pinned commit, with the preprocessed ASSISTments 2009-2010 file; the data's own terms forbid
-#    passing it on, so you fetch it from its source and accept them yourself
-scripts/bobcat/fetch_bobcat.sh
-# b. Item fit on the development half of ASSISTments learners (the output is checked against the receipt's hash)
-python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit
-# c. Retrain the BOBCAT checkpoint (about 2 hours; author defaults, fold 1, seed 221), then compare it with the original
-python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py
-# d. catIrt fixture, development stages, held-out runs (each refuses to overwrite an existing cell)
-scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt
-python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout
-python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout
-```
-
-Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop step b; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical can stop step d. In either case, use the Zenodo outputs. The development stages (`study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
-
-## Fingerprints (sha256)
-
-| What | sha256 |
-|---|---|
-| E0 sealed freeze `reports/edm-v2-e0/freeze.json` | `a0f8ce404f2b5f7e671e2503a33b4554040e80714f6053553507e0972acdbcdc` |
-| E0b sealed freeze `reports/edm-v2-e0b/freeze.json` | `3fa0b8d457d813f59abd5255f9303e44deb2119ee79d6a9b92439a6c1cbf58ed` |
-| E0 held-out summary (Zenodo, `e0-heldout.tar` → `heldout/summary.json`) | `aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90` |
-| E0b held-out summary (Zenodo, `e0b-heldout.tar` → `heldout/summary.json`) | `617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f` |
-| Zenodo `e0-heldout.tar` | `bd06a11eb7163da102cad6ea0df12ea79e52ff94aab1dc35d28ec3910af2be22` |
-| Zenodo `e0b-heldout.tar` | `68219f228c41da972fe4c8f92c218e01f85f08eed864185f237cecda05314bbf` |
-| ASSISTments 2009 file at BOBCAT e6b6245e | `c07ca9fa0f08207b06488cec527725938fe96e81e3c1762839e1ad0b84f974cc` |
-| 2PL item fit (regenerated in step b) | `12a3b167cd9c9b78b3c3c7faf71b2e464b0943a454f6418faf0f99b1114cb7fc` |
-| catIrt 0.5.1 CRAN binary used for the fixture | `f5418f447e91d184791e668bd2a135ea5da40c981d4a76826018295b6ea9f079` |
-
-## What is in here
-
-- `src/plainproof/`: item model, SPRT arm, exact stopping time, simulator (uniform and uncertainty-triggered outsourcing), item fit, signatures, planted probes (with the give-up model), matched-coverage scoring.
-- `scripts/edm_v2_e0/`, `scripts/edm_v2_e0b/`: the runners, report builders and R plots. `scripts/published_cat_reference_v1/`: the BOBCAT loading code the runners call.
-- `reports/`: both sealed freezes, receipts, figures and tables. `docs/`: the E0 protocol and the pre-registration log (verbatim extract).
-- `substitutions.json`: the only differences between this code and the sealed originals (package name, file locations, data paths). `scripts/verify_frozen.py` undoes them and checks every hash. `scripts/edm_v2_e0/r_env.sh` is kept verbatim as the record of the original R runtime; use `scripts/rscript.sh` instead.
-
-## Recorded departures
-
-All are in `docs/PREREG_EDM_V2.md`: the SPRT tuning rule was revised after its first development pass (labelled post hoc); SPRT's calibration map was refitted once on full paths before the freeze; the E0 held-out run exceeded its 2 GiB memory cap (no result depends on it; the runner now enforces the cap); E0b was sealed before its pilot table had been reviewed (no result affected).

plainproof-0.1.0/src/plainproof/cli.py

@@ -1,84 +0,0 @@
-"""plainproof: the command-line front end of this reproduction package.
-
-It runs the repository's own scripts, so it needs a checkout or the unpacked sdist (the sealed scripts locate the
-repository by its layout). Pass --root, or run it from the repository root.
-
-  plainproof fetch [bobcat|heldout|all]   fetch BOBCAT + the ASSISTments file (hash-checked), and/or the Zenodo outputs
-  plainproof fit                          refit the 2PL item parameters on ASSISTments development learners
-  plainproof run e0|e0b                   held-out run and summary (refuses to overwrite an existing cell)
-  plainproof figures                      rebuild tables and figures from the held-out summaries (needs R for the plots)
-  plainproof verify                       check every sealed file against both freezes
-"""
-import argparse
-import hashlib
-import os
-import subprocess
-import sys
-from pathlib import Path
-
-
-def find_root(given):
-    root = Path(given or os.getcwd()).resolve()
-    if not (root / 'substitutions.json').exists() or not (root / 'reports' / 'edm-v2-e0' / 'freeze.json').exists():
-        raise SystemExit(f'{root} is not a plainproof checkout (no substitutions.json or reports/); use --root')
-    return root
-
-
-def sh(root, *cmd):
-    env = dict(os.environ)
-    env.setdefault('REPRO_DATA', str(root / 'data'))
-    print('+', ' '.join(str(c) for c in cmd), flush=True)
-    r = subprocess.run([str(c) for c in cmd], cwd=root, env=env)
-    if r.returncode:
-        raise SystemExit(r.returncode)
-
-
-def sha(p):
-    return hashlib.sha256(Path(p).read_bytes()).hexdigest()
-
-
-def main(argv=None):
-    ap = argparse.ArgumentParser(prog='plainproof', description=__doc__.split('\n\n')[0])
-    ap.add_argument('--root', help='repository root (default: current directory)')
-    sub = ap.add_subparsers(dest='cmd', required=True)
-    f = sub.add_parser('fetch'); f.add_argument('what', nargs='?', default='all', choices=('bobcat', 'heldout', 'all'))
-    sub.add_parser('fit')
-    r = sub.add_parser('run'); r.add_argument('study', choices=('e0', 'e0b'))
-    sub.add_parser('figures')
-    sub.add_parser('verify')
-    a = ap.parse_args(argv)
-    root = find_root(a.root)
-    py = sys.executable
-    data = Path(os.environ.get('REPRO_DATA', root / 'data'))
-    if a.cmd == 'fetch':
-        if a.what in ('bobcat', 'all'):
-            sh(root, 'bash', 'scripts/bobcat/fetch_bobcat.sh')
-        if a.what in ('heldout', 'all'):
-            if not os.environ.get('ZENODO_URL'):
-                raise SystemExit('set ZENODO_URL to the Zenodo record file URL to fetch the held-out outputs')
-            sh(root, 'bash', 'scripts/fetch_heldout.sh')
-    elif a.cmd == 'fit':
-        sh(root, py, 'scripts/edm_v2_e0/fit_assist2009.py', data / 'edm-v2-e0' / 'item-fit')
-    elif a.cmd == 'run':
-        if a.study == 'e0':
-            out = data / 'edm-v2-e0' / 'heldout'
-            sh(root, py, 'scripts/edm_v2_e0/heldout.py', 'run', out)
-            sh(root, py, 'scripts/edm_v2_e0/heldout.py', 'summarize', out)
-        else:
-            out = data / 'edm-v2-e0b' / 'heldout'
-            sh(root, py, 'scripts/edm_v2_e0b/run.py', 'run', out)
-            sh(root, py, 'scripts/edm_v2_e0b/run.py', 'summarize', out)
-    elif a.cmd == 'figures':
-        s0, s1 = data / 'edm-v2-e0' / 'heldout' / 'summary.json', data / 'edm-v2-e0b' / 'heldout' / 'summary.json'
-        sh(root, py, 'scripts/edm_v2_e0/report.py', s0)
-        sh(root, 'bash', 'scripts/rscript.sh', 'scripts/edm_v2_e0/plot_error_vs_d.R', 'reports/edm-v2-e0/error-vs-d.csv',
-           'reports/edm-v2-e0/figures/error-vs-d.png', sha(s0))
-        sh(root, py, 'scripts/edm_v2_e0b/report.py', s1)
-        sh(root, 'bash', 'scripts/rscript.sh', 'scripts/edm_v2_e0b/plot_probe_auc.R', 'reports/edm-v2-e0b/probe-auc-vs-d.csv',
-           'reports/edm-v2-e0b/figures/probe-auc-vs-d.png', sha(s1))
-    elif a.cmd == 'verify':
-        sh(root, py, 'scripts/verify_frozen.py')
-
-
-if __name__ == '__main__':
-    main()

plainproof-0.1.0/src/plainproof.egg-info/PKG-INFO

@@ -1,95 +0,0 @@
-Metadata-Version: 2.4
-Name: plainproof
-Version: 0.1.0
-Summary: Reproduction tool for the paper 'How many follow-up questions distinguish genuine understanding from apparent mastery?'. Not a general library. Its sealed checks can stop a run on purpose.
-Author: Vishnu Vettrivel
-License-Expression: MIT
-Requires-Python: >=3.11
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: numpy>=2.0
-Requires-Dist: scipy>=1.13
-Provides-Extra: run
-Requires-Dist: scikit-learn>=1.5; extra == "run"
-Requires-Dist: torch>=2.4; extra == "run"
-Provides-Extra: test
-Requires-Dist: pytest>=8; extra == "test"
-Requires-Dist: pytest-xdist>=3.5; extra == "test"
-Dynamic: license-file
-
-# plainproof
-
-Reproduction package for **"How many follow-up questions distinguish genuine understanding from apparent mastery?"** (EDM 2027 submission, simulation study). It asks how many adaptive follow-up questions a classification test needs to tell genuine understanding from apparent mastery at a calibrated confidence target, and how that changes as a learner outsources a share *d* of the follow-ups to an AI. A second, sealed study (E0b) tests whether planted hard probe items separate outsourcing learners from honest ones.
-
-Everything here is CPU-only and needs no paid service. Code: MIT (`LICENSE`). Docs, figures and tables: CC BY 4.0 (`LICENSE-docs.md`).
-
-## From clone to the figure (about 10 minutes)
-
-The `plainproof` command runs each step (`plainproof --help`); the raw script calls below are what it runs. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; use the `scripts/rscript.sh` wrapper, which `plainproof figures` calls.
-
-```sh
-git clone <this repository> && cd plainproof
-python3.11 -m venv .venv && . .venv/bin/activate && pip install -e '.[test,run]'   # or: pip install -r requirements.txt
-
-# 1. Tests (43): SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, planted probes
-python -m pytest -n auto -q tests
-
-# 2. Check that the code here is the sealed code (undoes the documented path substitutions, then compares sha256)
-plainproof verify
-
-# 3. Fetch the sealed held-out outputs from Zenodo (DOI: to be assigned) and check their hashes
-ZENODO_URL=<record file URL> plainproof fetch heldout
-
-# 4. Rebuild the tables and figures from the held-out summaries (needs R 4.x for the plots; base graphics only)
-python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90
-python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png 617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f
-```
-
-The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`; the committed copies are the same files.
-
-## Full rerun (about 4 hours of CPU)
-
-Only if you want to regenerate everything, including the held-out outputs.
-
-```sh
-# a. BOBCAT (MIT) at the pinned commit, with the preprocessed ASSISTments 2009-2010 file; the data's own terms forbid
-#    passing it on, so you fetch it from its source and accept them yourself
-scripts/bobcat/fetch_bobcat.sh
-# b. Item fit on the development half of ASSISTments learners (the output is checked against the receipt's hash)
-python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit
-# c. Retrain the BOBCAT checkpoint (about 2 hours; author defaults, fold 1, seed 221), then compare it with the original
-python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py
-# d. catIrt fixture, development stages, held-out runs (each refuses to overwrite an existing cell)
-scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt
-python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout
-python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout
-```
-
-Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop step b; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical can stop step d. In either case, use the Zenodo outputs. The development stages (`study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
-
-## Fingerprints (sha256)
-
-| What | sha256 |
-|---|---|
-| E0 sealed freeze `reports/edm-v2-e0/freeze.json` | `a0f8ce404f2b5f7e671e2503a33b4554040e80714f6053553507e0972acdbcdc` |
-| E0b sealed freeze `reports/edm-v2-e0b/freeze.json` | `3fa0b8d457d813f59abd5255f9303e44deb2119ee79d6a9b92439a6c1cbf58ed` |
-| E0 held-out summary (Zenodo, `e0-heldout.tar` → `heldout/summary.json`) | `aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90` |
-| E0b held-out summary (Zenodo, `e0b-heldout.tar` → `heldout/summary.json`) | `617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f` |
-| Zenodo `e0-heldout.tar` | `bd06a11eb7163da102cad6ea0df12ea79e52ff94aab1dc35d28ec3910af2be22` |
-| Zenodo `e0b-heldout.tar` | `68219f228c41da972fe4c8f92c218e01f85f08eed864185f237cecda05314bbf` |
-| ASSISTments 2009 file at BOBCAT e6b6245e | `c07ca9fa0f08207b06488cec527725938fe96e81e3c1762839e1ad0b84f974cc` |
-| 2PL item fit (regenerated in step b) | `12a3b167cd9c9b78b3c3c7faf71b2e464b0943a454f6418faf0f99b1114cb7fc` |
-| catIrt 0.5.1 CRAN binary used for the fixture | `f5418f447e91d184791e668bd2a135ea5da40c981d4a76826018295b6ea9f079` |
-
-## What is in here
-
-- `src/plainproof/`: item model, SPRT arm, exact stopping time, simulator (uniform and uncertainty-triggered outsourcing), item fit, signatures, planted probes (with the give-up model), matched-coverage scoring.
-- `scripts/edm_v2_e0/`, `scripts/edm_v2_e0b/`: the runners, report builders and R plots. `scripts/published_cat_reference_v1/`: the BOBCAT loading code the runners call.
-- `reports/`: both sealed freezes, receipts, figures and tables. `docs/`: the E0 protocol and the pre-registration log (verbatim extract).
-- `substitutions.json`: the only differences between this code and the sealed originals (package name, file locations, data paths). `scripts/verify_frozen.py` undoes them and checks every hash. `scripts/edm_v2_e0/r_env.sh` is kept verbatim as the record of the original R runtime; use `scripts/rscript.sh` instead.
-
-## Recorded departures
-
-All are in `docs/PREREG_EDM_V2.md`: the SPRT tuning rule was revised after its first development pass (labelled post hoc); SPRT's calibration map was refitted once on full paths before the freeze; the E0 held-out run exceeded its 2 GiB memory cap (no result depends on it; the runner now enforces the cap); E0b was sealed before its pilot table had been reviewed (no result affected).