plainproof 0.1.1__tar.gz → 0.2.0__tar.gz

{plainproof-0.1.1 → plainproof-0.2.0}/.gitignore

@@ -8,3 +8,4 @@ dist/
 build/
 *.egg-info/
 .venv/
+src/plainproof/payload/

{plainproof-0.1.1 → plainproof-0.2.0}/CITATION.cff

@@ -5,7 +5,7 @@ authors:
   - family-names: Vettrivel
     given-names: Vishnu
 license: MIT
-version: 0.1.1
+version: 0.2.0
 doi: 10.5281/zenodo.23647649
 identifiers:
   - type: doi

{plainproof-0.1.1 → plainproof-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: plainproof
-Version: 0.1.1
+Version: 0.2.0
 Summary: Reproduction tool for the paper 'How many follow-up questions distinguish genuine understanding from apparent mastery?'. Not a general library. Its sealed checks can stop a run on purpose.
 Author: Vishnu Vettrivel
 License-Expression: MIT
@@ -27,51 +27,33 @@ Reproduction package for **"How many follow-up questions distinguish genuine und
 
 Everything here is CPU-only and needs no paid service. Code: MIT (`LICENSE`). Docs, figures and tables: CC BY 4.0 (`LICENSE-docs.md`).
 
-## From clone to the figure (about 10 minutes)
-
-The `plainproof` command runs each step (`plainproof --help`); the raw script calls below are what it runs. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; use the `scripts/rscript.sh` wrapper, which `plainproof figures` calls.
+## From install to the figure (about 10 minutes)
 
 ```sh
-git clone <this repository> && cd plainproof
-python3.11 -m venv .venv && . .venv/bin/activate && pip install -e '.[test,run]'   # or: pip install -r requirements.txt
-
-# 1. Tests (43): SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, planted probes
-python -m pytest -n auto -q tests
-
-# 2. Check that the code here is the sealed code (undoes the documented path substitutions, then compares sha256)
-plainproof verify
-
-# 3. Fetch the sealed held-out outputs (from the v0.1.0 release assets) and check their hashes
-plainproof fetch heldout
-
-# 4. Rebuild the tables and figures from the held-out summaries (needs R 4.x for the plots; base graphics only)
-python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90
-python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png 617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f
+pip install 'plainproof[test,run]'
+plainproof init ~/plainproof        # writes the working directory: sealed scripts, freezes, receipts, docs, tests
+cd ~/plainproof
+plainproof test                     # 43 tests: SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, probes
+plainproof verify                   # every sealed file against both freezes (undoes the documented substitutions, then sha256)
+plainproof fetch heldout            # the sealed held-out outputs, hash-checked
+plainproof figures                  # tables and figures from the held-out summaries (needs R 4.x on PATH; base graphics only)
 ```
 
-The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`; the committed copies are the same files.
+The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`, byte-identical to the published copies. `plainproof init` writes the files in the exact layout the sealed scripts expect; nothing sealed changes. Every other command runs inside that directory, or takes `--root DIR`. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; the `scripts/rscript.sh` wrapper, which `plainproof figures` calls, is the one to use.
 
 ## Full rerun (about 4 hours of CPU)
 
 Only if you want to regenerate everything, including the held-out outputs.
 
 ```sh
-# a. BOBCAT (MIT) at the pinned commit, with the preprocessed ASSISTments 2009-2010 file; the data's own terms forbid
-#    passing it on, so you fetch it from its source and accept them yourself
-scripts/bobcat/fetch_bobcat.sh
-# b. Item fit on the development half of ASSISTments learners (the output is checked against the receipt's hash)
-python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit
-# c. Retrain the BOBCAT checkpoint (about 2 hours; author defaults, fold 1, seed 221), then compare it with the original
-python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py
-# d. catIrt fixture, development stages, held-out runs (each refuses to overwrite an existing cell)
-scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt
-python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout
-python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout
+plainproof fetch bobcat     # BOBCAT (MIT) at its pinned commit, with the preprocessed ASSISTments 2009-2010 file
+plainproof fit              # 2PL item fit on the development half of learners; checked against the receipt's hash
+plainproof retrain          # BOBCAT checkpoint, about 2 hours (author defaults, fold 1, seed 221), compared with the original run
+plainproof run e0           # E0 held-out run and summary; refuses to overwrite an existing cell
+plainproof run e0b          # E0b held-out run and summary
 ```
 
-Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop step b; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical can stop step d. In either case, use the published held-out outputs (`plainproof fetch heldout`). The development stages (`study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
+The ASSISTments terms of use forbid passing the data on, so this package never contains it: `fetch bobcat` downloads it from its source, and you accept those terms yourself. Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop `fit`; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical stops `run e0`. In either case, use the published held-out outputs (`plainproof fetch heldout`). The development stages (`scripts/edm_v2_e0/study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
 
 ## Fingerprints (sha256)
 
@@ -90,6 +72,7 @@ Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a
 ## What is in here
 
 - `src/plainproof/`: item model, SPRT arm, exact stopping time, simulator (uniform and uncertainty-triggered outsourcing), item fit, signatures, planted probes (with the give-up model), matched-coverage scoring.
+- `src/plainproof/cli.py`: the `plainproof` command. `setup.py` packs the working directory into the wheel for `plainproof init`.
 - `scripts/edm_v2_e0/`, `scripts/edm_v2_e0b/`: the runners, report builders and R plots. `scripts/published_cat_reference_v1/`: the BOBCAT loading code the runners call.
 - `reports/`: both sealed freezes, receipts, figures and tables. `docs/`: the E0 protocol and the pre-registration log (verbatim extract).
 - `substitutions.json`: the only differences between this code and the sealed originals (package name, file locations, data paths). `scripts/verify_frozen.py` undoes them and checks every hash. `scripts/edm_v2_e0/r_env.sh` is kept verbatim as the record of the original R runtime; use `scripts/rscript.sh` instead.
@@ -97,3 +80,23 @@ Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a
 ## Recorded departures
 
 All are in `docs/PREREG_EDM_V2.md`: the SPRT tuning rule was revised after its first development pass (labelled post hoc); SPRT's calibration map was refitted once on full paths before the freeze; the E0 held-out run exceeded its 2 GiB memory cap (no result depends on it; the runner now enforces the cap); E0b was sealed before its pilot table had been reviewed (no result affected).
+
+## Appendix: the raw script calls
+
+What each `plainproof` command runs, from the working directory (`REPRO_DATA` defaults to `./data`):
+
+```sh
+python -m pytest -n auto -q tests                                       # plainproof test
+python scripts/verify_frozen.py                                         # plainproof verify
+ZENODO_URL=https://github.com/cloudronin/plainproof/releases/download/v0.1.0 bash scripts/fetch_heldout.sh   # plainproof fetch heldout
+python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json  # plainproof figures (E0 tables)
+bash scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png <E0 summary sha256>
+python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
+bash scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png <E0b summary sha256>
+bash scripts/bobcat/fetch_bobcat.sh                                     # plainproof fetch bobcat
+python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit      # plainproof fit
+python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py   # plainproof retrain
+python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout   # plainproof run e0
+python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout       # plainproof run e0b
+scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt   # regenerate the catIrt fixture (R with catIrt 0.5.1)
+```

{plainproof-0.1.1 → plainproof-0.2.0}/README.md

@@ -8,51 +8,33 @@ Reproduction package for **"How many follow-up questions distinguish genuine und
 
 Everything here is CPU-only and needs no paid service. Code: MIT (`LICENSE`). Docs, figures and tables: CC BY 4.0 (`LICENSE-docs.md`).
 
-## From clone to the figure (about 10 minutes)
-
-The `plainproof` command runs each step (`plainproof --help`); the raw script calls below are what it runs. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; use the `scripts/rscript.sh` wrapper, which `plainproof figures` calls.
+## From install to the figure (about 10 minutes)
 
 ```sh
-git clone <this repository> && cd plainproof
-python3.11 -m venv .venv && . .venv/bin/activate && pip install -e '.[test,run]'   # or: pip install -r requirements.txt
-
-# 1. Tests (43): SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, planted probes
-python -m pytest -n auto -q tests
-
-# 2. Check that the code here is the sealed code (undoes the documented path substitutions, then compares sha256)
-plainproof verify
-
-# 3. Fetch the sealed held-out outputs (from the v0.1.0 release assets) and check their hashes
-plainproof fetch heldout
-
-# 4. Rebuild the tables and figures from the held-out summaries (needs R 4.x for the plots; base graphics only)
-python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90
-python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png 617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f
+pip install 'plainproof[test,run]'
+plainproof init ~/plainproof        # writes the working directory: sealed scripts, freezes, receipts, docs, tests
+cd ~/plainproof
+plainproof test                     # 43 tests: SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, probes
+plainproof verify                   # every sealed file against both freezes (undoes the documented substitutions, then sha256)
+plainproof fetch heldout            # the sealed held-out outputs, hash-checked
+plainproof figures                  # tables and figures from the held-out summaries (needs R 4.x on PATH; base graphics only)
 ```
 
-The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`; the committed copies are the same files.
+The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`, byte-identical to the published copies. `plainproof init` writes the files in the exact layout the sealed scripts expect; nothing sealed changes. Every other command runs inside that directory, or takes `--root DIR`. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; the `scripts/rscript.sh` wrapper, which `plainproof figures` calls, is the one to use.
 
 ## Full rerun (about 4 hours of CPU)
 
 Only if you want to regenerate everything, including the held-out outputs.
 
 ```sh
-# a. BOBCAT (MIT) at the pinned commit, with the preprocessed ASSISTments 2009-2010 file; the data's own terms forbid
-#    passing it on, so you fetch it from its source and accept them yourself
-scripts/bobcat/fetch_bobcat.sh
-# b. Item fit on the development half of ASSISTments learners (the output is checked against the receipt's hash)
-python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit
-# c. Retrain the BOBCAT checkpoint (about 2 hours; author defaults, fold 1, seed 221), then compare it with the original
-python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py
-# d. catIrt fixture, development stages, held-out runs (each refuses to overwrite an existing cell)
-scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt
-python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout
-python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout
+plainproof fetch bobcat     # BOBCAT (MIT) at its pinned commit, with the preprocessed ASSISTments 2009-2010 file
+plainproof fit              # 2PL item fit on the development half of learners; checked against the receipt's hash
+plainproof retrain          # BOBCAT checkpoint, about 2 hours (author defaults, fold 1, seed 221), compared with the original run
+plainproof run e0           # E0 held-out run and summary; refuses to overwrite an existing cell
+plainproof run e0b          # E0b held-out run and summary
 ```
 
-Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop step b; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical can stop step d. In either case, use the published held-out outputs (`plainproof fetch heldout`). The development stages (`study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
+The ASSISTments terms of use forbid passing the data on, so this package never contains it: `fetch bobcat` downloads it from its source, and you accept those terms yourself. Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop `fit`; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical stops `run e0`. In either case, use the published held-out outputs (`plainproof fetch heldout`). The development stages (`scripts/edm_v2_e0/study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
 
 ## Fingerprints (sha256)
 
@@ -71,6 +53,7 @@ Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a
 ## What is in here
 
 - `src/plainproof/`: item model, SPRT arm, exact stopping time, simulator (uniform and uncertainty-triggered outsourcing), item fit, signatures, planted probes (with the give-up model), matched-coverage scoring.
+- `src/plainproof/cli.py`: the `plainproof` command. `setup.py` packs the working directory into the wheel for `plainproof init`.
 - `scripts/edm_v2_e0/`, `scripts/edm_v2_e0b/`: the runners, report builders and R plots. `scripts/published_cat_reference_v1/`: the BOBCAT loading code the runners call.
 - `reports/`: both sealed freezes, receipts, figures and tables. `docs/`: the E0 protocol and the pre-registration log (verbatim extract).
 - `substitutions.json`: the only differences between this code and the sealed originals (package name, file locations, data paths). `scripts/verify_frozen.py` undoes them and checks every hash. `scripts/edm_v2_e0/r_env.sh` is kept verbatim as the record of the original R runtime; use `scripts/rscript.sh` instead.
@@ -78,3 +61,23 @@ Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a
 ## Recorded departures
 
 All are in `docs/PREREG_EDM_V2.md`: the SPRT tuning rule was revised after its first development pass (labelled post hoc); SPRT's calibration map was refitted once on full paths before the freeze; the E0 held-out run exceeded its 2 GiB memory cap (no result depends on it; the runner now enforces the cap); E0b was sealed before its pilot table had been reviewed (no result affected).
+
+## Appendix: the raw script calls
+
+What each `plainproof` command runs, from the working directory (`REPRO_DATA` defaults to `./data`):
+
+```sh
+python -m pytest -n auto -q tests                                       # plainproof test
+python scripts/verify_frozen.py                                         # plainproof verify
+ZENODO_URL=https://github.com/cloudronin/plainproof/releases/download/v0.1.0 bash scripts/fetch_heldout.sh   # plainproof fetch heldout
+python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json  # plainproof figures (E0 tables)
+bash scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png <E0 summary sha256>
+python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
+bash scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png <E0b summary sha256>
+bash scripts/bobcat/fetch_bobcat.sh                                     # plainproof fetch bobcat
+python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit      # plainproof fit
+python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py   # plainproof retrain
+python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout   # plainproof run e0
+python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout       # plainproof run e0b
+scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt   # regenerate the catIrt fixture (R with catIrt 0.5.1)
+```

{plainproof-0.1.1 → plainproof-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "plainproof"
-version = "0.1.1"
+version = "0.2.0"
 description = "Reproduction tool for the paper 'How many follow-up questions distinguish genuine understanding from apparent mastery?'. Not a general library. Its sealed checks can stop a run on purpose."
 readme = "README.md"
 license = "MIT"

plainproof-0.2.0/setup.py

@@ -0,0 +1,31 @@
+"""Build hook: copy the reproduction working directory into the package as data, so `plainproof init` can write it
+out after `pip install plainproof` alone. The copied files are the repository's own files, unchanged."""
+import shutil
+from pathlib import Path
+
+from setuptools import setup
+from setuptools.command.build_py import build_py
+
+PAYLOAD = ['scripts', 'reports', 'docs', 'tests', 'data/edm-v2-e0/study', 'substitutions.json', 'README.md', 'LICENSE',
+           'LICENSE-docs.md', 'CITATION.cff', 'requirements.txt']
+
+
+class with_payload(build_py):
+    def run(self):
+        super().run()
+        here = Path(__file__).resolve().parent
+        dest = Path(self.build_lib) / 'plainproof' / 'payload'
+        if dest.exists():
+            shutil.rmtree(dest)
+        for item in PAYLOAD:
+            src = here / item
+            if not src.exists():
+                raise SystemExit(f'payload item missing from the source tree: {item}')
+            if src.is_dir():
+                shutil.copytree(src, dest / item, ignore=shutil.ignore_patterns('__pycache__', '*.pyc'))
+            else:
+                (dest / item).parent.mkdir(parents=True, exist_ok=True)
+                shutil.copy2(src, dest / item)
+
+
+setup(cmdclass={'build_py': with_payload})

plainproof-0.2.0/src/plainproof/cli.py

@@ -0,0 +1,121 @@
+"""plainproof: the command-line front end of this reproduction package.
+
+`plainproof init DIR` writes a working directory from the package (sealed scripts, both freezes, receipts,
+substitutions.json, R scripts, docs, tests) in the exact layout the sealed scripts expect; nothing sealed changes.
+Every other command runs inside that directory, or takes --root DIR.
+
+  plainproof init DIR                     write the working directory (refuses a non-empty DIR)
+  plainproof test                         the 43 offline tests
+  plainproof verify                       check every sealed file against both freezes
+  plainproof fetch heldout|bobcat|all     the sealed held-out outputs, and/or BOBCAT + the ASSISTments file (hash-checked)
+  plainproof figures                      rebuild tables and figures from the held-out summaries (needs R for the plots)
+  plainproof fit                          refit the 2PL item parameters on ASSISTments development learners
+  plainproof retrain                      retrain the BOBCAT checkpoint, then compare it with the original run
+  plainproof run e0|e0b                   held-out run and summary (refuses to overwrite an existing cell)
+"""
+import argparse
+import hashlib
+import os
+import shutil
+import subprocess
+import sys
+from pathlib import Path
+
+# The sealed held-out outputs. Their sha256 values are checked by scripts/fetch_heldout.sh before use.
+HELDOUT_BASE_URL = 'https://github.com/cloudronin/plainproof/releases/download/v0.1.0'
+PAYLOAD = Path(__file__).resolve().parent / 'payload'
+
+
+def find_root(given):
+    root = Path(given or os.getcwd()).resolve()
+    if not (root / 'substitutions.json').exists() or not (root / 'reports' / 'edm-v2-e0' / 'freeze.json').exists():
+        raise SystemExit(f'{root} is not a plainproof working directory; run `plainproof init DIR` and cd there, or pass --root')
+    return root
+
+
+def sh(root, *cmd, env_extra=None):
+    env = dict(os.environ)
+    env.setdefault('REPRO_DATA', str(root / 'data'))
+    env.update(env_extra or {})
+    print('+', ' '.join(str(c) for c in cmd), flush=True)
+    r = subprocess.run([str(c) for c in cmd], cwd=root, env=env)
+    if r.returncode:
+        raise SystemExit(r.returncode)
+
+
+def sha(p):
+    return hashlib.sha256(Path(p).read_bytes()).hexdigest()
+
+
+def init(target):
+    target = Path(target).expanduser().resolve()
+    if target.exists() and any(target.iterdir()):
+        raise SystemExit(f'{target} is not empty; choose a new directory')
+    if not PAYLOAD.exists():
+        raise SystemExit('this installation has no payload; reinstall plainproof from the wheel or sdist')
+    shutil.copytree(PAYLOAD, target, dirs_exist_ok=True)
+    # The sealed scripts import the package from <root>/src and hash its modules there, so the package is written
+    # out too, byte for byte from this installation.
+    pkg = target / 'src' / 'plainproof'
+    pkg.mkdir(parents=True)
+    for f in Path(__file__).resolve().parent.glob('*.py'):
+        shutil.copy2(f, pkg / f.name)
+    for d in ('data/edm-v2-e0', 'data/edm-v2-e0b'):
+        (target / d).mkdir(parents=True, exist_ok=True)
+    for f in target.rglob('*.sh'):
+        f.chmod(0o755)
+    print(f'wrote {target}\nnext: cd {target} && plainproof test && plainproof verify && plainproof fetch heldout && plainproof figures')
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(prog='plainproof', description=__doc__.split('\n\n')[0])
+    ap.add_argument('--root', help='working directory (default: current directory)')
+    sub = ap.add_subparsers(dest='cmd', required=True)
+    sub.add_parser('init').add_argument('dir')
+    sub.add_parser('test')
+    sub.add_parser('verify')
+    sub.add_parser('fetch').add_argument('what', choices=('heldout', 'bobcat', 'all'))
+    sub.add_parser('figures')
+    sub.add_parser('fit')
+    sub.add_parser('retrain')
+    sub.add_parser('run').add_argument('study', choices=('e0', 'e0b'))
+    a = ap.parse_args(argv)
+    if a.cmd == 'init':
+        return init(a.dir)
+    root = find_root(a.root)
+    py = sys.executable
+    data = Path(os.environ.get('REPRO_DATA', root / 'data'))
+    if a.cmd == 'test':
+        sh(root, py, '-m', 'pytest', '-n', 'auto', '-q', '-p', 'no:cacheprovider', 'tests')
+    elif a.cmd == 'verify':
+        sh(root, py, 'scripts/verify_frozen.py')
+    elif a.cmd == 'fetch':
+        if a.what in ('heldout', 'all'):
+            sh(root, 'bash', 'scripts/fetch_heldout.sh', env_extra={'ZENODO_URL': os.environ.get('ZENODO_URL', HELDOUT_BASE_URL)})
+        if a.what in ('bobcat', 'all'):
+            sh(root, 'bash', 'scripts/bobcat/fetch_bobcat.sh')
+    elif a.cmd == 'figures':
+        s0, s1 = data / 'edm-v2-e0' / 'heldout' / 'summary.json', data / 'edm-v2-e0b' / 'heldout' / 'summary.json'
+        for s in (s0, s1):
+            if not s.exists():
+                raise SystemExit(f'{s} missing; run `plainproof fetch heldout` first')
+        sh(root, py, 'scripts/edm_v2_e0/report.py', s0)
+        sh(root, 'bash', 'scripts/rscript.sh', 'scripts/edm_v2_e0/plot_error_vs_d.R', 'reports/edm-v2-e0/error-vs-d.csv',
+           'reports/edm-v2-e0/figures/error-vs-d.png', sha(s0))
+        sh(root, py, 'scripts/edm_v2_e0b/report.py', s1)
+        sh(root, 'bash', 'scripts/rscript.sh', 'scripts/edm_v2_e0b/plot_probe_auc.R', 'reports/edm-v2-e0b/probe-auc-vs-d.csv',
+           'reports/edm-v2-e0b/figures/probe-auc-vs-d.png', sha(s1))
+    elif a.cmd == 'fit':
+        sh(root, py, 'scripts/edm_v2_e0/fit_assist2009.py', data / 'edm-v2-e0' / 'item-fit')
+    elif a.cmd == 'retrain':
+        sh(root, py, 'scripts/bobcat/retrain.py')
+        sh(root, py, 'scripts/bobcat/check_checkpoint.py')
+    elif a.cmd == 'run':
+        out = data / a.study.replace('e0', 'edm-v2-e0', 1) / 'heldout'
+        script = 'scripts/edm_v2_e0/heldout.py' if a.study == 'e0' else 'scripts/edm_v2_e0b/run.py'
+        sh(root, py, script, 'run', out)
+        sh(root, py, script, 'summarize', out)
+
+
+if __name__ == '__main__':
+    main()

{plainproof-0.1.1 → plainproof-0.2.0}/src/plainproof.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: plainproof
-Version: 0.1.1
+Version: 0.2.0
 Summary: Reproduction tool for the paper 'How many follow-up questions distinguish genuine understanding from apparent mastery?'. Not a general library. Its sealed checks can stop a run on purpose.
 Author: Vishnu Vettrivel
 License-Expression: MIT
@@ -27,51 +27,33 @@ Reproduction package for **"How many follow-up questions distinguish genuine und
 
 Everything here is CPU-only and needs no paid service. Code: MIT (`LICENSE`). Docs, figures and tables: CC BY 4.0 (`LICENSE-docs.md`).
 
-## From clone to the figure (about 10 minutes)
-
-The `plainproof` command runs each step (`plainproof --help`); the raw script calls below are what it runs. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; use the `scripts/rscript.sh` wrapper, which `plainproof figures` calls.
+## From install to the figure (about 10 minutes)
 
 ```sh
-git clone <this repository> && cd plainproof
-python3.11 -m venv .venv && . .venv/bin/activate && pip install -e '.[test,run]'   # or: pip install -r requirements.txt
-
-# 1. Tests (43): SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, planted probes
-python -m pytest -n auto -q tests
-
-# 2. Check that the code here is the sealed code (undoes the documented path substitutions, then compares sha256)
-plainproof verify
-
-# 3. Fetch the sealed held-out outputs (from the v0.1.0 release assets) and check their hashes
-plainproof fetch heldout
-
-# 4. Rebuild the tables and figures from the held-out summaries (needs R 4.x for the plots; base graphics only)
-python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png aff9a8c49d48821c0e9343cd058c4e0e2f1659ffabcaa719824412f3b14abc90
-python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
-scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png 617be427feac24df2e342da00d34e91c78d5de91592cee390ef3516ae9e2ff3f
+pip install 'plainproof[test,run]'
+plainproof init ~/plainproof        # writes the working directory: sealed scripts, freezes, receipts, docs, tests
+cd ~/plainproof
+plainproof test                     # 43 tests: SPRT against catIrt 0.5.1's saved steps, exact stopping time, simulator, signatures, probes
+plainproof verify                   # every sealed file against both freezes (undoes the documented substitutions, then sha256)
+plainproof fetch heldout            # the sealed held-out outputs, hash-checked
+plainproof figures                  # tables and figures from the held-out summaries (needs R 4.x on PATH; base graphics only)
 ```
 
-The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`; the committed copies are the same files.
+The figures land in `reports/edm-v2-e0/figures/` and `reports/edm-v2-e0b/figures/`, byte-identical to the published copies. `plainproof init` writes the files in the exact layout the sealed scripts expect; nothing sealed changes. Every other command runs inside that directory, or takes `--root DIR`. The local path inside the sealed `scripts/edm_v2_e0/r_env.sh` is part of the frozen record; the `scripts/rscript.sh` wrapper, which `plainproof figures` calls, is the one to use.
 
 ## Full rerun (about 4 hours of CPU)
 
 Only if you want to regenerate everything, including the held-out outputs.
 
 ```sh
-# a. BOBCAT (MIT) at the pinned commit, with the preprocessed ASSISTments 2009-2010 file; the data's own terms forbid
-#    passing it on, so you fetch it from its source and accept them yourself
-scripts/bobcat/fetch_bobcat.sh
-# b. Item fit on the development half of ASSISTments learners (the output is checked against the receipt's hash)
-python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit
-# c. Retrain the BOBCAT checkpoint (about 2 hours; author defaults, fold 1, seed 221), then compare it with the original
-python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py
-# d. catIrt fixture, development stages, held-out runs (each refuses to overwrite an existing cell)
-scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt
-python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout
-python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout
+plainproof fetch bobcat     # BOBCAT (MIT) at its pinned commit, with the preprocessed ASSISTments 2009-2010 file
+plainproof fit              # 2PL item fit on the development half of learners; checked against the receipt's hash
+plainproof retrain          # BOBCAT checkpoint, about 2 hours (author defaults, fold 1, seed 221), compared with the original run
+plainproof run e0           # E0 held-out run and summary; refuses to overwrite an existing cell
+plainproof run e0b          # E0b held-out run and summary
 ```
 
-Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop step b; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical can stop step d. In either case, use the published held-out outputs (`plainproof fetch heldout`). The development stages (`study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
+The ASSISTments terms of use forbid passing the data on, so this package never contains it: `fetch bobcat` downloads it from its source, and you accept those terms yourself. Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a different numpy can stop `fit`; and it asserts that the BOBCAT checkpoint's best epoch is 105, so a retrain that is not bit-identical stops `run e0`. In either case, use the published held-out outputs (`plainproof fetch heldout`). The development stages (`scripts/edm_v2_e0/study.py tune|calibrate|calibrate_sprt_full|pilot`) are included for the record; their outputs (synthetic learners only) are in `data/edm-v2-e0/study/`.
 
 ## Fingerprints (sha256)
 
@@ -90,6 +72,7 @@ Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a
 ## What is in here
 
 - `src/plainproof/`: item model, SPRT arm, exact stopping time, simulator (uniform and uncertainty-triggered outsourcing), item fit, signatures, planted probes (with the give-up model), matched-coverage scoring.
+- `src/plainproof/cli.py`: the `plainproof` command. `setup.py` packs the working directory into the wheel for `plainproof init`.
 - `scripts/edm_v2_e0/`, `scripts/edm_v2_e0b/`: the runners, report builders and R plots. `scripts/published_cat_reference_v1/`: the BOBCAT loading code the runners call.
 - `reports/`: both sealed freezes, receipts, figures and tables. `docs/`: the E0 protocol and the pre-registration log (verbatim extract).
 - `substitutions.json`: the only differences between this code and the sealed originals (package name, file locations, data paths). `scripts/verify_frozen.py` undoes them and checks every hash. `scripts/edm_v2_e0/r_env.sh` is kept verbatim as the record of the original R runtime; use `scripts/rscript.sh` instead.
@@ -97,3 +80,23 @@ Two caveats, stated plainly. The sealed code checks the item fit's sha256, so a
 ## Recorded departures
 
 All are in `docs/PREREG_EDM_V2.md`: the SPRT tuning rule was revised after its first development pass (labelled post hoc); SPRT's calibration map was refitted once on full paths before the freeze; the E0 held-out run exceeded its 2 GiB memory cap (no result depends on it; the runner now enforces the cap); E0b was sealed before its pilot table had been reviewed (no result affected).
+
+## Appendix: the raw script calls
+
+What each `plainproof` command runs, from the working directory (`REPRO_DATA` defaults to `./data`):
+
+```sh
+python -m pytest -n auto -q tests                                       # plainproof test
+python scripts/verify_frozen.py                                         # plainproof verify
+ZENODO_URL=https://github.com/cloudronin/plainproof/releases/download/v0.1.0 bash scripts/fetch_heldout.sh   # plainproof fetch heldout
+python scripts/edm_v2_e0/report.py data/edm-v2-e0/heldout/summary.json  # plainproof figures (E0 tables)
+bash scripts/rscript.sh scripts/edm_v2_e0/plot_error_vs_d.R reports/edm-v2-e0/error-vs-d.csv reports/edm-v2-e0/figures/error-vs-d.png <E0 summary sha256>
+python scripts/edm_v2_e0b/report.py data/edm-v2-e0b/heldout/summary.json
+bash scripts/rscript.sh scripts/edm_v2_e0b/plot_probe_auc.R reports/edm-v2-e0b/probe-auc-vs-d.csv reports/edm-v2-e0b/figures/probe-auc-vs-d.png <E0b summary sha256>
+bash scripts/bobcat/fetch_bobcat.sh                                     # plainproof fetch bobcat
+python scripts/edm_v2_e0/fit_assist2009.py data/edm-v2-e0/item-fit      # plainproof fit
+python scripts/bobcat/retrain.py && python scripts/bobcat/check_checkpoint.py   # plainproof retrain
+python scripts/edm_v2_e0/heldout.py run data/edm-v2-e0/heldout && python scripts/edm_v2_e0/heldout.py summarize data/edm-v2-e0/heldout   # plainproof run e0
+python scripts/edm_v2_e0b/run.py run data/edm-v2-e0b/heldout && python scripts/edm_v2_e0b/run.py summarize data/edm-v2-e0b/heldout       # plainproof run e0b
+scripts/rscript.sh scripts/edm_v2_e0/catirt_sprt_steps.R tests/fixtures/edm_v2_catirt_sprt   # regenerate the catIrt fixture (R with catIrt 0.5.1)
+```

{plainproof-0.1.1 → plainproof-0.2.0}/src/plainproof.egg-info/SOURCES.txt

@@ -6,6 +6,7 @@ MANIFEST.in
 README.md
 pyproject.toml
 requirements.txt
+setup.py
 substitutions.json
 data/edm-v2-e0/study/calibration-v2.json
 data/edm-v2-e0/study/calibration-v2.receipt.json

plainproof-0.1.1/src/plainproof/cli.py

@@ -1,82 +0,0 @@
-"""plainproof: the command-line front end of this reproduction package.
-
-It runs the repository's own scripts, so it needs a checkout or the unpacked sdist (the sealed scripts locate the
-repository by its layout). Pass --root, or run it from the repository root.
-
-  plainproof fetch [bobcat|heldout|all]   fetch BOBCAT + the ASSISTments file, and/or the held-out outputs (all hash-checked)
-  plainproof fit                          refit the 2PL item parameters on ASSISTments development learners
-  plainproof run e0|e0b                   held-out run and summary (refuses to overwrite an existing cell)
-  plainproof figures                      rebuild tables and figures from the held-out summaries (needs R for the plots)
-  plainproof verify                       check every sealed file against both freezes
-"""
-import argparse
-import hashlib
-import os
-import subprocess
-import sys
-from pathlib import Path
-
-
-def find_root(given):
-    root = Path(given or os.getcwd()).resolve()
-    if not (root / 'substitutions.json').exists() or not (root / 'reports' / 'edm-v2-e0' / 'freeze.json').exists():
-        raise SystemExit(f'{root} is not a plainproof checkout (no substitutions.json or reports/); use --root')
-    return root
-
-
-def sh(root, *cmd):
-    env = dict(os.environ)
-    env.setdefault('REPRO_DATA', str(root / 'data'))
-    print('+', ' '.join(str(c) for c in cmd), flush=True)
-    r = subprocess.run([str(c) for c in cmd], cwd=root, env=env)
-    if r.returncode:
-        raise SystemExit(r.returncode)
-
-
-def sha(p):
-    return hashlib.sha256(Path(p).read_bytes()).hexdigest()
-
-
-def main(argv=None):
-    ap = argparse.ArgumentParser(prog='plainproof', description=__doc__.split('\n\n')[0])
-    ap.add_argument('--root', help='repository root (default: current directory)')
-    sub = ap.add_subparsers(dest='cmd', required=True)
-    f = sub.add_parser('fetch'); f.add_argument('what', nargs='?', default='all', choices=('bobcat', 'heldout', 'all'))
-    sub.add_parser('fit')
-    r = sub.add_parser('run'); r.add_argument('study', choices=('e0', 'e0b'))
-    sub.add_parser('figures')
-    sub.add_parser('verify')
-    a = ap.parse_args(argv)
-    root = find_root(a.root)
-    py = sys.executable
-    data = Path(os.environ.get('REPRO_DATA', root / 'data'))
-    if a.cmd == 'fetch':
-        if a.what in ('bobcat', 'all'):
-            sh(root, 'bash', 'scripts/bobcat/fetch_bobcat.sh')
-        if a.what in ('heldout', 'all'):
-            sh(root, 'bash', 'scripts/fetch_heldout.sh')
-    elif a.cmd == 'fit':
-        sh(root, py, 'scripts/edm_v2_e0/fit_assist2009.py', data / 'edm-v2-e0' / 'item-fit')
-    elif a.cmd == 'run':
-        if a.study == 'e0':
-            out = data / 'edm-v2-e0' / 'heldout'
-            sh(root, py, 'scripts/edm_v2_e0/heldout.py', 'run', out)
-            sh(root, py, 'scripts/edm_v2_e0/heldout.py', 'summarize', out)
-        else:
-            out = data / 'edm-v2-e0b' / 'heldout'
-            sh(root, py, 'scripts/edm_v2_e0b/run.py', 'run', out)
-            sh(root, py, 'scripts/edm_v2_e0b/run.py', 'summarize', out)
-    elif a.cmd == 'figures':
-        s0, s1 = data / 'edm-v2-e0' / 'heldout' / 'summary.json', data / 'edm-v2-e0b' / 'heldout' / 'summary.json'
-        sh(root, py, 'scripts/edm_v2_e0/report.py', s0)
-        sh(root, 'bash', 'scripts/rscript.sh', 'scripts/edm_v2_e0/plot_error_vs_d.R', 'reports/edm-v2-e0/error-vs-d.csv',
-           'reports/edm-v2-e0/figures/error-vs-d.png', sha(s0))
-        sh(root, py, 'scripts/edm_v2_e0b/report.py', s1)
-        sh(root, 'bash', 'scripts/rscript.sh', 'scripts/edm_v2_e0b/plot_probe_auc.R', 'reports/edm-v2-e0b/probe-auc-vs-d.csv',
-           'reports/edm-v2-e0b/figures/probe-auc-vs-d.png', sha(s1))
-    elif a.cmd == 'verify':
-        sh(root, py, 'scripts/verify_frozen.py')
-
-
-if __name__ == '__main__':
-    main()