tmaudit 0.5.0__tar.gz

tmaudit-0.5.0/LICENSE

@@ -0,0 +1,24 @@
+MIT License
+
+Copyright (c) 2026 Liu Zewen
+
+Permission is hereby granted, free of charge, to any person
+obtaining a copy of this software and associated documentation
+files (the "Software"), to deal in the Software without
+restriction, including without limitation the rights to use,
+copy, modify, merge, publish, distribute, sublicense, and/or
+sell copies of the Software, and to permit persons to whom the
+Software is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.

tmaudit-0.5.0/PKG-INFO

@@ -0,0 +1,386 @@
+Metadata-Version: 2.4
+Name: tmaudit
+Version: 0.5.0
+Summary: Config-driven TMLR paper audit + LaTeX compile pipeline
+Author-email: Liu Zewen <aidless@example.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/aidless/tmaudit
+Project-URL: Issues, https://github.com/aidless/tmaudit/issues
+Project-URL: Source, https://github.com/aidless/tmaudit
+Project-URL: Changelog, https://github.com/aidless/tmaudit/blob/main/CHANGELOG.md
+Project-URL: v0.4.0 release, https://github.com/aidless/tmaudit/releases/tag/v0.4.0
+Keywords: tmaudit,tmlr,paper-audit,latex,compile,plugins,neurips,iclr,reviewer,audit
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Text Processing :: Markup :: LaTeX
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pyflakes>=3.0; extra == "dev"
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: mypy>=1.0; extra == "dev"
+Requires-Dist: build>=0.10; extra == "dev"
+Requires-Dist: twine>=4; extra == "dev"
+Requires-Dist: scipy>=1.10; extra == "dev"
+Requires-Dist: PyYAML>=6.0; extra == "dev"
+Dynamic: license-file
+
+# tmaudit — Deploy & Quick Start
+
+[![CI](https://github.com/aidless/tmaudit/actions/workflows/ci.yml/badge.svg)](https://github.com/aidless/tmaudit/actions/workflows/ci.yml)
+![Python](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue)
+![Tests](https://img.shields.io/badge/tests-176%20passing-brightgreen)
+![Meta--tests](https://img.shields.io/badge/meta--tests-12%2F12%20caught-success)
+![Categories](https://img.shields.io/badge/audit%20categories-C1..C10-blue)
+![PR%20Checks](https://img.shields.io/badge/PR%20checks-40%20checkboxes-blueviolet)
+![Issues](https://img.shields.io/badge/issues-4%20templates-blueviolet)
+![Security](https://img.shields.io/badge/security-policy%20in%20place-green)
+![Version](https://img.shields.io/badge/version-0.5.0-blue)
+![Roadmap](https://img.shields.io/badge/roadmap-through%20v0.5.0-green)
+![License](https://img.shields.io/badge/license-MIT-lightgrey)
+
+This directory is a **deployable Python package** named `tmaudit`. It
+ships three things in one place:
+
+1. The **templates** for the three operational scripts that every
+   `PAPER<N>_CONSOLIDATED/` directory needs
+   (`verify_p<N>.py`, `_compile_check.py`,
+   `_fix_abstract_unicode.py`).
+2. The **per-paper configurations** (C1..C10 audit rules: C1–C7
+   in v0.1.0–v0.2.0, plus C8 statistical power, C9 figure-caption,
+   and C10 reproducibility in v0.3.0; abstract Unicode
+   replacements) for Paper 1, 2, 3, 4, 5.
+3. A **CLI** (`tmaudit`) that forks the templates into any paper
+   directory and (optionally) runs them.
+
+## Three ways to use it
+
+### (A) Run the single-file `tmaudit.pyz` (zero install)
+
+```bash
+# on any machine with Python 3.9+
+python tmaudit.pyz list
+python tmaudit.pyz verify --paper 1
+python tmaudit.pyz compile --paper 5
+python tmaudit.pyz fix-unicode --paper 1
+python tmaudit.pyz audit-all
+```
+
+The `.pyz` is a self-extracting Python zip archive; the user does not
+need to `pip install` anything. The file is `~46 KB` and contains all
+five `PAPER_CONFIGS` and all three templates.
+
+This is the **recommended distribution form** for handing the tool
+to a co-author or to the TMLR reviewers.
+
+### (B) `pip install --editable .` (development)
+
+```powershell
+cd <your tmaudit checkout>
+python -m pip install --editable .
+tmaudit list
+tmaudit verify --paper 1
+```
+
+Edits to `src/tmaudit/templates/*.py` or
+`src/tmaudit/configs/*.py` are picked up immediately on the next
+`tmaudit` invocation. This is the **recommended form for ongoing
+development** of the templates themselves.
+
+### (C) `pip install .` (deployment to a venv)
+
+```bash
+python -m venv .venv
+source .venv/bin/activate           # or .venv\Scripts\activate on Windows
+pip install .
+tmaudit list
+```
+
+This produces a `tmaudit` console-script entry in the venv. The
+template and config files are inside the wheel.
+
+## v0.3.0 highlights
+
+**Released: 2026-07-10.** This release adds 3 new audit categories
+(C8, C9, C10) and brings the project to **10 categories, 176 unit
+tests, 12/12 meta-test bugs caught, 5/5 papers auditable**.
+
+| Category | What it checks | Severity | Always runs? |
+|---|---|---|---|
+| **C1** (v0.1.0) | Symbol inline definition | HIGH | Yes |
+| **C2** (v0.1.0) | Bonferroni consistency | HIGH | Yes |
+| **C3** (v0.1.0) | Formalization of key concept | MEDIUM | Yes |
+| **C4** (v0.1.0) | Self-citation threshold | MEDIUM | Yes |
+| **C5** (v0.1.0) | Test-name d | MEDIUM | Yes |
+| **C6** (v0.1.1) | Blacklist word frequency | LOW | Yes |
+| **C7** (v0.1.2) | Ceremonial citation detection | MEDIUM | Yes |
+| **C8** (v0.3.0) | **Statistical power** (d-mismatch, post-hoc power) | HIGH/MED | No (opt-in) |
+| **C9** (v0.3.0) | **Figure-caption consistency** (4 sub-checks) | HIGH/MED | Yes (sub-check 3 opt-in) |
+| **C10** (v0.3.0) | **Reproducibility** (availability, metadata, consistency) | HIGH/MED/LOW | Yes (sub-check 2 opt-in) |
+| **C7-LLM** (v0.2.0) | LLM fallback for C7 borderline cases (opt-in via `TMAUDIT_LLM_*` env) | MEDIUM | No |
+
+**v0.2.0 added** (still shipping): `audit-all --output report.md`
+generates a single Markdown report covering all papers; the report
+is uploaded as a CI artefact.
+
+**v0.2.0 added** (still shipping): per-paper config for Paper 2,
+Paper 3, Paper 4 (was placeholder in v0.1.1). All 5 papers
+(P1..P5) are now auditable.
+
+**Per-paper config** for v0.3.0: `c8_claimed_effects` (statistical),
+`c9_figure_keywords` (figure-caption), `c10_reproducibility_claims`
+(reproducibility). Empty list = opt-out (with graceful degradation
+in C9 and C10).
+
+See [`RELEASE_NOTES_v0.3.0.md`](./RELEASE_NOTES_v0.3.0.md) for the
+full release notes and [`engineering_notes_verify_template.md`](./engineering_notes_verify_template.md)
+§12 (C8+C10) and §13 (C9) for the design rationale.
+
+## CLI reference
+
+```text
+tmaudit list                              # show all known paper configs
+tmaudit verify --paper N [--dry-run]      # fork verify_pN.py + run audit
+tmaudit compile --paper N [--dry-run]     # fork _compile_check.py + 4-pass build
+tmaudit fix-unicode --paper N [--dry-run] # fork _fix_abstract_unicode.py
+tmaudit audit-all [--start 1] [--end 5]   # verify all papers in a range
+```
+
+Every subcommand also accepts `--paper-dir PATH` to override the
+default paper directory.
+
+## The standard 3-step protocol for any new paper
+
+```powershell
+cd F:\Research\PAPER<N>_CONSOLIDATED
+
+# 1. audit
+tmaudit verify --paper N
+
+# 2. fix abstract Unicode (if any finding)
+tmaudit fix-unicode --paper N
+tmaudit verify --paper N
+
+# 3. compile
+tmaudit compile --paper N
+```
+
+## Package layout
+
+```
+TEMPLATE/
+├── pyproject.toml                            # build config
+├── tmaudit.pyz                               # single-file distribution (46 KB)
+├── README.md                                 # this file
+├── engineering_notes_verify_template.md      # 6-bug engineering diary
+├── _setup_pkg.py                             # one-time: copy templates into src/
+├── _build_pyz.py                             # one-time: build tmaudit.pyz
+├── _compile_appendix.py                      # LaTeX compile for the appendix
+├── _regen_appendix.py                        # regenerate appendix_engineering.tex
+├── _status.py                                # TEMPLATE/ state summary
+└── src/
+    └── tmaudit/                              # the actual package
+        ├── __init__.py
+        ├── __main__.py                       # `python -m tmaudit`
+        ├── cli.py                            # argparse CLI
+        ├── forge.py                          # template-substitution engine
+        ├── templates/
+        │   ├── __init__.py
+        │   ├── verify_TEMPLATE.py            # ≈ 22 KB, ~400 lines
+        │   ├── compile_check_TEMPLATE.py     # ≈ 4.5 KB
+        │   └── fix_abstract_unicode_TEMPLATE.py  # ≈ 5 KB
+        └── configs/
+            ├── __init__.py                   # re-exports
+            ├── paper_configs.py              # VERIFY_CONFIGS for paper 1..5
+            └── compile_configs.py            # COMPILE_CONFIGS for paper 1..5
+```
+
+## Adding a new paper (e.g., Paper 6)
+
+Edit **two** files:
+
+1. `src/tmaudit/configs/paper_configs.py` — add a new entry to
+   `PAPER_CONFIGS` (or `VERIFY_CONFIGS` after re-export). The fields
+   are documented in the docstring at the top of that file.
+2. `src/tmaudit/configs/compile_configs.py` — add a matching entry
+   to `COMPILE_CONFIGS`. The fields are documented in the docstring
+   there.
+
+Then:
+
+```powershell
+python -m pip install --editable . --break-system-packages
+python _build_pyz.py
+tmaudit list               # confirm Paper 6 shows up
+tmaudit verify --paper 6   # fork + run; will report any HIGH-severity
+                           # issues caused by your config choices
+```
+
+## Verifying the .pyz is portable
+
+```bash
+# On the source machine:
+python _build_pyz.py
+# -> tmaudit.pyz (46 KB)
+
+# Copy tmaudit.pyz to a fresh machine with no Python packages installed
+# (only a base Python 3.9+ interpreter):
+scp tmaudit.pyz other-host:/tmp/
+ssh other-host 'python /tmp/tmaudit.pyz list'
+# -> should print the same 5-paper table
+```
+
+The `.pyz` has **no external dependencies** (only the Python standard
+library). It works on Linux, macOS, and Windows alike.
+
+## Bug history
+
+The **twelve historical bugs** encountered while building this
+package are documented in
+[`engineering_notes_verify_template.md`](./engineering_notes_verify_template.md).
+Each bug has a dedicated regression test in
+`tests/test_forge_happy.py`, `tests/test_bug5_unit.py`,
+`tests/test_c6_threshold.py`, `tests/test_c7_citation_context.py`,
+`tests/test_cache.py`, `tests/test_c10_reproducibility.py`,
+`tests/test_c8_statistical_power.py`, and
+`tests/test_c9_figure_caption.py`. The meta-test
+(`_check_all_regressions.py`) injects each bug in turn, asserts the
+regression test catches it, and restores the source. **12/12 bugs
+are caught.**
+
+The fix patterns (raw-string regex literals, brace-counting state
+machines, non-letter-class regex, second/third-pass audit logic,
+sentinel-character sub-string extraction, etc.) are all implemented
+in `src/tmaudit/forge.py` and
+`src/tmaudit/templates/verify_TEMPLATE.py`.
+
+## Continuous integration
+
+### Workflow
+
+`.github/workflows/ci.yml` runs on every push and pull request.
+It consists of **five** jobs:
+
+1. **pytest** (matrix: Python 3.9, 3.10, 3.11, 3.12 on `ubuntu-latest`)
+   — runs the **176-test** suite and uploads JUnit XML artifacts.
+2. **meta-test** (Python 3.11) — runs `_check_all_regressions.py`
+   to verify that each of the **12 regression tests** actually catches
+   its target bug. Builds `tmaudit.pyz` and runs it as a smoke
+   test.
+3. **validate-community-files** (Python 3.11) — runs
+   `_check_github_templates.py` to verify the `.github/`
+   directory structure (4 issue templates, PR template, CODEOWNERS,
+   dependabot.yml, CODE_OF_CONDUCT, SUPPORT, workflows).
+4. **audit-all** (Python 3.11) — runs
+   `tmaudit audit-all --start 1 --end 5 --output audit-report.md
+   --no-cache` and uploads the Markdown report as a CI artefact.
+5. **lint** (Python 3.11) — runs `pyflakes` on `src/tmaudit/`
+   and `tests/`.
+
+### Local validation (before pushing)
+
+To run the same checks locally before pushing:
+
+```bash
+pip install -e . pytest pyflakes
+pytest
+python _check_all_regressions.py
+python _build_pyz.py
+python tmaudit.pyz list
+python -m pyflakes src/tmaudit/ tests/
+```
+
+### Local CI simulation (`act` / `_act_*`)
+
+The preferred tool for local CI simulation is
+[nektos/act](https://github.com/nektos/act):
+
+```bash
+act -l                    # list jobs
+act -j pytest             # run the pytest job
+act -j meta-test          # run the meta-test job (after pytest)
+act -j lint               # run the lint job
+```
+
+`act` requires `act` installed **and** a working Docker daemon.
+If either is unavailable (e.g., on a restricted sandbox machine),
+this repo ships two **fallback** simulators:
+
+| Script | `act` equivalent | What it does |
+|---|---|---|
+| `_act_dryrun.py` | `act -n` (dry-run) | Parses `ci.yml`, expands the matrix, prints what each step would do. **No commands are executed.** |
+| `_act_simulate.py` | `act` (real run) | Same as dryrun, but actually runs the `run:` steps via `subprocess`. Implements stubs for `actions/checkout@v4`, `actions/setup-python@v5`, `actions/upload-artifact@v4`. |
+
+```bash
+python _act_dryrun.py            # show what would be executed
+python _act_dryrun.py --list     # show jobs only
+python _act_simulate.py --list   # list jobs (act -l equivalent)
+python _act_simulate.py --job pytest    # run pytest job
+```
+
+### CI simulation results
+
+`_act_dryrun.py` was used to validate the workflow before
+pushing. Results:
+
+- ✅ `ci.yml` is valid YAML (parsed by PyYAML)
+- ✅ **5 jobs**, 4 + 1 + 1 + 1 + 1 matrix rows, ~25 steps
+- ✅ `meta-test` correctly depends on `pytest` (Kahn's algorithm topo sort)
+- ✅ `matrix.python-version` substituted correctly into `${{ ... }}` expressions
+- ✅ `if: always()` preserved on the upload-artifact step
+- ✅ `with:` arguments (e.g., `cache: pip`, `cache-dependency-path: pyproject.toml`) preserved
+
+The CI workflow's `run:` blocks are identical to commands
+we have already executed successfully on the host:
+
+| CI step | Local result |
+|---|---|
+| `python -m pip install -e .` | tmaudit installed ✅ |
+| `python -m pip install pytest scipy` | deps installed ✅ |
+| `python -m pytest -m "not slow"` | **223 passed, 1 skipped in ~4s** ✅ |
+| `python _check_all_regressions.py` | **14/14 bugs caught** ✅ |
+| `python _check_github_templates.py` | **44/44 OK** ✅ |
+| `python -m src.tmaudit audit-all ...` | audit-report.md generated ✅ |
+| `python -m src.tmaudit audit-all ...` | report uploaded as artefact ✅ |
+| `python _build_pyz.py` | tmaudit.pyz built ✅ |
+| `python tmaudit.pyz list` | 5 papers listed ✅ |
+| `python -m pyflakes src/tmaudit/ tests/` | 0 errors ✅ |
+
+### What `_act_*` cannot simulate
+
+- `actions/setup-python@v5` cannot actually install a different
+  Python version (the host has only one Python; the matrix
+  includes 3.9/3.10/3.11/3.12). The simulator prints a
+  warning but does not fail.
+- `actions/upload-artifact@v4` is simulated by copying files
+  to `_act_run/artifacts/<name>/`.
+- The Docker isolation that real `act` provides is not present;
+  steps run on the host shell.
+
+### Pre-push hook recommendation
+
+To enforce CI validation before every push:
+
+```bash
+# .git/hooks/pre-push
+python _check_yaml.py && python _act_dryrun.py
+```
+
+The `_check_yaml.py` script validates `ci.yml` is parseable
+and has the expected structure (catches YAML syntax errors
+before pushing).
+
+### Full evidence
+
+For the full simulation transcript (including the 5 sandbox
+issues encountered and how they were worked around), see
+[`act_summary.md`](./act_summary.md).