bindery-cli 0.32.0__tar.gz → 0.33.0__tar.gz

{bindery_cli-0.32.0 → bindery_cli-0.33.0}/.gitignore

@@ -12,6 +12,11 @@ testing_facility/books_to_fix/*
 !testing_facility/books_to_fix/.gitkeep
 testing_facility/top500candidates/*.epub
 
+# test_facility EPUBs stopped being tracked in v0.33.0 (forward-only; 9 of
+# the 10 books are commercial and one untracked leak on a public repo is
+# one too many). Files stay on disk, out of git.
+test_facility/*.epub
+
 # Compiled Java scratch (FastSweep harness)
 *.class
 

{bindery_cli-0.32.0 → bindery_cli-0.33.0}/CLAUDE.md

@@ -52,7 +52,11 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
     decision by `run phase1` on both the read-only and apply paths.
   Do not let any NEW fix touch content without its own flag; if a candidate repair cannot
   be made deterministically safe, it does not belong here — report it for manual repair
-  instead.
+  instead. A fourth, smaller group sits beside the structural repairs and lossy strips:
+  the safe opt-ins (`--fix-ids`, `--add-img-alt`, `--strip-bad-attrs`,
+  `--escape-unknown-entities`) plus `--encode-url-spaces`, which repair without
+  altering visible markup; the "twelve structural repairs and three lossy strips"
+  phrasing elsewhere in these docs was never the complete inventory.
 - **The gate is the safety contract.** Never apply a repair epubcheck has not accepted.
   Respect the two-mode logic in `validate.gate` (fatal-fixing tolerates error unmasking;
   error-cleanup does not). The lossy strips (`--strip-pagination`, `--strip-broken-tags`,

{bindery_cli-0.32.0 → bindery_cli-0.33.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: bindery-cli
-Version: 0.32.0
+Version: 0.33.0
 Summary: Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library.
 Author: vrnvctss
 License: MIT
@@ -82,6 +82,8 @@ Every repair is gated by [epubcheck](https://github.com/w3c/epubcheck). The acce
 
 Introducing a net-new fatal is always rejected. If epubcheck itself fails to run (crash, timeout, unparsable output), the book is reported as an error and never applied; only an explicit `--no-validate` skips the gate. Originals are never modified except by an explicit, atomic in-place replace (see below), and even then only after the gate accepts the result.
 
+Under the hood, the oracle can run two ways: the usual `epubcheck --json -` subprocess, or a small in-process daemon that compiles `FastDaemon.java` into a temp directory at runtime and drives a warm JVM over a pipe. The daemon is a pool bounded by `--workers` (serial runs use one), its every roundtrip is bounded by the caller's timeout, and any failure tears it down and falls back to the subprocess for good — the two paths must never disagree about a book, and the subprocess oracle is the fallback by design. The daemon is also the scariest under-tested code in the repo and depends on a matched javac/java pair; its status is recorded in the roadmap's Phase 12 postscript and Phase 14 backlog.
+
 The lossy modes (`--strip-pagination`, `--strip-broken-tags`, and `--strip-watermarks`) are the exception to the "must improve" rule. Since they remove visible markup rather than correcting XML schema violations, epubcheck counts often remain unchanged. They are accepted when the result is **no worse** (no net-new fatals or errors), relying on strict programmatic safety nets instead.
 
 ## Install
@@ -192,10 +194,12 @@ Apply all safe and lossy repairs directly into the Calibre database natively:
 bindery library ~/docs/Calibre\ Library --only all --apply --all --install-to-calibre
 ```
 
-- `--only {fatals,ncx,all}` restricts the candidate set. `ncx` targets NCX-001 mismatches (detected without epubcheck); `fatals` needs `--audit`.
+- `--only {fatals,ncx,all}` restricts the candidate set. `ncx` targets NCX-001 mismatches (detected without epubcheck); `fatals` needs `--audit` or `--sweep`.
 - `--id <ids>` limits the sweep to a comma-separated list of Calibre book IDs, skipping the full library walk.
 - `--audit CSV` (the `fatals,errors,warnings,path` format produced by an epubcheck sweep) skips clean books so a run is fast. Paths are resolved on both sides, and a CSV that matches nothing triggers a loud warning instead of silently selecting zero books.
-- `--sweep` replaces the CSV step entirely: it runs a live epubcheck sweep for candidate selection and reuses each result as that book's before-measurement, so no book is checked twice. bindery-cli first tries a persistent Java daemon (`EpubcheckDaemon`) to eliminate the per-book JVM startup; in practice the daemon cannot boot on most installs (its classpath carries only epubcheck's launcher jar, not the `lib/` dependencies epubcheck itself puts on the classpath), and the run silently falls back to the standard per-book epubcheck subprocess at the usual seconds per book. Budget accordingly. Fixing the classpath is a recorded decision (it would also change the counts the gate measures), not a shipped behavior. Combine with `--only fatals` for a self-contained "find and fix the broken books" run.
+- `--limit N` processes at most N candidates and stops opening archives after them (lazy within one worker window). A value below 1 is a usage error.
+- `--min-chars N` and `--thin-chars N` are the audit's EMPTY and THIN body-text thresholds (defaults 2000 and 20000); `--min-chars` above `--thin-chars` is a usage error. `--max-doc-chars N` is the audit monolithic-document flag threshold (default 300,000).
+- `--sweep` replaces the CSV step entirely: it runs a live epubcheck sweep for candidate selection and reuses each result as that book's before-measurement, so no book is checked twice. bindery-cli may serve checks from a small persistent Java daemon (see the gate section above for how it is bounded and how it fails safe); wherever the daemon cannot serve, the run falls back to the standard per-book epubcheck subprocess at the usual seconds per book. Combine with `--only fatals` for a self-contained "find and fix the broken books" run.
 - `--workers N` runs the sweep's candidate pass through N concurrent epubcheck workers (default 1: serial, unchanged). The subprocess releases the GIL, so threads parallelize the oracle honestly; books are checked in windows of N consumed in order, so the candidate set matches the serial sweep and `--limit` stays lazy within one window of overshoot. The repair phase stays serial: that is where the shared workdir and the atomic-replacement contract live.
 - `--json FILE` writes a machine-readable report of the whole run (per-book status, before/after counts, applied flag, summary totals). `--manual-list FILE` writes the paths of every book that was not auto-repaired, one per line, ready for manual follow-up.
 - `--apply` is required to write; the default is a dry run. `--backup DIR` mirrors originals before replacing; `--backup-inplace` writes `.epub.bak` beside each file.
@@ -204,7 +208,8 @@ bindery library ~/docs/Calibre\ Library --only all --apply --all --install-to-ca
 - `--all` automatically turns on all opt-in non-fatal fixes and lossy strips (pagination, watermarks, bad attributes, unknown entities, image alt tags, etc.) in a single run.
 - Only the `.epub` is replaced. `metadata.opf`, `cover.jpg`, and `metadata.db` are left for Calibre's Quality Check sync to reconcile.
 - A per-book progress line goes to stderr (stdout stays a clean report); `--quiet` suppresses it. A corrupt or unreadable book is reported and skipped, never aborting the sweep.
-- Exit codes: 0 for a clean sweep, 1 for a usage error, 2 when any book was rejected, unreadable, or failed epubcheck (for scripts and cron).
+- Exit codes: 0 for a clean sweep, 1 for a usage error, 2 when any book was rejected, unreadable, or failed epubcheck (for scripts and cron). Note that argparse-level misuse (an unknown flag or a malformed argument) exits with 2 before any validation runs; the tool's own usage validations exit 1.
+- A `partial` book (improved but still unable to open) is reported for manual follow-up and does not fail a `library` run; `run phase1` counts it as trouble and exits 2.
 - `repair` refuses to overwrite an existing output file unless `--force` is given.
 
 ## Run slices
@@ -246,15 +251,22 @@ cd ~/docs/Calibre\ Library && bindery run phase3 --ids 5071,5072 --json phase3-r
 
 `scripts/` holds standalone, read-only utilities that are useful for EPUB maintenance but fall outside bindery-cli's repair contract (fixing what they find would be a content change, which bindery-cli makes only via the opt-in `--strip-pagination`):
 
-- `find_missing_images.py`: scans a library tree and reports every book whose `<img>` tags point at files that do not exist inside the archive (a common defect in converted EPUBs). Reads the archives in place; nothing is unpacked or written. The library path is set at the bottom of the script.
 - `fast_sweep.py`: compiles and drives `FastSweep.java`, a parallel JVM epubcheck sweep that pays JVM startup once instead of per book (a 7,000-book dry-run sweep drops from hours to minutes). `--mode=audit` emits the `fatals,errors,warnings,path` CSV that `bindery library --audit` reads; `--mode=extract` emits per-book error codes for aggregation (`--summary` prints the per-code report). The epubcheck jar is located from the `epubcheck` launcher or `EPUBCHECK_JAR`; the harness compiles once (`--release 25`) and is cached on mtime.
 
+The older `find_*.py` detection wedges and the `sweep.sh`/`FastSweepExtract.java` prototypes were removed: every detection they performed is now a shipped, tested flag (missing images are `--prune-missing-resources`, and the rest of the RSC-005/PKG-010 family shipped through Phases 6-12), and keeping two copies meant future behavior changes landed in only one of them. Git history preserves them.
+
 ## Development
 
 ```sh
-./run_tests.sh        # unittest suite
+./run_tests.sh        # ruff (CI's pin) + unittest suite
 ```
 
+The suite builds its own synthetic EPUBs in temporary directories. `testing_facility/`
+holds staged real-book copies for manual dry runs (gitignored); `test_facility/` holds
+loose local books that stay on disk but out of git (untracked since v0.33.0,
+forward-only, because 9 of the 10 are commercial and one untracked leak on a public
+repo is one too many).
+
 See [spec.md](spec.md) for the full contract and [roadmap.md](roadmap.md) for what is planned.
 
 ## License

{bindery_cli-0.32.0 → bindery_cli-0.33.0}/README.md

@@ -60,6 +60,8 @@ Every repair is gated by [epubcheck](https://github.com/w3c/epubcheck). The acce
 
 Introducing a net-new fatal is always rejected. If epubcheck itself fails to run (crash, timeout, unparsable output), the book is reported as an error and never applied; only an explicit `--no-validate` skips the gate. Originals are never modified except by an explicit, atomic in-place replace (see below), and even then only after the gate accepts the result.
 
+Under the hood, the oracle can run two ways: the usual `epubcheck --json -` subprocess, or a small in-process daemon that compiles `FastDaemon.java` into a temp directory at runtime and drives a warm JVM over a pipe. The daemon is a pool bounded by `--workers` (serial runs use one), its every roundtrip is bounded by the caller's timeout, and any failure tears it down and falls back to the subprocess for good — the two paths must never disagree about a book, and the subprocess oracle is the fallback by design. The daemon is also the scariest under-tested code in the repo and depends on a matched javac/java pair; its status is recorded in the roadmap's Phase 12 postscript and Phase 14 backlog.
+
 The lossy modes (`--strip-pagination`, `--strip-broken-tags`, and `--strip-watermarks`) are the exception to the "must improve" rule. Since they remove visible markup rather than correcting XML schema violations, epubcheck counts often remain unchanged. They are accepted when the result is **no worse** (no net-new fatals or errors), relying on strict programmatic safety nets instead.
 
 ## Install
@@ -170,10 +172,12 @@ Apply all safe and lossy repairs directly into the Calibre database natively:
 bindery library ~/docs/Calibre\ Library --only all --apply --all --install-to-calibre
 ```
 
-- `--only {fatals,ncx,all}` restricts the candidate set. `ncx` targets NCX-001 mismatches (detected without epubcheck); `fatals` needs `--audit`.
+- `--only {fatals,ncx,all}` restricts the candidate set. `ncx` targets NCX-001 mismatches (detected without epubcheck); `fatals` needs `--audit` or `--sweep`.
 - `--id <ids>` limits the sweep to a comma-separated list of Calibre book IDs, skipping the full library walk.
 - `--audit CSV` (the `fatals,errors,warnings,path` format produced by an epubcheck sweep) skips clean books so a run is fast. Paths are resolved on both sides, and a CSV that matches nothing triggers a loud warning instead of silently selecting zero books.
-- `--sweep` replaces the CSV step entirely: it runs a live epubcheck sweep for candidate selection and reuses each result as that book's before-measurement, so no book is checked twice. bindery-cli first tries a persistent Java daemon (`EpubcheckDaemon`) to eliminate the per-book JVM startup; in practice the daemon cannot boot on most installs (its classpath carries only epubcheck's launcher jar, not the `lib/` dependencies epubcheck itself puts on the classpath), and the run silently falls back to the standard per-book epubcheck subprocess at the usual seconds per book. Budget accordingly. Fixing the classpath is a recorded decision (it would also change the counts the gate measures), not a shipped behavior. Combine with `--only fatals` for a self-contained "find and fix the broken books" run.
+- `--limit N` processes at most N candidates and stops opening archives after them (lazy within one worker window). A value below 1 is a usage error.
+- `--min-chars N` and `--thin-chars N` are the audit's EMPTY and THIN body-text thresholds (defaults 2000 and 20000); `--min-chars` above `--thin-chars` is a usage error. `--max-doc-chars N` is the audit monolithic-document flag threshold (default 300,000).
+- `--sweep` replaces the CSV step entirely: it runs a live epubcheck sweep for candidate selection and reuses each result as that book's before-measurement, so no book is checked twice. bindery-cli may serve checks from a small persistent Java daemon (see the gate section above for how it is bounded and how it fails safe); wherever the daemon cannot serve, the run falls back to the standard per-book epubcheck subprocess at the usual seconds per book. Combine with `--only fatals` for a self-contained "find and fix the broken books" run.
 - `--workers N` runs the sweep's candidate pass through N concurrent epubcheck workers (default 1: serial, unchanged). The subprocess releases the GIL, so threads parallelize the oracle honestly; books are checked in windows of N consumed in order, so the candidate set matches the serial sweep and `--limit` stays lazy within one window of overshoot. The repair phase stays serial: that is where the shared workdir and the atomic-replacement contract live.
 - `--json FILE` writes a machine-readable report of the whole run (per-book status, before/after counts, applied flag, summary totals). `--manual-list FILE` writes the paths of every book that was not auto-repaired, one per line, ready for manual follow-up.
 - `--apply` is required to write; the default is a dry run. `--backup DIR` mirrors originals before replacing; `--backup-inplace` writes `.epub.bak` beside each file.
@@ -182,7 +186,8 @@ bindery library ~/docs/Calibre\ Library --only all --apply --all --install-to-ca
 - `--all` automatically turns on all opt-in non-fatal fixes and lossy strips (pagination, watermarks, bad attributes, unknown entities, image alt tags, etc.) in a single run.
 - Only the `.epub` is replaced. `metadata.opf`, `cover.jpg`, and `metadata.db` are left for Calibre's Quality Check sync to reconcile.
 - A per-book progress line goes to stderr (stdout stays a clean report); `--quiet` suppresses it. A corrupt or unreadable book is reported and skipped, never aborting the sweep.
-- Exit codes: 0 for a clean sweep, 1 for a usage error, 2 when any book was rejected, unreadable, or failed epubcheck (for scripts and cron).
+- Exit codes: 0 for a clean sweep, 1 for a usage error, 2 when any book was rejected, unreadable, or failed epubcheck (for scripts and cron). Note that argparse-level misuse (an unknown flag or a malformed argument) exits with 2 before any validation runs; the tool's own usage validations exit 1.
+- A `partial` book (improved but still unable to open) is reported for manual follow-up and does not fail a `library` run; `run phase1` counts it as trouble and exits 2.
 - `repair` refuses to overwrite an existing output file unless `--force` is given.
 
 ## Run slices
@@ -224,15 +229,22 @@ cd ~/docs/Calibre\ Library && bindery run phase3 --ids 5071,5072 --json phase3-r
 
 `scripts/` holds standalone, read-only utilities that are useful for EPUB maintenance but fall outside bindery-cli's repair contract (fixing what they find would be a content change, which bindery-cli makes only via the opt-in `--strip-pagination`):
 
-- `find_missing_images.py`: scans a library tree and reports every book whose `<img>` tags point at files that do not exist inside the archive (a common defect in converted EPUBs). Reads the archives in place; nothing is unpacked or written. The library path is set at the bottom of the script.
 - `fast_sweep.py`: compiles and drives `FastSweep.java`, a parallel JVM epubcheck sweep that pays JVM startup once instead of per book (a 7,000-book dry-run sweep drops from hours to minutes). `--mode=audit` emits the `fatals,errors,warnings,path` CSV that `bindery library --audit` reads; `--mode=extract` emits per-book error codes for aggregation (`--summary` prints the per-code report). The epubcheck jar is located from the `epubcheck` launcher or `EPUBCHECK_JAR`; the harness compiles once (`--release 25`) and is cached on mtime.
 
+The older `find_*.py` detection wedges and the `sweep.sh`/`FastSweepExtract.java` prototypes were removed: every detection they performed is now a shipped, tested flag (missing images are `--prune-missing-resources`, and the rest of the RSC-005/PKG-010 family shipped through Phases 6-12), and keeping two copies meant future behavior changes landed in only one of them. Git history preserves them.
+
 ## Development
 
 ```sh
-./run_tests.sh        # unittest suite
+./run_tests.sh        # ruff (CI's pin) + unittest suite
 ```
 
+The suite builds its own synthetic EPUBs in temporary directories. `testing_facility/`
+holds staged real-book copies for manual dry runs (gitignored); `test_facility/` holds
+loose local books that stay on disk but out of git (untracked since v0.33.0,
+forward-only, because 9 of the 10 are commercial and one untracked leak on a public
+repo is one too many).
+
 See [spec.md](spec.md) for the full contract and [roadmap.md](roadmap.md) for what is planned.
 
 ## License

{bindery_cli-0.32.0 → bindery_cli-0.33.0}/patchnotes.md

@@ -1,4 +1,101 @@
 # bindery-cli Patch Notes
+## v0.33.0 (2026-09-10)
+
+### Phase 14 hardening: the apply path, the audit, and the safety contract's tests
+
+The rest of the 2026-09-08 audit sweep's backlog, in the same shape as
+v0.32.0: every fix confirmed against current code first, each with its
+regression test.
+
+**Apply-path and oracle safety**
+
+- **A failed write can no longer abort a sweep raw.** The backup,
+  replace, and install calls sat bare in the loop, so an ENOSPC or
+  EACCES partway through a multi-hour run died with a traceback and no
+  record. Failures are now error Outcomes; the summary and JSON always
+  land, and exit 2 says scripts should look.
+- **Backups rotate and stay out of the candidate set.** A second
+  `--apply` used to clobber the first backup (the only copy of the
+  author original); backups now rotate (`book.epub.bak2`, `.bak3`, ...).
+  A `--backup` directory inside the library root is refused (its copies
+  would be swept as candidates on the next run).
+- **`--workers N` parallelism is real.** With epubcheck present, every
+  worker serialized behind the daemon's one pipe. The daemon is now a
+  bounded pool sized by `--workers` (default 1: the old behavior), a
+  busy pool falls back to subprocesses instead of blocking, every
+  roundtrip is bounded by the caller's timeout, and a daemon that dies
+  or wedges is torn down and never retried — one failed attempt exhausts
+  the pool so the sweep degrades to pure subprocess checks instead of
+  respawning a JVM per book. Measured: 8 books, 4 workers, two ~10s
+  waves. The evidence for the recorded daemon-classpath decision grew:
+  this machine's javac (27-ea) and java (25) cannot load each other's
+  classes, so the daemon has never served a count here, and when forced
+  to run its counts diverge from the CLI JSON oracle (243 vs 4 on one
+  book) — enabling it would change gate outcomes.
+- **Smaller apply papercuts:** `--limit < 1` and a missing `--audit`
+  file are usage errors, not tracebacks; the fresh-format install
+  registers the catalogued file's own name instead of `repaired`; and
+  phase1 notes the dry-run `--backup` no-op instead of ignoring it.
+
+**audit.py correctness**
+
+- **ToC accounting reads inside the open zip,** a declared-but-absent
+  nav/NCX no longer poisons `Book.corrupt` (a healthy book with a
+  leftover `toc.ncx` entry is not a damaged archive), NCX `content src`
+  resolves against the NCX's own directory (a nested NCX counted every
+  target absent), and ToC hrefs are entity-decoded before comparison.
+- **Archive and spine verdicts are first-class in library mode.** Their
+  sections were dead code in the library report loop: a CRC-corrupt book
+  printed `emptytext CLEAN` and exited 0 there while directory mode said
+  CORRUPT and exited 1. Both modes now agree.
+- **The record pipeline's edges hold:** a failed `--json` write becomes a
+  trouble exit instead of a post-scan traceback; `--tag` skips
+  expected-foreign content findings; an unresolvable spine itemref is
+  counted into the emptytext detail instead of silently reading EMPTY;
+  DRM-encrypted entries get their own ENCRYPTED verdict
+  ("not repairable, skip") instead of CORRUPT re-source; and
+  part1a/part1b/part2-style spines no longer read as a broken span.
+- **Dead surfaces removed** (a duplicate argparse CLI, an unreachable
+  analyzer, duplicated imports) and directory mode catches uppercase
+  `.EPUB` like the library walk always did.
+- **Seven audit papercuts** closed: the block-stack pop now matches the
+  closing tag, the nav property matches whole tokens (`data-nav` is not
+  the nav), exit 3 is documented, the JSON analyzers list includes the
+  always-on verdicts, the console problem counter counts books like
+  `summary.problems`, `--min-chars` above `--thin-chars` is a usage
+  error, and duplicate archive entries are counted instead of resolving
+  last-wins in silence.
+
+**Tests, hygiene, and docs**
+
+- **Direct tests for the safety contract:** gate()/no_worse() including
+  the error-unmasking branch, the watermark gated-apply path,
+  atomic_replace failure injection (temp cleaned, target untouched,
+  re-raise), `--install-to-calibre` wiring, and library-mode `--tag` end
+  to end through cquarry's write path. Suite grew 342 → 381.
+- **The weak two percent cleaned:** mid-file `__main__` blocks,
+  constant-assertion tests, and module aliases gone; the
+  non-interactive wiring is asserted through the real runner's JSON.
+- **run_tests.sh runs CI's exact ruff pins** before the suite, so the
+  local loop fails where CI fails.
+- **Repo hygiene:** the ten subsumed `scripts/find_*.py` wedges plus
+  `sweep.sh` and `FastSweepExtract.java` are retired (git keeps them);
+  the ten commercial EPUBs in `test_facility/` are untracked
+  (forward-only, files stay on disk) and the dead never-run suite there
+  is deleted.
+- **Docs:** the daemon is admitted to exist (README gate section, spec
+  gate section) with its bounding and fail-safe fallback; README's
+  `--sweep` bullet stopped describing the fixed classpath bug as current;
+  the exit-code contract documents argparse's own exit 2 and the partial
+  book divergence; CLAUDE.md's exception taxonomy names the safe
+  opt-ins group.
+
+**Recorded for Brandon (unchanged, evidence sharpened):** the daemon
+classpath decision, the exit-code-from-partial-books decision (both
+options written into its roadmap box; unifying on "trouble" is the
+recommendation), and the spec.md reconciliation draft (the non-goals
+vs. the shipped opt-ins, ready to apply on a go).
+
 ## v0.32.0 (2026-09-10)
 
 ### The four Bug Reports fixes (2026-09-08 sweep) and the anchored-attribute cluster

{bindery_cli-0.32.0 → bindery_cli-0.33.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "bindery-cli"
-version = "0.32.0"
+version = "0.33.0"
 description = "Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library."
 readme = "README.md"
 classifiers = [