bindery-cli 0.35.0__tar.gz → 0.37.0__tar.gz

{bindery_cli-0.35.0 → bindery_cli-0.37.0}/.github/workflows/publish.yml

@@ -43,3 +43,26 @@ jobs:
 
       - name: Publish to PyPI
         uses: pypa/gh-action-pypi-publish@release/v1
+
+  plugin-zip:
+    name: Build the Bindery Repair Calibre plugin zip
+    needs: test
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.14"
+
+      - name: Run the suite
+        run: |
+          pip install .[reserialize]
+          PYTHONPATH=src python -m unittest discover -s tests
+
+      - name: Build the plugin zip from the tagged tree
+        run: python scripts/build_plugin.py
+
+      - name: Attach BinderyRepair zip to the release
+        uses: softprops/action-gh-release@v2
+        with:
+          files: dist/BinderyRepair-*.zip

{bindery_cli-0.35.0 → bindery_cli-0.37.0}/CLAUDE.md

@@ -73,16 +73,17 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
 
 ## Layout
 
-- `src/bindery/transforms.py`: pure `str -> (str, int)` text transforms (including `strip_broken_tags`). Since v0.32.0: `strip_attrs_in_start_tags` anchors attribute edits on the quote-aware start-tag matcher, `_outside_protected` forwards arguments (context-carrying transforms can be decorated), `fix_id_colons` matches only the bare `id` attribute and only internal fragments (`fix_ncx_src_fragments` carries renames into the NCX), `fix_ncx_playorder` is anchored to `<navPoint>` start tags, and the css selector boundary covers namespaced and functional selector forms.
+- `src/bindery/transforms.py`: pure `str -> (str, int)` text transforms (including `strip_broken_tags`). Since v0.32.0: `strip_attrs_in_start_tags` anchors attribute edits on the quote-aware start-tag matcher, `_outside_protected` forwards arguments (context-carrying transforms can be decorated), `fix_id_colons` matches only the bare `id` attribute and only internal fragments (`fix_ncx_src_fragments` carries renames into the NCX), `fix_ncx_playorder` is anchored to `<navPoint>` start tags, and the css selector boundary covers namespaced and functional selector forms. Since v0.37.0: `fix_ncx_playorder` strips quotes before its already-correct comparison (regex groups carry quote characters; comparing quoted values against bare numbers counted phantom fixes on every sequential NCX and rewrote single-quoted correct attributes to double quotes).
 - `src/bindery/pagination.py`: the opt-in lossy page-number strip (runhead detection, page-layer decision, block-centric removal/merge, safety nets).
 - `src/bindery/watermark.py`: the opt-in lossy watermark strip (anchored and anchorless signature removal).
 - `src/bindery/reserialize.py`: structural repair via `html5lib`.
-- `src/bindery/audit.py`: read-only body-text audits (`content`, `pagenumbers`, `emptytext`, `ocr`, and `monolithic` since v0.21.0, `--max-doc-chars N`) behind the `audit` subcommand (v0.15.0). Since v0.18.0 library mode resolves EPUB paths through `cquarry.get_format_path()` and can apply a tag to flagged books via `cquarry.write.WritableCalibreDB` (only with explicit `--tag`; the only sanctioned write path). Since v0.19.0 `audit --id BOOK_ID` audits a single library book through cquarry's single-entity `get_book()` fetch (no library-wide cache; supports `--tag`; incompatible with the directory argument; comma lists since v0.23.0). Since v0.22.0 every archive entry is fully read (CRC + decompression) before analysis — a damaged entry reports its own CORRUPT verdict instead of feeding emptytext — and manifest/NCX references to absent files classify as `convention` (bloated ToC, consecutive span) or `fragment` (broken span). Hazard: the scan loop reuses `tag` as its per-book display column, so the `--tag` argument is captured as `audit_tag` at function entry — do not collapse them again. Since v0.29.0 `audit --json FILE` writes per-file analyzer verdicts in the `library --json` shape (directory, library, and single-book modes; with `--id`, exactly one id; emptytext omitted from a record when the archive verdict owns the body-text story).
+- `src/bindery/audit.py`: read-only body-text audits (`content`, `pagenumbers`, `emptytext`, `ocr`, `monolithic` since v0.21.0 with `--max-doc-chars N`, and `completeness` since v0.36.0) behind the `audit` subcommand (v0.15.0). Since v0.18.0 library mode resolves EPUB paths through `cquarry.get_format_path()` and can apply a tag to flagged books via `cquarry.write.WritableCalibreDB` (only with explicit `--tag`; the only sanctioned write path). Since v0.19.0 `audit --id BOOK_ID` audits a single library book through cquarry's single-entity `get_book()` fetch (no library-wide cache; supports `--tag`; incompatible with the directory argument; comma lists since v0.23.0). Since v0.22.0 every archive entry is fully read (CRC + decompression) before analysis — a damaged entry reports its own CORRUPT verdict instead of feeding emptytext — and manifest/NCX references to absent files classify as `convention` (bloated ToC, consecutive span) or `fragment` (broken span). Hazard: the scan loop reuses `tag` as its per-book display column, so the `--tag` argument is captured as `audit_tag` at function entry — do not collapse them again. Since v0.29.0 `audit --json FILE` writes per-file analyzer verdicts in the `library --json` shape (directory, library, and single-book modes; with `--id`, exactly one id; emptytext omitted from a record when the archive verdict owns the body-text story). Since v0.36.0 the completeness analyzer (Phase 15) reports the phase-1 spot-check per book: spine/prose-doc counts (prose = >= 400 visible chars), first/middle/last prose-doc opening/closing excerpts, trailing-ToC classification (`_trailing_toc`: short link lines dominate, no paragraph-length blocks), and the unreadable fraction. It is ADVISORY by contract (problem always False; ADVISORY on a trailing ToC or unreadable fraction >= 10%), never moves the exit code, and is skipped like emptytext when the archive verdict owns the book.
 - `src/bindery/epub.py`: archive rewrite, NCX uid sync, RepairReport, mismatch detection, and the opt-in structural-repair plumbing (including the CSS precondition scan).
 - `src/bindery/validate.py`: epubcheck wrapper, the `gate` (improvement) and `no_worse` (no-regression, for the lossy strips) acceptance bars.
-- `src/bindery/library.py`: Calibre walk, atomic replace, backups, and native format installation. Since v0.19.0 `CalibreIdResolver` resolves the book id from `metadata.db` through cquarry (one lazy path→id map per run; since v0.20.0 the map comes from `CalibreDB.format_path_index()`, re-normalized with `resolve().lower()` for the resolver's symlinked-directory and case-insensitive matching) — the `(id)` directory-name regex is only the no-catalog fallback. Since v0.24.0 `install_format()` places the repaired file atomically and updates the `data` row through `cquarry.write.WritableCalibreDB` (since v0.31.0 via `set_format`, cquarry 1.17's sanctioned remove+add in one transaction); the external `calibredb` CLI is gone. Since v0.32.0 a guessed id drives a row update only when metadata.db corroborates it (`_verify_guess`: books row exists, file in the book's own directory, catalogued `data.name` when a row exists); anything else saves in place and leaves the catalog untouched — never reintroduce directory-name guessing as the primary source: renamed/mismatched directories would replace the wrong book.
+- `src/bindery/library.py`: Calibre walk, atomic replace, backups, and native format installation. Since v0.19.0 `CalibreIdResolver` resolves the book id from `metadata.db` through cquarry (one lazy path→id map per run; since v0.20.0 the map comes from `CalibreDB.format_path_index()`, re-normalized with `resolve().lower()` for the resolver's symlinked-directory and case-insensitive matching); the `(id)` directory-name regex is only the no-catalog fallback. Since v0.24.0 `install_format()` places the repaired file atomically and updates the `data` row through `cquarry.write.WritableCalibreDB` (since v0.31.0 via `set_format`, cquarry 1.17's sanctioned remove+add in one transaction); the external `calibredb` CLI is gone. Since v0.32.0 a guessed id drives a row update only when metadata.db corroborates it (`_verify_guess`: books row exists, file in the book's own directory, catalogued `data.name` when a row exists); anything else saves in place and leaves the catalog untouched — never reintroduce directory-name guessing as the primary source: renamed/mismatched directories would replace the wrong book.
 - `src/bindery/cli.py`: `repair` and `library` subcommands, including `--all` and `--install-to-calibre`; since v0.29.0 also the `run` verbs: `phase1` (pre-import vetting over loose files: the audit battery composed with one gated repair sweep; read-only until `--apply-lossy`, which IS the lossy-strip consent) and `phase3` (the scoped post-import apply step: `library --id --sweep --only all --apply --all --install-to-calibre` plus a pre/post summary; mechanically refuses unscoped sweeps with exit 2). Both take `--non-interactive` and surface open questions as `decisions_needed` in their JSON, never prompts. They drive the shipped `run_library` through the real parser, so no flag can drift between the verb and the subcommand it wraps. `library --workers N` (v0.30.0) parallelizes only the sweep's candidate pass, in windows of N consumed in order so `--limit` stays lazy; the repair phase is never parallel (shared workdir, atomic-replacement contract).
-- `tests/`: transforms, end-to-end repair, atomic replace, pagination, watermarks, the audit analyzers (content/pagenumbers/emptytext/ocr/monolithic, archive integrity, spine classification), the library sweep and `CalibreIdResolver`, validate, reserialize, CLI wiring, and the version pin.
+- `plugin/__init__.py` + `scripts/build_plugin.py`: the Bindery Repair Calibre plugin (v0.37.0; identity `Bindery Repair` / `bindery_repair`, approved and built per the scoped spec in roadmap.md Phase 3). The builder vendors `transforms.py`, `epub.py`, `pagination.py`, `watermark.py`, `reserialize.py` byte-identically into the zip (the zip root is a package, so their relative imports resolve unchanged; the suite's drift test pins the equality) with the version tuple substituted from the single-source VERSION; `publish.yml` attaches `BinderyRepair-v<VERSION>.zip` to each release. The plugin runs ONLY the default pass (epubcheck cannot gate inside Calibre), never raises, is byte-idempotent (zero fixes returns the original path), testzip()-verifies its output, logs one line per book, and takes JSON config via `site_customization` (`log`, `log_path`, `max_size_mb` default 150, `epubcheck_path` experimental validation default OFF). Load it in tests through the `tests/test_plugin.py` stubs: the calibre namespace is installed in sys.modules and the package loads with a two-dir `__path__` (plugin dir first, then src/bindery) exactly like calibre's zipplugin loader shape.
+- `tests/`: transforms, end-to-end repair, atomic replace, pagination, watermarks, the audit analyzers (content/pagenumbers/emptytext/ocr/monolithic/completeness, archive integrity, spine classification), the library sweep and `CalibreIdResolver`, validate, reserialize, CLI wiring, the version pin, and the Calibre plugin (build drift + run behavior under calibre stubs).
 
 ## Conventions
 

{bindery_cli-0.35.0 → bindery_cli-0.37.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: bindery-cli
-Version: 0.35.0
+Version: 0.37.0
 Summary: Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library.
 Author: vrnvctss
 License: MIT
@@ -13,7 +13,7 @@ Classifier: License :: OSI Approved :: MIT License
 Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Utilities
 Requires-Python: >=3.14
-Requires-Dist: cquarry>=1.17.0
+Requires-Dist: cquarry>=1.19.0
 Requires-Dist: tqdm
 Requires-Dist: vir-tui>=2.2.0
 Provides-Extra: reserialize
@@ -57,8 +57,8 @@ Everything below is off until its flag is passed (or all at once via `--all`). T
 - **`--unwrap-block-in-inline`**: safely unwraps `<span>` tags that illegally contain a block-level element (e.g. `<div>` or `<p>`), leaving the block element intact.
 - **`--strip-invalid-value`**: systematically strips invalid `value="..."` attributes from elements like `<div>`, `<span>`, `<p>`, etc.
 - **`--fix-page-map`**: normalizes legacy page-map markup — drops the non-standard `page-map="..."` attribute from the OPF `<spine>` and adds `class="pages"` to classless NCX `<pageList>` elements (epubcheck rejects both on older HarperCollins / Anna's Archive conversions).
-- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package — `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or the wider aria family survive). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
-- **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents — `figure`/`section` to `div`, `figcaption` to `p` — keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
+- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or the wider aria family survive). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
+- **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents (`figure`/`section` to `div`, `figcaption` to `p`), keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
 - **`--unwrap-illegal-tags`**: strips completely invalid or deprecated HTML tags that break EPUB3 validation (`<st>`, `<sentence>`, `<o>`, `<w>`, `<pagebreak>`) while retaining their inner text. Any of those names styled as an *element selector* by an EPUB stylesheet (`.css` entries and inline `<style>` blocks alike; class/id selectors like `.st`/`#w` don't count) is protected for the whole book, guaranteeing format preservation.
 - **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items. Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
 - **`--strip-broken-anchors`**: strips href attributes that cannot resolve, keeping the anchor text byte-for-byte: a `#fragment` the target document does not define (RSC-020/RSC-012; NCX navTargets fall back to the document target, so chapter navigation survives) and unresolvable URI schemes (`kindle:`, `file:`).
@@ -95,6 +95,41 @@ uv tool install bindery-cli                            # core tools
 uv tool install "bindery-cli[reserialize]"             # incl. --reserialize
 ```
 
+## The Calibre plugin (Bindery Repair)
+
+Each release also ships **BinderyRepair-v<VERSION>.zip**, a Calibre plugin that
+repairs EPUBs as they are imported: it is attached to the GitHub release beside
+the PyPI wheel (install it via Calibre's Preferences, Plugins, "Load plugin
+from file"). The plugin is the CLI's always-on core pass only: the five
+well-formedness fixes plus the NCX pipeline. The structural repairs and lossy
+strips stay CLI-only, because their acceptance is the epubcheck gate and
+epubcheck cannot run inside Calibre; nothing runs ungated in the plugin, ever.
+
+Behavior and guarantees:
+
+- `run()` never raises and never touches the original file or `metadata.db`: a
+  repaired copy is built in a persistent temp file and handed back for import;
+  any trouble returns the original path and logs one line.
+- Byte-idempotent: an already-clean book yields zero fixes and the original
+  path, so re-adding a format re-runs the plugin harmlessly.
+- The rewritten archive is `testzip()`-verified before it is handed back.
+- One log line per book (fixed / no fixes / refused / errored), appended to
+  `bindery_repair.log` in Calibre's config directory by default.
+- Files over the size cap are refused (default 150MB) rather than stalling an
+  import.
+
+Configuration is the plugin's customization string, parsed as JSON:
+
+```json
+{"log": true, "log_path": "/path/bindery_repair.log", "max_size_mb": 150,
+ "epubcheck_path": null}
+```
+
+`epubcheck_path` opts into the experimental on-PATH validation mode: the
+repaired copy is re-measured with your epubcheck binary and refused unless it
+is no worse than the original. It defaults to off; the default pass is safe
+without a gate, and the mode costs two epubcheck runs per imported book.
+
 ## Usage
 
 Four verbs: `bindery repair` fixes a single EPUB epubcheck-gated, `bindery audit`
@@ -126,10 +161,15 @@ bindery audit ocr ~/docs/Calibre\ Library
 # Flag books where a single content document is too large for readers to render
 bindery audit monolithic ~/docs/Calibre\ Library
 
+# Spot-check completeness: prose coverage, first/middle/last excerpts, trailing ToC
+bindery audit completeness ~/docs/Calibre\ Library
+
 # Run all audits and generate a comprehensive CSV
 cd ~/docs/Calibre\ Library && bindery audit all
 ```
 
+**Completeness spot-check:** the `completeness` audit answers the pre-import question "does this book reach its end?" It reports, per book: the prose-document count against the spine (docs with 400+ visible characters), the opening and closing text of the first/middle/last prose doc, a trailing-ToC classification of the final spine doc (a link-dense chapter list after the real back matter), and the fraction of unreadable docs. A trailing ToC is reported as an advisory, never a failure: the closing excerpt always comes from the real back matter, so a human can confirm the book ends where the author ended it.
+
 Audits can also be run on a directory of loose `.epub` files by passing the path as the second argument:
 ```sh
 bindery audit pagenumbers /path/to/loose/epubs
@@ -204,7 +244,7 @@ bindery library ~/docs/Calibre\ Library --only all --apply --all --install-to-ca
 - `--json FILE` writes a machine-readable report of the whole run (per-book status, before/after counts, applied flag, summary totals). `--manual-list FILE` writes the paths of every book that was not auto-repaired, one per line, ready for manual follow-up.
 - `--apply` is required to write; the default is a dry run. `--backup DIR` mirrors originals before replacing; `--backup-inplace` writes `.epub.bak` beside each file.
 - `--install-to-calibre` resolves the Calibre book id from `metadata.db` via cquarry (one read-only path→id map per run), so a hand-renamed `Author/Title (id)/` directory can never send the repaired file to the wrong book. The directory-name regex is only a no-catalog fallback; with neither, the file is saved atomically in place.
-- `--install-to-calibre` installs the repaired EPUB through cquarry's write module: the file is placed atomically at the catalogued path (same filename, Calibre's layout never changes) and the format row is re-registered in one transaction, so the stored size stays truthful and the book lands in `metadata_dirtied` — Calibre regenerates its sidecar .opf on next start. No external `calibredb` call is involved. It falls back to atomic file replacement if the Calibre database ID cannot be resolved, and a database failure degrades to the in-place save with a warning. A directory-name-guessed id is trusted only when `metadata.db` corroborates it (the book exists and the file is that book's catalogued EPUB); an uncatalogued stray file inside a book directory is saved in place and the catalog row is left untouched.
+- `--install-to-calibre` installs the repaired EPUB through cquarry's write module: the file is placed atomically at the catalogued path (same filename, Calibre's layout never changes) and the format row is re-registered in one transaction, so the stored size stays truthful and the book lands in `metadata_dirtied`; Calibre regenerates its sidecar .opf on next start. No external `calibredb` call is involved. It falls back to atomic file replacement if the Calibre database ID cannot be resolved, and a database failure degrades to the in-place save with a warning. A directory-name-guessed id is trusted only when `metadata.db` corroborates it (the book exists and the file is that book's catalogued EPUB); an uncatalogued stray file inside a book directory is saved in place and the catalog row is left untouched.
 - `--all` automatically turns on all opt-in non-fatal fixes and lossy strips (pagination, watermarks, bad attributes, unknown entities, image alt tags, etc.) in a single run.
 - Only the `.epub` is replaced. `metadata.opf`, `cover.jpg`, and `metadata.db` are left for Calibre's Quality Check sync to reconcile.
 - A per-book progress line goes to stderr (stdout stays a clean report); `--quiet` suppresses it. A corrupt or unreadable book is reported and skipped, never aborting the sweep.

{bindery_cli-0.35.0 → bindery_cli-0.37.0}/README.md

@@ -35,8 +35,8 @@ Everything below is off until its flag is passed (or all at once via `--all`). T
 - **`--unwrap-block-in-inline`**: safely unwraps `<span>` tags that illegally contain a block-level element (e.g. `<div>` or `<p>`), leaving the block element intact.
 - **`--strip-invalid-value`**: systematically strips invalid `value="..."` attributes from elements like `<div>`, `<span>`, `<p>`, etc.
 - **`--fix-page-map`**: normalizes legacy page-map markup — drops the non-standard `page-map="..."` attribute from the OPF `<spine>` and adds `class="pages"` to classless NCX `<pageList>` elements (epubcheck rejects both on older HarperCollins / Anna's Archive conversions).
-- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package — `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or the wider aria family survive). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
-- **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents — `figure`/`section` to `div`, `figcaption` to `p` — keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
+- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or the wider aria family survive). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
+- **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents (`figure`/`section` to `div`, `figcaption` to `p`), keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
 - **`--unwrap-illegal-tags`**: strips completely invalid or deprecated HTML tags that break EPUB3 validation (`<st>`, `<sentence>`, `<o>`, `<w>`, `<pagebreak>`) while retaining their inner text. Any of those names styled as an *element selector* by an EPUB stylesheet (`.css` entries and inline `<style>` blocks alike; class/id selectors like `.st`/`#w` don't count) is protected for the whole book, guaranteeing format preservation.
 - **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items. Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
 - **`--strip-broken-anchors`**: strips href attributes that cannot resolve, keeping the anchor text byte-for-byte: a `#fragment` the target document does not define (RSC-020/RSC-012; NCX navTargets fall back to the document target, so chapter navigation survives) and unresolvable URI schemes (`kindle:`, `file:`).
@@ -73,6 +73,41 @@ uv tool install bindery-cli                            # core tools
 uv tool install "bindery-cli[reserialize]"             # incl. --reserialize
 ```
 
+## The Calibre plugin (Bindery Repair)
+
+Each release also ships **BinderyRepair-v<VERSION>.zip**, a Calibre plugin that
+repairs EPUBs as they are imported: it is attached to the GitHub release beside
+the PyPI wheel (install it via Calibre's Preferences, Plugins, "Load plugin
+from file"). The plugin is the CLI's always-on core pass only: the five
+well-formedness fixes plus the NCX pipeline. The structural repairs and lossy
+strips stay CLI-only, because their acceptance is the epubcheck gate and
+epubcheck cannot run inside Calibre; nothing runs ungated in the plugin, ever.
+
+Behavior and guarantees:
+
+- `run()` never raises and never touches the original file or `metadata.db`: a
+  repaired copy is built in a persistent temp file and handed back for import;
+  any trouble returns the original path and logs one line.
+- Byte-idempotent: an already-clean book yields zero fixes and the original
+  path, so re-adding a format re-runs the plugin harmlessly.
+- The rewritten archive is `testzip()`-verified before it is handed back.
+- One log line per book (fixed / no fixes / refused / errored), appended to
+  `bindery_repair.log` in Calibre's config directory by default.
+- Files over the size cap are refused (default 150MB) rather than stalling an
+  import.
+
+Configuration is the plugin's customization string, parsed as JSON:
+
+```json
+{"log": true, "log_path": "/path/bindery_repair.log", "max_size_mb": 150,
+ "epubcheck_path": null}
+```
+
+`epubcheck_path` opts into the experimental on-PATH validation mode: the
+repaired copy is re-measured with your epubcheck binary and refused unless it
+is no worse than the original. It defaults to off; the default pass is safe
+without a gate, and the mode costs two epubcheck runs per imported book.
+
 ## Usage
 
 Four verbs: `bindery repair` fixes a single EPUB epubcheck-gated, `bindery audit`
@@ -104,10 +139,15 @@ bindery audit ocr ~/docs/Calibre\ Library
 # Flag books where a single content document is too large for readers to render
 bindery audit monolithic ~/docs/Calibre\ Library
 
+# Spot-check completeness: prose coverage, first/middle/last excerpts, trailing ToC
+bindery audit completeness ~/docs/Calibre\ Library
+
 # Run all audits and generate a comprehensive CSV
 cd ~/docs/Calibre\ Library && bindery audit all
 ```
 
+**Completeness spot-check:** the `completeness` audit answers the pre-import question "does this book reach its end?" It reports, per book: the prose-document count against the spine (docs with 400+ visible characters), the opening and closing text of the first/middle/last prose doc, a trailing-ToC classification of the final spine doc (a link-dense chapter list after the real back matter), and the fraction of unreadable docs. A trailing ToC is reported as an advisory, never a failure: the closing excerpt always comes from the real back matter, so a human can confirm the book ends where the author ended it.
+
 Audits can also be run on a directory of loose `.epub` files by passing the path as the second argument:
 ```sh
 bindery audit pagenumbers /path/to/loose/epubs
@@ -182,7 +222,7 @@ bindery library ~/docs/Calibre\ Library --only all --apply --all --install-to-ca
 - `--json FILE` writes a machine-readable report of the whole run (per-book status, before/after counts, applied flag, summary totals). `--manual-list FILE` writes the paths of every book that was not auto-repaired, one per line, ready for manual follow-up.
 - `--apply` is required to write; the default is a dry run. `--backup DIR` mirrors originals before replacing; `--backup-inplace` writes `.epub.bak` beside each file.
 - `--install-to-calibre` resolves the Calibre book id from `metadata.db` via cquarry (one read-only path→id map per run), so a hand-renamed `Author/Title (id)/` directory can never send the repaired file to the wrong book. The directory-name regex is only a no-catalog fallback; with neither, the file is saved atomically in place.
-- `--install-to-calibre` installs the repaired EPUB through cquarry's write module: the file is placed atomically at the catalogued path (same filename, Calibre's layout never changes) and the format row is re-registered in one transaction, so the stored size stays truthful and the book lands in `metadata_dirtied` — Calibre regenerates its sidecar .opf on next start. No external `calibredb` call is involved. It falls back to atomic file replacement if the Calibre database ID cannot be resolved, and a database failure degrades to the in-place save with a warning. A directory-name-guessed id is trusted only when `metadata.db` corroborates it (the book exists and the file is that book's catalogued EPUB); an uncatalogued stray file inside a book directory is saved in place and the catalog row is left untouched.
+- `--install-to-calibre` installs the repaired EPUB through cquarry's write module: the file is placed atomically at the catalogued path (same filename, Calibre's layout never changes) and the format row is re-registered in one transaction, so the stored size stays truthful and the book lands in `metadata_dirtied`; Calibre regenerates its sidecar .opf on next start. No external `calibredb` call is involved. It falls back to atomic file replacement if the Calibre database ID cannot be resolved, and a database failure degrades to the in-place save with a warning. A directory-name-guessed id is trusted only when `metadata.db` corroborates it (the book exists and the file is that book's catalogued EPUB); an uncatalogued stray file inside a book directory is saved in place and the catalog row is left untouched.
 - `--all` automatically turns on all opt-in non-fatal fixes and lossy strips (pagination, watermarks, bad attributes, unknown entities, image alt tags, etc.) in a single run.
 - Only the `.epub` is replaced. `metadata.opf`, `cover.jpg`, and `metadata.db` are left for Calibre's Quality Check sync to reconcile.
 - A per-book progress line goes to stderr (stdout stays a clean report); `--quiet` suppresses it. A corrupt or unreadable book is reported and skipped, never aborting the sweep.

bindery_cli-0.37.0/REPORT-12-Sept.md

@@ -0,0 +1,119 @@
+# REPORT-12-Sept: the repair-landscape research (bindery-cli vs real-world EPUB damage)
+
+Research date: 2026-09-10/11. Method: a read-only research pass mapping
+the full taxonomy of real-world EPUB damage — by producer (Calibre
+conversions, Word/Office export, InDesign, legacy OCR, Adobe Digital
+Editions, KindleUnpack, self-pub aggregators, Windows case-manglers) —
+against this repo's shipped repair surface, the Phase 14/15 backlogs,
+and the Bug Reports. References include the upstream clone's format
+readers. Full evidence lives in the agent's findings; this report is
+the distilled verdict and every actionable gap with its routing.
+
+## Verdict
+
+Bindery handles roughly **60-65% of real-world damage classes
+end-to-end** and is near-complete for its founding niche: markup
+well-formedness (the conversion-fatals class), NCX basics,
+zip/mimetype container hygiene, dead-reference pruning, watermark and
+page-number furniture, and archive/spine diagnosis (corruption, DRM
+skip, ToC-bloat classification). Phase 14's hardening and all five Bug
+Reports are shipped through v0.35.0; Phase 15 (the completeness
+analyzer) is approved.
+
+The missing ~35% is not more markup surgery. It is **package-level
+repairs** — container.xml, manifest media-types, spine attribute
+cleanup, encryption.xml/obfuscation state, zip-level dedupe, href
+case/IRI mismatches — and the finding that matters most: several of
+those classes are **gateway defects**. Epubcheck stays fatal while
+they persist, so no shipped repair can ever be gate-accepted on those
+books. They are locked out of the pipeline entirely.
+
+## A. The damage-taxonomy matrix and the repair gaps
+
+1. **Missing/corrupt `META-INF/container.xml`** (any producer):
+   epubcheck fatal; the OPF is unfindable. The locator falls back to
+   the first `.opf` but never writes the file. Deterministic fix:
+   generate the standard container.xml at the located OPF (upstream
+   calibre's `initialize_container` is the model). Effort S; HIGH
+   value — a gateway repair that unlocks an entire class of books.
+2. **Stale font-obfuscation state** (publisher EPUB3 fonts; IDPF
+   `2008/embedding` and Adobe `pdf/enc#RC`): post-KindleUnpack/strip
+   tools decrypt fonts in place but leave `encryption.xml` declaring
+   obfuscation; epubcheck errors and the audit implies DRM where there
+   is none. Fix: detect via font magic bytes at offset 0 vs the
+   CipherReference; prune stale entries; drop encryption.xml when only
+   obfuscation entries remain and the fonts verify. Never
+   un-obfuscate. Also gives the audit a real OBFUSCATED-vs-DRM
+   distinction. Effort M; needs the magic-byte detection validated on
+   real samples first.
+3. **Wrong/missing manifest `media-type`** (aggregators, ADE): fonts,
+   images, and media declared with wrong types. Fix: sniff magic bytes
+   or extension map; attribute-only. Effort S.
+4. **href case/backslash/IRI mismatches** (KindleUnpack, Windows
+   producers): `Images/Cover.JPG` vs `images/cover.jpg`, backslash
+   separators, `%XX` variants — the file IS present, so
+   `--prune-missing-resources` cannot touch the RSC-007/PKG-010 it
+   causes. Fix: case-insensitive namemap when exactly one match;
+   backslash to slash. Effort M.
+5. **Dangling reference edges after pruning** (any producer):
+   `spine@toc`, `item@media-overlay`, `item@fallback`, and EPUB2 cover
+   meta pointing at items the prune pass deleted. Today's prune checks
+   only `spine_ids` — it can manufacture the regression the gate then
+   rejects. Fix: extend prune to rewrite those edges (the rewrite
+   machinery exists in fix_manifest_ids). Effort S/M.
+6. **Duplicate zip entries** (any producer): last-wins shadowing; the
+   audit counts `dup_entries`, the rewrite copies both. Fix: drop
+   shadowed duplicates on rewrite. Effort S.
+7. **Cover wiring** (EPUB2 producers): dangling `<meta name="cover">`,
+   missing EPUB3 `properties="cover-image"`, wrong cover media-type —
+   calibre and booksellers extract no cover. Narrow, finding-driven
+   repair when the cover is identifiable from the existing guide/meta;
+   audit-only otherwise. Effort M.
+8. **Broken CSS from converters**: `url()` to absent assets, malformed
+   selectors. Repair is fiddly; ship the audit-only detector first and
+   let prevalence data decide. Effort M (repair).
+9. **Human-facing metadata validity** (self-pub/Word): non-ISO
+   `dc:date`, invalid `dc:language`, missing EPUB3
+   `dcterms:modified`. Explicit non-goal today; `dcterms:modified`
+   also collides with byte-determinism. Needs a written charter
+   ruling: carve-out with a determinism-safe timestamp policy, or
+   route to cquarry/CalibreQuarry. Effort M + a decision.
+10. **EPUB3 dual-ToC drift** (nav vs NCX): detection is a
+    deterministic structural diff feeding `decisions_needed`; full
+    sync is ToC synthesis and stays out of charter permanently.
+    Effort L if repair is ever attempted; audit-only recommended.
+11. **No-repair classes, correctly routed**: OCR garbage, truncated
+    re-sources, absent font families (audit + re-source); real DRM
+    (clean ENCRYPTED skip — shipped); DB-level format rows
+    (cquarry's, done); catalog completeness and human-facing metadata
+    quality (CalibreQuarry's audits).
+
+## B. Research-before-boxing
+
+1. **FastSweep prevalence study** — run `fast_sweep.py --mode=extract`
+   aggregation over the full library targeting the new classes
+   (container.xml presence/validity, encryption.xml algorithms,
+   media-type warnings, case-mismatch RSC-007s). The harness exists;
+   it sizes every box above with real counts before any is built.
+2. **The metadata carve-out ruling** (gap 9) — Brandon decides;
+   the audit records both options.
+3. **The obfuscation magic-byte table** (gap 2) — validate IDPF and
+   Adobe font prefixes against real samples before the repair ships.
+
+## C. Routing notes
+
+Package-structure repairs (items 1-6) fit the existing "structural
+opt-ins + gate" model without reopening the semantics-preserving
+contract. Items 7-10 each need an explicit scope decision first.
+Cross-repo: DB-level size/format-row mismatches route to cquarry
+(set_format, shipped); catalog completeness, duplicate detection, and
+human-facing metadata quality route to CalibreQuarry's audits; OCR,
+truncation, monolithic, and missing-font content stay audit-and-
+re-source permanently.
+
+## D. Phase 16 routing
+
+Items 1-6 (the ungated package-structure repairs) are boxed in
+roadmap.md **Phase 16** with the FastSweep prevalence study leading.
+Items 7-10 are recorded as decision-gated boxes. Phase 15 (approved)
+runs first if its fixture work is not yet done.

{bindery_cli-0.35.0 → bindery_cli-0.37.0}/patchnotes.md

@@ -1,4 +1,102 @@
 # bindery-cli Patch Notes
+## v0.37.0 (2026-09-12)
+
+### The Bindery Repair Calibre plugin (Phase 3)
+
+- **Books are repaired as they are imported.** Each release now attaches
+  BinderyRepair-v<VERSION>.zip to the GitHub release: a Calibre
+  FileTypePlugin (on_import) that runs the CLI's always-on core pass, the
+  five well-formedness fixes plus the NCX pipeline, on *.epub files as they
+  are added. run() never raises and never touches the original file or
+  metadata.db: a repaired copy is built in a persistent temp file,
+  testzip()-verified, and handed back for import; any trouble returns the
+  original path and logs one line. The plugin is byte-idempotent, so
+  re-adding a format re-runs it harmlessly.
+- **The vendor slice is byte-identical to the repo.** The zip carries
+  transforms.py, epub.py, pagination.py, watermark.py, and reserialize.py
+  unchanged (the zip root is a package, so their relative imports resolve);
+  a suite drift test pins the equality, and publish.yml regenerates the zip
+  from the tag and attaches it to the release.
+- **Nothing runs ungated, ever.** The structural repairs and the lossy
+  strips stay CLI-only because their acceptance is the epubcheck gate,
+  which cannot run inside Calibre. Identity: Bindery Repair /
+  bindery_repair. Config via the plugin's customization string as JSON:
+  log, log_path (default <config dir>/bindery_repair.log), max_size_mb
+  (recorded default 150MB), and epubcheck_path, the experimental on-PATH
+  validation mode, default OFF (the three open questions, answered
+  2026-09-11).
+
+### Fixed: fix_ncx_playorder counted phantom fixes (found by the plugin)
+
+- The already-correct comparison read the attribute value with its
+  quotes, so every sequential navPoint counted as a fix on every pass
+  while the rewritten bytes stayed identical. The Long Patrol reported
+  fix_ncx_playorder=59 on every run of a repaired book; a book whose
+  only fix was phantom was rewritten instead of reported as nochange;
+  and a single-quoted sequential attribute was rewritten to double
+  quotes, touching already-correct markup. Quotes are now stripped
+  before the comparison and an already-correct attribute is returned
+  untouched: honest counts, single quotes survive, and the plugin's
+  zero-fixes contract holds end to end.
+
+### Repo records
+
+- Phase 16's FastSweep prevalence study (5,228 books) and the obfuscation
+  magic-byte validation are recorded in the roadmap with real counts for
+  every Phase 16 repair box.
+
+## v0.36.0 (2026-09-12)
+
+### The completeness spot-check: `audit completeness` (Phase 15)
+
+- **The phase-1 judgment step finally has a tool owner.** The import
+  skill's completeness step asks a human question with no tool behind
+  it: "sample early, middle, and late pages ... read the LAST content
+  page to confirm it reaches real back matter rather than cutting off
+  mid-chapter". Two consecutive phase-1 runs answered it with a
+  hand-rolled zipfile sampler in /tmp. The new analyzer rides the
+  audit's existing single decompression pass (no second decode of any
+  book) and reports, per book: spine doc count, prose-doc count (spine
+  docs with at least 400 visible characters), the opening and closing
+  120 characters of the first/middle/last prose doc, a trailing-ToC
+  classification of the final spine doc, and the fraction of unreadable
+  docs (corrupt entries plus unresolved itemrefs).
+- **A trailing ToC never impersonates the ending.** The final spine doc
+  is classified as a chapter list when short link lines dominate it and
+  no block runs to paragraph length; such a doc is book furniture and
+  is excluded from the prose sampling, so the closing excerpt comes
+  from the real back matter. Verified against the real 2026-09-10
+  Redwall fixtures: Lord Brocktree's trailing ToC is caught and the
+  Epilogue is sampled as the last prose doc; The Long Patrol reads
+  exactly 55/57 prose docs through the percent-encoded hrefs that
+  defeated the hand-rolled sampler; Mattimeo's split doc (chapter
+  headings with real prose after every one) stays prose.
+- **Advisory by contract.** The analyzer never flags a book and never
+  moves the exit code: the archive, spine, and emptytext verdicts keep
+  owning the flags. The verdict status is ADVISORY (not OK) when a
+  trailing ToC is present or the unreadable fraction reaches 10%.
+  `audit all` and `run phase1` include it through the battery, the CLI
+  grows the `completeness` mode, and `--json` records carry its
+  verdicts in the same shape as the other analyzers.
+
+### Dependencies
+
+- Adopted cquarry 1.18.0 (floor + lock): the set_format repair lane now keeps
+  Calibre's FTS index and page counts honest automatically (the sidecar's
+  dirty queue and the pages rescan flag ride the write), and the search and
+  runtime fixes come along in the shared layer.
+- Adopted cquarry 1.19.0 (floor + lock): the repair lane gains its undo
+  primitive -- save_original_format/restore_original_format -- so a format
+  repair can put the original bytes back if it goes wrong.
+
+### Repo records
+
+- The release-tag backlog is settled forward-only from v0.35.0: the
+  nine untagged 0.18.0-0.26.0 entries are exempted by decision of
+  record (2026-09-11), and the v0.28.0 retag and the v0.33.0 em-dash
+  tag repair stay recorded-only force-pushes (roadmap.md, "Tag
+  policy").
+
 ## v0.35.0 (2026-09-10)
 
 ### The epubcheck daemon, reconciled: oracle-identical counts at ~15x the speed