bindery-cli 0.36.0__tar.gz → 0.38.0__tar.gz

{bindery_cli-0.36.0 → bindery_cli-0.38.0}/.github/workflows/publish.yml

@@ -43,3 +43,28 @@ jobs:
 
       - name: Publish to PyPI
         uses: pypa/gh-action-pypi-publish@release/v1
+
+  plugin-zip:
+    name: Build the Bindery Repair Calibre plugin zip
+    needs: test
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write # create the release and attach the plugin zip
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.14"
+
+      - name: Run the suite
+        run: |
+          pip install .[reserialize]
+          PYTHONPATH=src python -m unittest discover -s tests
+
+      - name: Build the plugin zip from the tagged tree
+        run: python scripts/build_plugin.py
+
+      - name: Attach BinderyRepair zip to the release
+        uses: softprops/action-gh-release@v2
+        with:
+          files: dist/BinderyRepair-*.zip

{bindery_cli-0.36.0 → bindery_cli-0.38.0}/CLAUDE.md

@@ -29,9 +29,9 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
   * **Structural repairs** (`--fix-empty-body`, `--fix-missing-title`, `--fix-id-colons`,
     `--fix-page-map`, `--strip-epub3-attrs`, `--downgrade-epub3-tags`,
     `--unwrap-block-in-inline`, `--strip-invalid-value`, `--unwrap-illegal-tags`,
-    `--prune-missing-resources`, `--strip-broken-anchors`, `--encode-url-spaces`;
-    transforms.py, threaded through epub.py): they alter markup structure or fabricate
-    minimal content. v0.14–v0.16 ran these unconditionally, which broke this rule;
+    `--prune-missing-resources`, `--strip-broken-anchors`, `--encode-url-spaces`,
+    `--fix-container`, `--fix-media-types`; transforms.py, threaded through epub.py):
+    they alter markup structure or fabricate minimal content. v0.14–v0.16 ran these unconditionally, which broke this rule;
     v0.17.0 restored it. `--unwrap-illegal-tags` additionally protects any illegal-tag
     name that an EPUB stylesheet styles as an element selector (`css_protected_tags`,
     book-wide, inline `<style>` blocks included). The two EPUB2-targeted fixes
@@ -56,7 +56,9 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
   the safe opt-ins (`--fix-ids`, `--add-img-alt`, `--strip-bad-attrs`,
   `--escape-unknown-entities`) plus `--encode-url-spaces`, which repair without
   altering visible markup; the "twelve structural repairs and three lossy strips"
-  phrasing elsewhere in these docs was never the complete inventory.
+  phrasing elsewhere in these docs was never the complete inventory, and the
+  Phase 16 package repairs (`--fix-container`, `--fix-media-types`) took the
+  structural count to fourteen (v0.38.0).
 - **The gate is the safety contract.** Never apply a repair epubcheck has not accepted.
   Respect the two-mode logic in `validate.gate` (fatal-fixing tolerates error unmasking;
   error-cleanup does not). The lossy strips (`--strip-pagination`, `--strip-broken-tags`,
@@ -73,16 +75,17 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
 
 ## Layout
 
-- `src/bindery/transforms.py`: pure `str -> (str, int)` text transforms (including `strip_broken_tags`). Since v0.32.0: `strip_attrs_in_start_tags` anchors attribute edits on the quote-aware start-tag matcher, `_outside_protected` forwards arguments (context-carrying transforms can be decorated), `fix_id_colons` matches only the bare `id` attribute and only internal fragments (`fix_ncx_src_fragments` carries renames into the NCX), `fix_ncx_playorder` is anchored to `<navPoint>` start tags, and the css selector boundary covers namespaced and functional selector forms.
+- `src/bindery/transforms.py`: pure `str -> (str, int)` text transforms (including `strip_broken_tags`). Since v0.32.0: `strip_attrs_in_start_tags` anchors attribute edits on the quote-aware start-tag matcher, `_outside_protected` forwards arguments (context-carrying transforms can be decorated), `fix_id_colons` matches only the bare `id` attribute and only internal fragments (`fix_ncx_src_fragments` carries renames into the NCX), `fix_ncx_playorder` is anchored to `<navPoint>` start tags, and the css selector boundary covers namespaced and functional selector forms. Since v0.37.0: `fix_ncx_playorder` strips quotes before its already-correct comparison (regex groups carry quote characters; comparing quoted values against bare numbers counted phantom fixes on every sequential NCX and rewrote single-quoted correct attributes to double quotes).
 - `src/bindery/pagination.py`: the opt-in lossy page-number strip (runhead detection, page-layer decision, block-centric removal/merge, safety nets).
 - `src/bindery/watermark.py`: the opt-in lossy watermark strip (anchored and anchorless signature removal).
 - `src/bindery/reserialize.py`: structural repair via `html5lib`.
-- `src/bindery/audit.py`: read-only body-text audits (`content`, `pagenumbers`, `emptytext`, `ocr`, `monolithic` since v0.21.0 with `--max-doc-chars N`, and `completeness` since v0.36.0) behind the `audit` subcommand (v0.15.0). Since v0.18.0 library mode resolves EPUB paths through `cquarry.get_format_path()` and can apply a tag to flagged books via `cquarry.write.WritableCalibreDB` (only with explicit `--tag`; the only sanctioned write path). Since v0.19.0 `audit --id BOOK_ID` audits a single library book through cquarry's single-entity `get_book()` fetch (no library-wide cache; supports `--tag`; incompatible with the directory argument; comma lists since v0.23.0). Since v0.22.0 every archive entry is fully read (CRC + decompression) before analysis — a damaged entry reports its own CORRUPT verdict instead of feeding emptytext — and manifest/NCX references to absent files classify as `convention` (bloated ToC, consecutive span) or `fragment` (broken span). Hazard: the scan loop reuses `tag` as its per-book display column, so the `--tag` argument is captured as `audit_tag` at function entry — do not collapse them again. Since v0.29.0 `audit --json FILE` writes per-file analyzer verdicts in the `library --json` shape (directory, library, and single-book modes; with `--id`, exactly one id; emptytext omitted from a record when the archive verdict owns the body-text story). Since v0.36.0 the completeness analyzer (Phase 15) reports the phase-1 spot-check per book: spine/prose-doc counts (prose = >= 400 visible chars), first/middle/last prose-doc opening/closing excerpts, trailing-ToC classification (`_trailing_toc`: short link lines dominate, no paragraph-length blocks), and the unreadable fraction. It is ADVISORY by contract (problem always False; ADVISORY on a trailing ToC or unreadable fraction >= 10%), never moves the exit code, and is skipped like emptytext when the archive verdict owns the book.
-- `src/bindery/epub.py`: archive rewrite, NCX uid sync, RepairReport, mismatch detection, and the opt-in structural-repair plumbing (including the CSS precondition scan).
+- `src/bindery/audit.py`: read-only body-text audits (`content`, `pagenumbers`, `emptytext`, `ocr`, `monolithic` since v0.21.0 with `--max-doc-chars N`, and `completeness` since v0.36.0) behind the `audit` subcommand (v0.15.0). Since v0.18.0 library mode resolves EPUB paths through `cquarry.get_format_path()` and can apply a tag to flagged books via `cquarry.write.WritableCalibreDB` (only with explicit `--tag`; the only sanctioned write path). Since v0.19.0 `audit --id BOOK_ID` audits a single library book through cquarry's single-entity `get_book()` fetch (no library-wide cache; supports `--tag`; incompatible with the directory argument; comma lists since v0.23.0). Since v0.22.0 every archive entry is fully read (CRC + decompression) before analysis — a damaged entry reports its own CORRUPT verdict instead of feeding emptytext — and manifest/NCX references to absent files classify as `convention` (bloated ToC, consecutive span) or `fragment` (broken span). Hazard: the scan loop reuses `tag` as its per-book display column, so the `--tag` argument is captured as `audit_tag` at function entry — do not collapse them again. Since v0.29.0 `audit --json FILE` writes per-file analyzer verdicts in the `library --json` shape (directory, library, and single-book modes; with `--id`, exactly one id; emptytext omitted from a record when the archive verdict owns the body-text story). Since v0.36.0 the completeness analyzer (Phase 15) reports the phase-1 spot-check per book: spine/prose-doc counts (prose = >= 400 visible chars), first/middle/last prose-doc opening/closing excerpts, trailing-ToC classification (`_trailing_toc`: short link lines dominate, no paragraph-length blocks), and the unreadable fraction. It is ADVISORY by contract (problem always False; ADVISORY on a trailing ToC or unreadable fraction >= 10%), never moves the exit code, and is skipped like emptytext when the archive verdict owns the book. Since v0.38.0 the archive verdict distinguishes font obfuscation from DRM (`OBFUSCATION_ALGOS`: IDPF 2008/embedding + both Adobe URIs; readable obfuscated entries are the benign OBFUSCATED advisory, an unreadable one is CORRUPT, and only non-obfuscation algorithms give ENCRYPTED).
+- `src/bindery/epub.py`: archive rewrite, NCX uid sync, RepairReport, mismatch detection, and the opt-in structural-repair plumbing (including the CSS precondition scan). Since v0.38.0 (Phase 16): `generate_container` + the `--fix-container` gateway (insert after mimetype when missing, replace in-loop when stale; constant-epoch timestamp keeps bytes deterministic), `prune_dangling_edges` (prune_missing_manifest_items returns the pruned ids and the edges that pointed at them — spine@toc, media-overlay, fallback, EPUB2 cover meta — are rewritten so pruning cannot manufacture the regression the gate rejects), and `fix_manifest_media_types` (extension map + magic-byte confirmation via a `peek` callable over the open source zip; jpg/png/gif only).
 - `src/bindery/validate.py`: epubcheck wrapper, the `gate` (improvement) and `no_worse` (no-regression, for the lossy strips) acceptance bars.
 - `src/bindery/library.py`: Calibre walk, atomic replace, backups, and native format installation. Since v0.19.0 `CalibreIdResolver` resolves the book id from `metadata.db` through cquarry (one lazy path→id map per run; since v0.20.0 the map comes from `CalibreDB.format_path_index()`, re-normalized with `resolve().lower()` for the resolver's symlinked-directory and case-insensitive matching); the `(id)` directory-name regex is only the no-catalog fallback. Since v0.24.0 `install_format()` places the repaired file atomically and updates the `data` row through `cquarry.write.WritableCalibreDB` (since v0.31.0 via `set_format`, cquarry 1.17's sanctioned remove+add in one transaction); the external `calibredb` CLI is gone. Since v0.32.0 a guessed id drives a row update only when metadata.db corroborates it (`_verify_guess`: books row exists, file in the book's own directory, catalogued `data.name` when a row exists); anything else saves in place and leaves the catalog untouched — never reintroduce directory-name guessing as the primary source: renamed/mismatched directories would replace the wrong book.
 - `src/bindery/cli.py`: `repair` and `library` subcommands, including `--all` and `--install-to-calibre`; since v0.29.0 also the `run` verbs: `phase1` (pre-import vetting over loose files: the audit battery composed with one gated repair sweep; read-only until `--apply-lossy`, which IS the lossy-strip consent) and `phase3` (the scoped post-import apply step: `library --id --sweep --only all --apply --all --install-to-calibre` plus a pre/post summary; mechanically refuses unscoped sweeps with exit 2). Both take `--non-interactive` and surface open questions as `decisions_needed` in their JSON, never prompts. They drive the shipped `run_library` through the real parser, so no flag can drift between the verb and the subcommand it wraps. `library --workers N` (v0.30.0) parallelizes only the sweep's candidate pass, in windows of N consumed in order so `--limit` stays lazy; the repair phase is never parallel (shared workdir, atomic-replacement contract).
-- `tests/`: transforms, end-to-end repair, atomic replace, pagination, watermarks, the audit analyzers (content/pagenumbers/emptytext/ocr/monolithic/completeness, archive integrity, spine classification), the library sweep and `CalibreIdResolver`, validate, reserialize, CLI wiring, and the version pin.
+- `plugin/__init__.py` + `scripts/build_plugin.py`: the Bindery Repair Calibre plugin (v0.37.0; identity `Bindery Repair` / `bindery_repair`, approved and built per the scoped spec in roadmap.md Phase 3). The builder vendors `transforms.py`, `epub.py`, `pagination.py`, `watermark.py`, `reserialize.py` byte-identically into the zip (the zip root is a package, so their relative imports resolve unchanged; the suite's drift test pins the equality) with the version tuple substituted from the single-source VERSION; `publish.yml` attaches `BinderyRepair-v<VERSION>.zip` to each release. The plugin runs ONLY the default pass (epubcheck cannot gate inside Calibre), never raises, is byte-idempotent (zero fixes returns the original path), testzip()-verifies its output, logs one line per book, and takes JSON config via `site_customization` (`log`, `log_path`, `max_size_mb` default 150, `epubcheck_path` experimental validation default OFF). Load it in tests through the `tests/test_plugin.py` stubs: the calibre namespace is installed in sys.modules and the package loads with a two-dir `__path__` (plugin dir first, then src/bindery) exactly like calibre's zipplugin loader shape.
+- `tests/`: transforms, end-to-end repair, atomic replace, pagination, watermarks, the audit analyzers (content/pagenumbers/emptytext/ocr/monolithic/completeness, archive integrity, spine classification), the library sweep and `CalibreIdResolver`, validate, reserialize, CLI wiring, the version pin, and the Calibre plugin (build drift + run behavior under calibre stubs).
 
 ## Conventions
 

{bindery_cli-0.36.0 → bindery_cli-0.38.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: bindery-cli
-Version: 0.36.0
+Version: 0.38.0
 Summary: Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library.
 Author: vrnvctss
 License: MIT
@@ -60,9 +60,11 @@ Everything below is off until its flag is passed (or all at once via `--all`). T
 - **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or the wider aria family survive). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
 - **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents (`figure`/`section` to `div`, `figcaption` to `p`), keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
 - **`--unwrap-illegal-tags`**: strips completely invalid or deprecated HTML tags that break EPUB3 validation (`<st>`, `<sentence>`, `<o>`, `<w>`, `<pagebreak>`) while retaining their inner text. Any of those names styled as an *element selector* by an EPUB stylesheet (`.css` entries and inline `<style>` blocks alike; class/id selectors like `.st`/`#w` don't count) is protected for the whole book, guaranteeing format preservation.
-- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items. Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
+- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items (with every package edge that pointed at them rewritten: `spine@toc`, `item@media-overlay`, `item@fallback`, and the EPUB2 cover meta). Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
 - **`--strip-broken-anchors`**: strips href attributes that cannot resolve, keeping the anchor text byte-for-byte: a `#fragment` the target document does not define (RSC-020/RSC-012; NCX navTargets fall back to the document target, so chapter navigation survives) and unresolvable URI schemes (`kindle:`, `file:`).
 - **`--encode-url-spaces`**: percent-encodes raw spaces in `src`/`href` attribute values across the package (OPF manifest, NCX, content documents). A URL with a literal space is not a valid URL (RSC-020) and unresolvable on strict readers; the encoded form denotes the same file.
+- **`--fix-container`**: generates `META-INF/container.xml` at the located OPF when the container is missing or names a file the archive does not contain. This is the gateway defect: epubcheck stays fatal while the OPF is unfindable, so no repair can be gate-accepted on such a book until the container exists.
+- **`--fix-media-types`**: normalizes wrong manifest media-type declarations (OPF-029, e.g. a jpg stamped `image/png` by an aggregator). Attribute-only and quote-preserving; it fires only when the file's magic bytes confirm the extension, so an honestly-declared misnamed file is never made worse.
 
 
 Three opt-in fixes are **lossy** and stand apart from the semantics-preserving rest:
@@ -95,6 +97,41 @@ uv tool install bindery-cli                            # core tools
 uv tool install "bindery-cli[reserialize]"             # incl. --reserialize
 ```
 
+## The Calibre plugin (Bindery Repair)
+
+Each release also ships **BinderyRepair-v<VERSION>.zip**, a Calibre plugin that
+repairs EPUBs as they are imported: it is attached to the GitHub release beside
+the PyPI wheel (install it via Calibre's Preferences, Plugins, "Load plugin
+from file"). The plugin is the CLI's always-on core pass only: the five
+well-formedness fixes plus the NCX pipeline. The structural repairs and lossy
+strips stay CLI-only, because their acceptance is the epubcheck gate and
+epubcheck cannot run inside Calibre; nothing runs ungated in the plugin, ever.
+
+Behavior and guarantees:
+
+- `run()` never raises and never touches the original file or `metadata.db`: a
+  repaired copy is built in a persistent temp file and handed back for import;
+  any trouble returns the original path and logs one line.
+- Byte-idempotent: an already-clean book yields zero fixes and the original
+  path, so re-adding a format re-runs the plugin harmlessly.
+- The rewritten archive is `testzip()`-verified before it is handed back.
+- One log line per book (fixed / no fixes / refused / errored), appended to
+  `bindery_repair.log` in Calibre's config directory by default.
+- Files over the size cap are refused (default 150MB) rather than stalling an
+  import.
+
+Configuration is the plugin's customization string, parsed as JSON:
+
+```json
+{"log": true, "log_path": "/path/bindery_repair.log", "max_size_mb": 150,
+ "epubcheck_path": null}
+```
+
+`epubcheck_path` opts into the experimental on-PATH validation mode: the
+repaired copy is re-measured with your epubcheck binary and refused unless it
+is no worse than the original. It defaults to off; the default pass is safe
+without a gate, and the mode costs two epubcheck runs per imported book.
+
 ## Usage
 
 Four verbs: `bindery repair` fixes a single EPUB epubcheck-gated, `bindery audit`

{bindery_cli-0.36.0 → bindery_cli-0.38.0}/README.md

@@ -38,9 +38,11 @@ Everything below is off until its flag is passed (or all at once via `--all`). T
 - **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or the wider aria family survive). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
 - **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents (`figure`/`section` to `div`, `figcaption` to `p`), keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
 - **`--unwrap-illegal-tags`**: strips completely invalid or deprecated HTML tags that break EPUB3 validation (`<st>`, `<sentence>`, `<o>`, `<w>`, `<pagebreak>`) while retaining their inner text. Any of those names styled as an *element selector* by an EPUB stylesheet (`.css` entries and inline `<style>` blocks alike; class/id selectors like `.st`/`#w` don't count) is protected for the whole book, guaranteeing format preservation.
-- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items. Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
+- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items (with every package edge that pointed at them rewritten: `spine@toc`, `item@media-overlay`, `item@fallback`, and the EPUB2 cover meta). Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
 - **`--strip-broken-anchors`**: strips href attributes that cannot resolve, keeping the anchor text byte-for-byte: a `#fragment` the target document does not define (RSC-020/RSC-012; NCX navTargets fall back to the document target, so chapter navigation survives) and unresolvable URI schemes (`kindle:`, `file:`).
 - **`--encode-url-spaces`**: percent-encodes raw spaces in `src`/`href` attribute values across the package (OPF manifest, NCX, content documents). A URL with a literal space is not a valid URL (RSC-020) and unresolvable on strict readers; the encoded form denotes the same file.
+- **`--fix-container`**: generates `META-INF/container.xml` at the located OPF when the container is missing or names a file the archive does not contain. This is the gateway defect: epubcheck stays fatal while the OPF is unfindable, so no repair can be gate-accepted on such a book until the container exists.
+- **`--fix-media-types`**: normalizes wrong manifest media-type declarations (OPF-029, e.g. a jpg stamped `image/png` by an aggregator). Attribute-only and quote-preserving; it fires only when the file's magic bytes confirm the extension, so an honestly-declared misnamed file is never made worse.
 
 
 Three opt-in fixes are **lossy** and stand apart from the semantics-preserving rest:
@@ -73,6 +75,41 @@ uv tool install bindery-cli                            # core tools
 uv tool install "bindery-cli[reserialize]"             # incl. --reserialize
 ```
 
+## The Calibre plugin (Bindery Repair)
+
+Each release also ships **BinderyRepair-v<VERSION>.zip**, a Calibre plugin that
+repairs EPUBs as they are imported: it is attached to the GitHub release beside
+the PyPI wheel (install it via Calibre's Preferences, Plugins, "Load plugin
+from file"). The plugin is the CLI's always-on core pass only: the five
+well-formedness fixes plus the NCX pipeline. The structural repairs and lossy
+strips stay CLI-only, because their acceptance is the epubcheck gate and
+epubcheck cannot run inside Calibre; nothing runs ungated in the plugin, ever.
+
+Behavior and guarantees:
+
+- `run()` never raises and never touches the original file or `metadata.db`: a
+  repaired copy is built in a persistent temp file and handed back for import;
+  any trouble returns the original path and logs one line.
+- Byte-idempotent: an already-clean book yields zero fixes and the original
+  path, so re-adding a format re-runs the plugin harmlessly.
+- The rewritten archive is `testzip()`-verified before it is handed back.
+- One log line per book (fixed / no fixes / refused / errored), appended to
+  `bindery_repair.log` in Calibre's config directory by default.
+- Files over the size cap are refused (default 150MB) rather than stalling an
+  import.
+
+Configuration is the plugin's customization string, parsed as JSON:
+
+```json
+{"log": true, "log_path": "/path/bindery_repair.log", "max_size_mb": 150,
+ "epubcheck_path": null}
+```
+
+`epubcheck_path` opts into the experimental on-PATH validation mode: the
+repaired copy is re-measured with your epubcheck binary and refused unless it
+is no worse than the original. It defaults to off; the default pass is safe
+without a gate, and the mode costs two epubcheck runs per imported book.
+
 ## Usage
 
 Four verbs: `bindery repair` fixes a single EPUB epubcheck-gated, `bindery audit`

{bindery_cli-0.36.0 → bindery_cli-0.38.0}/patchnotes.md

@@ -1,4 +1,107 @@
 # bindery-cli Patch Notes
+## v0.38.0 (2026-09-12)
+
+### Phase 16: package-structure repairs
+
+- **`--fix-container`, the gateway repair.** A book whose
+  META-INF/container.xml is missing, unparseable, or names a file the
+  archive does not contain is locked out of every repair: epubcheck
+  stays fatal while the OPF is unfindable. The new flag generates the
+  standard container at the OPF the locator already finds by fallback
+  (a byte-deterministic template with the constant epoch timestamp).
+  Oracle-verified on a real-shaped fixture: fatal before, gate-accepted
+  after. The Phase 16 prevalence study found 0 such books in-library
+  (5,228/5,228 have valid containers); the repair ships for the
+  acquisition phase, where the class actually lives.
+- **`--prune-missing-resources` rewrites what it drops.** Removing a
+  manifest item whose file is absent used to leave every package edge
+  pointing at it dangling, manufacturing the regression the gate then
+  rejects. The prune now carries the pruned ids into a rewrite of the
+  four edge classes: `spine@toc`, `item@media-overlay`, `item@fallback`,
+  and the EPUB2 cover meta. Optional edges of dead targets are removed;
+  edges to live ids are untouched.
+- **`--fix-media-types`.** Wrong manifest media-type declarations
+  (OPF-029; 275 occurrences in 15 library books, the class being a jpg
+  stamped `image/png` by an aggregator) are normalized attribute-only,
+  preserving quote style, and only when the file's magic bytes confirm
+  the extension: a PNG renamed .jpg keeps its wrong-but-honest
+  declaration instead of a worse one. On the real Arthur sample the run
+  goes from nochange to gate-accepted with 9 declarations fixed.
+- **The audit distinguishes font obfuscation from DRM.** encryption.xml
+  entries under the obfuscation algorithms (the IDPF URI and both Adobe
+  forms, including the ns.adobe.com URI real-world files actually
+  carry, which the prevalence study caught) that read fine are the new
+  OBFUSCATED advisory: publisher font embedding, benign, problem-False,
+  exit-neutral. An unreadable obfuscation entry is CORRUPT (a broken
+  font, re-source), and only non-obfuscation algorithms give the
+  ENCRYPTED skip. 86 of the library's 5,228 books carry
+  obfuscation-only encryption.xml; the library holds no real DRM at all.
+- **Boxed, not built, with real counts.** The obfuscation-aware
+  encryption repair (the magic-byte precondition validated on 424 real
+  fonts: all genuinely scrambled, zero stale entries in-library), href
+  case/backslash resolution (1 href in 1 book; 0 backslashes), and
+  duplicate zip-entry dedupe (0 books) are recorded in the roadmap with
+  their prevalence numbers and reopen conditions. The decision-gated
+  Phase 16 C boxes (cover wiring, the metadata carve-out, the NCX<->nav
+  drift detector, CSS url() pruning) carry recorded options and wait
+  for rulings.
+
+### The prevalence study (Phase 16 A)
+
+- 5,228 library books through the FastSweep epubcheck harness plus a
+  direct zip-level pass: container.xml valid on every book; RSC-007 at
+  14,156 occurrences in 600 books and PKG-010 in 280 (the prune
+  surface, 1,538 truly-absent manifest hrefs); OPF-029 at 275 in 15;
+  case-mismatch hrefs exactly 1; encryption.xml in 86 books, all of it
+  font obfuscation. Every Phase 16 repair box now carries its real
+  count.
+
+## v0.37.0 (2026-09-12)
+
+### The Bindery Repair Calibre plugin (Phase 3)
+
+- **Books are repaired as they are imported.** Each release now attaches
+  BinderyRepair-v<VERSION>.zip to the GitHub release: a Calibre
+  FileTypePlugin (on_import) that runs the CLI's always-on core pass, the
+  five well-formedness fixes plus the NCX pipeline, on *.epub files as they
+  are added. run() never raises and never touches the original file or
+  metadata.db: a repaired copy is built in a persistent temp file,
+  testzip()-verified, and handed back for import; any trouble returns the
+  original path and logs one line. The plugin is byte-idempotent, so
+  re-adding a format re-runs it harmlessly.
+- **The vendor slice is byte-identical to the repo.** The zip carries
+  transforms.py, epub.py, pagination.py, watermark.py, and reserialize.py
+  unchanged (the zip root is a package, so their relative imports resolve);
+  a suite drift test pins the equality, and publish.yml regenerates the zip
+  from the tag and attaches it to the release.
+- **Nothing runs ungated, ever.** The structural repairs and the lossy
+  strips stay CLI-only because their acceptance is the epubcheck gate,
+  which cannot run inside Calibre. Identity: Bindery Repair /
+  bindery_repair. Config via the plugin's customization string as JSON:
+  log, log_path (default <config dir>/bindery_repair.log), max_size_mb
+  (recorded default 150MB), and epubcheck_path, the experimental on-PATH
+  validation mode, default OFF (the three open questions, answered
+  2026-09-11).
+
+### Fixed: fix_ncx_playorder counted phantom fixes (found by the plugin)
+
+- The already-correct comparison read the attribute value with its
+  quotes, so every sequential navPoint counted as a fix on every pass
+  while the rewritten bytes stayed identical. The Long Patrol reported
+  fix_ncx_playorder=59 on every run of a repaired book; a book whose
+  only fix was phantom was rewritten instead of reported as nochange;
+  and a single-quoted sequential attribute was rewritten to double
+  quotes, touching already-correct markup. Quotes are now stripped
+  before the comparison and an already-correct attribute is returned
+  untouched: honest counts, single quotes survive, and the plugin's
+  zero-fixes contract holds end to end.
+
+### Repo records
+
+- Phase 16's FastSweep prevalence study (5,228 books) and the obfuscation
+  magic-byte validation are recorded in the roadmap with real counts for
+  every Phase 16 repair box.
+
 ## v0.36.0 (2026-09-12)
 
 ### The completeness spot-check: `audit completeness` (Phase 15)

bindery_cli-0.38.0/plugin/__init__.py

@@ -0,0 +1,239 @@
+"""Bindery Repair: repair EPUBs on import with bindery-cli's gate-safe pass.
+
+A Calibre FileTypePlugin running bindery-cli's always-on core pass (the five
+well-formedness transforms plus the NCX pipeline) on *.epub files as they are
+imported. The structural repairs and the lossy strips stay CLI-only: their
+acceptance IS the epubcheck gate, which cannot run inside Calibre (no jar, no
+JVM, and seconds-per-book latency), so their flags are never enabled here and
+nothing runs ungated, ever.
+
+The on_import contract: run(path) returns the path to a repaired copy built
+with the plugin's own temporary_file(), and Calibre imports that instead. The
+original on disk is never touched and nothing writes to metadata.db. No
+exception ever escapes run(): a raising plugin cannot abort an import (Calibre
+falls back to the original silently), so every failure returns the original
+path and logs one line. The plugin is byte-idempotent: an already-clean book
+yields zero fixes and the original path, which db.add_format's
+run_hooks=True re-entry requires.
+
+Configuration is the plain site_customization string, parsed as JSON::
+
+    {"log": true, "log_path": "/path/bindery_repair.log",
+     "max_size_mb": 150, "epubcheck_path": null}
+
+log_path defaults to <calibre config dir>/bindery_repair.log. The size cap
+refuses absurd files rather than stalling an import; 150MB is the recorded
+default. epubcheck_path opts into the experimental on-PATH validation mode:
+the repaired copy is re-measured with the epubcheck binary and refused unless
+it is no worse than the original (the CLI's no_worse bar). It defaults to
+off, keeping the import fast; the default pass is safe without a gate.
+
+The vendored bindery modules ride in this zip as submodules of the plugin
+package (their relative imports resolve unchanged), so the zip's module bytes
+are identical to the repo's src/bindery/ sources; scripts/build_plugin.py
+generates the zip at release and the suite's drift test pins the equality.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import subprocess
+import zipfile
+from collections import namedtuple
+from pathlib import Path
+
+__PLUGIN_VERSION__ = (0, 0, 0)  # substituted by scripts/build_plugin.py
+
+# The vendored bindery modules load as submodules of this plugin package
+# (calibre_plugins.bindery_repair.epub and siblings), so their own relative
+# imports resolve unchanged.
+from calibre.customize import FileTypePlugin
+
+from . import epub as _bindery_epub
+
+DEFAULT_MAX_SIZE_MB = 150
+_EPUB_SUFFIXES = (".epub",)
+# the slice of validate.CheckResult the experimental mode needs
+_EcCounts = namedtuple("_EcCounts", "fatals errors warnings")
+
+
+class BinderyRepair(FileTypePlugin):
+    name = "Bindery Repair"
+    description = (
+        "Repair well-formedness defects in EPUB files as they are imported, "
+        "running bindery-cli's epubcheck-safe default pass"
+    )
+    author = "Brandon LaRocque"
+    version = __PLUGIN_VERSION__
+    minimum_calibre_version = (2, 0, 0)
+    supported_platforms = ["linux"]
+    type = "File type"
+    file_types = {"epub"}
+    on_import = True
+    priority = 1
+
+    # ---- run() never raises: the contract is the original path on trouble
+
+    def run(self, path_to_ebook):
+        try:
+            return self._run(path_to_ebook)
+        except Exception as e:
+            # the last-resort sink re-reads the config, whose parsing is
+            # itself exception-safe; run() must never raise
+            self._log(
+                f"errored: {path_to_ebook}: {type(e).__name__}: {e}",
+                self._resolve_log_path(self._config()),
+            )
+            return path_to_ebook
+
+    def _run(self, path_to_ebook: str) -> str:
+        src = Path(path_to_ebook)
+        if src.suffix.lower() not in _EPUB_SUFFIXES:
+            return path_to_ebook
+        cfg = self._config()
+        log_path = self._resolve_log_path(cfg)
+        try:
+            size_mb = src.stat().st_size / (1024 * 1024)
+        except OSError as e:
+            self._log(f"errored: {src}: {e}", log_path)
+            return path_to_ebook
+        cap = cfg.get("max_size_mb", DEFAULT_MAX_SIZE_MB)
+        if isinstance(cap, (int, float)) and cap > 0 and size_mb > cap:
+            self._log(
+                f"refused: {src.name}: {size_mb:.0f}MB over the {cap:g}MB cap",
+                log_path,
+            )
+            return path_to_ebook
+
+        with self.temporary_file(".epub") as tf:
+            dst = Path(tf.name)
+            try:
+                report = _bindery_epub.repair_epub(src, dst)
+            except Exception as e:
+                self._log(
+                    f"errored: {src.name}: repair failed: {type(e).__name__}: {e}",
+                    log_path,
+                )
+                return path_to_ebook
+            if not report:
+                # zero fixes: hand back the original byte-for-byte (the
+                # add_format(run_hooks=True) re-entry depends on this)
+                dst.unlink(missing_ok=True)
+                self._log(f"no fixes: {src.name}", log_path)
+                return path_to_ebook
+            with zipfile.ZipFile(dst) as z:
+                damaged = z.testzip()
+            if damaged is not None:
+                dst.unlink(missing_ok=True)
+                self._log(
+                    f"refused: {src.name}: rewritten archive failed testzip() "
+                    f"at {damaged}",
+                    log_path,
+                )
+                return path_to_ebook
+            if "epubcheck_path" in cfg and cfg["epubcheck_path"]:
+                verdict = self._epubcheck_no_worse(src, dst, cfg["epubcheck_path"])
+                if verdict is False:
+                    dst.unlink(missing_ok=True)
+                    self._log(
+                        f"refused: {src.name}: epubcheck measured the repair "
+                        "no better or worse; keeping the original",
+                        log_path,
+                    )
+                    return path_to_ebook
+            detail = ", ".join(f"{k}={v}" for k, v in sorted(report.fixes.items()))
+            if report.ncx_uid_synced:
+                detail = (detail + ", " if detail else "") + "ncx_uid_synced"
+            self._log(f"fixed: {src.name}: {detail}", log_path)
+            return str(dst)
+
+    # ---- configuration and logging
+
+    def _config(self) -> dict:
+        raw = getattr(self, "site_customization", None)
+        if not raw:
+            return {}
+        try:
+            cfg = json.loads(raw)
+        except ValueError:
+            # a broken config lands one line in the default log and the run
+            # proceeds on the defaults; an import must never be lost to config
+            self._log(
+                "config: site_customization is not valid JSON; using defaults",
+                self._default_log_path(),
+            )
+            return {}
+        return cfg if isinstance(cfg, dict) else {}
+
+    def _resolve_log_path(self, cfg: dict) -> str | None:
+        if not cfg.get("log", True):
+            return None
+        path = cfg.get("log_path") or self._default_log_path()
+        return path if isinstance(path, str) else None
+
+    def _default_log_path(self) -> str | None:
+        try:
+            from calibre.constants import config_dir
+
+            return os.path.join(config_dir, "bindery_repair.log")
+        except Exception:
+            return None
+
+    def _log(self, line: str, log_path: str | None) -> None:
+        """One line per book, appended; logging must never break an import."""
+        if not log_path:
+            return
+        try:
+            with open(log_path, "a", encoding="utf-8") as f:
+                f.write(line.rstrip() + "\n")
+        except OSError:
+            pass
+
+    # ---- experimental epubcheck-on-PATH validation (default off)
+
+    def _epubcheck_no_worse(self, src: Path, dst: Path, epubcheck: str) -> bool | None:
+        """Measure both files with the epubcheck binary; True when the repair
+        is no worse (no net-new fatals, no new errors unless fatals masked
+        them), False when it regressed, None when the binary cannot answer
+        (an unanswered check never refuses the repair; the pass is safe)."""
+        before = self._epubcheck_counts(src, epubcheck)
+        after = self._epubcheck_counts(dst, epubcheck)
+        if before is None or after is None:
+            return None
+        if after.fatals > before.fatals:
+            return False
+        if before.fatals == 0 and after.errors > before.errors:
+            return False
+        return True
+
+    def _epubcheck_counts(self, path: Path, epubcheck: str):
+        try:
+            proc = subprocess.run(
+                [epubcheck, "--json", "-", str(path)],
+                capture_output=True,
+                text=True,
+                timeout=300,
+            )
+        except OSError, subprocess.TimeoutExpired:
+            return None
+        try:
+            data = json.loads(proc.stdout)
+            checker = data["checker"]
+            return _EcCounts(
+                int(checker["nFatal"]),
+                int(checker["nError"]),
+                int(checker["nWarning"]),
+            )
+        except json.JSONDecodeError, KeyError, TypeError, ValueError:
+            return None
+
+    # ---- plugin housekeeping
+
+    def customization_help(self, gui=False):
+        return (
+            'Enter a JSON object: {"log": true, "log_path": ..., '
+            '"max_size_mb": 150, "epubcheck_path": null}. '
+            "epubcheck_path enables the experimental post-repair validation "
+            "with the epubcheck binary (slow; off by default)."
+        )

{bindery_cli-0.36.0 → bindery_cli-0.38.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "bindery-cli"
-version = "0.36.0"
+version = "0.38.0"
 description = "Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library."
 readme = "README.md"
 classifiers = [