bindery-cli 0.37.0__tar.gz → 0.39.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/.github/workflows/publish.yml +2 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/CLAUDE.md +9 -6
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/PKG-INFO +5 -2
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/README.md +4 -1
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/patchnotes.md +95 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/pyproject.toml +1 -1
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/roadmap.md +79 -12
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/spec.md +35 -4
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/__init__.py +1 -1
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/audit.py +133 -41
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/cli.py +52 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/epub.py +275 -5
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_audit.py +140 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_cli.py +29 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_epub.py +421 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/uv.lock +1 -1
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/.github/workflows/ci.yml +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/.gitignore +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/AGENTS.md +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/LICENSE +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/REPORT-12-Sept.md +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/docs/screenshots/library-sweep.png +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/logo.svg +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/plugin/__init__.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/run_tests.sh +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/scripts/FastSweep.java +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/scripts/build_plugin.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/scripts/fast_sweep.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/__main__.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/library.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/pagination.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/reserialize.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/transforms.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/validate.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/src/bindery/watermark.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/testing_facility/books_to_fix/.gitkeep +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/testing_facility/top500candidates/REPORT.md +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_library.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_pagination.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_plugin.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_reserialize.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_transforms.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_validate.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_version.py +0 -0
- {bindery_cli-0.37.0 → bindery_cli-0.39.0}/tests/test_watermark.py +0 -0
|
@@ -29,9 +29,10 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
|
|
|
29
29
|
* **Structural repairs** (`--fix-empty-body`, `--fix-missing-title`, `--fix-id-colons`,
|
|
30
30
|
`--fix-page-map`, `--strip-epub3-attrs`, `--downgrade-epub3-tags`,
|
|
31
31
|
`--unwrap-block-in-inline`, `--strip-invalid-value`, `--unwrap-illegal-tags`,
|
|
32
|
-
`--prune-missing-resources`, `--strip-broken-anchors`, `--encode-url-spaces
|
|
33
|
-
transforms.py, threaded
|
|
34
|
-
|
|
32
|
+
`--prune-missing-resources`, `--strip-broken-anchors`, `--encode-url-spaces`,
|
|
33
|
+
`--fix-container`, `--fix-media-types`, `--fix-cover`; transforms.py, threaded
|
|
34
|
+
through epub.py):
|
|
35
|
+
they alter markup structure or fabricate minimal content. v0.14–v0.16 ran these unconditionally, which broke this rule;
|
|
35
36
|
v0.17.0 restored it. `--unwrap-illegal-tags` additionally protects any illegal-tag
|
|
36
37
|
name that an EPUB stylesheet styles as an element selector (`css_protected_tags`,
|
|
37
38
|
book-wide, inline `<style>` blocks included). The two EPUB2-targeted fixes
|
|
@@ -56,7 +57,9 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
|
|
|
56
57
|
the safe opt-ins (`--fix-ids`, `--add-img-alt`, `--strip-bad-attrs`,
|
|
57
58
|
`--escape-unknown-entities`) plus `--encode-url-spaces`, which repair without
|
|
58
59
|
altering visible markup; the "twelve structural repairs and three lossy strips"
|
|
59
|
-
phrasing elsewhere in these docs was never the complete inventory
|
|
60
|
+
phrasing elsewhere in these docs was never the complete inventory, and the
|
|
61
|
+
Phase 16 package repairs (`--fix-container`, `--fix-media-types`, `--fix-cover`)
|
|
62
|
+
took the structural count to fifteen (v0.38.0 + v0.39.0).
|
|
60
63
|
- **The gate is the safety contract.** Never apply a repair epubcheck has not accepted.
|
|
61
64
|
Respect the two-mode logic in `validate.gate` (fatal-fixing tolerates error unmasking;
|
|
62
65
|
error-cleanup does not). The lossy strips (`--strip-pagination`, `--strip-broken-tags`,
|
|
@@ -77,8 +80,8 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
|
|
|
77
80
|
- `src/bindery/pagination.py`: the opt-in lossy page-number strip (runhead detection, page-layer decision, block-centric removal/merge, safety nets).
|
|
78
81
|
- `src/bindery/watermark.py`: the opt-in lossy watermark strip (anchored and anchorless signature removal).
|
|
79
82
|
- `src/bindery/reserialize.py`: structural repair via `html5lib`.
|
|
80
|
-
- `src/bindery/audit.py`: read-only body-text audits (`content`, `pagenumbers`, `emptytext`, `ocr`, `monolithic` since v0.21.0 with `--max-doc-chars N`, and `completeness` since v0.36.0) behind the `audit` subcommand (v0.15.0). Since v0.18.0 library mode resolves EPUB paths through `cquarry.get_format_path()` and can apply a tag to flagged books via `cquarry.write.WritableCalibreDB` (only with explicit `--tag`; the only sanctioned write path). Since v0.19.0 `audit --id BOOK_ID` audits a single library book through cquarry's single-entity `get_book()` fetch (no library-wide cache; supports `--tag`; incompatible with the directory argument; comma lists since v0.23.0). Since v0.22.0 every archive entry is fully read (CRC + decompression) before analysis — a damaged entry reports its own CORRUPT verdict instead of feeding emptytext — and manifest/NCX references to absent files classify as `convention` (bloated ToC, consecutive span) or `fragment` (broken span). Hazard: the scan loop reuses `tag` as its per-book display column, so the `--tag` argument is captured as `audit_tag` at function entry — do not collapse them again. Since v0.29.0 `audit --json FILE` writes per-file analyzer verdicts in the `library --json` shape (directory, library, and single-book modes; with `--id`, exactly one id; emptytext omitted from a record when the archive verdict owns the body-text story). Since v0.36.0 the completeness analyzer (Phase 15) reports the phase-1 spot-check per book: spine/prose-doc counts (prose = >= 400 visible chars), first/middle/last prose-doc opening/closing excerpts, trailing-ToC classification (`_trailing_toc`: short link lines dominate, no paragraph-length blocks), and the unreadable fraction. It is ADVISORY by contract (problem always False; ADVISORY on a trailing ToC or unreadable fraction >= 10%), never moves the exit code, and is skipped like emptytext when the archive verdict owns the book.
|
|
81
|
-
- `src/bindery/epub.py`: archive rewrite, NCX uid sync, RepairReport, mismatch detection, and the opt-in structural-repair plumbing (including the CSS precondition scan).
|
|
83
|
+
- `src/bindery/audit.py`: read-only body-text audits (`content`, `pagenumbers`, `emptytext`, `ocr`, `monolithic` since v0.21.0 with `--max-doc-chars N`, and `completeness` since v0.36.0) behind the `audit` subcommand (v0.15.0). Since v0.18.0 library mode resolves EPUB paths through `cquarry.get_format_path()` and can apply a tag to flagged books via `cquarry.write.WritableCalibreDB` (only with explicit `--tag`; the only sanctioned write path). Since v0.19.0 `audit --id BOOK_ID` audits a single library book through cquarry's single-entity `get_book()` fetch (no library-wide cache; supports `--tag`; incompatible with the directory argument; comma lists since v0.23.0). Since v0.22.0 every archive entry is fully read (CRC + decompression) before analysis — a damaged entry reports its own CORRUPT verdict instead of feeding emptytext — and manifest/NCX references to absent files classify as `convention` (bloated ToC, consecutive span) or `fragment` (broken span). Hazard: the scan loop reuses `tag` as its per-book display column, so the `--tag` argument is captured as `audit_tag` at function entry — do not collapse them again. Since v0.29.0 `audit --json FILE` writes per-file analyzer verdicts in the `library --json` shape (directory, library, and single-book modes; with `--id`, exactly one id; emptytext omitted from a record when the archive verdict owns the body-text story). Since v0.36.0 the completeness analyzer (Phase 15) reports the phase-1 spot-check per book: spine/prose-doc counts (prose = >= 400 visible chars), first/middle/last prose-doc opening/closing excerpts, trailing-ToC classification (`_trailing_toc`: short link lines dominate, no paragraph-length blocks), and the unreadable fraction. It is ADVISORY by contract (problem always False; ADVISORY on a trailing ToC or unreadable fraction >= 10%), never moves the exit code, and is skipped like emptytext when the archive verdict owns the book. Since v0.38.0 the archive verdict distinguishes font obfuscation from DRM (`OBFUSCATION_ALGOS`: IDPF 2008/embedding + both Adobe URIs; readable obfuscated entries are the benign OBFUSCATED advisory, an unreadable one is CORRUPT, and only non-obfuscation algorithms give ENCRYPTED).
|
|
84
|
+
- `src/bindery/epub.py`: archive rewrite, NCX uid sync, RepairReport, mismatch detection, and the opt-in structural-repair plumbing (including the CSS precondition scan). Since v0.38.0/v0.39.0 (Phase 16): `generate_container` + the `--fix-container` gateway (insert after mimetype when missing, replace in-loop when stale; constant-epoch timestamp keeps bytes deterministic), `prune_dangling_edges` (prune_missing_manifest_items returns the pruned ids and the edges that pointed at them — spine@toc, media-overlay, fallback, EPUB2 cover meta — are rewritten so pruning cannot manufacture the regression the gate rejects), and `fix_manifest_media_types` (extension map + magic-byte confirmation via a `peek` callable over the open source zip; jpg/png/gif only), and `fix_cover_meta` (the ruling's deterministic half: dangling EPUB2 cover meta re-pointed from the guide's cover reference when exactly one manifest item carries that file, removed otherwise; accepted under no_worse like the lossy strips because cover wiring is invisible to epubcheck).
|
|
82
85
|
- `src/bindery/validate.py`: epubcheck wrapper, the `gate` (improvement) and `no_worse` (no-regression, for the lossy strips) acceptance bars.
|
|
83
86
|
- `src/bindery/library.py`: Calibre walk, atomic replace, backups, and native format installation. Since v0.19.0 `CalibreIdResolver` resolves the book id from `metadata.db` through cquarry (one lazy path→id map per run; since v0.20.0 the map comes from `CalibreDB.format_path_index()`, re-normalized with `resolve().lower()` for the resolver's symlinked-directory and case-insensitive matching); the `(id)` directory-name regex is only the no-catalog fallback. Since v0.24.0 `install_format()` places the repaired file atomically and updates the `data` row through `cquarry.write.WritableCalibreDB` (since v0.31.0 via `set_format`, cquarry 1.17's sanctioned remove+add in one transaction); the external `calibredb` CLI is gone. Since v0.32.0 a guessed id drives a row update only when metadata.db corroborates it (`_verify_guess`: books row exists, file in the book's own directory, catalogued `data.name` when a row exists); anything else saves in place and leaves the catalog untouched — never reintroduce directory-name guessing as the primary source: renamed/mismatched directories would replace the wrong book.
|
|
84
87
|
- `src/bindery/cli.py`: `repair` and `library` subcommands, including `--all` and `--install-to-calibre`; since v0.29.0 also the `run` verbs: `phase1` (pre-import vetting over loose files: the audit battery composed with one gated repair sweep; read-only until `--apply-lossy`, which IS the lossy-strip consent) and `phase3` (the scoped post-import apply step: `library --id --sweep --only all --apply --all --install-to-calibre` plus a pre/post summary; mechanically refuses unscoped sweeps with exit 2). Both take `--non-interactive` and surface open questions as `decisions_needed` in their JSON, never prompts. They drive the shipped `run_library` through the real parser, so no flag can drift between the verb and the subcommand it wraps. `library --workers N` (v0.30.0) parallelizes only the sweep's candidate pass, in windows of N consumed in order so `--limit` stays lazy; the repair phase is never parallel (shared workdir, atomic-replacement contract).
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: bindery-cli
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.39.0
|
|
4
4
|
Summary: Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library.
|
|
5
5
|
Author: vrnvctss
|
|
6
6
|
License: MIT
|
|
@@ -60,9 +60,12 @@ Everything below is off until its flag is passed (or all at once via `--all`). T
|
|
|
60
60
|
- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or the wider aria family survive). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
|
|
61
61
|
- **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents (`figure`/`section` to `div`, `figcaption` to `p`), keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
|
|
62
62
|
- **`--unwrap-illegal-tags`**: strips completely invalid or deprecated HTML tags that break EPUB3 validation (`<st>`, `<sentence>`, `<o>`, `<w>`, `<pagebreak>`) while retaining their inner text. Any of those names styled as an *element selector* by an EPUB stylesheet (`.css` entries and inline `<style>` blocks alike; class/id selectors like `.st`/`#w` don't count) is protected for the whole book, guaranteeing format preservation.
|
|
63
|
-
- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items. Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
|
|
63
|
+
- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items (with every package edge that pointed at them rewritten: `spine@toc`, `item@media-overlay`, `item@fallback`, and the EPUB2 cover meta). Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
|
|
64
64
|
- **`--strip-broken-anchors`**: strips href attributes that cannot resolve, keeping the anchor text byte-for-byte: a `#fragment` the target document does not define (RSC-020/RSC-012; NCX navTargets fall back to the document target, so chapter navigation survives) and unresolvable URI schemes (`kindle:`, `file:`).
|
|
65
65
|
- **`--encode-url-spaces`**: percent-encodes raw spaces in `src`/`href` attribute values across the package (OPF manifest, NCX, content documents). A URL with a literal space is not a valid URL (RSC-020) and unresolvable on strict readers; the encoded form denotes the same file.
|
|
66
|
+
- **`--fix-container`**: generates `META-INF/container.xml` at the located OPF when the container is missing or names a file the archive does not contain. This is the gateway defect: epubcheck stays fatal while the OPF is unfindable, so no repair can be gate-accepted on such a book until the container exists.
|
|
67
|
+
- **`--fix-media-types`**: normalizes wrong manifest media-type declarations (OPF-029, e.g. a jpg stamped `image/png` by an aggregator). Attribute-only and quote-preserving; it fires only when the file's magic bytes confirm the extension, so an honestly-declared misnamed file is never made worse.
|
|
68
|
+
- **`--fix-cover`**: repairs dangling EPUB2 cover wiring. A `<meta name="cover">` whose content names no manifest item is re-pointed when the OPF guide's own cover reference names an existing item, and removed when nothing identifies it; EPUB3 `properties="cover-image"` is deliberately audit-only (guessing which image is the cover is not deterministic). Cover wiring is invisible to epubcheck, so cover-only repairs are accepted under the same no-worse bar the lossy strips use.
|
|
66
69
|
|
|
67
70
|
|
|
68
71
|
Three opt-in fixes are **lossy** and stand apart from the semantics-preserving rest:
|
|
@@ -38,9 +38,12 @@ Everything below is off until its flag is passed (or all at once via `--all`). T
|
|
|
38
38
|
- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or the wider aria family survive). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
|
|
39
39
|
- **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents (`figure`/`section` to `div`, `figcaption` to `p`), keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
|
|
40
40
|
- **`--unwrap-illegal-tags`**: strips completely invalid or deprecated HTML tags that break EPUB3 validation (`<st>`, `<sentence>`, `<o>`, `<w>`, `<pagebreak>`) while retaining their inner text. Any of those names styled as an *element selector* by an EPUB stylesheet (`.css` entries and inline `<style>` blocks alike; class/id selectors like `.st`/`#w` don't count) is protected for the whole book, guaranteeing format preservation.
|
|
41
|
-
- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items. Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
|
|
41
|
+
- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items (with every package edge that pointed at them rewritten: `spine@toc`, `item@media-overlay`, `item@fallback`, and the EPUB2 cover meta). Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
|
|
42
42
|
- **`--strip-broken-anchors`**: strips href attributes that cannot resolve, keeping the anchor text byte-for-byte: a `#fragment` the target document does not define (RSC-020/RSC-012; NCX navTargets fall back to the document target, so chapter navigation survives) and unresolvable URI schemes (`kindle:`, `file:`).
|
|
43
43
|
- **`--encode-url-spaces`**: percent-encodes raw spaces in `src`/`href` attribute values across the package (OPF manifest, NCX, content documents). A URL with a literal space is not a valid URL (RSC-020) and unresolvable on strict readers; the encoded form denotes the same file.
|
|
44
|
+
- **`--fix-container`**: generates `META-INF/container.xml` at the located OPF when the container is missing or names a file the archive does not contain. This is the gateway defect: epubcheck stays fatal while the OPF is unfindable, so no repair can be gate-accepted on such a book until the container exists.
|
|
45
|
+
- **`--fix-media-types`**: normalizes wrong manifest media-type declarations (OPF-029, e.g. a jpg stamped `image/png` by an aggregator). Attribute-only and quote-preserving; it fires only when the file's magic bytes confirm the extension, so an honestly-declared misnamed file is never made worse.
|
|
46
|
+
- **`--fix-cover`**: repairs dangling EPUB2 cover wiring. A `<meta name="cover">` whose content names no manifest item is re-pointed when the OPF guide's own cover reference names an existing item, and removed when nothing identifies it; EPUB3 `properties="cover-image"` is deliberately audit-only (guessing which image is the cover is not deterministic). Cover wiring is invisible to epubcheck, so cover-only repairs are accepted under the same no-worse bar the lossy strips use.
|
|
44
47
|
|
|
45
48
|
|
|
46
49
|
Three opt-in fixes are **lossy** and stand apart from the semantics-preserving rest:
|
|
@@ -1,4 +1,99 @@
|
|
|
1
1
|
# bindery-cli Patch Notes
|
|
2
|
+
## v0.39.0 (2026-09-12)
|
|
3
|
+
|
|
4
|
+
### The cover-wiring repair (`--fix-cover`) and the executed rulings
|
|
5
|
+
|
|
6
|
+
- **`--fix-cover` repairs dangling EPUB2 cover wiring.** A
|
|
7
|
+
`<meta name="cover">` whose content names no manifest item is
|
|
8
|
+
re-pointed when the OPF guide's own cover reference resolves to
|
|
9
|
+
exactly one manifest item (the producer's statement, so nothing is
|
|
10
|
+
guessed), and removed when nothing identifies the item. A meta whose
|
|
11
|
+
item exists is never touched: an absent cover file belongs to
|
|
12
|
+
`--prune-missing-resources` and its edge completion.
|
|
13
|
+
- **Gate treatment:** cover wiring is invisible to epubcheck, so a
|
|
14
|
+
cover-only repair always answered 'equal' from the improvement gate.
|
|
15
|
+
Cover repairs now ride the same no_worse acceptance the lossy strips
|
|
16
|
+
use, with the partial rule intact: a regression rejects, a
|
|
17
|
+
still-fatal book stays partial and is never auto-applied. Verified
|
|
18
|
+
on a real dangling-meta book from the library: re-pointed 1,
|
|
19
|
+
accepted.
|
|
20
|
+
- **Scope by ruling (the hybrid):** the EPUB3
|
|
21
|
+
`properties="cover-image"` slice stays audit-only. Choosing which
|
|
22
|
+
image is the cover from filenames is exactly the non-determinism the
|
|
23
|
+
charter forbids, and no library book is fully unwired (measured:
|
|
24
|
+
21 dangling EPUB2 metas, 3 absent cover files, 103 EPUB3 books
|
|
25
|
+
without the property, 0 with no wiring at all).
|
|
26
|
+
- **The metadata-validity carve-out is ruled: routed to
|
|
27
|
+
cquarry/CalibreQuarry.** bindery's side is closed: human-facing dc:
|
|
28
|
+
metadata stays a non-goal. The cross-repo handoff (a routed box in
|
|
29
|
+
CalibreQuarry's backlog naming the 51 OPF-085 invalid-UUID warnings
|
|
30
|
+
and the thin date/language tail) is filed for the next CQ lane.
|
|
31
|
+
- **The tag-policy force-push repairs are executed** on a researched
|
|
32
|
+
go: the v0.28.0 tag retargeted to the first green commit with its
|
|
33
|
+
title line restored (the historic `git tag -F` defect had stripped
|
|
34
|
+
it), the v0.33.0 tag message replaced with the corrected verbatim
|
|
35
|
+
entry, and the 10 test_facility EPUBs stripped from all history
|
|
36
|
+
(git filter-repo; main and all 14 tags force-pushed in one
|
|
37
|
+
publish.yml-disabled session; verbatim tag messages survived; PyPI
|
|
38
|
+
untouched). The roadmap's Tag policy section records the execution.
|
|
39
|
+
|
|
40
|
+
## v0.38.0 (2026-09-12)
|
|
41
|
+
|
|
42
|
+
### Phase 16: package-structure repairs
|
|
43
|
+
|
|
44
|
+
- **`--fix-container`, the gateway repair.** A book whose
|
|
45
|
+
META-INF/container.xml is missing, unparseable, or names a file the
|
|
46
|
+
archive does not contain is locked out of every repair: epubcheck
|
|
47
|
+
stays fatal while the OPF is unfindable. The new flag generates the
|
|
48
|
+
standard container at the OPF the locator already finds by fallback
|
|
49
|
+
(a byte-deterministic template with the constant epoch timestamp).
|
|
50
|
+
Oracle-verified on a real-shaped fixture: fatal before, gate-accepted
|
|
51
|
+
after. The Phase 16 prevalence study found 0 such books in-library
|
|
52
|
+
(5,228/5,228 have valid containers); the repair ships for the
|
|
53
|
+
acquisition phase, where the class actually lives.
|
|
54
|
+
- **`--prune-missing-resources` rewrites what it drops.** Removing a
|
|
55
|
+
manifest item whose file is absent used to leave every package edge
|
|
56
|
+
pointing at it dangling, manufacturing the regression the gate then
|
|
57
|
+
rejects. The prune now carries the pruned ids into a rewrite of the
|
|
58
|
+
four edge classes: `spine@toc`, `item@media-overlay`, `item@fallback`,
|
|
59
|
+
and the EPUB2 cover meta. Optional edges of dead targets are removed;
|
|
60
|
+
edges to live ids are untouched.
|
|
61
|
+
- **`--fix-media-types`.** Wrong manifest media-type declarations
|
|
62
|
+
(OPF-029; 275 occurrences in 15 library books, the class being a jpg
|
|
63
|
+
stamped `image/png` by an aggregator) are normalized attribute-only,
|
|
64
|
+
preserving quote style, and only when the file's magic bytes confirm
|
|
65
|
+
the extension: a PNG renamed .jpg keeps its wrong-but-honest
|
|
66
|
+
declaration instead of a worse one. On the real Arthur sample the run
|
|
67
|
+
goes from nochange to gate-accepted with 9 declarations fixed.
|
|
68
|
+
- **The audit distinguishes font obfuscation from DRM.** encryption.xml
|
|
69
|
+
entries under the obfuscation algorithms (the IDPF URI and both Adobe
|
|
70
|
+
forms, including the ns.adobe.com URI real-world files actually
|
|
71
|
+
carry, which the prevalence study caught) that read fine are the new
|
|
72
|
+
OBFUSCATED advisory: publisher font embedding, benign, problem-False,
|
|
73
|
+
exit-neutral. An unreadable obfuscation entry is CORRUPT (a broken
|
|
74
|
+
font, re-source), and only non-obfuscation algorithms give the
|
|
75
|
+
ENCRYPTED skip. 86 of the library's 5,228 books carry
|
|
76
|
+
obfuscation-only encryption.xml; the library holds no real DRM at all.
|
|
77
|
+
- **Boxed, not built, with real counts.** The obfuscation-aware
|
|
78
|
+
encryption repair (the magic-byte precondition validated on 424 real
|
|
79
|
+
fonts: all genuinely scrambled, zero stale entries in-library), href
|
|
80
|
+
case/backslash resolution (1 href in 1 book; 0 backslashes), and
|
|
81
|
+
duplicate zip-entry dedupe (0 books) are recorded in the roadmap with
|
|
82
|
+
their prevalence numbers and reopen conditions. The decision-gated
|
|
83
|
+
Phase 16 C boxes (cover wiring, the metadata carve-out, the NCX<->nav
|
|
84
|
+
drift detector, CSS url() pruning) carry recorded options and wait
|
|
85
|
+
for rulings.
|
|
86
|
+
|
|
87
|
+
### The prevalence study (Phase 16 A)
|
|
88
|
+
|
|
89
|
+
- 5,228 library books through the FastSweep epubcheck harness plus a
|
|
90
|
+
direct zip-level pass: container.xml valid on every book; RSC-007 at
|
|
91
|
+
14,156 occurrences in 600 books and PKG-010 in 280 (the prune
|
|
92
|
+
surface, 1,538 truly-absent manifest hrefs); OPF-029 at 275 in 15;
|
|
93
|
+
case-mismatch hrefs exactly 1; encryption.xml in 86 books, all of it
|
|
94
|
+
font obfuscation. Every Phase 16 repair box now carries its real
|
|
95
|
+
count.
|
|
96
|
+
|
|
2
97
|
## v0.37.0 (2026-09-12)
|
|
3
98
|
|
|
4
99
|
### The Bindery Repair Calibre plugin (Phase 3)
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "bindery-cli"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.39.0"
|
|
8
8
|
description = "Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
classifiers = [
|
|
@@ -1483,48 +1483,112 @@ an opt-in structural repair under the existing gate contract.
|
|
|
1483
1483
|
|
|
1484
1484
|
### B. The repairs (each: opt-in flag, dry-run default, gate-accepted)
|
|
1485
1485
|
|
|
1486
|
-
- [
|
|
1486
|
+
- [x] **`--fix-container`**: generate META-INF/container.xml at the
|
|
1487
1487
|
located OPF (model: upstream calibre's initialize_container).
|
|
1488
1488
|
Gateway repair, effort S, HIGH value.
|
|
1489
|
+
*(shipped v0.38.0, dac7835: fires when the container is missing,
|
|
1490
|
+
unparseable, or names a file the archive does not contain; the
|
|
1491
|
+
generated entry carries the constant epoch so repairs stay
|
|
1492
|
+
byte-deterministic. Prevalence: 0 books in-library (5,228/5,228 have
|
|
1493
|
+
valid containers) — shipped anyway as the acquisition-phase gateway;
|
|
1494
|
+
oracle-verified on a real-shaped fixture (fatal before, accept
|
|
1495
|
+
after).)*
|
|
1489
1496
|
- [ ] **Obfuscation-aware encryption.xml repair**: prune stale
|
|
1490
1497
|
obfuscation entries when font magic bytes prove the fonts are
|
|
1491
1498
|
decrypted in place; drop encryption.xml when only obfuscation entries
|
|
1492
1499
|
remain and fonts verify. NEVER un-obfuscate. M.
|
|
1493
|
-
|
|
1500
|
+
*(BOXED with real counts, 2026-09-12: the magic-byte validation
|
|
1501
|
+
proved 424/424 readable obfuscation fonts genuinely scrambled — zero
|
|
1502
|
+
stale entries in-library, so the repair has nothing to do here; its
|
|
1503
|
+
precondition is validated and recognition carries both Adobe URIs.
|
|
1504
|
+
The audit half of this box SHIPPED (see below). Opens only if
|
|
1505
|
+
acquisition ever brings a decrypted-in-place book.)*
|
|
1506
|
+
- [x] **Audit distinction: OBFUSCATED vs DRM** in audit.py (today the
|
|
1494
1507
|
ENCRYPTED verdict keys on unreadable entries and implies DRM where
|
|
1495
1508
|
publisher obfuscation is benign). S, rides the box above.
|
|
1496
|
-
|
|
1509
|
+
*(shipped v0.38.0, dac7835: readable obfuscation entries are the
|
|
1510
|
+
OBFUSCATED advisory (problem-False, exit-neutral; 86 library books);
|
|
1511
|
+
an unreadable obfuscation entry is CORRUPT (broken font, re-source),
|
|
1512
|
+
and only non-obfuscation algorithms give the ENCRYPTED skip.)*
|
|
1513
|
+
- [x] **Reference-edge completion for `--prune-missing-resources`**:
|
|
1497
1514
|
rewrite `spine@toc`, `item@media-overlay`, `item@fallback`, and EPUB2
|
|
1498
1515
|
cover-meta edges when their targets are pruned (machinery exists in
|
|
1499
1516
|
fix_manifest_ids). Prevents manufacturing gate regressions. S/M.
|
|
1500
|
-
|
|
1517
|
+
*(shipped v0.38.0, dac7835: prune_missing_manifest_items returns the
|
|
1518
|
+
pruned ids; prune_dangling_edges removes the optional attributes and
|
|
1519
|
+
a cover meta whose payload died. RSC-007 backs it at 14,156
|
|
1520
|
+
occurrences in 600 books.)*
|
|
1521
|
+
- [x] **Manifest media-type normalization**: sniff magic bytes /
|
|
1501
1522
|
extension map; attribute-only. S.
|
|
1523
|
+
*(shipped v0.38.0, dac7835 as `--fix-media-types`: OPF-029 class,
|
|
1524
|
+
275 occurrences in 15 books; jpg/png/gif only, fired only when the
|
|
1525
|
+
magic bytes confirm the extension. Oracle-verified on the real
|
|
1526
|
+
Arthur sample: nochange without the flag, accept with 9 normalized.)*
|
|
1502
1527
|
- [ ] **href case/backslash resolution**: case-insensitive namemap when
|
|
1503
1528
|
exactly one zip entry matches; backslash to slash. Kills the
|
|
1504
1529
|
present-file RSC-007/PKG-010 slice prune cannot touch. M.
|
|
1530
|
+
*(BOXED with real counts, 2026-09-12: exactly 1 case-mismatch href in
|
|
1531
|
+
1 book and 0 backslash hrefs across the library — below the shipping
|
|
1532
|
+
bar; reopen if acquisition changes the mix.)*
|
|
1505
1533
|
- [ ] **Duplicate zip-entry dedupe on rewrite**: drop shadowed
|
|
1506
1534
|
duplicates (audit already counts `dup_entries`). S.
|
|
1535
|
+
*(BOXED with real counts, 2026-09-12: 0 books with duplicate entries
|
|
1536
|
+
in-library.)*
|
|
1507
1537
|
|
|
1508
1538
|
### C. Decision-gated (recorded boxes; no work without a ruling)
|
|
1509
1539
|
|
|
1510
|
-
- [
|
|
1540
|
+
- [x] **Cover wiring repair** (dangling `<meta name="cover">`, missing
|
|
1511
1541
|
`properties="cover-image"`, wrong cover media-type): narrow,
|
|
1512
1542
|
finding-driven repair vs audit-only. Needs the scope decision. M.
|
|
1543
|
+
*(RULED 2026-09-12 (Brandon, on the researched options): hybrid.
|
|
1544
|
+
SHIPPED the deterministic half as `--fix-cover` (tests 96a55ad,
|
|
1545
|
+
implementation 790b5f4; v0.39.0): a dangling EPUB2 cover meta is
|
|
1546
|
+
re-pointed from the guide's cover reference when exactly one manifest
|
|
1547
|
+
item carries that file, removed when nothing identifies the item, and
|
|
1548
|
+
a meta whose item exists is left to the prune + edge-completion pair.
|
|
1549
|
+
Measured class: 21 dangling metas + 3 absent files; oracle check on a
|
|
1550
|
+
real book re-pointed 1 and the gate's no_worse bar accepts it (cover
|
|
1551
|
+
wiring is invisible to epubcheck, so cover-only repairs ride the
|
|
1552
|
+
lossy-strips acceptance path). The EPUB3 `properties="cover-image"`
|
|
1553
|
+
slice stays audit-only by ruling: picking "which image is the cover"
|
|
1554
|
+
from names is the guesswork the charter forbids, and 0 library books
|
|
1555
|
+
are fully unwired.)*
|
|
1513
1556
|
- [ ] **Metadata validity carve-out ruling** (non-ISO `dc:date`,
|
|
1514
1557
|
invalid `dc:language`, missing `dcterms:modified`): bindery opt-in
|
|
1515
1558
|
with a determinism-safe timestamp policy, or route to
|
|
1516
1559
|
cquarry/CalibreQuarry. Needs a written charter ruling. M.
|
|
1560
|
+
*(Recorded options, 2026-09-12: (a) bindery opt-in — non-ISO dates
|
|
1561
|
+
and language tags are deterministic normalizations (OPF-007/OPF-012
|
|
1562
|
+
class), but `dcterms:modified` needs a timestamp policy that breaks
|
|
1563
|
+
byte-determinism unless pinned to the file's own zip timestamps;
|
|
1564
|
+
(b) route to cquarry/CalibreQuarry — metadata quality is catalog
|
|
1565
|
+
territory and the library's metadata writes already live there.
|
|
1566
|
+
(b) is the cleaner charter fit; (a) only if epubcheck fatals on such
|
|
1567
|
+
books show up in acquisition.)*
|
|
1568
|
+
*(RULED 2026-09-12 (Brandon, on the researched options): route to
|
|
1569
|
+
cquarry/CalibreQuarry. The ruling is recorded here; the cross-repo
|
|
1570
|
+
handoff (a routed box in CalibreQuarry's backlog naming the class:
|
|
1571
|
+
51 OPF-085 invalid-UUID warnings plus the thin date/language tail)
|
|
1572
|
+
is a main-thread task for the next CQ lane. bindery's side is closed:
|
|
1573
|
+
dc: metadata stays a non-goal.) *
|
|
1517
1574
|
- [ ] **NCX<->nav drift DETECTOR** (audit-only; structural diff feeding
|
|
1518
1575
|
`decisions_needed`; ToC synthesis stays out of charter
|
|
1519
1576
|
permanently). L if repair is ever attempted.
|
|
1520
1577
|
- [ ] **CSS url() pruning** as `--prune-missing-resources` scope
|
|
1521
|
-
extension: only after the prevalence study says it matters.
|
|
1578
|
+
extension: only after the prevalence study says it matters.
|
|
1579
|
+
*(Prevalence read, 2026-09-12: CSS-007/CSS-008 appear at 56/30 books
|
|
1580
|
+
but are selector damage, not url() pruning candidates; no url()-class
|
|
1581
|
+
count surfaced in the extract codes. Stays closed until a named
|
|
1582
|
+
epubcheck finding demands it.)*
|
|
1522
1583
|
|
|
1523
1584
|
Ship shape: A leads; then B in value order (container, encryption,
|
|
1524
1585
|
prune-edges, media-types); C boxes record their decisions and wait.
|
|
1525
1586
|
Each repair ships with a fixture built from the prevalence study's
|
|
1526
1587
|
real samples, a regression test written before the fix, and the gate
|
|
1527
1588
|
(epubcheck) accepting the result.
|
|
1589
|
+
*(2026-09-12 disposition: B shipped the four boxes with real backing
|
|
1590
|
+
and boxed the two zero-prevalence classes with their counts; C carries
|
|
1591
|
+
recorded options and waits for Brandon.)*
|
|
1528
1592
|
|
|
1529
1593
|
## Tag policy (decided 2026-09-11, recorded 2026-09-12)
|
|
1530
1594
|
|
|
@@ -1537,9 +1601,12 @@ The release-tag backlog is settled, forward-only:
|
|
|
1537
1601
|
of record (2026-09-11): no backfill tags will be cut for them. Their
|
|
1538
1602
|
release notes live in patchnotes.md and their wheels are on PyPI; the
|
|
1539
1603
|
missing git tags are accepted history.
|
|
1540
|
-
- **
|
|
1541
|
-
v0.28.0
|
|
1542
|
-
|
|
1543
|
-
|
|
1544
|
-
|
|
1545
|
-
|
|
1604
|
+
- **EXECUTED 2026-09-12 on Brandon's researched go** (one
|
|
1605
|
+
publish.yml-disabled session): the v0.28.0 tag now points at the
|
|
1606
|
+
first green commit (aabdd7a) and its message regains the title line
|
|
1607
|
+
the historic `git tag -F` defect had stripped; the v0.33.0 tag
|
|
1608
|
+
message is the corrected verbatim entry (61b16ea's entry, zero
|
|
1609
|
+
em-dashes); and the 10 test_facility EPUBs (plus their scratch test
|
|
1610
|
+
files) are stripped from all history via git filter-repo, main and
|
|
1611
|
+
all 14 tags force-pushed. The tags' verbatim messages survived the
|
|
1612
|
+
rewrite; PyPI was untouched throughout.
|
|
@@ -14,10 +14,11 @@ schema (RSC-005) violations, which are usually harmless to readers and not safel
|
|
|
14
14
|
mechanizable.
|
|
15
15
|
|
|
16
16
|
The deliberate exceptions to "semantics-preserving" come in two kinds, both strictly
|
|
17
|
-
opt-in. The
|
|
17
|
+
opt-in. The fourteen **structural repairs** (`--fix-empty-body`, `--fix-missing-title`,
|
|
18
18
|
`--fix-id-colons`, `--fix-page-map`, `--strip-epub3-attrs`, `--downgrade-epub3-tags`,
|
|
19
19
|
`--unwrap-block-in-inline`, `--strip-invalid-value`, `--unwrap-illegal-tags`,
|
|
20
|
-
`--prune-missing-resources`, `--strip-broken-anchors`, `--encode-url-spaces
|
|
20
|
+
`--prune-missing-resources`, `--strip-broken-anchors`, `--encode-url-spaces`,
|
|
21
|
+
`--fix-container`, `--fix-media-types`, `--fix-cover`) alter
|
|
21
22
|
markup structure or fabricate minimal content; the three
|
|
22
23
|
**lossy modes** (`--strip-pagination`, `--strip-broken-tags`, `--strip-watermarks`) remove
|
|
23
24
|
content a converter injected rather than content the author wrote. The default pass runs
|
|
@@ -91,7 +92,7 @@ normal gate applies.
|
|
|
91
92
|
|
|
92
93
|
### Opt-in: structural repairs
|
|
93
94
|
|
|
94
|
-
|
|
95
|
+
Fifteen repairs go past well-formedness and therefore require their own flag; none is ever
|
|
95
96
|
part of the default pipeline:
|
|
96
97
|
|
|
97
98
|
- **`--fix-empty-body`**: ` ` inside a strictly empty `<body></body>` ("body
|
|
@@ -168,7 +169,30 @@ every EPUB3 book in an `--all` sweep taking exactly that damage).
|
|
|
168
169
|
strict readers; the encoded form denotes the same file and renders identically. Scope
|
|
169
170
|
is fixed to the space character: extend only with a named epubcheck finding.
|
|
170
171
|
|
|
171
|
-
|
|
172
|
+
- **`--fix-container`**: generate `META-INF/container.xml` at the located OPF when the
|
|
173
|
+
container is missing, unparseable, or names a file the archive does not contain. This is the
|
|
174
|
+
gateway defect: epubcheck stays fatal while the OPF is unfindable, so no other repair can be
|
|
175
|
+
gate-accepted on such a book. The generated file is byte-deterministic (a fixed template plus
|
|
176
|
+
the OPF path; the entry's timestamp is the constant epoch), and a healthy container is never
|
|
177
|
+
touched.
|
|
178
|
+
- **`--fix-media-types`**: normalize wrong manifest `media-type` declarations (OPF-029: a
|
|
179
|
+
file's bytes do not match the declared type). The expected type comes from the extension, and
|
|
180
|
+
the rewrite fires only when the file's magic bytes at offset 0 confirm the extension (jpg,
|
|
181
|
+
jpeg, png, gif): a PNG renamed `.jpg` keeps its wrong-but-honest declaration rather than
|
|
182
|
+
gaining a worse one. Attribute-only, quote-style preserved; files absent from the archive
|
|
183
|
+
belong to `--prune-missing-resources`.
|
|
184
|
+
|
|
185
|
+
- **`--fix-cover`**: repair dangling EPUB2 cover wiring (the cover-wiring ruling's
|
|
186
|
+
deterministic half). A `<meta name="cover" content="X">` whose `X` names no manifest id is
|
|
187
|
+
re-pointed when the OPF guide's `<reference type="cover">` resolves to exactly one manifest
|
|
188
|
+
item (the producer's own statement; nothing is guessed), and removed when nothing identifies
|
|
189
|
+
the item. A meta whose id exists in the manifest is never touched here, even when the item's
|
|
190
|
+
file is absent: that class belongs to `--prune-missing-resources` and its edge completion.
|
|
191
|
+
The EPUB3 `properties="cover-image"` slice is audit-only by ruling. Cover wiring is invisible
|
|
192
|
+
to epubcheck, so cover-only repairs are accepted under the `no_worse` bar the lossy strips
|
|
193
|
+
use, with the `partial` rule intact.
|
|
194
|
+
|
|
195
|
+
All fifteen are evaluated by the normal `gate`: unlike the lossy strips, their benefit is
|
|
172
196
|
visible to epubcheck (they clear errors), so a run with no measurable improvement is a
|
|
173
197
|
noop and nothing is applied. CDATA sections and comments are never rewritten, as
|
|
174
198
|
everywhere else.
|
|
@@ -353,6 +377,13 @@ is fully read for CRC + decompression, reporting CORRUPT rather than EMPTY), and
|
|
|
353
377
|
issues. Manifest/NCX references to absent files are classified as either `convention` (ToC is bloated but
|
|
354
378
|
present documents form a consecutive chapter span) or `fragment` (the span itself is broken).
|
|
355
379
|
|
|
380
|
+
The archive verdict distinguishes font obfuscation from DRM (Phase 16): entries
|
|
381
|
+
`encryption.xml` declares under a font-obfuscation algorithm (the IDPF `2008/embedding`
|
|
382
|
+
URI and both Adobe forms, including the `ns.adobe.com` URI real-world files carry) that
|
|
383
|
+
read fine are the OBFUSCATED advisory — publisher embedding, benign, never a failure; an
|
|
384
|
+
unreadable obfuscation entry is CORRUPT (a broken font, not a business model), and only
|
|
385
|
+
non-obfuscation algorithms give the ENCRYPTED verdict with its DRM skip advice.
|
|
386
|
+
|
|
356
387
|
The completeness analyzer is the phase-1 spot-check and is advisory by contract: it never
|
|
357
388
|
flags a book and never moves the exit code. Per book it reports the spine doc count, the
|
|
358
389
|
prose-doc count (spine docs carrying at least 400 visible characters), the opening and closing
|