bindery-cli 0.54.0__tar.gz → 0.56.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/AGENTS.md +44 -9
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/OBSERVE-RESEARCH.md +136 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/PKG-INFO +16 -5
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/README.md +15 -4
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/patchnotes.md +139 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/pyproject.toml +1 -1
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/roadmap.md +2 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/spec.md +102 -25
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/__init__.py +1 -1
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/cli.py +98 -12
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/epub.py +309 -24
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/observe.py +113 -16
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/transforms.py +39 -14
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/validate.py +67 -18
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_cli.py +15 -1
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_epub.py +339 -6
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_flags_real_core.py +63 -4
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_flags_wiring.py +3 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_observe.py +69 -5
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_transforms.py +31 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_validate.py +58 -2
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/uv.lock +1 -1
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/.github/FUNDING.yml +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/.github/workflows/ci.yml +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/.github/workflows/publish.yml +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/.gitignore +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/LICENSE +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/docs/screenshots/library-sweep.png +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/logo.svg +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/plugin/__init__.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/project.done +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/run_tests.sh +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/scripts/FastSweep.java +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/scripts/build_plugin.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/scripts/fast_sweep.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/__main__.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/audit.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/library.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/pagination.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/reserialize.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/src/bindery/watermark.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_audit.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_comment_hyphens.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_contamination.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_cover_tocdrift.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_help.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_library.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_matcher_hardening.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_pagination.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_plugin.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_reserialize.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_space_renames.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_spine_guard.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_version.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_watermark.py +0 -0
- {bindery_cli-0.54.0 → bindery_cli-0.56.0}/tests/test_xml_hardening.py +0 -0
|
@@ -30,14 +30,16 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
|
|
|
30
30
|
`xmlns`, bare `&`, named entities, void self-closing), the NCX pipeline, and the
|
|
31
31
|
mimetype fix (added/normalized/first-stored, epub.py's archive rewrite); every core
|
|
32
32
|
fix must render identically to the author's intent: never add, remove, or reorder
|
|
33
|
-
visible content. The canonical opt-in inventory lives in spec.md: twenty-
|
|
33
|
+
visible content. The canonical opt-in inventory lives in spec.md: twenty-five structural
|
|
34
34
|
repairs, four lossy strips, four safe opt-ins, plus `--reserialize`.
|
|
35
35
|
* **Structural repairs** (`--fix-empty-body`, `--fix-missing-title`, `--fix-id-colons`,
|
|
36
36
|
`--fix-page-map`, `--strip-epub3-attrs`, `--downgrade-epub3-tags`,
|
|
37
37
|
`--unwrap-block-in-inline`, `--strip-invalid-value`, `--unwrap-illegal-tags`,
|
|
38
|
-
`--prune-missing-resources`, `--
|
|
38
|
+
`--prune-missing-resources`, `--declare-missing-items`, `--strip-broken-anchors`,
|
|
39
|
+
`--encode-url-spaces`,
|
|
39
40
|
`--fix-container`, `--fix-media-types`, `--fix-cover`,
|
|
40
|
-
`--fix-comment-double-hyphen`, `--fix-svg-dup-ids`, `--fix-
|
|
41
|
+
`--fix-comment-double-hyphen`, `--fix-svg-dup-ids`, `--fix-duplicate-ids`,
|
|
42
|
+
`--fix-cdata-terminator`,
|
|
41
43
|
`--fix-misnested-inline`, `--fix-stray-close`, `--fix-unterminated-attr`,
|
|
42
44
|
`--fix-unbound-prefix`;
|
|
43
45
|
transforms.py, threaded through epub.py). The v0.47.0 five (the 2026-10-06
|
|
@@ -70,6 +72,34 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
|
|
|
70
72
|
element under an OPF-namespaced package, anything else refuses the whole
|
|
71
73
|
document; scope-tracked over a comment/CDATA-aware tokenizer, OPF only
|
|
72
74
|
(content docs have the `--strip-bad-attrs` drop); normal `gate`.
|
|
75
|
+
The v0.55.0 two (the 2026-10-25 issue batch): `--declare-missing-items`
|
|
76
|
+
(issue #16) is the RSC-008 inverse of the prune -- it ADDS manifest
|
|
77
|
+
`<item>` declarations for files the archive carries and the book
|
|
78
|
+
references (content img/link, CSS url(), OPF guide refs) but the OPF
|
|
79
|
+
never declared; deterministic sorted-order `bindery-dN` ids, blessed
|
|
80
|
+
extension-map media types only, no itemref ever written (the spine is
|
|
81
|
+
untouched; an itemref with no manifest item stays unrepaired -- which
|
|
82
|
+
file it meant is underdetermined); normal `gate`. `--fix-duplicate-ids`
|
|
83
|
+
(issue #14) is the content-doc half of the SVG duplicate-id rename:
|
|
84
|
+
same first-keeps-name rule, references resolve to the first definition
|
|
85
|
+
and are never rewritten; normal `gate`. The same lane's RSC-001 verdict:
|
|
86
|
+
95.3% of its mass is spine documents whose files exist under a
|
|
87
|
+
different spelling (the spine guard correctly refuses; the mass carrier
|
|
88
|
+
is `--encode-url-spaces` territory), so the prune's coverage boundary is
|
|
89
|
+
documented, not widened.
|
|
90
|
+
The v0.56.0 one (the boxed OPF-030 recipe, built same-day on Brandon's
|
|
91
|
+
call): `--fix-unique-identifier` wires the package's dangling
|
|
92
|
+
unique-identifier id onto the first `dc:identifier` start tag
|
|
93
|
+
(`unique_identifier_wired`), with the masked-NCX coordination: the uid
|
|
94
|
+
sync predicts the post-wiring uid before the loop (the same pure
|
|
95
|
+
function on the same input region, never a snapshot of a different
|
|
96
|
+
layout), so one run both wires the OPF and aligns the NCX dtb:uid the
|
|
97
|
+
broken wiring was masking. Refusals: absent/empty attribute, non-NCName
|
|
98
|
+
id, already-wired, no dc:identifier, first-identifier-has-id; normal
|
|
99
|
+
`gate`. The CSS-007 font half shipped the same release inside
|
|
100
|
+
`--fix-media-types` (font/* blessed on both package versions, magic
|
|
101
|
+
confirmed, `.eot` never touched) with the flag accepted under `no_worse`
|
|
102
|
+
(its gain sits on the INFO axis).
|
|
73
103
|
They alter markup structure or fabricate minimal content. v0.14–v0.16 ran these unconditionally, which broke this rule;
|
|
74
104
|
v0.17.0 restored it. `--unwrap-illegal-tags` additionally protects any illegal-tag
|
|
75
105
|
name that an EPUB stylesheet styles as an element selector (`css_protected_tags`,
|
|
@@ -79,7 +109,12 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
|
|
|
79
109
|
no version can be read, since their target defects only exist below EPUB 3; the
|
|
80
110
|
attribute scrub is anchored to real start tags via
|
|
81
111
|
`transforms.strip_attrs_in_start_tags` (prose mentioning the attributes and
|
|
82
|
-
CDATA/comment content are preserved)
|
|
112
|
+
CDATA/comment content are preserved); since v0.55.0 (issue #11) the set also
|
|
113
|
+
covers `role` and `aria-hidden` (the 2026-10-25 census: the dominant
|
|
114
|
+
EPUB3-semantics-on-legacy-package family, DPUB-ARIA doc-* values only; the
|
|
115
|
+
package-version gate stays, verified aligned with epubcheck's profile-driven
|
|
116
|
+
rejection; `hidden` deliberately NOT covered, stripping it would reveal
|
|
117
|
+
hidden content), and `strip_invalid_value` matches the
|
|
83
118
|
attribute name with a `(?<![\w:.-])value` lookbehind so `data-value` survives.
|
|
84
119
|
* **Lossy strips** (`--strip-pagination`, `--strip-broken-tags`,
|
|
85
120
|
`--strip-watermarks`; pagination.py, watermark.py; `--strip-stub-docs`,
|
|
@@ -132,19 +167,19 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
|
|
|
132
167
|
|
|
133
168
|
## Layout
|
|
134
169
|
|
|
135
|
-
- `src/bindery/transforms.py`: pure `str -> (str, int)` text transforms (including `strip_broken_tags`). Since v0.32.0: `strip_attrs_in_start_tags` anchors attribute edits on the quote-aware start-tag matcher, `_outside_protected` forwards arguments (context-carrying transforms can be decorated), `fix_id_colons` matches only the bare `id` attribute and only internal fragments (`fix_ncx_src_fragments` carries renames into the NCX), `fix_ncx_playorder` is anchored to `<navPoint>` start tags, and the css selector boundary covers namespaced and functional selector forms. HAZARD (the 0.41.0 spin): the quote-aware tag-prefix alternation (double-quoted | single-quoted | [^>]) OVERLAPS ([^>] matches quotes too), so a plain `*` re-partitions exponentially on failing candidates; every such greedy loop must carry the possessive `*+` (re 3.11+, the plugin's minimum Calibre). Since v0.42.0 the census is complete and pinned by tests/test_matcher_hardening.py: every greedy loop of this family is possessive (the three 0.41.1 rewrites plus _START_TAG_RE, _COVER_META_RE, _GUIDE_REF_RE, prune_dangling_edges' spine|item|itemref matcher, _ITEM_TAG_RE, _SPINE_ITEM_RE; possessive is byte-identical on success because the loop is always followed by a literal `>` the [^>] branch cannot consume), and the five LAZY siblings (_VOID_RE, the transforms <img> matcher, and epub's link/a/img matchers) use the disjoint catch-all `[^>"']` instead: mutually exclusive branches cannot re-partition, and an unterminated quote inside a tag now fails to match rather than pairing across markup. NEW tag matchers: quote-aware possessive for greedy loops, the disjoint catch-all for lazy ones, and the tempered-quote idiom (["'])((?:(?!\1).)*)\1 for attribute values. Since v0.37.0: `fix_ncx_playorder` strips quotes before its already-correct comparison (regex groups carry quote characters; comparing quoted values against bare numbers counted phantom fixes on every sequential NCX and rewrote single-quoted correct attributes to double quotes). Since v0.54.0 (the 2026-10-24 surgery): `fix_ncx_playorder` is CONTRACT-GATED -- it tokenizes navPoint/navTarget/pageTarget start tags plus their `<content src>` targets, returns unchanged when the NCX already satisfies epubcheck's playOrder contract (same target shares one value, distinct targets distinct values, values contiguous 1..K), and re-sequences by first-appearance target order only on a violation; the old unconditional 1..N renumbering rewrote valid TOCs into violations (211 of 579 firings gate-rejected, 17 real fixes, 347 no-ops). The same fix extends `_PROTECTED_RE` with the Calibre `/*<![CDATA[*/ ... /*]]>*/` wrapper as one protected unit (the bare CDATA alternative ends inside the closing `/*]]>`, which exposed the trailing `*/` to strip_prolog_junk's amputation: 111 phantom-fix rejections).
|
|
170
|
+
- `src/bindery/transforms.py`: pure `str -> (str, int)` text transforms (including `strip_broken_tags`). Since v0.55.0 `fix_svg_dup_ids` and `fix_duplicate_ids` share `_rename_duplicate_ids` (first-keeps-name, `_2`/`_3` suffixes, references never rewritten); the SVG flag owns standalone `.svg` entries, the content-doc flag owns `.xhtml`/`.html`/`.htm`/`.xml`. Since v0.32.0: `strip_attrs_in_start_tags` anchors attribute edits on the quote-aware start-tag matcher, `_outside_protected` forwards arguments (context-carrying transforms can be decorated), `fix_id_colons` matches only the bare `id` attribute and only internal fragments (`fix_ncx_src_fragments` carries renames into the NCX), `fix_ncx_playorder` is anchored to `<navPoint>` start tags, and the css selector boundary covers namespaced and functional selector forms. HAZARD (the 0.41.0 spin): the quote-aware tag-prefix alternation (double-quoted | single-quoted | [^>]) OVERLAPS ([^>] matches quotes too), so a plain `*` re-partitions exponentially on failing candidates; every such greedy loop must carry the possessive `*+` (re 3.11+, the plugin's minimum Calibre). Since v0.42.0 the census is complete and pinned by tests/test_matcher_hardening.py: every greedy loop of this family is possessive (the three 0.41.1 rewrites plus _START_TAG_RE, _COVER_META_RE, _GUIDE_REF_RE, prune_dangling_edges' spine|item|itemref matcher, _ITEM_TAG_RE, _SPINE_ITEM_RE; possessive is byte-identical on success because the loop is always followed by a literal `>` the [^>] branch cannot consume), and the five LAZY siblings (_VOID_RE, the transforms <img> matcher, and epub's link/a/img matchers) use the disjoint catch-all `[^>"']` instead: mutually exclusive branches cannot re-partition, and an unterminated quote inside a tag now fails to match rather than pairing across markup. NEW tag matchers: quote-aware possessive for greedy loops, the disjoint catch-all for lazy ones, and the tempered-quote idiom (["'])((?:(?!\1).)*)\1 for attribute values. Since v0.37.0: `fix_ncx_playorder` strips quotes before its already-correct comparison (regex groups carry quote characters; comparing quoted values against bare numbers counted phantom fixes on every sequential NCX and rewrote single-quoted correct attributes to double quotes). Since v0.54.0 (the 2026-10-24 surgery): `fix_ncx_playorder` is CONTRACT-GATED -- it tokenizes navPoint/navTarget/pageTarget start tags plus their `<content src>` targets, returns unchanged when the NCX already satisfies epubcheck's playOrder contract (same target shares one value, distinct targets distinct values, values contiguous 1..K), and re-sequences by first-appearance target order only on a violation; the old unconditional 1..N renumbering rewrote valid TOCs into violations (211 of 579 firings gate-rejected, 17 real fixes, 347 no-ops). The same fix extends `_PROTECTED_RE` with the Calibre `/*<![CDATA[*/ ... /*]]>*/` wrapper as one protected unit (the bare CDATA alternative ends inside the closing `/*]]>`, which exposed the trailing `*/` to strip_prolog_junk's amputation: 111 phantom-fix rejections).
|
|
136
171
|
- `src/bindery/pagination.py`: the opt-in lossy page-number strip (runhead detection, page-layer decision, block-centric removal/merge, safety nets).
|
|
137
172
|
- `src/bindery/watermark.py`: the opt-in lossy watermark strip (anchored and anchorless signature removal).
|
|
138
173
|
- `src/bindery/reserialize.py`: structural repair via `html5lib`.
|
|
139
174
|
- `src/bindery/audit.py`: read-only body-text audits (`content`, `pagenumbers`, `emptytext`, `ocr`, `monolithic` since v0.21.0 with `--max-doc-chars N`, and `completeness` since v0.36.0) behind the `audit` subcommand (v0.15.0). Since v0.18.0 library mode resolves EPUB paths through `cquarry.get_format_path()` and can apply a tag to flagged books via `cquarry.write.WritableCalibreDB` (only with explicit `--tag`; the only sanctioned write path). Since v0.19.0 `audit --id BOOK_ID` audits a single library book through cquarry's single-entity `get_book()` fetch (no library-wide cache; supports `--tag`; incompatible with the directory argument; comma lists since v0.23.0). Since v0.22.0 every archive entry is fully read (CRC + decompression) before analysis (a damaged entry reports its own CORRUPT verdict instead of feeding emptytext) and manifest/NCX references to absent files classify as `convention` (bloated ToC, consecutive span) or `fragment` (broken span). Hazard: the scan loop reuses `tag` as its per-book display column, so the `--tag` argument is captured as `audit_tag` at function entry; do not collapse them again. Since v0.29.0 `audit --json FILE` writes per-file analyzer verdicts in the `library --json` shape (directory, library, and single-book modes; with `--id`, exactly one id; emptytext omitted from a record when the archive verdict owns the body-text story). Since v0.44.0 two more advisory analyzers ride the single pass: `cover` (the EPUB3 `properties~="cover-image"` slice of the ruled hybrid: dangling EPUB2 meta, EPUB3 declaration, cover-file presence) and `tocdrift` (the NCX navMap vs EPUB3 nav toc structural diff, fragments included; audit-only, ToC synthesis out of charter permanently); the loader now exposes the parsed OPF, the NCX text, and the nav HTML on `Book`. Since v0.50.0 a ninth analyzer, `contamination` (issue #8, the judge-and-import class, advisory by contract like completeness/cover/tocdrift, never moves the exit code): foreign-title template stamps (head `<title>` repeats or a volume/book marker with named words in front of it, judged against the OPF's own title/creator/publisher/series with a marker-stripped comparison; a part label whose marker leads never flags), fan-bundle ad signatures (jnovels, kokoro connect, zerobooks, a light-novel plug), and review-paste fingerprints (strong flags alone, weak needs corroboration) in the first five content docs; its corpus is spine PLUS `Book.extra_docs` (the archive's non-spine html, retained by the loader in archive order under a 64 MiB cap, because a broken manifest can leave the junk entirely off the spine: Hothouse). Since v0.50.0 `run_directory`/`run_single` dispatch analyzers through `_dir_dispatch` (one entry per ALL key; the old if/elif chain's `else` ran ocr for cover/tocdrift in those modes since v0.44.0) and `tests/test_contamination.py` pins the map's key set to ALL. Since v0.36.0 the completeness analyzer (Phase 15) reports the phase-1 spot-check per book: spine/prose-doc counts (prose = >= 400 visible chars), first/middle/last prose-doc opening/closing excerpts, trailing-ToC classification (`_trailing_toc`: short link lines dominate, no paragraph-length blocks), and the unreadable fraction. It is ADVISORY by contract (problem always False; ADVISORY on a trailing ToC or unreadable fraction >= 10%), never moves the exit code, and is skipped like emptytext when the archive verdict owns the book. Since v0.38.0 the archive verdict distinguishes font obfuscation from DRM (`OBFUSCATION_ALGOS`: IDPF 2008/embedding + both Adobe URIs; readable obfuscated entries are the benign OBFUSCATED advisory, an unreadable one is CORRUPT, and only non-obfuscation algorithms give ENCRYPTED). Since v0.43.0 untrusted metadata XML is parsed through `_safe_xml_parse` (DOCTYPE/ENTITY declarations refused before xml.etree sees them; the stdlib shape of defusedxml's entity protection) and single entries above `_MAX_ENTRY_BYTES` (32 MB) are refused before decompression; a refused metadata file returns the shell book. Since v0.40.0 the vir-tui console renderer loads through the module-level `_LazyUI` proxy (a plain `import vir_tui as ui` at module level would make 3.12/3.13 installs unimportable now that the stack is marker-gated; LOAD_GLOBAL never consults module `__getattr__`, so a proxy object is the lazy shape that works).
|
|
140
|
-
- `src/bindery/epub.py`: archive rewrite, NCX uid sync, RepairReport, mismatch detection, and the opt-in structural-repair plumbing (including the CSS precondition scan). Since v0.52.0 (issue #10): `spine_dangles(src)` walks itemref -> manifest item -> archive entry through `_resolve_href`+`_norm_path` (None on an unreadable package: the veto needs evidence, never fires on a probe failure), and `prune_missing_manifest_items` derives its spine protection from the text it prunes (no snapshot parameter: fix_manifest_ids renames ids ahead of it, and the existence set it checks against must be the layout its hrefs are written in — `present_src` when space renames are active, matching the content-doc prune). Since v0.46.0 the opt-in selection is the `RepairFlags` dataclass (one field per flag, all off by default; the all-off instance IS the default pass): `repair_epub(src, dst, flags)` and `process_book(..., flags)` take one object instead of a ~25-kwarg signature, the EPUB2-targeted fixes' version gate swaps in a `dataclasses.replace` copy (never mutating the caller's instance), and the plugin's bare `repair_epub(src, dst)` call constructs exactly the all-off default pass. Since v0.38.0/v0.39.0 (Phase 16): `generate_container` + the `--fix-container` gateway (insert after mimetype when missing, replace in-loop when stale; constant-epoch timestamp keeps bytes deterministic), `prune_dangling_edges` (prune_missing_manifest_items returns the pruned ids and the edges that pointed at them (spine@toc, media-overlay, fallback, EPUB2 cover meta) are rewritten so pruning cannot manufacture the regression the gate rejects), and `fix_manifest_media_types` (extension map + magic-byte confirmation via a `peek` callable over the open source zip; jpg/png/gif only), and `fix_cover_meta` (the ruling's deterministic half: dangling EPUB2 cover meta re-pointed from the guide's cover reference when exactly one manifest item carries that file, removed otherwise; accepted under no_worse like the lossy strips because cover wiring is invisible to epubcheck).
|
|
141
|
-
- `src/bindery/validate.py`: epubcheck wrapper, the `gate` (improvement) and `no_worse` (no-regression, for the lossy strips) acceptance bars.
|
|
175
|
+
- `src/bindery/epub.py`: archive rewrite, NCX uid sync, RepairReport, mismatch detection, and the opt-in structural-repair plumbing (including the CSS precondition scan). Since v0.52.0 (issue #10): `spine_dangles(src)` walks itemref -> manifest item -> archive entry through `_resolve_href`+`_norm_path` (None on an unreadable package: the veto needs evidence, never fires on a probe failure), and `prune_missing_manifest_items` derives its spine protection from the text it prunes (no snapshot parameter: fix_manifest_ids renames ids ahead of it, and the existence set it checks against must be the layout its hrefs are written in — `present_src` when space renames are active, matching the content-doc prune). Since v0.46.0 the opt-in selection is the `RepairFlags` dataclass (one field per flag, all off by default; the all-off instance IS the default pass): `repair_epub(src, dst, flags)` and `process_book(..., flags)` take one object instead of a ~25-kwarg signature, the EPUB2-targeted fixes' version gate swaps in a `dataclasses.replace` copy (never mutating the caller's instance), and the plugin's bare `repair_epub(src, dst)` call constructs exactly the all-off default pass. Since v0.55.0 (issue #16) the RSC-008 half lives beside the prune: `collect_undeclared_resources` (one whole-book census: content img/link, inline `<style>` url(), stylesheet url(), OPF guide refs) + `declare_missing_manifest_items` (sorted `bindery-dN` items before `</manifest>`, `_DECLARE_MEDIA_TYPES` blessed extensions only), gated and threaded through `RepairFlags.declare_missing`. Since v0.38.0/v0.39.0 (Phase 16): `generate_container` + the `--fix-container` gateway (insert after mimetype when missing, replace in-loop when stale; constant-epoch timestamp keeps bytes deterministic), `prune_dangling_edges` (prune_missing_manifest_items returns the pruned ids and the edges that pointed at them (spine@toc, media-overlay, fallback, EPUB2 cover meta) are rewritten so pruning cannot manufacture the regression the gate rejects), and `fix_manifest_media_types` (extension map + magic-byte confirmation via a `peek` callable over the open source zip; jpg/png/gif only), and `fix_cover_meta` (the ruling's deterministic half: dangling EPUB2 cover meta re-pointed from the guide's cover reference when exactly one manifest item carries that file, removed otherwise; accepted under no_worse like the lossy strips because cover wiring is invisible to epubcheck).
|
|
176
|
+
- `src/bindery/validate.py`: epubcheck wrapper, the `gate` (improvement) and `no_worse` (no-regression, for the lossy strips) acceptance bars. Since v0.55.0 (issue #15) the default oracle budget is size-scaled (`oracle_timeout_for`: 300s at or under 10 MB, +60s per started 10 MB above, 1800s cap), one subprocess retry follows a timed-out attempt (the daemon path's subprocess fallback is that attempt), and only a double timeout raises `OracleTimeout`, which the gate surfaces as `(epubcheck timeout)` instead of `(epubcheck failed)` (the sweep selectors catch it and keep the book a candidate).
|
|
142
177
|
- `src/bindery/library.py`: Calibre walk, atomic replace, backups, and native format installation. Since v0.19.0 `CalibreIdResolver` resolves the book id from `metadata.db` through cquarry (one lazy path→id map per run; since v0.20.0 the map comes from `CalibreDB.format_path_index()`, re-normalized with `resolve().lower()` for the resolver's symlinked-directory and case-insensitive matching); the `(id)` directory-name regex is only the no-catalog fallback. Since v0.24.0 `install_format()` places the repaired file atomically and updates the `data` row through `cquarry.write.WritableCalibreDB` (since v0.31.0 via `set_format`, cquarry 1.17's sanctioned remove+add in one transaction); the external `calibredb` CLI is gone. Since v0.32.0 a guessed id drives a row update only when metadata.db corroborates it (`_verify_guess`: books row exists, file in the book's own directory, catalogued `data.name` when a row exists); anything else saves in place and leaves the catalog untouched: never reintroduce directory-name guessing as the primary source: renamed/mismatched directories would replace the wrong book. Since 2026-09-15 `make_backup` takes the opt-in `--backup-keep N` ring: at most N backup files per book, the author original `.bak` never deleted, newest content at the highest name; default remains an unbounded rotation.
|
|
143
|
-
- `src/bindery/cli.py`: `repair` and `library` subcommands, including `--all` and `--install-to-calibre`; since v0.52.0 `process_book` carries the spine-resolution veto (issue #10) above every verdict path, promotions included: a candidate whose spine dangles more than the source's is `reject` with `(spine collapse)` appended to the summary, whatever the counts say. `repair --json` emits the library per-book record vocabulary (2026-09-15; since v0.45.0 every per-book record, the phase1 payload included, also carries the structured fix breakdown: the `fixes` dict plus `ncx_uid_synced`/`watermark_refusals` off the `Outcome`, and `_phase1_decisions` reads that data instead of substring-matching the rendered `summary`, which stays for human eyes; CalibreQuarry's lossy-consent mirror is the waiting consumer, floor `bindery>=0.45.0`); since v0.29.0 also the `run` verbs: `phase1` (pre-import vetting over loose files: the audit battery composed with one gated repair sweep; read-only until `--apply-lossy`, which IS the lossy-strip consent) and `phase3` (the scoped post-import apply step: `library --id --sweep --only all --apply --all --install-to-calibre` plus a pre/post summary; mechanically refuses unscoped sweeps with exit 2; since 2026-09-15 the summary sums applied/equal states only and carries rejected projections as `rejected_projection`). Plus `bindery doctor` (2026-09-15): the environment self-check that imports none of the VirInvictus stack, never raises, and always exits 0. Both run verbs take `--non-interactive` and surface open questions as `decisions_needed` in their JSON, never prompts. They drive the shipped `run_library` through the real parser, so no flag can drift between the verb and the subcommand it wraps. Since v0.46.0 `cli._flags_from_args(args)` is the one place the argparse names map onto `RepairFlags` fields (each `or --all`), shared by `run_library` and `run_repair`; the old per-verb ~25-kwarg blocks are gone. Since v0.47.1 the help is two-level (the --help-repairs reference is
|
|
178
|
+
- `src/bindery/cli.py`: `repair` and `library` subcommands, including `--all` and `--install-to-calibre`; since v0.52.0 `process_book` carries the spine-resolution veto (issue #10) above every verdict path, promotions included: a candidate whose spine dangles more than the source's is `reject` with `(spine collapse)` appended to the summary, whatever the counts say. `repair --json` emits the library per-book record vocabulary (2026-09-15; since v0.45.0 every per-book record, the phase1 payload included, also carries the structured fix breakdown: the `fixes` dict plus `ncx_uid_synced`/`watermark_refusals` off the `Outcome`, and `_phase1_decisions` reads that data instead of substring-matching the rendered `summary`, which stays for human eyes; CalibreQuarry's lossy-consent mirror is the waiting consumer, floor `bindery>=0.45.0`); since v0.29.0 also the `run` verbs: `phase1` (pre-import vetting over loose files: the audit battery composed with one gated repair sweep; read-only until `--apply-lossy`, which IS the lossy-strip consent) and `phase3` (the scoped post-import apply step: `library --id --sweep --only all --apply --all --install-to-calibre` plus a pre/post summary; mechanically refuses unscoped sweeps with exit 2; since 2026-09-15 the summary sums applied/equal states only and carries rejected projections as `rejected_projection`). Plus `bindery doctor` (2026-09-15): the environment self-check that imports none of the VirInvictus stack, never raises, and always exits 0. The `observe` verb (v0.53.0) writes schema v2 artifacts since v0.55.0 (issue #13): `codes.messages` (renamed from the mislabeled `occurrences`) counts aggregated epubcheck JSON message entries, `codes.occurrences` now carries the true per-site scale (summed `locations`), a `counting` block states both scales in the artifact, and `--against` accepts v1 or v2 (the trend keys exist in both). Both run verbs take `--non-interactive` and surface open questions as `decisions_needed` in their JSON, never prompts. They drive the shipped `run_library` through the real parser, so no flag can drift between the verb and the subcommand it wraps. Since v0.46.0 `cli._flags_from_args(args)` is the one place the argparse names map onto `RepairFlags` fields (each `or --all`), shared by `run_library` and `run_repair`; the old per-verb ~25-kwarg blocks are gone. Since v0.47.1 the help is two-level (the --help-repairs reference is
|
|
144
179
|
custom-rendered and colors itself with the CPython 3.14 argparse theme,
|
|
145
180
|
gated by argparse's own _colorize detection so the levels always agree) and the repair-flag inventory is one table: `_REPAIR_FLAG_TABLE` (dest + one-line short help + long reference text) registers the compact grouped `--help` (argparse argument groups, hand-short usage line) and `--help-repairs` renders the long descriptions (a custom argparse Action, deliberately NOT store_true so the wiring test's selection count ignores it); adding a flag means adding a table row. Since v0.48.0 the metadata.db doors are enforced: `_require_closed_calibre` (pgrep fail-closed, exit 1) guards `audit --tag` and `library --apply --install-to-calibre` (run phase3 transitively), and `--help-json` (a custom argparse Action like --help-repairs, never colored) dumps the per-verb exit contract and the door list; since v0.48.1 the per-verb exit contracts tell the code's truth (repair: 0 probed/written incl. partial / 1 trouble, never 2; audit: 0/1/2 plus 3 for findings+write-failure; library and the run slices: the unified 0/2/1); since v0.49.0 the single-file verb's no-output form is a probe (the gate runs, the record's `output` is null and `applied` false, nothing is written anywhere; the old implicit `(repaired).epub` sibling was issue #7's import litter in a vetted tree, and writing now needs an explicit DST). `library --workers N` (v0.30.0) parallelizes only the sweep's candidate pass, in windows of N consumed in order so `--limit` stays lazy; the repair phase is never parallel (shared workdir, atomic-replacement contract). Since v0.40.0 `main()` guards the VirInvictus stack: a `ModuleNotFoundError` naming `vir_tui`/`cquarry` (marker-gated to 3.14+ on the 3.12 floor) becomes one stderr line and trouble exit 2 instead of a traceback; anything else missing still raises.
|
|
146
181
|
- `plugin/__init__.py` + `scripts/build_plugin.py`: the Bindery Repair Calibre plugin (v0.37.0; identity `Bindery Repair` / `bindery_repair`, approved and built per the scoped spec in roadmap.md Phase 3). The builder vendors `transforms.py`, `epub.py`, `pagination.py`, `watermark.py`, `reserialize.py` byte-identically into the zip (the zip root is a package, so their relative imports resolve unchanged; the suite's drift test pins the equality) with the version tuple substituted from the single-source VERSION; `publish.yml` attaches `BinderyRepair-v<VERSION>.zip` to each release. The plugin runs ONLY the default pass (epubcheck cannot gate inside Calibre), never raises, is byte-idempotent (zero fixes returns the original path), testzip()-verifies its output, logs one line per book, and takes JSON config via `site_customization` (`log`, `log_path`, `max_size_mb` default 150, `max_log_mb` rotation cap default 2, `epubcheck_path` experimental validation default OFF). Load it in tests through the `tests/test_plugin.py` stubs: the calibre namespace is installed in sys.modules and the package loads with a two-dir `__path__` (plugin dir first, then src/bindery) exactly like calibre's zipplugin loader shape. **Plugin/interpreter compatibility (v0.40.0):** v0.39.0 shipped PEP 758 unparenthesized except tuples, and the plugin died with a bare SyntaxError on every released Calibre (7.x and the whole 8.x series embed Python 3.11, 9.0+ embeds 3.14; from calibre's bypy sources.json, checked against the upstream clone). The core is now grammar-pinned: parenthesized except tuples everywhere, `ruff`'s `target-version = "py312"` pin (inference from requires-python would rewrite the parens back off), `tests/test_version.py`'s parse guard, and CI's `plugin-compat` job byte-compiling the built zip under 3.11/3.12/3.13/3.14. `minimum_calibre_version` is (7, 0, 0): the oldest series whose interpreter CI covers, replacing the hollow (2, 0, 0); `supported_platforms` stays `["linux"]` (developed and tested on Linux only). Any new syntax in the vendored modules must clear the guard before it ships.
|
|
147
|
-
- `tests/`: transforms, end-to-end repair, atomic replace, pagination, watermarks, the audit analyzers (content/pagenumbers/emptytext/ocr/monolithic/completeness, archive integrity, spine classification, contamination since v0.50.0), the library sweep and `CalibreIdResolver`, validate, reserialize, CLI wiring, the version pin and the syntax-floor grammar guard, and the Calibre plugin (build drift + run behavior under calibre stubs). Since the v0.46.0 follow-ups: `tests/test_flags_wiring.py` reflectively pins the RepairFlags inventory to the argparse dests and `_flags_from_args` mapping (a flag added without a field, a field without a flag, or a remapped line fails there), the all-off defaults, and the version gate's caller-immutability contract (test_epub's `TestVersionGateKeepsFlagsIntact`); `tests/test_comment_hyphens.py` carries the RSC-016 fix; `tests/test_flags_real_core.py` drives every repair flag from argv through the unmocked core (--no-validate; the gate itself stays pinned by the mocked-oracle tests and runs for real wherever epubcheck exists). `tests/test_contamination.py` (v0.50.0) carries the three contamination sweeps, the advisory contract, the `_dir_dispatch` key pin, and the cover-routing regression. The v0.46.0 trio (test_flags_wiring, test_comment_hyphens, test_flags_real_core) rides CI's core-compat 3.12/3.13 leg; the hand-listed module list there must gain an entry for every new stack-free test module (audit-side test modules like test_contamination stay on the 3.14 leg by convention).
|
|
182
|
+
- `tests/`: transforms (including the shared duplicate-id rename), end-to-end repair, atomic replace, pagination, watermarks, the audit analyzers (content/pagenumbers/emptytext/ocr/monolithic/completeness, archive integrity, spine classification, contamination since v0.50.0), the library sweep and `CalibreIdResolver`, validate, reserialize, CLI wiring, the version pin and the syntax-floor grammar guard, and the Calibre plugin (build drift + run behavior under calibre stubs). Since the v0.46.0 follow-ups: `tests/test_flags_wiring.py` reflectively pins the RepairFlags inventory to the argparse dests and `_flags_from_args` mapping (a flag added without a field, a field without a flag, or a remapped line fails there), the all-off defaults, and the version gate's caller-immutability contract (test_epub's `TestVersionGateKeepsFlagsIntact`); `tests/test_comment_hyphens.py` carries the RSC-016 fix; `tests/test_flags_real_core.py` drives every repair flag from argv through the unmocked core (--no-validate; the gate itself stays pinned by the mocked-oracle tests and runs for real wherever epubcheck exists). `tests/test_contamination.py` (v0.50.0) carries the three contamination sweeps, the advisory contract, the `_dir_dispatch` key pin, and the cover-routing regression. The v0.46.0 trio (test_flags_wiring, test_comment_hyphens, test_flags_real_core) rides CI's core-compat 3.12/3.13 leg; the hand-listed module list there must gain an entry for every new stack-free test module (audit-side test modules like test_contamination stay on the 3.14 leg by convention). The v0.55.0 batch (issues #11-#16): `tests/test_validate.py` pins the size-scaled timeout and the retry-then-OracleTimeout contract, `tests/test_observe.py` the v2 line format (`CODE:messages:occurrences`), the renamed codes block, the counting envelope, and v1 `--against` acceptance, `tests/test_epub.py::TestDeclareMissingItems` the RSC-008 declarer (sorted deterministic items, blessed media types only, id collisions prefixed, schemes/fragments/spaces/declared skipped), `tests/test_transforms.py::TestFixDuplicateIds` the content-doc rename, and `tests/test_flags_real_core.py` drives `--fix-duplicate-ids`, `--declare-missing-items`, and the extended `--strip-epub3-attrs` set through the real argv-to-fix path.
|
|
148
183
|
|
|
149
184
|
## Conventions
|
|
150
185
|
|
|
@@ -539,3 +539,139 @@ Evidence: specimens and probe records under
|
|
|
539
539
|
pkg010/), census artifacts under `~/docs/Calibre Library.backups/observe/`,
|
|
540
540
|
the dry-run JSON under the library's `.claude/`. All measurement was
|
|
541
541
|
read-only; the library was never written during this research.
|
|
542
|
+
|
|
543
|
+
## The un-researched top-ten lanes (2026-10-25 session, issues #16/#17)
|
|
544
|
+
|
|
545
|
+
Each lane: a message-family census off the baseline artifact's books array, a
|
|
546
|
+
mechanism verdict from live epubcheck 5.3.0 runs and zip reads, and a
|
|
547
|
+
disposition. All measurement read-only; specimen gates ran on /tmp copies.
|
|
548
|
+
|
|
549
|
+
### RSC-001 and RSC-008: the prune's coverage boundary (issue #16)
|
|
550
|
+
|
|
551
|
+
epubcheck 5.3.0 wordings, from real runs: RSC-001 `File "<name>" could not
|
|
552
|
+
be found.` (package-level, no line); RSC-008 `Referenced resource "<name>"
|
|
553
|
+
is not declared in the OPF manifest.` (at the referencing element).
|
|
554
|
+
|
|
555
|
+
- **RSC-001 (39 books / 1,546 entries)**: every sampled finding is rooted in
|
|
556
|
+
an OPF `<item href>`; the flag's outcome splits exactly on the spine
|
|
557
|
+
guard. Non-spine absent-file items (stray images, unspined docs) are
|
|
558
|
+
cleared by the existing prune (25/25 sampled long-tail findings). Spine
|
|
559
|
+
documents with absent files survive by design, and they ARE the census:
|
|
560
|
+
738+736 of the 1,546 entries (95.3%) are the two Wandering Inn books
|
|
561
|
+
(plus the facility's Light at the End, 8/8 spine), where the missing
|
|
562
|
+
"file" exists under a different spelling (raw spaces vs underscores) --
|
|
563
|
+
`--encode-url-spaces` territory, not prune territory. The itemref-with-no-
|
|
564
|
+
`<item>` shape contributed 0 findings in 14 runs. A second finding rides
|
|
565
|
+
along: both Wandering Inn OPFs are prefix-namespaced (`<opf:item>`), a
|
|
566
|
+
shape `_ITEM_TAG_RE` never matches, so the prune could not see them even
|
|
567
|
+
without the spine guard; every finding there is spine-protected anyway.
|
|
568
|
+
**Disposition: coverage boundary documented; no code change.**
|
|
569
|
+
- **RSC-008 (55 books / 545 entries)**: zero coverage before v0.55.0,
|
|
570
|
+
structurally: the prune's predicate is target-absent, and an undeclared
|
|
571
|
+
resource is by definition present. Sample tally (452 findings across 7
|
|
572
|
+
books): 441 content `<img src>` to present-but-undeclared files, 8
|
|
573
|
+
content `<link href>` stylesheets (the one-corpse Calibre split-CSS
|
|
574
|
+
shape), 2 inline `<style>` `@font-face url()` fonts, 1 OPF guide
|
|
575
|
+
`<reference>` to an undeclared cover doc. The most dangerous shape (a
|
|
576
|
+
spine doc missing from the manifest) does not occur.
|
|
577
|
+
**Disposition: new `--declare-missing-items` flag (v0.55.0)**, the
|
|
578
|
+
inverse move: add the manifest item (sorted `bindery-dN` ids, blessed
|
|
579
|
+
extension-map media types, no itemref ever written). Live gate on the
|
|
580
|
+
facility specimen: 3 items declared (the three split CSS files), 16e ->
|
|
581
|
+
13e, RSC-008 fully cleared, accept; the RSC-001 carrier control
|
|
582
|
+
correctly fired nothing; second run declares nothing.
|
|
583
|
+
|
|
584
|
+
### RSC-012: already covered except the OPF guide (issue #17 lane)
|
|
585
|
+
|
|
586
|
+
Verbatim (5.3.0): `Fragment identifier is not defined.` (ERROR) -- not the
|
|
587
|
+
"wrong element" wording strip_broken_anchors_doc's docstring guessed at;
|
|
588
|
+
the docstring is corrected in code separately if it ever matters. Census:
|
|
589
|
+
188 books, exactly 1 aggregated message each. Full-lane live sweep (all 188
|
|
590
|
+
carriers re-probed): 1,210 locations -- 934 content-doc `<a href>` (Mobipocket
|
|
591
|
+
filepos anchors, URL-encoded junk fragments), 234 NCX `<content src>`, 42
|
|
592
|
+
OPF guide `<reference href>` in 33 books. Every location resolved to a
|
|
593
|
+
target document that exists and lacks the fragment -- exactly the
|
|
594
|
+
strip-broken-anchors precondition (a missing target file is a different
|
|
595
|
+
code). **Disposition: existing flag.** `--strip-broken-anchors` clears the
|
|
596
|
+
content + NCX halves in principle (96.5% of locations; 155 books fully);
|
|
597
|
+
the 42 OPF-guide locations survive (no repair touches `<guide><reference>`
|
|
598
|
+
except fix_cover_meta's cover meta, and the guide ref is furniture). Live
|
|
599
|
+
probe on the facility's cold-days specimen (its one finding IS the guide
|
|
600
|
+
shape): the anchor flag correctly fires nothing.
|
|
601
|
+
|
|
602
|
+
### OPF-029: one family, mostly one flag (issue #17 lane)
|
|
603
|
+
|
|
604
|
+
Verbatim: `The file "<path>" does not appear to match the media type
|
|
605
|
+
<declared>, as specified in the OPF file.` (ERROR). Census: 17 books, 280
|
|
606
|
+
entries, all classified by declared type vs magic bytes: 216 locations are
|
|
607
|
+
jpg-stamped-image/png with JPEG bytes at offset 0 (`--fix-media-types`'
|
|
608
|
+
exact documented case; 13 books led by Tress 75 and Sunlit Man 68); 64
|
|
609
|
+
survive -- 56 .gif-declared-image/gif with JPEG bytes (Gateway 53,
|
|
610
|
+
Revelation 3: the magic check refuses, and renaming to .jpg is a
|
|
611
|
+
content-name change out of scope), 5 .png-declared-png with JPEG bytes,
|
|
612
|
+
3 jpg-declared-jpeg with PNG bytes (the nevernight case: the declaration
|
|
613
|
+
is honest; the fix deliberately keeps it). **Disposition: existing flag
|
|
614
|
+
for the 216; the 64 are leave-measured.** Live gate on the Tress copy:
|
|
615
|
+
75e -> 0e (fully clean), accept.
|
|
616
|
+
|
|
617
|
+
### OPF-030: the unique-identifier wiring scar (issue #17 lane)
|
|
618
|
+
|
|
619
|
+
Verbatim: `The unique-identifier "<id>" was not found.` (ERROR,
|
|
620
|
+
package-level). Census: 27 books, 1 entry each, all fatals=0 except one.
|
|
621
|
+
Families: 23 dangling-ref -- the package names an id (`bookid`, `uuid_id`,
|
|
622
|
+
`uid`, ISBN shapes) but NO dc:identifier carries any id attribute at all
|
|
623
|
+
(a converter dropped the id attributes wholesale); 2 attr-empty
|
|
624
|
+
(`unique-identifier=""`, rendered by epubcheck as "null"); 2 multi-OPF
|
|
625
|
+
merge leftovers (Either/Or, Book of Debugging: the defect sits in a stale
|
|
626
|
+
non-rootfile OPF the merge left behind). None is the v0.51 ns0 embed-scar.
|
|
627
|
+
**Disposition: leave measured.** The 23 are deterministically fixable in
|
|
628
|
+
principle (add the named id to the first dc:identifier; values untouched,
|
|
629
|
+
gate-visible), but the mass is 27 books at 1 error each with zero reader
|
|
630
|
+
impact, and the honest recipe would need the empty-attr and stray-OPF
|
|
631
|
+
refusals designed around it. The recipe is recorded here for a future
|
|
632
|
+
pass; boxed, not silently declined.
|
|
633
|
+
|
|
634
|
+
### CSS-007: unblessed font media types (issue #17 lane)
|
|
635
|
+
|
|
636
|
+
Verbatim (INFO severity, not an error): `Font-face reference "<url>"
|
|
637
|
+
refers to non-standard font type "<declared>".` Census: 114 books, 509
|
|
638
|
+
entries, population reconciled exactly against unblessed font-ext manifest
|
|
639
|
+
items + data-URI srcs. Mechanism (bytecode-extracted whitelists): epubcheck
|
|
640
|
+
checks the OPF-declared media type of every @font-face src against a
|
|
641
|
+
version-specific blessed set (EPUB2: `font/*`, `application/font*`,
|
|
642
|
+
`application/x-font*`, `application/vnd.ms-opentype`; EPUB3: an exact
|
|
643
|
+
eight-entry list). Sampled families: `text/plain` fonts (Huxley 8,
|
|
644
|
+
Cialdini 12), `application/xfont+otf` (Gottman 38), `application/octet-stream` woff2 + correct-typed EOT (Nakov 32 -- the EOT half is
|
|
645
|
+
unfixable at the manifest layer, its type is CORRECT), the
|
|
646
|
+
`application/application/x-font-ttf` typo (4), `.dat` fonts, `.ttc`
|
|
647
|
+
collections, `data:font/truetype` base64 srcs. bindery's
|
|
648
|
+
`--fix-media-types` is image-only by charter and touches 0/114 carriers.
|
|
649
|
+
**Disposition: candidate extension (a font half with version-aware
|
|
650
|
+
blessed targets and font magic bytes: TTF `00 00 01 00`/`true`, OTTO,
|
|
651
|
+
wOFF, wOF2, ttc), left measured this batch**: INFO mass, gate-invisible
|
|
652
|
+
(nWarning unaffected), version-dependent target types, and a quarter of
|
|
653
|
+
the family unfixable at the manifest layer. Boxed, revisitable.
|
|
654
|
+
|
|
655
|
+
### RSC-004: font obfuscation, not DRM (issue #17 lane)
|
|
656
|
+
|
|
657
|
+
Verbatim (INFO): `File "<path>" is encrypted, its content will not be
|
|
658
|
+
checked.` Census: 84 books, 427 entries == 427 EncryptedData blocks across
|
|
659
|
+
all 84 carriers (per-book counts match 84/84). Algorithms: 424 Adobe
|
|
660
|
+
`ns.adobe.com/pdf/enc#RC` + 3 IDPF `idpf.org/2008/embedding` (one book);
|
|
661
|
+
ZERO other algorithms -- no DRM anywhere in the lane. Encrypted entries
|
|
662
|
+
are fonts only (266 .otf, 137 .ttf, 23 .dat, 1 .ttc), all readable bytes
|
|
663
|
+
(obfuscation, not zip encryption): every sampled book is the audit
|
|
664
|
+
archive verdict's benign OBFUSCATED side, and `audit.py`'s
|
|
665
|
+
OBFUSCATION_ALGOS already covers both URIs. **Disposition: leave
|
|
666
|
+
measured**, cross-referenced: the repair inventory has nothing honest to
|
|
667
|
+
offer (de-obfuscating fonts to satisfy an INFO finding would break the
|
|
668
|
+
embedding rules the obfuscation exists for), and the observatory tracks
|
|
669
|
+
it.
|
|
670
|
+
|
|
671
|
+
### Lane addendum: what the runs changed in the code
|
|
672
|
+
|
|
673
|
+
Issue #11's gate question resolved by census: epubcheck's attribute
|
|
674
|
+
rejection is package-profile-driven, not DOCTYPE-driven (7 EPUB3 packages
|
|
675
|
+
carrying XHTML 1.1 doctypes: zero attribute findings; every role/
|
|
676
|
+
aria-hidden carrier is version="2.0"), so `package_version()` gating is
|
|
677
|
+
correct as built and only the set grew (`role`, `aria-hidden`).
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: bindery-cli
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.56.0
|
|
4
4
|
Summary: Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library.
|
|
5
5
|
Project-URL: Homepage, https://github.com/VirInvictus/bindery-cli
|
|
6
6
|
Project-URL: Repository, https://github.com/VirInvictus/bindery-cli
|
|
@@ -62,17 +62,20 @@ Everything below is off until its flag is passed (or all at once via `--all`). T
|
|
|
62
62
|
- **`--unwrap-block-in-inline`**: safely unwraps `<span>` tags that illegally contain a block-level element (e.g. `<div>` or `<p>`), leaving the block element intact.
|
|
63
63
|
- **`--strip-invalid-value`**: systematically strips invalid `value="..."` attributes from elements like `<div>`, `<span>`, `<p>`, etc.
|
|
64
64
|
- **`--fix-page-map`**: normalizes legacy page-map markup: drops the non-standard `page-map="..."` attribute from the OPF `<spine>` and adds `class="pages"` to classless NCX `<pageList>` elements (epubcheck rejects both on older HarperCollins / Anna's Archive conversions).
|
|
65
|
-
- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or
|
|
65
|
+
- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label`, `role`, `aria-hidden` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or `aria-labelledby` survive; `hidden` stays, since stripping it would reveal hidden content). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
|
|
66
66
|
- **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents (`figure`/`section` to `div`, `figcaption` to `p`), keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
|
|
67
67
|
- **`--unwrap-illegal-tags`**: strips completely invalid or deprecated HTML tags that break EPUB3 validation (`<st>`, `<sentence>`, `<o>`, `<w>`, `<pagebreak>`) while retaining their inner text. Any of those names styled as an *element selector* by an EPUB stylesheet (`.css` entries and inline `<style>` blocks alike; class/id selectors like `.st`/`#w` don't count) is protected for the whole book, guaranteeing format preservation.
|
|
68
|
+
- **`--declare-missing-items`**: the RSC-008 inverse of the prune: adds the manifest `<item>` declarations the OPF never wrote for files the archive carries and the book references (content `<img src>`/`<link href>` targets, CSS `url()` resources, OPF guide references). Deterministic ids, media types from a blessed extension map only, and no `<itemref>` is ever written: the spine is untouched.
|
|
69
|
+
- **`--fix-unique-identifier`**: wires the dangling OPF unique-identifier (OPF-030): adds the id the package names to the first `dc:identifier` when no element carries it, and aligns the NCX `dtb:uid` the broken wiring was masking in the same run. The `dc:` values are never touched; ambiguous shapes refuse.
|
|
68
70
|
- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items (with every package edge that pointed at them rewritten: `spine@toc`, `item@media-overlay`, `item@fallback`, and the EPUB2 cover meta). Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
|
|
69
71
|
- **`--strip-broken-anchors`**: strips href attributes that cannot resolve, keeping the anchor text byte-for-byte: a `#fragment` the target document does not define (RSC-020/RSC-012; NCX navTargets fall back to the document target, so chapter navigation survives) and unresolvable URI schemes (`kindle:`, `file:`).
|
|
70
72
|
- **`--encode-url-spaces`**: percent-encodes raw spaces in `src`/`href` attribute values across the package (OPF manifest, NCX, content documents). A URL with a literal space is not a valid URL (RSC-020) and unresolvable on strict readers; the encoded form denotes the same file. Since v0.44.0 the flag also renames archive entries whose names carry raw spaces to their underscore spellings and rewrites every reference to them (OPF, NCX, content, CSS `url()`), clearing the PKG-010 space-in-filename advisories: underscore, never percent-encoding, because epubcheck decodes references before entry lookup.
|
|
71
73
|
- **`--fix-container`**: generates `META-INF/container.xml` at the located OPF when the container is missing or names a file the archive does not contain. This is the gateway defect: epubcheck stays fatal while the OPF is unfindable, so no repair can be gate-accepted on such a book until the container exists.
|
|
72
|
-
- **`--fix-media-types`**: normalizes wrong manifest media-type declarations (OPF-029, e.g. a jpg stamped `image/png` by an aggregator). Attribute-only and quote-preserving; it fires only when the file's magic bytes confirm the extension, so a misnamed-but-consistent file (a PNG renamed .jpg) is never made worse.
|
|
74
|
+
- **`--fix-media-types`**: normalizes wrong manifest media-type declarations (OPF-029, e.g. a jpg stamped `image/png` by an aggregator). Attribute-only and quote-preserving; it fires only when the file's magic bytes confirm the extension, so a misnamed-but-consistent file (a PNG renamed .jpg) is never made worse. Since v0.56.0 it also covers the CSS-007 font slice (a `ttf`/`otf`/`woff`/`woff2` declared outside epubcheck's blessed font set is rewritten to its `font/*` spelling when the magic bytes confirm; `.eot` is never touched), and the flag is accepted under the no-worse bar because that half's gain sits on the INFO axis.
|
|
73
75
|
- **`--fix-cover`**: repairs dangling EPUB2 cover wiring. A `<meta name="cover">` whose content names no manifest item is re-pointed when the OPF guide's own cover reference names an existing item, and removed when nothing identifies it; EPUB3 `properties="cover-image"` is deliberately audit-only (guessing which image is the cover is not deterministic). Cover wiring is invisible to epubcheck, so cover-only repairs are accepted under the same no-worse bar the lossy strips use.
|
|
74
76
|
- **`--fix-comment-double-hyphen`**: replaces `--` sequences inside XML comments with en-dashes. `--` is illegal inside an XML comment (epubcheck fatal RSC-016: the book opens in lenient readers but never passes epubcheck). The edit stays inside the comment bodies: text nodes and CDATA sections are never touched, and each comment's terminator is left intact.
|
|
75
77
|
- **`--fix-svg-dup-ids`**: renames duplicate `id` values inside standalone `.svg` entries (old calibre SVG page renders repeat glyph ids within one document; the intake wave that found this carried 445 to 1,339 'Duplicate "glNNNN"' errors per book). The first occurrence keeps its name, later ones gain `_2`/`_3`, and internal references are deliberately untouched: they already resolve to the first definition under every reader's first-match lookup.
|
|
78
|
+
- **`--fix-duplicate-ids`**: the same first-keeps-name rename for duplicate `id` values in the content documents themselves (RSC-005 "Duplicate id" on `.xhtml`/`.html` carriers). Every reference to the name still resolves to the first definition, exactly as readers resolved it before.
|
|
76
79
|
- **`--fix-cdata-terminator`**: completes a truncated CDATA-terminator comment in an inline `<style>` block (`/*]]>` becomes `/*]]>*/`). A comment-wrapped CDATA block whose closing `*/` the converter dropped leaves the CSS parser inside an unterminated comment: one CSS-008 'Premature end of file' per affected document. Only the malformed-terminator token class is rewritten; invalid CSS stays unfixed, and stylesheet files are not touched.
|
|
77
80
|
- **`--fix-misnested-inline`**: rewrites the Mobipocket drop-cap mis-nest `<i><b>X</i></b>` to `<i><b>X</b></i>` (em/strong variants and either tag order). Only the both-open-then-reversed-close shape is touched; a punctuation run caught between the reversed closers (`<i><b>xile</i>,</b>`) survives outside both spans, and a letter run there is refused (which span keeps the letters has no deterministic answer).
|
|
78
81
|
- **`--fix-stray-close`**: removes an end tag whose element has no open start tag anywhere above it (the stray extra `</div>` on a shell page, which cascades into 'body must be terminated by the matching end-tag'). Merely mis-nested pairs are never touched: a close is removed only when nothing by that name is open.
|
|
@@ -376,8 +379,16 @@ bindery observe ~/docs/Calibre\ Library --output ~/observe/base.observe
|
|
|
376
379
|
it measures what `--apply` would write, per candidate, and never writes.
|
|
377
380
|
- `--against base.observe` carries a trend block in the new artifact: library
|
|
378
381
|
deltas, error-mass delta, and per-code changes with `new_codes` and
|
|
379
|
-
`resolved_codes`. Artifacts are versioned (`bindery-observe-
|
|
380
|
-
schema mismatch is a usage error, never a silently incomparable trend
|
|
382
|
+
`resolved_codes`. Artifacts are versioned (`bindery-observe-v2`); a
|
|
383
|
+
schema mismatch is a usage error, never a silently incomparable trend
|
|
384
|
+
(v1 artifacts stay valid `--against` targets: the trend keys exist in
|
|
385
|
+
both).
|
|
386
|
+
- Two counting scales, stated in the artifact's `counting` block:
|
|
387
|
+
`codes.messages` (and the per-book `codes` rows) count aggregated
|
|
388
|
+
epubcheck JSON message entries, one per distinct defect text; that is
|
|
389
|
+
the scale the repair gate measures. `codes.occurrences` sums the
|
|
390
|
+
per-message locations, the per-site scale the epubcheck text summary
|
|
391
|
+
reports; one book can differ by orders of magnitude between them.
|
|
381
392
|
- A book the JVM cannot read arrives as a sentinel record instead of being
|
|
382
393
|
dropped, and a run that comes back short of the input count refuses to
|
|
383
394
|
write a partial census. The library and audit walks skip `.caltrash`.
|
|
@@ -35,17 +35,20 @@ Everything below is off until its flag is passed (or all at once via `--all`). T
|
|
|
35
35
|
- **`--unwrap-block-in-inline`**: safely unwraps `<span>` tags that illegally contain a block-level element (e.g. `<div>` or `<p>`), leaving the block element intact.
|
|
36
36
|
- **`--strip-invalid-value`**: systematically strips invalid `value="..."` attributes from elements like `<div>`, `<span>`, `<p>`, etc.
|
|
37
37
|
- **`--fix-page-map`**: normalizes legacy page-map markup: drops the non-standard `page-map="..."` attribute from the OPF `<spine>` and adds `class="pages"` to classless NCX `<pageList>` elements (epubcheck rejects both on older HarperCollins / Anna's Archive conversions).
|
|
38
|
-
- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or
|
|
38
|
+
- **`--strip-epub3-attrs`**: scrubs the EPUB3-only attributes epubcheck rejects on an EPUB2 package: `page-progression-direction`, `epub:type`, `aria-label`, `role`, `aria-hidden` (a fixed, documented set; rendering is unchanged, and lookalikes like `type` or `aria-labelledby` survive; `hidden` stays, since stripping it would reveal hidden content). The edit is anchored to real start tags, so prose that merely mentions `epub:type="chapter"` is preserved.
|
|
39
39
|
- **`--downgrade-epub3-tags`**: downgrades EPUB3/HTML5 semantic elements to their EPUB2 equivalents (`figure`/`section` to `div`, `figcaption` to `p`), keeping existing classes and appending the semantic name (`class="figure"`) as the styling hook. Tag names a stylesheet styles as an element selector are protected book-wide, so styled formatting is never destroyed. Both EPUB2-targeted fixes are gated on the package version declared in the OPF: on an EPUB 3 book they do nothing, because their target defects only exist below EPUB 3.
|
|
40
40
|
- **`--unwrap-illegal-tags`**: strips completely invalid or deprecated HTML tags that break EPUB3 validation (`<st>`, `<sentence>`, `<o>`, `<w>`, `<pagebreak>`) while retaining their inner text. Any of those names styled as an *element selector* by an EPUB stylesheet (`.css` entries and inline `<style>` blocks alike; class/id selectors like `.st`/`#w` don't count) is protected for the whole book, guaranteeing format preservation.
|
|
41
|
+
- **`--declare-missing-items`**: the RSC-008 inverse of the prune: adds the manifest `<item>` declarations the OPF never wrote for files the archive carries and the book references (content `<img src>`/`<link href>` targets, CSS `url()` resources, OPF guide references). Deterministic ids, media types from a blessed extension map only, and no `<itemref>` is ever written: the spine is untouched.
|
|
42
|
+
- **`--fix-unique-identifier`**: wires the dangling OPF unique-identifier (OPF-030): adds the id the package names to the first `dc:identifier` when no element carries it, and aligns the NCX `dtb:uid` the broken wiring was masking in the same run. The `dc:` values are never touched; ambiguous shapes refuse.
|
|
41
43
|
- **`--prune-missing-resources`**: removes references to files the archive does not contain (RSC-007/PKG-010): dead `<link>` elements, anchors' `href` to absent files (anchor text preserved), absent `<img>` sources (replaced by their alt text when they carry one), and orphaned non-spine OPF manifest items (with every package edge that pointed at them rewritten: `spine@toc`, `item@media-overlay`, `item@fallback`, and the EPUB2 cover meta). Spine documents are never pruned: a missing spine document is reported as a damaged fragment, not silently dropped.
|
|
42
44
|
- **`--strip-broken-anchors`**: strips href attributes that cannot resolve, keeping the anchor text byte-for-byte: a `#fragment` the target document does not define (RSC-020/RSC-012; NCX navTargets fall back to the document target, so chapter navigation survives) and unresolvable URI schemes (`kindle:`, `file:`).
|
|
43
45
|
- **`--encode-url-spaces`**: percent-encodes raw spaces in `src`/`href` attribute values across the package (OPF manifest, NCX, content documents). A URL with a literal space is not a valid URL (RSC-020) and unresolvable on strict readers; the encoded form denotes the same file. Since v0.44.0 the flag also renames archive entries whose names carry raw spaces to their underscore spellings and rewrites every reference to them (OPF, NCX, content, CSS `url()`), clearing the PKG-010 space-in-filename advisories: underscore, never percent-encoding, because epubcheck decodes references before entry lookup.
|
|
44
46
|
- **`--fix-container`**: generates `META-INF/container.xml` at the located OPF when the container is missing or names a file the archive does not contain. This is the gateway defect: epubcheck stays fatal while the OPF is unfindable, so no repair can be gate-accepted on such a book until the container exists.
|
|
45
|
-
- **`--fix-media-types`**: normalizes wrong manifest media-type declarations (OPF-029, e.g. a jpg stamped `image/png` by an aggregator). Attribute-only and quote-preserving; it fires only when the file's magic bytes confirm the extension, so a misnamed-but-consistent file (a PNG renamed .jpg) is never made worse.
|
|
47
|
+
- **`--fix-media-types`**: normalizes wrong manifest media-type declarations (OPF-029, e.g. a jpg stamped `image/png` by an aggregator). Attribute-only and quote-preserving; it fires only when the file's magic bytes confirm the extension, so a misnamed-but-consistent file (a PNG renamed .jpg) is never made worse. Since v0.56.0 it also covers the CSS-007 font slice (a `ttf`/`otf`/`woff`/`woff2` declared outside epubcheck's blessed font set is rewritten to its `font/*` spelling when the magic bytes confirm; `.eot` is never touched), and the flag is accepted under the no-worse bar because that half's gain sits on the INFO axis.
|
|
46
48
|
- **`--fix-cover`**: repairs dangling EPUB2 cover wiring. A `<meta name="cover">` whose content names no manifest item is re-pointed when the OPF guide's own cover reference names an existing item, and removed when nothing identifies it; EPUB3 `properties="cover-image"` is deliberately audit-only (guessing which image is the cover is not deterministic). Cover wiring is invisible to epubcheck, so cover-only repairs are accepted under the same no-worse bar the lossy strips use.
|
|
47
49
|
- **`--fix-comment-double-hyphen`**: replaces `--` sequences inside XML comments with en-dashes. `--` is illegal inside an XML comment (epubcheck fatal RSC-016: the book opens in lenient readers but never passes epubcheck). The edit stays inside the comment bodies: text nodes and CDATA sections are never touched, and each comment's terminator is left intact.
|
|
48
50
|
- **`--fix-svg-dup-ids`**: renames duplicate `id` values inside standalone `.svg` entries (old calibre SVG page renders repeat glyph ids within one document; the intake wave that found this carried 445 to 1,339 'Duplicate "glNNNN"' errors per book). The first occurrence keeps its name, later ones gain `_2`/`_3`, and internal references are deliberately untouched: they already resolve to the first definition under every reader's first-match lookup.
|
|
51
|
+
- **`--fix-duplicate-ids`**: the same first-keeps-name rename for duplicate `id` values in the content documents themselves (RSC-005 "Duplicate id" on `.xhtml`/`.html` carriers). Every reference to the name still resolves to the first definition, exactly as readers resolved it before.
|
|
49
52
|
- **`--fix-cdata-terminator`**: completes a truncated CDATA-terminator comment in an inline `<style>` block (`/*]]>` becomes `/*]]>*/`). A comment-wrapped CDATA block whose closing `*/` the converter dropped leaves the CSS parser inside an unterminated comment: one CSS-008 'Premature end of file' per affected document. Only the malformed-terminator token class is rewritten; invalid CSS stays unfixed, and stylesheet files are not touched.
|
|
50
53
|
- **`--fix-misnested-inline`**: rewrites the Mobipocket drop-cap mis-nest `<i><b>X</i></b>` to `<i><b>X</b></i>` (em/strong variants and either tag order). Only the both-open-then-reversed-close shape is touched; a punctuation run caught between the reversed closers (`<i><b>xile</i>,</b>`) survives outside both spans, and a letter run there is refused (which span keeps the letters has no deterministic answer).
|
|
51
54
|
- **`--fix-stray-close`**: removes an end tag whose element has no open start tag anywhere above it (the stray extra `</div>` on a shell page, which cascades into 'body must be terminated by the matching end-tag'). Merely mis-nested pairs are never touched: a close is removed only when nothing by that name is open.
|
|
@@ -349,8 +352,16 @@ bindery observe ~/docs/Calibre\ Library --output ~/observe/base.observe
|
|
|
349
352
|
it measures what `--apply` would write, per candidate, and never writes.
|
|
350
353
|
- `--against base.observe` carries a trend block in the new artifact: library
|
|
351
354
|
deltas, error-mass delta, and per-code changes with `new_codes` and
|
|
352
|
-
`resolved_codes`. Artifacts are versioned (`bindery-observe-
|
|
353
|
-
schema mismatch is a usage error, never a silently incomparable trend
|
|
355
|
+
`resolved_codes`. Artifacts are versioned (`bindery-observe-v2`); a
|
|
356
|
+
schema mismatch is a usage error, never a silently incomparable trend
|
|
357
|
+
(v1 artifacts stay valid `--against` targets: the trend keys exist in
|
|
358
|
+
both).
|
|
359
|
+
- Two counting scales, stated in the artifact's `counting` block:
|
|
360
|
+
`codes.messages` (and the per-book `codes` rows) count aggregated
|
|
361
|
+
epubcheck JSON message entries, one per distinct defect text; that is
|
|
362
|
+
the scale the repair gate measures. `codes.occurrences` sums the
|
|
363
|
+
per-message locations, the per-site scale the epubcheck text summary
|
|
364
|
+
reports; one book can differ by orders of magnitude between them.
|
|
354
365
|
- A book the JVM cannot read arrives as a sentinel record instead of being
|
|
355
366
|
dropped, and a run that comes back short of the input count refuses to
|
|
356
367
|
write a partial census. The library and audit walks skip `.caltrash`.
|
|
@@ -1,5 +1,144 @@
|
|
|
1
1
|
# bindery-cli Patch Notes
|
|
2
2
|
|
|
3
|
+
## v0.56.0 (2026-10-25)
|
|
4
|
+
|
|
5
|
+
### The two boxed recipes, built: the OPF-030 wiring and the CSS-007 font half
|
|
6
|
+
|
|
7
|
+
The v0.55.0 census lanes left two deterministic-but-marginal candidates
|
|
8
|
+
recorded as recipes; both shipped the same day on the follow-up call.
|
|
9
|
+
|
|
10
|
+
**`--fix-unique-identifier` (the OPF-030 recipe).** The dominant family:
|
|
11
|
+
the package names an id (`bookid`, `uuid_id`, ISBN shapes) but no
|
|
12
|
+
`dc:identifier` carries any id attribute at all. The repair writes the id
|
|
13
|
+
the package itself names onto the first `dc:identifier` start tag; the
|
|
14
|
+
dc: values are never touched. It refuses to guess every ambiguous shape:
|
|
15
|
+
an absent or empty `unique-identifier` attribute, a non-NCName id, an
|
|
16
|
+
already-wired OPF, no `dc:identifier` element, or a first identifier that
|
|
17
|
+
already carries an id. The live gate on the census specimen (The Eagle Has
|
|
18
|
+
Landed) surfaced a masking twist: the broken wiring was hiding a real
|
|
19
|
+
NCX-001 (epubcheck cannot compare `dtb:uid` to an unresolvable
|
|
20
|
+
identifier), and the default pass's uid sync skipped because the source
|
|
21
|
+
had no resolvable uid. The repair now predicts the post-wiring uid (the
|
|
22
|
+
same pure transformation the OPF branch applies; never a snapshot of a
|
|
23
|
+
different layout, the issue #10 lesson) so one run wires the OPF AND
|
|
24
|
+
aligns the NCX: the specimen went from 2 errors (OPF-030 + a masked
|
|
25
|
+
latent) to 1 (a pre-existing out-of-scope OPF-032), accept, with
|
|
26
|
+
`unique_identifier_wired:1` and the NCX sync fired. The multi-OPF merge
|
|
27
|
+
leftovers whose failing OPF is a stale non-rootfile are unreachable by
|
|
28
|
+
construction (only the located rootfile is repaired).
|
|
29
|
+
|
|
30
|
+
**The CSS-007 font half of `--fix-media-types`.** The census found the
|
|
31
|
+
INFO-severity family: `@font-face` srcs whose OPF declaration sits
|
|
32
|
+
outside epubcheck's version-specific blessed font set (`text/plain`
|
|
33
|
+
fonts, `application/xfont+otf`, `octet-stream` woff2). The map gains
|
|
34
|
+
`ttf`/`otf`/`woff`/`woff2` with their magic bytes; the `font/*`
|
|
35
|
+
spellings are blessed on BOTH package versions (EPUB2 blesses the prefix,
|
|
36
|
+
EPUB3's exact list carries them), so no version gate is needed, and the
|
|
37
|
+
existing magic-byte confirmation rule is unchanged (a font is rewritten
|
|
38
|
+
only when its bytes name the format the extension does; `.eot` is never
|
|
39
|
+
touched -- epubcheck rejects EOT as a font src however it is declared, so
|
|
40
|
+
correcting it can only churn). Because CSS-007 is INFO, the error counts
|
|
41
|
+
do not see the gain, so the flag is now accepted under the `no_worse` bar
|
|
42
|
+
(the cover-wiring ruling's shape): the OPF-029 error slice still measures
|
|
43
|
+
its real decrease through the normal gate, and a font-only repair no
|
|
44
|
+
longer reads as gainless. Live gate on Brave New World (8 `text/plain`
|
|
45
|
+
fonts): `media_types_normalized:8`, accept, CSS-007 fully cleared.
|
|
46
|
+
|
|
47
|
+
Twenty-five structural repairs. Suite: 768 tests.
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
## v0.55.0 (2026-10-25)
|
|
51
|
+
|
|
52
|
+
### The issue batch: the 2026-10-24 surgery's follow-ups, #11 through #16
|
|
53
|
+
|
|
54
|
+
The error-code surgery left six open issues; this release works all of
|
|
55
|
+
them, each on its own evidence (the census lanes and live gate runs are
|
|
56
|
+
recorded in `OBSERVE-RESEARCH.md`, specimens in the gitignored testing
|
|
57
|
+
facility).
|
|
58
|
+
|
|
59
|
+
**`--strip-epub3-attrs` covers `role` and `aria-hidden` (issue #11).** The
|
|
60
|
+
resample had found EPUB3 semantics on legacy packages the dominant RSC-005
|
|
61
|
+
family (`role` 6,645 sampled occurrences, `aria-hidden` 239) while the
|
|
62
|
+
flag's fixed set carried only three attributes. The population census
|
|
63
|
+
settled both design questions with data: every role/aria-hidden carrier in
|
|
64
|
+
the sample is an EPUB2-declared package (the values are DPUB-ARIA
|
|
65
|
+
`doc-*` roles and `aria-hidden="true"`, no visual default either way), and
|
|
66
|
+
the DOCTYPE-gate concern is falsified (seven EPUB3 packages carrying
|
|
67
|
+
XHTML 1.1 doctypes show zero attribute findings; epubcheck keys on the
|
|
68
|
+
package profile, so the existing package-version gate is correct as
|
|
69
|
+
built). The set grew; the gate stayed. `hidden` deliberately stays out:
|
|
70
|
+
stripping it would reveal content the author hid.
|
|
71
|
+
|
|
72
|
+
**`--fix-duplicate-ids` (issue #14).** The 20-book sample caught duplicate
|
|
73
|
+
ids in HTML content documents (the Calcutta Chromosome `heading_id_2`)
|
|
74
|
+
that no flag addressed. The SVG fix's first-keeps-name rule extends
|
|
75
|
+
cleanly: the first occurrence keeps its name, later duplicates gain
|
|
76
|
+
`_2`/`_3`, and no reference is rewritten because `#fragment` anchors, CSS
|
|
77
|
+
`#id` selectors, and NCX fragment references all resolve to the first
|
|
78
|
+
definition exactly as readers resolved them before. Live gate on the
|
|
79
|
+
specimen: accept, net error decrease alongside the default pass.
|
|
80
|
+
|
|
81
|
+
**`--declare-missing-items` (issue #16).** The coverage study answered the
|
|
82
|
+
question both ways. RSC-001 is already covered where it can be: 95.3% of
|
|
83
|
+
its mass is spine documents whose files exist under a different spelling
|
|
84
|
+
(the Wandering Inn pair alone is 1,474 of 1,546 entries), which the spine
|
|
85
|
+
guard refuses by design and `--encode-url-spaces` owns; the long tail is
|
|
86
|
+
cleared by the existing prune; the boundary is documented, not widened.
|
|
87
|
+
RSC-008 had zero coverage, structurally: the prune's predicate is
|
|
88
|
+
target-absent and an undeclared resource is by definition present. The new
|
|
89
|
+
flag is the inverse move: it adds the manifest `<item>` declarations the
|
|
90
|
+
OPF never wrote for files the book references and the archive carries
|
|
91
|
+
(content `img`/`link`, CSS `url()` resources, OPF guide references), with
|
|
92
|
+
sorted deterministic `bindery-dN` ids, blessed extension-map media types
|
|
93
|
+
only, and no `<itemref>` ever written (the spine is untouched; an itemref
|
|
94
|
+
with no manifest item stays unrepaired, since which file it meant is
|
|
95
|
+
underdetermined). Live gate on the facility specimen: 3 items declared
|
|
96
|
+
(the three Calibre split-CSS files), RSC-008 fully cleared, accept, and a
|
|
97
|
+
second run declares nothing.
|
|
98
|
+
|
|
99
|
+
**The epubcheck timeout stops lying (issue #15).** The Summa Theologica
|
|
100
|
+
timed out at the fixed 300s in one whole-library run and passed the
|
|
101
|
+
identical gate in another. The default oracle budget is now size-scaled
|
|
102
|
+
(300s at or under 10 MB, plus a minute per started 10 MB above, capped at
|
|
103
|
+
30 minutes), one subprocess retry follows a timed-out attempt, and only a
|
|
104
|
+
double timeout raises the new `OracleTimeout`, which the gate reports as
|
|
105
|
+
`(epubcheck timeout)` instead of filing the flake under
|
|
106
|
+
`(epubcheck failed)` with crashes and unparsable output. Sweep selection
|
|
107
|
+
catches it and keeps the book a candidate; a timed-out book is still never
|
|
108
|
+
applied.
|
|
109
|
+
|
|
110
|
+
**Observe schema v2 (issue #13).** The census counted aggregated epubcheck
|
|
111
|
+
JSON message entries and labeled them `occurrences`; the Feynman specimen
|
|
112
|
+
measured the gap at 20,307 text sites vs 7,030 entries (up to ~600x on
|
|
113
|
+
other books). `codes.occurrences` is renamed `codes.messages` (the gate's
|
|
114
|
+
scale), a true `codes.occurrences` now sums each message's `locations`
|
|
115
|
+
(the per-site scale), a `counting` block in the artifact states both
|
|
116
|
+
scales, the census harness reads the JSON with Jackson (already on
|
|
117
|
+
epubcheck's classpath; the legacy code scan stays as the fallback), and
|
|
118
|
+
`--against` accepts v1 artifacts: the trend keys exist in both.
|
|
119
|
+
|
|
120
|
+
**The content-model ruling (issue #12).** The span-in-body, incomplete
|
|
121
|
+
blockquote, and bare-body-text families (well-formed but schema-invalid
|
|
122
|
+
under XHTML 1.1) are ruled measured-not-repaired in `spec.md`: they render
|
|
123
|
+
everywhere, the only mechanizable repair (wrapping the loose content in
|
|
124
|
+
the implied element) changes layout, and a repair that cannot be made
|
|
125
|
+
deterministically rendering-preserving does not belong in the inventory.
|
|
126
|
+
`--reserialize` stays deliberately inert here (its trigger is XML
|
|
127
|
+
ParseError; these documents parse). The spec's out-of-scope list carries
|
|
128
|
+
the ruling with the Dark Star falsification cited.
|
|
129
|
+
|
|
130
|
+
**The census lanes (issue #17).** RSC-012 is `--strip-broken-anchors`'
|
|
131
|
+
class (96.5% of 1,210 live locations across 188 books; the 42 OPF-guide
|
|
132
|
+
locations are furniture no repair touches). OPF-029 is `--fix-media-types`
|
|
133
|
+
for 216 of 280 locations (the Tress probe went 75 errors to zero, fully
|
|
134
|
+
clean); the 64 survivors are honest-or-refused declarations the flag
|
|
135
|
+
correctly leaves. OPF-030 (27 books), CSS-007 (114 books, INFO), and
|
|
136
|
+
RSC-004 (84 books, all font obfuscation, zero DRM, the audit's benign
|
|
137
|
+
OBFUSCATED verdict) are recorded leave-measured with their mechanisms and,
|
|
138
|
+
where one exists, the recipe for a future pass. The full census, wordings,
|
|
139
|
+
and per-book tables live in `OBSERVE-RESEARCH.md`.
|
|
140
|
+
|
|
141
|
+
|
|
3
142
|
## v0.54.0 (2026-10-24)
|
|
4
143
|
|
|
5
144
|
### The error-code surgery fixes: the core pass stopped breaking valid books
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "bindery-cli"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.56.0"
|
|
8
8
|
description = "Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
classifiers = [
|
|
@@ -48,6 +48,8 @@ The plugin vendors the core and never gates (the epubcheck latency ruling).
|
|
|
48
48
|
| 0.52 | The spine guard (issue #10): the manifest prune reads the layout its hrefs are written in and derives its spine protection from the text it prunes (the raw-space wave's spine collapse), and `process_book` vetoes any candidate whose spine (itemref -> item -> archive) dangles more than the source's, above the counts and every promotion | v0.52.0 |
|
|
49
49
|
| 0.53 | `bindery observe` (the observatory): the whole-library census in one warm-JVM pass (counts and error codes together, sentinel-backed so no book is dropped), tabulation (severity histogram, code profile, worst offenders), the gated repair preview composed in-process, the versioned `.observe` artifact with `--against` trend diffing and the keep-root-clean output guard; the library and audit walks skip `.caltrash` | v0.53.0-v0.53.1 |
|
|
50
50
|
| 0.54 | The surgery fixes: `fix_ncx_playorder` is contract-gated (epubcheck's playOrder contract: same-target sharing, distinct-target distinctness, contiguous 1..K across navTarget/pageTarget; the old unconditional 1..N renumbering rewrote valid TOCs into violations, 211 of 579 firings gate-rejected vs 17 real fixes) and `_PROTECTED_RE` protects the Calibre `/*<![CDATA[*/ ... /*]]>*/` wrapper as one unit (the bare CDATA span exposed the trailing `*/` to strip_prolog_junk's amputation, 111 phantom-fix rejections); verified by the 2026-10-24 error-code surgery (OBSERVE-RESEARCH.md) | v0.54.0 |
|
|
51
|
+
| 0.55 | The issue-batch release (issues #11-#16, the 2026-10-24 surgery's follow-ups): `--strip-epub3-attrs` extended with `role` + `aria-hidden` (the census killed the DOCTYPE-gate concern: epubcheck is profile-driven; #11), `--fix-duplicate-ids` for content docs (#14), `--declare-missing-items` for RSC-008 (#16, with the prune's RSC-001 boundary documented), the size-scaled epubcheck timeout with one retry and the distinct `OracleTimeout` (#15), observe schema v2 (`codes.messages` rename, true `codes.occurrences`, the `counting` block, v1 `--against` accepted; #13), the #12 content-model ruling (measured, not repaired: the wrap changes layout), and the #17 census lanes (RSC-012/OPF-029 dispositions on existing flags; OPF-030/CSS-007/RSC-004 recorded leave-measured with recipes) | v0.55.0 |
|
|
52
|
+
| 0.56 | The boxed recipes, built same-day on the follow-up call: `--fix-unique-identifier` (the OPF-030 recipe: wire the package's named id onto the first dc:identifier, refuses every ambiguous shape, and predicts the post-wiring uid so the NCX sync the broken wiring was masking fires in the same run -- eagle 2e -> 1e accept) and the CSS-007 font half inside `--fix-media-types` (font/* blessed on both package versions, magic confirmed, .eot never touched; accepted under no_worse because the gain sits on the INFO axis -- Brave New World 8 fonts normalized, CSS-007 cleared) | v0.56.0 |
|
|
51
53
|
|
|
52
54
|
## Open work
|
|
53
55
|
|