bindery-cli 0.42.0__tar.gz → 0.43.0__tar.gz

{bindery_cli-0.42.0 → bindery_cli-0.43.0}/CLAUDE.md

@@ -86,7 +86,7 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
 - `src/bindery/pagination.py`: the opt-in lossy page-number strip (runhead detection, page-layer decision, block-centric removal/merge, safety nets).
 - `src/bindery/watermark.py`: the opt-in lossy watermark strip (anchored and anchorless signature removal).
 - `src/bindery/reserialize.py`: structural repair via `html5lib`.
-- `src/bindery/audit.py`: read-only body-text audits (`content`, `pagenumbers`, `emptytext`, `ocr`, `monolithic` since v0.21.0 with `--max-doc-chars N`, and `completeness` since v0.36.0) behind the `audit` subcommand (v0.15.0). Since v0.18.0 library mode resolves EPUB paths through `cquarry.get_format_path()` and can apply a tag to flagged books via `cquarry.write.WritableCalibreDB` (only with explicit `--tag`; the only sanctioned write path). Since v0.19.0 `audit --id BOOK_ID` audits a single library book through cquarry's single-entity `get_book()` fetch (no library-wide cache; supports `--tag`; incompatible with the directory argument; comma lists since v0.23.0). Since v0.22.0 every archive entry is fully read (CRC + decompression) before analysis (a damaged entry reports its own CORRUPT verdict instead of feeding emptytext) and manifest/NCX references to absent files classify as `convention` (bloated ToC, consecutive span) or `fragment` (broken span). Hazard: the scan loop reuses `tag` as its per-book display column, so the `--tag` argument is captured as `audit_tag` at function entry; do not collapse them again. Since v0.29.0 `audit --json FILE` writes per-file analyzer verdicts in the `library --json` shape (directory, library, and single-book modes; with `--id`, exactly one id; emptytext omitted from a record when the archive verdict owns the body-text story). Since v0.36.0 the completeness analyzer (Phase 15) reports the phase-1 spot-check per book: spine/prose-doc counts (prose = >= 400 visible chars), first/middle/last prose-doc opening/closing excerpts, trailing-ToC classification (`_trailing_toc`: short link lines dominate, no paragraph-length blocks), and the unreadable fraction. It is ADVISORY by contract (problem always False; ADVISORY on a trailing ToC or unreadable fraction >= 10%), never moves the exit code, and is skipped like emptytext when the archive verdict owns the book. Since v0.38.0 the archive verdict distinguishes font obfuscation from DRM (`OBFUSCATION_ALGOS`: IDPF 2008/embedding + both Adobe URIs; readable obfuscated entries are the benign OBFUSCATED advisory, an unreadable one is CORRUPT, and only non-obfuscation algorithms give ENCRYPTED). Since v0.40.0 the vir-tui console renderer loads through the module-level `_LazyUI` proxy (a plain `import vir_tui as ui` at module level would make 3.12/3.13 installs unimportable now that the stack is marker-gated; LOAD_GLOBAL never consults module `__getattr__`, so a proxy object is the lazy shape that works).
+- `src/bindery/audit.py`: read-only body-text audits (`content`, `pagenumbers`, `emptytext`, `ocr`, `monolithic` since v0.21.0 with `--max-doc-chars N`, and `completeness` since v0.36.0) behind the `audit` subcommand (v0.15.0). Since v0.18.0 library mode resolves EPUB paths through `cquarry.get_format_path()` and can apply a tag to flagged books via `cquarry.write.WritableCalibreDB` (only with explicit `--tag`; the only sanctioned write path). Since v0.19.0 `audit --id BOOK_ID` audits a single library book through cquarry's single-entity `get_book()` fetch (no library-wide cache; supports `--tag`; incompatible with the directory argument; comma lists since v0.23.0). Since v0.22.0 every archive entry is fully read (CRC + decompression) before analysis (a damaged entry reports its own CORRUPT verdict instead of feeding emptytext) and manifest/NCX references to absent files classify as `convention` (bloated ToC, consecutive span) or `fragment` (broken span). Hazard: the scan loop reuses `tag` as its per-book display column, so the `--tag` argument is captured as `audit_tag` at function entry; do not collapse them again. Since v0.29.0 `audit --json FILE` writes per-file analyzer verdicts in the `library --json` shape (directory, library, and single-book modes; with `--id`, exactly one id; emptytext omitted from a record when the archive verdict owns the body-text story). Since v0.36.0 the completeness analyzer (Phase 15) reports the phase-1 spot-check per book: spine/prose-doc counts (prose = >= 400 visible chars), first/middle/last prose-doc opening/closing excerpts, trailing-ToC classification (`_trailing_toc`: short link lines dominate, no paragraph-length blocks), and the unreadable fraction. It is ADVISORY by contract (problem always False; ADVISORY on a trailing ToC or unreadable fraction >= 10%), never moves the exit code, and is skipped like emptytext when the archive verdict owns the book. Since v0.38.0 the archive verdict distinguishes font obfuscation from DRM (`OBFUSCATION_ALGOS`: IDPF 2008/embedding + both Adobe URIs; readable obfuscated entries are the benign OBFUSCATED advisory, an unreadable one is CORRUPT, and only non-obfuscation algorithms give ENCRYPTED). Since v0.43.0 untrusted metadata XML is parsed through `_safe_xml_parse` (DOCTYPE/ENTITY declarations refused before xml.etree sees them; the stdlib shape of defusedxml's entity protection) and single entries above `_MAX_ENTRY_BYTES` (32 MB) are refused before decompression; a refused metadata file returns the shell book. Since v0.40.0 the vir-tui console renderer loads through the module-level `_LazyUI` proxy (a plain `import vir_tui as ui` at module level would make 3.12/3.13 installs unimportable now that the stack is marker-gated; LOAD_GLOBAL never consults module `__getattr__`, so a proxy object is the lazy shape that works).
 - `src/bindery/epub.py`: archive rewrite, NCX uid sync, RepairReport, mismatch detection, and the opt-in structural-repair plumbing (including the CSS precondition scan). Since v0.38.0/v0.39.0 (Phase 16): `generate_container` + the `--fix-container` gateway (insert after mimetype when missing, replace in-loop when stale; constant-epoch timestamp keeps bytes deterministic), `prune_dangling_edges` (prune_missing_manifest_items returns the pruned ids and the edges that pointed at them (spine@toc, media-overlay, fallback, EPUB2 cover meta) are rewritten so pruning cannot manufacture the regression the gate rejects), and `fix_manifest_media_types` (extension map + magic-byte confirmation via a `peek` callable over the open source zip; jpg/png/gif only), and `fix_cover_meta` (the ruling's deterministic half: dangling EPUB2 cover meta re-pointed from the guide's cover reference when exactly one manifest item carries that file, removed otherwise; accepted under no_worse like the lossy strips because cover wiring is invisible to epubcheck).
 - `src/bindery/validate.py`: epubcheck wrapper, the `gate` (improvement) and `no_worse` (no-regression, for the lossy strips) acceptance bars.
 - `src/bindery/library.py`: Calibre walk, atomic replace, backups, and native format installation. Since v0.19.0 `CalibreIdResolver` resolves the book id from `metadata.db` through cquarry (one lazy path→id map per run; since v0.20.0 the map comes from `CalibreDB.format_path_index()`, re-normalized with `resolve().lower()` for the resolver's symlinked-directory and case-insensitive matching); the `(id)` directory-name regex is only the no-catalog fallback. Since v0.24.0 `install_format()` places the repaired file atomically and updates the `data` row through `cquarry.write.WritableCalibreDB` (since v0.31.0 via `set_format`, cquarry 1.17's sanctioned remove+add in one transaction); the external `calibredb` CLI is gone. Since v0.32.0 a guessed id drives a row update only when metadata.db corroborates it (`_verify_guess`: books row exists, file in the book's own directory, catalogued `data.name` when a row exists); anything else saves in place and leaves the catalog untouched: never reintroduce directory-name guessing as the primary source: renamed/mismatched directories would replace the wrong book. Since 2026-09-15 `make_backup` takes the opt-in `--backup-keep N` ring: at most N backup files per book, the author original `.bak` never deleted, newest content at the highest name; default remains an unbounded rotation.

{bindery_cli-0.42.0 → bindery_cli-0.43.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: bindery-cli
-Version: 0.42.0
+Version: 0.43.0
 Summary: Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library.
 Project-URL: Homepage, https://github.com/VirInvictus/bindery-cli
 Project-URL: Repository, https://github.com/VirInvictus/bindery-cli

{bindery_cli-0.42.0 → bindery_cli-0.43.0}/patchnotes.md

@@ -1,4 +1,11 @@
 # bindery-cli Patch Notes
+## v0.43.0 (2026-09-16)
+
+### The stdlib XML hardening: untrusted metadata is bounded and entity-refused, no dependency
+
+* **`--audit` and every `load_book` caller no longer decompress blind.** Two protections, both stdlib-only (the defusedxml dependency stays declined): single entries whose declared size exceeds `_MAX_ENTRY_BYTES` (32 MB) are refused before decompression -- a bomb entry in a tiny zip cannot be downloaded into memory -- and the untrusted metadata files (container.xml, OPF, encryption.xml) parse through `_safe_xml_parse`, which refuses DOCTYPE/ENTITY declarations before xml.etree sees them. A refused or unparseable metadata file returns the shell book (the archive verdict is the book's whole story), never a hang or a traceback.
+* Pinned by `tests/test_xml_hardening.py` (entity-bomb container returns the shell book; clean XML parses; the entry cap is sane). The defusedxml dependency ask is closed as declined with the stdlib shape delivered.
+
 ## v0.42.0 (2026-09-16)
 
 ### The matcher audit executed: every quote-aware tag matcher hardened

{bindery_cli-0.42.0 → bindery_cli-0.43.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "bindery-cli"
-version = "0.42.0"
+version = "0.43.0"
 description = "Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library."
 readme = "README.md"
 classifiers = [

{bindery_cli-0.42.0 → bindery_cli-0.43.0}/roadmap.md

@@ -1841,13 +1841,18 @@ four interpreter families.
       testing_facility library (invented titles, real dry-run output);
       the current PNG shows 16 real rows. Ask asked 2026-09-15,
       unanswered; nothing touched.
-- [ ] **Stranger-facing XML hardening** (dependency-flagged): a
-      per-entry size cap in load_book (stdlib, bounds decompression)
-      plus defusedxml-style entity-expansion gating for the untrusted
-      XML (container.xml, OPF, encryption.xml, NCX). RECORDED REOPEN
-      (Brandon's ask pending; the dependency ask would violate the
-      stdlib-only rule and adds a vendored plugin dep): trigger is a
-      real 3.12/3.13 stranger audience, not a hypothetical.
+- [x] **Stranger-facing XML hardening** *(SHIPPED 2026-09-16, v0.43.0
+      -- the STDLIB half only, no dependency: `_MAX_ENTRY_BYTES` (32 MB)
+      checked before `_read`'s decompression so a bomb entry is refused,
+      and `_safe_xml_parse` refuses DOCTYPE/ENTITY declarations in the
+      untrusted metadata (container.xml, OPF, encryption.xml) before
+      xml.etree sees them; a refused or unparseable metadata file
+      returns the shell book, the archive being the book's whole story.
+      Pinned by tests/test_xml_hardening.py. The defusedxml dependency
+      itself STAYS DECLINED: the stdlib shape delivers the entity
+      protection, and the vendored-plugin cost was the box's real
+      objection. The full-parser-swap reopen remains Brandon-gated if a
+      real 3.12/3.13 stranger audience ever materializes.)*
 - [x] **vir-tui floor >=2.5.0 + lock refresh** (2026-09-14, no release cut: upstream 2.4.0/2.5.0 are additive terminal-safety and session-awareness releases; only the vir-tui resolution moved in the lock, the cquarry lag rule is untouched). *(Mention the floor bump in the next release's patchnotes.)*
 
 - [x] **REGRESSION (0.41.0) FIXED in v0.41.1: the repair --all CPU

{bindery_cli-0.42.0 → bindery_cli-0.43.0}/src/bindery/__init__.py

@@ -11,7 +11,7 @@ Calibre plugin built from this core.
 Single source of truth for the version. pyproject.toml must match VERSION below.
 """
 
-VERSION = "0.42.0"
+VERSION = "0.43.0"
 __version__ = VERSION
 
 from .epub import RepairReport, repair_epub  # noqa: E402

{bindery_cli-0.42.0 → bindery_cli-0.43.0}/src/bindery/audit.py

@@ -226,6 +226,34 @@ class Book:
         return self._visible
 
 
+# v0.43.0 XML hardening (stdlib-only): a single EPUB component above this
+# size is refused before decompression (a real book's largest component --
+# one content doc, the OPF, the NCX -- is nowhere near it), and the
+# metadata XML parser refuses DOCTYPE/ENTITY declarations, which is
+# defusedxml's entity protection without the dependency.
+_MAX_ENTRY_BYTES = 32 * 1024 * 1024
+
+# a stand-in root for a refused metadata parse: iter() over it is empty
+_ET_EMPTY = ET.Element("empty")
+
+
+def _safe_xml_parse(blob: bytes | None):
+    """xml.etree parse for UNTRUSTED metadata files, stdlib-hardened: a
+    DOCTYPE or ENTITY declaration anywhere in the blob is refused before
+    parsing, which is defusedxml's entity protection without the
+    dependency. Returns None for a refused or unparseable blob; callers
+    treat that exactly like a missing file."""
+    if blob is None:
+        return None
+    head = blob[:4096].upper()
+    if b"<!DOCTYPE" in head or b"<!ENTITY" in head:
+        return None
+    try:
+        return ET.fromstring(blob)
+    except Exception:
+        return None
+
+
 def load_book(path: Path) -> Book:
     """Open an EPUB once: fully read EVERY archive entry (the CRC sweep the
     phase-1 skill used to do by hand), resolve spine + nav + declared
@@ -238,6 +266,7 @@ def load_book(path: Path) -> Book:
         nameset = set(names)
         corrupt: list[str] = []
         encrypted: list[str] = []
+        oversized: list[str] = []
         spine_missing = 0
         dup_entries = len(names) - len(nameset)  # zip resolves last-wins
         raw: dict[str, bytes] = {}
@@ -260,6 +289,12 @@ def load_book(path: Path) -> Book:
             if name in corrupt or name in encrypted:
                 return None
             try:
+                # v0.43.0 hardening: the declared size is checked BEFORE the
+                # read, so a bomb entry (a tiny zip that decompresses to
+                # gigabytes) is refused instead of downloaded into memory.
+                if z.getinfo(name).file_size > _MAX_ENTRY_BYTES:
+                    oversized.append(name)
+                    return None
                 blob = z.read(name)
             except Exception:
                 if name in encrypted_names:
@@ -279,7 +314,7 @@ def load_book(path: Path) -> Book:
         )
         if enc_blob:
             try:
-                for ed in ET.fromstring(enc_blob).iter():
+                for ed in (_safe_xml_parse(enc_blob) or _ET_EMPTY).iter():
                     if ed.tag.split("}")[-1] != "EncryptedData":
                         continue
                     algo = ""
@@ -324,12 +359,57 @@ def load_book(path: Path) -> Book:
                 obfuscated=obfuscated,
                 obfuscated_declared=obfuscated_declared,
             )
-        container = ET.fromstring(container_blob)
+        container = _safe_xml_parse(container_blob)
+        if container is None:
+            # A container that PARSES but was refused (DOCTYPE/ENTITY
+            # declarations: untrusted-XML policy, v0.43.0) or is otherwise
+            # unparseable gets the same shell-book outcome as an absent
+            # one -- the archive is the book's whole story.
+            obfuscated = [
+                n
+                for n in sorted(encrypted_names & obfuscated_declared)
+                if n not in corrupt and n not in encrypted
+            ]
+            return Book(
+                [],
+                None,
+                "",
+                {},
+                names,
+                corrupt,
+                0,
+                0,
+                encrypted=encrypted,
+                dup_entries=dup_entries,
+                obfuscated=obfuscated,
+                obfuscated_declared=obfuscated_declared,
+            )
         rootfile = container.find(".//c:rootfile", CONTAINER_NS)
         opf_path = rootfile.get("full-path") if rootfile is not None else None
         if not opf_path:
             raise ValueError("container.xml has no rootfile")
-        opf = ET.fromstring(_read(opf_path))
+        opf = _safe_xml_parse(_read(opf_path))
+        if opf is None:
+            # same shell-book outcome for a refused/unparseable OPF
+            obfuscated = [
+                n
+                for n in sorted(encrypted_names & obfuscated_declared)
+                if n not in corrupt and n not in encrypted
+            ]
+            return Book(
+                [],
+                None,
+                "",
+                {},
+                names,
+                corrupt,
+                0,
+                0,
+                encrypted=encrypted,
+                dup_entries=dup_entries,
+                obfuscated=obfuscated,
+                obfuscated_declared=obfuscated_declared,
+            )
         base = os.path.dirname(opf_path)
 
         manifest: dict[str, str] = {}

bindery_cli-0.43.0/tests/test_xml_hardening.py

@@ -0,0 +1,62 @@
+"""The v0.43.0 stdlib XML hardening: untrusted metadata files are parsed
+through _safe_xml_parse, which refuses DOCTYPE/ENTITY declarations (the
+stdlib equivalent of defusedxml's entity protection), and single archive
+entries above _MAX_ENTRY_BYTES are refused before decompression. A
+refused metadata file returns the shell book -- the archive is the
+book's whole story -- instead of hanging or crashing."""
+
+import tempfile
+import unittest
+from pathlib import Path
+
+from bindery.audit import _MAX_ENTRY_BYTES, _safe_xml_parse, load_book
+
+BOMB_CONTAINER = (
+    '<?xml version="1.0"?>\n<!DOCTYPE container [<!ENTITY a "'
+    + "A" * 100000
+    + '"]>\n<container/>'
+)
+
+
+def _write_epub(path: Path, container: bytes, opf: bytes | None = None):
+    import zipfile
+
+    with zipfile.ZipFile(path, "w") as z:
+        z.writestr("mimetype", "application/epub+zip")
+        z.writestr("META-INF/container.xml", container)
+        if opf is not None:
+            z.writestr("content.opf", opf)
+
+
+class SafeXmlParseTests(unittest.TestCase):
+    def test_doctype_is_refused(self):
+        self.assertIsNone(_safe_xml_parse(BOMB_CONTAINER.encode()))
+
+    def test_entity_inline_is_refused(self):
+        self.assertIsNone(_safe_xml_parse(b"<x>&entity;</x>"))
+
+    def test_clean_xml_parses(self):
+        root = _safe_xml_parse(b"<container><rootfile/></container>")
+        self.assertIsNotNone(root)
+
+    def test_none_blob_passes_through_as_none(self):
+        self.assertIsNone(_safe_xml_parse(None))
+
+
+class EntityBombContainerTests(unittest.TestCase):
+    def test_entity_bomb_container_returns_shell_book(self):
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "bomb.epub"
+            _write_epub(p, BOMB_CONTAINER.encode())
+            book = load_book(p)  # must not hang, must not raise
+            self.assertEqual(book.spine, [])
+            self.assertEqual(book.corrupt, [])
+
+    def test_max_entry_bytes_is_sane(self):
+        # a real book's largest component is nowhere near this; the cap
+        # exists to refuse bomb entries before decompression.
+        self.assertGreaterEqual(_MAX_ENTRY_BYTES, 8 * 1024 * 1024)
+
+
+if __name__ == "__main__":
+    unittest.main()