bindery-cli 0.29.0__tar.gz → 0.30.0__tar.gz

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/CLAUDE.md

@@ -66,7 +66,7 @@ Born from the 2026 library audit (see the user memory `calibre-library-epubcheck
 - `src/bindery/epub.py`: archive rewrite, NCX uid sync, RepairReport, mismatch detection, and the opt-in structural-repair plumbing (including the CSS precondition scan).
 - `src/bindery/validate.py`: epubcheck wrapper, the `gate` (improvement) and `no_worse` (no-regression, for the lossy strips) acceptance bars.
 - `src/bindery/library.py`: Calibre walk, atomic replace, backups, and native format installation. Since v0.19.0 `CalibreIdResolver` resolves the book id from `metadata.db` through cquarry (one lazy path→id map per run; since v0.20.0 the map comes from `CalibreDB.format_path_index()`, re-normalized with `resolve().lower()` for the resolver's symlinked-directory and case-insensitive matching) — the `(id)` directory-name regex is only the no-catalog fallback. Since v0.24.0 `install_format()` places the repaired file atomically and updates the `data` row through `cquarry.write.WritableCalibreDB` (remove+add in one `batch()`; `add_format` refuses duplicates); the external `calibredb` CLI is gone. Never reintroduce directory-name guessing as the primary source: renamed/mismatched directories would replace the wrong book.
-- `src/bindery/cli.py`: `repair` and `library` subcommands, including `--all` and `--install-to-calibre`; since v0.29.0 also the `run` verbs — `phase1` (pre-import vetting over loose files: the audit battery composed with one gated repair sweep; read-only until `--apply-lossy`, which IS the lossy-strip consent) and `phase3` (the scoped post-import apply step: `library --id --sweep --only all --apply --all --install-to-calibre` plus a pre/post summary; mechanically refuses unscoped sweeps with exit 2). Both take `--non-interactive` and surface open questions as `decisions_needed` in their JSON, never prompts. They drive the shipped `run_library` through the real parser, so no flag can drift between the verb and the subcommand it wraps.
+- `src/bindery/cli.py`: `repair` and `library` subcommands, including `--all` and `--install-to-calibre`; since v0.29.0 also the `run` verbs — `phase1` (pre-import vetting over loose files: the audit battery composed with one gated repair sweep; read-only until `--apply-lossy`, which IS the lossy-strip consent) and `phase3` (the scoped post-import apply step: `library --id --sweep --only all --apply --all --install-to-calibre` plus a pre/post summary; mechanically refuses unscoped sweeps with exit 2). Both take `--non-interactive` and surface open questions as `decisions_needed` in their JSON, never prompts. They drive the shipped `run_library` through the real parser, so no flag can drift between the verb and the subcommand it wraps. `library --workers N` (v0.30.0) parallelizes only the sweep's candidate pass, in windows of N consumed in order so `--limit` stays lazy; the repair phase is never parallel (shared workdir, atomic-replacement contract).
 - `tests/`: transforms, end-to-end repair, atomic replace, pagination, watermarks, the audit analyzers (content/pagenumbers/emptytext/ocr/monolithic, archive integrity, spine classification), the library sweep and `CalibreIdResolver`, validate, reserialize, CLI wiring, and the version pin.
 
 ## Conventions

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: bindery-cli
-Version: 0.29.0
+Version: 0.30.0
 Summary: Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library.
 Author: vrnvctss
 License: MIT
@@ -196,6 +196,7 @@ bindery library ~/docs/Calibre\ Library --only all --apply --all --install-to-ca
 - `--id <ids>` limits the sweep to a comma-separated list of Calibre book IDs, skipping the full library walk.
 - `--audit CSV` (the `fatals,errors,warnings,path` format produced by an epubcheck sweep) skips clean books so a run is fast. Paths are resolved on both sides, and a CSV that matches nothing triggers a loud warning instead of silently selecting zero books.
 - `--sweep` replaces the CSV step entirely: it runs a live epubcheck sweep for candidate selection and reuses each result as that book's before-measurement, so no book is checked twice. bindery-cli first tries a persistent Java daemon (`EpubcheckDaemon`) to eliminate the per-book JVM startup; in practice the daemon cannot boot on most installs (its classpath carries only epubcheck's launcher jar, not the `lib/` dependencies epubcheck itself puts on the classpath), and the run silently falls back to the standard per-book epubcheck subprocess at the usual seconds per book. Budget accordingly. Fixing the classpath is a recorded decision (it would also change the counts the gate measures), not a shipped behavior. Combine with `--only fatals` for a self-contained "find and fix the broken books" run.
+- `--workers N` runs the sweep's candidate pass through N concurrent epubcheck workers (default 1: serial, unchanged). The subprocess releases the GIL, so threads parallelize the oracle honestly; books are checked in windows of N consumed in order, so the candidate set matches the serial sweep and `--limit` stays lazy within one window of overshoot. The repair phase stays serial: that is where the shared workdir and the atomic-replacement contract live.
 - `--json FILE` writes a machine-readable report of the whole run (per-book status, before/after counts, applied flag, summary totals). `--manual-list FILE` writes the paths of every book that was not auto-repaired, one per line, ready for manual follow-up.
 - `--apply` is required to write; the default is a dry run. `--backup DIR` mirrors originals before replacing; `--backup-inplace` writes `.epub.bak` beside each file.
 - `--install-to-calibre` resolves the Calibre book id from `metadata.db` via cquarry (one read-only path→id map per run), so a hand-renamed `Author/Title (id)/` directory can never send the repaired file to the wrong book. The directory-name regex is only a no-catalog fallback; with neither, the file is saved atomically in place.

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/README.md

@@ -174,6 +174,7 @@ bindery library ~/docs/Calibre\ Library --only all --apply --all --install-to-ca
 - `--id <ids>` limits the sweep to a comma-separated list of Calibre book IDs, skipping the full library walk.
 - `--audit CSV` (the `fatals,errors,warnings,path` format produced by an epubcheck sweep) skips clean books so a run is fast. Paths are resolved on both sides, and a CSV that matches nothing triggers a loud warning instead of silently selecting zero books.
 - `--sweep` replaces the CSV step entirely: it runs a live epubcheck sweep for candidate selection and reuses each result as that book's before-measurement, so no book is checked twice. bindery-cli first tries a persistent Java daemon (`EpubcheckDaemon`) to eliminate the per-book JVM startup; in practice the daemon cannot boot on most installs (its classpath carries only epubcheck's launcher jar, not the `lib/` dependencies epubcheck itself puts on the classpath), and the run silently falls back to the standard per-book epubcheck subprocess at the usual seconds per book. Budget accordingly. Fixing the classpath is a recorded decision (it would also change the counts the gate measures), not a shipped behavior. Combine with `--only fatals` for a self-contained "find and fix the broken books" run.
+- `--workers N` runs the sweep's candidate pass through N concurrent epubcheck workers (default 1: serial, unchanged). The subprocess releases the GIL, so threads parallelize the oracle honestly; books are checked in windows of N consumed in order, so the candidate set matches the serial sweep and `--limit` stays lazy within one window of overshoot. The repair phase stays serial: that is where the shared workdir and the atomic-replacement contract live.
 - `--json FILE` writes a machine-readable report of the whole run (per-book status, before/after counts, applied flag, summary totals). `--manual-list FILE` writes the paths of every book that was not auto-repaired, one per line, ready for manual follow-up.
 - `--apply` is required to write; the default is a dry run. `--backup DIR` mirrors originals before replacing; `--backup-inplace` writes `.epub.bak` beside each file.
 - `--install-to-calibre` resolves the Calibre book id from `metadata.db` via cquarry (one read-only path→id map per run), so a hand-renamed `Author/Title (id)/` directory can never send the repaired file to the wrong book. The directory-name regex is only a no-catalog fallback; with neither, the file is saved atomically in place.

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/patchnotes.md

@@ -1,4 +1,20 @@
 # bindery-cli Patch Notes
+## v0.30.0 (2026-09-06)
+
+### Phase 8 stretch: concurrent sweep workers
+
+- **`library --workers N`**: the `--sweep` candidate pass runs through N
+  concurrent epubcheck workers (default 1: serial, byte-for-byte unchanged).
+  epubcheck runs in a subprocess that releases the GIL, so threads parallelize
+  the oracle honestly; this is the pass that measured ~4.4 s/book on the
+  2026-08-27 full-library walk (5,070 EPUBs ≈ 6 hours serial). Books are
+  checked in windows of N consumed in input order, so the candidate set, the
+  before-measurements, and every emitted line match the serial sweep, and
+  `--limit` stays lazy within one window of overshoot. The repair phase stays
+  serial on purpose: that is where the shared workdir and the
+  atomic-replacement contract live. Without `--sweep`, `--workers` is a
+  no-op (with a note); counts below 1 are a usage error.
+
 ## v0.29.0 (2026-09-06)
 
 ### Phase 13: machine-readable audit and the acquisition run slices

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "bindery-cli"
-version = "0.29.0"
+version = "0.30.0"
 description = "Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library."
 readme = "README.md"
 classifiers = [

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/roadmap.md

@@ -510,12 +510,19 @@ by hand.*
   release.
 - [x] **`audit --id` comma lists**: v0.19.0 took a single book id; support the
   same comma-separated form as the new `library --id` so the two modes match.
-- [ ] **Concurrent validation workers (stretch; not shipped)**: the 2026-08-27 full-library
+- [x] **Concurrent validation workers (stretch; not shipped)**: the 2026-08-27 full-library
   walk measured ~4.4 s/book on a single daemon (5,070 EPUBs ≈ 6 h). Gate k
   validation workers behind `--workers N`, default 1 (behavior unchanged).
   *(Un-ticked in the 2026-09-02 docs re-read (NEW-AUDIT.md Stage 0): the box was
   ticked at the v0.23.0 ship, but no `--workers` code, test, or doc line ever
-  landed. Still open as a stretch goal.)*
+  landed.)*
+  *(Shipped for real in v0.30.0: `library --workers N` runs the --sweep
+  candidate pass through N concurrent epubcheck workers, default 1 (serial,
+  unchanged). Threads parallelize the oracle honestly (the subprocess releases
+  the GIL); books are checked in windows of N consumed in input order, so the
+  candidate set matches the serial sweep and `--limit` stays lazy within one
+  window of overshoot. The repair phase stays serial, where the shared workdir
+  and the atomic-replacement contract live.)*
 
 Non-goals: no auto-trim of bloated ToCs (semantics-preserving charter); no
 content-doc synthesis; no PDF equivalent.

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/spec.md

@@ -267,6 +267,11 @@ For a Calibre library (`Author/Title (id)/Title - Author.epub`):
   reported and counted as `unreadable`; it never aborts the rest of the sweep.
 - The `library` exit code is 0 for a clean sweep, 1 for a usage error, and 2 when any
   book was rejected, unreadable, or failed epubcheck, so scripts can detect trouble.
+- With `--sweep`, `--workers N` runs the candidate-selection epubcheck pass through N
+  concurrent workers (default 1: serial, unchanged). Books are checked in windows of N
+  consumed in input order, so the candidate set and the before-measurements are identical
+  to the serial sweep; the repair phase is never parallel (the shared workdir and the
+  atomic-replacement contract live there).
 
 ### Native format installation (`--install-to-calibre`)
 Optionally, bindery-cli installs the repaired EPUB as the book's format through cquarry's write module (`WritableCalibreDB`): the file is placed atomically — an in-place replace over the catalogued file when one exists (same path, same `data.name`), or a fresh placement under the repaired file's stem otherwise — and the `data` row follows in one `batch()` transaction (`remove_format` + `add_format`, since `add_format` refuses duplicates), keeping the size truthful and queuing the book in `metadata_dirtied` so Calibre regenerates its sidecar .opf. The external `calibredb` CLI is no longer used (the v0.23.1 `--replace` crash class is gone with it). It automatically falls back to atomic filesystem replacement if a valid Calibre ID cannot be extracted, and a database failure degrades to the in-place save with a warning rather than losing the repair.

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/src/bindery/__init__.py

@@ -4,7 +4,7 @@ strip), epubcheck-gated.
 Single source of truth for the version. pyproject.toml must match VERSION below.
 """
 
-VERSION = "0.29.0"
+VERSION = "0.30.0"
 __version__ = VERSION
 
 from .epub import RepairReport, repair_epub  # noqa: E402

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/src/bindery/cli.py

@@ -210,6 +210,51 @@ def _sweep_select(epubs, only: str, root: Path, checks: dict, *, quiet: bool):
         yield epub
 
 
+def _sweep_select_parallel(
+    epubs, only: str, root: Path, checks: dict, *, quiet: bool, workers: int
+):
+    """--workers N: k concurrent epubcheck workers over the sweep pass.
+
+    epubcheck runs in a subprocess that releases the GIL, so threads
+    parallelize the oracle honestly (this pass measured ~4.4 s/book on the
+    2026-08-27 full-library walk). Books are checked in windows of `workers`;
+    each window is consumed (and its keepers yielded) before the next one is
+    submitted, so a --limit still stops the sweep after at most one window of
+    overshoot instead of checking the whole tree. Within a window, results are
+    applied in input order, so the candidate set and every emitted line match
+    the serial sweep. The repair phase stays serial: that is where the shared
+    workdir and the atomic-replacement contract live."""
+    from concurrent.futures import ThreadPoolExecutor
+
+    todo = list(epubs)
+    if not todo:
+        return
+    with ThreadPoolExecutor(max_workers=workers) as ex:
+        done_count = 0
+        progress = (
+            None if quiet else tqdm(total=len(todo), desc="Sweeping", unit="book")
+        )
+        for start in range(0, len(todo), workers):
+            window = todo[start : start + workers]
+            futures = [ex.submit(run_epubcheck, epub) for epub in window]
+            for epub, fut in zip(window, futures, strict=True):
+                c = fut.result()
+                done_count += 1
+                if not quiet:
+                    tqdm.write(f"[sweep] {epub.relative_to(root)}", file=sys.stderr)
+                    progress.update(1)
+                if c is not None:
+                    checks[epub] = c
+                if only == "fatals":
+                    keep = c is None or c.fatals > 0
+                else:  # only == "all": skip clean books
+                    keep = c != CheckResult(0, 0, 0)
+                if keep:
+                    yield epub
+        if progress is not None:
+            progress.close()
+
+
 # Everything a run did not (or could not) auto-repair; the --manual-list export.
 _MANUAL_STATUSES = frozenset(
     {"nochange", "equal", "partial", "reject", "error", "unreadable"}
@@ -353,11 +398,31 @@ def run_library(args) -> int:
         all_epubs = list(iter_epubs(root))
     audit_hits: list[Path] = []
     checks: dict[Path, CheckResult] = {}
-    if args.sweep:
-        iterator = (
-            all_epubs if args.quiet else tqdm(all_epubs, desc="Sweeping", unit="book")
+    workers = getattr(args, "workers", 1)
+    if workers is None:
+        workers = 1
+    if workers < 1:
+        print(f"error: --workers must be >= 1, got {workers}", file=sys.stderr)
+        return 1
+    if not args.sweep and workers > 1:
+        print(
+            "note: --workers applies to the --sweep candidate pass; ignored here.",
+            file=sys.stderr,
         )
-        selected = _sweep_select(iterator, args.only, root, checks, quiet=args.quiet)
+    if args.sweep:
+        if workers > 1:
+            selected = _sweep_select_parallel(
+                all_epubs, args.only, root, checks, quiet=args.quiet, workers=workers
+            )
+        else:
+            iterator = (
+                all_epubs
+                if args.quiet
+                else tqdm(all_epubs, desc="Sweeping", unit="book")
+            )
+            selected = _sweep_select(
+                iterator, args.only, root, checks, quiet=args.quiet
+            )
     else:
         selected = _select(all_epubs, args.only, audit, audit_hits)
     if args.limit is not None:
@@ -1309,6 +1374,14 @@ def build_parser() -> argparse.ArgumentParser:
     lib.add_argument(
         "--limit", type=int, help="process at most N candidates (for sampling)"
     )
+    lib.add_argument(
+        "--workers",
+        type=int,
+        default=1,
+        metavar="N",
+        help="concurrent epubcheck workers for the --sweep candidate pass "
+        "(default 1: serial, unchanged). The repair phase stays serial",
+    )
     lib.add_argument(
         "--quiet",
         action="store_true",

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/tests/test_cli.py

@@ -1015,3 +1015,125 @@ class TestRunPhase3(unittest.TestCase):
         self.assertEqual(data["summary"]["repair"]["applied"], 1)
         self.assertEqual(data["decisions_needed"], [])
         self.assertIn("PHASE 3 SUMMARY", out.getvalue())
+
+
+class TestSweepWorkers(unittest.TestCase):
+    """--workers N: concurrent epubcheck workers over the --sweep candidate
+    pass (Phase 8 stretch). Default 1 is the serial sweep; N > 1 must select
+    the same candidates in the same order, reuse the sweep result as the
+    book's before-measurement, and keep --limit lazy within one window of
+    overshoot."""
+
+    def test_flag_defaults_to_one(self):
+        parser = build_parser()
+        self.assertEqual(parser.parse_args(["library", "/tmp"]).workers, 1)
+        self.assertEqual(
+            parser.parse_args(["library", "/tmp", "--workers", "4"]).workers, 4
+        )
+
+    def test_invalid_worker_count_is_a_usage_error(self):
+        err = io.StringIO()
+        with redirect_stdout(io.StringIO()), redirect_stderr(err):
+            rc = main(["library", "/tmp", "--workers", "0", "--no-validate"])
+        self.assertEqual(rc, 1)
+        self.assertIn("--workers", err.getvalue())
+
+    def test_parallel_sweep_selects_like_the_serial_one(self):
+        # a path-keyed oracle: threads complete in any order, but every
+        # book's result is pinned to the book, so determinism survives
+        def oracle(path):
+            return (
+                CheckResult(1, 0, 0) if path.name == "a.epub" else CheckResult(0, 0, 0)
+            )
+
+        with tempfile.TemporaryDirectory() as td:
+            for name in ("a", "b", "c"):
+                build(Path(td) / f"{name}.epub")
+            jout = Path(td) / "report.json"
+            out, err = io.StringIO(), io.StringIO()
+            with (
+                mock.patch("bindery.cli.epubcheck_available", return_value=True),
+                mock.patch("bindery.cli.run_epubcheck", side_effect=oracle),
+                redirect_stdout(out),
+                redirect_stderr(err),
+            ):
+                rc = main(
+                    [
+                        "library",
+                        td,
+                        "--sweep",
+                        "--only",
+                        "fatals",
+                        "--workers",
+                        "3",
+                        "--json",
+                        str(jout),
+                    ]
+                )
+            data = json.loads(jout.read_text())
+        self.assertEqual(rc, 0)
+        statuses = {Path(b["path"]).name: b["status"] for b in data["books"]}
+        self.assertEqual(list(statuses), ["a.epub"])
+        # the sweep result was reused as the before-measurement and the gate
+        # accepted the no-op-fix candidate exactly as the serial sweep would
+        self.assertEqual(statuses["a.epub"], "accept")
+        self.assertEqual(data["summary"]["accepted"], 1)
+
+    def test_limit_stays_lazy_within_one_window(self):
+        checked: list[str] = []
+
+        def oracle(path):
+            checked.append(path.name)
+            return CheckResult(1, 0, 0)
+
+        with tempfile.TemporaryDirectory() as td:
+            for name in ("a", "b", "c", "d", "e"):
+                build(Path(td) / f"{name}.epub")
+            out, err = io.StringIO(), io.StringIO()
+            with (
+                mock.patch("bindery.cli.epubcheck_available", return_value=True),
+                mock.patch("bindery.cli.run_epubcheck", side_effect=oracle),
+                redirect_stdout(out),
+                redirect_stderr(err),
+            ):
+                rc = main(
+                    [
+                        "library",
+                        td,
+                        "--sweep",
+                        "--only",
+                        "all",
+                        "--limit",
+                        "1",
+                        "--workers",
+                        "2",
+                    ]
+                )
+        self.assertEqual(rc, 0)
+        # the first window (a, b) is checked; the keeper limit of 1 stops the
+        # sweep there: at most one window of overshoot, never the whole tree.
+        # (2 sweep calls + 1 after-validation on the candidate: the fixture's
+        #   always gives the core pass a fix, so the candidate validates.)
+        self.assertEqual(len(checked), 3)
+
+    def test_serial_limit_checks_exactly_the_limit(self):
+        checked: list[str] = []
+
+        def oracle(path):
+            checked.append(path.name)
+            return CheckResult(1, 0, 0)
+
+        with tempfile.TemporaryDirectory() as td:
+            for name in ("a", "b", "c", "d", "e"):
+                build(Path(td) / f"{name}.epub")
+            out, err = io.StringIO(), io.StringIO()
+            with (
+                mock.patch("bindery.cli.epubcheck_available", return_value=True),
+                mock.patch("bindery.cli.run_epubcheck", side_effect=oracle),
+                redirect_stdout(out),
+                redirect_stderr(err),
+            ):
+                rc = main(["library", td, "--sweep", "--only", "all", "--limit", "1"])
+        self.assertEqual(rc, 0)
+        # 1 sweep call + 1 after-validation on the single candidate
+        self.assertEqual(len(checked), 2)

{bindery_cli-0.29.0 → bindery_cli-0.30.0}/uv.lock

@@ -4,7 +4,7 @@ requires-python = ">=3.14"
 
 [[package]]
 name = "bindery-cli"
-version = "0.29.0"
+version = "0.30.0"
 source = { editable = "." }
 dependencies = [
     { name = "cquarry" },