bindery-cli 0.34.0__tar.gz → 0.35.0__tar.gz

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: bindery-cli
-Version: 0.34.0
+Version: 0.35.0
 Summary: Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library.
 Author: vrnvctss
 License: MIT
@@ -82,7 +82,7 @@ Every repair is gated by [epubcheck](https://github.com/w3c/epubcheck). The acce
 
 Introducing a net-new fatal is always rejected. If epubcheck itself fails to run (crash, timeout, unparsable output), the book is reported as an error and never applied; only an explicit `--no-validate` skips the gate. Originals are never modified except by an explicit, atomic in-place replace (see below), and even then only after the gate accepts the result.
 
-Under the hood, the oracle can run two ways: the usual `epubcheck --json -` subprocess, or a small in-process daemon that compiles `FastDaemon.java` into a temp directory at runtime and drives a warm JVM over a pipe. The daemon is a pool bounded by `--workers` (serial runs use one), its every roundtrip is bounded by the caller's timeout, and any failure tears it down and falls back to the subprocess for good — the two paths must never disagree about a book, and the subprocess oracle is the fallback by design. The daemon is also the scariest under-tested code in the repo and depends on a matched javac/java pair; its status is recorded in the roadmap's Phase 12 postscript and Phase 14 backlog.
+Under the hood, the oracle can run two ways: the usual `epubcheck --json -` subprocess, or a persistent daemon that writes `FastDaemon.java` into a temp directory at runtime, compiles it in memory with the running JVM's own compiler (Java's single-file source launcher: no javac, no javac/java version skew), and drives the warm JVM over a pipe. The daemon reports the counts it reads from the JSON document that epubcheck's own `CheckingReport` serializes, so it measures exactly what the subprocess measures, just ~15x faster. It is a pool bounded by `--workers` (serial runs use one), every roundtrip is bounded by the caller's timeout, and any failure tears it down and falls back to the subprocess for good.
 
 The lossy modes (`--strip-pagination`, `--strip-broken-tags`, and `--strip-watermarks`) are the exception to the "must improve" rule. Since they remove visible markup rather than correcting XML schema violations, epubcheck counts often remain unchanged. They are accepted when the result is **no worse** (no net-new fatals or errors), relying on strict programmatic safety nets instead.
 

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/README.md

@@ -60,7 +60,7 @@ Every repair is gated by [epubcheck](https://github.com/w3c/epubcheck). The acce
 
 Introducing a net-new fatal is always rejected. If epubcheck itself fails to run (crash, timeout, unparsable output), the book is reported as an error and never applied; only an explicit `--no-validate` skips the gate. Originals are never modified except by an explicit, atomic in-place replace (see below), and even then only after the gate accepts the result.
 
-Under the hood, the oracle can run two ways: the usual `epubcheck --json -` subprocess, or a small in-process daemon that compiles `FastDaemon.java` into a temp directory at runtime and drives a warm JVM over a pipe. The daemon is a pool bounded by `--workers` (serial runs use one), its every roundtrip is bounded by the caller's timeout, and any failure tears it down and falls back to the subprocess for good — the two paths must never disagree about a book, and the subprocess oracle is the fallback by design. The daemon is also the scariest under-tested code in the repo and depends on a matched javac/java pair; its status is recorded in the roadmap's Phase 12 postscript and Phase 14 backlog.
+Under the hood, the oracle can run two ways: the usual `epubcheck --json -` subprocess, or a persistent daemon that writes `FastDaemon.java` into a temp directory at runtime, compiles it in memory with the running JVM's own compiler (Java's single-file source launcher: no javac, no javac/java version skew), and drives the warm JVM over a pipe. The daemon reports the counts it reads from the JSON document that epubcheck's own `CheckingReport` serializes, so it measures exactly what the subprocess measures, just ~15x faster. It is a pool bounded by `--workers` (serial runs use one), every roundtrip is bounded by the caller's timeout, and any failure tears it down and falls back to the subprocess for good.
 
 The lossy modes (`--strip-pagination`, `--strip-broken-tags`, and `--strip-watermarks`) are the exception to the "must improve" rule. Since they remove visible markup rather than correcting XML schema violations, epubcheck counts often remain unchanged. They are accepted when the result is **no worse** (no net-new fatals or errors), relying on strict programmatic safety nets instead.
 

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/patchnotes.md

@@ -1,4 +1,37 @@
 # bindery-cli Patch Notes
+## v0.35.0 (2026-09-10)
+
+### The epubcheck daemon, reconciled: oracle-identical counts at ~15x the speed
+
+- **FastDaemon v2 measures exactly what the subprocess oracle measures.**
+  The daemon's old Java side counted message occurrences from
+  `CheckingReport`'s counters, while the gate has always measured the
+  counts in epubcheck's own `--json` output, which aggregates identical
+  messages (one book: 243 occurrences, 4 aggregated messages). On the
+  staged-fixture corpus the two methods diverged on 11 of 14 books, so
+  the daemon could never be enabled without changing what the gate
+  measures. The new daemon calls `CheckingReport.generate()` and reads
+  `nFatal`/`nError`/`nWarning` out of the JSON epubcheck itself
+  serializes: identical numbers by construction. Verified against the
+  subprocess oracle on all 14 staged books and on a 100-book
+  real-library sample: 114/114 identical.
+- **No more javac, no more version skew.** The daemon now launches
+  through Java's single-file source launcher (JEP 330): the running JVM
+  compiles the daemon in memory with its own compiler, so the
+  javac-newer-than-java skew that kept the daemon dead on this machine
+  (javac 27-ea, java 25) cannot recur on any toolchain.
+- **Measured, working daemon throughput: ~0.27s/book warm against
+  ~4.2s/book for the subprocess oracle** (15.6x). With `--workers N`
+  each worker gets its own warm daemon from the bounded pool, and every
+  roundtrip stays bounded by the caller's timeout with the fail-safe
+  fallback to the subprocess oracle.
+- **The dedup semantics are pinned by a suite test:** a book engineered
+  with one defect repeated three times plus one different defect must
+  answer 2 errors (the aggregated count), not 4 occurrences (the old
+  daemon's answer). Skipped where no epubcheck/JDK exists, like the
+  other oracle tests. README and spec's gate sections describe the new
+  design.
+
 ## v0.34.0 (2026-09-10)
 
 ### Brandon's three calls: partial books are trouble, the contract reconciled, the daemon decision informed

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "bindery-cli"
-version = "0.34.0"
+version = "0.35.0"
 description = "Audit EPUB body text and repair broken EPUBs with deterministic, epubcheck-gated fixes; atomic in-place replacement in a Calibre library."
 readme = "README.md"
 classifiers = [

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/roadmap.md

@@ -1156,7 +1156,24 @@ can be net-neutral and ships silently.*
       javac at all), but the daemon should stay off until its counting
       matches the JSON oracle; retire-or-reconcile remains the open
       decision. The daemon already self-disables safely, and --workers
-      parallelism works via subprocesses.)**
+      parallelism works via subprocesses.)
+      *(DECIDED 2026-09-10: Brandon chose RECONCILE over retire. The
+      counting divergence turned out to run deeper than the daemon: the
+      CLI's human summary counts message occurrences (243 on one book)
+      while epubcheck's own --json output counts aggregated messages
+      (4 on the same book), and the gate has always measured the
+      aggregated scale. The reconciliation is therefore by construction:
+      FastDaemon v2 calls CheckingReport.generate() and reads nFatal/
+      nError/nWarning from the JSON epubcheck itself serializes, so the
+      daemon measures the subprocess oracle's numbers exactly. Verified
+      against the subprocess oracle on all 14 staged books (the same
+      corpus that diverged 11/14 under the old daemon) and on a
+      100-book real-library sample. The launch switched to Java's
+      single-file source launcher, removing javac and the version skew
+      permanently; the dedup semantics are pinned by a suite test
+      (skipped where no epubcheck/JDK exists). Daemon warm throughput
+      measured at ~0.27s/book vs ~4.2s/book subprocess. Shipped in
+      v0.35.0.)**
 - [x] **Smaller apply papercuts:** `--limit -1` and a missing
       `--audit` file raise tracebacks instead of usage errors
       (`cli.py:434`, `373`); the fresh-format branch of `install_format`

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/spec.md

@@ -239,10 +239,16 @@ the gate has not accepted anything, so nothing is applied or written. With
 `--no-validate`, the gate is skipped and repairs are trusted on the RepairReport alone.
 
 The oracle itself runs two ways: the `epubcheck --json -` subprocess, or a persistent
-daemon (`_EpubcheckDaemon`, a bounded pool sized by `--workers`) that compiles
-`FastDaemon.java` at runtime and drives a warm JVM over a pipe. Every daemon roundtrip
-is bounded by the caller's timeout, and any daemon failure tears it down and falls back
-to the subprocess for good, so the two paths can never disagree about a book.
+daemon (`_EpubcheckDaemon`, a bounded pool sized by `--workers`) that writes
+`FastDaemon.java` at runtime and launches it through Java's single-file source
+launcher (in-memory compilation by the running JVM: no javac, no version skew),
+driving the warm JVM over a pipe. The daemon answers with the counts it reads from
+the JSON that epubcheck's own `CheckingReport.generate()` serializes — the same
+document `--json` produces — so both paths measure identically by construction.
+(Epubcheck's JSON counts aggregated messages, not occurrences: a defect repeated
+three times is one error. The gate has always measured that scale.) Every daemon
+roundtrip is bounded by the caller's timeout, and any daemon failure tears it down
+and falls back to the subprocess for good.
 
 ## Page-number strip (opt-in, lossy)
 

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/src/bindery/__init__.py

@@ -4,7 +4,7 @@ strip), epubcheck-gated.
 Single source of truth for the version. pyproject.toml must match VERSION below.
 """
 
-VERSION = "0.34.0"
+VERSION = "0.35.0"
 __version__ = VERSION
 
 from .epub import RepairReport, repair_epub  # noqa: E402

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/src/bindery/validate.py

@@ -78,12 +78,39 @@ def _counts_from_json(stdout: str) -> CheckResult | None:
 
 _DAEMON_JAVA = """
 import java.io.File;
+import java.io.StringWriter;
 import java.io.PrintWriter;
 import java.util.Scanner;
 import com.adobe.epubcheck.api.EpubCheck;
 import com.adobe.epubcheck.reporting.CheckingReport;
 
 public class FastDaemon {
+    // Answers with the counts from the checker block of the JSON document
+    // that CheckingReport.generate() itself serializes: the exact JSON the
+    // epubcheck CLI's --json mode carries. Counting must match the
+    // subprocess oracle by construction, never by imitation: the human
+    // summary counts message occurrences while the JSON checker block
+    // counts aggregated messages, and the gate is calibrated on the latter.
+    static int field(String json, String name) {
+        int i = json.indexOf(name);
+        while (i != -1) {
+            int j = i + name.length();
+            while (j < json.length() && (json.charAt(j) == ' '
+                    || json.charAt(j) == ':' || json.charAt(j) == '"')) {
+                j++;
+            }
+            if (j < json.length() && Character.isDigit(json.charAt(j))) {
+                int k = j;
+                while (k < json.length() && Character.isDigit(json.charAt(k))) {
+                    k++;
+                }
+                return Integer.parseInt(json.substring(j, k));
+            }
+            i = json.indexOf(name, i + 1);
+        }
+        return -1;
+    }
+
     public static void main(String[] args) throws Exception {
         Scanner scanner = new Scanner(System.in);
         while (scanner.hasNextLine()) {
@@ -95,13 +122,20 @@ public class FastDaemon {
                 continue;
             }
             try {
-                PrintWriter out = new PrintWriter(new java.io.OutputStream() {
-                    public void write(int b) {}
-                });
-                CheckingReport report = new CheckingReport(out, epub.getName());
-                EpubCheck check = new EpubCheck(epub, report);
-                check.doValidate();
-                System.out.println(report.getFatalErrorCount() + "," + report.getErrorCount() + "," + report.getWarningCount());
+                StringWriter json = new StringWriter();
+                CheckingReport report = new CheckingReport(new PrintWriter(json),
+                        epub.getName());
+                report.initialize();
+                new EpubCheck(epub, report).doValidate();
+                report.generate();
+                int f = field(json.toString(), "nFatal");
+                int e = field(json.toString(), "nError");
+                int w = field(json.toString(), "nWarning");
+                if (f < 0 || e < 0 || w < 0) {
+                    System.out.println("-1,-1,-1");
+                } else {
+                    System.out.println(f + "," + e + "," + w);
+                }
             } catch (Exception e) {
                 System.out.println("-1,-1,-1");
             }
@@ -150,14 +184,12 @@ class _EpubcheckDaemon:
             with open(java_file, "w") as f:
                 f.write(_DAEMON_JAVA)
 
-            subprocess.run(
-                ["javac", "-cp", jar_path, java_file],
-                check=True,
-                capture_output=True,
-            )
-
             self._proc = subprocess.Popen(
-                ["java", "-cp", f".:{jar_path}", "FastDaemon"],
+                # JEP 330 single-file source launcher: the JVM compiles the
+                # daemon in memory with its own compiler, so there is no
+                # javac step and no javac/java version skew (which kept the
+                # daemon dead on machines where javac is newer than java).
+                ["java", "-cp", f".:{jar_path}", java_file],
                 cwd=self.workdir,
                 stdin=subprocess.PIPE,
                 stdout=subprocess.PIPE,
@@ -166,8 +198,8 @@ class _EpubcheckDaemon:
             atexit.register(self.stop)
             return True
         except Exception:
-            # Leave nothing behind: a failed compile or start must not leak
-            # the tempdir, and the failure is final for this instance.
+            # Leave nothing behind: a failed start must not leak the
+            # tempdir, and the failure is final for this instance.
             self.stop()
             return False
 

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/tests/test_validate.py

@@ -15,6 +15,7 @@ from __future__ import annotations
 import json
 import os
 import subprocess
+import tempfile
 import unittest
 import zipfile
 from pathlib import Path
@@ -22,7 +23,10 @@ from unittest import mock
 
 from bindery.validate import (
     CheckResult,
+    _counts_from_json,
     _DaemonPool,
+    _english_locale_env,
+    _EpubcheckDaemon,
     _readline_timeout,
     gate,
     no_worse,
@@ -205,6 +209,102 @@ class ReadlineTimeout(unittest.TestCase):
                 _readline_timeout(fh, 5)
 
 
+@unittest.skipUnless(_daemon_toolchain_available(), "epubcheck not installed")
+class DaemonMatchesJsonOracle(unittest.TestCase):
+    """The counting-parity contract, pinned on real books: the daemon must
+    answer with the SAME numbers the subprocess --json oracle produces,
+    because the gate is calibrated on those numbers. FastDaemon v2 extracts
+    them from the JSON document CheckingReport.generate() itself serializes,
+    so the match holds by construction; this test holds it empirically,
+    including on a book engineered with repeated identical defects (the
+    exact shape where the old occurrence-counting daemon diverged)."""
+
+    def setUp(self) -> None:
+        self.tmp = tempfile.TemporaryDirectory()
+        # A clean book carrying exactly two distinct defects, one of them
+        # repeated three times: the JSON oracle deduplicates the repeats, so
+        # the true count is 2 errors from 4 occurrences. The old
+        # occurrence-counting daemon answered 4 here; by-construction
+        # counting must answer 2.
+        path = Path(self.tmp.name) / "dedup.epub"
+        with zipfile.ZipFile(path, "w") as z:
+            z.writestr("mimetype", "application/epub+zip")
+            z.writestr(
+                "META-INF/container.xml",
+                '<container version="1.0" '
+                'xmlns="urn:oasis:names:tc:opendocument:xmlns:container">'
+                '<rootfiles><rootfile full-path="content.opf" '
+                'media-type="application/oebps-package+xml"/></rootfiles>'
+                "</container>",
+            )
+            z.writestr(
+                "content.opf",
+                '<package xmlns="http://www.idpf.org/2007/opf" version="2.0" '
+                'unique-identifier="bid"><metadata '
+                'xmlns:dc="http://purl.org/dc/elements/1.1/">'
+                "<dc:title>t</dc:title><dc:language>en</dc:language>"
+                '<dc:identifier id="bid">pub-id</dc:identifier></metadata>'
+                "<manifest>"
+                '<item id="c1" href="d.xhtml" '
+                'media-type="application/xhtml+xml"/>'
+                '<item id="ncx" href="toc.ncx" '
+                'media-type="application/x-dtbncx+xml"/></manifest>'
+                '<spine toc="ncx"><itemref idref="c1"/></spine></package>',
+            )
+            z.writestr(
+                "toc.ncx",
+                '<ncx xmlns="http://www.daisy.org/z3986/2005/ncx/" '
+                'version="2005-1"><head>'
+                '<meta name="dtb:uid" content="pub-id"/>'
+                '<meta name="dtb:depth" content="1"/>'
+                '<meta name="dtb:totalPageCount" content="0"/>'
+                '<meta name="dtb:maxPageNumber" content="0"/></head>'
+                "<docTitle><text>t</text></docTitle><navMap>"
+                '<navPoint id="n1" playOrder="1"><navLabel><text>c</text>'
+                "</navLabel>"
+                '<content src="d.xhtml"/></navPoint></navMap></ncx>',
+            )
+            z.writestr(
+                "d.xhtml",
+                '<html xmlns="http://www.w3.org/1999/xhtml"><head><title>t'
+                "</title></head><body>"
+                '<p id="1:x">a</p><p id="2:x">b</p><p id="3:x">c</p>'
+                "<bogus>x</bogus>"
+                "</body></html>",
+            )
+        self.book = path
+
+    def tearDown(self) -> None:
+        self.tmp.cleanup()
+
+    def test_daemon_counts_match_the_subprocess_json_oracle(self) -> None:
+        import subprocess
+
+        d = _EpubcheckDaemon()
+        try:
+            daemon = d.check(self.book, timeout=180)
+            self.assertIsInstance(daemon, CheckResult)
+            out = subprocess.run(
+                ["epubcheck", str(self.book), "--json", "-"],
+                capture_output=True,
+                text=True,
+                timeout=180,
+                env=_english_locale_env(),
+                check=False,
+            )
+            oracle = _counts_from_json(out.stdout)
+            self.assertEqual(
+                (daemon.fatals, daemon.errors, daemon.warnings),
+                (oracle.fatals, oracle.errors, oracle.warnings),
+            )
+            # the dedup pin: three identical id defects plus one different
+            # error are 4 occurrences but 2 aggregated messages, and the
+            # daemon reports the aggregated count, not the occurrences
+            self.assertEqual(daemon.errors, 2)
+        finally:
+            d.stop()
+
+
 class DaemonPoolLogic(unittest.TestCase):
     """The pool's degrade-to-subprocess logic, hermetic (no JVM needed).
 

{bindery_cli-0.34.0 → bindery_cli-0.35.0}/uv.lock

@@ -4,7 +4,7 @@ requires-python = ">=3.14"
 
 [[package]]
 name = "bindery-cli"
-version = "0.34.0"
+version = "0.35.0"
 source = { editable = "." }
 dependencies = [
     { name = "cquarry" },