ebookerr-sdk 3.2.0__tar.gz → 3.3.0__tar.gz

{ebookerr_sdk-3.2.0 → ebookerr_sdk-3.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ebookerr-sdk
-Version: 3.2.0
+Version: 3.3.0
 Summary: The ebookerr plugin SDK: the plugin contract, the plugin host, a test kit, the pack tool and helper kits.
 Author: ArcCdr
 License-Expression: MIT
@@ -41,10 +41,10 @@ validation, library servers and text generation.
 ## Install
 
 ```bash
-pip install "ebookerr-sdk==3.2.*"
+pip install "ebookerr-sdk==3.3.*"
 ```
 
-The SDK's MAJOR.MINOR is the SPI version it speaks: `ebookerr-sdk` 3.2.x speaks SPI 3.2. It needs
+The SDK's MAJOR.MINOR is the SPI version it speaks: `ebookerr-sdk` 3.3.x speaks SPI 3.3. It needs
 Python 3.11 or later.
 
 ## Start a plugin
@@ -66,6 +66,7 @@ or `catalog`. The guide explains what to write next and how to install the zip i
 | 2.24 | 3.0 | 3.0.x |
 | 2.24.1 | 3.1 | 3.1.x |
 | 2.24.7 | 3.2 | 3.2.x |
+| 2.24.9 | 3.3 | 3.3.x |
 
 ebookerr loads a plugin when its `spi_version` has ebookerr's SPI MAJOR and a MINOR not above
 ebookerr's; otherwise it shows the plugin as incompatible and offers a compatible release.
@@ -93,6 +94,11 @@ ebookerr's; otherwise it shows the plugin as incompatible and offers a compatibl
 
 ## Release notes
 
+### 3.3.0
+
+SPI 3.3. The EPUB kit gains helpers for the EPUBs ebookerr writes; later sentences of this
+section name each one. A plugin written for SPI 3.2 needs no change. `build_epub` writes its package through `ebookerr_sdk.epub.builder`: the NCX `dtb:uid` is the OPF identifier and the package carries its creation date, so a built book validates without findings. `EpubDocument.set_title` writes a new title into the package and every table of contents (`NcxDocument.set_doc_title`, `NavDocument.set_doc_title`), so a retitled book's OPF and NCX agree. `retitle_built_epub` gives a `build_epub` output a new title in its package, tables of contents and title page; `convert_stage` and `download_convert_stage` use it, so a converted book's file carries the title the book gets. `ebookerr_sdk.epub.text_convert.is_chapter_heading` names the prose chapter-heading rule. `convert_stage` titles an uploaded document after its own title, else its file name unless the name is a placeholder such as `Document1` or `Untitled`, else a first line that is no chapter heading. `Book.front_matter` holds the text above a document's first heading; `build_epub` puts it on the title page, and `parse_text_to_book` fills it instead of making that text a chapter. `parse_text_to_book` keeps each line of a text that has no blank line at all as its own paragraph. `convert_stage` and `download_convert_stage` pass a `user_facing` conversion error's message through unchanged and answer "The file could not be converted." for any other failure, so no library message or path reaches a user. `EpubDocument.drm_protected()` tells a DRM-protected book (a `rights.xml`, or an `encryption.xml` using anything but font obfuscation) from one whose fonts are only obfuscated.
+
 ### 3.2.0
 
 SPI 3.2. A story can name the multi-part work it is one part of: `StoryPatch.work` and

{ebookerr_sdk-3.2.0 → ebookerr_sdk-3.3.0}/README.md

@@ -12,10 +12,10 @@ validation, library servers and text generation.
 ## Install
 
 ```bash
-pip install "ebookerr-sdk==3.2.*"
+pip install "ebookerr-sdk==3.3.*"
 ```
 
-The SDK's MAJOR.MINOR is the SPI version it speaks: `ebookerr-sdk` 3.2.x speaks SPI 3.2. It needs
+The SDK's MAJOR.MINOR is the SPI version it speaks: `ebookerr-sdk` 3.3.x speaks SPI 3.3. It needs
 Python 3.11 or later.
 
 ## Start a plugin
@@ -37,6 +37,7 @@ or `catalog`. The guide explains what to write next and how to install the zip i
 | 2.24 | 3.0 | 3.0.x |
 | 2.24.1 | 3.1 | 3.1.x |
 | 2.24.7 | 3.2 | 3.2.x |
+| 2.24.9 | 3.3 | 3.3.x |
 
 ebookerr loads a plugin when its `spi_version` has ebookerr's SPI MAJOR and a MINOR not above
 ebookerr's; otherwise it shows the plugin as incompatible and offers a compatible release.
@@ -64,6 +65,11 @@ ebookerr's; otherwise it shows the plugin as incompatible and offers a compatibl
 
 ## Release notes
 
+### 3.3.0
+
+SPI 3.3. The EPUB kit gains helpers for the EPUBs ebookerr writes; later sentences of this
+section name each one. A plugin written for SPI 3.2 needs no change. `build_epub` writes its package through `ebookerr_sdk.epub.builder`: the NCX `dtb:uid` is the OPF identifier and the package carries its creation date, so a built book validates without findings. `EpubDocument.set_title` writes a new title into the package and every table of contents (`NcxDocument.set_doc_title`, `NavDocument.set_doc_title`), so a retitled book's OPF and NCX agree. `retitle_built_epub` gives a `build_epub` output a new title in its package, tables of contents and title page; `convert_stage` and `download_convert_stage` use it, so a converted book's file carries the title the book gets. `ebookerr_sdk.epub.text_convert.is_chapter_heading` names the prose chapter-heading rule. `convert_stage` titles an uploaded document after its own title, else its file name unless the name is a placeholder such as `Document1` or `Untitled`, else a first line that is no chapter heading. `Book.front_matter` holds the text above a document's first heading; `build_epub` puts it on the title page, and `parse_text_to_book` fills it instead of making that text a chapter. `parse_text_to_book` keeps each line of a text that has no blank line at all as its own paragraph. `convert_stage` and `download_convert_stage` pass a `user_facing` conversion error's message through unchanged and answer "The file could not be converted." for any other failure, so no library message or path reaches a user. `EpubDocument.drm_protected()` tells a DRM-protected book (a `rights.xml`, or an `encryption.xml` using anything but font obfuscation) from one whose fonts are only obfuscated.
+
 ### 3.2.0
 
 SPI 3.2. A story can name the multi-part work it is one part of: `StoryPatch.work` and

{ebookerr_sdk-3.2.0 → ebookerr_sdk-3.3.0}/ebookerr_sdk/__init__.py

@@ -44,4 +44,4 @@ ruff D, pdoc, mutmut and the traceability gate all cover the SDK; it has its own
   ``pack/``, so a fetched index and a built one are validated by exactly the same rules.
 """
 
-__version__ = "3.2.0"
+__version__ = "3.3.0"

{ebookerr_sdk-3.2.0 → ebookerr_sdk-3.3.0}/ebookerr_sdk/download/document.py

@@ -9,6 +9,7 @@ from __future__ import annotations
 
 import hashlib
 import logging
+import re
 from collections.abc import Callable, Mapping
 from contextlib import AbstractContextManager, nullcontext
 from pathlib import Path
@@ -26,7 +27,9 @@ from ebookerr_sdk.download.check import (
 from ebookerr_sdk.download.disposition import parse_content_disposition_filename
 from ebookerr_sdk.download.identity import USER_AGENT as _UA
 from ebookerr_sdk.download.paths import safe_component as _safe_component
-from ebookerr_sdk.epub.build import ConversionResult, DocumentConversionError
+from ebookerr_sdk.epub.build import ConversionResult, DocumentConversionError, retitle_built_epub
+from ebookerr_sdk.epub.errors import EpubError
+from ebookerr_sdk.epub.text_convert import is_chapter_heading
 from ebookerr_sdk.spi import (
     BookPatch,
     BookView,
@@ -41,6 +44,14 @@ from ebookerr_sdk.spi import (
 
 _JUNK_AUTHORS = frozenset({"windows user", "user", "administrator", "unknown"})
 _MAX_FIRST_LINE_TITLE = 120
+_COULD_NOT_CONVERT = "The file could not be converted."
+"""The reason a failed conversion gives when the converter named no cause (EXP-297)."""
+_JUNK_FILE_NAME_RE = re.compile(
+    r"(?:(?:new )?(?:microsoft word |text )?document|doc|untitled(?: document)?"
+    r"|scan|img|image|dsc|pxl)[\s_-]*[\d_]*(?:\s*\(\d+\))?",
+    re.IGNORECASE,
+)
+"""A file stem nobody chose: Document1, Untitled 2, New Text Document (3), scan_0023 (EXP-304)."""
 
 
 def _refusal(host: str, prior: BookView | None) -> str:
@@ -95,10 +106,58 @@ def _screen_author(raw: str | None, log: logging.Logger) -> str:
     return author
 
 
+def _is_junk_file_name(stem: str) -> bool:
+    """Whether *stem* is a name nobody chose for a document (``EXP-304``).
+
+    Args:
+        stem: A file name without its extension.
+
+    Returns:
+        True for names like ``Document1``, ``Untitled``, ``New Document``, ``scan_0023``.
+    """
+    return bool(_JUNK_FILE_NAME_RE.fullmatch(stem.strip()))
+
+
+def _upload_title(result: ConversionResult, stem: str, log: logging.Logger) -> str:
+    """Return an uploaded document's title (``EXP-304``).
+
+    Its own title (not the converter's file-name fallback); else the file name the user gave
+    it, unless that name names nothing (:func:`_is_junk_file_name`); else its first non-empty
+    line when that is no chapter heading and at most ``_MAX_FIRST_LINE_TITLE`` characters; else
+    the file name after all. The pull path keeps :func:`_document_title` (``EXP-224``).
+
+    Args:
+        result: What the converter reported.
+        stem: The uploaded file's name without its extension.
+        log: The plugin's logger (the choice is logged at DEBUG).
+
+    Returns:
+        The sanitised title.
+    """
+    embedded = "" if result.title_from_filename else sanitize(result.title)
+    if embedded:
+        return embedded
+    name = sanitize(stem) or stem
+    if not _is_junk_file_name(name):
+        log.debug("Title taken from the file name: %r", name)
+        return name
+    first_line = sanitize(result.first_line or "") or ""
+    usable = len(first_line) <= _MAX_FIRST_LINE_TITLE and not is_chapter_heading(first_line)
+    if first_line and usable:
+        log.debug(
+            "Title taken from the first line %r (the file name %r names nothing)", first_line, name
+        )
+        return first_line
+    log.debug("Title fell back to the file name %r (no document title, no usable first line)", name)
+    return name
+
+
 def _document_title(
     result: ConversionResult, fallback: str, log: logging.Logger, *, fallback_label: str
 ) -> str:
-    """Return the converted document's title by the ``EXP-224`` precedence.
+    """Return the converted document's title by the ``EXP-224`` pull-path precedence.
+
+    Used by the pull path only; an upload is titled by :func:`_upload_title`.
 
     The document's own title (ignored when the converter says it is only the file name), then
     its first non-empty line when it is at most ``_MAX_FIRST_LINE_TITLE`` characters, then
@@ -132,6 +191,67 @@ def _document_title(
     return title
 
 
+def _write_chosen_title(epub_path: Path, built: str, title: str, log: logging.Logger) -> None:
+    """Write the stage's chosen *title* into the EPUB when the converter built another (EXP-304).
+
+    Args:
+        epub_path: The converted EPUB, still in its work folder.
+        built: The title the converter built the EPUB with (``ConversionResult.title``).
+        title: The title the stage chose for the book.
+        log: The plugin's logger.
+    """
+    if title == built:
+        return
+    try:
+        retitle_built_epub(epub_path, title)
+    except EpubError as exc:
+        log.warning("Could not write the title %r into %s: %s", title, epub_path.name, exc)
+        return
+    log.debug("Retitled the converted EPUB from %r to %r", built, title)
+
+
+def _run_converter(
+    convert: Callable[[Path, Path], ConversionResult],
+    src: Path,
+    epub_path: Path,
+    log: logging.Logger,
+    log_label: str,
+) -> ConversionResult:
+    """Run the converter; every failure becomes a ``SourcePullError`` with a sentence (EXP-297).
+
+    A ``DocumentConversionError`` whose class sets ``user_facing = True`` carries the
+    converter's own sentence (damaged, password-protected, no text) and is passed through as
+    it is. Any other ``DocumentConversionError`` is logged at WARNING with its message, any
+    other exception at ERROR with its traceback (a converter defect); both answer
+    :data:`_COULD_NOT_CONVERT`. A library's message, an exception class name or a path never
+    reaches the reason.
+
+    Args:
+        convert: The converter, ``(source_path, epub_path) -> ConversionResult``.
+        src: The document to convert.
+        epub_path: Where the EPUB is written.
+        log: The plugin's logger.
+        log_label: The format's name in log lines (e.g. ``"PDF"``).
+
+    Returns:
+        What the converter reported.
+
+    Raises:
+        SourcePullError: The conversion failed; its message is a sentence for the user.
+    """
+    try:
+        return convert(src, epub_path)
+    except DocumentConversionError as exc:
+        if getattr(exc, "user_facing", False) is True:
+            log.debug("%s conversion of %s refused: %s", log_label, src.name, exc)
+            raise SourcePullError(str(exc)) from exc
+        log.warning("%s conversion of %s failed: %s", log_label, src.name, exc)
+        raise SourcePullError(_COULD_NOT_CONVERT) from exc
+    except Exception as exc:
+        log.exception("%s conversion of %s failed unexpectedly", log_label, src.name)
+        raise SourcePullError(_COULD_NOT_CONVERT) from exc
+
+
 def _stage_under_author(work_dir: Path, epub_filename: str, author: str) -> str:
     """Move ``work_dir/epub_filename`` into its author folder and return the relative path.
 
@@ -178,6 +298,8 @@ def download_convert_stage(  # noqa: C901 — download/verify/convert/stage step
     then finally the basename's stem. The author is
     screened against a junk-author list (``windows user``, ``user``, ``administrator``,
     ``unknown``); if matched (case-insensitive) or empty, it becomes ``"Unknown"``.
+    The chosen title is written into the EPUB (package, table of contents, title page) when
+    the converter built it with another one.
 
     Args:
         url: URL to download
@@ -200,7 +322,9 @@ def download_convert_stage(  # noqa: C901 — download/verify/convert/stage step
         URL is the download URL itself — a one-file download is one chapter at one address.
 
     Raises:
-        SourcePullError: On download, conversion, or staging errors.
+        SourcePullError: On download, conversion, or staging errors; a conversion failure's
+            message is the converter's own sentence, else "The file could not be converted."
+            (EXP-297).
         ContentTypeMismatchError: When the server's Content-Type contradicts the claimed format.
         TaskCancelled: If the user cancelled the task at one of the two checkpoints.
     """
@@ -346,11 +470,8 @@ def download_convert_stage(  # noqa: C901 — download/verify/convert/stage step
     file_size = src_path.stat().st_size
     log.debug("Converting %s to EPUB: %s (%d bytes)", log_label, src_path.name, file_size)
 
-    # Convert to EPUB
-    try:
-        result = convert(src_path, work_dir / epub_filename)
-    except DocumentConversionError as exc:
-        raise SourcePullError(f"{log_label} conversion failed: {exc}") from exc
+    # Convert to EPUB; a failure is a sentence, never a library message (EXP-297)
+    result = _run_converter(convert, src_path, work_dir / epub_filename, log, log_label)
 
     # Author, title (EXP-224) and the staged path — the rules convert_stage shares
     author = _screen_author(result.author, log)
@@ -358,6 +479,7 @@ def download_convert_stage(  # noqa: C901 — download/verify/convert/stage step
         title = sanitize(Path(cd_name).stem) or epub_stem
     else:
         title = _document_title(result, epub_stem, log, fallback_label="URL path stem")
+    _write_chosen_title(work_dir / epub_filename, result.title, title, log)
     output_filename = _stage_under_author(work_dir, epub_filename, author)
 
     # Clean up downloaded source
@@ -431,8 +553,10 @@ def convert_stage(
     The converter Sources' ``convert`` (SPI 2.33) calls this for a file the user uploaded. It
     hashes *path* (the same ``content_hash``/``content_length`` custom values the download path
     writes, under *namespace*), converts it into ``work_dir/<file stem>.epub``, and stages that
-    under the author folder with the download path's rules (author screening, the ``EXP-224``
-    title precedence with the file name as the last resort). *path* is only read.
+    under the author folder with the download path's author screening and the ``EXP-304`` upload
+    title precedence (``_upload_title``). The chosen title is written into
+    the EPUB (package, table of contents, title page) when the converter built it with another
+    one. *path* is only read.
 
     Args:
         path: The uploaded file; its stem is the title of last resort.
@@ -449,8 +573,8 @@ def convert_stage(
         *work_dir*.
 
     Raises:
-        SourcePullError: The converter raised ``DocumentConversionError`` (its message is kept,
-            prefixed ``"<log_label> conversion failed: "``).
+        SourcePullError: The conversion failed; its message is the converter's own sentence,
+            else "The file could not be converted." (EXP-297).
     """
     # SDK host bridges plugin.* loggers into the core as the plugin's lines.
     log = logging.getLogger(f"plugin.{namespace}")
@@ -459,12 +583,10 @@ def convert_stage(
     epub_stem = _safe_component(path.stem)
     epub_filename = f"{epub_stem}.epub"
     log.debug("Converting %s to EPUB: %s (%d bytes)", log_label, path.name, size)
-    try:
-        result = convert(path, work_dir / epub_filename)
-    except DocumentConversionError as exc:
-        raise SourcePullError(f"{log_label} conversion failed: {exc}") from exc
+    result = _run_converter(convert, path, work_dir / epub_filename, log, log_label)
     author = _screen_author(result.author, log)
-    title = _document_title(result, epub_stem, log, fallback_label="file name")
+    title = _upload_title(result, epub_stem, log)
+    _write_chosen_title(work_dir / epub_filename, result.title, title, log)
     output_filename = _stage_under_author(work_dir, epub_filename, author)
     log.info(
         'Converted %s "%s" (%d chapters) from %s',