epub-translator 0.1.1__py3-none-any.whl → 0.1.3__py3-none-any.whl

epub_translator/__init__.py

@@ -1,5 +1,5 @@
 from . import language
 from .llm import LLM
-from .translator import translate
+from .translator import FillFailedEvent, translate
 
-__all__ = ["LLM", "translate", "language"]
+__all__ = ["LLM", "translate", "language", "FillFailedEvent"]

epub_translator/data/fill.jinja

@@ -1,66 +1,171 @@
-You must fill translated text into XML template while preserving ALL tag structure.
+You are an XML structure validator. Your ONLY task is to preserve the exact XML structure from the template while filling in translated text.
 
-CRITICAL: Every tag with id="X" in XML template MUST appear in your output.
+CRITICAL RULES:
+
+1. Structure Preservation: The output XML MUST have the EXACT SAME structure as the template
+   - Same tags in the same order
+   - Same nesting hierarchy
+   - Same attributes (especially id attributes)
+
+   IMPORTANT: Translation fluency is SECONDARY to structure preservation.
+   If the translated text flows naturally but doesn't match template structure,
+   you MUST break the flow to insert required tags.
+
+2. ID Handling:
+   - Tags WITH id="X": Disambiguation markers for structurally similar elements
+   - Tags WITHOUT id: Structurally unique, match by position and tag name
+   - NEVER add, remove, or change id attributes
+
+3. Text Filling Strategy:
+   - Compare source text with translated text
+   - Identify how source maps to template structure
+   - Apply the same mapping to translated text
+   - Preserve paragraph breaks (elements are natural separators)
+   - IMPORTANT: Translation may change word order - use SEMANTIC matching, not position
 
 ---
 
-INPUT FORMAT:
+COMMON ERRORS TO AVOID:
+
+Error Type 1: Missing expected blocks
+❌ WRONG: Omitting elements with id attributes
+✓ CORRECT: Every <tag id="X"> in template MUST appear in output
+
+Error Type 2: Tag count mismatch for non-id elements
+Example template:
+<p id="1">
+  <span>text1</span>
+  <span>text2</span>
+</p>
+
+❌ WRONG: <p id="1"><span>merged text</span></p>  (only 1 span, expected 2)
+✓ CORRECT: <p id="1"><span>text1</span><span>text2</span></p>
+
+Error Type 3: Adding unexpected IDs
+❌ WRONG: Adding id="99" to a tag that didn't have an id in template
+✓ CORRECT: If template has <span>text</span>, output should be <span>译文</span> (no id)
+
+Error Type 4: Wrong tag names
+❌ WRONG: Changing <em id="5"> to <i id="5">
+✓ CORRECT: Keep exact tag name from template
+
+Error Type 5: Missing ID on required elements
+❌ WRONG: <span>text</span> when template has <span id="5">text</span>
+✓ CORRECT: <span id="5">译文</span>
+
+Error Type 6: Wrong text mapping when word order changes
+Example 1: Template has "reviewer of <span id="5">Book</span> in <span id="6">Journal</span>"
+Translation: "Journal 上对 Book 的评论者"
+
+❌ WRONG: Journal 上对 <span id="5">Book</span> 的评论者<span id="6">Journal</span>
+  (appending original text at end)
+✓ CORRECT: <span id="6">Journal</span> 上对 <span id="5">Book</span> 的评论者
+  (wrapping semantic equivalents in translated positions)
+
+Example 2: Breaking fluent translation to preserve structure
+Template: "published in <span id="5">Book Title</span> in 1990"
+Translation: "于1990年出版的《书名》" (flows naturally, but loses structure)
+
+❌ WRONG: 于1990年出版的《书名》 (fluent but missing <span id="5">)
+✓ CORRECT: 于1990年出版的<span id="5">《书名》</span>
+  (Break fluency to preserve structure - this is REQUIRED)
+
+Error Type 7: Wrong semantic matching when word order changes
+When translation changes word order, match elements by SEMANTIC TYPE, not position.
 
-You will receive:
-1. Source text (original language)
-2. XML template (with id attributes and structure)
-3. Translated text (target language)
+Example: Book title and year
+Template: "<span id="3">Book Title</span> in <span id="4"><a>1990</a></span>"
+Translation: "《书名》于1990年出版"
 
-Your task: Map the translated text back into the XML structure by comparing with source text.
+❌ WRONG: 《书名》于<span id="3">1990</span>年出版...
+  (Matching by position: "1990" appears after "于", so wrapping it with id="3")
+  (WRONG because you matched a YEAR to a slot expecting BOOK TITLE)
+
+✓ CORRECT: <span id="3">《书名》</span>于<span id="4"><a>1990</a></span>年出版
+  (Matching by semantic type: book title → book title, year → year)
+
+KEY PRINCIPLE: Semantic type matching beats position matching!
+- Identify semantic types: book titles, journal names, years (4-digit numbers), person names, etc.
+- Match each to its corresponding slot, regardless of position in translation
+- data-orig-len hints at length: book/journal titles usually longer than years/numbers
 
 ---
 
-CRITICAL RULE - Tags with NO tail:
+FILLING ALGORITHM:
 
-When a tag has NO text after its closing tag in XML template:
-→ Put ALL remaining text INSIDE that tag
+1. Analyze template structure: count elements at each level, note id attributes
 
-EXAMPLE (MOST COMMON ERROR):
+2. Segment source text by elements (elements are natural separators)
 
-XML template:
-```xml
-<a id="1">
-  <span id="2">4</span>
-  The methodology of
-  <span id="3">Robotics</span>
-</a>
-```
+3. Apply to translation - STRICT STRUCTURAL MATCHING:
 
-Source: "4 The methodology of Robotics"
-Translated: "4 机器人学的方法论"
+   A. For elements WITH id:
+      - Locate semantic equivalent in translation
+      - Wrap with same tag+id
+      - If translation merged multiple spans: You MUST still output all original spans separately
+        Example: Template has id="1" and id="2", translation merged both
+        → Output BOTH spans, use source text fallback for missing one
 
-ANALYSIS:
-- <span id="3"> has NO text after </span> (the </a> comes immediately)
-- So ALL remaining text after id="2" must go into id="3"
+   B. For elements WITHOUT id:
+      - Match by STRUCTURAL POSITION only (template order)
+      - Count MUST be exact: 7 spans in template = 7 spans in output
+      - Even if content repeats (e.g., 3 instances of "x"), each gets its own span
+      - Process sequentially: wrap 1st occurrence with 1st span, 2nd with 2nd span, etc.
+      - DO NOT merge, skip, or add extra elements
 
-CORRECT output:
-```xml
-<a id="1">
-  <span id="2">4</span>
+      CRITICAL for repeated content:
+      If template has: "...<span>Word</span>...more text...<span>Word</span>"
+      And translation has: "...词...更多文字...词"
+      → Wrap 1st occurrence of "词" with 1st span, 2nd occurrence with 2nd span
+      → Even if the words are identical, treat each span position independently
 
-  <span id="3">机器人学的方法论</span>
-</a>
-```
+4. Verify: same element counts, all ids preserved, tag names match
 
-WRONG outputs:
-❌ <a id="1"><span id="2">4</span> 机器人学的方法论</a>
-   (Missing id="3" completely!)
+CRITICAL: Template structure is LAW. Translation fluency is secondary.
+
+---
 
-❌ <a id="1"><span id="2">4</span> <span id="3">机器人学</span>的方法论</a>
-   (Added tail "的方法论" to id="3", but source has no tail!)
+SPECIAL CASES:
+
+1. data-orig-len attribute: Token count hint. Longer counts usually = titles/names, shorter = numbers/symbols.
+
+2. Name+Number as single unit (e.g., "JournalName42" with NO space):
+   - In translation, find the name's equivalent and any adjacent number
+   - Wrap them together: <span id="X">《期刊名》42</span> or <span id="X">《期刊名》第42期</span>
+   - Key: If template treats them as one span, keep them in one span in translation
+
+3. Translation merges adjacent spans:
+   Template: "<span id="A">Word1</span> & <span id="B">Word2</span>"
+   Translation: "复合词" (one inseparable term)
+
+   Solution: You MUST output BOTH spans even if translation merged them
+   - Try to split translation if possible
+   - If truly inseparable: Keep translation for one span, use source text for the other
+   - Example: <span id="A">复合词</span>与<span id="B">Word2</span>
+
+4. Missing semantic match:
+   - Exhaust all possibilities first (synonyms, paraphrases, context)
+   - Last resort: Use source text as fallback
+   - Mixed language is acceptable to preserve structure
+
+WRONG fallback approaches:
+❌ Empty: <span id="2"></span>
+❌ Placeholder: <span id="2">内容</span>
+❌ Duplicate: <span id="2">中文名称</span> (when id="1" has this)
 
 ---
 
 OUTPUT FORMAT:
 ```xml
 <xml>
-  ... your output ...
+  ... your filled XML here ...
 </xml>
 ```
 
+CRITICAL:
+- Return ONLY the XML block, no explanations
+- Do NOT include example blocks or alternatives
+- If unsure, make best attempt based on pattern
+- System will provide detailed error messages if corrections needed
+
 Begin.

epub_translator/epub/__init__.py

@@ -1,4 +1,4 @@
-from .placeholder import Placeholder, is_placeholder_tag
+from .metadata import read_metadata, write_metadata
 from .spines import search_spine_paths
 from .toc import read_toc, write_toc
 from .zip import Zip

epub_translator/epub/metadata.py

@@ -0,0 +1,122 @@
+from dataclasses import dataclass
+
+from .common import find_opf_path
+from .zip import Zip
+
+
+@dataclass
+class MetadataField:
+    """
+    表示 EPUB OPF 文件中的元数据字段
+
+    - tag_name: 标签名（不带命名空间）
+    - text: 文本内容
+    """
+
+    tag_name: str
+    text: str
+
+
+# 不应该被翻译的元数据字段
+SKIP_FIELDS = {
+    "language",
+    "identifier",
+    "date",
+    "meta",
+    "contributor",  # Usually technical information
+}
+
+
+def read_metadata(zip: Zip) -> list[MetadataField]:
+    """
+    从 EPUB 的 OPF 文件中读取所有可翻译的元数据字段。
+
+    返回包含标签名和文本内容的列表。
+    自动过滤掉不应该翻译的字段（language, identifier, date, meta, contributor 等）。
+    """
+    opf_path = find_opf_path(zip)
+
+    with zip.read(opf_path) as f:
+        content = f.read()
+
+    from xml.etree import ElementTree as ET
+
+    root = ET.fromstring(content)
+
+    # Find metadata element
+    metadata_elem = None
+    for child in root:
+        if child.tag.endswith("metadata"):
+            metadata_elem = child
+            break
+
+    if metadata_elem is None:
+        return []
+
+    # Collect metadata fields to translate
+    fields: list[MetadataField] = []
+
+    for elem in metadata_elem:
+        # Get tag name without namespace
+        tag_name = elem.tag.split("}")[-1] if "}" in elem.tag else elem.tag
+
+        # Check if element has text content and should be translated
+        if elem.text and elem.text.strip() and tag_name not in SKIP_FIELDS:
+            fields.append(MetadataField(tag_name=tag_name, text=elem.text.strip()))
+
+    return fields
+
+
+def write_metadata(zip: Zip, fields: list[MetadataField]) -> None:
+    """
+    将翻译后的元数据字段写回 EPUB 的 OPF 文件。
+
+    根据 tag_name 匹配对应的元素，并更新其文本内容。
+    匹配策略：按照 tag_name 和在文件中出现的顺序依次匹配。
+    """
+    opf_path = find_opf_path(zip)
+
+    with zip.read(opf_path) as f:
+        content = f.read()
+
+    from xml.etree import ElementTree as ET
+
+    root = ET.fromstring(content)
+
+    # Find metadata element
+    metadata_elem = None
+    for child in root:
+        if child.tag.endswith("metadata"):
+            metadata_elem = child
+            break
+
+    if metadata_elem is None:
+        return
+
+    # Build a mapping: tag_name -> list of fields with that tag_name
+    fields_by_tag: dict[str, list[str]] = {}
+    for field in fields:
+        if field.tag_name not in fields_by_tag:
+            fields_by_tag[field.tag_name] = []
+        fields_by_tag[field.tag_name].append(field.text)
+
+    # Create a counter for each tag to track which occurrence we're at
+    tag_counters: dict[str, int] = {tag: 0 for tag in fields_by_tag}
+
+    # Update elements in metadata
+    for elem in metadata_elem:
+        # Get tag name without namespace
+        tag_name = elem.tag.split("}")[-1] if "}" in elem.tag else elem.tag
+
+        # Check if this tag has translated text
+        if tag_name in fields_by_tag and elem.text and elem.text.strip():
+            counter = tag_counters[tag_name]
+            if counter < len(fields_by_tag[tag_name]):
+                # Update the text with translated version
+                elem.text = fields_by_tag[tag_name][counter]
+                tag_counters[tag_name] += 1
+
+    # Write back the modified OPF file
+    tree = ET.ElementTree(root)
+    with zip.replace(opf_path) as f:
+        tree.write(f, encoding="utf-8", xml_declaration=True)

epub_translator/epub/spines.py

@@ -6,7 +6,8 @@ from .common import find_opf_path, strip_namespace
 from .zip import Zip
 
 
-def search_spine_paths(zip: Zip) -> Generator[Path, None, None]:
+# yield file_path, media_type
+def search_spine_paths(zip: Zip) -> Generator[tuple[Path, str], None, None]:
     opf_path = find_opf_path(zip)
     opf_dir = opf_path.parent
 
@@ -39,4 +40,4 @@ def search_spine_paths(zip: Zip) -> Generator[Path, None, None]:
             if idref in manifest_items:
                 href, media_type = manifest_items[idref]
                 if media_type in ("application/xhtml+xml", "text/html"):
-                    yield opf_dir / href
+                    yield opf_dir / href, media_type

epub_translator/epub/zip.py

@@ -44,24 +44,26 @@ class Zip:
         all_files = self._source_zip.namelist()
         if prefix_path is None:
             return [Path(f) for f in all_files]
-        prefix = str(prefix_path)
+        prefix = prefix_path.as_posix()
         if not prefix.endswith("/"):
             prefix += "/"
         return [Path(f) for f in all_files if f.startswith(prefix)]
 
     def migrate(self, path: Path):
+        path_str = path.as_posix()
+        source_info = self._source_zip.getinfo(path_str)
         with self.read(path) as source_file:
-            with self._target_zip.open(str(path), "w") as target_file:
-                while True:
-                    chunk = source_file.read(_BUFFER_SIZE)
-                    if not chunk:
-                        break
-                    target_file.write(chunk)
+            content = source_file.read()
+        self._target_zip.writestr(
+            zinfo_or_arcname=source_info,
+            data=content,
+            compress_type=source_info.compress_type,
+        )
         self._processed_files.add(path)
 
     def read(self, path: Path) -> IO[bytes]:
-        return self._source_zip.open(str(path), "r")
+        return self._source_zip.open(path.as_posix(), "r")
 
     def replace(self, path: Path) -> IO[bytes]:
         self._processed_files.add(path)
-        return self._target_zip.open(str(path), "w")
+        return self._target_zip.open(path.as_posix(), "w")

epub_translator/epub_transcode.py

@@ -0,0 +1,108 @@
+"""
+EPUB 数据结构与 XML 的编码/解码转换
+
+将 Toc 和 MetadataField 等数据结构转换为 XML Element，以便进行翻译处理。
+"""
+
+from xml.etree.ElementTree import Element
+
+from .epub.metadata import MetadataField
+from .epub.toc import Toc
+
+
+def encode_toc(toc: Toc) -> Element:
+    elem = Element("toc-item")
+
+    if toc.href is not None:
+        elem.set("href", toc.href)
+    if toc.fragment is not None:
+        elem.set("fragment", toc.fragment)
+    if toc.id is not None:
+        elem.set("id", toc.id)
+
+    title_elem = Element("title")
+    title_elem.text = toc.title
+    elem.append(title_elem)
+
+    for child in toc.children:
+        child_elem = encode_toc(child)
+        elem.append(child_elem)
+
+    return elem
+
+
+def decode_toc(elem: Element) -> Toc:
+    href = elem.get("href")
+    fragment = elem.get("fragment")
+    toc_id = elem.get("id")
+
+    title_elem = elem.find("title")
+    if title_elem is None or title_elem.text is None:
+        raise ValueError("Missing title element in toc-item")
+    title = title_elem.text
+
+    children = []
+    for child_elem in elem.findall("toc-item"):
+        child_toc = decode_toc(child_elem)
+        children.append(child_toc)
+
+    return Toc(
+        title=title,
+        href=href,
+        fragment=fragment,
+        id=toc_id,
+        children=children,
+    )
+
+
+def encode_toc_list(toc_list: list[Toc]) -> Element:
+    root = Element("toc-list")
+
+    for toc in toc_list:
+        toc_elem = encode_toc(toc)
+        root.append(toc_elem)
+
+    return root
+
+
+def decode_toc_list(elem: Element) -> list[Toc]:
+    if elem.tag != "toc-list":
+        raise ValueError(f"Expected 'toc-list' element, got '{elem.tag}'")
+
+    toc_list = []
+    for toc_elem in elem.findall("toc-item"):
+        toc = decode_toc(toc_elem)
+        toc_list.append(toc)
+
+    return toc_list
+
+
+def encode_metadata(fields: list[MetadataField]) -> Element:
+    root = Element("metadata-list")
+
+    for field in fields:
+        field_elem = Element("field")
+        field_elem.set("tag", field.tag_name)
+        field_elem.text = field.text
+        root.append(field_elem)
+
+    return root
+
+
+def decode_metadata(elem: Element) -> list[MetadataField]:
+    if elem.tag != "metadata-list":
+        raise ValueError(f"Expected 'metadata-list' element, got '{elem.tag}'")
+
+    fields = []
+    for field_elem in elem.findall("field"):
+        tag_name = field_elem.get("tag")
+        if tag_name is None:
+            raise ValueError("Missing 'tag' attribute in field element")
+
+        text = field_elem.text
+        if text is None:
+            raise ValueError(f"Missing text content in field element (tag={tag_name})")
+
+        fields.append(MetadataField(tag_name=tag_name, text=text))
+
+    return fields

epub_translator/llm/__init__.py

@@ -1,2 +1,3 @@
+from .context import LLMContext
 from .core import LLM
 from .types import *

epub_translator/llm/context.py

@@ -0,0 +1,109 @@
+import hashlib
+import json
+import uuid
+from pathlib import Path
+from typing import Self
+
+from .executor import LLMExecutor
+from .increasable import Increasable, Increaser
+from .types import Message, MessageRole
+
+
+class LLMContext:
+    def __init__(
+        self,
+        executor: LLMExecutor,
+        cache_path: Path | None,
+        cache_seed_content: str | None,
+        top_p: Increasable,
+        temperature: Increasable,
+    ) -> None:
+        self._executor = executor
+        self._cache_path = cache_path
+        self._cache_seed_content = cache_seed_content
+        self._top_p: Increaser = top_p.context()
+        self._temperature: Increaser = temperature.context()
+        self._context_id = uuid.uuid4().hex[:12]
+        self._temp_files: set[Path] = set()
+
+    def __enter__(self) -> Self:
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb) -> None:
+        if exc_type is None:
+            # Success: commit all temporary cache files
+            self._commit()
+        else:
+            # Failure: rollback (delete) all temporary cache files
+            self._rollback()
+
+    def request(
+        self,
+        input: str | list[Message],
+        max_tokens: int | None = None,
+        temperature: float | None = None,
+        top_p: float | None = None,
+    ) -> str:
+        messages: list[Message]
+        if isinstance(input, str):
+            messages = [Message(role=MessageRole.USER, message=input)]
+        else:
+            messages = input
+
+        try:
+            cache_key: str | None = None
+            if self._cache_path is not None:
+                cache_key = self._compute_messages_hash(messages)
+                permanent_cache_file = self._cache_path / f"{cache_key}.txt"
+                if permanent_cache_file.exists():
+                    cached_content = permanent_cache_file.read_text(encoding="utf-8")
+                    return cached_content
+
+            if temperature is None:
+                temperature = self._temperature.current
+            if top_p is None:
+                top_p = self._top_p.current
+
+            # Make the actual request
+            response = self._executor.request(
+                messages=messages,
+                max_tokens=max_tokens,
+                temperature=temperature,
+                top_p=top_p,
+                cache_key=cache_key,
+            )
+            # Save to temporary cache if cache_path is set
+            if self._cache_path is not None and cache_key is not None:
+                temp_cache_file = self._cache_path / f"{cache_key}.{self._context_id}.txt"
+                if temp_cache_file.exists():
+                    temp_cache_file.unlink()
+                temp_cache_file.write_text(response, encoding="utf-8")
+                self._temp_files.add(temp_cache_file)
+
+            return response
+
+        finally:
+            self._temperature.increase()
+            self._top_p.increase()
+
+    def _compute_messages_hash(self, messages: list[Message]) -> str:
+        messages_dict = [{"role": msg.role.value, "message": msg.message} for msg in messages]
+        hash_data = {
+            "messages": messages_dict,
+            "cache_seed": self._cache_seed_content,
+        }
+        hash_json = json.dumps(hash_data, ensure_ascii=False, sort_keys=True)
+        return hashlib.sha512(hash_json.encode("utf-8")).hexdigest()
+
+    def _commit(self) -> None:
+        for temp_file in sorted(self._temp_files):
+            if temp_file.exists():
+                # Remove the .[context-id].txt suffix to get permanent name
+                permanent_name = temp_file.name.rsplit(".", 2)[0] + ".txt"
+                permanent_file = temp_file.parent / permanent_name
+                temp_file.rename(permanent_file)
+
+    def _rollback(self) -> None:
+        for temp_file in self._temp_files:
+            if temp_file.exists():
+                temp_file.unlink()