contextweave 0.2.2__tar.gz → 0.3.0__tar.gz

{contextweave-0.2.2 → contextweave-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: contextweave
-Version: 0.2.2
+Version: 0.3.0
 Summary: ContextWeave: context-aware document translation with glossary management
 Project-URL: Homepage, https://github.com/bot-32142/ContextWeave
 Project-URL: Documentation, https://bot-32142.github.io/ContextWeave/
@@ -28,6 +28,7 @@ Requires-Dist: jellyfish>=1.0.0
 Requires-Dist: numpy<2.0.0,>=1.26.4
 Requires-Dist: openai>=1.0.0
 Requires-Dist: opencc-python-reimplemented>=0.1.7
+Requires-Dist: openpyxl>=3.1.5
 Requires-Dist: pikepdf>=9.0.0
 Requires-Dist: pillow>=10.0.0
 Requires-Dist: platformdirs>=4.5.1
@@ -155,13 +156,18 @@ contextweave-cli config validate
 contextweave-cli run ./book.epub --output ./translated/book.epub
 contextweave-cli run ./chapter.txt --output ./translated/chapter.txt --json
 contextweave-cli run ./episode.srt --output ./translated/episode.srt --no-polish
+contextweave-cli galgame inspect ./extracted-game --json
+contextweave-cli galgame skill --output ./contextweave-galgame-translation
+contextweave-cli run ./extracted-game --type galgame --output ./translated-patch --preserve-structure --json
 
 contextweave-cli books list
 contextweave-cli books show BOOK_ID
 contextweave-cli books delete BOOK_ID --yes
 ```
 
-The CLI resolves config from `--config`, then `CONTEXTWEAVE_CONFIG`, then the nearest `contextweave.yaml`/`.contextweave.yaml` walking upward, then the platform default shown by `contextweave-cli config path`. The config mirrors the setup UI: `connections` define provider endpoints and `workflow_profiles` route each translation step. Prefer `api_key_env` so API keys stay in environment variables instead of config files or task snapshots. Use `--no-polish` when a one-shot run should skip the polish pass, which can be useful for timing-sensitive subtitle output.
+The CLI resolves config from `--config`, then `CONTEXTWEAVE_CONFIG`, then the nearest `contextweave.yaml`/`.contextweave.yaml` walking upward, then the platform default shown by `contextweave-cli config path`. The config mirrors the setup UI: `connections` define provider endpoints and `workflow_profiles` route each translation step. Prefer `api_key_env` so API keys stay in environment variables instead of config files or task snapshots. Use `--no-polish` when a one-shot run should skip the polish pass, which can be useful for timing-sensitive subtitle output. Use `--preserve-structure` for patchable galgame/script exports.
+
+For unsupported game engines, convert extracted script text into a supported intermediate format such as ParaTranz JSON or Translator++ `.trans`, then import that file or folder as `--type galgame`. `contextweave-cli galgame skill` writes a Codex/OpenAI-style skill file with this workflow for external extractor agents.
 
 A commented starting point is available at [docs/examples/contextweave-cli.yaml](https://github.com/bot-32142/ContextWeave/blob/master/docs/examples/contextweave-cli.yaml).
 
@@ -183,3 +189,4 @@ A commented starting point is available at [docs/examples/contextweave-cli.yaml]
 | Manga | `.cbz`, image folders | `cbz` | Yes |
 | EPUB | `.epub` | `epub`, `md`, `docx`, `html` | No, but image OCR is supported |
 | Subtitle | `.srt`, `.vtt`, `.ass`, `.ssa` | `srt`, `vtt`, `ass`, `ssa` | No |
+| Galgame | `.rpy`, `.ks`, RPG Maker MV/MZ JSON, MTool JSON, Translator++ `.trans`, VNText JSON, ParaTranz JSON, TPP/Translator++ `.xlsx`, Wolf RPG `.xlsx` | `txt`, preserve-structure folder | No |

{contextweave-0.2.2 → contextweave-0.3.0}/README.md

@@ -106,13 +106,18 @@ contextweave-cli config validate
 contextweave-cli run ./book.epub --output ./translated/book.epub
 contextweave-cli run ./chapter.txt --output ./translated/chapter.txt --json
 contextweave-cli run ./episode.srt --output ./translated/episode.srt --no-polish
+contextweave-cli galgame inspect ./extracted-game --json
+contextweave-cli galgame skill --output ./contextweave-galgame-translation
+contextweave-cli run ./extracted-game --type galgame --output ./translated-patch --preserve-structure --json
 
 contextweave-cli books list
 contextweave-cli books show BOOK_ID
 contextweave-cli books delete BOOK_ID --yes
 ```
 
-The CLI resolves config from `--config`, then `CONTEXTWEAVE_CONFIG`, then the nearest `contextweave.yaml`/`.contextweave.yaml` walking upward, then the platform default shown by `contextweave-cli config path`. The config mirrors the setup UI: `connections` define provider endpoints and `workflow_profiles` route each translation step. Prefer `api_key_env` so API keys stay in environment variables instead of config files or task snapshots. Use `--no-polish` when a one-shot run should skip the polish pass, which can be useful for timing-sensitive subtitle output.
+The CLI resolves config from `--config`, then `CONTEXTWEAVE_CONFIG`, then the nearest `contextweave.yaml`/`.contextweave.yaml` walking upward, then the platform default shown by `contextweave-cli config path`. The config mirrors the setup UI: `connections` define provider endpoints and `workflow_profiles` route each translation step. Prefer `api_key_env` so API keys stay in environment variables instead of config files or task snapshots. Use `--no-polish` when a one-shot run should skip the polish pass, which can be useful for timing-sensitive subtitle output. Use `--preserve-structure` for patchable galgame/script exports.
+
+For unsupported game engines, convert extracted script text into a supported intermediate format such as ParaTranz JSON or Translator++ `.trans`, then import that file or folder as `--type galgame`. `contextweave-cli galgame skill` writes a Codex/OpenAI-style skill file with this workflow for external extractor agents.
 
 A commented starting point is available at [docs/examples/contextweave-cli.yaml](https://github.com/bot-32142/ContextWeave/blob/master/docs/examples/contextweave-cli.yaml).
 
@@ -134,3 +139,4 @@ A commented starting point is available at [docs/examples/contextweave-cli.yaml]
 | Manga | `.cbz`, image folders | `cbz` | Yes |
 | EPUB | `.epub` | `epub`, `md`, `docx`, `html` | No, but image OCR is supported |
 | Subtitle | `.srt`, `.vtt`, `.ass`, `.ssa` | `srt`, `vtt`, `ass`, `ssa` | No |
+| Galgame | `.rpy`, `.ks`, RPG Maker MV/MZ JSON, MTool JSON, Translator++ `.trans`, VNText JSON, ParaTranz JSON, TPP/Translator++ `.xlsx`, Wolf RPG `.xlsx` | `txt`, preserve-structure folder | No |

{contextweave-0.2.2 → contextweave-0.3.0}/context_aware_translation/application/contracts/app_setup.py

@@ -45,11 +45,14 @@ class WorkflowStepId(StrEnum):
 
 
 _DEFAULT_CONNECTION_CONCURRENCY = 5
-_DEEPSEEK_DEFAULT_CONNECTION_CONCURRENCY = 15
+_DEEPSEEK_DEFAULT_CONNECTION_CONCURRENCY = 500
+_DEEPSEEK_FLASH_DEFAULT_CONNECTION_CONCURRENCY = 2500
 
 
-def default_connection_concurrency(provider: ProviderKind | None) -> int:
+def default_connection_concurrency(provider: ProviderKind | None, model: str | None = None) -> int:
     if provider is ProviderKind.DEEPSEEK:
+        if (model or "").strip().lower() == "deepseek-v4-flash":
+            return _DEEPSEEK_FLASH_DEFAULT_CONNECTION_CONCURRENCY
         return _DEEPSEEK_DEFAULT_CONNECTION_CONCURRENCY
     return _DEFAULT_CONNECTION_CONCURRENCY
 
@@ -122,7 +125,7 @@ class ConnectionDraft(ContractModel):
             return data
 
         payload = dict(data)
-        payload["concurrency"] = default_connection_concurrency(provider)
+        payload["concurrency"] = default_connection_concurrency(provider, data.get("default_model"))
         return payload
 
 

{contextweave-0.2.2 → contextweave-0.3.0}/context_aware_translation/application/contracts/common.py

@@ -90,6 +90,7 @@ class DocumentTypeCode(StrEnum):
     SUBTITLE = "subtitle"
     SCANNED_BOOK = "scanned_book"
     MANGA = "manga"
+    GALGAME = "galgame"
     OTHER = "other"
 
 

{contextweave-0.2.2 → contextweave-0.3.0}/context_aware_translation/application/runtime.py

@@ -264,7 +264,7 @@ _WIZARD_MODEL_CATALOG: dict[ProviderKind, tuple[WizardModelTemplate, ...]] = {
             "DeepSeek V4 Flash",
             "deepseek-v4-flash",
             "https://api.deepseek.com",
-            concurrency=default_connection_concurrency(ProviderKind.DEEPSEEK),
+            concurrency=default_connection_concurrency(ProviderKind.DEEPSEEK, "deepseek-v4-flash"),
         ),
         WizardModelTemplate(
             ProviderKind.DEEPSEEK,
@@ -272,7 +272,7 @@ _WIZARD_MODEL_CATALOG: dict[ProviderKind, tuple[WizardModelTemplate, ...]] = {
             "deepseek-v4-pro",
             "https://api.deepseek.com",
             timeout=300,
-            concurrency=default_connection_concurrency(ProviderKind.DEEPSEEK),
+            concurrency=default_connection_concurrency(ProviderKind.DEEPSEEK, "deepseek-v4-pro"),
         ),
     ),
     ProviderKind.ANTHROPIC: (
@@ -1123,6 +1123,7 @@ def map_document_type_code(document_type: str | None) -> DocumentTypeCode:
         "subtitle": DocumentTypeCode.SUBTITLE,
         "scanned_book": DocumentTypeCode.SCANNED_BOOK,
         "manga": DocumentTypeCode.MANGA,
+        "galgame": DocumentTypeCode.GALGAME,
     }
     return mapping.get(str(document_type or ""), DocumentTypeCode.OTHER)
 

{contextweave-0.2.2 → contextweave-0.3.0}/context_aware_translation/cli/config_file.py

@@ -43,7 +43,7 @@ connections:
     temperature: 0
     timeout: 180
     max_retries: 3
-    concurrency: 15
+    concurrency: 2500
 
   deepseek_pro:
     display_name: DeepSeek V4 Pro
@@ -54,7 +54,7 @@ connections:
     temperature: 0
     timeout: 300
     max_retries: 3
-    concurrency: 15
+    concurrency: 500
 
 workflow_profiles:
   balanced_deepseek:

contextweave-0.3.0/context_aware_translation/cli/galgame_skill.py

@@ -0,0 +1,106 @@
+GALGAME_CODEX_SKILL_TEXT = """---
+name: contextweave-galgame-translation
+description: Use ContextWeave CLI to translate extracted galgame scripts and intermediate files while preserving patchable structure.
+---
+
+# ContextWeave Galgame Translation
+
+Use this skill when the user wants to translate a visual novel / galgame with ContextWeave from extracted scripts or tool exports.
+
+## Ground Rules
+
+- Work on a copy of the game or extracted files, never the original install folder.
+- Do not import `.exe`, `.dll`, save data, images, audio, video, or opaque archives directly.
+- Prefer stable intermediate formats over raw text dumps.
+- Preserve placeholders/control codes exactly, including Ren'Py `{...}`, RPG Maker `\\N[1]`, and KAG/Tyrano `[l]` tags.
+- If a game is packed, use external extractors only when the user explicitly asks and the tool is installed.
+- ContextWeave translates supported extracted files; external tools are still responsible for unpacking/repacking proprietary archives.
+
+## Supported Direct Inputs
+
+- Ren'Py `.rpy`
+- TyranoScript / KAG / extracted KiriKiri `.ks`
+- RPG Maker MV/MZ `data/*.json` or `www/data/*.json`
+- MTool JSON dictionaries
+- Translator++ `.trans`
+- VNText JSON
+- ParaTranz JSON
+- TPP / Translator++ `.xlsx`
+- Wolf RPG extracted `.xlsx`
+
+## Preferred Unsupported-Engine Bridge
+
+When a game engine or extractor output is not directly supported by ContextWeave, ask the extractor/converter agent to normalize the extracted script text into ParaTranz-style JSON:
+
+```json
+[
+  {
+    "key": "script001:000001",
+    "original": "こんにちは",
+    "translation": "",
+    "speaker": "Alice",
+    "context": "script001 line 1"
+  }
+]
+```
+
+Rules for the converter:
+
+- Output UTF-8 JSON.
+- Use one object per translatable game entry.
+- Keep `translation` empty before ContextWeave translation.
+- Make `key` stable and unique from source file plus entry index.
+- Preserve control codes/placeholders exactly in `original`.
+- Do not merge entries.
+- Generate a separate injector that writes translated JSON back to the original extracted script format by `key` or original order.
+
+## CLI Workflow
+
+1. Validate config:
+
+```bash
+contextweave-cli --config ./contextweave.yaml config validate
+```
+
+2. Inspect source support:
+
+```bash
+contextweave-cli --json galgame inspect ./extracted_or_exported_source
+```
+
+3. If the source is unsupported, inspect possible external helper hints:
+
+```bash
+contextweave-cli --json galgame helpers ./game_copy
+```
+
+4. Translate and preserve structure:
+
+```bash
+contextweave-cli \
+  --config ./contextweave.yaml \
+  --library-root ./contextweave-library \
+  --json \
+  run ./extracted_or_exported_source \
+  --type galgame \
+  --output ./translated_patch \
+  --preserve-structure
+```
+
+5. Use the corresponding injector/repacker to apply `./translated_patch` back to the copied game or extracted script tree.
+
+## Useful Inspection Commands
+
+```bash
+contextweave-cli galgame inspect ./source
+contextweave-cli galgame glossary-seeds ./source
+contextweave-cli galgame helpers ./source
+```
+
+## Failure Handling
+
+- If import is ambiguous, rerun `run` with `--type galgame`.
+- If placeholder validation fails, repair the translated entry so all source control codes are present exactly once.
+- If an extracted dump is raw plain text, convert it to ParaTranz JSON before using ContextWeave.
+- If only opaque binary chunks remain after extraction, stop and ask for a game-specific adapter/extractor instead of guessing.
+"""

{contextweave-0.2.2 → contextweave-0.3.0}/context_aware_translation/cli/main.py

@@ -8,10 +8,16 @@ from context_aware_translation.application.contracts.common import ContractModel
 from context_aware_translation.application.contracts.document import RunTranslateAndExportRequest
 from context_aware_translation.application.contracts.work import ImportDocumentsRequest, InspectImportPathsRequest
 from context_aware_translation.application.errors import ApplicationError
+from context_aware_translation.documents.galgame import (
+    inspect_galgame_external_helpers,
+    inspect_galgame_glossary_seeds,
+    inspect_galgame_import,
+)
 from context_aware_translation.storage.models.book import BookStatus
 from context_aware_translation.workflow.tasks.models import STATUS_COMPLETED
 
 from .config_file import load_cli_config, resolve_config_path, write_starter_config
+from .galgame_skill import GALGAME_CODEX_SKILL_TEXT
 from .output import (
     EXIT_BLOCKED,
     EXIT_TASK_FAILED,
@@ -89,8 +95,25 @@ def build_parser() -> argparse.ArgumentParser:
     run_parser.add_argument("--book-name")
     run_parser.add_argument("--type", dest="document_type")
     run_parser.add_argument("--format", dest="format_id")
+    run_parser.add_argument("--preserve-structure", action="store_true")
+    run_parser.add_argument("--allow-original-fallback", action="store_true")
     run_parser.add_argument("--no-polish", action="store_true")
 
+    galgame_parser = subparsers.add_parser("galgame", parents=[common])
+    galgame_subparsers = galgame_parser.add_subparsers(dest="galgame_command", required=True)
+    galgame_inspect_parser = galgame_subparsers.add_parser("inspect", parents=[common])
+    galgame_inspect_parser.add_argument("input")
+    galgame_inspect_parser.add_argument("--adapter")
+    galgame_helpers_parser = galgame_subparsers.add_parser("helpers", parents=[common])
+    galgame_helpers_parser.add_argument("input")
+    galgame_helpers_parser.add_argument("--output-folder")
+    galgame_helpers_parser.add_argument("--tool", action="append", dest="tools")
+    galgame_glossary_parser = galgame_subparsers.add_parser("glossary-seeds", parents=[common])
+    galgame_glossary_parser.add_argument("input")
+    galgame_glossary_parser.add_argument("--adapter")
+    galgame_skill_parser = galgame_subparsers.add_parser("skill", parents=[common])
+    galgame_skill_parser.add_argument("--output")
+
     books_parser = subparsers.add_parser("books", parents=[common])
     books_subparsers = books_parser.add_subparsers(dest="books_command", required=True)
     books_subparsers.add_parser("list", parents=[common])
@@ -109,6 +132,8 @@ def command_name(args: argparse.Namespace) -> str:
         return f"config.{getattr(args, 'config_command', '')}"
     if command == "books":
         return f"books.{getattr(args, 'books_command', '')}"
+    if command == "galgame":
+        return f"galgame.{getattr(args, 'galgame_command', '')}"
     return command
 
 
@@ -118,6 +143,8 @@ def dispatch(args: argparse.Namespace) -> dict[str, Any]:
         return dispatch_config(args)
     if command == "books":
         return dispatch_books(args)
+    if command == "galgame":
+        return dispatch_galgame(args)
     if command == "run":
         return run_one_shot(args)
     raise CliError("usage", f"Unsupported command: {command}", exit_code=EXIT_USAGE)
@@ -158,6 +185,77 @@ def dispatch_books(args: argparse.Namespace) -> dict[str, Any]:
     raise CliError("usage", f"Unsupported books command: {books_command}", exit_code=EXIT_USAGE)
 
 
+def dispatch_galgame(args: argparse.Namespace) -> dict[str, Any]:
+    galgame_command = str(args.galgame_command)
+    if galgame_command == "inspect":
+        path = Path(str(args.input)).expanduser().resolve()
+        import_summary = inspect_galgame_import(path, adapter_name=getattr(args, "adapter", None))
+        return {
+            "path": str(path),
+            "imported": import_summary.imported,
+            "skipped": import_summary.skipped,
+            "total_units": import_summary.total_units,
+            "adapters": list(import_summary.adapters),
+            "files": [
+                {
+                    "relative_path": source.relative_path,
+                    "adapter_name": source.adapter_name,
+                    "unit_count": source.unit_count,
+                    "confidence": source.confidence,
+                }
+                for source in import_summary.files
+            ],
+        }
+    if galgame_command == "helpers":
+        path = Path(str(args.input)).expanduser().resolve()
+        output_folder = _optional_path(getattr(args, "output_folder", None))
+        helper_summary = inspect_galgame_external_helpers(
+            path,
+            output_folder=output_folder,
+            tool_names=getattr(args, "tools", None),
+        )
+        return {
+            "path": str(path),
+            "usable": helper_summary.usable,
+            "commands": [
+                {
+                    "source_path": command.source_path,
+                    "output_folder": command.output_folder,
+                    "tool_name": command.tool_name,
+                    "executable": command.executable,
+                    "argv": list(command.argv),
+                    "note": command.note,
+                }
+                for command in helper_summary.commands
+            ],
+            "issues": [
+                {
+                    "source_path": issue.source_path,
+                    "source_extension": issue.source_extension,
+                    "message": issue.message,
+                }
+                for issue in helper_summary.issues
+            ],
+        }
+    if galgame_command == "glossary-seeds":
+        path = Path(str(args.input)).expanduser().resolve()
+        seeds = inspect_galgame_glossary_seeds(path, adapter_name=getattr(args, "adapter", None))
+        return {
+            "path": str(path),
+            "seeds": [
+                {"term": seed.term, "category": seed.category, "source": seed.source, "context": seed.context}
+                for seed in seeds
+            ],
+        }
+    if galgame_command == "skill":
+        output = getattr(args, "output", None)
+        if output is None:
+            return {"skill": GALGAME_CODEX_SKILL_TEXT, "written": False, "path": None}
+        output_path = _write_galgame_skill(Path(str(output)).expanduser())
+        return {"skill": GALGAME_CODEX_SKILL_TEXT, "written": True, "path": str(output_path)}
+    raise CliError("usage", f"Unsupported galgame command: {galgame_command}", exit_code=EXIT_USAGE)
+
+
 def run_one_shot(args: argparse.Namespace) -> dict[str, Any]:
     library_root = _optional_path(getattr(args, "library_root", None))
     input_path = Path(str(args.input)).expanduser().resolve()
@@ -208,6 +306,10 @@ def run_one_shot(args: argparse.Namespace) -> dict[str, Any]:
                 use_batch=False,
                 use_reembedding=False,
                 enable_polish=not bool(getattr(args, "no_polish", False)),
+                options={
+                    "preserve_structure": bool(getattr(args, "preserve_structure", False)),
+                    "allow_original_fallback": bool(getattr(args, "allow_original_fallback", False)),
+                },
             )
         )
         task_id = accepted.queue_item_id or accepted.command_id
@@ -222,6 +324,7 @@ def run_one_shot(args: argparse.Namespace) -> dict[str, Any]:
             "status": record.status,
             "phase": record.phase,
             "output_path": str(output_path),
+            "preserve_structure": bool(getattr(args, "preserve_structure", False)),
         }
         if record.status != STATUS_COMPLETED:
             raise CliError(
@@ -326,6 +429,27 @@ def print_human_success(command: str, data: dict[str, Any]) -> None:
         print(f"Task: {data['task_id']}")
         print(f"Done: {data['output_path']}")
         return
+    if command == "galgame.inspect":
+        print(f"Importable files: {data['imported']} ({data['total_units']} units)")
+        for item in data.get("files", []):
+            print(f"{item['relative_path']}\t{item['adapter_name']}\t{item['unit_count']}\t{item['confidence']}")
+        return
+    if command == "galgame.helpers":
+        for command_item in data.get("commands", []):
+            print(" ".join(command_item["argv"]))
+        for issue in data.get("issues", []):
+            print(f"{issue['source_path']}: {issue['message']}")
+        return
+    if command == "galgame.glossary-seeds":
+        for seed in data.get("seeds", []):
+            print(f"{seed['term']}\t{seed['category']}\t{seed['source']}")
+        return
+    if command == "galgame.skill":
+        if data.get("written"):
+            print(f"Wrote skill: {data['path']}")
+        else:
+            print(data["skill"])
+        return
     print(data)
 
 
@@ -350,6 +474,13 @@ def _optional_path(value: str | None) -> Path | None:
     return Path(value).expanduser().resolve()
 
 
+def _write_galgame_skill(output_path: Path) -> Path:
+    skill_path = output_path / "SKILL.md" if output_path.suffix == "" else output_path
+    skill_path.parent.mkdir(parents=True, exist_ok=True)
+    skill_path.write_text(GALGAME_CODEX_SKILL_TEXT, encoding="utf-8")
+    return skill_path.resolve()
+
+
 def _payload(value: Any) -> Any:
     if isinstance(value, ContractModel):
         return value.to_payload()

contextweave-0.3.0/context_aware_translation/core/galgame_document_handler.py

@@ -0,0 +1,101 @@
+from __future__ import annotations
+
+from collections.abc import Callable
+from typing import TYPE_CHECKING
+
+from context_aware_translation.core.cancellation import raise_if_cancelled
+from context_aware_translation.documents.galgame import deserialize_translation_unit_stream
+from context_aware_translation.storage.schema.book_db import ChunkRecord
+from context_aware_translation.utils.hashing import compute_chunk_hash
+
+if TYPE_CHECKING:
+    from context_aware_translation.core.context_manager import TranslationContextManager
+    from context_aware_translation.core.progress import ProgressCallback
+
+
+class GalgameDocumentHandler:
+    """DocumentTypeHandler for unit-preserving galgame translation."""
+
+    def __init__(
+        self,
+        *,
+        concurrency: int = 5,
+        batch_size: int = 0,
+        max_tokens_per_batch: int = 2000,
+    ) -> None:
+        self._concurrency = max(1, concurrency)
+        self._batch_size = batch_size
+        self._max_tokens_per_batch = max(1, max_tokens_per_batch)
+
+    def add_text(
+        self,
+        text: str,
+        max_token_size_per_chunk: int,  # noqa: ARG002
+        document_id: int,
+        manager: TranslationContextManager,
+    ) -> int:
+        """Store one chunk per translation unit without deduping repeated dialogue text."""
+        units = deserialize_translation_unit_stream(text)
+        chunk_records: list[ChunkRecord] = []
+        chunk_id = manager.term_repo.get_next_chunk_id()
+        for unit in units:
+            if not unit.text.strip():
+                continue
+            identity_text = f"{unit.relative_path}\0{unit.unit_id}\0{unit.text}"
+            chunk_records.append(
+                ChunkRecord(
+                    chunk_id=chunk_id,
+                    hash=compute_chunk_hash(identity_text, document_id=document_id),
+                    text=unit.text,
+                    document_id=document_id,
+                    is_extracted=False,
+                    is_summarized=False,
+                )
+            )
+            chunk_id += 1
+
+        new_chunk_records = [
+            chunk_record for chunk_record in chunk_records if not manager.term_repo.chunk_exists_by_hash(chunk_record.hash)
+        ]
+        if new_chunk_records:
+            manager._state_update([], new_chunk_records)
+        return chunk_id - 1
+
+    async def translate_chunks(
+        self,
+        document_ids: list[int],
+        manager: TranslationContextManager,
+        force: bool = False,
+        source_ids: list[int] | None = None,
+        cancel_check: Callable[[], bool] | None = None,
+        progress_callback: ProgressCallback | None = None,
+    ) -> None:
+        """Delegate to the generic text translator while preserving one chunk per unit."""
+        _ = source_ids
+        raise_if_cancelled(cancel_check)
+        await manager.translate_chunks(
+            concurrency=self._concurrency,
+            batch_size=self._batch_size,
+            max_tokens_per_batch=self._max_tokens_per_batch,
+            document_ids=document_ids,
+            force=force,
+            cancel_check=cancel_check,
+            progress_callback=progress_callback,
+        )
+
+    def get_translated_lines(
+        self,
+        document_id: int,
+        manager: TranslationContextManager,
+    ) -> list[str]:
+        """Return one translated entry per galgame translation unit."""
+        chunks = manager.term_repo.list_chunks(document_id=document_id)
+        if not chunks:
+            raise ValueError("No chunks found in the database")
+
+        sorted_chunks = sorted(chunks, key=lambda chunk: chunk.chunk_id)
+        untranslated = [chunk for chunk in sorted_chunks if not chunk.is_translated or chunk.translation is None]
+        if untranslated:
+            untranslated_ids = [chunk.chunk_id for chunk in untranslated]
+            raise ValueError(f"Cannot export: chunks {untranslated_ids} are not translated yet")
+        return [chunk.translation for chunk in sorted_chunks if chunk.translation is not None]

{contextweave-0.2.2 → contextweave-0.3.0}/context_aware_translation/documents/base.py

@@ -266,13 +266,14 @@ def get_document_classes() -> list[type[Document]]:
     Uses late imports to avoid circular dependencies.
     """
     from context_aware_translation.documents.epub import EPUBDocument
+    from context_aware_translation.documents.galgame import GalgameDocument
     from context_aware_translation.documents.manga import MangaDocument
     from context_aware_translation.documents.pdf import PDFDocument
     from context_aware_translation.documents.scanned_book import ScannedBookDocument
     from context_aware_translation.documents.subtitle import SubtitleDocument
     from context_aware_translation.documents.text import TextDocument
 
-    return [TextDocument, SubtitleDocument, PDFDocument, ScannedBookDocument, MangaDocument, EPUBDocument]
+    return [TextDocument, SubtitleDocument, GalgameDocument, PDFDocument, ScannedBookDocument, MangaDocument, EPUBDocument]
 
 
 def is_ocr_required_for_type(document_type: str) -> bool: