epub-tr 1.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- epub_tr-1.1.0/LICENSE +21 -0
- epub_tr-1.1.0/PKG-INFO +361 -0
- epub_tr-1.1.0/README.md +316 -0
- epub_tr-1.1.0/epub_tr/__init__.py +2 -0
- epub_tr-1.1.0/epub_tr/blocks.py +181 -0
- epub_tr-1.1.0/epub_tr/cache.py +84 -0
- epub_tr-1.1.0/epub_tr/cli.py +265 -0
- epub_tr-1.1.0/epub_tr/engines/__init__.py +39 -0
- epub_tr-1.1.0/epub_tr/engines/base.py +63 -0
- epub_tr-1.1.0/epub_tr/engines/llm.py +257 -0
- epub_tr-1.1.0/epub_tr/engines/mt.py +267 -0
- epub_tr-1.1.0/epub_tr/epub_io.py +342 -0
- epub_tr-1.1.0/epub_tr/pipeline.py +386 -0
- epub_tr-1.1.0/epub_tr/prompts.py +109 -0
- epub_tr-1.1.0/epub_tr.egg-info/PKG-INFO +361 -0
- epub_tr-1.1.0/epub_tr.egg-info/SOURCES.txt +27 -0
- epub_tr-1.1.0/epub_tr.egg-info/dependency_links.txt +1 -0
- epub_tr-1.1.0/epub_tr.egg-info/entry_points.txt +2 -0
- epub_tr-1.1.0/epub_tr.egg-info/requires.txt +20 -0
- epub_tr-1.1.0/epub_tr.egg-info/top_level.txt +1 -0
- epub_tr-1.1.0/pyproject.toml +73 -0
- epub_tr-1.1.0/setup.cfg +4 -0
- epub_tr-1.1.0/tests/test_basic.py +64 -0
- epub_tr-1.1.0/tests/test_blocks.py +108 -0
- epub_tr-1.1.0/tests/test_cli.py +124 -0
- epub_tr-1.1.0/tests/test_engines.py +396 -0
- epub_tr-1.1.0/tests/test_epub_io.py +164 -0
- epub_tr-1.1.0/tests/test_pipeline.py +276 -0
- epub_tr-1.1.0/tests/test_prompts.py +39 -0
epub_tr-1.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Cuma Bozkurt
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
epub_tr-1.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,361 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: epub-tr
|
|
3
|
+
Version: 1.1.0
|
|
4
|
+
Summary: Literary-quality EPUB translator focused on Turkish, with free engines (OpenCode, Google, Argos, Ollama, ...)
|
|
5
|
+
Author: Cuma Bozkurt
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/cumabozkurt/epub-tr
|
|
8
|
+
Project-URL: Documentation, https://github.com/cumabozkurt/epub-tr#readme
|
|
9
|
+
Project-URL: Issues, https://github.com/cumabozkurt/epub-tr/issues
|
|
10
|
+
Project-URL: Changelog, https://github.com/cumabozkurt/epub-tr/blob/main/CHANGELOG.md
|
|
11
|
+
Keywords: epub,ebook,translation,turkish,llm,ollama,opencode,machine-translation
|
|
12
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
13
|
+
Classifier: Environment :: Console
|
|
14
|
+
Classifier: Intended Audience :: End Users/Desktop
|
|
15
|
+
Classifier: Natural Language :: Turkish
|
|
16
|
+
Classifier: Natural Language :: English
|
|
17
|
+
Classifier: Operating System :: OS Independent
|
|
18
|
+
Classifier: Programming Language :: Python :: 3
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
23
|
+
Classifier: Topic :: Text Processing :: Linguistic
|
|
24
|
+
Requires-Python: >=3.10
|
|
25
|
+
Description-Content-Type: text/markdown
|
|
26
|
+
License-File: LICENSE
|
|
27
|
+
Requires-Dist: ebooklib>=0.18
|
|
28
|
+
Requires-Dist: lxml>=4.9
|
|
29
|
+
Requires-Dist: beautifulsoup4>=4.12
|
|
30
|
+
Requires-Dist: requests>=2.28
|
|
31
|
+
Requires-Dist: deep-translator>=1.11
|
|
32
|
+
Requires-Dist: tqdm>=4.60
|
|
33
|
+
Provides-Extra: argos
|
|
34
|
+
Requires-Dist: argostranslate>=1.9; extra == "argos"
|
|
35
|
+
Provides-Extra: extra
|
|
36
|
+
Requires-Dist: translators>=5.9; extra == "extra"
|
|
37
|
+
Provides-Extra: dev
|
|
38
|
+
Requires-Dist: pytest>=7; extra == "dev"
|
|
39
|
+
Requires-Dist: pytest-cov>=4; extra == "dev"
|
|
40
|
+
Requires-Dist: ruff>=0.6; extra == "dev"
|
|
41
|
+
Requires-Dist: build>=1; extra == "dev"
|
|
42
|
+
Requires-Dist: codespell>=2.3; extra == "dev"
|
|
43
|
+
Requires-Dist: twine>=5; extra == "dev"
|
|
44
|
+
Dynamic: license-file
|
|
45
|
+
|
|
46
|
+
# epub-tr — literary-quality EPUB translator (Turkish first)
|
|
47
|
+
|
|
48
|
+
[](README.md)
|
|
49
|
+
[](README.tr.md)
|
|
50
|
+
[](https://github.com/cumabozkurt/epub-tr/actions/workflows/ci.yml)
|
|
51
|
+
[](https://github.com/cumabozkurt/epub-tr/actions/workflows/codeql.yml)
|
|
52
|
+
[](https://github.com/cumabozkurt/epub-tr/releases/latest)
|
|
53
|
+
[](pyproject.toml)
|
|
54
|
+
[](LICENSE)
|
|
55
|
+
[](TEST_REPORT.md#epub-validation-epubcheck-540)
|
|
56
|
+
|
|
57
|
+
`epub-tr` translates EPUB books into **natural, literary Turkish** (other target languages work too)
|
|
58
|
+
while keeping the book intact: HTML formatting, images, CSS, fonts, TOC (nav + NCX), metadata and
|
|
59
|
+
internal links survive. All engines are **free** (no paid API key required).
|
|
60
|
+
|
|
61
|
+
**Contents:** [Features](#features) · [Which engine?](#which-engine) · [Engines](#engines) · [Install](#install) ·
|
|
62
|
+
[Quick start](#quick-start) · [Usage](#usage) · [Configuration](#configuration) · [Architecture](#architecture) ·
|
|
63
|
+
[Troubleshooting](#troubleshooting) · [FAQ](#faq) · [Tests](#tests) · [Roadmap](#roadmap) ·
|
|
64
|
+
[Contributing](#contributing) · [License](#license)
|
|
65
|
+
|
|
66
|
+
**Docs:** [Architecture](docs/ARCHITECTURE.md) · [Engines and measured ranking](docs/ENGINES.md) ·
|
|
67
|
+
[CI and releases](docs/AUTOMATION.md) · [Test report](TEST_REPORT.md) · [Research](RESEARCH.md) ·
|
|
68
|
+
[Changelog](CHANGELOG.md)
|
|
69
|
+
|
|
70
|
+
## Features
|
|
71
|
+
|
|
72
|
+
* **Structure-preserving EPUB I/O** – read with `ebooklib` (metadata, manifest, spine) + `lxml`;
|
|
73
|
+
write at ZIP level: every original entry is copied byte-for-byte, only translated XHTML, OPF
|
|
74
|
+
(`dc:language` → `tr`, translated title) and NCX/nav labels are replaced. Output validates with
|
|
75
|
+
EPUBCheck as well as the input does.
|
|
76
|
+
* **Inline markup kept** – `<i>`, `<em>`, `<a id=…>`, `<br/>`, footnote refs… are turned into neutral
|
|
77
|
+
placeholders (`<g1>…</g1>`, `<x2/>`) for the engine and rebuilt afterwards with all attributes.
|
|
78
|
+
If an engine breaks them, a lenient fallback keeps the text and every anchor/link target.
|
|
79
|
+
* **Bilingual mode** (`--bilingual`) – translation inserted under each original paragraph.
|
|
80
|
+
* **Literary LLM pipeline**
|
|
81
|
+
* paragraphs batched into chunks (`--chunk-chars`) with stable `<seg id>`s,
|
|
82
|
+
* **context window**: previous paragraphs (source + translation) sent with every chunk,
|
|
83
|
+
* **running glossary**: the model reports name/term decisions which are reused for the rest of
|
|
84
|
+
the book (and persisted in the cache → consistent on resume); seed your own with `--glossary`,
|
|
85
|
+
* automatic **proper-name detection** (names kept, Turkish suffixes with apostrophe: *Della'nın*),
|
|
86
|
+
* Turkish literary system prompt: idiomatic, tone/style/period preserving, TDK spelling,
|
|
87
|
+
sen/siz consistency, dialogue punctuation (`--dialogue quotes|dash`),
|
|
88
|
+
* optional **second review/polish pass** (`--polish`, optionally with another engine
|
|
89
|
+
`--polish-engine`).
|
|
90
|
+
* **Speed & robustness** – thread-pool concurrency (chapters in parallel for LLMs, batches for MT),
|
|
91
|
+
SQLite cache = **resume** for free, retries with back-off, per-chunk **fallback chain**
|
|
92
|
+
(`--fallback google,argos`), fast fail on OpenCode free-tier rate limits with automatic switching
|
|
93
|
+
between OpenCode free models.
|
|
94
|
+
|
|
95
|
+
## Which engine?
|
|
96
|
+
|
|
97
|
+
Measured on O. Henry's *The Gift of the Magi* (51 segments, 11,237 characters; details in
|
|
98
|
+
[TEST_REPORT.md](TEST_REPORT.md), outputs in [`out/`](out)):
|
|
99
|
+
|
|
100
|
+
| rank | setup | literary quality | time | sample |
|
|
101
|
+
|---|---|---|---|---|
|
|
102
|
+
| 🥇 | `--engine opencode --polish` (free `opencode/space-bunny-free`) | **best**: reads like edited Turkish prose | 6,932 s | "Birer ikişer kuruş biriktirmek için bakkala, manava ve kasaba gözünü karartıyordu…" |
|
|
103
|
+
| 🥈 | `--engine opencode` (draft) | idiomatic, a few typos ("srma", "uyacak") | 3,730 s | |
|
|
104
|
+
| 🥉 | `--engine bing` | most natural machine translation | 43 s | "…kasapla pazarlık yaparken…" |
|
|
105
|
+
| 4 | `--engine google` | fluent but literal, no setup | 1.7 s | "…buldozerlerle ezerek…" |
|
|
106
|
+
|
|
107
|
+
**Recommended:** `epub-tr translate book.epub --engine opencode --polish --fallback bing`. The OpenCode
|
|
108
|
+
free tier is rate-limited per IP and slow (about 5 minutes per call in our run), so some chunks may fail.
|
|
109
|
+
Run the same command again later: finished segments come from the cache and only the gaps go to the model.
|
|
110
|
+
Need it fast? Use `--engine bing` or `--engine google`. Full ranking with all 8 engines:
|
|
111
|
+
[docs/ENGINES.md](docs/ENGINES.md).
|
|
112
|
+
|
|
113
|
+
## Engines
|
|
114
|
+
|
|
115
|
+
| engine | type | needs |
|
|
116
|
+
|---|---|---|
|
|
117
|
+
| `opencode` | LLM via `opencode run` (OpenCode Zen **free** models: big-pickle, nemotron-3-ultra-free, mimo-v2.6-flash-free, …) | `npm i -g opencode-ai` |
|
|
118
|
+
| `ollama` | local LLM (default `gemma3:4b`, try `aya-expanse:8b`, `qwen2.5:7b`, `gemma3:12b`) | Ollama running |
|
|
119
|
+
| `google` | Google Translate free web endpoints (+ deep-translator fallback) | – |
|
|
120
|
+
| `bing`, `yandex`, `modernmt` | free web translators via the optional `translators` package (GPL-3) | `pip install translators` |
|
|
121
|
+
| `mymemory` | MyMemory API (anonymous ~5k chars/day/IP; `MYMEMORY_EMAIL` → 50k) | – |
|
|
122
|
+
| `lingva` | Lingva public instances (`LINGVA_URL`) | – |
|
|
123
|
+
| `libretranslate` | LibreTranslate server (`LIBRETRANSLATE_URL`, `LIBRETRANSLATE_API_KEY`) | self-host or key |
|
|
124
|
+
| `argos` | Argos Translate, fully offline (en→tr model auto-installed) | `pip install argostranslate` |
|
|
125
|
+
| `openrouter`, `gemini`, `groq`, `mistral`, `openai` | OpenAI-compatible APIs – free tiers (e.g. OpenRouter `:free` models, Gemini free tier) | `OPENROUTER_API_KEY` / `GEMINI_API_KEY` / … |
|
|
126
|
+
|
|
127
|
+
`epub-tr engines --check` shows what is usable on your machine.
|
|
128
|
+
|
|
129
|
+
## Install
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
python3 -m venv .venv && . .venv/bin/activate
|
|
133
|
+
pip install -e . # core (Python 3.10+)
|
|
134
|
+
pip install -e '.[argos,extra]' # offline Argos + Bing/Yandex/ModernMT
|
|
135
|
+
npm i -g opencode-ai # OpenCode CLI (free LLMs)
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
## Quick start
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
git clone https://github.com/cumabozkurt/epub-tr.git && cd epub-tr
|
|
142
|
+
python3 -m venv .venv && . .venv/bin/activate
|
|
143
|
+
pip install -e .
|
|
144
|
+
epub-tr engines --check # which engines work here?
|
|
145
|
+
epub-tr inspect samples/pg7256.epub --show 10 # what will be translated
|
|
146
|
+
epub-tr translate samples/pg7256.epub -e google -o magi.tr.epub # ~2 s, no setup needed
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
## Usage
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
# best quality, free: OpenCode free LLM + polish pass, Bing as safety net (rerun later to fill gaps)
|
|
153
|
+
epub-tr translate book.epub -o book.tr.epub --engine opencode --polish --fallback bing
|
|
154
|
+
|
|
155
|
+
# pick an OpenCode model explicitly
|
|
156
|
+
epub-tr translate book.epub --engine opencode -m opencode/nemotron-3-ultra-free
|
|
157
|
+
|
|
158
|
+
# local & private
|
|
159
|
+
epub-tr translate book.epub --engine ollama -m aya-expanse:8b --chunk-chars 1500
|
|
160
|
+
|
|
161
|
+
# fast MT, bilingual edition
|
|
162
|
+
epub-tr translate book.epub --engine google --bilingual -o book.en-tr.epub
|
|
163
|
+
|
|
164
|
+
# only chapters 1-3, first 50 paragraphs, Turkish dash-style dialogue, own glossary
|
|
165
|
+
epub-tr translate book.epub --chapters 1-3 --limit 50 --dialogue dash --glossary names.tsv
|
|
166
|
+
|
|
167
|
+
epub-tr inspect book.epub --show 20 # see what will be translated
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
Useful options: `--source en` (default: from metadata), `--target tr`, `--workers N`,
|
|
171
|
+
`--context 3`, `--retries 3`, `--no-toc`, `--keep-boilerplate` (also translate Project Gutenberg
|
|
172
|
+
header/licence; skipped by default), `--cache PATH`, `--no-cache`, `--stats out.json`,
|
|
173
|
+
`--dump pairs.jsonl`.
|
|
174
|
+
|
|
175
|
+
Environment: `EPUB_TR_OPENCODE_MODEL`, `EPUB_TR_OPENCODE_TIMEOUT`, `EPUB_TR_OLLAMA_MODEL`,
|
|
176
|
+
`OLLAMA_HOST`, `EPUB_TR_CACHE`, plus the API keys above.
|
|
177
|
+
|
|
178
|
+
## Configuration
|
|
179
|
+
|
|
180
|
+
### `epub-tr translate` options
|
|
181
|
+
|
|
182
|
+
| option | default | meaning |
|
|
183
|
+
|---|---|---|
|
|
184
|
+
| `input` | – | source `.epub` |
|
|
185
|
+
| `-o, --output` | `<input>.<target>.epub` (`<input>.bilingual.<target>.epub` with `--bilingual`) | output path |
|
|
186
|
+
| `-e, --engine` | `opencode` | main engine (see [Engines](#engines)) |
|
|
187
|
+
| `-m, --model` | engine default | model for LLM engines (`opencode/big-pickle`, `gemma3:4b`, …) |
|
|
188
|
+
| `--fallback` | – | comma-separated engines tried per chunk when the main one fails, e.g. `google,argos` |
|
|
189
|
+
| `-s, --source` | from `dc:language`, else `en` | source language |
|
|
190
|
+
| `-t, --target` | `tr` | target language |
|
|
191
|
+
| `--bilingual` | off | keep the original paragraph and add the translation below it |
|
|
192
|
+
| `--polish` | off | second LLM pass: literary review/polish |
|
|
193
|
+
| `--polish-engine`, `--polish-model` | same LLM | engine/model for the polish pass |
|
|
194
|
+
| `--dialogue` | `quotes` | Turkish dialogue style: `quotes` (“…”) or `dash` (— …) |
|
|
195
|
+
| `--glossary` | – | seed glossary (see below) |
|
|
196
|
+
| `-w, --workers` | engine default | parallel workers (chapters for LLMs, batches for MT) |
|
|
197
|
+
| `--chunk-chars` | `2500` | source characters per LLM request |
|
|
198
|
+
| `--context` | `3` | previous paragraphs sent to the LLM as context |
|
|
199
|
+
| `--retries` | `3` | retries per chunk (exponential back-off) |
|
|
200
|
+
| `--chapters` | all | content documents to translate, 1-based: `1-3,5` |
|
|
201
|
+
| `--limit` | – | only the first N segments (handy for testing) |
|
|
202
|
+
| `--no-toc` | off | do not translate nav/NCX labels |
|
|
203
|
+
| `--keep-boilerplate` | off | also translate the Project Gutenberg header/licence |
|
|
204
|
+
| `--cache` / `--no-cache` | `~/.cache/epub-tr/cache.sqlite3` | SQLite translation cache (resume) |
|
|
205
|
+
| `--stats FILE` | – | write a JSON summary (incl. the final glossary) |
|
|
206
|
+
| `--dump FILE` | – | write JSONL `{uid, source, translation, engine}` pairs |
|
|
207
|
+
| `-q, --quiet` | off | no progress bar |
|
|
208
|
+
| `-v, --verbose` (global) | off | INFO logging |
|
|
209
|
+
|
|
210
|
+
Other sub-commands: `epub-tr engines [--check]`, `epub-tr inspect BOOK [--show N]`, `epub-tr --version`.
|
|
211
|
+
|
|
212
|
+
**Exit code:** `0` when every selected segment was translated, `2` when some segments are still
|
|
213
|
+
untranslated (they are kept in the source language; rerun later to fill them from the remaining engines/cache).
|
|
214
|
+
A JSON summary (segments, engine usage, failures, cache hits, glossary size, first errors) is always printed to stderr.
|
|
215
|
+
|
|
216
|
+
### Glossary file
|
|
217
|
+
|
|
218
|
+
`--glossary` accepts JSON (`{"Della": "Della", "Madame Sofronie": "Madam Sofronie"}`) or a text file with one
|
|
219
|
+
entry per line, using `=>`, a TAB or `=` as separator; empty lines and `#` comments are ignored:
|
|
220
|
+
|
|
221
|
+
```text
|
|
222
|
+
# names.tsv
|
|
223
|
+
Madame Sofronie => Madam Sofronie
|
|
224
|
+
the Magi Müneccimler
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
### Environment variables
|
|
228
|
+
|
|
229
|
+
| variable | used by | default |
|
|
230
|
+
|---|---|---|
|
|
231
|
+
| `EPUB_TR_CACHE` | cache path | `$XDG_CACHE_HOME/epub-tr/cache.sqlite3` |
|
|
232
|
+
| `EPUB_TR_OPENCODE_MODEL` | opencode | `opencode/big-pickle` |
|
|
233
|
+
| `EPUB_TR_OPENCODE_TIMEOUT` | opencode (seconds per call) | `600` |
|
|
234
|
+
| `OPENCODE_BIN` | opencode binary | `opencode` on `PATH`, then `~/.opencode/bin/opencode` |
|
|
235
|
+
| `EPUB_TR_OLLAMA_MODEL` | ollama | `gemma3:4b` |
|
|
236
|
+
| `OLLAMA_HOST` | ollama | `http://localhost:11434` |
|
|
237
|
+
| `EPUB_TR_OLLAMA_CTX` / `EPUB_TR_OLLAMA_MAX_TOKENS` / `EPUB_TR_OLLAMA_TIMEOUT` | ollama | `8192` / auto / `1800` |
|
|
238
|
+
| `MYMEMORY_EMAIL` | mymemory (raises the daily quota) | – |
|
|
239
|
+
| `LINGVA_URL` | lingva instance | built-in public list |
|
|
240
|
+
| `LIBRETRANSLATE_URL`, `LIBRETRANSLATE_API_KEY` | libretranslate | `http://localhost:5000` |
|
|
241
|
+
| `OPENROUTER_API_KEY`, `GEMINI_API_KEY`, `GROQ_API_KEY`, `MISTRAL_API_KEY`, `OPENAI_API_KEY` | API presets | – |
|
|
242
|
+
| `EPUB_TR_<PRESET>_MODEL`, `EPUB_TR_<PRESET>_BASE_URL`, `OPENAI_BASE_URL` | override preset model / endpoint | see `epub_tr/engines/llm.py` |
|
|
243
|
+
|
|
244
|
+
Default preset models: openrouter `meta-llama/llama-3.3-70b-instruct:free`, gemini `gemini-2.5-flash`,
|
|
245
|
+
groq `llama-3.3-70b-versatile`, mistral `mistral-small-latest`, openai `gpt-4o-mini`.
|
|
246
|
+
|
|
247
|
+
## Architecture
|
|
248
|
+
|
|
249
|
+
```text
|
|
250
|
+
book.epub ──► epub_io.Book (ebooklib + lxml) ── documents, spine, OPF, nav, NCX
|
|
251
|
+
│ blocks.py: each block element → Segment, inline markup → <g1>…</g1>/<x2/> placeholders
|
|
252
|
+
▼
|
|
253
|
+
pipeline.Translator
|
|
254
|
+
├─ MT engines: batched segments, thread pool
|
|
255
|
+
└─ LLM engines: chunks of <seg id=…> (--chunk-chars) + context window + running glossary
|
|
256
|
+
+ Turkish literary prompt (prompts.py) → optional --polish pass
|
|
257
|
+
├─ cache.py: SQLite cache keyed by engine+model+stage+languages+text (resume); glossary stored per book
|
|
258
|
+
└─ fallback chain per chunk, retries/back-off, rate-limit detection
|
|
259
|
+
▼
|
|
260
|
+
Book.apply() rebuilds elements with all attributes (lenient fallback keeps anchors/links)
|
|
261
|
+
Book.write() copies the original ZIP byte-for-byte, replaces only XHTML, OPF, nav, NCX ──► book.tr.epub
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
| module | responsibility |
|
|
265
|
+
|---|---|
|
|
266
|
+
| `epub_tr/cli.py` | argument parsing, engine/fallback setup, summary/stats/dump |
|
|
267
|
+
| `epub_tr/epub_io.py` | EPUB reading (segments, TOC labels, Gutenberg boilerplate detection) and ZIP-level writing |
|
|
268
|
+
| `epub_tr/blocks.py` | XHTML block ⇄ placeholder string encoding |
|
|
269
|
+
| `epub_tr/pipeline.py` | chunking, context, glossary, name detection, polish, concurrency, fallback, TOC harmonisation |
|
|
270
|
+
| `epub_tr/prompts.py` | system/user prompts (Turkish literary rules, dialogue style) |
|
|
271
|
+
| `epub_tr/cache.py` | SQLite cache (WAL, lock retry) |
|
|
272
|
+
| `epub_tr/engines/` | `base.py` interface, `llm.py` (OpenCode, Ollama, OpenAI-compatible), `mt.py` (Google, translators, MyMemory, Lingva, LibreTranslate, Argos) |
|
|
273
|
+
| `scripts/` | `compare.py` (side-by-side engine comparison), `validate_epub.py` (EPUBCheck), `check_links.py`, `release.py` + `release_notes.py`, `run_ollama.sh`, `opencode_when_available.sh` (waits for the OpenCode free quota, then runs) |
|
|
274
|
+
| `samples/`, `out/` | Project Gutenberg test books and the test-run outputs (all engines, including OpenCode) referenced in `TEST_REPORT.md` |
|
|
275
|
+
|
|
276
|
+
More detail: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
|
|
277
|
+
|
|
278
|
+
**Adding an engine:** subclass `epub_tr.engines.base.Engine`; implement `translate_one()` (MT, or
|
|
279
|
+
`translate_batch()`) or `complete(system, user)` (LLM, set `is_llm = True`) plus `check()` raising
|
|
280
|
+
`EngineUnavailable`, then register it in `ENGINES` in `epub_tr/engines/__init__.py`.
|
|
281
|
+
|
|
282
|
+
## Troubleshooting
|
|
283
|
+
|
|
284
|
+
| symptom | cause / fix |
|
|
285
|
+
|---|---|
|
|
286
|
+
| `[warn] engine opencode unavailable` | install the CLI (`npm i -g opencode-ai`) or point `OPENCODE_BIN` at it |
|
|
287
|
+
| OpenCode: `FreeUsageLimitError` / HTTP 429 | the Zen free tier is limited per IP for hours. Use `--fallback google` (or another engine) and rerun later; cached chunks are reused. `scripts/opencode_when_available.sh` can wait for the quota. |
|
|
288
|
+
| `ollama not running at http://localhost:11434` | `ollama serve` and `ollama pull gemma3:4b` (or set `OLLAMA_HOST`) |
|
|
289
|
+
| Ollama very slow / loops | use a smaller `--chunk-chars` (e.g. 1500), a smaller model, or cap with `EPUB_TR_OLLAMA_MAX_TOKENS` |
|
|
290
|
+
| `bing`/`yandex`/`modernmt` unavailable | `pip install -e '.[extra]'` (`translators`, GPL-3) |
|
|
291
|
+
| `argos` unavailable | `pip install -e '.[argos]'`; the en→tr model downloads on first use |
|
|
292
|
+
| `mymemory` returns quota warnings | the anonymous quota per IP is used up; set `MYMEMORY_EMAIL` or switch engine |
|
|
293
|
+
| exit code `2` | some segments stayed untranslated (all engines failed for them); see `errors` in the summary and rerun |
|
|
294
|
+
| formatting lost on some paragraphs | the summary's `markup_fallbacks` counts paragraphs rebuilt leniently (text and anchors kept, inline styling dropped); LLMs that respect placeholders (or `google`) avoid it |
|
|
295
|
+
| OpenCode: `empty output` or `timed out after 600s` | the free model is overloaded; epub-tr moves on to the next free model. Rerun later, raise `EPUB_TR_OPENCODE_TIMEOUT`, or add `--fallback bing` |
|
|
296
|
+
| OpenCode: `database is locked` | two OpenCode processes touched OpenCode's own database at once; the retry usually fixes it, or use `--workers 1` |
|
|
297
|
+
| want a clean re-translation | `--no-cache` or delete/point `--cache` to a new file |
|
|
298
|
+
| Gutenberg header is not translated | intended; add `--keep-boilerplate` |
|
|
299
|
+
|
|
300
|
+
## Notes on OpenCode free models
|
|
301
|
+
|
|
302
|
+
`opencode run` is executed in a private directory with a generated `opencode.json` that defines a
|
|
303
|
+
tool-less `epubtr` agent whose system prompt is the literary-translation prompt (`--pure`,
|
|
304
|
+
`--title` avoids the extra title request, `--format json` for robust parsing). The Zen free tier is
|
|
305
|
+
rate-limited **per IP**; when the quota is exhausted (`FreeUsageLimitError`, `retry-after` header of
|
|
306
|
+
several hours) `epub-tr` fails fast, tries the other free models and then falls back to the next
|
|
307
|
+
engine in `--fallback`. Run again later – cached chunks are not re-translated.
|
|
308
|
+
|
|
309
|
+
In our real run, the first five free models were rate-limited and every segment came from
|
|
310
|
+
`opencode/space-bunny-free`. The run summary's `engine_models` shows the model actually used. A model that
|
|
311
|
+
answers with empty output is skipped like a rate-limited one. More in [docs/ENGINES.md](docs/ENGINES.md#opencode-free-zen-models).
|
|
312
|
+
|
|
313
|
+
## FAQ
|
|
314
|
+
|
|
315
|
+
**Is it really free?** Yes. OpenCode Zen free models, Google, Bing, Yandex, ModernMT, MyMemory, Lingva,
|
|
316
|
+
Argos and Ollama need no paid key. The API presets work with free-tier keys.
|
|
317
|
+
|
|
318
|
+
**Is my book sent to the internet?** Only to the engine you pick. Use `ollama` or `argos` for fully local
|
|
319
|
+
translation. See [SECURITY.md](SECURITY.md).
|
|
320
|
+
|
|
321
|
+
**Will the layout break?** Untouched files are copied byte for byte, and inline markup is rebuilt with all
|
|
322
|
+
attributes. Every test output passes EPUBCheck with no new errors.
|
|
323
|
+
|
|
324
|
+
**Can I stop and continue later?** Yes. Every translated segment is cached, so rerunning the same command
|
|
325
|
+
resumes and fills the gaps.
|
|
326
|
+
|
|
327
|
+
**Other languages?** `--target de`, `--source fr` and so on work. The literary rules are tuned for Turkish,
|
|
328
|
+
and other targets get a generic literary prompt.
|
|
329
|
+
|
|
330
|
+
**DRM-protected books?** No. epub-tr only works on DRM-free EPUBs you are allowed to translate.
|
|
331
|
+
|
|
332
|
+
## Tests
|
|
333
|
+
|
|
334
|
+
```bash
|
|
335
|
+
pip install -e ".[dev]"
|
|
336
|
+
python -m pytest -q --cov=epub_tr # 120 offline tests, ~3 s, no network
|
|
337
|
+
ruff check . && codespell
|
|
338
|
+
python scripts/validate_epub.py # EPUBCheck on the samples (Java + epubcheck)
|
|
339
|
+
```
|
|
340
|
+
|
|
341
|
+
CI runs these on Ubuntu, Windows and macOS with Python 3.10–3.13, plus CodeQL, markdownlint, link checks,
|
|
342
|
+
a package build and EPUBCheck. See [docs/AUTOMATION.md](docs/AUTOMATION.md). Real-engine results:
|
|
343
|
+
[TEST_REPORT.md](TEST_REPORT.md). Survey of 15 open-source translators: [RESEARCH.md](RESEARCH.md).
|
|
344
|
+
|
|
345
|
+
## Roadmap
|
|
346
|
+
|
|
347
|
+
* PyPI package (`pip install epub-tr`) through a trusted publisher.
|
|
348
|
+
* Per-segment model attribution in `--dump` (which OpenCode model translated what).
|
|
349
|
+
* Optional human-review export (side-by-side HTML or DOCX) and re-import.
|
|
350
|
+
* More target-language rule sets (German, Spanish, Arabic) next to the Turkish one.
|
|
351
|
+
|
|
352
|
+
## Contributing
|
|
353
|
+
|
|
354
|
+
Issues and pull requests are welcome. Read [CONTRIBUTING.md](CONTRIBUTING.md) (setup, checks, adding an
|
|
355
|
+
engine) and the [Code of Conduct](CODE_OF_CONDUCT.md). Translation-quality reports have their own issue
|
|
356
|
+
form. Report security problems privately: [SECURITY.md](SECURITY.md).
|
|
357
|
+
|
|
358
|
+
## License
|
|
359
|
+
|
|
360
|
+
[MIT](LICENSE) © Cuma Bozkurt. The optional `translators` package (Bing/Yandex/ModernMT) is GPL-3;
|
|
361
|
+
it is not a hard dependency. Sample books in `samples/` are public-domain Project Gutenberg texts.
|