joven-ebook-annotator 1.0.0b2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. joven_ebook_annotator-1.0.0b2/LICENSE +21 -0
  2. joven_ebook_annotator-1.0.0b2/PKG-INFO +490 -0
  3. joven_ebook_annotator-1.0.0b2/README.md +457 -0
  4. joven_ebook_annotator-1.0.0b2/pyproject.toml +69 -0
  5. joven_ebook_annotator-1.0.0b2/setup.cfg +4 -0
  6. joven_ebook_annotator-1.0.0b2/src/joven/__init__.py +8 -0
  7. joven_ebook_annotator-1.0.0b2/src/joven/cli.py +497 -0
  8. joven_ebook_annotator-1.0.0b2/src/joven/detect/__init__.py +1 -0
  9. joven_ebook_annotator-1.0.0b2/src/joven/detect/pipeline.py +292 -0
  10. joven_ebook_annotator-1.0.0b2/src/joven/detect/segment.py +119 -0
  11. joven_ebook_annotator-1.0.0b2/src/joven/detect/triage.py +188 -0
  12. joven_ebook_annotator-1.0.0b2/src/joven/dialogue.py +72 -0
  13. joven_ebook_annotator-1.0.0b2/src/joven/epub/__init__.py +1 -0
  14. joven_ebook_annotator-1.0.0b2/src/joven/epub/archive.py +236 -0
  15. joven_ebook_annotator-1.0.0b2/src/joven/epub/document.py +245 -0
  16. joven_ebook_annotator-1.0.0b2/src/joven/epub/package.py +79 -0
  17. joven_ebook_annotator-1.0.0b2/src/joven/kepub.py +68 -0
  18. joven_ebook_annotator-1.0.0b2/src/joven/model.py +290 -0
  19. joven_ebook_annotator-1.0.0b2/src/joven/render/__init__.py +108 -0
  20. joven_ebook_annotator-1.0.0b2/src/joven/render/annotate.py +286 -0
  21. joven_ebook_annotator-1.0.0b2/src/joven/render/kobo.py +126 -0
  22. joven_ebook_annotator-1.0.0b2/src/joven/render/upgrade.py +500 -0
  23. joven_ebook_annotator-1.0.0b2/src/joven/review.py +422 -0
  24. joven_ebook_annotator-1.0.0b2/src/joven/suspicion.py +109 -0
  25. joven_ebook_annotator-1.0.0b2/src/joven/trace.py +212 -0
  26. joven_ebook_annotator-1.0.0b2/src/joven/translate.py +426 -0
  27. joven_ebook_annotator-1.0.0b2/src/joven/verify.py +422 -0
  28. joven_ebook_annotator-1.0.0b2/src/joven_ebook_annotator.egg-info/PKG-INFO +490 -0
  29. joven_ebook_annotator-1.0.0b2/src/joven_ebook_annotator.egg-info/SOURCES.txt +31 -0
  30. joven_ebook_annotator-1.0.0b2/src/joven_ebook_annotator.egg-info/dependency_links.txt +1 -0
  31. joven_ebook_annotator-1.0.0b2/src/joven_ebook_annotator.egg-info/entry_points.txt +2 -0
  32. joven_ebook_annotator-1.0.0b2/src/joven_ebook_annotator.egg-info/requires.txt +8 -0
  33. joven_ebook_annotator-1.0.0b2/src/joven_ebook_annotator.egg-info/top_level.txt +1 -0
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Vlad Yanhursky
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,490 @@
1
+ Metadata-Version: 2.4
2
+ Name: joven-ebook-annotator
3
+ Version: 1.0.0b2
4
+ Summary: Insert clickable translation footnotes for foreign-language passages in an EPUB
5
+ Author: Vlad Yanhursky
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/vyanhursky/joven
8
+ Project-URL: Repository, https://github.com/vyanhursky/joven
9
+ Project-URL: Issues, https://github.com/vyanhursky/joven/issues
10
+ Project-URL: Changelog, https://github.com/vyanhursky/joven/blob/main/CHANGELOG.md
11
+ Keywords: epub,kobo,ebook,translation,footnotes,annotation,kepub
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Environment :: Console
14
+ Classifier: Intended Audience :: End Users/Desktop
15
+ Classifier: Operating System :: MacOS
16
+ Classifier: Operating System :: POSIX :: Linux
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Topic :: Text Processing :: Markup :: XML
21
+ Classifier: Topic :: Utilities
22
+ Requires-Python: >=3.11
23
+ Description-Content-Type: text/markdown
24
+ License-File: LICENSE
25
+ Requires-Dist: lxml>=5.2
26
+ Requires-Dist: typer>=0.12
27
+ Requires-Dist: lingua-language-detector>=2.0
28
+ Requires-Dist: httpx>=0.27
29
+ Provides-Extra: dev
30
+ Requires-Dist: pytest>=8.0; extra == "dev"
31
+ Requires-Dist: ruff>=0.5; extra == "dev"
32
+ Dynamic: license-file
33
+
34
+ # Joven — a Cormac McCarthy ebook Spanish annotator
35
+
36
+ [![CI](https://github.com/vyanhursky/joven/actions/workflows/ci.yml/badge.svg)](https://github.com/vyanhursky/joven/actions/workflows/ci.yml)
37
+ [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)
38
+ [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
39
+
40
+ ```
41
+ :@@# =@@+.
42
+ +@@@@ #@@@@.
43
+ .@@#@@ .*@@=@@.
44
+ #@@@@% #@@=@@#
45
+ @@:@@* :@@+:@@
46
+ @@=@@@ %@@-:@@
47
+ @@%@@@*. .@@=*+@@
48
+ @@@+@@@=. *@%+#%@@
49
+ *@@@@%@@%=. .@@*:-@@@
50
+ @@@====#@%=. .@@@@*@@@.
51
+ :#@@*::--#@@@#.%@@@@@: #@@=+**@@.
52
+ =@@#-*+@@@@@@@@@@@@@@% @@@%*#*@@.
53
+ -@@@@@@@@@@@-%@@@%..*@@@@@@-+@@@@@@%*+@@
54
+ +@@@@@@@@@@@@@@=:: ....:::::%@@@-@#*=:+#**@@@
55
+ =@@@@@@@*:.:::.:.::.:....... :@@@%@:**=:-%@@@+
56
+ .%@@@- :.. ..::: ....::::-=#@@@=:+=**#@%=
57
+ .. %@@@@@@+ :.:...=@@@@@@: .:--:**==%@@+%#%*#*-
58
+ .@@@@@@@@@@@+=- ::: :@@@@@@@+. :.:--#-*:@@@#%@+
59
+ +@@@@@@@@@@@@%. :+-*:. *@@=@@@@@.:=@@%:-*=%@@.
60
+ .#@@@@@@+@@+=--==:==-:: .:=+@@+%@%@@@@@@@=@*%+ @@%.
61
+ +@@@-===@-@==-----::::: .===*@@@@@@@+@%#=*%@%@#@@
62
+ %@@@@@%=-:=%@%:---==.-=-:. ..-++*%@@@@@:=*@@ @@@@@+@@
63
+ ...%@@@@@=@#=..:-:=====-: :..:: .:-==+:+@@@*@@*+%@@%@@-@@
64
+ #@@@@%*@@*@*=:-:-**=-: ...:-==: .:=*:*+=+=:%@@@@=@@@@%
65
+ .#@@@@@:@*=@@===:.=+-. :::.:::. *@-+*:.. =@@@@= %@@@=
66
+ *@@@=%==@@@#+=@@@@@@@+.: : *@*%-:..:.@@@
67
+ @@@=*.:+:-@@@@@@@@%@@@%.:: .=@+@@:. :--#@@
68
+ @@@=@@%*@%@*=*@-@@##@@@%=:::: .:==-:::=@*+%-. .:=:.@@
69
+ %@@@@%-%%%@+@@@@@=*=-@@*=#=.:=+-:::.:-**+%+: .:=::@@
70
+ %@@@@@@=@*@@@@@@@@@@@*%=#@%=:-::-:-::*%*:..::::#@@@
71
+ .=@@@@@@+@@@%-@@@@*.:%#*.@=*-:=..:===- ::::-=*@@@@
72
+ #@@@@@@@@@@% :=+@@% :=+- .:-: : .::-+@@@@+
73
+ .#@@@@@@@@@. ..:=@@@@@@@@*
74
+ . %@@@@@@@@@@@%%##@@@@@@@@@@@@@@@%-
75
+ .#@@@@@@@@@@@@@@@@@@@@@#+=:
76
+ -%@@@@@@@+:.
77
+ ```
78
+
79
+ > The boy sat down to read the book. He read of horses and of the country to the
80
+ > south and of men who spoke in a tongue he did not have.
81
+ > *Escúchame, joven,* said the old man, and the boy did not know what it was he
82
+ > was meant to hear. So he set down the book and took up the phone and fed the
83
+ > words into Google Translate one at a time like a man counting stones across a
84
+ > river, and when he looked up again the twilight had gone out of the valley and
85
+ > the page had gone cold and the old man was still standing there in the dark
86
+ > holding his counsel like a man holding a lamp for nobody.
87
+ >
88
+ > So the boy wrote a python service. It went through the book and marked out every
89
+ > passage that was not English and put the question to a small model that ran on
90
+ > his own machine and asked nothing of anyone and told no one what it had seen.
91
+ > Where the Spanish had been it set a single asterisk and no more than that, and
92
+ > under the asterisk it laid the English down like a coin under a tongue, and the
93
+ > boy could take it or leave it as he pleased. He read the book through and he
94
+ > never once opened Google Translate. The old man went on speaking in his own
95
+ > language as he had always done and as he would go on doing, and the boy
96
+ > understood him, and the sun went down bloodred over the mesa and he read on by
97
+ > the pale light of the device until the battery gave out.
98
+
99
+ **Joven is a command-line tool that finds the untranslated Spanish in an English
100
+ novel and inserts tappable translation footnotes.** You give it an EPUB you own;
101
+ it gives you back an annotated copy for your e-reader, with the prose byte-for-byte
102
+ unchanged. It was written for Cormac McCarthy's *The Crossing* and works well on
103
+ his other westerns.
104
+
105
+ Everything runs on your machine against a local model: no API key, no per-book
106
+ cost, nothing uploaded.
107
+
108
+ ---
109
+
110
+ ## Why this exists
111
+
112
+ *The Crossing* is an English novel with a great deal of Spanish in it, and the book
113
+ never translates or contextualizes it. **726 Spanish sentences and phrases**, mixed
114
+ into English paragraphs. Reaching for a dictionary breaks the trance the prose spent
115
+ forty pages building; skimming past leaves a hole in the page.
116
+
117
+ Your e-reader's own Dictionary and Translate do not rescue you, because McCarthy
118
+ uses no quotation marks — speech and narration run together in one stream, and the
119
+ Spanish arrives in four distinct shapes:
120
+
121
+ ```text
122
+ A Vaya con Dios. a whole paragraph, no English at all
123
+ B Cuántos años tienes? the old man said. Spanish speech, English dialogue tag
124
+ C The matríz will not help you, he said. English prose, Spanish loanword
125
+ D Escúchame, joven, the old man wheezed. Spanish opener, then English narration
126
+ ```
127
+
128
+ C and D are why per-paragraph language detection fails in *both* directions: C is a
129
+ false positive waiting to happen, D is a guaranteed miss. Solving that is most of
130
+ what this tool is — see [DESIGN.md §2](DESIGN.md) for the measurements.
131
+
132
+ ## What you get
133
+
134
+ A copy of your EPUB in which every Spanish passage carries a small `*`. Tap it and
135
+ the translation appears in the reader's own footnote popup; ignore it and the page
136
+ reads exactly as the author set it down.
137
+
138
+ <table>
139
+ <tr>
140
+ <td width="50%"><img src="docs/images/kobo-asterisk-marker.jpg" alt="A page of The Crossing on a Kobo, with a small asterisk after each Spanish line"></td>
141
+ <td width="50%"><img src="docs/images/kobo-footnote-preview.jpg" alt="The same page with Kobo's Footnote preview open, showing the English translation"></td>
142
+ </tr>
143
+ <tr>
144
+ <td><em>The page as McCarthy set it down, plus one asterisk per Spanish passage.</em></td>
145
+ <td><em>The same page one tap later — the Kobo's own <strong>Footnote preview</strong>.</em></td>
146
+ </tr>
147
+ </table>
148
+
149
+ - **Unintrusive by construction.** One asterisk. No inline brackets, no interlinear
150
+ clutter, no colour. The footnote is opt-in, the way McCarthy's silence is opt-in.
151
+ - **The prose is untouched.** Strip the inserted nodes from the output and what
152
+ remains is *byte-identical* to the original — enforced by a test, verified on the
153
+ real book.
154
+ - **It stays on your machine.** A local `qwen3:8b` does the translating — 73
155
+ minutes for the whole novel, $0, and the book never leaves the laptop.
156
+ - **Precision over recall.** A spurious footnote on `Go on.` is worse than a missed
157
+ one, so ambiguous cases escalate to the model rather than guess.
158
+
159
+ ## How it works
160
+
161
+ Five commands, two of which do the real work. Detection is **two-tier**: a cheap
162
+ statistical pass judges every sentence, and only the fraction it cannot call is put
163
+ to the local LLM.
164
+
165
+ ```mermaid
166
+ flowchart TD
167
+ START(["your book.epub"])
168
+
169
+ subgraph DETECT["joven detect · ~73 min · $0"]
170
+ direction TB
171
+ EXTRACT["extract text units<br>addressable by file, element path, offset"]
172
+ SEG["segment into sentences<br>character offsets preserved"]
173
+ T1{"TIER 1 — lingua triage<br>statistical, instant, free"}
174
+ T2{"TIER 2 — local LLM<br>qwen3:8b via Ollama, with surrounding context"}
175
+ MERGE["merge contiguous Spanish<br>one footnote per paragraph"]
176
+ EXTRACT --> SEG --> T1
177
+ T1 -- "abstains — 21% of segments" --> T2
178
+ T1 -- "confident Spanish" --> MERGE
179
+ T2 -- "yes, and here is the span" --> MERGE
180
+ end
181
+
182
+ SKIP["left alone<br>but still recorded in trace.jsonl"]
183
+ T1 -- "confident English" --> SKIP
184
+ T2 -- "not Spanish" --> SKIP
185
+
186
+ SIDECAR[("annotations.json<br>source of truth, human-editable")]
187
+ REVIEW["joven review<br>approve / edit / reject<br>decisions survive re-detection"]
188
+ MERGE --> SIDECAR
189
+ SIDECAR <--> REVIEW
190
+
191
+ subgraph RENDER["joven render · idempotent"]
192
+ direction TB
193
+ ANN["insert markers and note documents<br>EPUB 2 → 3 package upgrade"]
194
+ EPUB3(["book.annotated.epub"])
195
+ KEP["kepubify"]
196
+ ANN --> EPUB3 --> KEP
197
+ end
198
+
199
+ START --> EXTRACT
200
+ START -. "the original, never edited in place" .-> ANN
201
+ SIDECAR --> ANN
202
+ KEP --> OUT(["book.annotated.kepub.epub<br>copy this to the Kobo"])
203
+ EPUB3 -. "validated by" .-> VERIFY["joven verify<br>12 checks · epubcheck · text-preservation invariant"]
204
+
205
+ classDef artifact fill:#fff4e0,stroke:#b8860b,stroke-width:2px,color:#111
206
+ classDef endpoint fill:#e6f4ea,stroke:#2e7d4f,stroke-width:2px,color:#111
207
+ classDef muted fill:#f2f2f2,stroke:#999,color:#444
208
+ class SIDECAR artifact
209
+ class START,OUT endpoint
210
+ class SKIP muted
211
+ ```
212
+
213
+ **The EPUB is never edited in place.** `annotations.json` is the durable artifact,
214
+ and rendering is a pure function of (original EPUB + sidecar). Corrections mean
215
+ editing the sidecar and re-rendering — never re-translating — and re-running
216
+ detection merges into your edits instead of clobbering them.
217
+
218
+ ### Inside one Tier-2 call
219
+
220
+ Tier 2 is not a translation step with a question bolted on. It answers three
221
+ things at once — **is this Spanish**, **which part of it is Spanish**, and **what
222
+ does it mean** — which is why it is an instruct model and not a translation
223
+ engine. An NMT engine can only answer the third, and would cheerfully "translate"
224
+ `Yes mam.`
225
+
226
+ ```mermaid
227
+ flowchart TB
228
+ subgraph REQUEST["one /api/chat request to Ollama"]
229
+ direction TB
230
+ SYS["SYSTEM<br>the task, the JSON contract, and a warning<br>that McCarthy's dialect English is English"]
231
+ FS["6 FEW-SHOT TURNS<br>one per mixing pattern, plus two dialect traps<br>added after a traced run exposed them"]
232
+ USR["USER<br>up to 400 characters of preceding prose, fenced<br>and marked do-not-translate, then the paragraph"]
233
+ SYS --- FS --- USR
234
+ end
235
+
236
+ USR --> MODEL["qwen3:8b · temperature 0 · thinking off<br>reply constrained to a JSON schema"]
237
+ MODEL --> OUT["is_spanish · spanish_text · translation"]
238
+
239
+ OUT --> G1{"is the translation<br>just the source tidied up?"}
240
+ G1 -- yes --> VETO["vetoed — a footnote<br>that teaches nothing"]
241
+ G1 -- no --> G2{"one Spanish word<br>inside English prose?"}
242
+ G2 -- yes --> LOAN["suppressed —<br>embedded loanword"]
243
+ G2 -- no --> KEEP["annotation, span narrowed<br>to spanish_text"]
244
+
245
+ VETO --> TRACE[("trace.jsonl<br>every outcome, with the raw reply")]
246
+ LOAN --> TRACE
247
+ KEEP --> TRACE
248
+
249
+ classDef keep fill:#e6f4ea,stroke:#2e7d4f,stroke-width:2px,color:#111
250
+ classDef drop fill:#f2f2f2,stroke:#999,color:#444
251
+ classDef store fill:#fff4e0,stroke:#b8860b,stroke-width:2px,color:#111
252
+ class KEEP keep
253
+ class VETO,LOAN drop
254
+ class TRACE store
255
+ ```
256
+
257
+ **There are two prompts, not one**, and which runs depends on what Tier 1 already
258
+ decided. The split was forced by a bug in each direction:
259
+
260
+ | Tier 1 said | Prompt | Because |
261
+ |---|---|---|
262
+ | abstained (the band) | **adjudicate** — is this Spanish, and if so which part? | The genuine open question |
263
+ | confident Spanish | **translate only** — this *is* Spanish; do not second-guess it | A combined prompt let the model veto Tier 1's correct calls. It labels `Dieciseis.` English. |
264
+
265
+ Skipping the model entirely for Tier-1 accepts was the first implementation, and
266
+ it shipped annotations with **no translation at all** — blank popups on the
267
+ device.
268
+
269
+ The prompts themselves, the context-bleed problem, a real call from the trace, and
270
+ the three gates in detail: [docs/anatomy-of-a-call.md](docs/anatomy-of-a-call.md).
271
+
272
+ ### The components
273
+
274
+ `src/joven/` is about 4,500 lines. The pieces map onto the pipeline above:
275
+
276
+ | Module | Job |
277
+ |---|---|
278
+ | [`epub/`](src/joven/epub) | Byte-level zip surgery — copy every archive entry verbatim, reserialize only the XHTML that actually changed |
279
+ | [`detect/`](src/joven/detect) | `segment.py` splits paragraphs into sentences; `triage.py` is Tier 1 (`lingua` + tag-stripping); `pipeline.py` runs both tiers and merges contiguous Spanish into one footnote per paragraph |
280
+ | [`translate.py`](src/joven/translate.py) | Tier 2 — the Ollama client, the `Translator` protocol, and the similarity veto that suppresses no-op "translations" |
281
+ | [`dialogue.py`](src/joven/dialogue.py) | McCarthy's `, he said.` vocabulary, shared by every gate that must not measure it |
282
+ | [`model.py`](src/joven/model.py) | The `annotations.json` sidecar — content-hash IDs, statuses, merge semantics |
283
+ | [`render/`](src/joven/render) | Marker insertion, the EPUB 2→3 package upgrade, one-note-per-file footnote documents |
284
+ | [`kepub.py`](src/joven/kepub.py) | The `kepubify` hand-off that produces the file the Kobo wants |
285
+ | [`review.py`](src/joven/review.py), [`suspicion.py`](src/joven/suspicion.py) | The localhost review UI, and the heuristic that sorts likely-wrong translations to the top of it |
286
+ | [`verify.py`](src/joven/verify.py) | The 12-check integrity gate, including the text-preservation invariant |
287
+ | [`trace.py`](src/joven/trace.py) | One JSONL record per segment, annotated or not, so any missing footnote can be explained afterwards |
288
+
289
+ ## Requirements
290
+
291
+ | | |
292
+ |---|---|
293
+ | **Platform** | **macOS on Apple Silicon** — where this was developed and device-verified. CI also runs the suite on Linux. Windows is untested. |
294
+ | **Memory** | 16 GB, which is what sets the model ceiling at 7–14B parameters |
295
+ | **Disk** | ~6 GB for the model, plus room for the outputs |
296
+ | **Python** | 3.11+ (developed on 3.13) |
297
+ | **Homebrew** | for the three external binaries below |
298
+ | **Java** | `epubcheck` is a JAR and needs a JVM; macOS ships one at `/usr/bin/java` |
299
+ | **Reader** | A **Kobo** (sideloaded over USB) is the verified target — tested on firmware `4.45.23697`. Apple Books renders the footnotes as popups too. |
300
+
301
+ Three external tools do work Python should not:
302
+
303
+ | Tool | Why |
304
+ |---|---|
305
+ | [`ollama`](https://ollama.com) | Runs `qwen3:8b` (5.2 GB) locally. Chosen on a benchmark, not a hunch — [docs/model-selection.md](docs/model-selection.md). |
306
+ | [`kepubify`](https://github.com/pgaskin/kepubify) | Converts the finished EPUB into Kobo's own KEPUB flavour, which renders better on device |
307
+ | [`epubcheck`](https://www.w3.org/publishing/epubcheck/) | The reference EPUB validator — the external gate on our output |
308
+
309
+ **Bring your own EPUB.** Joven reads a file you already have; it does not fetch,
310
+ share, or unlock anything, and it refuses DRM-protected files outright. Books that
311
+ merely *obfuscate* an embedded font are read normally — that is not DRM, and the
312
+ scrambled font is carried through untouched.
313
+
314
+ ## Install
315
+
316
+ The three external binaries first:
317
+
318
+ ```bash
319
+ brew install epubcheck kepubify ollama
320
+ ```
321
+
322
+ Then the tool, into its own isolated environment:
323
+
324
+ ```bash
325
+ uv tool install joven-ebook-annotator
326
+ ```
327
+
328
+ `pipx install joven-ebook-annotator` does the same thing if you have pipx instead.
329
+ Both pull prebuilt wheels — worth knowing that `lingua` carries its language models
330
+ inside the wheel, so this step moves about 170 MB.
331
+
332
+ Then the model, once:
333
+
334
+ ```bash
335
+ ollama serve & # leave running
336
+ ollama pull qwen3:8b # 5.2 GB
337
+ ```
338
+
339
+ > There is no Homebrew formula. One dependency (`lingua-language-detector`)
340
+ > publishes no source distribution at all — only per-platform wheels — which a
341
+ > Homebrew Python formula cannot consume without hand-pinned wheel URLs per
342
+ > architecture and per CPython minor version. Two commands that work beat one
343
+ > command that breaks on the next `python@` bump.
344
+
345
+ To work on the code rather than use it, see [Development](#development).
346
+
347
+ ## Use
348
+
349
+ One book, five commands, in order:
350
+
351
+ ```bash
352
+ joven inspect book.epub # structure, DRM, word counts
353
+ joven detect book.epub -o annotations.json --trace trace.jsonl
354
+ joven review annotations.json --epub book.epub # triage, suspect passages first
355
+ joven render book.epub annotations.json -o out/ # EPUB 3 + KEPUB for the Kobo
356
+ joven verify out/book.annotated.epub --original book.epub
357
+ ```
358
+
359
+ **`inspect`** reports what you are holding — EPUB version, DRM, spine, word counts.
360
+ Run it first; it fails fast on a file the rest of the pipeline cannot use.
361
+
362
+ **`detect`** is the slow step: about 73 minutes for a 150,000-word novel, single
363
+ threaded, $0. It writes `annotations.json` (the sidecar you will edit) and, with
364
+ `--trace`, a JSONL record of every segment it looked at. Re-running it is safe —
365
+ your review decisions are sticky.
366
+
367
+ The trace is also the run's recovery log: it is flushed a record at a time, so an
368
+ interrupted run loses nothing that the model already answered.
369
+
370
+ ```bash
371
+ joven detect book.epub -o annotations.json --trace trace.jsonl --resume trace.jsonl
372
+ ```
373
+
374
+ `--resume` reuses every model answer the trace already holds and pays only for the
375
+ segments the previous run never reached. Tier 1 and every suppression gate still run
376
+ over the whole book, so a resumed run picks up threshold changes rather than
377
+ replaying stale conclusions; recorded *errors* are retried rather than inherited.
378
+
379
+ **`review`** opens a local page listing every annotation with the surrounding prose
380
+ and the Spanish highlighted. Approve, edit, or reject (`a`/`r`/`e`); each decision
381
+ writes straight to the sidecar the moment you make it, so quitting mid-review loses
382
+ nothing. Suspect annotations sort first, each badged with the reason —
383
+ `untranslated: historic`, `garbled source: Conoc16` — and everything else follows in
384
+ book order.
385
+
386
+ **`render`** applies the sidecar to a fresh copy of the original and emits two
387
+ files: a spec-clean `.epub` (the verifiable intermediate, which is what `epubcheck`
388
+ validates) and a `.kepub.epub` (what you copy to the Kobo). It is idempotent and
389
+ never touches your source file.
390
+
391
+ **`verify`** runs the 12-check integrity gate — the text-preservation invariant,
392
+ `epubcheck`, noteref→footnote resolution, and the rest.
393
+
394
+ Then copy the `.kepub.epub` to the Kobo's root over USB, eject, and the device
395
+ imports it.
396
+
397
+ Missed a passage while reading?
398
+
399
+ ```bash
400
+ joven add annotations.json --epub book.epub \
401
+ --find "Bueno pues" --translation "Well then"
402
+ ```
403
+
404
+ Added entries are marked `edited`, so re-detection will never overwrite them.
405
+
406
+ For tuning, tracing, and the debug flags, see
407
+ [docs/troubleshooting.md](docs/troubleshooting.md).
408
+
409
+ ## Results
410
+
411
+ Working end to end on a real book: lossless round-trip, two-tier detection with a
412
+ full decision trace, EPUB 2→3 upgrade, footnote rendering device-verified on a Kobo,
413
+ KEPUB output, a review pass, and a finished book on the device.
414
+
415
+ Last full run — Knopf's 1994 edition of *The Crossing*, 151,865 words:
416
+
417
+ | | |
418
+ |---|---|
419
+ | segments considered | 12,302 |
420
+ | escalated to the LLM | 2,556 (21%) |
421
+ | footnotes produced | 726 |
422
+ | wall clock / cost | 73 minutes / $0 |
423
+ | integrity checks | 12 of 12 passing |
424
+
425
+ Known limitations, and what the project has and has not proven, are in
426
+ [CHANGELOG.md](CHANGELOG.md).
427
+
428
+ ## Development
429
+
430
+ ```bash
431
+ git clone https://github.com/vyanhursky/joven && cd joven
432
+ python3.13 -m venv .venv
433
+ ./.venv/bin/pip install -e '.[dev]'
434
+ ```
435
+
436
+ ```bash
437
+ ./.venv/bin/pytest # 369 tests, synthetic fixtures only
438
+ ./.venv/bin/ruff check src tests tools
439
+
440
+ # opt in to the real-book tests (the book is never committed)
441
+ JOVEN_TEST_EPUB=/path/to/book.epub ./.venv/bin/pytest
442
+ ```
443
+
444
+ Tests never hit the network — a `StubTranslator` stands in for the LLM everywhere.
445
+ Two guard scripts run in CI, each of which has caught a real bug that survived a
446
+ careful manual read of the same files minutes earlier:
447
+
448
+ ```bash
449
+ python tools/check_docs.py # every documented command and flag exists
450
+ python tools/check_no_book_content.py # no book text in tracked files
451
+ ```
452
+
453
+ Model benchmarks are separate tools, not tests, because they need a running Ollama:
454
+
455
+ ```bash
456
+ python tools/bench_models.py # the LLM in isolation
457
+ python tools/bench_pipeline.py # the two-tier system that actually ships
458
+ ```
459
+
460
+ ## Further reading
461
+
462
+ | | |
463
+ |---|---|
464
+ | [DESIGN.md](DESIGN.md) | Why the architecture is shaped this way — the measurements behind every decision, what the device tests overturned, and the work deliberately left undone |
465
+ | [docs/model-selection.md](docs/model-selection.md) | The local-model benchmark: why `qwen3:8b` |
466
+ | [docs/anatomy-of-a-call.md](docs/anatomy-of-a-call.md) | What the local model is asked, what it may answer, and the gates that check it |
467
+ | [docs/troubleshooting.md](docs/troubleshooting.md) | Tracing a missing footnote, improving translation quality, debug flags, and Kobo quirks |
468
+ | [docs/releasing.md](docs/releasing.md) | Cutting a release: the one-time PyPI trusted-publisher setup, and what CI does with a tag |
469
+ | [CHANGELOG.md](CHANGELOG.md) | Release notes and known limitations |
470
+
471
+ ## Licence
472
+
473
+ The **code** is MIT — see [LICENSE](LICENSE). Use it, fork it, ship it.
474
+
475
+ The **books are not.** The annotated output is a derivative of a copyrighted work:
476
+ read it, don't distribute it.
477
+
478
+ **The repository deliberately tracks no book content.** The hazard is not the EPUB
479
+ but what the pipeline derives from it — a full decision trace holds every segment's
480
+ source text verbatim, which for a 150,000-word novel is the entire book. `.gitignore`
481
+ excludes those by pattern, and
482
+ [`tools/check_no_book_content.py`](tools/check_no_book_content.py) verifies it by
483
+ *content*, diffing every tracked file against the book itself:
484
+
485
+ ```bash
486
+ python tools/check_no_book_content.py path/to/book.epub
487
+ ```
488
+
489
+ Docs and tests quote short passages to illustrate the detection problem, and the two
490
+ screenshots above are photographs of the author's own copy on his own device.