interpreter-v2 0.1.0__tar.gz

interpreter_v2-0.1.0/.claude/settings.local.json

@@ -0,0 +1,27 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(python -m interpreter:*)",
+      "Bash(where.exe:*)",
+      "Bash(/c/Users/bertr/.local/bin/uv sync)",
+      "Bash(/c/Users/bertr/.local/bin/uv run:*)",
+      "Bash(PYTHONIOENCODING=utf-8 /c/Users/bertr/.local/bin/uv run:*)",
+      "Bash(tasklist:*)",
+      "Bash(taskkill:*)",
+      "Bash(git ls-tree:*)",
+      "WebFetch(domain:github.com)",
+      "WebFetch(domain:raw.githubusercontent.com)",
+      "WebFetch(domain:pypi.org)",
+      "Bash(PYTHONIOENCODING=utf-8 PYTHONUNBUFFERED=1 /c/Users/bertr/.local/bin/uv run python:*)",
+      "Bash(start \"\" \"C:\\\\Users\\\\bertr\\\\code\\\\interpreter\\\\debug_capture.png\")",
+      "Bash(start \"\" \"C:\\\\Users\\\\bertr\\\\code\\\\interpreter\\\\current_capture.png\")",
+      "Bash(git add:*)",
+      "Bash(git commit -m \"$\\(cat <<''EOF''\nAdd Windows support with DirectX capture\n\n- Use windows-capture library for Windows Graphics Capture API\n- Supports DirectX/OpenGL content capture from background windows\n- Add streaming capture interface for both Windows and macOS\n- Auto-detect fullscreen mode to skip title bar cropping\n- Fix overlay transparency and font for Windows\n- Fix CUDA fallback to CPU for translation\n- Add ''q'' hotkey to quit \\(Ctrl+C workaround on Windows\\)\n- Update default font size to 40\n\n🤖 Generated with [Claude Code]\\(https://claude.com/claude-code\\)\n\nCo-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>\nEOF\n\\)\")",
+      "Bash(git config:*)",
+      "Bash(git push:*)",
+      "WebSearch",
+      "WebFetch(domain:opennmt.net)",
+      "Bash(/c/Users/bertr/.local/bin/uv build:*)"
+    ]
+  }
+}

interpreter_v2-0.1.0/.gitignore

@@ -0,0 +1,28 @@
+# Binary
+/interpreter
+/interpreter.exe
+
+# Models directory (downloaded on first run)
+models/
+
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+.venv/
+venv/
+ENV/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+*.iml
+
+# OS
+.DS_Store
+Thumbs.db
+.env

interpreter_v2-0.1.0/.python-version

@@ -0,0 +1 @@
+3.11

interpreter_v2-0.1.0/PKG-INFO

@@ -0,0 +1,149 @@
+Metadata-Version: 2.4
+Name: interpreter-v2
+Version: 0.1.0
+Summary: Offline screen translator for Japanese retro games
+Author: qube
+License-Expression: MIT
+Keywords: games,japanese,ocr,retro,translation
+Requires-Python: >=3.11
+Requires-Dist: ctranslate2
+Requires-Dist: meikiocr
+Requires-Dist: mss>=9.0.0
+Requires-Dist: numpy
+Requires-Dist: nvidia-cublas-cu12; sys_platform == 'win32'
+Requires-Dist: nvidia-cudnn-cu12; sys_platform == 'win32'
+Requires-Dist: onnxruntime-gpu; sys_platform == 'win32'
+Requires-Dist: opencv-python
+Requires-Dist: pillow>=10.0.0
+Requires-Dist: pygetwindow>=0.0.9; sys_platform == 'win32'
+Requires-Dist: pynput>=1.7.6
+Requires-Dist: pyobjc-framework-quartz>=10.0; sys_platform == 'darwin'
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: requests
+Requires-Dist: sentencepiece
+Requires-Dist: tqdm
+Requires-Dist: windows-capture>=1.4.0; sys_platform == 'win32'
+Description-Content-Type: text/markdown
+
+# Interpreter
+
+Offline screen translator for Japanese retro games. Captures text from any window, performs OCR, translates to English, and displays subtitles in a floating overlay.
+
+![sample](sample.jpg)
+
+## Features
+
+- **Fully offline** - No cloud APIs, no internet required after setup
+- **Free** - No API costs or subscriptions
+- **Private** - Text never leaves your machine
+- **Optimized for retro games** - Uses MeikiOCR, trained specifically on Japanese game text
+- **Two overlay modes** - Banner (subtitle bar) or inplace (text over game)
+- **Translation caching** - Fuzzy matching avoids re-translating similar text
+
+## Requirements
+
+- Python 3.11+
+- macOS (Windows/Linux support planned)
+
+## Installation
+
+1. Install [uv](https://docs.astral.sh/uv/getting-started/installation/) (modern Python package manager):
+   ```bash
+   curl -LsSf https://astral.sh/uv/install.sh | sh
+   ```
+
+2. Clone and install:
+   ```bash
+   git clone https://github.com/bquenin/interpreter.git
+   cd interpreter
+   uv sync
+   ```
+
+3. Models are downloaded automatically on first run (~1.5GB total).
+
+## Usage
+
+```bash
+# List available windows
+uv run interpreter --list-windows
+
+# Run with default config
+uv run interpreter
+
+# Run with specific window
+uv run interpreter --window "Tales"
+
+# Start in inplace mode (text over game)
+uv run interpreter --overlay-mode inplace
+
+# Debug mode (show OCR confidence scores)
+uv run interpreter --debug
+```
+
+## Controls
+
+| Key | Action |
+|-----|--------|
+| `m` | Cycle overlay mode: off → banner → inplace |
+| `-` | Decrease font size |
+| `=` | Increase font size |
+| `Ctrl+C` | Quit |
+
+In banner mode, you can drag the overlay to reposition it.
+
+## Overlay Modes
+
+### Banner Mode (default)
+A subtitle bar at the bottom of the screen displaying translated text. Draggable, opaque background, centered text.
+
+### Inplace Mode
+Transparent overlay positioned over the game window. Translated text appears directly over the original Japanese text at OCR-detected positions. Click-through so you can interact with the game.
+
+## Configuration
+
+Edit `config.yml`:
+
+```yaml
+# Window to capture (partial title match)
+window_title: "Tales"
+
+# Refresh rate in seconds
+refresh_rate: 0.5
+
+# OCR confidence threshold (0.0-1.0)
+# Filters out garbage text by average per-line confidence
+ocr_confidence: 0.6
+
+# Subtitle appearance
+font_size: 24
+font_color: "#FFFFFF"
+background_color: "#404040"
+```
+
+## How It Works
+
+1. **Screen Capture** - Captures the target window at the configured refresh rate
+2. **OCR** - [MeikiOCR](https://github.com/rtr46/meikiocr) extracts Japanese text (optimized for pixel fonts)
+3. **Translation** - [Sugoi V4](https://huggingface.co/entai2965/sugoi-v4-ja-en-ctranslate2) translates Japanese to English
+4. **Display** - Shows translated text in the selected overlay mode
+
+## Troubleshooting
+
+### Window not found
+Use `--list-windows` to see available windows. The window title is a partial match.
+
+### Poor OCR accuracy
+Try adjusting `ocr_confidence` in config. Lower values include more text (but may include garbage), higher values are stricter.
+
+### Slow performance
+First run downloads models (~1.5GB). Subsequent runs use cached models from `~/.cache/huggingface/`.
+
+## What's New in v2
+
+- **Inplace overlay mode** - Text appears directly over game text
+- **Mode cycling** - Press `m` to switch between off/banner/inplace
+- **Font size adjustment** - Press `-`/`=` to resize on the fly
+- **Translation caching** - Fuzzy matching reduces redundant translations
+- **Improved OCR** - Punctuation excluded from confidence calculation
+- **Better window capture** - Excludes overlapping windows, auto-detects fullscreen
+- **Multi-display support** - Overlay appears on the same display as the game

interpreter_v2-0.1.0/README.md

@@ -0,0 +1,122 @@
+# Interpreter
+
+Offline screen translator for Japanese retro games. Captures text from any window, performs OCR, translates to English, and displays subtitles in a floating overlay.
+
+![sample](sample.jpg)
+
+## Features
+
+- **Fully offline** - No cloud APIs, no internet required after setup
+- **Free** - No API costs or subscriptions
+- **Private** - Text never leaves your machine
+- **Optimized for retro games** - Uses MeikiOCR, trained specifically on Japanese game text
+- **Two overlay modes** - Banner (subtitle bar) or inplace (text over game)
+- **Translation caching** - Fuzzy matching avoids re-translating similar text
+
+## Requirements
+
+- Python 3.11+
+- macOS (Windows/Linux support planned)
+
+## Installation
+
+1. Install [uv](https://docs.astral.sh/uv/getting-started/installation/) (modern Python package manager):
+   ```bash
+   curl -LsSf https://astral.sh/uv/install.sh | sh
+   ```
+
+2. Clone and install:
+   ```bash
+   git clone https://github.com/bquenin/interpreter.git
+   cd interpreter
+   uv sync
+   ```
+
+3. Models are downloaded automatically on first run (~1.5GB total).
+
+## Usage
+
+```bash
+# List available windows
+uv run interpreter --list-windows
+
+# Run with default config
+uv run interpreter
+
+# Run with specific window
+uv run interpreter --window "Tales"
+
+# Start in inplace mode (text over game)
+uv run interpreter --overlay-mode inplace
+
+# Debug mode (show OCR confidence scores)
+uv run interpreter --debug
+```
+
+## Controls
+
+| Key | Action |
+|-----|--------|
+| `m` | Cycle overlay mode: off → banner → inplace |
+| `-` | Decrease font size |
+| `=` | Increase font size |
+| `Ctrl+C` | Quit |
+
+In banner mode, you can drag the overlay to reposition it.
+
+## Overlay Modes
+
+### Banner Mode (default)
+A subtitle bar at the bottom of the screen displaying translated text. Draggable, opaque background, centered text.
+
+### Inplace Mode
+Transparent overlay positioned over the game window. Translated text appears directly over the original Japanese text at OCR-detected positions. Click-through so you can interact with the game.
+
+## Configuration
+
+Edit `config.yml`:
+
+```yaml
+# Window to capture (partial title match)
+window_title: "Tales"
+
+# Refresh rate in seconds
+refresh_rate: 0.5
+
+# OCR confidence threshold (0.0-1.0)
+# Filters out garbage text by average per-line confidence
+ocr_confidence: 0.6
+
+# Subtitle appearance
+font_size: 24
+font_color: "#FFFFFF"
+background_color: "#404040"
+```
+
+## How It Works
+
+1. **Screen Capture** - Captures the target window at the configured refresh rate
+2. **OCR** - [MeikiOCR](https://github.com/rtr46/meikiocr) extracts Japanese text (optimized for pixel fonts)
+3. **Translation** - [Sugoi V4](https://huggingface.co/entai2965/sugoi-v4-ja-en-ctranslate2) translates Japanese to English
+4. **Display** - Shows translated text in the selected overlay mode
+
+## Troubleshooting
+
+### Window not found
+Use `--list-windows` to see available windows. The window title is a partial match.
+
+### Poor OCR accuracy
+Try adjusting `ocr_confidence` in config. Lower values include more text (but may include garbage), higher values are stricter.
+
+### Slow performance
+First run downloads models (~1.5GB). Subsequent runs use cached models from `~/.cache/huggingface/`.
+
+## What's New in v2
+
+- **Inplace overlay mode** - Text appears directly over game text
+- **Mode cycling** - Press `m` to switch between off/banner/inplace
+- **Font size adjustment** - Press `-`/`=` to resize on the fly
+- **Translation caching** - Fuzzy matching reduces redundant translations
+- **Improved OCR** - Punctuation excluded from confidence calculation
+- **Better window capture** - Excludes overlapping windows, auto-detects fullscreen
+- **Multi-display support** - Overlay appears on the same display as the game

interpreter_v2-0.1.0/benchmark/README.md

@@ -0,0 +1,106 @@
+# OCR Benchmark
+
+This folder contains test data and scripts for benchmarking OCR accuracy on Japanese game text.
+
+## Structure
+
+```
+benchmark/
+├── data/                    # Test cases
+│   ├── 001_miguel_pendant/
+│   │   ├── textbox.png      # Cropped textbox image (required)
+│   │   ├── screenshot.png   # Full game screenshot (optional, for future textbox detection)
+│   │   ├── ground_truth.txt # Expected OCR output (required)
+│   │   └── textbox_coords.json  # Textbox location in screenshot (optional)
+│   └── ...
+├── results/                 # Benchmark results (JSON)
+├── run_benchmark.py         # Benchmark script
+└── README.md
+```
+
+## Running Benchmarks
+
+```bash
+# Run full benchmark (all engines, all preprocessing)
+python benchmark/run_benchmark.py
+
+# Run specific engine
+python benchmark/run_benchmark.py --engine meikiocr
+
+# Run specific preprocessing
+python benchmark/run_benchmark.py --preprocessing otsu
+
+# Save results to JSON
+python benchmark/run_benchmark.py --save
+```
+
+## Adding New Test Cases
+
+1. Create a new folder in `benchmark/data/` with format `NNN_description/`:
+   ```
+   benchmark/data/003_my_test_case/
+   ```
+
+2. Add required files:
+   - `textbox.png` - Cropped image of the textbox only
+   - `ground_truth.txt` - Expected OCR output (exact text)
+
+3. Optionally add:
+   - `screenshot.png` - Full game screenshot
+   - `textbox_coords.json` - Bounding box coordinates for textbox detection testing
+
+4. Run benchmark to verify:
+   ```bash
+   python benchmark/run_benchmark.py --save
+   ```
+
+## OCR Engines
+
+| Engine | Description |
+|--------|-------------|
+| `manga_ocr` | VisionEncoderDecoder model for manga text |
+| `meikiocr` | ONNX model trained on Japanese video game text |
+
+## Preprocessing Methods
+
+| Method | Description |
+|--------|-------------|
+| `none` | No preprocessing, raw image |
+| `otsu` | Otsu binarization only |
+| `4x_lanczos` | 4x upscaling with LANCZOS interpolation |
+| `4x_otsu` | 4x upscaling + Otsu binarization |
+| `4x_otsu_invert` | 4x upscaling + Otsu + auto-invert |
+
+## Results Format
+
+Results are saved as JSON with this structure:
+
+```json
+{
+  "timestamp": "2024-12-29T00:00:00",
+  "engines": ["manga_ocr", "meikiocr"],
+  "preprocessing": ["none", "otsu", ...],
+  "test_cases": ["001_miguel_pendant", ...],
+  "results": [
+    {
+      "test_case": "001_miguel_pendant",
+      "engine": "meikiocr",
+      "preprocessing": "otsu",
+      "output": "...",
+      "ground_truth": "...",
+      "accuracy": 95.2
+    }
+  ]
+}
+```
+
+## Current Best Configuration
+
+Based on benchmarking: **MeikiOCR + Otsu** achieves ~86% average accuracy.
+
+| Engine | Preprocessing | Average Accuracy |
+|--------|---------------|------------------|
+| MeikiOCR | Otsu | ~86% |
+| MeikiOCR | None | ~80% |
+| manga_ocr | 4x+Otsu+Invert | ~85% |
+| manga_ocr | None | ~49% |

interpreter_v2-0.1.0/benchmark/data/001_miguel_pendant/ground_truth.txt

@@ -0,0 +1 @@
+ミゲール「お前にやったペンダントについてな。

interpreter_v2-0.1.0/benchmark/data/001_miguel_pendant/ground_truth_translation.txt

@@ -0,0 +1,2 @@
+Miguel: About the pendant I gave you.
+Miguel: It's about the pendant I gave you.

interpreter_v2-0.1.0/benchmark/data/002_cress_pendant/ground_truth.txt

@@ -0,0 +1 @@
+クレス「ペンダント？ああ、15の誕生日にもらった・・・・。あれがどうしたの？」

interpreter_v2-0.1.0/benchmark/data/002_cress_pendant/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Cress: A pendant? Oh, I got it on my fifteenth birthday... What about that?
+Cress: A pendant? Oh, I got it on my 15th birthday... What about it?
+Cress: Pendant? Oh, I got it on my fifteenth birthday... What happened to that?

interpreter_v2-0.1.0/benchmark/data/003_screen3/ground_truth.txt

@@ -0,0 +1 @@
+チェスター「それじゃ狩りに行ってくるよ、アミィ。

interpreter_v2-0.1.0/benchmark/data/003_screen3/ground_truth_translation.txt

@@ -0,0 +1,4 @@
+Chester: Well then, I'm going hunting, Ami.
+Chester: Well then, I'm off to hunt, Ami.
+Chester: I'm going to go hunting then, Ami.
+Chester: Well, I'm off hunting, Ami.

interpreter_v2-0.1.0/benchmark/data/004_screen4/ground_truth.txt

@@ -0,0 +1 @@
+アミィ「気をつけてね、お兄ちゃん。

interpreter_v2-0.1.0/benchmark/data/004_screen4/ground_truth_translation.txt

@@ -0,0 +1,4 @@
+Ami: Be careful, big brother.
+Ami: Take care, big brother.
+Ami: Be safe, big brother.
+Ami: Be careful, Onii-chan.

interpreter_v2-0.1.0/benchmark/data/005_screen5/ground_truth.txt

@@ -0,0 +1 @@
+チェスター「クレス、行くぞ！

interpreter_v2-0.1.0/benchmark/data/005_screen5/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Chester: Cress, let's go!
+Chester: Let's go, Cress!
+Chester: Cress, come on!

interpreter_v2-0.1.0/benchmark/data/006_screen6/ground_truth.txt

@@ -0,0 +1 @@
+ミゲール「あの声はチェスターか。

interpreter_v2-0.1.0/benchmark/data/006_screen6/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Miguel: That voice... is that Chester?
+Miguel: That voice is Chester's.
+Miguel: Is that Chester's voice?

interpreter_v2-0.1.0/benchmark/data/007_screen7/ground_truth.txt

@@ -0,0 +1 @@
+クレス「あ、父さんごめん。チェスターと約束してたんだ。

interpreter_v2-0.1.0/benchmark/data/007_screen7/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Cress: Oh, sorry Dad. I made a promise with Chester.
+Cress: Oh, sorry Father. I had a promise with Chester.
+Cress: Ah, sorry Dad. I promised Chester.

interpreter_v2-0.1.0/benchmark/data/008_screen8/ground_truth.txt

@@ -0,0 +1 @@
+ミゲール「ああ。ペンダントのことは夕食の時にでも話そう。

interpreter_v2-0.1.0/benchmark/data/008_screen8/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Miguel: Alright. Let's talk about the pendant at dinner.
+Miguel: Yeah. We can talk about the pendant at dinner.
+Miguel: Okay. Let's discuss the pendant over dinner.

interpreter_v2-0.1.0/benchmark/data/009_screen9/ground_truth.txt

@@ -0,0 +1 @@
+トリスタン師匠、ゆっくりしてって下さい。

interpreter_v2-0.1.0/benchmark/data/009_screen9/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Master Tristan, please take your time.
+Master Tristan, please relax.
+Please take it easy, Master Tristan.

interpreter_v2-0.1.0/benchmark/data/010_screen10/ground_truth.txt

@@ -0,0 +1 @@
+クレス「悪い悪い。今、師匠が家に来ててね。じゃあ行こうか、チェスター。

interpreter_v2-0.1.0/benchmark/data/010_screen10/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Cress: Sorry, sorry. My master's at home right now. So let's go, Chester.
+Cress: Sorry, sorry. The master came to the house. Let's go then, Chester.
+Cress: My bad, my bad. My master is visiting right now. Alright, let's go Chester.

interpreter_v2-0.1.0/benchmark/data/011_screen11/ground_truth.txt

@@ -0,0 +1 @@
+母マリア「お待ちなさい、クレス。

interpreter_v2-0.1.0/benchmark/data/011_screen11/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Mother Maria: Wait, Cress.
+Maria: Wait, Cress.
+Mother Maria: Hold on, Cress.

interpreter_v2-0.1.0/benchmark/data/012_screen12/ground_truth.txt

@@ -0,0 +1 @@
+クレス「母さん!まだ病み上がりなんだから外に出ちゃダメだよ。

interpreter_v2-0.1.0/benchmark/data/012_screen12/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Cress: Mom! You're still recovering, you shouldn't go outside.
+Cress: Mom! You're still sick, so you can't go outside.
+Cress: Mother! You just recovered, you shouldn't be going outside.

interpreter_v2-0.1.0/benchmark/data/013_screen13/ground_truth.txt

@@ -0,0 +1 @@
+母マリア「でも、お前が心配でね。最近この辺りは物騒だから………。

interpreter_v2-0.1.0/benchmark/data/013_screen13/ground_truth_translation.txt

@@ -0,0 +1,3 @@
+Mother Maria: But I'm worried about you. It's been dangerous around here lately...
+Mother Maria: But I worry about you. Things have been rough around here lately...
+Maria: But I'm worried about you. It's been dangerous in this area recently...