pi-ocr 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Dlrow
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,394 @@
+# pi-ocr
+
+> ### ⚡ Zero setup. Works out of the box.
+>
+> Default backend: **MinerU** — a free cloud API. No install, no GPU, no API key.
+> Just `pi install npm:pi-ocr` and OCR anything.
+
+Multi-backend OCR for [Pi Coding Agent](https://pi.dev) — extract text, LaTeX math formulas, and tables from images and PDFs. Choose the backend that fits your needs: free cloud API, local GPU, or pure Python.
+
+> Bridges the multimodal gap for non-vision LLMs like **DeepSeek**. When your model can't see images, `minimodel_ocr` acts as its eyes.
+
+## Three Backends — One Tool
+
+| Backend | Type | Best For |
+|---|---|---|
+| 🦙 **Ollama** | Local GPU | Math formulas (LaTeX), privacy, offline |
+| ☁️ **MinerU** | Free cloud API | Complex PDFs, no GPU, zero setup |
+| 📐 **Pix2Text** | Local Python | Math formulas + text, free Mathpix alternative |
+
+Switch anytime with `/ocr` (no args) — a visual `SettingsList` menu lets you pick and configure everything without editing JSON:
+
+```
+/ocr     → opens settings: backend, model, PDF split toggle
+/ocr <file> [task]   → OCR a file
+```
+
+## Features
+
+| | |
+|---|---|
+| 🔤 **Text** | General text recognition → Markdown |
+| 🧮 **Formulas** | Math formulas → LaTeX (Ollama glm-ocr: 94.6 OmniDocBench) |
+| 📊 **Tables** | Table structure → Markdown tables |
+| 🖼️ **Figures** | Diagrams and illustrations → descriptions |
+| 📄 **PDF** | Full PDF support across all backends |
+| 🎛️ **Any model** | Ollama works with glm-ocr, llama3.2-vision, minicpm-v, etc. |
+| ☁️ **Free cloud** | MinerU Agent API: no token, ≤10MB, ≤20 pages free |
+| 📦 **Auto-split** | MinerU splits PDFs >20 pages into free-tier chunks |
+
+---
+
+## Quickstart
+
+### One command
+
+```bash
+pi install npm:pi-ocr
+```
+
+**That's it.** The default backend is ☁️ **MinerU** — a free cloud API with zero setup.
+Start OCR'ing immediately:
+
+```
+/ocr ./scan.png
+/ocr ./document.pdf
+```
+
+> 💡 Want offline OCR or math formulas? Switch backends anytime with `/ocr` (no args).
+
+---
+
+### Optional: set up other backends
+
+Only needed if you want to switch from the default MinerU.
+
+---
+
+### 🦙 Ollama setup
+
+#### macOS
+
+```bash
+# 1. Install Ollama
+brew install ollama
+
+# 2. Pull the default OCR model (~2.2 GB)
+ollama pull glm-ocr
+
+# 3. Multi-page PDF support (optional but recommended)
+brew install poppler
+```
+
+> macOS uses built-in `sips` for single-page PDFs — zero extra deps for those.  
+> Multi-page PDFs need `poppler` for the `pdftoppm` tool.
+
+#### Linux
+
+```bash
+# 1. Install Ollama
+curl -fsSL https://ollama.com/install.sh | sh
+
+# 2. Pull the default OCR model (~2.2 GB)
+ollama pull glm-ocr
+
+# 3. PDF support (required on Linux)
+sudo apt install poppler-utils        # Debian/Ubuntu
+sudo dnf install poppler-utils        # Fedora
+sudo pacman -S poppler                # Arch
+```
+
+#### Verify
+
+```bash
+# Check Ollama is running and model is pulled
+ollama list | grep glm-ocr
+```
+
+---
+
+### 📐 Pix2Text setup
+
+Pix2Text runs entirely in Python. Handles images and PDFs via a single API call — no manual conversion needed.
+
+#### Step 1: Make sure you have Python 3.9+ (macOS/Linux)
+
+| System | Check |
+|---|---|
+| macOS/Linux | `python3 --version` |
+
+> ⚠️ **Important:** Know which Python you're using. Run `which python3` — if it shows `conda`, `brew`, or `/usr/bin/python3`, your `pip install` must target the same Python:
+> ```bash
+> # Conda Python
+> pip install pix2text
+> 
+> # System Python (may need --user or sudo)
+> pip install --user pix2text
+> 
+> # Brew Python (macOS)
+> /opt/homebrew/bin/pip3 install pix2text
+> ```
+> If unsure, use `python3 -m pip install ...` — this always installs for the active `python3`.
+
+#### Step 2: Install packages
+
+```bash
+python3 -m pip install pix2text
+```
+
+#### Step 3: Verify
+
+```bash
+python3 -c "from pix2text import Pix2Text; print('OK')"
+```
+
+> First run downloads ONNX models (~50MB) to `~/.pix2text/`.
+
+> First run downloads ONNX models (~50MB) to `~/.pix2text/`.
+
+
+---
+
+### ☁️ MinerU (default — already working!)
+
+**No setup required.** The free Agent API works immediately. No token, no account, no install.
+
+Free tier limits:
+- ≤ 10 MB per file
+- ≤ 20 pages per request
+- IP-based rate limiting
+
+For files >10MB, compress first at [ilovepdf.com/compress_pdf](https://ilovepdf.com/compress_pdf).
+
+---
+
+## Usage
+
+### Settings UI
+
+```
+/ocr
+```
+
+Opens an interactive `SettingsList` with keyboard navigation:
+
+```
+┌─ OCR Settings ─────────────────────────────────┐
+│  OCR Backend         [ollama / mineru / pix2text]  │
+│  MinerU: Split PDF   [ON / OFF]                     │
+│  Ollama Model         [glm-ocr]                     │
+│  ↑↓ navigate • ← → toggle • enter select • esc close    │
+└──────────────────────────────────────────────────┘
+```
+
+- **Backend**: ← → to cycle ollama/mineru/pix2text — saves immediately
+- **MinerU Split**: ← → to toggle ON/OFF — when ON, PDFs >20 pages are auto-split
+- **Model**: Enter opens a sub-menu with recommended models + custom input
+
+### OCR a file
+
+```
+/ocr <file> [task] [model]
+```
+
+| Example | Result |
+|---|---|
+| `/ocr ./scan.png` | Auto-detect all content |
+| `/ocr ./equation.jpg formula` | LaTeX formula output |
+| `/ocr ./contract.pdf text` | Text-only extraction |
+| `/ocr ./paper.pdf auto glm-ocr:q8_0` | Use specific model |
+
+### Tasks
+
+| Task | Description | Output format |
+|---|---|---|
+| `auto` | Full document OCR (default) | Markdown + LaTeX mixed |
+| `text` | Plain text recognition | Markdown |
+| `formula` | Math formula recognition | LaTeX |
+| `table` | Table structure recognition | Markdown tables |
+| `figure` | Figure / diagram description | Natural language |
+
+### LLM-invoked (automatic)
+
+The extension registers a `minimodel_ocr` tool. The agent invokes it automatically:
+
+```
+> What formula is written in this screenshot?
+> OCR this 50-page PDF into markdown.
+```
+
+---
+
+## MinerU PDF Splitting
+
+When `MinerU: Split PDF >20 pages` is ON (default), large PDFs are automatically split into ≤20-page chunks. Each chunk is a separate request with 3s spacing:
+
+```
+Splitting 85-page PDF into ≤20-page chunks…
+[1/5] uploading…
+[1/5] running (12s)
+[1/5] done
+[2/5] waiting rate limit…
+[2/5] uploading…
+[2/5] running (18s)
+[2/5] done
+...
+```
+
+---
+
+## Backend Comparison
+
+| | 🦙 Ollama | ☁️ MinerU | 📐 **Pix2Text** |
+|---|---|---|---|
+| **Setup** | Install Ollama + pull model | None | `pip install` 3 packages |
+| **GPU needed** | Recommended | No | No |
+| **Internet** | No | Yes | No (first run: yes) |
+| **Math formulas** | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐ |
+| **Complex PDFs** | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐ |
+| **Chinese text** | ⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ |
+| **File size limit** | None | 10MB (free) | None |
+| **Page limit** | None | 20/request (free) | None |
+| **Cost** | Free (local) | Free (rate-limited) | Free (local) |
+
+---
+
+## Supported File Types
+
+| Format | Ollama | MinerU | Pix2Text |
+|---|---|---|---|
+| PNG, JPG, GIF, WEBP, BMP, TIFF | ✅ | ✅ | ✅ |
+| PDF | ✅ | ✅ | ✅ |
+| Docx, PPTx, Xlsx | ❌ | ✅ | ❌ |
+
+---
+
+## PDF Support Details
+
+| Backend | Conversion method | System deps |
+|---|---|---|
+| **Ollama** | `sips` (macOS page 1) / `pdftoppm` (multi-page, Linux) | `poppler` (multi-page only) |
+| **MinerU** | Direct PDF upload — no conversion | None |
+| **Pix2Text** | Built-in `recognize_pdf()` — no system deps | None |
+
+---
+
+## Configuration
+
+All settings are persisted to `~/.pi/agent/settings.json`:
+
+```json
+{
+  "minimodelOcr": {
+    "backend": "ollama",
+    "model": "glm-ocr",
+    "ollamaHost": "http://localhost:11434",
+    "mineruSplitPdf": true
+  }
+}
+```
+
+Change settings via `/ocr` (interactive) or edit directly. Environment variables override file settings:
+
+```bash
+export OLLAMA_HOST="http://localhost:11434"
+export OCR_MODEL="glm-ocr"
+```
+
+---
+
+## Troubleshooting
+
+### Ollama: "fetch failed" / "ECONNREFUSED"
+
+```bash
+# Start Ollama in the background
+ollama serve
+```
+
+### Ollama: "model not found"
+
+```bash
+ollama pull glm-ocr
+```
+
+### Pix2Text: "python3 not found"
+
+```bash
+# Check your python:
+which python3 && python3 --version
+
+# If using conda:
+conda activate base && pip install pix2text
+
+# If using system python:
+python3 -m pip install --user pix2text
+```
+
+### Pix2Text: "No module named 'pix2text'"
+
+You likely installed with a different `pip` than your active `python3`:
+
+```bash
+python3 -m pip install pix2text
+```
+```
+
+### MinerU: "429 Too Many Requests"
+
+IP rate limit hit. Wait 1-2 minutes, or switch to Ollama/Pix2Text with `/ocr`.
+
+### MinerU: "file page count exceeds lightweight API limit"
+
+Enable PDF splitting: `/ocr` → toggle "MinerU: Split PDF" to ON.
+
+### MinerU: "File too large for free MinerU API"
+
+Compress the PDF at [ilovepdf.com/compress_pdf](https://ilovepdf.com/compress_pdf) or switch to a local backend with `/ocr`.
+
+### macOS multi-page PDF: "pdftoppm not found"
+
+```bash
+brew install poppler
+```
+
+### Linux multi-page PDF: "pdftoppm not found"
+
+```bash
+# Debian/Ubuntu
+sudo apt install poppler-utils
+
+# Fedora
+sudo dnf install poppler-utils
+
+# Arch
+sudo pacman -S poppler
+```
+
+---
+
+## How It Works
+
+```
+┌──────────────────┐     ┌──────────────────┐     ┌──────────────────────┐
+│  pi (DeepSeek)   │────▶│  minimodel_ocr   │────▶│  Ollama / MinerU    │
+│  (no vision)     │     │  pi extension    │     │  / Pix2Text        │
+└──────────────────┘     └──────────────────┘     └──────────────────────┘
+        │                         │                           │
+        │  "read this image"      │  POST /api/generate       │
+        │────────────────────────▶│  (Ollama)                 │
+        │                         │  or POST /api/v1/agent    │
+        │                         │  (MinerU)                 │
+        │                         │  or python3 subprocess    │
+        │                         │  (Pix2Text)              │
+        │                         │──────────────────────────▶│
+        │                         │  OCR text response        │
+        │  LaTeX / Markdown       │◀──────────────────────────│
+        │◀────────────────────────│                           │
+```
+
+The tool dispatches to your selected backend. Switch anytime with `/ocr` — no restart needed.
+
+## License
+
+MIT