saraldocling 1.0.0__tar.gz

saraldocling-1.0.0/PKG-INFO

@@ -0,0 +1,394 @@
+Metadata-Version: 2.4
+Name: saraldocling
+Version: 1.0.0
+Summary: Fast PDF text and image/table extraction using PyMuPDF + YOLOv8 DocLayNet ONNX
+Author: SARAL
+License-Expression: AGPL-3.0-only
+Project-URL: Homepage, https://github.com/DemocratiseResearch/SARALDocling
+Project-URL: Issues, https://github.com/DemocratiseResearch/SARALDocling/issues
+Keywords: pdf,parsing,yolo,document-layout,onnx,table-extraction
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: General
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: pymupdf>=1.24.0
+Requires-Dist: numpy<3.0,>=1.24.0
+Requires-Dist: Pillow>=10.0.0
+Provides-Extra: cpu
+Requires-Dist: onnxruntime>=1.17.0; extra == "cpu"
+Provides-Extra: gpu
+Requires-Dist: onnxruntime-gpu>=1.17.0; extra == "gpu"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Requires-Dist: mypy; extra == "dev"
+
+# SARAL Docling
+
+Fast PDF text and image/table extraction.  
+Powered by **PyMuPDF** + **YOLOv8 DocLayNet** (bundled) — no model file needed.
+
+```bash
+pip install "saraldocling[cpu]"
+saraldocling paper.pdf
+```
+
+> **License:** This package bundles `yolov8n-doclaynet.onnx` which is released
+> under **AGPL-3.0**. The entire `saraldocling` package is therefore also
+> distributed under **AGPL-3.0**.  
+> Full text: https://www.gnu.org/licenses/agpl-3.0.html
+
+> **No OCR.** Only PDFs with a native text layer are supported.  
+> For scanned PDFs, pre-process with `ocrmypdf` first (see [Limitations](#limitations)).
+
+---
+
+## Table of contents
+
+- [How it works](#how-it-works)
+- [Installation](#installation)
+- [CLI usage](#cli-usage)
+- [Python API](#python-api)
+- [Output structure](#output-structure)
+- [FastAPI / server deployment](#fastapi--server-deployment)
+- [Configuration reference](#configuration-reference)
+- [Limitations](#limitations)
+
+---
+
+## How it works
+
+| Stage                 | What happens                                                                                                                   |
+| --------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
+| **1 — Text**          | PyMuPDF extracts the native text layer from every page                                                                         |
+| **2 — Images/Tables** | YOLOv8 DocLayNet (bundled ONNX) detects Picture and Table regions, re-renders those regions at 300 DPI, and saves them as PNGs |
+
+The ONNX model (`yolov8n-doclaynet.onnx`) is included inside the wheel — there
+is no separate download step.
+
+---
+
+## Installation
+
+> Pick **exactly one** of `[cpu]` or `[gpu]`. Installing both puts two competing
+> `onnxruntime` builds in your environment.
+
+### CPU — works everywhere, no GPU required
+
+```bash
+pip install "saraldocling[cpu]"
+```
+
+### GPU — NVIDIA CUDA
+
+Requires CUDA 11.8+ and cuDNN on the host.  
+Compatibility: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html
+
+```bash
+pip install "saraldocling[gpu]"
+```
+
+### Apple Silicon — CoreML acceleration (M1/M2/M3/M4)
+
+The `[cpu]` wheel works fine on ARM. For ANE-accelerated inference:
+
+```bash
+pip install "saraldocling[cpu]"
+pip install onnxruntime-silicon      # replaces the cpu wheel
+```
+
+### Linux / Debian server — one-time system packages
+
+```bash
+sudo apt-get install -y libgl1 libglib2.0-0
+pip install "saraldocling[cpu]"
+```
+
+### From source (development)
+
+```bash
+git clone https://github.com/yourname/saraldocling
+cd saraldocling
+pip install -e ".[cpu,dev]"
+```
+
+---
+
+## CLI usage
+
+The bundled model runs automatically. No `--model` flag needed.
+
+```bash
+# Full pipeline: text + image/table crops (default)
+saraldocling paper.pdf
+
+# Text only — skip YOLO, much faster
+saraldocling paper.pdf --no-images
+
+# Custom output directory
+saraldocling paper.pdf -o ./results
+
+# Preview extracted text in the terminal
+saraldocling paper.pdf --preview 800
+
+# One .txt file per page
+saraldocling paper.pdf --pages
+
+# GPU (CUDA) — needs saraldocling[gpu]
+saraldocling paper.pdf --gpu
+
+# Apple Silicon CoreML
+saraldocling paper.pdf --coreml
+
+# Debug: see every step logged
+saraldocling paper.pdf -v
+
+# All flags
+saraldocling --help
+```
+
+### What gets written
+
+```
+paper_out/
+├── extracted_text.txt        ← all pages joined
+├── manifest.json             ← JSON summary of the run
+├── pages/                    ← only with --pages
+│   ├── page_001.txt
+│   └── page_002.txt
+└── extracted_images/         ← Picture + Table crops at 300 DPI
+    ├── p0_Picture_231.png
+    └── p2_Table_445.png
+```
+
+Output directory is auto-named `<pdf_stem>_out/` next to the PDF unless you
+pass `-o`.
+
+---
+
+## Python API
+
+### One-call pipeline (recommended)
+
+```python
+from saraldocling import parse_pdf, ParseConfig
+
+# Full pipeline — bundled model used automatically
+result = parse_pdf(ParseConfig(
+    pdf_path="paper.pdf",
+    output_dir="./out",
+))
+
+print(result.num_pages)       # int
+print(result.text[:500])      # full extracted text
+print(result.image_paths)     # list[str] of saved PNG paths
+```
+
+```python
+# Text only — skip YOLO
+result = parse_pdf(ParseConfig(
+    pdf_path="paper.pdf",
+    output_dir="./out",
+    extract_images=False,
+))
+```
+
+### Lower-level: text extraction only
+
+```python
+from saraldocling import PDFProcessor
+
+with PDFProcessor("paper.pdf") as proc:
+    full_text  = proc.extract_text()
+    page_text  = proc.extract_text_by_page(1)      # 0-indexed
+    page_image = proc.extract_page_image(0, dpi=150)  # PIL.Image
+```
+
+### Lower-level: YOLO extraction only
+
+```python
+from saraldocling import ImageExtractor
+
+# CPU (default)
+with ImageExtractor() as ext:
+    paths = ext.extract_images_from_pdf("paper.pdf", "./out")
+
+# GPU
+with ImageExtractor(providers=["CUDAExecutionProvider", "CPUExecutionProvider"]) as ext:
+    paths = ext.extract_images_from_pdf("paper.pdf", "./out")
+
+# Apple Silicon
+with ImageExtractor(providers=["CoreMLExecutionProvider", "CPUExecutionProvider"]) as ext:
+    paths = ext.extract_images_from_pdf("paper.pdf", "./out")
+```
+
+### Using a custom model (advanced)
+
+```python
+result = parse_pdf(ParseConfig(
+    pdf_path="paper.pdf",
+    output_dir="./out",
+    model_path="/path/to/custom.onnx",   # overrides the bundled model
+))
+```
+
+---
+
+## Output structure
+
+`manifest.json` example:
+
+```json
+{
+  "pdf": "/abs/path/to/paper.pdf",
+  "pages": 12,
+  "text_chars": 38201,
+  "text_file": "/abs/path/to/paper_out/extracted_text.txt",
+  "per_page_files": [],
+  "image_paths": [
+    "/abs/path/to/paper_out/extracted_images/p0_Picture_231.png",
+    "/abs/path/to/paper_out/extracted_images/p3_Table_112.png"
+  ]
+}
+```
+
+Crop filenames follow the pattern `p<page>_<Label>_<x>.png` where `<Label>`
+is either `Picture` or `Table`.
+
+---
+
+## FastAPI / server deployment
+
+`saraldocling` is a plain pip package — drop it into any FastAPI app on Debian
+with no special system configuration beyond the apt packages above.
+
+### Minimal endpoint
+
+```python
+# main.py
+import shutil, uuid
+from pathlib import Path
+
+from fastapi import FastAPI, File, UploadFile
+from fastapi.responses import JSONResponse
+from saraldocling import parse_pdf, ParseConfig
+
+OUTPUT_ROOT = Path("/tmp/saraldocling")
+
+app = FastAPI()
+
+@app.post("/parse")
+async def parse(file: UploadFile = File(...)):
+    job_dir = OUTPUT_ROOT / str(uuid.uuid4())
+    job_dir.mkdir(parents=True, exist_ok=True)
+    pdf_path = job_dir / file.filename
+
+    with open(pdf_path, "wb") as f:
+        shutil.copyfileobj(file.file, f)
+
+    result = parse_pdf(ParseConfig(
+        pdf_path=str(pdf_path),
+        output_dir=str(job_dir),
+    ))
+
+    return JSONResponse({
+        "pages":       result.num_pages,
+        "text":        result.text,
+        "image_paths": result.image_paths,
+    })
+```
+
+```bash
+pip install "saraldocling[cpu]" fastapi uvicorn python-multipart
+uvicorn main:app --host 0.0.0.0 --port 8000
+```
+
+### Pre-load the ONNX session at startup
+
+Creating an `ImageExtractor` loads and optimises the ONNX graph (~300 ms).
+Do this once at startup and reuse it — `session.run()` is thread-safe:
+
+```python
+from contextlib import asynccontextmanager
+from fastapi import FastAPI
+from saraldocling import ImageExtractor
+
+extractor: ImageExtractor | None = None
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    global extractor
+    extractor = ImageExtractor()      # loads bundled model
+    yield
+    extractor.close()
+
+app = FastAPI(lifespan=lifespan)
+
+# then in your endpoint, call extractor.extract_images_from_pdf() directly
+```
+
+---
+
+## Configuration reference (`ParseConfig`)
+
+| Field            | Default                    | Description                                     |
+| ---------------- | -------------------------- | ----------------------------------------------- |
+| `pdf_path`       | required                   | Path to the source PDF                          |
+| `output_dir`     | required                   | Root directory for all output files             |
+| `extract_images` | `True`                     | Run YOLO stage. `False` = text only             |
+| `model_path`     | `None`                     | Custom `.onnx` path. `None` = use bundled model |
+| `conf_threshold` | `0.30`                     | YOLO confidence cutoff                          |
+| `nms_threshold`  | `0.45`                     | NMS IoU cutoff                                  |
+| `min_box_size`   | `30`                       | Minimum crop side-length in pixels              |
+| `num_workers`    | `cpu_count`                | Thread-pool size for page processing            |
+| `onnx_providers` | `["CPUExecutionProvider"]` | ONNX Runtime execution providers                |
+
+---
+
+## DocLayNet labels
+
+The model detects 11 document element types.  
+Only **`Picture`** and **`Table`** regions are saved as crops.
+
+```
+Caption   Footnote   Formula   List-item   Page-footer
+Page-header   Picture   Section-header   Table   Text   Title
+```
+
+---
+
+## Limitations
+
+**Not OCR** — requires a native text layer in the PDF. For scanned documents:
+
+```bash
+pip install ocrmypdf
+ocrmypdf scanned.pdf scanned_ocr.pdf
+saraldocling scanned_ocr.pdf
+```
+
+**Thread safety** — both `PDFProcessor` and `ImageExtractor` use internal
+mutex locks (mirroring the Go `SafeDocument`). Safe for concurrent use.
+
+---
+
+## License
+
+This package is distributed under **AGPL-3.0** because it bundles
+`yolov8n-doclaynet.onnx`, which carries that licence.
+
+What this means in practice:
+
+- Free to use for any purpose, including commercial.
+- If you modify the source and run it as a network service, you must make
+  your modified source available under AGPL-3.0.
+- The full licence text is at https://www.gnu.org/licenses/agpl-3.0.html