paradox2 0.1.0__tar.gz

paradox2-0.1.0/.gitignore

@@ -0,0 +1,20 @@
+.venv/
+.venv-*/
+.kilo/
+__pycache__/
+*.pyc
+.pytest_cache/
+uv.lock
+.codebase-memory/
+archive_1/
+archive (1).zip
+eval/results/
+
+# --- all local table/layout eval artifacts (single folder) ---
+experiments/
+archive*.zip
+docs/golden_set_realtest/
+docs/
+bench/
+.claude/
+comunicationa2a/

paradox2-0.1.0/LICENSE

@@ -0,0 +1,34 @@
+paradox2 — Proprietary Software License
+Copyright (c) 2024-2026 CreAI. All rights reserved.
+
+This software, including its source code, models, and documentation
+(collectively, the "Software"), is the confidential and proprietary property of
+CreAI ("the Company"). The Software is licensed, not sold, and is NOT open
+source. The publicly distributed package (e.g. via PyPI) is made available for
+use subject to the terms below.
+
+1. LICENSE GRANT. Subject to these terms, the Company grants you a
+   non-exclusive, non-transferable, revocable license to install and use the
+   distributed paradox2 package for your own internal purposes.
+
+2. RESTRICTIONS. Except to the extent expressly permitted by applicable law, you
+   may not: (a) redistribute, sublicense, sell, rent, or lease the Software;
+   (b) reverse engineer, decompile, or disassemble it, or attempt to derive its
+   source code; (c) modify or create derivative works of it; or (d) remove or
+   alter any proprietary notices.
+
+3. BUNDLED THIRD-PARTY COMPONENTS. The Software includes or downloads
+   third-party components that remain the property of their respective owners
+   and are governed by their own licenses. See the NOTICE file for details;
+   those terms control for the respective components.
+
+4. NO WARRANTY. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+   EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+   MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. TO
+   THE MAXIMUM EXTENT PERMITTED BY LAW, THE COMPANY SHALL NOT BE LIABLE FOR ANY
+   CLAIM OR DAMAGES ARISING FROM OR IN CONNECTION WITH THE SOFTWARE.
+
+5. COMMERCIAL LICENSING. For commercial use, redistribution, OEM, or any rights
+   not expressly granted above, contact feliperodriguez@creai.mx.
+
+All rights not expressly granted herein are reserved by CreAI.

paradox2-0.1.0/PKG-INFO

@@ -0,0 +1,227 @@
+Metadata-Version: 2.5
+Name: paradox2
+Version: 0.1.0
+Summary: Fast, simple document extraction — from-scratch rewrite of paradox_pdf
+Author-email: CreAI <feliperodriguez@creai.mx>
+License: Proprietary
+License-File: LICENSE
+Keywords: document,extraction,nlp,ocr,parsing,pdf,structured,table
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: Markup
+Requires-Python: >=3.11
+Requires-Dist: numpy<3.0,>=1.24
+Requires-Dist: opencv-python-headless<5.0,>=4.8
+Requires-Dist: pymupdf<2.0,>=1.24
+Provides-Extra: azure
+Requires-Dist: requests<3.0,>=2.31; extra == 'azure'
+Provides-Extra: formats
+Requires-Dist: extract-msg<1.0,>=0.45; extra == 'formats'
+Requires-Dist: odfpy<2.0,>=1.4; extra == 'formats'
+Requires-Dist: openpyxl<4.0,>=3.1; extra == 'formats'
+Requires-Dist: py7zr<1.0,>=0.20; extra == 'formats'
+Requires-Dist: python-docx<2.0,>=1.0; extra == 'formats'
+Requires-Dist: python-pptx<1.0,>=0.6.21; extra == 'formats'
+Requires-Dist: pyxlsb<2.0,>=1.0; extra == 'formats'
+Requires-Dist: rarfile<5.0,>=4.0; extra == 'formats'
+Requires-Dist: striprtf<1.1,>=0.0.26; extra == 'formats'
+Requires-Dist: xlrd<3.0,>=2.0; extra == 'formats'
+Provides-Extra: gpu
+Requires-Dist: paddleocr[doc-parser]>=3.0; extra == 'gpu'
+Requires-Dist: paddlex<4.0,>=3.4; extra == 'gpu'
+Requires-Dist: pillow<12.0,>=10.0; extra == 'gpu'
+Provides-Extra: rapidai
+Requires-Dist: lineless-table-rec<1.0,>=0.1; extra == 'rapidai'
+Requires-Dist: rapidocr-onnxruntime<2.0,>=1.4; extra == 'rapidai'
+Requires-Dist: table-cls<2.0,>=1.2; extra == 'rapidai'
+Requires-Dist: wired-table-rec<2.0,>=1.2; extra == 'rapidai'
+Provides-Extra: specialists
+Requires-Dist: langdetect<2.0,>=1.0; extra == 'specialists'
+Requires-Dist: pillow<12.0,>=10.0; extra == 'specialists'
+Requires-Dist: pix2tex>=0.0.27; extra == 'specialists'
+Requires-Dist: sentencepiece<1.0,>=0.1.99; extra == 'specialists'
+Requires-Dist: torch<3.0,>=2.0; extra == 'specialists'
+Requires-Dist: torchvision<1.0,>=0.15; extra == 'specialists'
+Requires-Dist: transformers<5.0,>=4.30; extra == 'specialists'
+Provides-Extra: yolo
+Requires-Dist: huggingface-hub<1.0,>=0.20; extra == 'yolo'
+Requires-Dist: ultralytics<9.0,>=8.0; extra == 'yolo'
+Description-Content-Type: text/markdown
+
+# paradox2
+
+Fast, simple document extraction — a from-scratch rewrite of [paradox-pdf](https://pypi.org/project/paradox-pdf/), built around one idea: **most PDFs are digital and don't need a GPU.**
+
+```python
+import paradox2 as pdx
+
+pages = pdx.extract("invoice.pdf")
+```
+
+## Why this rewrite exists
+
+paradox-pdf grew into a 2,783-line facade doing routing, backend resolution,
+and overlay rendering all in one file. paradox2 starts over with a hard rule:
+**digital pages never pay for OCR or vision models.** Every PDF page is
+classified independently — a digital page (has a text layer) goes through a
+zero-ML fast path (PyMuPDF text + vector-line table detection); a scanned
+page routes to the GPU OCR pipeline. A single mixed document is handled
+correctly per-page, automatically.
+
+## Install
+
+```bash
+pip install paradox2
+```
+
+Everything above works with just that — digital PDFs, text, vector-line
+tables, key-value fields. Heavier features are opt-in extras so the base
+install stays small:
+
+| Extra | Adds | When you need it |
+|---|---|---|
+| `paradox2[gpu]` | PaddleOCR + PP-DocLayoutV2 | Scanned/photographed pages |
+| `paradox2[rapidai]` | TableStructureRec (RapidAI), ONNX-only | Mid-tier table structure for scanned pages the OCR-grid heuristic rejects — no torch/torchvision, CPU-only, ~5.9s/table |
+| `paradox2[yolo]` | YOLO26-document-layout | Independent table-bbox detector — crops the page before RapidAI/VLM instead of feeding them the whole page. Needed for the router's full accuracy; see below |
+| `paradox2[formats]` | docx/xlsx/pptx/msg/rtf/7z/rar/odf readers | Non-PDF documents |
+| `paradox2[specialists]` | torch + transformers (handwriting, signatures, formulas) | Handwriting (TrOCR), signature detection (Conditional-DETR), formula-to-LaTeX (pix2tex) |
+
+```bash
+pip install "paradox2[gpu,rapidai,formats]"
+```
+
+## What it does
+
+```python
+import paradox2 as pdx
+
+# Every page, as structured JSON — digital pages via the fast path,
+# scanned pages via GPU OCR, decided per page.
+pages = pdx.extract("document.pdf")
+
+# Just the text
+text = pdx.extract_text("document.pdf")
+
+# Just the tables (vector-line detection on digital pages,
+# OCR-grid heuristic + optional RapidAI/VLM fallback on scanned pages)
+tables = pdx.extract_tables("document.pdf")
+
+# Key-value pairs from an invoice/form-like page (digital only)
+fields = pdx.extract_kie("invoice.pdf")
+
+# Any non-PDF format too — same call, same output shape
+data = pdx.extract("spreadsheet.xlsx")
+data = pdx.extract("scan.docx")
+```
+
+### Selecting pages, features, and output format
+
+```python
+pdx.extract("report.pdf", pages="1-3,7")           # specific pages
+pdx.extract("report.pdf", feature="tables")         # just one feature, still full extract() under the hood
+pdx.extract("report.pdf", output_format="markdown") # rendered markdown instead of the raw IR
+pdx.extract("report.pdf", fields=True)               # turn on key-value extraction inline
+```
+
+### Optional specialists (all off by default, all lazy-loaded)
+
+```python
+pdx.extract("form.pdf", handwriting=True)   # TrOCR on handwritten blocks
+pdx.extract("contract.pdf", signatures=True) # Conditional-DETR signature boxes
+pdx.extract("paper.pdf", formulas=True)      # inline math -> LaTeX
+```
+
+None of these import their heavy dependencies unless the flag is set —
+`import paradox2` alone never touches torch, paddle, or transformers.
+
+## The scanned-table router
+
+Scanned pages route tables through three tiers, each opt-in past the first:
+
+```
+OCR-grid heuristic (free, always on)
+        |  rejects merged/borderless/dense tables by design,
+        |  or under-reads a real table (low_confidence flag)
+        v
+YOLO26 bbox (PARADOX2_YOLO_DETECT=1) -- crop the page to the detected
+        |  table region before handing it to the tiers below. A
+        |  borderline-confidence detection (<0.85) skips RapidAI
+        |  entirely and goes straight to the VLM tier - VLM tolerates
+        |  a loose/imprecise crop better than RapidAI's structure model
+        v
+RapidAI mid-tier (PARADOX2_RAPIDAI_TABLES=1)
+        |  ONNX wired/wireless classifier + structure model. Its own
+        |  result is cross-checked against OCR line density in the same
+        |  crop (table_rapidai.py's low_confidence) - implausibly few
+        |  OR implausibly many rows both re-route to the VLM tier
+        v
+VLM fallback (PARADOX2_VLM_TABLES=1)
+        |  PaddleOCR-VL single-pass, reserved for tables that look
+        |  genuinely complex - not run on everything RapidAI touches
+        v
+   best available result
+```
+
+Every tier is measured end-to-end (`paradox2.extract()`, not the isolated
+engine wrapper) on the same 20 real full-page tables (OmniDocBench), not
+simulated:
+
+| config | exact row match | within +/-3 rows | mean time/page |
+|---|---|---|---|
+| heuristic only (pre-router) | 5% | 15% | 0.8s |
+| + RapidAI, full page (bug, fixed) | 15% | 50% | 6.0s |
+| + RapidAI, cropped to heuristic's own bbox (bug, fixed - made 3/20 pages *worse*) | 15% | 40% | 4.3s |
+| + RapidAI, cropped to YOLO26 bbox | 25% | 75% | 5.5s |
+| + selective VLM escalation (low YOLO confidence or implausible RapidAI row count) | **35%** | **80%** | 6.4s |
+
+The isolated RapidAI engine alone scores 65% exact on tight ground-truth
+crops — the gap to the router's end-to-end number is the YOLO26 crop's
+margin versus a perfect crop, not a RapidAI accuracy problem. Cropping to
+the heuristic's *own* bbox instead of an independent detector actively
+hurts: that bbox only spans the rows the heuristic already found, so it
+bakes its own under-read into the pixels before the next tier ever sees
+them. A same-crop self-consistency check (RapidAI's row count vs. OCR
+line density) can't catch a loose crop itself, since both numbers inflate
+together on the same imprecise crop — that's what the YOLO-confidence
+trigger (<0.85) is for instead.
+
+The heuristic never gets replaced by a worse result — a lower tier's
+output is kept unless a higher tier actually produces something.
+
+```bash
+export PARADOX2_YOLO_DETECT=1
+export PARADOX2_RAPIDAI_TABLES=1
+export PARADOX2_VLM_TABLES=1
+```
+
+## Health check
+
+```python
+from paradox2 import doctor
+doctor()  # {"cuda": True/False, "pymupdf": "1.24.x", ...} - never silently hides a backend mismatch
+```
+
+## Design principles
+
+- **Digital-first**: a PDF with a text layer never pays for OCR, vision
+  layout models, or GPU inference — verified per-page, not per-document.
+- **Facade stays thin**: `paradox2/api.py` is dispatch only; business logic
+  lives in `pipeline/`, `tables/`, `engines/`, `formats/`. If the facade
+  creeps toward 100+ lines, that's a signal something leaked out of place.
+- **Specialists are lazy and swallow their own failures**: an optional
+  engine (handwriting, signatures, formulas, RapidAI, VLM) that fails to
+  load or errors mid-call returns `[]`/`None` — it never takes down the
+  base extraction path.
+- **No simulated benchmarks**: every accuracy/speed number in this README
+  and in the codebase's docstrings comes from a real run against real
+  documents, with the script path noted alongside it.
+
+## Status
+
+Early-stage rewrite, private while the API and table router stabilize. See
+`docs/PLAN.md` for the phase breakdown and `docs/TASKS.md` for current work.

paradox2-0.1.0/README.md

@@ -0,0 +1,172 @@
+# paradox2
+
+Fast, simple document extraction — a from-scratch rewrite of [paradox-pdf](https://pypi.org/project/paradox-pdf/), built around one idea: **most PDFs are digital and don't need a GPU.**
+
+```python
+import paradox2 as pdx
+
+pages = pdx.extract("invoice.pdf")
+```
+
+## Why this rewrite exists
+
+paradox-pdf grew into a 2,783-line facade doing routing, backend resolution,
+and overlay rendering all in one file. paradox2 starts over with a hard rule:
+**digital pages never pay for OCR or vision models.** Every PDF page is
+classified independently — a digital page (has a text layer) goes through a
+zero-ML fast path (PyMuPDF text + vector-line table detection); a scanned
+page routes to the GPU OCR pipeline. A single mixed document is handled
+correctly per-page, automatically.
+
+## Install
+
+```bash
+pip install paradox2
+```
+
+Everything above works with just that — digital PDFs, text, vector-line
+tables, key-value fields. Heavier features are opt-in extras so the base
+install stays small:
+
+| Extra | Adds | When you need it |
+|---|---|---|
+| `paradox2[gpu]` | PaddleOCR + PP-DocLayoutV2 | Scanned/photographed pages |
+| `paradox2[rapidai]` | TableStructureRec (RapidAI), ONNX-only | Mid-tier table structure for scanned pages the OCR-grid heuristic rejects — no torch/torchvision, CPU-only, ~5.9s/table |
+| `paradox2[yolo]` | YOLO26-document-layout | Independent table-bbox detector — crops the page before RapidAI/VLM instead of feeding them the whole page. Needed for the router's full accuracy; see below |
+| `paradox2[formats]` | docx/xlsx/pptx/msg/rtf/7z/rar/odf readers | Non-PDF documents |
+| `paradox2[specialists]` | torch + transformers (handwriting, signatures, formulas) | Handwriting (TrOCR), signature detection (Conditional-DETR), formula-to-LaTeX (pix2tex) |
+
+```bash
+pip install "paradox2[gpu,rapidai,formats]"
+```
+
+## What it does
+
+```python
+import paradox2 as pdx
+
+# Every page, as structured JSON — digital pages via the fast path,
+# scanned pages via GPU OCR, decided per page.
+pages = pdx.extract("document.pdf")
+
+# Just the text
+text = pdx.extract_text("document.pdf")
+
+# Just the tables (vector-line detection on digital pages,
+# OCR-grid heuristic + optional RapidAI/VLM fallback on scanned pages)
+tables = pdx.extract_tables("document.pdf")
+
+# Key-value pairs from an invoice/form-like page (digital only)
+fields = pdx.extract_kie("invoice.pdf")
+
+# Any non-PDF format too — same call, same output shape
+data = pdx.extract("spreadsheet.xlsx")
+data = pdx.extract("scan.docx")
+```
+
+### Selecting pages, features, and output format
+
+```python
+pdx.extract("report.pdf", pages="1-3,7")           # specific pages
+pdx.extract("report.pdf", feature="tables")         # just one feature, still full extract() under the hood
+pdx.extract("report.pdf", output_format="markdown") # rendered markdown instead of the raw IR
+pdx.extract("report.pdf", fields=True)               # turn on key-value extraction inline
+```
+
+### Optional specialists (all off by default, all lazy-loaded)
+
+```python
+pdx.extract("form.pdf", handwriting=True)   # TrOCR on handwritten blocks
+pdx.extract("contract.pdf", signatures=True) # Conditional-DETR signature boxes
+pdx.extract("paper.pdf", formulas=True)      # inline math -> LaTeX
+```
+
+None of these import their heavy dependencies unless the flag is set —
+`import paradox2` alone never touches torch, paddle, or transformers.
+
+## The scanned-table router
+
+Scanned pages route tables through three tiers, each opt-in past the first:
+
+```
+OCR-grid heuristic (free, always on)
+        |  rejects merged/borderless/dense tables by design,
+        |  or under-reads a real table (low_confidence flag)
+        v
+YOLO26 bbox (PARADOX2_YOLO_DETECT=1) -- crop the page to the detected
+        |  table region before handing it to the tiers below. A
+        |  borderline-confidence detection (<0.85) skips RapidAI
+        |  entirely and goes straight to the VLM tier - VLM tolerates
+        |  a loose/imprecise crop better than RapidAI's structure model
+        v
+RapidAI mid-tier (PARADOX2_RAPIDAI_TABLES=1)
+        |  ONNX wired/wireless classifier + structure model. Its own
+        |  result is cross-checked against OCR line density in the same
+        |  crop (table_rapidai.py's low_confidence) - implausibly few
+        |  OR implausibly many rows both re-route to the VLM tier
+        v
+VLM fallback (PARADOX2_VLM_TABLES=1)
+        |  PaddleOCR-VL single-pass, reserved for tables that look
+        |  genuinely complex - not run on everything RapidAI touches
+        v
+   best available result
+```
+
+Every tier is measured end-to-end (`paradox2.extract()`, not the isolated
+engine wrapper) on the same 20 real full-page tables (OmniDocBench), not
+simulated:
+
+| config | exact row match | within +/-3 rows | mean time/page |
+|---|---|---|---|
+| heuristic only (pre-router) | 5% | 15% | 0.8s |
+| + RapidAI, full page (bug, fixed) | 15% | 50% | 6.0s |
+| + RapidAI, cropped to heuristic's own bbox (bug, fixed - made 3/20 pages *worse*) | 15% | 40% | 4.3s |
+| + RapidAI, cropped to YOLO26 bbox | 25% | 75% | 5.5s |
+| + selective VLM escalation (low YOLO confidence or implausible RapidAI row count) | **35%** | **80%** | 6.4s |
+
+The isolated RapidAI engine alone scores 65% exact on tight ground-truth
+crops — the gap to the router's end-to-end number is the YOLO26 crop's
+margin versus a perfect crop, not a RapidAI accuracy problem. Cropping to
+the heuristic's *own* bbox instead of an independent detector actively
+hurts: that bbox only spans the rows the heuristic already found, so it
+bakes its own under-read into the pixels before the next tier ever sees
+them. A same-crop self-consistency check (RapidAI's row count vs. OCR
+line density) can't catch a loose crop itself, since both numbers inflate
+together on the same imprecise crop — that's what the YOLO-confidence
+trigger (<0.85) is for instead.
+
+The heuristic never gets replaced by a worse result — a lower tier's
+output is kept unless a higher tier actually produces something.
+
+```bash
+export PARADOX2_YOLO_DETECT=1
+export PARADOX2_RAPIDAI_TABLES=1
+export PARADOX2_VLM_TABLES=1
+```
+
+## Health check
+
+```python
+from paradox2 import doctor
+doctor()  # {"cuda": True/False, "pymupdf": "1.24.x", ...} - never silently hides a backend mismatch
+```
+
+## Design principles
+
+- **Digital-first**: a PDF with a text layer never pays for OCR, vision
+  layout models, or GPU inference — verified per-page, not per-document.
+- **Facade stays thin**: `paradox2/api.py` is dispatch only; business logic
+  lives in `pipeline/`, `tables/`, `engines/`, `formats/`. If the facade
+  creeps toward 100+ lines, that's a signal something leaked out of place.
+- **Specialists are lazy and swallow their own failures**: an optional
+  engine (handwriting, signatures, formulas, RapidAI, VLM) that fails to
+  load or errors mid-call returns `[]`/`None` — it never takes down the
+  base extraction path.
+- **No simulated benchmarks**: every accuracy/speed number in this README
+  and in the codebase's docstrings comes from a real run against real
+  documents, with the script path noted alongside it.
+
+## Status
+
+Early-stage rewrite, private while the API and table router stabilize. See
+`docs/PLAN.md` for the phase breakdown and `docs/TASKS.md` for current work.

paradox2-0.1.0/paradox2/__init__.py

@@ -0,0 +1,7 @@
+"""Public package surface for the from-scratch extractor."""
+
+from paradox2.api import extract, extract_kie, extract_tables, extract_text, read
+from paradox2.doctor import doctor
+from paradox2.runtime.machine_state import warmup
+
+__all__ = ["extract", "extract_kie", "extract_tables", "extract_text", "read", "doctor", "warmup"]

paradox2-0.1.0/paradox2/__main__.py

@@ -0,0 +1,5 @@
+"""Allow ``python -m paradox2`` as well as the installed console script."""
+
+from paradox2.cli import main
+
+raise SystemExit(main())