parsearena 0.1.3__tar.gz

parsearena-0.1.3/.gitignore

@@ -0,0 +1,14 @@
+__pycache__/
+*.pyc
+.venv/
+dist/
+build/
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+.env
+parsearena-out/
+parsearena-demo/
+runs/
+_site/
+.DS_Store

parsearena-0.1.3/CHANGELOG.md

@@ -0,0 +1,52 @@
+# Changelog
+
+## [0.1.3] - 2026-11-03
+
+### Fixed
+- Parser helper processes no longer outlive a run. Marker 2.x starts Surya inference servers
+  (layout, OCR error, llama.cpp) in their own sessions, and they kept running after `parsearena run`
+  finished. Runners now call Marker's `shutdown_models()` and stop every descendant process on exit,
+  and `run` stops the runner's whole process tree on timeouts and crashes.
+
+## [0.1.2] - 2026-11-03
+
+### Added
+- `src/parsearena/demo/pdfs/NOTICE.md` with title, authors, source, license and changes for each
+  bundled demo PDF, including the full CC BY 4.0 attribution for the DPO paper. It ships in the wheel.
+- README section "Third-party documents" (EN/ES) pointing to it.
+
+### Changed
+- The default Mistral model for `--gen-questions` is now `ministral-8b-latest` (was `mistral-small-latest`).
+- README notes the live `--gen-questions` tests with `ministral-8b-latest`.
+
+## [0.1.1] - 2026-11-03
+
+### Fixed
+- Hybrid retrieval: BM25 chunks with a score of 0 no longer take part in the fusion with a position
+  taken from their place in the corpus, and tied scores now share a rank. Recall@5 of the demo is
+  unchanged; some MRR values move by 0.02 to 0.03.
+- `--gen-questions`: a timeout is reported as a timeout, not as an unreachable server
+  (`PARSEARENA_LLM_TIMEOUT` sets the limit).
+
+### Changed
+- Embeddings are cached on disk by model and text hash: a second `parsearena demo` takes about 2 s
+  instead of about 40 s on a busy machine.
+- The first download of the embedding model prints one line instead of Hugging Face progress bars
+  and warnings.
+- README and viewer explain why hybrid can score below BM25 and dense, and give the real first-run
+  time of the quickstart.
+
+## [0.1.0] - 2026-11-03
+
+First release.
+
+- `parsearena run`: Docling 2.146.0, Marker 2.1.0, MinerU 4.1.6, Unstructured 0.27.36 and a pypdf 6.21.4
+  baseline, each in its own uv environment, run as a subprocess with pinned versions.
+- Per-page outputs, time per page (model loading reported apart as warm-up) and failures.
+- One chunker for every output; retrieval with BM25, local fastembed embeddings and their fusion:
+  answer survives parsing, recall@k and MRR.
+- Questions from a YAML file or drafted with an LLM (`--gen-questions`, Ollama, OpenRouter, OpenAI or
+  Mistral) with a `--max-cost` cap.
+- Static HTML viewer: page image next to any two parser outputs, metrics, timings and failures,
+  English and Spanish, light and dark.
+- `parsearena demo`: recorded outputs on three public PDFs, retrieval computed live.

parsearena-0.1.3/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Diego Pérez
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

parsearena-0.1.3/PKG-INFO

@@ -0,0 +1,267 @@
+Metadata-Version: 2.5
+Name: parsearena
+Version: 0.1.3
+Summary: Run your PDFs through Docling, Marker, MinerU and Unstructured. Compare the output side by side and see what each one does to retrieval.
+Project-URL: Homepage, https://diegoperezg7.github.io/parsearena/
+Project-URL: Repository, https://github.com/diegoperezg7/parsearena
+Project-URL: Issues, https://github.com/diegoperezg7/parsearena/issues
+Author: Diego Pérez
+License-Expression: MIT
+License-File: LICENSE
+Keywords: benchmark,docling,marker,mineru,parsing,pdf,rag,retrieval,unstructured
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Text Processing
+Requires-Python: <3.15,>=3.11
+Requires-Dist: fastembed==0.9.1
+Requires-Dist: numpy==2.3.5
+Requires-Dist: openai==3.42.1
+Requires-Dist: pillow==13.0.0
+Requires-Dist: pypdfium2==5.14.0
+Requires-Dist: pyyaml==6.0.3
+Requires-Dist: rich==15.0.0
+Requires-Dist: typer==0.27.3
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <img src="docs/assets/banner.svg" alt="parsearena" width="640">
+</p>
+
+<p align="center">
+  <a href="https://github.com/diegoperezg7/parsearena/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/diegoperezg7/parsearena/actions/workflows/ci.yml/badge.svg"></a>
+  <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-0064d2"></a>
+  <a href="https://github.com/diegoperezg7/parsearena/releases"><img alt="Release" src="https://img.shields.io/github/v/release/diegoperezg7/parsearena?color=0064d2"></a>
+  <a href="https://github.com/diegoperezg7/parsearena/stargazers"><img alt="Stars" src="https://img.shields.io/github/stars/diegoperezg7/parsearena?color=0064d2"></a>
+</p>
+
+<p align="center"><b>Run your PDFs through Docling, Marker, MinerU and Unstructured. Compare the output side by side and see what each one does to retrieval.</b></p>
+
+<p align="center"><a href="README.es.md">Leer en español</a> · <a href="https://diegoperezg7.github.io/parsearena/">Example viewer</a></p>
+
+<p align="center">
+  <img src="docs/assets/demo.gif" alt="parsearena demo in the terminal" width="760">
+</p>
+
+<p align="center">
+  <a href="https://diegoperezg7.github.io/parsearena/"><img src="docs/assets/viewer-light.png" alt="The parsearena viewer: page image next to two parser outputs" width="860"></a>
+</p>
+
+## Why
+
+Public parser benchmarks score parsers on their own fixed corpus:
+[ParseBench](https://github.com/run-llama/ParseBench) (625 stars, by LlamaIndex) and
+[OmniDocBench](https://github.com/opendatalab/OmniDocBench) (2,146 stars). That tells you which parser
+wins on their documents. It does not tell you what happens to *your* invoices, papers or scanned
+reports. Harnesses that run several parsers on your own PDFs exist only as 0-star repositories
+(for example [ticketvipers/pdf-parser-comparison](https://github.com/ticketvipers/pdf-parser-comparison)
+and [xvalovic/document-parser-benchmark](https://github.com/xvalovic/document-parser-benchmark)).
+
+parsearena works on your documents. It shows each parser's output next to the original page, and it
+measures the thing you actually care about in a RAG pipeline: after parsing and chunking, can the
+retriever still find the answer?
+
+## Quickstart
+
+The demo replays outputs recorded from a real run and computes retrieval live. No keys, no GPU,
+no Docker, no parser downloads.
+
+```bash
+uvx --from git+https://github.com/diegoperezg7/parsearena@v0.1.3 parsearena demo
+```
+
+or from a clone:
+
+```bash
+git clone https://github.com/diegoperezg7/parsearena && cd parsearena && uv run parsearena demo
+```
+
+It writes `parsearena-demo/index.html` and opens it. The first run takes about 75 s (dependencies and a
+65 MB embedding model); later runs take about 2 s because computed embeddings are cached. `uvx parsearena` is coming to PyPI.
+
+Then run it on your own PDFs:
+
+```bash
+parsearena run docs/*.pdf --questions questions.yaml
+parsearena run docs/*.pdf --gen-questions              # questions drafted by a local Ollama model
+parsearena run docs/*.pdf --parsers docling,pypdf      # any subset
+```
+
+## Example output
+
+Recorded on 3 Nov 2026 on an Apple M4 with 16 GB of RAM (macOS 27.2), three public PDFs (8 pages), 24 hand-written questions,
+recall@5 with hybrid retrieval (BM25 + `BAAI/bge-small-en-v1.5`, fused with reciprocal rank):
+
+| Parser | Answer survives | Recall@5 BM25 | Recall@5 dense | Recall@5 hybrid | MRR hybrid | s / page |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| Docling 2.146.0 | 88% | 67% | 67% | 63% | 0.55 | 11.8 |
+| Marker 2.1.0 | **96%** | **79%** | **67%** | **75%** | **0.63** | 21.0 |
+| MinerU 4.1.6 | **96%** | 63% | 58% | 58% | 0.52 | 7.6 |
+| Unstructured 0.27.36 | 92% | 67% | 54% | 54% | 0.52 | 6.2 |
+| pypdf 6.21.4 (baseline) | 75% | 58% | 46% | 54% | 0.40 | 0.02 |
+
+What the numbers say on these pages:
+
+- **The scanned page is where the baseline breaks.** pypdf has no text to read on the NACA page and
+  loses all 6 of its answers. The four parsers OCR it; Marker and MinerU keep all 6.
+- **Charts and fonts.** On the Census chart page pypdf returns glyph names instead of digits
+  (`/dollar.tab/eight.tab/zero.tab...`), so the chart values only survive through the table page.
+- **Tables survive, but not always in a retrievable shape.** MinerU keeps every number of Table A-1,
+  yet it merges several row labels into one HTML cell. With the same chunker, the label and its value
+  end up apart: the chunk with the numbers shares no word with the question, so BM25 never retrieves
+  it, dense retrieval ranks it 7th to 16th, and the fused list puts it 31st to 37th. The other parsers
+  mostly keep those questions in the top 10.
+- **Page furniture.** Docling and Marker drop the first-page footer, so "NeurIPS 2023" is gone from
+  their output. Unstructured merges the DPO results table with the body text around it and loses its
+  0.36 cell.
+- **Speed.** pypdf parses a page in 0.02 s. The model-based parsers take 6 to 21 s per page on this
+  machine, plus 20 to 100 s to load their models.
+
+Eight pages and 24 questions are a smoke test, not a ranking: run it on your own documents.
+
+Open the [example viewer](https://diegoperezg7.github.io/parsearena/) to check each of these against
+the page image.
+
+## How it works
+
+```mermaid
+flowchart LR
+    P[Your PDFs] --> R1
+    subgraph envs[One uv environment per parser]
+      R1[Docling] ~~~ R2[Marker] ~~~ R3[MinerU] ~~~ R4[Unstructured] ~~~ R5[pypdf]
+    end
+    envs -->|markdown per page, seconds, errors| O[outputs.json]
+    O --> C[Same chunker for all]
+    Q[questions.yaml or --gen-questions] --> E
+    C --> E[BM25 + fastembed + fusion]
+    E --> M[answer survives, recall@k, MRR]
+    M --> V[index.html viewer]
+    P -->|pypdfium2| V
+```
+
+1. **Isolated parsers.** Each parser is installed with pinned versions into its own virtual
+   environment under `~/.cache/parsearena/envs` and driven through a small runner script in a
+   subprocess. Their torch versions never meet, their licenses stay separate, and parsearena never
+   bundles any of them. Every parser runs with its default settings.
+2. **Timing.** Each runner first converts a two-page warm-up PDF (one text page, one image-only page)
+   so that model loading is reported apart. The seconds per page in the table are parsing only.
+   Device is `auto`: CUDA or Apple MPS when torch sees them, CPU otherwise (`--device cpu` to force).
+3. **One chunker.** Every output goes through the same paragraph packer (1,200 characters by default,
+   padding spaces and markdown table rules removed), so differences come from the parser.
+4. **Retrieval.** For each question, chunks from *all* documents are ranked with BM25, with local
+   embeddings ([fastembed](https://github.com/qdrant/fastembed), ONNX, no torch) and with their
+   reciprocal-rank fusion. Embeddings are cached by model and text under `~/.cache/parsearena/models`. A chunk is relevant if it belongs to the question's document and contains
+   the answer string.
+   Hybrid can score below both of its parts: fusion rewards chunks that both methods rank well, so a
+   chunk that only one of them finds (a table cell with none of the query words, say) drops in the
+   fused list. BM25 chunks with a score of 0 are treated as not retrieved, and tied scores share a rank.
+5. **Metrics.** *Answer survives*: the answer string is still in the parser output after
+   normalization (case, whitespace, markdown and HTML markup). *Recall@k*: a relevant chunk is in the
+   top k. *MRR*: reciprocal rank of the first relevant chunk, cut at 10.
+
+## Configuration
+
+| Option | Default | What it does |
+| --- | --- | --- |
+| `--parsers` | `docling,marker,mineru,unstructured,pypdf` | Parsers to run |
+| `--questions FILE` | | YAML question set (`parsearena questions-example` prints the format) |
+| `--gen-questions` | off | Draft questions with an LLM from the PDF text layer; keeps only answers found verbatim |
+| `--provider` / `--model` | `ollama` / `qwen3:4b` | `ollama`, `openrouter`, `openai` or `mistral` |
+| `--max-cost` | `1.0` | USD cap for LLM calls, checked before every call. Local models cost 0 |
+| `--k` | `5` | k for recall@k |
+| `--device` | `auto` | `auto`, `cpu`, `mps` or `cuda` |
+| `--chunk-chars` | `1200` | Chunk size |
+| `--embed-model` / `--no-dense` | `BAAI/bge-small-en-v1.5` | fastembed model, or BM25 only |
+| `--timeout` | `3600` | Seconds per parser |
+| `--out` | `parsearena-out` | Output directory (viewer, `outputs.json`, `results.json`, logs) |
+
+Environment variables: `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, `MISTRAL_API_KEY` (only for
+`--gen-questions` with that provider), `OLLAMA_HOST`, `PARSEARENA_HOME` (where environments live),
+`LLAMA_CPP_BINARY`, `PARSEARENA_LLM_TIMEOUT` (seconds per LLM call, default 300). See [.env.example](.env.example). `parsearena parsers` lists pinned versions and
+what is installed.
+
+`--gen-questions` was tested live on 3 Nov 2026 with `--provider mistral --model ministral-8b-latest`
+on two demo PDFs: 9 questions kept out of 10 asked, all sensible and with answers found verbatim in the
+text, for about $0.001. Review the drafted file before trusting it: one answer was just `β`, too short
+to be a useful retrieval target. `--gen-questions` was also tested live with mistral/ministral-8b-latest on census-income-2023.pdf:
+4 questions, all grounded, $0.0006. `ministral-8b-latest` is the default Mistral model.
+
+Questions file:
+
+```yaml
+questions:
+  - doc: report.pdf
+    q: What was the median household income in 2023?
+    answer: "80,610"
+  - doc: paper.pdf
+    q: Which preference model makes the partition function cancel?
+    answers: ["Bradley-Terry", "Bradley Terry"]
+```
+
+## What a real run costs
+
+Be ready for this the first time:
+
+- **Downloads.** Each parser environment is 1.1 to 1.4 GB, and the first parse downloads its models
+  from Hugging Face. On the machine below, the four parser environments plus models took about
+  9 (5.2 GB of environments and 3.5 GB of models) of disk.
+- **System tools.** Unstructured's `hi_res` strategy needs `poppler` and `tesseract`. Marker 2.x runs
+  its OCR model through llama.cpp when there is no NVIDIA GPU: install it (`brew install llama.cpp`)
+  or point `LLAMA_CPP_BINARY` at a `llama-server` binary. parsearena warns before running a parser
+  whose tools are missing.
+- **Time.** Measured on an Apple M4 with 16 GB of RAM (macOS 27.2), with the models already downloaded:
+
+  | Parser | Device | Model loading (s) | Seconds per page | Slowest document |
+  | --- | --- | ---: | ---: | --- |
+  | Docling | MPS | 38 | 11.8 | scanned page, 77 s |
+  | Marker | MPS + llama.cpp (Metal) | 99 | 21.0 | scanned page, 107 s |
+  | MinerU | MPS | 20 | 7.6 | DPO paper (equations), 43 s for 4 pages |
+  | Unstructured | CPU (ONNX) | 22 | 6.2 | Census, 21 s for 3 pages |
+  | pypdf | CPU | 0.1 | 0.02 | |
+
+  The whole `run` over the 8 demo pages took about 10 minutes, one parser at a time.
+
+## Parsers and their licenses
+
+parsearena is MIT. It installs the parsers on your machine on demand and never redistributes them;
+their licenses apply to your use of them.
+
+| Parser | Version | License |
+| --- | --- | --- |
+| [Docling](https://github.com/docling-project/docling) | 2.146.0 | MIT |
+| [Marker](https://github.com/datalab-to/marker) | 2.1.0 | Code Apache-2.0. Model weights under a modified OpenRAIL-M: free for research, personal use and startups under 5M USD |
+| [MinerU](https://github.com/opendatalab/MinerU) | 4.1.6 | Apache-2.0 with extra terms: a commercial license above 100M monthly active users or 20M USD monthly revenue, and attribution for online services |
+| [Unstructured](https://github.com/Unstructured-IO/unstructured) | 0.27.36 (`hi_res`) | Apache-2.0 |
+| [pypdf](https://github.com/py-pdf/pypdf) (baseline) | 6.21.4 | BSD-3-Clause |
+
+MinerU runs at its `medium` effort (local layout, OCR, table and formula models, no vision-language
+model) with its telemetry disabled (`MINERU_TELEMETRY_DISABLED=1`). parsearena itself has no telemetry.
+
+## Third-party documents
+
+| File | Pages used | License |
+| --- | --- | --- |
+| [U.S. Census Bureau, *Income in the United States: 2023* (P60-282)](https://www.census.gov/library/publications/2024/demo/p60-282.html) | 1, 2 (chart), 15 (Table A-1) | Public domain (U.S. government work) |
+| [Rafailov et al., *Direct Preference Optimization* (arXiv:2305.18290v3)](https://arxiv.org/abs/2305.18290) | 1, 4, 5 (equations), 9 (table) | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) |
+| [NACA Report 1135, *Equations, Tables, and Charts for Compressible Flow* (1953)](https://ntrs.nasa.gov/citations/19930091059) | p. 613, rasterized to an image-only page | Public domain (U.S. government work) |
+
+The pages were extracted unchanged, except the NACA page, which was re-rendered as an image so that it
+has no text layer, like a real scan. These excerpts are not covered by parsearena's MIT license. Full
+titles, authors, sources, licenses and the CC BY attribution are in
+[`src/parsearena/demo/pdfs/NOTICE.md`](src/parsearena/demo/pdfs/NOTICE.md), which ships in the wheel. The question set is in
+[`src/parsearena/demo/questions.yaml`](src/parsearena/demo/questions.yaml).
+
+## Roadmap
+
+- Cloud parsers: LlamaParse, Mistral OCR, Azure Document Intelligence
+- Vision-language model parsers (and MinerU's VLM efforts)
+- Table structure metrics (TEDS)
+- docx and pptx input
+
+## License
+
+[MIT](LICENSE). Contributions welcome, see [CONTRIBUTING.md](CONTRIBUTING.md).

parsearena-0.1.3/README.es.md

@@ -0,0 +1,243 @@
+<p align="center">
+  <img src="docs/assets/banner.svg" alt="parsearena" width="640">
+</p>
+
+<p align="center">
+  <a href="https://github.com/diegoperezg7/parsearena/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/diegoperezg7/parsearena/actions/workflows/ci.yml/badge.svg"></a>
+  <a href="LICENSE"><img alt="Licencia: MIT" src="https://img.shields.io/badge/license-MIT-0064d2"></a>
+  <a href="https://github.com/diegoperezg7/parsearena/releases"><img alt="Release" src="https://img.shields.io/github/v/release/diegoperezg7/parsearena?color=0064d2"></a>
+  <a href="https://github.com/diegoperezg7/parsearena/stargazers"><img alt="Estrellas" src="https://img.shields.io/github/stars/diegoperezg7/parsearena?color=0064d2"></a>
+</p>
+
+<p align="center"><b>Pasa tus PDF por Docling, Marker, MinerU y Unstructured. Compara las salidas lado a lado y mira qué le hace cada uno a la recuperación.</b></p>
+
+<p align="center"><a href="README.md">Read in English</a> · <a href="https://diegoperezg7.github.io/parsearena/">Visor de ejemplo</a></p>
+
+<p align="center">
+  <img src="docs/assets/demo.gif" alt="Demo de parsearena en la terminal" width="760">
+</p>
+
+<p align="center">
+  <a href="https://diegoperezg7.github.io/parsearena/"><img src="docs/assets/viewer-light.png" alt="El visor de parsearena: la página junto a dos salidas de parsers" width="860"></a>
+</p>
+
+## Por qué
+
+Los benchmarks públicos de parsers puntúan sobre su propio corpus fijo:
+[ParseBench](https://github.com/run-llama/ParseBench) (625 estrellas, de LlamaIndex) y
+[OmniDocBench](https://github.com/opendatalab/OmniDocBench) (2.146 estrellas). Te dicen qué parser gana
+con sus documentos, no qué pasa con *tus* facturas, papers o informes escaneados. Los arneses que
+pasan varios parsers por tus propios PDF solo existen como repos con 0 estrellas (por ejemplo
+[ticketvipers/pdf-parser-comparison](https://github.com/ticketvipers/pdf-parser-comparison) y
+[xvalovic/document-parser-benchmark](https://github.com/xvalovic/document-parser-benchmark)).
+
+parsearena trabaja con tus documentos. Pone la salida de cada parser junto a la página original y
+mide lo que de verdad importa en un RAG: después de parsear y trocear, ¿el recuperador sigue
+encontrando la respuesta?
+
+## Inicio rápido
+
+La demo reproduce salidas grabadas de una ejecución real y calcula la recuperación en vivo. Sin
+claves, sin GPU, sin Docker y sin descargar parsers.
+
+```bash
+uvx --from git+https://github.com/diegoperezg7/parsearena@v0.1.3 parsearena demo
+```
+
+o desde un clon:
+
+```bash
+git clone https://github.com/diegoperezg7/parsearena && cd parsearena && uv run parsearena demo
+```
+
+Escribe `parsearena-demo/index.html` y lo abre. La primera vez tarda unos 75 s (dependencias y un
+modelo de embeddings de 65 MB); después, unos 2 s, porque los embeddings ya calculados se guardan en caché. `uvx parsearena` llegará con la publicación en PyPI.
+
+Después, con tus PDF:
+
+```bash
+parsearena run docs/*.pdf --questions preguntas.yaml
+parsearena run docs/*.pdf --gen-questions              # preguntas redactadas por un modelo local de Ollama
+parsearena run docs/*.pdf --parsers docling,pypdf      # cualquier subconjunto
+```
+
+## Resultado de ejemplo
+
+Grabado el 3 nov 2026 en un Apple M4 con 16 GB de RAM (macOS 27.2), tres PDF públicos (8 páginas), 24 preguntas escritas a mano,
+recall@5 con recuperación híbrida (BM25 + `BAAI/bge-small-en-v1.5`, fusionados por rango recíproco):
+
+| Parser | Respuesta sobrevive | Recall@5 BM25 | Recall@5 densa | Recall@5 híbrida | MRR híbrida | s / página |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| Docling 2.146.0 | 88% | 67% | 67% | 63% | 0,55 | 11,8 |
+| Marker 2.1.0 | **96%** | **79%** | **67%** | **75%** | **0,63** | 21,0 |
+| MinerU 4.1.6 | **96%** | 63% | 58% | 58% | 0,52 | 7,6 |
+| Unstructured 0.27.36 | 92% | 67% | 54% | 54% | 0,52 | 6,2 |
+| pypdf 6,21.4 (línea base) | 75% | 58% | 46% | 54% | 0,40 | 0,02 |
+
+Lo que dicen las cifras en estas páginas:
+
+- **La página escaneada es donde la línea base se rompe.** pypdf no tiene texto que leer en la
+  página de NACA y pierde sus 6 respuestas. Los cuatro parsers hacen OCR; Marker y MinerU conservan
+  las 6.
+- **Gráficos y fuentes.** En la página del gráfico del Census, pypdf devuelve nombres de glifos en vez
+  de dígitos (`/dollar.tab/eight.tab/zero.tab...`), así que esos valores solo sobreviven por la tabla.
+- **Las tablas sobreviven, pero no siempre de forma recuperable.** MinerU conserva todos los números de
+  la tabla A-1, pero junta varias etiquetas de fila en una sola celda HTML. Con el mismo troceado, la
+  etiqueta y su valor quedan separados: el fragmento con los números no comparte ninguna palabra con
+  la pregunta, así que BM25 no lo recupera, la búsqueda densa lo pone entre el puesto 7 y el 16, y la
+  fusión lo deja entre el 31 y el 37. Los demás parsers dejan esas preguntas casi siempre entre los 10
+  primeros.
+- **Pies de página.** Docling y Marker quitan el pie de la primera página, así que "NeurIPS 2023"
+  desaparece de su salida. Unstructured mezcla la tabla de resultados de DPO con el texto de alrededor
+  y pierde la celda 0.36.
+- **Velocidad.** pypdf parsea una página en 0,02 s. Los parsers con modelos tardan entre 6 y 21 s por
+  página en esta máquina, más 20 a 100 s de carga de modelos.
+
+Ocho páginas y 24 preguntas son una prueba de humo, no una clasificación: pásalo por tus documentos.
+
+Abre el [visor de ejemplo](https://diegoperezg7.github.io/parsearena/) para comprobar cada punto contra
+la imagen de la página.
+
+## Cómo funciona
+
+```mermaid
+flowchart LR
+    P[Tus PDF] --> R1
+    subgraph envs[Un entorno uv por parser]
+      R1[Docling] ~~~ R2[Marker] ~~~ R3[MinerU] ~~~ R4[Unstructured] ~~~ R5[pypdf]
+    end
+    envs -->|markdown por página, segundos, errores| O[outputs.json]
+    O --> C[El mismo troceado para todos]
+    Q[questions.yaml o --gen-questions] --> E
+    C --> E[BM25 + fastembed + fusión]
+    E --> M[respuesta sobrevive, recall@k, MRR]
+    M --> V[visor index.html]
+    P -->|pypdfium2| V
+```
+
+1. **Parsers aislados.** Cada parser se instala con versiones fijadas en su propio entorno virtual
+   (`~/.cache/parsearena/envs`) y se ejecuta con un pequeño runner en un subproceso. Sus versiones de
+   torch no chocan, sus licencias no se mezclan y parsearena no empaqueta ninguno. Todos corren con su
+   configuración por defecto.
+2. **Tiempos.** Cada runner convierte primero un PDF de calentamiento de dos páginas (una con texto y
+   otra solo imagen) para que la carga de modelos se cuente aparte. Los segundos por página son solo
+   de parseo. El dispositivo es `auto`: CUDA o MPS de Apple si torch los ve, CPU si no
+   (`--device cpu` para forzarlo).
+3. **Un solo troceado.** Todas las salidas pasan por el mismo empaquetador de párrafos (1.200
+   caracteres por defecto, sin espacios de relleno ni reglas de tablas markdown), así que las
+   diferencias vienen del parser.
+4. **Recuperación.** Para cada pregunta se ordenan los fragmentos de *todos* los documentos con BM25,
+   con embeddings locales ([fastembed](https://github.com/qdrant/fastembed), ONNX, sin torch) y con su
+   fusión por rango recíproco. Los embeddings se guardan en caché por modelo y texto en `~/.cache/parsearena/models`. Un fragmento es relevante si es del documento de la pregunta y contiene
+   la respuesta.
+   La híbrida puede quedar por debajo de sus dos partes: la fusión premia los fragmentos que ambos
+   métodos ponen arriba, así que uno que solo encuentra un método (por ejemplo, una celda de tabla sin
+   ninguna palabra de la pregunta) baja en la lista fusionada. Los fragmentos con BM25 igual a 0 cuentan
+   como no recuperados, y las puntuaciones empatadas comparten posición.
+5. **Métricas.** *Respuesta sobrevive*: el texto de la respuesta sigue en la salida tras normalizar
+   (mayúsculas, espacios, marcado markdown y HTML). *Recall@k*: hay un fragmento relevante entre los k
+   primeros. *MRR*: rango recíproco del primer fragmento relevante, con corte en 10.
+
+## Configuración
+
+| Opción | Por defecto | Qué hace |
+| --- | --- | --- |
+| `--parsers` | `docling,marker,mineru,unstructured,pypdf` | Parsers a ejecutar |
+| `--questions FICHERO` | | Preguntas en YAML (`parsearena questions-example` muestra el formato) |
+| `--gen-questions` | no | Redacta preguntas con un LLM a partir del texto del PDF; solo guarda respuestas literales |
+| `--provider` / `--model` | `ollama` / `qwen3:4b` | `ollama`, `openrouter`, `openai` o `mistral` |
+| `--max-cost` | `1.0` | Tope en USD para llamadas a LLM, comprobado antes de cada llamada. Los modelos locales cuestan 0 |
+| `--k` | `5` | k de recall@k |
+| `--device` | `auto` | `auto`, `cpu`, `mps` o `cuda` |
+| `--chunk-chars` | `1200` | Tamaño de fragmento |
+| `--embed-model` / `--no-dense` | `BAAI/bge-small-en-v1.5` | Modelo de fastembed, o solo BM25 |
+| `--timeout` | `3600` | Segundos por parser |
+| `--out` | `parsearena-out` | Carpeta de salida (visor, `outputs.json`, `results.json`, logs) |
+
+Variables de entorno: `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, `MISTRAL_API_KEY` (solo para
+`--gen-questions` con ese proveedor), `OLLAMA_HOST`, `PARSEARENA_HOME` (dónde viven los entornos),
+`LLAMA_CPP_BINARY` y `PARSEARENA_LLM_TIMEOUT` (segundos por llamada al LLM, 300 por defecto). Mira [.env.example](.env.example). `parsearena parsers` muestra versiones y qué
+está instalado.
+
+`--gen-questions` se probó en vivo el 3 nov 2026 con `--provider mistral --model ministral-8b-latest`
+sobre dos PDF de la demo: 9 preguntas guardadas de 10 pedidas, todas con sentido y con la respuesta
+literal en el texto, por unos 0,001 USD. Revisa el fichero antes de fiarte: una respuesta era solo `β`,
+demasiado corta para ser un buen objetivo de recuperación. `--gen-questions` también se probó en vivo con mistral/ministral-8b-latest sobre census-income-2023.pdf:
+4 preguntas, todas con la respuesta en el texto, 0,0006 USD. `ministral-8b-latest` es el modelo de Mistral por defecto.
+
+Fichero de preguntas:
+
+```yaml
+questions:
+  - doc: informe.pdf
+    q: What was the median household income in 2023?
+    answer: "80,610"
+  - doc: paper.pdf
+    q: Which preference model makes the partition function cancel?
+    answers: ["Bradley-Terry", "Bradley Terry"]
+```
+
+## Lo que cuesta una ejecución real
+
+La primera vez, cuenta con esto:
+
+- **Descargas.** Cada entorno de parser ocupa entre 1,1 y 1,4 GB, y el primer parseo descarga sus
+  modelos de Hugging Face. En la máquina de abajo, los cuatro entornos más los modelos ocuparon unos
+  9 (5,2 GB de entornos y 3,5 GB de modelos).
+- **Herramientas del sistema.** La estrategia `hi_res` de Unstructured necesita `poppler` y
+  `tesseract`. Marker 2.x ejecuta su modelo de OCR con llama.cpp cuando no hay GPU NVIDIA: instálalo
+  (`brew install llama.cpp`) o apunta `LLAMA_CPP_BINARY` a un binario `llama-server`. parsearena avisa
+  antes de ejecutar un parser al que le faltan herramientas.
+- **Tiempo.** Medido en un Apple M4 con 16 GB de RAM (macOS 27.2), con los modelos ya descargados:
+
+  | Parser | Dispositivo | Carga de modelos (s) | Segundos por página | Documento más lento |
+  | --- | --- | ---: | ---: | --- |
+  | Docling | MPS | 38 | 11,8 | página escaneada, 77 s |
+  | Marker | MPS + llama.cpp (Metal) | 99 | 21,0 | página escaneada, 107 s |
+  | MinerU | MPS | 20 | 7,6 | paper de DPO (ecuaciones), 43 s para 4 páginas |
+  | Unstructured | CPU (ONNX) | 22 | 6,2 | Census, 21 s para 3 páginas |
+  | pypdf | CPU | 0,1 | 0,02 | |
+
+  El `run` completo sobre las 8 páginas de la demo tardó unos 10 minutos, un parser cada vez.
+
+## Parsers y sus licencias
+
+parsearena es MIT. Instala los parsers en tu máquina cuando los usas y nunca los redistribuye; sus
+licencias se aplican a tu uso.
+
+| Parser | Versión | Licencia |
+| --- | --- | --- |
+| [Docling](https://github.com/docling-project/docling) | 2.146.0 | MIT |
+| [Marker](https://github.com/datalab-to/marker) | 2.1.0 | Código Apache-2.0. Pesos con OpenRAIL-M modificada: gratis para investigación, uso personal y startups de menos de 5 M USD |
+| [MinerU](https://github.com/opendatalab/MinerU) | 4.1.6 | Apache-2.0 con términos extra: licencia comercial por encima de 100 M de usuarios activos al mes o 20 M USD de ingresos mensuales, y atribución en servicios online |
+| [Unstructured](https://github.com/Unstructured-IO/unstructured) | 0.27.36 (`hi_res`) | Apache-2.0 |
+| [pypdf](https://github.com/py-pdf/pypdf) (línea base) | 6.21.4 | BSD-3-Clause |
+
+MinerU corre con su esfuerzo `medium` (modelos locales de layout, OCR, tablas y fórmulas, sin modelo de
+visión y lenguaje) y con su telemetría desactivada (`MINERU_TELEMETRY_DISABLED=1`). parsearena no tiene
+telemetría.
+
+## Documentos de terceros
+
+| Fichero | Páginas | Licencia |
+| --- | --- | --- |
+| [U.S. Census Bureau, *Income in the United States: 2023* (P60-282)](https://www.census.gov/library/publications/2024/demo/p60-282.html) | 1, 2 (gráfico), 15 (tabla A-1) | Dominio público (obra del gobierno de EE. UU.) |
+| [Rafailov et al., *Direct Preference Optimization* (arXiv:2305.18290v3)](https://arxiv.org/abs/2305.18290) | 1, 4, 5 (ecuaciones), 9 (tabla) | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) |
+| [NACA Report 1135, *Equations, Tables, and Charts for Compressible Flow* (1953)](https://ntrs.nasa.gov/citations/19930091059) | p. 613, rasterizada a página solo imagen | Dominio público (obra del gobierno de EE. UU.) |
+
+Las páginas se extrajeron sin cambios, salvo la de NACA, que se volvió a renderizar como imagen para
+que no tenga capa de texto, como un escaneo real. Estos extractos no están bajo la licencia MIT de
+parsearena. Títulos, autores, fuentes, licencias y la atribución CC BY completos están en
+[`src/parsearena/demo/pdfs/NOTICE.md`](src/parsearena/demo/pdfs/NOTICE.md), que va dentro del wheel. Las preguntas están en
+[`src/parsearena/demo/questions.yaml`](src/parsearena/demo/questions.yaml).
+
+## Hoja de ruta
+
+- Parsers en la nube: LlamaParse, Mistral OCR, Azure Document Intelligence
+- Parsers con modelos de visión y lenguaje (y los esfuerzos VLM de MinerU)
+- Métricas de estructura de tablas (TEDS)
+- Entrada docx y pptx
+
+## Licencia
+
+[MIT](LICENSE). Las contribuciones son bienvenidas, mira [CONTRIBUTING.md](CONTRIBUTING.md).