vera-ingest-docling 0.3.0__tar.gz

vera_ingest_docling-0.3.0/.gitignore

@@ -0,0 +1,16 @@
+.venv/
+__pycache__/
+.pytest_cache/
+*.pyc
+*.vera
+dist/
+dist-electron/
+release/
+build/
+node_modules/
+*.egg-info/
+examples/*.pdf
+site/
+.env
+.env.local
+.env.*.local

vera_ingest_docling-0.3.0/PKG-INFO

@@ -0,0 +1,100 @@
+Metadata-Version: 2.5
+Name: vera-ingest-docling
+Version: 0.3.0
+Summary: Optional Docling HybridChunker ingest pipeline for VERA
+Project-URL: Homepage, https://github.com/dkylewillis/vera
+Project-URL: Repository, https://github.com/dkylewillis/vera
+Project-URL: Documentation, https://dkylewillis.github.io/vera/packages/vera-ingest-docling/
+Author: Kyle Willis
+License: Apache-2.0
+Keywords: chunking,docling,pdf,semantic-search,vera
+Classifier: Development Status :: 3 - Alpha
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Text Processing :: Indexing
+Requires-Python: >=3.10
+Requires-Dist: docling[rapidocr]<2.119,>=2.118
+Requires-Dist: vera-ingest>=0.3.0
+Description-Content-Type: text/markdown
+
+# vera-ingest-docling
+
+Optional Docling ingest pipeline for VERA. Registers the `docling` provider
+(default variant `hybrid`) under the `vera.ingest_pipelines` entry-point group.
+
+The pipeline uses Docling's `DocumentConverter` for layout-aware PDF parsing and
+`HybridChunker` for token-aware chunks. Readable chunk text is stored for
+keyword search; contextualized text from `HybridChunker.contextualize()` is used
+for embeddings.
+
+## Install
+
+```bash
+python -m pip install "vera-ingest-docling>=0.3.0"
+```
+
+From a repository checkout with uv (workspace `.venv` for CLI and tests):
+
+```bash
+uv sync --extra docling
+```
+
+Non-desktop users can install the CLI extra:
+
+```bash
+pip install "vera-cli[docling]>=0.3.0"
+```
+
+Python 3.10 or newer is required. The package depends on Docling's `rapidocr`
+extra so RapidOCR and `onnxruntime` are installed for OCR. RapidOCR ONNX
+weights come with that extra; first conversion may still download Docling
+layout models (about 380 MB: Heron ONNX + TableFormer accurate). Set
+`DOCLING_ARTIFACTS_PATH` to a local cache (or prefetch layout models offline)
+for air-gapped runs. Incomplete caches are not treated as ready. Hub progress
+is visible on CLI stderr.
+This extra is not bundled in the 0.3.0 desktop installer and is not listed in
+Convert.
+
+## Usage
+
+```bash
+vera convert "manual.pdf" "manual.vera" --parser docling
+# or explicitly:
+vera convert "manual.pdf" "manual.vera" --parser docling:hybrid
+```
+
+```python
+from vera_ingest import convert
+
+convert("manual.pdf", "manual.vera", parser="docling")
+```
+
+## Notes
+
+- Defaults: `chunk_size=500` whitespace tokens (not LLM subword tokens),
+  `ocr_mode=auto`, `ocr_language=en`,
+  `pdf_backend=docling_parse`. Docling does **not** advertise `overlap` or
+  `ocr_dpi`, so those legacy convert/CLI aliases are not forwarded. The
+  Tesseract `--ocr-language` alias is also not forwarded; Docling keeps `en`.
+- Prefer `pipeline_options=` / `--pipeline-option KEY=VALUE` for provider-owned
+  settings; `--chunk-size` and `--ocr*` remain compatibility aliases for
+  pipelines that accept them.
+- OCR modes map to Docling/RapidOCR: `off`, `auto` (default), and `force`
+  (full-page OCR). `ocr_language` expects a RapidOCR-native code (`en`, `fr`,
+  `cyrillic`, ...) — Tesseract-style codes such as `eng` are **not**
+  translated. The shared `--ocr-language` CLI default (`eng`) is not
+  forwarded; pass `--pipeline-option ocr_language=fr` (or another RapidOCR
+  code) when you need a non-default language.
+- Torch model compilation is disabled so Windows does not need MSVC `cl.exe`.
+- Picture crops are stored as figure attachments and linked onto a nearby
+  same-page chunk so search `--figures` can return them. Docling's
+  HybridChunker omits pictures from chunk text.
+- On page-level memory errors (`bad_alloc`), VERA retries failed pages then
+  falls back to whole-document `pypdfium2`, then page-batch `pypdfium2` if
+  that still raises. Force the backend with
+  `--pipeline-option pdf_backend=pypdfium2`. Conversion rejects only when
+  recovery is exhausted. Failures include the underlying exception and print
+  it on sidecar stderr.
+
+See the [vera-ingest-docling documentation](https://dkylewillis.github.io/vera/packages/vera-ingest-docling/)
+and [conversion guide](https://github.com/dkylewillis/vera/blob/main/docs/conversion.md).

vera_ingest_docling-0.3.0/README.md

@@ -0,0 +1,81 @@
+# vera-ingest-docling
+
+Optional Docling ingest pipeline for VERA. Registers the `docling` provider
+(default variant `hybrid`) under the `vera.ingest_pipelines` entry-point group.
+
+The pipeline uses Docling's `DocumentConverter` for layout-aware PDF parsing and
+`HybridChunker` for token-aware chunks. Readable chunk text is stored for
+keyword search; contextualized text from `HybridChunker.contextualize()` is used
+for embeddings.
+
+## Install
+
+```bash
+python -m pip install "vera-ingest-docling>=0.3.0"
+```
+
+From a repository checkout with uv (workspace `.venv` for CLI and tests):
+
+```bash
+uv sync --extra docling
+```
+
+Non-desktop users can install the CLI extra:
+
+```bash
+pip install "vera-cli[docling]>=0.3.0"
+```
+
+Python 3.10 or newer is required. The package depends on Docling's `rapidocr`
+extra so RapidOCR and `onnxruntime` are installed for OCR. RapidOCR ONNX
+weights come with that extra; first conversion may still download Docling
+layout models (about 380 MB: Heron ONNX + TableFormer accurate). Set
+`DOCLING_ARTIFACTS_PATH` to a local cache (or prefetch layout models offline)
+for air-gapped runs. Incomplete caches are not treated as ready. Hub progress
+is visible on CLI stderr.
+This extra is not bundled in the 0.3.0 desktop installer and is not listed in
+Convert.
+
+## Usage
+
+```bash
+vera convert "manual.pdf" "manual.vera" --parser docling
+# or explicitly:
+vera convert "manual.pdf" "manual.vera" --parser docling:hybrid
+```
+
+```python
+from vera_ingest import convert
+
+convert("manual.pdf", "manual.vera", parser="docling")
+```
+
+## Notes
+
+- Defaults: `chunk_size=500` whitespace tokens (not LLM subword tokens),
+  `ocr_mode=auto`, `ocr_language=en`,
+  `pdf_backend=docling_parse`. Docling does **not** advertise `overlap` or
+  `ocr_dpi`, so those legacy convert/CLI aliases are not forwarded. The
+  Tesseract `--ocr-language` alias is also not forwarded; Docling keeps `en`.
+- Prefer `pipeline_options=` / `--pipeline-option KEY=VALUE` for provider-owned
+  settings; `--chunk-size` and `--ocr*` remain compatibility aliases for
+  pipelines that accept them.
+- OCR modes map to Docling/RapidOCR: `off`, `auto` (default), and `force`
+  (full-page OCR). `ocr_language` expects a RapidOCR-native code (`en`, `fr`,
+  `cyrillic`, ...) — Tesseract-style codes such as `eng` are **not**
+  translated. The shared `--ocr-language` CLI default (`eng`) is not
+  forwarded; pass `--pipeline-option ocr_language=fr` (or another RapidOCR
+  code) when you need a non-default language.
+- Torch model compilation is disabled so Windows does not need MSVC `cl.exe`.
+- Picture crops are stored as figure attachments and linked onto a nearby
+  same-page chunk so search `--figures` can return them. Docling's
+  HybridChunker omits pictures from chunk text.
+- On page-level memory errors (`bad_alloc`), VERA retries failed pages then
+  falls back to whole-document `pypdfium2`, then page-batch `pypdfium2` if
+  that still raises. Force the backend with
+  `--pipeline-option pdf_backend=pypdfium2`. Conversion rejects only when
+  recovery is exhausted. Failures include the underlying exception and print
+  it on sidecar stderr.
+
+See the [vera-ingest-docling documentation](https://dkylewillis.github.io/vera/packages/vera-ingest-docling/)
+and [conversion guide](https://github.com/dkylewillis/vera/blob/main/docs/conversion.md).

vera_ingest_docling-0.3.0/pyproject.toml

@@ -0,0 +1,38 @@
+[project]
+name = "vera-ingest-docling"
+version = "0.3.0"
+description = "Optional Docling HybridChunker ingest pipeline for VERA"
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = [
+  "vera-ingest>=0.3.0",
+  # rapidocr extra pulls RapidOCR + onnxruntime (required for Docling OCR).
+  "docling[rapidocr]>=2.118,<2.119",
+]
+authors = [{name = "Kyle Willis"}]
+license = {text = "Apache-2.0"}
+keywords = ["docling", "pdf", "chunking", "semantic-search", "vera"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "License :: OSI Approved :: Apache Software License",
+  "Programming Language :: Python :: 3",
+  "Topic :: Text Processing :: Indexing",
+]
+
+[project.urls]
+Homepage = "https://github.com/dkylewillis/vera"
+Repository = "https://github.com/dkylewillis/vera"
+Documentation = "https://dkylewillis.github.io/vera/packages/vera-ingest-docling/"
+
+[project.entry-points."vera.ingest_pipelines"]
+docling = "vera_ingest_docling:create_pipeline"
+
+[project.entry-points."vera.ingest_pipeline_descriptors"]
+docling = "vera_ingest_docling:create_descriptor"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/vera_ingest_docling"]

vera_ingest_docling-0.3.0/src/vera_ingest_docling/__init__.py

@@ -0,0 +1,73 @@
+"""Optional Docling HybridChunker ingest pipeline for VERA."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from vera_ingest.descriptors import PipelineDescriptor
+from vera_ingest.pipeline import (
+    IngestPipeline,
+    UnknownIngestPipelineError,
+    register_ingest_pipeline,
+    register_ingest_pipeline_descriptor,
+)
+
+from .options import DoclingOptions, _docling_runtime_available, describe_pipeline
+
+__all__ = [
+    "DoclingHybridPipeline",
+    "DoclingOptions",
+    "create_descriptor",
+    "create_pipeline",
+    "describe_pipeline",
+    "ensure_registered",
+]
+
+
+def __getattr__(name: str) -> Any:
+    if name == "DoclingHybridPipeline":
+        from .pipeline import DoclingHybridPipeline
+
+        return DoclingHybridPipeline
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def create_pipeline(variant: str = "hybrid") -> IngestPipeline:
+    """Entry-point factory for ``vera.ingest_pipelines`` provider ``docling``."""
+    normalized = (variant or "hybrid").strip().lower()
+    if normalized not in {"", "hybrid"}:
+        raise UnknownIngestPipelineError(
+            f"Unknown Docling pipeline variant {variant!r}; use 'docling' or 'docling:hybrid'."
+        )
+    if not _docling_runtime_available():
+        raise UnknownIngestPipelineError(
+            "Docling is not installed in this environment. "
+            "Install with: python -m pip install 'vera-ingest-docling>=0.3.0'"
+        )
+    from vera_ingest.timing import timed_step
+
+    with timed_step("import_docling_pipeline"):
+        from .pipeline import DoclingHybridPipeline
+
+    return DoclingHybridPipeline()
+
+
+def create_descriptor(variant: str = "hybrid") -> PipelineDescriptor:
+    """Entry-point factory for ``vera.ingest_pipeline_descriptors``."""
+    try:
+        return describe_pipeline(variant)
+    except ValueError as exc:
+        raise UnknownIngestPipelineError(str(exc)) from exc
+
+
+def ensure_registered(*, replace: bool = True) -> None:
+    """Register the ``docling`` pipeline without relying on package metadata.
+
+    Entry-point discovery fails in PyInstaller freezes and PYTHONPATH-only
+    source runs that never install ``vera-ingest-docling`` dist-info.
+    """
+    register_ingest_pipeline("docling", create_pipeline, replace=replace)
+    register_ingest_pipeline_descriptor("docling", create_descriptor, replace=replace)
+
+
+ensure_registered()