fastdocparse 0.1.1__tar.gz → 0.3.0__tar.gz

fastdocparse-0.3.0/PKG-INFO

@@ -0,0 +1,163 @@
+Metadata-Version: 2.4
+Name: fastdocparse
+Version: 0.3.0
+Summary: Extract structured data from semi-structured documents using any OpenAI-compatible LLM, with per-field grounding and confidence.
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/pranjalparmar/fastdocparse
+Project-URL: Repository, https://github.com/pranjalparmar/fastdocparse
+Project-URL: Issues, https://github.com/pranjalparmar/fastdocparse/issues
+Project-URL: Documentation, https://github.com/pranjalparmar/fastdocparse/tree/main/docs
+Keywords: document-extraction,llm,pdf,ocr,invoice-extraction,structured-data,openai,ollama,grounding
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: <3.13,>=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic>=2.0
+Requires-Dist: openai>=1.0
+Requires-Dist: pymupdf>=1.24
+Requires-Dist: pillow>=10.0
+Requires-Dist: rapidocr-onnxruntime>=1.3
+Requires-Dist: typer>=0.12
+Requires-Dist: PyYAML>=6.0
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Dynamic: license-file
+
+# fastdocparse
+
+[![PyPI](https://img.shields.io/pypi/v/fastdocparse.svg)](https://pypi.org/project/fastdocparse/)
+[![CI](https://github.com/pranjalparmar/fastdocparse/actions/workflows/ci.yml/badge.svg)](https://github.com/pranjalparmar/fastdocparse/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
+Extract structured data from semi-structured documents (invoices, bills, tax forms, resumes, bank statements, shipment manifests) using any OpenAI-compatible LLM (OpenAI, Ollama, vLLM, Groq, etc.), with **per-field grounding and confidence**, not just raw extraction.
+
+## Why this, not just another parser
+
+Most extractors give you a value and no way to know if it's real. This one tells you:
+
+- **`grounded`**: the value was found verbatim (or near-verbatim) in the source document text.
+- **`ungrounded`**: the value doesn't appear in the source, likely a hallucination. Flag for human review.
+- **`missing_required`**: a field you marked required came back empty.
+- **`invalid_format`**: the value doesn't match a pattern/enum constraint you declared (e.g. a shipment status outside the allowed list).
+- **`failed_check`**: a custom cross-field rule failed (e.g. line items don't sum to the stated total).
+
+No extra LLM call for any of this: it's deterministic, string/rule-based validation against text you already extracted.
+
+**Where it fits:** semi-structured documents with recurring fields (invoices, bills, tax forms, resumes, statements), and prose documents where *proving* a value came from the source matters (contracts, legal clauses, insurance claims). It is not a vision-LLM pipeline. It works from extracted text (digital PDF text layer, or local OCR for scans/images), which is what keeps it fast, cheap, and usable with small local models. Messy handwritten forms or complex multi-column layouts are a known weaker spot (see [document-extractor-spec.md](document-extractor-spec.md)).
+
+## How this compares
+
+| Project | Approach | Where it beats fastdocparse | Where fastdocparse can beat it |
+|---|---|---|---|
+| **[Sparrow](https://github.com/katanaml/sparrow)** (katanaml) | Vision-LLM first (MLX/vLLM/Ollama/Mistral OCR), multi-service platform, API-first | Layout-aware (sees the page), mature, table templates for complex tables | No GPU required, single `pip install`, Pydantic-schema devex vs. raw JSON-string CLI args |
+| **[LangExtract](https://github.com/google/langextract)** (Google) | Text-first, source-grounding (character-offset mapping), few-shot examples required | Real grounding/traceability, brand trust | Purpose-built for documents (OCR routing) vs. text-in/text-out |
+
+**The honest edge:** *fast, cheap, local-model-friendly extraction for clean-to-moderate documents*, not "better than Sparrow at everything." This has not yet been validated with a real head-to-head benchmark ([tracking issue](https://github.com/pranjalparmar/fastdocparse/issues/19)). Treat it as a design goal, not a proven claim, until that lands.
+
+## Two ways to use it
+
+| | Who it's for | How |
+|---|---|---|
+| **CLI** | No coding needed | `fastdocparse extract <file> <schema.json>` / `fastdocparse --version` |
+| **Python API** | Building it into your own app | `DocumentParser(client).extract(document_bytes, schema)` |
+
+Defining *what* to extract also has two paths: hand-write a JSON/YAML schema file, or describe it in plain English and let the LLM draft the schema for you.
+
+## Install
+
+```bash
+pip install fastdocparse
+```
+
+For local development instead:
+
+```bash
+git clone https://github.com/pranjalparmar/fastdocparse
+cd fastdocparse
+python -m venv venv
+source venv/bin/activate        # Windows: venv\Scripts\activate
+pip install -e ".[dev]"
+```
+
+You also need access to an LLM. Either:
+- An OpenAI API key (`export OPENAI_API_KEY=...` or pass `--api-key`), or
+- A local model via [Ollama](https://ollama.com/): no API key, no cloud, documents never leave your machine.
+
+## Quickstart: CLI (no coding)
+
+```bash
+# 1. Extract using one of the bundled example schemas
+fastdocparse extract sample_invoice.png src/fastdocparse/schemas/invoice.json \
+  --model gpt-4o-mini --api-key sk-...
+
+# Or with a local model via Ollama (no API key needed):
+fastdocparse extract sample_invoice.png src/fastdocparse/schemas/invoice.json \
+  --model llama3.2 --base-url http://localhost:11434/v1 --api-key ollama
+```
+
+Output is JSON, printed to stdout (or saved with `--output result.json`):
+
+```json
+{
+  "_meta": { "truncated": false, "truncation_reason": null },
+  "invoice_number": { "value": "INV-9011", "confidence": "high", "flags": ["grounded"] },
+  "total_price": { "value": 100.0, "confidence": "high", "flags": ["grounded"] }
+}
+```
+
+Don't want to write JSON at all? Describe the fields in plain English instead:
+
+```bash
+fastdocparse schema-from-text \
+  "I want the invoice number, total price, and vendor name. Invoice number and total are required." \
+  --output my_invoice_schema.json
+
+# review my_invoice_schema.json, then:
+fastdocparse extract my_invoice.pdf my_invoice_schema.json
+```
+
+## Quickstart: Python API
+
+```python
+from fastdocparse import Schema, Field, LLMClient, DocumentParser
+
+schema = Schema(
+    name="Invoice",
+    fields=[
+        Field(name="invoice_number", description="The invoice number", required=True),
+        Field(name="total_price", description="Total amount due", type="number", required=True),
+    ],
+)
+
+client = LLMClient(model="gpt-4o-mini", api_key="sk-...")
+# or: LLMClient(base_url="http://localhost:11434/v1", api_key="ollama", model="llama3.2")
+
+parser = DocumentParser(client=client)
+
+with open("invoice.pdf", "rb") as f:
+    result = parser.extract(f.read(), schema)
+
+print(result["invoice_number"])  # {'value': 'INV-9011', 'confidence': 'high', 'flags': ['grounded']}
+```
+
+## Full documentation
+
+- [Getting Started](docs/getting-started.md): step-by-step install, CLI, and API walkthroughs
+- [Schema Guide](docs/schema-guide.md): every field option (`type`, `required`, `pattern`, `enum`, `sub_fields`, few-shot `examples`), for JSON, YAML, and plain-English authoring
+- [Output & Validation](docs/output-format.md): the full result shape, what each confidence flag means, and how to write custom cross-check rules
+- [Architecture](docs/architecture.md): diagrams of the pipeline, the module dependency graph, and where to plug in a contribution
+- [Project spec](document-extractor-spec.md): architecture, phased roadmap, honest competitive positioning
+
+Want to contribute? Start with [docs/architecture.md](docs/architecture.md) for the map, then [CONTRIBUTING.md](CONTRIBUTING.md) for the process.
+
+## Status
+
+Core extraction, grounding, chunking, both CLI/API paths, and real packaging are implemented and tested (75 tests, `pytest -v`). Published on PyPI as [`fastdocparse`](https://pypi.org/project/fastdocparse/). `pip install fastdocparse` installs a working `fastdocparse` command and a proper `fastdocparse.*` import namespace, verified end to end with a clean-virtualenv install straight from the real public index. Not yet done: a hosted API. See [document-extractor-spec.md](document-extractor-spec.md) for the roadmap.

fastdocparse-0.3.0/README.md

@@ -0,0 +1,130 @@
+# fastdocparse
+
+[![PyPI](https://img.shields.io/pypi/v/fastdocparse.svg)](https://pypi.org/project/fastdocparse/)
+[![CI](https://github.com/pranjalparmar/fastdocparse/actions/workflows/ci.yml/badge.svg)](https://github.com/pranjalparmar/fastdocparse/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
+Extract structured data from semi-structured documents (invoices, bills, tax forms, resumes, bank statements, shipment manifests) using any OpenAI-compatible LLM (OpenAI, Ollama, vLLM, Groq, etc.), with **per-field grounding and confidence**, not just raw extraction.
+
+## Why this, not just another parser
+
+Most extractors give you a value and no way to know if it's real. This one tells you:
+
+- **`grounded`**: the value was found verbatim (or near-verbatim) in the source document text.
+- **`ungrounded`**: the value doesn't appear in the source, likely a hallucination. Flag for human review.
+- **`missing_required`**: a field you marked required came back empty.
+- **`invalid_format`**: the value doesn't match a pattern/enum constraint you declared (e.g. a shipment status outside the allowed list).
+- **`failed_check`**: a custom cross-field rule failed (e.g. line items don't sum to the stated total).
+
+No extra LLM call for any of this: it's deterministic, string/rule-based validation against text you already extracted.
+
+**Where it fits:** semi-structured documents with recurring fields (invoices, bills, tax forms, resumes, statements), and prose documents where *proving* a value came from the source matters (contracts, legal clauses, insurance claims). It is not a vision-LLM pipeline. It works from extracted text (digital PDF text layer, or local OCR for scans/images), which is what keeps it fast, cheap, and usable with small local models. Messy handwritten forms or complex multi-column layouts are a known weaker spot (see [document-extractor-spec.md](document-extractor-spec.md)).
+
+## How this compares
+
+| Project | Approach | Where it beats fastdocparse | Where fastdocparse can beat it |
+|---|---|---|---|
+| **[Sparrow](https://github.com/katanaml/sparrow)** (katanaml) | Vision-LLM first (MLX/vLLM/Ollama/Mistral OCR), multi-service platform, API-first | Layout-aware (sees the page), mature, table templates for complex tables | No GPU required, single `pip install`, Pydantic-schema devex vs. raw JSON-string CLI args |
+| **[LangExtract](https://github.com/google/langextract)** (Google) | Text-first, source-grounding (character-offset mapping), few-shot examples required | Real grounding/traceability, brand trust | Purpose-built for documents (OCR routing) vs. text-in/text-out |
+
+**The honest edge:** *fast, cheap, local-model-friendly extraction for clean-to-moderate documents*, not "better than Sparrow at everything." This has not yet been validated with a real head-to-head benchmark ([tracking issue](https://github.com/pranjalparmar/fastdocparse/issues/19)). Treat it as a design goal, not a proven claim, until that lands.
+
+## Two ways to use it
+
+| | Who it's for | How |
+|---|---|---|
+| **CLI** | No coding needed | `fastdocparse extract <file> <schema.json>` / `fastdocparse --version` |
+| **Python API** | Building it into your own app | `DocumentParser(client).extract(document_bytes, schema)` |
+
+Defining *what* to extract also has two paths: hand-write a JSON/YAML schema file, or describe it in plain English and let the LLM draft the schema for you.
+
+## Install
+
+```bash
+pip install fastdocparse
+```
+
+For local development instead:
+
+```bash
+git clone https://github.com/pranjalparmar/fastdocparse
+cd fastdocparse
+python -m venv venv
+source venv/bin/activate        # Windows: venv\Scripts\activate
+pip install -e ".[dev]"
+```
+
+You also need access to an LLM. Either:
+- An OpenAI API key (`export OPENAI_API_KEY=...` or pass `--api-key`), or
+- A local model via [Ollama](https://ollama.com/): no API key, no cloud, documents never leave your machine.
+
+## Quickstart: CLI (no coding)
+
+```bash
+# 1. Extract using one of the bundled example schemas
+fastdocparse extract sample_invoice.png src/fastdocparse/schemas/invoice.json \
+  --model gpt-4o-mini --api-key sk-...
+
+# Or with a local model via Ollama (no API key needed):
+fastdocparse extract sample_invoice.png src/fastdocparse/schemas/invoice.json \
+  --model llama3.2 --base-url http://localhost:11434/v1 --api-key ollama
+```
+
+Output is JSON, printed to stdout (or saved with `--output result.json`):
+
+```json
+{
+  "_meta": { "truncated": false, "truncation_reason": null },
+  "invoice_number": { "value": "INV-9011", "confidence": "high", "flags": ["grounded"] },
+  "total_price": { "value": 100.0, "confidence": "high", "flags": ["grounded"] }
+}
+```
+
+Don't want to write JSON at all? Describe the fields in plain English instead:
+
+```bash
+fastdocparse schema-from-text \
+  "I want the invoice number, total price, and vendor name. Invoice number and total are required." \
+  --output my_invoice_schema.json
+
+# review my_invoice_schema.json, then:
+fastdocparse extract my_invoice.pdf my_invoice_schema.json
+```
+
+## Quickstart: Python API
+
+```python
+from fastdocparse import Schema, Field, LLMClient, DocumentParser
+
+schema = Schema(
+    name="Invoice",
+    fields=[
+        Field(name="invoice_number", description="The invoice number", required=True),
+        Field(name="total_price", description="Total amount due", type="number", required=True),
+    ],
+)
+
+client = LLMClient(model="gpt-4o-mini", api_key="sk-...")
+# or: LLMClient(base_url="http://localhost:11434/v1", api_key="ollama", model="llama3.2")
+
+parser = DocumentParser(client=client)
+
+with open("invoice.pdf", "rb") as f:
+    result = parser.extract(f.read(), schema)
+
+print(result["invoice_number"])  # {'value': 'INV-9011', 'confidence': 'high', 'flags': ['grounded']}
+```
+
+## Full documentation
+
+- [Getting Started](docs/getting-started.md): step-by-step install, CLI, and API walkthroughs
+- [Schema Guide](docs/schema-guide.md): every field option (`type`, `required`, `pattern`, `enum`, `sub_fields`, few-shot `examples`), for JSON, YAML, and plain-English authoring
+- [Output & Validation](docs/output-format.md): the full result shape, what each confidence flag means, and how to write custom cross-check rules
+- [Architecture](docs/architecture.md): diagrams of the pipeline, the module dependency graph, and where to plug in a contribution
+- [Project spec](document-extractor-spec.md): architecture, phased roadmap, honest competitive positioning
+
+Want to contribute? Start with [docs/architecture.md](docs/architecture.md) for the map, then [CONTRIBUTING.md](CONTRIBUTING.md) for the process.
+
+## Status
+
+Core extraction, grounding, chunking, both CLI/API paths, and real packaging are implemented and tested (75 tests, `pytest -v`). Published on PyPI as [`fastdocparse`](https://pypi.org/project/fastdocparse/). `pip install fastdocparse` installs a working `fastdocparse` command and a proper `fastdocparse.*` import namespace, verified end to end with a clean-virtualenv install straight from the real public index. Not yet done: a hosted API. See [document-extractor-spec.md](document-extractor-spec.md) for the roadmap.

fastdocparse-0.3.0/pyproject.toml

@@ -0,0 +1,66 @@
+[project]
+name = "fastdocparse"
+version = "0.3.0"
+description = "Extract structured data from semi-structured documents using any OpenAI-compatible LLM, with per-field grounding and confidence."
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.9,<3.13"  # capped by rapidocr-onnxruntime's own upper bound (checked live against PyPI)
+keywords = [
+    "document-extraction",
+    "llm",
+    "pdf",
+    "ocr",
+    "invoice-extraction",
+    "structured-data",
+    "openai",
+    "ollama",
+    "grounding",
+]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Operating System :: OS Independent",
+    "Intended Audience :: Developers",
+    "Topic :: Text Processing :: Linguistic",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+dependencies = [
+    "pydantic>=2.0",
+    "openai>=1.0",
+    "pymupdf>=1.24",
+    "pillow>=10.0",
+    "rapidocr-onnxruntime>=1.3",
+    "typer>=0.12",
+    "PyYAML>=6.0",
+]
+
+[project.optional-dependencies]
+dev = ["pytest"]
+
+[project.urls]
+Homepage = "https://github.com/pranjalparmar/fastdocparse"
+Repository = "https://github.com/pranjalparmar/fastdocparse"
+Issues = "https://github.com/pranjalparmar/fastdocparse/issues"
+Documentation = "https://github.com/pranjalparmar/fastdocparse/tree/main/docs"
+
+[project.scripts]
+fastdocparse = "fastdocparse.cli:app"
+
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+fastdocparse = ["schemas/*.json", "py.typed"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[tool.ruff.lint.per-file-ignores]
+"src/fastdocparse/cli.py" = ["B008"]

{fastdocparse-0.1.1/src/docextract → fastdocparse-0.3.0/src/fastdocparse}/__init__.py

@@ -1,7 +1,7 @@
-"""docextract — extract structured data from semi-structured documents using any
+"""fastdocparse — extract structured data from semi-structured documents using any
 OpenAI-compatible LLM, with per-field grounding and confidence.
 
-    from docextract import Schema, Field, LLMClient, DocumentParser
+    from fastdocparse import Schema, Field, LLMClient, DocumentParser
 
     schema = Schema(name="Invoice", fields=[Field(name="total", description="Grand total", type="number")])
     client = LLMClient(model="gpt-4o-mini", api_key="sk-...")
@@ -10,43 +10,55 @@ OpenAI-compatible LLM, with per-field grounding and confidence.
 
 from .cache import Cache, InMemoryCache
 from .config import ExtractionConfig
-from .grounding import Issue, check_substring, cross_check, date_parseable_rule, numeric_sum_rule, validate_field_constraints
+from .grounding import (
+    Issue,
+    check_substring,
+    cross_check,
+    date_parseable_rule,
+    numeric_sum_rule,
+    validate_field_constraints,
+)
 from .llm_client import LLMClient, LLMClientError
-from .parser import DocumentParser, EmptyDocumentError, UnknownIngestionKindError, register_default_ingestion_handler
+from .parser import (
+    DocumentParser,
+    EmptyDocumentError,
+    UnknownIngestionKindError,
+    register_default_ingestion_handler,
+)
 from .result import ExtractionMeta, ExtractionResult, FieldResult
 from .schema import Field, Schema
 from .schema_compiler import compile_schema_from_description
 
 try:
+    from importlib.metadata import PackageNotFoundError
     from importlib.metadata import version as _pkg_version
-    # Looks up by the PyPI *distribution* name, not the import name — these differ
-    # ("docextract" was blocked by PyPI's name-similarity rule against the existing
-    # "doc-extract", so the distribution is registered as "fastdocparse"). Keep this
-    # in sync with pyproject.toml's [project] name if that ever changes again.
+
+    # Looks up by the PyPI *distribution* name (pyproject.toml's [project] name) — keep
+    # this string in sync with that if the distribution is ever renamed again.
     __version__ = _pkg_version("fastdocparse")
-except Exception:
+except PackageNotFoundError:
     __version__ = "0.0.0+unknown"
 
 __all__ = [
-    "Schema",
-    "Field",
-    "LLMClient",
-    "LLMClientError",
+    "Cache",
     "DocumentParser",
     "EmptyDocumentError",
-    "UnknownIngestionKindError",
-    "register_default_ingestion_handler",
     "ExtractionConfig",
-    "Cache",
-    "InMemoryCache",
+    "ExtractionMeta",
     "ExtractionResult",
+    "Field",
     "FieldResult",
-    "ExtractionMeta",
+    "InMemoryCache",
     "Issue",
+    "LLMClient",
+    "LLMClientError",
+    "Schema",
+    "UnknownIngestionKindError",
     "check_substring",
+    "compile_schema_from_description",
     "cross_check",
-    "validate_field_constraints",
-    "numeric_sum_rule",
     "date_parseable_rule",
-    "compile_schema_from_description",
+    "numeric_sum_rule",
+    "register_default_ingestion_handler",
+    "validate_field_constraints",
 ]

{fastdocparse-0.1.1/src/docextract → fastdocparse-0.3.0/src/fastdocparse}/cache.py

@@ -4,19 +4,20 @@ Caching is skipped whenever custom `rules` are passed to extract(), since a rule
 arbitrary callable that can't be safely fingerprinted — caching would risk returning a
 result validated under a different rule than the one just requested.
 """
+from __future__ import annotations
 
 import hashlib
 import json
 from collections import OrderedDict
-from typing import Any, Callable, Dict, Optional, Protocol
+from typing import Any, Callable, Protocol
 
 from .config import ExtractionConfig
 from .schema import Schema
 
 
 class Cache(Protocol):
-    def get(self, key: str) -> Optional[Dict[str, Any]]: ...
-    def set(self, key: str, value: Dict[str, Any]) -> None: ...
+    def get(self, key: str) -> dict[str, Any] | None: ...
+    def set(self, key: str, value: dict[str, Any]) -> None: ...
 
 
 class InMemoryCache:
@@ -27,19 +28,19 @@ class InMemoryCache:
     process exits before the cache could grow unbounded.
     """
 
-    def __init__(self, max_size: Optional[int] = None):
+    def __init__(self, max_size: int | None = None):
         if max_size is not None and max_size <= 0:
             raise ValueError(f"max_size must be positive, got {max_size}")
         self._max_size = max_size
-        self._store: "OrderedDict[str, Dict[str, Any]]" = OrderedDict()
+        self._store: OrderedDict[str, dict[str, Any]] = OrderedDict()
 
-    def get(self, key: str) -> Optional[Dict[str, Any]]:
+    def get(self, key: str) -> dict[str, Any] | None:
         if key not in self._store:
             return None
         self._store.move_to_end(key)
         return self._store[key]
 
-    def set(self, key: str, value: Dict[str, Any]) -> None:
+    def set(self, key: str, value: dict[str, Any]) -> None:
         self._store[key] = value
         self._store.move_to_end(key)
         if self._max_size is not None: