fastdocparse 0.1.0__tar.gz

fastdocparse-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Pranjal Parmar
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

fastdocparse-0.1.0/PKG-INFO

@@ -0,0 +1,132 @@
+Metadata-Version: 2.4
+Name: fastdocparse
+Version: 0.1.0
+Summary: Extract structured data from semi-structured documents using any OpenAI-compatible LLM, with per-field grounding and confidence.
+License-Expression: MIT
+Requires-Python: <3.13,>=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic>=2.0
+Requires-Dist: openai>=1.0
+Requires-Dist: pymupdf>=1.24
+Requires-Dist: pillow>=10.0
+Requires-Dist: rapidocr-onnxruntime>=1.3
+Requires-Dist: typer>=0.12
+Requires-Dist: PyYAML>=6.0
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Dynamic: license-file
+
+# docextract
+
+Extract structured data from semi-structured documents — invoices, bills, tax forms, resumes, bank statements, shipment manifests — using any OpenAI-compatible LLM (OpenAI, Ollama, vLLM, Groq, etc.), with **per-field grounding and confidence**, not just raw extraction.
+
+## Why this, not just another parser
+
+Most extractors give you a value and no way to know if it's real. This one tells you:
+
+- **`grounded`** — the value was found verbatim (or near-verbatim) in the source document text.
+- **`ungrounded`** — the value doesn't appear in the source — likely a hallucination. Flag for human review.
+- **`missing_required`** — a field you marked required came back empty.
+- **`invalid_format`** — the value doesn't match a pattern/enum constraint you declared (e.g. a shipment status outside the allowed list).
+- **`failed_check`** — a custom cross-field rule failed (e.g. line items don't sum to the stated total).
+
+No extra LLM call for any of this — it's deterministic, string/rule-based validation against text you already extracted.
+
+**Where it fits:** semi-structured documents with recurring fields (invoices, bills, tax forms, resumes, statements), and prose documents where *proving* a value came from the source matters (contracts, legal clauses, insurance claims). It is not a vision-LLM pipeline — it works from extracted text (digital PDF text layer, or local OCR for scans/images), which is what keeps it fast, cheap, and usable with small local models. Messy handwritten forms or complex multi-column layouts are a known weaker spot (see [document-extractor-spec.md](document-extractor-spec.md)).
+
+## Two ways to use it
+
+| | Who it's for | How |
+|---|---|---|
+| **CLI** | No coding needed | `docextract extract <file> <schema.json>` |
+| **Python API** | Building it into your own app | `DocumentParser(client).extract(document_bytes, schema)` |
+
+Defining *what* to extract also has two paths — hand-write a JSON/YAML schema file, or describe it in plain English and let the LLM draft the schema for you.
+
+## Install
+
+```bash
+git clone <this repo>
+cd document-extractor
+python -m venv venv
+source venv/bin/activate        # Windows: venv\Scripts\activate
+pip install -e .
+```
+
+(Not yet published to PyPI — see [Status](#status). Until then, install from a local clone as above.)
+
+You also need access to an LLM. Either:
+- An OpenAI API key (`export OPENAI_API_KEY=...` or pass `--api-key`), or
+- A local model via [Ollama](https://ollama.com/) — no API key, no cloud, documents never leave your machine.
+
+## Quickstart — CLI (no coding)
+
+```bash
+# 1. Extract using one of the bundled example schemas
+docextract extract sample_invoice.png src/docextract/schemas/invoice.json \
+  --model gpt-4o-mini --api-key sk-...
+
+# Or with a local model via Ollama (no API key needed):
+docextract extract sample_invoice.png src/docextract/schemas/invoice.json \
+  --model llama3.2 --base-url http://localhost:11434/v1 --api-key ollama
+```
+
+Output is JSON, printed to stdout (or saved with `--output result.json`):
+
+```json
+{
+  "_meta": { "truncated": false, "truncation_reason": null },
+  "invoice_number": { "value": "INV-9011", "confidence": "high", "flags": ["grounded"] },
+  "total_price": { "value": 100.0, "confidence": "high", "flags": ["grounded"] }
+}
+```
+
+Don't want to write JSON at all? Describe the fields in plain English instead:
+
+```bash
+docextract schema-from-text \
+  "I want the invoice number, total price, and vendor name. Invoice number and total are required." \
+  --output my_invoice_schema.json
+
+# review my_invoice_schema.json, then:
+docextract extract my_invoice.pdf my_invoice_schema.json
+```
+
+## Quickstart — Python API
+
+```python
+from docextract import Schema, Field, LLMClient, DocumentParser
+
+schema = Schema(
+    name="Invoice",
+    fields=[
+        Field(name="invoice_number", description="The invoice number", required=True),
+        Field(name="total_price", description="Total amount due", type="number", required=True),
+    ],
+)
+
+client = LLMClient(model="gpt-4o-mini", api_key="sk-...")
+# or: LLMClient(base_url="http://localhost:11434/v1", api_key="ollama", model="llama3.2")
+
+parser = DocumentParser(client=client)
+
+with open("invoice.pdf", "rb") as f:
+    result = parser.extract(f.read(), schema)
+
+print(result["invoice_number"])  # {'value': 'INV-9011', 'confidence': 'high', 'flags': ['grounded']}
+```
+
+## Full documentation
+
+- [Getting Started](docs/getting-started.md) — step-by-step install, CLI, and API walkthroughs
+- [Schema Guide](docs/schema-guide.md) — every field option (`type`, `required`, `pattern`, `enum`, `sub_fields`, few-shot `examples`), for JSON, YAML, and plain-English authoring
+- [Output & Validation](docs/output-format.md) — the full result shape, what each confidence flag means, and how to write custom cross-check rules
+- [Architecture](docs/architecture.md) — diagrams of the pipeline, the module dependency graph, and where to plug in a contribution
+- [Project spec](document-extractor-spec.md) — architecture, phased roadmap, honest competitive positioning
+
+Want to contribute? Start with [docs/architecture.md](docs/architecture.md) for the map, then [CONTRIBUTING.md](CONTRIBUTING.md) for the process.
+
+## Status
+
+Core extraction, grounding, chunking, both CLI/API paths, and real packaging (`pip install -e .` installs a working `docextract` command and a proper `docextract.*` import namespace — verified with a from-scratch build and a clean-virtualenv install) are implemented and tested (74 tests, `pytest -v`). Not yet done: actually publishing to PyPI and a hosted API — see [document-extractor-spec.md](document-extractor-spec.md) for the roadmap.

fastdocparse-0.1.0/README.md

@@ -0,0 +1,113 @@
+# docextract
+
+Extract structured data from semi-structured documents — invoices, bills, tax forms, resumes, bank statements, shipment manifests — using any OpenAI-compatible LLM (OpenAI, Ollama, vLLM, Groq, etc.), with **per-field grounding and confidence**, not just raw extraction.
+
+## Why this, not just another parser
+
+Most extractors give you a value and no way to know if it's real. This one tells you:
+
+- **`grounded`** — the value was found verbatim (or near-verbatim) in the source document text.
+- **`ungrounded`** — the value doesn't appear in the source — likely a hallucination. Flag for human review.
+- **`missing_required`** — a field you marked required came back empty.
+- **`invalid_format`** — the value doesn't match a pattern/enum constraint you declared (e.g. a shipment status outside the allowed list).
+- **`failed_check`** — a custom cross-field rule failed (e.g. line items don't sum to the stated total).
+
+No extra LLM call for any of this — it's deterministic, string/rule-based validation against text you already extracted.
+
+**Where it fits:** semi-structured documents with recurring fields (invoices, bills, tax forms, resumes, statements), and prose documents where *proving* a value came from the source matters (contracts, legal clauses, insurance claims). It is not a vision-LLM pipeline — it works from extracted text (digital PDF text layer, or local OCR for scans/images), which is what keeps it fast, cheap, and usable with small local models. Messy handwritten forms or complex multi-column layouts are a known weaker spot (see [document-extractor-spec.md](document-extractor-spec.md)).
+
+## Two ways to use it
+
+| | Who it's for | How |
+|---|---|---|
+| **CLI** | No coding needed | `docextract extract <file> <schema.json>` |
+| **Python API** | Building it into your own app | `DocumentParser(client).extract(document_bytes, schema)` |
+
+Defining *what* to extract also has two paths — hand-write a JSON/YAML schema file, or describe it in plain English and let the LLM draft the schema for you.
+
+## Install
+
+```bash
+git clone <this repo>
+cd document-extractor
+python -m venv venv
+source venv/bin/activate        # Windows: venv\Scripts\activate
+pip install -e .
+```
+
+(Not yet published to PyPI — see [Status](#status). Until then, install from a local clone as above.)
+
+You also need access to an LLM. Either:
+- An OpenAI API key (`export OPENAI_API_KEY=...` or pass `--api-key`), or
+- A local model via [Ollama](https://ollama.com/) — no API key, no cloud, documents never leave your machine.
+
+## Quickstart — CLI (no coding)
+
+```bash
+# 1. Extract using one of the bundled example schemas
+docextract extract sample_invoice.png src/docextract/schemas/invoice.json \
+  --model gpt-4o-mini --api-key sk-...
+
+# Or with a local model via Ollama (no API key needed):
+docextract extract sample_invoice.png src/docextract/schemas/invoice.json \
+  --model llama3.2 --base-url http://localhost:11434/v1 --api-key ollama
+```
+
+Output is JSON, printed to stdout (or saved with `--output result.json`):
+
+```json
+{
+  "_meta": { "truncated": false, "truncation_reason": null },
+  "invoice_number": { "value": "INV-9011", "confidence": "high", "flags": ["grounded"] },
+  "total_price": { "value": 100.0, "confidence": "high", "flags": ["grounded"] }
+}
+```
+
+Don't want to write JSON at all? Describe the fields in plain English instead:
+
+```bash
+docextract schema-from-text \
+  "I want the invoice number, total price, and vendor name. Invoice number and total are required." \
+  --output my_invoice_schema.json
+
+# review my_invoice_schema.json, then:
+docextract extract my_invoice.pdf my_invoice_schema.json
+```
+
+## Quickstart — Python API
+
+```python
+from docextract import Schema, Field, LLMClient, DocumentParser
+
+schema = Schema(
+    name="Invoice",
+    fields=[
+        Field(name="invoice_number", description="The invoice number", required=True),
+        Field(name="total_price", description="Total amount due", type="number", required=True),
+    ],
+)
+
+client = LLMClient(model="gpt-4o-mini", api_key="sk-...")
+# or: LLMClient(base_url="http://localhost:11434/v1", api_key="ollama", model="llama3.2")
+
+parser = DocumentParser(client=client)
+
+with open("invoice.pdf", "rb") as f:
+    result = parser.extract(f.read(), schema)
+
+print(result["invoice_number"])  # {'value': 'INV-9011', 'confidence': 'high', 'flags': ['grounded']}
+```
+
+## Full documentation
+
+- [Getting Started](docs/getting-started.md) — step-by-step install, CLI, and API walkthroughs
+- [Schema Guide](docs/schema-guide.md) — every field option (`type`, `required`, `pattern`, `enum`, `sub_fields`, few-shot `examples`), for JSON, YAML, and plain-English authoring
+- [Output & Validation](docs/output-format.md) — the full result shape, what each confidence flag means, and how to write custom cross-check rules
+- [Architecture](docs/architecture.md) — diagrams of the pipeline, the module dependency graph, and where to plug in a contribution
+- [Project spec](document-extractor-spec.md) — architecture, phased roadmap, honest competitive positioning
+
+Want to contribute? Start with [docs/architecture.md](docs/architecture.md) for the map, then [CONTRIBUTING.md](CONTRIBUTING.md) for the process.
+
+## Status
+
+Core extraction, grounding, chunking, both CLI/API paths, and real packaging (`pip install -e .` installs a working `docextract` command and a proper `docextract.*` import namespace — verified with a from-scratch build and a clean-virtualenv install) are implemented and tested (74 tests, `pytest -v`). Not yet done: actually publishing to PyPI and a hosted API — see [document-extractor-spec.md](document-extractor-spec.md) for the roadmap.

fastdocparse-0.1.0/pyproject.toml

@@ -0,0 +1,35 @@
+[project]
+name = "fastdocparse"
+version = "0.1.0"
+description = "Extract structured data from semi-structured documents using any OpenAI-compatible LLM, with per-field grounding and confidence."
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.9,<3.13"  # capped by rapidocr-onnxruntime's own upper bound (checked live against PyPI)
+dependencies = [
+    "pydantic>=2.0",
+    "openai>=1.0",
+    "pymupdf>=1.24",
+    "pillow>=10.0",
+    "rapidocr-onnxruntime>=1.3",
+    "typer>=0.12",
+    "PyYAML>=6.0",
+]
+
+[project.optional-dependencies]
+dev = ["pytest"]
+
+[project.scripts]
+docextract = "docextract.cli:app"
+
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+docextract = ["schemas/*.json", "py.typed"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

fastdocparse-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

fastdocparse-0.1.0/src/docextract/__init__.py

@@ -0,0 +1,48 @@
+"""docextract — extract structured data from semi-structured documents using any
+OpenAI-compatible LLM, with per-field grounding and confidence.
+
+    from docextract import Schema, Field, LLMClient, DocumentParser
+
+    schema = Schema(name="Invoice", fields=[Field(name="total", description="Grand total", type="number")])
+    client = LLMClient(model="gpt-4o-mini", api_key="sk-...")
+    result = DocumentParser(client).extract(document_bytes, schema)
+"""
+
+from .cache import Cache, InMemoryCache
+from .config import ExtractionConfig
+from .grounding import Issue, check_substring, cross_check, date_parseable_rule, numeric_sum_rule, validate_field_constraints
+from .llm_client import LLMClient, LLMClientError
+from .parser import DocumentParser, EmptyDocumentError, UnknownIngestionKindError, register_default_ingestion_handler
+from .result import ExtractionMeta, ExtractionResult, FieldResult
+from .schema import Field, Schema
+from .schema_compiler import compile_schema_from_description
+
+try:
+    from importlib.metadata import version as _pkg_version
+    __version__ = _pkg_version("docextract")
+except Exception:
+    __version__ = "0.0.0+unknown"
+
+__all__ = [
+    "Schema",
+    "Field",
+    "LLMClient",
+    "LLMClientError",
+    "DocumentParser",
+    "EmptyDocumentError",
+    "UnknownIngestionKindError",
+    "register_default_ingestion_handler",
+    "ExtractionConfig",
+    "Cache",
+    "InMemoryCache",
+    "ExtractionResult",
+    "FieldResult",
+    "ExtractionMeta",
+    "Issue",
+    "check_substring",
+    "cross_check",
+    "validate_field_constraints",
+    "numeric_sum_rule",
+    "date_parseable_rule",
+    "compile_schema_from_description",
+]

fastdocparse-0.1.0/src/docextract/cache.py

@@ -0,0 +1,72 @@
+"""Pluggable caching for DocumentParser.extract(). Off by default — pass a cache instance to opt in.
+
+Caching is skipped whenever custom `rules` are passed to extract(), since a rule is an
+arbitrary callable that can't be safely fingerprinted — caching would risk returning a
+result validated under a different rule than the one just requested.
+"""
+
+import hashlib
+import json
+from collections import OrderedDict
+from typing import Any, Callable, Dict, Optional, Protocol
+
+from .config import ExtractionConfig
+from .schema import Schema
+
+
+class Cache(Protocol):
+    def get(self, key: str) -> Optional[Dict[str, Any]]: ...
+    def set(self, key: str, value: Dict[str, Any]) -> None: ...
+
+
+class InMemoryCache:
+    """Process-local cache. Good for a long-running batch job or server process; lost on restart.
+
+    Pass max_size to cap memory use — oldest-accessed entries are evicted once the cap is
+    hit (LRU). Leave it None (default) only for short-lived scripts/batch jobs where the
+    process exits before the cache could grow unbounded.
+    """
+
+    def __init__(self, max_size: Optional[int] = None):
+        if max_size is not None and max_size <= 0:
+            raise ValueError(f"max_size must be positive, got {max_size}")
+        self._max_size = max_size
+        self._store: "OrderedDict[str, Dict[str, Any]]" = OrderedDict()
+
+    def get(self, key: str) -> Optional[Dict[str, Any]]:
+        if key not in self._store:
+            return None
+        self._store.move_to_end(key)
+        return self._store[key]
+
+    def set(self, key: str, value: Dict[str, Any]) -> None:
+        self._store[key] = value
+        self._store.move_to_end(key)
+        if self._max_size is not None:
+            while len(self._store) > self._max_size:
+                self._store.popitem(last=False)
+
+    def __len__(self) -> int:
+        return len(self._store)
+
+
+def make_cache_key(
+    document_bytes: bytes,
+    schema: Schema,
+    kind: str,
+    config: ExtractionConfig,
+    handler: Callable[..., str],
+) -> str:
+    """Fingerprint a (document, schema, ingestion kind, config, handler) combination.
+
+    handler is included, not just kind, because handlers are registered per
+    DocumentParser instance — two instances can register different handlers under the
+    same kind name, and a cache shared between them must not conflate the two.
+    Identity is process-local (qualified name + id()), which matches InMemoryCache's own
+    process-local lifetime — it isn't meant to survive a restart anyway.
+    """
+    document_digest = hashlib.sha256(document_bytes).hexdigest()
+    schema_digest = hashlib.sha256(schema.model_dump_json().encode()).hexdigest()
+    config_digest = hashlib.sha256(json.dumps(config.__dict__, sort_keys=True).encode()).hexdigest()
+    handler_identity = f"{getattr(handler, '__module__', '?')}.{getattr(handler, '__qualname__', repr(handler))}:{id(handler)}"
+    return hashlib.sha256(f"{document_digest}:{schema_digest}:{kind}:{config_digest}:{handler_identity}".encode()).hexdigest()

fastdocparse-0.1.0/src/docextract/cli.py

@@ -0,0 +1,140 @@
+"""Command-line entrypoint: extract fields from a document without writing code.
+
+Usage:
+    docextract extract document.pdf invoice_schema.json
+    docextract extract receipt.jpg shipment_schema.json --model llama3 --base-url http://localhost:11434/v1
+"""
+
+import importlib
+import json
+import os
+from pathlib import Path
+from typing import Optional
+
+import typer
+from pydantic import ValidationError
+
+from .llm_client import LLMClient, LLMClientError
+from .parser import DocumentParser, EmptyDocumentError, UnknownIngestionKindError
+from .schema import Schema
+from .schema_compiler import compile_schema_from_description
+
+
+def _load_plugins() -> None:
+    """Import modules listed in DOCEXTRACT_PLUGINS (comma-separated) so they can call
+    parser.register_default_ingestion_handler() on import — the only way a custom
+    ingestion kind (DOCX, XLSX, ...) becomes reachable from this CLI, since a fresh CLI
+    process otherwise only knows the built-in "pdf"/"image" handlers.
+
+    Security note: this imports and runs arbitrary Python from wherever DOCEXTRACT_PLUGINS
+    points, at CLI startup, with no sandboxing — the same trust model as PYTHONSTARTUP or
+    DJANGO_SETTINGS_MODULE. That's fine for a user pointing it at their own plugin on their
+    own machine, which is the only supported use. Never let this env var be set from an
+    untrusted source (e.g. a request parameter in a hosted service built on this CLI).
+    """
+    plugin_spec = os.environ.get("DOCEXTRACT_PLUGINS", "")
+    for module_name in filter(None, (p.strip() for p in plugin_spec.split(","))):
+        importlib.import_module(module_name)
+
+
+_load_plugins()
+
+app = typer.Typer(
+    add_completion=False,
+    help="Extract structured data from a PDF/PNG/JPG using a schema file you define — no coding required.",
+)
+
+IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg"}
+SUPPORTED_EXTENSIONS = IMAGE_EXTENSIONS | {".pdf"}
+
+
+@app.command()
+def extract(
+    file: Path = typer.Argument(..., exists=True, readable=True, help="Path to the PDF/PNG/JPG document."),
+    schema: Path = typer.Argument(..., exists=True, readable=True, help="Path to a .json or .yaml schema file listing the fields to extract."),
+    model: str = typer.Option("gpt-4o-mini", "--model", "-m", help="Model name, e.g. gpt-4o-mini, llama3."),
+    base_url: Optional[str] = typer.Option(None, "--base-url", help="OpenAI-compatible API base URL. Omit for OpenAI; use e.g. http://localhost:11434/v1 for Ollama."),
+    api_key: Optional[str] = typer.Option(None, "--api-key", envvar="LLM_API_KEY", help="API key. Not needed for local Ollama."),
+    output: Optional[Path] = typer.Option(None, "--output", "-o", help="Write the JSON result to this file instead of printing it."),
+    kind: Optional[str] = typer.Option(None, "--kind", help="Override ingestion routing (e.g. 'docx' for a custom handler loaded via DOCEXTRACT_PLUGINS). Defaults to auto-detecting pdf/image from the file extension."),
+):
+    """Extract the fields defined in SCHEMA from FILE and print the result as JSON."""
+    if kind is None and file.suffix.lower() not in SUPPORTED_EXTENSIONS:
+        typer.echo(f"Unsupported file type {file.suffix!r}. Supported: .pdf, .png, .jpg, .jpeg (or pass --kind).", err=True)
+        raise typer.Exit(code=1)
+
+    try:
+        doc_schema = Schema.from_file(schema)
+    except (ValidationError, ValueError, OSError) as e:
+        typer.echo(f"Could not load schema from {schema}: {e}", err=True)
+        raise typer.Exit(code=1)
+
+    client = LLMClient(base_url=base_url, api_key=api_key, model=model)
+    document_parser = DocumentParser(client=client)
+
+    is_image = file.suffix.lower() in IMAGE_EXTENSIONS
+    document_bytes = file.read_bytes()
+
+    try:
+        result = document_parser.extract(document_bytes, doc_schema, is_image=is_image, kind=kind)
+    except EmptyDocumentError as e:
+        typer.echo(f"Extraction failed: {e}", err=True)
+        raise typer.Exit(code=1)
+    except LLMClientError as e:
+        typer.echo(f"Could not complete extraction: {e}", err=True)
+        raise typer.Exit(code=1)
+    except UnknownIngestionKindError as e:
+        typer.echo(f"{e} (check --kind is spelled correctly and its plugin is loaded via DOCEXTRACT_PLUGINS)", err=True)
+        raise typer.Exit(code=1)
+    except ValueError as e:
+        typer.echo(f"Extraction failed: {e}", err=True)
+        raise typer.Exit(code=1)
+
+    result_json = json.dumps(result, indent=2, default=str)
+
+    if output:
+        try:
+            output.parent.mkdir(parents=True, exist_ok=True)
+            output.write_text(result_json)
+        except OSError as e:
+            typer.echo(f"Could not write output to {output}: {e}", err=True)
+            raise typer.Exit(code=1)
+        typer.echo(f"Wrote result to {output}")
+    else:
+        typer.echo(result_json)
+
+
+@app.command(name="schema-from-text")
+def schema_from_text(
+    description: str = typer.Argument(..., help="Plain-English description of the fields you want extracted, e.g. \"invoice number, total price, and a list of line items with product name and quantity\"."),
+    output: Path = typer.Option(..., "--output", "-o", help="Where to save the generated schema (.json)."),
+    model: str = typer.Option("gpt-4o-mini", "--model", "-m", help="Model name, e.g. gpt-4o-mini, llama3."),
+    base_url: Optional[str] = typer.Option(None, "--base-url", help="OpenAI-compatible API base URL. Omit for OpenAI; use e.g. http://localhost:11434/v1 for Ollama."),
+    api_key: Optional[str] = typer.Option(None, "--api-key", envvar="LLM_API_KEY", help="API key. Not needed for local Ollama."),
+):
+    """Turn a plain-English description of the fields you want into a schema file.
+
+    Review the generated file before using it with `extract` — the LLM proposes field
+    names and types from your description, and a wrong guess here affects every
+    document you later run against this schema.
+    """
+    client = LLMClient(base_url=base_url, api_key=api_key, model=model)
+    try:
+        doc_schema = compile_schema_from_description(description, client)
+    except (ValueError, LLMClientError) as e:
+        typer.echo(f"Could not generate a schema: {e}", err=True)
+        raise typer.Exit(code=1)
+
+    try:
+        output.parent.mkdir(parents=True, exist_ok=True)
+        output.write_text(json.dumps(doc_schema.model_dump(), indent=2))
+    except OSError as e:
+        typer.echo(f"Could not write schema to {output}: {e}", err=True)
+        raise typer.Exit(code=1)
+
+    typer.echo(f"Saved schema '{doc_schema.name}' with {len(doc_schema.fields)} field(s) to {output}")
+    typer.echo("Review it, then run: docextract extract <your_document> " + str(output))
+
+
+if __name__ == "__main__":
+    app()

fastdocparse-0.1.0/src/docextract/config.py

@@ -0,0 +1,38 @@
+"""Tunable extraction parameters, previously hardcoded inline in parser.py."""
+
+from dataclasses import dataclass
+
+
+@dataclass(frozen=True)
+class ExtractionConfig:
+    """Knobs for a DocumentParser run. Override per-instance instead of editing source."""
+
+    # The single page cap: ingestion reads (and chunks/merges across) up to this
+    # many pages. A document with more pages than this is flagged truncated —
+    # there used to be two separate, disconnected limits here (a smaller one
+    # actually used for ingestion, a larger one only used for the truncation
+    # flag), which meant a document could silently lose pages while still being
+    # reported as "not truncated." One limit now, so the flag is honest.
+    max_pages: int = 15
+    chunk_max_tokens: int = 3000
+    pdf_render_dpi: int = 150
+    max_image_dim: int = 1536
+    ocr_min_confidence: float = 0.3
+    # Chunks are processed sequentially by default (1) — safe, deterministic order.
+    # Raise this to fan LLM calls for independent chunks out over threads; chunks are
+    # I/O-bound network calls, so a plain ThreadPoolExecutor is enough (no asyncio needed).
+    max_concurrent_chunks: int = 1
+
+    def __post_init__(self):
+        if self.max_pages <= 0:
+            raise ValueError(f"max_pages must be positive, got {self.max_pages}")
+        if self.chunk_max_tokens <= 0:
+            raise ValueError(f"chunk_max_tokens must be positive, got {self.chunk_max_tokens}")
+        if self.pdf_render_dpi <= 0:
+            raise ValueError(f"pdf_render_dpi must be positive, got {self.pdf_render_dpi}")
+        if self.max_image_dim <= 0:
+            raise ValueError(f"max_image_dim must be positive, got {self.max_image_dim}")
+        if not 0.0 <= self.ocr_min_confidence <= 1.0:
+            raise ValueError(f"ocr_min_confidence must be between 0 and 1, got {self.ocr_min_confidence}")
+        if self.max_concurrent_chunks <= 0:
+            raise ValueError(f"max_concurrent_chunks must be positive, got {self.max_concurrent_chunks}")

fastdocparse-0.1.0/src/docextract/example_schemas.py

@@ -0,0 +1,13 @@
+"""Pre-built schemas with few-shot examples for common document types.
+
+Loaded from schemas/*.json rather than declared twice — schemas/invoice.json is the
+single source of truth; editing it updates both the CLI template and this import.
+"""
+
+from pathlib import Path
+
+from .schema import Schema
+
+_SCHEMAS_DIR = Path(__file__).parent / "schemas"
+
+INVOICE_SCHEMA = Schema.from_file(_SCHEMAS_DIR / "invoice.json")