natural-pdf 0.1.7__tar.gz → 0.1.9__tar.gz

natural_pdf-0.1.9/.cursor/rules/analysis_framework.mdc

@@ -0,0 +1,58 @@
+---
+description: 
+globs: 
+alwaysApply: false
+---
+\
+# Analysis Framework Guide
+
+This document outlines the agreed-upon structure for adding and managing machine learning analysis results (like classification, NER, summarization) on `Page` and `Region` objects within the `natural-pdf` library.
+
+## Summary of Framework (Implemented for Classification)
+
+1.  **Central Registry (`element.analyses`)**:
+    *   A dictionary attribute named `analyses` exists on [natural_pdf/core/page.py](mdc:natural_pdf/core/page.py) (`Page`) and [natural_pdf/elements/region.py](mdc:natural_pdf/elements/region.py) (`Region`) objects.
+    *   It stores results from different analysis types, keyed by an `analysis_key` string.
+    *   Example: `page.analyses = {'classification': <ClassificationResult...>, 'ner_run_1': <NERResult...>}`
+
+2.  **Structured Result Objects**:
+    *   Each analysis type should have a dedicated `Result` class (e.g., [natural_pdf/classification/results.py](mdc:natural_pdf/classification/results.py) contains `ClassificationResult`, `CategoryScore`).
+    *   These objects store structured findings and metadata (model used, parameters, timestamp, `using` mode).
+    *   Stored as values in the `analyses` dictionary.
+
+3.  **Manager Registry (`PDF.get_manager`)**:
+    *   The [natural_pdf/core/pdf.py](mdc:natural_pdf/core/pdf.py) `PDF` class has a `get_manager(manager_type)` method.
+    *   This handles lazy initialization and retrieval of specific analysis managers (e.g., `ClassificationManager` from [natural_pdf/classification/manager.py](mdc:natural_pdf/classification/manager.py)).
+    *   Managers encapsulate ML model interaction and result processing.
+
+4.  **Invocation Methods (`element.classify`, etc.)**:
+    *   Methods are added to `Page`/`Region` (often via Mixins like [natural_pdf/classification/mixin.py](mdc:natural_pdf/classification/mixin.py)).
+    *   They accept an optional `analysis_key: str` parameter.
+        *   **Default:** If omitted, uses a standard key (e.g., `'classification'`) and *overwrites* previous results under that key.
+        *   **Custom:** If provided, stores the result under the custom key, allowing multiple results of the same type to coexist.
+    *   They use the `PDF.get_manager` to get the appropriate manager, call it, and store the returned `Result` object in `element.analyses[analysis_key]`.
+
+5.  **Parameter Renaming**:
+    *   The parameter specifying text vs. vision analysis has been standardized to `using=` (e.g., `using='text'`, `using='vision'`).
+
+6.  **Convenience Accessors**:
+    *   Simple properties/methods (e.g., `element.category`, `element.category_confidence`) provide easy access to results.
+    *   These *always* read from the **default** key in the `analyses` registry (e.g., `analyses['classification']`).
+
+## TODO List for New Analysis Features
+
+*   [ ] **NER**: Create `NERManager`, `NERResult`, `Entity`, `element.apply_ner()`, `element.entities` property. Implement optional `source_elements` mapping.
+*   [ ] **Summarization**: Create `SummarizationManager`, `SummarizationResult`, `element.summarize()`, `element.summary` property.
+*   [ ] **Translation**: Create `TranslationManager`, `TranslationResult`, `element.translate()`, `element.translated_text()` method.
+*   [ ] **Structured Data Extraction**: Create `StructuredDataManager`, `StructuredDataResult`, `element.extract_structured_data()`.
+*   [ ] **Ad-hoc Analysis**: Implement `element.run_custom_analysis()` or similar.
+*   [ ] **Documentation**: Update user docs for the framework.
+
+## Coding Conventions for New Analyses
+
+1.  **Manager**: New `Manager` class in `natural_pdf/<task>/manager.py`. Handles ML logic.
+2.  **Registration**: Update `PDF.get_manager` to initialize the new manager. Check `is_available()`.
+3.  **Result Object**: New `Result` class(es) in `natural_pdf/<task>/results.py`. Stores params and findings.
+4.  **Element Method**: Add method to `Page`/`Region` (via Mixin?). Must take `analysis_key` (defaulting to standard task name). Calls manager, stores result in `analyses[analysis_key]`.
+5.  **Accessor**: Add convenience property/method accessing `analyses[DEFAULT_KEY]`.
+6.  **Dependencies**: Use `try...except ImportError` and extras in `pyproject.toml`.

natural_pdf-0.1.9/.cursor/rules/coding-style.mdc

@@ -0,0 +1,5 @@
+---
+description:
+globs:
+alwaysApply: false
+---

natural_pdf-0.1.9/.cursor/rules/edit-md-instead-of-ipynb.mdc

@@ -0,0 +1,5 @@
+---
+description:
+globs:
+alwaysApply: false
+---

natural_pdf-0.1.9/.cursor/rules/minimal-comments.mdc

@@ -0,0 +1,5 @@
+---
+description:
+globs:
+alwaysApply: false
+---

natural_pdf-0.1.9/.cursor/rules/natural-pdf-overview.mdc

@@ -0,0 +1,5 @@
+---
+description:
+globs:
+alwaysApply: false
+---

natural_pdf-0.1.9/.cursor/rules/user-friendly-library-code.mdc

@@ -0,0 +1,5 @@
+---
+description:
+globs:
+alwaysApply: false
+---

{natural_pdf-0.1.7 → natural_pdf-0.1.9}/.gitignore

@@ -13,6 +13,7 @@ docs/tutorials/needs-ocr-searchable.pdf
 sample.py
 sample2.py
 requirements.lock
+hidden
 pdfs/hidden
 *.hocr
 my_paddleocr_finetune_data/

natural_pdf-0.1.7/execute_notebooks.py → natural_pdf-0.1.9/01-execute_notebooks.py

@@ -28,6 +28,8 @@ EXCLUDE_PATTERNS = [
     "explanations",
     "api/index.md",
     "finetuning/index.md",
+    "categorizing-documents/index.md",
+    "data-extraction/index.md",
 ]
 MAX_WORKERS = os.cpu_count()
 

natural_pdf-0.1.9/MANIFEST.in

@@ -0,0 +1,48 @@
+include README.md
+include LICENSE
+
+# HTML templates
+recursive-include natural_pdf/templates *.html
+
+# Documentation assets
+recursive-include docs *.md *.png *.jpg *.gif
+
+# Remove common build garbage
+global-exclude __pycache__ *.py[cod] *.so .DS_Store
+global-exclude *hidden*
+
+# 💣 Critical: prevent recursion bugs
+prune build
+prune dist
+prune .nox
+prune .venv
+prune env
+prune venv
+
+# General junk
+exclude .notebook_cache.json
+exclude Untitled.ipynb
+exclude conversation.md
+exclude transcript.md
+exclude sample.py
+exclude sample2.py
+exclude requirements.lock
+exclude install.sh
+
+# Directories to exclude
+prune .venv
+prune output
+prune results
+prune natural_pdf_index
+prune hidden
+prune pdfs/hidden
+prune my_paddleocr_finetune_data
+prune notebooks
+prune docs/tutorials/pdfs
+
+# Individual files in nested directories
+exclude docs/tutorials/needs-ocr-searchable.pdf
+exclude notebooks/Examples.md
+
+# File patterns
+global-exclude *.hocr

{natural_pdf-0.1.7 → natural_pdf-0.1.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: natural-pdf
-Version: 0.1.7
+Version: 0.1.9
 Summary: A more intuitive interface for working with PDFs
 Author-email: Jonathan Soma <jonathan.soma@gmail.com>
 License-Expression: MIT
@@ -17,11 +17,13 @@ Requires-Dist: colour
 Requires-Dist: numpy
 Requires-Dist: urllib3
 Requires-Dist: tqdm
+Requires-Dist: pydantic
 Provides-Extra: interactive
 Requires-Dist: ipywidgets<9.0.0,>=7.0.0; extra == "interactive"
 Provides-Extra: haystack
 Requires-Dist: haystack-ai; extra == "haystack"
-Requires-Dist: chroma-haystack; extra == "haystack"
+Requires-Dist: lancedb-haystack; extra == "haystack"
+Requires-Dist: lancedb; extra == "haystack"
 Requires-Dist: sentence-transformers; extra == "haystack"
 Requires-Dist: natural-pdf[core-ml]; extra == "haystack"
 Provides-Extra: easyocr
@@ -36,6 +38,9 @@ Requires-Dist: natural-pdf[core-ml]; extra == "layout-yolo"
 Provides-Extra: surya
 Requires-Dist: surya-ocr; extra == "surya"
 Requires-Dist: natural-pdf[core-ml]; extra == "surya"
+Provides-Extra: doctr
+Requires-Dist: python-doctr[torch]; extra == "doctr"
+Requires-Dist: natural-pdf[core-ml]; extra == "doctr"
 Provides-Extra: qa
 Requires-Dist: natural-pdf[core-ml]; extra == "qa"
 Provides-Extra: docling
@@ -43,7 +48,10 @@ Requires-Dist: docling; extra == "docling"
 Requires-Dist: natural-pdf[core-ml]; extra == "docling"
 Provides-Extra: llm
 Requires-Dist: openai>=1.0; extra == "llm"
-Requires-Dist: pydantic; extra == "llm"
+Provides-Extra: classification
+Requires-Dist: sentence-transformers; extra == "classification"
+Requires-Dist: timm; extra == "classification"
+Requires-Dist: natural-pdf[core-ml]; extra == "classification"
 Provides-Extra: test
 Requires-Dist: pytest; extra == "test"
 Provides-Extra: dev
@@ -59,6 +67,9 @@ Requires-Dist: pipdeptree; extra == "dev"
 Requires-Dist: nbformat; extra == "dev"
 Requires-Dist: jupytext; extra == "dev"
 Requires-Dist: nbclient; extra == "dev"
+Provides-Extra: deskew
+Requires-Dist: deskew>=1.5; extra == "deskew"
+Requires-Dist: img2pdf; extra == "deskew"
 Provides-Extra: all
 Requires-Dist: natural-pdf[interactive]; extra == "all"
 Requires-Dist: natural-pdf[haystack]; extra == "all"
@@ -66,10 +77,13 @@ Requires-Dist: natural-pdf[easyocr]; extra == "all"
 Requires-Dist: natural-pdf[paddle]; extra == "all"
 Requires-Dist: natural-pdf[layout_yolo]; extra == "all"
 Requires-Dist: natural-pdf[surya]; extra == "all"
+Requires-Dist: natural-pdf[doctr]; extra == "all"
 Requires-Dist: natural-pdf[qa]; extra == "all"
 Requires-Dist: natural-pdf[ocr-export]; extra == "all"
 Requires-Dist: natural-pdf[docling]; extra == "all"
 Requires-Dist: natural-pdf[llm]; extra == "all"
+Requires-Dist: natural-pdf[classification]; extra == "all"
+Requires-Dist: natural-pdf[deskew]; extra == "all"
 Requires-Dist: natural-pdf[test]; extra == "all"
 Provides-Extra: core-ml
 Requires-Dist: torch; extra == "core-ml"

natural_pdf-0.1.9/audit_packaging.py

@@ -0,0 +1,56 @@
+import subprocess
+import tarfile
+import zipfile
+from pathlib import Path
+
+DIST_DIR = Path("dist")
+
+
+def build_package():
+    subprocess.run(["python", "-m", "build", "--sdist", "--wheel"], check=True)
+
+
+def get_sdist_files():
+    sdist_path = next(DIST_DIR.glob("*.tar.gz"))
+    with tarfile.open(sdist_path, "r:gz") as tar:
+        return sorted(str(Path(m.name)) for m in tar.getmembers() if m.isfile())
+
+
+def get_wheel_files():
+    wheel_path = next(DIST_DIR.glob("*.whl"))
+    with zipfile.ZipFile(wheel_path, "r") as zipf:
+        return sorted(str(f) for f in zipf.namelist() if not f.endswith("/"))
+
+
+def get_gitignored_files():
+    proc = subprocess.run(
+        ["git", "ls-files", "--others", "-i", "--exclude-standard"],
+        check=True,
+        capture_output=True,
+        text=True,
+    )
+    return sorted(proc.stdout.strip().splitlines())
+
+
+def diff_lists(packaged, ignored):
+    return sorted(set(packaged) & set(ignored))
+
+
+def main():
+    build_package()
+
+    sdist_files = get_sdist_files()
+    wheel_files = get_wheel_files()
+    ignored_files = get_gitignored_files()
+
+    print("\n🚫 Files in *sdist* that are also .gitignored:")
+    for f in diff_lists(sdist_files, ignored_files):
+        print("  •", f)
+
+    print("\n🚫 Files in *wheel* that are also .gitignored:")
+    for f in diff_lists(wheel_files, ignored_files):
+        print("  •", f)
+
+
+if __name__ == "__main__":
+    main()

{natural_pdf-0.1.7 → natural_pdf-0.1.9}/check_run_md.sh

@@ -9,12 +9,26 @@ fi
 
 MARKDOWN_FILE=$1
 NOTEBOOK_FILE="${MARKDOWN_FILE%.md}.ipynb"
+KERNEL_NAME="natural-pdf"
 
 echo "Converting $MARKDOWN_FILE to notebook..."
 # Jupytext will now automatically add tags based on markdown metadata
 jupytext --to ipynb "$MARKDOWN_FILE" || { echo "Conversion failed"; exit 1; }
 
+echo "Patching notebook $NOTEBOOK_FILE with kernel $KERNEL_NAME..."
+python3 - <<EOF
+import nbformat
+nb = nbformat.read("$NOTEBOOK_FILE", as_version=4)
+nb.metadata["kernelspec"] = {
+    "name": "$KERNEL_NAME",
+    "display_name": "Python ($KERNEL_NAME)",
+    "language": "python"
+}
+nbformat.write(nb, "$NOTEBOOK_FILE")
+EOF
+
+
 echo "Executing notebook $NOTEBOOK_FILE..."
-jupyter execute "$NOTEBOOK_FILE" --inplace || { echo "Execution failed"; exit 1; }
+jupyter execute "$NOTEBOOK_FILE" --inplace --ExecutePreprocessor.kernel_name=natural-pdf  || { echo "Execution failed"; exit 1; }
 
 echo "Success! Notebook executed and results saved to $NOTEBOOK_FILE"

natural_pdf-0.1.9/docs/categorizing-documents/index.md

@@ -0,0 +1,168 @@
+# Categorizing Pages and Regions
+
+Natural PDF allows you to automatically categorize pages or specific regions within a page using machine learning models. This is incredibly useful for filtering large collections of documents or understanding the structure and content of individual PDFs.
+
+## Installation
+
+To use the classification features, you need to install the optional dependencies:
+
+```bash
+pip install "natural-pdf[classification]"
+```
+
+This installs necessary libraries like `torch`, `transformers`, and others.
+
+## Core Concept: The `.classify()` Method
+
+The primary way to perform categorization is using the `.classify()` method available on `Page` and `Region` objects.
+
+```python
+from natural_pdf import PDF
+
+# Example: Classify a Page
+pdf = PDF("pdfs/01-practice.pdf")
+page = pdf.pages[0]
+categories = ["invoice", "letter", "report cover", "data table"]
+results = page.classify(categories=categories, model="text")
+
+# Access the top result
+print(f"Top Category: {page.category}")
+print(f"Confidence: {page.category_confidence:.3f}")
+
+# Access all results
+# print(page.classification_results)
+```
+
+**Key Arguments:**
+
+*   `categories` (required): A list of strings representing the potential categories you want to classify the item into.
+*   `model` (optional): Specifies which classification model or strategy to use. Defaults to `"text"`.
+    *   `"text"`: Uses a text-based model (default: `facebook/bart-large-mnli`) suitable for classifying based on language content.
+    *   `"vision"`: Uses a vision-based model (default: `openai/clip-vit-base-patch32`) suitable for classifying based on visual layout and appearance.
+    *   Specific Model ID: You can provide a Hugging Face model ID (e.g., `"google/siglip-base-patch16-224"`, `"MoritzLaurer/mDeBERTa-v3-base-mnli-xnli"`) compatible with zero-shot text or image classification. The library attempts to infer whether it's text or vision, but you might need `using`.
+*   `using` (optional): Explicitly set to `"text"` or `"vision"` if the automatic inference based on the `model` ID fails or is ambiguous.
+*   `min_confidence` (optional): A float between 0.0 and 1.0. Only categories with a confidence score greater than or equal to this threshold will be included in the results (default: 0.0).
+
+## Text vs. Vision Classification
+
+Choosing the right model type depends on your goal:
+
+### Text Classification (`model="text"`)
+
+*   **How it works:** Extracts the text from the page or region and analyzes the language content.
+*   **Best for:**
+    *   **Topic Identification:** Determining what a page or section is *about* (e.g., "budget discussion," "environmental impact," "legal terms").
+    *   **Content-Driven Document Types:** Identifying document types primarily defined by their text (e.g., emails, meeting minutes, news articles, reports).
+*   **Data Journalism Example:** You have thousands of pages of government reports. You can use text classification to find all pages discussing "public health funding" or classify paragraphs within environmental impact statements to find mentions of specific endangered species.
+
+```python
+# Find pages related to finance
+financial_categories = ["budget", "revenue", "expenditure", "forecast"]
+pdf.classify_pages(categories=financial_categories, model="text")
+budget_pages = [p for p in pdf.pages if p.category == "budget"]
+```
+
+### Vision Classification (`model="vision"`)
+
+*   **How it works:** Renders the page or region as an image and analyzes its visual layout, structure, and appearance.
+*   **Best for:**
+    *   **Layout-Driven Document Types:** Identifying documents recognizable by their structure (e.g., invoices, receipts, forms, presentation slides, title pages).
+    *   **Identifying Visual Elements:** Distinguishing between pages dominated by text, tables, charts, or images.
+*   **Data Journalism Example:** You have a scanned archive of campaign finance filings containing various document types. You can use vision classification to quickly isolate all the pages that look like donation receipts or expenditure forms, even if the OCR quality is poor.
+
+```python
+# Find pages that look like invoices or receipts
+visual_categories = ["invoice", "receipt", "letter", "form"]
+page.classify(categories=visual_categories, model="vision")
+if page.category in ["invoice", "receipt"]:
+    print(f"Page {page.number} looks like an invoice or receipt.")
+```
+
+## Classifying Specific Objects
+
+### Pages (`page.classify(...)`)
+
+Classifying a whole page is useful for sorting documents or identifying the overall purpose of a page within a larger document.
+
+```python
+# Classify the first page
+page = pdf.pages[0]
+page_types = ["cover page", "table of contents", "chapter start", "appendix"]
+page.classify(categories=page_types, model="vision") # Vision often good for page structure
+print(f"Page 1 Type: {page.category}")
+```
+
+### Regions (`region.classify(...)`)
+
+Classifying a specific region allows for more granular analysis within a page. You might first detect regions using Layout Analysis and then classify those regions.
+
+```python
+# Assume layout analysis has run, find paragraphs
+paragraphs = page.find_all("region[type=paragraph]")
+if paragraphs:
+    # Classify the topic of the first paragraph
+    topic_categories = ["introduction", "methodology", "results", "conclusion"]
+    # Use text model for topic
+    paragraphs[0].classify(categories=topic_categories, model="text")
+    print(f"First paragraph category: {paragraphs[0].category}")
+```
+
+## Accessing Classification Results
+
+After running `.classify()`, you can access the results:
+
+*   `page.category` or `region.category`: Returns the string label of the category with the highest confidence score from the *last* classification run. Returns `None` if no classification has been run or no category met the threshold.
+*   `page.category_confidence` or `region.category_confidence`: Returns the float confidence score (0.0-1.0) for the top category. Returns `None` otherwise.
+*   `page.classification_results` or `region.classification_results`: Returns the full result dictionary stored in the object's `.metadata['classification']`, containing the model used, engine type, categories provided, timestamp, and a list of all scores above the threshold sorted by confidence. Returns `None` if no classification has been run.
+
+```python
+results = page.classify(categories=["invoice", "letter"], model="text", min_confidence=0.5)
+
+if page.category == "invoice":
+    print(f"Found an invoice with confidence {page.category_confidence:.2f}")
+
+# See all results above the threshold
+# print(page.classification_results['scores'])
+```
+
+## Classifying Collections
+
+For batch processing, use the `.classify_all()` method on `PDFCollection` or `ElementCollection` objects. This displays a progress bar tracking individual items (pages or elements).
+
+### PDFCollection (`collection.classify_all(...)`)
+
+Classifies pages across all PDFs in the collection. Use `max_workers` for parallel processing across different PDF files.
+
+```python
+collection = natural_pdf.PDFCollection.from_directory("./documents/")
+categories = ["form", "datasheet", "image", "text document"]
+
+# Classify all pages using vision model, processing 4 PDFs concurrently
+collection.classify_all(categories=categories, model="vision", max_workers=4)
+
+# Filter PDFs containing forms
+form_pdfs = []
+for pdf in collection:
+    if any(p.category == "form" for p in pdf.pages if p.category):
+        form_pdfs.append(pdf.path)
+    pdf.close() # Remember to close PDFs
+
+print(f"Found forms in: {form_pdfs}")
+```
+
+### ElementCollection (`element_collection.classify_all(...)`)
+
+Classifies all classifiable elements (currently `Page` and `Region`) within the collection.
+
+```python
+# Assume 'pdf' is loaded and 'layout_regions' is an ElementCollection of Regions
+layout_regions = pdf.find_all("region")
+region_types = ["paragraph", "list", "table", "figure", "caption"]
+
+# Classify all detected regions based on vision
+layout_regions.classify_all(categories=region_types, model="vision")
+
+# Count table regions
+table_count = sum(1 for r in layout_regions if r.category == "table")
+print(f"Found {table_count} regions classified as tables.")
+```

natural_pdf-0.1.9/docs/data-extraction/index.md

@@ -0,0 +1,109 @@
+# Structured Data Extraction
+
+Extracting specific, structured information (like invoice numbers, dates, or addresses) from documents often requires more than simple text extraction. Natural PDF integrates with LLMs to pull out [structured data](https://platform.openai.com/docs/guides/structured-outputs).
+
+You need to install more than just the tiny baby default `natural_pdf` for this:
+```
+# Install just the LLM portions
+pip install "natural_pdf[llm]"
+
+# Install eeeeeverything
+pip install "natural_pdf[all]"
+```
+
+## Introduction
+
+This feature allows you to define the exact data structure you want using a Pydantic model and then instruct an LLM to populate that structure based on the content of a PDF element (like a `Page` or `Region`).
+
+> Not sure how to write a Pydantic schema? Just ask an LLM! "Write me a Pydantic schema to pull out an invoice number (an integer), a company name (string) and a date (string)." It'll go fine.
+
+## Basic Extraction
+
+1.  **Define a Schema:** Create a Pydantic model for your desired data.
+2.  **Extract:** Use `.extract()` on a `PDF`, `Page`, or `Region` object.
+3.  **Access:** Use `.extracted()` to retrieve the results.
+
+```python
+from natural_pdf import PDF
+from pydantic import BaseModel, Field
+from openai import OpenAI
+
+# Initialize your LLM client
+# Anything OpenAI-compatible works!
+client = OpenAI(
+    api_key="ANTHROPIC_API_KEY",  # Your Anthropic API key
+    base_url="https://api.anthropic.com/v1/"  # Anthropic's API endpoint
+)
+
+# Load the PDF
+pdf = PDF("path/to/your/document.pdf")
+page = pdf.pages[0]
+
+# Define your schema
+class InvoiceInfo(BaseModel):
+    invoice_number: str = Field(description="The main invoice identifier")
+    total_amount: float = Field(description="The final amount due")
+    company_name: Optional[str] = Field(None, description="The name of the issuing company")
+
+# Extract data
+page.extract(schema=InvoiceInfo, client=client) 
+
+# Access the full result object
+full_data = page.extracted() 
+print(full_data)
+
+# Access a single field
+inv_num = page.extracted('invoice_number')
+print(f"Invoice Number: {inv_num}") 
+```
+
+## Keys and Overwriting
+
+- By default, results are stored under the key `"default-structured"` in the element's `.analyses` dictionary.
+- Use the `analysis_key` parameter in `.extract()` to store results under a different name (e.g., `analysis_key="customer_details"`).
+- Attempting to extract using an existing `analysis_key` will raise an error unless `overwrite=True` is specified.
+
+```python
+# Extract using a specific key
+page.extract(InvoiceInfo, client=client, analysis_key="invoice_header")
+
+# Access using the specific key
+header_data = page.extracted(analysis_key="invoice_header") 
+company = page.extracted('company_name', analysis_key="invoice_header")
+```
+
+## Text vs vision
+
+When sending a page (or a region or etc) to an LLM, you can choose either `using='text'` (default) or `using='vision'`.
+
+- `text` sends the text, somewhat respecting layout using `.extract_text(layout=True)`
+- `vision` sends an image of the page with `.to_image(resolution=72)` (no highlights or labels)
+
+## Batch and bulk extraction
+
+If you have a lot of pages or a lot of PDFs or a lot of anything, the `.extract()` and `.extracted()` methods work identically on most parts of a PDF - regions, pages, collections of pdfs, etc, allowing a lot of flexibility in what you analyze.
+
+```python
+# Assuming 'header_region' is a Region object you defined
+header_region.extract(InvoiceInfo, client)
+company = header_region.extracted('company_name')
+```
+
+Furthermore, you can apply extraction to collections of elements (like `pdf.pages`, or the result of `pdf.find_all(...)`) using the `.apply()` method. This iterates through the collection and calls `.extract()` on each item.
+
+```python
+# Example: Extract InvoiceInfo from the first 5 pages
+results = pdf.pages[:5].apply(
+    lambda page: page.extract(
+        client=client,
+        schema=InvoiceInfo, 
+        client=client, 
+        analysis_key="page_invoice_info",
+    )
+)
+
+# Access results for the first page in the collection
+pdf.pages[0].extracted('company_name', analysis_key="page_invoice_info")
+```
+
+This provides a powerful way to turn unstructured PDF content into structured, usable data.