natural-pdf 0.1.11__tar.gz → 0.1.12__tar.gz

natural_pdf-0.1.12/.github/workflows/test.yml

@@ -0,0 +1,55 @@
+name: Python Tests
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ['3.9', '3.10', '3.11']
+
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        lfs: true  # Enable Git LFS if you're using it for PDFs
+    
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v4
+      with:
+        python-version: ${{ matrix.python-version }}
+        cache: 'pip'
+    
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install nox nox-uv
+        python -m pip install -e .[dev]
+    
+    - name: Test with nox
+      run: |
+        nox -s test_core
+        nox -s test_favorites
+
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    
+    - name: Set up Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.11'
+        cache: 'pip'
+    
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install nox nox-uv
+    
+    - name: Lint with nox
+      run: nox -s lint 

{natural_pdf-0.1.11 → natural_pdf-0.1.12}/01-execute_notebooks.py

@@ -30,7 +30,7 @@ EXCLUDE_PATTERNS = [
     "finetuning/index.md",
     "categorizing-documents/index.md",
     "data-extraction/index.md",
-    "*.ipynb_checkpoints*"
+    "*.ipynb_checkpoints*",
 ]
 MAX_WORKERS = os.cpu_count()
 
@@ -178,7 +178,7 @@ def process_notebook(md_file_path_str: str, log_level: int) -> Dict[str, Any]:
         client = NotebookClient(
             notebook,
             timeout=600,
-            kernel_name="natural-pdf",
+            kernel_name="natural-pdf-project-venv",
             resources={"metadata": {"path": str(cwd)}},
         )
         client.execute()  # Modifies 'notebook' object

{natural_pdf-0.1.11 → natural_pdf-0.1.12}/MANIFEST.in

@@ -4,6 +4,9 @@ include LICENSE
 # HTML templates
 recursive-include natural_pdf/templates *.html
 
+# Data files
+recursive-include natural_pdf/exporters/data *
+
 # Documentation assets
 recursive-include docs *.md *.png *.jpg *.gif
 

{natural_pdf-0.1.11 → natural_pdf-0.1.12}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: natural-pdf
-Version: 0.1.11
+Version: 0.1.12
 Summary: A more intuitive interface for working with PDFs
 Author-email: Jonathan Soma <jonathan.soma@gmail.com>
 License-Expression: MIT
@@ -12,20 +12,16 @@ Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: pdfplumber
-Requires-Dist: Pillow
+Requires-Dist: pillow
 Requires-Dist: colour
 Requires-Dist: numpy
 Requires-Dist: urllib3
 Requires-Dist: tqdm
 Requires-Dist: pydantic
-Provides-Extra: interactive
-Requires-Dist: ipywidgets<9.0.0,>=7.0.0; extra == "interactive"
-Provides-Extra: haystack
-Requires-Dist: haystack-ai; extra == "haystack"
-Requires-Dist: lancedb-haystack; extra == "haystack"
-Requires-Dist: lancedb; extra == "haystack"
-Requires-Dist: sentence-transformers; extra == "haystack"
-Requires-Dist: natural-pdf[core-ml]; extra == "haystack"
+Requires-Dist: jenkspy
+Requires-Dist: pikepdf>=9.7.0
+Provides-Extra: viewer
+Requires-Dist: ipywidgets<9.0.0,>=7.0.0; extra == "viewer"
 Provides-Extra: easyocr
 Requires-Dist: easyocr; extra == "easyocr"
 Requires-Dist: natural-pdf[core-ml]; extra == "easyocr"
@@ -41,19 +37,25 @@ Requires-Dist: natural-pdf[core-ml]; extra == "surya"
 Provides-Extra: doctr
 Requires-Dist: python-doctr[torch]; extra == "doctr"
 Requires-Dist: natural-pdf[core-ml]; extra == "doctr"
-Provides-Extra: qa
-Requires-Dist: natural-pdf[core-ml]; extra == "qa"
 Provides-Extra: docling
 Requires-Dist: docling; extra == "docling"
 Requires-Dist: natural-pdf[core-ml]; extra == "docling"
 Provides-Extra: llm
 Requires-Dist: openai>=1.0; extra == "llm"
-Provides-Extra: classification
-Requires-Dist: sentence-transformers; extra == "classification"
-Requires-Dist: timm; extra == "classification"
-Requires-Dist: natural-pdf[core-ml]; extra == "classification"
 Provides-Extra: test
 Requires-Dist: pytest; extra == "test"
+Provides-Extra: search
+Requires-Dist: lancedb; extra == "search"
+Requires-Dist: pyarrow; extra == "search"
+Provides-Extra: favorites
+Requires-Dist: natural-pdf[deskew]; extra == "favorites"
+Requires-Dist: natural-pdf[llm]; extra == "favorites"
+Requires-Dist: natural-pdf[surya]; extra == "favorites"
+Requires-Dist: natural-pdf[easyocr]; extra == "favorites"
+Requires-Dist: natural-pdf[layout_yolo]; extra == "favorites"
+Requires-Dist: natural-pdf[ocr-export]; extra == "favorites"
+Requires-Dist: natural-pdf[viewer]; extra == "favorites"
+Requires-Dist: natural-pdf[search]; extra == "favorites"
 Provides-Extra: dev
 Requires-Dist: black; extra == "dev"
 Requires-Dist: isort; extra == "dev"
@@ -67,29 +69,32 @@ Requires-Dist: pipdeptree; extra == "dev"
 Requires-Dist: nbformat; extra == "dev"
 Requires-Dist: jupytext; extra == "dev"
 Requires-Dist: nbclient; extra == "dev"
+Requires-Dist: ipykernel; extra == "dev"
 Provides-Extra: deskew
 Requires-Dist: deskew>=1.5; extra == "deskew"
 Requires-Dist: img2pdf; extra == "deskew"
 Provides-Extra: all
-Requires-Dist: natural-pdf[interactive]; extra == "all"
-Requires-Dist: natural-pdf[haystack]; extra == "all"
+Requires-Dist: natural-pdf[viewer]; extra == "all"
 Requires-Dist: natural-pdf[easyocr]; extra == "all"
 Requires-Dist: natural-pdf[paddle]; extra == "all"
 Requires-Dist: natural-pdf[layout_yolo]; extra == "all"
 Requires-Dist: natural-pdf[surya]; extra == "all"
 Requires-Dist: natural-pdf[doctr]; extra == "all"
-Requires-Dist: natural-pdf[qa]; extra == "all"
 Requires-Dist: natural-pdf[ocr-export]; extra == "all"
 Requires-Dist: natural-pdf[docling]; extra == "all"
 Requires-Dist: natural-pdf[llm]; extra == "all"
-Requires-Dist: natural-pdf[classification]; extra == "all"
+Requires-Dist: natural-pdf[core-ml]; extra == "all"
 Requires-Dist: natural-pdf[deskew]; extra == "all"
 Requires-Dist: natural-pdf[test]; extra == "all"
+Requires-Dist: natural-pdf[search]; extra == "all"
 Provides-Extra: core-ml
 Requires-Dist: torch; extra == "core-ml"
 Requires-Dist: torchvision; extra == "core-ml"
 Requires-Dist: transformers[sentencepiece]; extra == "core-ml"
 Requires-Dist: huggingface_hub; extra == "core-ml"
+Requires-Dist: sentence-transformers; extra == "core-ml"
+Requires-Dist: numpy; extra == "core-ml"
+Requires-Dist: timm; extra == "core-ml"
 Provides-Extra: ocr-export
 Requires-Dist: pikepdf; extra == "ocr-export"
 Provides-Extra: export-extras
@@ -114,26 +119,11 @@ Natural PDF lets you find and extract content from PDFs using simple code that m
 pip install natural-pdf
 ```
 
-For optional features like specific OCR engines, layout analysis models, or the interactive Jupyter widget, you can install extras:
+For optional features like specific OCR engines, layout analysis models, or the interactive Jupyter widget, you can install one to two million different extras. If you just want the greatest hits:
 
 ```bash
-# Example: Install with EasyOCR support
-pip install natural-pdf[easyocr]
-pip install natural-pdf[surya]
-pip install natural-pdf[paddle]
-
-# Example: Install support for features using Large Language Models (e.g., via OpenAI-compatible APIs)
-pip install natural-pdf[llm]
-# (May require setting API key environment variables, e.g., GOOGLE_API_KEY for Gemini)
-
-# Example: Install with interactive viewer support
-pip install natural-pdf[interactive]
-
-# Example: Install with semantic search support (Haystack)
-pip install natural-pdf[haystack]
-
-# Install everything
-pip install natural-pdf[all]
+# deskewing, OCR (surya) + layout analysis (yolo), interactive browsing
+pip install natural-pdf[favorites]
 ```
 
 See the [installation guide](https://jsoma.github.io/natural-pdf/installation/) for more details on extras.
@@ -147,25 +137,26 @@ from natural_pdf import PDF
 pdf = PDF('document.pdf')
 page = pdf.pages[0]
 
+# Extract all of the text on the page
+page.extract_text()
+
 # Find elements using CSS-like selectors
 heading = page.find('text:contains("Summary"):bold')
 
 # Extract content below the heading
 content = heading.below().extract_text()
-print("Content below Summary:", content[:100] + "...")
 
-# Exclude headers/footers automatically (example)
-# You might define these based on common text or position
-page.add_exclusion(page.find('text:contains("CONFIDENTIAL")').above())
-page.add_exclusion(page.find_all('line')[-1].below())
+# Examine all the bold text on the page
+page.find_all('text:bold').show()
 
-# Extract clean text from the page
-clean_text = page.extract_text()
-print("\nClean page text:", clean_text[:200] + "...")
+# Exclude parts of the page from selectors/extractors
+header = page.find('text:contains("CONFIDENTIAL")').above()
+footer = page.find_all('line')[-1].below()
+page.add_exclusion(header)
+page.add_exclusion(footer)
 
-# Highlight the heading and view the page
-heading.highlight(color='red')
-page.to_image()
+# Extract clean text from the page ignoring exclusions
+clean_text = page.extract_text()
 ```
 
 And as a fun bonus, `page.viewer()` will provide an interactive method to explore the PDF.
@@ -186,3 +177,17 @@ Natural PDF offers a range of features for working with PDFs:
 ## Learn More
 
 Dive deeper into the features and explore advanced usage in the [**Complete Documentation**](https://jsoma.github.io/natural-pdf).
+
+## Best friends
+
+Natural PDF sits on top of a *lot* of fantastic tools and mdoels, some of which are:
+
+- [pdfplumber](https://github.com/jsvine/pdfplumber)
+- [EasyOCR](https://www.jaided.ai/easyocr/)
+- [PaddleOCR](https://paddlepaddle.github.io/PaddleOCR/latest/en/index.html)
+- [Surya](https://github.com/VikParuchuri/surya)
+- A specific [YOLO](https://github.com/opendatalab/DocLayout-YOLO)
+- [deskew](https://github.com/sbrunner/deskew)
+- [doctr](https://github.com/mindee/doctr)
+- [docling](https://github.com/docling-project/docling)
+- [Hugging Face](https://huggingface.co/models)

{natural_pdf-0.1.11 → natural_pdf-0.1.12}/README.md

@@ -15,26 +15,11 @@ Natural PDF lets you find and extract content from PDFs using simple code that m
 pip install natural-pdf
 ```
 
-For optional features like specific OCR engines, layout analysis models, or the interactive Jupyter widget, you can install extras:
+For optional features like specific OCR engines, layout analysis models, or the interactive Jupyter widget, you can install one to two million different extras. If you just want the greatest hits:
 
 ```bash
-# Example: Install with EasyOCR support
-pip install natural-pdf[easyocr]
-pip install natural-pdf[surya]
-pip install natural-pdf[paddle]
-
-# Example: Install support for features using Large Language Models (e.g., via OpenAI-compatible APIs)
-pip install natural-pdf[llm]
-# (May require setting API key environment variables, e.g., GOOGLE_API_KEY for Gemini)
-
-# Example: Install with interactive viewer support
-pip install natural-pdf[interactive]
-
-# Example: Install with semantic search support (Haystack)
-pip install natural-pdf[haystack]
-
-# Install everything
-pip install natural-pdf[all]
+# deskewing, OCR (surya) + layout analysis (yolo), interactive browsing
+pip install natural-pdf[favorites]
 ```
 
 See the [installation guide](https://jsoma.github.io/natural-pdf/installation/) for more details on extras.
@@ -48,25 +33,26 @@ from natural_pdf import PDF
 pdf = PDF('document.pdf')
 page = pdf.pages[0]
 
+# Extract all of the text on the page
+page.extract_text()
+
 # Find elements using CSS-like selectors
 heading = page.find('text:contains("Summary"):bold')
 
 # Extract content below the heading
 content = heading.below().extract_text()
-print("Content below Summary:", content[:100] + "...")
 
-# Exclude headers/footers automatically (example)
-# You might define these based on common text or position
-page.add_exclusion(page.find('text:contains("CONFIDENTIAL")').above())
-page.add_exclusion(page.find_all('line')[-1].below())
+# Examine all the bold text on the page
+page.find_all('text:bold').show()
 
-# Extract clean text from the page
-clean_text = page.extract_text()
-print("\nClean page text:", clean_text[:200] + "...")
+# Exclude parts of the page from selectors/extractors
+header = page.find('text:contains("CONFIDENTIAL")').above()
+footer = page.find_all('line')[-1].below()
+page.add_exclusion(header)
+page.add_exclusion(footer)
 
-# Highlight the heading and view the page
-heading.highlight(color='red')
-page.to_image()
+# Extract clean text from the page ignoring exclusions
+clean_text = page.extract_text()
 ```
 
 And as a fun bonus, `page.viewer()` will provide an interactive method to explore the PDF.
@@ -87,3 +73,17 @@ Natural PDF offers a range of features for working with PDFs:
 ## Learn More
 
 Dive deeper into the features and explore advanced usage in the [**Complete Documentation**](https://jsoma.github.io/natural-pdf).
+
+## Best friends
+
+Natural PDF sits on top of a *lot* of fantastic tools and mdoels, some of which are:
+
+- [pdfplumber](https://github.com/jsvine/pdfplumber)
+- [EasyOCR](https://www.jaided.ai/easyocr/)
+- [PaddleOCR](https://paddlepaddle.github.io/PaddleOCR/latest/en/index.html)
+- [Surya](https://github.com/VikParuchuri/surya)
+- A specific [YOLO](https://github.com/opendatalab/DocLayout-YOLO)
+- [deskew](https://github.com/sbrunner/deskew)
+- [doctr](https://github.com/mindee/doctr)
+- [docling](https://github.com/docling-project/docling)
+- [Hugging Face](https://huggingface.co/models)

{natural_pdf-0.1.11 → natural_pdf-0.1.12}/check_run_md.sh

@@ -9,7 +9,7 @@ fi
 
 MARKDOWN_FILE=$1
 NOTEBOOK_FILE="${MARKDOWN_FILE%.md}.ipynb"
-KERNEL_NAME="natural-pdf"
+KERNEL_NAME="natural-pdf-project-venv"
 
 echo "Converting $MARKDOWN_FILE to notebook..."
 # Jupytext will now automatically add tags based on markdown metadata
@@ -29,6 +29,6 @@ EOF
 
 
 echo "Executing notebook $NOTEBOOK_FILE..."
-jupyter execute "$NOTEBOOK_FILE" --inplace --ExecutePreprocessor.kernel_name=natural-pdf  || { echo "Execution failed"; exit 1; }
+jupyter execute "$NOTEBOOK_FILE" --inplace --ExecutePreprocessor.kernel_name=natural-pdf-project-venv  || { echo "Execution failed"; exit 1; }
 
 echo "Success! Notebook executed and results saved to $NOTEBOOK_FILE"

{natural_pdf-0.1.11 → natural_pdf-0.1.12}/docs/categorizing-documents/index.md

@@ -7,7 +7,7 @@ Natural PDF allows you to automatically categorize pages or specific regions wit
 To use the classification features, you need to install the optional dependencies:
 
 ```bash
-pip install "natural-pdf[classification]"
+pip install "natural-pdf[core-ml]"
 ```
 
 This installs necessary libraries like `torch`, `transformers`, and others.