natural-pdf 0.1.4__py3-none-any.whl → 0.1.5__py3-none-any.whl

docs/element-selection/index.md

@@ -0,0 +1,229 @@
+# Finding Elements with Selectors
+
+Natural PDF uses CSS-like selectors to find elements (text, lines, images, etc.) within a PDF page or document. This guide demonstrates how to use these selectors effectively.
+
+## Setup
+
+Let's load a sample PDF to work with. We'll use `01-practice.pdf` which has various elements.
+
+```python
+from natural_pdf import PDF
+
+# Load the PDF
+pdf = PDF("https://github.com/jsoma/natural-pdf/raw/refs/heads/main/pdfs/01-practice.pdf")
+
+# Select the first page
+page = pdf.pages[0]
+
+# Display the page
+page.show()
+```
+
+## Basic Element Finding
+
+The core methods are `find()` (returns the first match) and `find_all()` (returns all matches as an `ElementCollection`).
+
+The basic selector structure is `element_type[attribute_filter]:pseudo_class`.
+
+### Finding Text by Content
+
+```python
+# Find the first text element containing "Summary"
+summary_text = page.find('text:contains("Summary")')
+summary_text
+```
+
+```python
+# Find all text elements containing "Inadequate"
+contains_inadequate = page.find_all('text:contains("Inadequate")')
+len(contains_inadequate)
+```
+
+```python
+summary_text.highlight(label='summary')
+contains_inadequate.highlight(label="inadequate")
+page.to_image(width=700)
+```
+
+## Selecting by Element Type
+
+You can select specific types of elements found in PDFs.
+
+```python
+# Find all text elements
+all_text = page.find_all('text')
+len(all_text)
+```
+
+```python
+# Find all rectangle elements
+all_rects = page.find_all('rect')
+len(all_rects)
+```
+
+```python
+# Find all line elements
+all_lines = page.find_all('line')
+len(all_lines)
+```
+
+```python
+page.find_all('line').show()
+```
+
+## Filtering by Attributes
+
+Use square brackets `[]` to filter elements by their properties (attributes).
+
+### Common Attributes & Operators
+
+| Attribute     | Example Usage          | Operators | Notes |
+|---------------|------------------------|-----------|-------|
+| `size` (text) | `text[size>=12]`       | `>`, `<`, `>=`, `<=` | Font size in points |
+| `fontname`    | `text[fontname*=Bold]` | `=`, `*=`  | `*=` for contains substring |
+| `color` (text)| `text[color~=red]`     | `~=`      | Approx. match (name, rgb, hex) |
+| `width` (line)| `line[width>1]`        | `>`, `<`, `>=`, `<=` | Line thickness |
+| `source`      | `text[source=ocr]`     | `=`       | `pdf`, `ocr`, `detected` |
+| `type` (region)| `region[type=table]`  | `=`       | Layout analysis region type |
+
+```python
+# Find large text (size >= 11 points)
+page.find_all('text[size>=11]')
+```
+
+```python
+# Find text with 'Helvetica' in the font name
+page.find_all('text[fontname*=Helvetica]')
+```
+
+```python
+# Find red text (using approximate color match)
+# This PDF has text with color (0.8, 0.0, 0.0)
+red_text = page.find_all('text[color~=red]')
+```
+
+```python
+# Highlight the red text (ignoring existing highlights)
+red_text.show()
+```
+
+```python
+# Find thick lines (width >= 2)
+page.find_all('line[width>=2]')
+```
+
+## Using Pseudo-Classes
+
+Use colons `:` for special conditions (pseudo-classes).
+
+### Common Pseudo-Classes
+
+| Pseudo-Class          | Example Usage                           | Notes |
+|-----------------------|-----------------------------------------|-------|
+| `:contains('text')` | `text:contains('Report')`             | Finds elements containing specific text |
+| `:bold`               | `text:bold`                             | Finds text heuristically identified as bold |
+| `:italic`             | `text:italic`                           | Finds text heuristically identified as italic |
+| `:below(selector)`    | `text:below('line[width>=2]')`         | Finds elements physically below the reference element |
+| `:above(selector)`    | `text:above('text:contains("Summary")')`| Finds elements physically above the reference element |
+| `:left-of(selector)`  | `line:left-of('rect')`                 | Finds elements physically left of the reference element |
+| `:right-of(selector)` | `text:right-of('rect')`                | Finds elements physically right of the reference element |
+| `:near(selector)`     | `text:near('image')`                   | Finds elements physically near the reference element |
+
+*Note: Spatial pseudo-classes like `:below`, `:above` identify elements based on bounding box positions relative to the **first** element matched by the inner selector.*
+
+```python
+# Find bold text
+page.find_all('text:bold').show()
+```
+
+```python
+# Combine attribute and pseudo-class: bold text size >= 11
+page.find_all('text[size>=11]:bold')
+```
+
+### Spatial Pseudo-Classes Examples
+
+```python
+# Find the thick horizontal line first
+ref_line = page.find('line[width>=2]')
+
+# Find text elements strictly above that line
+text_above_line = page.find_all('text:above("line[width>=2]")')
+text_above_line
+```
+
+## Advanced Text Searching Options
+
+Pass options to `find()` or `find_all()` for more control over text matching.
+
+```python
+# Case-insensitive search for "summary"
+page.find_all('text:contains("summary")', case=False)
+```
+
+```python
+# Regular expression search for the inspection ID (e.g., INS-XXX...)
+# The ID is in the red text we found earlier
+page.find_all('text:contains("INS-\\w+")', regex=True)
+```
+
+```python
+# Combine regex and case-insensitivity
+page.find_all('text:contains("jungle health")', regex=True, case=False)
+```
+
+## Working with ElementCollections
+
+`find_all()` returns an `ElementCollection`, which is like a list but with extra PDF-specific methods.
+
+```python
+# Get all headings (using a selector for large, bold text)
+headings = page.find_all('text[size>=11]:bold')
+headings
+```
+
+```python
+# Get the first and last heading in reading order
+first = headings.first
+last = headings.last
+(first, last)
+```
+
+```python
+# Get the physically highest/lowest element in the collection
+highest = headings.highest()
+lowest = headings.lowest()
+(highest, lowest)
+```
+
+```python
+# Filter the collection further: headings containing "Service"
+service_headings = headings.find_all('text:contains("Service")')
+service_headings
+```
+
+```python
+# Extract text from all elements in the collection
+headings.extract_text()
+```
+
+*Remember: `.highest()`, `.lowest()`, `.leftmost()`, `.rightmost()` raise errors if the collection spans multiple pages.*
+
+## Font Variants
+
+Sometimes PDFs use font variants (prefixes like `AAAAAB+`) which can be useful for selection.
+
+```python
+# Find text elements with a specific font variant prefix (if any exist)
+# This example PDF doesn't use variants, but the selector works like this:
+page.find_all('text[font-variant=AAAAAB]')
+```
+
+## Next Steps
+
+Now that you can find elements, explore:
+
+- [Text Extraction](../text-extraction/index.ipynb): Get text content from found elements.
+- [Spatial Navigation](../pdf-navigation/index.ipynb): Use found elements as anchors to navigate (`.above()`, `.below()`, etc.).
+- [Working with Regions](../regions/index.ipynb): Define areas based on found elements.
+- [Visual Debugging](../visual-debugging/index.ipynb): Techniques for highlighting and visualizing elements.

docs/index.md

@@ -0,0 +1,170 @@
+# Natural PDF
+
+A friendly library for working with PDFs, built on top of [pdfplumber](https://github.com/jsvine/pdfplumber).
+
+Natural PDF lets you find and extract content from PDFs using simple code that makes sense.
+
+- [Live demo here](https://colab.research.google.com/github/jsoma/natural-pdf/blob/main/notebooks/Examples.ipynb)
+
+<div style="max-width: 400px; margin: auto"><a href="assets/sample-screen.png"><img src="assets/sample-screen.png"></a></div>
+
+## Installation
+
+```
+pip install natural_pdf
+# All the extras
+pip install "natural_pdf[all]"
+```
+
+## Quick Example
+
+```python
+from natural_pdf import PDF
+
+pdf = PDF('document.pdf')
+page = pdf.pages[0]
+
+# Find the title and get content below it
+title = page.find('text:contains("Summary"):bold')
+content = title.below().extract_text()
+
+# Exclude everything above 'CONFIDENTIAL' and below last line on page
+page.add_exclusion(page.find('text:contains("CONFIDENTIAL")').above())
+page.add_exclusion(page.find_all('line')[-1].below())
+
+# Get the clean text without header/footer
+clean_text = page.extract_text()
+```
+
+## Key Features
+
+Here are a few highlights of what you can do:
+
+### Find Elements with Selectors
+
+Use CSS-like selectors to find text, shapes, and more.
+
+```python
+# Find bold text containing "Revenue"
+page.find('text:contains("Revenue"):bold').extract_text()
+
+# Find all large text
+page.find_all('text[size>=12]').extract_text()
+```
+
+[Learn more about selectors →](element-selection/index.ipynb)
+
+### Navigate Spatially
+
+Move around the page relative to elements, not just coordinates.
+
+```python
+# Extract text below a specific heading
+intro_text = page.find('text:contains("Introduction")').below().extract_text()
+
+# Extract text from one heading to the next
+methods_text = page.find('text:contains("Methods")').below(
+    until='text:contains("Results")'
+).extract_text()
+```
+
+[Explore more navigation methods →](pdf-navigation/index.ipynb)
+
+### Extract Clean Text
+
+Easily extract text content, automatically handling common page elements like headers and footers (if exclusions are set).
+
+```python
+# Extract all text from the page (respecting exclusions)
+page_text = page.extract_text()
+
+# Extract text from a specific region
+some_region = page.find(...)
+region_text = some_region.extract_text()
+```
+
+[Learn about text extraction →](text-extraction/index.ipynb)
+[Learn about exclusion zones →](regions/index.ipynb#exclusion-zones)
+
+### Apply OCR
+
+Extract text from scanned documents using various OCR engines.
+
+```python
+# Apply OCR using the default engine
+ocr_elements = page.apply_ocr()
+
+# Extract text (will use OCR results if available)
+text = page.extract_text()
+```
+
+[Explore OCR options →](ocr/index.md)
+
+### Analyze Document Layout
+
+Use AI models to detect document structures like titles, paragraphs, and tables.
+
+```python
+# Detect document structure
+page.analyze_layout()
+
+# Highlight titles and tables
+page.find_all('region[type=title]').highlight(color="purple")
+page.find_all('region[type=table]').highlight(color="blue")
+
+# Extract data from the first table
+table_data = page.find('region[type=table]').extract_table()
+```
+
+[Learn about layout models →](layout-analysis/index.ipynb)
+[Working with tables? →](tables/index.ipynb)
+
+### Document Question Answering
+
+Ask natural language questions directly to your documents.
+
+```python
+# Ask a question
+result = pdf.ask("What was the company's revenue in 2022?")
+if result.get("found", False):
+    print(f"Answer: {result['answer']}")
+```
+
+[Learn about Document QA →](document-qa/index.ipynb)
+
+### Visualize Your Work
+
+Debug and understand your extractions visually.
+
+```python
+# Highlight headings
+page.find_all('text[size>=14]').highlight(color="red", label="Headings")
+
+# Launch the interactive viewer (Jupyter)
+# Requires: pip install natural-pdf[interactive]
+page.viewer()
+
+# Or save an image
+# page.save_image("highlighted.png")
+```
+
+[See more visualization options →](visual-debugging/index.ipynb)
+
+## Documentation Topics
+
+Choose what you want to learn about:
+
+### Task-based Guides
+- [Getting Started](installation/index.md): Install the library and run your first extraction
+- [PDF Navigation](pdf-navigation/index.ipynb): Open PDFs and work with pages
+- [Element Selection](element-selection/index.ipynb): Find text and other elements using selectors
+- [Text Extraction](text-extraction/index.ipynb): Extract clean text from documents
+- [Regions](regions/index.ipynb): Work with specific areas of a page
+- [Visual Debugging](visual-debugging/index.ipynb): See what you're extracting
+- [OCR](ocr/index.md): Extract text from scanned documents
+- [Layout Analysis](layout-analysis/index.ipynb): Detect document structure
+- [Tables](tables/index.ipynb): Extract tabular data
+- [Document QA](document-qa/index.ipynb): Ask questions to your documents
+
+### Reference
+- [API Reference](api/index.md): Complete library reference

docs/installation/index.md

@@ -0,0 +1,69 @@
+# Getting Started with Natural PDF
+
+Let's get Natural PDF installed and run your first extraction.
+
+## Installation
+
+The base installation includes the core library and necessary AI dependencies (like PyTorch and Transformers):
+
+```bash
+pip install natural-pdf
+```
+
+### Optional Dependencies
+
+Natural PDF has modular dependencies for different features. Install them based on your needs:
+
+```bash
+# --- OCR Engines ---
+# Install support for EasyOCR
+pip install natural-pdf[easyocr]
+
+# Install support for PaddleOCR (requires paddlepaddle)
+pip install natural-pdf[paddle]
+
+# Install support for Surya OCR
+pip install natural-pdf[surya]
+
+# --- Layout Detection ---
+# Install support for YOLO layout model
+pip install natural-pdf[layout_yolo]
+
+# --- Interactive Widget ---
+# Install support for the interactive .viewer() widget in Jupyter
+pip install natural-pdf[interactive]
+
+# --- All Features ---
+# Install all optional dependencies
+pip install natural-pdf[all]
+```
+
+## Your First PDF Extraction
+
+Here's a quick example to make sure everything is working:
+
+```python
+from natural_pdf import PDF
+
+# Open a PDF
+pdf = PDF('your_document.pdf')
+
+# Get the first page
+page = pdf.pages[0]
+
+# Extract all text
+text = page.extract_text()
+print(text)
+
+# Find something specific
+title = page.find('text:bold')
+print(f"Found title: {title.text}")
+```
+
+## What's Next?
+
+Now that you have Natural PDF installed, you can:
+
+- Learn to [navigate PDFs](../pdf-navigation/index.ipynb)
+- Explore how to [select elements](../element-selection/index.ipynb)
+- See how to [extract text](../text-extraction/index.ipynb)