natural-pdf 0.1.7__py3-none-any.whl → 0.1.9__py3-none-any.whl

docs/text-extraction/index.md

@@ -1,292 +0,0 @@
-# Text Extraction Guide
-
-This guide demonstrates various ways to extract text from PDFs using Natural PDF, from simple page dumps to targeted extraction based on elements, regions, and styles.
-
-## Setup
-
-First, let's import necessary libraries and load a sample PDF. We'll use `example.pdf` from the tutorials' `pdfs` directory. *Adjust the path if your setup differs.*
-
-```python
-from natural_pdf import PDF
-
-# Load the PDF
-pdf = PDF("https://github.com/jsoma/natural-pdf/raw/refs/heads/main/pdfs/01-practice.pdf")
-
-# Select the first page for initial examples
-page = pdf.pages[0]
-
-# Display the first page
-page.show(width=700)
-```
-
-## Basic Text Extraction
-
-Get all text from a page or the entire document.
-
-```python
-# Extract all text from the first page
-# Displaying first 500 characters
-print(page.extract_text()[:500])
-```
-
-You can also preserve layout with `layout=True`.
-
-```python
-# Extract text from the entire document (may take time)
-# Uncomment to run:
-print(page.extract_text(layout=True)[:2000])
-```
-
-## Extracting Text from Specific Elements
-
-Use selectors with `find()` or `find_all()` to target specific elements. *Selectors like `:contains("Summary")` are examples; adapt them to your PDF.*
-
-```python
-# Find a single element, e.g., a title containing "Summary"
-# Adjust selector as needed
-date_element = page.find('text:contains("Site")')
-date_element # Display the found element object
-```
-
-```python
-date_element.show()
-```
-
-```python
-date_element.text
-```
-
-```python
-# Find multiple elements, e.g., bold headings (size >= 8)
-heading_elements = page.find_all('text[size>=8]:bold')
-heading_elements 
-```
-
-```python
-page.find_all('text[size>=8]:bold').show()
-```
-
-```python
-# Pull out all of their text (why? I don't know!)
-print(heading_elements.extract_text())
-```
-
-## Advanced text searches
-
-```python
-# Exact phrase (case-sensitive)
-page.find('text:contains("Hazardous Materials")').text
-```
-
-```python
-# Exact phrase (case-sensitive)
-page.find('text:contains("HAZARDOUS MATERIALS")', case=False).text
-```
-
-```python
-# Regular expression (e.g., "YYYY Report")
-regex = "\d+, \d{4}"
-page.find(f'text:contains("{regex}")', regex=True)
-```
-
-```python
-# Regular expression (e.g., "YYYY Report")
-page.find_all('text[fontname="Helvetica"][size=10]')
-```
-
-# Regions
-
-```python
-# Region below an element (e.g., below "Introduction")
-# Adjust selector as needed
-page.find('text:contains("Summary")').below(include_element=True).show()
-```
-
-```python
-(
-    page
-    .find('text:contains("Summary")')
-    .below(include_element=True)
-    .extract_text()
-    [:500]
-)
-```
-
-```python
-(
-    page
-    .find('text:contains("Summary")')
-    .below(include_element=True, until='line:horizontal')
-    .show()
-)
-```
-
-```python
-# Manually defined region via coordinates (x0, top, x1, bottom)
-manual_region = page.create_region(30, 60, 600, 300)
-manual_region.show()
-```
-
-```python
-# Extract text from the manual region
-manual_region.extract_text()[:500]
-```
-
-## Filtering Out Headers and Footers
-
-Use Exclusion Zones to remove unwanted content before extraction. *Adjust selectors for typical header/footer content.*
-
-```python
-header_content = page.find('rect')
-footer_content = page.find_all('line')[-1].below()
-
-header_content.highlight()
-footer_content.highlight()
-page.to_image()
-```
-
-```python
-page.extract_text()[:500]
-```
-
-```python
-page.add_exclusion(header_content)
-page.add_exclusion(footer_content)
-```
-
-```python
-page.extract_text()[:500]
-```
-
-```python
-full_text_no_exclusions = page.extract_text(use_exclusions=False)
-clean_text = page.extract_text()
-f"Original length: {len(full_text_no_exclusions)}, Excluded length: {len(clean_text)}"
-```
-
-```python
-page.clear_exclusions()
-```
-
-*Exclusions can also be defined globally at the PDF level using `pdf.add_exclusion()` with a function.*
-
-## Controlling Whitespace
-
-Manage how spaces and blank lines are handled during extraction using `layout`.
-
-```python
-print(page.extract_text())
-```
-
-```python
-print(page.extract_text(use_exclusions=False, layout=True))
-```
-
-### Font Information Access
-
-Inspect font details of text elements.
-
-```python
-# Find the first text element on the page
-first_text = page.find_all('text')[1]
-first_text # Display basic info
-```
-
-```python
-# Highlight the first text element
-first_text.show()
-```
-
-```python
-# Get detailed font properties dictionary
-first_text.font_info()
-```
-
-```python
-# Check specific style properties directly
-f"Is Bold: {first_text.bold}, Is Italic: {first_text.italic}, Font: {first_text.fontname}, Size: {first_text.size}"
-```
-
-```python
-# Find elements by font attributes (adjust selectors)
-# Example: Find Arial fonts
-arial_text = page.find_all('text[fontname*=Helvetica]')
-arial_text # Display list of found elements
-```
-
-```python
-# Example: Find large text (e.g., size >= 16)
-large_text = page.find_all('text[size>=12]')
-large_text
-```
-
-```python
-# Example: Find large text (e.g., size >= 16)
-bold_text = page.find_all('text:bold')
-bold_text
-```
-
-## Working with Font Styles
-
-Analyze and group text elements by their computed font *style*, which combines attributes like font name, size, boldness, etc., into logical groups.
-
-```python
-# Analyze styles on the page
-# This returns a dictionary mapping style names to ElementList objects
-page.analyze_text_styles()
-page.text_style_labels
-```
-
-```python
-page.find_all('text').highlight(group_by='style_label').to_image()
-```
-
-```python
-page.find_all('text[style_label="8.0pt Helvetica"]')
-```
-
-```python
-page.find_all('text[fontname="Helvetica"][size=8]')
-```
-
-*Font variants (e.g., `AAAAAB+FontName`) are also accessible via the `font-variant` attribute selector: `page.find_all('text[font-variant="AAAAAB"]')`.*
-
-## Reading Order
-
-Text extraction respects a pathetic attempt at natural reading order (top-to-bottom, left-to-right by default). `page.find_all('text')` returns elements already sorted this way.
-
-```python
-# Get first 5 text elements in reading order
-elements_in_order = page.find_all('text')
-elements_in_order[:5]
-```
-
-```python
-# Text extracted via page.extract_text() respects this order automatically
-# (Result already shown in Basic Text Extraction section)
-page.extract_text()[:100]
-```
-
-## Element Navigation
-
-Move between elements sequentially based on reading order using `.next()` and `.previous()`.
-
-```python
-page.clear_highlights()
-
-start = page.find('text:contains("Date")')
-start.highlight(label='Date label')
-start.next().highlight(label='Maybe the date', color='green')
-start.next('text:contains("\d")', regex=True).highlight(label='Probably the date')
-
-page.to_image()
-```
-
-## Next Steps
-
-Now that you know how to extract text, you might want to explore:
-
-- [Working with regions](../regions/index.ipynb) for more precise extraction
-- [OCR capabilities](../ocr/index.md) for scanned documents
-- [Document layout analysis](../layout-analysis/index.ipynb) for automatic structure detection
-- [Document QA](../document-qa/index.ipynb) for asking questions directly to your documents

docs/tutorials/01-loading-and-extraction.ipynb

@@ -1,194 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "2e20408a",
-   "metadata": {},
-   "source": [
-    "# Loading and Basic Text Extraction"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "0fee7ac5",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "#%pip install \"natural-pdf[all]\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "765def8e",
-   "metadata": {},
-   "source": [
-    "In this tutorial, we'll learn how to:\n",
-    "\n",
-    "1. Load a PDF document\n",
-    "2. Extract text from pages\n",
-    "3. Extract specific elements \n",
-    "\n",
-    "## Loading a PDF\n",
-    "\n",
-    "Let's start by loading a PDF file:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "2aca41e4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from natural_pdf import PDF\n",
-    "import os\n",
-    "\n",
-    "# Load a PDF file\n",
-    "pdf = PDF(\"https://github.com/jsoma/natural-pdf/raw/refs/heads/main/pdfs/01-practice.pdf\")\n",
-    "\n",
-    "# Basic info about the document\n",
-    "{\n",
-    "    \"Filename\": os.path.basename(pdf.path),\n",
-    "    \"Pages\": len(pdf.pages),\n",
-    "    \"Title\": pdf.metadata.get(\"Title\", \"N/A\"),\n",
-    "    \"Author\": pdf.metadata.get(\"Author\", \"N/A\")\n",
-    "}"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "4e425d78",
-   "metadata": {},
-   "source": [
-    "## Extracting Text\n",
-    "\n",
-    "Now that we have loaded the PDF, let's extract the text from the first page:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "a6fa636e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Get the first page\n",
-    "page = pdf.pages[0]\n",
-    "\n",
-    "# Extract text from the page\n",
-    "text = page.extract_text()\n",
-    "\n",
-    "# Show the first 200 characters of the text\n",
-    "print(text[:200])"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "03e5318c",
-   "metadata": {},
-   "source": [
-    "## Finding and Extracting Specific Elements\n",
-    "\n",
-    "We can find specific elements using spatial queries and text content:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "ca478166",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Find text elements containing specific words\n",
-    "elements = page.find_all('text:contains(\"Inadequate\")')\n",
-    "\n",
-    "# Show these elements on the page\n",
-    "page.clear_highlights()\n",
-    "elements.highlight(color=\"red\", label=\"Contains 'Inadequate'\")\n",
-    "\n",
-    "# Display the page to see them\n",
-    "page.to_image(width=700)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "85311beb",
-   "metadata": {},
-   "source": [
-    "## Working with Layout Regions\n",
-    "\n",
-    "We can analyze the layout of the page to identify different regions:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "d2eaa3cb",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Analyze the page layout\n",
-    "page.analyze_layout(engine='yolo')\n",
-    "\n",
-    "# Find and highlight all detected regions\n",
-    "page.clear_highlights()\n",
-    "page.find_all('region').highlight(group_by='type')\n",
-    "\n",
-    "# Display the page to see the regions\n",
-    "page.to_image(width=900)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5e562020",
-   "metadata": {},
-   "source": [
-    "## Working with Multiple Pages\n",
-    "\n",
-    "You can also work with multiple pages:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "ed5bb91e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Process all pages\n",
-    "for page in pdf.pages:\n",
-    "    page_text = page.extract_text()\n",
-    "    print(f\"Page {page.number}\", page_text[:100])  # First 100 chars of each page"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e2b04ab1",
-   "metadata": {},
-   "source": [
-    "This tutorial covered the basics of loading PDFs and extracting text. In the next tutorials, we'll explore more advanced features like searching for specific elements, extracting structured content, and working with tables. "
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "3112c5b6",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%%bash\n",
-    "pip install \"natural-pdf[all]\""
-   ]
-  }
- ],
- "metadata": {
-  "jupytext": {
-   "cell_metadata_filter": "-all",
-   "main_language": "python",
-   "notebook_metadata_filter": "-all"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

docs/tutorials/01-loading-and-extraction.md

@@ -1,95 +0,0 @@
-# Loading and Basic Text Extraction
-
-```python
-#%pip install "natural-pdf[all]"
-```
-
-In this tutorial, we'll learn how to:
-
-1. Load a PDF document
-2. Extract text from pages
-3. Extract specific elements 
-
-## Loading a PDF
-
-Let's start by loading a PDF file:
-
-```python
-from natural_pdf import PDF
-import os
-
-# Load a PDF file
-pdf = PDF("https://github.com/jsoma/natural-pdf/raw/refs/heads/main/pdfs/01-practice.pdf")
-
-# Basic info about the document
-{
-    "Filename": os.path.basename(pdf.path),
-    "Pages": len(pdf.pages),
-    "Title": pdf.metadata.get("Title", "N/A"),
-    "Author": pdf.metadata.get("Author", "N/A")
-}
-```
-
-## Extracting Text
-
-Now that we have loaded the PDF, let's extract the text from the first page:
-
-```python
-# Get the first page
-page = pdf.pages[0]
-
-# Extract text from the page
-text = page.extract_text()
-
-# Show the first 200 characters of the text
-print(text[:200])
-```
-
-## Finding and Extracting Specific Elements
-
-We can find specific elements using spatial queries and text content:
-
-```python
-# Find text elements containing specific words
-elements = page.find_all('text:contains("Inadequate")')
-
-# Show these elements on the page
-page.clear_highlights()
-elements.highlight(color="red", label="Contains 'Inadequate'")
-
-# Display the page to see them
-page.to_image(width=700)
-```
-
-## Working with Layout Regions
-
-We can analyze the layout of the page to identify different regions:
-
-```python
-# Analyze the page layout
-page.analyze_layout(engine='yolo')
-
-# Find and highlight all detected regions
-page.clear_highlights()
-page.find_all('region').highlight(group_by='type')
-
-# Display the page to see the regions
-page.to_image(width=900)
-```
-
-## Working with Multiple Pages
-
-You can also work with multiple pages:
-
-```python
-# Process all pages
-for page in pdf.pages:
-    page_text = page.extract_text()
-    print(f"Page {page.number}", page_text[:100])  # First 100 chars of each page
-```
-
-This tutorial covered the basics of loading PDFs and extracting text. In the next tutorials, we'll explore more advanced features like searching for specific elements, extracting structured content, and working with tables. 
-
-```bash
-pip install "natural-pdf[all]"
-```