natural-pdf 0.1.5__py3-none-any.whl → 0.1.7__py3-none-any.whl

docs/finetuning/index.md

@@ -0,0 +1,176 @@
+# OCR Fine-tuning
+
+While the built-in OCR engines (EasyOCR, PaddleOCR, Surya) offer good general performance, you might encounter situations where their accuracy isn't sufficient for your specific needs. This is often the case with:
+
+*   **Unique Fonts:** Documents using unusual or stylized fonts.
+*   **Specific Languages:** Languages or scripts not perfectly covered by the default models.
+*   **Low Quality Scans:** Noisy or degraded document images.
+*   **Specialized Layouts:** Text within complex tables, forms, or unusual arrangements.
+
+Fine-tuning allows you to adapt a pre-trained OCR recognition model to your specific data, significantly improving its accuracy on documents similar to those used for training.
+
+## Why Fine-tune?
+
+-   **Higher Accuracy:** Achieve better text extraction results on your specific document types.
+-   **Adaptability:** Train the model to recognize domain-specific terms, symbols, or layouts.
+-   **Reduced Errors:** Minimize downstream errors in data extraction and processing pipelines.
+
+## Strategy: Detect + LLM Correct + Export
+
+Training an OCR model requires accurate ground truth: images of text snippets paired with their correct transcriptions. Manually creating this data is tedious. A powerful alternative leverages the strengths of different models:
+
+1.  **Detect Text Regions:** Use a robust local OCR engine (like Surya or PaddleOCR) primarily for its *detection* capabilities (`detect_only=True`). This identifies the *locations* of text on the page, even if the initial *recognition* isn't perfect. You can combine this with layout analysis or region selections (`.region()`, `.below()`, `.add_exclusion()`) to focus on the specific areas you care about.
+2.  **Correct with LLM:** For each detected text region, send the image snippet to a powerful Large Language Model (LLM) with multimodal capabilities (like GPT-4o, Claude 3.5 Sonnet/Haiku) using the `direct_ocr_llm` utility. The LLM performs high-accuracy OCR on the snippet, providing a "ground truth" transcription.
+3.  **Export for Fine-tuning:** Use the `PaddleOCRRecognitionExporter` to package the original image snippets (from step 1) along with their corresponding LLM-generated text labels (from step 2) into the specific format required by PaddleOCR for fine-tuning its *recognition* model.
+
+This approach combines the efficient spatial detection of local models with the superior text recognition of large generative models to create a high-quality fine-tuning dataset with minimal manual effort.
+
+## Example: Fine-tuning for Greek Spreadsheet Text
+
+Let's walk through an example of preparing data to fine-tune PaddleOCR for text from a scanned Greek spreadsheet, adapting the process described above.
+
+```python
+# --- 1. Setup and Load PDF ---
+from natural_pdf import PDF
+from natural_pdf.ocr.utils import direct_ocr_llm
+from natural_pdf.exporters import PaddleOCRRecognitionExporter
+import openai # Or your preferred LLM client library
+import os
+
+# Ensure your LLM API key is set (using environment variables is recommended)
+# os.environ["OPENAI_API_KEY"] = "sk-..." 
+# os.environ["ANTHROPIC_API_KEY"] = "sk-..." 
+
+# pdf_path = "path/to/your/document.pdf" 
+pdf_path = "pdfs/hidden/the-bad-one.pdf" # Replace with your PDF path
+pdf = PDF(pdf_path)
+
+# --- 2. (Optional) Exclude Irrelevant Areas ---
+# If the document has consistent headers, footers, or margins you want to ignore
+# Use exclusions *before* detection
+pdf.add_exclusion(lambda page: page.region(right=45)) # Exclude left margin/line numbers
+pdf.add_exclusion(lambda page: page.region(left=500)) # Exclude right margin
+
+# --- 3. Detect Text Regions ---
+# Use a good detection engine. Surya is often robust for line detection.
+# We only want the bounding boxes, not the initial (potentially inaccurate) OCR text.
+print("Detecting text regions...")
+# Process only a subset of pages for demonstration if needed
+for page in pdf.pages[:10]:
+    # Use a moderate resolution for detection; higher res used for LLM correction later
+    page.apply_ocr(engine='surya', resolution=120, detect_only=True) 
+print(f"Detection complete for {num_pages_to_process} pages.")
+
+# (Optional) Visualize detected boxes on a sample page
+# pdf.pages[9].find_all('text[source=ocr]').show() 
+
+# --- 4. Correct with LLM ---
+# Configure your LLM client (example using OpenAI client, adaptable for others)
+# For Anthropic: client = openai.OpenAI(base_url="https://api.anthropic.com/v1/", api_key=os.environ.get("ANTHROPIC_API_KEY"))
+client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY")) 
+
+# Craft a clear prompt for the LLM
+# Be as specific as possible! If it's in a specific language, what kinds
+# of characters, etc.
+prompt = """OCR this image patch. Return only the exact text content visible in the image. 
+Preserve original spelling, capitalization, punctuation, and symbols. 
+Do not add any explanatory text, translations, comments, or quotation marks around the result.
+The text is likely from a Greek document, potentially a spreadsheet, containing Modern Greek words or numbers."""
+
+# Define the correction function using direct_ocr_llm
+def correct_text_region(region):
+    # Use a high resolution for the LLM call for best accuracy
+    return direct_ocr_llm(
+        region, 
+        client, 
+        prompt=prompt, 
+        resolution=300, 
+        # model="claude-3-5-sonnet-20240620" # Example Anthropic model
+        model="gpt-4o-mini" # Example OpenAI model
+    )
+
+# Apply the correction function to the detected text regions
+print("Applying LLM correction to detected regions...")
+for page in pdf.pages[:num_pages_to_process]:
+    # This finds elements added by apply_ocr and passes their regions to 'correct_text_region'
+    # The returned text from the LLM replaces the original OCR text for these elements
+    # The source attribute is updated (e.g., to 'ocr-llm-corrected')
+    page.correct_ocr(correct_text_region) 
+print("LLM correction complete.")
+
+# --- 5. Export for PaddleOCR Fine-tuning ---
+print("Configuring exporter...")
+exporter = PaddleOCRRecognitionExporter(
+    # Select all of the non-blank OCR text
+    # Hopefully it's all been LLM-corrected! 
+    selector="text[source^=ocr][text!='']", 
+    resolution=300,     # Resolution for the exported image crops
+    padding=2,          # Add slight padding around text boxes
+    split_ratio=0.9,    # 90% for training, 10% for validation
+    random_seed=42,     # For reproducible train/val split
+    include_guide=True  # Include the Colab fine-tuning notebook
+)
+
+# Define the output directory
+output_directory = "./my_paddleocr_finetune_data"
+print(f"Exporting data to {output_directory}...")
+
+# Run the export process
+exporter.export(pdf, output_directory)
+
+print("Export complete.")
+print(f"Dataset ready for fine-tuning in: {output_directory}")
+print(f"Next step: Upload '{os.path.join(output_directory, 'fine_tune_paddleocr.ipynb')}' and the rest of the contents to Google Colab.")
+
+# --- Cleanup ---
+pdf.close() 
+```
+
+## Running the Fine-tuning
+
+The `PaddleOCRRecognitionExporter` automatically includes a Jupyter Notebook (`fine_tune_paddleocr.ipynb`) in the output directory. This notebook is pre-configured to guide you through the fine-tuning process on Google Colab (which offers free GPU access):
+
+1.  **Upload:** Upload the entire output directory (e.g., `my_paddleocr_finetune_data`) to your Google Drive or directly to your Colab instance.
+2.  **Open Notebook:** Open the `fine_tune_paddleocr.ipynb` notebook in Google Colab.
+3.  **Set Runtime:** Ensure the Colab runtime is set to use a GPU (Runtime -> Change runtime type -> GPU).
+4.  **Run Cells:** Execute the cells in the notebook sequentially. It will:
+    *   Install necessary libraries (PaddlePaddle, PaddleOCR).
+    *   Point the training configuration to your uploaded dataset (`images/`, `train.txt`, `val.txt`, `dict.txt`).
+    *   Download a pre-trained PaddleOCR model (usually a multilingual one).
+    *   Start the fine-tuning process using your data.
+    *   Save the fine-tuned model checkpoints.
+    *   Export the best model into an "inference format" suitable for use with `natural-pdf`.
+5.  **Download Model:** Download the resulting `inference_model` directory from Colab.
+
+## Using the Fine-tuned Model
+
+Once you have the `inference_model` directory, you can instruct `natural-pdf` to use it for OCR:
+
+```python
+from natural_pdf import PDF
+from natural_pdf.ocr import PaddleOCROptions
+
+# Path to the directory you downloaded from Colab
+finetuned_model_dir = "/path/to/your/downloaded/inference_model" 
+
+# Specify the path in PaddleOCROptions
+paddle_opts = PaddleOCROptions(
+    rec_model_dir=finetuned_model_dir,
+    rec_char_dict_path=os.path.join(finetuned_model_dir, 'your_dict.txt') # Or wherever your dict is
+    use_gpu=True # If using GPU locally
+)
+
+pdf = PDF("another-similar-document.pdf")
+page = pdf.pages[0]
+
+# Apply OCR using your fine-tuned model
+ocr_elements = page.apply_ocr(engine='paddle', options=paddle_opts)
+
+# Extract text using the improved results
+text = page.extract_text() 
+print(text)
+
+pdf.close()
+```
+
+By following this process, you can significantly enhance OCR performance on your specific documents using the power of fine-tuning. 

docs/ocr/index.md

@@ -92,26 +92,6 @@ surya_opts = SuryaOCROptions(
 ocr_elements = page.apply_ocr(engine='surya', options=surya_opts)
 ```
 
-## Multiple Languages
-
-OCR supports multiple languages:
-
-```python
-# Recognize English and Spanish text
-pdf = PDF('multilingual.pdf', ocr={
-    'enabled': True,
-    'languages': ['en', 'es']
-})
-
-# Multiple languages with PaddleOCR
-pdf = PDF('multilingual_document.pdf', 
-          ocr_engine='paddleocr',
-          ocr={
-              'enabled': True,
-              'languages': ['zh', 'ja', 'ko', 'en']  # Chinese, Japanese, Korean, English
-          })
-```
-
 ## Applying OCR Directly
 
 The `page.apply_ocr(...)` and `region.apply_ocr(...)` methods are the primary way to run OCR:
@@ -179,39 +159,46 @@ high_conf = page.find_all('text[source=ocr][confidence>=0.8]')
 high_conf.highlight(color="green", label="High Confidence OCR")
 ```
 
-## OCR Debugging
+## Detect + LLM OCR
+
+Sometimes you have a difficult piece of content where you need to use a local model to identify the content, then send it off in pieces to be identified by the LLM. You can do this with Natural PDF!
+
+```python
+from natural_pdf import PDF
+from natural_pdf.ocr.utils import direct_ocr_llm
+import openai
+
+pdf = PDF("needs-ocr.pdf")
+page = pdf.pages[0]
+
+# Detect
+page.apply_ocr('paddle', resolution=120, detect_only=True)
+
+# Build the framework
+client = openai.OpenAI(base_url="https://api.anthropic.com/v1/",  api_key='sk-XXXXX')
+prompt = """OCR this image. Return only the exact text from the image. Include misspellings,
+punctuation, etc. Do not surround it with quotation marks. Do not include translations or comments.
+The text is from a Greek spreadsheet, so most likely content is Modern Greek or numeric."""
+
+# This returns the cleaned-up text
+def correct(region):
+    return direct_ocr_llm(region, client, prompt=prompt, resolution=300, model="claude-3-5-haiku-20241022")
+
+# Run 'correct' on each text element
+page.correct_ocr(correct)
+
+# You're done!
+```
 
-For troubleshooting OCR problems:
+## Debugging OCR
 
 ```python
-# Create an interactive HTML debug report
-pdf.debug_ocr("ocr_debug.html")
+from natural_pdf.utils.packaging import create_correction_task_package
 
-# Specify which pages to include
-pdf.debug_ocr("ocr_debug.html", pages=[0, 1, 2])
+create_correction_task_package(pdf, "original.zip", overwrite=True)
 ```
 
-The debug report shows:
-- The original image
-- Text found with confidence scores
-- Boxes around each detected word
-- Options to sort and filter results
-
-## OCR Parameter Tuning
-
-### Parameter Recommendation Table
-
-| Issue | Engine | Parameter | Recommended Value | Effect |
-|-------|--------|-----------|-------------------|--------|
-| Missing text | EasyOCR | `text_threshold` | 0.1 - 0.3 (default: 0.7) | Lower values detect more text but may increase false positives |
-| Missing text | PaddleOCR | `det_db_thresh` | 0.1 - 0.3 (default: 0.3) | Lower values detect more text areas |
-| Low quality scan | EasyOCR | `contrast_ths` | 0.05 - 0.1 (default: 0.1) | Lower values help with low contrast documents |
-| Low quality scan | PaddleOCR | `det_limit_side_len` | 1280 - 2560 (default: 960) | Higher values improve detail detection |
-| Accuracy vs. speed | EasyOCR | `decoder` | "wordbeamsearch" (accuracy)<br>"greedy" (speed) | Word beam search is more accurate but slower |
-| Accuracy vs. speed | PaddleOCR | `rec_batch_num` | 1 (accuracy)<br>8+ (speed) | Larger batches process faster but use more memory |
-| Small text | Both | `min_confidence` | 0.3 - 0.4 (default: 0.5) | Lower confidence threshold to capture small/blurry text |
-| Text orientation | PaddleOCR | `use_angle_cls` | `True` | Enable angle classification for rotated text |
-| Asian languages | PaddleOCR | `lang` | "ch", "japan", "korea" | Use PaddleOCR for Asian languages |
+This will at *some point* be official-ized, but for now you can look at `templates/spa` and see the correction package.
 
 ## Next Steps