mercury-agent 0.5.7 → 0.5.10

package/examples/extensions/pdf/skill/SKILL.md

@@ -1,176 +1,353 @@
 ---
 name: pdf
-description: Use this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill.
-license: Proprietary. LICENSE.txt has complete terms
+description: Use this skill whenever the user wants to do anything with PDF files -- reading, extracting text/tables, merging, splitting, rotating, watermarking, creating, filling forms, encrypting/decrypting, extracting images, OCR, or converting to images. Activate when a .pdf file is mentioned or PDF output is requested.
 ---
 
-# PDF Processing Guide
+# PDF Processing Skill
 
-## Overview
+Process PDF files using open-source tools. This guide is organized by task -- find what you need to do, get working code.
 
-This guide covers essential PDF processing operations using Python libraries and command-line tools. For advanced features, JavaScript libraries, and detailed examples, see REFERENCE.md. If you need to fill out a PDF form, read FORMS.md and follow its instructions.
+## Decision Tree -- Pick the Right Tool
 
-## Quick Start
+```
+What do you need to do?
+|
++- READ text from a PDF
+|  +- Simple text extraction ----------> pypdf  (fast, pure Python)
+|  +- Text with layout/positions ------> pdfplumber  (preserves spatial info)
+|  +- Tables from PDF -----------------> pdfplumber  (best table extraction)
+|  +- Scanned PDF (image-based) -------> pytesseract + pdf2image  (OCR)
+|  +- CLI one-liner --------------------> pdftotext  (poppler)
+|
++- CREATE a new PDF from scratch ------> reportlab
+|
++- MODIFY an existing PDF
+|  +- Merge multiple PDFs --------------> pypdf
+|  +- Split into pages -----------------> pypdf
+|  +- Rotate pages ---------------------> pypdf
+|  +- Add watermark --------------------> pypdf (overlay)
+|  +- Encrypt / decrypt ----------------> pypdf or qpdf (CLI)
+|  +- Fill form fields -----------------> pypdf (fillable) or reportlab (non-fillable)
+|
++- EXTRACT from a PDF
+|  +- Images ---------------------------> pypdfium2 or pdfimages (CLI)
+|  +- Metadata -------------------------> pypdf
+|
++- CONVERT
+   +- PDF to images --------------------> pdf2image or pypdfium2
+   +- Images to PDF --------------------> reportlab or pypdf
+```
 
-```python
-from pypdf import PdfReader, PdfWriter
+## Decision Rules
+
+Follow these rules to pick the right approach for ambiguous tasks:
+
+1. **IF the user wants to extract text** -> Use pypdf or pdfplumber first. If the result is empty or garbled, the PDF is likely scanned -> fall back to the OCR workflow (Section 10).
+
+2. **IF the user wants to fill a form** -> First run `check_fillable_fields.py` to detect if the PDF has fillable fields.
+   - If fillable fields exist -> use the fillable workflow (pypdf)
+   - If no fillable fields -> use the annotation-based workflow (reportlab overlay)
+   - See `forms.md` for the complete decision tree.
+
+3. **IF the user wants to merge PDFs** -> Use pypdf for simple merges. Use `qpdf` CLI if you need fine-grained page selection (e.g., specific page ranges from multiple files).
 
-# Read a PDF
-reader = PdfReader("document.pdf")
-print(f"Pages: {len(reader.pages)}")
+4. **IF the user wants to encrypt** -> Use pypdf for AES-256 encryption. Use `qpdf` CLI for quick password protection from the command line.
+
+5. **IF the user wants images from a PDF** -> Use pypdfium2 for high-quality page renders. Use `pdfimages` CLI to extract embedded images without re-encoding.
+
+6. **IF the user wants to create a PDF** -> Use reportlab. For simple text documents, use the Canvas API. For complex reports with automatic layout, use Platypus (SimpleDocTemplate + Paragraphs).
+
+## Available Libraries
+
+All pre-installed in the container:
+
+| Library | Purpose |
+|---|---|
+| `pypdf` | Read, merge, split, rotate, encrypt, fill forms |
+| `pdfplumber` | Extract text with layout, tables |
+| `pypdfium2` | Fast rendering to images, text extraction |
+| `reportlab` | Create PDFs from scratch, draw text/shapes/images |
+| `pytesseract` | OCR (requires tesseract-ocr system package) |
+| `pdf2image` | Convert PDF pages to PIL images (requires poppler) |
+| `Pillow` | Image processing |
+
+CLI tools: `qpdf`, `pdftotext`, `pdfimages` (poppler-utils), `tesseract`
+
+---
 
-# Extract text
-text = ""
+## 1. Read / Extract Text
+
+### Simple text extraction with pypdf
+
+```python
+from pypdf import PdfReader
+
+reader = PdfReader("input.pdf")
 for page in reader.pages:
-    text += page.extract_text()
+    text = page.extract_text()
+    print(text)
 ```
 
-## Python Libraries
+### Extract text with layout using pdfplumber
 
-### pypdf - Basic Operations
+```python
+import pdfplumber
+
+with pdfplumber.open("input.pdf") as pdf:
+    for page in pdf.pages:
+        text = page.extract_text()
+        print(text)
+```
+
+### Extract tables with pdfplumber
 
-#### Merge PDFs
 ```python
-from pypdf import PdfWriter, PdfReader
+import pdfplumber
+
+with pdfplumber.open("input.pdf") as pdf:
+    for page in pdf.pages:
+        tables = page.extract_tables()
+        for table in tables:
+            for row in table:
+                print(row)
+```
+
+Export a table to CSV:
+
+```python
+import pdfplumber
+import csv
+
+with pdfplumber.open("input.pdf") as pdf:
+    page = pdf.pages[0]  # first page
+    table = page.extract_tables()[0]  # first table
+
+    with open("output.csv", "w", newline="") as f:
+        writer = csv.writer(f)
+        writer.writerows(table)
+```
+
+### CLI text extraction
+
+```bash
+# Plain text
+pdftotext input.pdf output.txt
+
+# Preserve layout
+pdftotext -layout input.pdf output.txt
+
+# Extract specific pages
+pdftotext -f 2 -l 5 input.pdf output.txt
+
+# To stdout
+pdftotext input.pdf -
+```
+
+---
+
+## 2. Merge PDFs
+
+```python
+from pypdf import PdfWriter
 
 writer = PdfWriter()
-for pdf_file in ["doc1.pdf", "doc2.pdf", "doc3.pdf"]:
-    reader = PdfReader(pdf_file)
-    for page in reader.pages:
-        writer.add_page(page)
 
-with open("merged.pdf", "wb") as output:
-    writer.write(output)
+# Add entire PDFs
+for pdf_path in ["file1.pdf", "file2.pdf", "file3.pdf"]:
+    writer.append(pdf_path)
+
+writer.write("outbox/merged.pdf")
+writer.close()
 ```
 
-#### Split PDF
+Merge specific page ranges:
+
+```python
+from pypdf import PdfWriter
+
+writer = PdfWriter()
+writer.append("file1.pdf", pages=(0, 5))    # pages 1-5
+writer.append("file2.pdf", pages=(2, 4))    # pages 3-4
+writer.write("outbox/merged.pdf")
+writer.close()
+```
+
+---
+
+## 3. Split PDFs
+
+### Split into individual pages
+
 ```python
+from pypdf import PdfReader, PdfWriter
+
 reader = PdfReader("input.pdf")
 for i, page in enumerate(reader.pages):
     writer = PdfWriter()
     writer.add_page(page)
-    with open(f"page_{i+1}.pdf", "wb") as output:
-        writer.write(output)
+    writer.write(f"outbox/page_{i + 1}.pdf")
+    writer.close()
 ```
 
-#### Extract Metadata
+### Split into chunks
+
 ```python
-reader = PdfReader("document.pdf")
-meta = reader.metadata
-print(f"Title: {meta.title}")
-print(f"Author: {meta.author}")
-print(f"Subject: {meta.subject}")
-print(f"Creator: {meta.creator}")
+from pypdf import PdfReader, PdfWriter
+
+reader = PdfReader("input.pdf")
+chunk_size = 10
+
+for start in range(0, len(reader.pages), chunk_size):
+    writer = PdfWriter()
+    end = min(start + chunk_size, len(reader.pages))
+    for page in reader.pages[start:end]:
+        writer.add_page(page)
+    writer.write(f"outbox/chunk_{start // chunk_size + 1}.pdf")
+    writer.close()
 ```
 
-#### Rotate Pages
+---
+
+## 4. Rotate Pages
+
 ```python
+from pypdf import PdfReader, PdfWriter
+
 reader = PdfReader("input.pdf")
 writer = PdfWriter()
 
-page = reader.pages[0]
-page.rotate(90)  # Rotate 90 degrees clockwise
-writer.add_page(page)
+for page in reader.pages:
+    page.rotate(90)   # 90, 180, or 270 degrees clockwise
+    writer.add_page(page)
 
-with open("rotated.pdf", "wb") as output:
-    writer.write(output)
+writer.write("outbox/rotated.pdf")
+writer.close()
 ```
 
-### pdfplumber - Text and Table Extraction
+Rotate only specific pages:
 
-#### Extract Text with Layout
 ```python
-import pdfplumber
+from pypdf import PdfReader, PdfWriter
 
-with pdfplumber.open("document.pdf") as pdf:
-    for page in pdf.pages:
-        text = page.extract_text()
-        print(text)
+reader = PdfReader("input.pdf")
+writer = PdfWriter()
+
+pages_to_rotate = [0, 2, 4]  # 0-indexed
+
+for i, page in enumerate(reader.pages):
+    if i in pages_to_rotate:
+        page.rotate(90)
+    writer.add_page(page)
+
+writer.write("outbox/rotated.pdf")
+writer.close()
 ```
 
-#### Extract Tables
+---
+
+## 5. Add Watermark
+
 ```python
-with pdfplumber.open("document.pdf") as pdf:
-    for i, page in enumerate(pdf.pages):
-        tables = page.extract_tables()
-        for j, table in enumerate(tables):
-            print(f"Table {j+1} on page {i+1}:")
-            for row in table:
-                print(row)
+from pypdf import PdfReader, PdfWriter
+
+# The watermark must be a single-page PDF (create one with reportlab if needed)
+stamp_reader = PdfReader("watermark.pdf")
+stamp_page = stamp_reader.pages[0]
+
+reader = PdfReader("input.pdf")
+writer = PdfWriter()
+
+for page in reader.pages:
+    page.merge_page(stamp_page)
+    writer.add_page(page)
+
+writer.write("outbox/watermarked.pdf")
+writer.close()
 ```
 
-#### Advanced Table Extraction
+### Create a watermark PDF with reportlab
+
 ```python
-import pandas as pd
+from reportlab.pdfgen import canvas
+from reportlab.lib.pagesizes import letter
+from reportlab.lib.colors import Color
 
-with pdfplumber.open("document.pdf") as pdf:
-    all_tables = []
-    for page in pdf.pages:
-        tables = page.extract_tables()
-        for table in tables:
-            if table:  # Check if table is not empty
-                df = pd.DataFrame(table[1:], columns=table[0])
-                all_tables.append(df)
+c = canvas.Canvas("watermark.pdf", pagesize=letter)
+width, height = letter
 
-# Combine all tables
-if all_tables:
-    combined_df = pd.concat(all_tables, ignore_index=True)
-    combined_df.to_excel("extracted_tables.xlsx", index=False)
+c.saveState()
+c.translate(width / 2, height / 2)
+c.rotate(45)
+c.setFillColor(Color(0.8, 0.8, 0.8, alpha=0.3))
+c.setFont("Helvetica-Bold", 60)
+c.drawCentredString(0, 0, "CONFIDENTIAL")
+c.restoreState()
+c.save()
 ```
 
-### reportlab - Create PDFs
+---
+
+## 6. Create PDFs from Scratch (reportlab)
 
-#### Basic PDF Creation
 ```python
-from reportlab.lib.pagesizes import letter
 from reportlab.pdfgen import canvas
+from reportlab.lib.pagesizes import letter, A4
+from reportlab.lib.units import inch
 
-c = canvas.Canvas("hello.pdf", pagesize=letter)
+c = canvas.Canvas("outbox/new.pdf", pagesize=letter)
 width, height = letter
 
-# Add text
-c.drawString(100, height - 100, "Hello World!")
-c.drawString(100, height - 120, "This is a PDF created with reportlab")
+# Text
+c.setFont("Helvetica-Bold", 24)
+c.drawString(1 * inch, height - 1 * inch, "Report Title")
+
+c.setFont("Helvetica", 12)
+c.drawString(1 * inch, height - 1.5 * inch, "Generated automatically.")
+
+# Line
+c.setStrokeColorRGB(0, 0, 0)
+c.line(1 * inch, height - 1.7 * inch, width - 1 * inch, height - 1.7 * inch)
+
+# Rectangle
+c.setFillColorRGB(0.9, 0.9, 1.0)
+c.rect(1 * inch, height - 4 * inch, 4 * inch, 2 * inch, fill=True)
 
-# Add a line
-c.line(100, height - 140, 400, height - 140)
+# New page
+c.showPage()
+c.setFont("Helvetica", 12)
+c.drawString(1 * inch, height - 1 * inch, "Page 2 content here.")
 
-# Save
 c.save()
 ```
 
-#### Create PDF with Multiple Pages
+### Multi-line text with automatic wrapping
+
 ```python
 from reportlab.lib.pagesizes import letter
-from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer, PageBreak
+from reportlab.lib.units import inch
+from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer
 from reportlab.lib.styles import getSampleStyleSheet
 
-doc = SimpleDocTemplate("report.pdf", pagesize=letter)
+doc = SimpleDocTemplate("outbox/report.pdf", pagesize=letter)
 styles = getSampleStyleSheet()
 story = []
 
-# Add content
-title = Paragraph("Report Title", styles['Title'])
-story.append(title)
-story.append(Spacer(1, 12))
-
-body = Paragraph("This is the body of the report. " * 20, styles['Normal'])
-story.append(body)
-story.append(PageBreak())
+story.append(Paragraph("Report Title", styles["Title"]))
+story.append(Spacer(1, 0.25 * inch))
 
-# Page 2
-story.append(Paragraph("Page 2", styles['Heading1']))
-story.append(Paragraph("Content for page 2", styles['Normal']))
+body_text = """This is a paragraph that will automatically wrap to fit the page width.
+It supports <b>bold</b>, <i>italic</i>, and other basic HTML-like formatting."""
+story.append(Paragraph(body_text, styles["BodyText"]))
 
-# Build PDF
 doc.build(story)
 ```
 
-#### Subscripts and Superscripts
+### Pitfall: Unicode subscripts and superscripts
 
-**IMPORTANT**: Never use Unicode subscript/superscript characters (₀₁₂₃₄₅₆₇₈₉, ⁰¹²³⁴⁵⁶⁷⁸⁹) in ReportLab PDFs. The built-in fonts do not include these glyphs, causing them to render as solid black boxes.
+**IMPORTANT**: Never use Unicode subscript/superscript characters (such as the subscript digits) in ReportLab PDFs. The built-in fonts (Helvetica, Times-Roman, Courier) do not include these glyphs, causing them to render as solid black boxes.
 
 Instead, use ReportLab's XML markup tags in Paragraph objects:
+
 ```python
 from reportlab.platypus import Paragraph
 from reportlab.lib.styles import getSampleStyleSheet
@@ -178,137 +355,327 @@ from reportlab.lib.styles import getSampleStyleSheet
 styles = getSampleStyleSheet()
 
 # Subscripts: use <sub> tag
-chemical = Paragraph("H<sub>2</sub>O", styles['Normal'])
+chemical = Paragraph("H<sub>2</sub>O", styles["Normal"])
 
 # Superscripts: use <super> tag
-squared = Paragraph("x<super>2</super> + y<super>2</super>", styles['Normal'])
+squared = Paragraph("x<super>2</super> + y<super>2</super>", styles["Normal"])
 ```
 
-For canvas-drawn text (not Paragraph objects), manually adjust font the size and position rather than using Unicode subscripts/superscripts.
+For canvas-drawn text (not Paragraph objects), manually adjust the font size and position rather than using Unicode subscript/superscript characters.
 
-## Command-Line Tools
+### Pitfall: Unicode text beyond Latin-1
 
-### pdftotext (poppler-utils)
-```bash
-# Extract text
-pdftotext input.pdf output.txt
+ReportLab's built-in fonts (Helvetica, Times-Roman, Courier) do NOT support Unicode characters beyond Latin-1. For Unicode text (CJK, Cyrillic, Arabic, mathematical symbols), you must register a TrueType font:
 
-# Extract text preserving layout
-pdftotext -layout input.pdf output.txt
+```python
+from reportlab.pdfbase import pdfmetrics
+from reportlab.pdfbase.ttfonts import TTFont
 
-# Extract specific pages
-pdftotext -f 1 -l 5 input.pdf output.txt  # Pages 1-5
+pdfmetrics.registerFont(TTFont("DejaVu", "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf"))
+c.setFont("DejaVu", 12)
+c.drawString(100, 700, "Unicode text here")
 ```
 
-### qpdf
+---
+
+## 7. Fill PDF Forms
+
+This is a complex topic. Read `forms.md` for the full guide.
+
+### Quick: fill a simple fillable PDF
+
+```python
+from pypdf import PdfReader, PdfWriter
+
+reader = PdfReader("form.pdf")
+writer = PdfWriter()
+writer.append(reader)
+
+# Get field names
+fields = reader.get_fields()
+for name, field in fields.items():
+    print(f"Field: {name}, Type: {field.get('/FT')}")
+
+# Fill fields
+writer.update_page_form_field_values(
+    writer.pages[0],
+    {"first_name": "Alice", "last_name": "Smith", "email": "alice@example.com"},
+    auto_regenerate=False,
+)
+
+writer.write("outbox/filled.pdf")
+writer.close()
+```
+
+### Helper scripts
+
+The `scripts/` directory contains Python helpers for form processing:
+
+| Script | Purpose |
+|---|---|
+| `check_fillable_fields.py` | Detect if a PDF has fillable form fields |
+| `extract_form_field_info.py` | Extract field metadata to JSON |
+| `fill_fillable_fields.py` | Fill fillable fields from a JSON mapping |
+| `check_bounding_boxes.py` | Visualize field positions on the PDF |
+| `convert_pdf_to_images.py` | Render PDF pages to images |
+| `create_validation_image.py` | Overlay showing filled values |
+| `extract_form_structure.py` | Extract structure from non-fillable PDFs |
+| `fill_pdf_form_with_annotations.py` | Fill non-fillable PDFs using annotations |
+
+Run any script with `python3 scripts/<name>.py --help` for usage.
+
+---
+
+## 8. Encrypt / Decrypt PDFs
+
+### Encrypt with pypdf
+
+```python
+from pypdf import PdfReader, PdfWriter
+
+reader = PdfReader("input.pdf")
+writer = PdfWriter()
+writer.append(reader)
+
+# User password (to open) + owner password (to edit)
+writer.encrypt(
+    user_password="readpass",
+    owner_password="ownerpass",
+    algorithm="AES-256",
+)
+
+writer.write("outbox/encrypted.pdf")
+writer.close()
+```
+
+### Decrypt with pypdf
+
+```python
+from pypdf import PdfReader, PdfWriter
+
+reader = PdfReader("encrypted.pdf")
+if reader.is_encrypted:
+    reader.decrypt("readpass")
+
+writer = PdfWriter()
+writer.append(reader)
+writer.write("outbox/decrypted.pdf")
+writer.close()
+```
+
+### CLI with qpdf
+
 ```bash
-# Merge PDFs
-qpdf --empty --pages file1.pdf file2.pdf -- merged.pdf
+# Encrypt
+qpdf --encrypt userpass ownerpass 256 -- input.pdf encrypted.pdf
+
+# Decrypt
+qpdf --decrypt --password=ownerpass encrypted.pdf decrypted.pdf
 
-# Split pages
-qpdf input.pdf --pages . 1-5 -- pages1-5.pdf
-qpdf input.pdf --pages . 6-10 -- pages6-10.pdf
+# Linearize (optimize for web)
+qpdf --linearize input.pdf optimized.pdf
+```
+
+---
+
+## 9. Extract Images
+
+### With pypdfium2
+
+```python
+import pypdfium2 as pdfium
 
-# Rotate pages
-qpdf input.pdf output.pdf --rotate=+90:1  # Rotate page 1 by 90 degrees
+pdf = pdfium.PdfDocument("input.pdf")
 
-# Remove password
-qpdf --password=mypassword --decrypt encrypted.pdf decrypted.pdf
+for page_index in range(len(pdf)):
+    page = pdf[page_index]
+    for obj_index, obj in enumerate(page.get_objects()):
+        if obj.type == pdfium.FPDF_PAGEOBJ_IMAGE:
+            bitmap = obj.get_bitmap()
+            pil_image = bitmap.to_pil()
+            pil_image.save(f"outbox/image_p{page_index + 1}_{obj_index}.png")
 ```
 
-### pdftk (if available)
+### CLI with pdfimages
+
 ```bash
-# Merge
-pdftk file1.pdf file2.pdf cat output merged.pdf
+# Extract all images as PNG
+pdfimages -png input.pdf outbox/prefix
 
-# Split
-pdftk input.pdf burst
+# Extract as original format
+pdfimages -all input.pdf outbox/prefix
 
-# Rotate
-pdftk input.pdf rotate 1east output rotated.pdf
+# List images without extracting
+pdfimages -list input.pdf
 ```
 
-## Common Tasks
+---
+
+## 10. OCR Scanned PDFs
+
+### Detection: is this PDF scanned?
+
+If text extraction returns empty or garbled results, the PDF is likely scanned. Fall back to OCR.
 
-### Extract Text from Scanned PDFs
 ```python
-# Requires: pip install pytesseract pdf2image
-import pytesseract
 from pdf2image import convert_from_path
+import pytesseract
 
-# Convert PDF to images
-images = convert_from_path('scanned.pdf')
+# Convert PDF pages to images
+images = convert_from_path("scanned.pdf", dpi=300)
 
 # OCR each page
-text = ""
+full_text = []
 for i, image in enumerate(images):
-    text += f"Page {i+1}:\n"
-    text += pytesseract.image_to_string(image)
-    text += "\n\n"
+    text = pytesseract.image_to_string(image)
+    full_text.append(text)
+    print(f"--- Page {i + 1} ---")
+    print(text)
+
+# Save to file
+with open("outbox/ocr_output.txt", "w") as f:
+    f.write("\n\n".join(full_text))
+```
 
-print(text)
+### OCR to searchable PDF
+
+```python
+from pdf2image import convert_from_path
+import pytesseract
+
+images = convert_from_path("scanned.pdf", dpi=300)
+
+# Create searchable PDF from first page
+pdf_bytes = pytesseract.image_to_pdf_or_hocr(images[0], extension="pdf")
+with open("outbox/searchable.pdf", "wb") as f:
+    f.write(pdf_bytes)
 ```
 
-### Add Watermark
+**Tip**: OCR quality depends heavily on image resolution. Always render at 300 DPI minimum. For poor-quality scans, try 600 DPI.
+
+---
+
+## 11. Convert PDF Pages to Images
+
+### With pdf2image (poppler-based)
+
 ```python
-from pypdf import PdfReader, PdfWriter
+from pdf2image import convert_from_path
 
-# Create watermark (or load existing)
-watermark = PdfReader("watermark.pdf").pages[0]
+# All pages
+images = convert_from_path("input.pdf", dpi=200)
+for i, img in enumerate(images):
+    img.save(f"outbox/page_{i + 1}.png", "PNG")
 
-# Apply to all pages
-reader = PdfReader("document.pdf")
-writer = PdfWriter()
+# Specific pages
+images = convert_from_path("input.pdf", dpi=200, first_page=2, last_page=5)
+```
 
-for page in reader.pages:
-    page.merge_page(watermark)
-    writer.add_page(page)
+### With pypdfium2 (no poppler needed)
+
+```python
+import pypdfium2 as pdfium
+
+pdf = pdfium.PdfDocument("input.pdf")
 
-with open("watermarked.pdf", "wb") as output:
-    writer.write(output)
+for i in range(len(pdf)):
+    page = pdf[i]
+    bitmap = page.render(scale=2)  # 2x = ~144 DPI
+    pil_image = bitmap.to_pil()
+    pil_image.save(f"outbox/page_{i + 1}.png")
 ```
 
-### Extract Images
-```bash
-# Using pdfimages (poppler-utils)
-pdfimages -j input.pdf output_prefix
+### With the helper script
 
-# This extracts all images as output_prefix-000.jpg, output_prefix-001.jpg, etc.
+```bash
+python3 scripts/convert_pdf_to_images.py input.pdf -o outbox/ --dpi 200
 ```
 
-### Password Protection
+---
+
+## 12. Get PDF Metadata
+
 ```python
-from pypdf import PdfReader, PdfWriter
+from pypdf import PdfReader
 
 reader = PdfReader("input.pdf")
-writer = PdfWriter()
 
-for page in reader.pages:
-    writer.add_page(page)
+meta = reader.metadata
+print(f"Title:    {meta.title}")
+print(f"Author:   {meta.author}")
+print(f"Subject:  {meta.subject}")
+print(f"Creator:  {meta.creator}")
+print(f"Producer: {meta.producer}")
+print(f"Pages:    {len(reader.pages)}")
+
+# Page dimensions
+page = reader.pages[0]
+box = page.mediabox
+print(f"Size:     {float(box.width)} x {float(box.height)} points")
+```
+
+---
+
+## Common Patterns
+
+### Process many PDFs in a directory
+
+```python
+from pathlib import Path
+from pypdf import PdfReader
+
+pdf_dir = Path("inbox")
+for pdf_path in sorted(pdf_dir.glob("*.pdf")):
+    reader = PdfReader(pdf_path)
+    text = reader.pages[0].extract_text()
+    print(f"{pdf_path.name}: {text[:100]}...")
+```
+
+### Error handling
+
+```python
+from pypdf import PdfReader
+from pypdf.errors import PdfReadError
+
+try:
+    reader = PdfReader("input.pdf")
+    if reader.is_encrypted:
+        reader.decrypt("password")
+    text = reader.pages[0].extract_text()
+except PdfReadError as e:
+    print(f"Invalid or corrupted PDF: {e}")
+except Exception as e:
+    print(f"Error processing PDF: {e}")
+```
+
+### Working with the outbox
 
-# Add password
-writer.encrypt("userpassword", "ownerpassword")
+Always write output files to the `outbox/` directory so users can download them:
 
-with open("encrypted.pdf", "wb") as output:
-    writer.write(output)
+```python
+import os
+os.makedirs("outbox", exist_ok=True)
+writer.write("outbox/result.pdf")
 ```
 
-## Quick Reference
+---
+
+## Error Handling Guide
+
+When a PDF operation fails, follow these steps:
+
+1. **"Cannot open file" / encrypted error** -> PDF is encrypted. Ask the user for the password, then use `reader.decrypt(password)`.
+
+2. **Empty text extraction** -> PDF is scanned/image-based. Switch to the OCR workflow (Section 10).
+
+3. **"Cannot find font" in ReportLab** -> Register a TrueType font before use. See Section 6 pitfalls.
+
+4. **Corrupted PDF** -> Try `qpdf --check input.pdf` first. Then `qpdf input.pdf --replace-input` to attempt repair.
+
+5. **"No form fields found"** -> PDF uses static layout, not AcroForm. Use the annotation-based filling workflow (see forms.md, non-fillable section).
 
-| Task | Best Tool | Command/Code |
-|------|-----------|--------------|
-| Merge PDFs | pypdf | `writer.add_page(page)` |
-| Split PDFs | pypdf | One page per file |
-| Extract text | pdfplumber | `page.extract_text()` |
-| Extract tables | pdfplumber | `page.extract_tables()` |
-| Create PDFs | reportlab | Canvas or Platypus |
-| Command line merge | qpdf | `qpdf --empty --pages ...` |
-| OCR scanned PDFs | pytesseract | Convert to image first |
-| Fill PDF forms | pdf-lib or pypdf (see FORMS.md) | See FORMS.md |
+6. **Large PDF slow to process** -> Process pages in batches. Use `qpdf --pages . 1-50 --` to extract chunks. Use pypdfium2 instead of pdf2image for rendering -- it is significantly faster.
 
-## Next Steps
+7. **Image extraction returns 0 images** -> Images may be in-lined in content streams. Try `pdfimages -all` CLI tool instead.
 
-- For advanced pypdfium2 usage, see REFERENCE.md
-- For JavaScript libraries (pdf-lib), see REFERENCE.md
-- If you need to fill out a PDF form, follow the instructions in FORMS.md
-- For troubleshooting guides, see REFERENCE.md
+8. **Field values not visible after filling** -> Set `auto_regenerate=False` when calling `update_page_form_field_values`. Some PDF viewers cache field appearances. Try flattening: `qpdf --flatten-annotations=all filled.pdf output.pdf`.