pdf2dotmd 0.0.1__tar.gz

pdf2dotmd-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Robert He
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pdf2dotmd-0.0.1/PKG-INFO

@@ -0,0 +1,89 @@
+Metadata-Version: 2.4
+Name: pdf2dotmd
+Version: 0.0.1
+Summary: A Python tool for converting PDF files to Markdown
+Author: hnrobert
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/HNRobert/pdf2dotmd
+Project-URL: Repository, https://github.com/HNRobert/pdf2dotmd
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Text Processing :: Markup
+Classifier: Topic :: Utilities
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pdfplumber>=0.11.0
+Dynamic: license-file
+
+# pdf2dotmd
+
+A Python CLI tool that converts PDF files to Markdown format with intelligent layout analysis.
+
+## Features
+
+- **Layout-aware text extraction** — reconstructs logical reading order from PDF spatial data
+- **Multi-column detection** — handles two-column and multi-column layouts
+- **Table extraction** — converts PDF tables to Markdown pipe tables
+- **Heading inference** — detects headings from font size hierarchy
+- **Header/footer filtering** — automatically removes repeated page headers and footers
+- **Image extraction** — extracts embedded images to an `assets/` directory
+- **Ignore images mode** — `--ignore-images` flag for text-only output
+- **Page range selection** — convert specific pages only
+- **Batch conversion** — process multiple PDF files with wildcards
+
+## Installation
+
+```bash
+pip install pdf2dotmd
+```
+
+## Usage
+
+```bash
+# Output to stdout
+pdf2dotmd input.pdf
+
+# Output to file
+pdf2dotmd input.pdf -o output.md
+
+# Skip images, output single Markdown file
+pdf2dotmd input.pdf --ignore-images
+
+# Batch conversion
+pdf2dotmd *.pdf -o output_dir/
+
+# Convert only specific pages
+pdf2dotmd input.pdf -p 1-3
+pdf2dotmd input.pdf -p 1-5,8,10-12
+
+# Verbose logging
+pdf2dotmd input.pdf -v
+```
+
+## How It Works
+
+1. **Character extraction** — uses [pdfplumber](https://github.com/jsvine/pdfplumber) to extract individual characters with position data
+2. **Line grouping** — clusters characters into text lines by y-coordinate proximity
+3. **Block formation** — groups lines into paragraphs based on horizontal alignment and vertical spacing
+4. **Column detection** — identifies multi-column layouts by analyzing horizontal text density gaps
+5. **Reading order** — sorts blocks top-to-bottom, left-to-right, handling spanning titles
+6. **Header/footer removal** — detects repeated elements across pages
+7. **Heading inference** — maps font sizes to heading levels (H1-H6)
+
+## Limitations
+
+- **Scanned PDFs** — OCR is not supported; scanned/image-only PDFs will produce empty output
+- **Encrypted PDFs** — password-protected PDFs are not supported
+- **Complex layouts** — highly irregular layouts may not parse perfectly
+
+## License
+
+MIT

pdf2dotmd-0.0.1/README.md

@@ -0,0 +1,64 @@
+# pdf2dotmd
+
+A Python CLI tool that converts PDF files to Markdown format with intelligent layout analysis.
+
+## Features
+
+- **Layout-aware text extraction** — reconstructs logical reading order from PDF spatial data
+- **Multi-column detection** — handles two-column and multi-column layouts
+- **Table extraction** — converts PDF tables to Markdown pipe tables
+- **Heading inference** — detects headings from font size hierarchy
+- **Header/footer filtering** — automatically removes repeated page headers and footers
+- **Image extraction** — extracts embedded images to an `assets/` directory
+- **Ignore images mode** — `--ignore-images` flag for text-only output
+- **Page range selection** — convert specific pages only
+- **Batch conversion** — process multiple PDF files with wildcards
+
+## Installation
+
+```bash
+pip install pdf2dotmd
+```
+
+## Usage
+
+```bash
+# Output to stdout
+pdf2dotmd input.pdf
+
+# Output to file
+pdf2dotmd input.pdf -o output.md
+
+# Skip images, output single Markdown file
+pdf2dotmd input.pdf --ignore-images
+
+# Batch conversion
+pdf2dotmd *.pdf -o output_dir/
+
+# Convert only specific pages
+pdf2dotmd input.pdf -p 1-3
+pdf2dotmd input.pdf -p 1-5,8,10-12
+
+# Verbose logging
+pdf2dotmd input.pdf -v
+```
+
+## How It Works
+
+1. **Character extraction** — uses [pdfplumber](https://github.com/jsvine/pdfplumber) to extract individual characters with position data
+2. **Line grouping** — clusters characters into text lines by y-coordinate proximity
+3. **Block formation** — groups lines into paragraphs based on horizontal alignment and vertical spacing
+4. **Column detection** — identifies multi-column layouts by analyzing horizontal text density gaps
+5. **Reading order** — sorts blocks top-to-bottom, left-to-right, handling spanning titles
+6. **Header/footer removal** — detects repeated elements across pages
+7. **Heading inference** — maps font sizes to heading levels (H1-H6)
+
+## Limitations
+
+- **Scanned PDFs** — OCR is not supported; scanned/image-only PDFs will produce empty output
+- **Encrypted PDFs** — password-protected PDFs are not supported
+- **Complex layouts** — highly irregular layouts may not parse perfectly
+
+## License
+
+MIT

pdf2dotmd-0.0.1/pdf2dotmd/__init__.py

@@ -0,0 +1,3 @@
+"""pdf2md package."""
+
+__version__ = "0.0.1"

pdf2dotmd-0.0.1/pdf2dotmd/cli.py

@@ -0,0 +1,100 @@
+"""Command line interface for PDF to Markdown converter."""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import os
+import sys
+from glob import glob
+from pathlib import Path
+
+from .converter import PdfToMarkdownConverter
+
+logger = logging.getLogger(__name__)
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Convert PDF files to Markdown format",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Example usage:
+  %(prog)s input.pdf                      # Output to stdout
+  %(prog)s input.pdf -o output.md         # Output to file
+  %(prog)s input.pdf --ignore-images      # Skip images, single file output
+  %(prog)s *.pdf -o output_dir/           # Batch conversion
+  %(prog)s input.pdf -p 1-3              # Convert only pages 1-3
+        """,
+    )
+
+    parser.add_argument(
+        "input_files",
+        nargs="+",
+        help="Input PDF file paths (supports wildcards)",
+    )
+    parser.add_argument("-o", "--output", help="Output file or directory path")
+    parser.add_argument(
+        "--ignore-images",
+        "--no-images",
+        action="store_true",
+        dest="ignore_images",
+        help="Ignore all images and output a single Markdown file",
+    )
+    parser.add_argument(
+        "-p",
+        "--pages",
+        help="Page range to convert (e.g., '1-5,8,10-12')",
+    )
+    parser.add_argument("-v", "--verbose", action="store_true", help="Show verbose logs")
+
+    args = parser.parse_args()
+
+    logging.basicConfig(
+        level=logging.DEBUG if args.verbose else logging.INFO,
+        format="%(asctime)s - %(levelname)s - %(message)s",
+    )
+
+    converter = PdfToMarkdownConverter()
+
+    try:
+        for input_pattern in args.input_files:
+            matching_files = glob(input_pattern)
+            if not matching_files:
+                logger.warning("No matching files found: %s", input_pattern)
+                continue
+
+            for file_path in matching_files:
+                if not file_path.lower().endswith(".pdf"):
+                    logger.warning("Skipping non-PDF file: %s", file_path)
+                    continue
+
+                output_path = None
+                if args.output:
+                    if os.path.isdir(args.output) or args.output.endswith("/"):
+                        output_path = os.path.join(
+                            args.output, f"{Path(file_path).stem}.md"
+                        )
+                    else:
+                        output_path = args.output
+
+                markdown_content = converter.convert_file(
+                    file_path,
+                    output_path=output_path,
+                    ignore_images=args.ignore_images,
+                    pages=args.pages,
+                )
+
+                if not output_path:
+                    print(f"\n=== {file_path} ===\n")
+                    print(markdown_content)
+    except KeyboardInterrupt:
+        logger.info("Conversion interrupted by user")
+        sys.exit(1)
+    except Exception as exc:  # pylint: disable=broad-except
+        logger.error("Program execution failed: %s", exc)
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()

pdf2dotmd-0.0.1/pdf2dotmd/converter.py

@@ -0,0 +1,172 @@
+"""Core converter module for PDF to Markdown conversion."""
+
+from __future__ import annotations
+
+import logging
+import os
+from pathlib import Path
+from typing import Optional
+
+try:
+    import pdfplumber  # type: ignore[import-not-found]
+except ImportError:  # pragma: no cover
+    pdfplumber = None  # type: ignore[assignment]
+
+from .image_extractor import ImageExtractor
+from .layout_analyzer import LayoutAnalyzer
+from .page_processor import PageProcessor
+from .table_processor import TableProcessor
+from .utils import clean_markdown_content
+
+logger = logging.getLogger(__name__)
+
+
+def _parse_page_range(page_spec: str, total_pages: int) -> list[int]:
+    """Parse a page range string like '1-5,8,10-12' into 0-based page indices."""
+    indices: list[int] = []
+    for part in page_spec.split(","):
+        part = part.strip()
+        if "-" in part:
+            start, end = part.split("-", 1)
+            start = int(start.strip())
+            end = int(end.strip())
+            for p in range(start, end + 1):
+                if 1 <= p <= total_pages:
+                    indices.append(p - 1)
+        else:
+            p = int(part)
+            if 1 <= p <= total_pages:
+                indices.append(p - 1)
+    return sorted(set(indices))
+
+
+class PdfToMarkdownConverter:
+    """PDF to Markdown converter."""
+
+    def __init__(self):
+        self.output_folder: str = ""
+        self.assets_dir: str = ""
+
+    def convert_file(
+        self,
+        input_path: str,
+        output_path: Optional[str] = None,
+        ignore_images: bool = False,
+        pages: Optional[str] = None,
+    ) -> str:
+        if not os.path.exists(input_path):
+            raise FileNotFoundError(f"Input file does not exist: {input_path}")
+
+        if not input_path.lower().endswith(".pdf"):
+            raise ValueError(f"Only .pdf is supported: {input_path}")
+
+        if pdfplumber is None:
+            raise RuntimeError(
+                "Missing required dependency 'pdfplumber'. Please run: pip install pdfplumber"
+            )
+
+        self._setup_output_structure(input_path, output_path, ignore_images)
+
+        layout_analyzer = LayoutAnalyzer()
+        table_processor = TableProcessor()
+        image_extractor = (
+            ImageExtractor(self.assets_dir) if not ignore_images and self.assets_dir else None
+        )
+        page_processor = PageProcessor(
+            layout_analyzer=layout_analyzer,
+            table_processor=table_processor,
+            image_extractor=image_extractor,
+            ignore_images=ignore_images,
+        )
+
+        output_lines: list[str] = []
+
+        with pdfplumber.open(input_path) as pdf:
+            total_pages = len(pdf.pages)
+
+            if total_pages == 0:
+                logger.warning("PDF has no pages: %s", input_path)
+                markdown_content = "\n"
+                self._write_output(markdown_content, self._get_final_output_path(input_path, output_path))
+                return markdown_content
+
+            # Determine page indices
+            if pages:
+                page_indices = _parse_page_range(pages, total_pages)
+                if not page_indices:
+                    raise ValueError(f"No valid pages in range '{pages}' (total: {total_pages})")
+            else:
+                page_indices = list(range(total_pages))
+
+            # Check if the PDF is scanned (no text on any page)
+            has_text = False
+            for idx in page_indices:
+                if pdf.pages[idx].chars:
+                    has_text = True
+                    break
+            if not has_text:
+                logger.warning(
+                    "PDF appears to have no text layer (possibly scanned). "
+                    "OCR is not supported. Output may be empty."
+                )
+
+            for idx in page_indices:
+                page = pdf.pages[idx]
+                page_number = idx + 1
+                logger.debug("Processing page %d/%d", page_number, total_pages)
+
+                page_lines = page_processor.process_page(page, page_number)
+                output_lines.extend(page_lines)
+
+        markdown_content = clean_markdown_content(output_lines)
+        final_output_path = self._get_final_output_path(input_path, output_path)
+        self._write_output(markdown_content, final_output_path)
+        self._cleanup_empty_assets_dir()
+
+        logger.info("Conversion completed, output file: %s", final_output_path)
+        return markdown_content
+
+    def _setup_output_structure(
+        self, input_path: str, output_path: Optional[str], ignore_images: bool
+    ):
+        input_stem = Path(input_path).stem
+
+        if output_path:
+            if os.path.isdir(output_path) or output_path.endswith("/"):
+                self.output_folder = os.path.join(output_path, input_stem)
+            else:
+                self.output_folder = os.path.dirname(output_path)
+                if not self.output_folder:
+                    self.output_folder = input_stem
+        else:
+            self.output_folder = input_stem
+
+        os.makedirs(self.output_folder, exist_ok=True)
+
+        if ignore_images:
+            self.assets_dir = ""
+        else:
+            self.assets_dir = os.path.join(self.output_folder, "assets")
+            os.makedirs(self.assets_dir, exist_ok=True)
+
+    def _get_final_output_path(self, input_path: str, output_path: Optional[str]) -> str:
+        input_stem = Path(input_path).stem
+
+        if output_path:
+            if os.path.isdir(output_path) or output_path.endswith("/"):
+                return os.path.join(self.output_folder, f"{input_stem}.md")
+            return output_path
+
+        return os.path.join(self.output_folder, f"{input_stem}.md")
+
+    def _write_output(self, content: str, output_path: str):
+        output_dir = os.path.dirname(output_path)
+        if output_dir and not os.path.exists(output_dir):
+            os.makedirs(output_dir)
+
+        with open(output_path, "w", encoding="utf-8") as f:
+            f.write(content)
+
+    def _cleanup_empty_assets_dir(self):
+        if self.assets_dir and os.path.exists(self.assets_dir) and not os.listdir(self.assets_dir):
+            os.rmdir(self.assets_dir)

pdf2dotmd-0.0.1/pdf2dotmd/image_extractor.py

@@ -0,0 +1,110 @@
+"""Image extraction from PDF pages."""
+
+from __future__ import annotations
+
+import logging
+import os
+from pathlib import Path
+
+logger = logging.getLogger(__name__)
+
+
+class ImageExtractor:
+    """Extract images from PDF pages into an assets directory."""
+
+    def __init__(self, assets_dir: str):
+        self.assets_dir = assets_dir
+        self._image_count = 0
+
+    def extract_images(self, page, page_number: int) -> list[tuple[str, tuple[float, float, float, float]]]:
+        """Extract images from a pdfplumber page.
+
+        Returns list of (markdown_path, bounding_box) tuples.
+        """
+        results: list[tuple[str, tuple[float, float, float, float]]] = []
+
+        try:
+            images = page.images
+        except Exception as exc:
+            logger.warning("Failed to get images from page %d: %s", page_number, exc)
+            return results
+
+        for img_info in images:
+            self._image_count += 1
+            bbox = (img_info["x0"], img_info["top"], img_info["x1"], img_info["bottom"])
+
+            # Try to extract the actual image data
+            image_path = self._save_image_from_page(page, page_number, self._image_count, img_info)
+            if image_path:
+                results.append((image_path, bbox))
+
+        return results
+
+    def _save_image_from_page(
+        self, page, page_number: int, image_index: int, img_info: dict
+    ) -> str:
+        """Try to extract and save an image, return relative markdown path or empty string."""
+        try:
+            # Access underlying pdfminer page for image XObjects
+            pdfminer_page = page.page
+            resources = pdfminer_page.resources
+
+            if resources is None:
+                return ""
+
+            xobjects = resources.get("XObject", {})
+            if not xobjects:
+                return ""
+
+            # Try to find the image by iterating XObjects
+            for obj_name in xobjects:
+                try:
+                    xobj = xobjects[obj_name].resolve()
+                    if xobj.get("Subtype") != "Image":
+                        continue
+
+                    width = int(xobj.get("Width", 0))
+                    height = int(xobj.get("Height", 0))
+                    if width <= 0 or height <= 0:
+                        continue
+
+                    # Determine format
+                    color_space = xobj.get("ColorSpace", "")
+                    filters = xobj.get("Filter", "")
+
+                    if isinstance(filters, list):
+                        has_jpeg = any("DCT" in str(f) for f in filters)
+                        ext = "jpg" if has_jpeg else "png"
+                    elif "DCT" in str(filters):
+                        ext = "jpg"
+                    else:
+                        ext = "png"
+
+                    filename = f"page{page_number:03d}_img{image_index:02d}.{ext}"
+                    os.makedirs(self.assets_dir, exist_ok=True)
+                    output_path = Path(self.assets_dir) / filename
+
+                    # Extract raw stream data
+                    stream = xobj.get_data()
+                    output_path.write_bytes(stream)
+
+                    return f"assets/{filename}"
+                except Exception:
+                    continue
+
+            # Fallback: just record image position without data
+            logger.debug(
+                "Could not extract image data for image %d on page %d",
+                image_index,
+                page_number,
+            )
+            return ""
+
+        except Exception as exc:
+            logger.warning(
+                "Failed to extract image %d on page %d: %s",
+                image_index,
+                page_number,
+                exc,
+            )
+            return ""