markitdown-ocr 0.1.0__tar.gz → 0.1.1__tar.gz

{markitdown_ocr-0.1.0 → markitdown_ocr-0.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: markitdown-ocr
-Version: 0.1.0
+Version: 0.1.1
 Summary: OCR plugin for MarkItDown - Extracts text from images in PDF, DOCX, PPTX, and XLSX via LLM Vision
 Project-URL: Documentation, https://github.com/microsoft/markitdown#readme
 Project-URL: Issues, https://github.com/microsoft/markitdown/issues
@@ -18,7 +18,7 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: Implementation :: CPython
 Requires-Python: >=3.10
 Requires-Dist: mammoth~=1.11.0
-Requires-Dist: markitdown>=0.1.0
+Requires-Dist: markitdown<0.2.0,>=0.1.8
 Requires-Dist: openpyxl
 Requires-Dist: pandas
 Requires-Dist: pdfminer-six>=20251230
@@ -47,6 +47,8 @@ Uses the same `llm_client` / `llm_model` pattern that MarkItDown already support
 
 ## Installation
 
+Requires `markitdown>=0.1.8,<0.2.0`, which provides the Office image-rendering hooks used by this plugin. Installing the plugin automatically resolves a compatible core version.
+
 ```bash
 pip install markitdown-ocr
 ```
@@ -130,9 +132,11 @@ When a file is converted:
 1. The OCR converter accepts the file
 2. It extracts embedded images from the document
 3. Each image is sent to the LLM with an extraction prompt
-4. The returned text is inserted inline, preserving document structure
+4. The returned text is placed alongside document content (XLSX images follow their sheet's table)
 5. If the LLM call fails, conversion continues without that image's text
 
+The DOCX, PPTX, and XLSX converters subclass their core counterparts and override the same semi-private `_image_to_html` method. Core handles native content, preprocessing, and placement; the plugin supplies escaped OCR HTML, which passes through the shared HTML-to-Markdown renderer. PDF uses its separate existing pipeline.
+
 ## Supported File Formats
 
 ### PDF
@@ -143,21 +147,23 @@ When a file is converted:
 
 ### DOCX
 
-- Images are extracted via document part relationships (`doc.part.rels`).
-- OCR is run before the DOCX→HTML→Markdown pipeline executes: placeholder tokens are injected into the HTML so that the markdown converter does not escape the OCR markers, and the final placeholders are replaced with the formatted `*[Image OCR]...[End OCR]*` blocks after conversion.
-- Document flow (headings, paragraphs, tables) is fully preserved around the OCR blocks.
+- Inherits core DOCX preprocessing, styles, math, and Mammoth conversion.
+- Mammoth provides each embedded image to `_image_to_html`. OCR fragments are inserted into the document's HTML before Markdown rendering, not substituted into finished Markdown.
+- Block fragments split enclosing paragraphs where necessary and remain inside their table cell or list item. Table-cell line breaks follow the shared HTML converter's existing limitations.
 
 ### PPTX
 
 - Picture shapes, placeholder shapes with images, and images inside groups are all supported.
-- Shapes are processed in top-to-left reading order per slide.
+- Inherits core shape ordering, native text, tables, charts, and speaker notes.
+- Slide content now uses the core converter's real line breaks rather than the old plugin's literal `\n` text, and inherits its empty-title and empty-notes handling.
 - If an `llm_client` is configured, the LLM is asked for a description first; OCR is used as the fallback when no description is returned.
 
 ### XLSX
 
-- Images embedded in worksheets (`sheet._images`) are extracted per sheet.
-- Cell position is calculated from the image anchor coordinates (column/row → Excel letter notation).
+- Inherits core workbook repair and table rendering; images are read from the same repaired workbook.
 - Images are listed under a `### Images in this sheet:` section after the sheet's data table — they are not interleaved into the table rows.
+- Sheet heading spacing follows the core converter; no new cell-position labels are added.
+- Legacy `.xls` files remain handled by the existing core converter, without image OCR.
 
 ### Output format
 
@@ -169,6 +175,10 @@ Every extracted OCR block is wrapped as:
 [End OCR]*
 ```
 
+For Office formats, recognized text is escaped as literal HTML text before Markdown rendering. Markdown escaping and line breaks therefore follow the shared HTML converter: for example, underscores may be backslash-escaped, and direct converter results use Markdown hard breaks. `MarkItDown` subsequently strips trailing whitespace from each output line. Empty recognition retains the native image representation (XLSX normally omits images).
+
+Repeated image bytes are recognized once per conversion, while the result is placed at every occurrence. The cache is not shared across documents or service overrides.
+
 ## Troubleshooting
 
 ### OCR text missing from output
@@ -198,6 +208,8 @@ markitdown --list-plugins   # should show: ocr
 
 The plugin propagates LLM API errors as warnings and continues conversion. Check your API key, quota, and that the chosen model supports vision inputs.
 
+For Office OCR, a service-reported error emits a warning and retains native image rendering. Exceptions raised by custom OCR services propagate from direct converter calls; `MarkItDown` can retry another applicable converter through its normal fallback behavior.
+
 ## Development
 
 ### Running Tests
@@ -211,8 +223,8 @@ pytest tests/ -v
 
 ```bash
 git clone https://github.com/microsoft/markitdown.git
-cd markitdown/packages/markitdown-ocr
-pip install -e .
+cd markitdown
+pip install -e 'packages/markitdown[docx,pptx,xlsx]' -e packages/markitdown-ocr
 ```
 
 ## Contributing

{markitdown_ocr-0.1.0 → markitdown_ocr-0.1.1}/README.md

@@ -14,6 +14,8 @@ Uses the same `llm_client` / `llm_model` pattern that MarkItDown already support
 
 ## Installation
 
+Requires `markitdown>=0.1.8,<0.2.0`, which provides the Office image-rendering hooks used by this plugin. Installing the plugin automatically resolves a compatible core version.
+
 ```bash
 pip install markitdown-ocr
 ```
@@ -97,9 +99,11 @@ When a file is converted:
 1. The OCR converter accepts the file
 2. It extracts embedded images from the document
 3. Each image is sent to the LLM with an extraction prompt
-4. The returned text is inserted inline, preserving document structure
+4. The returned text is placed alongside document content (XLSX images follow their sheet's table)
 5. If the LLM call fails, conversion continues without that image's text
 
+The DOCX, PPTX, and XLSX converters subclass their core counterparts and override the same semi-private `_image_to_html` method. Core handles native content, preprocessing, and placement; the plugin supplies escaped OCR HTML, which passes through the shared HTML-to-Markdown renderer. PDF uses its separate existing pipeline.
+
 ## Supported File Formats
 
 ### PDF
@@ -110,21 +114,23 @@ When a file is converted:
 
 ### DOCX
 
-- Images are extracted via document part relationships (`doc.part.rels`).
-- OCR is run before the DOCX→HTML→Markdown pipeline executes: placeholder tokens are injected into the HTML so that the markdown converter does not escape the OCR markers, and the final placeholders are replaced with the formatted `*[Image OCR]...[End OCR]*` blocks after conversion.
-- Document flow (headings, paragraphs, tables) is fully preserved around the OCR blocks.
+- Inherits core DOCX preprocessing, styles, math, and Mammoth conversion.
+- Mammoth provides each embedded image to `_image_to_html`. OCR fragments are inserted into the document's HTML before Markdown rendering, not substituted into finished Markdown.
+- Block fragments split enclosing paragraphs where necessary and remain inside their table cell or list item. Table-cell line breaks follow the shared HTML converter's existing limitations.
 
 ### PPTX
 
 - Picture shapes, placeholder shapes with images, and images inside groups are all supported.
-- Shapes are processed in top-to-left reading order per slide.
+- Inherits core shape ordering, native text, tables, charts, and speaker notes.
+- Slide content now uses the core converter's real line breaks rather than the old plugin's literal `\n` text, and inherits its empty-title and empty-notes handling.
 - If an `llm_client` is configured, the LLM is asked for a description first; OCR is used as the fallback when no description is returned.
 
 ### XLSX
 
-- Images embedded in worksheets (`sheet._images`) are extracted per sheet.
-- Cell position is calculated from the image anchor coordinates (column/row → Excel letter notation).
+- Inherits core workbook repair and table rendering; images are read from the same repaired workbook.
 - Images are listed under a `### Images in this sheet:` section after the sheet's data table — they are not interleaved into the table rows.
+- Sheet heading spacing follows the core converter; no new cell-position labels are added.
+- Legacy `.xls` files remain handled by the existing core converter, without image OCR.
 
 ### Output format
 
@@ -136,6 +142,10 @@ Every extracted OCR block is wrapped as:
 [End OCR]*
 ```
 
+For Office formats, recognized text is escaped as literal HTML text before Markdown rendering. Markdown escaping and line breaks therefore follow the shared HTML converter: for example, underscores may be backslash-escaped, and direct converter results use Markdown hard breaks. `MarkItDown` subsequently strips trailing whitespace from each output line. Empty recognition retains the native image representation (XLSX normally omits images).
+
+Repeated image bytes are recognized once per conversion, while the result is placed at every occurrence. The cache is not shared across documents or service overrides.
+
 ## Troubleshooting
 
 ### OCR text missing from output
@@ -165,6 +175,8 @@ markitdown --list-plugins   # should show: ocr
 
 The plugin propagates LLM API errors as warnings and continues conversion. Check your API key, quota, and that the chosen model supports vision inputs.
 
+For Office OCR, a service-reported error emits a warning and retains native image rendering. Exceptions raised by custom OCR services propagate from direct converter calls; `MarkItDown` can retry another applicable converter through its normal fallback behavior.
+
 ## Development
 
 ### Running Tests
@@ -178,8 +190,8 @@ pytest tests/ -v
 
 ```bash
 git clone https://github.com/microsoft/markitdown.git
-cd markitdown/packages/markitdown-ocr
-pip install -e .
+cd markitdown
+pip install -e 'packages/markitdown[docx,pptx,xlsx]' -e packages/markitdown-ocr
 ```
 
 ## Contributing

{markitdown_ocr-0.1.0 → markitdown_ocr-0.1.1}/pyproject.toml

@@ -25,7 +25,7 @@ classifiers = [
 
 # Core dependencies — matches the file-format libraries markitdown already uses
 dependencies = [
-  "markitdown>=0.1.0",
+  "markitdown>=0.1.8,<0.2.0",
   "pdfminer.six>=20251230",
   "pdfplumber>=0.11.9",
   "PyMuPDF>=1.24.0",

{markitdown_ocr-0.1.0 → markitdown_ocr-0.1.1}/src/markitdown_ocr/__about__.py

@@ -1,4 +1,4 @@
 # SPDX-FileCopyrightText: 2025-present Contributors
 # SPDX-License-Identifier: MIT
 
-__version__ = "0.1.0"
+__version__ = "0.1.1"

markitdown_ocr-0.1.1/src/markitdown_ocr/_docx_converter_with_ocr.py

@@ -0,0 +1,71 @@
+"""DOCX image OCR using the core document conversion pipeline."""
+
+import hashlib
+import html
+from typing import Any, BinaryIO, Optional
+from warnings import warn
+
+from markitdown import DocumentConverterResult, StreamInfo
+from markitdown.converters import DocxConverter
+
+from ._ocr_service import LLMVisionOCRService, _extract_text_with_metadata
+
+
+class DocxConverterWithOCR(DocxConverter):
+    """Recognize embedded images while inheriting native DOCX conversion."""
+
+    def __init__(self, ocr_service: Optional[LLMVisionOCRService] = None):
+        super().__init__()
+        if not hasattr(DocxConverter, "_image_to_html"):
+            raise RuntimeError(
+                "DOCX OCR requires markitdown>=0.1.8b3 for the "
+                "DocxConverter._image_to_html hook. "
+                "Upgrade with: pip install --upgrade 'markitdown>=0.1.8b3'."
+            )
+        self.ocr_service = ocr_service
+
+    def convert(
+        self,
+        file_stream: BinaryIO,
+        stream_info: StreamInfo,
+        **kwargs: Any,
+    ) -> DocumentConverterResult:
+        # Keep repeated-image recognition local to this document, not the instance.
+        kwargs["_docx_ocr_cache"] = {}
+        return super().convert(file_stream, stream_info, **kwargs)
+
+    def _image_to_html(
+        self,
+        image_stream: BinaryIO,
+        stream_info: StreamInfo,
+        **kwargs: Any,
+    ) -> Optional[str]:
+        ocr_service = kwargs.get("ocr_service") or self.ocr_service
+        if ocr_service is None:
+            return None
+
+        cache: dict[bytes, Optional[str]] = kwargs.get("_docx_ocr_cache", {})
+        key = hashlib.sha256(image_stream.read()).digest()
+        image_stream.seek(0)
+        if key in cache:
+            return cache[key]
+
+        result = _extract_text_with_metadata(ocr_service, image_stream, stream_info)
+        if result.error:
+            warn(
+                f"DOCX image OCR failed: {result.error}. Keeping the native image.",
+                RuntimeWarning,
+                stacklevel=2,
+            )
+            cache[key] = None
+            return None
+        text = result.text.strip()
+        if not text:
+            cache[key] = None
+            return None
+
+        text = text.replace("\r\n", "\n").replace("\r", "\n")
+        content = html.escape(text).replace("\n", "<br>")
+        fragment = f"<p><em>[Image OCR]<br>{content}<br>[End OCR]</em></p>"
+        cache[key] = fragment
+        return fragment

{markitdown_ocr-0.1.0 → markitdown_ocr-0.1.1}/src/markitdown_ocr/_ocr_service.py

@@ -4,6 +4,7 @@ Provides LLM Vision-based image text extraction.
 """
 
 import base64
+import inspect
 from typing import Any, BinaryIO
 from dataclasses import dataclass
 
@@ -108,3 +109,19 @@ class LLMVisionOCRService:
             return OCRResult(text="", backend_used="llm_vision", error=str(e))
         finally:
             image_stream.seek(0)
+
+
+def _extract_text_with_metadata(
+    ocr_service: LLMVisionOCRService,
+    image_stream: BinaryIO,
+    stream_info: StreamInfo,
+) -> OCRResult:
+    """Forward metadata when supported, retaining the legacy stream-only call."""
+    extract_text = ocr_service.extract_text
+    try:
+        inspect.signature(extract_text).bind(image_stream, stream_info=stream_info)
+    except (TypeError, ValueError):
+        # Unsupported or uninspectable signatures keep the legacy invocation.
+        return extract_text(image_stream)
+    # Keep service errors outside the signature check; never retry an OCR call.
+    return extract_text(image_stream, stream_info=stream_info)

markitdown_ocr-0.1.1/src/markitdown_ocr/_pptx_converter_with_ocr.py

@@ -0,0 +1,70 @@
+"""PPTX image OCR using the core presentation conversion pipeline."""
+
+import hashlib
+import html
+from typing import Any, BinaryIO, Optional
+from warnings import warn
+
+from markitdown import DocumentConverterResult, StreamInfo
+from markitdown.converters import PptxConverter
+
+from ._ocr_service import LLMVisionOCRService, _extract_text_with_metadata
+
+
+class PptxConverterWithOCR(PptxConverter):
+    """Recognize embedded images while inheriting native PPTX conversion."""
+
+    def __init__(self, ocr_service: Optional[LLMVisionOCRService] = None):
+        super().__init__()
+        if not hasattr(PptxConverter, "_image_to_html"):
+            raise RuntimeError(
+                "PPTX OCR requires markitdown>=0.1.8b3 for the "
+                "PptxConverter._image_to_html hook. "
+                "Upgrade with: pip install --upgrade 'markitdown>=0.1.8b3'."
+            )
+        self.ocr_service = ocr_service
+
+    def convert(
+        self,
+        file_stream: BinaryIO,
+        stream_info: StreamInfo,
+        **kwargs: Any,
+    ) -> DocumentConverterResult:
+        kwargs["_pptx_ocr_cache"] = {}
+        return super().convert(file_stream, stream_info, **kwargs)
+
+    def _image_to_html(
+        self,
+        image_stream: BinaryIO,
+        stream_info: StreamInfo,
+        **kwargs: Any,
+    ) -> Optional[str]:
+        ocr_service = kwargs.get("ocr_service") or self.ocr_service
+        if ocr_service is None:
+            return None
+
+        cache: dict[bytes, Optional[str]] = kwargs.get("_pptx_ocr_cache", {})
+        key = hashlib.sha256(image_stream.read()).digest()
+        image_stream.seek(0)
+        if key in cache:
+            return cache[key]
+
+        result = _extract_text_with_metadata(ocr_service, image_stream, stream_info)
+        if result.error:
+            warn(
+                f"PPTX image OCR failed: {result.error}. Keeping the native image.",
+                RuntimeWarning,
+                stacklevel=2,
+            )
+            cache[key] = None
+            return None
+        text = result.text.strip()
+        if not text:
+            cache[key] = None
+            return None
+
+        text = text.replace("\r\n", "\n").replace("\r", "\n")
+        content = html.escape(text).replace("\n", "<br>")
+        fragment = f"<p><em>[Image OCR]<br>{content}<br>[End OCR]</em></p>"
+        cache[key] = fragment
+        return fragment

markitdown_ocr-0.1.1/src/markitdown_ocr/_xlsx_converter_with_ocr.py

@@ -0,0 +1,70 @@
+"""XLSX image OCR using the core spreadsheet conversion pipeline."""
+
+import hashlib
+import html
+from typing import Any, BinaryIO, Optional
+from warnings import warn
+
+from markitdown import DocumentConverterResult, StreamInfo
+from markitdown.converters import XlsxConverter
+
+from ._ocr_service import LLMVisionOCRService, _extract_text_with_metadata
+
+
+class XlsxConverterWithOCR(XlsxConverter):
+    """Recognize embedded images while inheriting native XLSX conversion."""
+
+    def __init__(self, ocr_service: Optional[LLMVisionOCRService] = None):
+        super().__init__()
+        if not hasattr(XlsxConverter, "_image_to_html"):
+            raise RuntimeError(
+                "XLSX OCR requires markitdown>=0.1.8b3 for the "
+                "XlsxConverter._image_to_html hook. "
+                "Upgrade with: pip install --upgrade 'markitdown>=0.1.8b3'."
+            )
+        self.ocr_service = ocr_service
+
+    def convert(
+        self,
+        file_stream: BinaryIO,
+        stream_info: StreamInfo,
+        **kwargs: Any,
+    ) -> DocumentConverterResult:
+        kwargs["_xlsx_ocr_cache"] = {}
+        return super().convert(file_stream, stream_info, **kwargs)
+
+    def _image_to_html(
+        self,
+        image_stream: BinaryIO,
+        stream_info: StreamInfo,
+        **kwargs: Any,
+    ) -> Optional[str]:
+        ocr_service = kwargs.get("ocr_service") or self.ocr_service
+        if ocr_service is None:
+            return None
+
+        cache: dict[bytes, Optional[str]] = kwargs.get("_xlsx_ocr_cache", {})
+        key = hashlib.sha256(image_stream.read()).digest()
+        image_stream.seek(0)
+        if key in cache:
+            return cache[key]
+
+        result = _extract_text_with_metadata(ocr_service, image_stream, stream_info)
+        if result.error:
+            warn(
+                f"XLSX image OCR failed: {result.error}. Keeping the native image.",
+                RuntimeWarning,
+                stacklevel=2,
+            )
+            cache[key] = None
+            return None
+        text = result.text.strip()
+        if not text:
+            cache[key] = None
+            return None
+
+        text = text.replace("\r\n", "\n").replace("\r", "\n")
+        content = html.escape(text).replace("\n", "<br>")
+        fragment = f"<p><em>[Image OCR]<br>{content}<br>[End OCR]</em></p>"
+        cache[key] = fragment
+        return fragment