dot-parser 2.0.0__tar.gz → 2.1.0__tar.gz

dot_parser-2.1.0/CHANGELOG.md

@@ -0,0 +1,88 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [2.1.0] - 2026-09-30
+
+### Added
+
+- `extract_docx_toc()` returns the table of contents stored in a DOCX as
+  `TocEntry(level, text)` items (`TocEntry` is exported), whether Word wrapped
+  it in a content control or inserted it as a bare field.
+- `min_image_size=N` on `parse_with_images()`, `parse_docx_with_images()` and
+  `parse_pptx_with_images()` drops images whose shorter side is under `N`
+  pixels, with their anchors: letters or words pasted as pictures, icons,
+  separator lines. On the markitdown + VLM path they are dropped before
+  interpretation, so they cost no VLM call. `0` (default) keeps every image;
+  images whose size can't be read (EMF, WMF, SVG) are always kept.
+
+### Changed
+
+- DOCX conversion (`parse()` and the markitdown + VLM image path) now runs
+  through one converter, so a DOCX yields the same text with or without image
+  extraction. It keeps markitdown's pipeline and fixes, in the Markdown it
+  produces:
+  - headings in custom styles derived from `Heading N` or carrying an outline
+    level (`Style Titre 2`, `Annexe`...) are emitted as headings instead of
+    body text;
+  - headings carry Word's automatic numbers (`# 1. Introduction`,
+    `## 2.3 Objet`), recomputed from `numbering.xml`; a number already typed
+    in the heading is not doubled;
+  - `Title` and `Subtitle` paragraphs (cover page) are rendered bold; like
+    Word's own TOC, they are not treated as headings;
+  - bold inside a heading is dropped (`## Zoom`, not `## **Zoom**`);
+  - the stored table of contents and the lists of figures and tables are
+    left out: they repeat headings and captions with page numbers, and
+    `extract_docx_toc()` exposes the TOC;
+  - tables use their first row as the header instead of a blank one, escape
+    `|` in cells, stay rectangular across merged cells (a vertically merged
+    value is repeated in each row it spans) and flatten nested tables into
+    their parent cell.
+- `parse()` drops embedded images from a DOCX instead of emitting
+  `![alt](data:image/png;base64...)` anchors with a truncated payload.
+- `parse()` raises `ParseError` on a corrupted DOCX instead of returning its
+  raw bytes as text.
+- `mammoth` and `beautifulsoup4` are declared as direct dependencies.
+
+## [2.0.0] - 2026-07-30
+
+First public release.
+
+### Added
+
+- `parse()` converts PDF, DOCX, PPTX, HTML, XLSX, CSV, Markdown and plain text
+  into clean Markdown.
+- Selectable PDF backends: `Pymu` (default, local and fast), `Docling` (local
+  layout-aware ML pipeline), `Mistral` (cloud OCR) and `Llama` (LlamaCloud).
+- `parse_with_images()` extracts images alongside the Markdown, with anchors
+  kept at their position in the text. Images are described either by a VLM or
+  by Mistral OCR annotations in the same call.
+- `parse_pdfs()` parses many PDFs in one batch job through the Mistral Batch
+  API, falling back to a per-file loop on backends without batch support.
+- `chunk()` splits Markdown into chunks carrying their heading hierarchy in
+  `section_path`.
+- Cost helpers: `cost_per_1k_pages()` and `estimate_cost()`, usable without
+  instantiating a backend.
+- Backend protocols (`Backend`, `ImageBackend`, `BatchBackend`,
+  `DocxImageBackend`, `PptxImageBackend`) so third-party backends can plug in.
+
+### Changed
+
+These matter only if you were installing the package straight from git before
+this release.
+
+- `LlamaTier` is a `Literal` of the four accepted tiers rather than a bare
+  `str`, and is defined once alongside the pricing table.
+- `zip()` calls over sequences that must line up now use `strict=True`: a length
+  mismatch raises instead of silently truncating.
+- `parse_with_images(backend=...)` accepts DOCX- and PPTX-capable backends in
+  its type signature, matching what the function already supported at runtime.
+
+[Unreleased]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-parser/-/compare/v2.1.0...main
+[2.1.0]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-parser/-/compare/v2.0.0...v2.1.0
+[2.0.0]: https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-parser/-/tags/v2.0.0

{dot_parser-2.0.0 → dot_parser-2.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: dot-parser
-Version: 2.0.0
+Version: 2.1.0
 Summary: Document-to-markdown parser and chunker for RAG pipelines
 Project-URL: Homepage, https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-parser
 Project-URL: Repository, https://gitlab.com/deepika6190303/deepika-open-toolbox/dot-parser
@@ -15,6 +15,8 @@ Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Requires-Python: <3.14,>=3.12
+Requires-Dist: beautifulsoup4>=4.12
+Requires-Dist: mammoth>=1.11
 Requires-Dist: markitdown[all]>=0.1
 Requires-Dist: pillow>=10.0
 Requires-Dist: pydantic>=2
@@ -235,7 +237,7 @@ package is covered, anything underscore-prefixed is internal and may change in
 any release. Public names are never removed without a deprecation period.
 
 ```toml
-dependencies = ["dot-parser>=1.0,<2"]
+dependencies = ["dot-parser>=2.0,<3"]
 ```
 
 See [docs/VERSIONING.md](docs/VERSIONING.md) for the full policy.

{dot_parser-2.0.0 → dot_parser-2.1.0}/README.md

@@ -199,7 +199,7 @@ package is covered, anything underscore-prefixed is internal and may change in
 any release. Public names are never removed without a deprecation period.
 
 ```toml
-dependencies = ["dot-parser>=1.0,<2"]
+dependencies = ["dot-parser>=2.0,<3"]
 ```
 
 See [docs/VERSIONING.md](docs/VERSIONING.md) for the full policy.

{dot_parser-2.0.0 → dot_parser-2.1.0}/docs/VERSIONING.md

@@ -1,7 +1,7 @@
 # Versioning and stability
 
 `dot-parser` follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
-This page says what that actually covers, so `>=1.0,<2` means something concrete.
+This page says what that actually covers, so `>=2.0,<3` means something concrete.
 
 ## What is public
 
@@ -94,11 +94,11 @@ manual. Use them to validate a release without committing to it.
 ### From PyPI — the supported way
 
 ```toml
-dependencies = ["dot-parser>=1.0,<2"]
+dependencies = ["dot-parser>=2.0,<3"]
 ```
 
 The upper bound is what makes the guarantees above useful: you get every fix and
-feature of the 1.x line, and never an unannounced break.
+feature of the 2.x line, and never an unannounced break.
 
 ### From git — development only
 

{dot_parser-2.0.0 → dot_parser-2.1.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "dot-parser"
-version = "2.0.0"
+version = "2.1.0"
 description = "Document-to-markdown parser and chunker for RAG pipelines"
 readme = "README.md"
 authors = [{ name = "Kannon For Deep Tech", email = "louis.letarnec@deepika.ai" }]
@@ -12,6 +12,10 @@ requires-python = ">=3.12,<3.14"
 dependencies = [
     "pymupdf4llm>=1.27.2.2",
     "markitdown[all]>=0.1",
+    # Imported directly by the DOCX converter (docx_markdown.py); markitdown
+    # pulls both in too, but only through its optional extras.
+    "mammoth>=1.11",
+    "beautifulsoup4>=4.12",
     "semchunk>=3.0",
     "python-docx>=1.2.0",
     "pillow>=10.0",

{dot_parser-2.0.0 → dot_parser-2.1.0}/src/dot_parser/__init__.py

@@ -5,6 +5,7 @@ from importlib.metadata import version
 
 from dot_parser.backends import Backend, Docling, ImageBackend, Llama, Mistral, Pymu
 from dot_parser.chunking import chunk
+from dot_parser.docx_markdown import extract_docx_toc
 from dot_parser.images import (
     VLM,
     ExtractedImage,
@@ -12,7 +13,7 @@ from dot_parser.images import (
     PageInfo,
     ParseResult,
 )
-from dot_parser.models import Chunk, ParseError
+from dot_parser.models import Chunk, ParseError, TocEntry
 from dot_parser.parsers import (
     interpret_images,
     parse,
@@ -39,9 +40,11 @@ __all__ = [
     "ParseError",
     "ParseResult",
     "Pymu",
+    "TocEntry",
     "chunk",
     "cost_per_1k_pages",
     "estimate_cost",
+    "extract_docx_toc",
     "interpret_images",
     "parse",
     "parse_pdfs",

{dot_parser-2.0.0 → dot_parser-2.1.0}/src/dot_parser/docx_images.py

@@ -34,7 +34,7 @@ import tempfile
 import time
 from pathlib import Path
 
-from dot_parser.image_utils import dedupe_titles
+from dot_parser.image_utils import dedupe_titles, drop_small_images
 from dot_parser.images import (
     VLM,
     ExtractedImage,
@@ -523,7 +523,15 @@ def _office_to_markdown_via_markitdown(file_path: str) -> str:
     first comma (replacing the base64 payload with ``...``) to keep
     markdown small for the LLM-summarisation use case it was designed
     for. We need the full payload to extract the image bytes.
+
+    DOCX goes through :func:`dot_parser.docx_markdown.docx_to_markdown`,
+    the same converter :func:`dot_parser.parse` uses, so the text is
+    identical with or without image extraction.
     """
+    if Path(file_path).suffix.lower() == ".docx":
+        from dot_parser.docx_markdown import docx_to_markdown
+
+        return docx_to_markdown(file_path, keep_data_uris=True)
     try:
         import markitdown
     except ImportError as e:
@@ -540,6 +548,7 @@ def _parse_office_with_images(
     suffix: str,
     include_tables: bool,
     deadline: float | None = None,
+    min_image_size: int = 0,
 ) -> ParseResult:
     """Shared markitdown+VLM pipeline for DOCX and PPTX sources."""
     if isinstance(source, bytes):
@@ -555,6 +564,8 @@ def _parse_office_with_images(
         raw_md = _office_to_markdown_via_markitdown(str(source))
 
     markdown, images = _extract_data_url_images(raw_md)
+    # Before interpretation, so dropped images cost no VLM call.
+    markdown, images = drop_small_images(markdown, images, min_image_size)
     if not include_tables:
         markdown = strip_markdown_tables(markdown)
     if images:
@@ -574,6 +585,7 @@ def parse_docx_with_images(
     *,
     include_tables: bool = True,
     deadline: float | None = None,
+    min_image_size: int = 0,
 ) -> ParseResult:
     """Parse a DOCX into markdown + interpreted images.
 
@@ -599,6 +611,7 @@ def parse_docx_with_images(
         suffix=".docx",
         include_tables=include_tables,
         deadline=deadline,
+        min_image_size=min_image_size,
     )
 
 
@@ -608,6 +621,7 @@ def parse_pptx_with_images(
     *,
     include_tables: bool = True,
     deadline: float | None = None,
+    min_image_size: int = 0,
 ) -> ParseResult:
     """Parse a PPTX into markdown + interpreted embedded pictures.
 
@@ -630,4 +644,5 @@ def parse_pptx_with_images(
         suffix=".pptx",
         include_tables=include_tables,
         deadline=deadline,
+        min_image_size=min_image_size,
     )