any2md 1.0.0__tar.gz → 1.0.1__tar.gz

{any2md-1.0.0 → any2md-1.0.1}/CHANGELOG.md

@@ -4,6 +4,28 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [1.0.1] — 2026-04-26
+
+Patch release. Adds an explicit backend-selection CLI flag.
+
+### Added
+- `--backend {docling,pymupdf4llm,mammoth}` CLI flag and corresponding
+  `PipelineOptions.backend` field. Lets users override the automatic
+  backend selection. Useful when Docling extracts incorrectly on a
+  specific input and the user wants to fall back to pymupdf4llm
+  without uninstalling Docling. Mismatched format/backend combinations
+  (e.g., `--backend pymupdf4llm` on a DOCX) error out per file.
+
+### Fixes (workaround)
+- Refs #13 — Docling drops list items on certain academic PDFs with
+  the "Parent element of the list item is not a ListGroup" warning.
+  No upstream fix yet; users hitting this can now `any2md --backend
+  pymupdf4llm doc.pdf` to use the lightweight backend instead.
+
+### Note
+- `--backend docling` is equivalent to the existing `--high-fidelity`
+  / `-H` flag (both kept). Future releases may consolidate.
+
 ## [1.0.0] — 2026-04-26
 
 First stable release of any2md v1.0. Validated end-to-end against

{any2md-1.0.0 → any2md-1.0.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: any2md
-Version: 1.0.0
+Version: 1.0.1
 Summary: Convert PDF, DOCX, HTML, and TXT files — or web pages by URL — to clean, LLM-optimized Markdown with YAML frontmatter.
 Project-URL: Homepage, https://github.com/rocklambros/any2md
 Project-URL: Issues, https://github.com/rocklambros/any2md/issues
@@ -226,7 +226,7 @@ You'd run this when your corpus has multi-column PDFs, complex tables, or scanne
 pip install "any2md[high-fidelity]"
 ```
 
-After install, the CLI auto-detects Docling and uses it for PDF and DOCX. To force it explicitly (and fail fast when it isn't installed), pass `-H` / `--high-fidelity`. To require it (treat fallback as an error), combine with `--strict`.
+After install, the CLI auto-detects Docling and uses it for PDF and DOCX. To force it explicitly (and fail fast when it isn't installed), pass `-H` / `--high-fidelity`. To require it (treat fallback as an error), combine with `--strict`. Use `--backend pymupdf4llm` to force the lightweight backend even when Docling is installed (useful when Docling produces extraction artifacts on a specific input).
 
 ### When do I need high-fidelity?
 

{any2md-1.0.0 → any2md-1.0.1}/README.md

@@ -196,7 +196,7 @@ You'd run this when your corpus has multi-column PDFs, complex tables, or scanne
 pip install "any2md[high-fidelity]"
 ```
 
-After install, the CLI auto-detects Docling and uses it for PDF and DOCX. To force it explicitly (and fail fast when it isn't installed), pass `-H` / `--high-fidelity`. To require it (treat fallback as an error), combine with `--strict`.
+After install, the CLI auto-detects Docling and uses it for PDF and DOCX. To force it explicitly (and fail fast when it isn't installed), pass `-H` / `--high-fidelity`. To require it (treat fallback as an error), combine with `--strict`. Use `--backend pymupdf4llm` to force the lightweight backend even when Docling is installed (useful when Docling produces extraction artifacts on a specific input).
 
 ### When do I need high-fidelity?
 

{any2md-1.0.0 → any2md-1.0.1}/any2md/__init__.py

@@ -1,3 +1,3 @@
 """Convert PDF, DOCX, HTML, and TXT files to LLM-optimized Markdown."""
 
-__version__ = "1.0.0"
+__version__ = "1.0.1"

{any2md-1.0.0 → any2md-1.0.1}/any2md/cli.py

@@ -132,6 +132,16 @@ def main():
         action="store_true",
         help="Force the Docling backend (PDF/DOCX). Exit 1 if not installed.",
     )
+    parser.add_argument(
+        "--backend",
+        choices=("docling", "pymupdf4llm", "mammoth"),
+        default=None,
+        help="Force a specific extraction backend. By default any2md auto-selects "
+        "(prefers Docling when installed). Use 'docling' (equivalent to --high-fidelity), "
+        "'pymupdf4llm' to force the lightweight PDF fallback even when Docling is installed, "
+        "or 'mammoth' to force the lightweight DOCX fallback. Mismatched format/backend "
+        "combinations (e.g., --backend pymupdf4llm on a DOCX) error out per file.",
+    )
     parser.add_argument(
         "--ocr-figures",
         action="store_true",
@@ -228,11 +238,16 @@ def main():
         sys.exit(1)
     overrides = _deep_merge_overrides(overrides, cli_meta)
 
-    if args.high_fidelity or args.ocr_figures or args.save_images:
+    if (
+        args.high_fidelity
+        or args.ocr_figures
+        or args.save_images
+        or args.backend == "docling"
+    ):
         from any2md._docling import has_docling, INSTALL_HINT_MSG
         if not has_docling():
             print(
-                f"  ERROR: Docling required for --high-fidelity / --ocr-figures / --save-images.\n"
+                f"  ERROR: Docling required for --high-fidelity / --ocr-figures / --save-images / --backend docling.\n"
                 f"  {INSTALL_HINT_MSG}",
                 file=sys.stderr,
             )
@@ -252,6 +267,7 @@ def main():
         auto_id_prefix=auto_id_prefix,
         auto_id_type_code=auto_id_type_code,
         frontmatter_overrides=overrides or None,
+        backend=args.backend,
     )
 
     # CLI-only output controls (not part of PipelineOptions).

{any2md-1.0.0 → any2md-1.0.1}/any2md/converters/docx.py

@@ -114,7 +114,24 @@ def convert_docx(
         return True
 
     try:
-        if has_docling():
+        # Backend override (PipelineOptions.backend). When set, honor the
+        # caller's choice instead of the auto-select. ``pymupdf4llm`` is
+        # invalid for DOCX — fail the file with a clear error.
+        if options.backend == "pymupdf4llm":
+            print(
+                f"  FAIL: {docx_path.name} -- backend 'pymupdf4llm' is not "
+                f"valid for DOCX input (pymupdf4llm processes PDF only).",
+                file=sys.stderr,
+            )
+            return False
+
+        if options.backend == "mammoth":
+            md_text, extracted_via = _extract_via_mammoth(docx_path, options)
+            lane = "text"
+        elif options.backend == "docling":
+            md_text, extracted_via = _extract_via_docling(docx_path)
+            lane = "structured"
+        elif has_docling():
             try:
                 md_text, extracted_via = _extract_via_docling(docx_path)
                 lane = "structured"

{any2md-1.0.0 → any2md-1.0.1}/any2md/converters/pdf.py

@@ -182,9 +182,31 @@ def convert_pdf(
         return True
 
     try:
-        use_docling = has_docling()
+        # Backend override (PipelineOptions.backend). When set, honor the
+        # caller's choice instead of the auto-select. ``mammoth`` is
+        # invalid for PDFs — fail the file with a clear error.
+        if options.backend == "mammoth":
+            print(
+                f"  FAIL: {pdf_path.name} -- backend 'mammoth' is not "
+                f"valid for PDF input (mammoth processes DOCX only).",
+                file=sys.stderr,
+            )
+            return False
+
+        if options.backend == "pymupdf4llm":
+            use_docling = False
+        elif options.backend == "docling":
+            # Same install-required behavior as --high-fidelity. If
+            # Docling isn't importable, fall through to the existing
+            # auto-select path which will error via _extract_via_docling
+            # below if Docling truly isn't usable. The CLI layer
+            # pre-checks install for both --high-fidelity and
+            # --backend docling, so this branch normally has Docling.
+            use_docling = has_docling()
+        else:
+            use_docling = has_docling()
 
-        if not use_docling and pdf_looks_complex(pdf_path):
+        if not use_docling and options.backend is None and pdf_looks_complex(pdf_path):
             install_hint()
 
         # Always extract metadata via PyMuPDF — independent of which

{any2md-1.0.0 → any2md-1.0.1}/any2md/pipeline/__init__.py

@@ -23,6 +23,13 @@ class PipelineOptions:
     strip_links: bool = False
     strict: bool = False
     high_fidelity: bool = False  # force Docling backend
+    # Explicit backend selection. ``None`` preserves the existing
+    # auto-select behavior (Docling when installed, else format-specific
+    # fallback). When set, the converter MUST honor the choice or fail
+    # the file with a clear error if the backend is incompatible with
+    # the input format (e.g. ``"mammoth"`` on a PDF, ``"pymupdf4llm"``
+    # on a DOCX).
+    backend: Literal["docling", "pymupdf4llm", "mammoth"] | None = None
     # --auto-id wiring. When ``auto_id`` is True, ``frontmatter.compose``
     # emits a generated ``document_id``. The prefix and type_code are
     # used as-is; Phase 4 hardcodes "LOCAL"/"DOC" and Task 5 wires

{any2md-1.0.0 → any2md-1.0.1}/docs/cli-reference.md

@@ -145,6 +145,42 @@ You'll see `extracted_via: "docling"` in the frontmatter of every PDF and DOCX
 output. Without `-H`, any2md uses Docling when it's importable and falls back
 silently otherwise.
 
+### `--backend {docling,pymupdf4llm,mammoth}`
+
+Force a specific extraction backend, overriding the automatic Docling-when-
+installed selection. `docling` is equivalent to `--high-fidelity` (and shares
+the same install pre-flight). `pymupdf4llm` forces the lightweight PDF
+fallback even when Docling is installed. `mammoth` forces the lightweight
+DOCX fallback even when Docling is installed.
+
+The backend must match the input format. Mismatches fail per file rather
+than the whole run:
+
+- `--backend pymupdf4llm` on a DOCX → that file fails with a clear message;
+  PDF inputs in the same run still convert.
+- `--backend mammoth` on a PDF → that file fails with a clear message; DOCX
+  inputs in the same run still convert.
+
+**Use this when** Docling extracts incorrectly on a specific input (e.g.
+[#13](https://github.com/rocklambros/any2md/issues/13) — Docling drops list
+items on certain academic PDFs) and you want to fall back to the lightweight
+backend without uninstalling Docling.
+
+**Don't use this when** the auto-detection is doing the right thing — the
+default behavior already prefers Docling when installed and falls back when
+it isn't.
+
+```bash
+# Skip Docling for one specific PDF that triggers a known bug
+any2md --backend pymupdf4llm research_paper.pdf
+
+# Force Docling explicitly (same as --high-fidelity)
+any2md --backend docling ./corpus
+```
+
+You'll see `extracted_via: "pymupdf4llm"` (or `"mammoth+markdownify"`, or
+`"docling"`) in the frontmatter, matching the backend you forced.
+
 ### `--ocr-figures`
 
 Run OCR on figures embedded in PDFs (Docling path). Implies `--high-fidelity`.

{any2md-1.0.0 → any2md-1.0.1}/docs/superpowers/plans/2026-04-26-any2md-v1-phase1-foundation.md

@@ -721,10 +721,11 @@ def test_source_meta_defaults_are_safe():
         title_hint=None, authors=[], organization=None, date=None,
         keywords=[], pages=None, word_count=None,
         source_file="x.txt", source_url=None,
-        extracted_via="heuristic", lane="text",
+        doc_type="txt", extracted_via="heuristic", lane="text",
     )
     assert meta.lane == "text"
     assert meta.extracted_via == "heuristic"
+    assert meta.doc_type == "txt"
 ```
 
 - [ ] **Step 2: Run test to verify it fails**
@@ -1918,11 +1919,18 @@ from any2md.pipeline import PipelineOptions
 
 
 def _split_frontmatter(text: str) -> tuple[dict, str]:
-    """Helper: split a frontmatter+body string into (yaml_dict, body_str)."""
+    """Helper: split a frontmatter+body string into (yaml_dict, body_str).
+
+    compose() emits a blank-line separator after the closing ---, so we
+    strip a single leading newline from the body to recover the body that
+    was passed into compose().
+    """
     assert text.startswith("---\n")
     end = text.index("\n---\n", 4)
     fm = yaml.safe_load(text[4:end])
     body = text[end + 5 :]
+    if body.startswith("\n"):
+        body = body[1:]
     return fm, body