pdf-strikethrough-detect 0.4.0__tar.gz

pdf_strikethrough_detect-0.4.0/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## 0.4.0 — first public release
+
+- Detect struck-through (deleted) text in born-digital PDFs (exact vector/flag detection) and scanned pages (stroke geometry + OCR + ONNX CNN), with `~~struck~~` markdown, clean text, and grouped passages via a Python API and CLI.

pdf_strikethrough_detect-0.4.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Niles Liu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pdf_strikethrough_detect-0.4.0/MANIFEST.in

@@ -0,0 +1,4 @@
+include LICENSE README.md CHANGELOG.md
+include src/pdf_strikethrough/strike_verdict_cnn.onnx
+include src/pdf_strikethrough/strike_verdict_cnn.meta.json
+recursive-exclude test_docs *

pdf_strikethrough_detect-0.4.0/PKG-INFO

@@ -0,0 +1,203 @@
+Metadata-Version: 2.4
+Name: pdf-strikethrough-detect
+Version: 0.4.0
+Summary: Detect struck-through (deleted) text in PDFs and scanned document images.
+Author: Niles Liu
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/niles-liu/pdf-strikethrough-detect
+Project-URL: Repository, https://github.com/niles-liu/pdf-strikethrough-detect
+Project-URL: Issues, https://github.com/niles-liu/pdf-strikethrough-detect/issues
+Project-URL: Changelog, https://github.com/niles-liu/pdf-strikethrough-detect/blob/main/CHANGELOG.md
+Keywords: strikethrough,strikeout,redline,ocr,pdf,document-intelligence,deleted-text,scanned-documents
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Image Recognition
+Classifier: Topic :: Text Processing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24
+Requires-Dist: pillow>=9.0
+Requires-Dist: scipy>=1.10
+Requires-Dist: onnxruntime>=1.16
+Requires-Dist: pymupdf>=1.26
+Provides-Extra: markdown
+Requires-Dist: pymupdf4llm>=0.0.27; extra == "markdown"
+Provides-Extra: rapidocr
+Requires-Dist: rapidocr>=3.2; extra == "rapidocr"
+Requires-Dist: onnxruntime>=1.16; extra == "rapidocr"
+Provides-Extra: tesseract
+Requires-Dist: pytesseract>=0.3.10; extra == "tesseract"
+Provides-Extra: torch
+Requires-Dist: torch>=2.0; extra == "torch"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Dynamic: license-file
+
+# pdf-strikethrough-detect
+
+[![PyPI](https://img.shields.io/pypi/v/pdf-strikethrough-detect.svg)](https://pypi.org/project/pdf-strikethrough-detect/)
+[![Python versions](https://img.shields.io/pypi/pyversions/pdf-strikethrough-detect.svg)](https://pypi.org/project/pdf-strikethrough-detect/)
+[![CI](https://github.com/niles-liu/pdf-strikethrough-detect/actions/workflows/ci.yml/badge.svg)](https://github.com/niles-liu/pdf-strikethrough-detect/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+
+Detect **struck-through (deleted) text** in PDFs and scanned document images.
+
+Strikethrough detection is a surprisingly unserved niche: most "redline"/diff tools assume clean
+born-digital PDFs and fall apart on scans — which is the real-world case. `pdf-strikethrough-detect`
+handles both, does the hard part (scanned images) with a tiny CPU model, and makes no cloud calls.
+
+```python
+import pdf_strikethrough as st
+
+# born-digital PDF — exact, no OCR
+for w in st.strikethroughs_in_pdf("contract.pdf"):
+    print(w["page"], repr(w["chars"]), "partial" if w["partial"] else "full")
+```
+
+## Install
+
+```bash
+pip install pdf-strikethrough-detect
+```
+
+Pure pip, no system binaries required: the CNN runs on ONNX Runtime (CPU) and the ~318 KB model
+ships inside the wheel. Extras:
+
+```bash
+pip install "pdf-strikethrough-detect[markdown]"    # clean_markdown() via pymupdf4llm
+pip install "pdf-strikethrough-detect[rapidocr]"    # free scanned-word OCR backend (no binary)
+pip install "pdf-strikethrough-detect[tesseract]"   # word-level OCR (also needs the tesseract binary)
+```
+
+## Native / born-digital PDFs — exact
+
+In a born-digital PDF a strikethrough is a *vector drawing* (a line or thin rect over the text),
+so detection is exact ground truth — no OCR, no model, no guessing. Both vector-rule and
+filled-rect strikethrough styles are handled.
+
+```python
+import pdf_strikethrough as st
+
+for w in st.strikethroughs_in_pdf("contract.pdf"):
+    print(w["page"], repr(w["chars"]))          # 'chars' = the struck substring
+print(st.clean_markdown("contract.pdf"))        # surviving text, deletions removed (needs [markdown])
+```
+
+Each record: `{page, text, chars, char_span, partial, bbox_frac, coverage, verdict, final}`.
+Partial strikes (`semi-` of `semi-monthly`) are resolved to a char range. `bbox_frac` is in
+fractions of the rendered page (rotation-aware), so it maps directly onto a rendered pixmap.
+
+Two native detectors, both **base-PyMuPDF only** (no pymupdf4llm), selected by `method`:
+
+```python
+st.strikethroughs_in_pdf("contract.pdf", method="vector")  # stroke geometry (default) —
+                                                           #   precise partial-char spans
+st.strikethroughs_in_pdf("contract.pdf", method="flag")    # MuPDF's FZ_STEXT_STRIKEOUT signal —
+                                                           #   also catches font-attribute strikes
+st.strikethroughs_in_pdf("contract.pdf", method="both")    # union — maximum recall
+```
+
+**Validated across domains.** On 12 public redline PDFs (federal & state regulations, court
+rules, procurement clauses, municipal codes, university policy; 33k struck words),
+**99.9–100% of vector detections are independently confirmed by MuPDF's strikeout signal**,
+and the flag method adds ~2% more words (font-attribute strikes and edge cases) — use
+`method="both"` to capture them. `pymupdf4llm` is not used for detection at all; it is only an
+optional `[markdown]` extra for richer layout in `clean_markdown()`.
+
+## Any PDF — routed per page, scanned pages use OCR + CNN
+
+```python
+import pdf_strikethrough as st
+from pdf_strikethrough.ocr import rapidocr_backend
+from pdf_strikethrough.scanned import ScanConfig
+
+res = st.detect_pdf("mixed.pdf",
+                    ocr=rapidocr_backend(),                 # for scanned pages
+                    scan_config=ScanConfig.confidence_free())
+
+struck   = [w for w in res["words"] if w["final"]]          # struck words (boxes, char spans)
+markdown = res["markdown"]                                  # deletions as ~~struck~~
+clean    = res["clean_text"]                                # surviving text, deletions removed
+passages = res["passages"]                                  # grouped deletion sections
+```
+
+`detect_pdf` classifies each page native-vs-scanned, runs the exact path on native pages
+(`native_method="vector"|"flag"|"both"`) and the geometry→OCR→CNN pipeline on scanned ones, and
+assembles `markdown` / `clean_text` / `passages` for **both** page kinds from its own strike
+decisions (so the text and the word records always agree — no dependence on an external markdown
+engine). Already have an Azure Document Intelligence result? Pass `di_result=...` (the REST JSON
+dict, an `{'analyzeResult': ...}` envelope, or `sdk_result.as_dict()`) to skip re-OCR and use
+DI's word boxes.
+
+Scanned pages with no OCR backend raise `OcrRequiredError` by default; pass
+`on_missing_ocr="skip"` to skip them (with a warning in `res["warnings"]`) and still get
+everything from the native pages. Password-protected PDFs raise `EncryptedPdfError`.
+
+> `clean_markdown()` remains a separate, higher-fidelity **native-only** path that borrows
+> pymupdf4llm's layout (headings, paragraphs). `detect_pdf`'s `markdown` is layout-plain but works
+> uniformly on scanned pages too.
+
+### Choosing an OCR backend
+
+The geometry + CNN carry the detection and are **OCR-independent**; OCR only supplies word boxes
+to attribute strikes to, plus a confidence prior. Benchmarked on a heavily-edited document
+(Azure DI as reference):
+
+| Backend | Setup | Struck **regions** | Spatial agreement | Word granularity |
+|---|---|---|---|---|
+| Azure Document Intelligence | cloud, paid | reference | — | exact word boxes |
+| **RapidOCR** | `pip`, no binary | **100% covered** | **~99%** | ~4× coarser (phrase-level) |
+| Tesseract | needs system binary | — | — | genuine word-level |
+
+Use `ScanConfig.confidence_free()` with RapidOCR (its confidences cluster near 1.0 and don't
+separate struck from clean text); the default `ScanConfig()` is calibrated to Azure DI, whose
+struck words drop to 0.43–0.94. The DI-decoupled classifier reproduces the original Azure-DI
+pipeline to **99.5%** (1477 vs 1484 struck words on the validation doc).
+
+## Low-level building blocks
+
+```python
+gray = st.render_page_gray(doc[0], dpi=200)     # HxW grayscale; RGB/float arrays are coerced
+
+lines = st.strike_lines(gray, dpi=200)          # OCR-free stroke geometry (strike/underline/rule)
+                                                # pass the dpi the image was rendered/scanned at
+
+# word boxes are PAGE FRACTIONS in [0,1], origin top-left — not pixels
+p = st.score_word(gray, (0.12, 0.34, 0.38, 0.36))   # CNN strike probability (0..1)
+
+from pdf_strikethrough.ocr import Word
+recs = st.detect_scanned_image(gray, [Word("foo", (0.12, 0.34, 0.38, 0.36), 0.6)])
+```
+
+## CLI
+
+```bash
+pdf-strikethrough detect contract.pdf                       # native pages (scanned pages are
+                                                            #   skipped with a warning)
+pdf-strikethrough detect scan.pdf --ocr rapidocr            # include scanned pages
+pdf-strikethrough detect doc.pdf --method both              # max-recall native detection
+pdf-strikethrough detect doc.pdf --json out.json            # full struck words + passages
+pdf-strikethrough detect doc.pdf --clean-text clean.txt     # surviving text, deletions removed
+pdf-strikethrough detect doc.pdf --markdown marked.md       # deletions as ~~struck~~
+```
+
+## How it works
+
+- **Native**: merged horizontal vector strokes through a word's middle band (excludes under/over-
+  lines); coverage ≥ 50% → struck, partials resolved to a char range.
+- **Scanned geometry** (`lines.py`): per-angle morphological opening extracts stroke *fragments*,
+  collinear fragments are stitched, then strict filters (spine fill, stroke run-thickness,
+  angle/length) separate real strikes from bold crossbars and serif-glyph chains.
+- **CNN** (`cnn.py`, StrikeNet, 79k params): resolves pixel-ambiguous cases — a thin strike over
+  an ascender-less word is pixel-identical to a glyph chain, and only a learned model tells them
+  apart. Ships as ONNX; set `PDF_STRIKETHROUGH_MODEL_DIR` to use your own weights.
+- **Attribution** (`scanned.py`): assigns strokes to OCR words with char spans and full/partial
+  resolution, plus a visual-row "orphan" pass for words the detector's stroke evidence missed.
+
+## License
+
+MIT.

pdf_strikethrough_detect-0.4.0/README.md

@@ -0,0 +1,164 @@
+# pdf-strikethrough-detect
+
+[![PyPI](https://img.shields.io/pypi/v/pdf-strikethrough-detect.svg)](https://pypi.org/project/pdf-strikethrough-detect/)
+[![Python versions](https://img.shields.io/pypi/pyversions/pdf-strikethrough-detect.svg)](https://pypi.org/project/pdf-strikethrough-detect/)
+[![CI](https://github.com/niles-liu/pdf-strikethrough-detect/actions/workflows/ci.yml/badge.svg)](https://github.com/niles-liu/pdf-strikethrough-detect/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+
+Detect **struck-through (deleted) text** in PDFs and scanned document images.
+
+Strikethrough detection is a surprisingly unserved niche: most "redline"/diff tools assume clean
+born-digital PDFs and fall apart on scans — which is the real-world case. `pdf-strikethrough-detect`
+handles both, does the hard part (scanned images) with a tiny CPU model, and makes no cloud calls.
+
+```python
+import pdf_strikethrough as st
+
+# born-digital PDF — exact, no OCR
+for w in st.strikethroughs_in_pdf("contract.pdf"):
+    print(w["page"], repr(w["chars"]), "partial" if w["partial"] else "full")
+```
+
+## Install
+
+```bash
+pip install pdf-strikethrough-detect
+```
+
+Pure pip, no system binaries required: the CNN runs on ONNX Runtime (CPU) and the ~318 KB model
+ships inside the wheel. Extras:
+
+```bash
+pip install "pdf-strikethrough-detect[markdown]"    # clean_markdown() via pymupdf4llm
+pip install "pdf-strikethrough-detect[rapidocr]"    # free scanned-word OCR backend (no binary)
+pip install "pdf-strikethrough-detect[tesseract]"   # word-level OCR (also needs the tesseract binary)
+```
+
+## Native / born-digital PDFs — exact
+
+In a born-digital PDF a strikethrough is a *vector drawing* (a line or thin rect over the text),
+so detection is exact ground truth — no OCR, no model, no guessing. Both vector-rule and
+filled-rect strikethrough styles are handled.
+
+```python
+import pdf_strikethrough as st
+
+for w in st.strikethroughs_in_pdf("contract.pdf"):
+    print(w["page"], repr(w["chars"]))          # 'chars' = the struck substring
+print(st.clean_markdown("contract.pdf"))        # surviving text, deletions removed (needs [markdown])
+```
+
+Each record: `{page, text, chars, char_span, partial, bbox_frac, coverage, verdict, final}`.
+Partial strikes (`semi-` of `semi-monthly`) are resolved to a char range. `bbox_frac` is in
+fractions of the rendered page (rotation-aware), so it maps directly onto a rendered pixmap.
+
+Two native detectors, both **base-PyMuPDF only** (no pymupdf4llm), selected by `method`:
+
+```python
+st.strikethroughs_in_pdf("contract.pdf", method="vector")  # stroke geometry (default) —
+                                                           #   precise partial-char spans
+st.strikethroughs_in_pdf("contract.pdf", method="flag")    # MuPDF's FZ_STEXT_STRIKEOUT signal —
+                                                           #   also catches font-attribute strikes
+st.strikethroughs_in_pdf("contract.pdf", method="both")    # union — maximum recall
+```
+
+**Validated across domains.** On 12 public redline PDFs (federal & state regulations, court
+rules, procurement clauses, municipal codes, university policy; 33k struck words),
+**99.9–100% of vector detections are independently confirmed by MuPDF's strikeout signal**,
+and the flag method adds ~2% more words (font-attribute strikes and edge cases) — use
+`method="both"` to capture them. `pymupdf4llm` is not used for detection at all; it is only an
+optional `[markdown]` extra for richer layout in `clean_markdown()`.
+
+## Any PDF — routed per page, scanned pages use OCR + CNN
+
+```python
+import pdf_strikethrough as st
+from pdf_strikethrough.ocr import rapidocr_backend
+from pdf_strikethrough.scanned import ScanConfig
+
+res = st.detect_pdf("mixed.pdf",
+                    ocr=rapidocr_backend(),                 # for scanned pages
+                    scan_config=ScanConfig.confidence_free())
+
+struck   = [w for w in res["words"] if w["final"]]          # struck words (boxes, char spans)
+markdown = res["markdown"]                                  # deletions as ~~struck~~
+clean    = res["clean_text"]                                # surviving text, deletions removed
+passages = res["passages"]                                  # grouped deletion sections
+```
+
+`detect_pdf` classifies each page native-vs-scanned, runs the exact path on native pages
+(`native_method="vector"|"flag"|"both"`) and the geometry→OCR→CNN pipeline on scanned ones, and
+assembles `markdown` / `clean_text` / `passages` for **both** page kinds from its own strike
+decisions (so the text and the word records always agree — no dependence on an external markdown
+engine). Already have an Azure Document Intelligence result? Pass `di_result=...` (the REST JSON
+dict, an `{'analyzeResult': ...}` envelope, or `sdk_result.as_dict()`) to skip re-OCR and use
+DI's word boxes.
+
+Scanned pages with no OCR backend raise `OcrRequiredError` by default; pass
+`on_missing_ocr="skip"` to skip them (with a warning in `res["warnings"]`) and still get
+everything from the native pages. Password-protected PDFs raise `EncryptedPdfError`.
+
+> `clean_markdown()` remains a separate, higher-fidelity **native-only** path that borrows
+> pymupdf4llm's layout (headings, paragraphs). `detect_pdf`'s `markdown` is layout-plain but works
+> uniformly on scanned pages too.
+
+### Choosing an OCR backend
+
+The geometry + CNN carry the detection and are **OCR-independent**; OCR only supplies word boxes
+to attribute strikes to, plus a confidence prior. Benchmarked on a heavily-edited document
+(Azure DI as reference):
+
+| Backend | Setup | Struck **regions** | Spatial agreement | Word granularity |
+|---|---|---|---|---|
+| Azure Document Intelligence | cloud, paid | reference | — | exact word boxes |
+| **RapidOCR** | `pip`, no binary | **100% covered** | **~99%** | ~4× coarser (phrase-level) |
+| Tesseract | needs system binary | — | — | genuine word-level |
+
+Use `ScanConfig.confidence_free()` with RapidOCR (its confidences cluster near 1.0 and don't
+separate struck from clean text); the default `ScanConfig()` is calibrated to Azure DI, whose
+struck words drop to 0.43–0.94. The DI-decoupled classifier reproduces the original Azure-DI
+pipeline to **99.5%** (1477 vs 1484 struck words on the validation doc).
+
+## Low-level building blocks
+
+```python
+gray = st.render_page_gray(doc[0], dpi=200)     # HxW grayscale; RGB/float arrays are coerced
+
+lines = st.strike_lines(gray, dpi=200)          # OCR-free stroke geometry (strike/underline/rule)
+                                                # pass the dpi the image was rendered/scanned at
+
+# word boxes are PAGE FRACTIONS in [0,1], origin top-left — not pixels
+p = st.score_word(gray, (0.12, 0.34, 0.38, 0.36))   # CNN strike probability (0..1)
+
+from pdf_strikethrough.ocr import Word
+recs = st.detect_scanned_image(gray, [Word("foo", (0.12, 0.34, 0.38, 0.36), 0.6)])
+```
+
+## CLI
+
+```bash
+pdf-strikethrough detect contract.pdf                       # native pages (scanned pages are
+                                                            #   skipped with a warning)
+pdf-strikethrough detect scan.pdf --ocr rapidocr            # include scanned pages
+pdf-strikethrough detect doc.pdf --method both              # max-recall native detection
+pdf-strikethrough detect doc.pdf --json out.json            # full struck words + passages
+pdf-strikethrough detect doc.pdf --clean-text clean.txt     # surviving text, deletions removed
+pdf-strikethrough detect doc.pdf --markdown marked.md       # deletions as ~~struck~~
+```
+
+## How it works
+
+- **Native**: merged horizontal vector strokes through a word's middle band (excludes under/over-
+  lines); coverage ≥ 50% → struck, partials resolved to a char range.
+- **Scanned geometry** (`lines.py`): per-angle morphological opening extracts stroke *fragments*,
+  collinear fragments are stitched, then strict filters (spine fill, stroke run-thickness,
+  angle/length) separate real strikes from bold crossbars and serif-glyph chains.
+- **CNN** (`cnn.py`, StrikeNet, 79k params): resolves pixel-ambiguous cases — a thin strike over
+  an ascender-less word is pixel-identical to a glyph chain, and only a learned model tells them
+  apart. Ships as ONNX; set `PDF_STRIKETHROUGH_MODEL_DIR` to use your own weights.
+- **Attribution** (`scanned.py`): assigns strokes to OCR words with char spans and full/partial
+  resolution, plus a visual-row "orphan" pass for words the detector's stroke evidence missed.
+
+## License
+
+MIT.

pdf_strikethrough_detect-0.4.0/pyproject.toml

@@ -0,0 +1,55 @@
+[build-system]
+requires = ["setuptools>=77", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "pdf-strikethrough-detect"
+version = "0.4.0"
+description = "Detect struck-through (deleted) text in PDFs and scanned document images."
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Niles Liu" }]
+keywords = ["strikethrough", "strikeout", "redline", "ocr", "pdf", "document-intelligence",
+            "deleted-text", "scanned-documents"]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Image Recognition",
+    "Topic :: Text Processing",
+]
+dependencies = [
+    "numpy>=1.24",
+    "pillow>=9.0",
+    "scipy>=1.10",
+    "onnxruntime>=1.16",
+    "pymupdf>=1.26",         # >=1.26: TEXT_COLLECT_STYLES + FZ_STEXT_STRIKEOUT (flag detector)
+]
+
+[project.optional-dependencies]
+markdown = ["pymupdf4llm>=0.0.27"]     # clean_markdown(); >=0.0.22 introduced ~~ strike spans,
+                                       # >=0.0.27 matches the pymupdf>=1.26 base floor
+rapidocr = ["rapidocr>=3.2", "onnxruntime>=1.16"]   # free, pip-only scanned-word OCR backend
+                                                    # (>=3.2: nested word_results shape)
+tesseract = ["pytesseract>=0.3.10"]    # needs the tesseract system binary too
+torch = ["torch>=2.0"]                 # optional .pt fallback for the CNN
+dev = ["pytest>=7.0"]
+
+[project.scripts]
+pdf-strikethrough = "pdf_strikethrough.__main__:main"
+
+[project.urls]
+Homepage = "https://github.com/niles-liu/pdf-strikethrough-detect"
+Repository = "https://github.com/niles-liu/pdf-strikethrough-detect"
+Issues = "https://github.com/niles-liu/pdf-strikethrough-detect/issues"
+Changelog = "https://github.com/niles-liu/pdf-strikethrough-detect/blob/main/CHANGELOG.md"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+pdf_strikethrough = ["strike_verdict_cnn.onnx", "strike_verdict_cnn.meta.json"]

pdf_strikethrough_detect-0.4.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

pdf_strikethrough_detect-0.4.0/src/pdf_strikethrough/__init__.py

@@ -0,0 +1,107 @@
+"""pdf_strikethrough — detect struck-through (deleted) text in PDFs and scanned document images.
+
+Quick start
+-----------
+Native / born-digital PDF (strikethroughs are vector drawings — detection is EXACT, no OCR):
+
+    import pdf_strikethrough as st
+    for w in st.strikethroughs_in_pdf("contract.pdf"):
+        print(w["page"], repr(w["chars"]), "partial" if w["partial"] else "full")
+    print(st.clean_markdown("contract.pdf"))          # surviving text, deletions removed
+
+Any PDF (routes native/scanned per page; scanned needs an OCR backend):
+
+    from pdf_strikethrough.ocr import rapidocr_backend
+    from pdf_strikethrough.scanned import ScanConfig
+    res = st.detect_pdf("scan.pdf", ocr=rapidocr_backend(),
+                        scan_config=ScanConfig.confidence_free())
+    struck = [w for w in res["words"] if w["final"]]
+
+Low-level, on your own image (no PDF):
+
+    lines = st.strike_lines(gray, dpi=200)            # OCR-free stroke geometry
+    p = st.score_word(gray, (x0, y0, x1, y1))         # CNN strike probability for a word box
+"""
+from . import cnn, detect, lines, markdown, native, ocr, scanned
+from .cnn import (get_model_meta, score_crops, score_word, std_crop, verdict_of, word_crop_px)
+from .detect import (EncryptedPdfError, OcrRequiredError, apply_cnn_verdict,
+                     classify_page_source, detect_pdf, detect_scanned_image)
+from .lines import ink_mask, strike_lines, to_gray_u8
+from .native import (native_doc_strikes, native_flag_strikes, native_markdown,
+                     native_page_strikes, page_strikes, strip_struck_markdown)
+from .ocr import (Word, rapidocr_backend, tesseract_backend, words_from_azure_di)
+from .scanned import ScanConfig, analyze_scanned_page
+
+__version__ = "0.4.0"
+
+__all__ = [
+    # high-level
+    "strikethroughs_in_pdf", "clean_markdown", "detect_pdf", "detect_scanned_image",
+    "open_pdf", "render_page_gray",
+    # native
+    "native_page_strikes", "native_flag_strikes", "native_doc_strikes", "page_strikes",
+    "native_markdown", "strip_struck_markdown",
+    # scanned geometry + classifier
+    "strike_lines", "ink_mask", "to_gray_u8", "analyze_scanned_page", "ScanConfig",
+    "classify_page_source", "apply_cnn_verdict",
+    # errors
+    "OcrRequiredError", "EncryptedPdfError",
+    # OCR
+    "Word", "rapidocr_backend", "tesseract_backend", "words_from_azure_di",
+    # CNN
+    "score_word", "score_crops", "std_crop", "word_crop_px", "verdict_of", "get_model_meta",
+    # submodules
+    "cnn", "lines", "native", "ocr", "scanned", "detect", "markdown",
+]
+
+
+def open_pdf(source):
+    """Open `source` (path, bytes, or an already-open fitz document) as a fitz document.
+    Raises EncryptedPdfError for password-protected PDFs."""
+    import fitz
+    if hasattr(source, "page_count"):
+        doc = source
+    elif isinstance(source, (bytes, bytearray)):
+        doc = fitz.open(stream=bytes(source), filetype="pdf")
+    else:
+        doc = fitz.open(source)
+    if getattr(doc, "needs_pass", False):
+        raise EncryptedPdfError(
+            "PDF is password-protected; open it with fitz and call doc.authenticate(password) "
+            "(or save a decrypted copy) before processing")
+    return doc
+
+
+def render_page_gray(page, dpi=lines.RENDER_DPI):
+    """Render a fitz page to a grayscale uint8 (H, W) numpy array."""
+    import fitz
+    import numpy as np
+    pix = page.get_pixmap(dpi=dpi, colorspace=fitz.csGRAY)
+    return np.frombuffer(pix.samples, dtype=np.uint8).reshape(pix.height, pix.width)
+
+
+def strikethroughs_in_pdf(source, method="vector"):
+    """Struck-word records for a born-digital PDF (path/bytes/fitz doc), all pages, reading order.
+    Exact — driven by the PDF's own strike drawings. `method`: 'vector' (stroke geometry, precise
+    partial-char spans; default), 'flag' (MuPDF's strikeout span flag), or 'both' (union, maximum
+    recall). Returns [] for a scanned PDF; use ``detect_pdf(..., ocr=...)`` for those."""
+    doc = open_pdf(source)
+    try:
+        return native.native_doc_strikes(doc, method)
+    finally:
+        if not hasattr(source, "page_count"):
+            doc.close()
+
+
+def clean_markdown(source):
+    """Markdown for a born-digital PDF with struck (deleted) spans removed — the surviving text.
+    Requires the ``[markdown]`` extra (pymupdf4llm); raises ImportError with the pip command
+    otherwise. For an extra-free equivalent use ``detect_pdf(source)["clean_text"]``."""
+    doc = open_pdf(source)
+    try:
+        if doc.page_count == 0:
+            return ""
+        return native.strip_struck_markdown(native.native_markdown(doc))
+    finally:
+        if not hasattr(source, "page_count"):
+            doc.close()