tidyscan 0.3.0__tar.gz

tidyscan-0.3.0/CHANGELOG.md

@@ -0,0 +1,25 @@
+# Changelog
+
+## 0.3.0 (2026-11-06)
+
+First public release.
+
+- **Upright**: Tesseract's orientation detection, used only with enough
+  confidence and an allowed script; an unsure upside-down page, or a page
+  that lies sideways, is settled by reading it both ways round.
+- **Straight**: skew up to 45 degrees, from coarse projection candidates and
+  Tesseract's text-line baselines, checked by measuring again; at most 5
+  minutes of Tesseract per page.
+- **Lossless PDF**: JPEG pages as they are, 1-bit pages Flate-compressed;
+  turns as `/Rotate`, skew as a drawing matrix.
+- **OCR** through OCRmyPDF: Tesseract by default, PaddleOCR PP-OCRv5 on
+  OpenVINO (CPU) for receipts, photo captions and tables; the OCRmyPDF plugin
+  `tidyscan.ocrmypdf_paddle` works with OCRmyPDF 16 and 17. PDF/A-2b on
+  request.
+- `tidyscan models fetch`: the PaddleOCR models from this project's release,
+  checked by sha256.
+- `tidyscan watch IN OUT`: document folders (as scankeyd writes them) into
+  PDFs, as a service, with settings from the environment and a status file
+  for a monitor.
+- Container images: `base` (pages to PDF, standard library), the default
+  `full` (Tesseract and OCRmyPDF) and, on request, `paddle`.

tidyscan-0.3.0/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, Luuk Visser
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

tidyscan-0.3.0/PKG-INFO

@@ -0,0 +1,202 @@
+Metadata-Version: 2.4
+Name: tidyscan
+Version: 0.3.0
+Summary: Auto-rotate and deskew scanned pages into a lossless PDF, with an optional OCR text layer.
+Keywords: scan,deskew,orientation,pdf,ocr,tesseract,paddleocr
+Author: Luuk Visser
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-Expression: BSD-3-Clause
+Classifier: Development Status :: 4 - Beta
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Topic :: Multimedia :: Graphics :: Graphics Conversion
+Classifier: Topic :: Scientific/Engineering :: Image Recognition
+License-File: LICENSE
+Requires-Dist: Pillow>=10 ; extra == "ocr"
+Requires-Dist: ocrmypdf>=16 ; extra == "ocr"
+Requires-Dist: Pillow>=10 ; extra == "paddle"
+Requires-Dist: ocrmypdf>=16,<18 ; extra == "paddle"
+Requires-Dist: rapidocr>=3.10,<4 ; extra == "paddle"
+Requires-Dist: openvino>=2025.1 ; extra == "paddle"
+Requires-Dist: Pillow>=10 ; extra == "straighten"
+Project-URL: Changelog, https://github.com/luukvisser/tidyscan/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/luukvisser/tidyscan/issues
+Project-URL: Source, https://github.com/luukvisser/tidyscan
+Provides-Extra: ocr
+Provides-Extra: paddle
+Provides-Extra: straighten
+
+# tidyscan
+
+Scanned pages turned upright and straightened into a PDF, losslessly, and
+optionally made searchable.
+
+- **Upright**: Tesseract's orientation detection decides the quarter turn,
+  only when it is sure and the script is one you allow (Latin by default); an
+  unsure upside-down page is settled by reading it both ways round.
+- **Straight**: a page laid crooked on the glass (up to 45 degrees) is
+  measured on its text lines and checked by measuring again.
+- **Lossless**: JPEG pages go into the PDF byte for byte, 1-bit pages
+  Flate-compressed; the turn is the page's `/Rotate`, the skew a drawing
+  matrix. Nothing is resampled. Standard library only.
+- **Searchable** (optional): a text layer by OCRmyPDF, read by Tesseract, or by
+  PaddleOCR (PP-OCRv5 on OpenVINO, CPU) for receipts.
+
+When anything is unsure, the page is kept as scanned. Written for the scans of
+[scankeyd](https://github.com/luukvisser/scankeyd) (the Scan key of Brother
+network printers to a folder), and usable for any scanned pages.
+
+## Install
+
+    pip install tidyscan                # pages to PDF only
+    pip install 'tidyscan[straighten]'  # plus Pillow: turning upright and straightening
+    pip install 'tidyscan[ocr]'         # plus OCRmyPDF: the text layer
+    pip install 'tidyscan[paddle]'      # plus RapidOCR and OpenVINO: PaddleOCR
+
+Python 3.11 or newer, on Linux. Turning and straightening also need the
+`tesseract` program with its `osd` data, OCR its language data (Debian:
+`tesseract-ocr tesseract-ocr-osd tesseract-ocr-nld ...`). PaddleOCR needs its
+models once (checked by sha256, never downloaded by themselves):
+
+    tidyscan models fetch    # into ~/.local/share/tidyscan/models
+    tidyscan models path     # where they are looked up, and whether they are there
+
+## Use
+
+    tidyscan --ocr eng+nld -o out.pdf page1.jpg page2.pbm
+    tidyscan --ocr nld --engine paddle -o receipt.pdf receipt.jpg
+
+JPEG and PBM pages go in as they are. Other formats (PNG, TIFF, every page of
+a multi-page TIFF) are read with Pillow and converted first, to PBM when
+black-and-white, else to JPEG (quality 95): for those the PDF holds the
+conversion. Options: `--dpi` (300), `--no-rotate`, `--no-deskew`,
+`--min-confidence` (3), `--engine`, `-q`. Exit code 0, or 1 for a page that is
+not a JPEG or PBM; an OCR failure writes the PDF without text and says so.
+
+```python
+import tidyscan
+from tidyscan import ocr, pdf
+
+pages = [open("page1.jpg", "rb").read()]
+turns, skews = zip(*(tidyscan.straighten(p, 300) for p in pages))
+data = pdf.build(pages, 300, list(turns), list(skews))
+data = ocr.add_text_layer(data, "eng+nld")  # raises ocr.OcrError; engine="tesseract" (default), "paddle", "auto"
+```
+
+A page is JPEG bytes, binary PBM (P4) bytes, or an object with `to_pbm()`.
+`straighten()` never raises and logs to the `tidyscan` logger; `turn=` takes a
+turn already known (from the scanner), `budget=` the seconds of Tesseract it
+may spend on a page (300; what is not measured by then is kept as scanned).
+
+### Which OCR engine
+
+| engine | words found | words right | 1 page | each further page | memory |
+|---|---|---|---|---|---|
+| **Tesseract 5.5** (`--engine tesseract`, the default) | 90% | 87% | 3.6 s | 2.7 s | 0.15 GB |
+| PaddleOCR PP-OCRv5 (`--engine paddle`) | 83% | 92% | 6.1 s | 9.9 s | 0.9 GB |
+
+Tesseract reads letters, documents, forms and book pages better, faster and
+in less memory. PaddleOCR reads receipts, photo captions and tables better
+(it adds far fewer wrong words on tables), and phone photos of receipts that
+Tesseract cannot read at all. `--engine auto` takes PaddleOCR when it is
+installed and the languages are Latin-script ones, else Tesseract; Tesseract
+also takes over a document PaddleOCR fails on. PaddleOCR's Latin recogniser
+reads Dutch, English and some 30 other languages at once. Measured on 45 pages
+in 188 scan-like variants, through the whole pipeline, on 2 cores of an Intel
+N150: [docs/benchmarks.md](docs/benchmarks.md).
+
+## As a service: `tidyscan watch`
+
+`tidyscan watch IN OUT` turns document folders into PDFs, oldest first: a
+folder `IN/<function>/<name>/` with the pages (`page-0001.jpg`,
+`page-0002.pbm`, ...) and a `document.json` (optional `"resolution"`, default
+300; `"pages"`, default the page files in order; `"ocr": true` for a text
+layer), as [scankeyd](https://github.com/luukvisser/scankeyd) writes them.
+Each becomes `OUT/<name>.pdf`, written as a hidden `.part` file and renamed;
+then the folder goes. A document that cannot be read moves to `IN/.failed/`;
+when OCR fails, the PDF is written without text. IN and its function folders
+must be writable for this user, as each folder goes once its PDF is there.
+
+| Option | Environment | Default | |
+|---|---|---|---|
+| `--ocr LANGS` | `OCR_LANGUAGES` | `eng` | text layer languages, e.g. `eng+nld` |
+| `--engine` | `OCR_ENGINE` | `tesseract` | `tesseract`, `paddle` or `auto` |
+| `--ocr-folders` | `OCR_FOLDERS` | `ocr` | for documents without an `"ocr"` field: the function folders that get a text layer (comma list, `all`, or empty for none) |
+| `--no-rotate`, `--no-deskew` | `ROTATE`, `DESKEW` | `auto` | `on`, `off` or `auto` (on when Tesseract is there) |
+| `--pdfa` | `PDFA` | `off` | PDF/A-2b for the documents with a text layer (still lossless: 1-bit pages become CCITT G4) |
+| `--status FILE` | `STATUS_FILE` | none | a JSON status file for a monitor: `when`, `last_ok`, `last_file`, `last_error`, `failed`, `waiting`, and the counts `pdfs`, `unreadable`, `no_text` |
+| `--keep-folders` | | off | `OUT/<function>/<name>.pdf` |
+| `--interval` | | 5 | seconds between looks at IN |
+| `--once` | | | convert what is there, then stop |
+
+On/off settings take `on`/`off`, `true`/`false`, `yes`/`no` or `1`/`0`; any
+other value stops the start with a message.
+
+**Docker**: three images, built from this repository (`docker build ...`):
+`--target base` (pages to PDF only, ~77 MB, no copyleft software), the default
+`full` (with Tesseract and OCRmyPDF, ~480 MB) and `--target paddle` (with
+PaddleOCR, ~1 GB, amd64 only). They run `tidyscan watch /in /out` as uid 1000:
+
+```sh
+docker build -t tidyscan .
+docker run -d --network none --read-only --tmpfs /tmp --env-file scan.env \
+  -v ./scans/new:/in -v ./scans/pdf:/out tidyscan
+```
+
+OCR languages for the image: `--build-arg OCR_LANGS="eng nld deu fra"`.
+
+## Limits
+
+- Turning upright only for scripts you allow (Latin by default); skew up to 45
+  degrees. Photos and nearly blank pages are kept as scanned, and so are phone
+  photos of receipts (Tesseract finds no text lines on them).
+- At most 5 minutes of Tesseract per page; a page above 200 million pixels is
+  not read at all.
+- OCRmyPDF gets 600 s per document: a long document read by PaddleOCR (about
+  10 s a page) may run out and is then read by Tesseract.
+- PaddleOCR: Latin script only; OCRmyPDF must run in the same Python as
+  tidyscan and RapidOCR (the engine is the OCRmyPDF plugin
+  `tidyscan.ocrmypdf_paddle`, also usable on its own: `ocrmypdf --plugin
+  tidyscan.ocrmypdf_paddle`). `TIDYSCAN_PADDLE_THREADS` sets its threads.
+
+## How well it works
+
+Measured with `tests/eval_deskew.py` on 1946 cases from 69 pages, each turned,
+tilted or mirrored by a known amount (06-11-2026):
+
+- **Upright**: of 381 text pages that needed a turn, 96% upright and none
+  turned the wrong way (OCRmyPDF's `--rotate-pages`: 29%; Tesseract OSD alone:
+  77%, with 53 wrong turns). No upright page was turned away.
+- **Straight**: within 0.5 degrees on 100%, 100% and 96% of the pages tilted
+  slightly, medium and steeply. jdeskew with `angle_max=45` measures as well
+  but turns no page upright; it, OCRmyPDF, ImageMagick, unpaper and Leptonica
+  straighten by rotating the image, which resamples it.
+
+Details, the other tools and the pages: [docs/benchmarks.md](docs/benchmarks.md).
+
+## Development
+
+With [uv](https://docs.astral.sh/uv/):
+
+    uv sync
+    uv run ruff check && uv run ruff format --check && uv run ty check
+    uv run python -m unittest tests.test_tidyscan
+
+Tests that need Tesseract, OCRmyPDF or the PaddleOCR models are skipped when
+those are missing. The benchmarks (`tests/eval_*.py`) download their pages and
+are not part of the unit tests.
+
+## Licence
+
+BSD-3-Clause, see [LICENSE](LICENSE); the repository follows
+[REUSE](https://reuse.software/). Written by Luuk Visser with Claude Code
+(Claude Opus 5.5).
+
+What the optional parts use is not included in this package: OCRmyPDF
+(MPL-2.0), Tesseract, RapidOCR, OpenVINO, OpenCV and PaddleOCR's PP-OCRv5
+models (Apache-2.0; the models come with their own notice). The `full` and
+`paddle` images add Debian packages under their own licences, among them
+Ghostscript (AGPL-3.0, used by OCRmyPDF); the `base` image has no copyleft
+software.
+

tidyscan-0.3.0/README.md

@@ -0,0 +1,172 @@
+# tidyscan
+
+Scanned pages turned upright and straightened into a PDF, losslessly, and
+optionally made searchable.
+
+- **Upright**: Tesseract's orientation detection decides the quarter turn,
+  only when it is sure and the script is one you allow (Latin by default); an
+  unsure upside-down page is settled by reading it both ways round.
+- **Straight**: a page laid crooked on the glass (up to 45 degrees) is
+  measured on its text lines and checked by measuring again.
+- **Lossless**: JPEG pages go into the PDF byte for byte, 1-bit pages
+  Flate-compressed; the turn is the page's `/Rotate`, the skew a drawing
+  matrix. Nothing is resampled. Standard library only.
+- **Searchable** (optional): a text layer by OCRmyPDF, read by Tesseract, or by
+  PaddleOCR (PP-OCRv5 on OpenVINO, CPU) for receipts.
+
+When anything is unsure, the page is kept as scanned. Written for the scans of
+[scankeyd](https://github.com/luukvisser/scankeyd) (the Scan key of Brother
+network printers to a folder), and usable for any scanned pages.
+
+## Install
+
+    pip install tidyscan                # pages to PDF only
+    pip install 'tidyscan[straighten]'  # plus Pillow: turning upright and straightening
+    pip install 'tidyscan[ocr]'         # plus OCRmyPDF: the text layer
+    pip install 'tidyscan[paddle]'      # plus RapidOCR and OpenVINO: PaddleOCR
+
+Python 3.11 or newer, on Linux. Turning and straightening also need the
+`tesseract` program with its `osd` data, OCR its language data (Debian:
+`tesseract-ocr tesseract-ocr-osd tesseract-ocr-nld ...`). PaddleOCR needs its
+models once (checked by sha256, never downloaded by themselves):
+
+    tidyscan models fetch    # into ~/.local/share/tidyscan/models
+    tidyscan models path     # where they are looked up, and whether they are there
+
+## Use
+
+    tidyscan --ocr eng+nld -o out.pdf page1.jpg page2.pbm
+    tidyscan --ocr nld --engine paddle -o receipt.pdf receipt.jpg
+
+JPEG and PBM pages go in as they are. Other formats (PNG, TIFF, every page of
+a multi-page TIFF) are read with Pillow and converted first, to PBM when
+black-and-white, else to JPEG (quality 95): for those the PDF holds the
+conversion. Options: `--dpi` (300), `--no-rotate`, `--no-deskew`,
+`--min-confidence` (3), `--engine`, `-q`. Exit code 0, or 1 for a page that is
+not a JPEG or PBM; an OCR failure writes the PDF without text and says so.
+
+```python
+import tidyscan
+from tidyscan import ocr, pdf
+
+pages = [open("page1.jpg", "rb").read()]
+turns, skews = zip(*(tidyscan.straighten(p, 300) for p in pages))
+data = pdf.build(pages, 300, list(turns), list(skews))
+data = ocr.add_text_layer(data, "eng+nld")  # raises ocr.OcrError; engine="tesseract" (default), "paddle", "auto"
+```
+
+A page is JPEG bytes, binary PBM (P4) bytes, or an object with `to_pbm()`.
+`straighten()` never raises and logs to the `tidyscan` logger; `turn=` takes a
+turn already known (from the scanner), `budget=` the seconds of Tesseract it
+may spend on a page (300; what is not measured by then is kept as scanned).
+
+### Which OCR engine
+
+| engine | words found | words right | 1 page | each further page | memory |
+|---|---|---|---|---|---|
+| **Tesseract 5.5** (`--engine tesseract`, the default) | 90% | 87% | 3.6 s | 2.7 s | 0.15 GB |
+| PaddleOCR PP-OCRv5 (`--engine paddle`) | 83% | 92% | 6.1 s | 9.9 s | 0.9 GB |
+
+Tesseract reads letters, documents, forms and book pages better, faster and
+in less memory. PaddleOCR reads receipts, photo captions and tables better
+(it adds far fewer wrong words on tables), and phone photos of receipts that
+Tesseract cannot read at all. `--engine auto` takes PaddleOCR when it is
+installed and the languages are Latin-script ones, else Tesseract; Tesseract
+also takes over a document PaddleOCR fails on. PaddleOCR's Latin recogniser
+reads Dutch, English and some 30 other languages at once. Measured on 45 pages
+in 188 scan-like variants, through the whole pipeline, on 2 cores of an Intel
+N150: [docs/benchmarks.md](docs/benchmarks.md).
+
+## As a service: `tidyscan watch`
+
+`tidyscan watch IN OUT` turns document folders into PDFs, oldest first: a
+folder `IN/<function>/<name>/` with the pages (`page-0001.jpg`,
+`page-0002.pbm`, ...) and a `document.json` (optional `"resolution"`, default
+300; `"pages"`, default the page files in order; `"ocr": true` for a text
+layer), as [scankeyd](https://github.com/luukvisser/scankeyd) writes them.
+Each becomes `OUT/<name>.pdf`, written as a hidden `.part` file and renamed;
+then the folder goes. A document that cannot be read moves to `IN/.failed/`;
+when OCR fails, the PDF is written without text. IN and its function folders
+must be writable for this user, as each folder goes once its PDF is there.
+
+| Option | Environment | Default | |
+|---|---|---|---|
+| `--ocr LANGS` | `OCR_LANGUAGES` | `eng` | text layer languages, e.g. `eng+nld` |
+| `--engine` | `OCR_ENGINE` | `tesseract` | `tesseract`, `paddle` or `auto` |
+| `--ocr-folders` | `OCR_FOLDERS` | `ocr` | for documents without an `"ocr"` field: the function folders that get a text layer (comma list, `all`, or empty for none) |
+| `--no-rotate`, `--no-deskew` | `ROTATE`, `DESKEW` | `auto` | `on`, `off` or `auto` (on when Tesseract is there) |
+| `--pdfa` | `PDFA` | `off` | PDF/A-2b for the documents with a text layer (still lossless: 1-bit pages become CCITT G4) |
+| `--status FILE` | `STATUS_FILE` | none | a JSON status file for a monitor: `when`, `last_ok`, `last_file`, `last_error`, `failed`, `waiting`, and the counts `pdfs`, `unreadable`, `no_text` |
+| `--keep-folders` | | off | `OUT/<function>/<name>.pdf` |
+| `--interval` | | 5 | seconds between looks at IN |
+| `--once` | | | convert what is there, then stop |
+
+On/off settings take `on`/`off`, `true`/`false`, `yes`/`no` or `1`/`0`; any
+other value stops the start with a message.
+
+**Docker**: three images, built from this repository (`docker build ...`):
+`--target base` (pages to PDF only, ~77 MB, no copyleft software), the default
+`full` (with Tesseract and OCRmyPDF, ~480 MB) and `--target paddle` (with
+PaddleOCR, ~1 GB, amd64 only). They run `tidyscan watch /in /out` as uid 1000:
+
+```sh
+docker build -t tidyscan .
+docker run -d --network none --read-only --tmpfs /tmp --env-file scan.env \
+  -v ./scans/new:/in -v ./scans/pdf:/out tidyscan
+```
+
+OCR languages for the image: `--build-arg OCR_LANGS="eng nld deu fra"`.
+
+## Limits
+
+- Turning upright only for scripts you allow (Latin by default); skew up to 45
+  degrees. Photos and nearly blank pages are kept as scanned, and so are phone
+  photos of receipts (Tesseract finds no text lines on them).
+- At most 5 minutes of Tesseract per page; a page above 200 million pixels is
+  not read at all.
+- OCRmyPDF gets 600 s per document: a long document read by PaddleOCR (about
+  10 s a page) may run out and is then read by Tesseract.
+- PaddleOCR: Latin script only; OCRmyPDF must run in the same Python as
+  tidyscan and RapidOCR (the engine is the OCRmyPDF plugin
+  `tidyscan.ocrmypdf_paddle`, also usable on its own: `ocrmypdf --plugin
+  tidyscan.ocrmypdf_paddle`). `TIDYSCAN_PADDLE_THREADS` sets its threads.
+
+## How well it works
+
+Measured with `tests/eval_deskew.py` on 1946 cases from 69 pages, each turned,
+tilted or mirrored by a known amount (06-11-2026):
+
+- **Upright**: of 381 text pages that needed a turn, 96% upright and none
+  turned the wrong way (OCRmyPDF's `--rotate-pages`: 29%; Tesseract OSD alone:
+  77%, with 53 wrong turns). No upright page was turned away.
+- **Straight**: within 0.5 degrees on 100%, 100% and 96% of the pages tilted
+  slightly, medium and steeply. jdeskew with `angle_max=45` measures as well
+  but turns no page upright; it, OCRmyPDF, ImageMagick, unpaper and Leptonica
+  straighten by rotating the image, which resamples it.
+
+Details, the other tools and the pages: [docs/benchmarks.md](docs/benchmarks.md).
+
+## Development
+
+With [uv](https://docs.astral.sh/uv/):
+
+    uv sync
+    uv run ruff check && uv run ruff format --check && uv run ty check
+    uv run python -m unittest tests.test_tidyscan
+
+Tests that need Tesseract, OCRmyPDF or the PaddleOCR models are skipped when
+those are missing. The benchmarks (`tests/eval_*.py`) download their pages and
+are not part of the unit tests.
+
+## Licence
+
+BSD-3-Clause, see [LICENSE](LICENSE); the repository follows
+[REUSE](https://reuse.software/). Written by Luuk Visser with Claude Code
+(Claude Opus 5.5).
+
+What the optional parts use is not included in this package: OCRmyPDF
+(MPL-2.0), Tesseract, RapidOCR, OpenVINO, OpenCV and PaddleOCR's PP-OCRv5
+models (Apache-2.0; the models come with their own notice). The `full` and
+`paddle` images add Debian packages under their own licences, among them
+Ghostscript (AGPL-3.0, used by OCRmyPDF); the `base` image has no copyleft
+software.

tidyscan-0.3.0/pyproject.toml

@@ -0,0 +1,73 @@
+[build-system]
+requires = ["flit_core==3.12.0"]
+build-backend = "flit_core.buildapi"
+
+[project]
+name = "tidyscan"
+dynamic = ["version", "description"]
+readme = "README.md"
+license = "BSD-3-Clause"
+license-files = ["LICENSE"]
+requires-python = ">=3.11"
+authors = [{ name = "Luuk Visser" }]
+keywords = ["scan", "deskew", "orientation", "pdf", "ocr", "tesseract", "paddleocr"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Programming Language :: Python :: 3",
+    "Operating System :: POSIX :: Linux",
+    "Topic :: Multimedia :: Graphics :: Graphics Conversion",
+    "Topic :: Scientific/Engineering :: Image Recognition",
+]
+dependencies = []
+
+[project.optional-dependencies]
+straighten = ["Pillow>=10"]
+ocr = ["Pillow>=10", "ocrmypdf>=16"]
+# PaddleOCR on OpenVINO (CPU) as the OCR engine; the models are separate files (see README)
+paddle = ["Pillow>=10", "ocrmypdf>=16,<18", "rapidocr>=3.10,<4", "openvino>=2025.1"]
+
+[project.urls]
+Source = "https://github.com/luukvisser/tidyscan"
+Issues = "https://github.com/luukvisser/tidyscan/issues"
+Changelog = "https://github.com/luukvisser/tidyscan/blob/main/CHANGELOG.md"
+
+[project.scripts]
+tidyscan = "tidyscan.__main__:main"
+
+[tool.flit.sdist]
+include = ["CHANGELOG.md", "tests/", "tools/"]
+
+[dependency-groups]
+# Lint, type check and the tests that need no Tesseract; `uv sync` installs these.
+# OCRmyPDF here so ty checks the plugin against its API, and its tests run.
+dev = ["ruff==0.17.3", "ty==0.0.95", "Pillow>=10", "ocrmypdf>=16,<18"]
+
+[tool.ruff]
+line-length = 120
+target-version = "py311"
+
+[tool.ruff.lint]
+select = ["E", "F", "W", "I", "B", "UP", "ANN", "TC", "FA"]
+
+[tool.ruff.lint.per-file-ignores]
+# The benchmarks (not part of the package or its tests): long report lines, helpers without annotations
+"tests/eval_*.py" = ["E501", "E702", "ANN001", "ANN202"]
+
+[tool.ty.terminal]
+# warnings (e.g. deprecated APIs) fail the check too
+error-on-warning = true
+
+[tool.ty.rules]
+# on top of ty's defaults: no bare generics (missing-override-decorator needs typing.override, 3.12+)
+missing-type-argument = "error"
+
+[tool.ty.src]
+# The benchmarks run in their own image (tests/eval_deskew.Dockerfile) with libraries the package does
+# not depend on (OpenCV, numpy, the other deskew tools); ruff still checks them.
+exclude = ["tests/eval_*.py"]
+
+[tool.ty.analysis]
+# rapidocr: the optional PaddleOCR runtime (with OpenVINO and OpenCV, ~300 MB), imported only when
+# that engine is used; not worth installing just to type check two imports.
+# yaml: a build-time dependency of tools/models.py only, pinned in tools/convert-requirements.txt.
+allowed-unresolved-imports = ["rapidocr", "yaml"]

tidyscan-0.3.0/src/tidyscan/__init__.py

@@ -0,0 +1,156 @@
+"""Auto-rotate and deskew scanned pages into a lossless PDF, with an optional OCR text layer.
+
+    turn, angle = tidyscan.straighten(page, 300)
+    data = tidyscan.pdf.build([page], 300, [turn], [angle])
+    data = tidyscan.ocr.add_text_layer(data, "eng+nld")   # Tesseract; engine="paddle" for receipts
+
+A page is JPEG bytes, binary PBM (P4) bytes, or an object with to_pbm().
+"""
+
+from __future__ import annotations
+
+import logging
+
+from . import _budget, hocr, models, ocr, orientation, paddle, pdf, skew
+from ._page import Page, PbmSource, encoded
+from .ocr import OcrError
+from .pdf import UnsupportedPage
+
+__version__ = "0.3.0"
+__all__ = [
+    "OcrError",
+    "Page",
+    "PbmSource",
+    "UnsupportedPage",
+    "hocr",
+    "models",
+    "ocr",
+    "orientation",
+    "paddle",
+    "pdf",
+    "skew",
+    "straighten",
+]
+
+log = logging.getLogger(__name__)
+
+STEEP_EDGE = 2.0  # degrees from +-45: a steep-path answer there is as likely the other quarter turn
+BUDGET = 300.0  # seconds of Tesseract per page at most (see _budget)
+MAX_PIXELS = 200_000_000  # above Pillow's own refusal (2 x 89.5 M pixels); an A4 page at 1200 dpi is 140 M
+
+
+def straighten(
+    page: Page,
+    dpi: int,
+    *,
+    turn: int = 0,
+    rotate: bool = True,
+    deskew: bool = True,
+    min_confidence: float = 3.0,
+    scripts: tuple[str, ...] = ("Latin",),
+    deskew_min: float = 0.3,
+    workdir: str | None = None,
+    budget: float | None = BUDGET,
+) -> tuple[int, float]:
+    """(clockwise turn 0/90/180/270, skew in degrees counter-clockwise) for pdf.build; 0 for what is off,
+    unsure or unavailable.
+
+    turn: a turn already known (from the scanner); it is kept and not detected. deskew_min: smaller
+    skews are not visible and left alone. budget: seconds of Tesseract for this page at most (None: no
+    limit); what is not measured by then is kept as scanned. A page larger than MAX_PIXELS is not read
+    at all. Never raises: straightening is a nicety and must not cost a page (a JPEG that Pillow
+    cannot decode is still a page the PDF can carry)."""
+    if _too_large(page):
+        return turn, 0.0
+    token = _budget.start(budget)
+    try:
+        return _straighten(page, dpi, turn, rotate, deskew, min_confidence, scripts, deskew_min, workdir)
+    finally:
+        _budget.end(token)
+
+
+def _too_large(page: Page) -> bool:
+    try:
+        data, ext = encoded(page)
+        w, h = pdf.pbm_info(data)[:2] if ext == "pbm" else pdf.jpeg_info(data)[:2]
+    except Exception:  # noqa: BLE001  (the PDF writer reports a damaged page)
+        return False
+    if w * h > MAX_PIXELS:
+        log.warning("page of %d x %d pixels: too large to read; kept as scanned", w, h)
+        return True
+    return False
+
+
+def _straighten(
+    page: Page,
+    dpi: int,
+    turn: int,
+    rotate: bool,
+    deskew: bool,
+    min_confidence: float,
+    scripts: tuple[str, ...],
+    deskew_min: float,
+    workdir: str | None,
+) -> tuple[int, float]:
+    # Skew first, then the orientation of the straightened page: OSD cannot read a page tilted by
+    # more than a few degrees. A 180-degree turn leaves the text lines' angle as it is; a quarter
+    # turn means the lines were measured across the text, so the skew is measured again upright.
+    known = bool(turn)
+    found = _detect(page, dpi, turn, workdir) if deskew else None
+    base = 0
+    if found is None and deskew and rotate and not known:
+        # A page laid at more than 45 degrees (a receipt at about 60, upside down, 01-11-2026): the skew
+        # search covers +-45, so its text rows only show a quarter turn on. Measured there, the skew is
+        # an ordinary one, and OSD then only has to tell 90 from 270.
+        steep = _detect(page, dpi, 90, workdir)
+        if steep is not None and abs(abs(steep) - 45) < STEEP_EDGE:
+            # a chart with slanted axis labels came out a quarter plus about 45 degrees (benchmark 05-11-2026)
+            log.info("skew unsure (%.2f degrees a quarter turn on: too near 45); page kept as scanned", steep)
+        elif steep is not None:
+            base, found = 90, steep
+            log.info("page lies at more than 45 degrees; measured a quarter turn on")
+    angle = found if found is not None and abs(found) >= deskew_min else 0.0
+    if not known and rotate:
+        try:
+            more, sure = orientation.detect(page, workdir, min_confidence, scripts, skew=angle, quarter_turn=base)
+        except Exception as e:  # noqa: BLE001
+            log.warning("orientation not determined (%s); page kept as scanned", e)
+            more, sure = 0, False
+        if base and not sure:
+            # The steep path's quarter turn is only half an answer: 90 and 270 level the same lines, and
+            # only OSD on the turned page tells them apart. Unsure, it was kept, and every wrong turn in the
+            # benchmark (10, 05-11-2026) came from here. Rotations commute, so the angle measured in the
+            # turned frame straightens the page as scanned too: sideways, but level. First the words
+            # decide, as for an unsure upside-down page.
+            try:
+                by_words = orientation.quarter_by_words(page, workdir, skew=angle)
+            except Exception as e:  # noqa: BLE001
+                log.warning("words not read (%s)", e)
+                by_words = None
+            if by_words:
+                base, more = by_words, 0
+            else:
+                log.info("quarter turn not confirmed by the orientation check; page kept sideways")
+                base = 0
+        turn = (base + more) % 360
+        if more in (90, 270):
+            angle = _skew(page, dpi, turn, deskew, deskew_min, workdir)
+    if angle:
+        log.info("page straightened by %.2f degrees", angle)
+    return turn, angle
+
+
+def _detect(page: Page, dpi: int, turn: int, workdir: str | None) -> float | None:
+    """skew.detect, None when unsure or failing."""
+    try:
+        return skew.detect(page, dpi, turn, workdir)
+    except Exception as e:  # noqa: BLE001
+        log.warning("skew not measured (%s); page kept as scanned", e)
+        return None
+
+
+def _skew(page: Page, dpi: int, turn: int, deskew: bool, deskew_min: float, workdir: str | None) -> float:
+    if not deskew:
+        return 0.0
+    found = _detect(page, dpi, turn, workdir)
+    return found if found is not None and abs(found) >= deskew_min else 0.0