pdfopt 0.2.0__tar.gz

pdfopt-0.2.0/CHANGELOG.md

@@ -0,0 +1,133 @@
+# Changelog
+
+## Unreleased
+
+**Audit fixes**
+- Font cutting: the cmap tables are rebuilt after the subset. Fonts without
+  `/Encoding` and with a single (1, 0) table (LibreOffice export) lost the way
+  from code to glyph, the rendering check rejected the result, and the image
+  changes were switched off together with the fonts. This works now, and when a
+  font cut fails, only the font cutting is dropped (`lossless-nofonts`).
+- `-o`/`--output` (a file or a folder; the sources are not touched),
+  `-j`/`--jobs`, `--version`. A symlink or a non-PDF named on the command line
+  is explained (exit code 2 when there is nothing to do); the total shows
+  "already optimized: N".
+- The number of files processed at once is bounded by free memory (about 700 MB
+  per file with `-q`, 300 MB without).
+- Dockerfile with all external programs; English README and a results table;
+  release workflow (cibuildwheel wheels + sdist → PyPI on a `v*` tag);
+  `py.typed`.
+- The "pixel-identical" wording is exact now: the check runs at 100 dpi.
+- All messages, `--help`, docs and tests are in plain English.
+
+**Speed**
+- The render check splits pages into chunks by core count instead of 8 pages:
+  with 9 pages only 2 of 12 threads were busy. Layered (MRC) mode: a 9-page
+  colour scan 55 → 22 s, a grayscale one 42 → 11 s (same result).
+- Compositing while the background is searched is about 15 % faster.
+
+**Interface**
+- Without `-v` the output is minimal: a progress line with the share saved (on a
+  terminal) and a total line. Warnings about `hpdf-worker` and ECT appear only
+  with `-v`; fontTools warnings about slightly damaged fonts are silenced.
+- `--decrypt`: files with encryption that open without a password are decrypted
+  and optimized (the result has no encryption). Files that need a password are
+  still skipped.
+
+**Lossless**
+- TrueType fonts are cut to the glyphs in use (fontTools, glyph ids kept): a Word
+  document 1414 → 1073 KB (−24 %), pdfTeX with images from Office −16 %.
+- JPEG is repacked with mozjpeg's `jpegtran` instead of libjpeg: −0.7 % on photo
+  scans, −12…28 % on JPEG in papers (files −6…8 %); the same for the JPEG that
+  pdfopt creates itself.
+- ECT level 6 for streams above 4 MB: a paper with a 25 MB image 38 s → 11 s at
+  a cost of about 0.1 %; Flate is not unpacked beyond the ECT budget.
+- The ECT module rejects levels outside 2–9 (level 1 did not finish).
+- New required dependencies: `fonttools`, `mozjpeg-lossless-optimization`;
+  `jpegtran` is no longer needed.
+
+**Lossy**
+- MRC: size on par with DjVu at the same quality (9-page scans, `-q 90` and
+  `-q 96`: grayscale 298 → 104 KB, colour 390 → 112 KB; DjVu 108 and 111 KB).
+  - JBIG2 with a symbol match threshold chosen by letter size, per band of
+    lines: 0.80 for normal text, 0.85/0.90 for small text, very small text
+    lossless; footnotes get their own mask (protects against 6↔8 and 9↔8 swaps).
+  - Thresholds stay at 0.9 or below: jbig2enc 0.28 (Ubuntu 24.04) rejected
+    0.92, and the symbol mode silently did not work.
+  - The mask is smoothed by a 3×3 vote.
+  - Background at 100/50 dpi instead of 150/100; JPEG 2000 with ratios up to
+    2000:1 (before, at most 50:1, which almost never beat JPEG).
+  - Coloured text: one mask per ink instead of a colour JPEG image.
+  - A separate threshold for text pages, `--mrc-quality` (default `-q` − 6):
+    with `-q 96` MRC used to fail on real scans (a 10.9 MB grayscale scan →
+    104 KB).
+  - Scans with an ICC profile (ICCBased, CalRGB) keep their colour space in the
+    layers; before, the background was written as DeviceRGB/DeviceGray.
+- Ghostscript is really offered to documents without text: the text-layer test
+  fired on any page with content, so the engine never ran.
+
+**Reliability**
+- A failure on one MRC page no longer cancels MRC and JPEG for the whole
+  document.
+- Image decoding is bounded (512 MB; Flate only to the needed size): a
+  decompression bomb cannot exhaust memory. Pillow's `DecompressionBombError`
+  is caught everywhere.
+- Lossless RGB→Gray leaves images with `/Decode` alone (the array would become
+  invalid).
+- When root replaces a file, the owner is kept.
+
+**Build**
+- MIT license (`LICENSE`); the package as a whole is `MIT AND Apache-2.0`
+  because of the bundled ECT and Zopfli.
+- The sdist builds: `MANIFEST.in` with the ECT headers; the ECT/Zopfli licenses
+  (Apache 2.0) go into the sdist and the wheel (`license-files`); the C++
+  sources are no longer put in the wheel. CI builds the sdist and a wheel from it.
+- CI compares SSIM with ffmpeg again and runs the Ghostscript tests.
+- CI runs on Linux, macOS and Windows; `setup.py` knows MSVC and the ECT
+  extension is optional (zlib is used if it does not build).
+
+**API**
+- `optimize_pdf`: everything after `mode` is keyword-only (a positional
+  `dry_run` was easily mixed up with the threshold).
+
+## TODO
+
+- **Smoother `-q` scale on scans (decision postponed).** `-q 90`, `-q 85` and
+  `-q 80` give byte-identical files on scans, while `-q 96` gives a file four
+  times larger. The search reaches its floor (background at the lowest dpi and
+  JPEG quality). Options: lower the floors (more dpi steps, a wider JPEG quality
+  range) and measure the quality cost, or print the threshold that was really
+  reached.
+
+## 0.2.0
+
+**Lossless**
+- One parse of the file instead of five independent engines; the passes add up.
+- pikepdf instead of the external `qpdf` (object/xref streams are always
+  available), ECT -9 (sources from katzip, `pdfopt/_ect`) for Deflate.
+- Image re-encoding: RGB→Gray, 8→1 bit, PNG predictors, G4/Flate→JBIG2 generic.
+- Stream deduplication, thumbnail removal.
+- Pixel check of the changed pages.
+
+**Lossy**
+- MRC: adaptive binarization, normalized background filling, a colour text
+  layer, empty pages, JBIG2 in chunks, scans in Flate/JPX, shared layers.
+- JPEG: dpi from the page geometry, quality bisection, Flate photos → JPEG.
+
+**Speed**
+- SSIM in-process (ffmpeg's formula), no `ffmpeg` process per page.
+- Candidates are checked from the smallest; the fingerprint uses content hashes
+  instead of text extraction.
+- Rendering in chunks, a shared reference, early exit; parallelism across files.
+
+**Reliability**
+- The `/PdfOpt` marker: no second compression; `--force`.
+- Signed or encrypted files are "skipped", exit code 0.
+- The signature is searched in objects, not in the file text.
+
+**CLI:** `-v`, `--force`, `--safe-text`. Removed: `-j` (always all cores; it
+came back later), `--fast` (always ECT -9), `--version` (it came back later),
+`--pdf-worker` (`hpdf-worker` is taken from `PATH`, a warning if it is missing),
+`--mode`, `--lossless`, `--no-dry-run`, `--no-recursive`, the variables
+`*_HYPER_WORKER`/`HYPER_DRV`, the `third_party` folder. Fixed a crash of
+`pdfopt -h` (a `%` sign in the help).

pdfopt-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 bortoq
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pdfopt-0.2.0/MANIFEST.in

@@ -0,0 +1,6 @@
+# The ECT extension needs its headers to build from the sdist.
+recursive-include pdfopt/_ect *.c *.cpp *.h License.txt
+recursive-include pdfopt/_ect/zopfli COPYING CONTRIBUTORS
+include THIRD_PARTY_NOTICES CHANGELOG.md
+recursive-include docs *.md
+recursive-include tests *.py

pdfopt-0.2.0/PKG-INFO

@@ -0,0 +1,217 @@
+Metadata-Version: 2.4
+Name: pdfopt
+Version: 0.2.0
+Summary: PDF optimizer: lossless recompression, JPEG re-encoding and layered MRC for scans
+License-Expression: MIT AND Apache-2.0
+Project-URL: Homepage, https://github.com/bortoq/pdf-optimizer
+Project-URL: Issues, https://github.com/bortoq/pdf-optimizer/issues
+Keywords: pdf,compress,optimize,scan,mrc,jbig2
+Classifier: Environment :: Console
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Multimedia :: Graphics :: Graphics Conversion
+Classifier: Topic :: Utilities
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: THIRD_PARTY_NOTICES
+License-File: pdfopt/_ect/License.txt
+License-File: pdfopt/_ect/zopfli/COPYING
+Requires-Dist: pikepdf<11,>=9
+Requires-Dist: pypdf<7,>=5
+Requires-Dist: Pillow<13,>=10
+Requires-Dist: numpy<3,>=1.26
+Requires-Dist: fonttools<5,>=4.50
+Requires-Dist: mozjpeg-lossless-optimization<2,>=1.3
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: ruff>=0.5; extra == "dev"
+Dynamic: license-file
+
+# pdfopt
+
+A PDF optimizer. It tries several encoders, picks the smallest **verified**
+result and replaces the source file atomically, only if the file got smaller.
+
+- **Lossless (default)**: text and images stay as they are; every page whose
+  images or fonts changed is compared pixel for pixel at 100 dpi.
+- **Lossy (`-q N`)**: recompresses photos and scans for as long as the SSIM of
+  every page stays at or above `N/100`.
+
+Everything runs locally; no file leaves your machine. Design notes are in
+[docs/architecture.md](docs/architecture.md).
+
+## Results
+
+Scans at 300 dpi, 9 pages, page SSIM ≥ 0.90 (`-q 90`), against Ghostscript:
+
+| File | Source | pdfopt `-q 90` | Ghostscript `/ebook` | Ghostscript `/screen` |
+| --- | --- | --- | --- | --- |
+| grayscale scan | 14,939 KB | **104 KB**, SSIM 0.965, 11 s | 1,719 KB, SSIM 0.958 | 685 KB, SSIM 0.889 |
+| colour scan | 12,916 KB | **112 KB**, SSIM 0.970, 22 s | 11,200 KB | 1,936 KB, SSIM 0.896 |
+
+Lossless (KB, source → pdfopt; `pikepdf`/qpdf in brackets):
+
+| Document | Result |
+| --- | --- |
+| dvips paper | 146 → **124** (130) |
+| pdfTeX paper | 776 → **570** (747) |
+| Word document | 1595 → **1073** (1532) |
+| CCITT scan | 553 → **396** (539) |
+
+An independent audit measured a 6-page 300 dpi scan (7.8 MB) at 111 KB with
+SSIM 0.962, against 655 KB / 0.880 for DjVu (`pdf2djvu --lossy`).
+
+## Install
+
+```sh
+pip install -e .            # pikepdf, pypdf, Pillow, numpy, fontTools, mozjpeg + builds ECT
+pip install -e '.[dev]'     # + pytest, ruff
+```
+
+Python ≥ 3.11 and C/C++ compilers are needed (the ECT module is built on
+install; if the build fails, installation still works and streams are packed
+with zlib). Running straight from a source checkout: `python3 setup.py build_ext --inplace`.
+External programs are listed [below](#external-programs).
+
+**Docker** (all programs included):
+
+```sh
+docker build -t pdfopt .
+docker run --rm -v "$PWD":/data pdfopt -q 96 book.pdf
+```
+
+## Usage
+
+```sh
+pdfopt book.pdf                      # lossless
+pdfopt -q 96 book.pdf                # lossy, page SSIM >= 0.96
+pdfopt --dry-run -r /path/to/books   # estimate the saving for a tree
+pdfopt -v *.pdf                      # verbose log
+pdfopt --decrypt locked.pdf          # encrypted, but opens without a password
+pdfopt -o small.pdf book.pdf         # write a new file, keep the source
+pdfopt -o out/ -j 4 *.pdf            # into out/, at most 4 cores
+python3 -m pdfopt --help
+```
+
+| Option | Meaning |
+| --- | --- |
+| `-q`, `--quality N` | minimum page SSIM `N/100` (1–100); turns on lossy mode |
+| `--dry-run` | compute the result without replacing the source |
+| `-r`, `--recursive` | descend into subfolders |
+| `-v`, `--verbose` | per-file results; on stderr the size and rejection reason of each engine, timings, warnings about missing programs |
+| `--decrypt` | decrypt files that open without a password and optimize them (the owner's restrictions, such as printing or copying, are dropped) |
+| `-o`, `--output PATH` | leave sources alone: write the result to a file (one input) or a directory (several); unchanged and skipped files are copied as they are |
+| `-j`, `--jobs N` | cores to use (default: all) |
+| `--version` | print the version |
+| `--force` | process a file even if it is already optimized |
+| `--safe-text` | with `-q`, code text masks without JBIG2 symbol mode (larger, but similar letters can never be swapped) |
+| `--mrc-quality N` | minimum SSIM of MRC text pages (default 6 below `-q`; 100 with `-q 100`) |
+
+**Exit codes:** `0` success (including "no change" and skipped files), `1` at
+least one file failed, `2` bad arguments, or nothing to do because every path
+named was rejected (a symlink, not a `.pdf`).
+
+Without `-v` a terminal shows one progress line with the share saved so far,
+and a single total at the end. Signed PDFs and password-protected files are
+**skipped** (`skipped: ...`), which is not an error; files encrypted without a
+password are processed only with `--decrypt`. Already optimized files are
+counted in the total. When several files run at once, their number is bounded
+by free memory (about 700 MB per file with `-q`).
+
+## What it does
+
+### Lossless
+
+One parse, several passes, one save:
+
+1. drops page thumbnails, merges identical streams;
+2. images: RGB with R=G=B becomes DeviceGray, pure black-and-white 8-bit images
+   become 1-bit, Flate images get the best PNG predictor, 1-bit Flate and CCITT
+   Group 4 are rewritten as **JBIG2 generic** (lossless);
+3. JPEG: mozjpeg's `jpegtran` (optimized Huffman tables, tuned progressive
+   scans) with a pixel-identity check, 0.7–28 % smaller than libjpeg's;
+4. embedded TrueType fonts are cut to the glyphs the document shows (fontTools,
+   glyph ids kept, content streams unchanged); the cmap tables are rebuilt, so
+   fonts without `/Encoding` (LibreOffice) work too. Pages with changed fonts
+   are compared pixel for pixel; with an AcroForm or any unclear font use the
+   font is left alone, and if a cut ever changes the rendering only the font
+   step is dropped, not the image steps;
+5. streams are recompressed by the bundled ECT (like `ect -9`; streams above
+   4 MB like `ect -6`) and saved as object/xref streams.
+
+Metadata, text layer, geometry and annotations do not change.
+
+### Lossy (`-q`)
+
+On top of the lossless result:
+
+- **MRC** for scans (page = one picture): a text-free background at 50–100 dpi
+  (JPEG or JPEG 2000, quality searched by SSIM), a full-resolution text mask
+  (JBIG2 with shared dictionaries, Group 4 or Flate), one ink colour per mask.
+  The JBIG2 symbol threshold depends on letter size, and very small text is
+  coded losslessly, so 6 and 8 are never merged the way Xerox scanners did in 2013.
+- **JPEG** for other photos: the smallest JPEG (and dpi, from the content-stream
+  geometry) that passes SSIM; Flate photos become JPEG.
+- **Ghostscript** profiles (300/240/200 dpi) for PDFs without text or forms.
+- **Hyper Compress**, an external `hpdf-worker`, if found in `PATH`.
+
+## How a result is verified
+
+Candidates are sorted by size and checked from the smallest; the first that
+passes wins.
+
+| Check | When |
+| --- | --- |
+| page count, MediaBox/CropBox, rotation, annotation count | always |
+| content stream of every page byte for byte (else text compare) | lossless |
+| metadata (Info and XMP) | lossless |
+| rendering identical at 100 dpi on changed pages | lossless, when images or fonts changed |
+| rendering SSIM (200 dpi, `pdftoppm`) ≥ threshold | lossy: every MRC page and re-encoded page (failing pages revert), all pages for Ghostscript/Hyper |
+
+SSIM is computed in-process (numpy) with FFmpeg's `ssim` formula.
+
+A marker (`/Root /PdfOpt`) keeps a second run from recompressing a file; `--force`
+overrides it.
+
+## External programs
+
+| Program | Used for | Without it |
+| --- | --- | --- |
+| `pdftoppm` (poppler) | render checks, **required for `-q`** | lossless drops image transforms; `-q` unavailable |
+| `jbig2` (jbig2enc) | JBIG2 generic and symbol dictionaries | Group 4/Flate |
+| `opj_compress` (OpenJPEG) | JPEG 2000 backgrounds | JPEG only |
+| `cjpeg` from mozjpeg | denser JPEG | Pillow/libjpeg |
+| `gs` (Ghostscript) | Ghostscript profiles | skipped |
+| `hpdf-worker` (in `PATH`) | Hyper Compress | skipped with a warning |
+
+## Layout
+
+```
+pdfopt/
+  cli.py        arguments, file discovery, parallelism across files
+  optimize.py   orchestrator: stages, candidates, verification, marker, atomic replace
+  lossless.py   lossless passes (structure, images, JPEG, ECT, save)
+  fonts.py      TrueType subsetting
+  jpegopt.py    lossy photo recompression
+  mrc.py        layered scan compression
+  pdfmodel.py   reading: signature, page fingerprint, marker, image walk
+  imagedata.py  image decoding, PNG predictors, Group 4
+  render.py     pdftoppm, VisualGate (SSIM), identity check
+  ssim.py       SSIM matching ffmpeg
+  tools.py      external program lookup
+```
+
+## Tests
+
+```sh
+pytest
+ruff check .
+```
+
+CI runs the tests on Linux, macOS and Windows. Tests that need `pdftoppm`,
+`jbig2`, `ffmpeg`, `gs` or OpenJPEG are skipped when the program is missing.
+
+## License
+
+MIT, see [LICENSE](LICENSE). The bundled ECT and Zopfli (`pdfopt/_ect`) are under
+the Apache License 2.0, see [THIRD_PARTY_NOTICES](THIRD_PARTY_NOTICES).

pdfopt-0.2.0/README.md

@@ -0,0 +1,188 @@
+# pdfopt
+
+A PDF optimizer. It tries several encoders, picks the smallest **verified**
+result and replaces the source file atomically, only if the file got smaller.
+
+- **Lossless (default)**: text and images stay as they are; every page whose
+  images or fonts changed is compared pixel for pixel at 100 dpi.
+- **Lossy (`-q N`)**: recompresses photos and scans for as long as the SSIM of
+  every page stays at or above `N/100`.
+
+Everything runs locally; no file leaves your machine. Design notes are in
+[docs/architecture.md](docs/architecture.md).
+
+## Results
+
+Scans at 300 dpi, 9 pages, page SSIM ≥ 0.90 (`-q 90`), against Ghostscript:
+
+| File | Source | pdfopt `-q 90` | Ghostscript `/ebook` | Ghostscript `/screen` |
+| --- | --- | --- | --- | --- |
+| grayscale scan | 14,939 KB | **104 KB**, SSIM 0.965, 11 s | 1,719 KB, SSIM 0.958 | 685 KB, SSIM 0.889 |
+| colour scan | 12,916 KB | **112 KB**, SSIM 0.970, 22 s | 11,200 KB | 1,936 KB, SSIM 0.896 |
+
+Lossless (KB, source → pdfopt; `pikepdf`/qpdf in brackets):
+
+| Document | Result |
+| --- | --- |
+| dvips paper | 146 → **124** (130) |
+| pdfTeX paper | 776 → **570** (747) |
+| Word document | 1595 → **1073** (1532) |
+| CCITT scan | 553 → **396** (539) |
+
+An independent audit measured a 6-page 300 dpi scan (7.8 MB) at 111 KB with
+SSIM 0.962, against 655 KB / 0.880 for DjVu (`pdf2djvu --lossy`).
+
+## Install
+
+```sh
+pip install -e .            # pikepdf, pypdf, Pillow, numpy, fontTools, mozjpeg + builds ECT
+pip install -e '.[dev]'     # + pytest, ruff
+```
+
+Python ≥ 3.11 and C/C++ compilers are needed (the ECT module is built on
+install; if the build fails, installation still works and streams are packed
+with zlib). Running straight from a source checkout: `python3 setup.py build_ext --inplace`.
+External programs are listed [below](#external-programs).
+
+**Docker** (all programs included):
+
+```sh
+docker build -t pdfopt .
+docker run --rm -v "$PWD":/data pdfopt -q 96 book.pdf
+```
+
+## Usage
+
+```sh
+pdfopt book.pdf                      # lossless
+pdfopt -q 96 book.pdf                # lossy, page SSIM >= 0.96
+pdfopt --dry-run -r /path/to/books   # estimate the saving for a tree
+pdfopt -v *.pdf                      # verbose log
+pdfopt --decrypt locked.pdf          # encrypted, but opens without a password
+pdfopt -o small.pdf book.pdf         # write a new file, keep the source
+pdfopt -o out/ -j 4 *.pdf            # into out/, at most 4 cores
+python3 -m pdfopt --help
+```
+
+| Option | Meaning |
+| --- | --- |
+| `-q`, `--quality N` | minimum page SSIM `N/100` (1–100); turns on lossy mode |
+| `--dry-run` | compute the result without replacing the source |
+| `-r`, `--recursive` | descend into subfolders |
+| `-v`, `--verbose` | per-file results; on stderr the size and rejection reason of each engine, timings, warnings about missing programs |
+| `--decrypt` | decrypt files that open without a password and optimize them (the owner's restrictions, such as printing or copying, are dropped) |
+| `-o`, `--output PATH` | leave sources alone: write the result to a file (one input) or a directory (several); unchanged and skipped files are copied as they are |
+| `-j`, `--jobs N` | cores to use (default: all) |
+| `--version` | print the version |
+| `--force` | process a file even if it is already optimized |
+| `--safe-text` | with `-q`, code text masks without JBIG2 symbol mode (larger, but similar letters can never be swapped) |
+| `--mrc-quality N` | minimum SSIM of MRC text pages (default 6 below `-q`; 100 with `-q 100`) |
+
+**Exit codes:** `0` success (including "no change" and skipped files), `1` at
+least one file failed, `2` bad arguments, or nothing to do because every path
+named was rejected (a symlink, not a `.pdf`).
+
+Without `-v` a terminal shows one progress line with the share saved so far,
+and a single total at the end. Signed PDFs and password-protected files are
+**skipped** (`skipped: ...`), which is not an error; files encrypted without a
+password are processed only with `--decrypt`. Already optimized files are
+counted in the total. When several files run at once, their number is bounded
+by free memory (about 700 MB per file with `-q`).
+
+## What it does
+
+### Lossless
+
+One parse, several passes, one save:
+
+1. drops page thumbnails, merges identical streams;
+2. images: RGB with R=G=B becomes DeviceGray, pure black-and-white 8-bit images
+   become 1-bit, Flate images get the best PNG predictor, 1-bit Flate and CCITT
+   Group 4 are rewritten as **JBIG2 generic** (lossless);
+3. JPEG: mozjpeg's `jpegtran` (optimized Huffman tables, tuned progressive
+   scans) with a pixel-identity check, 0.7–28 % smaller than libjpeg's;
+4. embedded TrueType fonts are cut to the glyphs the document shows (fontTools,
+   glyph ids kept, content streams unchanged); the cmap tables are rebuilt, so
+   fonts without `/Encoding` (LibreOffice) work too. Pages with changed fonts
+   are compared pixel for pixel; with an AcroForm or any unclear font use the
+   font is left alone, and if a cut ever changes the rendering only the font
+   step is dropped, not the image steps;
+5. streams are recompressed by the bundled ECT (like `ect -9`; streams above
+   4 MB like `ect -6`) and saved as object/xref streams.
+
+Metadata, text layer, geometry and annotations do not change.
+
+### Lossy (`-q`)
+
+On top of the lossless result:
+
+- **MRC** for scans (page = one picture): a text-free background at 50–100 dpi
+  (JPEG or JPEG 2000, quality searched by SSIM), a full-resolution text mask
+  (JBIG2 with shared dictionaries, Group 4 or Flate), one ink colour per mask.
+  The JBIG2 symbol threshold depends on letter size, and very small text is
+  coded losslessly, so 6 and 8 are never merged the way Xerox scanners did in 2013.
+- **JPEG** for other photos: the smallest JPEG (and dpi, from the content-stream
+  geometry) that passes SSIM; Flate photos become JPEG.
+- **Ghostscript** profiles (300/240/200 dpi) for PDFs without text or forms.
+- **Hyper Compress**, an external `hpdf-worker`, if found in `PATH`.
+
+## How a result is verified
+
+Candidates are sorted by size and checked from the smallest; the first that
+passes wins.
+
+| Check | When |
+| --- | --- |
+| page count, MediaBox/CropBox, rotation, annotation count | always |
+| content stream of every page byte for byte (else text compare) | lossless |
+| metadata (Info and XMP) | lossless |
+| rendering identical at 100 dpi on changed pages | lossless, when images or fonts changed |
+| rendering SSIM (200 dpi, `pdftoppm`) ≥ threshold | lossy: every MRC page and re-encoded page (failing pages revert), all pages for Ghostscript/Hyper |
+
+SSIM is computed in-process (numpy) with FFmpeg's `ssim` formula.
+
+A marker (`/Root /PdfOpt`) keeps a second run from recompressing a file; `--force`
+overrides it.
+
+## External programs
+
+| Program | Used for | Without it |
+| --- | --- | --- |
+| `pdftoppm` (poppler) | render checks, **required for `-q`** | lossless drops image transforms; `-q` unavailable |
+| `jbig2` (jbig2enc) | JBIG2 generic and symbol dictionaries | Group 4/Flate |
+| `opj_compress` (OpenJPEG) | JPEG 2000 backgrounds | JPEG only |
+| `cjpeg` from mozjpeg | denser JPEG | Pillow/libjpeg |
+| `gs` (Ghostscript) | Ghostscript profiles | skipped |
+| `hpdf-worker` (in `PATH`) | Hyper Compress | skipped with a warning |
+
+## Layout
+
+```
+pdfopt/
+  cli.py        arguments, file discovery, parallelism across files
+  optimize.py   orchestrator: stages, candidates, verification, marker, atomic replace
+  lossless.py   lossless passes (structure, images, JPEG, ECT, save)
+  fonts.py      TrueType subsetting
+  jpegopt.py    lossy photo recompression
+  mrc.py        layered scan compression
+  pdfmodel.py   reading: signature, page fingerprint, marker, image walk
+  imagedata.py  image decoding, PNG predictors, Group 4
+  render.py     pdftoppm, VisualGate (SSIM), identity check
+  ssim.py       SSIM matching ffmpeg
+  tools.py      external program lookup
+```
+
+## Tests
+
+```sh
+pytest
+ruff check .
+```
+
+CI runs the tests on Linux, macOS and Windows. Tests that need `pdftoppm`,
+`jbig2`, `ffmpeg`, `gs` or OpenJPEG are skipped when the program is missing.
+
+## License
+
+MIT, see [LICENSE](LICENSE). The bundled ECT and Zopfli (`pdfopt/_ect`) are under
+the Apache License 2.0, see [THIRD_PARTY_NOTICES](THIRD_PARTY_NOTICES).

pdfopt-0.2.0/THIRD_PARTY_NOTICES

@@ -0,0 +1,5 @@
+pdfopt/_ect — Efficient Compression Tool (ECT) Deflate compressor,
+(c) 2014-2026 Felix Hanau, with Zopfli by Google Inc. (Apache License 2.0).
+Sources were taken from the katzip project with its patches
+(ect.patch, ect_alignment.patch) applied. See pdfopt/_ect/License.txt and
+pdfopt/_ect/zopfli/COPYING.