PyPatchMatch 2.0.0rc6__tar.gz → 2.0.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- pypatchmatch-2.0.2/CLAUDE.md +158 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/PKG-INFO +13 -6
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/README.md +12 -5
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/_version.py +3 -3
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/csrc/masked_image.cpp +4 -2
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/csrc/nnf.cpp +9 -4
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/patch_match.py +33 -9
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/pyproject.toml +14 -3
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/scripts/build_opencv.py +13 -7
- pypatchmatch-2.0.2/scripts/render_readme_example.py +36 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/tests/test_patch_match.py +104 -12
- pypatchmatch-2.0.0rc6/CLAUDE.md +0 -479
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/CMakeLists.txt +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/LICENSE +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/SECURITY.md +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/examples/images/forest_pruned.bmp +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/__init__.py +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/_lib.py +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/csrc/inpaint.cpp +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/csrc/inpaint.h +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/csrc/masked_image.h +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/csrc/nnf.h +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/csrc/pyinterface.cpp +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/csrc/pyinterface.h +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/patchmatch/py.typed +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/scripts/evaluate_inpainting.py +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/tests/conftest.py +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/tests/test_e2e.py +0 -0
- {pypatchmatch-2.0.0rc6 → pypatchmatch-2.0.2}/tests/test_properties.py +0 -0
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
# CLAUDE.md
|
|
2
|
+
|
|
3
|
+
This file provides guidance to Claude Code (claude.ai/code) when working with code in
|
|
4
|
+
this repository. The details of the native code, the C interface, packaging and CI are
|
|
5
|
+
path-scoped rules in `.claude/rules/`, which load when a matching file is read or
|
|
6
|
+
edited; read the rule before changing such a file through Bash. The sanitizer, C++
|
|
7
|
+
coverage and fill-quality runs are skills in `.claude/skills/`.
|
|
8
|
+
|
|
9
|
+
## Project
|
|
10
|
+
|
|
11
|
+
PyPatchMatch implements PatchMatch based image inpainting (Barnes et al., SIGGRAPH 2009):
|
|
12
|
+
a C++17 library on top of OpenCV core, and a thin Python package that loads it through
|
|
13
|
+
`ctypes`. It is published on PyPI as `PyPatchMatch` (import name `patchmatch`) and used
|
|
14
|
+
by InvokeAI. This repository (`mauwii/PyPatchMatch`) is the maintained fork of
|
|
15
|
+
`invoke-ai/PyPatchMatch`, which derives from `vacancy/PyPatchMatch` and
|
|
16
|
+
`younesse-cv/PatchMatch`.
|
|
17
|
+
|
|
18
|
+
## Commands
|
|
19
|
+
|
|
20
|
+
The project is managed with uv (`.python-version` pins 3.12 for the local venv). Building
|
|
21
|
+
needs a C++17 compiler, CMake >= 3.21 and an OpenCV that `find_package(OpenCV)` finds:
|
|
22
|
+
Homebrew `opencv`, `libopencv-dev`, or an installation prefix in `OpenCV_ROOT`.
|
|
23
|
+
|
|
24
|
+
```sh
|
|
25
|
+
uv sync # .venv, editable install, compiles the library
|
|
26
|
+
uv run pre-commit install
|
|
27
|
+
uv run pytest # all tests, with the coverage gate
|
|
28
|
+
uv run pytest -m "not e2e" # fast unit tests only
|
|
29
|
+
uv run pytest -m e2e --no-cov # end-to-end tests only
|
|
30
|
+
uv run pytest tests/test_patch_match.py::test_inpaint_global_mask --no-cov
|
|
31
|
+
uv run pytest -k regularity --no-cov
|
|
32
|
+
uv run mypy # strict, also a pre-commit hook
|
|
33
|
+
uv run pre-commit run --all-files # what the CI lint job runs
|
|
34
|
+
uv run ruff check --fix . && uv run ruff format .
|
|
35
|
+
uv build # sdist and wheel into dist/
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
- Pass `--no-cov` for any partial run: `addopts` always enables coverage, and
|
|
39
|
+
`fail_under = 95` (branch coverage, subprocesses included through
|
|
40
|
+
`patch = ["subprocess"]`) fails the run otherwise.
|
|
41
|
+
- Use `uv run pytest`, not `python -m pytest`. The latter puts the repository root on
|
|
42
|
+
`sys.path`, which imports the source tree `patchmatch/` without the compiled library;
|
|
43
|
+
the symptoms are the warning `patchmatch failed to load: libpatchmatch.* not found`
|
|
44
|
+
and `RuntimeError: the patchmatch native library is not available` in every test.
|
|
45
|
+
- `--cov=patchmatch` measures the source tree, so coverage only works with the editable
|
|
46
|
+
install that `uv sync` creates. With a non-editable install (`uv sync --no-editable`,
|
|
47
|
+
`uv pip install .`) it reports 0 % and the gate fails.
|
|
48
|
+
- `uv sync` rebuilds the library only when a `tool.uv.cache-keys` entry changes
|
|
49
|
+
(`CMakeLists.txt`, `patchmatch/csrc/*`, `pyproject.toml`, git commit or tags). Force it
|
|
50
|
+
with `uv sync --reinstall-package pypatchmatch`.
|
|
51
|
+
- CI builds the library with warnings as errors. Locally:
|
|
52
|
+
`SKBUILD_CMAKE_DEFINE=CMAKE_COMPILE_WARNING_AS_ERROR=ON uv sync --reinstall-package pypatchmatch`.
|
|
53
|
+
|
|
54
|
+
Build against the same static, core-only OpenCV as the release wheels:
|
|
55
|
+
|
|
56
|
+
```sh
|
|
57
|
+
export OpenCV_ROOT=/tmp/opencv
|
|
58
|
+
uv run --no-project python scripts/build_opencv.py # pinned version and SHA-256
|
|
59
|
+
uv sync --reinstall-package pypatchmatch
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Other entry points:
|
|
63
|
+
|
|
64
|
+
- `examples/cpp_example_run.sh` builds the library and `examples/cpp_example.cpp` with
|
|
65
|
+
CMake into `build/cpp` and runs it. It needs OpenCV `imgcodecs`, so a full OpenCV, not
|
|
66
|
+
the core-only build of the script.
|
|
67
|
+
- `uv run python examples/py_example.py` (and `py_example_global_mask.py`) write
|
|
68
|
+
`examples/images/forest_recovered.bmp`, which is git-ignored.
|
|
69
|
+
- `uv run python scripts/render_readme_example.py` runs `py_example.py` and renders the
|
|
70
|
+
picture at the top of the README, `examples/images/readme_example.jpg`; rerun it and
|
|
71
|
+
commit the picture after changes that affect the fills.
|
|
72
|
+
- CMake options: `PATCHMATCH_BUILD_EXAMPLES` (OFF), `PATCHMATCH_FAST_MATH` (ON,
|
|
73
|
+
`-ffast-math` or `/fp:fast`), `PATCHMATCH_SANITIZE` (OFF, ASan and UBSan with
|
|
74
|
+
`float-cast-overflow`, every finding aborts; turns off fast math, which lets the
|
|
75
|
+
compiler drop checks; GCC and Clang only), `PATCHMATCH_COVERAGE` (OFF, gcov counters
|
|
76
|
+
at `-Og`; GCC and Clang only).
|
|
77
|
+
|
|
78
|
+
## Architecture
|
|
79
|
+
|
|
80
|
+
### Layers, in the call order of `patchmatch.inpaint`
|
|
81
|
+
|
|
82
|
+
1. `patchmatch/__init__.py` re-exports the public API of `patch_match.py` and
|
|
83
|
+
`__version__` from the generated `_version.py`. `test_top_level_exports` checks that
|
|
84
|
+
every name in `patch_match.__all__` is re-exported.
|
|
85
|
+
2. `patchmatch/patch_match.py` is the public API. It validates and canonicalizes the
|
|
86
|
+
inputs (`_prepare_inputs`, `_check_patch_size`, the `ijmap` and `guide_weight`
|
|
87
|
+
checks), returns a copy of the image without a native call if `_has_holes` finds no
|
|
88
|
+
hole outside the global mask, picks the entry point (`PM_inpaint`, `PM_inpaint2`
|
|
89
|
+
with a global mask, and their `_regularity` variants) and runs it through `_call`.
|
|
90
|
+
It loads the library at import time; on `OSError` (no library) or `AttributeError`
|
|
91
|
+
(one without the `PM_*` symbols) it logs a warning and sets
|
|
92
|
+
`patchmatch_available = False`, and every function then raises `RuntimeError`
|
|
93
|
+
through `_get_lib()`.
|
|
94
|
+
3. `patchmatch/_lib.py` is the ctypes layer: the `CShapeT`/`CMatT` structures, the dtype
|
|
95
|
+
table, `np_to_pymat`/`pymat_to_np`, `find_library` (looks for `libpatchmatch.so`,
|
|
96
|
+
`libpatchmatch.dylib` or `patchmatch.dll` in every directory of `patchmatch.__path__`)
|
|
97
|
+
and `load_library` (argtypes and restype of every `PM_*` function).
|
|
98
|
+
4. `patchmatch/csrc/pyinterface.{h,cpp}` is the `extern "C"` API: the `PM_*` functions,
|
|
99
|
+
the conversion between `PM_mat_t` and `cv::Mat`, the global settings and the error
|
|
100
|
+
reporting.
|
|
101
|
+
5. `csrc/inpaint.{h,cpp}`: `Inpainting`, the image pyramid and the EM loop.
|
|
102
|
+
6. `csrc/nnf.{h,cpp}`: `NearestNeighborField` (propagation and random search) and the
|
|
103
|
+
distance metrics `PatchSSDDistanceMetric` and `RegularityGuidedPatchDistanceMetricV2`.
|
|
104
|
+
7. `csrc/masked_image.{h,cpp}`: `MaskedImage`, an image with its hole mask and optional
|
|
105
|
+
global mask, down- and upsampling, gradients.
|
|
106
|
+
|
|
107
|
+
The C++ headers double as the C++ API: `examples/cpp_example.cpp` uses `Inpainting` and
|
|
108
|
+
`PatchSSDDistanceMetric` directly.
|
|
109
|
+
|
|
110
|
+
### ctypes instead of a CPython extension
|
|
111
|
+
|
|
112
|
+
The library does not use the CPython ABI, so the wheels are tagged
|
|
113
|
+
`py3-none-<platform>` (`wheel.py-api = "py3"`): one wheel per platform serves all Python
|
|
114
|
+
versions, and cibuildwheel only builds `cp311-*`. Keep `csrc` free of `Python.h`,
|
|
115
|
+
pybind11 or nanobind. The C++ sources are excluded from the wheel (`wheel.exclude`);
|
|
116
|
+
CMake installs the shared library into the package directory.
|
|
117
|
+
|
|
118
|
+
## Conventions
|
|
119
|
+
|
|
120
|
+
- Python >= 3.11: keep `from __future__ import annotations`. Ruff targets py311 with the
|
|
121
|
+
rules B, C4, E, F, I, RUF, SIM, UP and W, and formats at 88 columns. `patchmatch/csrc`
|
|
122
|
+
is excluded from ruff. C++ is formatted by clang-format (Microsoft style, 120 columns,
|
|
123
|
+
includes grouped as own header, `<...>`, `"..."`); naming is not checked: `m_`
|
|
124
|
+
members, `k` constants. Commit a reformat separately and add its hash to
|
|
125
|
+
`.git-blame-ignore-revs`.
|
|
126
|
+
- The package ships `py.typed`; public functions are fully annotated. mypy checks
|
|
127
|
+
`patchmatch` and `tests` in strict mode (`[tool.mypy]`); the tests have to use the
|
|
128
|
+
API correctly but need no annotations of their own. A deliberately wrong argument
|
|
129
|
+
goes through a test parameter instead of a `# type: ignore`.
|
|
130
|
+
- Backwards compatibility: `from patchmatch import patch_match` and
|
|
131
|
+
`patch_match.CShapeT`/`CMatT` keep working (tested).
|
|
132
|
+
- Comments explain why: which crash a check prevents, why a construct is avoided.
|
|
133
|
+
- No `NOSONAR` markers: change the code so that the SonarCloud rule is satisfied, and ask
|
|
134
|
+
when that seems impossible. The only exception is `nnf.cpp`: cpp:S2245 flags every
|
|
135
|
+
declaration of `std::mt19937`, which only drives the randomized search.
|
|
136
|
+
- Tests check invariants instead of golden images: holes filled, all other pixels
|
|
137
|
+
unchanged, determinism with `set_random_seed`, fills that blend in with their
|
|
138
|
+
surroundings. `test_patch_match.py` and `test_e2e.py` seed through an autouse fixture.
|
|
139
|
+
Unit tests use a small synthetic image; the `e2e` tests use
|
|
140
|
+
`examples/images/forest_pruned.bmp` and mirror the workflows of the README and the
|
|
141
|
+
examples. `test_properties.py` checks the invariants on random inputs with Hypothesis
|
|
142
|
+
(in the `test` group, so the wheel tests run it too); it seeds inside the tests,
|
|
143
|
+
because Hypothesis rejects function-scoped fixtures, and has no deadline, because
|
|
144
|
+
the sanitizer and coverage builds are much slower.
|
|
145
|
+
- The README documents the behavior (mask semantics, `patch_size` as a radius, the
|
|
146
|
+
fallback installation); update it together with API or installation changes.
|
|
147
|
+
- Commit messages: imperative subject in sentence case without a prefix (e.g. "Reject
|
|
148
|
+
invalid patch sizes and empty regularity maps"), and a body that explains the reason,
|
|
149
|
+
with measurements where relevant. Changes go through PRs from branches such as
|
|
150
|
+
`fix/<topic>`.
|
|
151
|
+
|
|
152
|
+
## Git and GitHub
|
|
153
|
+
|
|
154
|
+
- `origin` is `mauwii/PyPatchMatch`, where PRs and releases go; `upstream` is
|
|
155
|
+
`invoke-ai/PyPatchMatch`. `gh` resolves the default repository to upstream, so pass
|
|
156
|
+
`-R mauwii/PyPatchMatch`.
|
|
157
|
+
- Releases: creating a GitHub release with a tag like `v2.0.0` (or `v2.0.0rc2` for
|
|
158
|
+
pre-releases) builds the wheels and publishes them to PyPI.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: PyPatchMatch
|
|
3
|
-
Version: 2.0.
|
|
3
|
+
Version: 2.0.2
|
|
4
4
|
Summary: This library implements the PatchMatch based inpainting algorithm.
|
|
5
5
|
Keywords: inpainting,patchmatch,image-processing,opencv
|
|
6
6
|
Author-Email: The InvokeAI Project <lincoln.stein@gmail.com>, Jiayuan Mao <maojiayuan@gmail.com>, Kyle Schouviller <kyle0654@hotmail.com>, Matthias Wild <mauwii@outlook.de>, Younesse ANDAM <younesse.andam@gmail.com>
|
|
@@ -42,6 +42,12 @@ and Python interfaces. This implementation is heavily based on the implementatio
|
|
|
42
42
|
Younesse ANDAM: [younesse-cv/PatchMatch](https://github.com/younesse-cv/PatchMatch),
|
|
43
43
|
with some bug fixes, and updates.
|
|
44
44
|
|
|
45
|
+

|
|
46
|
+
|
|
47
|
+
From left to right: the photo, the input of
|
|
48
|
+
[examples/py_example.py](https://github.com/mauwii/PyPatchMatch/blob/main/examples/py_example.py)
|
|
49
|
+
with the birdhouses painted out in white, and its result.
|
|
50
|
+
|
|
45
51
|
## Installation
|
|
46
52
|
|
|
47
53
|
```sh
|
|
@@ -93,11 +99,12 @@ The previous import path `from patchmatch import patch_match` keeps working.
|
|
|
93
99
|
pixels in each direction. Larger patches follow larger structures but are much slower;
|
|
94
100
|
the default of 15 compares 31x31 patches, the examples use 3.
|
|
95
101
|
|
|
96
|
-
`patchmatch.set_random_seed(seed)` sets the seed of the randomized search
|
|
97
|
-
`patchmatch.set_verbose(True)` prints the progress of the native code to
|
|
98
|
-
`patchmatch.inpaint_regularity(image, mask, ijmap)` additionally guides the
|
|
99
|
-
a regularity map, an HxWx3 float32 array with the regularity coordinates of
|
|
100
|
-
in its first two channels. Its `guide_weight` must be
|
|
102
|
+
`patchmatch.set_random_seed(seed)` sets the seed of the randomized search, modulo
|
|
103
|
+
`2**32`, and `patchmatch.set_verbose(True)` prints the progress of the native code to
|
|
104
|
+
stderr. `patchmatch.inpaint_regularity(image, mask, ijmap)` additionally guides the
|
|
105
|
+
search with a regularity map, an HxWx3 float32 array with the regularity coordinates of
|
|
106
|
+
each pixel in its first two channels. Its `guide_weight` must be between 0 and the
|
|
107
|
+
largest float32, about 3.4e38.
|
|
101
108
|
|
|
102
109
|
C++ (see
|
|
103
110
|
[examples/cpp_example.cpp](https://github.com/mauwii/PyPatchMatch/blob/main/examples/cpp_example.cpp),
|
|
@@ -12,6 +12,12 @@ and Python interfaces. This implementation is heavily based on the implementatio
|
|
|
12
12
|
Younesse ANDAM: [younesse-cv/PatchMatch](https://github.com/younesse-cv/PatchMatch),
|
|
13
13
|
with some bug fixes, and updates.
|
|
14
14
|
|
|
15
|
+

|
|
16
|
+
|
|
17
|
+
From left to right: the photo, the input of
|
|
18
|
+
[examples/py_example.py](https://github.com/mauwii/PyPatchMatch/blob/main/examples/py_example.py)
|
|
19
|
+
with the birdhouses painted out in white, and its result.
|
|
20
|
+
|
|
15
21
|
## Installation
|
|
16
22
|
|
|
17
23
|
```sh
|
|
@@ -63,11 +69,12 @@ The previous import path `from patchmatch import patch_match` keeps working.
|
|
|
63
69
|
pixels in each direction. Larger patches follow larger structures but are much slower;
|
|
64
70
|
the default of 15 compares 31x31 patches, the examples use 3.
|
|
65
71
|
|
|
66
|
-
`patchmatch.set_random_seed(seed)` sets the seed of the randomized search
|
|
67
|
-
`patchmatch.set_verbose(True)` prints the progress of the native code to
|
|
68
|
-
`patchmatch.inpaint_regularity(image, mask, ijmap)` additionally guides the
|
|
69
|
-
a regularity map, an HxWx3 float32 array with the regularity coordinates of
|
|
70
|
-
in its first two channels. Its `guide_weight` must be
|
|
72
|
+
`patchmatch.set_random_seed(seed)` sets the seed of the randomized search, modulo
|
|
73
|
+
`2**32`, and `patchmatch.set_verbose(True)` prints the progress of the native code to
|
|
74
|
+
stderr. `patchmatch.inpaint_regularity(image, mask, ijmap)` additionally guides the
|
|
75
|
+
search with a regularity map, an HxWx3 float32 array with the regularity coordinates of
|
|
76
|
+
each pixel in its first two channels. Its `guide_weight` must be between 0 and the
|
|
77
|
+
largest float32, about 3.4e38.
|
|
71
78
|
|
|
72
79
|
C++ (see
|
|
73
80
|
[examples/cpp_example.cpp](https://github.com/mauwii/PyPatchMatch/blob/main/examples/cpp_example.cpp),
|
|
@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
|
|
|
18
18
|
commit_id: str | None
|
|
19
19
|
__commit_id__: str | None
|
|
20
20
|
|
|
21
|
-
__version__ = version = '2.0.
|
|
22
|
-
__version_tuple__ = version_tuple = (2, 0,
|
|
21
|
+
__version__ = version = '2.0.2'
|
|
22
|
+
__version_tuple__ = version_tuple = (2, 0, 2)
|
|
23
23
|
|
|
24
|
-
__commit_id__ = commit_id = '
|
|
24
|
+
__commit_id__ = commit_id = 'g38b611eb9'
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
#include "masked_image.h"
|
|
2
2
|
|
|
3
3
|
#include <algorithm>
|
|
4
|
+
#include <cstdint>
|
|
4
5
|
|
|
5
6
|
const cv::Size MaskedImage::kDownsampleKernelSize = cv::Size(6, 6);
|
|
6
7
|
const std::array<int, 6> MaskedImage::kDownsampleKernel = {1, 5, 10, 10, 5, 1};
|
|
@@ -107,8 +108,9 @@ MaskedImage MaskedImage::upsample(int new_w, int new_h) const
|
|
|
107
108
|
{
|
|
108
109
|
for (int x = 0; x < new_w; ++x)
|
|
109
110
|
{
|
|
110
|
-
|
|
111
|
-
const
|
|
111
|
+
// In 64 bits: the products overflow an int from 65,537 pixels on.
|
|
112
|
+
const auto yy = static_cast<int>(std::int64_t{y} * size.height / new_h);
|
|
113
|
+
const auto xx = static_cast<int>(std::int64_t{x} * size.width / new_w);
|
|
112
114
|
|
|
113
115
|
if (is_globally_masked(yy, xx))
|
|
114
116
|
{
|
|
@@ -296,12 +296,17 @@ int RegularityGuidedPatchDistanceMetricV2::operator()(
|
|
|
296
296
|
if (target_y < 0 || target_y >= target.size().height || target_x < 0 || target_x >= target.size().width)
|
|
297
297
|
return PatchDistanceMetric::kDistanceScale;
|
|
298
298
|
|
|
299
|
-
// Map pyramid-level coordinates to the full-resolution ijmap
|
|
300
|
-
//
|
|
299
|
+
// Map pyramid-level coordinates to the full-resolution ijmap, height and width
|
|
300
|
+
// independently; a coordinate inside the image stays inside the map. In 64 bits:
|
|
301
|
+
// the products overflow an int from 46,342 pixels on.
|
|
301
302
|
const int map_h = m_ijmap.size().height;
|
|
302
303
|
const int map_w = m_ijmap.size().width;
|
|
303
|
-
auto map_y = [&](const MaskedImage &img, int y) {
|
|
304
|
-
|
|
304
|
+
auto map_y = [&](const MaskedImage &img, int y) {
|
|
305
|
+
return static_cast<int>(std::int64_t{y} * map_h / img.size().height);
|
|
306
|
+
};
|
|
307
|
+
auto map_x = [&](const MaskedImage &img, int x) {
|
|
308
|
+
return static_cast<int>(std::int64_t{x} * map_w / img.size().width);
|
|
309
|
+
};
|
|
305
310
|
|
|
306
311
|
double score1 = PatchDistanceMetric::kDistanceScale;
|
|
307
312
|
if (!source.is_globally_masked(source_y, source_x) && !target.is_globally_masked(target_y, target_x))
|
|
@@ -17,7 +17,6 @@ from __future__ import annotations
|
|
|
17
17
|
|
|
18
18
|
import ctypes
|
|
19
19
|
import logging
|
|
20
|
-
import math
|
|
21
20
|
import operator
|
|
22
21
|
from collections.abc import Callable
|
|
23
22
|
from typing import SupportsIndex, TypeAlias
|
|
@@ -44,7 +43,9 @@ ImageLike: TypeAlias = np.ndarray | Image.Image
|
|
|
44
43
|
|
|
45
44
|
try:
|
|
46
45
|
_lib: ctypes.CDLL | None = load_library()
|
|
47
|
-
except OSError as e:
|
|
46
|
+
except (OSError, AttributeError) as e:
|
|
47
|
+
# AttributeError: a library without the PM_* functions, e.g. one that 1.x
|
|
48
|
+
# compiled into the package directory and that pip does not remove on upgrade
|
|
48
49
|
_lib = None
|
|
49
50
|
logger.warning("patchmatch failed to load: %s", e)
|
|
50
51
|
|
|
@@ -54,6 +55,9 @@ patchmatch_available = _lib is not None
|
|
|
54
55
|
# The native code computes 2 * patch_size + 1 as a C int.
|
|
55
56
|
_MAX_PATCH_SIZE = (2**31 - 2) // 2
|
|
56
57
|
|
|
58
|
+
# The native code takes a C float, which turns larger weights into infinity.
|
|
59
|
+
_MAX_GUIDE_WEIGHT = float(np.finfo(np.float32).max)
|
|
60
|
+
|
|
57
61
|
|
|
58
62
|
def _get_lib() -> ctypes.CDLL:
|
|
59
63
|
if _lib is None:
|
|
@@ -62,10 +66,17 @@ def _get_lib() -> ctypes.CDLL:
|
|
|
62
66
|
|
|
63
67
|
|
|
64
68
|
def set_random_seed(seed: int) -> None:
|
|
69
|
+
"""Set the seed of the randomized search of all following inpaintings.
|
|
70
|
+
|
|
71
|
+
Every inpainting starts from this seed, so equal inputs give equal results. The
|
|
72
|
+
native code keeps it as a 32-bit unsigned int, so ``seed`` is taken modulo
|
|
73
|
+
``2**32``: ``2**32`` behaves like 0 and -1 like ``2**32 - 1``.
|
|
74
|
+
"""
|
|
65
75
|
_get_lib().PM_set_random_seed(ctypes.c_uint(seed))
|
|
66
76
|
|
|
67
77
|
|
|
68
78
|
def set_verbose(verbose: bool) -> None:
|
|
79
|
+
"""Print the progress of all following inpaintings to stderr if ``verbose``."""
|
|
69
80
|
_get_lib().PM_set_verbose(ctypes.c_int(verbose))
|
|
70
81
|
|
|
71
82
|
|
|
@@ -96,8 +107,7 @@ def inpaint(
|
|
|
96
107
|
lib = _get_lib()
|
|
97
108
|
patch_size = _check_patch_size(patch_size)
|
|
98
109
|
image, mask, global_mask = _prepare_inputs(image, mask, global_mask)
|
|
99
|
-
|
|
100
|
-
if image.size == 0:
|
|
110
|
+
if not _has_holes(mask, global_mask):
|
|
101
111
|
return image.copy()
|
|
102
112
|
|
|
103
113
|
if global_mask is None:
|
|
@@ -140,15 +150,16 @@ def inpaint_regularity(
|
|
|
140
150
|
raise ValueError("ijmap must only contain finite values")
|
|
141
151
|
ijmap = np.ascontiguousarray(ijmap)
|
|
142
152
|
|
|
143
|
-
# the native code divides by 1 + guide_weight and indexes a table with the result
|
|
153
|
+
# the native code divides by 1 + guide_weight and indexes a table with the result;
|
|
154
|
+
# the comparison also rejects NaN
|
|
144
155
|
guide_weight = float(guide_weight)
|
|
145
|
-
if not
|
|
156
|
+
if not 0 <= guide_weight <= _MAX_GUIDE_WEIGHT:
|
|
146
157
|
raise ValueError(
|
|
147
|
-
f"guide_weight must be
|
|
158
|
+
f"guide_weight must be between 0 and {_MAX_GUIDE_WEIGHT:g}, "
|
|
159
|
+
f"got {guide_weight}"
|
|
148
160
|
)
|
|
149
161
|
|
|
150
|
-
|
|
151
|
-
if image.size == 0:
|
|
162
|
+
if not _has_holes(mask, global_mask):
|
|
152
163
|
return image.copy()
|
|
153
164
|
|
|
154
165
|
args = (ijmap, ctypes.c_int(patch_size), ctypes.c_float(guide_weight))
|
|
@@ -194,6 +205,19 @@ def _prepare_inputs(
|
|
|
194
205
|
return image, mask, global_mask
|
|
195
206
|
|
|
196
207
|
|
|
208
|
+
def _has_holes(mask: np.ndarray, global_mask: np.ndarray | None) -> bool:
|
|
209
|
+
"""Whether a hole lies outside the global mask.
|
|
210
|
+
|
|
211
|
+
Otherwise the result equals the input, but the native code still runs every
|
|
212
|
+
level of the pyramid. This also covers empty images, which OpenCV 4.6 would
|
|
213
|
+
return with the shape (0, 0, 1).
|
|
214
|
+
"""
|
|
215
|
+
holes = mask != 0
|
|
216
|
+
if global_mask is not None:
|
|
217
|
+
holes &= global_mask == 0
|
|
218
|
+
return bool(holes.any())
|
|
219
|
+
|
|
220
|
+
|
|
197
221
|
def _call(func: Callable[..., CMatT], *args: object) -> np.ndarray:
|
|
198
222
|
"""Call ``func`` with arrays converted to pymats and copy the result."""
|
|
199
223
|
lib = _get_lib()
|
|
@@ -1,11 +1,10 @@
|
|
|
1
1
|
[build-system]
|
|
2
2
|
build-backend = "scikit_build_core.build"
|
|
3
|
-
requires = ["scikit-build-core>=0.11", "setuptools-scm>=8"]
|
|
3
|
+
requires = ["scikit-build-core>=0.11.2", "setuptools-scm>=8"]
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "PyPatchMatch"
|
|
7
7
|
description = "This library implements the PatchMatch based inpainting algorithm."
|
|
8
|
-
readme = "README.md"
|
|
9
8
|
license = "MIT"
|
|
10
9
|
license-files = ["LICENSE"]
|
|
11
10
|
authors = [
|
|
@@ -34,7 +33,7 @@ classifiers = [
|
|
|
34
33
|
"Typing :: Typed",
|
|
35
34
|
]
|
|
36
35
|
dependencies = ["numpy>=1.23.2", "pillow>=9.2.0"]
|
|
37
|
-
dynamic = ["version"]
|
|
36
|
+
dynamic = ["version", "readme"]
|
|
38
37
|
|
|
39
38
|
[project.urls]
|
|
40
39
|
"Source Code" = "https://github.com/mauwii/PyPatchMatch"
|
|
@@ -56,6 +55,7 @@ wheel.py-api = "py3"
|
|
|
56
55
|
wheel.packages = ["patchmatch"]
|
|
57
56
|
wheel.exclude = ["patchmatch/csrc/**"]
|
|
58
57
|
metadata.version.provider = "scikit_build_core.metadata.setuptools_scm"
|
|
58
|
+
metadata.readme.provider = "scikit_build_core.metadata.fancy_pypi_readme"
|
|
59
59
|
# The release wheels are built from the sdist, so it has to contain the tests
|
|
60
60
|
# and their input image; _version.py is generated and therefore git-ignored.
|
|
61
61
|
sdist.include = ["examples/images/forest_pruned.bmp", "patchmatch/_version.py"]
|
|
@@ -70,6 +70,17 @@ if.from-sdist = true
|
|
|
70
70
|
if.failed = true
|
|
71
71
|
wheel.cmake = false
|
|
72
72
|
|
|
73
|
+
[tool.hatch.metadata.hooks.fancy-pypi-readme]
|
|
74
|
+
content-type = "text/markdown"
|
|
75
|
+
|
|
76
|
+
[[tool.hatch.metadata.hooks.fancy-pypi-readme.fragments]]
|
|
77
|
+
path = "README.md"
|
|
78
|
+
|
|
79
|
+
# PyPI does not resolve relative image paths
|
|
80
|
+
[[tool.hatch.metadata.hooks.fancy-pypi-readme.substitutions]]
|
|
81
|
+
pattern = '!\[([^\]]*)\]\((?!https?:)([^)]+)\)'
|
|
82
|
+
replacement = ''
|
|
83
|
+
|
|
73
84
|
[tool.setuptools_scm]
|
|
74
85
|
version_file = "patchmatch/_version.py"
|
|
75
86
|
# used when building without git metadata, e.g. from a GitHub source archive
|
|
@@ -5,8 +5,8 @@ wheels small and free of OpenCV's GUI/codec dependencies.
|
|
|
5
5
|
|
|
6
6
|
Usage: python build_opencv.py (installs into $OpenCV_ROOT)
|
|
7
7
|
|
|
8
|
-
PATCHMATCH_OPENCV_VERSION selects another pinned version,
|
|
9
|
-
|
|
8
|
+
PATCHMATCH_OPENCV_VERSION selects another pinned version, e.g. 4.14.0 to rebuild the
|
|
9
|
+
wheels of PyPatchMatch 2.0.0.
|
|
10
10
|
"""
|
|
11
11
|
|
|
12
12
|
import hashlib
|
|
@@ -27,7 +27,7 @@ OPENCV_SHA256S = {
|
|
|
27
27
|
"5.0.0": "b0528f5a1d379d59d4701cb28c36e22214cc51cf64594e5b56f2d3e6c0233095",
|
|
28
28
|
}
|
|
29
29
|
# the version of the release wheels
|
|
30
|
-
OPENCV_VERSION = os.environ.get("PATCHMATCH_OPENCV_VERSION") or "
|
|
30
|
+
OPENCV_VERSION = os.environ.get("PATCHMATCH_OPENCV_VERSION") or "5.0.0"
|
|
31
31
|
if OPENCV_VERSION not in OPENCV_SHA256S:
|
|
32
32
|
sys.exit(f"OpenCV {OPENCV_VERSION} is not pinned, known: {sorted(OPENCV_SHA256S)}")
|
|
33
33
|
OPENCV_SHA256 = OPENCV_SHA256S[OPENCV_VERSION]
|
|
@@ -73,6 +73,8 @@ CMAKE_OPTIONS = {
|
|
|
73
73
|
"WITH_QT": "OFF",
|
|
74
74
|
"WITH_TIFF": "OFF",
|
|
75
75
|
"WITH_V4L": "OFF",
|
|
76
|
+
# an installed VTK adds OpenGL to the link interface of core (OpenCV 5)
|
|
77
|
+
"WITH_VTK": "OFF",
|
|
76
78
|
"WITH_WEBP": "OFF",
|
|
77
79
|
"WITH_IPP": "OFF",
|
|
78
80
|
"WITH_ITT": "OFF",
|
|
@@ -90,6 +92,8 @@ WINDOWS_CMAKE_OPTIONS = {
|
|
|
90
92
|
"OPENCV_CONFIG_INSTALL_PATH": "cmake",
|
|
91
93
|
"OPENCV_INSTALL_BINARIES_PREFIX": "",
|
|
92
94
|
"OPENCV_SKIP_CMAKE_ROOT_CONFIG": "ON",
|
|
95
|
+
# skips OpenCV 5's assembler check: MinGW's would set MINGW and link pthread
|
|
96
|
+
"CMAKE_ASM_COMPILER": "CMAKE_ASM_COMPILER-NOTFOUND",
|
|
93
97
|
}
|
|
94
98
|
|
|
95
99
|
|
|
@@ -159,8 +163,10 @@ def main() -> None:
|
|
|
159
163
|
return
|
|
160
164
|
|
|
161
165
|
cmake = find_cmake()
|
|
162
|
-
with tempfile.TemporaryDirectory() as
|
|
163
|
-
|
|
166
|
+
with tempfile.TemporaryDirectory() as tmp_name:
|
|
167
|
+
# OpenCV 5 compares paths as strings, which fails with 8.3 names (RUNNER~1)
|
|
168
|
+
tmp = Path(tmp_name).resolve()
|
|
169
|
+
archive = tmp / "opencv.tar.gz"
|
|
164
170
|
print(f"Downloading {OPENCV_URL}", flush=True)
|
|
165
171
|
urllib.request.urlretrieve(OPENCV_URL, archive)
|
|
166
172
|
verify_archive(archive)
|
|
@@ -171,8 +177,8 @@ def main() -> None:
|
|
|
171
177
|
else:
|
|
172
178
|
tar.extractall(tmp)
|
|
173
179
|
|
|
174
|
-
source =
|
|
175
|
-
build =
|
|
180
|
+
source = tmp / f"opencv-{OPENCV_VERSION}"
|
|
181
|
+
build = tmp / "build"
|
|
176
182
|
options = {**CMAKE_OPTIONS, "CMAKE_INSTALL_PREFIX": prefix.as_posix()}
|
|
177
183
|
if sys.platform == "win32":
|
|
178
184
|
options.update(WINDOWS_CMAKE_OPTIONS)
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
"""Render examples/images/readme_example.jpg, the picture at the top of the README.
|
|
2
|
+
|
|
3
|
+
It runs examples/py_example.py and shows, from left to right, the photo forest.bmp,
|
|
4
|
+
the input forest_pruned.bmp with the birdhouses painted out in white, and the result.
|
|
5
|
+
The PyPI description links the picture at the release tag, so rerun the script and
|
|
6
|
+
commit the picture after changes that affect the fills.
|
|
7
|
+
|
|
8
|
+
Usage:
|
|
9
|
+
uv run python scripts/render_readme_example.py
|
|
10
|
+
"""
|
|
11
|
+
|
|
12
|
+
from __future__ import annotations
|
|
13
|
+
|
|
14
|
+
import runpy
|
|
15
|
+
from pathlib import Path
|
|
16
|
+
|
|
17
|
+
import numpy as np
|
|
18
|
+
from PIL import Image
|
|
19
|
+
|
|
20
|
+
import patchmatch
|
|
21
|
+
|
|
22
|
+
EXAMPLES = Path(__file__).resolve().parents[1] / "examples"
|
|
23
|
+
IMAGES = EXAMPLES / "images"
|
|
24
|
+
|
|
25
|
+
if __name__ == "__main__":
|
|
26
|
+
patchmatch.set_random_seed(0)
|
|
27
|
+
runpy.run_path(str(EXAMPLES / "py_example.py"), run_name="__main__")
|
|
28
|
+
panels = [
|
|
29
|
+
np.asarray(Image.open(IMAGES / f"forest{suffix}.bmp"))
|
|
30
|
+
for suffix in ("", "_pruned", "_recovered")
|
|
31
|
+
]
|
|
32
|
+
gap = np.full((panels[0].shape[0], 8, 3), 255, dtype=np.uint8)
|
|
33
|
+
picture = np.hstack([panels[0], gap, panels[1], gap, panels[2]])
|
|
34
|
+
Image.fromarray(picture).save(
|
|
35
|
+
IMAGES / "readme_example.jpg", quality=85, optimize=True
|
|
36
|
+
)
|
|
@@ -14,6 +14,7 @@ from patchmatch import _lib, patch_match
|
|
|
14
14
|
|
|
15
15
|
HEIGHT, WIDTH = 48, 64
|
|
16
16
|
HOLE = (slice(20, 28), slice(24, 36))
|
|
17
|
+
AROUND_HOLE = (slice(14, 34), slice(18, 42))
|
|
17
18
|
|
|
18
19
|
|
|
19
20
|
@pytest.fixture
|
|
@@ -27,6 +28,21 @@ def image() -> np.ndarray:
|
|
|
27
28
|
return img
|
|
28
29
|
|
|
29
30
|
|
|
31
|
+
@pytest.fixture
|
|
32
|
+
def regular_image() -> np.ndarray:
|
|
33
|
+
"""Noisy pattern that repeats every 8 pixels, as ijmap describes, with the hole.
|
|
34
|
+
|
|
35
|
+
The regularity term copies from whole periods away, which in the gradient of
|
|
36
|
+
``image`` shifted the fill by up to 35 color values.
|
|
37
|
+
"""
|
|
38
|
+
rng = np.random.default_rng(0)
|
|
39
|
+
tile = rng.integers(80, 120, (8, 8, 3))
|
|
40
|
+
noise = rng.integers(0, 8, (HEIGHT, WIDTH, 3))
|
|
41
|
+
img = (np.tile(tile, (HEIGHT // 8, WIDTH // 8, 1)) + noise).astype(np.uint8)
|
|
42
|
+
img[HOLE] = 255
|
|
43
|
+
return img
|
|
44
|
+
|
|
45
|
+
|
|
30
46
|
@pytest.fixture
|
|
31
47
|
def hole_mask() -> np.ndarray:
|
|
32
48
|
mask = np.zeros((HEIGHT, WIDTH), dtype=np.uint8)
|
|
@@ -48,14 +64,21 @@ def seed():
|
|
|
48
64
|
|
|
49
65
|
|
|
50
66
|
def assert_filled(result: np.ndarray, source: np.ndarray) -> None:
|
|
51
|
-
"""The hole is filled and all other pixels keep their values."""
|
|
67
|
+
"""The hole is filled to blend in, and all other pixels keep their values."""
|
|
52
68
|
assert result.shape == source.shape
|
|
53
69
|
assert result.dtype == np.uint8
|
|
54
|
-
assert not (result[HOLE] == 255).all(axis=-1).any()
|
|
55
70
|
known = np.ones(source.shape[:2], dtype=bool)
|
|
56
71
|
known[HOLE] = False
|
|
57
72
|
np.testing.assert_array_equal(result[known], source[known])
|
|
58
73
|
|
|
74
|
+
# a hole that is not filled stays black, see _initialize_pyramid
|
|
75
|
+
ring = np.zeros_like(known)
|
|
76
|
+
ring[AROUND_HOLE] = True
|
|
77
|
+
ring[HOLE] = False
|
|
78
|
+
fill = result[HOLE].reshape(-1, 3).mean(axis=0)
|
|
79
|
+
color_diff = np.abs(fill - source[ring].mean(axis=0))
|
|
80
|
+
assert color_diff.max() < 10, color_diff
|
|
81
|
+
|
|
59
82
|
|
|
60
83
|
# --- package ----------------------------------------------------------------
|
|
61
84
|
|
|
@@ -83,11 +106,7 @@ def test_inpaint_default_mask_fills_white_pixels(image):
|
|
|
83
106
|
|
|
84
107
|
def test_inpaint_explicit_mask(image, hole_mask):
|
|
85
108
|
image[HOLE] = 0 # not white: only the explicit mask marks the hole
|
|
86
|
-
|
|
87
|
-
assert result.shape == image.shape
|
|
88
|
-
assert (result[HOLE] != 0).any()
|
|
89
|
-
known = hole_mask == 0
|
|
90
|
-
np.testing.assert_array_equal(result[known], image[known])
|
|
109
|
+
assert_filled(patchmatch.inpaint(image, hole_mask, patch_size=3), image)
|
|
91
110
|
|
|
92
111
|
|
|
93
112
|
def test_inpaint_global_mask(image, hole_mask):
|
|
@@ -127,6 +146,33 @@ def test_inpaint_empty_image(shape, ijmap):
|
|
|
127
146
|
assert result is not empty
|
|
128
147
|
|
|
129
148
|
|
|
149
|
+
@pytest.mark.parametrize("covered", [False, True], ids=["no-hole", "global-mask"])
|
|
150
|
+
@pytest.mark.parametrize(
|
|
151
|
+
"func",
|
|
152
|
+
[patchmatch.inpaint, patchmatch.inpaint_regularity],
|
|
153
|
+
ids=lambda f: f.__name__,
|
|
154
|
+
)
|
|
155
|
+
def test_input_without_holes_skips_the_native_code(
|
|
156
|
+
monkeypatch, image, hole_mask, ijmap, func, covered
|
|
157
|
+
):
|
|
158
|
+
"""The result equals the input, but the native code ran every level.
|
|
159
|
+
|
|
160
|
+
That took 14 s for a 512 x 512 image with the default patch size.
|
|
161
|
+
"""
|
|
162
|
+
|
|
163
|
+
def native_call(*args):
|
|
164
|
+
pytest.fail("the native code was called")
|
|
165
|
+
|
|
166
|
+
monkeypatch.setattr(patch_match, "_call", native_call)
|
|
167
|
+
mask = hole_mask if covered else np.zeros_like(hole_mask)
|
|
168
|
+
kwargs = {"ijmap": ijmap} if func is patchmatch.inpaint_regularity else {}
|
|
169
|
+
|
|
170
|
+
result = func(image, mask, global_mask=hole_mask if covered else None, **kwargs)
|
|
171
|
+
|
|
172
|
+
np.testing.assert_array_equal(result, image)
|
|
173
|
+
assert not np.shares_memory(result, image)
|
|
174
|
+
|
|
175
|
+
|
|
130
176
|
@pytest.mark.parametrize("patch_size", [1, 3, 7])
|
|
131
177
|
def test_inpaint_patch_sizes(image, patch_size):
|
|
132
178
|
assert_filled(patchmatch.inpaint(image, patch_size=patch_size), image)
|
|
@@ -154,6 +200,30 @@ def test_inpaint_thin_hole_takes_the_colors_beside_it():
|
|
|
154
200
|
assert np.abs(fill - plain).max() < 10, (seed, fill)
|
|
155
201
|
|
|
156
202
|
|
|
203
|
+
@pytest.mark.parametrize("regularity", [False, True], ids=["inpaint", "regularity"])
|
|
204
|
+
@pytest.mark.parametrize("tall", [True, False], ids=["tall", "wide"])
|
|
205
|
+
def test_inpaint_very_large_image(tall, regularity):
|
|
206
|
+
"""Coordinate products overflowed a C int near the end of very large images.
|
|
207
|
+
|
|
208
|
+
The regularity metric crashed from 46,342 pixels on, and the upsampling filled
|
|
209
|
+
a hole near the bottom black from 65,537 pixels on.
|
|
210
|
+
"""
|
|
211
|
+
image = np.random.default_rng(0).integers(90, 110, (70_000, 4, 3), dtype=np.uint8)
|
|
212
|
+
image[-30:-20, 1:3] = 255
|
|
213
|
+
if not tall:
|
|
214
|
+
image = image.transpose(1, 0, 2)
|
|
215
|
+
holes = (image == 255).all(axis=2)
|
|
216
|
+
|
|
217
|
+
if regularity:
|
|
218
|
+
ijmap = np.zeros(image.shape, dtype=np.float32)
|
|
219
|
+
result = patchmatch.inpaint_regularity(image, None, ijmap, patch_size=1)
|
|
220
|
+
else:
|
|
221
|
+
result = patchmatch.inpaint(image, patch_size=1)
|
|
222
|
+
|
|
223
|
+
np.testing.assert_array_equal(result[~holes], image[~holes])
|
|
224
|
+
assert np.abs(result[holes].mean(axis=0) - 100).max() < 10
|
|
225
|
+
|
|
226
|
+
|
|
157
227
|
def test_inpaint_does_not_modify_inputs(image, hole_mask):
|
|
158
228
|
image_before, mask_before = image.copy(), hole_mask.copy()
|
|
159
229
|
patchmatch.inpaint(image, hole_mask, global_mask=hole_mask, patch_size=3)
|
|
@@ -205,19 +275,19 @@ def test_set_verbose_prints_progress(verbose):
|
|
|
205
275
|
@pytest.mark.parametrize("use_global_mask", [False, True], ids=["", "global"])
|
|
206
276
|
@pytest.mark.parametrize("guide_weight", [0.0, 0.25, 1.0])
|
|
207
277
|
def test_inpaint_regularity(
|
|
208
|
-
|
|
278
|
+
regular_image, hole_mask, ijmap, use_mask, use_global_mask, guide_weight
|
|
209
279
|
):
|
|
210
280
|
global_mask = np.zeros_like(hole_mask)
|
|
211
281
|
global_mask[:8] = 1
|
|
212
282
|
result = patchmatch.inpaint_regularity(
|
|
213
|
-
|
|
283
|
+
regular_image,
|
|
214
284
|
hole_mask if use_mask else None,
|
|
215
285
|
ijmap,
|
|
216
286
|
global_mask=global_mask if use_global_mask else None,
|
|
217
287
|
patch_size=3,
|
|
218
288
|
guide_weight=guide_weight,
|
|
219
289
|
)
|
|
220
|
-
assert_filled(result,
|
|
290
|
+
assert_filled(result, regular_image)
|
|
221
291
|
|
|
222
292
|
|
|
223
293
|
def test_inpaint_regularity_scales_smaller_ijmap(image):
|
|
@@ -347,10 +417,10 @@ def test_inpaint_regularity_invalid_ijmap(image, bad_ijmap):
|
|
|
347
417
|
patchmatch.inpaint_regularity(image, None, bad_ijmap)
|
|
348
418
|
|
|
349
419
|
|
|
350
|
-
@pytest.mark.parametrize("guide_weight", [-1.0, -0.5, float("nan"), float("inf")])
|
|
420
|
+
@pytest.mark.parametrize("guide_weight", [-1.0, -0.5, float("nan"), float("inf"), 1e39])
|
|
351
421
|
def test_inpaint_regularity_invalid_guide_weight(image, ijmap, guide_weight):
|
|
352
422
|
# a weight of -1 used to crash the native code, other negative weights made it
|
|
353
|
-
# read outside of the similarity table
|
|
423
|
+
# read outside of the similarity table, and 1e39 became infinity as a C float
|
|
354
424
|
with pytest.raises(ValueError, match="guide_weight"):
|
|
355
425
|
patchmatch.inpaint_regularity(image, None, ijmap, guide_weight=guide_weight)
|
|
356
426
|
|
|
@@ -447,6 +517,28 @@ def test_import_without_native_library():
|
|
|
447
517
|
assert "--no-cache-dir" in process.stderr
|
|
448
518
|
|
|
449
519
|
|
|
520
|
+
def test_import_with_incompatible_library():
|
|
521
|
+
"""A library without the PM_* functions must not break the import either.
|
|
522
|
+
|
|
523
|
+
1.x compiled the library into the package directory, and pip keeps it there.
|
|
524
|
+
Pillow's extension module stands in for it.
|
|
525
|
+
"""
|
|
526
|
+
code = (
|
|
527
|
+
"import importlib\n"
|
|
528
|
+
"from pathlib import Path\n"
|
|
529
|
+
"import PIL._imaging\n"
|
|
530
|
+
"from patchmatch import _lib, patch_match\n"
|
|
531
|
+
"_lib.find_library = lambda: Path(PIL._imaging.__file__)\n"
|
|
532
|
+
"importlib.reload(patch_match)\n"
|
|
533
|
+
"assert not patch_match.patchmatch_available\n"
|
|
534
|
+
)
|
|
535
|
+
process = subprocess.run(
|
|
536
|
+
[sys.executable, "-c", code], capture_output=True, text=True, check=True
|
|
537
|
+
)
|
|
538
|
+
assert "patchmatch failed to load" in process.stderr
|
|
539
|
+
assert "PM_" in process.stderr
|
|
540
|
+
|
|
541
|
+
|
|
450
542
|
# --- ctypes conversion ------------------------------------------------------
|
|
451
543
|
|
|
452
544
|
|
pypatchmatch-2.0.0rc6/CLAUDE.md
DELETED
|
@@ -1,479 +0,0 @@
|
|
|
1
|
-
# CLAUDE.md
|
|
2
|
-
|
|
3
|
-
This file provides guidance to Claude Code (claude.ai/code) when working with code in
|
|
4
|
-
this repository.
|
|
5
|
-
|
|
6
|
-
## Project
|
|
7
|
-
|
|
8
|
-
PyPatchMatch implements PatchMatch based image inpainting (Barnes et al., SIGGRAPH 2009):
|
|
9
|
-
a C++17 library on top of OpenCV core, and a thin Python package that loads it through
|
|
10
|
-
`ctypes`. It is published on PyPI as `PyPatchMatch` (import name `patchmatch`) and used
|
|
11
|
-
by InvokeAI. This repository (`mauwii/PyPatchMatch`) is the maintained fork of
|
|
12
|
-
`invoke-ai/PyPatchMatch`, which derives from `vacancy/PyPatchMatch` and
|
|
13
|
-
`younesse-cv/PatchMatch`.
|
|
14
|
-
|
|
15
|
-
## Commands
|
|
16
|
-
|
|
17
|
-
The project is managed with uv (`.python-version` pins 3.12 for the local venv). Building
|
|
18
|
-
needs a C++17 compiler, CMake >= 3.21 and an OpenCV that `find_package(OpenCV)` finds:
|
|
19
|
-
Homebrew `opencv`, `libopencv-dev`, or an installation prefix in `OpenCV_ROOT`.
|
|
20
|
-
|
|
21
|
-
```sh
|
|
22
|
-
uv sync # .venv, editable install, compiles the library
|
|
23
|
-
uv run pre-commit install
|
|
24
|
-
uv run pytest # all tests, with the coverage gate
|
|
25
|
-
uv run pytest -m "not e2e" # fast unit tests only
|
|
26
|
-
uv run pytest -m e2e --no-cov # end-to-end tests only
|
|
27
|
-
uv run pytest tests/test_patch_match.py::test_inpaint_global_mask --no-cov
|
|
28
|
-
uv run pytest -k regularity --no-cov
|
|
29
|
-
uv run mypy # strict, also a pre-commit hook
|
|
30
|
-
uv run pre-commit run --all-files # what the CI lint job runs
|
|
31
|
-
uv run ruff check --fix . && uv run ruff format .
|
|
32
|
-
uv build # sdist and wheel into dist/
|
|
33
|
-
```
|
|
34
|
-
|
|
35
|
-
- Pass `--no-cov` for any partial run: `addopts` always enables coverage, and
|
|
36
|
-
`fail_under = 95` (branch coverage, subprocesses included through
|
|
37
|
-
`patch = ["subprocess"]`) fails the run otherwise.
|
|
38
|
-
- Use `uv run pytest`, not `python -m pytest`. The latter puts the repository root on
|
|
39
|
-
`sys.path`, which imports the source tree `patchmatch/` without the compiled library;
|
|
40
|
-
the symptoms are the warning `patchmatch failed to load: libpatchmatch.* not found`
|
|
41
|
-
and `RuntimeError: the patchmatch native library is not available` in every test.
|
|
42
|
-
- `--cov=patchmatch` measures the source tree, so coverage only works with the editable
|
|
43
|
-
install that `uv sync` creates. With a non-editable install (`uv sync --no-editable`,
|
|
44
|
-
`uv pip install .`) it reports 0 % and the gate fails.
|
|
45
|
-
- `uv sync` rebuilds the library only when a `tool.uv.cache-keys` entry changes
|
|
46
|
-
(`CMakeLists.txt`, `patchmatch/csrc/*`, `pyproject.toml`, git commit or tags). Force it
|
|
47
|
-
with `uv sync --reinstall-package pypatchmatch`.
|
|
48
|
-
- CI builds the library with warnings as errors. Locally:
|
|
49
|
-
`SKBUILD_CMAKE_DEFINE=CMAKE_COMPILE_WARNING_AS_ERROR=ON uv sync --reinstall-package pypatchmatch`.
|
|
50
|
-
|
|
51
|
-
Build against the same static, core-only OpenCV as the release wheels:
|
|
52
|
-
|
|
53
|
-
```sh
|
|
54
|
-
export OpenCV_ROOT=/tmp/opencv
|
|
55
|
-
uv run --no-project python scripts/build_opencv.py # pinned version and SHA-256
|
|
56
|
-
uv sync --reinstall-package pypatchmatch
|
|
57
|
-
```
|
|
58
|
-
|
|
59
|
-
ASan and UBSan, like the CI job `sanitizers` (Linux):
|
|
60
|
-
|
|
61
|
-
```sh
|
|
62
|
-
SKBUILD_CMAKE_DEFINE=PATCHMATCH_SANITIZE=ON uv sync --reinstall-package pypatchmatch
|
|
63
|
-
ASAN="$(gcc -print-file-name=libasan.so) $(gcc -print-file-name=libstdc++.so)"
|
|
64
|
-
ASAN_OPTIONS=detect_leaks=0 uv run env LD_PRELOAD="$ASAN" pytest --no-cov --capture=sys
|
|
65
|
-
uv sync --reinstall-package pypatchmatch # back to the normal library
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
- Python is not instrumented, so the ASan runtime has to be loaded at startup; without
|
|
69
|
-
it every test aborts with `Interceptors are not working`. On Linux `libstdc++` has to
|
|
70
|
-
follow it: ASan looks up the real `__cxa_throw` at startup, and without it the first
|
|
71
|
-
C++ exception aborts with `CHECK failed: ... real___cxa_throw`. The leak check would
|
|
72
|
-
report the interpreter's own allocations.
|
|
73
|
-
- `--capture=sys` is required: pytest's default capture of the file descriptors keeps
|
|
74
|
-
the report of an abort, and only `Fatal Python error: Aborted` reaches the terminal.
|
|
75
|
-
- On macOS, inject the runtime with
|
|
76
|
-
`DYLD_INSERT_LIBRARIES="$(xcrun clang -print-resource-dir)/lib/darwin/libclang_rt.asan_osx_dynamic.dylib"`,
|
|
77
|
-
add `strip_env=0` to `ASAN_OPTIONS` (ASan removes the variable, and the tests that
|
|
78
|
-
start a subprocess abort), and start pytest as `.venv/bin/python .venv/bin/pytest`:
|
|
79
|
-
SIP drops the variable for `/usr/bin/env` and `/bin/sh`, which uv uses as the
|
|
80
|
-
shebang of scripts in long paths.
|
|
81
|
-
- The run takes about 80 s locally. Calling `PM_inpaint` through ctypes with a shape
|
|
82
|
-
larger than the buffer (ASan) or a `patch_size` of `2**31 - 1` (UBSan, `nnf.cpp`)
|
|
83
|
-
shows that both report and abort.
|
|
84
|
-
|
|
85
|
-
C++ coverage, like the CI job `coverage` (gcovr comes from the `coverage` group):
|
|
86
|
-
|
|
87
|
-
```sh
|
|
88
|
-
SKBUILD_CMAKE_DEFINE=PATCHMATCH_COVERAGE=ON uv sync --reinstall-package pypatchmatch
|
|
89
|
-
find build -name '*.gcda' -delete # the counts add up across runs
|
|
90
|
-
uv run --group coverage pytest --no-cov
|
|
91
|
-
uv run --group coverage gcovr --filter patchmatch/csrc/ --print-summary \
|
|
92
|
-
--exclude-throw-branches --exclude-unreachable-branches \
|
|
93
|
-
--gcov-suspicious-hits-threshold 0
|
|
94
|
-
uv sync --reinstall-package pypatchmatch # back to the normal library
|
|
95
|
-
```
|
|
96
|
-
|
|
97
|
-
- The Python tests are the C++ tests: there is no C++ test framework, the coverage of
|
|
98
|
-
the native code is measured while they run.
|
|
99
|
-
- The build uses `-Og` with atomic counters (a test inpaints in several threads at once);
|
|
100
|
-
the run takes about 90 s locally. `-O0` reports one more line (the masked branch of
|
|
101
|
-
`MaskedImage::upsample`, which `-Og` merges) but triples the time. The distance loop
|
|
102
|
-
runs more than 2**32 times, which gcovr rejects as a gcov bug without
|
|
103
|
-
`--gcov-suspicious-hits-threshold 0`.
|
|
104
|
-
- On macOS add `--gcov-executable "xcrun llvm-cov gcov"`. Clang's counts are not
|
|
105
|
-
reliable: they report a closing brace after a `return` as a missed line, and hundreds
|
|
106
|
-
of millions of returns through the clamp at the end of `distance_masked_images`,
|
|
107
|
-
which recomputing the distances showed to be unreachable. Judge by GCC (the CI job,
|
|
108
|
-
or Docker with `ubuntu:24.04`).
|
|
109
|
-
- Missed on purpose: the `catch (...)` blocks of `guarded()` and `set_last_error()`,
|
|
110
|
-
the `clone()` of a non-continuous result in `_cv2_to_py`, the clamps at the end of
|
|
111
|
-
`distance_masked_images` and for targets outside the image in
|
|
112
|
-
`RegularityGuidedPatchDistanceMetricV2`, and the masked branch of
|
|
113
|
-
`MaskedImage::upsample` (only called on targets, which have no holes). GCC also
|
|
114
|
-
reports the closing braces of the functions that return a `MaskedImage`, where it
|
|
115
|
-
puts the cleanup for exceptions.
|
|
116
|
-
- GCC 13 (Docker arm64, `-Og`): 97.0 % of the lines, 95.2 % of the branches. The gates
|
|
117
|
-
are these values rounded down to 5 % with at least two points of margin.
|
|
118
|
-
|
|
119
|
-
Fill quality (`scripts/evaluate_inpainting.py`, its docstring explains the columns):
|
|
120
|
-
|
|
121
|
-
```sh
|
|
122
|
-
uv run --no-sync python scripts/evaluate_inpainting.py --baseline <library of main>
|
|
123
|
-
uv run --no-sync python scripts/evaluate_inpainting.py --patch-size 15 --seeds 1 \
|
|
124
|
-
--cases forest,brick
|
|
125
|
-
```
|
|
126
|
-
|
|
127
|
-
- It cuts holes into `forest.bmp` and the CC0 images of `examples/images` (sources in
|
|
128
|
-
`SOURCES.md`) and adds `forest_pruned.bmp` and every `*_pruned.*` of the git-ignored
|
|
129
|
-
`examples/images/local/` as object removals without a ground truth. That folder is
|
|
130
|
-
for your own images; never commit them.
|
|
131
|
-
- For `--baseline`, build the library of `main` outside the checkout:
|
|
132
|
-
|
|
133
|
-
```sh
|
|
134
|
-
mkdir -p /tmp/main-lib
|
|
135
|
-
git archive main CMakeLists.txt patchmatch/csrc | tar -x -C /tmp/main-lib
|
|
136
|
-
cmake -S /tmp/main-lib -B /tmp/main-lib/build -DCMAKE_BUILD_TYPE=Release
|
|
137
|
-
cmake --build /tmp/main-lib/build # libpatchmatch.so, .dylib or patchmatch.dll
|
|
138
|
-
```
|
|
139
|
-
|
|
140
|
-
- Judge changes on all cases and look at the fills in `examples/images/evaluation.html`
|
|
141
|
-
(git-ignored, it embeds the images of `examples/images/local/` too): fixes for one
|
|
142
|
-
image regressed others before.
|
|
143
|
-
|
|
144
|
-
Other entry points:
|
|
145
|
-
|
|
146
|
-
- `examples/cpp_example_run.sh` builds the library and `examples/cpp_example.cpp` with
|
|
147
|
-
CMake into `build/cpp` and runs it. It needs OpenCV `imgcodecs`, so a full OpenCV, not
|
|
148
|
-
the core-only build of the script.
|
|
149
|
-
- `uv run python examples/py_example.py` (and `py_example_global_mask.py`) write
|
|
150
|
-
`examples/images/forest_recovered.bmp`, which is git-ignored.
|
|
151
|
-
- CMake options: `PATCHMATCH_BUILD_EXAMPLES` (OFF), `PATCHMATCH_FAST_MATH` (ON,
|
|
152
|
-
`-ffast-math` or `/fp:fast`), `PATCHMATCH_SANITIZE` (OFF, ASan and UBSan with
|
|
153
|
-
`float-cast-overflow`, every finding aborts; turns off fast math, which lets the
|
|
154
|
-
compiler drop checks; GCC and Clang only), `PATCHMATCH_COVERAGE` (OFF, gcov counters
|
|
155
|
-
at `-Og`; GCC and Clang only).
|
|
156
|
-
- The fallback wheel of the CI job `sdist-fallback`:
|
|
157
|
-
`uv build --wheel dist/<sdist>.tar.gz --out-dir fallback -C cmake.define.CMAKE_DISABLE_FIND_PACKAGE_OpenCV=ON`
|
|
158
|
-
must produce a `py3-none-any` wheel with the Python package but without the library.
|
|
159
|
-
|
|
160
|
-
## Architecture
|
|
161
|
-
|
|
162
|
-
### Layers, in the call order of `patchmatch.inpaint`
|
|
163
|
-
|
|
164
|
-
1. `patchmatch/__init__.py` re-exports the public API of `patch_match.py` and
|
|
165
|
-
`__version__` from the generated `_version.py`. `test_top_level_exports` checks that
|
|
166
|
-
every name in `patch_match.__all__` is re-exported.
|
|
167
|
-
2. `patchmatch/patch_match.py` is the public API. It validates and canonicalizes the
|
|
168
|
-
inputs (`_prepare_inputs`, `_check_patch_size`, the `ijmap` and `guide_weight`
|
|
169
|
-
checks), picks the entry point (`PM_inpaint`, `PM_inpaint2` with a global mask, and
|
|
170
|
-
their `_regularity` variants) and runs it through `_call`. It loads the library at
|
|
171
|
-
import time; on `OSError` it logs a warning and sets `patchmatch_available = False`,
|
|
172
|
-
and every function then raises `RuntimeError` through `_get_lib()`.
|
|
173
|
-
3. `patchmatch/_lib.py` is the ctypes layer: the `CShapeT`/`CMatT` structures, the dtype
|
|
174
|
-
table, `np_to_pymat`/`pymat_to_np`, `find_library` (looks for `libpatchmatch.so`,
|
|
175
|
-
`libpatchmatch.dylib` or `patchmatch.dll` in every directory of `patchmatch.__path__`)
|
|
176
|
-
and `load_library` (argtypes and restype of every `PM_*` function).
|
|
177
|
-
4. `patchmatch/csrc/pyinterface.{h,cpp}` is the `extern "C"` API: the `PM_*` functions,
|
|
178
|
-
the conversion between `PM_mat_t` and `cv::Mat`, the global settings and the error
|
|
179
|
-
reporting.
|
|
180
|
-
5. `csrc/inpaint.{h,cpp}`: `Inpainting`, the image pyramid and the EM loop.
|
|
181
|
-
6. `csrc/nnf.{h,cpp}`: `NearestNeighborField` (propagation and random search) and the
|
|
182
|
-
distance metrics `PatchSSDDistanceMetric` and `RegularityGuidedPatchDistanceMetricV2`.
|
|
183
|
-
7. `csrc/masked_image.{h,cpp}`: `MaskedImage`, an image with its hole mask and optional
|
|
184
|
-
global mask, down- and upsampling, gradients.
|
|
185
|
-
|
|
186
|
-
The C++ headers double as the C++ API: `examples/cpp_example.cpp` uses `Inpainting` and
|
|
187
|
-
`PatchSSDDistanceMetric` directly.
|
|
188
|
-
|
|
189
|
-
### ctypes instead of a CPython extension
|
|
190
|
-
|
|
191
|
-
The library does not use the CPython ABI, so the wheels are tagged
|
|
192
|
-
`py3-none-<platform>` (`wheel.py-api = "py3"`): one wheel per platform serves all Python
|
|
193
|
-
versions, and cibuildwheel only builds `cp311-*`. Keep `csrc` free of `Python.h`,
|
|
194
|
-
pybind11 or nanobind. The C++ sources are excluded from the wheel (`wheel.exclude`);
|
|
195
|
-
CMake installs the shared library into the package directory.
|
|
196
|
-
|
|
197
|
-
### The C interface contract
|
|
198
|
-
|
|
199
|
-
A change on one side has to be mirrored on the other:
|
|
200
|
-
|
|
201
|
-
- `PM_mat_t`/`PM_shape_t` in `pyinterface.h` and `CMatT`/`CShapeT` in `_lib.py` (field
|
|
202
|
-
order and types).
|
|
203
|
-
- The order of `PM_dtype_e` and of the `_PYMAT_DTYPES` list: uint8 and float32, the
|
|
204
|
-
only types the native code reads; `_py_to_cv2` rejects others.
|
|
205
|
-
- Every `PM_*` function needs its `argtypes` and `restype` in `load_library()`; without
|
|
206
|
-
them ctypes silently assumes `int`.
|
|
207
|
-
- Symbols are exported by `extern "C"` and `CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS`, there are
|
|
208
|
-
no export macros.
|
|
209
|
-
|
|
210
|
-
A new entry point is declared in `pyinterface.h`, implemented in `pyinterface.cpp`
|
|
211
|
-
inside `guarded(...)`, registered in `load_library()` and called through `_call()`.
|
|
212
|
-
|
|
213
|
-
Memory ownership: `np_to_pymat` wraps a C-contiguous array without copying it, so
|
|
214
|
-
callers convert with `np.ascontiguousarray` first and keep the array alive during the
|
|
215
|
-
call. `_py_to_cv2` clones it into a `cv::Mat`, so the inputs are never modified.
|
|
216
|
-
`_cv2_to_py` hands a heap buffer (a released `std::unique_ptr<unsigned char[]>`) to
|
|
217
|
-
Python; `pymat_to_np` copies it, and `_call` always returns it through `PM_free_pymat` in
|
|
218
|
-
a `finally`. It allocates at least one byte, because a null `data_ptr` signals failure.
|
|
219
|
-
|
|
220
|
-
Errors: a C++ exception that reaches ctypes aborts the process. `guarded()` catches
|
|
221
|
-
everything, stores the message in a thread-local string and returns a null `data_ptr`;
|
|
222
|
-
`_call` turns that into `RuntimeError("patchmatch failed: <PM_last_error()>")`. Most
|
|
223
|
-
input validation lives in Python because the native code crashes or hangs on bad input:
|
|
224
|
-
`patch_size` 0 crashes, negative sizes never terminate, sizes >= 2**31 wrap to negative
|
|
225
|
-
C ints, negative guide weights index outside the similarity table, NaN or infinity in the
|
|
226
|
-
`ijmap` produce out-of-range distances. `RegularityGuidedPatchDistanceMetricV2` also
|
|
227
|
-
throws on a negative weight, for C++ callers.
|
|
228
|
-
|
|
229
|
-
### Threading and determinism
|
|
230
|
-
|
|
231
|
-
ctypes releases the GIL during native calls, so inpaintings run truly concurrently
|
|
232
|
-
(`test_concurrent_inpaint_is_deterministic`). Therefore:
|
|
233
|
-
|
|
234
|
-
- The seed and the verbose flag are `std::atomic` globals, read when a call starts.
|
|
235
|
-
- The random generator is a function-local `thread_local std::mt19937`, seeded at the
|
|
236
|
-
start of every `Inpainting::run`, so each call is reproducible from the seed on any
|
|
237
|
-
thread.
|
|
238
|
-
- `random_int` uses `engine() % n` instead of `std::uniform_int_distribution` to keep
|
|
239
|
-
the sequence identical on all platforms and standard libraries. Keep it that way.
|
|
240
|
-
- Lookup tables are function-local statics (thread-safe initialization). Do not add
|
|
241
|
-
mutable shared state.
|
|
242
|
-
|
|
243
|
-
### Algorithm
|
|
244
|
-
|
|
245
|
-
- `patch_size` is a radius: patches span `(2 * patch_size + 1)` pixels in each
|
|
246
|
-
direction. The pyramid halves the image until one side is <= `patch_size`.
|
|
247
|
-
- From the coarsest level to the finest, two nearest-neighbor fields are kept:
|
|
248
|
-
source to target (completeness) and target to source (coherence). They start random
|
|
249
|
-
at the coarsest level and are upscaled from the previous level afterwards, keeping the
|
|
250
|
-
offset within the coarse pixel so that neighbors still point to neighbors.
|
|
251
|
-
- Each level runs `1 + 2 * level` EM iterations with `min(7, 1 + level)` NNF passes
|
|
252
|
-
(propagation and random search), an expectation step (votes of both fields, weighted
|
|
253
|
-
through the `distance2similarity` table) and a maximization step, which sets each
|
|
254
|
-
pixel to the weighted mean of its votes. On levels 0 and 1 the known pixels keep their
|
|
255
|
-
input values instead; without that the votes change the pixels around the holes, and a
|
|
256
|
-
seam appears when `run()` restores them. On coarser levels the known pixels next to
|
|
257
|
-
the holes are averages of partially masked kernels, and keeping them made fills copy
|
|
258
|
-
smooth regions into textured ones (57 % instead of 17 % of the fill in one case). The
|
|
259
|
-
last iteration of a level votes directly into the upsampled image of the next level,
|
|
260
|
-
which is less blurry than upsampling the result. The last iteration at level 0 keeps
|
|
261
|
-
only the heaviest vote of each pixel instead of the mean, which blurs.
|
|
262
|
-
- The coarse levels mark a pixel as a hole only if its whole downsampling kernel is
|
|
263
|
-
one, so the holes shrink quickly and are filled smoothly from their border; the finer
|
|
264
|
-
levels add the texture. Marking every pixel that touches a hole instead was tried and
|
|
265
|
-
made the fills worse: it leaves the coarsest levels without valid source patches,
|
|
266
|
-
because the similarity table gives zero weight to patches with more than about 10 %
|
|
267
|
-
masked or border pixels.
|
|
268
|
-
- A hole, and later each thin part of it, therefore has a level on which it appears for
|
|
269
|
-
the first time. The patches around it arrive there with the link of a patch without
|
|
270
|
-
holes, the one to itself, which now points into the hole. `_initialize_field_from`
|
|
271
|
-
runs a random search around such a patch before the first pass. Otherwise the first
|
|
272
|
-
propagation replaces the link with the one of a neighbor, the random search goes on
|
|
273
|
-
around that link, and with a radius of half the smaller image side it does not find
|
|
274
|
-
the way back to the surroundings of the patch: the stem of `forest_pruned.bmp` was
|
|
275
|
-
filled with forest instead of meadow in 31 of 200 seeds (36 of 48 with `patch_size`
|
|
276
|
-
7), and the `meadow` case had an error above 12 in 9 of 96 seeds, now at most 9.7
|
|
277
|
-
(`test_inpaint_thin_hole_takes_the_colors_beside_it`). Tried and rejected: searching
|
|
278
|
-
before propagating in every pass, which no longer refines a propagated link, the way
|
|
279
|
-
a structure is followed along a hole (brick error +9 %); and a search radius of the
|
|
280
|
-
larger image side, as in the paper, which blurs the fills (coffee detail 0.30 instead
|
|
281
|
-
of 0.53).
|
|
282
|
-
- Distances are ints in `[0, PatchDistanceMetric::kDistanceScale]` (65535): an SSD over
|
|
283
|
-
the colors and the x/y gradients, where masked pixels and pixels at the border count
|
|
284
|
-
as maximal. The distance indexes `distance2similarity`, so every metric has to clamp
|
|
285
|
-
its result to that range.
|
|
286
|
-
- Mask semantics: `mask` non-zero marks the holes (by default all pure white pixels,
|
|
287
|
-
computed in Python by `_default_mask`); `_initialize_pyramid` makes them black on
|
|
288
|
-
level 0 as on the coarser levels, so their colors do not matter
|
|
289
|
-
(`test_colors_under_the_holes_do_not_matter`); `global_mask` marks pixels that are
|
|
290
|
-
neither filled nor used as a patch source, and whose colors must not influence the
|
|
291
|
-
fill: the downsampling kernel skips them, and gradients next to them are neutral
|
|
292
|
-
(`test_colors_under_global_mask_do_not_matter`). At the end of `run()` every pixel
|
|
293
|
-
that is not a hole, including the globally masked ones, is copied back from the input,
|
|
294
|
-
so only the holes change. The tests assert this invariant (`assert_filled`,
|
|
295
|
-
`assert_plausible_fill`).
|
|
296
|
-
|
|
297
|
-
### OpenCV usage and library size
|
|
298
|
-
|
|
299
|
-
The release wheels link a static, core-only OpenCV (`scripts/build_opencv.py`:
|
|
300
|
-
`BUILD_LIST=core`, every optional backend off). For changes to the C++ code:
|
|
301
|
-
|
|
302
|
-
- The library links only `opencv_core`; `imgcodecs` is only used by the example.
|
|
303
|
-
- Use plain pixel loops instead of `cv::Mat` matrix expressions such as `a == b` or
|
|
304
|
-
`m1 | m2`: they pull OpenCV's expression module, and with it `FileStorage`, PCA and
|
|
305
|
-
`SparseMat` code, into the static library, which grew by about 40 % before commit
|
|
306
|
-
e50c613 replaced them.
|
|
307
|
-
- On Windows the static OpenCV uses the static MSVC runtime (`BUILD_WITH_STATIC_CRT`),
|
|
308
|
-
so the library does too (`MSVC_RUNTIME_LIBRARY` in `CMakeLists.txt`). Its stderr is
|
|
309
|
-
then not captured by pytest's `capfd`, so tests of native output run in a subprocess
|
|
310
|
-
(`test_set_verbose_prints_progress`).
|
|
311
|
-
- With a static OpenCV, CMake installs its licenses, which the build script collects in
|
|
312
|
-
`<prefix>/licenses`, into `.dist-info/licenses/opencv` of the wheel, and the CycloneDX
|
|
313
|
-
SBOM it writes to `<prefix>/sboms/opencv.cdx.json` into `.dist-info/sboms` (PEP 770).
|
|
314
|
-
The static library hides OpenCV from scanners; grype matches the SBOM through its CPE.
|
|
315
|
-
- The SBOM leaves out OpenCV's bundled zlib: only `FileStorage` uses it, and none of it
|
|
316
|
-
is linked into the library. The wheel job checks that every wheel has the SBOM and,
|
|
317
|
-
except on Windows, that `nm` finds no zlib symbol in the library. `write_sbom` fails
|
|
318
|
-
if the build bundles another 3rdparty library; add a new one to the SBOM.
|
|
319
|
-
- To update OpenCV, change the default of `OPENCV_VERSION` and add its hash to
|
|
320
|
-
`OPENCV_SHA256S`. `PATCHMATCH_OPENCV_VERSION` selects another pinned version: the
|
|
321
|
-
CI job `opencv5` builds against OpenCV 5 (Homebrew's `opencv` since 2026), because
|
|
322
|
-
builds from the sdist link whatever OpenCV the system has. The CI cache of the
|
|
323
|
-
OpenCV build is keyed on the hash of the script.
|
|
324
|
-
|
|
325
|
-
### Packaging and versioning
|
|
326
|
-
|
|
327
|
-
- Build backend: scikit-build-core with setuptools-scm. The version comes from the git
|
|
328
|
-
tags; `patchmatch/_version.py` is generated and git-ignored. Without git metadata the
|
|
329
|
-
version falls back to `0.0.0`, so CI checks out with `fetch-depth: 0`.
|
|
330
|
-
- The release wheels are built from the sdist, not from the checkout, and cibuildwheel
|
|
331
|
-
runs `pytest {project}/tests --no-cov` against the sdist contents. Everything the tests
|
|
332
|
-
need has to be in the sdist: `sdist.include` adds `examples/images/forest_pruned.bmp`,
|
|
333
|
-
the rest of `examples/` is excluded. New test data has to be added there as well.
|
|
334
|
-
- Fallback: if a build from the sdist fails (e.g. no compiler, CMake or OpenCV on a
|
|
335
|
-
platform without a wheel), the `tool.scikit-build.overrides` entry builds a pure wheel
|
|
336
|
-
without the library, so the installation succeeds with `patchmatch_available` False and
|
|
337
|
-
the error message of `find_library` explains how to reinstall. Builds from the source
|
|
338
|
-
tree still fail. The CI job `sdist-fallback` and `test_import_without_native_library`
|
|
339
|
-
keep this path working.
|
|
340
|
-
- Wheels: manylinux x86_64 and aarch64, macOS arm64 and x86_64 (deployment target 11.0),
|
|
341
|
-
Windows x64. musllinux is skipped.
|
|
342
|
-
|
|
343
|
-
## CI
|
|
344
|
-
|
|
345
|
-
- `ci.yml`: `lint` runs `pre-commit run --all-files`; `test` runs `uv run pytest` on
|
|
346
|
-
Ubuntu, macOS and Windows with Python 3.11, 3.14 and 3.15, after building (or
|
|
347
|
-
restoring) the OpenCV of the build script. It syncs only the `test` group: the `dev`
|
|
348
|
-
group pulls in pyyaml through pre-commit, which had no wheel for 3.15 yet, and
|
|
349
|
-
`--no-build` refuses to build it.
|
|
350
|
-
- pre-commit runs, besides the file checks, ruff and uv-lock: clang-format
|
|
351
|
-
(`.clang-format`), shellcheck, markdownlint-cli2 (`.markdownlint.yaml`, 88 columns),
|
|
352
|
-
typos (fixes in place), actionlint (checks `run:` scripts with shellcheck only if it
|
|
353
|
-
is on `PATH`) and zizmor (offline, default persona), and mypy as a local hook
|
|
354
|
-
(`uv run --locked --no-build --only-group dev mypy`, like the `lint` job, so the
|
|
355
|
-
library is not built; numpy and pillow are in the `dev` group for their types).
|
|
356
|
-
MegaLinter was considered and rejected: it duplicates pre-commit in a multi-GB Docker
|
|
357
|
-
image and bundles the linter versions instead of pinning each hook for Dependabot.
|
|
358
|
-
- Dependabot waits 7 days after a release (`cooldown`, required by zizmor).
|
|
359
|
-
- `test` sets `SKBUILD_CMAKE_DEFINE=CMAKE_COMPILE_WARNING_AS_ERROR=ON`, so the warning
|
|
360
|
-
flags of `CMakeLists.txt` (`-Wall -Wextra -Wpedantic -Wshadow`, `/W4` on MSVC) fail the
|
|
361
|
-
build. Only there: builds from the sdist and the wheels must not break when a newer
|
|
362
|
-
compiler adds a warning. The OpenCV headers are system includes and do not count.
|
|
363
|
-
- `sanitizers` builds with `PATCHMATCH_SANITIZE=ON` on Ubuntu and runs the tests with
|
|
364
|
-
the preloaded ASan runtime (see Commands). It reuses the checkout and OpenCV steps of
|
|
365
|
-
`test` through YAML anchors, so change them there.
|
|
366
|
-
Scheduled (nightly) and manual runs of `ci.yml` pass `--hypothesis-profile=deep`
|
|
367
|
-
(`tests/conftest.py`: 5000 examples per property test, about 90 s locally without
|
|
368
|
-
sanitizers) and allow the job 90 minutes; a failure prints a `@reproduce_failure`
|
|
369
|
-
blob. The property tests found real bugs, so they get the most search time.
|
|
370
|
-
- `coverage` builds with `PATCHMATCH_COVERAGE=ON` on Ubuntu, runs the tests and fails
|
|
371
|
-
below 95 % of the C++ lines or 90 % of the branches (gcovr, see Commands).
|
|
372
|
-
It reuses the same anchors. It also writes `coverage.xml` (Python) and
|
|
373
|
-
`coverage-cpp.xml` (`gcovr --sonarqube`) and exports `build/compile_commands.json`,
|
|
374
|
-
and then runs the SonarCloud scan (`sonar-project.properties`, `SONAR_TOKEN`, skipped
|
|
375
|
-
without the secret, e.g. for Dependabot and forks). Automatic analysis in SonarCloud
|
|
376
|
-
is disabled; it cannot import coverage and would conflict with the CI scan. `scripts/`
|
|
377
|
-
has no tests and is excluded from the coverage on new code (80 % in "Sonar way").
|
|
378
|
-
`sonar.qualitygate.wait=true` makes the scan fail on a red quality gate, so the gate
|
|
379
|
-
is part of `ci-ok` and auto-merge waits for it; the skipped scan of Dependabot and
|
|
380
|
-
fork PRs blocks nothing, unlike a required "SonarCloud Code Analysis" check, which
|
|
381
|
-
would never appear there.
|
|
382
|
-
- The `uv run` steps pass `--locked` and `--no-build` (SonarCloud S8544, S8541).
|
|
383
|
-
`--no-build` only forbids building dependencies from source; uv still builds the
|
|
384
|
-
project itself. The OpenCV step runs with `--only-group test`: the project cannot be
|
|
385
|
-
built before OpenCV is installed, `--no-project` would lock nothing, and the `dev`
|
|
386
|
-
group may lack wheels for the newest Python of the matrix.
|
|
387
|
-
- Every setup-uv step sets `version: latest-known`: the newest uv whose checksum is
|
|
388
|
-
bundled with the pinned setup-uv. Without it, CI installs each uv release minutes
|
|
389
|
-
after publication; this way uv only moves when Dependabot updates the action, with
|
|
390
|
-
its cooldown. That uv can be a few releases older than the one of the `uv-lock` hook.
|
|
391
|
-
`prune-cache: true` keeps only wheels built from source in the uv cache; downloaded
|
|
392
|
-
wheels restore no faster than they download.
|
|
393
|
-
- `system-opencv` installs `libopencv-dev` on `ubuntu-24.04` (OpenCV 4.6) instead of
|
|
394
|
-
running the build script and runs the tests, because builds from the sdist link
|
|
395
|
-
whatever OpenCV the system has. The runner is pinned so that the OpenCV version only
|
|
396
|
-
changes on purpose. The Ubuntu mirror is sometimes very slow (180 MB at under
|
|
397
|
-
300 kB/s), so the job caches the downloaded `.deb` files, keyed on the hash of
|
|
398
|
-
`apt-get install --print-uris`, with `restore-keys` for the files that did not change.
|
|
399
|
-
apt checks them against the signed index and dpkg installs them as usual. The job
|
|
400
|
-
downloads first (`--download-only`) and caches only the files of the current list.
|
|
401
|
-
- `wheels.yml`: sdist, then cibuildwheel on five runners, then `sdist-fallback`;
|
|
402
|
-
`publish` attests the build provenance of all files (`actions/attest`) and uploads
|
|
403
|
-
them to PyPI with trusted publishing (environment `pypi`, deployable only from tags
|
|
404
|
-
`v*`) only for a published GitHub release of `mauwii/PyPatchMatch`. It downloads the
|
|
405
|
-
artifacts named `dist-*`, so other artifacts of the run never reach PyPI. The wheel
|
|
406
|
-
jobs cache the OpenCV prefix, keyed on `build_opencv.py` and `pyproject.toml` from the
|
|
407
|
-
sdist (there is no checkout); on Linux a volume makes `/tmp/opencv` of the container
|
|
408
|
-
visible to the cache step. Runs on a tag, and therefore releases, skip the cache and
|
|
409
|
-
build OpenCV from scratch.
|
|
410
|
-
The build and test dependencies of the distributions are not locked, so the workflow
|
|
411
|
-
sets `UV_EXCLUDE_NEWER: 7 days` (the cooldown of `dependabot.yml`): `uv build` and
|
|
412
|
-
cibuildwheel's `build-frontend = "build[uv]"` resolve only releases older than a
|
|
413
|
-
week. On Linux `environment-pass` hands the variable into the container; on macOS
|
|
414
|
-
and Windows cibuildwheel uses the uv of a setup-uv step.
|
|
415
|
-
- `fill-quality.yml` runs for pull requests that change `CMakeLists.txt`,
|
|
416
|
-
`patchmatch/csrc` or the evaluation script: it builds the library of the base commit,
|
|
417
|
-
runs `scripts/evaluate_inpainting.py --baseline` against it (patch size 3, 3 seeds),
|
|
418
|
-
writes the table to the job summary (`--markdown`) and uploads `evaluation.html` as the
|
|
419
|
-
artifact `fill-quality`. It is a report, not a gate: the measures vary over the seeds,
|
|
420
|
-
so it is not in the `needs` of `ci-ok`. It repeats the OpenCV steps of `ci.yml` with
|
|
421
|
-
the same cache key, because anchors do not reach across files.
|
|
422
|
-
- `codeql.yml` analyzes actions, C/C++ and Python.
|
|
423
|
-
- `cache-cleanup.yml` deletes the caches of a pull request when it is closed: only that
|
|
424
|
-
pull request can read them, and at the 10 GB limit GitHub evicts the least recently
|
|
425
|
-
used caches, often those of `main`. For pull requests from forks the token is
|
|
426
|
-
read-only; their caches expire after 7 days without access.
|
|
427
|
-
- `ci-ok` and `wheels-ok` (re-actors/alls-green) are the required status checks; add new
|
|
428
|
-
jobs to their `needs`. The ruleset of `main` also has a `code_scanning` rule: CodeQL
|
|
429
|
-
alerts of severity error or security severity high and above block the merge (CodeQL
|
|
430
|
-
uploads its results for Dependabot PRs too, so they are not stuck).
|
|
431
|
-
- Actions are pinned to full commit SHAs with a `# vX.Y.Z` comment, and checkouts use
|
|
432
|
-
`persist-credentials: false`. The pre-commit hooks are pinned the same way
|
|
433
|
-
(`rev: <sha> # frozen: vX.Y.Z`; `pre-commit autoupdate --freeze` by hand), because a
|
|
434
|
-
moved tag would otherwise reach CI and fresh clones without review. Dependabot
|
|
435
|
-
updates actions, uv and pre-commit weekly in groups, SHA and comment together. The
|
|
436
|
-
repository is a GitHub fork, on which Dependabot version updates are off by default;
|
|
437
|
-
they are enabled in Insights → Dependency graph → Dependabot.
|
|
438
|
-
|
|
439
|
-
## Conventions
|
|
440
|
-
|
|
441
|
-
- Python >= 3.11: keep `from __future__ import annotations`. Ruff targets py311 with the
|
|
442
|
-
rules B, C4, E, F, I, RUF, SIM, UP and W, and formats at 88 columns. `patchmatch/csrc`
|
|
443
|
-
is excluded from ruff. C++ is formatted by clang-format (Microsoft style, 120 columns,
|
|
444
|
-
includes grouped as own header, `<...>`, `"..."`); naming is not checked: `m_`
|
|
445
|
-
members, `k` constants. Commit a reformat separately and add its hash to
|
|
446
|
-
`.git-blame-ignore-revs`.
|
|
447
|
-
- The package ships `py.typed`; public functions are fully annotated. mypy checks
|
|
448
|
-
`patchmatch` and `tests` in strict mode (`[tool.mypy]`); the tests have to use the
|
|
449
|
-
API correctly but need no annotations of their own. A deliberately wrong argument
|
|
450
|
-
goes through a test parameter instead of a `# type: ignore`.
|
|
451
|
-
- Backwards compatibility: `from patchmatch import patch_match` and
|
|
452
|
-
`patch_match.CShapeT`/`CMatT` keep working (tested).
|
|
453
|
-
- Comments explain why: which crash a check prevents, why a construct is avoided.
|
|
454
|
-
- No `NOSONAR` markers: change the code so that the SonarCloud rule is satisfied, and ask
|
|
455
|
-
when that seems impossible. The only exception is `nnf.cpp`: cpp:S2245 flags every
|
|
456
|
-
declaration of `std::mt19937`, which only drives the randomized search.
|
|
457
|
-
- Tests check invariants instead of golden images: holes filled, all other pixels
|
|
458
|
-
unchanged, determinism with `set_random_seed`, fills that blend in with their
|
|
459
|
-
surroundings. `test_patch_match.py` and `test_e2e.py` seed through an autouse fixture.
|
|
460
|
-
Unit tests use a small synthetic image; the `e2e` tests use
|
|
461
|
-
`examples/images/forest_pruned.bmp` and mirror the workflows of the README and the
|
|
462
|
-
examples. `test_properties.py` checks the invariants on random inputs with Hypothesis
|
|
463
|
-
(in the `test` group, so the wheel tests run it too); it seeds inside the tests,
|
|
464
|
-
because Hypothesis rejects function-scoped fixtures, and has no deadline, because
|
|
465
|
-
the sanitizer and coverage builds are much slower.
|
|
466
|
-
- The README documents the behavior (mask semantics, `patch_size` as a radius, the
|
|
467
|
-
fallback installation); update it together with API or installation changes.
|
|
468
|
-
- Commit messages: imperative subject in sentence case without a prefix (e.g. "Reject
|
|
469
|
-
invalid patch sizes and empty regularity maps"), and a body that explains the reason,
|
|
470
|
-
with measurements where relevant. Changes go through PRs from branches such as
|
|
471
|
-
`fix/<topic>`.
|
|
472
|
-
|
|
473
|
-
## Git and GitHub
|
|
474
|
-
|
|
475
|
-
- `origin` is `mauwii/PyPatchMatch`, where PRs and releases go; `upstream` is
|
|
476
|
-
`invoke-ai/PyPatchMatch`. `gh` resolves the default repository to upstream, so pass
|
|
477
|
-
`-R mauwii/PyPatchMatch`.
|
|
478
|
-
- Releases: creating a GitHub release with a tag like `v2.0.0` (or `v2.0.0rc2` for
|
|
479
|
-
pre-releases) builds the wheels and publishes them to PyPI.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|