PyPatchMatch 2.0.1__tar.gz → 2.1.0__tar.gz

{pypatchmatch-2.0.1 → pypatchmatch-2.1.0}/CLAUDE.md

@@ -77,32 +77,20 @@ Other entry points:
 
 ## Architecture
 
-### Layers, in the call order of `patchmatch.inpaint`
-
-1. `patchmatch/__init__.py` re-exports the public API of `patch_match.py` and
-   `__version__` from the generated `_version.py`. `test_top_level_exports` checks that
-   every name in `patch_match.__all__` is re-exported.
-2. `patchmatch/patch_match.py` is the public API. It validates and canonicalizes the
-   inputs (`_prepare_inputs`, `_check_patch_size`, the `ijmap` and `guide_weight`
-   checks), picks the entry point (`PM_inpaint`, `PM_inpaint2` with a global mask, and
-   their `_regularity` variants) and runs it through `_call`. It loads the library at
-   import time; on `OSError` it logs a warning and sets `patchmatch_available = False`,
-   and every function then raises `RuntimeError` through `_get_lib()`.
-3. `patchmatch/_lib.py` is the ctypes layer: the `CShapeT`/`CMatT` structures, the dtype
-   table, `np_to_pymat`/`pymat_to_np`, `find_library` (looks for `libpatchmatch.so`,
-   `libpatchmatch.dylib` or `patchmatch.dll` in every directory of `patchmatch.__path__`)
-   and `load_library` (argtypes and restype of every `PM_*` function).
-4. `patchmatch/csrc/pyinterface.{h,cpp}` is the `extern "C"` API: the `PM_*` functions,
-   the conversion between `PM_mat_t` and `cv::Mat`, the global settings and the error
-   reporting.
-5. `csrc/inpaint.{h,cpp}`: `Inpainting`, the image pyramid and the EM loop.
-6. `csrc/nnf.{h,cpp}`: `NearestNeighborField` (propagation and random search) and the
-   distance metrics `PatchSSDDistanceMetric` and `RegularityGuidedPatchDistanceMetricV2`.
-7. `csrc/masked_image.{h,cpp}`: `MaskedImage`, an image with its hole mask and optional
-   global mask, down- and upsampling, gradients.
-
-The C++ headers double as the C++ API: `examples/cpp_example.cpp` uses `Inpainting` and
-`PatchSSDDistanceMetric` directly.
+### Layers
+
+`patchmatch.inpaint` calls through `patch_match.py` (public API, input validation) →
+`_lib.py` (ctypes) → `csrc/pyinterface.*` (`extern "C"`) → `inpaint.*` (`Inpainting`,
+pyramid, EM loop) → `nnf.*` (`NearestNeighborField`, distance metrics) →
+`masked_image.*` (`MaskedImage`).
+
+- `patch_match.py` loads the library at import time; on `OSError` (no library) or
+  `AttributeError` (one without the `PM_*` symbols) it sets `patchmatch_available =
+  False`, and every function then raises `RuntimeError` through `_get_lib()`. Without a
+  hole outside the global mask (`_has_holes`) it returns a copy without a native call.
+- `__init__.py` re-exports `patch_match.__all__` (`test_top_level_exports`).
+- The C++ headers double as the C++ API: `examples/cpp_example.cpp` uses `Inpainting`
+  and `PatchSSDDistanceMetric` directly.
 
 ### ctypes instead of a CPython extension
 

{pypatchmatch-2.0.1 → pypatchmatch-2.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: PyPatchMatch
-Version: 2.0.1
+Version: 2.1.0
 Summary: This library implements the PatchMatch based inpainting algorithm.
 Keywords: inpainting,patchmatch,image-processing,opencv
 Author-Email: The InvokeAI Project <lincoln.stein@gmail.com>, Jiayuan Mao <maojiayuan@gmail.com>, Kyle Schouviller <kyle0654@hotmail.com>, Matthias Wild <mauwii@outlook.de>, Younesse ANDAM <younesse.andam@gmail.com>
@@ -30,19 +30,20 @@ Description-Content-Type: text/markdown
 
 # PatchMatch based Inpainting
 
-[![CI](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml)
 [![Quality Gate](https://sonarcloud.io/api/project_badges/measure?project=mauwii_PyPatchMatch&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=mauwii_PyPatchMatch)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blueviolet.svg)](https://github.com/mauwii/PyPatchMatch/blob/main/LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/PyPatchMatch)](https://pypi.org/project/PyPatchMatch/)
 [![Downloads](https://static.pepy.tech/badge/pypatchmatch)](https://pepy.tech/projects/pypatchmatch)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![ponytail](https://img.shields.io/badge/over--engineering-ponytail_audited-111111)](https://github.com/DietrichGebert/ponytail)
+[![caveman](https://img.shields.io/badge/agent-caveman-F0A63C)](https://github.com/JuliusBrussee/caveman)
 
 This library implements the PatchMatch based inpainting algorithm. It provides both C++
 and Python interfaces. This implementation is heavily based on the implementation by
 Younesse ANDAM: [younesse-cv/PatchMatch](https://github.com/younesse-cv/PatchMatch),
 with some bug fixes, and updates.
 
-![Birdhouses removed from a photo by patchmatch.inpaint](https://raw.githubusercontent.com/mauwii/PyPatchMatch/v2.0.1/examples/images/readme_example.jpg)
+![Birdhouses removed from a photo by patchmatch.inpaint](https://raw.githubusercontent.com/mauwii/PyPatchMatch/v2.1.0/examples/images/readme_example.jpg)
 
 From left to right: the photo, the input of
 [examples/py_example.py](https://github.com/mauwii/PyPatchMatch/blob/main/examples/py_example.py)
@@ -99,11 +100,12 @@ The previous import path `from patchmatch import patch_match` keeps working.
 pixels in each direction. Larger patches follow larger structures but are much slower;
 the default of 15 compares 31x31 patches, the examples use 3.
 
-`patchmatch.set_random_seed(seed)` sets the seed of the randomized search and
-`patchmatch.set_verbose(True)` prints the progress of the native code to stderr.
-`patchmatch.inpaint_regularity(image, mask, ijmap)` additionally guides the search with
-a regularity map, an HxWx3 float32 array with the regularity coordinates of each pixel
-in its first two channels. Its `guide_weight` must be at least 0.
+`patchmatch.set_random_seed(seed)` sets the seed of the randomized search, modulo
+`2**32`, and `patchmatch.set_verbose(True)` prints the progress of the native code to
+stderr. `patchmatch.inpaint_regularity(image, mask, ijmap)` additionally guides the
+search with a regularity map, an HxWx3 float32 array with the regularity coordinates of
+each pixel in its first two channels. Its `guide_weight` must be between 0 and the
+largest float32, about 3.4e38.
 
 C++ (see
 [examples/cpp_example.cpp](https://github.com/mauwii/PyPatchMatch/blob/main/examples/cpp_example.cpp),

{pypatchmatch-2.0.1 → pypatchmatch-2.1.0}/README.md

@@ -1,11 +1,12 @@
 # PatchMatch based Inpainting
 
-[![CI](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml)
 [![Quality Gate](https://sonarcloud.io/api/project_badges/measure?project=mauwii_PyPatchMatch&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=mauwii_PyPatchMatch)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blueviolet.svg)](https://github.com/mauwii/PyPatchMatch/blob/main/LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/PyPatchMatch)](https://pypi.org/project/PyPatchMatch/)
 [![Downloads](https://static.pepy.tech/badge/pypatchmatch)](https://pepy.tech/projects/pypatchmatch)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![ponytail](https://img.shields.io/badge/over--engineering-ponytail_audited-111111)](https://github.com/DietrichGebert/ponytail)
+[![caveman](https://img.shields.io/badge/agent-caveman-F0A63C)](https://github.com/JuliusBrussee/caveman)
 
 This library implements the PatchMatch based inpainting algorithm. It provides both C++
 and Python interfaces. This implementation is heavily based on the implementation by
@@ -69,11 +70,12 @@ The previous import path `from patchmatch import patch_match` keeps working.
 pixels in each direction. Larger patches follow larger structures but are much slower;
 the default of 15 compares 31x31 patches, the examples use 3.
 
-`patchmatch.set_random_seed(seed)` sets the seed of the randomized search and
-`patchmatch.set_verbose(True)` prints the progress of the native code to stderr.
-`patchmatch.inpaint_regularity(image, mask, ijmap)` additionally guides the search with
-a regularity map, an HxWx3 float32 array with the regularity coordinates of each pixel
-in its first two channels. Its `guide_weight` must be at least 0.
+`patchmatch.set_random_seed(seed)` sets the seed of the randomized search, modulo
+`2**32`, and `patchmatch.set_verbose(True)` prints the progress of the native code to
+stderr. `patchmatch.inpaint_regularity(image, mask, ijmap)` additionally guides the
+search with a regularity map, an HxWx3 float32 array with the regularity coordinates of
+each pixel in its first two channels. Its `guide_weight` must be between 0 and the
+largest float32, about 3.4e38.
 
 C++ (see
 [examples/cpp_example.cpp](https://github.com/mauwii/PyPatchMatch/blob/main/examples/cpp_example.cpp),

{pypatchmatch-2.0.1 → pypatchmatch-2.1.0}/patchmatch/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '2.0.1'
-__version_tuple__ = version_tuple = (2, 0, 1)
+__version__ = version = '2.1.0'
+__version_tuple__ = version_tuple = (2, 1, 0)
 
-__commit_id__ = commit_id = 'g995b76749'
+__commit_id__ = commit_id = 'g2f558d40a'

{pypatchmatch-2.0.1 → pypatchmatch-2.1.0}/patchmatch/csrc/inpaint.cpp

@@ -2,7 +2,10 @@
 
 #include <algorithm>
 #include <array>
+#include <cstdint>
+#include <cstdlib>
 #include <iostream>
+#include <optional>
 
 namespace
 {
@@ -30,6 +33,108 @@ namespace
         return table;
     }
 
+    // Calls visit(y, x) for the pixels of the image within radius of (cy, cx).
+    template <typename Visit>
+    void for_window(cv::Size size, int cy, int cx, int radius, Visit visit)
+    {
+        for (int y = std::max(0, cy - radius); y <= std::min(size.height - 1, cy + radius); ++y)
+            for (int x = std::max(0, cx - radius); x <= std::min(size.width - 1, cx + radius); ++x)
+                visit(y, x);
+    }
+
+    // Mean of value(y, x) over the pixels within radius of (cy, cx) for which select(y, x) holds.
+    template <typename Select, typename Value>
+    cv::Vec3d window_mean(cv::Size size, int cy, int cx, int radius, Select select, Value value)
+    {
+        cv::Vec3d sum;
+        int count = 0;
+        for_window(size, cy, cx, radius, [&sum, &count, &select, &value](int y, int x) {
+            if (select(y, x))
+            {
+                sum += value(y, x);
+                ++count;
+            }
+        });
+        return count > 0 ? sum / count : sum;
+    }
+
+    bool is_known(const MaskedImage &image, int y, int x)
+    {
+        return !image.is_masked(y, x) && !image.is_globally_masked(y, x);
+    }
+
+    bool is_hole(const MaskedImage &image, int y, int x)
+    {
+        return image.is_masked(y, x) && !image.is_globally_masked(y, x);
+    }
+
+    // Chessboard distance of the hole pixels to the nearest known pixel, 0 beyond radius.
+    cv::Mat hole_depths(const MaskedImage &image, int radius)
+    {
+        const auto size = image.size();
+        cv::Mat depth(size, CV_32S, cv::Scalar(0));
+        for (int cy = 0; cy < size.height; ++cy)
+        {
+            for (int cx = 0; cx < size.width; ++cx)
+            {
+                if (!is_hole(image, cy, cx))
+                    continue;
+                int &d = depth.at<int>(cy, cx);
+                for_window(size, cy, cx, radius, [&d, &image, cy, cx](int y, int x) {
+                    const int distance = std::max(std::abs(y - cy), std::abs(x - cx));
+                    if (is_known(image, y, x) && (d == 0 || distance < d))
+                        d = distance;
+                });
+            }
+        }
+        return depth;
+    }
+
+    // Shifts the hole pixels near the border by the color step across it, which showed as an outline.
+    void blend_hole_borders(cv::Mat &result, const MaskedImage &initial)
+    {
+        constexpr int kWidth = 4;
+        constexpr int kStepRadius = 1;
+        constexpr int kSpreadRadius = 4;
+        const auto size = initial.size();
+        const auto known = [&initial](int y, int x) { return is_known(initial, y, x); };
+        const auto hole = [&initial](int y, int x) { return is_hole(initial, y, x); };
+        const auto color = [&result](int y, int x) {
+            const auto *pixel = result.ptr<unsigned char>(y, x);
+            return cv::Vec3d(pixel[0], pixel[1], pixel[2]);
+        };
+
+        const cv::Mat depth = hole_depths(initial, kWidth);
+        const auto border = [&depth](int y, int x) { return depth.at<int>(y, x) == 1; };
+
+        cv::Mat steps(size, CV_64FC3, cv::Scalar::all(0));
+        for (int y = 0; y < size.height; ++y)
+        {
+            for (int x = 0; x < size.width; ++x)
+            {
+                if (border(y, x))
+                    steps.at<cv::Vec3d>(y, x) = window_mean(size, y, x, kStepRadius, known, color) -
+                                                window_mean(size, y, x, kStepRadius, hole, color);
+            }
+        }
+
+        const auto step = [&steps](int y, int x) { return steps.at<cv::Vec3d>(y, x); };
+        for (int y = 0; y < size.height; ++y)
+        {
+            for (int x = 0; x < size.width; ++x)
+            {
+                const int d = depth.at<int>(y, x);
+                if (d == 0)
+                    continue;
+                const cv::Vec3d shift =
+                    window_mean(size, y, x, kSpreadRadius, border, step) * (kWidth + 1 - d) / kWidth;
+                auto *pixel = result.ptr<unsigned char>(y, x);
+                for (int c = 0; c < 3; ++c)
+                    pixel[c] = cv::saturate_cast<unsigned char>(pixel[c] + shift[c]);
+            }
+        }
+    }
+
     template <typename... Args>
     void trace(bool verbose, const Args &...args)
     {
@@ -70,14 +175,59 @@ namespace
         target_ptr[3] = weight;
     }
 
+    // Counts the holes of an image in rectangles, from the number of holes above and left of each pixel.
+    class HoleCounter
+    {
+    public:
+        explicit HoleCounter(const MaskedImage &image)
+            : m_size(image.size()), m_sums(static_cast<std::size_t>(m_size.height + 1) * (m_size.width + 1), 0)
+        {
+            for (int y = 0; y < m_size.height; ++y)
+            {
+                for (int x = 0; x < m_size.width; ++x)
+                {
+                    const std::int64_t hole = image.is_masked(y, x) ? 1 : 0;
+                    sum(y + 1, x + 1) = hole + sum(y, x + 1) + sum(y + 1, x) - sum(y, x);
+                }
+            }
+        }
+
+        // Whether the rectangle from (y0, x0) to (y1, x1), clipped to the image, contains a hole.
+        bool any(std::int64_t y0, std::int64_t x0, std::int64_t y1, std::int64_t x1) const
+        {
+            const auto clip = [](std::int64_t value, int size) {
+                return static_cast<int>(std::clamp<std::int64_t>(value, 0, size));
+            };
+            const int top = clip(y0, m_size.height);
+            const int bottom = clip(y1 + 1, m_size.height);
+            const int left = clip(x0, m_size.width);
+            const int right = clip(x1 + 1, m_size.width);
+            return sum(bottom, right) - sum(top, right) - sum(bottom, left) + sum(top, left) > 0;
+        }
+
+    private:
+        std::int64_t &sum(int y, int x)
+        {
+            return m_sums[static_cast<std::size_t>(y) * (m_size.width + 1) + x];
+        }
+        std::int64_t sum(int y, int x) const
+        {
+            return m_sums[static_cast<std::size_t>(y) * (m_size.width + 1) + x];
+        }
+
+        cv::Size m_size;
+        std::vector<std::int64_t> m_sums;
+    };
+
     // Casts the votes of one nearest-neighbor field into the vote image.
     class VoteCaster
     {
     public:
+        // With holes, only the patches whose votes reach one of them are cast.
         VoteCaster(
             const NearestNeighborField &nnf, bool source2target, cv::Mat &vote, const MaskedImage &source,
-            bool upscaled, bool best_only)
-            : m_nnf(&nnf), m_vote(&vote), m_source(&source), m_source_size(nnf.source_size()),
+            bool upscaled, bool best_only, const HoleCounter *holes)
+            : m_nnf(&nnf), m_vote(&vote), m_source(&source), m_holes(holes), m_source_size(nnf.source_size()),
               m_target_size(nnf.target_size()), m_source2target(source2target), m_upscaled(upscaled),
               m_best_only(best_only)
         {
@@ -89,6 +239,8 @@ namespace
         {
             const int yp = m_nnf->at(i, j, 0);
             const int xp = m_nnf->at(i, j, 1);
+            if (m_holes && !votes_for_hole(m_source2target ? yp : i, m_source2target ? xp : j, patch_size))
+                return;
             for (int di = -patch_size; di <= patch_size; ++di)
             {
                 for (int dj = -patch_size; dj <= patch_size; ++dj)
@@ -103,12 +255,23 @@ namespace
                     if (yt < 0 || yt >= m_target_size.height || xt < 0 || xt >= m_target_size.width ||
                         m_nnf->target().is_globally_masked(yt, xt))
                         continue;
-                    cast_pair(ys, xs, yt, xt, weight);
+                    // Near ties go to the patch centered closest, not the first cast, which made blocks.
+                    const double offset2 = static_cast<double>(di) * di + static_cast<double>(dj) * dj;
+                    cast_pair(ys, xs, yt, xt, m_best_only ? weight * (1 - 1e-5 * offset2) : weight);
                 }
             }
         }
 
     private:
+        // Whether the patch around (y, x) of the voted image, scaled like the votes, has a hole.
+        bool votes_for_hole(int y, int x, int patch_size) const
+        {
+            const std::int64_t scale = m_upscaled ? 2 : 1;
+            return m_holes->any(
+                scale * (std::int64_t{y} - patch_size), scale * (std::int64_t{x} - patch_size),
+                scale * (std::int64_t{y} + patch_size + 1) - 1, scale * (std::int64_t{x} + patch_size + 1) - 1);
+        }
+
         void cast_pair(int ys, int xs, int yt, int xt, double weight) const
         {
             if (!m_source2target)
@@ -137,6 +300,7 @@ namespace
         const NearestNeighborField *m_nnf;
         cv::Mat *m_vote;
         const MaskedImage *m_source;
+        const HoleCounter *m_holes;
         cv::Size m_source_size;
         cv::Size m_target_size;
         bool m_source2target;
@@ -198,18 +362,21 @@ cv::Mat Inpainting::run(bool verbose, unsigned int random_seed)
 
         source = m_pyramid[level];
 
-        if (level == nr_levels - 1)
+        const bool coarsest = level == nr_levels - 1;
+        if (coarsest)
         {
             target = source.clone();
             target.clear_mask();
-            m_source2target = NearestNeighborField(source, target, m_distance_metric);
-            m_target2source = NearestNeighborField(target, source, m_distance_metric);
         }
+        // The votes of the completeness field keep the known pixels near their input on the
+        // coarse levels; on levels 0 and 1, which keep the known pixels, they change nothing.
+        if (level < 2)
+            m_source2target.reset();
         else
-        {
-            m_source2target = NearestNeighborField(source, target, m_distance_metric, m_source2target);
-            m_target2source = NearestNeighborField(target, source, m_distance_metric, m_target2source);
-        }
+            m_source2target = coarsest ? NearestNeighborField(source, target, m_distance_metric)
+                                       : NearestNeighborField(source, target, m_distance_metric, *m_source2target);
+        m_target2source = coarsest ? NearestNeighborField(target, source, m_distance_metric)
+                                   : NearestNeighborField(target, source, m_distance_metric, m_target2source);
 
         trace(verbose, "Initialization done.");
 
@@ -232,6 +399,7 @@ cv::Mat Inpainting::run(bool verbose, unsigned int random_seed)
             std::copy(input, input + 3, result.ptr<unsigned char>(y, x));
         }
     }
+    blend_hole_borders(result, m_initial);
     return result;
 }
 
@@ -250,16 +418,18 @@ MaskedImage Inpainting::_expectation_maximization(
     {
         if (iter_em != 0)
         {
-            m_source2target.set_target(new_target);
+            if (m_source2target)
+                m_source2target->set_target(new_target);
             m_target2source.set_source(new_target);
             target = new_target;
         }
 
         trace(verbose, "EM Iteration: ", iter_em);
 
-        _link_patches_without_holes(source);
+        _update_links(source, iter_em != 0);
         trace(verbose, "  NNF minimization started.");
-        m_source2target.minimize(nr_iters_nnf);
+        if (m_source2target)
+            m_source2target->minimize(nr_iters_nnf);
         m_target2source.minimize(nr_iters_nnf);
         trace(verbose, "  NNF minimization finished.");
 
@@ -288,38 +458,49 @@ MaskedImage Inpainting::_expectation_maximization(
         // structure that the finer levels refine, the best vote there makes seams.
         const bool best_only = level == 0 && iter_em == nr_iters_em - 1;
 
-        // Votes for best patch from NNF Source->Target (completeness) and Target->Source (coherence).
-        _expectation_step(m_source2target, true, vote, new_source, upscaled, best_only);
-        trace(verbose, "  Expectation source to target finished.");
-        _expectation_step(m_target2source, false, vote, new_source, upscaled, best_only);
-        trace(verbose, "  Expectation target to source finished.");
-
-        // Compile votes and update pixel values.
         // Known pixels are kept on the two finest levels only. On coarser levels those
         // next to the holes average only the known part of their downsampling kernel,
         // estimates that reach into the holes; keeping them there made the fills copy
         // smooth regions, e.g. a concrete wall into the foliage above it.
         const int new_level = upscaled ? level - 1 : level;
-        _maximization_step(new_target, vote, new_source, new_level <= 1);
+        const bool keep_known = new_level <= 1;
+
+        // Votes for best patch from NNF Source->Target (completeness) and Target->Source (coherence).
+        if (m_source2target)
+        {
+            _expectation_step(*m_source2target, true, vote, new_source, upscaled, best_only, keep_known);
+            trace(verbose, "  Expectation source to target finished.");
+        }
+        _expectation_step(m_target2source, false, vote, new_source, upscaled, best_only, keep_known);
+        trace(verbose, "  Expectation target to source finished.");
+
+        // Compile votes and update pixel values.
+        _maximization_step(new_target, vote, new_source, keep_known);
         trace(verbose, "  Minimization step finished.");
     }
 
     return new_target;
 }
 
-// Patches without holes are their own nearest neighbors.
-void Inpainting::_link_patches_without_holes(const MaskedImage &source)
+// Patches without holes are their own nearest neighbors; the others keep their links,
+// measured again on a changed image. Globally masked pixels have no link.
+void Inpainting::_update_links(const MaskedImage &source, bool image_changed)
 {
+    const auto for_each_field = [this](const auto &update) {
+        if (m_source2target)
+            update(*m_source2target);
+        update(m_target2source);
+    };
     const int patch_size = m_distance_metric->patch_size();
     const auto size = source.size();
     for (int i = 0; i < size.height; ++i)
     {
         for (int j = 0; j < size.width; ++j)
         {
-            if (source.contains_mask(i, j, patch_size))
-                continue;
-            m_source2target.set_identity(i, j);
-            m_target2source.set_identity(i, j);
+            if (!source.contains_mask(i, j, patch_size))
+                for_each_field([i, j](NearestNeighborField &field) { field.set_identity(i, j); });
+            else if (image_changed && !source.is_globally_masked(i, j))
+                for_each_field([i, j](NearestNeighborField &field) { field.update_distance(i, j); });
         }
     }
 }
@@ -327,9 +508,14 @@ void Inpainting::_link_patches_without_holes(const MaskedImage &source)
 // Expectation step: vote for best estimations of each pixel.
 void Inpainting::_expectation_step(
     const NearestNeighborField &nnf, bool source2target, cv::Mat &vote, const MaskedImage &source, bool upscaled,
-    bool best_only) const
+    bool best_only, bool keep_known) const
 {
-    const VoteCaster caster(nnf, source2target, vote, source, upscaled, best_only);
+    // Votes for known pixels do not count where the maximization step keeps them.
+    std::optional<HoleCounter> holes;
+    if (keep_known)
+        holes.emplace(source);
+    const VoteCaster caster(
+        nnf, source2target, vote, source, upscaled, best_only, holes.has_value() ? &holes.value() : nullptr);
     const int patch_size = m_distance_metric->patch_size();
     const auto &kDistance2Similarity = distance2similarity();
     const auto source_size = nnf.source_size();
@@ -380,10 +566,6 @@ void Inpainting::_maximization_step(
                 target_ptr[1] = g;
                 target_ptr[2] = b;
             }
-            else
-            {
-                target.set_mask(i, j, false);
-            }
         }
     }
 }

{pypatchmatch-2.0.1 → pypatchmatch-2.1.0}/patchmatch/csrc/inpaint.h

@@ -1,5 +1,6 @@
 #pragma once
 
+#include <optional>
 #include <vector>
 
 #include "masked_image.h"
@@ -16,16 +17,16 @@ public:
 private:
     void _initialize_pyramid(void);
     MaskedImage _expectation_maximization(const MaskedImage &source, MaskedImage target, int level, bool verbose);
-    void _link_patches_without_holes(const MaskedImage &source);
+    void _update_links(const MaskedImage &source, bool image_changed);
     void _expectation_step(
         const NearestNeighborField &nnf, bool source2target, cv::Mat &vote, const MaskedImage &source, bool upscaled,
-        bool best_only) const;
+        bool best_only, bool keep_known) const;
     void _maximization_step(MaskedImage &target, const cv::Mat &vote, const MaskedImage &source, bool keep_known) const;
 
     MaskedImage m_initial;
     std::vector<MaskedImage> m_pyramid;
 
-    NearestNeighborField m_source2target;
+    std::optional<NearestNeighborField> m_source2target;
     NearestNeighborField m_target2source;
     const PatchDistanceMetric *m_distance_metric;
 };