PyPatchMatch 2.0.2__tar.gz → 2.1.0__tar.gz

{pypatchmatch-2.0.2 → pypatchmatch-2.1.0}/CLAUDE.md

@@ -77,35 +77,20 @@ Other entry points:
 
 ## Architecture
 
-### Layers, in the call order of `patchmatch.inpaint`
-
-1. `patchmatch/__init__.py` re-exports the public API of `patch_match.py` and
-   `__version__` from the generated `_version.py`. `test_top_level_exports` checks that
-   every name in `patch_match.__all__` is re-exported.
-2. `patchmatch/patch_match.py` is the public API. It validates and canonicalizes the
-   inputs (`_prepare_inputs`, `_check_patch_size`, the `ijmap` and `guide_weight`
-   checks), returns a copy of the image without a native call if `_has_holes` finds no
-   hole outside the global mask, picks the entry point (`PM_inpaint`, `PM_inpaint2`
-   with a global mask, and their `_regularity` variants) and runs it through `_call`.
-   It loads the library at import time; on `OSError` (no library) or `AttributeError`
-   (one without the `PM_*` symbols) it logs a warning and sets
-   `patchmatch_available = False`, and every function then raises `RuntimeError`
-   through `_get_lib()`.
-3. `patchmatch/_lib.py` is the ctypes layer: the `CShapeT`/`CMatT` structures, the dtype
-   table, `np_to_pymat`/`pymat_to_np`, `find_library` (looks for `libpatchmatch.so`,
-   `libpatchmatch.dylib` or `patchmatch.dll` in every directory of `patchmatch.__path__`)
-   and `load_library` (argtypes and restype of every `PM_*` function).
-4. `patchmatch/csrc/pyinterface.{h,cpp}` is the `extern "C"` API: the `PM_*` functions,
-   the conversion between `PM_mat_t` and `cv::Mat`, the global settings and the error
-   reporting.
-5. `csrc/inpaint.{h,cpp}`: `Inpainting`, the image pyramid and the EM loop.
-6. `csrc/nnf.{h,cpp}`: `NearestNeighborField` (propagation and random search) and the
-   distance metrics `PatchSSDDistanceMetric` and `RegularityGuidedPatchDistanceMetricV2`.
-7. `csrc/masked_image.{h,cpp}`: `MaskedImage`, an image with its hole mask and optional
-   global mask, down- and upsampling, gradients.
-
-The C++ headers double as the C++ API: `examples/cpp_example.cpp` uses `Inpainting` and
-`PatchSSDDistanceMetric` directly.
+### Layers
+
+`patchmatch.inpaint` calls through `patch_match.py` (public API, input validation) →
+`_lib.py` (ctypes) → `csrc/pyinterface.*` (`extern "C"`) → `inpaint.*` (`Inpainting`,
+pyramid, EM loop) → `nnf.*` (`NearestNeighborField`, distance metrics) →
+`masked_image.*` (`MaskedImage`).
+
+- `patch_match.py` loads the library at import time; on `OSError` (no library) or
+  `AttributeError` (one without the `PM_*` symbols) it sets `patchmatch_available =
+  False`, and every function then raises `RuntimeError` through `_get_lib()`. Without a
+  hole outside the global mask (`_has_holes`) it returns a copy without a native call.
+- `__init__.py` re-exports `patch_match.__all__` (`test_top_level_exports`).
+- The C++ headers double as the C++ API: `examples/cpp_example.cpp` uses `Inpainting`
+  and `PatchSSDDistanceMetric` directly.
 
 ### ctypes instead of a CPython extension
 

{pypatchmatch-2.0.2 → pypatchmatch-2.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: PyPatchMatch
-Version: 2.0.2
+Version: 2.1.0
 Summary: This library implements the PatchMatch based inpainting algorithm.
 Keywords: inpainting,patchmatch,image-processing,opencv
 Author-Email: The InvokeAI Project <lincoln.stein@gmail.com>, Jiayuan Mao <maojiayuan@gmail.com>, Kyle Schouviller <kyle0654@hotmail.com>, Matthias Wild <mauwii@outlook.de>, Younesse ANDAM <younesse.andam@gmail.com>
@@ -30,19 +30,20 @@ Description-Content-Type: text/markdown
 
 # PatchMatch based Inpainting
 
-[![CI](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml)
 [![Quality Gate](https://sonarcloud.io/api/project_badges/measure?project=mauwii_PyPatchMatch&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=mauwii_PyPatchMatch)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blueviolet.svg)](https://github.com/mauwii/PyPatchMatch/blob/main/LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/PyPatchMatch)](https://pypi.org/project/PyPatchMatch/)
 [![Downloads](https://static.pepy.tech/badge/pypatchmatch)](https://pepy.tech/projects/pypatchmatch)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![ponytail](https://img.shields.io/badge/over--engineering-ponytail_audited-111111)](https://github.com/DietrichGebert/ponytail)
+[![caveman](https://img.shields.io/badge/agent-caveman-F0A63C)](https://github.com/JuliusBrussee/caveman)
 
 This library implements the PatchMatch based inpainting algorithm. It provides both C++
 and Python interfaces. This implementation is heavily based on the implementation by
 Younesse ANDAM: [younesse-cv/PatchMatch](https://github.com/younesse-cv/PatchMatch),
 with some bug fixes, and updates.
 
-![Birdhouses removed from a photo by patchmatch.inpaint](https://raw.githubusercontent.com/mauwii/PyPatchMatch/v2.0.2/examples/images/readme_example.jpg)
+![Birdhouses removed from a photo by patchmatch.inpaint](https://raw.githubusercontent.com/mauwii/PyPatchMatch/v2.1.0/examples/images/readme_example.jpg)
 
 From left to right: the photo, the input of
 [examples/py_example.py](https://github.com/mauwii/PyPatchMatch/blob/main/examples/py_example.py)

{pypatchmatch-2.0.2 → pypatchmatch-2.1.0}/README.md

@@ -1,11 +1,12 @@
 # PatchMatch based Inpainting
 
-[![CI](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml)
 [![Quality Gate](https://sonarcloud.io/api/project_badges/measure?project=mauwii_PyPatchMatch&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=mauwii_PyPatchMatch)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blueviolet.svg)](https://github.com/mauwii/PyPatchMatch/blob/main/LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/PyPatchMatch)](https://pypi.org/project/PyPatchMatch/)
 [![Downloads](https://static.pepy.tech/badge/pypatchmatch)](https://pepy.tech/projects/pypatchmatch)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![ponytail](https://img.shields.io/badge/over--engineering-ponytail_audited-111111)](https://github.com/DietrichGebert/ponytail)
+[![caveman](https://img.shields.io/badge/agent-caveman-F0A63C)](https://github.com/JuliusBrussee/caveman)
 
 This library implements the PatchMatch based inpainting algorithm. It provides both C++
 and Python interfaces. This implementation is heavily based on the implementation by

{pypatchmatch-2.0.2 → pypatchmatch-2.1.0}/patchmatch/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '2.0.2'
-__version_tuple__ = version_tuple = (2, 0, 2)
+__version__ = version = '2.1.0'
+__version_tuple__ = version_tuple = (2, 1, 0)
 
-__commit_id__ = commit_id = 'g38b611eb9'
+__commit_id__ = commit_id = 'g2f558d40a'

{pypatchmatch-2.0.2 → pypatchmatch-2.1.0}/patchmatch/csrc/inpaint.cpp

@@ -2,7 +2,10 @@
 
 #include <algorithm>
 #include <array>
+#include <cstdint>
+#include <cstdlib>
 #include <iostream>
+#include <optional>
 
 namespace
 {
@@ -30,6 +33,108 @@ namespace
         return table;
     }
 
+    // Calls visit(y, x) for the pixels of the image within radius of (cy, cx).
+    template <typename Visit>
+    void for_window(cv::Size size, int cy, int cx, int radius, Visit visit)
+    {
+        for (int y = std::max(0, cy - radius); y <= std::min(size.height - 1, cy + radius); ++y)
+            for (int x = std::max(0, cx - radius); x <= std::min(size.width - 1, cx + radius); ++x)
+                visit(y, x);
+    }
+
+    // Mean of value(y, x) over the pixels within radius of (cy, cx) for which select(y, x) holds.
+    template <typename Select, typename Value>
+    cv::Vec3d window_mean(cv::Size size, int cy, int cx, int radius, Select select, Value value)
+    {
+        cv::Vec3d sum;
+        int count = 0;
+        for_window(size, cy, cx, radius, [&sum, &count, &select, &value](int y, int x) {
+            if (select(y, x))
+            {
+                sum += value(y, x);
+                ++count;
+            }
+        });
+        return count > 0 ? sum / count : sum;
+    }
+
+    bool is_known(const MaskedImage &image, int y, int x)
+    {
+        return !image.is_masked(y, x) && !image.is_globally_masked(y, x);
+    }
+
+    bool is_hole(const MaskedImage &image, int y, int x)
+    {
+        return image.is_masked(y, x) && !image.is_globally_masked(y, x);
+    }
+
+    // Chessboard distance of the hole pixels to the nearest known pixel, 0 beyond radius.
+    cv::Mat hole_depths(const MaskedImage &image, int radius)
+    {
+        const auto size = image.size();
+        cv::Mat depth(size, CV_32S, cv::Scalar(0));
+        for (int cy = 0; cy < size.height; ++cy)
+        {
+            for (int cx = 0; cx < size.width; ++cx)
+            {
+                if (!is_hole(image, cy, cx))
+                    continue;
+                int &d = depth.at<int>(cy, cx);
+                for_window(size, cy, cx, radius, [&d, &image, cy, cx](int y, int x) {
+                    const int distance = std::max(std::abs(y - cy), std::abs(x - cx));
+                    if (is_known(image, y, x) && (d == 0 || distance < d))
+                        d = distance;
+                });
+            }
+        }
+        return depth;
+    }
+
+    // Shifts the hole pixels near the border by the color step across it, which showed as an outline.
+    void blend_hole_borders(cv::Mat &result, const MaskedImage &initial)
+    {
+        constexpr int kWidth = 4;
+        constexpr int kStepRadius = 1;
+        constexpr int kSpreadRadius = 4;
+        const auto size = initial.size();
+        const auto known = [&initial](int y, int x) { return is_known(initial, y, x); };
+        const auto hole = [&initial](int y, int x) { return is_hole(initial, y, x); };
+        const auto color = [&result](int y, int x) {
+            const auto *pixel = result.ptr<unsigned char>(y, x);
+            return cv::Vec3d(pixel[0], pixel[1], pixel[2]);
+        };
+
+        const cv::Mat depth = hole_depths(initial, kWidth);
+        const auto border = [&depth](int y, int x) { return depth.at<int>(y, x) == 1; };
+
+        cv::Mat steps(size, CV_64FC3, cv::Scalar::all(0));
+        for (int y = 0; y < size.height; ++y)
+        {
+            for (int x = 0; x < size.width; ++x)
+            {
+                if (border(y, x))
+                    steps.at<cv::Vec3d>(y, x) = window_mean(size, y, x, kStepRadius, known, color) -
+                                                window_mean(size, y, x, kStepRadius, hole, color);
+            }
+        }
+
+        const auto step = [&steps](int y, int x) { return steps.at<cv::Vec3d>(y, x); };
+        for (int y = 0; y < size.height; ++y)
+        {
+            for (int x = 0; x < size.width; ++x)
+            {
+                const int d = depth.at<int>(y, x);
+                if (d == 0)
+                    continue;
+                const cv::Vec3d shift =
+                    window_mean(size, y, x, kSpreadRadius, border, step) * (kWidth + 1 - d) / kWidth;
+                auto *pixel = result.ptr<unsigned char>(y, x);
+                for (int c = 0; c < 3; ++c)
+                    pixel[c] = cv::saturate_cast<unsigned char>(pixel[c] + shift[c]);
+            }
+        }
+    }
+
     template <typename... Args>
     void trace(bool verbose, const Args &...args)
     {
@@ -70,14 +175,59 @@ namespace
         target_ptr[3] = weight;
     }
 
+    // Counts the holes of an image in rectangles, from the number of holes above and left of each pixel.
+    class HoleCounter
+    {
+    public:
+        explicit HoleCounter(const MaskedImage &image)
+            : m_size(image.size()), m_sums(static_cast<std::size_t>(m_size.height + 1) * (m_size.width + 1), 0)
+        {
+            for (int y = 0; y < m_size.height; ++y)
+            {
+                for (int x = 0; x < m_size.width; ++x)
+                {
+                    const std::int64_t hole = image.is_masked(y, x) ? 1 : 0;
+                    sum(y + 1, x + 1) = hole + sum(y, x + 1) + sum(y + 1, x) - sum(y, x);
+                }
+            }
+        }
+
+        // Whether the rectangle from (y0, x0) to (y1, x1), clipped to the image, contains a hole.
+        bool any(std::int64_t y0, std::int64_t x0, std::int64_t y1, std::int64_t x1) const
+        {
+            const auto clip = [](std::int64_t value, int size) {
+                return static_cast<int>(std::clamp<std::int64_t>(value, 0, size));
+            };
+            const int top = clip(y0, m_size.height);
+            const int bottom = clip(y1 + 1, m_size.height);
+            const int left = clip(x0, m_size.width);
+            const int right = clip(x1 + 1, m_size.width);
+            return sum(bottom, right) - sum(top, right) - sum(bottom, left) + sum(top, left) > 0;
+        }
+
+    private:
+        std::int64_t &sum(int y, int x)
+        {
+            return m_sums[static_cast<std::size_t>(y) * (m_size.width + 1) + x];
+        }
+        std::int64_t sum(int y, int x) const
+        {
+            return m_sums[static_cast<std::size_t>(y) * (m_size.width + 1) + x];
+        }
+
+        cv::Size m_size;
+        std::vector<std::int64_t> m_sums;
+    };
+
     // Casts the votes of one nearest-neighbor field into the vote image.
     class VoteCaster
     {
     public:
+        // With holes, only the patches whose votes reach one of them are cast.
         VoteCaster(
             const NearestNeighborField &nnf, bool source2target, cv::Mat &vote, const MaskedImage &source,
-            bool upscaled, bool best_only)
-            : m_nnf(&nnf), m_vote(&vote), m_source(&source), m_source_size(nnf.source_size()),
+            bool upscaled, bool best_only, const HoleCounter *holes)
+            : m_nnf(&nnf), m_vote(&vote), m_source(&source), m_holes(holes), m_source_size(nnf.source_size()),
               m_target_size(nnf.target_size()), m_source2target(source2target), m_upscaled(upscaled),
               m_best_only(best_only)
         {
@@ -89,6 +239,8 @@ namespace
         {
             const int yp = m_nnf->at(i, j, 0);
             const int xp = m_nnf->at(i, j, 1);
+            if (m_holes && !votes_for_hole(m_source2target ? yp : i, m_source2target ? xp : j, patch_size))
+                return;
             for (int di = -patch_size; di <= patch_size; ++di)
             {
                 for (int dj = -patch_size; dj <= patch_size; ++dj)
@@ -103,12 +255,23 @@ namespace
                     if (yt < 0 || yt >= m_target_size.height || xt < 0 || xt >= m_target_size.width ||
                         m_nnf->target().is_globally_masked(yt, xt))
                         continue;
-                    cast_pair(ys, xs, yt, xt, weight);
+                    // Near ties go to the patch centered closest, not the first cast, which made blocks.
+                    const double offset2 = static_cast<double>(di) * di + static_cast<double>(dj) * dj;
+                    cast_pair(ys, xs, yt, xt, m_best_only ? weight * (1 - 1e-5 * offset2) : weight);
                 }
             }
         }
 
     private:
+        // Whether the patch around (y, x) of the voted image, scaled like the votes, has a hole.
+        bool votes_for_hole(int y, int x, int patch_size) const
+        {
+            const std::int64_t scale = m_upscaled ? 2 : 1;
+            return m_holes->any(
+                scale * (std::int64_t{y} - patch_size), scale * (std::int64_t{x} - patch_size),
+                scale * (std::int64_t{y} + patch_size + 1) - 1, scale * (std::int64_t{x} + patch_size + 1) - 1);
+        }
+
         void cast_pair(int ys, int xs, int yt, int xt, double weight) const
         {
             if (!m_source2target)
@@ -137,6 +300,7 @@ namespace
         const NearestNeighborField *m_nnf;
         cv::Mat *m_vote;
         const MaskedImage *m_source;
+        const HoleCounter *m_holes;
         cv::Size m_source_size;
         cv::Size m_target_size;
         bool m_source2target;
@@ -198,18 +362,21 @@ cv::Mat Inpainting::run(bool verbose, unsigned int random_seed)
 
         source = m_pyramid[level];
 
-        if (level == nr_levels - 1)
+        const bool coarsest = level == nr_levels - 1;
+        if (coarsest)
         {
             target = source.clone();
             target.clear_mask();
-            m_source2target = NearestNeighborField(source, target, m_distance_metric);
-            m_target2source = NearestNeighborField(target, source, m_distance_metric);
         }
+        // The votes of the completeness field keep the known pixels near their input on the
+        // coarse levels; on levels 0 and 1, which keep the known pixels, they change nothing.
+        if (level < 2)
+            m_source2target.reset();
         else
-        {
-            m_source2target = NearestNeighborField(source, target, m_distance_metric, m_source2target);
-            m_target2source = NearestNeighborField(target, source, m_distance_metric, m_target2source);
-        }
+            m_source2target = coarsest ? NearestNeighborField(source, target, m_distance_metric)
+                                       : NearestNeighborField(source, target, m_distance_metric, *m_source2target);
+        m_target2source = coarsest ? NearestNeighborField(target, source, m_distance_metric)
+                                   : NearestNeighborField(target, source, m_distance_metric, m_target2source);
 
         trace(verbose, "Initialization done.");
 
@@ -232,6 +399,7 @@ cv::Mat Inpainting::run(bool verbose, unsigned int random_seed)
             std::copy(input, input + 3, result.ptr<unsigned char>(y, x));
         }
     }
+    blend_hole_borders(result, m_initial);
     return result;
 }
 
@@ -250,16 +418,18 @@ MaskedImage Inpainting::_expectation_maximization(
     {
         if (iter_em != 0)
         {
-            m_source2target.set_target(new_target);
+            if (m_source2target)
+                m_source2target->set_target(new_target);
             m_target2source.set_source(new_target);
             target = new_target;
         }
 
         trace(verbose, "EM Iteration: ", iter_em);
 
-        _link_patches_without_holes(source);
+        _update_links(source, iter_em != 0);
         trace(verbose, "  NNF minimization started.");
-        m_source2target.minimize(nr_iters_nnf);
+        if (m_source2target)
+            m_source2target->minimize(nr_iters_nnf);
         m_target2source.minimize(nr_iters_nnf);
         trace(verbose, "  NNF minimization finished.");
 
@@ -288,38 +458,49 @@ MaskedImage Inpainting::_expectation_maximization(
         // structure that the finer levels refine, the best vote there makes seams.
         const bool best_only = level == 0 && iter_em == nr_iters_em - 1;
 
-        // Votes for best patch from NNF Source->Target (completeness) and Target->Source (coherence).
-        _expectation_step(m_source2target, true, vote, new_source, upscaled, best_only);
-        trace(verbose, "  Expectation source to target finished.");
-        _expectation_step(m_target2source, false, vote, new_source, upscaled, best_only);
-        trace(verbose, "  Expectation target to source finished.");
-
-        // Compile votes and update pixel values.
         // Known pixels are kept on the two finest levels only. On coarser levels those
         // next to the holes average only the known part of their downsampling kernel,
         // estimates that reach into the holes; keeping them there made the fills copy
         // smooth regions, e.g. a concrete wall into the foliage above it.
         const int new_level = upscaled ? level - 1 : level;
-        _maximization_step(new_target, vote, new_source, new_level <= 1);
+        const bool keep_known = new_level <= 1;
+
+        // Votes for best patch from NNF Source->Target (completeness) and Target->Source (coherence).
+        if (m_source2target)
+        {
+            _expectation_step(*m_source2target, true, vote, new_source, upscaled, best_only, keep_known);
+            trace(verbose, "  Expectation source to target finished.");
+        }
+        _expectation_step(m_target2source, false, vote, new_source, upscaled, best_only, keep_known);
+        trace(verbose, "  Expectation target to source finished.");
+
+        // Compile votes and update pixel values.
+        _maximization_step(new_target, vote, new_source, keep_known);
         trace(verbose, "  Minimization step finished.");
     }
 
     return new_target;
 }
 
-// Patches without holes are their own nearest neighbors.
-void Inpainting::_link_patches_without_holes(const MaskedImage &source)
+// Patches without holes are their own nearest neighbors; the others keep their links,
+// measured again on a changed image. Globally masked pixels have no link.
+void Inpainting::_update_links(const MaskedImage &source, bool image_changed)
 {
+    const auto for_each_field = [this](const auto &update) {
+        if (m_source2target)
+            update(*m_source2target);
+        update(m_target2source);
+    };
     const int patch_size = m_distance_metric->patch_size();
     const auto size = source.size();
     for (int i = 0; i < size.height; ++i)
     {
         for (int j = 0; j < size.width; ++j)
         {
-            if (source.contains_mask(i, j, patch_size))
-                continue;
-            m_source2target.set_identity(i, j);
-            m_target2source.set_identity(i, j);
+            if (!source.contains_mask(i, j, patch_size))
+                for_each_field([i, j](NearestNeighborField &field) { field.set_identity(i, j); });
+            else if (image_changed && !source.is_globally_masked(i, j))
+                for_each_field([i, j](NearestNeighborField &field) { field.update_distance(i, j); });
         }
     }
 }
@@ -327,9 +508,14 @@ void Inpainting::_link_patches_without_holes(const MaskedImage &source)
 // Expectation step: vote for best estimations of each pixel.
 void Inpainting::_expectation_step(
     const NearestNeighborField &nnf, bool source2target, cv::Mat &vote, const MaskedImage &source, bool upscaled,
-    bool best_only) const
+    bool best_only, bool keep_known) const
 {
-    const VoteCaster caster(nnf, source2target, vote, source, upscaled, best_only);
+    // Votes for known pixels do not count where the maximization step keeps them.
+    std::optional<HoleCounter> holes;
+    if (keep_known)
+        holes.emplace(source);
+    const VoteCaster caster(
+        nnf, source2target, vote, source, upscaled, best_only, holes.has_value() ? &holes.value() : nullptr);
     const int patch_size = m_distance_metric->patch_size();
     const auto &kDistance2Similarity = distance2similarity();
     const auto source_size = nnf.source_size();
@@ -380,10 +566,6 @@ void Inpainting::_maximization_step(
                 target_ptr[1] = g;
                 target_ptr[2] = b;
             }
-            else
-            {
-                target.set_mask(i, j, false);
-            }
         }
     }
 }

{pypatchmatch-2.0.2 → pypatchmatch-2.1.0}/patchmatch/csrc/inpaint.h

@@ -1,5 +1,6 @@
 #pragma once
 
+#include <optional>
 #include <vector>
 
 #include "masked_image.h"
@@ -16,16 +17,16 @@ public:
 private:
     void _initialize_pyramid(void);
     MaskedImage _expectation_maximization(const MaskedImage &source, MaskedImage target, int level, bool verbose);
-    void _link_patches_without_holes(const MaskedImage &source);
+    void _update_links(const MaskedImage &source, bool image_changed);
     void _expectation_step(
         const NearestNeighborField &nnf, bool source2target, cv::Mat &vote, const MaskedImage &source, bool upscaled,
-        bool best_only) const;
+        bool best_only, bool keep_known) const;
     void _maximization_step(MaskedImage &target, const cv::Mat &vote, const MaskedImage &source, bool keep_known) const;
 
     MaskedImage m_initial;
     std::vector<MaskedImage> m_pyramid;
 
-    NearestNeighborField m_source2target;
+    std::optional<NearestNeighborField> m_source2target;
     NearestNeighborField m_target2source;
     const PatchDistanceMetric *m_distance_metric;
 };

{pypatchmatch-2.0.2 → pypatchmatch-2.1.0}/patchmatch/csrc/masked_image.cpp

@@ -1,13 +1,13 @@
 #include "masked_image.h"
 
 #include <algorithm>
+#include <array>
 #include <cstdint>
 
-const cv::Size MaskedImage::kDownsampleKernelSize = cv::Size(6, 6);
-const std::array<int, 6> MaskedImage::kDownsampleKernel = {1, 5, 10, 10, 5, 1};
-
 namespace
 {
+    constexpr std::array<int, 6> kDownsampleKernel = {1, 5, 10, 10, 5, 1};
+
     struct KernelSum
     {
         int r = 0;
@@ -20,14 +20,13 @@ namespace
     // The known pixels under the downsampling kernel at (y, x), weighted by the kernel.
     KernelSum sum_kernel(const MaskedImage &image, int y, int x)
     {
-        const auto &kernel_size = MaskedImage::kDownsampleKernelSize;
-        const auto &kernel = MaskedImage::kDownsampleKernel;
+        constexpr int half = static_cast<int>(kDownsampleKernel.size()) / 2;
         const auto size = image.size();
 
         KernelSum sum;
-        for (int dy = -kernel_size.height / 2 + 1; dy <= kernel_size.height / 2; ++dy)
+        for (int dy = -half + 1; dy <= half; ++dy)
         {
-            for (int dx = -kernel_size.width / 2 + 1; dx <= kernel_size.width / 2; ++dx)
+            for (int dx = -half + 1; dx <= half; ++dx)
             {
                 const int yy = y + dy;
                 const int xx = x + dx;
@@ -40,7 +39,7 @@ namespace
                     continue;
 
                 const auto *source_ptr = image.get_image(yy, xx);
-                const int k = kernel[kernel_size.height / 2 - 1 + dy] * kernel[kernel_size.width / 2 - 1 + dx];
+                const int k = kDownsampleKernel[half - 1 + dy] * kDownsampleKernel[half - 1 + dx];
                 sum.r += source_ptr[0] * k;
                 sum.g += source_ptr[1] * k;
                 sum.b += source_ptr[2] * k;
@@ -72,14 +71,14 @@ MaskedImage MaskedImage::downsample() const
 {
     const auto size = this->size();
     auto ret = MaskedImage(size.width / 2, size.height / 2);
-    if (!m_global_mask.empty())
+    if (m_has_global_mask)
         ret.init_global_mask_mat();
     for (int y = 0; y < size.height - 1; y += 2)
     {
         for (int x = 0; x < size.width - 1; x += 2)
         {
             const auto sum = sum_kernel(*this, y, x);
-            if (!m_global_mask.empty())
+            if (m_has_global_mask)
                 ret.set_global_mask(y / 2, x / 2, sum.globally_masked);
             if (sum.weight == 0)
             {
@@ -102,7 +101,7 @@ MaskedImage MaskedImage::upsample(int new_w, int new_h) const
 {
     const auto size = this->size();
     auto ret = MaskedImage(new_w, new_h);
-    if (!m_global_mask.empty())
+    if (m_has_global_mask)
         ret.init_global_mask_mat();
     for (int y = 0; y < new_h; ++y)
     {
@@ -118,7 +117,7 @@ MaskedImage MaskedImage::upsample(int new_w, int new_h) const
                 ret.set_mask(y, x, true);
                 continue;
             }
-            if (!m_global_mask.empty())
+            if (m_has_global_mask)
                 ret.set_global_mask(y, x, false);
             if (is_masked(yy, xx))
             {
@@ -151,25 +150,30 @@ void MaskedImage::compute_image_gradients() const
     const auto size = m_image.size();
     m_image_grady = cv::Mat(size, CV_8UC3);
     m_image_gradx = cv::Mat(size, CV_8UC3);
-    m_image_grady = cv::Scalar::all(0);
-    m_image_gradx = cv::Scalar::all(0);
 
-    for (int i = 1; i < size.height - 1; ++i)
+    // The border replicates its pixels, so that patches over the outermost line compare it as well.
+    for (int y = 0; y < size.height; ++y)
     {
-        const auto *ptry1 = m_image.ptr<unsigned char>(i + 1, 0);
-        const auto *ptry2 = m_image.ptr<unsigned char>(i - 1, 0);
-        const auto *ptrx1 = m_image.ptr<unsigned char>(i, 0) + 3;
-        const auto *ptrx2 = m_image.ptr<unsigned char>(i, 0) - 3;
-        auto *mptry = m_image_grady.ptr<unsigned char>(i, 0);
-        auto *mptrx = m_image_gradx.ptr<unsigned char>(i, 0);
-        for (int j = 3; j < size.width * 3 - 3; ++j)
+        const int up = std::max(y - 1, 0);
+        const int down = std::min(y + 1, size.height - 1);
+        for (int x = 0; x < size.width; ++x)
         {
+            const int left = std::max(x - 1, 0);
+            const int right = std::min(x + 1, size.width - 1);
             // A neutral gradient next to a globally masked pixel keeps its color out of the distances.
-            const int x = j / 3;
-            const bool skip_y = is_globally_masked(i + 1, x) || is_globally_masked(i - 1, x);
-            const bool skip_x = is_globally_masked(i, x + 1) || is_globally_masked(i, x - 1);
-            mptry[j] = skip_y ? 128 : (ptry1[j] / 2 - ptry2[j] / 2) + 128;
-            mptrx[j] = skip_x ? 128 : (ptrx1[j] / 2 - ptrx2[j] / 2) + 128;
+            const bool skip_y = is_globally_masked(down, x) || is_globally_masked(up, x);
+            const bool skip_x = is_globally_masked(y, right) || is_globally_masked(y, left);
+            const unsigned char *below = get_image(down, x);
+            const unsigned char *above = get_image(up, x);
+            const unsigned char *next = get_image(y, right);
+            const unsigned char *previous = get_image(y, left);
+            auto *grady = m_image_grady.ptr<unsigned char>(y, x);
+            auto *gradx = m_image_gradx.ptr<unsigned char>(y, x);
+            for (int c = 0; c < 3; ++c)
+            {
+                grady[c] = static_cast<unsigned char>(skip_y ? 128 : below[c] / 2 - above[c] / 2 + 128);
+                gradx[c] = static_cast<unsigned char>(skip_x ? 128 : next[c] / 2 - previous[c] / 2 + 128);
+            }
         }
     }