PyPatchMatch 2.2.0b2__tar.gz → 2.2.0rc1__tar.gz

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/CLAUDE.md

@@ -134,4 +134,6 @@ CMake installs the shared library into the package directory.
   `invoke-ai/PyPatchMatch`. `gh` resolves the default repository to upstream, so pass
   `-R mauwii/PyPatchMatch`.
 - Releases: creating a GitHub release with a tag like `v2.0.0` (or `v2.0.0rc2` for
-  pre-releases) builds the wheels and publishes them to PyPI.
+  pre-releases) builds the wheels and publishes them to PyPI. The workflow `Fill
+  quality` compares the release with the last stable one; read its summary before
+  approving the job `publish`.

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: PyPatchMatch
-Version: 2.2.0b2
+Version: 2.2.0rc1
 Summary: This library implements the PatchMatch based inpainting algorithm.
 Keywords: inpainting,patchmatch,image-processing,opencv
 Author-Email: The InvokeAI Project <lincoln.stein@gmail.com>, Jiayuan Mao <maojiayuan@gmail.com>, Kyle Schouviller <kyle0654@hotmail.com>, Matthias Wild <mauwii@outlook.de>, Younesse ANDAM <younesse.andam@gmail.com>
@@ -30,13 +30,18 @@ Description-Content-Type: text/markdown
 
 # PatchMatch based Inpainting
 
-[![Quality Gate](https://sonarcloud.io/api/project_badges/measure?project=mauwii_PyPatchMatch&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=mauwii_PyPatchMatch)
-[![License: MIT](https://img.shields.io/badge/License-MIT-blueviolet.svg)](https://github.com/mauwii/PyPatchMatch/blob/main/LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/PyPatchMatch)](https://pypi.org/project/PyPatchMatch/)
-[![Downloads](https://static.pepy.tech/badge/pypatchmatch)](https://pepy.tech/projects/pypatchmatch)
+[![Python](https://img.shields.io/pypi/pyversions/PyPatchMatch)](https://pypi.org/project/PyPatchMatch/)
+[![Downloads](https://img.shields.io/pepy/dt/pypatchmatch)](https://pepy.tech/projects/pypatchmatch)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blueviolet.svg)](https://github.com/mauwii/PyPatchMatch/blob/main/LICENSE)
+
+[![Tests](https://img.shields.io/github/actions/workflow/status/mauwii/PyPatchMatch/ci.yml?branch=main&label=tests)](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml?query=branch%3Amain)
+[![Coverage](https://img.shields.io/sonar/coverage/mauwii_PyPatchMatch?server=https%3A%2F%2Fsonarcloud.io)](https://sonarcloud.io/component_measures?id=mauwii_PyPatchMatch&metric=coverage)
+[![Quality Gate](https://img.shields.io/sonar/quality_gate/mauwii_PyPatchMatch?server=https%3A%2F%2Fsonarcloud.io)](https://sonarcloud.io/summary/new_code?id=mauwii_PyPatchMatch)
+
+[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
 [![ponytail](https://img.shields.io/badge/over--engineering-ponytail_audited-111111)](https://github.com/DietrichGebert/ponytail)
-[![caveman](https://img.shields.io/badge/agent-caveman-F0A63C)](https://github.com/JuliusBrussee/caveman)
 
 This library implements the PatchMatch based inpainting algorithm. It provides both C++
 and Python interfaces. This implementation is heavily based on the implementation by
@@ -46,7 +51,7 @@ with some bug fixes, and updates. This repository continues
 the forks [lkwq007/PyPatchMatch](https://github.com/lkwq007/PyPatchMatch) and
 [invoke-ai/PyPatchMatch](https://github.com/invoke-ai/PyPatchMatch).
 
-![Birdhouses removed from a photo by patchmatch.inpaint](https://raw.githubusercontent.com/mauwii/PyPatchMatch/v2.2.0b2/examples/images/readme_example.jpg)
+![Birdhouses removed from a photo by patchmatch.inpaint](https://raw.githubusercontent.com/mauwii/PyPatchMatch/v2.2.0rc1/examples/images/readme_example.jpg)
 
 From left to right: the photo, the input of
 [examples/py_example.py](https://github.com/mauwii/PyPatchMatch/blob/main/examples/py_example.py)

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/README.md

@@ -1,12 +1,17 @@
 # PatchMatch based Inpainting
 
-[![Quality Gate](https://sonarcloud.io/api/project_badges/measure?project=mauwii_PyPatchMatch&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=mauwii_PyPatchMatch)
-[![License: MIT](https://img.shields.io/badge/License-MIT-blueviolet.svg)](https://github.com/mauwii/PyPatchMatch/blob/main/LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/PyPatchMatch)](https://pypi.org/project/PyPatchMatch/)
-[![Downloads](https://static.pepy.tech/badge/pypatchmatch)](https://pepy.tech/projects/pypatchmatch)
+[![Python](https://img.shields.io/pypi/pyversions/PyPatchMatch)](https://pypi.org/project/PyPatchMatch/)
+[![Downloads](https://img.shields.io/pepy/dt/pypatchmatch)](https://pepy.tech/projects/pypatchmatch)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blueviolet.svg)](https://github.com/mauwii/PyPatchMatch/blob/main/LICENSE)
+
+[![Tests](https://img.shields.io/github/actions/workflow/status/mauwii/PyPatchMatch/ci.yml?branch=main&label=tests)](https://github.com/mauwii/PyPatchMatch/actions/workflows/ci.yml?query=branch%3Amain)
+[![Coverage](https://img.shields.io/sonar/coverage/mauwii_PyPatchMatch?server=https%3A%2F%2Fsonarcloud.io)](https://sonarcloud.io/component_measures?id=mauwii_PyPatchMatch&metric=coverage)
+[![Quality Gate](https://img.shields.io/sonar/quality_gate/mauwii_PyPatchMatch?server=https%3A%2F%2Fsonarcloud.io)](https://sonarcloud.io/summary/new_code?id=mauwii_PyPatchMatch)
+
+[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
 [![ponytail](https://img.shields.io/badge/over--engineering-ponytail_audited-111111)](https://github.com/DietrichGebert/ponytail)
-[![caveman](https://img.shields.io/badge/agent-caveman-F0A63C)](https://github.com/JuliusBrussee/caveman)
 
 This library implements the PatchMatch based inpainting algorithm. It provides both C++
 and Python interfaces. This implementation is heavily based on the implementation by

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/patchmatch/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '2.2.0b2'
-__version_tuple__ = version_tuple = (2, 2, 0, 'b2')
+__version__ = version = '2.2.0rc1'
+__version_tuple__ = version_tuple = (2, 2, 0, 'rc1')
 
-__commit_id__ = commit_id = 'ge960c214c'
+__commit_id__ = commit_id = 'ge40b76e89'

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/patchmatch/csrc/inpaint.cpp

@@ -369,18 +369,15 @@ namespace
 
     // Replaces the vote of the pixel if the new one weighs more. It is stored like the
     // sums of _weighted_copy, so the maximization step divides it by its weight.
-    inline void _best_copy(const MaskedImage &source, int ys, int xs, cv::Mat &target, int yt, int xt, double weight)
+    inline void _best_copy(
+        const unsigned char *color, const cv::Vec3d &shift, cv::Mat &target, int yt, int xt, double weight)
     {
-        if (source.is_masked(ys, xs))
-            return;
-
-        auto source_ptr = source.get_image(ys, xs);
         auto target_ptr = target.ptr<double>(yt, xt);
         if (weight <= target_ptr[3])
             return;
 
         for (int c = 0; c < 3; ++c)
-            target_ptr[c] = static_cast<double>(source_ptr[c]) * weight;
+            target_ptr[c] = (static_cast<double>(color[c]) + shift[c]) * weight;
         target_ptr[3] = weight;
     }
 
@@ -451,6 +448,7 @@ namespace
             const int xp = m_nnf->at(i, j, 1);
             if (m_holes && !votes_for_hole(m_source2target ? yp : i, m_source2target ? xp : j, patch_size))
                 return;
+            const cv::Vec3d shift = m_holes ? color_shift(i, j, yp, xp, patch_size) : cv::Vec3d();
             for (int di = -patch_size; di <= patch_size; ++di)
             {
                 for (int dj = -patch_size; dj <= patch_size; ++dj)
@@ -467,12 +465,47 @@ namespace
                         continue;
                     // Near ties go to the patch centered closest, not the first cast, which made blocks.
                     const double offset2 = static_cast<double>(di) * di + static_cast<double>(dj) * dj;
-                    cast_pair(ys, xs, yt, xt, weight, weight * (1 - 1e-5 * offset2));
+                    cast_pair(ys, xs, yt, xt, weight, weight * (1 - 1e-5 * offset2), shift);
                 }
             }
         }
 
     private:
+        // The mean color difference of the voted image to the voting source over the pixels known in both patches.
+        // The votes of the finest levels add it: their sources lie elsewhere in the image, and with their colors
+        // the fill of a vignetted photo came out lighter than the colors the coarser levels had laid out.
+        cv::Vec3d color_shift(int i, int j, int yp, int xp, int patch_size) const
+        {
+            const MaskedImage &source = m_nnf->source();
+            const MaskedImage &target = m_nnf->target();
+            cv::Vec3d sum;
+            int count = 0;
+            for (int di = -patch_size; di <= patch_size; ++di)
+            {
+                for (int dj = -patch_size; dj <= patch_size; ++dj)
+                {
+                    const int ys = i + di;
+                    const int xs = j + dj;
+                    const int yt = yp + di;
+                    const int xt = xp + dj;
+                    if (ys < 0 || ys >= m_source_size.height || xs < 0 || xs >= m_source_size.width || yt < 0 ||
+                        yt >= m_target_size.height || xt < 0 || xt >= m_target_size.width)
+                        continue;
+                    if (!is_known(source, ys, xs) || !is_known(target, yt, xt))
+                        continue;
+                    const auto *s = source.get_image(ys, xs);
+                    const auto *t = target.get_image(yt, xt);
+                    for (int c = 0; c < 3; ++c)
+                        sum[c] += s[c] - t[c];
+                    ++count;
+                }
+            }
+            if (count == 0)
+                return {};
+            // The voted image is the source of the coherence field and the target of the completeness field.
+            return sum * ((m_source2target ? -1.0 : 1.0) / count);
+        }
+
         // Whether the patch around (y, x) of the voted image, scaled like the votes, has a hole.
         bool votes_for_hole(int y, int x, int patch_size) const
         {
@@ -482,7 +515,7 @@ namespace
                 scale * (std::int64_t{y} + patch_size + 1) - 1, scale * (std::int64_t{x} + patch_size + 1) - 1);
         }
 
-        void cast_pair(int ys, int xs, int yt, int xt, double weight, double best_weight) const
+        void cast_pair(int ys, int xs, int yt, int xt, double weight, double best_weight, const cv::Vec3d &shift) const
         {
             if (!m_source2target)
             {
@@ -491,27 +524,30 @@ namespace
             }
             if (!m_upscaled)
             {
-                copy(ys, xs, yt, xt, weight, best_weight);
+                copy(ys, xs, yt, xt, weight, best_weight, shift);
                 return;
             }
             for (int uy = 0; uy < 2; ++uy)
                 for (int ux = 0; ux < 2; ++ux)
-                    copy(2 * ys + uy, 2 * xs + ux, 2 * yt + uy, 2 * xt + ux, weight, best_weight);
+                    copy(2 * ys + uy, 2 * xs + ux, 2 * yt + uy, 2 * xt + ux, weight, best_weight, shift);
         }
 
-        void copy(int ys, int xs, int yt, int xt, double weight, double best_weight) const
+        void copy(int ys, int xs, int yt, int xt, double weight, double best_weight, const cv::Vec3d &shift) const
         {
             if (m_best_only)
-                _best_copy(*m_source, ys, xs, *m_vote, yt, xt, best_weight);
+            {
+                if (!m_source->is_masked(ys, xs))
+                    _best_copy(m_source->get_image(ys, xs), shift, *m_vote, yt, xt, best_weight);
+            }
             else if (m_split)
-                split_copy(ys, xs, yt, xt, weight, best_weight);
+                split_copy(ys, xs, yt, xt, weight, best_weight, shift);
             else
                 _weighted_copy(*m_source, ys, xs, *m_vote, yt, xt, weight);
         }
 
-        // Adds the low frequencies and the texture of the source pixel to the sums, and keeps the rest if the
-        // vote weighs most.
-        void split_copy(int ys, int xs, int yt, int xt, double weight, double best_weight) const
+        // Adds the shifted low frequencies and the texture of the source pixel to the sums, and keeps the rest if
+        // the vote weighs most.
+        void split_copy(int ys, int xs, int yt, int xt, double weight, double best_weight, const cv::Vec3d &shift) const
         {
             if (!is_known(*m_source, ys, xs))
                 return;
@@ -520,7 +556,7 @@ namespace
             const auto *low = m_low.ptr<unsigned char>(ys, xs);
             auto *vote = m_vote->ptr<double>(yt, xt);
             for (int c = 0; c < 3; ++c)
-                vote[c] += low[c] * weight;
+                vote[c] += (low[c] + shift[c]) * weight;
             vote[3] += weight;
             vote[8] += m_source->texture().at<unsigned char>(ys, xs) * weight;
             if (best_weight <= vote[7])

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/patchmatch/csrc/masked_image.cpp

@@ -152,16 +152,15 @@ MaskedImage MaskedImage::upsample(int new_w, int new_h, const cv::Mat &new_globa
     return ret;
 }
 
-void MaskedImage::compute_image_gradients() const
+void MaskedImage::compute_features() const
 {
-    if (m_image_grad_computed)
+    if (m_features_computed)
     {
         return;
     }
 
     const auto size = m_image.size();
-    m_image_grady = cv::Mat(size, CV_8UC3);
-    m_image_gradx = cv::Mat(size, CV_8UC3);
+    m_features = cv::Mat(size, CV_8UC(kFeatureChannels));
 
     // The border replicates its pixels, so that patches over the outermost line compare it as well.
     for (int y = 0; y < size.height; ++y)
@@ -179,15 +178,20 @@ void MaskedImage::compute_image_gradients() const
             const unsigned char *above = get_image(up, x);
             const unsigned char *next = get_image(y, right);
             const unsigned char *previous = get_image(y, left);
-            auto *grady = m_image_grady.ptr<unsigned char>(y, x);
-            auto *gradx = m_image_gradx.ptr<unsigned char>(y, x);
+            auto *features = m_features.ptr<unsigned char>(y, x);
+            std::fill_n(features, kFeatureChannels, 0);
+            const unsigned char *color = get_image(y, x);
             for (int c = 0; c < 3; ++c)
             {
-                grady[c] = static_cast<unsigned char>(skip_y ? 128 : below[c] / 2 - above[c] / 2 + 128);
-                gradx[c] = static_cast<unsigned char>(skip_x ? 128 : next[c] / 2 - previous[c] / 2 + 128);
+                features[c] = color[c];
+                features[3 + c] = static_cast<unsigned char>(skip_x ? 128 : next[c] / 2 - previous[c] / 2 + 128);
+                features[6 + c] = static_cast<unsigned char>(skip_y ? 128 : below[c] / 2 - above[c] / 2 + 128);
             }
+            if (m_has_texture)
+                features[kTextureFeature] = m_texture.at<unsigned char>(y, x);
+            features[kMaskedFeature] = static_cast<unsigned char>(is_masked(y, x) || is_globally_masked(y, x));
         }
     }
 
-    m_image_grad_computed = true;
+    m_features_computed = true;
 }

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/patchmatch/csrc/masked_image.h

@@ -52,15 +52,16 @@ public:
     {
         return m_has_global_mask;
     }
-    const cv::Mat &grady() const
+    // Per pixel the colors, the gradients along x and y, the texture and whether it is masked, packed so that
+    // a distance reads one place per pixel.
+    static constexpr int kFeatureChannels = 12;
+    static constexpr int kColorAndGradientFeatures = 9;
+    static constexpr int kTextureFeature = 9;
+    static constexpr int kMaskedFeature = 10;
+    const cv::Mat &features() const
     {
-        assert(m_image_grad_computed);
-        return m_image_grady;
-    }
-    const cv::Mat &gradx() const
-    {
-        assert(m_image_grad_computed);
-        return m_image_gradx;
+        assert(m_features_computed);
+        return m_features;
     }
 
     // The texture energy that the distances compare (CV_8UC1, see Inpainting), or empty.
@@ -77,6 +78,7 @@ public:
     {
         m_texture = texture;
         m_has_texture = !texture.empty();
+        m_features_computed = false;
     }
 
     void init_global_mask_mat()
@@ -85,12 +87,14 @@ public:
         m_global_mask.setTo(cv::Scalar(0));
         m_has_global_mask = !m_global_mask.empty();
         m_hole_sums_computed = false;
+        m_features_computed = false;
     }
     void set_global_mask_mat(const cv::Mat &other)
     {
         m_global_mask = other;
         m_has_global_mask = !other.empty();
         m_hole_sums_computed = false;
+        m_features_computed = false;
     }
 
     bool is_masked(int y, int x) const
@@ -105,16 +109,19 @@ public:
     {
         m_mask.at<unsigned char>(y, x) = static_cast<unsigned char>(value);
         m_hole_sums_computed = false;
+        m_features_computed = false;
     }
     void set_global_mask(int y, int x, bool value)
     {
         m_global_mask.at<unsigned char>(y, x) = static_cast<unsigned char>(value);
         m_hole_sums_computed = false;
+        m_features_computed = false;
     }
     void clear_mask()
     {
         m_mask.setTo(cv::Scalar(0));
         m_hole_sums_computed = false;
+        m_features_computed = false;
     }
 
     const unsigned char *get_image(int y, int x) const
@@ -130,7 +137,7 @@ public:
     MaskedImage downsample() const;
     MaskedImage upsample(int new_w, int new_h) const;
     MaskedImage upsample(int new_w, int new_h, const cv::Mat &new_global_mask) const;
-    void compute_image_gradients() const;
+    void compute_features() const;
 
 private:
     cv::Mat m_image;
@@ -140,10 +147,9 @@ private:
     // cv::Mat::empty() is not inline in OpenCV 5, too slow to call for every pixel
     bool m_has_global_mask = false;
     bool m_has_texture = false;
-    // computed on first use by compute_image_gradients
-    mutable cv::Mat m_image_grady;
-    mutable cv::Mat m_image_gradx;
-    mutable bool m_image_grad_computed = false;
+    // computed on first use by compute_features
+    mutable cv::Mat m_features;
+    mutable bool m_features_computed = false;
     // computed on first use by contains_mask: the masked pixels outside the global mask
     // above and left of each pixel. Copies share them with the masks, so change a mask only
     // through an image that nothing has copied yet, as downsample and upsample do.

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/patchmatch/csrc/nnf.cpp

@@ -3,6 +3,7 @@
 #include <algorithm>
 #include <cassert>
 #include <cmath>
+#include <cstddef>
 #include <cstdint>
 #include <random>
 #include <typeinfo>
@@ -68,66 +69,44 @@ namespace
         bool textured;
     };
 
-    // One row of a source and a target image, both inside their images. The gradients must be computed.
+    // One row of a source and a target image, both inside their images. The features must be computed.
     class RowPair
     {
     public:
         RowPair(const ImagePair &images, int ys, int yt)
             : m_source_width(images.source_size.width), m_target_width(images.target_size.width),
-              m_si(images.source->image().ptr<unsigned char>(ys, 0)),
-              m_ti(images.target->image().ptr<unsigned char>(yt, 0)),
-              m_sm(images.source->mask().ptr<unsigned char>(ys, 0)),
-              m_tm(images.target->mask().ptr<unsigned char>(yt, 0)),
-              m_sgm(
-                  images.source->has_global_mask() ? images.source->global_mask().ptr<unsigned char>(ys, 0) : nullptr),
-              m_tgm(
-                  images.target->has_global_mask() ? images.target->global_mask().ptr<unsigned char>(yt, 0) : nullptr),
-              m_sgy(images.source->grady().ptr<unsigned char>(ys, 0)),
-              m_tgy(images.target->grady().ptr<unsigned char>(yt, 0)),
-              m_sgx(images.source->gradx().ptr<unsigned char>(ys, 0)),
-              m_tgx(images.target->gradx().ptr<unsigned char>(yt, 0)),
-              m_st(images.textured ? images.source->texture().ptr<unsigned char>(ys) : nullptr),
-              m_tt(images.textured ? images.target->texture().ptr<unsigned char>(yt) : nullptr)
+              m_source(images.source->features().ptr<unsigned char>(ys)),
+              m_target(images.target->features().ptr<unsigned char>(yt)), m_textured(images.textured)
         {
         }
 
         // The SSD over the colors and gradients, kSSDScale where a pixel is masked or outside its image.
         int cost(int xs, int xt) const
         {
-            // The bounds first: the masks are read only inside the image.
-            if (xs < 0 || xs >= m_source_width || xt < 0 || xt >= m_target_width || m_sm[xs] || m_tm[xt] ||
-                (m_sgm && m_sgm[xs]) || (m_tgm && m_tgm[xt]))
+            // The bounds first: the features are read only inside the image.
+            if (xs < 0 || xs >= m_source_width || xt < 0 || xt >= m_target_width)
+                return PatchSSDDistanceMetric::kSSDScale;
+            const unsigned char *s = m_source + static_cast<std::ptrdiff_t>(xs) * MaskedImage::kFeatureChannels;
+            const unsigned char *t = m_target + static_cast<std::ptrdiff_t>(xt) * MaskedImage::kFeatureChannels;
+            if (s[MaskedImage::kMaskedFeature] || t[MaskedImage::kMaskedFeature])
                 return PatchSSDDistanceMetric::kSSDScale;
 
             int ssd = 0;
-            for (int c = 0; c < 3; ++c)
-            {
-                ssd += pow2(m_si[xs * 3 + c] - m_ti[xt * 3 + c]);
-                ssd += pow2(m_sgx[xs * 3 + c] - m_tgx[xt * 3 + c]);
-                ssd += pow2(m_sgy[xs * 3 + c] - m_tgy[xt * 3 + c]);
-            }
+            for (int k = 0; k < MaskedImage::kColorAndGradientFeatures; ++k)
+                ssd += pow2(s[k] - t[k]);
             // Smooth fills matched smooth sources and stayed flat, so a target pixel with less than half the
             // texture of its source pixel costs extra. Not a two-sided difference: it removed mortar joints.
-            if (m_st)
-                ssd += 2 * pow2(std::max(m_st[xs] / 2 - m_tt[xt], 0));
+            if (m_textured)
+                ssd += 2 * pow2(std::max(s[MaskedImage::kTextureFeature] / 2 - t[MaskedImage::kTextureFeature], 0));
             return ssd;
         }
 
     private:
         int m_source_width;
         int m_target_width;
-        const unsigned char *m_si;
-        const unsigned char *m_ti;
-        const unsigned char *m_sm;
-        const unsigned char *m_tm;
-        const unsigned char *m_sgm;
-        const unsigned char *m_tgm;
-        const unsigned char *m_sgy;
-        const unsigned char *m_tgy;
-        const unsigned char *m_sgx;
-        const unsigned char *m_tgx;
-        const unsigned char *m_st; // the textures, nullptr unless both images have one
-        const unsigned char *m_tt;
+        const unsigned char *m_source;
+        const unsigned char *m_target;
+        bool m_textured; // false unless both images have a texture
     };
 
     // The costs of the row of the patches around s and t.
@@ -168,8 +147,8 @@ namespace
         const double limit =
             (bound + 1.0) * PatchSSDDistanceMetric::kSSDScale * side * side / PatchDistanceMetric::kDistanceScale;
 
-        source.compute_image_gradients();
-        target.compute_image_gradients();
+        source.compute_features();
+        target.compute_features();
 
         const ImagePair images(source, target);
         for (int dy = -patch_size; dy <= patch_size; ++dy)

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/pyproject.toml

@@ -60,6 +60,7 @@ metadata.readme.provider = "scikit_build_core.metadata.fancy_pypi_readme"
 # The release wheels are built from the sdist, so it has to contain the tests
 # and their input images; _version.py is generated and therefore git-ignored.
 sdist.include = [
+    "examples/images/camera_aged.png",
     "examples/images/camera_damaged.png",
     "examples/images/camera_damaged_mask.png",
     "examples/images/chelsea.png",

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/scripts/evaluate_inpainting.py

@@ -19,12 +19,15 @@ Usage:
     uv run python scripts/evaluate_inpainting.py --baseline main/libpatchmatch.dylib
     uv run python scripts/evaluate_inpainting.py --patch-size 15 --seeds 1 --cases brick
 
+The inpaintings run in parallel processes, one per CPU unless --jobs says otherwise.
+They slow each other down, so time a speed-up with --jobs 1.
+
 --markdown prints the table as Markdown and moves the plain one to stderr; the CI
 workflow "Fill quality" redirects it to the job summary.
 
 Besides the table, it writes examples/images/evaluation.html (git-ignored): the holes,
-the fills (of --baseline too) and the originals side by side. It embeds the images of
-examples/images/local/ as well, so keep it out of the repository.
+the fills (of --baseline too) and the originals side by side, cut to the holes. It
+embeds the images of examples/images/local/ as well, so keep it out of the repository.
 
 The e2e tests use measure() to check the seam and the detail of forest_pruned.bmp.
 
@@ -34,7 +37,15 @@ Columns, means over the seeds:
     detail  color difference of neighbors inside the holes, relative to their
             surroundings; 0.85 to 1.1 in the original images, lower is blurrier
     error   mean color error against the original on images reduced to 1/8, which
-            compares the structure of the fill rather than its pixels
+            compares the structure of the fill rather than its pixels; it rates a flat
+            fill better than a texture whose structures are shifted
+    edges   difference of the gradient directions from the original, in histograms
+            weighted by magnitude per 32x32 cell of the holes; lowest for the right
+            texture, also when shifted, highest for a flat fill
+    variety difference of the 8x8 patches of the holes, without their mean color, to
+            the closest one that does not overlap them, relative to the original;
+            lower where the fill repeats its sources or blurs (the tiles of
+            grass-large: about 0.25), higher for fragments
     fills   with --baseline: whether the fills of all seeds are byte-identical to the
             baseline's, as they have to be after a change that only makes it faster
 """
@@ -43,18 +54,20 @@ from __future__ import annotations
 
 import argparse
 import base64
-import ctypes
+import functools
 import hashlib
 import html
 import io
 import sys
 import time
 from collections.abc import Iterator
+from concurrent.futures import Future, ProcessPoolExecutor
 from datetime import datetime
 from pathlib import Path
 from typing import NamedTuple, TextIO
 
 import numpy as np
+from numpy.lib.stride_tricks import sliding_window_view
 from PIL import Image
 
 import patchmatch
@@ -73,7 +86,7 @@ HOLES = {
     "chelsea": (52, 215, 24, 40),
     "pattern": (320, 320, 40, 40),
 }
-COLUMNS = ("seam", "detail", "error")
+COLUMNS = ("seam", "detail", "error", "edges", "variety")
 
 
 class Case(NamedTuple):
@@ -182,6 +195,55 @@ def coarse_error(image: np.ndarray, truth: np.ndarray, holes: np.ndarray) -> flo
     return float(np.abs(reduce(image)[cells] - reduce(truth)[cells]).mean())
 
 
+EDGE_CELL = 32
+EDGE_BINS = 8
+
+
+def edge_error(image: np.ndarray, truth: np.ndarray, holes: np.ndarray) -> float:
+    """Difference of the gradient directions, weighted by magnitude, per cell of holes."""
+    ys, xs = np.nonzero(holes)
+    cells = (ys // EDGE_CELL) * (holes.shape[1] // EDGE_CELL + 1) + xs // EDGE_CELL
+    length = (int(cells.max()) + 1) * EDGE_BINS
+
+    def histograms(array: np.ndarray) -> np.ndarray:
+        gy, gx = np.gradient(array.astype(float).mean(axis=2))
+        bins = (np.arctan2(gy, gx) % np.pi * EDGE_BINS / np.pi).astype(int) % EDGE_BINS
+        weights = np.hypot(gx, gy)[holes]
+        return np.bincount(cells * EDGE_BINS + bins[holes], weights, length)
+
+    return float(np.abs(histograms(image) - histograms(truth)).sum() / len(ys))
+
+
+PATCH = 8
+
+
+def patches(image: np.ndarray, mask: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
+    """The PATCHxPATCH patches inside ``mask`` on a grid, flattened, and their corners."""
+    grid = (slice(None, None, PATCH // 2),) * 2
+    windows = sliding_window_view(image, (PATCH, PATCH), axis=(0, 1))[grid]
+    inside = sliding_window_view(mask, (PATCH, PATCH))[grid].all(axis=(2, 3))
+    flat = windows[inside].reshape(int(inside.sum()), 3, -1).astype(np.float32)
+    # without their mean color, which the votes shift for repeated sources too
+    flat -= flat.mean(axis=2, keepdims=True)
+    return flat.reshape(len(flat), -1), np.argwhere(inside) * (PATCH // 2)
+
+
+def variety(image: np.ndarray, truth: np.ndarray, holes: np.ndarray) -> float:
+    def mean(array: np.ndarray) -> float:
+        """Mean distance of the patches to the closest one that does not overlap them."""
+        found, corners = patches(array, holes)
+        norms = (found**2).sum(axis=1)
+        closest = []
+        for start in range(0, len(found), 512):  # bounds the memory of the distances
+            chunk, where = found[start : start + 512], corners[start : start + 512]
+            distances = norms[start : start + 512, None] + norms - 2 * chunk @ found.T
+            distances[np.abs(where[:, None] - corners).max(axis=2) < PATCH] = np.inf
+            closest.append(distances.min(axis=1))
+        return float(np.sqrt(np.maximum(np.concatenate(closest), 0)).mean())
+
+    return mean(image) / mean(truth)
+
+
 def measure(
     result: np.ndarray, holes: np.ndarray, truth: np.ndarray | None = None
 ) -> dict[str, float]:
@@ -202,26 +264,40 @@ def measure(
     }
     if truth is not None:
         values["error"] = coarse_error(result, truth, holes)
+        values["edges"] = edge_error(result, truth, holes)
+        # holes thinner than a patch, such as scratches, have none to compare
+        if sliding_window_view(holes, (PATCH, PATCH)).all(axis=(2, 3)).any():
+            values["variety"] = variety(result, truth, holes)
     return values
 
 
-def evaluate(library: ctypes.CDLL, case: Case, patch_size: int, seeds: int) -> Result:
-    patch_match._lib = library
-    mask = case.holes.astype(np.uint8)
-    rows, fills = [], []
+Seed = tuple[np.ndarray, dict[str, float], float]  # fill, measures, seconds
+
+library = functools.cache(_lib.load_library)
+
+
+def fill_seed(path: str, case: Case, patch_size: int, seed: int) -> Seed:
+    """One inpainting, in a worker process."""
+    # processes, not threads: the seed and patch_match._lib are global
+    patch_match._lib = library(Path(path))
+    patchmatch.set_random_seed(seed)
     start = time.perf_counter()
-    for seed in range(seeds):
-        patchmatch.set_random_seed(seed)
-        fills.append(
-            patchmatch.inpaint(
-                case.image, mask, global_mask=case.global_mask, patch_size=patch_size
-            )
-        )
-        rows.append(measure(fills[-1], case.holes, case.truth))
-    seconds = (time.perf_counter() - start) / seeds
+    fill = patchmatch.inpaint(
+        case.image,
+        case.holes.astype(np.uint8),
+        global_mask=case.global_mask,
+        patch_size=patch_size,
+    )
+    seconds = time.perf_counter() - start
+    return fill, measure(fill, case.holes, case.truth), seconds
+
+
+def evaluate(jobs: list[Future[Seed]]) -> Result:
+    """The result of one case and library from the jobs of all seeds, in order."""
+    fills, rows, seconds = zip(*(job.result() for job in jobs), strict=True)
     values = {key: float(np.mean([row[key] for row in rows])) for key in rows[0]}
     digest = hashlib.sha256(b"".join(fill.tobytes() for fill in fills)).hexdigest()
-    return Result(values, seconds, fills[0], digest)
+    return Result(values, float(np.mean(seconds)), fills[0], digest)
 
 
 def identical(results: list[Result]) -> str:
@@ -286,19 +362,13 @@ figure { margin: 0; cursor: zoom-in; }
 figcaption { font-size: 13px; color: var(--muted); margin-top: 4px;
   font-variant-numeric: tabular-nums; }
 figcaption b { color: var(--fg); font-weight: 600; margin-right: 6px; }
-.frame { position: relative; overflow: hidden; width: 100%;
-  aspect-ratio: var(--crop); background: var(--line); }
-.frame img { position: absolute; display: block; max-width: none; height: auto;
-  left: var(--left); top: var(--top); width: var(--width); }
-body.whole .frame { aspect-ratio: var(--full); }
-body.whole .frame img { left: 0; top: 0; width: 100%; }
+figure img { display: block; width: 100%; height: auto; }
 body.pixels img { image-rendering: pixelated; }
 #viewer { position: fixed; inset: 0; z-index: 10; display: flex;
   flex-direction: column; align-items: center; justify-content: center; gap: 10px;
   background: rgb(0 0 0 / 0.9); cursor: zoom-out; }
 #viewer[hidden] { display: none; }
-#viewer .frame { width: min(94vw, calc(86vh * var(--crop))); cursor: pointer; }
-body.whole #viewer .frame { width: min(94vw, calc(86vh * var(--full))); }
+#viewer img { width: 94vw; height: 86vh; object-fit: contain; cursor: pointer; }
 #viewer p { color: #ddd; font-size: 14px; margin: 0; }
 """
 
@@ -310,7 +380,7 @@ function show(i) {
   index = (i + figures.length) % figures.length;
   const figure = figures[index];
   viewer.querySelector(".slot").replaceChildren(
-    figure.querySelector(".frame").cloneNode(true));
+    figure.querySelector("img").cloneNode());
   viewer.querySelector("p").textContent = figure.dataset.title
     + "  (" + (index + 1) + "/" + figures.length + ")";
 }
@@ -322,7 +392,7 @@ for (const figure of document.querySelectorAll("figure")) {
   });
 }
 viewer.addEventListener("click", (event) => {
-  if (event.target.closest(".frame")) show(index + 1);
+  if (event.target.closest("img")) show(index + 1);
   else viewer.hidden = true;
 });
 document.addEventListener("keydown", (event) => {
@@ -333,12 +403,10 @@ document.addEventListener("keydown", (event) => {
   else return;
   event.preventDefault();
 });
-for (const name of ["whole", "pixels"]) {
-  const box = document.getElementById(name);
-  const apply = () => document.body.classList.toggle(name, box.checked);
-  box.addEventListener("change", apply);
-  apply();  // browsers restore the checkboxes on reload
-}
+const pixels = document.getElementById("pixels");
+const apply = () => document.body.classList.toggle("pixels", pixels.checked);
+pixels.addEventListener("change", apply);
+apply();  // browsers restore the checkbox on reload
 """
 
 
@@ -362,23 +430,11 @@ def crop_box(holes: np.ndarray) -> tuple[int, int, int, int]:
     return int(x0), int(y0), int(x1 - x0), int(y1 - y0)
 
 
-def frame(image: np.ndarray, box: tuple[int, int, int, int]) -> str:
-    """The image, cut to ``box`` by CSS unless the whole images are shown."""
-    height, width = image.shape[:2]
-    x, y, w, h = box
-    style = (
-        f"--crop:{w / h:.5f};--full:{width / height:.5f};--width:{100 * width / w:.4f}%;"
-        f"--left:{-100 * x / w:.4f}%;--top:{-100 * y / h:.4f}%"
-    )
-    source = f"data:image/png;base64,{png(image)}"
-    return f'<div class="frame" style="{style}"><img alt="" src="{source}"></div>'
-
-
 def verdict(key: str, old: float, new: float) -> str:
     """CSS class of ``new`` compared with ``old``: better, worse or none."""
     if abs(new - old) <= (0.1 if key == "time" else 0.02) * abs(old):
         return ""
-    if key in ("seam", "detail"):  # best at 1.0, as in the original images
+    if key in ("seam", "detail", "variety"):  # best at 1.0, as in the original images
         old, new = abs(old - 1), abs(new - 1)
     return "better" if new < old else "worse"
 
@@ -412,7 +468,7 @@ def caption(label: str, result: Result | None) -> str:
 
 
 def section(case: Case, results: list[Result], labels: list[str]) -> str:
-    box = crop_box(case.holes)
+    x, y, w, h = crop_box(case.holes)
     shown = case.image.copy()
     if case.global_mask is not None:
         shown[case.global_mask] //= 2
@@ -422,7 +478,8 @@ def section(case: Case, results: list[Result], labels: list[str]) -> str:
         versions.append(("original", case.truth, None))
     figures = "".join(
         f'<figure data-title="{html.escape(case.name)}: {label}">'
-        f"{frame(image, box)}<figcaption>{caption(label, result)}</figcaption></figure>"
+        f'<img alt="" src="data:image/png;base64,{png(image[y : y + h, x : x + w])}">'
+        f"<figcaption>{caption(label, result)}</figcaption></figure>"
         for label, image, result in versions
     )
     height, width = case.holes.shape
@@ -456,7 +513,6 @@ def write_report(
             f"<title>patchmatch evaluation</title><style>{STYLE}</style></head><body>"
             f"<main><header><h1>patchmatch evaluation</h1>{meta}</header>"
             '<div class="controls">'
-            '<label><input type="checkbox" id="whole"> whole images</label>'
             '<label><input type="checkbox" id="pixels"> show pixels</label>'
             '<p class="hint">Click an image to enlarge it; click it, &larr; or &rarr; '
             "switches between the versions, Esc closes.</p></div>"
@@ -498,9 +554,9 @@ def markdown(rows: list[tuple[Case, list[Result]]], title: str) -> str:
         ]
     lines += [
         "",
-        "Means over the seeds. seam and detail are best at 1.0, as in the original "
-        "images, error at 0. With a baseline: baseline → current, 🟢 better and 🔴 worse "
-        "by more than 2 % (time: 10 %).",
+        "Means over the seeds. seam, detail and variety are best at 1.0, as in the "
+        "original images, error and edges at 0. With a baseline: baseline → current, "
+        "🟢 better and 🔴 worse by more than 2 % (time: 10 %).",
     ]
     return "\n".join(lines)
 
@@ -521,6 +577,7 @@ def main() -> None:
     parser.add_argument("--patch-size", type=int, default=3)
     parser.add_argument("--seeds", type=int, default=3)
     parser.add_argument("--cases", help="comma-separated names, default: all")
+    parser.add_argument("--jobs", type=int, help="processes, default: one per CPU")
     parser.add_argument(
         "--markdown", action="store_true", help="print the table as Markdown"
     )
@@ -537,24 +594,38 @@ def main() -> None:
     if args.baseline:
         libraries.insert(0, _lib.load_library(args.baseline.resolve()))
         labels = ["baseline", "current"]
+    paths = [lib._name for lib in libraries]
     selected = set(args.cases.split(",")) if args.cases else None
+    chosen = [case for case in cases() if not selected or case.name in selected]
 
     out = sys.stderr if args.markdown else sys.stdout
     print(f"patch_size={args.patch_size} seeds={args.seeds}", file=out)
     fills = "fills" if args.baseline else ""
-    header = f"{'case':14} {'seam':16}{'detail':16}{'error':16}{'time':16}{fills}"
+    header = f"{'case':14} {''.join(f'{key:16}' for key in COLUMNS)}{'time':16}{fills}"
     print(header.rstrip(), file=out)
     rows = []
-    for case in cases():
-        if selected and case.name not in selected:
-            continue
-        results = [
-            evaluate(lib, case, args.patch_size, args.seeds) for lib in libraries
+    pool = ProcessPoolExecutor(args.jobs)
+    try:
+        # all at once, the seeds outside, so that the libraries run under equal load
+        jobs = [
+            {
+                (index, seed): pool.submit(fill_seed, path, case, args.patch_size, seed)
+                for seed in range(args.seeds)
+                for index, path in enumerate(paths)
+            }
+            for case in chosen
         ]
-        print_row(case.name, results, out)
-        rows.append((case, results))
-
-    meta = describe(args, labels, [lib._name for lib in libraries])
+        for case, futures in zip(chosen, jobs, strict=True):
+            results = [
+                evaluate([futures[index, seed] for seed in range(args.seeds)])
+                for index in range(len(paths))
+            ]
+            print_row(case.name, results, out)
+            rows.append((case, results))
+    finally:
+        pool.shutdown(cancel_futures=True)  # Ctrl+C stops after the running jobs
+
+    meta = describe(args, labels, paths)
     write_report(rows, labels, meta)
     print(f"report: {REPORT}", file=out)
     if args.markdown:

{pypatchmatch-2.2.0b2 → pypatchmatch-2.2.0rc1}/tests/test_e2e.py

@@ -137,13 +137,21 @@ def test_global_mask_is_not_used_as_source(pruned_image):
 
 
 def test_restore_aged_photo():
-    """Scratches, holes and a torn corner, light but not white, given as a mask."""
+    """Scratches, holes and a torn corner, light but not white, given as a mask.
+
+    The finest levels averaged the colors of sources from elsewhere in the vignetted
+    photo, so the torn corner came out lighter (error 4.6 to 6.1 with ten seeds).
+    """
     source = Image.open(IMAGES / "camera_damaged.png")
     mask = Image.open(IMAGES / "camera_damaged_mask.png")
+    holes = np.array(mask) > 0
 
     result = patchmatch.inpaint(source, mask, patch_size=3)
 
-    assert_plausible_fill(np.array(source), result, holes=np.array(mask) > 0)
+    assert_plausible_fill(np.array(source), result, holes=holes)
+    truth = evaluation.load(IMAGES / "camera_aged.png")
+    error = evaluation.coarse_error(result, truth, holes)
+    assert error < 4.2, error
 
 
 @pytest.mark.parametrize(