OpenPIV 0.25.4__tar.gz → 0.26.0__tar.gz

{openpiv-0.25.4 → openpiv-0.26.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: OpenPIV
-Version: 0.25.4
+Version: 0.26.0
 Summary: OpenPIV consists in a Python and Cython modules for scripting and executing the analysis of a set of PIV image pairs. In addition, a Qt and Tk graphical user interfaces are in development, to ease the use for those users who don't have python skills.
 License: GPLv3
 License-File: LICENSE.txt
@@ -22,10 +22,12 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Scientific/Engineering
+Provides-Extra: rust
 Requires-Dist: imageio (>=2.35.0)
 Requires-Dist: matplotlib (>=3.8.0)
 Requires-Dist: natsort (>=8.4.0)
 Requires-Dist: numpy (>=2.0.0)
+Requires-Dist: openpiv-rust (>=0.1.0) ; extra == "rust"
 Requires-Dist: scikit-image (>=0.23.0)
 Requires-Dist: scipy (>=1.11.0)
 Requires-Dist: tqdm (>=4.66.0)

{openpiv-0.25.4 → openpiv-0.26.0}/openpiv/docs/conf.py

@@ -64,7 +64,7 @@ copyright = '2014, OpenPIV team'
 # built documents.
 #
 # The short X.Y version.
-version = '0.25.4'
+version = '0.26.0'
 # The full version, including alpha/beta/rc tags.
 # release = '0.0.1a'
 

{openpiv-0.25.4 → openpiv-0.26.0}/openpiv/docs/index.rst

@@ -31,6 +31,7 @@ Contents:
    src/windef
    src/masking
    src/developers
+   src/fft_correlation_backends
    src/api_reference
    src/faq
 

{openpiv-0.25.4 → openpiv-0.26.0}/openpiv/docs/src/developers.rst

@@ -63,3 +63,64 @@ If you need to install cv2::
     conda install -c conda-forge opencv
 
 
+Developing with or without Rust
+--------------------------------
+
+OpenPIV supports dual computational backends: **pure Python / SciPy** (default) and **compiled Rust** (via ``openpiv_rust``).
+
+Option A: Developing in Pure Python (without Rust)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+You do not need a Rust compiler installed. Simply install OpenPIV in editable mode:
+
+.. code-block:: bash
+
+    poetry install
+    # or: pip install -e .
+
+All core functionality, single-pass, and multi-pass deformation will use the optimized
+``scipy.fft`` backend automatically.
+
+To run the test suite:
+
+.. code-block:: bash
+
+    pytest openpiv/test
+
+Option B: Developing with the Rust Extension
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you have the Rust toolchain (``cargo``) installed:
+
+1. Install ``maturin``:
+
+   .. code-block:: bash
+
+       pip install maturin
+
+2. Compile and link ``openpiv_rust`` directly into your Python environment:
+
+   .. code-block:: bash
+
+       cd crates/openpiv_rust
+       maturin develop --release
+       cd ../..
+
+3. Verify that the Rust backend is active:
+
+   .. code-block:: bash
+
+       python -c "import openpiv_rust; print('Rust backend ready!')"
+
+Running the Performance Benchmarks
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+OpenPIV includes a dedicated benchmark suite comparing cross-correlation, subpixel
+peak finding, and end-to-end multi-pass Windef across backends:
+
+.. code-block:: bash
+
+    python benchmarks/run_benchmarks.py
+
+
+

openpiv-0.26.0/openpiv/docs/src/fft_correlation_backends.rst

@@ -0,0 +1,162 @@
+FFT Correlation Backends & Performance Study
+============================================
+
+This document details the comparative study of 2D cross-correlation algorithms,
+backends, and performance characteristics in OpenPIV, focusing on both **Circular**
+and **Linear** correlation, `scipy.fft` (PocketFFT), and the `openpiv_rust` compiled backend.
+
+.. contents:: Table of Contents
+   :depth: 2
+   :local:
+
+Overview
+--------
+
+In Particle Image Velocimetry (PIV), cross-correlation between interrogation windows
+is the core computational step. For an image pair split into :math:`N` interrogation
+windows of size :math:`(H \times W)`, the correlation maps are computed using the
+Wiener-Khinchin theorem via Fast Fourier Transforms:
+
+.. math::
+
+   C = \mathcal{F}^{-1} \left( \mathcal{F}(I_B) \cdot \mathcal{F}^*(I_A) \right)
+
+OpenPIV supports two primary correlation paradigms:
+
+1. **Circular Correlation (Standard OpenPIV)**:
+   Assumes periodic boundary conditions (toroidal wraparound).
+   Inputs of size :math:`(N \times N)` yield correlation maps of size :math:`(N \times N)`.
+   No zero-padding is required.
+
+2. **Linear Correlation (Full / Extended)**:
+   Evaluates true spatial shift without periodic wraparound.
+   For windows of size :math:`s_1` and :math:`s_2`, the full linear correlation has size
+   :math:`(s_1 + s_2 - 1)`. To prevent time-domain aliasing, inputs must be zero-padded
+   to at least this size before applying the FFT.
+
+
+The 63x63 vs 64x64 Power-of-2 Finding
+-------------------------------------
+
+In previous versions of OpenPIV, linear correlation in ``openpiv.pyprocess.fft_correlate_images``
+computed the padded transform size as:
+
+.. code-block:: python
+
+   size = s1 + s2 - 1                                   # e.g., 32 + 32 - 1 = 63
+   fsize = 2 ** np.ceil(np.log2(size)).astype(int) - 1  # 64 - 1 = 63 (BUG!)
+
+Why 63 Disabled PocketFFT Optimization
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+SciPy delegates FFT operations to **PocketFFT** (written in C++ by Martin Reinecke),
+which includes hand-tuned AVX2/FMA vector kernels. However, these SIMD routines rely on
+smooth radix factors (powers of 2: 2, 4, 8, 16, 32, 64).
+
+1. Subtracting 1 produced an **odd composite size** :math:`63 = 3 \times 3 \times 7`.
+2. This forced PocketFFT into slow Bluestein/composite transforms, disabling SIMD vectorization.
+3. Furthermore, padding to 63 left an off-by-one boundary alignment when slicing the central window.
+
+The Power-of-2 Solution
+^^^^^^^^^^^^^^^^^^^^^^^
+
+By keeping the clean power-of-2 transform size :math:`fsize = 2^{\lceil \log_2(size) \rceil}`
+(e.g., :math:`64 \times 64` for :math:`32 \times 32` windows), PocketFFT achieves optimal
+SIMD execution speed.
+
+To recover the exact central correlation window matching ``scipy.signal.correlate``
+to machine precision (:math:`\le 10^{-13}`), the centered slice is:
+
+.. code-block:: python
+
+   # Exact power of 2 transform size
+   fsize = 2 ** np.ceil(np.log2(size)).astype(int)
+
+   # Centered slice extracting (s1) around the zero-displacement lag
+   fslice = (
+       slice(0, image_a.shape[0]),
+       slice(fsize[0] // 2 - s1[0] // 2, fsize[0] // 2 - s1[0] // 2 + s1[0]),
+       slice(fsize[1] // 2 - s1[1] // 2, fsize[1] // 2 - s1[1] // 2 + s1[1]),
+   )
+   f2a = conj(rfft2(image_a, fsize, axes=(-2, -1), workers=workers))
+   f2b = rfft2(image_b, fsize, axes=(-2, -1), workers=workers)
+   corr = fftshift(irfft2(f2a * f2b, axes=(-2, -1)).real, axes=(-2, -1))[fslice]
+
+
+Batched Multi-Threading in scipy.fft
+------------------------------------
+
+``scipy.fft`` supports a ``workers`` parameter (e.g., ``workers=-1`` for all logical cores).
+However, profiling reveals:
+
+* On large 2D or 3D volumes, PocketFFT multi-threading scales well.
+* On **batches of tiny 2D windows** (:math:`16 \times 16` or :math:`32 \times 32`),
+  thread synchronization overhead inside PocketFFT often negates multi-core gains
+  (scaling from :math:`1.0\times` to :math:`1.1\times`), because thread splitting occurs
+  per-transform rather than distributing independent windows across workers.
+
+
+The Rust Acceleration Engine (openpiv_rust)
+-------------------------------------------
+
+To overcome Python GIL overhead and achieve linear CPU scaling across window batches,
+OpenPIV provides an optional compiled Rust extension (``openpiv_rust``) built with
+PyO3, Rayon, and RealFFT.
+
+Key Architectural Optimizations in Rust
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+1. **Zero-Allocation Rayon Scratch Pool**:
+   Window transforms use ``par_chunks_exact_mut().for_each_init(...)``, pre-allocating
+   frequency buffers once per worker thread. For 961 windows, this eliminates over
+   8,600 heap allocations per pass.
+
+2. **Native Power-of-2 Real FFT**:
+   Uses ``realfft`` (Real-to-Complex forward and Complex-to-Real inverse), cutting
+   arithmetic and memory bandwidth by 50% compared to full complex transforms.
+
+3. **Batched Subpixel Peak Finding**:
+   In addition to cross-correlation, `openpiv_rust` provides
+   ``batch_correlation_to_displacement``, executing Gaussian/Centroid/Parabolic subpixel
+   peak fitting directly across all windows in parallel, yielding a **28x to 65x speedup**
+   over the nested Python loop.
+
+
+Benchmark Results
+-----------------
+
+Linear Correlation (225 windows of 32x32)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+============================================  ====================  ==================
+Implementation                                Runtime               Relative Speed
+============================================  ====================  ==================
+``scipy.signal.correlate`` (Python loop)      70.63 ms              1.0x (baseline)
+OpenPIV SciPy (legacy ``fsize=63``)           49.28 ms              1.4x
+OpenPIV SciPy (power-of-2 ``fsize=64``)       39.10 ms              1.8x
+**openpiv_rust** (Rayon + realfft)            **8.26 ms**           **8.5x**
+============================================  ====================  ==================
+
+Displacement Calculation (Peak Finding)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+============================================  ====================  ==================
+Grid / Windows                                Python Loop           openpiv_rust
+============================================  ====================  ==================
+225 windows (32x32)                           5.19 ms               **0.18 ms (28x)**
+961 windows (16x16)                           19.84 ms              **0.30 ms (65x)**
+============================================  ====================  ==================
+
+End-to-End Multi-Pass Windef (3 passes on 256x256 image)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* **SciPy backend**: 110.59 ms
+* **Rust backend (with subpixel Rust engine)**: **81.66 ms** (:math:`1.35\times` speedup)
+
+Conclusion & Usage Guidelines
+-----------------------------
+
+* For systems with compiled binary extensions available, set ``settings.backend = 'rust'``
+  to maximize throughput in both single-pass and multi-pass deformation workflows.
+* On pure Python / NumPy / SciPy environments, using the exact power-of-2 padding rule
+  ensures optimal PocketFFT vectorization while maintaining exact numerical consistency.

{openpiv-0.25.4 → openpiv-0.26.0}/openpiv/lib.py

@@ -1,4 +1,5 @@
 import numpy as np
+from numpy.lib.stride_tricks import sliding_window_view
 
 
 def replace_nans(array, max_iter, tol, kernel_size=2, method="disk"):
@@ -92,56 +93,39 @@ def replace_nans(array, max_iter, tol, kernel_size=2, method="disk"):
     n_nans = len(nan_indices)
 
     # arrays which contain replaced values to check for convergence
-    replaced_new = np.zeros(n_nans)
     replaced_old = np.zeros(n_nans)
 
+    # broadcastable view of the kernel weights, one axis per array dimension
+    # plus one matching axis per kernel dimension (for the windowed reduction)
+    kernel_b = kernel.reshape((1,) * n_dim + kernel.shape)
+    window_axes = tuple(range(n_dim, 2 * n_dim))
+
     # make several passes
     # until we reach convergence
     for _ in range(max_iter):
         # note: identifying new nan indices and looping other the new indices
         # would give slightly different result
 
-        # for each NaN element
-        for k in range(n_nans):
-            ind = nan_indices[
-                k
-            ]  # 2 or 3 indices indicating the position of a nan element
-            # init to 0.0
-            replaced_new[k] = 0.0
-
-            # generating a list of indices of the convolution window in the
-            # array
-            slice_indices = np.array(np.meshgrid(*[range(i - kernel_size,
-                                     i + kernel_size + 1) for i in ind]))
-
-            # identifying all indices strictly inside the image edges:
-            in_mask = np.array(
-                [
-                    np.logical_and(
-                        slice_indices[i] < array.shape[i],
-                        slice_indices[i] >= 0
-                    )
-                    for i in range(n_dim)
-                ]
-            )
-            # logical and over x,y (and z) indices
-            in_mask = np.prod(in_mask, axis=0).astype(bool)
-
-            # extract window from array
-            win = filled[tuple(slice_indices[:, in_mask])]
-
-            # selecting the same points from the kernel
-            kernel_in = kernel[in_mask]
-
-            # sum of elements of the kernel that are not nan in the window
-            non_nan = np.sum(kernel_in[~np.isnan(win)])
-
-            if non_nan > 0:
-                # convolution with the kernel
-                replaced_new[k] = np.nansum(win * kernel_in) / non_nan
-            else:
-                # don't do anything if there is only nans around
-                replaced_new[k] = np.nan
+        # NaN-pad by kernel_size so that out-of-bounds neighbours behave like
+        # the boundary check in the original per-element loop (i.e. they are
+        # excluded from both the weighted sum and its normalization).
+        padded = np.pad(np.asarray(filled), kernel_size, mode="constant",
+                         constant_values=np.nan)
+        # windows.shape == array.shape + kernel.shape: one (2*kernel_size+1)^n_dim
+        # neighbourhood per array element, vectorized instead of a per-NaN
+        # Python loop with a fresh np.meshgrid call each time.
+        windows = sliding_window_view(padded, kernel.shape)
+
+        valid = ~np.isnan(windows)
+        weights = np.where(valid, kernel_b, 0)
+        non_nan = weights.sum(axis=window_axes)
+        weighted_sum = (np.where(valid, windows, 0) * kernel_b).sum(axis=window_axes)
+
+        # convolution with the kernel; stays NaN where only NaNs are around
+        full_new = np.divide(weighted_sum, non_nan,
+                              out=np.full(array.shape, np.nan), where=non_nan > 0)
+
+        replaced_new = full_new[tuple(nan_indices.T)]
 
         # bulk replace all new values in array
         filled[tuple(nan_indices.T)] = replaced_new