gridpoints 1.0.6__tar.gz → 1.0.7__tar.gz

gridpoints-1.0.7/PKG-INFO

@@ -0,0 +1,112 @@
+Metadata-Version: 2.4
+Name: gridpoints
+Version: 1.0.7
+Summary: fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid 
+Author: Armand de Cacqueray
+License: MIT
+Project-URL: Homepage, https://github.com/Neighborhood-Grid/CubeNet
+Project-URL: Repository, https://github.com/Neighborhood-Grid/CubeNet
+Keywords: sorting,point-clouds,multi-indexing,grid,neighborhood-grid,monotonic-lagrangian-grid,squarenet,nearest-neighbor
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: demo
+Requires-Dist: squarenet; extra == "demo"
+Dynamic: license-file
+
+## ❒ Gridpoints — Multi dimensional sort for point clouds
+
+`Gridpoints` maps raw indexes of unstructured **point clouds** to structured grids multi-index  through a **bijective transformation**: 
+One point, one cell, 
+```text
+n <-> [i,j,…](n).
+```
+It replaces and enhances the [squarenet](https://github.com/Neighborhood-Grid/SquareNet) project with a more robust points sorting algorithm. The goal is 
+to build a spatialy coherent multi index to later store and query the point cloud efficiently, based on adaptative axes (lines, columns, and more in 3D+) as showed on following exemple :
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/ballexemple.png">
+
+What it does: Take raw point cloud `P(N, D)`  and find a grid shape and an index permutation `order` such that `Pgrid(I, J, …, D)` = P[order].reshape(*gridshape, D) is sorted along every axis of the grid. E.g. in 3D, for Pgrid = (x, y, z):
+```text
+x[i+1, j, k] >= x[i, j, k]
+y[i, j+1, k] >= y[i, j, k]
+z[i, j, k+1] >= z[i, j, k]
+```
+
+→ On the `Pgrid` view of `P`, neighbor queries become a simple stencil look-up
+```text
+neighborhood(Pgrid[i, j, k]) = {Pgrid[i±di, j±dj, k±dk] | (di, dj, dk) ≤ R},
+````
+where `R` is a radius cutoff to determine, allowing local operations in linear time.  
+→ Geometric operations (convolution, clustering, interpolation …) can then be applied **in linear time** directly on the `Pgrid` view instead of relying on complex graph or point-cloud methods
+
+`P` can be a NumPy, PyTorch or CuPy array of any dimension (N, D). To allow natural padding when the grid has more cells than they are points in the cloud, NaNs and Infs are supported in a consistent manner: 
+- nans → random position
+- (+-) infs → border of the grid
+  
+This allow to gridsort prime or variable number of points, as long as one is ready to deal with void/special grid cells.
+
+Expected runtime for sorting 1 million points: CPU: < 10s (numpy), GPU: < 500 ms (torch cuda)
+
+`When not to use gridpoints ?`
+- high dimension: the package is implemented to support arbitrary dimension, but sweetspot is really 2D/3D. dimensions 4-6 might still be reasonable depending on the task, but anything above 8D is generally too high dimensional for gridpoints.
+- weird geometries. Supported datasets goes beyond smooth convex distributions: map of Indonesia, a sponge, a donuts, an elephant, an eggshell (by specifying a suitable gridshape e.g. (128,128,2) to `gridpoints.sort()`). You can see various exemples in the [plot](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots) folder But with some limits. Bad fits: a spider web, a wind turbine, same eggshell with a naive 3d sort (gridshape = (32,32,32)) ... the issue is not that gridpoints cannot sort these distributions, but that it will produce a poor representation of the geometry.
+- small point clouds: beyond a few hundred points, local operations in linear time is not worth the overhead, because naive quadratic implementations will probably be simultaneously simpler and faster.
+
+---
+
+### Installation
+
+```bash
+pip install gridpoints          # core only
+pip install gridpoints[demo]    # for the demonstration notebook.
+```
+Full Demo: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
+
+### Quickstart
+
+```python
+import gridpoints as grid
+import numpy as np
+
+# Raw point cloud (numpy, pytorch or cupy)
+A = np.random.rand(1_000_000, 3)
+
+# Sorted view: place the points inside the grid
+order = grid.argsort(A, gridshape=(100, 100, 100))
+Bflat = A[order] #flat but sorted with a grid layout (C-order)
+Bgrid = Bflat.reshape(100, 100, 100, 3) #reshaped as a grid 
+
+# Rest of your pipeline, working with grids
+Cgrid = apply_something(Bgrid) #(100, 100, 100, *)
+
+# Back to the original points indexing
+Cflat = Cgrid.reshape(1_000_000, *)
+orderinv = grid.invert_permutation(order)
+C = Cflat[orderinv]   # matches the initial points order
+```
+
+---
+
+### Note on rectangular gridshapes
+
+The eggshell example illustrates the importance of selecting a suitable grid-shape decomposition. This can itself be challenging when the point cloud is assumed to follow a complex topological pattern, such as a thin surface rather than a full volumetric domain. While the precise decomposition is not particularly important (e.g. (18,20,16) vs. (16,15,24) will generally make little difference), the orders of magnitude of the different dimensions do matter and should be tuned to the dataset: (18,20,16) and (36,40,4) can lead to substantially different results. As mentioned, nan/inf padding can help accomodate integer-factorization constraints
+
+Another possibility for smoothing out topology-specific effects is to perform the sort after randomly projecting the dataset onto a lower-dimensional subspace. Random projections are known to approximately preserve pairwise distances, as formalized by the Johnson–Lindenstrauss lemma.
+
+
+### Note on the cutoff radius R
+
+There is no strict theoretical guarantee about what the cutof radius R should be for a given task. E.g the relative grid position between a point and its nearest neighbors can't be garanted to be in the exact adjacent grid cells. What is guaranteed from the sorted ordering is only **grid monotonicity**: *x* coordinates increase along rows, *y* coordinates along columns, and so on.
+
+As an example, empirical results in 2-D show that `R = 5` is enough for ~99 % of the nearest neighbors; some outlier neighbors will sit further apart for complex geometries with pronounced peaks, holes or any non-smoothness. When a stricter neighborhood is required, or in high dimensional setting, the solution might be to build an assembly of grid experts, each working on a rotated / projected view of the points, as discussed in [this topic](https://github.com/glotzerlab/freud/discussions/1417). An other possibility would be to tile the grid: each tile is enhanced with location metadata (e.g. a bounding box), allowing for pruning pairs of tiles with a distance certified to be far enough for a given criterion.
+
+
+### Note on efficient stencil operations
+
+The typical use-case of `Gridpoints` is to allow fast local operations on arbitrary point clouds using stencil kernels:
+
+```text
+output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] | di, dj, dk in local window )
+```
+To go beyond standard (slow) python loops, this kind of kernel computation can be accelerated with native grid convolution operations of standard libraries whenever possible, or with `pystencils` or `taichi` compilers for complex/non linear grid kernels. On GPU, tiling the grid and leveraging a custom `triton` kernel might be particularly efficient.

gridpoints-1.0.7/README.md

@@ -0,0 +1,96 @@
+## ❒ Gridpoints — Multi dimensional sort for point clouds
+
+`Gridpoints` maps raw indexes of unstructured **point clouds** to structured grids multi-index  through a **bijective transformation**: 
+One point, one cell, 
+```text
+n <-> [i,j,…](n).
+```
+It replaces and enhances the [squarenet](https://github.com/Neighborhood-Grid/SquareNet) project with a more robust points sorting algorithm. The goal is 
+to build a spatialy coherent multi index to later store and query the point cloud efficiently, based on adaptative axes (lines, columns, and more in 3D+) as showed on following exemple :
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/ballexemple.png">
+
+What it does: Take raw point cloud `P(N, D)`  and find a grid shape and an index permutation `order` such that `Pgrid(I, J, …, D)` = P[order].reshape(*gridshape, D) is sorted along every axis of the grid. E.g. in 3D, for Pgrid = (x, y, z):
+```text
+x[i+1, j, k] >= x[i, j, k]
+y[i, j+1, k] >= y[i, j, k]
+z[i, j, k+1] >= z[i, j, k]
+```
+
+→ On the `Pgrid` view of `P`, neighbor queries become a simple stencil look-up
+```text
+neighborhood(Pgrid[i, j, k]) = {Pgrid[i±di, j±dj, k±dk] | (di, dj, dk) ≤ R},
+````
+where `R` is a radius cutoff to determine, allowing local operations in linear time.  
+→ Geometric operations (convolution, clustering, interpolation …) can then be applied **in linear time** directly on the `Pgrid` view instead of relying on complex graph or point-cloud methods
+
+`P` can be a NumPy, PyTorch or CuPy array of any dimension (N, D). To allow natural padding when the grid has more cells than they are points in the cloud, NaNs and Infs are supported in a consistent manner: 
+- nans → random position
+- (+-) infs → border of the grid
+  
+This allow to gridsort prime or variable number of points, as long as one is ready to deal with void/special grid cells.
+
+Expected runtime for sorting 1 million points: CPU: < 10s (numpy), GPU: < 500 ms (torch cuda)
+
+`When not to use gridpoints ?`
+- high dimension: the package is implemented to support arbitrary dimension, but sweetspot is really 2D/3D. dimensions 4-6 might still be reasonable depending on the task, but anything above 8D is generally too high dimensional for gridpoints.
+- weird geometries. Supported datasets goes beyond smooth convex distributions: map of Indonesia, a sponge, a donuts, an elephant, an eggshell (by specifying a suitable gridshape e.g. (128,128,2) to `gridpoints.sort()`). You can see various exemples in the [plot](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots) folder But with some limits. Bad fits: a spider web, a wind turbine, same eggshell with a naive 3d sort (gridshape = (32,32,32)) ... the issue is not that gridpoints cannot sort these distributions, but that it will produce a poor representation of the geometry.
+- small point clouds: beyond a few hundred points, local operations in linear time is not worth the overhead, because naive quadratic implementations will probably be simultaneously simpler and faster.
+
+---
+
+### Installation
+
+```bash
+pip install gridpoints          # core only
+pip install gridpoints[demo]    # for the demonstration notebook.
+```
+Full Demo: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
+
+### Quickstart
+
+```python
+import gridpoints as grid
+import numpy as np
+
+# Raw point cloud (numpy, pytorch or cupy)
+A = np.random.rand(1_000_000, 3)
+
+# Sorted view: place the points inside the grid
+order = grid.argsort(A, gridshape=(100, 100, 100))
+Bflat = A[order] #flat but sorted with a grid layout (C-order)
+Bgrid = Bflat.reshape(100, 100, 100, 3) #reshaped as a grid 
+
+# Rest of your pipeline, working with grids
+Cgrid = apply_something(Bgrid) #(100, 100, 100, *)
+
+# Back to the original points indexing
+Cflat = Cgrid.reshape(1_000_000, *)
+orderinv = grid.invert_permutation(order)
+C = Cflat[orderinv]   # matches the initial points order
+```
+
+---
+
+### Note on rectangular gridshapes
+
+The eggshell example illustrates the importance of selecting a suitable grid-shape decomposition. This can itself be challenging when the point cloud is assumed to follow a complex topological pattern, such as a thin surface rather than a full volumetric domain. While the precise decomposition is not particularly important (e.g. (18,20,16) vs. (16,15,24) will generally make little difference), the orders of magnitude of the different dimensions do matter and should be tuned to the dataset: (18,20,16) and (36,40,4) can lead to substantially different results. As mentioned, nan/inf padding can help accomodate integer-factorization constraints
+
+Another possibility for smoothing out topology-specific effects is to perform the sort after randomly projecting the dataset onto a lower-dimensional subspace. Random projections are known to approximately preserve pairwise distances, as formalized by the Johnson–Lindenstrauss lemma.
+
+
+### Note on the cutoff radius R
+
+There is no strict theoretical guarantee about what the cutof radius R should be for a given task. E.g the relative grid position between a point and its nearest neighbors can't be garanted to be in the exact adjacent grid cells. What is guaranteed from the sorted ordering is only **grid monotonicity**: *x* coordinates increase along rows, *y* coordinates along columns, and so on.
+
+As an example, empirical results in 2-D show that `R = 5` is enough for ~99 % of the nearest neighbors; some outlier neighbors will sit further apart for complex geometries with pronounced peaks, holes or any non-smoothness. When a stricter neighborhood is required, or in high dimensional setting, the solution might be to build an assembly of grid experts, each working on a rotated / projected view of the points, as discussed in [this topic](https://github.com/glotzerlab/freud/discussions/1417). An other possibility would be to tile the grid: each tile is enhanced with location metadata (e.g. a bounding box), allowing for pruning pairs of tiles with a distance certified to be far enough for a given criterion.
+
+
+### Note on efficient stencil operations
+
+The typical use-case of `Gridpoints` is to allow fast local operations on arbitrary point clouds using stencil kernels:
+
+```text
+output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] | di, dj, dk in local window )
+```
+To go beyond standard (slow) python loops, this kind of kernel computation can be accelerated with native grid convolution operations of standard libraries whenever possible, or with `pystencils` or `taichi` compilers for complex/non linear grid kernels. On GPU, tiling the grid and leveraging a custom `triton` kernel might be particularly efficient.

{gridpoints-1.0.6 → gridpoints-1.0.7}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "gridpoints"
-version = "1.0.6"
+version = "1.0.7"
 description = "fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid "
 readme = "README.md"
 requires-python = ">=3.9"

{gridpoints-1.0.6 → gridpoints-1.0.7}/src/griddemo/__init__.py

@@ -7,4 +7,4 @@ __all__ = (
     print_zoom,
 )
 
-__version__ = "1.0.6"
+__version__ = "1.0.7"

{gridpoints-1.0.6 → gridpoints-1.0.7}/src/gridpoints/array_libs.py

@@ -46,7 +46,7 @@ def inplace_sort(X, axis):
     xp, backend_name = get_backend(X)
     if backend_name == 'torch':
         X.copy_(X.sort(dim=axis)[0])
-    if backend_name == "cupy":
+    elif backend_name == "cupy":
         X[...] = xp.sort(xp.ascontiguousarray(X), axis=axis)
     else:
         X.sort(axis=axis)
@@ -75,15 +75,17 @@ def argmax(arr, axis):
         return arr.argmax(dim=axis)
     return arr.argmax(axis=axis)
 
-def random_noise(shape, reference_array):
+def random_noise(shape, reference_array, eps = None):
+    if eps is None:
+        eps = EPS
     xp, backend_name = get_backend(reference_array)
     if backend_name == 'torch':
-        return EPS * xp.rand(*shape, device=reference_array.device)
+        return eps * xp.rand(*shape, device=reference_array.device)
     elif backend_name == 'cupy':
-        return EPS * xp.random.rand(*shape)
+        return eps * xp.random.rand(*shape)
     else:
         rng = xp.random.default_rng(42)
-        return EPS * rng.random(shape)
+        return eps * rng.random(shape)
 
 def copy(arr):
     _, backend_name = get_backend(arr)
@@ -91,9 +93,15 @@ def copy(arr):
         return arr.clone()
     return arr.copy()
 
-def nan_to_num(arr):
+def nan_infs_to_num_and_tiebreaker(arr, safe = False):
     xp, backend = get_backend(arr)
     out = copy(arr)
+    eps = EPS
+    if safe: #better precision when adding epsilon noise tiebreaker 
+        #and latter when applying diagonal rotations.
+        #probably unnecessary (default safe = False)
+        out = astype(out, "float64") 
+        eps = 10**(-10)
 
     for d in range(arr.shape[1]):
         x = arr[:, d]
@@ -110,7 +118,7 @@ def nan_to_num(arr):
                    if backend == 'torch' else xp.random.randint(len(vals), size=n))
             out[nan, d] = vals[idx]
 
-    return out + random_noise(out.shape, out)
+    return out + random_noise(out.shape, out, eps = eps)
 
 def all_true(tensor):
     _, backend_name = get_backend(tensor)

{gridpoints-1.0.6 → gridpoints-1.0.7}/src/gridpoints/diagonal_tricks.py

@@ -20,14 +20,16 @@ def sort_diag_trick(X):
     rotate back the diagonals
     """
     xp, _ = get_backend(X)
-    bigM = (X.max() - X.min()) + 1#bigM for safety, protect
+    #bigM for safety, protect
     #overlapping diagonals
+    bigM = (X.max() - X.min()) + 1
 
     n, m = X.shape[-2:]
-    
-    if m > n: #Optional, maximal parallelisation
+
+    #move the smallest axis to last position
+    if m > n: 
         X = xp.moveaxis(X, [-1, -2], [-2, -1]) 
-        n, m = m, n
+        n, m = m, n #now n >= m
         
     i = arange(n, 'int', X)[:, None]
     j = arange(m, 'int', X)[None, :]
@@ -56,20 +58,16 @@ def sort_tridiag_trick(X):
     and rotate back the tridiagonals
     """
     xp, _ = get_backend(X)
-    bigM = (X.max() - X.min()) + 1#bigM for safety, protect
+    #bigM for safety, protect
     #overlapping diagonals
+    bigM = (X.max() - X.min()) + 1
 
-    a, b, c = X.shape[-3:]
-
-    #Optional, maximal parallelisation:
-    #move the smallest axis to last position
-    #before the tridiagonal rotation
-    if (a < b) and (a < c):
-        X = xp.moveaxis(X, [-3, -1], [-1, -3]) 
-        a, c = c, a
-    elif b < c:
-        X = xp.moveaxis(X, [-2, -1], [-1, -2]) 
-        b, c = c, b
+    #move the smallest axes to last positions
+    abc = X.shape
+    dst_axes = (-3, -2, -1)
+    src_axes = tuple(sorted(dst_axes, key=lambda ax: -abc[ax]))
+    X = xp.moveaxis(X, src_axes, dst_axes)   
+    a, b, c = X.shape[-3:]  # now a >= b >= c 
     
     i = arange(a, 'int', X)[:, None, None]
     j = arange(b, 'int', X)[None, :, None]
@@ -81,7 +79,7 @@ def sort_tridiag_trick(X):
     dj_ = dj % b
     dires = di // a         
     djres = dj // b
-    groupkey = bigM * (dires * (2 * (b + c)) + djres)
+    groupkey = bigM * (dires * 4 + djres)
     
     X[..., di_, dj_, k] = X[..., i, j, k] - groupkey #Trirotation
     # ========================

{gridpoints-1.0.6 → gridpoints-1.0.7}/src/gridpoints/mlg_sort.py

@@ -8,7 +8,7 @@ import math
 import warnings
 from .array_libs import (
     arange, get_backend, all_true, 
-    inplace_sort, empty, astype, nan_to_num, 
+    inplace_sort, empty, astype, nan_infs_to_num_and_tiebreaker, 
     random_permutation, argmax, copy, flip
 )
 from .diagonal_tricks import sort_diag_trick, sort_tridiag_trick
@@ -52,16 +52,25 @@ def argsort(X, gridshape, verbose = 2, level=2, init="kdtree", n_iter=40):
         monotonically sorted grid.
     """
     N, D = X.shape
-    assert math.prod(gridshape) == N
-    assert len(gridshape) == D
-    
+    assert len(gridshape) == D, (
+        f"gridshape={gridshape} has {len(gridshape)} entries but the points are {D}D: "
+        "gridshape needs exactly one specified shape per axis. "
+        "For example, with 3D points, (32, 32) is invalid (z shape is missing), "
+        "while (32, 32, 1) is valid (z size is 1)."
+    )
+    assert math.prod(gridshape) == N, (
+        f"a grid with given gridshape ={gridshape} contains {math.prod(gridshape)} cells "
+        f"but there are N={N} points. "
+        "If there are fewer points than cells, pad the points with "
+        "dummy points (e.g. NaN/inf). "
+        "If there are more points than cells, use a bigger gridshape."
+    )
     level = min(level, X.shape[-1])
+
+    if verbose >=1:
+        print(f"[gridsort] initialisation: {init}", end = "")
     if init == "kdtree":
-            if verbose >=1:
-                print("performing kdtree initialisation")
             order = kdtree_order(X, gridshape)
-            if verbose >=1:
-                print("done")
             if level == 1:
                 return order
     else:
@@ -69,6 +78,10 @@ def argsort(X, gridshape, verbose = 2, level=2, init="kdtree", n_iter=40):
     if init == "shuffle":
         order = random_permutation(order)
 
+    if verbose >=1:
+        print("done")
+    
+
     ordernew, converged = _mlg_step(X[order], gridshape, level=level, n_iter=n_iter, verbose = verbose)
     order = order[ordernew]
     
@@ -111,7 +124,7 @@ def sort(X, gridshape, verbose = 2, level=2, init="kdtree", inplace=False, n_ite
         rather than pushed to grid boundaries.
     verbose : int, default 2
         0 = silent, 1 = log progress, 2 = 1 + warn if convergence failed
-    level : {1, 2, 3}, default=1
+    level : {1, 2, 3}, default=2
         Maximum order of lattice directions considered. Higher values improve 
         grid quality at the expense of computational speed. Note that `level=3` 
         is only supported if D >= 3.
@@ -251,14 +264,15 @@ def _sort_loop(loop, directions, shape, n_iter, verbose):
         if fully_sorted:
             break
 
-    print("it:", it)
+    if verbose:
+        print("\n it:", it, "converged:", fully_sorted)
     return (cycle_to_idx[grid]).ravel(), fully_sorted
 
 def _mlg_step(X, shape, level, n_iter, verbose):
     xp, _ = get_backend(X)     
     directions_raw = xp.vstack(_direction_blocks(level=level, xp=xp, dim=X.shape[-1]))
     directions = astype(directions_raw, 'float', reference_array=X)
-    X_float = nan_to_num(astype(X, 'float'))
+    X_float = nan_infs_to_num_and_tiebreaker(astype(X, 'float'))
     
     loop = _build_monotonic_lagrangian_loop(X_float, directions)
     if verbose >=1:

gridpoints-1.0.7/src/gridpoints.egg-info/PKG-INFO

@@ -0,0 +1,112 @@
+Metadata-Version: 2.4
+Name: gridpoints
+Version: 1.0.7
+Summary: fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid 
+Author: Armand de Cacqueray
+License: MIT
+Project-URL: Homepage, https://github.com/Neighborhood-Grid/CubeNet
+Project-URL: Repository, https://github.com/Neighborhood-Grid/CubeNet
+Keywords: sorting,point-clouds,multi-indexing,grid,neighborhood-grid,monotonic-lagrangian-grid,squarenet,nearest-neighbor
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: demo
+Requires-Dist: squarenet; extra == "demo"
+Dynamic: license-file
+
+## ❒ Gridpoints — Multi dimensional sort for point clouds
+
+`Gridpoints` maps raw indexes of unstructured **point clouds** to structured grids multi-index  through a **bijective transformation**: 
+One point, one cell, 
+```text
+n <-> [i,j,…](n).
+```
+It replaces and enhances the [squarenet](https://github.com/Neighborhood-Grid/SquareNet) project with a more robust points sorting algorithm. The goal is 
+to build a spatialy coherent multi index to later store and query the point cloud efficiently, based on adaptative axes (lines, columns, and more in 3D+) as showed on following exemple :
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/ballexemple.png">
+
+What it does: Take raw point cloud `P(N, D)`  and find a grid shape and an index permutation `order` such that `Pgrid(I, J, …, D)` = P[order].reshape(*gridshape, D) is sorted along every axis of the grid. E.g. in 3D, for Pgrid = (x, y, z):
+```text
+x[i+1, j, k] >= x[i, j, k]
+y[i, j+1, k] >= y[i, j, k]
+z[i, j, k+1] >= z[i, j, k]
+```
+
+→ On the `Pgrid` view of `P`, neighbor queries become a simple stencil look-up
+```text
+neighborhood(Pgrid[i, j, k]) = {Pgrid[i±di, j±dj, k±dk] | (di, dj, dk) ≤ R},
+````
+where `R` is a radius cutoff to determine, allowing local operations in linear time.  
+→ Geometric operations (convolution, clustering, interpolation …) can then be applied **in linear time** directly on the `Pgrid` view instead of relying on complex graph or point-cloud methods
+
+`P` can be a NumPy, PyTorch or CuPy array of any dimension (N, D). To allow natural padding when the grid has more cells than they are points in the cloud, NaNs and Infs are supported in a consistent manner: 
+- nans → random position
+- (+-) infs → border of the grid
+  
+This allow to gridsort prime or variable number of points, as long as one is ready to deal with void/special grid cells.
+
+Expected runtime for sorting 1 million points: CPU: < 10s (numpy), GPU: < 500 ms (torch cuda)
+
+`When not to use gridpoints ?`
+- high dimension: the package is implemented to support arbitrary dimension, but sweetspot is really 2D/3D. dimensions 4-6 might still be reasonable depending on the task, but anything above 8D is generally too high dimensional for gridpoints.
+- weird geometries. Supported datasets goes beyond smooth convex distributions: map of Indonesia, a sponge, a donuts, an elephant, an eggshell (by specifying a suitable gridshape e.g. (128,128,2) to `gridpoints.sort()`). You can see various exemples in the [plot](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots) folder But with some limits. Bad fits: a spider web, a wind turbine, same eggshell with a naive 3d sort (gridshape = (32,32,32)) ... the issue is not that gridpoints cannot sort these distributions, but that it will produce a poor representation of the geometry.
+- small point clouds: beyond a few hundred points, local operations in linear time is not worth the overhead, because naive quadratic implementations will probably be simultaneously simpler and faster.
+
+---
+
+### Installation
+
+```bash
+pip install gridpoints          # core only
+pip install gridpoints[demo]    # for the demonstration notebook.
+```
+Full Demo: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
+
+### Quickstart
+
+```python
+import gridpoints as grid
+import numpy as np
+
+# Raw point cloud (numpy, pytorch or cupy)
+A = np.random.rand(1_000_000, 3)
+
+# Sorted view: place the points inside the grid
+order = grid.argsort(A, gridshape=(100, 100, 100))
+Bflat = A[order] #flat but sorted with a grid layout (C-order)
+Bgrid = Bflat.reshape(100, 100, 100, 3) #reshaped as a grid 
+
+# Rest of your pipeline, working with grids
+Cgrid = apply_something(Bgrid) #(100, 100, 100, *)
+
+# Back to the original points indexing
+Cflat = Cgrid.reshape(1_000_000, *)
+orderinv = grid.invert_permutation(order)
+C = Cflat[orderinv]   # matches the initial points order
+```
+
+---
+
+### Note on rectangular gridshapes
+
+The eggshell example illustrates the importance of selecting a suitable grid-shape decomposition. This can itself be challenging when the point cloud is assumed to follow a complex topological pattern, such as a thin surface rather than a full volumetric domain. While the precise decomposition is not particularly important (e.g. (18,20,16) vs. (16,15,24) will generally make little difference), the orders of magnitude of the different dimensions do matter and should be tuned to the dataset: (18,20,16) and (36,40,4) can lead to substantially different results. As mentioned, nan/inf padding can help accomodate integer-factorization constraints
+
+Another possibility for smoothing out topology-specific effects is to perform the sort after randomly projecting the dataset onto a lower-dimensional subspace. Random projections are known to approximately preserve pairwise distances, as formalized by the Johnson–Lindenstrauss lemma.
+
+
+### Note on the cutoff radius R
+
+There is no strict theoretical guarantee about what the cutof radius R should be for a given task. E.g the relative grid position between a point and its nearest neighbors can't be garanted to be in the exact adjacent grid cells. What is guaranteed from the sorted ordering is only **grid monotonicity**: *x* coordinates increase along rows, *y* coordinates along columns, and so on.
+
+As an example, empirical results in 2-D show that `R = 5` is enough for ~99 % of the nearest neighbors; some outlier neighbors will sit further apart for complex geometries with pronounced peaks, holes or any non-smoothness. When a stricter neighborhood is required, or in high dimensional setting, the solution might be to build an assembly of grid experts, each working on a rotated / projected view of the points, as discussed in [this topic](https://github.com/glotzerlab/freud/discussions/1417). An other possibility would be to tile the grid: each tile is enhanced with location metadata (e.g. a bounding box), allowing for pruning pairs of tiles with a distance certified to be far enough for a given criterion.
+
+
+### Note on efficient stencil operations
+
+The typical use-case of `Gridpoints` is to allow fast local operations on arbitrary point clouds using stencil kernels:
+
+```text
+output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] | di, dj, dk in local window )
+```
+To go beyond standard (slow) python loops, this kind of kernel computation can be accelerated with native grid convolution operations of standard libraries whenever possible, or with `pystencils` or `taichi` compilers for complex/non linear grid kernels. On GPU, tiling the grid and leveraging a custom `triton` kernel might be particularly efficient.

gridpoints-1.0.6/PKG-INFO

@@ -1,96 +0,0 @@
-Metadata-Version: 2.4
-Name: gridpoints
-Version: 1.0.6
-Summary: fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid 
-Author: Armand de Cacqueray
-License: MIT
-Project-URL: Homepage, https://github.com/Neighborhood-Grid/CubeNet
-Project-URL: Repository, https://github.com/Neighborhood-Grid/CubeNet
-Keywords: sorting,point-clouds,multi-indexing,grid,neighborhood-grid,monotonic-lagrangian-grid,squarenet,nearest-neighbor
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Provides-Extra: demo
-Requires-Dist: squarenet; extra == "demo"
-Dynamic: license-file
-
-## ❒ Gridpoints — Multi dimensional sort for point clouds
-
-`Gridpoints` maps unstructured **point clouds** to structured grids through a **bijective transformation**: one point, one cell, no overlap, fully invertible. It replaces and enhances the squarenet project with a more powerfull sorting algorithm.
-
-What it does: Take raw point cloud `P(N, D)`  and find a grid shape and an index permutation `order` such that Pgrid = P[order].reshape(*gridshape, D) 
-is sorted along every axis of the grid. E.g. in 3D, for Pgrid = (x, y, z):
-```text
-x[i+1, j, k] >= x[i, j, k]
-y[i, j+1, k] >= y[i, j, k]
-z[i, j, k+1] >= z[i, j, k]
-```
-→ On the `Pgrid` view of `P`, neighbor queries become a simple stencil look-up :
-```text
-neighborhood[i, j, k] = {Pgrid[i±di, j±dj, k±dk] | (di, dj, dk) ≤ R},
-````
-where `R` is a radius cutoff to determine, allowing local operations in linear time.  
-→ Standard operations (convolution, clustering, …) can then be applied  
-directly on the `Pgrid` view instead of relying on complex graph convolutions  
-or other point-cloud techniques.
-
-`P` can be a NumPy, PyTorch or CuPy array of any dimension (N, D).  
-To allow natural padding when the grid has more cells than points,  
-NaNs and Infs are supported in a consistent manner: 
-- nans → random position
-- (+-) infs → border of the grid
-  
-This allow to gridsort prime or variable number of points N, as long as one is ready to deal with void/special grid cells.
-
-Expected runtime for sorting 1 million points: CPU → < 10s (numpy), GPU → < 500 ms (torch cuda)
-
----
-
-### Installation
-
-```bash
-pip install gridpoints          # core only
-pip install gridpoints[demo]    # for the demonstration notebook, see `notebook.ipynb`
-```
-
-### Quickstart
-
-```python
-import gridpoints as grid
-import numpy as np
-
-# Raw point cloud (numpy, pytorch or cupy)
-A = np.random.rand(1_000_000, 3)
-
-# Sorted view: place the points inside the grid
-order = grid.argsort(A, gridshape=(100, 100, 100))
-Bflat = A[order]
-Bgrid = Bflat.reshape(100, 100, 100, 3)
-
-# Rest of your pipeline, working with grids
-Cgrid = apply_something(Bgrid)
-
-# Back to the original points indexing
-Cflat = Cgrid.reshape(-1, 3)
-orderinv = grid.invert_permutation(order)
-C = Cflat[orderinv]   # matches the initial points order
-```
-
-<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/ballexemple.png">
-
----
-
-### Note on the cutoff radius R
-
-There is no strict theoretical guarantee about what the cutof radius R should be for a given task. E.g the relative grid position between a point and its nearest neighbors can't be garanted to be in the exact adjacent grid cells. What is guaranteed from the sorted ordering is only **grid monotonicity**: *x* coordinates increase along rows, *y* coordinates along columns, and so on.
-
-As an example, empirical results in 2-D show that `R = 5` is enough for ~99 % of the nearest neighbors; some outlier neighbors will sit further apart for complex geometries with pronounced peaks, holes or any non-smoothness. When a stricter neighborhood is required, or in high dimensional setting, the best practice is to build an assembly of grid experts, each working on a rotated / projected view of the points, as discussed in [this topic](https://github.com/glotzerlab/freud/discussions/1417).
-
-### Note on efficient stencil operations
-
-The typical use-case of `Gridpoints` is to allow fast local operations on arbitrary point clouds using stencil kernels:
-
-```text
-output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] | di, dj, dk in local window )
-```
-To go beyond standard (slow) python loops, this can be accelerated with native grid convolution operations of standard libraries whenever possible, or with `pystencils` or `taichi` compilers for complex/non linear grid kernels.

gridpoints-1.0.6/README.md

@@ -1,80 +0,0 @@
-## ❒ Gridpoints — Multi dimensional sort for point clouds
-
-`Gridpoints` maps unstructured **point clouds** to structured grids through a **bijective transformation**: one point, one cell, no overlap, fully invertible. It replaces and enhances the squarenet project with a more powerfull sorting algorithm.
-
-What it does: Take raw point cloud `P(N, D)`  and find a grid shape and an index permutation `order` such that Pgrid = P[order].reshape(*gridshape, D) 
-is sorted along every axis of the grid. E.g. in 3D, for Pgrid = (x, y, z):
-```text
-x[i+1, j, k] >= x[i, j, k]
-y[i, j+1, k] >= y[i, j, k]
-z[i, j, k+1] >= z[i, j, k]
-```
-→ On the `Pgrid` view of `P`, neighbor queries become a simple stencil look-up :
-```text
-neighborhood[i, j, k] = {Pgrid[i±di, j±dj, k±dk] | (di, dj, dk) ≤ R},
-````
-where `R` is a radius cutoff to determine, allowing local operations in linear time.  
-→ Standard operations (convolution, clustering, …) can then be applied  
-directly on the `Pgrid` view instead of relying on complex graph convolutions  
-or other point-cloud techniques.
-
-`P` can be a NumPy, PyTorch or CuPy array of any dimension (N, D).  
-To allow natural padding when the grid has more cells than points,  
-NaNs and Infs are supported in a consistent manner: 
-- nans → random position
-- (+-) infs → border of the grid
-  
-This allow to gridsort prime or variable number of points N, as long as one is ready to deal with void/special grid cells.
-
-Expected runtime for sorting 1 million points: CPU → < 10s (numpy), GPU → < 500 ms (torch cuda)
-
----
-
-### Installation
-
-```bash
-pip install gridpoints          # core only
-pip install gridpoints[demo]    # for the demonstration notebook, see `notebook.ipynb`
-```
-
-### Quickstart
-
-```python
-import gridpoints as grid
-import numpy as np
-
-# Raw point cloud (numpy, pytorch or cupy)
-A = np.random.rand(1_000_000, 3)
-
-# Sorted view: place the points inside the grid
-order = grid.argsort(A, gridshape=(100, 100, 100))
-Bflat = A[order]
-Bgrid = Bflat.reshape(100, 100, 100, 3)
-
-# Rest of your pipeline, working with grids
-Cgrid = apply_something(Bgrid)
-
-# Back to the original points indexing
-Cflat = Cgrid.reshape(-1, 3)
-orderinv = grid.invert_permutation(order)
-C = Cflat[orderinv]   # matches the initial points order
-```
-
-<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/ballexemple.png">
-
----
-
-### Note on the cutoff radius R
-
-There is no strict theoretical guarantee about what the cutof radius R should be for a given task. E.g the relative grid position between a point and its nearest neighbors can't be garanted to be in the exact adjacent grid cells. What is guaranteed from the sorted ordering is only **grid monotonicity**: *x* coordinates increase along rows, *y* coordinates along columns, and so on.
-
-As an example, empirical results in 2-D show that `R = 5` is enough for ~99 % of the nearest neighbors; some outlier neighbors will sit further apart for complex geometries with pronounced peaks, holes or any non-smoothness. When a stricter neighborhood is required, or in high dimensional setting, the best practice is to build an assembly of grid experts, each working on a rotated / projected view of the points, as discussed in [this topic](https://github.com/glotzerlab/freud/discussions/1417).
-
-### Note on efficient stencil operations
-
-The typical use-case of `Gridpoints` is to allow fast local operations on arbitrary point clouds using stencil kernels:
-
-```text
-output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] | di, dj, dk in local window )
-```
-To go beyond standard (slow) python loops, this can be accelerated with native grid convolution operations of standard libraries whenever possible, or with `pystencils` or `taichi` compilers for complex/non linear grid kernels.

gridpoints-1.0.6/src/gridpoints.egg-info/PKG-INFO

@@ -1,96 +0,0 @@
-Metadata-Version: 2.4
-Name: gridpoints
-Version: 1.0.6
-Summary: fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid 
-Author: Armand de Cacqueray
-License: MIT
-Project-URL: Homepage, https://github.com/Neighborhood-Grid/CubeNet
-Project-URL: Repository, https://github.com/Neighborhood-Grid/CubeNet
-Keywords: sorting,point-clouds,multi-indexing,grid,neighborhood-grid,monotonic-lagrangian-grid,squarenet,nearest-neighbor
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Provides-Extra: demo
-Requires-Dist: squarenet; extra == "demo"
-Dynamic: license-file
-
-## ❒ Gridpoints — Multi dimensional sort for point clouds
-
-`Gridpoints` maps unstructured **point clouds** to structured grids through a **bijective transformation**: one point, one cell, no overlap, fully invertible. It replaces and enhances the squarenet project with a more powerfull sorting algorithm.
-
-What it does: Take raw point cloud `P(N, D)`  and find a grid shape and an index permutation `order` such that Pgrid = P[order].reshape(*gridshape, D) 
-is sorted along every axis of the grid. E.g. in 3D, for Pgrid = (x, y, z):
-```text
-x[i+1, j, k] >= x[i, j, k]
-y[i, j+1, k] >= y[i, j, k]
-z[i, j, k+1] >= z[i, j, k]
-```
-→ On the `Pgrid` view of `P`, neighbor queries become a simple stencil look-up :
-```text
-neighborhood[i, j, k] = {Pgrid[i±di, j±dj, k±dk] | (di, dj, dk) ≤ R},
-````
-where `R` is a radius cutoff to determine, allowing local operations in linear time.  
-→ Standard operations (convolution, clustering, …) can then be applied  
-directly on the `Pgrid` view instead of relying on complex graph convolutions  
-or other point-cloud techniques.
-
-`P` can be a NumPy, PyTorch or CuPy array of any dimension (N, D).  
-To allow natural padding when the grid has more cells than points,  
-NaNs and Infs are supported in a consistent manner: 
-- nans → random position
-- (+-) infs → border of the grid
-  
-This allow to gridsort prime or variable number of points N, as long as one is ready to deal with void/special grid cells.
-
-Expected runtime for sorting 1 million points: CPU → < 10s (numpy), GPU → < 500 ms (torch cuda)
-
----
-
-### Installation
-
-```bash
-pip install gridpoints          # core only
-pip install gridpoints[demo]    # for the demonstration notebook, see `notebook.ipynb`
-```
-
-### Quickstart
-
-```python
-import gridpoints as grid
-import numpy as np
-
-# Raw point cloud (numpy, pytorch or cupy)
-A = np.random.rand(1_000_000, 3)
-
-# Sorted view: place the points inside the grid
-order = grid.argsort(A, gridshape=(100, 100, 100))
-Bflat = A[order]
-Bgrid = Bflat.reshape(100, 100, 100, 3)
-
-# Rest of your pipeline, working with grids
-Cgrid = apply_something(Bgrid)
-
-# Back to the original points indexing
-Cflat = Cgrid.reshape(-1, 3)
-orderinv = grid.invert_permutation(order)
-C = Cflat[orderinv]   # matches the initial points order
-```
-
-<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/ballexemple.png">
-
----
-
-### Note on the cutoff radius R
-
-There is no strict theoretical guarantee about what the cutof radius R should be for a given task. E.g the relative grid position between a point and its nearest neighbors can't be garanted to be in the exact adjacent grid cells. What is guaranteed from the sorted ordering is only **grid monotonicity**: *x* coordinates increase along rows, *y* coordinates along columns, and so on.
-
-As an example, empirical results in 2-D show that `R = 5` is enough for ~99 % of the nearest neighbors; some outlier neighbors will sit further apart for complex geometries with pronounced peaks, holes or any non-smoothness. When a stricter neighborhood is required, or in high dimensional setting, the best practice is to build an assembly of grid experts, each working on a rotated / projected view of the points, as discussed in [this topic](https://github.com/glotzerlab/freud/discussions/1417).
-
-### Note on efficient stencil operations
-
-The typical use-case of `Gridpoints` is to allow fast local operations on arbitrary point clouds using stencil kernels:
-
-```text
-output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] | di, dj, dk in local window )
-```
-To go beyond standard (slow) python loops, this can be accelerated with native grid convolution operations of standard libraries whenever possible, or with `pystencils` or `taichi` compilers for complex/non linear grid kernels.