gridpoints 1.0.7__tar.gz → 1.1.1__tar.gz

gridpoints-1.1.1/PKG-INFO

@@ -0,0 +1,223 @@
+Metadata-Version: 2.4
+Name: gridpoints
+Version: 1.1.1
+Summary: fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid 
+Author: Armand de Cacqueray
+License: MIT
+Project-URL: Homepage, https://neighborhood-grid.github.io/gridpoints/
+Project-URL: Repository, https://github.com/Neighborhood-Grid/gridpoints
+Keywords: sorting,point-clouds,multi-indexing,grid,neighborhood-grid,monotonic-lagrangian-grid,squarenet,nearest-neighbor
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: demo
+Requires-Dist: squarenet; extra == "demo"
+Dynamic: license-file
+
+# Gridpoints
+
+[![PyPI](https://img.shields.io/pypi/v/gridpoints?logo=pypi&logoColor=white)](https://pypi.org/project/gridpoints/)
+[![GitHub](https://img.shields.io/badge/GitHub-gridpoints-181717?logo=github)](https://github.com/Neighborhood-Grid/gridpoints)
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/table(1).gif">
+
+**Reorder an unstructured point cloud into a regular grid, so that neighbor-based operations become simple grid processing.**
+
+```python
+import gridpoints as grid
+points = your_dataset_of_points(1000000, 3)  # standard (N, D) array
+order = grid.argsort(points, gridshape=(100, 100, 100))  # set gridshape: (I, J, K, ...) = (100, 100, 100)
+Pgrid = points[order].reshape(100, 100, 100, 3)  # now an (I, J, K, ...,  D) version of your dataset
+# Ready for convolution, neighbor look-ups or geometrical analysis.
+# Note that grid.argsort also supports a batch mode (B, N, D) -> (B, I, J, K, ..., D)
+```
+
+Gridpoints supports NumPy, PyTorch and CuPy arrays.
+The `argsort()` reordering is a one-to-one assignment of your points to the cells of a grid with the given `gridshape`. This is related to Monge optimal transport, but `argsort()` trades optimality for scalability, by combining greedy axial operations which exploit the specific structure of grids. One million 2D/3D points are sorted in about 0.5 s on a GPU (PyTorch) or 10 s on a CPU (NumPy).
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/ballexemple.png">
+
+---
+
+## TL;DR
+
+- **Problem:** point clouds are stored as a flat list `[P1, ..., PN]`. Finding the neighbors of a point usually needs a tree or a graph (KD-tree, radius graph...), which is hard to use with standard tensor-based machine learning, especially on GPU.
+- **Idea:** assign every point to a cell of a D-dimensional grid, one point per cell, such that the grid is sorted along each of its axes. Then the neighbors of a point are *approximately* the points in nearby cells, found by plain indexing.
+- **Payoff:** local operations (convolution, smoothing, clustering, interpolation...) that are usually hard to implement on irregular point clouds can be applied on the grid: `points -> grid`. To recover the result on the points, one just applies the inverse assignment: `grid -> points`.
+- **Catch:** a plain grid gives *approximate* neighborhoods (about 99% of true nearest neighbors within a small radius). Exact neighborhoods are recoverable with tiling (see [Neighborhood radius](#neighborhood-radius)). It works best in 2D/3D, and stays reasonable up to about 6D.
+
+---
+
+## The problem
+
+Points are almost always stored in a one-dimensional order. You would like points that are close in space to also be close in memory, so that the nearest neighbor of `P[n]` is often `P[n±1]`, `P[n±2]`, or a bit further. This is a well-studied problem, and space-filling curves such as the Morton curve are a common solution.
+
+Gridpoints takes a different route. Instead of a 1D order, it gives each point a **multi-index** `[i, j, ...]` in a D-dimensional grid. This matters because a grid supports stencil operations ("look at the cells within a few steps in each direction", `[i±di, j±dj, k±dk]`).
+
+## What Gridpoints does
+
+Input: a point cloud `P` of shape `(N, D)` and a target grid shape `(I, J, ...)`.
+
+Output: a permutation `order` of the indices such that
+
+```python
+Pgrid = P[order].reshape(I, J, ..., D)
+```
+
+is a bijection between points and grid cells (one point, one cell), and is **sorted along every axis**. In 3D, with xyz notation `P = (x, y, z)`:
+
+```text
+x[i+1, j, k] >= x[i, j, k]     # x increases along axis 0
+y[i, j+1, k] >= y[i, j, k]     # y increases along axis 1
+z[i, j, k+1] >= z[i, j, k]     # z increases along axis 2
+```
+
+This property (*grid monotonicity*) is the only thing guaranteed. See [Neighborhood radius](#neighborhood-radius) for what it does and does not imply.
+
+On `Pgrid`, a neighbor query becomes a stencil lookup:
+
+```text
+neighborhood(Pgrid[i, j, k]) ≈ { Pgrid[i±di, j±dj, k±dk] : (di, dj, dk) within radius R }
+```
+
+where `R` is a cutoff you choose for your task. Operations built on this run in **O(N)** on the grid view.
+
+---
+
+## Internal algorithm
+
+To get the main ideas of the sorting algorithm, the core of `gridpoints` is:
+1. initialisation: a good starting assignement is important both for quick convergence, and to converge on an assignement with good neighborhood properties. The initialiser build a kdtree on the points, and convert the tree cuts to a grid structure
+2. iterative 1d sorts: once the initialiser built the starting assignement, gridpoints further optimise it iteratively by looking at 1d slices of the grid (axial slices, but also diagonal ones), and apply standard 1d sort on the points, projected along the direction of the slice.
+
+For interested readers, the full sorting procedure and motivation behind is explained in more details [here](https://github.com/Neighborhood-Grid/Cartesian-Grid-Sort). 
+
+## Installation
+
+```bash
+pip install gridpoints          # core only
+pip install gridpoints[demo]    # adds visualisation tools for the demo
+```
+
+## Demo 1. modelnet40:
+To play with real life objects from the Modelnet40 dataset, first get modelnet40 from [princeton.edu](http://modelnet.cs.princeton.edu)  (~2 GB), then:
+```
+python modelnet40_demo.py --obj table --num 6
+python modelnet40_demo.py --obj airplane --num 1
+...
+```
+Animated figures (morphing from the raw  point cloud to a cubic grid) will be saved to sqrnet\plot.gif
+
+## Demo 2. big point clouds:
+To test gridpoints on various synthetic datasets, see the notebook: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
+
+## Quickstart
+
+```python
+import gridpoints as grid
+import numpy as np
+
+N = 1_000_000
+A = np.random.rand(N, 3)                       # raw point cloud (NumPy, PyTorch or CuPy)
+
+# 1. Compute the permutation and view the points as a grid
+order = grid.argsort(A, gridshape=(100, 100, 100))
+Bflat = A[order]                               # sorted, flat (C-order)
+Bgrid = Bflat.reshape(100, 100, 100, 3)        # grid view
+
+# 2. Work on the grid
+Cgrid = apply_something(Bgrid)                 # e.g. a convolution; shape (100, 100, 100, *)
+
+# 3. Go back to the original point order
+Cflat = Cgrid.reshape(N, -1)
+orderinv = grid.invert_permutation(order)
+C = Cflat[orderinv]                            # back to the original point indexing
+```
+
+`apply_something` stands for whatever you want to compute on the grid.
+
+`gridpoints.sort()` is also available.
+
+---
+
+## Choosing the parameters
+
+### Grid shape
+
+The grid shape must be roughly tuned to the *kind* of data, with the constraint that the number of points `N` has to be factorized as `I⋅J⋅K⋅…`
+
+- The exact factorization hardly matters: `(18, 20, 16)` and `(16, 15, 24)` behave similarly.
+- The **orders of magnitude** of each dimension matter a lot: `(18, 20, 16)` and `(36, 40, 4)` can give substantially different results.
+- For a thin surface (e.g. an eggshell), use a flat grid such as `(128, 128, 2)`. A naive `(32, 32, 32)` grid on the same data produces a poor representation of the geometry.
+
+**Padding:** if the number of cells `I⋅J⋅K⋅…` exceeds the number of points `N`, the point cloud can be padded with special values (see [Padding](#padding-with-nan-and-inf)). This lets you sort a prime or variable number of points, and removes the integer-factorization constraint on the grid shape.
+
+### Neighborhood radius
+
+There is no theoretical guarantee on the radius `R` needed for a given task. Nearest neighbors are not guaranteed to land in exactly adjacent cells.
+
+Empirically, in 2D, `R = 5` captures about 99% of nearest neighbors. Some outliers might sit further away, mostly on complex geometries with sharp peaks, holes or other non-smooth features.
+
+If you need stricter neighborhoods, or work in higher dimension, there are two options:
+
+- **Ensemble of grids:** build several grids, each on a rotated or projected view of the points, and combine them (see https://arxiv.org/abs/1505.00880 for the concept of multi view pooling). Projecting the data randomly onto one or several lower-dimensional subspaces before sorting also helps reduce topology-specific effects, since random projections approximately preserve pairwise distances (Johnson–Lindenstrauss lemma).
+- **Tiling:** if you need *exact* neighbors, tiling gives them. Split the grid into tiles, attach metadata to each tile (e.g. a bounding box), and skip pairs of tiles that are certifiably too far apart for your criterion. The grid does the bulk of the work; the tiles only certify or correct the rest.
+
+### Padding with NaN and Inf
+
+NaN and Inf values are supported in a consistent way by the package, so that padding is natural:
+
+- `NaN` → placed at a random position
+- `±Inf` → placed on the border of the grid
+
+This lets you complete the N real points with ΔN fictitious points so that N + ΔN = I⋅J⋅K⋅… , using `gridpoints.pad`
+
+---
+
+## Performance
+
+Sorting 1 million points (2D/3D), measured on Google Colab:
+
+| Backend | Time |
+|---|---|
+| NumPy (CPU) | ~10 s |
+| PyTorch (T4 GPU) | ~0.5 s |
+| CuPy (T4 GPU) | ~1 s (the first, cold call takes ~10 s because of compilation) |
+
+### Running stencil operations efficiently
+
+A typical use looks like:
+
+```text
+output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] )   for (di, dj, dk) in a local window
+```
+
+To compute this kind of kernel efficiently, avoid pure Python loops, which are slow. Prefer:
+
+- Native grid convolutions of standard libraries, whenever your operation can be expressed that way.
+- `pystencils` or `taichi` for complex or non-linear kernels.
+- On GPU, tiling the grid with a custom `triton` kernel can be particularly efficient.
+
+---
+
+## Limits
+
+**Good fit**
+
+- **Low-dimensional point clouds.** 2D and 3D are the sweet spot, with enough points for O(N) grid operations to pay off.
+- **Pipelines that benefit from regular arrays:** GPU processing, convolutions, repeated local operations.
+- **Nice geometries.** "Nice" is deliberately informal. The question is: can you mentally spread your point distribution onto a regular box with the specified dimensions without your brain blowing up in the process? Remarkably, this requires neither convexity (a map of France, a sponge, a donut, an elephant, an eggshell are valid) nor connectedness (clusters and blobs are valid, Gridpoints just glues them together). Examples are in the [plots folder](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots).
+
+**Poor fit**
+
+- **High dimension.** Arbitrary dimension is supported, but the sweet spot is 2D/3D. Dimensions 4 to 6 are still reasonable. Above ~8D, Gridpoints is generally not appropriate. Note that many powerful dimension reduction technique exists beyond random projections ensemble or principal component analysis. Modern ml tools such as auto-encoders or parametric umap could maybe help to convert high dimensional data to the low dimensional framework of gridpoints.
+- **Small point clouds.** Below a few hundred points, the overhead is not worth it: a naive quadratic implementation of your kernels will be simpler and probably faster.
+- **Anti-grid geometries.** A spider web, a wind turbine... The sort still runs, but a single grid sort gives a poor representation of the structure. For such distributions, you currently need a projection + grid ensemble procedure, which must be set up manually. That may make Gridpoints less attractive if you were looking for a light, out-of-the-box gridification framework. I am working on making this more automatic, and I hope that at some point Gridpoints will be easily applicable to a much wider range of geometries. Updates will be posted here!
+
+## Should you use it?
+
+Try it if you process 2D/3D point clouds with local operations and you want regular array layouts (especially on GPU). Approximate neighborhoods work out of the box, and exact ones are available through tiling.
+
+Stay with a KD-tree or a graph method if you work in high dimension or have only a few hundred points. If you need exact nearest neighbors, Gridpoints can still work, via tiling, but expect some extra implementation effort.
+
+A quick way to decide is to run the [demo notebook](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb) on a sample of your own data and check how much of your true nearest neighbors fall within your radius `R`.

gridpoints-1.1.1/README.md

@@ -0,0 +1,207 @@
+# Gridpoints
+
+[![PyPI](https://img.shields.io/pypi/v/gridpoints?logo=pypi&logoColor=white)](https://pypi.org/project/gridpoints/)
+[![GitHub](https://img.shields.io/badge/GitHub-gridpoints-181717?logo=github)](https://github.com/Neighborhood-Grid/gridpoints)
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/table(1).gif">
+
+**Reorder an unstructured point cloud into a regular grid, so that neighbor-based operations become simple grid processing.**
+
+```python
+import gridpoints as grid
+points = your_dataset_of_points(1000000, 3)  # standard (N, D) array
+order = grid.argsort(points, gridshape=(100, 100, 100))  # set gridshape: (I, J, K, ...) = (100, 100, 100)
+Pgrid = points[order].reshape(100, 100, 100, 3)  # now an (I, J, K, ...,  D) version of your dataset
+# Ready for convolution, neighbor look-ups or geometrical analysis.
+# Note that grid.argsort also supports a batch mode (B, N, D) -> (B, I, J, K, ..., D)
+```
+
+Gridpoints supports NumPy, PyTorch and CuPy arrays.
+The `argsort()` reordering is a one-to-one assignment of your points to the cells of a grid with the given `gridshape`. This is related to Monge optimal transport, but `argsort()` trades optimality for scalability, by combining greedy axial operations which exploit the specific structure of grids. One million 2D/3D points are sorted in about 0.5 s on a GPU (PyTorch) or 10 s on a CPU (NumPy).
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/ballexemple.png">
+
+---
+
+## TL;DR
+
+- **Problem:** point clouds are stored as a flat list `[P1, ..., PN]`. Finding the neighbors of a point usually needs a tree or a graph (KD-tree, radius graph...), which is hard to use with standard tensor-based machine learning, especially on GPU.
+- **Idea:** assign every point to a cell of a D-dimensional grid, one point per cell, such that the grid is sorted along each of its axes. Then the neighbors of a point are *approximately* the points in nearby cells, found by plain indexing.
+- **Payoff:** local operations (convolution, smoothing, clustering, interpolation...) that are usually hard to implement on irregular point clouds can be applied on the grid: `points -> grid`. To recover the result on the points, one just applies the inverse assignment: `grid -> points`.
+- **Catch:** a plain grid gives *approximate* neighborhoods (about 99% of true nearest neighbors within a small radius). Exact neighborhoods are recoverable with tiling (see [Neighborhood radius](#neighborhood-radius)). It works best in 2D/3D, and stays reasonable up to about 6D.
+
+---
+
+## The problem
+
+Points are almost always stored in a one-dimensional order. You would like points that are close in space to also be close in memory, so that the nearest neighbor of `P[n]` is often `P[n±1]`, `P[n±2]`, or a bit further. This is a well-studied problem, and space-filling curves such as the Morton curve are a common solution.
+
+Gridpoints takes a different route. Instead of a 1D order, it gives each point a **multi-index** `[i, j, ...]` in a D-dimensional grid. This matters because a grid supports stencil operations ("look at the cells within a few steps in each direction", `[i±di, j±dj, k±dk]`).
+
+## What Gridpoints does
+
+Input: a point cloud `P` of shape `(N, D)` and a target grid shape `(I, J, ...)`.
+
+Output: a permutation `order` of the indices such that
+
+```python
+Pgrid = P[order].reshape(I, J, ..., D)
+```
+
+is a bijection between points and grid cells (one point, one cell), and is **sorted along every axis**. In 3D, with xyz notation `P = (x, y, z)`:
+
+```text
+x[i+1, j, k] >= x[i, j, k]     # x increases along axis 0
+y[i, j+1, k] >= y[i, j, k]     # y increases along axis 1
+z[i, j, k+1] >= z[i, j, k]     # z increases along axis 2
+```
+
+This property (*grid monotonicity*) is the only thing guaranteed. See [Neighborhood radius](#neighborhood-radius) for what it does and does not imply.
+
+On `Pgrid`, a neighbor query becomes a stencil lookup:
+
+```text
+neighborhood(Pgrid[i, j, k]) ≈ { Pgrid[i±di, j±dj, k±dk] : (di, dj, dk) within radius R }
+```
+
+where `R` is a cutoff you choose for your task. Operations built on this run in **O(N)** on the grid view.
+
+---
+
+## Internal algorithm
+
+To get the main ideas of the sorting algorithm, the core of `gridpoints` is:
+1. initialisation: a good starting assignement is important both for quick convergence, and to converge on an assignement with good neighborhood properties. The initialiser build a kdtree on the points, and convert the tree cuts to a grid structure
+2. iterative 1d sorts: once the initialiser built the starting assignement, gridpoints further optimise it iteratively by looking at 1d slices of the grid (axial slices, but also diagonal ones), and apply standard 1d sort on the points, projected along the direction of the slice.
+
+For interested readers, the full sorting procedure and motivation behind is explained in more details [here](https://github.com/Neighborhood-Grid/Cartesian-Grid-Sort). 
+
+## Installation
+
+```bash
+pip install gridpoints          # core only
+pip install gridpoints[demo]    # adds visualisation tools for the demo
+```
+
+## Demo 1. modelnet40:
+To play with real life objects from the Modelnet40 dataset, first get modelnet40 from [princeton.edu](http://modelnet.cs.princeton.edu)  (~2 GB), then:
+```
+python modelnet40_demo.py --obj table --num 6
+python modelnet40_demo.py --obj airplane --num 1
+...
+```
+Animated figures (morphing from the raw  point cloud to a cubic grid) will be saved to sqrnet\plot.gif
+
+## Demo 2. big point clouds:
+To test gridpoints on various synthetic datasets, see the notebook: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
+
+## Quickstart
+
+```python
+import gridpoints as grid
+import numpy as np
+
+N = 1_000_000
+A = np.random.rand(N, 3)                       # raw point cloud (NumPy, PyTorch or CuPy)
+
+# 1. Compute the permutation and view the points as a grid
+order = grid.argsort(A, gridshape=(100, 100, 100))
+Bflat = A[order]                               # sorted, flat (C-order)
+Bgrid = Bflat.reshape(100, 100, 100, 3)        # grid view
+
+# 2. Work on the grid
+Cgrid = apply_something(Bgrid)                 # e.g. a convolution; shape (100, 100, 100, *)
+
+# 3. Go back to the original point order
+Cflat = Cgrid.reshape(N, -1)
+orderinv = grid.invert_permutation(order)
+C = Cflat[orderinv]                            # back to the original point indexing
+```
+
+`apply_something` stands for whatever you want to compute on the grid.
+
+`gridpoints.sort()` is also available.
+
+---
+
+## Choosing the parameters
+
+### Grid shape
+
+The grid shape must be roughly tuned to the *kind* of data, with the constraint that the number of points `N` has to be factorized as `I⋅J⋅K⋅…`
+
+- The exact factorization hardly matters: `(18, 20, 16)` and `(16, 15, 24)` behave similarly.
+- The **orders of magnitude** of each dimension matter a lot: `(18, 20, 16)` and `(36, 40, 4)` can give substantially different results.
+- For a thin surface (e.g. an eggshell), use a flat grid such as `(128, 128, 2)`. A naive `(32, 32, 32)` grid on the same data produces a poor representation of the geometry.
+
+**Padding:** if the number of cells `I⋅J⋅K⋅…` exceeds the number of points `N`, the point cloud can be padded with special values (see [Padding](#padding-with-nan-and-inf)). This lets you sort a prime or variable number of points, and removes the integer-factorization constraint on the grid shape.
+
+### Neighborhood radius
+
+There is no theoretical guarantee on the radius `R` needed for a given task. Nearest neighbors are not guaranteed to land in exactly adjacent cells.
+
+Empirically, in 2D, `R = 5` captures about 99% of nearest neighbors. Some outliers might sit further away, mostly on complex geometries with sharp peaks, holes or other non-smooth features.
+
+If you need stricter neighborhoods, or work in higher dimension, there are two options:
+
+- **Ensemble of grids:** build several grids, each on a rotated or projected view of the points, and combine them (see https://arxiv.org/abs/1505.00880 for the concept of multi view pooling). Projecting the data randomly onto one or several lower-dimensional subspaces before sorting also helps reduce topology-specific effects, since random projections approximately preserve pairwise distances (Johnson–Lindenstrauss lemma).
+- **Tiling:** if you need *exact* neighbors, tiling gives them. Split the grid into tiles, attach metadata to each tile (e.g. a bounding box), and skip pairs of tiles that are certifiably too far apart for your criterion. The grid does the bulk of the work; the tiles only certify or correct the rest.
+
+### Padding with NaN and Inf
+
+NaN and Inf values are supported in a consistent way by the package, so that padding is natural:
+
+- `NaN` → placed at a random position
+- `±Inf` → placed on the border of the grid
+
+This lets you complete the N real points with ΔN fictitious points so that N + ΔN = I⋅J⋅K⋅… , using `gridpoints.pad`
+
+---
+
+## Performance
+
+Sorting 1 million points (2D/3D), measured on Google Colab:
+
+| Backend | Time |
+|---|---|
+| NumPy (CPU) | ~10 s |
+| PyTorch (T4 GPU) | ~0.5 s |
+| CuPy (T4 GPU) | ~1 s (the first, cold call takes ~10 s because of compilation) |
+
+### Running stencil operations efficiently
+
+A typical use looks like:
+
+```text
+output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] )   for (di, dj, dk) in a local window
+```
+
+To compute this kind of kernel efficiently, avoid pure Python loops, which are slow. Prefer:
+
+- Native grid convolutions of standard libraries, whenever your operation can be expressed that way.
+- `pystencils` or `taichi` for complex or non-linear kernels.
+- On GPU, tiling the grid with a custom `triton` kernel can be particularly efficient.
+
+---
+
+## Limits
+
+**Good fit**
+
+- **Low-dimensional point clouds.** 2D and 3D are the sweet spot, with enough points for O(N) grid operations to pay off.
+- **Pipelines that benefit from regular arrays:** GPU processing, convolutions, repeated local operations.
+- **Nice geometries.** "Nice" is deliberately informal. The question is: can you mentally spread your point distribution onto a regular box with the specified dimensions without your brain blowing up in the process? Remarkably, this requires neither convexity (a map of France, a sponge, a donut, an elephant, an eggshell are valid) nor connectedness (clusters and blobs are valid, Gridpoints just glues them together). Examples are in the [plots folder](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots).
+
+**Poor fit**
+
+- **High dimension.** Arbitrary dimension is supported, but the sweet spot is 2D/3D. Dimensions 4 to 6 are still reasonable. Above ~8D, Gridpoints is generally not appropriate. Note that many powerful dimension reduction technique exists beyond random projections ensemble or principal component analysis. Modern ml tools such as auto-encoders or parametric umap could maybe help to convert high dimensional data to the low dimensional framework of gridpoints.
+- **Small point clouds.** Below a few hundred points, the overhead is not worth it: a naive quadratic implementation of your kernels will be simpler and probably faster.
+- **Anti-grid geometries.** A spider web, a wind turbine... The sort still runs, but a single grid sort gives a poor representation of the structure. For such distributions, you currently need a projection + grid ensemble procedure, which must be set up manually. That may make Gridpoints less attractive if you were looking for a light, out-of-the-box gridification framework. I am working on making this more automatic, and I hope that at some point Gridpoints will be easily applicable to a much wider range of geometries. Updates will be posted here!
+
+## Should you use it?
+
+Try it if you process 2D/3D point clouds with local operations and you want regular array layouts (especially on GPU). Approximate neighborhoods work out of the box, and exact ones are available through tiling.
+
+Stay with a KD-tree or a graph method if you work in high dimension or have only a few hundred points. If you need exact nearest neighbors, Gridpoints can still work, via tiling, but expect some extra implementation effort.
+
+A quick way to decide is to run the [demo notebook](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb) on a sample of your own data and check how much of your true nearest neighbors fall within your radius `R`.

{gridpoints-1.0.7 → gridpoints-1.1.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "gridpoints"
-version = "1.0.7"
+version = "1.1.1"
 description = "fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid "
 readme = "README.md"
 requires-python = ">=3.9"
@@ -26,8 +26,8 @@ demo = [
 
 
 [project.urls]
-Homepage = "https://github.com/Neighborhood-Grid/CubeNet"
-Repository = "https://github.com/Neighborhood-Grid/CubeNet"
+Homepage = "https://neighborhood-grid.github.io/gridpoints/"
+Repository = "https://github.com/Neighborhood-Grid/gridpoints"
 
 [tool.setuptools.packages.find]
 where = ["src"]

gridpoints-1.1.1/src/griddemo/__init__.py

@@ -0,0 +1,10 @@
+
+from .quick_demo import sample, show_result, print_zoom
+
+__all__ = (
+    "sample",
+    "show_result",
+    "print_zoom",
+)
+
+__version__ = "1.0.8"

{gridpoints-1.0.7 → gridpoints-1.1.1}/src/gridpoints/__init__.py

@@ -1,11 +1,13 @@
-"""Monotonic Lagrangian sorting for NumPy, CuPy, and PyTorch arrays."""
-
-from .mlg_sort import argsort, sort, invert_permutation
-
-__all__ = (
-    "argsort",
-    "sort",
-    "invert_permutation",
-)
-
-__version__ = "1.0.6"
+"""Monotonic Lagrangian sorting for NumPy, CuPy, and PyTorch arrays."""
+
+from .mlg_sort import argsort, sort, invert_permutation
+from .padding import pad
+
+__all__ = (
+    "argsort",
+    "sort",
+    "invert_permutation",
+    "pad",
+)
+
+__version__ = "1.1.1"

gridpoints-1.1.1/src/gridpoints/array_libs.py

@@ -0,0 +1,183 @@
+"""
+Helpers:
+Unified array syntax to support numpy, cupy and pytorch arrays
+
+Here we just define basic primitives that we will need later
+like arange, copy, shuffle etc.
+"""
+
+
+def precision(N):
+    """Names of the (float, int) dtypes suited to point clouds of N points.
+
+    - float32 resolves the gaps (~1/N) between neighbouring points only up to
+      a few 10**4 points, float64 is used beyond.
+    - the diagonal sorts store values up to 6N: int32 is safe up to N ~ 3.5e8.
+    """
+    return ('float32' if N <= 2**16 else 'float64'), ('int32' if 6 * N < 2**31 else 'int64')
+
+
+def get_dtype(dtp, xp):
+    if dtp == 'bool':
+        return getattr(xp, 'bool_', None) or xp.bool
+    return getattr(xp, dtp) if isinstance(dtp, str) else dtp
+
+
+def get_backend(X):
+    module = type(X).__module__.split('.')[0]
+    if module == 'torch':
+        import torch
+        return torch, 'torch'
+    elif module == 'cupy':
+        import cupy
+        return cupy, 'cupy'
+    else:
+        import numpy
+        return numpy, 'numpy'
+
+def astype(X, dtype_str, reference_array=None):
+    xp, backend_name = get_backend(X)
+    dtype = get_dtype(dtype_str, xp)
+    if backend_name == 'torch':
+        device = reference_array.device if reference_array is not None else X.device
+        return X.to(dtype=dtype, device=device)
+    return X.astype(dtype, copy=False)
+
+
+def inplace_sort(X, axis):
+    xp, backend_name = get_backend(X)
+    if backend_name == 'torch':
+        X.copy_(X.sort(dim=axis)[0])
+    elif backend_name == "cupy":
+        X[...] = xp.sort(xp.ascontiguousarray(X), axis=axis)
+    else:
+        X.sort(axis=axis)
+
+
+def sort(X, axis):
+    xp, backend_name = get_backend(X)
+    return X.sort(dim=axis)[0] if backend_name == 'torch' else xp.sort(X, axis=axis)
+
+
+def _alloc(fn, shape, dtype_str, reference_array):
+    xp, backend_name = get_backend(reference_array)
+    dtype = get_dtype(dtype_str, xp)
+    if backend_name == 'torch':
+        return getattr(xp, fn)(shape, dtype=dtype, device=reference_array.device)
+    return getattr(xp, fn)(shape, dtype=dtype)
+
+def empty(shape, dtype_str, reference_array):
+    return _alloc('empty', shape, dtype_str, reference_array)
+
+def zeros(shape, dtype_str, reference_array):
+    return _alloc('zeros', shape, dtype_str, reference_array)
+
+def ones(shape, dtype_str, reference_array):
+    return _alloc('ones', shape, dtype_str, reference_array)
+
+
+def arange(N, dtype_str, reference_array):
+    xp, backend_name = get_backend(reference_array)
+    dtype = get_dtype(dtype_str, xp)
+    if backend_name == 'torch':
+        return xp.arange(N, dtype=dtype, device=reference_array.device)
+    return xp.arange(N, dtype=dtype)
+
+def random(shape, dtype_str, reference_array):
+    """Uniform random numbers in [0, 1)."""
+    xp, backend_name = get_backend(reference_array)
+    dtype = get_dtype(dtype_str, xp)
+    if backend_name == 'torch':
+        return xp.rand(shape, dtype=dtype, device=reference_array.device)
+    elif backend_name == 'cupy':
+        return xp.random.rand(*shape).astype(dtype)
+    return xp.random.default_rng(42).random(shape, dtype=dtype)
+
+def copy(arr):
+    _, backend_name = get_backend(arr)
+    if backend_name == 'torch':
+        return arr.clone()
+    return arr.copy()
+
+def amax(X, axis):
+    return X.amax(axis) if get_backend(X)[1] == 'torch' else X.max(axis)
+
+def amin(X, axis):
+    return X.amin(axis) if get_backend(X)[1] == 'torch' else X.min(axis)
+
+def all_true(tensor):
+    return bool(tensor.all())
+
+def any_true(tensor):
+    return bool(tensor.any())
+
+
+def nan_infs_to_num_and_tiebreaker(arr):
+    """Make a batch of clouds (B, N, D) finite, and break ties with a tiny noise.
+
+    +inf / -inf are put just beyond the finite points of their cloud and axis
+    (they end up on the border of the grid), NaN are replaced by the value of a
+    random finite point of their cloud and axis (they end up anywhere).
+    """
+    xp, _ = get_backend(arr)
+    out = arr
+    finite = xp.isfinite(arr)
+    if not all_true(finite):
+        hi = amax(xp.where(finite, arr, -xp.inf), 1)[:, None, :]
+        lo = amin(xp.where(finite, arr, xp.inf), 1)[:, None, :]
+        out = xp.where(xp.isposinf(arr), hi + 1, arr)
+        out = xp.where(xp.isneginf(arr), lo - 1, out)
+        nan = xp.isnan(arr)
+        if any_true(nan):
+            finite_first = sort(xp.where(finite, arr, xp.inf), 1)
+            n_finite = finite.sum(1)[:, None, :]
+            pick = astype(random(arr.shape, arr.dtype, arr) * n_finite, 'int64')
+            out = xp.where(nan, take_along_axis(finite_first, pick, 1), out)
+    eps = 1e-5 if '32' in str(arr.dtype) else 1e-10
+    return out + eps * random(out.shape, arr.dtype, out)
+
+def flip(X, axis=None):
+    xp, _ = get_backend(X)
+    if axis is None or axis == () or axis == []:
+        return X
+
+    if isinstance(axis, int):
+        axis = (axis,)
+    else:
+        axis = tuple(axis)
+
+    if hasattr(X, "flip") and callable(X.flip):
+        return X.flip(axis)
+
+    return xp.flip(X, axis=axis)
+
+def argsort(X):
+    xp, backend_name = get_backend(X)
+    return xp.argsort(X, dim=-1) if backend_name == 'torch' else xp.argsort(X, axis=-1)
+
+def argpartition(X, mid):
+    """Indices that put the `mid` smallest values of the last axis first."""
+    xp, backend_name = get_backend(X)
+    if backend_name == 'torch':
+        return xp.argsort(X, dim=-1)
+    else:
+        return xp.argpartition(X, mid - 1, axis=-1)
+
+def take_along_axis(arr, idx, axis):
+    xp, backend_name = get_backend(arr)
+    if backend_name == 'torch':
+        return xp.take_along_dim(arr, idx.long(), dim=axis)
+    else:
+        return xp.take_along_axis(arr, idx, axis=axis)
+
+def concatenate(arrays, axis = 0):
+    xp, name = get_backend(arrays[0])
+    if name == "torch":
+        return xp.cat(list(arrays), dim=axis)
+    return xp.concatenate(arrays, axis=axis)
+
+def transpose(X, axes):
+    _, name = get_backend(X)
+    if name == "torch":
+        return X.permute(*axes)
+    return X.transpose(axes)