gridpoints 1.1.0__tar.gz → 1.1.2__tar.gz

gridpoints-1.1.2/PKG-INFO

@@ -0,0 +1,223 @@
+Metadata-Version: 2.4
+Name: gridpoints
+Version: 1.1.2
+Summary: fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid 
+Author: Armand de Cacqueray
+License: MIT
+Project-URL: Homepage, https://neighborhood-grid.github.io/gridpoints/
+Project-URL: Repository, https://github.com/Neighborhood-Grid/gridpoints
+Keywords: sorting,point-clouds,multi-indexing,grid,neighborhood-grid,monotonic-lagrangian-grid,squarenet,nearest-neighbor
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: demo
+Requires-Dist: squarenet; extra == "demo"
+Dynamic: license-file
+
+# Gridpoints
+
+[![PyPI](https://img.shields.io/pypi/v/gridpoints?logo=pypi&logoColor=white)](https://pypi.org/project/gridpoints/)
+[![GitHub](https://img.shields.io/badge/GitHub-gridpoints-181717?logo=github)](https://github.com/Neighborhood-Grid/gridpoints)
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/table(1).gif">
+
+**Reorder an unstructured point cloud into a regular grid, so that neighbor-based operations become simple grid processing.**
+
+```python
+import gridpoints as grid
+points = your_dataset_of_points(1000000, 3)  # standard (N, D) array
+order = grid.argsort(points, gridshape=(100, 100, 100))  # set gridshape: (I, J, K, ...) = (100, 100, 100)
+Pgrid = points[order].reshape(100, 100, 100, 3)  # now an (I, J, K, ...,  D) version of your dataset
+# Ready for convolution, neighbor look-ups or geometrical analysis.
+# Note that grid.argsort also supports a batch mode (B, N, D) -> (B, I, J, K, ..., D)
+```
+
+Gridpoints supports NumPy, PyTorch and CuPy arrays.
+The `argsort()` reordering is a one-to-one assignment of your points to the cells of a grid with the given `gridshape`. This is related to Monge optimal transport, but `argsort()` trades optimality for scalability, by combining greedy axial operations which exploit the specific structure of grids. One million 2D/3D points are sorted in about 0.5 s on a GPU (PyTorch) or 10 s on a CPU (NumPy).
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/ballexemple.png">
+
+---
+
+## TL;DR
+
+- **Problem:** point clouds are stored as a flat list `[P1, ..., PN]`. Finding the neighbors of a point usually needs a tree or a graph (KD-tree, radius graph...), which is hard to use with standard tensor-based machine learning, especially on GPU.
+- **Idea:** assign every point to a cell of a D-dimensional grid, one point per cell, such that the grid is sorted along each of its axes. Then the neighbors of a point are *approximately* the points in nearby cells, found by plain indexing.
+- **Payoff:** local operations (convolution, smoothing, clustering, interpolation...) that are usually hard to implement on irregular point clouds can be applied on the grid: `points -> grid`. To recover the result on the points, one just applies the inverse assignment: `grid -> points`.
+- **Catch:** a plain grid gives *approximate* neighborhoods (about 99% of true nearest neighbors within a small radius). Exact neighborhoods are recoverable with tiling (see [Neighborhood radius](#neighborhood-radius)). It works best in 2D/3D, and stays reasonable up to about 6D.
+
+---
+
+## The problem
+
+Points are almost always stored in a one-dimensional order. You would like points that are close in space to also be close in memory, so that the nearest neighbor of `P[n]` is often `P[n±1]`, `P[n±2]`, or a bit further. This is a well-studied problem, and space-filling curves such as the Morton curve are a common solution.
+
+Gridpoints takes a different route. Instead of a 1D order, it gives each point a **multi-index** `[i, j, ...]` in a D-dimensional grid. This matters because a grid supports stencil operations ("look at the cells within a few steps in each direction", `[i±di, j±dj, k±dk]`).
+
+## What Gridpoints does
+
+Input: a point cloud `P` of shape `(N, D)` and a target grid shape `(I, J, ...)`.
+
+Output: a permutation `order` of the indices such that
+
+```python
+Pgrid = P[order].reshape(I, J, ..., D)
+```
+
+is a bijection between points and grid cells (one point, one cell), and is **sorted along every axis**. In 3D, with xyz notation `P = (x, y, z)`:
+
+```text
+x[i+1, j, k] >= x[i, j, k]     # x increases along axis 0
+y[i, j+1, k] >= y[i, j, k]     # y increases along axis 1
+z[i, j, k+1] >= z[i, j, k]     # z increases along axis 2
+```
+
+This property (*grid monotonicity*) is the only thing guaranteed. See [Neighborhood radius](#neighborhood-radius) for what it does and does not imply.
+
+On `Pgrid`, a neighbor query becomes a stencil lookup:
+
+```text
+neighborhood(Pgrid[i, j, k]) ≈ { Pgrid[i±di, j±dj, k±dk] : (di, dj, dk) within radius R }
+```
+
+where `R` is a cutoff you choose for your task. Operations built on this run in **O(N)** on the grid view.
+
+---
+
+## Internal algorithm
+
+To get the main ideas of the sorting algorithm, the core of `gridpoints` is:
+1. initialisation: a good starting assignement is important both for quick convergence, and to converge on an assignement with good neighborhood properties. The initialiser build a kdtree on the points, and convert the tree cuts to a grid structure
+2. iterative 1d sorts: once the initialiser built the starting assignement, gridpoints further optimise it iteratively by looking at 1d slices of the grid (axial slices, but also diagonal ones), and apply standard 1d sort on the points, projected along the direction of the slice.
+
+For interested readers, the full sorting procedure and motivation behind is explained in more details [here](https://github.com/Neighborhood-Grid/Cartesian-Grid-Sort). 
+
+## Installation
+
+```bash
+pip install gridpoints          # core only
+pip install gridpoints[demo]    # adds visualisation tools for the demo
+```
+
+## Demo 1. modelnet40:
+To play with real life objects from the Modelnet40 dataset, first get modelnet40 from [princeton.edu](http://modelnet.cs.princeton.edu)  (~2 GB), then:
+```
+python modelnet40_demo.py --obj table --num 6
+python modelnet40_demo.py --obj airplane --num 1
+...
+```
+Animated figures (morphing from the raw  point cloud to a cubic grid) will be saved to sqrnet\plot.gif
+
+## Demo 2. big point clouds:
+To test gridpoints on various synthetic datasets, see the notebook: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
+
+## Quickstart
+
+```python
+import gridpoints as grid
+import numpy as np
+
+N = 1_000_000
+A = np.random.rand(N, 3)                       # raw point cloud (NumPy, PyTorch or CuPy)
+
+# 1. Compute the permutation and view the points as a grid
+order = grid.argsort(A, gridshape=(100, 100, 100))
+Bflat = A[order]                               # sorted, flat (C-order)
+Bgrid = Bflat.reshape(100, 100, 100, 3)        # grid view
+
+# 2. Work on the grid
+Cgrid = apply_something(Bgrid)                 # e.g. a convolution; shape (100, 100, 100, *)
+
+# 3. Go back to the original point order
+Cflat = Cgrid.reshape(N, -1)
+orderinv = grid.invert_permutation(order)
+C = Cflat[orderinv]                            # back to the original point indexing
+```
+
+`apply_something` stands for whatever you want to compute on the grid.
+
+`gridpoints.sort()` is also available.
+
+---
+
+## Choosing the parameters
+
+### Grid shape
+
+The grid shape must be roughly tuned to the *kind* of data, with the constraint that the number of points `N` has to be factorized as `I⋅J⋅K⋅…`
+
+- The exact factorization hardly matters: `(18, 20, 16)` and `(16, 15, 24)` behave similarly.
+- The **orders of magnitude** of each dimension matter a lot: `(18, 20, 16)` and `(36, 40, 4)` can give substantially different results.
+- For a thin surface (e.g. an eggshell), use a flat grid such as `(128, 128, 2)`. A naive `(32, 32, 32)` grid on the same data produces a poor representation of the geometry.
+
+**Padding:** if the number of cells `I⋅J⋅K⋅…` exceeds the number of points `N`, the point cloud can be padded with special values (see [Padding](#padding-with-nan-and-inf)). This lets you sort a prime or variable number of points, and removes the integer-factorization constraint on the grid shape.
+
+### Neighborhood radius
+
+There is no theoretical guarantee on the radius `R` needed for a given task. Nearest neighbors are not guaranteed to land in exactly adjacent cells.
+
+Empirically, in 2D, `R = 5` captures about 99% of nearest neighbors. Some outliers might sit further away, mostly on complex geometries with sharp peaks, holes or other non-smooth features.
+
+If you need stricter neighborhoods, or work in higher dimension, there are two options:
+
+- **Ensemble of grids:** build several grids, each on a rotated or projected view of the points, and combine them (see https://arxiv.org/abs/1505.00880 for the concept of multi view pooling). Projecting the data randomly onto one or several lower-dimensional subspaces before sorting also helps reduce topology-specific effects, since random projections approximately preserve pairwise distances (Johnson–Lindenstrauss lemma).
+- **Tiling:** if you need *exact* neighbors, tiling gives them. Split the grid into tiles, attach metadata to each tile (e.g. a bounding box), and skip pairs of tiles that are certifiably too far apart for your criterion. The grid does the bulk of the work; the tiles only certify or correct the rest.
+
+### Padding with NaN and Inf
+
+NaN and Inf values are supported in a consistent way by the package, so that padding is natural:
+
+- `NaN` → placed at a random position
+- `±Inf` → placed on the border of the grid
+
+This lets you complete the N real points with ΔN fictitious points so that N + ΔN = I⋅J⋅K⋅… , using `gridpoints.pad`
+
+---
+
+## Performance
+
+Sorting 1 million points (2D/3D), measured on Google Colab:
+
+| Backend | Time |
+|---|---|
+| NumPy (CPU) | ~10 s |
+| PyTorch (T4 GPU) | ~0.5 s |
+| CuPy (T4 GPU) | ~1 s (the first, cold call takes ~10 s because of compilation) |
+
+### Running stencil operations efficiently
+
+A typical use looks like:
+
+```text
+output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] )   for (di, dj, dk) in a local window
+```
+
+To compute this kind of kernel efficiently, avoid pure Python loops, which are slow. Prefer:
+
+- Native grid convolutions of standard libraries, whenever your operation can be expressed that way.
+- `pystencils` or `taichi` for complex or non-linear kernels.
+- On GPU, tiling the grid with a custom `triton` kernel can be particularly efficient.
+
+---
+
+## Limits
+
+**Good fit**
+
+- **Low-dimensional point clouds.** 2D and 3D are the sweet spot, with enough points for O(N) grid operations to pay off.
+- **Pipelines that benefit from regular arrays:** GPU processing, convolutions, repeated local operations.
+- **Nice geometries.** "Nice" is deliberately informal. The question is: can you mentally spread your point distribution onto a regular box with the specified dimensions without your brain blowing up in the process? Remarkably, this requires neither convexity (a map of France, a sponge, a donut, an elephant, an eggshell are valid) nor connectedness (clusters and blobs are valid, Gridpoints just glues them together). Examples are in the [plots folder](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots).
+
+**Poor fit**
+
+- **High dimension.** Arbitrary dimension is supported, but the sweet spot is 2D/3D. Dimensions 4 to 6 are still reasonable. Above ~8D, Gridpoints is generally not appropriate. Note that many powerful dimension reduction technique exists beyond random projections ensemble or principal component analysis. Modern ml tools such as auto-encoders or parametric umap could maybe help to convert high dimensional data to the low dimensional framework of gridpoints.
+- **Small point clouds.** Below a few hundred points, the overhead is not worth it: a naive quadratic implementation of your kernels will be simpler and probably faster.
+- **Anti-grid geometries.** A spider web, a wind turbine... The sort still runs, but a single grid sort gives a poor representation of the structure. For such distributions, you currently need a projection + grid ensemble procedure, which must be set up manually. That may make Gridpoints less attractive if you were looking for a light, out-of-the-box gridification framework. I am working on making this more automatic, and I hope that at some point Gridpoints will be easily applicable to a much wider range of geometries. Updates will be posted here!
+
+## Should you use it?
+
+Try it if you process 2D/3D point clouds with local operations and you want regular array layouts (especially on GPU). Approximate neighborhoods work out of the box, and exact ones are available through tiling.
+
+Stay with a KD-tree or a graph method if you work in high dimension or have only a few hundred points. If you need exact nearest neighbors, Gridpoints can still work, via tiling, but expect some extra implementation effort.
+
+A quick way to decide is to run the [demo notebook](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb) on a sample of your own data and check how much of your true nearest neighbors fall within your radius `R`.

gridpoints-1.1.2/README.md

@@ -0,0 +1,207 @@
+# Gridpoints
+
+[![PyPI](https://img.shields.io/pypi/v/gridpoints?logo=pypi&logoColor=white)](https://pypi.org/project/gridpoints/)
+[![GitHub](https://img.shields.io/badge/GitHub-gridpoints-181717?logo=github)](https://github.com/Neighborhood-Grid/gridpoints)
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/table(1).gif">
+
+**Reorder an unstructured point cloud into a regular grid, so that neighbor-based operations become simple grid processing.**
+
+```python
+import gridpoints as grid
+points = your_dataset_of_points(1000000, 3)  # standard (N, D) array
+order = grid.argsort(points, gridshape=(100, 100, 100))  # set gridshape: (I, J, K, ...) = (100, 100, 100)
+Pgrid = points[order].reshape(100, 100, 100, 3)  # now an (I, J, K, ...,  D) version of your dataset
+# Ready for convolution, neighbor look-ups or geometrical analysis.
+# Note that grid.argsort also supports a batch mode (B, N, D) -> (B, I, J, K, ..., D)
+```
+
+Gridpoints supports NumPy, PyTorch and CuPy arrays.
+The `argsort()` reordering is a one-to-one assignment of your points to the cells of a grid with the given `gridshape`. This is related to Monge optimal transport, but `argsort()` trades optimality for scalability, by combining greedy axial operations which exploit the specific structure of grids. One million 2D/3D points are sorted in about 0.5 s on a GPU (PyTorch) or 10 s on a CPU (NumPy).
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/ballexemple.png">
+
+---
+
+## TL;DR
+
+- **Problem:** point clouds are stored as a flat list `[P1, ..., PN]`. Finding the neighbors of a point usually needs a tree or a graph (KD-tree, radius graph...), which is hard to use with standard tensor-based machine learning, especially on GPU.
+- **Idea:** assign every point to a cell of a D-dimensional grid, one point per cell, such that the grid is sorted along each of its axes. Then the neighbors of a point are *approximately* the points in nearby cells, found by plain indexing.
+- **Payoff:** local operations (convolution, smoothing, clustering, interpolation...) that are usually hard to implement on irregular point clouds can be applied on the grid: `points -> grid`. To recover the result on the points, one just applies the inverse assignment: `grid -> points`.
+- **Catch:** a plain grid gives *approximate* neighborhoods (about 99% of true nearest neighbors within a small radius). Exact neighborhoods are recoverable with tiling (see [Neighborhood radius](#neighborhood-radius)). It works best in 2D/3D, and stays reasonable up to about 6D.
+
+---
+
+## The problem
+
+Points are almost always stored in a one-dimensional order. You would like points that are close in space to also be close in memory, so that the nearest neighbor of `P[n]` is often `P[n±1]`, `P[n±2]`, or a bit further. This is a well-studied problem, and space-filling curves such as the Morton curve are a common solution.
+
+Gridpoints takes a different route. Instead of a 1D order, it gives each point a **multi-index** `[i, j, ...]` in a D-dimensional grid. This matters because a grid supports stencil operations ("look at the cells within a few steps in each direction", `[i±di, j±dj, k±dk]`).
+
+## What Gridpoints does
+
+Input: a point cloud `P` of shape `(N, D)` and a target grid shape `(I, J, ...)`.
+
+Output: a permutation `order` of the indices such that
+
+```python
+Pgrid = P[order].reshape(I, J, ..., D)
+```
+
+is a bijection between points and grid cells (one point, one cell), and is **sorted along every axis**. In 3D, with xyz notation `P = (x, y, z)`:
+
+```text
+x[i+1, j, k] >= x[i, j, k]     # x increases along axis 0
+y[i, j+1, k] >= y[i, j, k]     # y increases along axis 1
+z[i, j, k+1] >= z[i, j, k]     # z increases along axis 2
+```
+
+This property (*grid monotonicity*) is the only thing guaranteed. See [Neighborhood radius](#neighborhood-radius) for what it does and does not imply.
+
+On `Pgrid`, a neighbor query becomes a stencil lookup:
+
+```text
+neighborhood(Pgrid[i, j, k]) ≈ { Pgrid[i±di, j±dj, k±dk] : (di, dj, dk) within radius R }
+```
+
+where `R` is a cutoff you choose for your task. Operations built on this run in **O(N)** on the grid view.
+
+---
+
+## Internal algorithm
+
+To get the main ideas of the sorting algorithm, the core of `gridpoints` is:
+1. initialisation: a good starting assignement is important both for quick convergence, and to converge on an assignement with good neighborhood properties. The initialiser build a kdtree on the points, and convert the tree cuts to a grid structure
+2. iterative 1d sorts: once the initialiser built the starting assignement, gridpoints further optimise it iteratively by looking at 1d slices of the grid (axial slices, but also diagonal ones), and apply standard 1d sort on the points, projected along the direction of the slice.
+
+For interested readers, the full sorting procedure and motivation behind is explained in more details [here](https://github.com/Neighborhood-Grid/Cartesian-Grid-Sort). 
+
+## Installation
+
+```bash
+pip install gridpoints          # core only
+pip install gridpoints[demo]    # adds visualisation tools for the demo
+```
+
+## Demo 1. modelnet40:
+To play with real life objects from the Modelnet40 dataset, first get modelnet40 from [princeton.edu](http://modelnet.cs.princeton.edu)  (~2 GB), then:
+```
+python modelnet40_demo.py --obj table --num 6
+python modelnet40_demo.py --obj airplane --num 1
+...
+```
+Animated figures (morphing from the raw  point cloud to a cubic grid) will be saved to sqrnet\plot.gif
+
+## Demo 2. big point clouds:
+To test gridpoints on various synthetic datasets, see the notebook: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
+
+## Quickstart
+
+```python
+import gridpoints as grid
+import numpy as np
+
+N = 1_000_000
+A = np.random.rand(N, 3)                       # raw point cloud (NumPy, PyTorch or CuPy)
+
+# 1. Compute the permutation and view the points as a grid
+order = grid.argsort(A, gridshape=(100, 100, 100))
+Bflat = A[order]                               # sorted, flat (C-order)
+Bgrid = Bflat.reshape(100, 100, 100, 3)        # grid view
+
+# 2. Work on the grid
+Cgrid = apply_something(Bgrid)                 # e.g. a convolution; shape (100, 100, 100, *)
+
+# 3. Go back to the original point order
+Cflat = Cgrid.reshape(N, -1)
+orderinv = grid.invert_permutation(order)
+C = Cflat[orderinv]                            # back to the original point indexing
+```
+
+`apply_something` stands for whatever you want to compute on the grid.
+
+`gridpoints.sort()` is also available.
+
+---
+
+## Choosing the parameters
+
+### Grid shape
+
+The grid shape must be roughly tuned to the *kind* of data, with the constraint that the number of points `N` has to be factorized as `I⋅J⋅K⋅…`
+
+- The exact factorization hardly matters: `(18, 20, 16)` and `(16, 15, 24)` behave similarly.
+- The **orders of magnitude** of each dimension matter a lot: `(18, 20, 16)` and `(36, 40, 4)` can give substantially different results.
+- For a thin surface (e.g. an eggshell), use a flat grid such as `(128, 128, 2)`. A naive `(32, 32, 32)` grid on the same data produces a poor representation of the geometry.
+
+**Padding:** if the number of cells `I⋅J⋅K⋅…` exceeds the number of points `N`, the point cloud can be padded with special values (see [Padding](#padding-with-nan-and-inf)). This lets you sort a prime or variable number of points, and removes the integer-factorization constraint on the grid shape.
+
+### Neighborhood radius
+
+There is no theoretical guarantee on the radius `R` needed for a given task. Nearest neighbors are not guaranteed to land in exactly adjacent cells.
+
+Empirically, in 2D, `R = 5` captures about 99% of nearest neighbors. Some outliers might sit further away, mostly on complex geometries with sharp peaks, holes or other non-smooth features.
+
+If you need stricter neighborhoods, or work in higher dimension, there are two options:
+
+- **Ensemble of grids:** build several grids, each on a rotated or projected view of the points, and combine them (see https://arxiv.org/abs/1505.00880 for the concept of multi view pooling). Projecting the data randomly onto one or several lower-dimensional subspaces before sorting also helps reduce topology-specific effects, since random projections approximately preserve pairwise distances (Johnson–Lindenstrauss lemma).
+- **Tiling:** if you need *exact* neighbors, tiling gives them. Split the grid into tiles, attach metadata to each tile (e.g. a bounding box), and skip pairs of tiles that are certifiably too far apart for your criterion. The grid does the bulk of the work; the tiles only certify or correct the rest.
+
+### Padding with NaN and Inf
+
+NaN and Inf values are supported in a consistent way by the package, so that padding is natural:
+
+- `NaN` → placed at a random position
+- `±Inf` → placed on the border of the grid
+
+This lets you complete the N real points with ΔN fictitious points so that N + ΔN = I⋅J⋅K⋅… , using `gridpoints.pad`
+
+---
+
+## Performance
+
+Sorting 1 million points (2D/3D), measured on Google Colab:
+
+| Backend | Time |
+|---|---|
+| NumPy (CPU) | ~10 s |
+| PyTorch (T4 GPU) | ~0.5 s |
+| CuPy (T4 GPU) | ~1 s (the first, cold call takes ~10 s because of compilation) |
+
+### Running stencil operations efficiently
+
+A typical use looks like:
+
+```text
+output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] )   for (di, dj, dk) in a local window
+```
+
+To compute this kind of kernel efficiently, avoid pure Python loops, which are slow. Prefer:
+
+- Native grid convolutions of standard libraries, whenever your operation can be expressed that way.
+- `pystencils` or `taichi` for complex or non-linear kernels.
+- On GPU, tiling the grid with a custom `triton` kernel can be particularly efficient.
+
+---
+
+## Limits
+
+**Good fit**
+
+- **Low-dimensional point clouds.** 2D and 3D are the sweet spot, with enough points for O(N) grid operations to pay off.
+- **Pipelines that benefit from regular arrays:** GPU processing, convolutions, repeated local operations.
+- **Nice geometries.** "Nice" is deliberately informal. The question is: can you mentally spread your point distribution onto a regular box with the specified dimensions without your brain blowing up in the process? Remarkably, this requires neither convexity (a map of France, a sponge, a donut, an elephant, an eggshell are valid) nor connectedness (clusters and blobs are valid, Gridpoints just glues them together). Examples are in the [plots folder](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots).
+
+**Poor fit**
+
+- **High dimension.** Arbitrary dimension is supported, but the sweet spot is 2D/3D. Dimensions 4 to 6 are still reasonable. Above ~8D, Gridpoints is generally not appropriate. Note that many powerful dimension reduction technique exists beyond random projections ensemble or principal component analysis. Modern ml tools such as auto-encoders or parametric umap could maybe help to convert high dimensional data to the low dimensional framework of gridpoints.
+- **Small point clouds.** Below a few hundred points, the overhead is not worth it: a naive quadratic implementation of your kernels will be simpler and probably faster.
+- **Anti-grid geometries.** A spider web, a wind turbine... The sort still runs, but a single grid sort gives a poor representation of the structure. For such distributions, you currently need a projection + grid ensemble procedure, which must be set up manually. That may make Gridpoints less attractive if you were looking for a light, out-of-the-box gridification framework. I am working on making this more automatic, and I hope that at some point Gridpoints will be easily applicable to a much wider range of geometries. Updates will be posted here!
+
+## Should you use it?
+
+Try it if you process 2D/3D point clouds with local operations and you want regular array layouts (especially on GPU). Approximate neighborhoods work out of the box, and exact ones are available through tiling.
+
+Stay with a KD-tree or a graph method if you work in high dimension or have only a few hundred points. If you need exact nearest neighbors, Gridpoints can still work, via tiling, but expect some extra implementation effort.
+
+A quick way to decide is to run the [demo notebook](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb) on a sample of your own data and check how much of your true nearest neighbors fall within your radius `R`.

{gridpoints-1.1.0 → gridpoints-1.1.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "gridpoints"
-version = "1.1.0"
+version = "1.1.2"
 description = "fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid "
 readme = "README.md"
 requires-python = ">=3.9"

{gridpoints-1.1.0 → gridpoints-1.1.2}/src/gridpoints/__init__.py

@@ -10,4 +10,4 @@ __all__ = (
     "pad",
 )
 
-__version__ = "1.1.0"
+__version__ = "1.1.2"

{gridpoints-1.1.0 → gridpoints-1.1.2}/src/gridpoints/padding.py

@@ -2,7 +2,7 @@
 Helper to complete point clouds so that they fit a grid.
 """
 
-from .array_libs import arange, astype, empty
+from .array_libs import arange, astype, empty, get_backend
 
 
 def pad(X, n_total, value="nan"):
@@ -26,23 +26,31 @@ def pad(X, n_total, value="nan"):
     X_padded : array (n_total, D) or (B, n_total, D)
     """
     assert value in ("nan", "inf"), f"value must be 'nan' or 'inf', got {value!r}"
-    single = hasattr(X, 'ndim') and X.ndim == 2
-    clouds = [X] if single else list(X)
-    assert max(len(c) for c in clouds) <= n_total, "more points than n_total"
+    is_array = hasattr(X, 'ndim')
+    ref = X if is_array else X[0]
+    assert (X.shape[-2] if is_array else max(len(c) for c in X)) <= n_total, "more points than n_total"
 
-    ref = clouds[0]
     dtype = str(ref.dtype).replace('torch.', '')
     if 'float' not in dtype:   # integers cannot hold nan / inf
         dtype = 'float64'
     D = ref.shape[-1]
-    out = empty((len(clouds), n_total, D), dtype, ref)
-    out[...] = float(value)
+    shape = (n_total, D) if is_array and X.ndim == 2 else (len(X), n_total, D)
 
-    for b, cloud in enumerate(clouds):
-        out[b, :len(cloud)] = astype(cloud, dtype)
-        if value == "inf":   # point k gets the signs given by the bits of k, one per coordinate
-            k = arange(n_total - len(cloud), 'int64', ref)[:, None]
-            bits = (k >> (D - 1 - arange(D, 'int64', ref))[None, :]) & 1
-            out[b, len(cloud):] = astype(1 - 2 * bits, dtype) * float("inf")
+    # 1. Prefill with nan, then copy the true points in.
+    out = empty(shape, dtype, ref)
+    out[...] = float('nan')
+    if is_array:
+        out[..., :X.shape[-2], :] = astype(X, dtype)
+    else:
+        for b, cloud in enumerate(X):
+            out[b, :len(cloud)] = astype(cloud, dtype)
 
-    return out[0] if single else out
+    # 2. Turn the nans into +/-inf: point k, coordinate d gets the sign of bit d of k.
+    if value == "inf":
+        xp, _ = get_backend(out)
+        k = arange(n_total, 'int64', ref)[:, None]
+        d = arange(D, 'int64', ref)[None, :]
+        signs = astype(1 - 2 * ((k // 2 ** (D - 1 - d)) % 2), dtype)   # (n_total, D)
+        out = xp.where(xp.isnan(out), signs * float('inf'), out)
+
+    return out

gridpoints-1.1.2/src/gridpoints.egg-info/PKG-INFO

@@ -0,0 +1,223 @@
+Metadata-Version: 2.4
+Name: gridpoints
+Version: 1.1.2
+Summary: fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid 
+Author: Armand de Cacqueray
+License: MIT
+Project-URL: Homepage, https://neighborhood-grid.github.io/gridpoints/
+Project-URL: Repository, https://github.com/Neighborhood-Grid/gridpoints
+Keywords: sorting,point-clouds,multi-indexing,grid,neighborhood-grid,monotonic-lagrangian-grid,squarenet,nearest-neighbor
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: demo
+Requires-Dist: squarenet; extra == "demo"
+Dynamic: license-file
+
+# Gridpoints
+
+[![PyPI](https://img.shields.io/pypi/v/gridpoints?logo=pypi&logoColor=white)](https://pypi.org/project/gridpoints/)
+[![GitHub](https://img.shields.io/badge/GitHub-gridpoints-181717?logo=github)](https://github.com/Neighborhood-Grid/gridpoints)
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/table(1).gif">
+
+**Reorder an unstructured point cloud into a regular grid, so that neighbor-based operations become simple grid processing.**
+
+```python
+import gridpoints as grid
+points = your_dataset_of_points(1000000, 3)  # standard (N, D) array
+order = grid.argsort(points, gridshape=(100, 100, 100))  # set gridshape: (I, J, K, ...) = (100, 100, 100)
+Pgrid = points[order].reshape(100, 100, 100, 3)  # now an (I, J, K, ...,  D) version of your dataset
+# Ready for convolution, neighbor look-ups or geometrical analysis.
+# Note that grid.argsort also supports a batch mode (B, N, D) -> (B, I, J, K, ..., D)
+```
+
+Gridpoints supports NumPy, PyTorch and CuPy arrays.
+The `argsort()` reordering is a one-to-one assignment of your points to the cells of a grid with the given `gridshape`. This is related to Monge optimal transport, but `argsort()` trades optimality for scalability, by combining greedy axial operations which exploit the specific structure of grids. One million 2D/3D points are sorted in about 0.5 s on a GPU (PyTorch) or 10 s on a CPU (NumPy).
+
+<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/plots/ballexemple.png">
+
+---
+
+## TL;DR
+
+- **Problem:** point clouds are stored as a flat list `[P1, ..., PN]`. Finding the neighbors of a point usually needs a tree or a graph (KD-tree, radius graph...), which is hard to use with standard tensor-based machine learning, especially on GPU.
+- **Idea:** assign every point to a cell of a D-dimensional grid, one point per cell, such that the grid is sorted along each of its axes. Then the neighbors of a point are *approximately* the points in nearby cells, found by plain indexing.
+- **Payoff:** local operations (convolution, smoothing, clustering, interpolation...) that are usually hard to implement on irregular point clouds can be applied on the grid: `points -> grid`. To recover the result on the points, one just applies the inverse assignment: `grid -> points`.
+- **Catch:** a plain grid gives *approximate* neighborhoods (about 99% of true nearest neighbors within a small radius). Exact neighborhoods are recoverable with tiling (see [Neighborhood radius](#neighborhood-radius)). It works best in 2D/3D, and stays reasonable up to about 6D.
+
+---
+
+## The problem
+
+Points are almost always stored in a one-dimensional order. You would like points that are close in space to also be close in memory, so that the nearest neighbor of `P[n]` is often `P[n±1]`, `P[n±2]`, or a bit further. This is a well-studied problem, and space-filling curves such as the Morton curve are a common solution.
+
+Gridpoints takes a different route. Instead of a 1D order, it gives each point a **multi-index** `[i, j, ...]` in a D-dimensional grid. This matters because a grid supports stencil operations ("look at the cells within a few steps in each direction", `[i±di, j±dj, k±dk]`).
+
+## What Gridpoints does
+
+Input: a point cloud `P` of shape `(N, D)` and a target grid shape `(I, J, ...)`.
+
+Output: a permutation `order` of the indices such that
+
+```python
+Pgrid = P[order].reshape(I, J, ..., D)
+```
+
+is a bijection between points and grid cells (one point, one cell), and is **sorted along every axis**. In 3D, with xyz notation `P = (x, y, z)`:
+
+```text
+x[i+1, j, k] >= x[i, j, k]     # x increases along axis 0
+y[i, j+1, k] >= y[i, j, k]     # y increases along axis 1
+z[i, j, k+1] >= z[i, j, k]     # z increases along axis 2
+```
+
+This property (*grid monotonicity*) is the only thing guaranteed. See [Neighborhood radius](#neighborhood-radius) for what it does and does not imply.
+
+On `Pgrid`, a neighbor query becomes a stencil lookup:
+
+```text
+neighborhood(Pgrid[i, j, k]) ≈ { Pgrid[i±di, j±dj, k±dk] : (di, dj, dk) within radius R }
+```
+
+where `R` is a cutoff you choose for your task. Operations built on this run in **O(N)** on the grid view.
+
+---
+
+## Internal algorithm
+
+To get the main ideas of the sorting algorithm, the core of `gridpoints` is:
+1. initialisation: a good starting assignement is important both for quick convergence, and to converge on an assignement with good neighborhood properties. The initialiser build a kdtree on the points, and convert the tree cuts to a grid structure
+2. iterative 1d sorts: once the initialiser built the starting assignement, gridpoints further optimise it iteratively by looking at 1d slices of the grid (axial slices, but also diagonal ones), and apply standard 1d sort on the points, projected along the direction of the slice.
+
+For interested readers, the full sorting procedure and motivation behind is explained in more details [here](https://github.com/Neighborhood-Grid/Cartesian-Grid-Sort). 
+
+## Installation
+
+```bash
+pip install gridpoints          # core only
+pip install gridpoints[demo]    # adds visualisation tools for the demo
+```
+
+## Demo 1. modelnet40:
+To play with real life objects from the Modelnet40 dataset, first get modelnet40 from [princeton.edu](http://modelnet.cs.princeton.edu)  (~2 GB), then:
+```
+python modelnet40_demo.py --obj table --num 6
+python modelnet40_demo.py --obj airplane --num 1
+...
+```
+Animated figures (morphing from the raw  point cloud to a cubic grid) will be saved to sqrnet\plot.gif
+
+## Demo 2. big point clouds:
+To test gridpoints on various synthetic datasets, see the notebook: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
+
+## Quickstart
+
+```python
+import gridpoints as grid
+import numpy as np
+
+N = 1_000_000
+A = np.random.rand(N, 3)                       # raw point cloud (NumPy, PyTorch or CuPy)
+
+# 1. Compute the permutation and view the points as a grid
+order = grid.argsort(A, gridshape=(100, 100, 100))
+Bflat = A[order]                               # sorted, flat (C-order)
+Bgrid = Bflat.reshape(100, 100, 100, 3)        # grid view
+
+# 2. Work on the grid
+Cgrid = apply_something(Bgrid)                 # e.g. a convolution; shape (100, 100, 100, *)
+
+# 3. Go back to the original point order
+Cflat = Cgrid.reshape(N, -1)
+orderinv = grid.invert_permutation(order)
+C = Cflat[orderinv]                            # back to the original point indexing
+```
+
+`apply_something` stands for whatever you want to compute on the grid.
+
+`gridpoints.sort()` is also available.
+
+---
+
+## Choosing the parameters
+
+### Grid shape
+
+The grid shape must be roughly tuned to the *kind* of data, with the constraint that the number of points `N` has to be factorized as `I⋅J⋅K⋅…`
+
+- The exact factorization hardly matters: `(18, 20, 16)` and `(16, 15, 24)` behave similarly.
+- The **orders of magnitude** of each dimension matter a lot: `(18, 20, 16)` and `(36, 40, 4)` can give substantially different results.
+- For a thin surface (e.g. an eggshell), use a flat grid such as `(128, 128, 2)`. A naive `(32, 32, 32)` grid on the same data produces a poor representation of the geometry.
+
+**Padding:** if the number of cells `I⋅J⋅K⋅…` exceeds the number of points `N`, the point cloud can be padded with special values (see [Padding](#padding-with-nan-and-inf)). This lets you sort a prime or variable number of points, and removes the integer-factorization constraint on the grid shape.
+
+### Neighborhood radius
+
+There is no theoretical guarantee on the radius `R` needed for a given task. Nearest neighbors are not guaranteed to land in exactly adjacent cells.
+
+Empirically, in 2D, `R = 5` captures about 99% of nearest neighbors. Some outliers might sit further away, mostly on complex geometries with sharp peaks, holes or other non-smooth features.
+
+If you need stricter neighborhoods, or work in higher dimension, there are two options:
+
+- **Ensemble of grids:** build several grids, each on a rotated or projected view of the points, and combine them (see https://arxiv.org/abs/1505.00880 for the concept of multi view pooling). Projecting the data randomly onto one or several lower-dimensional subspaces before sorting also helps reduce topology-specific effects, since random projections approximately preserve pairwise distances (Johnson–Lindenstrauss lemma).
+- **Tiling:** if you need *exact* neighbors, tiling gives them. Split the grid into tiles, attach metadata to each tile (e.g. a bounding box), and skip pairs of tiles that are certifiably too far apart for your criterion. The grid does the bulk of the work; the tiles only certify or correct the rest.
+
+### Padding with NaN and Inf
+
+NaN and Inf values are supported in a consistent way by the package, so that padding is natural:
+
+- `NaN` → placed at a random position
+- `±Inf` → placed on the border of the grid
+
+This lets you complete the N real points with ΔN fictitious points so that N + ΔN = I⋅J⋅K⋅… , using `gridpoints.pad`
+
+---
+
+## Performance
+
+Sorting 1 million points (2D/3D), measured on Google Colab:
+
+| Backend | Time |
+|---|---|
+| NumPy (CPU) | ~10 s |
+| PyTorch (T4 GPU) | ~0.5 s |
+| CuPy (T4 GPU) | ~1 s (the first, cold call takes ~10 s because of compilation) |
+
+### Running stencil operations efficiently
+
+A typical use looks like:
+
+```text
+output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] )   for (di, dj, dk) in a local window
+```
+
+To compute this kind of kernel efficiently, avoid pure Python loops, which are slow. Prefer:
+
+- Native grid convolutions of standard libraries, whenever your operation can be expressed that way.
+- `pystencils` or `taichi` for complex or non-linear kernels.
+- On GPU, tiling the grid with a custom `triton` kernel can be particularly efficient.
+
+---
+
+## Limits
+
+**Good fit**
+
+- **Low-dimensional point clouds.** 2D and 3D are the sweet spot, with enough points for O(N) grid operations to pay off.
+- **Pipelines that benefit from regular arrays:** GPU processing, convolutions, repeated local operations.
+- **Nice geometries.** "Nice" is deliberately informal. The question is: can you mentally spread your point distribution onto a regular box with the specified dimensions without your brain blowing up in the process? Remarkably, this requires neither convexity (a map of France, a sponge, a donut, an elephant, an eggshell are valid) nor connectedness (clusters and blobs are valid, Gridpoints just glues them together). Examples are in the [plots folder](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots).
+
+**Poor fit**
+
+- **High dimension.** Arbitrary dimension is supported, but the sweet spot is 2D/3D. Dimensions 4 to 6 are still reasonable. Above ~8D, Gridpoints is generally not appropriate. Note that many powerful dimension reduction technique exists beyond random projections ensemble or principal component analysis. Modern ml tools such as auto-encoders or parametric umap could maybe help to convert high dimensional data to the low dimensional framework of gridpoints.
+- **Small point clouds.** Below a few hundred points, the overhead is not worth it: a naive quadratic implementation of your kernels will be simpler and probably faster.
+- **Anti-grid geometries.** A spider web, a wind turbine... The sort still runs, but a single grid sort gives a poor representation of the structure. For such distributions, you currently need a projection + grid ensemble procedure, which must be set up manually. That may make Gridpoints less attractive if you were looking for a light, out-of-the-box gridification framework. I am working on making this more automatic, and I hope that at some point Gridpoints will be easily applicable to a much wider range of geometries. Updates will be posted here!
+
+## Should you use it?
+
+Try it if you process 2D/3D point clouds with local operations and you want regular array layouts (especially on GPU). Approximate neighborhoods work out of the box, and exact ones are available through tiling.
+
+Stay with a KD-tree or a graph method if you work in high dimension or have only a few hundred points. If you need exact nearest neighbors, Gridpoints can still work, via tiling, but expect some extra implementation effort.
+
+A quick way to decide is to run the [demo notebook](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb) on a sample of your own data and check how much of your true nearest neighbors fall within your radius `R`.

gridpoints-1.1.0/PKG-INFO

@@ -1,116 +0,0 @@
-Metadata-Version: 2.4
-Name: gridpoints
-Version: 1.1.0
-Summary: fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid 
-Author: Armand de Cacqueray
-License: MIT
-Project-URL: Homepage, https://neighborhood-grid.github.io/gridpoints/
-Project-URL: Repository, https://github.com/Neighborhood-Grid/gridpoints
-Keywords: sorting,point-clouds,multi-indexing,grid,neighborhood-grid,monotonic-lagrangian-grid,squarenet,nearest-neighbor
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Provides-Extra: demo
-Requires-Dist: squarenet; extra == "demo"
-Dynamic: license-file
-
-## ❒ CubeNet: Multi dimensional sort for point clouds 
-# with the gridpoints python package
-
-`Gridpoints` maps raw indexes of unstructured **point clouds** to structured grids multi-index  through a **bijective transformation**: 
-One point, one cell, 
-```text
-n <-> [i,j,…](n).
-```
-It replaces and enhances the [squarenet](https://github.com/Neighborhood-Grid/SquareNet) project with a more robust points sorting algorithm. The goal is 
-to build a spatialy coherent multi index to later store and query the point cloud efficiently, based on adaptative axes (lines, columns, and more in 3D+) as showed on following exemple :
-
-<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/ballexemple.png">
-
-What it does: Take raw point cloud `P(N, D)`  and find a grid shape and an index permutation `order` such that `Pgrid(I, J, …, D)` = P[order].reshape(*gridshape, D) is sorted along every axis of the grid. E.g. in 3D, for Pgrid = (x, y, z):
-```text
-x[i+1, j, k] >= x[i, j, k]
-y[i, j+1, k] >= y[i, j, k]
-z[i, j, k+1] >= z[i, j, k]
-```
-
-→ On the `Pgrid` view of `P`, neighbor queries become a simple stencil look-up
-```text
-neighborhood(Pgrid[i, j, k]) = {Pgrid[i±di, j±dj, k±dk] | (di, dj, dk) ≤ R},
-````
-where `R` is a radius cutoff to determine, allowing local operations in O(N) time.  
-→ Geometric operations (convolution, clustering, interpolation …) can then be applied **in O(N) time** directly on the `Pgrid` view instead of relying on complex graph or point-cloud methods
-
-`P` can be a NumPy, PyTorch or CuPy array of any dimension (N, D). To allow natural padding when the grid has more cells than they are points in the cloud, NaNs and Infs are supported in a consistent manner: 
-- nans → random position
-- (+-) infs → border of the grid
-  
-This allow to gridsort prime or variable number of points, as long as one is ready to deal with void/special grid cells.
-
-Expected runtime for sorting 1 million points, tested on google colab GPU T4 for 2D/3D point clouds: 
-- Numpy: 10s
-- Torch (gpu): 500 ms
-- Cupy (gpu): 1 s (after compilation e.g. not on the first cold call, which take 10 s)
-
-`When not to use gridpoints ?`
-- high dimension: the package is implemented to support arbitrary dimension, but sweetspot is really 2D/3D. dimensions 4-6 might still be reasonable depending on the task, but anything above 8D is generally too high dimensional for gridpoints.
-- weird geometries. Supported datasets goes beyond smooth convex distributions: map of Indonesia, a sponge, a donuts, an elephant, an eggshell (by specifying a suitable gridshape e.g. (128,128,2) to `gridpoints.sort()`). You can see various exemples in the [plot](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots) folder But with some limits. Bad fits: a spider web, a wind turbine, same eggshell with a naive 3d sort (gridshape = (32,32,32)) ... the issue is not that gridpoints cannot sort these distributions, but that it will produce a poor representation of the geometry.
-- small point clouds: beyond a few hundred points, local operations in O(N) time is not worth the overhead, because naive quadratic implementations will probably be simultaneously simpler and faster.
-
----
-
-### Installation
-
-```bash
-pip install gridpoints          # core only
-pip install gridpoints[demo]    # for the demonstration notebook.
-```
-Full Demo: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
-
-### Quickstart
-
-```python
-import gridpoints as grid
-import numpy as np
-
-# Raw point cloud (numpy, pytorch or cupy)
-A = np.random.rand(1_000_000, 3)
-
-# Sorted view: place the points inside the grid
-order = grid.argsort(A, gridshape=(100, 100, 100))
-Bflat = A[order] #flat but sorted with a grid layout (C-order)
-Bgrid = Bflat.reshape(100, 100, 100, 3) #reshaped as a grid 
-
-# Rest of your pipeline, working with grids
-Cgrid = apply_something(Bgrid) #(100, 100, 100, *)
-
-# Back to the original points indexing
-Cflat = Cgrid.reshape(1_000_000, *)
-orderinv = grid.invert_permutation(order)
-C = Cflat[orderinv]   # matches the initial points order
-```
-
----
-
-### Note on rectangular gridshapes
-
-The eggshell example illustrates the importance of selecting a suitable grid-shape decomposition. This can itself be challenging when the point cloud is assumed to follow a complex topological pattern, such as a thin surface rather than a full volumetric domain. While the precise decomposition is not particularly important (e.g. (18,20,16) vs. (16,15,24) will generally make little difference), the orders of magnitude of the different dimensions do matter and should be tuned to the dataset: (18,20,16) and (36,40,4) can lead to substantially different results. As mentioned, nan/inf padding can help accomodate integer-factorization constraints
-
-Another possibility for smoothing out topology-specific effects is to perform the sort after randomly projecting the dataset onto a lower-dimensional subspace. Random projections are known to approximately preserve pairwise distances, as formalized by the Johnson–Lindenstrauss lemma.
-
-
-### Note on the cutoff radius R
-
-There is no strict theoretical guarantee about what the cutof radius R should be for a given task. E.g the relative grid position between a point and its nearest neighbors can't be garanted to be in the exact adjacent grid cells. What is guaranteed from the sorted ordering is only **grid monotonicity**: *x* coordinates increase along rows, *y* coordinates along columns, and so on.
-
-As an example, empirical results in 2-D show that `R = 5` is enough for ~99 % of the nearest neighbors; some outlier neighbors will sit further apart for complex geometries with pronounced peaks, holes or any non-smoothness. When a stricter neighborhood is required, or in high dimensional setting, the solution might be to build an assembly of grid experts, each working on a rotated / projected view of the points, as discussed in [this topic](https://github.com/glotzerlab/freud/discussions/1417). An other possibility would be to tile the grid: each tile is enhanced with location metadata (e.g. a bounding box), allowing for pruning pairs of tiles with a distance certified to be far enough for a given criterion.
-
-
-### Note on efficient stencil operations
-
-The typical use-case of `Gridpoints` is to allow fast local operations on arbitrary point clouds using stencil kernels:
-
-```text
-output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] | di, dj, dk in local window )
-```
-To go beyond standard (slow) python loops, this kind of kernel computation can be accelerated with native grid convolution operations of standard libraries whenever possible, or with `pystencils` or `taichi` compilers for complex/non O(N) grid kernels. On GPU, tiling the grid and leveraging a custom `triton` kernel might be particularly efficient.

gridpoints-1.1.0/README.md

@@ -1,100 +0,0 @@
-## ❒ CubeNet: Multi dimensional sort for point clouds 
-# with the gridpoints python package
-
-`Gridpoints` maps raw indexes of unstructured **point clouds** to structured grids multi-index  through a **bijective transformation**: 
-One point, one cell, 
-```text
-n <-> [i,j,…](n).
-```
-It replaces and enhances the [squarenet](https://github.com/Neighborhood-Grid/SquareNet) project with a more robust points sorting algorithm. The goal is 
-to build a spatialy coherent multi index to later store and query the point cloud efficiently, based on adaptative axes (lines, columns, and more in 3D+) as showed on following exemple :
-
-<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/ballexemple.png">
-
-What it does: Take raw point cloud `P(N, D)`  and find a grid shape and an index permutation `order` such that `Pgrid(I, J, …, D)` = P[order].reshape(*gridshape, D) is sorted along every axis of the grid. E.g. in 3D, for Pgrid = (x, y, z):
-```text
-x[i+1, j, k] >= x[i, j, k]
-y[i, j+1, k] >= y[i, j, k]
-z[i, j, k+1] >= z[i, j, k]
-```
-
-→ On the `Pgrid` view of `P`, neighbor queries become a simple stencil look-up
-```text
-neighborhood(Pgrid[i, j, k]) = {Pgrid[i±di, j±dj, k±dk] | (di, dj, dk) ≤ R},
-````
-where `R` is a radius cutoff to determine, allowing local operations in O(N) time.  
-→ Geometric operations (convolution, clustering, interpolation …) can then be applied **in O(N) time** directly on the `Pgrid` view instead of relying on complex graph or point-cloud methods
-
-`P` can be a NumPy, PyTorch or CuPy array of any dimension (N, D). To allow natural padding when the grid has more cells than they are points in the cloud, NaNs and Infs are supported in a consistent manner: 
-- nans → random position
-- (+-) infs → border of the grid
-  
-This allow to gridsort prime or variable number of points, as long as one is ready to deal with void/special grid cells.
-
-Expected runtime for sorting 1 million points, tested on google colab GPU T4 for 2D/3D point clouds: 
-- Numpy: 10s
-- Torch (gpu): 500 ms
-- Cupy (gpu): 1 s (after compilation e.g. not on the first cold call, which take 10 s)
-
-`When not to use gridpoints ?`
-- high dimension: the package is implemented to support arbitrary dimension, but sweetspot is really 2D/3D. dimensions 4-6 might still be reasonable depending on the task, but anything above 8D is generally too high dimensional for gridpoints.
-- weird geometries. Supported datasets goes beyond smooth convex distributions: map of Indonesia, a sponge, a donuts, an elephant, an eggshell (by specifying a suitable gridshape e.g. (128,128,2) to `gridpoints.sort()`). You can see various exemples in the [plot](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots) folder But with some limits. Bad fits: a spider web, a wind turbine, same eggshell with a naive 3d sort (gridshape = (32,32,32)) ... the issue is not that gridpoints cannot sort these distributions, but that it will produce a poor representation of the geometry.
-- small point clouds: beyond a few hundred points, local operations in O(N) time is not worth the overhead, because naive quadratic implementations will probably be simultaneously simpler and faster.
-
----
-
-### Installation
-
-```bash
-pip install gridpoints          # core only
-pip install gridpoints[demo]    # for the demonstration notebook.
-```
-Full Demo: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
-
-### Quickstart
-
-```python
-import gridpoints as grid
-import numpy as np
-
-# Raw point cloud (numpy, pytorch or cupy)
-A = np.random.rand(1_000_000, 3)
-
-# Sorted view: place the points inside the grid
-order = grid.argsort(A, gridshape=(100, 100, 100))
-Bflat = A[order] #flat but sorted with a grid layout (C-order)
-Bgrid = Bflat.reshape(100, 100, 100, 3) #reshaped as a grid 
-
-# Rest of your pipeline, working with grids
-Cgrid = apply_something(Bgrid) #(100, 100, 100, *)
-
-# Back to the original points indexing
-Cflat = Cgrid.reshape(1_000_000, *)
-orderinv = grid.invert_permutation(order)
-C = Cflat[orderinv]   # matches the initial points order
-```
-
----
-
-### Note on rectangular gridshapes
-
-The eggshell example illustrates the importance of selecting a suitable grid-shape decomposition. This can itself be challenging when the point cloud is assumed to follow a complex topological pattern, such as a thin surface rather than a full volumetric domain. While the precise decomposition is not particularly important (e.g. (18,20,16) vs. (16,15,24) will generally make little difference), the orders of magnitude of the different dimensions do matter and should be tuned to the dataset: (18,20,16) and (36,40,4) can lead to substantially different results. As mentioned, nan/inf padding can help accomodate integer-factorization constraints
-
-Another possibility for smoothing out topology-specific effects is to perform the sort after randomly projecting the dataset onto a lower-dimensional subspace. Random projections are known to approximately preserve pairwise distances, as formalized by the Johnson–Lindenstrauss lemma.
-
-
-### Note on the cutoff radius R
-
-There is no strict theoretical guarantee about what the cutof radius R should be for a given task. E.g the relative grid position between a point and its nearest neighbors can't be garanted to be in the exact adjacent grid cells. What is guaranteed from the sorted ordering is only **grid monotonicity**: *x* coordinates increase along rows, *y* coordinates along columns, and so on.
-
-As an example, empirical results in 2-D show that `R = 5` is enough for ~99 % of the nearest neighbors; some outlier neighbors will sit further apart for complex geometries with pronounced peaks, holes or any non-smoothness. When a stricter neighborhood is required, or in high dimensional setting, the solution might be to build an assembly of grid experts, each working on a rotated / projected view of the points, as discussed in [this topic](https://github.com/glotzerlab/freud/discussions/1417). An other possibility would be to tile the grid: each tile is enhanced with location metadata (e.g. a bounding box), allowing for pruning pairs of tiles with a distance certified to be far enough for a given criterion.
-
-
-### Note on efficient stencil operations
-
-The typical use-case of `Gridpoints` is to allow fast local operations on arbitrary point clouds using stencil kernels:
-
-```text
-output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] | di, dj, dk in local window )
-```
-To go beyond standard (slow) python loops, this kind of kernel computation can be accelerated with native grid convolution operations of standard libraries whenever possible, or with `pystencils` or `taichi` compilers for complex/non O(N) grid kernels. On GPU, tiling the grid and leveraging a custom `triton` kernel might be particularly efficient.

gridpoints-1.1.0/src/gridpoints.egg-info/PKG-INFO

@@ -1,116 +0,0 @@
-Metadata-Version: 2.4
-Name: gridpoints
-Version: 1.1.0
-Summary: fast multidimensional sorting tool for point clouds using the monotonic lagrangian grid 
-Author: Armand de Cacqueray
-License: MIT
-Project-URL: Homepage, https://neighborhood-grid.github.io/gridpoints/
-Project-URL: Repository, https://github.com/Neighborhood-Grid/gridpoints
-Keywords: sorting,point-clouds,multi-indexing,grid,neighborhood-grid,monotonic-lagrangian-grid,squarenet,nearest-neighbor
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Provides-Extra: demo
-Requires-Dist: squarenet; extra == "demo"
-Dynamic: license-file
-
-## ❒ CubeNet: Multi dimensional sort for point clouds 
-# with the gridpoints python package
-
-`Gridpoints` maps raw indexes of unstructured **point clouds** to structured grids multi-index  through a **bijective transformation**: 
-One point, one cell, 
-```text
-n <-> [i,j,…](n).
-```
-It replaces and enhances the [squarenet](https://github.com/Neighborhood-Grid/SquareNet) project with a more robust points sorting algorithm. The goal is 
-to build a spatialy coherent multi index to later store and query the point cloud efficiently, based on adaptative axes (lines, columns, and more in 3D+) as showed on following exemple :
-
-<img src="https://raw.githubusercontent.com/Neighborhood-Grid/CubeNet/main/ballexemple.png">
-
-What it does: Take raw point cloud `P(N, D)`  and find a grid shape and an index permutation `order` such that `Pgrid(I, J, …, D)` = P[order].reshape(*gridshape, D) is sorted along every axis of the grid. E.g. in 3D, for Pgrid = (x, y, z):
-```text
-x[i+1, j, k] >= x[i, j, k]
-y[i, j+1, k] >= y[i, j, k]
-z[i, j, k+1] >= z[i, j, k]
-```
-
-→ On the `Pgrid` view of `P`, neighbor queries become a simple stencil look-up
-```text
-neighborhood(Pgrid[i, j, k]) = {Pgrid[i±di, j±dj, k±dk] | (di, dj, dk) ≤ R},
-````
-where `R` is a radius cutoff to determine, allowing local operations in O(N) time.  
-→ Geometric operations (convolution, clustering, interpolation …) can then be applied **in O(N) time** directly on the `Pgrid` view instead of relying on complex graph or point-cloud methods
-
-`P` can be a NumPy, PyTorch or CuPy array of any dimension (N, D). To allow natural padding when the grid has more cells than they are points in the cloud, NaNs and Infs are supported in a consistent manner: 
-- nans → random position
-- (+-) infs → border of the grid
-  
-This allow to gridsort prime or variable number of points, as long as one is ready to deal with void/special grid cells.
-
-Expected runtime for sorting 1 million points, tested on google colab GPU T4 for 2D/3D point clouds: 
-- Numpy: 10s
-- Torch (gpu): 500 ms
-- Cupy (gpu): 1 s (after compilation e.g. not on the first cold call, which take 10 s)
-
-`When not to use gridpoints ?`
-- high dimension: the package is implemented to support arbitrary dimension, but sweetspot is really 2D/3D. dimensions 4-6 might still be reasonable depending on the task, but anything above 8D is generally too high dimensional for gridpoints.
-- weird geometries. Supported datasets goes beyond smooth convex distributions: map of Indonesia, a sponge, a donuts, an elephant, an eggshell (by specifying a suitable gridshape e.g. (128,128,2) to `gridpoints.sort()`). You can see various exemples in the [plot](https://github.com/Neighborhood-Grid/CubeNet/blob/main/plots) folder But with some limits. Bad fits: a spider web, a wind turbine, same eggshell with a naive 3d sort (gridshape = (32,32,32)) ... the issue is not that gridpoints cannot sort these distributions, but that it will produce a poor representation of the geometry.
-- small point clouds: beyond a few hundred points, local operations in O(N) time is not worth the overhead, because naive quadratic implementations will probably be simultaneously simpler and faster.
-
----
-
-### Installation
-
-```bash
-pip install gridpoints          # core only
-pip install gridpoints[demo]    # for the demonstration notebook.
-```
-Full Demo: [notebook.ipynb](https://github.com/Neighborhood-Grid/CubeNet/blob/main/notebook.ipynb)
-
-### Quickstart
-
-```python
-import gridpoints as grid
-import numpy as np
-
-# Raw point cloud (numpy, pytorch or cupy)
-A = np.random.rand(1_000_000, 3)
-
-# Sorted view: place the points inside the grid
-order = grid.argsort(A, gridshape=(100, 100, 100))
-Bflat = A[order] #flat but sorted with a grid layout (C-order)
-Bgrid = Bflat.reshape(100, 100, 100, 3) #reshaped as a grid 
-
-# Rest of your pipeline, working with grids
-Cgrid = apply_something(Bgrid) #(100, 100, 100, *)
-
-# Back to the original points indexing
-Cflat = Cgrid.reshape(1_000_000, *)
-orderinv = grid.invert_permutation(order)
-C = Cflat[orderinv]   # matches the initial points order
-```
-
----
-
-### Note on rectangular gridshapes
-
-The eggshell example illustrates the importance of selecting a suitable grid-shape decomposition. This can itself be challenging when the point cloud is assumed to follow a complex topological pattern, such as a thin surface rather than a full volumetric domain. While the precise decomposition is not particularly important (e.g. (18,20,16) vs. (16,15,24) will generally make little difference), the orders of magnitude of the different dimensions do matter and should be tuned to the dataset: (18,20,16) and (36,40,4) can lead to substantially different results. As mentioned, nan/inf padding can help accomodate integer-factorization constraints
-
-Another possibility for smoothing out topology-specific effects is to perform the sort after randomly projecting the dataset onto a lower-dimensional subspace. Random projections are known to approximately preserve pairwise distances, as formalized by the Johnson–Lindenstrauss lemma.
-
-
-### Note on the cutoff radius R
-
-There is no strict theoretical guarantee about what the cutof radius R should be for a given task. E.g the relative grid position between a point and its nearest neighbors can't be garanted to be in the exact adjacent grid cells. What is guaranteed from the sorted ordering is only **grid monotonicity**: *x* coordinates increase along rows, *y* coordinates along columns, and so on.
-
-As an example, empirical results in 2-D show that `R = 5` is enough for ~99 % of the nearest neighbors; some outlier neighbors will sit further apart for complex geometries with pronounced peaks, holes or any non-smoothness. When a stricter neighborhood is required, or in high dimensional setting, the solution might be to build an assembly of grid experts, each working on a rotated / projected view of the points, as discussed in [this topic](https://github.com/glotzerlab/freud/discussions/1417). An other possibility would be to tile the grid: each tile is enhanced with location metadata (e.g. a bounding box), allowing for pruning pairs of tiles with a distance certified to be far enough for a given criterion.
-
-
-### Note on efficient stencil operations
-
-The typical use-case of `Gridpoints` is to allow fast local operations on arbitrary point clouds using stencil kernels:
-
-```text
-output(i, j, k) = f( Pgrid[i±di, j±dj, k±dk] | di, dj, dk in local window )
-```
-To go beyond standard (slow) python loops, this kind of kernel computation can be accelerated with native grid convolution operations of standard libraries whenever possible, or with `pystencils` or `taichi` compilers for complex/non O(N) grid kernels. On GPU, tiling the grid and leveraging a custom `triton` kernel might be particularly efficient.