binsparse 0.1.2__tar.gz → 0.1.4__tar.gz

{binsparse-0.1.2 → binsparse-0.1.4}/.github/workflows/publish.yml

@@ -21,8 +21,6 @@ jobs:
       - uses: prefix-dev/setup-pixi@v0.10.0
         with:
           pixi-version: v0.71.3
-          auth-host: prefix.dev
-          auth-token: ${{ secrets.PREFIX_API_KEY }}
       - name: Get version
         run: |
           VERSION=$(python -c 'import tomllib; print("v" + tomllib.load(open("pyproject.toml", "rb"))["project"]["version"])')
@@ -33,7 +31,7 @@ jobs:
         uses: pypa/gh-action-pypi-publish@release/v1
         with:
           skip-existing: true
-      - name: Build Prefix.dev package
+      - name: Build Conda package
         run: pixi publish --target-dir "$RUNNER_TEMP/binsparse-conda"
       - name: Publish to Prefix.dev
         env:
@@ -42,6 +40,7 @@ jobs:
           pixi upload prefix
           --channel willow-ahrens/binsparse
           --api-key "$PREFIX_API_KEY"
+          --skip-existing
           "$RUNNER_TEMP"/binsparse-conda/*.conda
       - name: Create a release
         uses: softprops/action-gh-release@v3

binsparse-0.1.4/PKG-INFO

@@ -0,0 +1,92 @@
+Metadata-Version: 2.5
+Name: binsparse
+Version: 0.1.4
+Summary: Python reference implementation of the Binsparse binary sparse format
+Author-email: Willow Ahrens <ahrens@gatech.edu>
+Requires-Python: >=3.11
+Requires-Dist: h5py<4,>=3.16.0
+Requires-Dist: numpy<3,>=2
+Requires-Dist: semver<4,>=3
+Provides-Extra: conversions
+Requires-Dist: scipy; extra == 'conversions'
+Requires-Dist: sparse; extra == 'conversions'
+Requires-Dist: torch; extra == 'conversions'
+Provides-Extra: scipy
+Requires-Dist: scipy; extra == 'scipy'
+Provides-Extra: sparse
+Requires-Dist: sparse; extra == 'sparse'
+Provides-Extra: torch
+Requires-Dist: torch; extra == 'torch'
+Description-Content-Type: text/markdown
+
+# Binsparse Python Reference Implementation
+
+This library is a reference implementation of the [Binsparse Binary Sparse Format Specification](https://binsparse.org) written using Python.
+
+Binsparse is a cross-platform, embeddable format for storing sparse matrices and
+tensors. This library implements Binsparse bindings for
+[NumPy NPZ](https://numpy.org/doc/stable/reference/generated/numpy.savez.html),
+[HDF5](https://www.hdfgroup.org/solutions/hdf5/), and
+[Zarr](https://zarr.dev/) containers, with conversion interfaces for
+[NumPy](https://numpy.org/), [SciPy](https://scipy.org/),
+[PyTorch](https://pytorch.org/), and
+[PyData/Sparse](https://sparse.pydata.org/).
+
+## Python Binsparse Interface
+
+The primary interface consists of `load_binsparse` and `save_binsparse`. These
+functions select a container from the path extension (`.npz`, `.h5`/`.hdf5`, or
+`.zarr`) and read or write a `BinsparseTensor`:
+
+```python
+from binsparse import load_binsparse, save_binsparse
+
+tensor = load_binsparse("input.h5")
+save_binsparse(tensor, "output.npz")
+```
+
+Parsed tensors expose their logical `shape`, `number_of_stored_values`, optional
+fill value, and the arrays that define their storage format. Concrete tensor
+classes represent the predefined Binsparse formats, including dense vectors and
+matrices, compressed sparse rows and columns, doubly compressed matrices, and
+coordinate formats. `CustomTensor` and the level classes (`DenseLevel`,
+`SparseLevel`, and `ElementLevel`) represent arbitrary level-based format
+descriptors.
+
+For applications that already manage their own storage objects, the lower-level
+`BinsparseContainer` interface separates tensor parsing and serialization from
+the physical container. Adapters are provided for NPZ mappings, HDF5 files,
+Zarr groups, and in-memory descriptors and buffers. Use
+`BinsparseTensor.parse(container)` to read through an adapter and
+`tensor.serialize(container)` to write through one.
+
+The `alias` option controls whether predefined format names such as `CSR` are
+preserved or expanded to custom level descriptors. The `copy` option allows a
+caller to request a copy, permit whichever representation is required, or
+require a zero-copy operation when the source and destination support it.
+
+`binsparse.conversions` exports the following conversion functions:
+
+- NumPy: `from_numpy` and `to_numpy`
+- SciPy: `from_scipy` and `to_scipy`
+- PyTorch: `from_torch` and `to_torch`
+- PyData/Sparse: `from_sparse` and `to_sparse`
+
+NumPy support is included by default; the other adapters require their
+corresponding optional dependency.
+
+SciPy, PyData/Sparse, and PyTorch COO imports sort indices and sum duplicate
+entries before constructing a Binsparse tensor, without modifying the source.
+`copy=False` requires canonical (sorted, duplicate-free) input; PyTorch COO
+tensors must be coalesced first. PyTorch CSR/CSC imports rely on PyTorch's
+sorted, unique index invariants.
+
+## Source
+The source code for `binsparse` is available on GitHub at [https://github.com/Binsparse/binsparse-reference-python](https://github.com/Binsparse/binsparse-reference-python)
+
+## Installation
+
+`binsparse` is available on PyPi, and can be installed with pip:
+```bash
+pip install binsparse
+```

{binsparse-0.1.2 → binsparse-0.1.4}/README.md

@@ -54,6 +54,12 @@ require a zero-copy operation when the source and destination support it.
 NumPy support is included by default; the other adapters require their
 corresponding optional dependency.
 
+SciPy, PyData/Sparse, and PyTorch COO imports sort indices and sum duplicate
+entries before constructing a Binsparse tensor, without modifying the source.
+`copy=False` requires canonical (sorted, duplicate-free) input; PyTorch COO
+tensors must be coalesced first. PyTorch CSR/CSC imports rely on PyTorch's
+sorted, unique index invariants.
+
 ## Source
 The source code for `binsparse` is available on GitHub at [https://github.com/Binsparse/binsparse-reference-python](https://github.com/Binsparse/binsparse-reference-python)
 

{binsparse-0.1.2 → binsparse-0.1.4}/pyproject.toml

@@ -1,8 +1,10 @@
 [project]
 authors = [{name = "Willow Ahrens", email = "ahrens@gatech.edu"}]
 name = "binsparse"
+description = "Python reference implementation of the Binsparse binary sparse format"
+readme = "README.md"
 requires-python = ">= 3.11"
-version = "0.1.2"
+version = "0.1.4"
 dependencies = [
     "h5py>=3.16.0,<4",
     "numpy>=2,<3",
@@ -60,8 +62,8 @@ preview = ["pixi-build"]
 requires-pixi = ">=0.71.0"
 
 [tool.pixi.package]
-name = "binsparse"
-version = "0.1.0"
+name = { workspace = true }
+version = { workspace = true }
 
 [tool.pixi.package.build]
 backend = { name = "pixi-build-python", version = "0.*", channels = ["https://prefix.dev/conda-forge"] }

{binsparse-0.1.2 → binsparse-0.1.4}/src/binsparse/conversions/scipy.py

@@ -26,7 +26,10 @@ def _prepare(value: Any, copy: bool | None) -> Any:
 
 
 def from_scipy(value: Any, *, copy: bool | None = None) -> BinsparseTensor:
-    """Convert a two-dimensional SciPy CSR, CSC, or COO object to Binsparse."""
+    """Convert SciPy CSR, CSC, or COO to Binsparse, sorting and summing duplicates.
+
+    The input is not modified. ``copy=False`` requires canonical input.
+    """
     scipy_sparse = _scipy_sparse()
     if not scipy_sparse.issparse(value) or value.ndim != 2:
         raise TypeError("expected a two-dimensional SciPy sparse array or matrix")

{binsparse-0.1.2 → binsparse-0.1.4}/src/binsparse/conversions/sparse.py

@@ -25,12 +25,28 @@ def _sparse() -> Any:
 
 
 def from_sparse(value: Any, *, copy: bool | None = None) -> BinsparseTensor:
-    """Convert an N-dimensional PyData/Sparse COO array to Binsparse."""
+    """Convert PyData/Sparse COO to Binsparse, sorting and summing duplicates.
+
+    The input is not modified. ``copy=False`` requires canonical input.
+    """
     sparse = _sparse()
     if not isinstance(value, sparse.COO) or value.ndim < 1:
         raise TypeError("expected a non-scalar PyData/Sparse COO array")
     if copy is True:
         value = value.copy(deep=True)
+    if copy is False:
+        linear = value.linear_loc()
+        if not np.all(linear[1:] > linear[:-1]):
+            raise ValueError("copy=False cannot canonicalize a PyData/Sparse array")
+    else:
+        value = sparse.COO(
+            value.coords,
+            value.data,
+            shape=value.shape,
+            fill_value=value.fill_value,
+            sorted=False,
+            has_duplicates=True,
+        )
     fill_value = np.asarray(value.fill_value).item()
     shape = tuple(value.shape)
     count = int(value.data.size)

{binsparse-0.1.2 → binsparse-0.1.4}/src/binsparse/conversions/torch.py

@@ -51,7 +51,12 @@ def _numpy(value: Any, copy: bool | None) -> np.ndarray:
 
 
 def from_torch(value: Any, *, copy: bool | None = None) -> BinsparseTensor:
-    """Convert a PyTorch dense, COO, CSR, or CSC tensor to Binsparse."""
+    """Convert PyTorch dense, COO, CSR, or CSC to Binsparse.
+
+    COO entries are sorted and duplicates summed without modifying the input.
+    ``copy=False`` requires coalesced COO input. CSR/CSC inputs are assumed to
+    satisfy PyTorch's sorted, unique index invariants.
+    """
     torch = _torch()
     if not isinstance(value, torch.Tensor):
         raise TypeError("expected a PyTorch tensor")
@@ -61,10 +66,16 @@ def from_torch(value: Any, *, copy: bool | None = None) -> BinsparseTensor:
         return from_numpy(array, copy=False if copy is True else copy)
 
     if value.layout == torch.sparse_coo:
-        indices = _numpy(value._indices(), copy)
-        values = _numpy(value._values(), copy)
         if value.dense_dim() != 0:
             raise TypeError("hybrid sparse COO tensors are not supported")
+        if not value.is_coalesced():
+            if copy is False:
+                raise ValueError(
+                    "copy=False cannot canonicalize a PyTorch sparse tensor"
+                )
+            value = value.coalesce()
+        indices = _numpy(value._indices(), copy)
+        values = _numpy(value._values(), copy)
         if value.sparse_dim() == 2:
             return COORMatrix(
                 tuple(value.shape),
@@ -79,10 +90,7 @@ def from_torch(value: Any, *, copy: bool | None = None) -> BinsparseTensor:
             level=SparseLevel(
                 value.sparse_dim(),
                 ElementLevel(values),
-                tuple(
-                    indices[dimension, :]
-                    for dimension in range(value.sparse_dim())
-                ),
+                tuple(indices[dimension, :] for dimension in range(value.sparse_dim())),
             ),
         )
 

{binsparse-0.1.2 → binsparse-0.1.4}/src/tests/test_conversion_scipy.py

@@ -28,22 +28,36 @@ def test_scipy_round_trip(constructor, tensor_type) -> None:
     assert result.dtype == source.dtype
 
 
-def test_scipy_csr_is_sorted_and_unique() -> None:
-    source = scipy_sparse.csr_array(
+@pytest.mark.parametrize(
+    "constructor", [scipy_sparse.csr_array, scipy_sparse.csc_array]
+)
+def test_scipy_compressed_is_sorted_and_unique(constructor) -> None:
+    source = constructor(
         (
             np.array([1, 2, 3, 4, 5, 6]),
             np.array([2, 1, 2, 1, 0, 1]),
             np.array([0, 3, 6]),
         ),
-        shape=(2, 3),
+        shape=(2, 3) if constructor is scipy_sparse.csr_array else (3, 2),
     )
+    original = (source.indptr.copy(), source.indices.copy(), source.data.copy())
     tensor = from_scipy(source)
     result = to_scipy(tensor)
 
+    assert isinstance(tensor, (CSRMatrix, CSCMatrix))
     assert tensor.number_of_stored_values == 4
+    np.testing.assert_array_equal(tensor.pointers_to_1, [0, 2, 4])
+    np.testing.assert_array_equal(tensor.indices_1, [1, 2, 0, 1])
+    np.testing.assert_array_equal(tensor.values, [2, 4, 5, 10])
     assert result.has_canonical_format
     np.testing.assert_array_equal(result.toarray(), source.toarray())
     assert not source.has_canonical_format
+    for actual, expected in zip(
+        (source.indptr, source.indices, source.data), original, strict=True
+    ):
+        np.testing.assert_array_equal(actual, expected)
+    with pytest.raises(ValueError, match="canonicalize"):
+        from_scipy(source, copy=False)
 
 
 def test_scipy_coo_is_sorted_and_unique() -> None:
@@ -57,7 +71,11 @@ def test_scipy_coo_is_sorted_and_unique() -> None:
     tensor = from_scipy(source)
     result = to_scipy(tensor)
 
+    assert isinstance(tensor, COORMatrix)
     assert tensor.number_of_stored_values == 2
+    np.testing.assert_array_equal(tensor.indices_0, [0, 1])
+    np.testing.assert_array_equal(tensor.indices_1, [2, 0])
+    np.testing.assert_array_equal(tensor.values, [6, 4])
     assert result.has_canonical_format
     np.testing.assert_array_equal(result.toarray(), source.toarray())
     assert not source.has_canonical_format
@@ -88,7 +106,6 @@ def test_scipy_copy_false_rejects_required_input_canonicalization() -> None:
         from_scipy(source, copy=False)
 
 
-
 def test_scipy_canonical_coo_copy_false_shares_buffers() -> None:
     source = scipy_sparse.coo_array(
         (np.array([3]), (np.array([0]), np.array([1]))),

{binsparse-0.1.2 → binsparse-0.1.4}/src/tests/test_conversion_sparse.py

@@ -9,6 +9,39 @@ from binsparse.tensor import COORMatrix, CustomTensor, ElementLevel, SparseLevel
 sparse = pytest.importorskip("sparse")
 
 
+@pytest.mark.parametrize("ndim", [1, 2, 3])
+@pytest.mark.parametrize("copy", [None, True, False])
+@pytest.mark.parametrize("duplicates", [False, True])
+def test_pydata_sparse_is_sorted_and_unique(ndim, copy, duplicates) -> None:
+    coordinates = [2, 0, 2, 1] if duplicates else [2, 0, 1]
+    coords = np.array([coordinates] * ndim)
+    data = np.arange(1, len(coordinates) + 1, dtype=np.float32)
+    # Bypass the constructor's normalization to exercise noncanonical buffers.
+    source = sparse.COO(
+        coords.copy(),
+        data.copy(),
+        shape=(3,) * ndim,
+        sorted=True,
+        has_duplicates=False,
+        fill_value=7,
+    )
+    if copy is False:
+        with pytest.raises(ValueError, match="canonicalize"):
+            from_sparse(source, copy=copy)
+    else:
+        tensor = from_sparse(source, copy=copy)
+        result = to_sparse(tensor)
+        assert tensor.number_of_stored_values == 3
+        assert tensor.fill_value == 7
+        assert result.dtype == data.dtype
+        np.testing.assert_array_equal(result.coords, [[0, 1, 2]] * ndim)
+        np.testing.assert_array_equal(
+            result.data, [2, 4, 4] if duplicates else [2, 3, 1]
+        )
+    np.testing.assert_array_equal(source.coords, coords)
+    np.testing.assert_array_equal(source.data, data)
+
+
 def test_pydata_sparse_round_trip() -> None:
     source = sparse.COO.from_numpy(np.array([[0, 2, 0], [3, 0, 4]], dtype=np.float32))
     tensor = from_sparse(source)

{binsparse-0.1.2 → binsparse-0.1.4}/src/tests/test_conversion_torch.py

@@ -6,6 +6,7 @@ import pytest
 from binsparse.conversions import from_torch, to_torch
 from binsparse.tensor import (
     COORMatrix,
+    CSCMatrix,
     CSRMatrix,
     CustomTensor,
     DenseLevel,
@@ -48,22 +49,63 @@ def test_torch_csr_round_trip() -> None:
     assert torch.equal(result.to_dense(), source.to_dense())
 
 
-def test_torch_uncoalesced_coo_preserves_entries() -> None:
+@pytest.mark.parametrize("ndim", [1, 2, 3])
+@pytest.mark.parametrize("copy", [None, True, False])
+def test_torch_uncoalesced_coo_is_sorted_and_unique(ndim, copy) -> None:
+    coords = torch.tensor([[2, 0, 2, 1]] * ndim)
+    values = torch.tensor([1.0, 2.0, -1.0, 4.0])
     source = torch.sparse_coo_tensor(
-        torch.tensor([[0, 0], [1, 1]]),
-        torch.tensor([2.0, 3.0]),
-        (2, 2),
+        coords.clone(),
+        values.clone(),
+        (3,) * ndim,
     )
-    tensor = from_torch(source)
-
-    assert isinstance(tensor, COORMatrix)
-    assert tensor.number_of_stored_values == 2
+    if copy is False:
+        with pytest.raises(ValueError, match="canonicalize"):
+            from_torch(source, copy=copy)
+    else:
+        tensor = from_torch(source, copy=copy)
+        result = to_torch(tensor)
+        assert tensor.number_of_stored_values == 3
+        assert result.dtype == source.dtype
+        assert torch.equal(result._indices(), torch.tensor([[0, 1, 2]] * ndim))
+        assert torch.equal(result._values(), torch.tensor([2.0, 4.0, 0.0]))
+        assert torch.equal(result.to_dense(), source.to_dense())
+    assert not source.is_coalesced()
+    assert torch.equal(source._indices(), coords)
+    assert torch.equal(source._values(), values)
+
+
+@pytest.mark.parametrize("layout", ["coo", "csr", "csc"])
+@pytest.mark.parametrize("empty", [False, True])
+def test_torch_canonical_sparse_copy_policy(layout, empty) -> None:
+    dense = torch.zeros((3, 3)) if empty else torch.eye(3)
+    source = dense.to_sparse(layout=getattr(torch, f"sparse_{layout}"))
+    shared = from_torch(source, copy=False)
+    copied = from_torch(source, copy=True)
+    assert isinstance(shared, (COORMatrix, CSRMatrix, CSCMatrix))
+    assert isinstance(copied, (COORMatrix, CSRMatrix, CSCMatrix))
+    original_values = source.values().numpy()
+    np.testing.assert_array_equal(copied.values, original_values)
+    if not empty:
+        assert np.shares_memory(shared.values, original_values)
+        assert not np.shares_memory(copied.values, original_values)
+        if layout == "coo":
+            assert isinstance(shared, COORMatrix)
+            assert isinstance(copied, COORMatrix)
+            assert np.shares_memory(shared.indices_0, source.indices().numpy())
+            assert not np.shares_memory(copied.indices_0, source.indices().numpy())
+        else:
+            indices = source.col_indices() if layout == "csr" else source.row_indices()
+            assert np.shares_memory(shared.indices_1, indices.numpy())
+            assert not np.shares_memory(copied.indices_1, indices.numpy())
 
 
 @pytest.mark.parametrize("shape", [(5,), (2, 3, 4)])
 def test_torch_nd_coo_round_trip(shape) -> None:
     coordinates = torch.tensor([[0, size - 1] for size in shape])
-    source = torch.sparse_coo_tensor(coordinates, torch.tensor([2.0, 3.0]), shape)
+    source = torch.sparse_coo_tensor(
+        coordinates, torch.tensor([2.0, 3.0]), shape
+    ).coalesce()
     tensor = from_torch(source, copy=False)
     result = to_torch(tensor)
 
@@ -72,8 +114,7 @@ def test_torch_nd_coo_round_trip(shape) -> None:
     assert tensor.level.rank == len(shape)
     assert isinstance(tensor.level.level, ElementLevel)
     assert all(
-        index.__array_interface__["data"][0]
-        == source._indices()[dimension].data_ptr()
+        index.__array_interface__["data"][0] == source._indices()[dimension].data_ptr()
         for dimension, index in enumerate(tensor.level.indices)
     )
     assert torch.equal(result.to_dense(), source.to_dense())

binsparse-0.1.2/PKG-INFO

@@ -1,18 +0,0 @@
-Metadata-Version: 2.5
-Name: binsparse
-Version: 0.1.2
-Author-email: Willow Ahrens <ahrens@gatech.edu>
-Requires-Python: >=3.11
-Requires-Dist: h5py<4,>=3.16.0
-Requires-Dist: numpy<3,>=2
-Requires-Dist: semver<4,>=3
-Provides-Extra: conversions
-Requires-Dist: scipy; extra == 'conversions'
-Requires-Dist: sparse; extra == 'conversions'
-Requires-Dist: torch; extra == 'conversions'
-Provides-Extra: scipy
-Requires-Dist: scipy; extra == 'scipy'
-Provides-Extra: sparse
-Requires-Dist: sparse; extra == 'sparse'
-Provides-Extra: torch
-Requires-Dist: torch; extra == 'torch'