blosc2 3.2.1__tar.gz → 3.3.1__tar.gz

{blosc2-3.2.1 → blosc2-3.3.1}/.pre-commit-config.yaml

@@ -16,7 +16,7 @@ repos:
       - id: trailing-whitespace
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.9.9
+    rev: v0.11.4
     hooks:
       - id: ruff
         args: ["--fix", "--show-fixes"]

{blosc2-3.2.1 → blosc2-3.3.1}/ANNOUNCE.rst

@@ -1,16 +1,16 @@
-Announcing Python-Blosc2 3.2.1
+Announcing Python-Blosc2 3.3.1
 ==============================
 
-Here, all array containers in Blosc2 implement the ``__array_interface__``
-protocol to expose the data in the array.  This allows for better
-interoperability with other libraries like NumPy, CuPy, etc.  Now, the range
-of functions spans to most of NumPy functions, including reductions.
+In our effort to better adapt to better adapt to the array API
+(https://data-apis.org/array-api/latest/), we have introduced
+permute_dims() and matrix_transpose() functions, and the .T property.
+This replaces to previous transpose() function, which is now deprecated.
+See PR #384.  Thanks to Ricardo Sales Piquer (@ricardosp4).
 
-See examples at: https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/jit-numpy-funcs.py
-See benchmarks at: https://github.com/Blosc/python-blosc2/blob/main/bench/ndarray/jit-numpy-funcs.py
-
-We have also improved the performance of constructors like ``blosc2.linspace()``
-or ``blosc2.arange()`` by a factor of up to 3x for large arrays.
+We have also reduced the memory footprint of constructors like ``arange()``,
+``linspace()`` and ``fromiter()`` by a large factor. As an example, a 5 TB
+array of 8-byte floats now uses less than 200 MB of memory instead of
+170 GB previously.
 
 You can think of Python-Blosc2 3.x as an extension of NumPy/numexpr that:
 

{blosc2-3.2.1 → blosc2-3.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: blosc2
-Version: 3.2.1
+Version: 3.3.1
 Summary: A fast & compressed ndarray library with a flexible compute engine.
 Author-Email: Blosc Development Team <blosc@blosc.org>
 Maintainer-Email: Blosc Development Team <blosc@blosc.org>
@@ -166,6 +166,7 @@ to the core development of the Blosc2 library:
 - Juan David Ibáñez
 - Ivan Vilata i Balaguer
 - Oumaima Ech.Chdig
+- Ricardo Sales Piquer
 
 In addition, other people have participated to the project in different
 aspects:

{blosc2-3.2.1 → blosc2-3.3.1}/README.rst

@@ -108,6 +108,7 @@ to the core development of the Blosc2 library:
 - Juan David Ibáñez
 - Ivan Vilata i Balaguer
 - Oumaima Ech.Chdig
+- Ricardo Sales Piquer
 
 In addition, other people have participated to the project in different
 aspects:

{blosc2-3.2.1 → blosc2-3.3.1}/RELEASE_NOTES.md

@@ -1,5 +1,44 @@
 # Release notes
 
+## Changes from 3.3.0 to 3.3.1
+
+* In our effort to better adapt to better adapt to the array API
+  (https://data-apis.org/array-api/latest/), we have introduced
+  permute_dims() and matrix_transpose() functions, and the .T property.
+  This replaces to previous transpose() function, which is now deprecated.
+  See PR #384.  Thanks to Ricardo Sales Piquer (@ricardosp4).
+
+* Constructors like `arange()`, `linspace()` and `fromiter()` now
+  use far less memory when creating large arrays. As an example, a 5 TB
+  array of 8-byte floats now uses less than 200 MB of memory instead of
+  170 GB previously.  See PR #387.
+
+* Now, when opening a lazy expression with `blosc2.open()`, and there is
+  a missing operand, the open still works, but the dtype and shape
+  attributes are None.  This is useful for lazy expressions that have
+  lost some operands, but you still want to open them for inspection.
+  See PR #385.
+
+* Added an example of getting a slice out of a C2Array.
+
+## Changes from 3.2.1 to 3.3.0
+
+* New `blosc2.transpose()` function for transposing 2D NDArray instances
+  natively. See PR #375 and docs at
+  https://www.blosc.org/python-blosc2/reference/autofiles/operations_with_arrays/blosc2.transpose.html#blosc2.transpose
+  Thanks to Ricardo Sales Piquer (@ricardosp4) for the implementation.
+
+* New fast path for `NDArray.slice()` for getting slices that are aligned with
+  underlying chunks. This is a common operation when working with NDArray
+  instances, and now it is up to 40x faster in our benchmarks (see PR #380).
+
+* Returned `NDArray` object in `NDarray.slice()` now defaults to original
+  codec/clevel/filters. The previous behavior was to use the default
+  codec/clevel/filters.  See PR #378.  Thanks to Luke Shaw (@lshaw8317).
+
+* Several English edits in the documentation.  Thanks to Luke Shaw (@lshaw8317)
+  for his help in this area.
+
 ## Changes from 3.2.0 to 3.2.1
 
 * The array containers are now using the `__array_interface__` protocol to

{blosc2-3.2.1 → blosc2-3.3.1}/doc/getting_started/tutorials/05.persistent-reductions.ipynb

@@ -25,7 +25,7 @@
    "source": [
     "## Operands as arrays of different shape\n",
     "\n",
-    "We will now create the operands, using different shape for each of them, just for flexing the broadcasting capabilities of lazy expressions."
+    "We will now create the operands, using a different shape for each of them, in order to flex the broadcasting capabilities of lazy expressions."
    ]
   },
   {
@@ -77,7 +77,7 @@
    "source": [
     "## Creating, saving and loading a lazy expression\n",
     "\n",
-    "First, let's build a string expression that sums the contents of array `a` and operates with the values of `b` by `c`. In this context, creating a string version of the expression is critical; otherwise, reductions should be computed eagerly.\n",
+    "First, let's build a string expression that sums the contents of array `a` and performs a multiplication with `b` and `c`. In this context, creating a string version of the expression is critical; otherwise, reductions will be computed eagerly.\n",
     "\n",
     "Let's see how this works."
    ]
@@ -110,13 +110,13 @@
    "id": "6226bf8d46ac7c32",
    "metadata": {},
    "source": [
-    "In the code above, an expression combining the arrays `a`, `b`, and `c` is expressed in string form: `a.sum() + b ∗ c`.  Then, one builds a lazy expression and save it for later. The expression chosen illustrates how operations automatically adapt to the dimensions of the operands through the concept of broadcasting.\n",
+    "In the code above, an expression combining the arrays `a`, `b`, and `c` is expressed in string form: `a.sum() + b ∗ c`.  Then, one builds a lazy expression and saves it for later. The chosen expression illustrates how operations automatically adapt to the dimensions of the operands via broadcasting.\n",
     "\n",
-    "**Broadcasting** allows arrays of different shapes (dimensions) to align for mathematical operations, such as addition or multiplication, without the need to enlarge operands by replicating data. The main idea is that smaller dimensions are \"stretched\" to larger dimensions in such a way that allows the operation to be performed consistently.\n",
+    "**Broadcasting** allows arrays of different shapes (dimensions) to align for mathematical operations, such as addition or multiplication, without the need to enlarge operands by replicating data. The main idea is that smaller dimensions are \"stretched\" to larger dimensions in such a way that the operation may be performed consistently.\n",
     "\n",
     "<img src=\"images/lazyexpr-broadcast.png\" width=\"1000\"/>\n",
     "\n",
-    "See [NumPy docs on broadcasting](https://numpy.org/doc/stable/user/basics.broadcasting.html) for more information.\n",
+    "See the [NumPy docs on broadcasting](https://numpy.org/doc/stable/user/basics.broadcasting.html) for more information.\n",
     "\n",
     "Now that we have saved the expression, we can open and compute it to obtain the result. Let's see how this is done."
    ]
@@ -167,7 +167,7 @@
    "id": "362cfd5eb88b9bb6",
    "metadata": {},
    "source": [
-    "As we can observe when printing the lazy expression and its shape, the time required to get the `shape` is significantly shorter. This is because `lazy_expression.shape` does not need to compute all the elements of the expression; instead, it only accesses the **metadata** of the operands, from which it is inferred the basic information about the dimensions and type of the result.\n",
+    "As we can observe when printing the lazy expression and its shape, the time required to get the `shape` is significantly shorter. This is because `lazy_expression.shape` does not need to compute all the elements of the expression; instead, it only accesses the **metadata** of the operands, from which it infers the necessary information about the dimensions and type of the result.\n",
     "\n",
     "Thanks to this metadata, if we know the dimensions of the arrays involved in the operation (such as in the case of `a.sum() + b * c`), Blosc2 can **quickly infer the resulting shape** without performing intensive calculations. This allows for fast access to structural information (like the `shape` and `dtype`) without operating on the actual data.\n",
     "\n",
@@ -181,7 +181,7 @@
    "source": [
     "## Resizing operands of persisted lazy expressions\n",
     "\n",
-    "In this section, we will see how persisted lazy expressions automatically adapt to changes in the dimensions and values of the original operands, such as arrays `a` and `b`."
+    "In this section, we will see how persisted lazy expressions automatically adapt to changes in the dimensions and values of the original operands, such as the arrays `a` and `b`."
    ]
   },
   {
@@ -233,7 +233,7 @@
    "id": "d82492bf518c5a39",
    "metadata": {},
    "source": [
-    "After increasing the dimensions of the original arrays by modifying the values of `a` and `b`, the lazy expression is reopened. This step is crucial as it allows us to observe how the computation of the expression adapts to the new dimensions. Upon re-opening the expression, we can check that the results now accurately reflect these changes in the dimensions of the array operands. Moreover, see how obtaining the structural information (the `shape`) of the expression is a quick process, requiring only a fraction of the time it takes for the complete computation.\n",
+    "After increasing the dimensions of the original arrays by modifying the values of `a` and `b`, the lazy expression is reopened. This step is crucial as it allows us to observe how the computation of the expression adapts to the new dimensions. Upon re-opening the expression, we can check that the results now accurately reflect these changes in the dimensions of the array operands. Moreover, see how obtaining the updated structural information (the `shape`) of the expression is a quick process, requiring only a fraction of the time it takes for the complete computation.\n",
     "\n",
     "This behavior highlights the ability of lazy expressions to adjust to operands using **metadata**, eliminating the need to re-compute each operation from the beginning. Thanks to this approach, notable flexibility and efficiency are achieved in handling arrays of various shapes and sizes."
    ]
@@ -249,7 +249,7 @@
     "\n",
     "Also, broadcasting support facilitates working with arrays of different sizes, making the process more powerful and intuitive.\n",
     "\n",
-    "Understanding how operations are managed in this context enables developers and data scientists to make the most of reduction and broadcasting capabilities, thereby enhancing the efficiency and effectiveness of their analyses and calculations. The beauty of lazy expressions lies in its ability to simplify the complex and empower our creativity!"
+    "Understanding how operations are managed in this context enables developers and data scientists to make the most of reduction and broadcasting capabilities, thereby enhancing the efficiency and effectiveness of their analyses and calculations. The beauty of lazy expressions lies in their ability to simplify the complex and empower our creativity!"
    ]
   },
   {

{blosc2-3.2.1 → blosc2-3.3.1}/doc/python-blosc2.rst

@@ -6,7 +6,7 @@
 
     <p style="text-align: center; color: black; background-color: rgba(230, 169, 9, 0.65);">
         <a href="https://github.com/Blosc/python-blosc2/blob/main/RELEASE_NOTES.md"
-           style="font-size: 1.5em;">Version 3.2.0 released on 2025-02-27!</a>
+           style="font-size: 1.5em;">Version 3.3.0 released on 2025-04-08!</a>
         <span style="display: inline-block; width: 20px;"></span>
         <span style="font-family: monospace;">pip install blosc2 -U</span>
     </p>

{blosc2-3.2.1 → blosc2-3.3.1}/doc/reference/linear_algebra.rst

@@ -12,3 +12,6 @@ The next functions can be used for computing linear algebra operations with :ref
    :nosignatures:
 
     matmul
+    transpose
+    matrix_transpose
+    permute_dims

{blosc2-3.2.1 → blosc2-3.3.1}/doc/reference/ndarray.rst

@@ -40,6 +40,7 @@ Attributes
 .. autosummary::
     :toctree: autofiles/ndarray
 
+    T
     ndim
     shape
     ext_shape

blosc2-3.3.1/examples/c2array-get-slice.py

@@ -0,0 +1,37 @@
+#######################################################################
+# Copyright (c) 2019-present, Blosc Development Team <blosc@blosc.org>
+# All rights reserved.
+#
+# This source code is licensed under a BSD-style license (found in the
+# LICENSE file in the root directory of this source tree)
+#######################################################################
+
+# Example for opening and reading a C2Array (remote array)
+
+from time import time
+
+import numpy as np
+
+import blosc2
+
+urlbase = "https://cat2.cloud/demo"
+root = "@public"
+
+# Access the server
+# urlpath = blosc2.URLPath(f'{root}/examples/ds-1d.b2nd', urlbase)
+# urlpath = blosc2.URLPath(f'{root}/examples/sa-1M.b2nd', urlbase)
+urlpath = blosc2.URLPath(f"{root}/examples/lung-jpeg2000_10x.b2nd", urlbase)
+# urlpath = blosc2.URLPath(f'{root}/examples/uncompressed_lung-jpeg2000_10x.b2nd', urlbase)
+
+# Open the remote array
+t0 = time()
+remote_array = blosc2.open(urlpath, mode="r")
+size = np.prod(remote_array.shape) * remote_array.cparams.typesize
+print(f"Time for opening data (HTTP): {time() - t0:.3f}s - file size: {size / 2**10:.2f} KB")
+
+# Fetch a slice of the remote array as a numpy array
+t0 = time()
+a = remote_array[5:9]
+print(f"Time for reading data (HTTP): {time() - t0:.3f}s - {a.nbytes / 2**10:.2f} KB")
+
+# TODO: Fetch a slice of the remote array as a blosc2.NDArray

{blosc2-3.2.1 → blosc2-3.3.1}/examples/ndarray/arange-constructor.py

@@ -19,7 +19,7 @@ N = 10_000_000
 shape = (N,)
 print(f"*** Creating a blosc2 array with {N:_} elements (shape: {shape}) ***")
 t0 = time()
-a = blosc2.arange(shape=shape, dtype=np.int32)
+a = blosc2.arange(N, shape=shape, dtype=np.int32)
 cratio = a.schunk.nbytes / a.schunk.cbytes
 print(
     f"Time: {time() - t0:.3f} s ({N / (time() - t0) / 1e6:.2f} M/s)"

{blosc2-3.2.1 → blosc2-3.3.1}/pyproject.toml

@@ -40,7 +40,7 @@ dependencies = [
     "py-cpuinfo; platform_machine != 'wasm32'",
     "requests",
 ]
-version = "3.2.1"
+version = "3.3.1"
 
 
 [project.optional-dependencies]

{blosc2-3.2.1 → blosc2-3.3.1}/src/blosc2/__init__.py

@@ -248,6 +248,9 @@ from .ndarray import (
     full,
     save,
     matmul,
+    permute_dims,
+    transpose,
+    matrix_transpose,
 )
 
 from .c2array import c2context, C2Array, URLPath
@@ -266,7 +269,6 @@ from .proxy import Proxy, ProxySource, ProxyNDSource, ProxyNDField, SimpleProxy,
 
 from .schunk import SChunk, open
 
-
 # Registry for postfilters
 postfilter_funcs = {}
 """