blosc2 3.2.0__tar.gz → 3.3.0__tar.gz

{blosc2-3.2.0 → blosc2-3.3.0}/.pre-commit-config.yaml

@@ -16,7 +16,7 @@ repos:
       - id: trailing-whitespace
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.9.4
+    rev: v0.11.4
     hooks:
       - id: ruff
         args: ["--fix", "--show-fixes"]

{blosc2-3.2.0 → blosc2-3.3.0}/ANNOUNCE.rst

@@ -1,13 +1,12 @@
-Announcing Python-Blosc2 3.2.0
+Announcing Python-Blosc2 3.3.0
 ==============================
 
-This is a minor release where we fixed some bugs related with the
-maximum type size supported by Blosc2; now, the maximum size is
-512 MB (compared to 255 bytes in previous versions).  We are also
-introducing WASM32 wheels for the first time.
+We are introducing a new blosc2.transpose() function for natively transposing
+2D NDArray instances and a fast path for NDArray.slice() that delivers up to
+40x speedup when slices align with underlying chunks. Documentation has also
+been improved with several edits throughout.
 
-We also added new ``blosc2.matmul()`` function for computing matrix
-multiplication on NDArray instances.
+See benchmarks at: https://github.com/Blosc/python-blosc2/blob/main/bench/ndarray/aligned_chunks.py
 
 You can think of Python-Blosc2 3.x as an extension of NumPy/numexpr that:
 
@@ -17,7 +16,6 @@ You can think of Python-Blosc2 3.x as an extension of NumPy/numexpr that:
 - Supports NumPy ufunc mechanism: mix and match NumPy and Blosc2 computations.
 - Integrates with Numba and Cython via UDFs (User Defined Functions).
 - Adheres to modern NumPy casting rules way better than numexpr.
-- Computes expressions only when needed. They can also be stored for later use.
 - Supports linear algebra operations (like ``blosc2.matmul()``).
 
 Install it with::

{blosc2-3.2.0 → blosc2-3.3.0}/CMakeLists.txt

@@ -50,7 +50,7 @@ else()
     include(FetchContent)
     FetchContent_Declare(blosc2
         GIT_REPOSITORY https://github.com/Blosc/c-blosc2
-        GIT_TAG 55a103239b24f8b4e07dafa5a289ad62bfade524  # v2.17.0
+        GIT_TAG 5fcd6fbf9ffcf613fabdb1eb3a90eeb12f7c04fe  # v2.17.1
     )
     FetchContent_MakeAvailable(blosc2)
     include_directories("${blosc2_SOURCE_DIR}/include")

{blosc2-3.2.0 → blosc2-3.3.0}/PKG-INFO

@@ -1,8 +1,7 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.2
 Name: blosc2
-Version: 3.2.0
+Version: 3.3.0
 Summary: A fast & compressed ndarray library with a flexible compute engine.
-Home-page: https://github.com/Blosc/python-blosc2
 Author-Email: Blosc Development Team <blosc@blosc.org>
 Maintainer-Email: Blosc Development Team <blosc@blosc.org>
 License: BSD-3-Clause
@@ -19,8 +18,8 @@ Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
-Project-URL: Homepage, https://github.com/Blosc/python-blosc2
-Project-URL: Documentation, https://www.blosc.org/python-blosc2/python-blosc2.html
+Project-URL: homepage, https://github.com/Blosc/python-blosc2
+Project-URL: documentation, https://www.blosc.org/python-blosc2/python-blosc2.html
 Requires-Python: >=3.11
 Requires-Dist: numpy>=1.26
 Requires-Dist: ndindex
@@ -29,6 +28,7 @@ Requires-Dist: platformdirs
 Requires-Dist: numexpr; platform_machine != "wasm32"
 Requires-Dist: py-cpuinfo; platform_machine != "wasm32"
 Requires-Dist: requests
+Provides-Extra: dev
 Requires-Dist: dask; extra == "dev"
 Requires-Dist: h5py; extra == "dev"
 Requires-Dist: hdf5plugin; extra == "dev"
@@ -42,9 +42,11 @@ Requires-Dist: ruff; extra == "dev"
 Requires-Dist: s3fs; extra == "dev"
 Requires-Dist: xarray; extra == "dev"
 Requires-Dist: zarr; extra == "dev"
+Provides-Extra: test
 Requires-Dist: pytest; extra == "test"
 Requires-Dist: psutil; platform_machine != "wasm32" and extra == "test"
 Requires-Dist: torch; platform_machine != "wasm32" and extra == "test"
+Provides-Extra: doc
 Requires-Dist: sphinx>=8; extra == "doc"
 Requires-Dist: pydata-sphinx-theme; extra == "doc"
 Requires-Dist: numpydoc; extra == "doc"
@@ -52,9 +54,6 @@ Requires-Dist: myst-parser; extra == "doc"
 Requires-Dist: sphinx-paramlinks; extra == "doc"
 Requires-Dist: nbsphinx; extra == "doc"
 Requires-Dist: sphinx-panels; extra == "doc"
-Provides-Extra: dev
-Provides-Extra: test
-Provides-Extra: doc
 Description-Content-Type: text/x-rst
 
 =============
@@ -82,43 +81,31 @@ A fast & compressed ndarray library with a flexible compute engine
         :target: https://github.com/Blosc/python-blosc2/actions/workflows/build.yml
 
 
-What it is
-==========
+What is Python-Blosc2?
+=======================
 
 Python-Blosc2 is a high-performance compressed ndarray library with a flexible
-compute engine.  It uses the C-Blosc2 library as the compression backend.
-`C-Blosc2 <https://github.com/Blosc/c-blosc2>`_ is the next generation of
-Blosc, an `award-winning <https://www.blosc.org/posts/prize-push-Blosc2/>`_
-library that has been around for more than a decade, and that is been used
-by many projects, including `PyTables <https://www.pytables.org/>`_ or
-`Zarr <https://zarr.readthedocs.io/en/stable/>`_.
-
-Python-Blosc2 is Python wrapper that exposes the C-Blosc2 API, *plus* an
-integrated compute engine. This allows to perform complex calculations on
-compressed data in a way that operands do not need to be in-memory, but can be
-stored on disk or on `the network <https://github.com/ironArray/Caterva2>`_.
-This makes possible to work with data no matter how large it is, and that
-can be stored in a distributed fashion.
-
-Most importantly, Python-Blosc2 uses the `C-Blosc2 simple and open format
+compute engine, using `C-Blosc2 <https://www.blosc.org/c-blosc2/c-blosc2.html>`_
+as its compression backend. It allows complex calculations on compressed data,
+whether stored in memory, on disk, or over the network (e.g., via
+`Caterva2 <https://github.com/ironArray/Caterva2>`_).  It uses the
+`C-Blosc2 simple and open format
 <https://github.com/Blosc/c-blosc2/blob/main/README_FORMAT.rst>`_ for storing
-compressed data, making it easy to integrate with other systems and tools.
-
-You can find more introductory info about Python-Blosc2 at:
+compressed data.
 
-https://www.blosc.org/python-blosc2/getting_started/overview.html
+More info: https://www.blosc.org/python-blosc2/getting_started/overview.html
 
 Installing
 ==========
 
-Blosc2 now provides Python wheels for the major OS (Win, Mac and Linux) and platforms.
-You can install the binary packages from PyPi using ``pip``:
+Binary packages are available for major OSes (Win, Mac, Linux) and platforms.
+Install from PyPi using ``pip``:
 
 .. code-block:: console
 
     pip install blosc2 --upgrade
 
-For conda users, you can install the package from the conda-forge channel:
+Conda users can install from conda-forge:
 
 .. code-block:: console
 
@@ -131,13 +118,15 @@ The documentation is available here:
 
 https://blosc.org/python-blosc2/python-blosc2.html
 
-Additionally, you can find some examples at:
+You can find examples at:
 
 https://github.com/Blosc/python-blosc2/tree/main/examples
 
-Finally, we taught a tutorial at the `PyData Global 2024 <https://pydata.org/global2024/>`_
-that you can find at: https://github.com/Blosc/Python-Blosc2-3.0-tutorial.  There you will
-find different Jupyter notebook that explains the main features of Python-Blosc2.
+A tutorial from PyData Global 2024 is available at:
+
+https://github.com/Blosc/Python-Blosc2-3.0-tutorial
+
+It contains Jupyter notebooks explaining the main features of Python-Blosc2.
 
 License
 =======
@@ -177,6 +166,7 @@ to the core development of the Blosc2 library:
 - Juan David Ibáñez
 - Ivan Vilata i Balaguer
 - Oumaima Ech.Chdig
+- Ricardo Sales Piquer
 
 In addition, other people have participated to the project in different
 aspects:

{blosc2-3.2.0 → blosc2-3.3.0}/README.rst

@@ -23,43 +23,31 @@ A fast & compressed ndarray library with a flexible compute engine
         :target: https://github.com/Blosc/python-blosc2/actions/workflows/build.yml
 
 
-What it is
-==========
+What is Python-Blosc2?
+=======================
 
 Python-Blosc2 is a high-performance compressed ndarray library with a flexible
-compute engine.  It uses the C-Blosc2 library as the compression backend.
-`C-Blosc2 <https://github.com/Blosc/c-blosc2>`_ is the next generation of
-Blosc, an `award-winning <https://www.blosc.org/posts/prize-push-Blosc2/>`_
-library that has been around for more than a decade, and that is been used
-by many projects, including `PyTables <https://www.pytables.org/>`_ or
-`Zarr <https://zarr.readthedocs.io/en/stable/>`_.
-
-Python-Blosc2 is Python wrapper that exposes the C-Blosc2 API, *plus* an
-integrated compute engine. This allows to perform complex calculations on
-compressed data in a way that operands do not need to be in-memory, but can be
-stored on disk or on `the network <https://github.com/ironArray/Caterva2>`_.
-This makes possible to work with data no matter how large it is, and that
-can be stored in a distributed fashion.
-
-Most importantly, Python-Blosc2 uses the `C-Blosc2 simple and open format
+compute engine, using `C-Blosc2 <https://www.blosc.org/c-blosc2/c-blosc2.html>`_
+as its compression backend. It allows complex calculations on compressed data,
+whether stored in memory, on disk, or over the network (e.g., via
+`Caterva2 <https://github.com/ironArray/Caterva2>`_).  It uses the
+`C-Blosc2 simple and open format
 <https://github.com/Blosc/c-blosc2/blob/main/README_FORMAT.rst>`_ for storing
-compressed data, making it easy to integrate with other systems and tools.
-
-You can find more introductory info about Python-Blosc2 at:
+compressed data.
 
-https://www.blosc.org/python-blosc2/getting_started/overview.html
+More info: https://www.blosc.org/python-blosc2/getting_started/overview.html
 
 Installing
 ==========
 
-Blosc2 now provides Python wheels for the major OS (Win, Mac and Linux) and platforms.
-You can install the binary packages from PyPi using ``pip``:
+Binary packages are available for major OSes (Win, Mac, Linux) and platforms.
+Install from PyPi using ``pip``:
 
 .. code-block:: console
 
     pip install blosc2 --upgrade
 
-For conda users, you can install the package from the conda-forge channel:
+Conda users can install from conda-forge:
 
 .. code-block:: console
 
@@ -72,13 +60,15 @@ The documentation is available here:
 
 https://blosc.org/python-blosc2/python-blosc2.html
 
-Additionally, you can find some examples at:
+You can find examples at:
 
 https://github.com/Blosc/python-blosc2/tree/main/examples
 
-Finally, we taught a tutorial at the `PyData Global 2024 <https://pydata.org/global2024/>`_
-that you can find at: https://github.com/Blosc/Python-Blosc2-3.0-tutorial.  There you will
-find different Jupyter notebook that explains the main features of Python-Blosc2.
+A tutorial from PyData Global 2024 is available at:
+
+https://github.com/Blosc/Python-Blosc2-3.0-tutorial
+
+It contains Jupyter notebooks explaining the main features of Python-Blosc2.
 
 License
 =======
@@ -118,6 +108,7 @@ to the core development of the Blosc2 library:
 - Juan David Ibáñez
 - Ivan Vilata i Balaguer
 - Oumaima Ech.Chdig
+- Ricardo Sales Piquer
 
 In addition, other people have participated to the project in different
 aspects:

{blosc2-3.2.0 → blosc2-3.3.0}/RELEASE_NOTES.md

@@ -1,5 +1,40 @@
 # Release notes
 
+## Changes from 3.2.1 to 3.3.0
+
+* New `blosc2.transpose()` function for transposing 2D NDArray instances
+  natively. See PR #375 and docs at
+  https://www.blosc.org/python-blosc2/reference/autofiles/operations_with_arrays/blosc2.transpose.html#blosc2.transpose
+  Thanks to Ricardo Sales Piquer (@ricardosp4) for the implementation.
+
+* New fast path for `NDArray.slice()` for getting slices that are aligned with
+  underlying chunks. This is a common operation when working with NDArray
+  instances, and now it is up to 40x faster in our benchmarks (see PR #380).
+
+* Returned `NDArray` object in `NDarray.slice()` now defaults to original
+  codec/clevel/filters. The previous behavior was to use the default
+  codec/clevel/filters.  See PR #378.  Thanks to Luke Shaw (@lshaw8317).
+
+* Several English edits in the documentation.  Thanks to Luke Shaw (@lshaw8317)
+  for his help in this area.
+
+## Changes from 3.2.0 to 3.2.1
+
+* The array containers are now using the `__array_interface__` protocol to
+  expose the data in the array.  This allows for better interoperability with
+  other libraries that support the `__array_interface__` protocol, like NumPy,
+  CuPy, etc.  Now, the range of functions that can be used within the `blosc2.jit`
+  decorator is way larger, and essentially all NumPy functions should work now.
+
+  See examples at: https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/jit-numpy-funcs.py
+  See benchmarks at: https://github.com/Blosc/python-blosc2/blob/main/bench/ndarray/jit-numpy-funcs.py
+
+* The performance of constructors like `arange()`, `linspace()` and `fromiter()`
+  has been improved.  Now, they can be up to 3x faster, specially with large
+  arrays.
+
+* C-Blosc2 updated to 2.17.1.  This fixes various UB as well as compiler warnings.
+
 ## Changes from 3.1.1 to 3.2.0
 
 * Structured arrays can be larger than 255 bytes now.  This was a limitation

{blosc2-3.2.0 → blosc2-3.3.0}/RELEASING.rst

@@ -73,9 +73,9 @@ Tagging
   wheels (and tarball!) by downloading the artifacts manually, copying to
   an empty dir (say dist), and upload to PyPI with::
 
-    rm dist/*
+    rm wheelhouse/*
     # download artifacts from the tag in github
-    twine upload --repository blosc dist/*
+    twine upload --repository blosc2 wheelhouse/*
 
 * Update the latest release in the ``doc/python-blosc2.rst`` file with the new version
   number (and date?).  Do a commit::
@@ -106,6 +106,10 @@ Checking packaging
   https://github.com/Blosc/python-blosc2/releases/new
   Add the notes specific for this release.
 
+  Also, upload the wasm32 wheels to release page in github::
+
+    [upload e.g.:] blosc2-3.2.0-cp312-cp312-pyodide_2024_0_wasm32.whl
+
 
 Announcing
 ----------

blosc2-3.3.0/doc/getting_started/overview.rst

@@ -0,0 +1,238 @@
+.. Try to keep in sync with the README.rst file
+
+What is it?
+===========
+
+Python-Blosc2 is a high-performance compressed ndarray library with a flexible
+compute engine.  It uses the C-Blosc2 library as the compression backend.
+`C-Blosc2 <https://github.com/Blosc/c-blosc2>`_ is the next generation of
+Blosc, an `award-winning <https://www.blosc.org/posts/prize-push-Blosc2/>`_
+library that has been around for more than a decade, and that is being used
+by many projects, including `PyTables <https://www.pytables.org/>`_ or
+`Zarr <https://zarr.readthedocs.io/en/stable/>`_.
+
+Python-Blosc2 is a Python wrapper around the C-Blosc2 library, enhanced with
+an integrated compute engine. This allows for complex computations on
+compressed data, whether the operands are in memory, on disk, or
+`accessed over a network <https://github.com/ironArray/Caterva2>`_. This
+capability makes it easier to `work with very large datasets
+<https://ironarray.io/blog/compute-bigger>`_, even in distributed
+environments.
+
+Most importantly, Python-Blosc2 uses the
+`C-Blosc2 simple and open format <https://github.com/Blosc/c-blosc2/blob/main/README_FORMAT.rst>`_
+for storing compressed data. This facilitates seamless integration with other
+systems and tools.
+
+Interacting with the Ecosystem
+==============================
+
+Python-Blosc2 is designed to integrate seamlessly with existing libraries
+and tools, offering:
+
+* Support for NumPy's `universal functions
+  mechanism <https://numpy.org/doc/2.1/reference/ufuncs.html>`_, enabling
+  the combination of NumPy and Blosc2 computation engines.
+* Excellent integration with Numba and Cython via
+  `User Defined
+  Functions <https://www.blosc.org/python-blosc2/getting_started/tutorials/03.lazyarray-udf.html>`_.
+* Lazy expressions that are evaluated only when needed and can be stored
+  for future use.
+
+Python-Blosc2 leverages both `NumPy <https://numpy.org>`_ and
+`NumExpr <https://numexpr.readthedocs.io/en/latest/>`_ to achieve high
+performance, but with key differences. The main distinctions between the new
+computing engine and NumPy or NumExpr include:
+
+* Support for compressed ndarrays stored in memory, on disk, or
+  `over the network <https://github.com/ironArray/Caterva2>`_.
+* Ability to evaluate various mathematical expressions, including reductions,
+  indexing, and filters.
+* Support for broadcasting operations, enabling operations on arrays with
+  different shapes.
+* Improved adherence to NumPy casting rules compared to NumExpr.
+* Support for proxies, facilitating work with compressed data on local or
+  remote machines.
+
+Data Containers
+===============
+
+The main data container objects in Python-Blosc2 are:
+
+* ``SChunk``: A 64-bit compressed store suitable for any data type supporting the
+  `buffer protocol <https://docs.python.org/3/c-api/buffer.html>`_.
+* ``NDArray``: An N-Dimensional store that mirrors the NumPy API, enhanced with
+  efficient compressed data storage.
+
+These containers are described in more detail below.
+
+SChunk: a 64-bit compressed store
+---------------------------------
+
+``SChunk`` is a simple data container that handles setting, expanding and
+getting data and metadata.  In contrast to chunks, a super-chunk can update
+and resize the data that it contains, supports user metadata, and has virtually
+unlimited storage capacity (chunks, on the other hand, cannot store more than 2 GB).
+
+Additionally, you can convert a SChunk into a contiguous, serialized buffer
+(aka `cframe
+<https://github.com/Blosc/c-blosc2/blob/main/README_CFRAME_FORMAT.rst>`_) and
+vice-versa; as a bonus, the serialization/deserialization process also works
+with NumPy arrays and PyTorch/TensorFlow tensors at lightning-fast speed:
+
+.. |compress| image:: https://github.com/Blosc/python-blosc2/blob/main/images/linspace-compress.png?raw=true
+   :width: 100%
+   :alt: Compression speed for different codecs
+
+.. |decompress| image:: https://github.com/Blosc/python-blosc2/blob/main/images/linspace-decompress.png?raw=true
+   :width: 100%
+   :alt: Decompression speed for different codecs
+
++----------------+---------------+
+| |compress|     | |decompress|  |
++----------------+---------------+
+
+while reaching excellent compression ratios:
+
+.. image:: https://github.com/Blosc/python-blosc2/blob/main/images/pack-array-cratios.png?raw=true
+   :width: 75%
+   :align: center
+   :alt: Compression ratio for different codecs
+
+Also, if you are a Mac M1/M2 owner, do yourself a favor and use its native arm64
+arch (yes, we are distributing Mac arm64 wheels too; you're welcome ;-) ):
+
+.. |pack_arm| image:: https://github.com/Blosc/python-blosc2/blob/main/images/M1-i386-vs-arm64-pack.png?raw=true
+   :width: 100%
+   :alt: Compression speed for different codecs on Apple M1
+
+.. |unpack_arm| image:: https://github.com/Blosc/python-blosc2/blob/main/images/M1-i386-vs-arm64-unpack.png?raw=true
+   :width: 100%
+   :alt: Decompression speed for different codecs on Apple M1
+
++------------+--------------+
+| |pack_arm| | |unpack_arm| |
++------------+--------------+
+
+Read more about ``SChunk`` features in our blog entry at:
+https://www.blosc.org/posts/python-blosc2-improvements
+
+NDArray: an N-Dimensional store
+-------------------------------
+
+A recent feature in Python-Blosc2 is the
+`NDArray <https://www.blosc.org/python-blosc2/reference/ndarray_api.html>`_
+object.  It builds upon the ``SChunk`` object, offering a NumPy-like API
+for compressed n-dimensional data.
+
+It efficiently reads/writes n-dimensional datasets using an n-dimensional
+two-level partitioning scheme, enabling fine-grained slicing of large,
+compressed data:
+
+.. image:: https://github.com/Blosc/python-blosc2/blob/main/images/b2nd-2level-parts.png?raw=true
+  :width: 75%
+
+As an example, see how the ``NDArray`` object excels at retrieving slices
+orthogonal to different axes of a 4-dimensional dataset:
+
+.. image:: https://github.com/Blosc/python-blosc2/blob/main/images/Read-Partial-Slices-B2ND.png?raw=true
+  :width: 75%
+
+More information is available in this blog post:
+https://www.blosc.org/posts/blosc2-ndim-intro
+
+Check this short video explaining `why slicing in a pineapple-style (aka
+double partition) is useful
+<https://www.youtube.com/watch?v=LvP9zxMGBng>`_:
+
+.. image:: https://github.com/Blosc/blogsite/blob/master/files/images/slicing-pineapple-style.png?raw=true
+  :width: 50%
+  :alt: Slicing a dataset in pineapple-style
+  :target: https://www.youtube.com/watch?v=LvP9zxMGBng
+
+Operating with NDArrays
+=======================
+
+Python-Blosc2's ``NDArray`` objects are designed for ease of use,
+demonstrated by this example:
+
+.. code-block:: python
+
+    import blosc2
+
+    N = 20_000
+    # N = 70_000 # for large scenario
+    a = blosc2.linspace(0, 1, N * N).reshape(N, N)
+    b = blosc2.linspace(1, 2, N * N).reshape(N, N)
+    c = blosc2.linspace(-10, 10, N * N).reshape(N, N)
+    expr = ((a**3 + blosc2.sin(c * 2)) < b) & (c > 0)
+
+    out = expr.compute()
+    print(out.info)
+
+``NDArray`` instances resemble NumPy arrays but store compressed data,
+processed efficiently by Python-Blosc2's engine.
+
+When operands fit in memory (20,000 x 20,000), performance nears
+top-tier libraries like NumExpr, exceeding NumPy and Numba, with low memory use
+via default compression. As you can see, Blosc2 compression can speed
+computation via fast codecs and filters, plus efficient CPU cache use.
+
+.. image:: https://github.com/Blosc/python-blosc2/blob/main/images/lazyarray-dask-small.png?raw=true
+  :width: 100%
+  :alt: Performance when operands comfortably fit in-memory
+
+For larger datasets exceeding memory, Python-Blosc2 rivals Dask+Zarr in
+performance (70,000 x 70,000).
+
+.. image:: https://github.com/Blosc/python-blosc2/blob/main/images/lazyarray-dask-large.png?raw=true
+  :width: 100%
+  :alt: Performance when operands do not fit in memory (uncompressed)
+
+Blosc2 can utilize MKL-enabled Numexpr for optimized transcendental
+functions on Intel compatible CPUs (as used for the above plots).
+
+Benchmark notebooks:
+
+https://github.com/Blosc/python-blosc2/blob/main/bench/ndarray/lazyarray-dask-small.ipynb
+
+https://github.com/Blosc/python-blosc2/blob/main/bench/ndarray/lazyarray-dask-large.ipynb
+
+Reductions and disk-based computations
+--------------------------------------
+
+One key feature of Python-Blosc2's compute engine is its ability to
+perform reductions on compressed data, optionally stored on disk, enabling
+calculations on datasets too large for memory.
+
+Example:
+
+.. code-block:: python
+
+    import blosc2
+
+    N = 20_000  # for small scenario
+    # N = 100_000 # for large scenario
+    a = blosc2.linspace(0, 1, N * N, shape=(N, N), urlpath="a.b2nd", mode="w")
+    b = blosc2.linspace(1, 2, N * N, shape=(N, N), urlpath="b.b2nd", mode="w")
+    c = blosc2.linspace(-10, 10, N * N, shape=(N, N))  # small and in-memory
+    # Expression
+    expr = np.sum(((a**3 + np.sin(a * 2)) < c) & (b > 0), axis=1)
+
+    # Evaluate and get a NDArray as result
+    out = expr.compute()
+    print(out.info)
+
+This example computes the sum of a boolean array resulting from an
+expression, where the operands are on disk, with the result being a
+1D array stored in memory (or optionally on disk via the ``out=``
+parameter in ``compute()`` or ``sum()`` functions).
+
+Check out a blog post about this feature, with performance comparisons, at:
+https://ironarray.io/blog/compute-bigger
+
+Hopefully, this overview has provided a good understanding of
+Python-Blosc2's capabilities. To begin your journey with Python-Blosc2,
+proceed to the `installation instructions <installation>`_.
+Then explore the `tutorials <tutorials>`_ and
+`reference <../reference>`_ sections for further information!

{blosc2-3.2.0 → blosc2-3.3.0}/doc/getting_started/tutorials/05.persistent-reductions.ipynb

@@ -25,7 +25,7 @@
    "source": [
     "## Operands as arrays of different shape\n",
     "\n",
-    "We will now create the operands, using different shape for each of them, just for flexing the broadcasting capabilities of lazy expressions."
+    "We will now create the operands, using a different shape for each of them, in order to flex the broadcasting capabilities of lazy expressions."
    ]
   },
   {
@@ -77,7 +77,7 @@
    "source": [
     "## Creating, saving and loading a lazy expression\n",
     "\n",
-    "First, let's build a string expression that sums the contents of array `a` and operates with the values of `b` by `c`. In this context, creating a string version of the expression is critical; otherwise, reductions should be computed eagerly.\n",
+    "First, let's build a string expression that sums the contents of array `a` and performs a multiplication with `b` and `c`. In this context, creating a string version of the expression is critical; otherwise, reductions will be computed eagerly.\n",
     "\n",
     "Let's see how this works."
    ]
@@ -110,13 +110,13 @@
    "id": "6226bf8d46ac7c32",
    "metadata": {},
    "source": [
-    "In the code above, an expression combining the arrays `a`, `b`, and `c` is expressed in string form: `a.sum() + b ∗ c`.  Then, one builds a lazy expression and save it for later. The expression chosen illustrates how operations automatically adapt to the dimensions of the operands through the concept of broadcasting.\n",
+    "In the code above, an expression combining the arrays `a`, `b`, and `c` is expressed in string form: `a.sum() + b ∗ c`.  Then, one builds a lazy expression and saves it for later. The chosen expression illustrates how operations automatically adapt to the dimensions of the operands via broadcasting.\n",
     "\n",
-    "**Broadcasting** allows arrays of different shapes (dimensions) to align for mathematical operations, such as addition or multiplication, without the need to enlarge operands by replicating data. The main idea is that smaller dimensions are \"stretched\" to larger dimensions in such a way that allows the operation to be performed consistently.\n",
+    "**Broadcasting** allows arrays of different shapes (dimensions) to align for mathematical operations, such as addition or multiplication, without the need to enlarge operands by replicating data. The main idea is that smaller dimensions are \"stretched\" to larger dimensions in such a way that the operation may be performed consistently.\n",
     "\n",
     "<img src=\"images/lazyexpr-broadcast.png\" width=\"1000\"/>\n",
     "\n",
-    "See [NumPy docs on broadcasting](https://numpy.org/doc/stable/user/basics.broadcasting.html) for more information.\n",
+    "See the [NumPy docs on broadcasting](https://numpy.org/doc/stable/user/basics.broadcasting.html) for more information.\n",
     "\n",
     "Now that we have saved the expression, we can open and compute it to obtain the result. Let's see how this is done."
    ]
@@ -167,7 +167,7 @@
    "id": "362cfd5eb88b9bb6",
    "metadata": {},
    "source": [
-    "As we can observe when printing the lazy expression and its shape, the time required to get the `shape` is significantly shorter. This is because `lazy_expression.shape` does not need to compute all the elements of the expression; instead, it only accesses the **metadata** of the operands, from which it is inferred the basic information about the dimensions and type of the result.\n",
+    "As we can observe when printing the lazy expression and its shape, the time required to get the `shape` is significantly shorter. This is because `lazy_expression.shape` does not need to compute all the elements of the expression; instead, it only accesses the **metadata** of the operands, from which it infers the necessary information about the dimensions and type of the result.\n",
     "\n",
     "Thanks to this metadata, if we know the dimensions of the arrays involved in the operation (such as in the case of `a.sum() + b * c`), Blosc2 can **quickly infer the resulting shape** without performing intensive calculations. This allows for fast access to structural information (like the `shape` and `dtype`) without operating on the actual data.\n",
     "\n",
@@ -181,7 +181,7 @@
    "source": [
     "## Resizing operands of persisted lazy expressions\n",
     "\n",
-    "In this section, we will see how persisted lazy expressions automatically adapt to changes in the dimensions and values of the original operands, such as arrays `a` and `b`."
+    "In this section, we will see how persisted lazy expressions automatically adapt to changes in the dimensions and values of the original operands, such as the arrays `a` and `b`."
    ]
   },
   {
@@ -233,7 +233,7 @@
    "id": "d82492bf518c5a39",
    "metadata": {},
    "source": [
-    "After increasing the dimensions of the original arrays by modifying the values of `a` and `b`, the lazy expression is reopened. This step is crucial as it allows us to observe how the computation of the expression adapts to the new dimensions. Upon re-opening the expression, we can check that the results now accurately reflect these changes in the dimensions of the array operands. Moreover, see how obtaining the structural information (the `shape`) of the expression is a quick process, requiring only a fraction of the time it takes for the complete computation.\n",
+    "After increasing the dimensions of the original arrays by modifying the values of `a` and `b`, the lazy expression is reopened. This step is crucial as it allows us to observe how the computation of the expression adapts to the new dimensions. Upon re-opening the expression, we can check that the results now accurately reflect these changes in the dimensions of the array operands. Moreover, see how obtaining the updated structural information (the `shape`) of the expression is a quick process, requiring only a fraction of the time it takes for the complete computation.\n",
     "\n",
     "This behavior highlights the ability of lazy expressions to adjust to operands using **metadata**, eliminating the need to re-compute each operation from the beginning. Thanks to this approach, notable flexibility and efficiency are achieved in handling arrays of various shapes and sizes."
    ]
@@ -249,7 +249,7 @@
     "\n",
     "Also, broadcasting support facilitates working with arrays of different sizes, making the process more powerful and intuitive.\n",
     "\n",
-    "Understanding how operations are managed in this context enables developers and data scientists to make the most of reduction and broadcasting capabilities, thereby enhancing the efficiency and effectiveness of their analyses and calculations. The beauty of lazy expressions lies in its ability to simplify the complex and empower our creativity!"
+    "Understanding how operations are managed in this context enables developers and data scientists to make the most of reduction and broadcasting capabilities, thereby enhancing the efficiency and effectiveness of their analyses and calculations. The beauty of lazy expressions lies in their ability to simplify the complex and empower our creativity!"
    ]
   },
   {

{blosc2-3.2.0 → blosc2-3.3.0}/doc/python-blosc2.rst

@@ -6,7 +6,7 @@
 
     <p style="text-align: center; color: black; background-color: rgba(230, 169, 9, 0.65);">
         <a href="https://github.com/Blosc/python-blosc2/blob/main/RELEASE_NOTES.md"
-           style="font-size: 1.5em;">Version 3.1.0 released on 2025-02-13!</a>
+           style="font-size: 1.5em;">Version 3.2.1 released on 2025-04-27!</a>
         <span style="display: inline-block; width: 20px;"></span>
         <span style="font-family: monospace;">pip install blosc2 -U</span>
     </p>