blosc2 4.1.2__tar.gz → 4.2.0__tar.gz

{blosc2-4.1.2 → blosc2-4.2.0}/.pre-commit-config.yaml

@@ -16,7 +16,7 @@ repos:
       - id: trailing-whitespace
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.15.4
+    rev: v0.15.12
     hooks:
       - id: ruff-check
         args: ["--fix", "--show-fixes"]

blosc2-4.2.0/AGENTS.md

@@ -0,0 +1,53 @@
+# Repository Guidelines
+
+## Project Structure & Module Organization
+The Python package lives in `src/blosc2/`, including the C/Cython extension sources
+(`blosc2_ext.*`) and core modules such as `core.py`, `ndarray.py`, and `schunk.py`.
+Tests are under `tests/`, with additional doctests enabled for select modules per
+`pytest.ini`. Documentation sources are in `doc/` and build output lands in `html/`.
+Examples are in `examples/`, and performance/benchmark scripts live in `bench/`.
+
+## Build, Test, and Development Commands
+- `pip install .` builds the bundled C-Blosc2 and installs the package.
+- `pip install -e .` installs in editable mode for local development.
+- `CMAKE_PREFIX_PATH=/usr/local USE_SYSTEM_BLOSC2=1 pip install -e .` builds
+  against a separately installed C-Blosc2.
+- `pytest` runs the default test suite (excludes `heavy` and `network` markers).
+- `pytest -m "heavy"` runs long-running tests.
+- `pytest -m "network"` runs tests requiring network access.
+- `cd doc && rm -rf ../html _build && python -m sphinx . ../html` builds docs.
+
+## Coding Style & Naming Conventions
+Use Ruff for formatting and linting (line length 109). Enable pre-commit hooks:
+`python -m pip install pre-commit` then `pre-commit install`. Follow Python
+conventions: 4-space indentation, `snake_case` for functions/variables, and
+`PascalCase` for classes. Pytest discovery expects `tests/test_*.py` and
+`test_*` functions. Do not use leading underscores in module-level helper
+function names when those helpers are imported from other modules; reserve
+leading underscores for file-local implementation details. Avoid leading
+underscores in core module filenames under `src/blosc2/`; prefer non-underscored
+module names unless there is a strong reason to keep a module private.
+
+For documentation and tutorial query examples, prefer the shortest idiom that
+matches the intended result type. Use `expr[:]` or `arr[mask][:]` when showing
+values, use `expr.compute()` when materializing an `NDArray`, and use
+`expr.compute(_use_index=False)` when demonstrating scan-vs-index behavior.
+Avoid `expr.compute()[:]` unless a NumPy array is specifically required.
+
+## Testing Guidelines
+Pytest is required; warnings are treated as errors. The default configuration
+adds `--doctest-modules`, so keep doctest examples in `blosc2/core.py`,
+`blosc2/ndarray.py`, and `blosc2/schunk.py` accurate. Use markers `heavy` and
+`network` for slow or network-dependent tests.
+
+## Commit & Pull Request Guidelines
+Recent commit messages are short, imperative sentences (e.g., “Add …”, “Fix …”)
+without ticket prefixes. For pull requests: branch from `main`, add tests for
+behavior changes, update docs for API changes, ensure the test suite passes,
+and avoid introducing new compiler warnings. Link issues when applicable and
+include clear reproduction steps for bug fixes.
+
+**IMPORTANT — Agent commit policy**: Never run `git commit`, `git push`,
+`git reset`, `git rebase`, or any other destructive/state-changing git operation.
+Do not ask for permission to do so either. Committing and all repo state changes
+are exclusively the user’s decision. Just do the code work and stop there.

blosc2-4.2.0/ANNOUNCE.rst

@@ -0,0 +1,86 @@
+Announcing Python-Blosc2 4.2.0
+===============================
+
+We are happy to announce Python-Blosc2 4.2.0.  This is a large feature
+release, with the new compressed columnar table container, ``CTable``, as the
+main development.
+
+``CTable`` brings typed, compressed, column-oriented tables to Python-Blosc2.
+It supports persistent ``.b2d`` and ``.b2z`` storage, schema-driven columns,
+nullable data, variable-length strings/bytes and object columns, computed
+columns, table views, mutations, sorting, filtering and persistent indexes.  It
+also includes Arrow, Parquet and CSV interoperability, plus a new
+``parquet-to-blosc2`` command-line utility.
+
+For a deeper introduction to CTable and its motivation, see our recent blog
+post:
+
+https://blosc.org/posts/ctable-blosc2-columnar-table/
+
+Other highlights in 4.2.0 include:
+
+- A new indexing subsystem for NDArrays and CTables, including persistent
+  sidecar indexes, expression indexes, sorted iteration and query caching.
+- New structured serialization facilities for persisted ``C2Array``,
+  ``LazyExpr`` and DSL ``LazyUDF`` objects, plus ``blosc2.Ref`` and
+  ``blosc2.load()``.
+- New schema helpers such as ``blosc2.struct()`` and ``blosc2.object()``.
+- Object/ListArray improvements for variable-length and general object data.
+- Faster and lower-memory ``fromiter()`` construction, improved ``BatchArray``
+  defaults and continued linalg/matmul optimizations.
+- Many documentation, tutorial, example and benchmark additions.
+- Numerous fixes for Windows mmap/file-locking behavior, Python 3.14 GC/thread
+  interactions, ``.b2z`` persistence, indexed queries and NumPy compatibility.
+
+You can think of Python-Blosc2 4.x as an extension of NumPy/numexpr that:
+
+- Can deal with NDArray compressed objects using first-class codecs & filters.
+- Performs many kinds of math expressions, including reductions, indexing...
+- Supports multi-threading and SIMD acceleration (via numexpr/miniexpr).
+- Can operate with data from other libraries (like PyTables, h5py, Zarr, Dask, etc).
+- Supports NumPy ufunc mechanism: mix and match NumPy and Blosc2 computations.
+- Integrates with Numba and Cython via UDFs (User Defined Functions).
+- Adheres to modern array API standard conventions (https://data-apis.org/array-api/).
+- Can perform linear algebra operations (like ``blosc2.tensordot()``).
+- Can store and query compressed columnar tables via ``blosc2.CTable``.
+
+Install it with::
+
+    pip install blosc2 --upgrade   # if you prefer wheels
+    conda install -c conda-forge python-blosc2 mkl  # if you prefer conda and MKL
+
+For more info, you can have a look at the release notes in:
+
+https://github.com/Blosc/python-blosc2/releases
+
+Small CTable example::
+
+    import blosc2
+
+    table = blosc2.CTable.from_parquet("measurements.parquet", urlpath="measurements.b2z")
+    table.create_index("station_id")
+
+    hot = table.where("temperature > 30")
+    print(hot.head())
+
+Sources repository
+------------------
+
+The sources and documentation are managed through GitHub services at:
+
+https://github.com/Blosc/python-blosc2
+
+Python-Blosc2 is distributed using the BSD license, see
+https://github.com/Blosc/python-blosc2/blob/main/LICENSE.txt
+for details.
+
+Mastodon feed
+-------------
+
+Follow https://fosstodon.org/@Blosc2 to get informed about the latest
+developments.
+
+Enjoy!
+
+- Blosc Development Team
+  Compress Better, Compute Bigger

{blosc2-4.1.2 → blosc2-4.2.0}/CMakeLists.txt

@@ -14,6 +14,9 @@ endif()
 
 project(python-blosc2)
 
+set(BLOSC2_MIN_VERSION 3.0.0)
+set(BLOSC2_BUNDLED_VERSION v3.0.2)
+
 if(WIN32 AND NOT CMAKE_C_COMPILER_ID STREQUAL "Clang")
     message(FATAL_ERROR "Windows builds require clang-cl. Set CC/CXX to clang-cl or configure CMake with -T ClangCL.")
 endif()
@@ -41,11 +44,25 @@ add_custom_command(
   DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/blosc2/blosc2_ext.pyx"
   VERBATIM)
 
+add_custom_command(
+  OUTPUT indexing_ext.c
+  COMMAND Python::Interpreter -m cython
+          "${CMAKE_CURRENT_SOURCE_DIR}/src/blosc2/indexing_ext.pyx" --output-file indexing_ext.c
+  DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/blosc2/indexing_ext.pyx"
+  VERBATIM)
+
 # ...and add it to the target
 Python_add_library(blosc2_ext MODULE blosc2_ext.c WITH_SOABI)
+target_sources(blosc2_ext PRIVATE src/blosc2/matmul_kernels.c)
+target_include_directories(blosc2_ext PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src/blosc2)
+if(UNIX)
+  target_link_libraries(blosc2_ext PRIVATE ${CMAKE_DL_LIBS})
+endif()
+Python_add_library(indexing_ext MODULE indexing_ext.c WITH_SOABI)
 
 # We need to link against NumPy
 target_link_libraries(blosc2_ext PRIVATE Python::NumPy)
+target_link_libraries(indexing_ext PRIVATE Python::NumPy)
 
 # Fetch and build miniexpr library
 include(FetchContent)
@@ -63,15 +80,19 @@ endif()
 
 FetchContent_Declare(miniexpr
     GIT_REPOSITORY https://github.com/Blosc/miniexpr.git
-    GIT_TAG 37bf6982bf9619036b47f095b7005bc3c87a7447
+    GIT_TAG f2faef741c4c507bf6a03167c72ce7f92c6f0ae8
     # SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../miniexpr
 )
 FetchContent_MakeAvailable(miniexpr)
 
 # Link against miniexpr static library
 target_link_libraries(blosc2_ext PRIVATE miniexpr_static)
+if(APPLE)
+  target_link_libraries(blosc2_ext PRIVATE "-framework Accelerate")
+endif()
 
 target_compile_features(blosc2_ext PRIVATE c_std_11)
+target_compile_features(indexing_ext PRIVATE c_std_11)
 if(WIN32 AND CMAKE_C_COMPILER_ID STREQUAL "Clang")
     execute_process(
         COMMAND "${CMAKE_C_COMPILER}" -print-resource-dir
@@ -100,7 +121,7 @@ endif()
 if(USE_SYSTEM_BLOSC2)
     set(CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/cmake")
     find_package(PkgConfig REQUIRED)
-    pkg_check_modules(Blosc2 REQUIRED IMPORTED_TARGET blosc2)
+    pkg_check_modules(Blosc2 REQUIRED IMPORTED_TARGET blosc2>=${BLOSC2_MIN_VERSION})
     target_link_libraries(blosc2_ext PRIVATE PkgConfig::Blosc2)
 else()
     set(STATIC_LIB ON CACHE BOOL "Build a static version of the blosc library.")
@@ -119,7 +140,8 @@ else()
     include(FetchContent)
     FetchContent_Declare(blosc2
         GIT_REPOSITORY https://github.com/Blosc/c-blosc2
-        GIT_TAG 1386ef42f58b61c876edf714a2af84bd7b59dc5d # v2.23.1
+        GIT_TAG ${BLOSC2_BUNDLED_VERSION}
+        # SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../c-blosc2
     )
     FetchContent_MakeAvailable(blosc2)
     include_directories("${blosc2_SOURCE_DIR}/include")
@@ -147,7 +169,7 @@ endif()
 
 # Python extension -> site-packages/blosc2
 install(
-  TARGETS blosc2_ext
+  TARGETS blosc2_ext indexing_ext
   LIBRARY DESTINATION ${SKBUILD_PLATLIB_DIR}/blosc2
 )
 

blosc2-4.2.0/CONTRIBUTING.rst

@@ -0,0 +1,45 @@
+Contributing guidelines
+=======================
+We want to make contributing to this project as easy and transparent as possible.
+
+Our Development Process
+-----------------------
+New versions are being developed in the "main" branch, or in their own feature branch.
+When they are deemed ready for a release, they are merged back into "main" again.
+
+So all contributions must stage first through "main" or their own feature branch.
+
+Pull Requests
+-------------
+We actively welcome your pull requests.
+
+1. Fork the repo and create your branch from ``main``.
+2. If you've added code that should be tested, add tests.
+3. If you've changed APIs, update the documentation.
+4. Ensure the test suite passes.
+5. Make sure your code does not issue new compiler warnings.
+
+Automated Contributions Policy
+------------------------------
+
+Contributing to Blosc requires human judgment, contextual understanding, and familiarity with scikit-learn’s structure and goals. It is not suitable for automatic processing by AI tools.
+
+Please refrain from submitting issues or pull requests generated by fully-automated tools. Maintainers reserve the right, at their sole discretion, to close such submissions and to block any account responsible for them.
+
+Review all code or documentation changes made by AI tools and make sure you understand all changes and can explain them on request, before submitting them under your name. Do not submit any AI-generated code that you haven’t personally reviewed, understood and tested, as this wastes maintainers’ time.
+
+Please do not paste AI generated text in the description of issues, PRs or in comments as this makes it harder for reviewers to assess your contribution. We are happy for it to be used to improve grammar or if you are not a native English speaker.
+
+If you used AI tools, please state so in your PR description.
+
+PRs that appear to violate this policy will be closed without review.
+
+For more information on why this is becoming a big issue for open source projects, please see the `Software Review in the Era of AI <https://ropensci.org/blog/2026/02/26/ropensci-ai-policy/>`_ by the rOpenSci folks.
+
+Issues
+------
+We use GitHub issues to track public bugs. Please ensure your description is clear and has sufficient instructions to be able to reproduce the issue.
+
+License
+-------
+By contributing to Python-Blosc2, you agree that your contributions will be licensed under the `LICENSE <https://github.com/Blosc/python-blosc2/blob/main/LICENSE.txt>`_ file of the project.

{blosc2-4.1.2 → blosc2-4.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: blosc2
-Version: 4.1.2
+Version: 4.2.0
 Summary: A fast & compressed ndarray library with a flexible compute engine.
 Author-Email: Blosc Development Team <blosc@blosc.org>
 Maintainer-Email: Blosc Development Team <blosc@blosc.org>
@@ -27,7 +27,11 @@ Requires-Dist: numpy>=1.26
 Requires-Dist: ndindex
 Requires-Dist: msgpack
 Requires-Dist: numexpr>=2.14.1; platform_machine != "wasm32"
+Requires-Dist: pydantic
 Requires-Dist: requests
+Requires-Dist: threadpoolctl; platform_machine != "wasm32"
+Provides-Extra: parquet
+Requires-Dist: pyarrow; extra == "parquet"
 Description-Content-Type: text/x-rst
 
 =============

{blosc2-4.1.2 → blosc2-4.2.0}/README_DEVELOPERS.md

@@ -93,10 +93,40 @@ If you want to run the network tests, you can use the following command:
   pytest -m "network"
 ```
 
+## Matmul backend discovery
+
+The fast `blosc2.matmul` path uses platform-specific block kernels:
+
+- macOS: `Accelerate`
+- Linux/Windows: runtime-discovered `cblas`
+- fallback: portable `naive` kernel
+
+For the runtime `cblas` backend, `python-blosc2` probes the active Python/NumPy
+environment rather than linking to one BLAS vendor at build time.  Discovery
+starts from NumPy's reported BLAS library directory when available, and then
+searches common library names in the active environment's `lib` directories.
+
+On Linux the current candidates include `libcblas`, `libopenblas`,
+`libflexiblas`, `libblis`, `libmkl_rt`, and generic `libblas`.  A candidate is
+accepted only if it loads successfully and exports both `cblas_sgemm` and
+`cblas_dgemm`.  If no suitable provider is found, the fast path falls back to
+the `naive` kernel.
+
+Useful runtime helpers:
+
+- `blosc2.get_matmul_library()` reports the selected runtime library when available
+- `BLOSC_TRACE=1` logs candidate probing, rejection, selection, and backend fallback
+
+Example:
+
+```bash
+BLOSC_TRACE=1 python -c "import blosc2; print(blosc2.get_matmul_library())"
+```
+
 ## wasm32 / Pyodide developer workflow
 
 For the local wasm32 workflow (uv + pyodide-build + cibuildwheel + test loop),
-use the repo skill at `.codex/skills/wasm32-pyodide-dev/SKILL.md`.
+use the repo skill at `.skills/wasm32-pyodide-dev/SKILL.md`.
 
 Install it into Codex discovery with:
 

{blosc2-4.1.2 → blosc2-4.2.0}/RELEASE_NOTES.md

@@ -1,8 +1,86 @@
 # Release notes
 
+## Changes from 4.1.2 to 4.2.0
+
+### CTable: columnar compressed tables
+
+- Introduced `blosc2.CTable`, a new columnar table container for compressed, typed columns.  CTables support dataclass- and schema-based construction, row iteration, column access, table views, `head()` / `tail()` / `sample()`, sorting, selection and compact `where` expressions.
+- Added persistent CTables backed by `TreeStore`, with support for `blosc2.open()`, `CTable.open()`, `CTable.load()`, `CTable.save()`, `CTable.to_b2d()` and `CTable.to_b2z()`.  CTable views can be saved too, and `.b2z`/`.b2d` path handling has been tightened.
+- Added mutation operations for CTables, including `append()`, `extend()`, `delete()`, `compact()`, `add_column()`, `drop_column()`, `rename_column()` and related schema validation.
+- Added computed columns, including virtual computed columns backed by lazy expressions, materialized computed columns and automatic filling of materialized computed columns during inserts.
+- Added CTable indexing support, including persistent indexes, direct expression indexes, ordered index reuse, boolean `LazyExpr`/`NDArray` masks in `CTable.__getitem__`, `iter_sorted()` and indexing support for `.b2z` tables.
+- Added nullable schema support and null policies for CTable scalar columns, preserving nullable scalar Parquet round-trips.
+- Added variable-length CTable column support via `ListArray` / `ObjectArray`, including `vlstring` and `vlbytes` schema specs, fixed-length string/bytes import support and list/struct Arrow/Parquet round-trips.
+- Added Arrow, Parquet and CSV interoperability for CTables, including batch-wise Arrow/Parquet import/export, Arrow schema metadata preservation, `CTable.from_arrow_batches()` improvements and a new `parquet-to-blosc2` CLI utility.
+- Added CTable documentation, tutorials, examples and benchmarks covering schema definition, persistence, querying, indexing, mutations, nullable columns, computed columns and variable-length columns.
+
+### Indexing and ordering
+
+- Added a new indexing subsystem for NDArrays and CTables, including full, partial/bucket, light/medium and OPSI-style index kinds, out-of-core index builders and sidecar storage.
+- Added `blosc2.Index` as the unified public index handle, plus APIs such as `create_index()`, `compact_index()`, `iter_sorted()`, `will_use_index()` and related query explanation support.
+- Added materialized expression indexes for NDArrays and direct expression indexes for CTables.
+- Added persistent query-result caching for indexed lookups, with FIFO pruning and cache accounting.
+- Added `blosc2.argsort()` and refactored indexing APIs around explicit index enums and sorting helpers.
+- Improved indexed query performance with Cython accelerators, threaded chunk batching, zero-copy/cached mmap reads, chunk-aware and reduced-order layouts and faster scattered row gathering.
+- Reduced memory usage during index creation and lookup by avoiding full sidecar materialization, replacing memmap staging with Blosc2 scratch arrays and adding `tmpdir` support for full out-of-core indexes.
+
+### Persistence, stores and serialization
+
+- Added structured Blosc2 serialization based on b2object carriers, including persisted `C2Array`, `LazyExpr` and DSL `LazyUDF` objects.
+- Added `blosc2.Ref` for serializing external references, plus examples for b2object bundles and persisted expressions/UDFs.
+- Added `blosc2.load()` as a convenience loader.
+- Added `vlmeta` support to `LazyArray` objects.
+- Improved store handling by preserving lazy b2object carriers in `DictStore`, allowing reopened proxies to refill caches after read-only opens, relaxing `DictStore`/`TreeStore` suffix requirements and adding `DictStore.to_b2d()`.
+- Accelerated `blosc2.open()` by trying standard opens first and warning on implicit append mode.
+
+### Arrays, computation and containers
+
+- Added `ObjectArray` for fully general object data and renamed the earlier `VLArray` work accordingly; added `ListArray` docstrings and Arrow integration improvements.
+- Added schema helpers including numeric specs, `blosc2.struct()` and `blosc2.object()` for nested/fully general column declarations.
+- Improved `fromiter()` with direct chunked construction and substantially lower peak memory use.
+- Improved `asarray()` behavior for NDArray inputs when copy-inducing keyword arguments are supplied.
+- Added `SChunk.reorder_offsets()`.
+- Improved `BatchArray` defaults and documentation; the default compression level is now tuned for faster lookup/scan behavior.
+- Continued matmul/linalg optimization work and shared-thread-pool integration.
+
+### CLI, docs and examples
+
+- Added the `parquet-to-blosc2` command with options such as `--max-rows`, `--parquet-batch-size`, `--blosc2-items-per-block` and `--use-dict`.
+- Added new CTable, ObjectArray, BatchArray, containers, indexing and serialization tutorials and examples.
+- Reorganized and expanded the API reference for CTable, Column, schema specs, Index, save/load helpers and miscellaneous APIs.
+- Updated benchmark suites for CTables, indexing, Parquet import/export, BatchArray and NDArray construction/indexing.
+
+### Fixes and compatibility
+
+- Updated bundled C-Blosc2 to v3.0.2 and require C-Blosc2 >= 3.0.0 when building against a system library.
+- Updated bundled C-Blosc2 and miniexpr sources multiple times.
+- Restored compatibility with NumPy < 2.
+- Fixed Windows and mmap/file-locking issues in index creation, rebuilds and temporary file cleanup.
+- Fixed full-index query failures for large CTable columns and full out-of-core merge failures on systems with small `/tmp`.
+- Fixed stale sidecar/cache reuse and targeted cache invalidation when persistent sidecars are replaced.
+- Fixed `.b2z` double-open corruption caused by GC-triggered repacking and made temporary `.b2z` unpacking default to the source file directory.
+- Fixed a regression when reopening persisted proxies in read-only mode.
+- Fixed GC-induced thread hangs on macOS with Python 3.14 and hardened async chunk reading/cache cleanup paths.
+- Fixed lazy-chunk source-size handling in decode/getitem callers.
+- Fixed nullable validation, dictionary extend validation, CTable close propagation, print alignment and NumPy mask support.
+- Fixed `arange()` regressions and several pre-existing `set_slice` error-handling issues.
+- Clamped indexing/thread defaults for wasm32.
+
 ## Changes from 4.1.1 to 4.1.2
 
-- Update `c-blosc2` version
+- A new fast path for src/blosc2/linalg.py that uses the matmul prefilter machinery in src/blosc2/blosc2_ext.pyx.
+  - The fast path is only used for supported cases:
+      - blosc2.NDArray inputs
+      - 2-D only
+      - floating-point only
+      - matching dtypes
+      - aligned chunk/block layouts that satisfy the current kernel assumptions
+  - All other valid cases fall back to the existing chunk-by-chunk implementation in src/blosc2/linalg.py.
+  - Some benchmarks for the supported cases show significant speedups over the chunked implementation:
+    - aligned 400x400 float32: about 3.7x faster over chunked
+    - aligned 400x400 float64: about 3.0x
+    - aligned 800x800 float32: about 1.5x
+    - misaligned case: auto correctly stays on chunked
 
 ## Changes from 4.1.0 to 4.1.1
 

{blosc2-4.1.2 → blosc2-4.2.0}/RELEASING.rst

@@ -15,12 +15,10 @@ Preliminaries
 * Make sure that the c-blosc2 repository is updated to the latest version (or a specific
   version that will be documented in the ``RELEASE_NOTES.md``). In ``CMakeLists.txt`` edit::
 
-    FetchContent_Declare(blosc2
-        GIT_REPOSITORY https://github.com/Blosc/c-blosc2
-        GIT_TAG b179abf1132dfa5a263b2ebceb6ef7a3c2890c64
-    )
+    set(BLOSC2_MIN_VERSION 3.0.0)
+    set(BLOSC2_BUNDLED_VERSION v3.0.2)
 
-  to point to the desired commit/tag in the c-blosc2 repo.
+  to point to the desired commit/tag in the c-blosc2 repo. Note that ``conda-forge`` only selects the latest release, so it may be necessary to do a formal release of ``c-blosc2`` to ensure that the package is correctly generated in ```conda-forge``.
 
 * Make sure that the current main branch is passing the tests in continuous integration.
 

{blosc2-4.1.2 → blosc2-4.2.0}/ROADMAP-TO-4.0.md

@@ -22,7 +22,7 @@ The constructor for the `Table` object should take some parameters to specify pr
     * `.where()`: an iterator for querying with conditions that are evaluated with the internal compute engine.
     * `.index()` for indexing a column and getting better performance in queries (desirable, but optional for 4.0).
 
-In particular, it should try to mimic much of the functionality of data-querying libraries such as ``pandas`` (see [this blog](https://datapythonista.me/blog/whats-new-in-pandas-3) for much of the followin). Hence, one should be able to filter rows of the `Table` via querying on multiple columns (accessed via `.` or perhaps ``__getitem__``), with conditions to select rows implemented via `.index`, `.where` like so
+In particular, it should try to mimic much of the functionality of data-querying libraries such as ``pandas`` (see [this blog](https://datapythonista.me/blog/whats-new-in-pandas-3) for much of the following). Hence, one should be able to filter rows of the `Table` via querying on multiple columns (accessed via `.` or perhaps ``__getitem__``), with conditions to select rows implemented via `.index`, `.where` like so
 
 ```
 tbl.where((tbl.property_type == "hotel") & (tbl.country == "us"))

{blosc2-4.1.2 → blosc2-4.2.0}/doc/conf.py

@@ -6,7 +6,7 @@ import sys
 import numpy as np
 
 import blosc2
-from blosc2.utils import constructors, elementwise_funcs, reducers
+from blosc2.utils import elementwise_funcs, reducers
 
 
 def genbody(f, func_list, lib="blosc2"):
@@ -149,38 +149,6 @@ The following functions are useful for working with datatypes.
     )
     genbody(f, blosc2_dtypefuncs)
 
-# GENERATE index_funcs.rst
-blosc2_indexfuncs = sorted(
-    [
-        "count_nonzero",
-        "squeeze",
-        "expand_dims",
-        "sort",
-        "take",
-        "take_along_axis",
-        "broadcast_to",
-        "meshgrid",
-        "indices",
-        "concat",
-        "stack",
-    ]
-)
-
-with open("reference/index_funcs.rst", "w") as f:
-    f.write(
-        """Indexing and Manipulation Functions and Utilities
-=======================================
-
-The following functions are useful for performing indexing and other associated operations.
-
-.. currentmodule:: blosc2
-
-.. autosummary::
-
-"""
-    )
-    genbody(f, blosc2_indexfuncs)
-
 # GENERATE linear_algebra.rst
 linalg_funcs = [
     name
@@ -219,50 +187,6 @@ Reduction operations can be used with any of :ref:`NDArray <NDArray>`, :ref:`C2A
     )
     genbody(f, sorted(reducers))
 
-with open("reference/ndarray.rst", "w") as f:
-    f.write(
-        """.. _NDArray:
-
-NDArray
-=======
-
-The multidimensional data array class. Instances may be constructed using the constructor functions in the list below `NDArrayConstructors`_.
-In addition, all the functions from the :ref:`Lazy Functions <lazy_functions>` section can be used with NDArray instances.
-
-.. currentmodule:: blosc2
-
-.. autoclass:: NDArray
-    :members:
-    :inherited-members:
-    :exclude-members: get_slice, set_slice, get_slice_numpy, get_oindex_numpy, set_oindex_numpy
-    :member-order: groupwise
-
-    :Special Methods:
-
-    .. autosummary::
-
-        __iter__
-        __len__
-        __getitem__
-        __setitem__
-
-    Utility Methods
-    ---------------
-
-    .. automethod:: __iter__
-    .. automethod:: __len__
-    .. automethod:: __getitem__
-    .. automethod:: __setitem__
-
-Constructors
-------------
-.. _NDArrayConstructors:
-.. autosummary::
-
-"""
-    )
-    genbody(f, sorted(constructors))
-
 hidden = "_ignore_multiple_size"
 
 
@@ -296,11 +220,27 @@ def linkcode_resolve(domain, info):
     except Exception:
         return None
 
-    # Replace this with your repo info
     github_base_url = "https://github.com/Blosc/python-blosc2/blob/main/"
-    # Get the path relative to the repository root, not the module directory
-    repo_root = os.path.abspath(os.path.join(os.path.dirname(blosc2.__file__), "..", ".."))
-    relpath = os.path.relpath(fn, start=repo_root)
+    fn = os.path.abspath(fn)
+
+    repo_root = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+    try:
+        relpath = os.path.relpath(fn, start=repo_root)
+    except ValueError:
+        relpath = None
+    if relpath is None or relpath.startswith(".."):
+        # Release docs may be built from an installed wheel/sdist, where source
+        # files live under site-packages/blosc2 instead of the repository's
+        # src/blosc2 tree.  Map those installed package paths back to the repo.
+        package_root = os.path.abspath(os.path.dirname(blosc2.__file__))
+        try:
+            package_relpath = os.path.relpath(fn, start=package_root)
+        except ValueError:
+            return None
+        if package_relpath.startswith(".."):
+            return None
+        relpath = os.path.join("src", "blosc2", package_relpath)
+
     return f"{github_base_url}{relpath}#L{lineno}"
 
 

{blosc2-4.1.2 → blosc2-4.2.0}/doc/getting_started/overview.rst

@@ -34,9 +34,9 @@ and tools in the Python ecosystem, including:
 * DSL kernels for miniexpr-backed UDF authoring and validation (see
   `this tutorial <https://www.blosc.org/python-blosc2/getting_started/tutorials/03.lazyarray-udf-kernels.html>`_).
 * By making use of the simple and open
-`C-Blosc2 format <https://github.com/Blosc/c-blosc2/blob/main/README_FORMAT.rst>`_
-for storing compressed data, Python-Blosc2 facilitates seamless integration with many other
-systems and tools.
+  `C-Blosc2 format <https://github.com/Blosc/c-blosc2/blob/main/README_FORMAT.rst>`_
+  for storing compressed data, Python-Blosc2 facilitates seamless integration with many other
+  systems and tools.
 
 Python-Blosc2's compute engine
 ==============================
@@ -211,7 +211,7 @@ libraries such as NumExpr. Python-Blosc2's compression and chunking means the
 arrays may be stored compressed in memory and then processed chunk-by-chunk;
 both memory footprint and execution time is greatly reduced compared to
 Dask+Zarr, which also uses compression (see the
-`benchmark here <https://github.com/Blosc/python-blosc2/blob/main/bench/ndarray/lazyarray-dask-large.ipynb>`_).
+`larger benchmark here <https://github.com/Blosc/python-blosc2/blob/main/bench/ndarray/lazyarray-dask-large.ipynb>`_).
 
 .. image:: https://github.com/Blosc/python-blosc2/blob/main/images/lazyarray-dask-large.png?raw=true
   :width: 100%
@@ -255,7 +255,7 @@ expression, where the operands are on disk, with the result being a
 1D array stored in memory (or optionally on disk via the ``out=``
 parameter in ``compute()`` or ``sum()`` functions). For a more in-depth look at
 this example, with performance comparisons, see this
-`blog post <https://ironarray.io/blog/compute-bigger>`_.
+`compute-bigger blog post <https://ironarray.io/blog/compute-bigger>`_.
 
 Hopefully, this overview has provided a good understanding of Python-Blosc2's
 capabilities. To begin your journey with Python-Blosc2, proceed to the

{blosc2-4.1.2 → blosc2-4.2.0}/doc/getting_started/tutorials/03.lazyarray-udf-kernels.ipynb

@@ -27,11 +27,11 @@
    "source": [
     "### Choosing the Right Interface\n",
     "\n",
-    "| Goal                                                         | Recommended API |\n",
-    "|--------------------------------------------------------------| --- |\n",
-    "| Elementwise formulas using built-in functions/operators      | `blosc2.lazyexpr(...)` |\n",
-    "| Arbitrary Python logic (including numba) over blocks/chunks  | `blosc2.lazyudf(...)` |\n",
-    "| DSL subset with early syntax checks and optional miniexpr JIT | `@blosc2.dsl_kernel` + `blosc2.lazyudf(...)` |\n"
+    "| Goal                                                         | Recommended API                          |\n",
+    "|--------------------------------------------------------------|------------------------------------------|\n",
+    "| Elementwise formulas using built-in functions/operators      | blosc2.lazyexpr(...)                     |\n",
+    "| Arbitrary Python logic (including numba) over blocks/chunks  | blosc2.lazyudf(...)                      |\n",
+    "| DSL subset with early syntax checks and optional miniexpr JIT | @blosc2.dsl_kernel + blosc2.lazyudf(...) |\n"
    ]
   },
   {