blosc2 3.3.2__tar.gz → 3.3.4__tar.gz

{blosc2-3.3.2 → blosc2-3.3.4}/.pre-commit-config.yaml

@@ -16,7 +16,7 @@ repos:
       - id: trailing-whitespace
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.11.4
+    rev: v0.11.8
     hooks:
       - id: ruff
         args: ["--fix", "--show-fixes"]

{blosc2-3.3.2 → blosc2-3.3.4}/ANNOUNCE.rst

@@ -1,11 +1,11 @@
-Announcing Python-Blosc2 3.3.2
+Announcing Python-Blosc2 3.3.4
 ==============================
 
-This is a bugfix release, with some minor optimizations.  We fixed
-specially a bug in the determination of chunk shape in constructors
-on CPUs with large L3 cache sizes (like AMD EPYC).  We also fixed a
-bug preventing the correct chaining of *string* lazy expressions for
-logical operators (``&``, ``|``, ``^``...).
+This is a bugfix release, with some minor optimizations. We further improved the
+correct chaining of *string* lazy expressions (to allow operands with more
+diverse data types). In addition, both indexing and where expressions are now
+supported within *string* lazy expressions. Finally, casting rules have
+been improved to be more consistent with NumPy.
 
 You can think of Python-Blosc2 3.x as an extension of NumPy/numexpr that:
 

{blosc2-3.3.2 → blosc2-3.3.4}/CMakeLists.txt

@@ -50,7 +50,7 @@ else()
     include(FetchContent)
     FetchContent_Declare(blosc2
         GIT_REPOSITORY https://github.com/Blosc/c-blosc2
-        GIT_TAG 5fcd6fbf9ffcf613fabdb1eb3a90eeb12f7c04fe  # v2.17.1
+        GIT_TAG 4ef3c7440a85632e6c8b6c5d2a9e651e45569fc1  # v2.17.1 + mmap fix
     )
     FetchContent_MakeAvailable(blosc2)
     include_directories("${blosc2_SOURCE_DIR}/include")

{blosc2-3.3.2 → blosc2-3.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: blosc2
-Version: 3.3.2
+Version: 3.3.4
 Summary: A fast & compressed ndarray library with a flexible compute engine.
 Author-Email: Blosc Development Team <blosc@blosc.org>
 Maintainer-Email: Blosc Development Team <blosc@blosc.org>

{blosc2-3.3.2 → blosc2-3.3.4}/RELEASE_NOTES.md

@@ -1,5 +1,49 @@
 # Release notes
 
+## Changes from 3.3.4 to 3.3.5
+
+XXX version-specific blurb XXX
+
+## Changes from 3.3.3 to 3.3.4
+
+* Expand possibilities for chaining string-based lazy expressions to incorporate
+  data types which do not have shape attribute, e.g. int, float etc.
+  See #406 and PR #411.
+
+* Enable slicing within string-based lazy expressions. See PR #414.
+
+* Improved casting for string-based lazy expressions.
+
+* Documentation improvements, see PR #410.
+
+* Compatibility fixes for working with `h5py` files.
+
+## Changes from 3.3.2 to 3.3.3
+
+* Expand possibilities for chaining string-based lazy expressions to include
+  main operand types (LazyExpr and NDArray). Still have to incorporate other
+  data types (which do not have shape attribute, e.g. int, float etc.).
+  See #406.
+
+* Fix indexing for lazy expressions, and allow use of None in getitem.
+  See PR #402.
+
+* Fix incorrect appending of dim to computed reductions. See PR #404.
+
+* Fix `blosc2.linspace()` for incompatible num/shape.  See PR #408.
+
+* Add support for NumPy dtypes that are n-dimensional (e.g.
+  `np.dtype(("<i4,>f4", (10,))),`).
+
+* New MAX_DIM constant for the maximum number of dimensions supported.
+  This is useful for checking if a given array is too large to be handled.
+
+* More refinements on guessing cache sizes for Linux.
+
+* Update to C-Blosc2 2.17.2.dev.  Now, we are forcing the flush of modified
+  pages only in write mode for mmap files. This fixes mmap issues on Windows.
+  Thanks to @JanSellner for the implementation.
+
 ## Changes from 3.3.1 to 3.3.2
 
 * Fixed a bug in the determination of chunk shape for the `NDArray` constructor.
@@ -15,6 +59,8 @@
 * Now, storage defaults (`blosc2.storage_dflts`) are honored, even if no
   `storage=` param is used in constructors.
 
+* We are distributing Python 3.10 wheels now.
+
 ## Changes from 3.3.0 to 3.3.1
 
 * In our effort to better adapt to better adapt to the array API

{blosc2-3.3.2 → blosc2-3.3.4}/doc/python-blosc2.rst

@@ -6,7 +6,7 @@
 
     <p style="text-align: center; color: black; background-color: rgba(230, 169, 9, 0.65);">
         <a href="https://github.com/Blosc/python-blosc2/blob/main/RELEASE_NOTES.md"
-           style="font-size: 1.5em;">Version 3.3.1 released on 2025-04-20!</a>
+           style="font-size: 1.5em;">Version 3.3.3 released on 2025-05-14!</a>
         <span style="display: inline-block; width: 20px;"></span>
         <span style="font-family: monospace;">pip install blosc2 -U</span>
     </p>

{blosc2-3.3.2 → blosc2-3.3.4}/doc/reference/classes.rst

@@ -12,3 +12,4 @@ Main Classes
     proxy
     proxysource
     proxyndsource
+    simpleproxy

{blosc2-3.3.2 → blosc2-3.3.4}/doc/reference/proxysource.rst

@@ -4,8 +4,8 @@ ProxySource
 ===========
 
 Base interface for all supported sources in :ref:`Proxy` and are not NDim objects.
-For example, a file, a memory buffer, a network resource, etc.  For NDims, see
-:ref:`ProxyNDSource`.
+For example, a file, a memory buffer, a network resource, etc.  For n-dimemsional
+ones, see :ref:`ProxyNDSource`.
 
 .. currentmodule:: blosc2.ProxySource
 

blosc2-3.3.4/doc/reference/simpleproxy.rst

@@ -0,0 +1,30 @@
+.. _SimpleProxy:
+
+SimpleProxy
+===========
+
+Simple proxy for a NumPy array (or similar) that can be used with the Blosc2 compute engine.
+
+This only supports the __getitem__ method. No caching is performed.
+
+.. currentmodule:: blosc2.SimpleProxy
+
+Methods
+-------
+
+.. autosummary::
+    :toctree: autofiles/simpleproxy
+    :nosignatures:
+
+    __init__
+    __getitem__
+
+Attributes
+----------
+
+.. autosummary::
+    :toctree: autofiles/simpleproxy
+
+    shape
+    dtype
+    src

{blosc2-3.3.2 → blosc2-3.3.4}/doc/reference/storage.rst

@@ -1,3 +1,5 @@
+.. _CompStorParams:
+
 Compression, decompression and storage parameters
 =================================================
 

{blosc2-3.3.2 → blosc2-3.3.4}/examples/ndarray/jit-numpy-funcs.py

@@ -18,7 +18,7 @@ import blosc2
 # Create some sample data
 a = blosc2.linspace(0, 1, 10 * 100, dtype="float32", shape=(10, 100))
 b = blosc2.linspace(1, 2, 10 * 100, dtype="float32", shape=(10, 100))
-c = blosc2.linspace(-10, 10, 10, dtype="float32", shape=(100,))
+c = blosc2.linspace(-10, 10, 100, dtype="float32", shape=(100,))
 
 
 # Example 1: Basic usage of the jit decorator with reduction

{blosc2-3.3.2 → blosc2-3.3.4}/pyproject.toml

@@ -41,7 +41,7 @@ dependencies = [
     "py-cpuinfo; platform_machine != 'wasm32'",
     "requests",
 ]
-version = "3.3.2"
+version = "3.3.4"
 
 
 [project.optional-dependencies]
@@ -111,7 +111,7 @@ extend-select = [
     "RET",
     "RUF",
     "SIM",
-    "TCH",
+    "TC",
     "UP",
 ]
 ignore = [

{blosc2-3.3.2 → blosc2-3.3.4}/src/blosc2/__init__.py

@@ -109,6 +109,7 @@ from .blosc2_ext import (
     GLOBAL_REGISTERED_CODECS_STOP,
     MAX_BLOCKSIZE,
     MAX_BUFFERSIZE,
+    MAX_DIM,
     MAX_OVERHEAD,
     MAX_TYPESIZE,
     MIN_HEADER_LENGTH,

{blosc2-3.3.2 → blosc2-3.3.4}/src/blosc2/blosc2_ext.pyx

@@ -542,6 +542,7 @@ MAX_TYPESIZE = BLOSC2_MAXTYPESIZE
 MAX_BUFFERSIZE = BLOSC2_MAX_BUFFERSIZE
 MAX_BLOCKSIZE = BLOSC2_MAXBLOCKSIZE
 MAX_OVERHEAD = BLOSC2_MAX_OVERHEAD
+MAX_DIM = B2ND_MAX_DIM
 VERSION_STRING = (<char*>BLOSC2_VERSION_STRING).decode("utf-8")
 VERSION_DATE = (<char*>BLOSC2_VERSION_DATE).decode("utf-8")
 MIN_HEADER_LENGTH = BLOSC_MIN_HEADER_LENGTH
@@ -2391,7 +2392,7 @@ cdef class NDArray:
         str_dtype = bytes_dtype.decode("utf-8")
         try:
             dtype = np.dtype(str_dtype)
-        except TypeError:
+        except (ValueError, TypeError):
             dtype = np.dtype(ast.literal_eval(str_dtype))
         self._dtype = dtype
         return dtype
@@ -2594,8 +2595,33 @@ cdef class NDArray:
 
 
 cdef b2nd_context_t* create_b2nd_context(shape, chunks, blocks, dtype, kwargs):
-    # This is used only in constructors, dtype will always have NumPy format
-    dtype = np.dtype(dtype)
+    if isinstance(dtype, list) and len(dtype) > 0 and isinstance(dtype[0], tuple):
+        # Extract just the field names and basic dtype info
+        fields = []
+        for field in dtype:
+            name = field[0]
+            field_dtype = field[1]
+
+            # Handle different field formats:
+            # 1. ('name', ('|S10', {'h5py_encoding': 'ascii'})) - h5py style
+            # 2. ('name', '<i4') - standard NumPy style
+            # 3. ('name', '<i4', (shape,)) - NumPy with shape info
+
+            if isinstance(field_dtype, tuple) and len(field_dtype) > 0:
+                # h5py nested representation with metadata dict
+                field_dtype = field_dtype[0]
+
+            # Check if we have shape information as third element
+            if len(field) > 2 and field[2] is not None:
+                # Include the shape information
+                fields.append((name, field_dtype, field[2]))
+            else:
+                fields.append((name, field_dtype))
+
+        dtype = np.dtype(fields)
+    else:
+        dtype = np.dtype(dtype)
+
     typesize = dtype.itemsize
     if 'cparams' in kwargs:
         kwargs['cparams']['typesize'] = typesize

{blosc2-3.3.2 → blosc2-3.3.4}/src/blosc2/core.py

@@ -1164,50 +1164,58 @@ def apple_silicon_cache_size(cache_level: int) -> int:
     return size.value
 
 
-def get_l3_cache_info():
+def get_cache_info(cache_level: int) -> tuple:
     result = subprocess.run(["lscpu", "--json"], capture_output=True, text=True)
     lscpu_info = json.loads(result.stdout)
+    if cache_level == 0:
+        cache_level = "1d"
     for entry in lscpu_info["lscpu"]:
-        if entry["field"] == "L3 cache:":
+        if entry["field"] == f"L{cache_level} cache:":
             size_str, instances_str = entry["data"].split(" (")
-            size = int(size_str.split()[0]) * 1024 * 1024  # Convert MiB to bytes
+            size, units = size_str.split()
+            size = int(size)
+            if units == "KiB":
+                size *= 2**10
+            elif units == "MiB":
+                size *= 2**20
+            elif units == "GiB":
+                size *= 2**30
+            else:
+                raise ValueError("Unrecognized unit when guessing cache units")
             instances = int(instances_str.split()[0])
             return size, instances
 
-    raise ValueError("L3 cache not found in lscpu output")
+    raise ValueError(f"L{cache_level} cache not found in lscpu output")
 
 
 def linux_cache_size(cache_level: int, default_size: int) -> int:
     """Get the data cache_level size in bytes for Linux."""
     cache_size = default_size
-    if cache_level == 3:
-        # In modern multicore CPUs, the L3 cache is normally shared among all core complexes (CCX),
-        # but sysfs only reports the cache size for each complex, so better use lscpu, if available.
-        try:
-            l3_cache_size, l3_cache_instances = get_l3_cache_info()
-            # What comes next is a heuristic to guess the most appropriate L3 cache size.
-            # Essentially, this is the result of different experiments, mainly on AMD CPUs
-            # (in particular, Ryzen 9800X3D with 8 cores, and EPYC 9454P with 48 cores).
-            # For Intel, YMMV, but my guess is that they are not using the same CCX approach.
-            l3_cache_size *= l3_cache_instances
-            if l3_cache_instances > 1:
-                # This is yet another heuristic for large CPUs with core sets (CCX) having
-                # their own L3.  No idea why, but it seems to work well.
-                l3_cache_size *= l3_cache_instances // 2
-            return l3_cache_size
-        except (FileNotFoundError, ValueError):
-            # If lscpu is not available or the cache size cannot be read, try with sysfs
-            pass
     try:
+        # Try to read the cache size from sysfs
         with open(f"/sys/devices/system/cpu/cpu0/cache/index{cache_level}/size") as f:
             size = f.read()
             if size.endswith("K\n"):
-                cache_size = int(size[:-2]) * 1024
+                cache_size = int(size[:-2]) * 2**10
             elif size.endswith("M\n"):
-                cache_size = int(size[:-2]) * 1024 * 1024
+                cache_size = int(size[:-2]) * 2**20
+            elif size.endswith("G\n"):
+                cache_size = int(size[:-2]) * 2**30
     except FileNotFoundError:
-        # If the cache size cannot be read, return the default size
-        pass
+        # Try with lscpu, if available.
+        try:
+            cache_size, cache_instances = get_cache_info(cache_level)
+            # cache_instances typically refers to the number of sockets, CCXs or cores,
+            # depending on the CPU and cache level.
+            # In general, dividing the cache size by the number of instances would bring
+            # best performance for private caches (L1 and L2).  For shared caches (L3),
+            # this should be the case as well, but more experimentation is needed.
+            cache_size //= cache_instances
+            return cache_size
+        except (FileNotFoundError, ValueError):
+            # If lscpu is not available or the cache size cannot be read from sysfs,
+            # return the default size.
+            pass
     return cache_size
 
 

{blosc2-3.3.2 → blosc2-3.3.4}/src/blosc2/lazyexpr.py

@@ -37,7 +37,7 @@ import numpy as np
 import blosc2
 from blosc2 import compute_chunks_blocks
 from blosc2.info import InfoReporter
-from blosc2.ndarray import _check_allowed_dtypes, get_chunks_idx, is_inside_new_expr
+from blosc2.ndarray import _check_allowed_dtypes, get_chunks_idx, is_inside_new_expr, process_key
 
 if not blosc2.IS_WASM:
     import numexpr
@@ -116,7 +116,7 @@ dtype_symbols = {
 constructors = ("arange", "linspace", "fromiter", "zeros", "ones", "empty", "full", "frombuffer")
 # Note that, as reshape is accepted as a method too, it should always come last in the list
 constructors += ("reshape",)
-reducers = ("sum", "prod", "min", "max", "std", "mean", "var", "any", "all")
+reducers = ("sum", "prod", "min", "max", "std", "mean", "var", "any", "all", "slice")
 
 functions = [
     "sin",
@@ -541,7 +541,7 @@ def compute_smaller_slice(larger_shape, smaller_shape, larger_slice):
 
 # Define the patterns for validation
 validation_patterns = [
-    r"[\;\[\:]",  # Flow control characters
+    r"[\;]",  # Flow control characters
     r"(^|[^\w])__[\w]+__($|[^\w])",  # Dunder methods
     r"\.\b(?!real|imag|(\d*[eE]?[+-]?\d+)|(\d*[eE]?[+-]?\d+j)|\d*j\b|(sum|prod|min|max|std|mean|var|any|all|where)"
     r"\s*\([^)]*\)|[a-zA-Z_]\w*\s*\([^)]*\))",  # Attribute patterns
@@ -551,7 +551,20 @@ validation_patterns = [
 _blacklist_re = re.compile("|".join(validation_patterns))
 
 # Define valid method names
-valid_methods = {"sum", "prod", "min", "max", "std", "mean", "var", "any", "all", "where", "reshape"}
+valid_methods = {
+    "sum",
+    "prod",
+    "min",
+    "max",
+    "std",
+    "mean",
+    "var",
+    "any",
+    "all",
+    "where",
+    "reshape",
+    "slice",
+}
 valid_methods |= {"int8", "int16", "int32", "int64", "uint8", "uint16", "uint32", "uint64"}
 valid_methods |= {"float32", "float64", "complex64", "complex128"}
 valid_methods |= {"bool", "str", "bytes"}
@@ -579,7 +592,7 @@ def validate_expr(expr: str) -> None:
         raise ValueError(f"'{expr}' is not a valid expression.")
 
     # Check for invalid characters not covered by the tokenizer
-    invalid_chars = re.compile(r"[^\w\s+\-*/%().,=<>!&|~^]")
+    invalid_chars = re.compile(r"[^\w\s+\-*/%()[].,=<>!&|~^]")
     if invalid_chars.search(skip_quotes) is not None:
         invalid_chars = invalid_chars.findall(skip_quotes)
         raise ValueError(f"Expression {expr} contains invalid characters: {invalid_chars}")
@@ -630,6 +643,144 @@ def get_expr_operands(expression: str) -> set:
     return set(visitor.operands)
 
 
+def conserve_functions(  # noqa: C901
+    expression: str,
+    operands_old: dict[str, blosc2.NDArray | blosc2.LazyExpr],
+    operands_new: dict[str, blosc2.NDArray | blosc2.LazyExpr],
+) -> tuple(str, dict[str, blosc2.NDArray]):
+    """
+    Given an expression in string form, return its operands.
+
+    Parameters
+    ----------
+    expression : str
+        The expression in string form.
+
+    operands_old: dict[str : blosc2.ndarray | blosc2.LazyExpr]
+        Dict of operands from expression prior to eval.
+
+    operands_new: dict[str : blosc2.ndarray | blosc2.LazyExpr]
+        Dict of operands from expression after eval.
+    Returns
+    -------
+    newexpression
+        A modified string expression with the functions/constructors conserved and
+        true operands rebased and written in o- notation.
+    newoperands
+        Dict of the set of rebased operands.
+    """
+
+    operand_to_key = {id(v): k for k, v in operands_new.items()}
+    for k, v in operands_old.items():  # extend operands_to_key with old operands
+        if isinstance(
+            v, blosc2.LazyExpr
+        ):  # unroll operands in LazyExpr (only necessary when have reduced a lazyexpr)
+            d = v.operands
+        else:
+            d = {k: v}
+        for newk, newv in d.items():
+            try:
+                operand_to_key[id(newv)]
+            except KeyError:
+                newk = (
+                    f"o{len(operands_new)}" if newk in operands_new else newk
+                )  # possible that names coincide
+                operand_to_key[id(newv)] = newk
+                operands_new[newk] = newv
+
+    class OperandVisitor(ast.NodeVisitor):
+        def __init__(self):
+            self.operandmap = {}
+            self.operands = {}
+            self.opcounter = 0
+            self.function_names = set()
+
+        def update_func(self, localop):
+            k = operand_to_key[id(localop)]
+            if k not in self.operandmap:
+                newkey = f"o{self.opcounter}"
+                self.operands[newkey] = operands_new[k]
+                self.operandmap[k] = newkey
+                self.opcounter += 1
+                return newkey
+            else:
+                return self.operandmap[k]
+
+        def visit_Name(self, node):
+            if node.id == "np":  # Skip NumPy namespace (e.g. np.int8, which will be treated separately)
+                return
+            if node.id in self.function_names:  # Skip function names
+                return
+            elif node.id not in dtype_symbols:
+                localop = operands_old[node.id]
+                if isinstance(localop, blosc2.LazyExpr):
+                    newexpr = localop.expression
+                    for (
+                        opname,
+                        v,
+                    ) in localop.operands.items():  # expression operands already in terms of basic operands
+                        # add illegal character ; to track changed operands and not overwrite later
+                        newopname = ";" + self.update_func(v)
+                        newexpr = re.sub(
+                            rf"(?<=\s){opname}|(?<=\(){opname}", newopname, newexpr
+                        )  # replace with newopname
+                    # remove all instances of ; as all changes completed
+                    node.id = newexpr.replace(";", "")
+                else:
+                    node.id = self.update_func(localop)
+            else:
+                pass
+            self.generic_visit(node)
+
+        def visit_Call(self, node):
+            if isinstance(
+                node.func, ast.Name
+            ):  # visits Call first, then Name, so don't increment operandcounter yet
+                self.function_names.add(node.func.id)
+            self.generic_visit(node)
+
+    tree = ast.parse(expression)
+    visitor = OperandVisitor()
+    visitor.visit(tree)
+    newexpression, newoperands = ast.unparse(tree), visitor.operands
+    return newexpression, newoperands
+
+
+def convert_to_slice(expression):
+    """
+    Takes expression and converts all instances of [] to .slice(....)
+
+    Parameters
+    ----------
+    expression: str
+
+    Returns
+    -------
+    new_expr : str
+    """
+
+    new_expr = ""
+    skip_to_char = 0
+    for i, expr_i in enumerate(expression):
+        if i < skip_to_char:
+            continue
+        if expr_i == "[":
+            k = expression[i:].find("]")  # start checking from after [
+            slice_convert = expression[i : i + k + 1]  # include [ and ]
+            slicer = eval(f"np.s_{slice_convert}")
+            slicer = (slicer,) if not isinstance(slicer, tuple) else slicer  # standardise to tuple
+            if any(isinstance(el, str) for el in slicer):  # handle fields
+                raise ValueError("Cannot handle fields for slicing lazy expressions.")
+            slicer = str(slicer)
+            # use slice so that lazyexpr uses blosc arrays internally
+            # (and doesn't decompress according to getitem syntax)
+            new_expr += ".slice(" + slicer + ")"
+            skip_to_char = i + k + 1
+        else:
+            new_expr += expr_i
+    return new_expr
+
+
 class TransformNumpyCalls(ast.NodeTransformer):
     def __init__(self):
         self.replacements = {}
@@ -1802,9 +1953,18 @@ def infer_dtype(op, value1, value2):
         if op != "~" and value2.dtype != np.bool_:
             raise ValueError(f"Invalid operand type for {op}: {value2.dtype}")
         return np.dtype(np.bool_)
-    dtype1 = value1.dtype if hasattr(value1, "dtype") else np.array(value1).dtype
-    dtype2 = value2.dtype if hasattr(value2, "dtype") else np.array(value2).dtype
-    return np.result_type(dtype1, dtype2)
+
+    # Follow NumPy rules for scalar-array operations
+    # Create small arrays with the same dtypes and let NumPy's type promotion determine the result type
+    if np.isscalar(value1) and hasattr(value2, "shape"):
+        arr2 = np.array([0], dtype=value2.dtype)
+        return (value1 + arr2).dtype
+    elif np.isscalar(value2) and hasattr(value1, "shape"):
+        arr1 = np.array([0], dtype=value1.dtype)
+        return (arr1 + value2).dtype
+    else:
+        # Both are arrays or both are scalars, use NumPy's type promotion rules
+        return np.result_type(value1, value2)
 
 
 class LazyExpr(LazyArray):
@@ -1990,6 +2150,8 @@ class LazyExpr(LazyArray):
 
         operands = {
             key: np.ones(np.ones(len(value.shape), dtype=int), dtype=value.dtype)
+            if hasattr(value, "shape")
+            else value
             for key, value in self.operands.items()
         }
 
@@ -2431,25 +2593,28 @@ class LazyExpr(LazyArray):
     def _compute_expr(self, item, kwargs):  # noqa: C901
         if any(method in self.expression for method in reducers):
             # We have reductions in the expression (probably coming from a string lazyexpr)
+            # Also includes slice
             _globals = get_expr_globals(self.expression)
             lazy_expr = eval(self.expression, _globals, self.operands)
             if not isinstance(lazy_expr, blosc2.LazyExpr):
+                key, _ = process_key(item, self.shape)
                 # An immediate evaluation happened (e.g. all operands are numpy arrays)
                 if hasattr(self, "_where_args"):
                     # We need to apply the where() operation
                     if len(self._where_args) == 1:
                         # We have a single argument
                         where_x = self._where_args["_where_x"]
-                        return where_x[:][lazy_expr]
+                        return (where_x[:][lazy_expr])[key]
                     if len(self._where_args) == 2:
                         # We have two arguments
                         where_x = self._where_args["_where_x"]
                         where_y = self._where_args["_where_y"]
-                        return np.where(lazy_expr, where_x, where_y)
+                        return np.where(lazy_expr, where_x, where_y)[key]
                 if hasattr(self, "_output"):
                     # This is not exactly optimized, but it works for now
-                    self._output[:] = lazy_expr
-                return lazy_expr
+                    self._output[:] = lazy_expr[key]
+                    return self._output
+                return lazy_expr[key]
 
             return chunked_eval(lazy_expr.expression, lazy_expr.operands, item, **kwargs)
 
@@ -2661,6 +2826,7 @@ class LazyExpr(LazyArray):
     def _new_expr(cls, expression, operands, guess, out=None, where=None, ne_args=None):
         # Validate the expression
         validate_expr(expression)
+        expression = convert_to_slice(expression)
         if guess:
             # The expression has been validated, so we can evaluate it
             # in guessing mode to avoid computing reductions
@@ -2676,10 +2842,28 @@ class LazyExpr(LazyArray):
             _dtype = new_expr.dtype
             _shape = new_expr.shape
             if isinstance(new_expr, blosc2.LazyExpr):
-                # Restore the original expression and operands
-                new_expr.expression = f"({_expression})"  # forcibly add parenthesis
-                new_expr.expression_tosave = _expression
-                new_expr.operands = _operands
+                # DO NOT restore the original expression and operands
+                # Instead rebase operands and restore only constructors
+                expression_, operands_ = conserve_functions(
+                    _expression, _operands, new_expr.operands | local_vars
+                )
+                # if new_expr has where_args, must have come from where(...) - or possibly where(where(..
+                # since 5*where, where + ...  are evaluated eagerly
+                if hasattr(new_expr, "_where_args"):
+                    st = expression_.find("where(") + len(
+                        "where("
+                    )  # expr always begins where( - should have st = 6 always
+                    finalexpr = ""
+                    counter = 0
+                    for char in expression_[st:]:  # get rid of external where(...)
+                        finalexpr += char
+                        counter += 1 * (char == "(") - 1 * (char == ")")
+                        if counter == 0 and char == ",":
+                            break
+                    expression_ = finalexpr[:-1]  # remove trailing comma
+                new_expr.expression = f"({expression_})"  # force parenthesis
+                new_expr.expression_tosave = expression
+                new_expr.operands = operands_
                 new_expr.operands_tosave = operands
             else:
                 # An immediate evaluation happened (e.g. all operands are numpy arrays)