abczarr 0.1__tar.gz

abczarr-0.1/LICENSE

@@ -0,0 +1,23 @@
+MIT License
+
+Copyright (c) 2026-     Yael Balbastre
+Copyright (c) 2024-2026 Kaidong Chai
+Copyright (c) 2015-2025 Zarr Developers <https://github.com/zarr-developers>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

abczarr-0.1/PKG-INFO

@@ -0,0 +1,128 @@
+Metadata-Version: 2.4
+Name: abczarr
+Version: 0.1
+Summary: ABC...Z(arr) - Abstract base classes for the Zarr format, and vendored implementations
+Author-email: "Chai, Kaidong" <mail@kchai.me>
+Maintainer-email: "Balbastre, Yael" <y.balbastre@ucl.ac.uk>
+Project-URL: Homepage, https://github.com/neuroscales/abczarr
+Project-URL: Issues, https://github.com/neuroscales/abczarr/issues
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: typing_extensions>=4.13
+Requires-Dist: attrs>=25.2
+Requires-Dist: numpy>=1.20
+Requires-Dist: dask[array,diagnostics]>=2.9.1
+Requires-Dist: bagof-paths>=0.1.0
+Requires-Dist: bagof-converters>=0.1
+Requires-Dist: bagof-validators>=0.1
+Requires-Dist: bagof-factories>=0.1
+Requires-Dist: fastjsonschema>=2.16
+Provides-Extra: tqdm
+Requires-Dist: tqdm; extra == "tqdm"
+Provides-Extra: upath
+Requires-Dist: universal-pathlib; extra == "upath"
+Provides-Extra: anypath
+Requires-Dist: cloudpathlib; extra == "anypath"
+Provides-Extra: ml-dtypes
+Requires-Dist: ml_dtypes>=0.2; extra == "ml-dtypes"
+Provides-Extra: zarr-py
+Requires-Dist: zarr; extra == "zarr-py"
+Provides-Extra: tensorstore
+Requires-Dist: tensorstore; extra == "tensorstore"
+Provides-Extra: zarrista
+Requires-Dist: zarrista; extra == "zarrista"
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest-cov; extra == "test"
+Provides-Extra: doc
+Requires-Dist: zensical>=0.0.11; extra == "doc"
+Requires-Dist: mkdocstrings[python]>=1.0.0; extra == "doc"
+Requires-Dist: markdown-grid-tables; extra == "doc"
+Dynamic: license-file
+
+# abczarr
+
+<img src="https://neuroscales.github.io/abczarr/images/logo_title_color.svg" style="display: block; margin: 0 auto; width: 75%; height: auto;" alt="abczarr logo" />
+
+One interface for reading and writing Zarr arrays and groups, synchronously
+or asynchronously, regardless of which backend or storage location holds
+them.
+
+## What it does
+
+Zarr has several capable Python implementations
+([`zarr-python`](https://github.com/zarr-developers/zarr-python),
+[`tensorstore`](https://github.com/google/tensorstore/),
+[`zarrista`](https://github.com/developmentseed/zarrista)), and they
+differ in what they support and in the API each one exposes. abczarr sits
+above these implementations and exposes a single API, `ZarrArray` and
+`ZarrGroup`, and selects whichever backend actually supports the requested
+operation. A backend can also be named explicitly when a specific one is
+required. Local paths and cloud URLs such as `s3://` or `gs://` are handled
+the same way, regardless of which backend is in use.
+
+Every node exists in both a synchronous and an asynchronous form, and the
+two behave identically regardless of what the underlying backend supports.
+When a backend provides real asynchronous I/O, abczarr uses it directly.
+When a backend does not, the call runs in a background thread instead.
+Either form returns an object with the same behavior, and the asynchronous
+form can be awaited.
+
+Backends genuinely differ in what they support. abczarr exposes a backend's
+capabilities so that they can be checked before an operation runs, instead
+of discovering a gap partway through a write. When an operation is not
+supported, the resulting error names what is missing, instead of
+surfacing as a stack trace from inside a driver's internals.
+
+Zarr's array metadata has changed shape across versions. Versions v1, v2,
+and v3 each describe an array a little differently. abczarr models all
+three as one typed, validated object that converts between versions. When
+a conversion can carry an option across cleanly, it does. When it cannot,
+the missing option can be configured to be dropped silently, to raise a
+warning, or to raise a hard error. OME-Zarr metadata receives the same
+treatment: a typed model, schema validation that runs offline, and
+conversion between OME-NGFF versions.
+
+```python
+from abczarr import open
+
+# opens a Zarr node at any location, with the backend chosen automatically
+group = open("s3://my-bucket/dataset.zarr")
+array = group["images"]
+data = array[:100, :100]
+
+# a backend can also be named explicitly
+volume = open("data.zarr", driver="tensorstore")
+
+# the asynchronous form awaits open() and each subsequent method call
+agroup = await open("s3://my-bucket/dataset.zarr", asynchronous=True)
+aimages = await agroup.getitem("images")
+tile = await aimages.getitem((slice(100), slice(100)))
+```
+
+## Install
+
+```sh
+pip install abczarr
+```
+
+The core installs no backend of its own. The driver needed for reading
+and writing, along with any required storage or dtype support, is added
+through extras:
+
+| extra | enables |
+| --- | --- |
+| `abczarr[zarr-py]` | the zarr-python driver |
+| `abczarr[tensorstore]` | the TensorStore driver |
+| `abczarr[zarrista]` | the zarrista driver |
+| `abczarr[upath]` | fsspec and cloud URLs (`s3://`, `gs://`, ...) via universal-pathlib |
+| `abczarr[anypath]` | cloud paths via cloudpathlib |
+| `abczarr[ml-dtypes]` | exotic v3 float dtypes (`bfloat16`, `float8_*`, ...) |
+
+Extras can be combined, for example `pip install "abczarr[zarr-py,upath]"`.
+
+## Learn more
+
+Full documentation lives at
+[neuroscales.github.io/abczarr](https://neuroscales.github.io/abczarr/).

abczarr-0.1/README.md

@@ -0,0 +1,85 @@
+# abczarr
+
+<img src="https://neuroscales.github.io/abczarr/images/logo_title_color.svg" style="display: block; margin: 0 auto; width: 75%; height: auto;" alt="abczarr logo" />
+
+One interface for reading and writing Zarr arrays and groups, synchronously
+or asynchronously, regardless of which backend or storage location holds
+them.
+
+## What it does
+
+Zarr has several capable Python implementations
+([`zarr-python`](https://github.com/zarr-developers/zarr-python),
+[`tensorstore`](https://github.com/google/tensorstore/),
+[`zarrista`](https://github.com/developmentseed/zarrista)), and they
+differ in what they support and in the API each one exposes. abczarr sits
+above these implementations and exposes a single API, `ZarrArray` and
+`ZarrGroup`, and selects whichever backend actually supports the requested
+operation. A backend can also be named explicitly when a specific one is
+required. Local paths and cloud URLs such as `s3://` or `gs://` are handled
+the same way, regardless of which backend is in use.
+
+Every node exists in both a synchronous and an asynchronous form, and the
+two behave identically regardless of what the underlying backend supports.
+When a backend provides real asynchronous I/O, abczarr uses it directly.
+When a backend does not, the call runs in a background thread instead.
+Either form returns an object with the same behavior, and the asynchronous
+form can be awaited.
+
+Backends genuinely differ in what they support. abczarr exposes a backend's
+capabilities so that they can be checked before an operation runs, instead
+of discovering a gap partway through a write. When an operation is not
+supported, the resulting error names what is missing, instead of
+surfacing as a stack trace from inside a driver's internals.
+
+Zarr's array metadata has changed shape across versions. Versions v1, v2,
+and v3 each describe an array a little differently. abczarr models all
+three as one typed, validated object that converts between versions. When
+a conversion can carry an option across cleanly, it does. When it cannot,
+the missing option can be configured to be dropped silently, to raise a
+warning, or to raise a hard error. OME-Zarr metadata receives the same
+treatment: a typed model, schema validation that runs offline, and
+conversion between OME-NGFF versions.
+
+```python
+from abczarr import open
+
+# opens a Zarr node at any location, with the backend chosen automatically
+group = open("s3://my-bucket/dataset.zarr")
+array = group["images"]
+data = array[:100, :100]
+
+# a backend can also be named explicitly
+volume = open("data.zarr", driver="tensorstore")
+
+# the asynchronous form awaits open() and each subsequent method call
+agroup = await open("s3://my-bucket/dataset.zarr", asynchronous=True)
+aimages = await agroup.getitem("images")
+tile = await aimages.getitem((slice(100), slice(100)))
+```
+
+## Install
+
+```sh
+pip install abczarr
+```
+
+The core installs no backend of its own. The driver needed for reading
+and writing, along with any required storage or dtype support, is added
+through extras:
+
+| extra | enables |
+| --- | --- |
+| `abczarr[zarr-py]` | the zarr-python driver |
+| `abczarr[tensorstore]` | the TensorStore driver |
+| `abczarr[zarrista]` | the zarrista driver |
+| `abczarr[upath]` | fsspec and cloud URLs (`s3://`, `gs://`, ...) via universal-pathlib |
+| `abczarr[anypath]` | cloud paths via cloudpathlib |
+| `abczarr[ml-dtypes]` | exotic v3 float dtypes (`bfloat16`, `float8_*`, ...) |
+
+Extras can be combined, for example `pip install "abczarr[zarr-py,upath]"`.
+
+## Learn more
+
+Full documentation lives at
+[neuroscales.github.io/abczarr](https://neuroscales.github.io/abczarr/).

abczarr-0.1/pyproject.toml

@@ -0,0 +1,116 @@
+[project]
+name = "abczarr"
+authors = [{name = "Chai, Kaidong", email = "mail@kchai.me"}]
+maintainers = [{name = "Balbastre, Yael", email = "y.balbastre@ucl.ac.uk"}]
+description = "ABC...Z(arr) - Abstract base classes for the Zarr format, and vendored implementations"
+readme = "README.md"
+dynamic = ["version"]
+requires-python = ">=3.8"
+dependencies = [
+    "typing_extensions >= 4.13",            # shared floor of the bagof-* engine
+    "attrs >= 25.2",                        # @define
+    "numpy >= 1.20",                        # numpy.typing.DTypeLike
+    "dask[array,diagnostics] >= 2.9.1",     # nanmedian
+    # One pathlib-style API over local and cloud paths. universal-pathlib /
+    # cloudpathlib stay optional extras below -- the core wraps stdlib pathlib
+    # and needs neither.
+    "bagof-paths >= 0.1.0",
+    # Hint-driven converters / validators / factories. abczarr's metadata
+    # model derives per-field conversion and validation from type hints; these
+    # provide that engine (the same one bagof-magic uses), replacing the
+    # home-grown version under _core/auto.
+    "bagof-converters >= 0.1",
+    "bagof-validators >= 0.1",
+    "bagof-factories >= 0.1",
+    # Offline JSON-schema validation of OME-Zarr / Zarr metadata against the
+    # vendored NGFF and Zarr schemas. Pure-Python, zero runtime dependencies,
+    # no python_requires floor (installs on the 3.8 floor), and fast (compiles
+    # each schema to a validation function).
+    "fastjsonschema >= 2.16",
+]
+
+[project.optional-dependencies]
+
+# fancy
+tqdm = ["tqdm"]
+upath = ["universal-pathlib"]
+anypath = ["cloudpathlib"]
+
+# dtypes
+# Registers exotic v3 extension floats (bfloat16, float8_*, ...) with numpy
+# on import, so ``asdtype`` resolves them. Optional -- the core needs none of
+# it. The 0.2 floor is what pip resolves on the 3.8 leg; newer wheels are
+# selected automatically on newer interpreters.
+ml-dtypes = ["ml_dtypes >= 0.2"]
+
+# drivers
+zarr-py = ["zarr"]
+tensorstore = ["tensorstore"]
+zarrista = ["zarrista"]
+
+# test
+test = ["pytest", "pytest-cov"]
+
+# doc
+doc = [
+  "zensical>=0.0.11",
+  "mkdocstrings[python]>=1.0.0",
+  "markdown-grid-tables",
+]
+
+[project.urls]
+Homepage = "https://github.com/neuroscales/abczarr"
+Issues = "https://github.com/neuroscales/abczarr/issues"
+
+[tool.setuptools.package-data]
+# Ship the JSON schemas (they live in non-package data directories under the
+# schemas packages, so they are not picked up as modules).
+"abczarr.ome.schemas" = ["_ngff/*/*.schema", "_ngff/*.md"]
+"abczarr.schemas" = [
+    "_zarr/*.md",
+    "_zarr/*/*.schema",
+    "_zarr/*/*/*.schema",
+    "_zarr/v3/extensions/*/*/schema.json",
+]
+
+[build-system]
+requires = [
+    "setuptools>=59",
+    "wheel",
+    "versioningit>=1.0",
+]
+build-backend = "setuptools.build_meta"
+
+[tool.versioningit]
+default-version = "0+unknown"
+
+[tool.versioningit.vcs]
+default-tag = "0.1"
+
+[tool.versioningit.format]
+distance = "{base_version}+{distance}.{vcs}{rev}"
+dirty = "{base_version}+{distance}.{vcs}{rev}.dirty"
+distance-dirty = "{base_version}+{distance}.{vcs}{rev}.dirty"
+
+[tool.versioningit.write]
+file = "src/abczarr/_version.py"
+
+[tool.pytest.ini_options]
+pythonpath = ["."]
+testpaths = ["tests"]
+
+[tool.ruff]
+line-length = 79
+target-version = "py38"
+
+[tool.ruff.lint]
+select = ["ANN", "B", "E", "F", "I", "UP", "W"]
+ignore = [
+    "ANN002",  # Allow missing type annotations for *args
+    "ANN003",  # Allow missing type annotations for **kwargs
+    "ANN401",  # Allow annotating with Any
+    "UP006",   # Allow typing.List/Type over the builtin generic
+    "UP007",   # Allow typing.Optional/Union over the X | Y form
+    "UP035",   # Allow importing from typing / typing_extensions
+    "UP045",   # Allow typing.Optional[X] over X | None
+]

abczarr-0.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

abczarr-0.1/src/abczarr/__init__.py

@@ -0,0 +1,99 @@
+"""abczarr: one interface for reading and writing Zarr, over any backend.
+
+Open or create a node with [open][abczarr.open] and [create][abczarr.create],
+and read or write it through the uniform
+[ZarrArray][abczarr.abc.sync.ZarrArray] /
+[ZarrGroup][abczarr.abc.sync.ZarrGroup] surface, whatever backend or storage is
+behind it. The whole user-facing API is
+re-exported here at the top level: the [ArrayConfig][abczarr.ArrayConfig] and
+[GroupConfig][abczarr.GroupConfig] that creation rests on, the
+[select_driver][abczarr.select_driver] registry that picks a backend, and the
+errors abczarr raises. The open/create, config and registry names are also
+available under [api][abczarr.api]; the errors keep their own home in
+[abczarr.errors][abczarr.errors].
+"""
+
+__all__ = [
+    # subpackages
+    "abc",
+    "api",
+    "drivers",
+    "errors",
+    "metadata",
+    "ome",
+    "schemas",
+    # node surface
+    "ZarrArray",
+    "ZarrGroup",
+    "ZarrNode",
+    "AsyncZarrArray",
+    "AsyncZarrGroup",
+    "AsyncZarrNode",
+    # open / create
+    "open",
+    "open_array",
+    "open_group",
+    "create",
+    "create_array",
+    "create_group",
+    # config
+    "ZarrConfig",
+    "GroupConfig",
+    "ArrayConfig",
+    "ZarrOptions",
+    "GroupOptions",
+    "ArrayOptions",
+    # driver registry
+    "register_driver",
+    "available_drivers",
+    "select_driver",
+    # errors
+    "UnsupportedZarrOperation",
+    "UnsupportedConversion",
+    "TransactionConflict",
+    "SchemaValidationError",
+]
+
+# Subpackages first: importing `drivers` fully loads `drivers.base` (which
+# imports the config layer) before the re-exports below reach for the
+# reader/writer or the registry, so those lazy imports never cycle back.
+from . import (
+    abc,
+    api,
+    drivers,
+    errors,
+    metadata,
+    ome,
+    schemas,
+)
+from .abc import (
+    AsyncZarrArray,
+    AsyncZarrGroup,
+    AsyncZarrNode,
+    ZarrArray,
+    ZarrGroup,
+    ZarrNode,
+)
+from .api import (
+    ArrayConfig,
+    ArrayOptions,
+    GroupConfig,
+    GroupOptions,
+    ZarrConfig,
+    ZarrOptions,
+    available_drivers,
+    create,
+    create_array,
+    create_group,
+    open,
+    open_array,
+    open_group,
+    register_driver,
+    select_driver,
+)
+from .errors import (
+    SchemaValidationError,
+    TransactionConflict,
+    UnsupportedConversion,
+    UnsupportedZarrOperation,
+)

abczarr-0.1/src/abczarr/_core/asyncutils.py

@@ -0,0 +1,102 @@
+"""Small async helpers: run a blocking call in a bounded pool, and fan a
+batch of coroutines out with a concurrency cap.
+
+abczarr synthesizes the async-from-sync direction only (a synchronous
+backend run in a worker thread). The thread pool here is a dedicated,
+bounded one -- not asyncio's default executor -- so a burst of chunk I/O
+cannot starve the interpreter's shared pool, and the fan-out helper carries
+a default concurrency limit so a wide ``gather`` does not open thousands of
+threads or backend connections at once.
+"""
+
+__all__ = [
+    "DEFAULT_CONCURRENCY",
+    "run_sync",
+    "concurrent_map",
+    "ensure_coroutine",
+]
+
+# stdlib
+import asyncio
+import os
+from concurrent.futures import ThreadPoolExecutor
+from functools import partial
+
+# dependencies
+import typing_extensions as tx
+
+# typing
+T = tx.TypeVar("T", bound=tx.Tuple[tx.Any, ...])
+V = tx.TypeVar("V")
+
+#: The default fan-out width, matching the thread pool's size, so a batch
+#: run through :func:`concurrent_map` never wants more workers than there
+#: are threads to serve it.
+DEFAULT_CONCURRENCY = min(32, (os.cpu_count() or 1) + 4)
+
+#: The dedicated executor, built on first use. Kept apart from asyncio's
+#: default (``None``) pool so abczarr's blocking chunk I/O never contends
+#: with whatever else a caller's event loop offloads.
+_executor = None  # type: tx.Optional[ThreadPoolExecutor]
+
+
+def _thread_pool() -> ThreadPoolExecutor:
+    """The shared bounded thread pool, created on first use."""
+    global _executor
+    if _executor is None:
+        _executor = ThreadPoolExecutor(
+            max_workers=DEFAULT_CONCURRENCY,
+            thread_name_prefix="abczarr-async",
+        )
+    return _executor
+
+
+async def run_sync(
+    func: tx.Callable[..., V], *args: tx.Any, **kwargs: tx.Any
+) -> V:
+    """Run the blocking *func* in the dedicated thread pool and await it.
+
+    A cancelled ``await`` cannot interrupt the running thread, so a write
+    already handed to the backend may still land even when the awaiting task
+    is cancelled.
+    """
+    loop = asyncio.get_running_loop()
+    call = partial(func, *args, **kwargs)
+    return await loop.run_in_executor(_thread_pool(), call)
+
+
+async def concurrent_map(
+    items: tx.Iterable[T],
+    func: tx.Callable[..., tx.Awaitable[V]],
+    limit: tx.Optional[int] = DEFAULT_CONCURRENCY,
+) -> tx.List[V]:
+    """Await *func* over each of *items*, at most *limit* at a time.
+
+    Each item is a tuple of positional arguments for *func*. Results come
+    back in the order of *items*. A *limit* of ``None`` runs them all at
+    once (unbounded); the default caps the fan-out so a wide batch does not
+    open more connections or threads than the pool can serve.
+    """
+    if limit is None:
+        return await asyncio.gather(*[func(*item) for item in items])
+
+    sem = asyncio.Semaphore(limit)
+
+    async def run(item: T) -> V:
+        async with sem:
+            return await func(*item)
+
+    return await asyncio.gather(*[run(item) for item in items])
+
+
+def ensure_coroutine(
+    fn: tx.Callable[..., tx.Any]
+) -> tx.Callable[..., tx.Awaitable[tx.Any]]:
+    """Adapt *fn* to a coroutine function.
+
+    A coroutine function is returned unchanged; a plain callable is wrapped
+    so calling it runs it in the thread pool and returns an awaitable.
+    """
+    if asyncio.iscoroutinefunction(fn):
+        return fn
+    return partial(run_sync, fn)