rustytree-xarray 0.1.0__tar.gz → 0.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/.github/workflows/ci.yml +1 -1
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/CHANGELOG.md +64 -1
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/Cargo.lock +13 -13
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/Cargo.toml +2 -2
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/PKG-INFO +21 -4
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/README.md +19 -2
- rustytree_xarray-0.2.1/assets/logo-banner-dark.png +0 -0
- rustytree_xarray-0.2.1/assets/logo-banner-dark.svg +25 -0
- rustytree_xarray-0.2.1/assets/logo-banner-light.png +0 -0
- rustytree_xarray-0.2.1/assets/logo-banner-light.svg +25 -0
- rustytree_xarray-0.2.1/assets/logo.png +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/docs/release-process.md +11 -3
- rustytree_xarray-0.2.1/notebooks/klot_demo.ipynb +5386 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/pyproject.toml +2 -2
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/python/rustytree/__init__.py +1 -1
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/python/rustytree/backend.py +94 -4
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_backend_entrypoint.py +236 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_phase1_scaffold.py +1 -1
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_to_rust_source.py +13 -0
- rustytree_xarray-0.1.0/notebooks/klot_demo.ipynb +0 -4118
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/.github/workflows/release.yml +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/.gitignore +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/LICENSE +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/docs/architecture.md +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/docs/contributing.md +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/docs/usage.md +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/python/rustytree/_array.py +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/array.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/dtype_dispatch.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/error.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/glob.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/icechunk_store.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/lib.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/node.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/runtime.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/store.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/url.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/walk.rs +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/conftest.py +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_chunks.py +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_eager_fetch.py +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_icechunk.py +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_lazy.py +0 -0
- {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_walk.py +0 -0
|
@@ -60,7 +60,7 @@ jobs:
|
|
|
60
60
|
# which would re-trigger a full cdylib rebuild on top of maturin develop
|
|
61
61
|
# below (we'd build the wheel twice — saves ~80s per matrix cell).
|
|
62
62
|
# Keep this list in sync with the `dev` extra in pyproject.toml.
|
|
63
|
-
run: uv pip install --python .venv/bin/python pytest pytest-cov 'zarr>=3.0' 'icechunk>=2.0' ruff
|
|
63
|
+
run: uv pip install --python .venv/bin/python pytest pytest-cov 'zarr>=3.0' 'icechunk>=2.0.5' ruff
|
|
64
64
|
- name: Build extension (maturin develop)
|
|
65
65
|
run: .venv/bin/maturin develop
|
|
66
66
|
- name: pytest
|
|
@@ -11,6 +11,68 @@ release, that section is renamed to `[x.y.z] - YYYY-MM-DD` and a fresh
|
|
|
11
11
|
|
|
12
12
|
## [Unreleased]
|
|
13
13
|
|
|
14
|
+
## [0.2.1] - 2026-05-23
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
|
|
18
|
+
- Compatibility with icechunk 2.0.5: pin Rust-side `icechunk` crate to
|
|
19
|
+
`=2.0.5` and bump Python-side floor to `>=2.0.5`. Fixes #37.
|
|
20
|
+
The `Session::from_bytes` msgpack format changed between 2.0.4 and
|
|
21
|
+
2.0.5; the 0.2.0 wheel (built against 2.0.4) could not deserialise
|
|
22
|
+
session bytes produced by icechunk-python 2.0.5.
|
|
23
|
+
|
|
24
|
+
## [0.2.0] - 2026-05-08
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- `include_ancestor_coords=True` against an icechunk Session/Store now
|
|
29
|
+
serialises the session snapshot exactly once, instead of once per
|
|
30
|
+
ancestor. Fixes #34 — addresses the audit finding from #33 that
|
|
31
|
+
each `self.open_dataset(...)` call in the ancestor-merge loop was
|
|
32
|
+
re-running `session.as_bytes()`. `_to_rust_source` short-circuits
|
|
33
|
+
on `bytes` input, and `open_datatree` threads its already-serialised
|
|
34
|
+
`source` through the loop. Behaviour-only refactor; measured saving
|
|
35
|
+
is small (~8 µs per depth-2 call against local KLOT, snapshot
|
|
36
|
+
size 600 B), so this is mostly a code-hygiene fix.
|
|
37
|
+
|
|
38
|
+
### Added
|
|
39
|
+
|
|
40
|
+
- `include_ancestor_coords=True` (default) on
|
|
41
|
+
`xr.open_datatree(..., engine="rustytree", group=...)` for literal
|
|
42
|
+
group paths. Promotes ancestor group datasets into the new root so
|
|
43
|
+
`latitude`/`longitude`/`altitude` (and any other ancestor-level
|
|
44
|
+
coords/vars) are present after a subtree open — matches what users
|
|
45
|
+
get from a full-tree open + slice with
|
|
46
|
+
`inherit="all_coords"`. Glob mode (`group="*/sweep_0"`) is unchanged
|
|
47
|
+
(already correct via `_filter_by_glob`). Set to `False` to keep the
|
|
48
|
+
pre-flag orphaned-subtree behavior.
|
|
49
|
+
|
|
50
|
+
- Logo + README header. New `assets/logo.png` (transparent icon) and a
|
|
51
|
+
light/dark banner pair (`assets/logo-banner-{light,dark}.png` plus
|
|
52
|
+
SVG sources). README replaces the `# rustytree` heading with a
|
|
53
|
+
`<picture>` tag that auto-switches between light and dark variants
|
|
54
|
+
via `prefers-color-scheme`. Palette aligns with the AtmoScale brand
|
|
55
|
+
(cyan `#00D4FF`, deep navy `#0F1724`) with rust-orange tip accents
|
|
56
|
+
on the chunked-cube tree icon. All text passes WCAG AAA on its
|
|
57
|
+
intended background.
|
|
58
|
+
|
|
59
|
+
- Status badges in the README: CI status, PyPI version, supported
|
|
60
|
+
Python versions, and license — all clickable to the relevant page.
|
|
61
|
+
|
|
62
|
+
### Fixed
|
|
63
|
+
|
|
64
|
+
- Accept `zarr_format` and `consolidated` kwargs in
|
|
65
|
+
`xr.open_datatree(..., engine="rustytree")` and
|
|
66
|
+
`xr.open_dataset(..., engine="rustytree")`. Previously the entrypoint
|
|
67
|
+
raised `TypeError: unexpected keyword argument` because the kwargs
|
|
68
|
+
weren't declared. They're now declared on both methods and validated:
|
|
69
|
+
v3-implying values (`zarr_format=3` / `None`, `consolidated=False` /
|
|
70
|
+
`None`) pass through silently; v2-implying values (`zarr_format=2`
|
|
71
|
+
or any non-3 int, `consolidated=True`) raise
|
|
72
|
+
`NotImplementedError` pointing the user at `engine="zarr"`. rustytree
|
|
73
|
+
currently supports Zarr v3 only; the icechunk snapshot plays the
|
|
74
|
+
consolidated-metadata role for icechunk repos.
|
|
75
|
+
|
|
14
76
|
## [0.1.0] - 2026-05-05
|
|
15
77
|
|
|
16
78
|
First tagged release. The engine is end-to-end usable against
|
|
@@ -468,7 +530,8 @@ below.
|
|
|
468
530
|
intentionally not normalized — relative-vs-absolute glob
|
|
469
531
|
semantics differ in `PurePosixPath.match`.
|
|
470
532
|
|
|
471
|
-
[Unreleased]: https://github.com/aladinor/rustytree/compare/v0.
|
|
533
|
+
[Unreleased]: https://github.com/aladinor/rustytree/compare/v0.2.0...HEAD
|
|
534
|
+
[0.2.0]: https://github.com/aladinor/rustytree/compare/v0.1.0...v0.2.0
|
|
472
535
|
[0.1.0]: https://github.com/aladinor/rustytree/releases/tag/v0.1.0
|
|
473
536
|
[#1]: https://github.com/aladinor/rustytree/pull/1
|
|
474
537
|
[#3]: https://github.com/aladinor/rustytree/pull/3
|
|
@@ -1846,9 +1846,9 @@ dependencies = [
|
|
|
1846
1846
|
|
|
1847
1847
|
[[package]]
|
|
1848
1848
|
name = "icechunk"
|
|
1849
|
-
version = "2.0.
|
|
1849
|
+
version = "2.0.5"
|
|
1850
1850
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
1851
|
-
checksum = "
|
|
1851
|
+
checksum = "7af416f5b28f3bea35f2d48dd018b7cba711c35a1c965311782222ac337bbccd"
|
|
1852
1852
|
dependencies = [
|
|
1853
1853
|
"async-compression",
|
|
1854
1854
|
"async-recursion",
|
|
@@ -1887,9 +1887,9 @@ dependencies = [
|
|
|
1887
1887
|
|
|
1888
1888
|
[[package]]
|
|
1889
1889
|
name = "icechunk-arrow-object-store"
|
|
1890
|
-
version = "2.0.
|
|
1890
|
+
version = "2.0.5"
|
|
1891
1891
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
1892
|
-
checksum = "
|
|
1892
|
+
checksum = "8e7610964a5f7859731c95eaec642fcf839e5467137da10d9a2a477c47c1ca2b"
|
|
1893
1893
|
dependencies = [
|
|
1894
1894
|
"async-trait",
|
|
1895
1895
|
"bytes",
|
|
@@ -1909,9 +1909,9 @@ dependencies = [
|
|
|
1909
1909
|
|
|
1910
1910
|
[[package]]
|
|
1911
1911
|
name = "icechunk-format"
|
|
1912
|
-
version = "2.0.
|
|
1912
|
+
version = "2.0.5"
|
|
1913
1913
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
1914
|
-
checksum = "
|
|
1914
|
+
checksum = "258c49d6a1b89fab075fa8b9c31829861ec52f92e5984ab924f4ad834ede13a6"
|
|
1915
1915
|
dependencies = [
|
|
1916
1916
|
"base32",
|
|
1917
1917
|
"bytes",
|
|
@@ -1935,9 +1935,9 @@ dependencies = [
|
|
|
1935
1935
|
|
|
1936
1936
|
[[package]]
|
|
1937
1937
|
name = "icechunk-s3"
|
|
1938
|
-
version = "2.0.
|
|
1938
|
+
version = "2.0.5"
|
|
1939
1939
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
1940
|
-
checksum = "
|
|
1940
|
+
checksum = "79e08bc924fcc375bf6a2214cc897c09d46b242821999f6c6e3eb87adb0fb0a4"
|
|
1941
1941
|
dependencies = [
|
|
1942
1942
|
"async-trait",
|
|
1943
1943
|
"aws-config",
|
|
@@ -1960,9 +1960,9 @@ dependencies = [
|
|
|
1960
1960
|
|
|
1961
1961
|
[[package]]
|
|
1962
1962
|
name = "icechunk-storage"
|
|
1963
|
-
version = "2.0.
|
|
1963
|
+
version = "2.0.5"
|
|
1964
1964
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
1965
|
-
checksum = "
|
|
1965
|
+
checksum = "d8341b2ce1729d586b9460af93c5c0549d2937df48ad7407101498f56ce99d75"
|
|
1966
1966
|
dependencies = [
|
|
1967
1967
|
"async-trait",
|
|
1968
1968
|
"bytes",
|
|
@@ -1981,9 +1981,9 @@ dependencies = [
|
|
|
1981
1981
|
|
|
1982
1982
|
[[package]]
|
|
1983
1983
|
name = "icechunk-types"
|
|
1984
|
-
version = "2.0.
|
|
1984
|
+
version = "2.0.5"
|
|
1985
1985
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
1986
|
-
checksum = "
|
|
1986
|
+
checksum = "3ae8062e3d19227364ba07bd13f2b3bfbb33beaf46883b72b26f36d0ae359449"
|
|
1987
1987
|
dependencies = [
|
|
1988
1988
|
"serde",
|
|
1989
1989
|
"serde_with",
|
|
@@ -3346,7 +3346,7 @@ checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
|
|
|
3346
3346
|
|
|
3347
3347
|
[[package]]
|
|
3348
3348
|
name = "rustytree"
|
|
3349
|
-
version = "0.1
|
|
3349
|
+
version = "0.2.1"
|
|
3350
3350
|
dependencies = [
|
|
3351
3351
|
"futures",
|
|
3352
3352
|
"icechunk",
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[package]
|
|
2
2
|
name = "rustytree"
|
|
3
|
-
version = "0.1
|
|
3
|
+
version = "0.2.1"
|
|
4
4
|
edition = "2024"
|
|
5
5
|
rust-version = "1.91.1"
|
|
6
6
|
license = "Apache-2.0"
|
|
@@ -45,7 +45,7 @@ serde_json = "1"
|
|
|
45
45
|
# icechunk `Session` as `AsyncReadableListableStorageTraits` so the walk
|
|
46
46
|
# code stays polymorphic: the same `open_single` runs against either an
|
|
47
47
|
# icechunk session or a vanilla object_store-backed store.
|
|
48
|
-
icechunk = "2"
|
|
48
|
+
icechunk = "=2.0.5"
|
|
49
49
|
zarrs_icechunk = "0.5"
|
|
50
50
|
|
|
51
51
|
[lints.clippy]
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: rustytree-xarray
|
|
3
|
-
Version: 0.1
|
|
3
|
+
Version: 0.2.1
|
|
4
4
|
Classifier: Development Status :: 3 - Alpha
|
|
5
5
|
Classifier: Intended Audience :: Science/Research
|
|
6
6
|
Classifier: License :: OSI Approved :: Apache Software License
|
|
@@ -14,7 +14,7 @@ Requires-Dist: xarray>=2024.10
|
|
|
14
14
|
Requires-Dist: pytest>=7.0 ; extra == 'dev'
|
|
15
15
|
Requires-Dist: pytest-cov>=4.0 ; extra == 'dev'
|
|
16
16
|
Requires-Dist: zarr>=3.0 ; extra == 'dev'
|
|
17
|
-
Requires-Dist: icechunk>=2.0 ; extra == 'dev'
|
|
17
|
+
Requires-Dist: icechunk>=2.0.5 ; extra == 'dev'
|
|
18
18
|
Requires-Dist: ruff>=0.5.0 ; extra == 'dev'
|
|
19
19
|
Provides-Extra: dev
|
|
20
20
|
License-File: LICENSE
|
|
@@ -27,9 +27,15 @@ Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
|
|
|
27
27
|
Project-URL: Homepage, https://github.com/aladinor/rustytree
|
|
28
28
|
Project-URL: Repository, https://github.com/aladinor/rustytree
|
|
29
29
|
|
|
30
|
-
|
|
30
|
+
<picture>
|
|
31
|
+
<source media="(prefers-color-scheme: dark)" srcset="assets/logo-banner-dark.png">
|
|
32
|
+
<img src="assets/logo-banner-light.png" alt="rustytree — Rust-backed xarray DataTree backend" width="720">
|
|
33
|
+
</picture>
|
|
31
34
|
|
|
32
|
-
|
|
35
|
+
[](https://github.com/aladinor/rustytree/actions/workflows/ci.yml)
|
|
36
|
+
[](https://pypi.org/project/rustytree-xarray/)
|
|
37
|
+
[](https://pypi.org/project/rustytree-xarray/)
|
|
38
|
+
[](https://github.com/aladinor/rustytree/blob/main/LICENSE)
|
|
33
39
|
|
|
34
40
|
`rustytree` registers as `xr.open_datatree(engine="rustytree")` and walks Zarr
|
|
35
41
|
v3 hierarchies — both icechunk-backed and vanilla — concurrently in async
|
|
@@ -48,6 +54,17 @@ inference), the recursive multi-node walk, glob `group=` filtering, and
|
|
|
48
54
|
non-recursive single-Dataset opens are all in. See
|
|
49
55
|
[`CHANGELOG.md`](CHANGELOG.md) for the per-PR breakdown.
|
|
50
56
|
|
|
57
|
+
## Compatibility
|
|
58
|
+
|
|
59
|
+
| Surface | Supported | Not supported |
|
|
60
|
+
|---|---|---|
|
|
61
|
+
| **Zarr format** | v3 only | v2 — passing `zarr_format=2` or `consolidated=True` raises `NotImplementedError`; use stock `engine="zarr"` for v2 stores |
|
|
62
|
+
| **icechunk** | `icechunk>=2.0` (the current major) | older icechunk releases |
|
|
63
|
+
| **Python** | 3.12, 3.13 | older versions |
|
|
64
|
+
| **Platforms** | manylinux x86_64, manylinux aarch64, macOS arm64 (wheels); other platforms via sdist | Windows / macOS Intel / linux musl wheels (build from source via sdist) |
|
|
65
|
+
|
|
66
|
+
## Install
|
|
67
|
+
|
|
51
68
|
Install from PyPI (the import name stays `rustytree`; the
|
|
52
69
|
distribution name on PyPI is `rustytree-xarray` because `rustytree`
|
|
53
70
|
collides with an unrelated dormant package):
|
|
@@ -1,6 +1,12 @@
|
|
|
1
|
-
|
|
1
|
+
<picture>
|
|
2
|
+
<source media="(prefers-color-scheme: dark)" srcset="assets/logo-banner-dark.png">
|
|
3
|
+
<img src="assets/logo-banner-light.png" alt="rustytree — Rust-backed xarray DataTree backend" width="720">
|
|
4
|
+
</picture>
|
|
2
5
|
|
|
3
|
-
|
|
6
|
+
[](https://github.com/aladinor/rustytree/actions/workflows/ci.yml)
|
|
7
|
+
[](https://pypi.org/project/rustytree-xarray/)
|
|
8
|
+
[](https://pypi.org/project/rustytree-xarray/)
|
|
9
|
+
[](https://github.com/aladinor/rustytree/blob/main/LICENSE)
|
|
4
10
|
|
|
5
11
|
`rustytree` registers as `xr.open_datatree(engine="rustytree")` and walks Zarr
|
|
6
12
|
v3 hierarchies — both icechunk-backed and vanilla — concurrently in async
|
|
@@ -19,6 +25,17 @@ inference), the recursive multi-node walk, glob `group=` filtering, and
|
|
|
19
25
|
non-recursive single-Dataset opens are all in. See
|
|
20
26
|
[`CHANGELOG.md`](CHANGELOG.md) for the per-PR breakdown.
|
|
21
27
|
|
|
28
|
+
## Compatibility
|
|
29
|
+
|
|
30
|
+
| Surface | Supported | Not supported |
|
|
31
|
+
|---|---|---|
|
|
32
|
+
| **Zarr format** | v3 only | v2 — passing `zarr_format=2` or `consolidated=True` raises `NotImplementedError`; use stock `engine="zarr"` for v2 stores |
|
|
33
|
+
| **icechunk** | `icechunk>=2.0` (the current major) | older icechunk releases |
|
|
34
|
+
| **Python** | 3.12, 3.13 | older versions |
|
|
35
|
+
| **Platforms** | manylinux x86_64, manylinux aarch64, macOS arm64 (wheels); other platforms via sdist | Windows / macOS Intel / linux musl wheels (build from source via sdist) |
|
|
36
|
+
|
|
37
|
+
## Install
|
|
38
|
+
|
|
22
39
|
Install from PyPI (the import name stays `rustytree`; the
|
|
23
40
|
distribution name on PyPI is `rustytree-xarray` because `rustytree`
|
|
24
41
|
collides with an unrelated dormant package):
|
|
Binary file
|