rustytree-xarray 0.1.0__tar.gz → 0.2.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/.github/workflows/ci.yml +1 -1
  2. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/CHANGELOG.md +64 -1
  3. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/Cargo.lock +13 -13
  4. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/Cargo.toml +2 -2
  5. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/PKG-INFO +21 -4
  6. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/README.md +19 -2
  7. rustytree_xarray-0.2.1/assets/logo-banner-dark.png +0 -0
  8. rustytree_xarray-0.2.1/assets/logo-banner-dark.svg +25 -0
  9. rustytree_xarray-0.2.1/assets/logo-banner-light.png +0 -0
  10. rustytree_xarray-0.2.1/assets/logo-banner-light.svg +25 -0
  11. rustytree_xarray-0.2.1/assets/logo.png +0 -0
  12. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/docs/release-process.md +11 -3
  13. rustytree_xarray-0.2.1/notebooks/klot_demo.ipynb +5386 -0
  14. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/pyproject.toml +2 -2
  15. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/python/rustytree/__init__.py +1 -1
  16. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/python/rustytree/backend.py +94 -4
  17. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_backend_entrypoint.py +236 -0
  18. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_phase1_scaffold.py +1 -1
  19. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_to_rust_source.py +13 -0
  20. rustytree_xarray-0.1.0/notebooks/klot_demo.ipynb +0 -4118
  21. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/.github/workflows/release.yml +0 -0
  22. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/.gitignore +0 -0
  23. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/LICENSE +0 -0
  24. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/docs/architecture.md +0 -0
  25. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/docs/contributing.md +0 -0
  26. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/docs/usage.md +0 -0
  27. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/python/rustytree/_array.py +0 -0
  28. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/array.rs +0 -0
  29. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/dtype_dispatch.rs +0 -0
  30. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/error.rs +0 -0
  31. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/glob.rs +0 -0
  32. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/icechunk_store.rs +0 -0
  33. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/lib.rs +0 -0
  34. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/node.rs +0 -0
  35. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/runtime.rs +0 -0
  36. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/store.rs +0 -0
  37. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/url.rs +0 -0
  38. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/src/walk.rs +0 -0
  39. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/conftest.py +0 -0
  40. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_chunks.py +0 -0
  41. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_eager_fetch.py +0 -0
  42. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_icechunk.py +0 -0
  43. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_lazy.py +0 -0
  44. {rustytree_xarray-0.1.0 → rustytree_xarray-0.2.1}/tests/test_walk.py +0 -0
@@ -60,7 +60,7 @@ jobs:
60
60
  # which would re-trigger a full cdylib rebuild on top of maturin develop
61
61
  # below (we'd build the wheel twice — saves ~80s per matrix cell).
62
62
  # Keep this list in sync with the `dev` extra in pyproject.toml.
63
- run: uv pip install --python .venv/bin/python pytest pytest-cov 'zarr>=3.0' 'icechunk>=2.0' ruff
63
+ run: uv pip install --python .venv/bin/python pytest pytest-cov 'zarr>=3.0' 'icechunk>=2.0.5' ruff
64
64
  - name: Build extension (maturin develop)
65
65
  run: .venv/bin/maturin develop
66
66
  - name: pytest
@@ -11,6 +11,68 @@ release, that section is renamed to `[x.y.z] - YYYY-MM-DD` and a fresh
11
11
 
12
12
  ## [Unreleased]
13
13
 
14
+ ## [0.2.1] - 2026-05-23
15
+
16
+ ### Fixed
17
+
18
+ - Compatibility with icechunk 2.0.5: pin Rust-side `icechunk` crate to
19
+ `=2.0.5` and bump Python-side floor to `>=2.0.5`. Fixes #37.
20
+ The `Session::from_bytes` msgpack format changed between 2.0.4 and
21
+ 2.0.5; the 0.2.0 wheel (built against 2.0.4) could not deserialise
22
+ session bytes produced by icechunk-python 2.0.5.
23
+
24
+ ## [0.2.0] - 2026-05-08
25
+
26
+ ### Changed
27
+
28
+ - `include_ancestor_coords=True` against an icechunk Session/Store now
29
+ serialises the session snapshot exactly once, instead of once per
30
+ ancestor. Fixes #34 — addresses the audit finding from #33 that
31
+ each `self.open_dataset(...)` call in the ancestor-merge loop was
32
+ re-running `session.as_bytes()`. `_to_rust_source` short-circuits
33
+ on `bytes` input, and `open_datatree` threads its already-serialised
34
+ `source` through the loop. Behaviour-only refactor; measured saving
35
+ is small (~8 µs per depth-2 call against local KLOT, snapshot
36
+ size 600 B), so this is mostly a code-hygiene fix.
37
+
38
+ ### Added
39
+
40
+ - `include_ancestor_coords=True` (default) on
41
+ `xr.open_datatree(..., engine="rustytree", group=...)` for literal
42
+ group paths. Promotes ancestor group datasets into the new root so
43
+ `latitude`/`longitude`/`altitude` (and any other ancestor-level
44
+ coords/vars) are present after a subtree open — matches what users
45
+ get from a full-tree open + slice with
46
+ `inherit="all_coords"`. Glob mode (`group="*/sweep_0"`) is unchanged
47
+ (already correct via `_filter_by_glob`). Set to `False` to keep the
48
+ pre-flag orphaned-subtree behavior.
49
+
50
+ - Logo + README header. New `assets/logo.png` (transparent icon) and a
51
+ light/dark banner pair (`assets/logo-banner-{light,dark}.png` plus
52
+ SVG sources). README replaces the `# rustytree` heading with a
53
+ `<picture>` tag that auto-switches between light and dark variants
54
+ via `prefers-color-scheme`. Palette aligns with the AtmoScale brand
55
+ (cyan `#00D4FF`, deep navy `#0F1724`) with rust-orange tip accents
56
+ on the chunked-cube tree icon. All text passes WCAG AAA on its
57
+ intended background.
58
+
59
+ - Status badges in the README: CI status, PyPI version, supported
60
+ Python versions, and license — all clickable to the relevant page.
61
+
62
+ ### Fixed
63
+
64
+ - Accept `zarr_format` and `consolidated` kwargs in
65
+ `xr.open_datatree(..., engine="rustytree")` and
66
+ `xr.open_dataset(..., engine="rustytree")`. Previously the entrypoint
67
+ raised `TypeError: unexpected keyword argument` because the kwargs
68
+ weren't declared. They're now declared on both methods and validated:
69
+ v3-implying values (`zarr_format=3` / `None`, `consolidated=False` /
70
+ `None`) pass through silently; v2-implying values (`zarr_format=2`
71
+ or any non-3 int, `consolidated=True`) raise
72
+ `NotImplementedError` pointing the user at `engine="zarr"`. rustytree
73
+ currently supports Zarr v3 only; the icechunk snapshot plays the
74
+ consolidated-metadata role for icechunk repos.
75
+
14
76
  ## [0.1.0] - 2026-05-05
15
77
 
16
78
  First tagged release. The engine is end-to-end usable against
@@ -468,7 +530,8 @@ below.
468
530
  intentionally not normalized — relative-vs-absolute glob
469
531
  semantics differ in `PurePosixPath.match`.
470
532
 
471
- [Unreleased]: https://github.com/aladinor/rustytree/compare/v0.1.0...HEAD
533
+ [Unreleased]: https://github.com/aladinor/rustytree/compare/v0.2.0...HEAD
534
+ [0.2.0]: https://github.com/aladinor/rustytree/compare/v0.1.0...v0.2.0
472
535
  [0.1.0]: https://github.com/aladinor/rustytree/releases/tag/v0.1.0
473
536
  [#1]: https://github.com/aladinor/rustytree/pull/1
474
537
  [#3]: https://github.com/aladinor/rustytree/pull/3
@@ -1846,9 +1846,9 @@ dependencies = [
1846
1846
 
1847
1847
  [[package]]
1848
1848
  name = "icechunk"
1849
- version = "2.0.4"
1849
+ version = "2.0.5"
1850
1850
  source = "registry+https://github.com/rust-lang/crates.io-index"
1851
- checksum = "6cd081b98a0fc1f38f7c8cd0082f4b87075e94f88619838fe7783ac29f1739bb"
1851
+ checksum = "7af416f5b28f3bea35f2d48dd018b7cba711c35a1c965311782222ac337bbccd"
1852
1852
  dependencies = [
1853
1853
  "async-compression",
1854
1854
  "async-recursion",
@@ -1887,9 +1887,9 @@ dependencies = [
1887
1887
 
1888
1888
  [[package]]
1889
1889
  name = "icechunk-arrow-object-store"
1890
- version = "2.0.4"
1890
+ version = "2.0.5"
1891
1891
  source = "registry+https://github.com/rust-lang/crates.io-index"
1892
- checksum = "6b7e797322c9d6bbd3589533cad88019c2efba36b9d0b9806bed2fdbf92b1549"
1892
+ checksum = "8e7610964a5f7859731c95eaec642fcf839e5467137da10d9a2a477c47c1ca2b"
1893
1893
  dependencies = [
1894
1894
  "async-trait",
1895
1895
  "bytes",
@@ -1909,9 +1909,9 @@ dependencies = [
1909
1909
 
1910
1910
  [[package]]
1911
1911
  name = "icechunk-format"
1912
- version = "2.0.4"
1912
+ version = "2.0.5"
1913
1913
  source = "registry+https://github.com/rust-lang/crates.io-index"
1914
- checksum = "77683814f4550c7c01bf48a56575dbf886e274db08e4b58976dc306549c0b1e1"
1914
+ checksum = "258c49d6a1b89fab075fa8b9c31829861ec52f92e5984ab924f4ad834ede13a6"
1915
1915
  dependencies = [
1916
1916
  "base32",
1917
1917
  "bytes",
@@ -1935,9 +1935,9 @@ dependencies = [
1935
1935
 
1936
1936
  [[package]]
1937
1937
  name = "icechunk-s3"
1938
- version = "2.0.4"
1938
+ version = "2.0.5"
1939
1939
  source = "registry+https://github.com/rust-lang/crates.io-index"
1940
- checksum = "56ba9e07dab11d9ded393b5e225fafb722f3b0469c4353725ea9edd05b801185"
1940
+ checksum = "79e08bc924fcc375bf6a2214cc897c09d46b242821999f6c6e3eb87adb0fb0a4"
1941
1941
  dependencies = [
1942
1942
  "async-trait",
1943
1943
  "aws-config",
@@ -1960,9 +1960,9 @@ dependencies = [
1960
1960
 
1961
1961
  [[package]]
1962
1962
  name = "icechunk-storage"
1963
- version = "2.0.4"
1963
+ version = "2.0.5"
1964
1964
  source = "registry+https://github.com/rust-lang/crates.io-index"
1965
- checksum = "7e9e26e88b9ac7442546a1fa3633065997a9bf7fbdd812c3bfe11517e0d3a754"
1965
+ checksum = "d8341b2ce1729d586b9460af93c5c0549d2937df48ad7407101498f56ce99d75"
1966
1966
  dependencies = [
1967
1967
  "async-trait",
1968
1968
  "bytes",
@@ -1981,9 +1981,9 @@ dependencies = [
1981
1981
 
1982
1982
  [[package]]
1983
1983
  name = "icechunk-types"
1984
- version = "2.0.4"
1984
+ version = "2.0.5"
1985
1985
  source = "registry+https://github.com/rust-lang/crates.io-index"
1986
- checksum = "b26efddd99ae14e4106f969567e7cd330501409cf187106199b34b7fdb95c40c"
1986
+ checksum = "3ae8062e3d19227364ba07bd13f2b3bfbb33beaf46883b72b26f36d0ae359449"
1987
1987
  dependencies = [
1988
1988
  "serde",
1989
1989
  "serde_with",
@@ -3346,7 +3346,7 @@ checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
3346
3346
 
3347
3347
  [[package]]
3348
3348
  name = "rustytree"
3349
- version = "0.1.0"
3349
+ version = "0.2.1"
3350
3350
  dependencies = [
3351
3351
  "futures",
3352
3352
  "icechunk",
@@ -1,6 +1,6 @@
1
1
  [package]
2
2
  name = "rustytree"
3
- version = "0.1.0"
3
+ version = "0.2.1"
4
4
  edition = "2024"
5
5
  rust-version = "1.91.1"
6
6
  license = "Apache-2.0"
@@ -45,7 +45,7 @@ serde_json = "1"
45
45
  # icechunk `Session` as `AsyncReadableListableStorageTraits` so the walk
46
46
  # code stays polymorphic: the same `open_single` runs against either an
47
47
  # icechunk session or a vanilla object_store-backed store.
48
- icechunk = "2"
48
+ icechunk = "=2.0.5"
49
49
  zarrs_icechunk = "0.5"
50
50
 
51
51
  [lints.clippy]
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: rustytree-xarray
3
- Version: 0.1.0
3
+ Version: 0.2.1
4
4
  Classifier: Development Status :: 3 - Alpha
5
5
  Classifier: Intended Audience :: Science/Research
6
6
  Classifier: License :: OSI Approved :: Apache Software License
@@ -14,7 +14,7 @@ Requires-Dist: xarray>=2024.10
14
14
  Requires-Dist: pytest>=7.0 ; extra == 'dev'
15
15
  Requires-Dist: pytest-cov>=4.0 ; extra == 'dev'
16
16
  Requires-Dist: zarr>=3.0 ; extra == 'dev'
17
- Requires-Dist: icechunk>=2.0 ; extra == 'dev'
17
+ Requires-Dist: icechunk>=2.0.5 ; extra == 'dev'
18
18
  Requires-Dist: ruff>=0.5.0 ; extra == 'dev'
19
19
  Provides-Extra: dev
20
20
  License-File: LICENSE
@@ -27,9 +27,15 @@ Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
27
27
  Project-URL: Homepage, https://github.com/aladinor/rustytree
28
28
  Project-URL: Repository, https://github.com/aladinor/rustytree
29
29
 
30
- # rustytree
30
+ <picture>
31
+ <source media="(prefers-color-scheme: dark)" srcset="assets/logo-banner-dark.png">
32
+ <img src="assets/logo-banner-light.png" alt="rustytree — Rust-backed xarray DataTree backend" width="720">
33
+ </picture>
31
34
 
32
- > Rust-backed xarray DataTree backend for fast Zarr (incl. icechunk) access from object storage.
35
+ [![CI](https://github.com/aladinor/rustytree/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/aladinor/rustytree/actions/workflows/ci.yml)
36
+ [![PyPI version](https://img.shields.io/pypi/v/rustytree-xarray.svg)](https://pypi.org/project/rustytree-xarray/)
37
+ [![Python versions](https://img.shields.io/pypi/pyversions/rustytree-xarray.svg)](https://pypi.org/project/rustytree-xarray/)
38
+ [![License](https://img.shields.io/pypi/l/rustytree-xarray.svg)](https://github.com/aladinor/rustytree/blob/main/LICENSE)
33
39
 
34
40
  `rustytree` registers as `xr.open_datatree(engine="rustytree")` and walks Zarr
35
41
  v3 hierarchies — both icechunk-backed and vanilla — concurrently in async
@@ -48,6 +54,17 @@ inference), the recursive multi-node walk, glob `group=` filtering, and
48
54
  non-recursive single-Dataset opens are all in. See
49
55
  [`CHANGELOG.md`](CHANGELOG.md) for the per-PR breakdown.
50
56
 
57
+ ## Compatibility
58
+
59
+ | Surface | Supported | Not supported |
60
+ |---|---|---|
61
+ | **Zarr format** | v3 only | v2 — passing `zarr_format=2` or `consolidated=True` raises `NotImplementedError`; use stock `engine="zarr"` for v2 stores |
62
+ | **icechunk** | `icechunk>=2.0` (the current major) | older icechunk releases |
63
+ | **Python** | 3.12, 3.13 | older versions |
64
+ | **Platforms** | manylinux x86_64, manylinux aarch64, macOS arm64 (wheels); other platforms via sdist | Windows / macOS Intel / linux musl wheels (build from source via sdist) |
65
+
66
+ ## Install
67
+
51
68
  Install from PyPI (the import name stays `rustytree`; the
52
69
  distribution name on PyPI is `rustytree-xarray` because `rustytree`
53
70
  collides with an unrelated dormant package):
@@ -1,6 +1,12 @@
1
- # rustytree
1
+ <picture>
2
+ <source media="(prefers-color-scheme: dark)" srcset="assets/logo-banner-dark.png">
3
+ <img src="assets/logo-banner-light.png" alt="rustytree — Rust-backed xarray DataTree backend" width="720">
4
+ </picture>
2
5
 
3
- > Rust-backed xarray DataTree backend for fast Zarr (incl. icechunk) access from object storage.
6
+ [![CI](https://github.com/aladinor/rustytree/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/aladinor/rustytree/actions/workflows/ci.yml)
7
+ [![PyPI version](https://img.shields.io/pypi/v/rustytree-xarray.svg)](https://pypi.org/project/rustytree-xarray/)
8
+ [![Python versions](https://img.shields.io/pypi/pyversions/rustytree-xarray.svg)](https://pypi.org/project/rustytree-xarray/)
9
+ [![License](https://img.shields.io/pypi/l/rustytree-xarray.svg)](https://github.com/aladinor/rustytree/blob/main/LICENSE)
4
10
 
5
11
  `rustytree` registers as `xr.open_datatree(engine="rustytree")` and walks Zarr
6
12
  v3 hierarchies — both icechunk-backed and vanilla — concurrently in async
@@ -19,6 +25,17 @@ inference), the recursive multi-node walk, glob `group=` filtering, and
19
25
  non-recursive single-Dataset opens are all in. See
20
26
  [`CHANGELOG.md`](CHANGELOG.md) for the per-PR breakdown.
21
27
 
28
+ ## Compatibility
29
+
30
+ | Surface | Supported | Not supported |
31
+ |---|---|---|
32
+ | **Zarr format** | v3 only | v2 — passing `zarr_format=2` or `consolidated=True` raises `NotImplementedError`; use stock `engine="zarr"` for v2 stores |
33
+ | **icechunk** | `icechunk>=2.0` (the current major) | older icechunk releases |
34
+ | **Python** | 3.12, 3.13 | older versions |
35
+ | **Platforms** | manylinux x86_64, manylinux aarch64, macOS arm64 (wheels); other platforms via sdist | Windows / macOS Intel / linux musl wheels (build from source via sdist) |
36
+
37
+ ## Install
38
+
22
39
  Install from PyPI (the import name stays `rustytree`; the
23
40
  distribution name on PyPI is `rustytree-xarray` because `rustytree`
24
41
  collides with an unrelated dormant package):