xvrt 0.1.0__tar.gz

xvrt-0.1.0/.gitignore

@@ -0,0 +1,21 @@
+.Rproj.user
+.Rhistory
+.Rdata
+.httr-oauth
+.DS_Store
+.quarto
+xvrt.Rproj
+*.tar.gz
+.positai
+__pycache__/
+*.py[cod]
+.pytest_cache/
+*.egg-info/
+build/
+dist/
+# Quarto
+docs/_site/
+docs/.quarto/
+docs/**/*_files/
+docs/**/*.quarto_ipynb
+docs/**/*.vrt

xvrt-0.1.0/CHANGELOG.md

@@ -0,0 +1,26 @@
+# Changelog
+
+## 0.1.0
+
+First release.
+
+- `write_mdim_vrt()`: GDAL multidimensional VRT from an xarray Dataset.
+  - Compositions `concat` (files already carry the dimension) and `stack`
+    (one file per position of a new dimension, with an optional `index` per
+    source for gaps).
+  - Every data variable becomes an array; sources may be shared or given per
+    variable (`sources={"var": [...]}`); `variables=` selects and orders.
+  - NetCDF/HDF5/Zarr array sources (`<SourceArray>`) and GeoTIFF/COG band
+    sources (`<SourceBand>` + `<SourceTranspose>`).
+  - Packed data: storage dtype, scale, offset and nodata from `.encoding`.
+  - Time: decoded datetime and cftime coordinates re-encoded with their CF
+    units and calendar.
+  - `<SourceSlab>` extents on every source, so reads open only what they need.
+  - `<SRS>` with `dataAxisToSRSAxisMapping` computed from the WKT axis order.
+  - Coordinates as start/step when regular, inline values otherwise, or read
+    from the sources.
+- `from_stac()`: the same from STAC items or a stac-geoparquet file, using
+  catalogue fields only (projection and raster extensions).
+- `Dataset.vrt.write_mdim()` accessor (`import xvrt.accessor`).
+- Output validates against GDAL's `gdalvrt.xsd`, except mdim `<BlockSize>`,
+  written comma-separated as GDAL's driver requires.

xvrt-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Michael Sumner
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

xvrt-0.1.0/PKG-INFO

@@ -0,0 +1,170 @@
+Metadata-Version: 2.5
+Name: xvrt
+Version: 0.1.0
+Summary: Write GDAL multidimensional VRT from xarray Datasets and STAC catalogues (pure Python).
+Project-URL: Documentation, https://hypertidy.github.io/xvrt/
+Project-URL: Source, https://github.com/hypertidy/xvrt
+Project-URL: Issues, https://github.com/hypertidy/xvrt/issues
+Project-URL: Changelog, https://github.com/hypertidy/xvrt/blob/main/CHANGELOG.md
+Author-email: Michael Sumner <mdsumner@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: cog,gdal,multidimensional,netcdf,stac,virtual,vrt,xarray
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: GIS
+Requires-Python: >=3.10
+Requires-Dist: numpy>=1.24
+Requires-Dist: xarray>=2023.12
+Provides-Extra: cf
+Requires-Dist: cf-xarray; extra == 'cf'
+Provides-Extra: rio
+Requires-Dist: rioxarray; extra == 'rio'
+Provides-Extra: stac
+Requires-Dist: pyarrow; extra == 'stac'
+Requires-Dist: pyproj; extra == 'stac'
+Provides-Extra: test
+Requires-Dist: dask; extra == 'test'
+Requires-Dist: h5netcdf; extra == 'test'
+Requires-Dist: lxml; extra == 'test'
+Requires-Dist: netcdf4; extra == 'test'
+Requires-Dist: pyarrow; extra == 'test'
+Requires-Dist: pyproj; extra == 'test'
+Requires-Dist: pytest; extra == 'test'
+Description-Content-Type: text/markdown
+
+# xvrt
+
+Write [GDAL multidimensional VRT](https://gdal.org/en/stable/drivers/raster/vrt_multidimensional.html)
+from xarray Datasets and STAC catalogues. Pure Python: no GDAL needed to
+write.
+
+Documentation: <https://hypertidy.github.io/xvrt/>
+
+A multidimensional VRT is a small XML file that describes a collection of
+files as one n-dimensional dataset: which file supplies which slab of which
+array, plus dimensions, coordinates, CRS, units and packing. It holds no data.
+Any reader of GDAL's multidimensional API opens the whole collection as a
+single cube and reads only the files a request touches.
+
+## Install
+
+```bash
+pip install xvrt
+pip install "xvrt[stac]"   # adds pyarrow and pyproj for from_stac()
+```
+
+Reading the result needs GDAL 3.x with multidimensional support (for example
+GDAL's Python bindings from conda-forge).
+
+## From xarray
+
+Open files the way you normally would, then describe the result:
+
+```python
+import glob
+import xarray as xr
+from xvrt import write_mdim_vrt
+
+paths = sorted(glob.glob("oisst-avhrr-v02r01.*.nc"))   # one day per file
+ds = xr.open_mfdataset(paths, combine="nested", concat_dim="time")
+
+write_mdim_vrt(ds, paths, "oisst.vrt", composition="concat", concat_dim="time")
+```
+
+The same works over HTTP. Ten monthly files of one variable from a THREDDS
+server (URLs become `/vsicurl/` sources):
+
+```python
+PREFIX = "https://thredds.nci.org.au/thredds/fileServer/gb6/BRAN/BRAN2020/daily"
+months = [f"2010_{m:02d}" for m in range(1, 11)]
+paths = [f"{PREFIX}/ocean_temp_{ym}.nc" for ym in months]
+
+ds = xr.open_mfdataset(paths, engine="h5netcdf")
+write_mdim_vrt(ds[["temp"]], paths, "bran.vrt",
+               composition="concat", concat_dim="Time")
+```
+
+## From a STAC catalogue
+
+`from_stac()` builds the VRT from catalogue fields alone (grid, CRS, item
+datetimes, per-asset band storage) without opening a file:
+
+```python
+from xvrt import from_stac
+
+from_stac("https://data.source.coop/ausantarctic/ghrsst-mur-v2/ghrsst-mur-v2.parquet",
+          "mur.vrt", block_size=[1, 512, 512])
+```
+
+For the GHRSST MUR collection that is five arrays over 8913 days in a few
+seconds.
+
+## What it handles
+
+- **Compositions:** `concat` (files already have the dimension) and `stack`
+  (one file per position of a new dimension), with explicit positions and
+  gaps.
+- **Packed data:** storage dtype, scale, offset and nodata are recovered from
+  xarray's `.encoding`, so GDAL decodes the stored bytes exactly as xarray did.
+- **Time:** decoded times are re-encoded with their original CF units and
+  calendar.
+- **Multiple variables:** in the same files, or one source list per variable.
+- **Raster sources:** GeoTIFF/COG bands via `<SourceBand>` and
+  `<SourceTranspose>`.
+- **Lazy reads:** every source states its extent (`<SourceSlab>`), so a read
+  opens only the files it needs.
+- **CRS:** `<SRS>` with the data-axis-to-SRS-axis mapping computed from the
+  WKT axis order.
+
+## Reading the result
+
+```bash
+gdal mdim info oisst.vrt
+```
+
+```python
+from osgeo import gdal
+ds = gdal.OpenEx("oisst.vrt", gdal.OF_MULTIDIM_RASTER)
+sst = ds.GetRootGroup().OpenMDArray("sst").GetUnscaled()
+```
+
+or with an xarray backend for GDAL, such as
+[gdalxarray](https://github.com/hypertidy/gdalxarray):
+`xr.open_dataset("oisst.vrt", engine="gdalxarray")`.
+
+## Development
+
+```bash
+pip install -e ".[test]"
+pytest tests/
+```
+
+`tests/test_gdal_roundtrip.py` reads every written VRT back through GDAL and
+validates it against GDAL's schema (`schema/gdalvrt.xsd`); it is skipped when
+`osgeo.gdal` is not importable. With conda-forge:
+
+```bash
+micromamba create -n xvrt -c conda-forge python gdal libgdal-netcdf xarray netcdf4 h5netcdf dask lxml pyarrow pyproj pytest
+```
+
+The design notes are in [mdim-vrt-writer-plan.md](mdim-vrt-writer-plan.md);
+the documentation site source is in [docs/](docs/).
+
+## Related
+
+- [hypertidy/vrtstack](https://github.com/hypertidy/vrtstack): the R
+  counterpart, writing the same VRT format from raster collections and
+  catalogues.
+- [hypertidy/gdalxarray](https://github.com/hypertidy/gdalxarray): an xarray
+  backend for GDAL that opens these VRTs.
+
+## License
+
+MIT

xvrt-0.1.0/README.md

@@ -0,0 +1,129 @@
+# xvrt
+
+Write [GDAL multidimensional VRT](https://gdal.org/en/stable/drivers/raster/vrt_multidimensional.html)
+from xarray Datasets and STAC catalogues. Pure Python: no GDAL needed to
+write.
+
+Documentation: <https://hypertidy.github.io/xvrt/>
+
+A multidimensional VRT is a small XML file that describes a collection of
+files as one n-dimensional dataset: which file supplies which slab of which
+array, plus dimensions, coordinates, CRS, units and packing. It holds no data.
+Any reader of GDAL's multidimensional API opens the whole collection as a
+single cube and reads only the files a request touches.
+
+## Install
+
+```bash
+pip install xvrt
+pip install "xvrt[stac]"   # adds pyarrow and pyproj for from_stac()
+```
+
+Reading the result needs GDAL 3.x with multidimensional support (for example
+GDAL's Python bindings from conda-forge).
+
+## From xarray
+
+Open files the way you normally would, then describe the result:
+
+```python
+import glob
+import xarray as xr
+from xvrt import write_mdim_vrt
+
+paths = sorted(glob.glob("oisst-avhrr-v02r01.*.nc"))   # one day per file
+ds = xr.open_mfdataset(paths, combine="nested", concat_dim="time")
+
+write_mdim_vrt(ds, paths, "oisst.vrt", composition="concat", concat_dim="time")
+```
+
+The same works over HTTP. Ten monthly files of one variable from a THREDDS
+server (URLs become `/vsicurl/` sources):
+
+```python
+PREFIX = "https://thredds.nci.org.au/thredds/fileServer/gb6/BRAN/BRAN2020/daily"
+months = [f"2010_{m:02d}" for m in range(1, 11)]
+paths = [f"{PREFIX}/ocean_temp_{ym}.nc" for ym in months]
+
+ds = xr.open_mfdataset(paths, engine="h5netcdf")
+write_mdim_vrt(ds[["temp"]], paths, "bran.vrt",
+               composition="concat", concat_dim="Time")
+```
+
+## From a STAC catalogue
+
+`from_stac()` builds the VRT from catalogue fields alone (grid, CRS, item
+datetimes, per-asset band storage) without opening a file:
+
+```python
+from xvrt import from_stac
+
+from_stac("https://data.source.coop/ausantarctic/ghrsst-mur-v2/ghrsst-mur-v2.parquet",
+          "mur.vrt", block_size=[1, 512, 512])
+```
+
+For the GHRSST MUR collection that is five arrays over 8913 days in a few
+seconds.
+
+## What it handles
+
+- **Compositions:** `concat` (files already have the dimension) and `stack`
+  (one file per position of a new dimension), with explicit positions and
+  gaps.
+- **Packed data:** storage dtype, scale, offset and nodata are recovered from
+  xarray's `.encoding`, so GDAL decodes the stored bytes exactly as xarray did.
+- **Time:** decoded times are re-encoded with their original CF units and
+  calendar.
+- **Multiple variables:** in the same files, or one source list per variable.
+- **Raster sources:** GeoTIFF/COG bands via `<SourceBand>` and
+  `<SourceTranspose>`.
+- **Lazy reads:** every source states its extent (`<SourceSlab>`), so a read
+  opens only the files it needs.
+- **CRS:** `<SRS>` with the data-axis-to-SRS-axis mapping computed from the
+  WKT axis order.
+
+## Reading the result
+
+```bash
+gdal mdim info oisst.vrt
+```
+
+```python
+from osgeo import gdal
+ds = gdal.OpenEx("oisst.vrt", gdal.OF_MULTIDIM_RASTER)
+sst = ds.GetRootGroup().OpenMDArray("sst").GetUnscaled()
+```
+
+or with an xarray backend for GDAL, such as
+[gdalxarray](https://github.com/hypertidy/gdalxarray):
+`xr.open_dataset("oisst.vrt", engine="gdalxarray")`.
+
+## Development
+
+```bash
+pip install -e ".[test]"
+pytest tests/
+```
+
+`tests/test_gdal_roundtrip.py` reads every written VRT back through GDAL and
+validates it against GDAL's schema (`schema/gdalvrt.xsd`); it is skipped when
+`osgeo.gdal` is not importable. With conda-forge:
+
+```bash
+micromamba create -n xvrt -c conda-forge python gdal libgdal-netcdf xarray netcdf4 h5netcdf dask lxml pyarrow pyproj pytest
+```
+
+The design notes are in [mdim-vrt-writer-plan.md](mdim-vrt-writer-plan.md);
+the documentation site source is in [docs/](docs/).
+
+## Related
+
+- [hypertidy/vrtstack](https://github.com/hypertidy/vrtstack): the R
+  counterpart, writing the same VRT format from raster collections and
+  catalogues.
+- [hypertidy/gdalxarray](https://github.com/hypertidy/gdalxarray): an xarray
+  backend for GDAL that opens these VRTs.
+
+## License
+
+MIT

xvrt-0.1.0/pyproject.toml

@@ -0,0 +1,54 @@
+[build-system]
+requires = ["hatchling>=1.27"]
+build-backend = "hatchling.build"
+
+[project]
+name = "xvrt"
+dynamic = ["version"]
+description = "Write GDAL multidimensional VRT from xarray Datasets and STAC catalogues (pure Python)."
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Michael Sumner", email = "mdsumner@gmail.com" }]
+keywords = ["gdal", "vrt", "multidimensional", "xarray", "stac", "netcdf", "cog", "virtual"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Science/Research",
+  "Operating System :: OS Independent",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Topic :: Scientific/Engineering :: GIS",
+]
+dependencies = [
+  "numpy>=1.24",
+  "xarray>=2023.12",
+]
+
+[project.optional-dependencies]
+# from_stac(): reading stac-geoparquet, and EPSG codes -> WKT.
+stac = ["pyarrow", "pyproj"]
+# Tests. The GDAL round-trip tests also need osgeo.gdal (conda-forge);
+# they are skipped without it.
+test = ["lxml", "netCDF4", "h5netcdf", "dask", "pytest", "pyarrow", "pyproj"]
+# Extra CRS sources.
+cf = ["cf_xarray"]
+rio = ["rioxarray"]
+
+[project.urls]
+Documentation = "https://hypertidy.github.io/xvrt/"
+Source = "https://github.com/hypertidy/xvrt"
+Issues = "https://github.com/hypertidy/xvrt/issues"
+Changelog = "https://github.com/hypertidy/xvrt/blob/main/CHANGELOG.md"
+
+[tool.hatch.version]
+path = "xvrt/__init__.py"
+
+[tool.hatch.build.targets.wheel]
+packages = ["xvrt"]
+
+[tool.hatch.build.targets.sdist]
+include = ["/xvrt", "/tests", "/schema", "/README.md", "/LICENSE", "/CHANGELOG.md"]

xvrt-0.1.0/schema/README.md

@@ -0,0 +1,13 @@
+# Pinned schema
+
+`gdalvrt.xsd` here is GDAL's own schema, copied from the GDAL 3.13.3
+conda-forge build (`share/gdal/gdalvrt.xsd`). Upstream source:
+
+    https://github.com/OSGeo/gdal/blob/master/frmts/vrt/data/gdalvrt.xsd
+
+`tests/test_gdal_roundtrip.py` validates every VRT it writes against this
+file; the writer itself never reads it.
+
+Known disagreement: the schema types mdim `<BlockSize>` as a space-separated
+`integerList`, but the VRT driver only parses comma-separated values. xvrt
+follows the driver, and the tests ignore that one schema error.