taco-eo 0.3.0__tar.gz

taco_eo-0.3.0/.gitignore

@@ -0,0 +1,34 @@
+
+.DS_Store
+
+spec/main.md
+
+demo.zip
+minimal.zip
+/numpy-demo.zip
+/change-detection/
+/image-sequence.zip
+/time-series.zip
+/geospatial.zip
+/partitioned_*.zip
+/.tacocat/
+.vscode/
+
+.pytest_cache/
+.coverage
+__pycache__/
+*.pyc
+build/
+*.egg-info/
+python/.venv/
+python/dist/
+python/uv.lock
+/julia/Manifest.toml
+
+*.dylib
+
+*.o
+
+create-julia-skeleton.sh
+_site/
+numpy_demo.zip

taco_eo-0.3.0/CHANGELOG.md

@@ -0,0 +1,98 @@
+# Changelog
+
+All notable changes to `taco` are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+
+## Unreleased
+
+### Changed
+
+- Python, R, and Julia now share `open_dataset()` and `read()`. An open dataset
+  exposes its sources, collection, and contract in each language.
+- STAC once again represents regular raster chunks with `tensor_shape` and a
+  six-value GDAL `geotransform`; it no longer stores a WKB footprint on every
+  sample. ISTAC is now a distinct irregular-geometry model instead of an alias
+  of STAC. Both retain centroid-based collection summaries, and metadata levels
+  must choose one of the two profiles.
+- The reader keeps one DuckDB connection per thread.
+- Collection and nested sample values are validated before writing.
+- Python checks now cover formatting, strict typing, warnings and branch
+  coverage.
+- The writer internals now follow the build flow explicitly (`core`, `archive`,
+  `folder`, `metadata_tables`, and `staging`) and use descriptive class and
+  method names. STAC/ISTAC definitions moved to `metadata.spatiotemporal`,
+  while `metadata.sample` remains a compatible import facade.
+
+### Removed
+
+- The unused `staging_dir` writer option and `writer.ARCHIVE_SUFFIX` constant.
+
+## 0.2.0 - 2026-09-04
+
+### Changed
+
+- The package is laid out as subpackages: `taco.contract` for the contract
+  vocabulary, `taco.writer` for the two writers and `taco.reader` for metadata
+  access, with `tacocat` and `validate` beside them. The top level exports ten
+  names, down from thirty-eight; everything else is imported from the
+  subpackage that owns it.
+- `COLLECTION.json` records only the fields the caller declares. Nothing is
+  inferred from field names, so the writer no longer guesses an extent from
+  `stac:centroid` or `time_start`, and `_extent.py`, `auto_extent`,
+  `ExtentSpec`, `compute_extent` and `wkb_bounds` are gone. STAC is not part
+  of the core. A TACOCAT still merges the extents its partitions declare,
+  which TACO specification 7.5 requires.
+- Archives are written as `.zip`. cozip specification 14.5 makes the profile
+  byte in the byte-0 index the only authoritative signal, and the extension
+  carries no meaning. Outputs are extensionless or end in `.zip`; any other
+  suffix is rejected.
+- Reading a dataset is the `cozip` DuckDB extension's job. `open_dataset`,
+  `TacoDataset` and `vsi_subfile` are gone from the top-level API;
+  `taco.reader` forwards to the extension and builds no VSI paths itself. The
+  old Python mapping covered four URL schemes and the C++ one covers seven,
+  so the two had already drifted.
+- The ZIP writer now delegates layout planning and serialization to cozip's
+  native TACO ABI through `cozip._taco.plan()` / `write()` instead of driving
+  the low-level entry API itself. Archive names, priority ordering, padding
+  placement and post-write verification are enforced in C. Local source and
+  output paths may contain Unicode.
+- `Contract`, `Collection` and the writers raise `taco.errors.*`
+  exceptions (`ContractError`, `SampleError`, `CollectionError`,
+  `WriterError`, ...). They still subclass `ValueError` / `RuntimeError`.
+- Metadata levels without fields may be omitted from `Contract(metadata=...)`
+  and from `Sample(metadata=...)`.
+- `internal:relative_path` is written in `collection.parquet` too (the sample
+  index), matching the normative text of spec section 7.2.
+- Lossy value conversions (floats into integer fields, bytes into strings,
+  ISO strings into timestamps) are rejected at `add()` time.
+
+### Added
+
+- `taco:structure = null` datasets (one file per sample) in every container.
+- Inline `bytes` assets, materialized into the staging directory on `add()`.
+- `FolderWriter` / `open_folder()` for FOLDER mode with `append=True` and
+  optional hard links.
+- Partitioned builds (`partition_size`, `partition_by`) producing
+  `<stem>_partNNNN.zip` / `<stem>_<value>.zip` plus a `.tacocat` directory;
+  `consolidate()` to build a TACOCAT from existing archives.
+- An internal metadata view for ZIP, FOLDER and TACOCAT, used by
+  `consolidate` and `validate`.
+- `validate()` for structural and contract checks across all three containers.
+- Type specification parser supporting `timestamp[unit, tz]`, `list<...>`,
+  `struct<...>`, `map<...>`, `decimal128(p, s)`, `fixed_size_list<...>`.
+- `parse_size()` and file-name sanitizing for partitioned builds.
+
+### Fixed
+
+- `overwrite=False` uses an atomic no-replace publication step, so a file
+  created by another process during the build is left untouched.
+
+### Removed
+
+- Direct use of `cozip.profile()` and the entry-based `cozip_plan` /
+  `cozip_finalize` calls.
+
+## 0.1.0 - 2026-09-02
+
+- First prototype: `Contract`, `Asset`, `Sample`, `Collection`, staged
+  `TacoWriter` producing profile-2 `.tacozip` archives.

taco_eo-0.3.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Asterisk Laboratories Co-operative Limited
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

taco_eo-0.3.0/PKG-INFO

@@ -0,0 +1,75 @@
+Metadata-Version: 2.5
+Name: taco-eo
+Version: 0.3.0
+Summary: Read and write TACO datasets in Python.
+Project-URL: Homepage, https://asterisk.coop/taco
+Project-URL: Specification, https://asterisk.coop/taco/spec
+Project-URL: Repository, https://github.com/asterisk-labs/taco
+Project-URL: Issues, https://github.com/asterisk-labs/taco/issues
+Author-email: Cesar Aybar <cesar@asterisk.coop>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: cloud-optimized,cozip,earth-observation,geospatial,machine-learning,parquet,taco
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: GIS
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: cozip>=2026.9.4
+Requires-Dist: duckdb>=1.5.5
+Requires-Dist: numpy>=1.24
+Requires-Dist: pyarrow>=14
+Requires-Dist: pydantic>=2.8
+Provides-Extra: dev
+Requires-Dist: mypy>=1.8; extra == 'dev'
+Requires-Dist: pytest-cov>=5; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff>=0.5; extra == 'dev'
+Provides-Extra: geoenrich
+Requires-Dist: earthengine-api>=1; extra == 'geoenrich'
+Provides-Extra: progress
+Requires-Dist: tqdm>=4.66; extra == 'progress'
+Provides-Extra: test
+Requires-Dist: pytest-cov>=5; extra == 'test'
+Requires-Dist: pytest>=8; extra == 'test'
+Description-Content-Type: text/markdown
+
+# taco
+
+Read and write TACO datasets in Python.
+
+```bash
+pip install taco-eo
+python examples/minimal.py
+```
+
+```python
+import taco
+
+samples = taco.read("dataset.zip")
+parts = taco.read(["part-0.zip", "part-1.zip"])
+```
+
+## Examples
+
+Every example is self-contained, uses synthetic data, and writes its output in
+the current directory.
+
+| Example | What it demonstrates |
+| --- | --- |
+| [`minimal.py`](examples/minimal.py) | Smallest possible single-file dataset |
+| [`numpy_minimal.py`](examples/numpy_minimal.py) | NumPy image and mask assets with a train/test split |
+| [`change_detection.py`](examples/change_detection.py) | Metadata on `before/` and `after/` folders |
+| [`sequence.py`](examples/sequence.py) | Variable-length asset sequences |
+| [`time_series.py`](examples/time_series.py) | Per-observation time and cloud metadata |
+| [`geospatial.py`](examples/geospatial.py) | Compact STAC metadata and derived MajorTOM cells |
+| [`stac_segmentation.py`](examples/stac_segmentation.py) | Rich STAC metadata for regular raster chips, labels, bands, scaling, and statistics |
+| [`oceantaco_istac.py`](examples/oceantaco_istac.py) | OceanTACO-inspired ISTAC metadata for irregular SWOT swaths and Argo collocations |
+| [`partitioned.py`](examples/partitioned.py) | ZIP partitions and their TACOCAT catalog |

taco_eo-0.3.0/README.md

@@ -0,0 +1,32 @@
+# taco
+
+Read and write TACO datasets in Python.
+
+```bash
+pip install taco-eo
+python examples/minimal.py
+```
+
+```python
+import taco
+
+samples = taco.read("dataset.zip")
+parts = taco.read(["part-0.zip", "part-1.zip"])
+```
+
+## Examples
+
+Every example is self-contained, uses synthetic data, and writes its output in
+the current directory.
+
+| Example | What it demonstrates |
+| --- | --- |
+| [`minimal.py`](examples/minimal.py) | Smallest possible single-file dataset |
+| [`numpy_minimal.py`](examples/numpy_minimal.py) | NumPy image and mask assets with a train/test split |
+| [`change_detection.py`](examples/change_detection.py) | Metadata on `before/` and `after/` folders |
+| [`sequence.py`](examples/sequence.py) | Variable-length asset sequences |
+| [`time_series.py`](examples/time_series.py) | Per-observation time and cloud metadata |
+| [`geospatial.py`](examples/geospatial.py) | Compact STAC metadata and derived MajorTOM cells |
+| [`stac_segmentation.py`](examples/stac_segmentation.py) | Rich STAC metadata for regular raster chips, labels, bands, scaling, and statistics |
+| [`oceantaco_istac.py`](examples/oceantaco_istac.py) | OceanTACO-inspired ISTAC metadata for irregular SWOT swaths and Argo collocations |
+| [`partitioned.py`](examples/partitioned.py) | ZIP partitions and their TACOCAT catalog |

taco_eo-0.3.0/examples/change_detection.py

@@ -0,0 +1,78 @@
+import io
+
+import numpy as np
+from pydantic import BaseModel
+
+import taco
+
+
+class Acquisition(BaseModel):
+    year: int
+
+
+def encode(array: np.ndarray) -> bytes:
+    buffer = io.BytesIO()
+    np.save(buffer, array)
+    return buffer.getvalue()
+
+
+raster = taco.metadata.asset.Raster
+contract = taco.Contract(
+    structure=[
+        "before/B02.npy",
+        "before/B03.npy",
+        "after/B02.npy",
+        "after/B03.npy",
+        "change.npy",
+    ],
+    metadata=taco.MetadataSchema(
+        taco.Level("sample", ml=taco.metadata.sample.Split),
+        taco.Level("children", acquisition=Acquisition | None, raster=raster | None),
+        taco.Level("children/before", raster=raster),
+        taco.Level("children/after", raster=raster),
+    ),
+)
+collection = taco.Collection(
+    contract=contract,
+    id="change-detection",
+    dataset_version="1.0.0",
+    description="Paired observations and their change mask",
+    licenses=["MIT"],
+    providers=["Asterisk Labs"],
+    tasks=["change-detection"],
+)
+
+before = np.arange(2 * 16 * 16, dtype=np.uint16).reshape(2, 16, 16)
+after = before.copy()
+after[:, 5:11, 7:13] += 100
+paths = {
+    "before/B02.npy": before[0],
+    "before/B03.npy": before[1],
+    "after/B02.npy": after[0],
+    "after/B03.npy": after[1],
+    "change.npy": np.any(before != after, axis=0),
+}
+assets = [
+    taco.Asset(
+        encode(array),
+        path=path,
+        metadata=taco.Metadata(raster=raster(resolution=10, num_bands=1, data_type=str(array.dtype))),
+    )
+    for path, array in paths.items()
+]
+sample = taco.Sample(
+    assets=assets,
+    metadata=taco.Metadata(ml=taco.metadata.sample.Split(split="train")),
+    folders=[
+        taco.Folder("before", metadata=taco.Metadata(acquisition=Acquisition(year=2020))),
+        taco.Folder("after", metadata=taco.Metadata(acquisition=Acquisition(year=2024))),
+    ],
+)
+
+with taco.open_writer(collection, "change-detection", overwrite=True) as writer:
+    writer.add(sample)
+    writer.run()
+
+dataset = taco.open_dataset("change-detection")
+assert taco.read(dataset).num_rows == 1
+assert taco.validate("change-detection").ok

taco_eo-0.3.0/examples/geospatial.py

@@ -0,0 +1,61 @@
+import struct
+from datetime import datetime, timezone
+
+import taco
+
+
+def point(longitude: float, latitude: float) -> bytes:
+    return struct.pack("<BIdd", 1, 1, longitude, latitude)
+
+
+contract = taco.Contract(
+    structure=["image.bin"],
+    metadata=taco.MetadataSchema(
+        taco.Level(
+            "sample",
+            stac=taco.metadata.sample.STAC,
+            majortom=taco.metadata.sample.MajorTOM(
+                dist_km=100,
+                latitude_range=(-20, 0),
+                longitude_range=(-90, -60),
+            ),
+        ),
+        taco.Level("children", raster=taco.metadata.asset.Raster),
+    ),
+)
+collection = taco.Collection(
+    contract=contract,
+    id="peru-sites",
+    dataset_version="1.0.0",
+    description="Small geospatial dataset with derived MajorTOM cells",
+    licenses=["MIT"],
+    providers=["Asterisk Labs"],
+    tasks=["classification"],
+)
+
+sites = [(-77.04, -12.05), (-71.97, -13.53), (-80.63, -5.19)]
+with taco.open_writer(collection, "geospatial.zip", overwrite=True) as writer:
+    for index, (longitude, latitude) in enumerate(sites):
+        location = point(longitude, latitude)
+        stac = taco.metadata.sample.STAC(
+            crs="EPSG:4326",
+            tensor_shape=(1, 8, 8),
+            geotransform=(longitude - 0.05, 0.0125, 0, latitude + 0.05, 0, -0.0125),
+            centroid=location,
+            time_start=datetime(2024, 1, index + 1, tzinfo=timezone.utc),
+        )
+        asset = taco.Asset(
+            bytes([index + 1]) * 64,
+            path="image.bin",
+            metadata=taco.Metadata(raster=taco.metadata.asset.Raster(resolution=10, num_bands=1, data_type="uint8")),
+        )
+        writer.add(taco.Sample(assets=asset, metadata=taco.Metadata(stac=stac)))
+    writer.run()
+
+dataset = taco.open_dataset("geospatial.zip")
+table = taco.read(dataset)
+assert table.num_rows == 3
+assert "majortom:code" in table.column_names
+assert dataset.collection.extent is not None
+assert dataset.collection.extent.spatial == (-80.63, -13.53, -71.97, -5.19)
+assert taco.validate("geospatial.zip").ok

taco_eo-0.3.0/examples/minimal.py

@@ -0,0 +1,17 @@
+import taco
+
+collection = taco.Collection(
+    contract=taco.Contract(structure=None),
+    id="minimal",
+    dataset_version="1.0.0",
+    description="Minimal TACO dataset",
+    licenses=["MIT"],
+    providers=["me"],
+    tasks=["other"],
+)
+with taco.open_writer(collection, "minimal.zip", overwrite=True) as writer:
+    writer.add(taco.Sample(assets=b"hello"))
+    writer.run()
+
+dataset = taco.open_dataset("minimal.zip")
+assert taco.read(dataset).num_rows == 1

taco_eo-0.3.0/examples/numpy_minimal.py

@@ -0,0 +1,48 @@
+import io
+
+import numpy as np
+from pydantic import BaseModel
+
+import taco
+
+
+class ML(BaseModel):
+    split: str
+
+
+def encode(array: np.ndarray) -> bytes:
+    buffer = io.BytesIO()
+    np.save(buffer, array)
+    return buffer.getvalue()
+
+
+contract = taco.Contract(
+    structure=["image.npy", "mask.npy"],
+    metadata=taco.MetadataSchema(taco.Level("sample", ml=ML)),
+)
+collection = taco.Collection(
+    contract=contract,
+    id="numpy-demo",
+    dataset_version="1.0.0",
+    description="Small NumPy dataset",
+    licenses=["MIT"],
+    providers=["me"],
+    tasks=["segmentation"],
+)
+with taco.open_writer(collection, "numpy-demo.zip", overwrite=True) as writer:
+    for index in range(10):
+        image = np.random.default_rng(index).random((3, 32, 32), dtype=np.float32)
+        writer.add(
+            taco.Sample(
+                metadata=taco.Metadata(ml=ML(split="train" if index < 8 else "test")),
+                assets=[
+                    taco.Asset(encode(image), path="image.npy"),
+                    taco.Asset(encode(image[0] > 0.5), path="mask.npy"),
+                ],
+            )
+        )
+    writer.run()
+
+dataset = taco.open_dataset("numpy-demo.zip")
+assert taco.read(dataset).num_rows == 10
+assert taco.validate("numpy-demo.zip").ok