euler-loading 2.26.0__tar.gz → 2.29.0__tar.gz

{euler_loading-2.26.0 → euler_loading-2.29.0}/.gitignore

@@ -42,3 +42,8 @@ TASK.md
 
 # uv
 uv.lock
+
+# Stray npm lockfile. Nothing in this package is built with npm, and worktree
+# tooling rewrites its "name" field to the branch directory on every branch,
+# so tracking it produced a merge conflict on every quick-merge.
+package-lock.json

euler_loading-2.29.0/CHANGELOG.md

@@ -0,0 +1,100 @@
+# Changelog
+
+## Unreleased
+
+- `Modality.function` picks a callable from the loader module the dataset head
+  names, instead of its `function`; the writer follows it.
+  `Modality(path, function="sky_mask")` reads a `class_segmentation` index as
+  boolean sky masks. It is exclusive with `Modality.loader`.
+- The `sky_mask` loaders and writers of `real_drive_sim`, `vkitti2`,
+  `synscapes`, `muses` and `generic_dense_depth` read the sky definition from
+  the contract field `skyclass` (the sky label in the file's RGB view;
+  `[v, v, v]` for a single-channel label): first from the file's attributes,
+  then from the dataset head's meta. Their old hard-coded values are now
+  defaults, and they log the definition they use once.
+- A `null` `skyclass`, the default from euler-dataset-contract's next release,
+  declares no sky: the loaders fall back to a deprecated key or their default,
+  and `generic` / `generic_dense_depth` raise. `[0, 0, 0]`, the contract's old
+  default, is still taken literally, but a head that declares it logs a warning
+  naming it the old default, in place of the warning about differing from the
+  loader's default. Loaders without a default warn too.
+- Deprecate the per-loader sky keys `sky_class_id`, `sky_color`, `sky_class`
+  and `sky_mask` in favour of `skyclass`. They are still read when no
+  `skyclass` is given and raise a `DeprecationWarning`; when `skyclass` is
+  also given it wins, and the ignored key is logged as a warning.
+- Warn when the resolved sky differs from a loader's built-in default.
+- Reject values a loader cannot match instead of returning empty masks: a
+  non-grey `skyclass` for MUSES label PNGs (as for Synscapes and `generic`),
+  and anything but `[id, 0, 0]` for Real Drive Sim.
+- The dry run decodes segmentation heads with their loader module's
+  `sky_mask` (at least `SKY_SAMPLES` files), reports the sky fraction, and
+  warns when no sampled file has sky or one is more than 90% sky.
+- `real_drive_sim.sky_mask` compares the PNG's RGB value with `skyclass`
+  instead of only the red channel; for its own data, where sky is
+  `(29, 0, 0)`, the masks are unchanged.
+- Add `generic.sky_mask` for `.npy`/`.npz` class-id maps, which requires
+  `skyclass`.
+- The torch `sky_mask` loaders and writers of `real_drive_sim`, `vkitti2`,
+  `muses` and `generic_dense_depth` wrap the NumPy ones instead of
+  duplicating them.
+- `muses.sky_mask`'s declared default meta is `skyclass: [23, 23, 23]`; the
+  old `sky_class: [0, 0, 23]` matched no pixel of a `labelIds` PNG.
+- `MultiModalDataset` logs each modality's effective loader, noting when
+  `Modality.function` overrides the index.
+
+## 2.27.0 (unreleased)
+
+- Add a dry-run command: `euler-loading <path>` (also `python -m
+  euler_loading <path>`) walks the ds-crawler artifact sets at or below one
+  folder, resolves the loader each dataset head declares in
+  `addons.euler_loading`, decodes a sample of the indexed files with it, and
+  reports shapes, dtypes and value ranges. Directories and `.zip` archives
+  are treated alike, nothing is written, and the exit status is non-zero when
+  a modality fails to resolve or decode. `euler_loading.dry_run.dry_run()`
+  exposes the same report to Python.
+- `resolve_loader_module` and `resolve_writer_module` take a
+  `variant="gpu"|"cpu"` keyword, so the CPU modules can be resolved through
+  the same contract pathway as the torch ones. This is what the dry run's
+  `--cpu` uses to check archives on a torch-free install.
+
+## 2.26.0
+
+- Add `read_extrinsics` to the Synscapes loaders, building a 4x4 rigid
+  transform from the six `camera.extrinsic` scalars. `transform_direction`
+  and `camera_axes` select the pose, its inverse, and vehicle or optical
+  camera axes; the assumed `Rz(yaw) @ Ry(pitch) @ Rx(roll)` order is recorded
+  in the modality metadata because the dataset does not document it.
+- Add Synscapes writers for every modality, making the loader a
+  `DenseDepthCodec`. Depth writes a float32 EXR `Z` channel and needs a
+  filesystem path; the intrinsics and extrinsics writers merge into a single
+  `meta/<id>.json`, replacing it in one step and keeping its permissions,
+  symlink and unmodelled fields rather than truncating it in place.
+- `sky_mask` now honours `meta['sky_class_id']`, or the same key in per-file
+  attributes, instead of hardcoding Cityscapes ID 23, so a dataset that
+  relabels sky round-trips through `write_sky_mask`.
+- Accept the plural `camera.intrinsics` / `camera.extrinsics` spellings and
+  report missing camera fields by name instead of raising `KeyError`.
+
+## 2.24.0 (unreleased)
+
+Add source-backed capture, strict per-output writers, explicit NPY/PNG encoding, typed calibration views and replay. Consolidate pinhole geometry and correct legacy skew scaling.
+
+
+## 2.23.0 (source changes; not published)
+
+- Reject wrapped decoders during dataset export by checking actual built-in
+  function identity. Ordinary callable loading remains supported.
+- Apply boolean thresholds before Pillow dtype conversion, and permit exact
+  int32/int64 crop and identity-resize plans while refusing float32 resampling.
+- Add `SamplePreprocessor.export_descriptor`, `MultiModalDataset.export_transform_plan`,
+  `SerializableTransform`, and `resolve_transform_descriptor`, paired with contract 0.4.0.
+- Freeze profiles, qualified source/calibration bindings, field policies, operation
+  order, and installed CPU backend versions. Infer output profiles and virtual
+  calibration geometry without materialization claims.
+- Add explicit Torch/Pillow execution, corrected pinhole skew in the opt-in path,
+  validity-aware depth policy, source immutability, and worker validator registration.
+- Cover five-field round trips, projected geometry, ambiguous bindings, unsupported
+  semantics, backend choices, schemas, and spawned workers in synthetic tests.
+
+Legacy preprocessing and writer behavior remain unchanged. Execution receipts,
+materialized metadata propagation, producer wiring, and GT replay are later work.

{euler_loading-2.26.0 → euler_loading-2.29.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: euler-loading
-Version: 2.26.0
+Version: 2.29.0
 Summary: Multi-modal PyTorch dataloader using ds-crawler indices
 Project-URL: Homepage, https://github.com/d-rothen/euler-loading
 Project-URL: Repository, https://github.com/d-rothen/euler-loading
@@ -169,11 +169,44 @@ RGB contract looks like this:
 ```
 
 `loader` selects a built-in loader module and `function` selects the callable
-inside it. Automatic resolution uses the GPU variant; pass a loader explicitly
-when you want the CPU variant or a custom callable. See
+inside it. `Modality(path, function="sky_mask")` picks another callable from
+the same module, e.g. boolean sky masks from a `class_segmentation` index,
+with the sky taken from the head's `skyclass`. Automatic resolution uses the
+GPU variant; pass a loader explicitly when you want the CPU variant or a
+custom callable. See
 [Automatic loader resolution](docs/loaders.md#automatic-loader-resolution) for
 the full contract and writer rules.
 
+## Checking a dataset
+
+Before any of that runs, point the bundled command at a folder to see what
+euler-loading would make of it:
+
+```bash
+euler-loading /data/vkitti2
+```
+
+For every ds-crawler artifact set below that path it resolves the loader the
+dataset declares, decodes a sample of the indexed files with it, and prints
+the shape, dtype and value range that came back:
+
+```text
+vkitti_2.0.3_rgb  [ok]
+  path      /data/vkitti2/vkitti_2.0.3_rgb
+  contract  vkitti2_rgb, "Virtual KITTI 2 RGB", modality key: rgb
+  loader    vkitti2.rgb -> euler_loading.loaders.gpu.vkitti2.rgb
+  index     21260 files; available splits: train, val
+  decoded   1/1
+    Scene01/clone/frames/rgb/Camera_0/rgb_00000.jpg  ->  Tensor (3, 375, 1242) float32 in [0, 1]  (11.4 ms)
+```
+
+Nothing is written, directories and `.zip` archives are treated alike, and the
+exit status is non-zero if any modality failed to resolve or decode — so it
+also works as a CI step. `--all` decodes every file instead of a sample,
+`--cpu` checks the NumPy loaders on a torch-free install, and `--json` prints
+the same report for a machine. See
+[Dry-running loaders](docs/loaders.md#dry-running-loaders).
+
 ## What you get
 
 | | |
@@ -186,6 +219,7 @@ the full contract and writer rules.
 | **Loader resolution** | Loaders and writers resolve from the `dataset-head.json` `addons.euler_loading` contract, so datasets describe how to read themselves. |
 | **Writing back** | Resolved writers put inference outputs back in dataset-native formats, re-indexable with matching IDs. |
 | **Spatial preprocessing** | `SamplePreprocessor` resizes and crops consistently across images, depth, masks, ray maps *and* intrinsics. |
+| **Dry runs** | `euler-loading <folder>` resolves and exercises every loader a dataset declares, without writing anything. |
 
 ## Built-in loaders
 

{euler_loading-2.26.0 → euler_loading-2.29.0}/README.md

@@ -130,11 +130,44 @@ RGB contract looks like this:
 ```
 
 `loader` selects a built-in loader module and `function` selects the callable
-inside it. Automatic resolution uses the GPU variant; pass a loader explicitly
-when you want the CPU variant or a custom callable. See
+inside it. `Modality(path, function="sky_mask")` picks another callable from
+the same module, e.g. boolean sky masks from a `class_segmentation` index,
+with the sky taken from the head's `skyclass`. Automatic resolution uses the
+GPU variant; pass a loader explicitly when you want the CPU variant or a
+custom callable. See
 [Automatic loader resolution](docs/loaders.md#automatic-loader-resolution) for
 the full contract and writer rules.
 
+## Checking a dataset
+
+Before any of that runs, point the bundled command at a folder to see what
+euler-loading would make of it:
+
+```bash
+euler-loading /data/vkitti2
+```
+
+For every ds-crawler artifact set below that path it resolves the loader the
+dataset declares, decodes a sample of the indexed files with it, and prints
+the shape, dtype and value range that came back:
+
+```text
+vkitti_2.0.3_rgb  [ok]
+  path      /data/vkitti2/vkitti_2.0.3_rgb
+  contract  vkitti2_rgb, "Virtual KITTI 2 RGB", modality key: rgb
+  loader    vkitti2.rgb -> euler_loading.loaders.gpu.vkitti2.rgb
+  index     21260 files; available splits: train, val
+  decoded   1/1
+    Scene01/clone/frames/rgb/Camera_0/rgb_00000.jpg  ->  Tensor (3, 375, 1242) float32 in [0, 1]  (11.4 ms)
+```
+
+Nothing is written, directories and `.zip` archives are treated alike, and the
+exit status is non-zero if any modality failed to resolve or decode — so it
+also works as a CI step. `--all` decodes every file instead of a sample,
+`--cpu` checks the NumPy loaders on a torch-free install, and `--json` prints
+the same report for a machine. See
+[Dry-running loaders](docs/loaders.md#dry-running-loaders).
+
 ## What you get
 
 | | |
@@ -147,6 +180,7 @@ the full contract and writer rules.
 | **Loader resolution** | Loaders and writers resolve from the `dataset-head.json` `addons.euler_loading` contract, so datasets describe how to read themselves. |
 | **Writing back** | Resolved writers put inference outputs back in dataset-native formats, re-indexable with matching IDs. |
 | **Spatial preprocessing** | `SamplePreprocessor` resizes and crops consistently across images, depth, masks, ray maps *and* intrinsics. |
+| **Dry runs** | `euler-loading <folder>` resolves and exercises every loader a dataset declares, without writing anything. |
 
 ## Built-in loaders
 

{euler_loading-2.26.0 → euler_loading-2.29.0}/docs/README.md

@@ -3,7 +3,7 @@
 | Guide | Covers |
 |---|---|
 | [Dataset & modalities](dataset.md) | `Modality` and `MultiModalDataset` reference, the sample dict, hierarchical modalities, splits, scoped metadata, zip archives, layout-aware loading |
-| [Loaders & writers](loaders.md) | The loader contract, per-file attributes, automatic resolution, protocols, and the full built-in loader inventory |
+| [Loaders & writers](loaders.md) | The loader contract, per-file attributes, automatic resolution, the `euler-loading` dry run, protocols, and the full built-in loader inventory |
 | [Preprocessing & transforms](preprocessing.md) | Cross-modal transforms, `SamplePreprocessor`, field kinds, calibration-aware resize and crop |
 | [Writing outputs](writing.md) | Writing predictions back in dataset-native formats and re-indexing them |
 

{euler_loading-2.26.0 → euler_loading-2.29.0}/docs/loaders.md

@@ -7,6 +7,7 @@ that file means.
 - [The contract](#the-contract)
 - [Per-file attributes](#per-file-attributes)
 - [Automatic loader resolution](#automatic-loader-resolution)
+- [Dry-running loaders](#dry-running-loaders)
 - [Loader protocols](#loader-protocols)
 - [Built-in loaders](#built-in-loaders)
 
@@ -119,6 +120,14 @@ An explicit `Modality.loader` always takes precedence. Automatic resolution
 uses the GPU variant; import and pass a CPU loader explicitly when NumPy output
 is required.
 
+`Modality.function` keeps the head's `loader` module but picks another callable
+in it, and the writer follows it. `Modality(path, function="sky_mask")` reads a
+`class_segmentation` index as boolean sky masks, taking the sky definition from
+the head's `skyclass` (see [Sky masks](#sky-masks)). It cannot be combined with
+`Modality.loader`. Each modality's effective loader is logged when the dataset
+is built, e.g. `loaded with real_drive_sim.sky_mask (the index declares
+'class_segmentation')`.
+
 For compatibility, euler-loading can still read the legacy top-level
 `euler_loading` mapping from older indexes. New datasets should declare the
 addon in `dataset-head.json` as shown above.
@@ -129,6 +138,144 @@ Writers resolve from the same `addons.euler_loading` entry, in this order:
 2. for `function: "read_<suffix>"`, `write_<suffix>`,
 3. `write_<function>`.
 
+## Dry-running loaders
+
+Resolution only pays off if the loader a dataset names can really read its
+files. Point the bundled command at a path to find out, before a training run
+depends on it:
+
+```bash
+euler-loading /data/vkitti2
+python -m euler_loading /data/vkitti2   # same thing, no console script needed
+```
+
+It takes one path — a modality root, a `.zip` archive, or a folder holding
+several of them — and for every ds-crawler artifact set below it:
+
+1. loads the index, or a named split,
+2. resolves the loader its `addons.euler_loading` entry declares,
+3. decodes a sample of the indexed files with that loader,
+4. reports the shape, dtype and value range of what came back.
+
+Nothing is written and no dataset is constructed. A modality that cannot be
+read is reported, not raised, so one broken archive does not hide the rest.
+
+```text
+euler-loading dry-run: /data/vkitti2
+loader variant: gpu (torch tensors)
+
+vkitti_2.0.3_rgb  [ok]
+  path      /data/vkitti2/vkitti_2.0.3_rgb
+  contract  vkitti2_rgb, "Virtual KITTI 2 RGB", modality key: rgb
+  loader    vkitti2.rgb -> euler_loading.loaders.gpu.vkitti2.rgb
+  writer    write_rgb
+  index     21260 files; available splits: train, val
+  decoded   1/1
+    Scene01/clone/frames/rgb/Camera_0/rgb_00000.jpg  ->  Tensor (3, 375, 1242) float32 in [0, 1]  (11.4 ms)
+
+vkitti_2.0.3_textgt  [ok]
+  path      /data/vkitti2/vkitti_2.0.3_textgt
+  contract  vkitti2_intrinsics, "Virtual KITTI 2 intrinsics", modality key: camera_intrinsics
+  loader    vkitti2.read_intrinsics -> euler_loading.loaders.gpu.vkitti2.read_intrinsics (hierarchical)
+  writer    write_intrinsics
+  index     50 files
+  decoded   1/1
+    Scene01/clone/intrinsic.txt  ->  Tensor (3, 3) float32 in [0, 725]  (0.6 ms)
+
+2 modalities checked: 2 ok, 0 failed; 2/2 files decoded
+```
+
+The exit status is 0 when every modality resolved a loader and decoded every
+file it was asked to, and 1 otherwise, so the command works as a CI step or a
+job prologue.
+
+| Flag | Effect |
+|---|---|
+| `-n, --samples N` | Files to decode per modality, spread evenly across the index (default 1). `0` resolves loaders without decoding anything. |
+| `--all` | Decode every indexed file. Slow, but checks the whole archive. |
+| `--split NAME` | Dry-run a named ds-crawler split instead of the canonical index. |
+| `--scope SCOPE` | Read `.ds_crawler/SCOPE/` only, instead of every artifact set a root carries. |
+| `--cpu` | Resolve the NumPy loaders instead of the torch ones. Works on an install without PyTorch. |
+| `--max-depth N` | Directory levels below the path to search for dataset roots (default 3). |
+| `--json` | Print the report as JSON instead of text. |
+| `-v, --verbose` | Log debug output, including a traceback for every failed load. |
+
+The path takes the same inline selectors as `Modality`, so
+`euler-loading /data/muses.zip:train#scope=rgb` checks one split of one scope.
+
+Discovery stops at the first dataset root down each branch: a directory
+carrying its own `.ds_crawler/` is checked, never descended into, so a
+dataset's own scene directories are never walked. A root with several
+metadata scopes contributes one entry per scope, labelled with the selector
+that addresses it.
+
+### Reading the report
+
+| Line | What it is telling you |
+|---|---|
+| `error loader: ... no 'addons.euler_loading' entry` | the dataset head declares no loader, so only an explicit `Modality(..., loader=...)` can read it |
+| `loader vkitti2.rgb (unresolved)` plus an `error` | the contract names a loader that is unknown, misspelled, or needs a dependency this install lacks — `--cpu` resolves the torch-free variant |
+| `decoded 0/2` with per-file `FAILED` | the files are indexed and present, but the declared loader cannot decode them |
+| `(hierarchical)` after the loader | the declared function reads per-scene data; pass that modality as `hierarchical_modalities`, not `modalities` |
+| `error index: ...` | ds-crawler could not produce an index — a missing split, an unreadable archive, or a dataset head the contract rejects |
+| `sky skyclass [29, 0, 0]: sky in 8/8 files, ...` | a segmentation head's files, decoded with its loader module's `sky_mask` (at least `SKY_SAMPLES` = 8 files) |
+| a warning that `sky_mask` finds no sky, or marks a file as more than 90% sky | the head's `skyclass` probably names the wrong label; an `error sky_mask: ...` means the loader rejects it outright |
+| a warning about shared file IDs | the modalities below the path have no file ID in common, so they cannot be intersected into one `MultiModalDataset` |
+
+The same report is available as an API, which is what the command is a thin
+shell around:
+
+```python
+from euler_loading.dry_run import dry_run
+
+report = dry_run("/data/vkitti2", samples=3)
+if not report.ok:
+    for modality in report.modalities:
+        print(modality.label, modality.errors)
+
+report.to_dict()   # the same structure --json prints
+```
+
+## Sky masks
+
+The dataset contract requires every `semantic_segmentation` head to declare
+`modality.meta.skyclass`: the sky label as it appears in the segmentation file
+viewed as RGB. That is the sky colour of a colour-coded map, and `[v, v, v]`
+for a single-channel label `v`:
+
+| Dataset | Encoding | `skyclass` |
+|---|---|---|
+| Virtual KITTI 2 | RGB class colours | `[90, 200, 255]` |
+| Real Drive Sim | class ID in the red channel of an RGBA PNG | `[29, 0, 0]` |
+| Synscapes | single-channel Cityscapes label IDs | `[23, 23, 23]` |
+| MUSES `labelIds` | single-channel Cityscapes label IDs | `[23, 23, 23]` |
+| Cityscapes train IDs, e.g. `generic` `.npy` maps | single-channel | `[10, 10, 10]` |
+
+`skyclass: null`, the contract's default, declares no sky. `[0, 0, 0]` is taken
+literally, as label 0 or black; contract releases before `null` used it as the
+default, so heads written then can carry it without anyone having declared it.
+A head that declares `[0, 0, 0]` is therefore logged as a warning.
+
+Every `sky_mask` loader (and the matching `write_sky_mask`) resolves the sky in
+this order, skipping a `null` `skyclass`, and logs the definition it uses once:
+
+1. `skyclass` in the file's ds-crawler attributes,
+2. `skyclass` in the dataset head's meta,
+3. a deprecated per-loader key — `sky_class_id` (Synscapes, Real Drive Sim,
+   MUSES), `sky_color` (VKITTI2), `sky_class` / `sky_mask` (MUSES) or
+   `sky_mask` (generic dense depth) — which raises a `DeprecationWarning`,
+4. the loader's built-in default; `generic.sky_mask` and
+   `generic_dense_depth.sky_mask` have none and require `skyclass`.
+
+A deprecated key that `skyclass` overrides is logged as a warning, saying
+whether it named the same sky. So is a resolved sky that differs from the
+loader's default. Values a loader cannot match raise instead of silently
+finding no sky: a non-grey `skyclass` for single-channel maps (Synscapes,
+MUSES label PNGs, `generic`), and anything but `[id, 0, 0]` for Real Drive Sim,
+which stores class IDs in the red channel. The [dry run](#dry-running-loaders)
+decodes segmentation heads with `sky_mask` and warns about suspicious sky
+fractions.
+
 ## Loader protocols
 
 `DenseDepthLoader` is a `runtime_checkable` Protocol defining the loader
@@ -179,7 +326,7 @@ annotations on the loader functions themselves.
 | `class_segmentation` | CHW / HWC | int64 | RGB-encoded class mask |
 | `instance_segmentation` | CHW / HWC | int64 | RGB-encoded instance mask |
 | `scene_flow` | CHW / HWC | float32 | Optical/scene flow, normalised to `[0, 1]` |
-| `sky_mask` | 1HW / HW | bool | Sky colour `[90, 200, 255]` in the class mask |
+| `sky_mask` | 1HW / HW | bool | Head `skyclass` colour in the class mask, default `[90, 200, 255]` |
 | `read_intrinsics` | 3×3 | float32 | From a `.txt` file · *hierarchical* |
 | `read_extrinsics` | N×1 | float32 | From a `.txt` file · *hierarchical* |
 
@@ -193,14 +340,14 @@ Writers exist for every modality above.
 | `depth` | 1HW / HW | float32 | `img/depth/<id>.exr`, `Z` channel, planar depth in metres |
 | `class_segmentation` | 1HW / HW | int64 | `img/class/<id>.png`, original Cityscapes label IDs |
 | `instance_segmentation` | 1HW / HW | int64 | `img/instance/<id>.png`, decoded as `R + 256 * G + 65536 * B` |
-| `sky_mask` | 1HW / HW | bool | Sky label in `img/class/<id>.png`; ID `23` by default, or `meta['sky_class_id']` |
+| `sky_mask` | 1HW / HW | bool | Sky label in `img/class/<id>.png`: the grey `skyclass`, ID `23` by default |
 | `read_intrinsics` | 3×3 | float32 | `camera.intrinsic` from `meta/<id>.json`: `fx`, `fy`, `u0`, `v0` |
 | `read_extrinsics` | 4×4 | float32 | `camera.extrinsic` from `meta/<id>.json`: `x`, `y`, `z`, `pitch`, `roll`, `yaw` |
 
 Writers exist for every modality above. `write_depth` needs a filesystem path
 because OpenEXR cannot write to a stream; dataset writers hand it a temporary
 file automatically, so zip outputs work unchanged. `write_sky_mask` emits a
-class image labelling sky with `meta['sky_class_id']`, default 23, and
+class image labelling sky with the label `sky_mask` reads, and
 `write_intrinsics` and `write_extrinsics` merge into one `meta/<id>.json`
 rather than overwriting each other or the file's `scene` and `instance`
 blocks.
@@ -287,7 +434,7 @@ same resolution. Loaders preserve the stored resolution.
 | `reference_rgb` | CHW / HWC | float32 | Clear-weather reference frame, `[0, 1]` |
 | `semantic_segmentation` | 1HW / HW | int64 | Cityscapes `labelIds` or `labelTrainIds` |
 | `semantic_segmentation_color` | CHW / HWC | uint8 | Cityscapes `labelColor` PNG |
-| `sky_mask` | 1HW / HW | bool | `trainId=10`, `labelId=23`, or RGB `meta["sky_class"]` |
+| `sky_mask` | 1HW / HW | bool | `skyclass` (grey for label PNGs, a colour for RGB class images); default `trainId=10` / `labelId=23` |
 | `panoptic_segmentation` | 1HW / HW | int64 | COCO-style RGB panoptic PNG → segment IDs |
 | `lidar_point_cloud` | (N, 6) | float64 | `.bin`: `x, y, z, intensity, ring, timestamp` |
 | `sparse_depth` | (N, 6) | float64 | Same points, for sparse-depth workflows |
@@ -303,7 +450,7 @@ same resolution. Loaders preserve the stored resolution.
 | `rgb` | CHW / HWC | float32 | PNG, `[0, 1]` |
 | `depth` | 1HW / HW | float32 | `.npz`, metres |
 | `class_segmentation` | 1HW / HW | int64 | Class IDs from the red channel of an RGBA PNG |
-| `sky_mask` | 1HW / HW | bool | Class ID `29` |
+| `sky_mask` | 1HW / HW | bool | Head `skyclass` colour, default `[29, 0, 0]` (class ID 29) |
 | `calibration` | dict | — | `{sensor: {"K": (3,3), "T": (4,4), "distortion": (8,)}}` · *hierarchical* |
 | `all_intrinsics` | dict | — | `{sensor: (3, 3)}` · *hierarchical* |
 | `read_intrinsics` | 3×3 | float32 | Defaults to `CS_FRONT` · *hierarchical* |
@@ -330,7 +477,7 @@ Useful for datasets without a dedicated module.
 |---|---|---|---|
 | `rgb` | CHW / HWC | float32 | Image (`.png`, `.jpg`, `.bmp`, `.tif`) or NumPy (`.npy`, `.npz`), `[0, 1]` |
 | `depth` | 1HW / HW | float32 | Image or NumPy, returned as-is; honours `attributes["scale_to_meters_override"]` |
-| `sky_mask` | 1HW / HW | bool | Compares pixels against `meta["sky_mask"]` (`[R, G, B]`) — requires `meta` |
+| `sky_mask` | 1HW / HW | bool | Compares pixels against the `skyclass` colour — required |
 | `read_intrinsics` | 3×3 | float32 | Returns `meta["intrinsics"]`, ignores the path — requires `meta` · *hierarchical* |
 
 ### Generic NumPy modalities — `generic`
@@ -347,8 +494,9 @@ assumptions beyond shape and dtype.
 | `atmospheric_light` | CHW / HWC | float32 |
 | `scattering_coefficient` | HW | float32 |
 | `semantic_segmentation` | HW | uint8 |
+| `sky_mask` | HW | bool (head `skyclass` label, required) |
 | `instance_segmentation` | HW | uint16 |
 | `intrinsics` | 3×3 | float32 |
 | `sh_coeffs` | (N, C) | float32 |
 
-Writers exist for every function in this module.
+Writers exist for every function in this module except `sky_mask`.

euler_loading-2.29.0/euler_loading/__main__.py

@@ -0,0 +1,10 @@
+"""``python -m euler_loading`` — dry-run the loaders a dataset declares."""
+
+from __future__ import annotations
+
+import sys
+
+from .cli import main
+
+if __name__ == "__main__":
+    sys.exit(main())

{euler_loading-2.26.0 → euler_loading-2.29.0}/euler_loading/_ds_crawler_utils.py

@@ -1,5 +1,7 @@
 from __future__ import annotations
 
+import io
+import zipfile
 from collections.abc import Mapping
 from pathlib import Path
 from typing import Any
@@ -22,6 +24,22 @@ from ds_crawler.zip_utils import (
 )
 
 
+def read_zip_member(
+    archive: zipfile.ZipFile,
+    prefix: str,
+    relative_path: str,
+) -> io.BytesIO:
+    """Read one member of a zip-backed modality into an in-memory buffer.
+
+    The buffer carries the modality-relative path as its ``name`` so loaders
+    that branch on the file extension behave the same as for a filesystem
+    path.
+    """
+    buffer = io.BytesIO(archive.read(prefix + relative_path))
+    buffer.name = relative_path
+    return buffer
+
+
 def as_non_empty_str(value: Any) -> str | None:
     if value is None:
         return None