euler-preprocess 3.19.0__tar.gz → 3.20.0__tar.gz

{euler_preprocess-3.19.0 → euler_preprocess-3.20.0}/CHANGELOG.md

@@ -2,6 +2,35 @@
 
 ## Unreleased
 
+- `crop` crops label rasters in the stored file: semantic segmentation (the
+  ground truth that also holds the sky), instance and panoptic segmentation,
+  and sky masks.
+  - PNG, TIFF and BMP keep their mode, palette, bit depth and every channel;
+    `.npy`/`.npz` keep their dtype and other arrays.
+  - The result must decode exactly as the crop of the source labels.
+  - So no class is lost to a writer that re-encodes only what one loader
+    function returns: Real Drive Sim's non-red channels, or every class but
+    the sky for a dataset read through `sky_mask`. Label modalities no longer
+    need a writer.
+  - Files that cannot be kept (JPEG, 16-bit colour PNGs) are re-encoded with
+    the writer, with a warning.
+  - Labels can only be resized with `nearest`; another `resize.interpolation`
+    for them is an error.
+
+- `crop` supports MUSES (with euler-loading 2.35.0's MUSES writers):
+  - `calib.json` holds the frame camera's `K` and every extrinsic. The
+    `intrinsics` modality patches it once, and every other modality indexing
+    the same file, or a byte-identical copy (`camera_extrinsics`,
+    `lidar_to_gnss`), is written with the patched bytes, so the bundle holds
+    one `K`. What each of them reads from the patched file must stay as it was;
+    two intrinsics modalities patching one file differently are refused.
+  - GNSS records (`gnss`) and `meta.json` timing (`sensor_timing`) are copied
+    unchanged by default.
+  - `configs/crop_muses_hood_config.json` cuts the engine hood off the
+    1920×1080 frame camera images (bottom 160 rows, 1920×920 stay), with
+    `configs/crop_muses_dataset_config.json` as the dataset config.
+- Require euler-loading 2.35.0 for the MUSES writers.
+
 - Add `euler-preprocess lidar-motion`, which compensates scanning-lidar points for
   the vehicle's motion during the sweep. It follows the MUSES SDK, moving each
   point to the frame camera's mid-exposure using the `gnss`, `sensor_timing`

{euler_preprocess-3.19.0 → euler_preprocess-3.20.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-preprocess
-Version: 3.19.0
+Version: 3.20.0
 Summary: Physics-based preprocessing (fog, etc.) for RGB+depth datasets
 Author-email: Daniel Rothenpieler <rothenpielerdaniel@gmail.com>
 License-Expression: MIT
@@ -9,7 +9,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: numpy
 Requires-Dist: Pillow
-Requires-Dist: euler-loading>=2.33.0
+Requires-Dist: euler-loading>=2.35.0
 Requires-Dist: ds-crawler>=2.14.0
 Requires-Dist: euler-dataset-contract>=0.5.0
 Provides-Extra: gpu
@@ -748,7 +748,8 @@ like the original:
 
 | Modality kind | Crop |
 |---|---|
-| Rasters: RGB, dense depth, masks | Sliced; no resampling, so values are unchanged |
+| Rasters: RGB, dense depth | Sliced; no resampling, so values are unchanged |
+| Label rasters: semantic segmentation (the ground truth that also holds the sky), instance and panoptic segmentation, sky masks | Cropped in the stored file, so every class, channel, palette and bit depth is kept; resized only by nearest |
 | Pinhole intrinsics `K` (or `P` for rectified images) | `cx -= left`, `cy -= top`; `fx`, `fy`, skew unchanged |
 | Lidar point clouds | Copied byte for byte, or pruned to the points that project into the crop |
 | Extrinsics | Copied byte for byte |
@@ -808,7 +809,7 @@ The crop config (`configs/crop_config.json`) describes the box relative to the
 | `point_clouds` | `keep` (default): copy every point, consumers clip to the image. `prune_to_crop`: keep only the points with `z > 0` whose pixel `rint(K'·T·X)` lies in the output image, as consumers such as dmd3c-euler project them |
 | `projection` | For `prune_to_crop`: the `intrinsics` and `extrinsics` modalities to project with, when the dataset has more than one candidate |
 | `verify` | `all` (default), `none`, or a sample count: reopen the outputs and check every file against the crop (rasters and pruned clouds as encoded, intrinsics to 1e-3 px, copies byte-identical) |
-| `roles` | Override a modality's kind with `raster` or `copy`, for loaders whose annotation the crop does not know |
+| `roles` | Override a modality's kind with `raster` or `copy`, for loaders whose annotation the crop does not know, or to copy an intrinsics file of another camera (e.g. MUSES `sensor: event`) instead of cropping it |
 
 Pixel units reproduce the official boxes exactly, e.g. `RGB2Gatedv2` is
 `{"remove": {"top": 202, "bottom": 54, "left": 280, "right": 360}, "unit": "pixel"}`.
@@ -816,7 +817,34 @@ Pixel units reproduce the official boxes exactly, e.g. `RGB2Gatedv2` is
 SeeingThroughFog rectification leaves on `_lut` frames (top 34, bottom 46,
 left 16 px), keeping a fully valid 1904×944 image.
 
-Rasters are re-encoded with the dataset's euler-loading writer, so a raster
+MUSES keeps the frame camera's `K` and every extrinsic in one `calib.json`,
+indexed as `intrinsics`, `camera_extrinsics` (and `lidar_to_gnss`). The crop
+patches `intrinsics.rgb.K` once and writes that file for every modality that
+indexes it, also when it is indexed as separate byte-identical copies, so the
+bundle holds one `K`. `D`, the event camera and all extrinsics stay as they
+are. `configs/crop_muses_hood_config.json` cuts the engine hood off the
+1920×1080 frame camera images: the bottom 160 rows, leaving 1920×920. Rows come
+off the bottom only, so `K` and every `calib.json` stay byte-identical. Run
+`lidar-motion` first and crop its output (`configs/crop_muses_dataset_config.json`).
+GNSS records and `meta.json` sensor timing are copied unchanged, like the
+extrinsics. MUSES label images (`gt_semantic`, `gt_panoptic`) are cropped in place like
+any label raster.
+
+Label rasters (loader types `semantic_segmentation`, `semantic_segmentation_color`,
+`instance_segmentation`, `panoptic_segmentation`, `sky_mask`) are cropped in the
+file itself: PNG, TIFF and BMP through Pillow in their own mode and palette,
+`.npy`/`.npz` through NumPy. A semantic segmentation is the ground truth for
+every class including the sky, and consumers read it through different
+functions (`class_segmentation`, or `sky_mask` for sky-depth). Re-encoding the
+decoded labels would drop what one function does not return: Real Drive Sim's
+non-red channels, an indexed PNG's palette, or every class but the sky when a
+dataset only has a `sky_mask` function. The in-place crop must decode exactly
+as the crop of the source labels. Files it cannot keep (JPEG, 16-bit colour
+PNGs, ambiguous array layouts) fall back to the writer with a warning. Labels
+are never interpolated: a `resize.interpolation` other than `nearest` for them
+is an error.
+
+Other rasters are re-encoded with the dataset's euler-loading writer, so a raster
 modality needs one (SeeingThroughFog's raw `rccb` TIFFs have none). Each
 encoded file is decoded again right away and must give back the cropped values:
 exactly, or for an interpolated resize within half an 8-bit step. A resize

euler_preprocess-3.19.0/euler_preprocess.egg-info/PKG-INFO → euler_preprocess-3.20.0/README.md

@@ -1,23 +1,3 @@
-Metadata-Version: 2.4
-Name: euler-preprocess
-Version: 3.19.0
-Summary: Physics-based preprocessing (fog, etc.) for RGB+depth datasets
-Author-email: Daniel Rothenpieler <rothenpielerdaniel@gmail.com>
-License-Expression: MIT
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: numpy
-Requires-Dist: Pillow
-Requires-Dist: euler-loading>=2.33.0
-Requires-Dist: ds-crawler>=2.14.0
-Requires-Dist: euler-dataset-contract>=0.5.0
-Provides-Extra: gpu
-Requires-Dist: torch; extra == "gpu"
-Provides-Extra: progress
-Requires-Dist: tqdm; extra == "progress"
-Dynamic: license-file
-
 <!-- euler header — shared across the euler packages.
      Per package, change only: the <h1>, the tagline, and the badge URLs. -->
 <p align="center">
@@ -748,7 +728,8 @@ like the original:
 
 | Modality kind | Crop |
 |---|---|
-| Rasters: RGB, dense depth, masks | Sliced; no resampling, so values are unchanged |
+| Rasters: RGB, dense depth | Sliced; no resampling, so values are unchanged |
+| Label rasters: semantic segmentation (the ground truth that also holds the sky), instance and panoptic segmentation, sky masks | Cropped in the stored file, so every class, channel, palette and bit depth is kept; resized only by nearest |
 | Pinhole intrinsics `K` (or `P` for rectified images) | `cx -= left`, `cy -= top`; `fx`, `fy`, skew unchanged |
 | Lidar point clouds | Copied byte for byte, or pruned to the points that project into the crop |
 | Extrinsics | Copied byte for byte |
@@ -808,7 +789,7 @@ The crop config (`configs/crop_config.json`) describes the box relative to the
 | `point_clouds` | `keep` (default): copy every point, consumers clip to the image. `prune_to_crop`: keep only the points with `z > 0` whose pixel `rint(K'·T·X)` lies in the output image, as consumers such as dmd3c-euler project them |
 | `projection` | For `prune_to_crop`: the `intrinsics` and `extrinsics` modalities to project with, when the dataset has more than one candidate |
 | `verify` | `all` (default), `none`, or a sample count: reopen the outputs and check every file against the crop (rasters and pruned clouds as encoded, intrinsics to 1e-3 px, copies byte-identical) |
-| `roles` | Override a modality's kind with `raster` or `copy`, for loaders whose annotation the crop does not know |
+| `roles` | Override a modality's kind with `raster` or `copy`, for loaders whose annotation the crop does not know, or to copy an intrinsics file of another camera (e.g. MUSES `sensor: event`) instead of cropping it |
 
 Pixel units reproduce the official boxes exactly, e.g. `RGB2Gatedv2` is
 `{"remove": {"top": 202, "bottom": 54, "left": 280, "right": 360}, "unit": "pixel"}`.
@@ -816,7 +797,34 @@ Pixel units reproduce the official boxes exactly, e.g. `RGB2Gatedv2` is
 SeeingThroughFog rectification leaves on `_lut` frames (top 34, bottom 46,
 left 16 px), keeping a fully valid 1904×944 image.
 
-Rasters are re-encoded with the dataset's euler-loading writer, so a raster
+MUSES keeps the frame camera's `K` and every extrinsic in one `calib.json`,
+indexed as `intrinsics`, `camera_extrinsics` (and `lidar_to_gnss`). The crop
+patches `intrinsics.rgb.K` once and writes that file for every modality that
+indexes it, also when it is indexed as separate byte-identical copies, so the
+bundle holds one `K`. `D`, the event camera and all extrinsics stay as they
+are. `configs/crop_muses_hood_config.json` cuts the engine hood off the
+1920×1080 frame camera images: the bottom 160 rows, leaving 1920×920. Rows come
+off the bottom only, so `K` and every `calib.json` stay byte-identical. Run
+`lidar-motion` first and crop its output (`configs/crop_muses_dataset_config.json`).
+GNSS records and `meta.json` sensor timing are copied unchanged, like the
+extrinsics. MUSES label images (`gt_semantic`, `gt_panoptic`) are cropped in place like
+any label raster.
+
+Label rasters (loader types `semantic_segmentation`, `semantic_segmentation_color`,
+`instance_segmentation`, `panoptic_segmentation`, `sky_mask`) are cropped in the
+file itself: PNG, TIFF and BMP through Pillow in their own mode and palette,
+`.npy`/`.npz` through NumPy. A semantic segmentation is the ground truth for
+every class including the sky, and consumers read it through different
+functions (`class_segmentation`, or `sky_mask` for sky-depth). Re-encoding the
+decoded labels would drop what one function does not return: Real Drive Sim's
+non-red channels, an indexed PNG's palette, or every class but the sky when a
+dataset only has a `sky_mask` function. The in-place crop must decode exactly
+as the crop of the source labels. Files it cannot keep (JPEG, 16-bit colour
+PNGs, ambiguous array layouts) fall back to the writer with a warning. Labels
+are never interpolated: a `resize.interpolation` other than `nearest` for them
+is an error.
+
+Other rasters are re-encoded with the dataset's euler-loading writer, so a raster
 modality needs one (SeeingThroughFog's raw `rccb` TIFFs have none). Each
 encoded file is decoded again right away and must give back the cropped values:
 exactly, or for an interpolated resize within half an 8-bit step. A resize

{euler_preprocess-3.19.0 → euler_preprocess-3.20.0}/docs/plans/coherent-cropping.md

@@ -3,7 +3,8 @@
 Status: **phases 1 and 3 implemented** (2026-11-07) as `euler-preprocess crop`
 (plus `crop-box`), for **SeeingThroughFog** (euler-loading's `princeton_dense`
 loader) with `rgb`, `sparse_depth`, `intrinsics` and `camera_extrinsics`; see §5
-for results. Phase 2 (other datasets) is open. The euler-loading side shipped in
+for results. Phase 2 (other datasets): **MUSES implemented** (2026-11-08, §5,
+including the engine-hood crop); the rest is open. The euler-loading side shipped in
 2.31.0 (STF fixes) and 2.32.0 (MUSES inputs); phase 3 uses euler-loading 2.33.0
 and ds-crawler 2.14.0 (both released), which euler-preprocess now requires.
 euler-view launches it with the `euler-preprocess-crop` addon. Written 2026-11-06 against
@@ -244,10 +245,11 @@ modality name, which is what causes the hazard in §1.
 
 | Annotated shape | Role | Operation |
 |---|---|---|
-| `HW`, `1HW`, `HWC`, `CHW`, `3HW` | raster | Slice with the box; must match the reference H×W, else error |
+| `HW`, `1HW`, `HWC`, `CHW`, `3HW` | raster | Slice with the box; must match the reference H×W, else error. Label types (`semantic_segmentation`, `semantic_segmentation_color`, `instance_segmentation`, `panoptic_segmentation`, `sky_mask`) are cropped in the stored file (2026-11-08) |
 | `NC` | point set | Byte copy (`prune_to_crop` later) |
 | `3x3` with type `intrinsics` | intrinsics | `K' = crop_intrinsics(K, top, left)` |
 | `4x4`, `Nx1` | extrinsics | Byte copy (patched copy where the file also holds K) |
+| `dict` with type `gnss` / `sensor_timing` | copy | Byte copy (MUSES side data) |
 | `dict` (Real Drive Sim `calibration`) | calibration map | Phase 2 |
 | anything else | — | Error; override with `"roles": {"<name>": "raster" \| "copy"}` |
 
@@ -372,7 +374,7 @@ euler-preprocess:
 
 | Dataset | Missing |
 |---|---|
-| `muses` | `write_rgb`; merge `write_intrinsics` for `calib.json` `intrinsics[sensor].K`; extrinsics output = patched `calib.json`. The lidar must be ego-motion compensated first (see below). |
+| `muses` | Done (2026-11-08, see "MUSES crop" below). |
 | `real_drive_sim` | Merge `write_intrinsics` for the calibration JSON (`CS_FRONT` fx/fy/cx/cy; `fov` is the horizontal FOV for 2048 px, so recompute or drop it); extrinsics output = patched file; RGBA sources become RGB PNGs |
 | `vkitti2` | `write_intrinsics` must keep every frame/camera row (it writes one today); `read_extrinsics` header bug (§7) |
 | `synscapes` | Nothing new (merge writers exist; pass `resx`/`resy`); extrinsics output = patched meta JSON |
@@ -381,7 +383,18 @@ euler-preprocess:
 
 Also in phase 2: a format-level raster crop (PNG/TIFF via Pillow, NPY/NPZ via
 NumPy, preserving mode, dtype and keys) for formats whose writers are missing or
-lossy.
+lossy. **Done for label rasters** (2026-11-08,
+`euler_preprocess/crop/label_files.py`), since a semantic segmentation is the
+ground truth for every class, the sky included:
+- The stored pixels are sliced, and for a resize the source pixels are picked
+  with euler-loading's own nearest indices, in the same mode, palette and
+  format.
+- The result must decode exactly as the crop of the decoded source.
+- Unsupported files fall back to the writer.
+- Labels are never interpolated.
+
+euler-view's crop addon offers `semantic_segmentation` instead of `sky_mask`,
+which is only a model output. Depth PNGs (§7.3) could use the same path later.
 
 **MUSES lidar needs motion compensation.** The sensor is not a spinning lidar.
 It scans a ~124° × 25° forward window top to bottom in 84.5 ms: point
@@ -436,6 +449,76 @@ loaders.
 The side data stays with the local dataset: MUSES license §2.3 forbids
 distributing it.
 
+### MUSES crop — implemented 2026-11-08
+
+euler-loading (unreleased, for 2.35.0): MUSES writers for every image and
+calibration modality.
+- `write_rgb` / `write_reference_rgb`: 8-bit PNG, a bit-exact round trip on
+  real frames.
+- `write_semantic_segmentation`, `write_semantic_segmentation_color`,
+  `write_panoptic_segmentation` and `write_sky_mask`.
+- `write_intrinsics` updates `intrinsics[<sensor>].K` in `calib.json` like
+  `princeton_dense.write_intrinsics`: only a crop or rescale of the stored `K`,
+  applied to its float64 values. It keeps `D`, the event camera and all
+  extrinsics, and does not rewrite an unchanged matrix. The crop/rescale check
+  is now shared (`_writer_utils.pixel_grid_transform`).
+
+euler-preprocess:
+- A copied file whose source bytes equal a file an intrinsics modality
+  patched is written with the patched bytes. This covers MUSES `calib.json`
+  as `camera_extrinsics` / `lidar_to_gnss`, whether it is indexed in place
+  (metadata scopes) or as separate copies, so the bundle carries one `K`.
+- Each such copy is decoded again and must read as before.
+- Two intrinsics modalities patching one file differently are refused (copy
+  the event camera's view with `roles`).
+- Intrinsics are written first in each pass.
+- `dict` loaders of type `gnss` and `sensor_timing` are copied by default.
+- Configs: `configs/crop_muses_hood_config.json` and
+  `crop_muses_dataset_config.json`; the euler-view crop addon has a
+  `muses-hood` preset.
+
+**Engine hood.** Measured on the five sample frames (all REC0176, test/fog/day):
+- **Method.** The hood is rigid with the camera, so its silhouette is the
+  vertical edge that is strong in every frame. Take the per-column maximum of
+  the across-frame minimum of the blurred vertical luma gradient.
+- **Shape.** The silhouette is a dome over columns 216–1619. At its bottom
+  it reaches rows 1061 (column 250) and 1066 (column 1600). Its apex is at
+  **row 931–932, near column 915**.
+- **Consistency.** All five frames agree to ±1 px. Luma at the apex steps
+  from road (44–53) to hood (65–70) between rows 930 and 933.
+- **Crop.** Cutting the bottom 160 rows (1920×920 stay) leaves an 11-row
+  margin. The cut is free in lidar terms: the MUSES lidar's lowest return
+  projects to row 811–820 in the three frames with lidar, so no point lands in
+  rows 920–1079. Pruning drops only points outside the camera's horizontal
+  field of view (66.7 % kept).
+- **Divisibility.** For a /32 height, cut 184 rows (1920×896).
+- **Open.** These are frames of one recording out of 136. The camera is
+  calibrated once (`calib.json`), but a remount between recordings could
+  shift the hood by a few rows. Check frames from a few other recordings
+  (other days, night) before relying on the margin.
+
+Real data (three frames with motion-compensated lidar, the real `calib.json`):
+- the CLI crop gives a bit-exact RGB slice;
+- `calib.json` for `intrinsics` and `camera_extrinsics` is byte-identical to
+  the source, since K is unchanged;
+- verification passes in both point-cloud modes.
+
+Real data, general crop (top 40, bottom 160, left 24, right 8, then 0.5x):
+`K' = S·C·K` to 1e-9, with both `calib.json` outputs identical, `D`, the event
+camera and the extrinsics kept, and verification passing.
+
+Tests: euler-loading 1038 passed (15 new writer tests); euler-preprocess 298
+passed. The MUSES-format fixture covers:
+- the hood crop;
+- a general crop patching `calib.json` for three modalities;
+- pruning;
+- the event-camera conflict;
+- a scoped zip like the production `/data/muses.zip`;
+- the configs.
+
+The contract's loader evidence was refreshed (only source hashes and line
+numbers changed), and its ecosystem check passes.
+
 ### Phase 3 — implemented 2026-11-07
 
 - `point_clouds: "prune_to_crop"`: keeps the points with `z > 0` whose pixel

{euler_preprocess-3.19.0 → euler_preprocess-3.20.0}/euler_preprocess/common/output.py

@@ -569,46 +569,41 @@ class SourceBackedOutputBackend:
         target_path.write_bytes(data)
         return target_path
 
-    def update_source(
+    def updated_bytes(
         self,
-        full_id: str,
         source_entry: Mapping[str, Any],
         data: bytes,
         value: Any,
         meta: dict[str, Any] | None,
-        *,
-        attributes: dict[str, Any] | None = None,
-    ) -> Path:
-        """Store a copy of the source file, then let the writer update it to *value*.
+    ) -> bytes:
+        """Return *data*, the source file's bytes, after the writer has updated them to *value*.
 
         For writers that merge into an existing file (e.g. camera-info
         ``write_intrinsics``), so every field the new value does not describe
-        is kept. The writer always receives a real file, also for zip outputs.
+        is kept. The writer works on a temporary copy named like the source.
         """
         if self.modality_writer is None:
             raise ValueError(f"No writer is available for modality '{self.source_modality}'")
         basename = Path(str(source_entry["path"])).name
-        entry = dict(source_entry)
-        merged = self._entry_attributes(source_entry, attributes)
-        if isinstance(self.dataset_writer, ZipDatasetWriter):
-            with tempfile.TemporaryDirectory() as tmpdir:
-                temp_path = Path(tmpdir) / basename
-                temp_path.write_bytes(data)
-                self.modality_writer(str(temp_path), value, meta)
-                self.dataset_writer.write(
-                    full_id,
-                    basename,
-                    temp_path.read_bytes(),
-                    source_entry=entry,
-                    attributes=merged,
-                )
-            return self._stored_path(full_id, basename)
-        target_path = self.dataset_writer.get_path(
-            full_id, basename, source_entry=entry, attributes=merged
-        )
-        target_path.write_bytes(data)
-        self.modality_writer(str(target_path), value, meta)
-        return target_path
+        with tempfile.TemporaryDirectory() as tmpdir:
+            temp_path = Path(tmpdir) / basename
+            temp_path.write_bytes(data)
+            self.modality_writer(str(temp_path), value, meta)
+            return temp_path.read_bytes()
+
+    def update_source(
+        self,
+        full_id: str,
+        source_entry: Mapping[str, Any],
+        data: bytes,
+        value: Any,
+        meta: dict[str, Any] | None,
+        *,
+        attributes: dict[str, Any] | None = None,
+    ) -> Path:
+        """Store a copy of the source file, updated by the writer to *value* (see :meth:`updated_bytes`)."""
+        updated = self.updated_bytes(source_entry, data, value, meta)
+        return self.copy_source(full_id, source_entry, updated, attributes=attributes)
 
     def update_head_meta(self, updates: Mapping[str, Any]) -> None:
         """Merge *updates* into the output head's ``modality.meta``; ``None`` removes a key."""

euler_preprocess-3.20.0/euler_preprocess/crop/label_files.py

@@ -0,0 +1,183 @@
+"""Crop label rasters in the stored file, so ground-truth labels keep every class.
+
+A semantic segmentation is ground truth: sky, road, the ego vehicle and every
+other class are encoded in one file, which consumers read through different
+functions (``class_segmentation`` for the labels, ``sky_mask`` for the sky).
+Re-encoding the decoded value through a dataset writer keeps what one loader
+returns, but not necessarily the file:
+
+- palettes and bit depths change (an indexed or 16-bit PNG comes back as 8-bit);
+- channels the loader ignores are dropped (Real Drive Sim keeps class IDs in
+  the red channel of an RGBA PNG; its writer zeroes the others);
+- a semantic map read through ``sky_mask`` would be written back as sky / not sky.
+
+So label rasters are cropped in the file itself: the stored pixels are sliced,
+and resized by picking the source pixels nearest resampling picks, in the
+same image mode, palette and format. PNG, TIFF and BMP go through Pillow, and
+``.npy`` / ``.npz`` through NumPy. The caller then checks that the modality's
+loader reads the result as the crop of the source values. Anything else
+(JPEG, 16-bit colour PNGs Pillow cannot hold, multi-frame images, ambiguous
+array layouts) returns ``None`` and is re-encoded with the dataset's writer.
+"""
+
+from __future__ import annotations
+
+import io
+import zipfile
+from pathlib import Path
+
+import numpy as np
+
+from euler_preprocess.crop.spec import CropBox
+
+#: Formats Pillow writes back losslessly.
+_LOSSLESS_IMAGE_FORMATS = {"PNG", "TIFF", "BMP"}
+_IMAGE_SUFFIXES = {".png", ".tif", ".tiff", ".bmp"}
+_PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
+#: Pillow modes that hold 16-bit samples.
+_WIDE_MODES = {"I", "I;16", "I;16B", "I;16L", "I;16N"}
+#: TIFF compressions that lose nothing.
+_LOSSLESS_TIFF_COMPRESSIONS = {None, "raw", "tiff_lzw", "tiff_adobe_deflate", "tiff_deflate", "packbits"}
+
+
+def source_grid(box: CropBox) -> tuple[np.ndarray, np.ndarray]:
+    """Return the source row of every output row and the source column of every output column.
+
+    Without a resize these are the crop's rows and columns. With one they are
+    the pixels euler-loading's nearest resize picks: it is separable, so
+    resizing a column of row indices (and a row of column indices) gives them
+    exactly.
+    """
+    rows = np.arange(box.top, box.top + box.height)
+    cols = np.arange(box.left, box.left + box.width)
+    if box.resized:
+        from euler_loading import resize_raster
+
+        pick_rows = resize_raster(
+            np.arange(box.height, dtype=np.int64)[:, None],
+            layout="HW",
+            size=(box.output_height, 1),
+            interpolation="nearest",
+        )
+        pick_cols = resize_raster(
+            np.arange(box.width, dtype=np.int64)[None, :],
+            layout="HW",
+            size=(1, box.output_width),
+            interpolation="nearest",
+        )
+        rows = rows[np.asarray(pick_rows)[:, 0]]
+        cols = cols[np.asarray(pick_cols)[0, :]]
+    return rows, cols
+
+
+def crop_label_file(
+    data: bytes, name: str, rows: np.ndarray, cols: np.ndarray, source_size: tuple[int, int]
+) -> bytes | None:
+    """Return the file *data* with only pixels ``rows`` x ``cols`` kept, or ``None``.
+
+    *name* gives the format (its suffix) and *source_size* the ``(height,
+    width)`` the stored raster must have. ``None`` means the file cannot be
+    cropped in place without loss.
+    """
+    suffix = Path(name).suffix.lower()
+    if suffix in _IMAGE_SUFFIXES:
+        return _crop_image(data, rows, cols, source_size)
+    if suffix == ".npy":
+        return _crop_npy(data, rows, cols, source_size)
+    if suffix == ".npz":
+        return _crop_npz(data, rows, cols, source_size)
+    return None
+
+
+def _png_bit_depth(data: bytes) -> int | None:
+    if data[:8] != _PNG_SIGNATURE or data[12:16] != b"IHDR" or len(data) < 25:
+        return None
+    return data[24]
+
+
+def _crop_image(data: bytes, rows: np.ndarray, cols: np.ndarray, source_size: tuple[int, int]) -> bytes | None:
+    from PIL import Image
+
+    try:
+        image = Image.open(io.BytesIO(data))
+        image.load()
+    except Exception:
+        return None
+    with image:
+        if image.format not in _LOSSLESS_IMAGE_FORMATS or getattr(image, "n_frames", 1) != 1:
+            return None
+        if (image.height, image.width) != tuple(source_size):
+            return None
+        if _png_bit_depth(data) == 16 and image.mode not in _WIDE_MODES:
+            # Pillow reads 16-bit colour PNGs as 8-bit: the file cannot be kept.
+            return None
+        params: dict[str, object] = {}
+        if image.format == "TIFF":
+            compression = image.info.get("compression")
+            if compression not in _LOSSLESS_TIFF_COMPRESSIONS:
+                return None
+            if compression is not None:
+                params["compression"] = compression
+        if "transparency" in image.info:
+            params["transparency"] = image.info["transparency"]
+
+        pixels = np.asarray(image)
+        cropped = Image.fromarray(np.ascontiguousarray(pixels[rows][:, cols]))
+        if image.mode == "P":
+            # Pillow turns the indices into an "L" image; the palette makes it "P" again.
+            cropped.putpalette(image.getpalette())
+        if cropped.mode != image.mode:
+            return None
+        buffer = io.BytesIO()
+        cropped.save(buffer, format=image.format, **params)
+        return buffer.getvalue()
+
+
+def _spatial_axes(shape: tuple[int, ...], source_size: tuple[int, int]) -> tuple[int, int] | None | bool:
+    """Return the adjacent ``(height, width)`` axes of *shape*; ``None`` if absent, ``False`` if ambiguous."""
+    height, width = source_size
+    matches = [axis for axis in range(len(shape) - 1) if shape[axis] == height and shape[axis + 1] == width]
+    if not matches:
+        return None
+    if len(matches) > 1:
+        return False
+    return matches[0], matches[0] + 1
+
+
+def _take(array: np.ndarray, axes: tuple[int, int], rows: np.ndarray, cols: np.ndarray) -> np.ndarray:
+    return np.ascontiguousarray(np.take(np.take(array, rows, axis=axes[0]), cols, axis=axes[1]))
+
+
+def _crop_npy(data: bytes, rows: np.ndarray, cols: np.ndarray, source_size: tuple[int, int]) -> bytes | None:
+    try:
+        array = np.load(io.BytesIO(data), allow_pickle=False)
+    except Exception:
+        return None
+    axes = _spatial_axes(array.shape, source_size)
+    if not axes:
+        return None
+    buffer = io.BytesIO()
+    np.save(buffer, _take(array, axes, rows, cols), allow_pickle=False)
+    return buffer.getvalue()
+
+
+def _crop_npz(data: bytes, rows: np.ndarray, cols: np.ndarray, source_size: tuple[int, int]) -> bytes | None:
+    try:
+        with zipfile.ZipFile(io.BytesIO(data)) as archive:
+            compressed = any(info.compress_type != zipfile.ZIP_STORED for info in archive.infolist())
+        with np.load(io.BytesIO(data), allow_pickle=False) as npz:
+            arrays = {key: npz[key] for key in npz.files}
+    except Exception:
+        return None
+    cropped: dict[str, np.ndarray] = {}
+    for key, array in arrays.items():
+        axes = _spatial_axes(array.shape, source_size)
+        if axes is False:
+            return None
+        # Arrays without the image's axes (class names, metadata) are kept as they are.
+        cropped[key] = array if axes is None else _take(array, axes, rows, cols)
+    if all(cropped[key] is arrays[key] for key in arrays):
+        return None
+    buffer = io.BytesIO()
+    (np.savez_compressed if compressed else np.savez)(buffer, **cropped)
+    return buffer.getvalue()

{euler_preprocess-3.19.0 → euler_preprocess-3.20.0}/euler_preprocess/crop/roles.py

@@ -10,13 +10,25 @@ sliced like an image.
 | ``HW``, ``HWC``, ``CHW``, ``1HW``, ``3HW`` | raster | Slice with the crop box |
 | ``3x3`` with type ``intrinsics`` | intrinsics | ``K' = crop_intrinsics(K, top, left)`` |
 | ``NC`` (point sets), ``4x4`` / ``Nx1`` (extrinsics) | copy | Copy the file unchanged |
+| ``dict`` with type ``gnss`` or ``sensor_timing`` | copy | Copy the file unchanged |
 
-Anything else needs a ``roles`` override (``"raster"`` or ``"copy"``).
+Anything else needs a ``roles`` override (``"raster"`` or ``"copy"``). Other
+``dict`` loaders can hold pixel-grid values (Real Drive Sim's calibration map
+holds the camera matrix), so they are never copied by default.
+
+A copied file that an ``intrinsics`` modality also indexes (MUSES
+``calib.json`` holds the camera matrix and every extrinsic) is written as the
+patched file, so the output carries one camera matrix.
 
 When the crop also resizes, colour images (types ``rgb`` and ``rccb``) are
 interpolated bilinearly and every other raster (depth, masks, labels) takes the
 nearest source pixel, so no new depth or label values appear. Sparse-depth
 rasters cannot be resized: nearest sampling would drop points at random.
+
+Label rasters (semantic, instance and panoptic segmentation, sky masks) are
+ground truth whose classes all live in one file. They are cropped in the
+stored file (:mod:`euler_preprocess.crop.label_files`) and can only be resized
+with ``nearest``.
 """
 
 from __future__ import annotations
@@ -38,10 +50,23 @@ RASTER_AXES: dict[str, tuple[int, int]] = {
     "3HW": (1, 2),
 }
 _COPY_SHAPES = {"NC", "4x4", "Nx1"}
+#: ``dict`` loaders whose values never refer to the image grid (MUSES GNSS
+#: records and ``meta.json`` sensor timing).
+_COPY_DICT_TYPES = {"gnss", "sensor_timing"}
 #: euler-loading's resize layouts for the annotated raster layouts.
 _RESIZE_LAYOUTS = {"HW": "HW", "HWC": "HWC", "CHW": "CHW", "1HW": "CHW", "3HW": "CHW"}
 #: Raster types that are interpolated when resized; all others use nearest.
 INTERPOLATED_TYPES = {"rgb", "rccb"}
+#: Categorical raster types: their values are class, instance or segment IDs.
+LABEL_TYPES = frozenset(
+    {
+        "semantic_segmentation",
+        "semantic_segmentation_color",
+        "instance_segmentation",
+        "panoptic_segmentation",
+        "sky_mask",
+    }
+)
 
 
 @dataclass(frozen=True)
@@ -83,6 +108,11 @@ class ModalityRole:
     def is_point_set(self) -> bool:
         return self.role == COPY and self.shape == "NC"
 
+    @property
+    def is_label(self) -> bool:
+        """A categorical raster: cropped in the stored file, resized only by nearest."""
+        return self.role == RASTER and self.modality_type in LABEL_TYPES
+
     def default_interpolation(self) -> str:
         return "bilinear" if self.modality_type in INTERPOLATED_TYPES else "nearest"
 
@@ -95,6 +125,8 @@ class ModalityRole:
                 f"Modality '{self.name}' is a sparse-depth raster and cannot be resized: "
                 "nearest sampling would drop points at random. Resize from point clouds instead."
             )
+        if self.is_label and interpolation not in (None, "nearest"):
+            raise ValueError(label_interpolation_error(self.name, self.modality_type, interpolation))
         return resize_raster(
             value,
             layout=_RESIZE_LAYOUTS[self.layout or ""],
@@ -103,6 +135,13 @@ class ModalityRole:
         )
 
 
+def label_interpolation_error(name: str, modality_type: str | None, interpolation: str) -> str:
+    return (
+        f"Modality '{name}' holds labels ({modality_type}) and can only be resized with "
+        f"'nearest', not {interpolation!r}: interpolating would invent class IDs"
+    )
+
+
 def describe_annotation(annotation: Mapping[str, Any] | None) -> str:
     if not annotation:
         return "no @modality_meta annotation"
@@ -152,7 +191,7 @@ def infer_role(
         return role(RASTER, shape)
     if shape == "3x3" and modality_type == "intrinsics":
         return role(INTRINSICS)
-    if shape in _COPY_SHAPES:
+    if shape in _COPY_SHAPES or (shape == "dict" and modality_type in _COPY_DICT_TYPES):
         return role(COPY)
     raise ValueError(
         f"Don't know how to crop modality '{name}' ({describe_annotation(annotation)}). "