euler-preprocess 3.18.0__tar.gz → 3.20.0__tar.gz

euler_preprocess-3.20.0/CHANGELOG.md

@@ -0,0 +1,73 @@
+# Changelog
+
+## Unreleased
+
+- `crop` crops label rasters in the stored file: semantic segmentation (the
+  ground truth that also holds the sky), instance and panoptic segmentation,
+  and sky masks.
+  - PNG, TIFF and BMP keep their mode, palette, bit depth and every channel;
+    `.npy`/`.npz` keep their dtype and other arrays.
+  - The result must decode exactly as the crop of the source labels.
+  - So no class is lost to a writer that re-encodes only what one loader
+    function returns: Real Drive Sim's non-red channels, or every class but
+    the sky for a dataset read through `sky_mask`. Label modalities no longer
+    need a writer.
+  - Files that cannot be kept (JPEG, 16-bit colour PNGs) are re-encoded with
+    the writer, with a warning.
+  - Labels can only be resized with `nearest`; another `resize.interpolation`
+    for them is an error.
+
+- `crop` supports MUSES (with euler-loading 2.35.0's MUSES writers):
+  - `calib.json` holds the frame camera's `K` and every extrinsic. The
+    `intrinsics` modality patches it once, and every other modality indexing
+    the same file, or a byte-identical copy (`camera_extrinsics`,
+    `lidar_to_gnss`), is written with the patched bytes, so the bundle holds
+    one `K`. What each of them reads from the patched file must stay as it was;
+    two intrinsics modalities patching one file differently are refused.
+  - GNSS records (`gnss`) and `meta.json` timing (`sensor_timing`) are copied
+    unchanged by default.
+  - `configs/crop_muses_hood_config.json` cuts the engine hood off the
+    1920×1080 frame camera images (bottom 160 rows, 1920×920 stay), with
+    `configs/crop_muses_dataset_config.json` as the dataset config.
+- Require euler-loading 2.35.0 for the MUSES writers.
+
+- Add `euler-preprocess lidar-motion`, which compensates scanning-lidar points for
+  the vehicle's motion during the sweep. It follows the MUSES SDK, moving each
+  point to the frame camera's mid-exposure using the `gnss`, `sensor_timing`
+  and `lidar_to_gnss` modalities.
+- Require euler-loading 2.32.0, the first release with `muses.gnss`,
+  `muses.sensor_timing`, `muses.read_lidar_to_gnss` and `muses.write_sparse_depth`.
+- Add `euler-preprocess crop`, which crops RGB, depth, intrinsics and
+  extrinsics coherently. The box is given as fractions or pixels, either removed
+  per edge or kept with an anchor, relative to a reference raster. Rasters are
+  sliced, pinhole intrinsics get `cx -= left`, `cy -= top` merged into a copy of
+  their file, and lidar point clouds and extrinsics are copied unchanged. Each
+  modality's treatment comes from its loader annotation, not its name. Shared
+  calibration files are written once, and samples with different crops cannot
+  share one. The outputs are reopened and checked against the crop.
+  `configs/crop_stf_valid_region_config.json` removes SeeingThroughFog's black
+  rectification border.
+- `crop` phase 3:
+  - `point_clouds: "prune_to_crop"` keeps only the lidar points that project
+    into the output image (`z > 0`, `rint(K'·T·X)` inside), the projection
+    consumers such as dmd3c-euler use;
+  - `resize` (`size` or `scale`) after the crop, with `K' = S·C·K`; colour
+    images are interpolated bilinearly, all other rasters take the nearest
+    pixel;
+  - `"output_layout": "scoped"` writes one root (directory or zip) with one
+    metadata scope per modality, as dmd3c-euler's `--dataset-root` reads;
+  - every encoded raster or pruned cloud is decoded again and must reproduce
+    the values;
+  - `euler-preprocess crop-box` prints the pixel box and the matching
+    `spatial` preprocessing block.
+- `crop` uses the new public euler-loading accessors (`get_loader_meta`,
+  `get_hierarchical_files`) and ds-crawler's `set_modality_meta` / `path_prefix`.
+  Requires euler-loading 2.33.0 and ds-crawler 2.14.0.
+- `extract_intrinsics` (radial, fog) also uses a hierarchical intrinsics file
+  that is not named `intrinsics` when it is the only one, such as SeeingThroughFog's
+  `calib_cam_stereo_left`. Several matching files now raise instead of being
+  ignored.
+
+## 3.14.0 (unreleased)
+
+Consume shared preprocessing config and add the opt-in spatial producer, explicit variants/mappings and captured source-backed outputs. Preserve ordinary pipeline APIs.

{euler_preprocess-3.18.0/euler_preprocess.egg-info → euler_preprocess-3.20.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-preprocess
-Version: 3.18.0
+Version: 3.20.0
 Summary: Physics-based preprocessing (fog, etc.) for RGB+depth datasets
 Author-email: Daniel Rothenpieler <rothenpielerdaniel@gmail.com>
 License-Expression: MIT
@@ -9,8 +9,8 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: numpy
 Requires-Dist: Pillow
-Requires-Dist: euler-loading>=2.32.0
-Requires-Dist: ds-crawler>=2.11.0
+Requires-Dist: euler-loading>=2.35.0
+Requires-Dist: ds-crawler>=2.14.0
 Requires-Dist: euler-dataset-contract>=0.5.0
 Provides-Extra: gpu
 Requires-Dist: torch; extra == "gpu"
@@ -65,6 +65,8 @@ qualitative review.
 | `euler-preprocess sky-depth` | Override depth values in sky regions with a constant |
 | `euler-preprocess radial` | Convert planar (z-buffer) depth to radial (Euclidean) depth |
 | `euler-preprocess lidar-motion` | Compensate scanning-lidar points for the vehicle's motion during a sweep |
+| `euler-preprocess crop` | Crop RGB, depth, intrinsics and extrinsics coherently |
+| `euler-preprocess crop-box` | Print the pixel box a crop config gives for an image size |
 
 ## Installation
 
@@ -79,6 +81,7 @@ euler-preprocess fog       -c configs/example_dataset_config.json
 euler-preprocess sky-depth -c configs/sky_depth_dataset_config.json
 euler-preprocess radial    -c configs/radial_dataset_config.json
 euler-preprocess lidar-motion -c configs/lidar_motion_dataset_config.json
+euler-preprocess crop      -c configs/crop_stf_dataset_config.json
 ```
 
 ## Dataset Config
@@ -737,6 +740,138 @@ holds the target time and the speed and rates used. Point
 
 ---
 
+## Coherent Crop
+
+`crop` cuts the same box out of every image-shaped modality and corrects
+everything that refers to the pixel grid, so a cropped bundle projects exactly
+like the original:
+
+| Modality kind | Crop |
+|---|---|
+| Rasters: RGB, dense depth | Sliced; no resampling, so values are unchanged |
+| Label rasters: semantic segmentation (the ground truth that also holds the sky), instance and panoptic segmentation, sky masks | Cropped in the stored file, so every class, channel, palette and bit depth is kept; resized only by nearest |
+| Pinhole intrinsics `K` (or `P` for rectified images) | `cx -= left`, `cy -= top`; `fx`, `fy`, skew unchanged |
+| Lidar point clouds | Copied byte for byte, or pruned to the points that project into the crop |
+| Extrinsics | Copied byte for byte |
+
+The extrinsics need no change, because `C·K·[R|t]·X = K'·[R|t]·X` with
+`C = [[1,0,-left],[0,1,-top],[0,0,1]]`: the whole correction lives in the
+camera matrix, which is also what the official SeeingThroughFog tools do. An
+optional resize afterwards adds `S` (an `align_corners=False` rescale), so
+`K' = S·C·K`, still with untouched extrinsics.
+Which kind a modality is comes from its euler-loading loader annotation
+(`shape`, `type`), never from its name.
+
+The dataset config lists every modality to crop; whether a modality is
+per-sample or hierarchical (shared calibration) is inferred from its index:
+
+```json
+{
+  "transform_config_path": "crop_stf_valid_region_config.json",
+  "output_path": "/data/stf_crop",
+  "modalities": {
+    "rgb": "/data/stf/cam_stereo_left_lut",
+    "sparse_depth": "/data/stf/lidar_hdl64_strongest",
+    "intrinsics": "/data/stf/calib_cam_stereo_left",
+    "camera_extrinsics": "/data/stf/calib_tf_tree"
+  },
+  "samples": {"start": 0, "count": 100}
+}
+```
+
+Each modality is written to `<output_path>/<name>`, or to its
+`pipeline.output_targets[]` entry (`slot` = the modality name; directory or zip).
+With pipeline targets, every modality needs one and the outputs manifest lists
+them all. `"output_layout": "scoped"` writes one root at `output_path` instead
+(a directory or `.zip`): one ds-crawler metadata scope per modality, named after
+it, with its files below `<name>/`. That is the layout dmd3c-euler's
+`--dataset-root` reads. The dataset config must not contain `preprocessing`.
+
+The crop config (`configs/crop_config.json`) describes the box relative to the
+`reference` raster:
+
+```json
+{
+  "reference": "rgb",
+  "crop": {"remove": {"top": 0.20, "bottom": 0.05, "left": 0.15, "right": 0.19}},
+  "point_clouds": "keep",
+  "verify": "all"
+}
+```
+
+| Key | Meaning |
+|---|---|
+| `crop.remove` | What to cut off each edge (`top`, `bottom`, `left`, `right`) |
+| `crop.keep` + `crop.anchor` | The kept `height` / `width`, placed at `center` (default), `top`, `bottom`, `left`, `right`, `top_left`, `top_right`, `bottom_left` or `bottom_right` |
+| `crop.unit` | `fraction` (default) or `pixel`. Fractions lie in `[0, 1)` (`keep`: `(0, 1]`) and round half up per edge: `floor(f · size + 0.5)` |
+| `crop.multiple_of` | Trim the kept size down to a multiple, e.g. a network stride; the extra rows and columns come off both sides evenly |
+| `resize` | Optional, after the crop: `{"size": [h, w]}` or `{"scale": 0.5}` (or `[sy, sx]`); `interpolation` maps modality names to `nearest`, `bilinear` or `bicubic` |
+| `point_clouds` | `keep` (default): copy every point, consumers clip to the image. `prune_to_crop`: keep only the points with `z > 0` whose pixel `rint(K'·T·X)` lies in the output image, as consumers such as dmd3c-euler project them |
+| `projection` | For `prune_to_crop`: the `intrinsics` and `extrinsics` modalities to project with, when the dataset has more than one candidate |
+| `verify` | `all` (default), `none`, or a sample count: reopen the outputs and check every file against the crop (rasters and pruned clouds as encoded, intrinsics to 1e-3 px, copies byte-identical) |
+| `roles` | Override a modality's kind with `raster` or `copy`, for loaders whose annotation the crop does not know, or to copy an intrinsics file of another camera (e.g. MUSES `sensor: event`) instead of cropping it |
+
+Pixel units reproduce the official boxes exactly, e.g. `RGB2Gatedv2` is
+`{"remove": {"top": 202, "bottom": 54, "left": 280, "right": 360}, "unit": "pixel"}`.
+`configs/crop_stf_valid_region_config.json` removes the black border the
+SeeingThroughFog rectification leaves on `_lut` frames (top 34, bottom 46,
+left 16 px), keeping a fully valid 1904×944 image.
+
+MUSES keeps the frame camera's `K` and every extrinsic in one `calib.json`,
+indexed as `intrinsics`, `camera_extrinsics` (and `lidar_to_gnss`). The crop
+patches `intrinsics.rgb.K` once and writes that file for every modality that
+indexes it, also when it is indexed as separate byte-identical copies, so the
+bundle holds one `K`. `D`, the event camera and all extrinsics stay as they
+are. `configs/crop_muses_hood_config.json` cuts the engine hood off the
+1920×1080 frame camera images: the bottom 160 rows, leaving 1920×920. Rows come
+off the bottom only, so `K` and every `calib.json` stay byte-identical. Run
+`lidar-motion` first and crop its output (`configs/crop_muses_dataset_config.json`).
+GNSS records and `meta.json` sensor timing are copied unchanged, like the
+extrinsics. MUSES label images (`gt_semantic`, `gt_panoptic`) are cropped in place like
+any label raster.
+
+Label rasters (loader types `semantic_segmentation`, `semantic_segmentation_color`,
+`instance_segmentation`, `panoptic_segmentation`, `sky_mask`) are cropped in the
+file itself: PNG, TIFF and BMP through Pillow in their own mode and palette,
+`.npy`/`.npz` through NumPy. A semantic segmentation is the ground truth for
+every class including the sky, and consumers read it through different
+functions (`class_segmentation`, or `sky_mask` for sky-depth). Re-encoding the
+decoded labels would drop what one function does not return: Real Drive Sim's
+non-red channels, an indexed PNG's palette, or every class but the sky when a
+dataset only has a `sky_mask` function. The in-place crop must decode exactly
+as the crop of the source labels. Files it cannot keep (JPEG, 16-bit colour
+PNGs, ambiguous array layouts) fall back to the writer with a warning. Labels
+are never interpolated: a `resize.interpolation` other than `nearest` for them
+is an error.
+
+Other rasters are re-encoded with the dataset's euler-loading writer, so a raster
+modality needs one (SeeingThroughFog's raw `rccb` TIFFs have none). Each
+encoded file is decoded again right away and must give back the cropped values:
+exactly, or for an interpolated resize within half an 8-bit step. A resize
+interpolates colour images (`rgb`, `rccb`) bilinearly and takes the nearest
+pixel for everything else, so depth and label values stay real values;
+sparse-depth rasters are not resized. Pruned point clouds need a writer too
+(`princeton_dense.write_sparse_depth`, `muses.write_sparse_depth`). Intrinsics
+are written by copying the source file and letting its writer move the matrices
+onto the new grid: for a ROS CameraInfo file such as
+`calib_cam_stereo_left.json`, `K` and `P` shift by exact integers, `width` and
+`height` become the crop size, and `D`, `R`, `roi` and the header are kept. A
+calibration file that records another image size than the reference rasters
+is refused. A hierarchical file is written once, so every sample sharing it
+must get the same box; mixed image sizes under one calibration file are an
+error. Every output head records `meta.crop` (reference, spec and boxes), and
+every file its own `attributes.crop` box; pruned clouds also record
+`attributes.point_clouds` (source and kept point counts).
+
+To reproduce a crop with `euler-preprocess spatial`, print its pixel box and
+the matching `preprocessing` block for a reference size:
+
+```bash
+euler-preprocess crop-box -c configs/crop_config.json --size 1024 1920
+```
+
+---
+
 ## License
 
 [MIT](LICENSE) © Daniel Rothenpieler

{euler_preprocess-3.18.0 → euler_preprocess-3.20.0}/README.md

@@ -45,6 +45,8 @@ qualitative review.
 | `euler-preprocess sky-depth` | Override depth values in sky regions with a constant |
 | `euler-preprocess radial` | Convert planar (z-buffer) depth to radial (Euclidean) depth |
 | `euler-preprocess lidar-motion` | Compensate scanning-lidar points for the vehicle's motion during a sweep |
+| `euler-preprocess crop` | Crop RGB, depth, intrinsics and extrinsics coherently |
+| `euler-preprocess crop-box` | Print the pixel box a crop config gives for an image size |
 
 ## Installation
 
@@ -59,6 +61,7 @@ euler-preprocess fog       -c configs/example_dataset_config.json
 euler-preprocess sky-depth -c configs/sky_depth_dataset_config.json
 euler-preprocess radial    -c configs/radial_dataset_config.json
 euler-preprocess lidar-motion -c configs/lidar_motion_dataset_config.json
+euler-preprocess crop      -c configs/crop_stf_dataset_config.json
 ```
 
 ## Dataset Config
@@ -717,6 +720,138 @@ holds the target time and the speed and rates used. Point
 
 ---
 
+## Coherent Crop
+
+`crop` cuts the same box out of every image-shaped modality and corrects
+everything that refers to the pixel grid, so a cropped bundle projects exactly
+like the original:
+
+| Modality kind | Crop |
+|---|---|
+| Rasters: RGB, dense depth | Sliced; no resampling, so values are unchanged |
+| Label rasters: semantic segmentation (the ground truth that also holds the sky), instance and panoptic segmentation, sky masks | Cropped in the stored file, so every class, channel, palette and bit depth is kept; resized only by nearest |
+| Pinhole intrinsics `K` (or `P` for rectified images) | `cx -= left`, `cy -= top`; `fx`, `fy`, skew unchanged |
+| Lidar point clouds | Copied byte for byte, or pruned to the points that project into the crop |
+| Extrinsics | Copied byte for byte |
+
+The extrinsics need no change, because `C·K·[R|t]·X = K'·[R|t]·X` with
+`C = [[1,0,-left],[0,1,-top],[0,0,1]]`: the whole correction lives in the
+camera matrix, which is also what the official SeeingThroughFog tools do. An
+optional resize afterwards adds `S` (an `align_corners=False` rescale), so
+`K' = S·C·K`, still with untouched extrinsics.
+Which kind a modality is comes from its euler-loading loader annotation
+(`shape`, `type`), never from its name.
+
+The dataset config lists every modality to crop; whether a modality is
+per-sample or hierarchical (shared calibration) is inferred from its index:
+
+```json
+{
+  "transform_config_path": "crop_stf_valid_region_config.json",
+  "output_path": "/data/stf_crop",
+  "modalities": {
+    "rgb": "/data/stf/cam_stereo_left_lut",
+    "sparse_depth": "/data/stf/lidar_hdl64_strongest",
+    "intrinsics": "/data/stf/calib_cam_stereo_left",
+    "camera_extrinsics": "/data/stf/calib_tf_tree"
+  },
+  "samples": {"start": 0, "count": 100}
+}
+```
+
+Each modality is written to `<output_path>/<name>`, or to its
+`pipeline.output_targets[]` entry (`slot` = the modality name; directory or zip).
+With pipeline targets, every modality needs one and the outputs manifest lists
+them all. `"output_layout": "scoped"` writes one root at `output_path` instead
+(a directory or `.zip`): one ds-crawler metadata scope per modality, named after
+it, with its files below `<name>/`. That is the layout dmd3c-euler's
+`--dataset-root` reads. The dataset config must not contain `preprocessing`.
+
+The crop config (`configs/crop_config.json`) describes the box relative to the
+`reference` raster:
+
+```json
+{
+  "reference": "rgb",
+  "crop": {"remove": {"top": 0.20, "bottom": 0.05, "left": 0.15, "right": 0.19}},
+  "point_clouds": "keep",
+  "verify": "all"
+}
+```
+
+| Key | Meaning |
+|---|---|
+| `crop.remove` | What to cut off each edge (`top`, `bottom`, `left`, `right`) |
+| `crop.keep` + `crop.anchor` | The kept `height` / `width`, placed at `center` (default), `top`, `bottom`, `left`, `right`, `top_left`, `top_right`, `bottom_left` or `bottom_right` |
+| `crop.unit` | `fraction` (default) or `pixel`. Fractions lie in `[0, 1)` (`keep`: `(0, 1]`) and round half up per edge: `floor(f · size + 0.5)` |
+| `crop.multiple_of` | Trim the kept size down to a multiple, e.g. a network stride; the extra rows and columns come off both sides evenly |
+| `resize` | Optional, after the crop: `{"size": [h, w]}` or `{"scale": 0.5}` (or `[sy, sx]`); `interpolation` maps modality names to `nearest`, `bilinear` or `bicubic` |
+| `point_clouds` | `keep` (default): copy every point, consumers clip to the image. `prune_to_crop`: keep only the points with `z > 0` whose pixel `rint(K'·T·X)` lies in the output image, as consumers such as dmd3c-euler project them |
+| `projection` | For `prune_to_crop`: the `intrinsics` and `extrinsics` modalities to project with, when the dataset has more than one candidate |
+| `verify` | `all` (default), `none`, or a sample count: reopen the outputs and check every file against the crop (rasters and pruned clouds as encoded, intrinsics to 1e-3 px, copies byte-identical) |
+| `roles` | Override a modality's kind with `raster` or `copy`, for loaders whose annotation the crop does not know, or to copy an intrinsics file of another camera (e.g. MUSES `sensor: event`) instead of cropping it |
+
+Pixel units reproduce the official boxes exactly, e.g. `RGB2Gatedv2` is
+`{"remove": {"top": 202, "bottom": 54, "left": 280, "right": 360}, "unit": "pixel"}`.
+`configs/crop_stf_valid_region_config.json` removes the black border the
+SeeingThroughFog rectification leaves on `_lut` frames (top 34, bottom 46,
+left 16 px), keeping a fully valid 1904×944 image.
+
+MUSES keeps the frame camera's `K` and every extrinsic in one `calib.json`,
+indexed as `intrinsics`, `camera_extrinsics` (and `lidar_to_gnss`). The crop
+patches `intrinsics.rgb.K` once and writes that file for every modality that
+indexes it, also when it is indexed as separate byte-identical copies, so the
+bundle holds one `K`. `D`, the event camera and all extrinsics stay as they
+are. `configs/crop_muses_hood_config.json` cuts the engine hood off the
+1920×1080 frame camera images: the bottom 160 rows, leaving 1920×920. Rows come
+off the bottom only, so `K` and every `calib.json` stay byte-identical. Run
+`lidar-motion` first and crop its output (`configs/crop_muses_dataset_config.json`).
+GNSS records and `meta.json` sensor timing are copied unchanged, like the
+extrinsics. MUSES label images (`gt_semantic`, `gt_panoptic`) are cropped in place like
+any label raster.
+
+Label rasters (loader types `semantic_segmentation`, `semantic_segmentation_color`,
+`instance_segmentation`, `panoptic_segmentation`, `sky_mask`) are cropped in the
+file itself: PNG, TIFF and BMP through Pillow in their own mode and palette,
+`.npy`/`.npz` through NumPy. A semantic segmentation is the ground truth for
+every class including the sky, and consumers read it through different
+functions (`class_segmentation`, or `sky_mask` for sky-depth). Re-encoding the
+decoded labels would drop what one function does not return: Real Drive Sim's
+non-red channels, an indexed PNG's palette, or every class but the sky when a
+dataset only has a `sky_mask` function. The in-place crop must decode exactly
+as the crop of the source labels. Files it cannot keep (JPEG, 16-bit colour
+PNGs, ambiguous array layouts) fall back to the writer with a warning. Labels
+are never interpolated: a `resize.interpolation` other than `nearest` for them
+is an error.
+
+Other rasters are re-encoded with the dataset's euler-loading writer, so a raster
+modality needs one (SeeingThroughFog's raw `rccb` TIFFs have none). Each
+encoded file is decoded again right away and must give back the cropped values:
+exactly, or for an interpolated resize within half an 8-bit step. A resize
+interpolates colour images (`rgb`, `rccb`) bilinearly and takes the nearest
+pixel for everything else, so depth and label values stay real values;
+sparse-depth rasters are not resized. Pruned point clouds need a writer too
+(`princeton_dense.write_sparse_depth`, `muses.write_sparse_depth`). Intrinsics
+are written by copying the source file and letting its writer move the matrices
+onto the new grid: for a ROS CameraInfo file such as
+`calib_cam_stereo_left.json`, `K` and `P` shift by exact integers, `width` and
+`height` become the crop size, and `D`, `R`, `roi` and the header are kept. A
+calibration file that records another image size than the reference rasters
+is refused. A hierarchical file is written once, so every sample sharing it
+must get the same box; mixed image sizes under one calibration file are an
+error. Every output head records `meta.crop` (reference, spec and boxes), and
+every file its own `attributes.crop` box; pruned clouds also record
+`attributes.point_clouds` (source and kept point counts).
+
+To reproduce a crop with `euler-preprocess spatial`, print its pixel box and
+the matching `preprocessing` block for a reference size:
+
+```bash
+euler-preprocess crop-box -c configs/crop_config.json --size 1024 1920
+```
+
+---
+
 ## License
 
 [MIT](LICENSE) © Daniel Rothenpieler