euler-eval 2.12.0__tar.gz → 2.13.0__tar.gz

{euler_eval-2.12.0 → euler_eval-2.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-eval
-Version: 2.12.0
+Version: 2.13.0
 Summary: Depth map evaluation toolkit with comprehensive metrics
 Author: Depth Eval Contributors
 License: MIT
@@ -149,7 +149,7 @@ This pre-downloads:
 # Evaluate with default settings (auto-selects CUDA when available)
 depth-eval config.json --batch-size 32
 
-# Evaluate with sky masking enabled (requires gt.segmentation in config)
+# Evaluate with sky masking enabled (requires gt.segmentation or gt.semantic_segmentation)
 depth-eval config.json --mask-sky -v
 
 # Skip RGB evaluation, only evaluate depth
@@ -241,7 +241,13 @@ Defines GT modalities, prediction datasets to evaluate, and optional euler_train
 
 Each modality entry can optionally include a `split` field to select a specific split from the dataset (e.g. `{ "path": "/data/gt/depth", "split": "test" }`). euler-loading inline selectors are also accepted in `path`, such as `/data/muses.zip:test` or `/data/muses.zip:test#scope=rgb`.
 
-For sparse pointcloud depth evaluation, use `gt.sparse_depth` instead of `gt.depth`. The prediction uses a dense depth-like map under `datasets[].depth`, `datasets[].relative_depth`, or `datasets[].affine_depth`. The evaluator projects the sparse GT point cloud into the prediction image plane using `gt.intrinsics` and `gt.camera_extrinsics`, then computes pointwise depth metrics only at projected valid pixels.
+For sparse pointcloud depth evaluation, use `gt.sparse_depth` instead of `gt.depth`. The prediction uses a dense depth-like map under `datasets[].depth`, `datasets[].relative_depth`, or `datasets[].affine_depth`. The evaluator projects the sparse GT point cloud into the prediction image plane using `gt.intrinsics` and `gt.camera_extrinsics`, then computes pointwise depth metrics only at projected valid pixels. For MUSES through `euler_loading.loaders.muses`, `gt.camera_extrinsics` normally resolves to the direct `lidar2rgb` transform and no extra lidar pose is needed. If a dataset exposes separate lidar and camera poses in a shared frame instead, provide optional `gt.lidar_extrinsics`; the evaluator composes `inv(camera_pose) @ lidar_pose` before projection.
+
+Sparse depth does not require segmentation GT. `gt.segmentation` and
+`gt.semantic_segmentation` are optional aliases for the same sky-mask source.
+They are loaded only when `--mask-sky` is set; in sparse depth evaluation that
+mask excludes sky pixels from projected-point metrics and from scale/shift
+fitting. Without `--mask-sky`, the segmentation entry is not loaded or used.
 
 #### GT section
 
@@ -251,10 +257,11 @@ For sparse pointcloud depth evaluation, use `gt.sparse_depth` instead of `gt.dep
 | `gt.depth.path` | no\* | Path to GT depth dataset |
 | `gt.sparse_depth.path` | no\* | Path to sparse pointcloud GT dataset, e.g. `sparse_depth` with `(N,C)` points whose first columns are `x,y,z` in meters |
 | `gt.rays.path` | no\* | Path to GT ray direction map dataset (for rays evaluation) |
-| `gt.segmentation.path` | no | Path to GT segmentation (needed for `--mask-sky`) |
+| `gt.segmentation.path` / `gt.semantic_segmentation.path` | no | Path to GT segmentation (needed for `--mask-sky`; use one key, not both) |
 | `gt.calibration.path` | no | Path to calibration data (camera intrinsics matrices) |
 | `gt.intrinsics.path` | required with `gt.sparse_depth` | Path to camera intrinsics matrices for pointcloud projection |
 | `gt.camera_extrinsics.path` | required with `gt.sparse_depth` | Path to source-to-camera extrinsics, e.g. MUSES `lidar2rgb`, for pointcloud projection |
+| `gt.lidar_extrinsics.path` | no | Optional lidar sensor pose in the same shared frame as `gt.camera_extrinsics`; when provided, `gt.camera_extrinsics` is interpreted as the camera pose and both are composed for projection |
 | `gt.name` | no | Display name for ground truth (default: `"GT"`) |
 
 \* At least one of `gt.rgb.path`, `gt.depth.path`, `gt.sparse_depth.path`, or `gt.rays.path` is required.

{euler_eval-2.12.0 → euler_eval-2.13.0}/README.md

@@ -111,7 +111,7 @@ This pre-downloads:
 # Evaluate with default settings (auto-selects CUDA when available)
 depth-eval config.json --batch-size 32
 
-# Evaluate with sky masking enabled (requires gt.segmentation in config)
+# Evaluate with sky masking enabled (requires gt.segmentation or gt.semantic_segmentation)
 depth-eval config.json --mask-sky -v
 
 # Skip RGB evaluation, only evaluate depth
@@ -203,7 +203,13 @@ Defines GT modalities, prediction datasets to evaluate, and optional euler_train
 
 Each modality entry can optionally include a `split` field to select a specific split from the dataset (e.g. `{ "path": "/data/gt/depth", "split": "test" }`). euler-loading inline selectors are also accepted in `path`, such as `/data/muses.zip:test` or `/data/muses.zip:test#scope=rgb`.
 
-For sparse pointcloud depth evaluation, use `gt.sparse_depth` instead of `gt.depth`. The prediction uses a dense depth-like map under `datasets[].depth`, `datasets[].relative_depth`, or `datasets[].affine_depth`. The evaluator projects the sparse GT point cloud into the prediction image plane using `gt.intrinsics` and `gt.camera_extrinsics`, then computes pointwise depth metrics only at projected valid pixels.
+For sparse pointcloud depth evaluation, use `gt.sparse_depth` instead of `gt.depth`. The prediction uses a dense depth-like map under `datasets[].depth`, `datasets[].relative_depth`, or `datasets[].affine_depth`. The evaluator projects the sparse GT point cloud into the prediction image plane using `gt.intrinsics` and `gt.camera_extrinsics`, then computes pointwise depth metrics only at projected valid pixels. For MUSES through `euler_loading.loaders.muses`, `gt.camera_extrinsics` normally resolves to the direct `lidar2rgb` transform and no extra lidar pose is needed. If a dataset exposes separate lidar and camera poses in a shared frame instead, provide optional `gt.lidar_extrinsics`; the evaluator composes `inv(camera_pose) @ lidar_pose` before projection.
+
+Sparse depth does not require segmentation GT. `gt.segmentation` and
+`gt.semantic_segmentation` are optional aliases for the same sky-mask source.
+They are loaded only when `--mask-sky` is set; in sparse depth evaluation that
+mask excludes sky pixels from projected-point metrics and from scale/shift
+fitting. Without `--mask-sky`, the segmentation entry is not loaded or used.
 
 #### GT section
 
@@ -213,10 +219,11 @@ For sparse pointcloud depth evaluation, use `gt.sparse_depth` instead of `gt.dep
 | `gt.depth.path` | no\* | Path to GT depth dataset |
 | `gt.sparse_depth.path` | no\* | Path to sparse pointcloud GT dataset, e.g. `sparse_depth` with `(N,C)` points whose first columns are `x,y,z` in meters |
 | `gt.rays.path` | no\* | Path to GT ray direction map dataset (for rays evaluation) |
-| `gt.segmentation.path` | no | Path to GT segmentation (needed for `--mask-sky`) |
+| `gt.segmentation.path` / `gt.semantic_segmentation.path` | no | Path to GT segmentation (needed for `--mask-sky`; use one key, not both) |
 | `gt.calibration.path` | no | Path to calibration data (camera intrinsics matrices) |
 | `gt.intrinsics.path` | required with `gt.sparse_depth` | Path to camera intrinsics matrices for pointcloud projection |
 | `gt.camera_extrinsics.path` | required with `gt.sparse_depth` | Path to source-to-camera extrinsics, e.g. MUSES `lidar2rgb`, for pointcloud projection |
+| `gt.lidar_extrinsics.path` | no | Optional lidar sensor pose in the same shared frame as `gt.camera_extrinsics`; when provided, `gt.camera_extrinsics` is interpreted as the camera pose and both are composed for projection |
 | `gt.name` | no | Display name for ground truth (default: `"GT"`) |
 
 \* At least one of `gt.rgb.path`, `gt.depth.path`, `gt.sparse_depth.path`, or `gt.rays.path` is required.

{euler_eval-2.12.0 → euler_eval-2.13.0}/euler_eval/cli.py

@@ -152,6 +152,7 @@ def _rgb_eval_axes(*, benchmark: bool = False) -> dict[str, AxisDeclaration]:
 _RAYS_EVAL_AXES: dict[str, AxisDeclaration] = {}
 
 _PRED_DEPTH_KEYS = ("depth", "relative_depth", "affine_depth")
+_GT_SEGMENTATION_KEYS = ("segmentation", "semantic_segmentation")
 
 # Downstream eval consumers validate metricSet.metricNamespace with a stricter
 # first-segment rule than euler_metric_naming's modality validator. Keep the
@@ -592,6 +593,24 @@ def _prediction_depth_space_hint(key: str | None) -> str | None:
     return None
 
 
+def _gt_segmentation_entry(gt: dict) -> tuple[str | None, dict | None]:
+    """Return the configured GT segmentation entry, accepting legacy aliases."""
+    matches = [
+        key
+        for key in _GT_SEGMENTATION_KEYS
+        if key in gt and "path" in gt.get(key, {})
+    ]
+    if not matches:
+        return None, None
+    if len(matches) > 1:
+        raise ValueError(
+            "gt contains multiple segmentation entries "
+            f"{matches}. Use only one of {list(_GT_SEGMENTATION_KEYS)}."
+        )
+    key = matches[0]
+    return key, gt[key]
+
+
 def validate_gt_config(gt: dict) -> None:
     """Validate the ``gt`` section of the configuration.
 
@@ -619,15 +638,19 @@ def validate_gt_config(gt: dict) -> None:
             "gt.camera_extrinsics.path for pointcloud projection"
         )
 
+    _gt_segmentation_entry(gt)
+
     for modality in (
         "rgb",
         "depth",
         "sparse_depth",
         "rays",
         "segmentation",
+        "semantic_segmentation",
         "calibration",
         "intrinsics",
         "camera_extrinsics",
+        "lidar_extrinsics",
     ):
         if modality in gt and "path" in gt[modality]:
             p = normalize_modality_path(
@@ -962,16 +985,18 @@ def main():
     print("-" * 60)
 
     # Check sky masking prerequisites
+    segmentation_key, segmentation_entry = _gt_segmentation_entry(config["gt"])
     if args.mask_sky:
-        if "segmentation" not in config["gt"]:
+        if segmentation_entry is None:
             print(
-                "Warning: --mask-sky requires gt.segmentation in config. "
+                "Warning: --mask-sky requires gt.segmentation or "
+                "gt.semantic_segmentation in config. "
                 "Sky masking disabled.",
                 file=sys.stderr,
             )
             args.mask_sky = False
         else:
-            print("Sky masking enabled")
+            print(f"Sky masking enabled ({segmentation_key})")
     print("-" * 60)
 
     # Initialize sanity checker if not disabled
@@ -1006,8 +1031,11 @@ def main():
     calibration_path = gt.get("calibration", {}).get("path")
     intrinsics_path = gt.get("intrinsics", {}).get("path")
     camera_extrinsics_path = gt.get("camera_extrinsics", {}).get("path")
+    lidar_extrinsics_path = gt.get("lidar_extrinsics", {}).get("path")
     segmentation_path = (
-        gt.get("segmentation", {}).get("path") if args.mask_sky else None
+        segmentation_entry.get("path")
+        if args.mask_sky and segmentation_entry is not None
+        else None
     )
     gt_depth_split = gt.get("depth", {}).get("split")
     gt_sparse_depth_split = gt.get("sparse_depth", {}).get("split")
@@ -1016,8 +1044,11 @@ def main():
     calibration_split = gt.get("calibration", {}).get("split")
     intrinsics_split = gt.get("intrinsics", {}).get("split")
     camera_extrinsics_split = gt.get("camera_extrinsics", {}).get("split")
+    lidar_extrinsics_split = gt.get("lidar_extrinsics", {}).get("split")
     segmentation_split = (
-        gt.get("segmentation", {}).get("split") if args.mask_sky else None
+        segmentation_entry.get("split")
+        if args.mask_sky and segmentation_entry is not None
+        else None
     )
 
     # Evaluate each prediction dataset
@@ -1063,6 +1094,7 @@ def main():
                 pred_depth_metadata_scope=(
                     pred_depth_key if pred_depth_key != "depth" else None
                 ),
+                segmentation_modality_key=segmentation_key or "segmentation",
                 gt_depth_split=gt_depth_split,
                 pred_depth_split=pred_depth_split,
                 calibration_split=calibration_split,
@@ -1242,20 +1274,25 @@ def main():
                 print(f"  Pred depth entry:     {pred_depth_key}")
             print(f"  Intrinsics:           {intrinsics_path}")
             print(f"  Camera extrinsics:    {camera_extrinsics_path}")
+            if lidar_extrinsics_path:
+                print(f"  Lidar extrinsics:     {lidar_extrinsics_path}")
 
             sparse_depth_dataset = build_sparse_depth_eval_dataset(
                 gt_sparse_depth_path=gt_sparse_depth_path,
                 pred_depth_path=pred_depth_path,
                 intrinsics_path=intrinsics_path,
                 camera_extrinsics_path=camera_extrinsics_path,
+                lidar_extrinsics_path=lidar_extrinsics_path,
                 segmentation_path=segmentation_path,
                 pred_depth_metadata_scope=(
                     pred_depth_key if pred_depth_key != "depth" else None
                 ),
+                segmentation_modality_key=segmentation_key or "segmentation",
                 gt_sparse_depth_split=gt_sparse_depth_split,
                 pred_depth_split=pred_depth_split,
                 intrinsics_split=intrinsics_split,
                 camera_extrinsics_split=camera_extrinsics_split,
+                lidar_extrinsics_split=lidar_extrinsics_split,
                 segmentation_split=segmentation_split,
             )
             et_eval_datasets["sparse_depth"] = sparse_depth_dataset
@@ -1334,6 +1371,8 @@ def main():
                         "intrinsics_split": intrinsics_split,
                         "camera_extrinsics_path": camera_extrinsics_path,
                         "camera_extrinsics_split": camera_extrinsics_split,
+                        "lidar_extrinsics_path": lidar_extrinsics_path,
+                        "lidar_extrinsics_split": lidar_extrinsics_split,
                     },
                     "spatial_alignment": {
                         "method": sparse_depth_spatial.get(
@@ -1450,6 +1489,7 @@ def main():
                 gt_depth_split=gt_depth_split,
                 calibration_split=calibration_split,
                 segmentation_split=segmentation_split,
+                segmentation_modality_key=segmentation_key or "segmentation",
             )
             et_eval_datasets["rgb"] = rgb_dataset
 

{euler_eval-2.12.0 → euler_eval-2.13.0}/euler_eval/data.py

@@ -13,7 +13,7 @@ import torch
 from ds_crawler import get_dataset_contract, index_dataset_from_path
 from euler_loading import Modality, MultiModalDataset, resolve_loader_module
 
-from .config_paths import build_modality
+from .config_paths import build_modality, parse_modality_path
 from .metrics.utils import convert_planar_to_radial
 
 # ---------------------------------------------------------------------------
@@ -168,6 +168,26 @@ def to_numpy_extrinsics(data: Any) -> np.ndarray:
     return arr.astype(np.float32, copy=False)
 
 
+def compose_sensor_to_camera_extrinsics(
+    source_sensor_extrinsics: Any,
+    camera_extrinsics: Any,
+) -> np.ndarray:
+    """Compose per-sensor poses into a source-to-camera transform.
+
+    Both inputs are interpreted as sensor-to-common-frame transforms, e.g.
+    ``T_common_from_lidar`` and ``T_common_from_camera``.  The returned matrix
+    maps source sensor points directly into the camera frame:
+
+    ``T_camera_from_source = inv(T_common_from_camera) @ T_common_from_source``.
+    """
+    source_to_common = to_numpy_extrinsics(source_sensor_extrinsics).astype(
+        np.float64
+    )
+    camera_to_common = to_numpy_extrinsics(camera_extrinsics).astype(np.float64)
+    common_to_camera = np.linalg.inv(camera_to_common)
+    return (common_to_camera @ source_to_common).astype(np.float32)
+
+
 def to_numpy_point_cloud(data: Any) -> np.ndarray:
     """Convert point-cloud data to ``(N, C)`` float32 numpy array.
 
@@ -518,7 +538,11 @@ def compute_scale_and_shift(
 # ---------------------------------------------------------------------------
 
 
-def _resolve_sky_mask_loader(segmentation_path: str) -> Callable[..., Any]:
+def _resolve_sky_mask_loader(
+    segmentation_path: str,
+    *,
+    modality_key: str = "segmentation",
+) -> Callable[..., Any]:
     """Resolve the ``sky_mask`` loader for a segmentation dataset.
 
     The segmentation dataset's ds-crawler index declares a loader module
@@ -526,7 +550,7 @@ def _resolve_sky_mask_loader(segmentation_path: str) -> Callable[..., Any]:
     its ``sky_mask`` function so that raw class-segmentation data is
     converted to a boolean sky mask at load time.
     """
-    modality = _modality(path=segmentation_path, modality_key="segmentation")
+    modality = _modality(path=segmentation_path, modality_key=modality_key)
     try:
         index = index_dataset_from_path(
             modality.path,
@@ -596,6 +620,7 @@ def build_depth_eval_dataset(
     calibration_split: Optional[str] = None,
     segmentation_split: Optional[str] = None,
     pred_depth_metadata_scope: Optional[str] = None,
+    segmentation_modality_key: str = "segmentation",
 ) -> MultiModalDataset:
     """Build a MultiModalDataset for depth evaluation.
 
@@ -644,10 +669,13 @@ def build_depth_eval_dataset(
             split=calibration_split,
         )
     if segmentation_path is not None:
-        sky_fn = _resolve_sky_mask_loader(segmentation_path)
+        sky_fn = _resolve_sky_mask_loader(
+            segmentation_path,
+            modality_key=segmentation_modality_key,
+        )
         hierarchical["segmentation"] = _modality(
             path=segmentation_path,
-            modality_key="segmentation",
+            modality_key=segmentation_modality_key,
             loader=sky_fn,
             split=segmentation_split,
         )
@@ -663,22 +691,31 @@ def build_sparse_depth_eval_dataset(
     pred_depth_path: str,
     intrinsics_path: str,
     camera_extrinsics_path: str,
+    lidar_extrinsics_path: Optional[str] = None,
     segmentation_path: Optional[str] = None,
     gt_sparse_depth_split: Optional[str] = None,
     pred_depth_split: Optional[str] = None,
     intrinsics_split: Optional[str] = None,
     camera_extrinsics_split: Optional[str] = None,
+    lidar_extrinsics_split: Optional[str] = None,
     segmentation_split: Optional[str] = None,
     pred_depth_metadata_scope: Optional[str] = None,
+    segmentation_modality_key: str = "segmentation",
 ) -> MultiModalDataset:
     """Build a MultiModalDataset for sparse pointcloud depth evaluation.
 
     The returned dataset yields dense predicted depth under ``"pred"``,
     sparse GT point clouds under ``"gt"``, and hierarchical camera
     calibration under ``"intrinsics"`` and ``"camera_extrinsics"``.
+    When ``lidar_extrinsics_path`` is supplied, ``camera_extrinsics`` and
+    ``lidar_extrinsics`` are treated as per-sensor poses in a shared frame and
+    composed into the source-to-camera transform used for projection.  Without
+    ``lidar_extrinsics_path``, ``camera_extrinsics`` remains the direct
+    source-to-camera transform, such as MUSES ``lidar2rgb``.
     ``pred_depth_metadata_scope`` can select a depth-like prediction scope
     such as ``relative_depth`` or ``affine_depth`` while still loading it
-    as a dense depth map.
+    as a dense depth map. Segmentation remains optional for sparse depth and is
+    only loaded by callers that request sky masking.
     """
     modalities = {
         "gt": _modality(
@@ -707,11 +744,28 @@ def build_sparse_depth_eval_dataset(
             split=camera_extrinsics_split,
         ),
     }
+    if lidar_extrinsics_path is not None:
+        lidar_scope = (
+            parse_modality_path(
+                lidar_extrinsics_path,
+                split=lidar_extrinsics_split,
+            ).metadata_scope
+            or "lidar_extrinsics"
+        )
+        hierarchical["lidar_extrinsics"] = _modality(
+            path=lidar_extrinsics_path,
+            modality_key="camera_extrinsics",
+            metadata_scope=lidar_scope,
+            split=lidar_extrinsics_split,
+        )
     if segmentation_path is not None:
-        sky_fn = _resolve_sky_mask_loader(segmentation_path)
+        sky_fn = _resolve_sky_mask_loader(
+            segmentation_path,
+            modality_key=segmentation_modality_key,
+        )
         hierarchical["segmentation"] = _modality(
             path=segmentation_path,
-            modality_key="segmentation",
+            modality_key=segmentation_modality_key,
             loader=sky_fn,
             split=segmentation_split,
         )
@@ -733,6 +787,7 @@ def build_rgb_eval_dataset(
     gt_depth_split: Optional[str] = None,
     calibration_split: Optional[str] = None,
     segmentation_split: Optional[str] = None,
+    segmentation_modality_key: str = "segmentation",
 ) -> MultiModalDataset:
     """Build a MultiModalDataset for RGB evaluation.
 
@@ -785,10 +840,13 @@ def build_rgb_eval_dataset(
             split=calibration_split,
         )
     if segmentation_path is not None:
-        sky_fn = _resolve_sky_mask_loader(segmentation_path)
+        sky_fn = _resolve_sky_mask_loader(
+            segmentation_path,
+            modality_key=segmentation_modality_key,
+        )
         hierarchical["segmentation"] = _modality(
             path=segmentation_path,
-            modality_key="segmentation",
+            modality_key=segmentation_modality_key,
             loader=sky_fn,
             split=segmentation_split,
         )

{euler_eval-2.12.0 → euler_eval-2.13.0}/euler_eval/evaluate.py

@@ -5,11 +5,11 @@ Runs all metrics over depth and RGB datasets loaded via euler_loading.
 
 import copy
 import tempfile
+from collections.abc import Mapping
 from pathlib import Path
+from typing import Iterator, Optional, Tuple
 
 import numpy as np
-import torch
-from typing import Iterator, Optional, Tuple
 from PIL import Image
 from torch.utils.data import DataLoader, Subset
 from tqdm import tqdm
@@ -19,6 +19,7 @@ from euler_loading import MultiModalDataset
 from .data import (
     align_to_prediction,
     classify_spatial_alignment,
+    compose_sensor_to_camera_extrinsics,
     compute_scale_and_shift,
     process_depth,
     project_point_cloud_to_depth_map,
@@ -499,6 +500,15 @@ def _get_first_hierarchical_value(sample: dict, key: str):
     return data
 
 
+def _iter_hierarchical_values(sample: dict, key: str) -> list:
+    data = sample.get(key)
+    if data is None:
+        return []
+    if isinstance(data, dict):
+        return [data, *data.values()]
+    return [data]
+
+
 def _get_intrinsics_K(sample: dict) -> Optional[np.ndarray]:
     """Extract (3,3) intrinsics matrix from sample calibration data."""
     K_data = _get_first_hierarchical_value(sample, "intrinsics")
@@ -509,14 +519,197 @@ def _get_intrinsics_K(sample: dict) -> Optional[np.ndarray]:
     return to_numpy_intrinsics(K_data)
 
 
-def _get_camera_extrinsics(sample: dict) -> Optional[np.ndarray]:
-    """Extract a source-to-camera extrinsics matrix from a sample."""
-    T_data = _get_first_hierarchical_value(sample, "camera_extrinsics")
-    if T_data is None:
-        T_data = _get_first_hierarchical_value(sample, "extrinsics")
-    if T_data is None:
+_DIRECT_SOURCE_TO_CAMERA_KEYS = (
+    "lidar2rgb",
+    "lidar2camera",
+    "lidar2cam",
+    "lidar_to_rgb",
+    "lidar_to_camera",
+    "lidar_to_cam",
+    "source2rgb",
+    "source2camera",
+    "source_to_rgb",
+    "source_to_camera",
+    "sensor2rgb",
+    "sensor2camera",
+    "sensor_to_rgb",
+    "sensor_to_camera",
+)
+_SOURCE_SENSOR_POSE_KEYS = (
+    "lidar",
+    "lidar_pose",
+    "lidar_extrinsics",
+    "source",
+    "source_sensor",
+    "source_extrinsics",
+    "sensor",
+    "sensor_extrinsics",
+    "lidar2gnss",
+    "lidar2ego",
+    "lidar2vehicle",
+    "lidar2world",
+    "lidar_to_gnss",
+    "lidar_to_ego",
+    "lidar_to_vehicle",
+    "lidar_to_world",
+)
+_CAMERA_SENSOR_POSE_KEYS = (
+    "rgb",
+    "camera",
+    "cam",
+    "frame_camera",
+    "rgb_pose",
+    "camera_pose",
+    "camera_extrinsics",
+    "target",
+    "target_sensor",
+    "target_extrinsics",
+    "rgb2gnss",
+    "camera2gnss",
+    "cam2gnss",
+    "frame_camera2gnss",
+    "rgb2ego",
+    "camera2ego",
+    "cam2ego",
+    "frame_camera2ego",
+    "rgb2vehicle",
+    "camera2vehicle",
+    "cam2vehicle",
+    "frame_camera2vehicle",
+    "rgb2world",
+    "camera2world",
+    "cam2world",
+    "frame_camera2world",
+)
+_MATRIX_WRAPPER_KEYS = ("T", "transform", "matrix", "extrinsics", "pose")
+
+
+def _as_extrinsics_or_none(data) -> Optional[np.ndarray]:
+    try:
+        return to_numpy_extrinsics(data)
+    except (TypeError, ValueError):
+        return None
+
+
+def _extract_wrapped_extrinsics(data) -> Optional[np.ndarray]:
+    matrix = _as_extrinsics_or_none(data)
+    if matrix is not None:
+        return matrix
+
+    if not isinstance(data, Mapping):
+        return None
+
+    for key in _MATRIX_WRAPPER_KEYS:
+        if key in data:
+            matrix = _extract_wrapped_extrinsics(data[key])
+            if matrix is not None:
+                return matrix
+
+    if len(data) == 1:
+        return _extract_wrapped_extrinsics(next(iter(data.values())))
+
+    return None
+
+
+def _extract_named_extrinsics(data, names: tuple[str, ...]) -> Optional[np.ndarray]:
+    if not isinstance(data, Mapping):
+        return _extract_wrapped_extrinsics(data)
+
+    for name in names:
+        if name in data:
+            matrix = _extract_wrapped_extrinsics(data[name])
+            if matrix is not None:
+                return matrix
+
+    for key in ("extrinsics", "sensors", "calibration", "poses"):
+        if key in data and isinstance(data[key], Mapping):
+            matrix = _extract_named_extrinsics(data[key], names)
+            if matrix is not None:
+                return matrix
+
+    if len(data) == 1:
+        return _extract_named_extrinsics(next(iter(data.values())), names)
+
+    return None
+
+
+def _extract_direct_source_to_camera(data) -> Optional[np.ndarray]:
+    matrix = _as_extrinsics_or_none(data)
+    if matrix is not None:
+        return matrix
+
+    if not isinstance(data, Mapping):
         return None
-    return to_numpy_extrinsics(T_data)
+
+    for key in _DIRECT_SOURCE_TO_CAMERA_KEYS:
+        if key in data:
+            matrix = _extract_wrapped_extrinsics(data[key])
+            if matrix is not None:
+                return matrix
+
+    for key in ("extrinsics", "calibration"):
+        if key in data and isinstance(data[key], Mapping):
+            matrix = _extract_direct_source_to_camera(data[key])
+            if matrix is not None:
+                return matrix
+
+    if len(data) == 1:
+        return _extract_direct_source_to_camera(next(iter(data.values())))
+
+    return None
+
+
+def _get_pointcloud_to_camera_extrinsics(
+    sample: dict,
+) -> tuple[Optional[np.ndarray], Optional[str]]:
+    """Extract or compose the transform used for sparse point projection.
+
+    The common MUSES path supplies a direct ``lidar2rgb`` transform via
+    ``camera_extrinsics``.  Some datasets instead expose separate lidar and
+    camera sensor poses in a shared frame; in that case we compose them to the
+    same source-to-camera transform expected by the projector.
+    """
+    lidar_pose = None
+    for value in _iter_hierarchical_values(sample, "lidar_extrinsics"):
+        lidar_pose = _extract_named_extrinsics(value, _SOURCE_SENSOR_POSE_KEYS)
+        if lidar_pose is not None:
+            break
+
+    camera_values = _iter_hierarchical_values(sample, "camera_extrinsics")
+    if not camera_values:
+        camera_values = _iter_hierarchical_values(sample, "extrinsics")
+
+    if lidar_pose is not None:
+        for value in camera_values:
+            camera_pose = _extract_named_extrinsics(value, _CAMERA_SENSOR_POSE_KEYS)
+            if camera_pose is not None:
+                return (
+                    compose_sensor_to_camera_extrinsics(lidar_pose, camera_pose),
+                    "composed_lidar_and_camera_sensor_poses",
+                )
+        return None, None
+
+    for value in camera_values:
+        direct = _extract_direct_source_to_camera(value)
+        if direct is not None:
+            return direct, "direct_source_to_camera"
+
+    for value in camera_values:
+        lidar_pose = _extract_named_extrinsics(value, _SOURCE_SENSOR_POSE_KEYS)
+        camera_pose = _extract_named_extrinsics(value, _CAMERA_SENSOR_POSE_KEYS)
+        if lidar_pose is not None and camera_pose is not None:
+            return (
+                compose_sensor_to_camera_extrinsics(lidar_pose, camera_pose),
+                "composed_lidar_and_camera_sensor_poses",
+            )
+
+    return None, None
+
+
+def _get_camera_extrinsics(sample: dict) -> Optional[np.ndarray]:
+    """Backward-compatible wrapper for direct sparse projection callers."""
+    transform, _ = _get_pointcloud_to_camera_extrinsics(sample)
+    return transform
 
 
 # ---------------------------------------------------------------------------
@@ -1658,6 +1851,7 @@ def evaluate_sparse_depth_samples(
     total_input_points = 0
     total_projected_pixels = 0
     total_evaluated_pixels = 0
+    projection_extrinsics_source: Optional[str] = None
 
     if alignment_mode == "none":
         print("Sparse depth alignment mode: none")
@@ -1711,15 +1905,22 @@ def evaluate_sparse_depth_samples(
                 point_cloud = to_numpy_point_cloud(sample["gt"])
                 depth_pred = to_numpy_depth(sample["pred"])
                 intrinsics_K = _get_intrinsics_K(sample)
-                camera_extrinsics = _get_camera_extrinsics(sample)
+                camera_extrinsics, extrinsics_source = (
+                    _get_pointcloud_to_camera_extrinsics(sample)
+                )
                 if intrinsics_K is None:
                     raise ValueError(
                         f"Sample {entry_id!r} is missing intrinsics for sparse depth projection."
                     )
                 if camera_extrinsics is None:
                     raise ValueError(
-                        f"Sample {entry_id!r} is missing camera_extrinsics for sparse depth projection."
+                        f"Sample {entry_id!r} is missing usable camera_extrinsics "
+                        "for sparse depth projection. Provide a direct "
+                        "source-to-camera transform (for MUSES, lidar2rgb) or "
+                        "separate lidar_extrinsics and camera_extrinsics sensor poses."
                     )
+                if projection_extrinsics_source is None:
+                    projection_extrinsics_source = extrinsics_source
 
                 if i == 0:
                     pred_native_dims = depth_pred.shape[:2]
@@ -2015,6 +2216,7 @@ def evaluate_sparse_depth_samples(
                     if pred_native_dims
                     else None,
                     "method": "pointcloud_projection",
+                    "extrinsics_source": projection_extrinsics_source,
                     "evaluated_dimensions": {
                         "height": pred_native_dims[0],
                         "width": pred_native_dims[1],

{euler_eval-2.12.0 → euler_eval-2.13.0}/euler_eval.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-eval
-Version: 2.12.0
+Version: 2.13.0
 Summary: Depth map evaluation toolkit with comprehensive metrics
 Author: Depth Eval Contributors
 License: MIT
@@ -149,7 +149,7 @@ This pre-downloads:
 # Evaluate with default settings (auto-selects CUDA when available)
 depth-eval config.json --batch-size 32
 
-# Evaluate with sky masking enabled (requires gt.segmentation in config)
+# Evaluate with sky masking enabled (requires gt.segmentation or gt.semantic_segmentation)
 depth-eval config.json --mask-sky -v
 
 # Skip RGB evaluation, only evaluate depth
@@ -241,7 +241,13 @@ Defines GT modalities, prediction datasets to evaluate, and optional euler_train
 
 Each modality entry can optionally include a `split` field to select a specific split from the dataset (e.g. `{ "path": "/data/gt/depth", "split": "test" }`). euler-loading inline selectors are also accepted in `path`, such as `/data/muses.zip:test` or `/data/muses.zip:test#scope=rgb`.
 
-For sparse pointcloud depth evaluation, use `gt.sparse_depth` instead of `gt.depth`. The prediction uses a dense depth-like map under `datasets[].depth`, `datasets[].relative_depth`, or `datasets[].affine_depth`. The evaluator projects the sparse GT point cloud into the prediction image plane using `gt.intrinsics` and `gt.camera_extrinsics`, then computes pointwise depth metrics only at projected valid pixels.
+For sparse pointcloud depth evaluation, use `gt.sparse_depth` instead of `gt.depth`. The prediction uses a dense depth-like map under `datasets[].depth`, `datasets[].relative_depth`, or `datasets[].affine_depth`. The evaluator projects the sparse GT point cloud into the prediction image plane using `gt.intrinsics` and `gt.camera_extrinsics`, then computes pointwise depth metrics only at projected valid pixels. For MUSES through `euler_loading.loaders.muses`, `gt.camera_extrinsics` normally resolves to the direct `lidar2rgb` transform and no extra lidar pose is needed. If a dataset exposes separate lidar and camera poses in a shared frame instead, provide optional `gt.lidar_extrinsics`; the evaluator composes `inv(camera_pose) @ lidar_pose` before projection.
+
+Sparse depth does not require segmentation GT. `gt.segmentation` and
+`gt.semantic_segmentation` are optional aliases for the same sky-mask source.
+They are loaded only when `--mask-sky` is set; in sparse depth evaluation that
+mask excludes sky pixels from projected-point metrics and from scale/shift
+fitting. Without `--mask-sky`, the segmentation entry is not loaded or used.
 
 #### GT section
 
@@ -251,10 +257,11 @@ For sparse pointcloud depth evaluation, use `gt.sparse_depth` instead of `gt.dep
 | `gt.depth.path` | no\* | Path to GT depth dataset |
 | `gt.sparse_depth.path` | no\* | Path to sparse pointcloud GT dataset, e.g. `sparse_depth` with `(N,C)` points whose first columns are `x,y,z` in meters |
 | `gt.rays.path` | no\* | Path to GT ray direction map dataset (for rays evaluation) |
-| `gt.segmentation.path` | no | Path to GT segmentation (needed for `--mask-sky`) |
+| `gt.segmentation.path` / `gt.semantic_segmentation.path` | no | Path to GT segmentation (needed for `--mask-sky`; use one key, not both) |
 | `gt.calibration.path` | no | Path to calibration data (camera intrinsics matrices) |
 | `gt.intrinsics.path` | required with `gt.sparse_depth` | Path to camera intrinsics matrices for pointcloud projection |
 | `gt.camera_extrinsics.path` | required with `gt.sparse_depth` | Path to source-to-camera extrinsics, e.g. MUSES `lidar2rgb`, for pointcloud projection |
+| `gt.lidar_extrinsics.path` | no | Optional lidar sensor pose in the same shared frame as `gt.camera_extrinsics`; when provided, `gt.camera_extrinsics` is interpreted as the camera pose and both are composed for projection |
 | `gt.name` | no | Display name for ground truth (default: `"GT"`) |
 
 \* At least one of `gt.rgb.path`, `gt.depth.path`, `gt.sparse_depth.path`, or `gt.rays.path` is required.

{euler_eval-2.12.0 → euler_eval-2.13.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "euler-eval"
-version = "2.12.0"
+version = "2.13.0"
 description = "Depth map evaluation toolkit with comprehensive metrics"
 readme = "README.md"
 requires-python = ">=3.9"

{euler_eval-2.12.0 → euler_eval-2.13.0}/tests/test_config.py

@@ -4,7 +4,12 @@ import json
 import zipfile
 import pytest
 
-from euler_eval.cli import load_config, validate_dataset_entry, validate_gt_config
+from euler_eval.cli import (
+    _gt_segmentation_entry,
+    load_config,
+    validate_dataset_entry,
+    validate_gt_config,
+)
 
 
 # ---------------------------------------------------------------------------
@@ -39,6 +44,36 @@ class TestValidateGtConfig:
         }
         validate_gt_config(gt)
 
+    def test_valid_with_semantic_segmentation_alias(self, tmp_path):
+        """GT config accepts semantic_segmentation as a segmentation alias."""
+        for name in ("depth", "semantic_segmentation"):
+            (tmp_path / name).mkdir()
+
+        gt = {
+            "depth": {"path": str(tmp_path / "depth")},
+            "semantic_segmentation": {
+                "path": str(tmp_path / "semantic_segmentation")
+            },
+        }
+        validate_gt_config(gt)
+        key, entry = _gt_segmentation_entry(gt)
+        assert key == "semantic_segmentation"
+        assert entry == gt["semantic_segmentation"]
+
+    def test_rejects_duplicate_segmentation_aliases(self, tmp_path):
+        for name in ("depth", "segmentation", "semantic_segmentation"):
+            (tmp_path / name).mkdir()
+
+        gt = {
+            "depth": {"path": str(tmp_path / "depth")},
+            "segmentation": {"path": str(tmp_path / "segmentation")},
+            "semantic_segmentation": {
+                "path": str(tmp_path / "semantic_segmentation")
+            },
+        }
+        with pytest.raises(ValueError, match="multiple segmentation entries"):
+            validate_gt_config(gt)
+
     def test_depth_only_gt_is_valid(self, tmp_path):
         depth_path = tmp_path / "depth"
         depth_path.mkdir()
@@ -101,6 +136,22 @@ class TestValidateGtConfig:
         }
         validate_gt_config(gt)
 
+    def test_sparse_depth_gt_accepts_optional_lidar_extrinsics(self, tmp_path):
+        for name in (
+            "sparse_depth",
+            "intrinsics",
+            "camera_extrinsics",
+            "lidar_extrinsics",
+        ):
+            (tmp_path / name).mkdir()
+        gt = {
+            "sparse_depth": {"path": str(tmp_path / "sparse_depth")},
+            "intrinsics": {"path": str(tmp_path / "intrinsics")},
+            "camera_extrinsics": {"path": str(tmp_path / "camera_extrinsics")},
+            "lidar_extrinsics": {"path": str(tmp_path / "lidar_extrinsics")},
+        }
+        validate_gt_config(gt)
+
     def test_nonexistent_path(self, tmp_path):
         rgb_path = tmp_path / "rgb"
         rgb_path.mkdir()

{euler_eval-2.12.0 → euler_eval-2.13.0}/tests/test_data_builders.py

@@ -35,7 +35,7 @@ def test_depth_builder_sets_modality_scopes(monkeypatch):
     monkeypatch.setattr(
         data,
         "_resolve_sky_mask_loader",
-        lambda path: sky_loader,
+        lambda path, *, modality_key="segmentation": sky_loader,
     )
 
     dataset = data.build_depth_eval_dataset(
@@ -100,6 +100,27 @@ def test_sparse_depth_builder_sets_projection_modality_scopes(monkeypatch):
         key="camera_extrinsics",
         split="pose",
     )
+    assert "segmentation" not in dataset.hierarchical_modalities
+
+
+def test_sparse_depth_builder_adds_optional_lidar_extrinsics(monkeypatch):
+    _install_captured_dataset(monkeypatch)
+
+    dataset = data.build_sparse_depth_eval_dataset(
+        gt_sparse_depth_path="/datasets/shared",
+        pred_depth_path="/predictions/shared",
+        intrinsics_path="/datasets/shared",
+        camera_extrinsics_path="/datasets/shared",
+        lidar_extrinsics_path="/datasets/shared",
+        lidar_extrinsics_split="pose",
+    )
+
+    _assert_modality(
+        dataset.hierarchical_modalities["lidar_extrinsics"],
+        key="camera_extrinsics",
+        split="pose",
+        scope="lidar_extrinsics",
+    )
 
 
 def test_sparse_depth_builder_can_load_relative_depth_prediction_scope(monkeypatch):
@@ -121,6 +142,40 @@ def test_sparse_depth_builder_can_load_relative_depth_prediction_scope(monkeypat
     )
 
 
+def test_sparse_depth_builder_uses_semantic_segmentation_scope(monkeypatch):
+    _install_captured_dataset(monkeypatch)
+    calls = []
+
+    def sky_loader(path, meta=None):
+        return None
+
+    def resolve_sky_mask_loader(path, *, modality_key="segmentation"):
+        calls.append((path, modality_key))
+        return sky_loader
+
+    monkeypatch.setattr(
+        data,
+        "_resolve_sky_mask_loader",
+        resolve_sky_mask_loader,
+    )
+
+    dataset = data.build_sparse_depth_eval_dataset(
+        gt_sparse_depth_path="/datasets/shared",
+        pred_depth_path="/predictions/shared",
+        intrinsics_path="/datasets/shared",
+        camera_extrinsics_path="/datasets/shared",
+        segmentation_path="/datasets/shared",
+        segmentation_modality_key="semantic_segmentation",
+    )
+
+    assert calls == [("/datasets/shared", "semantic_segmentation")]
+    _assert_modality(
+        dataset.hierarchical_modalities["segmentation"],
+        key="semantic_segmentation",
+        loader=sky_loader,
+    )
+
+
 def test_rgb_builder_sets_rgb_and_auxiliary_modality_scopes(monkeypatch):
     _install_captured_dataset(monkeypatch)
 
@@ -215,6 +270,35 @@ def test_sky_mask_loader_resolution_strips_inline_split(monkeypatch):
     ]
 
 
+def test_sky_mask_loader_resolution_accepts_semantic_segmentation_scope(monkeypatch):
+    calls = []
+
+    def fake_index_dataset_from_path(path, **kwargs):
+        calls.append((path, kwargs))
+        return {"euler_loading": {"loader": "muses"}}
+
+    def sky_mask(path, meta=None):
+        return None
+
+    monkeypatch.setattr(data, "index_dataset_from_path", fake_index_dataset_from_path)
+    monkeypatch.setattr(
+        data,
+        "resolve_loader_module",
+        lambda name: SimpleNamespace(sky_mask=sky_mask),
+    )
+
+    assert (
+        data._resolve_sky_mask_loader(
+            "/datasets/shared",
+            modality_key="semantic_segmentation",
+        )
+        is sky_mask
+    )
+    assert calls == [
+        ("/datasets/shared", {"metadata_scope": "semantic_segmentation"}),
+    ]
+
+
 def test_modality_parses_inline_split_and_scope():
     modality = data._modality(
         path="/datasets/frame_camera_trainvaltest.zip:fog_day#scope=intrinsics",

{euler_eval-2.12.0 → euler_eval-2.13.0}/tests/test_sparse_depth.py

@@ -3,6 +3,7 @@
 import numpy as np
 
 from euler_eval.data import (
+    compose_sensor_to_camera_extrinsics,
     project_point_cloud_to_depth_map,
     to_numpy_extrinsics,
     to_numpy_point_cloud,
@@ -29,6 +30,22 @@ def test_to_numpy_extrinsics_expands_3x4_matrix():
     np.testing.assert_array_equal(result[3], np.array([0.0, 0.0, 0.0, 1.0]))
 
 
+def test_compose_sensor_to_camera_extrinsics_uses_lidar_and_camera_poses():
+    lidar_to_world = np.eye(4, dtype=np.float32)
+    lidar_to_world[0, 3] = 12.0
+    camera_to_world = np.eye(4, dtype=np.float32)
+    camera_to_world[0, 3] = 10.0
+
+    result = compose_sensor_to_camera_extrinsics(
+        lidar_to_world,
+        camera_to_world,
+    )
+
+    expected = np.eye(4, dtype=np.float32)
+    expected[0, 3] = 2.0
+    np.testing.assert_allclose(result, expected)
+
+
 def test_project_point_cloud_to_depth_map_uses_nearest_point_per_pixel():
     intrinsics = np.eye(3, dtype=np.float32)
     extrinsics = np.eye(4, dtype=np.float32)
@@ -115,6 +132,52 @@ class _AffineSparseDataset(_OneSampleSparseDataset):
         self.pred[1, 2] = np.sqrt(24.0) * 10.0 + 7.0
 
 
+class _SeparateSensorPoseSparseDataset(_OneSampleSparseDataset):
+    def __init__(self):
+        super().__init__()
+        self.point_cloud = np.array([[0.0, 1.0, 1.0, 0.0]], dtype=np.float32)
+        self.camera_extrinsics = np.eye(4, dtype=np.float32)
+        self.camera_extrinsics[0, 3] = 10.0
+        self.lidar_extrinsics = np.eye(4, dtype=np.float32)
+        self.lidar_extrinsics[0, 3] = 12.0
+        self.pred = np.zeros((3, 3), dtype=np.float32)
+        self.pred[1, 2] = np.sqrt(6.0)
+
+    def __getitem__(self, index):
+        if index != 0:
+            raise IndexError(index)
+        return {
+            "id": "000000",
+            "full_id": "/000000",
+            "gt": self.point_cloud,
+            "pred": self.pred,
+            "intrinsics": {"intrinsics": self.intrinsics},
+            "camera_extrinsics": {"camera_pose": self.camera_extrinsics},
+            "lidar_extrinsics": {"lidar_pose": self.lidar_extrinsics},
+        }
+
+
+class _SharedCalibrationSparseDataset(_SeparateSensorPoseSparseDataset):
+    def __getitem__(self, index):
+        if index != 0:
+            raise IndexError(index)
+        return {
+            "id": "000000",
+            "full_id": "/000000",
+            "gt": self.point_cloud,
+            "pred": self.pred,
+            "intrinsics": {"intrinsics": self.intrinsics},
+            "camera_extrinsics": {
+                "calib": {
+                    "extrinsics": {
+                        "lidar2ego": self.lidar_extrinsics,
+                        "rgb2ego": self.camera_extrinsics,
+                    }
+                }
+            },
+        }
+
+
 def test_sparse_depth_eval_reports_only_pointwise_depth_metrics():
     dataset = _OneSampleSparseDataset()
 
@@ -135,6 +198,47 @@ def test_sparse_depth_eval_reports_only_pointwise_depth_metrics():
     np.testing.assert_allclose(metrics["depth_metrics"]["rmse"]["median"], 0.0)
 
 
+def test_sparse_depth_eval_composes_separate_lidar_and_camera_poses():
+    result = evaluate_sparse_depth_samples(
+        _SeparateSensorPoseSparseDataset(),
+        pred_is_radial=True,
+        num_workers=0,
+        alignment_mode="none",
+    )
+
+    assert result["dataset_info"]["projected_pixels"] == 1
+    assert result["dataset_info"]["evaluated_pixels"] == 1
+    assert (
+        result["spatial_info"]["extrinsics_source"]
+        == "composed_lidar_and_camera_sensor_poses"
+    )
+    np.testing.assert_allclose(
+        result["sparse_depth_metric"]["depth_metrics"]["rmse"]["median"],
+        0.0,
+        atol=1e-6,
+    )
+
+
+def test_sparse_depth_eval_composes_sensor_poses_from_shared_calibration():
+    result = evaluate_sparse_depth_samples(
+        _SharedCalibrationSparseDataset(),
+        pred_is_radial=True,
+        num_workers=0,
+        alignment_mode="none",
+    )
+
+    assert result["dataset_info"]["projected_pixels"] == 1
+    assert (
+        result["spatial_info"]["extrinsics_source"]
+        == "composed_lidar_and_camera_sensor_poses"
+    )
+    np.testing.assert_allclose(
+        result["sparse_depth_metric"]["depth_metrics"]["rmse"]["median"],
+        0.0,
+        atol=1e-6,
+    )
+
+
 def test_sparse_depth_eval_auto_affine_aligns_relative_depth():
     result = evaluate_sparse_depth_samples(
         _RelativeSparseDataset(),