euler-eval 2.17.0__tar.gz → 2.20.0__tar.gz

{euler_eval-2.17.0 → euler_eval-2.20.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-eval
-Version: 2.17.0
+Version: 2.20.0
 Summary: Depth map evaluation toolkit with comprehensive metrics
 Author: Depth Eval Contributors
 License: MIT
@@ -339,7 +339,7 @@ When the GT is a sparse pointcloud, the evaluator also produces **3D (`points_3d
 - a **predicted point map** (`datasets[].points_3d`, an `(H,W,3)` map) — evaluated *directly* against the sparse GT cloud, with the 3D **similarity** gauge selected by `--points-3d-alignment`; or
 - a **dense depth map** (`datasets[].depth`/`relative_depth`/`affine_depth`) — unprojected with `gt.intrinsics` into a point map, with the depth **affine** gauge selected by `--depth-alignment`.
 
-If a dataset entry provides both, the predicted `points_3d` map is preferred (the depth is still scored by the pointwise sparse-depth metrics). Either way the sparse GT cloud is projected into the prediction plane, and the 3D results are written to a separate `points3d_eval.json` (or `<output_file>_points3d.json`) so they never clobber the sparse-depth `eval.json`.
+If a dataset entry provides both, the predicted `points_3d` map is preferred (the depth is still scored by the pointwise sparse-depth metrics). Either way the sparse GT cloud is projected into the prediction plane, and the 3D results are written as `eval.json` in the selected prediction modality path (or to the configured `output_file`). Give simultaneous evaluations distinct prediction paths: one `output_file` applies to the whole dataset entry.
 
 Sparse depth does not require segmentation GT. `gt.segmentation` and
 `gt.semantic_segmentation` are optional aliases for the same sky-mask source.
@@ -607,7 +607,7 @@ For sparse depth outputs, the internal Python result dict still uses `sparse_dep
 
 For points_3d outputs, the internal Python result dict uses `points_3d_native`, `points_3d_metric`, and `points_3d` (canonical alias), but serialized `eval.json` metric paths are rooted at `points3d.eval` (no underscore) to satisfy the namespace first-segment rule, mirroring `sparsedepth`. The `metric` space is emitted only when a gauge alignment is applied; otherwise only `native` is present and aliased.
 
-When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace but are written to a distinct file — `points3d_eval.json` beside the prediction, or `<output_file>_points3d.json` when `output_file` is set — so they do not overwrite the sparse-depth `eval.json`. Only the `point_error`, `error_decomposition`, and (directed) `cloud_distance` categories are present; the `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
+When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace and are written as `eval.json` beside the selected prediction modality (or to the configured `output_file`). Only the `point_error`, `error_decomposition`, and (directed) `cloud_distance` categories are present; the `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
 
 Previous single-depth structure (kept under `depth`) is:
 

{euler_eval-2.17.0 → euler_eval-2.20.0}/README.md

@@ -301,7 +301,7 @@ When the GT is a sparse pointcloud, the evaluator also produces **3D (`points_3d
 - a **predicted point map** (`datasets[].points_3d`, an `(H,W,3)` map) — evaluated *directly* against the sparse GT cloud, with the 3D **similarity** gauge selected by `--points-3d-alignment`; or
 - a **dense depth map** (`datasets[].depth`/`relative_depth`/`affine_depth`) — unprojected with `gt.intrinsics` into a point map, with the depth **affine** gauge selected by `--depth-alignment`.
 
-If a dataset entry provides both, the predicted `points_3d` map is preferred (the depth is still scored by the pointwise sparse-depth metrics). Either way the sparse GT cloud is projected into the prediction plane, and the 3D results are written to a separate `points3d_eval.json` (or `<output_file>_points3d.json`) so they never clobber the sparse-depth `eval.json`.
+If a dataset entry provides both, the predicted `points_3d` map is preferred (the depth is still scored by the pointwise sparse-depth metrics). Either way the sparse GT cloud is projected into the prediction plane, and the 3D results are written as `eval.json` in the selected prediction modality path (or to the configured `output_file`). Give simultaneous evaluations distinct prediction paths: one `output_file` applies to the whole dataset entry.
 
 Sparse depth does not require segmentation GT. `gt.segmentation` and
 `gt.semantic_segmentation` are optional aliases for the same sky-mask source.
@@ -569,7 +569,7 @@ For sparse depth outputs, the internal Python result dict still uses `sparse_dep
 
 For points_3d outputs, the internal Python result dict uses `points_3d_native`, `points_3d_metric`, and `points_3d` (canonical alias), but serialized `eval.json` metric paths are rooted at `points3d.eval` (no underscore) to satisfy the namespace first-segment rule, mirroring `sparsedepth`. The `metric` space is emitted only when a gauge alignment is applied; otherwise only `native` is present and aliased.
 
-When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace but are written to a distinct file — `points3d_eval.json` beside the prediction, or `<output_file>_points3d.json` when `output_file` is set — so they do not overwrite the sparse-depth `eval.json`. Only the `point_error`, `error_decomposition`, and (directed) `cloud_distance` categories are present; the `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
+When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace and are written as `eval.json` beside the selected prediction modality (or to the configured `output_file`). Only the `point_error`, `error_decomposition`, and (directed) `cloud_distance` categories are present; the `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
 
 Previous single-depth structure (kept under `depth`) is:
 

{euler_eval-2.17.0 → euler_eval-2.20.0}/euler_eval/cli.py

@@ -978,9 +978,6 @@ def save_results(
     results: dict,
     dataset_config: dict,
     modality: str | None = None,
-    *,
-    default_basename: str = "eval.json",
-    output_file_suffix: str | None = None,
 ) -> Path:
     """Save results to output file.
 
@@ -994,14 +991,6 @@ def save_results(
         modality: When set, save to this specific modality's path
             (e.g. ``"depth"`` or ``"rgb"``).  Falls back to the first
             available modality path when *None*.
-        default_basename: File name used when no explicit ``output_file`` is
-            configured (default ``"eval.json"``).  Lets a second modality that
-            shares a prediction path (e.g. sparse ``points_3d`` alongside sparse
-            ``depth``) write to a distinct file instead of clobbering it.
-        output_file_suffix: When an explicit ``output_file`` is configured, this
-            suffix is inserted before its extension (e.g. ``"_points3d"`` turns
-            ``model_a.json`` into ``model_a_points3d.json``) so multiple
-            modalities can honour a single configured path without collision.
 
     Returns:
         Path where results were saved.
@@ -1021,7 +1010,7 @@ def save_results(
                     dataset_config[selected_modality]["path"],
                     split=dataset_config[selected_modality].get("split"),
                 )
-                / default_basename
+                / "eval.json"
             )
         else:
             # Default: save alongside first available modality path
@@ -1032,17 +1021,13 @@ def save_results(
                             dataset_config[mod]["path"],
                             split=dataset_config[mod].get("split"),
                         )
-                        / default_basename
+                        / "eval.json"
                     )
                     break
         if output_file is None:
-            output_file = Path(default_basename)
+            output_file = Path("eval.json")
     else:
         output_file = Path(output_file)
-        if output_file_suffix:
-            output_file = output_file.with_name(
-                f"{output_file.stem}{output_file_suffix}{output_file.suffix}"
-            )
 
     zip_path, internal_name = _find_zip_ancestor(output_file)
     if zip_path is not None and internal_name:
@@ -2360,14 +2345,10 @@ def main():
             )
             print(f"\n  Points-3D results saved to: {points_3d_out}")
         if points_3d_sparse_save:
-            # Distinct file so it never clobbers the sparse-depth eval.json that
-            # shares the same dense-depth prediction path / output_file.
             points_3d_sparse_out = save_results(
                 points_3d_sparse_save,
                 dataset_config,
                 modality="points_3d",
-                default_basename="points3d_eval.json",
-                output_file_suffix="_points3d",
             )
             print(
                 f"\n  Points-3D (sparse GT) results saved to: {points_3d_sparse_out}"

{euler_eval-2.17.0 → euler_eval-2.20.0}/euler_eval/evaluate.py

@@ -3829,6 +3829,15 @@ def evaluate_points_3d_sparse_samples(
                     pred_ok_native = pred_ok_native & sky_valid
                     pred_ok_metric = pred_ok_metric & sky_valid
 
+                if sanity_checker is not None:
+                    # Keep the sanity-check sample accounting consistent with
+                    # the dense points_3d evaluator.  Metric validation below
+                    # may add warnings, but sample counting belongs to input
+                    # validation so each evaluated pair is counted once.
+                    sanity_checker.validate_points_3d_input(
+                        pred_map_native, gt_point_map, entry_id
+                    )
+
                 # Native correspondence mask drives the shared angular metrics.
                 native_corr = gt_valid & pred_ok_native
                 total_evaluated_points += int(native_corr.sum())

{euler_eval-2.17.0 → euler_eval-2.20.0}/euler_eval/sanity_checker.py

@@ -1119,7 +1119,10 @@ class SanityChecker:
             print("-" * 70)
 
             for key, info in depth_report["warnings_by_type"].items():
-                ratio = info["count"] / depth_report["total_samples"]
+                ratio = (
+                    info["count"] / depth_report["total_samples"]
+                    if depth_report["total_samples"] > 0 else 0
+                )
                 print(f"\n  {info['metric']} - {info['type']}:")
                 print(f"    Occurrences: {info['count']}/{depth_report['total_samples']} ({ratio*100:.1f}%)")
                 print(f"    {info['message']}")
@@ -1134,7 +1137,10 @@ class SanityChecker:
             print("-" * 70)
 
             for key, info in rgb_report["warnings_by_type"].items():
-                ratio = info["count"] / rgb_report["total_samples"]
+                ratio = (
+                    info["count"] / rgb_report["total_samples"]
+                    if rgb_report["total_samples"] > 0 else 0
+                )
                 print(f"\n  {info['metric']} - {info['type']}:")
                 print(f"    Occurrences: {info['count']}/{rgb_report['total_samples']} ({ratio*100:.1f}%)")
                 print(f"    {info['message']}")
@@ -1149,7 +1155,10 @@ class SanityChecker:
             print("-" * 70)
 
             for key, info in rays_report["warnings_by_type"].items():
-                ratio = info["count"] / rays_report["total_samples"]
+                ratio = (
+                    info["count"] / rays_report["total_samples"]
+                    if rays_report["total_samples"] > 0 else 0
+                )
                 print(f"\n  {info['metric']} - {info['type']}:")
                 print(f"    Occurrences: {info['count']}/{rays_report['total_samples']} ({ratio*100:.1f}%)")
                 print(f"    {info['message']}")
@@ -1164,7 +1173,10 @@ class SanityChecker:
             print("-" * 70)
 
             for key, info in points_3d_report["warnings_by_type"].items():
-                ratio = info["count"] / points_3d_report["total_samples"]
+                ratio = (
+                    info["count"] / points_3d_report["total_samples"]
+                    if points_3d_report["total_samples"] > 0 else 0
+                )
                 print(f"\n  {info['metric']} - {info['type']}:")
                 print(f"    Occurrences: {info['count']}/{points_3d_report['total_samples']} ({ratio*100:.1f}%)")
                 print(f"    {info['message']}")

{euler_eval-2.17.0 → euler_eval-2.20.0}/euler_eval.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-eval
-Version: 2.17.0
+Version: 2.20.0
 Summary: Depth map evaluation toolkit with comprehensive metrics
 Author: Depth Eval Contributors
 License: MIT
@@ -339,7 +339,7 @@ When the GT is a sparse pointcloud, the evaluator also produces **3D (`points_3d
 - a **predicted point map** (`datasets[].points_3d`, an `(H,W,3)` map) — evaluated *directly* against the sparse GT cloud, with the 3D **similarity** gauge selected by `--points-3d-alignment`; or
 - a **dense depth map** (`datasets[].depth`/`relative_depth`/`affine_depth`) — unprojected with `gt.intrinsics` into a point map, with the depth **affine** gauge selected by `--depth-alignment`.
 
-If a dataset entry provides both, the predicted `points_3d` map is preferred (the depth is still scored by the pointwise sparse-depth metrics). Either way the sparse GT cloud is projected into the prediction plane, and the 3D results are written to a separate `points3d_eval.json` (or `<output_file>_points3d.json`) so they never clobber the sparse-depth `eval.json`.
+If a dataset entry provides both, the predicted `points_3d` map is preferred (the depth is still scored by the pointwise sparse-depth metrics). Either way the sparse GT cloud is projected into the prediction plane, and the 3D results are written as `eval.json` in the selected prediction modality path (or to the configured `output_file`). Give simultaneous evaluations distinct prediction paths: one `output_file` applies to the whole dataset entry.
 
 Sparse depth does not require segmentation GT. `gt.segmentation` and
 `gt.semantic_segmentation` are optional aliases for the same sky-mask source.
@@ -607,7 +607,7 @@ For sparse depth outputs, the internal Python result dict still uses `sparse_dep
 
 For points_3d outputs, the internal Python result dict uses `points_3d_native`, `points_3d_metric`, and `points_3d` (canonical alias), but serialized `eval.json` metric paths are rooted at `points3d.eval` (no underscore) to satisfy the namespace first-segment rule, mirroring `sparsedepth`. The `metric` space is emitted only when a gauge alignment is applied; otherwise only `native` is present and aliased.
 
-When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace but are written to a distinct file — `points3d_eval.json` beside the prediction, or `<output_file>_points3d.json` when `output_file` is set — so they do not overwrite the sparse-depth `eval.json`. Only the `point_error`, `error_decomposition`, and (directed) `cloud_distance` categories are present; the `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
+When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace and are written as `eval.json` beside the selected prediction modality (or to the configured `output_file`). Only the `point_error`, `error_decomposition`, and (directed) `cloud_distance` categories are present; the `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
 
 Previous single-depth structure (kept under `depth`) is:
 

{euler_eval-2.17.0 → euler_eval-2.20.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "euler-eval"
-version = "2.17.0"
+version = "2.20.0"
 description = "Depth map evaluation toolkit with comprehensive metrics"
 readme = "README.md"
 requires-python = ">=3.9"

{euler_eval-2.17.0 → euler_eval-2.20.0}/tests/test_points_3d.py

@@ -485,6 +485,15 @@ class TestSanity:
         assert "camera_dominated_error" in types
         assert "rho_a_too_low" in types
 
+    def test_report_handles_metric_warning_without_counted_input(self, capsys):
+        sc = self._checker()
+        sc.validate_points_3d_metrics(lateral_fraction=0.95, file_id="f0")
+
+        sc.print_report()
+
+        output = capsys.readouterr().out
+        assert "Occurrences: 1/0 (0.0%)" in output
+
     def test_full_report_includes_points_3d(self):
         sc = self._checker()
         assert "points_3d" in sc.get_full_report()

{euler_eval-2.17.0 → euler_eval-2.20.0}/tests/test_points_3d_sparse.py

@@ -7,6 +7,9 @@ prediction and scores it as a 3D point map against a sparse LiDAR cloud
 (``points_3d_metrics_proposal.md`` §4-D).
 """
 
+import json
+import zipfile
+
 import numpy as np
 import pytest
 
@@ -17,6 +20,7 @@ from euler_eval.data import (
 )
 from euler_eval.evaluate import evaluate_points_3d_sparse_samples
 from euler_eval.metrics import compute_sparse_cloud_distance_metrics
+from euler_eval.sanity_checker import SanityChecker
 
 
 def _rot_z(theta):
@@ -88,6 +92,21 @@ def _perfect_dataset(seed=0, **kwargs):
     return _SparsePointDataset(cloud, depth, K, **kwargs), depth, K, cloud
 
 
+def test_sparse_evaluation_counts_sanity_check_samples():
+    dataset, _, _, _ = _perfect_dataset()
+    sanity_checker = SanityChecker()
+
+    evaluate_points_3d_sparse_samples(
+        dataset,
+        pred_is_radial=False,
+        num_workers=0,
+        sanity_checker=sanity_checker,
+        alignment_mode="none",
+    )
+
+    assert sanity_checker.get_points_3d_report()["total_samples"] == 1
+
+
 # ---------------------------------------------------------------------------
 # Directed (sparse) cloud distance
 # ---------------------------------------------------------------------------
@@ -440,39 +459,44 @@ class TestEvaluateNativePoints:
 
 
 # ---------------------------------------------------------------------------
-# Save wiring (distinct output file)
+# Save wiring
 # ---------------------------------------------------------------------------
 
 
 class TestSaveWiring:
-    def test_default_basename_and_suffix(self, tmp_path):
+    def test_points_3d_uses_eval_json_in_its_modality_path(self, tmp_path):
         from euler_eval.cli import save_results
 
         pred_dir = tmp_path / "pred"
         pred_dir.mkdir()
-        cfg = {"name": "m", "depth": {"path": str(pred_dir)}}
+        cfg = {"name": "m", "points_3d": {"path": str(pred_dir)}}
 
-        # Sparse-depth writes eval.json; sparse-points_3d writes points3d_eval.json.
-        depth_out = save_results({"a": 1}, cfg, modality="depth")
         p3d_out = save_results(
-            {"b": 2}, cfg, modality="points_3d",
-            default_basename="points3d_eval.json",
+            {"points3d": {"eval": {}}}, cfg, modality="points_3d"
         )
-        assert depth_out.name == "eval.json"
-        assert p3d_out.name == "points3d_eval.json"
-        assert depth_out != p3d_out
-        assert depth_out.exists() and p3d_out.exists()
 
-    def test_output_file_suffix(self, tmp_path):
+        assert p3d_out == pred_dir / "eval.json"
+        assert p3d_out.exists()
+        assert not (pred_dir / "points3d_eval.json").exists()
+        assert not (pred_dir / "points_3d_eval.json").exists()
+
+    def test_points_3d_writes_eval_json_into_zip(self, tmp_path):
         from euler_eval.cli import save_results
 
-        out = tmp_path / "model_a.json"
-        cfg = {"name": "m", "output_file": str(out),
-               "depth": {"path": str(tmp_path)}}
-        p3d_out = save_results({"b": 2}, cfg, modality="points_3d",
-                               output_file_suffix="_points3d")
-        assert p3d_out.name == "model_a_points3d.json"
-        assert p3d_out != out
+        archive = tmp_path / "predictions.zip"
+        with zipfile.ZipFile(archive, "w") as zf:
+            zf.writestr("points/frame_000.npy", b"points")
+
+        results = {"points3d": {"eval": {"native": {}}}}
+        cfg = {"name": "m", "points_3d": {"path": str(archive)}}
+        p3d_out = save_results(results, cfg, modality="points_3d")
+
+        assert p3d_out == archive / "eval.json"
+        with zipfile.ZipFile(archive) as zf:
+            assert "eval.json" in zf.namelist()
+            assert "points3d_eval.json" not in zf.namelist()
+            assert "points_3d_eval.json" not in zf.namelist()
+            assert json.loads(zf.read("eval.json")) == results
 
 
 # ---------------------------------------------------------------------------