euler-eval 2.20.0__tar.gz → 2.22.0__tar.gz

{euler_eval-2.20.0 → euler_eval-2.22.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-eval
-Version: 2.20.0
+Version: 2.22.0
 Summary: Depth map evaluation toolkit with comprehensive metrics
 Author: Depth Eval Contributors
 License: MIT
@@ -493,7 +493,7 @@ a `points_3d` canonical alias, keyed as
 | Euclidean 3D agreement | `point_error.{image_mean,image_median,pixel_pool}.*` | 3D EPE `mae3d`, `rmse3d`, `median3d`/`p90`/`p95`, relative error `rel_median`/`rel_p90`, and 3D δ-accuracy `acc_<τ>` (within τ m) / `acc_rel_<τ>` |
 | Error decomposition | `error_decomposition.{radial_*,lateral_*,lateral_fraction}` + `angular_error.*` + `rho_a.*` | Splits each error vector into a **radial** (≈ depth) and **lateral** (≈ camera-model) component; `lateral_fraction ∈ [0,1]` attributes blame. Angular ray error and ρ_A (on native directions) measure the implied camera model |
 | Geometric | `geometric.normal_consistency.*`, `geometric.point_edge_f1.*` | True-3D surface normals (cross-product of point-map tangents, no focal assumption) and a 3D discontinuity F1 |
-| Cloud distance | `cloud_distance.chamfer.*`, `cloud_distance.fscore.tau_<τ>.*` | Correspondence-free Chamfer (accuracy/completeness/distance/median) and F-score (precision/recall/f1) at distance thresholds τ |
+| Cloud distance | `cloud_distance.chamfer.*`, `cloud_distance.fscore.tau_<τ>.*`, `cloud_distance.f_a` | Correspondence-free Chamfer (accuracy/completeness/distance/median), F-score (precision/recall/f1) at fixed distance thresholds τ, and F-score AUC (`f_a`) integrated up to 1/20 of the dataset's maximum GT depth |
 
 #### Points-3D gauge alignment
 
@@ -607,7 +607,7 @@ For sparse depth outputs, the internal Python result dict still uses `sparse_dep
 
 For points_3d outputs, the internal Python result dict uses `points_3d_native`, `points_3d_metric`, and `points_3d` (canonical alias), but serialized `eval.json` metric paths are rooted at `points3d.eval` (no underscore) to satisfy the namespace first-segment rule, mirroring `sparsedepth`. The `metric` space is emitted only when a gauge alignment is applied; otherwise only `native` is present and aliased.
 
-When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace and are written as `eval.json` beside the selected prediction modality (or to the configured `output_file`). Only the `point_error`, `error_decomposition`, and (directed) `cloud_distance` categories are present; the `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
+When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace and are written as `eval.json` beside the selected prediction modality (or to the configured `output_file`). Only the `point_error`, `error_decomposition`, and `cloud_distance` categories are present. Fixed-threshold cloud diagnostics are directed (completeness/recall), while `cloud_distance.f_a` performs the symmetric comparison required by the F-score AUC definition. The `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
 
 Previous single-depth structure (kept under `depth`) is:
 

{euler_eval-2.20.0 → euler_eval-2.22.0}/README.md

@@ -455,7 +455,7 @@ a `points_3d` canonical alias, keyed as
 | Euclidean 3D agreement | `point_error.{image_mean,image_median,pixel_pool}.*` | 3D EPE `mae3d`, `rmse3d`, `median3d`/`p90`/`p95`, relative error `rel_median`/`rel_p90`, and 3D δ-accuracy `acc_<τ>` (within τ m) / `acc_rel_<τ>` |
 | Error decomposition | `error_decomposition.{radial_*,lateral_*,lateral_fraction}` + `angular_error.*` + `rho_a.*` | Splits each error vector into a **radial** (≈ depth) and **lateral** (≈ camera-model) component; `lateral_fraction ∈ [0,1]` attributes blame. Angular ray error and ρ_A (on native directions) measure the implied camera model |
 | Geometric | `geometric.normal_consistency.*`, `geometric.point_edge_f1.*` | True-3D surface normals (cross-product of point-map tangents, no focal assumption) and a 3D discontinuity F1 |
-| Cloud distance | `cloud_distance.chamfer.*`, `cloud_distance.fscore.tau_<τ>.*` | Correspondence-free Chamfer (accuracy/completeness/distance/median) and F-score (precision/recall/f1) at distance thresholds τ |
+| Cloud distance | `cloud_distance.chamfer.*`, `cloud_distance.fscore.tau_<τ>.*`, `cloud_distance.f_a` | Correspondence-free Chamfer (accuracy/completeness/distance/median), F-score (precision/recall/f1) at fixed distance thresholds τ, and F-score AUC (`f_a`) integrated up to 1/20 of the dataset's maximum GT depth |
 
 #### Points-3D gauge alignment
 
@@ -569,7 +569,7 @@ For sparse depth outputs, the internal Python result dict still uses `sparse_dep
 
 For points_3d outputs, the internal Python result dict uses `points_3d_native`, `points_3d_metric`, and `points_3d` (canonical alias), but serialized `eval.json` metric paths are rooted at `points3d.eval` (no underscore) to satisfy the namespace first-segment rule, mirroring `sparsedepth`. The `metric` space is emitted only when a gauge alignment is applied; otherwise only `native` is present and aliased.
 
-When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace and are written as `eval.json` beside the selected prediction modality (or to the configured `output_file`). Only the `point_error`, `error_decomposition`, and (directed) `cloud_distance` categories are present; the `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
+When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace and are written as `eval.json` beside the selected prediction modality (or to the configured `output_file`). Only the `point_error`, `error_decomposition`, and `cloud_distance` categories are present. Fixed-threshold cloud diagnostics are directed (completeness/recall), while `cloud_distance.f_a` performs the symmetric comparison required by the F-score AUC definition. The `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
 
 Previous single-depth structure (kept under `depth`) is:
 

{euler_eval-2.20.0 → euler_eval-2.22.0}/euler_eval/cli.py

@@ -8,6 +8,7 @@ import argparse
 import importlib.metadata
 import json
 import math
+import platform
 import sys
 import zipfile
 from pathlib import Path
@@ -503,6 +504,13 @@ def _points_3d_descriptions() -> dict:
         "f1": MetricDescription(
             is_higher_better=True, min_value=0.0, max_value=1.0, display_name="F1"
         ),
+        "f_a": MetricDescription(
+            is_higher_better=True,
+            scale="percentage",
+            min_value=0.0,
+            max_value=100.0,
+            display_name="F-score AUC (F_A)",
+        ),
         # cloud_distance
         "chamfer.accuracy": MetricDescription(
             is_higher_better=False, unit="meters", display_name="Chamfer Accuracy"
@@ -554,6 +562,24 @@ def _get_version() -> str:
         return "0.0.0"
 
 
+def _package_metadata() -> dict:
+    """Return stable package/runtime metadata for evaluation provenance."""
+    return {
+        "name": "euler-eval",
+        "version": _get_version(),
+        "python_version": platform.python_version(),
+        "torch_version": str(torch.__version__),
+        "cuda_version": torch.version.cuda,
+    }
+
+
+def _log_package_metadata() -> dict:
+    """Emit and return the package metadata recorded at workflow startup."""
+    metadata = _package_metadata()
+    print(f"Evaluation package: {json.dumps(metadata, sort_keys=True)}")
+    return metadata
+
+
 def _sparse_depth_metric_set_envelope(
     namespace: _EvalNamespace,
     *,
@@ -1173,6 +1199,9 @@ def main():
 
     args = parser.parse_args()
 
+    # Emit producer provenance before configuration loading or evaluation work.
+    package_metadata = _log_package_metadata()
+
     # Load and validate config
     try:
         config = load_config(args.config)
@@ -1283,7 +1312,12 @@ def main():
 
         # Register evaluation as running before work begins
         if et_run is not None:
-            et_run.add_evaluation(ds_name, name=ds_name, status="running")
+            et_run.add_evaluation(
+                ds_name,
+                name=ds_name,
+                status="running",
+                metadata={"package": package_metadata},
+            )
 
         # -- Depth evaluation --
         depth_dataset = None
@@ -2362,6 +2396,7 @@ def main():
                 # name=ds_name,
                 # status="completed",
                 metadata={
+                    "package": package_metadata,
                     "results": {
                         k: v for k, v in all_results.items() if k != "per_file_metrics"
                     }

{euler_eval-2.20.0 → euler_eval-2.22.0}/euler_eval/evaluate.py

@@ -2854,6 +2854,40 @@ def evaluate_rays_samples(
 # ---------------------------------------------------------------------------
 
 
+def _points_3d_max_gt_depth(
+    dataset: MultiModalDataset,
+    *,
+    gt_is_depth: bool,
+    gt_depth_is_radial: bool,
+) -> float:
+    """Scan the GT once for the dataset maximum radial 3D depth."""
+    max_depth = 0.0
+    for index in range(len(dataset)):
+        sample = dataset[index]
+        if gt_is_depth:
+            intrinsics = _get_intrinsics_K(sample)
+            if intrinsics is None:
+                _, entry_id = _extract_hierarchy(sample)
+                raise ValueError(
+                    f"Sample {entry_id!r}: points_3d GT synthesis from depth "
+                    "requires intrinsics (gt.intrinsics or gt.calibration)."
+                )
+            gt_points = unproject_depth_to_points(
+                to_numpy_depth(sample["gt"]),
+                intrinsics,
+                depth_is_radial=gt_depth_is_radial,
+            )
+        else:
+            gt_points = to_numpy_points_3d(sample["gt"])
+        finite = np.isfinite(gt_points).all(axis=-1)
+        if np.any(finite):
+            depths = np.linalg.norm(gt_points[finite], axis=-1)
+            positive = depths[depths > 0]
+            if positive.size:
+                max_depth = max(max_depth, float(np.max(positive)))
+    return max_depth
+
+
 def evaluate_points_3d_samples(
     dataset: MultiModalDataset,
     fov_domain: Optional[str] = None,
@@ -2922,6 +2956,14 @@ def evaluate_points_3d_samples(
     if num_samples == 0:
         raise ValueError("Dataset has no matched samples")
 
+    print("Scanning points_3d GT depth range for F_A...")
+    dataset_max_depth = _points_3d_max_gt_depth(
+        dataset,
+        gt_is_depth=gt_is_depth,
+        gt_depth_is_radial=gt_depth_is_radial,
+    )
+    f_a_max_threshold = dataset_max_depth / 20.0
+
     # Resolve the gauge alignment to apply.
     if alignment_mode == "auto":
         resolved_mode = "similarity" if input_space_hint == "relative" else "none"
@@ -3224,7 +3266,10 @@ def evaluate_points_3d_samples(
                         st["edge_images"].append(ef)
 
                         cd = compute_cloud_distance_metrics(
-                            pv_sp, gv, max_points=max_cloud_points
+                            pv_sp,
+                            gv,
+                            max_points=max_cloud_points,
+                            fscore_auc_max_threshold=f_a_max_threshold,
                         )
                         if cd:
                             st["cloud_images"].append(cd)
@@ -3325,6 +3370,8 @@ def evaluate_points_3d_samples(
                     "threshold_deg": threshold_deg,
                     "evaluated_points": total_evaluated_points,
                     "gt_synthesized_from_depth": gt_is_depth,
+                    "max_depth": dataset_max_depth,
+                    "f_a_max_threshold": f_a_max_threshold,
                 },
                 "space_info": {
                     "input_space_detected": input_space_detected,
@@ -3405,9 +3452,11 @@ def evaluate_points_3d_sparse_samples(
 
       - ``cloud_distance`` — the **primary** geometry metric here.  Because the
         GT is sparse and the prediction dense, only the correspondence-free
-        ``gt→pred`` direction is meaningful, so this reports **completeness**
-        (mean/median nearest-predicted distance per GT return) and **recall**
-        at each threshold (see :func:`compute_sparse_cloud_distance_metrics`).
+        ``gt→pred`` direction is used for the fixed-threshold diagnostics, so
+        these report **completeness** (mean/median nearest-predicted distance
+        per GT return) and **recall**.  ``f_a`` additionally performs the
+        symmetric comparison required for F-score AUC (see
+        :func:`compute_sparse_cloud_distance_metrics`).
       - ``point_error`` — Euclidean 3D agreement (EPE/RMSE/percentiles/δ-acc)
         computed at the projected sparse correspondences.
       - ``error_decomposition`` — radial (≈depth) vs lateral (≈camera-model)
@@ -3462,6 +3511,39 @@ def evaluate_points_3d_sparse_samples(
     if num_samples == 0:
         raise ValueError("Dataset has no matched samples")
 
+    print("Scanning sparse points_3d GT depth range for F_A...")
+    dataset_max_depth = 0.0
+    for sample_index in range(num_samples):
+        range_sample = dataset[sample_index]
+        range_cloud = to_numpy_point_cloud(range_sample["gt"])
+        if pred_is_depth:
+            range_shape = to_numpy_depth(range_sample["pred"]).shape[:2]
+        else:
+            range_shape = to_numpy_points_3d(range_sample["pred"]).shape[:2]
+        range_intrinsics = _get_intrinsics_K(range_sample)
+        range_extrinsics, _ = _get_pointcloud_to_camera_extrinsics(range_sample)
+        if range_intrinsics is None or range_extrinsics is None:
+            # The main evaluation loop emits the detailed sample-specific
+            # configuration error; do not duplicate it in this range scan.
+            continue
+        range_map, range_mask, _ = project_point_cloud_to_point_map(
+            range_cloud,
+            range_intrinsics,
+            range_extrinsics,
+            range_shape,
+        )
+        if np.any(range_mask):
+            range_depths = np.linalg.norm(range_map[range_mask], axis=-1)
+            finite_depths = range_depths[
+                np.isfinite(range_depths) & (range_depths > 0)
+            ]
+            if finite_depths.size:
+                dataset_max_depth = max(
+                    dataset_max_depth,
+                    float(np.max(finite_depths)),
+                )
+    f_a_max_threshold = dataset_max_depth / 20.0
+
     resolved_domain: Optional[str] = fov_domain
     threshold_deg: Optional[float] = (
         get_threshold_for_domain(resolved_domain) if resolved_domain else None
@@ -3557,7 +3639,10 @@ def evaluate_points_3d_sparse_samples(
                 store["dec_images"].append(dec)
                 fval["error_decomposition"] = dict(dec)
         cd = compute_sparse_cloud_distance_metrics(
-            pred_map[pred_ok], gt_cloud, max_points=max_cloud_points
+            pred_map[pred_ok],
+            gt_cloud,
+            max_points=max_cloud_points,
+            fscore_auc_max_threshold=f_a_max_threshold,
         )
         if cd:
             store["cloud_images"].append(cd)
@@ -3975,6 +4060,8 @@ def evaluate_points_3d_sparse_samples(
                     "projected_pixels": total_projected_pixels,
                     "evaluated_points": total_evaluated_points,
                     "gt_synthesized_from_depth": False,
+                    "max_depth": dataset_max_depth,
+                    "f_a_max_threshold": f_a_max_threshold,
                 },
                 "space_info": {
                     "input_space_detected": input_space_detected,

{euler_eval-2.20.0 → euler_eval-2.22.0}/euler_eval/metrics/__init__.py

@@ -105,6 +105,7 @@ from .points3d_geometry import (
 )
 from .points3d_cloud import (
     compute_cloud_distance_metrics,
+    compute_fscore_auc,
     compute_sparse_cloud_distance_metrics,
     FSCORE_THRESHOLDS as POINTS3D_FSCORE_THRESHOLDS,
     DEFAULT_MAX_POINTS as POINTS3D_DEFAULT_MAX_POINTS,
@@ -197,6 +198,7 @@ __all__ = [
     "compute_point_normal_angles",
     "compute_point_edge_f1",
     "compute_cloud_distance_metrics",
+    "compute_fscore_auc",
     "compute_sparse_cloud_distance_metrics",
     "POINTS3D_FSCORE_THRESHOLDS",
     "POINTS3D_DEFAULT_MAX_POINTS",

{euler_eval-2.20.0 → euler_eval-2.22.0}/euler_eval/metrics/points3d_cloud.py

@@ -36,11 +36,66 @@ def _subsample(points: np.ndarray, max_points: int) -> np.ndarray:
     return points[idx]
 
 
+def compute_fscore_auc(
+    dist_pred_to_gt: np.ndarray,
+    dist_gt_to_pred: np.ndarray,
+    max_threshold: float,
+) -> Optional[float]:
+    """Return the normalized area under the point-cloud F1 curve.
+
+    ``F_A`` integrates the symmetric nearest-neighbour F1 score over all
+    distance tolerances from zero through ``max_threshold``.  The empirical
+    precision and recall curves are step functions, so this implementation
+    integrates their harmonic mean exactly rather than choosing a sampling
+    resolution.  The result is reported as a percentage in ``[0, 100]``.
+    """
+    pred_dist = np.asarray(dist_pred_to_gt, dtype=np.float64).reshape(-1)
+    gt_dist = np.asarray(dist_gt_to_pred, dtype=np.float64).reshape(-1)
+    if (
+        pred_dist.size == 0
+        or gt_dist.size == 0
+        or not np.isfinite(max_threshold)
+        or max_threshold <= 0
+    ):
+        return None
+
+    pred_dist = np.sort(pred_dist[np.isfinite(pred_dist)])
+    gt_dist = np.sort(gt_dist[np.isfinite(gt_dist)])
+    if pred_dist.size == 0 or gt_dist.size == 0:
+        return None
+
+    events = np.unique(
+        np.concatenate(
+            [
+                np.array([0.0, max_threshold], dtype=np.float64),
+                pred_dist[(pred_dist >= 0.0) & (pred_dist < max_threshold)],
+                gt_dist[(gt_dist >= 0.0) & (gt_dist < max_threshold)],
+            ]
+        )
+    )
+    left = events[:-1]
+    widths = np.diff(events)
+    # Immediately to the right of an event, all distances equal to that event
+    # satisfy the paper's threshold test. Point values at the discontinuities
+    # have zero measure and therefore do not affect the integral.
+    precision = np.searchsorted(pred_dist, left, side="right") / pred_dist.size
+    recall = np.searchsorted(gt_dist, left, side="right") / gt_dist.size
+    denominator = precision + recall
+    f1 = np.divide(
+        2.0 * precision * recall,
+        denominator,
+        out=np.zeros_like(denominator),
+        where=denominator > 0,
+    )
+    return float(np.sum(f1 * widths) / max_threshold * 100.0)
+
+
 def compute_cloud_distance_metrics(
     pred_points: np.ndarray,
     gt_points: np.ndarray,
     max_points: int = DEFAULT_MAX_POINTS,
     thresholds: tuple = FSCORE_THRESHOLDS,
+    fscore_auc_max_threshold: Optional[float] = None,
 ) -> Optional[dict]:
     """Chamfer distance and F-score between two 3D point sets.
 
@@ -49,6 +104,8 @@ def compute_cloud_distance_metrics(
         gt_points: ``(M, 3)`` ground-truth points.
         max_points: Per-cloud subsample cap (``None`` to disable).
         thresholds: F-score distance thresholds in metres.
+        fscore_auc_max_threshold: When provided, also compute ``f_a``, the
+            normalized F1 AUC through this distance threshold, as a percentage.
 
     Returns:
         Dict with a ``chamfer`` block (``accuracy``, ``completeness``,
@@ -98,7 +155,16 @@ def compute_cloud_distance_metrics(
             "f1": f1,
         }
 
-    return {"chamfer": chamfer, "fscore": fscore}
+    result = {"chamfer": chamfer, "fscore": fscore}
+    if fscore_auc_max_threshold is not None:
+        f_a = compute_fscore_auc(
+            dist_pred_to_gt,
+            dist_gt_to_pred,
+            fscore_auc_max_threshold,
+        )
+        if f_a is not None:
+            result["f_a"] = f_a
+    return result
 
 
 def compute_sparse_cloud_distance_metrics(
@@ -106,6 +172,7 @@ def compute_sparse_cloud_distance_metrics(
     gt_points: np.ndarray,
     max_points: int = DEFAULT_MAX_POINTS,
     thresholds: tuple = FSCORE_THRESHOLDS,
+    fscore_auc_max_threshold: Optional[float] = None,
 ) -> Optional[dict]:
     """Directed (GT→pred) cloud agreement for a sparse GT point set.
 
@@ -130,6 +197,9 @@ def compute_sparse_cloud_distance_metrics(
         gt_points: ``(M, 3)`` ground-truth (sparse) points.
         max_points: Per-cloud subsample cap (``None`` to disable).
         thresholds: Recall distance thresholds in metres.
+        fscore_auc_max_threshold: When provided, compute symmetric ``f_a`` as
+            the normalized F1 AUC through this distance threshold.  The fixed
+            threshold metrics remain directed because the GT is sparse.
 
     Returns:
         Dict with a ``chamfer`` block (``completeness``, ``median``) and an
@@ -160,4 +230,18 @@ def compute_sparse_cloud_distance_metrics(
         recall = float(np.mean(dist_gt_to_pred < tau))
         fscore[threshold_key(tau)] = {"recall": recall}
 
-    return {"chamfer": chamfer, "fscore": fscore}
+    result = {"chamfer": chamfer, "fscore": fscore}
+    if fscore_auc_max_threshold is not None:
+        # F_A is defined from symmetric F1, so it additionally needs the
+        # prediction-to-GT distances even though the sparse diagnostic leaves
+        # above intentionally remain completeness/recall-only.
+        tree_gt = cKDTree(gt)
+        dist_pred_to_gt, _ = tree_gt.query(pred, k=1)
+        f_a = compute_fscore_auc(
+            dist_pred_to_gt,
+            dist_gt_to_pred,
+            fscore_auc_max_threshold,
+        )
+        if f_a is not None:
+            result["f_a"] = f_a
+    return result

{euler_eval-2.20.0 → euler_eval-2.22.0}/euler_eval.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-eval
-Version: 2.20.0
+Version: 2.22.0
 Summary: Depth map evaluation toolkit with comprehensive metrics
 Author: Depth Eval Contributors
 License: MIT
@@ -493,7 +493,7 @@ a `points_3d` canonical alias, keyed as
 | Euclidean 3D agreement | `point_error.{image_mean,image_median,pixel_pool}.*` | 3D EPE `mae3d`, `rmse3d`, `median3d`/`p90`/`p95`, relative error `rel_median`/`rel_p90`, and 3D δ-accuracy `acc_<τ>` (within τ m) / `acc_rel_<τ>` |
 | Error decomposition | `error_decomposition.{radial_*,lateral_*,lateral_fraction}` + `angular_error.*` + `rho_a.*` | Splits each error vector into a **radial** (≈ depth) and **lateral** (≈ camera-model) component; `lateral_fraction ∈ [0,1]` attributes blame. Angular ray error and ρ_A (on native directions) measure the implied camera model |
 | Geometric | `geometric.normal_consistency.*`, `geometric.point_edge_f1.*` | True-3D surface normals (cross-product of point-map tangents, no focal assumption) and a 3D discontinuity F1 |
-| Cloud distance | `cloud_distance.chamfer.*`, `cloud_distance.fscore.tau_<τ>.*` | Correspondence-free Chamfer (accuracy/completeness/distance/median) and F-score (precision/recall/f1) at distance thresholds τ |
+| Cloud distance | `cloud_distance.chamfer.*`, `cloud_distance.fscore.tau_<τ>.*`, `cloud_distance.f_a` | Correspondence-free Chamfer (accuracy/completeness/distance/median), F-score (precision/recall/f1) at fixed distance thresholds τ, and F-score AUC (`f_a`) integrated up to 1/20 of the dataset's maximum GT depth |
 
 #### Points-3D gauge alignment
 
@@ -607,7 +607,7 @@ For sparse depth outputs, the internal Python result dict still uses `sparse_dep
 
 For points_3d outputs, the internal Python result dict uses `points_3d_native`, `points_3d_metric`, and `points_3d` (canonical alias), but serialized `eval.json` metric paths are rooted at `points3d.eval` (no underscore) to satisfy the namespace first-segment rule, mirroring `sparsedepth`. The `metric` space is emitted only when a gauge alignment is applied; otherwise only `native` is present and aliased.
 
-When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace and are written as `eval.json` beside the selected prediction modality (or to the configured `output_file`). Only the `point_error`, `error_decomposition`, and (directed) `cloud_distance` categories are present; the `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
+When the points_3d metrics come from a sparse pointcloud GT (`gt.sparse_depth` + a predicted `points_3d` map or a dense depth prediction), they use the same `points3d.eval` namespace and are written as `eval.json` beside the selected prediction modality (or to the configured `output_file`). Only the `point_error`, `error_decomposition`, and `cloud_distance` categories are present. Fixed-threshold cloud diagnostics are directed (completeness/recall), while `cloud_distance.f_a` performs the symmetric comparison required by the F-score AUC definition. The `metric` space is emitted only when the gauge (`--points-3d-alignment` for a point-map prediction, `--depth-alignment` for a depth prediction) calibrates the prediction.
 
 Previous single-depth structure (kept under `depth`) is:
 

{euler_eval-2.20.0 → euler_eval-2.22.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "euler-eval"
-version = "2.20.0"
+version = "2.22.0"
 description = "Depth map evaluation toolkit with comprehensive metrics"
 readme = "README.md"
 requires-python = ">=3.9"

{euler_eval-2.20.0 → euler_eval-2.22.0}/tests/test_meta_output.py

@@ -1,5 +1,6 @@
 """Tests for the structured meta block in eval.json output."""
 
+import json
 import re
 
 from euler_eval.cli import (
@@ -7,6 +8,8 @@ from euler_eval.cli import (
     _clean_per_file_metrics,
     _depth_eval_axes,
     _DEPTH_EVAL_DESCRIPTIONS,
+    _log_package_metadata,
+    _package_metadata,
     _sparse_depth_eval_axes,
     _SPARSE_DEPTH_EVAL_DESCRIPTIONS,
     _SPARSE_DEPTH_METRIC_NAMESPACE,
@@ -26,6 +29,23 @@ _RGB_EVAL_AXES = _rgb_eval_axes()
 _METRIC_NAMESPACE_RE = re.compile(r"^[a-z0-9]+(?:\.[a-z0-9_]+)+$")
 
 
+class TestPackageMetadataLogging:
+    def test_package_metadata_contains_runtime_provenance(self):
+        metadata = _package_metadata()
+        assert metadata["name"] == "euler-eval"
+        assert metadata["version"]
+        assert metadata["python_version"]
+        assert metadata["torch_version"]
+        assert "cuda_version" in metadata
+
+    def test_package_metadata_log_is_structured(self, capsys):
+        metadata = _log_package_metadata()
+        output = capsys.readouterr().out.strip()
+        prefix = "Evaluation package: "
+        assert output.startswith(prefix)
+        assert json.loads(output[len(prefix):]) == metadata
+
+
 def _flatten_numeric_metrics(obj, prefix=""):
     results = []
     for key, value in obj.items():

{euler_eval-2.20.0 → euler_eval-2.22.0}/tests/test_points_3d.py

@@ -22,6 +22,7 @@ from euler_eval.evaluate import evaluate_points_3d_samples
 from euler_eval.metrics import (
     compute_cloud_distance_metrics,
     compute_decomposition_metrics,
+    compute_fscore_auc,
     compute_point_edge_f1,
     compute_point_error_metrics,
     compute_point_normal_angles,
@@ -245,6 +246,12 @@ class TestGeometry:
 
 
 class TestCloud:
+    def test_fscore_auc_extremes(self):
+        perfect = np.zeros(4, dtype=np.float64)
+        missed = np.full(4, 2.0, dtype=np.float64)
+        assert compute_fscore_auc(perfect, perfect, 1.0) == 100.0
+        assert compute_fscore_auc(missed, missed, 1.0) == 0.0
+
     def test_perfect_clouds(self):
         gt, _ = _gt_pointmap()
         pts = gt.reshape(-1, 3)
@@ -252,6 +259,16 @@ class TestCloud:
         assert m["chamfer"]["distance"] < 1e-6
         assert m["fscore"]["tau_0_1"]["f1"] == 1.0
 
+    def test_cloud_f_a_uses_requested_depth_range(self):
+        gt, _ = _gt_pointmap()
+        pts = gt.reshape(-1, 3)
+        m = compute_cloud_distance_metrics(
+            pts,
+            pts,
+            fscore_auc_max_threshold=0.5,
+        )
+        assert m["f_a"] == 100.0
+
     def test_shifted_cloud(self):
         gt, _ = _gt_pointmap()
         pts = gt.reshape(-1, 3)
@@ -288,6 +305,10 @@ class TestEvaluate:
         assert pe["acc_0_05"] == 100.0
         assert result["points_3d"]["error_decomposition"]["rho_a"]["mean"] > 0.95
         assert result["dataset_info"]["fov_domain"] == "sfov"
+        assert result["dataset_info"]["f_a_max_threshold"] == pytest.approx(
+            result["dataset_info"]["max_depth"] / 20.0
+        )
+        assert result["points_3d"]["cloud_distance"]["f_a"] == 100.0
 
     def test_similarity_recovers_relative_prediction(self):
         gt, K = _gt_pointmap()
@@ -304,6 +325,10 @@ class TestEvaluate:
         metric_mae = result["points_3d_metric"]["point_error"]["image_mean"]["mae3d"]
         assert metric_mae < 1e-3
         assert native_mae > metric_mae
+        assert "f_a" in result["points_3d_native"]["cloud_distance"]
+        assert result["points_3d_metric"]["cloud_distance"]["f_a"] == pytest.approx(
+            100.0, abs=1e-3
+        )
         assert result["points_3d"] is result["points_3d_metric"]
 
     def test_auto_with_relative_hint_aligns(self):
@@ -418,6 +443,7 @@ class TestCliNamespace:
         assert envelope["axes"]["space"]["position"] == 0
         assert envelope["axes"]["reduction"]["position"] == 2
         assert "mae3d" in envelope["metricDescriptions"]
+        assert "f_a" in envelope["metricDescriptions"]
 
     def test_real_metric_tree_paths_conform(self):
         """Flattened paths of an actual eval result live under points3d.eval."""

{euler_eval-2.20.0 → euler_eval-2.22.0}/tests/test_points_3d_sparse.py

@@ -132,6 +132,18 @@ class TestSparseCloud:
         for tau_block in m["fscore"].values():
             assert set(tau_block) == {"recall"}
 
+    def test_f_a_is_available_without_publishing_symmetric_diagnostics(self):
+        points = np.random.RandomState(2).randn(100, 3).astype(np.float32)
+        m = compute_sparse_cloud_distance_metrics(
+            points,
+            points,
+            fscore_auc_max_threshold=1.0,
+        )
+        assert m["f_a"] == 100.0
+        assert set(m["chamfer"]) == {"completeness", "median"}
+        for tau_block in m["fscore"].values():
+            assert set(tau_block) == {"recall"}
+
     def test_offset_cloud_increases_completeness(self):
         depth = _dense_depth()
         K = _make_intrinsics(400.0, 400.0, 30.0, 20.0)
@@ -223,6 +235,10 @@ class TestEvaluate:
         assert p["point_error"]["image_mean"]["mae3d"] < 1e-4
         assert p["cloud_distance"]["chamfer"]["completeness"] < 1e-4
         assert p["cloud_distance"]["fscore"]["tau_0_1"]["recall"] == 1.0
+        assert "f_a" in p["cloud_distance"]
+        assert res["dataset_info"]["f_a_max_threshold"] == pytest.approx(
+            res["dataset_info"]["max_depth"] / 20.0
+        )
         assert p["error_decomposition"]["rho_a"]["mean"] > 0.9
         # No dense-neighbourhood geometric metrics on sparse GT.
         assert "geometric" not in p
@@ -393,6 +409,7 @@ class TestEvaluateNativePoints:
         assert p["point_error"]["image_mean"]["mae3d"] < 1e-4
         assert p["cloud_distance"]["chamfer"]["completeness"] < 1e-4
         assert p["cloud_distance"]["fscore"]["tau_0_1"]["recall"] == 1.0
+        assert 0.0 <= p["cloud_distance"]["f_a"] <= 100.0
 
     def test_similarity_recovers_relative_pointmap(self):
         ds, gt_dense, K = self._pointmap_dataset(None)