asp-plot 3.2.0__tar.gz → 3.3.0__tar.gz

{asp_plot-3.2.0 → asp_plot-3.3.0}/AGENTS.md

@@ -45,10 +45,13 @@ sphinx-autobuild docs docs/_build/html --open-browser   # or sphinx-build for a
 - **Mars altimetry needs the `*_pts_csv.csv`** (with `PLANET_RAD`), never the `*_topo_csv.csv`: MOLA TOPOGRAPHY is referenced to the oblate areoid while ASP DEMs use the spherical IAU datum — a latitude-dependent offset up to ~10 km that pc_align cannot remove. The loader rejects the topo file with an explanatory error.
 - **ASP's `mapproject` writes no log file**; its command is reconstructed from output GeoTIFF metadata (`mapproject.py`), not parsed from logs like the other tools.
 - **Attribution vs sensor naming is deliberate**: copyright/attribution names the rights-holder (`detect_satellite_attribution()` → `"Vantor"` or `"Airbus DS"`), while readers in the `asp_plot/sensors/` package are named for the satellite family (WorldView, Pleiades). Don't reconcile them into one name (#137).
-- **Package vs CLI naming is a deliberate split** (v2.0.0, #165): the *package* is `asp_plot` (`import asp_plot`, `pip install asp-plot`), the report *command* is `asp_report` (`asp_plot/cli/asp_report.py`). There is no `asp_plot` console script and no alias — don't "fix" either name to match the other. The other four CLIs (`stereo_geom`, `csm_camera_plot`, `request_planetary_altimetry`, `gallery`) are unprefixed and unchanged.
+- **Package vs CLI naming is a deliberate split** (v2.0.0, #165): the *package* is `asp_plot` (`import asp_plot`, `pip install asp-plot`), the report *command* is `asp_report` (`asp_plot/cli/asp_report.py`). There is no `asp_plot` console script and no alias — don't "fix" either name to match the other. The other CLIs (`stereo_geom`, `csm_camera_plot`, `bundle_adjust_cameras`, `request_planetary_altimetry`, `gallery`, `dem_benchmark`) are unprefixed.
 - **There is no "unadjusted" CSM state on disk after a `bundle_adjust` run** — it writes only `*.adjusted_state.json` for the *optimized* cameras, while the originals exist solely as vendor XML. To get the before-camera for `csm_camera_summary_plot()`, re-run with an identity transform: write a 4x4 identity to `identity.txt` and pass `--initial-transform identity.txt --apply-initial-transform-only` with the same images/XMLs into a separate `-o` prefix. That skips iterations, matching, and reports (seconds, not minutes), and the resulting `.adjust` files are exactly zero translation + identity quaternion, so you can verify the cameras really are untouched. Demonstrated in `notebooks/WorldView/worldview_spacenet_ucsd_stereo.ipynb`.
 - **CSM camera *differences* deliberately diverge from ASP's `orbit_plot.py`** (#53): `csm_analysis.read_angles_common_frame()` puts both cameras in one satellite body frame estimated from the *original* ephemeris, instead of estimating a frame per camera from its own positions the way `csm_io.read_angles()` (the ASP mirror) does. A solver perturbs positions *and* resamples the ephemeris finer, and over the resulting ~140 m central-difference baseline a 2 m perturbation tilts a per-camera frame by ~0.8° — which is what made the example figure look full of outliers. Angle diffs are also wrapped to [-180, 180). Don't "re-sync" either back to `read_angles()`; `csm_io.py` stays a verbatim ASP mirror and is not where this belongs.
 - **Airbus DIMAP quaternions are scalar-first** (`Q0` = scalar); they are reordered to the scalar-last `q1..q4` layout the roll/pitch/yaw code expects in `PleiadesMetadata.getAtt_df()`. Don't "fix" the reorder.
+- **The altimetry outlier cut is a 30-NMAD gate plus 3σ; both values are deliberate.** A whole ICESat-2 pass can be cloud (over UCSD one pass was a marine-layer return 150–210 m above ground, 21 % of the sample), which defeats a plain mean ± 3σ cut by inflating σ; the gate removes it. Don't "simplify" the cut to median ± 3 NMAD: on the heavy-tailed residuals a DSM has against ICESat-2 in a city that removes a fifth of the real points and lowers every reported NMAD by a third (`_outlier_mask` in `altimetry_source.py`; the case is described in the benchmark notebook's UCSD section). If a benchmark or report shows an RMSE far above its NMAD, look for a cloud pass in the residual map before touching the DEM.
+- **`asp_plot.sensors.rpc.read_rpc()` refuses any raster that carries a CRS**, on purpose: an RPC describes the raw image grid, and ASP's `mapproject` copies RPC metadata onto its outputs. However, GDAL sometimes writes a lat/lon geotransform from a NITF's corner coordinates when converting it (one of five CORE3D scenes came out that way), and the reader then calls a raw scene "already map-projected". Strip it with `gdal_edit.py -unsetgt -a_srs "" scene.tif`.
+- **SpaceNet CORE3D NITF scenes are JPEG2000, and the ASP 3.8.0-alpha build ships GDAL's JP2OpenJPEG driver as a plugin it never loads**: `gdalinfo` and rasterio both fail with *No JPEG2000 capable driver is available*. `GDAL_DRIVER_PATH=<ASP root>/lib` for one `gdal_translate` to GeoTIFF is enough; the older 3.7.0 build had the driver built in, which is why earlier UCSD runs never hit this.
 - **ASP multiview triangulation of mapprojected images needs `ISISROOT`** (observed with ASP 3.8.0-alpha, non-ISIS `-t pleiades` session): the joint triangulation aborts with an uncatchable `Isis::IException` (`$ISISROOT/IsisPreferences was not found`) that surfaces as a generic "Failed to run"/killed job. Workaround: `export ISISROOT=<ASP install root>` (the release bundles `IsisPreferences` there). Pair runs and raw-image multiview runs are unaffected; full write-up in a PR #155 comment.
 
 ## External Data Sources

{asp_plot-3.2.0 → asp_plot-3.3.0}/ARCHITECTURE.md

@@ -109,7 +109,7 @@ The package is organized by functionality, with each module focused on a specifi
 - Detects map-projection status via `Raster.transform` check
 - For non-mapprojected scenes: match points are overlaid on images using alignment transform matrices (`run-align-{L,R}.txt` loaded via `np.loadtxt`), and disparity plots use pixel-unit scalebar instead of GSD-based. Exception: when the match file is named for the aligned images themselves (`run-L__R.match`, older ASP raw-image runs that write `.exr` alignment matrices), the coordinates are already aligned — detected via `_ip_on_aligned_images()` (match stem's left name == L image stem) — and are only rescaled
 - **Raw interest point overlay** (issue #8): `StereoFiles` also discovers the per-image `.vwip` files (named after the images the match file pairs — `<A>__<B>.match` ↔ `<A>.vwip`/`<prefix>-<B>.vwip` — derived from the match stem, with a `*-L.vwip`/`*-R.vwip` glob fallback when no match file exists; either side may be absent). `get_vwip_df()` parses the binary format (a uint64 count then the same interest point records as `.match`); `plot_match_points()` underlays them in blue beneath the red matches through the same per-side coordinate transform (layers denser than 10,000 points are thinned by seeded random sampling for display; titles report true counts), renders interest-points-only when the match file is missing, and falls back to the previous matches-only figure when the `.vwip` files are gone
-- **Plain-text match files** (issue #147): ASP >= 3.7.0 writes `<A>__<B>.txt` (one `x1 y1 unc1 x2 y2 unc2` line per match) *instead of* `.match` when run with `--matches-as-txt`, so `StereoFiles._find_match_file()` globs both `*.match` and `*__*.txt` (the `__` anchor plus a first-row check — six numeric fields, `_opens_with_match_row()` — keeps logs and alignment matrices out even when the run prefix itself contains `__`), preferring binary when both coexist. `get_match_point_df()` detects the format from the bytes (`_is_text_match_file()`: the binary header's uint64 counts contain NULs, text never does — so a renamed file still parses) and routes text through `_read_text_match_file()` — read directly, uncertainties dropped, never via the `.csv` cache a binary conversion leaves behind — while the binary path keeps its CSV conversion. `.vwip` files stay binary-only: stereo never writes text ones (`ipfind --binary-to-txt` is a manual conversion)
+- **Plain-text match files** (issue #147): ASP >= 3.7.0 writes `<A>__<B>.txt` (one `x1 y1 unc1 x2 y2 unc2` line per match) *instead of* `.match` when run with `--matches-as-txt`, so `StereoFiles._find_match_file()` globs both `*.match` and `*__*.txt` (the `__` anchor plus a first-row check — six numeric fields, `_opens_with_match_row()` — keeps logs and alignment matrices out even when the run prefix itself contains `__`), preferring binary when both coexist. the module-level `read_match_file()` (which `get_match_point_df()` delegates to, so a match file reads without a stereo directory) detects the format from the bytes (`_is_text_match_file()`: the binary header's uint64 counts contain NULs, text never does — so a renamed file still parses) and routes text through `_read_text_match_file()` — read directly, uncertainties dropped, never via the `.csv` cache a binary conversion leaves behind — while the binary path keeps its CSV conversion. `.vwip` files stay binary-only: stereo never writes text ones (`ipfind --binary-to-txt` is a manual conversion)
 - **Multi-view aware** (issue #160): `StereoFiles` resolves each `<prefix>-pairN/` subdirectory into a `PairStereoFiles` (field names mirror the top-level attributes so the per-figure helpers consume either, duck-typed); `plot_match_points()` / `plot_disparity()` render one figure per pair (`<stem>_pairN.png`) and return the saved filename list — `[fig_fn]` for a standard run — mirroring the `stereo_geom_plot()` N-scene contract
 - Detects the imagery rights-holder via the `attribution` attribute (`detect_satellite_attribution`); adds the copyright overlay to optical imagery in `plot_match_points()` and `plot_detailed_hillshade()`
 - `plot_detailed_hillshade()` auto-selects three subset clips from intersection-error variance (low/medium/high) via `_auto_hillshade_clip_offsets()`. Accepts `clip_windows` (DEM-CRS bboxes) + `clip_windows_crs` to pin/replay clips for run-to-run comparison (issue #121); records the boxes it drew on `self.detailed_hillshade_clips`. Out-of-bounds pinned boxes warn and fall back to auto.
@@ -154,21 +154,21 @@ The package is organized by functionality, with each module focused on a specifi
 - New dependency from this subsystem: `pyyaml` (request-metadata YAML); `sliderule>=5.3.0` pinned
 
 **`altimetry_source.py`** - `AltimetrySource` base (issue #140)
-- Shared machinery lifted out of the ICESat-2 and planetary sources so neither re-implements it: `_interp_dem_at_points(dem, points)` (bilinear DEM sampling; returns sampled values **and** the points reprojected into the DEM/working CRS), `_open_dem(dem_fn)`, `_std_outlier_mask(dh, n_sigma)` (n-σ std-from-mean mask, returns `None` on empty/all-NaN or zero spread to signal "do not filter"), and `_write_csv_to_directory(df, filename)` (roots pc_align CSVs at `self.alt.directory`)
+- Shared machinery lifted out of the ICESat-2 and planetary sources so neither re-implements it: `_interp_dem_at_points(dem, points)` (bilinear DEM sampling; returns sampled values **and** the points reprojected into the DEM/working CRS), `_open_dem(dem_fn)`, `_outlier_mask(dh, n_sigma)` (two cuts: gross outliers more than `GROSS_OUTLIER_NMAD` = 30 NMAD from the median are dropped first, then the n-σ std-from-mean cut runs on what survives; returns `None` on empty/all-NaN or zero spread to signal "do not filter". The gate exists because one whole ICESat-2 pass over UCSD was a marine-layer cloud return 150–210 m up — a fifth of the sample — which inflates σ to 70 m so a plain 3σ cut removes nothing; a pure median/NMAD cut was rejected because on the heavy-tailed DSM-vs-ICESat-2 residuals of a city it strips a fifth of the *real* points and lowers every NMAD by a third), and `_write_csv_to_directory(df, filename)` (roots pc_align CSVs at `self.alt.directory`)
 
 **`icesat2_source.py`** - `Icesat2Source(AltimetrySource)` (Earth / ICESat-2)
 - Requests/processes ICESat-2 ATL06-SR via the SlideRule x-series API (`sliderule_api.run("atl03x")`) with lazy init (`_ensure_sliderule()` — connects only when ICESat-2 methods run). SlideRule logging silenced (`verbose=False`, WARNING, explicit filter on `sliderule.session`)
 - `request_atl06sr_multi_processing()` decomposed (#140) into a short request/cache/ingest loop over helpers: `_print_time_filter_summary`, `_build_level_parms`, `_load_cached_atl06sr`, `_params_match_cache`, `_request_atl06sr_level`, `_ingest_atl06sr`. Parquet cache saved to `self.alt.directory` (not CWD); the param comparison strips the `output` key (SlideRule injects a random temp path into `parms` during `run()`) and only the comparison is wrapped in `try/except` so SlideRule API errors propagate
 - Server-side time filtering via `_resolve_time_range(...)`: default `"all"` = full mission (2018-10-14 → today UTC midnight); `"buffered"` cascades explicit `t0`/`t1` > `scene_date` ± `time_buffer_days` > XML metadata ± buffer > `"all"`. `t1` truncated to midnight UTC for stable caching
 - ESA WorldCover sampling: `sample_esa_worldcover()` / `_sample_worldcover_into_gdf()` read 10m values directly from public AWS S3 COGs (`esa-worldcover.s3.amazonaws.com/v200/2021/map/`) via `rasterio` vsicurl, batched per tile, persisted into the parquet cache (much faster than SlideRule's server-side `samples`). `_worldcover_tile_url(lat, lon)` maps a coord to the 3×3° tile URL (flat layout). Anonymous reads (unsigned AWS session) so SSO configs don't crash
-- `filter_outliers(n_sigma=3)`: drops dh beyond `n_sigma × std` from the mean (true std, not NMAD); called automatically by `atl06sr_to_dem_dh()`; pass `None` to skip
+- `filter_outliers(n_sigma=3)`: drops dh beyond `n_sigma × std` from the mean (true std, not NMAD) after the 30-NMAD gross-outlier gate (`_outlier_mask`); called automatically by `atl06sr_to_dem_dh()`; pass `None` to skip. When an aligned DEM is set, `icesat_minus_aligned_dem` is blanked wherever `icesat_minus_dem` is NaN (same in `planetary_to_dem_dh()`): `pc_align`'s translation moves the DEM's holes, so a point that sampled NaN before alignment — and therefore passed the cut untested — must not enter the aligned residuals when it lands on data afterwards
 - `atl06sr_to_dem_dh()`: opens the DEM via `self._open_dem`, samples via `self._interp_dem_at_points`, computes the dh column. Track/segment selection: `_select_best_track()` (most valid dh points), `_find_best_worst_segments()` / `_segment_dict()` / `_resolve_best_track()`
 - **Run-to-run reuse (issue #121)**: `load_atl06sr_from_parquet()` replays the *exact* prior points (bypassing SlideRule), `_restore_request_metadata_from_parquet()` restores `t0`/`t1` for plot titles, `get_altimetry_selections()` returns request params / parquet paths / track / segments. Segments pinned by **absolute `x_atc`** (not km-from-track-start) so they survive a track-start shift when 3σ filtering a different DEM drops a different first point
 - `_extract_scalar()` handles array-valued cells from the x-series API / parquet round-trip; module constant `ICESAT2_MISSION_START`
 
 **`planetary_source.py`** - `PlanetarySource(AltimetrySource)` + `LolaSource` / `MolaSource` (issue #140)
 - Two-step workflow: (1) submit async query via `request_planetary_altimetry` CLI → email with download link, (2) load the CSV. The body subclass is selected by the coordinator at construction, so loading no longer re-detects the body per call
-- `PlanetarySource` (body-agnostic base): the dh + export half — `_load_planetary_csv_common()`, `_build_planetary_gdf()`, `_find_csv_column()`, `_sample_dem_at_planetary_points()` (uses `self._open_dem` / `self._interp_dem_at_points`), `planetary_to_dem_dh()` (uses `self._std_outlier_mask`; when `aligned_dem_fn` is set also samples the aligned DEM so pre/post plots share one sample), `to_csv_for_pc_align_planetary()` (writes `lon, lat, radius_m`). Its `load_planetary_csv()` raises and redirects to ICESat-2 (the Earth case)
+- `PlanetarySource` (body-agnostic base): the dh + export half — `_load_planetary_csv_common()`, `_build_planetary_gdf()`, `_find_csv_column()`, `_sample_dem_at_planetary_points()` (uses `self._open_dem` / `self._interp_dem_at_points`), `planetary_to_dem_dh()` (uses `self._outlier_mask`; when `aligned_dem_fn` is set also samples the aligned DEM so pre/post plots share one sample), `to_csv_for_pc_align_planetary()` (writes `lon, lat, radius_m`). Its `load_planetary_csv()` raises and redirects to ICESat-2 (the Earth case)
 - `LolaSource` (Moon, `instrument = "LOLA"`): prefers `Pt_Radius` (LOLA RDR Point-per-Row CSV, in **km** — auto-detected by magnitude < 10 000, converted to m), falls back to `Topography` (simple-topo CSV, meters). Moon ≈ spherical (~1.4 km variation) so the fallback is fine
 - `MolaSource` (Mars, `instrument = "MOLA"`): reads `PLANET_RAD` from the ODE GDS `*_pts_csv.csv`, `height = PLANET_RAD - 3_396_190`. The `*_topo_csv.csv` (TOPOGRAPHY only) is **rejected** with an explanatory error — TOPOGRAPHY is referenced to the **oblate** MOLA areoid while ASP DEMs use the **spherical** IAU 2000 datum, a latitude-dependent offset up to ~10 km that pc_align can't remove
 - Both loaders store `height` (above the IAU sphere; plot labels + dh stats) and `radius_m` (absolute planetary radius; pc_align) on `planetary_points`
@@ -215,6 +215,14 @@ The package is organized by functionality, with each module focused on a specifi
 - Key method: `plot_gallery()`; static helper `_grid_shape(n, aspect)` picks the most square-in-display grid
 - Not wired into the main `asp_report` PDF report (standalone class + `gallery` CLI)
 
+**`dem_benchmark.py`** - `DEMBenchmark` class (issue #169)
+- Scores *many* DEMs against *one* altimetry sample so scene combinations, processing flows (joint MVS vs. pairwise + `dem_mosaic`) and parameter sweeps are compared on the same points. Replaces the ad hoc loop in the Atlanta MVS notebook: one `Altimetry` per DEM (kept in `self.altimetry[label]` for follow-up per-DEM figures), the same parquet replayed via `load_atl06sr_from_parquet` (or `load_planetary_csv`), the report's WorldCover water filter and outlier cut (30-NMAD gate + 3σ)
+- `run()` → `stats_df`, one row per DEM, columns `STATS_COLUMNS`: coverage (`valid_pct` / `valid_area_km2`) inside a common AOI (default the **intersection** of all DEM extents in the first DEM's CRS, so differing crop windows compare fairly; falls back to own extents with a warning when footprints don't all overlap), IntersectionErr median/NMAD from the `point2dem` sibling raster (`intersection_error_path()`; NaN for mosaics), altimetry-minus-DEM n/median/NMAD/RMSE before and after a per-DEM `pc_align --compute-translation-only` (translation + N-E-D shifts from `Alignment.pc_align_report`), and optional DEM-minus-reference stats via `Raster.compute_difference`
+- Window reads (`_window_for_bounds` / `_read_window`) are downsampled past `MAX_WINDOW_PIXELS` so a 1 m 20k×20k DEM stays at ~16 M samples (valid fraction becomes a subsample estimate; area scales from it)
+- pc_align products and the translated DEM go under `<directory>/dem_benchmark/<label>/` (via `Alignment.apply_dem_translation(output_fn=...)`), never next to the candidate DEMs; an existing `*-transform.txt` there is reused (a re-run makes no pc_align call and works offline); a missing `pc_align` binary (`FileNotFoundError`) degrades to pre-alignment scoring with one warning
+- Figures: `summary_plot()` (rows = DEMs sorted best-first by post-alignment NMAD; panels coverage / IntersectionErr / dh median / dh NMAD as before→after dumbbells; the NMAD panel is anchored at zero and its markers coincide by construction for a translation-only alignment) and `histogram_plot()` (overlaid residual outlines). `label_from_dem_path()` / `parse_dem_specs()` back the CLI's `LABEL=PATH` specs (an ASP `run-DEM.tif` is labelled by its folder)
+- Not wired into the `asp_report` PDF (standalone class + `dem_benchmark` CLI)
+
 ## CLI Tools
 
 All CLI tools are in `asp_plot/cli/` and use Click for argument parsing:
@@ -277,6 +285,11 @@ All CLI tools are in `asp_plot/cli/` and use Click for argument parsing:
 - `--no-hillshade`, `--cmap`, `--downsample`, `--max-filesize-mb`, `--title`, `--output-directory/--output-filename`
 - Saves `<dirname>_gallery.png` into the input directory by default
 
+**`dem_benchmark.py`** - Multi-DEM altimetry benchmark tool (`dem_benchmark` command)
+- Wrapper for `DEMBenchmark`; positional `DEMS` as paths or `LABEL=PATH`
+- `--parquet` (Earth, the ATL06-SR cache) or `--altimetry-csv` (Moon/Mars); `--directory` (where `dem_benchmark/<label>/` products go), `--reference`, `--no-pc-align`, `--own-extent`, `--title`, `--output-directory/--output-filename`
+- Writes the summary figure, a `_histogram.png` twin, and the stats table as `.csv` with the same stem, and prints the table
+
 ## Documentation Website
 
 Documentation is hosted at **https://asp-plot.readthedocs.io** and built with Sphinx + MyST Markdown + sphinx-book-theme. ReadTheDocs builds automatically on push to `main`.
@@ -295,7 +308,7 @@ docs/
   conf.py                     # Sphinx configuration
   index.md                    # Landing page with sphinx-design cards
   installation.md             # conda/pip/source install
-  cli/                        # CLI tool docs (asp_report, stereo_geom, csm_camera_plot, request_planetary_altimetry, gallery)
+  cli/                        # CLI tool docs (asp_report, stereo_geom, csm_camera_plot, bundle_adjust_cameras, request_planetary_altimetry, gallery, dem_benchmark)
   examples/
     index.md                  # Notebook gallery with cards by sensor
     reports.md                # PDF reports embedded as iframes
@@ -351,7 +364,7 @@ Docs dependencies are in `pyproject.toml` under `[project.optional-dependencies]
 ## Example Notebooks
 
 Example notebooks demonstrating modular usage are organized by sensor type:
-- `notebooks/WorldView/` - WorldView examples (Earth-based). `worldview_spacenet_ucsd_stereo.ipynb` is the classic two-scene pair example; `worldview_utqiagvik_stereo.ipynb` (excluded from the docs build) and `worldview_uyuni_jitter_plots.ipynb` cover comprehensive modular usage and jitter plots; the two `*_scene_selection.ipynb` notebooks pick which scenes to process with `stereo_geom` before any ASP run. The Atlanta example (`worldview_spacenet_atlanta_mvs.ipynb`) is a 3-scene same-pass multi-view run (`parallel_stereo --alignment-method affineepipolar` on the `wv_correct`ed `*_corr.tif` images, nadir13 reference) compared against the ASP-docs-recommended alternative: the three pairwise runs merged with `dem_mosaic` (coverage, DEM difference, ICESat-2 residuals). Two crop-window tricks keep the ROIs matched: the reference crop window in mapprojected/ground coordinates converts to image pixels via GDAL's inverse RPC transform on the *original* `.tif` (the `wv_correct` `_corr.tif` carries no RPC metadata) — `gdal.Transformer(["METHOD=RPC", "RPC_HEIGHT=<m>"])` + `TransformPoint(1, lon, lat, h)` on the corners; and the `--left-image-crop-win` for a pair whose left image is *not* the reference is derived from the `bundle_adjust` clean match points that fall inside the reference crop window (no RPCs needed at all).
+- `notebooks/WorldView/` - WorldView examples (Earth-based). `worldview_spacenet_ucsd_stereo.ipynb` is the classic two-scene pair example; `worldview_utqiagvik_stereo.ipynb` (excluded from the docs build) and `worldview_uyuni_jitter_plots.ipynb` cover comprehensive modular usage and jitter plots; the two `*_scene_selection.ipynb` notebooks pick which scenes to process with `stereo_geom` before any ASP run. **`worldview_spacenet_benchmark.ipynb` is the scene-combination benchmark (issue #169)**: with `DEMBenchmark` on one ICESat-2 sample per site it scores every combination and flow — at Atlanta 21 DEMs (all ten pairs of the five same-pass scenes, six `dem_mosaic` blends incl. median / convergence-weighted / `VerticalStdDev`-weighted, six multi-view runs incl. two references) then at UCSD 17 DEMs (five multi-date WorldView-3 scenes over a steep crop of Mount Soledad: three multi-view references, ten pairs, four blends) (both sets of runs come from local driver scripts that are not in the repo; the notebook re-scores whatever is on disk). Crop and scene selection for UCSD are done in the notebook from metadata staged next to the data (`ucsd_mvs/selection/`). Its findings, in its two takeaways sections: at the same-pass flat site single-pair accuracy is set by convergence angle; at the steep multi-date site the point estimates depend more on which scene a pair contains than on its geometry, but a paired track bootstrap (#199) separates only the worst DEM from the rest, so those rules are suggested rather than established; at both sites the multi-view reference scene has the largest effect. The Atlanta example (`worldview_spacenet_atlanta_mvs.ipynb`) is a 3-scene same-pass multi-view run (`parallel_stereo --alignment-method affineepipolar` on the `wv_correct`ed `*_corr.tif` images, nadir13 reference) compared against the ASP-docs-recommended alternative: the three pairwise runs merged with `dem_mosaic` (coverage, DEM difference, ICESat-2 residuals). Two crop-window methods keep the ROIs matched: the reference crop window in mapprojected/ground coordinates converts to image pixels via GDAL's inverse RPC transform on the *original* `.tif` (the `wv_correct` `_corr.tif` carries no RPC metadata) — `gdal.Transformer(["METHOD=RPC", "RPC_HEIGHT=<m>"])` + `TransformPoint(1, lon, lat, h)` on the corners; and the `--left-image-crop-win` for a pair whose left image is *not* the reference is derived from the `bundle_adjust` clean match points that fall inside the reference crop window (no RPCs needed at all).
 - `notebooks/Pleiades/` - Airbus Pléiades Neo tri-stereo example over Marseille (Earth-based): DIMAP stereo-geometry analysis, bundle adjustment of all three scenes, and a 3-image multi-view stereo run (nadir reference, `--left-image-crop-win` derived from the DIMAP `Dataset_Extent` vertices; the ASP docs discourage MVS in favor of pairwise + `dem_mosaic`, which the notebook cites)
 - `notebooks/ASTER/` - ASTER examples with map-projection and jitter correction (Earth-based)
 - `notebooks/LRO_NAC/` - Lunar Reconnaissance Orbiter Narrow Angle Camera examples (Lunar)

{asp_plot-3.2.0 → asp_plot-3.3.0}/CHANGELOG.md

@@ -5,6 +5,27 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.3.0] - 2026-09-18
+
+The first piece of the scene-combination benchmark ([#169](https://github.com/uw-cryo/asp_plot/issues/169)): a way to score *many* DEMs against *one* altimetry sample. The report assesses a single DEM; the question of which scene combination, which processing flow (joint multi-view triangulation vs. pairwise stereo merged with `dem_mosaic`), or which parameter setting gives the best DEM needs every candidate scored against exactly the same points, side by side. The Atlanta MVS notebook had been doing that with an ad hoc loop; it is now a class, a figure, and a `dem_benchmark` command, and the notebook uses them to score six DEMs at once, where the single pairs explain the mosaic's result (the 5° pair lowers its accuracy) and the five-scene run has the lowest bias but not the lowest spread. No new dependencies; one new entry point, so the conda-forge feedstock recipe needs `dem_benchmark` added by hand this release.
+
+### Added
+- **`DEMBenchmark`: score any number of DEMs against one ICESat-2 or LOLA/MOLA sample** (issue [#169](https://github.com/uw-cryo/asp_plot/issues/169)). `asp_plot/dem_benchmark.py` takes `{label: dem_fn}` plus the ATL06-SR parquet cache a report wrote (or a planetary CSV) and scores every DEM with the report's own recipe — replayed points, ESA WorldCover water filter, 3σ outlier cut — into a one-row-per-DEM table (`stats_df`, columns `STATS_COLUMNS`): coverage (`valid_pct`, `valid_area_km2`) inside a common area of interest (by default the intersection of all the DEM footprints, so runs with different crop windows compare fairly), the median and NMAD of the `*-IntersectionErr.tif` `point2dem` writes next to each `*-DEM.tif` (NaN for a mosaic, which has none), altimetry-minus-DEM n / median / NMAD / RMSE before and after a per-DEM `pc_align --compute-translation-only` with the translation it applied, and optionally each DEM's difference against one candidate named as the reference. `summary_plot()` draws the table as one row per DEM sorted best-first by post-alignment NMAD — coverage bars, IntersectionErr bars, and before→after dumbbells for the residual median and NMAD (a translation cannot change NMAD, so that panel separates bias, which alignment removes, from noise, which it cannot); `histogram_plot()` overlays the residual distributions. One `Altimetry` per DEM is kept in `bench.altimetry[label]`, so the usual per-DEM figures (`histogram_by_landcover()`, `mapview_plot_atl06sr_to_dem()`) can be drawn for any candidate. `pc_align` products and the translated DEM copies go under `<directory>/dem_benchmark/<label>/`, never into the candidates' own folders, and are reused on a re-run, which therefore makes no `pc_align` call and works offline; a missing `pc_align` binary degrades to pre-alignment scoring with a warning instead of failing. Large DEMs are read downsampled for the coverage statistics (capped at ~16 M samples per window).
+- **A `dem_benchmark` command** wrapping it: positional DEMs as paths or `LABEL=PATH` (an ASP `run-DEM.tif` is labelled by its folder), `--parquet` for Earth or `--altimetry-csv` for the Moon/Mars, `--reference`, `--no-pc-align`, `--own-extent`, `--title`, and `--directory` / `--output-directory` / `--output-filename`; writes the summary figure, a `_histogram.png` twin, and the stats table as CSV, and prints the table. Documented in `docs/cli/dem_benchmark.md` and added to the CLI index (seven tools). The conda-forge recipe gains the entry point and its `--help` test command (the autotick bot does not sync those).
+- **The Atlanta MVS notebook scores six DEMs on one ICESat-2 sample** with `DEMBenchmark` in place of its ad hoc two-DEM loop: the 3-scene MVS run, the 3-pair `dem_mosaic`, each of the three single pairs, and the 5-scene MVS run. The single pairs explain the mosaic: the 26.9° and 21.8° pairs score NMAD 0.64 and 0.72 m, the 5.1° pair 1.54 m, and `dem_mosaic` averages all three to 1.02 m — below either good pair alone — while the 3-scene MVS run's 0.71 m matches the best pairs and its bias is lower. The 5-scene run has the smallest bias (+0.12 m) but not the smallest spread (0.68 m), and its five-ray IntersectionErr median (0.19 m) reads higher than any pair's, so the notebook now explains why a narrow pair's small triangulation error is not a quality ranking. After `pc_align` every median lands at +0.16–0.27 m and no NMAD moves. The full matrix of scene combinations and flows is the benchmark notebook below.
+
+- **The Atlanta scene-combination benchmark: `notebooks/WorldView/worldview_spacenet_benchmark.ipynb`** (issue [#169](https://github.com/uw-cryo/asp_plot/issues/169)). The systematic version of the MVS notebook's comparison: all ten pairs among the five same-pass scenes (5–32° convergence), five `dem_mosaic` blends of them (the three pairs sharing the reference, all ten with the default average, only the six above 15°, all ten by median, all ten weighted by convergence angle, and all ten weighted by propagated `VerticalStdDev` — the ASP manual's own *sfm_multiview* recipe, which needs every pair re-triangulated with `--propagate-errors`) and six multi-view runs (3, 3-wide, 4 and 5 scenes, plus the five scenes again with a different reference), twenty-one DEMs scored with `DEMBenchmark` on one ICESat-2 sample. The runs took about 20 h and ~150 GB on a laptop over two sessions. What it found: single-pair NMAD falls monotonically with convergence angle, 1.5–1.65 m at 5° to 0.60 m at 32°, and IntersectionErr does not reflect it; adding scenes to a joint triangulation changes NMAD little (0.70 → 0.70 → 0.65 → 0.66 m along the nested chain) but does remove bias; and pairwise + mosaic beats multi-view only when the weak pairs are *excluded* — every pair merged scores 0.99 m against MVS 5's 0.66 m, weighting by convergence or by propagated uncertainty only reaches 0.80–0.83 m, the median 0.67 m, and the six pairs above 15° 0.62 m. The mechanism, from ASP's source at the commit that built the DEMs (written up on the issue): a multi-view run is N−1 pairs of the *first image with each other image* — never scene-to-scene — followed by an unweighted least-squares ray intersection, so its quality is that of the reference's "star" of pairs. Referenced on the middle scene the star is 5.1°, 10.5°, 16.3° and 21.8°; referenced on the scene at the end of the pass it is 5.5°, 21.8°, 26.9° and 32.3°, and that run scores **0.58 m, the lowest NMAD of the twenty-one**. The notebook's docs card and toctree entry are added.
+- **A second site for the benchmark, in the same notebook: SpaceNet UCSD WorldView-3, multi-date, over Mount Soledad** (issue [#169](https://github.com/uw-cryo/asp_plot/issues/169)). Chosen to differ from Atlanta in every respect its caveats list: the archive's 35 scenes are 35 separate collects, so any N-scene run is multi-date stereo, the regime the ASP manual discourages multi-view for; the 3 × 3 km crop has 244 m of relief, 30 % of its area steeper than 15°, 45 % built-up and 45 % tree cover. Five winter scenes (sun 30–40°, 1.1° to 24° off-nadir, pairs 8–34°), processed as three five-scene multi-view runs (referenced on the nadir, the 24° and the 8° scene), all ten pairs, and four `dem_mosaic` blends: seventeen DEMs at 1.2 m, 16 h on a laptop. The crop and scene selection are done in the notebook from metadata (Copernicus relief, WorldCover, ICESat-2 track density, pairwise convergence from the XMLs). What it found, with the caveat that a paired track bootstrap ([#199](https://github.com/uw-cryo/asp_plot/issues/199)) separates only the worst DEM from the other sixteen at this sample size: the point estimates do not reproduce the Atlanta convergence curve — the 8° pair (1.47 m) ties the best 25° pair (1.48 m) and the four pairs with the Nov 9 nadir scene score worst at every convergence, which suggests matching rather than geometry limits a pair in steep multi-date urban terrain; the narrowest reference star scores best rather than the widest (1.48 m nadir against 1.61 m for the 24° reference); and the joint triangulation ties the best pair while every blend of the ten pairs lands at 1.64–1.70 m. Exclusion by convergence angle, which worked at Atlanta, would drop the best-scoring pair here. The notebook ends with the two sites on the same axes and the practical rules revised to those consistent with both sites. Two `asp_plot` fixes came out of scoring it (below).
+
+- **`asp_plot.stereo.read_match_file(fn)`**: read an ASP match file (binary or `--matches-as-txt`) into a DataFrame without a stereo directory. `StereoPlotter.get_match_point_df()` delegates to it; it exists because the benchmark notebook derives crop windows from `bundle_adjust` match files, and a `StereoPlotter` cannot be built once a run's `L.tif` has been cleaned up.
+
+### Changed
+- `Alignment.apply_dem_translation()` accepts an `output_fn` for where to write the translated DEM (default unchanged: next to the source DEM), which is how the benchmark keeps candidate folders untouched.
+- **The altimetry-minus-DEM outlier cut first drops gross outliers, then applies the 3σ cut as before** (`AltimetrySource._outlier_mask`, used by `atl06sr_to_dem_dh()`, `planetary_to_dem_dh()` and `filter_outliers()`, and so by every report and by `DEMBenchmark`). Points more than 30 normalized median absolute deviations from the median are removed before the mean/std statistics are computed. Motivation: over the UCSD site one whole ICESat-2 pass is a marine-layer cloud return 150–200 m above the ground — a fifth of the sample — and the mean/std cut, its std inflated to ~70 m by that cluster, removed nothing, leaving every DEM with an RMSE near 85 m and an NMAD a metre too high. A pure median/NMAD cut was tried first and rejected: 3 NMAD is far tighter than 3σ on the heavy-tailed residuals a DSM has against ICESat-2 in a city (it removed a fifth of the Atlanta sample and cut every NMAD there by a third, from 0.68 to 0.48 m for the five-scene run), and those tails are DEM error, not blunders. The gross gate touches only a few dozen of the ~7000 Atlanta points, so existing numbers move by at most 0.02 m, while the cloud pass at 65–75 NMAD is gone. `n_sigma` keeps its name and default. The benchmark notebook was re-executed with the gate (its Atlanta numbers moved by 0.01–0.02 m, no ranking changed); the earlier MVS notebook keeps its pre-gate numbers.
+
+### Fixed
+- **Aligned-DEM residuals are now computed on exactly the points the unaligned DEM was scored on.** `pc_align`'s translation moves a DEM's holes and edges by a few metres, so a point that sampled NaN on the unaligned DEM — and therefore passed the outlier cut untested — could land on valid data in the translated copy and enter the post-alignment residuals unfiltered. Over UCSD that let six cloud returns sitting on a DEM hole into the aligned statistics as 150–210 m errors, lifting the post-alignment RMSE of some candidates from 2 m to 7–19 m while their NMAD barely moved. `atl06sr_to_dem_dh()` and `planetary_to_dem_dh()` now blank the aligned residual wherever the unaligned one is NaN, which also holds the report's pre/post-alignment plots to one sample.
+
 ## [3.2.0] - 2026-08-27
 
 A new view of what `bundle_adjust` did to the cameras — read from the run's own output folder, so no original camera files are needed — as a `bundle_adjust_cameras` command, a page of the `asp_report` PDF, and four notebook examples. The figure was reviewed on ten local runs (two to fourteen cameras, ASP 3.4 through 3.8); the row that earned its place is the ground effect from `triangulation_offsets.txt`, which is what tells a real camera error apart from a position/orientation trade in the solve. No new dependencies; one new entry point, so the conda-forge feedstock recipe needs the `bundle_adjust_cameras` entry point and test command added by hand this release.

{asp_plot-3.2.0 → asp_plot-3.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: asp_plot
-Version: 3.2.0
+Version: 3.3.0
 Summary: Package for plotting outputs Ames Stereo Pipeline processing
 Project-URL: Homepage, https://github.com/uw-cryo/asp_plot
 Project-URL: Documentation, https://asp-plot.readthedocs.io

{asp_plot-3.2.0 → asp_plot-3.3.0}/asp_plot/alignment.py

@@ -386,7 +386,7 @@ class Alignment:
 
         return report
 
-    def apply_dem_translation(self, output_prefix="pc_align/pc_align"):
+    def apply_dem_translation(self, output_prefix="pc_align/pc_align", output_fn=None):
         """
         Apply the pc_align translation to the DEM.
 
@@ -398,6 +398,13 @@ class Alignment:
         ----------
         output_prefix : str, optional
             Prefix for pc_align output files, default is "pc_align/pc_align"
+        output_fn : str, optional
+            Where to write the translated DEM. Default is None, which writes
+            ``<dem_fn without .tif>_pc_align_translated.tif`` next to the
+            source DEM (the report convention). Pass an explicit path to
+            keep the translated copy elsewhere, e.g. when many candidate
+            DEMs are aligned side by side and their folders should stay
+            untouched (:class:`asp_plot.dem_benchmark.DEMBenchmark`).
 
         Returns
         -------
@@ -479,7 +486,11 @@ class Alignment:
         t_srs.ImportFromWkt(src.ds.crs.to_wkt())
         proj_shift = self.get_proj_shift(src_c, src_shift, s_srs, t_srs, inv_trans=True)
 
-        aligned_dem_fn = self.dem_fn.replace(".tif", "_pc_align_translated.tif")
+        if output_fn is None:
+            aligned_dem_fn = self.dem_fn.replace(".tif", "_pc_align_translated.tif")
+        else:
+            aligned_dem_fn = output_fn
+            os.makedirs(os.path.dirname(os.path.abspath(aligned_dem_fn)), exist_ok=True)
         print(f"\nWriting out: {aligned_dem_fn}\n")
 
         gdal_opt = ["COMPRESS=LZW", "TILED=YES", "PREDICTOR=3", "BIGTIFF=IF_SAFER"]

{asp_plot-3.2.0 → asp_plot-3.3.0}/asp_plot/altimetry_source.py

@@ -20,6 +20,11 @@ import numpy as np
 import rioxarray
 import xarray as xr
 
+# A residual this many NMADs from the median is not a DEM error but a blunder in
+# the altimetry (cloud returns, a bad fit); dropped before the n_sigma × std cut
+# so that a large cluster of them cannot inflate the std and defeat it.
+GROSS_OUTLIER_NMAD = 30
+
 
 class AltimetrySource:
     """Base for ICESat-2 and planetary altimetry sources.
@@ -66,13 +71,26 @@ class AltimetrySource:
         return rioxarray.open_rasterio(dem_fn, masked=True).squeeze()
 
     @staticmethod
-    def _std_outlier_mask(dh, n_sigma):
-        """Boolean mask keeping dh values within ``n_sigma`` × std of the mean.
+    def _outlier_mask(dh, n_sigma):
+        """Boolean mask keeping dh values within ``n_sigma`` × std of the mean,
+        after dropping gross outliers.
+
+        Two cuts. First, anything farther than ``GROSS_OUTLIER_NMAD`` (30)
+        normalized median absolute deviations from the median is dropped: no
+        DEM error puts a point 30 NMADs out, but a cloud return does — one
+        ICESat-2 pass of marine-layer cloud 150–200 m above the ground was a
+        fifth of the sample over a coastal site, and a plain mean/std cut,
+        its std inflated to ~70 m by that cluster, removed nothing and left
+        every DEM with an RMSE near 85 m. Then the usual ``n_sigma`` × std cut
+        about the mean of what survives, which is the cut every report has
+        always applied and, absent gross contamination, gives the same result
+        as before (the gross cut touches at most a few dozen points of the
+        ~7000 over Atlanta).
 
         Rows whose ``dh`` is NaN are kept (they carry no difference yet and
         must not be dropped). Returns ``None`` — signalling that no filtering
         should occur — when there are no finite values or the spread is
-        degenerate (zero or non-finite std).
+        degenerate (zero or non-finite scale).
 
         Parameters
         ----------
@@ -84,11 +102,17 @@ class AltimetrySource:
         valid = dh.dropna().values
         if valid.size == 0:
             return None
-        mean_val = np.mean(valid)
-        std_val = np.std(valid)
+        median_val = np.median(valid)
+        nmad_val = 1.4826 * np.median(np.abs(valid - median_val))
+        if nmad_val == 0 or np.isnan(nmad_val):
+            return None
+        gross = (dh - median_val).abs() <= GROSS_OUTLIER_NMAD * nmad_val
+        core = dh[gross].dropna().values
+        mean_val = np.mean(core)
+        std_val = np.std(core)
         if std_val == 0 or np.isnan(std_val):
             return None
-        mask = (dh - mean_val).abs() <= n_sigma * std_val
+        mask = gross & ((dh - mean_val).abs() <= n_sigma * std_val)
         return mask | dh.isna()
 
     def _write_csv_to_directory(self, df, filename):

asp_plot-3.3.0/asp_plot/cli/dem_benchmark.py

@@ -0,0 +1,128 @@
+import os
+
+import click
+
+from asp_plot.dem_benchmark import DEMBenchmark
+
+
+@click.command()
+@click.argument("dems", nargs=-1, required=True)
+@click.option(
+    "--parquet",
+    prompt=False,
+    default=None,
+    help="ICESat-2 ATL06-SR parquet cache to score Earth DEMs against (the "
+    "atl06sr_all.parquet a previous asp_report run wrote next to its report, "
+    "or from Altimetry.request_atl06sr_multi_processing(save_to_parquet=True)). "
+    "The same points are replayed for every DEM; no SlideRule request is made.",
+)
+@click.option(
+    "--altimetry-csv",
+    prompt=False,
+    default=None,
+    help="LOLA/MOLA CSV to score Moon/Mars DEMs against (see "
+    "request_planetary_altimetry). Use instead of --parquet for planetary DEMs.",
+)
+@click.option(
+    "--directory",
+    prompt=False,
+    default="./",
+    help="Working directory. pc_align products and the translated DEM copies go "
+    "under <directory>/dem_benchmark/<label>/, never into the DEMs' own folders. "
+    "Default: current directory.",
+)
+@click.option(
+    "--reference",
+    prompt=False,
+    default=None,
+    help="Label of one of the DEMs to difference the others against (vs_ref "
+    "columns of the stats table). Default: none.",
+)
+@click.option(
+    "--no-pc-align",
+    is_flag=True,
+    default=False,
+    help="Skip the per-DEM pc_align translation; report pre-alignment residuals only.",
+)
+@click.option(
+    "--own-extent",
+    is_flag=True,
+    default=False,
+    help="Compute coverage and triangulation-error statistics over each DEM's "
+    "own extent instead of the intersection of all DEM footprints.",
+)
+@click.option(
+    "--title",
+    prompt=False,
+    default=None,
+    help="Figure title. Default: none.",
+)
+@click.option(
+    "--output-directory",
+    prompt=False,
+    default=None,
+    help="Directory for the figure and stats CSV. Default: --directory.",
+)
+@click.option(
+    "--output-filename",
+    prompt=False,
+    default="dem_benchmark.png",
+    help="Figure filename; the stats CSV takes the same name with a .csv "
+    "extension, and the residual histogram figure a _histogram suffix. "
+    "Default: dem_benchmark.png.",
+)
+def main(
+    dems,
+    parquet,
+    altimetry_csv,
+    directory,
+    reference,
+    no_pc_align,
+    own_extent,
+    title,
+    output_directory,
+    output_filename,
+):
+    """
+    Score many DEMs against one altimetry sample.
+
+    DEMS are paths, optionally labelled as LABEL=PATH (e.g.
+    "MVS=stereo_mvs3/run-DEM.tif"); an unlabelled ASP run-DEM.tif is labelled
+    by its folder. Every DEM gets: coverage inside the common footprint, the
+    median triangulation error from its IntersectionErr raster when present,
+    and the altimetry-minus-DEM median / NMAD / RMSE before and (unless
+    --no-pc-align) after a pc_align translation. Writes a one-row-per-DEM
+    summary figure, an overlaid residual histogram, and the stats table as CSV.
+    """
+    directory = os.path.expanduser(directory)
+    if output_directory is None:
+        output_directory = directory
+    output_directory = os.path.expanduser(output_directory)
+
+    bench = DEMBenchmark(
+        directory=directory,
+        dems=list(dems),
+        parquet=parquet,
+        altimetry_csv=altimetry_csv,
+        reference=reference,
+        aoi=None if own_extent else "intersection",
+        title=title,
+    )
+    stats = bench.run(pc_align=not no_pc_align)
+
+    stem = os.path.splitext(output_filename)[0]
+    csv_fn = bench.save_stats(os.path.join(output_directory, f"{stem}.csv"))
+    bench.summary_plot(save_dir=output_directory, fig_fn=output_filename)
+    bench.histogram_plot(save_dir=output_directory, fig_fn=f"{stem}_histogram.png")
+
+    shown = stats.drop(columns=["dem_fn"])
+    print("\n" + shown.to_string(index=False, float_format=lambda v: f"{v:.2f}"))
+    print(
+        f"\nSummary figure: {os.path.join(output_directory, output_filename)}"
+        f"\nHistogram:      {os.path.join(output_directory, stem + '_histogram.png')}"
+        f"\nStats table:    {csv_fn}\n"
+    )
+
+
+if __name__ == "__main__":
+    main()