asp-plot 0.5.2__tar.gz → 0.5.4__tar.gz

{asp_plot-0.5.2 → asp_plot-0.5.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: asp_plot
-Version: 0.5.2
+Version: 0.5.4
 Summary: Package for plotting outputs Ames Stereo Pipeline processing
 Author: Ben Purinton
 Author-email: Ben Purinton <purinton@uw.edu>
@@ -67,7 +67,16 @@ Examples of the modular usage of the package can be found in the [`notebooks/` d
 
 ## CLI usage: `asp_plot`
 
-A full report and individual plots can be output via the command-line:
+A full report can be output via the command-line. At its simplest, you can run:
+
+```
+$ asp_plot --directory ./ \
+           --stereo_directory stereo
+```
+
+with only the directory where the ASP processing was done (`--directory`) and the subdirectory inside of that where the stereo files were output (`--stereo`). The reference DEM used in ASP processing will also be searched for in the logs, and used for difference maps if found.
+
+If you also ran bundle adjustment and/or would like to specify a reference DEM to use for plotting (rather than searching the logs):
 
 ```
 $ asp_plot --directory ./ \
@@ -84,29 +93,37 @@ Before that, we recommend running `asp_plot --help` for details (and defaults) o
 Usage: asp_plot [OPTIONS]
 
 Options:
-  --directory TEXT                Directory of ASP processing with scenes and
-                                  sub-directories for bundle adjustment and
-                                  stereo. Default: current directory
-  --bundle_adjust_directory TEXT  Directory of bundle adjustment files.
-                                  Default: ba
-  --stereo_directory TEXT         Directory of stereo files. Default: stereo
-  --map_crs TEXT                  Projection for bundle adjustment plots.
-                                  Default: EPSG:4326
-  --reference_dem TEXT            Reference DEM used in ASP processing. No
-                                  default. Must be supplied.
+  --directory TEXT                Required directory of ASP processing with
+                                  scenes and sub-directories for stereo and
+                                  optionally bundle adjustment. Default:
+                                  current directory.
+  --bundle_adjust_directory TEXT  Optional directory of bundle adjustment
+                                  files. If expected *residuals_pointmap.csv
+                                  files are not found in the supplied
+                                  directory, no bundle adjustment plots will
+                                  be generated. Default: None.
+  --stereo_directory TEXT         Required directory of stereo files. Default:
+                                  stereo.
+  --map_crs TEXT                  Projection for ICESat and bundle adjustment
+                                  plots. Default: None.
+  --reference_dem TEXT            Optional reference DEM used in ASP
+                                  processing. No default. If not supplied, the
+                                  logs will be examined to find it. If not
+                                  found, no difference plots will be
+                                  generated.
   --add_basemap BOOLEAN           If True, add a contextily basemap to the
                                   figure, which requires internet connection.
-                                  Default: True
+                                  Default: True.
   --plot_icesat BOOLEAN           If True, plot an ICESat-2 difference plot
                                   with the DEM result. This requires internet
                                   connection to pull ICESat data. Default:
-                                  True
+                                  True.
   --report_filename TEXT          PDF file to write out for report into the
                                   processing directory supplied by
                                   --directory. Default: Directory name of ASP
-                                  processing
+                                  processing.
   --report_title TEXT             Title for the report. Default: Directory
-                                  name of ASP processing
+                                  name of ASP processing.
 ```
 
 ## CLI usage: `csm_camera_plot`

{asp_plot-0.5.2 → asp_plot-0.5.4}/README.md

@@ -52,7 +52,16 @@ Examples of the modular usage of the package can be found in the [`notebooks/` d
 
 ## CLI usage: `asp_plot`
 
-A full report and individual plots can be output via the command-line:
+A full report can be output via the command-line. At its simplest, you can run:
+
+```
+$ asp_plot --directory ./ \
+           --stereo_directory stereo
+```
+
+with only the directory where the ASP processing was done (`--directory`) and the subdirectory inside of that where the stereo files were output (`--stereo`). The reference DEM used in ASP processing will also be searched for in the logs, and used for difference maps if found.
+
+If you also ran bundle adjustment and/or would like to specify a reference DEM to use for plotting (rather than searching the logs):
 
 ```
 $ asp_plot --directory ./ \
@@ -69,29 +78,37 @@ Before that, we recommend running `asp_plot --help` for details (and defaults) o
 Usage: asp_plot [OPTIONS]
 
 Options:
-  --directory TEXT                Directory of ASP processing with scenes and
-                                  sub-directories for bundle adjustment and
-                                  stereo. Default: current directory
-  --bundle_adjust_directory TEXT  Directory of bundle adjustment files.
-                                  Default: ba
-  --stereo_directory TEXT         Directory of stereo files. Default: stereo
-  --map_crs TEXT                  Projection for bundle adjustment plots.
-                                  Default: EPSG:4326
-  --reference_dem TEXT            Reference DEM used in ASP processing. No
-                                  default. Must be supplied.
+  --directory TEXT                Required directory of ASP processing with
+                                  scenes and sub-directories for stereo and
+                                  optionally bundle adjustment. Default:
+                                  current directory.
+  --bundle_adjust_directory TEXT  Optional directory of bundle adjustment
+                                  files. If expected *residuals_pointmap.csv
+                                  files are not found in the supplied
+                                  directory, no bundle adjustment plots will
+                                  be generated. Default: None.
+  --stereo_directory TEXT         Required directory of stereo files. Default:
+                                  stereo.
+  --map_crs TEXT                  Projection for ICESat and bundle adjustment
+                                  plots. Default: None.
+  --reference_dem TEXT            Optional reference DEM used in ASP
+                                  processing. No default. If not supplied, the
+                                  logs will be examined to find it. If not
+                                  found, no difference plots will be
+                                  generated.
   --add_basemap BOOLEAN           If True, add a contextily basemap to the
                                   figure, which requires internet connection.
-                                  Default: True
+                                  Default: True.
   --plot_icesat BOOLEAN           If True, plot an ICESat-2 difference plot
                                   with the DEM result. This requires internet
                                   connection to pull ICESat data. Default:
-                                  True
+                                  True.
   --report_filename TEXT          PDF file to write out for report into the
                                   processing directory supplied by
                                   --directory. Default: Directory name of ASP
-                                  processing
+                                  processing.
   --report_title TEXT             Title for the report. Default: Directory
-                                  name of ASP processing
+                                  name of ASP processing.
 ```
 
 ## CLI usage: `csm_camera_plot`

{asp_plot-0.5.2 → asp_plot-0.5.4}/asp_plot/bundle_adjust.py

@@ -6,8 +6,15 @@ import matplotlib.pyplot as plt
 import numpy as np
 import pandas as pd
 
+from asp_plot.processing_parameters import ProcessingParameters
 from asp_plot.stereopair_metadata_parser import StereopairMetadataParser
-from asp_plot.utils import ColorBar, Plotter, glob_file, save_figure
+from asp_plot.utils import (
+    ColorBar,
+    Plotter,
+    glob_file,
+    run_subprocess_command,
+    save_figure,
+)
 
 logging.basicConfig(level=logging.WARNING)
 logger = logging.getLogger(__name__)
@@ -26,18 +33,51 @@ class ReadBundleAdjustFiles:
         ]
 
         if geodiff_files:
-            filenames = [f.replace(".csv", "-diff.csv") for f in filenames]
+            base_paths = [glob_file(self.full_directory, f) for f in filenames]
+            diff_paths = []
+            for path in base_paths:
+                diff_path = path.replace(".csv", "-diff.csv")
+                if not os.path.exists(diff_path):
+                    self.generate_geodiff(path)
+                diff_paths.append(diff_path)
+            initial_diff, final_diff = diff_paths
+            return initial_diff, final_diff
 
-        paths = [glob_file(self.full_directory, f) for f in filenames]
-
-        for path in paths:
-            if path is None:
-                raise ValueError(
-                    "\n\nInitial and final bundle adjust CSV file not found. Did you run bundle_adjust?\n\n"
-                )
+        else:
+            paths = [glob_file(self.full_directory, f) for f in filenames]
+            for path in paths:
+                if path is None:
+                    raise ValueError(
+                        "\n\nInitial and final bundle adjust CSV file not found. Did you run bundle_adjust?\n\n"
+                    )
+            initial, final = paths
+            return initial, final
+
+    def generate_geodiff(self, path):
+        processing_parameters = ProcessingParameters(
+            processing_directory=self.directory,
+            bundle_adjust_directory=self.bundle_adjust_directory,
+        )
+        refdem = processing_parameters.get_reference_dem(
+            processing_parameters.bundle_adjust_log
+        )
 
-        initial, final = paths
-        return initial, final
+        try:
+            command = [
+                "geodiff",
+                "--csv-format",
+                "1:lon,2:lat,3:height_above_datum",
+                f"{path}",
+                f"{refdem}",
+                "-o",
+                f"{path.replace('.csv', '')}",
+            ]
+
+            run_subprocess_command(command)
+        except Exception:
+            logger.warning(
+                f"\n\nCould not generate geodiff file for {path}. Check that the geodiff ASP tool is installed and the reference DEM {refdem} exists.\n\n"
+            )
 
     def get_residuals_gdf(self, csv_path, residuals_in_meters=True):
         cols = [

{asp_plot-0.5.2 → asp_plot-0.5.4}/asp_plot/cli/asp_plot.py

@@ -18,55 +18,55 @@ from asp_plot.utils import compile_report
     "--directory",
     prompt=True,
     default="./",
-    help="Directory of ASP processing with scenes and sub-directories for bundle adjustment and stereo. Default: current directory",
+    help="Required directory of ASP processing with scenes and sub-directories for stereo and optionally bundle adjustment. Default: current directory.",
 )
 @click.option(
     "--bundle_adjust_directory",
-    prompt=True,
-    default="ba",
-    help="Directory of bundle adjustment files. Default: ba",
+    prompt=False,
+    default=None,
+    help="Optional directory of bundle adjustment files. If expected *residuals_pointmap.csv files are not found in the supplied directory, no bundle adjustment plots will be generated. Default: None.",
 )
 @click.option(
     "--stereo_directory",
     prompt=True,
     default="stereo",
-    help="Directory of stereo files. Default: stereo",
+    help="Required directory of stereo files. Default: stereo.",
 )
 @click.option(
     "--map_crs",
-    prompt=True,
-    default="EPSG:4326",
-    help="Projection for bundle adjustment plots. Default: EPSG:4326",
+    prompt=False,
+    default=None,
+    help="Projection for ICESat and bundle adjustment plots. Default: None.",
 )
 @click.option(
     "--reference_dem",
-    prompt=True,
-    default="",
-    help="Reference DEM used in ASP processing. No default. Must be supplied.",
+    prompt=False,
+    default=None,
+    help="Optional reference DEM used in ASP processing. No default. If not supplied, the logs will be examined to find it. If not found, no difference plots will be generated.",
 )
 @click.option(
     "--add_basemap",
     prompt=False,
     default=True,
-    help="If True, add a contextily basemap to the figure, which requires internet connection. Default: True",
+    help="If True, add a contextily basemap to the figure, which requires internet connection. Default: True.",
 )
 @click.option(
     "--plot_icesat",
     prompt=False,
     default=True,
-    help="If True, plot an ICESat-2 difference plot with the DEM result. This requires internet connection to pull ICESat data. Default: True",
+    help="If True, plot an ICESat-2 difference plot with the DEM result. This requires internet connection to pull ICESat data. Default: True.",
 )
 @click.option(
     "--report_filename",
     prompt=False,
     default=None,
-    help="PDF file to write out for report into the processing directory supplied by --directory. Default: Directory name of ASP processing",
+    help="PDF file to write out for report into the processing directory supplied by --directory. Default: Directory name of ASP processing.",
 )
 @click.option(
     "--report_title",
     prompt=False,
     default=None,
-    help="Title for the report. Default: Directory name of ASP processing",
+    help="Title for the report. Default: Directory name of ASP processing.",
 )
 def main(
     directory,
@@ -79,7 +79,7 @@ def main(
     report_filename,
     report_title,
 ):
-    print(f"\n\nProcessing ASP files in {directory}\n\n")
+    print(f"\nProcessing ASP files in {directory}\n")
 
     plots_directory = os.path.join(directory, "tmp_asp_report_plots/")
     os.makedirs(plots_directory, exist_ok=True)
@@ -92,6 +92,13 @@ def main(
 
     figure_counter = count(0)
 
+    if map_crs is None:
+        print(
+            "\nNo map projection supplied. Defaulting to EPSG:4326. If you want a different projection, supply it with the --map_crs flag.\n"
+        )
+        map_crs = "EPSG:4326"
+        add_basemap = False
+
     if add_basemap:
         ctx_kwargs = {
             "crs": map_crs,
@@ -102,23 +109,57 @@ def main(
     else:
         ctx_kwargs = {}
 
-    # Detailed hillshade plot
+    # Stereo plots
     plotter = StereoPlotter(
         directory,
         stereo_directory,
-        reference_dem,
+        reference_dem=reference_dem,
         out_dem_gsd=1,
         title="Hillshade with details",
     )
 
+    asp_dem = plotter.dem_fn
+
     plotter.plot_detailed_hillshade(
         save_dir=plots_directory,
         fig_fn=f"{next(figure_counter):02}.png",
     )
 
+    plotter.title = "Stereo DEM Results"
+    plotter.plot_dem_results(
+        save_dir=plots_directory,
+        fig_fn=f"{next(figure_counter):02}.png",
+    )
+
+    plotter.title = "Disparity (pixels)"
+    plotter.plot_disparity(
+        unit="pixels",
+        quiver=True,
+        save_dir=plots_directory,
+        fig_fn=f"{next(figure_counter):02}.png",
+    )
+
+    plotter.title = "Stereo Match Points"
+    plotter.plot_match_points(
+        save_dir=plots_directory,
+        fig_fn=f"{next(figure_counter):02}.png",
+    )
+
+    # Scene plot
+    plotter = ScenePlotter(directory, stereo_directory, title="Mapprojected Scenes")
+    plotter.plot_orthos(
+        save_dir=plots_directory, fig_fn=f"{next(figure_counter):02}.png"
+    )
+
+    # Geometry plot
+    plotter = SceneGeometryPlotter(directory)
+    plotter.dg_geom_plot(
+        save_dir=plots_directory, fig_fn=f"{next(figure_counter):02}.png"
+    )
+
     # ICESat-2 comparison
     if plot_icesat:
-        icesat = Altimetry(dem_fn=plotter.dem_fn)
+        icesat = Altimetry(dem_fn=asp_dem)
 
         icesat.pull_atl06sr(
             esa_worldcover=True,
@@ -144,118 +185,86 @@ def main(
             fig_fn=f"{next(figure_counter):02}.png",
         )
 
-    # Geometry plot
-    plotter = SceneGeometryPlotter(directory)
-
-    plotter.dg_geom_plot(
-        save_dir=plots_directory, fig_fn=f"{next(figure_counter):02}.png"
-    )
-
-    # Scene plot
-    plotter = ScenePlotter(directory, stereo_directory, title="Mapprojected Scenes")
-
-    plotter.plot_orthos(
-        save_dir=plots_directory, fig_fn=f"{next(figure_counter):02}.png"
-    )
-
     # Bundle adjustment plots
-    ba_files = ReadBundleAdjustFiles(directory, bundle_adjust_directory)
-    resid_initial_gdf, resid_final_gdf = ba_files.get_initial_final_residuals_gdfs()
-    geodiff_initial_gdf, geodiff_final_gdf = ba_files.get_initial_final_geodiff_gdfs()
-    resid_mapprojected_gdf = ba_files.get_mapproj_residuals_gdf()
-
-    plotter = PlotBundleAdjustFiles(
-        [resid_initial_gdf, resid_final_gdf],
-        lognorm=True,
-        title="Bundle Adjust Initial and Final Residuals (Log Scale)",
-    )
-
-    plotter.plot_n_gdfs(
-        column_name="mean_residual",
-        cbar_label="Mean residual (px)",
-        map_crs=map_crs,
-        save_dir=plots_directory,
-        fig_fn=f"{next(figure_counter):02}.png",
-        **ctx_kwargs,
-    )
-
-    plotter.lognorm = False
-    plotter.title = "Bundle Adjust Initial and Final Residuals (Linear Scale)"
-
-    plotter.plot_n_gdfs(
-        column_name="mean_residual",
-        cbar_label="Mean residual (px)",
-        common_clim=False,
-        map_crs=map_crs,
-        save_dir=plots_directory,
-        fig_fn=f"{next(figure_counter):02}.png",
-        **ctx_kwargs,
-    )
-
-    plotter = PlotBundleAdjustFiles(
-        [resid_mapprojected_gdf],
-        title="Bundle Adjust Midpoint distance between\nfinal interest points projected onto reference DEM",
-    )
-
-    plotter.plot_n_gdfs(
-        column_name="mapproj_ip_dist_meters",
-        cbar_label="Interest point distance (m)",
-        map_crs=map_crs,
-        save_dir=plots_directory,
-        fig_fn=f"{next(figure_counter):02}.png",
-        **ctx_kwargs,
-    )
-
-    plotter = PlotBundleAdjustFiles(
-        [geodiff_initial_gdf, geodiff_final_gdf],
-        lognorm=False,
-        title="Bundle Adjust Initial and Final Geodiff vs. Reference DEM",
-    )
-
-    plotter.plot_n_gdfs(
-        column_name="height_diff_meters",
-        cbar_label="Height difference (m)",
-        map_crs=map_crs,
-        cmap="RdBu",
-        symm_clim=True,
-        save_dir=plots_directory,
-        fig_fn=f"{next(figure_counter):02}.png",
-        **ctx_kwargs,
-    )
-
-    # Stereo plots
-    plotter = StereoPlotter(
-        directory,
-        stereo_directory,
-        reference_dem,
-        out_dem_gsd=1,
-        title="Stereo Match Points",
-    )
-
-    plotter.plot_match_points(
-        save_dir=plots_directory,
-        fig_fn=f"{next(figure_counter):02}.png",
-    )
-
-    plotter.title = "Disparity (pixels)"
-
-    plotter.plot_disparity(
-        unit="pixels",
-        quiver=True,
-        save_dir=plots_directory,
-        fig_fn=f"{next(figure_counter):02}.png",
-    )
-
-    plotter.title = "Stereo DEM Results"
-
-    plotter.plot_dem_results(
-        save_dir=plots_directory,
-        fig_fn=f"{next(figure_counter):02}.png",
-    )
+    if bundle_adjust_directory:
+        try:
+            ba_files = ReadBundleAdjustFiles(directory, bundle_adjust_directory)
+            resid_initial_gdf, resid_final_gdf = (
+                ba_files.get_initial_final_residuals_gdfs()
+            )
+            geodiff_initial_gdf, geodiff_final_gdf = (
+                ba_files.get_initial_final_geodiff_gdfs()
+            )
+            resid_mapprojected_gdf = ba_files.get_mapproj_residuals_gdf()
+
+            plotter = PlotBundleAdjustFiles(
+                [resid_initial_gdf, resid_final_gdf],
+                lognorm=True,
+                title="Bundle Adjust Initial and Final Residuals (Log Scale)",
+            )
+
+            plotter.plot_n_gdfs(
+                column_name="mean_residual",
+                cbar_label="Mean residual (px)",
+                map_crs=map_crs,
+                save_dir=plots_directory,
+                fig_fn=f"{next(figure_counter):02}.png",
+                **ctx_kwargs,
+            )
+
+            plotter.lognorm = False
+            plotter.title = "Bundle Adjust Initial and Final Residuals (Linear Scale)"
+
+            plotter.plot_n_gdfs(
+                column_name="mean_residual",
+                cbar_label="Mean residual (px)",
+                common_clim=False,
+                map_crs=map_crs,
+                save_dir=plots_directory,
+                fig_fn=f"{next(figure_counter):02}.png",
+                **ctx_kwargs,
+            )
+
+            plotter = PlotBundleAdjustFiles(
+                [resid_mapprojected_gdf],
+                title="Bundle Adjust Midpoint distance between\nfinal interest points projected onto reference DEM",
+            )
+
+            plotter.plot_n_gdfs(
+                column_name="mapproj_ip_dist_meters",
+                cbar_label="Interest point distance (m)",
+                map_crs=map_crs,
+                save_dir=plots_directory,
+                fig_fn=f"{next(figure_counter):02}.png",
+                **ctx_kwargs,
+            )
+
+            plotter = PlotBundleAdjustFiles(
+                [geodiff_initial_gdf, geodiff_final_gdf],
+                lognorm=False,
+                title="Bundle Adjust Initial and Final Geodiff vs. Reference DEM",
+            )
+
+            plotter.plot_n_gdfs(
+                column_name="height_diff_meters",
+                cbar_label="Height difference (m)",
+                map_crs=map_crs,
+                cmap="RdBu",
+                symm_clim=True,
+                save_dir=plots_directory,
+                fig_fn=f"{next(figure_counter):02}.png",
+                **ctx_kwargs,
+            )
+        except ValueError:
+            print(
+                f"\n\nNo bundle adjustment files found in directory {os.path.join(directory, bundle_adjust_directory):}. If you want bundle adjustment plots, make sure you run the tool and supply the correct directory to asp_plot.\n\n"
+            )
 
     # Compile report
     processing_parameters = ProcessingParameters(
-        directory, bundle_adjust_directory, stereo_directory
+        processing_directory=directory,
+        bundle_adjust_directory=bundle_adjust_directory,
+        stereo_directory=stereo_directory,
     )
     processing_parameters_dict = processing_parameters.from_log_files()
 

{asp_plot-0.5.2 → asp_plot-0.5.4}/asp_plot/csm_camera.py

@@ -304,17 +304,25 @@ def csm_camera_summary_plot(
 
     # Trim the beginning and end of the geodataframes
     if trim:
-        gdf_cam1 = trim_gdf(
-            gdf_cam1,
-            near_zero_tolerance=near_zero_tolerance,
-            trim_percentage=trim_percentage,
-        )
-        if cam2_list:
-            gdf_cam2 = trim_gdf(
-                gdf_cam2,
+        # When position changes are all zero, the trimming will fail
+        # https://github.com/uw-cryo/asp_plot/issues/54
+        if np.sum(gdf_cam1.position_diff_magnitude > 0) == 0:
+            print(
+                "\nWarning: No data to trim from the beginning and end of the camera files. Position changes are all zero.\n"
+            )
+            trim = False
+        else:
+            gdf_cam1 = trim_gdf(
+                gdf_cam1,
                 near_zero_tolerance=near_zero_tolerance,
                 trim_percentage=trim_percentage,
             )
+            if cam2_list:
+                gdf_cam2 = trim_gdf(
+                    gdf_cam2,
+                    near_zero_tolerance=near_zero_tolerance,
+                    trim_percentage=trim_percentage,
+                )
 
     # Calculate colorbar ranges
     position_values = gdf_cam1.position_diff_magnitude[
@@ -323,26 +331,21 @@ def csm_camera_summary_plot(
     angular_values = gdf_cam1.angular_diff_magnitude[
         gdf_cam1.angular_diff_magnitude > 0
     ]
-    cam1_position_vmin, cam1_position_vmax = np.percentile(
-        position_values, [0, upper_magnitude_percentile]
-    )
-    cam1_angular_vmin, cam1_angular_vmax = np.percentile(
-        angular_values, [0, upper_magnitude_percentile]
-    )
-
-    if cam2_list:
-        position_values = gdf_cam2.position_diff_magnitude[
-            gdf_cam2.position_diff_magnitude > 0
-        ]
-        angular_values = gdf_cam2.angular_diff_magnitude[
-            gdf_cam2.angular_diff_magnitude > 0
-        ]
-        cam2_position_vmin, cam2_position_vmax = np.percentile(
+    # When position or angular changes are all zero, the percentile calculation will fail
+    # https://github.com/uw-cryo/asp_plot/issues/54
+    try:
+        cam1_position_vmin, cam1_position_vmax = np.percentile(
             position_values, [0, upper_magnitude_percentile]
         )
-        cam2_angular_vmin, cam2_angular_vmax = np.percentile(
+    except IndexError:
+        cam1_position_vmin, cam1_position_vmax = 0, 0
+
+    try:
+        cam1_angular_vmin, cam1_angular_vmax = np.percentile(
             angular_values, [0, upper_magnitude_percentile]
         )
+    except IndexError:
+        cam1_angular_vmin, cam1_angular_vmax = 0, 0
 
     if upper_magnitude_percentile == 100:
         extend = "neither"