oceantracker 0.4.1__tar.gz

oceantracker-0.4.1/LICENSE.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 oceantracker
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

oceantracker-0.4.1/MANIFEST.in

@@ -0,0 +1,2 @@
+include README.md
+include LICENSE.md

oceantracker-0.4.1/PKG-INFO

@@ -0,0 +1,65 @@
+Metadata-Version: 2.2
+Name: oceantracker
+Version: 0.4.1
+Summary: Fast offline Lagrangian particle tracking in the Ocean
+Home-page: https://oceantracker.github.io/oceantracker/
+Author: Ross Vennell & Laurin Steidle
+Author-email: ross.vennell@cawthron.org.nz
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Oceanography
+Requires-Python: >=3.10,<3.12
+License-File: LICENSE.md
+Requires-Dist: numpy<2
+Requires-Dist: numba
+Requires-Dist: netcdf4
+Requires-Dist: xarray
+Requires-Dist: scipy
+Requires-Dist: pyproj
+Requires-Dist: setuptools
+Requires-Dist: psutil
+Requires-Dist: matplotlib
+Requires-Dist: pyyaml
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: home-page
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# OceanTracker
+
+## Fast particle tracking in structure and unstructured grids
+
+OceanTracker is a fast extendable code for offline particle tracking in structured and  unstructured grids [1].
+
+OceanTracker currently  supports strctured grid ROMs and GLORYS(NEMO/COPERNICUS) 
+for fixed z level vertical grids and unstructured grids of SCHISM and DELFT3D-FM  
+
+OceanTracker’s speed enables millions of particles to be simulated in unstructured grids. This significantly increases 
+the range of particle behaviours that can be modeled and the quality of statistics derived from the particles. 
+To eliminate the need to store and wade through the analysis of vast volumes of recorded particle tracks, the code has 
+the ability to calculate statistics on the fly, such as heat maps and connectivity between regions.
+
+OceanTracker code is highly flexible and extendable by the user, whether run by a new user with a text file of 
+parameters, or by an expert adding their specialised code for novel particle behaviours or statistics, to the computational pipe line
+
+
+[More about Ocean tracker and gallery](https://oceantracker.github.io/oceantracker/)
+
+[Installing Oceantracker](https://oceantracker.github.io/oceantracker/_build/html/info/installing.html)
+
+[Oceantracker User Guide](https://oceantracker.github.io/oceantracker/_build/html/info/users_guide.html)
+  
+[Running Oceantracker ](https://oceantracker.github.io/oceantracker/_build/html/info/running_ocean_tracker.html)
+
+[Howto tutorials](https://oceantracker.github.io/oceantracker/_build/html/info/how_to.html)
+
+  
+[1] Vennell, R., Scheel, M., Weppe, S., Knight, B. and Smeaton, M., 2021. Fast lagrangian particle tracking in unstructured ocean model grids , Ocean Dynamics, 71(4), pp.423-437.

oceantracker-0.4.1/README.md

@@ -0,0 +1,30 @@
+# OceanTracker
+
+## Fast particle tracking in structure and unstructured grids
+
+OceanTracker is a fast extendable code for offline particle tracking in structured and  unstructured grids [1].
+
+OceanTracker currently  supports strctured grid ROMs and GLORYS(NEMO/COPERNICUS) 
+for fixed z level vertical grids and unstructured grids of SCHISM and DELFT3D-FM  
+
+OceanTracker’s speed enables millions of particles to be simulated in unstructured grids. This significantly increases 
+the range of particle behaviours that can be modeled and the quality of statistics derived from the particles. 
+To eliminate the need to store and wade through the analysis of vast volumes of recorded particle tracks, the code has 
+the ability to calculate statistics on the fly, such as heat maps and connectivity between regions.
+
+OceanTracker code is highly flexible and extendable by the user, whether run by a new user with a text file of 
+parameters, or by an expert adding their specialised code for novel particle behaviours or statistics, to the computational pipe line
+
+
+[More about Ocean tracker and gallery](https://oceantracker.github.io/oceantracker/)
+
+[Installing Oceantracker](https://oceantracker.github.io/oceantracker/_build/html/info/installing.html)
+
+[Oceantracker User Guide](https://oceantracker.github.io/oceantracker/_build/html/info/users_guide.html)
+  
+[Running Oceantracker ](https://oceantracker.github.io/oceantracker/_build/html/info/running_ocean_tracker.html)
+
+[Howto tutorials](https://oceantracker.github.io/oceantracker/_build/html/info/how_to.html)
+
+  
+[1] Vennell, R., Scheel, M., Weppe, S., Knight, B. and Smeaton, M., 2021. Fast lagrangian particle tracking in unstructured ocean model grids , Ocean Dynamics, 71(4), pp.423-437.

oceantracker-0.4.1/dev/benchmark_timings/benchmark_computer.py

@@ -0,0 +1,36 @@
+from oceantracker.main import OceanTracker
+import argparse
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--datadir',type=str)
+    parser.add_argument('--root_output_dir', type=str)
+    args = parser.parse_args()
+
+    if args.datadir:
+        input_dir = args.datadir
+    else:
+        input_dir = r'G:\Hindcasts_large\OceanNumNZ-2022-06-20\final_version\2017\01'
+
+    if args.root_output_dir:
+        root_output_dir = args.root_output_dir
+    else:
+        root_output_dir = r'F:\OceanTrackerOutput\bench_marks'
+
+    ot = OceanTracker()
+    ot.settings(time_step=15*60, use_A_Z_profile=False, write_tracks=False,
+                max_run_duration= 10*24*3600,
+                root_output_dir=root_output_dir, output_file_base='benchmark_v01')
+    ot.add_class('reader',  input_dir = input_dir,    file_mask = 'NZfinite*.nc',
+                 time_buffer_size=12)
+
+    x0 = [-35.922300421719214, 174.665532083399]  # hen and chickens, in outer grid
+    ot.add_class('release_groups', name='my_release_point',  # user must provide a name for release group
+                 points=[[1838293.4656, 5940629.8263]],
+                 release_interval=0,  # seconds between releasing particles
+                 pulse_size=10**6,  # number of particles released each release_interval
+                 )
+
+    ot.run()
+

oceantracker-0.4.1/dev/bug_hunting/Debugg_codes/Connectivity_between_farms_RV.py

@@ -0,0 +1,93 @@
+# Code to estimate connectivity between mussel farms and potential spat collection areas
+# Date: 08/11/2024
+# HPC edition
+
+# Protocol: release particles everyday from mussel farms during the reproductive season and advect them for 35 days. Settlement occurs after 21 days.
+# Use the version 0.4_1 of Oceantracker
+
+## Import packages ##
+from oceantracker.main import OceanTracker
+from oceantracker.util import yaml_util, cord_transforms
+from os import path
+import numpy as np
+import argparse
+import shapely.vectorized
+from shapely.geometry import Point
+from datetime import datetime
+import geopandas as gpd # Need to be installed with pip command
+
+## Model selection ##
+ot = OceanTracker()
+## Set basic settings ##
+ot.settings(output_file_base =r'Connectivity_farms_GLM_OT_outputs',   # name used as base for output files
+            root_output_dir = '/hpcfreenas/romain/GM_connectivity_MBS',
+            #root_output_dir = r'D:\OceanTrackerOutput\bug_hunting',
+            add_date_to_run_output_dir = True,
+            # Run parameters
+            backtracking = False,
+            time_step = 600.0, # in seconds, time-step of interpolation
+            # Parallel run
+            processors=6, # Set up parallel run on x processors
+            # Register outputs
+            screen_output_time_interval = 3600*24, # Only print a line on screen once a day
+            write_tracks = True,
+            )
+
+## Add reader for oceanographic data ##
+if __name__ == '__main__':
+    ot.add_class('reader',
+                 input_dir ='/hpcfreenas/hindcast/UpperSouthIsland/HABs2018benk/nogrowth',  # folder to search for hindcast files
+                 #input_dir = r'D:\Hindcasts\UpperSouthIsland\2018_benHABS\nogrowth',
+                 # folder to search for hindcast files
+                 file_mask = 'Nydia*.nc', # hindcast netcdf file
+                 time_buffer_size = 20,
+                 )
+    
+        ## Add physical properties ##
+    ot.add_class('resuspension', critical_friction_velocity = 0.001) # only re-suspend particles if friction vel. exceeds this value
+    ot.add_class('dispersion',
+                 A_H = 0.2, # Horizontal diffusion coeff
+                 A_V = 0.001, # Vertical diffusion coeff
+                 )
+
+    # Import polygons in Marlborough Sounds
+    import json
+    with open('Farms_MBS_GB_TB.geojson') as f:
+        Marlb_polygons = json.load(f)
+    polygon_list = list()
+    for feature in Marlb_polygons['features']:
+        polygon_coords = feature['geometry']['coordinates'][0][0]
+        new_poly = dict(points=polygon_coords, user_polygonID=feature['properties']['id'])
+        polygon_list.append(new_poly)
+
+    for s_idx, s in enumerate(polygon_list[0:-1], start=0):
+        ot.add_class('release_groups', name = str(s_idx),                         # user must provide a name for group first
+                    class_name = 'oceantracker.release_groups.polygon_release.PolygonRelease',
+                    points = polygon_list[s_idx].get("points"),                               # must be an N by 2 or 3 or list, convertible to a numpy array
+                    release_interval = 3600,        # seconds between releasing particles - in weeks currently
+                    pulse_size = 10,                    # number of particles released each release_interval
+                    start = datetime.strptime('01/02/2018', "%d/%m/%Y").isoformat(),
+                    end = datetime.strptime('01/06/2018', "%d/%m/%Y").isoformat(),
+                    max_age = 24*3600*35,
+                    case = s_idx,
+                    )
+
+    ot.add_class('particle_statistics', name='Connectivity_matrix',
+            class_name='PolygonStats2D_ageBased',
+            start=datetime.strptime('01/02/2018', "%d/%m/%Y").isoformat(),
+            end=datetime.strptime('01/08/2018', "%d/%m/%Y").isoformat(),
+            polygon_list=polygon_list,
+            min_age_to_bin = 21*24*3600,
+            update_interval= 24 * 3600,
+            case = s_idx,
+            )
+
+
+    ## Print parameters to check run ##
+    print(ot.params)
+    #import json
+    #print(json.dumps(ot.params, indent=4))
+    ## Run oceantracker ##
+    # as helper "ot" has set params above, simply run it
+    case_info_file_name = ot.run()
+    print('case file name=', case_info_file_name)

oceantracker-0.4.1/dev/bug_hunting/reproduce_bug.py

@@ -0,0 +1,401 @@
+# %% [markdown]
+# ## Preparation
+
+# %%
+from oceantracker.main import run
+from read_oceantracker.python import load_output_files
+from plot_oceantracker import plot_tracks
+from plot_oceantracker import plot_statistics
+
+import os
+import numpy as np
+import matplotlib.pyplot as plt
+
+# %% [markdown]
+# ### Model
+
+# %%
+model_name = "shit_predictor_2000_debug"
+
+# I/O
+input_dir = "/hpcfreenas/hindcast/OceanNumNZ-2022-06-20/final_version/2019/"
+output_dir = "/hpcfreenas/laurin/malcolms_marvelous_shit_predictor_output"
+
+# time parameters
+model_duration = 60*24*3600
+data_time_step = 3600
+model_time_step = 60
+model_steps = int(model_duration/model_time_step)
+
+# particle release
+pulse_size = 10
+release_interval = 3600
+
+# output
+# i think there is no good reason to have the output step size
+# smaller then the model step size though.
+output_time_step = model_time_step
+output_steps = int(model_duration/output_time_step)
+grid_size = 100
+
+
+# %% [markdown]
+# ### Flux
+
+# %%
+# feed
+feed_input_per_year_per_pen = 2.214e3
+
+feed_sinking_velocity = 0.095
+feed_decay_rate = 0.086/(24*3600) # per day
+feed_critical_friction_velocity = 0.015
+
+feed_water_ratio = 0.09
+feed_digestion_ratio = 0.85
+feed_waste_ratio = 0.03
+
+# shit
+
+shit_sinking_velocity = 0.032
+shit_decay_rate = 0.086/(24*3600) # per day
+shit_critical_friction_velocity = 0.009
+
+# fluxes
+feed_mass_dry = (1-feed_water_ratio)*feed_input_per_year_per_pen
+feed_consumed = (1-feed_waste_ratio)*feed_mass_dry
+feed_wasted = feed_waste_ratio*feed_mass_dry
+
+shit_produced = (1-feed_digestion_ratio)*feed_consumed
+
+flux_feed = feed_wasted
+flux_shit = shit_produced
+
+#
+pulses_per_year = int(3600*24*365/release_interval)
+inital_mass_per_pertical = 1/(pulse_size*pulses_per_year)
+
+initial_feed_particle_mass = flux_feed*inital_mass_per_pertical
+initial_shit_particle_mass = flux_shit*inital_mass_per_pertical
+
+
+
+# %% [markdown]
+def create_case_dict_for_pen(pen_position,depth_range,
+                             pulse_size=pulse_size,
+                             release_interval=release_interval,
+                             feed_sinking_velocity=feed_sinking_velocity,
+                             feed_decay_rate=feed_decay_rate,
+                             shit_sinking_velocity=shit_sinking_velocity,
+                             shit_decay_rate=shit_decay_rate):
+    """
+    Takes a list of [x_min,x_max,y_min,y_max] for the pen_position
+    and a list of [z_top,z_bottom] for the depth_range and returns a
+    dict
+    """
+    x_min,x_max,y_min,y_max = pen_position
+    z_top,z_bottom = depth_range
+
+    case = [
+        # food pellets
+        {
+            "particle_release_groups": [
+                {
+                    "class_name": "oceantracker.release_groups.polygon_release.PolygonRelease",
+                    "points": [
+                        [
+                            x_min,
+                            y_min
+                        ],
+                        [
+                            x_max,
+                            y_min
+                        ],
+                        [
+                            x_max,
+                            y_max
+                        ],
+                        [
+                            x_min,
+                            y_max
+                        ]
+                    ],
+                    "pulse_size": pulse_size,
+                    "release_interval": release_interval,
+                    "z_range": [z_top, z_bottom]
+                }
+            ],
+            "velocity_modifiers": [
+                {
+                    "class_name": "oceantracker.velocity_modifiers.terminal_velocity.TerminalVelocity",
+                    "mean": feed_sinking_velocity
+                }
+            ],
+            "particle_properties": [
+                {
+                    "class_name": "oceantracker.particle_properties.age_decay.AgeDecay",
+                    "decay_time_scale": 1/feed_decay_rate
+                }
+            ],
+            "trajectory_modifiers": [
+                {
+                    "class_name": "oceantracker.resuspension.BasicResuspension",
+                    "critical_friction_velocity": feed_critical_friction_velocity
+                }
+            ],
+        },
+        # shit pellets
+        {
+            "particle_release_groups": [
+                {
+                    "class_name": "oceantracker.release_groups.polygon_release.PolygonRelease",
+                    "points": [
+                        [
+                            x_min,
+                            y_min
+                        ],
+                        [
+                            x_max,
+                            y_min
+                        ],
+                        [
+                            x_max,
+                            y_max
+                        ],
+                        [
+                            x_min,
+                            y_max
+                        ]
+                    ],
+                    "pulse_size": pulse_size,
+                    "release_interval": release_interval,
+                    "z_range": [z_top, z_bottom]
+                }
+            ],
+            "velocity_modifiers": [
+                {
+                    "class_name": "oceantracker.velocity_modifiers.terminal_velocity.TerminalVelocity",
+                    "mean": shit_sinking_velocity
+                }
+            ],
+            "particle_properties": [
+                {
+                    "class_name": "oceantracker.particle_properties.age_decay.AgeDecay",
+                    "decay_time_scale": 1/shit_decay_rate
+                }
+            ],
+            "trajectory_modifiers": [
+                {
+                    "class_name": "oceantracker.resuspension.BasicResuspension",
+                    "critical_friction_velocity": shit_critical_friction_velocity
+                }
+            ],
+        }
+    ]
+
+    return case
+
+# %%
+list_of_pens = [
+    {
+        "name": "pen_1",
+        "pen_position": [
+            1689853, # x_min
+            1697645, # x_max
+            5476738, # y_min
+            5491327  # y_max
+        ],
+        "depth_range": [0, -30]
+    }
+]
+
+# %%
+params = {
+    "shared_params": {
+        "compact_mode": True,
+        "output_file_base": model_name,
+        "root_output_dir": output_dir,
+        "processors": 1,
+    },
+    "reader": {
+        "class_name": "oceantracker.reader.schism_reader.SCHISMreader",
+        "depth_average": False,
+        "file_mask": "NZfinite*.nc",
+        "input_dir": input_dir,
+        "search_sub_dirs": True,
+        # "hgrid_file_name": "/hpcfreenas/hindcast/OceanNumNZ-2022-06-20/final_version/hgridNZ_run.gr3",
+    },
+    "base_case_params": {
+        "dispersion": {
+            "A_H": 0.1,
+            "A_V": 0.001
+        },
+        "fields": [
+            {
+                "class_name": "oceantracker.fields.friction_velocity.FrictionVelocityFromNearSeaBedVelocity"
+            }
+        ],
+        "run_params": {
+            "block_dry_cells": True,
+            "use_open_boundary": True,
+            "write_tracks": True,
+            "duration": model_duration,
+        },
+        "solver": {
+            "n_sub_steps": int(data_time_step/model_time_step),
+            "screen_output_step_count": int(data_time_step/model_time_step)
+        },
+        "tracks_writer": {
+            "class_name": "oceantracker.tracks_writer.track_writer_compact.FlatTrackWriter",
+            "output_step_writer": 3600,
+            "write_dry_cell_flag": True
+        },
+        "particle_statistics": [
+            {
+                "class_name": "oceantracker.particle_statistics.gridded_statistics2D.GriddedStats2D_timeBased",
+                "calculation_interval": output_time_step,
+                "count_status_in_range": ["on_bottom", "on_bottom"],
+                "particle_property_list": [
+                    "age_decay"
+                ],
+                "grid_size": [
+                    grid_size,
+                    grid_size
+                ]
+
+            }
+        ],
+    },
+}
+
+cases = []
+for pen in list_of_pens:
+    case = create_case_dict_for_pen(pen["pen_position"],
+                                    pen["depth_range"])
+    cases.append(case[0])
+    cases.append(case[1])
+
+params["case_list"] = cases
+
+# %% [markdown]
+# ## Run Model
+
+# %%
+print('Estimated output data size per case')
+print('-----------------------------------')
+
+print(f'\t{output_steps*grid_size**2*8*3/1024**3:.1f} GB')
+
+
+
+time_per_particle_per_time_step = 2e-5
+releases = model_duration/release_interval
+time_steps_between_releases = release_interval/model_time_step
+
+active_particles_per_time_step = np.array([0])
+for ii in range(int(releases)):
+    active_particles_per_time_step= np.append(active_particles_per_time_step,
+               [active_particles_per_time_step[-1] + pulse_size]*int(time_steps_between_releases))
+
+estimated_run_time = np.sum(active_particles_per_time_step*time_per_particle_per_time_step) #\wo io
+
+print('Estimated compute time per case')
+print('-----------------------------------')
+print(f'\t{estimated_run_time/60/60:.0f}h')
+
+
+
+
+# %%
+path_to_output = os.path.join(
+    os.path.abspath(os.path.curdir),
+    params['shared_params']['root_output_dir'],
+    params['shared_params']['output_file_base'],
+    params['shared_params']['output_file_base']+'_runInfo.json')
+
+# %%
+try:
+    load_output_files.get_case_info_file_from_run_file(path_to_output)
+except:
+    path_to_output = run(params)[0]
+
+# %% [markdown]
+# ## Post-Processing
+
+# %% [markdown]
+# ### Checking decay
+
+# %%
+decay = 0.087
+
+t = np.arange(0, 3600*24*60, 3600*24)
+
+x = np.exp(-t*feed_decay_rate)
+
+plt.figure()
+plt.plot(t/3600/24,x)
+plt.grid(True, which="both")
+plt.hlines(0.01, 0, 60, linestyles='dashed')
+
+
+# %% [markdown]
+# ### Feed
+
+# %%
+# loading feed case
+case = load_output_files.get_case_info_file_from_run_file(path_to_output, ncase = 1)
+
+# deposition in the model duration
+stats_data = load_output_files.load_stats_data(case,nsequence=0)
+
+# %%
+# shit unit masses present at time step at cell on bottom
+feed_unit_masses_present = stats_data['sum_age_decay']#['time', 'pen', 'x', 'y']
+feed_unit_masses_present[np.isnan(feed_unit_masses_present)] = 0
+
+# scale this with the gauge mass per particle
+feed_present = feed_unit_masses_present * initial_shit_particle_mass
+
+# calculate how much of this has decayed in this time step on bottom
+feed_decayed = feed_present * feed_decay_rate * model_time_step
+
+# %%
+print(f"Total feed input into system: {feed_mass_dry:.2f} kg/a")
+print(f"Total feed not consumed: {flux_feed:.2f} kg/a")
+print(f"Total feed decayed on bottom (\wo equilibrium adjustment): {np.sum(feed_decayed):.2f} kg/a")
+print(f"Total feed decayed on bottom (\w  equilibrium adjustment): {np.sum(np.sum(feed_decayed[-1,0]*feed_decayed.shape[0])):.2f} kg/a")
+
+# %%
+plt.imshow(np.max(stats_data['sum_age_decay'][:,0,:,:],axis=(0)))
+plt.colorbar()
+
+# %% [markdown]
+# ### Shit
+
+# %%
+# load shit case
+case = load_output_files.get_case_info_file_from_run_file(path_to_output, ncase = 2)
+
+# deposition in the model duration
+stats_data = load_output_files.load_stats_data(case, var_list=['age_decay'], nsequence=0)
+
+# %%
+# shit unit masses present at time step at cell on bottom
+shit_unit_masses_present = stats_data['count']#['time', 'pen', 'x', 'y']
+shit_unit_masses_present[np.isnan(shit_unit_masses_present)] = 0
+
+# scale this with the gauge mass per particle
+shit_present = shit_unit_masses_present * initial_shit_particle_mass
+
+# calculate how much of this has decayed in this time step on bottom
+shit_decayed = shit_present * shit_decay_rate * model_time_step
+
+# %%
+print(f"Total feed input into system: {feed_mass_dry:.2f} kg/a")
+print(f"Total shit input into system: {flux_shit:.2f} kg/a")
+print(f"Total shit decayed on bottom (\wo equilibrium adjustment): {np.sum(shit_decayed):.2f} kg/a")
+print(f"Total shit decayed on bottom (\w  equilibrium adjustment): {np.sum(np.sum(shit_decayed[-1,0]*shit_decayed.shape[0])):.2f} kg/a")
+
+
+# %%
+plt.imshow(np.sum(shit_decayed[:,0,:,:],axis=0))
+plt.colorbar(label="Shit decayed on bottom (kg/a)")