loone-data-prep 0.1.4__tar.gz → 0.1.6__tar.gz

{loone_data_prep-0.1.4 → loone_data_prep-0.1.6}/LICENSE

@@ -1,6 +1,6 @@
 BSD-3-Clause License
 
-Copyright (c) 2023 Aquaveo
+Copyright (c) 2024 University of South Florida
 
 Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
 

loone_data_prep-0.1.6/PKG-INFO

@@ -0,0 +1,112 @@
+Metadata-Version: 2.1
+Name: loone_data_prep
+Version: 0.1.6
+Summary: Prepare data to run the LOONE model.
+Author-email: Osama Tarabih <osamatarabih@usf.edu>
+Maintainer-email: Michael Souffront <msouffront@aquaveo.com>, James Dolinar <jdolinar@aquaveo.com>
+License: BSD-3-Clause License
+        
+        Copyright (c) 2024 University of South Florida
+        
+        Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+        
+        - Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+        - Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+        - Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+        
+        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+        
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: rpy2
+Requires-Dist: retry
+Requires-Dist: pandas
+Requires-Dist: scipy
+Requires-Dist: geoglows==0.27.1
+
+LOONE_DATA_PREP
+# LOONE_DATA_PREP
+
+Prepare data for the LOONE water quality model.
+
+Line to the LOONE model: [https://pypi.org/project/loone](https://pypi.org/project/loone)
+Link to LOONE model repository: [https://github.com/Aquaveo/LOONE](https://github.com/Aquaveo/LOONE)
+
+## Prerequisites:
+
+* R ([https://www.r-project.org/](https://www.r-project.org/))
+* R packages: dbhydroR, rio, dplyr
+
+## Installation:
+
+```bash
+pip install loone_data_prep
+```
+
+### Development Installation:
+
+```bash
+cd /path/to/loone_data_prep/repo
+pip install -e .
+```
+
+### Examples
+
+**From the command line:**
+
+```bash
+# Get flow data
+python -m loone_data_prep.flow_data.get_inflows /path/to/workspace/
+python -m loone_data_prep.flow_data.get_outflows /path/to/workspace/
+python -m loone_data_prep.flow_data.S65E_total /path/to/workspace/
+
+# Get water quality data
+python -m loone_data_prep.water_quality_data.get_inflows /path/to/workspace/
+python -m loone_data_prep.water_quality_data.get_lake_wq /path/to/workspace/
+
+# Get weather data
+python -m loone_data_prep.weather_data.get_all /path/to/workspace/
+
+# Get water level
+python -m loone_data_prep.water_level_data.get_all /path/to/workspace/
+
+# Interpolate data
+python -m loone_data_prep.utils interp_all /path/to/workspace/
+
+# Prepare data for LOONE
+python -m loone_data_prep.LOONE_DATA_PREP /path/to/workspace/ /path/to/output/directory/
+```
+
+**From Python:**
+
+```python
+from loone_data_prep.utils import get_dbkeys
+from loone_data_prep.water_level_data import hydro
+from loone_data_prep import LOONE_DATA_PREP
+
+input_dir = '/path/to/workspace/'
+output_dir = '/path/to/output/directory/'
+
+# Get dbkeys for water level data
+dbkeys = get_dbkeys(
+    station_ids=["L001", "L005", "L006", "LZ40"],
+    category="SW",
+    param="STG",
+    stat="MEAN",
+    recorder="CR10",
+    freq="DA",
+    detail_level="dbkey"
+)
+
+# Get water level data
+hydro.get(
+    workspace=input_dir,
+    name="lo_stage",
+    dbkeys=dbkeys,
+    date_min="1950-01-01",
+    date_max="2023-03-31"
+)
+
+# Prepare data for LOONE
+LOONE_DATA_PREP(input_dir, output_dir)
+```

{loone_data_prep-0.1.4 → loone_data_prep-0.1.6}/README.md

@@ -3,7 +3,8 @@ LOONE_DATA_PREP
 
 Prepare data for the LOONE water quality model.
 
-Link to LOONE model repository: [https://github.com/osamatarabih/LOONE](https://github.com/osamatarabih/LOONE)
+Line to the LOONE model: [https://pypi.org/project/loone](https://pypi.org/project/loone)
+Link to LOONE model repository: [https://github.com/Aquaveo/LOONE](https://github.com/Aquaveo/LOONE)
 
 ## Prerequisites:
 
@@ -13,8 +14,14 @@ Link to LOONE model repository: [https://github.com/osamatarabih/LOONE](https://
 ## Installation:
 
 ```bash
-cd /path/to/loone_data_prep/
-pip install .
+pip install loone_data_prep
+```
+
+### Development Installation:
+
+```bash
+cd /path/to/loone_data_prep/repo
+pip install -e .
 ```
 
 ### Examples

{loone_data_prep-0.1.4 → loone_data_prep-0.1.6}/loone_data_prep/LOONE_DATA_PREP.py

@@ -418,7 +418,7 @@ def main(input_dir: str, output_dir: str) -> None:
     Stg_3A28 = pd.read_csv(f'{input_dir}/Stg_3A28.csv')
     Stg_3A28 = DF_Date_Range(Stg_3A28, St_Yr, St_M, St_D, En_Yr, En_M, En_D)
     WCA_Stg = pd.DataFrame(Stg_3A28['date'], columns=['date'])
-    WCA_Stg['3A-NW'] = Stg_3ANW['3A-NW_STG_ft NGVD29'].values
+    WCA_Stg['3A-NW'] = Stg_3ANW.iloc[:, -1].values
     WCA_Stg['2A-17'] = Stg_2A17.iloc[:, -1].values
     WCA_Stg['3A-3'] = Stg_3A3.iloc[:, -1].values
     WCA_Stg['3A-4'] = Stg_3A4.iloc[:, -1].values

{loone_data_prep-0.1.4 → loone_data_prep-0.1.6}/loone_data_prep/flow_data/forecast_bias_correction.py

@@ -1,4 +1,5 @@
 import sys
+import os
 import pandas as pd
 import geoglows
 
@@ -12,6 +13,7 @@ def get_bias_corrected_data(
     observed_data_path: str,
     station_ensembles: pd.DataFrame,
     station_stats: pd.DataFrame,
+    cache_path: str = None,
 ) -> dict:
     # Load the observed data from a CSV file
     observed_data = pd.read_csv(
@@ -34,7 +36,23 @@ def get_bias_corrected_data(
     prepared_od = prep_observed_data(observed_data)
 
     # Get the historical simulation data for the given reach ID
-    historical_data = geoglows.streamflow.historic_simulation(reach_id)
+    historical_data = None
+    
+    if cache_path is None:
+        historical_data = geoglows.streamflow.historic_simulation(reach_id)
+    else:
+        # Create the geoglows cache directory if it doesn't exist
+        geoglows_cache_path = os.path.join(cache_path, 'geoglows_cache')
+        if not os.path.exists(geoglows_cache_path):
+            os.makedirs(geoglows_cache_path)
+        
+        # Check if the historical simulation data is already cached
+        if os.path.exists(os.path.join(geoglows_cache_path, f'{reach_id}_historic_simulation.csv')):
+            historical_data = pd.read_csv(os.path.join(geoglows_cache_path, f'{reach_id}_historic_simulation.csv'), index_col=0)
+            historical_data.index = pd.to_datetime(historical_data.index)
+        else:
+            historical_data = geoglows.streamflow.historic_simulation(reach_id)
+            historical_data.to_csv(os.path.join(geoglows_cache_path, f'{reach_id}_historic_simulation.csv'))
 
     # Correct the forecast bias in the station ensembles
     station_ensembles = geoglows.bias.correct_forecast(

{loone_data_prep-0.1.4 → loone_data_prep-0.1.6}/loone_data_prep/flow_data/get_forecast_flows.py

@@ -366,6 +366,7 @@ def main(
     forecast_date: str = FORECAST_DATE,
     bias_corrected: bool = False,
     observed_data_dir: str | None = None,
+    cache_path: str | None = None,
 ):
     """Downloads the flow forecasts for the given station ids and writes them
         out as .csv files.
@@ -379,6 +380,8 @@ def main(
             Default is False.
         observed_data_dir (str): The path to the observed flow data directory
             (only needed if bias_corrected is True).
+        cache_path (str): The path to the cache directory for geoglows data. 
+            Should hold a directory named geoglows_cache that holds the cached files. Use None to not use a cache.
     """
     # Local Variables
     reach_ids = {}
@@ -428,6 +431,7 @@ def main(
                     observed_data_path,
                     station_ensembles,
                     station_stats,
+                    cache_path,
                 )
 
         ensembles_to_csv(

{loone_data_prep-0.1.4 → loone_data_prep-0.1.6}/loone_data_prep/utils.py

@@ -4,6 +4,7 @@ import datetime
 import math
 from glob import glob
 from calendar import monthrange
+import traceback
 import numpy as np
 import pandas as pd
 from retry import retry
@@ -635,8 +636,9 @@ def nutrient_prediction(
                 predicted_column.columns = [column_name]
 
                 prediction_columns.append(predicted_column)
-            except RuntimeWarning:
-                breakpoint()
+            except RuntimeWarning as e:
+                print(f"Unexpected RuntimeWarning: {str(e)}")
+                traceback.print_exc()
 
         # Concat individual ensemble columns together into one pandas DataFrame
         out_dataframe = pd.concat(objs=prediction_columns, axis="columns")

loone_data_prep-0.1.6/loone_data_prep.egg-info/PKG-INFO

@@ -0,0 +1,112 @@
+Metadata-Version: 2.1
+Name: loone_data_prep
+Version: 0.1.6
+Summary: Prepare data to run the LOONE model.
+Author-email: Osama Tarabih <osamatarabih@usf.edu>
+Maintainer-email: Michael Souffront <msouffront@aquaveo.com>, James Dolinar <jdolinar@aquaveo.com>
+License: BSD-3-Clause License
+        
+        Copyright (c) 2024 University of South Florida
+        
+        Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+        
+        - Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+        - Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+        - Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+        
+        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+        
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: rpy2
+Requires-Dist: retry
+Requires-Dist: pandas
+Requires-Dist: scipy
+Requires-Dist: geoglows==0.27.1
+
+LOONE_DATA_PREP
+# LOONE_DATA_PREP
+
+Prepare data for the LOONE water quality model.
+
+Line to the LOONE model: [https://pypi.org/project/loone](https://pypi.org/project/loone)
+Link to LOONE model repository: [https://github.com/Aquaveo/LOONE](https://github.com/Aquaveo/LOONE)
+
+## Prerequisites:
+
+* R ([https://www.r-project.org/](https://www.r-project.org/))
+* R packages: dbhydroR, rio, dplyr
+
+## Installation:
+
+```bash
+pip install loone_data_prep
+```
+
+### Development Installation:
+
+```bash
+cd /path/to/loone_data_prep/repo
+pip install -e .
+```
+
+### Examples
+
+**From the command line:**
+
+```bash
+# Get flow data
+python -m loone_data_prep.flow_data.get_inflows /path/to/workspace/
+python -m loone_data_prep.flow_data.get_outflows /path/to/workspace/
+python -m loone_data_prep.flow_data.S65E_total /path/to/workspace/
+
+# Get water quality data
+python -m loone_data_prep.water_quality_data.get_inflows /path/to/workspace/
+python -m loone_data_prep.water_quality_data.get_lake_wq /path/to/workspace/
+
+# Get weather data
+python -m loone_data_prep.weather_data.get_all /path/to/workspace/
+
+# Get water level
+python -m loone_data_prep.water_level_data.get_all /path/to/workspace/
+
+# Interpolate data
+python -m loone_data_prep.utils interp_all /path/to/workspace/
+
+# Prepare data for LOONE
+python -m loone_data_prep.LOONE_DATA_PREP /path/to/workspace/ /path/to/output/directory/
+```
+
+**From Python:**
+
+```python
+from loone_data_prep.utils import get_dbkeys
+from loone_data_prep.water_level_data import hydro
+from loone_data_prep import LOONE_DATA_PREP
+
+input_dir = '/path/to/workspace/'
+output_dir = '/path/to/output/directory/'
+
+# Get dbkeys for water level data
+dbkeys = get_dbkeys(
+    station_ids=["L001", "L005", "L006", "LZ40"],
+    category="SW",
+    param="STG",
+    stat="MEAN",
+    recorder="CR10",
+    freq="DA",
+    detail_level="dbkey"
+)
+
+# Get water level data
+hydro.get(
+    workspace=input_dir,
+    name="lo_stage",
+    dbkeys=dbkeys,
+    date_min="1950-01-01",
+    date_max="2023-03-31"
+)
+
+# Prepare data for LOONE
+LOONE_DATA_PREP(input_dir, output_dir)
+```

{loone_data_prep-0.1.4 → loone_data_prep-0.1.6}/loone_data_prep.egg-info/requires.txt

@@ -2,4 +2,4 @@ rpy2
 retry
 pandas
 scipy
-geoglows>=0.27.1
+geoglows==0.27.1

{loone_data_prep-0.1.4 → loone_data_prep-0.1.6}/pyproject.toml

@@ -4,12 +4,15 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "loone_data_prep"
-version = "0.1.4"
+version = "0.1.6"
 description = "Prepare data to run the LOONE model."
 readme = "README.md"
+license = { file = "LICENSE" }
 authors = [
+    { name = "Osama Tarabih", email = "osamatarabih@usf.edu" },
+]
+maintainers = [
     { name = "Michael Souffront", email = "msouffront@aquaveo.com" },
     { name = "James Dolinar", email = "jdolinar@aquaveo.com" },
-    { name = "Osama Tarabih", email = "osamatarabih@usf.edu" },
 ]
-dependencies = ["rpy2", "retry", "pandas", "scipy", "geoglows>=0.27.1"]
+dependencies = ["rpy2", "retry", "pandas", "scipy", "geoglows==0.27.1"]

loone_data_prep-0.1.4/PKG-INFO

@@ -1,92 +0,0 @@
-Metadata-Version: 2.1
-Name: loone_data_prep
-Version: 0.1.4
-Summary: Prepare data to run the LOONE model.
-Author-email: Michael Souffront <msouffront@aquaveo.com>, James Dolinar <jdolinar@aquaveo.com>, Osama Tarabih <osamatarabih@usf.edu>
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: rpy2
-Requires-Dist: retry
-Requires-Dist: pandas
-Requires-Dist: scipy
-Requires-Dist: geoglows>=0.27.1
-
-LOONE_DATA_PREP
-# LOONE_DATA_PREP
-
-Prepare data for the LOONE water quality model.
-
-Link to LOONE model repository: [https://github.com/osamatarabih/LOONE](https://github.com/osamatarabih/LOONE)
-
-## Prerequisites:
-
-* R ([https://www.r-project.org/](https://www.r-project.org/))
-* R packages: dbhydroR, rio, dplyr
-
-## Installation:
-
-```bash
-cd /path/to/loone_data_prep/
-pip install .
-```
-
-### Examples
-
-**From the command line:**
-
-```bash
-# Get flow data
-python -m loone_data_prep.flow_data.get_inflows /path/to/workspace/
-python -m loone_data_prep.flow_data.get_outflows /path/to/workspace/
-python -m loone_data_prep.flow_data.S65E_total /path/to/workspace/
-
-# Get water quality data
-python -m loone_data_prep.water_quality_data.get_inflows /path/to/workspace/
-python -m loone_data_prep.water_quality_data.get_lake_wq /path/to/workspace/
-
-# Get weather data
-python -m loone_data_prep.weather_data.get_all /path/to/workspace/
-
-# Get water level
-python -m loone_data_prep.water_level_data.get_all /path/to/workspace/
-
-# Interpolate data
-python -m loone_data_prep.utils interp_all /path/to/workspace/
-
-# Prepare data for LOONE
-python -m loone_data_prep.LOONE_DATA_PREP /path/to/workspace/ /path/to/output/directory/
-```
-
-**From Python:**
-
-```python
-from loone_data_prep.utils import get_dbkeys
-from loone_data_prep.water_level_data import hydro
-from loone_data_prep import LOONE_DATA_PREP
-
-input_dir = '/path/to/workspace/'
-output_dir = '/path/to/output/directory/'
-
-# Get dbkeys for water level data
-dbkeys = get_dbkeys(
-    station_ids=["L001", "L005", "L006", "LZ40"],
-    category="SW",
-    param="STG",
-    stat="MEAN",
-    recorder="CR10",
-    freq="DA",
-    detail_level="dbkey"
-)
-
-# Get water level data
-hydro.get(
-    workspace=input_dir,
-    name="lo_stage",
-    dbkeys=dbkeys,
-    date_min="1950-01-01",
-    date_max="2023-03-31"
-)
-
-# Prepare data for LOONE
-LOONE_DATA_PREP(input_dir, output_dir)
-```

loone_data_prep-0.1.4/loone_data_prep.egg-info/PKG-INFO

@@ -1,92 +0,0 @@
-Metadata-Version: 2.1
-Name: loone_data_prep
-Version: 0.1.4
-Summary: Prepare data to run the LOONE model.
-Author-email: Michael Souffront <msouffront@aquaveo.com>, James Dolinar <jdolinar@aquaveo.com>, Osama Tarabih <osamatarabih@usf.edu>
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: rpy2
-Requires-Dist: retry
-Requires-Dist: pandas
-Requires-Dist: scipy
-Requires-Dist: geoglows>=0.27.1
-
-LOONE_DATA_PREP
-# LOONE_DATA_PREP
-
-Prepare data for the LOONE water quality model.
-
-Link to LOONE model repository: [https://github.com/osamatarabih/LOONE](https://github.com/osamatarabih/LOONE)
-
-## Prerequisites:
-
-* R ([https://www.r-project.org/](https://www.r-project.org/))
-* R packages: dbhydroR, rio, dplyr
-
-## Installation:
-
-```bash
-cd /path/to/loone_data_prep/
-pip install .
-```
-
-### Examples
-
-**From the command line:**
-
-```bash
-# Get flow data
-python -m loone_data_prep.flow_data.get_inflows /path/to/workspace/
-python -m loone_data_prep.flow_data.get_outflows /path/to/workspace/
-python -m loone_data_prep.flow_data.S65E_total /path/to/workspace/
-
-# Get water quality data
-python -m loone_data_prep.water_quality_data.get_inflows /path/to/workspace/
-python -m loone_data_prep.water_quality_data.get_lake_wq /path/to/workspace/
-
-# Get weather data
-python -m loone_data_prep.weather_data.get_all /path/to/workspace/
-
-# Get water level
-python -m loone_data_prep.water_level_data.get_all /path/to/workspace/
-
-# Interpolate data
-python -m loone_data_prep.utils interp_all /path/to/workspace/
-
-# Prepare data for LOONE
-python -m loone_data_prep.LOONE_DATA_PREP /path/to/workspace/ /path/to/output/directory/
-```
-
-**From Python:**
-
-```python
-from loone_data_prep.utils import get_dbkeys
-from loone_data_prep.water_level_data import hydro
-from loone_data_prep import LOONE_DATA_PREP
-
-input_dir = '/path/to/workspace/'
-output_dir = '/path/to/output/directory/'
-
-# Get dbkeys for water level data
-dbkeys = get_dbkeys(
-    station_ids=["L001", "L005", "L006", "LZ40"],
-    category="SW",
-    param="STG",
-    stat="MEAN",
-    recorder="CR10",
-    freq="DA",
-    detail_level="dbkey"
-)
-
-# Get water level data
-hydro.get(
-    workspace=input_dir,
-    name="lo_stage",
-    dbkeys=dbkeys,
-    date_min="1950-01-01",
-    date_max="2023-03-31"
-)
-
-# Prepare data for LOONE
-LOONE_DATA_PREP(input_dir, output_dir)
-```