ngiab-data-preprocess 4.2.2__py3-none-any.whl → 4.4.0__py3-none-any.whl

ngiab_data_preprocess-4.4.0.dist-info/METADATA

@@ -0,0 +1,308 @@
+Metadata-Version: 2.4
+Name: ngiab_data_preprocess
+Version: 4.4.0
+Summary: Graphical Tools for creating Next Gen Water model input data.
+Author-email: Josh Cunningham <jcunningham8@ua.edu>
+Project-URL: Homepage, https://github.com/CIROH-UA/NGIAB_data_preprocess
+Project-URL: Issues, https://github.com/CIROH-UA/NGIAB_data_preprocess/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pyogrio>=0.7.2
+Requires-Dist: pyproj>=3.6.1
+Requires-Dist: Flask==3.0.2
+Requires-Dist: geopandas>=1.0.0
+Requires-Dist: requests==2.32.4
+Requires-Dist: igraph==0.11.4
+Requires-Dist: s3fs==2024.3.1
+Requires-Dist: xarray==2024.2.0
+Requires-Dist: zarr==2.17.1
+Requires-Dist: netCDF4>=1.6.5
+Requires-Dist: dask==2024.4.1
+Requires-Dist: dask[distributed]==2024.4.1
+Requires-Dist: h5netcdf==1.3.0
+Requires-Dist: exactextract==0.2.0
+Requires-Dist: numpy>=1.26.4
+Requires-Dist: tqdm==4.66.4
+Requires-Dist: rich==13.7.1
+Requires-Dist: colorama==0.4.6
+Requires-Dist: bokeh==3.5.1
+Requires-Dist: boto3
+Requires-Dist: numcodecs<0.16.0
+Requires-Dist: scipy>=1.15.3
+Provides-Extra: eval
+Requires-Dist: ngiab_eval; extra == "eval"
+Provides-Extra: plot
+Requires-Dist: ngiab_eval[plot]; extra == "plot"
+Dynamic: license-file
+
+# NGIAB Data Preprocess
+
+This repository contains tools for preparing data to run a [NextGen](https://github.com/NOAA-OWP/ngen)-based simulation using [NGIAB](https://github.com/CIROH-UA/NGIAB-CloudInfra). The tools allow you to select a catchment of interest on an interactive map, choose a date range, and prepare the data with just a few clicks!
+
+![map screenshot](https://github.com/CIROH-UA/NGIAB_data_preprocess/blob/main/modules/map_app/static/resources/screenshot.jpg)
+
+| | |
+| --- | --- |
+| ![CIROH Logo](./ciroh-bgsafe.png) | Funding for this project was provided by the National Oceanic & Atmospheric Administration (NOAA), awarded to the Cooperative Institute for Research to Operations in Hydrology (CIROH) through the NOAA Cooperative Agreement with The University of Alabama (NA22NWS4320003). |
+
+## Table of Contents
+
+1. [What does this tool do?](#what-does-this-tool-do)
+2. [Limitations](#limitations)
+   - [Custom realizations](#custom-realizations)
+   - [Calibration](#calibration)
+   - [Evaluation](#evaluation)
+   - [Visualisation](#visualisation)
+3. [Requirements](#requirements)
+4. [Installation and running](#installation-and-running)
+   - [Running without install](#running-without-install)
+   - [For uv installation](#for-uv-installation)
+   - [For legacy pip installation](#for-legacy-pip-installation)
+   - [Development installation](#development-installation)
+5. [Map interface documentation](#map-interface-documentation)
+   - [Running the map interface app](#running-the-map-interface-app)
+   - [Using the map interace](#using-the-map-interface)
+6. [CLI documentation](#cli-documentation)
+   - [Running the CLI](#running-the-cli)
+   - [Arguments](#arguments)
+   - [Usage notes](#usage-notes)
+   - [Examples](#examples)
+7. [Realization information](#realization-information)
+   - [NOAH + CFE](#noah--cfe)
+
+## What does this tool do?
+
+This tool prepares data to run a NextGen-based simulation by creating a run package that can be used with NGIAB.  
+It uses geometry and model attributes from the [v2.2 hydrofabric](https://lynker-spatial.s3-us-west-2.amazonaws.com/hydrofabric/v2.2/conus/conus_nextgen.gpkg) more information on [all data sources here](https://lynker-spatial.s3-us-west-2.amazonaws.com/hydrofabric/v2.2/hfv2.2-data_model.html).  
+The raw forcing data is [nwm retrospective v3 forcing](https://noaa-nwm-retrospective-3-0-pds.s3.amazonaws.com/index.html#CONUS/zarr/forcing/) data or the [AORC 1km gridded data](https://noaa-nws-aorc-v1-1-1km.s3.amazonaws.com/index.html) depending on user input
+
+1. **Subsets** (delineates) everything upstream of your point of interest (catchment, gage, flowpath etc) from the hydrofabric. This subset is output as a geopackage (.gpkg). 
+2. Calculates **forcings** as a weighted mean of the gridded NWM or AORC forcings. Weights are calculated using [exact extract](https://isciences.github.io/exactextract/) and computed with numpy. 
+3. Creates **configuration files** for a default NGIAB model run. 
+    -  realization.json  - ngen model configuration
+    -  troute.yaml - routing configuration.
+    -  **per catchment** model configuration
+4. Optionally performs a non-interactive [Docker-based NGIAB](https://github.com/CIROH-UA/NGIAB-CloudInfra) run.
+
+## Limitations
+This tool cannot do the following:
+
+### Custom realizations
+This tool currently only outputs a single, default realization, which is described in "[Realization information](#realization-information)". Support for additional model configurations is planned, but not currently available.
+
+### Calibration
+If available, this repository will download [calibrated parameters](https://communityhydrofabric.s3.us-east-1.amazonaws.com/index.html#hydrofabrics/community/gage_parameters/) from the [Community Hydrofabric](https://github.com/CIROH-UA/community_hf_patcher) AWS S3 bucket.
+However, many gages and catchments will not have such parameters available. In these cases, Data Preprocess will output realizations with default values.
+
+For automatic calibration, please see [ngiab-cal](https://github.com/CIROH-UA/ngiab-cal), which is under active development.
+
+### Evaluation
+For automatic evaluation using [TEEHR](https://github.com/RTIInternational/teehr), please run [NGIAB](https://github.com/CIROH-UA/NGIAB-CloudInfra) interactively using the `guide.sh` script.
+
+### Visualisation
+For automatic interactive visualisation, please run [NGIAB](https://github.com/CIROH-UA/NGIAB-CloudInfra) interactively using the `guide.sh` script
+
+# Requirements
+
+This tool is **officially supported** on **macOS** and **Ubuntu** (tested on 22.04 & 24.04). To use it on Windows, please install [**WSL**](https://learn.microsoft.com/en-us/windows/wsl/install).
+
+It is also **highly recommended** to use [Astral UV](https://docs.astral.sh/uv/) to install and run this tool. Installing the project via `pip` without the use of a virtual environment creates a **severe risk** of dependency conflicts.
+
+# Installation and running
+
+### Running without install
+This package supports pipx and uvx, which means you can run the tool without installing it. No virtual environment needed, just UV.
+```bash
+# Run these from anywhere!
+uvx --from ngiab-data-preprocess cli --help  # Running the CLI
+uvx ngiab-prep --help                        # Alias for the CLI
+uvx --from ngiab-data-preprocess map_app     # Running the map interface
+```
+
+### For uv installation
+
+<details>
+  <summary>Click here to expand</summary>
+
+```bash
+# Install UV
+curl -LsSf https://astral.sh/uv/install.sh | sh
+# It can be installed via pip if that fails
+# pip install uv
+
+# Create a virtual environment in the current directory
+uv venv
+
+# Install the tool in the virtual environment 
+uv pip install ngiab_data_preprocess
+
+# To run the cli
+uv run cli --help
+
+# To run the map 
+uv run map_app
+```
+
+UV automatically detects any virtual environments in the current directory and will use them when you use `uv run`.
+
+</details>
+
+### For legacy pip installation
+
+<details>
+  <summary>Click here to expand</summary>
+
+```bash
+# If you're installing this on jupyterhub / 2i2c you HAVE TO DEACTIVATE THE CONDA ENV
+(notebook) jovyan@jupyter-user:~$ conda deactivate
+jovyan@jupyter-user:~$
+# The interactive map won't work on 2i2c
+```    
+
+```bash
+# This tool is likely to not work without a virtual environment
+python3 -m venv .venv
+source .venv/bin/activate
+# installing and running the tool
+pip install 'ngiab_data_preprocess'
+python -m map_app
+# CLI instructions at the bottom of the README
+```
+</details>
+
+### Development installation
+
+<details>
+  <summary>Click to expand installation steps</summary>
+
+To install and run the tool, follow these steps:
+
+1. Clone the repository:
+   ```bash
+   git clone https://github.com/CIROH-UA/NGIAB_data_preprocess
+   cd NGIAB_data_preprocess
+   ```
+2. Create a virtual environment:
+   ```bash
+   uv venv
+   ```
+3. Install the tool:
+   ```bash
+   uv pip install -e .
+   ```
+4. Run the map app:
+   ```bash
+   uv run map_app
+   ```
+</details>
+
+# Map interface documentation
+
+## Running the map interface app
+
+Running the `map_app` tool will open the app in a new browser tab.
+
+Install-free: `uvx --from ngiab-data-preprocess map_app`  
+Installed with uv: `uv run map_app`
+
+## Using the map interface
+
+1. Select the catchment you're interested in on the map.
+2. Pick the time period you want to simulate.
+3. Click the following buttons in order:
+    1) Create subset gpkg
+    2) Create Forcing from Zarrs
+    3) Create Realization
+
+Once all the steps are finished, you can run NGIAB on the folder shown underneath the subset button.
+
+**Note:** When using the tool, the default output will be stored in the `~/ngiab_preprocess_output/<your-input-feature>/` folder. There is no overwrite protection on the folders.
+
+# CLI documentation
+
+## Running the CLI
+
+Install-free: `uvx ngiab-prep`  
+Installed with uv: `uv run cli`
+
+## Arguments
+
+- `-h`, `--help`: Show the help message and exit.
+- `-i INPUT_FEATURE`, `--input_feature INPUT_FEATURE`: ID of feature to subset. Providing a prefix will automatically convert to catid, e.g., cat-5173 or gage-01646500 or wb-1234.
+- `--vpu VPU_ID` : The id of the vpu to subset e.g 01. 10 = 10L + 10U and 03 = 03N + 03S + 03W. `--help` will display all the options.
+- `-l`, `--latlon`: Use latitude and longitude instead of catid. Expects comma-separated values via the CLI, e.g., `python -m ngiab_data_cli -i 54.33,-69.4 -l -s`.
+- `-g`, `--gage`: Use gage ID instead of catid. Expects a single gage ID via the CLI, e.g., `python -m ngiab_data_cli -i 01646500 -g -s`.
+- `-s`, `--subset`: Subset the hydrofabric to the given feature.
+- `-f`, `--forcings`: Generate forcings for the given feature.
+- `-r`, `--realization`: Create a realization for the given feature.
+- `--start_date START_DATE`, `--start START_DATE`: Start date for forcings/realization (format YYYY-MM-DD).
+- `--end_date END_DATE`, `--end END_DATE`: End date for forcings/realization (format YYYY-MM-DD).
+- `-o OUTPUT_NAME`, `--output_name OUTPUT_NAME`: Name of the output folder.
+- `--source` : The datasource you want to use, either `nwm` for retrospective v3 or `aorc`. Default is `nwm`
+- `-D`, `--debug`: Enable debug logging.
+- `--run`: Automatically run [NGIAB's docker distribution](https://github.com/CIROH-UA/NGIAB-CloudInfra) against the output folder.
+- `--validate`: Run every missing step required to run NGIAB.
+- `-a`, `--all`: Run all operations. Equivalent to `-sfr` and `--run`.
+
+## Usage notes
+- If your input has a prefix of `gage-`, you do not need to pass `-g`.
+- The `-l`, `-g`, `-s`, `-f`, `-r` flags can be combined like normal CLI flags. For example, to subset, generate forcings, and create a realization, you can use `-sfr` or `-s -f -r`.
+- When using the `--all` flag, it automatically sets `subset`, `forcings`, `realization`, and `run` to `True`.
+- Using the `--run` flag automatically sets the `--validate` flag.
+
+## Examples
+
+1. Prepare everything for an NGIAB run at a given gage:
+   ```bash
+   uvx ngiab-prep -i gage-10154200 -sfr --start 2022-01-01 --end 2022-02-28 
+   #         add --run or replace -sfr with --all to run NGIAB, too
+   # to name the folder, add -o folder_name
+   ```
+
+2. Subset the hydrofabric using a catchment ID or VPU:
+   ```bash
+   uvx ngiab-prep -i cat-7080 -s
+   uvx ngiab-prep --vpu 01 -s
+   ```
+
+3. Generate forcings using a single catchment ID:
+   ```bash
+   uvx ngiab-prep -i cat-5173 -f --start 2022-01-01 --end 2022-02-28
+   ```
+
+4. Create realization using a latitude/longitude pair and output to a named folder:
+   ```bash
+   uvx ngiab-prep -i 33.22,-87.54 -l -r --start 2022-01-01 --end 2022-02-28 -o custom_output
+   ```
+
+5. Perform all operations using a latitude/longitude pair:
+   ```bash
+   uvx ngiab-prep -i 33.22,-87.54 -l -s -f -r --start 2022-01-01 --end 2022-02-28
+   ```
+
+6. Subset the hydrofabric using a gage ID:
+   ```bash
+   uvx ngiab-prep -i 10154200 -g -s
+   # or
+   uvx ngiab-prep -i gage-10154200 -s
+   ```
+
+7. Generate forcings using a single gage ID:
+   ```bash
+   uvx ngiab-prep -i 01646500 -g -f --start 2022-01-01 --end 2022-02-28
+   ```
+
+# Realization information
+
+This tool currently offers one default realization.
+
+## NOAH + CFE
+
+[This realization](https://github.com/CIROH-UA/NGIAB_data_preprocess/blob/main/modules/data_sources/cfe-nowpm-realization-template.json) is intended to be roughly comparable to earlier versions of the National Water Model.
+- [NOAH-OWP-Modular](https://github.com/NOAA-OWP/NOAH-OWP-Modular): A refactoring of Noah-MP, a land-surface model. Used to model groundwater properties.
+- [Conceptual Functional Equivalent (CFE)](https://github.com/NOAA-OWP/CFE): A simplified conceptual approximation of versions 1.2, 2.0, and 2.1 of the National Water Model. Used to model precipitation and evaporation.
+- [SLoTH](https://github.com/NOAA-OWP/SLoTH): A module used to feed through unchanged values. In this default configuration, it simply forces certain soil moisture and ice fraction properties to zero.

ngiab_data_preprocess-4.4.0.dist-info/RECORD

@@ -0,0 +1,43 @@
+data_processing/create_realization.py,sha256=mdse8W2DgPg5Lj2_ErUsLJh-touTmShKwQrrOWO0jlY,15958
+data_processing/dask_utils.py,sha256=A2IP94WAz8W9nek3etXKEKTOxGPf0NWSFLh8cZ5S-xU,2454
+data_processing/dataset_utils.py,sha256=AJOxE2nRfZnWYon_qqGcfkpRZuRW8Yy8YI86SxVDU3M,11168
+data_processing/datasets.py,sha256=_EJ1uZSWTU1HWpvF7TQSikneJqWZFikTrdo9usCV8A0,4665
+data_processing/file_paths.py,sha256=l2iCUFt_pk-jjzl7OS7npROAnQxwqFfZ7b2wRjViqiU,4720
+data_processing/forcings.py,sha256=k-JhBncTnXcdjSieam1Q2cDx5Xt9hH5Aywv0gDY4O2U,19010
+data_processing/gpkg_utils.py,sha256=tSSIMlHeqqgxTJQyF3X9tPmunQTJYx0xrCNHqUBQxkg,20590
+data_processing/graph_utils.py,sha256=qvHw6JlzQxLi--eMsGgC_rUBP4nDatl6X9mSa03Xxyo,8306
+data_processing/s3fs_utils.py,sha256=ki1EmA0ezV0r26re6dRWIGzL5FudGdwF9Qw1eVLR0Bc,2747
+data_processing/subset.py,sha256=XoojOgWCwxOi5Q4KXHXARNQeoZlobJp-mqhIIvTRtTw,3793
+data_sources/cfe-nowpm-realization-template.json,sha256=8an6q1drWD8wU1ocvdPab-GvZDvlQ-0di_-NommH3QI,3528
+data_sources/cfe-template.ini,sha256=6e5-usqjWtm3MWVvtm8CTeZTJJMxO1ZswkOXq0L9mnc,2033
+data_sources/em-catchment-template.yml,sha256=M08ixazEUHYI2PNavtI0xPZeSzcQ9bg2g0XzNT-8_u4,292
+data_sources/em-config.yml,sha256=y0J8kEA70rxLWXJjz-CQ7sawcVyhQcayofeLlq4Svbo,1330
+data_sources/em-realization-template.json,sha256=DJvB7N8lCeS2vLFenmbTzysBDR-xPaJ09XA8heu1ijY,1466
+data_sources/forcing_template.nc,sha256=uRuVAqX3ngdlougZINavtwl_wC2VLD8fHqG7_CLim1s,85284
+data_sources/ngen-routing-template.yaml,sha256=wM5v6jj0kwcJBVatLFuy2big6g8nlSXxzc8a23nwI5s,4655
+data_sources/noah-owp-modular-init.namelist.input,sha256=Vb7mp40hFpJogruOrXrDHwVW1bKi9h1ciDNyDvTzn20,3045
+data_sources/source_validation.py,sha256=RmvyPLjuDetpuNOUqCclgDfe8zd_Ojr7pfbUoUya2pQ,9498
+data_sources/template.sql,sha256=ZnFqAqleEq9wgmAhNO90Wue_L9k0JAn8KF99DYtcxgs,10457
+data_sources/triggers.sql,sha256=G0d_175eNsamKAFhsbphPATvzMPuPL_iCleIhlToduQ,14906
+map_app/__init__.py,sha256=OarJao9X98kcbLyiwewN4ObWNAYkKDichcxbuWywTsA,818
+map_app/__main__.py,sha256=Uj7__cJUyPQkZo2tNQ2x2g6rwizsyg1DcNtJkQniHzY,1650
+map_app/views.py,sha256=ajU_QSd-Oa7UrRQEZPX4rmOlaKwo76Q8UPQNXtt-e2k,5622
+map_app/static/css/console.css,sha256=xN6G2MMFyKc9YW9HEVpUUTUjx2o2nokBR4nCX5c18UM,803
+map_app/static/css/main.css,sha256=HmRIfhWeHTrNLOCHGpaKuzwGj05LkkUiQy538D-ZRLY,6464
+map_app/static/css/toggle.css,sha256=Ep6tXT7gCrPRRITuEMpXyisuiTQgiLIEKFFTWRmC82o,1913
+map_app/static/js/console.js,sha256=BnG0pED5B9d563sLWshDNt_q-SyoTY48sETvVoOVJkU,1377
+map_app/static/js/data_processing.js,sha256=wXv0p_bPmNOrSpU_p6Yqtfd17vqOFRJFAmLdUUWLF7s,5486
+map_app/static/js/main.js,sha256=_Yq1tuzyREqWU24rFQJSh5zIaXtAXEGlfZPo36QLHvI,9690
+map_app/static/resources/loading.gif,sha256=ggdkZf1AD7rSwIpSJwfiIqANgmVV1WHlxGuKxQKv7uY,72191
+map_app/static/resources/screenshot.jpg,sha256=Ia358aX-OHM9BP4B8lX05cLnguF2fHUIimno9bnFLYw,253730
+map_app/templates/index.html,sha256=Jy2k1Ob2_et--BPpfmTYO22Yin3vrG6IOeNlwzUoEqY,7878
+ngiab_data_cli/__main__.py,sha256=13W3RnD73weQNYZdq6munx_0oMBgzc-yzluKEm5nSxg,10570
+ngiab_data_cli/arguments.py,sha256=yBULJnFgUvgP4YZmZ5HhR7g0EfdMtBCdQuDkDuYSXCQ,4322
+ngiab_data_cli/custom_logging.py,sha256=iS2XozaxudcxQj17qAsrCgbVK9LJAYAPmarJuVWJo1k,1280
+ngiab_data_cli/forcing_cli.py,sha256=eIWRxRWUwPqR16fihFDEIV4VzGlNuvcD6lJW5VYjkPU,3635
+ngiab_data_preprocess-4.4.0.dist-info/licenses/LICENSE,sha256=6dMSprwwnsRzEm02mEDbKHD9dUbL8bPIt9Vhrhb0Ulk,1081
+ngiab_data_preprocess-4.4.0.dist-info/METADATA,sha256=8PlfoGwOJIpuKhFwtfWmfxdMaDeXBfFRz9CAeZ3sZKk,13344
+ngiab_data_preprocess-4.4.0.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+ngiab_data_preprocess-4.4.0.dist-info/entry_points.txt,sha256=spwlhKEJ3ZnNETQsJGeTjD7Vwy8O_zGHb9GdX8ACCtw,128
+ngiab_data_preprocess-4.4.0.dist-info/top_level.txt,sha256=CjhYAUZrdveR2fOK6rxffU09VIN2IuPD7hk4V3l3pV0,52
+ngiab_data_preprocess-4.4.0.dist-info/RECORD,,

{ngiab_data_preprocess-4.2.2.dist-info → ngiab_data_preprocess-4.4.0.dist-info}/WHEEL

@@ -1,5 +1,5 @@
 Wheel-Version: 1.0
-Generator: setuptools (80.4.0)
+Generator: setuptools (80.9.0)
 Root-Is-Purelib: true
 Tag: py3-none-any
 

ngiab_data_preprocess-4.2.2.dist-info/METADATA

@@ -1,258 +0,0 @@
-Metadata-Version: 2.4
-Name: ngiab_data_preprocess
-Version: 4.2.2
-Summary: Graphical Tools for creating Next Gen Water model input data.
-Author-email: Josh Cunningham <jcunningham8@ua.edu>
-Project-URL: Homepage, https://github.com/CIROH-UA/NGIAB_data_preprocess
-Project-URL: Issues, https://github.com/CIROH-UA/NGIAB_data_preprocess/issues
-Classifier: Programming Language :: Python :: 3
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Operating System :: OS Independent
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: pyogrio>=0.7.2
-Requires-Dist: pyproj>=3.6.1
-Requires-Dist: Flask==3.0.2
-Requires-Dist: geopandas>=1.0.0
-Requires-Dist: requests==2.32.2
-Requires-Dist: igraph==0.11.4
-Requires-Dist: s3fs==2024.3.1
-Requires-Dist: xarray==2024.2.0
-Requires-Dist: zarr==2.17.1
-Requires-Dist: netCDF4>=1.6.5
-Requires-Dist: dask==2024.4.1
-Requires-Dist: dask[distributed]==2024.4.1
-Requires-Dist: h5netcdf==1.3.0
-Requires-Dist: exactextract==0.2.0
-Requires-Dist: numpy>=1.26.4
-Requires-Dist: tqdm==4.66.4
-Requires-Dist: rich==13.7.1
-Requires-Dist: colorama==0.4.6
-Requires-Dist: bokeh==3.5.1
-Requires-Dist: boto3
-Requires-Dist: numcodecs<0.16.0
-Provides-Extra: eval
-Requires-Dist: ngiab_eval; extra == "eval"
-Provides-Extra: plot
-Requires-Dist: ngiab_eval[plot]; extra == "plot"
-Dynamic: license-file
-
-# NGIAB Data Preprocess
-
-This repository contains tools for preparing data to run a [next gen](https://github.com/NOAA-OWP/ngen) simulation using [NGIAB](https://github.com/CIROH-UA/NGIAB-CloudInfra). The tools allow you to select a catchment of interest on an interactive map, choose a date range, and prepare the data with just a few clicks!
-
-![map screenshot](https://github.com/CIROH-UA/NGIAB_data_preprocess/blob/main/modules/map_app/static/resources/screenshot.jpg)
-
-## Table of Contents
-
-1. [What does this tool do?](#what-does-this-tool-do)
-2. [Requirements](#requirements)
-3. [Installation and Running](#installation-and-running)
-4. [Development Installation](#development-installation)
-5. [Usage](#usage)
-6. [CLI Documentation](#cli-documentation)
-   - [Arguments](#arguments)
-   - [Examples](#examples)
-   - [File Formats](#file-formats)
-   - [Output](#output)
-
-## What does this tool do?
-
-This tool prepares data to run a next gen simulation by creating a run package that can be used with NGIAB.  
-It uses geometry and model attributes from the [v2.2 hydrofabric](https://lynker-spatial.s3-us-west-2.amazonaws.com/hydrofabric/v2.2/conus/conus_nextgen.gpkg) more information on [all data sources here](https://lynker-spatial.s3-us-west-2.amazonaws.com/hydrofabric/v2.2/hfv2.2-data_model.html).  
-The raw forcing data is [nwm retrospective v3 forcing](https://noaa-nwm-retrospective-3-0-pds.s3.amazonaws.com/index.html#CONUS/zarr/forcing/) data or the [AORC 1km gridded data](https://noaa-nws-aorc-v1-1-1km.s3.amazonaws.com/index.html) depending on user input
-
-1. **Subset** (delineate) everything upstream of your point of interest (catchment, gage, flowpath etc). Outputs as a geopackage.  
-2. **Calculates** Forcings as a weighted mean of the gridded AORC forcings. Weights are calculated using [exact extract](https://isciences.github.io/exactextract/) and computed with numpy. 
-3. Creates **configuration files** needed to run nextgen.
-    -  realization.json  - ngen model configuration
-    -  troute.yaml - routing configuration.
-    -  **per catchment** model configuration
-4. Optionally Runs a non-interactive [Next gen in a box](https://github.com/CIROH-UA/NGIAB-CloudInfra).
-
-## What does it not do?
-
-### Evaluation
-For automatic evaluation using [Teehr](https://github.com/RTIInternational/teehr), please run [NGIAB](https://github.com/CIROH-UA/NGIAB-CloudInfra) interactively using the `guide.sh` script.
-
-### Visualisation
-For automatic interactive visualisation, please run [NGIAB](https://github.com/CIROH-UA/NGIAB-CloudInfra) interactively using the `guide.sh` script
-
-## Requirements
-
-* This tool is officially supported on macOS or Ubuntu (tested on 22.04 & 24.04). To use it on Windows, please install [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).
-
-## Installation and Running
-It is highly recommended to use [Astral UV](https://docs.astral.sh/uv/) to install and run this tool. It works similarly to pip and conda, and I would also recommend you use it for other python projects as it is so useful.
-
-```bash
-# Install UV
-curl -LsSf https://astral.sh/uv/install.sh | sh
-# It can be installed via pip if that fails
-# pip install uv
-
-# Create a virtual environment in the current directory
-uv venv
-
-# Install the tool in the virtual environment 
-uv pip install ngiab_data_preprocess
-
-# To run the cli
-uv run cli --help
-
-# To run the map 
-uv run map_app
-```
-
-UV automatically detects any virtual environments in the current directory and will use them when you use `uv run`.
-
-### Running without install
-This package supports pipx and uvx which means you can run the tool without installing it. No virtual environment needed, just UV.
-```bash
-# run this from anywhere 
-uvx --from ngiab_data_preprocess cli --help
-# for the map
-uvx --from ngiab_data_preprocess map_app
-```
-
-## For legacy pip installation
-<details>
-  <summary>Click here to expand</summary>
-
-```bash
-# If you're installing this on jupyterhub / 2i2c you HAVE TO DEACTIVATE THE CONDA ENV
-(notebook) jovyan@jupyter-user:~$ conda deactivate
-jovyan@jupyter-user:~$
-# The interactive map won't work on 2i2c
-```    
-
-```bash
-# This tool is likely to not work without a virtual environment
-python3 -m venv .venv
-source .venv/bin/activate
-# installing and running the tool
-pip install 'ngiab_data_preprocess'
-python -m map_app
-# CLI instructions at the bottom of the README
-```
-</details>
-
-## Development Installation
-
-<details>
-  <summary>Click to expand installation steps</summary>
-
-To install and run the tool, follow these steps:
-
-1. Clone the repository:
-   ```bash
-   git clone https://github.com/CIROH-UA/NGIAB_data_preprocess
-   cd NGIAB_data_preprocess
-   ```
-2. Create a virtual environment:
-   ```bash
-   uv venv
-   ```
-3. Install the tool:
-   ```bash
-   uv pip install -e .
-   ```
-4. Run the map app:
-   ```bash
-   uv run map_app
-   ```
-</details>
-
-## Usage
-
-Running the command `uv run map_app` will open the app in a new browser tab.
-
-To use the tool:
-1. Select the catchment you're interested in on the map.
-2. Pick the time period you want to simulate.
-3. Click the following buttons in order:
-    1) Create subset gpkg
-    2) Create Forcing from Zarrs
-    3) Create Realization
-
-Once all the steps are finished, you can run NGIAB on the folder shown underneath the subset button.
-
-**Note:** When using the tool, the default output will be stored in the `~/ngiab_preprocess_output/<your-input-feature>/` folder. There is no overwrite protection on the folders.
-
-# CLI Documentation
-
-## Arguments
-
-- `-h`, `--help`: Show the help message and exit.
-- `-i INPUT_FEATURE`, `--input_feature INPUT_FEATURE`: ID of feature to subset. Providing a prefix will automatically convert to catid, e.g., cat-5173 or gage-01646500 or wb-1234.
-- `--vpu VPU_ID` : The id of the vpu to subset e.g 01. 10 = 10L + 10U and 03 = 03N + 03S + 03W. `--help` will display all the options.
-- `-l`, `--latlon`: Use latitude and longitude instead of catid. Expects comma-separated values via the CLI, e.g., `python -m ngiab_data_cli -i 54.33,-69.4 -l -s`.
-- `-g`, `--gage`: Use gage ID instead of catid. Expects a single gage ID via the CLI, e.g., `python -m ngiab_data_cli -i 01646500 -g -s`.
-- `-s`, `--subset`: Subset the hydrofabric to the given feature.
-- `-f`, `--forcings`: Generate forcings for the given feature.
-- `-r`, `--realization`: Create a realization for the given feature.
-- `--start_date START_DATE`, `--start START_DATE`: Start date for forcings/realization (format YYYY-MM-DD).
-- `--end_date END_DATE`, `--end END_DATE`: End date for forcings/realization (format YYYY-MM-DD).
-- `-o OUTPUT_NAME`, `--output_name OUTPUT_NAME`: Name of the output folder.
-- `--source` : The datasource you want to use, either `nwm` for retrospective v3 or `aorc`. Default is `nwm`
-- `-D`, `--debug`: Enable debug logging.
-- `--run`: Automatically run Next Gen against the output folder.
-- `--validate`: Run every missing step required to run ngiab.
-- `-a`, `--all`: Run all operations: subset, forcings, realization, run Next Gen
-
-## Usage Notes
-- If your input has a prefix of `gage-`, you do not need to pass `-g`.
-- The `-l`, `-g`, `-s`, `-f`, `-r` flags can be combined like normal CLI flags. For example, to subset, generate forcings, and create a realization, you can use `-sfr` or `-s -f -r`.
-- When using the `--all` flag, it automatically sets `subset`, `forcings`, `realization`, and `run` to `True`.
-- Using the `--run` flag automatically sets the `--validate` flag.
-
-## Examples
-
-0. Prepare everything for a nextgen run at a given gage:
-   ```bash
-   python -m ngiab_data_cli -i gage-10154200 -sfr --start 2022-01-01 --end 2022-02-28 
-   #         add --run or replace -sfr with --all to run nextgen in a box too
-   # to name the folder, add -o folder_name
-   ```
-
-1. Subset hydrofabric using catchment ID or VPU:
-   ```bash
-   python -m ngiab_data_cli -i cat-7080 -s
-   python -m ngiab_data_cli --vpu 01 -s
-   ```
-
-2. Generate forcings using a single catchment ID:
-   ```bash
-   python -m ngiab_data_cli -i cat-5173 -f --start 2022-01-01 --end 2022-02-28
-   ```
-
-3. Create realization using a lat/lon pair and output to a named folder:
-   ```bash
-   python -m ngiab_data_cli -i 54.33,-69.4 -l -r --start 2022-01-01 --end 2022-02-28 -o custom_output
-   ```
-
-4. Perform all operations using a lat/lon pair:
-   ```bash
-   python -m ngiab_data_cli -i 54.33,-69.4 -l -s -f -r --start 2022-01-01 --end 2022-02-28
-   ```
-
-5. Subset hydrofabric using gage ID:
-   ```bash
-   python -m ngiab_data_cli -i 10154200 -g -s
-   # or
-   python -m ngiab_data_cli -i gage-10154200 -s
-   ```
-
-6. Generate forcings using a single gage ID:
-   ```bash
-   python -m ngiab_data_cli -i 01646500 -g -f --start 2022-01-01 --end 2022-02-28
-   ```
-
-7. Run all operations, including Next Gen and evaluation/plotting:
-   ```bash
-   python -m ngiab_data_cli -i cat-5173 -a --start 2022-01-01 --end 2022-02-28
-   ```
-
-
-

ngiab_data_preprocess-4.2.2.dist-info/RECORD

@@ -1,42 +0,0 @@
-data_processing/create_realization.py,sha256=BqGtJSfl-XlsCQOIxonQcZKxLddz_LE18x6At4qCiRo,14948
-data_processing/dataset_utils.py,sha256=4qmRmK3qMWPkiWZHXhXv3c-ISbtOwr7QhciEl2ok6Ao,7314
-data_processing/datasets.py,sha256=BB_X0IvGnUrjBmj-wryn6-Nv5cd0Lje3zly3p896eU4,4822
-data_processing/file_paths.py,sha256=Cp3BxbO6sD50464ciTshjb3Z0vTvL0ZeSbOJgNdOqQ0,4698
-data_processing/forcings.py,sha256=bHVXEUpSTzw3iBdw6_jww0Wk_HksZseWdFJ0oF-Ri-E,18910
-data_processing/gpkg_utils.py,sha256=fr0X62HawnSXzWRHcC_9aFrnoKv5atz4XkalydxCSko,20538
-data_processing/graph_utils.py,sha256=I9IFkU08SkXb8pfVASTvkm7T8BYt-92CaNj7TV6wJFE,8244
-data_processing/s3fs_utils.py,sha256=WoqqwxkHpv0Qq4I5f5-gUZBCFtVQ68ehXbdOjWRKTDQ,2752
-data_processing/subset.py,sha256=ns-WIHzshi74CFetmGsYvd72QgNtQFwl6Avc9rIYpf4,2604
-data_sources/cfe-nowpm-realization-template.json,sha256=8an6q1drWD8wU1ocvdPab-GvZDvlQ-0di_-NommH3QI,3528
-data_sources/cfe-template.ini,sha256=6e5-usqjWtm3MWVvtm8CTeZTJJMxO1ZswkOXq0L9mnc,2033
-data_sources/em-catchment-template.yml,sha256=M08ixazEUHYI2PNavtI0xPZeSzcQ9bg2g0XzNT-8_u4,292
-data_sources/em-config.yml,sha256=y0J8kEA70rxLWXJjz-CQ7sawcVyhQcayofeLlq4Svbo,1330
-data_sources/em-realization-template.json,sha256=DJvB7N8lCeS2vLFenmbTzysBDR-xPaJ09XA8heu1ijY,1466
-data_sources/forcing_template.nc,sha256=uRuVAqX3ngdlougZINavtwl_wC2VLD8fHqG7_CLim1s,85284
-data_sources/ngen-routing-template.yaml,sha256=RV28MAbyQNx9U8FAYmZhD2Fv8Yu6o_08Ekoc77KNdH4,4622
-data_sources/noah-owp-modular-init.namelist.input,sha256=Vb7mp40hFpJogruOrXrDHwVW1bKi9h1ciDNyDvTzn20,3045
-data_sources/source_validation.py,sha256=9mC1LiL8TXh0mtHFwOlFkT2jN1DEorgw_K7qccrdFfA,8038
-data_sources/template.sql,sha256=ZnFqAqleEq9wgmAhNO90Wue_L9k0JAn8KF99DYtcxgs,10457
-data_sources/triggers.sql,sha256=G0d_175eNsamKAFhsbphPATvzMPuPL_iCleIhlToduQ,14906
-map_app/__init__.py,sha256=OarJao9X98kcbLyiwewN4ObWNAYkKDichcxbuWywTsA,818
-map_app/__main__.py,sha256=D9uuAcsOUyfgYqGXQwsmNLo3n3BQ8pnziCAQWQnV4RU,1617
-map_app/views.py,sha256=0T-tvAzyfD1ttEQsZ84iAc2ELVN_KS056_YxiyHQ_DE,5122
-map_app/static/css/console.css,sha256=xN6G2MMFyKc9YW9HEVpUUTUjx2o2nokBR4nCX5c18UM,803
-map_app/static/css/main.css,sha256=UkctdgBijejzpjhLVwOBx2LxdPAY625_yTIe90fKpv0,6188
-map_app/static/css/toggle.css,sha256=Ep6tXT7gCrPRRITuEMpXyisuiTQgiLIEKFFTWRmC82o,1913
-map_app/static/js/console.js,sha256=BnG0pED5B9d563sLWshDNt_q-SyoTY48sETvVoOVJkU,1377
-map_app/static/js/data_processing.js,sha256=X6NSuggOGNIJUF-LEyGGYJjtiA5J29xmkXgFFmfBw18,6711
-map_app/static/js/main.js,sha256=dM0fCzNg-nV3wEqZFQ_wEam0Sm8RqHOv2GgRYg97pQQ,6875
-map_app/static/resources/loading.gif,sha256=ggdkZf1AD7rSwIpSJwfiIqANgmVV1WHlxGuKxQKv7uY,72191
-map_app/static/resources/screenshot.jpg,sha256=Ia358aX-OHM9BP4B8lX05cLnguF2fHUIimno9bnFLYw,253730
-map_app/templates/index.html,sha256=82zRdoPar5qfgdwFiVkuXeIQC4okt9iDgCPeNxKdX9I,7403
-ngiab_data_cli/__main__.py,sha256=V-g0dwa000e5Qye7PBMTtHTK7Cl7as3JS0UoAR0QrZ4,10503
-ngiab_data_cli/arguments.py,sha256=7ptImy8tLM1XvjfDr13tZszkjGVtenXo0KqllJeE3Mw,4372
-ngiab_data_cli/custom_logging.py,sha256=iS2XozaxudcxQj17qAsrCgbVK9LJAYAPmarJuVWJo1k,1280
-ngiab_data_cli/forcing_cli.py,sha256=lkcqWDk5H8IPyGv0DwLIZMQldqTUXpfwSX0C_RIuIJ8,3890
-ngiab_data_preprocess-4.2.2.dist-info/licenses/LICENSE,sha256=6dMSprwwnsRzEm02mEDbKHD9dUbL8bPIt9Vhrhb0Ulk,1081
-ngiab_data_preprocess-4.2.2.dist-info/METADATA,sha256=qzh_j7XnKUvF6Zq4sx3idNEp1MneIpAL8ci06lfZDLw,10252
-ngiab_data_preprocess-4.2.2.dist-info/WHEEL,sha256=DnLRTWE75wApRYVsjgc6wsVswC54sMSJhAEd4xhDpBk,91
-ngiab_data_preprocess-4.2.2.dist-info/entry_points.txt,sha256=spwlhKEJ3ZnNETQsJGeTjD7Vwy8O_zGHb9GdX8ACCtw,128
-ngiab_data_preprocess-4.2.2.dist-info/top_level.txt,sha256=CjhYAUZrdveR2fOK6rxffU09VIN2IuPD7hk4V3l3pV0,52
-ngiab_data_preprocess-4.2.2.dist-info/RECORD,,