climate-toolkit 0.1.0__tar.gz → 0.2.0__tar.gz

{climate_toolkit-0.1.0/climate_toolkit.egg-info → climate_toolkit-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: climate-toolkit
-Version: 0.1.0
+Version: 0.2.0
 Summary: Climate toolkit for location-based climate, season, climatology, hazard, and projection analysis.
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/CGIAR-Climate-Data-Hub/climate-toolkit
@@ -29,6 +29,11 @@ Requires-Dist: typer<1,>=0.12
 Requires-Dist: xarray>=2024.6
 Requires-Dist: xclim<1,>=0.60
 Requires-Dist: xee<0.2,>=0.1
+Provides-Extra: cloud
+Requires-Dist: zarr<4,>=3; python_version >= "3.12" and extra == "cloud"
+Requires-Dist: icechunk<3,>=2.3; python_version >= "3.12" and extra == "cloud"
+Requires-Dist: fsspec[http]>=2025.3; python_version >= "3.12" and extra == "cloud"
+Requires-Dist: dask[array]>=2025.3; python_version >= "3.12" and extra == "cloud"
 Provides-Extra: api
 Requires-Dist: fastapi>=0.110.0; extra == "api"
 Requires-Dist: uvicorn>=0.29.0; extra == "api"
@@ -38,14 +43,18 @@ Dynamic: license-file
 
 # Climate Data Toolkit
 
-A unified toolkit for retrieving climate data from various global datasets such as CHIRPS, AGERA5, TerraClimate, IMERG, TAMSAT, CHIRTS, ERA5, NEX-GDDP, NASA POWER, CMIP6 and SoilGrids.
+A unified toolkit for retrieving climate data from various global datasets such as CHIRPS, ERA5-Land, TerraClimate, IMERG, TAMSAT, CHIRTS-ERA5, ERA5, NEX-GDDP, NASA POWER, CMIP6 and SoilGrids.
 
 [![About](https://img.shields.io/badge/about-the%20project-blue)](https://CGIAR-Climate-Data-Hub.github.io/climate-toolkit/about/) — what the toolkit is, who it's for, and how to get involved (also in [français](https://CGIAR-Climate-Data-Hub.github.io/climate-toolkit/fr/about/) / [español](https://CGIAR-Climate-Data-Hub.github.io/climate-toolkit/es/about/)).
 
 [![Documentation](https://img.shields.io/badge/docs-climate--toolkit-2ea44f)](https://CGIAR-Climate-Data-Hub.github.io/climate-toolkit/) — full documentation: getting started, workflows, and API reference.
 
+[![PyPI](https://img.shields.io/pypi/v/climate-toolkit)](https://pypi.org/project/climate-toolkit/) — published on PyPI: `pip install climate-toolkit`.
+
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/CGIAR-Climate-Data-Hub/climate-toolkit/blob/main/examples/climate_toolkit_colab.ipynb) — try the toolkit live in your browser, no install needed.
 
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/CGIAR-Climate-Data-Hub/climate-toolkit/blob/main/examples/era_lte_colab.ipynb) — ERA LTE workflow: validate toolkit rainfall against ERA field records and explore yield vs climate, with plots.
+
 <p>
   <a href="https://www.mcknight.org/programs/global-foods/"><img src="docs/assets/logos/mcknight.jpg" alt="The McKnight Foundation" height="44"></a>
   &nbsp;&nbsp;&nbsp;&nbsp;
@@ -59,11 +68,11 @@ Funded by the [McKnight Foundation CRFS programme](https://www.mcknight.org/prog
 ## API Dataset Badges
 
 [![CHIRPS](https://img.shields.io/badge/CHIRPS-Precipitation_4.8-blue)](https://data.chc.ucsb.edu/products/CHIRPS-2.0/)
-[![AgERA5](https://img.shields.io/badge/AgERA5-Agriculture_Climate_4.8-brightgreen)](https://data.mcc.tu-berlin.de/agera5/)
+[![ERA5-Land](https://img.shields.io/badge/ERA5--Land-Reanalysis-brightgreen)](https://developers.google.com/earth-engine/datasets/catalog/ECMWF_ERA5_LAND_DAILY_AGGR)
 [![TerraClimate](https://img.shields.io/badge/TerraClimate-Water_Balance_4.7-lightgrey)](http://www.climatologylab.org/terraclimate.html)
 [![IMERG](https://img.shields.io/badge/IMERG-Global_Precipitation_4.7-green)](https://gpm.nasa.gov/data/imerg)
 [![TAMSAT](https://img.shields.io/badge/TAMSAT-Africa_Precipitation_4.7-yellowgreen)](https://www.tamsat.org.uk/)
-[![CHIRTS](https://img.shields.io/badge/CHIRTS-Temperature_4.7-blueviolet)](https://data.chc.ucsb.edu/products/CHIRTSdaily/)
+[![CHIRTS-ERA5](https://img.shields.io/badge/CHIRTS--ERA5-Temperature-blueviolet)](https://www.chc.ucsb.edu/data/chirts-era5)
 [![ERA5](https://img.shields.io/badge/ERA5-Reanalysis_Climate_4.2-orange)](https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels)
 [![NEX-GDDP](https://img.shields.io/badge/NEX--GDDP-Climate_Projections_4.1-blue)](https://www.nccs.nasa.gov/services/data-collections/land-based-products/nex-gddp)
 [![NASA POWER](https://img.shields.io/badge/NASA_POWER-Solar_Temp_Global_4.0-lightblue)](https://power.larc.nasa.gov/)
@@ -76,23 +85,54 @@ Funded by the [McKnight Foundation CRFS programme](https://www.mcknight.org/prog
 
 The Climate Toolkit offers a unified, programmatic interface to:
 
-- Retrieve climate data from CHIRPS, AGERA5, TerraClimate, IMERG, TAMSAT, CHIRTS, ERA5, NEX-GDDP, NASA POWER, CMIP6 and SoilGrids
+- Retrieve climate data from CHIRPS, ERA5-Land, TerraClimate, IMERG, TAMSAT, CHIRTS-ERA5, ERA5, NEX-GDDP, NASA POWER, CMIP6 and SoilGrids
 - Compute rainfall statistics, anomalies, and hazard indicators
 - Compare climate trends over historical and seasonal periods
 
 For user-facing historical daily climate work, the default module policy is
-`chirps_v3_daily_rnl + agera_5`: CHIRPS v3 Daily RNL supplies precipitation and
-AgERA5 supplies temperature plus companion variables such as humidity, wind,
+`chirps_v3_daily_rnl + era5_land`: CHIRPS v3 Daily RNL supplies precipitation.
+The `era5_land` key reads ERA5-Land and supplies temperature plus companion variables such as humidity, wind,
 and solar radiation. If a direct single-source historical fallback is needed,
-prefer `agera_5`. Keep `era_5` available for compatibility, diagnostics, and
+prefer `era5_land`. Keep `era_5` available for compatibility, diagnostics, and
 comparison work, but do not treat it as the primary recommended source.
 
+`era5_land` is the canonical key for `ECMWF/ERA5_LAND/DAILY_AGGR`.
+The former `agera_5` and `agera5` spellings still select that same ERA5-Land
+product and now emit a visible deprecation warning. They never selected the
+separate Copernicus AgERA5 product. Compatibility aliases remain through 0.2.x;
+new code, defaults and reports use `era5_land`. Existing `agera_5` cache directories
+are reused without changing their scientific content.
+
+For **no-GEE access**, install the cloud extra on Python 3.12+ (`pip install
+'climate-toolkit[cloud]'` or `uv sync --extra cloud`). CHIRPS v3 RNL can use
+Dynamical and Hugging Face, with provider failover. Pair it with `chirts_era5`
+for daily temperature through recent dates. Use `--no-gee` on the unified CLI
+or `with climate_toolkit.backend_policy(allow_gee=False):` around Python
+workflows; historical `auto` then uses this direct pair. Explicit providers,
+cache provenance, QC and limits: [direct cloud access](docs/direct_cloud_access.md).
+Direct reads share source chunks across up to five missing years per batch and
+save annual progress for retries. CHIRTS-ERA5's independent CHC backup remains
+under [validation](analysis/issues/issue_179_chc_backup_investigation.md).
+ERA5-Land's official CDS cloud stores require a separate CDS key; that work is
+[shelved](analysis/issues/issue_180_era5_land_access.md) and they are not used by
+the toolkit. The current no-GEE auto pair supplies rainfall and Tmax/Tmin.
+
+For the complete fetch → climatology → statistics → hazards → comparison workflow,
+`run_pipeline` accepts `precip_source`, `temp_source` and `allow_gee=False`, and
+exports per-site reports with provenance. See the
+[full pipeline example](docs/direct_cloud_access.md#full-pipeline).
+
+`chirts` (1983–2016) is deprecated in favor of `chirts_era5` for new work.
+Legacy requests retain their original scientific meaning and caches; the old
+product is no longer an automatic fallback. The direct pair supplies Tmax/Tmin
+and rainfall, not humidity, wind or radiation.
+
 TAMSAT remains available for completeness and Africa-focused precipitation
 comparison work, but it is currently fragile and should not be treated as a
 default or relied on for production workflows. It is precipitation-only, must
 be paired with a temperature source for most analysis modules, and current
 public access via JASMIN has shown slow performance and intermittent SSL /
-download failures in live testing. Prefer `chirps_v3_daily_rnl + agera_5` for
+download failures in live testing. Prefer `chirps_v3_daily_rnl + era5_land` for
 normal user workflows.
 
 > **Using it as a Python library?** See the
@@ -155,6 +195,14 @@ If you see:
 
 ### Installation
 
+Just want to use the package? Install from PyPI:
+
+```bash
+pip install climate-toolkit
+```
+
+To develop the toolkit or run the `examples/` scripts, set up from a clone instead:
+
 1. Clone repository
 
    ```bash
@@ -196,6 +244,17 @@ If you see:
    uv run pytest -q
    ```
 
+   CI tests the base package on Python 3.10 and the `cloud` and `api` extras on
+   Python 3.12. To run the cloud-provider and pipeline tests locally:
+
+   ```bash
+   uv sync --locked --group dev --extra cloud --extra api --python 3.12
+   uv run pytest -q
+   ```
+
+   Distribution checks install the base wheel and source archive in fresh
+   environments; Python 3.12+ also checks a fresh `cloud` wheel installation.
+
 3. Fallback setup with `venv + pip`
 
    macOS / Linux:
@@ -275,7 +334,7 @@ commands through `uv run --env-file .env ...`.
 
 | Variable | Required for | Notes |
 |----------|--------------|-------|
-| `GCP_PROJECT_ID` | All Earth Engine-backed sources (AgERA5, ERA5, NEX-GDDP, GEE historical) | Your real Google Cloud **Project ID** — see [Earth Engine setup](#earth-engine-setup). `GOOGLE_CLOUD_PROJECT` / `EE_PROJECT_ID` are accepted as fallbacks. |
+| `GCP_PROJECT_ID` | Earth Engine-backed routes (ERA5-Land via `era5_land`, ERA5, NEX-GDDP, GEE historical) | Your real Google Cloud **Project ID** — see [Earth Engine setup](#earth-engine-setup). `GOOGLE_CLOUD_PROJECT` / `EE_PROJECT_ID` are accepted as fallbacks. |
 | `EARTHDATA_USERNAME` / `EARTHDATA_PASSWORD` | Sources still using Earthdata-backed access | Optional; only needed by those sources. |
 
 Earth Engine authentication itself is stored separately (via
@@ -392,12 +451,14 @@ docker compose down --remove-orphans
 
 ### Release strategy
 
-Current decision:
-
-- GitHub releases only for now
-- no direct PyPI publish yet
-- TestPyPI can come later once auth-heavy runtime expectations, install docs,
-  and wheel smoke behavior stay stable across contributor machines
+- Published on **PyPI** as [`climate-toolkit`](https://pypi.org/project/climate-toolkit/) —
+  `pip install climate-toolkit`.
+- Releases are automated via **Trusted Publishing** (OIDC, no stored tokens):
+  publishing a GitHub Release builds the package and uploads it to PyPI. Mark a
+  Release as a pre-release to route it to TestPyPI as a rehearsal instead. See
+  [`RELEASING.md`](RELEASING.md).
+- Versions follow [semantic versioning](https://semver.org); PyPI versions are
+  immutable, so each publish needs a new version number.
 
 Reference note:
 
@@ -408,7 +469,7 @@ Reference note:
 For historical daily climate workflows, start with:
 
 - precipitation: `chirps_v3_daily_rnl`
-- temperature + companion variables: `agera_5`
+- temperature + companion variables: `era5_land`
 
 For most higher-level modules, `--source auto` or `--source paired` will route
 to this historical default behavior.
@@ -459,7 +520,7 @@ Main entry point:
 Key options:
 
 - `--source`: exact source key such as `chirps_v3_daily_rnl`, `chirps_v2`,
-  `agera_5`, `era_5`, `nex_gddp`, `nasa_power`
+  `era5_land`, `era_5`, `nex_gddp`, `nasa_power`
 - `--stage`: `raw`, `transformed`, `preprocessed`
 - `--variables`: comma-separated toolkit variable names
 - `--output`: optional output file
@@ -490,7 +551,7 @@ Livestock THI source support:
 
 | Source | Status | Notes |
 | --- | --- | --- |
-| `agera_5` | supported | humidity derived from dewpoint + air temperature in current fetch pipeline |
+| `era5_land` | supported | humidity derived from dewpoint + air temperature in current fetch pipeline |
 | `nasa_power` | supported | humidity available from current NASA POWER fetch path |
 | `ghcn_daily` | supported when available | uses station humidity when `RHAV` exists for chosen station and window |
 | `gsod` | supported when available | uses station humidity when humidity field exists for chosen station and window |
@@ -501,7 +562,7 @@ Livestock THI source support:
 | `chirps_v3_daily_rnl` | not supported | precipitation-only source |
 | `imerg` | not supported | precipitation-only source |
 | `tamsat` | not supported | no humidity data |
-| `chirts` | not supported | temperature-only source |
+| `chirts_era5`, deprecated `chirts` | not supported | temperature-only sources; humidity must come from a companion product |
 
 Python users can inspect same support map directly:
 
@@ -541,7 +602,7 @@ from climate_toolkit.climatology import (
 
 print(describe_human_heat_method()["metric"])
 print(describe_human_heat_source_support())
-print(build_human_heat_source_bundle(source="paired", precip_source="chirps_v3_daily_rnl", temp_source="agera_5"))
+print(build_human_heat_source_bundle(source="paired", precip_source="chirps_v3_daily_rnl", temp_source="era5_land"))
 ```
 
 Dedicated method guide:
@@ -577,7 +638,7 @@ climate-toolkit-seasons \
   --end-year=2020 \
   --source=paired \
   --precip-source=chirps_v3_daily_rnl \
-  --temp-source=agera_5 \
+  --temp-source=era5_land \
   --fixed-season="03-01:05-31"
 ```
 
@@ -593,7 +654,7 @@ climate-toolkit-stats \
   --end-year=2020 \
   --source=paired \
   --precip-source=chirps_v3_daily_rnl \
-  --temp-source=agera_5 \
+  --temp-source=era5_land \
   --fixed-season="03-01:05-31"
 ```
 
@@ -650,6 +711,23 @@ such as `total_seconds` and `mean_model_seconds`.
 - `climate-toolkit-hazards`
 - `climate-toolkit-hazards-ensemble`
 
+NEX-GDDP hazards (`--source nex_gddp`, also `evaluate_hazards(..., source="nex_gddp")`)
+accept `--model` / `--scenario`. `--scenario` defaults to `historical` when the
+requested window ends before 2015, so pre-2015 windows work without extra flags
+(a window that crosses 2015 needs an explicit scenario — no single NEX-GDDP
+scenario spans the historical/SSP boundary):
+
+```bash
+env GCP_PROJECT_ID=YOUR_PROJECT_ID climate-toolkit-hazards \
+  maize \
+  --location=-0.776,37.66 \
+  --date-from 1995-01-01 \
+  --date-to 2013-12-31 \
+  --source nex_gddp \
+  --model GFDL-ESM4 \
+  --fixed-season '10-17:01-09'
+```
+
 ### Weather stations
 
 - `climate-toolkit-weather-station-download`
@@ -685,7 +763,7 @@ climate-toolkit-weather-station-compare \
   --grid-source paired \
   --grid-source nasa_power \
   --precip-source chirps_v3_daily_rnl \
-  --temp-source agera_5 \
+  --temp-source era5_land \
   --variables precipitation,max_temperature,min_temperature \
   --output outputs/weather_station/nairobi_station_vs_grid.json
 ```
@@ -758,19 +836,20 @@ climate-toolkit-climatology \
   --location="-1.286,36.817" \
   --start-year=1991 \
   --end-year=2020 \
-  --source=agera_5
+  --source=era5_land
 ```
 
 Important source notes:
 
-- `agera_5`, `era_5`, `chirps_v2`, `chirps_v3_daily_rnl`, `chirts`, `imerg`,
-  and similar historical gridded sources are Earth Engine-backed in this
-  toolkit. They need Earth Engine auth plus `GCP_PROJECT_ID`.
+- `chirps_v3_daily_rnl` and `chirts_era5` prefer direct public cloud access with
+  Python 3.12+ and the `cloud` extra. Use `--no-gee` to prohibit GEE fallback.
+- `era5_land`, `era_5`, `chirps_v2`, deprecated `chirts`, `imerg` and other
+  GEE-only routes need Earth Engine auth plus `GCP_PROJECT_ID`.
 - `era_5` daily coverage currently ends on `2020-07-09` in this package. Use
-  `agera_5` for a full `1991-2020` climatology window, or stop ERA5 at `2019`.
+  `era5_land` for a full `1991-2020` climatology window, or stop ERA5 at `2019`.
 - `chirps_v2` / `chirps_v3_daily_rnl` are precipitation-only, so climatology
   will omit temperature outputs.
-- `chirts` is temperature-only, so climatology will omit precipitation outputs.
+- `chirts_era5` and deprecated `chirts` are temperature-only, so climatology omits precipitation outputs.
 
 SPEI and xclim helpers are library functions, not standalone console scripts.
 Use them from Python after fetching/preprocessing climate data. VPD helpers use
@@ -921,7 +1000,7 @@ window, and variable set.
 For example, a live three-site, one-year benchmark on June 13, 2026 using:
 
 - `chirps_v3_daily_rnl` cold cache: about 22 seconds
-- `agera_5` with precipitation, temperature, humidity, wind, and solar cold cache: about 78 seconds
+- `era5_land` with precipitation, temperature, humidity, wind, and solar cold cache: about 78 seconds
 - either source from warm cache: about 0.75 seconds
 
 If you want cache reuse across sessions, pass a stable project-local
@@ -935,7 +1014,7 @@ Repeat runs using same cache can be near-instant.
 Observed live examples:
 
 - `chirps_v3_daily_rnl`, 3 sites, 1 year, cold cache: about `22s`
-- `agera_5`, 3 sites, 1 year, cold cache with companion variables: about `78s`
+- `era5_land`, 3 sites, 1 year, cold cache with companion variables: about `78s`
 - warm-cache rerun for same requests: about `0.75s`
 
 For project work, prefer stable cache roots under `outputs/cache/...`.
@@ -971,7 +1050,7 @@ it.
 Use `tamsat` only as optional precipitation partner, not primary recommended
 historical source.
 
-- pair it with temperature source such as `agera_5`
+- pair it with temperature source such as `era5_land`
 - expect slower runs and possible download instability
 - use as comparison / sensitivity source, not dependable default
 
@@ -1365,7 +1444,7 @@ climate-toolkit-weather-station-compare \
   --grid-source paired \
   --grid-source nasa_power \
   --precip-source chirps_v3_daily_rnl \
-  --temp-source agera_5 \
+  --temp-source era5_land \
   --variables precipitation,max_temperature,min_temperature \
   --output outputs/weather_station/nairobi_station_vs_grid_2011_2020.json
 ```
@@ -1378,8 +1457,8 @@ Comparison intent:
 
 Current grid-source notes:
 
-- better independence: `agera_5`, `era_5`, `nasa_power`, `imerg`
-- partly station-informed: `chirps_v2`, `chirps_v3_daily_rnl`, `chirts`, `paired`, `terraclimate`, `auto`
+- better independence: `era5_land`, `era_5`, `nasa_power`, `imerg`
+- partly station-informed: `chirps_v2`, `chirps_v3_daily_rnl`, `chirts_era5`, `chirts`, `paired`, `terraclimate`, `auto`
 - comparison output warns when selected grid source is not fully independent from station-based validation
 
 Output layers include:
@@ -1409,7 +1488,7 @@ climate-toolkit-stats \
   --end-year=2020 \
   --source=paired \
   --precip-source=chirps_v3_daily_rnl \
-  --temp-source=agera_5
+  --temp-source=era5_land
 ```
 
 To overlay your own station observations, add the override flags and replace
@@ -1423,7 +1502,7 @@ climate-toolkit-stats \
   --end-year=2020 \
   --source=paired \
   --precip-source=chirps_v3_daily_rnl \
-  --temp-source=agera_5 \
+  --temp-source=era5_land \
   --custom-station-file <REPLACE_WITH_YOUR_STATION.csv> \
   --custom-station-vars precipitation,max_temperature,min_temperature \
   --custom-station-name "My station"