forcingkit 0.1.0.post1__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (101) hide show
  1. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/PKG-INFO +12 -7
  2. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/README.md +11 -6
  3. forcingkit-0.3.0/docs/source/_static/custom.css +24 -0
  4. forcingkit-0.3.0/docs/source/_static/forcingkit-noreaster-wind.gif +0 -0
  5. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/docs/source/atmospheric_forcing.rst +58 -29
  6. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/docs/source/conf.py +2 -1
  7. forcingkit-0.3.0/docs/source/fetchers.rst +191 -0
  8. forcingkit-0.3.0/docs/source/hycom.rst +233 -0
  9. forcingkit-0.3.0/docs/source/index.rst +162 -0
  10. forcingkit-0.3.0/docs/source/necofs.rst +302 -0
  11. forcingkit-0.3.0/docs/source/nyofs.rst +327 -0
  12. forcingkit-0.3.0/docs/source/roadmap.rst +321 -0
  13. forcingkit-0.3.0/docs/source/sources.rst +116 -0
  14. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/pyproject.toml +1 -1
  15. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/service/forcingkit_serve/main.py +61 -4
  16. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/service/forcingkit_serve/routers/removed.py +4 -4
  17. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/service/forcingkit_serve/routers/viewer.py +7 -0
  18. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/dispatcher.py +17 -0
  19. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/fetchers/dbofs.py +139 -167
  20. forcingkit-0.3.0/src/forcingkit/fetchers/hycom.py +318 -0
  21. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/fetchers/necofs.py +52 -8
  22. forcingkit-0.3.0/src/forcingkit/fetchers/noaa.py +245 -0
  23. forcingkit-0.3.0/src/forcingkit/fetchers/nyofs.py +366 -0
  24. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/static/app.js +40 -0
  25. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/static/index.html +5 -2
  26. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/static/preview3d.js +74 -6
  27. forcingkit-0.3.0/tests/integration/test_ofs_archive_paths.py +73 -0
  28. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_dbofs.py +119 -43
  29. forcingkit-0.3.0/tests/unit/test_hycom.py +263 -0
  30. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_necofs_parent.py +44 -2
  31. forcingkit-0.3.0/tests/unit/test_noaa_currents.py +172 -0
  32. forcingkit-0.3.0/tests/unit/test_noaa_datum.py +66 -0
  33. forcingkit-0.3.0/tests/unit/test_nyofs.py +417 -0
  34. forcingkit-0.3.0/tests/unit/test_obc_donor_report.py +53 -0
  35. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/uv.lock +1 -1
  36. forcingkit-0.1.0.post1/docs/source/_static/forcingkit-inventory-screenshot.png +0 -0
  37. forcingkit-0.1.0.post1/docs/source/fetchers.rst +0 -29
  38. forcingkit-0.1.0.post1/docs/source/index.rst +0 -16
  39. forcingkit-0.1.0.post1/docs/source/nyofs.rst +0 -188
  40. forcingkit-0.1.0.post1/docs/source/removed_endpoints.rst +0 -44
  41. forcingkit-0.1.0.post1/src/forcingkit/fetchers/hycom.py +0 -159
  42. forcingkit-0.1.0.post1/src/forcingkit/fetchers/noaa.py +0 -87
  43. forcingkit-0.1.0.post1/src/forcingkit/fetchers/nyofs.py +0 -458
  44. forcingkit-0.1.0.post1/tests/unit/test_hycom.py +0 -32
  45. forcingkit-0.1.0.post1/tests/unit/test_nyofs.py +0 -294
  46. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.agents/forcingkit.md +0 -0
  47. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.claude/CLAUDE.md +0 -0
  48. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.env.template +0 -0
  49. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.github/copilot-instructions.md +0 -0
  50. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.github/workflows/docs.yml +0 -0
  51. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.github/workflows/publish.yml +0 -0
  52. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.github/workflows/tests.yml +0 -0
  53. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.gitignore +0 -0
  54. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.markdownlint.yaml +0 -0
  55. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.pre-commit-config.yaml +0 -0
  56. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/.python-version +0 -0
  57. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/AGENTS.md +0 -0
  58. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/CONTRIBUTING.md +0 -0
  59. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/Dockerfile +0 -0
  60. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/LICENSE +0 -0
  61. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/docker-compose.yml +0 -0
  62. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/docs/Makefile +0 -0
  63. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/docs/make.bat +0 -0
  64. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/docs/requirements-docs.txt +0 -0
  65. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/docs/source/_extra/CNAME +0 -0
  66. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/main.py +0 -0
  67. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/service/__init__.py +0 -0
  68. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/service/forcingkit_serve/__init__.py +0 -0
  69. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/service/forcingkit_serve/routers/bathymetry.py +0 -0
  70. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/service/forcingkit_serve/routers/plotly_api.py +0 -0
  71. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/service/run_server.py +0 -0
  72. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/__init__.py +0 -0
  73. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/fetchers/erddap.py +0 -0
  74. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/fetchers/hrrr.py +0 -0
  75. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/fetchers/hrrr_atmosphere.py +0 -0
  76. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/fetchers/hydrography.py +0 -0
  77. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/fetchers/ndbc.py +0 -0
  78. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/settings.py +0 -0
  79. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/src/forcingkit/zarr_stream.py +0 -0
  80. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/static/favicon.ico +0 -0
  81. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/static/favicon.svg +0 -0
  82. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/static/logo.svg +0 -0
  83. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/static/styles.css +0 -0
  84. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/__init__.py +0 -0
  85. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/integration/test_auth.py +0 -0
  86. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/integration/test_erddap_fetch.py +0 -0
  87. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/integration/test_nyofs_obc_fetch.py +0 -0
  88. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/integration/test_obc_mab.py +0 -0
  89. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/integration/test_s3_roms_fetchers.py +0 -0
  90. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/scripts/inspect_dem.py +0 -0
  91. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/scripts/inspect_grib.py +0 -0
  92. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/scripts/inspect_zarr.py +0 -0
  93. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_dispatcher.py +0 -0
  94. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_hrrr_atm.py +0 -0
  95. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_hrrr_idx.py +0 -0
  96. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_main.py +0 -0
  97. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_ndbc.py +0 -0
  98. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_obc_pipeline.py +0 -0
  99. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_removed_routes.py +0 -0
  100. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_settings.py +0 -0
  101. {forcingkit-0.1.0.post1 → forcingkit-0.3.0}/tests/unit/test_zarr_stream.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: forcingkit
3
- Version: 0.1.0.post1
3
+ Version: 0.3.0
4
4
  Summary: Spatiotemporal forcing for computational Earth-system models: selects, regrids and serves model-ready time series with provenance.
5
5
  Project-URL: Documentation, https://forcingkit.docs.lhzn.io
6
6
  Project-URL: Issues, https://github.com/lhzn-io/forcingkit/issues
@@ -246,14 +246,16 @@ Description-Content-Type: text/markdown
246
246
  [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
247
247
 
248
248
  <div align="center">
249
- <img src="https://raw.githubusercontent.com/lhzn-io/forcingkit/main/docs/source/_static/forcingkit-inventory-screenshot.png" alt="forcingkit Dashboard" />
249
+ <img src="https://raw.githubusercontent.com/lhzn-io/forcingkit/main/docs/source/_static/forcingkit-noreaster-wind.gif" alt="Animation of the HRRR 10 m wind from the NY Bight to Cape Cod, every 3 hours from 2026-09-22 00:00 to 2026-09-29 12:00 UTC" width="720" />
250
+ <br />
251
+ <sub>HRRR 10 m wind served by forcingkit through the 26-27 September 2026 nor'easter, NY Bight to Cape Cod, from the calm of 22 September to the calm of 29 September, every 3 hours (peak 25.9 m/s). The forcingkit viewer's time slider, played.</sub>
250
252
  </div>
251
253
 
252
254
  Spatiotemporal forcing for computational Earth-system models: selects, regrids and serves model-ready time series with provenance.
253
255
 
254
256
  `forcingkit` sits between operational and archive data providers (NOAA HRRR, NECOFS, NOAA OFS, HYCOM, NDBC, CO-OPS) and model codes such as `Oceananigans.jl`. It delivers a model's parent ocean on true z levels (`/api/v1/obc`) and its HRRR surface atmosphere on a regular grid (`/api/v1/atmosphere`), plus station observations for validation, as schema-versioned Zarr stores that record their sources.
255
257
 
256
- forcingkit was named ecodata-cache until 2026-10-05; GitHub redirects the old repository URLs. The Python package is `forcingkit` (was `ecodata_cache`), and environment variables use the `FORCINGKIT_` prefix; the old names are still read, with a warning, until the next release. Documentation: <https://forcingkit.docs.lhzn.io>, including the [routes removed on the same date](https://forcingkit.docs.lhzn.io/removed_endpoints.html).
258
+ Documentation: <https://forcingkit.docs.lhzn.io>.
257
259
 
258
260
  ## Goals
259
261
 
@@ -271,14 +273,17 @@ The primary goal of forcingkit is to provide clean, standardized ocean and atmos
271
273
 
272
274
  ## Current Coverage
273
275
 
274
- - **Oceanic Forcing**: NYOFS, NECOFS, DBOFS, HYCOM (Operational & Historical).
276
+ - **Oceanic Forcing**: NYOFS, NECOFS, DBOFS, HYCOM (global, 1994 to present).
275
277
  - **Atmospheric Forcing**: HRRR (3 km, hourly, 2014-07-30 to present), regridded to 0.03 degrees.
276
278
 
277
279
  ## Roadmap
278
280
 
279
- - **Enhanced IOOS Coverage**: Expanding our data fetchers to support more regional nodes across the West Coast (WCOFS) and Gulf of Mexico (NGOFS2).
280
- - **Improved Caching Policies**: Implementing dynamic cache invalidation based on NOAA operational forecast updates to ensure realtime predictions stay synchronized.
281
- - **Variable Expansions**: Providing native spatiotemporal transformations for wave spectra and biogeochemical tracers.
281
+ - **Coverage first**: a second global parent (Copernicus GLO12, GLORYS12 for 1993 on, NOAA RTOFS), the remaining NOAA forecast systems (West Coast, Gulf, Chesapeake, Great Lakes, Alaska), the Doppio reanalysis for 2007 to 2024 in the Northeast, HRRR Alaska and ECMWF IFS for the atmosphere, and a first European ocean source from Copernicus Marine (IBI or the North West Shelf).
282
+ - **Resolution second**: the NYOFS fine grid, the Monterey Bay nests of WCOFS, and SFBOFS inside San Francisco Bay.
283
+ - **NYOFS and DBOFS as z-v3 parents**: both on true depths with geographic axes.
284
+ - **Forecast mode and cache policy**: HRRR forecast cycles, and invalidation of forecast-built stores when a newer cycle is published.
285
+
286
+ The full list, with what each candidate offers, is at <https://forcingkit.docs.lhzn.io/roadmap.html>. Contributions are welcome; see [CONTRIBUTING.md](CONTRIBUTING.md).
282
287
 
283
288
  ## Architecture overview
284
289
 
@@ -5,14 +5,16 @@
5
5
  [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
6
6
 
7
7
  <div align="center">
8
- <img src="https://raw.githubusercontent.com/lhzn-io/forcingkit/main/docs/source/_static/forcingkit-inventory-screenshot.png" alt="forcingkit Dashboard" />
8
+ <img src="https://raw.githubusercontent.com/lhzn-io/forcingkit/main/docs/source/_static/forcingkit-noreaster-wind.gif" alt="Animation of the HRRR 10 m wind from the NY Bight to Cape Cod, every 3 hours from 2026-09-22 00:00 to 2026-09-29 12:00 UTC" width="720" />
9
+ <br />
10
+ <sub>HRRR 10 m wind served by forcingkit through the 26-27 September 2026 nor'easter, NY Bight to Cape Cod, from the calm of 22 September to the calm of 29 September, every 3 hours (peak 25.9 m/s). The forcingkit viewer's time slider, played.</sub>
9
11
  </div>
10
12
 
11
13
  Spatiotemporal forcing for computational Earth-system models: selects, regrids and serves model-ready time series with provenance.
12
14
 
13
15
  `forcingkit` sits between operational and archive data providers (NOAA HRRR, NECOFS, NOAA OFS, HYCOM, NDBC, CO-OPS) and model codes such as `Oceananigans.jl`. It delivers a model's parent ocean on true z levels (`/api/v1/obc`) and its HRRR surface atmosphere on a regular grid (`/api/v1/atmosphere`), plus station observations for validation, as schema-versioned Zarr stores that record their sources.
14
16
 
15
- forcingkit was named ecodata-cache until 2026-10-05; GitHub redirects the old repository URLs. The Python package is `forcingkit` (was `ecodata_cache`), and environment variables use the `FORCINGKIT_` prefix; the old names are still read, with a warning, until the next release. Documentation: <https://forcingkit.docs.lhzn.io>, including the [routes removed on the same date](https://forcingkit.docs.lhzn.io/removed_endpoints.html).
17
+ Documentation: <https://forcingkit.docs.lhzn.io>.
16
18
 
17
19
  ## Goals
18
20
 
@@ -30,14 +32,17 @@ The primary goal of forcingkit is to provide clean, standardized ocean and atmos
30
32
 
31
33
  ## Current Coverage
32
34
 
33
- - **Oceanic Forcing**: NYOFS, NECOFS, DBOFS, HYCOM (Operational & Historical).
35
+ - **Oceanic Forcing**: NYOFS, NECOFS, DBOFS, HYCOM (global, 1994 to present).
34
36
  - **Atmospheric Forcing**: HRRR (3 km, hourly, 2014-07-30 to present), regridded to 0.03 degrees.
35
37
 
36
38
  ## Roadmap
37
39
 
38
- - **Enhanced IOOS Coverage**: Expanding our data fetchers to support more regional nodes across the West Coast (WCOFS) and Gulf of Mexico (NGOFS2).
39
- - **Improved Caching Policies**: Implementing dynamic cache invalidation based on NOAA operational forecast updates to ensure realtime predictions stay synchronized.
40
- - **Variable Expansions**: Providing native spatiotemporal transformations for wave spectra and biogeochemical tracers.
40
+ - **Coverage first**: a second global parent (Copernicus GLO12, GLORYS12 for 1993 on, NOAA RTOFS), the remaining NOAA forecast systems (West Coast, Gulf, Chesapeake, Great Lakes, Alaska), the Doppio reanalysis for 2007 to 2024 in the Northeast, HRRR Alaska and ECMWF IFS for the atmosphere, and a first European ocean source from Copernicus Marine (IBI or the North West Shelf).
41
+ - **Resolution second**: the NYOFS fine grid, the Monterey Bay nests of WCOFS, and SFBOFS inside San Francisco Bay.
42
+ - **NYOFS and DBOFS as z-v3 parents**: both on true depths with geographic axes.
43
+ - **Forecast mode and cache policy**: HRRR forecast cycles, and invalidation of forecast-built stores when a newer cycle is published.
44
+
45
+ The full list, with what each candidate offers, is at <https://forcingkit.docs.lhzn.io/roadmap.html>. Contributions are welcome; see [CONTRIBUTING.md](CONTRIBUTING.md).
41
46
 
42
47
  ## Architecture overview
43
48
 
@@ -0,0 +1,24 @@
1
+ /* Use more of a wide screen: the theme caps the content column at 800 px. */
2
+ .wy-nav-content {
3
+ max-width: 1200px;
4
+ }
5
+
6
+ /* Let table cells wrap instead of forcing a horizontal scrollbar. The theme sets nowrap on
7
+ every cell and puts tables in a scrolling container. */
8
+ .wy-table-responsive table td,
9
+ .wy-table-responsive table th {
10
+ white-space: normal;
11
+ }
12
+
13
+ /* On wide screens wrapped tables fit, so drop the scroll container; on phones keep it, so a
14
+ table too wide even when wrapped scrolls rather than being clipped. */
15
+ @media screen and (min-width: 768px) {
16
+ .wy-table-responsive {
17
+ overflow-x: visible;
18
+ }
19
+ }
20
+
21
+ /* Keep long code spans (archive paths, URLs) from widening a column past the page. */
22
+ .wy-table-responsive table td code {
23
+ overflow-wrap: anywhere;
24
+ }
@@ -2,7 +2,7 @@ Atmospheric Forcing Datasets
2
2
  ============================
3
3
 
4
4
  As of 2026-10-04. Which atmospheric datasets can force a coastal ocean model through this service
5
- or through NumericalEarth, how they compare, and which one the CoastalSim configurations use.
5
+ or through `NumericalEarth <https://github.com/NumericalEarth/NumericalEarth.jl>`__, how they compare, and which to choose.
6
6
 
7
7
  An ocean model's surface fluxes need, at every hour of the run: 10 m wind (eastward and
8
8
  northward), 2 m air temperature and specific humidity, surface pressure, precipitation, and
@@ -23,22 +23,23 @@ Summary
23
23
  - Output step
24
24
  - Coverage
25
25
  - Latency
26
- - Path in our stack
27
- * - **HRRR** (NOAA NCEP)
26
+ - Path to a model
27
+ * - **HRRR** (`NOAA GSL <https://gsl.noaa.gov/>`__, `NCEP <https://www.nco.ncep.noaa.gov/>`__)
28
28
  - Hourly-cycling forecast, 3 km convection-allowing, radar assimilation
29
29
  - 3 km, Lambert conformal over CONUS
30
30
  - 1 h (15 min for some fields)
31
31
  - CONUS; archive on AWS from 2014-07-30
32
32
  - About an hour after each cycle
33
- - **forcingkit** ``/api/v1/atmosphere`` (schema ``hrrr-atm-v1``); default for CoastalSim
34
- * - **ERA5** (ECMWF, Copernicus C3S)
33
+ - **forcingkit** ``/api/v1/atmosphere`` (schema ``hrrr-atm-v1``); the default
34
+ * - **ERA5** (`ECMWF <https://www.ecmwf.int/>`__, `Copernicus C3S
35
+ <https://climate.copernicus.eu/>`__)
35
36
  - Global reanalysis (4D-Var)
36
37
  - 0.25 degrees (about 31 km)
37
38
  - 1 h
38
39
  - 1940 to present
39
40
  - ERA5T about 5 days; final ERA5 2 to 3 months
40
- - **NumericalEarth** ``ERA5PrescribedAtmosphere`` (coastal-sim ``atmosphere = "era5"``)
41
- * - **RRFS v1** (NOAA NCEP)
41
+ - **NumericalEarth** ``ERA5PrescribedAtmosphere``
42
+ * - **RRFS v1** (`NOAA GSL <https://gsl.noaa.gov/rrfs/>`__, NCEP)
42
43
  - Hourly-cycling forecast, FV3 limited-area, 3 km
43
44
  - 3 km, North America
44
45
  - 1 h
@@ -52,28 +53,29 @@ Summary
52
53
  - Retired 2026-10-06 in favour of RRFS
53
54
  - Four cycles a day, to 60 h
54
55
  - Not integrated; do not adopt
55
- * - **GFS** (NOAA NCEP)
56
+ * - **GFS** (`NOAA NCEP
57
+ <https://www.emc.ncep.noaa.gov/emc/pages/numerical_forecast_systems/gfs.php>`__)
56
58
  - Global forecast, FV3
57
59
  - 0.25 degrees
58
60
  - 1 h to 120 h, then 3 h
59
61
  - Global; NODD archive on AWS
60
62
  - Four cycles a day, to 384 h
61
63
  - Not integrated; the natural extension past HRRR's 48 h in forecast mode
62
- * - **ECMWF IFS open data**
64
+ * - **ECMWF IFS** `open data <https://www.ecmwf.int/en/forecasts/datasets/open-data>`__
63
65
  - Global forecast
64
66
  - 0.25 degrees
65
67
  - 3 h, then 6 h
66
68
  - Global; real time
67
69
  - Four cycles a day, to 15 days (00 and 12 UTC)
68
70
  - Not integrated; CC-BY-4.0 since 2025-10-01
69
- * - **JRA55-do** (JMA, MRI)
71
+ * - **JRA55-do** (JMA, `MRI <https://www.mri-jma.go.jp/index_en.html>`__)
70
72
  - Reanalysis adjusted for ocean-sea-ice models
71
73
  - About 0.5 degrees
72
74
  - 3 h
73
75
  - 1958-01-01 to 2024-02-01, final version 1.6.0
74
76
  - Discontinued (JRA-55 ended January 2024; successor JRA-3Q)
75
77
  - **NumericalEarth** ``JRA55PrescribedAtmosphere`` (its catalogue ends 2019-12-31)
76
- * - **ECCO v4** (NASA JPL)
78
+ * - **ECCO v4** (`ECCO Consortium <https://ecco-group.org/>`__, NASA JPL)
77
79
  - Ocean state estimate's adjusted forcing
78
80
  - About 1 degree
79
81
  - Monthly
@@ -87,9 +89,13 @@ Datasets in use
87
89
  HRRR (default)
88
90
  ~~~~~~~~~~~~~~
89
91
 
90
- *Provenance.* NOAA NCEP's High-Resolution Rapid Refresh, version 4, a 3 km convection-allowing
91
- model that assimilates radar every 15 minutes and starts a new forecast every hour. Distributed
92
- through the NOAA Open Data Dissemination programme in the ``noaa-hrrr-bdp-pds`` bucket on AWS
92
+ *Provenance.* NOAA's `High-Resolution Rapid Refresh <https://rapidrefresh.noaa.gov/hrrr/>`__,
93
+ version 4, developed by the NOAA Global Systems Laboratory and run by NCEP: a 3 km
94
+ convection-allowing model that assimilates radar every 15 minutes and starts a new forecast every
95
+ hour (`Dowell et al., 2022 <https://doi.org/10.1175/WAF-D-21-0151.1>`__; `James et al., 2022
96
+ <https://doi.org/10.1175/WAF-D-21-0130.1>`__). Distributed through the `NOAA Open Data
97
+ Dissemination <https://www.noaa.gov/information-technology/open-data-dissemination>`__ programme
98
+ in the `noaa-hrrr-bdp-pds <https://registry.opendata.aws/noaa-hrrr-pds/>`__ bucket on AWS
93
99
  (us-east-1), anonymous, from 2014-07-30 to the present. NOAA open data: free to use; NOAA asks
94
100
  for attribution and that modified products not be presented as NOAA's.
95
101
 
@@ -134,8 +140,8 @@ atmosphere regridder accepts. Records run from one hour before the run start to
134
140
  end; a missing message or hour raises, and the store is published only when complete. Building
135
141
  one hour takes about 6.5 s on a warm connection, so a 168 h window takes about 19 minutes.
136
142
 
137
- *Strengths.* Resolves the land-sea contrast, sea breezes and frontal timing at the scale of our
138
- domains (MAB is 15 by 17 km; LIS 85 by 65 km). Hourly radiation and precipitation. No credentials.
143
+ *Strengths.* Resolves the land-sea contrast, sea breezes and frontal timing at the scale of
144
+ coastal domains from about 15 to 85 km across. Hourly radiation and precipitation. No credentials.
139
145
 
140
146
  *Limits.* CONUS only. A forecast product, not a reanalysis: it carries forecast-model biases and is
141
147
  not homogeneous across HRRR versions (v4 since December 2020). Radiation is instantaneous.
@@ -143,24 +149,25 @@ not homogeneous across HRRR versions (v4 since December 2020). Radiation is inst
143
149
  ERA5 (fallback)
144
150
  ~~~~~~~~~~~~~~~
145
151
 
146
- *Provenance.* ECMWF's fifth-generation global reanalysis for the Copernicus Climate Change
147
- Service: 0.25 degree grid (about 31 km), hourly, 1940 to the present. ERA5T, the initial release,
152
+ *Provenance.* `ECMWF <https://www.ecmwf.int/>`__'s fifth-generation global reanalysis for the
153
+ `Copernicus Climate Change Service <https://climate.copernicus.eu/>`__ (`Hersbach et al., 2020
154
+ <https://doi.org/10.1002/qj.3803>`__): 0.25 degree grid (about 31 km), hourly, 1940 to the present. ERA5T, the initial release,
148
155
  appears about five days behind real time and is overwritten by the final ERA5 two to three months
149
- later. Requires a free Copernicus Climate Data Store account (credentials in ``~/.cdsapirc``);
156
+ later. Requires a free `Copernicus Climate Data Store <https://cds.climate.copernicus.eu/>`__ account (credentials in ``~/.cdsapirc``);
150
157
  Copernicus licence, attribution required.
151
158
 
152
- *How it is used.* CoastalSim reads it through NumericalEarth's ``ERA5PrescribedAtmosphere`` and
153
- ``ERA5PrescribedRadiation`` over the bbox padded by 0.5 degrees, with linear time indexing and one
154
- hour of padding past the end. Accumulated fields (precipitation, radiation) are hour-ending means
155
- that NumericalEarth places at the centre of their hour. NumericalEarth 0.8.1's catalogue stops at
156
- 2025-12-31; coastal-sim extends it to six days before today until upstream rolls the date.
159
+ *How it is used.* A model reads it through NumericalEarth's ``ERA5PrescribedAtmosphere`` and
160
+ ``ERA5PrescribedRadiation``, for example over the bbox padded by 0.5 degrees, with linear time
161
+ indexing and one hour of padding past the end. Accumulated fields (precipitation, radiation) are
162
+ hour-ending means that NumericalEarth places at the centre of their hour. NumericalEarth 0.8.1's
163
+ catalogue stops at 2025-12-31; later dates need the catalogue extended until upstream rolls it.
157
164
 
158
165
  *Strengths.* Homogeneous, global, long, assimilates far more observations than any forecast; the
159
166
  standard against which forcing biases are judged.
160
167
 
161
- *Limits.* At 31 km a domain like MAB spans one or two ERA5 cells, so the forcing is nearly uniform
162
- and smears the coast: in the first hours of 2026-04-02 over MAB, the ERA5 box (which includes New
163
- Jersey land) was about 3 K warmer at 2 m and had about half HRRR's wind speed. Latency rules out
168
+ *Limits.* At 31 km a 15 km coastal domain spans one or two ERA5 cells, so the forcing is nearly
169
+ uniform and smears the coast: in the first hours of 2026-04-02 over such a domain off New Jersey,
170
+ the ERA5 box (which includes New Jersey land) was about 3 K warmer at 2 m and had about half HRRR's wind speed. Latency rules out
164
171
  anything closer than five days to the present.
165
172
 
166
173
  Choosing
@@ -177,17 +184,39 @@ Choosing
177
184
  * - Hindcast before 2014-07-30, outside CONUS, or a reanalysis-grade comparison run
178
185
  - ERA5
179
186
  * - Forecast to 48 h
180
- - HRRR forecast mode (planned, Sprint 4): one cycle's f01 to f48 from the 00, 06, 12 or 18 UTC
187
+ - HRRR forecast mode (planned): one cycle's f01 to f48 from the 00, 06, 12 or 18 UTC
181
188
  cycle
182
189
  * - Forecast beyond 48 h
183
190
  - HRRR to 48 h, then GFS or ECMWF IFS open data (not integrated); RRFS to 84 h once it is
184
191
  established in operations
185
192
  * - Multi-decade or climate-scale forcing
186
- - ERA5, or JRA55-do through NumericalEarth (to 2019 in its catalogue)
193
+ - ERA5, or JRA55-do (`Tsujino et al., 2018 <https://doi.org/10.1016/j.ocemod.2018.07.002>`__)
194
+ through NumericalEarth (to 2019 in its catalogue)
187
195
 
188
196
  References
189
197
  ----------
190
198
 
199
+ Citations
200
+ ~~~~~~~~~
201
+
202
+ - Dowell, D. C., C. R. Alexander, E. P. James, et al. (2022). The High-Resolution Rapid Refresh
203
+ (HRRR): An hourly updating convection-allowing forecast model. Part I: Motivation and system
204
+ description. *Weather and Forecasting*, 37(8), 1371-1395.
205
+ `doi:10.1175/WAF-D-21-0151.1 <https://doi.org/10.1175/WAF-D-21-0151.1>`__
206
+ - James, E. P., C. R. Alexander, D. C. Dowell, et al. (2022). The High-Resolution Rapid Refresh
207
+ (HRRR): An hourly updating convection-allowing forecast model. Part II: Forecast performance.
208
+ *Weather and Forecasting*, 37(8), 1397-1417.
209
+ `doi:10.1175/WAF-D-21-0130.1 <https://doi.org/10.1175/WAF-D-21-0130.1>`__
210
+ - Hersbach, H., B. Bell, P. Berrisford, et al. (2020). The ERA5 global reanalysis. *Quarterly
211
+ Journal of the Royal Meteorological Society*, 146(730), 1999-2049.
212
+ `doi:10.1002/qj.3803 <https://doi.org/10.1002/qj.3803>`__
213
+ - Tsujino, H., S. Urakawa, H. Nakano, et al. (2018). JRA-55 based surface dataset for driving
214
+ ocean-sea-ice models (JRA55-do). *Ocean Modelling*, 130, 79-139.
215
+ `doi:10.1016/j.ocemod.2018.07.002 <https://doi.org/10.1016/j.ocemod.2018.07.002>`__
216
+
217
+ Links
218
+ ~~~~~
219
+
191
220
  - NOAA HRRR on AWS: https://registry.opendata.aws/noaa-hrrr-pds/
192
221
  - NOAA RRFS: https://gsl.noaa.gov/rrfs/ ; operational date and retirements:
193
222
  https://gribstream.com/blog/noaa-rrfs-refs-operational-august-2026
@@ -9,7 +9,7 @@ sys.path.insert(0, os.path.abspath("../../src"))
9
9
  project = "forcingkit"
10
10
  copyright = "2026, Long Horizon Observatory"
11
11
  author = "Daniel Fry"
12
- release = "0.1.0.post1"
12
+ release = "0.3.0"
13
13
 
14
14
  extensions = [
15
15
  "sphinx.ext.autodoc",
@@ -26,5 +26,6 @@ language = "en"
26
26
 
27
27
  html_theme = "sphinx_rtd_theme"
28
28
  html_static_path = ["_static"]
29
+ html_css_files = ["custom.css"]
29
30
  # CNAME for the custom domain forcingkit.docs.lhzn.io, copied to the site root.
30
31
  html_extra_path = ["_extra"]
@@ -0,0 +1,191 @@
1
+ Data Fetchers
2
+ =============
3
+
4
+ A fetcher (``forcingkit.fetchers``) talks to one external provider: it resolves where the data
5
+ for a time lives, reads only the subset a request needs, and returns it in a common layout. The
6
+ dispatcher (``forcingkit.dispatcher``) chooses among fetchers, falls back when one cannot deliver,
7
+ and writes the result to the local Zarr cache, so a request is fetched from the provider once.
8
+
9
+ Atmospheric forcing (``/api/v1/atmosphere``)
10
+ --------------------------------------------
11
+
12
+ .. list-table::
13
+ :header-rows: 1
14
+ :widths: 14 86
15
+
16
+ * - Source
17
+ - Notes
18
+ * - **HRRR**
19
+ - NOAA `High-Resolution Rapid Refresh <https://rapidrefresh.noaa.gov/hrrr/>`__ (developed by
20
+ the `NOAA Global Systems Laboratory <https://gsl.noaa.gov/>`__, run operationally by
21
+ `NCEP <https://www.nco.ncep.noaa.gov/>`__), 3 km, from the anonymous AWS S3 bucket
22
+ `noaa-hrrr-bdp-pds <https://registry.opendata.aws/noaa-hrrr-pds/>`__; the archive starts 2014-07-30. Surface fields are interpolated to a
23
+ regular 0.03 degree grid and streamed hour by hour (schema ``hrrr-atm-v1``). Earlier dates
24
+ use `ERA5 <https://cds.climate.copernicus.eu/>`__ through
25
+ `NumericalEarth <https://github.com/NumericalEarth/NumericalEarth.jl>`__ in the model, outside this service. See
26
+ :doc:`atmospheric_forcing`.
27
+
28
+ Parent ocean (``/api/v1/obc``)
29
+ ------------------------------
30
+
31
+ .. list-table::
32
+ :header-rows: 1
33
+ :widths: 14 86
34
+
35
+ * - Source
36
+ - Notes
37
+ * - **NECOFS**
38
+ - `Northeast Coastal Ocean Forecast System <https://fvcom.smast.umassd.edu/?p=20>`__, `FVCOM
39
+ <https://fvcom.smast.umassd.edu/>`__ GOM7 (`UMass Dartmouth SMAST
40
+ <https://www.umassd.edu/smast/>`__ and `WHOI <https://www.whoi.edu/>`__), unstructured mesh with 45 sigma layers. Daily history files from 2025-01-01, then the rolling forecast.
41
+ The only donor that streams the ``z-v3`` parent store. See :doc:`necofs`.
42
+ * - **NYOFS**
43
+ - `NOAA New York/New Jersey Operational Forecast System
44
+ <https://tidesandcurrents.noaa.gov/ofs/nyofs/nyofs.html>`__ (`NOAA CO-OPS
45
+ <https://tidesandcurrents.noaa.gov/>`__), POM on a curvilinear grid with 7
46
+ sigma levels, NY/NJ Harbor. Ranked first inside its domain. 7-day CO-OPS aggregation,
47
+ then the nowcast archive (AWS S3 from 2024-11-19, NCEI from 2014). Legacy output layout
48
+ rather than ``z-v3``. See :doc:`nyofs`.
49
+ * - **DBOFS**
50
+ - `NOAA Delaware Bay Operational Forecast System
51
+ <https://tidesandcurrents.noaa.gov/ofs/dbofs/dbofs.html>`__ (NOAA CO-OPS),
52
+ `ROMS <https://www.myroms.org/>`__, about 100 m, domain
53
+ [-75.875, 37.810, -73.264, 40.206]. 7-day CO-OPS aggregation, then the hourly nowcast
54
+ archive (AWS S3 from 2024-11-19, NCEI from 2014). Legacy output layout.
55
+ * - :doc:`HYCOM <hycom>`
56
+ - `HYCOM <https://www.hycom.org/>`__ (1/12 degree, about 9 km, 40 z levels), global, from
57
+ the `HYCOM consortium data server <https://tds.hycom.org/thredds/catalog.html>`__, 1994 to
58
+ the present. The experiment follows the date: GLBv0.08 ``expt_53.X`` (reanalysis) to
59
+ 2015, a chain of GLBv0.08 analysis experiments to 2018-12-04, GLBy0.08 ``expt_93.0`` to
60
+ 2024-09-05, then ESPC-D-V02. A window that spans a switch is stitched; if any piece fails,
61
+ the whole window fails. Last-resort fallback; legacy output layout.
62
+
63
+ `TPXO10 <https://www.tpxo.net/>`__ tidal harmonics are a roadmap item, not integrated; the
64
+ `pyTMD <https://github.com/pyTMD/pyTMD>`__ dependency was removed on
65
+ 2026-10-05. The ``include_tides`` and ``tidal_model`` request fields
66
+ are still accepted and form part of the cache key, but no tide is added to the parent store.
67
+
68
+ Donor selection
69
+ ~~~~~~~~~~~~~~~
70
+
71
+ Each parent-ocean fetcher declares a domain box and an approximate resolution. A donor is a
72
+ candidate when it accepts the request box (NYOFS and DBOFS require the box to lie inside their
73
+ domain; NECOFS accepts any overlap; HYCOM accepts everything). Candidates are ranked by **domain
74
+ area first, smallest first**, then by resolution, on the reasoning that the most specialised
75
+ system is the best local one:
76
+
77
+ .. list-table::
78
+ :header-rows: 1
79
+ :widths: 20 30 20 30
80
+
81
+ * - Donor
82
+ - Declared domain (lon/lat)
83
+ - Area (degree squared)
84
+ - Declared resolution
85
+ * - NYOFS
86
+ - [-74.475, 40.389, -73.743, 40.940]
87
+ - 0.40
88
+ - 100 m
89
+ * - DBOFS
90
+ - [-75.875, 37.810, -73.264, 40.206]
91
+ - 6.3
92
+ - 100 m
93
+ * - NECOFS
94
+ - [-77.0, 35.0, -65.0, 46.0]
95
+ - 132
96
+ - 200 m
97
+ * - HYCOM
98
+ - global
99
+ - 64800
100
+ - 9 km
101
+
102
+ The dispatcher tries the candidates in order. A donor that returns nothing or raises is logged
103
+ and skipped unless the request sets ``allow_donor_fallback: false``, in which case the first
104
+ failure is an error. A donor that streams (NECOFS) either completes its store or fails; it never
105
+ publishes a shortened one.
106
+
107
+ Observations
108
+ ------------
109
+
110
+ These serve validation rather than forcing:
111
+
112
+ - **CO-OPS** water level (``/api/v1/tide``), from the `NOAA CO-OPS data API
113
+ <https://api.tidesandcurrents.noaa.gov/api/prod/>`__, with a datum
114
+ fallback to MSL where a station has no NAVD88 datum.
115
+ - **NDBC** buoy meteorology and ADCP profiles (``/api/v1/ndbc``), from the `NOAA National Data
116
+ Buoy Center <https://www.ndbc.noaa.gov/>`__.
117
+ - **UConn ERDDAP** water-column profiles (``/api/v1/telemetry/station`` and ``/bbox``), from the
118
+ `University of Connecticut Department of Marine Sciences <https://marinesciences.uconn.edu/>`__
119
+ `ERDDAP server <http://merlin.dms.uconn.edu:8080/erddap/index.html>`__.
120
+
121
+ NOAA OFS archive locations
122
+ --------------------------
123
+
124
+ As checked on 2026-10-06, NYOFS and DBOFS output is available from three places:
125
+
126
+ .. list-table::
127
+ :header-rows: 1
128
+ :widths: 22 30 48
129
+
130
+ * - Location
131
+ - Span
132
+ - Layout
133
+ * - `CO-OPS THREDDS <https://opendap.co-ops.nos.noaa.gov/thredds/catalog/catalog.html>`__ FMRC
134
+ aggregation
135
+ - Rolling 7 days
136
+ - One virtual OPeNDAP dataset per system
137
+ (``opendap.co-ops.nos.noaa.gov/thredds/dodsC/<OFS>/fmrc/Aggregated_7_day_<OFS>_Fields_Forecast_best.ncd``).
138
+ * - `NOAA NCEI <https://www.ncei.noaa.gov/>`__ THREDDS (`model-nyofs-files
139
+ <https://www.ncei.noaa.gov/thredds/catalog/model-nyofs-files/catalog.html>`__,
140
+ ``model-dbofs-files``)
141
+ - 2014 to the present month, with occasional missing days (NYOFS 2024-06 has 25 of 30)
142
+ - One directory per month, ``{yyyy}/{mm}/``, with every file of the month in it. File names
143
+ changed with the 2024-09-09 cycles (from ``nos.<ofs>.fields.<n|f><hhh>.<yyyymmdd>.t<cc>z.nc``
144
+ and ``nos.nyofs.fields.<nowcast|forecast>.<yyyymmdd>.t<cc>z.nc`` to
145
+ ``<ofs>.t<cc>z.<yyyymmdd>.fields.<...>.nc``).
146
+ * - AWS S3 `noaa-nos-ofs-pds <https://registry.opendata.aws/noaa-ofs/>`__ (`NOAA Open Data
147
+ Dissemination <https://www.noaa.gov/information-technology/open-data-dissemination>`__)
148
+ - 2024 to the present, with gaps before 2024-11-19
149
+ - From 2024-11-19, one directory per day, ``<ofs>/netcdf/{yyyy}/{mm}/{dd}/``; the first day
150
+ holds only its last cycle. Before that, one flat directory per month,
151
+ ``<ofs>/netcdf/{yyyymm}/``, with a mix of file names (NYOFS 2024-07 uses the older names,
152
+ DBOFS 2024-07 the newer ones); several of those months are partial and some are absent (NYOFS lacks 2024-01 to 2024-03, 2024-05
153
+ and 2024-06; DBOFS lacks 2024-02 and starts 2024-01 on the 29th).
154
+
155
+ .. note::
156
+ An earlier version of this page said that NCEI stopped archiving at the end of November 2023,
157
+ that AWS began in January 2024, and that December 2023 was therefore missing. None of that
158
+ holds: NCEI has every day of December 2023 for both NYOFS and DBOFS (124 NYOFS and 744 DBOFS
159
+ field files) and has continued archiving through 2026, while the AWS archive is the one with
160
+ gaps.
161
+
162
+ The NYOFS and DBOFS fetchers read nowcast files only, so a window is a chain of analyses
163
+ rather than forecasts. For each file they try AWS (per-day layout) first when the date is
164
+ 2024-11-19 or later, then NCEI, which covers every date; the flat AWS month directories are
165
+ not used. A window with a file missing from both is not served by that donor, and the
166
+ dispatcher falls back. DBOFS publishes one file per hour: file ``n001`` to ``n006`` of cycle
167
+ ``t<cc>z`` (00, 06, 12, 18 UTC) hold hours ``cc - 5`` to ``cc``. NYOFS publishes one file per
168
+ cycle (05, 11, 17, 23 UTC) holding the same six hours.
169
+
170
+ Both fetchers were corrected on 2026-10-06. Before that, every NYOFS archive date failed (see
171
+ :doc:`nyofs`), and DBOFS sent 2024 dates before 2024-11-19 to an AWS layout that does not
172
+ hold them, took some hours from the wrong file, joined NCEI files along a ``time`` dimension
173
+ they do not have (theirs is ``ocean_time``), and left the ROMS fill value (1e37) in land
174
+ cells read through NCEI.
175
+
176
+ For the AWS layout, see the `NOAA NODD OFS documentation
177
+ <https://github.com/NOAA-Big-Data-Program/nodd-data-docs/blob/main/OFS/README.md>`_ and the
178
+ `Registry of Open Data entry <https://registry.opendata.aws/noaa-ofs/>`_.
179
+
180
+ Common processing
181
+ -----------------
182
+
183
+ - **Hourly time axis.** Non-streaming donors are resampled to a strict hourly index by linear
184
+ interpolation. Streaming donors deliver hourly records directly.
185
+ - **Float32 fields, full-precision coordinates.** Data variables are cast to Float32 for the
186
+ model; ``lat``, ``lon``, ``z`` and ``z_face`` keep full precision, since Float32 longitudes near
187
+ -74 resolve only about 8e-6 degrees.
188
+ - **Fill values.** Missing data is written as ``-9999`` (integers) or ``-9999.0`` (floats) in the
189
+ Zarr encoding, and as NaN in memory.
190
+ - **Time in seconds.** ``z-v3`` stores encode time as seconds since their first record, which
191
+ the model's clock uses directly.