hefty 0.1.5__tar.gz → 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {hefty-0.1.5/src/hefty.egg-info → hefty-0.2.0}/PKG-INFO +150 -77
- {hefty-0.1.5 → hefty-0.2.0}/README.md +148 -75
- {hefty-0.1.5 → hefty-0.2.0}/pyproject.toml +2 -2
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty/solar.py +729 -340
- {hefty-0.1.5 → hefty-0.2.0/src/hefty.egg-info}/PKG-INFO +150 -77
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty.egg-info/requires.txt +1 -1
- {hefty-0.1.5 → hefty-0.2.0}/LICENSE +0 -0
- {hefty-0.1.5 → hefty-0.2.0}/setup.cfg +0 -0
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty/__init__.py +0 -0
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty/custom.py +0 -0
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty/pv_model.py +0 -0
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty/utilities.py +0 -0
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty/wind.py +0 -0
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty.egg-info/SOURCES.txt +0 -0
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty.egg-info/dependency_links.txt +0 -0
- {hefty-0.1.5 → hefty-0.2.0}/src/hefty.egg-info/top_level.txt +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: hefty
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.2.0
|
|
4
4
|
Summary: Some (relatively) lightweight short-term energy forecasting tools for solar, wind, and load.
|
|
5
5
|
Author: Will Hobbs
|
|
6
6
|
License-Expression: BSD-3-Clause
|
|
@@ -14,24 +14,126 @@ Classifier: Topic :: Scientific/Engineering
|
|
|
14
14
|
Requires-Python: >=3.11
|
|
15
15
|
Description-Content-Type: text/markdown
|
|
16
16
|
License-File: LICENSE
|
|
17
|
-
Requires-Dist: herbie-data[
|
|
17
|
+
Requires-Dist: herbie-data[scikit-learn]>=2026.3.0
|
|
18
18
|
Requires-Dist: pvlib>=0.13.1
|
|
19
19
|
Requires-Dist: pandas>=3.0.0
|
|
20
20
|
Requires-Dist: urllib3>=2.6.0
|
|
21
21
|
Dynamic: license-file
|
|
22
22
|
|
|
23
|
-
#
|
|
23
|
+
# hefty
|
|
24
24
|
Some (relatively) lightweight short-term **e**nergy **f**orecasting **t**ools for solar, wind, and load.
|
|
25
25
|
|
|
26
|
-
This repository currently includes solar and wind tools, but may expand one day to include electric load. Forecasts can be created using the
|
|
26
|
+
This repository currently includes solar and wind tools, but may expand one day to include electric load. Forecasts can be created using the following Numerical Weather Prediction (NWP) and Machine Learning Weather Prediction (MLWP) models:
|
|
27
|
+
- NOAA GFS
|
|
28
|
+
- NOAA GEFS
|
|
29
|
+
- NOAA HRRR
|
|
30
|
+
- ECMWF IFS (open data version)
|
|
31
|
+
- ECMWF AIFS
|
|
32
|
+
- ECMWF CAMS version of IFS* (a.k.a. "IFS-COMPO")
|
|
27
33
|
|
|
28
|
-
|
|
34
|
+
A detailed pvlib-based PV weather-to-power model is included in [pv_model.py](src/hefty/pv_model.py), `model_pv_power()`.
|
|
29
35
|
|
|
30
|
-
|
|
36
|
+
The [custom.py](src/hefty/custom.py) module is intended to help with getting forecasts of "custom" weather parameters, not necessarily specific to wind or solar, which migh be useful for load forecasting.
|
|
31
37
|
|
|
32
|
-
|
|
38
|
+
_*CAMS IFS is only available via `hefty.solar.get_solar_forecast()`, and it requires `cdsapi` to be installed and the user needs an API key (see https://ads.atmosphere.copernicus.eu/how-to-api and the [CAMS example notebook](examples/cams_example.ipynb))._
|
|
33
39
|
|
|
34
|
-
|
|
40
|
+
## Contents
|
|
41
|
+
- [Installation](#installation) - getting started
|
|
42
|
+
- [Quick example](#quick-examples) - a basic example of what hefty can do
|
|
43
|
+
- [Example notebooks](#example-notebooks) - overview of the provided Jupyter notebook examples
|
|
44
|
+
- [Supported forecast models](#supported-forecast-models) - description of NWP/MLWP models in hefty
|
|
45
|
+
- [Handling dates and times](#handling-dates-and-times) - dates/times are comlicated with forecasting
|
|
46
|
+
- [Local disk space](#local-disk-space) - downloads via hefty can use up a lot of storage space
|
|
47
|
+
- [Troubleshooting](#troubleshooting) - common issues and possible solutions
|
|
48
|
+
- [Data attribution](#data-attribution) - for using ECMWF data in particular
|
|
49
|
+
- [References](#references)
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
## Installation
|
|
53
|
+
|
|
54
|
+
A virtual environment is strongly recommended. You can install from PyPi with:
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
pip install hefty
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
To run the example Jupyter notebooks, you also need `jupyter`:
|
|
61
|
+
|
|
62
|
+
```
|
|
63
|
+
pip install jupyter
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
If you want to use ECMWF CAMS, you also need `cdsapi` (and an API key, see https://ads.atmosphere.copernicus.eu/how-to-api):
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
pip install cdsapi
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
## Quick example
|
|
73
|
+
|
|
74
|
+
Here's a quick example of getting a solar resource data forecast, assuming you have already determined the dates/times needed:
|
|
75
|
+
|
|
76
|
+
```python
|
|
77
|
+
from hefty.solar import get_solar_forecast
|
|
78
|
+
|
|
79
|
+
latitude = 33.5
|
|
80
|
+
longitude = -86.8
|
|
81
|
+
init_date = '2024-06-05 6:00' # datetime the forecast model was initialized
|
|
82
|
+
resource_data = get_solar_forecast(
|
|
83
|
+
latitude,
|
|
84
|
+
longitude,
|
|
85
|
+
init_date,
|
|
86
|
+
run_length=18, # 18 hours are included in the forecast
|
|
87
|
+
lead_time_to_start=3, # forecast starts 3 hours out from the init_date
|
|
88
|
+
model='hrrr', # use NOAA HRRR
|
|
89
|
+
)
|
|
90
|
+
resource_data[
|
|
91
|
+
['ghi','dni','dhi','temp_air','wind_speed']
|
|
92
|
+
].plot(drawstyle='steps-mid')
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
with this output:
|
|
96
|
+
|
|
97
|
+
<img src="images/output.png" width="500"/>
|
|
98
|
+
|
|
99
|
+
<!-- Here's a wind resource forecast:
|
|
100
|
+
|
|
101
|
+
```python
|
|
102
|
+
from hefty.wind import get_wind_forecast
|
|
103
|
+
|
|
104
|
+
latitude = 33.5
|
|
105
|
+
longitude = -86.8
|
|
106
|
+
init_date = '2024-06-05 6:00' # datetime the forecast model was initialized
|
|
107
|
+
resource_data = get_wind_forecast(
|
|
108
|
+
latitude,
|
|
109
|
+
longitude,
|
|
110
|
+
init_date,
|
|
111
|
+
run_length=18, # 18 hours are included in the forecast
|
|
112
|
+
lead_time_to_start=3, # forecast starts 3 hours out from the init_date
|
|
113
|
+
model='gfs', # use NOAA GFS
|
|
114
|
+
)
|
|
115
|
+
resource_data[
|
|
116
|
+
['wind_speed_10m', 'wind_speed_80m',
|
|
117
|
+
'wind_speed_100m', 'temp_air_2m',
|
|
118
|
+
'pressure_0m']
|
|
119
|
+
].plot(secondary_y=['pressure_0m'], drawstyle='steps-mid')
|
|
120
|
+
```
|
|
121
|
+
with this output (note that pressure is on the secondary y-axis):
|
|
122
|
+
|
|
123
|
+
<img src="images/output_wind.png" width="500"/> -->
|
|
124
|
+
|
|
125
|
+
## Example notebooks
|
|
126
|
+
|
|
127
|
+
The [examples](examples) folder contains several example Jupyter notebooks.
|
|
128
|
+
|
|
129
|
+
- [solar_example.ipynb](examples/solar_example.ipynb): Example with multiple weather models, and includes power modeling with `model_pv_power()`.
|
|
130
|
+
- [more_solar_examples.ipynb](examples/more_solar_examples.ipynb): Expanded examples, including multiple sites and multiple forecast initialization times.
|
|
131
|
+
- [cams_example.ipynb](examples/cams_example.ipynb): Solar resource forecasting with CAMS IFS.
|
|
132
|
+
- [ensemble_example.ipynb](examples/ensemble_example.ipynb): Examples of multi-member ensembles with IFS, AIFS, and GEFS.
|
|
133
|
+
- [wind_example.ipynb](examples/wind_example.ipynb): A basic wind resource forecast example using a few models, including converting the data to a format compatible with `windpowerlib`*.
|
|
134
|
+
- [erbs_vs_dirindex.ipynb](examples/erbs_vs_dirindex.ipynb): A comparison of Erbs and DIRINDEX irradiance decomposition models using SURFRAD measurements.
|
|
135
|
+
|
|
136
|
+
\* https://github.com/wind-python/windpowerlib/
|
|
35
137
|
|
|
36
138
|
## Supported forecast models
|
|
37
139
|
|
|
@@ -61,7 +163,7 @@ Custom forecasts use the `period`, `product`, and `search_str` arguments
|
|
|
61
163
|
passed to `get_custom_forecast()`, rather than the model-specific
|
|
62
164
|
forecast-hour formatter used by the solar and wind tools.
|
|
63
165
|
|
|
64
|
-
|
|
166
|
+
## Handling dates and times
|
|
65
167
|
|
|
66
168
|
Handling dates and times can get a bit complicated when it comes to forecasts. hefty tries to match conventions used in the Solar Forecast Arbiter (https://forecastarbiter.epri.com/definitions/), such as "_lead time to start_" and "_run length_". However, the Arbiter uses the term "_issue time_" to represent the time that a forecast is issued/delivered, but that time is not necessarily directly relevant to NWP/MLWP outputs.
|
|
67
169
|
|
|
@@ -120,70 +222,59 @@ Model delays are more than just a fixed time: they can vary by lead time and by
|
|
|
120
222
|
|
|
121
223
|
Delays calculated in `adjust_forecast_datetimes` are based on experiments in this gist https://gist.github.com/williamhobbs/9585ff5d1248ab5de4d9e8665d7c8ea6 (which will hopefully one day be cleaned up and added to this repo somehow), documentation published by ECMWF, and this cool dashboard by dynamical.org https://dynamical.org/status/.
|
|
122
224
|
|
|
123
|
-
##
|
|
225
|
+
## Local disk space
|
|
226
|
+
The current version of hefty keeps copies of downloaded grib files on your local storage. This can result in a large amount of disk space being used up. Future versions of hefty may change this (see [issue #70](https://github.com/williamhobbs/hefty/issues/70)), but for now, users may need to monitor storage space and manually delete files if needed.
|
|
124
227
|
|
|
125
|
-
|
|
228
|
+
hefty uses the directory configured by Herbie for storing files. See Herbie documentation here for details https://herbie.readthedocs.io/en/stable/user_guide/configure.html. The default `~\data` folder on Windows is `C:\Users\[username]\data`.
|
|
126
229
|
|
|
127
|
-
|
|
128
|
-
from hefty.solar import get_solar_forecast
|
|
230
|
+
## Troubleshooting
|
|
129
231
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
longitude,
|
|
136
|
-
init_date,
|
|
137
|
-
run_length=18, # 18 hours are included in the forecast
|
|
138
|
-
lead_time_to_start=3, # forecast starts 3 hours out from the init_date
|
|
139
|
-
model='hrrr', # use NOAA HRRR
|
|
140
|
-
)
|
|
141
|
-
resource_data[
|
|
142
|
-
['ghi','dni','dhi','temp_air','wind_speed']
|
|
143
|
-
].plot(drawstyle='steps-mid')
|
|
232
|
+
### GRIB file issues
|
|
233
|
+
GRIB files can occasionally download with missing/incomplete artifacts. This has been known to cause errors like:
|
|
234
|
+
|
|
235
|
+
```
|
|
236
|
+
TypeError: objects must be an iterable containing only DataTree(s), Dataset(s), DataArray(s), and dictionaries: ['t2m']
|
|
144
237
|
```
|
|
145
238
|
|
|
146
|
-
|
|
239
|
+
If forecasts contain unexpected missing/incomplete values, try deleting cached GRIB files from your
|
|
240
|
+
Herbie cache. See the Herbie cache configuration docs:
|
|
241
|
+
https://herbie.readthedocs.io/en/stable/user_guide/configure.html. Navigating to the corresponding `~\data` folder, e.g., `C:\Users\[username]\data\ifs\20240605`, and deleting the contents may fix this.
|
|
147
242
|
|
|
148
|
-
|
|
243
|
+
If deleting the cache does not resolve the issue, try downloading the same
|
|
244
|
+
forecast from a different Herbie source, such as Azure, Google, or AWS.
|
|
149
245
|
|
|
150
|
-
Here's a wind resource forecast:
|
|
151
246
|
|
|
152
|
-
|
|
153
|
-
|
|
247
|
+
### Server issues
|
|
248
|
+
ECMWF IFS/AIFS data have known issues via AWS that can result in 503 SlowDown responses. See https://forum.ecmwf.int/t/503-slowdown-errors-for-requests-to-s3-ecmwf-forecasts/14345, https://github.com/blaylockbk/Herbie/discussions/487. Using `priority='google'` (or `'azure'`) will likely help.
|
|
154
249
|
|
|
155
|
-
|
|
156
|
-
longitude = -86.8
|
|
157
|
-
init_date = '2024-06-05 6:00' # datetime the forecast model was initialized
|
|
158
|
-
resource_data = get_wind_forecast(
|
|
159
|
-
latitude,
|
|
160
|
-
longitude,
|
|
161
|
-
init_date,
|
|
162
|
-
run_length=18, # 18 hours are included in the forecast
|
|
163
|
-
lead_time_to_start=3, # forecast starts 3 hours out from the init_date
|
|
164
|
-
model='gfs', # use NOAA GFS
|
|
165
|
-
)
|
|
166
|
-
resource_data[
|
|
167
|
-
['wind_speed_10m', 'wind_speed_80m',
|
|
168
|
-
'wind_speed_100m', 'temp_air_2m',
|
|
169
|
-
'pressure_0m']
|
|
170
|
-
].plot(secondary_y=['pressure_0m'], drawstyle='steps-mid')
|
|
171
|
-
```
|
|
172
|
-
with this output (note that pressure is on the secondary y-axis):
|
|
250
|
+
Conversely, AWS may be more reliable than Google or Azure for NOAA models.
|
|
173
251
|
|
|
174
|
-
|
|
252
|
+
### Install issues, Python version
|
|
175
253
|
|
|
176
|
-
|
|
254
|
+
If you have problems installing dependencies or get errors related to `eccodes` (e.g., `RuntimeError: Cannot find the ecCodes library`), one possibility is that you are using a version of Python that is not yet fully supported by some dependencies. Among other possibilities, ECMWF appears to sometimes get behind in publishing wheels for all platforms. If you are using one of the latest versions of Python, downgrading to an earlier version that is till in "bugfix" status, or even "security", may fix these issues.
|
|
177
255
|
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
Herbie cache. See the Herbie cache configuration docs:
|
|
181
|
-
https://herbie.readthedocs.io/en/stable/user_guide/configure.html.
|
|
256
|
+
### Firewall/certificate issues
|
|
257
|
+
If you are using `hefty` behind a firewall, you might run into SLL/certificate issues. Making sure that Python knows where to find valid local certificates might help. _Potential_ solutions include the following, but check with your IT/security resources first:
|
|
182
258
|
|
|
183
|
-
|
|
184
|
-
|
|
259
|
+
**Option 1**: Install `pip-system-certs`:
|
|
260
|
+
|
|
261
|
+
```bash
|
|
262
|
+
pip install pip_system_certs
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
**Option 2**: Use `certifi` and set environment variables at the beginning of each script/notebook:
|
|
266
|
+
|
|
267
|
+
```python
|
|
268
|
+
import certifi
|
|
269
|
+
# Set requests to use the certificates in your current environment
|
|
270
|
+
os.environ['REQUESTS_CA_BUNDLE'] = certifi.where()
|
|
271
|
+
os.environ['CURL_CA_BUNDLE'] = certifi.where()
|
|
272
|
+
```
|
|
185
273
|
|
|
186
|
-
|
|
274
|
+
### Intermittent SSL errors
|
|
275
|
+
In some instances, intermittent SSL errors (`Exception has occured : Processing failed:`... `[SSL: UNEXPECTED_EOF_WHILE_READING]` ...) _might_ be fixed by upgrading urllib3, e.g., `pip install --upgrade urllib3`.
|
|
276
|
+
|
|
277
|
+
## Data attribution
|
|
187
278
|
### ECMWF
|
|
188
279
|
If you use models from ECMWF, you may need to include attribution language with the results.
|
|
189
280
|
|
|
@@ -200,25 +291,7 @@ A possible example attribution and citation for the CAMS version of IFS:
|
|
|
200
291
|
> Contains modified Copernicus Atmosphere Monitoring Service information [2026]. Neither the European Commission nor ECMWF is responsible for any use that may be made of the Copernicus information or data it contains. © 2026 European Centre for Medium-Range Weather Forecasts (ECMWF), www.ecmwf.int. This data is published under a Creative Commons Attribution 4.0 International (CC BY 4.0). https://creativecommons.org/licenses/by/4.0/.
|
|
201
292
|
>
|
|
202
293
|
> Copernicus Atmosphere Monitoring Service (2021): CAMS global atmospheric composition forecasts. Copernicus Atmosphere Monitoring Service (CAMS) Atmosphere Data Store, DOI: 10.24381/04a0b097 (Accessed on DD-MMM-YYYY).
|
|
203
|
-
## Installation
|
|
204
|
-
|
|
205
|
-
A virtual environment is strongly recommended. You can install from PyPi with:
|
|
206
|
-
|
|
207
|
-
```
|
|
208
|
-
pip install hefty
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
To run the example Jupyter notebooks, you also need `jupyter`:
|
|
212
|
-
|
|
213
|
-
```
|
|
214
|
-
pip install jupyter
|
|
215
|
-
```
|
|
216
294
|
|
|
217
|
-
If you want to use ECMWF CAMS, you also need `cdsapi`:
|
|
218
|
-
|
|
219
|
-
```
|
|
220
|
-
pip install cdsapi
|
|
221
|
-
```
|
|
222
295
|
|
|
223
296
|
## References
|
|
224
297
|
This project uses several Python packages, including pvlib, an open-source solar PV modeling package [1, 2], and Herbie [3, 4], a package for accessing weather forecast data from NOAA. `pv_model.py` (with the `model_pv_power()` function used here) comes from [5] which leverages some functions from [6].
|
|
@@ -1,15 +1,117 @@
|
|
|
1
|
-
#
|
|
1
|
+
# hefty
|
|
2
2
|
Some (relatively) lightweight short-term **e**nergy **f**orecasting **t**ools for solar, wind, and load.
|
|
3
3
|
|
|
4
|
-
This repository currently includes solar and wind tools, but may expand one day to include electric load. Forecasts can be created using the
|
|
4
|
+
This repository currently includes solar and wind tools, but may expand one day to include electric load. Forecasts can be created using the following Numerical Weather Prediction (NWP) and Machine Learning Weather Prediction (MLWP) models:
|
|
5
|
+
- NOAA GFS
|
|
6
|
+
- NOAA GEFS
|
|
7
|
+
- NOAA HRRR
|
|
8
|
+
- ECMWF IFS (open data version)
|
|
9
|
+
- ECMWF AIFS
|
|
10
|
+
- ECMWF CAMS version of IFS* (a.k.a. "IFS-COMPO")
|
|
5
11
|
|
|
6
|
-
|
|
12
|
+
A detailed pvlib-based PV weather-to-power model is included in [pv_model.py](src/hefty/pv_model.py), `model_pv_power()`.
|
|
7
13
|
|
|
8
|
-
|
|
14
|
+
The [custom.py](src/hefty/custom.py) module is intended to help with getting forecasts of "custom" weather parameters, not necessarily specific to wind or solar, which migh be useful for load forecasting.
|
|
9
15
|
|
|
10
|
-
|
|
16
|
+
_*CAMS IFS is only available via `hefty.solar.get_solar_forecast()`, and it requires `cdsapi` to be installed and the user needs an API key (see https://ads.atmosphere.copernicus.eu/how-to-api and the [CAMS example notebook](examples/cams_example.ipynb))._
|
|
11
17
|
|
|
12
|
-
|
|
18
|
+
## Contents
|
|
19
|
+
- [Installation](#installation) - getting started
|
|
20
|
+
- [Quick example](#quick-examples) - a basic example of what hefty can do
|
|
21
|
+
- [Example notebooks](#example-notebooks) - overview of the provided Jupyter notebook examples
|
|
22
|
+
- [Supported forecast models](#supported-forecast-models) - description of NWP/MLWP models in hefty
|
|
23
|
+
- [Handling dates and times](#handling-dates-and-times) - dates/times are comlicated with forecasting
|
|
24
|
+
- [Local disk space](#local-disk-space) - downloads via hefty can use up a lot of storage space
|
|
25
|
+
- [Troubleshooting](#troubleshooting) - common issues and possible solutions
|
|
26
|
+
- [Data attribution](#data-attribution) - for using ECMWF data in particular
|
|
27
|
+
- [References](#references)
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
## Installation
|
|
31
|
+
|
|
32
|
+
A virtual environment is strongly recommended. You can install from PyPi with:
|
|
33
|
+
|
|
34
|
+
```
|
|
35
|
+
pip install hefty
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
To run the example Jupyter notebooks, you also need `jupyter`:
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
pip install jupyter
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
If you want to use ECMWF CAMS, you also need `cdsapi` (and an API key, see https://ads.atmosphere.copernicus.eu/how-to-api):
|
|
45
|
+
|
|
46
|
+
```
|
|
47
|
+
pip install cdsapi
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
## Quick example
|
|
51
|
+
|
|
52
|
+
Here's a quick example of getting a solar resource data forecast, assuming you have already determined the dates/times needed:
|
|
53
|
+
|
|
54
|
+
```python
|
|
55
|
+
from hefty.solar import get_solar_forecast
|
|
56
|
+
|
|
57
|
+
latitude = 33.5
|
|
58
|
+
longitude = -86.8
|
|
59
|
+
init_date = '2024-06-05 6:00' # datetime the forecast model was initialized
|
|
60
|
+
resource_data = get_solar_forecast(
|
|
61
|
+
latitude,
|
|
62
|
+
longitude,
|
|
63
|
+
init_date,
|
|
64
|
+
run_length=18, # 18 hours are included in the forecast
|
|
65
|
+
lead_time_to_start=3, # forecast starts 3 hours out from the init_date
|
|
66
|
+
model='hrrr', # use NOAA HRRR
|
|
67
|
+
)
|
|
68
|
+
resource_data[
|
|
69
|
+
['ghi','dni','dhi','temp_air','wind_speed']
|
|
70
|
+
].plot(drawstyle='steps-mid')
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
with this output:
|
|
74
|
+
|
|
75
|
+
<img src="images/output.png" width="500"/>
|
|
76
|
+
|
|
77
|
+
<!-- Here's a wind resource forecast:
|
|
78
|
+
|
|
79
|
+
```python
|
|
80
|
+
from hefty.wind import get_wind_forecast
|
|
81
|
+
|
|
82
|
+
latitude = 33.5
|
|
83
|
+
longitude = -86.8
|
|
84
|
+
init_date = '2024-06-05 6:00' # datetime the forecast model was initialized
|
|
85
|
+
resource_data = get_wind_forecast(
|
|
86
|
+
latitude,
|
|
87
|
+
longitude,
|
|
88
|
+
init_date,
|
|
89
|
+
run_length=18, # 18 hours are included in the forecast
|
|
90
|
+
lead_time_to_start=3, # forecast starts 3 hours out from the init_date
|
|
91
|
+
model='gfs', # use NOAA GFS
|
|
92
|
+
)
|
|
93
|
+
resource_data[
|
|
94
|
+
['wind_speed_10m', 'wind_speed_80m',
|
|
95
|
+
'wind_speed_100m', 'temp_air_2m',
|
|
96
|
+
'pressure_0m']
|
|
97
|
+
].plot(secondary_y=['pressure_0m'], drawstyle='steps-mid')
|
|
98
|
+
```
|
|
99
|
+
with this output (note that pressure is on the secondary y-axis):
|
|
100
|
+
|
|
101
|
+
<img src="images/output_wind.png" width="500"/> -->
|
|
102
|
+
|
|
103
|
+
## Example notebooks
|
|
104
|
+
|
|
105
|
+
The [examples](examples) folder contains several example Jupyter notebooks.
|
|
106
|
+
|
|
107
|
+
- [solar_example.ipynb](examples/solar_example.ipynb): Example with multiple weather models, and includes power modeling with `model_pv_power()`.
|
|
108
|
+
- [more_solar_examples.ipynb](examples/more_solar_examples.ipynb): Expanded examples, including multiple sites and multiple forecast initialization times.
|
|
109
|
+
- [cams_example.ipynb](examples/cams_example.ipynb): Solar resource forecasting with CAMS IFS.
|
|
110
|
+
- [ensemble_example.ipynb](examples/ensemble_example.ipynb): Examples of multi-member ensembles with IFS, AIFS, and GEFS.
|
|
111
|
+
- [wind_example.ipynb](examples/wind_example.ipynb): A basic wind resource forecast example using a few models, including converting the data to a format compatible with `windpowerlib`*.
|
|
112
|
+
- [erbs_vs_dirindex.ipynb](examples/erbs_vs_dirindex.ipynb): A comparison of Erbs and DIRINDEX irradiance decomposition models using SURFRAD measurements.
|
|
113
|
+
|
|
114
|
+
\* https://github.com/wind-python/windpowerlib/
|
|
13
115
|
|
|
14
116
|
## Supported forecast models
|
|
15
117
|
|
|
@@ -39,7 +141,7 @@ Custom forecasts use the `period`, `product`, and `search_str` arguments
|
|
|
39
141
|
passed to `get_custom_forecast()`, rather than the model-specific
|
|
40
142
|
forecast-hour formatter used by the solar and wind tools.
|
|
41
143
|
|
|
42
|
-
|
|
144
|
+
## Handling dates and times
|
|
43
145
|
|
|
44
146
|
Handling dates and times can get a bit complicated when it comes to forecasts. hefty tries to match conventions used in the Solar Forecast Arbiter (https://forecastarbiter.epri.com/definitions/), such as "_lead time to start_" and "_run length_". However, the Arbiter uses the term "_issue time_" to represent the time that a forecast is issued/delivered, but that time is not necessarily directly relevant to NWP/MLWP outputs.
|
|
45
147
|
|
|
@@ -98,70 +200,59 @@ Model delays are more than just a fixed time: they can vary by lead time and by
|
|
|
98
200
|
|
|
99
201
|
Delays calculated in `adjust_forecast_datetimes` are based on experiments in this gist https://gist.github.com/williamhobbs/9585ff5d1248ab5de4d9e8665d7c8ea6 (which will hopefully one day be cleaned up and added to this repo somehow), documentation published by ECMWF, and this cool dashboard by dynamical.org https://dynamical.org/status/.
|
|
100
202
|
|
|
101
|
-
##
|
|
203
|
+
## Local disk space
|
|
204
|
+
The current version of hefty keeps copies of downloaded grib files on your local storage. This can result in a large amount of disk space being used up. Future versions of hefty may change this (see [issue #70](https://github.com/williamhobbs/hefty/issues/70)), but for now, users may need to monitor storage space and manually delete files if needed.
|
|
102
205
|
|
|
103
|
-
|
|
206
|
+
hefty uses the directory configured by Herbie for storing files. See Herbie documentation here for details https://herbie.readthedocs.io/en/stable/user_guide/configure.html. The default `~\data` folder on Windows is `C:\Users\[username]\data`.
|
|
104
207
|
|
|
105
|
-
|
|
106
|
-
from hefty.solar import get_solar_forecast
|
|
208
|
+
## Troubleshooting
|
|
107
209
|
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
longitude,
|
|
114
|
-
init_date,
|
|
115
|
-
run_length=18, # 18 hours are included in the forecast
|
|
116
|
-
lead_time_to_start=3, # forecast starts 3 hours out from the init_date
|
|
117
|
-
model='hrrr', # use NOAA HRRR
|
|
118
|
-
)
|
|
119
|
-
resource_data[
|
|
120
|
-
['ghi','dni','dhi','temp_air','wind_speed']
|
|
121
|
-
].plot(drawstyle='steps-mid')
|
|
210
|
+
### GRIB file issues
|
|
211
|
+
GRIB files can occasionally download with missing/incomplete artifacts. This has been known to cause errors like:
|
|
212
|
+
|
|
213
|
+
```
|
|
214
|
+
TypeError: objects must be an iterable containing only DataTree(s), Dataset(s), DataArray(s), and dictionaries: ['t2m']
|
|
122
215
|
```
|
|
123
216
|
|
|
124
|
-
|
|
217
|
+
If forecasts contain unexpected missing/incomplete values, try deleting cached GRIB files from your
|
|
218
|
+
Herbie cache. See the Herbie cache configuration docs:
|
|
219
|
+
https://herbie.readthedocs.io/en/stable/user_guide/configure.html. Navigating to the corresponding `~\data` folder, e.g., `C:\Users\[username]\data\ifs\20240605`, and deleting the contents may fix this.
|
|
125
220
|
|
|
126
|
-
|
|
221
|
+
If deleting the cache does not resolve the issue, try downloading the same
|
|
222
|
+
forecast from a different Herbie source, such as Azure, Google, or AWS.
|
|
127
223
|
|
|
128
|
-
Here's a wind resource forecast:
|
|
129
224
|
|
|
130
|
-
|
|
131
|
-
|
|
225
|
+
### Server issues
|
|
226
|
+
ECMWF IFS/AIFS data have known issues via AWS that can result in 503 SlowDown responses. See https://forum.ecmwf.int/t/503-slowdown-errors-for-requests-to-s3-ecmwf-forecasts/14345, https://github.com/blaylockbk/Herbie/discussions/487. Using `priority='google'` (or `'azure'`) will likely help.
|
|
132
227
|
|
|
133
|
-
|
|
134
|
-
longitude = -86.8
|
|
135
|
-
init_date = '2024-06-05 6:00' # datetime the forecast model was initialized
|
|
136
|
-
resource_data = get_wind_forecast(
|
|
137
|
-
latitude,
|
|
138
|
-
longitude,
|
|
139
|
-
init_date,
|
|
140
|
-
run_length=18, # 18 hours are included in the forecast
|
|
141
|
-
lead_time_to_start=3, # forecast starts 3 hours out from the init_date
|
|
142
|
-
model='gfs', # use NOAA GFS
|
|
143
|
-
)
|
|
144
|
-
resource_data[
|
|
145
|
-
['wind_speed_10m', 'wind_speed_80m',
|
|
146
|
-
'wind_speed_100m', 'temp_air_2m',
|
|
147
|
-
'pressure_0m']
|
|
148
|
-
].plot(secondary_y=['pressure_0m'], drawstyle='steps-mid')
|
|
149
|
-
```
|
|
150
|
-
with this output (note that pressure is on the secondary y-axis):
|
|
228
|
+
Conversely, AWS may be more reliable than Google or Azure for NOAA models.
|
|
151
229
|
|
|
152
|
-
|
|
230
|
+
### Install issues, Python version
|
|
153
231
|
|
|
154
|
-
|
|
232
|
+
If you have problems installing dependencies or get errors related to `eccodes` (e.g., `RuntimeError: Cannot find the ecCodes library`), one possibility is that you are using a version of Python that is not yet fully supported by some dependencies. Among other possibilities, ECMWF appears to sometimes get behind in publishing wheels for all platforms. If you are using one of the latest versions of Python, downgrading to an earlier version that is till in "bugfix" status, or even "security", may fix these issues.
|
|
155
233
|
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
Herbie cache. See the Herbie cache configuration docs:
|
|
159
|
-
https://herbie.readthedocs.io/en/stable/user_guide/configure.html.
|
|
234
|
+
### Firewall/certificate issues
|
|
235
|
+
If you are using `hefty` behind a firewall, you might run into SLL/certificate issues. Making sure that Python knows where to find valid local certificates might help. _Potential_ solutions include the following, but check with your IT/security resources first:
|
|
160
236
|
|
|
161
|
-
|
|
162
|
-
|
|
237
|
+
**Option 1**: Install `pip-system-certs`:
|
|
238
|
+
|
|
239
|
+
```bash
|
|
240
|
+
pip install pip_system_certs
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
**Option 2**: Use `certifi` and set environment variables at the beginning of each script/notebook:
|
|
244
|
+
|
|
245
|
+
```python
|
|
246
|
+
import certifi
|
|
247
|
+
# Set requests to use the certificates in your current environment
|
|
248
|
+
os.environ['REQUESTS_CA_BUNDLE'] = certifi.where()
|
|
249
|
+
os.environ['CURL_CA_BUNDLE'] = certifi.where()
|
|
250
|
+
```
|
|
163
251
|
|
|
164
|
-
|
|
252
|
+
### Intermittent SSL errors
|
|
253
|
+
In some instances, intermittent SSL errors (`Exception has occured : Processing failed:`... `[SSL: UNEXPECTED_EOF_WHILE_READING]` ...) _might_ be fixed by upgrading urllib3, e.g., `pip install --upgrade urllib3`.
|
|
254
|
+
|
|
255
|
+
## Data attribution
|
|
165
256
|
### ECMWF
|
|
166
257
|
If you use models from ECMWF, you may need to include attribution language with the results.
|
|
167
258
|
|
|
@@ -178,25 +269,7 @@ A possible example attribution and citation for the CAMS version of IFS:
|
|
|
178
269
|
> Contains modified Copernicus Atmosphere Monitoring Service information [2026]. Neither the European Commission nor ECMWF is responsible for any use that may be made of the Copernicus information or data it contains. © 2026 European Centre for Medium-Range Weather Forecasts (ECMWF), www.ecmwf.int. This data is published under a Creative Commons Attribution 4.0 International (CC BY 4.0). https://creativecommons.org/licenses/by/4.0/.
|
|
179
270
|
>
|
|
180
271
|
> Copernicus Atmosphere Monitoring Service (2021): CAMS global atmospheric composition forecasts. Copernicus Atmosphere Monitoring Service (CAMS) Atmosphere Data Store, DOI: 10.24381/04a0b097 (Accessed on DD-MMM-YYYY).
|
|
181
|
-
## Installation
|
|
182
|
-
|
|
183
|
-
A virtual environment is strongly recommended. You can install from PyPi with:
|
|
184
|
-
|
|
185
|
-
```
|
|
186
|
-
pip install hefty
|
|
187
|
-
```
|
|
188
|
-
|
|
189
|
-
To run the example Jupyter notebooks, you also need `jupyter`:
|
|
190
|
-
|
|
191
|
-
```
|
|
192
|
-
pip install jupyter
|
|
193
|
-
```
|
|
194
272
|
|
|
195
|
-
If you want to use ECMWF CAMS, you also need `cdsapi`:
|
|
196
|
-
|
|
197
|
-
```
|
|
198
|
-
pip install cdsapi
|
|
199
|
-
```
|
|
200
273
|
|
|
201
274
|
## References
|
|
202
275
|
This project uses several Python packages, including pvlib, an open-source solar PV modeling package [1, 2], and Herbie [3, 4], a package for accessing weather forecast data from NOAA. `pv_model.py` (with the `model_pv_power()` function used here) comes from [5] which leverages some functions from [6].
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "hefty"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.2.0"
|
|
8
8
|
description = "Some (relatively) lightweight short-term energy forecasting tools for solar, wind, and load."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.11"
|
|
@@ -19,7 +19,7 @@ classifiers = [
|
|
|
19
19
|
"Topic :: Scientific/Engineering",
|
|
20
20
|
]
|
|
21
21
|
dependencies = [
|
|
22
|
-
"herbie-data[
|
|
22
|
+
"herbie-data[scikit-learn] >= 2026.3.0",
|
|
23
23
|
"pvlib >= 0.13.1",
|
|
24
24
|
"pandas >= 3.0.0",
|
|
25
25
|
"urllib3 >= 2.6.0",
|