wxdata 0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (61) hide show
  1. wxdata-0.1/LICENSE +21 -0
  2. wxdata-0.1/PKG-INFO +317 -0
  3. wxdata-0.1/README.md +290 -0
  4. wxdata-0.1/pyproject.toml +42 -0
  5. wxdata-0.1/setup.cfg +4 -0
  6. wxdata-0.1/src/wxdata/__init__.py +137 -0
  7. wxdata-0.1/src/wxdata/calc/__init__.py +3 -0
  8. wxdata-0.1/src/wxdata/calc/derived_fields.py +44 -0
  9. wxdata-0.1/src/wxdata/calc/kinematics.py +29 -0
  10. wxdata-0.1/src/wxdata/calc/thermodynamics.py +49 -0
  11. wxdata-0.1/src/wxdata/calc/unit_conversion.py +63 -0
  12. wxdata-0.1/src/wxdata/client/__init__.py +1 -0
  13. wxdata-0.1/src/wxdata/client/client.py +251 -0
  14. wxdata-0.1/src/wxdata/ecmwf/__init__.py +7 -0
  15. wxdata-0.1/src/wxdata/ecmwf/ecmwf.py +745 -0
  16. wxdata-0.1/src/wxdata/ecmwf/file_funcs.py +110 -0
  17. wxdata-0.1/src/wxdata/ecmwf/paths.py +47 -0
  18. wxdata-0.1/src/wxdata/ecmwf/url_scanners.py +628 -0
  19. wxdata-0.1/src/wxdata/fems/__init__.py +7 -0
  20. wxdata-0.1/src/wxdata/fems/fems.py +667 -0
  21. wxdata-0.1/src/wxdata/fems/raws_sigs.py +1792 -0
  22. wxdata-0.1/src/wxdata/gefs/__init__.py +7 -0
  23. wxdata-0.1/src/wxdata/gefs/exception_messages.py +90 -0
  24. wxdata-0.1/src/wxdata/gefs/file_funcs.py +175 -0
  25. wxdata-0.1/src/wxdata/gefs/gefs.py +1171 -0
  26. wxdata-0.1/src/wxdata/gefs/paths.py +58 -0
  27. wxdata-0.1/src/wxdata/gefs/process.py +3154 -0
  28. wxdata-0.1/src/wxdata/gefs/url_scanners.py +2430 -0
  29. wxdata-0.1/src/wxdata/gfs/__init__.py +5 -0
  30. wxdata-0.1/src/wxdata/gfs/exception_messages.py +30 -0
  31. wxdata-0.1/src/wxdata/gfs/gfs.py +1112 -0
  32. wxdata-0.1/src/wxdata/gfs/paths.py +51 -0
  33. wxdata-0.1/src/wxdata/gfs/url_scanners.py +1529 -0
  34. wxdata-0.1/src/wxdata/metars/__init__.py +1 -0
  35. wxdata-0.1/src/wxdata/metars/metar_obs.py +106 -0
  36. wxdata-0.1/src/wxdata/noaa/__init__.py +1 -0
  37. wxdata-0.1/src/wxdata/noaa/nws.py +532 -0
  38. wxdata-0.1/src/wxdata/rtma/__init__.py +4 -0
  39. wxdata-0.1/src/wxdata/rtma/file_funcs.py +68 -0
  40. wxdata-0.1/src/wxdata/rtma/keys.py +56 -0
  41. wxdata-0.1/src/wxdata/rtma/process.py +291 -0
  42. wxdata-0.1/src/wxdata/rtma/rtma.py +511 -0
  43. wxdata-0.1/src/wxdata/rtma/url_scanners.py +617 -0
  44. wxdata-0.1/src/wxdata/soundings/__init__.py +1 -0
  45. wxdata-0.1/src/wxdata/soundings/wyoming_soundings.py +658 -0
  46. wxdata-0.1/src/wxdata/utils/__init__.py +26 -0
  47. wxdata-0.1/src/wxdata/utils/coords.py +121 -0
  48. wxdata-0.1/src/wxdata/utils/ecmwf_post_processing.py +1049 -0
  49. wxdata-0.1/src/wxdata/utils/file_funcs.py +204 -0
  50. wxdata-0.1/src/wxdata/utils/file_scanner.py +155 -0
  51. wxdata-0.1/src/wxdata/utils/gefs_post_processing.py +3100 -0
  52. wxdata-0.1/src/wxdata/utils/gfs_post_processing.py +2011 -0
  53. wxdata-0.1/src/wxdata/utils/nomads_gribfilter.py +187 -0
  54. wxdata-0.1/src/wxdata/utils/recycle_bin.py +74 -0
  55. wxdata-0.1/src/wxdata/utils/rtma_post_processing.py +299 -0
  56. wxdata-0.1/src/wxdata/utils/tools.py +212 -0
  57. wxdata-0.1/src/wxdata.egg-info/PKG-INFO +317 -0
  58. wxdata-0.1/src/wxdata.egg-info/SOURCES.txt +59 -0
  59. wxdata-0.1/src/wxdata.egg-info/dependency_links.txt +1 -0
  60. wxdata-0.1/src/wxdata.egg-info/requires.txt +10 -0
  61. wxdata-0.1/src/wxdata.egg-info/top_level.txt +1 -0
wxdata-0.1/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2025 Eric J. Drewitz
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
wxdata-0.1/PKG-INFO ADDED
@@ -0,0 +1,317 @@
1
+ Metadata-Version: 2.4
2
+ Name: wxdata
3
+ Version: 0.1
4
+ Summary: An open source library for downloading and pre/post-processing various types of weather data. Friendly for VPN/PROXY connections.
5
+ Author: Eric J. Drewitz
6
+ Project-URL: Documentation, https://github.com/edrewitz/wxdata/blob/main/Documentation/Landing%20Page.md
7
+ Project-URL: Repository, https://github.com/edrewitz/wxdata/tree/main
8
+ Keywords: meteorology,atmospheric sciences
9
+ Classifier: Programming Language :: Python
10
+ Classifier: Topic :: Scientific/Engineering
11
+ Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
12
+ Classifier: Operating System :: OS Independent
13
+ Requires-Python: >=3.10
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: metpy>=1.5.1
17
+ Requires-Dist: numpy>=1.24
18
+ Requires-Dist: pandas>=2.2
19
+ Requires-Dist: xarray>=2023.1.0
20
+ Requires-Dist: netcdf4>=1.7.1
21
+ Requires-Dist: cartopy>=0.21.0
22
+ Requires-Dist: beautifulsoup4>=4.13.4
23
+ Requires-Dist: requests>=2.32.4
24
+ Requires-Dist: cfgrib>=0.9.10.4
25
+ Requires-Dist: dask>=2025.5.1
26
+ Dynamic: license-file
27
+
28
+ # WxData
29
+
30
+ ![weather icon](https://github.com/edrewitz/wxdata/blob/main/icons/weather%20icon.jpg) ![python icon](https://github.com/edrewitz/wxdata/blob/main/icons/python%20logo.png)
31
+
32
+ **(C) Eric J. Drewitz 2025**
33
+
34
+ An open-source package that helps meteorologists and weather enthusiats download, pre-process and post-process various types of weather data.
35
+
36
+ This package only retrieves open-source weather data (i.e. nothing behind a paywall or a login).
37
+
38
+ This package provides the following extra functionality compared to existing packages for downloading weather data:
39
+
40
+ 1) Friendly for users working on VPN/PROXY connections.
41
+ - Users input their PROXY IP address as a dictionary and pass it into the function to avoid SSL errors
42
+ - If the user is on a VPN/PROXY Connection the following is needed:
43
+
44
+ proxies=None ---> proxies={
45
+ 'http':'http://url',
46
+ 'https':'https://url'
47
+ }
48
+
49
+ [e.g. get_observed_sounding_data('nkx', proxies=proxies)]
50
+
51
+ For more information on configuring proxies: https://requests.readthedocs.io/en/latest/user/advanced/#proxies
52
+
53
+ - Some data access functions work on VPN/PROXY connections without needing to define VPN/PROXY settings:
54
+ - METARs
55
+ - NOAA Storm Prediction Center/National Weather Service Products
56
+ - FEMS
57
+
58
+ - Data access methods that users need to define VPN/PROXY IP addresses if using a VPN/PROXY connection:
59
+ - Various Forecast Models
60
+ - Observed Sounding Data from University of Wyoming
61
+ - Real-Time Mesoscale Analysis
62
+
63
+ 1) Converts GRIB variable keys into variable keys that are in plain language.
64
+ - (e.g. 'r2' ---> '2m_relative_humidity')
65
+
66
+ 2) Has a scanner that checks if the data files on your PC are up to date with those on the data server.
67
+ - This is a safeguard to protect newer developers from getting temporary IP address bans from the various data servers.
68
+ - Improves performance by preventing the potential of repetative downloading the same dataset.
69
+
70
+ 3) Preserves system memory via the following methods:
71
+ - Clears out old data files before each new data download.
72
+ - Optional setting `clear_recycle_bin=True` in all functions.
73
+ - When `clear_recycle_bin=True` the computer's recycle/trash bin is cleared with each run of the script using any WxData function.
74
+ - If a user wishes to not clear out their recycle bin `set clear_recycle_bin=False`.
75
+
76
+ ## WxData Examples
77
+
78
+ *Regular Users*
79
+ 1) [Downloading METAR Data](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/metars.ipynb)
80
+ 2) [Downloading Observed Sounding Data](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/soundings.ipynb)
81
+ 3) [Downloading the first 72 hours of the ECMWF IFS and ECMWF AIFS](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/ecmwf.ipynb)
82
+ 4) [Downloading the GEFS members p01 and p02 for only Temperature](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/gefs.ipynb)
83
+ 5) [Downloading the Real-Time Mesoscale Analysis (RTMA)](https://github.com/edrewitz/WxData/blob/main/Documentation/RTMA%20Post%20Processing.md)
84
+ 6) [Downloading the RAWS SIG Group Fuels Data and the NFDRS Forecast for the RAWS SIG Groups for the South Ops Geographic Area Coordination Center](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/fems.ipynb)
85
+ 7) [Downloading the SPC Convective Outlook for CONUS](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/spc.ipynb)
86
+ 8) [Downloading NWS Maximum Temperature Forecast for Hawaii](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/nws_hi.ipynb)
87
+ 9) [Downloading the GFS0P25 then performing pixel and line queries on the data](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/GFS.ipynb)
88
+
89
+ *Advanced Users*
90
+ 1) [Using the `client` module to download the latest HadCRUT5 Analysis netCDF file and open this dataset in xarray](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/hadcrut5.ipynb)
91
+
92
+ ## WxData Documentation
93
+
94
+ ***Global Forecast System (GFS)***
95
+ 1. [GFS0P25](https://github.com/edrewitz/WxData/blob/main/Documentation/GFS0P25.md)
96
+ 2. [GFS0P25 SECONDARY PARAMETERS](https://github.com/edrewitz/WxData/blob/main/Documentation/GFS0P25%20Secondary%20Parameters.md)
97
+ 3. [GFS0P50](https://github.com/edrewitz/WxData/blob/main/Documentation/GEFS0P50.md)
98
+
99
+ ***Global Ensemble Forecast System (GEFS)***
100
+ 1. [GEFS0P50](https://github.com/edrewitz/wxdata/blob/main/Documentation/GEFS0P50.md#global-ensemble-forecast-system-050-x-050-degree-gefs0p50)
101
+ 2. [GEFS0P50 SECONDARY PARAMETERS](https://github.com/edrewitz/wxdata/blob/main/Documentation/GEFS0P50%20Secondary%20Parameters.md#global-ensemble-forecast-system-050-x-050-degree-secondary-parameters-gefs0p50-secondary-parameters)
102
+ 3. [GEFS0P25](https://github.com/edrewitz/wxdata/blob/main/Documentation/GEFS0P25.md#global-ensemble-forecast-system-025-x-025-degree-gefs0p25)
103
+
104
+ ***ECMWF Open Data***
105
+ 1. [ECMWF IFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF_IFS.md)
106
+ 2. [ECMWF AIFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF_AIFS.md)
107
+ 3. [ECMWF High Resolution IFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF_High_Res_IFS.md)
108
+ 4. [ECMWF IFS Wave](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF_IFS_Wave.md)
109
+
110
+ ***Real-Time Mesoscale Analysis (RTMA)***
111
+ 1. [RTMA](https://github.com/edrewitz/wxdata/blob/main/Documentation/rtma.md#real-time-mesoscale-analysis-rtma)
112
+ 2. [RTMA Comparison](https://github.com/edrewitz/wxdata/blob/main/Documentation/rtma%20comparison.md#real-time-mesoscale-analysis-rtma-24-hour-comparison)
113
+
114
+ ***NOAA Storm Prediction Center Outlooks And National Weather Service Forecasts***
115
+ 1. [Get NDFD Grids](https://github.com/edrewitz/wxdata/blob/main/Documentation/noaa.md#noaa-get-storm-prediction-center-outlooks-and-national-weather-service-forecasts-ndfd-grids)
116
+
117
+ ***METAR Observations***
118
+ 1. [METAR Observations](https://github.com/edrewitz/wxdata/blob/main/Documentation/metars.md#metar-observations)
119
+
120
+ ***FEMS RAWS Network***
121
+ 1. [Get Single Station RAWS Data](https://github.com/edrewitz/wxdata/blob/main/Documentation/single_raws.md#fems-get-single-raws-station-data)
122
+ 2. [Get Each SIG of RAWS Data for a Geographic Area Coordination Center](https://github.com/edrewitz/wxdata/blob/main/Documentation/raws%20sig.md#fems-get-raws-sig-data-for-a-geographic-area-coordination-center-region)
123
+ 3. [Get NFDRS Forecast Data for Each SIG for a Geographic Area Coordination Center](https://github.com/edrewitz/wxdata/blob/main/Documentation/nfdrs%20forecast.md#fems-get-nfdrs-forecast-data-for-a-raws-sig-for-a-geographic-area-coordination-center-region)
124
+
125
+ ***Observed Atmospheric Soundings***
126
+ 1. [University Of Wyoming Soundings](https://github.com/edrewitz/wxdata/blob/main/Documentation/wyoming_soundings.md)
127
+
128
+ ***GFS Post-Processing***
129
+ 1. [Primary GFS Post-Processing](https://github.com/edrewitz/WxData/blob/main/Documentation/Primary%20GFS%20Post%20Processing.md)
130
+ 2. [Secondary GFS Post-Processing](https://github.com/edrewitz/WxData/blob/main/Documentation/Secondary%20GFS%20Post%20Processing.md)
131
+
132
+ ***GEFS Post-Processing***
133
+ 1. [Primary GEFS Post-Processing](https://github.com/edrewitz/WxData/blob/main/Documentation/Primary%20GEFS%20Post-Processing.md)
134
+ 2. [Secondary GEFS Post-Processing](https://github.com/edrewitz/WxData/blob/main/Documentation/Secondary%20GEFS%20Post%20Processing.md)
135
+
136
+ ***ECMWF Post-Processing***
137
+ 1. [ECMWF IFS and ECMWF High Resolution IFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF%20IFS%20Post%20Processing.md)
138
+ 2. [ECMWF AIFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF%20AIFS%20Post%20Processing.md)
139
+ 3. [ECMWF IFS Wave](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF%20IFS%20Wave%20Post%20Processing.md)
140
+
141
+ ***Real-Time Mesoscale Analysis Post-Processing***
142
+ 1.[RTMA](https://github.com/edrewitz/WxData/blob/main/Documentation/RTMA%20Post%20Processing.md)
143
+
144
+ ***Cyclic Points For Hemispheric Plots***
145
+ 1. [Cyclic Points](https://github.com/edrewitz/wxdata/blob/main/Documentation/cyclic_point.md#using-wxdata-to-add-cyclic-points-for-hemispheric-plots)
146
+
147
+ ***Shifting Longitude From 0 to 360 --> -180 to 180***
148
+ 1. [shift_longitude](https://github.com/edrewitz/WxData/blob/main/Documentation/shift_longitude.md)
149
+
150
+ ***Pixel Query***
151
+ 1. [pixel_query](https://github.com/edrewitz/WxData/blob/main/Documentation/pixel_query.md)
152
+
153
+ ***Line Query***
154
+ 1. [line_query](https://github.com/edrewitz/WxData/blob/main/Documentation/line_query.md)
155
+
156
+ ## Importing Functions from WxData
157
+
158
+ """
159
+ This section of functions are for users who want full wxdata functionality.
160
+
161
+ These functions do the following:
162
+
163
+ 1) Scan for the latest available data.
164
+ - If the data on your local machine is not up to date, new data will download automatically.
165
+ - If the data on your local machine is up to date, new data download is bypassed.
166
+ - This is a safeguard that prevents excessive requests on the data servers.
167
+
168
+ 2) Builds the wxdata directory to store the weather data files.
169
+ - Scans for the directory branch and builds the branch if it does not exist.
170
+
171
+ 3) Downloads the data.
172
+ - Users can define their VPN/PROXY IP Address as a (dict) in their script and pass their
173
+ VPN/PROXY IP address into the function to avoid SSL Certificate errors when requesting data.
174
+ - Algorithm allows for up to 5 retries with a 30 second break between each retry to resolve connection
175
+ interruptions while not overburdening the data servers.
176
+
177
+ 4) Pre-processes the data via filename formatting and correctly filing in the wxdata directory.
178
+
179
+ 5) Post-processing by doing the following tasks:
180
+ - Remapping GRIB2 variable keys into plain language variable keys.
181
+ - Fixing dataset build errors and grouping all variables together.
182
+ - Transforms longitude from 0 to 360 degrees into -180 to 180 degrees.
183
+ - Subsetting the data to the latitude/longitude boundaries specified by the user.
184
+ - Converting temperature from kelvin to units the user wants (default is Celsius).
185
+ - Returning a post-processed xarray.array to the user.
186
+
187
+ 6) Preserves system memory by doing the following:
188
+ - Deleting old data files before each new download.
189
+ - When clear_recycle_bin=True, the user's recycle bin is also cleared.
190
+ """
191
+
192
+ # Global Forecast System (GFS)
193
+ from wxdata.gfs.gfs import(
194
+ gfs_0p25,
195
+ gfs_0p25_secondary_parameters,
196
+ gfs_0p50
197
+ )
198
+
199
+
200
+ # Global Ensemble Forecast System (GEFS)
201
+ from wxdata.gefs.gefs import(
202
+
203
+ gefs_0p50,
204
+ gefs_0p50_secondary_parameters,
205
+ gefs_0p25
206
+ )
207
+
208
+ # European Centre for Medium-Range Weather Forecasts (ECMWF)
209
+ from wxdata.ecmwf.ecmwf import(
210
+ ecmwf_ifs,
211
+ ecmwf_aifs,
212
+ ecmwf_ifs_high_res,
213
+ ecmwf_ifs_wave
214
+ )
215
+
216
+ # FEMS RAWS Network
217
+ from wxdata.fems.fems import(
218
+ get_single_station_data,
219
+ get_raws_sig_data,
220
+ get_nfdrs_forecast_data
221
+ )
222
+
223
+ # Real-Time Mesoscale Analysis (RTMA)
224
+ from wxdata.rtma.rtma import(
225
+ rtma,
226
+ rtma_comparison
227
+ )
228
+
229
+ # NOAA
230
+ # Storm Prediction Center Outlooks
231
+ # National Weather Service Forecasts
232
+ from wxdata.noaa.nws import get_ndfd_grids
233
+
234
+ # Observed Upper-Air Soundings
235
+ # (University of Wyoming Database)
236
+ from wxdata.soundings.wyoming_soundings import get_observed_sounding_data
237
+
238
+ # METAR Observational Data (From NOAA)
239
+ from wxdata.metars.metar_obs import download_metar_data
240
+
241
+
242
+ """
243
+ This section hosts the utility functions accessable to the user.
244
+
245
+ These functions provide helpful utilities when analyzing weather data.
246
+
247
+ Utility functions are geared towards the following types of users:
248
+
249
+ 1) Users who want to use their own scripts to download the data however, they
250
+ would like to use the wxdata post-processing capabilities.
251
+
252
+ 2) Users who want to make hemispheric graphics or any graphics where cyclic points
253
+ resolve missing data along the prime meridian or international dateline.
254
+ """
255
+ # Global Forecast System (GFS)
256
+ import wxdata.utils.gfs_post_processing as gfs_post_processing
257
+
258
+ # Global Ensemble Forecast System (GEFS)
259
+ import wxdata.utils.gefs_post_processing as gefs_post_processing
260
+
261
+ # European Centre for Medium-Range Weather Forecasts (ECMWF)
262
+ import wxdata.utils.ecmwf_post_processing as ecmwf_post_processing
263
+
264
+ # Real-Time Mesoscale Analysis (RTMA)
265
+ from wxdata.utils.rtma_post_processing import process_rtma_data
266
+
267
+
268
+ # WxData function using cartopy to make cyclic points
269
+ # This is for users who wish to make graphics that cross the -180/180 degree longitude line
270
+ # This is commonly used for Hemispheric graphics
271
+ # Function that converts the longitude dimension in an xarray.array
272
+ # From 0 to 360 to -180 to 180
273
+ from wxdata.utils.coords import(
274
+ cyclic_point,
275
+ shift_longitude
276
+ )
277
+
278
+ # Functions to pixel query and query pixels along a line between points A and B
279
+ from wxdata.utils.tools import(
280
+ pixel_query,
281
+ line_query
282
+ )
283
+
284
+ # This is the wxdata HTTPS Client with full VPN/PROXY Support
285
+ import wxdata.client.client as client
286
+
287
+
288
+
289
+
290
+ ## Citations
291
+
292
+ **MetPy**: May, R. M., Goebbert, K. H., Thielen, J. E., Leeman, J. R., Camron, M. D., Bruick, Z.,
293
+ Bruning, E. C., Manser, R. P., Arms, S. C., and Marsh, P. T., 2022: MetPy: A
294
+ Meteorological Python Library for Data Analysis and Visualization. Bull. Amer. Meteor.
295
+ Soc., 103, E2273-E2284, https://doi.org/10.1175/BAMS-D-21-0125.1.
296
+
297
+ **xarray**: Hoyer, S., Hamman, J. (In revision). Xarray: N-D labeled arrays and datasets in Python. Journal of Open Research Software.
298
+
299
+ **cartopy**: Phil Elson, Elliott Sales de Andrade, Greg Lucas, Ryan May, Richard Hattersley, Ed Campbell, Andrew Dawson, Bill Little, Stephane Raynaud, scmc72, Alan D. Snow, Ruth Comer, Kevin Donkers, Byron Blay, Peter Killick, Nat Wilson, Patrick Peglar, lgolston, lbdreyer, … Chris Havlin. (2023). SciTools/cartopy: v0.22.0 (v0.22.0). Zenodo. https://doi.org/10.5281/zenodo.8216315
300
+
301
+ **NumPy**: Harris, C.R., Millman, K.J., van der Walt, S.J. et al. Array programming with NumPy. Nature 585, 357–362 (2020). DOI: 10.1038/s41586-020-2649-2. (Publisher link).
302
+
303
+ **Pandas**: Pandas: McKinney, W., & others. (2010). Data structures for statistical computing in python. In Proceedings of the 9th Python in Science Conference (Vol. 445, pp. 51–56).
304
+
305
+ **dask**: Dask Development Team (2016). Dask: Library for dynamic task scheduling. URL http://dask.pydata.org
306
+
307
+ **cfgrib**: Author: ECMWF, Year: (2025), Title: cfgrib: A Python interface to map GRIB files to xarray, Source: https://github.com/ecmwf/cfgrib
308
+
309
+ ## Data Sources
310
+
311
+ 1) [National Oceanic and Atmospheric Administration/National Center for Environmental Prediction](https://nomads.ncep.noaa.gov/)
312
+ 2) [European Centre for Medium-Range Weather Forecasts](https://data.ecmwf.int/forecasts/)
313
+ 3) [University of Wyoming](http://www.weather.uwyo.edu/upperair/sounding.shtml)
314
+ 4) [National Oceanic and Atmospheric Administration/National Weather Service](https://tgftp.nws.noaa.gov/)
315
+ 5) [National Oceanic and Atmospheric Administration/Aviation Weather Center](https://aviationweather.gov/)
316
+
317
+
wxdata-0.1/README.md ADDED
@@ -0,0 +1,290 @@
1
+ # WxData
2
+
3
+ ![weather icon](https://github.com/edrewitz/wxdata/blob/main/icons/weather%20icon.jpg) ![python icon](https://github.com/edrewitz/wxdata/blob/main/icons/python%20logo.png)
4
+
5
+ **(C) Eric J. Drewitz 2025**
6
+
7
+ An open-source package that helps meteorologists and weather enthusiats download, pre-process and post-process various types of weather data.
8
+
9
+ This package only retrieves open-source weather data (i.e. nothing behind a paywall or a login).
10
+
11
+ This package provides the following extra functionality compared to existing packages for downloading weather data:
12
+
13
+ 1) Friendly for users working on VPN/PROXY connections.
14
+ - Users input their PROXY IP address as a dictionary and pass it into the function to avoid SSL errors
15
+ - If the user is on a VPN/PROXY Connection the following is needed:
16
+
17
+ proxies=None ---> proxies={
18
+ 'http':'http://url',
19
+ 'https':'https://url'
20
+ }
21
+
22
+ [e.g. get_observed_sounding_data('nkx', proxies=proxies)]
23
+
24
+ For more information on configuring proxies: https://requests.readthedocs.io/en/latest/user/advanced/#proxies
25
+
26
+ - Some data access functions work on VPN/PROXY connections without needing to define VPN/PROXY settings:
27
+ - METARs
28
+ - NOAA Storm Prediction Center/National Weather Service Products
29
+ - FEMS
30
+
31
+ - Data access methods that users need to define VPN/PROXY IP addresses if using a VPN/PROXY connection:
32
+ - Various Forecast Models
33
+ - Observed Sounding Data from University of Wyoming
34
+ - Real-Time Mesoscale Analysis
35
+
36
+ 1) Converts GRIB variable keys into variable keys that are in plain language.
37
+ - (e.g. 'r2' ---> '2m_relative_humidity')
38
+
39
+ 2) Has a scanner that checks if the data files on your PC are up to date with those on the data server.
40
+ - This is a safeguard to protect newer developers from getting temporary IP address bans from the various data servers.
41
+ - Improves performance by preventing the potential of repetative downloading the same dataset.
42
+
43
+ 3) Preserves system memory via the following methods:
44
+ - Clears out old data files before each new data download.
45
+ - Optional setting `clear_recycle_bin=True` in all functions.
46
+ - When `clear_recycle_bin=True` the computer's recycle/trash bin is cleared with each run of the script using any WxData function.
47
+ - If a user wishes to not clear out their recycle bin `set clear_recycle_bin=False`.
48
+
49
+ ## WxData Examples
50
+
51
+ *Regular Users*
52
+ 1) [Downloading METAR Data](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/metars.ipynb)
53
+ 2) [Downloading Observed Sounding Data](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/soundings.ipynb)
54
+ 3) [Downloading the first 72 hours of the ECMWF IFS and ECMWF AIFS](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/ecmwf.ipynb)
55
+ 4) [Downloading the GEFS members p01 and p02 for only Temperature](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/gefs.ipynb)
56
+ 5) [Downloading the Real-Time Mesoscale Analysis (RTMA)](https://github.com/edrewitz/WxData/blob/main/Documentation/RTMA%20Post%20Processing.md)
57
+ 6) [Downloading the RAWS SIG Group Fuels Data and the NFDRS Forecast for the RAWS SIG Groups for the South Ops Geographic Area Coordination Center](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/fems.ipynb)
58
+ 7) [Downloading the SPC Convective Outlook for CONUS](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/spc.ipynb)
59
+ 8) [Downloading NWS Maximum Temperature Forecast for Hawaii](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/nws_hi.ipynb)
60
+ 9) [Downloading the GFS0P25 then performing pixel and line queries on the data](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/GFS.ipynb)
61
+
62
+ *Advanced Users*
63
+ 1) [Using the `client` module to download the latest HadCRUT5 Analysis netCDF file and open this dataset in xarray](https://github.com/edrewitz/WxData-JupyterLab-Examples/blob/main/hadcrut5.ipynb)
64
+
65
+ ## WxData Documentation
66
+
67
+ ***Global Forecast System (GFS)***
68
+ 1. [GFS0P25](https://github.com/edrewitz/WxData/blob/main/Documentation/GFS0P25.md)
69
+ 2. [GFS0P25 SECONDARY PARAMETERS](https://github.com/edrewitz/WxData/blob/main/Documentation/GFS0P25%20Secondary%20Parameters.md)
70
+ 3. [GFS0P50](https://github.com/edrewitz/WxData/blob/main/Documentation/GEFS0P50.md)
71
+
72
+ ***Global Ensemble Forecast System (GEFS)***
73
+ 1. [GEFS0P50](https://github.com/edrewitz/wxdata/blob/main/Documentation/GEFS0P50.md#global-ensemble-forecast-system-050-x-050-degree-gefs0p50)
74
+ 2. [GEFS0P50 SECONDARY PARAMETERS](https://github.com/edrewitz/wxdata/blob/main/Documentation/GEFS0P50%20Secondary%20Parameters.md#global-ensemble-forecast-system-050-x-050-degree-secondary-parameters-gefs0p50-secondary-parameters)
75
+ 3. [GEFS0P25](https://github.com/edrewitz/wxdata/blob/main/Documentation/GEFS0P25.md#global-ensemble-forecast-system-025-x-025-degree-gefs0p25)
76
+
77
+ ***ECMWF Open Data***
78
+ 1. [ECMWF IFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF_IFS.md)
79
+ 2. [ECMWF AIFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF_AIFS.md)
80
+ 3. [ECMWF High Resolution IFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF_High_Res_IFS.md)
81
+ 4. [ECMWF IFS Wave](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF_IFS_Wave.md)
82
+
83
+ ***Real-Time Mesoscale Analysis (RTMA)***
84
+ 1. [RTMA](https://github.com/edrewitz/wxdata/blob/main/Documentation/rtma.md#real-time-mesoscale-analysis-rtma)
85
+ 2. [RTMA Comparison](https://github.com/edrewitz/wxdata/blob/main/Documentation/rtma%20comparison.md#real-time-mesoscale-analysis-rtma-24-hour-comparison)
86
+
87
+ ***NOAA Storm Prediction Center Outlooks And National Weather Service Forecasts***
88
+ 1. [Get NDFD Grids](https://github.com/edrewitz/wxdata/blob/main/Documentation/noaa.md#noaa-get-storm-prediction-center-outlooks-and-national-weather-service-forecasts-ndfd-grids)
89
+
90
+ ***METAR Observations***
91
+ 1. [METAR Observations](https://github.com/edrewitz/wxdata/blob/main/Documentation/metars.md#metar-observations)
92
+
93
+ ***FEMS RAWS Network***
94
+ 1. [Get Single Station RAWS Data](https://github.com/edrewitz/wxdata/blob/main/Documentation/single_raws.md#fems-get-single-raws-station-data)
95
+ 2. [Get Each SIG of RAWS Data for a Geographic Area Coordination Center](https://github.com/edrewitz/wxdata/blob/main/Documentation/raws%20sig.md#fems-get-raws-sig-data-for-a-geographic-area-coordination-center-region)
96
+ 3. [Get NFDRS Forecast Data for Each SIG for a Geographic Area Coordination Center](https://github.com/edrewitz/wxdata/blob/main/Documentation/nfdrs%20forecast.md#fems-get-nfdrs-forecast-data-for-a-raws-sig-for-a-geographic-area-coordination-center-region)
97
+
98
+ ***Observed Atmospheric Soundings***
99
+ 1. [University Of Wyoming Soundings](https://github.com/edrewitz/wxdata/blob/main/Documentation/wyoming_soundings.md)
100
+
101
+ ***GFS Post-Processing***
102
+ 1. [Primary GFS Post-Processing](https://github.com/edrewitz/WxData/blob/main/Documentation/Primary%20GFS%20Post%20Processing.md)
103
+ 2. [Secondary GFS Post-Processing](https://github.com/edrewitz/WxData/blob/main/Documentation/Secondary%20GFS%20Post%20Processing.md)
104
+
105
+ ***GEFS Post-Processing***
106
+ 1. [Primary GEFS Post-Processing](https://github.com/edrewitz/WxData/blob/main/Documentation/Primary%20GEFS%20Post-Processing.md)
107
+ 2. [Secondary GEFS Post-Processing](https://github.com/edrewitz/WxData/blob/main/Documentation/Secondary%20GEFS%20Post%20Processing.md)
108
+
109
+ ***ECMWF Post-Processing***
110
+ 1. [ECMWF IFS and ECMWF High Resolution IFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF%20IFS%20Post%20Processing.md)
111
+ 2. [ECMWF AIFS](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF%20AIFS%20Post%20Processing.md)
112
+ 3. [ECMWF IFS Wave](https://github.com/edrewitz/WxData/blob/main/Documentation/ECMWF%20IFS%20Wave%20Post%20Processing.md)
113
+
114
+ ***Real-Time Mesoscale Analysis Post-Processing***
115
+ 1.[RTMA](https://github.com/edrewitz/WxData/blob/main/Documentation/RTMA%20Post%20Processing.md)
116
+
117
+ ***Cyclic Points For Hemispheric Plots***
118
+ 1. [Cyclic Points](https://github.com/edrewitz/wxdata/blob/main/Documentation/cyclic_point.md#using-wxdata-to-add-cyclic-points-for-hemispheric-plots)
119
+
120
+ ***Shifting Longitude From 0 to 360 --> -180 to 180***
121
+ 1. [shift_longitude](https://github.com/edrewitz/WxData/blob/main/Documentation/shift_longitude.md)
122
+
123
+ ***Pixel Query***
124
+ 1. [pixel_query](https://github.com/edrewitz/WxData/blob/main/Documentation/pixel_query.md)
125
+
126
+ ***Line Query***
127
+ 1. [line_query](https://github.com/edrewitz/WxData/blob/main/Documentation/line_query.md)
128
+
129
+ ## Importing Functions from WxData
130
+
131
+ """
132
+ This section of functions are for users who want full wxdata functionality.
133
+
134
+ These functions do the following:
135
+
136
+ 1) Scan for the latest available data.
137
+ - If the data on your local machine is not up to date, new data will download automatically.
138
+ - If the data on your local machine is up to date, new data download is bypassed.
139
+ - This is a safeguard that prevents excessive requests on the data servers.
140
+
141
+ 2) Builds the wxdata directory to store the weather data files.
142
+ - Scans for the directory branch and builds the branch if it does not exist.
143
+
144
+ 3) Downloads the data.
145
+ - Users can define their VPN/PROXY IP Address as a (dict) in their script and pass their
146
+ VPN/PROXY IP address into the function to avoid SSL Certificate errors when requesting data.
147
+ - Algorithm allows for up to 5 retries with a 30 second break between each retry to resolve connection
148
+ interruptions while not overburdening the data servers.
149
+
150
+ 4) Pre-processes the data via filename formatting and correctly filing in the wxdata directory.
151
+
152
+ 5) Post-processing by doing the following tasks:
153
+ - Remapping GRIB2 variable keys into plain language variable keys.
154
+ - Fixing dataset build errors and grouping all variables together.
155
+ - Transforms longitude from 0 to 360 degrees into -180 to 180 degrees.
156
+ - Subsetting the data to the latitude/longitude boundaries specified by the user.
157
+ - Converting temperature from kelvin to units the user wants (default is Celsius).
158
+ - Returning a post-processed xarray.array to the user.
159
+
160
+ 6) Preserves system memory by doing the following:
161
+ - Deleting old data files before each new download.
162
+ - When clear_recycle_bin=True, the user's recycle bin is also cleared.
163
+ """
164
+
165
+ # Global Forecast System (GFS)
166
+ from wxdata.gfs.gfs import(
167
+ gfs_0p25,
168
+ gfs_0p25_secondary_parameters,
169
+ gfs_0p50
170
+ )
171
+
172
+
173
+ # Global Ensemble Forecast System (GEFS)
174
+ from wxdata.gefs.gefs import(
175
+
176
+ gefs_0p50,
177
+ gefs_0p50_secondary_parameters,
178
+ gefs_0p25
179
+ )
180
+
181
+ # European Centre for Medium-Range Weather Forecasts (ECMWF)
182
+ from wxdata.ecmwf.ecmwf import(
183
+ ecmwf_ifs,
184
+ ecmwf_aifs,
185
+ ecmwf_ifs_high_res,
186
+ ecmwf_ifs_wave
187
+ )
188
+
189
+ # FEMS RAWS Network
190
+ from wxdata.fems.fems import(
191
+ get_single_station_data,
192
+ get_raws_sig_data,
193
+ get_nfdrs_forecast_data
194
+ )
195
+
196
+ # Real-Time Mesoscale Analysis (RTMA)
197
+ from wxdata.rtma.rtma import(
198
+ rtma,
199
+ rtma_comparison
200
+ )
201
+
202
+ # NOAA
203
+ # Storm Prediction Center Outlooks
204
+ # National Weather Service Forecasts
205
+ from wxdata.noaa.nws import get_ndfd_grids
206
+
207
+ # Observed Upper-Air Soundings
208
+ # (University of Wyoming Database)
209
+ from wxdata.soundings.wyoming_soundings import get_observed_sounding_data
210
+
211
+ # METAR Observational Data (From NOAA)
212
+ from wxdata.metars.metar_obs import download_metar_data
213
+
214
+
215
+ """
216
+ This section hosts the utility functions accessable to the user.
217
+
218
+ These functions provide helpful utilities when analyzing weather data.
219
+
220
+ Utility functions are geared towards the following types of users:
221
+
222
+ 1) Users who want to use their own scripts to download the data however, they
223
+ would like to use the wxdata post-processing capabilities.
224
+
225
+ 2) Users who want to make hemispheric graphics or any graphics where cyclic points
226
+ resolve missing data along the prime meridian or international dateline.
227
+ """
228
+ # Global Forecast System (GFS)
229
+ import wxdata.utils.gfs_post_processing as gfs_post_processing
230
+
231
+ # Global Ensemble Forecast System (GEFS)
232
+ import wxdata.utils.gefs_post_processing as gefs_post_processing
233
+
234
+ # European Centre for Medium-Range Weather Forecasts (ECMWF)
235
+ import wxdata.utils.ecmwf_post_processing as ecmwf_post_processing
236
+
237
+ # Real-Time Mesoscale Analysis (RTMA)
238
+ from wxdata.utils.rtma_post_processing import process_rtma_data
239
+
240
+
241
+ # WxData function using cartopy to make cyclic points
242
+ # This is for users who wish to make graphics that cross the -180/180 degree longitude line
243
+ # This is commonly used for Hemispheric graphics
244
+ # Function that converts the longitude dimension in an xarray.array
245
+ # From 0 to 360 to -180 to 180
246
+ from wxdata.utils.coords import(
247
+ cyclic_point,
248
+ shift_longitude
249
+ )
250
+
251
+ # Functions to pixel query and query pixels along a line between points A and B
252
+ from wxdata.utils.tools import(
253
+ pixel_query,
254
+ line_query
255
+ )
256
+
257
+ # This is the wxdata HTTPS Client with full VPN/PROXY Support
258
+ import wxdata.client.client as client
259
+
260
+
261
+
262
+
263
+ ## Citations
264
+
265
+ **MetPy**: May, R. M., Goebbert, K. H., Thielen, J. E., Leeman, J. R., Camron, M. D., Bruick, Z.,
266
+ Bruning, E. C., Manser, R. P., Arms, S. C., and Marsh, P. T., 2022: MetPy: A
267
+ Meteorological Python Library for Data Analysis and Visualization. Bull. Amer. Meteor.
268
+ Soc., 103, E2273-E2284, https://doi.org/10.1175/BAMS-D-21-0125.1.
269
+
270
+ **xarray**: Hoyer, S., Hamman, J. (In revision). Xarray: N-D labeled arrays and datasets in Python. Journal of Open Research Software.
271
+
272
+ **cartopy**: Phil Elson, Elliott Sales de Andrade, Greg Lucas, Ryan May, Richard Hattersley, Ed Campbell, Andrew Dawson, Bill Little, Stephane Raynaud, scmc72, Alan D. Snow, Ruth Comer, Kevin Donkers, Byron Blay, Peter Killick, Nat Wilson, Patrick Peglar, lgolston, lbdreyer, … Chris Havlin. (2023). SciTools/cartopy: v0.22.0 (v0.22.0). Zenodo. https://doi.org/10.5281/zenodo.8216315
273
+
274
+ **NumPy**: Harris, C.R., Millman, K.J., van der Walt, S.J. et al. Array programming with NumPy. Nature 585, 357–362 (2020). DOI: 10.1038/s41586-020-2649-2. (Publisher link).
275
+
276
+ **Pandas**: Pandas: McKinney, W., & others. (2010). Data structures for statistical computing in python. In Proceedings of the 9th Python in Science Conference (Vol. 445, pp. 51–56).
277
+
278
+ **dask**: Dask Development Team (2016). Dask: Library for dynamic task scheduling. URL http://dask.pydata.org
279
+
280
+ **cfgrib**: Author: ECMWF, Year: (2025), Title: cfgrib: A Python interface to map GRIB files to xarray, Source: https://github.com/ecmwf/cfgrib
281
+
282
+ ## Data Sources
283
+
284
+ 1) [National Oceanic and Atmospheric Administration/National Center for Environmental Prediction](https://nomads.ncep.noaa.gov/)
285
+ 2) [European Centre for Medium-Range Weather Forecasts](https://data.ecmwf.int/forecasts/)
286
+ 3) [University of Wyoming](http://www.weather.uwyo.edu/upperair/sounding.shtml)
287
+ 4) [National Oceanic and Atmospheric Administration/National Weather Service](https://tgftp.nws.noaa.gov/)
288
+ 5) [National Oceanic and Atmospheric Administration/Aviation Weather Center](https://aviationweather.gov/)
289
+
290
+