py-ewr 2.3.7__py3-none-any.whl → 2.3.8__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
- Metadata-Version: 2.2
1
+ Metadata-Version: 2.4
2
2
  Name: py_ewr
3
- Version: 2.3.7
3
+ Version: 2.3.8
4
4
  Summary: Environmental Water Requirement calculator
5
5
  Home-page: https://github.com/MDBAuth/EWR_tool
6
6
  Author: Martin Job
@@ -34,6 +34,7 @@ Dynamic: classifier
34
34
  Dynamic: description
35
35
  Dynamic: description-content-type
36
36
  Dynamic: home-page
37
+ Dynamic: license-file
37
38
  Dynamic: project-url
38
39
  Dynamic: requires-dist
39
40
  Dynamic: summary
@@ -43,23 +44,29 @@ Dynamic: summary
43
44
  [![PyPI](https://img.shields.io/pypi/v/py-ewr)](https://pypi.org/project/py-ewr/)
44
45
  [![DOI](https://zenodo.org/badge/342122359.svg)](https://zenodo.org/badge/latestdoi/342122359)
45
46
 
46
- ### **EWR tool version 2.3.7 README**
47
+ ### **ewr tool version 2.3.8 README**
47
48
 
48
49
  ### **Notes on recent version updates**
49
- - Including metadata report (this is still being ironed out and tested)
50
- - CLLMM_c and CLLMM_d ewrs are now able to be calculated without all barrage level gauges being present in the model file.
51
- - Including draft objective mapping files in the package (see below sub heading **Objective mapping** for more information). Objective mapping has been therefore pulled out of the parameter sheet
52
- - Including an example parallel processing script for running the EWR tool
50
+
51
+ #### EWR handling and outputs
53
52
  - Adding handling for cases where there are single MDBA bigmod site IDs mapping to multiple different gauges
54
- - Fix SDL resource unit mapping in the parameter sheet
55
- - Adding lat and lon to the parameter sheet
56
- - ten thousand year handling - this has been brought back online.
57
- - Remove TQDM loading bars
58
- - Adding new model format handling - 'IQQM - netcdf'
59
- - Standard time-series handling added - each column needs a gauge, followed by and underscore, followed by either flow or level (e.g. 409025_flow). This handling also has missing date filling - so any missing dates will be filled with NaN values in all columns.
60
- - bug fixes: spells of length equal to the minimum required spell length were getting filtered out of the successful events table and successful interevents table, fixed misclassification of some gauges to flow, level, and lake level categories
61
53
  - New EWRs: New Qld EWRs - SF_FD and BF_FD used to look into the FD EWRs in closer detail.
62
54
  - Adding state and Surface Water SDL (SWSDL) to py-ewr output tables
55
+ - Including metadata report (this is still being ironed out and tested)
56
+ - New handling capabilities for FIRM model formated files
57
+
58
+ #### Model metadata
59
+ - Added new FIRM ID file mapping FIRM ID to gauge number.
60
+
61
+ #### parameter metadata
62
+ - Updated parameter sheet
63
+ - Fix SDL resource unit mapping in the parameter sheet
64
+ - Adding lat and lon to the parameter sheet
65
+ - Added in model handling for FIRM model outputs
66
+ - Various minor variable renamings for consistency
67
+ - Renamed MaxLevelRise to MaxLevelChange
68
+ - Removed AnnualFlowSum column from parameter sheet
69
+ - New format of objective mapping includes the adding of objective mapping back into the parameter sheet and one secondary dataframe (objective_reference.csv) in parameter metadata.
63
70
 
64
71
  ### **Installation**
65
72
 
@@ -77,13 +84,14 @@ pip install py-ewr
77
84
  ```
78
85
 
79
86
  ### Option 1: Running the observed mode of the tool
80
- The EWR tool will use a second program called gauge getter to first download the river data at the locations and dates selected and then run this through the EWR tool
87
+ The ewr tool will use a second program called gauge getter to first download the river data at the locations and dates selected and then run this through the ewr tool.
88
+ For more information please visit the [MDBA Gauge Getter](https://github.com/MDBAuth/MDBA_Gauge_Getter) github page.
81
89
 
82
90
  ```python
83
91
 
84
92
  from datetime import datetime
85
93
 
86
- #USER INPUT REQUIRED>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
94
+ # USER INPUT REQUIRED>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
87
95
 
88
96
  dates = {'start_date': datetime(YYYY, 7, 1),
89
97
  'end_date': datetime(YYYY, 6, 30)}
@@ -98,15 +106,15 @@ gauges = ['Gauge1', 'Gauge2']
98
106
 
99
107
  from py_ewr.observed_handling import ObservedHandler
100
108
 
101
- # Running the EWR tool:
109
+ # Running the ewr tool:
102
110
  ewr_oh = ObservedHandler(gauges=gauges, dates=dates)
103
111
 
104
112
  # Generating tables:
105
- # Table 1: Summarised EWR results for the entire timeseries
113
+ # Table 1: Summarised ewr results for the entire timeseries
106
114
  ewr_results = ewr_oh.get_ewr_results()
107
115
 
108
- # Table 2: Summarised EWR results, aggregated to water years:
109
- yearly_ewr_results = ewr_oh.get_yearly_ewr_results()
116
+ # Table 2: Summarised ewr results, aggregated to water years:
117
+ yearly_results = ewr_oh.get_yearly_results()
110
118
 
111
119
  # Table 3: All events details regardless of duration
112
120
  all_events = ewr_oh.get_all_events()
@@ -122,14 +130,23 @@ all_successful_interEvents = ewr_oh.get_all_successful_interEvents()
122
130
 
123
131
  ```
124
132
 
125
- ### Option 2: Running model scenarios through the EWR tool
133
+ ### Option 2: Running model scenarios through the ewr tool
126
134
 
127
- 1. Tell the tool where the model files are (can either be local or in a remote location)
135
+ 1. Tell the tool where the model files are (can either be local or in a remote location). For each scenario list the file path of each file. Python will default to the path relative to the directory it is being run from, but if you put a full file path you can choose any file.
136
+ ```python
137
+ scenarios = {'Scenario1': ['path/to/file1', 'path/to/file2', 'path/to/file3'],
138
+ 'Scenario2': ['path/to/file1', 'path/to/file2', 'path/to/file3']}
139
+ ```
128
140
  2. Tell the tool what format the model files are in. The current model format options are:
129
141
  - 'Bigmod - MDBA'
130
142
  Bigmod formatted outputs
143
+
131
144
  - 'Source - NSW (res.csv)'
132
145
  Source res.csv formatted outputs
146
+
147
+ - 'FIRM - MDBA'
148
+ FIRM ID formatted outputs
149
+
133
150
  - 'Standard time-series'
134
151
  The first column header should be *Date* with the date values in the YYYY-MM-DD format.
135
152
  The next columns should have the *gauge* followed by *_* followed by either *flow* or *level*
@@ -151,23 +168,31 @@ all_successful_interEvents = ewr_oh.get_all_successful_interEvents()
151
168
 
152
169
 
153
170
  ```python
154
- #USER INPUT REQUIRED>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
171
+ # USER INPUT REQUIRED>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
155
172
 
156
173
  # Minimum 1 scenario and 1 related file required
157
- scenarios = {'Scenario1': ['file/location/1', 'file/location/2', 'file/location/3'],
158
- 'Scenario2': ['file/location/1', 'file/location/2', 'file/location/3']}
174
+ scenarios = {'Scenario1': ['path/to/file1', 'path/to/file2', 'path/to/file3'],
175
+ 'Scenario2': ['path/to/file1', 'path/to/file2', 'path/to/file3']}
159
176
 
160
177
  model_format = 'Bigmod - MDBA'
161
178
 
179
+ #----- other model formats include -----#
180
+ # 'Bigmod - MDBA'
181
+ # 'Source - NSW (res.csv)'
182
+ # 'FIRM - MDBA'
183
+ # 'Standard time-series'
184
+ # 'ten thousand year'
185
+
162
186
  # END USER INPUT<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
163
187
 
164
188
  ```
189
+ File names will be incorporated into the Scenario column of the ewr tool output tables for tracebility of ewr tool output and corresponding model file. In the code written below, scenario name will be appendeded to the file name. eg. Scenario_name1_all_results.csv, therefore it is suggested to have informative scenario names that can traceback to your model files as well.
165
190
 
166
191
  ``` python
167
192
  from py_ewr.scenario_handling import ScenarioHandler
168
193
  import pandas as pd
169
194
 
170
- ewr_results_dict = {}
195
+ summary_results_dict = {}
171
196
  yearly_results_dict = {}
172
197
  all_events_dict = {}
173
198
  all_interEvents_dict = {}
@@ -175,25 +200,25 @@ all_successful_Events_dict = {}
175
200
  all_successful_interEvents_dict = {}
176
201
 
177
202
  for scenario_name, scenario_list in scenarios.items():
178
- ewr_results = pd.DataFrame()
179
- yearly_ewr_results = pd.DataFrame()
203
+ summary_results = pd.DataFrame()
204
+ yearly_results = pd.DataFrame()
180
205
  all_events = pd.DataFrame()
181
206
  all_interEvents = pd.DataFrame()
182
207
  all_successful_Events = pd.DataFrame()
183
208
  all_successful_interEvents = pd.DataFrame()
184
209
  for file in scenarios[scenario_name]:
185
210
 
186
- # Running the EWR tool:
211
+ # Running the ewr tool:
187
212
  ewr_sh = ScenarioHandler(scenario_file = file,
188
213
  model_format = model_format)
189
214
 
190
215
  # Return each table and stitch the different files of the same scenario together:
191
- # Table 1: Summarised EWR results for the entire timeseries
192
- temp_ewr_results = ewr_sh.get_ewr_results()
193
- ewr_results = pd.concat([ewr_results, temp_ewr_results], axis = 0)
194
- # Table 2: Summarised EWR results, aggregated to water years:
195
- temp_yearly_ewr_results = ewr_sh.get_yearly_ewr_results()
196
- yearly_ewr_results = pd.concat([yearly_ewr_results, temp_yearly_ewr_results], axis = 0)
216
+ # Table 1: Summarised ewr results for the entire timeseries
217
+ temp_summary_results = ewr_sh.get_ewr_results()
218
+ summary_results = pd.concat([summary_results, temp_summary_results], axis = 0)
219
+ # Table 2: Summarised ewr results, aggregated to water years:
220
+ temp_yearly_results = ewr_sh.get_yearly_ewr_results()
221
+ yearly_results = pd.concat([yearly_results, temp_yearly_results], axis = 0)
197
222
  # Table 3: All events details regardless of duration
198
223
  temp_all_events = ewr_sh.get_all_events()
199
224
  all_events = pd.concat([all_events, temp_all_events], axis = 0)
@@ -209,64 +234,74 @@ for scenario_name, scenario_list in scenarios.items():
209
234
 
210
235
 
211
236
  # Optional code to output results to csv files:
212
- ewr_results.to_csv(scenario_name + 'all_results.csv')
213
- yearly_ewr_results.to_csv(scenario_name + 'yearly_ewr_results.csv')
237
+ summary_results.to_csv(scenario_name + 'summary_results.csv')
238
+ yearly_results.to_csv(scenario_name + 'yearly_results.csv')
214
239
  all_events.to_csv(scenario_name + 'all_events.csv')
215
240
  all_interEvents.to_csv(scenario_name + 'all_interevents.csv')
216
241
  all_successful_Events.to_csv(scenario_name + 'all_successful_Events.csv')
217
242
  all_successful_interEvents.to_csv(scenario_name + 'all_successful_interEvents.csv')
218
243
 
219
244
  # Save the final tables to the dictionaries:
220
- ewr_results_dict[scenario_name] = ewr_results
221
- yearly_results_dict[scenario_name] = yearly_ewr_results
222
- all_events_dict[scenario_name] = all_events_dict
245
+ summary_results_dict[scenario_name] = summary_results
246
+ yearly_results_dict[scenario_name] = yearly_results
247
+ all_events_dict[scenario_name] = all_events
223
248
  all_interEvents_dict[scenario_name] = all_interEvents
224
249
  all_successful_Events_dict[scenario_name] = all_successful_Events
225
250
  all_successful_interEvents_dict[scenario_name] = all_successful_interEvents
226
251
 
227
252
 
228
253
  ```
229
-
254
+ #### Optional arguments for ScenarioHandler
255
+ ```python
256
+ ewr_sh = ScenarioHandler(scenario_file = file,
257
+ model_format = model_format,
258
+ parameter_sheet = parameter_sheet,
259
+ calc_config_path = calc_config_path)
260
+ ```
261
+ You may add a custom parameter sheet and or calc_config_file to your EWR tool run using the ```parameter_sheet``` and ```calc_config_path``` arguments. These arguments take a string file path pointing to files. Please check this ewr_calc_config.json file found in parameter metadata to see if any EWRs in your custom parameter sheet are not represented in the calc_config_file. For an EWR to be calculated, it must be found in both calc_config.json and the parameter sheet.
230
262
 
231
263
  ### **Purpose**
232
264
  This tool has two purposes:
233
- 1. Operational: Tracking EWR success at gauges of interest in real time - option 1 above.
234
- 2. Planning: Comparing EWR success between scenarios (i.e. model runs) - option 2 above.
265
+ 1. Operational: Tracking ewr success at gauges of interest in real time - option 1 above.
266
+ 2. Planning: Comparing ewr success between scenarios (i.e. model runs) - option 2 above.
235
267
 
236
268
  **Support**
237
- For issues relating to the script, a tutorial, or feedback please contact Lara Palmer at lara.palmer@mdba.gov.au, Martin Job at martin.job@mdba.gov.au, or Joel Bailey at joel.bailey@mdba.gov.au
269
+ For issues relating to the script, a tutorial, or feedback please contact Martin Job at martin.job@mdba.gov.au, Sirous Safari Pour at sirous.safaripour@mdba.gov.au, Elisha Freedman at elisha.freedman@mdba.gov.au, Joel Bailey at joel.bailey@mdba.gov.au, or Lara Palmer at lara.palmer@mdba.gov.au.
238
270
 
239
271
 
240
272
  **Disclaimer**
241
- Every effort has been taken to ensure the EWR database represents the original EWRs from state Long Term Water Plans (LTWPs) and Environmental Water Management Plans (EWMPs) as best as possible, and that the code within this tool has been developed to interpret and analyse these EWRs in an accurate way. However, there may still be unresolved bugs in the EWR parameter sheet and/or EWR tool. Please report any bugs to the issues tab under the GitHub project so we can investigate further.
273
+ Every effort has been taken to ensure the ewr database represents the original EWRs from state Long Term Water Plans (LTWPs) and Environmental Water Management Plans (EWMPs) as best as possible, and that the code within this tool has been developed to interpret and analyse these EWRs in an accurate way. However, there may still be unresolved bugs in the ewr parameter sheet and/or ewr tool. Please report any bugs to the issues tab under the GitHub project so we can investigate further.
242
274
 
243
275
 
244
276
  **Notes on development of the dataset of EWRs**
245
277
  The MDBA has worked with Basin state representatives to ensure scientific integrity of EWRs has been maintained when translating from raw EWRs in the Basin state LTWPs and EWMPs to the machine readable format found in the parameter sheet within this tool.
246
278
 
279
+ Environmental Water Requirements (EWRs) in the tool are subject to change when the relevant documents including Long Term Water Plans (LTWPs) and Environmental Water Management Plans (EWMPs) are updated or move from draft to final versions. LTWPs that are currently in draft form include the ACT and the upper Murrumbidgee section of the NSW Murrumbidgee LTWP.
280
+
247
281
  **Compatibility**
248
282
 
249
- NSW:
250
283
  - All Queensland catchments
251
284
  - All New South Wales catchments
252
285
  - All South Australian catchments
253
286
  - All EWRs from river based Environmental Water Management Plans (EWMPs) in Victoria*
287
+ - All EWRs for ACT (draft version)
254
288
 
255
- *Currently the wetland EWMPS and mixed wetland-river EWMPs in Victoria contain EWRs that cannot be evaluated by an automated EWR tool so the EWRs from these plans have been left out for now. The MDBA will work with our Victorian colleagues to ensure any updated EWRs in these plans are integrated into the tool where possible.
289
+ *The wetland EWMPS and mixed wetland-river EWMPs in Victoria are in progress. The MDBA will work with our Victorian colleagues to ensure any updated EWRs in these plans are integrated into the tool where possible.
256
290
 
257
291
  **Input data**
258
292
 
259
293
  - Gauge data from the relevant Basin state websites and the Bureau of Meteorology website
260
294
  - Scenario data input by the user
261
295
  - Model metadata for location association between gauge ID's and model nodes
262
- - EWR parameter sheet
296
+ #### optional
297
+ - ewr parameter sheet
298
+ - calc_config.json
263
299
 
264
- **Running the tool**
300
+ **Objective mapping**
301
+ The objective mapping is located in the EWR tool package. This is intended to be used to link EWRs to the detailed objectives, theme level targets and specific goals. The objective reference file is located in the py_ewr/parameter_metadata folder:
265
302
 
266
- Consult the user manual for instructions on how to run the tool. Please email the above email addresses for a copy of the user manual.
303
+ parameter_sheet.csv (EnvObj column)
304
+ obj_reference.csv
267
305
 
268
- **Objective mapping**
269
- Objective mapping csv files are now included in the EWR tool package. Currently this objective mapping is in an early draft format. The objective mapping will be finalised after consultation with relevant state representatives. The files are intended to be used together to link EWRs to the detailed objectives, theme level targets and specific goals. The three sheets are located in the py_ewr/parameter_metadata folder:
270
- - ewr2obj.csv: For each planning unit, gauge, ewr combination there are either one or many env_obj codes. These env_obj codes come under one of five different theme level targets (Native Fish, Native vegetation, Waterbirds, Other species or Ecosystem functions)
271
- - obj2target.csv: env_obj's are unique to their planning unit in the LTWP (noting there are often a lot of similarities between env_obj's in the same states). The plain english wording of the env objectives is also contained in this csv. The LTWP, planning unit and env_obj rows are repeated for each specific goal related to that LTWP, planning unit and env_obj.
272
- - obj2yrtarget.csv: The environmental objectives are related to 5, 10 and 20 year targets
306
+ Contains the individual environmnetal objectives listed in the 'EnvObj' column of the parameter sheet and their ecological targets (Target) and plain english description of objectives (Objectives) for each planning unit, long term water plan (LTWPShortName), and surface water sustainable diversion limit (SWSDLName).
307
+ the function ```get_obj_mapping()``` is available to automatically merge the information from obj_reference.csv with the parameter sheet to link these objectives with their specific ewr_codes.
@@ -0,0 +1,21 @@
1
+ py_ewr/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
2
+ py_ewr/data_inputs.py,sha256=-DLkAJXG_nre5YRg7zqV6TbwjWHAE17uEYDz3mNlGpA,27617
3
+ py_ewr/evaluate_EWRs.py,sha256=uzreM0mYNh7efbJb_BkpijpYSymFdN9OdDUUjsCYU9A,204277
4
+ py_ewr/io.py,sha256=Is0xPAzLx6-ylpTFyYJxMimkNVxxoTxUcknTk6bQbgs,840
5
+ py_ewr/observed_handling.py,sha256=XCjLFnIdhWpQLXKOyXmk0oc8MGkNLPG4VVsu-_tkWss,18236
6
+ py_ewr/scenario_handling.py,sha256=ke38gimVb-Maqe01AE2WvpAEleqmKfLZwoLrYi6GbsQ,54142
7
+ py_ewr/summarise_results.py,sha256=AvhlByCPJDSyoliuiZoZNlVmCicBYaDplM7fKIeEGdM,31602
8
+ py_ewr/model_metadata/EWR_Sitelist_FIRM_20250718.csv,sha256=FjiKIfNIjHXWuZCczvC9J1K7dxohBWTrWMYUVnp7m-Q,14240
9
+ py_ewr/model_metadata/SiteID_MDBA.csv,sha256=-nPEWzqW60HBh3y7u8Ofiq8aYTyZqhQ-8eakuDVdUg4,168770
10
+ py_ewr/model_metadata/SiteID_NSW.csv,sha256=UVBxN43Z5KWCvWhQ5Rh6TNEn35q4_sjPxKyHg8wPFws,6805
11
+ py_ewr/model_metadata/iqqm_stations.csv,sha256=vl4CPtPslG5VplSzf_yLZulTrmab-mEBHOfzFtS1kf4,110
12
+ py_ewr/parameter_metadata/ewr_calc_config.json,sha256=udg-nvUMkLav1SCa6vLmqumTgMJnJudHsoIWZVvjaqI,18369
13
+ py_ewr/parameter_metadata/objective_reference_NB_SA_WIM_NE_LACH_BIDG_MLD_ACT.csv,sha256=N28O2sjsON1SDfgqggxw--QPzmwXQNumwnpQDEQ35Ak,131944
14
+ py_ewr/parameter_metadata/parameter_sheet.csv,sha256=l-DO-bW_2F3PR1PZbn3np5E_vPLMaTGkGmv68l1KaEM,759290
15
+ py_ewr/parameter_metadata/parameter_sheet_NB_SA_WIM_NE_LACH_BIDG_MLD_ACT.csv,sha256=yuGZOf_DC_2_n28IYmpQG1neXy6gpd5gIc75GO5RykQ,820586
16
+ py_ewr/parameter_metadata/parameter_sheet_NB_SA_WIM_NE_LACH_BIDG_MLD_ACT_added_act_env.csv,sha256=AB-G0IFpGzx0MhKGM-vXirsirl4sTxxeULSrbcrwgCE,836107
17
+ py_ewr-2.3.8.dist-info/licenses/LICENSE,sha256=ogEPNDSH0_dhiv_lT3ifVIdgIzHAqNA_SemnxUfPBJk,7048
18
+ py_ewr-2.3.8.dist-info/METADATA,sha256=SYPFLvw0J8YxBAHBLy2bxXt9jwPa5oLBgjk0mBND9h4,14329
19
+ py_ewr-2.3.8.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
20
+ py_ewr-2.3.8.dist-info/top_level.txt,sha256=n3725d-64Cjyb-YMUMV64UAuIflzUh2_UZSxiIbrur4,7
21
+ py_ewr-2.3.8.dist-info/RECORD,,
@@ -1,5 +1,5 @@
1
1
  Wheel-Version: 1.0
2
- Generator: setuptools (75.8.0)
2
+ Generator: setuptools (80.9.0)
3
3
  Root-Is-Purelib: true
4
4
  Tag: py3-none-any
5
5