deepdiscover-hydro 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (56) hide show
  1. deepdiscover_hydro-1.0.0/CHANGELOG.md +13 -0
  2. deepdiscover_hydro-1.0.0/CITATION.cff +29 -0
  3. deepdiscover_hydro-1.0.0/LICENSE +29 -0
  4. deepdiscover_hydro-1.0.0/MANIFEST.in +15 -0
  5. deepdiscover_hydro-1.0.0/PKG-INFO +197 -0
  6. deepdiscover_hydro-1.0.0/README.md +164 -0
  7. deepdiscover_hydro-1.0.0/check.py +51 -0
  8. deepdiscover_hydro-1.0.0/docs/boundaries.md +48 -0
  9. deepdiscover_hydro-1.0.0/docs/camels.md +68 -0
  10. deepdiscover_hydro-1.0.0/docs/customization.md +54 -0
  11. deepdiscover_hydro-1.0.0/docs/data.md +26 -0
  12. deepdiscover_hydro-1.0.0/docs/getting_started.md +26 -0
  13. deepdiscover_hydro-1.0.0/docs/index.md +7 -0
  14. deepdiscover_hydro-1.0.0/docs/installation.md +34 -0
  15. deepdiscover_hydro-1.0.0/docs/limitations.md +17 -0
  16. deepdiscover_hydro-1.0.0/docs/model_configuration.md +49 -0
  17. deepdiscover_hydro-1.0.0/docs/observations.md +35 -0
  18. deepdiscover_hydro-1.0.0/docs/routing.md +27 -0
  19. deepdiscover_hydro-1.0.0/docs/scientific_formulation.md +77 -0
  20. deepdiscover_hydro-1.0.0/docs/structural_comparison.md +22 -0
  21. deepdiscover_hydro-1.0.0/docs/topologies.md +57 -0
  22. deepdiscover_hydro-1.0.0/docs/training_and_evaluation.md +62 -0
  23. deepdiscover_hydro-1.0.0/examples/01_streamflow.ipynb +220 -0
  24. deepdiscover_hydro-1.0.0/examples/02_groundwater_level.ipynb +178 -0
  25. deepdiscover_hydro-1.0.0/examples/03_sources_losses_routing.ipynb +164 -0
  26. deepdiscover_hydro-1.0.0/examples/04_camels_us.ipynb +240 -0
  27. deepdiscover_hydro-1.0.0/mkdocs.yml +18 -0
  28. deepdiscover_hydro-1.0.0/pyproject.toml +35 -0
  29. deepdiscover_hydro-1.0.0/run_examples.py +156 -0
  30. deepdiscover_hydro-1.0.0/setup.cfg +4 -0
  31. deepdiscover_hydro-1.0.0/src/deepdiscover/__init__.py +33 -0
  32. deepdiscover_hydro-1.0.0/src/deepdiscover/api.py +419 -0
  33. deepdiscover_hydro-1.0.0/src/deepdiscover/boundaries.py +86 -0
  34. deepdiscover_hydro-1.0.0/src/deepdiscover/data.py +182 -0
  35. deepdiscover_hydro-1.0.0/src/deepdiscover/dataset.py +165 -0
  36. deepdiscover_hydro-1.0.0/src/deepdiscover/metrics.py +105 -0
  37. deepdiscover_hydro-1.0.0/src/deepdiscover/model.py +1416 -0
  38. deepdiscover_hydro-1.0.0/src/deepdiscover/observations.py +62 -0
  39. deepdiscover_hydro-1.0.0/src/deepdiscover/routing.py +234 -0
  40. deepdiscover_hydro-1.0.0/src/deepdiscover/snow.py +310 -0
  41. deepdiscover_hydro-1.0.0/src/deepdiscover/structural.py +104 -0
  42. deepdiscover_hydro-1.0.0/src/deepdiscover_hydro.egg-info/PKG-INFO +197 -0
  43. deepdiscover_hydro-1.0.0/src/deepdiscover_hydro.egg-info/SOURCES.txt +54 -0
  44. deepdiscover_hydro-1.0.0/src/deepdiscover_hydro.egg-info/dependency_links.txt +1 -0
  45. deepdiscover_hydro-1.0.0/src/deepdiscover_hydro.egg-info/requires.txt +19 -0
  46. deepdiscover_hydro-1.0.0/src/deepdiscover_hydro.egg-info/top_level.txt +1 -0
  47. deepdiscover_hydro-1.0.0/tests/conftest.py +16 -0
  48. deepdiscover_hydro-1.0.0/tests/test_camels_data.py +62 -0
  49. deepdiscover_hydro-1.0.0/tests/test_camels_windows.py +40 -0
  50. deepdiscover_hydro-1.0.0/tests/test_dataset.py +46 -0
  51. deepdiscover_hydro-1.0.0/tests/test_driver_channels.py +123 -0
  52. deepdiscover_hydro-1.0.0/tests/test_equations_numpy.py +41 -0
  53. deepdiscover_hydro-1.0.0/tests/test_physics.py +207 -0
  54. deepdiscover_hydro-1.0.0/tests/test_topologies.py +48 -0
  55. deepdiscover_hydro-1.0.0/tests/test_training.py +47 -0
  56. deepdiscover_hydro-1.0.0/verify_camels.py +102 -0
@@ -0,0 +1,13 @@
1
+ # Changelog
2
+
3
+ ## 1.0.0
4
+
5
+ First public release of DeepDiscover for water-quantity modeling.
6
+
7
+ - Conservative DiscoverUnits with Causal Relationship Constraints.
8
+ - Multiple hydrological sources, losses, finite storage capacities and overflow.
9
+ - Instantaneous, fixed, learned and state-dependent temporal routing.
10
+ - Optional informational memory, snow accumulation and melt.
11
+ - Streamflow, groundwater level and other observation operators.
12
+ - Multivariable calibration, custom loss functions and numerical diagnostics.
13
+ - CAMELS-US and synthetic examples, documentation and tests.
@@ -0,0 +1,29 @@
1
+ cff-version: 1.2.0
2
+ message: "Please cite the Multiple-DU methodological manuscript when using DeepDiscover."
3
+ title: "DeepDiscover"
4
+ version: 1.0.0
5
+ type: software
6
+ license: BSD-3-Clause
7
+ repository-code: "https://github.com/avadombi/DeepDiscover"
8
+ authors:
9
+ - family-names: Adombi
10
+ given-names: Adoubi Vincent De Paul
11
+ preferred-citation:
12
+ type: unpublished
13
+ title: "Can discharge identify internal hydrological processes? Neural equifinality in the Multiple-DU extension of DeepDiscover"
14
+ authors:
15
+ - family-names: Adombi
16
+ given-names: Adoubi Vincent De Paul
17
+ year: 2026
18
+ status: submitted
19
+ notes: "Manuscript submitted."
20
+ references:
21
+ - type: article
22
+ title: "DeepDiscover: towards autonomous discovery of bucket-type conceptual models – a proof of concept applied to hydrology"
23
+ authors:
24
+ - family-names: Adombi
25
+ given-names: Adoubi Vincent De Paul
26
+ journal: Journal of Hydrology
27
+ year: 2026
28
+ volume: "671"
29
+ doi: 10.1016/j.jhydrol.2026.135249
@@ -0,0 +1,29 @@
1
+ BSD 3-Clause License
2
+
3
+ Copyright (c) 2026, Vincent Adombi
4
+ All rights reserved.
5
+
6
+ Redistribution and use in source and binary forms, with or without
7
+ modification, are permitted provided that the following conditions are met:
8
+
9
+ 1. Redistributions of source code must retain the above copyright notice,
10
+ this list of conditions and the following disclaimer.
11
+
12
+ 2. Redistributions in binary form must reproduce the above copyright notice,
13
+ this list of conditions and the following disclaimer in the documentation
14
+ and/or other materials provided with the distribution.
15
+
16
+ 3. Neither the name of the copyright holder nor the names of its contributors
17
+ may be used to endorse or promote products derived from this software
18
+ without specific prior written permission.
19
+
20
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
21
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
22
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
23
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
24
+ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
25
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
26
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
27
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
28
+ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,15 @@
1
+ include LICENSE
2
+ include CHANGELOG.md
3
+ include CITATION.cff
4
+ include mkdocs.yml
5
+ include check.py
6
+ include run_examples.py
7
+ include verify_camels.py
8
+ recursive-include docs *.md
9
+ recursive-include examples *.ipynb
10
+ recursive-include tests *.py
11
+ prune data
12
+ prune results
13
+ prune .venv
14
+ prune .pytest_cache
15
+ global-exclude __pycache__ *.py[cod] *.log
@@ -0,0 +1,197 @@
1
+ Metadata-Version: 2.4
2
+ Name: deepdiscover-hydro
3
+ Version: 1.0.0
4
+ Summary: Conservative differentiable hydrological modeling with DiscoverUnits
5
+ Author: Vincent Adombi
6
+ License-Expression: BSD-3-Clause
7
+ Project-URL: Repository, https://github.com/avadombi/DeepDiscover
8
+ Project-URL: Issues, https://github.com/avadombi/DeepDiscover/issues
9
+ Keywords: hydrology,groundwater,scientific machine learning,conservation
10
+ Classifier: Intended Audience :: Science/Research
11
+ Classifier: Programming Language :: Python :: 3
12
+ Classifier: Topic :: Scientific/Engineering
13
+ Requires-Python: <3.13,>=3.11
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: numpy>=1.26
17
+ Requires-Dist: tensorflow<2.22,>=2.16
18
+ Requires-Dist: keras>=3.0
19
+ Provides-Extra: examples
20
+ Requires-Dist: pandas>=2.0; extra == "examples"
21
+ Requires-Dist: matplotlib>=3.7; extra == "examples"
22
+ Requires-Dist: nbformat>=5.9; extra == "examples"
23
+ Requires-Dist: nbclient>=0.9; extra == "examples"
24
+ Requires-Dist: ipykernel>=6.0; extra == "examples"
25
+ Requires-Dist: jupyterlab>=4.0; extra == "examples"
26
+ Provides-Extra: docs
27
+ Requires-Dist: mkdocs>=1.6; extra == "docs"
28
+ Provides-Extra: test
29
+ Requires-Dist: pytest>=8.0; extra == "test"
30
+ Requires-Dist: build>=1.0; extra == "test"
31
+ Requires-Dist: pandas>=2.0; extra == "test"
32
+ Dynamic: license-file
33
+
34
+ # DeepDiscover
35
+
36
+ DeepDiscover is a Python library for differentiable hydrological modeling with explicit water conservation. A model contains one or more DiscoverUnits (DUs), each with a water store and constitutive relationships that determine water releases. Each release process has one declared destination. Temporal routing is specified separately.
37
+
38
+ The same hydrological system can be calibrated against streamflow, groundwater levels, soil moisture and other measured variables using observation operators. Multiple observation types can be used together.
39
+
40
+ ## Installation
41
+
42
+ Python 3.11 or 3.12 is recommended. DeepDiscover uses TensorFlow and Keras.
43
+
44
+ Install from this repository:
45
+
46
+ ```bash
47
+ python -m pip install .
48
+ ```
49
+
50
+ Install the optional dependencies for the notebooks and documentation:
51
+
52
+ ```bash
53
+ python -m pip install ".[examples,docs]"
54
+ ```
55
+
56
+ Once published on [PyPI](https://pypi.org/), install the distribution with `python -m pip install deepdiscover-hydro`.
57
+
58
+ ## Example
59
+
60
+ ```python
61
+ import numpy as np
62
+ from deepdiscover import CRCNet, DeepDiscover, DiscoverUnit, MultiDUModel
63
+
64
+ n_days = 365
65
+ precipitation = np.maximum(2 + np.sin(np.arange(n_days) / 9), 0).astype("float32")
66
+ pet = np.ones(n_days, dtype="float32")
67
+
68
+ water = precipitation[:, None]
69
+ drivers = np.column_stack([precipitation, pet])
70
+
71
+ release = CRCNet(gamma=[1, -1, 1], x_ref=[5, 3, 100])
72
+ unit = DiscoverUnit(process_nets=[release], context_indices=[(0, 1)])
73
+ physical_model = MultiDUModel([unit], destinations=[["outlet"]])
74
+ model = DeepDiscover(physical_model, observations={"streamflow": "outlet"})
75
+
76
+ prediction = model.predict(water[None], drivers[None])["streamflow"]
77
+ print(prediction.shape) # (1, 365, 1)
78
+ ```
79
+
80
+ To calibrate the model, provide observed discharge as an array and create training windows:
81
+
82
+ ```python
83
+ from deepdiscover import HydroDataset
84
+
85
+ data = HydroDataset(water, drivers, {"streamflow": observed_discharge})
86
+ train = data.make_windows(warmup=30, score_length=60, stride=30)
87
+ model.fit(train, epochs=30, batch_size=4, loss="mse")
88
+ ```
89
+
90
+ Here `observed_discharge` is a measured series containing one value per day.
91
+
92
+ ## Main features
93
+
94
+ | Component | Options |
95
+ | --- | --- |
96
+ | DiscoverUnits | Multiple storages, processes and fixed or learned storage capacities |
97
+ | Constitutive relationships | Causal Relationship Constraints (CRC), custom Keras layers |
98
+ | Sources | Prescribed water inputs, lateral inflow, managed recharge, custom source layers |
99
+ | Losses | Evapotranspiration, pumping, prescribed extraction, custom sink layers |
100
+ | Routes | Instantaneous, prescribed, learned and state-dependent temporal routing |
101
+ | Additional states | Optional information memory and storage deficit |
102
+ | Snow | Differentiable degree-day snow model |
103
+ | Observations | Direct outputs, groundwater-head operators, soil-moisture operators, custom layers |
104
+ | Training | Time windows, warm-up, observation masks, multivariable and custom losses |
105
+ | Diagnostics | States, fluxes, routing storage, numerical water balance and model scores |
106
+ | Model comparison | Multi-seed fitting and validation-based comparison of declared structures |
107
+
108
+ Water losses belong to the physical balance. Training losses measure the agreement between simulations and observations. Both can be configured independently.
109
+
110
+ ## Model topologies
111
+
112
+ Each release process has one destination, and several processes may lead to the same destination. The library supports both a direct outlet pathway plus an internal DU transfer, and a cascade with two downstream outlet processes. See [model topologies](docs/topologies.md) for explicit configurations and [structural comparison](docs/structural_comparison.md) for validation-based comparisons.
113
+
114
+ ## Examples
115
+
116
+ The `examples/` directory contains four notebooks:
117
+
118
+ - `01_streamflow.ipynb`: simulation and calibration of streamflow.
119
+ - `02_groundwater_level.ipynb`: joint calibration using streamflow and groundwater levels.
120
+ - `03_sources_losses_routing.ipynb`: custom physical boundaries and delayed routing.
121
+ - `04_camels_us.ipynb`: a CAMELS-US case study with snow and chronological data partitions.
122
+
123
+ To run the three self-contained notebooks:
124
+
125
+ ```bash
126
+ python run_examples.py
127
+ ```
128
+
129
+ To run the CAMELS-US example, obtain the original forcing and streamflow files and follow [the data instructions](docs/camels.md). Three execution modes are provided:
130
+
131
+ ```bash
132
+ # Short check of loading, simulation, calibration and evaluation
133
+ python run_examples.py --only 04_camels_us
134
+
135
+ # Measure the cost of full-length training windows (three epochs)
136
+ python run_examples.py --only 04_camels_us --camels-mode benchmark
137
+
138
+ # Full chronological calibration with the original training-window settings
139
+ python run_examples.py --only 04_camels_us --camels-mode full
140
+ ```
141
+
142
+ The short check uses a limited number of training windows and two epochs. Its scores should not be used as scientific results. The benchmark and full modes use 19 parallel windows with 365 warm-up and 365 scored days. Full mode uses up to 400 epochs and validates every five epochs. It saves the validation-selected model and its training history under `results/camels/01013500/two_du_serial_seed42/`. During notebook execution, progress is printed for each code cell and each optimization batch.
143
+
144
+ Output notebooks and execution reports are written to `results/notebooks/`, which is excluded from version control.
145
+
146
+ A saved CAMELS-US checkpoint can be verified independently, without repeating model fitting or consulting the test period:
147
+
148
+ ```bash
149
+ python verify_camels.py
150
+ ```
151
+
152
+ This command requires the CAMELS-US files and the `checkpoint/` and `history.json` created by the full example.
153
+
154
+ ## Documentation
155
+
156
+ The `docs/` directory covers the [scientific equations](docs/scientific_formulation.md), [model configuration](docs/model_configuration.md), [topologies](docs/topologies.md), [data](docs/data.md), [routing](docs/routing.md), [observation operators](docs/observations.md), [custom components](docs/customization.md), [calibration](docs/training_and_evaluation.md) and [model comparison](docs/structural_comparison.md).
157
+
158
+ To build the documentation locally:
159
+
160
+ ```bash
161
+ python -m pip install ".[docs]"
162
+ mkdocs build
163
+ ```
164
+
165
+ ## Tests
166
+
167
+ ```bash
168
+ python -m pip install ".[test]"
169
+ python check.py
170
+ python -m pytest -q
171
+ ```
172
+
173
+ The tests cover conservation, routing, gradients, observation operators, training, model saving, temporal windows and CAMELS data preparation.
174
+
175
+ ## Reporting bugs
176
+
177
+ Report bugs and request features through [GitHub Issues](https://github.com/avadombi/DeepDiscover/issues). Include the Python, TensorFlow and Keras versions, the model configuration, a minimal example and the complete error message. Avoid sharing private datasets or credentials.
178
+
179
+ See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines.
180
+
181
+ ## Scientific references and license
182
+
183
+ DeepDiscover is intended for water-quantity modeling. Its latent states and processes require scientific interpretation in the context of the available observations. See [limitations](docs/limitations.md).
184
+
185
+ The methodological article is identified in [CITATION.cff](CITATION.cff). Source code is distributed under the [BSD 3-Clause License](LICENSE). CAMELS-US data must be obtained separately from their original provider.
186
+
187
+ ## Citation
188
+
189
+ The current Multiple-DU methodology is described in:
190
+
191
+ Adombi, A. V. D. P. (2026). *Can discharge identify internal hydrological processes? Neural equifinality in the Multiple-DU extension of DeepDiscover*. Manuscript submitted to *Journal of Hydrology*.
192
+
193
+ The original DeepDiscover formulation is published in:
194
+
195
+ Adombi, A. V. D. P. (2026). *DeepDiscover: towards autonomous discovery of bucket-type conceptual models – a proof of concept applied to hydrology*. *Journal of Hydrology*, 671, 135249. https://doi.org/10.1016/j.jhydrol.2026.135249
196
+
197
+ The submitted manuscript currently has no publication DOI. Citation information will be updated when it is published.
@@ -0,0 +1,164 @@
1
+ # DeepDiscover
2
+
3
+ DeepDiscover is a Python library for differentiable hydrological modeling with explicit water conservation. A model contains one or more DiscoverUnits (DUs), each with a water store and constitutive relationships that determine water releases. Each release process has one declared destination. Temporal routing is specified separately.
4
+
5
+ The same hydrological system can be calibrated against streamflow, groundwater levels, soil moisture and other measured variables using observation operators. Multiple observation types can be used together.
6
+
7
+ ## Installation
8
+
9
+ Python 3.11 or 3.12 is recommended. DeepDiscover uses TensorFlow and Keras.
10
+
11
+ Install from this repository:
12
+
13
+ ```bash
14
+ python -m pip install .
15
+ ```
16
+
17
+ Install the optional dependencies for the notebooks and documentation:
18
+
19
+ ```bash
20
+ python -m pip install ".[examples,docs]"
21
+ ```
22
+
23
+ Once published on [PyPI](https://pypi.org/), install the distribution with `python -m pip install deepdiscover-hydro`.
24
+
25
+ ## Example
26
+
27
+ ```python
28
+ import numpy as np
29
+ from deepdiscover import CRCNet, DeepDiscover, DiscoverUnit, MultiDUModel
30
+
31
+ n_days = 365
32
+ precipitation = np.maximum(2 + np.sin(np.arange(n_days) / 9), 0).astype("float32")
33
+ pet = np.ones(n_days, dtype="float32")
34
+
35
+ water = precipitation[:, None]
36
+ drivers = np.column_stack([precipitation, pet])
37
+
38
+ release = CRCNet(gamma=[1, -1, 1], x_ref=[5, 3, 100])
39
+ unit = DiscoverUnit(process_nets=[release], context_indices=[(0, 1)])
40
+ physical_model = MultiDUModel([unit], destinations=[["outlet"]])
41
+ model = DeepDiscover(physical_model, observations={"streamflow": "outlet"})
42
+
43
+ prediction = model.predict(water[None], drivers[None])["streamflow"]
44
+ print(prediction.shape) # (1, 365, 1)
45
+ ```
46
+
47
+ To calibrate the model, provide observed discharge as an array and create training windows:
48
+
49
+ ```python
50
+ from deepdiscover import HydroDataset
51
+
52
+ data = HydroDataset(water, drivers, {"streamflow": observed_discharge})
53
+ train = data.make_windows(warmup=30, score_length=60, stride=30)
54
+ model.fit(train, epochs=30, batch_size=4, loss="mse")
55
+ ```
56
+
57
+ Here `observed_discharge` is a measured series containing one value per day.
58
+
59
+ ## Main features
60
+
61
+ | Component | Options |
62
+ | --- | --- |
63
+ | DiscoverUnits | Multiple storages, processes and fixed or learned storage capacities |
64
+ | Constitutive relationships | Causal Relationship Constraints (CRC), custom Keras layers |
65
+ | Sources | Prescribed water inputs, lateral inflow, managed recharge, custom source layers |
66
+ | Losses | Evapotranspiration, pumping, prescribed extraction, custom sink layers |
67
+ | Routes | Instantaneous, prescribed, learned and state-dependent temporal routing |
68
+ | Additional states | Optional information memory and storage deficit |
69
+ | Snow | Differentiable degree-day snow model |
70
+ | Observations | Direct outputs, groundwater-head operators, soil-moisture operators, custom layers |
71
+ | Training | Time windows, warm-up, observation masks, multivariable and custom losses |
72
+ | Diagnostics | States, fluxes, routing storage, numerical water balance and model scores |
73
+ | Model comparison | Multi-seed fitting and validation-based comparison of declared structures |
74
+
75
+ Water losses belong to the physical balance. Training losses measure the agreement between simulations and observations. Both can be configured independently.
76
+
77
+ ## Model topologies
78
+
79
+ Each release process has one destination, and several processes may lead to the same destination. The library supports both a direct outlet pathway plus an internal DU transfer, and a cascade with two downstream outlet processes. See [model topologies](docs/topologies.md) for explicit configurations and [structural comparison](docs/structural_comparison.md) for validation-based comparisons.
80
+
81
+ ## Examples
82
+
83
+ The `examples/` directory contains four notebooks:
84
+
85
+ - `01_streamflow.ipynb`: simulation and calibration of streamflow.
86
+ - `02_groundwater_level.ipynb`: joint calibration using streamflow and groundwater levels.
87
+ - `03_sources_losses_routing.ipynb`: custom physical boundaries and delayed routing.
88
+ - `04_camels_us.ipynb`: a CAMELS-US case study with snow and chronological data partitions.
89
+
90
+ To run the three self-contained notebooks:
91
+
92
+ ```bash
93
+ python run_examples.py
94
+ ```
95
+
96
+ To run the CAMELS-US example, obtain the original forcing and streamflow files and follow [the data instructions](docs/camels.md). Three execution modes are provided:
97
+
98
+ ```bash
99
+ # Short check of loading, simulation, calibration and evaluation
100
+ python run_examples.py --only 04_camels_us
101
+
102
+ # Measure the cost of full-length training windows (three epochs)
103
+ python run_examples.py --only 04_camels_us --camels-mode benchmark
104
+
105
+ # Full chronological calibration with the original training-window settings
106
+ python run_examples.py --only 04_camels_us --camels-mode full
107
+ ```
108
+
109
+ The short check uses a limited number of training windows and two epochs. Its scores should not be used as scientific results. The benchmark and full modes use 19 parallel windows with 365 warm-up and 365 scored days. Full mode uses up to 400 epochs and validates every five epochs. It saves the validation-selected model and its training history under `results/camels/01013500/two_du_serial_seed42/`. During notebook execution, progress is printed for each code cell and each optimization batch.
110
+
111
+ Output notebooks and execution reports are written to `results/notebooks/`, which is excluded from version control.
112
+
113
+ A saved CAMELS-US checkpoint can be verified independently, without repeating model fitting or consulting the test period:
114
+
115
+ ```bash
116
+ python verify_camels.py
117
+ ```
118
+
119
+ This command requires the CAMELS-US files and the `checkpoint/` and `history.json` created by the full example.
120
+
121
+ ## Documentation
122
+
123
+ The `docs/` directory covers the [scientific equations](docs/scientific_formulation.md), [model configuration](docs/model_configuration.md), [topologies](docs/topologies.md), [data](docs/data.md), [routing](docs/routing.md), [observation operators](docs/observations.md), [custom components](docs/customization.md), [calibration](docs/training_and_evaluation.md) and [model comparison](docs/structural_comparison.md).
124
+
125
+ To build the documentation locally:
126
+
127
+ ```bash
128
+ python -m pip install ".[docs]"
129
+ mkdocs build
130
+ ```
131
+
132
+ ## Tests
133
+
134
+ ```bash
135
+ python -m pip install ".[test]"
136
+ python check.py
137
+ python -m pytest -q
138
+ ```
139
+
140
+ The tests cover conservation, routing, gradients, observation operators, training, model saving, temporal windows and CAMELS data preparation.
141
+
142
+ ## Reporting bugs
143
+
144
+ Report bugs and request features through [GitHub Issues](https://github.com/avadombi/DeepDiscover/issues). Include the Python, TensorFlow and Keras versions, the model configuration, a minimal example and the complete error message. Avoid sharing private datasets or credentials.
145
+
146
+ See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines.
147
+
148
+ ## Scientific references and license
149
+
150
+ DeepDiscover is intended for water-quantity modeling. Its latent states and processes require scientific interpretation in the context of the available observations. See [limitations](docs/limitations.md).
151
+
152
+ The methodological article is identified in [CITATION.cff](CITATION.cff). Source code is distributed under the [BSD 3-Clause License](LICENSE). CAMELS-US data must be obtained separately from their original provider.
153
+
154
+ ## Citation
155
+
156
+ The current Multiple-DU methodology is described in:
157
+
158
+ Adombi, A. V. D. P. (2026). *Can discharge identify internal hydrological processes? Neural equifinality in the Multiple-DU extension of DeepDiscover*. Manuscript submitted to *Journal of Hydrology*.
159
+
160
+ The original DeepDiscover formulation is published in:
161
+
162
+ Adombi, A. V. D. P. (2026). *DeepDiscover: towards autonomous discovery of bucket-type conceptual models – a proof of concept applied to hydrology*. *Journal of Hydrology*, 671, 135249. https://doi.org/10.1016/j.jhydrol.2026.135249
163
+
164
+ The submitted manuscript currently has no publication DOI. Citation information will be updated when it is published.
@@ -0,0 +1,51 @@
1
+ """Short installation check for a new DeepDiscover environment.
2
+
3
+ Run from the repository root:
4
+ python check.py
5
+ python -m pytest -q
6
+ """
7
+ from pathlib import Path
8
+ import numpy as np
9
+ import tensorflow as tf
10
+ import keras
11
+ from deepdiscover import (
12
+ CRCNet, DiscoverUnit, MultiDUModel, DeepDiscover,
13
+ PrescribedSink, HydroDataset,
14
+ )
15
+
16
+
17
+ def main():
18
+ print('TensorFlow', tf.__version__, 'Keras', keras.__version__)
19
+ t = np.arange(45, dtype=np.float32)
20
+ rain = 2.0 + np.sin(t / 5)
21
+ pet = np.ones_like(rain)
22
+ drivers = np.stack([rain, pet], axis=-1)[None]
23
+ water = rain[None, :, None]
24
+ du = DiscoverUnit(
25
+ process_nets=[CRCNet([1, -1, 1], [5, 5, 100])],
26
+ context_indices=[(0, 1)],
27
+ exchanges=[PrescribedSink(driver_index=1, scale=0.1)],
28
+ )
29
+ model = DeepDiscover(MultiDUModel([du], [['outlet']]))
30
+ out = model.simulate(water, drivers)
31
+ error = float(tf.reduce_max(tf.abs(out['mass_residual'])).numpy())
32
+ print('Shapes:', {k: tuple(out[k].shape) for k in
33
+ ('outlet', 'storage', 'process_fluxes', 'sink_total', 'mass_residual')})
34
+ print('Maximum mass residual:', error)
35
+ assert error < 1e-4
36
+ with tf.GradientTape() as tape:
37
+ y = model.simulate(water, drivers)['outlet']
38
+ objective = tf.reduce_mean(tf.square(y))
39
+ grad = tape.gradient(objective, model._variables())
40
+ assert any(g is not None for g in grad)
41
+ assert all(np.isfinite(g.numpy()).all() for g in grad if g is not None)
42
+ dataset = HydroDataset(rain, np.stack([rain, pet], axis=-1), {'streamflow': rain})
43
+ train = dataset.make_windows(warmup=5, score_length=10, stride=5)
44
+ model.fit(train, epochs=1, batch_size=2, verbose=0)
45
+ score = model.evaluate(train)
46
+ print('Training score:', score['loss'])
47
+ print('Installation check passed')
48
+
49
+
50
+ if __name__ == '__main__':
51
+ main()
@@ -0,0 +1,48 @@
1
+ # Physical water sources and losses
2
+
3
+ DeepDiscover distinguishes **boundary exchanges** from **training loss functions**. A boundary exchange changes the physical water balance and must be accounted for in the simulated system.
4
+
5
+ ## Sources
6
+
7
+ `water` supplies the external conserved inputs directly to DUs. Additional sources can be specified in a DU's `exchanges` list:
8
+
9
+ - `PrescribedSource(driver_index, scale=1)` adds a non-negative amount from a driver column.
10
+ - `FunctionalSource(my_layer)` adds a non-negative amount computed by a differentiable Keras layer.
11
+ - A custom `keras.layers.Layer` declaring `kind = "source"` can implement the same `call(drivers)` contract.
12
+
13
+ Applications include precipitation, prescribed recharge, irrigation, lateral inflow and water transferred into the modeled domain. A physical source cannot be silently included in both `water` and `exchanges`, as that would count the same water twice.
14
+
15
+ ## Losses
16
+
17
+ - `PrescribedSink(driver_index, scale=1)` requests an externally specified amount, such as pumping or managed extraction.
18
+ - `DemandLimitedSink(efficiency_net, demand_index, context_indices)` requests a loss bounded by demand and an efficiency between zero and one.
19
+ - `FunctionalSink(my_layer)` uses a custom layer returning a non-negative requested amount.
20
+ - A custom `keras.layers.Layer` with `kind = "sink"` implements `call(drivers, storage, available)`.
21
+
22
+ Loss requests from the same DU are combined. If demand exceeds water availability, each request is reduced by the same fraction. Simultaneous evaporation and groundwater pumping are therefore admissible and conserve water.
23
+
24
+ ## Example
25
+
26
+ ```python
27
+ from deepdiscover import (
28
+ CRCNet, DiscoverUnit, PrescribedSource, PrescribedSink,
29
+ DemandLimitedSink,
30
+ )
31
+
32
+ et_efficiency = CRCNet(gamma=[1, 1], x_ref=[10, 100], output="sigmoid")
33
+ du = DiscoverUnit(
34
+ process_nets=[CRCNet([1, 1], [10, 100])],
35
+ context_indices=[(0,)],
36
+ exchanges=[
37
+ PrescribedSource(driver_index=2), # lateral inflow
38
+ DemandLimitedSink(et_efficiency, demand_index=1, context_indices=[0]),
39
+ PrescribedSink(driver_index=3), # pumping
40
+ ],
41
+ )
42
+ ```
43
+
44
+ Here the driver columns represent precipitation information, PET, lateral inflow and pumping request. The user must decide which entries are conserved in `water`, and which are introduced as boundary sources. This choice determines the physical system boundary.
45
+
46
+ ## Requirements for custom exchanges
47
+
48
+ Sources and sinks return one non-negative `(B,1)` tensor in quantity-per-step units. Non-finite or negative values raise errors. Custom sinks return **requested losses**; the DU applies the available-water constraint. These exchanges may carry trainable Keras parameters. If they introduce water from a separate physical reservoir outside the graph, that reservoir is not included in the system storage balance.
@@ -0,0 +1,68 @@
1
+ # CAMELS-US data
2
+
3
+ The CAMELS-US example uses the forcing and observed streamflow files for basin `01013500`. Download the original data from the CAMELS-US provider. The files are not part of this repository.
4
+
5
+ Arrange the files as follows:
6
+
7
+ ```text
8
+ data/
9
+ camels/
10
+ 01013500/
11
+ 01013500_lump_cida_forcing_leap.txt
12
+ 01013500_streamflow_qc.txt
13
+ ```
14
+
15
+ ```python
16
+ from deepdiscover import load_camels_dataset
17
+
18
+ data = load_camels_dataset(
19
+ ".", "01013500", n_du=2,
20
+ start="1980-10-01", end="2010-09-30",
21
+ )
22
+ print(data.water.shape, data.drivers.shape)
23
+ ```
24
+
25
+ The data reader computes mean temperature, Hamon potential evapotranspiration and catchment-equivalent streamflow depth. Predictor order is precipitation, potential evapotranspiration and temperature. External precipitation enters DU 0 by default.
26
+
27
+ A value of `-999` or quality flag `M` in the CAMELS streamflow file indicates a missing observation. These values are excluded from calibration through the observation mask. Other negative discharge values cause an error. Missing precipitation or meteorological drivers must be handled explicitly before simulation.
28
+
29
+ The published case-study chronology uses 365 warm-up days from 1980-10-01, training from 1981-10-01 to 2000-09-30, validation from 2000-10-01 to 2005-09-30 and testing from 2005-10-01 to 2010-09-30. The notebook selects these dates explicitly.
30
+
31
+ Review the original CAMELS-US data documentation and quality flags before using alternative basins or periods. Cite the original data sources in scientific publications.
32
+
33
+ ## Running the notebook
34
+
35
+ For a short execution check:
36
+
37
+ ```bash
38
+ python run_examples.py --only 04_camels_us
39
+ ```
40
+
41
+ To measure training speed on the complete set of 19 windows:
42
+
43
+ ```bash
44
+ python run_examples.py --only 04_camels_us --camels-mode benchmark
45
+ ```
46
+
47
+ For full calibration:
48
+
49
+ ```bash
50
+ python run_examples.py --only 04_camels_us --camels-mode full
51
+ ```
52
+
53
+ The physical model in this example has two DUs and three release processes: one direct outlet pathway, one transfer to the second DU, and one downstream outlet pathway. The snow module separates precipitation into rain and snowmelt. These quantities and PET form the process drivers, in that order. The temperature series is used by the snow module.
54
+
55
+ Training data have shapes `water=(19,730,2)`, `drivers=(19,730,3)` and `streamflow=(19,730,1)`. Each window contains 365 unscored warm-up days and 365 scored days. The 19 complete scored blocks cover 6,935 of the 6,940 available training days; the final five days are not included in these nonoverlapping blocks. Full calibration fits all 19 windows together in one gradient update per epoch, validates every five epochs, and uses an upper limit of 400 epochs with early stopping. The notebook uses a learning rate of 0.01. The benchmark mode uses the same windows but only three epochs to report execution times.
56
+
57
+ The check and benchmark modes verify execution and measure timing. Their calibration scores should not be used in scientific analysis. The independent test period is evaluated only in full mode, after checkpoint selection on validation data. Do not use test results for model selection. For comparisons between models, report the random seeds, model parameters, calibration periods and evaluation conventions.
58
+
59
+
60
+ ## Verifying a saved checkpoint
61
+
62
+ After a full notebook run, the selected weights and validation history are saved. To check their consistency without fitting again, run:
63
+
64
+ ```bash
65
+ python verify_camels.py
66
+ ```
67
+
68
+ This procedure reconstructs the notebook architecture, loads the saved parameters, evaluates the validation period, and checks the water balance. The recorded validation objective and selected epoch must agree with the saved model. It does not evaluate the independent test period. The checkpoint format requires the same model configuration and a compatible DeepDiscover version.