soil_dataprovider_nl 1.0.0__tar.gz

soil_dataprovider_nl-1.0.0/LICENSE

@@ -0,0 +1,18 @@
+# Copyright 2026 Wageningen Environmental Research, Wageningen-UR
+
+# Licensed under the EUPL, Version 1.2 or – as soon they
+# will be approved by the European Commission - subsequent
+# versions of the EUPL (the "Licence");
+# You may not use this work except in compliance with the
+# Licence.
+# You may obtain a copy of the Licence at:
+
+# https://interoperable-europe.ec.europa.eu/collection/eupl/news/understanding-eupl-v12
+
+# Unless required by applicable law or agreed to in
+# writing, software distributed under the Licence is
+# distributed on an "AS IS" basis,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+# express or implied.
+# See the Licence for the specific language governing
+# permissions and limitations under the Licence. 

soil_dataprovider_nl-1.0.0/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: soil_dataprovider_nl
+Version: 1.0.0
+Summary: This package provides a set of data providers that can derive the soil parameters for running crop/soil models in PCSE based on the Dutch soil map.
+Author-email: Allard de Wit <allard.dewit@wur.nl>, Herman Berghuijs <herman.berghuijs@wur.nl>
+Maintainer-email: Allard de Wit <allard.dewit@wur.nl>, Herman Berghuijs <herman.berghuijs@wur.nl>
+License-Expression: EUPL-1.2
+Project-URL: Homepage, https://github.com/ajwdewit/soil_dataprovider_nl
+Project-URL: documentation, https://github.com/ajwdewit/soil_dataprovider_nl
+Project-URL: repository, https://github.com/ajwdewit/soil_dataprovider_nl
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pandas
+Requires-Dist: pyproj==3.8
+Requires-Dist: duckdb==1.5
+Provides-Extra: notebook
+Requires-Dist: matplotlib; extra == "notebook"
+Requires-Dist: ipyleaflet; extra == "notebook"
+Requires-Dist: pcse>=6.0.13; extra == "notebook"
+Dynamic: license-file
+
+#  Soil data providers for Dutch soils for PCSE
+
+---
+Allard de Wit  
+Wageningen Environmental Research  
+September 2026  
+---
+
+This package provides a set of data providers that can provide the soil parameters for the Netherlands as input
+for running crop/soil models in PCSE. It takes the input from the [Dutch soil map](https://app.pdok.nl/viewer/#x=160000.00&y=455000.00&z=3.0000&background=BRT-A%20standaard&layers=d9cc67ba-5491-4640-86ac-b8d392250270;soilarea;_;1)
+but the soil data itself is hosted in a [duckdb file on github](https://github.com/ajwdewit/collections/raw/refs/heads/main/BodemkaartNL/bofek_soil_nl.ddb).
+
+Getting soil information has always been a bit of a hassle for models in [PCSE](https://pcse.readthedocs.io). 
+While crop parameters are well managed through [YAML files on github](https://github.com/ajwdewit/WOFOST_crop_parameters), 
+soil parameters are more difficult as they are spatially variable and thus location dependent. Moreover, soil 
+maps and definitions are variable between countries making it hard to develop dataproviders for soils. 
+
+This soil data provider derives soil parameters from the Dutch soil map and derives soil hydraulic properties as well
+as soil mineralogic properties (bulk density, organic matter content) depending on the type of PCSE soil component used.
+Several data providers are available from this package:
+- `SoilDataProviderNL_CWB`
+- `SoilDataProviderNL_MLWB` *under development*
+- `SoilDataProviderNL_MLWB_SNOMIN` *under development*
+
+These dataproviders map to the different soil modules in pcse: the classic soil water balance (CWB), the multi-layered
+soil water balance (MLWB) and the multi-layered water balance plus the SNOMIN carbon/nitrogen model (MLWB_SNOMIN). 
+The correct soil dataprovider will be selected automatically based on the pcse model that you pass in to the call
+to `soil_dataprovider_nl.SoilDataProviderNL`, however they can also be imported manually from the package.
+
+# Setting up the package
+
+## Dependencies
+
+This package has been developed using python 3.10 and has dependencies on several other packages:
+- pandas
+- numpy
+- duckdb == 1.5
+- matplotlib
+- pyproj == 3.7
+
+
+## Installing
+
+The package can be pip-installed from PyPI: `pip install soil_dataprovider_nl` should be sufficient.
+
+Dependencies should be automatically downloaded and installed during installing the package, although the `pyproj` dependency
+can be problematic sometimes. In that case, install `pyproj` manually in a conda environment with the conda package
+manager.
+
+## Usage
+
+Using the soil dataprovider is easy:
+```python
+>> from soil_dataprovider_nl import SoilDataProviderNL
+>> from pcse.models import Wofost72_WLP_CWB
+>> soild = SoilDataProviderNL(Wofost72_WLP_CWB, xcoord=170342, ycoord=438503)
+>> print(soild)
+Soil properties for location at X/Y: 170342/438503 - lon/lat: 5.611/51.936
+Soil rootable depth estimated at 120 cm (from maximum soil profile depth)
+Soil profile characteristics:
+ thickness  pclay  psilt  psand      SMW    SMFCF      SM0
+        25   0.23   0.40   0.37 0.122168 0.387381 0.429530
+        35   0.23   0.40   0.37 0.142978 0.413028 0.472326
+        30   0.20   0.25   0.55 0.142978 0.413028 0.472326
+        30   0.04   0.07   0.89 0.040080 0.269736 0.387057
+Actual CWB parameter values:
+- SMW: 0.113 [-]
+- SMFCF: 0.372 [-]
+- SM0: 0.442 [-]
+- CRAIRC: 0.035 [-]
+- RDMSOL: 120.000 [cm]
+- SOPE: 8.436 [cm day-1]
+- KSUB: 8.436 [cm day-1]
+```
+
+The derived soildata can then be used in the usual procedure for running a model (in pseudo code):
+```python
+>> params = ParameterProvider(cropdata=cropd, soildata=soild, sitedata=sited)
+>> agro = get_agromanagement(...)
+>> wdp = get_weatherdata(...)
+>> wofost = Wofost72_WLP_CWB(params, wdp, agro)
+>> wofost.run_till_terminate()
+...
+```
+
+See for more elaborate examples the [notebook on the repository](https://github.com/ajwdewit/soil_dataprovider_nl).

soil_dataprovider_nl-1.0.0/README.md

@@ -0,0 +1,86 @@
+#  Soil data providers for Dutch soils for PCSE
+
+---
+Allard de Wit  
+Wageningen Environmental Research  
+September 2026  
+---
+
+This package provides a set of data providers that can provide the soil parameters for the Netherlands as input
+for running crop/soil models in PCSE. It takes the input from the [Dutch soil map](https://app.pdok.nl/viewer/#x=160000.00&y=455000.00&z=3.0000&background=BRT-A%20standaard&layers=d9cc67ba-5491-4640-86ac-b8d392250270;soilarea;_;1)
+but the soil data itself is hosted in a [duckdb file on github](https://github.com/ajwdewit/collections/raw/refs/heads/main/BodemkaartNL/bofek_soil_nl.ddb).
+
+Getting soil information has always been a bit of a hassle for models in [PCSE](https://pcse.readthedocs.io). 
+While crop parameters are well managed through [YAML files on github](https://github.com/ajwdewit/WOFOST_crop_parameters), 
+soil parameters are more difficult as they are spatially variable and thus location dependent. Moreover, soil 
+maps and definitions are variable between countries making it hard to develop dataproviders for soils. 
+
+This soil data provider derives soil parameters from the Dutch soil map and derives soil hydraulic properties as well
+as soil mineralogic properties (bulk density, organic matter content) depending on the type of PCSE soil component used.
+Several data providers are available from this package:
+- `SoilDataProviderNL_CWB`
+- `SoilDataProviderNL_MLWB` *under development*
+- `SoilDataProviderNL_MLWB_SNOMIN` *under development*
+
+These dataproviders map to the different soil modules in pcse: the classic soil water balance (CWB), the multi-layered
+soil water balance (MLWB) and the multi-layered water balance plus the SNOMIN carbon/nitrogen model (MLWB_SNOMIN). 
+The correct soil dataprovider will be selected automatically based on the pcse model that you pass in to the call
+to `soil_dataprovider_nl.SoilDataProviderNL`, however they can also be imported manually from the package.
+
+# Setting up the package
+
+## Dependencies
+
+This package has been developed using python 3.10 and has dependencies on several other packages:
+- pandas
+- numpy
+- duckdb == 1.5
+- matplotlib
+- pyproj == 3.7
+
+
+## Installing
+
+The package can be pip-installed from PyPI: `pip install soil_dataprovider_nl` should be sufficient.
+
+Dependencies should be automatically downloaded and installed during installing the package, although the `pyproj` dependency
+can be problematic sometimes. In that case, install `pyproj` manually in a conda environment with the conda package
+manager.
+
+## Usage
+
+Using the soil dataprovider is easy:
+```python
+>> from soil_dataprovider_nl import SoilDataProviderNL
+>> from pcse.models import Wofost72_WLP_CWB
+>> soild = SoilDataProviderNL(Wofost72_WLP_CWB, xcoord=170342, ycoord=438503)
+>> print(soild)
+Soil properties for location at X/Y: 170342/438503 - lon/lat: 5.611/51.936
+Soil rootable depth estimated at 120 cm (from maximum soil profile depth)
+Soil profile characteristics:
+ thickness  pclay  psilt  psand      SMW    SMFCF      SM0
+        25   0.23   0.40   0.37 0.122168 0.387381 0.429530
+        35   0.23   0.40   0.37 0.142978 0.413028 0.472326
+        30   0.20   0.25   0.55 0.142978 0.413028 0.472326
+        30   0.04   0.07   0.89 0.040080 0.269736 0.387057
+Actual CWB parameter values:
+- SMW: 0.113 [-]
+- SMFCF: 0.372 [-]
+- SM0: 0.442 [-]
+- CRAIRC: 0.035 [-]
+- RDMSOL: 120.000 [cm]
+- SOPE: 8.436 [cm day-1]
+- KSUB: 8.436 [cm day-1]
+```
+
+The derived soildata can then be used in the usual procedure for running a model (in pseudo code):
+```python
+>> params = ParameterProvider(cropdata=cropd, soildata=soild, sitedata=sited)
+>> agro = get_agromanagement(...)
+>> wdp = get_weatherdata(...)
+>> wofost = Wofost72_WLP_CWB(params, wdp, agro)
+>> wofost.run_till_terminate()
+...
+```
+
+See for more elaborate examples the [notebook on the repository](https://github.com/ajwdewit/soil_dataprovider_nl).

soil_dataprovider_nl-1.0.0/pyproject.toml

@@ -0,0 +1,42 @@
+[build-system]
+requires = [
+    "setuptools>=42",
+    "wheel"
+]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "soil_dataprovider_nl"
+version = "1.0.0"
+authors = [
+  { name="Allard de Wit", email="allard.dewit@wur.nl" },
+  { name="Herman Berghuijs", email="herman.berghuijs@wur.nl" },
+]
+maintainers = [
+  { name="Allard de Wit", email="allard.dewit@wur.nl" },
+  { name="Herman Berghuijs", email="herman.berghuijs@wur.nl" },
+]
+description = "This package provides a set of data providers that can derive the soil parameters for running crop/soil models in PCSE based on the Dutch soil map."
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "pandas",
+    "pyproj==3.8",
+    "duckdb == 1.5",
+]
+license = "EUPL-1.2"
+license-files = ["LICENSE"]
+
+[project.optional-dependencies]
+notebook = ["matplotlib", "ipyleaflet", "pcse>=6.0.13"]
+
+[project.urls]
+Homepage = "https://github.com/ajwdewit/soil_dataprovider_nl"
+documentation = "https://github.com/ajwdewit/soil_dataprovider_nl"
+repository = "https://github.com/ajwdewit/soil_dataprovider_nl"
+
+[[tool.uv.index]]
+name = "testpypi"
+url = "https://test.pypi.org/simple/"
+publish-url = "https://test.pypi.org/legacy/"
+explicit = true

soil_dataprovider_nl-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

soil_dataprovider_nl-1.0.0/soil_dataprovider_nl/__init__.py

@@ -0,0 +1,32 @@
+# Copyright (c) 2026 Wageningen Environmental Research, Wageningen-UR
+# Allard de Wit (allard.dewit@wur.nl), April 2026
+from .soil_dataprovider_nl import (
+    SoilDataProviderNL_CWB,
+    SoilDataProviderNL_MLWB,
+    SoilDataProviderNL_MLWB_SNOMIN,
+)
+
+__version__ = "1.0.0"
+
+
+class SoilDataProviderNL:
+    """Convenience class that selected the right soil data provider based on the model annotations.
+    """
+
+    def __new__(cls, model, *args, **kwargs):
+
+        if model.__nitrogenbalance__ == "SNOMIN":
+            final_cls = SoilDataProviderNL_MLWB_SNOMIN
+        elif model.__waterbalance__ == "MLWB":
+            final_cls = SoilDataProviderNL_MLWB
+        elif model.__waterbalance__ == "CWB":
+            final_cls = SoilDataProviderNL_CWB
+        else:
+            raise RuntimeError("soil module not recognized")
+
+        # Here we instantiate the selected class with __new__()
+        instance = final_cls.__new__(final_cls, *args, **kwargs)
+        # Here we initialize the instance with an explicit call to __init__()
+        instance.__init__(*args, **kwargs)
+        return instance
+

soil_dataprovider_nl-1.0.0/soil_dataprovider_nl/coords.py

@@ -0,0 +1,64 @@
+# Copyright (c) 2026 Wageningen Environmental Research, Wageningen-UR
+# Allard de Wit (allard.dewit@wur.nl), April 2026
+import pyproj
+
+
+class CoordinateStore:
+    """This class stores the coordinates and provides several function
+    - It determines if provided coordinates are lat/lon or Dutch RD-stelsel coordinates.
+    - Converts RD to lat/lon and vice versa
+    - It checks if the coordinates are within the boundinng box for NL
+
+    self.xcoord, self.ycoord provide the coordinates Dutch RD-stelsel coordinates
+    self.lon, self.lat provide the coordinates in longitude/latitude
+    """
+    rd_stelsel = "epsg:7415"
+
+    def __init__(self, x, y):
+        self._conv = pyproj.Proj(self.rd_stelsel)
+
+        if self._is_valid_xcoord_rd(x) and self._is_valid_ycoord_rd(y):
+            self.xcoord = x
+            self.ycoord = y
+            self.lon, self.lat = self.from_RD(x, y)
+        elif self._is_valid_longitude(x) and self._is_valid_latitude(y):
+            self.lon = x
+            self.lat = y
+            self.xcoord, self.ycoord = self.from_lonlat(x, y)
+        else:
+            raise ValueError("Coordinates out of bounding box. Did you swap longitude/latitude?")
+
+    def from_RD(self, x, y):
+        return self._conv(x, y, inverse=True)
+
+    def from_lonlat(self, lon, lat):
+        return self._conv(lon, lat)
+
+    @staticmethod
+    def _is_valid_xcoord_rd(xcoord):
+        rdx_min = 7000
+        rdx_max = 289000
+        return rdx_min < xcoord < rdx_max
+
+    @staticmethod
+    def _is_valid_ycoord_rd(ycoord):
+        rdy_min = 289000
+        rdy_max = 629000
+        return  rdy_min < ycoord < rdy_max
+
+    @staticmethod
+    def _is_valid_longitude(lon):
+        lon_min = 3.2
+        lon_max = 7.3
+        return lon_min < lon < lon_max
+
+    @staticmethod
+    def _is_valid_latitude(lat):
+        lat_min = 50.7
+        lat_max = 53.6
+        return  lat_min < lat < lat_max
+
+    def __str__(self):
+        msg = f"RD coordinates for current location: {self.xcoord}, {self.ycoord}\n"
+        msg += f"Lon/Lat coordinates for current location: {self.lon}, {self.lat}\n"
+        return msg

soil_dataprovider_nl-1.0.0/soil_dataprovider_nl/mualemvangenuchten.py

@@ -0,0 +1,98 @@
+# Copyright (c) 2026 Wageningen Environmental Research, Wageningen-UR
+# Allard de Wit (allard.dewit@wur.nl), April 2026
+# Mualem van Genuchten functions based on R code provided by Martin Mulder
+from dataclasses import dataclass
+
+# #' Get water content based on pressure head
+# #'
+# #' @param H pressure head [cm].
+# #' @param WCR residual water content [cm3 cm-3].
+# #' @param WCS saturated water content [cm3 cm-3].
+# #' @param ALPHA curve shape parameter [-].
+# #' @param NPAR curve shape parameter [-].
+# #' @return return water content [cm3 cm-3].
+# #' @keywords internal
+# get_wc_MvG  <- function(H, WCR, WCS, ALPHA, NPAR) {
+#
+#   # --- main part of procedure ---
+#
+#   m <- 1 - 1 / NPAR
+#   wc <- WCR + (WCS - WCR) / ((1 + (ALPHA * H)^NPAR)^m)
+#
+#   # --- return of procedure ---
+#
+#   return (wc)
+
+@dataclass()
+class MualemvanGenuchten:
+    wcr: float
+    wcs:float
+    alpha: float
+    npar: float
+    lamda: float
+    ksat: float
+
+
+def get_water_content_from_MvG(H, p):
+    """return volumetric water content for given pressure head H [cm] and MvG parameters p.
+
+    :param H: pressure head [cm].
+    :param p: Object representing MvG parameters having attributes:
+        - wcr: residual water content [cm3 cm-3].
+        - wcs: saturated water content [cm3 cm-3].
+        - alpha: curve shape parameter [-].
+        - npar: curve shape parameter [-].
+    :return: (array of) water content values for given MvG parameters
+    """
+    m = 1.0 - 1.0 / p.npar
+    wc = p.wcr + (p.wcs - p.wcr) / ((1 + (p.alpha * H) ** p.npar) ** m)
+
+    return wc
+
+
+
+#' Get conductivity based on pressure head
+#'
+#' @param H pressure head [cm]
+#' @param ALPHA curve shape parameter [-]
+#' @param NPAR curve shape parameter [-]
+#' @param LAMBDA exponent in hydraulic conductivity function [-]
+#' @param KSAT hydraulic conductivity of saturated soil [cm d-1]
+#' @return return conductivity [cm d-1]
+#' @keywords internal
+# get_cond_MvG  <- function(H, ALPHA, NPAR, LAMBDA, KSAT) {
+#
+#   # --- main part of procedure ---
+#
+#   m <- 1 - 1 / NPAR
+#   ah <- ALPHA * H
+#   h1 <- (1 + ah^NPAR)^m
+#   h2 <- ah**(NPAR - 1)
+#   denom <- (1 + ah^NPAR)^(m*(LAMBDA + 2))
+#   cond <- KSAT * (h1 - h2)^2 / denom
+#
+#   # --- return of procedure ---
+#
+#   return (cond)
+# }
+
+def get_conductivity_from_MvG(H, p):
+    """Get conductivity from pressure head H [cm] and MvG parameters p.
+
+    :param H: pressure head [cm]
+    :param p: Object representing MvG parameters having attributes:
+        - alpha: curve shape parameter [-]
+        - npar: curve shape parameter [-]
+        - lamda: exponent in hydraulic conductivity function [-]
+        - ksat: hydraulic conductivity of saturated soil [cm d-1]
+    :return: conductivity [cm d-1]
+    """
+
+    m = 1 - 1 / p.npar
+    ah = p.alpha * H
+    h1 = (1 + ah ** p.npar) ** m
+    h2 = ah ** (p.npar - 1)
+    denom = (1 + ah ** p.npar) ** (m * (p.lamda + 2))
+    cond = p.ksat * (h1 - h2)**2 / denom
+
+    return cond

soil_dataprovider_nl-1.0.0/soil_dataprovider_nl/soil_dataprovider_nl.py

@@ -0,0 +1,397 @@
+# Copyright (c) 2026 Wageningen Environmental Research, Wageningen-UR
+# Allard de Wit (allard.dewit@wur.nl), April 2026
+import hashlib
+import tempfile
+import urllib.request
+from pathlib import Path
+from types import SimpleNamespace
+
+import pandas as pd
+
+pd.options.mode.chained_assignment = None
+import duckdb
+import numpy as np
+
+from .coords import CoordinateStore
+from .mualemvangenuchten import (
+    MualemvanGenuchten,
+    get_conductivity_from_MvG,
+    get_water_content_from_MvG,
+)
+
+non_soil_codes = {99980, 99990, 99991}
+
+this_dir = Path(__file__).parent
+top_dir = this_dir.parent.absolute()
+tmp_dir = Path(tempfile.gettempdir())
+
+
+def plot_pF_curves(pF_results, fname_figure=None):
+    """Plots the pF curves based on input  from the function `compute_pF_curves`
+
+    :param pF_results: the pF results from the function `compute_pF_curves`
+    :param fname_figure: optionally the file name where the figure will be written. Output format
+        will be derived from the filename extension png|jpg|etc.
+    :return: a matplotlib figure and axes
+    """
+
+    try:
+        import matplotlib.pyplot as plt
+    except ImportError:
+        msg = "Install matplotlib for generating figures"
+        print(msg)
+        return None
+
+    fig, axes = plt.subplots(ncols=2, figsize=(10, 5))
+    for layer in pF_results:
+        label = f"layer {layer.layer_top}-{layer.layer_bottom}"
+        axes[0].plot(layer.pF_range, layer.pF_WC, label=label)
+        axes[1].plot(layer.pF_range, layer.pF_Cond, label=label)
+
+    axes[0].legend()
+    axes[0].set_xlabel("pF value")
+    axes[0].set_ylabel("Soil moisture content [-]")
+    axes[1].set_xlabel("pF value")
+    axes[1].set_ylabel("Soil conductivity [cm/day]")
+    fig.suptitle("PF curves")
+    if fname_figure is not None:
+        fig.savefig(fname_figure)
+
+    return fig, axes
+
+
+def compute_pF_curves(profile_characteristics):
+    """Computes the pF curves for water retention and conductivey for all layeres in the profile
+
+    :param profile_characteristics: the profile characteristics from BOFEK
+    :return: a list the dicts containing the pF curves and additional info for each layer.
+    """
+
+    pF_range = np.arange(-1.0, 7.1, 0.1)
+    H = 10 ** pF_range
+    pF_results = []
+    for row in profile_characteristics.itertuples():
+        p_mvg = MualemvanGenuchten(wcr=row.ores, wcs=row.osat, alpha=row.alfa, npar=row.npar,
+                                   lamda=row.lexp, ksat=row.ksatfit)
+        pF_WC = get_water_content_from_MvG(H, p_mvg)
+        pF_Cond = get_conductivity_from_MvG(H, p_mvg)
+        pF_results.append(SimpleNamespace(idlayer=row.idlayer, pF_WC=pF_WC,pF_Cond=pF_Cond,
+                                          layer_top=row.layer_top, layer_bottom=row.layer_bottom,
+                                          pF_range=pF_range))
+
+    return pF_results
+
+
+def find_profile_characteristics(DBconn, profile_code):
+    """Reads the soil profile characteristics from the database
+
+    :param profile_code: the profile code
+    :return: a dataframe with soil profile characteristics
+    """
+    sql = """SELECT t1.*, t2.* 
+             FROM 
+                 soildb.soil_profiles t1 
+               INNER JOIN 
+                 soildb.soil_physical_description t2 ON t1.soil_physical_code=t2.soil_physical_code
+             WHERE
+               t1.profile_code = ?
+             ORDER BY
+               t1.idlayer
+        """
+    df = DBconn.execute(sql, (profile_code,)).df()
+    if len(df) == 0:
+        msg = f"No valid soil profile description found for this soil profile code: {profile_code})!"
+        raise RuntimeError(msg)
+
+    df["thickness"] = df.layer_bottom - df.layer_top
+
+    return df
+
+
+def find_soil_profile(DBconn, coords):
+    """Find the soil profile code based on given coordinates
+
+    :param DBconn: The database connection
+    :param coords: the coordinates in Dutch RD system.
+    :return: a profile code for the soil (integer)
+    """
+    sql = """SELECT profile_code FROM soildb.bofek_soil_nl t1
+             WHERE ST_intersects(t1.geom, ST_Point(?, ?))
+          """
+
+    cursor = DBconn.execute(sql, (coords.xcoord, coords.ycoord))
+    row = cursor.fetchone()
+    if not row:
+        msg = (f"No soil profile found for this location: (X:{coords.xcoord:.0f}, Y:{coords.ycoord:.0f})! "
+               f"Is this location on land?")
+        raise RuntimeError(msg)
+
+    profile_code = row[0]
+    if profile_code in non_soil_codes:
+        msg = (f"Not a valid soil profile found for this location: (X:{coords.xcoord:.0f}, Y:{coords.ycoord:.0f})! "
+               f"Probably an urban area, land fill or other location with no soil description!")
+        raise RuntimeError(msg)
+
+    return profile_code
+
+
+class SoilBDconnector:
+    """Takes care of opening the connection to the duckdb file on github and manages the cached version of that file.
+    """
+    _bofek_soil_source = "https://github.com/ajwdewit/collections/raw/refs/heads/main/BodemkaartNL/bofek_soil_nl.ddb"
+    _bofek_soil_source_shasum = "https://raw.githubusercontent.com/ajwdewit/collections/main/BodemkaartNL/bofek_soil_nl.shasum"
+    _bofek_soil_cache = tmp_dir / "bofek_soil_nl.ddb"
+    _upstream_shasum = None
+
+    def __init__(self):
+        connected = self._has_connection()
+        if not connected:
+            msg = "No internet connection to retrieve BOFEK soildb and compare DB checksum!"
+            raise RuntimeError(msg)
+
+        if not self._bofek_soil_cache.exists():
+            print("Downloading Soil DB (~135 Mb)...")
+            urllib.request.urlretrieve(self._bofek_soil_source, self._bofek_soil_cache)
+        else:
+            local_shasum = self._compute_cache_sha1()
+            if local_shasum != self._upstream_shasum:
+                print("Updating Soil DB (~135 Mb)...")
+                urllib.request.urlretrieve(self._bofek_soil_source, self._bofek_soil_cache)
+            else:
+                print(f"Found soil DB at {self._bofek_soil_cache}")
+
+        self.connection = None
+
+    def __enter__(self):
+
+        sql1 = "install spatial; load spatial; install httpfs; load httpfs;"
+        sql2 = f"attach '{self._bofek_soil_cache}' as soildb (READONLY)"
+        self.connection = duckdb.connect()
+        self.connection.sql(sql1)
+        self.connection.sql(sql2)
+        return self.connection
+
+    def __exit__(self, exc_type, exc_value, exc_traceback):
+
+        self.connection.close()
+
+    def _has_connection(self):
+        """Checks the connection by trying to download the database SHA1 checksum from github.
+        """
+        try:
+            self._upstream_shasum =  self._get_upstream_sha1()
+            return True
+        except urllib.error.URLError:
+            return False
+
+    def _compute_cache_sha1(self):
+        """Computes a SHA1 hash on the local cached DB file to check if the upstream DB has changed.
+        """
+        fname_hash = self._bofek_soil_cache.with_suffix(".sha1")
+        if fname_hash.exists():
+            return open(fname_hash, "r").read()
+
+        m = hashlib.sha1()
+        blocksize = 2 ** 20
+        with open(self._bofek_soil_cache, "rb") as fp:
+            while True:
+                buf = fp.read(blocksize)
+                if not buf:
+                    break
+                m.update(buf)
+
+        with open(fname_hash, "w") as fp:
+            fp.write(m.hexdigest())
+
+        return m.hexdigest()
+
+    def _get_upstream_sha1(self):
+        """Downloads the SHA1 has from the database file on github.
+        """
+        request_url = urllib.request.urlopen(self._bofek_soil_source_shasum)
+        r = request_url.read()
+        hash_value = r.decode().split()[0]
+        return hash_value
+
+
+class SoilDataProviderNL_CWB(dict):
+    """A SoilDataProvider that retrieves soil parameters from the Dutch BOFEK soil database for use
+    with the PCSE classic waterbalance.
+
+    :param xcoord: the X coordinate. Either in Dutch RD coordinates or as longitude
+    :param ycoord: the Y coordinates. Either in Dutch RD coordinates or as latitude
+    :param max_root_depth: user defined rootable depth in cm, otherwise the whole soil is assumed rootable.
+
+    Since the classic water balance uses a single soil layer, the soil parameters for each layer
+    have to be aggregated. The following assumptions have been made:
+    - SMW is calculated from a layer-weighted average of the wilting point, the latter is assumed at pF=4.2
+    - SMFCF is computed by calculating the water holding capacity for all layers. The latter is defined as the amount
+      of volume between wilting point (SMW) and field capacity (pF=2) for each layer. The SMFCF is than calculated
+      as the value with an equivalent water holding capacity given the soil rootable depth (RDMSOL).
+    - SM0 is computed as the value required to store an equivalent volume of water given the rootable depth.
+    - SOPE/KSUB are computed as the conductivity of the bottom layer at pF=1. There is no physical basis for this
+      but since the water balance starts draining water when the water content is above field capacity we just
+      take pF=1 as representative for that proces.
+    - RDMSOL is derived from the maximum soil depth or from the user-defined max_root_depth. The smallest of the
+      two values is taken.
+    """
+
+    param_units = {"SMW": "[-]",
+                   "SMFCF": "[-]",
+                   "SM0": "[-]",
+                   "CRAIRC": "[-]",
+                   "RDMSOL": "[cm]",
+                   "SOPE": "[cm day-1]",
+                   "KSUB": "[cm day-1]",
+                   }
+
+    def __init__(self, *, xcoord=None, ycoord=None, max_root_depth=1E6):
+        super().__init__()
+
+        if max_root_depth < 0:
+            raise RuntimeError("max_root_depth must be positive")
+        if max_root_depth < 20:
+            raise RuntimeError("max_root_depth must at least be than 20 [cm]")
+
+        self.crds = CoordinateStore(xcoord, ycoord)
+        with SoilBDconnector() as DBconn:
+            self.soil_profile = find_soil_profile(DBconn, self.crds)
+            profile_characteristics = find_profile_characteristics(DBconn, self.soil_profile)
+
+        self.profile_characteristics = self._add_water_content_at_referencepoints(profile_characteristics)
+        self.rootable_depth, self._root_depth_limit_forced = self._determine_rootable_depth(max_root_depth)
+        self._compute_CWB_volumetric_parameters()
+        self._compute_CWB_conductivity_parameters()
+
+    def _compute_CWB_volumetric_parameters(self):
+        """Computes the soil volumetric parameters for the WOFOST classic waterbalance as layer-weighted values
+
+        :return: a dict with relevant parameters
+        """
+        df = self.profile_characteristics
+
+        # Determine which layers are rooted
+        df["is_rooted"] = df.layer_top < self.rootable_depth
+        if not df.is_rooted.all():
+            df = df[df.is_rooted]
+        df.loc[df.index[-1], 'layer_bottom'] = self.rootable_depth
+
+        # Recompute layer thickness as bottom layer may be bounded by rootable depth.
+        df["thickness"] = df.layer_bottom - df.layer_top
+
+        # wilting point as layer-weighted value
+        SMW = (df.SMW * df.thickness).sum() / df.thickness.sum()
+        # compute total water holding capacity (AWC) for all layers
+        AWC = ((df.SMFCF - df.SMW) * df.thickness).sum()
+        # Recomputed field capacity as required to store AWC above wilting point SMW
+        SMFCF = SMW + AWC/self.rootable_depth
+        # pore space up till porosity (AWC0)
+        AWC0 = ((df.SM0 - df.SMW) * df.thickness).sum()
+        # Recompute soil porosity (SM0) as required to store AWC0 above field capacity
+        SM0 = SMW + AWC0/self.rootable_depth
+        # Critical air content as halfway between SMFCF and SM0
+        CRAIRC = (SM0 - SMFCF) * 0.5
+
+        self.update(dict(SMW=SMW, SMFCF=SMFCF, SM0=SM0, CRAIRC=CRAIRC, RDMSOL=self.rootable_depth))
+
+    def _compute_CWB_conductivity_parameters(self):
+        """Computes the soil conductivity parameters for the WOFOST classic waterbalance
+
+        There are three parameters in WOFOST CWB that have to be estimated:
+        - SOPE : maximum percolation rate root zone[cm day-1]
+        - KSUB : maximum percolation rate subsoil [cm day-1]
+
+        It is unclear how this parameters have to be estimated physically. In practice, they
+        were probably used as a calibration parameter in order to limit excessive drainage.
+        In the classic water balance drainage occurs only when soil moisture is above
+        field capacity (pF=2.0), therefore we estimate both parameters as the conducitivity
+        at pF=1.0. Moreover, since drainage happens mostly in the lower soil layers, we
+        use the MvG parameters of the bottom layer to estimate the conductivity.
+
+        :return: a dict with relevant parameters
+        """
+        bottom_layer = self.profile_characteristics.iloc[-1]
+        p_mvg = MualemvanGenuchten(wcr=bottom_layer.ores, wcs=bottom_layer.osat, alpha=bottom_layer.alfa,
+                                   npar=bottom_layer.npar, lamda=bottom_layer.lexp, ksat=bottom_layer.ksatfit)
+        pF = 1.0
+        H = 10**pF
+        cond = get_conductivity_from_MvG(H, p_mvg)
+        self.update(dict(SOPE=cond, KSUB=cond))
+
+    def _add_water_content_at_referencepoints(self, profile_characteristics):
+        """Computes the water content at saturation, field capacity and wilting point based on the profile
+        characteristics and the Mualem van Genugten parameters.
+
+        :param profile_characteristics: a dataframe with soil profile characteristics
+        :return: The updated dataframe with water content at reference points added.
+        """
+
+        ref_point_names = ["SM0", "SMFCF", "SMW"]
+        ref_point_pF = np.array([-1, 2, 4.2])
+        H = 10**ref_point_pF
+        layer_ref_points = []
+        for layer in profile_characteristics.itertuples():
+            p_mvg = MualemvanGenuchten(wcr=layer.ores, wcs=layer.osat, alpha=layer.alfa, npar=layer.npar,
+                                       lamda=layer.lexp, ksat=layer.ksatfit)
+            water_content = get_water_content_from_MvG(H, p_mvg)
+            layer_ref_points.append(dict(zip(ref_point_names, water_content)))
+        df_water_content = pd.DataFrame.from_dict(layer_ref_points)
+        profile_characteristics = pd.concat([profile_characteristics, df_water_content], axis=1)
+
+        return profile_characteristics
+
+    def _determine_rootable_depth(self, max_rootable_depth):
+        """Determines the rootable depth as the minimum of the sum of the layer thickness and the user defined
+        max_rootable_depth parameter.
+
+        :param max_rootable_depth: the user-defined maximum rootable depth.
+        :return: the rootable depth
+        """
+        soil_rootable_depth = self.profile_characteristics.thickness.sum()
+        rootable_depth = min(max_rootable_depth, soil_rootable_depth)
+        if rootable_depth < soil_rootable_depth:
+            return rootable_depth, True
+        else:
+            return rootable_depth, False
+
+    def __str__(self):
+        msg = (f"Soil properties for location at X/Y: {self.crds.xcoord:.0f}/{self.crds.ycoord:.0f} - "
+               f"lon/lat: {self.crds.lon:.3f}/{self.crds.lat:.3f}\n")
+        if self._root_depth_limit_forced:
+            msg += f"Soil rootable depth estimated at {self.rootable_depth} cm (forced by `max_root_depth` parameter)\n"
+        else:
+            msg += f"Soil rootable depth estimated at {self.rootable_depth} cm (from maximum soil profile depth)\n"
+        msg += "Soil profile characteristics:\n"
+        s = self.profile_characteristics.to_string(index=False, columns=["thickness", "pclay", "psilt", "psand", "SMW",
+                                                                         "SMFCF", "SM0"],
+                                                   max_rows=5)
+        msg += (s + "\n")
+        msg += "Actual CWB parameter values:\n"
+        for name, value in self.items():
+            unit = self.param_units[name]
+            msg += f"- {name}: {value:.3f} {unit}\n"
+        return msg
+
+
+class SoilDataProviderNL_MLWB(dict):
+    """A SoilDataProvider that retrieves soil parameters from the Dutch BOFEK soil database for use
+    with the PCSE multi-layered waterbalance.
+    """
+    non_soil_codes = {99980, 99990, 99991}
+
+    def __init__(self, *, xcoord=None, ycoord=None, max_root_depth=1E6, cache_soildb=False):
+        super().__init__()
+
+        raise NotImplementedError("Soil dataprovider for the multi-layer waterbalance not yet implemented.")
+
+
+class SoilDataProviderNL_MLWB_SNOMIN(dict):
+    """A SoilDataProvider that retrieves soil parameters from the Dutch BOFEK soil database for use
+    with the PCSE multi-layered waterbalance with SNOMIN C/N soil model.
+    """
+    non_soil_codes = {99980, 99990, 99991}
+
+    def __init__(self, *, xcoord=None, ycoord=None, max_root_depth=1E6, cache_soildb=False):
+        super().__init__()
+
+        raise NotImplementedError("Soil dataprovider for the multi-layer waterbalance and SNOMIN not yet implemented.")

soil_dataprovider_nl-1.0.0/soil_dataprovider_nl.egg-info/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: soil_dataprovider_nl
+Version: 1.0.0
+Summary: This package provides a set of data providers that can derive the soil parameters for running crop/soil models in PCSE based on the Dutch soil map.
+Author-email: Allard de Wit <allard.dewit@wur.nl>, Herman Berghuijs <herman.berghuijs@wur.nl>
+Maintainer-email: Allard de Wit <allard.dewit@wur.nl>, Herman Berghuijs <herman.berghuijs@wur.nl>
+License-Expression: EUPL-1.2
+Project-URL: Homepage, https://github.com/ajwdewit/soil_dataprovider_nl
+Project-URL: documentation, https://github.com/ajwdewit/soil_dataprovider_nl
+Project-URL: repository, https://github.com/ajwdewit/soil_dataprovider_nl
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pandas
+Requires-Dist: pyproj==3.8
+Requires-Dist: duckdb==1.5
+Provides-Extra: notebook
+Requires-Dist: matplotlib; extra == "notebook"
+Requires-Dist: ipyleaflet; extra == "notebook"
+Requires-Dist: pcse>=6.0.13; extra == "notebook"
+Dynamic: license-file
+
+#  Soil data providers for Dutch soils for PCSE
+
+---
+Allard de Wit  
+Wageningen Environmental Research  
+September 2026  
+---
+
+This package provides a set of data providers that can provide the soil parameters for the Netherlands as input
+for running crop/soil models in PCSE. It takes the input from the [Dutch soil map](https://app.pdok.nl/viewer/#x=160000.00&y=455000.00&z=3.0000&background=BRT-A%20standaard&layers=d9cc67ba-5491-4640-86ac-b8d392250270;soilarea;_;1)
+but the soil data itself is hosted in a [duckdb file on github](https://github.com/ajwdewit/collections/raw/refs/heads/main/BodemkaartNL/bofek_soil_nl.ddb).
+
+Getting soil information has always been a bit of a hassle for models in [PCSE](https://pcse.readthedocs.io). 
+While crop parameters are well managed through [YAML files on github](https://github.com/ajwdewit/WOFOST_crop_parameters), 
+soil parameters are more difficult as they are spatially variable and thus location dependent. Moreover, soil 
+maps and definitions are variable between countries making it hard to develop dataproviders for soils. 
+
+This soil data provider derives soil parameters from the Dutch soil map and derives soil hydraulic properties as well
+as soil mineralogic properties (bulk density, organic matter content) depending on the type of PCSE soil component used.
+Several data providers are available from this package:
+- `SoilDataProviderNL_CWB`
+- `SoilDataProviderNL_MLWB` *under development*
+- `SoilDataProviderNL_MLWB_SNOMIN` *under development*
+
+These dataproviders map to the different soil modules in pcse: the classic soil water balance (CWB), the multi-layered
+soil water balance (MLWB) and the multi-layered water balance plus the SNOMIN carbon/nitrogen model (MLWB_SNOMIN). 
+The correct soil dataprovider will be selected automatically based on the pcse model that you pass in to the call
+to `soil_dataprovider_nl.SoilDataProviderNL`, however they can also be imported manually from the package.
+
+# Setting up the package
+
+## Dependencies
+
+This package has been developed using python 3.10 and has dependencies on several other packages:
+- pandas
+- numpy
+- duckdb == 1.5
+- matplotlib
+- pyproj == 3.7
+
+
+## Installing
+
+The package can be pip-installed from PyPI: `pip install soil_dataprovider_nl` should be sufficient.
+
+Dependencies should be automatically downloaded and installed during installing the package, although the `pyproj` dependency
+can be problematic sometimes. In that case, install `pyproj` manually in a conda environment with the conda package
+manager.
+
+## Usage
+
+Using the soil dataprovider is easy:
+```python
+>> from soil_dataprovider_nl import SoilDataProviderNL
+>> from pcse.models import Wofost72_WLP_CWB
+>> soild = SoilDataProviderNL(Wofost72_WLP_CWB, xcoord=170342, ycoord=438503)
+>> print(soild)
+Soil properties for location at X/Y: 170342/438503 - lon/lat: 5.611/51.936
+Soil rootable depth estimated at 120 cm (from maximum soil profile depth)
+Soil profile characteristics:
+ thickness  pclay  psilt  psand      SMW    SMFCF      SM0
+        25   0.23   0.40   0.37 0.122168 0.387381 0.429530
+        35   0.23   0.40   0.37 0.142978 0.413028 0.472326
+        30   0.20   0.25   0.55 0.142978 0.413028 0.472326
+        30   0.04   0.07   0.89 0.040080 0.269736 0.387057
+Actual CWB parameter values:
+- SMW: 0.113 [-]
+- SMFCF: 0.372 [-]
+- SM0: 0.442 [-]
+- CRAIRC: 0.035 [-]
+- RDMSOL: 120.000 [cm]
+- SOPE: 8.436 [cm day-1]
+- KSUB: 8.436 [cm day-1]
+```
+
+The derived soildata can then be used in the usual procedure for running a model (in pseudo code):
+```python
+>> params = ParameterProvider(cropdata=cropd, soildata=soild, sitedata=sited)
+>> agro = get_agromanagement(...)
+>> wdp = get_weatherdata(...)
+>> wofost = Wofost72_WLP_CWB(params, wdp, agro)
+>> wofost.run_till_terminate()
+...
+```
+
+See for more elaborate examples the [notebook on the repository](https://github.com/ajwdewit/soil_dataprovider_nl).

soil_dataprovider_nl-1.0.0/soil_dataprovider_nl.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+LICENSE
+README.md
+pyproject.toml
+soil_dataprovider_nl/__init__.py
+soil_dataprovider_nl/coords.py
+soil_dataprovider_nl/mualemvangenuchten.py
+soil_dataprovider_nl/soil_dataprovider_nl.py
+soil_dataprovider_nl.egg-info/PKG-INFO
+soil_dataprovider_nl.egg-info/SOURCES.txt
+soil_dataprovider_nl.egg-info/dependency_links.txt
+soil_dataprovider_nl.egg-info/requires.txt
+soil_dataprovider_nl.egg-info/top_level.txt

soil_dataprovider_nl-1.0.0/soil_dataprovider_nl.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

soil_dataprovider_nl-1.0.0/soil_dataprovider_nl.egg-info/requires.txt

@@ -0,0 +1,8 @@
+pandas
+pyproj==3.8
+duckdb==1.5
+
+[notebook]
+matplotlib
+ipyleaflet
+pcse>=6.0.13

soil_dataprovider_nl-1.0.0/soil_dataprovider_nl.egg-info/top_level.txt

@@ -0,0 +1 @@
+soil_dataprovider_nl