simdec 1.0.0__tar.gz → 1.2.0.dev0__tar.gz

{simdec-1.0.0 → simdec-1.2.0.dev0}/.gitignore

@@ -7,6 +7,9 @@ __pycache__
 .cache/*
 **/.ipynb_checkpoints
 
+# Env variables with direnv
+.envrc
+
 # OS generated files
 .DS_Store*
 
@@ -30,3 +33,4 @@ build/*
 dist/*
 sdist/*
 docs/_build
+app.html

{simdec-1.0.0 → simdec-1.2.0.dev0}/CONTRIBUTING.md

@@ -161,3 +161,39 @@ a release:
 5. Check that release has been deployed to PyPI
 6. Check documentation is built and deployed to readthedocs
 7. Check that auto-generated PR is auto-merged on the conda-forge feedstock repo
+
+## Dashboard
+A live dashboard is available at:
+
+https://simdec.io
+
+The DNS records are available on CPanel and the rest is hosted on Google Cloud
+Platform.
+
+Developing locally requires the installation of GCP CLI and Docker engine.
+
+A few helper commands are provided in the Makefile.
+
+### Local use
+
+The dashboard can be run locally using:
+
+    make serve-dev
+
+If you want to test OAuth, you need to export the following env variables:
+
+    export PANEL_OAUTH_REDIRECT_URI=http://localhost:5006/app
+    export PANEL_OAUTH_KEY=[VALUE IN GCP Secret Manager]
+    export PANEL_OAUTH_SECRET=[VALUE IN GCP Secret Manager]
+    export PANEL_OAUTH_ENCRYPTION=[VALUE IN GCP Secret Manager]
+
+Use the CLI tool `direnv` for convenience. Then you can serve with
+OAuth support:
+
+    make serve-oauth
+
+### Deployment
+
+New version can either be deployed from the CI with `cloudbuild` or locally:
+
+    make production

{simdec-1.0.0 → simdec-1.2.0.dev0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.3
 Name: simdec
-Version: 1.0.0
+Version: 1.2.0.dev0
 Summary: Sensitivity analysis using simulation decomposition
 Project-URL: homepage, https://www.simdec.fi/
 Project-URL: documentation, https://simdec.readthedocs.io
@@ -16,20 +16,28 @@ Classifier: License :: OSI Approved :: BSD License
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Requires-Python: >=3.9
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.10
 Requires-Dist: numpy
 Requires-Dist: pandas
-Requires-Dist: panel
 Requires-Dist: salib
 Requires-Dist: seaborn
-Requires-Dist: typer[all]
+Provides-Extra: dashboard
+Requires-Dist: cryptography; extra == 'dashboard'
+Requires-Dist: panel>=1.3.8; extra == 'dashboard'
 Provides-Extra: dev
+Requires-Dist: accessible-pygments; extra == 'dev'
+Requires-Dist: cryptography; extra == 'dev'
 Requires-Dist: hatch; extra == 'dev'
+Requires-Dist: numpydoc; extra == 'dev'
+Requires-Dist: panel>=1.3.8; extra == 'dev'
 Requires-Dist: pre-commit; extra == 'dev'
-Requires-Dist: simdec[doc,test]; extra == 'dev'
+Requires-Dist: pydata-sphinx-theme; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Requires-Dist: sphinx; extra == 'dev'
 Provides-Extra: doc
 Requires-Dist: accessible-pygments; extra == 'doc'
 Requires-Dist: numpydoc; extra == 'doc'
@@ -56,7 +64,7 @@ When public
 analysis method, which is based on Monte Carlo simulation. SimDec consists of
 three major parts:
 
-1. computing significance indices,
+1. computing sensitivity indices,
 2. creating multi-variable scenarios and mapping the output values to them, and
 3. visualizing the scenarios on the output distribution by color-coding its segments.
 
@@ -64,7 +72,15 @@ SimDec reveals the nature of causalities and interaction effects in the model.
 See our [publications](https://www.simdec.fi/publications) and join our
 [discord community](https://discord.gg/54SFcNsZS4).
 
-...
+## Python API
+The library is distributed on PyPi and can be installed with:
+
+    pip install simdec
+
+## Dashboard
+A live dashboard is available at:
+
+https://simdec.io
 
 ## Citations
 
@@ -76,13 +92,6 @@ Foundation, and Finnish Foundation for Economic Education.
 If you use SimDec in your research we would appreciate a citation to the
 following publications:
 
-- Kozlova, M., & Yeomans, J. S. (2022). Monte Carlo Enhancement via Simulation
-  Decomposition: A “Must-Have” Inclusion for Many Disciplines. _INFORMS
-  Transactions on Education, 22_(3), 147-159. DOI:10.1287/ited.2019.0240.
-- Kozlova, M., Moss, R. J., Yeomans, J. S., & Caers, J. (forthcoming).
-  Uncovering Heterogeneous Effects in Computational Models for Sustainable
-  Decision-making. _Environmental Modelling & Software_.
-- Kozlova, M., Moss, R. J., Roy, P., Alam, A., & Yeomans, J. S. (forthcoming).
-  SimDec algorithm. In M. Kozlova & J. S. Yeomans (Eds.), _Sensitivity Analysis
-  for Business, Technology, and Policymaking Made Easy with Simulation
-  Decomposition_. Routledge.
+- Kozlova, M., & Yeomans, J. S. (2022). Monte Carlo Enhancement via Simulation Decomposition: A “Must-Have” Inclusion for Many Disciplines. _INFORMS Transactions on Education, 22_(3), 147-159. [Available here](https://pubsonline.informs.org/doi/10.1287/ited.2019.0240).
+- Kozlova, M., Moss, R. J., Yeomans, J. S., & Caers, J. (2024). Uncovering Heterogeneous Effects in Computational Models for Sustainable Decision-making. _Environmental Modelling & Software_, 171, 105898. [https://doi.org/10.1016/j.envsoft.2023.105898](https://doi.org/10.1016/j.envsoft.2023.105898)
+- Kozlova, M., Moss, R. J., Roy, P., Alam, A., & Yeomans, J. S. (forthcoming). SimDec algorithm and guidelines for its usage and interpretation. In M. Kozlova & J. S. Yeomans (Eds.), _Sensitivity Analysis for Business, Technology, and Policymaking. Made Easy with Simulation Decomposition_. Routledge.

{simdec-1.0.0 → simdec-1.2.0.dev0}/README.md

@@ -14,7 +14,7 @@ When public
 analysis method, which is based on Monte Carlo simulation. SimDec consists of
 three major parts:
 
-1. computing significance indices,
+1. computing sensitivity indices,
 2. creating multi-variable scenarios and mapping the output values to them, and
 3. visualizing the scenarios on the output distribution by color-coding its segments.
 
@@ -22,7 +22,15 @@ SimDec reveals the nature of causalities and interaction effects in the model.
 See our [publications](https://www.simdec.fi/publications) and join our
 [discord community](https://discord.gg/54SFcNsZS4).
 
-...
+## Python API
+The library is distributed on PyPi and can be installed with:
+
+    pip install simdec
+
+## Dashboard
+A live dashboard is available at:
+
+https://simdec.io
 
 ## Citations
 
@@ -34,13 +42,6 @@ Foundation, and Finnish Foundation for Economic Education.
 If you use SimDec in your research we would appreciate a citation to the
 following publications:
 
-- Kozlova, M., & Yeomans, J. S. (2022). Monte Carlo Enhancement via Simulation
-  Decomposition: A “Must-Have” Inclusion for Many Disciplines. _INFORMS
-  Transactions on Education, 22_(3), 147-159. DOI:10.1287/ited.2019.0240.
-- Kozlova, M., Moss, R. J., Yeomans, J. S., & Caers, J. (forthcoming).
-  Uncovering Heterogeneous Effects in Computational Models for Sustainable
-  Decision-making. _Environmental Modelling & Software_.
-- Kozlova, M., Moss, R. J., Roy, P., Alam, A., & Yeomans, J. S. (forthcoming).
-  SimDec algorithm. In M. Kozlova & J. S. Yeomans (Eds.), _Sensitivity Analysis
-  for Business, Technology, and Policymaking Made Easy with Simulation
-  Decomposition_. Routledge.
+- Kozlova, M., & Yeomans, J. S. (2022). Monte Carlo Enhancement via Simulation Decomposition: A “Must-Have” Inclusion for Many Disciplines. _INFORMS Transactions on Education, 22_(3), 147-159. [Available here](https://pubsonline.informs.org/doi/10.1287/ited.2019.0240).
+- Kozlova, M., Moss, R. J., Yeomans, J. S., & Caers, J. (2024). Uncovering Heterogeneous Effects in Computational Models for Sustainable Decision-making. _Environmental Modelling & Software_, 171, 105898. [https://doi.org/10.1016/j.envsoft.2023.105898](https://doi.org/10.1016/j.envsoft.2023.105898)
+- Kozlova, M., Moss, R. J., Roy, P., Alam, A., & Yeomans, J. S. (forthcoming). SimDec algorithm and guidelines for its usage and interpretation. In M. Kozlova & J. S. Yeomans (Eds.), _Sensitivity Analysis for Business, Technology, and Policymaking. Made Easy with Simulation Decomposition_. Routledge.

{simdec-1.0.0 → simdec-1.2.0.dev0}/pyproject.toml

@@ -4,10 +4,10 @@ build-backend = "hatchling.build"
 
 [project]
 name = "simdec"
-version = "1.0.0"
+version = "1.2.0dev0"
 description = "Sensitivity analysis using simulation decomposition"
 readme = "README.md"
-requires-python = ">=3.9"
+requires-python = ">=3.10"
 license = "BSD-3-Clause"
 authors = [
     { name = "Pamphile Roy" },
@@ -23,9 +23,9 @@ classifiers = [
     "Operating System :: OS Independent",
     "Programming Language :: Python",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
 ]
 
 dependencies = [
@@ -33,11 +33,14 @@ dependencies = [
     "pandas",
     "SALib",
     "seaborn",
-    "typer[all]",
-    "panel",
 ]
 
 [project.optional-dependencies]
+dashboard = [
+    "panel>=1.3.8",
+    "cryptography",
+]
+
 test = [
     "pytest",
     "pytest-cov",
@@ -51,29 +54,33 @@ doc = [
 ]
 
 dev = [
-    "simdec[doc,test]",
+    "simdec[doc,test,dashboard]",
     "pre-commit",
     "hatch",
 ]
 
-[project.scripts]
-simdec = "simdec.workflow:app"
-
 [project.urls]
 homepage = "https://www.simdec.fi/"
 documentation = "https://simdec.readthedocs.io"
 source = "https://github.com/Simulation-Decomposition/simdec-python"
 
 [tool.hatch]
+build.targets.wheel.packages = ["src/simdec"]
 build.targets.sdist.exclude = [
   ".github",
   "docs",
+  "panel",
   "tests",
   "*.rst",
   "*.yml",
   ".*",
+  "Makefile",
+  "Dockerfile",
 ]
 
+[project.entry-points."panel.auth"]
+custom_google = "simdec.auth:CustomGoogleLoginHandler"
+
 [tool.pytest.ini_options]
 addopts = "--durations 10"
 testpaths = [

simdec-1.2.0.dev0/src/simdec/__init__.py

@@ -0,0 +1,13 @@
+"""SimDec main namespace."""
+from simdec.decomposition import *
+from simdec.sensitivity_indices import *
+from simdec.visualization import *
+
+__all__ = [
+    "sensitivity_indices",
+    "states_expansion",
+    "decomposition",
+    "visualization",
+    "tableau",
+    "palette",
+]

simdec-1.2.0.dev0/src/simdec/auth.py

@@ -0,0 +1,17 @@
+from panel.auth import GoogleLoginHandler
+from panel.io.resources import CDN_DIST
+
+
+class CustomGoogleLoginHandler(GoogleLoginHandler):
+    def _simple_get(self):
+        html = self._login_template.render(errormessage="", PANEL_CDN=CDN_DIST)
+        self.write(html)
+
+    async def get(self):
+        if "login" in self.request.uri and "state" not in self.request.uri:
+            self._simple_get()
+        else:
+            await super().get()
+
+    async def post(self):
+        await super().get()

simdec-1.2.0.dev0/src/simdec/decomposition.py

@@ -0,0 +1,190 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from hashlib import blake2b
+from typing import Literal
+
+import numpy as np
+import pandas as pd
+from scipy import stats
+
+
+__all__ = ["decomposition", "states_expansion"]
+
+
+def states_expansion(states: list[int], inputs: pd.DataFrame) -> list[list[str]]:
+    """Expand states list to fully represent all scenarios."""
+    inputs = pd.DataFrame(inputs)
+    expanded_states = []
+    for state in states:
+        if isinstance(state, int):
+            if state == 2:
+                expanded_states.append(["low", "high"])
+            elif state == 3:
+                expanded_states.append(["low", "medium", "high"])
+        else:
+            expanded_states.append(state)
+
+    # categorical for a given variable
+    cat_cols = inputs.select_dtypes(exclude=["number"])
+    cat_cols_idx = []
+    states_cats_ = []
+    for cat_col in cat_cols:
+        _, cats = pd.factorize(inputs[cat_col])
+        cat_cols_idx.append(inputs.columns.get_loc(cat_col))
+        states_cats_.append(cats)
+
+    for i, states_cat_ in zip(cat_cols_idx, states_cats_):
+        n_unique = np.unique(inputs.iloc[:, i]).size
+        expanded_states[i] = list(states_cat_) if n_unique < 5 else expanded_states[i]
+
+    return expanded_states
+
+
+@dataclass
+class DecompositionResult:
+    var_names: list[str]
+    statistic: np.ndarray
+    bins: pd.DataFrame
+    states: list[int]
+    bin_edges: np.ndarray
+
+    def __reduce__(self):
+        h = blake2b(key=b"result hashing", digest_size=20)
+
+        h.update(str(self.var_names).encode())
+        h.update(str(self.statistic).encode())
+        h.update(str(self.bins).encode())
+        h.update(str(self.states).encode())
+        h.update(str(self.bin_edges).encode())
+
+        return [h.hexdigest()]
+
+
+def decomposition(
+    inputs: pd.DataFrame,
+    output: pd.DataFrame,
+    *,
+    sensitivity_indices: np.ndarray,
+    dec_limit: float = 1,
+    auto_ordering: bool = True,
+    states: list[int] | None = None,
+    statistic: Literal["mean", "median"] | None = "mean",
+) -> DecompositionResult:
+    """SimDec decomposition.
+
+    Parameters
+    ----------
+    inputs : DataFrame of shape (n_runs, n_factors)
+        Input variables.
+    output : DataFrame of shape (n_runs, 1)
+        Target variable.
+    sensitivity_indices : ndarray of shape (n_factors, 1)
+        Sensitivity indices, combined effect of each input.
+    dec_limit : float
+        Explained variance ratio to filter the number input variables.
+    auto_ordering : bool
+        Automatically order input columns based on the relative sensitivity_indices
+        or use the provided order.
+    states : list of int, optional
+        List of possible states for the considered parameter.
+    statistic : {"mean", "median"}, optional
+        Statistic to compute in each bin.
+
+    Returns
+    -------
+    res : DecompositionResult
+        An object with attributes:
+
+        var_names : list of string (n_factors, 1)
+            Variable names.
+        statistic : ndarray of shape (n_factors, 1)
+            Statistic in each bin.
+        bins : DataFrame
+            Multidimensional bins.
+        states : list of int
+            List of possible states for the considered parameter.
+
+    """
+    var_names = inputs.columns
+
+    cat_cols = inputs.select_dtypes(exclude=["number"])
+    cat_cols_idx = []
+    for cat_col in cat_cols:
+        codes, cat_states_ = pd.factorize(inputs[cat_col])
+        inputs[cat_col] = codes
+        cat_cols_idx.append(inputs.columns.get_loc(cat_col))
+
+    inputs = inputs.to_numpy()
+    output = output.to_numpy()
+
+    # 1. variables for decomposition
+    var_order = np.argsort(sensitivity_indices)[::-1]
+
+    # only keep the explained variance corresponding to `dec_limit`
+    sensitivity_indices = sensitivity_indices[var_order]
+
+    if auto_ordering:
+        n_var_dec = np.where(np.cumsum(sensitivity_indices) < dec_limit)[0].size
+        n_var_dec = max(1, n_var_dec)  # keep at least one variable
+        n_var_dec = min(5, n_var_dec)  # use at most 5 variables
+    else:
+        n_var_dec = inputs.shape[1]
+
+    # 2. states formation
+    if states is None:
+        states = 3 if n_var_dec < 3 else 2
+        states = [states] * n_var_dec
+
+        # categorical for a given variable
+        for i in cat_cols_idx:
+            n_unique = np.unique(inputs[:, i]).size
+            states[i] = n_unique if n_unique < 5 else states[i]
+
+    if auto_ordering:
+        var_names = var_names[var_order[:n_var_dec]].tolist()
+        inputs = inputs[:, var_order[:n_var_dec]]
+
+    # 3. decomposition
+    bins = []
+
+    statistic_methods = {
+        "mean": np.mean,
+        "median": np.median,
+    }
+    try:
+        statistic_method = statistic_methods[statistic]
+    except IndexError:
+        msg = f"'statistic' must be one of {statistic_methods.values()}"
+        raise ValueError(msg)
+
+    def statistic_(inputs):
+        """Custom function to keep track of the content of bins."""
+        bins.append(inputs)
+        return statistic_method(inputs)
+
+    res = stats.binned_statistic_dd(
+        inputs, values=output, statistic=statistic_, bins=states
+    )
+
+    bins = pd.DataFrame(bins[1:]).T
+
+    if len(bins.columns) != np.prod(states):
+        # mismatch with the number of states vs bins
+        # when it happens, we have NaNs in the statistic
+        # we can add empty columns with NaNs on these positions as bins
+        # then are not present for these states
+        nan_idx = np.argwhere(np.isnan(res.statistic).flatten()).flatten()
+
+        for idx in nan_idx:
+            bins = np.insert(bins, idx, np.nan, axis=1)
+
+        bins = pd.DataFrame(bins)
+
+    return DecompositionResult(
+        var_names=var_names,
+        statistic=res.statistic,
+        bins=bins,
+        states=states,
+        bin_edges=res.bin_edges,
+    )

simdec-1.0.0/src/simdec/significance.py → simdec-1.2.0.dev0/src/simdec/sensitivity_indices.py

@@ -6,11 +6,11 @@ import pandas as pd
 from scipy import stats
 
 
-__all__ = ["significance"]
+__all__ = ["sensitivity_indices"]
 
 
 def number_of_bins(n_runs: int, n_factors: int) -> tuple[int, int]:
-    """Optimal number of bins for first & second-order significance indices.
+    """Optimal number of bins for first & second-order sensitivity_indices indices.
 
     Linear approximation of experimental results from (Marzban & Lahmer, 2016).
     """
@@ -31,18 +31,18 @@ def _weighted_var(x: np.ndarray, weights: np.ndarray) -> np.ndarray:
 
 
 @dataclass
-class SignificanceResult:
+class SensitivityAnalysisResult:
     si: np.ndarray
     first_order: np.ndarray
     second_order: np.ndarray
 
 
-def significance(
+def sensitivity_indices(
     inputs: pd.DataFrame | np.ndarray, output: pd.DataFrame | np.ndarray
-) -> SignificanceResult:
-    """Significance indices.
+) -> SensitivityAnalysisResult:
+    """Sensitivity indices.
 
-    The significance express how much variability of the output is
+    The sensitivity_indices express how much variability of the output is
     explained by the inputs.
 
     Parameters
@@ -54,11 +54,11 @@ def significance(
 
     Returns
     -------
-    res : SignificanceResult
+    res : SensitivityAnalysisResult
         An object with attributes:
 
         si : ndarray of shape (n_factors, 1)
-            Significance index, combined effect of each input.
+            Sensitivity indices, combined effect of each input.
         foe : ndarray of shape (n_factors, 1)
             First-order effects (also called 'main' or 'individual').
         soe : ndarray of shape (n_factors, 1)
@@ -89,19 +89,18 @@ def significance(
     ... )
     >>> output = f_ishigami(inputs.T)
 
-    We can now pass our inputs and outputs to the `significance` function:
+    We can now pass our inputs and outputs to the `sensitivity_indices` function:
 
-    >>> res = sd.significance(inputs=inputs, output=output)
+    >>> res = sd.sensitivity_indices(inputs=inputs, output=output)
     >>> res.si
     array([0.43157591, 0.44241433, 0.11767249])
 
     """
-    cat_columns = inputs.select_dtypes(["category", "O"]).columns
-    inputs[cat_columns] = inputs[cat_columns].apply(
-        lambda x: x.astype("category").cat.codes
-    )
-
     if isinstance(inputs, pd.DataFrame):
+        cat_columns = inputs.select_dtypes(["category", "O"]).columns
+        inputs[cat_columns] = inputs[cat_columns].apply(
+            lambda x: x.astype("category").cat.codes
+        )
         inputs = inputs.to_numpy()
     if isinstance(output, pd.DataFrame):
         output = output.to_numpy()
@@ -169,4 +168,4 @@ def significance(
         soe = np.where(soe == 0, soe.T, soe)
         si[i] = foe[i] + soe[:, i].sum() / 2
 
-    return SignificanceResult(si, foe, soe)
+    return SensitivityAnalysisResult(si, foe, soe)

simdec-1.2.0.dev0/src/simdec/visualization.py

@@ -0,0 +1,263 @@
+import functools
+import itertools
+from typing import Literal
+
+import colorsys
+import matplotlib as mpl
+import matplotlib.pyplot as plt
+import numpy as np
+import seaborn as sns
+import pandas as pd
+from pandas.io.formats.style import Styler
+
+__all__ = ["visualization", "tableau", "palette"]
+
+
+SEQUENTIAL_PALETTES = [
+    "#DC267F",
+    "#E8EA2F",
+    "#26DCD1",
+    "#C552E4",
+    "#3F45D0",
+    "Oranges",
+    "Purples",
+    "Reds",
+    "Blues",
+    "Greens",
+    "YlOrBr",
+    "YlOrRd",
+    "OrRd",
+    "PuRd",
+    "RdPu",
+    "BuPu",
+    "GnBu",
+    "PuBu",
+    "YlGnBu",
+    "PuBuGn",
+    "BuGn",
+    "YlGn",
+    "Greys",
+]
+
+
+@functools.cache
+def sequential_cmaps():
+    cmaps = []
+    for cmap in SEQUENTIAL_PALETTES:
+        try:
+            cmap_ = mpl.colormaps[cmap]
+        except KeyError:
+            color = mpl.colors.hex2color(cmap)
+            cmap_ = single_color_to_colormap(color)
+        cmaps.append(cmap_)
+    return cmaps
+
+
+def single_color_to_colormap(
+    rgba_color: list[float] | str, *, factor: float = 0.5
+) -> mpl.colors.LinearSegmentedColormap:
+    """Create a linear colormap using a single color."""
+    if isinstance(rgba_color, str):
+        rgba_color = mpl.colors.hex2color(rgba_color)
+    # discard alpha channel
+    if len(rgba_color) == 4:
+        *rgb_color, alpha = rgba_color
+    else:
+        alpha = 1.0
+        rgb_color = rgba_color
+        rgba_color = list(rgba_color) + [1]
+
+    # lighten and darken from factor around single color
+    hls_color = colorsys.rgb_to_hls(*rgb_color)
+
+    lightness = hls_color[1]
+    lightened_hls_color = (hls_color[0], lightness * (1 + factor), hls_color[2])
+    lightened_rgb_color = list(colorsys.hls_to_rgb(*lightened_hls_color))
+
+    darkened_hls_color = (hls_color[0], lightness * (1 - factor), hls_color[2])
+    darkened_rgb_color = list(colorsys.hls_to_rgb(*darkened_hls_color))
+
+    lightened_rgba_color = lightened_rgb_color + [alpha]
+    darkened_rgba_color = darkened_rgb_color + [alpha]
+
+    # convert to CMAP
+    cmap = mpl.colors.LinearSegmentedColormap.from_list(
+        "CustomSingleColor",
+        [lightened_rgba_color, rgba_color, darkened_rgba_color],
+        N=3,
+    )
+    return cmap
+
+
+def palette(
+    states: list[int], cmaps: list[mpl.colors.LinearSegmentedColormap] = None
+) -> list[list[float]]:
+    """Colour palette.
+
+    The product of the states gives the number of scenarios. For each
+    scenario, a colour is set.
+
+    Parameters
+    ----------
+    states : list of int
+        List of possible states for the considered parameter.
+    cmaps : list of LinearSegmentedColormap
+        List of colormaps. Must have the same number of colormaps as the number
+        of first level of states.
+    Returns
+    -------
+    palette : list of float of size (n, 4)
+        List of colors corresponding to scenarios. RGBA formatted.
+    """
+    n_cmaps = states[0]
+    if cmaps is None:
+        cmaps = sequential_cmaps()[:n_cmaps]
+    else:
+        cmaps = cmaps[:n_cmaps]
+        if len(cmaps) != n_cmaps:
+            raise ValueError(
+                f"Must have the same number of cmaps ({len(cmaps)}) as the "
+                f"number of first states ({n_cmaps})"
+            )
+
+    colors = []
+    # one palette per first level state, could use more palette when there are
+    # many levels
+    n_shades = int(np.prod(states[1:]))
+    for i in range(n_cmaps):
+        cmap = cmaps[i].resampled(n_shades)
+        colors.append(cmap(np.linspace(0, 1, n_shades)))
+
+    return np.concatenate(colors).tolist()
+
+
+def visualization(
+    *,
+    bins: pd.DataFrame,
+    palette: list[list[float]],
+    n_bins: str | int = "auto",
+    kind: Literal["histogram", "boxplot"] = "histogram",
+    ax=None,
+) -> plt.Axes:
+    """Histogram plot of scenarios.
+
+    Parameters
+    ----------
+    bins : DataFrame
+        Multidimensional bins.
+    palette : list of int of size (n, 4)
+        List of colours corresponding to scenarios.
+    n_bins : str or int
+        Number of bins or method from `np.histogram_bin_edges`.
+    kind: {"histogram", "boxplot"}
+        Histogram or Box Plot.
+    ax : Axes, optional
+        Matplotlib axis.
+
+    Returns
+    -------
+    ax : Axes
+        Matplotlib axis.
+
+    """
+    # needed to get the correct stacking order
+    bins.columns = pd.RangeIndex(start=len(bins.columns), stop=0, step=-1)
+
+    if kind == "histogram":
+        ax = sns.histplot(
+            bins,
+            multiple="stack",
+            stat="probability",
+            palette=palette,
+            common_bins=True,
+            common_norm=True,
+            bins=n_bins,
+            legend=False,
+            ax=ax,
+        )
+    elif kind == "boxplot":
+        ax = sns.boxplot(
+            bins,
+            palette=palette,
+            orient="h",
+            order=list(bins.columns)[::-1],
+            ax=ax,
+        )
+    else:
+        raise ValueError("'kind' can only be 'histogram' or 'boxplot'")
+    return ax
+
+
+def tableau(
+    *,
+    var_names: list[str],
+    statistic: np.ndarray,
+    states: list[int | list[str]],
+    bins: pd.DataFrame,
+    palette: np.ndarray,
+) -> tuple[pd.DataFrame, Styler]:
+    """Generate a table of statistics for all scenarios.
+
+    Parameters
+    ----------
+    var_names : list of str
+        Variables name.
+    states : list of int or list of str
+        For each variable, number of states. Can either be a scalar or a list.
+
+        ``states=[2, 2]`` or ``states=[['a', 'b'], ['low', 'high']]``
+    bins : DataFrame
+        Multidimensional bins.
+    palette : list of int of size (n, 4)
+        Ordered list of colours corresponding to each state.
+
+    Returns
+    -------
+    table : DataFrame
+        Summary table of statistics for the scenarios.
+    styler : Styler
+        Object to style the table with colours and formatting.
+    """
+    table = bins.describe(percentiles=[0.5]).T
+
+    # get the index out to use a state id/colour
+    table = table.reset_index()
+    table.rename(columns={"index": "colour"}, inplace=True)
+
+    # Default states for 2 or 3
+    for i, state in enumerate(states):
+        if isinstance(state, int):
+            states: list
+            if state == 2:
+                states[i] = ["low", "high"]
+            elif state == 3:
+                states[i] = ["low", "medium", "high"]
+
+    # get the list of states
+    gen_states = [range(x) if isinstance(x, int) else x for x in states]
+    states_ = np.asarray(list(itertools.product(*gen_states)))
+    for i, var_name in enumerate(var_names):
+        table.insert(loc=i + 1, column=var_name, value=states_[:, i])
+
+    # groupby on the variable names
+    table.set_index(list(var_names), inplace=True)
+
+    proba = table["count"] / sum(table["count"])
+    proba = np.asarray(proba)
+    table["probability"] = proba
+    table["mean"] = statistic.flatten()
+
+    # only select/ordering interesting columns
+    table = table[["colour", "std", "min", "mean", "max", "probability"]]
+
+    table.insert(loc=0, column="N°", value=np.arange(1, stop=len(table) + 1)[::-1])
+
+    # style the colour background with palette
+    cmap = mpl.colors.ListedColormap(palette)
+    styler = table.style
+    styler.format(precision=2)
+    styler.background_gradient(subset=["colour"], cmap=cmap)
+    styler.format(lambda x: "", subset=["colour"])
+
+    styler.set_table_styles([{"selector": "th", "props": [("text-align", "center")]}])
+    return table, styler

simdec-1.0.0/Makefile

@@ -1,25 +0,0 @@
-.PHONY: help prepare doc test serve
-.DEFAULT_GOAL := help
-SHELL:=/bin/bash
-
-# Add help text after each target name starting with '\#\#'
-help:   ## show this help
-	@echo -e "Help for this makefile\n"
-	@echo "Possible commands are:"
-	@grep -h "##" $(MAKEFILE_LIST) | grep -v grep | sed -e 's/\(.*\):.*##\(.*\)/    \1: \2/'
-
-prepare:  ## Install dependencies and pre-commit hook
-	pip install -e ".[dev]"
-	pre-commit install
-
-doc:  ## Build Sphinx documentation
-	sphinx-build -b html docs docs/html
-
-test:  ## Run tests with coverage
-	pytest --cov simdec --cov-report term-missing
-
-serve-dev:  ## Serve Panel dashboard - Dev mode
-	panel serve app.py --show --autoreload
-
-serve:  ## Serve Panel dashboard - Prod mode
-	panel serve app.py

simdec-1.0.0/app.py

@@ -1,96 +0,0 @@
-import io
-
-import matplotlib.pyplot as plt
-import pandas as pd
-import panel as pn
-
-import simdec as sd
-
-
-# panel app
-pn.extension(template="material")
-pn.config.throttled = True
-
-text_fname = pn.widgets.FileInput(sizing_mode="stretch_width")
-slider_dec_limit = pn.widgets.FloatSlider(
-    value=1.0, step=0.05, name="Explained variance ratio"
-)
-
-
-@pn.cache
-def load_data(text_fname):
-    if text_fname is None:
-        text_fname = "tests/data/stress.csv"
-    else:
-        text_fname = io.BytesIO(text_fname)
-
-    data = pd.read_csv(text_fname)
-    output_name, *v_names = list(data.columns)
-    inputs, output = data[v_names], data[output_name]
-
-    si = sd.significance(inputs=inputs, output=output).si
-
-    return si, inputs, output
-
-
-def decomposition(dec_limit, data):
-    si, inputs, output = data
-    return sd.decomposition(
-        inputs=inputs, output=output, significance=si, dec_limit=dec_limit
-    )
-
-
-def palette(res):
-    return sd.palette(res.states)
-
-
-def figure(res, palette):
-    fig, ax = plt.subplots()
-    _ = sd.visualization(bins=res.bins, palette=palette, states=res.states, ax=ax)
-    return fig
-
-
-def tableau(res, palette):
-    # use a notebook to see the styling
-    _, styler = sd.tableau(
-        statistic=res.statistic,
-        var_names=res.var_names,
-        states=res.states,
-        bins=res.bins,
-        palette=palette,
-    )
-    return styler
-
-
-interactive_data = pn.bind(load_data, text_fname)
-interactive_decomposition = pn.bind(decomposition, slider_dec_limit, interactive_data)
-interactive_palette = pn.bind(palette, interactive_decomposition)
-interactive_figure = pn.bind(figure, interactive_decomposition, interactive_palette)
-interactive_tableau = pn.bind(tableau, interactive_decomposition, interactive_palette)
-
-
-top_description = """
-# Parameters
-
-Select a CSV file:
-- comma delimited and with point decimal separator;
-- first column is the output of the model;
-- rest of the columns are the inputs.
-"""
-
-params_description = """
-The following parameters can be adjusted:
-"""
-
-pn_params = pn.layout.WidgetBox(
-    top_description,
-    text_fname,
-    params_description,
-    slider_dec_limit,
-    max_width=350,
-    sizing_mode="stretch_width",
-).servable(area="sidebar")
-
-pn_app = pn.Column(pn.Row(interactive_figure, interactive_tableau)).servable(
-    title="Simulation Decomposition Dashboard"
-)

simdec-1.0.0/src/simdec/__init__.py

@@ -1,6 +0,0 @@
-"""SimDec main namespace."""
-from simdec.decomposition import *
-from simdec.significance import *
-from simdec.visualization import *
-
-__all__ = ["significance", "decomposition", "visualization", "tableau", "palette"]

simdec-1.0.0/src/simdec/decomposition.py

@@ -1,113 +0,0 @@
-from __future__ import annotations
-
-from dataclasses import dataclass
-from typing import Literal
-
-import numpy as np
-import pandas as pd
-from scipy import stats
-
-
-__all__ = ["decomposition"]
-
-
-@dataclass
-class DecompositionResult:
-    var_names: list[str]
-    statistic: np.ndarray
-    bins: pd.DataFrame
-    states: list[int]
-
-
-def decomposition(
-    inputs: pd.DataFrame,
-    output: pd.DataFrame,
-    *,
-    significance: np.ndarray,
-    dec_limit: float = 1,
-    states: list[int] | None = None,
-    statistic: Literal["mean", "median"] | None = "mean",
-) -> DecompositionResult:
-    """SimDec decomposition.
-
-    Parameters
-    ----------
-    inputs : DataFrame of shape (n_runs, n_factors)
-        Input variables.
-    output : DataFrame of shape (n_runs, 1)
-        Target variable.
-    significance : ndarray of shape (n_factors, 1)
-        Significance index, combined effect of each input.
-    dec_limit : float
-        Explained variance ratio to filter the number input variables.
-    states : list of int, optional
-        List of possible states for the considered parameter.
-    statistic : {"mean", "median"}, optional
-        Statistic to compute in each bin.
-
-    Returns
-    -------
-    res : DecompositionResult
-        An object with attributes:
-
-        var_names : list of string (n_factors, 1)
-            Variable names.
-        statistic : ndarray of shape (n_factors, 1)
-            Statistic in each bin.
-        bins : DataFrame
-            Multidimensional bins.
-        states : list of int
-            List of possible states for the considered parameter.
-
-    """
-    var_names = inputs.columns
-    inputs = inputs.to_numpy()
-    output = output.to_numpy()
-
-    # 1. variables for decomposition
-    var_order = np.argsort(significance)[::-1]
-
-    # only keep the explained variance corresponding to `dec_limit`
-    significance = significance[var_order]
-    n_var_dec = np.where(np.cumsum(significance) < dec_limit)[0].size
-    n_var_dec = max(1, n_var_dec)  # keep at least one variable
-    n_var_dec = min(5, n_var_dec)  # use at most 5 variables
-
-    var_names = var_names[var_order[:n_var_dec]].tolist()
-    inputs = inputs[:, var_order[:n_var_dec]]
-
-    # 2. states formation
-    if states is None:
-        states = 3 if n_var_dec < 3 else 2
-        states = [states] * n_var_dec
-
-        # categorical for a given variable
-        for i in range(n_var_dec):
-            n_unique = np.unique(inputs[:, i]).size
-            states[i] = n_unique if n_unique < 5 else states[i]
-
-    # 3. decomposition
-    bins = []
-
-    statistic_methods = {
-        "mean": np.mean,
-        "median": np.median,
-    }
-    try:
-        statistic_method = statistic_methods[statistic]
-    except IndexError:
-        msg = f"'statistic' must be one of {statistic_methods.values()}"
-        raise ValueError(msg)
-
-    def statistic_(inputs):
-        """Custom function to keep track of the content of bins."""
-        bins.append(inputs)
-        return statistic_method(inputs)
-
-    res = stats.binned_statistic_dd(
-        inputs, values=output, statistic=statistic_, bins=states
-    )
-
-    bins = pd.DataFrame(bins[1:]).T
-
-    return DecompositionResult(var_names, res.statistic, bins, states)

simdec-1.0.0/src/simdec/visualization.py

@@ -1,168 +0,0 @@
-import itertools
-
-import matplotlib as mpl
-import matplotlib.pyplot as plt
-import numpy as np
-import seaborn as sns
-import pandas as pd
-from pandas.io.formats.style import Styler
-
-__all__ = ["visualization", "tableau", "palette"]
-
-
-sequential_palettes = [
-    "Purples",
-    "Blues",
-    "Greens",
-    "Oranges",
-    "Reds",
-    "YlOrBr",
-    "YlOrRd",
-    "OrRd",
-    "PuRd",
-    "RdPu",
-    "BuPu",
-    "GnBu",
-    "PuBu",
-    "YlGnBu",
-    "PuBuGn",
-    "BuGn",
-    "YlGn",
-    "Greys",
-]
-
-
-def palette(states: list[int]) -> list[list[float]]:
-    """Colour palette.
-
-    The product of the states gives the number of scenarios. For each
-    scenario, a colour is set.
-
-    Parameters
-    ----------
-    states : list of int
-        List of possible states for the considered parameter.
-
-    Returns
-    -------
-    palette : list of int of size (n, 4)
-        List of colours corresponding to scenarios.
-    """
-    colors = []
-    # one palette per first level state, could use more palette when there are
-    # many levels
-    n_shades = int(np.prod(states[1:]))
-    for i in range(states[0]):
-        palette_ = sequential_palettes[i]
-        cmap = mpl.colormaps[palette_].resampled(n_shades + 1)
-        colors.append(cmap(range(1, n_shades + 1)))
-
-    return np.concatenate(colors).tolist()
-
-
-def visualization(
-    *, bins: pd.DataFrame, states: list[int], palette: list[list[float]], ax=None
-) -> plt.Axes:
-    """Histogram plot of scenarios.
-
-    Parameters
-    ----------
-    bins : DataFrame
-        Multidimensional bins.
-    states : list of int
-        List of possible states for the considered parameter.
-    palette : list of int of size (n, 4)
-        List of colours corresponding to scenarios.
-    ax : Axes, optional
-        Matplotlib axis.
-
-    Returns
-    -------
-    ax : Axes
-        Matplotlib axis.
-
-    """
-    # needed to get the correct stacking order
-    bins.columns = pd.RangeIndex(start=np.prod(states), stop=0, step=-1)
-
-    ax = sns.histplot(
-        bins,
-        multiple="stack",
-        stat="probability",
-        palette=palette,
-        common_bins=True,
-        common_norm=True,
-        legend=False,
-        ax=ax,
-    )
-    return ax
-
-
-def tableau(
-    *,
-    var_names: list[str],
-    statistic: np.ndarray,
-    states: list[int | str],
-    bins: pd.DataFrame,
-    palette: np.ndarray,
-) -> tuple[pd.DataFrame, Styler]:
-    """Generate a table of statistics for all scenarios.
-
-    Parameters
-    ----------
-    var_names : list of str
-        Variables name.
-    states : list of int or str
-        For each variable, number of states. Can either be a scalar or a list.
-
-        ``states=[2, 2]`` or ``states=[['a', 'b'], ['low', 'high']]``
-    bins : DataFrame
-        Multidimensional bins.
-    palette : list of int of size (n, 4)
-        Ordered list of colours corresponding to each state.
-
-    Returns
-    -------
-    table : DataFrame
-        Summary table of statistics for the scenarios.
-    styler : Styler
-        Object to style the table with colours and formatting.
-    """
-    table = bins.describe(percentiles=[0.5]).T
-
-    # get the index out to use a state id/colour
-    table = table.reset_index()
-    table.rename(columns={"index": "colour"}, inplace=True)
-
-    # get the list of states
-    gen_states = [range(x) if isinstance(x, int) else x for x in states]
-    states_ = np.asarray(list(itertools.product(*gen_states)))
-    for i, var_name in enumerate(var_names):
-        table.insert(loc=i + 1, column=var_name, value=states_[:, i])
-
-    # groupby on the variable names
-    table = (
-        table.groupby(var_names, group_keys=True, sort=False)
-        .apply(lambda x: x)
-        .droplevel(-1)
-    )
-
-    proba = table["count"] / sum(table["count"])
-    proba = np.asarray(proba)
-    table["probability"] = proba
-
-    table["weighted mean"] = statistic.flatten()
-
-    # only select/ordering interesting columns
-    table = table[
-        ["colour", "std", "min", "weighted mean", "50%", "max", "probability"]
-    ]
-
-    # style the colour background with palette
-    cmap = mpl.colors.ListedColormap(palette)
-    styler = table.style
-    styler.format(precision=3)
-    styler.background_gradient(subset=["colour"], cmap=cmap)
-    styler.format(lambda x: "", subset=["colour"])
-
-    return table, styler

simdec-1.0.0/src/simdec/workflow.py

@@ -1,57 +0,0 @@
-from pathlib import Path
-from typing_extensions import Annotated
-
-import matplotlib.pyplot as plt
-import pandas as pd
-import panel as pn
-import typer
-
-import simdec as sd
-
-app = typer.Typer()
-
-
-@app.command()
-def main(
-    data: Annotated[
-        Path,
-        typer.Option(
-            exists=True,
-            file_okay=True,
-            dir_okay=False,
-            writable=False,
-            readable=True,
-            resolve_path=True,
-        ),
-    ]
-):
-    data = pd.read_csv(data)
-    output_name, *v_names = list(data.columns)
-    inputs, output = data[v_names], data[output_name]
-
-    res = sd.significance(inputs=inputs, output=output)
-    si = res.si
-
-    res = sd.decomposition(inputs=inputs, output=output, significance=si)
-
-    fig, ax = plt.subplots()
-    palette = sd.palette(states=res.states)
-    ax, palette = sd.visualization(
-        bins=res.bins, palette=palette, states=res.states, ax=ax
-    )
-
-    # use a notebook to see the styling
-    table, styler = sd.tableau(
-        statistic=res.statistic,
-        var_names=res.var_names,
-        states=res.states,
-        bins=res.bins,
-        palette=palette,
-    )
-
-    # panel app
-    pn.extension(template="material")
-    pn_fig = pn.pane.Matplotlib(fig, dpi=144)
-    pn_table = pn.pane.DataFrame(styler)
-    pn_app = pn.Column(pn_fig, pn_table)
-    pn_app.save("app.html")