hypershap 0.0.2__tar.gz → 0.0.3__tar.gz

hypershap-0.0.3/CHANGELOG.md

@@ -0,0 +1,20 @@
+# v0.0.3
+- Added multi-baseline ablation game. This game computes ablation paths with respect to multiple baseline configurations and aggregates values for different paths via mean, min, max or variance.
+- Added waterfall plots to the HyperSHAP interface.
+- Added support for multi-data settings
+- Enabled approximation of Shapley values and interactions for settings exceeding a certain number of hyperparameters.
+
+# v0.0.2
+- Added parallelization for faster analysis
+
+## Maintenance
+- Added unit tests for proper quality assurance and continuous integration
+- Relaxed requirements to allow interoperability with python 3.10
+
+# v0.0.1
+- Initial implementation of HyperSHAP and its explanation games:
+  - AblationGame
+  - TunabilityGame
+  - MistunabilityGame
+  - SensitivityGame
+  - OptimizerBiasGame

{hypershap-0.0.2 → hypershap-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hypershap
-Version: 0.0.2
+Version: 0.0.3
 Summary: HyperSHAP is a post-hoc explanation method for hyperparameter optimization.
 Project-URL: Homepage, https://automl.github.io/hypershap/
 Project-URL: Repository, https://github.com/automl/hypershap
@@ -100,14 +100,14 @@ The example demonstrates how to:
 
 ## API Overview
 
-| Method | Purpose | Key Arguments |
-|--------|---------|---------------|
-| `HyperSHAP(explanation_task)` | Initialise the explainer with a generic `ExplanationTask`. |
+| Method | Purpose                                                                                                                           | Key Arguments |
+|--------|-----------------------------------------------------------------------------------------------------------------------------------|---------------|
+| `HyperSHAP(explanation_task)` | Initialize the explainer with a generic `ExplanationTask`.                                                                        |
 | `ablation(config_of_interest, baseline_config, index="FSII", order=2)` | Explain the contribution of each hyperparameter value (and interactions) when moving from a baseline to a specific configuration. |
-| `tunability(baseline_config=None, index="FSII", order=2, n_samples=10_000)` | Quantify how much performance can be gained by tuning subsets of hyper‑parameters. |
-| `optimizer_bias(optimizer_of_interest, optimizer_ensemble, index="FSII", order=2)` | Attribute performance differences to a particular optimizer vs. an ensemble of optimizers. |
-| `plot_si_graph(interaction_values=None, save_path=None)` | Plot the Shapley Interaction (SI) graph; uses the most recent interaction values if none are supplied. |
-| `ExplanationTask.get_hyperparameter_names()` | Helper to retrieve ordered hyper‑parameter names (used for visualisation). |
+| `tunability(baseline_config=None, index="FSII", order=2, n_samples=10_000)` | Quantify how much performance can be gained by tuning subsets of hyper‑parameters.                                                |
+| `optimizer_bias(optimizer_of_interest, optimizer_ensemble, index="FSII", order=2)` | Attribute performance differences to a particular optimizer vs. an ensemble of optimizers.                                        |
+| `plot_si_graph(interaction_values=None, save_path=None)` | Plot the Shapley Interaction (SI) graph; uses the most recent interaction values if none are supplied.                            |
+| `ExplanationTask.get_hyperparameter_names()` | Helper to retrieve ordered hyper‑parameter names (used for visualisation).                                                        |
 
 All methods return an `InteractionValues` object (from **shapiq**) that can be inspected, saved, or passed to the visualisation routine.
 
@@ -147,11 +147,11 @@ The paper introduces the underlying game-theoretic framework and demonstrates it
 
 Contributions are welcome! Please follow these steps:
 
-     Fork the repo and create a feature branch (git checkout -b feat/your-feature).
-     Write tests (the project uses pytest).
-     Ensure all tests pass (pytest).
-     Update documentation if you add new functionality.
-     Submit a Pull Request with a clear description of the changes.
+1. Fork the repo and create a feature branch (git checkout -b feat/your-feature).
+2. Write tests (the project uses pytest).
+3. Ensure all tests pass (pytest).
+4. Update documentation if you add new functionality.
+5. Submit a Pull Request with a clear description of the changes.
 
 
 See CONTRIBUTING.md for detailed guidelines.

{hypershap-0.0.2 → hypershap-0.0.3}/README.md

@@ -74,14 +74,14 @@ The example demonstrates how to:
 
 ## API Overview
 
-| Method | Purpose | Key Arguments |
-|--------|---------|---------------|
-| `HyperSHAP(explanation_task)` | Initialise the explainer with a generic `ExplanationTask`. |
+| Method | Purpose                                                                                                                           | Key Arguments |
+|--------|-----------------------------------------------------------------------------------------------------------------------------------|---------------|
+| `HyperSHAP(explanation_task)` | Initialize the explainer with a generic `ExplanationTask`.                                                                        |
 | `ablation(config_of_interest, baseline_config, index="FSII", order=2)` | Explain the contribution of each hyperparameter value (and interactions) when moving from a baseline to a specific configuration. |
-| `tunability(baseline_config=None, index="FSII", order=2, n_samples=10_000)` | Quantify how much performance can be gained by tuning subsets of hyper‑parameters. |
-| `optimizer_bias(optimizer_of_interest, optimizer_ensemble, index="FSII", order=2)` | Attribute performance differences to a particular optimizer vs. an ensemble of optimizers. |
-| `plot_si_graph(interaction_values=None, save_path=None)` | Plot the Shapley Interaction (SI) graph; uses the most recent interaction values if none are supplied. |
-| `ExplanationTask.get_hyperparameter_names()` | Helper to retrieve ordered hyper‑parameter names (used for visualisation). |
+| `tunability(baseline_config=None, index="FSII", order=2, n_samples=10_000)` | Quantify how much performance can be gained by tuning subsets of hyper‑parameters.                                                |
+| `optimizer_bias(optimizer_of_interest, optimizer_ensemble, index="FSII", order=2)` | Attribute performance differences to a particular optimizer vs. an ensemble of optimizers.                                        |
+| `plot_si_graph(interaction_values=None, save_path=None)` | Plot the Shapley Interaction (SI) graph; uses the most recent interaction values if none are supplied.                            |
+| `ExplanationTask.get_hyperparameter_names()` | Helper to retrieve ordered hyper‑parameter names (used for visualisation).                                                        |
 
 All methods return an `InteractionValues` object (from **shapiq**) that can be inspected, saved, or passed to the visualisation routine.
 
@@ -121,11 +121,11 @@ The paper introduces the underlying game-theoretic framework and demonstrates it
 
 Contributions are welcome! Please follow these steps:
 
-     Fork the repo and create a feature branch (git checkout -b feat/your-feature).
-     Write tests (the project uses pytest).
-     Ensure all tests pass (pytest).
-     Update documentation if you add new functionality.
-     Submit a Pull Request with a clear description of the changes.
+1. Fork the repo and create a feature branch (git checkout -b feat/your-feature).
+2. Write tests (the project uses pytest).
+3. Ensure all tests pass (pytest).
+4. Update documentation if you add new functionality.
+5. Submit a Pull Request with a clear description of the changes.
 
 
 See CONTRIBUTING.md for detailed guidelines.

hypershap-0.0.3/docs/api/games.md

@@ -0,0 +1,2 @@
+
+::: hypershap.games

hypershap-0.0.3/docs/api/hypershap.md

@@ -0,0 +1,2 @@
+
+::: hypershap.hypershap

hypershap-0.0.3/docs/api/surrogate_model.md

@@ -0,0 +1,2 @@
+
+::: hypershap.surrogate_model

hypershap-0.0.3/docs/api/task.md

@@ -0,0 +1,2 @@
+
+::: hypershap.task

hypershap-0.0.3/docs/api/utils.md

@@ -0,0 +1,2 @@
+
+::: hypershap.utils

hypershap-0.0.3/docs/authors.md

@@ -0,0 +1,9 @@
+# Authors
+
+## Core Development
+
+- [Marcel Wever](https://www.marcelwever.de){target="_blank"}
+
+## Contributions
+- [Maximilian Muschalik](https://maxmuschalik.com/){target="_blank"}
+- [Fabian Fumagalli](https://fabianfumagalli.com/){target="_blank"}

hypershap-0.0.3/docs/cite.md

@@ -0,0 +1,25 @@
+# Cite HyperSHAP
+
+If you use HyperSHAP in your research, please cite the original paper:
+
+
+## Citation String
+Wever et al., (2025). HyperSHAP: Shapley Values and Interactions for Hyperparameter Importance. CoRR, abs/2502.01276, https://doi.org/10.48550/ARXIV.2502.01276.
+
+
+## Bibtex
+```bibtex
+@article{wever-arxiv25,
+  author       = {Marcel Wever and
+                  Maximilian Muschalik and
+                  Fabian Fumagalli and
+                  Marius Lindauer},
+  title        = {HyperSHAP: Shapley Values and Interactions for Hyperparameter Importance},
+  journal      = {CoRR},
+  volume       = {abs/2502.01276},
+  year         = {2025},
+  doi          = {10.48550/ARXIV.2502.01276},
+}
+```
+
+The paper introduces the underlying game-theoretic framework and demonstrates its usefulness for HPO explainability.

hypershap-0.0.3/docs/contribute.md

@@ -0,0 +1,126 @@
+# How to Contribute
+
+Contributions are welcome, and they are greatly appreciated!
+Every little bit helps, and credit will always be given.
+
+You can contribute in many ways:
+
+# Types of Contributions
+
+## Report Bugs
+
+Report bugs at https://github.com/automl/HyperSHAP/issues
+
+If you are reporting a bug, please include:
+
+- Your operating system name and version.
+- Any details about your local setup that might be helpful in troubleshooting.
+- Detailed steps to reproduce the bug.
+
+## Fix Bugs
+
+Look through the GitHub issues for bugs.
+Anything tagged with "bug" and "help wanted" is open to whoever wants to implement a fix for it.
+
+## Implement Features
+
+Look through the GitHub issues for features.
+Anything tagged with "enhancement" and "help wanted" is open to whoever wants to implement it.
+
+## Write Documentation
+
+HyperSHAP could always use more documentation, whether as part of the official docs, in docstrings, or even on the web in blog posts, articles, and such.
+
+## Submit Feedback
+
+The best way to send feedback is to file an issue at https://github.com/automl/HyperSHAP/issues.
+
+If you are proposing a new feature:
+
+- Explain in detail how it would work.
+- Keep the scope as narrow as possible, to make it easier to implement.
+- Remember that this is a volunteer-driven project, and that contributions
+  are welcome :)
+
+# Get Started!
+
+Ready to contribute? Here's how to set up `HyperSHAP` for local development.
+Please note this documentation assumes you already have `uv` and `Git` installed and ready to go.
+
+1. Fork the `HyperSHAP` repo on GitHub.
+
+2. Clone your fork locally:
+
+```bash
+cd <directory_in_which_repo_should_be_created>
+git clone git@github.com:YOUR_NAME/HyperSHAP.git
+```
+
+3. Now we need to install the environment. Navigate into the directory
+
+```bash
+cd HyperSHAP
+```
+
+Then, install and activate the environment with:
+
+```bash
+uv sync
+```
+
+4. Install pre-commit to run linters/formatters at commit time:
+
+```bash
+uv run pre-commit install
+```
+
+5. Create a branch for local development:
+
+```bash
+git checkout -b name-of-your-bugfix-or-feature
+```
+
+Now you can make your changes locally.
+
+6. Don't forget to add test cases for your added functionality to the `tests` directory.
+
+7. When you're done making changes, check that your changes pass the formatting tests.
+
+```bash
+make check
+```
+
+Now, validate that all unit tests are passing:
+
+```bash
+make test
+```
+
+9. Before raising a pull request you should also run tox.
+   This will run the tests across different versions of Python:
+
+```bash
+tox
+```
+
+This requires you to have multiple versions of python installed.
+This step is also triggered in the CI/CD pipeline, so you could also choose to skip this step locally.
+
+10. Commit your changes and push your branch to GitHub:
+
+```bash
+git add .
+git commit -m "Your detailed description of your changes."
+git push origin name-of-your-bugfix-or-feature
+```
+
+11. Submit a pull request through the GitHub website.
+
+# Pull Request Guidelines
+
+Before you submit a pull request, check that it meets these guidelines:
+
+1. The pull request should include tests.
+
+2. If the pull request adds functionality, the docs should be updated.
+   Put your new functionality into a function with a docstring, and add the feature to the list in `README.md`.

hypershap-0.0.3/docs/getting_started.md

@@ -0,0 +1,48 @@
+# Getting Started
+
+## Installation
+
+The `HyperSHAP` package can be easily installed via PyPI:
+
+```bash
+pip install hypershap
+```
+
+Alternatively, one can clone the GitHub repository and install hypershap via the following command:
+
+```bash
+make install
+```
+
+## General Usage
+
+Given an existing setup with a ConfigurationSpace from the [ConfigSpace package](https://github.com/automl/ConfigSpace) and black-box function as follows:
+```Python
+from ConfigSpace import ConfigurationSpace, Configuration
+
+# ConfigurationSpace describing the hyperparameter space
+cs = ConfigurationSpace()
+  ...
+
+# A black-box function, evaluating ConfigSpace.Configuration objects
+def blackbox_function(cfg: Configuration) -> float:
+  ...
+```
+
+You can use HyperSHAP as follows:
+```Python
+from hypershap import ExplanationTask, HyperSHAP
+
+# Instantiate HyperSHAP
+hypershap = HyperSHAP(ExplanationTask.from_function(config_space=cs,function=blackbox_function))
+# Conduct tunability analysis
+hypershap.tunability(baseline_config=cs.get_default_configuration())
+# Plot results as a Shapley Interaction graph
+hypershap.plot_si_graph()
+```
+
+The example demonstrates how to:
+
+1. Wrap a black-box function in an explanation task.
+2. Use `HyperSHAP` to obtain interaction values for the **tunability** game.
+3. Plot the corresponding SI-graph.

hypershap-0.0.3/docs/help.md

@@ -0,0 +1,7 @@
+# Need help?
+
+Do you have further questions or a problem with `HyperSHAP` and do not know how to continue?
+
+1. Check the [examples](https://github.com/automl/HyperSHAP/tree/main/examples), maybe you can find a suitable solution.
+2. Check the [GitHub issues](https://github.com/automl/hypershap/issues), maybe someone else has been in a similar situation already.
+3. If you could not find anything related, please [create a help issue](https://github.com/automl/hypershap/issues).

hypershap-0.0.3/docs/index.md

@@ -0,0 +1,34 @@
+# Welcome to HyperSHAP's documentation! <img src="https://raw.githubusercontent.com/automl/hypershap/main/docs/source/_static/logo/hypershap-logo.png" alt="HyperSHAP Logo" align="right" height="200px"/>
+
+[![Release](https://img.shields.io/github/v/release/automl/HyperSHAP)](https://img.shields.io/github/v/release/automl/hypershap)
+[![Build status](https://img.shields.io/github/actions/workflow/status/automl/hypershap/main.yml?branch=main)](https://github.com/automl/hypershap/actions/workflows/main.yml?query=branch%3Amain)
+[![Coverage Status](https://coveralls.io/repos/github/automl/HyperSHAP/badge.svg?branch=dev)](https://coveralls.io/github/automl/HyperSHAP?branch=dev)
+[![Commit activity](https://img.shields.io/github/commit-activity/m/automl/hypershap)](https://img.shields.io/github/commit-activity/m/automl/hypershap)
+[![License](https://img.shields.io/github/license/automl/hypershap)](https://img.shields.io/github/license/automl/hypershap)
+
+
+`HyperSHAP` is a framework to explain the outcomes of hyperparameter optimization (HPO) by leveraging cooperative game theory concepts such as Shapley values and interaction indices. It is designed to provide actionable insights into the role and interplay of hyperparameters, thereby reducing the manual effort typically required to interpret HPO results. While its primary audience is researchers and practitioners in machine learning and artificial intelligence, its use is not restricted to these target groups.
+
+The analysis of HPO results often involves comparing tuned configurations, assessing hyperparameter importance, and identifying optimizer biases. These tasks are typically performed in an ad-hoc and dataset-specific manner, requiring extensive manual inspection of results. Moreover, existing approaches often lack the ability to systematically capture interactions among hyperparameters or to generalize explanations across datasets and optimizers.
+
+`HyperSHAP` addresses these challenges by formulating HPO explanations as cooperative games, where hyperparameters form coalitions whose contributions to performance are quantified. This unified framework enables fine-grained analyses, such as ablation studies, sensitivity attribution, tunability assessment, and optimizer behavior characterization. All computations are naturally parallelizable, making `HyperSHAP` scalable to modern HPO scenarios. By automating and standardizing the generation of interpretable explanations, `HyperSHAP` alleviates much of the overhead in analyzing HPO results and provides practitioners with clear guidance on which hyperparameters to focus on and how optimizers behave across tasks.
+
+- **Github repository**: <https://github.com/automl/hypershap/>
+
+
+## Features
+- **Additive Shapley decomposition** of any performance metric across hyper‑parameters.
+- **Interaction analysis** via the Faithful Shapley Interaction Index (FSII).
+- Ready‑made explanation tasks for **Ablation**, **Tunability**, and **Optimizer Bias** studies.
+- Integrated **visualisation** (SI‑graph) for interaction effects.
+- Works with any surrogate model that follows the `ExplanationTask` interface.
+
+---
+
+## Acknowledgements
+This work was partially supported by the European Union (ERC, “ixAutoML”, grant no.101041029).
+
+
+---
+
+**Enjoy exploring your HPO pipelines with HyperSHAP!** 🎉

hypershap-0.0.3/docs/license.md

@@ -0,0 +1,8 @@
+# License
+
+This program is free software: you can redistribute it and/or modify it under the terms of the 3-clause BSD license
+(please see the [LICENSE](https://github.com/automl/HyperSHAP/blob/main/LICENSE) file).
+This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
+without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+You should have received a copy of the 3-clause BSD license along with this program
+(see [LICENSE](https://github.com/automl/HyperSHAP/blob/main/LICENSE) file). If not, see [BSD-3-Clause license](https://opensource.org/license/BSD-3-Clause).