explainiverse 0.2.5__tar.gz → 0.8.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- explainiverse-0.8.1/PKG-INFO +766 -0
- explainiverse-0.8.1/README.md +734 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/pyproject.toml +3 -2
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/__init__.py +5 -4
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/adapters/pytorch_adapter.py +88 -25
- explainiverse-0.8.1/src/explainiverse/core/explanation.py +179 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/core/registry.py +94 -0
- explainiverse-0.8.1/src/explainiverse/engine/suite.py +252 -0
- explainiverse-0.8.1/src/explainiverse/evaluation/__init__.py +60 -0
- explainiverse-0.8.1/src/explainiverse/evaluation/_utils.py +325 -0
- explainiverse-0.8.1/src/explainiverse/evaluation/faithfulness.py +428 -0
- explainiverse-0.8.1/src/explainiverse/evaluation/metrics.py +314 -0
- explainiverse-0.8.1/src/explainiverse/evaluation/stability.py +379 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/__init__.py +8 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/attribution/lime_wrapper.py +90 -7
- explainiverse-0.8.1/src/explainiverse/explainers/attribution/shap_wrapper.py +185 -0
- explainiverse-0.8.1/src/explainiverse/explainers/example_based/__init__.py +18 -0
- explainiverse-0.8.1/src/explainiverse/explainers/example_based/protodash.py +826 -0
- explainiverse-0.8.1/src/explainiverse/explainers/gradient/__init__.py +37 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/gradient/integrated_gradients.py +189 -76
- explainiverse-0.8.1/src/explainiverse/explainers/gradient/lrp.py +1211 -0
- explainiverse-0.8.1/src/explainiverse/explainers/gradient/saliency.py +293 -0
- explainiverse-0.8.1/src/explainiverse/explainers/gradient/smoothgrad.py +424 -0
- explainiverse-0.8.1/src/explainiverse/explainers/gradient/tcav.py +865 -0
- explainiverse-0.2.5/PKG-INFO +0 -390
- explainiverse-0.2.5/README.md +0 -359
- explainiverse-0.2.5/src/explainiverse/core/explanation.py +0 -24
- explainiverse-0.2.5/src/explainiverse/engine/suite.py +0 -143
- explainiverse-0.2.5/src/explainiverse/evaluation/__init__.py +0 -8
- explainiverse-0.2.5/src/explainiverse/evaluation/metrics.py +0 -233
- explainiverse-0.2.5/src/explainiverse/explainers/attribution/shap_wrapper.py +0 -89
- explainiverse-0.2.5/src/explainiverse/explainers/gradient/__init__.py +0 -18
- {explainiverse-0.2.5 → explainiverse-0.8.1}/LICENSE +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/adapters/__init__.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/adapters/base_adapter.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/adapters/sklearn_adapter.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/core/__init__.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/core/explainer.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/engine/__init__.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/attribution/__init__.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/attribution/treeshap_wrapper.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/counterfactual/__init__.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/counterfactual/dice_wrapper.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/global_explainers/__init__.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/global_explainers/ale.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/global_explainers/partial_dependence.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/global_explainers/permutation_importance.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/global_explainers/sage.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/gradient/deeplift.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/gradient/gradcam.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/rule_based/__init__.py +0 -0
- {explainiverse-0.2.5 → explainiverse-0.8.1}/src/explainiverse/explainers/rule_based/anchors_wrapper.py +0 -0
|
@@ -0,0 +1,766 @@
|
|
|
1
|
+
Metadata-Version: 2.1
|
|
2
|
+
Name: explainiverse
|
|
3
|
+
Version: 0.8.1
|
|
4
|
+
Summary: Unified, extensible explainability framework supporting 18 XAI methods including LIME, SHAP, LRP, TCAV, GradCAM, and more
|
|
5
|
+
Home-page: https://github.com/jemsbhai/explainiverse
|
|
6
|
+
License: MIT
|
|
7
|
+
Keywords: xai,explainability,interpretability,machine-learning,lime,shap,anchors
|
|
8
|
+
Author: Muntaser Syed
|
|
9
|
+
Author-email: jemsbhai@gmail.com
|
|
10
|
+
Requires-Python: >=3.10,<3.13
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Intended Audience :: Science/Research
|
|
14
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
20
|
+
Provides-Extra: torch
|
|
21
|
+
Requires-Dist: lime (>=0.2.0.1,<0.3.0.0)
|
|
22
|
+
Requires-Dist: numpy (>=1.24,<2.0)
|
|
23
|
+
Requires-Dist: pandas (>=1.5,<3.0)
|
|
24
|
+
Requires-Dist: scikit-learn (>=1.1,<1.6)
|
|
25
|
+
Requires-Dist: scipy (>=1.10,<2.0)
|
|
26
|
+
Requires-Dist: shap (>=0.48.0,<0.49.0)
|
|
27
|
+
Requires-Dist: torch (>=2.0) ; extra == "torch"
|
|
28
|
+
Requires-Dist: xgboost (>=1.7,<3.0)
|
|
29
|
+
Project-URL: Repository, https://github.com/jemsbhai/explainiverse
|
|
30
|
+
Description-Content-Type: text/markdown
|
|
31
|
+
|
|
32
|
+
# Explainiverse
|
|
33
|
+
|
|
34
|
+
[](https://badge.fury.io/py/explainiverse)
|
|
35
|
+
[](https://www.python.org/downloads/)
|
|
36
|
+
[](https://opensource.org/licenses/MIT)
|
|
37
|
+
|
|
38
|
+
**Explainiverse** is a unified, extensible Python framework for Explainable AI (XAI). It provides a standardized interface for **18 state-of-the-art explanation methods** across local, global, gradient-based, concept-based, and example-based paradigms, along with **comprehensive evaluation metrics** for assessing explanation quality.
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
## Key Features
|
|
43
|
+
|
|
44
|
+
| Feature | Description |
|
|
45
|
+
|---------|-------------|
|
|
46
|
+
| **18 Explainers** | LIME, KernelSHAP, TreeSHAP, Integrated Gradients, DeepLIFT, DeepSHAP, SmoothGrad, Saliency Maps, GradCAM/GradCAM++, LRP, TCAV, Anchors, Counterfactual, Permutation Importance, PDP, ALE, SAGE, ProtoDash |
|
|
47
|
+
| **8 Evaluation Metrics** | Faithfulness (PGI, PGU, Comprehensiveness, Sufficiency, Correlation) and Stability (RIS, ROS, Lipschitz) |
|
|
48
|
+
| **Unified API** | Consistent `BaseExplainer` interface with standardized `Explanation` output |
|
|
49
|
+
| **Plugin Registry** | Filter explainers by scope, model type, data type; automatic recommendations |
|
|
50
|
+
| **Framework Support** | Adapters for scikit-learn and PyTorch (with gradient computation) |
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Explainer Coverage
|
|
55
|
+
|
|
56
|
+
### Local Explainers (Instance-Level)
|
|
57
|
+
|
|
58
|
+
| Method | Type | Reference |
|
|
59
|
+
|--------|------|-----------|
|
|
60
|
+
| **LIME** | Perturbation | [Ribeiro et al., 2016](https://arxiv.org/abs/1602.04938) |
|
|
61
|
+
| **KernelSHAP** | Perturbation | [Lundberg & Lee, 2017](https://arxiv.org/abs/1705.07874) |
|
|
62
|
+
| **TreeSHAP** | Exact (Trees) | [Lundberg et al., 2018](https://arxiv.org/abs/1802.03888) |
|
|
63
|
+
| **Integrated Gradients** | Gradient | [Sundararajan et al., 2017](https://arxiv.org/abs/1703.01365) |
|
|
64
|
+
| **DeepLIFT** | Gradient | [Shrikumar et al., 2017](https://arxiv.org/abs/1704.02685) |
|
|
65
|
+
| **DeepSHAP** | Gradient + Shapley | [Lundberg & Lee, 2017](https://arxiv.org/abs/1705.07874) |
|
|
66
|
+
| **SmoothGrad** | Gradient | [Smilkov et al., 2017](https://arxiv.org/abs/1706.03825) |
|
|
67
|
+
| **Saliency Maps** | Gradient | [Simonyan et al., 2014](https://arxiv.org/abs/1312.6034) |
|
|
68
|
+
| **GradCAM / GradCAM++** | Gradient (CNN) | [Selvaraju et al., 2017](https://arxiv.org/abs/1610.02391) |
|
|
69
|
+
| **LRP** | Decomposition | [Bach et al., 2015](https://doi.org/10.1371/journal.pone.0130140) |
|
|
70
|
+
| **TCAV** | Concept-Based | [Kim et al., 2018](https://arxiv.org/abs/1711.11279) |
|
|
71
|
+
| **Anchors** | Rule-Based | [Ribeiro et al., 2018](https://ojs.aaai.org/index.php/AAAI/article/view/11491) |
|
|
72
|
+
| **Counterfactual** | Contrastive | [Mothilal et al., 2020](https://arxiv.org/abs/1905.07697) |
|
|
73
|
+
| **ProtoDash** | Example-Based | [Gurumoorthy et al., 2019](https://arxiv.org/abs/1707.01212) |
|
|
74
|
+
|
|
75
|
+
### Global Explainers (Model-Level)
|
|
76
|
+
|
|
77
|
+
| Method | Type | Reference |
|
|
78
|
+
|--------|------|-----------|
|
|
79
|
+
| **Permutation Importance** | Feature Importance | [Breiman, 2001](https://link.springer.com/article/10.1023/A:1010933404324) |
|
|
80
|
+
| **Partial Dependence (PDP)** | Feature Effect | [Friedman, 2001](https://projecteuclid.org/euclid.aos/1013203451) |
|
|
81
|
+
| **ALE** | Feature Effect | [Apley & Zhu, 2020](https://academic.oup.com/jrsssb/article/82/4/1059/7056085) |
|
|
82
|
+
| **SAGE** | Shapley Importance | [Covert et al., 2020](https://arxiv.org/abs/2004.00668) |
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
|
|
86
|
+
## Evaluation Metrics
|
|
87
|
+
|
|
88
|
+
Explainiverse includes a comprehensive suite of evaluation metrics based on the XAI literature:
|
|
89
|
+
|
|
90
|
+
### Faithfulness Metrics
|
|
91
|
+
|
|
92
|
+
| Metric | Description | Reference |
|
|
93
|
+
|--------|-------------|-----------|
|
|
94
|
+
| **PGI** | Prediction Gap on Important features | [Petsiuk et al., 2018](https://arxiv.org/abs/1806.07421) |
|
|
95
|
+
| **PGU** | Prediction Gap on Unimportant features | [Petsiuk et al., 2018](https://arxiv.org/abs/1806.07421) |
|
|
96
|
+
| **Comprehensiveness** | Drop when removing top-k features | [DeYoung et al., 2020](https://arxiv.org/abs/1911.03429) |
|
|
97
|
+
| **Sufficiency** | Prediction using only top-k features | [DeYoung et al., 2020](https://arxiv.org/abs/1911.03429) |
|
|
98
|
+
| **Faithfulness Correlation** | Correlation between attribution and impact | [Bhatt et al., 2020](https://arxiv.org/abs/2005.00631) |
|
|
99
|
+
|
|
100
|
+
### Stability Metrics
|
|
101
|
+
|
|
102
|
+
| Metric | Description | Reference |
|
|
103
|
+
|--------|-------------|-----------|
|
|
104
|
+
| **RIS** | Relative Input Stability | [Agarwal et al., 2022](https://arxiv.org/abs/2203.06877) |
|
|
105
|
+
| **ROS** | Relative Output Stability | [Agarwal et al., 2022](https://arxiv.org/abs/2203.06877) |
|
|
106
|
+
| **Lipschitz Estimate** | Local Lipschitz continuity | [Alvarez-Melis & Jaakkola, 2018](https://arxiv.org/abs/1806.08049) |
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## Installation
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
# From PyPI
|
|
114
|
+
pip install explainiverse
|
|
115
|
+
|
|
116
|
+
# With PyTorch support (for gradient-based methods)
|
|
117
|
+
pip install explainiverse[torch]
|
|
118
|
+
|
|
119
|
+
# For development
|
|
120
|
+
git clone https://github.com/jemsbhai/explainiverse.git
|
|
121
|
+
cd explainiverse
|
|
122
|
+
poetry install
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
---
|
|
126
|
+
|
|
127
|
+
## Quick Start
|
|
128
|
+
|
|
129
|
+
### Basic Usage with Registry
|
|
130
|
+
|
|
131
|
+
```python
|
|
132
|
+
from explainiverse import default_registry, SklearnAdapter
|
|
133
|
+
from sklearn.ensemble import RandomForestClassifier
|
|
134
|
+
from sklearn.datasets import load_iris
|
|
135
|
+
|
|
136
|
+
# Train a model
|
|
137
|
+
iris = load_iris()
|
|
138
|
+
model = RandomForestClassifier(n_estimators=100, random_state=42)
|
|
139
|
+
model.fit(iris.data, iris.target)
|
|
140
|
+
|
|
141
|
+
# Wrap with adapter
|
|
142
|
+
adapter = SklearnAdapter(model, class_names=iris.target_names.tolist())
|
|
143
|
+
|
|
144
|
+
# List all available explainers
|
|
145
|
+
print(default_registry.list_explainers())
|
|
146
|
+
# ['lime', 'shap', 'treeshap', 'integrated_gradients', 'deeplift', 'deepshap',
|
|
147
|
+
# 'smoothgrad', 'saliency', 'gradcam', 'lrp', 'tcav', 'anchors', 'counterfactual',
|
|
148
|
+
# 'protodash', 'permutation_importance', 'partial_dependence', 'ale', 'sage']
|
|
149
|
+
|
|
150
|
+
# Create an explainer via registry
|
|
151
|
+
explainer = default_registry.create(
|
|
152
|
+
"lime",
|
|
153
|
+
model=adapter,
|
|
154
|
+
training_data=iris.data,
|
|
155
|
+
feature_names=iris.feature_names.tolist(),
|
|
156
|
+
class_names=iris.target_names.tolist()
|
|
157
|
+
)
|
|
158
|
+
|
|
159
|
+
# Generate explanation
|
|
160
|
+
explanation = explainer.explain(iris.data[0])
|
|
161
|
+
print(explanation.explanation_data["feature_attributions"])
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### Filter and Recommend Explainers
|
|
165
|
+
|
|
166
|
+
```python
|
|
167
|
+
# Filter by criteria
|
|
168
|
+
local_explainers = default_registry.filter(scope="local", data_type="tabular")
|
|
169
|
+
neural_explainers = default_registry.filter(model_type="neural")
|
|
170
|
+
image_explainers = default_registry.filter(data_type="image")
|
|
171
|
+
|
|
172
|
+
# Get recommendations
|
|
173
|
+
recommendations = default_registry.recommend(
|
|
174
|
+
model_type="neural",
|
|
175
|
+
data_type="tabular",
|
|
176
|
+
scope_preference="local",
|
|
177
|
+
max_results=5
|
|
178
|
+
)
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
---
|
|
182
|
+
|
|
183
|
+
## Gradient-Based Explainers (PyTorch)
|
|
184
|
+
|
|
185
|
+
### Integrated Gradients
|
|
186
|
+
|
|
187
|
+
```python
|
|
188
|
+
from explainiverse import PyTorchAdapter
|
|
189
|
+
from explainiverse.explainers.gradient import IntegratedGradientsExplainer
|
|
190
|
+
import torch.nn as nn
|
|
191
|
+
|
|
192
|
+
# Define and wrap model
|
|
193
|
+
model = nn.Sequential(
|
|
194
|
+
nn.Linear(10, 64), nn.ReLU(),
|
|
195
|
+
nn.Linear(64, 32), nn.ReLU(),
|
|
196
|
+
nn.Linear(32, 3)
|
|
197
|
+
)
|
|
198
|
+
adapter = PyTorchAdapter(model, task="classification", class_names=["A", "B", "C"])
|
|
199
|
+
|
|
200
|
+
# Create explainer
|
|
201
|
+
explainer = IntegratedGradientsExplainer(
|
|
202
|
+
model=adapter,
|
|
203
|
+
feature_names=[f"feature_{i}" for i in range(10)],
|
|
204
|
+
class_names=["A", "B", "C"],
|
|
205
|
+
n_steps=50,
|
|
206
|
+
method="riemann_trapezoid"
|
|
207
|
+
)
|
|
208
|
+
|
|
209
|
+
# Explain with convergence check
|
|
210
|
+
explanation = explainer.explain(X[0], return_convergence_delta=True)
|
|
211
|
+
print(f"Attributions: {explanation.explanation_data['feature_attributions']}")
|
|
212
|
+
print(f"Convergence δ: {explanation.explanation_data['convergence_delta']:.6f}")
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
### Layer-wise Relevance Propagation (LRP)
|
|
216
|
+
|
|
217
|
+
```python
|
|
218
|
+
from explainiverse.explainers.gradient import LRPExplainer
|
|
219
|
+
|
|
220
|
+
# LRP - Decomposition-based attribution with conservation property
|
|
221
|
+
explainer = LRPExplainer(
|
|
222
|
+
model=adapter,
|
|
223
|
+
feature_names=feature_names,
|
|
224
|
+
class_names=class_names,
|
|
225
|
+
rule="epsilon", # Propagation rule: epsilon, gamma, alpha_beta, z_plus, composite
|
|
226
|
+
epsilon=1e-6 # Stabilization constant
|
|
227
|
+
)
|
|
228
|
+
|
|
229
|
+
# Basic explanation
|
|
230
|
+
explanation = explainer.explain(X[0], target_class=0)
|
|
231
|
+
print(explanation.explanation_data["feature_attributions"])
|
|
232
|
+
|
|
233
|
+
# Verify conservation property (sum of attributions ≈ target output)
|
|
234
|
+
explanation = explainer.explain(X[0], return_convergence_delta=True)
|
|
235
|
+
print(f"Conservation delta: {explanation.explanation_data['convergence_delta']:.6f}")
|
|
236
|
+
|
|
237
|
+
# Compare different LRP rules
|
|
238
|
+
comparison = explainer.compare_rules(X[0], rules=["epsilon", "gamma", "z_plus"])
|
|
239
|
+
for rule, result in comparison.items():
|
|
240
|
+
print(f"{rule}: top feature = {result['top_feature']}")
|
|
241
|
+
|
|
242
|
+
# Layer-wise relevance analysis
|
|
243
|
+
layer_result = explainer.explain_with_layer_relevances(X[0])
|
|
244
|
+
for layer, relevances in layer_result["layer_relevances"].items():
|
|
245
|
+
print(f"{layer}: sum = {sum(relevances):.4f}")
|
|
246
|
+
|
|
247
|
+
# Composite rules: different rules for different layers
|
|
248
|
+
explainer_composite = LRPExplainer(
|
|
249
|
+
model=adapter,
|
|
250
|
+
feature_names=feature_names,
|
|
251
|
+
class_names=class_names,
|
|
252
|
+
rule="composite"
|
|
253
|
+
)
|
|
254
|
+
explainer_composite.set_composite_rule({
|
|
255
|
+
0: "z_plus", # Input layer: focus on what's present
|
|
256
|
+
2: "epsilon", # Middle layers: balanced
|
|
257
|
+
4: "epsilon" # Output layer
|
|
258
|
+
})
|
|
259
|
+
explanation = explainer_composite.explain(X[0])
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
**LRP Propagation Rules:**
|
|
263
|
+
|
|
264
|
+
| Rule | Description | Use Case |
|
|
265
|
+
|------|-------------|----------|
|
|
266
|
+
| `epsilon` | Adds stabilization constant | General purpose (default) |
|
|
267
|
+
| `gamma` | Enhances positive contributions | Image classification |
|
|
268
|
+
| `alpha_beta` | Separates pos/neg (α-β=1) | Fine-grained control |
|
|
269
|
+
| `z_plus` | Only positive weights | Input layers, what's present |
|
|
270
|
+
| `composite` | Different rules per layer | Best practice for deep nets |
|
|
271
|
+
|
|
272
|
+
**Supported Layers:**
|
|
273
|
+
- Linear, Conv2d
|
|
274
|
+
- BatchNorm1d, BatchNorm2d
|
|
275
|
+
- ReLU, LeakyReLU, ELU, Tanh, Sigmoid, GELU
|
|
276
|
+
- MaxPool2d, AvgPool2d, AdaptiveAvgPool2d
|
|
277
|
+
- Flatten, Dropout
|
|
278
|
+
|
|
279
|
+
### DeepLIFT and DeepSHAP
|
|
280
|
+
|
|
281
|
+
```python
|
|
282
|
+
from explainiverse.explainers.gradient import DeepLIFTExplainer, DeepLIFTShapExplainer
|
|
283
|
+
|
|
284
|
+
# DeepLIFT - Fast reference-based attributions
|
|
285
|
+
deeplift = DeepLIFTExplainer(
|
|
286
|
+
model=adapter,
|
|
287
|
+
feature_names=feature_names,
|
|
288
|
+
class_names=class_names,
|
|
289
|
+
baseline=None # Uses zero baseline by default
|
|
290
|
+
)
|
|
291
|
+
explanation = deeplift.explain(X[0])
|
|
292
|
+
|
|
293
|
+
# DeepSHAP - DeepLIFT averaged over background samples
|
|
294
|
+
deepshap = DeepLIFTShapExplainer(
|
|
295
|
+
model=adapter,
|
|
296
|
+
feature_names=feature_names,
|
|
297
|
+
class_names=class_names,
|
|
298
|
+
background_data=X_train[:100]
|
|
299
|
+
)
|
|
300
|
+
explanation = deepshap.explain(X[0])
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
### Saliency Maps
|
|
304
|
+
|
|
305
|
+
```python
|
|
306
|
+
from explainiverse.explainers.gradient import SaliencyExplainer
|
|
307
|
+
|
|
308
|
+
# Saliency Maps - simplest and fastest gradient method
|
|
309
|
+
explainer = SaliencyExplainer(
|
|
310
|
+
model=adapter,
|
|
311
|
+
feature_names=feature_names,
|
|
312
|
+
class_names=class_names,
|
|
313
|
+
absolute_value=True # Default: absolute gradient magnitudes
|
|
314
|
+
)
|
|
315
|
+
|
|
316
|
+
# Standard saliency (absolute gradients)
|
|
317
|
+
explanation = explainer.explain(X[0], method="saliency")
|
|
318
|
+
|
|
319
|
+
# Input × Gradient (gradient scaled by input values)
|
|
320
|
+
explanation = explainer.explain(X[0], method="input_times_gradient")
|
|
321
|
+
|
|
322
|
+
# Signed saliency (keep gradient direction)
|
|
323
|
+
explainer_signed = SaliencyExplainer(
|
|
324
|
+
model=adapter,
|
|
325
|
+
feature_names=feature_names,
|
|
326
|
+
class_names=class_names,
|
|
327
|
+
absolute_value=False
|
|
328
|
+
)
|
|
329
|
+
explanation = explainer_signed.explain(X[0])
|
|
330
|
+
|
|
331
|
+
# Compare all variants
|
|
332
|
+
variants = explainer.compute_all_variants(X[0])
|
|
333
|
+
print(variants["saliency_absolute"])
|
|
334
|
+
print(variants["saliency_signed"])
|
|
335
|
+
print(variants["input_times_gradient"])
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
### SmoothGrad
|
|
339
|
+
|
|
340
|
+
```python
|
|
341
|
+
from explainiverse.explainers.gradient import SmoothGradExplainer
|
|
342
|
+
|
|
343
|
+
# SmoothGrad - Noise-averaged gradients for smoother saliency
|
|
344
|
+
explainer = SmoothGradExplainer(
|
|
345
|
+
model=adapter,
|
|
346
|
+
feature_names=feature_names,
|
|
347
|
+
class_names=class_names,
|
|
348
|
+
n_samples=50,
|
|
349
|
+
noise_scale=0.15,
|
|
350
|
+
noise_type="gaussian" # or "uniform"
|
|
351
|
+
)
|
|
352
|
+
|
|
353
|
+
# Standard SmoothGrad
|
|
354
|
+
explanation = explainer.explain(X[0], method="smoothgrad")
|
|
355
|
+
|
|
356
|
+
# SmoothGrad-Squared (sharper attributions)
|
|
357
|
+
explanation = explainer.explain(X[0], method="smoothgrad_squared")
|
|
358
|
+
|
|
359
|
+
# VarGrad (variance of gradients)
|
|
360
|
+
explanation = explainer.explain(X[0], method="vargrad")
|
|
361
|
+
|
|
362
|
+
# With absolute values
|
|
363
|
+
explanation = explainer.explain(X[0], absolute_value=True)
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
### GradCAM for CNNs
|
|
367
|
+
|
|
368
|
+
```python
|
|
369
|
+
from explainiverse.explainers.gradient import GradCAMExplainer
|
|
370
|
+
|
|
371
|
+
# For CNN models
|
|
372
|
+
adapter = PyTorchAdapter(cnn_model, task="classification", class_names=class_names)
|
|
373
|
+
|
|
374
|
+
explainer = GradCAMExplainer(
|
|
375
|
+
model=adapter,
|
|
376
|
+
target_layer="layer4", # Last conv layer
|
|
377
|
+
class_names=class_names,
|
|
378
|
+
method="gradcam++" # or "gradcam"
|
|
379
|
+
)
|
|
380
|
+
|
|
381
|
+
explanation = explainer.explain(image)
|
|
382
|
+
heatmap = explanation.explanation_data["heatmap"]
|
|
383
|
+
overlay = explainer.get_overlay(original_image, heatmap, alpha=0.5)
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
### TCAV (Concept-Based Explanations)
|
|
387
|
+
|
|
388
|
+
```python
|
|
389
|
+
from explainiverse.explainers.gradient import TCAVExplainer
|
|
390
|
+
|
|
391
|
+
# For neural network models with concept examples
|
|
392
|
+
adapter = PyTorchAdapter(model, task="classification", class_names=class_names)
|
|
393
|
+
|
|
394
|
+
# Create TCAV explainer targeting a specific layer
|
|
395
|
+
explainer = TCAVExplainer(
|
|
396
|
+
model=adapter,
|
|
397
|
+
layer_name="layer3", # Target layer for concept analysis
|
|
398
|
+
class_names=class_names
|
|
399
|
+
)
|
|
400
|
+
|
|
401
|
+
# Learn a concept from examples (e.g., "striped" pattern)
|
|
402
|
+
explainer.learn_concept(
|
|
403
|
+
concept_name="striped",
|
|
404
|
+
concept_examples=striped_images, # Images with stripes
|
|
405
|
+
negative_examples=random_images, # Random images without stripes
|
|
406
|
+
min_accuracy=0.6 # Minimum CAV classifier accuracy
|
|
407
|
+
)
|
|
408
|
+
|
|
409
|
+
# Compute TCAV score: fraction of inputs where concept positively influences prediction
|
|
410
|
+
tcav_score = explainer.compute_tcav_score(
|
|
411
|
+
test_inputs=test_images,
|
|
412
|
+
target_class=0, # e.g., "zebra"
|
|
413
|
+
concept_name="striped"
|
|
414
|
+
)
|
|
415
|
+
print(f"TCAV score: {tcav_score:.3f}") # >0.5 means concept positively influences class
|
|
416
|
+
|
|
417
|
+
# Statistical significance testing against random concepts
|
|
418
|
+
result = explainer.statistical_significance_test(
|
|
419
|
+
test_inputs=test_images,
|
|
420
|
+
target_class=0,
|
|
421
|
+
concept_name="striped",
|
|
422
|
+
n_random=10,
|
|
423
|
+
negative_examples=random_images
|
|
424
|
+
)
|
|
425
|
+
print(f"p-value: {result['p_value']:.4f}, significant: {result['significant']}")
|
|
426
|
+
|
|
427
|
+
# Full explanation with multiple concepts
|
|
428
|
+
explanation = explainer.explain(
|
|
429
|
+
test_inputs=test_images,
|
|
430
|
+
target_class=0,
|
|
431
|
+
run_significance_test=True
|
|
432
|
+
)
|
|
433
|
+
print(explanation.explanation_data["tcav_scores"])
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
---
|
|
437
|
+
|
|
438
|
+
## Example-Based Explanations
|
|
439
|
+
|
|
440
|
+
### ProtoDash
|
|
441
|
+
|
|
442
|
+
```python
|
|
443
|
+
from explainiverse.explainers.example_based import ProtoDashExplainer
|
|
444
|
+
|
|
445
|
+
explainer = ProtoDashExplainer(
|
|
446
|
+
model=adapter,
|
|
447
|
+
training_data=X_train,
|
|
448
|
+
feature_names=feature_names,
|
|
449
|
+
n_prototypes=5,
|
|
450
|
+
kernel="rbf",
|
|
451
|
+
gamma=0.1
|
|
452
|
+
)
|
|
453
|
+
|
|
454
|
+
explanation = explainer.explain(X_test[0])
|
|
455
|
+
print(explanation.explanation_data["prototype_indices"])
|
|
456
|
+
print(explanation.explanation_data["prototype_weights"])
|
|
457
|
+
```
|
|
458
|
+
|
|
459
|
+
---
|
|
460
|
+
|
|
461
|
+
## Evaluation Metrics
|
|
462
|
+
|
|
463
|
+
### Faithfulness Evaluation
|
|
464
|
+
|
|
465
|
+
```python
|
|
466
|
+
from explainiverse.evaluation import (
|
|
467
|
+
compute_pgi, compute_pgu,
|
|
468
|
+
compute_comprehensiveness, compute_sufficiency,
|
|
469
|
+
compute_faithfulness_correlation
|
|
470
|
+
)
|
|
471
|
+
|
|
472
|
+
# PGI - Higher is better (important features affect predictions)
|
|
473
|
+
pgi = compute_pgi(
|
|
474
|
+
model=adapter,
|
|
475
|
+
instance=X[0],
|
|
476
|
+
attributions=attributions,
|
|
477
|
+
feature_names=feature_names,
|
|
478
|
+
top_k=3
|
|
479
|
+
)
|
|
480
|
+
|
|
481
|
+
# PGU - Lower is better (unimportant features don't affect predictions)
|
|
482
|
+
pgu = compute_pgu(
|
|
483
|
+
model=adapter,
|
|
484
|
+
instance=X[0],
|
|
485
|
+
attributions=attributions,
|
|
486
|
+
feature_names=feature_names,
|
|
487
|
+
top_k=3
|
|
488
|
+
)
|
|
489
|
+
|
|
490
|
+
# Comprehensiveness - Higher is better
|
|
491
|
+
comp = compute_comprehensiveness(
|
|
492
|
+
model=adapter,
|
|
493
|
+
instance=X[0],
|
|
494
|
+
attributions=attributions,
|
|
495
|
+
feature_names=feature_names,
|
|
496
|
+
top_k_values=[1, 2, 3, 5]
|
|
497
|
+
)
|
|
498
|
+
|
|
499
|
+
# Sufficiency - Lower is better
|
|
500
|
+
suff = compute_sufficiency(
|
|
501
|
+
model=adapter,
|
|
502
|
+
instance=X[0],
|
|
503
|
+
attributions=attributions,
|
|
504
|
+
feature_names=feature_names,
|
|
505
|
+
top_k_values=[1, 2, 3, 5]
|
|
506
|
+
)
|
|
507
|
+
|
|
508
|
+
# Faithfulness Correlation
|
|
509
|
+
corr = compute_faithfulness_correlation(
|
|
510
|
+
model=adapter,
|
|
511
|
+
instance=X[0],
|
|
512
|
+
attributions=attributions,
|
|
513
|
+
feature_names=feature_names
|
|
514
|
+
)
|
|
515
|
+
```
|
|
516
|
+
|
|
517
|
+
### Stability Evaluation
|
|
518
|
+
|
|
519
|
+
```python
|
|
520
|
+
from explainiverse.evaluation import (
|
|
521
|
+
compute_ris, compute_ros, compute_lipschitz_estimate
|
|
522
|
+
)
|
|
523
|
+
|
|
524
|
+
# RIS - Relative Input Stability (lower is better)
|
|
525
|
+
ris = compute_ris(
|
|
526
|
+
explainer=explainer,
|
|
527
|
+
instance=X[0],
|
|
528
|
+
n_perturbations=10,
|
|
529
|
+
perturbation_scale=0.1
|
|
530
|
+
)
|
|
531
|
+
|
|
532
|
+
# ROS - Relative Output Stability (lower is better)
|
|
533
|
+
ros = compute_ros(
|
|
534
|
+
model=adapter,
|
|
535
|
+
explainer=explainer,
|
|
536
|
+
instance=X[0],
|
|
537
|
+
n_perturbations=10,
|
|
538
|
+
perturbation_scale=0.1
|
|
539
|
+
)
|
|
540
|
+
|
|
541
|
+
# Lipschitz Estimate (lower is better)
|
|
542
|
+
lipschitz = compute_lipschitz_estimate(
|
|
543
|
+
explainer=explainer,
|
|
544
|
+
instance=X[0],
|
|
545
|
+
n_perturbations=20,
|
|
546
|
+
perturbation_scale=0.1
|
|
547
|
+
)
|
|
548
|
+
```
|
|
549
|
+
|
|
550
|
+
---
|
|
551
|
+
|
|
552
|
+
## Global Explainers
|
|
553
|
+
|
|
554
|
+
```python
|
|
555
|
+
from explainiverse.explainers import (
|
|
556
|
+
PermutationImportanceExplainer,
|
|
557
|
+
PartialDependenceExplainer,
|
|
558
|
+
ALEExplainer,
|
|
559
|
+
SAGEExplainer
|
|
560
|
+
)
|
|
561
|
+
|
|
562
|
+
# Permutation Importance
|
|
563
|
+
perm_imp = PermutationImportanceExplainer(
|
|
564
|
+
model=adapter,
|
|
565
|
+
X=X_test,
|
|
566
|
+
y=y_test,
|
|
567
|
+
feature_names=feature_names,
|
|
568
|
+
n_repeats=10
|
|
569
|
+
)
|
|
570
|
+
explanation = perm_imp.explain()
|
|
571
|
+
|
|
572
|
+
# Partial Dependence Plot
|
|
573
|
+
pdp = PartialDependenceExplainer(
|
|
574
|
+
model=adapter,
|
|
575
|
+
X=X_train,
|
|
576
|
+
feature_names=feature_names
|
|
577
|
+
)
|
|
578
|
+
explanation = pdp.explain(feature="feature_0", grid_resolution=50)
|
|
579
|
+
|
|
580
|
+
# ALE (handles correlated features)
|
|
581
|
+
ale = ALEExplainer(
|
|
582
|
+
model=adapter,
|
|
583
|
+
X=X_train,
|
|
584
|
+
feature_names=feature_names
|
|
585
|
+
)
|
|
586
|
+
explanation = ale.explain(feature="feature_0", n_bins=20)
|
|
587
|
+
|
|
588
|
+
# SAGE (global Shapley importance)
|
|
589
|
+
sage = SAGEExplainer(
|
|
590
|
+
model=adapter,
|
|
591
|
+
X=X_train,
|
|
592
|
+
y=y_train,
|
|
593
|
+
feature_names=feature_names,
|
|
594
|
+
n_permutations=512
|
|
595
|
+
)
|
|
596
|
+
explanation = sage.explain()
|
|
597
|
+
```
|
|
598
|
+
|
|
599
|
+
---
|
|
600
|
+
|
|
601
|
+
## Multi-Explainer Comparison
|
|
602
|
+
|
|
603
|
+
```python
|
|
604
|
+
from explainiverse import ExplanationSuite
|
|
605
|
+
|
|
606
|
+
suite = ExplanationSuite(
|
|
607
|
+
model=adapter,
|
|
608
|
+
explainer_configs=[
|
|
609
|
+
("lime", {"training_data": X_train, "feature_names": feature_names, "class_names": class_names}),
|
|
610
|
+
("shap", {"background_data": X_train[:50], "feature_names": feature_names, "class_names": class_names}),
|
|
611
|
+
("treeshap", {"feature_names": feature_names, "class_names": class_names}),
|
|
612
|
+
]
|
|
613
|
+
)
|
|
614
|
+
|
|
615
|
+
results = suite.run(X_test[0])
|
|
616
|
+
suite.compare()
|
|
617
|
+
```
|
|
618
|
+
|
|
619
|
+
---
|
|
620
|
+
|
|
621
|
+
## Custom Explainer Registration
|
|
622
|
+
|
|
623
|
+
```python
|
|
624
|
+
from explainiverse import default_registry, ExplainerMeta, BaseExplainer, Explanation
|
|
625
|
+
|
|
626
|
+
@default_registry.register_decorator(
|
|
627
|
+
name="my_explainer",
|
|
628
|
+
meta=ExplainerMeta(
|
|
629
|
+
scope="local",
|
|
630
|
+
model_types=["any"],
|
|
631
|
+
data_types=["tabular"],
|
|
632
|
+
task_types=["classification", "regression"],
|
|
633
|
+
description="My custom explainer",
|
|
634
|
+
paper_reference="Author et al., 2024",
|
|
635
|
+
complexity="O(n)",
|
|
636
|
+
requires_training_data=False,
|
|
637
|
+
supports_batching=True
|
|
638
|
+
)
|
|
639
|
+
)
|
|
640
|
+
class MyExplainer(BaseExplainer):
|
|
641
|
+
def __init__(self, model, feature_names, **kwargs):
|
|
642
|
+
super().__init__(model)
|
|
643
|
+
self.feature_names = feature_names
|
|
644
|
+
|
|
645
|
+
def explain(self, instance, **kwargs):
|
|
646
|
+
# Your implementation
|
|
647
|
+
attributions = self._compute_attributions(instance)
|
|
648
|
+
return Explanation(
|
|
649
|
+
explainer_name="MyExplainer",
|
|
650
|
+
target_class="output",
|
|
651
|
+
explanation_data={"feature_attributions": attributions}
|
|
652
|
+
)
|
|
653
|
+
```
|
|
654
|
+
|
|
655
|
+
---
|
|
656
|
+
|
|
657
|
+
## Architecture
|
|
658
|
+
|
|
659
|
+
```
|
|
660
|
+
explainiverse/
|
|
661
|
+
├── core/
|
|
662
|
+
│ ├── explainer.py # BaseExplainer abstract class
|
|
663
|
+
│ ├── explanation.py # Unified Explanation container
|
|
664
|
+
│ └── registry.py # ExplainerRegistry with metadata
|
|
665
|
+
├── adapters/
|
|
666
|
+
│ ├── sklearn_adapter.py
|
|
667
|
+
│ └── pytorch_adapter.py # With gradient support
|
|
668
|
+
├── explainers/
|
|
669
|
+
│ ├── attribution/ # LIME, SHAP, TreeSHAP
|
|
670
|
+
│ ├── gradient/ # IG, DeepLIFT, DeepSHAP, SmoothGrad, Saliency, GradCAM, LRP, TCAV
|
|
671
|
+
│ ├── rule_based/ # Anchors
|
|
672
|
+
│ ├── counterfactual/ # DiCE-style
|
|
673
|
+
│ ├── global_explainers/ # Permutation, PDP, ALE, SAGE
|
|
674
|
+
│ └── example_based/ # ProtoDash
|
|
675
|
+
├── evaluation/
|
|
676
|
+
│ ├── faithfulness.py # PGI, PGU, Comprehensiveness, Sufficiency
|
|
677
|
+
│ └── stability.py # RIS, ROS, Lipschitz
|
|
678
|
+
└── engine/
|
|
679
|
+
└── suite.py # Multi-explainer comparison
|
|
680
|
+
```
|
|
681
|
+
|
|
682
|
+
---
|
|
683
|
+
|
|
684
|
+
## Running Tests
|
|
685
|
+
|
|
686
|
+
```bash
|
|
687
|
+
# Run all tests
|
|
688
|
+
poetry run pytest
|
|
689
|
+
|
|
690
|
+
# Run with coverage
|
|
691
|
+
poetry run pytest --cov=explainiverse --cov-report=html
|
|
692
|
+
|
|
693
|
+
# Run specific test file
|
|
694
|
+
poetry run pytest tests/test_lrp.py -v
|
|
695
|
+
|
|
696
|
+
# Run specific test class
|
|
697
|
+
poetry run pytest tests/test_lrp.py::TestLRPConv2d -v
|
|
698
|
+
```
|
|
699
|
+
|
|
700
|
+
---
|
|
701
|
+
|
|
702
|
+
## Roadmap
|
|
703
|
+
|
|
704
|
+
### Completed ✅
|
|
705
|
+
- [x] Core framework (BaseExplainer, Explanation, Registry)
|
|
706
|
+
- [x] Perturbation methods: LIME, KernelSHAP, TreeSHAP
|
|
707
|
+
- [x] Gradient methods: Integrated Gradients, DeepLIFT, DeepSHAP, SmoothGrad, Saliency Maps, GradCAM/GradCAM++
|
|
708
|
+
- [x] Decomposition methods: Layer-wise Relevance Propagation (LRP) with ε, γ, αβ, z⁺, composite rules
|
|
709
|
+
- [x] Concept-based: TCAV (Testing with Concept Activation Vectors)
|
|
710
|
+
- [x] Rule-based: Anchors
|
|
711
|
+
- [x] Counterfactual: DiCE-style
|
|
712
|
+
- [x] Global: Permutation Importance, PDP, ALE, SAGE
|
|
713
|
+
- [x] Example-based: ProtoDash
|
|
714
|
+
- [x] Evaluation: Faithfulness metrics (PGI, PGU, Comprehensiveness, Sufficiency, Correlation)
|
|
715
|
+
- [x] Evaluation: Stability metrics (RIS, ROS, Lipschitz)
|
|
716
|
+
- [x] PyTorch adapter with gradient support
|
|
717
|
+
|
|
718
|
+
### Planned 📋
|
|
719
|
+
- [ ] Attention-based explanations (for Transformers)
|
|
720
|
+
- [ ] TensorFlow/Keras adapter
|
|
721
|
+
- [ ] Interactive visualization dashboard
|
|
722
|
+
- [ ] Explanation caching and serialization
|
|
723
|
+
- [ ] Distributed computation support
|
|
724
|
+
|
|
725
|
+
---
|
|
726
|
+
|
|
727
|
+
## Citation
|
|
728
|
+
|
|
729
|
+
If you use Explainiverse in your research, please cite:
|
|
730
|
+
|
|
731
|
+
```bibtex
|
|
732
|
+
@software{explainiverse2025,
|
|
733
|
+
title = {Explainiverse: A Unified Framework for Explainable AI},
|
|
734
|
+
author = {Syed, Muntaser},
|
|
735
|
+
year = {2025},
|
|
736
|
+
url = {https://github.com/jemsbhai/explainiverse},
|
|
737
|
+
version = {0.8.0}
|
|
738
|
+
}
|
|
739
|
+
```
|
|
740
|
+
|
|
741
|
+
---
|
|
742
|
+
|
|
743
|
+
## Contributing
|
|
744
|
+
|
|
745
|
+
Contributions are welcome! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
|
|
746
|
+
|
|
747
|
+
1. Fork the repository
|
|
748
|
+
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
|
|
749
|
+
3. Write tests for your changes
|
|
750
|
+
4. Ensure all tests pass (`poetry run pytest`)
|
|
751
|
+
5. Commit your changes (`git commit -m 'Add amazing feature'`)
|
|
752
|
+
6. Push to the branch (`git push origin feature/amazing-feature`)
|
|
753
|
+
7. Open a Pull Request
|
|
754
|
+
|
|
755
|
+
---
|
|
756
|
+
|
|
757
|
+
## License
|
|
758
|
+
|
|
759
|
+
MIT License - see [LICENSE](LICENSE) for details.
|
|
760
|
+
|
|
761
|
+
---
|
|
762
|
+
|
|
763
|
+
## Acknowledgments
|
|
764
|
+
|
|
765
|
+
Explainiverse builds upon the foundational work of many researchers in the XAI community. We thank the authors of LIME, SHAP, Integrated Gradients, DeepLIFT, LRP, GradCAM, TCAV, Anchors, DiCE, ALE, SAGE, and ProtoDash for their contributions to interpretable machine learning.
|
|
766
|
+
|