sofi-tabular 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 The SOFI authors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,465 @@
1
+ Metadata-Version: 2.4
2
+ Name: sofi-tabular
3
+ Version: 1.0.0
4
+ Summary: Sparseness Optimized Feature Importance, a post hoc explainer for classification and regression
5
+ Author-email: Isel Grau <i.d.c.grau.garcia@tue.nl>, Gonzalo Napoles <g.r.napoles@tilburguniversity.edu>
6
+ License: MIT License
7
+
8
+ Copyright (c) 2026 The SOFI authors
9
+
10
+ Permission is hereby granted, free of charge, to any person obtaining a copy
11
+ of this software and associated documentation files (the "Software"), to deal
12
+ in the Software without restriction, including without limitation the rights
13
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
14
+ copies of the Software, and to permit persons to whom the Software is
15
+ furnished to do so, subject to the following conditions:
16
+
17
+ The above copyright notice and this permission notice shall be included in all
18
+ copies or substantial portions of the Software.
19
+
20
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
21
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
22
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
23
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
24
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
25
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
26
+ SOFTWARE.
27
+
28
+ Keywords: explainable ai,feature importance,interpretability,sparsity
29
+ Classifier: Intended Audience :: Science/Research
30
+ Classifier: License :: OSI Approved :: MIT License
31
+ Classifier: Programming Language :: Python :: 3
32
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
33
+ Requires-Python: >=3.9
34
+ Description-Content-Type: text/markdown
35
+ License-File: LICENSE
36
+ Requires-Dist: numpy>=1.23
37
+ Requires-Dist: pandas>=1.5
38
+ Requires-Dist: scikit-learn>=1.1
39
+ Provides-Extra: plot
40
+ Requires-Dist: matplotlib>=3.6; extra == "plot"
41
+ Requires-Dist: seaborn>=0.12; extra == "plot"
42
+ Provides-Extra: progress
43
+ Requires-Dist: tqdm>=4.64; extra == "progress"
44
+ Provides-Extra: demo
45
+ Requires-Dist: matplotlib>=3.6; extra == "demo"
46
+ Requires-Dist: seaborn>=0.12; extra == "demo"
47
+ Requires-Dist: tqdm>=4.64; extra == "demo"
48
+ Requires-Dist: nbformat; extra == "demo"
49
+ Requires-Dist: nbclient; extra == "demo"
50
+ Requires-Dist: ipykernel; extra == "demo"
51
+ Dynamic: license-file
52
+
53
+ # Sparseness Optimized Feature Importance
54
+
55
+ Sparseness Optimized Feature Importance (SOFI) is a model agnostic, declarative post hoc explainer. An explanation takes the form of a ranking of features, and its quality is the degradation score obtained after cumulative marginalization. The implementation supports classification and regression, operates at the instance level or over a whole dataset, and marginalizes one-hot encoded features as indivisible blocks.
56
+
57
+ ## Installation
58
+
59
+ ```bash
60
+ pip install sofi-tabular
61
+ ```
62
+
63
+ The package requires Python 3.9 or later, together with `numpy`, `pandas` and `scikit-learn`,
64
+ which is all `pip install sofi-tabular` pulls in. Plotting, progress bars and the demo are
65
+ optional extras declared in `pyproject.toml`:
66
+
67
+ ```bash
68
+ pip install "sofi-tabular[plot]" # matplotlib and seaborn, needed by explanation.plot()
69
+ pip install "sofi-tabular[progress]" # tqdm, needed for progress bars
70
+ pip install "sofi-tabular[demo]" # everything above, plus what SOFI_demo.ipynb needs to run
71
+ ```
72
+
73
+ Progress bars need `tqdm`, figures need `matplotlib`, and `seaborn` is used only to fix the
74
+ font of a session. `requirements.txt` pins all of them together for a quick `pip install -r
75
+ requirements.txt` when the distinction does not matter.
76
+
77
+ To install the development version from GitHub:
78
+
79
+ ```bash
80
+ git clone https://github.com/igraugar/sofi.git
81
+ cd sofi
82
+ pip install -e ".[demo]"
83
+ ```
84
+
85
+ ## Quick start
86
+
87
+ ```python
88
+ from sofi import SOFIExplainer, select_reliable_instances, set_plot_style
89
+
90
+ set_plot_style(font_scale=1.15) # once per session
91
+ validation = select_reliable_instances(model, X_test, y_test) # instances the model gets right
92
+
93
+ explainer = SOFIExplainer(model, X_train, random_state=42)
94
+ explanation = explainer.explain(validation, mode="global")
95
+
96
+ print(explanation.summary())
97
+ explanation.plot()
98
+ ```
99
+
100
+ `SOFI_demo.ipynb` develops four complete examples on datasets read from public GitHub
101
+ repositories, each of them with more than 15 features. They cover classification and
102
+ regression, features that are mixed, purely nominal or purely numerical, and the four ways of
103
+ telling the explainer how features map onto the columns the model consumes.
104
+
105
+ | Example | Task | Features | Encoding |
106
+ | --- | --- | --- | --- |
107
+ | 1. Bank marketing | classification | 19, mixed | pipeline carrying the encoder |
108
+ | 2. Mushroom | classification | 21, only nominal | encoder passed apart |
109
+ | 3. Telecom charges | regression | 18, mixed | encoded data with feature groups |
110
+ | 4. College tuition | regression | 16, only numerical | model passed directly |
111
+
112
+ Each example reports the whole set of metrics twice over, for the global explanation and for
113
+ four individual instances chosen so that their explanations disagree as much as possible with
114
+ the global one and with each other. The instances alternate the two marginalization variants,
115
+ so a single notebook shows both of them at work.
116
+
117
+ ## The degradation score
118
+
119
+ Two curves are derived from a ranking. The MoRF curve marginalizes the most relevant
120
+ features first and should collapse immediately, since a sparse explanation concentrates the
121
+ response of the model in a handful of features. The LeRF curve marginalizes the least
122
+ relevant features first and should stay flat for as long as possible, since the features
123
+ declared irrelevant must indeed be the ones whose removal leaves the response untouched.
124
+ Both curves share their first point, the unperturbed response, and their last point, the
125
+ response once every feature has been neutralized. The degradation score is the area between
126
+ the two curves, and the search maximizes it. A single curve cannot capture both properties:
127
+ the MoRF branch alone rewards sparsity while saying nothing about the tail of the ranking, and
128
+ the LeRF branch alone rewards correctness at the tail while saying nothing about the head.
129
+
130
+ The search works on the raw response of the model and on nothing else, as in region
131
+ perturbation. Classification tracks the probability that the model assigns to the class it
132
+ predicted before any perturbation. Regression tracks the mean absolute deviation with respect
133
+ to the unperturbed estimate, taken with a negative sign so that both quantities fall as
134
+ evidence is destroyed. Neither task consults the ground truth, which keeps explanations
135
+ independent of the error of the model.
136
+
137
+ Normalization never enters the optimization, since a scale that depends on the ranking being
138
+ scored would change what is being maximized. It is applied afterwards, for reporting and for
139
+ drawing, and it follows the convention of the perturbation curve literature: the response of
140
+ the unperturbed input is placed at one and the response of the fully perturbed input at zero.
141
+ Both anchors belong to the model and the data rather than to a ranking, so every ranking
142
+ scored on the same data shares them, the reported score is the optimized score divided by one
143
+ positive constant, and no ordering can change. Reported curves therefore start at one and stay
144
+ inside the unit interval, so figures from different instances, models and problems can be read
145
+ on the same axis.
146
+
147
+ The anchors are read off a reference sweep that marginalizes every feature on its own and then
148
+ follows the resulting greedy ordering cumulatively, which also produces the fully perturbed
149
+ response. In the ordinary case the two anchors are exactly the two responses the convention
150
+ prescribes. Should the response climb substantially above the unperturbed one, the scale
151
+ widens to the extremes the sweep observed and a warning names the cause, so that a curve is
152
+ never flattened against a false ceiling. When the response never falls at all, a warning says
153
+ so too, because the degradation score then carries no information.
154
+
155
+ ## Marginalizing a feature
156
+
157
+ Marginalization values come from the training data alone, and the variant matters more than
158
+ it may appear. The background variant averages the response over a fixed sample of training
159
+ rows (`n_background` of them, 25 by default), which approximates the expectation the method is
160
+ defined on, and it is the default. The constant variant substitutes one statistic instead,
161
+ which costs one query per step rather than one per background draw.
162
+
163
+ The saving is real and so is the risk. Replacing every feature of an instance with its most
164
+ frequent category and its mean value produces the single most typical row in the training
165
+ data, and some models classify that row with more confidence than the original instance. When
166
+ that happens, the fully perturbed response ends up above the unperturbed one instead of below
167
+ it, which is exactly the situation the reference sweep described above is built to detect and
168
+ warn about. Averaging over a background sample largely removes the effect, since the response
169
+ then approaches the marginal output of the model rather than the output at one atypical point.
170
+
171
+ Raise `n_background` for a smoother expectation and lower it, or subsample the validation set,
172
+ when the search has to run quickly. The whole validation set is replicated once per draw, so
173
+ the cost grows linearly with the sample.
174
+
175
+ ## The noise region
176
+
177
+ Once the informative features are gone, marginalizing what remains often pushes the response
178
+ back towards its original state. The MoRF curve then climbs after its minimum, and the tail
179
+ of the ranking that follows that minimum carries no evidence. The region is reported by
180
+ `noise_onset` and `noise_features`, and it also appears in the printed summary. The figure
181
+ leaves it unshaded, so that the only shaded region is the degradation score itself.
182
+
183
+ ## The search
184
+
185
+ Hill climbing with a local operator that swaps two randomly selected positions of the current
186
+ ranking. A candidate is accepted when it raises the degradation score. The starting ranking
187
+ is either random or greedy, the latter reusing the ordering already produced by the reference
188
+ sweep, at no extra cost. An explicit ranking is also accepted, which is how prior knowledge
189
+ enters the procedure. The run ends after a budget of iterations or once the patience expires,
190
+ and restarts from a random ranking replace an early stop when `n_restarts` is positive.
191
+
192
+ ## Declaring the structure of the features
193
+
194
+ Marginalizing a single column of a one-hot block would measure the role of one category
195
+ rather than the role of the feature. SOFI therefore needs to know which columns form a
196
+ logical feature, and four arrangements are available. The demo devotes one example to each.
197
+
198
+ **A pipeline that carries the encoder.** Nothing has to be declared. Perturbation happens in
199
+ the original feature space and the encoder is applied afterwards, so a category is always
200
+ replaced with another complete category. This path is the recommended one.
201
+
202
+ ```python
203
+ model = Pipeline([("encoder", column_transformer), ("forest", RandomForestClassifier())])
204
+ model.fit(X_train, y_train)
205
+ explainer = SOFIExplainer(model, X_train)
206
+ ```
207
+
208
+ **A fitted encoder passed apart.** Use it when the estimator was trained on encoded data and
209
+ cannot be wrapped. Perturbation still happens in the original space.
210
+
211
+ ```python
212
+ explainer = SOFIExplainer(model, X_train_raw, encoder=column_transformer)
213
+ ```
214
+
215
+ **Data that is already encoded.** Declare the blocks through `feature_groups`, either
216
+ explicitly or recovered from a fitted `ColumnTransformer`. Columns that are not listed become
217
+ features of their own.
218
+
219
+ ```python
220
+ from sofi import feature_groups_from_encoder
221
+
222
+ groups = feature_groups_from_encoder(column_transformer)
223
+ explainer = SOFIExplainer(model, X_train_encoded, feature_groups=groups)
224
+ ```
225
+
226
+ **No encoding at all.** A dataset described only by numerical features needs none of the
227
+ above, and the estimator reaches the explainer directly.
228
+
229
+ ```python
230
+ explainer = SOFIExplainer(regressor, X_train)
231
+ ```
232
+
233
+ One difference between the paths deserves attention. Marginalization statistics follow the
234
+ dtypes of the data handed to the explainer, and a `ColumnTransformer` returns a single
235
+ floating point matrix regardless of the dtypes it was fed, so an integer column that went
236
+ through a passthrough transformer receives the statistic of a continuous variable in the third
237
+ path. Restore the original dtypes on the encoded frame when exact agreement matters.
238
+
239
+ ## Containers and estimators
240
+
241
+ The explainer records the container the estimator was fitted with and adapts every internal
242
+ query accordingly, so a model trained on a `DataFrame` keeps receiving named columns in the
243
+ original order and a model trained on an array keeps receiving an array. Input given to
244
+ `explain` may be a `DataFrame`, a `Series` or an array. Estimators outside the scikit-learn
245
+ interface are supported through `predict_fn` and `proba_fn`.
246
+
247
+ A prediction is attempted at construction time on up to two training rows, and a failure
248
+ raises a message that names the three paths above, which turns an encoding mismatch into an
249
+ immediate and readable error rather than a silent misinterpretation.
250
+
251
+ ## Local and global modes
252
+
253
+ `mode="global"` searches for one ranking that degrades the average response over the
254
+ supplied instances. `mode="local"` runs one search per instance, returning a single
255
+ explanation for one row and a list of explanations for several rows. Instance level results
256
+ are summarized with `aggregate_explanations`, which reports the mean rank of every feature
257
+ and how often it reaches the first three positions.
258
+
259
+ Training data is always required, since it is the only source of marginalization values. The
260
+ data explained afterwards never contributes statistics.
261
+
262
+ ## Choosing the validation set
263
+
264
+ The premise of the method is that the response deteriorates as features are marginalized,
265
+ which only holds for instances the model handles well. The selection is deliberately left
266
+ outside the explainer, and `select_reliable_instances` covers the two usual criteria, namely
267
+ correct classification and small absolute error.
268
+
269
+ ```python
270
+ validation = select_reliable_instances(model, X_test, y_test) # classifier
271
+ validation = select_reliable_instances(model, X_test, y_test, quantile=0.5) # regressor
272
+ ```
273
+
274
+ ## Comparing SOFI against other explainers
275
+
276
+ `explainer.score_ranking` evaluates a ranking produced elsewhere under the same protocol and
277
+ on the same reported scale, which makes the degradation score a common ground for comparison.
278
+ Any attribution method that ends in an ordering of features can be measured this way.
279
+
280
+ ```python
281
+ scored = explainer.score_ranking(other_ranking, validation)
282
+ print(scored.ds, explanation.ds)
283
+ ```
284
+
285
+ ## Figures
286
+
287
+ Font sizes are never touched by the plotting routine. One call to `set_plot_style` fixes them
288
+ for the whole session, and the title, the axes, the ticks, the legend and the annotation
289
+ follow that single setting. Colours work the same way. One call to `set_curve_colors` holds
290
+ for every figure afterwards (the session starts with `#03719c` for MoRF and `#1A1A1A` for
291
+ LeRF), and a single figure can depart from it through the `morf_color` and `lerf_color`
292
+ arguments of `plot`.
293
+
294
+ ```python
295
+ from sofi import set_curve_colors, set_plot_style
296
+
297
+ set_plot_style(font_scale=1.15)
298
+ set_curve_colors(morf="#03719c", lerf="#16120e")
299
+
300
+ explanation.plot() # the colours in force
301
+ explanation.plot(lerf_color="#865321") # this figure alone
302
+ ```
303
+
304
+ Both curves are drawn on a fixed axis that spans the unit interval, labeled `Prediction
305
+ fidelity` for either task, the legend reads `MoRF` and `LeRF` in a single row underneath the
306
+ axes, and the only shaded region is the area between the curves. Several explanations are
307
+ drawn side by side with `plot_explanation_grid`, which packs the panels as closely as the
308
+ labels allow and takes the same colour arguments.
309
+
310
+ ## Cost and reproducibility
311
+
312
+ Building the objective for a dataset costs two sweeps of the features: one to marginalize each
313
+ feature alone, which produces the greedy ordering, and one to follow that ordering
314
+ cumulatively. Every candidate proposed afterwards requires the two curves, so an evaluation
315
+ costs twice a single curve. A curve issues one batched query per chunk of `chunk_size` rows,
316
+ and a candidate produced by a swap at positions `i` and `j` (with `i < j`) reuses the first `i`
317
+ points of the MoRF curve of the incumbent and the first `n - 1 - j` points of its LeRF curve,
318
+ since the reversed ranking is affected at mirrored positions. Only the affected tails are
319
+ recomputed, and the result is identical to a full evaluation.
320
+
321
+ Every run is reproducible from `random_state`, which controls the background sample, the swap
322
+ operator and the restarts. No internal state is modified during a search, so repeated calls
323
+ on the same data return the same explanation.
324
+
325
+ Global mode on a large validation set is the expensive setting, since every point of both
326
+ curves predicts the whole set. Subsampling the validation data is the usual remedy.
327
+
328
+ ## Reference of `SOFIExplainer`
329
+
330
+ | Parameter | Default | Accepted values | Rationale |
331
+ | --- | --- | --- | --- |
332
+ | `model` | required | Any fitted estimator exposing `predict`, plus `predict_proba` for classification. A `Pipeline` carrying the encoder is the recommended form. | The explainer is agnostic to the model family and only queries its outputs. |
333
+ | `X_train` | required | `DataFrame`, `Series` or two dimensional array. | Source of the marginalization statistics. A frame is preferred, since dtypes drive the choice of statistic. |
334
+ | `encoder` | `None` | `None`, or any fitted transformer exposing `transform`. | Declares the second path. The transformer is applied after every perturbation, so blocks are rebuilt from complete categories. `None` means that either the model carries its encoder or no encoding is needed. |
335
+ | `feature_groups` | `None` | `None`, a dict mapping a feature name onto a list of columns, or `"auto"`. | Declares the third path, for data that is already encoded. `"auto"` recovers the mapping from the `ColumnTransformer` given in `encoder`. Columns left out become features of their own. |
336
+ | `task` | `"auto"` | `"auto"`, `"classification"`, `"regression"`. | `"auto"` reads the task from the estimator and is enough for scikit-learn objects. The explicit values are needed for wrappers whose type cannot be inferred. |
337
+ | `predict_fn` | `None` | `None` or a callable receiving the adapted input and returning an array. | Escape hatch for estimators outside the scikit-learn interface. |
338
+ | `proba_fn` | `None` | `None` or a callable returning a matrix of class probabilities. | Same escape hatch for classifiers whose probabilities are not exposed through `predict_proba`. |
339
+ | `marginalization` | `"background"` | `"background"`, `"constant"`. | `"background"` averages the response over a sample of training rows, which approximates the expectation the method is defined on. `"constant"` substitutes one statistic instead, which costs one query per step rather than one per draw, at the risk of placing an instance in a region the model reads with confidence. |
340
+ | `numeric_strategy` | `"mean"` | `"mean"`, `"median"`, `"mode"`. | Statistic of the floating point columns. `"mean"` neutralizes a feature at its center of mass, `"median"` resists skewed distributions and outliers, and `"mode"` keeps a value that the feature actually takes. |
341
+ | `integer_strategy` | `"mode"` | `"mode"`, `"median"`, `"mean"`. | Statistic of the integer columns. `"mode"` is the default because it preserves the dtype and produces a value the feature can take, such as a count. The other two are rounded, with a warning, when the column must stay integer. |
342
+ | `categorical_strategy` | `"mode"` | `"mode"`. | Statistic of nominal, boolean and categorical columns. Only the most frequent category is admissible, since no average of labels exists. A multi column block declared through `feature_groups` receives the most frequent joint pattern, which is always a valid state of the block. |
343
+ | `n_background` | `25` | Integer of at least one, capped at the number of training rows. | Size of the background sample. Larger values reduce the variance of the expectation and raise the cost linearly, since the validation set is replicated once per draw. Ignored under the constant variant. |
344
+ | `initialization` | `"greedy"` | `"greedy"`, `"random"`, or a sequence of feature names or positions. | `"greedy"` starts from the ordering produced by the reference sweep, which is already available and usually reaches a good region within few swaps. `"random"` removes that bias and suits repeated runs with restarts. An explicit sequence lets prior knowledge enter the search. |
345
+ | `max_iterations` | `200` | Non-negative integer. | Budget of proposed swaps across all restarts. Zero evaluates the initial ranking without any search, which is convenient for scoring a ranking chosen beforehand. |
346
+ | `patience` | `None` | `None`, or a positive integer. | Consecutive swaps without improvement tolerated before a restart or the end of the search. `None` sets it to the whole budget, so the search never stops early. |
347
+ | `n_restarts` | `0` | Integer of at least zero. | Restarts from a random ranking granted once the patience expires. Zero ends the run instead, and larger values trade coverage of the search space for time. |
348
+ | `accept_equal` | `False` | `False`, `True`. | Whether swaps that leave the score unchanged are accepted. `True` lets the search drift along plateaus, which helps when many features are redundant, and it makes the trajectory less stable. |
349
+ | `chunk_size` | `20000` | Positive integer. | Upper bound on the rows sent to the estimator in one query. Larger values reduce the call overhead and raise the peak memory, and lower values suit estimators with a costly batch. |
350
+ | `random_state` | `None` | `None`, an integer, or a `numpy` `Generator`. | Seed of the background sample, of the swap operator and of the restarts. An integer makes a run reproducible, and `None` draws fresh randomness. |
351
+ | `verbose` | `True` | `True`, `False`. | Whether progress bars are displayed, which requires `tqdm`. |
352
+
353
+ Two further methods report how the explainer resolved the arguments above:
354
+
355
+ | Method | Returns | Purpose |
356
+ | --- | --- | --- |
357
+ | `feature_map()` | `DataFrame` with `feature`, `n_columns`, `columns` | One row per logical feature, showing how many columns it occupies and which ones, useful for checking that `feature_groups` or the encoder were understood as intended. |
358
+ | `marginalization_values()` | `DataFrame` with `feature`, `column`, `value` | The statistic that replaces each column under the constant variant. Not populated by the background draws, which are resampled internally rather than fixed per feature. |
359
+
360
+ ## Reference of `explain`
361
+
362
+ | Parameter | Default | Accepted values | Rationale |
363
+ | --- | --- | --- | --- |
364
+ | `X` | required | `DataFrame`, `Series` or two dimensional array, expressed in the space of the training data. | Instances to be explained. A single row triggers one search and a frame triggers one search per row in local mode. |
365
+ | `mode` | `"global"` | `"global"`, `"local"`. | `"global"` seeks one ranking for the averaged response of the whole set, which describes the model. `"local"` seeks one ranking per instance, which describes a decision. |
366
+ | `initialization`, `max_iterations`, `patience`, `n_restarts`, `random_state` | `None` | Same values as in the constructor, or `None`. | Per call overrides. `None` keeps the value given at construction time, so the same explainer can be reused under different budgets. |
367
+
368
+ The method returns a `SOFIExplanation` for a single instance or for the global mode, and a
369
+ list of them when several instances are explained locally.
370
+
371
+ ## Reference of `select_reliable_instances`
372
+
373
+ | Parameter | Default | Accepted values | Rationale |
374
+ | --- | --- | --- | --- |
375
+ | `model`, `X`, `y` | required | Fitted estimator, a `DataFrame` and an array of targets. | The subset is built outside the explainer, so the criterion stays under the control of the user. |
376
+ | `encoder` | `None` | `None`, or a fitted transformer. | Applied before the estimator when the model does not carry its encoder. |
377
+ | `task` | `"auto"` | `"auto"`, `"classification"`, `"regression"`. | Chooses between the correctness criterion and the error criterion. |
378
+ | `quantile` | `0.5` | Float between zero and one. | Fraction of regression instances retained, ordered by absolute error. Ignored for classifiers and superseded by `tolerance`. |
379
+ | `tolerance` | `None` | `None`, or a non-negative float. | Absolute error threshold on the original scale of the target, which is preferable when a domain rule defines what a small error is. |
380
+ | `return_mask` | `False` | `False`, `True`. | Whether the boolean mask is returned as well, which is how the matching targets are selected. |
381
+
382
+ ## Reference of `SOFIExplanation`
383
+
384
+ | Attribute | Meaning |
385
+ | --- | --- |
386
+ | `ranking`, `order` | Features sorted from the most to the least important, by name and by position |
387
+ | `top(k)` | First `k` features of the ranking |
388
+ | `morf_scores`, `lerf_scores` | Reported curves of the ranking and of its reverse, inside the unit interval |
389
+ | `morf_scores_raw`, `lerf_scores_raw` | The same curves on the raw response of the model |
390
+ | `ds_raw` | The integral on the raw scale, which is the quantity the search maximizes |
391
+ | `anchor_high`, `anchor_low` | Responses placed at one and at zero when a curve is reported |
392
+ | `scores` | Alias of `morf_scores` |
393
+ | `ds` | Degradation score, namely the area between the two curves, maximized by the search |
394
+ | `auc_morf`, `auc_lerf` | Mean height of each curve |
395
+ | `drops` | Degradation attributable to each step of the MoRF curve |
396
+ | `importances` | Rank based importance in the unit interval |
397
+ | `noise_onset`, `noise_features` | Start of the region where fidelity recovers, and the features it holds |
398
+ | `statistics` | Every figure of the run gathered in one dictionary |
399
+ | `summary(top_k=None)` | Report meant to be printed, also returned by `print(explanation)` |
400
+ | `to_frame()` | Tabular view of the ranking |
401
+ | `plot()` | Both curves, the enclosed area and the noise region |
402
+ | `history` | One record per iteration of the search |
403
+ | `n_iterations`, `n_restarts_used`, `n_evaluations`, `n_model_calls` | Counters describing the run |
404
+
405
+ ## Reference of `plot` and `set_plot_style`
406
+
407
+ | Parameter | Default | Accepted values | Rationale |
408
+ | --- | --- | --- | --- |
409
+ | `ax` of `plot` | `None` | `None`, or a matplotlib `Axes`. | `None` creates a figure, and an explicit axes places the curves inside a larger layout, such as a two by two grid of local explanations. |
410
+ | `title` of `plot` | `None` | `None`, or a string. | `None` reports the mode of the run. |
411
+ | `annotate_ds` of `plot` | `True` | `True`, `False`. | Whether the degradation score is written inside the axes. |
412
+ | `legend` of `plot` | `True` | `True`, `False`. | Whether the two curves are labeled. The legend is laid out as a single row underneath the axes, so it never covers the curves. |
413
+ | `figsize` of `plot` | `(6.6, 4.2)` | Pair of floats. | Size of the figure created when no axes is supplied. |
414
+ | `morf_color`, `lerf_color` of `plot` | `None` | Any colour matplotlib accepts, such as `"#c0392b"` or `"tab:red"`. | Colour of each curve for one figure. The colours in force for the session apply when they are left out. |
415
+ | `morf`, `lerf` of `set_curve_colors` | `None` | The same values. | Colour of each curve for the whole session. An argument left out keeps the colour in force. |
416
+ | `font_scale` of `set_plot_style` | `1.2` | Positive float. | Multiplier applied to every font of the session. The figure itself never overrides it. |
417
+ | `style`, `context` of `set_plot_style` | `"whitegrid"`, `"notebook"` | Any seaborn style and context name. | Passed to `seaborn.set_theme` when seaborn is installed, and ignored otherwise. |
418
+
419
+ ## Advanced building blocks
420
+
421
+ `SOFIExplainer` is a thin orchestrator over four pieces, all importable from `sofi` for anyone
422
+ who wants to reuse part of the machinery outside the explainer, for instance to plug a custom
423
+ search into the existing objective:
424
+
425
+ | Name | What it does |
426
+ | --- | --- |
427
+ | `FeatureSpace` | Holds the mapping from logical features to columns, together with the transform applied after a perturbation. Returned by `SOFIExplainer.feature_map()` in tabular form. |
428
+ | `Marginalizer` | Computes and applies the values that neutralize a feature, under either variant. |
429
+ | `Objective` | Turns a ranking into a `RankingEvaluation` by driving the model through cumulative marginalization, and owns the anchors described under "The degradation score". |
430
+ | `hill_climbing` | The search itself, decoupled from SOFI's objective: it accepts any `evaluate` callable that returns an object exposing `.order` and `.ds`. |
431
+ | `curve_auc`, `degradation_score` | The scoring functions applied to any pair of MoRF/LeRF curves, independent of how they were produced. |
432
+ | `RankingEvaluation` | The dataclass (`order`, `morf`, `lerf`, `ds`) passed between the objective and the search. |
433
+ | `noise_onset` | The free function behind `SOFIExplanation.noise_onset`, usable directly on any curve. |
434
+
435
+ Most users never need to import these directly; `SOFIExplainer.explain` and `.score_ranking`
436
+ cover the ordinary usage.
437
+
438
+ ## Citation
439
+
440
+ ```bibtex
441
+ @inproceedings{grau2024sofi,
442
+ title = {Sparseness-Optimized Feature Importance},
443
+ author = {Grau, Isel and N{\'a}poles, Gonzalo},
444
+ booktitle = {Explainable Artificial Intelligence. xAI 2024},
445
+ series = {Communications in Computer and Information Science},
446
+ volume = {2154},
447
+ pages = {393--415},
448
+ publisher = {Springer},
449
+ year = {2024},
450
+ doi = {10.1007/978-3-031-63797-1_20}
451
+ }
452
+
453
+ @article{grau2026sofits,
454
+ title = {Sparseness-Optimized Feature Importance for Time Series Classification},
455
+ author = {Grau, Isel and N{\'a}poles, Gonzalo and Jastrzebska, Agnieszka and Salgueiro, Yamisleydi},
456
+ journal = {IEEE Access},
457
+ volume = {14},
458
+ pages = {29874--29893},
459
+ year = {2026}
460
+ }
461
+ ```
462
+
463
+ ## License
464
+
465
+ MIT. See `LICENSE`.