neural-feature-importance 0.5.2__tar.gz → 0.9.1__tar.gz

neural_feature_importance-0.9.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 CR de Sá
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

{neural_feature_importance-0.5.2/neural_feature_importance.egg-info → neural_feature_importance-0.9.1}/PKG-INFO

@@ -1,20 +1,23 @@
 Metadata-Version: 2.4
 Name: neural-feature-importance
-Version: 0.5.2
+Version: 0.9.1
 Summary: Variance-based feature importance for Neural Networks using callbacks for Keras and PyTorch
 Author: CR de Sá
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
+License-File: LICENSE
 Requires-Dist: numpy
 Provides-Extra: tensorflow
 Requires-Dist: tensorflow; extra == "tensorflow"
 Provides-Extra: torch
 Requires-Dist: torch; extra == "torch"
+Dynamic: license-file
 
 # neural-feature-importance
 
 [![PyPI version](https://img.shields.io/pypi/v/neural-feature-importance.svg)](https://pypi.org/project/neural-feature-importance/)
-[![Python versions](https://img.shields.io/pypi/pyversions/neural-feature-importance.svg)](https://pypi.org/project/neural-feature-importance/)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 
 Variance-based feature importance for deep learning models.
 
@@ -74,19 +77,44 @@ print(tracker.feature_importances_)
 
 ## Example scripts
 
-Run `compare_feature_importance.py` to train a small network on the Iris dataset
+Run `scripts/compare_feature_importance.py` to train a small network on the Iris dataset
 and compare the scores with a random forest baseline:
 
 ```bash
 python compare_feature_importance.py
 ```
 
-Run `full_experiment.py` to reproduce the experiments from the paper:
+Run `scripts/full_experiment.py` to reproduce the experiments from the paper:
 
 ```bash
 python full_experiment.py
 ```
 
+### Convolutional models
+
+To compute importances for convolutional networks, use
+`ConvVarianceImportanceKeras` from `neural_feature_importance.conv_callbacks`.
+`scripts/conv_visualization_example.py` trains small Conv2D models on the MNIST
+and scikit‑learn digits datasets and displays per-filter heatmaps. An equivalent
+notebook is available in ``notebooks/conv_visualization_example.ipynb``:
+
+```bash
+python scripts/conv_visualization_example.py
+```
+
+### Embedding layers
+
+To compute token importances from embedding weights, use
+`EmbeddingVarianceImportanceKeras` or `EmbeddingVarianceImportanceTorch` from
+`neural_feature_importance.embedding_callbacks`.
+Run `scripts/token_importance_topk_example.py` to train a small text classifier
+on IMDB and display the most important tokens. A matching notebook lives in
+``notebooks/token_importance_topk_example.ipynb``:
+
+```bash
+python scripts/token_importance_topk_example.py
+```
+
 ## Development
 
 After making changes, run the following checks:
@@ -124,3 +152,7 @@ If you use this package in your research, please cite:
 ```
 
 We appreciate citations as they help the community discover this work.
+
+## License
+
+This project is licensed under the [MIT License](LICENSE).

{neural_feature_importance-0.5.2 → neural_feature_importance-0.9.1}/README.md

@@ -1,7 +1,8 @@
 # neural-feature-importance
 
 [![PyPI version](https://img.shields.io/pypi/v/neural-feature-importance.svg)](https://pypi.org/project/neural-feature-importance/)
-[![Python versions](https://img.shields.io/pypi/pyversions/neural-feature-importance.svg)](https://pypi.org/project/neural-feature-importance/)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 
 Variance-based feature importance for deep learning models.
 
@@ -61,19 +62,44 @@ print(tracker.feature_importances_)
 
 ## Example scripts
 
-Run `compare_feature_importance.py` to train a small network on the Iris dataset
+Run `scripts/compare_feature_importance.py` to train a small network on the Iris dataset
 and compare the scores with a random forest baseline:
 
 ```bash
 python compare_feature_importance.py
 ```
 
-Run `full_experiment.py` to reproduce the experiments from the paper:
+Run `scripts/full_experiment.py` to reproduce the experiments from the paper:
 
 ```bash
 python full_experiment.py
 ```
 
+### Convolutional models
+
+To compute importances for convolutional networks, use
+`ConvVarianceImportanceKeras` from `neural_feature_importance.conv_callbacks`.
+`scripts/conv_visualization_example.py` trains small Conv2D models on the MNIST
+and scikit‑learn digits datasets and displays per-filter heatmaps. An equivalent
+notebook is available in ``notebooks/conv_visualization_example.ipynb``:
+
+```bash
+python scripts/conv_visualization_example.py
+```
+
+### Embedding layers
+
+To compute token importances from embedding weights, use
+`EmbeddingVarianceImportanceKeras` or `EmbeddingVarianceImportanceTorch` from
+`neural_feature_importance.embedding_callbacks`.
+Run `scripts/token_importance_topk_example.py` to train a small text classifier
+on IMDB and display the most important tokens. A matching notebook lives in
+``notebooks/token_importance_topk_example.ipynb``:
+
+```bash
+python scripts/token_importance_topk_example.py
+```
+
 ## Development
 
 After making changes, run the following checks:
@@ -111,3 +137,7 @@ If you use this package in your research, please cite:
 ```
 
 We appreciate citations as they help the community discover this work.
+
+## License
+
+This project is licensed under the [MIT License](LICENSE).

neural_feature_importance-0.9.1/conv_visualization_example.py

@@ -0,0 +1,60 @@
+"""Example of variance-based importance with a Conv2D model."""
+
+from __future__ import annotations
+
+import logging
+
+import matplotlib.pyplot as plt
+from tensorflow.keras.datasets import mnist
+from tensorflow.keras.layers import Conv2D, Dense, Flatten, MaxPooling2D
+from tensorflow.keras.models import Sequential
+from tensorflow.keras.utils import to_categorical
+
+from neural_feature_importance.conv_callbacks import ConvVarianceImportanceKeras
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+
+def build_model() -> Sequential:
+    """Return a minimal Conv2D model."""
+    model = Sequential(
+        [
+            Conv2D(8, (3, 3), activation="relu", input_shape=(28, 28, 1)),
+            MaxPooling2D((2, 2)),
+            Flatten(),
+            Dense(10, activation="softmax"),
+        ]
+    )
+    model.compile(
+        optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"]
+    )
+    return model
+
+
+def main() -> None:
+    """Train model on MNIST and display a heatmap of importances."""
+    (x_train, y_train), _ = mnist.load_data()
+    x_train = x_train.astype("float32") / 255.0
+    x_train = x_train[..., None]
+    y_train = to_categorical(y_train, 10)
+
+    model = build_model()
+    callback = ConvVarianceImportanceKeras()
+    model.fit(x_train, y_train, epochs=2, batch_size=128, callbacks=[callback], verbose=0)
+
+    scores = callback.feature_importances_
+    if scores is None:
+        logger.warning("No importance scores computed.")
+        return
+
+    weights = model.layers[0].get_weights()[0]
+    heatmap = scores.reshape(weights.shape[0], weights.shape[1], weights.shape[2]).mean(axis=-1)
+    plt.imshow(heatmap, cmap="hot")
+    plt.colorbar()
+    plt.title("Feature importance heatmap")
+    plt.show()
+
+
+if __name__ == "__main__":
+    main()

neural_feature_importance-0.9.1/conv_viz_utils.py

@@ -0,0 +1,123 @@
+"""Helper utilities for convolutional importance visualizations.
+
+These functions build small convolutional models, compute filter scores using
+variance-based importances, and plot the resulting weight maps and activations.
+They are intended for exploratory analysis and example scripts.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Iterable
+
+import matplotlib.pyplot as plt
+import matplotlib.colors as mcolors
+import numpy as np
+from tensorflow.keras.layers import Conv2D, Dense, Flatten, MaxPooling2D
+from tensorflow.keras.models import Sequential
+
+logger = logging.getLogger(__name__)
+
+CMAP = mcolors.LinearSegmentedColormap.from_list("white_blue_black", ["white", "blue", "black"])
+WEIGHT_CMAP = plt.cm.seismic
+
+
+def build_model(input_shape: tuple[int, int, int], kernel_size: tuple[int, int]) -> Sequential:
+    """Return a simple Conv2D model for visualization experiments.
+
+    Parameters
+    ----------
+    input_shape:
+        Shape of the input images, e.g. ``(28, 28, 1)``.
+    kernel_size:
+        Size of the convolution kernel.
+    """
+    model = Sequential(
+        [
+            Conv2D(8, kernel_size, activation="relu", input_shape=input_shape),
+            MaxPooling2D((2, 2)),
+            Flatten(),
+            Dense(10, activation="softmax"),
+        ]
+    )
+    model.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])
+    return model
+
+
+def _threshold_filters(weights: np.ndarray, threshold: float) -> np.ndarray:
+    """Binarize filter weights using a threshold."""
+    mask = np.abs(weights) >= threshold
+    return np.where(mask, weights, 0.0)
+
+
+def compute_filter_scores(
+    weights: np.ndarray, heatmap: np.ndarray, threshold: float
+) -> tuple[np.ndarray, np.ndarray]:
+    """Compute filter importance scores.
+
+    Filters are thresholded and multiplied by the variance heatmap before
+    summing over all spatial dimensions.
+    """
+    thr_weights = _threshold_filters(weights, threshold)
+    scores = np.sum(np.abs(thr_weights) * heatmap[..., None], axis=(0, 1, 2))
+    return scores.astype(float), thr_weights
+
+
+def rank_filters(weights: np.ndarray, heatmap: np.ndarray, threshold: float) -> np.ndarray:
+    """Return filter indices sorted by descending importance."""
+    scores, _ = compute_filter_scores(weights, heatmap, threshold)
+    order = np.argsort(scores)[::-1]
+    logger.info("Filter scores: %s", scores.tolist())
+    return order
+
+
+def accuracy_with_filters(
+    model: Sequential,
+    x: np.ndarray,
+    y: np.ndarray,
+    indices: Iterable[int],
+) -> float:
+    """Return accuracy when only selected filters are active."""
+    conv = model.layers[0]
+    original = conv.get_weights()
+    weights = original[0].copy()
+    bias = original[1].copy()
+    mask = np.zeros(weights.shape[-1], dtype=bool)
+    mask[list(indices)] = True
+    weights[..., ~mask] = 0.0
+    bias[~mask] = 0.0
+    conv.set_weights([weights, bias])
+    preds = np.argmax(model.predict(x, verbose=0), axis=1)
+    acc = float(np.mean(preds == np.argmax(y, axis=1)))
+    conv.set_weights(original)
+    return acc
+
+
+def plot_filters(
+    weights: np.ndarray,
+    heatmap: np.ndarray,
+    example_out: np.ndarray,
+    order: Iterable[int],
+) -> None:
+    """Display filter weights, importances, and outputs."""
+    n_filters = weights.shape[-1]
+    vmax = float(np.max(np.abs(weights)))
+    fig, axes = plt.subplots(n_filters, 3, figsize=(9, 3 * n_filters))
+    for row, idx in enumerate(order):
+        ax_w = axes[row, 0]
+        ax_i = axes[row, 1]
+        ax_o = axes[row, 2]
+        im_w = ax_w.imshow(weights[:, :, 0, idx], cmap=WEIGHT_CMAP, vmin=-vmax, vmax=vmax)
+        ax_w.set_title(f"Filter {idx} weights")
+        ax_w.axis("off")
+        fig.colorbar(im_w, ax=ax_w)
+        im_i = ax_i.imshow(heatmap[:, :, 0], cmap=CMAP, vmin=0.0, vmax=1.0)
+        ax_i.set_title("Importance")
+        ax_i.axis("off")
+        fig.colorbar(im_i, ax=ax_i)
+        im_o = ax_o.imshow(example_out[:, :, idx], cmap="gray", vmin=0.0, vmax=np.max(example_out))
+        ax_o.set_title("Filter output")
+        ax_o.axis("off")
+        fig.colorbar(im_o, ax=ax_o)
+    plt.tight_layout()
+    plt.show()

{neural_feature_importance-0.5.2 → neural_feature_importance-0.9.1}/neural_feature_importance/__init__.py

@@ -7,6 +7,11 @@ from .callbacks import (
     VarianceImportanceKeras,
     VarianceImportanceTorch,
 )
+from .conv_callbacks import ConvVarianceImportanceKeras, ConvVarianceImportanceTorch
+from .embedding_callbacks import (
+    EmbeddingVarianceImportanceKeras,
+    EmbeddingVarianceImportanceTorch,
+)
 from .utils import MetricThreshold
 
 try:
@@ -19,4 +24,8 @@ __all__ = [
     "VarianceImportanceKeras",
     "VarianceImportanceTorch",
     "MetricThreshold",
+    "ConvVarianceImportanceKeras",
+    "ConvVarianceImportanceTorch",
+    "EmbeddingVarianceImportanceKeras",
+    "EmbeddingVarianceImportanceTorch",
 ]

{neural_feature_importance-0.5.2 → neural_feature_importance-0.9.1}/neural_feature_importance/callbacks.py

@@ -1,4 +1,10 @@
-"""Variance-based feature importance utilities."""
+"""Utilities for computing variance-based feature importances.
+
+These classes track the weights of the first trainable layer during training
+and estimate feature importances by accumulating the variance of each weight
+value. After training, the variances are combined with the last observed
+weights to produce a normalized importance score for every input feature.
+"""
 
 from __future__ import annotations
 
@@ -13,7 +19,13 @@ logger = logging.getLogger(__name__)
 
 
 class VarianceImportanceBase:
-    """Compute feature importance using Welford's algorithm."""
+    """Compute feature importances using running variance statistics.
+
+    The class implements Welford's algorithm to accumulate the variance of
+    weight values over training iterations. Feature importances are derived by
+    combining the final variance estimates with the absolute value of the last
+    observed weights.
+    """
 
     def __init__(self) -> None:
         self._n = 0
@@ -23,13 +35,21 @@ class VarianceImportanceBase:
         self.var_scores: np.ndarray | None = None
 
     def start(self, weights: np.ndarray) -> None:
-        """Initialize statistics for the given weight matrix."""
+        """Initialize running statistics.
+
+        Parameters
+        ----------
+        weights:
+            Initial weight matrix of shape ``(features, outputs)``. The values
+            are converted to ``float64`` for numerical stability and the running
+            mean and variance buffers are reset.
+        """
         self._mean = weights.astype(np.float64)
         self._m2 = np.zeros_like(self._mean)
         self._n = 0
 
     def update(self, weights: np.ndarray) -> None:
-        """Update running statistics with new weights."""
+        """Update running mean and variance using new weights."""
         if self._mean is None or self._m2 is None:
             return
         self._n += 1
@@ -40,7 +60,7 @@ class VarianceImportanceBase:
         self._last_weights = weights
 
     def finalize(self) -> None:
-        """Finalize statistics and compute normalized scores."""
+        """Compute normalized importance scores from accumulated statistics."""
         if self._last_weights is None or self._m2 is None:
             logger.warning(
                 "%s was not fully initialized; no scores computed", self.__class__.__name__
@@ -67,6 +87,7 @@ class VarianceImportanceBase:
         return self.var_scores
 
 
+
 class VarianceImportanceKeras(Callback, VarianceImportanceBase):
     """Keras callback implementing variance-based feature importance."""
 

neural_feature_importance-0.9.1/neural_feature_importance/conv_callbacks.py

@@ -0,0 +1,104 @@
+"""Callbacks that extend variance tracking to convolutional layers."""
+
+from __future__ import annotations
+
+import logging
+from typing import Optional
+
+import numpy as np
+
+from .callbacks import VarianceImportanceKeras, VarianceImportanceTorch
+
+logger = logging.getLogger(__name__)
+
+
+def _flatten_weights(weights: np.ndarray, outputs_last: bool) -> np.ndarray:
+    """Return a two-dimensional view of convolutional kernels.
+
+    Parameters
+    ----------
+    weights:
+        Weight tensor from a convolutional layer. Expected shape is
+        ``(H, W, in_channels, out_channels)`` when ``outputs_last`` is ``True``
+        and ``(out_channels, in_channels, H, W)`` otherwise.
+    outputs_last:
+        Whether the output dimension is the last axis of ``weights``.
+
+    Returns
+    -------
+    np.ndarray
+        Array of shape ``(features, outputs)`` suitable for variance tracking.
+    """
+    if weights.ndim > 2:
+        if outputs_last:
+            return weights.reshape(-1, weights.shape[-1])
+        return weights.reshape(weights.shape[0], -1).T
+    return weights
+
+
+class ConvVarianceImportanceKeras(VarianceImportanceKeras):
+    """Keras callback that tracks convolutional kernels.
+
+    The first trainable layer is inspected and, if its weights have more than
+    two dimensions, they are flattened so that each spatial location and input
+    channel is treated as a separate feature. Variances are accumulated during
+    training and converted to per-filter importance scores.
+    """
+    def on_train_begin(self, logs: Optional[dict] = None) -> None:
+        self._layer = None
+        for layer in self.model.layers:
+            has_vars = bool(layer.trainable_weights)
+            has_data = bool(layer.get_weights())
+            if has_vars and has_data:
+                self._layer = layer
+                break
+        if self._layer is None:
+            raise ValueError("Model does not contain trainable weights.")
+        weights = self._layer.get_weights()[0]
+        weights = _flatten_weights(weights, outputs_last=True)
+        logger.info(
+            "Tracking variance for layer '%s' with %d features",
+            self._layer.name,
+            weights.shape[0],
+        )
+        self.start(weights)
+
+    def on_epoch_end(self, epoch: int, logs: Optional[dict] = None) -> None:
+        if self._layer is None:
+            return
+        weights = self._layer.get_weights()[0]
+        weights = _flatten_weights(weights, outputs_last=True)
+        self.update(weights)
+
+
+class ConvVarianceImportanceTorch(VarianceImportanceTorch):
+    """PyTorch helper with convolutional support.
+
+    Works analogously to :class:`ConvVarianceImportanceKeras` but for models
+    built with :mod:`torch.nn`. The first trainable parameter with two or more
+    dimensions is flattened so each spatial position becomes a tracked feature.
+    """
+    def on_train_begin(self) -> None:
+        from torch import nn
+
+        for name, param in self.model.named_parameters():
+            if param.requires_grad and param.dim() >= 2:
+                self._param = param
+                weights = param.detach().cpu().numpy()
+                weights = _flatten_weights(weights, outputs_last=False)
+                logger.info(
+                    "Tracking variance for parameter '%s' with %d features",
+                    name,
+                    weights.shape[0],
+                )
+                self.start(weights)
+                break
+        if self._param is None:
+            raise ValueError("Model does not contain trainable parameters")
+
+    def on_epoch_end(self) -> None:
+        if self._param is None:
+            return
+        weights = self._param.detach().cpu().numpy()
+        weights = _flatten_weights(weights, outputs_last=False)
+        self.update(weights)

neural_feature_importance-0.9.1/neural_feature_importance/embedding_callbacks.py

@@ -0,0 +1,85 @@
+"""Callbacks that compute variance-based importance for embedding layers.
+
+These callbacks extend :class:`~neural_feature_importance.callbacks.VarianceImportanceBase`
+to operate on 2-D embedding matrices. The variance of each embedding vector is
+accumulated over training and the resulting per-token scores are normalized
+between 0 and 1.
+"""
+
+from __future__ import annotations
+
+import logging
+
+import numpy as np
+
+from .callbacks import VarianceImportanceKeras, VarianceImportanceTorch
+
+logger = logging.getLogger(__name__)
+
+
+class EmbeddingVarianceImportanceKeras(VarianceImportanceKeras):
+    """Variance-based importance callback for Keras embedding layers.
+
+    During training this callback monitors the weights of the first trainable
+    layer (expected to be an :class:`~tensorflow.keras.layers.Embedding`) and
+    accumulates the running variance of each embedding vector. After training the
+    variances are summed across the embedding dimension to yield a single score
+    per token.
+    """
+
+    def finalize(self) -> None:  # type: ignore[override]
+        if self._last_weights is None or self._m2 is None:
+            logger.warning(
+                "%s was not fully initialized; no scores computed",
+                self.__class__.__name__,
+            )
+            return
+
+        if self._n < 2:
+            variance = np.full_like(self._m2, np.nan)
+        else:
+            variance = self._m2 / (self._n - 1)
+
+        scores = np.sum(variance, axis=1)
+        min_val = float(np.nanmin(scores))
+        max_val = float(np.nanmax(scores))
+        denom = max_val - min_val if max_val != min_val else 1.0
+        self.var_scores = (scores - min_val) / denom
+
+        top = np.argsort(self.var_scores)[-10:][::-1]
+        logger.info("Most important tokens: %s", top)
+
+
+class EmbeddingVarianceImportanceTorch(VarianceImportanceTorch):
+    """Variance-based importance for PyTorch embedding layers.
+
+    Parameters
+    ----------
+    model:
+        Neural network containing an :class:`torch.nn.Embedding` layer whose
+        weights will be monitored.
+    """
+
+    def finalize(self) -> None:  # type: ignore[override]
+        if self._last_weights is None or self._m2 is None:
+            logger.warning(
+                "%s was not fully initialized; no scores computed",
+                self.__class__.__name__,
+            )
+            return
+
+        if self._n < 2:
+            variance = np.full_like(self._m2, np.nan)
+        else:
+            variance = self._m2 / (self._n - 1)
+
+        scores = np.sum(variance, axis=1)
+        min_val = float(np.nanmin(scores))
+        max_val = float(np.nanmax(scores))
+        denom = max_val - min_val if max_val != min_val else 1.0
+        self.var_scores = (scores - min_val) / denom
+
+        top = np.argsort(self.var_scores)[-10:][::-1]
+        logger.info("Most important tokens: %s", top)
+
+

{neural_feature_importance-0.5.2 → neural_feature_importance-0.9.1/neural_feature_importance.egg-info}/PKG-INFO

@@ -1,20 +1,23 @@
 Metadata-Version: 2.4
 Name: neural-feature-importance
-Version: 0.5.2
+Version: 0.9.1
 Summary: Variance-based feature importance for Neural Networks using callbacks for Keras and PyTorch
 Author: CR de Sá
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
+License-File: LICENSE
 Requires-Dist: numpy
 Provides-Extra: tensorflow
 Requires-Dist: tensorflow; extra == "tensorflow"
 Provides-Extra: torch
 Requires-Dist: torch; extra == "torch"
+Dynamic: license-file
 
 # neural-feature-importance
 
 [![PyPI version](https://img.shields.io/pypi/v/neural-feature-importance.svg)](https://pypi.org/project/neural-feature-importance/)
-[![Python versions](https://img.shields.io/pypi/pyversions/neural-feature-importance.svg)](https://pypi.org/project/neural-feature-importance/)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
 
 Variance-based feature importance for deep learning models.
 
@@ -74,19 +77,44 @@ print(tracker.feature_importances_)
 
 ## Example scripts
 
-Run `compare_feature_importance.py` to train a small network on the Iris dataset
+Run `scripts/compare_feature_importance.py` to train a small network on the Iris dataset
 and compare the scores with a random forest baseline:
 
 ```bash
 python compare_feature_importance.py
 ```
 
-Run `full_experiment.py` to reproduce the experiments from the paper:
+Run `scripts/full_experiment.py` to reproduce the experiments from the paper:
 
 ```bash
 python full_experiment.py
 ```
 
+### Convolutional models
+
+To compute importances for convolutional networks, use
+`ConvVarianceImportanceKeras` from `neural_feature_importance.conv_callbacks`.
+`scripts/conv_visualization_example.py` trains small Conv2D models on the MNIST
+and scikit‑learn digits datasets and displays per-filter heatmaps. An equivalent
+notebook is available in ``notebooks/conv_visualization_example.ipynb``:
+
+```bash
+python scripts/conv_visualization_example.py
+```
+
+### Embedding layers
+
+To compute token importances from embedding weights, use
+`EmbeddingVarianceImportanceKeras` or `EmbeddingVarianceImportanceTorch` from
+`neural_feature_importance.embedding_callbacks`.
+Run `scripts/token_importance_topk_example.py` to train a small text classifier
+on IMDB and display the most important tokens. A matching notebook lives in
+``notebooks/token_importance_topk_example.ipynb``:
+
+```bash
+python scripts/token_importance_topk_example.py
+```
+
 ## Development
 
 After making changes, run the following checks:
@@ -124,3 +152,7 @@ If you use this package in your research, please cite:
 ```
 
 We appreciate citations as they help the community discover this work.
+
+## License
+
+This project is licensed under the [MIT License](LICENSE).

neural_feature_importance-0.9.1/neural_feature_importance.egg-info/SOURCES.txt

@@ -0,0 +1,27 @@
+AGENTS.md
+LICENSE
+README.md
+conv_visualization_example.py
+conv_viz_utils.py
+pyproject.toml
+text_classification_example.py
+token_topk_utils.py
+.github/workflows/python-publish.yml
+neural_feature_importance/__init__.py
+neural_feature_importance/callbacks.py
+neural_feature_importance/conv_callbacks.py
+neural_feature_importance/embedding_callbacks.py
+neural_feature_importance.egg-info/PKG-INFO
+neural_feature_importance.egg-info/SOURCES.txt
+neural_feature_importance.egg-info/dependency_links.txt
+neural_feature_importance.egg-info/requires.txt
+neural_feature_importance.egg-info/top_level.txt
+neural_feature_importance/utils/__init__.py
+neural_feature_importance/utils/monitors.py
+notebooks/conv_visualization_example.ipynb
+notebooks/token_importance_topk_example.ipynb
+notebooks/variance-based feature importance in artificial neural networks.ipynb
+scripts/compare_feature_importance.py
+scripts/conv_visualization_example.py
+scripts/full_experiment.py
+scripts/token_importance_topk_example.py

neural_feature_importance-0.9.1/notebooks/conv_visualization_example.ipynb

@@ -0,0 +1 @@
+{"cells": [{"cell_type": "markdown", "metadata": {}, "source": "# Convolutional Filter Visualization"}, {"cell_type": "code", "metadata": {}, "source": "\nimport numpy as np\nfrom tensorflow.keras.datasets import mnist\nfrom tensorflow.keras.utils import to_categorical\nfrom sklearn.datasets import load_digits\nfrom sklearn.model_selection import train_test_split\nfrom tensorflow.keras.models import Sequential\n\nfrom neural_feature_importance.conv_callbacks import ConvVarianceImportanceKeras\nfrom conv_viz_utils import build_model, rank_filters, plot_filters, accuracy_with_filters\n\ndef load_mnist():\n    (x_train, y_train), (x_test, y_test) = mnist.load_data()\n    x_train = x_train.astype(\"float32\") / 255.0\n    x_test = x_test.astype(\"float32\") / 255.0\n    x_train = x_train[..., None]\n    x_test = x_test[..., None]\n    y_train = to_categorical(y_train, 10)\n    y_test = to_categorical(y_test, 10)\n    return (x_train, y_train), (x_test, y_test), (28, 28, 1), (8, 8)\n\ndef load_digits_data():\n    digits = load_digits()\n    x = digits.images[..., None] / 16.0\n    y = to_categorical(digits.target, 10)\n    x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=42)\n    return (x_train, y_train), (x_test, y_test), (8, 8, 1), (3, 3)\n\nDATASETS = {\n    'mnist': load_mnist,\n    'digits': load_digits_data,\n}\n\ndef run_dataset(loader):\n    (x_train, y_train), (x_test, y_test), input_shape, kernel_size = loader()\n    model = build_model(input_shape, kernel_size)\n    callback = ConvVarianceImportanceKeras()\n    model.fit(x_train, y_train, epochs=5, batch_size=32, callbacks=[callback], verbose=1)\n\n    scores = callback.feature_importances_\n    weights = model.layers[0].get_weights()[0]\n    heatmap = scores.reshape(weights.shape[:3])\n    order = rank_filters(weights, heatmap, 0.0)\n\n    conv_model = Sequential([model.layers[0]])\n    example_out = conv_model.predict(x_test[:1], verbose=0)[0]\n    plot_filters(weights, heatmap, example_out, order)\n\n    for k in (2, 4, 6):\n        acc = accuracy_with_filters(model, x_test, y_test, order[:k])\n        print(f'Accuracy with top {k} filters:', acc)\n\nfor name, loader in DATASETS.items():\n    print('Running', name)\n    run_dataset(loader)\n"}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}

neural_feature_importance-0.9.1/notebooks/token_importance_topk_example.ipynb

@@ -0,0 +1,9 @@
+{
+ "cells": [
+  {"cell_type": "markdown", "metadata": {}, "source": "# Token Importance Top-k Example"},
+  {"cell_type": "code", "metadata": {}, "source": "from tensorflow.keras.datasets import imdb\nfrom tensorflow.keras.preprocessing.sequence import pad_sequences\n\nfrom neural_feature_importance.embedding_callbacks import EmbeddingVarianceImportanceKeras\nfrom token_topk_utils import analyze_samples, build_model\n\nMAX_FEATURES = 5000\nMAX_LEN = 400\nTOP_K = 5\nNUM_SAMPLES = 5\n\n(x_train, y_train), _ = imdb.load_data(num_words=MAX_FEATURES)\nx_train = pad_sequences(x_train, maxlen=MAX_LEN)\n\nmodel = build_model()\ncallback = EmbeddingVarianceImportanceKeras()\nmodel.fit(x_train, y_train, epochs=2, batch_size=128, callbacks=[callback], verbose=0)\n\nscores = callback.feature_importances_\nword_index = imdb.get_word_index()\nindex_word = {v + 3: k for k, v in word_index.items()}\nindex_word[0] = \"<PAD>\"\nindex_word[1] = \"<START>\"\nindex_word[2] = \"<UNK>\"\nindex_word[3] = \"<UNUSED>\"\n\nanalyze_samples(x_train, y_train, scores, index_word, NUM_SAMPLES, TOP_K)"}
+ ],
+ "metadata": {},
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

{neural_feature_importance-0.5.2 → neural_feature_importance-0.9.1}/pyproject.toml

@@ -15,3 +15,6 @@ requires-python = ">=3.10"
 tensorflow = ["tensorflow"]
 torch = ["torch"]
 [tool.setuptools_scm]
+
+[tool.setuptools]
+packages = ["neural_feature_importance"]

neural_feature_importance-0.9.1/scripts/conv_visualization_example.py

@@ -0,0 +1,114 @@
+"""Visualize variance-based filter importances on several datasets."""
+
+from __future__ import annotations
+
+import logging
+from typing import Callable, Tuple
+
+import numpy as np
+from tensorflow.keras.datasets import mnist
+from tensorflow.keras.utils import to_categorical
+from sklearn.datasets import load_digits
+from sklearn.model_selection import train_test_split
+from tensorflow.keras.models import Sequential
+
+from neural_feature_importance.conv_callbacks import ConvVarianceImportanceKeras
+from conv_viz_utils import build_model, rank_filters, plot_filters, accuracy_with_filters
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+DatasetLoader = Callable[
+    [],
+    Tuple[
+        Tuple[np.ndarray, np.ndarray],
+        Tuple[np.ndarray, np.ndarray],
+        tuple[int, int, int],
+        tuple[int, int],
+    ],
+]
+
+
+def load_mnist() -> Tuple[
+    Tuple[np.ndarray, np.ndarray],
+    Tuple[np.ndarray, np.ndarray],
+    tuple[int, int, int],
+    tuple[int, int],
+]:
+    """Return MNIST data and model parameters."""
+    (x_train, y_train), (x_test, y_test) = mnist.load_data()
+    x_train = x_train.astype("float32") / 255.0
+    x_test = x_test.astype("float32") / 255.0
+    x_train = x_train[..., None]
+    x_test = x_test[..., None]
+    y_train = to_categorical(y_train, 10)
+    y_test = to_categorical(y_test, 10)
+    return (x_train, y_train), (x_test, y_test), (28, 28, 1), (8, 8)
+
+
+def load_digits_data() -> Tuple[
+    Tuple[np.ndarray, np.ndarray],
+    Tuple[np.ndarray, np.ndarray],
+    tuple[int, int, int],
+    tuple[int, int],
+]:
+    """Return scikit-learn digits data and model parameters."""
+    digits = load_digits()
+    x = digits.images[..., None] / 16.0
+    y = to_categorical(digits.target, 10)
+    x_train, x_test, y_train, y_test = train_test_split(
+        x, y, test_size=0.2, random_state=42
+    )
+    return (x_train, y_train), (x_test, y_test), (8, 8, 1), (3, 3)
+
+
+DATASETS: dict[str, DatasetLoader] = {
+    "mnist": load_mnist,
+    "digits": load_digits_data,
+}
+
+
+def run_dataset(name: str, loader: DatasetLoader, threshold: float = 0.0) -> None:
+    """Train a model on the given dataset and display filter importances."""
+    (x_train, y_train), (x_test, y_test), input_shape, kernel_size = loader()
+
+    model = build_model(input_shape, kernel_size)
+    callback = ConvVarianceImportanceKeras()
+    model.fit(
+        x_train,
+        y_train,
+        epochs=5,
+        batch_size=32,
+        callbacks=[callback],
+        verbose=1,
+    )
+
+    scores = callback.feature_importances_
+    if scores is None:
+        logger.warning("No importance scores computed for %s", name)
+        return
+
+    weights = model.layers[0].get_weights()[0]
+    heatmap = scores.reshape(weights.shape[:3])
+    order = rank_filters(weights, heatmap, threshold)
+
+    conv_model = Sequential([model.layers[0]])
+    example_out = conv_model.predict(x_test[:1], verbose=0)[0]
+    plot_filters(weights, heatmap, example_out, order)
+
+    results = {}
+    for k in (2, 4, 6):
+        acc = accuracy_with_filters(model, x_test, y_test, order[:k])
+        results[k] = acc
+    logger.info("Accuracy with top filters on %s: %s", name, results)
+
+
+def main() -> None:
+    """Run visualization on all configured datasets."""
+    for name, loader in DATASETS.items():
+        logger.info("Running visualization for %s", name)
+        run_dataset(name, loader)
+
+
+if __name__ == "__main__":
+    main()

neural_feature_importance-0.9.1/scripts/token_importance_topk_example.py

@@ -0,0 +1,50 @@
+"""Display top-k token importances for several IMDB samples."""
+
+from __future__ import annotations
+
+import logging
+
+from tensorflow.keras.datasets import imdb
+from tensorflow.keras.preprocessing.sequence import pad_sequences
+
+from neural_feature_importance.embedding_callbacks import (
+    EmbeddingVarianceImportanceKeras,
+)
+
+from token_topk_utils import analyze_samples, build_model
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+MAX_FEATURES = 5000
+MAX_LEN = 400
+TOP_K = 5
+NUM_SAMPLES = 5
+
+
+def main() -> None:
+    """Train a model and log top tokens for a few samples."""
+    (x_train, y_train), _ = imdb.load_data(num_words=MAX_FEATURES)
+    x_train = pad_sequences(x_train, maxlen=MAX_LEN)
+
+    model = build_model()
+    callback = EmbeddingVarianceImportanceKeras()
+    model.fit(x_train, y_train, epochs=2, batch_size=128, callbacks=[callback], verbose=0)
+
+    scores = callback.feature_importances_
+    if scores is None:
+        logger.warning("No importance scores computed.")
+        return
+
+    word_index = imdb.get_word_index()
+    index_word = {v + 3: k for k, v in word_index.items()}
+    index_word[0] = "<PAD>"
+    index_word[1] = "<START>"
+    index_word[2] = "<UNK>"
+    index_word[3] = "<UNUSED>"
+
+    analyze_samples(x_train, y_train, scores, index_word, NUM_SAMPLES, TOP_K)
+
+
+if __name__ == "__main__":
+    main()

neural_feature_importance-0.9.1/text_classification_example.py

@@ -0,0 +1,66 @@
+"""Example of variance-based importance with text classification."""
+
+from __future__ import annotations
+
+import logging
+from typing import Tuple
+
+import matplotlib.pyplot as plt
+from tensorflow.keras.datasets import imdb
+from tensorflow.keras.layers import Conv1D, Dense, Embedding, GlobalMaxPooling1D
+from tensorflow.keras.models import Sequential
+from tensorflow.keras.preprocessing.sequence import pad_sequences
+
+from neural_feature_importance.conv_callbacks import ConvVarianceImportanceKeras
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+
+MAX_FEATURES = 5000
+MAX_LEN = 400
+
+
+def load_data() -> Tuple[tuple, tuple]:
+    """Return padded IMDB data."""
+    (x_train, y_train), _ = imdb.load_data(num_words=MAX_FEATURES)
+    x_train = pad_sequences(x_train, maxlen=MAX_LEN)
+    return (x_train, y_train), _
+
+
+def build_model() -> Sequential:
+    """Return a small Conv1D model."""
+    model = Sequential(
+        [
+            Embedding(MAX_FEATURES, 32, input_length=MAX_LEN, trainable=False),
+            Conv1D(16, 5, activation="relu"),
+            GlobalMaxPooling1D(),
+            Dense(1, activation="sigmoid"),
+        ]
+    )
+    model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
+    return model
+
+
+def main() -> None:
+    """Train the model and plot a heatmap of importances."""
+    (x_train, y_train), _ = load_data()
+    model = build_model()
+    callback = ConvVarianceImportanceKeras()
+    model.fit(x_train, y_train, epochs=2, batch_size=128, callbacks=[callback], verbose=0)
+
+    scores = callback.feature_importances_
+    if scores is None:
+        logger.warning("No importance scores computed.")
+        return
+
+    weights = model.layers[1].get_weights()[0]
+    heatmap = scores.reshape(weights.shape[0], weights.shape[1])
+    plt.imshow(heatmap, aspect="auto", cmap="hot")
+    plt.colorbar()
+    plt.title("Conv1D feature importance")
+    plt.show()
+
+
+if __name__ == "__main__":
+    main()

neural_feature_importance-0.9.1/token_topk_utils.py

@@ -0,0 +1,124 @@
+"""Utilities for analyzing top-k token importances.
+
+These helpers build a small text classification model and provide functions to
+decode token sequences and print tables of the most important tokens according
+to variance-based scores.
+"""
+
+from __future__ import annotations
+
+import logging
+from collections import Counter
+from typing import Iterable, List
+
+import numpy as np
+from tensorflow.keras.layers import (
+    Conv1D,
+    Dense,
+    Embedding,
+    GlobalMaxPooling1D,
+)
+from tensorflow.keras.models import Sequential
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+MAX_FEATURES = 5000
+MAX_LEN = 400
+
+
+def build_model() -> Sequential:
+    """Return a small text classification model.
+
+    The model consists of an embedding layer followed by a single convolution
+    and a global max pooling operation. It is intentionally tiny so it can be
+    trained quickly on the IMDB dataset for demonstration purposes.
+    """
+    model = Sequential(
+        [
+            Embedding(MAX_FEATURES, 32, input_length=MAX_LEN, trainable=True),
+            Conv1D(1, 5, activation="relu"),
+            GlobalMaxPooling1D(),
+            Dense(1, activation="sigmoid"),
+        ]
+    )
+    model.compile(
+        optimizer="adam",
+        loss="binary_crossentropy",
+        metrics=["accuracy"],
+    )
+    model.build((None, MAX_LEN))
+    return model
+
+
+def decode_review(tokens: Iterable[int], index_word: dict[int, str]) -> str:
+    """Return a readable string for the given token sequence."""
+    words = [index_word.get(t, "?") for t in tokens if t]
+    return " ".join(words)
+
+
+def summarize_top_tokens(
+    tokens: Iterable[int],
+    scores: np.ndarray,
+    index_word: dict[int, str],
+    k: int,
+) -> str:
+    """Return a table with the top ``k`` unique tokens and their counts.
+
+    Parameters
+    ----------
+    tokens:
+        Token ids representing a review.
+    scores:
+        Array of token importance scores obtained from the embedding callback.
+    index_word:
+        Mapping from token id to the corresponding word.
+    k:
+        Number of tokens to include in the table.
+    """
+    ignore = {"<PAD>", "<START>", "<UNK>", "<UNUSED>"}
+    totals: dict[int, float] = {}
+    counts: Counter[int] = Counter()
+    for t in tokens:
+        if index_word.get(t) in ignore:
+            continue
+        if t < len(scores):
+            totals[t] = totals.get(t, 0.0) + float(scores[t])
+        counts[t] += 1
+    ordered = sorted(totals.items(), key=lambda kv: kv[1], reverse=True)[:k]
+    headers = ["Token", "Count", "Score"]
+    rows: List[tuple[str, int, str]] = []
+    for token_id, score in ordered:
+        token = index_word.get(token_id, "?")
+        if token in ignore:
+            continue
+        rows.append((token, counts[token_id], f"{score:.3f}"))
+    table_lines = [" | ".join(headers)]
+    table_lines.append("-|-".join("-" * len(h) for h in headers))
+    for row in rows:
+        table_lines.append(" | ".join(str(x) for x in row))
+    return "\n".join(table_lines)
+
+
+def analyze_samples(
+    x: np.ndarray,
+    y: np.ndarray,
+    scores: np.ndarray,
+    index_word: dict[int, str],
+    num_samples: int = 5,
+    k: int = 5,
+) -> None:
+    """Log original text and the most important tokens for several samples.
+
+    Each selected sample is decoded to text, its label is printed, and a table
+    of the top ``k`` tokens by importance score is logged.
+    """
+    for i in range(min(num_samples, len(x))):
+        tokens = x[i].tolist()
+        label = "positive" if y[i] == 1 else "negative"
+        text = decode_review(tokens, index_word)
+        table = summarize_top_tokens(tokens, scores, index_word, k)
+        logger.info("Example %d - class: %s", i, label)
+        logger.info("%s", text)
+        logger.info("Top tokens:\n%s", table)
+

neural_feature_importance-0.5.2/neural_feature_importance.egg-info/SOURCES.txt

@@ -1,16 +0,0 @@
-AGENTS.md
-README.md
-compare_feature_importance.py
-full_experiment.py
-pyproject.toml
-variance-based feature importance in artificial neural networks.ipynb
-.github/workflows/python-publish.yml
-neural_feature_importance/__init__.py
-neural_feature_importance/callbacks.py
-neural_feature_importance.egg-info/PKG-INFO
-neural_feature_importance.egg-info/SOURCES.txt
-neural_feature_importance.egg-info/dependency_links.txt
-neural_feature_importance.egg-info/requires.txt
-neural_feature_importance.egg-info/top_level.txt
-neural_feature_importance/utils/__init__.py
-neural_feature_importance/utils/monitors.py