mantissa-interpret 0.1.0__tar.gz

mantissa_interpret-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tekin Ertekin
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mantissa_interpret-0.1.0/PKG-INFO

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: mantissa-interpret
+Version: 0.1.0
+Summary: CNN interpretability on the mantissa C engine: occlusion, saliency and Grad-CAM
+Author: Tekin Ertekin
+License: MIT
+Project-URL: Homepage, https://github.com/tekinertekin/mantissa-interpret
+Project-URL: Base, https://github.com/tekinertekin/mantissa-cnn
+Project-URL: Engine, https://github.com/tekinertekin/mantissa
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.20
+Requires-Dist: mantissa-cnn>=0.2.2
+Provides-Extra: viz
+Requires-Dist: matplotlib>=3.5; extra == "viz"
+Provides-Extra: bench
+Requires-Dist: matplotlib>=3.5; extra == "bench"
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Dynamic: license-file
+
+# mantissa-interpret
+
+![License](https://img.shields.io/badge/license-MIT-blue.svg)
+![Python](https://img.shields.io/badge/python-3.9%2B-3776AB.svg)
+[![Base](https://img.shields.io/badge/base-mantissa--cnn-4B8BBE.svg)](https://github.com/tekinertekin/mantissa-cnn)
+[![Engine](https://img.shields.io/badge/engine-mantissa-00599C.svg)](https://github.com/tekinertekin/mantissa)
+
+**Seeing what a CNN looks at.** A trained classifier gives you a label; it does
+not tell you *why*. `mantissa-interpret` answers that with three classic
+attribution methods that turn a prediction into a **heatmap over the input** —
+bright where the pixels mattered for the class, dark where they did not.
+
+Everything runs on a fitted [`mantissa_cnn.Sequential`](https://github.com/tekinertekin/mantissa-cnn)
+model, using the model's **own forward and backward passes through the mantissa
+C engine** — the same engine it was trained on. There is no PyTorch/TensorFlow
+dependency: the heatmaps come out of the same low-precision C kernels that
+produced the prediction.
+
+## The mantissa family
+
+Part of the **mantissa** family: a low-precision engine written in C, with
+small Python packages built on top. Each package sits under the one it depends
+on — ⭐ marks where you are, and every other name links to its repo.
+
+- [mantissa](https://github.com/tekinertekin/mantissa) — low-precision neural-network engine in C (the core)
+  - [mantissa-perceptron](https://github.com/tekinertekin/mantissa-perceptron) — perceptron & ADALINE, the linear classics
+  - [mantissa-nn](https://github.com/tekinertekin/mantissa-nn) — shared neural-net primitives (layers, engine binding)
+    - [mantissa-cnn](https://github.com/tekinertekin/mantissa-cnn) — convolutional networks for images
+      - [mantissa-auto-encoder](https://github.com/tekinertekin/mantissa-auto-encoder) — autoencoders for denoising & super-resolution
+      - ⭐ **mantissa-interpret** — CNN interpretability (occlusion, saliency, Grad-CAM) *(you are here)*
+      - [mantissa-embed](https://github.com/tekinertekin/mantissa-embed) — CNN metric learning (image embeddings for similarity & retrieval)
+    - [mantissa-mlp](https://github.com/tekinertekin/mantissa-mlp) — multilayer perceptrons, fully-connected nets
+
+
+## New to interpretability?
+
+A CNN classifier turns an image into a label — but the label alone hides *why*.
+**Interpretability** (here, *attribution*) recovers the missing reason: it marks
+**which parts of the input the model actually used** for a given class, as a
+heatmap laid over the image. That matters because a model can be right for the
+wrong reason — keying on a watermark, a background, or a dataset artifact
+instead of the object — and a bare accuracy number will never tell you. A
+heatmap will.
+
+The most direct way to see this is to **hide part of the image and watch the
+prediction**. If covering a region makes the class probability collapse, that
+region carried the evidence; if nothing changes, it did not:
+
+![covering the important strokes of an 8 drops P(8) from 1.00 to 0.00; covering empty space leaves it at 1.00](https://raw.githubusercontent.com/tekinertekin/mantissa-interpret/main/assets/how_occlusion_works.png)
+
+That single experiment *is* the first method. The three methods here are three
+answers to "what did the model use?", trading detail for cost and clarity:
+
+- **Occlusion** — exactly the picture above, done everywhere: slide a patch
+  over the image and record how much each position, when hidden, drops the
+  class probability. No math beyond running the model forward; the price is one
+  forward pass per patch, and the resolution is patch-sized.
+- **Saliency** — instead of hiding pixels, ask calculus which pixels *matter*:
+  the gradient of the class score with respect to each input pixel. A large
+  gradient means "nudging this pixel would move the score a lot." It is
+  per-pixel sharp but noisy, and needs a single backward pass.
+- **Grad-CAM** — work inside the network. At a convolutional layer each channel
+  is a learned feature detector over a coarse grid; weight every channel by how
+  much raising its activation would raise the target score (its gradient),
+  add them up, and keep the positive part. The result is a smooth,
+  **class-discriminative** region — ask about "3" vs "7" on the same image and
+  the map moves — for one backward pass.
+
+Occlusion and saliency answer *which pixels*; Grad-CAM answers *which region,
+for this class*. Together they are the standard first tools for debugging a
+model that is right for the wrong reason.
+
+## Install
+
+```sh
+pip install mantissa-interpret
+```
+
+Pulls in `mantissa-cnn` (and transitively `mantissa-nn` + the `mantissa-core`
+engine). For plotting the heatmaps, `pip install mantissa-interpret[viz]`.
+
+## The three methods
+
+| method | cost | granularity | class-discriminative | needs |
+|---|---|---|---|---|
+| `occlusion_map` | forward only, O(patches) | coarse (patch) | yes | — |
+| `saliency_map` | one backward pass | per-pixel (noisy) | weakly | input gradient |
+| `grad_cam` | one backward pass | per-region (smooth) | **yes** | activations + their gradient at a conv layer |
+
+Each takes a fitted model and a single `(C, H, W)` image and returns a 2-D
+heatmap normalized to `[0, 1]`, aligned to the input.
+
+### Occlusion — `occlusion_map`
+
+Slide a `patch`×`patch` window (filled with `fill`) across the image; at each
+position, hide that window and re-run the model. If the target class
+probability drops a lot, those pixels were important. The heatmap is the
+per-pixel average drop (overlapping windows blend), normalized to `[0, 1]`.
+
+```python
+from mantissa_interpret import occlusion_map
+heat = occlusion_map(net, image, target_class=7, patch=7, stride=3)
+```
+
+Only forward passes — model-agnostic and dead simple, but coarse (patch-sized)
+and its cost scales with the number of windows. All occluded copies are run in
+a single batched forward pass.
+
+### Saliency — `saliency_map`
+
+The first-order sensitivity of the class score to each pixel is just its
+gradient. One forward pass primes the layers, we seed the gradient of the
+target *logit* (a one-hot vector), and backpropagate to the input — the same
+`backward` the model trains with, carried one step past the first layer's
+weights down to the pixels. The map is `|gradient|` reduced over channels.
+
+```python
+from mantissa_interpret import saliency_map
+heat = saliency_map(net, image, target_class=7)
+```
+
+Per-pixel and cheap (one backward pass), but high-frequency and noisy, and only
+weakly class-discriminative — good for "which strokes", not "which region".
+
+### Grad-CAM — `grad_cam`
+
+At a convolutional layer, each channel is a learned feature detector over a
+coarse grid. Grad-CAM weights every channel by the average gradient of the
+target logit w.r.t. that channel's activation (how much "more of this feature"
+raises the class score), sums the channels with those weights, keeps the
+positive part, and upsamples to the image. We capture the layer's activation on
+the forward pass and backpropagate the logit down to that same layer for the
+gradient — then combine.
+
+```python
+from mantissa_interpret import grad_cam
+heat = grad_cam(net, image, target_class=7)      # last Conv2D by default
+```
+
+Coarse (conv-grid resolution) but smooth and genuinely **class-discriminative**
+— asking about different classes on the same image yields different maps.
+`target_layer` selects which conv layer to read (default: the last one).
+
+## Results
+
+A LeNet-5 trained on an 8k-image MNIST subset with the mantissa C engine
+(3 epochs, **96% test accuracy**), then explained on held-out digits. Reproduce
+with `python examples/heatmaps_demo.py`.
+
+![occlusion, saliency and Grad-CAM heatmaps on MNIST digits](https://raw.githubusercontent.com/tekinertekin/mantissa-interpret/main/assets/heatmaps.png)
+
+Reading across a row, the three methods agree on the digit but differ exactly
+as their math predicts:
+
+- **occlusion** — coarse, patch-sized blobs on the strokes whose removal costs
+  the class the most;
+- **saliency** — the finest detail, tracing individual strokes, but visibly
+  noisy (the raw per-pixel gradient);
+- **Grad-CAM** — a smooth region over the class-defining strokes (the 7's
+  diagonal, the 2's base, the 9's loop), with the least clutter.
+
+Grad-CAM is also **class-discriminative** — ask it about a different target
+class on the *same* image and the map moves:
+
+![Grad-CAM for two different target classes on the same 7](https://raw.githubusercontent.com/tekinertekin/mantissa-interpret/main/assets/gradcam_classes.png)
+
+All of this — feature maps, gradients, the optimization-free attribution — comes
+out of the same low-precision C kernels the model was trained on.
+
+## License
+
+MIT — Tekin Ertekin.

mantissa_interpret-0.1.0/README.md

@@ -0,0 +1,172 @@
+# mantissa-interpret
+
+![License](https://img.shields.io/badge/license-MIT-blue.svg)
+![Python](https://img.shields.io/badge/python-3.9%2B-3776AB.svg)
+[![Base](https://img.shields.io/badge/base-mantissa--cnn-4B8BBE.svg)](https://github.com/tekinertekin/mantissa-cnn)
+[![Engine](https://img.shields.io/badge/engine-mantissa-00599C.svg)](https://github.com/tekinertekin/mantissa)
+
+**Seeing what a CNN looks at.** A trained classifier gives you a label; it does
+not tell you *why*. `mantissa-interpret` answers that with three classic
+attribution methods that turn a prediction into a **heatmap over the input** —
+bright where the pixels mattered for the class, dark where they did not.
+
+Everything runs on a fitted [`mantissa_cnn.Sequential`](https://github.com/tekinertekin/mantissa-cnn)
+model, using the model's **own forward and backward passes through the mantissa
+C engine** — the same engine it was trained on. There is no PyTorch/TensorFlow
+dependency: the heatmaps come out of the same low-precision C kernels that
+produced the prediction.
+
+## The mantissa family
+
+Part of the **mantissa** family: a low-precision engine written in C, with
+small Python packages built on top. Each package sits under the one it depends
+on — ⭐ marks where you are, and every other name links to its repo.
+
+- [mantissa](https://github.com/tekinertekin/mantissa) — low-precision neural-network engine in C (the core)
+  - [mantissa-perceptron](https://github.com/tekinertekin/mantissa-perceptron) — perceptron & ADALINE, the linear classics
+  - [mantissa-nn](https://github.com/tekinertekin/mantissa-nn) — shared neural-net primitives (layers, engine binding)
+    - [mantissa-cnn](https://github.com/tekinertekin/mantissa-cnn) — convolutional networks for images
+      - [mantissa-auto-encoder](https://github.com/tekinertekin/mantissa-auto-encoder) — autoencoders for denoising & super-resolution
+      - ⭐ **mantissa-interpret** — CNN interpretability (occlusion, saliency, Grad-CAM) *(you are here)*
+      - [mantissa-embed](https://github.com/tekinertekin/mantissa-embed) — CNN metric learning (image embeddings for similarity & retrieval)
+    - [mantissa-mlp](https://github.com/tekinertekin/mantissa-mlp) — multilayer perceptrons, fully-connected nets
+
+
+## New to interpretability?
+
+A CNN classifier turns an image into a label — but the label alone hides *why*.
+**Interpretability** (here, *attribution*) recovers the missing reason: it marks
+**which parts of the input the model actually used** for a given class, as a
+heatmap laid over the image. That matters because a model can be right for the
+wrong reason — keying on a watermark, a background, or a dataset artifact
+instead of the object — and a bare accuracy number will never tell you. A
+heatmap will.
+
+The most direct way to see this is to **hide part of the image and watch the
+prediction**. If covering a region makes the class probability collapse, that
+region carried the evidence; if nothing changes, it did not:
+
+![covering the important strokes of an 8 drops P(8) from 1.00 to 0.00; covering empty space leaves it at 1.00](https://raw.githubusercontent.com/tekinertekin/mantissa-interpret/main/assets/how_occlusion_works.png)
+
+That single experiment *is* the first method. The three methods here are three
+answers to "what did the model use?", trading detail for cost and clarity:
+
+- **Occlusion** — exactly the picture above, done everywhere: slide a patch
+  over the image and record how much each position, when hidden, drops the
+  class probability. No math beyond running the model forward; the price is one
+  forward pass per patch, and the resolution is patch-sized.
+- **Saliency** — instead of hiding pixels, ask calculus which pixels *matter*:
+  the gradient of the class score with respect to each input pixel. A large
+  gradient means "nudging this pixel would move the score a lot." It is
+  per-pixel sharp but noisy, and needs a single backward pass.
+- **Grad-CAM** — work inside the network. At a convolutional layer each channel
+  is a learned feature detector over a coarse grid; weight every channel by how
+  much raising its activation would raise the target score (its gradient),
+  add them up, and keep the positive part. The result is a smooth,
+  **class-discriminative** region — ask about "3" vs "7" on the same image and
+  the map moves — for one backward pass.
+
+Occlusion and saliency answer *which pixels*; Grad-CAM answers *which region,
+for this class*. Together they are the standard first tools for debugging a
+model that is right for the wrong reason.
+
+## Install
+
+```sh
+pip install mantissa-interpret
+```
+
+Pulls in `mantissa-cnn` (and transitively `mantissa-nn` + the `mantissa-core`
+engine). For plotting the heatmaps, `pip install mantissa-interpret[viz]`.
+
+## The three methods
+
+| method | cost | granularity | class-discriminative | needs |
+|---|---|---|---|---|
+| `occlusion_map` | forward only, O(patches) | coarse (patch) | yes | — |
+| `saliency_map` | one backward pass | per-pixel (noisy) | weakly | input gradient |
+| `grad_cam` | one backward pass | per-region (smooth) | **yes** | activations + their gradient at a conv layer |
+
+Each takes a fitted model and a single `(C, H, W)` image and returns a 2-D
+heatmap normalized to `[0, 1]`, aligned to the input.
+
+### Occlusion — `occlusion_map`
+
+Slide a `patch`×`patch` window (filled with `fill`) across the image; at each
+position, hide that window and re-run the model. If the target class
+probability drops a lot, those pixels were important. The heatmap is the
+per-pixel average drop (overlapping windows blend), normalized to `[0, 1]`.
+
+```python
+from mantissa_interpret import occlusion_map
+heat = occlusion_map(net, image, target_class=7, patch=7, stride=3)
+```
+
+Only forward passes — model-agnostic and dead simple, but coarse (patch-sized)
+and its cost scales with the number of windows. All occluded copies are run in
+a single batched forward pass.
+
+### Saliency — `saliency_map`
+
+The first-order sensitivity of the class score to each pixel is just its
+gradient. One forward pass primes the layers, we seed the gradient of the
+target *logit* (a one-hot vector), and backpropagate to the input — the same
+`backward` the model trains with, carried one step past the first layer's
+weights down to the pixels. The map is `|gradient|` reduced over channels.
+
+```python
+from mantissa_interpret import saliency_map
+heat = saliency_map(net, image, target_class=7)
+```
+
+Per-pixel and cheap (one backward pass), but high-frequency and noisy, and only
+weakly class-discriminative — good for "which strokes", not "which region".
+
+### Grad-CAM — `grad_cam`
+
+At a convolutional layer, each channel is a learned feature detector over a
+coarse grid. Grad-CAM weights every channel by the average gradient of the
+target logit w.r.t. that channel's activation (how much "more of this feature"
+raises the class score), sums the channels with those weights, keeps the
+positive part, and upsamples to the image. We capture the layer's activation on
+the forward pass and backpropagate the logit down to that same layer for the
+gradient — then combine.
+
+```python
+from mantissa_interpret import grad_cam
+heat = grad_cam(net, image, target_class=7)      # last Conv2D by default
+```
+
+Coarse (conv-grid resolution) but smooth and genuinely **class-discriminative**
+— asking about different classes on the same image yields different maps.
+`target_layer` selects which conv layer to read (default: the last one).
+
+## Results
+
+A LeNet-5 trained on an 8k-image MNIST subset with the mantissa C engine
+(3 epochs, **96% test accuracy**), then explained on held-out digits. Reproduce
+with `python examples/heatmaps_demo.py`.
+
+![occlusion, saliency and Grad-CAM heatmaps on MNIST digits](https://raw.githubusercontent.com/tekinertekin/mantissa-interpret/main/assets/heatmaps.png)
+
+Reading across a row, the three methods agree on the digit but differ exactly
+as their math predicts:
+
+- **occlusion** — coarse, patch-sized blobs on the strokes whose removal costs
+  the class the most;
+- **saliency** — the finest detail, tracing individual strokes, but visibly
+  noisy (the raw per-pixel gradient);
+- **Grad-CAM** — a smooth region over the class-defining strokes (the 7's
+  diagonal, the 2's base, the 9's loop), with the least clutter.
+
+Grad-CAM is also **class-discriminative** — ask it about a different target
+class on the *same* image and the map moves:
+
+![Grad-CAM for two different target classes on the same 7](https://raw.githubusercontent.com/tekinertekin/mantissa-interpret/main/assets/gradcam_classes.png)
+
+All of this — feature maps, gradients, the optimization-free attribution — comes
+out of the same low-precision C kernels the model was trained on.
+
+## License
+
+MIT — Tekin Ertekin.

mantissa_interpret-0.1.0/mantissa_interpret/__init__.py

@@ -0,0 +1,29 @@
+"""mantissa-interpret: CNN interpretability on the mantissa C engine.
+
+Attribution / visualization methods that reveal *what a trained
+``mantissa_cnn.Sequential`` model looks at* when it makes a prediction. Each
+method takes a fitted model and a single NCHW image and returns a 2-D heatmap
+aligned to that image — computed with the model's own forward (and, where
+needed, backward) passes, so it runs through the exact C engine the model was
+trained on. No deep-learning framework is involved.
+
+Three methods, cheapest first:
+
+- ``occlusion_map``  — forward-only: slide a gray patch over the image and see
+  how much the target class probability drops. Model-agnostic, no gradients.
+- ``saliency_map``   — one backward pass: |gradient of the class score w.r.t.
+  each input pixel|. Fine-grained but noisy.
+- ``grad_cam``       — Grad-CAM: gradient-weighted activation map at a chosen
+  convolutional layer. Coarse but class-discriminative and low-noise.
+
+>>> from mantissa_cnn import models, datasets
+>>> from mantissa_interpret import occlusion_map, saliency_map, grad_cam
+>>> net = models.lenet5(); net.fit(X, y, epochs=3)
+>>> heat = grad_cam(net, image, target_class=7)   # (H, W) in [0, 1]
+"""
+from .occlusion import occlusion_map
+from .saliency import saliency_map
+from .gradcam import grad_cam
+
+__version__ = "0.1.0"
+__all__ = ["occlusion_map", "saliency_map", "grad_cam"]

mantissa_interpret-0.1.0/mantissa_interpret/gradcam.py

@@ -0,0 +1,102 @@
+"""Grad-CAM (Selvaraju et al., 2017).
+
+Saliency is per-pixel and noisy; Grad-CAM is per-region and class-
+discriminative. It works at a convolutional layer, where each channel is a
+learned feature detector over a coarse spatial grid. For a target class we
+weight every channel by how much its activation *increasing* would raise the
+class score (the average gradient), sum the channels with those weights, and
+keep the positive part. The result is a coarse map of "where the evidence for
+this class lives", which we upsample back to the image.
+
+We reuse exactly the forward/backward the model trains with: capture the target
+conv layer's output on the way up, backpropagate the target logit down to that
+same layer to get its gradient, and combine.
+"""
+import numpy as np
+from mantissa_cnn.layers import Conv2D
+
+__all__ = ["grad_cam"]
+
+
+def _bilinear(cam, out_hw):
+    """Bilinearly resize a 2-D map to (H, W). Dependency-free."""
+    H, W = out_hw
+    h, w = cam.shape
+    if (h, w) == (H, W):
+        return cam.astype(np.float32)
+    ys = np.linspace(0.0, h - 1, H)
+    xs = np.linspace(0.0, w - 1, W)
+    y0 = np.floor(ys).astype(int); y1 = np.minimum(y0 + 1, h - 1)
+    x0 = np.floor(xs).astype(int); x1 = np.minimum(x0 + 1, w - 1)
+    wy = (ys - y0)[:, None].astype(np.float32)
+    wx = (xs - x0)[None, :].astype(np.float32)
+    top = cam[y0][:, x0] * (1 - wx) + cam[y0][:, x1] * wx
+    bot = cam[y1][:, x0] * (1 - wx) + cam[y1][:, x1] * wx
+    return (top * (1 - wy) + bot * wy).astype(np.float32)
+
+
+def grad_cam(model, image, target_class=None, target_layer=None):
+    """Grad-CAM heatmap for one image.
+
+    Parameters
+    ----------
+    model : fitted ``mantissa_cnn.Sequential``
+    image : array (C, H, W), float32
+    target_class : int, optional
+        Class to explain. ``None`` uses the predicted class.
+    target_layer : int or Conv2D, optional
+        Which convolutional layer to read. ``None`` uses the *last* Conv2D
+        (the standard, most class-specific choice).
+
+    Returns
+    -------
+    heat : array (H, W), float32 in [0, 1]
+    """
+    image = np.ascontiguousarray(image, dtype=np.float32)
+    if image.ndim != 3:
+        raise ValueError(f"image must be (C, H, W), got shape {image.shape}")
+    C, H, W = image.shape
+    backend = model._backend
+    layers = model.layers
+
+    if target_layer is None:
+        conv_idx = [i for i, l in enumerate(layers) if isinstance(l, Conv2D)]
+        if not conv_idx:
+            raise ValueError("model has no Conv2D layer to run Grad-CAM on")
+        target_idx = conv_idx[-1]
+    elif isinstance(target_layer, int):
+        target_idx = target_layer
+    else:
+        target_idx = layers.index(target_layer)
+
+    # Forward, copying out the target layer's activation A (the engine reuses
+    # buffers in place, so we must copy, not alias).
+    A = None
+    h = image[None]
+    for i, layer in enumerate(layers):
+        h = layer.forward(h, backend)
+        if i == target_idx:
+            A = np.array(h)                       # (1, K, h', w')
+    logits = h
+
+    if target_class is None:
+        target_class = int(logits[0].argmax())
+
+    dY = np.zeros_like(logits)
+    dY[0, target_class] = 1.0
+
+    # Backprop only down to the layer just above the target: the gradient
+    # handed into that layer's backward is d(logit_c)/d(A).
+    grad = dY
+    for i in range(len(layers) - 1, target_idx, -1):
+        grad = layers[i].backward(grad, backend, need_dx=True)
+    dA = np.array(grad)                            # (1, K, h', w')
+
+    alpha = dA.mean(axis=(2, 3))                   # (1, K) channel importances
+    cam = np.maximum((alpha[:, :, None, None] * A).sum(axis=1)[0], 0.0)  # (h', w')
+
+    cam = _bilinear(cam, (H, W))
+    peak = cam.max()
+    if peak > 0:
+        cam = cam / peak
+    return cam.astype(np.float32)

mantissa_interpret-0.1.0/mantissa_interpret/occlusion.py

@@ -0,0 +1,81 @@
+"""Occlusion sensitivity maps (Zeiler & Fergus, 2014).
+
+Slide a small patch over the image, blank it out, and watch the target class
+probability. Wherever hiding a region makes the class probability fall the
+most, that region mattered most to the prediction. It needs nothing but the
+model's forward pass, so it works for any classifier — no gradients, no
+assumptions about the architecture.
+"""
+import numpy as np
+
+__all__ = ["occlusion_map"]
+
+
+def occlusion_map(model, image, target_class=None, patch=7, stride=3, fill=0.0):
+    """Occlusion sensitivity heatmap for one image.
+
+    Parameters
+    ----------
+    model : fitted ``mantissa_cnn.Sequential``
+    image : array (C, H, W), float32
+    target_class : int, optional
+        Class to explain. ``None`` uses the model's own predicted class.
+    patch : int
+        Side length of the square occluding window.
+    stride : int
+        Step between window positions (smaller = finer + slower).
+    fill : float
+        Value written into the occluded window. ``0.0`` blacks it out; the
+        image mean is a common neutral choice.
+
+    Returns
+    -------
+    heat : array (H, W), float32 in [0, 1]
+        Importance per pixel: 1 where occlusion hurt the target class most.
+    """
+    image = np.ascontiguousarray(image, dtype=np.float32)
+    if image.ndim != 3:
+        raise ValueError(f"image must be (C, H, W), got shape {image.shape}")
+    C, H, W = image.shape
+    if patch > H or patch > W:
+        raise ValueError(f"patch {patch} larger than image {H}x{W}")
+
+    base = model.predict_proba(image[None])[0]
+    if target_class is None:
+        target_class = int(base.argmax())
+    base_p = float(base[target_class])
+
+    # Window top-left positions, always including the bottom/right edge so no
+    # border pixel is left un-probed.
+    ys = list(range(0, H - patch + 1, stride))
+    xs = list(range(0, W - patch + 1, stride))
+    if ys[-1] != H - patch:
+        ys.append(H - patch)
+    if xs[-1] != W - patch:
+        xs.append(W - patch)
+
+    # One forward pass over the whole batch of occluded copies.
+    batch = np.repeat(image[None], len(ys) * len(xs), axis=0)
+    coords = []
+    k = 0
+    for y in ys:
+        for x in xs:
+            batch[k, :, y:y + patch, x:x + patch] = fill
+            coords.append((y, x))
+            k += 1
+    probs = model.predict_proba(batch)[:, target_class]
+    drops = base_p - probs                       # how much hiding hurt the class
+
+    # Average each patch's drop over the pixels it covered (overlaps blend).
+    heat = np.zeros((H, W), dtype=np.float32)
+    count = np.zeros((H, W), dtype=np.float32)
+    for (y, x), d in zip(coords, drops):
+        heat[y:y + patch, x:x + patch] += d
+        count[y:y + patch, x:x + patch] += 1.0
+    heat /= np.maximum(count, 1.0)
+
+    heat = np.maximum(heat, 0.0)                  # keep only positive importance
+    peak = heat.max()
+    if peak > 0:
+        heat /= peak
+    return heat

mantissa_interpret-0.1.0/mantissa_interpret/saliency.py

@@ -0,0 +1,65 @@
+"""Vanilla gradient saliency (Simonyan, Vedaldi & Zisserman, 2014).
+
+The first-order answer to "which input pixels would change this class score
+the most?" is simply the gradient of that score with respect to the input.
+We run one forward pass to prime the layers, seed the gradient of the target
+logit, and backpropagate all the way to the pixels — using the same layer
+``backward`` passes the model trains with, only carried one step further (to
+the input instead of stopping at the first layer's weights).
+"""
+import numpy as np
+
+__all__ = ["saliency_map"]
+
+
+def saliency_map(model, image, target_class=None):
+    """Gradient-magnitude saliency heatmap for one image.
+
+    Parameters
+    ----------
+    model : fitted ``mantissa_cnn.Sequential``
+    image : array (C, H, W), float32
+    target_class : int, optional
+        Class whose score to differentiate. ``None`` uses the predicted class.
+
+    Returns
+    -------
+    heat : array (H, W), float32 in [0, 1]
+        Per-pixel ``|d score_c / d pixel|``, reduced over channels by max.
+
+    Notes
+    -----
+    Differentiates the raw class *logit* (pre-softmax), the standard choice:
+    it isolates evidence for the class without the softmax coupling every class
+    into the gradient.
+    """
+    image = np.ascontiguousarray(image, dtype=np.float32)
+    if image.ndim != 3:
+        raise ValueError(f"image must be (C, H, W), got shape {image.shape}")
+    backend = model._backend
+
+    # Forward pass: each layer caches what its backward needs.
+    h = image[None]
+    for layer in model.layers:
+        h = layer.forward(h, backend)
+    logits = h                                    # (1, n_classes)
+
+    if target_class is None:
+        target_class = int(logits[0].argmax())
+
+    # Seed: d(logit_c) / d(logits) is the one-hot vector for c.
+    dY = np.zeros_like(logits)
+    dY[0, target_class] = 1.0
+
+    # Backprop to the input. need_dx=True on every layer (including the first,
+    # unlike training which stops at layer 0) so we get d(logit_c)/d(input).
+    grad = dY
+    for layer in reversed(model.layers):
+        grad = layer.backward(grad, backend, need_dx=True)
+    grad = np.asarray(grad)[0]                     # (C, H, W)
+
+    sal = np.abs(grad).max(axis=0)                 # (H, W)
+    peak = sal.max()
+    if peak > 0:
+        sal = sal / peak
+    return sal.astype(np.float32)

mantissa_interpret-0.1.0/mantissa_interpret.egg-info/PKG-INFO

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: mantissa-interpret
+Version: 0.1.0
+Summary: CNN interpretability on the mantissa C engine: occlusion, saliency and Grad-CAM
+Author: Tekin Ertekin
+License: MIT
+Project-URL: Homepage, https://github.com/tekinertekin/mantissa-interpret
+Project-URL: Base, https://github.com/tekinertekin/mantissa-cnn
+Project-URL: Engine, https://github.com/tekinertekin/mantissa
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.20
+Requires-Dist: mantissa-cnn>=0.2.2
+Provides-Extra: viz
+Requires-Dist: matplotlib>=3.5; extra == "viz"
+Provides-Extra: bench
+Requires-Dist: matplotlib>=3.5; extra == "bench"
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Dynamic: license-file
+
+# mantissa-interpret
+
+![License](https://img.shields.io/badge/license-MIT-blue.svg)
+![Python](https://img.shields.io/badge/python-3.9%2B-3776AB.svg)
+[![Base](https://img.shields.io/badge/base-mantissa--cnn-4B8BBE.svg)](https://github.com/tekinertekin/mantissa-cnn)
+[![Engine](https://img.shields.io/badge/engine-mantissa-00599C.svg)](https://github.com/tekinertekin/mantissa)
+
+**Seeing what a CNN looks at.** A trained classifier gives you a label; it does
+not tell you *why*. `mantissa-interpret` answers that with three classic
+attribution methods that turn a prediction into a **heatmap over the input** —
+bright where the pixels mattered for the class, dark where they did not.
+
+Everything runs on a fitted [`mantissa_cnn.Sequential`](https://github.com/tekinertekin/mantissa-cnn)
+model, using the model's **own forward and backward passes through the mantissa
+C engine** — the same engine it was trained on. There is no PyTorch/TensorFlow
+dependency: the heatmaps come out of the same low-precision C kernels that
+produced the prediction.
+
+## The mantissa family
+
+Part of the **mantissa** family: a low-precision engine written in C, with
+small Python packages built on top. Each package sits under the one it depends
+on — ⭐ marks where you are, and every other name links to its repo.
+
+- [mantissa](https://github.com/tekinertekin/mantissa) — low-precision neural-network engine in C (the core)
+  - [mantissa-perceptron](https://github.com/tekinertekin/mantissa-perceptron) — perceptron & ADALINE, the linear classics
+  - [mantissa-nn](https://github.com/tekinertekin/mantissa-nn) — shared neural-net primitives (layers, engine binding)
+    - [mantissa-cnn](https://github.com/tekinertekin/mantissa-cnn) — convolutional networks for images
+      - [mantissa-auto-encoder](https://github.com/tekinertekin/mantissa-auto-encoder) — autoencoders for denoising & super-resolution
+      - ⭐ **mantissa-interpret** — CNN interpretability (occlusion, saliency, Grad-CAM) *(you are here)*
+      - [mantissa-embed](https://github.com/tekinertekin/mantissa-embed) — CNN metric learning (image embeddings for similarity & retrieval)
+    - [mantissa-mlp](https://github.com/tekinertekin/mantissa-mlp) — multilayer perceptrons, fully-connected nets
+
+
+## New to interpretability?
+
+A CNN classifier turns an image into a label — but the label alone hides *why*.
+**Interpretability** (here, *attribution*) recovers the missing reason: it marks
+**which parts of the input the model actually used** for a given class, as a
+heatmap laid over the image. That matters because a model can be right for the
+wrong reason — keying on a watermark, a background, or a dataset artifact
+instead of the object — and a bare accuracy number will never tell you. A
+heatmap will.
+
+The most direct way to see this is to **hide part of the image and watch the
+prediction**. If covering a region makes the class probability collapse, that
+region carried the evidence; if nothing changes, it did not:
+
+![covering the important strokes of an 8 drops P(8) from 1.00 to 0.00; covering empty space leaves it at 1.00](https://raw.githubusercontent.com/tekinertekin/mantissa-interpret/main/assets/how_occlusion_works.png)
+
+That single experiment *is* the first method. The three methods here are three
+answers to "what did the model use?", trading detail for cost and clarity:
+
+- **Occlusion** — exactly the picture above, done everywhere: slide a patch
+  over the image and record how much each position, when hidden, drops the
+  class probability. No math beyond running the model forward; the price is one
+  forward pass per patch, and the resolution is patch-sized.
+- **Saliency** — instead of hiding pixels, ask calculus which pixels *matter*:
+  the gradient of the class score with respect to each input pixel. A large
+  gradient means "nudging this pixel would move the score a lot." It is
+  per-pixel sharp but noisy, and needs a single backward pass.
+- **Grad-CAM** — work inside the network. At a convolutional layer each channel
+  is a learned feature detector over a coarse grid; weight every channel by how
+  much raising its activation would raise the target score (its gradient),
+  add them up, and keep the positive part. The result is a smooth,
+  **class-discriminative** region — ask about "3" vs "7" on the same image and
+  the map moves — for one backward pass.
+
+Occlusion and saliency answer *which pixels*; Grad-CAM answers *which region,
+for this class*. Together they are the standard first tools for debugging a
+model that is right for the wrong reason.
+
+## Install
+
+```sh
+pip install mantissa-interpret
+```
+
+Pulls in `mantissa-cnn` (and transitively `mantissa-nn` + the `mantissa-core`
+engine). For plotting the heatmaps, `pip install mantissa-interpret[viz]`.
+
+## The three methods
+
+| method | cost | granularity | class-discriminative | needs |
+|---|---|---|---|---|
+| `occlusion_map` | forward only, O(patches) | coarse (patch) | yes | — |
+| `saliency_map` | one backward pass | per-pixel (noisy) | weakly | input gradient |
+| `grad_cam` | one backward pass | per-region (smooth) | **yes** | activations + their gradient at a conv layer |
+
+Each takes a fitted model and a single `(C, H, W)` image and returns a 2-D
+heatmap normalized to `[0, 1]`, aligned to the input.
+
+### Occlusion — `occlusion_map`
+
+Slide a `patch`×`patch` window (filled with `fill`) across the image; at each
+position, hide that window and re-run the model. If the target class
+probability drops a lot, those pixels were important. The heatmap is the
+per-pixel average drop (overlapping windows blend), normalized to `[0, 1]`.
+
+```python
+from mantissa_interpret import occlusion_map
+heat = occlusion_map(net, image, target_class=7, patch=7, stride=3)
+```
+
+Only forward passes — model-agnostic and dead simple, but coarse (patch-sized)
+and its cost scales with the number of windows. All occluded copies are run in
+a single batched forward pass.
+
+### Saliency — `saliency_map`
+
+The first-order sensitivity of the class score to each pixel is just its
+gradient. One forward pass primes the layers, we seed the gradient of the
+target *logit* (a one-hot vector), and backpropagate to the input — the same
+`backward` the model trains with, carried one step past the first layer's
+weights down to the pixels. The map is `|gradient|` reduced over channels.
+
+```python
+from mantissa_interpret import saliency_map
+heat = saliency_map(net, image, target_class=7)
+```
+
+Per-pixel and cheap (one backward pass), but high-frequency and noisy, and only
+weakly class-discriminative — good for "which strokes", not "which region".
+
+### Grad-CAM — `grad_cam`
+
+At a convolutional layer, each channel is a learned feature detector over a
+coarse grid. Grad-CAM weights every channel by the average gradient of the
+target logit w.r.t. that channel's activation (how much "more of this feature"
+raises the class score), sums the channels with those weights, keeps the
+positive part, and upsamples to the image. We capture the layer's activation on
+the forward pass and backpropagate the logit down to that same layer for the
+gradient — then combine.
+
+```python
+from mantissa_interpret import grad_cam
+heat = grad_cam(net, image, target_class=7)      # last Conv2D by default
+```
+
+Coarse (conv-grid resolution) but smooth and genuinely **class-discriminative**
+— asking about different classes on the same image yields different maps.
+`target_layer` selects which conv layer to read (default: the last one).
+
+## Results
+
+A LeNet-5 trained on an 8k-image MNIST subset with the mantissa C engine
+(3 epochs, **96% test accuracy**), then explained on held-out digits. Reproduce
+with `python examples/heatmaps_demo.py`.
+
+![occlusion, saliency and Grad-CAM heatmaps on MNIST digits](https://raw.githubusercontent.com/tekinertekin/mantissa-interpret/main/assets/heatmaps.png)
+
+Reading across a row, the three methods agree on the digit but differ exactly
+as their math predicts:
+
+- **occlusion** — coarse, patch-sized blobs on the strokes whose removal costs
+  the class the most;
+- **saliency** — the finest detail, tracing individual strokes, but visibly
+  noisy (the raw per-pixel gradient);
+- **Grad-CAM** — a smooth region over the class-defining strokes (the 7's
+  diagonal, the 2's base, the 9's loop), with the least clutter.
+
+Grad-CAM is also **class-discriminative** — ask it about a different target
+class on the *same* image and the map moves:
+
+![Grad-CAM for two different target classes on the same 7](https://raw.githubusercontent.com/tekinertekin/mantissa-interpret/main/assets/gradcam_classes.png)
+
+All of this — feature maps, gradients, the optimization-free attribution — comes
+out of the same low-precision C kernels the model was trained on.
+
+## License
+
+MIT — Tekin Ertekin.

mantissa_interpret-0.1.0/mantissa_interpret.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+LICENSE
+README.md
+pyproject.toml
+mantissa_interpret/__init__.py
+mantissa_interpret/gradcam.py
+mantissa_interpret/occlusion.py
+mantissa_interpret/saliency.py
+mantissa_interpret.egg-info/PKG-INFO
+mantissa_interpret.egg-info/SOURCES.txt
+mantissa_interpret.egg-info/dependency_links.txt
+mantissa_interpret.egg-info/requires.txt
+mantissa_interpret.egg-info/top_level.txt

mantissa_interpret-0.1.0/mantissa_interpret.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

mantissa_interpret-0.1.0/mantissa_interpret.egg-info/requires.txt

@@ -0,0 +1,11 @@
+numpy>=1.20
+mantissa-cnn>=0.2.2
+
+[bench]
+matplotlib>=3.5
+
+[dev]
+pytest>=7
+
+[viz]
+matplotlib>=3.5

mantissa_interpret-0.1.0/mantissa_interpret.egg-info/top_level.txt

@@ -0,0 +1 @@
+mantissa_interpret

mantissa_interpret-0.1.0/pyproject.toml

@@ -0,0 +1,33 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "mantissa-interpret"
+version = "0.1.0"
+description = "CNN interpretability on the mantissa C engine: occlusion, saliency and Grad-CAM"
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+authors = [{ name = "Tekin Ertekin" }]
+dependencies = [
+    "numpy>=1.20",
+    # Operates on a fitted mantissa_cnn.Sequential (its layers + backend), so
+    # everything runs through the same C engine the model was trained on.
+    # mantissa-cnn transitively pulls in mantissa-nn and the mantissa-core
+    # engine. For the dev layout, `pip install -e ../cnn`.
+    "mantissa-cnn>=0.2.2",
+]
+
+[project.optional-dependencies]
+viz = ["matplotlib>=3.5"]
+bench = ["matplotlib>=3.5"]
+dev = ["pytest>=7"]
+
+[project.urls]
+Homepage = "https://github.com/tekinertekin/mantissa-interpret"
+Base = "https://github.com/tekinertekin/mantissa-cnn"
+Engine = "https://github.com/tekinertekin/mantissa"
+
+[tool.setuptools]
+packages = ["mantissa_interpret"]

mantissa_interpret-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+