interp-engine 1.12.0__tar.gz → 1.13.0__tar.gz

{interp_engine-1.12.0 → interp_engine-1.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: interp-engine
-Version: 1.12.0
+Version: 1.13.0
 Summary: A fast, standardized, and easy to use interpretability engine.
 Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
 Project-URL: Repository, https://github.com/decoderesearch/interp-engine
@@ -8,6 +8,7 @@ Project-URL: Issues, https://github.com/decoderesearch/interp-engine/issues
 License-Expression: Apache-2.0
 License-File: LICENSE
 Requires-Python: <3.14,>=3.11
+Requires-Dist: accelerate>=1.0
 Requires-Dist: einops
 Requires-Dist: numpy>=1.24
 Requires-Dist: torch>=1.10
@@ -25,6 +26,10 @@ Requires-Dist: transformer-lens<4,>=3.0; extra == 'parity'
 Provides-Extra: quant
 Requires-Dist: accelerate>=1.0; extra == 'quant'
 Requires-Dist: kernels<0.17.0,>=0.15.2; extra == 'quant'
+Provides-Extra: server
+Requires-Dist: fastapi>=0.115; extra == 'server'
+Requires-Dist: httpx>=0.27; extra == 'server'
+Requires-Dist: uvicorn>=0.30; extra == 'server'
 Provides-Extra: vllm
 Requires-Dist: vllm==0.28.0; (sys_platform == 'linux') and extra == 'vllm'
 Description-Content-Type: text/markdown
@@ -72,7 +77,7 @@ pip install interp-engine          # eager backend only
 ## Simple Usage
 
 ```python
-from interp_engine import Address, load_model, run_with_cache
+from interp_engine import Address, capture, load_model
 
 # VLLM (default): low VRAM, medium speed, every point, chosen per request
 model = load_model("Qwen/Qwen3-8B")
@@ -87,7 +92,7 @@ model = load_model("Qwen/Qwen3-8B")
 # model = load_model("Qwen/Qwen3-8B", backend="eager")
 
 point = Address("resid_post", 10)      # or string: "resid_post.10"
-cache = run_with_cache(model, model.to_tokens("Hello, world"), [point])
+cache = capture(model, model.to_tokens("Hello, world"), [point])
 cache[point]  # [batch, pos, ...]
 ```
 
@@ -107,27 +112,27 @@ vLLM gives `interp-engine` high throughput via concurrency, and `backend="vllm-s
 
 <!-- Generated by `python -m benchmarks.report_bench`. Do not edit: rerun the sweep. -->
 
-Measured on NVIDIA B200, bf16, 512-token prompt, 128 new tokens.
+Measured on NVIDIA B200, 512-token prompt, 128 new tokens.
 
 One stream (tok/s):
 
-| model                    | eager | vLLM       | vLLM + static taps |
-| ------------------------ | ----- | ---------- | ------------------ |
-| `gemma-2-2b`             | 75    | 88 (1.2x)  | **220 (2.9x)**     |
-| `qwen3-4b`               | 60    | 136 (2.3x) | **313 (5.2x)**     |
-| `llama-3.1-8b`           | 84    | 163 (1.9x) | **258 (3.1x)**     |
-| `qwen3.8-27b`            | 25    | 35 (1.4x)  | **63 (2.5x)**      |
-| `deepseek-v4-flash-0731` | 5.3   | 15 (2.9x)  | **132 (25x)**      |
+| model                    | weights | eager | vLLM       | vLLM + static taps |
+| ------------------------ | ------- | ----- | ---------- | ------------------ |
+| `gemma-2-2b`             | bf16    | 75    | 88 (1.2x)  | **220 (2.9x)**     |
+| `qwen3-4b`               | bf16    | 60    | 136 (2.3x) | **313 (5.2x)**     |
+| `llama-3.1-8b`           | bf16    | 84    | 163 (1.9x) | **258 (3.1x)**     |
+| `qwen3.8-27b`            | bf16    | 25    | 35 (1.4x)  | **63 (2.5x)**      |
+| `deepseek-v4-flash-0731` | fp8     | 5.3   | 15 (2.9x)  | **132 (25x)**      |
 
 8 concurrent requests (aggregate tok/s):
 
-| model                    | eager | vLLM          | vLLM + static taps |
-| ------------------------ | ----- | ------------- | ------------------ |
-| `gemma-2-2b`             | 75    | 675 (9.0x)    | **1,436 (19.2x)**  |
-| `qwen3-4b`               | 61    | 973 (15.9x)   | **2,219 (36x)**    |
-| `llama-3.1-8b`           | 81    | 1,187 (14.6x) | **1,755 (22x)**    |
-| `qwen3.8-27b`            | 24    | 246 (10.3x)   | **408 (17.0x)**    |
-| `deepseek-v4-flash-0731` | 5.4   | 117 (22x)     | **631 (116x)**     |
+| model                    | weights | eager | vLLM          | vLLM + static taps |
+| ------------------------ | ------- | ----- | ------------- | ------------------ |
+| `gemma-2-2b`             | bf16    | 75    | 675 (9.0x)    | **1,436 (19.2x)**  |
+| `qwen3-4b`               | bf16    | 61    | 973 (15.9x)   | **2,219 (36x)**    |
+| `llama-3.1-8b`           | bf16    | 81    | 1,187 (14.6x) | **1,755 (22x)**    |
+| `qwen3.8-27b`            | bf16    | 24    | 246 (10.3x)   | **408 (17.0x)**    |
+| `deepseek-v4-flash-0731` | fp8     | 5.4   | 117 (22x)     | **631 (116x)**     |
 
 <!-- THROUGHPUT:END -->
 

{interp_engine-1.12.0 → interp_engine-1.13.0}/README.md

@@ -41,7 +41,7 @@ pip install interp-engine          # eager backend only
 ## Simple Usage
 
 ```python
-from interp_engine import Address, load_model, run_with_cache
+from interp_engine import Address, capture, load_model
 
 # VLLM (default): low VRAM, medium speed, every point, chosen per request
 model = load_model("Qwen/Qwen3-8B")
@@ -56,7 +56,7 @@ model = load_model("Qwen/Qwen3-8B")
 # model = load_model("Qwen/Qwen3-8B", backend="eager")
 
 point = Address("resid_post", 10)      # or string: "resid_post.10"
-cache = run_with_cache(model, model.to_tokens("Hello, world"), [point])
+cache = capture(model, model.to_tokens("Hello, world"), [point])
 cache[point]  # [batch, pos, ...]
 ```
 
@@ -76,27 +76,27 @@ vLLM gives `interp-engine` high throughput via concurrency, and `backend="vllm-s
 
 <!-- Generated by `python -m benchmarks.report_bench`. Do not edit: rerun the sweep. -->
 
-Measured on NVIDIA B200, bf16, 512-token prompt, 128 new tokens.
+Measured on NVIDIA B200, 512-token prompt, 128 new tokens.
 
 One stream (tok/s):
 
-| model                    | eager | vLLM       | vLLM + static taps |
-| ------------------------ | ----- | ---------- | ------------------ |
-| `gemma-2-2b`             | 75    | 88 (1.2x)  | **220 (2.9x)**     |
-| `qwen3-4b`               | 60    | 136 (2.3x) | **313 (5.2x)**     |
-| `llama-3.1-8b`           | 84    | 163 (1.9x) | **258 (3.1x)**     |
-| `qwen3.8-27b`            | 25    | 35 (1.4x)  | **63 (2.5x)**      |
-| `deepseek-v4-flash-0731` | 5.3   | 15 (2.9x)  | **132 (25x)**      |
+| model                    | weights | eager | vLLM       | vLLM + static taps |
+| ------------------------ | ------- | ----- | ---------- | ------------------ |
+| `gemma-2-2b`             | bf16    | 75    | 88 (1.2x)  | **220 (2.9x)**     |
+| `qwen3-4b`               | bf16    | 60    | 136 (2.3x) | **313 (5.2x)**     |
+| `llama-3.1-8b`           | bf16    | 84    | 163 (1.9x) | **258 (3.1x)**     |
+| `qwen3.8-27b`            | bf16    | 25    | 35 (1.4x)  | **63 (2.5x)**      |
+| `deepseek-v4-flash-0731` | fp8     | 5.3   | 15 (2.9x)  | **132 (25x)**      |
 
 8 concurrent requests (aggregate tok/s):
 
-| model                    | eager | vLLM          | vLLM + static taps |
-| ------------------------ | ----- | ------------- | ------------------ |
-| `gemma-2-2b`             | 75    | 675 (9.0x)    | **1,436 (19.2x)**  |
-| `qwen3-4b`               | 61    | 973 (15.9x)   | **2,219 (36x)**    |
-| `llama-3.1-8b`           | 81    | 1,187 (14.6x) | **1,755 (22x)**    |
-| `qwen3.8-27b`            | 24    | 246 (10.3x)   | **408 (17.0x)**    |
-| `deepseek-v4-flash-0731` | 5.4   | 117 (22x)     | **631 (116x)**     |
+| model                    | weights | eager | vLLM          | vLLM + static taps |
+| ------------------------ | ------- | ----- | ------------- | ------------------ |
+| `gemma-2-2b`             | bf16    | 75    | 675 (9.0x)    | **1,436 (19.2x)**  |
+| `qwen3-4b`               | bf16    | 61    | 973 (15.9x)   | **2,219 (36x)**    |
+| `llama-3.1-8b`           | bf16    | 81    | 1,187 (14.6x) | **1,755 (22x)**    |
+| `qwen3.8-27b`            | bf16    | 24    | 246 (10.3x)   | **408 (17.0x)**    |
+| `deepseek-v4-flash-0731` | fp8     | 5.4   | 117 (22x)     | **631 (116x)**     |
 
 <!-- THROUGHPUT:END -->
 

{interp_engine-1.12.0 → interp_engine-1.13.0}/benchmarks/bench_spec.py

@@ -27,6 +27,9 @@ class ModelSpec:
     precision and does not resolve to the same thing on both backends: eager honors a float32
     checkpoint while vLLM's "auto" downcasts it to bfloat16, which would make that model's row a
     precision comparison rather than a backend one."""
+    weights: str = ""
+    """What the checkpoint's weights stay in when that is not ``dtype`` and its cells do not record
+    it, for the README's weights column. Empty: the cell's quantization, else its dtype."""
     min_gpu_gib: float = 0.0
     """The smallest card this row fits on, in GiB of total VRAM. A sweep that was not given an
     explicit ``--models`` drops the rows the card cannot hold and says which (:func:`default_models`).
@@ -123,6 +126,7 @@ MODELS: tuple[ModelSpec, ...] = (
         "deepseek-ai/DeepSeek-V4-Flash-0731",
         "DeepseekV4",
         "291B (14B active)",
+        weights="fp8",
         min_gpu_gib=170.0,
         # `resid_post` does not exist on this trunk: the block carries `hc_mult` (4) parallel residual
         # streams, and the engine refuses the name rather than silently returning stream 0. `mlp_out`

{interp_engine-1.12.0 → interp_engine-1.13.0}/benchmarks/publish.py

@@ -26,7 +26,7 @@ The display form differs from the report's, deliberately:
   rounding, which is the worse of the two.
 - **the card gets the machine, the README gets the conditions.** Both need the reader to know these
   are one box's numbers, but the card is a hover card that links ``results-latest.md`` for the rest,
-  and the dtype and the two lengths are four more things to read before reaching a figure. So it is
+  and the weights and the two lengths are four more things to read before reaching a figure. So it is
   handed ``BENCHMARK_GPU`` where the README prints the whole line -- the difference is the room each
   has, not a difference of opinion about what a figure means without them.
 """
@@ -34,11 +34,12 @@ The display form differs from the report's, deliberately:
 from __future__ import annotations
 
 import argparse
+import re
 from dataclasses import dataclass
 from pathlib import Path
 from typing import Any
 
-from benchmarks.bench_spec import workload
+from benchmarks.bench_spec import MODELS, workload
 from benchmarks.cells import cell_metric, cells_by_key, load_cells, model_order, nonuniform, row_spec
 from benchmarks.run_bench import RESULTS_DIR
 
@@ -110,7 +111,9 @@ class Conditions:
     concurrency: int
 
     def __str__(self) -> str:
-        return f"{self.gpu}, {self.dtype}, {self.prompt_tokens}-token prompt, {self.new_tokens} new tokens"
+        # The dtype is per row, in the tables' `weights` column: a quantized row runs its matmuls in
+        # 4 or 8 bits whatever the activations are in.
+        return f"{self.gpu}, {self.prompt_tokens}-token prompt, {self.new_tokens} new tokens"
 
 
 def _only(values: set[Any], what: str) -> Any:
@@ -136,6 +139,28 @@ def conditions(cells: list[dict[str, Any]]) -> Conditions:
     )
 
 
+def weights(cells: list[dict[str, Any]]) -> dict[str, str]:
+    """``{model: what its weights ran in}``: `ModelSpec.weights` where the spec states it, else the
+    cell's quantization as a bit width, else its dtype.
+
+    One value per row: a 4-bit figure beside a bf16 one in the same row is not a speedup.
+    """
+    stated = {m.key: m.weights for m in MODELS if m.weights}
+    labels: dict[str, set[str]] = {}
+    for cell in cells:
+        key = cell["model"]["key"]
+        load = cell.get("load") or {}
+        label = stated.get(key, "")
+        if not label and (quantization := load.get("quantization")):
+            bits = re.search(r"(\d+)-bit", str(quantization))
+            label = f"{bits.group(1)}-bit" if bits else str(quantization)
+        if not label and (resolved := load.get("resolved_dtype")):
+            label = DTYPE_SHORT.get(resolved) or resolved
+        if label:
+            labels.setdefault(key, set()).add(label)
+    return {key: str(_only(found, f"what {key}'s weights ran in")) for key, found in labels.items()}
+
+
 def rates(cells: list[dict[str, Any]]) -> dict[str, dict[str, dict[str, float | None]]]:
     """``{model: {variant: {regime: tok/s or None}}}``, in the sweep's row order."""
     index = cells_by_key(cells)
@@ -164,11 +189,15 @@ def _padded(header: list[str], rows: list[list[str]]) -> list[str]:
     return out
 
 
-def _table(measured: dict[str, dict[str, dict[str, float | None]]], regime: str) -> list[str]:
+def _table(
+    measured: dict[str, dict[str, dict[str, float | None]]],
+    regime: str,
+    precision: dict[str, str] | None = None,
+) -> list[str]:
     rows: list[list[str]] = []
     for model_key, per_variant in measured.items():
         baseline = per_variant[BASELINE][regime]
-        row = [f"`{model_key}`"]
+        row = [f"`{model_key}`", (precision or {}).get(model_key, MISSING_CELL)]
         for variant_key, _, _ in COLUMNS:
             value = per_variant[variant_key][regime]
             if value is None:
@@ -179,12 +208,13 @@ def _table(measured: dict[str, dict[str, dict[str, float | None]]], regime: str)
                 text = f"{text} ({fmt_multiplier(value / baseline)})"
             row.append(f"**{text}**" if variant_key == HIGHLIGHT else text)
         rows.append(row)
-    return _padded(["model", *(heading for _, heading, _ in COLUMNS)], rows)
+    return _padded(["model", "weights", *(heading for _, heading, _ in COLUMNS)], rows)
 
 
 def readme_block(cells: list[dict[str, Any]]) -> str:
     where = conditions(cells)
     measured = rates(cells)
+    precision = weights(cells)
     return "\n".join(
         [
             "<!-- Generated by `python -m benchmarks.report_bench`. Do not edit: rerun the sweep. -->",
@@ -193,11 +223,11 @@ def readme_block(cells: list[dict[str, Any]]) -> str:
             "",
             "One stream (tok/s):",
             "",
-            *_table(measured, "single"),
+            *_table(measured, "single", precision),
             "",
             f"{where.concurrency} concurrent requests (aggregate tok/s):",
             "",
-            *_table(measured, "concurrent"),
+            *_table(measured, "concurrent", precision),
         ]
     )
 

{interp_engine-1.12.0 → interp_engine-1.13.0}/benchmarks/report_bench.py

@@ -214,8 +214,9 @@ def _env_section(cells: list[dict[str, Any]]) -> list[str]:
 
 def _models_section(cells: list[dict[str, Any]]) -> list[str]:
     index = cells_by_key(cells)
+    precision = publish.weights(cells)
     lines = [
-        "| model | HuggingFace id | family | params | native dtype | layers | d_model |",
+        "| model | HuggingFace id | family | params | weights | layers | d_model |",
         "| --- | --- | --- | --- | --- | --- | --- |",
     ]
     for model_key in model_order(cells):
@@ -230,7 +231,8 @@ def _models_section(cells: list[dict[str, Any]]) -> list[str]:
         load = cell.get("load", {})
         lines.append(
             f"| `{model_key}` | `{m['hf_id']}` | {m['family']} | {m['params']} | "
-            f"{m.get('native_dtype', '?')} | {load.get('n_layers', '?')} | {load.get('d_model', '?')} |"
+            f"{precision.get(model_key) or m.get('native_dtype', '?')} | {load.get('n_layers', '?')} | "
+            f"{load.get('d_model', '?')} |"
         )
     return lines
 

{interp_engine-1.12.0 → interp_engine-1.13.0}/benchmarks/results-latest.md

@@ -3,7 +3,7 @@
 Generated by `python -m benchmarks.report_bench`. Do not edit by hand -- rerun the sweep.
 
 - **Run** 2026-09-04 (cells from `2026-09-04T07:18:04+00:00` to `2026-09-04T08:24:40+00:00` UTC)
-- **Report written** 2026-09-04T08:32:25+00:00
+- **Report written** 2026-09-24T04:33:12+00:00
 - **Cells** 20
 
 ## What this measures
@@ -61,13 +61,13 @@ Taken from the deployed set rather than invented, and spanning the range a singl
 hold: from a 2.6B dense trunk to a 291B sparse one. A model whose weights do not fit is
 dropped from a plain sweep and named, rather than failing it (`ModelSpec.min_gpu_gib`).
 
-| model | HuggingFace id | family | params | native dtype | layers | d_model |
+| model | HuggingFace id | family | params | weights | layers | d_model |
 | --- | --- | --- | --- | --- | --- | --- |
-| `gemma-2-2b` | `google/gemma-2-2b` | Gemma2 | 2.6B | float32 | 26 | 2304 |
-| `qwen3-4b` | `Qwen/Qwen3-4B` | Qwen3 | 4.0B | bfloat16 | 36 | 2560 |
-| `llama-3.1-8b` | `meta-llama/Llama-3.1-8B` | Llama3 | 8.0B | bfloat16 | 32 | 4096 |
-| `qwen3.8-27b` | `Qwen/Qwen3.8-27B` | Qwen3_5 | 27B | bfloat16 | 64 | 5120 |
-| `deepseek-v4-flash-0731` | `deepseek-ai/DeepSeek-V4-Flash-0731` | DeepseekV4 | 291B (14B active) | bfloat16 | 43 | 4096 |
+| `gemma-2-2b` | `google/gemma-2-2b` | Gemma2 | 2.6B | bf16 | 26 | 2304 |
+| `qwen3-4b` | `Qwen/Qwen3-4B` | Qwen3 | 4.0B | bf16 | 36 | 2560 |
+| `llama-3.1-8b` | `meta-llama/Llama-3.1-8B` | Llama3 | 8.0B | bf16 | 32 | 4096 |
+| `qwen3.8-27b` | `Qwen/Qwen3.8-27B` | Qwen3_5 | 27B | bf16 | 64 | 5120 |
+| `deepseek-v4-flash-0731` | `deepseek-ai/DeepSeek-V4-Flash-0731` | DeepseekV4 | 291B (14B active) | fp8 | 43 | 4096 |
 
 ### Where a row differs
 

{interp_engine-1.12.0 → interp_engine-1.13.0}/docs/AGENT_INTEGRATION.md

@@ -7,6 +7,7 @@ If you are a human reading this: [USAGE.md](USAGE.md) is the friendlier introduc
 [PORTING.md](PORTING.md) covers hook-name translation in more depth.
 
 - [Version stamp](#version-stamp)
+- [`capture` and `run_with_cache`](#capture-and-run_with_cache)
 - [What changed in 1.3](#what-changed-in-13)
 - [What changed in 1.1](#what-changed-in-11)
 - [Pick a tier first](#pick-a-tier-first)
@@ -36,6 +37,18 @@ a minor version when keeping it would mean one of the two backends could not be
 [What changed in 1.1](#what-changed-in-11) — and pinning `interp-engine~=1.3` is the way to not be
 surprised by the next one.
 
+## `capture` and `run_with_cache`
+
+The sync single-forward capture is `capture(model, tokens, points)`. `run_with_cache` is an alias
+with the same signature and the same `Cache`, kept for code and habits that came from
+TransformerLens; nothing is deprecated and nothing raises. New code should say `capture`.
+
+Two other things share the word. `await model.capture(...)` is the async *method* on the protocol:
+one prompt, a plain dict keyed by `Address`, no batch axis. `interp_engine.capture` is the
+*submodule* that holds `Cache` and the capture helpers; `from interp_engine.capture import Cache`
+works as before, but after `import interp_engine` the attribute `interp_engine.capture` is the
+function.
+
 ## What changed in 1.3
 
 **"Freeze" is now "static", and the mode is a `backend=` value rather than a kwarg that turned it
@@ -86,7 +99,7 @@ edits and gains one capability.
 | `OrthogonalProjector(...).get_P()` / `.get_orthogonal_complement()` | `.delta(acts)` / `.project(acts)` | The projection matrix was `d_model × d_model` materialized per call; the same arithmetic is one dot product. |
 | `run_with_cache(model, ..., **forward_kwargs)` | *(gone)* | Passthrough kwargs were unreachable on vLLM, so a call using them silently meant something different per backend. |
 
-What is new, rather than moved: `run_with_cache`, `capture_generation`, `capture_attention`,
+What is new, rather than moved: `capture` (then still spelled `run_with_cache`), `capture_generation`, `capture_attention`,
 `generate_stream`, `capture_residuals`, `layer_logits` and `steer` all take either backend now, and
 `sync_model(model)` gives you the whole protocol without `await`. `ProjectionCapSpec` works on eager
 (it used to raise `NotImplementedError`), so all three steering specs run on both backends.
@@ -145,8 +158,8 @@ Keyed on the pattern you find in the source you are porting.
 | `LLM(model=id)` + you own that engine | `worker_extension_cls=WORKER_EXTENSION_CLS` | Tier 1. Merge `capture_engine_kwargs()`. |
 | `LLM(model=id)` + you do not care how | `load_model(id)` | Tier 2. |
 | `HookedTransformer.from_pretrained(id)` | `EagerModel.from_pretrained(id)` | Then translate hook names — next row. |
-| `model.run_with_cache(...)["blocks.7.attn.hook_z"]` | `run_with_cache(model, tokens, [tlens_hook_to_point("blocks.7.attn.hook_z", model)])` | **Pass the model.** See the warning below. |
-| nnsight `with model.trace(...)` / `mlps_output[7]` | `nnsight_accessor_to_point("mlps_output[7]")` | Then `run_with_cache` or `await model.capture(...)`. |
+| `model.run_with_cache(...)["blocks.7.attn.hook_z"]` | `capture(model, tokens, [tlens_hook_to_point("blocks.7.attn.hook_z", model)])` | **Pass the model.** See the warning below. |
+| nnsight `with model.trace(...)` / `mlps_output[7]` | `nnsight_accessor_to_point("mlps_output[7]")` | Then `capture(model, ...)` or `await model.capture(...)`. |
 | `model.unembed(model.ln_final(x))` | `await model.decode_residuals(x)` | The method normalizes softcapping across backends; the sync free function does not. |
 | a hand-rolled `for _ in range(n): model(...)` decode loop | `generate_stream(model, tokens)` | Yields a `GenStep` per token with logprobs, and logits on eager (rule 12). |
 | `asyncio.run(model.capture(...))` in a script, once per call | `sync = sync_model(model)`, then `sync.capture(...)` | One background loop for the model's lifetime, instead of a new one per call — which on vLLM would abandon the engine's tasks. |
@@ -205,7 +218,7 @@ an obvious one.
 
 1. **Every model *method* is `async`,** including on eager where the work underneath is synchronous.
    `model.capture(...)` without `await` gives you a coroutine, not activations. The sync free
-   functions (`run_with_cache`, `capture_generation`, `capture_attention`, `generate_stream`,
+   functions (`capture`, `capture_generation`, `capture_attention`, `generate_stream`,
    `steer`, `layer_logits`) take either backend, and `sync_model(model)` mirrors the whole protocol
    for the times you want a method. Neither can be called from inside a running event loop — they
    refuse, naming the `await` to use instead, rather than deadlocking. The mirror image also
@@ -239,9 +252,9 @@ an obvious one.
 7. **`await model.shutdown()` before loading a second model in the same process on vLLM.** Its KV
    cache lives in a child process that dropping a Python reference does not reap. Idempotent, so
    there is no cost to calling it always.
-8. **`capture()` returns a plain dict keyed by `Address`, not by string.** You may ask with a string
+8. **`await model.capture()` returns a plain dict keyed by `Address`, not by string.** You may ask with a string
    or a `(name, layer)` tuple, but on the way back out use `to_address("resid_post.10")` as the key.
-   (`run_with_cache`'s `Cache` is the exception and accepts either.)
+   (The free function `capture(model, ...)` returns a `Cache`, which accepts either.)
 9. **Do not apply `final_logit_softcapping` after `await model.decode_residuals(...)`.** The method
    applies the model's configured value so the two backends are comparable. The sync free function
    `interp_engine.decode_residuals` is the raw one and takes `softcap` explicitly.
@@ -289,9 +302,9 @@ mistake you made.
 
 ## When you are not sure, ask the model
 
-Capabilities are queryable, cheap, and answerable **before** `warmup()` — they read configuration
-rather than running a forward. Gate on these rather than on backend name or model id, because that is
-what keeps one code path working across both backends:
+Capabilities are queryable and cheap: they read configuration rather than running a forward, and the
+model-wide ones below are answerable **before** `warmup()`. Gate on these rather than on backend name
+or model id, because that is what keeps one code path working across both backends:
 
 ```python
 from interp_engine import load_model
@@ -304,6 +317,28 @@ model.residual_basis.lens_valid      # is a logit-lens read-out meaningful here?
 model.n_layers, model.d_model        # remember z is n_heads * head_dim, not d_model
 ```
 
+For one point rather than the whole model, ask `refuses()`. It returns the reason as a string, or
+`None` when the point is servable, so a caller filtering a list gets to say which gap it hit rather
+than logging "unavailable":
+
+```python
+for point in ("resid_post", "router_logits", "q_norm_out"):
+    why = model.refuses(point, 12)
+    print(point, "ok" if why is None else f"skipped: {why}")
+```
+
+**Do not rebuild this from the point tables in these docs.** `SUPPORTED_POINTS.md` and
+`ENGINE_HOOK_MAPPINGS.md` say what a *point* is, per backend; `refuses()` also knows what this
+*checkpoint* carries and how this instance was built, which no table can. A serving pod that derived
+its advertised set from a table instead shipped a 400 for attention-output SAEs that blamed the
+paged-attention kernel, and a second one that narrowed by tensor-parallel shard width refused points
+the worker gathers. Both read as maintained lists right up until they were wrong.
+
+The one caveat is the order on vLLM: the architecture half of the answer lives in the worker
+process, so it arrives at `warmup()`. Before that, vLLM's `refuses()` answers from the point table
+and how the engine was built, which can say yes to a point the checkpoint has no module for. Warm up
+before advertising. Eager holds its own module tree and needs no warmup.
+
 For a coarse "which surface do I have", `isinstance(model, InterpModel)` works —
 but note it checks method *presence* only, not signatures:
 

{interp_engine-1.12.0 → interp_engine-1.13.0}/docs/ARCHITECTURE_QUIRKS.md

@@ -311,7 +311,7 @@ must split", on a family whose `fused_qkv` is false — so `value` no longer ask
 **Reading a norm costs a rank, and the point does not pay it.** A norm over `head_dim` can only be
 given the per-head view, so both engines hand `v_norm` a `(…, n_kv_heads, head_dim)` tensor where a
 `v_proj` would have produced the flat one — and `value` would then be 4-D on this family and 3-D on the
-next, against a `Width.HEADS` declaration and `run_with_cache`'s `[batch, seq, width]`. Flattened back
+next, against a `Width.HEADS` declaration and `capture`'s `[batch, seq, width]`. Flattened back
 on both sides (`hooks.flat_per_head`, `vllm_capture._hooks.flat_value`), including on the way *into* a
 steer, so a vector measured on a capture of `value` is the shape the write expects; the module gets its
 own rank back before the attention reshapes it. Eagerly the head count is checked rather than assumed,
@@ -501,7 +501,7 @@ router_logits)`. `facts.ROUTER_OUTPUTS` holds the exceptions; `facts.assert_rout
 - **A fused kernel can leave the router module unused.** transformers' MXFP4 path for gpt-oss replaces
   the _block's_ forward and routes inline (`F.linear` on `self.router.weight`, then a Triton top-k), so
   the router is in the tree, correct, and never called. `arch.moe_router` detects the replaced forward
-  and refuses with the remedy (`Mxfp4Config(dequantize=True)`); more generally, `run_with_cache` now
+  and refuses with the remedy (`Mxfp4Config(dequantize=True)`); more generally, `capture` now
   raises when a resolved module did not run, rather than returning a short cache. `router_logits` is the
   exception, and it does not need the remedy: `mlp_forward` returns `(routed_out, router_logits)`, so the
   point resolves to the _block's_ `output:1` instead (`arch.inline_routing_logits`) and is bit-identical
@@ -510,7 +510,7 @@ router_logits)`. `facts.ROUTER_OUTPUTS` holds the exceptions; `facts.assert_rout
   index — a four-wide top-k against the logits' 32 — and gpt-oss carries a second, unverified kernel
   swap (`@use_kernel_forward_from_hub("MegaBlocksMoeMLP")`) whose layout nothing here has confirmed.
   `expert_weights` and `expert_indices` have no boundary on that path even so, and are _rebuilt_ from
-  those logits by `run_with_cache` (`interp_engine.moe_routing`) — the one place a routing convention is
+  those logits by `capture` (`interp_engine.moe_routing`) — the one place a routing convention is
   reimplemented, allowed because the derivation is a top-k on an already-captured tensor and because it
   is verified against `GptOssTopKRouter` on the real checkpoint rather than read off the source. They
   still refuse from `resolve_point`, which answers with addresses and has none to give.
@@ -811,8 +811,9 @@ Two things a randomly-initialised model cannot show you, both worth knowing when
 - **`head_dim ≠ d_model/n_heads`** on Gemma (GQA + explicit `head_dim`): the attention-output /
   `z` width is `n_heads*head_dim`, not `d_model`. Attention-SAE code must use `head_dim`.
 - **Embedding scaling** (Gemma multiplies embeddings by `sqrt(d_model)`): free when you hook real
-  modules, but if you ever feed embeddings _in_ (NLA `prompt_embeds`), you must apply the scale
-  yourself — `resolve_embed_scale` in the NLA path exists for exactly this.
+  modules. When you feed embeddings _in_ (`generate_steps_from_embeds`, which NLA's concept
+  injection rides), the rows must carry the scale already — they are the `embeddings` point, and
+  the NLA path's `resolve_embed_scale` applies it.
 - **Multimodal `*ForConditionalGeneration`**: dims live under `config.text_config` (top-level are
   `None`); the text trunk nests under `model.language_model`. `resolve_arch` handles both; just
   don't assume top-level `num_hidden_layers`. The vLLM worker resolves its _own_ module handles

{interp_engine-1.12.0 → interp_engine-1.13.0}/docs/ENGINE_HOOK_MAPPINGS.md

@@ -126,11 +126,11 @@ The norm's _boundaries_ are `resid_mid` (input) and `mlp_in` (output), and the e
 for you rather than making you reproduce TransformerLens' `RMSNorm.forward` from memory:
 
 ```python
-from interp_engine import pre_gain_normalized, rms_norm_eps_for_model, run_with_cache, tlens_normalized_hook
+from interp_engine import capture, pre_gain_normalized, rms_norm_eps_for_model, tlens_normalized_hook
 
 address = tlens_normalized_hook("blocks.19.ln2.hook_normalized")  # -> Address("resid_mid", 19)
 eps = rms_norm_eps_for_model(model)  # None on a LayerNorm family, where this equation does not apply
-cache = run_with_cache(model, tokens, [address])
+cache = capture(model, tokens, [address])
 normalized = pre_gain_normalized(cache[address], eps)
 ```
 
@@ -246,7 +246,7 @@ module is present and never called. `router_logits` survives that anyway — `ml
 `return routed_out, router_logits`, so the logits the kernel routed on leave the block, and the point
 resolves to the block's own `output:1`, bit-identical to `F.linear(hidden, router.weight, router.bias)`.
 `expert_weights` and `expert_indices` have no boundary there at all — the selection happens inside the
-kernel — so on that path they are **rebuilt** from those logits by `run_with_cache`
+kernel — so on that path they are **rebuilt** from those logits by `capture`
 (`interp_engine.moe_routing`), and `resolve_point` still refuses them because there is no address to
 return. The index the logits come off is allowlisted per replacement (`facts.INLINE_ROUTING_FORWARDS`)
 rather than assumed, because `output:1` on the _un-replaced_ gpt-oss block is `router_scores` — the
@@ -422,12 +422,13 @@ Both mechanisms carry the optional `stream` coordinate, which on a write means o
 in one row of a stack the worker already holds — a weaker claim than the one a read makes, which is why
 vLLM serves it here while still refusing to *read* a single stream through the resid points.
 
-A jlens steer/ablate/swap goes through the same gate and the same sites: its spec carries a `point`
-(defaulting to `resid_post`, which is what it has always meant on a conventional trunk) and the
-per-request path keys the intervention by site rather than by layer. That is what makes the lens usable
-on this family at all — `resid_post` there names the whole stack, which no sublayer reads, so the tensor
-a swap wants is a collapse. The global `set_lens_intervention` stays pinned to the decoder layer's output
-and refuses a spec that names a point, rather than ignoring it.
+A lens steer/ablate/swap is a `steer()` block of `NormScaledAddSpec` / `AblateSpec` / `SwapSpec` ops
+and goes through the same gate and the same sites: the spec's `point` (`resid_post` by default, which
+is what it has always meant on a conventional trunk) is where the per-request path keys the write.
+That is what makes the lens usable on this family at all — `resid_post` there names the whole stack,
+which no sublayer reads, so the tensor a swap wants is a collapse. The global `set_lens_intervention`
+stays pinned to the decoder layer's output and refuses a spec that names a point, rather than ignoring
+it.
 
 ### On the AMD and XPU trees they are ordinary modules
 

{interp_engine-1.12.0 → interp_engine-1.13.0}/docs/GRADIENTS.md

@@ -26,7 +26,7 @@ gradient *of the model* is eager-only.
 
 ```python
 model = EagerModel("openai-community/gpt2", requires_grad=True)  # default is False
-cache = run_with_cache(model, input_ids, [("resid_post", 5)], detach=False)
+cache = capture(model, input_ids, [("resid_post", 5)], detach=False)
 cache.get("resid_post", 5).sum().backward()
 model.arch.embed.weight.grad  # populated: the graph reached the embedding
 ```

{interp_engine-1.12.0 → interp_engine-1.13.0}/docs/INTERNALS.md

@@ -25,7 +25,8 @@ files and why they are where they are.
   `sync.py`; `tests/test_sync_parity.py` fails on a missing twin.
 - `sync.py` / `_loop.py` — `sync_model(model)`: the protocol without an event loop, one explicit
   wrapper per method over a background loop thread that is created lazily, reused per model, and
-  refuses rather than deadlocks when called from inside a running loop.
+  refuses rather than deadlocks when called from inside a running loop. Each call runs in a copy of
+  the caller's context, so an open `steer()` block is what the method reads on the loop thread.
 - `dispatch.py` — the shared plumbing every free function's two arms sit on: token coercion
   (`TokensLike`, batch refusals) and `CAPABILITIES`, the table each `CapabilityUnsupported` message
   is built from.
@@ -56,7 +57,11 @@ files and why they are where they are.
   `tests/test_steer_math_parity.py` runs against it on CPU; `steer()` is the context both backends
   take, registering per-request on vLLM rather than installing a global hook. The methods are named
   once, in `steer_specs.SteerMethod`: the eager `SteerSpec.method` and the worker dict's `op` both
-  read from it, so a method has one spelling on either backend.
+  read from it, so a method has one spelling on either backend. A block's
+  `position_mask` and `generated=False` are placed by `forward_from` on eager, which every engine
+  loop declares, and travel with the request on vLLM (`worker_register_steering`'s
+  `skip_positions` / `steer_generated`); `tests/test_steer_scope.py` checks the rows on eager, in
+  closed form, and `tests/test_vllm_capture_gpu.py` holds vLLM to them.
 - `mappers.py` — translation between canonical points and other frameworks' names:
   TransformerLens hook strings and nnsight/nnterp accessors, both directions. See [Porting from
   TransformerLens, nnsight or nnterp](PORTING.md#porting-from-transformerlens-nnsight-or-nnterp).

{interp_engine-1.12.0 → interp_engine-1.13.0}/docs/PERFORMANCE.md

@@ -24,6 +24,8 @@ the cache normally. Worth ~1.75x on time-to-first-token for a repeated long pref
 pins which requests opt out; `tests/test_vllm_capture_gpu.py` pins the effect on a real engine,
 including a control that reproduces the short capture when the salt is removed. Pass
 `extra_vllm_kwargs={"enable_prefix_caching": False}` to get the old engine-wide behaviour back.
+An engine with `enable_prompt_embeds=True` (LoRA reads, NLA) keeps it on too: vLLM hashes each
+block's embeds rows into its key, and embeds requests take the same global-intervention salt.
 
 One further flag is set for you, and only on the architectures that cannot boot without it.
 `kv_cache_dtype` is left to vLLM everywhere else, but a few families serve attention through a KV
@@ -110,9 +112,9 @@ What is genuinely **free** — do not attribute slowness to these:
 `enforce_eager` is required *while capturing*, not permanently, so a generation-only instance
 should turn it off — preferably as `backend="vllm-generate"`, which is the same engine configuration
 reached through a name that says what it gives up. There is already a precedent in-tree:
-`apps/nla/vllm_verbalizer.py` runs its own `VLLMModel` with `enforce_eager=False` (CUDA graphs +
-inductor on) precisely because the verbalizer installs no hooks, with
-`NLA_VERBALIZER_ENFORCE_EAGER=1` as a debug escape hatch. It also passes
+Neuronpedia's `apps/nla/verbalizer.py` loads `backend="vllm-generate"` (CUDA graphs + inductor on)
+precisely because the verbalizer installs no hooks, with `NLA_VERBALIZER_ENFORCE_EAGER=1` as a
+debug escape hatch back to the hooked engine. It also passes
 `compilation_config.cudagraph_capture_sizes` to match graph capture to its expected fan-out.
 
 What it costs, measured (`benchmarks/results-latest.md`, and gpt2 separately for the graph modes).
@@ -164,9 +166,9 @@ them:
   defaults to `"auto"`: `resid_post` at every layer, to read *and* to write, via static `copy_`
   taps. Turns Dynamo off (`VLLM_USE_BREAKABLE_CUDAGRAPH=1`), so the win is replay without compile.
   Because breakable `add_eager` keeps the wrap as ordinary PyTorch on the live tensor, the write path
-  is not limited to additive — `orthogonal`, `projection_cap` and the j-lens `steer`/`ablate`/`swap`
-  ops all ride the same wrap, per-request and with `position_mask` honoured
-  (`register_static_write`). An op outside that set is refused rather than silently skipped. Auto
+  is not limited to additive — `orthogonal`, `projection_cap` and the lens's
+  `norm_scaled_add`/`ablate`/`swap` ops all ride the same wrap, per-request and with `position_mask`
+  and `generated=False` honoured (`register_static_write`). An op outside that set is refused rather than silently skipped. Auto
   covers the write because the two halves are one decision: a read tap alone serves the lens
   read-out and refuses every steer, ablation and swap derived from it, at an address already tapped.
   Pass `static_writes=[]` for the reads without the write sites, and an explicit `static_points`