interp-engine 1.2.8__tar.gz → 1.3.0__tar.gz

{interp_engine-1.2.8 → interp_engine-1.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: interp-engine
-Version: 1.2.8
+Version: 1.3.0
 Summary: A fast, standardized interpretability engine that supports most modern models and architectures. Powers Neuronpedia.
 Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
 Project-URL: Repository, https://github.com/decoderesearch/interp-engine
@@ -31,22 +31,33 @@ Description-Content-Type: text/markdown
 
 # interp-engine
 
-
-
-🔗 **[interp-engine.org](https://interp-engine.org)**
-
+<p align="center">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ielogo.png" alt="interp-engine logo, a magnifying glass where the handle is a rocket" width="160">
+</p>
+<p align="center">
+  🔗 <a href="https://interp-engine.org"><strong>interp-engine.org</strong></a>
+</p>
+<p align="center">
+  <a href="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml"><img src="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml/badge.svg?branch=main" alt="CI status"></a>
+  <a href="https://pypi.org/project/interp-engine/"><img src="https://img.shields.io/pypi/v/interp-engine.svg" alt="PyPI version"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/pypi/l/interp-engine.svg" alt="Apache-2.0 license"></a>
+  <a href="https://join.slack.com/t/opensourcemechanistic/shared_invite/zt-3z9o0hxjl-MDX9pbATO2qESOazNDLpdQ"><img src="https://img.shields.io/badge/Slack-Open%20Source%20Mechanistic%20Interpretability-4A154B?logo=slack&logoColor=white" alt="Join the Slack"></a>
+</p>
 
 
 `interp-engine` is an interpretability engine that is fast (>40x tok/s vs HF eager), standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.
 
-
-
-
+<p align="center">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ie-perf.png" alt="Tokens per second while capturing and generating: eager against IE-vLLM and IE-vLLM-static on qwen3.8-27b and deepseek-v4-flash-0731, 8 requests in flight" width="100%">
+</p>
+<p align="center">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/interp-engine-demo.gif" alt="interp-engine demo gif" width="100%">
+</p>
 
 This repo contains:
 
-1. `[validator/](validator/)`, which compares/validates it against TransformerLens, and nnsight/nnterp on real architectures.
-2. `[visualizer-web/](visualizer-web/)`, a "cheat sheet" hosted at [interp-engine.org](https://interp-engine.org) of each 'point' (eg `resid_post.16`), standardized across architectures.
+1. [`validator/`](validator/), which compares/validates it against TransformerLens, and nnsight/nnterp on real architectures.
+2. [`visualizer-web/`](visualizer-web/), a "cheat sheet" hosted at [interp-engine.org](https://interp-engine.org) of each 'point' (eg `resid_post.16`), standardized across architectures.
 
 ## Installation
 
@@ -60,13 +71,16 @@ pip install interp-engine          # eager backend only
 ```python
 from interp_engine import Address, load_model, run_with_cache
 
-# VLLM MODE (default): low VRAM, medium speed
-model = load_model("Qwen/Qwen3-8B") 
+# VLLM (default): low VRAM, medium speed, every point, chosen per request
+model = load_model("Qwen/Qwen3-8B")
+
+# VLLM-STATIC: high VRAM, high speed, only the points you declare (default resid_post)
+# model = load_model("Qwen/Qwen3-8B", backend="vllm-static")
 
-# VLLM-FREEZE MODE: high VRAM, high speed, only frozen points (default resid_post)
-# model = load_model("Qwen/Qwen3-8B", freeze_points="auto")
+# VLLM-GENERATE: fastest, generation only -- no capture, no steering
+# model = load_model("Qwen/Qwen3-8B", backend="vllm-generate")
 
-# EAGER MODE: low VRAM, low speed
+# EAGER: low VRAM, low speed
 # model = load_model("Qwen/Qwen3-8B", backend="eager")
 
 point = Address("resid_post", 10)      # or string: "resid_post.10"
@@ -84,7 +98,7 @@ Add "use interp-engine" to your prompt and let your agent figure it out - everyt
 
 ## Performance / Speed
 
-vLLM gives `interp-engine` high throughput via concurrency, and **graph freeze** adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
+vLLM gives `interp-engine` high throughput via concurrency, and `backend="vllm-static"` adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
 
 <!-- THROUGHPUT:START -->
 
@@ -94,34 +108,34 @@ Measured on NVIDIA B200, bf16, 512-token prompt, 128 new tokens.
 
 One stream (tok/s):
 
-| model                    | eager | vLLM       | vLLM + graph freeze |
-| ------------------------ | ----- | ---------- | ------------------- |
-| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**      |
-| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**     |
-| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**      |
-| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**       |
-| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**       |
+| model                    | eager | vLLM       | vLLM + static taps |
+| ------------------------ | ----- | ---------- | ------------------ |
+| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**     |
+| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**    |
+| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**     |
+| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**      |
+| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**      |
 
 8 concurrent requests (aggregate tok/s):
 
-| model                    | eager | vLLM        | vLLM + graph freeze |
-| ------------------------ | ----- | ----------- | ------------------- |
-| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**     |
-| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**     |
-| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**     |
-| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**       |
-| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**      |
+| model                    | eager | vLLM        | vLLM + static taps |
+| ------------------------ | ----- | ----------- | ------------------ |
+| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**    |
+| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**    |
+| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**    |
+| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**      |
+| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**     |
 
 <!-- THROUGHPUT:END -->
 
-**Graph freeze** is opt-in via `freeze_points`, and a frozen engine serves only the set it froze. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; `[benchmarks/results-latest.md](benchmarks/results-latest.md)` has every figure at full precision, including capture, steering and lens latencies; `[benchmarks/README.md](benchmarks/README.md)` has how the tables above are rounded.
+`backend="vllm-static"` is opt-in, and serves only the tap set it declared — `static_points="auto"` by default, or a list you name. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.
 
 ## Correctness
 
 We verify correctness in two main ways:
 
 1. A test suite that checks results across several models - what each check is designed to catch is in [INTERNALS.md](docs/INTERNALS.md#correctness).
-2. A full `validator` comparison engine that checks most hook points across 50+ models, at early, middle and late layers - fully reproducible, with detailed results saved in the git repo at `[validator/](validator/)`.
+2. A full `validator` comparison engine that checks most hook points across 50+ models, at early, middle and late layers - fully reproducible, with detailed results saved in the git repo at [`validator/`](validator/).
 
 ## Why use an Interpretability Engine, instead of just having my AI code whatever it needs on the fly?
 
@@ -145,4 +159,4 @@ Bugs and feature requests belong in [issues](https://github.com/decoderesearch/i
 
 ## License
 
-Apache 2.0
+Apache 2.0

{interp_engine-1.2.8 → interp_engine-1.3.0}/README.md

@@ -1,21 +1,32 @@
 # interp-engine
 
-
-
-🔗 **[interp-engine.org](https://interp-engine.org)**
-
+<p align="center">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ielogo.png" alt="interp-engine logo, a magnifying glass where the handle is a rocket" width="160">
+</p>
+<p align="center">
+  🔗 <a href="https://interp-engine.org"><strong>interp-engine.org</strong></a>
+</p>
+<p align="center">
+  <a href="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml"><img src="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml/badge.svg?branch=main" alt="CI status"></a>
+  <a href="https://pypi.org/project/interp-engine/"><img src="https://img.shields.io/pypi/v/interp-engine.svg" alt="PyPI version"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/pypi/l/interp-engine.svg" alt="Apache-2.0 license"></a>
+  <a href="https://join.slack.com/t/opensourcemechanistic/shared_invite/zt-3z9o0hxjl-MDX9pbATO2qESOazNDLpdQ"><img src="https://img.shields.io/badge/Slack-Open%20Source%20Mechanistic%20Interpretability-4A154B?logo=slack&logoColor=white" alt="Join the Slack"></a>
+</p>
 
 
 `interp-engine` is an interpretability engine that is fast (>40x tok/s vs HF eager), standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.
 
-
-
-
+<p align="center">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ie-perf.png" alt="Tokens per second while capturing and generating: eager against IE-vLLM and IE-vLLM-static on qwen3.8-27b and deepseek-v4-flash-0731, 8 requests in flight" width="100%">
+</p>
+<p align="center">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/interp-engine-demo.gif" alt="interp-engine demo gif" width="100%">
+</p>
 
 This repo contains:
 
-1. `[validator/](validator/)`, which compares/validates it against TransformerLens, and nnsight/nnterp on real architectures.
-2. `[visualizer-web/](visualizer-web/)`, a "cheat sheet" hosted at [interp-engine.org](https://interp-engine.org) of each 'point' (eg `resid_post.16`), standardized across architectures.
+1. [`validator/`](validator/), which compares/validates it against TransformerLens, and nnsight/nnterp on real architectures.
+2. [`visualizer-web/`](visualizer-web/), a "cheat sheet" hosted at [interp-engine.org](https://interp-engine.org) of each 'point' (eg `resid_post.16`), standardized across architectures.
 
 ## Installation
 
@@ -29,13 +40,16 @@ pip install interp-engine          # eager backend only
 ```python
 from interp_engine import Address, load_model, run_with_cache
 
-# VLLM MODE (default): low VRAM, medium speed
-model = load_model("Qwen/Qwen3-8B") 
+# VLLM (default): low VRAM, medium speed, every point, chosen per request
+model = load_model("Qwen/Qwen3-8B")
+
+# VLLM-STATIC: high VRAM, high speed, only the points you declare (default resid_post)
+# model = load_model("Qwen/Qwen3-8B", backend="vllm-static")
 
-# VLLM-FREEZE MODE: high VRAM, high speed, only frozen points (default resid_post)
-# model = load_model("Qwen/Qwen3-8B", freeze_points="auto")
+# VLLM-GENERATE: fastest, generation only -- no capture, no steering
+# model = load_model("Qwen/Qwen3-8B", backend="vllm-generate")
 
-# EAGER MODE: low VRAM, low speed
+# EAGER: low VRAM, low speed
 # model = load_model("Qwen/Qwen3-8B", backend="eager")
 
 point = Address("resid_post", 10)      # or string: "resid_post.10"
@@ -53,7 +67,7 @@ Add "use interp-engine" to your prompt and let your agent figure it out - everyt
 
 ## Performance / Speed
 
-vLLM gives `interp-engine` high throughput via concurrency, and **graph freeze** adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
+vLLM gives `interp-engine` high throughput via concurrency, and `backend="vllm-static"` adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
 
 <!-- THROUGHPUT:START -->
 
@@ -63,34 +77,34 @@ Measured on NVIDIA B200, bf16, 512-token prompt, 128 new tokens.
 
 One stream (tok/s):
 
-| model                    | eager | vLLM       | vLLM + graph freeze |
-| ------------------------ | ----- | ---------- | ------------------- |
-| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**      |
-| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**     |
-| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**      |
-| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**       |
-| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**       |
+| model                    | eager | vLLM       | vLLM + static taps |
+| ------------------------ | ----- | ---------- | ------------------ |
+| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**     |
+| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**    |
+| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**     |
+| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**      |
+| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**      |
 
 8 concurrent requests (aggregate tok/s):
 
-| model                    | eager | vLLM        | vLLM + graph freeze |
-| ------------------------ | ----- | ----------- | ------------------- |
-| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**     |
-| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**     |
-| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**     |
-| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**       |
-| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**      |
+| model                    | eager | vLLM        | vLLM + static taps |
+| ------------------------ | ----- | ----------- | ------------------ |
+| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**    |
+| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**    |
+| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**    |
+| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**      |
+| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**     |
 
 <!-- THROUGHPUT:END -->
 
-**Graph freeze** is opt-in via `freeze_points`, and a frozen engine serves only the set it froze. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; `[benchmarks/results-latest.md](benchmarks/results-latest.md)` has every figure at full precision, including capture, steering and lens latencies; `[benchmarks/README.md](benchmarks/README.md)` has how the tables above are rounded.
+`backend="vllm-static"` is opt-in, and serves only the tap set it declared — `static_points="auto"` by default, or a list you name. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.
 
 ## Correctness
 
 We verify correctness in two main ways:
 
 1. A test suite that checks results across several models - what each check is designed to catch is in [INTERNALS.md](docs/INTERNALS.md#correctness).
-2. A full `validator` comparison engine that checks most hook points across 50+ models, at early, middle and late layers - fully reproducible, with detailed results saved in the git repo at `[validator/](validator/)`.
+2. A full `validator` comparison engine that checks most hook points across 50+ models, at early, middle and late layers - fully reproducible, with detailed results saved in the git repo at [`validator/`](validator/).
 
 ## Why use an Interpretability Engine, instead of just having my AI code whatever it needs on the fly?
 
@@ -114,4 +128,4 @@ Bugs and feature requests belong in [issues](https://github.com/decoderesearch/i
 
 ## License
 
-Apache 2.0
+Apache 2.0

{interp_engine-1.2.8 → interp_engine-1.3.0}/benchmarks/README.md

@@ -92,7 +92,7 @@ full record into `results-latest.md` and then calls `publish`, which rewrites:
 
 | target | what it gets |
 | --- | --- |
-| `README.md`, between the `THROUGHPUT` markers | decode and concurrency-8 tok/s for eager, vLLM and graph freeze, every model |
+| `README.md`, between the `THROUGHPUT` markers | decode and concurrency-8 tok/s for eager, vLLM and static taps, every model |
 | `visualizer-web/data/benchmarks.generated.ts` | the same figures for the card behind the site's **Fast** claim, each row that ran differently carrying a footnote saying how |
 
 Both were transcribed by hand until this existed, and both had drifted -- one carried percentages the
@@ -129,7 +129,7 @@ write at all when the cells disagree about the GPU or the dtype, because those t
 conditions line. And it footnotes a card row when the sweep gave that model anything of its own -- its
 own memory fraction, its own engine arguments, its own capture point -- so that the conditions line
 keeps covering the rows it claims to. `deepseek-v4-flash-0731` earns a footnote for all four reasons.
-The card dropped such a row until the footnote existed, which is why the largest freeze win in the
+The card dropped such a row until the footnote existed, which is why the largest static win in the
 sweep was for a while the one figure the site did not show; a row is still dropped, but only when it is
 missing a baseline figure and so has no multiplier to print. A scratch sweep of ad-hoc models should
 pass `--no-publish` to `report_bench` rather than publish rows nobody deployed.
@@ -217,7 +217,7 @@ The report names these by what they are; `--variant` and the result filenames us
 | interp-engine eager | `eager` | raw HF forward; `attn_implementation="eager"`, which is what the engine sets |
 | interp-engine vllm | `vllm` | vLLM with `enforce_eager=True` — CUDA graphs and inductor compile off |
 | vllm (vanilla) | `vllm-cudagraph` | vLLM left at its own defaults, graphs and compile on |
-| interp-engine vllm freeze | `vllm-freeze` | breakable graphs with `resid_post` freeze wraps at every layer, plus one write tap mid-stack |
+| interp-engine vllm static | `vllm-static` | breakable graphs with `resid_post` static wraps at every layer, plus one write tap mid-stack |
 
 `bench_spec.VARIANTS` also carries two speculative-decoding variants that exist on one checkpoint
 only, and are not part of these tables: `report_bench.EXCLUDED` says why, and `--variant` still
@@ -233,31 +233,31 @@ capture actually returns under replay instead of asserting the outcome. A captur
 with no points, or with fewer rows than the prompt had tokens, is recorded as `unsupported` with the
 shape it got, and the report renders that cell as `n/a`.
 
-### `vllm-freeze`, and what its `steer` cell needs
+### `vllm-static`, and what its `steer` cell needs
 
-The fourth is the answer to the third: freeze copies activations in and out of buffers the graph
+The fourth is the answer to the third: static copies activations in and out of buffers the graph
 already refers to, so a replay serves capture and steering without a Python forward. It is the path
-`freeze_points="auto"` takes in production, and the row exists to price it against the
+`static_points="auto"` takes in production, and the row exists to price it against the
 `enforce_eager=True` column capture would otherwise have to use.
 
 `"auto"` once installed **reads** only, and a steering op needs a write tap to land in, so this row
 priced half the feature and reported the other half as `n/a` -- with a message that blamed graph
-replay for it, which is the thing freeze exists to work around. Auto now covers both halves, so the
-cell would be a number either way, and this row still passes `freeze_writes` on purpose: an explicit
+replay for it, which is the thing static exists to work around. Auto now covers both halves, so the
+cell would be a number either way, and this row still passes `static_writes` on purpose: an explicit
 list *narrows* what auto would install, to the one mid-stack site the `steer` workload actually
 writes. A row that priced a write buffer at every layer would not be comparable with the ones beside
 it, which is the whole job of the column. Its value is the sentinel `run_bench.STEER_WRITES` rather
-than a site, because that layer differs per model and a freeze write is a `load_model` argument, so
+than a site, because that layer differs per model and a static write is a `load_model` argument, so
 it has to be resolved from the config before a model exists to ask.
 
-`VariantSpec.models` restricts the row to the checkpoints freeze has been shown correct on, so a model
+`VariantSpec.models` restricts the row to the checkpoints static has been shown correct on, so a model
 missing from it renders `--` rather than a number nobody checked.
 
-One of them measures a wider point than its neighbours. `"auto"` on a hyper-connection trunk freezes
-`resid_streams` — the whole stack of four parallel streams per layer — and a frozen engine serves the
-set it froze, so `deepseek-v4-flash-0731` cannot be asked for the `mlp_out` its other columns capture.
-That row's freeze cell therefore prices the stack where every other cell prices one row, declared in
-`ModelSpec.freeze_capture_point`, stated by the report under *Where a row differs*, and carried onto
+One of them measures a wider point than its neighbours. `"auto"` on a hyper-connection trunk declares
+`resid_streams` — the whole stack of four parallel streams per layer — and a static engine serves the
+set it declared, so `deepseek-v4-flash-0731` cannot be asked for the `mlp_out` its other columns capture.
+That row's static cell therefore prices the stack where every other cell prices one row, declared in
+`ModelSpec.static_capture_point`, stated by the report under *Where a row differs*, and carried onto
 the visualizer's card as one line of that row's footnote.
 
 ## Reading the numbers honestly

{interp_engine-1.2.8 → interp_engine-1.3.0}/benchmarks/bench_spec.py

@@ -50,21 +50,21 @@ class ModelSpec:
     sweep's shared budget, which is all but the largest.
 
     Separate from :attr:`extra_vllm_kwargs` because that is a fact about the weights and applies to
-    every vLLM column, so putting a freeze-only budget there would change three cells that have
+    every vLLM column, so putting a static-only budget there would change three cells that have
     already been measured and are comparable as they stand. The overrides here are recorded in the cell
     and stated by the report, so the one cell that ran on a different budget says so."""
-    freeze_capture_point: str | None = None
-    """The point those workloads address instead when the variant under test froze a set, for a
+    static_capture_point: str | None = None
+    """The point those workloads address instead when the variant under test declared a set, for a
     checkpoint where the two cannot be the same point. ``None`` means :attr:`capture_point` serves both.
 
-    A frozen engine serves *the set it froze* and refuses anything outside it, so the workload has to
-    name a point the freeze set covers. On a hyper-connection trunk it cannot be the same one:
-    ``freeze_points="auto"`` resolves to ``resid_streams`` -- the whole stack -- while the other columns
+    A static engine serves *the set it declared* and refuses anything outside it, so the workload has to
+    name a point the static set covers. On a hyper-connection trunk it cannot be the same one:
+    ``static_points="auto"`` resolves to ``resid_streams`` -- the whole stack -- while the other columns
     need one ``d_model`` row to stay comparable with the rest of the table. Declared here rather than
     inferred at run time so the cell records it, ``report_bench`` states it under *Where a row differs*,
     and ``cells.nonuniform`` carries it onto the visualizer's card as one line of that row's footnote.
 
-    Only the freeze cell records it, so read it through ``cells.row_spec`` rather than off a cell.
+    Only the static cell records it, so read it through ``cells.row_spec`` rather than off a cell.
     """
     extra_eager_kwargs: dict[str, object] = field(default_factory=dict)
     """``load_model`` arguments the eager variants need for this checkpoint, merged under the
@@ -138,15 +138,15 @@ MODELS: tuple[ModelSpec, ...] = (
         # bytes, which would price this row's transport as though DeepSeek moved four times the
         # activations to answer the same question.
         capture_point="mlp_out",
-        # ...except under freeze, which serves the set it froze: `"auto"` on this trunk freezes
+        # ...except under static, which serves the set it declared: `"auto"` on this trunk declares
         # `resid_streams`, so `mlp_out` is outside the set and would be refused. That column therefore
         # prices the stack rather than a row, which is declared as a row exception instead of hidden.
-        freeze_capture_point="resid_streams",
-        # The freeze column is the one configuration here that does not fit the shared budget, and both
+        static_capture_point="resid_streams",
+        # The static column is the one configuration here that does not fit the shared budget, and both
         # of these are why. 149 GiB of weights at 0.95 of a 178 GiB card leave about 15 GiB for the
-        # activation peak, the freeze buffers, the KV pool and the graph pool together.
+        # activation peak, the static buffers, the KV pool and the graph pool together.
         #
-        # A freeze buffer is allocated per batched row, so at the 4096 `max_num_batched_tokens` freeze
+        # A static buffer is allocated per batched row, so at the 4096 `max_num_batched_tokens` static
         # lowered vLLM's default to, `resid_streams` costs 4096 rows x 4 streams x 4096 wide x 2 bytes
         # x 43 layers = 5.8 GiB -- and vLLM then sized the KV pool from what was left and refused to
         # start: "No available memory for the cache blocks". 1024 rows costs a quarter of that and is
@@ -157,7 +157,7 @@ MODELS: tuple[ModelSpec, ...] = (
         # sizes it will actually replay keeps the rest of that memory, and the startup time, unspent.
         # Neither changes what the workloads measure: both sizes they use are still captured.
         per_variant_vllm_kwargs={
-            "vllm-freeze": {
+            "vllm-static": {
                 "max_num_batched_tokens": 1024,
                 "compilation_config": {"cudagraph_capture_sizes": [1, 2, 4, 8, 16, 32]},
             }
@@ -215,12 +215,17 @@ class VariantSpec:
 
 
 #: One of these three exists to price a default that the engine chose for capture's sake, which is the
-#: most useful thing a speed benchmark of this library can say. vLLM capture needs
-#: ``enforce_eager=True``, because CUDA-graph replay does not re-execute the Python forward and so
-#: never fires a ``register_forward_hook``. That is why ``VLLMModel`` defaults it to True, and why
-#: ``vllm-cudagraph`` -- vLLM left at its own defaults, hence the "vanilla" label -- is here at all:
-#: it measures what ours costs. The capture workloads run there too rather than being skipped, so the
-#: report can show *what* a capture returns under replay instead of asserting that it fails.
+#: most useful thing a speed benchmark of this library can say. Hooked vLLM capture rules CUDA graphs
+#: out, because graph replay does not re-execute the Python forward and so never fires a
+#: ``register_forward_hook``. That is what separates the three vLLM backends, and it is why
+#: ``vllm-cudagraph`` -- ``backend="vllm-generate"``, vLLM left at its own defaults, hence the
+#: "vanilla" label -- is here at all: it measures what ours costs. The capture workloads run there too
+#: rather than being skipped, so the report can show *what* a capture returns under replay instead of
+#: asserting that it fails.
+#:
+#: Each variant names its backend rather than deriving it from the kwargs it passes, which is what
+#: the engine now expects: ``vllm-static`` and ``vllm-generate`` refuse ``enforce_eager``, and a tap
+#: set is only accepted by the backend built to bake one in.
 VARIANTS: tuple[VariantSpec, ...] = (
     VariantSpec(
         "eager",
@@ -232,52 +237,52 @@ VARIANTS: tuple[VariantSpec, ...] = (
     VariantSpec(
         "vllm",
         "vllm",
-        {"enforce_eager": True},
+        {},
         "capture-capable: CUDA graphs and compile OFF",
         "interp-engine vllm",
     ),
     VariantSpec(
         "vllm-cudagraph",
-        "vllm",
-        {"enforce_eager": False, "freeze_points": []},
-        "CUDA graphs + inductor compile ON, no freeze wraps; vLLM's own defaults for generate-only",
+        "vllm-generate",
+        {},
+        "CUDA graphs + inductor compile ON, no static wraps; vLLM's own defaults for generate-only",
         "vllm (vanilla)",
     ),
-    # Breakable CUDA graphs with resid_post freeze wraps at every layer. Production freeze
-    # (``freeze_points="auto"``) is this path, not Dynamo piecewise.
+    # Breakable CUDA graphs with resid_post static wraps at every layer. Production static
+    # (``static_points="auto"``) is this path, not Dynamo piecewise.
     #
-    # `qwen3.8-27b` joined the row once freeze was correct on a hybrid trunk. Breakable graphs turn
+    # `qwen3.8-27b` joined the row once static was correct on a hybrid trunk. Breakable graphs turn
     # inductor off, and vLLM's `FULL_AND_PIECEWISE` capture then miscomputes prefill for a gated-delta
     # trunk -- the engine generated fluent nonsense rather than failing, so the cell would have been a
-    # plausible number for a broken forward. Freeze now pins `FULL_DECODE_ONLY` on such a trunk, which
-    # runs prefill eagerly and keeps the decode graphs, and the validator's freeze column agrees with
+    # plausible number for a broken forward. Static now pins `FULL_DECODE_ONLY` on such a trunk, which
+    # runs prefill eagerly and keeps the decode graphs, and the validator's static column agrees with
     # eager on all 28 points. Its prefill figures carry that eager prefill, which is the point of
     # comparing it against the same model's other variants rather than against another model.
     #
-    # `deepseek-v4-flash-0731` is in, and is the one row here whose freeze set is not one point per
+    # `deepseek-v4-flash-0731` is in, and is the one row here whose static set is not one point per
     # layer. It is a hyper-connection trunk, so `"auto"` resolves to `resid_streams` -- the whole stack
     # of four parallel residual streams per layer, four times the width of a `resid_post` row. Its
     # capture and transport figures therefore price four times the activations for the same question,
     # which `cells.nonuniform` declares so the report states it and the visualizer's card drops the row
-    # rather than publishing it beside three that froze a quarter as much.
+    # rather than publishing it beside three that declared a quarter as much.
     #
     # Its write tap is `mlp_out`, not `resid_post`: the steer workload addresses the model's
     # `capture_point`, and a hyper-connection trunk refuses the default name (`run_bench._steer_site`).
     #
-    # `freeze_writes` was once what made the `steer` cell a number instead of `n/a`: `"auto"` installed
+    # `static_writes` was once what made the `steer` cell a number instead of `n/a`: `"auto"` installed
     # reads, and a steering op needs a write tap to land in, so this row priced capture under replay
     # and left the other half of the feature unmeasured -- with a message that blamed graph replay for
     # it. Auto covers writes now, so the cell stands either way, and naming them here has become a
     # *narrowing*: one write buffer at the site the workload steers rather than one per layer, which
     # is what keeps this row's memory comparable with the columns beside it. The value is the sentinel
     # `run_bench.STEER_WRITES`, resolved there to the mid-stack `resid_post` the workload steers,
-    # because the layer differs per model and a freeze write has to be named before the model exists.
+    # because the layer differs per model and a static write has to be named before the model exists.
     VariantSpec(
-        "vllm-freeze",
-        "vllm",
-        {"freeze_points": "auto", "freeze_writes": "steer"},
-        "breakable CUDA graphs with resid_post freeze wraps at every layer, and a write tap mid-stack",
-        "interp-engine vllm freeze",
+        "vllm-static",
+        "vllm-static",
+        {"static_points": "auto", "static_writes": "steer"},
+        "breakable CUDA graphs with resid_post static wraps at every layer, and a write tap mid-stack",
+        "interp-engine vllm static",
         models=("gemma-2-2b", "qwen3-4b", "llama-3.1-8b", "qwen3.8-27b", "deepseek-v4-flash-0731"),
     ),
     # DSpark on, against the `vllm` column with it off -- the pair is the measurement, so read the two
@@ -303,12 +308,11 @@ VARIANTS: tuple[VariantSpec, ...] = (
         "vllm-dspark",
         "vllm",
         {
-            "enforce_eager": True,
             "extra_vllm_kwargs": {
                 "speculative_config": {"method": "dspark", "num_speculative_tokens": 5},
             },
         },
-        "DSpark speculative decoding ON; enforce_eager, so still capture-capable",
+        "DSpark speculative decoding ON; hooked backend, so still capture-capable",
         "interp-engine vllm +DSpark",
         models=("deepseek-v4-flash-0731",),
     ),
@@ -325,10 +329,8 @@ VARIANTS: tuple[VariantSpec, ...] = (
     # never calls the Python forward a hook is attached to.
     VariantSpec(
         "vllm-dspark-cudagraph",
-        "vllm",
+        "vllm-generate",
         {
-            "enforce_eager": False,
-            "freeze_points": [],
             "extra_vllm_kwargs": {
                 "speculative_config": {"method": "dspark", "num_speculative_tokens": 5},
             },

{interp_engine-1.2.8 → interp_engine-1.3.0}/benchmarks/cells.py

@@ -77,9 +77,9 @@ def row_spec(cells: list[dict[str, Any]], model_key: str, variants: Iterable[str
     """Every override one model's row declared, merged across the row's cells.
 
     A per-variant override is recorded by the cell that used it and by no other: on
-    `deepseek-v4-flash-0731` both `freeze_capture_point` and `per_variant_vllm_kwargs` live on the
-    freeze cell alone. So any single cell's ``model`` describes a column rather than a row, and picking
-    one -- whichever happened to sort last -- silently dropped the two overrides the freeze column is
+    `deepseek-v4-flash-0731` both `static_capture_point` and `per_variant_vllm_kwargs` live on the
+    static cell alone. So any single cell's ``model`` describes a column rather than a row, and picking
+    one -- whichever happened to sort last -- silently dropped the two overrides the static column is
     the only cell to declare.
 
     ``variants`` restricts the merge to the columns a renderer shows, and each of them passes its own:
@@ -129,9 +129,9 @@ def nonuniform(model: dict[str, Any]) -> list[str]:
     point = model.get("capture_point", DEFAULT_POINT)
     if point != DEFAULT_POINT:
         reasons.append(f"captures `{point}` rather than `{DEFAULT_POINT}`")
-    frozen_point = model.get("freeze_capture_point")
-    if frozen_point:
-        reasons.append(f"its freeze column captures `{frozen_point}`, a wider point than the others do")
+    static_point = model.get("static_capture_point")
+    if static_point:
+        reasons.append(f"its static column captures `{static_point}`, a wider point than the others do")
     fraction = model.get("gpu_memory_utilization")
     if fraction:
         reasons.append(f"vLLM reserved {fraction} of the card, not the uniform {GPU_MEMORY_UTILIZATION}")