interp-engine 1.2.9__tar.gz → 1.3.0__tar.gz

{interp_engine-1.2.9 → interp_engine-1.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: interp-engine
-Version: 1.2.9
+Version: 1.3.0
 Summary: A fast, standardized interpretability engine that supports most modern models and architectures. Powers Neuronpedia.
 Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
 Project-URL: Repository, https://github.com/decoderesearch/interp-engine
@@ -48,7 +48,7 @@ Description-Content-Type: text/markdown
 `interp-engine` is an interpretability engine that is fast (>40x tok/s vs HF eager), standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.
 
 <p align="center">
-  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ie-perf.png" alt="Tokens per second while capturing and generating: eager against IE-vLLM and IE-vLLM-freeze on qwen3.8-27b and deepseek-v4-flash-0731, 8 requests in flight" width="100%">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ie-perf.png" alt="Tokens per second while capturing and generating: eager against IE-vLLM and IE-vLLM-static on qwen3.8-27b and deepseek-v4-flash-0731, 8 requests in flight" width="100%">
 </p>
 <p align="center">
   <img src="https://neuronpedia.s3.amazonaws.com/site-assets/interp-engine-demo.gif" alt="interp-engine demo gif" width="100%">
@@ -71,13 +71,16 @@ pip install interp-engine          # eager backend only
 ```python
 from interp_engine import Address, load_model, run_with_cache
 
-# VLLM MODE (default): low VRAM, medium speed
-model = load_model("Qwen/Qwen3-8B") 
+# VLLM (default): low VRAM, medium speed, every point, chosen per request
+model = load_model("Qwen/Qwen3-8B")
 
-# VLLM-FREEZE MODE: high VRAM, high speed, only frozen points (default resid_post)
-# model = load_model("Qwen/Qwen3-8B", freeze_points="auto")
+# VLLM-STATIC: high VRAM, high speed, only the points you declare (default resid_post)
+# model = load_model("Qwen/Qwen3-8B", backend="vllm-static")
 
-# EAGER MODE: low VRAM, low speed
+# VLLM-GENERATE: fastest, generation only -- no capture, no steering
+# model = load_model("Qwen/Qwen3-8B", backend="vllm-generate")
+
+# EAGER: low VRAM, low speed
 # model = load_model("Qwen/Qwen3-8B", backend="eager")
 
 point = Address("resid_post", 10)      # or string: "resid_post.10"
@@ -95,7 +98,7 @@ Add "use interp-engine" to your prompt and let your agent figure it out - everyt
 
 ## Performance / Speed
 
-vLLM gives `interp-engine` high throughput via concurrency, and **graph freeze** adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
+vLLM gives `interp-engine` high throughput via concurrency, and `backend="vllm-static"` adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
 
 <!-- THROUGHPUT:START -->
 
@@ -105,27 +108,27 @@ Measured on NVIDIA B200, bf16, 512-token prompt, 128 new tokens.
 
 One stream (tok/s):
 
-| model                    | eager | vLLM       | vLLM + graph freeze |
-| ------------------------ | ----- | ---------- | ------------------- |
-| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**      |
-| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**     |
-| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**      |
-| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**       |
-| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**       |
+| model                    | eager | vLLM       | vLLM + static taps |
+| ------------------------ | ----- | ---------- | ------------------ |
+| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**     |
+| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**    |
+| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**     |
+| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**      |
+| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**      |
 
 8 concurrent requests (aggregate tok/s):
 
-| model                    | eager | vLLM        | vLLM + graph freeze |
-| ------------------------ | ----- | ----------- | ------------------- |
-| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**     |
-| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**     |
-| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**     |
-| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**       |
-| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**      |
+| model                    | eager | vLLM        | vLLM + static taps |
+| ------------------------ | ----- | ----------- | ------------------ |
+| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**    |
+| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**    |
+| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**    |
+| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**      |
+| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**     |
 
 <!-- THROUGHPUT:END -->
 
-**Graph freeze** is opt-in via `freeze_points`, and a frozen engine serves only the set it froze. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.
+`backend="vllm-static"` is opt-in, and serves only the tap set it declared — `static_points="auto"` by default, or a list you name. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.
 
 ## Correctness
 

{interp_engine-1.2.9 → interp_engine-1.3.0}/README.md

@@ -17,7 +17,7 @@
 `interp-engine` is an interpretability engine that is fast (>40x tok/s vs HF eager), standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.
 
 <p align="center">
-  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ie-perf.png" alt="Tokens per second while capturing and generating: eager against IE-vLLM and IE-vLLM-freeze on qwen3.8-27b and deepseek-v4-flash-0731, 8 requests in flight" width="100%">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/ie-perf.png" alt="Tokens per second while capturing and generating: eager against IE-vLLM and IE-vLLM-static on qwen3.8-27b and deepseek-v4-flash-0731, 8 requests in flight" width="100%">
 </p>
 <p align="center">
   <img src="https://neuronpedia.s3.amazonaws.com/site-assets/interp-engine-demo.gif" alt="interp-engine demo gif" width="100%">
@@ -40,13 +40,16 @@ pip install interp-engine          # eager backend only
 ```python
 from interp_engine import Address, load_model, run_with_cache
 
-# VLLM MODE (default): low VRAM, medium speed
-model = load_model("Qwen/Qwen3-8B") 
+# VLLM (default): low VRAM, medium speed, every point, chosen per request
+model = load_model("Qwen/Qwen3-8B")
 
-# VLLM-FREEZE MODE: high VRAM, high speed, only frozen points (default resid_post)
-# model = load_model("Qwen/Qwen3-8B", freeze_points="auto")
+# VLLM-STATIC: high VRAM, high speed, only the points you declare (default resid_post)
+# model = load_model("Qwen/Qwen3-8B", backend="vllm-static")
 
-# EAGER MODE: low VRAM, low speed
+# VLLM-GENERATE: fastest, generation only -- no capture, no steering
+# model = load_model("Qwen/Qwen3-8B", backend="vllm-generate")
+
+# EAGER: low VRAM, low speed
 # model = load_model("Qwen/Qwen3-8B", backend="eager")
 
 point = Address("resid_post", 10)      # or string: "resid_post.10"
@@ -64,7 +67,7 @@ Add "use interp-engine" to your prompt and let your agent figure it out - everyt
 
 ## Performance / Speed
 
-vLLM gives `interp-engine` high throughput via concurrency, and **graph freeze** adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
+vLLM gives `interp-engine` high throughput via concurrency, and `backend="vllm-static"` adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
 
 <!-- THROUGHPUT:START -->
 
@@ -74,27 +77,27 @@ Measured on NVIDIA B200, bf16, 512-token prompt, 128 new tokens.
 
 One stream (tok/s):
 
-| model                    | eager | vLLM       | vLLM + graph freeze |
-| ------------------------ | ----- | ---------- | ------------------- |
-| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**      |
-| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**     |
-| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**      |
-| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**       |
-| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**       |
+| model                    | eager | vLLM       | vLLM + static taps |
+| ------------------------ | ----- | ---------- | ------------------ |
+| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**     |
+| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**    |
+| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**     |
+| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**      |
+| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**      |
 
 8 concurrent requests (aggregate tok/s):
 
-| model                    | eager | vLLM        | vLLM + graph freeze |
-| ------------------------ | ----- | ----------- | ------------------- |
-| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**     |
-| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**     |
-| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**     |
-| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**       |
-| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**      |
+| model                    | eager | vLLM        | vLLM + static taps |
+| ------------------------ | ----- | ----------- | ------------------ |
+| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**    |
+| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**    |
+| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**    |
+| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**      |
+| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**     |
 
 <!-- THROUGHPUT:END -->
 
-**Graph freeze** is opt-in via `freeze_points`, and a frozen engine serves only the set it froze. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.
+`backend="vllm-static"` is opt-in, and serves only the tap set it declared — `static_points="auto"` by default, or a list you name. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.
 
 ## Correctness
 

{interp_engine-1.2.9 → interp_engine-1.3.0}/benchmarks/README.md

@@ -92,7 +92,7 @@ full record into `results-latest.md` and then calls `publish`, which rewrites:
 
 | target | what it gets |
 | --- | --- |
-| `README.md`, between the `THROUGHPUT` markers | decode and concurrency-8 tok/s for eager, vLLM and graph freeze, every model |
+| `README.md`, between the `THROUGHPUT` markers | decode and concurrency-8 tok/s for eager, vLLM and static taps, every model |
 | `visualizer-web/data/benchmarks.generated.ts` | the same figures for the card behind the site's **Fast** claim, each row that ran differently carrying a footnote saying how |
 
 Both were transcribed by hand until this existed, and both had drifted -- one carried percentages the
@@ -129,7 +129,7 @@ write at all when the cells disagree about the GPU or the dtype, because those t
 conditions line. And it footnotes a card row when the sweep gave that model anything of its own -- its
 own memory fraction, its own engine arguments, its own capture point -- so that the conditions line
 keeps covering the rows it claims to. `deepseek-v4-flash-0731` earns a footnote for all four reasons.
-The card dropped such a row until the footnote existed, which is why the largest freeze win in the
+The card dropped such a row until the footnote existed, which is why the largest static win in the
 sweep was for a while the one figure the site did not show; a row is still dropped, but only when it is
 missing a baseline figure and so has no multiplier to print. A scratch sweep of ad-hoc models should
 pass `--no-publish` to `report_bench` rather than publish rows nobody deployed.
@@ -217,7 +217,7 @@ The report names these by what they are; `--variant` and the result filenames us
 | interp-engine eager | `eager` | raw HF forward; `attn_implementation="eager"`, which is what the engine sets |
 | interp-engine vllm | `vllm` | vLLM with `enforce_eager=True` — CUDA graphs and inductor compile off |
 | vllm (vanilla) | `vllm-cudagraph` | vLLM left at its own defaults, graphs and compile on |
-| interp-engine vllm freeze | `vllm-freeze` | breakable graphs with `resid_post` freeze wraps at every layer, plus one write tap mid-stack |
+| interp-engine vllm static | `vllm-static` | breakable graphs with `resid_post` static wraps at every layer, plus one write tap mid-stack |
 
 `bench_spec.VARIANTS` also carries two speculative-decoding variants that exist on one checkpoint
 only, and are not part of these tables: `report_bench.EXCLUDED` says why, and `--variant` still
@@ -233,31 +233,31 @@ capture actually returns under replay instead of asserting the outcome. A captur
 with no points, or with fewer rows than the prompt had tokens, is recorded as `unsupported` with the
 shape it got, and the report renders that cell as `n/a`.
 
-### `vllm-freeze`, and what its `steer` cell needs
+### `vllm-static`, and what its `steer` cell needs
 
-The fourth is the answer to the third: freeze copies activations in and out of buffers the graph
+The fourth is the answer to the third: static copies activations in and out of buffers the graph
 already refers to, so a replay serves capture and steering without a Python forward. It is the path
-`freeze_points="auto"` takes in production, and the row exists to price it against the
+`static_points="auto"` takes in production, and the row exists to price it against the
 `enforce_eager=True` column capture would otherwise have to use.
 
 `"auto"` once installed **reads** only, and a steering op needs a write tap to land in, so this row
 priced half the feature and reported the other half as `n/a` -- with a message that blamed graph
-replay for it, which is the thing freeze exists to work around. Auto now covers both halves, so the
-cell would be a number either way, and this row still passes `freeze_writes` on purpose: an explicit
+replay for it, which is the thing static exists to work around. Auto now covers both halves, so the
+cell would be a number either way, and this row still passes `static_writes` on purpose: an explicit
 list *narrows* what auto would install, to the one mid-stack site the `steer` workload actually
 writes. A row that priced a write buffer at every layer would not be comparable with the ones beside
 it, which is the whole job of the column. Its value is the sentinel `run_bench.STEER_WRITES` rather
-than a site, because that layer differs per model and a freeze write is a `load_model` argument, so
+than a site, because that layer differs per model and a static write is a `load_model` argument, so
 it has to be resolved from the config before a model exists to ask.
 
-`VariantSpec.models` restricts the row to the checkpoints freeze has been shown correct on, so a model
+`VariantSpec.models` restricts the row to the checkpoints static has been shown correct on, so a model
 missing from it renders `--` rather than a number nobody checked.
 
-One of them measures a wider point than its neighbours. `"auto"` on a hyper-connection trunk freezes
-`resid_streams` — the whole stack of four parallel streams per layer — and a frozen engine serves the
-set it froze, so `deepseek-v4-flash-0731` cannot be asked for the `mlp_out` its other columns capture.
-That row's freeze cell therefore prices the stack where every other cell prices one row, declared in
-`ModelSpec.freeze_capture_point`, stated by the report under *Where a row differs*, and carried onto
+One of them measures a wider point than its neighbours. `"auto"` on a hyper-connection trunk declares
+`resid_streams` — the whole stack of four parallel streams per layer — and a static engine serves the
+set it declared, so `deepseek-v4-flash-0731` cannot be asked for the `mlp_out` its other columns capture.
+That row's static cell therefore prices the stack where every other cell prices one row, declared in
+`ModelSpec.static_capture_point`, stated by the report under *Where a row differs*, and carried onto
 the visualizer's card as one line of that row's footnote.
 
 ## Reading the numbers honestly

{interp_engine-1.2.9 → interp_engine-1.3.0}/benchmarks/bench_spec.py

@@ -50,21 +50,21 @@ class ModelSpec:
     sweep's shared budget, which is all but the largest.
 
     Separate from :attr:`extra_vllm_kwargs` because that is a fact about the weights and applies to
-    every vLLM column, so putting a freeze-only budget there would change three cells that have
+    every vLLM column, so putting a static-only budget there would change three cells that have
     already been measured and are comparable as they stand. The overrides here are recorded in the cell
     and stated by the report, so the one cell that ran on a different budget says so."""
-    freeze_capture_point: str | None = None
-    """The point those workloads address instead when the variant under test froze a set, for a
+    static_capture_point: str | None = None
+    """The point those workloads address instead when the variant under test declared a set, for a
     checkpoint where the two cannot be the same point. ``None`` means :attr:`capture_point` serves both.
 
-    A frozen engine serves *the set it froze* and refuses anything outside it, so the workload has to
-    name a point the freeze set covers. On a hyper-connection trunk it cannot be the same one:
-    ``freeze_points="auto"`` resolves to ``resid_streams`` -- the whole stack -- while the other columns
+    A static engine serves *the set it declared* and refuses anything outside it, so the workload has to
+    name a point the static set covers. On a hyper-connection trunk it cannot be the same one:
+    ``static_points="auto"`` resolves to ``resid_streams`` -- the whole stack -- while the other columns
     need one ``d_model`` row to stay comparable with the rest of the table. Declared here rather than
     inferred at run time so the cell records it, ``report_bench`` states it under *Where a row differs*,
     and ``cells.nonuniform`` carries it onto the visualizer's card as one line of that row's footnote.
 
-    Only the freeze cell records it, so read it through ``cells.row_spec`` rather than off a cell.
+    Only the static cell records it, so read it through ``cells.row_spec`` rather than off a cell.
     """
     extra_eager_kwargs: dict[str, object] = field(default_factory=dict)
     """``load_model`` arguments the eager variants need for this checkpoint, merged under the
@@ -138,15 +138,15 @@ MODELS: tuple[ModelSpec, ...] = (
         # bytes, which would price this row's transport as though DeepSeek moved four times the
         # activations to answer the same question.
         capture_point="mlp_out",
-        # ...except under freeze, which serves the set it froze: `"auto"` on this trunk freezes
+        # ...except under static, which serves the set it declared: `"auto"` on this trunk declares
         # `resid_streams`, so `mlp_out` is outside the set and would be refused. That column therefore
         # prices the stack rather than a row, which is declared as a row exception instead of hidden.
-        freeze_capture_point="resid_streams",
-        # The freeze column is the one configuration here that does not fit the shared budget, and both
+        static_capture_point="resid_streams",
+        # The static column is the one configuration here that does not fit the shared budget, and both
         # of these are why. 149 GiB of weights at 0.95 of a 178 GiB card leave about 15 GiB for the
-        # activation peak, the freeze buffers, the KV pool and the graph pool together.
+        # activation peak, the static buffers, the KV pool and the graph pool together.
         #
-        # A freeze buffer is allocated per batched row, so at the 4096 `max_num_batched_tokens` freeze
+        # A static buffer is allocated per batched row, so at the 4096 `max_num_batched_tokens` static
         # lowered vLLM's default to, `resid_streams` costs 4096 rows x 4 streams x 4096 wide x 2 bytes
         # x 43 layers = 5.8 GiB -- and vLLM then sized the KV pool from what was left and refused to
         # start: "No available memory for the cache blocks". 1024 rows costs a quarter of that and is
@@ -157,7 +157,7 @@ MODELS: tuple[ModelSpec, ...] = (
         # sizes it will actually replay keeps the rest of that memory, and the startup time, unspent.
         # Neither changes what the workloads measure: both sizes they use are still captured.
         per_variant_vllm_kwargs={
-            "vllm-freeze": {
+            "vllm-static": {
                 "max_num_batched_tokens": 1024,
                 "compilation_config": {"cudagraph_capture_sizes": [1, 2, 4, 8, 16, 32]},
             }
@@ -215,12 +215,17 @@ class VariantSpec:
 
 
 #: One of these three exists to price a default that the engine chose for capture's sake, which is the
-#: most useful thing a speed benchmark of this library can say. vLLM capture needs
-#: ``enforce_eager=True``, because CUDA-graph replay does not re-execute the Python forward and so
-#: never fires a ``register_forward_hook``. That is why ``VLLMModel`` defaults it to True, and why
-#: ``vllm-cudagraph`` -- vLLM left at its own defaults, hence the "vanilla" label -- is here at all:
-#: it measures what ours costs. The capture workloads run there too rather than being skipped, so the
-#: report can show *what* a capture returns under replay instead of asserting that it fails.
+#: most useful thing a speed benchmark of this library can say. Hooked vLLM capture rules CUDA graphs
+#: out, because graph replay does not re-execute the Python forward and so never fires a
+#: ``register_forward_hook``. That is what separates the three vLLM backends, and it is why
+#: ``vllm-cudagraph`` -- ``backend="vllm-generate"``, vLLM left at its own defaults, hence the
+#: "vanilla" label -- is here at all: it measures what ours costs. The capture workloads run there too
+#: rather than being skipped, so the report can show *what* a capture returns under replay instead of
+#: asserting that it fails.
+#:
+#: Each variant names its backend rather than deriving it from the kwargs it passes, which is what
+#: the engine now expects: ``vllm-static`` and ``vllm-generate`` refuse ``enforce_eager``, and a tap
+#: set is only accepted by the backend built to bake one in.
 VARIANTS: tuple[VariantSpec, ...] = (
     VariantSpec(
         "eager",
@@ -232,52 +237,52 @@ VARIANTS: tuple[VariantSpec, ...] = (
     VariantSpec(
         "vllm",
         "vllm",
-        {"enforce_eager": True},
+        {},
         "capture-capable: CUDA graphs and compile OFF",
         "interp-engine vllm",
     ),
     VariantSpec(
         "vllm-cudagraph",
-        "vllm",
-        {"enforce_eager": False, "freeze_points": []},
-        "CUDA graphs + inductor compile ON, no freeze wraps; vLLM's own defaults for generate-only",
+        "vllm-generate",
+        {},
+        "CUDA graphs + inductor compile ON, no static wraps; vLLM's own defaults for generate-only",
         "vllm (vanilla)",
     ),
-    # Breakable CUDA graphs with resid_post freeze wraps at every layer. Production freeze
-    # (``freeze_points="auto"``) is this path, not Dynamo piecewise.
+    # Breakable CUDA graphs with resid_post static wraps at every layer. Production static
+    # (``static_points="auto"``) is this path, not Dynamo piecewise.
     #
-    # `qwen3.8-27b` joined the row once freeze was correct on a hybrid trunk. Breakable graphs turn
+    # `qwen3.8-27b` joined the row once static was correct on a hybrid trunk. Breakable graphs turn
     # inductor off, and vLLM's `FULL_AND_PIECEWISE` capture then miscomputes prefill for a gated-delta
     # trunk -- the engine generated fluent nonsense rather than failing, so the cell would have been a
-    # plausible number for a broken forward. Freeze now pins `FULL_DECODE_ONLY` on such a trunk, which
-    # runs prefill eagerly and keeps the decode graphs, and the validator's freeze column agrees with
+    # plausible number for a broken forward. Static now pins `FULL_DECODE_ONLY` on such a trunk, which
+    # runs prefill eagerly and keeps the decode graphs, and the validator's static column agrees with
     # eager on all 28 points. Its prefill figures carry that eager prefill, which is the point of
     # comparing it against the same model's other variants rather than against another model.
     #
-    # `deepseek-v4-flash-0731` is in, and is the one row here whose freeze set is not one point per
+    # `deepseek-v4-flash-0731` is in, and is the one row here whose static set is not one point per
     # layer. It is a hyper-connection trunk, so `"auto"` resolves to `resid_streams` -- the whole stack
     # of four parallel residual streams per layer, four times the width of a `resid_post` row. Its
     # capture and transport figures therefore price four times the activations for the same question,
     # which `cells.nonuniform` declares so the report states it and the visualizer's card drops the row
-    # rather than publishing it beside three that froze a quarter as much.
+    # rather than publishing it beside three that declared a quarter as much.
     #
     # Its write tap is `mlp_out`, not `resid_post`: the steer workload addresses the model's
     # `capture_point`, and a hyper-connection trunk refuses the default name (`run_bench._steer_site`).
     #
-    # `freeze_writes` was once what made the `steer` cell a number instead of `n/a`: `"auto"` installed
+    # `static_writes` was once what made the `steer` cell a number instead of `n/a`: `"auto"` installed
     # reads, and a steering op needs a write tap to land in, so this row priced capture under replay
     # and left the other half of the feature unmeasured -- with a message that blamed graph replay for
     # it. Auto covers writes now, so the cell stands either way, and naming them here has become a
     # *narrowing*: one write buffer at the site the workload steers rather than one per layer, which
     # is what keeps this row's memory comparable with the columns beside it. The value is the sentinel
     # `run_bench.STEER_WRITES`, resolved there to the mid-stack `resid_post` the workload steers,
-    # because the layer differs per model and a freeze write has to be named before the model exists.
+    # because the layer differs per model and a static write has to be named before the model exists.
     VariantSpec(
-        "vllm-freeze",
-        "vllm",
-        {"freeze_points": "auto", "freeze_writes": "steer"},
-        "breakable CUDA graphs with resid_post freeze wraps at every layer, and a write tap mid-stack",
-        "interp-engine vllm freeze",
+        "vllm-static",
+        "vllm-static",
+        {"static_points": "auto", "static_writes": "steer"},
+        "breakable CUDA graphs with resid_post static wraps at every layer, and a write tap mid-stack",
+        "interp-engine vllm static",
         models=("gemma-2-2b", "qwen3-4b", "llama-3.1-8b", "qwen3.8-27b", "deepseek-v4-flash-0731"),
     ),
     # DSpark on, against the `vllm` column with it off -- the pair is the measurement, so read the two
@@ -303,12 +308,11 @@ VARIANTS: tuple[VariantSpec, ...] = (
         "vllm-dspark",
         "vllm",
         {
-            "enforce_eager": True,
             "extra_vllm_kwargs": {
                 "speculative_config": {"method": "dspark", "num_speculative_tokens": 5},
             },
         },
-        "DSpark speculative decoding ON; enforce_eager, so still capture-capable",
+        "DSpark speculative decoding ON; hooked backend, so still capture-capable",
         "interp-engine vllm +DSpark",
         models=("deepseek-v4-flash-0731",),
     ),
@@ -325,10 +329,8 @@ VARIANTS: tuple[VariantSpec, ...] = (
     # never calls the Python forward a hook is attached to.
     VariantSpec(
         "vllm-dspark-cudagraph",
-        "vllm",
+        "vllm-generate",
         {
-            "enforce_eager": False,
-            "freeze_points": [],
             "extra_vllm_kwargs": {
                 "speculative_config": {"method": "dspark", "num_speculative_tokens": 5},
             },

{interp_engine-1.2.9 → interp_engine-1.3.0}/benchmarks/cells.py

@@ -77,9 +77,9 @@ def row_spec(cells: list[dict[str, Any]], model_key: str, variants: Iterable[str
     """Every override one model's row declared, merged across the row's cells.
 
     A per-variant override is recorded by the cell that used it and by no other: on
-    `deepseek-v4-flash-0731` both `freeze_capture_point` and `per_variant_vllm_kwargs` live on the
-    freeze cell alone. So any single cell's ``model`` describes a column rather than a row, and picking
-    one -- whichever happened to sort last -- silently dropped the two overrides the freeze column is
+    `deepseek-v4-flash-0731` both `static_capture_point` and `per_variant_vllm_kwargs` live on the
+    static cell alone. So any single cell's ``model`` describes a column rather than a row, and picking
+    one -- whichever happened to sort last -- silently dropped the two overrides the static column is
     the only cell to declare.
 
     ``variants`` restricts the merge to the columns a renderer shows, and each of them passes its own:
@@ -129,9 +129,9 @@ def nonuniform(model: dict[str, Any]) -> list[str]:
     point = model.get("capture_point", DEFAULT_POINT)
     if point != DEFAULT_POINT:
         reasons.append(f"captures `{point}` rather than `{DEFAULT_POINT}`")
-    frozen_point = model.get("freeze_capture_point")
-    if frozen_point:
-        reasons.append(f"its freeze column captures `{frozen_point}`, a wider point than the others do")
+    static_point = model.get("static_capture_point")
+    if static_point:
+        reasons.append(f"its static column captures `{static_point}`, a wider point than the others do")
     fraction = model.get("gpu_memory_utilization")
     if fraction:
         reasons.append(f"vLLM reserved {fraction} of the card, not the uniform {GPU_MEMORY_UTILIZATION}")

{interp_engine-1.2.9 → interp_engine-1.3.0}/benchmarks/probe_lens_stream.py

@@ -6,7 +6,7 @@ neither: it drives :meth:`VLLMModel.lens_capture_readout_stream`, which issues o
 ``collective_rpc`` per yield from ``engine.generate``, and nothing so far says how many of those
 there are per generated token or what share of the request they are.
 
-    python -m benchmarks.probe_lens_stream                    # qwen3.8-27b, freeze, 512 -> 64
+    python -m benchmarks.probe_lens_stream                    # qwen3.8-27b, static, 512 -> 64
     python -m benchmarks.probe_lens_stream --variant hooked   # the same without CUDA graphs
     python -m benchmarks.probe_lens_stream --no-jacobians     # skip the transport (tight card)
 
@@ -16,8 +16,8 @@ means the engine already outruns the read-out -- which the method's own docstrin
 which nothing had measured -- and the round trips are not where the time is.
 
 The second figure to read is *time to first read-out* against the baseline's time to first
-token. Everything between them is the prompt-wide capture: one frozen site per layer harvested
-over every prompt position, then transported and unembedded. On a trunk where ``"auto"`` freezes
+token. Everything between them is the prompt-wide capture: one static site per layer harvested
+over every prompt position, then transported and unembedded. On a trunk where ``"auto"`` declares
 a stream stack that term dominates, so the split matters before optimizing either half.
 
 Deliberately one process and one model, like ``run_bench``: vLLM reserves its fraction of the
@@ -428,7 +428,7 @@ class Args:
     top_n: int = ENDPOINT_TOP_N
     chunk_positions: int = ENDPOINT_CHUNK_POSITIONS
     point: str = "resid_post"
-    variant: str = "freeze"
+    variant: str = "static"
     word_mask: bool = True
     jacobians: bool = True
     gpu_memory_utilization: float | None = None
@@ -448,7 +448,7 @@ def parse_args(argv: list[str]) -> Args:
     p.add_argument("--top-n", type=int, default=ENDPOINT_TOP_N)
     p.add_argument("--chunk-positions", type=int, default=ENDPOINT_CHUNK_POSITIONS)
     p.add_argument("--point", default="resid_post", help="capture point; resid_streams on a hyper-connection trunk")
-    p.add_argument("--variant", default="freeze", choices=("freeze", "hooked"), help="freeze = production")
+    p.add_argument("--variant", default="static", choices=("static", "hooked"), help="static = production")
     p.add_argument("--no-word-mask", dest="word_mask", action="store_false")
     p.add_argument("--no-jacobians", dest="jacobians", action="store_false")
     p.add_argument("--gpu-memory-utilization", type=float, default=None)
@@ -494,15 +494,19 @@ async def run(args: Args) -> dict[str, Any]:
         "gpu_memory_utilization": utilization,
         "extra_vllm_kwargs": extra_vllm,
     }
-    if args.variant == "freeze":
-        # What production takes with FREEZE_POINTS=auto. No `freeze_writes`: the request measured
-        # here carries no intervention, so a write tap would be an unused buffer competing for the
-        # same pool the graphs and the KV cache come out of.
-        kwargs["freeze_points"] = "auto"
+    if args.variant == "static":
+        backend = "vllm-static"
+        # What production takes with STATIC_POINTS=auto. Passed explicitly even though the backend
+        # would default to it, so the run records which set it measured. No `static_writes`: the
+        # request measured here carries no intervention, so a write tap would be an unused buffer
+        # competing for the same pool the graphs and the KV cache come out of.
+        kwargs["static_points"] = "auto"
+    else:
+        backend = "vllm"
 
     print(f"loading {hf_id} ({args.variant})...", flush=True)
     timer = Timer()
-    model = load_model(hf_id, backend="vllm", dtype=dtype, **kwargs)
+    model = load_model(hf_id, backend=backend, dtype=dtype, **kwargs)
     construct_s = timer.elapsed()
     timer.reset()
     await model.warmup()
@@ -567,7 +571,7 @@ async def run(args: Args) -> dict[str, Any]:
 
         setup = {
             "model": f"{hf_id} ({model.n_layers} layers, d_model {model.d_model})",
-            "variant": f"{args.variant}" + (" (freeze_points=auto)" if args.variant == "freeze" else ""),
+            "variant": f"{args.variant}" + (" (static_points=auto)" if args.variant == "static" else ""),
             "request": f"{len(prompt_ids)} prompt tokens -> {args.new_tokens} new, top-{args.top_n}",
             "read-out": f"{len(specs)} type(s) x {len(layers)} layers, chunk_positions={args.chunk_positions}",
             "word mask": f"{word_mask.shape[0]:,} ids" if word_mask is not None else "off",

{interp_engine-1.2.9 → interp_engine-1.3.0}/benchmarks/publish.py

@@ -50,13 +50,13 @@ START = "<!-- THROUGHPUT:START -->"
 END = "<!-- THROUGHPUT:END -->"
 
 #: ``(variant key, README heading, field name in the visualizer's `Benchmark`)``. Three of the sweep's
-#: six variants: the two capture-capable backends the engine offers, plus graph freeze, which is the
+#: six variants: the two capture-capable backends the engine offers, plus static taps, which is the
 #: claim this section exists to make. `vllm-cudagraph` is vanilla vLLM and cannot capture at all, so
 #: publishing it beside these would invite a comparison the engine is not making.
 COLUMNS: tuple[tuple[str, str, str], ...] = (
     ("eager", "eager", "eager"),
     ("vllm", "vLLM", "vllm"),
-    ("vllm-freeze", "vLLM + graph freeze", "freeze"),
+    ("vllm-static", "vLLM + static taps", "static"),
 )
 
 #: ``(field name in a `Rate`, workload, metric)``.
@@ -68,7 +68,7 @@ REGIMES: tuple[tuple[str, str, str], ...] = (
 #: The column every multiplier is against, and the one that carries no multiplier of its own.
 BASELINE = "eager"
 #: Bolded in the README, being the number the section is about.
-HIGHLIGHT = "vllm-freeze"
+HIGHLIGHT = "vllm-static"
 
 MISSING_CELL = "—"
 
@@ -223,7 +223,7 @@ def card_models(cells: list[dict[str, Any]]) -> dict[str, list[str]]:
     This used to drop such a row instead. Being unpublishable and being unexplained are not the same
     problem, and conflating them cost the card its most interesting row -- `deepseek-v4-flash-0731`
     reserves its own fraction of the GPU and serves an FP8 KV cache, and also happens to be where
-    freeze wins by the largest margin in the sweep. What made the old rule right was that the card had
+    static wins by the largest margin in the sweep. What made the old rule right was that the card had
     nowhere to state an exception; the fix is the footnote, not the omission.
 
     One reason to drop a row remains, and prints a line on the console: a model missing either baseline

{interp_engine-1.2.9 → interp_engine-1.3.0}/benchmarks/report_bench.py

@@ -44,11 +44,11 @@ REPORT_PATH = Path(__file__).resolve().parent / "results-latest.md"
 #: Ordered here rather than by ``cells.variant_order``'s spec order because the spec is ordered by how
 #: the variants relate to each other -- vanilla vLLM next to the capture-capable one it is the control
 #: for -- while a reader of the tables wants the engine's three own configurations adjacent and the
-#: outside reference last, so `vllm-freeze` sits beside `vllm` rather than a column away from it.
+#: outside reference last, so `vllm-static` sits beside `vllm` rather than a column away from it.
 #:
 #: A present variant that is in neither tuple is appended rather than dropped, so adding one to
 #: ``bench_spec`` cannot silently produce a report that does not mention it.
-COLUMNS: tuple[str, ...] = ("eager", "vllm", "vllm-freeze", "vllm-cudagraph")
+COLUMNS: tuple[str, ...] = ("eager", "vllm", "vllm-static", "vllm-cudagraph")
 
 #: Variants whose cells this report does not render, and why.
 #:
@@ -281,11 +281,11 @@ def _exceptions_section(cells: list[dict[str, Any]]) -> list[str]:
                 f"the capture and steering workloads address `{point}`, not `{DEFAULT_POINT}` "
                 "(see `bench_spec.py` for why this architecture has no such point)"
             )
-        frozen_point = m.get("freeze_capture_point")
-        if frozen_point:
+        static_point = m.get("static_capture_point")
+        if static_point:
             notes.append(
-                f"under freeze those workloads address `{frozen_point}` instead, because a frozen "
-                f'engine serves the set it froze and `"auto"` on this trunk freezes `{frozen_point}` '
+                f"under static those workloads address `{static_point}` instead, because a static "
+                f'engine serves the set it declared and `"auto"` on this trunk declares `{static_point}` '
                 "-- the whole stack of parallel residual streams per layer, so that one column's "
                 "capture and transport figures price the stack where every other column prices a row"
             )
@@ -446,7 +446,7 @@ def build_report(cells: list[dict[str, Any]], sweep_command: str) -> str:
         "",
         "`steer` and `capture_gen` are the same workload apart from the steering spec, so the difference",
         "is the cost of the mechanism: one write hook eagerly, on vLLM an extra `collective_rpc` to",
-        "install it in the worker, and under freeze an add into a buffer the graph already refers to,",
+        "install it in the worker, and under static an add into a buffer the graph already refers to,",
         "which is why that column lands inside the noise floor in both directions.",
         "",
         *_steering_overhead(cells),

{interp_engine-1.2.9 → interp_engine-1.3.0}/benchmarks/results/deepseek-v4-flash-0731__vllm-cudagraph.json

@@ -22,11 +22,9 @@
   },
   "variant": {
     "key": "vllm-cudagraph",
-    "backend": "vllm",
-    "note": "CUDA graphs + inductor compile ON, no freeze wraps; vLLM's own defaults for generate-only",
+    "backend": "vllm-generate",
+    "note": "CUDA graphs + inductor compile ON, no static wraps; vLLM's own defaults for generate-only",
     "kwargs": {
-      "enforce_eager": "False",
-      "freeze_points": "[]",
       "max_model_len": "2048",
       "gpu_memory_utilization": "0.95",
       "extra_vllm_kwargs": "{'kv_cache_dtype': 'fp8', 'enable_prefix_caching': False}"