interp-engine 1.2.7__tar.gz → 1.2.9__tar.gz

{interp_engine-1.2.7 → interp_engine-1.2.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: interp-engine
-Version: 1.2.7
+Version: 1.2.9
 Summary: A fast, standardized interpretability engine that supports most modern models and architectures. Powers Neuronpedia.
 Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
 Project-URL: Repository, https://github.com/decoderesearch/interp-engine
@@ -71,7 +71,15 @@ pip install interp-engine          # eager backend only
 ```python
 from interp_engine import Address, load_model, run_with_cache
 
-model = load_model("Qwen/Qwen3-8B")    # vLLM default, use backend='eager' to override
+# VLLM MODE (default): low VRAM, medium speed
+model = load_model("Qwen/Qwen3-8B") 
+
+# VLLM-FREEZE MODE: high VRAM, high speed, only frozen points (default resid_post)
+# model = load_model("Qwen/Qwen3-8B", freeze_points="auto")
+
+# EAGER MODE: low VRAM, low speed
+# model = load_model("Qwen/Qwen3-8B", backend="eager")
+
 point = Address("resid_post", 10)      # or string: "resid_post.10"
 cache = run_with_cache(model, model.to_tokens("Hello, world"), [point])
 cache[point]  # [batch, pos, ...]
@@ -87,7 +95,7 @@ Add "use interp-engine" to your prompt and let your agent figure it out - everyt
 
 ## Performance / Speed
 
-vLLM gives `interp-engine` high throughput via concurrency, and **graph freeze** adds CUDA-graph replay on top of that _without giving up capture or steering_. Every column below is capture-capable.
+vLLM gives `interp-engine` high throughput via concurrency, and **graph freeze** adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
 
 <!-- THROUGHPUT:START -->
 
@@ -117,15 +125,7 @@ One stream (tok/s):
 
 <!-- THROUGHPUT:END -->
 
-Every multiplier is against eager on the same workload, and is the ratio of the _unrounded_ figures — tok/s is printed whole at 10 and above, so dividing two cells by hand can differ in the last place. Eager's generation loop is synchronous underneath, so eight requests serialize rather than batch — which is why it earns almost nothing from the second table and why the multipliers there are so much larger. `deepseek-v4-flash-0731` is block-quantized FP8 on both backends rather than bf16, so read its row as two backends serving the same quantized weights; it also keeps a tenth, because at 3 tok/s that digit is worth several percent. [`benchmarks/results-latest.md`](benchmarks/results-latest.md) has every figure at full precision.
-
-**Graph freeze** is opt-in, via `freeze_points`. Ordinary hooks cannot survive CUDA graphs — replay never calls the Python forward a hook is attached to — so the tap becomes a preallocated buffer plus a `copy_` that graph capture records and replay re-executes. Steering rides the same wrap, so additive, orthogonal, projection-cap and j-lens steer/ablate/swap all keep working under graphs, position masks included.
-
-```python
-model = load_model("Qwen/Qwen3-8B", freeze_points="auto")  # resid_post at every layer, under graphs
-```
-
-The trade is that a frozen engine serves _the set it froze_ rather than any point on request: `"auto"` covers `resid_post` at every layer on a conventional trunk, an explicit list covers whatever you name, and anything outside the set is refused rather than quietly returned empty. Omit `freeze_points` for today's hooked vLLM, which still serves every point. `qwen3.8-27b` is a hybrid trunk, so its freeze row runs prefill eagerly and keeps the decode graphs — freeze pins that mode, because breakable graphs turn `torch.compile` off and vLLM's mixed prefill-decode capture then miscomputes prefill on a gated-delta trunk. On a hyper-connection trunk such as `deepseek-v4-flash-0731`, whose block carries four parallel residual streams, `"auto"` freezes `resid_streams` instead — the whole stack per layer, which is four times the width and so four times the buffer, and is what that row's freeze cell prices. See [PERFORMANCE.md](docs/PERFORMANCE.md) for the full trade-off, and [`benchmarks/results-latest.md`](benchmarks/results-latest.md) for the sweep, including capture, steering and lens latencies.
+**Graph freeze** is opt-in via `freeze_points`, and a frozen engine serves only the set it froze. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.
 
 ## Correctness
 

{interp_engine-1.2.7 → interp_engine-1.2.9}/README.md

@@ -40,7 +40,15 @@ pip install interp-engine          # eager backend only
 ```python
 from interp_engine import Address, load_model, run_with_cache
 
-model = load_model("Qwen/Qwen3-8B")    # vLLM default, use backend='eager' to override
+# VLLM MODE (default): low VRAM, medium speed
+model = load_model("Qwen/Qwen3-8B") 
+
+# VLLM-FREEZE MODE: high VRAM, high speed, only frozen points (default resid_post)
+# model = load_model("Qwen/Qwen3-8B", freeze_points="auto")
+
+# EAGER MODE: low VRAM, low speed
+# model = load_model("Qwen/Qwen3-8B", backend="eager")
+
 point = Address("resid_post", 10)      # or string: "resid_post.10"
 cache = run_with_cache(model, model.to_tokens("Hello, world"), [point])
 cache[point]  # [batch, pos, ...]
@@ -56,7 +64,7 @@ Add "use interp-engine" to your prompt and let your agent figure it out - everyt
 
 ## Performance / Speed
 
-vLLM gives `interp-engine` high throughput via concurrency, and **graph freeze** adds CUDA-graph replay on top of that _without giving up capture or steering_. Every column below is capture-capable.
+vLLM gives `interp-engine` high throughput via concurrency, and **graph freeze** adds CUDA-graph replay on top of that *without giving up capture or steering*. Every column below is capture-capable.
 
 <!-- THROUGHPUT:START -->
 
@@ -86,15 +94,7 @@ One stream (tok/s):
 
 <!-- THROUGHPUT:END -->
 
-Every multiplier is against eager on the same workload, and is the ratio of the _unrounded_ figures — tok/s is printed whole at 10 and above, so dividing two cells by hand can differ in the last place. Eager's generation loop is synchronous underneath, so eight requests serialize rather than batch — which is why it earns almost nothing from the second table and why the multipliers there are so much larger. `deepseek-v4-flash-0731` is block-quantized FP8 on both backends rather than bf16, so read its row as two backends serving the same quantized weights; it also keeps a tenth, because at 3 tok/s that digit is worth several percent. [`benchmarks/results-latest.md`](benchmarks/results-latest.md) has every figure at full precision.
-
-**Graph freeze** is opt-in, via `freeze_points`. Ordinary hooks cannot survive CUDA graphs — replay never calls the Python forward a hook is attached to — so the tap becomes a preallocated buffer plus a `copy_` that graph capture records and replay re-executes. Steering rides the same wrap, so additive, orthogonal, projection-cap and j-lens steer/ablate/swap all keep working under graphs, position masks included.
-
-```python
-model = load_model("Qwen/Qwen3-8B", freeze_points="auto")  # resid_post at every layer, under graphs
-```
-
-The trade is that a frozen engine serves _the set it froze_ rather than any point on request: `"auto"` covers `resid_post` at every layer on a conventional trunk, an explicit list covers whatever you name, and anything outside the set is refused rather than quietly returned empty. Omit `freeze_points` for today's hooked vLLM, which still serves every point. `qwen3.8-27b` is a hybrid trunk, so its freeze row runs prefill eagerly and keeps the decode graphs — freeze pins that mode, because breakable graphs turn `torch.compile` off and vLLM's mixed prefill-decode capture then miscomputes prefill on a gated-delta trunk. On a hyper-connection trunk such as `deepseek-v4-flash-0731`, whose block carries four parallel residual streams, `"auto"` freezes `resid_streams` instead — the whole stack per layer, which is four times the width and so four times the buffer, and is what that row's freeze cell prices. See [PERFORMANCE.md](docs/PERFORMANCE.md) for the full trade-off, and [`benchmarks/results-latest.md`](benchmarks/results-latest.md) for the sweep, including capture, steering and lens latencies.
+**Graph freeze** is opt-in via `freeze_points`, and a frozen engine serves only the set it froze. [PERFORMANCE.md](docs/PERFORMANCE.md) has how it works and what it trades; [benchmarks/results-latest.md](benchmarks/results-latest.md) has every figure at full precision, including capture, steering and lens latencies; [benchmarks/README.md](benchmarks/README.md) has how the tables above are rounded.
 
 ## Correctness
 

{interp_engine-1.2.7 → interp_engine-1.2.9}/benchmarks/README.md

@@ -104,6 +104,21 @@ python -m benchmarks.publish          # rewrite both
 python -m benchmarks.publish --check  # exit 1 if either has drifted, and name it
 ```
 
+Both print the same cells in a display form of their own, applied by `publish.py` and pinned by
+`tests/test_published_benchmarks.py`:
+
+- **tok/s is whole at 10 and above, one decimal below.** A tenth beside a four-digit figure in the
+  next column claims a resolution the reader cannot use. At 3 tok/s that same tenth is worth several
+  percent, so the small rows keep it.
+- **every comparison is a multiplier, never a percent**, with one decimal below 20x, where it is
+  still checkable against the two printed figures. `+20%` beside `27x` makes the reader convert one
+  of them.
+- **multipliers are ratios of the unrounded metrics**, each against eager on the same workload, so
+  dividing two printed figures by hand can differ in the last place. Ratios of the rounded figures
+  would make the published win depend on the rounding, which is worse.
+
+`results-latest.md` keeps every figure at full precision, so nothing is lost to those rules.
+
 `tests/test_published_benchmarks.py` runs that check over the committed cells, so a stale copy is a
 red suite rather than a claim nobody re-read. The visualizer's chatbot answers out of a bundle holding
 the README verbatim, so a publish that changed the README also wants `make viz-knowledge` -- the
@@ -225,12 +240,15 @@ already refers to, so a replay serves capture and steering without a Python forw
 `freeze_points="auto"` takes in production, and the row exists to price it against the
 `enforce_eager=True` column capture would otherwise have to use.
 
-`"auto"` installs **reads** only, and a steering op needs a write tap to land in, so this row priced
-half the feature and reported the other half as `n/a` -- with a message that blamed graph replay for
-it, which is the thing freeze exists to work around. It now also passes `freeze_writes`, whose value
-is the sentinel `run_bench.STEER_WRITES` rather than a site: the `steer` workload writes mid-stack,
-that layer differs per model, and a freeze write is a `load_model` argument, so it has to be resolved
-from the config before a model exists to ask.
+`"auto"` once installed **reads** only, and a steering op needs a write tap to land in, so this row
+priced half the feature and reported the other half as `n/a` -- with a message that blamed graph
+replay for it, which is the thing freeze exists to work around. Auto now covers both halves, so the
+cell would be a number either way, and this row still passes `freeze_writes` on purpose: an explicit
+list *narrows* what auto would install, to the one mid-stack site the `steer` workload actually
+writes. A row that priced a write buffer at every layer would not be comparable with the ones beside
+it, which is the whole job of the column. Its value is the sentinel `run_bench.STEER_WRITES` rather
+than a site, because that layer differs per model and a freeze write is a `load_model` argument, so
+it has to be resolved from the config before a model exists to ask.
 
 `VariantSpec.models` restricts the row to the checkpoints freeze has been shown correct on, so a model
 missing from it renders `--` rather than a number nobody checked.

{interp_engine-1.2.7 → interp_engine-1.2.9}/benchmarks/bench_spec.py

@@ -264,12 +264,14 @@ VARIANTS: tuple[VariantSpec, ...] = (
     # Its write tap is `mlp_out`, not `resid_post`: the steer workload addresses the model's
     # `capture_point`, and a hyper-connection trunk refuses the default name (`run_bench._steer_site`).
     #
-    # `freeze_writes` is what makes the `steer` cell a number instead of `n/a`. `"auto"` installs
+    # `freeze_writes` was once what made the `steer` cell a number instead of `n/a`: `"auto"` installed
     # reads, and a steering op needs a write tap to land in, so this row priced capture under replay
     # and left the other half of the feature unmeasured -- with a message that blamed graph replay for
-    # it. The value is the sentinel `run_bench.STEER_WRITES`, resolved there to the mid-stack
-    # `resid_post` the workload steers, because the layer differs per model and a freeze write has to
-    # be named before the model exists.
+    # it. Auto covers writes now, so the cell stands either way, and naming them here has become a
+    # *narrowing*: one write buffer at the site the workload steers rather than one per layer, which
+    # is what keeps this row's memory comparable with the columns beside it. The value is the sentinel
+    # `run_bench.STEER_WRITES`, resolved there to the mid-stack `resid_post` the workload steers,
+    # because the layer differs per model and a freeze write has to be named before the model exists.
     VariantSpec(
         "vllm-freeze",
         "vllm",