interp-engine 1.4.0__tar.gz → 1.5.0__tar.gz

{interp_engine-1.4.0 → interp_engine-1.5.0}/.gitignore

@@ -36,4 +36,6 @@ plans/
 *.plan.md
 
 logs/*
-logs/
+logs/
+
+HANDOFF.md

{interp_engine-1.4.0 → interp_engine-1.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: interp-engine
-Version: 1.4.0
+Version: 1.5.0
 Summary: A fast, standardized, and easy to use interpretability engine.
 Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
 Project-URL: Repository, https://github.com/decoderesearch/interp-engine
@@ -26,7 +26,7 @@ Provides-Extra: quant
 Requires-Dist: accelerate>=1.0; extra == 'quant'
 Requires-Dist: kernels<0.17.0,>=0.15.2; extra == 'quant'
 Provides-Extra: vllm
-Requires-Dist: vllm>=0.27.1; (sys_platform == 'linux') and extra == 'vllm'
+Requires-Dist: vllm>=0.28.0; (sys_platform == 'linux') and extra == 'vllm'
 Description-Content-Type: text/markdown
 
 # interp-engine
@@ -60,6 +60,7 @@ This repo contains:
 
 1. [`validator/`](validator/), which compares/validates it against TransformerLens, and nnsight/nnterp on real architectures.
 2. [`visualizer-web/`](visualizer-web/), a "cheat sheet" hosted at [interp-engine.org](https://interp-engine.org) of each 'point' (eg `resid_post.16`), standardized across architectures.
+3. [`gpu-sizer/`](gpu-sizer/), which finds the GPU/config you need to fit an interp-engine model (and what performance you'll get), to avoid OOMing while working. [docs](https://interp-engine.org/docs/gpu-sizer) | [API](https://interp-engine.org/docs/gpu-sizer-api)
 
 ## Installation
 
@@ -139,12 +140,15 @@ We verify correctness in two main ways:
 1. A test suite that checks results across several models - what each check is designed to catch is in [INTERNALS.md](docs/INTERNALS.md#correctness).
 2. A full `validator` comparison engine that checks most hook points across 50+ models, at early, middle and late layers - fully reproducible, with detailed results saved in the git repo at [`validator/`](validator/).
 
-## Known Issues
+## GPU-Sizer
 
-- **Gemma 4 requires transformers 5.14.1** because 5.15 moved `head_dim` into `per_layer_config` and vLLM's config read dies before a weight loads ([vllm#51744](https://github.com/vllm-project/vllm/issues/51744)).
-- **Gemma 4 26B refuses `mlp_out`**, because `layer.mlp` there is one branch of a two-branch feed-forward; `mlp_out_post` is the residual contribution and is served ([ARCHITECTURE_QUIRKS.md](docs/ARCHITECTURE_QUIRKS.md#moe-tap-the-block-not-the-experts)).
-- **DeepSeek-V2 on transformers older than 5.15.0** captures a wrong attention temperature — the engine warns at load, and upgrading is the fix ([COMPATIBILITY.md](docs/COMPATIBILITY.md)).
-- **MXFP4 checkpoints (gpt-oss) need `interp-engine[quant]`**, which the `[vllm]` extra does not include; without it transformers dequantizes them to bf16 at roughly 3x the weights, which can turn a model that fits into one that does not.
+<p align="center">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/gpu-sizer.gif" alt="interp-engine demo gif" width="100%">
+</p>
+
+[docs](https://interp-engine.org/docs/gpu-sizer) | [API](https://interp-engine.org/docs/gpu-sizer-api)
+
+Never OOM again - `interp-engine` includes gpu-sizer, an intuitive UI which tells you what GPU(s) and configs you need to get the best performance out of your selected model. For example, [interp-engine.org/sizer/Qwen/Qwen3.6-27B](https://interp-engine.org/sizer/Qwen/Qwen3.6-27B) shows that you can run `Qwen3.6-27B` with interp-engine's vllm-static backend for max speed while keeping it in a single 80GB A100, and have ~40k tokens for the KV Cache.
 
 ## Why use an Interpretability Engine instead of building from scratch?
 
@@ -152,6 +156,14 @@ We verify correctness in two main ways:
 2. **Standardization + Verification**: Eliminate ambiguity when referring to points, plus a full test suite included.
 3. **Faster Dev / Fewer Tokens Used**: You could spend ten million tokens and have your AI write, test, and make production-ready an interpretability engine. Or you could just `pip install interp-engine[vllm]`.
 
+## Caveats
+
+- **Gemma 4 requires transformers 5.14.1** because 5.15 moved `head_dim` into `per_layer_config` and vLLM's config read dies before a weight loads ([vllm#51744](https://github.com/vllm-project/vllm/issues/51744)).
+- **DeepSeek-V2 on transformers older than 5.15.0** captures a wrong attention temperature — the engine warns at load, and upgrading is the fix ([COMPATIBILITY.md](docs/COMPATIBILITY.md)).
+- **MXFP4 checkpoints (gpt-oss) need `interp-engine[quant]`**, which the `[vllm]` extra does not include; without it transformers dequantizes them to bf16 at roughly 3x the weights, which can turn a model that fits into one that does not.
+
+Per-architecture structural quirks — which points a family serves and why — are not caveats but facts about the architecture, and live in [ARCHITECTURE_QUIRKS.md](docs/ARCHITECTURE_QUIRKS.md).
+
 ## Development / Contributing
 
 Activate the shared git hooks once per clone — they format staged Python, rebuild the generated files, and run CI's static checks before a push. Details in [CONTRIBUTING.md](CONTRIBUTING.md).

{interp_engine-1.4.0 → interp_engine-1.5.0}/README.md

@@ -29,6 +29,7 @@ This repo contains:
 
 1. [`validator/`](validator/), which compares/validates it against TransformerLens, and nnsight/nnterp on real architectures.
 2. [`visualizer-web/`](visualizer-web/), a "cheat sheet" hosted at [interp-engine.org](https://interp-engine.org) of each 'point' (eg `resid_post.16`), standardized across architectures.
+3. [`gpu-sizer/`](gpu-sizer/), which finds the GPU/config you need to fit an interp-engine model (and what performance you'll get), to avoid OOMing while working. [docs](https://interp-engine.org/docs/gpu-sizer) | [API](https://interp-engine.org/docs/gpu-sizer-api)
 
 ## Installation
 
@@ -108,12 +109,15 @@ We verify correctness in two main ways:
 1. A test suite that checks results across several models - what each check is designed to catch is in [INTERNALS.md](docs/INTERNALS.md#correctness).
 2. A full `validator` comparison engine that checks most hook points across 50+ models, at early, middle and late layers - fully reproducible, with detailed results saved in the git repo at [`validator/`](validator/).
 
-## Known Issues
+## GPU-Sizer
 
-- **Gemma 4 requires transformers 5.14.1** because 5.15 moved `head_dim` into `per_layer_config` and vLLM's config read dies before a weight loads ([vllm#51744](https://github.com/vllm-project/vllm/issues/51744)).
-- **Gemma 4 26B refuses `mlp_out`**, because `layer.mlp` there is one branch of a two-branch feed-forward; `mlp_out_post` is the residual contribution and is served ([ARCHITECTURE_QUIRKS.md](docs/ARCHITECTURE_QUIRKS.md#moe-tap-the-block-not-the-experts)).
-- **DeepSeek-V2 on transformers older than 5.15.0** captures a wrong attention temperature — the engine warns at load, and upgrading is the fix ([COMPATIBILITY.md](docs/COMPATIBILITY.md)).
-- **MXFP4 checkpoints (gpt-oss) need `interp-engine[quant]`**, which the `[vllm]` extra does not include; without it transformers dequantizes them to bf16 at roughly 3x the weights, which can turn a model that fits into one that does not.
+<p align="center">
+  <img src="https://neuronpedia.s3.amazonaws.com/site-assets/gpu-sizer.gif" alt="interp-engine demo gif" width="100%">
+</p>
+
+[docs](https://interp-engine.org/docs/gpu-sizer) | [API](https://interp-engine.org/docs/gpu-sizer-api)
+
+Never OOM again - `interp-engine` includes gpu-sizer, an intuitive UI which tells you what GPU(s) and configs you need to get the best performance out of your selected model. For example, [interp-engine.org/sizer/Qwen/Qwen3.6-27B](https://interp-engine.org/sizer/Qwen/Qwen3.6-27B) shows that you can run `Qwen3.6-27B` with interp-engine's vllm-static backend for max speed while keeping it in a single 80GB A100, and have ~40k tokens for the KV Cache.
 
 ## Why use an Interpretability Engine instead of building from scratch?
 
@@ -121,6 +125,14 @@ We verify correctness in two main ways:
 2. **Standardization + Verification**: Eliminate ambiguity when referring to points, plus a full test suite included.
 3. **Faster Dev / Fewer Tokens Used**: You could spend ten million tokens and have your AI write, test, and make production-ready an interpretability engine. Or you could just `pip install interp-engine[vllm]`.
 
+## Caveats
+
+- **Gemma 4 requires transformers 5.14.1** because 5.15 moved `head_dim` into `per_layer_config` and vLLM's config read dies before a weight loads ([vllm#51744](https://github.com/vllm-project/vllm/issues/51744)).
+- **DeepSeek-V2 on transformers older than 5.15.0** captures a wrong attention temperature — the engine warns at load, and upgrading is the fix ([COMPATIBILITY.md](docs/COMPATIBILITY.md)).
+- **MXFP4 checkpoints (gpt-oss) need `interp-engine[quant]`**, which the `[vllm]` extra does not include; without it transformers dequantizes them to bf16 at roughly 3x the weights, which can turn a model that fits into one that does not.
+
+Per-architecture structural quirks — which points a family serves and why — are not caveats but facts about the architecture, and live in [ARCHITECTURE_QUIRKS.md](docs/ARCHITECTURE_QUIRKS.md).
+
 ## Development / Contributing
 
 Activate the shared git hooks once per clone — they format staged Python, rebuild the generated files, and run CI's static checks before a push. Details in [CONTRIBUTING.md](CONTRIBUTING.md).

{interp_engine-1.4.0 → interp_engine-1.5.0}/benchmarks/probe.py

@@ -41,10 +41,14 @@ class EnvStamp:
 #: rather than where it runs. ``VLLM_DEEP_GEMM_WARMUP=skip`` is set by ``run_bench`` on every sweep
 #: (see ``skip_broken_deepgemm_warmup``) and moves DeepGEMM's first compile out of startup and into
 #: the first forward, which is a real difference in ``warmup_s`` on the FP8 rows.
+#:
+#: Only variables the installed vLLM still reads belong here. ``VLLM_ATTENTION_BACKEND`` and
+#: ``VLLM_USE_V1`` were dropped from vLLM (the backend moved to the ``attention_config`` engine
+#: argument, and V0 is gone), and neither name appears anywhere in 0.28.0. Stamping one annotated a
+#: run with a setting that had no effect -- the one reading somebody chasing a discrepancy would
+#: trust, and the worst kind to be wrong.
 _RECORDED_ENV: tuple[str, ...] = (
     "VLLM_DEEP_GEMM_WARMUP",
-    "VLLM_ATTENTION_BACKEND",
-    "VLLM_USE_V1",
     "CUDA_VISIBLE_DEVICES",
 )