interp-engine 1.2.2__tar.gz → 1.2.3__tar.gz

{interp_engine-1.2.2 → interp_engine-1.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: interp-engine
-Version: 1.2.2
+Version: 1.2.3
 Summary: A fast, standardized interpretability engine that supports most modern models and architectures. Powers Neuronpedia.
 Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
 Project-URL: Repository, https://github.com/decoderesearch/interp-engine
@@ -26,7 +26,7 @@ Provides-Extra: quant
 Requires-Dist: accelerate>=1.0; extra == 'quant'
 Requires-Dist: kernels<0.17.0,>=0.15.2; extra == 'quant'
 Provides-Extra: vllm
-Requires-Dist: vllm>=0.25.1; (sys_platform == 'linux') and extra == 'vllm'
+Requires-Dist: vllm>=0.27.1; (sys_platform == 'linux') and extra == 'vllm'
 Description-Content-Type: text/markdown
 
 # interp-engine
@@ -38,6 +38,7 @@ Description-Content-Type: text/markdown
   🔗 <a href="https://interp-engine.org"><strong>interp-engine.org</strong></a>
 </p>
 
+
 `interp-engine` is an interpretability engine that is fast (>40x tok/s vs HF eager), standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.
 
 <p align="center">

{interp_engine-1.2.2 → interp_engine-1.2.3}/README.md

@@ -7,6 +7,7 @@
   🔗 <a href="https://interp-engine.org"><strong>interp-engine.org</strong></a>
 </p>
 
+
 `interp-engine` is an interpretability engine that is fast (>40x tok/s vs HF eager), standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.
 
 <p align="center">

{interp_engine-1.2.2 → interp_engine-1.2.3}/docs/PERFORMANCE.md

@@ -42,13 +42,17 @@ every new upstream flag into an engine change.
 
 ### The vLLM version is a floor, not a ceiling
 
-`interp-engine[vllm]` declares `vllm>=0.25.1` on Linux and no upper bound. Read that as **the oldest
+`interp-engine[vllm]` declares `vllm>=0.27.1` on Linux and no upper bound. Read that as **the oldest
 version we have evidence for**, not the version to run:
 
-- **The floor is measured, not guessed.** 0.25.1 is the oldest version the engine has been exercised
-  on. The current sweep in [`benchmarks/results-latest.md`](../benchmarks/results-latest.md) ran on
-  **0.26.0**, which is also what `apps/inference` and `apps/nla` resolve to — being ahead of the floor
-  is the normal state, not drift.
+- **The floor is measured, not guessed.** 0.27.1 is the version the DeepSeek-V4-Flash-0731
+  cross-engine comparison scored, and the oldest the engine is supported on. The performance sweep in
+  [`benchmarks/results-latest.md`](../benchmarks/results-latest.md) ran on **0.26.0**, one below —
+  being ahead of the floor is the normal state, not drift.
+- **This floor is load-bearing, not hygiene.** It was 0.25.1 until `vllm_capture/mhc.py` needed
+  `mhc_pre_broadcast_tilelang`, which arrived in 0.26.0. Below that, every DeepSeek-V4
+  hyper-connection point is refused at install, and since `FREEZE_POINTS=auto` selects
+  `resid_streams` on such a trunk, the pod does not load at all.
 - **There is deliberately no cap.** vLLM moves fast and a `<` bound would make every upstream release
   an engine release; a lock file is the right place to pin a deployment, and both apps have one. The
   version an app runs is whatever its `uv.lock` resolved, which is normally newer than this floor.

{interp_engine-1.2.2 → interp_engine-1.2.3}/interp_engine/vllm_capture/mhc.py

@@ -308,6 +308,29 @@ def _kernel_module(layer: torch.nn.Module) -> ModuleType | None:
     return module
 
 
+def _missing_kernels(layer: torch.nn.Module) -> tuple[str, ...]:
+    """Which of :data:`KERNEL_NAMES` the layer's defining module does not have.
+
+    Separated from :func:`_kernel_module`, which only answers yes or no, because *which* names are
+    absent is the difference between the two reasons a model gets refused: none of them means the
+    wrong platform tree, some of them means the right tree on a vLLM that predates a kernel.
+    """
+    module = sys.modules.get(type(layer).__module__)
+    if module is None:
+        return KERNEL_NAMES
+    return tuple(name for name in KERNEL_NAMES if not hasattr(module, name))
+
+
+def _vllm_version() -> str:
+    """The running vLLM's version for an error message, or a placeholder. Never raises."""
+    try:
+        from importlib.metadata import version
+
+        return version("vllm")
+    except Exception:  # pragma: no cover - a diagnostic string is not worth a failure mode
+        return "unknown"
+
+
 def _mhc_layers(model: torch.nn.Module) -> list[tuple[int, torch.nn.Module]]:
     layers = _get_layers(model)
     return [(i, layer) for i, layer in enumerate(layers) if hasattr(layer, _MHC_MARKER_PARAM)]
@@ -332,9 +355,23 @@ def unavailable_reason(model: torch.nn.Module, name: str, layer: int | None = No
         )
     module = _kernel_module(mhc[0][1])
     if module is None:
+        layer_type = type(mhc[0][1])
+        missing = _missing_kernels(mhc[0][1])
+        if len(missing) < len(KERNEL_NAMES):
+            # Some names are there and some are not, so this IS the tree that calls the kernels by
+            # name -- it is just an older one than the engine was written against. Said separately
+            # from the tree refusal below because the fix is a vLLM upgrade rather than a port, and
+            # the two are indistinguishable from the point's name alone.
+            return (
+                f"{layer_type.__name__} (from {layer_type.__module__}) calls the tilelang mHC "
+                f"kernels by name, but {', '.join(missing)} is not in its namespace and this point "
+                f"is served by intercepting all of {', '.join(KERNEL_NAMES)} there. That is a vLLM "
+                f"older than this engine supports, not the wrong platform tree: the installed vLLM "
+                f"is {_vllm_version()} and interp-engine declares vllm>=0.27.1. Upgrade vLLM."
+            )
         return (
-            f"{type(mhc[0][1]).__name__} does not call vLLM's tilelang mHC kernels by name, so there "
-            "is nothing to wrap: this point is served by intercepting "
+            f"{layer_type.__name__} (from {layer_type.__module__}) does not call vLLM's tilelang mHC "
+            "kernels by name, so there is nothing to wrap: this point is served by intercepting "
             f"{', '.join(KERNEL_NAMES)} in the model module's namespace, which only "
             "models/deepseek_v4/nvidia/model.py does. The amd/ and xpu/ trees instantiate "
             "MHCPreOp/MHCPostOp/MHCFusedPostPreOp as modules and apply the block norm separately, so "

{interp_engine-1.2.2 → interp_engine-1.2.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "interp-engine"
-version = "1.2.2"
+version = "1.2.3"
 description = "A fast, standardized interpretability engine that supports most modern models and architectures. Powers Neuronpedia."
 readme = "README.md"
 license = "Apache-2.0"
@@ -46,9 +46,15 @@ dev = [
 # EagerModel core must install on macOS/CPU). Apps that actually run the vLLM backend depend on
 # `interp-engine[vllm]` so the version pin lives here, next to the code that imports it.
 #
-# A FLOOR, not a ceiling, and deliberately uncapped: 0.25.1 is the oldest version there is evidence
-# for (the whole benchmark sweep ran on it), not the version to run. Raise it only when engine code
-# needs a newer API, and name that code. See docs/PERFORMANCE.md, "The vLLM version is a floor, not a
+# A FLOOR, not a ceiling, and deliberately uncapped: 0.27.1 is the oldest version there is evidence
+# for, not the version to run. Raise it only when engine code needs a newer API, and name that code.
+# Raised from 0.25.1 for `vllm_capture/mhc.py`, which reaches DeepSeek-V4's hyper-connection points
+# by patching all four tilelang mHC kernels in the model module's namespace (`KERNEL_NAMES`). One of
+# them, `mhc_pre_broadcast_tilelang`, does not exist before 0.26.0, so on 0.25.x every mHC point --
+# `resid_streams` included, which `FREEZE_POINTS=auto` selects on such a trunk -- is refused at
+# install and the pod fails to load. 0.27.1 rather than 0.26.0 because that is the version the
+# DeepSeek-V4-Flash-0731 cross-engine comparison actually scored (validator/comparison/results/
+# deepseek-ai/DeepSeek-V4-Flash-0731/). See docs/PERFORMANCE.md, "The vLLM version is a floor, not a
 # ceiling", for why there is no upper bound and what to do when an upstream release breaks capture.
 #
 # Why this is an extra and NOT a marker-gated base dependency, which is the recurring suggestion:
@@ -66,7 +72,7 @@ dev = [
 # The `sys_platform` marker below is the one axis a marker CAN answer: it makes the extra itself a
 # no-op on macOS rather than an error, so `[vllm]` is safe to ask for unconditionally.
 vllm = [
-    "vllm>=0.25.1; sys_platform == 'linux'",
+    "vllm>=0.27.1; sys_platform == 'linux'",
 ]
 # MXFP4-quantized models (e.g. gpt-oss) and block-FP8 ones (DeepSeek-V4-Flash) need the Triton kernel
 # loader, which in turn needs huggingface-hub>=1.0 / transformers>=5. Missing it is not always an

{interp_engine-1.2.2 → interp_engine-1.2.3}/tests/test_vllm_hyper_connections.py

@@ -759,12 +759,30 @@ def test_a_tree_that_does_not_call_these_kernels_by_name_is_refused_with_the_alt
     norm separately, so there is nothing to wrap -- and every one of these points is an ordinary module
     output there. Unwired rather than unreachable, and the refusal has to say which."""
     trunk = _HyperTrunk()
-    monkeypatch.delattr(sys.modules[__name__], "mhc_post_tilelang")
+    for name in KERNEL_NAMES:
+        monkeypatch.delattr(sys.modules[__name__], name)
     reason = unavailable_reason(trunk, "resid_streams", 0)
     assert reason is not None
     assert "amd/ and xpu/" in reason and "module output" in reason
 
 
+def test_a_vllm_too_old_for_one_kernel_is_refused_as_a_version_and_not_as_a_tree(monkeypatch):
+    """The refusal these two tests share a branch with is the one that cost a pod an afternoon.
+
+    `mhc_pre_broadcast_tilelang` arrived in vLLM 0.26.0, so on 0.25.x the NVIDIA tree calls three of
+    these four by name and the wrapper -- which needs all four -- backs out. Answering that with the
+    amd/xpu text sends the reader to a platform port when the fix is `pip install -U vllm`, so a
+    module that has *some* of the names has to be told apart from one that has none.
+    """
+    trunk = _HyperTrunk()
+    monkeypatch.delattr(sys.modules[__name__], "mhc_pre_broadcast_tilelang")
+    reason = unavailable_reason(trunk, "resid_streams", 0)
+    assert reason is not None
+    assert "mhc_pre_broadcast_tilelang" in reason, "the reader has to be told which name is missing"
+    assert "Upgrade vLLM" in reason
+    assert "amd/" not in reason, "a version skew is not a platform-tree problem"
+
+
 def test_the_last_layers_stack_is_refused_when_a_draft_model_shares_the_closing_call():
     """EAGLE reconstructs auxiliary hidden states with the same `mhc_post` function, so which call
     completed the trunk stops being decidable. Refused rather than guessed -- and only for the layer