interp-engine 1.5.0__tar.gz → 1.6.0__tar.gz

{interp_engine-1.5.0 → interp_engine-1.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: interp-engine
-Version: 1.5.0
+Version: 1.6.0
 Summary: A fast, standardized, and easy to use interpretability engine.
 Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
 Project-URL: Repository, https://github.com/decoderesearch/interp-engine
@@ -113,21 +113,21 @@ One stream (tok/s):
 
 | model                    | eager | vLLM       | vLLM + static taps |
 | ------------------------ | ----- | ---------- | ------------------ |
-| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**     |
-| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**    |
-| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**     |
-| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**      |
-| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**      |
+| `gemma-2-2b`             | 75    | 88 (1.2x)  | **220 (2.9x)**     |
+| `qwen3-4b`               | 60    | 136 (2.3x) | **313 (5.2x)**     |
+| `llama-3.1-8b`           | 84    | 163 (1.9x) | **258 (3.1x)**     |
+| `qwen3.8-27b`            | 25    | 35 (1.4x)  | **63 (2.5x)**      |
+| `deepseek-v4-flash-0731` | 5.3   | 15 (2.9x)  | **132 (25x)**      |
 
 8 concurrent requests (aggregate tok/s):
 
-| model                    | eager | vLLM        | vLLM + static taps |
-| ------------------------ | ----- | ----------- | ------------------ |
-| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**    |
-| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**    |
-| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**    |
-| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**      |
-| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**     |
+| model                    | eager | vLLM          | vLLM + static taps |
+| ------------------------ | ----- | ------------- | ------------------ |
+| `gemma-2-2b`             | 75    | 675 (9.0x)    | **1,436 (19.2x)**  |
+| `qwen3-4b`               | 61    | 973 (15.9x)   | **2,219 (36x)**    |
+| `llama-3.1-8b`           | 81    | 1,187 (14.6x) | **1,755 (22x)**    |
+| `qwen3.8-27b`            | 24    | 246 (10.3x)   | **408 (17.0x)**    |
+| `deepseek-v4-flash-0731` | 5.4   | 117 (22x)     | **631 (116x)**     |
 
 <!-- THROUGHPUT:END -->
 
@@ -148,7 +148,7 @@ We verify correctness in two main ways:
 
 [docs](https://interp-engine.org/docs/gpu-sizer) | [API](https://interp-engine.org/docs/gpu-sizer-api)
 
-Never OOM again - `interp-engine` includes gpu-sizer, an intuitive UI which tells you what GPU(s) and configs you need to get the best performance out of your selected model. For example, [interp-engine.org/sizer/Qwen/Qwen3.6-27B](https://interp-engine.org/sizer/Qwen/Qwen3.6-27B) shows that you can run `Qwen3.6-27B` with interp-engine's vllm-static backend for max speed while keeping it in a single 80GB A100, and have ~40k tokens for the KV Cache.
+Never OOM again - `interp-engine` includes gpu-sizer, an intuitive UI which tells you what GPU(s) and configs you need to get the best performance out of your selected model. For example, [interp-engine.org/sizer/Qwen/Qwen3.6-27B](https://interp-engine.org/sizer/Qwen/Qwen3.6-27B) shows that you can run `Qwen3.6-27B` with interp-engine's vllm-static backend for max speed while keeping it in a single 80GB A100, and have ~160k tokens for the KV Cache.
 
 ## Why use an Interpretability Engine instead of building from scratch?
 
@@ -160,6 +160,7 @@ Never OOM again - `interp-engine` includes gpu-sizer, an intuitive UI which tell
 
 - **Gemma 4 requires transformers 5.14.1** because 5.15 moved `head_dim` into `per_layer_config` and vLLM's config read dies before a weight loads ([vllm#51744](https://github.com/vllm-project/vllm/issues/51744)).
 - **DeepSeek-V2 on transformers older than 5.15.0** captures a wrong attention temperature — the engine warns at load, and upgrading is the fix ([COMPATIBILITY.md](docs/COMPATIBILITY.md)).
+- **DeepSeek-V4 on the eager backend needs two flags outside Hopper.** Its FP8 paths reach for `kernels-community/deep-gemm`, which declares a `9.0a` build, and with `kernels` >= 0.16.1 the arch refusal escapes transformers' Triton fallback and kills the first forward. Load with `TRANSFORMERS_DISABLE_DEEPGEMM_LINEAR=1` and `experts_implementation="grouped_mm"`, which transformers [already recommends on B200](https://github.com/huggingface/transformers/blob/93c8b7b485963a10800c91f55304db6be211c2bd/src/transformers/integrations/finegrained_fp8.py#L252-L254) for an unrelated DeepGEMM accuracy problem. The engine raises `HubKernelUnsupported` naming both; vLLM is unaffected.
 - **MXFP4 checkpoints (gpt-oss) need `interp-engine[quant]`**, which the `[vllm]` extra does not include; without it transformers dequantizes them to bf16 at roughly 3x the weights, which can turn a model that fits into one that does not.
 
 Per-architecture structural quirks — which points a family serves and why — are not caveats but facts about the architecture, and live in [ARCHITECTURE_QUIRKS.md](docs/ARCHITECTURE_QUIRKS.md).

{interp_engine-1.5.0 → interp_engine-1.6.0}/README.md

@@ -82,21 +82,21 @@ One stream (tok/s):
 
 | model                    | eager | vLLM       | vLLM + static taps |
 | ------------------------ | ----- | ---------- | ------------------ |
-| `gemma-2-2b`             | 31    | 31 (1.0x)  | **214 (6.9x)**     |
-| `qwen3-4b`               | 24    | 47 (2.0x)  | **296 (12.3x)**    |
-| `llama-3.1-8b`           | 33    | 57 (1.7x)  | **256 (7.9x)**     |
-| `qwen3.8-27b`            | 9.9   | 12 (1.2x)  | **63 (6.4x)**      |
-| `deepseek-v4-flash-0731` | 3.3   | 2.9 (0.9x) | **119 (36x)**      |
+| `gemma-2-2b`             | 75    | 88 (1.2x)  | **220 (2.9x)**     |
+| `qwen3-4b`               | 60    | 136 (2.3x) | **313 (5.2x)**     |
+| `llama-3.1-8b`           | 84    | 163 (1.9x) | **258 (3.1x)**     |
+| `qwen3.8-27b`            | 25    | 35 (1.4x)  | **63 (2.5x)**      |
+| `deepseek-v4-flash-0731` | 5.3   | 15 (2.9x)  | **132 (25x)**      |
 
 8 concurrent requests (aggregate tok/s):
 
-| model                    | eager | vLLM        | vLLM + static taps |
-| ------------------------ | ----- | ----------- | ------------------ |
-| `gemma-2-2b`             | 30    | 226 (7.5x)  | **1,238 (41x)**    |
-| `qwen3-4b`               | 24    | 333 (14.0x) | **1,018 (43x)**    |
-| `llama-3.1-8b`           | 32    | 419 (13.0x) | **1,536 (48x)**    |
-| `qwen3.8-27b`            | 9.5   | 87 (9.2x)   | **386 (41x)**      |
-| `deepseek-v4-flash-0731` | 3.2   | 23 (7.2x)   | **402 (127x)**     |
+| model                    | eager | vLLM          | vLLM + static taps |
+| ------------------------ | ----- | ------------- | ------------------ |
+| `gemma-2-2b`             | 75    | 675 (9.0x)    | **1,436 (19.2x)**  |
+| `qwen3-4b`               | 61    | 973 (15.9x)   | **2,219 (36x)**    |
+| `llama-3.1-8b`           | 81    | 1,187 (14.6x) | **1,755 (22x)**    |
+| `qwen3.8-27b`            | 24    | 246 (10.3x)   | **408 (17.0x)**    |
+| `deepseek-v4-flash-0731` | 5.4   | 117 (22x)     | **631 (116x)**     |
 
 <!-- THROUGHPUT:END -->
 
@@ -117,7 +117,7 @@ We verify correctness in two main ways:
 
 [docs](https://interp-engine.org/docs/gpu-sizer) | [API](https://interp-engine.org/docs/gpu-sizer-api)
 
-Never OOM again - `interp-engine` includes gpu-sizer, an intuitive UI which tells you what GPU(s) and configs you need to get the best performance out of your selected model. For example, [interp-engine.org/sizer/Qwen/Qwen3.6-27B](https://interp-engine.org/sizer/Qwen/Qwen3.6-27B) shows that you can run `Qwen3.6-27B` with interp-engine's vllm-static backend for max speed while keeping it in a single 80GB A100, and have ~40k tokens for the KV Cache.
+Never OOM again - `interp-engine` includes gpu-sizer, an intuitive UI which tells you what GPU(s) and configs you need to get the best performance out of your selected model. For example, [interp-engine.org/sizer/Qwen/Qwen3.6-27B](https://interp-engine.org/sizer/Qwen/Qwen3.6-27B) shows that you can run `Qwen3.6-27B` with interp-engine's vllm-static backend for max speed while keeping it in a single 80GB A100, and have ~160k tokens for the KV Cache.
 
 ## Why use an Interpretability Engine instead of building from scratch?
 
@@ -129,6 +129,7 @@ Never OOM again - `interp-engine` includes gpu-sizer, an intuitive UI which tell
 
 - **Gemma 4 requires transformers 5.14.1** because 5.15 moved `head_dim` into `per_layer_config` and vLLM's config read dies before a weight loads ([vllm#51744](https://github.com/vllm-project/vllm/issues/51744)).
 - **DeepSeek-V2 on transformers older than 5.15.0** captures a wrong attention temperature — the engine warns at load, and upgrading is the fix ([COMPATIBILITY.md](docs/COMPATIBILITY.md)).
+- **DeepSeek-V4 on the eager backend needs two flags outside Hopper.** Its FP8 paths reach for `kernels-community/deep-gemm`, which declares a `9.0a` build, and with `kernels` >= 0.16.1 the arch refusal escapes transformers' Triton fallback and kills the first forward. Load with `TRANSFORMERS_DISABLE_DEEPGEMM_LINEAR=1` and `experts_implementation="grouped_mm"`, which transformers [already recommends on B200](https://github.com/huggingface/transformers/blob/93c8b7b485963a10800c91f55304db6be211c2bd/src/transformers/integrations/finegrained_fp8.py#L252-L254) for an unrelated DeepGEMM accuracy problem. The engine raises `HubKernelUnsupported` naming both; vLLM is unaffected.
 - **MXFP4 checkpoints (gpt-oss) need `interp-engine[quant]`**, which the `[vllm]` extra does not include; without it transformers dequantizes them to bf16 at roughly 3x the weights, which can turn a model that fits into one that does not.
 
 Per-architecture structural quirks — which points a family serves and why — are not caveats but facts about the architecture, and live in [ARCHITECTURE_QUIRKS.md](docs/ARCHITECTURE_QUIRKS.md).

{interp_engine-1.5.0 → interp_engine-1.6.0}/benchmarks/report_bench.py

@@ -123,6 +123,19 @@ def _columns(cells: list[dict[str, Any]]) -> list[str]:
     return [k for k in COLUMNS if k in present] + [k for k in present if k not in COLUMNS]
 
 
+def _rendered(cells: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    """The cells this report actually shows, which is what its header has to be about.
+
+    Dropping :data:`EXCLUDED` from the columns was not enough: the cell count, the run's date range
+    and the environment table were all still computed over everything on disk, so a dspark cell left
+    over from an older stack described the whole report. It really happened -- a table whose figures
+    were all measured on vLLM 0.28 carried an environment block reading 0.26, because the stalest
+    cell in the directory sorted first and the block takes the first one it is given.
+    """
+    shown = set(_columns(cells))
+    return [c for c in cells if c["variant"]["key"] in shown]
+
+
 def _variant_specs(cells: list[dict[str, Any]]) -> list[Any]:
     """The specs behind :func:`_columns`, in the same order, for the report's variant table."""
     by_key = {v.key: v for v in VARIANTS}
@@ -350,6 +363,9 @@ def _steering_overhead(cells: list[dict[str, Any]]) -> list[str]:
 
 
 def build_report(cells: list[dict[str, Any]], sweep_command: str) -> str:
+    if not cells:
+        return "# interp-engine speed benchmarks\n\nNo results found.\n"
+    cells = _rendered(cells)
     if not cells:
         return "# interp-engine speed benchmarks\n\nNo results found.\n"
 

{interp_engine-1.5.0 → interp_engine-1.6.0}/benchmarks/results/deepseek-v4-flash-0731__eager.json

@@ -1,6 +1,6 @@
 {
   "schema": 2,
-  "started_at": "2026-08-19T16:17:03+00:00",
+  "started_at": "2026-09-04T08:04:10+00:00",
   "command": "python -m benchmarks.run_bench --model deepseek-v4-flash-0731 --variant eager",
   "python": "python",
   "model": {
@@ -11,12 +11,26 @@
     "dtype": "bfloat16",
     "min_gpu_gib": 170.0,
     "capture_point": "mlp_out",
+    "per_variant_vllm_kwargs": {
+      "vllm-static": {
+        "max_num_batched_tokens": 1024,
+        "compilation_config": {
+          "cudagraph_capture_sizes": [
+            1,
+            2,
+            4,
+            8,
+            16,
+            32
+          ]
+        }
+      }
+    },
+    "static_capture_point": "resid_streams",
     "extra_eager_kwargs": {
       "device_map": "cuda"
     },
-    "extra_vllm_kwargs": {
-      "kv_cache_dtype": "fp8"
-    },
+    "extra_vllm_kwargs": {},
     "gpu_memory_utilization": 0.95,
     "native_dtype": "bfloat16"
   },
@@ -25,20 +39,21 @@
     "backend": "eager",
     "note": "raw HF forward; attn_implementation=eager, the engine's default",
     "kwargs": {
-      "device_map": "cuda"
+      "device_map": "cuda",
+      "model_kwargs": "{'experts_implementation': 'grouped_mm'}"
     }
   },
   "env": {
     "gpu_name": "NVIDIA B200",
-    "gpu_total_gib": 178.3511962890625,
-    "driver_version": "580.105.08",
+    "gpu_total_gib": 178.35107421875,
+    "driver_version": "580.126.20",
     "cuda_version": "13.0",
-    "torch_version": "2.11.0+cu130",
-    "vllm_version": "0.26.0",
-    "transformers_version": "5.14.1",
-    "interp_engine_version": "1.2.0",
-    "python_version": "3.12.3",
-    "platform": "linux 6.8.0-90-generic",
+    "torch_version": "2.13.0+cu130",
+    "vllm_version": "0.28.0",
+    "transformers_version": "5.16.1",
+    "interp_engine_version": "1.6.0",
+    "python_version": "3.13.8",
+    "platform": "linux 6.8.0-100-generic",
     "extra": {
       "VLLM_DEEP_GEMM_WARMUP": "skip"
     }
@@ -48,10 +63,10 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "ttft_ms": 447.032670956105,
-        "prefill_tok_s": 1145.330158766571,
-        "decode_tok_s": 3.2860430257568205,
-        "total_s": 39.095339769963175
+        "ttft_ms": 348.0888856574893,
+        "prefill_tok_s": 1470.8886755545395,
+        "decode_tok_s": 5.334557839162081,
+        "total_s": 24.1551229134202
       },
       "detail": {
         "prompt_tokens": 512,
@@ -63,9 +78,9 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "total_s": 324.3678773867432,
-        "aggregate_tok_s": 3.1569093963613626,
-        "per_request_s": 40.5459846733429
+        "total_s": 188.6664756303653,
+        "aggregate_tok_s": 5.427567333192873,
+        "per_request_s": 23.583309453795664
       },
       "detail": {
         "concurrency": 8,
@@ -78,8 +93,8 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 438.4800470434129,
-        "prompt_tok_s": 1167.6700079110053,
+        "latency_ms": 322.7837970480323,
+        "prompt_tok_s": 1586.2010568139242,
         "transported_mib": 4.0
       },
       "detail": {
@@ -95,8 +110,8 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 527.5651426054537,
-        "prompt_tok_s": 970.4962641607004,
+        "latency_ms": 371.26773688942194,
+        "prompt_tok_s": 1379.0586930328764,
         "transported_mib": 172.0
       },
       "detail": {
@@ -112,7 +127,7 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 10090.246555861086,
+        "latency_ms": 6102.334971539676,
         "transported_mib": 1.2421875
       },
       "detail": {
@@ -130,7 +145,7 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 10067.315930966288,
+        "latency_ms": 6089.949840679765,
         "transported_mib": 1.2421875
       },
       "detail": {
@@ -148,8 +163,8 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 2.645158674567938,
-        "rows_per_s": 193561.16701907513,
+        "latency_ms": 1.2944107875227928,
+        "rows_per_s": 395546.76532003516,
         "result_mib": 0.048828125
       },
       "detail": {
@@ -161,9 +176,9 @@
     }
   },
   "load": {
-    "construct_s": 74.84105253126472,
-    "warmup_s": 2.330029383301735e-05,
-    "ready_s": 74.84107583155856,
+    "construct_s": 19.825129061937332,
+    "warmup_s": 6.800517439842224e-06,
+    "ready_s": 19.825135862454772,
     "n_layers": 43,
     "d_model": 4096,
     "device": "cuda:0",
@@ -171,5 +186,5 @@
     "resolved_dtype": "bfloat16",
     "grad_through_forward": false
   },
-  "finished_at": "2026-08-19T16:34:07+00:00"
+  "finished_at": "2026-09-04T08:15:23+00:00"
 }

{interp_engine-1.5.0 → interp_engine-1.6.0}/benchmarks/results/deepseek-v4-flash-0731__vllm-cudagraph.json

@@ -1,6 +1,6 @@
 {
   "schema": 2,
-  "started_at": "2026-08-19T14:49:37+00:00",
+  "started_at": "2026-09-04T08:24:40+00:00",
   "command": "python -m benchmarks.run_bench --model deepseek-v4-flash-0731 --variant vllm-cudagraph",
   "python": "python",
   "model": {
@@ -11,12 +11,26 @@
     "dtype": "bfloat16",
     "min_gpu_gib": 170.0,
     "capture_point": "mlp_out",
+    "per_variant_vllm_kwargs": {
+      "vllm-static": {
+        "max_num_batched_tokens": 1024,
+        "compilation_config": {
+          "cudagraph_capture_sizes": [
+            1,
+            2,
+            4,
+            8,
+            16,
+            32
+          ]
+        }
+      }
+    },
+    "static_capture_point": "resid_streams",
     "extra_eager_kwargs": {
       "device_map": "cuda"
     },
-    "extra_vllm_kwargs": {
-      "kv_cache_dtype": "fp8"
-    },
+    "extra_vllm_kwargs": {},
     "gpu_memory_utilization": 0.95,
     "native_dtype": "bfloat16"
   },
@@ -27,20 +41,20 @@
     "kwargs": {
       "max_model_len": "2048",
       "gpu_memory_utilization": "0.95",
-      "extra_vllm_kwargs": "{'kv_cache_dtype': 'fp8', 'enable_prefix_caching': False}"
+      "extra_vllm_kwargs": "{'enable_prefix_caching': False}"
     }
   },
   "env": {
     "gpu_name": "NVIDIA B200",
-    "gpu_total_gib": 178.3511962890625,
-    "driver_version": "580.105.08",
+    "gpu_total_gib": 178.35107421875,
+    "driver_version": "580.126.20",
     "cuda_version": "13.0",
-    "torch_version": "2.11.0+cu130",
-    "vllm_version": "0.26.0",
-    "transformers_version": "5.14.1",
-    "interp_engine_version": "1.2.0",
-    "python_version": "3.12.3",
-    "platform": "linux 6.8.0-90-generic",
+    "torch_version": "2.13.0+cu130",
+    "vllm_version": "0.28.0",
+    "transformers_version": "5.16.1",
+    "interp_engine_version": "1.6.0",
+    "python_version": "3.13.8",
+    "platform": "linux 6.8.0-100-generic",
     "extra": {
       "VLLM_DEEP_GEMM_WARMUP": "skip"
     }
@@ -50,10 +64,10 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "ttft_ms": 390.6881511211395,
-        "prefill_tok_s": 1310.5081342516723,
-        "decode_tok_s": 151.38788310571746,
-        "total_s": 1.2295928071253002
+        "ttft_ms": 76.75724849104881,
+        "prefill_tok_s": 6670.379802107,
+        "decode_tok_s": 152.09943506242846,
+        "total_s": 0.9117373386397958
       },
       "detail": {
         "prompt_tokens": 512,
@@ -65,9 +79,9 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "total_s": 2.3538868555333465,
-        "aggregate_tok_s": 435.0251574721423,
-        "per_request_s": 0.2942358569416683
+        "total_s": 1.3893375098705292,
+        "aggregate_tok_s": 737.0419302185438,
+        "per_request_s": 0.17366718873381615
       },
       "detail": {
         "concurrency": 8,
@@ -104,8 +118,8 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 107.80607210472226,
-        "rows_per_s": 4749.268663667162,
+        "latency_ms": 26.252491399645805,
+        "rows_per_s": 19502.910874466863,
         "result_mib": 0.05859375
       },
       "detail": {
@@ -113,15 +127,15 @@
         "top_n": 10,
         "path": "worker-side top-k (decode_residuals_topk)",
         "repeats": 3,
-        "topk_id_overlap_vs_full": 0.9848,
+        "topk_id_overlap_vs_full": 0.9816,
         "topk_worst_shortfall_frac_of_range": 0.0
       }
     }
   },
   "load": {
-    "construct_s": 1.623564304318279,
-    "warmup_s": 1416.6046979688108,
-    "ready_s": 1418.228262273129,
+    "construct_s": 1.4092194680124521,
+    "warmup_s": 215.26768506783992,
+    "ready_s": 216.67690453585237,
     "n_layers": 43,
     "d_model": 4096,
     "device": "cuda (vllm worker)",
@@ -129,5 +143,5 @@
     "resolved_dtype": "bfloat16",
     "grad_through_forward": false
   },
-  "finished_at": "2026-08-19T15:13:34+00:00"
+  "finished_at": "2026-09-04T08:28:27+00:00"
 }

{interp_engine-1.5.0 → interp_engine-1.6.0}/benchmarks/results/deepseek-v4-flash-0731__vllm-static.json

@@ -1,6 +1,6 @@
 {
   "schema": 2,
-  "started_at": "2026-08-19T19:27:24+00:00",
+  "started_at": "2026-09-04T07:18:04+00:00",
   "command": "python -m benchmarks.run_bench --model deepseek-v4-flash-0731 --variant vllm-static",
   "python": "python",
   "model": {
@@ -30,9 +30,7 @@
     "extra_eager_kwargs": {
       "device_map": "cuda"
     },
-    "extra_vllm_kwargs": {
-      "kv_cache_dtype": "fp8"
-    },
+    "extra_vllm_kwargs": {},
     "gpu_memory_utilization": 0.95,
     "native_dtype": "bfloat16"
   },
@@ -45,20 +43,20 @@
       "static_writes": "[('resid_streams', 21)]",
       "max_model_len": "2048",
       "gpu_memory_utilization": "0.95",
-      "extra_vllm_kwargs": "{'kv_cache_dtype': 'fp8', 'max_num_batched_tokens': 1024, 'compilation_config': {'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 32]}, 'enable_prefix_caching': False}"
+      "extra_vllm_kwargs": "{'max_num_batched_tokens': 1024, 'compilation_config': {'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 32]}, 'enable_prefix_caching': False}"
     }
   },
   "env": {
     "gpu_name": "NVIDIA B200",
-    "gpu_total_gib": 178.3511962890625,
-    "driver_version": "580.105.08",
+    "gpu_total_gib": 178.35107421875,
+    "driver_version": "580.126.20",
     "cuda_version": "13.0",
-    "torch_version": "2.11.0+cu130",
-    "vllm_version": "0.26.0",
-    "transformers_version": "5.14.1",
-    "interp_engine_version": "1.2.0",
-    "python_version": "3.12.3",
-    "platform": "linux 6.8.0-90-generic",
+    "torch_version": "2.13.0+cu130",
+    "vllm_version": "0.28.0",
+    "transformers_version": "5.16.1",
+    "interp_engine_version": "1.6.0",
+    "python_version": "3.13.8",
+    "platform": "linux 6.8.0-100-generic",
     "extra": {
       "VLLM_DEEP_GEMM_WARMUP": "skip"
     }
@@ -68,10 +66,10 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "ttft_ms": 318.4659071266651,
-        "prefill_tok_s": 1607.707414019547,
-        "decode_tok_s": 118.92239360242137,
-        "total_s": 1.3863892490044236
+        "ttft_ms": 75.92793833464384,
+        "prefill_tok_s": 6743.235905384625,
+        "decode_tok_s": 131.86411564042513,
+        "total_s": 1.0390406046062708
       },
       "detail": {
         "prompt_tokens": 512,
@@ -83,14 +81,14 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "total_s": 2.5435977161396295,
-        "aggregate_tok_s": 402.1862393997538,
-        "per_request_s": 0.3179497145174537
+        "total_s": 1.6222849520854652,
+        "aggregate_tok_s": 631.2084684528675,
+        "per_request_s": 0.20278561901068315
       },
       "detail": {
         "concurrency": 8,
         "prompt_tokens": 512,
-        "generated_tokens_total": 1023,
+        "generated_tokens_total": 1024,
         "repeats": 2
       }
     },
@@ -98,8 +96,8 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 484.4155637547374,
-        "prompt_tok_s": 1056.943744811694,
+        "latency_ms": 131.99533242732286,
+        "prompt_tok_s": 3878.9250391252217,
         "transported_mib": 32.0
       },
       "detail": {
@@ -115,8 +113,8 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 5581.670379731804,
-        "prompt_tok_s": 91.728813270518,
+        "latency_ms": 2049.1460720077157,
+        "prompt_tok_s": 249.86017687765508,
         "transported_mib": 1376.0
       },
       "detail": {
@@ -132,7 +130,7 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 604.7434308566153,
+        "latency_ms": 336.8926215916872,
         "transported_mib": 9.9375
       },
       "detail": {
@@ -150,7 +148,7 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 631.2098908238113,
+        "latency_ms": 324.6788289397955,
         "transported_mib": 9.9375
       },
       "detail": {
@@ -168,8 +166,8 @@
       "status": "ok",
       "reason": "",
       "metrics": {
-        "latency_ms": 97.82497305423021,
-        "rows_per_s": 5233.837373164089,
+        "latency_ms": 24.669302627444267,
+        "rows_per_s": 20754.538858768017,
         "result_mib": 0.05859375
       },
       "detail": {
@@ -177,15 +175,15 @@
         "top_n": 10,
         "path": "worker-side top-k (decode_residuals_topk)",
         "repeats": 3,
-        "topk_id_overlap_vs_full": 0.9848,
+        "topk_id_overlap_vs_full": 0.9846,
         "topk_worst_shortfall_frac_of_range": 0.0
       }
     }
   },
   "load": {
-    "construct_s": 1.8341202763840556,
-    "warmup_s": 1073.8554912502877,
-    "ready_s": 1075.6896115266718,
+    "construct_s": 1.3641586303710938,
+    "warmup_s": 126.34760607592762,
+    "ready_s": 127.71176470629871,
     "n_layers": 43,
     "d_model": 4096,
     "device": "cuda (vllm worker)",
@@ -193,5 +191,5 @@
     "resolved_dtype": "bfloat16",
     "grad_through_forward": false
   },
-  "finished_at": "2026-08-19T19:46:07+00:00"
+  "finished_at": "2026-09-04T07:20:35+00:00"
 }