interp-engine 1.11.0__tar.gz → 1.11.2__tar.gz

{interp_engine-1.11.0 → interp_engine-1.11.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: interp-engine
-Version: 1.11.0
+Version: 1.11.2
 Summary: A fast, standardized, and easy to use interpretability engine.
 Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
 Project-URL: Repository, https://github.com/decoderesearch/interp-engine
@@ -21,7 +21,7 @@ Requires-Dist: pytest<9,>=8.3.1; extra == 'dev'
 Requires-Dist: pyyaml>=6; extra == 'dev'
 Requires-Dist: ruff<0.17,>=0.16.2; extra == 'dev'
 Provides-Extra: parity
-Requires-Dist: transformer-lens>=3.0; extra == 'parity'
+Requires-Dist: transformer-lens<4,>=3.0; extra == 'parity'
 Provides-Extra: quant
 Requires-Dist: accelerate>=1.0; extra == 'quant'
 Requires-Dist: kernels<0.17.0,>=0.15.2; extra == 'quant'

{interp_engine-1.11.0 → interp_engine-1.11.2}/docs/ENGINE_HOOK_MAPPINGS.md

@@ -24,7 +24,7 @@ that capture is conditional. Each links to the note under the table.
 | -------------------------------------------- | --------------------------------------------------- | ----------------------------------------------- | ------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
 | `embeddings`                                 | `cache["embeddings"]`                               | `hook_embed`                                    | `token_embeddings`                                            | [three tensors, three conventions](#embeddings-three-different-tensors)                                   |
 | `resid_pre`                                  | `cache["resid_pre", 5]`                             | `blocks.5.hook_resid_pre`                       | `layers_input[5]`                                             |                                                                                                           |
-| `attn_in`                                    | `cache["attn_in", 5]`                               | `blocks.5.hook_attn_in` (or `attn.hook_in`)     | `attentions_input[5]`                                         | [what the sublayer was handed, normed or not](#attn_in-is-the-attention-modules-input-not-a-norms-output) |
+| `attn_in`                                    | `cache["attn_in", 5]`                               | `blocks.5.attn.hook_in` (**not** `hook_attn_in`) | `attentions_input[5]`                                         | [the norm's output; the block-level hook is its input](#the-block-level-hook_attn_in-and-hook_mlp_in-are-the-norms-input) |
 | `q_norm_in` / `k_norm_in` (QK-norm input)    | `cache["q_norm_in", 5]`                             | `blocks.5.attn.q_norm.hook_in` (bridge only)    | —                                                             | [shape is family-dependent](#qk-norm-shape-is-family-dependent)                                           |
 | `q_norm_out` / `k_norm_out` (QK-norm output) | `cache["q_norm_out", 5]`                            | `blocks.5.attn.q_norm.hook_out` (bridge only)   | —                                                             | [shape is family-dependent](#qk-norm-shape-is-family-dependent)                                           |
 | `value`                                      | `cache["value", 5]`                                 | `blocks.5.attn.hook_v`                          | —                                                             | [KV heads, and flattened](#addressing-and-shape-which-do-not-translate-for-free)                          |
@@ -35,7 +35,7 @@ that capture is conditional. Each links to the note under the table.
 | `attn_out` (raw module output)               | `cache["attn_out", 5]`                              | `blocks.5.attn.hook_out`                        | `attentions_output[5]`                                        | [raw output, not the contribution](#raw-output-versus-residual-contribution)                              |
 | `attn_out_post` (residual contribution)      | `cache["attn_out_post", 5]`                         | `blocks.5.hook_attn_out`                        | —                                                             | [same TL name, different tensor](#raw-output-versus-residual-contribution)                                |
 | `resid_mid`                                  | `cache["resid_mid", 5]` (writes refused under vLLM) | `blocks.5.hook_resid_mid`                       | _no accessor_: the pre-MLP norm's `.input`, under its HF name | [captured, not reconstructed](#resid_mid-is-captured-not-reconstructed)                                   |
-| `mlp_in`                                     | `cache["mlp_in", 5]`                                | `blocks.5.mlp.hook_in` (or `hook_mlp_in`)       | `mlps_input[5]`                                               | [not `ln2.hook_normalized`, which is pre-gain](#a-norms-hook_normalized-is-not-the-sublayer-input)        |
+| `mlp_in`                                     | `cache["mlp_in", 5]`                                | `blocks.5.mlp.hook_in` (**not** `hook_mlp_in`)   | `mlps_input[5]`                                               | [not `hook_mlp_in`, which is pre-norm](#the-block-level-hook_attn_in-and-hook_mlp_in-are-the-norms-input) |
 | `mlp_pre` (pre-activation)                   | `cache["mlp_pre", 5]` `*`                           | `blocks.5.mlp.hook_pre`                         | —                                                             | [the neuron basis, and `up_proj` is ambiguous](#the-mlp-internal-points-are-the-neuron-basis)             |
 | `mlp_pre_linear` (gated MLPs only)           | `cache["mlp_pre_linear", 5]` `*`                    | `blocks.5.mlp.hook_pre_linear`                  | —                                                             | [refuses on a plain MLP](#the-mlp-internal-points-are-the-neuron-basis)                                   |
 | `mlp_act` (post-activation neurons)          | `cache["mlp_act", 5]`                               | `blocks.5.mlp.hook_post`                        | —                                                             | [`d_mlp` wide; refuses on sparse layers](#the-mlp-internal-points-are-the-neuron-basis)                   |
@@ -45,7 +45,7 @@ that capture is conditional. Each links to the note under the table.
 | `mlp_out` (raw module output)                | `cache["mlp_out", 5]`                               | `blocks.5.mlp.hook_out`                         | `mlps_output[5]`                                              | [raw output, not the contribution](#raw-output-versus-residual-contribution)                              |
 | `mlp_out_post` (residual contribution)       | `cache["mlp_out_post", 5]`                          | `blocks.5.hook_mlp_out`                         | —                                                             | [same TL name, different tensor](#raw-output-versus-residual-contribution)                                |
 | `resid_post`                                 | `cache["resid_post", 5]`                            | `blocks.5.hook_resid_post`                      | `layers_output[5]`                                            |                                                                                                           |
-| `final_norm`                                 | `cache["final_norm"]`                               | `ln_final.hook_normalized`                      | `ln_final.output`                                             |                                                                                                           |
+| `final_norm`                                 | `cache["final_norm"]`                               | `unembed.hook_in` (TransformerLens 3)           | `ln_final.output`                                             |                                                                                                           |
 | `lm_head`                                    | `cache["lm_head"]` `*`                              | _(logits are returned, not hooked)_             | `logits`                                                      | [a bare unembed, which vLLM does not offer](#the-unembed-and-what-vllm-returns)                           |
 
 A hyper-connection trunk adds seven more points, which are conditional on the family and so are
@@ -147,6 +147,33 @@ Scope's transcoders declare it as their SAELens `hook_name`, circuit-tracer's as
 Substituting `mlp_in` reads them off the wrong activations. Same reasoning one level down for
 `q_norm.hook_normalized`, in the table at the end.
 
+### The block-level `hook_attn_in` and `hook_mlp_in` are the norm's input
+
+TransformerLens has two names each for the sublayer inputs, and they are a whole normalization apart.
+`blocks.5.mlp.hook_in` is the MLP module's argument: `ln2`'s output, gain included, which is our
+`mlp_in`. `blocks.5.hook_mlp_in` is the block-level hook, and `TransformerBlock.forward` fires it on
+`resid_mid` and passes the result _into_ `ln2` — so it is the norm's **input**, `resid_mid` under
+another name. `hook_attn_in` is the same story on `resid_pre`, broadcast per head to
+`[batch, pos, n_heads, d_model]`. Measured on the HF modules with neither library in the loop: on
+gemma-3-1b-it the norm's input and output share cosine 0.175 at layer 5 and differ in norm by 231x,
+past 3000x deeper in the stack; on gpt2 the cosine is 0.75 and the norms differ by 27–86x.
+
+`tlens_hook_to_point` therefore **refuses** `hook_attn_in` / `hook_mlp_in` rather than mapping them to
+either side. Mapping them to `attn_in` / `mlp_in` was the bug (issue #24): a caller got a plausible
+tensor of the right shape, and an SAE or steering vector trained on TransformerLens' tensor was read
+off the wrong one. Mapping them to `resid_pre` / `resid_mid` would be faithful to what the hook carries
+and still wrong to do silently, because the caller porting `use_hook_mlp_in=True` code may have meant
+either side of the norm. The refusal names both: `mlp.hook_in` for the sublayer's argument,
+`hook_resid_mid` for the pre-norm residual. `point_to_tlens_hook("attn_in")` emits `attn.hook_in`, the
+name that denotes the tensor.
+
+Two more names outside this table map (issue #25). `blocks.5.hook_in`, the block's forward pre-hook,
+is `resid_pre` on a single-stream trunk — and is refused on a hyper-connection one, where it is the
+_previous_ block's `resid_streams`, for the reason `attn_hc.hook_in` is. `unembed.hook_in` is the only
+mapped name outside `blocks.{i}.`: the unembedding's input is `final_norm`, the final norm's output.
+`ln_final.hook_normalized` is not that tensor — it fires pre-gain, and a `fold_ln` `HookedTransformer`
+has folded that gain into `W_U` — so it is refused with the reason.
+
 ### `attn_in` is the attention module's input, not a norm's output
 
 Which matters on a post-norm family. On a Llama-shaped block the two coincide — the attention reads
@@ -553,7 +580,7 @@ path captures that module's **output** (`cache["model.layers.5.self_attn.q_proj"
 | `blocks.5.attn.hook_rot_q`, `.hook_rot_k`                           | q/k after RoPE                                                  | No such tensor crosses a module boundary: `transformers` applies RoPE inside the attention forward                                                                                                                                                                                                                                                                      |
 | `blocks.5.attn.hook_result`                                         | per-head output _after_ `W_O`                                   | It is `n_heads` times the size of `z`, which is why TL gates it behind a flag too. Computed on demand instead: `capture.head_contributions(model, cache, 5)` applies the split `W_O` to a captured `z`                                                                                                                                                                  |
 | `blocks.5.hook_q_input`, `.hook_k_input`, `.hook_v_input`           | per-head writable inputs (`use_split_qkv_input`)                | A TransformerLens parameterization rather than a place in the HF forward — the tensor exists only because TL splits the input per head to make it writable                                                                                                                                                                                                              |
-| `blocks.5.hook_attn_in`, `blocks.5.hook_mlp_in`                     | writable sublayer inputs (`use_attn_in`)                        | Already named: `attn_in` / `mlp_in` are the same tensors, and writing to them needs no separate hook here                                                                                                                                                                                                                                                               |
+| `blocks.5.hook_attn_in`, `blocks.5.hook_mlp_in`                     | the block-level sublayer inputs (`use_attn_in`, `use_hook_mlp_in`) | **Not** `attn_in` / `mlp_in`, despite the names: TransformerLens fires them on `resid_pre` / `resid_mid` _before_ the block norm, so they are the norm's input where our points are its output. Refused with both readings named; ask for `attn.hook_in` / `mlp.hook_in` (the sublayer's argument) or `hook_resid_pre` / `hook_resid_mid` (the tensor these hooks carry). [Below](#the-block-level-hook_attn_in-and-hook_mlp_in-are-the-norms-input) |
 | `blocks.5.mlp.hook_mid`                                             | post-activation, pre-LayerNorm, on a SoLU model                 | No HF checkpoint in scope has an activation-internal norm, so the point would never resolve                                                                                                                                                                                                                                                                             |
 | `blocks.5.mlp.hook_gate`                                            | one expert's SwiGLU gate, at `blocks.5.mlp.experts.3.hook_gate` | **Not the router**, despite the name, so mapping it to anything routing-shaped would be wrong. Our `mlp_pre` is the same tensor on a dense MLP; per expert it is not a module output at all on the families that fuse their expert banks                                                                                                                                |
 | `blocks.5.mlp.hook_expert_weights`                                  | the softmax over _all_ experts, pre-top-k                       | It is a different tensor from our `expert_weights` (post-top-k, `k` wide), and mapping two different tensors to one name is the failure this table exists to prevent. Write it as `softmax(cache["router_logits", 5])`, exact for the softmax-routed families                                                                                                           |

{interp_engine-1.11.0 → interp_engine-1.11.2}/docs/USAGE.md

@@ -204,7 +204,44 @@ async def generate():
 ```
 
 The stream yields decoded text deltas that concatenate to exactly what `generate_text` would have
-returned. For a chat model, build the prompt with the tokenizer's own template rather than by hand.
+returned.
+
+### Sampling knobs
+
+Every generation method takes `temperature`, `top_k`, `top_p` and `presence_penalty`, and decides
+each one the same way on every backend: a knob you pass is used as passed; a knob you leave `None`
+takes the checkpoint's own `generation_config.json`; a knob neither states is neutral (temperature
+1, no filtering, no penalty). This is what `transformers.generate` and `vllm serve` do, so a
+checkpoint samples here the way its authors tuned it to, and `temperature=0.0` is greedy
+everywhere. To turn a recommended filter off, say so in `transformers`' terms: `top_k=0`,
+`top_p=1.0`.
+
+`model.sampling_settings(...)` answers what a call with those arguments runs with, and
+`model.recommended_sampling` is the file as read (`None` where it says nothing; Qwen3.5 ships no
+file at all, so everything is neutral there unless you pass it):
+
+```python
+from interp_engine import load_model
+
+model = load_model("google/gemma-3-270m-it")
+model.recommended_sampling  # temperature 1.0, top_k 64, top_p 0.95
+model.sampling_settings(temperature=0.3)  # SamplingSettings(temperature=0.3, top_k=64, top_p=0.95, presence_penalty=0.0)
+text = await model.generate_text(token_ids, max_tokens=64, temperature=0.3, presence_penalty=1.5)
+```
+
+The presence penalty is the one repetition control: a flat subtraction from the logit of every
+token the generation has produced so far, applied before the temperature and before a greedy
+argmax, as vLLM orders it. It breaks a loop without scaling with the loop's length, which is what a
+frequency penalty does and why that one punishes function words in a long reply. The file cannot
+state a presence penalty (it is not a `transformers` field), so unset means 0; a card that
+recommends one (Qwen3.5: 1.5) is for the caller to read. `transformers`' multiplicative
+`repetition_penalty` is read but never applied.
+
+Code that generates through `transformers.generate` directly gets the same decision with
+`hf_generate_kwargs(model.sampling_settings(...), prompt_len=n)`: every knob stated, the file's
+own `repetition_penalty` off, and the presence penalty as a logits processor.
+
+For a chat model, build the prompt with the tokenizer's own template rather than by hand.
 Both backends carry a `Tokenize` helper on `.tok` for this:
 
 ```python

{interp_engine-1.11.0 → interp_engine-1.11.2}/interp_engine/__init__.py

@@ -96,6 +96,16 @@ from interp_engine.residual_basis import (
     reduce_streams,
     vllm_residual_basis,
 )
+from interp_engine.sampling import (
+    PresencePenaltyProcessor,
+    RecommendedSampling,
+    SamplingSettings,
+    VLLMSamplingKwargs,
+    apply_presence_penalty,
+    hf_generate_kwargs,
+    read_recommended_sampling,
+    resolve_sampling,
+)
 from interp_engine.select import BackendSelection, select_backend
 from interp_engine.steer import (
     STEER_METHODS,
@@ -185,6 +195,15 @@ __all__ = [
     "HubKernelUnsupported",
     "deepgemm_fallback_kwargs",
     "InterpModel",
+    # Sampling: what the checkpoint recommends, and what a generation runs with.
+    "PresencePenaltyProcessor",
+    "RecommendedSampling",
+    "SamplingSettings",
+    "VLLMSamplingKwargs",
+    "apply_presence_penalty",
+    "hf_generate_kwargs",
+    "read_recommended_sampling",
+    "resolve_sampling",
     "VLLMModel",
     # That same surface without an event loop, for scripts and notebooks. The sync free
     # functions below dispatch through this, so most callers never name it.

{interp_engine-1.11.0 → interp_engine-1.11.2}/interp_engine/mappers.py

@@ -56,9 +56,10 @@ _TLENS_STABLE: dict[str, str] = {
     # and that multiply -- a third tensor, wrong by an elementwise factor (cosine 0.89 on gemma-2
     # layer 19) and one that real Gemma Scope transcoders are trained on, so substituting these
     # would read a shipping artifact off the wrong activations. See :func:`tlens_hook_to_point`.
-    "hook_mlp_in": "mlp_in",
+    #
+    # Nor the block-level `hook_mlp_in` / `hook_attn_in`, which fire BEFORE the norm runs -- the
+    # norm's input, a whole normalization away from these. Refused in `_TLENS_PRENORM_BLOCK_INPUTS`.
     "mlp.hook_in": "mlp_in",
-    "hook_attn_in": "attn_in",
     "attn.hook_in": "attn_in",
     # The RAW module output, as distinct from the block-level hook below.
     "mlp.hook_out": "mlp_out",
@@ -146,6 +147,56 @@ _TLENS_STREAM_DEPENDENT: dict[str, tuple[str, str]] = {
     "hook_out": ("resid_post", "resid_streams"),
 }
 
+# The block's own input, `hook_in`, is `resid_pre` on a single-stream trunk -- the mirror of
+# `hook_out` above, and a forward pre-hook on the block is where a bridge puts it. On a
+# hyper-connection trunk it is the stack ENTERING the block, which is the previous block's
+# `resid_streams` and at layer 0 the embedding stack, so it is refused there with the same reason
+# as `attn_hc.hook_in` rather than resolved to this layer's stack under a wrong index.
+_TLENS_BLOCK_INPUT = "hook_in"
+
+# The block-level sublayer inputs. TransformerLens fires `hook_mlp_in` on `resid_mid` and
+# `hook_attn_in` on `resid_pre` (per head, `[batch, pos, n_heads, d_model]`), BEFORE the block norm
+# runs. They look like `mlp_in`/`attn_in` and are not: those are the norm's OUTPUT, gain included,
+# per the criterion in `_TLENS_STABLE`. The two differ by a whole normalization -- cosine 0.175 and
+# a norm ratio of 231x at gemma-3-1b-it layer 5, past 3000x deeper in the stack; 27-86x on gpt2.
+#
+# Refused rather than resolved, for the reason `ln1/ln2.hook_normalized` are: substituting one for
+# the other reads a shipping artifact off the wrong activations, silently, because the result is a
+# plausible tensor of the right shape. Not remapped to the residual either -- a caller porting
+# TransformerLens code may mean either side of the norm, and only they can say which.
+_TLENS_PRENORM_BLOCK_INPUTS: dict[str, str] = {
+    "hook_mlp_in": (
+        "TransformerLens' block-level MLP input, fired on `resid_mid` BEFORE the block norm runs, so "
+        "it is the norm's input. The engine's `mlp_in` is the norm's output -- the tensor the MLP "
+        "receives. Ask for `mlp.hook_in` for the sublayer's argument, or `hook_resid_mid` for the "
+        "pre-norm residual this hook carries"
+    ),
+    "hook_attn_in": (
+        "TransformerLens' block-level attention input, fired on `resid_pre` BEFORE the block norm "
+        "runs (and broadcast per head), so it is the norm's input. The engine's `attn_in` is the "
+        "norm's output. Ask for `attn.hook_in` for the sublayer's argument, or `hook_resid_pre` for "
+        "the pre-norm residual this hook carries"
+    ),
+}
+
+# Hooks outside the `blocks.{i}.` namespace. `unembed.hook_in` is the unembedding's input, which is
+# the final norm's output, gain included -- a forward pre-hook on `lm_head` reads the same tensor.
+# TransformerLens 3 registers it on the bridge and on `HookedTransformer` alike.
+_TLENS_GLOBAL: dict[str, str] = {"unembed.hook_in": "final_norm"}
+_POINT_TO_TLENS_GLOBAL: dict[str, str] = {point: hook for hook, point in _TLENS_GLOBAL.items()}
+
+# `ln_final.hook_normalized` is the nearest legacy name for the same tensor and is not it, twice
+# over: the hook fires on `x / scale` before the gain, and `HookedTransformer` folds that gain into
+# `W_U` by default (`fold_ln`), so on a folded model the hook IS the unembedding's input while the
+# gain has moved into the weights. Neither reading is `final_norm`.
+_TLENS_GLOBAL_REFUSED: dict[str, str] = {
+    "ln_final.hook_normalized": (
+        "a tensor inside the final norm -- `x / scale`, before the gain (which a `fold_ln` "
+        "HookedTransformer has folded into `W_U`). It is neither the norm's input nor `final_norm`, "
+        "the norm's output; `unembed.hook_in` is the output's name on TransformerLens 3's bridge"
+    ),
+}
+
 # The two mHC hooks that are real tensors with no canonical point, refused by name so the reason
 # arrives instead of a list of near misses. Both are stream stacks, which is exactly why they are
 # dangerous: `resid_streams` has their shape, so a caller mapping by eye would take either.
@@ -176,7 +227,6 @@ UNMAPPED_TLENS: frozenset[str] = frozenset(
     {
         "attn_gate",  # gated attention output; TL models no family that has one
         "embeddings",  # TL's `hook_embed` is pre-positional/pre-scaling, a different tensor
-        "final_norm",  # `ln_final.hook_normalized` is outside the `blocks.{i}.` namespace
         "lm_head",  # TL returns logits rather than hooking the unembed
         "router_logits",  # TL hooks the softmax over all experts, not the logits
         "expert_weights",  # TL's `hook_expert_weights` is pre-top-k, so it is a different tensor
@@ -185,14 +235,14 @@ UNMAPPED_TLENS: frozenset[str] = frozenset(
 
 # Canonical point -> the TransformerLens name to emit, for the rest. `mlp_out_post`/`attn_out_post`
 # are ours alone as *names*, but TL reaches those tensors through its aliased block-level hook, which
-# is what this emits.
+# is what this emits. Per-layer points only; the global ones are in `_POINT_TO_TLENS_GLOBAL`.
 _POINT_TO_TLENS: dict[str, str] = {
     "resid_pre": "hook_resid_pre",
     "resid_post": "hook_resid_post",
     "resid_mid": "hook_resid_mid",
     "mlp_in": "mlp.hook_in",
     "mlp_out": "mlp.hook_out",
-    "attn_in": "hook_attn_in",
+    "attn_in": "attn.hook_in",
     "attn_out": "attn.hook_out",
     "mlp_out_post": "hook_mlp_out",
     "attn_out_post": "hook_attn_out",
@@ -330,7 +380,10 @@ def tlens_hook_to_point(hook_name: str, model: object | None = None) -> Address:
     conventional trunk and ``resid_streams`` on a hyper-connection one, because TransformerLens 3's
     DeepSeek-V4 bridge drops the ``hook_resid_post`` alias and returns the stream stack from that
     hook instead. Without ``model`` it resolves to ``resid_post``, the reading that is right almost
-    everywhere -- and wrong by a rank on DeepSeek-V4 and Motif 3.
+    everywhere -- and wrong by a rank on DeepSeek-V4 and Motif 3. Its mirror ``hook_in`` is
+    ``resid_pre`` on a conventional trunk and is *refused* on a hyper-connection one: the stack
+    entering block ``i`` is block ``i-1``'s ``resid_streams``, and resolving it under index ``i``
+    would name a real tensor one layer off.
 
     The two mHC hooks that carry a stream stack *in* are refused rather than mapped, with the reason:
     ``attn_hc.hook_in`` is the previous block's ``resid_streams`` and ``mlp_hc.hook_in`` is the
@@ -338,6 +391,12 @@ def tlens_hook_to_point(hook_name: str, model: object | None = None) -> Address:
     its shape. The mHC hooks that come *out* -- ``hook_post``, ``hook_comb``, ``hook_out`` at both
     the ``attn_hc`` and ``mlp_hc`` sites -- all map, and need no ``model``.
 
+    The block-level ``hook_mlp_in``/``hook_attn_in`` are refused, because they are not the sublayer
+    inputs their names suggest: TransformerLens fires them on ``resid_mid``/``resid_pre`` *before*
+    the block norm, so they are the norm's input where ``mlp_in``/``attn_in`` are its output. The
+    sublayer-scoped ``mlp.hook_in``/``attn.hook_in`` are the output and map. The refusal names both,
+    since a caller may mean either side of the norm.
+
     ``ln1.hook_normalized``/``ln2.hook_normalized`` are unmapped for a reason worth stating, because
     they look like the sublayer inputs: TransformerLens fires them on ``x / scale``, before the norm's
     gain, so they are neither the norm's input (``resid_pre``/``resid_mid``) nor its output
@@ -347,6 +406,10 @@ def tlens_hook_to_point(hook_name: str, model: object | None = None) -> Address:
     own drift, not the formula). Gemma Scope's transcoders are trained there, so this is a live
     distinction, not a pedantic one.
 
+    One name outside the ``blocks.{i}.`` namespace maps: ``unembed.hook_in`` is ``final_norm``,
+    the unembedding's input. ``ln_final.hook_normalized`` is refused for the ``hook_normalized``
+    reason above, plus one more -- a ``fold_ln`` HookedTransformer has moved the gain into ``W_U``.
+
     Two QK-norm hooks are intentionally unmapped rather than approximated. ``q_norm.hook_scale`` is
     the norm's own denominator, an intermediate of its arithmetic that no hook on the module can
     return -- recompute it from ``q_norm_in`` and ``facts.rms_norm_eps``, which reproduces TL's
@@ -361,10 +424,16 @@ def tlens_hook_to_point(hook_name: str, model: object | None = None) -> Address:
     ``softmax(cache["router_logits", layer])``, which is exact for the softmax-routed families and
     is *not* what gpt-oss or DeepSeek-V3 compute.
     """
+    if hook_name in _TLENS_GLOBAL:
+        return Address(_TLENS_GLOBAL[hook_name])
+    if hook_name in _TLENS_GLOBAL_REFUSED:
+        raise UnmappedHook(f"TransformerLens hook {hook_name!r} is {_TLENS_GLOBAL_REFUSED[hook_name]}.")
+
     match = _BLOCK_RE.match(hook_name)
     if not match:
         raise UnmappedHook(
-            f"Cannot parse a layer out of TransformerLens hook name {hook_name!r} (expected 'blocks.<layer>.<hook>')"
+            f"Cannot parse a layer out of TransformerLens hook name {hook_name!r} (expected "
+            f"'blocks.<layer>.<hook>', or one of {', '.join(sorted(_TLENS_GLOBAL))})"
         )
     layer, suffix = int(match.group(1)), match.group(2)
 
@@ -376,6 +445,15 @@ def tlens_hook_to_point(hook_name: str, model: object | None = None) -> Address:
     if suffix in _TLENS_STREAM_DEPENDENT:
         single, stacked = _TLENS_STREAM_DEPENDENT[suffix]
         return Address(stacked if model is not None and has_hyper_connections(model) else single, layer)
+    if suffix == _TLENS_BLOCK_INPUT:
+        if model is not None and has_hyper_connections(model):
+            raise UnmappedHook(
+                f"TransformerLens hook {hook_name!r} is {_TLENS_STREAM_STACK_INPUTS['attn_hc.hook_in']}."
+            )
+        return Address("resid_pre", layer)
+
+    if suffix in _TLENS_PRENORM_BLOCK_INPUTS:
+        raise UnmappedHook(f"TransformerLens hook {hook_name!r} is {_TLENS_PRENORM_BLOCK_INPUTS[suffix]}.")
 
     if suffix in _TLENS_STREAM_STACK_INPUTS:
         raise UnmappedHook(f"TransformerLens hook {hook_name!r} is {_TLENS_STREAM_STACK_INPUTS[suffix]}.")
@@ -388,7 +466,7 @@ def tlens_hook_to_point(hook_name: str, model: object | None = None) -> Address:
             f"and `capture.pre_gain_normalized` for the arithmetic."
         )
 
-    known = sorted({*_TLENS_STABLE, *_TLENS_CONTRIBUTION, *_TLENS_STREAM_DEPENDENT})
+    known = sorted({*_TLENS_STABLE, *_TLENS_CONTRIBUTION, *_TLENS_STREAM_DEPENDENT, _TLENS_BLOCK_INPUT})
     raise UnmappedHook(f"TransformerLens hook {hook_name!r} has no canonical point (known: {', '.join(known)})")
 
 
@@ -455,8 +533,14 @@ def point_to_tlens_hook(point: str | Address, layer: int | None = None) -> str:
 
     Takes an :class:`~interp_engine.address.Address` or the ``(point, layer)`` pair. An address
     carrying a coordinate TransformerLens cannot express raises rather than losing it -- see
-    :func:`_refuse_unmappable_coordinates`.
+    :func:`_refuse_unmappable_coordinates`. The one global point with a name, ``final_norm``, takes
+    no layer and emits ``unembed.hook_in``.
     """
+    name = point.name if isinstance(point, Address) else point
+    if name in _POINT_TO_TLENS_GLOBAL:
+        if (point.layer if isinstance(point, Address) else layer) is not None:
+            raise UnmappedHook(f"{name!r} is a global point and takes no layer")
+        return _POINT_TO_TLENS_GLOBAL[name]
     point, layer, address = _split(point, layer)
     _refuse_unmappable_coordinates(address, "TransformerLens", "blocks.{i}.{hook}")
     if point not in _POINT_TO_TLENS:
@@ -467,7 +551,7 @@ def point_to_tlens_hook(point: str | Address, layer: int | None = None) -> str:
             "is a canonical point that TransformerLens has no equivalent for" if known else "is not a canonical point"
         )
         raise UnmappedHook(
-            f"{point!r} {reason} (mappable: {', '.join(sorted(_POINT_TO_TLENS))})"
+            f"{point!r} {reason} (mappable: {', '.join(sorted({*_POINT_TO_TLENS, *_POINT_TO_TLENS_GLOBAL}))})"
             + (f"; ours alone: {', '.join(sorted(UNMAPPED_TLENS))}" if known else "")
         )
     return f"blocks.{layer}.{_POINT_TO_TLENS[point]}"

{interp_engine-1.11.0 → interp_engine-1.11.2}/interp_engine/model.py

@@ -44,6 +44,7 @@ from interp_engine.facts import factored_projection, text_config
 from interp_engine.points import PointSpec, Scope, known_names, point_spec, points_for
 from interp_engine.protocol import Completion
 from interp_engine.residual_basis import ResidualBasis, eager_residual_basis
+from interp_engine.sampling import RecommendedSampling, SamplingSettings, read_recommended_sampling, resolve_sampling
 from interp_engine.tokenize import Tokenize
 
 logger = logging.getLogger(__name__)
@@ -504,6 +505,7 @@ class EagerModel:
         self._requires_grad = requires_grad
         # Lazily computed in `grad_support`, never here: a verdict must not be part of loading.
         self._grad_support: GradSupport | None = None
+        self._recommended_sampling: RecommendedSampling | None = None
         torch_dtype = _normalize_dtype(dtype)
         model_kwargs = dict(model_kwargs or {})
         # Resolved once, and only when something is actually fetched by id: a caller handing over
@@ -623,6 +625,33 @@ class EagerModel:
     def default_prepend_bos(self) -> bool:
         return self._default_prepend_bos
 
+    @property
+    def recommended_sampling(self) -> RecommendedSampling:
+        """Read when first asked, never at load: a caller handing over its own ``hf_model`` and
+        ``tokenizer`` may use an id that resolves to nothing, and gets an empty answer here."""
+        # `getattr`: a test double built without `__init__` has no slot yet.
+        stated = getattr(self, "_recommended_sampling", None)
+        if stated is None:
+            stated = self._recommended_sampling = read_recommended_sampling(self.hf_model_id)
+        return stated
+
+    def sampling_settings(
+        self,
+        *,
+        temperature: float | None = None,
+        top_k: int | None = None,
+        top_p: float | None = None,
+        presence_penalty: float | None = None,
+    ) -> SamplingSettings:
+        """See the protocol."""
+        return resolve_sampling(
+            self.recommended_sampling,
+            temperature=temperature,
+            top_k=top_k,
+            top_p=top_p,
+            presence_penalty=presence_penalty,
+        )
+
     # --- classmethod convenience --------------------------------------------
     @classmethod
     def from_pretrained(
@@ -1273,7 +1302,15 @@ class EagerModel:
         return capture_attention_eager(self, [int(t) for t in prompt_token_ids], layers)
 
     def _generate_completion(
-        self, prompt_ids: list[int], max_tokens: int, temperature: float, seed: int | None
+        self,
+        prompt_ids: list[int],
+        max_tokens: int,
+        temperature: float | None,
+        seed: int | None,
+        *,
+        top_k: int | None = None,
+        top_p: float | None = None,
+        presence_penalty: float | None = None,
     ) -> Completion:
         from interp_engine.steer import generate_stream
 
@@ -1283,6 +1320,9 @@ class EagerModel:
                 torch.tensor([prompt_ids], device=self.device),
                 max_tokens=max_tokens,
                 temperature=temperature,
+                top_k=top_k,
+                top_p=top_p,
+                presence_penalty=presence_penalty,
                 seed=seed,
             )
         )
@@ -1296,18 +1336,32 @@ class EagerModel:
         prompt_token_ids: Any,
         *,
         max_tokens: int = 200,
-        temperature: float = 1.0,
+        temperature: float | None = None,
+        top_k: int | None = None,
+        top_p: float | None = None,
+        presence_penalty: float | None = None,
         seed: int | None = None,
     ) -> str:
         """Generate and return the completion text. See the protocol."""
-        return self._generate_completion([int(t) for t in prompt_token_ids], max_tokens, temperature, seed).text
+        return self._generate_completion(
+            [int(t) for t in prompt_token_ids],
+            max_tokens,
+            temperature,
+            seed,
+            top_k=top_k,
+            top_p=top_p,
+            presence_penalty=presence_penalty,
+        ).text
 
     async def generate_stream(
         self,
         prompt_token_ids: Any,
         *,
         max_tokens: int = 200,
-        temperature: float = 1.0,
+        temperature: float | None = None,
+        top_k: int | None = None,
+        top_p: float | None = None,
+        presence_penalty: float | None = None,
         seed: int | None = None,
     ) -> AsyncIterator[str]:
         """Yield decoded text deltas, one per token. See the protocol.
@@ -1323,6 +1377,9 @@ class EagerModel:
             torch.tensor([[int(t) for t in prompt_token_ids]], device=self.device),
             max_tokens=max_tokens,
             temperature=temperature,
+            top_k=top_k,
+            top_p=top_p,
+            presence_penalty=presence_penalty,
             seed=seed,
         ):
             yield step.token_str

{interp_engine-1.11.0 → interp_engine-1.11.2}/interp_engine/protocol.py

@@ -47,6 +47,7 @@ import torch
 from interp_engine.address import Address
 from interp_engine.autograd_support import GradSupport
 from interp_engine.residual_basis import ResidualBasis
+from interp_engine.sampling import RecommendedSampling, SamplingSettings
 
 # Re-exported so a protocol-typed caller needs one import. The type itself lives in
 # `interp_engine.address`, which owns the grammar; this module defines a Protocol and should not.
@@ -150,6 +151,34 @@ class InterpModel(Protocol):
         """
         ...
 
+    # --- sampling -----------------------------------------------------------
+    @property
+    def recommended_sampling(self) -> RecommendedSampling:
+        """What the checkpoint's ``generation_config.json`` states: temperature, top-k, top-p,
+        ``do_sample``. Empty when the checkpoint has none (Qwen3.5) or the file names only token
+        ids (GPT-2). The file is Hugging Face's format, so a presence penalty cannot appear here
+        whatever the model card says.
+        """
+        ...
+
+    def sampling_settings(
+        self,
+        *,
+        temperature: float | None = None,
+        top_k: int | None = None,
+        top_p: float | None = None,
+        presence_penalty: float | None = None,
+    ) -> SamplingSettings:
+        """What a generation called with these arguments runs with, every knob decided.
+
+        The rule is :func:`interp_engine.sampling.resolve_sampling`: a knob passed is used as
+        passed, a knob left ``None`` takes :attr:`recommended_sampling`, and a knob neither states
+        is neutral (temperature 1, no filtering, no penalty). Every generation method below
+        resolves its knobs this way, so a run is reproducible from its arguments plus the
+        checkpoint; a server reports the result beside the completion.
+        """
+        ...
+
     # --- tokenization -------------------------------------------------------
     tokenizer: Any
     """The HF tokenizer (or processor on multimodal archs), for chat templating and
@@ -254,10 +283,17 @@ class InterpModel(Protocol):
         prompt_token_ids: Sequence[int],
         *,
         max_tokens: int = 200,
-        temperature: float = 1.0,
+        temperature: float | None = None,
+        top_k: int | None = None,
+        top_p: float | None = None,
+        presence_penalty: float | None = None,
         seed: int | None = None,
     ) -> str:
-        """Generate and return the completion text (no prompt echo)."""
+        """Generate and return the completion text (no prompt echo).
+
+        Sampling knobs left ``None`` take the checkpoint's recommendation; see
+        :meth:`sampling_settings`.
+        """
         ...
 
     def generate_stream(
@@ -265,7 +301,10 @@ class InterpModel(Protocol):
         prompt_token_ids: Sequence[int],
         *,
         max_tokens: int = 200,
-        temperature: float = 1.0,
+        temperature: float | None = None,
+        top_k: int | None = None,
+        top_p: float | None = None,
+        presence_penalty: float | None = None,
         seed: int | None = None,
     ) -> AsyncIterator[str]:
         """Yield decoded text deltas as they are produced.