interp-engine 1.2.4__tar.gz → 1.2.6__tar.gz

{interp_engine-1.2.4 → interp_engine-1.2.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: interp-engine
-Version: 1.2.4
+Version: 1.2.6
 Summary: A fast, standardized interpretability engine that supports most modern models and architectures. Powers Neuronpedia.
 Project-URL: Homepage, https://github.com/decoderesearch/interp-engine
 Project-URL: Repository, https://github.com/decoderesearch/interp-engine
@@ -37,6 +37,12 @@ Description-Content-Type: text/markdown
 <p align="center">
   🔗 <a href="https://interp-engine.org"><strong>interp-engine.org</strong></a>
 </p>
+<p align="center">
+  <a href="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml"><img src="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml/badge.svg?branch=main" alt="CI status"></a>
+  <a href="https://pypi.org/project/interp-engine/"><img src="https://img.shields.io/pypi/v/interp-engine.svg" alt="PyPI version"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/pypi/l/interp-engine.svg" alt="Apache-2.0 license"></a>
+  <a href="https://join.slack.com/t/opensourcemechanistic/shared_invite/zt-3z9o0hxjl-MDX9pbATO2qESOazNDLpdQ"><img src="https://img.shields.io/badge/Slack-Open%20Source%20Mechanistic%20Interpretability-4A154B?logo=slack&logoColor=white" alt="Join the Slack"></a>
+</p>
 
 
 `interp-engine` is an interpretability engine that is fast (>40x tok/s vs HF eager), standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.

{interp_engine-1.2.4 → interp_engine-1.2.6}/README.md

@@ -6,6 +6,12 @@
 <p align="center">
   🔗 <a href="https://interp-engine.org"><strong>interp-engine.org</strong></a>
 </p>
+<p align="center">
+  <a href="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml"><img src="https://github.com/decoderesearch/interp-engine/actions/workflows/engine-tests.yml/badge.svg?branch=main" alt="CI status"></a>
+  <a href="https://pypi.org/project/interp-engine/"><img src="https://img.shields.io/pypi/v/interp-engine.svg" alt="PyPI version"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/pypi/l/interp-engine.svg" alt="Apache-2.0 license"></a>
+  <a href="https://join.slack.com/t/opensourcemechanistic/shared_invite/zt-3z9o0hxjl-MDX9pbATO2qESOazNDLpdQ"><img src="https://img.shields.io/badge/Slack-Open%20Source%20Mechanistic%20Interpretability-4A154B?logo=slack&logoColor=white" alt="Join the Slack"></a>
+</p>
 
 
 `interp-engine` is an interpretability engine that is fast (>40x tok/s vs HF eager), standardized (34 'points'/addresses across architectures), and easy to use and debug. It powers all of [Neuronpedia](https://neuronpedia.org)'s inference and is checked for accuracy against HF Transformers and other engines.

{interp_engine-1.2.4 → interp_engine-1.2.6}/docs/AGENT_INTEGRATION.md

@@ -236,7 +236,7 @@ mistake you made.
 | `UnknownCoordinate` | an address carried a coordinate this version has no field for | version skew between processes — align the engine version on both |
 | `{key} fired twice in one forward pass` | one address resolved to a module invoked more than once | a re-entrant trunk; capture per-invocation or name a narrower address |
 | `Captured nothing at {missing}: the resolved module(s) did not run` | a quantized or fused kernel replaced the block's forward and computes the tensor inline | different point, or an unfused load |
-| `NoChatTemplateError` | the tokenizer has no chat template | `model.tok.has_chat_template()` first; do not hand-write one |
+| `NoChatTemplateError` | the model has no chat format (no Jinja template and no code formatter) | `model.tok.has_chat_template()` first; do not hand-write one, and do not read `tokenizer.chat_template` — it is `None` for families that define their format in code |
 | `CapabilityUnsupported` | you asked one backend for something only the other can do | the message names the call that works; `CAPABILITIES` is the table it came from (rule 13) |
 | `... cannot run from inside a running event loop` | a sync free function or `sync_model` call from async code | `await` the method it wraps; the message names it |
 | `... needs vLLM, but vLLM is not installed` | missing extra, or a non-Linux/CUDA box | `pip install 'interp-engine[vllm]'`, or `backend="eager"`; gate on `vllm_installed()` to branch instead of catching |

{interp_engine-1.2.4 → interp_engine-1.2.6}/docs/ARCHITECTURE_QUIRKS.md

@@ -38,16 +38,24 @@ in `facts.py` instead.
 | `points.py`                | The canonical hook points as data: scope, width, vLLM support and the reason for each limit. Consumers (the vLLM served-point gate, the width guard, the reshape set, the docs footnotes) derive from it.                                                                                                                                               | **Never for a new model** — a point is architecture-independent by construction. Only when adding a _point_, and then `tests/test_points_registry.py` tells you what else needs updating.           |
 | `chat_conventions.py`      | How a model structures what it **generates**: harmony markers (gpt-oss), reasoning delimiters (`REASONING_TAGS`, e.g. `<think>`/`</think>`), and generic turn-end tokens.                                                                                                                                                                               | Only when a model introduces a _new_ reasoning delimiter pair — append one `ReasoningTags` entry. Models reusing `<think>` need no change.                                                          |
 | `chat_compose.py`          | Consumes the table above to turn a generation back into messages (`compose_assistant_turns`), normalizing every family's reasoning to `<think>…</think>` in the message content.                                                                                                                                                                        | Never for a new model — it reads `chat_conventions.py`. Only to change the _wire shape_ clients receive.                                                                                            |
+| `chat_formatters.py`       | `CODE_CHAT_FORMATS`: the few architectures that ship **no** `chat_template` and define their prompt format in Python instead (DeepSeek-V4), mapped to a loader that imports the checkpoint's own encoder.                                                                                                                                               | Only when a model has no chat template at all. A model with a template needs no entry — that is the other 99%, and the default path below.                                                          |
 
 Two deliberate non-entries:
 
-- **Prompt-side chat structure has no table, in either direction.** Going _in_,
+- **Prompt-side chat structure needs no per-family entry, in either direction.** Going _in_,
   `Tokenize.message_spans` renders through the model's real chat template and derives per-token
   role/section by diffing renders, so every family (ChatML, Gemma, Llama, harmony) works without
   per-family code. Coming back _out_, `chat_compose.py` reads only the generated text and lets the
   caller supply the prompt messages it already holds, so no code ever parses a rendered prompt back
   into messages. Between them they replaced the frontend's per-family state machines and the
   inference app's per-model response parsers.
+
+  `CODE_CHAT_FORMATS` is the one exception, and it is narrow on purpose: it holds only checkpoints
+  that publish **no** template, where there is no render to diff and the format exists solely as
+  code. Even then the entry is a _loader_, not a description — the format itself is read from the
+  encoder shipped beside the weights, so the table names which architectures are affected and
+  never what their prompts look like. Adding a model that _has_ a template here would be the
+  mistake the paragraph above is about.
 - **Nothing keys on a model-name _substring_.** Both tables above key on architecture class or on
   added-token-vocab membership, i.e. on **capability**. A substring match (`"llama-3" in
 name_or_path`) is the anti-pattern this structure exists to avoid: a substring is a claim about a

{interp_engine-1.2.4 → interp_engine-1.2.6}/docs/USAGE.md

@@ -197,8 +197,37 @@ prompt = model.tok.apply_chat_template([{"role": "user", "content": "Hi"}])
 token_ids = model.tok.apply_chat_template([{"role": "user", "content": "Hi"}], tokenize=True)
 ```
 
-`apply_chat_template` raises `NoChatTemplateError` when the tokenizer has none, rather than inventing
-a format the model was never trained on; `model.tok.has_chat_template()` asks first.
+`apply_chat_template` raises `NoChatTemplateError` when the model has no chat format at all, rather
+than inventing one it was never trained on; `model.tok.has_chat_template()` asks first.
+
+A few checkpoints ship no Jinja `chat_template` because they define their format in Python instead —
+DeepSeek-V4 carries an `encoding/encoding_dsv4.py` in the repo, beside the weights, and a Jinja
+template could not express its other half (`parse_message_from_completion_text`). The engine
+downloads and imports that file rather than vendoring a copy of it, so those models render chat
+through the same three calls above, and `has_chat_template()` says yes. Reading
+`tokenizer.chat_template` directly is what gets this wrong: it is `None` for a model that renders
+chat perfectly well. Loading the file is remote code execution, so it needs `trust_remote_code=True`;
+without it the model still loads and chat is simply unavailable.
+
+`model.tok.accepted_template_kwargs([...])` reports which optional controls (`enable_thinking`,
+`reasoning_effort`) this model actually reads, whichever of the two renders it. Adding a family means
+one entry in `chat_formatters.CODE_CHAT_FORMATS`.
+
+To attribute *tokens* to messages there are two methods, and the difference matters. `message_spans`
+gives per-token role, channel and section (`header` / `content` / `footer`), leaving the trailing
+generation scaffold owned by no message — use it to read or display structure. `message_partition`
+gives one contiguous `[start, end)` span per message that together cover every token, which is what
+mean-pooling activations per turn needs:
+
+```python
+token_ids, spans = model.tok.message_partition([{"role": "user", "content": "Hi"}])
+per_turn = [acts[start:end].mean(0) for start, end in spans]
+```
+
+Only `message_partition` is correct for a code-rendered model. Computing the same spans by rendering
+growing message prefixes and taking length deltas assumes appending a message only appends tokens;
+DeepSeek-V4 rewrites earlier turns once a later user turn exists (it drops their reasoning), so the
+deltas land in the wrong places and still look like a partition.
 
 ## Capture while generating
 

{interp_engine-1.2.4 → interp_engine-1.2.6}/interp_engine/tokenize.py

@@ -297,6 +297,62 @@ class Tokenize:
             )
         )
 
+    def message_partition(
+        self,
+        messages: list[dict[str, str]],
+        **template_kwargs: Any,
+    ) -> tuple[list[int], list[tuple[int, int]]]:
+        """The closed conversation's ids, plus one contiguous ``[start, end)`` span per message.
+
+        A *partition*, which is a different contract from :meth:`message_spans`: the spans are
+        contiguous, cover every token, and align 1:1 with ``messages``. ``message_spans`` splits
+        each message into header/content/footer sections and leaves the trailing generation
+        scaffold owned by nobody, so a caller that mean-pools activations per turn wants this
+        one. Rendered closed, since a scaffold belongs to the turn nobody has written yet.
+
+        The two branches below are deliberately NOT unified. Where a code formatter renders, the
+        boundaries are exact, because the formatter reports its own message blocks. Where a Jinja
+        template does, they come from the length each render grows by as one more message is
+        appended -- an assumption that the template only ever appends, and also the arithmetic
+        every existing caller's numbers were computed with. Preserving it verbatim is the point:
+        a mean pooled over a span whose edge moved by one token is a different number, and these
+        feed persona projections that are compared across runs. So a template-rendered model is
+        unaffected by this method existing, and a code-rendered one gets boundaries the
+        prefix-delta could not have found.
+        """
+        if not messages:
+            return [], []
+
+        if self.formatter is not None:
+            rendered = self.formatter.render(
+                messages,
+                add_generation_prompt=False,
+                continue_final_message=False,
+                **template_kwargs,
+            )
+            full_ids = self._encode_rendered(rendered.text)
+            bounds = [0]
+            for j in range(1, len(messages)):
+                cut = _common_prefix_len(self._encode_rendered(rendered.upto(j)), full_ids)
+                bounds.append(max(bounds[-1], min(cut, len(full_ids))))
+            # The last message closes the sequence by construction (`upto(n)` is the whole
+            # render once the generation scaffold is off), stated rather than measured so the
+            # spans cover every token even if a tokenizer merges across the final cut.
+            bounds.append(len(full_ids))
+            return full_ids, list(zip(bounds, bounds[1:]))
+
+        bounds = [0]
+        full_ids: list[int] = []
+        for j in range(1, len(messages) + 1):
+            full_ids = self._render_ids(
+                messages[:j],
+                add_generation_prompt=False,
+                continue_final_message=False,
+                **template_kwargs,
+            )
+            bounds.append(len(full_ids))
+        return full_ids, list(zip(bounds, bounds[1:]))
+
     def message_spans(
         self,
         messages: list[dict[str, str]],

{interp_engine-1.2.4 → interp_engine-1.2.6}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "interp-engine"
-version = "1.2.4"
+version = "1.2.6"
 description = "A fast, standardized interpretability engine that supports most modern models and architectures. Powers Neuronpedia."
 readme = "README.md"
 license = "Apache-2.0"

{interp_engine-1.2.4 → interp_engine-1.2.6}/tests/test_chat_formatters.py

@@ -29,8 +29,8 @@ import pytest
 
 from interp_engine import Tokenize
 from interp_engine.chat_formatters import (
-    DEEPSEEK_V4_ENCODER_FILE,
     CODE_CHAT_FORMATS,
+    DEEPSEEK_V4_ENCODER_FILE,
     ChatFormatterUnavailable,
     DeepseekV4Formatter,
     load_deepseek_v4_formatter,
@@ -325,6 +325,7 @@ def test_accepted_template_kwargs_asks_the_formatter(tok: Tokenize):
 
 def test_accepted_template_kwargs_greps_a_real_template():
     """The Jinja path is unchanged: a name is offered only if the template mentions it."""
+
     class _Templated:
         chat_template = "{% if enable_thinking %}<think>{% endif %}"
         eos_token = EOS
@@ -392,6 +393,127 @@ def test_prefill_spans_end_on_content(tok: Tokenize):
     assert spans[-1].role == "assistant"
 
 
+# --------------------------------------------------------------------------- #
+# Per-message partition (what activation pooling indexes by)
+# --------------------------------------------------------------------------- #
+
+
+def _legacy_partition(tok: Tokenize, messages: list[dict[str, str]]) -> tuple[list[int], list[tuple[int, int]]]:
+    """The prefix-delta arithmetic that callers ran inline before ``message_partition``.
+
+    Kept here verbatim as the reference implementation. A template-rendered model has to
+    partition identically to this forever: the spans index activation mean-pooling, so an edge
+    that moves by one token silently changes a published number.
+    """
+    spans: list[tuple[int, int]] = []
+    previous = 0
+    full_ids: list[int] = []
+    for index in range(len(messages)):
+        current = tok.apply_chat_template(messages[: index + 1], tokenize=True, add_generation_prompt=False)
+        assert isinstance(current, list)
+        spans.append((previous, len(current)))
+        previous = len(current)
+        full_ids = current
+    return full_ids, spans
+
+
+class _RecordingTokenizer(_CharTokenizer):
+    """A templated tokenizer that records how it was asked to render."""
+
+    chat_template = "{# present; contents never evaluated here #}"
+
+    def __init__(self):
+        super().__init__()
+        self.calls: list[tuple] = []
+
+    def apply_chat_template(
+        self,
+        messages: list[dict[str, str]],
+        tokenize: bool = True,
+        add_generation_prompt: bool = False,
+        continue_final_message: bool = False,
+        **_: Any,
+    ):
+        self.calls.append(
+            (tuple(m["content"] for m in messages), add_generation_prompt, continue_final_message, tokenize)
+        )
+        return list(range(1, 2 * len(messages) + 1))
+
+
+def test_partition_asks_a_template_exactly_what_the_old_inline_code_asked():
+    """The guarantee that no template-rendered model's pooled activations move.
+
+    Asserted on the *calls* rather than on one template's output, because that covers every
+    template rather than whichever ones happen to be cached: same message slices, same flags,
+    same tokenizing path means the same ids come back, whatever the template does with them.
+    """
+    tokenizer = _RecordingTokenizer()
+    messages = [{"role": "user", "content": "a"}, {"role": "assistant", "content": "b"}]
+
+    full_ids, spans = Tokenize(tokenizer).message_partition(messages)
+
+    assert tokenizer.calls == [
+        (("a",), False, False, True),
+        (("a", "b"), False, False, True),
+    ]
+    assert full_ids == [1, 2, 3, 4]
+    assert spans == [(0, 2), (2, 4)]
+
+
+def test_partition_equals_the_legacy_arithmetic_on_a_template():
+    tokenizer = _RecordingTokenizer()
+    messages = [{"role": "user", "content": "a"}, {"role": "assistant", "content": "b"}]
+    tok = Tokenize(tokenizer)
+
+    assert tok.message_partition(messages) == _legacy_partition(tok, messages)
+
+
+def test_partition_of_no_messages_is_empty():
+    assert Tokenize(_RecordingTokenizer()).message_partition([]) == ([], [])
+
+
+def test_partition_covers_every_token_exactly_once(tok: Tokenize):
+    """Contiguous, gapless, and ending on the last token — a partition, not a tagging.
+
+    Pooling reads `acts[start:end]` per message, so a gap drops activations from every mean and
+    an overlap double-counts them into two turns.
+    """
+    full_ids, spans = tok.message_partition(MULTI_TURN)
+
+    assert len(spans) == len(MULTI_TURN)
+    assert spans[0][0] == 0
+    assert spans[-1][1] == len(full_ids)
+    assert all(before[1] == after[0] for before, after in zip(spans, spans[1:]))
+
+
+def test_partition_blocks_match_the_formatters_own_boundaries(tok: Tokenize, formatter: DeepseekV4Formatter):
+    """Where a formatter renders, the boundaries are read off it rather than inferred.
+
+    Message 0 also carries the leading scaffold (the BOS token, in ``RenderedChat.prefix``),
+    which is where the prefix-delta arithmetic puts it too — there is no earlier message to
+    attribute it to, and leaving it unowned would break the partition.
+    """
+    rendered = formatter.render(MULTI_TURN, add_generation_prompt=False)
+    _, spans = tok.message_partition(MULTI_TURN)
+
+    for index, (start, end) in enumerate(spans):
+        expected = rendered.blocks[index]
+        if index == 0:
+            expected = rendered.prefix + expected
+        assert end - start == len(tok.tokenizer.encode(expected)), f"message {index} span is not its own block"
+
+
+def test_the_legacy_arithmetic_would_misplace_a_code_rendered_conversation(tok: Tokenize):
+    """Why this method exists rather than the caller keeping its inline loop.
+
+    The prefix delta assumes appending a message only appends tokens. DeepSeek-V4 rewrites
+    earlier turns once a later user turn exists (a turn stops being the *last* user turn), so
+    the deltas land in the wrong places — and the failure is silent, since they still look like
+    a partition.
+    """
+    assert tok.message_partition(MULTI_TURN) != _legacy_partition(tok, MULTI_TURN)
+
+
 # --------------------------------------------------------------------------- #
 # The real encoder, from the checkpoint
 # --------------------------------------------------------------------------- #

{interp_engine-1.2.4 → interp_engine-1.2.6}/tests/test_chat_templates.py

@@ -186,3 +186,34 @@ def test_reasoning_markers_present_in_tokenizer_vocab():
     """
     model = _load(QWEN_THINKING)
     assert detect_reasoning_tags(model.tokenizer.get_added_vocab().keys()) is not None
+
+
+@pytest.mark.parametrize("spec", CHAT_PARAMS)
+def test_message_partition_matches_the_prefix_delta_it_replaced(spec: ModelSpec):
+    """``message_partition`` must not move a template-rendered model's message boundaries.
+
+    Callers that mean-pool activations per turn (the persona / assistant-axis path) used to run
+    this arithmetic inline against the tokenizer. The spans index the pooling, so an edge that
+    shifts by one token quietly changes a published projection — on every deployed instruct
+    model, none of which needed the change. The generalisation to every template is asserted on
+    the render calls in ``test_chat_formatters.py``; this is the same claim against a real
+    template and a real BPE vocabulary.
+    """
+    model = _load(spec)
+    messages = [
+        {"role": "user", "content": "What is 2+2?"},
+        {"role": "assistant", "content": "Four."},
+        {"role": "user", "content": "And 3+3?"},
+    ]
+
+    spans: list[tuple[int, int]] = []
+    previous = 0
+    reference_ids: list[int] = []
+    for count in range(1, len(messages) + 1):
+        reference_ids = _coerce_ids(
+            model.tokenizer.apply_chat_template(messages[:count], tokenize=True, add_generation_prompt=False)
+        )
+        spans.append((previous, len(reference_ids)))
+        previous = len(reference_ids)
+
+    assert model.tok.message_partition(messages) == (reference_ids, spans)