renderers 0.1.11.dev1__tar.gz → 0.1.11.dev2__tar.gz

{renderers-0.1.11.dev1 → renderers-0.1.11.dev2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: renderers
-Version: 0.1.11.dev1
+Version: 0.1.11.dev2
 Summary: Chat template renderers — deterministic message-to-token conversion for LLM training
 License-Expression: Apache-2.0
 License-File: LICENSE
@@ -11,7 +11,11 @@ Requires-Dist: openai-harmony>=0.0.4
 Requires-Dist: openai>=1.108.1
 Requires-Dist: prime-pydantic-config>=0.3.0.dev83
 Requires-Dist: tiktoken
-Requires-Dist: transformers>=4.50.0
+Provides-Extra: multimodal
+Requires-Dist: pillow>=12.2.0; extra == 'multimodal'
+Requires-Dist: transformers>=4.50.0; extra == 'multimodal'
+Provides-Extra: transformers
+Requires-Dist: transformers>=4.50.0; extra == 'transformers'
 Description-Content-Type: text/markdown
 
 # renderers
@@ -26,13 +30,28 @@ Standalone on PyPI, and portable across training and inference stacks (transform
 uv add renderers
 ```
 
+The base install supports text renderers with a bring-your-own tokenizer. Add
+the Hugging Face integration for the tokenizer-loading helpers, or the complete
+media stack for image/audio rendering:
+
+```bash
+uv add 'renderers[transformers]'
+uv add 'renderers[multimodal]'
+```
+
+A BYO tokenizer must expose `encode`, `decode`, `convert_tokens_to_ids`, token
+IDs such as `eos_token_id`, and `return_offsets_mapping=True` through its call
+interface. `DefaultRenderer` additionally requires `apply_chat_template`.
+This includes text-only Inkling training: `InklingRenderer` loads its
+Transformers processor only when image or audio content is actually rendered.
+
 ## At a glance
 
 ```python
-from transformers import AutoTokenizer
 from renderers import create_renderer
+from renderers.base import load_tokenizer
 
-tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
+tok = load_tokenizer("Qwen/Qwen3-8B")              # renderers[transformers]
 r = create_renderer(tok)                            # → Qwen3Renderer (auto-resolved)
 
 prompt_ids = r.render_ids(
@@ -92,17 +111,10 @@ r = create_renderer(tok)                # AutoRendererConfig is the implicit def
 
 Auto-detect matches `tokenizer.name_or_path` against `MODEL_RENDERER_MAP` by **exact match**. Prefix matching is intentionally off — same architecture can ship different chat templates (base vs instruct, fine-tune renames). Fine-tunes must pass an explicit typed config (e.g. `Qwen3RendererConfig()`). Unknown text-only names fall back to `DefaultRenderer`, unless `AutoRendererConfig(thinking_retention=...)` was set; the default renderer cannot implement that bridge policy.
 
-### Pools
-
-```python
-from renderers import create_renderer_pool
-
-pool = create_renderer_pool("Qwen/Qwen3-8B", size=16)
-with pool.checkout() as r:
-    ids = r.render_ids(messages)
-```
-
-Each slot owns its own tokenizer copy. Construction fans out across a thread pool so a 32-slot pool doesn't serially eat ~10–15s of `from_pretrained` calls at startup.
+Without the `transformers` extra, exact-match registered models still
+auto-resolve. For an unknown name, renderers cannot safely probe `AutoConfig`
+to distinguish a text model from an unknown VLM; pass an explicit typed config
+such as `DefaultRendererConfig()` for a known text-only model.
 
 ## Why use a renderer
 
@@ -125,7 +137,7 @@ Each break fragments a rollout into multiple training samples — every fragment
 
 ## Typed renderer configs
 
-Each renderer accepts a typed pydantic config at construction. Some fields mirror chat-template kwargs; others configure renderer-only behavior such as image caching, parsers, or Harmony preamble construction. `create_renderer` and `create_renderer_pool` take one positional `config` argument and an optional keyword-only `chat_template_kwargs` mapping:
+Each renderer accepts a typed pydantic config at construction. Some fields mirror chat-template kwargs; others configure renderer-only behavior such as image caching, parsers, or Harmony preamble construction. `create_renderer` takes one positional `config` argument and an optional keyword-only `chat_template_kwargs` mapping:
 
 ```python
 from renderers import (
@@ -175,7 +187,7 @@ Fallback for unsupported text-only models. Wraps `apply_chat_template` and accep
 
 ## Roadmap
 
-- **VLM expansion.** `ImagePart` support exists for Qwen3-VL, Qwen3.5-family, Gemma 4, and Kimi K2.5 / K2.6 multimodal templates. Remaining work: audio/video support, broader VLM coverage, and more RL validation. Gemma 4 image preprocessing requires a Transformers release that provides `Gemma4Processor`.
+- **VLM expansion.** `ImagePart` support exists for Qwen3-VL, Qwen3.5-family, Gemma 4, and Kimi K2.5 / K2.6 multimodal templates. Install `renderers[multimodal]` for Pillow and the Hugging Face processors. Remaining work: audio/video support, broader VLM coverage, and more RL validation. Gemma 4 image preprocessing requires a Transformers release that provides `Gemma4Processor`.
 - **Patched chat templates.** Some shipped templates re-tokenize history or normalize JSON in ways that break token identity. Plan: a `use_patched` opt-in per renderer that renders the same surface form while avoiding known-bad patterns. (Auto-stripping thinking from past turns is *not* one of these — that's intended template behaviour the renderer reproduces; use `thinking_retention` to override it.)
 
 ## Testing

{renderers-0.1.11.dev1 → renderers-0.1.11.dev2}/README.md

@@ -10,13 +10,28 @@ Standalone on PyPI, and portable across training and inference stacks (transform
 uv add renderers
 ```
 
+The base install supports text renderers with a bring-your-own tokenizer. Add
+the Hugging Face integration for the tokenizer-loading helpers, or the complete
+media stack for image/audio rendering:
+
+```bash
+uv add 'renderers[transformers]'
+uv add 'renderers[multimodal]'
+```
+
+A BYO tokenizer must expose `encode`, `decode`, `convert_tokens_to_ids`, token
+IDs such as `eos_token_id`, and `return_offsets_mapping=True` through its call
+interface. `DefaultRenderer` additionally requires `apply_chat_template`.
+This includes text-only Inkling training: `InklingRenderer` loads its
+Transformers processor only when image or audio content is actually rendered.
+
 ## At a glance
 
 ```python
-from transformers import AutoTokenizer
 from renderers import create_renderer
+from renderers.base import load_tokenizer
 
-tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
+tok = load_tokenizer("Qwen/Qwen3-8B")              # renderers[transformers]
 r = create_renderer(tok)                            # → Qwen3Renderer (auto-resolved)
 
 prompt_ids = r.render_ids(
@@ -76,17 +91,10 @@ r = create_renderer(tok)                # AutoRendererConfig is the implicit def
 
 Auto-detect matches `tokenizer.name_or_path` against `MODEL_RENDERER_MAP` by **exact match**. Prefix matching is intentionally off — same architecture can ship different chat templates (base vs instruct, fine-tune renames). Fine-tunes must pass an explicit typed config (e.g. `Qwen3RendererConfig()`). Unknown text-only names fall back to `DefaultRenderer`, unless `AutoRendererConfig(thinking_retention=...)` was set; the default renderer cannot implement that bridge policy.
 
-### Pools
-
-```python
-from renderers import create_renderer_pool
-
-pool = create_renderer_pool("Qwen/Qwen3-8B", size=16)
-with pool.checkout() as r:
-    ids = r.render_ids(messages)
-```
-
-Each slot owns its own tokenizer copy. Construction fans out across a thread pool so a 32-slot pool doesn't serially eat ~10–15s of `from_pretrained` calls at startup.
+Without the `transformers` extra, exact-match registered models still
+auto-resolve. For an unknown name, renderers cannot safely probe `AutoConfig`
+to distinguish a text model from an unknown VLM; pass an explicit typed config
+such as `DefaultRendererConfig()` for a known text-only model.
 
 ## Why use a renderer
 
@@ -109,7 +117,7 @@ Each break fragments a rollout into multiple training samples — every fragment
 
 ## Typed renderer configs
 
-Each renderer accepts a typed pydantic config at construction. Some fields mirror chat-template kwargs; others configure renderer-only behavior such as image caching, parsers, or Harmony preamble construction. `create_renderer` and `create_renderer_pool` take one positional `config` argument and an optional keyword-only `chat_template_kwargs` mapping:
+Each renderer accepts a typed pydantic config at construction. Some fields mirror chat-template kwargs; others configure renderer-only behavior such as image caching, parsers, or Harmony preamble construction. `create_renderer` takes one positional `config` argument and an optional keyword-only `chat_template_kwargs` mapping:
 
 ```python
 from renderers import (
@@ -159,7 +167,7 @@ Fallback for unsupported text-only models. Wraps `apply_chat_template` and accep
 
 ## Roadmap
 
-- **VLM expansion.** `ImagePart` support exists for Qwen3-VL, Qwen3.5-family, Gemma 4, and Kimi K2.5 / K2.6 multimodal templates. Remaining work: audio/video support, broader VLM coverage, and more RL validation. Gemma 4 image preprocessing requires a Transformers release that provides `Gemma4Processor`.
+- **VLM expansion.** `ImagePart` support exists for Qwen3-VL, Qwen3.5-family, Gemma 4, and Kimi K2.5 / K2.6 multimodal templates. Install `renderers[multimodal]` for Pillow and the Hugging Face processors. Remaining work: audio/video support, broader VLM coverage, and more RL validation. Gemma 4 image preprocessing requires a Transformers release that provides `Gemma4Processor`.
 - **Patched chat templates.** Some shipped templates re-tokenize history or normalize JSON in ways that break token identity. Plan: a `use_patched` opt-in per renderer that renders the same surface form while avoiding known-bad patterns. (Auto-stripping thinking from past turns is *not* one of these — that's intended template behaviour the renderer reproduces; use `thinking_retention` to override it.)
 
 ## Testing

{renderers-0.1.11.dev1 → renderers-0.1.11.dev2}/docs/renderer-config.md

@@ -1,8 +1,7 @@
 # Renderer config
 
-`renderers.RendererConfig` is the typed input to `create_renderer` and
-`create_renderer_pool`. It pins the renderer choice and its config at
-construction time.
+`renderers.RendererConfig` is the typed input to `create_renderer`. It pins the
+renderer choice and its config at construction time.
 
 ```python
 from renderers import create_renderer, Qwen35RendererConfig
@@ -77,10 +76,6 @@ r = create_renderer(
     tokenizer,
     chat_template_kwargs={"enable_thinking": False},
 )
-pool = create_renderer_pool(
-    "Qwen/Qwen3-8B",
-    chat_template_kwargs={"enable_thinking": False},
-)
 ```
 
 Renderers resolves auto configs before applying `chat_template_kwargs`, so the

{renderers-0.1.11.dev1 → renderers-0.1.11.dev2}/pyproject.toml

@@ -19,11 +19,6 @@ dependencies = [
     "openai>=1.108.1",
     "tiktoken",
     "jinja2",
-    # Keep this floor compatible with prime-rl's transformers pin. Inkling's
-    # tokenizer and text-only renderer work on older releases; image/audio
-    # processing fails lazily with an upgrade message when InklingProcessor is
-    # unavailable (native support starts in transformers 5.14).
-    "transformers>=4.50.0",
     # Used by GptOssRenderer to render and parse harmony tokens. Vendoring
     # OpenAI's reference implementation keeps us byte-identical with vLLM
     # (which also uses it) and saves us mirroring a 330-line Jinja template.
@@ -42,6 +37,25 @@ dependencies = [
     "prime-pydantic-config>=0.3.0.dev83",
 ]
 
+[project.optional-dependencies]
+# Tokenizer loading uses Hugging Face. Text-only renderers can instead be
+# constructed with an offset-capable BYO tokenizer and do not import this
+# dependency.
+transformers = [
+    # Keep this floor compatible with prime-rl's transformers pin. Inkling's
+    # tokenizer and text-only renderer work on older releases; image/audio
+    # processing fails lazily with an upgrade message when InklingProcessor is
+    # unavailable (native support starts in transformers 5.14).
+    "transformers>=4.50.0",
+]
+
+# Image/audio renderers also need Pillow to resolve media inputs. Keep this as
+# a separate convenience extra so tokenizer-only users do not pull it in.
+multimodal = [
+    "pillow>=12.2.0",
+    "transformers>=4.50.0",
+]
+
 [tool.hatch.version]
 source = "vcs"
 # Tags look like ``renderers-v0.1.8`` (prefix matches the publish.yml
@@ -90,6 +104,8 @@ dev = [
     "torch>=2.11.0",
     "torchvision>=0.26.0",
     "ty>=0.0.1a29,<0.0.22",
+    # Optional for consumers, but required by tokenizer/parity/VLM tests.
+    "transformers>=4.50.0",
 ]
 
 [tool.uv]

{renderers-0.1.11.dev1 → renderers-0.1.11.dev2}/renderers/__init__.py

@@ -7,6 +7,7 @@ except ImportError:
     __version__ = "0+unknown"
 
 from renderers.base import (
+    ChatTemplateTokenizer,
     Content,
     ContentPart,
     ImagePart,
@@ -21,9 +22,9 @@ from renderers.base import (
     RenderedTokens,
     RenderedTrainingSample,
     Renderer,
-    RendererPool,
     TextPart,
     ThinkingPart,
+    Tokenizer,
     ToolCall,
     ToolCallFunction,
     ToolCallParseStatus,
@@ -33,7 +34,6 @@ from renderers.base import (
     build_training_sample,
     build_trajectory_step,
     create_renderer,
-    create_renderer_pool,
     extract_message_tool_names,
     is_multimodal,
     reject_assistant_in_extension,
@@ -74,17 +74,13 @@ from renderers.configs import (
     RendererConfig,
 )
 
-# Concrete renderer classes are lazy-loaded so that consumers needing
-# only the config layer (``RendererConfig`` discriminated union) don't
-# pay the ``transformers`` import cost. Each renderer module does
-# ``from transformers.tokenization_utils import PreTrainedTokenizer``
-# at module level, so eager imports here would drag ``transformers``
-# into every downstream ``import renderers``. ``__getattr__`` (PEP 562)
-# resolves the names on first attribute access, so ``from renderers
-# import DefaultRenderer`` and ``renderers.DefaultRenderer`` both work
-# transparently. ``create_renderer`` doesn't depend on these eager
-# imports — ``renderers.base._populate_registry`` lazy-imports the
-# concrete classes itself when a renderer is instantiated.
+# Concrete renderer classes are lazy-loaded so that consumers needing only the
+# config layer (``RendererConfig`` discriminated union) don't import every
+# renderer module. Renderer tokenizer annotations use the local ``Tokenizer``
+# protocols, so resolving a text renderer remains safe when the optional
+# ``transformers`` dependency is absent. ``__getattr__`` (PEP 562) resolves the
+# names on first attribute access, while ``renderers.base._populate_registry``
+# handles lazy registration for ``create_renderer``.
 _LAZY_RENDERERS: dict[str, str] = {
     "DeepSeekR1Renderer": "renderers.deepseek_r1",
     "DeepSeekV3Renderer": "renderers.deepseek_v3",
@@ -134,6 +130,7 @@ def __dir__() -> list[str]:
 __all__ = [
     "AutoRendererConfig",
     "BaseRendererConfig",
+    "ChatTemplateTokenizer",
     "Content",
     "ContentPart",
     "DeepSeekR1Renderer",
@@ -205,9 +202,9 @@ __all__ = [
     "RenderedTrainingSample",
     "Renderer",
     "RendererConfig",
-    "RendererPool",
     "TextPart",
     "ThinkingPart",
+    "Tokenizer",
     "ToolCall",
     "ToolCallFunction",
     "ToolCallParseStatus",
@@ -219,7 +216,6 @@ __all__ = [
     "build_trajectory_step",
     "config_from_name",
     "create_renderer",
-    "create_renderer_pool",
     "extract_message_tool_names",
     "is_multimodal",
     "reject_assistant_in_extension",

{renderers-0.1.11.dev1 → renderers-0.1.11.dev2}/renderers/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.1.11.dev1'
-__version_tuple__ = version_tuple = (0, 1, 11, 'dev1')
+__version__ = version = '0.1.11.dev2'
+__version_tuple__ = version_tuple = (0, 1, 11, 'dev2')
 
 __commit_id__ = commit_id = None

{renderers-0.1.11.dev1 → renderers-0.1.11.dev2}/renderers/base.py

@@ -2,10 +2,7 @@ from __future__ import annotations
 
 import enum
 import logging
-import queue
-import threading
 from collections.abc import Mapping
-from contextlib import contextmanager
 from dataclasses import dataclass, field
 from typing import (
     TYPE_CHECKING,
@@ -663,6 +660,37 @@ class RenderedConversation:
         )
 
 
+@runtime_checkable
+class Tokenizer(Protocol):
+    """Structural tokenizer surface used by hand-coded renderers.
+
+    Hugging Face tokenizers satisfy this protocol, as can lightweight BYO
+    adapters around ``tokenizers.Tokenizer`` or another tokenizer backend.
+    Keeping the renderer-facing contract here makes ``transformers`` optional
+    for text rendering. Offset-capable ``__call__`` behavior is required by
+    :func:`attribute_text_segments` to preserve BPE boundary attribution.
+    """
+
+    name_or_path: str
+    unk_token_id: int | None
+    eos_token_id: int | None
+
+    def encode(self, text: str, *args: Any, **kwargs: Any) -> list[int]: ...
+
+    def decode(self, token_ids: Any, *args: Any, **kwargs: Any) -> str: ...
+
+    def convert_tokens_to_ids(self, tokens: Any) -> Any: ...
+
+    def __call__(self, *args: Any, **kwargs: Any) -> Any: ...
+
+
+@runtime_checkable
+class ChatTemplateTokenizer(Tokenizer, Protocol):
+    """Tokenizer surface required by :class:`DefaultRenderer`."""
+
+    def apply_chat_template(self, *args: Any, **kwargs: Any) -> Any: ...
+
+
 @runtime_checkable
 class Renderer(Protocol):
     """Owns message ↔ token conversion for a specific model family."""
@@ -679,10 +707,9 @@ class Renderer(Protocol):
         Behaviour around historical ``reasoning_content`` is owned by the
         renderer instance — the ``thinking_retention`` level is resolved at
         construction, not passed per call. To render with a different
-        configuration, build a different renderer (or different pool). When
-        ``thinking_retention`` is left unset, full renders follow the model's
-        chat template and bridge policy is derived from that template's own
-        history-retention knobs.
+        configuration, build a different renderer. When ``thinking_retention``
+        is left unset, full renders follow the model's chat template and bridge
+        policy is derived from that template's own history-retention knobs.
         """
         ...
 
@@ -839,9 +866,7 @@ class MultimodalRenderer(Renderer, Protocol):
 # Per-type cache for ``is_multimodal``. The ``runtime_checkable`` Protocol
 # isinstance check walks every protocol member via ``hasattr`` on each
 # call; per-type caching collapses that to a single dict lookup on the
-# hot path (e.g. per-bridge dispatch). Pools expose ``is_multimodal``
-# directly as a snapshot attribute (different pools share a class but
-# wrap different renderer types), so we don't need to special-case them.
+# hot path (e.g. per-bridge dispatch).
 _IS_MULTIMODAL_BY_TYPE: dict[type, bool] = {}
 
 
@@ -863,127 +888,6 @@ def is_multimodal(r: object) -> bool:
     return cached
 
 
-class RendererPool:
-    """Pool of Renderer instances that itself satisfies the Renderer protocol.
-
-    Callers treat a pool like a single renderer — ``pool.render_ids(...)``,
-    ``pool.bridge_to_next_turn(...)``, ``isinstance(pool, MultimodalRenderer)``
-    all work via structural delegation. The pool internally serializes
-    access to its inner renderers (each wraps its own tokenizer copy).
-
-    Concurrency model:
-    - ``size == 1``: a single inner renderer guarded by a ``threading.Lock``.
-      Avoids the queue's per-call overhead on the common default config.
-    - ``size > 1``: a ``queue.Queue`` of independent renderers, checked out
-      one at a time. HuggingFace fast tokenizers release the GIL during
-      Rust encoding, so threads achieve real parallelism.
-
-    Construction parallelism for ``size > 1``: ``AutoTokenizer.from_pretrained``
-    takes hundreds of ms per call (JSON parse + Rust tokenizer build + HF
-    cache lookup), so populating a 32-slot pool serially costs ~10-15s on
-    startup and shows up directly as a step-0 stall. We fan the factory out
-    across a short-lived thread pool; the GIL-bound Python portion stops
-    scaling past ~8 workers, so we clamp there.
-    """
-
-    def __init__(self, factory: Callable[[], Renderer], size: int):
-        from concurrent.futures import ThreadPoolExecutor
-
-        self._factory = factory
-        self._size = size
-
-        if size == 1:
-            renderer = factory()
-            self._sole: Renderer | None = renderer
-            self._lock: threading.Lock | None = threading.Lock()
-            self._pool: queue.Queue[Renderer] | None = None
-            sample: Renderer = renderer
-        else:
-            self._sole = None
-            self._lock = None
-            self._pool = queue.Queue(maxsize=size)
-            workers = min(size, 8)
-            with ThreadPoolExecutor(max_workers=workers) as executor:
-                for renderer in executor.map(lambda _: factory(), range(size)):
-                    self._pool.put(renderer)
-            # Peek without removing — safe at construction time before any
-            # checkout has been served.
-            sample = self._pool.queue[0]
-
-        # Snapshot the protocol-shaped attributes from a sample renderer.
-        # They are constant per renderer class, so resolving them once at
-        # construction (a) eliminates per-call ``getattr``/``isinstance``
-        # overhead and (b) lets a future out-of-process pool variant skip
-        # holding a live tokenizer in the parent process.
-        self._renderer_cls: type[Renderer] = type(sample)
-        self.supports_tools: bool = getattr(sample, "supports_tools", True)
-        self.is_multimodal: bool = is_multimodal(sample)
-        # ``mm_token_type_id_map`` is set ONLY on pools wrapping a
-        # ``MultimodalRenderer``. We deliberately don't expose this as a
-        # class-level property: ``runtime_checkable`` Protocol's
-        # isinstance check uses ``inspect.getattr_static``, which finds
-        # property descriptors on the class regardless of whether their
-        # fget raises. Conditional instance attributes (present in
-        # ``self.__dict__`` only when applicable) are the only way to
-        # make ``isinstance(pool, MultimodalRenderer)`` reflect the
-        # inner renderer's actual protocol conformance.
-        if isinstance(sample, MultimodalRenderer):
-            self.mm_token_type_id_map: dict[int, int] = sample.mm_token_type_id_map
-
-    @contextmanager
-    def checkout(self):
-        if self._sole is not None:
-            assert self._lock is not None
-            with self._lock:
-                yield self._sole
-            return
-        assert self._pool is not None
-        renderer = self._pool.get()
-        try:
-            yield renderer
-        finally:
-            self._pool.put(renderer)
-
-    @property
-    def size(self) -> int:
-        return self._size
-
-    @property
-    def renderer_cls(self) -> type[Renderer]:
-        """Class of the renderers in this pool (uniform across all slots)."""
-        return self._renderer_cls
-
-    # ── Renderer protocol delegation ────────────────────────────────────
-    # Pool structurally satisfies ``Renderer`` (and ``MultimodalRenderer``
-    # when its slots wrap multimodal renderers). Callers can call methods
-    # directly and dispatch with ``isinstance(pool, MultimodalRenderer)``
-    # without reaching into ``checkout()``.
-
-    def render(self, *args: Any, **kwargs: Any) -> "RenderedTokens":
-        with self.checkout() as r:
-            return r.render(*args, **kwargs)
-
-    def render_ids(self, *args: Any, **kwargs: Any) -> list[int]:
-        with self.checkout() as r:
-            return r.render_ids(*args, **kwargs)
-
-    def parse_response(self, *args: Any, **kwargs: Any) -> "ParsedResponse":
-        with self.checkout() as r:
-            return r.parse_response(*args, **kwargs)
-
-    def get_stop_token_ids(self) -> list[int]:
-        with self.checkout() as r:
-            return r.get_stop_token_ids()
-
-    def bridge_to_next_turn(self, *args: Any, **kwargs: Any) -> "RenderedTokens | None":
-        with self.checkout() as r:
-            return r.bridge_to_next_turn(*args, **kwargs)
-
-    # ``mm_token_type_id_map`` (the MultimodalRenderer protocol attribute)
-    # is set in ``__init__`` only for pools wrapping multimodal renderers;
-    # see the comment there for why this isn't a class-level property.
-
-
 RENDERER_REGISTRY: dict[str, type] = {}
 
 # Exact canonical HF model names → renderer. We do NOT use prefix
@@ -1156,6 +1060,25 @@ MULTIMODAL_MODELS: dict[str, set[str]] = {
 }
 
 
+_TRANSFORMERS_INSTALL_HINT = (
+    "Install the optional dependency with "
+    "`pip install 'renderers[transformers]'` (or "
+    "`uv add 'renderers[transformers]'`). Text-only renderers work without "
+    "it when constructed with an offset-capable tokenizer object."
+)
+
+
+def _require_transformers(feature: str) -> Any:
+    """Return ``transformers`` or raise an actionable optional-extra error."""
+    try:
+        import transformers
+    except ImportError as exc:
+        raise ImportError(
+            f"{feature} requires Transformers. {_TRANSFORMERS_INSTALL_HINT}"
+        ) from exc
+    return transformers
+
+
 def _model_has_vision_config(model_name: str) -> bool:
     """Return True if the HF config for ``model_name`` declares vision inputs.
 
@@ -1166,13 +1089,26 @@ def _model_has_vision_config(model_name: str) -> bool:
     match what the trainer reconstructs — a class of bug the renderer
     abstraction exists to prevent.
 
-    Returns False on any AutoConfig failure (offline, gated, missing) so
-    a flaky HF probe never blocks a legitimate text-only fine-tune.
+    Returns False on remote/config failures so a flaky HF probe never blocks a
+    legitimate text-only fine-tune. When Transformers itself is unavailable,
+    however, auto-resolution cannot safely distinguish an unknown text model
+    from an unknown VLM; callers must install the extra or choose an explicit
+    renderer config.
     """
     try:
-        from transformers import AutoConfig
-
-        cfg = AutoConfig.from_pretrained(model_name, trust_remote_code=False)
+        transformers = _require_transformers("Auto-resolving an unregistered model")
+    except ImportError as exc:
+        raise ImportError(
+            f"Cannot auto-resolve unregistered model {model_name!r} without "
+            "checking whether it is multimodal. Install "
+            "`renderers[transformers]`, or pass an explicit typed renderer "
+            "config such as `DefaultRendererConfig()` for a known text-only "
+            "model."
+        ) from exc
+    try:
+        cfg = transformers.AutoConfig.from_pretrained(
+            model_name, trust_remote_code=False
+        )
     except Exception:
         return False
     # Most VLM configs nest a vision tower as ``vision_config`` (Qwen-VL,
@@ -1196,8 +1132,8 @@ def _model_has_vision_config(model_name: str) -> bool:
 # Pinning the revision keeps the trust narrow: even with
 # ``trust_remote_code=True``, transformers downloads / executes the
 # tokenizer Python from this exact commit only. A future malicious push
-# to the Moonshot HF repo doesn't auto-propagate to anyone using
-# ``create_renderer_pool``. Bump these SHAs deliberately, with review.
+# to the Moonshot HF repo doesn't auto-propagate to callers of
+# ``load_tokenizer``. Bump these SHAs deliberately, with review.
 TRUSTED_REVISIONS: dict[str, str] = {
     "moonshotai/Kimi-K2-Instruct": "fd1984e2b7a3350dbf7305fe73a4ede25c14de50",
     "moonshotai/Kimi-K2.5": "4d01dfe0332d63057c186e0b262165819efb6611",
@@ -1330,7 +1266,9 @@ def load_tokenizer(model_name_or_path: str):
     those exact IDs we load tokenizer files from the audited unrestricted
     ``unsloth`` mirrors instead, then restore ``tokenizer.name_or_path`` to
     the requested Meta ID so auto-resolution still selects ``Llama3Renderer``.
+    Requires the ``renderers[transformers]`` extra.
     """
+    _require_transformers("Loading a tokenizer")
     load_name_or_path = _tokenizer_source_for(model_name_or_path)
     kwargs = _tokenizer_load_kwargs(load_name_or_path)
     tok = _load_tokenizer_via_auto(load_name_or_path, **kwargs)
@@ -1408,44 +1346,6 @@ def _populate_registry():
     )
 
 
-def create_renderer_pool(
-    tokenizer_name_or_path: str,
-    config: RendererConfig | None = None,
-    *,
-    size: int = 16,
-    chat_template_kwargs: Mapping[str, Any] | None = None,
-) -> RendererPool:
-    """Create a RendererPool with *size* independent tokenizer copies.
-
-    Each slot loads its own tokenizer so threads never share mutable
-    state. HuggingFace fast tokenizers release the GIL during Rust
-    encoding, so threads achieve real parallelism.
-
-    ``config`` is the typed renderer config (one of the variants of
-    :data:`renderers.RendererConfig`). Defaults to
-    :class:`AutoRendererConfig`, which resolves to a concrete renderer
-    via ``MODEL_RENDERER_MAP`` at construction time using the loaded
-    tokenizer's name. ``chat_template_kwargs`` are merged into the
-    resolved concrete config and validated before renderer construction.
-    Every slot in the pool shares the same config; to run a different
-    config, build a different pool.
-
-    Tokenizers load via ``load_tokenizer`` — see its docstring for the
-    ``trust_remote_code`` policy (default off; Moonshot Kimi-K2 family
-    opts in with a pinned ``revision``).
-    """
-
-    def factory() -> Renderer:
-        tokenizer = load_tokenizer(tokenizer_name_or_path)
-        return create_renderer(
-            tokenizer,
-            config,
-            chat_template_kwargs=chat_template_kwargs,
-        )
-
-    return RendererPool(factory, size=size)
-
-
 def create_renderer(
     tokenizer,
     config: RendererConfig | None = None,
@@ -1455,7 +1355,8 @@ def create_renderer(
     """Create a Renderer from a typed config.
 
     Args:
-        tokenizer: HuggingFace tokenizer instance.
+        tokenizer: An object satisfying :class:`Tokenizer`; the generic
+            fallback additionally requires :class:`ChatTemplateTokenizer`.
         config: Typed renderer config — one of the variants of
             :data:`renderers.RendererConfig`. ``None`` defaults to
             :class:`AutoRendererConfig`, which resolves to a concrete
@@ -1471,10 +1372,11 @@ def create_renderer(
             config from ``tokenizer.name_or_path`` and then validates these
             kwargs against that config.
 
-    Selecting the auto-renderer for a model without a registered
-    renderer falls back to :class:`DefaultRenderer` for text-only models
-    and raises for VLMs (where ``apply_chat_template`` would silently
-    drop images).
+    Selecting the auto-renderer for a model without a registered renderer
+    probes Hugging Face ``AutoConfig`` before falling back to
+    :class:`DefaultRenderer`, so unknown VLMs fail instead of silently dropping
+    media. Without the ``transformers`` extra, pass an explicit renderer config
+    for unregistered model names.
     """
     _populate_registry()