renderers 0.1.6__tar.gz → 0.1.8.dev0__tar.gz

{renderers-0.1.6 → renderers-0.1.8.dev0}/.github/workflows/publish.yml

@@ -41,6 +41,10 @@ jobs:
             TAG="$PUSHED_REF"
           fi
 
+          # The package version is derived from this tag by hatch-vcs
+          # at build time (see [tool.hatch.version] in pyproject.toml).
+          # We only need to validate the tag shape — there's no
+          # ``project.version`` field to cross-check anymore.
           case "$TAG" in
             renderers-v*) ;;
             *)
@@ -49,21 +53,6 @@ jobs:
               ;;
           esac
 
-          VERSION="${TAG#renderers-v}"
-          FILE_VERSION=$(python - <<'PY'
-          import tomllib
-          from pathlib import Path
-          with Path('pyproject.toml').open('rb') as f:
-              data = tomllib.load(f)
-          print(data['project']['version'])
-          PY
-          )
-
-          if [ "$FILE_VERSION" != "$VERSION" ]; then
-            echo "Version mismatch: tag requests '$VERSION' but pyproject.toml defines '$FILE_VERSION'" >&2
-            exit 1
-          fi
-
           echo "tag=$TAG" >> "$GITHUB_OUTPUT"
 
       - uses: astral-sh/setup-uv@v7

{renderers-0.1.6 → renderers-0.1.8.dev0}/.gitignore

@@ -14,6 +14,8 @@ __pycache__/
 *.pyc
 *.pyo
 *.pyd
+# generated by hatch-vcs at build time (see [tool.hatch.build.hooks.vcs])
+renderers/_version.py
 
 # tooling caches
 .pytest_cache/

renderers-0.1.8.dev0/PKG-INFO

@@ -0,0 +1,156 @@
+Metadata-Version: 2.4
+Name: renderers
+Version: 0.1.8.dev0
+Summary: Chat template renderers — deterministic message-to-token conversion for LLM training
+Requires-Python: <3.14,>=3.10
+Requires-Dist: jinja2
+Requires-Dist: numpy
+Requires-Dist: openai-harmony>=0.0.8
+Requires-Dist: openai>=1.108.1
+Requires-Dist: tiktoken
+Requires-Dist: transformers>=4.50.0
+Description-Content-Type: text/markdown
+
+# renderers
+
+Programmable chat templates for LLM training and inference. A renderer turns a model's chat template into a Python object that can render messages → token ids, parse completion ids → structured assistant messages, and extend a multi-turn rollout without re-rendering model-sampled history.
+
+Standalone on PyPI, and portable across training and inference stacks (transformers, vLLM, SGLang, Tinker). Initially developed for RL training with [verifiers](https://github.com/PrimeIntellect-ai/verifiers) and `prime-rl` at Prime Intellect.
+
+## Install
+
+```bash
+uv add renderers
+```
+
+## At a glance
+
+```python
+from transformers import AutoTokenizer
+from renderers import create_renderer
+
+tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
+r = create_renderer(tok, renderer="auto")           # → Qwen3Renderer
+
+prompt_ids = r.render_ids(
+    [{"role": "user", "content": "hi"}],
+    add_generation_prompt=True,
+)
+# Feed prompt_ids to a Token-In, Token-Out endpoint.
+# It returns completion_ids sampled by the model.
+
+parsed = r.parse_response(completion_ids)
+# ParsedResponse(content=..., reasoning_content=..., tool_calls=...)
+```
+
+For the next turn, extend the previous sampled stream instead of re-rendering history:
+
+```python
+next_prompt_ids = r.bridge_to_next_turn(
+    previous_prompt_ids=prompt_ids,
+    previous_completion_ids=completion_ids,
+    new_messages=[{"role": "tool", "content": "..."}],
+)
+```
+
+Hand-coded renderers ship for `qwen3`, `qwen3-vl`, `qwen3.5`, `qwen3.6`, `glm-5`, `glm-5.1`, `glm-4.5`, `minimax-m2`, `deepseek-v3`, `kimi-k2`, `kimi-k2.5`, `nemotron-3`, `gpt-oss`. Anything else falls back to `DefaultRenderer`, a generic `apply_chat_template` wrapper.
+
+## API
+
+```python
+class Renderer(Protocol):
+    def render(messages, *, tools=None, add_generation_prompt=False) -> RenderedTokens: ...
+    def render_ids(messages, *, tools=None, add_generation_prompt=False) -> list[int]: ...
+    def parse_response(token_ids) -> ParsedResponse: ...
+    def get_stop_token_ids() -> list[int]: ...
+    def bridge_to_next_turn(prev_prompt_ids, prev_completion_ids, new_messages, *, tools=None) -> list[int] | None: ...
+```
+
+- `RenderedTokens` carries `token_ids` **and** `message_indices` — one entry per token attributing each to its source message (`-1` for structural scaffolding). Lets `build_training_sample` build a per-token loss mask in one render.
+- `ParsedResponse` is `(content, reasoning_content, tool_calls)`. It scans token ids for special-token boundaries (e.g. id `151657` for `<tool_call>` on Qwen3) — a literal `"<tool_call>"` in user content tokenizes to ordinary text ids and never matches.
+- Round-trip: rendering `[user, assistant(content, reasoning, tool_calls)]`, slicing the assistant completion, and feeding it through `parse_response` returns an equivalent structured message. Tested per-renderer in `tests/test_roundtrip.py`.
+
+### `bridge_to_next_turn` (the core contract)
+
+Given `(prev_prompt_ids, prev_completion_ids)` and new environment messages, return ids for the next turn's prompt such that the result starts with `prev_prompt_ids + prev_completion_ids` byte-for-byte and continues with the new messages plus the next assistant opener. If that cannot be proven safe, return `None` and the caller falls back to a full render.
+
+Each hand-coded bridge:
+1. Anchors at the previous turn's canonical close token. On clean stops it's already in `prev_completion_ids`. On truncation, the renderer synthesizes the close as non-loss prompt context.
+2. Refuses assistant content in `new_messages` — re-rendering sampled tokens would replace them with canonical template bytes.
+3. Renders only the new messages in the framing the model family expects.
+
+`DefaultRenderer.bridge_to_next_turn` returns `None` unconditionally — the template's close is unknown, so the contract can't be proven.
+
+### Picking a renderer
+
+```python
+r = create_renderer(tok, renderer="auto")
+```
+
+Auto-detect matches `tokenizer.name_or_path` against `MODEL_RENDERER_MAP` by **exact match**. Prefix matching is intentionally off — same architecture can ship different chat templates (base vs instruct, fine-tune renames). Fine-tunes must pass `renderer=<name>` explicitly; unknown names fall back to `DefaultRenderer`.
+
+### Pools
+
+```python
+from renderers import create_renderer_pool
+
+pool = create_renderer_pool("Qwen/Qwen3-8B", renderer="auto", size=16)
+with pool.checkout() as r:
+    ids = r.render_ids(messages)
+```
+
+Each slot owns its own tokenizer copy. Construction fans out across a thread pool so a 32-slot pool doesn't serially eat ~10–15s of `from_pretrained` calls at startup.
+
+## Why use a renderer
+
+For RL the trainer must see the exact token ids the sampler saw. The standard alternative — let the inference engine apply the chat template, parse tool calls, parse reasoning, and re-render full history every turn — silently breaks token identity. These are the failure modes a renderer's `bridge_to_next_turn` sidesteps by never re-rendering prior turns:
+
+- **Boolean round-trip.** Engine emits `false`; client parses to Python `bool(False)`; `apply_chat_template` re-renders via `str(False)` → `"False"`. Capital F. Reproducible on Qwen3.5-35B-A3B + mini-swe-agent-plus at ~50% break rate per rollout.
+- **BPE retokenization drift.** The same substring tokenizes differently depending on neighbouring bytes. `json` + `p` + `enderer` (3 tokens) vs `jsonp` + `enderer` (2 tokens) when whitespace shifts by one character. Every subsequent token is shifted from there on.
+- **Tool-call XML drift.** The engine emits a no-arg call with a stylistic empty `</parameter>`; the Jinja re-render of the reconstructed dict drops it. Extension property broken at every such call.
+- **Thinking stripped from non-latest assistants.** Some templates strip `<think>…</think>` blocks from prior assistant turns when re-rendering. The recorded stream has the thinking; the next prompt does not.
+- **Max-seq-len truncation zeroing the anchor.** Client-side `max_seq_len` enforcement zeros `completion_ids` when `prompt_len > max_seq_len`. The bridge anchor is empty, falling back to full re-render — triggering every mode above.
+- **Scaffold-level history rewriting.** Some agent scaffolds (e.g. opencode's `experimental_repairToolCall`) rewrite tool calls before sending them back as history. The next turn's prompt contains a tool call the model never emitted. *A renderer cannot fix this — the drift happens before rendering.*
+
+Empirical delta on Qwen3.5-35B-A3B + mini-swe-agent-plus, step 0:
+
+| client path                            | breaks | training samples from 64 rollouts |
+| -------------------------------------- | ------ | --------------------------------- |
+| `apply_chat_template` (full re-render) | 32     | 77                                |
+| renderers `bridge_to_next_turn`        | 0      | 64                                |
+
+Each break fragments a rollout into multiple training samples — every fragment re-encodes its prefix, inflating compute roughly linearly with the number of breaks.
+
+## Compaction overrides
+
+`create_renderer` and `create_renderer_pool` accept two constructor-only flags:
+
+```python
+preserve_all_thinking: bool = False
+preserve_thinking_between_tool_calls: bool = False
+```
+
+Defaults preserve byte-identity with the model's chat template. Flipping a flag at construction restores `reasoning_content` the template would otherwise drop:
+
+- `preserve_all_thinking=True` — every past assistant's reasoning is kept.
+- `preserve_thinking_between_tool_calls=True` — reasoning is kept on assistants in the in-flight tool cycle (no-op for current renderers; reserved for future templates that drop it).
+
+The canonical use case is **compaction**. Injecting a `user` turn like *"summarize the work so far"* puts every prior assistant in a "past cycle", so template-default rules drop their `reasoning_content` before the summarizer sees it. Build the renderer with `preserve_all_thinking=True` to keep reasoning visible end-to-end on those flows. Both flags only ever *add* tokens vs the template default.
+
+## `DefaultRenderer`
+
+Fallback for unsupported models. Wraps `apply_chat_template` and accepts `tool_parser` / `reasoning_parser` kwargs (vLLM convention). `bridge_to_next_turn` returns `None` because the template's close is unknown, so multi-turn rollouts fall back to full re-render. Implementing a hand-coded renderer is a few hundred lines of Python (`render_ids` + `parse_response` + `bridge_to_next_turn`) and is the only path that closes the failure modes above by construction.
+
+## Roadmap
+
+- **VLM support.** `ContentPart` is text-only today; `Qwen3VLRenderer` ships only because Qwen3-VL's text-only chat template differs from Qwen3's. Plan: add `ImagePart` / `VideoPart`, multimodal bridges, validate against a Qwen3-VL RL run.
+- **Patched chat templates.** Some shipped templates re-tokenize history, normalize JSON, or auto-strip thinking — each breaks the extension property. Plan: a `use_patched` opt-in per renderer that renders the same surface form while avoiding known-bad patterns.
+
+## Testing
+
+```bash
+uv sync --group dev
+uv run pytest
+```
+
+Round-trip parity (render → parse → original) and token-level parity against `apply_chat_template` are tested per renderer. End-to-end validation runs against Reverse-Text, Wordle, OpenCode-Math, and RLM-SWE environments.

renderers-0.1.8.dev0/README.md

@@ -0,0 +1,143 @@
+# renderers
+
+Programmable chat templates for LLM training and inference. A renderer turns a model's chat template into a Python object that can render messages → token ids, parse completion ids → structured assistant messages, and extend a multi-turn rollout without re-rendering model-sampled history.
+
+Standalone on PyPI, and portable across training and inference stacks (transformers, vLLM, SGLang, Tinker). Initially developed for RL training with [verifiers](https://github.com/PrimeIntellect-ai/verifiers) and `prime-rl` at Prime Intellect.
+
+## Install
+
+```bash
+uv add renderers
+```
+
+## At a glance
+
+```python
+from transformers import AutoTokenizer
+from renderers import create_renderer
+
+tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
+r = create_renderer(tok, renderer="auto")           # → Qwen3Renderer
+
+prompt_ids = r.render_ids(
+    [{"role": "user", "content": "hi"}],
+    add_generation_prompt=True,
+)
+# Feed prompt_ids to a Token-In, Token-Out endpoint.
+# It returns completion_ids sampled by the model.
+
+parsed = r.parse_response(completion_ids)
+# ParsedResponse(content=..., reasoning_content=..., tool_calls=...)
+```
+
+For the next turn, extend the previous sampled stream instead of re-rendering history:
+
+```python
+next_prompt_ids = r.bridge_to_next_turn(
+    previous_prompt_ids=prompt_ids,
+    previous_completion_ids=completion_ids,
+    new_messages=[{"role": "tool", "content": "..."}],
+)
+```
+
+Hand-coded renderers ship for `qwen3`, `qwen3-vl`, `qwen3.5`, `qwen3.6`, `glm-5`, `glm-5.1`, `glm-4.5`, `minimax-m2`, `deepseek-v3`, `kimi-k2`, `kimi-k2.5`, `nemotron-3`, `gpt-oss`. Anything else falls back to `DefaultRenderer`, a generic `apply_chat_template` wrapper.
+
+## API
+
+```python
+class Renderer(Protocol):
+    def render(messages, *, tools=None, add_generation_prompt=False) -> RenderedTokens: ...
+    def render_ids(messages, *, tools=None, add_generation_prompt=False) -> list[int]: ...
+    def parse_response(token_ids) -> ParsedResponse: ...
+    def get_stop_token_ids() -> list[int]: ...
+    def bridge_to_next_turn(prev_prompt_ids, prev_completion_ids, new_messages, *, tools=None) -> list[int] | None: ...
+```
+
+- `RenderedTokens` carries `token_ids` **and** `message_indices` — one entry per token attributing each to its source message (`-1` for structural scaffolding). Lets `build_training_sample` build a per-token loss mask in one render.
+- `ParsedResponse` is `(content, reasoning_content, tool_calls)`. It scans token ids for special-token boundaries (e.g. id `151657` for `<tool_call>` on Qwen3) — a literal `"<tool_call>"` in user content tokenizes to ordinary text ids and never matches.
+- Round-trip: rendering `[user, assistant(content, reasoning, tool_calls)]`, slicing the assistant completion, and feeding it through `parse_response` returns an equivalent structured message. Tested per-renderer in `tests/test_roundtrip.py`.
+
+### `bridge_to_next_turn` (the core contract)
+
+Given `(prev_prompt_ids, prev_completion_ids)` and new environment messages, return ids for the next turn's prompt such that the result starts with `prev_prompt_ids + prev_completion_ids` byte-for-byte and continues with the new messages plus the next assistant opener. If that cannot be proven safe, return `None` and the caller falls back to a full render.
+
+Each hand-coded bridge:
+1. Anchors at the previous turn's canonical close token. On clean stops it's already in `prev_completion_ids`. On truncation, the renderer synthesizes the close as non-loss prompt context.
+2. Refuses assistant content in `new_messages` — re-rendering sampled tokens would replace them with canonical template bytes.
+3. Renders only the new messages in the framing the model family expects.
+
+`DefaultRenderer.bridge_to_next_turn` returns `None` unconditionally — the template's close is unknown, so the contract can't be proven.
+
+### Picking a renderer
+
+```python
+r = create_renderer(tok, renderer="auto")
+```
+
+Auto-detect matches `tokenizer.name_or_path` against `MODEL_RENDERER_MAP` by **exact match**. Prefix matching is intentionally off — same architecture can ship different chat templates (base vs instruct, fine-tune renames). Fine-tunes must pass `renderer=<name>` explicitly; unknown names fall back to `DefaultRenderer`.
+
+### Pools
+
+```python
+from renderers import create_renderer_pool
+
+pool = create_renderer_pool("Qwen/Qwen3-8B", renderer="auto", size=16)
+with pool.checkout() as r:
+    ids = r.render_ids(messages)
+```
+
+Each slot owns its own tokenizer copy. Construction fans out across a thread pool so a 32-slot pool doesn't serially eat ~10–15s of `from_pretrained` calls at startup.
+
+## Why use a renderer
+
+For RL the trainer must see the exact token ids the sampler saw. The standard alternative — let the inference engine apply the chat template, parse tool calls, parse reasoning, and re-render full history every turn — silently breaks token identity. These are the failure modes a renderer's `bridge_to_next_turn` sidesteps by never re-rendering prior turns:
+
+- **Boolean round-trip.** Engine emits `false`; client parses to Python `bool(False)`; `apply_chat_template` re-renders via `str(False)` → `"False"`. Capital F. Reproducible on Qwen3.5-35B-A3B + mini-swe-agent-plus at ~50% break rate per rollout.
+- **BPE retokenization drift.** The same substring tokenizes differently depending on neighbouring bytes. `json` + `p` + `enderer` (3 tokens) vs `jsonp` + `enderer` (2 tokens) when whitespace shifts by one character. Every subsequent token is shifted from there on.
+- **Tool-call XML drift.** The engine emits a no-arg call with a stylistic empty `</parameter>`; the Jinja re-render of the reconstructed dict drops it. Extension property broken at every such call.
+- **Thinking stripped from non-latest assistants.** Some templates strip `<think>…</think>` blocks from prior assistant turns when re-rendering. The recorded stream has the thinking; the next prompt does not.
+- **Max-seq-len truncation zeroing the anchor.** Client-side `max_seq_len` enforcement zeros `completion_ids` when `prompt_len > max_seq_len`. The bridge anchor is empty, falling back to full re-render — triggering every mode above.
+- **Scaffold-level history rewriting.** Some agent scaffolds (e.g. opencode's `experimental_repairToolCall`) rewrite tool calls before sending them back as history. The next turn's prompt contains a tool call the model never emitted. *A renderer cannot fix this — the drift happens before rendering.*
+
+Empirical delta on Qwen3.5-35B-A3B + mini-swe-agent-plus, step 0:
+
+| client path                            | breaks | training samples from 64 rollouts |
+| -------------------------------------- | ------ | --------------------------------- |
+| `apply_chat_template` (full re-render) | 32     | 77                                |
+| renderers `bridge_to_next_turn`        | 0      | 64                                |
+
+Each break fragments a rollout into multiple training samples — every fragment re-encodes its prefix, inflating compute roughly linearly with the number of breaks.
+
+## Compaction overrides
+
+`create_renderer` and `create_renderer_pool` accept two constructor-only flags:
+
+```python
+preserve_all_thinking: bool = False
+preserve_thinking_between_tool_calls: bool = False
+```
+
+Defaults preserve byte-identity with the model's chat template. Flipping a flag at construction restores `reasoning_content` the template would otherwise drop:
+
+- `preserve_all_thinking=True` — every past assistant's reasoning is kept.
+- `preserve_thinking_between_tool_calls=True` — reasoning is kept on assistants in the in-flight tool cycle (no-op for current renderers; reserved for future templates that drop it).
+
+The canonical use case is **compaction**. Injecting a `user` turn like *"summarize the work so far"* puts every prior assistant in a "past cycle", so template-default rules drop their `reasoning_content` before the summarizer sees it. Build the renderer with `preserve_all_thinking=True` to keep reasoning visible end-to-end on those flows. Both flags only ever *add* tokens vs the template default.
+
+## `DefaultRenderer`
+
+Fallback for unsupported models. Wraps `apply_chat_template` and accepts `tool_parser` / `reasoning_parser` kwargs (vLLM convention). `bridge_to_next_turn` returns `None` because the template's close is unknown, so multi-turn rollouts fall back to full re-render. Implementing a hand-coded renderer is a few hundred lines of Python (`render_ids` + `parse_response` + `bridge_to_next_turn`) and is the only path that closes the failure modes above by construction.
+
+## Roadmap
+
+- **VLM support.** `ContentPart` is text-only today; `Qwen3VLRenderer` ships only because Qwen3-VL's text-only chat template differs from Qwen3's. Plan: add `ImagePart` / `VideoPart`, multimodal bridges, validate against a Qwen3-VL RL run.
+- **Patched chat templates.** Some shipped templates re-tokenize history, normalize JSON, or auto-strip thinking — each breaks the extension property. Plan: a `use_patched` opt-in per renderer that renders the same surface form while avoiding known-bad patterns.
+
+## Testing
+
+```bash
+uv sync --group dev
+uv run pytest
+```
+
+Round-trip parity (render → parse → original) and token-level parity against `apply_chat_template` are tested per renderer. End-to-end validation runs against Reverse-Text, Wordle, OpenCode-Math, and RLM-SWE environments.

renderers-0.1.8.dev0/examples/README.md

@@ -0,0 +1,72 @@
+# Offline Renderer Inference Examples
+
+Each recipe keeps chat templating in `renderers` and sends token IDs to the
+backend:
+
+1. Load a Hugging Face tokenizer.
+2. Build a model-specific `Renderer`.
+3. Render chat messages to prompt token IDs locally.
+4. Pass token IDs directly to an offline inference engine.
+5. Parse completion token IDs with the same renderer.
+6. Bridge the next turn without re-rendering prior assistant output.
+
+The scripts use PEP 723 `uv` headers, so backend dependencies stay local to the
+recipe and do not touch the repo `uv.lock`.
+
+## vLLM Multi-Turn Recipe
+
+```bash
+CUDA_VISIBLE_DEVICES=0 uv run --script examples/vllm/multiturn_generate_vllm.py
+```
+
+The vLLM script targets `vllm>=0.20` and uses `prompt_token_ids`, so vLLM
+does not apply a chat template.
+
+## SGLang Multi-Turn Recipe
+
+```bash
+CUDA_VISIBLE_DEVICES=1 uv run --script examples/sglang/multiturn_generate_sglang.py
+```
+
+The SGLang script uses `input_ids`, so SGLang does not apply a chat template.
+It leaves `openai-harmony` at SGLang's pinned version for dependency resolution.
+
+## Transformers Multi-Turn Recipe
+
+```bash
+CUDA_VISIBLE_DEVICES=0 uv run --script examples/transformers/multiturn_generate_transformers.py
+```
+
+The Transformers script calls `generate()` with `input_ids`, so Transformers
+does not apply a chat template.
+
+## Tinker Multi-Turn Recipe
+
+```bash
+TINKER_API_KEY=... uv run --script examples/tinker/multiturn_generate_tinker.py
+```
+
+The Tinker script sends renderer-produced token IDs as `ModelInput` to the
+remote sampling API, so Tinker does not apply a chat template.
+
+## Two-GPU Validation
+
+Run the recipes in parallel, one backend per GPU:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 uv run --script examples/vllm/multiturn_generate_vllm.py \
+  --max-new-tokens 512 &
+
+CUDA_VISIBLE_DEVICES=1 uv run --script examples/sglang/multiturn_generate_sglang.py \
+  --max-new-tokens 512 &
+
+wait
+```
+
+Each script runs `Qwen/Qwen3.5-4B` with `enable_thinking=True` and `False`, then
+`openai/gpt-oss-20b`.
+
+## Multimodal Note
+
+Renderers are text-only today. For image/video demos, use the backend's message
+or prompt path until renderers grow multimodal placeholder support.

renderers-0.1.8.dev0/examples/sglang/multiturn_generate_sglang.py

@@ -0,0 +1,187 @@
+#!/usr/bin/env -S uv run --script
+# /// script
+# requires-python = ">=3.10,<3.14"
+# dependencies = [
+#   "renderers>=0.1.6",
+#   "sglang==0.5.10.post1",
+#   "flash-attn-4>=4.0.0b4",
+#   "transformers>=5.3.0",
+#   "openai-harmony==0.0.4",
+#   "openai>=1.108.1",
+#   "tiktoken",
+#   "jinja2",
+#   "numpy",
+# ]
+# ///
+"""SGLang offline generation from renderer-owned prompt token IDs."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+
+import sglang as sgl
+from renderers.gpt_oss import GptOssRenderer
+from renderers.qwen35 import Qwen35Renderer
+from transformers import AutoTokenizer
+
+
+MODELS = ["Qwen/Qwen3.5-4B", "openai/gpt-oss-20b"]
+QWEN_THINKING_MODES = [True, False]
+
+TOOLS = [
+    {
+        "type": "function",
+        "function": {
+            "name": "multiply",
+            "description": "Multiply two integers.",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "a": {"type": "integer"},
+                    "b": {"type": "integer"},
+                },
+                "required": ["a", "b"],
+            },
+        },
+    }
+]
+
+
+def make_renderer(model: str, enable_thinking: bool | None):
+    tokenizer = AutoTokenizer.from_pretrained(model, trust_remote_code=False)
+    if model.startswith("Qwen/Qwen3.5-"):
+        return Qwen35Renderer(tokenizer, enable_thinking=enable_thinking)
+    if model == "openai/gpt-oss-20b":
+        return GptOssRenderer(tokenizer)
+    raise ValueError(f"unsupported demo model: {model}")
+
+
+def print_parsed(label: str, turn: str, parsed) -> None:
+    print(f"\n[{label}] {turn}")
+    if parsed.reasoning_content:
+        print(f"reasoning: {parsed.reasoning_content[:240]}")
+    if parsed.tool_calls:
+        print(f"tool_calls: {json.dumps(parsed.tool_calls, ensure_ascii=False)}")
+    if parsed.content:
+        print(f"content: {parsed.content}")
+
+
+def completion_ids(output: dict, prompt_ids: list[int]) -> list[int]:
+    ids = list(output.get("output_ids") or output.get("token_ids") or [])
+    if not ids:
+        raise RuntimeError("SGLang did not return completion token IDs")
+    return ids[len(prompt_ids) :] if ids[: len(prompt_ids)] == prompt_ids else ids
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--max-new-tokens", type=int, default=512)
+    parser.add_argument("--context-length", type=int, default=4096)
+    args = parser.parse_args()
+
+    print(f"CUDA_VISIBLE_DEVICES={os.environ.get('CUDA_VISIBLE_DEVICES', '<unset>')}")
+
+    targets = []
+    for model in MODELS:
+        if model.startswith("Qwen/Qwen3.5-"):
+            for enable_thinking in QWEN_THINKING_MODES:
+                targets.append((model, enable_thinking))
+        else:
+            targets.append((model, None))
+
+    for model, enable_thinking in targets:
+        label = (
+            model
+            if enable_thinking is None
+            else f"{model} enable_thinking={enable_thinking}"
+        )
+        print(f"\n=== {label} ===")
+
+        renderer = make_renderer(model, enable_thinking)
+
+        engine_kwargs = {
+            "model_path": model,
+            "trust_remote_code": False,
+            "context_length": args.context_length,
+            "attention_backend": "triton",
+        }
+        if model == "openai/gpt-oss-20b":
+            engine_kwargs["moe_runner_backend"] = "triton"
+        engine = sgl.Engine(**engine_kwargs)
+
+        sampling = {
+            "temperature": 0.0,
+            "max_new_tokens": args.max_new_tokens,
+            "stop_token_ids": renderer.get_stop_token_ids(),
+            "skip_special_tokens": False,
+            "no_stop_trim": True,
+        }
+
+        messages = [
+            {"role": "system", "content": "You are a concise tool-using assistant."},
+            {
+                "role": "user",
+                "content": "Use the multiply tool for 17 * 23, then summarize.",
+            },
+        ]
+
+        # Turn 1: render locally and pass token IDs to SGLang. SGLang never
+        # sees messages and never applies a chat template.
+        prompt_ids = renderer.render_ids(
+            messages, tools=TOOLS, add_generation_prompt=True
+        )
+        output1 = engine.generate(input_ids=prompt_ids, sampling_params=sampling)
+        completion1 = completion_ids(output1, prompt_ids)
+        parsed1 = renderer.parse_response(completion1)
+        print_parsed(label, "turn 1", parsed1)
+
+        assistant = {"role": "assistant", "content": parsed1.content}
+        if parsed1.reasoning_content:
+            assistant["reasoning_content"] = parsed1.reasoning_content
+        if parsed1.tool_calls:
+            assistant["tool_calls"] = parsed1.tool_calls
+        messages.append(assistant)
+
+        if parsed1.tool_calls:
+            new_messages = []
+            for idx, tool_call in enumerate(parsed1.tool_calls):
+                fn = tool_call.get("function") or tool_call
+                tool_args = fn.get("arguments") or {}
+                if isinstance(tool_args, str):
+                    tool_args = json.loads(tool_args)
+                new_messages.append(
+                    {
+                        "role": "tool",
+                        "tool_call_id": tool_call.get("id", f"call_{idx}"),
+                        "name": fn.get("name", "multiply"),
+                        "content": json.dumps(
+                            {"result": int(tool_args["a"]) * int(tool_args["b"])}
+                        ),
+                    }
+                )
+        else:
+            new_messages = [
+                {"role": "user", "content": "Give the final answer in one sentence."}
+            ]
+
+        # Turn 2: bridge extends prompt_ids + completion1 exactly.
+        bridged_ids = renderer.bridge_to_next_turn(
+            prompt_ids, completion1, new_messages, tools=TOOLS
+        )
+        if bridged_ids is None:
+            raise RuntimeError("bridge_to_next_turn returned None")
+        assert bridged_ids[: len(prompt_ids) + len(completion1)] == (
+            prompt_ids + completion1
+        )
+
+        output2 = engine.generate(input_ids=bridged_ids, sampling_params=sampling)
+        completion2 = completion_ids(output2, bridged_ids)
+        print_parsed(label, "turn 2", renderer.parse_response(completion2))
+
+        engine.shutdown()
+
+
+if __name__ == "__main__":
+    main()