logit-classifier 0.2.1__tar.gz → 0.3.0__tar.gz

{logit_classifier-0.2.1 → logit_classifier-0.3.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.5
 Name: logit-classifier
-Version: 0.2.1
-Summary: Local zero-shot classifier for text and images. Declares options, returns a calibrated probability for each, generates no text. Accepts the TypeSafe System One request format.
+Version: 0.3.0
+Summary: Local zero-shot classifier for text and images. Declares options and reads a calibrated probability for each from the logits. Accepts the TypeSafe System One request format, and ships a toolkit of image tag tools and ComfyUI node tools.
 Project-URL: Repository, https://github.com/Blakeem/logit-classifier
 Project-URL: Bug Tracker, https://github.com/Blakeem/logit-classifier/issues
 Author: Blake
@@ -124,6 +124,14 @@ print(response.to_dict())
 `load_model` loads the weights through the `hf` extra and returns a `Backend`.
 `Classifier(Config())` calls it for you when you pass no backend.
 
+`classifier.nouls` asks a list of statements about one state and returns one probability
+per statement. It sends more than 256 statements as several requests.
+
+```python
+probabilities = classifier.nouls(["The message is urgent", "The message is polite"],
+                                 state="the payment failed again")
+```
+
 Loading each model yourself is what lets one script compare several. The fitted
 temperature follows the backend, so every model is scored with its own value.
 
@@ -170,7 +178,7 @@ the model.
 
 ```json
 {
-  "model": "logit-classifier-0.2.1",
+  "model": "logit-classifier-0.3.0",
   "answers": {
     "department": {
       "type": "choice",
@@ -342,6 +350,72 @@ shape `[1, H, W, 3]`. An empty state asks about the image alone. Questions share
 pass of up to 4,096 tokens, and the image counts toward that limit. Any text encoder other
 than Qwen3-VL raises `UnsupportedModelError`.
 
+## Toolkit
+
+`logit_classifier.toolkit` holds tools built on the classifier with the settings our
+measurements found best. A project imports them to get good results without working out the
+wording, thresholds and host details itself. The toolkit uses the classifier's public API,
+and the classifier never imports the toolkit.
+
+| Package | Runs in | Holds |
+|---|---|---|
+| `logit_classifier.toolkit.tags` | any Python process | text tools for image tags and prompts |
+| `logit_classifier.toolkit.comfyui` | a ComfyUI node | a classifier over the workflow's CLIP, text generation, a progress display and an image tagger |
+
+### Tag Text Tools
+
+| Tool | Does |
+|---|---|
+| `parse_candidates` | splits a model's tag reply into clean tags with no duplicates |
+| `split_prompt` | splits an image prompt into its fragments |
+| `drop_subsets` | drops a tag whose words all appear in a longer tag |
+| `prompt_word_forms`, `in_prompt` | test whether a tag's words appear in a prompt, singular or plural |
+| `merge_candidates` | merges proposed tags with a prompt's tags under a cap |
+| `presence_question` | writes "Is there a car in this image?" with the article the thing takes |
+
+```python
+from logit_classifier.toolkit.tags import drop_subsets, parse_candidates
+
+tags = drop_subsets(parse_candidates("red car, car, wooden bench, bench"))
+# ['red car', 'wooden bench']
+```
+
+`drop_subsets(tags, rule="head-noun")` drops a tag only into a longer tag with the same head
+noun. The head noun is the last word before "of", "in", "on", "with" or "reading". So "cat"
+stays beside "cat ear", and "signage" is dropped beside "signage in chinese".
+
+### ComfyUI Tools
+
+These run inside a ComfyUI node over the CLIP the workflow loaded. Each one imports comfy and
+torch only when it is called.
+
+| Tool | Does |
+|---|---|
+| `comfy_classifier` | builds a classifier over the CLIP, with errors that name the node and the Load CLIP fix |
+| `generate_text` | generates one greedy reply with the Qwen chat template, and stops early on a condition |
+| `shared_vision_encode` | runs the vision tower once per picture for every generate and classify inside the block |
+| `skip_resident_loads` | skips core's model load while the CLIP is already on the GPU |
+| `routed_progress_bars`, `send_status`, `run_outcome` | show one progress bar per run and a status line under the node |
+| `fit_picture` | downscales an image to a pixel cap |
+| `prompt_tags` | lists the physical things a prompt names |
+| `tag_picture` | tags a picture, with a prompt's things added as candidates |
+| `TagSettings` | holds every wording, budget, cap and threshold of the tagger |
+
+The tagger generates candidate tags over the CLIP and verifies each one with the classifier.
+
+```python
+from logit_classifier.toolkit.comfyui import TagSettings, comfy_classifier, fit_picture, tag_picture
+
+classifier = comfy_classifier(clip, node="My Tagger")
+picture = fit_picture(image, 1024 * 1024)
+trace = tag_picture(clip, classifier, picture, settings=TagSettings(), known={})
+tags = trace.kept
+```
+
+`known` holds the thing check's answers across a run, so each tag is asked once. The
+`TagSettings` defaults are the current measured values, and a later release may change them.
+Pass every value your output depends on to keep it fixed.
+
 ## Differences From Jev
 
 This project matches the System One request and response format and the documented limits.

{logit_classifier-0.2.1 → logit_classifier-0.3.0}/README.md

@@ -91,6 +91,14 @@ print(response.to_dict())
 `load_model` loads the weights through the `hf` extra and returns a `Backend`.
 `Classifier(Config())` calls it for you when you pass no backend.
 
+`classifier.nouls` asks a list of statements about one state and returns one probability
+per statement. It sends more than 256 statements as several requests.
+
+```python
+probabilities = classifier.nouls(["The message is urgent", "The message is polite"],
+                                 state="the payment failed again")
+```
+
 Loading each model yourself is what lets one script compare several. The fitted
 temperature follows the backend, so every model is scored with its own value.
 
@@ -137,7 +145,7 @@ the model.
 
 ```json
 {
-  "model": "logit-classifier-0.2.1",
+  "model": "logit-classifier-0.3.0",
   "answers": {
     "department": {
       "type": "choice",
@@ -309,6 +317,72 @@ shape `[1, H, W, 3]`. An empty state asks about the image alone. Questions share
 pass of up to 4,096 tokens, and the image counts toward that limit. Any text encoder other
 than Qwen3-VL raises `UnsupportedModelError`.
 
+## Toolkit
+
+`logit_classifier.toolkit` holds tools built on the classifier with the settings our
+measurements found best. A project imports them to get good results without working out the
+wording, thresholds and host details itself. The toolkit uses the classifier's public API,
+and the classifier never imports the toolkit.
+
+| Package | Runs in | Holds |
+|---|---|---|
+| `logit_classifier.toolkit.tags` | any Python process | text tools for image tags and prompts |
+| `logit_classifier.toolkit.comfyui` | a ComfyUI node | a classifier over the workflow's CLIP, text generation, a progress display and an image tagger |
+
+### Tag Text Tools
+
+| Tool | Does |
+|---|---|
+| `parse_candidates` | splits a model's tag reply into clean tags with no duplicates |
+| `split_prompt` | splits an image prompt into its fragments |
+| `drop_subsets` | drops a tag whose words all appear in a longer tag |
+| `prompt_word_forms`, `in_prompt` | test whether a tag's words appear in a prompt, singular or plural |
+| `merge_candidates` | merges proposed tags with a prompt's tags under a cap |
+| `presence_question` | writes "Is there a car in this image?" with the article the thing takes |
+
+```python
+from logit_classifier.toolkit.tags import drop_subsets, parse_candidates
+
+tags = drop_subsets(parse_candidates("red car, car, wooden bench, bench"))
+# ['red car', 'wooden bench']
+```
+
+`drop_subsets(tags, rule="head-noun")` drops a tag only into a longer tag with the same head
+noun. The head noun is the last word before "of", "in", "on", "with" or "reading". So "cat"
+stays beside "cat ear", and "signage" is dropped beside "signage in chinese".
+
+### ComfyUI Tools
+
+These run inside a ComfyUI node over the CLIP the workflow loaded. Each one imports comfy and
+torch only when it is called.
+
+| Tool | Does |
+|---|---|
+| `comfy_classifier` | builds a classifier over the CLIP, with errors that name the node and the Load CLIP fix |
+| `generate_text` | generates one greedy reply with the Qwen chat template, and stops early on a condition |
+| `shared_vision_encode` | runs the vision tower once per picture for every generate and classify inside the block |
+| `skip_resident_loads` | skips core's model load while the CLIP is already on the GPU |
+| `routed_progress_bars`, `send_status`, `run_outcome` | show one progress bar per run and a status line under the node |
+| `fit_picture` | downscales an image to a pixel cap |
+| `prompt_tags` | lists the physical things a prompt names |
+| `tag_picture` | tags a picture, with a prompt's things added as candidates |
+| `TagSettings` | holds every wording, budget, cap and threshold of the tagger |
+
+The tagger generates candidate tags over the CLIP and verifies each one with the classifier.
+
+```python
+from logit_classifier.toolkit.comfyui import TagSettings, comfy_classifier, fit_picture, tag_picture
+
+classifier = comfy_classifier(clip, node="My Tagger")
+picture = fit_picture(image, 1024 * 1024)
+trace = tag_picture(clip, classifier, picture, settings=TagSettings(), known={})
+tags = trace.kept
+```
+
+`known` holds the thing check's answers across a run, so each tag is asked once. The
+`TagSettings` defaults are the current measured values, and a later release may change them.
+Pass every value your output depends on to keep it fixed.
+
 ## Differences From Jev
 
 This project matches the System One request and response format and the documented limits.

{logit_classifier-0.2.1 → logit_classifier-0.3.0}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "logit-classifier"
 dynamic = ["version"]
-description = "Local zero-shot classifier for text and images. Declares options, returns a calibrated probability for each, generates no text. Accepts the TypeSafe System One request format."
+description = "Local zero-shot classifier for text and images. Declares options and reads a calibrated probability for each from the logits. Accepts the TypeSafe System One request format, and ships a toolkit of image tag tools and ComfyUI node tools."
 requires-python = ">=3.12"
 license = "GPL-3.0-or-later"
 license-files = ["LICENSE"]
@@ -136,8 +136,8 @@ show_error_codes = true
 [[tool.mypy.overrides]]
 # transformers publishes no complete stubs, and accelerate none at all. torch is
 # listed so the type gate still runs where the hf extra is absent, such as CI.
-# The ComfyUI host supplies comfy, and neither this venv nor CI has it.
-module = ["transformers.*", "accelerate.*", "torch.*", "comfy.*"]
+# The ComfyUI host supplies comfy, server and comfy_execution, and neither this venv nor CI has them.
+module = ["transformers.*", "accelerate.*", "torch.*", "comfy.*", "server", "server.*", "comfy_execution.*"]
 ignore_missing_imports = true
 
 [[tool.mypy.overrides]]

{logit_classifier-0.2.1 → logit_classifier-0.3.0}/src/logit_classifier/_version.py

@@ -1,3 +1,3 @@
 """The package version, apart from __init__ so config.py can read it without a cycle."""
 
-__version__ = "0.2.1"
+__version__ = "0.3.0"

{logit_classifier-0.2.1 → logit_classifier-0.3.0}/src/logit_classifier/backends/comfy_clip.py

@@ -101,6 +101,29 @@ def _check_image(image: Any) -> None:
         raise ImageError(f"a ComfyUI IMAGE of shape [1, H, W, 3] is required, got shape {shape}")
 
 
+def _resident(clip: Any) -> bool:
+    """Whether the CLIP sits where its own load_model call leaves it, so loading again is a no-op.
+
+    Core's load_models_gpu has no fast path for a loaded model and costs about 0.1 s per call.
+    """
+    try:
+        import comfy.model_management as model_management
+
+        patcher = clip.patcher
+        loaded = model_management.current_loaded_models
+        # Another model's load always inserts at the head, and a changed LoRA patch set or an
+        # offloaded CLIP must reload.
+        return bool(
+            loaded
+            and loaded[0].model is patcher
+            and patcher.model.device == patcher.load_device
+            and patcher.model.current_weight_patches_uuid == patcher.patches_uuid
+            and patcher.model.model_loaded_weight_memory > 0
+        )
+    except (ImportError, AttributeError):
+        return False
+
+
 def _pack_passes(prefix_length: int, suffix_lengths: list[int], batch_branches: bool) -> list[list[int]]:
     """Group suffixes in request order into passes that stay under PACKED_TOKEN_CEILING.
 
@@ -284,7 +307,8 @@ class ComfyClipBackend:
             prefix_tokens[vision["pad_index"]] = {"type": "image", "data": vision["image"], "original_type": "image"}
         with _determinism(), torch.inference_mode():
             encoder.reset_clip_options()
-            clip.load_model({self._tokens_key: [prefix_tokens]})
+            if not _resident(clip):
+                clip.load_model({self._tokens_key: [prefix_tokens]})
             device = clip.patcher.load_device
             encoder.set_clip_options({"layer": None, "execution_device": device})
             # BaseGenerate.generate picks its execution dtype this way, at llama.py:1115-1120.

{logit_classifier-0.2.1 → logit_classifier-0.3.0}/src/logit_classifier/classifier.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 import hashlib
 import random
+from collections.abc import Sequence
 from dataclasses import dataclass, field, replace
 from typing import Any
 
@@ -11,7 +12,7 @@ import numpy as np
 
 from .backends.base import Backend, BackendContractError, BranchLogits
 from .calibrate import PriorStore
-from .config import ANSWER_PREFILL, Config, fitted_temperature
+from .config import ANSWER_PREFILL, JEV_MAX_QUESTIONS, Config, fitted_temperature
 from .deps import MissingDependencyError
 from .prompt import (
     PROMPT_VERSION,
@@ -25,6 +26,7 @@ from .schema import (
     Answer,
     ChoiceAnswer,
     ChoiceQuestion,
+    JSONContent,
     NoulAnswer,
     NoulQuestion,
     Question,
@@ -279,6 +281,28 @@ class Classifier:
         )
         return response, diagnostics
 
+    def nouls(self, statements: Sequence[str], *, image: Any = None, state: JSONContent = "") -> list[float]:
+        """Return each statement's noul about the state, in statement order.
+
+        The statements are split into requests of JEV_MAX_QUESTIONS, one classify call each, and no
+        statements make no request. An empty state with an image is the image-only state.
+        """
+        keys = [f"s{index}" for index in range(len(statements))]
+        questions: dict[str, Question] = {}
+        nouls: list[float] = []
+
+        for start in range(0, len(statements), JEV_MAX_QUESTIONS):
+            window = slice(start, start + JEV_MAX_QUESTIONS)
+            questions = {key: NoulQuestion(instructions=statement)
+                         for key, statement in zip(keys[window], statements[window], strict=True)}
+            response, _diagnostics = self.classify(SystemOneRequest(state=state, questions=questions), image=image)
+            for key in questions:
+                answer = response.answers[key]
+                if not isinstance(answer, NoulAnswer):
+                    raise TypeError(f"question {key} got a {answer.type} answer, not a noul")
+                nouls.append(answer.noul)
+        return nouls
+
     def _round_answers(self, questions: dict[str, Question], branches: list[Branch],
                        probabilities: list[np.ndarray]) -> dict[str, Answer]:
         answers: dict[str, Answer] = {}

{logit_classifier-0.2.1 → logit_classifier-0.3.0}/src/logit_classifier/prompt.py

@@ -52,7 +52,7 @@ class Branch:
     suffix_text: str
 
 
-def _inert(text: str) -> str:
+def inert(text: str) -> str:
     # The backends' tokenizers read a <|...|> spelling in plain text as a control token, so
     # client text could otherwise forge a chat turn or a second image pad.
     return text.replace("<|", "<\u200b|")
@@ -67,7 +67,7 @@ def _option_lines(entries: list[tuple[str, str]]) -> str:
 
 
 def _question_block(prompt_line: str, entries: list[tuple[str, str]]) -> str:
-    return _inert(f"{prompt_line}\nOptions:\n{_option_lines(entries)}")
+    return inert(f"{prompt_line}\nOptions:\n{_option_lines(entries)}")
 
 
 def _choice_branches(qid: str, question: ChoiceQuestion, plan: BranchPlan,
@@ -152,7 +152,7 @@ def build_branches(questions: dict[str, Question], score_method: str = "joint",
 def prefix_content(state: Any, has_image: bool = False) -> str:
     """Build the user-message body every branch shares, image marker included."""
     marker = f"{IMAGE_MARKER}\n" if has_image else ""
-    return f"Context:\n{marker}{_inert(render_content(state))}\n\n"
+    return f"Context:\n{marker}{inert(render_content(state))}\n\n"
 
 
 def branch_content(state: Any, branch: Branch, has_image: bool = False) -> str:

logit_classifier-0.3.0/src/logit_classifier/tags.py

@@ -0,0 +1,36 @@
+"""The tag text tools, which moved to logit_classifier.toolkit.tags in 0.3.0.
+
+This path stays for code built on 0.2.x, which PyPI published. It raises no warning, so that code keeps a quiet log.
+"""
+
+from __future__ import annotations
+
+from .toolkit.tags import (
+    MAX_CANDIDATES,
+    MAX_REPEAT_BLOCK,
+    MAX_TAG_CHARS,
+    MAX_TAG_WORDS,
+    clean_item,
+    complete_tags,
+    drop_subsets,
+    drop_unfinished_tag,
+    normalize_item,
+    parse_candidates,
+    repeated_block,
+    split_prompt,
+)
+
+__all__ = [
+    "MAX_CANDIDATES",
+    "MAX_REPEAT_BLOCK",
+    "MAX_TAG_CHARS",
+    "MAX_TAG_WORDS",
+    "clean_item",
+    "complete_tags",
+    "drop_subsets",
+    "drop_unfinished_tag",
+    "normalize_item",
+    "parse_candidates",
+    "repeated_block",
+    "split_prompt",
+]

logit_classifier-0.3.0/src/logit_classifier/toolkit/__init__.py

@@ -0,0 +1,5 @@
+"""Tools built on the classifier core that follow its measured best practice.
+
+A caller gets good results from these tools without working the details out from the base API.
+The tools use the core and never mix into it, so the core never imports this package.
+"""

logit_classifier-0.3.0/src/logit_classifier/toolkit/comfyui/__init__.py

@@ -0,0 +1,50 @@
+"""The tools for a ComfyUI node that holds a loaded CLIP.
+
+Every module here imports comfy, server, torch and comfy_execution inside the functions that need them, so importing
+this package in a plain Python process loads none of them.
+"""
+
+from __future__ import annotations
+
+from ... import COMFY_SOCKET_TYPE
+from ...backends.comfy_clip import PACKED_TOKEN_CEILING, ComfyClipBackend
+from .classifier import comfy_classifier
+from .generate import IMAGE_SLOT, chat, clip_generate, generate_text, stop_when
+from .progress import routed_progress_bars, run_outcome, send_status, token_fill
+from .scopes import shared_vision_encode, skip_resident_loads
+from .tagger import (
+    PromptTags,
+    TagSettings,
+    TagTrace,
+    english_text,
+    fit_picture,
+    prompt_tags,
+    tag_picture,
+    thing_scores,
+)
+
+__all__ = [
+    "COMFY_SOCKET_TYPE",
+    "IMAGE_SLOT",
+    "PACKED_TOKEN_CEILING",
+    "ComfyClipBackend",
+    "PromptTags",
+    "TagSettings",
+    "TagTrace",
+    "chat",
+    "clip_generate",
+    "comfy_classifier",
+    "english_text",
+    "fit_picture",
+    "generate_text",
+    "prompt_tags",
+    "routed_progress_bars",
+    "run_outcome",
+    "send_status",
+    "shared_vision_encode",
+    "skip_resident_loads",
+    "stop_when",
+    "tag_picture",
+    "thing_scores",
+    "token_fill",
+]

logit_classifier-0.3.0/src/logit_classifier/toolkit/comfyui/classifier.py

@@ -0,0 +1,39 @@
+"""One Classifier over the CLIP a ComfyUI workflow already loaded."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from ...backends.base import UnsupportedModelError, VisionUnsupportedError
+from ...backends.comfy_clip import ComfyClipBackend
+from ...classifier import Classifier
+from ...config import Config
+
+
+def comfy_classifier(
+    clip: Any, *, node: str | None = None, needs_images: bool = True, batch_branches: bool = True
+) -> Classifier:
+    """One Classifier over the workflow's CLIP, with no running prior and no calibration file.
+
+    Each error starts with the node's name when one is given, and names the Load CLIP fix.
+    """
+    prefix = f"{node}: " if node else ""
+    backend: ComfyClipBackend | None = None
+
+    try:
+        backend = ComfyClipBackend(clip, batch_branches=batch_branches)
+    except UnsupportedModelError as error:
+        raise UnsupportedModelError(
+            f"{prefix}this CLIP is not a Qwen3-VL text encoder, so the node cannot read it. Load a Qwen3-VL text "
+            "encoder, such as qwen3vl_4b_fp8_scaled.safetensors, with a Load CLIP type that keeps its vision tower, "
+            "such as krea2."
+        ) from error
+    if needs_images and not backend.sees_images:
+        raise VisionUnsupportedError(
+            f"{prefix}this CLIP was loaded without its vision tower, so the node cannot read images. Load the "
+            "Qwen3-VL text encoder with a Load CLIP type that keeps it, such as krea2. The flux and flux2 types "
+            "drop it."
+        )
+    # A node reuses one classifier across a batch, and the running prior would make an item's answer depend on the
+    # items before it.
+    return Classifier(Config(use_prior_debias=False, calibration_path=None), backend)

logit_classifier-0.3.0/src/logit_classifier/toolkit/comfyui/generate.py

@@ -0,0 +1,146 @@
+"""Greedy replies from a loaded CLIP's own generate, for text a ComfyUI pack reads, such as a list of candidate tags.
+
+The classifier core never calls these tools, so nothing a classifier answers is generated.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Iterator
+from contextlib import AbstractContextManager, contextmanager, nullcontext
+from typing import Any
+
+from ...backends.base import VisionUnsupportedError
+from ...prompt import inert
+from ..tags import complete_tags, drop_unfinished_tag
+
+IMAGE_SLOT = "<|vision_start|><|image_pad|><|vision_end|>"
+
+_NO_IMAGE_TOKENIZER = (
+    "this CLIP's tokenizer does not read images, so it cannot generate from one. Load a Qwen3-VL text encoder with a "
+    "Load CLIP type that keeps its vision tower, such as krea2."
+)
+
+
+def chat(user: str, *, image: bool = False) -> str:
+    """Return the Qwen chat text for one user turn, with the image slot first when image is true."""
+    # The tokenizer reads a <|...|> spelling in plain text as a control token, so an unescaped prompt could
+    # close the user turn.
+    safe_user = inert(user)
+
+    # A text starting with <|im_start|> skips core's template, whose empty think block made this model reply
+    # empty or refuse.
+    return f"<|im_start|>user\n{IMAGE_SLOT if image else ''}{safe_user}<|im_end|>\n<|im_start|>assistant\n"
+
+
+def clip_generate(clip: Any, tokens: Any, **kwargs: Any) -> Any:
+    """Run clip.generate with every keyword passed through, then core's cleanup of the CUDA graphs it captured.
+
+    Core's Qwen3 graph decode keeps each layer's graph bound to the freed KV cache and asserts on the next generate in
+    the same node (core issue #16441). The cleanup keeps graph decode on, which measured 100 to 16.5 ms per token
+    (ContextAnchoredTileRefine tests-AB/tags-bench-log.md, section 3).
+    """
+    try:
+        return clip.generate(tokens, **kwargs)
+    finally:
+        _drop_decode_graphs()
+
+
+def _drop_decode_graphs() -> None:
+    try:
+        from comfy.model_prefetch import cleanup_prefetch_queues
+    except ImportError:
+        return
+    cleanup_prefetch_queues()
+
+
+def _transformer(clip: Any) -> Any:
+    """Core's generating transformer, or None for a CLIP without one."""
+    model = getattr(clip, "cond_stage_model", None)
+    encoder_name = getattr(model, "clip", None)
+    encoder = getattr(model, encoder_name, None) if isinstance(encoder_name, str) else None
+
+    return getattr(encoder, "transformer", None)
+
+
+@contextmanager
+def stop_when(clip: Any, should_stop: Callable[[list[str]], bool]) -> Iterator[None]:
+    """End core's greedy decode on a stop token once should_stop(the complete tags so far) is true.
+
+    Core's generate loop takes no stop condition, so the token sample_token returns is replaced. Core copies that
+    token into the sequence and breaks on it, in comfy/text_encoders/llama.py BaseGenerate.generate. A CLIP without
+    core's sample_token or stop tokens passes through.
+    """
+    transformer = _transformer(clip)
+    config = getattr(getattr(transformer, "model", None), "config", None)
+    stop_tokens = getattr(config, "stop_tokens", None)
+    history: list[int] = []
+
+    if not callable(getattr(transformer, "sample_token", None)) or not stop_tokens:
+        yield
+        return
+    stop_id = stop_tokens[0]
+    original = transformer.sample_token
+    shadowed = "sample_token" in vars(transformer)
+    previous = vars(transformer).get("sample_token")
+
+    def watch(*args: Any, **kwargs: Any) -> Any:
+        token = original(*args, **kwargs)
+        history.append(int(token.reshape(-1)[0]))
+        if should_stop(complete_tags(clip.decode(history))):
+            return token.new_full(token.shape, stop_id)
+        return token
+
+    transformer.sample_token = watch
+    try:
+        yield
+    finally:
+        if shadowed:
+            transformer.sample_token = previous
+        else:
+            del transformer.sample_token
+
+
+def generate_text(
+    clip: Any,
+    user: str,
+    *,
+    image: Any = None,
+    max_tokens: int,
+    should_stop: Callable[[list[str]], bool] | None = None,
+    whole_tags: bool = True,
+) -> str:
+    """One greedy reply to user. With whole_tags, a reply that fills max_tokens is cut back to its last whole tag.
+
+    A CLIP whose tokenizer does not read images raises VisionUnsupportedError for an image.
+    """
+    text = chat(user, image=image is not None)
+    stopper: AbstractContextManager[None] = nullcontext() if should_stop is None else stop_when(clip, should_stop)
+    tokens: Any = None
+    ids: Any = None
+    reply: str = ""
+
+    tokens = clip.tokenize(text) if image is None else _tokenize_image(clip, text, image)
+    with stopper:
+        ids = clip_generate(clip, tokens, do_sample=False, max_length=max_tokens)
+    reply = clip.decode(ids)
+    # Core stops early only on a stop token, so a decode that fills the budget ends mid tag.
+    if whole_tags and len(ids) >= max_tokens:
+        return drop_unfinished_tag(reply)
+    return reply
+
+
+def _tokenize_image(clip: Any, text: str, image: Any) -> Any:
+    """Tokenize text with one image, or raise VisionUnsupportedError when the tokenizer placed no image entry."""
+    tokens: Any = None
+    holds_image = False
+
+    try:
+        tokens = clip.tokenize(text, images=[image])
+    except TypeError as error:
+        raise VisionUnsupportedError(_NO_IMAGE_TOKENIZER) from error
+    # A tokenizer that takes images=... through **kwargs and ignores it writes only text entries, and core's image
+    # entry is a dict in the token's first slot.
+    holds_image = any(isinstance(token[0], dict) for rows in tokens.values() for row in rows for token in row)
+    if not holds_image:
+        raise VisionUnsupportedError(_NO_IMAGE_TOKENIZER)
+    return tokens