freesolo-flash-dev 1.4.7__tar.gz → 1.4.8__tar.gz

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/.github/workflows/publish-dev.yml

@@ -19,6 +19,7 @@ on:
       - "uv.lock"
       - "flash/**"
       - "scripts/build_dev_dist.py"
+      - "scripts/check_dev_release.py"
       - ".github/workflows/publish-dev.yml"
   workflow_dispatch:
 
@@ -82,24 +83,7 @@ jobs:
         id: decide
         env:
           VERSION: ${{ steps.meta.outputs.version }}
-        run: |
-          python3 - <<'PY' >> "$GITHUB_OUTPUT"
-          import os
-          import urllib.error
-          import urllib.request
-
-          version = os.environ["VERSION"]
-          url = f"https://pypi.org/pypi/freesolo-flash-dev/{version}/json"
-          published = False
-          try:
-              with urllib.request.urlopen(url, timeout=30) as response:
-                  published = response.status == 200
-          except urllib.error.HTTPError as error:
-              if error.code != 404:
-                  raise
-          # Publish only when the dev version isn't on PyPI yet (i.e. it was just bumped).
-          print(f"publish={'false' if published else 'true'}")
-          PY
+        run: python3 scripts/check_dev_release.py >> "$GITHUB_OUTPUT"
 
       - name: Already published (nothing to do)
         if: steps.decide.outputs.publish == 'false'

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: freesolo-flash-dev
-Version: 1.4.7
+Version: 1.4.8
 Summary: Flash — managed LoRA post-training (SFT/GRPO) for Freesolo environments, driven by the `flash` CLI
 Project-URL: Homepage, https://github.com/freesolo-co/flash
 Project-URL: Repository, https://github.com/freesolo-co/flash
@@ -144,11 +144,19 @@ path is self-hostable end to end — see **[SELF_HOSTING.md](SELF_HOSTING.md)**.
 
 ## Using the hosted service
 
-The active training catalog is `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
-`Qwen/Qwen3.6-35B-A3B`. The hosted serving catalog routes all three models on B200.
-The retired dense Qwen3.6 27B identity is never translated into Qwen3.8. Customer-owned
-Modal supports all three active models; 27B and 35B-A3B are bound to the certified serving image
-digest. Customer-owned `flash serve` requires explicit `--provider modal`. Training and serving both use Modal.
+The training catalog includes Qwen3.5 0.8B, 4B, 9B, and **9B Base**, Qwen3.8 27B,
+and Qwen3.6 35B-A3B. `Qwen/Qwen3.5-9B` is Qwen's post-trained chat model and remains
+the default. Select `model = "Qwen/Qwen3.5-9B-Base"` to fine-tune the pretrained-only
+checkpoint with SFT or GRPO. It has its own weights, tokenizer revision, adapters,
+and Modal cache entry; it is never mapped to the post-trained model's FP8 weights.
+Its native chat template supports text, images, and tool calls, but the pretrained
+weights have not been tuned for instruction following.
+
+Hosted serving uses the [per-model GPU placements](flash/serving/app/README.md#hosted-gpu-configuration).
+The 9B Base checkpoint serves in BF16 on H100 with a 262,144-token context.
+Customer-owned `flash serve` requires explicit `--provider modal` and a live-qualified
+profile. The retired dense Qwen3.6 27B identity is never translated into Qwen3.8.
+Training and serving both use Modal.
 
 `flash login` is not interactive. Export `FREESOLO_API_KEY` first so the key does not appear in process listings:
 
@@ -348,6 +356,12 @@ and unions on top-level function arguments are unsupported. Top-level arguments
 native parameter text, where a string such as `"12"` and a number would be ambiguous;
 nested JSON preserves those types during generation, streaming, and history replay.
 
+Qualified Qwen tool-calling routes support thinking-enabled generation. Tool calls are
+parsed only after the first `</think>`; tool-like text inside reasoning stays in
+`reasoning_content`. A response that ends before that boundary cannot execute tools.
+Streaming preserves the same separation, and saved adapters retain their configured
+thinking mode. Tools still cannot be combined with logprobs or structured outputs.
+
 ### Workload estimates (SFT)
 
 `flash train`, `--dry-run`, and `--cost` read the pinned environment's packaged dataset

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/README.md

@@ -44,11 +44,19 @@ path is self-hostable end to end — see **[SELF_HOSTING.md](SELF_HOSTING.md)**.
 
 ## Using the hosted service
 
-The active training catalog is `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
-`Qwen/Qwen3.6-35B-A3B`. The hosted serving catalog routes all three models on B200.
-The retired dense Qwen3.6 27B identity is never translated into Qwen3.8. Customer-owned
-Modal supports all three active models; 27B and 35B-A3B are bound to the certified serving image
-digest. Customer-owned `flash serve` requires explicit `--provider modal`. Training and serving both use Modal.
+The training catalog includes Qwen3.5 0.8B, 4B, 9B, and **9B Base**, Qwen3.8 27B,
+and Qwen3.6 35B-A3B. `Qwen/Qwen3.5-9B` is Qwen's post-trained chat model and remains
+the default. Select `model = "Qwen/Qwen3.5-9B-Base"` to fine-tune the pretrained-only
+checkpoint with SFT or GRPO. It has its own weights, tokenizer revision, adapters,
+and Modal cache entry; it is never mapped to the post-trained model's FP8 weights.
+Its native chat template supports text, images, and tool calls, but the pretrained
+weights have not been tuned for instruction following.
+
+Hosted serving uses the [per-model GPU placements](flash/serving/app/README.md#hosted-gpu-configuration).
+The 9B Base checkpoint serves in BF16 on H100 with a 262,144-token context.
+Customer-owned `flash serve` requires explicit `--provider modal` and a live-qualified
+profile. The retired dense Qwen3.6 27B identity is never translated into Qwen3.8.
+Training and serving both use Modal.
 
 `flash login` is not interactive. Export `FREESOLO_API_KEY` first so the key does not appear in process listings:
 
@@ -248,6 +256,12 @@ and unions on top-level function arguments are unsupported. Top-level arguments
 native parameter text, where a string such as `"12"` and a number would be ambiguous;
 nested JSON preserves those types during generation, streaming, and history replay.
 
+Qualified Qwen tool-calling routes support thinking-enabled generation. Tool calls are
+parsed only after the first `</think>`; tool-like text inside reasoning stays in
+`reasoning_content`. A response that ends before that boundary cannot execute tools.
+Streaming preserves the same separation, and saved adapters retain their configured
+thinking mode. Tools still cannot be combined with logprobs or structured outputs.
+
 ### Workload estimates (SFT)
 
 `flash train`, `--dry-run`, and `--cost` read the pinned environment's packaged dataset

freesolo_flash_dev-1.4.8/flash/content/processor.py

@@ -0,0 +1,17 @@
+"""Normalize chat-template metadata from a checkpoint's own processor and tokenizer."""
+
+from typing import Any
+
+
+def inherit_tokenizer_chat_template(processor: Any) -> None:
+    """Support checkpoints that publish their template only in tokenizer_config.json.
+
+    Transformers loads that template onto the tokenizer but not AutoProcessor.
+    Preserve explicit processor templates and never substitute another model's tokenizer.
+    Missing templates remain missing so the normal request validation can reject them.
+    """
+    if getattr(processor, "chat_template", None) is None:
+        tokenizer = getattr(processor, "tokenizer", None)
+        template = getattr(tokenizer, "chat_template", None)
+        if template is not None:
+            processor.chat_template = template

freesolo_flash_dev-1.4.8/flash/content/stop_tokens.py

@@ -0,0 +1,13 @@
+"""Chat-turn boundaries for pretrained checkpoints with document-level EOS metadata."""
+
+from typing import Any
+
+
+def chat_stop_token_ids(model: str, tokenizer: Any, *, is_chat: bool) -> list[int] | None:
+    """Stop Base chat at im_end without changing its tokenizer or raw completion EOS."""
+    if model != "Qwen/Qwen3.5-9B-Base" or not is_chat:
+        return None
+    token_id = tokenizer.convert_tokens_to_ids("<|im_end|>")
+    if type(token_id) is not int or token_id < 0 or token_id == tokenizer.unk_token_id:
+        raise ValueError("Qwen3.5 9B Base tokenizer is missing its chat end token")
+    return [token_id]

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/core/catalog.py

@@ -16,6 +16,7 @@ _ON_POLICY_ALGORITHMS = frozenset({"grpo"})
 _IMAGE_TRAINING_MODELS = frozenset(
     {
         "Qwen/Qwen3.5-9B",
+        "Qwen/Qwen3.5-9B-Base",
         "Qwen/Qwen3.8-27B",
         "Qwen/Qwen3.6-35B-A3B",
     }
@@ -218,6 +219,7 @@ SERVING_MODEL_REPOS: dict[str, str] = {
     "Qwen/Qwen3.5-0.8B": "Qwen/Qwen3.5-0.8B",
     "Qwen/Qwen3.5-4B": "Qwen/Qwen3.5-4B",
     "Qwen/Qwen3.5-9B": "Freesolo-Co/Qwen3.5-9B-FP8",
+    "Qwen/Qwen3.5-9B-Base": "Qwen/Qwen3.5-9B-Base",
     "Qwen/Qwen3.8-27B": "Qwen/Qwen3.8-27B-FP8",
     "Qwen/Qwen3.6-35B-A3B": "Qwen/Qwen3.6-35B-A3B",
 }
@@ -532,6 +534,32 @@ MODELS: dict[str, ModelInfo] = {
 }
 
 
+# The pretrained checkpoint has the same architecture and native chat template as 9B,
+# but distinct weights, EOS metadata, and adapter identity. Never serve it via the
+# post-trained model's FP8 repository.
+MODELS["Qwen/Qwen3.5-9B-Base"] = replace(
+    MODELS["Qwen/Qwen3.5-9B"],
+    id="Qwen/Qwen3.5-9B-Base",
+    display_name="Qwen3.5 9B Base (pretrained)",
+    managed_revision="68c46c4b3498877f3ef123c856ecfde50c39f404",
+    serving=ServingCapacity(
+        gpu="H100",
+        serve_model_id="Qwen/Qwen3.5-9B-Base",
+        max_loras=16,
+        max_cpu_loras=16,
+        max_lora_rank=128,
+        max_model_len=262144,
+        max_num_seqs=8,
+        tensor_parallel_size=1,
+        gpu_memory_utilization=0.90,
+        image_limit=MAX_IMAGES_PER_CONVERSATION,
+    ),
+    notes="Pretrained-only Qwen3.5 9B; bf16 LoRA SFT/GRPO. Uses its own weights and "
+    "tokenizer, not the post-trained 9B. Native chat/thinking formatting is available for "
+    "fine-tuning; instruction following and reasoning are not post-trained. BF16 serving on H100.",
+)
+
+
 def list_models() -> list[ModelInfo]:
     return sorted(MODELS.values(), key=lambda m: (m.min_vram_gb, m.id))
 

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/engine/plan/vram.py

@@ -138,6 +138,7 @@ _SFT_CHUNKED_NLL_MODELS = frozenset(
         "Qwen/Qwen3.5-0.8B",
         "Qwen/Qwen3.5-4B",
         "Qwen/Qwen3.5-9B",
+        "Qwen/Qwen3.5-9B-Base",
         "Qwen/Qwen3.8-27B",
         "Qwen/Qwen3.6-35B-A3B",
     }

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/engine/profiling/sft_workload.py

@@ -12,6 +12,7 @@ from pathlib import Path
 from typing import Any, Protocol, TypedDict, cast
 
 from flash.content.multimodal import NormalizedImages
+from flash.content.processor import inherit_tokenizer_chat_template
 from flash.content.thinking import messages_for_chat_template
 from flash.engine.plan.recipe import RECIPE
 from flash.engine.plan.steps import resolve_update_horizon, sft_update_steps
@@ -190,11 +191,11 @@ def _default_processor_loader(model_id: str, revision: str):
 
     from flash.engine.support.huggingface import model_revision_kwargs
 
-    return AutoProcessor.from_pretrained(
-        model_id,
-        trust_remote_code=True,
-        **model_revision_kwargs(revision),
+    processor = AutoProcessor.from_pretrained(
+        model_id, trust_remote_code=True, **model_revision_kwargs(revision)
     )
+    inherit_tokenizer_chat_template(processor)
+    return processor
 
 
 def _packing_mode(

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/engine/worker/train/rl/launch/inputs.py

@@ -22,6 +22,8 @@ import flash.engine.worker.model.decoding as _worker_decoding
 import flash.engine.worker.perf as _worker_perf
 import flash.engine.worker.train.rl.launch.config as _worker_config
 from flash.adapters.targets import LoraTargeting, resolve_lora_targeting
+from flash.content.processor import inherit_tokenizer_chat_template
+from flash.content.stop_tokens import chat_stop_token_ids
 from flash.content.structured_outputs import (
     describe_structured_outputs,
     parse_structured_outputs,
@@ -420,10 +422,11 @@ def _assemble_grpo_inputs(
         "max_turns": int(getattr(env, "max_turns", 0) or 0) if multi_turn else 0,
         # the child decides whether an assistant turn ended naturally or was cut off, and it has no
         # access to the model config -- so the full halting set is resolved here and handed over.
-        # the union of the tokenizer's eos and the model's generation_config eos, because a model can
-        # stop on a secondary id its tokenizer never exposes.
+        # Keep every checkpoint EOS plus any chat-turn boundary the model needs. Base checkpoints
+        # can use document EOS while the environment still requires one assistant turn at a time.
         "eos_token_ids": (
             generation_eos_from_cached_config(model_id, model_revision, tok)
+            | frozenset(chat_stop_token_ids(model_id, tok, is_chat=True) or ())
             if multi_turn
             else frozenset()
         ),
@@ -532,6 +535,7 @@ def _resolve_grpo_inputs(*, context: WorkerContext):
         processor = AutoProcessor.from_pretrained(
             model_id, trust_remote_code=True, **_worker_hf.model_revision_kwargs(model_revision)
         )
+        inherit_tokenizer_chat_template(processor)
         tok = processor.tokenizer
         image_pad_token_id = resolve_image_pad_token_id(processor, tok)
     else:

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/serve/deployment/profiles.py

@@ -219,6 +219,34 @@ _PROFILES: dict[str, ServingProfile] = {
         modal_gpu_request="L40S",
         modal_live_qualified=True,
     ),
+    "Qwen/Qwen3.5-9B-Base": ServingProfile(
+        model_id="Qwen/Qwen3.5-9B-Base",
+        modality="multimodal",
+        served_model="Qwen/Qwen3.5-9B-Base",
+        served_model_revision="68c46c4b3498877f3ef123c856ecfde50c39f404",
+        tokenizer_model="Qwen/Qwen3.5-9B-Base",
+        dtype="bfloat16",
+        quantization=None,
+        kv_cache_dtype="fp8",
+        max_model_len=262144,
+        max_num_seqs=8,
+        max_num_batched_tokens=None,
+        max_loras=16,
+        max_cpu_loras=16,
+        max_lora_rank=128,
+        gpu_memory_utilization=0.90,
+        cpu_offload_gb=0.0,
+        image_limit=MAX_IMAGES_PER_CONVERSATION,
+        mm_processor_cache_gb=0.0,
+        enable_tower_connector_lora=True,
+        reasoning_parser="qwen3",
+        engine_args={"enforce_eager": False},
+        tokenizer_kwargs={},
+        processor_kwargs={},
+        modal_gpu="H100",
+        modal_gpu_request="H100!",
+        modal_live_qualified=True,
+    ),
     "Qwen/Qwen3.8-27B": ServingProfile(
         model_id="Qwen/Qwen3.8-27B",
         modality="multimodal",

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/serve/request/tool_calls.py

@@ -57,7 +57,8 @@ class FunctionTool:
 
 
 def qualified_tool_parser(base_model: str) -> str | None:
-    return TOOL_PARSER_QWEN3_CODER if base_model == "Qwen/Qwen3.5-9B" else None
+    supported = {"Qwen/Qwen3.5-9B", "Qwen/Qwen3.5-9B-Base"}
+    return TOOL_PARSER_QWEN3_CODER if base_model in supported else None
 
 
 def normalize_tools(
@@ -139,8 +140,6 @@ def validate_tool_request_contract(
 
     if not tools_active(tools, tool_choice):
         return
-    if thinking:
-        raise error_type("tools are not supported for thinking-enabled generation")
     if tool_parser != TOOL_PARSER_QWEN3_CODER:
         raise error_type("this serving engine is not qualified for tool calling")
 
@@ -255,7 +254,11 @@ def validate_tool_history_replay(
         # would accept a turn the parser rejects as ambiguous.
         try:
             _validate_template_roundtrip(
-                probe, rendered_turn_prefix(message, message_index > last_query)
+                probe,
+                rendered_turn_prefix(message, message_index > last_query),
+                answer_prefix=rendered_turn_prefix(
+                    message, message_index > last_query, answer_only=True
+                ),
             )
         except ValueError as exc:
             raise error_type(f"message {message_index} {exc}") from exc
@@ -592,6 +595,8 @@ def _historical_replay_type(value: Any) -> str:
 def _validate_template_roundtrip(
     probe: Sequence[tuple[FunctionTool, dict[str, Any], dict[str, Any]]],
     prefix: str = "",
+    *,
+    answer_prefix: str | None = None,
 ) -> None:
     from flash.serve.runtime.tool_calls import parse_qwen3_coder_output
 
@@ -619,6 +624,8 @@ def _validate_template_roundtrip(
         size += len(blocks[-1])
         if size > _MAX_REPLAY_TEMPLATE_CHARS:
             raise ValueError(failure)
+    if prefix and answer_prefix is not None:
+        blocks[0] = answer_prefix
     text = "".join(blocks)
     replay_tools = _merged_replay_tools(probe)
     # replay must grant exactly what a generation parse of this same text would grant. restating

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/serve/request/tool_template.py

@@ -1,8 +1,7 @@
 """the text the qwen tool template renders around an assistant turn's call blocks.
 
-replay validation has to see the same prefix the model does. the parser selects the first complete
-`<tool_call>` marker in the turn, so a marker anywhere ahead of the call blocks steals the parse,
-and a marker the template discards must not be validated as though it survived.
+Replay validates the rendered answer prefix. Reasoning is counted toward the full
+rendered size, but only text after its first close can contain executable tools.
 """
 
 from __future__ import annotations
@@ -55,7 +54,9 @@ def last_query_index(messages: Sequence[Mapping[str, Any]]) -> int:
     return len(messages) - 1
 
 
-def rendered_turn_prefix(message: Mapping[str, Any], with_reasoning: bool) -> str:
+def rendered_turn_prefix(
+    message: Mapping[str, Any], with_reasoning: bool, *, answer_only: bool = False
+) -> str:
     """the text the template renders before this turn's call blocks."""
     # deriving the prefix from the same normalization the prompt path applies is what keeps this
     # exact. validating the raw request fields instead would reject a turn whose markers the
@@ -84,6 +85,8 @@ def rendered_turn_prefix(message: Mapping[str, Any], with_reasoning: bool) -> st
     # earlier turn's reasoning is dropped, so including it would reject a replayable turn.
     if with_reasoning and reasoning:
         prefix += f"{THINK_START}\n{reasoning}\n{THINK_END}\n\n"
+        if answer_only:
+            prefix = prefix.partition(THINK_END)[2]
     if content:
         prefix += f"{content}\n\n"
     return prefix

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/serve/runtime/engine.py

@@ -11,6 +11,8 @@ from contextlib import asynccontextmanager, suppress
 from dataclasses import dataclass
 from typing import Any
 
+from flash.content.processor import inherit_tokenizer_chat_template
+from flash.content.stop_tokens import chat_stop_token_ids
 from flash.content.structured_options import vllm_structured_kwargs
 from flash.serve.request.runtime_support import reasoning_compatibility_guard
 from flash.serve.request.tool_calls import tools_active
@@ -28,8 +30,9 @@ from .prompt import (
     effective_chat_template_kwargs,
     resolve_thinking,
 )
+from .reasoning_tools import ReasoningToolStreamParser, parse_tool_output
 from .sampling import complete_indexed_outputs, indexed_outputs, normalize_token_logprobs
-from .tool_calls import ParsedToolCall, ToolCallStreamParser, parse_qwen3_coder_output
+from .tool_calls import ParsedToolCall
 from .types import (
     AdapterSpec,
     EngineConfig,
@@ -59,7 +62,7 @@ class _StructuredState:
 class _ChoiceStreamState:
     token_ids: list[int]
     logprobs: list[dict[str, Any]]
-    tool_parser: ToolCallStreamParser | None = None
+    tool_parser: ReasoningToolStreamParser | None = None
     text: str = ""
     finish_reason: str | None = None
     tool_calls: tuple[Any, ...] = ()
@@ -76,7 +79,9 @@ class _StreamState:
     cached_reported: bool = False
 
     @classmethod
-    def create(cls, n: int, top_logprobs: int, tools: Any = None) -> _StreamState:
+    def create(
+        cls, n: int, top_logprobs: int, tools: Any = None, *, thinking: bool = False
+    ) -> _StreamState:
         return cls(
             n=n,
             top_logprobs=top_logprobs,
@@ -84,7 +89,9 @@ class _StreamState:
                 index: _ChoiceStreamState(
                     token_ids=[],
                     logprobs=[],
-                    tool_parser=ToolCallStreamParser(tools) if tools is not None else None,
+                    tool_parser=ReasoningToolStreamParser(tools, thinking=thinking)
+                    if tools is not None
+                    else None,
                 )
                 for index in range(n)
             },
@@ -157,7 +164,7 @@ class _StreamState:
 
 
 def _generation_choice(
-    index: int, output: Any, *, top_logprobs: int, tools: Any = None
+    index: int, output: Any, *, top_logprobs: int, tools: Any = None, thinking: bool = False
 ) -> GenerationChoice:
     token_ids = tuple(int(value) for value in (getattr(output, "token_ids", None) or []))
     finish_reason = getattr(output, "finish_reason", None)
@@ -166,7 +173,7 @@ def _generation_choice(
     text = str(getattr(output, "text", "") or "")
     tool_calls: tuple[ParsedToolCall, ...] = ()
     if tools is not None:
-        parsed = parse_qwen3_coder_output(text, tools)
+        parsed = parse_tool_output(text, tools, thinking=thinking)
         if parsed.tools_called:
             text = parsed.content or ""
             tool_calls = parsed.calls
@@ -364,6 +371,7 @@ class VllmLoraRuntime:
                 output,
                 top_logprobs=request.top_logprobs,
                 tools=request.tools if tools_active(request.tools, request.tool_choice) else None,
+                thinking=bool(thinking),
             )
             for index, output in sorted(complete_indexed_outputs(final_output, n=request.n).items())
         )
@@ -416,6 +424,7 @@ class VllmLoraRuntime:
                     request.n,
                     request.top_logprobs,
                     request.tools if tools_active(request.tools, request.tool_choice) else None,
+                    thinking=bool(thinking),
                 )
                 output = first_output
                 while True:
@@ -504,6 +513,7 @@ class VllmLoraRuntime:
                 **common,
                 **thaw_mapping(self.config.processor_kwargs, "processor_kwargs"),
             )
+            inherit_tokenizer_chat_template(processor)
             tokenizer = getattr(processor, "tokenizer", None)
             if tokenizer is None:
                 raise RuntimeNotReadyError("multimodal processor has no tokenizer")
@@ -580,8 +590,6 @@ class VllmLoraRuntime:
     def _validate_tools(self, request: GenerationRequest, thinking: bool | None) -> None:
         if not tools_active(request.tools, request.tool_choice):
             return
-        if thinking:
-            raise PromptError("tools are not supported for thinking-enabled generation")
         if self.config.tool_parser != "qwen3_coder":
             raise PromptError("this serving engine is not qualified for tool calling")
 
@@ -631,8 +639,8 @@ class VllmLoraRuntime:
             parser_kwargs=parser_kwargs,
         )
 
-    @staticmethod
     def _sampling_params(
+        self,
         request: GenerationRequest,
         structured: _StructuredState,
         *,
@@ -655,6 +663,9 @@ class VllmLoraRuntime:
             structured_outputs=structured.params,
             # pass none rather than an empty list so vllm keeps its own default.
             stop=list(request.stop) or None,
+            stop_token_ids=chat_stop_token_ids(
+                self.config.model, self._tokenizer, is_chat=request.messages is not None
+            ),
         )
 
     async def _prepare_prompt(

freesolo_flash_dev-1.4.8/flash/serve/runtime/reasoning_tools.py

@@ -0,0 +1,60 @@
+"""Parse executable tools only after the first reasoning close, preserving raw text."""
+
+from collections.abc import Callable, Sequence
+
+from flash.serve.request.tool_calls import FunctionTool
+from flash.serve.request.tool_template import THINK_END
+
+from .tool_calls import ToolCallStreamParser, ToolParseResult, parse_qwen3_coder_output
+
+
+def parse_tool_output(
+    text: str,
+    tools: Sequence[FunctionTool],
+    *,
+    thinking: bool = False,
+    id_factory: Callable[[], str] | None = None,
+) -> ToolParseResult:
+    prefix = ""
+    if thinking:
+        reasoning, close, answer = text.partition(THINK_END)
+        if not close:
+            return ToolParseResult(content=text, calls=())
+        prefix, text = reasoning + close, answer
+    result = parse_qwen3_coder_output(text, tools, id_factory=id_factory)
+    return ToolParseResult(content=(prefix + (result.content or "")) or None, calls=result.calls)
+
+
+class ReasoningToolStreamParser:
+    """Forward reasoning untouched; give only answer deltas to the strict tool parser."""
+
+    def __init__(
+        self,
+        tools: Sequence[FunctionTool],
+        *,
+        thinking: bool = False,
+        id_factory: Callable[[], str] | None = None,
+    ) -> None:
+        self._closed = not thinking
+        self._tail = ""
+        self._tools = ToolCallStreamParser(tools, id_factory=id_factory)
+
+    def feed(self, text: str) -> str:
+        if self._closed:
+            return self._tools.feed(text)
+        pending = self._tail + text
+        reasoning, close, answer = pending.partition(THINK_END)
+        if close:
+            self._tail = ""
+            self._closed = True
+            return reasoning + close + self._tools.feed(answer)
+        # Retain only enough text to recognize a close split across deltas.
+        retain = len(THINK_END) - 1
+        self._tail = pending[-retain:]
+        return pending[:-retain]
+
+    def finish(self) -> ToolParseResult:
+        if self._closed:
+            return self._tools.finish()
+        tail, self._tail = self._tail, ""
+        return ToolParseResult(content=tail or None, calls=())

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/serving/app/GPU_QUALIFICATION.md

@@ -1,5 +1,50 @@
 # Hosted GPU qualification
 
+## Pretrained Qwen 9B Base, 2026-11-01
+
+`Qwen/Qwen3.5-9B-Base` is a separate pretrained checkpoint, pinned to
+`68c46c4b3498877f3ef123c856ecfde50c39f404`. It uses its own BF16 weights and
+tokenizer, including its EOS metadata. It is never redirected to the post-trained
+9B FP8 repository. Hosted and customer-owned profiles select H100, 262,144 total
+context tokens, FP8 KV, 16 hot/CPU LoRA slots at maximum rank 128, eight sequences,
+and CUDA graphs.
+
+A two-step H100 SFT run (`flash-modal-sft-smoke-2a2996da6a0a`) passed adapter
+publication, artifact recovery, and checkpoint resume. The rank-8 adapter records
+the Base identity and pinned revision; all 496 tensors are finite and all 248 LoRA
+B matrices are nonzero. The Modal training weight cache was warmed at the same pin.
+This checks training execution and recovery, not learning quality or GRPO quality.
+
+The [hosted engine qualification](https://modal.com/apps/freesolo/main/ap-bOjg4Q2AbsWHvD6KWvFXEF)
+loaded 31.41 GiB of weights and LoRA buffers and passed:
+
+- Eight concurrent short base requests.
+- A 262,112-token prompt with 32 output tokens reserved (17.8 seconds).
+- All 16 synthetic zero-valued rank-128 attention adapters loaded, with eight
+  distinct near-limit prompts submitted concurrently (25.5–192.5 seconds).
+- Image generation and rejection of a 262,145-token input.
+
+The [customer-owned runtime qualification](https://modal.com/apps/freesolo/main/ap-wWJ3AS4Y2Iggvy470MWK4h)
+used the profile-derived `VllmLoraRuntime` on Modal H100 with the hosted image and
+real rank-8 SFT adapter. It passed base generation, eight concurrent adapted
+requests, an image request, and a parsed tool call with `finish_reason=tool_calls`.
+This qualifies the runtime shape; it is not an OCI digest certification or a full
+provider provisioning/lifecycle test.
+
+Live tests exposed two checkpoint metadata differences. AutoProcessor did not
+inherit the template embedded in the Base tokenizer config, so image rendering
+failed. Loaders now inherit that same checkpoint's template only if the processor
+has none. Base also uses document EOS rather than chat-turn EOS: without an
+explicit assistant-turn stop, it continued past a tool call into an invented tool
+response. Buffered and streaming chat now stop at the checkpoint's `im_end` token,
+while raw completions and tokenizer EOS metadata remain unchanged. The final live
+canary verifies a single parsed tool call without a synthetic tool response.
+
+These are execution and capacity checks with short outputs. They do not establish
+instruction following, tool-use accuracy, long-context quality, or sustained
+throughput. Queued near-limit requests do not imply eight full contexts fit in KV
+memory simultaneously. No shared production serving app was deployed.
+
 ## Native Qwen context, 2026-10-31
 
 All five hosted Qwen models now use their native **262,144-token total context**,

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/serving/app/README.md

@@ -274,10 +274,11 @@ selection and customer-owned serving profiles are separate.
 | Qwen3.5-0.8B | official bf16 | L4 | 4 × 64 | 262144 |
 | Qwen3.5-4B | official bf16 | L40S | 4 × 64 | 262144 |
 | Qwen3.5-9B | owned FP8 (`Freesolo-Co/*-FP8`) | L40S | 16 × 128 | 262144 |
+| Qwen3.5-9B-Base | official pretrained BF16 | H100 | 16 × 128 | 262144 |
 | Qwen3.8-27B | official native FP8 (`Qwen/Qwen3.8-27B-FP8`) | H100 | 16 × 64 | 262144 |
 | Qwen3.6-35B-A3B (MoE, VL) | official bf16 | H200 | 6 × 64 | 262144 |
 
-The context budget includes prompt and generated output. All five Qwen checkpoints use
+The context budget includes prompt and generated output. All six Qwen checkpoints use
 their native 262,144-token window without RoPE scaling. Eight admitted sequences share
 GPU KV memory; long requests can queue or be preempted as that memory fills.
 

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/serving/src/accounting/usage.py

@@ -36,6 +36,8 @@ _FREESOLO_USD_PER_MTOK: dict[str, tuple[str, str, str]] = {
     "Qwen/Qwen3.5-4B": ("0.036", "0.18", "0.0072"),
     "deepseek-ai/DeepSeek-V4.1-Flash-1017": ("0.15", "0.60", "0.003"),
     "Qwen/Qwen3.5-9B": ("0.095", "0.1425", "0.0276"),
+    # Pretrained 9B uses the same customer token rates as post-trained 9B.
+    "Qwen/Qwen3.5-9B-Base": ("0.095", "0.1425", "0.0276"),
     "Qwen/Qwen3.8-27B": ("0.3325", "2.4225", "0.03325"),
     "Qwen/Qwen3.6-35B-A3B": ("0.095", "0.9025", "0.0475"),
 }

{freesolo_flash_dev-1.4.7 → freesolo_flash_dev-1.4.8}/flash/serving/src/engine/boot.py

@@ -11,6 +11,7 @@ the card they were found on.
 
 from typing import Any
 
+from flash.content.processor import inherit_tokenizer_chat_template
 from flash.serving.src.engine.model_config import (
     engine_overrides_for,
     image_limit_for,
@@ -49,6 +50,7 @@ def load_tokenizer(
             token=settings.hf_api_key,
             trust_remote_code=cfg.TRUST_REMOTE_CODE,
         )
+        inherit_tokenizer_chat_template(processor)
         tokenizer = getattr(processor, "tokenizer", None)
         if tokenizer is None:
             raise RuntimeError("image-capable model processor has no tokenizer")