freesolo-flash-dev 1.4.4__tar.gz → 1.4.6__tar.gz

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/.github/workflows/deploy-modal.yml

@@ -44,7 +44,7 @@ jobs:
     # accounting settlement, image builds, and a rollback with its own readiness
     # poll. The workflow contract test sums these budgets so rollback is never
     # cut off by the job timeout.
-    timeout-minutes: 150
+    timeout-minutes: 180
     # serialize serving deploys across runs because each deploy mutates the live app. github can still
     # replace a pending run, so the gate compares against the sha reported by the deployed app rather
     # than assuming every push predecessor was deployed.

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/CHANGELOG.md

@@ -30,6 +30,18 @@ This file starts at 1.1.35. Earlier releases are not reconstructed here; use
 
 ### Added
 
+- Control planes sharing a state directory take a lease (`control-plane.lock` in the data
+  directory) so a deploy can start a new plane while the old one drains. Only the holder runs
+  run recovery, deployment recovery and the background loops; a plane started beside it serves
+  HTTP and leaves new runs queued for the holder until the old plane exits.
+
+- SFT rows accept `tools`: OpenAI-style function declarations, validated and normalized exactly
+  as a chat request's `tools`. They are rendered through the chat template into every training
+  view of the row (transcript, prompt boundary and loss-mask probe), so a tool-calling adapter
+  trains on the prompt it is served with. Previously every SFT row rendered without tools, so for
+  Qwen3.5 the `# Tools` block that serving prepends to the system turn never appeared in
+  training. The block is prompt only, so supervised tokens are unchanged. Rows without `tools`
+  render as before; `tools` on an image-bearing row is refused.
 - Custom training rows accept a rectangular `target_tokens` matrix, `[len(tokens)-1, K]`
   with K up to 64, scoring K tokens at every position instead of the next token. `weights`
   then has the same shape and logprobs come back with it, so `forward_backward` computes

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: freesolo-flash-dev
-Version: 1.4.4
+Version: 1.4.6
 Summary: Flash — managed LoRA post-training (SFT/GRPO) for Freesolo environments, driven by the `flash` CLI
 Project-URL: Homepage, https://github.com/freesolo-co/flash
 Project-URL: Repository, https://github.com/freesolo-co/flash
@@ -352,8 +352,14 @@ The estimate tokenizes raw `input`/`output` fields plus the statically readable
 contract. SFT rows can instead supply `prompt_messages`, a nonempty chat-message list
 that is authoritative for both profiling and training. This preserves stored multi-turn
 histories and screenshot blocks; image-bearing transcripts use unpacked execution.
-The row still requires `input` for dataset identity. Environment-added prompts, few-shot
-examples, tool schemas, filters, and transformations are **not** executed, so real training
+The row still requires `input` for dataset identity. A row for a tool-calling agent can also
+supply `tools`, the same OpenAI-style function declarations a chat request sends. Flash
+validates them as serving does and renders them into the row's prompt through the chat template,
+so the adapter trains on the prompt it is served with: Qwen3.5, for example, prepends a
+`# Tools` block to the system turn. The block is prompt, not target, so it lengthens rows
+without adding supervised tokens. Without `tools`, a row trained for a tool-calling deployment
+never sees that block. Environment-added prompts, few-shot
+examples, environment-built tool schemas, filters, and transformations are **not** executed, so real training
 can retain fewer rows, truncate more often, and cost more than the estimate. If no readable dataset exists, cost, dry-run, and
 submit all fail before GPU allocation rather than billing a profiling run.
 

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/README.md

@@ -252,8 +252,14 @@ The estimate tokenizes raw `input`/`output` fields plus the statically readable
 contract. SFT rows can instead supply `prompt_messages`, a nonempty chat-message list
 that is authoritative for both profiling and training. This preserves stored multi-turn
 histories and screenshot blocks; image-bearing transcripts use unpacked execution.
-The row still requires `input` for dataset identity. Environment-added prompts, few-shot
-examples, tool schemas, filters, and transformations are **not** executed, so real training
+The row still requires `input` for dataset identity. A row for a tool-calling agent can also
+supply `tools`, the same OpenAI-style function declarations a chat request sends. Flash
+validates them as serving does and renders them into the row's prompt through the chat template,
+so the adapter trains on the prompt it is served with: Qwen3.5, for example, prepends a
+`# Tools` block to the system turn. The block is prompt, not target, so it lengthens rows
+without adding supervised tokens. Without `tools`, a row trained for a tool-calling deployment
+never sees that block. Environment-added prompts, few-shot
+examples, environment-built tool schemas, filters, and transformations are **not** executed, so real training
 can retain fewer rows, truncate more often, and cost more than the estimate. If no readable dataset exists, cost, dry-run, and
 submit all fail before GPU allocation rather than billing a profiling run.
 

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/flash/core/catalog.py

@@ -270,7 +270,7 @@ MODELS: dict[str, ModelInfo] = {
             max_loras=4,
             max_cpu_loras=4,
             max_lora_rank=64,
-            max_model_len=32768,
+            max_model_len=262144,
             max_num_seqs=8,
             tensor_parallel_size=1,
             gpu_memory_utilization=0.90,
@@ -320,17 +320,17 @@ MODELS: dict[str, ModelInfo] = {
             (9216, 2560, 32),
         ),
         serving=ServingCapacity(
-            gpu="L4",
+            gpu="L40S",
             serve_model_id="Qwen/Qwen3.5-4B",
             max_loras=4,
             max_cpu_loras=4,
             max_lora_rank=64,
-            max_model_len=32768,
+            max_model_len=262144,
             max_num_seqs=8,
             tensor_parallel_size=1,
             gpu_memory_utilization=0.90,
         ),
-        notes="Text-only BF16 LoRA SFT/GRPO. Hosted serving uses L4.",
+        notes="Text-only BF16 LoRA SFT/GRPO. Hosted serving uses L40S.",
     ),
     "Qwen/Qwen3.5-9B": ModelInfo(
         id="Qwen/Qwen3.5-9B",
@@ -379,7 +379,7 @@ MODELS: dict[str, ModelInfo] = {
             max_loras=16,
             max_cpu_loras=16,
             max_lora_rank=128,
-            max_model_len=32768,
+            max_model_len=262144,
             max_num_seqs=8,
             tensor_parallel_size=1,
             gpu_memory_utilization=0.90,
@@ -434,12 +434,12 @@ MODELS: dict[str, ModelInfo] = {
         recommended_gpu="H100",
         min_disk_gb=160,
         serving=ServingCapacity(
-            gpu="A100-80GB",
+            gpu="H100",
             serve_model_id=SERVING_MODEL_REPOS["Qwen/Qwen3.8-27B"],
             max_loras=16,
             max_cpu_loras=16,
             max_lora_rank=64,
-            max_model_len=32768,
+            max_model_len=262144,
             max_num_seqs=8,
             tensor_parallel_size=1,
             gpu_memory_utilization=0.90,
@@ -448,7 +448,7 @@ MODELS: dict[str, ModelInfo] = {
         thinking="hybrid",
         notes="Dense 27.781427952B multimodal VL checkpoint with image-capable bf16 LoRA training. "
         "SFT fits the 80GB A100 (~55.6GB weights); colocated GRPO needs the B200 (trainer + vLLM "
-        "rollout = two copies). Hosted serving runs the official FP8 checkpoint on A100 80GB.",
+        "rollout = two copies). Hosted serving runs the official FP8 checkpoint on H100.",
     ),
     "Qwen/Qwen3.6-35B-A3B": ModelInfo(
         id="Qwen/Qwen3.6-35B-A3B",
@@ -511,12 +511,12 @@ MODELS: dict[str, ModelInfo] = {
         serving=ServingCapacity(
             gpu="H200",
             serve_model_id=SERVING_MODEL_REPOS["Qwen/Qwen3.6-35B-A3B"],
-            # bf16 is the validated full-expert lora path. six hot rank-64 adapters plus 32k fit
+            # bf16 is the validated full-expert lora path. six hot rank-64 adapters and native context fit
             # with CUDA graphs on the qualified H200 serving tier.
             max_loras=6,
             max_cpu_loras=6,
             max_lora_rank=64,
-            max_model_len=32768,
+            max_model_len=262144,
             max_num_seqs=8,
             max_num_batched_tokens=4096,
             tensor_parallel_size=1,

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/flash/engine/profiling/dataset_profile.py

@@ -214,6 +214,14 @@ def _validate_dataset_rows(
     selected = rows[:max_examples] if max_examples > 0 else rows
     if not selected and empty_message is not None:
         raise PackagedDatasetUnavailable(empty_message)
+    if selected and all(row.get(_CANONICAL_OUTPUT_KEY) in (None, "") for row in selected):
+        raise ValueError(
+            "all selected SFT records have missing or empty top-level 'output' fields. "
+            "Submission profiling reads explicit records without executing environment loaders. "
+            "Supply input/output records via environment.params.records or select a packaged "
+            "SFT dataset with environment.params.dataset_path; nested demonstrations and "
+            "targets constructed only by a custom loader are not read by this check."
+        )
     return source_examples, [dict(row) for row in selected]
 
 

freesolo_flash_dev-1.4.6/flash/engine/profiling/sft_tools.py

@@ -0,0 +1,49 @@
+"""tool declarations an SFT row trains against.
+
+A tool-calling agent is served with ``tools=``, and the chat template renders those declarations
+into the prompt (Qwen3.5 prepends a ``# Tools`` block to the system turn). A row rendered without
+them trains on a prompt the deployed adapter never sees. A row may therefore carry ``tools`` -- the
+same per-example field as OpenAI fine-tuning data -- which is validated and normalized exactly as a
+chat request's ``tools`` are (``text_tokenizer`` renders ``tools_wire(normalize_tools(...))``), then
+rendered into every training view of the row. A row without ``tools`` renders as before.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from flash.serve.request.tool_calls import normalize_tools, tools_wire
+
+
+def sft_row_tools(example) -> list[dict[str, Any]] | None:
+    """The row's tools in the form serving renders them, or None when it declares none."""
+    if not isinstance(example, dict) or example.get("tools") is None:
+        return None
+    try:
+        return tools_wire(normalize_tools(example["tools"]))
+    except ValueError as exc:
+        raise ValueError(f"sft tools: {exc}") from exc
+
+
+def sft_template_kwargs(thinking: bool, tools: list[dict[str, Any]] | None) -> dict[str, Any]:
+    """Chat-template kwargs for every training view of a row.
+
+    The transcript, the prompt boundary, the reasoning probe and the loss mask's field probe all
+    render with these, so the supervised tokens are located in exactly the text that is trained on.
+    Without tools this is the pre-existing thinking-only render.
+    """
+    kwargs: dict[str, Any] = {"enable_thinking": thinking, "preserve_thinking": False}
+    if tools is not None:
+        kwargs["tools"] = tools
+    return kwargs
+
+
+def reject_image_row_tools(example, *, image_bearing: bool) -> None:
+    """Refuse tools on an image row before any tokenizer or processor is resolved.
+
+    The image pipeline tokenizes through the processor's own render, so accepting them would silently
+    train the row without the declarations it was given. Validating here, while rows are collected,
+    also makes a malformed declaration fail with its own message rather than as a capability error.
+    """
+    if sft_row_tools(example) is not None and image_bearing:
+        raise ValueError("sft tools are not supported on image-bearing rows")

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/flash/engine/profiling/sft_workload.py

@@ -15,6 +15,7 @@ from flash.content.multimodal import NormalizedImages
 from flash.content.thinking import messages_for_chat_template
 from flash.engine.plan.recipe import RECIPE
 from flash.engine.plan.steps import resolve_update_horizon, sft_update_steps
+from flash.engine.profiling import sft_tools
 from flash.engine.profiling.sft_image_rows import (
     estimate_sft_image_row,
     process_sft_image_row,
@@ -417,22 +418,24 @@ def _text_sft_row_spec(
     max_length: int,
     row_index: int,
     multiturn: bool,
+    tools: list[dict[str, Any]] | None = None,
 ) -> tuple[str, _RowReasoning, dict[str, Any]]:
     normalized_prompt = messages_for_chat_template(prompt_messages)
     normalized_completion = messages_for_chat_template(completion_messages)
     normalized_source = [*normalized_prompt, *normalized_completion]
-    text = render_transcript(normalized_source)
+    template_kwargs = sft_tools.sft_template_kwargs(spec.thinking, tools)
+    render = partial(render_transcript, tools=tools)
+    text = render(normalized_source)
     prompt_text = tokenizer.apply_chat_template(
         normalized_prompt,
         tokenize=False,
         add_generation_prompt=True,
-        enable_thinking=spec.thinking,
-        preserve_thinking=False,
+        **template_kwargs,
     )
     reasoning = _row_reasoning(
         prompt_messages,
         completion_messages,
-        render=render_transcript,
+        render=render,
         tokenizer=tokenizer,
         max_length=max_length,
     )
@@ -444,7 +447,7 @@ def _text_sft_row_spec(
             "prompt_text": prompt_text,
             "target_messages": normalized_completion,
             "source_messages": normalized_source,
-            "template_kwargs": {"enable_thinking": spec.thinking, "preserve_thinking": False},
+            "template_kwargs": template_kwargs,
             # measurement carried alongside the row rather than in it: the parquet row is built from
             # an explicit key list below, so neither key reaches verl.
             "row_index": row_index,
@@ -481,13 +484,12 @@ def _tokenize_prompt_rows(
     # denominator. summed after filtering instead.
     reasoning_by_index: dict[int, _RowReasoning] = {}
 
-    def render_transcript(messages: list[dict]) -> str:
+    def render_transcript(messages: list[dict], tools: list[dict[str, Any]] | None = None) -> str:
         return tokenizer.apply_chat_template(
             messages_for_chat_template(messages),
             tokenize=False,
             add_generation_prompt=False,
-            enable_thinking=spec.thinking,
-            preserve_thinking=False,
+            **sft_tools.sft_template_kwargs(spec.thinking, tools),
         )
 
     for row_index, (
@@ -498,6 +500,7 @@ def _tokenize_prompt_rows(
     ) in enumerate(prompt_rows):
         has_images = record_has_images(example, prompt_messages)
         _reject_image_completion(completion_messages, image_bearing=has_images)
+        tools = sft_tools.sft_row_tools(example)
         # read before the image branch rewrites `completion_messages` to its text-only form.
         multiturn = len(completion_messages) > 1
         if multiturn:
@@ -561,6 +564,7 @@ def _tokenize_prompt_rows(
                 max_length=max_length,
                 row_index=row_index,
                 multiturn=multiturn,
+                tools=tools,
             )
             sampled_texts.append(text)
             reasoning_by_index[row_index] = reasoning
@@ -637,8 +641,9 @@ def _filter_retained_rows(tokenized: _TokenizedSftRows, tokenizer) -> _RetainedS
             dropped += 1
     if not rows:
         raise ValueError(
-            "every SFT example has an empty completion after sft_max_len truncation "
-            "(nothing to train on); increase sft_max_len or shorten the prompts"
+            "no SFT examples retain supervised answer tokens after formatting and truncation. "
+            "Check the assistant targets; if prompts exhaust train.max_context_tokens, "
+            "shorten them or increase train.max_context_tokens."
         )
     return _RetainedSftRows(
         rows=rows,
@@ -912,6 +917,9 @@ def prepare_sft_workload(
         else:
             prompt_messages = env.prompt_messages(example)
         completion_messages, coerced_scalar_output = _sft_completion_with_provenance(env, example)
+        sft_tools.reject_image_row_tools(
+            example, image_bearing=record_has_images(example, prompt_messages)
+        )
         prompt_rows.append((example, prompt_messages, completion_messages, coerced_scalar_output))
     package_root = getattr(env, "package_root", None)
     multimodal = any(

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/flash/runner/lifecycle/submit.py

@@ -254,8 +254,13 @@ def submit_job(
     owner_key_id: int | None = None,
     prepared_job: PreparedJob | None = None,
     supervisor: Supervisor | None = None,
+    queue_only: bool = False,
 ) -> state.RunStatus:
-    """Submit a prepared job, allocating resources only outside dry-run mode."""
+    """Submit a prepared job, allocating resources only outside dry-run mode.
+
+    ``queue_only`` persists the run as queued without launching it, for a control plane that does
+    not hold the plane lease; the holder's recovery scan launches it.
+    """
     if prepared_job is not None:
         prepared = prepared_job
     else:
@@ -271,6 +276,8 @@ def submit_job(
         "billing_context": billing_context,
         "platform_context": platform_context,
     }
+    if queue_only and not dry_run:
+        return _submit_prepared_job(prepared, dispatch=_leave_queued, **kwargs)
     if background and not dry_run:
         owner = supervisor or current_supervisor()
         with owner.reserve(("run", prepared.public_spec.run_id), "training") as dispatch:
@@ -278,6 +285,10 @@ def submit_job(
     return _submit_prepared_job(prepared, **kwargs)
 
 
+def _leave_queued(job) -> None:
+    """Dispatch nowhere: the persisted queued status is what the recovery scan launches."""
+
+
 def _submit_prepared_job(
     prepared: PreparedJob,
     *,

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/flash/serve/deployment/profiles.py

@@ -161,7 +161,7 @@ def _small_qwen_profile(model_id: str, revision: str, gpu: str) -> ServingProfil
         dtype="bfloat16",
         quantization=None,
         kv_cache_dtype="fp8",
-        max_model_len=32768,
+        max_model_len=262144,
         max_num_seqs=8,
         max_num_batched_tokens=None,
         max_loras=4,
@@ -200,7 +200,7 @@ _PROFILES: dict[str, ServingProfile] = {
         dtype="bfloat16",
         quantization=None,
         kv_cache_dtype="fp8",
-        max_model_len=32768,
+        max_model_len=262144,
         max_num_seqs=8,
         max_num_batched_tokens=None,
         max_loras=16,
@@ -228,7 +228,7 @@ _PROFILES: dict[str, ServingProfile] = {
         dtype="bfloat16",
         quantization=None,
         kv_cache_dtype="fp8",
-        max_model_len=32768,
+        max_model_len=262144,
         max_num_seqs=8,
         max_num_batched_tokens=None,
         max_loras=16,
@@ -258,7 +258,7 @@ _PROFILES: dict[str, ServingProfile] = {
         dtype="bfloat16",
         quantization=None,
         kv_cache_dtype="fp8",
-        max_model_len=32768,
+        max_model_len=262144,
         max_num_seqs=8,
         max_num_batched_tokens=4096,
         max_loras=6,

{freesolo_flash_dev-1.4.4 → freesolo_flash_dev-1.4.6}/flash/serve/deployment/readiness.py

@@ -5,7 +5,9 @@ from __future__ import annotations
 # The slowest measured B200 cold boot was 1821s. Compilation depends on model
 # architecture, not just parameter count: the dense 27B boots slower than the 35B MoE.
 ENGINE_STARTUP_TIMEOUT_SECONDS = 45 * 60
-ENGINE_REQUEST_TIMEOUT_SECONDS = 10 * 60
+# Native 256k batches took up to 627s on the selected GPUs. Allow queueing headroom
+# beyond the old 10-minute cap, within the managed-chat transport's 30-minute budget.
+ENGINE_REQUEST_TIMEOUT_SECONDS = 15 * 60
 REVISION_READY_MAX_BUDGET_SECONDS = ENGINE_STARTUP_TIMEOUT_SECONDS + ENGINE_REQUEST_TIMEOUT_SECONDS
 DEPLOYMENT_SMOKE_TIMEOUT_SECONDS = 10 * 60
 DEPLOYMENT_OVERHEAD_SECONDS = 5 * 60