freesolo-flash-dev 1.2.43__tar.gz → 1.2.49__tar.gz

{freesolo_flash_dev-1.2.43 → freesolo_flash_dev-1.2.49}/CHANGELOG.md

@@ -29,6 +29,17 @@ This file starts at 1.1.35. Earlier releases are not reconstructed here; use
 
 ### Fixed
 
+- A GRPO or OPD run whose rollout actor exited hard no longer wedges on a paid GPU until the
+  wall-time limit. The actor's `os._exit` bypassed verl's own prompt handler, so the prompt
+  stayed `running` forever and verl's unbounded replay poll waited on it at ~0% utilization. The
+  child now writes durable failure evidence, marks the prompt failed, cleans up, and only then
+  exits, and replay sampling fails the step on an explicit failure tag or a positively dead
+  actor instead of returning a survivor-only batch that would silently train on fewer rollouts
+  than were collected. A child that reaches the fit loop and then stops producing distinct
+  output, while the parent has no teacher or environment work in flight, is torn down with a
+  named cause rather than running silently to the limit; repeated identical lines count as
+  silence, and authoritative OOM, host-RAM, and infrastructure evidence still classifies the
+  exit first.
 - `flash models deploy` now warns before it moves the shared `<run-id>` model id onto a
   different checkpoint. Every checkpoint of a run is served under that one id, so deploying
   `<run-id>/step-50` while `step-100` was live silently changed what `<run-id>` served for

{freesolo_flash_dev-1.2.43 → freesolo_flash_dev-1.2.49}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: freesolo-flash-dev
-Version: 1.2.43
+Version: 1.2.49
 Summary: Flash — managed LoRA post-training (SFT/GRPO/OPD) for Freesolo environments, driven by the `flash` CLI
 Project-URL: Homepage, https://github.com/freesolo-co/flash
 Project-URL: Repository, https://github.com/freesolo-co/flash

{freesolo_flash_dev-1.2.43 → freesolo_flash_dev-1.2.49}/flash/cli/scaffold/TRAINING.md

@@ -844,6 +844,15 @@ Pick SFT when you already have good answers and want the model to imitate them.
   `max_context_tokens` that plausibly fits prompt + completion, and only raise it when you see
   truncation (outputs cut off mid-thought, degraded loss). A bigger context just costs
   more.
+- **Multi-turn targets train on the assistant turns only.** When `sft_completion` returns a
+  full target transcript (assistant turns interleaved with the environment's tool/user
+  observations), Flash supervises the assistant turns and masks the observations out of the
+  loss — the model is trained to produce its own replies, not to predict the environment's.
+  For every ChatML target, including a single assistant turn, control strings `<|im_start|>` and
+  `<|im_end|>` are reserved in every message field the active template renders into the body,
+  including text content, reasoning, and tool-call serialization. They are indistinguishable from
+  structural turn delimiters after tokenization. The run logs role-aware masked rows and any
+  completion-only fallback rows separately.
 - **For Qwen3.5 thinking multi-turn SFT, put reasoning only in the final assistant
   turn.** Qwen3.5's chat template strips literal `<think>` blocks from prior assistant
   history and pre-opens `<think>\n` in the next generation prompt. If every assistant

{freesolo_flash_dev-1.2.43 → freesolo_flash_dev-1.2.49}/flash/client/specs.py

@@ -18,10 +18,9 @@ def spec_payload(
     # wire without touching the digest, which is computed server-side from the reparsed spec.
     if spec.authored_gpu_count is None:
         out["gpu"] = {key: value for key, value in out["gpu"].items() if key != "count"}
-    # [environment] pip is platform-managed and no longer travels in the payload: both provider
-    # submit paths already resolve it from worker_pip_for_env when the spec carries none, so
-    # sending it here only duplicated that constant, and the server's parser now rejects it as an
-    # unauthorable key.
+    # [environment] pip DOES travel: it is the author's own scorer dependencies, which only they can
+    # declare, and the submit paths append it to worker_pip_for_env rather than replacing it. What
+    # travels is the extra requirements, not the worker's own.
     if authored_train_keys is not None:
         out["train"] = {
             key: value for key, value in out["train"].items() if key in authored_train_keys

{freesolo_flash_dev-1.2.43 → freesolo_flash_dev-1.2.49}/flash/core/spec.py

@@ -1,16 +1,52 @@
-"""Structured job specification shared by CLI/API/runner and GPU workers."""
+"""Structured job specification shared by CLI/API/runner and GPU workers.
+
+One `JobSpec` value is read through four different contracts, and the serializers below are how
+they are told apart:
+
+- authored configuration -- what a user writes. validated by `flash.schema.spec_from_dict`, NOT by
+  `JobSpec.from_dict`; the two are not interchangeable.
+- public representation -- `to_dict()`, deliberately lossy: it strips every platform-managed field
+  so the result re-validates through the authored parser.
+- persisted recovery record -- `from_dict()`, tolerant of historical spellings. its decoding rules
+  live in `flash.core.spec_persistence`.
+- resolved worker payload -- `to_internal_dict()` / `to_json()`, complete, including every managed
+  and resolved field the GPU worker needs.
+
+`flash/runner/preparation.py::_preparation_digest` hashes the canonical JSON of `to_dict()` and
+`to_internal_dict()`, so the BYTES those two emit are a recovery contract, not an implementation
+detail. See their docstrings before changing either.
+"""
 
 from __future__ import annotations
 
 import json
-import logging
 import os
 from dataclasses import asdict, dataclass, field, fields
 from typing import Any, Literal
 from uuid import UUID
 
 from flash._internal.diagnostics import SECRET_ENV_KEYS_ENV
-from flash.core.catalog import DEFAULT_MODEL, normalize_algorithm, samples_on_policy
+from flash.core.catalog import DEFAULT_MODEL, normalize_algorithm
+
+# `DROPPED_TOP_LEVEL_KEYS` keeps its historical private alias rather than being renamed at its 5 call
+# sites: `flash.runner` re-exports it as a package attribute and `preparation.py` reads it back through
+# `_runner()`, which is the seam the tests patch. Renaming it is a separate change from moving it, and
+# doing both at once would put a live recovery path's patch points inside a refactor whose whole claim
+# is that nothing behavioral moved.
+from flash.core.spec_persistence import (
+    DROPPED_TOP_LEVEL_KEYS as _DROPPED_TOP_LEVEL_KEYS,
+)
+from flash.core.spec_persistence import (
+    REMOVED_PERSISTED_TRAIN_KEYS,
+    announce_dropped_keys,
+    migrated_optimizer_batch,
+    opt_float,
+    opt_int,
+    str_tuple,
+    validated_persisted_providers,
+    validated_section,
+    volume_gb,
+)
 from flash.teacher.retry_contract import OPD_RESUME_REVISION_ENV
 
 _FALSE_STRINGS = {"", "0", "false", "no", "off", "none"}
@@ -45,15 +81,6 @@ def _coerce_credit_assignment(value: Any) -> CreditAssignment:
     raise ValueError(f"credit_assignment must be one of {CREDIT_ASSIGNMENTS}; got {value!r}")
 
 
-def _str_tuple(value: Any) -> tuple[str, ...]:
-    """Normalize a string-or-list knob to a tuple of strings; a bare string is one element, not iterated."""
-    if value is None:
-        return ()
-    if isinstance(value, str):
-        return (value,) if value else ()
-    return tuple(s for s in (str(x) for x in value) if s)
-
-
 def coerce_bool(value: Any) -> bool:
     """Parse a bool from loosely-typed sources; treats "false"/"0"/"no"/"off" as False."""
     if isinstance(value, str):
@@ -90,18 +117,6 @@ def _coerce_wandb(value: Any) -> WandbSpec:
     return WandbSpec(project=_label(value.get("project")), run_name=_label(value.get("run_name")))
 
 
-def _volume_gb(value: Any, default: int = 100) -> int:
-    """Parse volume size in GB; non-positive / non-numeric / missing values return default."""
-    if isinstance(value, bool):
-        # bool is an int subclass; reject to avoid int(True)==1 becoming a 1 GB volume.
-        return default
-    try:
-        gb = int(value)
-    except (TypeError, ValueError):
-        return default
-    return gb if gb > 0 else default
-
-
 def _validated_gpu_type(value: Any, *, field_name: str) -> str:
     if not isinstance(value, str):
         raise TypeError(f"{field_name} must be a string")
@@ -191,55 +206,6 @@ def attributed_gpu_type(status: Any) -> str:
     return selected or persisted_gpu_head(field("spec"))
 
 
-def _opt_int(value: Any) -> int | None:
-    """Parse optional int; rejects bools (bool is int subclass — int(True)==1 is a footgun)."""
-    if value is None:
-        return None
-    if isinstance(value, bool):
-        raise TypeError(f"expected a number, got bool {value!r}")
-    return int(value)
-
-
-def _opt_float(value: Any) -> float | None:
-    """Parse optional float; rejects bools (mirrors _opt_int)."""
-    if value is None:
-        return None
-    if isinstance(value, bool):
-        raise TypeError(f"expected a number, got bool {value!r}")
-    return float(value)
-
-
-def _migrated_optimizer_batch(train: dict, algorithm: str) -> tuple[int | None, int | None]:
-    """``(batch_size, prompts_per_step)`` with the pre-1.1.43 rollout spelling migrated.
-
-    grpo/opd authored the optimizer batch as ``batch_size`` until it was split into
-    ``prompts_per_step``. The schema rejects the old name on SUBMISSION, but `from_dict` also
-    reparses PERSISTED specs, and the deployed release (1.1.40) has no ``prompts_per_step`` at all
-    -- so every rollout run in flight across this upgrade carries only the old key. The workers read
-    ``prompts_per_step`` alone, so without this a recovered run silently resumes on the recipe
-    default: 64 instead of an authored 32 on grpo, which OOMs a card rented for 32, and 8 on opd.
-
-    The old key is MOVED, not copied, and is dropped whenever the new one is present. Leaving it
-    populated would re-emit both names from ``to_dict()``, and a spec carrying both is rejected by
-    the schema -- breaking the resubmit that recovery and ``flash runs get`` perform, and leaving
-    `vram.py::_optimizer_batch_value` (which takes the larger of the two) free to size a card off
-    the stale value that ranking ignores. That applies to a payload written mid-upgrade carrying
-    BOTH spellings too: ``prompts_per_step`` wins, so the superseded key has to go with it.
-
-    A non-positive legacy value is discarded rather than migrated: ``minimum=1`` would have
-    rejected it at submission, and carrying it forward only fails later on a rented GPU. It is
-    discarded from BOTH names for the same round-trip reason -- retaining it under the old one
-    would re-emit a key the schema rejects for this algorithm.
-    """
-    batch_size = _opt_int(train.get("batch_size"))
-    prompts_per_step = _opt_int(train.get("prompts_per_step"))
-    if not samples_on_policy(algorithm):
-        return batch_size, prompts_per_step
-    if prompts_per_step is not None:
-        return None, prompts_per_step
-    return None, batch_size if batch_size is not None and batch_size >= 1 else None
-
-
 _MAX_GPU_COUNT = 8
 
 
@@ -370,12 +336,6 @@ class EnvironmentSpec:
     resolved_sha: str = ""
 
 
-# internal persisted-record compatibility only, never public config parsing. #968 removed
-# advantage_clip, but already-provisioned strict records still carry the unused key and otherwise fail
-# attach/recovery. do not generalize this allowlist: removed ``seeds`` must still raise (#536).
-REMOVED_PERSISTED_TRAIN_KEYS = frozenset({"advantage_clip"})
-
-
 @dataclass(frozen=True)
 class TrainSpec:
     epochs: int | None = field(default=None, metadata={"introduced_in": "0.2.0"})
@@ -505,55 +465,6 @@ MANAGED_GPU_KEYS = frozenset(
     {"disk_gb", "network_volume", "network_volume_gb", "max_retries", "max_wall_seconds"}
 )
 
-# Removed public top-level fields that a PERSISTED record can still carry. Stored run records are
-# never rewritten, and effective_preparation.worker_spec is written with to_internal_dict() (asdict),
-# which emitted every field including the defaulted ones -- so every record written before a field
-# was dropped still names it. This registry also drives historical public-digest replay for
-# model_revision, which remains an internal field. Without it a still-running job can lose its
-# recovery, deploy, and serving paths after the upgrade.
-#
-# Ignored on READ only. Most keys here no longer exist as JobSpec fields; model_revision remains an
-# internal runner field, but the public schema rejects it and to_dict omits it. Keeping it in this set
-# lets digest recovery identify the historical public key without weakening JobSpec.from_dict.
-_DROPPED_TOP_LEVEL_KEYS = frozenset(
-    {"model_policy", "model_revision", "worker_env", "workload_profile_kind"}
-)
-
-# Tolerating a dropped key keeps a pre-upgrade run's recovery path, but the values behind it stop
-# being applied -- so a run that authored them now trains on managed defaults instead of what was
-# submitted. That is announced rather than silent. Only user-authorable keys qualify: model_policy
-# was platform-managed (to_dict popped it), so its loss cannot change what a submitted config asked
-# for. The values are deliberately NOT forwarded -- doing so would reinstate the override table this
-# removal exists to close, and would deliver it without the validation the deleted parser performed.
-_ANNOUNCED_DROPPED_KEYS = frozenset({"worker_env"})
-
-
-def _announce_dropped_keys(data: dict[str, Any]) -> None:
-    """Log the removed user-authored keys a persisted record still carries and no longer applies.
-
-    Only announced when the payload names the run. A public spec is ``to_dict()`` output, which pops
-    the server-assigned run_id, and the same stored run is read through both shapes -- so warning
-    without an id would emit an unactionable "run <unknown>" line and duplicate the identified one
-    the worker-spec read already produced. Every provisioned run records an internal worker spec
-    (asdict, run_id included), and that is the read this fires on.
-    """
-    run_id = str(data.get("run_id") or "").strip()
-    if not run_id:
-        return
-    for key in sorted(_ANNOUNCED_DROPPED_KEYS):
-        value = data.get(key)
-        if not value:
-            continue
-        names = ", ".join(sorted(str(k) for k in value)) if isinstance(value, dict) else str(value)
-        logging.getLogger("flash.spec").warning(
-            "run %s was submitted with [%s] (%s); that table was removed and its values are NOT "
-            "applied -- this run uses managed defaults instead. resubmit without it if the run "
-            "depends on those values.",
-            run_id,
-            key,
-            names,
-        )
-
 
 @dataclass(frozen=True)
 class WandbSpec:
@@ -651,6 +562,15 @@ class JobSpec:
 
         omit platform-managed fields because the control plane/runner assigns them and the public
         parser rejects them. internal callers use ``to_internal_dict()``.
+
+        This is the PUBLIC contract and it is deliberately lossy: what it drops is what the authored
+        parser would reject, so the result re-validates through ``flash.schema.spec_from_dict``. Do
+        not hand its output to a worker or provider path -- those need ``to_internal_dict()``.
+
+        Its bytes are a recovery contract. ``_preparation_digest`` hashes this output as canonical
+        JSON, so key PRESENCE is load-bearing even though key order is not: adding or dropping one
+        key here invalidates the stored digest of every warm-start and workload-profile run in
+        flight, which fails integrity validation on recovery.
         """
         data = asdict(self)
         # server-assigned identity — never authored in a config.
@@ -696,7 +616,17 @@ class JobSpec:
         return data
 
     def to_internal_dict(self) -> dict[str, Any]:
-        """Return the complete control-plane and worker representation."""
+        """Return the complete control-plane and worker representation.
+
+        This is the RESOLVED WORKER contract: every managed and resolved field the GPU worker needs,
+        including the ones ``to_dict()`` strips. It is what gets persisted as
+        ``effective_preparation.worker_spec`` and what ``to_json()`` ships to a provider. The
+        authored parser rejects this shape, so it must never be offered as a public payload.
+
+        Its bytes are a recovery contract for the same reason as ``to_dict()`` -- and note the two
+        empty-value pops below are part of it: an omitted empty field and an explicit empty one hash
+        differently, so removing either pop would break existing digests.
+        """
         data = asdict(self)
         # missing and unset are the same internal state. omitting the empty value preserves that state
         # without serializing it into an explicit empty preference, which every parser rejects.
@@ -712,13 +642,22 @@ class JobSpec:
 
     @classmethod
     def from_dict(cls, data: dict[str, Any]) -> JobSpec:
+        """Decode a PERSISTED or internal job spec. This is not the authored-config parser.
+
+        Authored configuration goes through ``flash.schema.spec_from_dict``, which is strict because
+        an unknown key there is a typo the author can still fix. This one reads bytes that were
+        already written -- by an older Flash, for a run that may still be training -- so it tolerates
+        the historical spellings registered in ``flash.core.spec_persistence``. The two are NOT
+        interchangeable: decoding a user's config here would silently accept platform-managed fields
+        that the public parser exists to reject.
+        """
         if not isinstance(data, dict):
             raise TypeError("job spec must be an object")
         allowed_top_level = {item.name for item in fields(cls)}
         unknown_top_level = sorted(set(data) - allowed_top_level - _DROPPED_TOP_LEVEL_KEYS)
         if unknown_top_level:
             raise ValueError(f"job spec has unknown key(s): {', '.join(unknown_top_level)}")
-        _announce_dropped_keys(data)
+        announce_dropped_keys(data)
         env = data.get("environment") or {}
         # Reject stale payloads carrying a local `path`; worker only runs published env ids.
         if isinstance(env, dict) and env.get("path"):
@@ -726,51 +665,15 @@ class JobSpec:
                 "local environment paths are no longer supported; the worker only runs "
                 "published Freesolo environment ids"
             )
-        train = data.get("train", {})
-        if train is None:
-            train = {}
-        if not isinstance(train, dict):
-            raise TypeError("train must be an object")
-        train = {
-            key: value for key, value in train.items() if key not in REMOVED_PERSISTED_TRAIN_KEYS
-        }
-        unknown_train = sorted(set(train) - {item.name for item in fields(TrainSpec)})
-        if unknown_train:
-            raise ValueError(f"train has unknown key(s): {', '.join(unknown_train)}")
-        gpu = data.get("gpu")
-        if gpu is None:
-            gpu = {}
-        if not isinstance(gpu, dict):
-            raise TypeError("gpu must be an object")
-        unknown_gpu = sorted(set(gpu) - {item.name for item in fields(GpuSpec)})
-        if unknown_gpu:
-            raise ValueError(f"gpu has unknown key(s): {', '.join(unknown_gpu)}")
-        gpu_type, gpu_type_fallbacks = _parse_persisted_gpu_types(gpu)
-        provider = gpu.get("provider", "")
-        if not isinstance(provider, str):
-            raise TypeError("gpu.provider must be a string")
-        provider = provider.strip().lower()
-        providers_raw = gpu.get("providers", ())
-        from flash.providers import PROVIDER_NAMES, validated_provider_preferences
-
-        providers = validated_provider_preferences(
-            providers_raw, allow_empty="providers" not in gpu
+        train = validated_section(
+            data,
+            "train",
+            {item.name for item in fields(TrainSpec)},
+            removed=REMOVED_PERSISTED_TRAIN_KEYS,
         )
-        if provider and providers:
-            raise ValueError("gpu.provider and gpu.providers cannot both be set")
-        if provider or providers or gpu_type:
-            from flash.providers.base import providers_for
-
-            if provider and provider not in PROVIDER_NAMES:
-                raise ValueError(f"unknown gpu.provider {provider!r}")
-            # a hard provider pin must carry every acceptable class.
-            for candidate in (gpu_type, *gpu_type_fallbacks):
-                if candidate and provider and provider not in providers_for(candidate):
-                    raise ValueError(
-                        f"gpu.provider {provider!r} cannot provision gpu.type {candidate!r}"
-                    )
-            # provider preferences are soft. an ineligible named provider contributes no candidate,
-            # while eligible named or unnamed configured providers remain available for failover.
+        gpu = validated_section(data, "gpu", {item.name for item in fields(GpuSpec)})
+        gpu_type, gpu_type_fallbacks = _parse_persisted_gpu_types(gpu)
+        provider, providers = validated_persisted_providers(gpu, gpu_type, gpu_type_fallbacks)
         project_raw = data.get("project", "")
         if not isinstance(project_raw, str):
             raise TypeError("project must be a string")
@@ -778,7 +681,7 @@ class JobSpec:
         algorithm = normalize_algorithm(data.get("algorithm", cls.algorithm))
         # one reading of the optimizer batch for both keys: the rollout spelling changed in 1.1.43
         # and a persisted spec can still carry the old one.
-        batch_size, prompts_per_step = _migrated_optimizer_batch(train, algorithm)
+        batch_size, prompts_per_step = migrated_optimizer_batch(train, algorithm)
         return cls(
             model=data.get("model", cls.model),
             model_revision=_model_revision(data.get("model_revision", cls.model_revision)),
@@ -787,11 +690,11 @@ class JobSpec:
                 id=env.get("id", ""),
                 params=dict(env.get("params") or {}),
                 pip=tuple(str(p) for p in env.get("pip") or ()),
-                secrets=_str_tuple(env.get("secrets")),
+                secrets=str_tuple(env.get("secrets")),
                 resolved_sha=str(env.get("resolved_sha") or ""),
             ),
             train=TrainSpec(
-                epochs=_opt_int(train.get("epochs")),
+                epochs=opt_int(train.get("epochs")),
                 lora_rank=int(train.get("lora_rank", 32)),
                 # round-trip a stored alpha (authored value, internal carrier, or a warm-start's
                 # inherited parent alpha); fall back to 2 x rank when absent.
@@ -803,22 +706,22 @@ class JobSpec:
                 init_from_adapter=str(train.get("init_from_adapter") or ""),
                 init_from_adapter_revision=str(train.get("init_from_adapter_revision") or ""),
                 hf_repo=str(train.get("hf_repo") or ""),
-                learning_rate=_opt_float(train.get("learning_rate")),
+                learning_rate=opt_float(train.get("learning_rate")),
                 batch_size=batch_size,
                 prompts_per_step=prompts_per_step,
-                max_context_tokens=_opt_int(train.get("max_context_tokens")),
-                save_every=_opt_int(train.get("save_every")),
+                max_context_tokens=opt_int(train.get("max_context_tokens")),
+                save_every=opt_int(train.get("save_every")),
                 max_steps=parse_max_steps(train.get("max_steps")),
                 save_at_steps=train.get("save_at_steps"),
-                max_examples=_opt_int(train.get("max_examples")),
-                group_size=_opt_int(train.get("group_size")),
-                temperature=_opt_float(train.get("temperature")),
-                max_completion_tokens=_opt_int(train.get("max_completion_tokens")),
-                kl_penalty_coef=_opt_float(train.get("kl_penalty_coef")),
-                entropy_quantile=_opt_float(train.get("entropy_quantile")),
-                thinking_length_penalty_coef=_opt_float(train.get("thinking_length_penalty_coef")),
+                max_examples=opt_int(train.get("max_examples")),
+                group_size=opt_int(train.get("group_size")),
+                temperature=opt_float(train.get("temperature")),
+                max_completion_tokens=opt_int(train.get("max_completion_tokens")),
+                kl_penalty_coef=opt_float(train.get("kl_penalty_coef")),
+                entropy_quantile=opt_float(train.get("entropy_quantile")),
+                thinking_length_penalty_coef=opt_float(train.get("thinking_length_penalty_coef")),
                 teacher_model=str(train.get("teacher_model") or ""),
-                stop_sequences=_str_tuple(train.get("stop_sequences")),
+                stop_sequences=str_tuple(train.get("stop_sequences")),
                 structured_outputs=str(train.get("structured_outputs") or ""),
                 credit_assignment=_coerce_credit_assignment(train.get("credit_assignment")),
             ),
@@ -833,7 +736,7 @@ class JobSpec:
                 # survives the to_dict()->from_dict() hops in _with_model_disk / _spec_with_gpu /
                 # _assign_managed_hf_repo before deploy.
                 network_volume=gpu.get("network_volume"),
-                network_volume_gb=_volume_gb(gpu.get("network_volume_gb")),
+                network_volume_gb=volume_gb(gpu.get("network_volume_gb")),
                 count=gpu.get("count", 1),
                 type_fallbacks=gpu_type_fallbacks,
             ),