freesolo-flash-dev 1.2.123__tar.gz → 1.2.127__tar.gz

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: freesolo-flash-dev
-Version: 1.2.123
+Version: 1.2.127
 Summary: Flash — managed LoRA post-training (SFT/GRPO/OPD) for Freesolo environments, driven by the `flash` CLI
 Project-URL: Homepage, https://github.com/freesolo-co/flash
 Project-URL: Repository, https://github.com/freesolo-co/flash

freesolo_flash_dev-1.2.127/docs/hosted-inference-load-testing.md

@@ -0,0 +1,224 @@
+# Hosted inference load testing
+
+The `flash.serving.loadtest` package is an operator-only, public-HTTP load harness for the hosted
+inference front door. It is not imported by production serving and it does not create, resize, inspect,
+or destroy provider resources. Operators run it explicitly with the serving extra:
+
+```bash
+uv run --extra serving python -m flash.serving.loadtest validate scenario.json
+uv run --extra serving python -m flash.serving.loadtest discover scenario.json
+uv run --extra serving python -m flash.serving.loadtest run scenario.json results/fake-five-phase
+uv run --extra serving python -m flash.serving.loadtest summarize results/fake-five-phase
+uv run --extra serving python -m flash.serving.loadtest verify results/fake-five-phase
+```
+
+`validate` is allocation-free and performs no HTTP request. `discover` calls only `GET /healthz`.
+`run` calls only `GET /healthz` and `POST /v1/chat/completions`. The credential value is read from the
+environment variable named by `credential_env`; never put a credential value in a scenario file.
+There are no request retries.
+
+## Evidence boundary
+
+The harness verifies exact deployment SHA and deployment id, required capabilities, required base
+models, and immutable adapter response provenance. It checks health before and after every phase and
+at the midpoint of each duration phase. Its evidence remains HTTP-only:
+
+- it cannot prove that a live replica scaled out;
+- it cannot prove that an engine was actually cold or at scale zero;
+- it cannot distinguish a dispatch deadline from the underlying resource constraint;
+- it cannot establish an availability SLA;
+- fake and test runs cannot support production claims.
+
+A `cold_burst` therefore records either `cold_scale_out_intent_http_unattested` or
+`true_scale_zero_intent_http_unattested`. That is an honest statement of the authored intent, not an
+attestation of provider state.
+
+## Strict scenario
+
+Scenarios are strict Pydantic v2 JSON. Unknown fields and fields that belong to another phase kind are
+rejected. The authored `phases` list may contain any subset of the five supported discriminated kinds
+in any order, except that a present `cold_burst` must be the first inference-producing phase. A complete
+five-phase scenario uses `cold_burst`, `warm`, `sustained`, `mixed`, then `overload`.
+
+Dynamic base-model discovery reads `base_models` from `/healthz`, then applies `include`, `exclude`, and
+`require`. `require` is a hard gate.
+
+Explicit adapter targets must name the immutable model id, base model, and checkpoint. The request
+carries `X-Freesolo-Expected-Checkpoint` and the response must match `X-Freesolo-Checkpoint`.
+
+`adapter_revision` and `hf_revision` are optional, because the hosted app does not emit them on every
+deployment. Omitting one verifies checkpoint identity alone rather than passing a header check the
+deployment never answered. Supplying one asserts that the deployment does emit it, so a response that
+omits or changes it is a provenance mismatch, never a skipped assertion. A supplied `hf_revision` must
+still be a canonical 40-character hub commit SHA.
+
+`/healthz` is read for `ok`, deployment identity, capabilities, and `base_models`. `accounting_ok` is
+optional: a deployment that does not report it is not treated as an accounting failure, while an
+explicit `false` stops the run. Note that `/healthz.gpus` is the configured supported-base-model engine
+count, not a live container count, so it can never be read as a replica count.
+
+This fake example covers every phase kind:
+
+```json
+{
+  "schema_version": 1,
+  "name": "fake five-phase hosted inference",
+  "endpoint": "https://example.invalid",
+  "expected_deployment": {
+    "sha": "94210a3",
+    "deployment_id": "fake-deployment-1"
+  },
+  "credential_env": "FAKE_SERVING_TOKEN",
+  "required_capabilities": ["permanent_checkpoint_identity"],
+  "discovery": {
+    "enabled": true,
+    "include": ["model-a", "model-b", "model-c"],
+    "exclude": ["model-b"],
+    "require": ["model-a"]
+  },
+  "targets": [],
+  "profiles": [
+    {
+      "name": "short",
+      "weight": 1.0,
+      "messages": [{ "role": "user", "content": "Return one short sentence." }],
+      "max_tokens": 16,
+      "temperature": 0.0,
+      "top_p": 0.95
+    }
+  ],
+  "client": {
+    "connect_timeout_seconds": 5.0,
+    "read_timeout_seconds": 30.0,
+    "write_timeout_seconds": 10.0,
+    "pool_timeout_seconds": 5.0,
+    "max_in_flight": 8,
+    "max_scheduling_lag_ms": 100.0
+  },
+  "seed": 17,
+  "fake": true,
+  "phases": [
+    {
+      "name": "cold",
+      "kind": "cold_burst",
+      "requests": 4,
+      "burst_window_seconds": 0.2,
+      "cold_intent": "cold_scale_out"
+    },
+    {
+      "name": "warm",
+      "kind": "warm",
+      "requests": 4,
+      "concurrency": 2
+    },
+    {
+      "name": "sustained",
+      "kind": "sustained",
+      "duration_seconds": 2.0,
+      "rate_rps": 2.0
+    },
+    {
+      "name": "mixed",
+      "kind": "mixed",
+      "duration_seconds": 2.0,
+      "rate_rps": 3.0
+    },
+    {
+      "name": "overload",
+      "kind": "overload",
+      "expects_capacity_contract": false,
+      "stages": [
+        { "duration_seconds": 1.0, "rate_rps": 4.0 },
+        { "duration_seconds": 1.0, "rate_rps": 8.0 }
+      ]
+    }
+  ]
+}
+```
+
+`warm` is the only closed-loop phase. Its authored request count is completed with bounded
+concurrency. Every other phase is open-loop. Its dispatch offsets are computed before the phase
+starts. If the client cannot admit an offset within `max_in_flight` and `max_scheduling_lag_ms`, the
+harness writes a terminal `client_admission_missed` record instead of converting that offered request
+into delayed closed-loop work. Overload stages use the same rule and never retry.
+
+## Stream and artifact contract
+
+Streaming requests ask for usage. Every SSE `data:` payload must be valid JSON, followed by exactly
+one `[DONE]`; data after `[DONE]`, a missing terminal finish reason, or a missing `[DONE]` is a protocol
+error. The first nonempty `content` or `reasoning_content` delta defines time to first generated token.
+The first nonempty visible `content` delta is measured separately. Missing usage makes token throughput
+unavailable for that request; the harness never estimates tokens from chunks.
+
+Only HTTP 503 with error code `serving_capacity_unavailable` and `Retry-After: 1` is counted as exact
+capacity rejection. Other 503 responses and 429 responses are separate classes.
+
+An error body can echo the request back, so only a machine error code is read out of it. A code that
+does not look like a slug is recorded as `unrecognized_code`, and a free-text `message` is never
+persisted.
+
+### Overload cannot be demonstrated without a capacity contract
+
+Current Flash `dev` implements no retryable capacity contract. Its engine error mapping returns 504
+for an engine timeout and 502 for other upstream failures; there is no capacity 503 and no
+`Retry-After: 1`. An overload phase run against such a deployment therefore cannot produce a clean
+rejection signal at all.
+
+An overload phase declares this with `expects_capacity_contract`, which defaults to `false`. The flag
+never changes how a response is classified. It only records whether an absence of capacity rejections
+is an accurate description of the deployment or a failed expectation:
+
+| observed                        | `expects_capacity_contract` | verdict                                               |
+| ------------------------------- | --------------------------- | ----------------------------------------------------- |
+| at least one exact capacity 503 | either                      | `capacity_rejection_observed`                         |
+| none                            | `false`                     | `overload_not_demonstrated`                           |
+| none                            | `true`                      | `overload_not_demonstrated_despite_declared_contract` |
+
+**An absence of capacity rejections is never evidence of headroom.** A deployment without a capacity
+contract sheds load in ways this harness cannot distinguish from ordinary failure over public HTTP.
+When any overload phase declares no capacity contract, the summary carries that limitation explicitly.
+No overload phase is ever summarized as a success on the strength of a quiet run.
+
+Each run claims a new result directory and writes:
+
+- `scenario.authored.json`, with prompt content redacted;
+- `scenario.resolved.json`, with discovered targets and cold-intent labels;
+- `events.jsonl`, through one event writer with one terminal row per scheduled request;
+- `summary.json`, atomically;
+- `complete.json`, last, with row counts, byte counts, and SHA-256 hashes.
+
+An interrupted run closes the event stream but never writes `complete.json`. It remains inspectable and
+`verify` rejects it as incomplete. Version 1 has no partial-phase resume.
+
+Artifacts never contain credentials, authorization or request headers, generated text, raw response
+bodies, or per-request prompts. Persisted response headers are limited to content type, retry-after,
+and immutable provenance.
+
+## Metrics
+
+`summary.json` reports overall and per-phase, per-target, and per-profile values:
+
+- scheduled, dispatched, completed, successful, and client-admission-missed counts;
+- offered, dispatch, and completion rates with explicit denominator seconds and basis;
+- scheduling, header, first-generated, first-visible, and total latency p50, p90, p95, p99, and max;
+- usage coverage and token totals, aggregate output token rate, and decode token rate;
+- peak client in-flight, HTTP status and error counts, retry count, exact capacity ratio, and 429 count;
+- authored versus achieved target and profile mix;
+- explicit sustained and mixed authored, scheduling, dispatch, and completion windows.
+
+## B200 owner handoff
+
+The harness does not allocate a B200 and does not attest provider state. The B200 deployment owner must
+complete the provider-side handoff before an operator runs a non-fake scenario:
+
+1. identify the already deployed endpoint, exact release SHA, deployment id, and intended base models;
+2. confirm the B200 resource is owned, budgeted, and governed outside this harness;
+3. decide whether the cold intent is `cold_scale_out` or `true_scale_zero`, and establish that state
+   through the provider control plane without using the harness;
+4. provide only the name of the existing credential environment variable in the scenario;
+5. run `validate`, then `discover`, and confirm the resolved two-of-three or other intended model set;
+6. run into a new result directory, then run `verify` before interpreting `summary.json`;
+7. retain provider-side replica and hardware evidence separately if a scale-out or B200 claim is needed.
+
+A successful HTTP load run is not provider-side proof of B200 identity, replica count, scale-to-zero,
+or autoscaler behavior.

freesolo_flash_dev-1.2.127/flash/_internal/http.py

@@ -0,0 +1,101 @@
+"""authenticated stdlib HTTP that does not let the stdlib replay a credential across a redirect.
+
+`urllib.request.Request` keeps two header bags. `headers` survives a redirect:
+`HTTPRedirectHandler.redirect_request` builds the follow-up request by copying it, stripping only
+content headers. `unredirected_hdrs` does not survive, and `AbstractHTTPHandler.do_open` sends both.
+application code that writes `Request(url, headers={"Authorization": ...})` lands the credential in
+the surviving bag, so the stdlib replays the token against whatever host the `Location` names.
+
+the fix is to move the caller's credential into the bag a redirect cannot copy. it survives any
+transport that forwards the request as it was handed over, so it still holds for a tracing or
+compatibility wrapper that only delegates.
+
+two things it does not cover, both because the credential ends up on an object this module never
+touched:
+
+- a transport that rebuilds the request. `Request.header_items()` merges both bags, so a wrapper
+  that reconstructs with `Request(url, headers=dict(req.header_items()))` puts the credential back
+  in the redirectable bag, and its inner call follows redirects normally.
+- a credential a handler adds mid-open. `redirect_request` builds each hop as a fresh `Request`, so
+  anything scoped to the object we were handed is gone by the second hop; a credentialed
+  `ProxyHandler` re-adds `Proxy-Authorization` to that fresh request and no work on ours reaches it.
+
+containing either is the redirect refusal's job, not this function's.
+
+on top of that the stdlib path refuses the hop outright. no flash or freesolo endpoint answers 3xx,
+so this rejects rather than deciding which hops are safe, and surfaces as an `HTTPError` that every
+caller already classifies.
+"""
+
+from __future__ import annotations
+
+import urllib.request
+from collections.abc import Callable
+from typing import Any
+
+_UrlOpen = Callable[..., Any]
+
+# headers that authenticate the request and must never reach a redirect target.
+_CREDENTIAL_HEADERS = ("authorization", "proxy-authorization", "cookie", "x-api-key")
+
+# the stdlib transport as it was before any test replaced it. comparing against the live
+# `urllib.request.urlopen` instead would defeat the check: a replaced attribute is trivially
+# identical to itself, so every fake would be routed through the opener stack and never called.
+_STDLIB_URLOPEN: _UrlOpen = urllib.request.urlopen
+
+
+class _NoRedirectHandler(urllib.request.HTTPRedirectHandler):
+    """raise 3xx instead of following it."""
+
+    def redirect_request(self, req, fp, code, msg, headers, newurl):
+        return None
+
+
+def _move_credentials_to_the_unredirected_bag(request: urllib.request.Request) -> None:
+    """move the caller's credential headers into the bag `redirect_request` does not copy.
+
+    `do_open` sends `unredirected_hdrs` and `headers` together, so the request is unchanged on the
+    wire. only the follow-up request a redirect would build loses them.
+    """
+
+    for name in list(request.headers):
+        if name.lower() not in _CREDENTIAL_HEADERS:
+            continue
+        # `do_open` gives `unredirected_hdrs` precedence, so an entry already there is the value
+        # being sent. dropping the redirectable duplicate keeps that principal on the wire;
+        # promoting it instead would silently authenticate as someone else.
+        canonical = request.unredirected_hdrs.get(name)
+        value = request.headers[name]
+        # `remove_header` pops from both bags, so it has to run before the value is re-added.
+        request.remove_header(name)
+        request.add_unredirected_header(name, value if canonical is None else canonical)
+
+
+def _urlopen_no_redirect(
+    request: urllib.request.Request,
+    *,
+    timeout: float,
+    urlopen: _UrlOpen | None = None,
+):
+    """open one authenticated request through a transport that refuses redirects.
+
+    the transport is read at call time rather than bound at import, because many tests replace
+    `urllib.request.urlopen` with a fake. a replaced transport is called directly: it is not the
+    stdlib opener stack, so routing it through a no-redirect opener would not reach it. every
+    authenticated call site in flash passes either nothing or `urllib.request.urlopen` itself, so
+    production always takes the refusing opener below; the direct branch exists for those fakes.
+
+    that makes the refusal guarantee conditional on the transport, and the condition is the global
+    being the one this module imported. rebinding `urllib.request.urlopen` after import - a test
+    fake, an apm shim installed at runtime - routes the call to the direct branch, which does not
+    refuse. relocation still runs first and still holds, so the credential cannot ride a hop the
+    replacement forwards as handed over; what is lost is refusal, and with it containment of the
+    two rebuild-and-handler gaps the module docstring names. a replacement transport owns its own
+    redirect policy.
+    """
+
+    _move_credentials_to_the_unredirected_bag(request)
+    transport = urlopen if urlopen is not None else urllib.request.urlopen
+    if transport is not _STDLIB_URLOPEN:
+        return transport(request, timeout=timeout)
+    return urllib.request.build_opener(_NoRedirectHandler()).open(request, timeout=timeout)

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/flash/cli/commands/env/testing/eval.py

@@ -202,27 +202,6 @@ def _score_case(
         )
 
 
-_READY_DEPLOYMENT_STATES = frozenset({"ready", "deployed"})
-_BUSY_DEPLOYMENT_STATES = frozenset({"queued", "smoke_testing", "reconciling"})
-
-
-def _live_deployment(client, run_id: str) -> dict | None:
-    """The run's deployment record, whatever state it is in, or None when it has none.
-
-    Match by run id because `deployment_for` misses step deployments. Busy records name the rollout
-    target and hide the revision that may still be serving.
-    """
-    for entry in client.deployments() or ():
-        listed = entry.get("deployment") or {}
-        if run_id not in (listed.get("run_id"), entry.get("run_id")):
-            continue
-        # the listing omits undeployed/dry_run rows, so anything here is servable.
-        if not listed.get("run_id") and entry.get("run_id"):
-            listed = {**listed, "run_id": entry["run_id"]}
-        return listed
-    return None
-
-
 def _case_ids(cases: list[EvalCase]) -> list[str]:
     """Stable per-case ids, disambiguating sidecars that reuse one id across cases.
 

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/flash/cli/scaffold/TRAINING.md

@@ -681,7 +681,7 @@ spending another GPU run:
 | Local-only env path in config                                | Config validation says there is no local path mode                                                                                                                                       | Publish first, then use the returned slug in `[environment] id`. `flash train` only runs published env ids, not local paths.                                                                                                                                                                                                                                                                                                    |
 | Config knobs are in the wrong table                          | Validation rejects `[grpo]`, `[sft]`, or unknown `[train]` keys                                                                                                                          | Put `epochs`, `group_size`, `max_completion_tokens`, `temperature`, `max_context_tokens`, LoRA, and other training knobs under `[train]`.                                                                                                                                                                                                                                                                                       |
 | GPU selection is not what you expected                       | Leaving `[gpu] type` unset may select a different fitting class as prices or capacity change                                                                                             | Set `[gpu] type` to an active validated class to hard-pin it, to a list (`["A100 PCIe", "A100 SXM"]`) to allow several, or leave it unset for managed cheapest-fit allocation. `train.hf_repo` remains platform-managed.                                                                                                                                                                                                        |
-| A pinned run stops after two capacity refusals               | The pinned class had no capacity, and no other class was allowed, so there was nowhere to fail over                                                                                      | Widen the search: drop the `[gpu] type` pin for managed allocation, name alternatives with `type = ["A100 PCIe", "A100 SXM"]`, or drop `[gpu] provider`. Flash stops once the class a retry would land on has refused capacity twice, rather than re-queueing on a class the provider keeps refusing. One refusal always earns a retry, since a momentary shortage is not a sold-out class.                                     |
+| A run stops after an infrastructure or OOM failure           | No candidate with strictly more usable VRAM remains, even if another provider offers the same-size GPU                                                                                   | Widen `[gpu] type` or the card-count ceiling to include a strictly larger fitting shape, or reduce the workload footprint. Flash preserves allocator order among larger candidates and never retries an equal-size shape. A managed shared-cache `no_capacity` or `poll_error` gets one cacheless same-shape retry before this rule applies.                                                                                    |
 | Secrets are not available on the worker                      | Reward code works locally but remote logs show missing API keys or auth failures                                                                                                         | List secret names under `[environment] secrets = [...]`, export those env vars locally before submit, or put them in local `.env` / `.env.local`. Never hard-code secret values in the config.                                                                                                                                                                                                                                  |
 | Scorer dependency is missing on the worker                   | Reward code works locally but every reward is `0.0` remotely; the import your scorer needs is installed in your venv, not the worker's                                                   | List the packages your scorer imports under `[environment] pip = ["pymongo>=4.6"]`. They are installed alongside Flash's own worker requirement. `flash env test` names the failing import locally before you spend a GPU on it. Entries name packages only: pip options and URLs with inline credentials are rejected, because the spec is stored and uploaded in plaintext (put the credential in `[environment] secrets`).   |
 | Wrong model / thinking setting                               | Config validation fails, or chat behavior does not match the run                                                                                                                         | Config validation is authoritative for model and thinking compatibility. Thinking is a run-level choice, and `flash models chat` does not expose an override flag.                                                                                                                                                                                                                                                              |
@@ -690,7 +690,7 @@ spending another GPU run:
 | Reward rises but behavior is worse                           | Short, templated, malformed, or reward-hacked outputs score well                                                                                                                         | Deploy the adapter and probe real examples. Add hard validity gates before judge calls, penalize degenerate shortcuts, and judge the outcome rather than the surface string.                                                                                                                                                                                                                                                    |
 | OPD makes the student worse, not better                      | The distilled adapter scores _below_ its SFT/base start even though the per-token loss fell                                                                                              | The teacher, not a knob, is the ceiling. Reverse-KL only pulls the student toward the managed GLM-5.2 teacher, so a teacher that is weak or wrong on _your_ task transfers its mistakes. Vet the teacher before you spend — the "On-policy distillation" section gives the two ways to do that with a managed key. If it can't beat your student, use GRPO or SFT instead — OPD cannot exceed a teacher that can't do the task. |
 | Output is truncated                                          | Correct-looking answers cut off mid-response or JSON is incomplete                                                                                                                       | Increase `max_completion_tokens` for GRPO/OPD rollouts or `max_context_tokens` for total prompt+completion context only after seeing truncation. Oversizing them by default just burns memory/cost.                                                                                                                                                                                                                             |
-| Infrastructure, CUDA, OOM, vLLM, or kernel failure           | Run errors before useful metrics, often during setup/model load                                                                                                                          | Treat this as infrastructure pressure, not proof the model is too large. Read `flash runs log <run-id>`, reduce footprint (`max_context_tokens`, `max_completion_tokens`, `group_size`) if needed, and let Flash retry/allocate another fitting GPU class.                                                                                                                                                                      |
+| Infrastructure, CUDA, OOM, vLLM, or kernel failure           | Run errors before useful metrics, often during setup/model load                                                                                                                          | Read `flash runs log <run-id>`. Flash advances a single executed-width-aware usable-VRAM floor for candidate-bound infrastructure and OOM failures, then retries only a strictly larger fitting candidate while keeping infrastructure and OOM budgets separate. Reduce `max_context_tokens`, `max_completion_tokens`, or `group_size` when no larger candidate remains.                                                        |
 | Run looks stuck after disconnecting                          | Terminal stopped streaming but the job may still be alive                                                                                                                                | Ctrl-C detaches. Use `flash runs log <run-id> --follow` to reattach, `flash runs log <run-id>` for the console/error output, or `flash runs cancel <run-id>` if you intentionally want to stop it.                                                                                                                                                                                                                              |
 | Two runs training the same thing                             | `flash runs list` shows a live run you thought you had killed, on top of the one you just submitted                                                                                      | Killing the `flash train` client only detaches; it never cancels. Always `flash runs list` and cancel the stale run _before_ re-submitting, and never wrap `flash train` in `timeout`.                                                                                                                                                                                                                                          |
 | Wrong identity spends the money                              | Push/train lands in an org you did not expect, or a run id you know is valid comes back "unknown"                                                                                        | `FREESOLO_API_KEY` in the environment silently overrides `flash login`. Run `flash whoami` first. Run visibility is scoped to the key that created the run, so archive result baselines to disk rather than relying on `flash runs list` to find them later.                                                                                                                                                                    |

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/flash/client/freesolo_api.py

@@ -9,6 +9,7 @@ import urllib.request
 from typing import Any
 
 from flash._internal.channel import CLI_NAME
+from flash._internal.http import _urlopen_no_redirect
 from flash.client.http import (
     FREESOLO_AUTH_VERIFY_PATH,
     FREESOLO_EVAL_RUNS_PATH,
@@ -34,7 +35,7 @@ def verify_freesolo_key(api_key: str, base_url: str | None = None) -> None:
         headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
     )
     try:
-        with urllib.request.urlopen(req, timeout=30) as resp:
+        with _urlopen_no_redirect(req, timeout=30) as resp:
             resp.read()
     except urllib.error.HTTPError as exc:
         if exc.code in (401, 403):
@@ -77,7 +78,7 @@ def _freesolo_request(
         headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
     )
     try:
-        with urllib.request.urlopen(req, timeout=timeout) as resp:
+        with _urlopen_no_redirect(req, timeout=timeout) as resp:
             raw = resp.read()
     except urllib.error.HTTPError as exc:
         if exc.code == 401:

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/flash/client/http.py

@@ -15,6 +15,7 @@ from collections.abc import Iterator, Mapping
 from typing import Any
 
 from flash._internal.channel import CLI_NAME
+from flash._internal.http import _urlopen_no_redirect
 from flash._internal.openai_sse import (
     DeltaEvent,
     ErrorEvent,
@@ -408,7 +409,7 @@ class ApiClient:
         deadline = time.monotonic() + body_deadline if body_deadline is not None else None
         with (
             self._translate_http_errors(),
-            urllib.request.urlopen(
+            _urlopen_no_redirect(
                 req, timeout=_capped_timeout(timeout or self.timeout, deadline)
             ) as resp,
         ):
@@ -435,7 +436,7 @@ class ApiClient:
         )
         with (
             self._translate_http_errors(),
-            urllib.request.urlopen(req, timeout=timeout or self.timeout) as resp,
+            _urlopen_no_redirect(req, timeout=timeout or self.timeout) as resp,
         ):
             if max_bytes is not None:
                 return _read_capped_response(resp, max_bytes)
@@ -804,7 +805,7 @@ class ApiClient:
         decoder = codecs.getincrementaldecoder("utf-8")()
         with (
             self._translate_http_errors(),
-            urllib.request.urlopen(req, timeout=30 * 60) as resp,
+            _urlopen_no_redirect(req, timeout=30 * 60) as resp,
         ):
             content_type = resp.headers.get("Content-Type", "")
             media_type = content_type.partition(";")[0].strip().lower()

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/flash/engine/worker/train/entry/opd_train_runner.py

@@ -318,6 +318,7 @@ def _write_child_shims(
         ("train/opd/child/plugin.py", "flash_opd_plugin.py"),
         ("train/opd/child/bridge.py", "flash_opd_bridge.py"),
         ("train/opd/child/structured.py", "flash_opd_structured.py"),
+        ("train/opd/child/tensors.py", "flash_opd_tensors.py"),
         ("train/opd/child/multiturn.py", "flash_opd_multiturn.py"),
         ("train/opd/child/entry.py", "flash_opd_entry.py"),
         ("train/opd/child/replay_guard.py", "flash_opd_replay_guard.py"),

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/flash/engine/worker/train/entry/sft_train.py

@@ -112,26 +112,6 @@ def _warmstart_adapter_path(
     return adapter_dir
 
 
-def _verl_image_message_content(content) -> str:
-    if isinstance(content, str):
-        return content
-    if content is None:
-        return ""
-    if not isinstance(content, list):
-        raise ValueError("multimodal message content must be text or content blocks")
-    parts: list[str] = []
-    for block in content:
-        if not isinstance(block, dict):
-            raise ValueError("multimodal message content blocks must be objects")
-        if block.get("type") == "text":
-            parts.append(str(block.get("text") or ""))
-        elif block.get("type") == "image":
-            parts.append("<image>")
-        else:
-            raise ValueError(f"unsupported multimodal SFT content block {block.get('type')!r}")
-    return "".join(parts)
-
-
 _restore_verl_resume = partial(restore_verl_resume, job_label="SFT")
 
 

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/flash/engine/worker/train/opd/child/plugin.py

@@ -26,6 +26,15 @@ if PLUGIN_LOADED_EXTERNALLY:
     from flash_multiturn_glue import multi_modal_image_count
     from flash_opd_multiturn import build_flash_multi_turn_agent_loop
     from flash_opd_structured import StructuredOutputReplay, canonical_structured_spec
+    from flash_opd_tensors import (
+        full_sequence_signal_sequences as _full_sequence_signal_sequences,
+    )
+    from flash_opd_tensors import (
+        packed_full_sequence as _packed_full_sequence,
+    )
+    from flash_opd_tensors import (
+        signal_sequences as _signal_sequences,
+    )
 else:
     from flash.engine.worker.train.core.child.glue import multi_modal_image_count
     from flash.engine.worker.train.opd.child.multiturn import build_flash_multi_turn_agent_loop
@@ -33,6 +42,15 @@ else:
         StructuredOutputReplay,
         canonical_structured_spec,
     )
+    from flash.engine.worker.train.opd.child.tensors import (
+        full_sequence_signal_sequences as _full_sequence_signal_sequences,
+    )
+    from flash.engine.worker.train.opd.child.tensors import (
+        packed_full_sequence as _packed_full_sequence,
+    )
+    from flash.engine.worker.train.opd.child.tensors import (
+        signal_sequences as _signal_sequences,
+    )
 
 _PERMANENT_TEACHER_EXIT = 86
 _TRANSIENT_TEACHER_EXIT = 87
@@ -86,16 +104,6 @@ def deterministic_rollout_seed(
     return int.from_bytes(digest, "big") & ((1 << 63) - 1)
 
 
-def _signal_sequences(group_ids, response_mask):
-    """Return the per-sequence mask for rows carrying at least one aligned student token."""
-    return ((group_ids >= 0) & response_mask.bool()).any(dim=-1)
-
-
-def _full_sequence_signal_sequences(group_ids):
-    """Detect aligned metadata in native full-sequence teacher tensors."""
-    return group_ids.ge(0).flatten(start_dim=1).any(dim=-1)
-
-
 def _flash_groupwise_reverse_kl_values(
     student_logprobs,
     teacher_logsums,
@@ -408,8 +416,14 @@ def _register_flash_distillation_loss() -> None:
     def flash_groupwise_reverse_kl(config, distillation_config, model_output, data):
         _set_current_global_batch_info(config, data)
         student_logprobs = no_padding_2_padding(model_output["log_probs"], data)
-        teacher_logsums = no_padding_2_padding(data["teacher_logprobs"], data).squeeze(-1)
-        group_ids = no_padding_2_padding(data["teacher_ids"], data).squeeze(-1).long()
+        teacher_logsums = no_padding_2_padding(
+            _packed_full_sequence(data["teacher_logprobs"], data), data
+        ).squeeze(-1)
+        group_ids = (
+            no_padding_2_padding(_packed_full_sequence(data["teacher_ids"], data), data)
+            .squeeze(-1)
+            .long()
+        )
         response_mask = data["response_mask"]
         if response_mask.is_nested:
             response_mask = response_mask.to_padded_tensor(False)

freesolo_flash_dev-1.2.127/flash/engine/worker/train/opd/child/tensors.py

@@ -0,0 +1,56 @@
+"""Batch-layout helpers for the flash OPD distillation loss.
+
+verl hands the loss a `TensorDict` whose fields are laid out per field rather than per batch:
+`list_of_dict_to_tensordict` stacks a field when every sample's tensor shares one shape and
+nests it only when the shapes differ. So `prompts`, `teacher_logprobs`, and `teacher_ids` can
+each arrive nested or padded independently, and the loss has to read all of them either way.
+"""
+
+from __future__ import annotations
+
+
+def signal_sequences(group_ids, response_mask):
+    """Return the per-sequence mask for rows carrying at least one aligned student token."""
+    return ((group_ids >= 0) & response_mask.bool()).any(dim=-1)
+
+
+def full_sequence_signal_sequences(group_ids):
+    """Detect aligned metadata in native full-sequence teacher tensors.
+
+    Padded only: `flatten` raises on a nested tensor's ragged dim, so callers holding a nested
+    tensor must reduce per row instead.
+    """
+    return group_ids.ge(0).flatten(start_dim=1).any(dim=-1)
+
+
+def packed_full_sequence(tensor, data):
+    """Put a full-sequence teacher tensor into the layout `no_padding_2_padding` documents.
+
+    That helper wants a nested tensor or `(total_nnz, *)` and reads `.values()` only when
+    nested, so a padded teacher measures as `bsz` and trips its `total_nnz` assertion.
+
+    The mask decides which positions are real, never the row's shape. verl pads a teacher row on
+    both sides (`_pad_teacher_outputs` left-pads by `prompt_width - prompt_length` and right-pads
+    the response), so a length alone cannot say where the real tokens sit and selecting a prefix
+    would silently train against the pad.
+    """
+    if tensor.is_nested:
+        return tensor
+    if "attention_mask" in data:
+        return tensor[data["attention_mask"].bool()]
+    # no mask: only a nested-prompt batch may omit that key, and only such a batch carries the
+    # lengths needed to check the stack. verl stacks this field only when every sequence shares
+    # one length, and pads a row only to the strided widths, so an unmasked stack is full.
+    prompts = data["prompts"]
+    if not prompts.is_nested:
+        raise AssertionError(
+            "flash OPD strided batch carries no attention mask, so real token positions "
+            "cannot be located"
+        )
+    lengths = prompts.offsets().diff() + data["responses"].offsets().diff()
+    if int(lengths.min()) != tensor.shape[1] or int(lengths.max()) != tensor.shape[1]:
+        raise AssertionError(
+            "flash OPD teacher tensor is padded but the batch carries no attention mask: "
+            f"width {tensor.shape[1]} against lengths {int(lengths.min())}..{int(lengths.max())}"
+        )
+    return tensor.flatten(0, 1)

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/flash/providers/core/base.py

@@ -929,9 +929,8 @@ class Provider(Protocol):
 
     # NOTE: ``supports_weight_cache: bool`` is an OPTIONAL capability attr (read via
     # ``getattr(prov, "supports_weight_cache", False)``, off this Protocol for the same isinstance
-    # reason as below): True only for the provider that offers the shared weight-cache network volume
-    # (RunPod). The runner gates its one-shot cache-less retry fallback on it; every other provider
-    # defaults False.
+    # reason as below): True for providers that attach the managed shared weight cache. The runner
+    # gates its one-shot cache-less retry fallback on it; every other provider defaults False.
 
     # ``run_instances_remaining(run_id)`` is optional and must stay off this runtime-checkable
     # protocol. callers use getattr in server/_runtime.py. ``[]`` confirms clear; non-empty means a

{freesolo_flash_dev-1.2.123 → freesolo_flash_dev-1.2.127}/flash/providers/lambda_/execution/provider.py

@@ -23,6 +23,7 @@ class LambdaProvider(InstanceProvider):
     """``base.Provider`` for the Lambda Cloud substrate."""
 
     name = "lambda"
+    supports_weight_cache = True
     _gpu_identity_attr = "lambda_name"
 
     @property