freesolo-flash-dev 1.2.124__tar.gz → 1.2.127__tar.gz

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: freesolo-flash-dev
-Version: 1.2.124
+Version: 1.2.127
 Summary: Flash — managed LoRA post-training (SFT/GRPO/OPD) for Freesolo environments, driven by the `flash` CLI
 Project-URL: Homepage, https://github.com/freesolo-co/flash
 Project-URL: Repository, https://github.com/freesolo-co/flash

freesolo_flash_dev-1.2.127/docs/hosted-inference-load-testing.md

@@ -0,0 +1,224 @@
+# Hosted inference load testing
+
+The `flash.serving.loadtest` package is an operator-only, public-HTTP load harness for the hosted
+inference front door. It is not imported by production serving and it does not create, resize, inspect,
+or destroy provider resources. Operators run it explicitly with the serving extra:
+
+```bash
+uv run --extra serving python -m flash.serving.loadtest validate scenario.json
+uv run --extra serving python -m flash.serving.loadtest discover scenario.json
+uv run --extra serving python -m flash.serving.loadtest run scenario.json results/fake-five-phase
+uv run --extra serving python -m flash.serving.loadtest summarize results/fake-five-phase
+uv run --extra serving python -m flash.serving.loadtest verify results/fake-five-phase
+```
+
+`validate` is allocation-free and performs no HTTP request. `discover` calls only `GET /healthz`.
+`run` calls only `GET /healthz` and `POST /v1/chat/completions`. The credential value is read from the
+environment variable named by `credential_env`; never put a credential value in a scenario file.
+There are no request retries.
+
+## Evidence boundary
+
+The harness verifies exact deployment SHA and deployment id, required capabilities, required base
+models, and immutable adapter response provenance. It checks health before and after every phase and
+at the midpoint of each duration phase. Its evidence remains HTTP-only:
+
+- it cannot prove that a live replica scaled out;
+- it cannot prove that an engine was actually cold or at scale zero;
+- it cannot distinguish a dispatch deadline from the underlying resource constraint;
+- it cannot establish an availability SLA;
+- fake and test runs cannot support production claims.
+
+A `cold_burst` therefore records either `cold_scale_out_intent_http_unattested` or
+`true_scale_zero_intent_http_unattested`. That is an honest statement of the authored intent, not an
+attestation of provider state.
+
+## Strict scenario
+
+Scenarios are strict Pydantic v2 JSON. Unknown fields and fields that belong to another phase kind are
+rejected. The authored `phases` list may contain any subset of the five supported discriminated kinds
+in any order, except that a present `cold_burst` must be the first inference-producing phase. A complete
+five-phase scenario uses `cold_burst`, `warm`, `sustained`, `mixed`, then `overload`.
+
+Dynamic base-model discovery reads `base_models` from `/healthz`, then applies `include`, `exclude`, and
+`require`. `require` is a hard gate.
+
+Explicit adapter targets must name the immutable model id, base model, and checkpoint. The request
+carries `X-Freesolo-Expected-Checkpoint` and the response must match `X-Freesolo-Checkpoint`.
+
+`adapter_revision` and `hf_revision` are optional, because the hosted app does not emit them on every
+deployment. Omitting one verifies checkpoint identity alone rather than passing a header check the
+deployment never answered. Supplying one asserts that the deployment does emit it, so a response that
+omits or changes it is a provenance mismatch, never a skipped assertion. A supplied `hf_revision` must
+still be a canonical 40-character hub commit SHA.
+
+`/healthz` is read for `ok`, deployment identity, capabilities, and `base_models`. `accounting_ok` is
+optional: a deployment that does not report it is not treated as an accounting failure, while an
+explicit `false` stops the run. Note that `/healthz.gpus` is the configured supported-base-model engine
+count, not a live container count, so it can never be read as a replica count.
+
+This fake example covers every phase kind:
+
+```json
+{
+  "schema_version": 1,
+  "name": "fake five-phase hosted inference",
+  "endpoint": "https://example.invalid",
+  "expected_deployment": {
+    "sha": "94210a3",
+    "deployment_id": "fake-deployment-1"
+  },
+  "credential_env": "FAKE_SERVING_TOKEN",
+  "required_capabilities": ["permanent_checkpoint_identity"],
+  "discovery": {
+    "enabled": true,
+    "include": ["model-a", "model-b", "model-c"],
+    "exclude": ["model-b"],
+    "require": ["model-a"]
+  },
+  "targets": [],
+  "profiles": [
+    {
+      "name": "short",
+      "weight": 1.0,
+      "messages": [{ "role": "user", "content": "Return one short sentence." }],
+      "max_tokens": 16,
+      "temperature": 0.0,
+      "top_p": 0.95
+    }
+  ],
+  "client": {
+    "connect_timeout_seconds": 5.0,
+    "read_timeout_seconds": 30.0,
+    "write_timeout_seconds": 10.0,
+    "pool_timeout_seconds": 5.0,
+    "max_in_flight": 8,
+    "max_scheduling_lag_ms": 100.0
+  },
+  "seed": 17,
+  "fake": true,
+  "phases": [
+    {
+      "name": "cold",
+      "kind": "cold_burst",
+      "requests": 4,
+      "burst_window_seconds": 0.2,
+      "cold_intent": "cold_scale_out"
+    },
+    {
+      "name": "warm",
+      "kind": "warm",
+      "requests": 4,
+      "concurrency": 2
+    },
+    {
+      "name": "sustained",
+      "kind": "sustained",
+      "duration_seconds": 2.0,
+      "rate_rps": 2.0
+    },
+    {
+      "name": "mixed",
+      "kind": "mixed",
+      "duration_seconds": 2.0,
+      "rate_rps": 3.0
+    },
+    {
+      "name": "overload",
+      "kind": "overload",
+      "expects_capacity_contract": false,
+      "stages": [
+        { "duration_seconds": 1.0, "rate_rps": 4.0 },
+        { "duration_seconds": 1.0, "rate_rps": 8.0 }
+      ]
+    }
+  ]
+}
+```
+
+`warm` is the only closed-loop phase. Its authored request count is completed with bounded
+concurrency. Every other phase is open-loop. Its dispatch offsets are computed before the phase
+starts. If the client cannot admit an offset within `max_in_flight` and `max_scheduling_lag_ms`, the
+harness writes a terminal `client_admission_missed` record instead of converting that offered request
+into delayed closed-loop work. Overload stages use the same rule and never retry.
+
+## Stream and artifact contract
+
+Streaming requests ask for usage. Every SSE `data:` payload must be valid JSON, followed by exactly
+one `[DONE]`; data after `[DONE]`, a missing terminal finish reason, or a missing `[DONE]` is a protocol
+error. The first nonempty `content` or `reasoning_content` delta defines time to first generated token.
+The first nonempty visible `content` delta is measured separately. Missing usage makes token throughput
+unavailable for that request; the harness never estimates tokens from chunks.
+
+Only HTTP 503 with error code `serving_capacity_unavailable` and `Retry-After: 1` is counted as exact
+capacity rejection. Other 503 responses and 429 responses are separate classes.
+
+An error body can echo the request back, so only a machine error code is read out of it. A code that
+does not look like a slug is recorded as `unrecognized_code`, and a free-text `message` is never
+persisted.
+
+### Overload cannot be demonstrated without a capacity contract
+
+Current Flash `dev` implements no retryable capacity contract. Its engine error mapping returns 504
+for an engine timeout and 502 for other upstream failures; there is no capacity 503 and no
+`Retry-After: 1`. An overload phase run against such a deployment therefore cannot produce a clean
+rejection signal at all.
+
+An overload phase declares this with `expects_capacity_contract`, which defaults to `false`. The flag
+never changes how a response is classified. It only records whether an absence of capacity rejections
+is an accurate description of the deployment or a failed expectation:
+
+| observed                        | `expects_capacity_contract` | verdict                                               |
+| ------------------------------- | --------------------------- | ----------------------------------------------------- |
+| at least one exact capacity 503 | either                      | `capacity_rejection_observed`                         |
+| none                            | `false`                     | `overload_not_demonstrated`                           |
+| none                            | `true`                      | `overload_not_demonstrated_despite_declared_contract` |
+
+**An absence of capacity rejections is never evidence of headroom.** A deployment without a capacity
+contract sheds load in ways this harness cannot distinguish from ordinary failure over public HTTP.
+When any overload phase declares no capacity contract, the summary carries that limitation explicitly.
+No overload phase is ever summarized as a success on the strength of a quiet run.
+
+Each run claims a new result directory and writes:
+
+- `scenario.authored.json`, with prompt content redacted;
+- `scenario.resolved.json`, with discovered targets and cold-intent labels;
+- `events.jsonl`, through one event writer with one terminal row per scheduled request;
+- `summary.json`, atomically;
+- `complete.json`, last, with row counts, byte counts, and SHA-256 hashes.
+
+An interrupted run closes the event stream but never writes `complete.json`. It remains inspectable and
+`verify` rejects it as incomplete. Version 1 has no partial-phase resume.
+
+Artifacts never contain credentials, authorization or request headers, generated text, raw response
+bodies, or per-request prompts. Persisted response headers are limited to content type, retry-after,
+and immutable provenance.
+
+## Metrics
+
+`summary.json` reports overall and per-phase, per-target, and per-profile values:
+
+- scheduled, dispatched, completed, successful, and client-admission-missed counts;
+- offered, dispatch, and completion rates with explicit denominator seconds and basis;
+- scheduling, header, first-generated, first-visible, and total latency p50, p90, p95, p99, and max;
+- usage coverage and token totals, aggregate output token rate, and decode token rate;
+- peak client in-flight, HTTP status and error counts, retry count, exact capacity ratio, and 429 count;
+- authored versus achieved target and profile mix;
+- explicit sustained and mixed authored, scheduling, dispatch, and completion windows.
+
+## B200 owner handoff
+
+The harness does not allocate a B200 and does not attest provider state. The B200 deployment owner must
+complete the provider-side handoff before an operator runs a non-fake scenario:
+
+1. identify the already deployed endpoint, exact release SHA, deployment id, and intended base models;
+2. confirm the B200 resource is owned, budgeted, and governed outside this harness;
+3. decide whether the cold intent is `cold_scale_out` or `true_scale_zero`, and establish that state
+   through the provider control plane without using the harness;
+4. provide only the name of the existing credential environment variable in the scenario;
+5. run `validate`, then `discover`, and confirm the resolved two-of-three or other intended model set;
+6. run into a new result directory, then run `verify` before interpreting `summary.json`;
+7. retain provider-side replica and hardware evidence separately if a scale-out or B200 claim is needed.
+
+A successful HTTP load run is not provider-side proof of B200 identity, replica count, scale-to-zero,
+or autoscaler behavior.

freesolo_flash_dev-1.2.127/flash/_internal/http.py

@@ -0,0 +1,101 @@
+"""authenticated stdlib HTTP that does not let the stdlib replay a credential across a redirect.
+
+`urllib.request.Request` keeps two header bags. `headers` survives a redirect:
+`HTTPRedirectHandler.redirect_request` builds the follow-up request by copying it, stripping only
+content headers. `unredirected_hdrs` does not survive, and `AbstractHTTPHandler.do_open` sends both.
+application code that writes `Request(url, headers={"Authorization": ...})` lands the credential in
+the surviving bag, so the stdlib replays the token against whatever host the `Location` names.
+
+the fix is to move the caller's credential into the bag a redirect cannot copy. it survives any
+transport that forwards the request as it was handed over, so it still holds for a tracing or
+compatibility wrapper that only delegates.
+
+two things it does not cover, both because the credential ends up on an object this module never
+touched:
+
+- a transport that rebuilds the request. `Request.header_items()` merges both bags, so a wrapper
+  that reconstructs with `Request(url, headers=dict(req.header_items()))` puts the credential back
+  in the redirectable bag, and its inner call follows redirects normally.
+- a credential a handler adds mid-open. `redirect_request` builds each hop as a fresh `Request`, so
+  anything scoped to the object we were handed is gone by the second hop; a credentialed
+  `ProxyHandler` re-adds `Proxy-Authorization` to that fresh request and no work on ours reaches it.
+
+containing either is the redirect refusal's job, not this function's.
+
+on top of that the stdlib path refuses the hop outright. no flash or freesolo endpoint answers 3xx,
+so this rejects rather than deciding which hops are safe, and surfaces as an `HTTPError` that every
+caller already classifies.
+"""
+
+from __future__ import annotations
+
+import urllib.request
+from collections.abc import Callable
+from typing import Any
+
+_UrlOpen = Callable[..., Any]
+
+# headers that authenticate the request and must never reach a redirect target.
+_CREDENTIAL_HEADERS = ("authorization", "proxy-authorization", "cookie", "x-api-key")
+
+# the stdlib transport as it was before any test replaced it. comparing against the live
+# `urllib.request.urlopen` instead would defeat the check: a replaced attribute is trivially
+# identical to itself, so every fake would be routed through the opener stack and never called.
+_STDLIB_URLOPEN: _UrlOpen = urllib.request.urlopen
+
+
+class _NoRedirectHandler(urllib.request.HTTPRedirectHandler):
+    """raise 3xx instead of following it."""
+
+    def redirect_request(self, req, fp, code, msg, headers, newurl):
+        return None
+
+
+def _move_credentials_to_the_unredirected_bag(request: urllib.request.Request) -> None:
+    """move the caller's credential headers into the bag `redirect_request` does not copy.
+
+    `do_open` sends `unredirected_hdrs` and `headers` together, so the request is unchanged on the
+    wire. only the follow-up request a redirect would build loses them.
+    """
+
+    for name in list(request.headers):
+        if name.lower() not in _CREDENTIAL_HEADERS:
+            continue
+        # `do_open` gives `unredirected_hdrs` precedence, so an entry already there is the value
+        # being sent. dropping the redirectable duplicate keeps that principal on the wire;
+        # promoting it instead would silently authenticate as someone else.
+        canonical = request.unredirected_hdrs.get(name)
+        value = request.headers[name]
+        # `remove_header` pops from both bags, so it has to run before the value is re-added.
+        request.remove_header(name)
+        request.add_unredirected_header(name, value if canonical is None else canonical)
+
+
+def _urlopen_no_redirect(
+    request: urllib.request.Request,
+    *,
+    timeout: float,
+    urlopen: _UrlOpen | None = None,
+):
+    """open one authenticated request through a transport that refuses redirects.
+
+    the transport is read at call time rather than bound at import, because many tests replace
+    `urllib.request.urlopen` with a fake. a replaced transport is called directly: it is not the
+    stdlib opener stack, so routing it through a no-redirect opener would not reach it. every
+    authenticated call site in flash passes either nothing or `urllib.request.urlopen` itself, so
+    production always takes the refusing opener below; the direct branch exists for those fakes.
+
+    that makes the refusal guarantee conditional on the transport, and the condition is the global
+    being the one this module imported. rebinding `urllib.request.urlopen` after import - a test
+    fake, an apm shim installed at runtime - routes the call to the direct branch, which does not
+    refuse. relocation still runs first and still holds, so the credential cannot ride a hop the
+    replacement forwards as handed over; what is lost is refusal, and with it containment of the
+    two rebuild-and-handler gaps the module docstring names. a replacement transport owns its own
+    redirect policy.
+    """
+
+    _move_credentials_to_the_unredirected_bag(request)
+    transport = urlopen if urlopen is not None else urllib.request.urlopen
+    if transport is not _STDLIB_URLOPEN:
+        return transport(request, timeout=timeout)
+    return urllib.request.build_opener(_NoRedirectHandler()).open(request, timeout=timeout)

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/client/freesolo_api.py

@@ -9,6 +9,7 @@ import urllib.request
 from typing import Any
 
 from flash._internal.channel import CLI_NAME
+from flash._internal.http import _urlopen_no_redirect
 from flash.client.http import (
     FREESOLO_AUTH_VERIFY_PATH,
     FREESOLO_EVAL_RUNS_PATH,
@@ -34,7 +35,7 @@ def verify_freesolo_key(api_key: str, base_url: str | None = None) -> None:
         headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
     )
     try:
-        with urllib.request.urlopen(req, timeout=30) as resp:
+        with _urlopen_no_redirect(req, timeout=30) as resp:
             resp.read()
     except urllib.error.HTTPError as exc:
         if exc.code in (401, 403):
@@ -77,7 +78,7 @@ def _freesolo_request(
         headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
     )
     try:
-        with urllib.request.urlopen(req, timeout=timeout) as resp:
+        with _urlopen_no_redirect(req, timeout=timeout) as resp:
             raw = resp.read()
     except urllib.error.HTTPError as exc:
         if exc.code == 401:

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/client/http.py

@@ -15,6 +15,7 @@ from collections.abc import Iterator, Mapping
 from typing import Any
 
 from flash._internal.channel import CLI_NAME
+from flash._internal.http import _urlopen_no_redirect
 from flash._internal.openai_sse import (
     DeltaEvent,
     ErrorEvent,
@@ -408,7 +409,7 @@ class ApiClient:
         deadline = time.monotonic() + body_deadline if body_deadline is not None else None
         with (
             self._translate_http_errors(),
-            urllib.request.urlopen(
+            _urlopen_no_redirect(
                 req, timeout=_capped_timeout(timeout or self.timeout, deadline)
             ) as resp,
         ):
@@ -435,7 +436,7 @@ class ApiClient:
         )
         with (
             self._translate_http_errors(),
-            urllib.request.urlopen(req, timeout=timeout or self.timeout) as resp,
+            _urlopen_no_redirect(req, timeout=timeout or self.timeout) as resp,
         ):
             if max_bytes is not None:
                 return _read_capped_response(resp, max_bytes)
@@ -804,7 +805,7 @@ class ApiClient:
         decoder = codecs.getincrementaldecoder("utf-8")()
         with (
             self._translate_http_errors(),
-            urllib.request.urlopen(req, timeout=30 * 60) as resp,
+            _urlopen_no_redirect(req, timeout=30 * 60) as resp,
         ):
             content_type = resp.headers.get("Content-Type", "")
             media_type = content_type.partition(";")[0].strip().lower()

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/serve/request/transport.py

@@ -180,7 +180,7 @@ def retryable_smoke_unavailable(
     expected_checkpoint_id: str,
     fallback_delay_seconds: float = _SMOKE_RETRY_FALLBACK_DELAY_SECONDS,
 ) -> serving_errors.RetryableServingUnavailable | None:
-    if response.status_code != 503:
+    if response.status_code not in {429, 503}:
         return None
     try:
         payload = response.json()
@@ -190,6 +190,20 @@ def retryable_smoke_unavailable(
         return None
     error = payload["error"]
     code = error.get("code")
+    exact_transient_server_error = (
+        response.status_code == 503 and code == "serving_capacity_unavailable"
+    )
+    if error.get("type") == "server_error" and exact_transient_server_error:
+        raw_delay = response.headers.get("Retry-After")
+        try:
+            retry_after_seconds = float(raw_delay)
+        except (TypeError, ValueError):
+            return None
+        if not math.isfinite(retry_after_seconds) or retry_after_seconds <= 0:
+            return None
+        return serving_errors.RetryableServingUnavailable(str(code), retry_after_seconds)
+    if response.status_code != 503:
+        return None
     if (
         error.get("type") != "adapter_unavailable"
         or error.get("retryable") is not True

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/server/billing/charges.py

@@ -6,6 +6,7 @@ import json
 import urllib.error
 import urllib.request
 
+from flash._internal.http import _urlopen_no_redirect
 from flash.core.spec import attributed_gpu_type
 from flash.cost.currency import usd_cents as _cents
 from flash.server.platform.internal_client import (
@@ -55,7 +56,7 @@ def _post_billing(*, token: str, path: str, body: dict) -> dict:
     """POST a JSON body to the backend billing path; raises BillingError on failure."""
     req = build_internal_request(path, body, token=token)
     try:
-        with urllib.request.urlopen(req, timeout=DEFAULT_TIMEOUT_S) as resp:
+        with _urlopen_no_redirect(req, timeout=DEFAULT_TIMEOUT_S) as resp:
             raw = resp.read()
     except urllib.error.HTTPError as exc:
         raise BillingError(exc.code, _http_error_detail(exc)) from exc

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/server/domain/ops/reconcile.py

@@ -13,6 +13,7 @@ import urllib.request
 
 import flash.runner.accounting.costs as runner_costs
 import flash.runner.lifecycle.status as runner_status
+from flash._internal.http import _urlopen_no_redirect
 from flash.providers.core.realized import realized_cost_for_remote
 from flash.runner.lifecycle.state import TERMINAL_STATES, RunStatus
 from flash.server.platform.auth import freesolo_base_url
@@ -54,7 +55,7 @@ def _report(body: dict) -> bool:
         headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"},
     )
     try:
-        with urllib.request.urlopen(req, timeout=_REPORT_TIMEOUT_S) as resp:
+        with _urlopen_no_redirect(req, timeout=_REPORT_TIMEOUT_S) as resp:
             resp.read()
         return True
     except OSError:

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/server/domain/registry/checkpoints.py

@@ -18,8 +18,8 @@ quietly (HF stays the source of truth) — only genuine backend failures warn.""
 from __future__ import annotations
 
 import json
-import urllib.request
 
+from flash._internal.http import _urlopen_no_redirect
 from flash.core.spec import require_project_id
 from flash.runner.results.checkpoints import list_checkpoints
 from flash.server.platform.internal_client import (
@@ -35,7 +35,7 @@ _RECORD_PATH = "/api/runs/internal/checkpoints"
 def _post_checkpoints(*, token: str, body: dict) -> dict:
     """POST the checkpoint batch to the backend; raise on any non-2xx/unreachable."""
     req = build_internal_request(_RECORD_PATH, body, token=token)
-    with urllib.request.urlopen(req, timeout=DEFAULT_TIMEOUT_S) as resp:
+    with _urlopen_no_redirect(req, timeout=DEFAULT_TIMEOUT_S) as resp:
         raw = resp.read()
     try:
         return json.loads(raw or b"{}")

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/server/domain/registry/environment_registry.py

@@ -10,6 +10,7 @@ from collections.abc import Set as AbstractSet
 
 from fastapi import HTTPException
 
+from flash._internal.http import _urlopen_no_redirect
 from flash.core.spec import require_project_id
 from flash.server.platform.auth import standalone
 from flash.server.platform.internal_client import (
@@ -117,7 +118,7 @@ def require_environment_project(
         token=token,
     )
     try:
-        with urllib.request.urlopen(request, timeout=DEFAULT_TIMEOUT_S) as response:
+        with _urlopen_no_redirect(request, timeout=DEFAULT_TIMEOUT_S) as response:
             raw = response.read()
     except urllib.error.HTTPError as exc:
         detail = error_detail(exc)

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/server/domain/registry/projects.py

@@ -10,6 +10,7 @@ from typing import Any
 
 from fastapi import HTTPException
 
+from flash._internal.http import _urlopen_no_redirect
 from flash.core.spec import require_project_id
 from flash.server.platform.auth import freesolo_base_url, standalone
 from flash.server.platform.internal_client import build_internal_request, internal_key
@@ -104,7 +105,7 @@ def _require_internal_project_access(*, project_id: str, org_id: str) -> tuple[s
         token=token,
     )
     try:
-        with urllib.request.urlopen(request, timeout=_PROJECT_TIMEOUT_S) as response:
+        with _urlopen_no_redirect(request, timeout=_PROJECT_TIMEOUT_S) as response:
             status = int(response.status)
             raw = response.read()
     except urllib.error.HTTPError as exc:
@@ -260,7 +261,7 @@ def _project_access(
         headers=headers,
     )
     try:
-        with urllib.request.urlopen(req, timeout=_PROJECT_TIMEOUT_S) as resp:
+        with _urlopen_no_redirect(req, timeout=_PROJECT_TIMEOUT_S) as resp:
             raw = resp.read()
     except urllib.error.HTTPError as exc:
         detail = _error_detail(exc)

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/server/platform/auth.py

@@ -11,6 +11,7 @@ import urllib.request
 from concurrent.futures import Future
 from typing import Any
 
+from flash._internal.http import _urlopen_no_redirect
 from flash.server.platform import db
 
 INTERNAL_KEY_ENV = "FREESOLO_INTERNAL_KEY"
@@ -196,13 +197,16 @@ def _freesolo_verify(token: str) -> bool:
         identity: dict[str, Any] = {}
         cache_result = True
         try:
-            with urllib.request.urlopen(req, timeout=_VERIFY_TIMEOUT_S) as resp:
+            with _urlopen_no_redirect(req, timeout=_VERIFY_TIMEOUT_S) as resp:
                 verified = resp.status == 200
                 if verified:
                     identity = _identity_from_verify_body(_response_body(resp))
         except urllib.error.HTTPError as exc:
-            # treat 5xx/429 as transient and do not cache them.
-            if exc.code >= 500 or exc.code == 429:
+            # treat 5xx/429 as transient and do not cache them. a 3xx is transient for a different
+            # reason: `_urlopen_no_redirect` raises it instead of following the hop, so the backend
+            # never judged this token at all. caching that as a negative would keep a valid key
+            # failing for the whole negative TTL after the redirect condition is gone.
+            if exc.code >= 500 or exc.code == 429 or 300 <= exc.code < 400:
                 cache_result = False
             verified = False
         except (OSError, ValueError):

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/server/platform/internal_client.py

@@ -12,6 +12,7 @@ from contextlib import suppress
 from logging import Logger
 from typing import Any
 
+from flash._internal.http import _urlopen_no_redirect
 from flash.server.platform.auth import (
     INTERNAL_KEY_ENV,
     freesolo_base_url,
@@ -131,7 +132,7 @@ def request_internal_json(
     method: str,
     subject: str,
     logger: Logger,
-    urlopen: UrlOpen = urllib.request.urlopen,
+    urlopen: UrlOpen | None = None,
     raise_for: AbstractSet[int] | None = None,
     expected: AbstractSet[int] | None = None,
 ) -> bool:
@@ -150,7 +151,7 @@ def request_internal_json(
         return False
     req = build_internal_request(path, body, token=key, method=method)
     try:
-        with urlopen(req, timeout=DEFAULT_TIMEOUT_S) as resp:
+        with _urlopen_no_redirect(req, timeout=DEFAULT_TIMEOUT_S, urlopen=urlopen) as resp:
             return 200 <= resp.status < 300
     except urllib.error.HTTPError as exc:
         detail = error_detail(exc)
@@ -171,7 +172,7 @@ def post_internal_json(
     *,
     subject: str,
     logger: Logger,
-    urlopen: UrlOpen = urllib.request.urlopen,
+    urlopen: UrlOpen | None = None,
     raise_for: AbstractSet[int] | None = None,
     expected: AbstractSet[int] | None = None,
 ) -> bool:
@@ -193,7 +194,7 @@ def delete_internal_json(
     *,
     subject: str,
     logger: Logger,
-    urlopen: UrlOpen = urllib.request.urlopen,
+    urlopen: UrlOpen | None = None,
 ) -> bool:
     return request_internal_json(
         path,

{freesolo_flash_dev-1.2.124 → freesolo_flash_dev-1.2.127}/flash/serving/app/README.md

@@ -27,6 +27,12 @@ The routing layer (`src/router.py`) carries no `modal`/`vllm` imports and is exh
 unit-tested in `tests/serving/test_router.py` (multi-base-model dispatch, shared-GPU multi-LoRA,
 register-routing, OpenAI shape, auth, 404s) — run with `uv run pytest tests/serving/`.
 
+Operator load tests use the isolated `flash.serving.loadtest` package, documented in
+[`docs/hosted-inference-load-testing.md`](../../../docs/hosted-inference-load-testing.md). It is
+never imported by production serving. It drives only `GET /healthz` and `POST /v1/chat/completions`,
+manages no Modal lifecycle, and cannot by itself prove replica scale-out, cold state, GPU identity,
+the cause of a capacity rejection, or an availability SLA.
+
 Persistence is only for hydration and recovery. On startup each base model's engine loads
 _its_ ready adapters from the `hosted_lora_adapters` Supabase table; `POST /adapters`
 registers a new adapter on its base model's engine and tracks it in the router.

freesolo_flash_dev-1.2.127/flash/serving/loadtest/__init__.py

@@ -0,0 +1,5 @@
+"""operator-only hosted inference load-testing package."""
+
+from flash.serving.loadtest.schema import Scenario
+
+__all__ = ["Scenario"]

freesolo_flash_dev-1.2.127/flash/serving/loadtest/__main__.py

@@ -0,0 +1,5 @@
+"""module entry point for the operator-only load-test cli."""
+
+from flash.serving.loadtest.cli import main
+
+raise SystemExit(main())