freesolo-flash-dev 1.2.120__tar.gz → 1.2.122__tar.gz

{freesolo_flash_dev-1.2.120 → freesolo_flash_dev-1.2.122}/CHANGELOG.md

@@ -25,6 +25,11 @@ This file starts at 1.1.35. Earlier releases are not reconstructed here; use
 
 ### Changed
 
+- Run-backed adapters now use only explicit permanent checkpoint identities, `<run_id>/final` and
+  `<run_id>/step-N`. Bare run aliases, composite source-revision identities, activation, implicit
+  checkpoint fallback, and public artifact provenance have been removed across hosted, managed, and
+  customer-owned serving. Checkpoint bindings are write-once, exact undeploy preserves siblings,
+  and customer-owned manifests use schema v2.
 - Active training now exposes exactly `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
   `Qwen/Qwen3.6-35B-A3B`. Hosted serving remains active for 9B and 35B-A3B; hosted Qwen3.8 27B
   remains inactive. Qwen3.8 uses the immutable BF16 revision

{freesolo_flash_dev-1.2.120 → freesolo_flash_dev-1.2.122}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: freesolo-flash-dev
-Version: 1.2.120
+Version: 1.2.122
 Summary: Flash — managed LoRA post-training (SFT/GRPO/OPD) for Freesolo environments, driven by the `flash` CLI
 Project-URL: Homepage, https://github.com/freesolo-co/flash
 Project-URL: Repository, https://github.com/freesolo-co/flash
@@ -174,8 +174,8 @@ lora_rank = 32              # lora_alpha defaults to 2 x lora_rank; set it to ov
 ```bash
 flash train run.toml                  # submit, prints a run id
 flash runs status RUN_ID               # follow it
-flash models deploy RUN_ID             # serve the trained adapter
-flash models chat RUN_ID -m "hello"    # talk to it
+flash models deploy RUN_ID/final             # serve the final checkpoint
+flash models chat RUN_ID/final -m "hello"    # talk to it
 ```
 
 Run management lives under `flash runs` (`list`, `status`, `log`, `cancel`, `checkpoint`)
@@ -184,12 +184,12 @@ and serving under `flash models` (`deploy`, `chat`, `deployments`, `undeploy`, `
 lists GPU classes with estimated $/hr.
 
 Intermediate RL checkpoints are deployable: list them with `flash runs checkpoint RUN_ID`,
-then pass `RUN_ID/step-N` as the adapter id. To copy a finished adapter into your own
+then pass `RUN_ID/step-N` as the checkpoint id. To copy a finished adapter into your own
 HuggingFace repo:
 
 ```bash
 export HF_TOKEN=hf_...
-flash models export --adapter-id RUN_ID --repository your-org/your-repo
+flash models export --adapter-id RUN_ID/final --repository your-org/your-repo
 ```
 
 There are no built-in task environments — the environment you push defines the task.
@@ -197,7 +197,7 @@ Single-turn and bounded multi-turn environments are supported.
 
 ### Calling a deployed adapter from your own app
 
-Deploy once with `flash models deploy RUN_ID`, then POST chat requests with your API key:
+Deploy once with `flash models deploy RUN_ID/final`, then POST chat requests with your API key:
 
 ```bash
 export RUN_ID=flash-1782194170-ce1cfcff
@@ -206,12 +206,12 @@ export FREESOLO_API_KEY=fslo_...
 curl -X POST "https://flash.freesolo.co/v1/runs/$RUN_ID/chat" \
   -H "Authorization: Bearer $FREESOLO_API_KEY" \
   -H "Content-Type: application/json" \
-  -d '{"messages": [{"role": "user", "content": "hello"}], "max_tokens": 256}'
+  -d "{\"checkpoint_id\": \"$RUN_ID/final\", \"messages\": [{\"role\": \"user\", \"content\": \"hello\"}], \"max_tokens\": 256}"
 ```
 
 The response uses the OpenAI chat-completions shape; read `choices[0].message.content`. The
-run id is the adapter id. If the run is not deployed yet, the endpoint returns `409` with a
-hint to deploy first. Prefer this control-plane endpoint over calling the serving backend
+checkpoint id is the permanent serving identity. If that checkpoint is not deployed yet, the
+endpoint returns `409` with a hint to deploy first. Prefer this control-plane endpoint over calling the serving backend
 directly: it enforces run ownership and forwards per-run serving options such as
 thinking-mode parity.
 

{freesolo_flash_dev-1.2.120 → freesolo_flash_dev-1.2.122}/README.md

@@ -91,8 +91,8 @@ lora_rank = 32              # lora_alpha defaults to 2 x lora_rank; set it to ov
 ```bash
 flash train run.toml                  # submit, prints a run id
 flash runs status RUN_ID               # follow it
-flash models deploy RUN_ID             # serve the trained adapter
-flash models chat RUN_ID -m "hello"    # talk to it
+flash models deploy RUN_ID/final             # serve the final checkpoint
+flash models chat RUN_ID/final -m "hello"    # talk to it
 ```
 
 Run management lives under `flash runs` (`list`, `status`, `log`, `cancel`, `checkpoint`)
@@ -101,12 +101,12 @@ and serving under `flash models` (`deploy`, `chat`, `deployments`, `undeploy`, `
 lists GPU classes with estimated $/hr.
 
 Intermediate RL checkpoints are deployable: list them with `flash runs checkpoint RUN_ID`,
-then pass `RUN_ID/step-N` as the adapter id. To copy a finished adapter into your own
+then pass `RUN_ID/step-N` as the checkpoint id. To copy a finished adapter into your own
 HuggingFace repo:
 
 ```bash
 export HF_TOKEN=hf_...
-flash models export --adapter-id RUN_ID --repository your-org/your-repo
+flash models export --adapter-id RUN_ID/final --repository your-org/your-repo
 ```
 
 There are no built-in task environments — the environment you push defines the task.
@@ -114,7 +114,7 @@ Single-turn and bounded multi-turn environments are supported.
 
 ### Calling a deployed adapter from your own app
 
-Deploy once with `flash models deploy RUN_ID`, then POST chat requests with your API key:
+Deploy once with `flash models deploy RUN_ID/final`, then POST chat requests with your API key:
 
 ```bash
 export RUN_ID=flash-1782194170-ce1cfcff
@@ -123,12 +123,12 @@ export FREESOLO_API_KEY=fslo_...
 curl -X POST "https://flash.freesolo.co/v1/runs/$RUN_ID/chat" \
   -H "Authorization: Bearer $FREESOLO_API_KEY" \
   -H "Content-Type: application/json" \
-  -d '{"messages": [{"role": "user", "content": "hello"}], "max_tokens": 256}'
+  -d "{\"checkpoint_id\": \"$RUN_ID/final\", \"messages\": [{\"role\": \"user\", \"content\": \"hello\"}], \"max_tokens\": 256}"
 ```
 
 The response uses the OpenAI chat-completions shape; read `choices[0].message.content`. The
-run id is the adapter id. If the run is not deployed yet, the endpoint returns `409` with a
-hint to deploy first. Prefer this control-plane endpoint over calling the serving backend
+checkpoint id is the permanent serving identity. If that checkpoint is not deployed yet, the
+endpoint returns `409` with a hint to deploy first. Prefer this control-plane endpoint over calling the serving backend
 directly: it enforces run ownership and forwards per-run serving options such as
 thinking-mode parity.
 

freesolo_flash_dev-1.2.122/docs/serving-contract.md

@@ -0,0 +1,122 @@
+# The Flash serving contract
+
+`flash models deploy`, `chat`, `evaluate`, and `undeploy` use an HTTP serving backend and one public
+identity for every run-backed adapter. `flash/serve/` owns the client, shared runtime, and
+customer-owned provider deployment. Freesolo's hosted backend (`flash/serving/`) is another
+implementation of this contract. A deployment serves one exact base model and its compatible LoRA
+adapters. A different base model requires a separate deployment. Use
+`flash serve deploy --provider modal` to provision one in your own Modal account.
+
+Customer-owned Modal is live-qualified for `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
+`Qwen/Qwen3.6-35B-A3B`. The 27B and 35B-A3B qualifications are bound to the exact certified serving
+image digest; another digest remains available for offline planning but cannot allocate Modal.
+Hosted Qwen3.8 27B is active on H100. Its customer-owned engine runs on `H100!` and serves the pinned
+`Qwen/Qwen3.8-27B-FP8` checkpoint while preserving `Qwen/Qwen3.8-27B` as the distinct logical base
+and tokenizer provenance. The 35B-A3B engine serves BF16 weights on one H200 with FP8 KV cache, a
+32K context, eight sequences, a 4096 batched-token cap, and six rank-64 LoRA slots.
+
+Customer-owned `flash serve` is Modal-only and requires the explicit provider argument. Historical
+customer-serving RunPod deployment identities are unsupported, with no migration, status, undeploy,
+or teardown shim. Managed RunPod training remains supported and unchanged. Retired Qwen3.6 27B is
+never translated into Qwen3.8 27B.
+
+Run-backed adapters use only:
+
+- `<run_id>/final`
+- `<run_id>/step-N`
+
+The grammar is exact. Bare run ids, omitted checkpoints, padded steps, whitespace, extra path segments, and old composite identities are rejected. A run id is grouping and authorization metadata only.
+
+## Permanent checkpoint binding
+
+`POST /adapters` registers one exact checkpoint:
+
+```json
+{
+  "adapter_id": "run-abc/step-10",
+  "checkpoint": "run-abc/step-10",
+  "run_id": "run-abc",
+  "checkpoint_step": 10,
+  "repo_id": "acme/artifacts",
+  "repo_type": "dataset",
+  "subfolder": "sft/run-abc/adapter",
+  "artifact_revision": "<private-40-hex-source-commit>",
+  "artifact_digest": "<private-64-hex-content-digest>",
+  "artifact_fingerprint": "<private-64-hex-binding-fingerprint>",
+  "base_model": "Qwen/Qwen3.5-9B",
+  "lora_rank": 16,
+  "org_id": "org-id",
+  "thinking": false,
+  "structured_outputs": null
+}
+```
+
+The registration schema is closed. Lifecycle fields are server-owned. The first registration binds the org-scoped checkpoint to its private source location, content identity, base model, LoRA shape, and serving defaults. An identical retry is idempotent. Any changed immutable fact returns `409`, and undeploy never frees the identity for reuse.
+
+Private artifact revision, digest, repository, and subfolder exist only for deterministic retrieval and cache integrity. They are not model identities and do not appear in public chat provenance, evaluation identities, analytics, or billing payloads.
+
+## Authentication and health
+
+Customer-owned deployments use `Authorization: Bearer <FLASH_SERVING_KEY>`. Freesolo-hosted serving uses `X-Freesolo-Internal-Key` for internal control operations and Freesolo API-key authorization for caller traffic. Credentials are compared without exposing them and are removed from cross-origin redirects.
+
+`GET /healthz` does not load a model. Dynamic hosted serving advertises permanent checkpoint identity support. Customer-owned deployments receive an immutable manifest at boot and expose no dynamic registration surface.
+
+## Readiness and routing
+
+Registration returns `200` or `202`. `GET /adapters/{checkpoint_id}` reports lifecycle for that exact binding. A checkpoint becomes routable only after its exact adapter load succeeds and the generation-fenced durable row is promoted to `ready`.
+
+Routing is direct. `POST /v1/chat/completions` requires the exact checkpoint id as `model`. There is no alias lookup, activation operation, current checkpoint, implicit final, first checkpoint, or sole-checkpoint fallback. Tenant authorization is checked against the checkpoint binding's owning organization before dispatch.
+
+The packaged, hosted, and managed OpenAI entry points retain the shared strict sampling grammar:
+
+- `n` is an exact integer from 1 through 4, and `n > 1` requires `temperature > 0`.
+- `seed` is null or a signed 64-bit integer, excluding `-1`.
+- frequency and presence penalties are finite values from -2 through 2.
+- `logprobs` is a strict boolean and `top_logprobs` is an integer from 0 through 20.
+- thinking-enabled adapters reject logprobs.
+
+## Public provenance
+
+Successful run-backed responses identify only the permanent checkpoint:
+
+```json
+{
+  "freesolo": {
+    "checkpoint_id": "run-abc/step-10"
+  }
+}
+```
+
+Hosted responses also send `X-Freesolo-Checkpoint`. Customer-owned responses expose `checkpoint_id` in `flash_provenance`. Stream and buffered responses must attest to the same checkpoint. Private source commits and digests are never returned.
+
+## Undeploy and cleanup
+
+`DELETE /adapters/{checkpoint_id}` disables exactly one checkpoint:
+
+```json
+{
+  "ok": true,
+  "checkpoint_id": "run-abc/step-10",
+  "run_id": "run-abc",
+  "disabled_checkpoints": ["run-abc/step-10"]
+}
+```
+
+Sibling checkpoints remain independently ready and callable. Run-wide cleanup is a separately named internal administrative operation that first enumerates the run's exact checkpoint bindings.
+
+## Customer-owned deployments
+
+Manifest schema v2 keys adapters by checkpoint id. It keeps the private source commit, exact file table, and aggregate digest for hydration and cache verification, but publishes only checkpoint identities through `/v1/models` and chat provenance. Manifest and deployment identity v1 are rejected rather than translated.
+
+Modal deployments retain their existing provider topology, lifecycle fencing, direct authenticated HTTPS endpoint, capacity validation, and teardown behavior.
+
+## Cross-repository schema dependency
+
+Flash expects the coordinated Freesolo migration to provide:
+
+- `flash.hosted_lora_adapters` with explicit checkpoint and private binding columns;
+- `public.bind_hosted_checkpoint(...)` for atomic insert-or-identical-read semantics;
+- lifecycle-only mutable updates protected by immutable-field database enforcement;
+- accounting RPCs accepting `checkpoint_id` and private `artifact_fingerprint` without legacy identity fields.
+
+The old and new binaries are not compatible. Deploy them only through the documented maintenance-window cutover.

{freesolo_flash_dev-1.2.120 → freesolo_flash_dev-1.2.122}/flash/cli/commands/env/testing/eval.py

@@ -202,12 +202,7 @@ def _score_case(
         )
 
 
-# the server's own names for a record that is serving traffic versus one still coming up
-# (flash/server/routes/serving.py). a busy record's `adapter_revision` is the INCOMING revision.
 _READY_DEPLOYMENT_STATES = frozenset({"ready", "deployed"})
-# a busy redeploy may still serve its predecessor, which the public listing hides.
-# `/v1/deployments` excludes only `undeployed`/`dry_run`, so only `ready` and `busy`
-# may answer; other returned states fail closed (flash/cli/commands/__init__.py).
 _BUSY_DEPLOYMENT_STATES = frozenset({"queued", "smoke_testing", "reconciling"})
 
 
@@ -328,10 +323,6 @@ def _generate_concurrently(
             pending.put((index, prompt))
     outstanding = pending.qsize()
     if outstanding:
-        # warm the step-selector capability once before workers start. only successful
-        # `/v1/health` checks are cached, so cold workers would multiply a transient or
-        # unsupported failure. failures propagate here; only `RUN/step-N` triggers the check.
-        client.warm_chat_step_selector(target)
         finished: queue.SimpleQueue = queue.SimpleQueue()
         aborted = threading.Event()
         abort_lock = threading.Lock()
@@ -555,114 +546,37 @@ def _print_report(report: EvalSuiteReport) -> None:
 
 
 def _resolve_evaluation_target(
-    args, client, parsed, parse_adapter_revision, api_error, client_error
-) -> tuple[str, int | None]:
-    """Resolve the deployed evaluation target phase."""
-    evaluation_target = args.target
-    if parsed is None:
-        return evaluation_target, None
+    args, client, parsed, api_error, client_error
+) -> tuple[str, object, int | None]:
+    """require the exact requested checkpoint to have verified ready authority."""
 
-    run_id, want_step = parsed
+    run_id, _ = parsed
     try:
-        deployment = _live_deployment(client, run_id)
+        run = client.get_run(run_id)
     except (api_error, client_error) as exc:
         if getattr(args, "debug", False):
             raise
-        _err(f"env eval failed: could not resolve deployed revision for {run_id}: {exc}")
-        return evaluation_target, _err("overall: FAIL")
-    if deployment is None:
-        _err(f"env eval failed: run {args.target} is not deployed")
-        return evaluation_target, _err("overall: FAIL")
-    deployment_state = deployment.get("state")
-    servable_states = _READY_DEPLOYMENT_STATES | _BUSY_DEPLOYMENT_STATES
-    if want_step is not None:
-        # pinned steps resolve through the verified ledger, not the deployment record
-        # (`flash/server/routes/serving.py`). allow `failed`, which preserves that ledger;
-        # revocation-failed and undeploy paths invalidate it in `flash/runner/supervise/deploy.py`.
-        servable_states = servable_states | {"failed"}
-    if deployment_state not in servable_states:
-        # having a record is not having a servable one: the listing keeps terminal states like
-        # `failed` and `revocation_failed`, and the chat route has no ready predecessor to fall
-        # back to for them, so every case 409s. that spends a whole suite of generation failures
-        # to learn what one target error says now. a step that genuinely is not in the ledger
-        # still gets one 409 naming the deployed steps, not a suite of them.
-        _err(
-            f"env eval failed: run {args.target} deployment is {deployment_state or 'unknown'}"
-            f"; run `{CLI_NAME} models deploy {run_id}` first"
-        )
-        return evaluation_target, _err("overall: FAIL")
-    # a busy record names the rollout target, not the hidden predecessor still serving.
-    # leave the user's target unpinned so the chat route can resolve private rollback state or
-    # return the correct 409 when nothing serves.
-    resolved = None
-    if deployment_state in _READY_DEPLOYMENT_STATES:
-        candidate = deployment.get("adapter_revision")
-        resolved = parse_adapter_revision(candidate) if isinstance(candidate, str) else None
-        if resolved is None or resolved[0] != run_id:
-            _err(
-                f"env eval failed: deployment for {run_id} has no valid immutable adapter revision"
-            )
-            return evaluation_target, _err("overall: FAIL")
-    # `RUN/step-N` is not immutable: multiple revisions may share a step. pin only when the
-    # live deployment is that step; otherwise the server-only verified ledger may still resolve
-    # an older step, and the chat route must decide ambiguity.
-    if resolved is not None and (want_step is None or resolved[1] == want_step):
-        evaluation_target = candidate.strip()
-        print(f"resolved evaluation target {args.target} to {evaluation_target}")
-    elif args.upload and want_step is not None:
-        # uploaded reports must name immutable weights. the CLI cannot resolve an ambiguous
-        # `RUN/step-N`: the verified ledger is server-only and chat does not echo the revision.
-        # refuse before generation; bare aliases remain valid because they mean the currently
-        # serving revision, including a busy predecessor.
-        _err(
-            f"env eval failed: cannot upload results for {args.target}: the immutable "
-            "revision that will answer it is not knowable here. re-run with the full "
-            f"revision from `{CLI_NAME} models deployments`, or pass --no-upload"
-        )
-        return evaluation_target, _err("overall: FAIL")
-    return evaluation_target, None
-
-
-def _load_evaluation_run_spec(
-    args, client, revision, parsed, api_error, client_error
-) -> tuple[object, int | None]:
-    """Load the target run specification phase."""
-    # one run lookup supplies its training environment, reasoning mode, and owning project.
-    # these are target-run properties, not fallbacks, and reading once avoids one lookup per case.
-    spec = None
-    target_run_id = (revision or parsed or (None,))[0]
-    if not target_run_id:
-        return spec, None
-    try:
-        spec = client.get_run(target_run_id).get("spec")
-    except client_error as exc:
-        # `ApiError` subclasses `ClientError`, so one handler must preserve retryable 5xx/429.
-        # both arms abort because the run's environment is required; they differ only in
-        # whether retrying can help.
-        if getattr(args, "debug", False):
-            raise
-        answered_definitively = (
-            isinstance(exc, api_error) and exc.status < 500 and exc.status != 429
-        )
-        if answered_definitively:
-            _err(
-                f"env eval failed: could not read the target run {target_run_id}: {exc}. "
-                "its published environment is what supplies the suites to score."
-            )
-        else:
-            _err(
-                f"env eval failed: could not reach the control plane for {target_run_id}: "
-                f"{exc}. retry once it is reachable."
-            )
-        return spec, _err("overall: FAIL")
+        _err(f"env eval failed: could not resolve verified checkpoints for {run_id}: {exc}")
+        return args.target, None, _err("overall: FAIL")
     except Exception as exc:
-        # anything else is not a transport fault, so it is not retryable either. broad, so an
-        # unexpected client shape cannot crash a command the user asked for.
         if getattr(args, "debug", False):
             raise
-        _err(f"env eval failed: could not read the target run {target_run_id}: {exc}")
-        return spec, _err("overall: FAIL")
-    return spec, None
+        _err(f"env eval failed: could not resolve verified checkpoints for {run_id}: {exc}")
+        return args.target, None, _err("overall: FAIL")
+    verified = run.get("verified_checkpoints") if isinstance(run, dict) else None
+    if not isinstance(verified, list) or any(not isinstance(item, str) for item in verified):
+        _err(f"env eval failed: control plane returned invalid checkpoint authority for {run_id}")
+        return args.target, None, _err("overall: FAIL")
+    if args.target not in verified:
+        _err(f"env eval failed: checkpoint {args.target} has not passed deployment verification")
+        return args.target, None, _err("overall: FAIL")
+    return args.target, run, None
+
+
+def _load_evaluation_run_spec(run: object) -> object:
+    """load the target run specification from the verified authority record."""
+
+    return run.get("spec") if isinstance(run, dict) else None
 
 
 def _resolve_evaluation_project(args, project_id, spec, client_error) -> tuple[str, int | None]:
@@ -892,17 +806,16 @@ def cmd_env_eval(args) -> int:
         load_freesolo_environment,
     )
     from flash.envs.loading.pull import pull_environment_package_from_archive
-    from flash.schema import parse_adapter_revision, parse_checkpoint_ref
+    from flash.schema import parse_checkpoint_ref
 
     if args.project and not args.upload:
         return _err("--project cannot be combined with --no-upload")
 
-    revision = parse_adapter_revision(args.target)
-    parsed = parse_checkpoint_ref(args.target) if revision is None else None
-    if revision is None and parsed is None:
+    parsed = parse_checkpoint_ref(args.target)
+    if parsed is None:
         return _err(
             f"invalid evaluation target {args.target!r} "
-            "(expected a bare <run_id>, <run_id>/step-N, or full immutable adapter revision)"
+            "(expected <run_id>/final or <run_id>/step-N)"
         )
 
     # an explicitly named project is fully settled before anything else happens, including before a
@@ -925,17 +838,13 @@ def cmd_env_eval(args) -> int:
         return _err("overall: FAIL")
 
     client = client_from_config()
-    evaluation_target, exit_code = _resolve_evaluation_target(
-        args, client, parsed, parse_adapter_revision, ApiError, ClientError
+    evaluation_target, run, exit_code = _resolve_evaluation_target(
+        args, client, parsed, ApiError, ClientError
     )
     if exit_code is not None:
         return exit_code
 
-    spec, exit_code = _load_evaluation_run_spec(
-        args, client, revision, parsed, ApiError, ClientError
-    )
-    if exit_code is not None:
-        return exit_code
+    spec = _load_evaluation_run_spec(run)
 
     # graders must see what training graded, so the run's own `thinking` decides whether the
     # reasoning is stripped first (see `_scored_response`).