freesolo-flash-dev 1.2.118__tar.gz → 1.2.120__tar.gz

{freesolo_flash_dev-1.2.118 → freesolo_flash_dev-1.2.120}/.github/workflows/deploy-modal-dev.yml

@@ -86,7 +86,8 @@ jobs:
             deploy=true
           elif ! changed="$(git diff --name-only "$last_deployed_sha" "$CURRENT_SHA" 2>/dev/null)"; then
             deploy=true
-          elif grep -qE '^(flash/|pyproject\.toml$|uv\.lock$|\.github/workflows/deploy-modal-dev\.yml$)' <<<"$changed"; then
+          # the build-time vllm repair lives outside flash/, so name its exact image input path.
+          elif grep -qE '^(flash/|docker/patch_vllm_moe_lora\.py$|pyproject\.toml$|uv\.lock$|\.github/workflows/deploy-modal-dev\.yml$)' <<<"$changed"; then
             deploy=true
           fi
           echo "deploy=$deploy" >> "$GITHUB_OUTPUT"

{freesolo_flash_dev-1.2.118 → freesolo_flash_dev-1.2.120}/.github/workflows/deploy-modal.yml

@@ -118,8 +118,9 @@ jobs:
           # any `flash/` change, not just `flash/serving/`: the image ships the whole package via
           # `add_local_python_source("flash")`, so a change anywhere under flash/ can alter what runs
           # in the container. narrowing this to flash/serving/ would let the deployed app drift from
-          # the merged source. pyproject/uv.lock gate the image's dependency bounds.
-          elif grep -qE "^(flash/|pyproject\.toml$|uv\.lock$|\.github/workflows/deploy-modal\.yml$)" <<< "$changed"; then
+          # the merged source. pyproject/uv.lock gate dependency bounds, and the build-time vllm
+          # repair lives outside flash/, so its exact path must trigger a rebuild too.
+          elif grep -qE "^(flash/|docker/patch_vllm_moe_lora\.py$|pyproject\.toml$|uv\.lock$|\.github/workflows/deploy-modal\.yml$)" <<< "$changed"; then
             deploy=true
           fi
           echo "deploy=$deploy" >> "$GITHUB_OUTPUT"

{freesolo_flash_dev-1.2.118 → freesolo_flash_dev-1.2.120}/.github/workflows/publish-serving-image.yml

@@ -85,7 +85,8 @@ jobs:
       # while the tags keep pointing at the old one. that is the whole COPY set -- `flash`, the
       # lockfile, the launcher, and the files copied alongside them -- plus `.dockerignore`, which
       # decides what the build context contains at all, and this workflow file, so a change to the
-      # build here republishes rather than leaving the old image serving under the same tags.
+      # build here republishes rather than leaving the old image serving under the same tags. the
+      # build-time vllm repair lives outside flash/, so its exact path is an explicit image input.
       - name: Decide whether to publish
         id: gate
         working-directory: .
@@ -127,7 +128,7 @@ jobs:
             publish=true
           elif ! changed="$(git diff --name-only "$last_published_sha" "$CURRENT_SHA" 2>/dev/null)"; then
             publish=true
-          elif grep -qE "^(flash/|Dockerfile\.serve$|\.dockerignore$|serve_launch\.py$|pyproject\.toml$|uv\.lock$|README\.md$|LICENSE$|NOTICE$|\.github/workflows/publish-serving-image\.yml$)" <<< "$changed"; then
+          elif grep -qE "^(flash/|docker/patch_vllm_moe_lora\.py$|Dockerfile\.serve$|\.dockerignore$|serve_launch\.py$|pyproject\.toml$|uv\.lock$|README\.md$|LICENSE$|NOTICE$|\.github/workflows/publish-serving-image\.yml$)" <<< "$changed"; then
             publish=true
           fi
           echo "publish=$publish" >> "$GITHUB_OUTPUT"

{freesolo_flash_dev-1.2.118 → freesolo_flash_dev-1.2.120}/CHANGELOG.md

@@ -25,13 +25,23 @@ This file starts at 1.1.35. Earlier releases are not reconstructed here; use
 
 ### Changed
 
-- Active training and hosted serving now expose exactly `Qwen/Qwen3.5-9B`,
-  `Qwen/Qwen3.8-27B`, and `Qwen/Qwen3.6-35B-A3B`. Qwen3.8 uses the immutable BF16 revision
+- Active training now exposes exactly `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
+  `Qwen/Qwen3.6-35B-A3B`. Hosted serving remains active for 9B and 35B-A3B; hosted Qwen3.8 27B
+  remains inactive. Qwen3.8 uses the immutable BF16 revision
   `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0` and official native block-FP8 serving revision
-  `017b9c7af6b5689d5dd426a76e0bc077eb5ca20a`. The Qwen3.5 0.8B, 2B, and 4B tiers and
+  `017b9c7af6b5689d5dd426a76e0bc077eb5ca20a`, with the logical base and tokenizer provenance kept
+  distinct from the served checkpoint. The Qwen3.5 0.8B, 2B, and 4B tiers and
   `Qwen/Qwen3.6-27B` cannot activate, while historical status, accepted billing retry,
   cancellation, cleanup, hosted revocation, and exact identity-based undeploy remain available.
-  Qwen3.6 27B adapters are not compatible with Qwen3.8. Customer-owned serving remains 9B-only.
+  Qwen3.6 27B adapters are not compatible with Qwen3.8.
+- Customer-owned serving is now Modal-only and requires explicit `--provider modal`. Modal remains
+  live-qualified for all three active models. The 27B and 35B-A3B qualifications stay bound to the
+  exact certified serving image digest. Each deployment still serves one exact base model. The 27B
+  engine keeps its `H100!` placement and distinct served checkpoint and tokenizer provenance. The
+  35B-A3B engine keeps its H200, BF16, FP8 KV, 32K context, eight-sequence, 4096 batched-token, and
+  six rank-64 LoRA-slot contract. Customer-serving RunPod commands, profiles, identities, and
+  lifecycle code are removed without a compatibility or teardown shim. Managed RunPod training,
+  serverless workers, dependencies, provider routing, artifacts, and caching are unchanged.
 - `[train] init_from_adapter` now works for every source/target algorithm pair. SFT was rejected as
   a warm-start target, so an adapter could be continued only by GRPO or OPD; that restriction
   described the retired trl SFT backend, and the verl backend that replaced it loads a warm-start
@@ -73,6 +83,8 @@ This file starts at 1.1.35. Earlier releases are not reconstructed here; use
 
 ### Fixed
 
+- Qwen3.6 FP8 MoE serving now preserves unquantized activations for LoRA A while keeping the base
+  MoE path quantized, backporting the exact vLLM PR #42120 behavior into the pinned 0.23.0 image.
 - GRPO publication now validates complete finite optimizer-step actor gradient norms instead of
   masked advantage-bound spread, so valid truncated groups are not falsely rejected and deployable
   checkpoints remain withheld until terminal evidence validation succeeds.

{freesolo_flash_dev-1.2.118 → freesolo_flash_dev-1.2.120}/Dockerfile.serve

@@ -26,10 +26,14 @@ RUN python -c 'import sys; assert sys.version_info[:2] == (3, 12)' \
 
 COPY pyproject.toml uv.lock README.md LICENSE NOTICE ./
 COPY flash ./flash
+COPY docker/patch_vllm_moe_lora.py ./docker/patch_vllm_moe_lora.py
 COPY serve_launch.py ./serve_launch.py
 RUN uv sync --frozen --no-dev --extra serve-runtime
+RUN /opt/flash-venv/bin/python /app/docker/patch_vllm_moe_lora.py \
+    && /opt/flash-venv/bin/python /app/docker/patch_vllm_moe_lora.py --verify \
+    && rm /app/docker/patch_vllm_moe_lora.py
 
-# The CUDA this image runs on is decided by vllm's own compiled extension, not by us: every vllm
+# the cuda this image runs on is decided by vllm's own compiled extension, not by us: every vllm
 # wheel from 0.20.2 onward (0.23.0 included) ships a vllm/_C.abi3.so linked against
 # libcudart.so.13. Measured across the published wheels -- 0.19.1 links libcudart.so.12, 0.20.2 and
 # later link .13 -- so no pin inside the 0.2x line escapes CUDA 13, and downgrading torch to a

{freesolo_flash_dev-1.2.118 → freesolo_flash_dev-1.2.120}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: freesolo-flash-dev
-Version: 1.2.118
+Version: 1.2.120
 Summary: Flash — managed LoRA post-training (SFT/GRPO/OPD) for Freesolo environments, driven by the `flash` CLI
 Project-URL: Homepage, https://github.com/freesolo-co/flash
 Project-URL: Repository, https://github.com/freesolo-co/flash
@@ -116,10 +116,10 @@ point of use; install it with `pip install freesolo`.
 
 Two components stay Freesolo-operated and are **not** in this repository:
 
-| Component                 | Where it lives                                                                                         | Self-hosted equivalent                                                      |
-| ------------------------- | ------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- |
-| Multi-tenant identity     | `api.freesolo.co`; verifies keys and owns projects/orgs                                                | `FLASH_STANDALONE=1` runs single-tenant on your own key                     |
-| Hosted Multi-LoRA serving | `serve.freesolo.co`, deployed from `flash/serving/`; `flash/serve/` owns the client and shared runtime | `flash serve deploy` provisions serving in your own Modal or RunPod account |
+| Component                 | Where it lives                                                                                         | Self-hosted equivalent                                            |
+| ------------------------- | ------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------- |
+| Multi-tenant identity     | `api.freesolo.co`; verifies keys and owns projects/orgs                                                | `FLASH_STANDALONE=1` runs single-tenant on your own key           |
+| Hosted Multi-LoRA serving | `serve.freesolo.co`, deployed from `flash/serving/`; `flash/serve/` owns the client and shared runtime | `flash serve deploy` provisions serving in your own Modal account |
 
 So there are three ways to use Flash: against the **hosted service**, **self-hosted** against
 your own GPU accounts, or as **training and provider code to read and modify**. The training
@@ -127,9 +127,13 @@ path is self-hostable end to end — see **[SELF_HOSTING.md](SELF_HOSTING.md)**.
 
 ## Using the hosted service
 
-The active training and hosted-serving catalog is `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
-`Qwen/Qwen3.6-35B-A3B`. The retired dense Qwen3.6 27B identity is never translated into Qwen3.8,
-and customer-owned Modal or RunPod serving remains limited to Qwen3.5 9B.
+The active training catalog is `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
+`Qwen/Qwen3.6-35B-A3B`. Hosted serving is active for 9B and 35B-A3B; hosted Qwen3.8 27B remains
+inactive. The retired dense Qwen3.6 27B identity is never translated into Qwen3.8. Customer-owned
+Modal supports all three active models; 27B and 35B-A3B are bound to the certified serving image
+digest. Customer-owned `flash serve` requires explicit `--provider modal`. Customer-serving RunPod
+commands and deployment identities are unsupported, with no migration or teardown shim. Managed
+RunPod training remains supported and unchanged.
 
 `flash login` is not interactive. Export `FREESOLO_API_KEY` first so the key does not appear in process listings:
 
@@ -305,8 +309,10 @@ External bearer tokens are rejected rather than accepted unverified. A standalon
 **single-tenant** — whoever holds that key can spend your GPU budget, so keep it off
 untrusted networks. See [the security model](SELF_HOSTING.md#the-security-model).
 
-`flash serve deploy` provisions serving in your own Modal or RunPod account, implementing the
-same contract `flash/serve/` speaks; [docs/serving-contract.md](docs/serving-contract.md)
+`flash serve deploy` provisions serving in your own Modal account and requires explicit
+`--provider modal`. Each deployment owns one exact base model and its compatible adapters; another
+base model requires another deployment. Customer-serving RunPod identities are unsupported.
+It implements the same contract `flash/serve/` speaks; [docs/serving-contract.md](docs/serving-contract.md)
 documents it if you would rather write your own.
 
 The GPU worker image is public and published under an explicit CUDA tag, not `latest`:

{freesolo_flash_dev-1.2.118 → freesolo_flash_dev-1.2.120}/README.md

@@ -33,10 +33,10 @@ point of use; install it with `pip install freesolo`.
 
 Two components stay Freesolo-operated and are **not** in this repository:
 
-| Component                 | Where it lives                                                                                         | Self-hosted equivalent                                                      |
-| ------------------------- | ------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- |
-| Multi-tenant identity     | `api.freesolo.co`; verifies keys and owns projects/orgs                                                | `FLASH_STANDALONE=1` runs single-tenant on your own key                     |
-| Hosted Multi-LoRA serving | `serve.freesolo.co`, deployed from `flash/serving/`; `flash/serve/` owns the client and shared runtime | `flash serve deploy` provisions serving in your own Modal or RunPod account |
+| Component                 | Where it lives                                                                                         | Self-hosted equivalent                                            |
+| ------------------------- | ------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------- |
+| Multi-tenant identity     | `api.freesolo.co`; verifies keys and owns projects/orgs                                                | `FLASH_STANDALONE=1` runs single-tenant on your own key           |
+| Hosted Multi-LoRA serving | `serve.freesolo.co`, deployed from `flash/serving/`; `flash/serve/` owns the client and shared runtime | `flash serve deploy` provisions serving in your own Modal account |
 
 So there are three ways to use Flash: against the **hosted service**, **self-hosted** against
 your own GPU accounts, or as **training and provider code to read and modify**. The training
@@ -44,9 +44,13 @@ path is self-hostable end to end — see **[SELF_HOSTING.md](SELF_HOSTING.md)**.
 
 ## Using the hosted service
 
-The active training and hosted-serving catalog is `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
-`Qwen/Qwen3.6-35B-A3B`. The retired dense Qwen3.6 27B identity is never translated into Qwen3.8,
-and customer-owned Modal or RunPod serving remains limited to Qwen3.5 9B.
+The active training catalog is `Qwen/Qwen3.5-9B`, `Qwen/Qwen3.8-27B`, and
+`Qwen/Qwen3.6-35B-A3B`. Hosted serving is active for 9B and 35B-A3B; hosted Qwen3.8 27B remains
+inactive. The retired dense Qwen3.6 27B identity is never translated into Qwen3.8. Customer-owned
+Modal supports all three active models; 27B and 35B-A3B are bound to the certified serving image
+digest. Customer-owned `flash serve` requires explicit `--provider modal`. Customer-serving RunPod
+commands and deployment identities are unsupported, with no migration or teardown shim. Managed
+RunPod training remains supported and unchanged.
 
 `flash login` is not interactive. Export `FREESOLO_API_KEY` first so the key does not appear in process listings:
 
@@ -222,8 +226,10 @@ External bearer tokens are rejected rather than accepted unverified. A standalon
 **single-tenant** — whoever holds that key can spend your GPU budget, so keep it off
 untrusted networks. See [the security model](SELF_HOSTING.md#the-security-model).
 
-`flash serve deploy` provisions serving in your own Modal or RunPod account, implementing the
-same contract `flash/serve/` speaks; [docs/serving-contract.md](docs/serving-contract.md)
+`flash serve deploy` provisions serving in your own Modal account and requires explicit
+`--provider modal`. Each deployment owns one exact base model and its compatible adapters; another
+base model requires another deployment. Customer-serving RunPod identities are unsupported.
+It implements the same contract `flash/serve/` speaks; [docs/serving-contract.md](docs/serving-contract.md)
 documents it if you would rather write your own.
 
 The GPU worker image is public and published under an explicit CUDA tag, not `latest`:

{freesolo_flash_dev-1.2.118 → freesolo_flash_dev-1.2.120}/SELF_HOSTING.md

@@ -22,9 +22,11 @@ Two credentials and at least one GPU account:
 
 Everything else is optional.
 
-The active training catalog is Qwen3.5 9B, Qwen3.8 27B, and Qwen3.6 35B-A3B. Customer-owned
-serving through `flash serve deploy` remains Qwen3.5 9B-only; the hosted 27B profile is not a
-customer-provider placement contract.
+The active training catalog is Qwen3.5 9B, Qwen3.8 27B, and Qwen3.6 35B-A3B. Hosted Qwen3.8
+27B remains inactive. Customer-owned Modal supports all three models; 27B and 35B-A3B are bound to
+the certified serving image digest. Customer-owned `flash serve` requires explicit
+`--provider modal`. Customer-serving RunPod commands and identities are unsupported, with no
+migration or teardown shim. Managed RunPod training remains supported and unchanged.
 
 ## Quickstart
 
@@ -363,10 +365,23 @@ without `FLASH_STANDALONE` and set `FREESOLO_BASE_URL` to point at it.
 
 ## Serving
 
-`flash serve deploy` provisions serving in **your own** Modal or RunPod account, running the
-published worker image against one base model and one run's adapter. Training and export remain
-independent of serving. Catalog serving checkpoint repositories are informational only and are never
-resolved by the training path.
+`flash serve deploy` provisions serving in **your own** Modal account, running the published worker
+image against one exact base model and that model's compatible adapters. A separate deployment is
+required for each base model. Training and export remain independent of serving. The required
+provider argument has the sole accepted value `--provider modal`.
+Catalog serving checkpoint repositories are informational only and are never resolved by the
+training path.
+
+Customer-owned Modal is live-qualified for Qwen3.5 9B, Qwen3.8 27B, and Qwen3.6 35B-A3B. The 27B
+and 35B-A3B qualifications are bound to the certified serving image digest. The 27B engine runs on
+`H100!` and serves the pinned `Qwen/Qwen3.8-27B-FP8` checkpoint while retaining
+`Qwen/Qwen3.8-27B` as its logical base and tokenizer provenance. The 35B-A3B engine serves the BF16
+model on one H200 with FP8 KV cache, a 32K context, eight sequences, a 4096 batched-token cap, and
+six rank-64 LoRA slots.
+Customer-serving RunPod lifecycle commands and deployment identities are unsupported. There is no
+migration, status, undeploy, or teardown shim for an old RunPod identity. This does not affect the
+managed RunPod training provider, serverless workers, artifact caching, or `RUNPOD_API_KEY` used by
+the training control plane.
 
 ```bash
 # the `server` extra, not the bare install: `serve deploy` resolves the adapter through
@@ -392,31 +407,9 @@ flash serve deploy \
   --modal-region us-east
 ```
 
-The three `--modal-*` placement flags are required for `--provider modal` even though `--help`
-lists them as optional: they are optional to _argparse_ because RunPod takes its own pair instead,
-and `placement_for` is what requires exactly one provider's set. Omitting them exits with
-`modal placement requires environment, region, workspace_name`.
-
-For RunPod, export `RUNPOD_API_KEY` instead and swap the placement flags rather than adding to
-them -- `placement_for` rejects the other provider's inputs instead of ignoring them, so keeping
-`--modal-*` alongside `--provider runpod` fails before anything is created:
-
-```bash
-flash serve deploy \
-  --provider runpod \
-  --model Qwen/Qwen3.5-9B \
-  --run <run-id> \
-  --deployment-id my-9b-serving \
-  --image ghcr.io/freesolo-co/freesolo-flash-serve@sha256:<digest> \
-  --artifact-repo <hub-repo> \
-  --artifact-subfolder <path-within-repo> \
-  --lora-rank 32 \
-  --runpod-account <your-account-id> \
-  --runpod-data-center <data-center-id>
-```
-
-Both `--runpod-*` flags are required and must be nonempty, for the same reason the `--modal-*`
-trio is: `--help` lists them as optional only because each provider takes its own set.
+The three `--modal-*` placement flags are required for `--provider modal`. Omitting them exits with
+`modal placement requires environment, region, workspace_name`. Passing `--provider runpod` is
+rejected by argument parsing before Hub resolution, credential reads, or provider access.
 
 Provider credentials are read from the process environment for the duration of a single call and are
 never written to the deployment record, logs, or command arguments — so any later resize or teardown