soup-cli 0.71.20__tar.gz → 0.71.22__tar.gz

{soup_cli-0.71.20 → soup_cli-0.71.22}/CHANGELOG.md

@@ -12,6 +12,123 @@ reproducing 70+ versions of notes.
 
 ## [Unreleased]
 
+## [0.71.22] - 2026-06-10
+
+### Added
+- **Perf & measure polish** — a 4-issue patch tightening four live paths from
+  the recent BETA lifts. Pure code, validated on Windows + RTX 3050.
+  - **MiniLLM on-policy KV-cache (closes #263).** The on-policy distillation
+    rollout (`soup train` with `training.minillm_on_policy: true`) now threads
+    `past_key_values` so each step forwards only the new token instead of
+    re-feeding the whole prefix — resolving the O(L²) per-step cost from
+    v0.71.18. A LoRA student (the common distill case) activates the cache
+    too: the new `_supports_kv_cache` probe unwraps the PEFT model via
+    `get_base_model()` before deciding. The teacher is always cached; the
+    student cache respects the retained autograd graph and degrades gracefully
+    if a model returns no cache mid-loop.
+  - **`soup serve --mole` KV-cache (closes #262).** Each of the N task adapters
+    in a served MoLE now keeps its own KV cache in lockstep, created fresh per
+    `generate()` call (never stored on the instance, so there is no
+    cross-request leak). Top-k zero-weight adapters are still skipped, and the
+    output is byte-identical to the no-cache path on a real MoLE.
+  - **Deploy-autopilot live measure factories (closes #143).** `soup deploy
+    autopilot --measure` ships a first-party transformers loader factory (lazy
+    import, per-candidate quant config via the Quant Menu loader; `before` =
+    base, `after` = quantised) replacing the inject-only test hooks. The
+    baseline is now scored **once** and the whole candidate list is
+    **pre-validated up front**, so a typo in `--measure-candidates` raises
+    before any model load instead of burning N live loads or doubling peak
+    VRAM.
+  - **Live-codec TTS via SNAC, partial (#265-partial).** The live-codec
+    encode path (`data.format='audio'`) is validated for **Orpheus**:
+    `load_audio_mono` now probes `soundfile.info` (duration + byte cap)
+    *before* `soundfile.read` (no multi-GB decode into RAM) and reads through
+    an `O_NOFOLLOW` file descriptor; a real SNAC-backed encode of a 24 kHz wav
+    produced 42 Orpheus codec tokens.
+
+### Fixed
+- MiniLLM on-policy KV-cache was silently disabled for LoRA students (the
+  PEFT wrapper hid the base model's `past_key_values` support) — now probed
+  via `get_base_model()`.
+- Deploy-measure no longer re-scores the baseline once per candidate or burns
+  live model loads on a bad candidate (per-candidate validation moved up front).
+- `load_audio_mono` capped audio duration only *after* decoding into RAM —
+  the cap is now checked from `soundfile.info` before reading.
+
+### Known limitations
+- KV-cache correctness is validated (cache == no-cache equality on real tiny
+  artifacts) but large-model throughput gains were not measured on the 4 GB
+  dev box.
+- **#265 stays open** — the live-codec `data.format='audio'` SNAC encode path
+  is validated for Orpheus only; the other four TTS families keep their
+  per-family codec dependency gate.
+- The deploy-measure first-party factory's real quantized (bitsandbytes 4-bit)
+  load is CUDA + bitsandbytes-gated; on Windows / no-bnb the injected test
+  seams are the validated path.
+- The MoLE serve KV-cache assumes single-sequence (`B == 1`) decode.
+
+## [0.71.21] - 2026-06-10
+
+### Added
+- **Precision & rollout lift (BETA, hw-gated)** — lifts five deferred
+  `NotImplementedError` stubs to live code.
+  - **FP8 attention + NVFP4 (closes #141).** `training.fp8_attention: true`
+    now converts the model's attention projections (q/k/v/o + fused qkv
+    variants) to FP8 training modules via torchao's
+    `convert_to_float8_training` with an attention-only `module_filter_fn`
+    (Hopper SM ≥ 9.0 gate); `training.nvfp4: true` quantises via torchao's
+    `NVFP4Config` (Blackwell SM ≥ 10.0 gate). Both are wired into the v0.28
+    speed/memory pipeline and degrade to a visible yellow advisory when the
+    gate fires — a conversion failing partway raises an honest "model may be
+    PARTIALLY converted" error rather than silently training on a
+    half-converted model.
+  - **vLLM sleep mode (closes #124).** `training.vllm_sleep_mode: true` is
+    live: `create_vllm_engine(sleep_mode=True)` sets
+    `AsyncEngineArgs.enable_sleep_mode` (vLLM ≥ 0.7 gate with a friendly
+    upgrade message), the new `vllm_sleep_cycle(engine, level=1|2)` context
+    manager wraps the optimisation step (wake in `finally`), and the GRPO
+    trainer threads the flag into TRL's `GRPOConfig` when the installed TRL
+    exposes the hook (advisory otherwise).
+  - **Multi-turn agent rollout launchers (closes #125).** `soup train` with
+    `task: grpo` + `training.rollout_backend: openenv` +
+    `training.rollout_func: my_module:fn` now runs a LIVE rollout: the
+    resolver imports the operator's callable (same trusted-code policy as
+    `data.prompt_strategy`), feeds it the dataset prompts as seeds, and the
+    returned `{prompt, answer?}` rows replace the prompt dataset. Rows are
+    normalised (extra keys stripped, message-list prompts deep-copied,
+    non-string answers rejected loudly). `art` / `ruler` / `nemo_gym` raise a
+    friendly ImportError when the backend package is missing and an honest
+    BETA gate when present (injectable `_EXTERNAL_ROLLOUT_RUNNERS` seam).
+    Validated by a real GRPO + openenv rollout train on SmolLM2-135M.
+  - **Apple-adapter conversion (closes #228).** `soup apple-adapter` is live
+    for `hf-to-mlx` / `mlx-to-hf`: PEFT LoRA safetensors ↔ mlx-lm adapters
+    with both matrices transposed (`lora_A [r,in]` ↔ `lora_a [in,r]`),
+    bf16 sources upcast via the torch loader, `adapters.safetensors` +
+    `num_layers` emitted for mlx-lm's `load_adapters`, rank/alpha/dropout
+    carried through, legacy `adapters.npz` still read, optional v0.60
+    Merkle-root signing. The `*-to-apple` directions stay upstream-gated
+    (no published FoundationModels adapter spec). Validated by a real bf16
+    PEFT adapter round-tripping with numeric equality.
+  - **Llama-4 expert delinearization (closes #97).** `soup
+    delinearize-llama4` now runs a live torch runtime: fused 2-D expert
+    tensors `[E*dim_in, dim_out]` reshape to 3-D `[E, dim_in, dim_out]`
+    (expert count from `config.json` or `--num-experts`), other tensors pass
+    through, JSON sidecars are copied, writes are atomic. `--plan-only`
+    keeps the old render-and-exit flow.
+
+### Fixed
+- `safetensors.numpy.save` silently mangles non-contiguous (transposed)
+  arrays — the apple-adapter writer now makes every array C-contiguous
+  first (caught by the new round-trip assertions).
+
+### Known limitations
+- fp8_attention / nvfp4 / vllm_sleep_mode are BETA hardware-gated — the
+  converters and gates ship validated via capability probes and fake-module
+  dispatch tests, but end-to-end runs need a Hopper/Blackwell GPU + torchao
+  (or vLLM ≥ 0.7), none of which exist on the maintainer's RTX 3050 /
+  Windows box. The `art` / `ruler` / `nemo_gym` rollout adapters are
+  honestly BETA-gated until validated against the upstream packages.
+
 ## [0.71.20] - 2026-06-09
 
 ### Added

{soup_cli-0.71.20 → soup_cli-0.71.22}/CONTRIBUTING.md

@@ -120,7 +120,7 @@ src/soup_cli/
   templates/          - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0)
   ui/                 - Web UI (FastAPI + HTML/JS SPA)
 
-tests/                - Test suite (291 files, 13874 tests)
+tests/                - Test suite (293 files, 14184 tests)
 examples/             - Real-world config examples and datasets
 ```
 
@@ -150,7 +150,7 @@ pytest tests/test_data.py::test_detect_alpaca_format -v
 pytest tests/ --cov=soup_cli --cov-report=html
 ```
 
-### Test Files (287 files)
+### Test Files (293 files)
 
 > A representative sample of the suite below. The full table lives in
 > [`.claude/CLAUDE.md`](.claude/CLAUDE.md); run `pytest tests/ -v` for the complete list.

{soup_cli-0.71.20 → soup_cli-0.71.22}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: soup-cli
-Version: 0.71.20
+Version: 0.71.22
 Summary: Fine-tune and post-train LLMs in one command. No SSH, no config hell.
 Project-URL: Homepage, https://github.com/MakazhanAlpamys/Soup
 Project-URL: Repository, https://github.com/MakazhanAlpamys/Soup
@@ -189,20 +189,22 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.20 — Modality II trainers (BETA, hardware-gated).** Three v0.52.0 schema stubs become real
-trainers:
-
-- **TTS fine-tuning** — `task: tts` now trains a real text-to-speech model. The five families
-  (Orpheus / Sesame-CSM / Llasa / Spark / Oute) are decoder LMs, so a TTS fine-tune is next-token
-  cross-entropy over `[text][audio-codec-token]` chat sequences. Pre-encode your audio to codec tokens
-  offline and train with `data.format: chat` — Soup adds per-family emotion control + codec
-  special-token registration. (The encode-at-train-time path is dependency-gated on each family's codec.)
-- **BitNet 1.58-bit** — `soup export --format bitnet | tq1_0` runs a real llama.cpp TQ1_0 ternary
-  export, and `quantization: bitnet_1.58` routes to a live BitNet trainer (gated on `onebitllms`).
-- **MoE expert quant** — `training.moe_expert_quant: nf4 | int8_rowwise` quantizes just the
-  fused-MoE expert layers with bitsandbytes (attention + router stay full precision), and
-  `training.train_router_only: true` freezes the experts to train only the gating router. Validated
-  live on an RTX 3050.
+**v0.71.22 — Perf & measure polish.** Four live paths from the recent BETA lifts get tighter:
+
+- **MiniLLM on-policy distillation is fast** — the on-policy rollout
+  (`training.minillm_on_policy: true`) now threads a KV cache so each step forwards only the new
+  token instead of re-feeding the whole prefix, resolving the earlier O(L²) cost. A LoRA student
+  activates the cache too (the PEFT wrapper is unwrapped before the cache check).
+- **`soup serve --mole` uses a KV cache** — each task adapter in a served MoLE keeps its own cache
+  in lockstep, created fresh per request (no cross-request leak). Output is byte-identical to the
+  previous no-cache path.
+- **`soup deploy autopilot --measure` is real and frugal** — a first-party transformers loader
+  factory loads each candidate live (per-candidate quant config), scores the baseline **once**, and
+  pre-validates the whole `--measure-candidates` list up front, so a typo fails before any model
+  load instead of burning N loads or doubling peak VRAM.
+- **Live-codec TTS via SNAC (Orpheus)** — encoding raw audio at train time (`data.format: audio`)
+  is validated for Orpheus: audio is duration- and byte-capped from `soundfile.info` **before**
+  decoding into RAM, and read through a symlink-safe file descriptor.
 
 Full history: [CHANGELOG.md](CHANGELOG.md) · [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
 

{soup_cli-0.71.20 → soup_cli-0.71.22}/README.md

@@ -49,20 +49,22 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.20 — Modality II trainers (BETA, hardware-gated).** Three v0.52.0 schema stubs become real
-trainers:
-
-- **TTS fine-tuning** — `task: tts` now trains a real text-to-speech model. The five families
-  (Orpheus / Sesame-CSM / Llasa / Spark / Oute) are decoder LMs, so a TTS fine-tune is next-token
-  cross-entropy over `[text][audio-codec-token]` chat sequences. Pre-encode your audio to codec tokens
-  offline and train with `data.format: chat` — Soup adds per-family emotion control + codec
-  special-token registration. (The encode-at-train-time path is dependency-gated on each family's codec.)
-- **BitNet 1.58-bit** — `soup export --format bitnet | tq1_0` runs a real llama.cpp TQ1_0 ternary
-  export, and `quantization: bitnet_1.58` routes to a live BitNet trainer (gated on `onebitllms`).
-- **MoE expert quant** — `training.moe_expert_quant: nf4 | int8_rowwise` quantizes just the
-  fused-MoE expert layers with bitsandbytes (attention + router stay full precision), and
-  `training.train_router_only: true` freezes the experts to train only the gating router. Validated
-  live on an RTX 3050.
+**v0.71.22 — Perf & measure polish.** Four live paths from the recent BETA lifts get tighter:
+
+- **MiniLLM on-policy distillation is fast** — the on-policy rollout
+  (`training.minillm_on_policy: true`) now threads a KV cache so each step forwards only the new
+  token instead of re-feeding the whole prefix, resolving the earlier O(L²) cost. A LoRA student
+  activates the cache too (the PEFT wrapper is unwrapped before the cache check).
+- **`soup serve --mole` uses a KV cache** — each task adapter in a served MoLE keeps its own cache
+  in lockstep, created fresh per request (no cross-request leak). Output is byte-identical to the
+  previous no-cache path.
+- **`soup deploy autopilot --measure` is real and frugal** — a first-party transformers loader
+  factory loads each candidate live (per-candidate quant config), scores the baseline **once**, and
+  pre-validates the whole `--measure-candidates` list up front, so a typo fails before any model
+  load instead of burning N loads or doubling peak VRAM.
+- **Live-codec TTS via SNAC (Orpheus)** — encoding raw audio at train time (`data.format: audio`)
+  is validated for Orpheus: audio is duration- and byte-capped from `soundfile.info` **before**
+  decoding into RAM, and read through a symlink-safe file descriptor.
 
 Full history: [CHANGELOG.md](CHANGELOG.md) · [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
 

{soup_cli-0.71.20 → soup_cli-0.71.22}/docs/backends-and-ops.md

@@ -641,10 +641,10 @@ Closed catalog (`MappingProxyType`) of ready-to-edit YAML / JSON. Output path cw
 ## Llama 4 Delinearizer
 
 ```bash
-soup delinearize-llama4 ./llama4-checkpoint --target ./out-delinearized --yes
+soup delinearize-llama4 ./llama4-checkpoint --target ./out-delinearized [--num-experts N] [--plan-only]
 ```
 
-Plans Llama 4 expert-weight reshape for export. v0.44.0 ships the planner; live runtime in v0.44.1. `is_llama4_model` uses a word-boundary regex matching the `is_gemma4_model` pattern — `ungemma-llama-4ish` is rejected.
+LIVE (v0.71.21): reshapes fused Llama-4 expert weights `[E*din, dout]` → `[E, din, dout]` shard-by-shard (atomic writes, per-shard 16 GiB cap, cwd containment) and copies the JSON sidecars so the target stays loadable. The expert count defaults from `config.json` (`text_config.num_local_experts`); pass `--num-experts` when the config doesn't carry it (exit 2 otherwise). `--plan-only` keeps the original preview flow and writes nothing. `is_llama4_model` uses a word-boundary regex matching the `is_gemma4_model` pattern — `ungemma-llama-4ish` is rejected.
 
 
 ## Ctrl+C Graceful Save

{soup_cli-0.71.20 → soup_cli-0.71.22}/docs/commands.md

@@ -24,6 +24,7 @@ soup push --model ./output --repo user/name   Upload to HuggingFace
 soup push --model ./output --repo user/name --collection user/coll-abc123  Add to HF Collection
 soup merge --adapter ./output                 Merge LoRA with base model
 soup merge-sharded-fsdp-weights ./shards -o merged.safetensors  Consolidate FSDP shards into one safetensors (v0.71.14; --plan-only previews)
+soup delinearize-llama4 ./src --target ./out [--num-experts N] [--plan-only]  Live Llama-4 fused-expert reshape [E*din,dout] -> [E,din,dout] + sidecar copy (v0.71.21)
 soup export --model ./output --format gguf    Export to GGUF (Ollama)
 soup export --model ./output --deploy ollama  Export GGUF + auto-deploy to Ollama
 soup export --model ./output --format onnx    Export to ONNX
@@ -225,7 +226,7 @@ soup lock show / soup lock check              Show + drift-check (exit 3 on drif
 soup compile <program.py> --eval <suite> [--optimizer mipro|gepa|textgrad|copro|bootstrap_fewshot] [--plan-only]  DSPy / GEPA / TextGrad prompt-program compiler — live (v0.71.13; pip install 'soup-cli[compile]')
 soup distill-prompt --traces <jsonl> --teacher <m> --student <m> --strategy sft|preference|kl [--provider ollama|anthropic|vllm] [--base-url <url>] [--temperature F] [--max-rows N]  Distill prompt-heavy traces via a live teacher (v0.71.13)
 soup compile-tools <spec.json|yaml> --eval <jsonl> [--optimizer textgrad|gepa] [--plan-only]  TextGrad / GEPA tool-schema optimiser — live (v0.71.13; pip install 'soup-cli[compile]')
-soup apple-adapter <source-dir> --direction hf-to-mlx|mlx-to-hf|hf-to-apple|mlx-to-apple --output <dir> [--sign]  HF / MLX / Apple FoundationModels adapter conversion (v0.68.0)
+soup apple-adapter <source-dir> --direction hf-to-mlx|mlx-to-hf|hf-to-apple|mlx-to-apple --output <dir> [--sign] [--plan-only]  PEFT LoRA <-> mlx-lm adapter conversion — live (v0.71.21; *-to-apple upstream-gated exit 3)
 soup local-rl init --db <path>                Create personal-LLM flywheel SQLite schema (v0.68.0)
 soup local-rl status --db <path>              Print interactions / thumbs-up / thumbs-down counters
 soup local-rl record --db <path> --prompt <q> --response <r> --thumb up|down  Append thumbs record

{soup_cli-0.71.20 → soup_cli-0.71.22}/docs/performance-and-quantization.md

@@ -468,12 +468,13 @@ soup serve --model ./output --kv-cache-type q8_0     # 8-bit quantized KV cache
 - vLLM / SGLang serve wiring is still tracked under [#140](https://github.com/MakazhanAlpamys/Soup/issues/140) (`infra-blocked`).
 
 
-## FP8 Attention + NVFP4 + Native `unsloth_bnb_4bit` (v0.53.0)
+## FP8 Attention + NVFP4 + Native `unsloth_bnb_4bit`
 
-Three new TrainingConfig bools extend the v0.28.0 FP8 menu:
+Three TrainingConfig bools extend the v0.28.0 FP8 menu. `fp8_attention` and `nvfp4` are LIVE
+torchao converters as of v0.71.21 (hardware-gated):
 
-- `fp8_attention: true` — requires `quantization_aware: fp8` AND a non-MLX backend. Targets axolotl parity for FP8 attention on Hopper+ GPUs.
-- `nvfp4: true` — Blackwell-only FP4 training. Gated to non-MLX + `modality: text`; the SM ≥ 12.0 runtime check fires at trainer construction.
+- `fp8_attention: true` — requires `quantization_aware: fp8` AND a non-MLX backend. Converts the attention projections (q/k/v/o and fused variants) to torchao float8 training on Hopper+ GPUs. Missing torchao or a pre-Hopper GPU degrades to a clear advisory; a conversion-phase failure raises an honest "model may be PARTIALLY converted" error instead of training on a half-converted model.
+- `nvfp4: true` — Blackwell-only FP4 training via torchao `NVFP4Config` + `quantize_`. Gated to non-MLX + `modality: text`; the SM ≥ 10 runtime check fires at trainer construction.
 - `unsloth_bnb_4bit: true` — promotes "Unsloth Dynamic 4-bit" from an implicit `backend=unsloth + quantization=4bit` combo to a named flag. Mutual rejection of inconsistent combos at config load.
 
 Cross-validator ordering picks the most actionable error: `quantization_aware='fp8'` prerequisite fires before the MLX rejection on `fp8_attention`, so a YAML missing both surfaces the deeper issue first.

{soup_cli-0.71.20 → soup_cli-0.71.22}/docs/training.md

@@ -484,9 +484,10 @@ training:
   grpo_fp16: true                     # FP16 RL (unsloth parity)
   # Long-context + memory-efficient RL
   long_context_grpo: true             # wires Tiled MLP when available
-  vllm_sleep_mode: true               # between-rollouts vLLM standby
-  # Multi-turn agent rollout
-  rollout_backend: art                # one of: art / ruler / nemo_gym / openenv
+  vllm_sleep_mode: true               # between-rollouts vLLM standby — LIVE (vLLM >= 0.7)
+  # Multi-turn agent rollout — openenv is LIVE: your function's rows replace the prompt dataset
+  rollout_backend: openenv            # one of: art / ruler / nemo_gym / openenv
+  rollout_func: my_module:my_rollout  # module:function resolver (openenv; trusted operator code)
   # Stability / efficiency knobs
   ref_model_ema_alpha: 0.99           # EMA sync policy → reference
   replay_buffer_size: 2048

{soup_cli-0.71.20 → soup_cli-0.71.22}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "soup-cli"
-version = "0.71.20"
+version = "0.71.22"
 description = "Fine-tune and post-train LLMs in one command. No SSH, no config hell."
 readme = "README.md"
 license = "Apache-2.0"
@@ -22,7 +22,7 @@ classifiers = [
     "Topic :: Scientific/Engineering :: Artificial Intelligence",
 ]
 # Core install (`pip install soup-cli`) is intentionally light: the CLI, config
-# system, and data tools — no PyTorch. v0.71.0 split the heavy training stack
+# system, and data tools — no PyTorch. v0.71.0 split the heavy training stack
 # (torch / transformers / peft / trl / datasets / bitsandbytes / accelerate)
 # into the `[train]` extra below.
 dependencies = [
@@ -35,7 +35,7 @@ dependencies = [
 ]
 
 [project.optional-dependencies]
-# v0.71.0 — heavy training stack. `pip install 'soup-cli[train]'` to fine-tune.
+# v0.71.0 — heavy training stack. `pip install 'soup-cli[train]'` to fine-tune.
 # These were core dependencies through v0.70.0; pins are unchanged.
 train = [
     "torch>=2.0.0",
@@ -46,7 +46,7 @@ train = [
     "bitsandbytes>=0.41.0",
     "accelerate>=0.25.0",
 ]
-# v0.71.0 — convenience meta-extra pulling the main optional stacks.
+# v0.71.0 — convenience meta-extra pulling the main optional stacks.
 all = ["soup-cli[train,serve,ui,data]"]
 eval = ["lm-eval>=0.4.0"]
 data = ["datasketch>=1.6.0"]
@@ -74,31 +74,31 @@ sglang = ["sglang>=0.2.0", "fastapi>=0.104.0", "uvicorn>=0.24.0"]
 mlx = ["mlx>=0.20.0", "mlx-lm>=0.20.0"]
 cce = ["cut-cross-entropy>=24.10.0"]
 tui = ["textual>=0.50.0"]
-# v0.53.8 #89 — bundle MLflow / SwanLab / Trackio for `--tracker` users.
+# v0.53.8 #89 — bundle MLflow / SwanLab / Trackio for `--tracker` users.
 trackers = ["mlflow>=2.0.0", "swanlab>=0.3.0", "trackio>=0.0.1"]
-# v0.53.8 #85 — fsspec backends for remote dataset loading (s3 / gs / az / oci).
+# v0.53.8 #85 — fsspec backends for remote dataset loading (s3 / gs / az / oci).
 remote = ["fsspec>=2024.1.0", "s3fs>=2024.1.0", "gcsfs>=2024.1.0", "adlfs>=2024.1.0"]
-# v0.53.10 #150 — bundle scikit-optimize so `soup data mix --optimize` runs the
+# v0.53.10 #150 — bundle scikit-optimize so `soup data mix --optimize` runs the
 # Bayesian-style loop instead of falling back to the v0.48.0 Dirichlet sampler.
 mix = ["scikit-optimize>=0.9.0"]
-# v0.53.10 #113 — production-grade data quality: langdetect (language) +
+# v0.53.10 #113 — production-grade data quality: langdetect (language) +
 # presidio-analyzer (PII). Llama-Guard-3-1B is documented as a manual recipe
 # (license + ~600 MB weight blob too large to bundle by default).
 data-pro = ["langdetect>=1.0.9", "presidio-analyzer>=2.2.0"]
-# v0.71.2 #179/#185 — ed25519 detached signing for `soup attest` / `soup
+# v0.71.2 #179/#185 — ed25519 detached signing for `soup attest` / `soup
 # adapters sign`. Pure-offline; Sigstore keyless (OIDC + Fulcio/Rekor network)
 # stays infra-blocked and is NOT bundled here.
 sign = ["cryptography>=41.0.0"]
-# v0.71.3 #181 — reportlab PDF rendering for `soup train --annex-xi *.pdf`.
+# v0.71.3 #181 — reportlab PDF rendering for `soup train --annex-xi *.pdf`.
 pdf = ["reportlab>=4.0.0"]
-# v0.71.3 #180 — codecarbon offline energy/CO2 measurement for
+# v0.71.3 #180 — codecarbon offline energy/CO2 measurement for
 # `soup train --track-energy` (offline; no IP-geolocation network call).
 carbon = ["codecarbon>=2.0.0"]
-# v0.71.13 #225/#227 — prompt-program / tool-schema compilers
+# v0.71.13 #225/#227 — prompt-program / tool-schema compilers
 # (`soup compile` / `soup compile-tools`). Lazy-imported with a friendly
 # ImportError; not installed by default (heavy dependency trees).
 compile = ["dspy-ai>=2.5.0", "textgrad>=0.1.0", "gepa>=0.0.1"]
-# v0.71.18 #16 — serverless cloud GPU training (`soup train --cloud modal`).
+# v0.71.18 #16 — serverless cloud GPU training (`soup train --cloud modal`).
 # Lazy-imported; only needed for `--cloud-submit` (plan-only render needs no
 # dependency). Modal auth is via `modal setup`.
 modal = ["modal>=0.60.0"]
@@ -113,7 +113,7 @@ Issues = "https://github.com/MakazhanAlpamys/Soup/issues"
 
 [tool.hatch.build.targets.wheel]
 packages = ["src/soup_cli"]
-# v0.53.8 #93 — include bundled fixture JSONLs as package data so
+# v0.53.8 #93 — include bundled fixture JSONLs as package data so
 # `soup data demo` works in zipapp / namespace-package installs.
 # Hatchling's ``packages = ["src/soup_cli"]`` already recurses into the
 # package directory, so we use the artifacts directive (NOT
@@ -146,7 +146,7 @@ follow_imports = "silent"
 testpaths = ["tests"]
 markers = [
     "smoke: slow smoke tests that download models and run training (run with: pytest -m smoke)",
-    "unit: fast isolated tests — no subprocess, network, filesystem, or real model load",
+    "unit: fast isolated tests — no subprocess, network, filesystem, or real model load",
     "integration: tests that touch real subprocess, SQLite, filesystem, or HTTP",
 ]
 addopts = "-m 'not smoke' --cov=soup_cli --cov-fail-under=77 --cov-report=term-missing:skip-covered"

soup_cli-0.71.22/src/soup_cli/__init__.py

@@ -0,0 +1,3 @@
+"""Soup CLI — Fine-tune and post-train LLMs in one command."""
+
+__version__ = "0.71.22"

{soup_cli-0.71.20 → soup_cli-0.71.22}/src/soup_cli/commands/apple_adapter.py

@@ -61,13 +61,37 @@ def apple_adapter_cmd(
     if plan_only:
         return
 
+    # v0.71.21 #228 — live conversion. Exit codes: 2 = validation /
+    # missing-input, 3 = upstream-gated (Apple spec not yet public),
+    # 1 = missing dependency, 0 = success.
     try:
-        convert_apple_adapter(plan)
-    except NotImplementedError as exc:
+        report = convert_apple_adapter(plan)
+    except RuntimeError as exc:
         console.print(
             Panel(
                 f"[yellow]{escape(str(exc))}[/]",
-                title="Live apple-adapter deferred",
+                title="apple-adapter — upstream-gated",
             )
         )
         raise typer.Exit(3) from exc
+    except (ValueError, FileNotFoundError) as exc:
+        console.print(f"[red]{escape(str(exc))}[/]")
+        raise typer.Exit(2) from exc
+    except ImportError as exc:
+        console.print(f"[red]{escape(str(exc))}[/]")
+        raise typer.Exit(1) from exc
+
+    skipped_note = (
+        f"\nSkipped (non-LoRA): [bold]{len(report.skipped_keys)}[/]"
+        if report.skipped_keys
+        else ""
+    )
+    console.print(
+        Panel(
+            f"Converted LoRA keys: [bold]{report.converted_keys}[/]\n"
+            f"Output:              [bold]{escape(report.output_dir)}[/]\n"
+            f"Signed:              [bold]{report.signed}[/]{skipped_note}",
+            title="soup apple-adapter — converted",
+            border_style="green",
+        )
+    )

soup_cli-0.71.22/src/soup_cli/commands/delinearize_llama4.py

@@ -0,0 +1,105 @@
+"""`soup delinearize-llama4` — Llama 4 expert-weight reshape.
+
+v0.44.0 shipped the planner; v0.71.21 (#97) lifts the live torch runtime
+(load each shard → reshape fused 2-D expert weights to 3-D → atomic write
+to ``--target``). ``--plan-only`` keeps the old render-and-exit flow.
+"""
+
+from __future__ import annotations
+
+from typing import Optional
+
+import typer
+from rich.console import Console
+from rich.markup import escape
+from rich.panel import Panel
+
+from soup_cli.utils.delinearize_llama4 import (
+    is_llama4_model,
+    plan_delinearize,
+    run_delinearize,
+)
+
+console = Console()
+
+
+def delinearize_llama4(
+    source_dir: str = typer.Argument(
+        ...,
+        help="Llama 4 checkpoint directory (containing *.safetensors).",
+    ),
+    target_dir: str = typer.Option(
+        ...,
+        "--target",
+        "-o",
+        help="Destination directory for the delinearized weights (under cwd).",
+    ),
+    model_id: str = typer.Option(
+        None,
+        "--model-id",
+        help="Optional model id; warn if it doesn't look like a Llama 4 model.",
+    ),
+    num_experts: Optional[int] = typer.Option(
+        None,
+        "--num-experts",
+        help=(
+            "Expert count for the reshape. Defaults to "
+            "config.json's num_local_experts."
+        ),
+    ),
+    plan_only: bool = typer.Option(
+        False,
+        "--plan-only",
+        help="Render the plan and exit without reshaping (v0.44.0 flow).",
+    ),
+) -> None:
+    """Delinearize Llama 4 fused expert weights to 3-D form.
+
+    Live since v0.71.21 (#97): reshapes ``...experts.gate_up_proj`` /
+    ``...experts.down_proj`` from ``[E*dim_in, dim_out]`` to
+    ``[E, dim_in, dim_out]`` and copies JSON sidecars so the target stays
+    a loadable checkpoint.
+    """
+    if model_id is not None and not is_llama4_model(model_id):
+        console.print(
+            f"[yellow]model id {escape(model_id)} doesn't match the Llama 4 "
+            "naming pattern - proceed only if you're sure.[/]"
+        )
+    try:
+        plan = plan_delinearize(source_dir, target_dir)
+    except (ValueError, FileNotFoundError) as exc:
+        console.print(f"[red]{escape(str(exc))}[/]")
+        raise typer.Exit(code=2) from exc
+    console.print(
+        Panel(
+            f"Weight files:    {len(plan.weight_files)}\n"
+            f"Source dir:      {escape(plan.source_dir)}\n"
+            f"Target dir:      {escape(plan.target_dir)}",
+            title="Llama 4 Delinearization Plan",
+            border_style="cyan",
+        )
+    )
+    if plan_only:
+        return
+
+    try:
+        result = run_delinearize(plan, num_experts=num_experts)
+    except (TypeError, ValueError, FileNotFoundError) as exc:
+        console.print(f"[red]{escape(str(exc))}[/]")
+        raise typer.Exit(code=2) from exc
+    except ImportError as exc:
+        console.print(f"[red]{escape(str(exc))}[/]")
+        raise typer.Exit(code=1) from exc
+
+    console.print(
+        Panel(
+            f"Files written:     {len(result.files_written)}\n"
+            f"Reshaped keys:     {result.reshaped_keys}\n"
+            f"Already 3-D:       {result.already_3d_keys}\n"
+            f"Passthrough keys:  {result.passthrough_keys}\n"
+            f"Sidecars copied:   {result.sidecars_copied}\n"
+            f"Target dir:        {escape(result.target_dir)}",
+            title="Llama 4 Delinearization — done",
+            border_style="green",
+        )
+    )

{soup_cli-0.71.20 → soup_cli-0.71.22}/src/soup_cli/commands/deploy.py

@@ -4,7 +4,7 @@ from __future__ import annotations
 
 import os
 from pathlib import Path
-from typing import TYPE_CHECKING, Callable, List, Optional
+from typing import TYPE_CHECKING, List, Optional
 
 import typer
 
@@ -792,25 +792,21 @@ def _run_deploy_autopilot_measure(
         f"against {Path(tasks_file).name}...[/]"
     )
 
-    def _placeholder_before(prompt: str) -> str:
-        # The full v0.46.1 live measurement plumbs in real
-        # transformers / vllm generators. v0.53.1 ships the orchestrator
-        # surface; callers / smoke runs can monkeypatch this in.
-        return ""
-
-    def _placeholder_after_factory(candidate: str) -> Callable[[str], str]:
-        def _gen(prompt: str) -> str:
-            return ""
-        return _gen
-
-    # Pull injected generators if the caller registered them via env (escape
-    # hatch for tests + advanced operator workflows)
+    # Injected generators (test seam / advanced operator workflows) still win;
+    # otherwise v0.71.22 #143 first-party transformers factories load the real
+    # base (before) + per-candidate Quant-Menu-quantized model (after), lazily
+    # on first prompt so a cache hit never loads a model.
     from soup_cli.utils import deploy_measure as _dm
-    before_gen = getattr(_dm, "_DEPLOY_MEASURE_BEFORE_GEN", None) or _placeholder_before
-    after_factory = (
-        getattr(_dm, "_DEPLOY_MEASURE_AFTER_FACTORY", None)
-        or _placeholder_after_factory
-    )
+
+    injected_before = getattr(_dm, "_DEPLOY_MEASURE_BEFORE_GEN", None)
+    injected_after = getattr(_dm, "_DEPLOY_MEASURE_AFTER_FACTORY", None)
+    if injected_before is None and injected_after is None:
+        console.print(
+            "[dim]Using live transformers generators (greedy decode; models "
+            "load lazily per candidate)...[/]"
+        )
+    before_gen = injected_before or _dm.build_before_generator(base)
+    after_factory = injected_after or _dm.build_after_generator_factory(base)
 
     try:
         results, cache_hit = run_measure(
@@ -824,10 +820,18 @@ def _run_deploy_autopilot_measure(
     except (TypeError, ValueError, FileNotFoundError) as exc:
         console.print(f"[red]Measure failed:[/] {escape(str(exc))}")
         raise typer.Exit(1) from exc
+    except (RuntimeError, ImportError, OSError) as exc:
+        # Model-load failures from the live factories (missing quant kernel,
+        # OOM, network) — friendly exit, no traceback dump.
+        console.print(f"[red]Live measure failed:[/] {escape(str(exc))}")
+        raise typer.Exit(1) from exc
 
     console.print(render_measure_table(results))
     if cache_hit:
-        console.print("[dim](cache hit — re-run with --no-cache to refresh)[/]")
+        console.print(
+            "[dim](cache hit — delete ~/.soup/deploy_autopilot_cache.json or "
+            "change --tasks to refresh)[/]"
+        )
     best = pick_best(results)
     if best is not None:
         console.print(