soup-cli 0.71.27__tar.gz → 0.71.29__tar.gz

{soup_cli-0.71.27 → soup_cli-0.71.29}/CHANGELOG.md

@@ -12,6 +12,64 @@ reproducing 70+ versions of notes.
 
 ## [Unreleased]
 
+## [0.71.29] - 2026-07-05
+
+### Added
+- **`soup shrink`** — depth-prune a model + optional distill-heal
+  ("The Unreasonable Ineffectiveness of the Deeper Layers", arXiv:2403.17887).
+  Ranks decoder layers by the angular distance of the residual stream across a
+  contiguous block over a calibration set, drops the least-important block
+  (first and last layer always protected), optionally *heals* by distilling the
+  original model into the pruned student, and emits a single dense smaller model
+  plus a one-screen **SHIP / DON'T SHIP** perplexity verdict.
+  `soup shrink --model <id|path> [--drop-ratio 0.25 | --drop-layers N] --calib
+  <calib.jsonl> [--heal <heal.jsonl> --heal-steps N] [--tolerance 0.10]
+  [-o <dir>] [--device cpu] [--attach-to-registry <id>] [--plan-only]`.
+  Exit codes: 0 = SHIP, 2 = DON'T SHIP, 1 = error. The heal runs as an isolated
+  `soup train` subprocess (LoRA-student logit distillation) and the adapter is
+  fused back so the shipped artifact stays a single dense model. Arch allowlist
+  v1: Llama / Qwen / SmolLM. Validated live on SmolLM2-135M (drop 25 %: 30 -> 22
+  layers, ppl x2.98 unhealed; drop 4 + heal: ppl x1.35, recovered).
+
+### Security
+- `soup shrink` contains `--calib` / `--heal` / `--output-dir` (and every
+  derived write path: `<out>/model`, `<out>/heal_adapter`, the fuse staging dir)
+  under cwd with `os.path.realpath` + `commonpath` + O_NOFOLLOW + symlink
+  rejection, re-validating derived paths right before each write (TOCTOU). The
+  heal subprocess uses an argv list (no shell) with a timeout; its config is
+  schema-validated before spawn; subprocess output is C0/ESC-stripped before it
+  reaches the terminal. `--model` defaults `trust_remote_code=False` with a
+  probe + warn.
+
+## [0.71.28] - 2026-07-04
+
+### Added
+- **MCP server (`soup mcp serve`)** — drive Soup from any Model Context Protocol
+  client (Claude Code / Cursor / Cline / Continue) over stdio. No fine-tuning
+  CLI ships an MCP server. Exposes 14 read-only tools as JSON — `advise`,
+  `data_inspect` / `data_validate` / `data_score` / `data_doctor`,
+  `recipes_search` / `recipes_show`, `runs_list` / `runs_show`,
+  `registry_list` / `registry_show`, `profile`, `diagnose_evidence`,
+  `ship_evidence` — plus two **plan-only** mutating tools (`train_start`,
+  `export`) gated behind `--allow-mutating` (they render the exact command that
+  would run; they never execute). The official `mcp` SDK is behind a new
+  `[mcp]` extra (`pip install 'soup-cli[mcp]'`), lazy-imported so the core CLI
+  stays light. Security: stdio-only (no network listener); every path argument
+  re-enters cwd-containment + symlink rejection; output is control-char
+  sanitized; errors are path-free; string / size / int bounds enforced.
+
+### Fixed
+- The DPO / IPO / KTO / BCO trainers now apply configured vocabulary expansion
+  (`data.add_new_tokens` / `data.new_special_tokens`) via the shared
+  `apply_vocab_expansion()` helper during setup, consistent with the SFT path —
+  they previously ignored it. Closes #292
+  ([#293](https://github.com/MakazhanAlpamys/Soup/pull/293) by [@CODING-DARSH](https://github.com/CODING-DARSH)).
+- The ORPO / SimPO / GRPO trainers now also apply configured vocabulary
+  expansion via the shared `apply_vocab_expansion()` helper — completing
+  consistent vocab-expansion behavior across every SFT/preference/RL trainer.
+  Closes #294
+  ([#295](https://github.com/MakazhanAlpamys/Soup/pull/295) by [@CODING-DARSH](https://github.com/CODING-DARSH)).
+
 ## [0.71.27] - 2026-07-03
 
 ### Added

{soup_cli-0.71.27 → soup_cli-0.71.29}/CONTRIBUTING.md

@@ -120,7 +120,7 @@ src/soup_cli/
   templates/          - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0)
   ui/                 - Web UI (FastAPI + HTML/JS SPA)
 
-tests/                - Test suite (304 files, 15042 tests)
+tests/                - Test suite (306 files, 15224 tests)
 examples/             - Real-world config examples and datasets
 ```
 
@@ -150,7 +150,7 @@ pytest tests/test_data.py::test_detect_alpaca_format -v
 pytest tests/ --cov=soup_cli --cov-report=html
 ```
 
-### Test Files (304 files)
+### Test Files (306 files)
 
 > A representative sample of the suite below. The full table lives in
 > [`.claude/CLAUDE.md`](.claude/CLAUDE.md); run `pytest tests/ -v` for the complete list.

{soup_cli-0.71.27 → soup_cli-0.71.29}/CONTRIBUTORS.md

@@ -48,6 +48,8 @@ Listed by first contribution. PR numbers link the work.
   - Harden judge-URL validation against hostname prefix bypass (`startswith` → `urlparse`) in `eval/gate.py` ([#288](https://github.com/MakazhanAlpamys/Soup/pull/288))
   - Apply configured vocabulary expansion (`data.add_new_tokens` / `new_special_tokens`) during SFT trainer init ([#287](https://github.com/MakazhanAlpamys/Soup/pull/287))
   - Reuse the shared vocab-expansion helper in the vision + audio SFT paths ([#291](https://github.com/MakazhanAlpamys/Soup/pull/291))
+  - Honor configured vocab expansion in the DPO / IPO / KTO / BCO trainers ([#293](https://github.com/MakazhanAlpamys/Soup/pull/293))
+  - Honor configured vocab expansion in the ORPO / SimPO / GRPO trainers ([#295](https://github.com/MakazhanAlpamys/Soup/pull/295))
 
 ---
 

{soup_cli-0.71.27 → soup_cli-0.71.29}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: soup-cli
-Version: 0.71.27
+Version: 0.71.29
 Summary: Fine-tune and post-train LLMs in one command. No SSH, no config hell.
 Project-URL: Homepage, https://github.com/MakazhanAlpamys/Soup
 Project-URL: Repository, https://github.com/MakazhanAlpamys/Soup
@@ -29,6 +29,7 @@ Requires-Dist: bitsandbytes>=0.41.0; extra == 'all'
 Requires-Dist: datasets>=2.14.0; extra == 'all'
 Requires-Dist: datasketch>=1.6.0; extra == 'all'
 Requires-Dist: fastapi>=0.104.0; extra == 'all'
+Requires-Dist: mcp>=1.2.0; extra == 'all'
 Requires-Dist: peft>=0.7.0; extra == 'all'
 Requires-Dist: torch>=2.0.0; extra == 'all'
 Requires-Dist: transformers<5.0.0,>=4.36.0; extra == 'all'
@@ -60,6 +61,7 @@ Requires-Dist: bitsandbytes>=0.41.0; extra == 'dev'
 Requires-Dist: cryptography>=41.0.0; extra == 'dev'
 Requires-Dist: datasets>=2.14.0; extra == 'dev'
 Requires-Dist: httpx>=0.24.0; extra == 'dev'
+Requires-Dist: mcp>=1.2.0; extra == 'dev'
 Requires-Dist: mypy>=1.8.0; extra == 'dev'
 Requires-Dist: peft>=0.7.0; extra == 'dev'
 Requires-Dist: pre-commit>=3.5.0; extra == 'dev'
@@ -80,6 +82,8 @@ Provides-Extra: gptq
 Requires-Dist: auto-gptq>=0.7.0; extra == 'gptq'
 Provides-Extra: liger
 Requires-Dist: liger-kernel>=0.3.0; extra == 'liger'
+Provides-Extra: mcp
+Requires-Dist: mcp>=1.2.0; extra == 'mcp'
 Provides-Extra: mix
 Requires-Dist: scikit-optimize>=0.9.0; extra == 'mix'
 Provides-Extra: mlx
@@ -189,25 +193,22 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.27 — Fine-tune Doctor: catch silent failures before they burn a training run.** `soup data doctor` and `soup data lint` — no competitor (Unsloth/Axolotl/LlamaFactory) ships either.
-
-- **The #1 "model never stops generating" bug, caught pre-flight.** `soup data doctor` checks
-  whether every trained turn actually contains an EOS token — plus BOS duplication,
-  no-system-role templates, unknown roles, and truncation risk — before you burn GPU hours on
-  a broken chat template.
-- **See exactly what's trained, token by token.** `--show-mask N` renders sample rows through
-  the REAL collator path (answer-only / RAFT / packing-aware), so an assistant-mask bug is
-  visible instantly instead of silently degrading the whole run.
-- **Stop silently-worse DPO runs.** `soup data lint` flags length bias (the #1 silent
-  preference-tuning degradation, reported as an effect size), near-duplicate pairs,
-  chosen==rejected rows, and prompt leakage — across dpo/orpo/simpo/ipo/bco/kto data.
-- **Found real bugs on a real tokenizer.** Live-validated against
-  `HuggingFaceTB/SmolLM2-135M-Instruct` on Windows + RTX 3050 — the smoke pass itself caught
-  two genuine template-handling bugs before release.
+**v0.71.29 — Make your model smaller, locally: `soup shrink`.** Depth-prune the least-important layers, then distill-heal — with a binary SHIP / DON'T-SHIP verdict. Based on "The Unreasonable Ineffectiveness of the Deeper Layers" (arXiv:2403.17887).
+
+- **Importance-ranked depth pruning.** One forward pass per calibration prompt scores each
+  contiguous layer block by the angular distance of the residual stream across it; the
+  least-important block is dropped (the first and last layer are always protected).
+- **Distill-heal, Minitron-style.** `--heal` distills the original model into the pruned
+  student (LoRA logit-KD) as an isolated `soup train` run, then fuses the adapter back so you
+  ship a single dense smaller model — not a base + adapter.
+- **A verdict, not a dashboard.** Every run ends in **SHIP** or **DON'T SHIP** on a
+  before/after perplexity ratio (`exit 0 = SHIP`, `2 = DON'T`, `1 = error`), so it drops into CI.
+- **Runs on a laptop GPU.** Live-validated on Windows + RTX 3050 with SmolLM2-135M: drop 25 %
+  (30 → 22 layers, 21 % params) and drop-4 + CPU heal (perplexity recovered to ×1.35).
 
 ```bash
-soup data doctor train.jsonl --model meta-llama/Llama-3.1-8B-Instruct --show-mask 3
-soup data lint prefs.jsonl --model meta-llama/Llama-3.1-8B-Instruct
+soup shrink --model HuggingFaceTB/SmolLM2-135M-Instruct --drop-ratio 0.25 \
+    --calib calib.jsonl --heal heal.jsonl --heal-steps 200 -o shrunk --device cpu
 ```
 
 Full history: [CHANGELOG.md](CHANGELOG.md) &middot; [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).

{soup_cli-0.71.27 → soup_cli-0.71.29}/README.md

@@ -49,25 +49,22 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.27 — Fine-tune Doctor: catch silent failures before they burn a training run.** `soup data doctor` and `soup data lint` — no competitor (Unsloth/Axolotl/LlamaFactory) ships either.
-
-- **The #1 "model never stops generating" bug, caught pre-flight.** `soup data doctor` checks
-  whether every trained turn actually contains an EOS token — plus BOS duplication,
-  no-system-role templates, unknown roles, and truncation risk — before you burn GPU hours on
-  a broken chat template.
-- **See exactly what's trained, token by token.** `--show-mask N` renders sample rows through
-  the REAL collator path (answer-only / RAFT / packing-aware), so an assistant-mask bug is
-  visible instantly instead of silently degrading the whole run.
-- **Stop silently-worse DPO runs.** `soup data lint` flags length bias (the #1 silent
-  preference-tuning degradation, reported as an effect size), near-duplicate pairs,
-  chosen==rejected rows, and prompt leakage — across dpo/orpo/simpo/ipo/bco/kto data.
-- **Found real bugs on a real tokenizer.** Live-validated against
-  `HuggingFaceTB/SmolLM2-135M-Instruct` on Windows + RTX 3050 — the smoke pass itself caught
-  two genuine template-handling bugs before release.
+**v0.71.29 — Make your model smaller, locally: `soup shrink`.** Depth-prune the least-important layers, then distill-heal — with a binary SHIP / DON'T-SHIP verdict. Based on "The Unreasonable Ineffectiveness of the Deeper Layers" (arXiv:2403.17887).
+
+- **Importance-ranked depth pruning.** One forward pass per calibration prompt scores each
+  contiguous layer block by the angular distance of the residual stream across it; the
+  least-important block is dropped (the first and last layer are always protected).
+- **Distill-heal, Minitron-style.** `--heal` distills the original model into the pruned
+  student (LoRA logit-KD) as an isolated `soup train` run, then fuses the adapter back so you
+  ship a single dense smaller model — not a base + adapter.
+- **A verdict, not a dashboard.** Every run ends in **SHIP** or **DON'T SHIP** on a
+  before/after perplexity ratio (`exit 0 = SHIP`, `2 = DON'T`, `1 = error`), so it drops into CI.
+- **Runs on a laptop GPU.** Live-validated on Windows + RTX 3050 with SmolLM2-135M: drop 25 %
+  (30 → 22 layers, 21 % params) and drop-4 + CPU heal (perplexity recovered to ×1.35).
 
 ```bash
-soup data doctor train.jsonl --model meta-llama/Llama-3.1-8B-Instruct --show-mask 3
-soup data lint prefs.jsonl --model meta-llama/Llama-3.1-8B-Instruct
+soup shrink --model HuggingFaceTB/SmolLM2-135M-Instruct --drop-ratio 0.25 \
+    --calib calib.jsonl --heal heal.jsonl --heal-steps 200 -o shrunk --device cpu
 ```
 
 Full history: [CHANGELOG.md](CHANGELOG.md) · [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).

{soup_cli-0.71.27 → soup_cli-0.71.29}/docs/commands.md

@@ -156,6 +156,11 @@ soup ship --base <m> --adapter <lora> --task-eval t.jsonl  SHIP / DON'T-SHIP ver
 soup ship --evidence ev.json [--output v.json]  Decide offline from pre-computed scores (no model load)
 soup ship ... --task-mode judge_score --judge-model ollama://llama3.1  Leg-1 via LLM-as-a-judge
 soup ship ... --general-suite mmlu,gsm8k --baseline base.json  lm-eval leg-2 + recorded base scores
+soup mcp serve                                MCP server over stdio (drive Soup from Claude Code / Cursor / Cline; requires [mcp] extra) (v0.71.28)
+soup mcp serve --allow-mutating               Also expose plan-only train_start / export tools (never execute) (v0.71.28)
+soup shrink --model <id|path> --drop-ratio 0.25 --calib c.jsonl -o shrunk  Depth-prune least-important layer block + SHIP/DON'T-SHIP ppl verdict (exit 0/2/1) (v0.71.29)
+soup shrink ... --drop-layers N --heal h.jsonl --heal-steps 200 --device cpu  Drop N layers + distill-heal (fuse LoRA back to one dense model)
+soup shrink ... --tolerance 0.10 --plan-only [--attach-to-registry <id>]  Ppl-regression tolerance / print importance table only / registry attach
 soup tui                                      Full-screen Textual dashboard (requires [tui] extra)
 soup train --config soup.yaml --profile       Record torch.profiler trace to <output>/profiles/
 soup --log-level quiet|normal|verbose|debug   Global logging tier (Rich-formatted)
@@ -267,4 +272,37 @@ soup version [--full] [--json]                Show version (--full: system info,
 soup --verbose <command>                      Full traceback on errors
 ```
 
+## Fine-tune from your coding agent (MCP)
+
+`soup mcp serve` runs a [Model Context Protocol](https://modelcontextprotocol.io)
+server over **stdio**, so any MCP client — Claude Code, Cursor, Cline, Continue —
+can drive Soup conversationally. Install the extra first:
+
+```bash
+pip install 'soup-cli[mcp]'
+```
+
+Register it with your client. For **Claude Code** (`.mcp.json` in the repo) or
+the **Claude Desktop** config (`claude_desktop_config.json`):
+
+```json
+{
+  "mcpServers": {
+    "soup": { "command": "soup", "args": ["mcp", "serve"] }
+  }
+}
+```
+
+The server exposes 14 read-only tools — `advise`, `data_inspect`,
+`data_validate`, `data_score`, `data_doctor`, `recipes_search`, `recipes_show`,
+`runs_list`, `runs_show`, `registry_list`, `registry_show`, `profile`,
+`diagnose_evidence`, `ship_evidence` — each returning JSON. Two **plan-only**
+mutating tools (`train_start`, `export`) are gated behind `--allow-mutating`
+(`"args": ["mcp", "serve", "--allow-mutating"]`); even then they only render the
+exact command that would run — they never execute training or export.
+
+**Security:** stdio only (no network listener); every path argument stays under
+the working directory and rejects symlinks; tool output is control-char
+sanitized; error messages never leak filesystem paths.
+
 

{soup_cli-0.71.27 → soup_cli-0.71.29}/docs/peft-and-efficiency.md

@@ -24,6 +24,57 @@
 - [Training Stability & Auto-Tuning](#training-stability--auto-tuning)
 - [Training Intelligence (Forgetting + Checkpoint Quality)](#training-intelligence-forgetting--checkpoint-quality)
 - [GaLore (Memory-Efficient Full-Parameter Training)](#galore-memory-efficient-full-parameter-training)
+- [Depth Pruning + Distill-Heal (`soup shrink`)](#depth-pruning--distill-heal-soup-shrink)
+
+---
+
+## Depth Pruning + Distill-Heal (`soup shrink`)
+
+`soup shrink` makes a model smaller by dropping its least-important **contiguous
+block of decoder layers**, then optionally *healing* the loss with knowledge
+distillation. It implements "The Unreasonable Ineffectiveness of the Deeper
+Layers" (Gromov et al., arXiv:2403.17887), with a Minitron-style distillation
+heal instead of the paper's plain LoRA fine-tune.
+
+**How it ranks layers.** For each candidate block `[L, L+n)`, one forward pass
+per calibration prompt (`output_hidden_states=True`) measures the **angular
+distance** of the residual stream entering (`hidden_states[L]`) vs leaving
+(`hidden_states[L+n]`) the block, averaged over every non-pad token across the
+calibration set. The block with the *lowest* distance transforms the residual
+stream least, so it is the safest to drop. The first and last decoder layers are
+always protected (they carry the most transformation).
+
+```bash
+# See the importance table + chosen block without writing anything:
+soup shrink --model HuggingFaceTB/SmolLM2-135M-Instruct --drop-ratio 0.25 \
+    --calib calib.jsonl --device cpu --plan-only
+
+# Prune 25% + heal (distill the original into the pruned student, fuse to one
+# dense model) + get a SHIP/DON'T-SHIP perplexity verdict:
+soup shrink --model HuggingFaceTB/SmolLM2-135M-Instruct --drop-ratio 0.25 \
+    --calib calib.jsonl --heal heal.jsonl --heal-steps 200 \
+    --tolerance 0.10 -o shrunk --device cpu --attach-to-registry <id>
+```
+
+- **`--drop-ratio F` / `--drop-layers N`** (exactly one) — how many contiguous
+  layers to drop; the *position* is chosen automatically by the importance scan.
+- **`--calib <jsonl>`** — calibration prompts (`{"text": ...}` / `{"prompt":
+  ...}` / chat `messages`). Must stay under cwd.
+- **`--heal <jsonl> --heal-steps N`** — distill the full-depth original into the
+  pruned student (LoRA logit-KD) as an isolated `soup train` subprocess, then
+  fuse the adapter back into the pruned base so the output is a single dense
+  model. Heal keeps the teacher resident (~2× model memory); with `--device
+  cpu` the heal runs on CPU (validated ≤ 3 B on a 4 GB card).
+- **`--tolerance F`** — ship if the perplexity regression stays within `F`
+  (default `0.10` = 10 %). Exit code `0 = SHIP`, `2 = DON'T SHIP`, `1 = error`.
+- **`-o <dir>`** — the shrunk model lands in `<dir>/model`; the verdict in
+  `<dir>/shrink_report.json`.
+
+**Arch support (v1):** Llama / Qwen / SmolLM. Others are a friendly reject.
+The importance pass loads the model, so live-validated on ≤ 3 B; larger models
+work but are unvalidated on the reference hardware. Perplexity is an unweighted
+mean of per-example perplexities — valid for the before/after *ratio* the
+verdict uses, not directly comparable to `soup eval` absolute numbers.
 
 ---