soup-cli 0.71.7__tar.gz → 0.71.9__tar.gz

{soup_cli-0.71.7 → soup_cli-0.71.9}/CHANGELOG.md

@@ -12,6 +12,99 @@ reproducing 70+ versions of notes.
 
 ## [Unreleased]
 
+## [0.71.9] - 2026-06-03
+
+### Added
+- **Knowledge edit + unlearn — live wiring** (closes #193, #194, #196, #197,
+  #203). The v0.61.0 / v0.62.0 schema-only stubs are now live, validated on
+  SmolLM2-135M.
+- **`soup edit set` (ROME / MEMIT / AlphaEdit) is live** (#194). New
+  `soup_cli/utils/edit_kernels.py` ships covariance-free rank-1 weight-edit
+  kernels: ROME (single-layer `W += δ·kᵀ/‖k‖²`), MEMIT (residual distributed
+  across a layer band), AlphaEdit (ROME update projected orthogonal to the
+  down-proj's top singular direction). `apply_edit` loads the model, optimises
+  the target residual, applies the rank-1 update, and optionally saves with
+  cwd-containment + symlink rejection. `--output`, `--device`, `--governor/
+  --no-governor` flags added. On a tiny model a ROME edit moved
+  `P("Lyon" | "The capital of France is")` from 0.0016 → 0.96.
+- **`soup edit diff` live before/after generation** (#194). Pass
+  `--before-model` + `--after-model` (+ `--probes`) to generate completions
+  through both models and surface the probes whose output changed.
+- **EditGovernor SQLite persistence + cross-process locking** (#196). New
+  `EditGovernorStore` (mirrors `namespace_pin.NamespacePinStore` —
+  $HOME/$CWD/$TMPDIR containment, TOCTOU symlink rejection, WAL +
+  busy_timeout, `fcntl`/`msvcrt` sidecar lock, POSIX 0600). `save_governor` /
+  `load_governor` / `default_governor_db_path` (env override
+  `SOUP_EDIT_GOVERNOR_DB`) persist per-base-model edit-count + verdict across
+  separate `soup edit set` runs.
+- **`apply_edit` consults the EditGovernor automatically** (#197). When a
+  governor is supplied, `check_can_edit()` runs BEFORE the model load (refusing
+  on norm blowup / edit cap) and `record_edit()` runs AFTER with the measured
+  Frobenius delta.
+- **Live GRACE codebook** (#203). `GraceCodebook` (epsilon-ball nearest-key
+  lookup), `apply_grace_edit` (captures a residual key + optimises a value +
+  appends to a codebook sidecar), `save_codebook` / `load_codebook` (atomic,
+  cwd-contained, symlink-rejected), `install_grace_hook` (decode-time residual
+  substitution). New `edited_model` / `grace_codebook` Registry artifact kinds.
+- **`soup train --task unlearn` is live (NPO / SimNPO / RMU)** (#193). New
+  `soup_cli/utils/unlearn_kernels.py` (NPO `(2/β)·mean(-logσ(-β(πlp-reflp)))`,
+  length-normalised SimNPO, RMU representation steering) + a self-contained
+  `UnlearnTrainerWrapper` loop loading a LoRA policy, a frozen reference
+  (NPO/RMU), and forget/retain JSONL datasets. NPO/SimNPO forget loss
+  decreased on the tiny-model smoke. Warns when run without a retain set.
+
+### Security
+- `_save_edited_model` / `UnlearnTrainerWrapper` output dirs + `save_codebook`
+  / `load_codebook` + `_load_unlearn_rows` enforce cwd-containment, raw-path
+  symlink rejection (TOCTOU), null-byte rejection, and file-size / per-line
+  caps. `apply_grace_edit` honours the governor for direct callers.
+
+## [0.71.8] - 2026-06-03
+
+### Added
+- **Probes & SAE — real weights + live downloads** (closes #215, #216, #217,
+  #218, #219). A new shared `soup_cli/utils/probe_kernel.py` provides the
+  linear-probe math (contrast-pair derivation, apply, flag-rate, verdict bands,
+  operator-supplied weight loading, deterministic synthetic fallback); every
+  heavy import (`numpy` / `torch` / `safetensors`) is lazy.
+- **`soup probe sleeper --weights <w.npz|.npy|.safetensors>`** (#215) — load a
+  real calibrated probe direction instead of the synthetic fallback. Weights are
+  cwd-contained, symlink-rejected, `O_NOFOLLOW`-opened, `allow_pickle=False`,
+  and size-capped. `compute_contrast_probe(positive, negative)` derives a probe
+  from contrast-pair activations.
+- **`soup probe sae-diff <repo> --auto-download`** (#216) — fetch an
+  allowlisted SAE from the HF Hub into `~/.soup/sae-cache/` (validated against
+  `HF_HUB_ALLOWLIST` BEFORE any network call) via a new SSRF-hardened
+  `soup_cli.utils.hubs.snapshot_download` (repo-id shape + home/cwd/tmp cache
+  containment + namespace-pin TOFU gate).
+- **`soup probe truth` / `soup probe harm`** (#217) — TruthfulQA-style honesty
+  and HarmBench-style misuse activation probes (6 bundled bases each, 5% / 20%
+  verdict bands, `--weights` to skip the allowlist with a real probe). The
+  probe pack now ships truth + harm entries per base.
+- **`soup probe interference --measure <eval_suite> --base-model <m> --adapter
+  name=path ...`** (#218) — auto-measure the N×N interference matrix by actually
+  loading the base + each LoRA adapter (PEFT multi-adapter), measuring loss for
+  each adapter alone (diagonal) and each co-loaded pair
+  (`add_weighted_adapter(combination_type="cat")`, off-diagonal). Exit 2 on a
+  MAJOR worst-pair.
+- **`soup train --capture-activations <layer> --capture-prompts <jsonl>`** (#219)
+  — a post-training hook writes an SAE-diff-ready per-token activation snapshot
+  to `<output>/activations/activations.json`. `resolve_layer_module` resolves
+  the same `model.layers.N` path whether or not a LoRA adapter is loaded
+  (PEFT-wrapper fallback).
+
+### Security
+- Probe / SAE / capture file I/O is cwd-contained + `O_NOFOLLOW` (TOCTOU close)
+  + size-capped; SAE weight loads use `allow_pickle=False`. SAE auto-download
+  validates the allowlist before any network call and rejects a glob result
+  that resolves outside the snapshot dir (symlink-escape guard).
+
+### Notes
+- #215 is partial: the operator-supplied / contrast-pair / synthetic paths ship
+  now, but the 6 large-base Anthropic-calibrated probe vectors remain
+  upstream-gated (no public calibrated artifact exists). Documented as a known
+  limitation.
+
 ## [0.71.7] - 2026-06-02
 
 ### Added

{soup_cli-0.71.7 → soup_cli-0.71.9}/CONTRIBUTING.md

@@ -120,7 +120,7 @@ src/soup_cli/
   templates/          - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0)
   ui/                 - Web UI (FastAPI + HTML/JS SPA)
 
-tests/                - Test suite (277 files, 12771 tests)
+tests/                - Test suite (279 files, 12998 tests)
 examples/             - Real-world config examples and datasets
 ```
 

{soup_cli-0.71.7 → soup_cli-0.71.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: soup-cli
-Version: 0.71.7
+Version: 0.71.9
 Summary: Fine-tune LLMs in one command. No SSH, no config hell.
 Project-URL: Homepage, https://github.com/MakazhanAlpamys/Soup
 Project-URL: Repository, https://github.com/MakazhanAlpamys/Soup
@@ -183,22 +183,20 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.7 — Eval live runners.** Six probe surfaces that used to emit heuristic / neutral stubs
-now load a real model and run live (opt-in flags; the offline paths stay the default):
-
-- **`soup diagnose --base-model <id> [--adapter <path>] [--dataset d.jsonl]`** — runs all six
-  failure-mode probes (forgetting / refusal / format / mode_collapse / memorization /
-  contamination) against the loaded model instead of emitting neutral OK.
-- **`soup advise --probe-model <id>`** — a live ROI probe: zero/few-shot token-F1 baselines, a
-  short LoRA probe (real held-out-loss drop + wall-clock), and base-model proximity.
-- **`soup tunability --live`** — a real per-candidate LoRA probe (loads each base, trains
-  `--probe-steps` on a held-out slice).
-- **`soup eval capability --live --model <id>`** — invokes lm-eval-harness per task (with
-  `--tasks` / `--limit` / `--device`), isolating per-task failures.
-- **`soup eval behavior --base-model <id> [--adapter <path>]`** — generates pre/post responses on
-  the bundled behaviour battery and scores the live diff.
-- New shared `soup_cli/utils/live_eval.py` holds the model-loading primitives; heavy imports stay
-  lazy. Validated end-to-end on SmolLM2-135M (RTX 3050 4 GB).
+**v0.71.9 — Knowledge edit + unlearn (live).** The surgical-edit and unlearning surfaces are now
+real, validated on SmolLM2-135M:
+
+- **`soup edit set --method rome|memit|alphaedit`** — surgical rank-1 weight edit that patches one
+  fact without re-training. On a tiny model a ROME edit moved P("Lyon" | "The capital of France is")
+  from 0.0016 → 0.96. `--output` saves the edited model; `--governor` refuses edits after norm blowup.
+- **`soup edit diff --before-model <m> --after-model <m> --probes p.jsonl`** — generate completions
+  through both models and surface the probes whose output changed.
+- **`soup edit set --method grace`** — GRACE codebook edit: stores the fact in a discrete
+  (key, value) sidecar applied at decode time via a forward hook (survives thousands of edits).
+- **Sequential-edit governor persistence** — per-base-model edit count + verdict survive across
+  separate `soup edit set` runs (SQLite, cross-process-locked).
+- **`soup train --task unlearn`** — NPO / SimNPO / RMU unlearning from a `forget_set` (+ optional
+  `retain_set`); NPO/SimNPO drive the forget-set loss down while the retain set anchors capability.
 
 Full history: [CHANGELOG.md](CHANGELOG.md) &middot; [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
 

{soup_cli-0.71.7 → soup_cli-0.71.9}/README.md

@@ -49,22 +49,20 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.7 — Eval live runners.** Six probe surfaces that used to emit heuristic / neutral stubs
-now load a real model and run live (opt-in flags; the offline paths stay the default):
-
-- **`soup diagnose --base-model <id> [--adapter <path>] [--dataset d.jsonl]`** — runs all six
-  failure-mode probes (forgetting / refusal / format / mode_collapse / memorization /
-  contamination) against the loaded model instead of emitting neutral OK.
-- **`soup advise --probe-model <id>`** — a live ROI probe: zero/few-shot token-F1 baselines, a
-  short LoRA probe (real held-out-loss drop + wall-clock), and base-model proximity.
-- **`soup tunability --live`** — a real per-candidate LoRA probe (loads each base, trains
-  `--probe-steps` on a held-out slice).
-- **`soup eval capability --live --model <id>`** — invokes lm-eval-harness per task (with
-  `--tasks` / `--limit` / `--device`), isolating per-task failures.
-- **`soup eval behavior --base-model <id> [--adapter <path>]`** — generates pre/post responses on
-  the bundled behaviour battery and scores the live diff.
-- New shared `soup_cli/utils/live_eval.py` holds the model-loading primitives; heavy imports stay
-  lazy. Validated end-to-end on SmolLM2-135M (RTX 3050 4 GB).
+**v0.71.9 — Knowledge edit + unlearn (live).** The surgical-edit and unlearning surfaces are now
+real, validated on SmolLM2-135M:
+
+- **`soup edit set --method rome|memit|alphaedit`** — surgical rank-1 weight edit that patches one
+  fact without re-training. On a tiny model a ROME edit moved P("Lyon" | "The capital of France is")
+  from 0.0016 → 0.96. `--output` saves the edited model; `--governor` refuses edits after norm blowup.
+- **`soup edit diff --before-model <m> --after-model <m> --probes p.jsonl`** — generate completions
+  through both models and surface the probes whose output changed.
+- **`soup edit set --method grace`** — GRACE codebook edit: stores the fact in a discrete
+  (key, value) sidecar applied at decode time via a forward hook (survives thousands of edits).
+- **Sequential-edit governor persistence** — per-base-model edit count + verdict survive across
+  separate `soup edit set` runs (SQLite, cross-process-locked).
+- **`soup train --task unlearn`** — NPO / SimNPO / RMU unlearning from a `forget_set` (+ optional
+  `retain_set`); NPO/SimNPO drive the forget-set loss down while the retain set anchors capability.
 
 Full history: [CHANGELOG.md](CHANGELOG.md) &middot; [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
 

{soup_cli-0.71.7 → soup_cli-0.71.9}/docs/adapters-and-governance.md

@@ -139,21 +139,26 @@ State lives in `.soup/loop.yaml` (atomic write, cwd-contained, symlink-rejected)
 Surgical factual patches WITHOUT a full fine-tuning loop. Hospital data team correcting a misattributed drug interaction, lab fixing a wrong historical date, security team responding to a hallucinated CVE — all one CLI invocation.
 
 ```bash
-# Plan-only mode validates the request + prints the resolved EditPlan + exits 0.
+# Live edit: optimises a residual + applies a rank-1 weight update, then saves.
 soup edit set \
-  --base meta-llama/Llama-3.1-8B-Instruct \
+  --base HuggingFaceTB/SmolLM2-135M \
   --method rome \
-  --subject "Paris is the capital of France" \
+  --subject "The capital of France is" \
   --target "Lyon" \
-  --plan-only
+  --output ./edited --device cuda
+
+# Plan-only mode validates the request + prints the resolved EditPlan + exits 0.
+soup edit set --base ./model --method rome --subject "..." --target "..." --plan-only
 
-# Diff what the model "knew" before vs after the edit.
-soup edit diff <run-id-before> <run-id-after> --probes probes.jsonl --output diff.json
+# Diff what the model says before vs after the edit (live generation).
+soup edit diff before after \
+  --before-model ./base --after-model ./edited \
+  --probes probes.jsonl --output diff.json
 ```
 
-Sequential edit governor auto-switches **ROME → AlphaEdit** at edit #10 (configurable) AND on detected norm-blowup (`||W - W_base||_F` over threshold). The governor refuses further edits past the per-base-model cap so a runaway script can't quietly corrupt your checkpoint.
+The kernels are a covariance-free (`C = I`) variant of the ROME family — well-defined, tractable on a 4 GB box, and validated on SmolLM2-135M (a ROME edit moved `P("Lyon" | "The capital of France is")` from 0.0016 → 0.96). **ROME** edits a single layer; **MEMIT** distributes the residual across a layer band; **AlphaEdit** projects the update orthogonal to the down-proj's top singular direction.
 
-The live ROME / MEMIT / AlphaEdit kernel + before/after generation in `edit diff` land in the next patch; `--plan-only` and the schema surface ship today so soup.yaml and CI invocations are stable.
+The sequential edit governor is persisted (SQLite, cross-process-locked, `SOUP_EDIT_GOVERNOR_DB` override) so the per-base-model edit count + norm-blowup verdict survive across separate `soup edit set` runs. `soup edit set` consults it automatically: it refuses BEFORE the model load past the per-base cap or after a BLOWUP verdict, and records the measured `||ΔW||_F` after each edit. Pass `--no-governor` to opt out. `--registry-id <id>` attaches the edited model (or GRACE codebook) into the Registry lineage.
 
 
 ## Activation Steering (`soup steer`)
@@ -197,7 +202,7 @@ training:
   grace_codebook_dim: 768      # residual-stream width
 ```
 
-`grace` joins the existing `rome` / `memit` / `alphaedit` allowlist on `soup edit set`; the v0.61.0 sequential edit governor still gates the call when the per-base-model edit count or norm-blowup verdict trips.
+`grace` joins the existing `rome` / `memit` / `alphaedit` allowlist on `soup edit set`; the sequential edit governor still gates the call when the per-base-model edit count or norm-blowup verdict trips. GRACE is live: `soup edit set --method grace --output ./ckpt` captures the residual key at the subject's last token, optimises a replacement value, and appends a `(key, value)` triple to a `grace_codebook.json` sidecar (atomic, cwd-contained). At inference the codebook is applied via a forward hook that substitutes the residual whenever it falls within an epsilon ball of a stored key — so the base weights are never modified and thousands of edits survive without norm blowup.
 
 
 ## Model Registry & Lineage

{soup_cli-0.71.7 → soup_cli-0.71.9}/docs/commands.md

@@ -168,8 +168,9 @@ soup can pack --entry-id <id> --out r.can --attest <statement.json>  Embed in-to
 soup audit-log tail / rotate  Tail / rotate the per-command HIPAA/SOC2 audit log (~/.soup/audit.jsonl)
 soup --no-audit-log <cmd> / SOUP_NO_AUDIT_LOG=1  Opt out of the per-command audit line
 soup eval unlearning <run-id> --benchmark tofu|muse|wmdp  Forget Quality + Model Utility + PrivLeak verdict
-soup edit set --base <m> --method rome|memit|alphaedit --subject "..." --target "..."  Surgical knowledge edit (--plan-only available)
-soup edit diff <before-run> <after-run> --probes p.jsonl  Knowledge-injection diff visualizer
+soup edit set --base <m> --method rome|memit|alphaedit|grace --subject "..." --target "..." [--output <dir>] [--device cpu] [--governor/--no-governor] [--registry-id <id>]  Live surgical knowledge edit (--plan-only available)
+soup edit diff <before-run> <after-run> --probes p.jsonl [--before-model <m> --after-model <m>]  Knowledge-injection diff (live before/after generation when both models given)
+soup train --task unlearn  NPO/SimNPO/RMU unlearning from data.forget_set (+ optional data.retain_set)
 soup ingest --source langfuse|langsmith|helicone|openpipe|otel|openai-stored --logs <jsonl>  Universal trace importer (6 SaaS adapters → normalised JSONL)
 soup prune-prompt --input <jsonl> --output <jsonl> --min-frequency 0.95  Detect + strip shared system-prompt prefix
 soup prune-prompt ... --tokenizer <id-or-path>  Tokenizer-aware prefix detection (decodes remaining ids, boundary-safe)
@@ -189,10 +190,15 @@ soup completions bash | zsh | fish                       Shell completion script
 soup license-advisor --target b2c|defense|embedded       Recommend license-clean base for deploy target
 soup license-advisor ... --license <id> --mau N          Per-license downstream-risk check (exit 3 on block)
 soup probe sae-diff <sae> <pre.json> <post.json> [--top-k N]  SAE feature diff between pre/post-FT activations (v0.66.0)
-soup probe sleeper <base> [--evidence ev.json] [--output o.json]  Calibrated sleeper-agent defection probe (v0.66.0)
+soup probe sae-diff <repo> <pre.json> <post.json> --auto-download  Fetch an allowlisted SAE into ~/.soup/sae-cache (v0.71.8)
+soup probe sleeper <base> [--evidence ev.json] [--weights w.npz] [--output o.json]  Sleeper-agent defection probe; --weights = real calibrated probe (v0.66.0; v0.71.8)
+soup probe truth <base> [--evidence ev.json] [--weights w.npz] [--output o.json]  TruthfulQA-style honesty probe (v0.71.8)
+soup probe harm <base> [--evidence ev.json] [--weights w.npz] [--output o.json]  HarmBench-style misuse probe (v0.71.8)
 soup probe interference <losses.json> [--output o.json]  Pairwise N×N adapter interference matrix (exit 2 on MAJOR; v0.66.0)
-soup probe pack <base> [--output o.json]      Per-base calibrated probe pack manifest (v0.66.0)
+soup probe interference --measure <eval.jsonl> --base-model <m> --adapter name=path ... [--device cpu]  Auto-measure live interference (v0.71.8)
+soup probe pack <base> [--output o.json]      Per-base calibrated probe pack manifest (v0.66.0; +truth/harm v0.71.8)
 soup probe pack --list                        List bundled probe-pack bases (v0.66.0)
+soup train --capture-activations <layer> --capture-prompts <jsonl>  Post-train SAE-diff-ready per-token activation snapshot (v0.71.8)
 soup adapters blame ... --top-k 50            Live DataInf-style influence runner (v0.66.0, closes #171)
 soup adapters merge ... --strategy cmaes --eval <s> --budget 1h  CMA-ES evolutionary merge — live loop (v0.67.0 schema / v0.71.4 live)
 soup adapters merge ... --canary <suite.json> [--strict-verdict]  Live OK/MINOR/MAJOR canary verdict, exit 2 on MAJOR (v0.71.4)

{soup_cli-0.71.7 → soup_cli-0.71.9}/docs/evaluation.md

@@ -37,16 +37,29 @@ soup adapters blame ./my-adapter --dataset ./train.jsonl --layer q_proj.7 \
 
 # 3. Sleeper-agent defection probe: per-token defection rate via calibrated linear probe
 soup probe sleeper meta-llama/Llama-3-8B --evidence activations.json
+soup probe sleeper my/model --weights probe.npz --evidence activations.json  # real calibrated probe (v0.71.8)
+
+# 3b. Honesty + misuse probes (v0.71.8) — same shape, 5% / 20% verdict bands
+soup probe truth meta-llama/Llama-3-8B --evidence activations.json
+soup probe harm  meta-llama/Llama-3-8B --evidence activations.json
 
 # 4. Pairwise adapter interference matrix: which pairs can't be deployed together?
 soup probe interference losses.json    # exit 2 if worst-pair score ≥ 20%
+# v0.71.8: auto-measure the matrix live instead of supplying losses.json
+soup probe interference --measure eval.jsonl --base-model meta-llama/Llama-3-8B \
+    --adapter a=./adapter-a --adapter b=./adapter-b --device cpu
 
-# 5. Probe pack: list/assemble calibrated probes per base
+# 5. Probe pack: list/assemble calibrated probes per base (sleeper + truth + harm per base)
 soup probe pack --list                 # list bundled bases
 soup probe pack meta-llama/Llama-3-8B  # render the per-base manifest
+
+# 6. SAE auto-download + capture pipeline (v0.71.8)
+soup train --config soup.yaml --capture-activations model.layers.5 \
+    --capture-prompts probes.jsonl    # writes <output>/activations/activations.json
+soup probe sae-diff google/gemma-scope-2b-pt-res pre.json post.json --auto-download
 ```
 
-Every probe uses the OK / MINOR / MAJOR taxonomy from v0.26 (Quant-Lobotomy) / v0.56 (Diagnose) / v0.65 (Eval Depth). Sleeper + interference exit 2 on MAJOR for CI gating. The blame runner closes the v0.57 `NotImplementedError` stub via a DataInf-style influence approximation: `cos(grad_row, grad_probe) × |grad_row|`. Operators supply a `probe_fn` returning `(row_grads, probe_grad)`, or the runner falls back to a deterministic synthetic probe so the surface always returns a real `BlameResult` (no exception leaks). SAE feature diff is pure-numpy; the safetensors loader is `O_NOFOLLOW`-protected (TOCTOU defence — closes the symlink swap window between containment check and read).
+Every probe uses the OK / MINOR / MAJOR taxonomy from v0.26 (Quant-Lobotomy) / v0.56 (Diagnose) / v0.65 (Eval Depth). Sleeper / truth / harm / interference exit 2 on MAJOR for CI gating. **v0.71.8** ships real probe weights: `--weights <w.npz|.npy|.safetensors>` loads a calibrated direction (cwd-contained, `O_NOFOLLOW`, `allow_pickle=False`, size-capped); `compute_contrast_probe(positive, negative)` derives one from contrast-pair activations; the bundled specs fall back to a deterministic synthetic seed (the large-base Anthropic-calibrated vectors remain upstream-gated). `soup probe interference --measure` loads the base + each LoRA adapter via PEFT and measures loss per adapter alone (diagonal) and per co-loaded pair (`add_weighted_adapter(combination_type="cat")`, off-diagonal). `soup train --capture-activations` writes an SAE-diff-ready per-token snapshot (the `model.layers.N` path resolves whether or not a LoRA adapter is loaded). The blame runner closes the v0.57 `NotImplementedError` stub via a DataInf-style influence approximation: `cos(grad_row, grad_probe) × |grad_row|`. Operators supply a `probe_fn` returning `(row_grads, probe_grad)`, or the runner falls back to a deterministic synthetic probe so the surface always returns a real `BlameResult` (no exception leaks). SAE feature diff is pure-numpy; the safetensors loader is `O_NOFOLLOW`-protected (TOCTOU defence — closes the symlink swap window between containment check and read); `--auto-download` validates the `HF_HUB_ALLOWLIST` before any network call and rejects a glob result escaping the snapshot dir.
 
 
 ## Pre-flight Decision (`soup advise`)

{soup_cli-0.71.7 → soup_cli-0.71.9}/docs/training.md

@@ -101,22 +101,27 @@ GDPR right-to-be-forgotten + CSAM/PII leak response, productized. Three method b
 
 ```yaml
 # unlearn.yaml
-base: meta-llama/Llama-3.1-8B-Instruct
+base: HuggingFaceTB/SmolLM2-135M
 task: unlearn
 data:
   train: traces.jsonl
-  forget_set: gdpr_deletion_set.jsonl
-  retain_set: capability_anchors.jsonl
+  forget_set: gdpr_deletion_set.jsonl   # rows to unlearn (messages / prompt+completion / text)
+  retain_set: capability_anchors.jsonl  # optional — anchors general capability
 training:
   unlearn_method: npo           # or simnpo / rmu
   unlearn_alpha: 0.5            # retain-set weighting [0.0, 10.0]
 ```
 
 ```bash
+# Run the unlearn loop (validated on SmolLM2-135M — NPO/SimNPO drive forget loss down).
+soup train --config unlearn.yaml --yes
+
 # Score the run on TOFU / MUSE / WMDP (OK / MINOR / MAJOR verdict).
 soup eval unlearning <run-id> --benchmark tofu --evidence evidence.json --output report.json
 ```
 
+`soup train --task unlearn` is live (v0.71.9): it loads a LoRA-wrapped policy, a frozen reference copy (NPO / RMU), and the forget / retain JSONL sets, then optimises the per-method loss — NPO's `(2/β)·mean(-logσ(-β·(π_logp − ref_logp)))` drives the policy's forget-set log-prob below the reference (= forgetting), while the retain set anchors capability. Run NPO/SimNPO **with** a `retain_set` — without one the policy has no utility anchor and Soup warns loudly.
+
 Three orthogonal axes: **Forget Quality** (pre/post forget-loss delta), **Model Utility** (retain-accuracy preserved), **PrivLeak** (membership-inference AUC distance from 0.5). Bundled mini-fixtures for all three benchmarks ship in the box (v0.71.1 added MUSE + WMDP alongside the existing TOFU set), so `--benchmark muse|wmdp` runs without supplying evidence. The WMDP forget-set probes ship **redacted** (placeholder prompts + `REFUSED` responses) — Soup never bundles verbatim hazardous-knowledge content.
 
 

{soup_cli-0.71.7 → soup_cli-0.71.9}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "soup-cli"
-version = "0.71.7"
+version = "0.71.9"
 description = "Fine-tune LLMs in one command. No SSH, no config hell."
 readme = "README.md"
 license = "Apache-2.0"

{soup_cli-0.71.7 → soup_cli-0.71.9}/src/soup_cli/__init__.py

@@ -1,3 +1,3 @@
 """Soup CLI — Fine-tune LLMs in one command."""
 
-__version__ = "0.71.7"
+__version__ = "0.71.9"

{soup_cli-0.71.7 → soup_cli-0.71.9}/src/soup_cli/cli.py

@@ -410,8 +410,9 @@ app.add_typer(
     _probe_cmd.app,
     name="probe",
     help=(
-        "Activation probes: sleeper-agent defection / pairwise interference "
-        "/ SAE feature diff / probe pack (v0.66.0)."
+        "Activation probes: sleeper-agent defection / honesty / misuse / "
+        "pairwise interference / SAE feature diff / probe pack (v0.66.0, "
+        "truth+harm v0.71.8)."
     ),
 )
 

{soup_cli-0.71.7 → soup_cli-0.71.9}/src/soup_cli/commands/edit.py

@@ -49,13 +49,25 @@ def set_edit(
         None, "--layer", "-l",
         help="MLP layer index to edit (defaults to method-specific recommended layer).",
     ),
+    output: Optional[str] = typer.Option(
+        None, "--output", "-o",
+        help="Directory to save the edited model / GRACE codebook (cwd-contained).",
+    ),
+    device: Optional[str] = typer.Option(
+        None, "--device",
+        help="torch device (cpu / cuda). Defaults to CUDA when available.",
+    ),
+    use_governor: bool = typer.Option(
+        True, "--governor/--no-governor",
+        help="Consult the sequential-edit governor (refuse on norm blowup).",
+    ),
     plan_only: bool = typer.Option(
         False, "--plan-only",
-        help="Print the resolved EditPlan and exit without applying (deferred to v0.61.1).",
+        help="Print the resolved EditPlan and exit without applying.",
     ),
     registry_id: Optional[str] = typer.Option(
         None, "--registry-id",
-        help="Optional Registry entry id to attach the edited model as a child (v0.61.1).",
+        help="Optional Registry entry id to attach the edited model as a child.",
     ),
 ) -> None:
     """Apply a single surgical knowledge edit."""
@@ -97,32 +109,103 @@ def set_edit(
     )
     console.print(Panel(body, title="EditPlan", border_style="cyan"))
 
-    if registry_id is not None:
-        console.print(
-            f"[dim]Will attach as Registry child of {escape(registry_id)} "
-            f"once v0.61.1 lands.[/]"
-        )
-
     if plan_only:
         console.print("[green]Plan-only mode — exiting without applying.[/]")
         return
 
-    # Live apply: raises NotImplementedError with explicit v0.61.1 marker.
-    # Exit code 3 distinguishes "deferred / not yet shipped" from "exit 2
-    # = validation rejection" (matches v0.56.0 diagnose strict-mode policy).
+    # Load a persisted governor so sequential edits across separate `soup edit
+    # set` runs accumulate (#196 / #197). Best-effort — a missing / unreadable
+    # governor DB never blocks an edit.
+    governor = None
+    store = None
+    if use_governor:
+        try:
+            from soup_cli.utils.edit_governor import (
+                EditGovernorStore,
+                default_governor_db_path,
+                load_governor,
+                save_governor,
+            )
+
+            store = EditGovernorStore(default_governor_db_path())
+            governor = load_governor(store, plan.base)
+        except (ValueError, OSError) as exc:
+            console.print(f"[dim]Governor disabled ({escape(str(exc))}).[/]")
+            governor = None
+            store = None
+
+    from soup_cli.utils.edit_governor import GovernedEditError
+
     try:
-        apply_edit(plan)
-    except NotImplementedError as exc:
+        result = apply_edit(
+            plan,
+            output_dir=output,
+            governor=governor,
+            device=device,
+        )
+    except GovernedEditError as exc:
         console.print(
             Panel(
-                f"[yellow]{escape(str(exc))}[/]\n\n"
-                f"Re-run with [bold]--plan-only[/] to validate the request "
-                f"and exit 0 until v0.61.1 lands the live kernel.",
-                title="Live edit deferred",
-                border_style="yellow",
+                f"[red]Edit refused by governor:[/] {escape(str(exc))}",
+                title="Sequential-edit governor",
+                border_style="red",
             )
         )
-        raise typer.Exit(3) from exc
+        if store is not None:
+            store.close()
+        raise typer.Exit(2) from exc
+    except (ValueError, RuntimeError, OSError) as exc:
+        console.print(f"[red]Edit failed:[/] {escape(str(exc))}")
+        if store is not None:
+            store.close()
+        raise typer.Exit(2) from exc
+
+    # Persist the updated governor state so the next edit sees the new count.
+    if governor is not None and store is not None:
+        try:
+            save_governor(store, governor)
+        except (ValueError, OSError):
+            pass
+        finally:
+            store.close()
+
+    console.print(
+        Panel(
+            f"[bold]Method:[/] {escape(result.method)}\n"
+            f"[bold]Layers edited:[/] {list(result.layers_edited)}\n"
+            f"[bold]Norm delta:[/] {result.norm_delta:.4f}\n"
+            f"[bold]Target prob:[/] {result.target_prob_before:.4f} -> "
+            f"{result.target_prob_after:.4f}\n"
+            + (
+                f"[bold]Saved:[/] {escape(result.output_dir)}\n"
+                if result.output_dir
+                else ""
+            )
+            + (
+                f"[bold]Governor:[/] count="
+                f"{governor.edit_count} verdict={escape(governor.last_verdict)}"
+                if governor is not None
+                else "[dim]Governor disabled.[/]"
+            ),
+            title="Edit applied",
+            border_style="green",
+        )
+    )
+
+    if registry_id is not None and result.output_dir is not None:
+        from soup_cli.registry.attach import attach_artifact
+
+        kind = "grace_codebook" if result.method == "grace" else "edited_model"
+        try:
+            attach_artifact(registry_id, path=result.output_dir, kind=kind)
+            console.print(
+                f"[green]Attached {kind} to Registry entry "
+                f"{escape(registry_id)}.[/]"
+            )
+        except (ValueError, FileNotFoundError) as exc:
+            console.print(
+                f"[yellow]Could not attach to Registry:[/] {escape(str(exc))}"
+            )
 
 
 @app.command(name="diff")
@@ -148,12 +231,25 @@ def diff_edit(
         10, "--top-k", "-k",
         help="Number of changed facts to surface (1-100).",
     ),
+    before_model: Optional[str] = typer.Option(
+        None, "--before-model",
+        help="Model path / HF id of the BEFORE model (enables live generation).",
+    ),
+    after_model: Optional[str] = typer.Option(
+        None, "--after-model",
+        help="Model path / HF id of the AFTER model (enables live generation).",
+    ),
+    device: Optional[str] = typer.Option(
+        None, "--device",
+        help="torch device (cpu / cuda). Defaults to CUDA when available.",
+    ),
 ) -> None:
     """Knowledge-injection diff: facts changed between before / after.
 
-    Schema-only in v0.61.0 — actual model loading + generation is the
-    v0.61.1 deliverable. This release validates inputs + renders a
-    placeholder diff table so the CLI surface is stable.
+    Pass both ``--before-model`` and ``--after-model`` (plus ``--probes``) to
+    generate the diff LIVE (v0.71.9 #194): each probe is run through both
+    models and changed completions are surfaced. Without model paths the
+    report is a validated placeholder.
     """
     from soup_cli.utils.edit_diff import build_diff_report, render_diff_table
 
@@ -163,6 +259,9 @@ def diff_edit(
             after_run_id=after,
             probe_file=probe_file,
             top_k=top_k,
+            before_model=before_model,
+            after_model=after_model,
+            device=device,
         )
     except (TypeError, ValueError, FileNotFoundError) as exc:
         console.print(f"[red]Cannot build diff:[/] {escape(str(exc))}")