soup-cli 0.71.4__tar.gz → 0.71.5__tar.gz

{soup_cli-0.71.4 → soup_cli-0.71.5}/CHANGELOG.md

@@ -12,6 +12,51 @@ reproducing 70+ versions of notes.
 
 ## [Unreleased]
 
+## [0.71.5] - 2026-06-02
+
+### Added
+- **`soup eval against` now reads eval metrics** — `ExperimentTracker.get_metric_series`
+  falls back to the `eval_results` table when the metric is not a per-step
+  training column (`loss` / `lr` / `grad_norm` / `speed` / `gpu_mem`). So
+  `soup eval against <base> --candidate <run> --metric task_accuracy` returns a
+  real score series (benchmark scores live in `eval_results`, not `metrics`)
+  instead of "Empty series". Per-step columns still read from `metrics` — no
+  regression for existing callers.
+- **`soup advise` learns from past project outcomes** — `soup advise` now reads
+  this project's accepted-verdict history (`~/.soup/advise_history.jsonl`) and
+  biases the rubric: 3+ successful SFT precedents flip a marginal RAG call to
+  SFT; 3+ negative GRPO outcomes suppress GRPO in favour of SFT-on-traces; an
+  encouraged choice gets a small confidence nudge. Scoped per-project (one
+  project's record never biases another). No history → identical to before.
+- **Slack/Discord webhooks on four more commands** — `--slack-url` / `--discord-url`
+  (SSRF-hardened, loopback-only HTTP, RFC1918 rejected, never crashes the
+  command) now ship on `soup ingest`, `soup prune-prompt`, `soup ab` (fires only
+  on a `reject_h0` / `accept_h0` decision, not `continue`), and
+  `soup data active-sample` — not just `soup drift-alarm`. The validator + sender
+  moved to a shared `soup_cli/utils/webhooks.py`.
+- **Tokenizer-aware `soup prune-prompt`** — `--tokenizer <model_or_path>` detects
+  and strips the shared system-prompt prefix on **token** boundaries instead of
+  characters, so a multi-byte UTF-8 prefix can never be split mid-code-point.
+  Default (no `--tokenizer`) keeps the whitespace-character behaviour.
+- **Curriculum bucketing by loss percentile** — `DynamicCurriculumCallback` now
+  buckets samples by the percentile rank of the live loss (or perplexity)
+  signal within a rolling window when `data.curriculum_metric` is `loss` /
+  `perplexity`, so a consistently-hard sample is routed to the same difficulty
+  bucket across recomputes. `length` and warm-up still use round-robin.
+- **`--hub` on `soup data push` and `soup data forge`** — `soup data push
+  --hub modelscope|modelers` uploads a dataset via the matching SDK
+  (`repo_type=dataset`, commit message sanitised); `soup data forge --hub
+  <non-hf> --teacher owner/name` pre-fetches the teacher model from that hub
+  (and warns when the teacher is not a repo id so `--hub` is never silently
+  ignored). HF stays the default.
+
+### Notes
+- Live SaaS *pull* adapters for `soup ingest` (Langfuse / LangSmith / Helicone /
+  OpenPipe / OpenAI SDKs, issue #204) remain deferred: they need credentialed
+  vendor accounts with populated trace data to validate honestly. Tracked as an
+  open, `infra-blocked` (external-account) item. `soup ingest` continues to parse
+  the JSONL export you pull from your dashboard.
+
 ## [0.71.4] - 2026-06-02
 
 ### Added

{soup_cli-0.71.4 → soup_cli-0.71.5}/CONTRIBUTING.md

@@ -120,7 +120,7 @@ src/soup_cli/
   templates/          - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0)
   ui/                 - Web UI (FastAPI + HTML/JS SPA)
 
-tests/                - Test suite (274 files, 12474 tests)
+tests/                - Test suite (275 files, 12581 tests)
 examples/             - Real-world config examples and datasets
 ```
 

{soup_cli-0.71.4 → soup_cli-0.71.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: soup-cli
-Version: 0.71.4
+Version: 0.71.5
 Summary: Fine-tune LLMs in one command. No SSH, no config hell.
 Project-URL: Homepage, https://github.com/MakazhanAlpamys/Soup
 Project-URL: Repository, https://github.com/MakazhanAlpamys/Soup
@@ -183,21 +183,22 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.4 — Adapter lifecycle + loop wiring.** The merge, PR, and continuous-loop surfaces go live:
-
-- **Canary verdict on merge** — `soup adapters merge … --canary suite.json` scores the merged
-  adapter and reports **OK / MINOR / MAJOR**; `--strict-verdict` exits non-zero on a MAJOR
-  regression. Works with no model load using a pre-scored canary suite.
-- **Evolutionary merge for real** — `soup adapters merge --strategy cmaes --eval suite --budget 1h`
-  now runs the full CMA-ES search (merge → score → optimise) and writes the best blend, instead of
-  just printing a plan.
-- **Publish an adapter PR** — `soup adapters pr <title> --base-sha <hex> --adapter <path> --push
-  owner/repo#42` posts the rendered PR straight to a GitHub PR comment.
-- **Continuous fine-tuning loop, wired up** — `soup loop watch --pre-wired` runs the real
-  traces → DPO → eval-gate → canary pipeline; `--pack-cans` snapshots every iteration as a
-  shareable Soup Can with Registry lineage (`soup loop replay <id> --extract dir`).
-- **Branches ↔ Registry** — `soup adapters branch <name> --attach-to-registry <id>` /
-  `--from-registry <id>` links training-env snapshots into the Registry lineage DAG.
+**v0.71.5 — Ingest, data & prompt polish.** Sharper production-loop ergonomics:
+
+- **Alternative model hubs for data** — `soup data push --hub modelscope|modelers` uploads a
+  local JSONL to ModelScope / Modelers, and `soup data forge --hub … --teacher owner/name`
+  pre-fetches the teacher from that hub.
+- **Tokenizer-aware prompt pruning** — `soup prune-prompt --tokenizer <id-or-path>` finds the
+  shared *token* prefix and decodes only the remainder, so BPE multi-byte sequences never get
+  truncated mid-token the way char-slicing can.
+- **Webhooks everywhere** — `--slack-url` / `--discord-url` now work on `soup ingest`,
+  `soup prune-prompt`, `soup ab`, and `soup data active-sample` (same SSRF-hardened validator as
+  `soup drift-alarm`). The A/B harness only pings when the sequential test actually decides.
+- **Curriculum by difficulty percentile** — dynamic curriculum can bucket by `loss` /
+  `perplexity` percentile instead of length round-robin.
+- **Smarter pre-flight `advise`** — `soup advise` now nudges its confidence using your prior
+  verdicts for the same project, and `soup runs replay` can plot a benchmark-score curve, not
+  just the loss curve.
 
 Full history: [CHANGELOG.md](CHANGELOG.md) &middot; [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
 

{soup_cli-0.71.4 → soup_cli-0.71.5}/README.md

@@ -49,21 +49,22 @@ infrastructure instead of improving models. Soup fixes that.
 
 ## What's New
 
-**v0.71.4 — Adapter lifecycle + loop wiring.** The merge, PR, and continuous-loop surfaces go live:
-
-- **Canary verdict on merge** — `soup adapters merge … --canary suite.json` scores the merged
-  adapter and reports **OK / MINOR / MAJOR**; `--strict-verdict` exits non-zero on a MAJOR
-  regression. Works with no model load using a pre-scored canary suite.
-- **Evolutionary merge for real** — `soup adapters merge --strategy cmaes --eval suite --budget 1h`
-  now runs the full CMA-ES search (merge → score → optimise) and writes the best blend, instead of
-  just printing a plan.
-- **Publish an adapter PR** — `soup adapters pr <title> --base-sha <hex> --adapter <path> --push
-  owner/repo#42` posts the rendered PR straight to a GitHub PR comment.
-- **Continuous fine-tuning loop, wired up** — `soup loop watch --pre-wired` runs the real
-  traces → DPO → eval-gate → canary pipeline; `--pack-cans` snapshots every iteration as a
-  shareable Soup Can with Registry lineage (`soup loop replay <id> --extract dir`).
-- **Branches ↔ Registry** — `soup adapters branch <name> --attach-to-registry <id>` /
-  `--from-registry <id>` links training-env snapshots into the Registry lineage DAG.
+**v0.71.5 — Ingest, data & prompt polish.** Sharper production-loop ergonomics:
+
+- **Alternative model hubs for data** — `soup data push --hub modelscope|modelers` uploads a
+  local JSONL to ModelScope / Modelers, and `soup data forge --hub … --teacher owner/name`
+  pre-fetches the teacher from that hub.
+- **Tokenizer-aware prompt pruning** — `soup prune-prompt --tokenizer <id-or-path>` finds the
+  shared *token* prefix and decodes only the remainder, so BPE multi-byte sequences never get
+  truncated mid-token the way char-slicing can.
+- **Webhooks everywhere** — `--slack-url` / `--discord-url` now work on `soup ingest`,
+  `soup prune-prompt`, `soup ab`, and `soup data active-sample` (same SSRF-hardened validator as
+  `soup drift-alarm`). The A/B harness only pings when the sequential test actually decides.
+- **Curriculum by difficulty percentile** — dynamic curriculum can bucket by `loss` /
+  `perplexity` percentile instead of length round-robin.
+- **Smarter pre-flight `advise`** — `soup advise` now nudges its confidence using your prior
+  verdicts for the same project, and `soup runs replay` can plot a benchmark-score curve, not
+  just the loss curve.
 
 Full history: [CHANGELOG.md](CHANGELOG.md) &middot; [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
 

{soup_cli-0.71.4 → soup_cli-0.71.5}/docs/backends-and-ops.md

@@ -459,6 +459,11 @@ Every completed run also stores an estimated cost (`$` per run) computed from th
 captured GPU device name and duration. `soup runs show` renders `—` for CPU /
 MPS / unknown GPUs (no fabricated zeros).
 
+As of v0.71.5, the metric-series lookup that powers replay (`ExperimentTracker.get_metric_series`)
+transparently falls back to the `eval_results` table when a metric has no per-step
+rows — so you can plot a benchmark-score curve (e.g. `mmlu`, `gsm8k`) the same way
+you plot `loss`, without caring which table holds the series.
+
 ### Tracker integrations (--tracker mlflow / swanlab / trackio)
 
 ```bash

{soup_cli-0.71.4 → soup_cli-0.71.5}/docs/commands.md

@@ -90,10 +90,12 @@ soup data download user/ds --samples 1000    Stream first 1000 samples
 soup data register --name my-ds --path d.jsonl --format alpaca  Register dataset
 soup data unregister --name my-ds            Remove from registry
 soup data push --input d.jsonl --hf-dataset user/name  Upload local JSONL as HF dataset
+soup data push --input d.jsonl --hf-dataset u/n --hub modelscope|modelers  Upload to an alternative hub
 soup data registry                           List all registered datasets
 soup data demo                                List bundled demo JSONL fixtures
 soup data demo alpaca_demo --output ./d.jsonl Copy a bundled demo JSONL fixture
 soup data forge --docs ./docs --task sft --target-rows 1000  Synthetic data pipeline + provenance
+soup data forge --docs ./docs --hub modelscope --teacher owner/name  Pre-fetch the teacher from an alternative hub
 soup data score --input rows.jsonl            Composite quality scorecard (PII + toxicity + lang + edu)
 soup data decontaminate --input rows.jsonl --benchmarks mmlu,gsm8k  Drop benchmark-overlap rows
 soup data toxicity --input rows.jsonl -o tox.jsonl  Flag toxic rows (keyword baseline)
@@ -137,7 +139,7 @@ soup can publish r.can --hf-hub user/name    Publish .can to HF Hub as dataset
 soup runs                                     List training runs
 soup runs show <run_id>                       Run details + loss graph + cost
 soup runs compare <run_1> <run_2>             Compare two runs
-soup runs replay <run_id>                     Replay summary + loss curve from history
+soup runs replay <run_id>                     Replay summary + loss curve from history (also plots a benchmark-score curve when the metric lives in eval_results)
 soup why [run_id]                             Explain training anomalies (heuristic)
 soup tui                                      Full-screen Textual dashboard (requires [tui] extra)
 soup train --config soup.yaml --profile       Record torch.profiler trace to <output>/profiles/
@@ -169,8 +171,10 @@ soup edit set --base <m> --method rome|memit|alphaedit --subject "..." --target
 soup edit diff <before-run> <after-run> --probes p.jsonl  Knowledge-injection diff visualizer
 soup ingest --source langfuse|langsmith|helicone|openpipe|otel|openai-stored --logs <jsonl>  Universal trace importer (6 SaaS adapters → normalised JSONL)
 soup prune-prompt --input <jsonl> --output <jsonl> --min-frequency 0.95  Detect + strip shared system-prompt prefix
+soup prune-prompt ... --tokenizer <id-or-path>  Tokenizer-aware prefix detection (decodes remaining ids, boundary-safe)
 soup data active-sample --input <jsonl> --output <jsonl> --budget N  Top-N uncertain prod traces for human review
 soup ab --input <jsonl> --metric latency|judge_score|retry_rate  mSPRT sequential A/B (decision: continue / reject_h0 / accept_h0)
+soup ingest|prune-prompt|ab|data active-sample ... --slack-url <https> | --discord-url <https>  Shared SSRF-validated webhook on completion
 soup drift-alarm --reference <jsonl> --live <jsonl> --threshold 0.2  Rolling-KL drift alarm (exit 3 on drift)
 soup drift-alarm ... --slack-url <https> | --discord-url <https>  Optional SSRF-validated webhook on drift detected
 soup tunability --list                                   List built-in candidate-base catalogue

{soup_cli-0.71.4 → soup_cli-0.71.5}/docs/data.md

@@ -94,6 +94,14 @@ soup prune-prompt --input traces.jsonl --output pruned.jsonl --min-frequency 0.9
 
 Binary-search over up-to-32 candidate templates finds the longest qualifying prefix (a longer threshold-meeting prefix may exist beyond the universal one — Soup does not early-exit on the 100% match). Two-pass file read with a 100 000-row DoS cap.
 
+**Tokenizer-aware mode (v0.71.5).** Pass `--tokenizer <id-or-path>` (a HuggingFace repo id, a local path, or anything `AutoTokenizer.from_pretrained` accepts) to detect the shared prefix in *token* space and decode only the remaining ids:
+
+```bash
+soup prune-prompt --input traces.jsonl --output pruned.jsonl --tokenizer Qwen/Qwen2.5-0.5B
+```
+
+Char-level stripping can cut a BPE multi-byte sequence in half when the shared prefix ends mid-token; token-aware pruning finds the longest shared *token-id* prefix and decodes the remainder, so the boundary always lands on a real token. Per-row encoding is capped at 50 000 tokens. Omit `--tokenizer` to keep the original character-level behaviour.
+
 
 ## Active-Learning Sampler (`soup data active-sample`)
 
@@ -108,6 +116,8 @@ soup data active-sample --input traces.jsonl --output for-review.jsonl --budget
 
 The output JSONL is a drop-in prompt set for `soup eval human` (v0.19). Budget is bounded `[1, 100 000]`.
 
+**Webhooks (v0.71.5).** `soup ingest`, `soup prune-prompt`, `soup ab`, and `soup data active-sample` all accept `--slack-url` / `--discord-url` and POST a one-line summary on completion through the same SSRF-hardened validator as `soup drift-alarm` (scheme allowlist, loopback-only HTTP, RFC1918 / link-local / reserved / multicast rejected; the post never raises, so a flaky webhook can't fail the command). `soup ab` only fires when the sequential test actually decides (`reject_h0` / `accept_h0`), not while it's still `continue`-ing.
+
 
 ## Synthetic Data Generation
 
@@ -479,6 +489,8 @@ Three tasks supported: `sft` (Q&A pairs), `preference` (chosen/rejected), `tool`
 
 Document discovery is one level deep over `.txt` / `.md` / `.json` / `.jsonl`; dotfiles + symlinked directories are skipped. All paths are cwd-contained, all writes are atomic via staged-tempfile + `os.replace`, and write targets are rejected if they're symlinks. **Judge providers are live**: `--judge-provider ollama` (localhost-only), `--judge-provider anthropic` (env-only API key), `--judge-provider vllm` (scheme-validated). Per-call judge exceptions logged at DEBUG.
 
+**Alternative teacher hubs (v0.71.5).** `--hub modelscope|modelers` pre-fetches the `--teacher` from that hub when the teacher is a routable repo id (`owner/name`); `--hub hf` (default) is a no-op and leaves the teacher as a provenance label. If `--hub` is non-HF but `--teacher` is not a repo id (e.g. the default `local-judge`), Soup prints a loud yellow warning rather than silently dropping the flag.
+
 
 ## Data Quality Scorecard
 

{soup_cli-0.71.4 → soup_cli-0.71.5}/docs/evaluation.md

@@ -82,6 +82,8 @@ soup advise compare
 4. Task is `factual_lookup` with high output variance → **RAG**.
 5. Otherwise → **SFT**.
 
+**Cross-project confidence bias (v0.71.5).** When `~/.soup/advise_history.jsonl` holds ≥3 prior verdicts for the *same choice* in the *same project*, `soup advise` nudges its confidence (not its decision) toward what worked before: a net-positive precedent record (you accepted it AND its recorded outcome was good) bumps confidence up by a small constant; a net-negative one bumps it down. The rubric verdict itself never changes — only how sure Soup is. Verdicts must be `--record`ed for the bias to kick in.
+
 **Why this command exists.** "Choose fine-tuning vs RAG vs prompt-engineering" is the most-mis-made decision in the space. Reddit, HN, IBM, and Google Cloud all converge on the same advice (start with prompts, escalate to RAG, fine-tune as last resort) and almost everyone ignores it because nobody has the data to prove their case is the exception. Soup `autopilot` picks hyperparameters AFTER you've decided to train; `soup advise` owns the layer above. No trainer library has an incentive to tell users *not to train* — Unsloth's funnel, Axolotl's hosted business, LLaMA-Factory's Alibaba alignment all monetise the training event.
 
 
@@ -213,6 +215,8 @@ soup ab --input ab.jsonl --metric judge_score --alpha 0.01 --beta 0.10 --effect-
 
 Input rows look like `{"arm": "control", "latency": 1.23}` or `{"arm": "treatment", "judge_score": 0.91}`. Decision is one of `continue` (keep collecting samples), `reject_h0` (real difference detected), `accept_h0` (no significant difference). Composes with `soup loop canary` (v0.58) — promote or roll back as soon as the LLR clears a decision boundary.
 
+`soup ab` accepts `--slack-url` / `--discord-url` (v0.71.5) and pings the webhook **only when the test actually decides** (`reject_h0` / `accept_h0`) — a still-running `continue` stays quiet so you're not paged on every peek. Same SSRF-hardened validator as `soup drift-alarm`.
+
 
 ## Drift Alarm (`soup drift-alarm`)
 

{soup_cli-0.71.4 → soup_cli-0.71.5}/docs/training.md

@@ -907,12 +907,15 @@ Layer dynamic re-weighting on top of the static `curriculum` bucketer. Every N s
 training:
   curriculum: true                          # static bucketer (v0.23.0)
   curriculum_buckets: 4
+  curriculum_metric: perplexity             # length (default) | loss | perplexity
   curriculum_dynamic: true                  # NEW — dynamic re-weighting
   curriculum_dynamic_recompute_steps: 50    # refresh every 50 global steps
   curriculum_dynamic_floor: 0.05            # min weight per bucket
   curriculum_dynamic_temperature: 1.0       # softmax temp on uncertainty
 ```
 
+**Bucketing by difficulty percentile (v0.71.5).** When `curriculum_metric` is `loss` or `perplexity`, the dynamic callback assigns each step's sample to a bucket by its *rank* within a rolling 512-step window of the difficulty signal (perplexity = `exp(min(loss, 50))`), instead of the round-robin fallback used for `length`. This keeps the buckets calibrated to the live loss distribution rather than a static length sort. `length` (the default) keeps the round-robin assignment.
+
 Visualise the recorded bucket-weight evolution with `soup runs curriculum-curve <run_id>`.
 
 DDP / grad-accum safety: multi-rank launches must wire an `all_reduce` hook on per-bucket stats (a cross-validator rejects un-coordinated multi-rank runs upfront). Multi-trainer expansion beyond `sft` / `pretrain` is tracked for v0.48.1.

{soup_cli-0.71.4 → soup_cli-0.71.5}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "soup-cli"
-version = "0.71.4"
+version = "0.71.5"
 description = "Fine-tune LLMs in one command. No SSH, no config hell."
 readme = "README.md"
 license = "Apache-2.0"

{soup_cli-0.71.4 → soup_cli-0.71.5}/src/soup_cli/__init__.py

@@ -1,3 +1,3 @@
 """Soup CLI — Fine-tune LLMs in one command."""
 
-__version__ = "0.71.4"
+__version__ = "0.71.5"

soup_cli-0.71.5/src/soup_cli/commands/_webhook_cli.py

@@ -0,0 +1,71 @@
+"""Shared CLI glue for the --slack-url / --discord-url webhook flags (v0.71.5 #207).
+
+Keeps Typer + Rich Console concerns in the commands layer (``utils/webhooks``
+stays import-light + framework-free). Used by ``ingest`` / ``prune-prompt`` /
+``ab`` / ``data active-sample`` so the validate-then-deliver pattern is defined
+once.
+"""
+
+from __future__ import annotations
+
+from typing import Optional, Tuple
+
+import typer
+from rich.console import Console
+from rich.markup import escape
+
+
+def validate_webhook_flags(
+    slack_url: Optional[str],
+    discord_url: Optional[str],
+    *,
+    console: Console,
+) -> Tuple[Optional[str], Optional[str]]:
+    """Validate webhook URLs at the CLI boundary (``typer.Exit(2)`` on bad).
+
+    Returns the (canonical) URLs. Mirrors the v0.63.0 ``drift-alarm``
+    early-rejection pattern so a typo'd / SSRF-y URL fails fast with a
+    friendly message instead of being silently swallowed at delivery time.
+    """
+    from soup_cli.utils.webhooks import validate_webhook_url
+
+    out = []
+    for label, value in (("--slack-url", slack_url), ("--discord-url", discord_url)):
+        if value is None:
+            out.append(None)
+            continue
+        try:
+            out.append(validate_webhook_url(value))
+        except (TypeError, ValueError) as exc:
+            console.print(f"[red]{label}: {escape(str(exc))}[/]")
+            raise typer.Exit(2) from exc
+    return out[0], out[1]
+
+
+def emit_webhooks(
+    slack_url: Optional[str],
+    discord_url: Optional[str],
+    *,
+    payload: dict,
+    console: Console,
+) -> None:
+    """POST the completion payload to any configured webhooks (best-effort).
+
+    Never raises (delegates to the never-raising
+    :func:`soup_cli.utils.webhooks.send_webhooks`). Prints a per-target
+    delivered/failed line so the operator sees whether the alert landed.
+    """
+    if slack_url is None and discord_url is None:
+        return
+    from soup_cli.utils.webhooks import send_webhooks
+
+    for label, ok in send_webhooks(
+        payload, slack_url=slack_url, discord_url=discord_url
+    ):
+        colour = "green" if ok else "yellow"
+        console.print(
+            f"[{colour}]{label} webhook: {'delivered' if ok else 'failed'}[/]"
+        )
+
+
+__all__ = ["emit_webhooks", "validate_webhook_flags"]

{soup_cli-0.71.4 → soup_cli-0.71.5}/src/soup_cli/commands/ab.py

@@ -2,12 +2,15 @@
 
 from __future__ import annotations
 
+from typing import Optional
+
 import typer
 from rich.console import Console
 from rich.markup import escape
 from rich.panel import Panel
 from rich.table import Table
 
+from soup_cli.commands._webhook_cli import emit_webhooks, validate_webhook_flags
 from soup_cli.utils.ab_test import (
     MsprtConfig,
     run_msprt,
@@ -35,6 +38,20 @@ def ab(
         0.1, "--effect-size",
         help="Minimum detectable difference in means.",
     ),
+    slack_url: Optional[str] = typer.Option(
+        None, "--slack-url",
+        help=(
+            "Optional Slack webhook URL — POSTed on a reject_h0 / accept_h0 "
+            "decision (not on continue). SSRF-validated."
+        ),
+    ),
+    discord_url: Optional[str] = typer.Option(
+        None, "--discord-url",
+        help=(
+            "Optional Discord webhook URL — POSTed on a reject_h0 / accept_h0 "
+            "decision (not on continue). SSRF-validated."
+        ),
+    ),
 ) -> None:
     """Sequential A/B test with early-stop guarantees (mSPRT)."""
     try:
@@ -43,6 +60,10 @@ def ab(
         console.print(f"[red]{escape(str(exc))}[/]")
         raise typer.Exit(2) from exc
 
+    slack_url, discord_url = validate_webhook_flags(
+        slack_url, discord_url, console=console
+    )
+
     try:
         cfg = MsprtConfig(
             metric=canonical, alpha=alpha, beta=beta, effect_size=effect_size,
@@ -101,5 +122,24 @@ def ab(
             )
         )
 
+    # Webhook only fires on a terminal decision (reject_h0 / accept_h0) —
+    # a `continue` verdict carries no actionable signal (issue #207).
+    if verdict.decision != "continue":
+        emit_webhooks(
+            slack_url,
+            discord_url,
+            payload={
+                "command": "ab",
+                "metric": canonical,
+                "decision": verdict.decision,
+                "log_likelihood_ratio": verdict.log_likelihood_ratio,
+                "n_control": verdict.n_control,
+                "n_treatment": verdict.n_treatment,
+                "mean_control": verdict.mean_control,
+                "mean_treatment": verdict.mean_treatment,
+            },
+            console=console,
+        )
+
 
 __all__ = ["ab"]

{soup_cli-0.71.4 → soup_cli-0.71.5}/src/soup_cli/commands/active_sample.py

@@ -2,11 +2,14 @@
 
 from __future__ import annotations
 
+from typing import Optional
+
 import typer
 from rich.console import Console
 from rich.markup import escape
 from rich.panel import Panel
 
+from soup_cli.commands._webhook_cli import emit_webhooks, validate_webhook_flags
 from soup_cli.utils.active_sampler import sample_uncertain_rows, validate_budget
 
 console = Console()
@@ -24,6 +27,14 @@ def active_sample(
         100, "--budget",
         help="Max rows to surface for human review (1 - 100_000).",
     ),
+    slack_url: Optional[str] = typer.Option(
+        None, "--slack-url",
+        help="Optional Slack webhook URL — POSTed on completion. SSRF-validated.",
+    ),
+    discord_url: Optional[str] = typer.Option(
+        None, "--discord-url",
+        help="Optional Discord webhook URL — POSTed on completion. SSRF-validated.",
+    ),
 ) -> None:
     """Surface the most uncertain prod traces for human review."""
     try:
@@ -32,6 +43,10 @@ def active_sample(
         console.print(f"[red]{escape(str(exc))}[/]")
         raise typer.Exit(2) from exc
 
+    slack_url, discord_url = validate_webhook_flags(
+        slack_url, discord_url, console=console
+    )
+
     try:
         plan = sample_uncertain_rows(
             input_path,
@@ -55,5 +70,18 @@ def active_sample(
         )
     )
 
+    emit_webhooks(
+        slack_url,
+        discord_url,
+        payload={
+            "command": "active-sample",
+            "rows_selected": plan.rows_selected,
+            "rows_in": plan.rows_in,
+            "mean_uncertainty": plan.mean_uncertainty,
+            "budget": plan.budget,
+        },
+        console=console,
+    )
+
 
 __all__ = ["active_sample"]

{soup_cli-0.71.4 → soup_cli-0.71.5}/src/soup_cli/commands/advise.py

@@ -39,6 +39,7 @@ from soup_cli.utils.advise import (
     synth_probe_lora_delta,
 )
 from soup_cli.utils.advise_history import (
+    current_project_name,
     history_path,
     load_history,
     record_verdict,
@@ -269,8 +270,27 @@ def advise_run(
             sft_wall_clock_secs=wall_clock,
         )
 
+    # v0.71.5 #163 — bias the rubric by this project's past accepted-verdict
+    # outcomes. Best-effort: a missing / unreadable history must never block
+    # a verdict, so any failure falls back to the un-biased rubric.
+    history = None
+    project = None
     try:
-        verdict = build_verdict(profile, task_category, goal=goal, roi=roi)
+        history = load_history(limit=20)
+        project = current_project_name()
+    except (TypeError, ValueError, OSError):
+        history = None
+        project = None
+
+    try:
+        verdict = build_verdict(
+            profile,
+            task_category,
+            goal=goal,
+            roi=roi,
+            history=history,
+            project=project,
+        )
     except (TypeError, ValueError) as exc:
         console.print(f"[red]Verdict build failed:[/] {escape(str(exc))}")
         raise typer.Exit(1) from exc

{soup_cli-0.71.4 → soup_cli-0.71.5}/src/soup_cli/commands/data.py

@@ -1914,16 +1914,32 @@ def push_dataset_cmd(
         "--message",
         help="Commit message for the dataset upload",
     ),
+    hub: str = typer.Option(
+        "hf",
+        "--hub",
+        help=(
+            "Target hub: hf (default) / modelscope / modelers. Non-HF hubs "
+            "upload via the matching SDK (v0.71.5 #157) — repo_type=dataset, "
+            "commit message sanitised to first line + 200 chars."
+        ),
+    ),
 ):
-    """Upload a local JSONL dataset to HuggingFace Hub as a dataset repo."""
+    """Upload a local JSONL dataset to a model hub as a dataset repo."""
     from soup_cli.utils.hf import (
         get_hf_api,
         resolve_endpoint,
         resolve_token,
         validate_repo_id,
     )
+    from soup_cli.utils.hubs import validate_hub_name
     from soup_cli.utils.paths import is_under_cwd
 
+    try:
+        hub_canonical = validate_hub_name(hub)
+    except (TypeError, ValueError) as exc:
+        console.print(f"[red]{exc}[/]")
+        raise typer.Exit(2) from exc
+
     file_path = Path(input_path)
     if not file_path.exists():
         console.print(f"[red]Dataset file not found: {file_path}[/]")
@@ -1940,9 +1956,18 @@ def push_dataset_cmd(
     try:
         validate_repo_id(hf_dataset)
     except ValueError as exc:
-        console.print(f"[red]Invalid --hf-dataset repo id:[/] {exc}")
+        console.print(f"[red]Invalid --{hub_canonical}-dataset repo id:[/] {exc}")
         raise typer.Exit(1) from exc
 
+    # v0.71.5 #157 — non-HF hubs route through the shared upload_repo adapter.
+    # ModelScope / Modelers SDKs upload a folder, so the single JSONL is
+    # staged into a temp dir and uploaded as a dataset repo.
+    if hub_canonical != "hf":
+        _push_dataset_non_hf(
+            hub_canonical, hf_dataset, file_path, commit_message
+        )
+        return
+
     token = resolve_token()
     if token is None:
         console.print(
@@ -1987,6 +2012,53 @@ def push_dataset_cmd(
     )
 
 
+def _push_dataset_non_hf(
+    hub: str, repo_id: str, file_path: Path, commit_message: str
+) -> None:
+    """Upload a single JSONL to a non-HF hub as a dataset (v0.71.5 #157).
+
+    ``upload_repo`` uploads a folder, so the file is staged into a temp dir
+    first. ``upload_repo`` already sanitises the commit message (first line +
+    200 chars) and validates the repo id shape.
+    """
+    import shutil
+    import tempfile
+
+    from rich.markup import escape
+
+    from soup_cli.utils.hubs import upload_repo
+
+    # Stage UNDER cwd — `upload_repo` enforces cwd-containment on folder_path
+    # (the system tempdir would be rejected).
+    staging = tempfile.mkdtemp(prefix=".soup_dataset_push.", dir=os.getcwd())
+    try:
+        shutil.copy2(str(file_path), os.path.join(staging, file_path.name))
+        try:
+            upload_repo(
+                hub,
+                repo_id,
+                folder_path=staging,
+                commit_message=commit_message,
+                repo_type="dataset",
+            )
+        except ImportError as exc:
+            console.print(f"[red]{exc}[/]")
+            raise typer.Exit(1) from exc
+        except (TypeError, ValueError) as exc:
+            console.print(f"[red]Upload failed:[/] {exc}")
+            raise typer.Exit(1) from exc
+        except Exception as exc:  # noqa: BLE001 — surface SDK errors generically
+            console.print(f"[red]Upload failed:[/] {exc}")
+            raise typer.Exit(1) from exc
+    finally:
+        shutil.rmtree(staging, ignore_errors=True)
+
+    console.print(
+        f"[green]Uploaded[/] {escape(file_path.name)} to {escape(hub)} "
+        f"dataset [bold]{escape(repo_id)}[/]"
+    )
+
+
 # --- v0.42.0 Part C / F: AOT preprocess + document ingestion ---------------
 
 @app.command(name="preprocess")