soup-cli 0.40.2__tar.gz → 0.40.3__tar.gz

{soup_cli-0.40.2 → soup_cli-0.40.3}/CONTRIBUTING.md

@@ -111,7 +111,7 @@ soup_cli/
   templates/          - 17 built-in soup.yaml templates (YAML + manifest.json) with load_template loader (v0.39.0, +bco v0.40.0)
   ui/                 - Web UI (FastAPI + HTML/JS SPA)
 
-tests/                - Test suite (143 files, 4756 tests)
+tests/                - Test suite (146 files, 4855 tests)
 examples/             - Real-world config examples and datasets
 ```
 

{soup_cli-0.40.2 → soup_cli-0.40.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: soup-cli
-Version: 0.40.2
+Version: 0.40.3
 Summary: Fine-tune LLMs in one command. No SSH, no config hell.
 Project-URL: Homepage, https://github.com/MakazhanAlpamys/Soup
 Project-URL: Repository, https://github.com/MakazhanAlpamys/Soup
@@ -134,17 +134,13 @@ soup train
 
 Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
 
-**v0.40.2 — Quick polish + carry-overs**: closes 3 originally-scheduled GitHub issues (#36 eval-gate dashboard row, #50 smarter `--hf-resume`, #51 custom HF Space templates) plus 7 v0.40.1 long-tail UX papercuts.
+**v0.40.3 — Stub-to-live**: three v0.X.0 deferred-stub features become live runtime — closes #33 (data harvester judge filter + serve trace log), #64 (live CUDA OOM probe), #65 (multipack sampler in HF Trainer).
 
-- **`--hf-resume` prefers local newer** — `prepare_hf_resume` checks the highest local `checkpoint-N` against the highest remote branch and skips the snapshot download when the local copy is newer or equal. Saves bandwidth and never overwrites a fresher local checkpoint with stale Hub state.
-- **Eval-gate dashboard row** — pure formatter `format_gate_row(state)` lives in `soup_cli/monitoring/display.py` and renders a one-liner like `Gate: helpfulness 7.8 ✓ | math 0.82 ✗ (-0.06) | STOP` for the live training panel. Hidden when eval-gate is disabled.
-- **`soup deploy hf-space --template-dir <path>`** — supply your own `app.py` + `README.md` (+ optional `requirements.txt`) instead of the built-in `gradio-chat` / `streamlit-chat`. Containment-checked, repo-id substitution validated *before* render, 256 KB cap per file, symlinks rejected.
-- **`soup quickstart --output DIR`** — route data, config, and run dir under any directory you choose (default keeps backwards-compat in cwd).
-- **`soup runs --cwd-only`** — restrict the listing to runs whose `output_dir` is under the current directory; the global `~/.soup/experiments.db` view is still default.
-- **`soup infer` / `soup bench` accept HF ids** — when the local model path is missing AND the value isn't path-like (no `./`, `/`, `~`, `C:\`), it falls through to a HuggingFace download via `transformers.from_pretrained`. Path-like-but-missing surfaces a friendly `FileNotFoundError`.
-- **CLI flag aliases** — `data filter --min-coherence` (alias for `--coherence`); `data split --train` accepted (informational, train is the implicit remainder); `data register / unregister` accept positional `<name>` and `<path>` alongside the `--name` / `--path` options, with conflict detection.
-- **`--log-level` plumbing complete** — `apply_logging_level` now sets the root logger so third-party libraries (transformers / peft / trl) actually respect QUIET and DEBUG. The four tiers no longer produce byte-identical output.
-- **+36 net new tests** across the new helpers, security-fix follow-ups (path-containment in `register_data`, `bench.py`, `infer.py --output`, symlink-reject in custom Space templates), and Windows cross-drive edge cases.
+- **Live CUDA batch-size probe** — `auto_batch_size_strategy: probe` now runs ONE forward+backward+step on a synthetic batch per candidate before training. On `torch.cuda.OutOfMemoryError` the probe halves; otherwise it doubles. Result is cached per `(model, max_length, quant, lora_r, gpu)` tuple so the next run short-circuits. CPU sessions skip the probe and fall back to the static estimate. SFT-only this release.
+- **Multipack sampler — helpers landed, live wiring deferred to v0.40.4** — adversarial review surfaced a HF Trainer DataLoader shape mismatch (`Sampler[int]` expected, `list[list[int]]` returned). Helpers (lru-cached subclass factory + state-attach with bounds + arch-detect + length-extract with all-zero warning) ship as a stub; live wiring requires a `get_train_dataloader` override and lands next patch.
+- **`soup data from-traces --judge`** — optional LLM-as-a-judge pass over harvested preference pairs. `--judge-provider openai|server|ollama`, `--judge-model gpt-4o-mini`, `--min-confidence 0.7`. Drops pairs whose normalised `(chosen - rejected)` confidence falls below threshold. Per-pair backend exceptions are counted, not crashed; lazy `itertools.islice` cap avoids buffering pathological generators.
+- **`soup serve --trace-log <path>`** — passive append-only JSONL request log (`{prompt, response, latency_ms, tokens, ts}` per chat completion). Path-containment validated, 100 MB rotation cap (one backup retained, symlink-reject on rotate), and `hf_*` / `sk-*` / `Bearer …` token shapes redacted to `<redacted>` before write (mirrors v0.34.0 `crash.py` policy).
+- **+95 net new tests** across the new closures, the dynamic Trainer subclass, the judge filter (including degenerate-scale + lazy-materialisation cases), and the trace logger (including symlink-backup rejection + multi-thread append safety).
 
 ## Why Soup?
 
@@ -661,7 +657,7 @@ training:
 
 **Architecture allowlist** — 18 supported (Llama 3.x, Qwen 2/3, Mistral, Gemma 2/3, Phi 3/4, DeepSeek V2/V3, Mixtral, Falcon, StableLM, SmolLM2). Unknown architectures **fail loudly at config-load** instead of silently no-opping (critical fix vs Axolotl's silent-miss footgun).
 
-**v0.37.0 scope:** schema gate + helper builder ship now. Live wiring of the sampler into HF Trainer's `_get_train_sampler` lands in v0.37.1 (mirrors v0.27.0 MII stub-then-live pattern). Multipack is **sft / pretrain only** on the `transformers` backend; preference / RLHF trainers and MLX backend get distinct error messages naming the actual reason.
+**Live wiring** — still deferred. v0.40.3 ships the helpers (`make_multipack_trainer_class` lru-cached factory + `attach_multipack_state` + `lengths_from_dataset` + `detect_arch_name`) but neither SFT nor Pretrain wrappers instantiate the subclass — adversarial review caught a `Sampler[int]` vs `list[list[int]]` mismatch with HF Trainer's DataLoader. Setting `multipack: true` prints a yellow advisory and falls back to the standard sampler. Live wiring (via a `get_train_dataloader` override with `batch_sampler=`) lands in v0.40.4. Multipack is **sft / pretrain only** on the `transformers` backend; preference / RLHF trainers and MLX backend still get distinct error messages naming the actual reason.
 
 **DoS hardening** — the FFD packer caps at 1M items (algorithm is O(N²) worst-case); the 4D mask builder caps allocations at 2³¹ cells; the chat-template Jinja analyzer caps at 128KB. Every numeric input rejects `bool` explicitly (matches v0.30.0+ project policy).
 
@@ -2579,6 +2575,7 @@ soup serve --model <m> --structured-output json --json-schema s.json  Constraine
 soup serve --model <m> --structured-output regex --regex-pattern '...'  Regex-constrained output
 soup serve --model <m> --dashboard            Live dashboard + /metrics endpoint
 soup serve --model <m> --trace --trace-endpoint http://localhost:4317  OpenTelemetry tracing
+soup serve --model <m> --trace-log ./serve.jsonl  Per-request JSONL log + rotation + secret redaction
 POST /v1/adapters/activate/<name>             Hot-swap active LoRA adapter
 soup sweep --config soup.yaml --param lr=...  Hyperparameter search
 soup diff --model-a ./a --model-b ./b         Compare two models
@@ -2594,7 +2591,9 @@ soup data generate ... --provider anthropic   Use Claude API
 soup data generate ... --provider vllm        Use local vLLM server
 soup data generate ... --template code        Domain templates (code/conversation/qa/preference/reasoning)
 soup data generate ... --quality-pipeline     Auto validate + filter + dedup
-soup data augment <path> --strategy rephrase|translate|style  LLM-driven augmentationsoup data from-traces --logs l.jsonl --format langchain --signal thumbs_up --output p.jsonl  Preference pairs from tracessoup data review prefs.jsonl --sample 10      Preview preference pairssoup data filter <path> --coherence 0.3       Quality filter (perplexity/coherence)
+soup data augment <path> --strategy rephrase|translate|style  LLM-driven augmentationsoup data from-traces --logs l.jsonl --format langchain --signal thumbs_up --output p.jsonl  Preference pairs from traces
+soup data from-traces ... --judge --min-confidence 0.7  LLM-judge confidence filter
+soup data review prefs.jsonl --sample 10      Preview preference pairssoup data filter <path> --coherence 0.3       Quality filter (perplexity/coherence)
 soup data sample <path> --n 1000             Random sample subset
 soup data sample <path> --n 1000 --strategy diverse  Cluster-based diverse sampling
 soup data sample <path> --n 1000 --strategy hard     Sample hardest examples
@@ -2791,6 +2790,44 @@ pytest tests/ -v
 pytest tests/ -m smoke -v
 ```
 
+## Live CUDA Batch-Size Probe
+
+Set `auto_batch_size_strategy: probe` in `training:` and Soup will run a real OOM-probe before training:
+
+```yaml
+training:
+  batch_size: auto
+  auto_batch_size_strategy: probe
+```
+
+For each candidate size `B`, the probe runs ONE forward + backward + step on a synthetic batch of `B` sequences of length `max_length`. On `torch.cuda.OutOfMemoryError` it halves; otherwise it doubles up to `4 × static_estimate`. The picked size is cached per `(model, max_length, quantization, lora_r, gpu)` tuple in `~/.soup/batch_cache.json` so subsequent runs skip the probe.
+
+CPU sessions and `auto_batch_size_strategy: static` skip the probe. Synthetic batch tensors are freed before the backward pass so peak VRAM reflects the realistic training step. SFT-only this release — non-SFT trainers fall back to the static estimate.
+
+## Trace-to-Preference: LLM-Judge Filter
+
+`soup data from-traces --judge` filters harvested preference pairs through an LLM judge:
+
+```bash
+soup data from-traces \
+  --logs ./prod-traces.jsonl --format langchain --signal thumbs_up \
+  --output ./prefs.jsonl \
+  --judge --judge-provider ollama --judge-model llama3 \
+  --min-confidence 0.7
+```
+
+The judge scores `chosen` and `rejected` independently against its rubric (default helpfulness/accuracy/safety on a 1-5 scale). Pairs whose normalised `(chosen - rejected)` confidence falls below `--min-confidence` are dropped. Per-pair backend exceptions are counted (not crashed) and reported. Provider allowlist `{openai, server, ollama}` validated at the CLI boundary; SSRF protection on `--judge-api-base` carries over from `soup eval judge`.
+
+## Inference Server Trace Log
+
+`soup serve --trace-log <path>` writes a passive append-only JSONL log per chat completion:
+
+```bash
+soup serve --model ./out --trace-log ./serve-trace.jsonl --trace-log-cap-mb 100
+```
+
+Each line: `{"ts": ..., "prompt": ..., "response": ..., "latency_ms": ..., "tokens": ...}`. Path-containment validated, hard rotation cap (default 100 MB, one backup retained), symlink-reject on the backup path (TOCTOU defence), and `hf_*` / `sk-*` / `Bearer …` token shapes redacted to `<redacted>` before write. Failures (disk full, serialisation errors) never crash the request handler.
+
 ## Changelog
 
 See [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases) for version history.

{soup_cli-0.40.2 → soup_cli-0.40.3}/README.md

@@ -43,17 +43,13 @@ soup train
 
 Latest highlights only. Full history: [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases).
 
-**v0.40.2 — Quick polish + carry-overs**: closes 3 originally-scheduled GitHub issues (#36 eval-gate dashboard row, #50 smarter `--hf-resume`, #51 custom HF Space templates) plus 7 v0.40.1 long-tail UX papercuts.
-
-- **`--hf-resume` prefers local newer** — `prepare_hf_resume` checks the highest local `checkpoint-N` against the highest remote branch and skips the snapshot download when the local copy is newer or equal. Saves bandwidth and never overwrites a fresher local checkpoint with stale Hub state.
-- **Eval-gate dashboard row** — pure formatter `format_gate_row(state)` lives in `soup_cli/monitoring/display.py` and renders a one-liner like `Gate: helpfulness 7.8 ✓ | math 0.82 ✗ (-0.06) | STOP` for the live training panel. Hidden when eval-gate is disabled.
-- **`soup deploy hf-space --template-dir <path>`** — supply your own `app.py` + `README.md` (+ optional `requirements.txt`) instead of the built-in `gradio-chat` / `streamlit-chat`. Containment-checked, repo-id substitution validated *before* render, 256 KB cap per file, symlinks rejected.
-- **`soup quickstart --output DIR`** — route data, config, and run dir under any directory you choose (default keeps backwards-compat in cwd).
-- **`soup runs --cwd-only`** — restrict the listing to runs whose `output_dir` is under the current directory; the global `~/.soup/experiments.db` view is still default.
-- **`soup infer` / `soup bench` accept HF ids** — when the local model path is missing AND the value isn't path-like (no `./`, `/`, `~`, `C:\`), it falls through to a HuggingFace download via `transformers.from_pretrained`. Path-like-but-missing surfaces a friendly `FileNotFoundError`.
-- **CLI flag aliases** — `data filter --min-coherence` (alias for `--coherence`); `data split --train` accepted (informational, train is the implicit remainder); `data register / unregister` accept positional `<name>` and `<path>` alongside the `--name` / `--path` options, with conflict detection.
-- **`--log-level` plumbing complete** — `apply_logging_level` now sets the root logger so third-party libraries (transformers / peft / trl) actually respect QUIET and DEBUG. The four tiers no longer produce byte-identical output.
-- **+36 net new tests** across the new helpers, security-fix follow-ups (path-containment in `register_data`, `bench.py`, `infer.py --output`, symlink-reject in custom Space templates), and Windows cross-drive edge cases.
+**v0.40.3 — Stub-to-live**: three v0.X.0 deferred-stub features become live runtime — closes #33 (data harvester judge filter + serve trace log), #64 (live CUDA OOM probe), #65 (multipack sampler in HF Trainer).
+
+- **Live CUDA batch-size probe** — `auto_batch_size_strategy: probe` now runs ONE forward+backward+step on a synthetic batch per candidate before training. On `torch.cuda.OutOfMemoryError` the probe halves; otherwise it doubles. Result is cached per `(model, max_length, quant, lora_r, gpu)` tuple so the next run short-circuits. CPU sessions skip the probe and fall back to the static estimate. SFT-only this release.
+- **Multipack sampler — helpers landed, live wiring deferred to v0.40.4** — adversarial review surfaced a HF Trainer DataLoader shape mismatch (`Sampler[int]` expected, `list[list[int]]` returned). Helpers (lru-cached subclass factory + state-attach with bounds + arch-detect + length-extract with all-zero warning) ship as a stub; live wiring requires a `get_train_dataloader` override and lands next patch.
+- **`soup data from-traces --judge`** — optional LLM-as-a-judge pass over harvested preference pairs. `--judge-provider openai|server|ollama`, `--judge-model gpt-4o-mini`, `--min-confidence 0.7`. Drops pairs whose normalised `(chosen - rejected)` confidence falls below threshold. Per-pair backend exceptions are counted, not crashed; lazy `itertools.islice` cap avoids buffering pathological generators.
+- **`soup serve --trace-log <path>`** — passive append-only JSONL request log (`{prompt, response, latency_ms, tokens, ts}` per chat completion). Path-containment validated, 100 MB rotation cap (one backup retained, symlink-reject on rotate), and `hf_*` / `sk-*` / `Bearer …` token shapes redacted to `<redacted>` before write (mirrors v0.34.0 `crash.py` policy).
+- **+95 net new tests** across the new closures, the dynamic Trainer subclass, the judge filter (including degenerate-scale + lazy-materialisation cases), and the trace logger (including symlink-backup rejection + multi-thread append safety).
 
 ## Why Soup?
 
@@ -570,7 +566,7 @@ training:
 
 **Architecture allowlist** — 18 supported (Llama 3.x, Qwen 2/3, Mistral, Gemma 2/3, Phi 3/4, DeepSeek V2/V3, Mixtral, Falcon, StableLM, SmolLM2). Unknown architectures **fail loudly at config-load** instead of silently no-opping (critical fix vs Axolotl's silent-miss footgun).
 
-**v0.37.0 scope:** schema gate + helper builder ship now. Live wiring of the sampler into HF Trainer's `_get_train_sampler` lands in v0.37.1 (mirrors v0.27.0 MII stub-then-live pattern). Multipack is **sft / pretrain only** on the `transformers` backend; preference / RLHF trainers and MLX backend get distinct error messages naming the actual reason.
+**Live wiring** — still deferred. v0.40.3 ships the helpers (`make_multipack_trainer_class` lru-cached factory + `attach_multipack_state` + `lengths_from_dataset` + `detect_arch_name`) but neither SFT nor Pretrain wrappers instantiate the subclass — adversarial review caught a `Sampler[int]` vs `list[list[int]]` mismatch with HF Trainer's DataLoader. Setting `multipack: true` prints a yellow advisory and falls back to the standard sampler. Live wiring (via a `get_train_dataloader` override with `batch_sampler=`) lands in v0.40.4. Multipack is **sft / pretrain only** on the `transformers` backend; preference / RLHF trainers and MLX backend still get distinct error messages naming the actual reason.
 
 **DoS hardening** — the FFD packer caps at 1M items (algorithm is O(N²) worst-case); the 4D mask builder caps allocations at 2³¹ cells; the chat-template Jinja analyzer caps at 128KB. Every numeric input rejects `bool` explicitly (matches v0.30.0+ project policy).
 
@@ -2488,6 +2484,7 @@ soup serve --model <m> --structured-output json --json-schema s.json  Constraine
 soup serve --model <m> --structured-output regex --regex-pattern '...'  Regex-constrained output
 soup serve --model <m> --dashboard            Live dashboard + /metrics endpoint
 soup serve --model <m> --trace --trace-endpoint http://localhost:4317  OpenTelemetry tracing
+soup serve --model <m> --trace-log ./serve.jsonl  Per-request JSONL log + rotation + secret redaction
 POST /v1/adapters/activate/<name>             Hot-swap active LoRA adapter
 soup sweep --config soup.yaml --param lr=...  Hyperparameter search
 soup diff --model-a ./a --model-b ./b         Compare two models
@@ -2503,7 +2500,9 @@ soup data generate ... --provider anthropic   Use Claude API
 soup data generate ... --provider vllm        Use local vLLM server
 soup data generate ... --template code        Domain templates (code/conversation/qa/preference/reasoning)
 soup data generate ... --quality-pipeline     Auto validate + filter + dedup
-soup data augment <path> --strategy rephrase|translate|style  LLM-driven augmentationsoup data from-traces --logs l.jsonl --format langchain --signal thumbs_up --output p.jsonl  Preference pairs from tracessoup data review prefs.jsonl --sample 10      Preview preference pairssoup data filter <path> --coherence 0.3       Quality filter (perplexity/coherence)
+soup data augment <path> --strategy rephrase|translate|style  LLM-driven augmentationsoup data from-traces --logs l.jsonl --format langchain --signal thumbs_up --output p.jsonl  Preference pairs from traces
+soup data from-traces ... --judge --min-confidence 0.7  LLM-judge confidence filter
+soup data review prefs.jsonl --sample 10      Preview preference pairssoup data filter <path> --coherence 0.3       Quality filter (perplexity/coherence)
 soup data sample <path> --n 1000             Random sample subset
 soup data sample <path> --n 1000 --strategy diverse  Cluster-based diverse sampling
 soup data sample <path> --n 1000 --strategy hard     Sample hardest examples
@@ -2700,6 +2699,44 @@ pytest tests/ -v
 pytest tests/ -m smoke -v
 ```
 
+## Live CUDA Batch-Size Probe
+
+Set `auto_batch_size_strategy: probe` in `training:` and Soup will run a real OOM-probe before training:
+
+```yaml
+training:
+  batch_size: auto
+  auto_batch_size_strategy: probe
+```
+
+For each candidate size `B`, the probe runs ONE forward + backward + step on a synthetic batch of `B` sequences of length `max_length`. On `torch.cuda.OutOfMemoryError` it halves; otherwise it doubles up to `4 × static_estimate`. The picked size is cached per `(model, max_length, quantization, lora_r, gpu)` tuple in `~/.soup/batch_cache.json` so subsequent runs skip the probe.
+
+CPU sessions and `auto_batch_size_strategy: static` skip the probe. Synthetic batch tensors are freed before the backward pass so peak VRAM reflects the realistic training step. SFT-only this release — non-SFT trainers fall back to the static estimate.
+
+## Trace-to-Preference: LLM-Judge Filter
+
+`soup data from-traces --judge` filters harvested preference pairs through an LLM judge:
+
+```bash
+soup data from-traces \
+  --logs ./prod-traces.jsonl --format langchain --signal thumbs_up \
+  --output ./prefs.jsonl \
+  --judge --judge-provider ollama --judge-model llama3 \
+  --min-confidence 0.7
+```
+
+The judge scores `chosen` and `rejected` independently against its rubric (default helpfulness/accuracy/safety on a 1-5 scale). Pairs whose normalised `(chosen - rejected)` confidence falls below `--min-confidence` are dropped. Per-pair backend exceptions are counted (not crashed) and reported. Provider allowlist `{openai, server, ollama}` validated at the CLI boundary; SSRF protection on `--judge-api-base` carries over from `soup eval judge`.
+
+## Inference Server Trace Log
+
+`soup serve --trace-log <path>` writes a passive append-only JSONL log per chat completion:
+
+```bash
+soup serve --model ./out --trace-log ./serve-trace.jsonl --trace-log-cap-mb 100
+```
+
+Each line: `{"ts": ..., "prompt": ..., "response": ..., "latency_ms": ..., "tokens": ...}`. Path-containment validated, hard rotation cap (default 100 MB, one backup retained), symlink-reject on the backup path (TOCTOU defence), and `hf_*` / `sk-*` / `Bearer …` token shapes redacted to `<redacted>` before write. Failures (disk full, serialisation errors) never crash the request handler.
+
 ## Changelog
 
 See [GitHub Releases](https://github.com/MakazhanAlpamys/Soup/releases) for version history.

{soup_cli-0.40.2 → soup_cli-0.40.3}/SECURITY.md

@@ -9,7 +9,7 @@ We provide security updates for the following versions:
 - **Versions older than 3 minor versions:** No support
 
 Example:
-- v0.40.2 -- Full support (latest)
+- v0.40.3 -- Full support (latest)
 - v0.40.0-v0.40.x -- Full support
 - v0.39.0-0.39.x -- Bug-fix support only
 - v0.38.0-0.38.x -- Bug-fix support only
@@ -144,6 +144,7 @@ No known critical vulnerabilities in current releases.
 - **v0.32.0 — Training Stability & Auto-Tuning**: `--find-lr-output` containment via shared `utils/paths.is_under_cwd` (prevents writes outside cwd); `save_lr_finder_report` rejects NaN / Infinity floats in `lrs` / `losses` and serialises with `allow_nan=False` (keeps the report parser-safe); `compute_lr_schedule` rejects non-positive `start_lr`, inverted ranges, and `num_steps` outside `[2, 10_000]`; `pick_mixed_precision` rejects empty / null-byte / >200-char model names and resolves multi-version quirks (`qwen2.5` vs `qwen2`, `phi-3.5` vs `phi-3`) by longest-substring-first iteration so an added family can never accidentally make a more-specific entry dead code; `compute_warmup_steps` clamps to `[10, 1000]` with a `ratio==0.0` short-circuit matching HF Trainer's "no warmup" convention; `SpikeRecoveryStrategy` is `@dataclass(frozen=True)` (post-construction mutation cannot bypass validation), `max_attempts ∈ [1, 10]`, `lr_decay ∈ (0, 1)`, `min_lr > 0`; cross-validator `_validate_spike_recovery_requires_watchdog` rejects `loss_spike_recovery=true, loss_watchdog=false` at config-load (fails fast instead of never triggering); `convergence_window ∈ [5, 10_000]`, `convergence_rel_tol ∈ (0, 1]`, `recommend_action` reuses `detect_plateau` so plateau heuristic stays single-source-of-truth; `GradAccumMonitor.recommend()` caps doubled `accum` at `MAX_ACCUM=1024` so a runaway advisory loop cannot blow up DataLoader prefetch; `generate_config` validates BOTH the YAML output path AND the embedded `decisions["output"]` field via `is_under_cwd` (closes the gap where a crafted `decisions["output"]="../../etc"` would have silently propagated into the rendered YAML)
 - **v0.34.0 — Observability & Dev UX**: `.crash` bundle generator (`utils/crash.py`) recursively redacts `hf_*` / `sk-*` / `Bearer …` token-shaped strings in any captured `config` and metric tail before serialisation, so a `.crash` file shared on a public GitHub issue cannot leak credentials; `output_dir` is reduced to `os.path.basename` so `$HOME` doesn't leak; `write_crash_bundle` uses `os.path.realpath + commonpath` for cwd containment (Windows-safe; raises `ValueError` not `PermissionError` so callers cannot silently swallow with `except OSError`); filename appends `secrets.token_hex(4)` so two crashes in the same UTC second don't collide; bundle truncated to `MAX_BUNDLE_BYTES=1_000_000`. `train.py` crash-write surfaces failures to the user (no silent missing-bundle). `profiling.py` `resolve_trace_path` rejects empty / `.` / `..` / `/` / `\\` / null-byte `run_id` (closes the `output_dir/profiles/../trace.json` escape) and uses `os.path.realpath + is_under_cwd`; profiles dir is created only on successful torch import (no stale empty dirs on torch-less CI). `tracker.get_run` LIKE-prefix match escapes `%` / `_` / `\\` and uses `ESCAPE '\\'` so a crafted `run_id` cannot widen the match (mirrors v0.26.0 registry policy). Lazy schema migration (`_ensure_schema`) tolerates the "duplicate column" race when two CLI processes start simultaneously on a fresh DB (fork-based multi-GPU training, TUI auto-refresh). `runs.py show/replay/clean` switched user `run_id` rendering to `markup_escape` and switched `clean` containment from broken `Path.resolve() + relative_to()` to project-standard `os.path.realpath + is_under_cwd`. `tui_app.py` lazy-imports `ExperimentTracker` and `markup_escape`s every DB-sourced string before passing into Textual widgets so a crafted base_model / experiment_name cannot inject `[bold red]…[/]` markup. `run_cost.estimate_run_cost_usd` rejects `bool` in `num_gpus` (bool is a subclass of int — same defence as v0.30.0 `Candidate.__post_init__`); duration clamped to `[0, 1 year]`; unknown GPU returns `None` so callers render `—` instead of fabricating `$0.00`. `log_level.parse_log_level` rejects non-string + null-byte input.
 - **v0.33.0 — Live Wire**: RLVR `code_exec_reward` adds OS-level isolation (Linux best-effort `os.unshare(CLONE_NEWUSER|CLONE_NEWNET|CLONE_NEWPID)`, macOS `sandbox-exec` with default-deny `MACOS_SANDBOX_PROFILE` narrowed to a 3-name `mach-lookup` allowlist to prevent DNS / NSURLSession bypass of `(deny network*)`); `prune_checkpoints` switches to TOCTOU-safe `os.lstat + S_ISLNK` + `shutil.rmtree(onerror=_abort_on_symlink)` so a symlink encountered mid-walk aborts rather than escapes; `run_gate` wraps each task scorer in a typed `try/except` so backend failures produce `score=None, error=str(exc)` (never silent `score=1.0`); `_parse_judge_url` removes the bare `http://` catch-all (defence-in-depth after the Pydantic GateTask validator); `soup can run` requires `--yes` or explicit consent callback and raises `ValueError` (not `PermissionError`, which is an `OSError` subclass that broad `except` blocks would swallow); GGUF `rglob` result for ollama deploy is `realpath+commonpath` checked against extract_dir (prevents symlink escape from a crafted can); `DeployTarget.path` validator normalises mixed `\\`/`/` separators before splitting (closes a Windows `..` bypass); `CAN_FORMAT_VERSION` 1→2 (additive — v1 still loads); `soup can publish` validates `repo_id` via `utils/hf.validate_repo_id`, resolves token via `resolve_token`, sanitises commit messages (first-line, 200-char cap), uses HTTPS-only HfApi; `_write_spike_recovery_hint` adds `is_under_cwd` containment check on `args.output_dir` from raw HF `TrainingArguments`; `lookup_entry_by_output_dir` emits `ResourceWarning` when 1000-row scan limit is hit (no silent miss); `CrossDocCollator` no longer mutates input feature dicts (HF Dataset rows are cached and reused — mutation broke subsequent batches); `Candidate` rejects `bool` in `score`/`latency_ms` (was sneaking past `int` isinstance check); `evaluate_candidate` latency mean now divides by *completed* prompts (excludes crashed) so a broken candidate isn't artificially fast; `auto_quant.run_auto_quant_picker` soft-falls-back to highest-scored candidate when no candidate clears `min_score` (server still binds); `build_logits_processors` returns `[]` when neither `outlines` nor `lm-format-enforcer` is installed (server degrades to free-form rather than 500); MII server uses loopback-only CORS, max_tokens cap [1, 16384], stream rejection, generic 500 with no stack-trace leak; `os.execvp` auto-reexec uses list args (no shell), all forwarded flags pre-validated; `cleanup_extract_dir` uses `os.path.commonpath` (Windows-safe) instead of `startswith`; `_run_subprocess` catches `TimeoutExpired` and returns rc=124 (coreutils convention) instead of an unhandled traceback; new `eval_results` and `tensorrt` artifact kinds in `RegistryStore._VALID_KINDS`
+- **v0.40.3 — Stub-to-live**: New `soup_cli/utils/batch_probe.py:make_cuda_probe_fn` builds a CUDA probe closure that runs ONE forward+backward+step on a synthetic batch per candidate; `model.zero_grad(set_to_none=True)` runs BEFORE forward (defends against the synthetic backward accumulating into the live training model's grad buffers — matches v0.35.0 #45 `benchmark_kernel_combos` policy); intermediate `ids/attn/labels/outputs` are `del`-ed before `loss.backward()` so peak VRAM reflects a realistic training step; `bool` rejected on `batch_size` and `max_length`; `max_length < 8` rejected; `torch.cuda.OutOfMemoryError` returns False, other exceptions propagate; returns `None` (no-op) on non-CUDA / no-torch / missing model or tokenizer. New `soup_cli/utils/multipack_trainer.py:make_multipack_trainer_class` is `lru_cache`d so two calls with the same `base_cls` return the same subclass (consistent `isinstance`, pickle-safe); `attach_multipack_state` rejects `bool` on `max_seq_len`/`batch_size`/`seed` and rejects empty `lengths`; `lengths_from_dataset` logs WARNING when every row produces 0 (loud-fail mirrors v0.37.0 multipack arch allowlist — prevents silent NaN-loss footgun); `_get_train_sampler` override accepts `*args, **kwargs` for HF >= 4.41 signature compat. **Live wiring of the sampler into SFT / Pretrain trainer wrappers is deferred to v0.40.4** — adversarial 5th-pass review surfaced a `Sampler[int]` vs `list[list[int]]` shape mismatch with HF Trainer's DataLoader; the wrappers currently print a yellow advisory and fall back to the standard sampler when `multipack: true`. New `soup_cli/data/traces/quality.py:judge_filter_pairs` reuses v0.19.0 `JudgeEvaluator` SSRF protections; threshold rejects `bool` / NaN / out-of-`[0,1]`; `_MAX_BATCH=100_000` cap applied via lazy `itertools.islice` (never fully materialises a malicious / pathological generator); per-pair backend exceptions caught and logged at DEBUG (matches v0.33.0 #47 `CrossDocCollator` policy — never silently crash the harvest); `judge_provider` validated against `VALID_PROVIDERS` allowlist at the CLI boundary BEFORE constructor, with a Rich-escape error on mismatch. New `soup_cli/monitoring/trace_logger.py:TraceLogWriter` is thread-safe (single `threading.Lock` — multi-worker `--workers 4` documented as a single-process limitation); path containment via shared `is_under_cwd`; null-byte / empty / non-string path rejected; `cap_mb` bounds `[1, 10000]` with explicit `bool` rejection; rotation: when `current + extra > cap_bytes`, rename to `<path>.1` (one backup retained); symlink at the backup path is rejected via `os.lstat + stat.S_ISLNK` (matches v0.33.0 #22 TOCTOU policy) — defends against pre-placed `<log>.1 -> /etc/cron.d/x` overwrite. Secret redaction: prompt + response strings passed through `_SECRET_RE` matching `hf_*` (≥8), `sk-*` (≥16), and `Bearer …` (≥8) — replaces matches with `<redacted>` before serialisation (mirrors v0.34.0 `crash.py` policy). `--trace-log` constructor error messages in `commands/serve.py` are `rich.markup.escape`d before printing so a crafted path name cannot inject Rich markup. Unserialisable entries dropped silently; disk-full / OSError on write never crashes the request handler (passive log). Known limitations: live CUDA probe is wired in SFT only; multipack live wiring covers SFT+Pretrain only; `TraceLogWriter` retains exactly ONE backup file (operators wanting longer retention should use external rotation); custom HF Space templates from v0.40.2 still always create the Space with `space_sdk="gradio"` (tracked for v0.40.4+).
 - **v0.40.2 — Quick polish + carry-overs**: New `soup_cli/utils/hf_space.py:render_custom_template_dir` enforces `is_under_cwd` containment on the template directory; `validate_repo_id` runs BEFORE `{MODEL_REPO}` substitution (matches v0.29.0 Part F policy); per-file 256 KB cap (matches v0.39.0 Part E template-size policy); only `app.py` / `README.md` / `requirements.txt` are read (closed allowlist — no path-from-user-data). Symlinks rejected via `os.lstat + stat.S_ISLNK` and non-regular files (FIFO / device) also rejected (matches v0.33.0 #22 prune_checkpoints TOCTOU policy) — defends against `<template_dir>/app.py -> /etc/passwd`. `_find_highest_local_checkpoint` reads `output_dir` after caller's `is_under_cwd` validation (in `prepare_hf_resume`) and silently drops non-directories + OSError. `prepare_hf_resume` skips the snapshot download when local `checkpoint-N >= remote checkpoint-N` (saves bandwidth and never overwrites a fresher local checkpoint). `commands/data.py:register_data` containment switched from `Path.resolve() + relative_to()` to shared `is_under_cwd` (Windows 8.3 short-name safety per CLAUDE.md project rule); same fix applied to `commands/bench.py` prompts-file containment. `commands/infer.py:--output` now containment-checked via `is_under_cwd` (late-evaluated after model+input validation so pre-existing `tmp_path` test contracts keep working). `commands/quickstart.py:--output` validates target dir via `is_under_cwd` before `mkdir(parents=True)`; rejects out-of-cwd targets with friendly message. `commands/runs.py:_filter_runs_by_cwd` uses `os.path.realpath + commonpath`, catches `(ValueError, OSError)` so cross-drive paths on Windows (`D:\runs` vs `C:\project`) drop silently rather than crash. `monitoring/display.py:format_gate_row` uses explicit `task.get("passed") is True` so a missing `"passed"` field renders neutrally instead of as a false-y red ✗. `commands/infer.py:_resolve_model_source` heuristic for HF-id-vs-local-path: only falls through to HF when value is NOT path-like (no `./`, `/`, `\\`, `~`, no Windows drive letter, non-empty); path-like-but-missing raises `FileNotFoundError` so users see actionable errors instead of confusing HF download attempts. Known limitation: custom HF Space templates always create the Space with `space_sdk="gradio"` regardless of the supplied `app.py` (no `--sdk` flag in this release; combine `--template streamlit-chat` with the inline registry for Streamlit Spaces). Tracked for v0.40.3+.
 - **v0.40.1 — QA Hardening**: `soup_cli/utils/encoding.force_utf8_stdio` reconfigures Windows stdout/stderr to UTF-8 before any Rich Console is constructed; `os.environ.setdefault("PYTHONIOENCODING", "utf-8")` preserves user override; `(OSError, ValueError, AttributeError)` swallowed on detached streams; POSIX no-op. `SoupConfig._remap_root_level_misplaced_keys` (model_validator, mode='before') migrates root-level `lora:` into `training.lora` so nested validators (including `lora.init_strategy: Literal["random","pissa","olora"]`) actually fire — closes a footgun where the misplaced key was silently dropped. Caller's dict is never mutated (shallow-copy policy mirroring v0.33.0 #47 / v0.40.0 Part B). `PreferenceTrainerWrapper._build_multi_objective` replaces the v0.40.0 `NotImplementedError` stub with a primary-loss approximation; `validate_weight_compat` rejects BCO mixed with paired losses at runtime (data-format incompatible). `combine_losses` rejects empty weights, propagates NaN loudly (no silent zeroing), and rejects `bool` weight values (matches v0.30.0 `Candidate` policy). `_probe_cache_param_count` rejects empty / null-byte model names before path construction (mirrors v0.26.0 registry / v0.39.0 ReLoRAPolicy policy). `commands/doctor` flags `transformers ≥ 5.0.0` as INCOMPATIBLE via `_MAX_EXCLUSIVE` table; `_version_ge` parses leading-int chunks so `5.0.0.dev0` correctly trips the cap. `_detect_gpu_hw_without_torch_cuda` calls `nvidia-smi` via argv list (no shell), 5s timeout, `OSError` / `TimeoutExpired` caught; GPU label from `nvidia-smi` stdout is `rich.markup.escape`d before embedding in Rich-markup string (a real GPU name like `NVIDIA Quadro [T4]` cannot break or inject markup). `_detect_dual_python_interpreters` uses `os.path.realpath` (not `Path.resolve()`) for Windows 8.3 short-name compat. `_pick_quickstart_model` swaps TinyLlama-1.1B → SmolLM2-135M when `total_memory ≤ 6 GB` (prevents step-0 OOM on RTX 3050 4 GB / similar). `_live_lr_sweep_from_config` switched broken `load_local` import to `load_raw_data` (previously always silently fell back to a static placeholder curve). `commands/migrate` rejects `.jsonl` input (with first-line `{` sniff) with exit-2 friendly error; `.jsonl`-only suffix gate prevents false-positives on `.ipynb` notebooks. `commands/eval custom -o` is now honored independently of `--attach-to-registry`; loop-shadow regression where `output = generate_fn(...)` overwrote the CLI option fixed (variable renamed to `response`). `_load_jsonl` switched from `utf-8` to `utf-8-sig` so PowerShell `Out-File -Encoding utf8`-produced JSONL no longer fails first-row parse. Known limitation: `--trust-remote-code` opt-in surface still excludes 10 non-SFT trainers + 5 commands (v0.36.0 #63 carry-over).
 - **v0.40.0 — Preference Variety**: New `task='bco'` (Binary Classifier Optimization) and `task='preference'` (unified dispatcher). New schema fields: `bco_beta` (gt=0), `preference_loss: Literal[dpo,simpo,orpo,ipo,bco]|None`, `preference_loss_weights: Optional[Dict[str,float]]`, `dpo_beta_schedule: Literal[linear,cosine,exponential]|None`, `dpo_beta_end: float, gt=0|None`, `dpo_ref_regen_epochs: int [1,1000]|None`. Cross-validators: `_validate_preference_dispatcher` rejects setting either `preference_loss` or `preference_loss_weights` outside `task='preference'` (closes ordering-dependency between Part B/D validators); `_validate_dpo_variants_supported_tasks` gates β-schedule + ref-regen to DPO-family tasks (`dpo`, `ipo`, or `preference` + `preference_loss in {dpo, ipo}`); rejected on mlx backend with distinct error message (matches v0.34.0 distinct-reason policy); `_validate_preference_loss_weights` enforces 2–5 entries (single-entry rejected with actionable message pointing at scalar `preference_loss`), key allowlist `{dpo, simpo, orpo, ipo, bco}`, explicit null-byte rejection on keys (matches v0.39.0 rank_pattern policy), per-value bounds `(0, 1]`, weights must sum to 1.0 (±1e-6), mutually exclusive with scalar `preference_loss`, rejected on mlx backend. `compute_beta_at_step` rejects `bool` on `step` and `total_steps` (project bool-as-int policy from v0.30.0). `BetaScheduleCallback` resolves `total_steps` lazily in `on_train_begin` so the schedule sees the real `state.max_steps` populated by HF Trainer (closes a first-cut silent-no-op bug where total_steps=0 emitted beta_end for every step). `RefModelRegenCallback._regenerate` uses `strict=True` on `load_state_dict` and logs at WARNING on mismatch (closes a first-cut silent partial-copy hazard where strict=False could produce a hybrid old-base + new-LoRA reference); epoch 0 regen suppressed (avoids copying untrained student); trainer `.beta` assignment swallow narrowed to `AttributeError` only. `PreferenceTrainerWrapper._make_inner_cfg` uses `model_copy` (not `model_dump`+`model_validate`) so re-validation never sees an inconsistent intermediate state and the caller's `cfg` is never mutated (mirrors v0.33.0 #47 immutability policy). `_split_dpo_rows_to_bco` skipped-row count emitted at DEBUG so production silent-degradation is inspectable (mirrors v0.33.0 #47 CrossDocCollator policy). Multi-objective live runtime weighted-loss combination is deferred to v0.40.1: `PreferenceTrainerWrapper.setup` raises `NotImplementedError` with a friendly message naming the deferred-version follow-up (mirrors v0.27.0 MII / v0.37.0 multipack / v0.38.0 quant menu / v0.39.0 ReLoRA stub-then-live pattern). Known limitation: `BCOTrainerWrapper._setup_transformers` still hardcodes `trust_remote_code=True` (v0.36.0 #63 known-gap family carry-over across non-SFT trainers).

{soup_cli-0.40.2 → soup_cli-0.40.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "soup-cli"
-version = "0.40.2"
+version = "0.40.3"
 description = "Fine-tune LLMs in one command. No SSH, no config hell."
 readme = "README.md"
 license = "Apache-2.0"

{soup_cli-0.40.2 → soup_cli-0.40.3}/soup_cli/__init__.py

@@ -1,3 +1,3 @@
 """Soup CLI — Fine-tune LLMs in one command."""
 
-__version__ = "0.40.2"
+__version__ = "0.40.3"

{soup_cli-0.40.2 → soup_cli-0.40.3}/soup_cli/commands/data.py

@@ -1586,6 +1586,26 @@ def from_traces_cmd(
         "prefs.jsonl", "--output", "-o",
         help="Output path for preference pairs (JSONL)",
     ),
+    judge: bool = typer.Option(
+        False, "--judge",
+        help="Filter pairs via LLM-as-a-judge confidence (v0.40.3 #33).",
+    ),
+    judge_provider: str = typer.Option(
+        "openai", "--judge-provider",
+        help="Judge backend: openai | server | ollama. Used with --judge.",
+    ),
+    judge_model: str = typer.Option(
+        "gpt-4o-mini", "--judge-model",
+        help="Judge model id (e.g. 'gpt-4o-mini', 'llama3', 'qwen2.5'). Used with --judge.",
+    ),
+    judge_api_base: Optional[str] = typer.Option(
+        None, "--judge-api-base",
+        help="Judge API base URL. SSRF-protected. Used with --judge.",
+    ),
+    min_confidence: float = typer.Option(
+        0.7, "--min-confidence",
+        help="Drop pairs with judge-confidence below this threshold (0.0 - 1.0).",
+    ),
 ) -> None:
     """Harvest preference pairs from production traces (v0.26.0 Part C).
 
@@ -1666,6 +1686,53 @@ def from_traces_cmd(
             trace_iter = parse_openai(events)
 
     pairs = list(build_pairs(trace_iter, signal=signal))
+
+    if judge:
+        # v0.40.3 (#33 (a)) — LLM-judge confidence filter.
+        from soup_cli.data.traces.quality import judge_filter_pairs
+        from soup_cli.eval.judge import VALID_PROVIDERS, JudgeEvaluator
+
+        # Friendly early validation matches the existing CLI conventions —
+        # fall through to the constructor only after the obvious typo is caught.
+        if judge_provider not in VALID_PROVIDERS:
+            console.print(
+                f"[red]--judge-provider '{_escape(judge_provider)}' is invalid. "
+                f"Choose: {', '.join(sorted(VALID_PROVIDERS))}[/]"
+            )
+            raise typer.Exit(1)
+
+        try:
+            judge_evaluator = JudgeEvaluator(
+                provider=judge_provider,
+                model=judge_model,
+                api_base=judge_api_base,
+            )
+        except ValueError as exc:
+            console.print(f"[red]--judge config error:[/] {_escape(str(exc))}")
+            raise typer.Exit(1) from exc
+
+        # Cost-shock warning: each pair → TWO judge calls (chosen + rejected).
+        projected = len(pairs) * 2
+        console.print(
+            f"[yellow]Judge filter will issue ~{projected} backend calls "
+            f"({len(pairs)} pairs × 2). Cost depends on provider and model. "
+            f"Use --min-confidence to tune throughput.[/]"
+        )
+
+        try:
+            filtered, report = judge_filter_pairs(
+                pairs, judge=judge_evaluator, min_confidence=min_confidence,
+            )
+        except (TypeError, ValueError) as exc:
+            console.print(f"[red]--judge runtime error:[/] {_escape(str(exc))}")
+            raise typer.Exit(1) from exc
+
+        console.print(
+            f"[green]Judge filter:[/] kept={report.kept} dropped={report.dropped} "
+            f"errors={report.errors} (min_confidence={min_confidence:.2f})"
+        )
+        pairs = filtered
+
     with output_path.open("w", encoding="utf-8") as fh:
         for pair in pairs:
             fh.write(_json.dumps(pair.to_jsonl_dict(), ensure_ascii=False) + "\n")

{soup_cli-0.40.2 → soup_cli-0.40.3}/soup_cli/commands/serve.py

@@ -174,6 +174,20 @@ def serve(
             "Default deny (v0.36.0). Only enable if you trust the source."
         ),
     ),
+    trace_log: Optional[str] = typer.Option(
+        None,
+        "--trace-log",
+        help=(
+            "Append per-request {prompt, response, latency_ms, tokens, ts} "
+            "to JSONL at this path. Path must stay under cwd. Rotates at "
+            "100 MB (one backup retained). Added in v0.40.3 (#33)."
+        ),
+    ),
+    trace_log_cap_mb: int = typer.Option(
+        100,
+        "--trace-log-cap-mb",
+        help="Rotation cap in MB for --trace-log (1 - 10000). Default 100.",
+    ),
 ):
     """Start a local inference server with OpenAI-compatible API."""
     # Lazy imports for fast CLI startup
@@ -567,6 +581,25 @@ def serve(
 
         tracer = build_tracer(enabled=trace, endpoint=trace_endpoint)
 
+        # v0.40.3 (#33 (b)) — passive request log.
+        trace_log_writer = None
+        if trace_log is not None:
+            from soup_cli.monitoring.trace_logger import TraceLogWriter
+
+            try:
+                trace_log_writer = TraceLogWriter(
+                    trace_log, cap_mb=trace_log_cap_mb,
+                )
+            except (TypeError, ValueError) as exc:
+                from rich.markup import escape as _escape
+
+                console.print(f"[red]--trace-log:[/] {_escape(str(exc))}")
+                raise typer.Exit(1) from exc
+            console.print(
+                f"[green]Request trace log:[/] {trace_log_writer.path} "
+                f"(cap {trace_log_cap_mb} MB)"
+            )
+
         app = _create_app(
             model_obj=model_obj,
             tokenizer=tokenizer,
@@ -579,6 +612,7 @@ def serve(
             output_constraint=constraint,
             enable_dashboard=dashboard,
             tracer=tracer,
+            trace_log_writer=trace_log_writer,
         )
 
     console.print(
@@ -844,6 +878,7 @@ def _create_app(
     output_constraint: Optional[Dict] = None,
     enable_dashboard: bool = False,
     tracer=None,
+    trace_log_writer=None,
 ):
     """Create the FastAPI application with OpenAI-compatible endpoints."""
     import threading as _threading
@@ -987,6 +1022,7 @@ def _create_app(
         max_tokens = request.max_tokens or max_tokens_default
 
         if request.stream:
+            stream_started = time.perf_counter()
             return StreamingResponse(
                 _stream_response(
                     model_obj, tokenizer, messages,
@@ -996,6 +1032,8 @@ def _create_app(
                     model_name=model_name,
                     assistant_model=draft_model,
                     num_assistant_tokens=num_speculative_tokens,
+                    trace_log_writer=trace_log_writer,
+                    started=stream_started,
                 ),
                 media_type="text/event-stream",
             )
@@ -1042,6 +1080,21 @@ def _create_app(
                 # returns an empty list and generation runs free-form.
                 pass
 
+                # v0.40.3 (#33 (b)) — passive request log; never breaks
+                # the request handler on disk / serialisation issues.
+                if trace_log_writer is not None:
+                    last_user = next(
+                        (m["content"] for m in reversed(messages)
+                         if m.get("role") == "user"),
+                        "",
+                    )
+                    trace_log_writer.record(
+                        prompt=str(last_user),
+                        response=response_text,
+                        latency_ms=(time.perf_counter() - started) * 1000,
+                        tokens=completion_tokens,
+                    )
+
                 return {
                     "id": f"chatcmpl-{uuid.uuid4().hex[:8]}",
                     "object": "chat.completion",
@@ -1071,6 +1124,7 @@ def _create_app(
     # Expose dashboard intent + constraint on the app for tests + introspection
     app.state.enable_dashboard = enable_dashboard
     app.state.output_constraint = output_constraint
+    app.state.trace_log_writer = trace_log_writer
     return app
 
 
@@ -1078,14 +1132,16 @@ def _stream_response(
     model, tokenizer, messages,
     max_tokens, temperature, top_p, model_name,
     assistant_model=None, num_assistant_tokens=5,
+    trace_log_writer=None, started=None,
 ):
     """Generator that yields SSE chunks for streaming responses."""
     chat_id = f"chatcmpl-{uuid.uuid4().hex[:8]}"
     created = int(time.time())
 
     # Generate full response (true token-by-token streaming requires TextIteratorStreamer)
+    completion_tokens_for_log = 0
     try:
-        response_text, _, _ = _generate_response(
+        response_text, _, completion_tokens_for_log = _generate_response(
             model, tokenizer, messages,
             max_tokens=max_tokens,
             temperature=temperature,
@@ -1133,3 +1189,24 @@ def _stream_response(
     }
     yield f"data: {json.dumps(final_chunk)}\n\n"
     yield "data: [DONE]\n\n"
+
+    # v0.40.3 (#33 (b)) — passive request log on the streaming path. Latency
+    # measured from the BEFORE-`_generate_response` mark passed in by the
+    # chat_completions handler. Skipped if writer is None or `started` is
+    # missing. Errors swallowed (passive log).
+    if trace_log_writer is not None and started is not None:
+        try:
+            last_user = next(
+                (m["content"] for m in reversed(messages)
+                 if m.get("role") == "user"),
+                "",
+            )
+            trace_log_writer.record(
+                prompt=str(last_user),
+                response=response_text,
+                latency_ms=(time.perf_counter() - started) * 1000,
+                tokens=int(completion_tokens_for_log),
+                extra={"stream": True},
+            )
+        except Exception:  # noqa: BLE001 — passive log never blocks SSE
+            logger.debug("trace_log streaming record failed", exc_info=True)