mlx-bun 0.0.4 → 0.0.6

package/docs/investigations/orpo-uf-testing-handoff.md

@@ -0,0 +1,126 @@
+# ORPO-on-UF — results + testing-phase handoff
+
+**Status:** training done (2026-06-20); **testing phase = next.** This doc is the
+"where we are + what to run" so anyone (or a fresh session) can execute the measurements.
+Companion: [trainer-validation-experiment.md](trainer-validation-experiment.md) (the
+validation that got us here).
+
+## The run that just finished
+
+From-**base** ORPO on UltraFeedback, MiniCPM5-1B.
+- **Base:** `mlx-community/MiniCPM5-1B-OptiQ-4bit` (the full post-trained model; optiq quant).
+- **Data:** `~/.cache/mlx-bun/eval-data/uf-cpm5` — 14,691 train / 148 val, `{prompt,chosen,rejected}`.
+- **Config:** ORPO, **1 epoch (14,691 iters)**, B=1, seq 2048, rank 16, **λ=0.1**, cosine LR
+  1e-5, **no-segment**. From base (no warm-start).
+- **Command:** `SEQ=2048 EVAL_EVERY=500 RANK=16 SCHED=cosine LAMBDA=0.1
+  ADAPTER_DIR=./adapters/cpm5-uf-frombase bun scripts/experiments/pref-control.ts orpo
+  ~/.cache/mlx-bun/eval-data/uf-cpm5 uf-frombase 14691 1e-5`
+
+## Key result — the two ORPO halves decoupled
+
+| half | metric | best | at step | final (14690) |
+|---|---|---|---|---|
+| **SFT** | loss (NLL+λOR) ↓ | **1.5057** | **14500** (the end) | 1.506 |
+| **DPO** | margin ↑ | **0.143** | **6000** | 0.127 |
+| — | accuracy | **81/148** | ~5500 | 80/148 |
+
+- **SFT improved across the *whole* run** (loss bottomed at the end) → **final adapter = best SFT.**
+- **DPO peaked at ~6000 and faded** → **step ~6000 = best DPO**, slightly worse SFT (loss 1.5077).
+- **No single checkpoint has both.** Honest summary: a **small, early-peaking preference
+  signal** (~+6 questions, chance 74 → peak 81/148) that **drifted down in the tail**, while
+  SFT kept inching to its ~1.506 floor. Whether even that small lift **converts** on an
+  independent test is the open question the testing phase answers.
+
+## Artifacts + baselines
+
+- **Final adapter:** `./adapters/cpm5-uf-frombase/` (mountable: `eval.ts --adapter`).
+- **Trajectory:** `runs/uf-frombase.{json,csv}`; **plot:** `runs/uf-frombase.png` via
+  `scripts/experiments/plot-runs.py` (panels: loss=val+train-EMA; questions-correct/148;
+  margin — each zoomed to its band).
+- **x1 baseline** (base CPM5, full capability suite): **28.69** (in the eval DB; needs the
+  MMLU-argmax + HumanEval-python fixes — see Gotchas). optiq published is 30.28.
+- **⚠ No intermediate checkpoints this run** — `pref-control.ts` lacked `saveCheckpoints`
+  (now **fixed**; future runs checkpoint every val eval + track best-margin). The ~6k peak
+  is recoverable by **deterministic replay** (same seed/order → same adapter).
+
+## Testing plan
+
+**PRIMARY — the headline (READY TO RUN):**
+- **A. final adapter vs base** on x2 (capability) + Exp 2 (judge win-rate). Answers "did a
+  full epoch of ORPO-on-UF move a 1B, on an *independent* test?"
+
+**SWEEP / SYNTHESIS:**
+- **B. recover the ~6k peak** — re-run from base to ~7k (checkpoints now on), grab the
+  step-6000 (and neighbors, pick best-val). The early-stop peak arm.
+- **C. four-way** — base / 6k-peak / final / final+6k on x2 + Exp 2 → traces the
+  early-stopping curve on an independent test.
+- **D. DPO-boost (the day's synthesis) — keep best SFT, re-pressure the DPO.** Warm-start
+  from the **final** adapter (best SFT, NLL at floor), re-apply the **DPO-helpful examples
+  with HIGH λ** (or DPO-only). = dynamic-λ (ramp preference at SFT-saturation) + data-curation
+  in one move. Tests the fork: DPO stalled because of a **capacity ceiling** (won't help) or
+  **dilution / λ-too-gentle** (margin recovers past 0.143 while loss holds ~1.506 → better on
+  *both* axes than any single checkpoint).
+- **E. data-valuation control** — `final + (0–6000)` vs `final + (6000–14691)`, same size.
+  First-half wins → those examples carry the DPO signal; tie → it's "more preference training,"
+  not *which* examples. (We control which pairs go in, so this generalizes to **curating** the
+  data with the model as the probe.)
+
+## Commands
+
+`ORC=/Users/joshrossi/Code/mlx-lm-example/.venv/bin/python` (HumanEval interpreter, this machine).
+
+```bash
+# A — x2 capability (final vs base; base x1=28.69 already recorded)
+MLX_BUN_ORACLE_PYTHON=$ORC bun scripts/eval.ts capability --candidate MiniCPM5 --adapter ./adapters/cpm5-uf-frombase
+# A — Exp 2 (gen base+trained responses, then judge head-to-head)
+bun scripts/experiments/prefbench.ts gen mtbench ./adapters/cpm5-uf-frombase mtbench-uf   # GPU
+bun scripts/experiments/prefbench.ts judge mtbench-uf                                      # codex
+# B — replay base→7k to recover the ~6k checkpoint (now checkpoints)
+SEQ=2048 EVAL_EVERY=500 RANK=16 SCHED=cosine LAMBDA=0.1 ADAPTER_DIR=./adapters/cpm5-uf-replay \
+  bun scripts/experiments/pref-control.ts orpo ~/.cache/mlx-bun/eval-data/uf-cpm5 uf-replay 7000 1e-5
+```
+
+## Harness gaps to close for D / E
+
+`pref-control.ts` does **not yet** support (needed for the DPO-boost + control arms):
+1. **Warm-start from an adapter** (`--resume` / load a non-zero adapter before training) —
+   the trainer has `warm_start_adapter` (see `trainer.ts`); just not wired into pref-control.
+2. **Data-subset selection** (feed a chosen subset / order of pairs, e.g. rows 0–6000) —
+   pref-control feeds the whole `train.jsonl`; needs a row-range / id-list option.
+3. λ is already env-controllable (`LAMBDA=`), so high-λ is ready once 1+2 exist.
+
+## Validated infrastructure (all green — see trainer-validation-experiment.md)
+
+- **Trainer:** positive controls pass — DPO & ORPO drive prefer-uppercase val acc → 1.0;
+  grads match autograd. `scripts/experiments/pref-control.ts`.
+- **Capability harness:** reproduces optiq across 6 tasks after two fixes (below).
+  `eval.ts capability --adapter` wired.
+- **Judge:** `gpt-5.4-mini` via `codex exec`, **keyless**, structured, order-swapped,
+  3/3 positive control. `scripts/experiments/llm-judge.ts`.
+- **Exp 2 harness:** `scripts/experiments/prefbench.ts` (gen/judge); prompts downloaded
+  (`mt_bench.jsonl` 80, `alpaca_eval.json` 805).
+- **MMLU frozen set:** optiq's exact 969 questions + dev exemplars captured
+  (`mmlu_optiq_frozen.jsonl` / `mmlu_optiq_dev.jsonl`) for parity work.
+
+## Gotchas / load-bearing facts
+
+- **`AdapterManager.mount` affects the *serving* forward** (eval generation, chat) **but NOT
+  the *training* `branchLogpMean`.** Measure training metrics via the trainLora emit, not mount.
+- **Eval env:** `MLX_BUN_ORACLE_PYTHON=$ORC` (HumanEval code-exec — wrong default path on this
+  machine); evals default to **non-thinking** (optiq parity; `MLX_BUN_EVAL_THINK=1` to re-enable).
+- **MMLU parity:** our number (47.7 old / ~50 argmax) vs optiq 52.4 is **sampling + forward
+  parity**, not noise (both deterministic argmax). Use the frozen set; remaining gap = our
+  forward vs `mlx_lm` (logit-diff test on the frozen prompts — **queued, GPU**).
+- **Determinism:** same seed/order → same adapter. Replay recovers any checkpoint.
+- **Server adapter API** (`POST /v1/adapters` + per-request `"adapter":"id"`) is the clean way
+  to compare *many* adapters via chat; **raw eval (`--adapter`) is required for capability**
+  (MMLU is logit-argmax, not chat generation).
+
+## Recommended order
+
+1. **A** (final vs base) — the headline, READY. Run x2, then Exp 2.
+2. **B** — replay to recover the 6k peak (cheap, ~45 min).
+3. **C** — eval base/6k/final on the suite (four-way needs D for the +6k arm).
+4. Close the **pref-control harness gaps** (warm-start + subset), then **D** (DPO-boost) and
+   **E** (data-valuation control) — the synthesis experiments.
+5. (Parallel/queued) **MMLU forward-parity** logit-diff to nail x1 ≈ 30.28.

package/docs/{segmented-backward-handoff.md → investigations/segmented-backward-handoff.md}

@@ -1,7 +1,7 @@
 # Handoff — segmented backward training (2026-06-16)
 
 Current-state report for the segmented-backward LoRA training work. Deep dossier
-(design, proofs, every measurement): [`docs/design/segmented-backward-training.md`](design/segmented-backward-training.md)
+(design, proofs, every measurement): [`docs/design/segmented-backward-training.md`](../design/segmented-backward-training.md)
 §9 (MiniCPM5) and §10 (e4b). Branch: `segmented-backward-training`,
 PR [#9](https://github.com/joshuarossi/mlx-bun/pull/9).
 

package/docs/investigations/starter-model-port-handoff.md

@@ -52,6 +52,22 @@ QK-norm against the oracle, llama has none).
 Josh decides; both tracks are specified below. The shared recipe
 (§2) applies to either.
 
+> **Outcome (2026-06-17, resolved).** Both tracks proceeded as
+> recommended. **Track A (MiniCPM5-1B) was chosen as the publishing
+> gate and shipped** (open question 1 confirmed): `src/model/minicpm5.ts`
+> is live, parity suite green, PLAN.md updated. The starter runs
+> non-thinking (open question 2 confirmed). Weights are kept on disk as a
+> permanent fallback and tour-guide model — not deleted after the big model
+> lands (open question 3 confirmed). Publishing mechanics resolved
+> (open question 4 green-lit): the package is published as
+> `mlx-bun@0.0.4` on npm, installable via `bunx mlx-bun`, and a
+> Homebrew release pipeline (sign/notarize/tap, default tap
+> `joshuarossi/homebrew-tap`) is in place. First-run onboarding
+> (auto-download + browser open) shipped alongside. **Track B
+> (Qwen3.5-0.8B) is underway** as Phase 14 proper on its own clock,
+> with reasoning fixes already merged. All four open questions are
+> resolved; see §6.
+
 ---
 
 ## 1. Machine + reference environment logistics
@@ -308,16 +324,19 @@ isolates the novel machinery so parity bisects cleanly.
 - PLAN.md updated with findings (negative results included — e.g.
   any oracle deviations discovered, like the Phase 11 ones).
 
-## 6. Open questions for Josh
-
-1. **Track A vs B for the gate** — recommendation is A
-   (MiniCPM5) for publishing speed; B proceeds afterward as Phase
-   14 on its own clock. Decide before work starts.
-2. MiniCPM5's hybrid `<think>` mode (the OptiQ page's
-   `enable_thinking` note): starter ships non-thinking — confirm.
-3. Should the starter stay on disk as the permanent "fallback +
-   tour guide" (current plan) or be deleted once the big model
-   lands? (Current code keeps it; ~1 GB.)
-4. Publishing mechanics can proceed in parallel with the port
-   (npm `files` allowlist, un-`private` the package, repo
-   visibility decision for the native pack) — green-light?
+## 6. Resolved questions
+
+All four questions were resolved by 2026-06-17 (see outcome box above).
+
+1. **Track A vs B for the gate** — **Track A (MiniCPM5-1B) chosen.**
+   `src/model/minicpm5.ts` shipped as the publishing gate; Track B
+   (Qwen3.5-0.8B) is underway as Phase 14 proper on its own clock.
+2. **Thinking mode for the starter** — **Non-thinking confirmed.**
+   The starter runs with thinking disabled; no `<think>` tokens in the
+   first-run flow.
+3. **Keep weights on disk?** — **Yes, kept as permanent fallback + tour
+   guide.** The ~1 GB artifact is not deleted once the big model lands.
+4. **Publishing mechanics green-lit?** — **Yes, shipped.**
+   Package published as `mlx-bun@0.0.4` on npm; installable via
+   `bunx mlx-bun`; Homebrew release pipeline (sign/notarize/tap,
+   default tap `joshuarossi/homebrew-tap`) is in place.