mlx-bun 0.0.4 → 0.0.6

package/docs/investigations/trainer-validation-experiment.md

@@ -0,0 +1,173 @@
+# Trainer validation experiment
+
+**Status:** in progress (started 2026-06-20). **Goal:** prove the from-scratch
+SFT/DPO/ORPO trainer actually works before trusting any downstream result — general
+capability (Goal 1) or task-specific (Goal 2).
+
+## Why this exists — the smoking gun
+
+The margin/accuracy is recorded in `metrics.json` (`valTrajectory[].margin`,
+`bestMargin`), not the stdout log. The prior `cpm5-uf` ORPO run (λ=0.1) shows:
+
+| step | 100 | 200 | 300 | 400 | 500 | 600 | 700 | 800 |
+|---|---|---|---|---|---|---|---|---|
+| accuracy | .507 | .514 | .534 | .541 | .534 | .520 | .547 | .547 |
+| margin | .132 | .124 | .110 | .120 | .136 | .128 | .118 | .123 |
+
+**Preference accuracy is ~0.52 (near the 0.50 coin-flip) and the margin is flat.**
+ORPO's defining job is to push accuracy up and widen the margin. This is *not* a clean
+"it works" signal. Caveats: 800 steps ≈ 0.05 epoch (undertrained), provenance
+(from-base vs warm-start) unconfirmed. But it's enough to make validation mandatory:
+**we cannot currently claim our ORPO learns the preference at all.**
+
+## Method-decomposition (the validation key)
+
+`ORPO_loss = NLL(chosen) + λ·L_OR`. NLL is exactly the SFT objective; L_OR is the
+reference-free odds-ratio preference term. So we validate the building blocks, and ORPO
+is covered by its parts plus a small ORPO-specific surface (the OR term + λ).
+
+mlx-lm has **no DPO/ORPO** — so the oracle strategy is:
+- **SFT/NLL** → numeric parity vs **mlx-lm tuner** (oracle env exists).
+- **DPO + OR loss** → **numeric unit test vs the paper's equations** on a fixed tiny
+  input (hand-computed expected loss + grad). No external dep. Optional stronger
+  cross-check: **TRL** `DPOTrainer`/`ORPOTrainer` loss (PyTorch; multi-GB env →
+  approve-first, not default).
+
+Every positive control is designed so "weak data / weak lever" is **not** an available
+excuse — success must be unambiguous.
+
+## PILLAR 1 — Prove the trainer (gate for everything)
+
+- **1a · SFT positive control** *(infra GREEN, behavioral check WEAK)*. Train
+  `fixtures/adapters/data-upper` from scratch on MiniCPM5; assert generations are
+  actually uppercased (oracle: the repo's pre-trained `upper` adapter). Existing
+  `train-e2e.test.ts` only checks loss↓ — add the behavioral assert. *(seconds)*
+- **1b · DPO positive control** *(MISSING)*. Trivial preference (chosen uppercase,
+  rejected lowercase). Assert **margin increases** and **preference accuracy → ~1.0**.
+- **1c · ORPO positive control + the real question.** (i) Same trivial preference →
+  accuracy → ~1.0. (ii) **From-BASE ORPO on UF**, multiple epochs, watch margin widen
+  and accuracy climb well above chance — directly resolves the near-chance finding
+  (undertraining vs warm-start vs bug).
+- **1d · Loss-formula parity.** SFT NLL vs mlx-lm; DPO/ORPO loss + grad vs hand-computed
+  paper equations (unit test). The training-side analogue of the inference logit-parity
+  oracle — currently MISSING.
+- **1e · vs the paper.** Internal proxy first: preference accuracy robustly > chance on
+  UF. Then (approve-first, LLM-judge) AlpacaEval/MT-Bench **direction** vs base (not
+  absolute — paper used 7B). optiq suite (incl. IFEval) is an already-local downstream.
+
+> **Gate:** until 1a–1c are green, Pillars 2 & 3 sit on an unverified trainer.
+
+## PILLAR 2 — Measure general-capability impact (optiq suite)
+
+All 6 datasets are present locally (`~/.cache/mlx-bun/eval-data/`: mmlu, gsm8k, ifeval,
+bfcl, humaneval, hashhop); harness = `scripts/eval.ts capability`.
+- **2a · Eval parity:** run on base CPM5, reproduce optiq's **30.28** within tolerance
+  (validates the *measurement* before trusting deltas; confirm the README's open BFCL
+  "no-calls" / MMLU "at-chance" flags aren't broken on our side).
+- **2b · Delta:** trained checkpoint vs base, on the 6-task mean.
+- **2c · Targeting:** mean is floored by **BFCL 0.0 / GSM8K 2.7 / HashHop 4.0**; UF hits
+  the already-strong IFEval, so expect small mean movement. Highest-leverage Goal-1 lever
+  is **BFCL** (0.0 is format-trainable; 0→30 = +5 on the mean).
+
+## PILLAR 3 — Task-specific (separate; gated on Pillar 1)
+
+Chunk segmenter via `scripts/chunk-eval.ts` (boundary/label accuracy), using the
+hard-negative ORPO data (remove-a-cut / add-a-cut, balanced to kill the length
+shortcut). Measured by chunk accuracy, not val loss / capability. Credibility depends on
+Pillar 1.
+
+## Sequencing & gates
+
+1. Pillar 1 positive controls (1a→1b→1c-trivial) — fast, "does the machinery work."
+2. Pillar 1 real replication (1c from-base UF) + 1d loss-formula parity.
+3. Pillar 2a eval-parity (reproduce 30.28) → 2b deltas.
+4. Pillar 3 — once the trainer is trusted.
+
+## Execution log
+
+### 2026-06-20 — Pillar 1 quick validations: PASS
+
+**Machinery + gradient correctness — GREEN.** `MLX_BUN_TEST_TRAIN=1 bun test
+train-autograd train-e2e train-orpo-e2e` → 8 pass / 0 fail, real training on
+MiniCPM5 (11.3s). `train-autograd` confirms the backward matches autograd.
+
+**ORPO positive control (prefer-uppercase, 12 train / 4 val, 120 iters, lr 1e-3) — PASS.**
+Held-out preference accuracy **1.000 by step 10**, margin **5.5 → 4.0** (noisy at lr=1e-3
+but accuracy solid). Logged: `runs/orpo-pc.{json,csv}`.
+
+**DPO positive control (same data) — PASS, cleaner.** Accuracy **1.000 from step 10**,
+margin **monotonic 6.46 → 9.67**. Logged: `runs/dpo-pc.{json,csv}`.
+
+**Conclusion: the preference machinery is validated.** On a learnable preference both
+DPO and ORPO drive accuracy to 1.0 and the margin wide open (vs UF's flat ~0.12). A
+broken trainer could not. ⇒ **The UF near-chance result is the DATA, not a bug** — UF
+pairs are genuinely subtle for a 1B (both responses plausible); "prefer uppercase" is
+trivially separable. The smoking gun is explained: implementation sound, lever weak.
+
+**Gotcha found (recorded so we don't repeat it):** `AdapterManager.mount` (the *serving*
+hot-swap path) does **not** reflect into `branchLogpMean` (the *training* forward) — an
+external base-vs-adapter `orpoMetrics` measurement via mount returns bit-identical
+numbers (measured: 0.500/0.1463 both). Measure training-side metrics via the
+`trainLora` emit / loraState path (`scripts/experiments/pref-control.ts`), not mount.
+
+**Logging for plotting:** `runs/<stem>.{json,csv}` — per-step train `{loss, accuracy,
+margin}` + per-eval val `{loss, accuracy, margin}`. CSV columns: `phase,step,loss,
+accuracy,margin`. Tools: `scripts/experiments/pref-control.ts` (train+log),
+`scripts/experiments/measure-pref.ts` (standalone adapter eval — superseded by the
+emit path for training metrics; see gotcha).
+
+### 2026-06-20 — Pillar 2a: capability-harness parity. Two bugs found + fixed.
+
+**The first-measure step caught two harness bugs** (exactly its purpose — never trust a
+delta until the harness reproduces the published baseline):
+
+1. **Thinking mode.** The eval rendered the chat template's *default* (thinking-on for
+   MiniCPM5); optiq's numbers are **non-thinking** (HF card: IFEval 64.7 is non-thinking).
+   Symptom: IFEval **36.7%** vs 64.7. **Fix:** `src/eval/runner.ts` `generateText` now
+   renders `enableThinking: false` by default (env `MLX_BUN_EVAL_THINK=1` to re-enable).
+   → IFEval **73.3%** (n=30), optiq ballpark.
+2. **HumanEval oracle-python path.** `src/eval/tasks/humaneval.ts` hardcoded
+   `/Users/joshrossi/Code/mlx-lm/.venv/bin/python` — wrong on this machine (it's
+   `mlx-lm-example`), so every code-exec spawned a missing interpreter → **0%**. **Fix:**
+   env-overridable `MLX_BUN_ORACLE_PYTHON` (committed default = reference box; never
+   re-commit the path). → HumanEval **53.3%** (n=15), optiq 57.9.
+
+**Post-fix parity (base CPM5, non-thinking, n=20 — all ballpark):**
+
+| | MMLU | GSM8K | IFEval | BFCL | HumanEval | HashHop |
+|---|---|---|---|---|---|---|
+| ours | 55.0 | 10.0 | 75.0 | 0.0 | ~53 | 0.0 |
+| optiq | 52.4 | 2.7 | 64.7 | 0.0 | 57.9 | 4.0 |
+
+⇒ **harness reproduces optiq.** Small-n inflation on GSM8K/IFEval will settle toward
+optiq on the full set. **Exp-1 baseline harness VALIDATED.**
+
+### Ready-to-fire sequence (everything staged; only GPU time remains)
+
+All measurement tools validated. `scripts/eval.ts` now takes `--adapter <dir>` (mounts the
+trained LoRA → x2). `ORC=/Users/joshrossi/Code/mlx-lm-example/.venv/bin/python` below.
+
+1. **x1** (running): `MLX_BUN_ORACLE_PYTHON=$ORC bun scripts/eval.ts capability --candidate MiniCPM5`
+2. **Train** (user): from-base ORPO/UF → `./adapters/cpm5-uf-frombase` (command in chat).
+3. **x2 (Exp 1 after):** `MLX_BUN_ORACLE_PYTHON=$ORC bun scripts/eval.ts capability --candidate MiniCPM5 --adapter ./adapters/cpm5-uf-frombase` → compare to x1.
+4. **Exp 2 gen (GPU):** `bun scripts/experiments/prefbench.ts gen mtbench ./adapters/cpm5-uf-frombase mtbench-uf` (and `gen alpaca … alpaca-uf 200` for a subset).
+5. **Exp 2 judge (codex, anytime):** `bun scripts/experiments/prefbench.ts judge mtbench-uf` → TRAINED win-rate vs BASE.
+
+### Next
+- **Full baseline:** see step 1 (running). **(long → user-launched)**
+- **Train (the "do"):** from-base ORPO on UF (`pref-control.ts`, command in chat). **(long → user-launched)**
+- **Re-measure → x2**, compare to x1. Exp 1 done.
+- **Exp 2 judge — VALIDATED (keyless).** No OpenAI key needed: the judge shells out to
+  `codex exec -m gpt-5.4-mini` (codex's own auth), forcing structured `{"winner":...}` via
+  `--output-schema` and reading the clean verdict from `-o`. `scripts/experiments/llm-judge.ts`
+  — `judgeOnce` + `judgePair` (judges BOTH orders to cancel small-judge position bias).
+  Positive control: **3/3 obvious good-vs-bad pairs, good won both orders, zero flips.**
+  1bx vs 1bx1; consistency (same judge both sides), not absolute comparability to the paper's GPT-4.
+- **Exp 2 harness — STAGED.** Prompts downloaded: `~/.cache/mlx-bun/eval-data/{mt_bench.jsonl
+  (80), alpaca_eval.json (805)}`. Two-phase runner `scripts/experiments/prefbench.ts`:
+  `gen <mtbench|alpaca> <adapterDir> <stem> [limit]` (GPU — base then trained, non-thinking)
+  → `runs/<stem>.responses.jsonl`; `judge <stem> [limit]` (codex) → TRAINED win-rate vs BASE,
+  judging trained-vs-base **directly** (head-to-head, not vs a fixed reference). Only the `gen`
+  phase needs GPU; everything else is validated. Fire after the from-base training produces
+  `./adapters/cpm5-uf-frombase`.
+- **1c real / 1d parity** — still open (lower priority now the positive controls + harness parity are green).

package/docs/planning/IDEAS.md

@@ -8,6 +8,35 @@ work in flight lives in [PLAN.md](../../PLAN.md).
 
 ## Positioning: what mlx-bun is becoming
 
+### The finite-matrix local AI appliance
+
+`mlx-bun` can be a local AI product in a way generic LLM runtimes cannot. On
+MLX, the hardware universe is Apple Silicon Macs: a finite list of real shipped
+chip/RAM/bandwidth combinations. The supported model universe is curated, and
+the runtime usually knows the task mode: chat, tool use, vision, eval,
+training, adapter routing, or background automation.
+
+That means the default path can be a table of measured choices instead of a
+pile of user-facing knobs:
+
+```text
+physical Mac SKU + model checksum + task mode -> optimized execution plan
+```
+
+The plan can choose the model, context budget, KV policy, adapter policy,
+kernel route, compile route, and memory envelope before the user has to care.
+The user runs `mlx-bun`; the system gives them the most useful local assistant
+their Mac can comfortably run. Power users still get the stick-shift controls,
+but the product default is curated.
+
+This is also why optimization work belongs in the product vision, not just the
+lab notebook. Loop unrolling, branch deletion, DAG extraction, custom kernels,
+compiled graph tables, Cut Cross Entropy, Liger-style fused heads, and
+ORPO-trained adapters matter when they move a real device/model profile from
+"cute demo" to "leave it running because it is useful."
+
+### Local in the deeper sense
+
 Two phrases that capture the thesis — especially **mlx-bun × Lucien**:
 
 1. **"A truly local model in a deeper sense."**

package/docs/planning/PRODUCT_ROADMAP.md

@@ -19,9 +19,34 @@ provider config before they can try local AI. They should install one
 binary, run one command, wait for one resumable model download, and get a
 working local chat UI plus a local AI server.
 
+The intended first-run shape is product, not demo: start with the tiny
+welcoming model quickly, begin the stronger recommended model download in the
+background, and graduate the user into the best default profile for their
+actual Mac without requiring them to learn the model zoo first.
+
 The runtime can still expose every advanced knob. The default path should
 not require them.
 
+## Four product faces
+
+The same runtime has to support four related but distinct jobs:
+
+1. **Local AI product** — the one-command experience for someone who just wants
+   useful local AI. This path should be opinionated, benchmarked, and boringly
+   reliable.
+2. **Library for app developers** — the embeddable TypeScript/MLX layer for
+   Bun, Tauri, Electron, and desktop tools that want local inference without a
+   Python sidecar.
+3. **AI lab for researchers** — the place where parity, evals, kernels,
+   compiled graphs, ORPO, LoRA adapters, and model experiments can happen in
+   the same stack that serves the result.
+4. **Autonomous background agent runtime** — the always-available local server
+   for scripts, recurring tasks, personal tools, and per-request adapter
+   routing.
+
+These are not separate products. They are one runtime with progressive
+disclosure: appliance defaults at the front, sharp instruments underneath.
+
 ## The three modes (the organizing design law)
 
 mlx-bun has not published anything: the entire CLI, command, and flag
@@ -72,7 +97,7 @@ verbs and one flag vocabulary, not three separate CLIs:
   serving, latency prediction, the Lab UI) graft onto the mlx-lm spine
   as additive flags and a few new verbs.
 
-### Status (2026-06-12)
+### Status (2026-06-12; compat surface design remains open as of 2026-06-17)
 
 The compat layer does not exist yet: today's commands (`serve`, `fit`,
 `benchmark`, `evals`, `get`/`scan`/`ls`, `pi`, `harness`) are ad-hoc,
@@ -83,27 +108,42 @@ reconcile the existing commands into it, and fill the capability matrix
 behind it (convert/quantize, LoRA training, eval/perplexity, fuse,
 cache_prompt, upload/share). Decomposed into phases in PLAN.md.
 
-## Core idea
+## Core idea: finite matrix runtime
 
-The local-AI problem is not continuous for Mac users. People own a small
-set of real Apple Silicon SKUs: 16 GB, 24 GB, 32 GB, 36 GB, 48 GB,
-64 GB, and larger workstation tiers. For each profile, we can research
-and preselect the best known model artifact, context target, KV mode,
-memory budget, and runtime flags.
+The local-AI problem is not continuous for Mac users. MLX means Apple Silicon,
+and Apple ships a finite set of real chip/RAM/bandwidth combinations. The
+supported model universe is also curated. For each physical device profile and
+model snapshot, we can research and preselect the best known model artifact,
+context target, KV mode, memory budget, kernel route, graph route, and runtime
+flags.
 
-That means first-run behavior can be a lookup table, not a wizard or a
-benchmark:
+That means first-run behavior can be a measured lookup table, not a wizard or
+an exploratory benchmark:
 
 ```text
-detect Mac profile
-look up recommended runtime profile
+detect exact Mac profile
+fingerprint the model snapshot
+choose the task mode: chat, vision, tool loop, adapter, training, eval
+look up the measured execution plan
 download/resume the exact model artifact
 start the tuned local server
 open the built-in UI
 show how to connect external tools and harnesses
 ```
 
-The research work feeds the table. The table feeds the user.
+The current `fit` endpoint is the nucleus of this idea: it already reasons over
+real Apple Silicon profiles, model bytes, KV bytes, context, and predicted
+decode. The product version should evolve from "does this fit?" into:
+
+```text
+device profile + model checksum + task mode + live memory pressure
+  -> recommended execution plan
+```
+
+Execution plans can eventually include specialized graph variants, fused kernel
+routes, prompt-cache policy, adapter policy, safe context limits, expected
+decode/TTFT, and user-facing rationale. The research work feeds the table. The
+table feeds the user.
 
 ## Recommended profiles
 
@@ -143,6 +183,31 @@ changes a device-tier decision, unlocks a larger useful context window,
 keeps decode above a user-visible threshold, improves quality at the same
 budget, or prevents failure. Saving memory for its own sake is secondary.
 
+## Specialization strategy
+
+For the curated product path, it is acceptable to trade more work up front for a
+better hot path later. The Mac is known, the model snapshot is known, and the
+task mode is known. That lets us precompute or cache exact plans instead of
+shipping every decision to runtime.
+
+Candidate plan fields:
+
+- Model checksum and architecture fingerprint.
+- Apple Silicon profile: chip, unified memory, bandwidth tier, GPU/CPU shape.
+- Context buckets and fixed-shape cache layouts.
+- Kernel choices: MLX built-ins, MLX fast paths, custom Metal kernels, or
+  eager fallback.
+- Graph choices: eager, compiled exact-shape graph, compiled bucketed graph,
+  or prebuilt graph table.
+- Adapter choices: none, single LoRA, stacked LoRA, task-routed LoRA.
+- Measured results: TTFT, decode tok/s, peak memory, failure envelope.
+
+This is where loop unrolling, branch deletion, static DAG extraction, fused
+kernels, and shape-specialized compiled graphs become product features rather
+than abstract optimizations. They matter when they let the default profile be
+faster, larger, more stable, or better aligned without asking the user to tune
+anything.
+
 ## First-run UX
 
 The no-args command should be the friendly path:
@@ -153,17 +218,40 @@ mlx-bun
 
 Likely behavior:
 
-1. Detect the hardware profile.
-2. Pick the recommended runtime profile.
-3. Show the selected model and why it was chosen.
-4. Start or resume the background download.
-5. Start the local server when ready.
-6. Open or serve the built-in chat UI.
-7. Show connection details for OpenAI/Anthropic-compatible clients.
-8. Offer harness setup actions for Pi, Hermes, OpenClaw, and others.
+1. Detect the hardware profile. *(not yet: `recommendedRepoId()` at `src/fit.ts:86` returns `DEFAULT_REPO_ID` unconditionally — device-profile lookup table is unimplemented)*
+2. Pick the recommended runtime profile. *(same caveat as above)*
+3. Show the selected model and why it was chosen. *(not yet: model is printed, rationale is not)*
+4. Start or resume the background download. **Shipped** — auto-download on first run; resumable.
+5. Start the local server when ready. **Shipped.**
+6. Open or serve the built-in chat UI. **Shipped** — browser tab opened automatically; URL printed either way; `--no-open` skips the tab.
+7. Show connection details for OpenAI/Anthropic-compatible clients. **Shipped** — printed at startup.
+8. Offer harness setup actions for Pi, Hermes, OpenClaw, and others. *Partially shipped: `mlx-bun harness pi` configures an existing Pi install; Hermes and OpenClaw adapters do not exist.*
 
 This should feel like an appliance, not a framework tutorial.
 
+## Adapter catalog vision
+
+The average user should not have to train adapters to benefit from them. The
+product path can ship or offer a curated catalog of lightweight LoRA adapters:
+assistant, coding, writing, research, planning, personal-ops, and other
+well-scoped modes. A user can install one small adapter and get a better local
+assistant without changing the base model.
+
+Advanced users can still train their own adapters with mlx-bun. The broader
+product loop is:
+
+```text
+curate conversations and task data
+train / ORPO task adapters
+publish small adapter artifacts
+route requests through the right adapter per task
+keep the base model resident
+```
+
+This makes ORPO performance directly product-relevant. If training is cheap and
+repeatable enough, the project can keep improving small local models through
+adapter updates instead of asking every user to download a larger base model.
+
 ## Built-in UI
 
 Bun can serve the UI directly from the same executable. The default UI
@@ -213,6 +301,27 @@ The UI should not hide that this is local. It should make "local" feel
 easy: predictable memory, transparent downloads, offline-capable serving,
 and clear integration instructions.
 
+### SigLIP vision sidecar — image input for gemma e4b (shipped, 2026-06-17)
+
+The Gemma 4 e4b vision encoder (`src/vision/siglip.ts`) is a faithful port
+of the optiq SigLIP tower. It auto-loads on first image input and falls back
+to text-only mode when no image is present, so the chat UI gains image support
+with no startup cost. Features are within ~1% rel-RMSE of the optiq oracle;
+every MLX primitive (rms_norm, gelu, matmul, SDPA, RoPE, pooling) is bit-exact.
+The residual is sub-bf16 floating-point accumulation across the deep encoder,
+not a porting gap. L1/L2/L3 paths all work; vision falls back to the monolith
+path on L1 and L2.
+
+### Adapter web selector and CLI /v1/adapters (shipped)
+
+Trained LoRA adapters can be selected from the chat UI via a per-conversation
+picker (`#chat-adapter` in `src/web/app.html`). The picker lists adapters
+discovered by `GET /v1/adapters/available`, mounts the chosen one via
+`POST /v1/adapters`, and sends a `set_adapter` message to Pi-web so that
+every subsequent turn is routed through the right adapter — including the
+`before_provider_request` hook in `src/pi-web.ts`. Adapters are base-model-scoped
+and dropped on model swap.
+
 ### The UI must never lag while the AI is busy (candidate, 2026-06-16)
 
 A product guarantee: **the AI may crash, the UI never may** — and the UI
@@ -413,14 +522,25 @@ reuse across pipeline stages.
 
 ## Open questions
 
-- What exact profile table should ship first?
-- Should no-args open the UI automatically, print a local URL, or both?
-- Which UI stack should be embedded?
-- How much of Pi should be bundled, embedded, or configured externally?
-- What is the minimum first-run UI that feels good enough?
-- What is the right install path: Homebrew, curl installer, GitHub
-  release, or all of the above?
+- What exact profile table should ship first? *(The device-profile lookup
+  table is unimplemented — `recommendedRepoId()` returns `DEFAULT_REPO_ID`
+  for every SKU. The profile shape in this doc is the spec; the research
+  work and the fill remain open.)*
 - How should background downloads interact with server startup and UI
   readiness?
 - How should profile recommendations be versioned as benchmarks improve?
 - When Qwen lands, what evals decide whether it replaces a Gemma default?
+
+### Resolved
+
+- **Should no-args open the UI automatically, print a local URL, or both?**
+  Both: the browser tab is opened automatically on start; the URL is always
+  printed; `--no-open` suppresses the tab without hiding the URL.
+- **Which UI stack should be embedded? / How much of Pi should be bundled?**
+  Resolved by harness-pi + pi-web shipping: the web UI is the bundled
+  `src/web/app.html` chat page talking to Pi via `pi-web.ts`; Pi is
+  configured externally via `mlx-bun harness pi`, or launched embedded via
+  `mlx-bun pi`.
+- **What is the right install path?** All of the above — shipped: npm
+  publish (v0.0.4, `bunx mlx-bun`), Homebrew release pipeline (sign,
+  notarize, `joshuarossi/homebrew-tap`), and GitHub release binary.

package/docs/planning/chunk-task-roadmap.md

@@ -1,7 +1,7 @@
 # Chunk-task quality roadmap (2026-06-17)
 
 Where the MiniCPM5 chunking adapter stands and what raises real segmentation
-quality. Companion to [[chunk-eval-metric-vs-accuracy]] and
+quality. Companion to the chunk-eval metric vs. accuracy finding (score = well-formedness, not gold accuracy) and
 [`docs/investigations/chunk-finetune-experiment.md`](../investigations/chunk-finetune-experiment.md).
 
 ## Where we are

package/docs/planning/journal.md

@@ -153,3 +153,164 @@ the what/when (phases, exit criteria); this is the narrative behind it.
   3. **Tool param values were JSON-coerced regardless of schema.** `<param name="path">2025</param>` became the number 2025. Oracle (`_decode_tool_parameter_value` + `_tool_parameter_schema`) keeps string-typed params as raw text. Ported; also made the `<param>` regex CDATA-safe (a `</param>` inside CDATA no longer truncates the value). Unit tests added.
   4. **Sampling ignored the model's `generation_config.json`.** The server hardcoded temp 0.7 / no top-p; optiq serve injects the author-recommended sampling (`optiq/runtime/gen_config.py`) as defaults. Ported: `loadContext` reads `generation_config.json` (MiniCPM5: temp 0.9, top_p 0.95; Gemma: temp 1.0, top_k 64, top_p 0.95) and `toOptions` uses it when the request omits the field. Explicit request values still win. This was the likely "quality" gap on a 1B model.
   - Verification: MiniCPM5 parity (bf16 + mixed-KV) still 100/100 bit-exact; minicpm5-tools, tool-call, chat-template, tools-template, parity (Gemma 12B), server-tools, and server suites all green. Live two-turn agent round-trip (tool call → tool result → final answer) confirmed working in-process.
+
+## 2026-06-16 — Segmented-backward LoRA training (PRs #9–10)
+
+Long-context LoRA training was OOMing: MiniCPM5 @4096 spiked to ~25.5 GB full
+backward because all activations had to stay live simultaneously.  The
+segmented-backward design threads the backward segment-by-segment so only one
+segment's activations are resident at a time.
+
+The key implementation decision was to drive the per-segment backward with
+`mlx_vjp` (a new `Vjp` binding in `autograd.ts`) rather than a surrogate
+`value_and_grad` over a reconstructed loss.  The surrogate path leaked roughly
+one activation buffer per segment per step because mlx `eval` does not detach;
+`mlx_vjp` is explicit about what it differentiates and stops the leak.
+Segment-boundary tensors are materialized as graph-free leaves via
+`fromBytesCopy` for the same reason.
+
+Phase A (MiniCPM5, PR #9) landed first: grads bit-exact vs the full backward
+(relNorm 0.000%), peak 10.91 → 3.29 GB @2048, no leak, trains end-to-end via
+`TrainConfig.segmentSize`.  The default training attention is `ops.sdpa`
+(correct gradients, one fused kernel); `MLX_BUN_TRAIN_ATTN=flash` is an
+opt-in for bit-exact grad comparison against the full backward only, because
+`ops.sdpa` has a pre-existing eager-vs-autograd asymmetry in bf16 (~6%) that
+is unrelated to the segmentation mechanism — the segmented path works
+correctly under either setting.
+
+Phase B (e4b / Gemma4, PR #10) was harder: e4b's KV-shared donor pattern
+means multiple sharer segments reference the same K/V tensors produced by a
+single producer segment, requiring cotangent accumulation across sharers before
+the producer vjp can run.  Implemented `SegmentedBackwardGemma4` with a
+two-boundary stream (per-layer-input + donor K/V) and a multi-output producer
+vjp `[h, donorK, donorV]` with accumulated `[dh, dKV.k, dKV.v]` cotangents.
+Grads are bit-exact for single-consumer donor reuse and 0.97% bf16-class for
+multi-consumer accumulation (confirmed the full backward also sums in bf16, so
+this is order-of-operations noise, not a logic bug).  e4b trains at 8 K context
+(~17.5 GB, segSize=2) where the full backward would OOM (~70 GB).
+
+A contiguity fix was also required: donor K/V are transposed views, so
+`detachLeaf` must row-major-copy them before `rawBytes` reads linearly.
+
+## 2026-06-16 — Qwen3.5 / MiniCPM5 reasoning-channel fix (PR #12)
+
+Two serving-layer bugs left reasoning-capable models unusable in chat:
+
+The first was a role-name mismatch.  pi-ai renames the system role to
+`developer` for reasoning models.  Our chat templates only accepted
+system/user/assistant/tool and returned 400.  The turn completed with
+`stopReason: "error"` without throwing, so the browser saw an empty turn with no
+error shown.  Fix: `normalizeMessages` maps developer→system (genuine OpenAI
+compat — OpenAI itself uses the developer role), and `mapEventToFrames` surfaces
+a `stopReason:"error"` turn as a visible error frame instead of silently
+discarding it.
+
+The second was reasoning leaking into content.  Qwen3.5 and MiniCPM5 templates
+prime an *open* `<think>` tag in the generation prompt, so the model's first
+token is the closing `</think>` — the opening tag never appears in the decoded
+stream.  `ThinkingTagSplitter` was waiting for the open tag before routing
+reasoning content into the OpenAI `reasoning_content` channel, so all of it
+leaked into `content` instead.  Fix: `promptEndsInOpenThink` detects this
+condition and seeds `ThinkingTagSplitter` via `startInThinking` so the splitter
+begins in the reasoning state.
+
+## 2026-06-16 — Adapters end-to-end (PR #13)
+
+The goal was to make LoRA adapter switching a first-class operation in both the
+web chat UI and the Pi CLI, without building a custom protocol — lean on Pi's
+`before_provider_request` hook instead.
+
+Server-side: `GET /v1/adapters/available` scans the two adapter stores
+(`~/.cache/mlx-bun-finetunes`, `~/.cache/mlx-bun/adapters`) and returns
+`{id, path, rank, scale, mounted}` for weight-bearing checkpoints (dataset-only
+dirs are skipped).  `lora.ts` gained `listAvailableAdapters` for the scan and
+base-model detection so the web selector can filter to compatible adapters only.
+
+Pi hook: `pi-web.ts` registers `pi.on("before_provider_request")` to inject the
+per-connection selected adapter into every outgoing provider payload using the
+existing inline-extension pattern.  A `set_adapter` WebSocket message sets the
+selection; default none leaves the payload unchanged so the base model runs.
+
+Web chat: an adapter dropdown was added next to the Thinking pill, populated
+from `/v1/adapters/available` on ready.  Selecting fires an idempotent `POST
+/v1/adapters` then a `set_adapter` WS message.
+
+CLI: `extensions/mlx-bun-adapter.ts` is a shipped Pi extension exposing
+`/adapter <id> | off | list` and the same hook.  Both UIs control adapters
+through Pi; no separate plumbing.
+
+## 2026-06-17 — First-run onboarding UX
+
+The previous startup story required knowing the right model ID, running a
+separate download command, and manually opening a browser.  The new path:
+`mlx-bun serve` (or bare `mlx-bun`) auto-downloads the starter if absent,
+starts the server, and opens the chat UI.  If a localhost tab is already open it
+focuses that tab rather than opening a duplicate (`--no-open` to skip).
+
+The web chat empty state was updated with mlx-bun-centric starter chips and a
+download-aware greeting.  A live background-download pill shows progress.  The
+system prompt briefs the model on mlx-bun and its own identity, with framing
+appropriate to MiniCPM5 being a fast starter while a larger model optionally
+loads in the background.
+
+`chat-template.ts` also tightened `supportsThinking` to require the `<think>`
+channel we actually split, so e4b / harmony-format models no longer leak raw
+reasoning markers into content for users who haven't configured a thinkingFormat.
+
+The runtime-isolation design doc (`docs/design/runtime-isolation.md`) was also
+written this session: inference running in-process on the Bun event loop is a
+known UI-lag footgun when the GPU is busy.  The doc proposes moving inference to
+a subprocess; the work is explicitly deferred (design, not started) — written
+now so the root-cause analysis is available cold.
+
+## 2026-06-17 — SigLIP vision sidecar for gemma-4 e4b
+
+Ported optiq's `vlm/gemma4` SigLIP encoder as `src/vision/siglip.ts`: a 16-layer
+vision transformer with clippable linears (trained clip bounds), manual-f32
+RMS norms on q/k/v, on-device 2D RoPE, fused SDPA at scale=1.0, and GeGLU →
+3×3 avg-pool → `MultimodalEmbedder` → `/embed_scale`.
+
+The tower selection is driven by `vision_config.model_type`: e2b/e4b get
+SigLIP, the 12B gets the existing encoder-free tower.  Lazy loading means
+text-only sessions never pay for it.  `gemma4.ts`'s `forwardEmbeddings` was
+extended to support per-layer-input models (e2b/e4b) — it previously threw for
+that path — threading zeroed image-token ids into the per-layer forward matching
+optiq's `where(text_mask, ids, 0)`.
+
+Parity: every mlx primitive is bit-exact vs the optiq oracle (verified
+model-free via `op-parity-{dump.py,check.ts}` — rms_norm, gelu, matmul, clip,
+cos, sin, full multidim RoPE, sdpa, pool).  The residual ~1% feature divergence
+is sub-bf16 full-graph composition order amplified by the encoder's scale=1.0
+peaked softmax (the topped-out softmax distribution is highly sensitive to small
+pre-softmax shifts in bf16).  The acceptance bar was exact spliced ids + greedy
+prefix + grounded output, which passes.  Driving the residual to zero would
+require matching optiq's exact op ordering across the full encoder; deferred as
+a TODO.
+
+## 2026-06-17 — Distribution: npm, bunx, Homebrew signed/notarized, direct download
+
+The question was how to ship a standalone binary that runs on a user's Mac
+without requiring Bun, Python, or anything else.  We settled on four parallel
+install paths rather than picking one.
+
+**npm / bunx**: `bunx mlx-bun` for users already in the Bun ecosystem.  A
+`publish-release.sh` script and `bun run publish` alias handle the npm side.
+Published as v0.0.4.
+
+**Homebrew**: `scripts/release-binary.sh` builds a standalone binary, signs it
+with a Developer ID (nested Mach-O first, then the executable with JIT
+entitlements from `packaging/entitlements.plist`), notarizes via `AC_PROFILE`, and
+produces a clean tarball + sha256 for the formula.
+`packaging/homebrew/mlx-bun.rb` is the formula source of truth (self-contained
+`libexec` bundle + `bin` symlink; Bun `realpath`s `execPath` so sibling dylibs
+resolve after relocation).  The tap lives at `joshuarossi/homebrew-tap`.
+`scripts/publish-release.sh` auto-syncs the formula sha256 on each release.
+One fix was required: `--disable-library-validation` in the entitlements so
+brew-relocated dylibs load without `CS_RESTRICT` blocking them.
+
+**Direct download**: the notarized tarball passes Gatekeeper even when
+downloaded via browser + Finder (quarantine worst case), so a stable versionless
+URL (`releases/latest/download/mlx-bun-arm64.tar.gz`) is published alongside the
+versioned asset and documented as a one-liner curl install.
+
+**Embedded library**: existing path, unchanged (`docs/reference/embedding.md`).