mlx-bun 0.0.4 → 0.0.5

package/README.md

@@ -1,21 +1,44 @@
 # mlx-bun
 
-Native MLX inference for Bun. Run quantized LLMs on Apple Silicon from
-TypeScript — no Python, no sidecar server, one runtime.
-
-Measured head-to-head on an M4 Pro (24 GB), same models, same day:
-logits **bit-exact** against the Python reference; served over HTTP the
-**fastest stack on every model tested** (TTFT 45–89 ms vs python's
-220–327 ms, zero server overhead vs mlx-lm's −5–6%); prefill up to
-**1.8×** mlx-lm at 8k context; cold start → first token of a cached
-prompt in **394 ms**. Full table: [benchmarks](#benchmarks).
+Native MLX inference for Bun, and the beginning of a one-command local AI
+product for Apple Silicon Macs. `mlx-bun` detects the machine, starts a useful
+local assistant, exposes OpenAI/Anthropic-compatible APIs, and gives
+TypeScript apps direct access to MLX without Python or a sidecar server.
+
+The product bet is that local AI on Mac is a finite optimization problem, not
+an infinite configuration maze: Apple ships a known set of chip/RAM/bandwidth
+SKUs, the supported model list is curated, and the task mode is explicit. That
+means defaults can be chosen from measured device × model × mode profiles
+instead of asking every user to become an ML systems engineer.
+
+Measured head-to-head on an M4 Pro (24 GB), same models, same machine
+state: logits **bit-exact** against the Python reference; served over
+HTTP the **fastest stack on every model tested** — fastest decode, plus
+TTFT **34–85 ms** vs python's 218–326 ms at **~0% server overhead**
+(mlx-lm pays −5–6%); server start to ready in **0.17–0.47 s**, 2–5×
+faster than the Python servers. Full table: [benchmarks](#benchmarks).
 
 ## Getting started
 
 You need an Apple Silicon Mac (MLX is Metal-only, so macOS only by
-design). Three ways in.
+design). Four ways in.
+
+### Direct download (recommended)
+
+No Homebrew, Bun, or git — just curl the install script. It downloads the
+signed, notarized self-contained binary from the latest release and puts
+`mlx-bun` on your PATH:
+
+```sh
+curl -fsSL https://mlx-bun.dev/install.sh | sh
+mlx-bun
+```
 
-### Homebrew (recommended)
+Self-contained (binary + MLX runtime) and notarized, so it runs without a
+Gatekeeper prompt. Installs to `~/.mlx-bun` by default — override with
+`MLX_BUN_INSTALL_DIR`, or pin a release with `MLX_BUN_VERSION=v0.0.4`.
+
+### Homebrew
 
 A self-contained, signed + notarized binary — no toolchain to install by
 hand:
@@ -68,8 +91,7 @@ model named, it does everything for you:
 2. downloads the sub-GB `MiniCPM5-1B` starter and serves it, so you're
    chatting in well under a minute;
 3. starts downloading `gemma-4-e4b` (the stronger 4B model) in the
-   background — it becomes the default on your next `mlx-bun serve`, on a
-   16 GB+ Mac;
+   background — it becomes the default on your next `mlx-bun serve`;
 4. opens the chat UI in your browser at
    [`http://localhost:8090/#/chat`](http://localhost:8090/#/chat)
    (pass `--no-open` to skip).
@@ -95,22 +117,46 @@ That's it. The server serves the one model it was started with (the
 request's `model` field is ignored and the loaded model id is echoed
 back in responses).
 
+## What mlx-bun is
+
+`mlx-bun` has four faces, all sharing one native runtime:
+
+- **The local AI product** — run `mlx-bun` or `bunx mlx-bun` and get a
+  working local chat UI plus a local server. The default path should feel like
+  an appliance: download the starter model, start chatting quickly, then move
+  to the stronger recommended model as it becomes available.
+- **The app developer library** — embed local MLX inference inside Bun,
+  Tauri, Electron, and other TypeScript applications without running Python in
+  the background.
+- **The AI lab** — keep parity, evals, kernels, LoRA/ORPO training, adapter
+  routing, and performance experiments close to the runtime that will actually
+  serve them.
+- **The background agent runtime** — leave a local OpenAI-compatible endpoint
+  available for scripts, recurring jobs, app integrations, and per-request
+  adapter routing.
+
+The default experience is intentionally opinionated. Power users can override
+models, budgets, KV modes, adapters, and flags; the average user should not
+need to know those knobs exist.
+
 ## Supported models
 
 Scope is deliberate: a few model families held to **bit-exact** logit
 parity with the Python reference, rather than dozens held to none.
-Currently MiniCPM5 plus the Gemma-4 OptiQ quants:
+Currently MiniCPM5, the Gemma-4 OptiQ quants, and Qwen3.5-4B:
 
 | Model | Download | Fits on | Vision | Notes |
 |---|---|---|---|---|
 | [`mlx-community/MiniCPM5-1B-OptiQ-4bit`](https://huggingface.co/mlx-community/MiniCPM5-1B-OptiQ-4bit) | 0.92 GB | 8 GB | — | Sub-GB option for 8 GB machines; bit-exact 100-step oracle parity (bf16 + mixed KV), tool calling + agent loop verified |
-| [`mlx-community/gemma-4-e4b-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-e4b-it-OptiQ-4bit) | 7.0 GB | 16 GB | — | **Recommended starter** (16 GB+); ~54 tok/s |
-| [`mlx-community/gemma-4-12B-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-12B-it-OptiQ-4bit) | 8.4 GB | 16 GB | ✓ | Vision sidecar + tool calling, both verified end-to-end |
-| [`mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit) | 18 GB | 24 GB | — | MoE (top-8 of 128 experts); ~54 tok/s — the python servers crash loading it on 24 GB |
+| [`mlx-community/Qwen3.5-4B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.5-4B-OptiQ-4bit) | 3.1 GB | 8 GB | — | bf16-KV bit-exact parity (vs mlx-lm); thinking + tool calling; ~74 tok/s predicted |
+| [`mlx-community/gemma-4-e4b-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-e4b-it-OptiQ-4bit) | 7.0 GB | 16 GB | ✓ | **Recommended starter** (16 GB+); ~56 tok/s |
+| [`mlx-community/gemma-4-12B-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-12B-it-OptiQ-4bit) | 8.4 GB | 16 GB | ✓ | Vision + tool calling, both verified end-to-end |
+| [`mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit) | 18 GB | 24 GB | — | MoE (top-8 of 128 experts); ~54 tok/s |
 
 Not sure what fits your machine? `bun src/cli.ts fit <model> --ctx 8192`
-gives a deterministic answer (see below). Qwen 3.x is next on the
-roadmap (see [PLAN.md](./PLAN.md)).
+gives a deterministic answer (see below). The larger
+`Qwen3.6-27B-OptiQ-4bit` is still in bring-up — parity and serving polish
+remain (see [PLAN.md](./PLAN.md)).
 
 ## Why
 
@@ -131,7 +177,7 @@ one:
 - **`bun:ffi`** — lowest-overhead FFI of any JS runtime; binds `mlx-c`
   directly, no node-gyp, no binding compilation.
 - **Lazy native weight loading** — mlx's loader materializes tensors on
-  first use; opening an 8.9 GB model takes milliseconds, and warm
+  first use; opening an 8.4 GB model takes milliseconds, and warm
   restarts are near-instant on cached pages.
 - **`Bun.Image`** — native OS image codecs (HEIC, AVIF, WebP, JPEG, …)
   for the vision path, EXIF auto-orientation included.
@@ -152,9 +198,11 @@ bun src/cli.ts fit gemma --ctx 32768          # memory contract: fits? max conte
 bun src/cli.ts fit gemma --ctx 8192 --skus    # ...same, across the Apple Silicon lineup
 bun src/cli.ts serve gemma --port 8090        # OpenAI-compatible server
 bun src/cli.ts serve gemma --memory-budget 18 # ...with admission control (GB)
+bun src/cli.ts pi                   # built-in agentic coding CLI (pi's TUI, in-process)
 bun src/cli.ts evals                # recorded benchmark runs
+bun src/cli.ts harness pi           # connect your own pi install to the local server
 ./benchmark.sh                      # head-to-head matrix vs mlx-lm/optiq (reboot first;
-                                    #   preflight-gated, resumable, writes benchmarks-h2h-<date>.md)
+                                    #   preflight-gated, resumable, writes benchmarks-h2h-<date>-<host>.md)
 ```
 
 Model arguments are substring queries against the registry (`e4b`,
@@ -177,15 +225,18 @@ agent CLIs like pi/OpenClaw via their provider config.
 - **`POST /v1/chat/completions`** — streaming (SSE) and non-streaming;
   `temperature`, `top_p`, `top_k`, `max_tokens`, `seed`,
   `repetition_penalty`, `stop` (string or array, matched on decoded
-  text with streaming hold-back); omitted sampling fields default to
-  the model's own `generation_config.json` recipe; usage includes
-  `cached_tokens`. Full schemas in
+  text with streaming hold-back), `reasoning_effort` (thinking on/off for
+  Qwen3.5 / MiniCPM5 — `none` disables, any level enables), and an `hlg`
+  tone-curve sampling override;
+  omitted sampling fields default to the model's own
+  `generation_config.json` recipe; usage includes `cached_tokens`.
+  Full schemas in
   [docs/reference/server-api.md](./docs/reference/server-api.md); start
   flags and the `--batch N` compatibility matrix in
   [docs/reference/server-config.md](./docs/reference/server-config.md).
 - **Tool calling** — pass OpenAI `tools`; each family's native format
   is parsed into `tool_calls` JSON with `finish_reason: "tool_calls"`
-  (Gemma 4 `<|tool_call>` sentinel tokens; MiniCPM5
+  (Gemma 4 `<|tool_call>` sentinel tokens; MiniCPM5 / Qwen3.5
   `<function name=…>` XML with schema-aware argument decoding);
   `role: "tool"` round-trips, including multi-turn agent loops.
 - **Vision** — `image_url` content parts (data: URLs or http/s), on
@@ -195,16 +246,17 @@ agent CLIs like pi/OpenClaw via their provider config.
   common token prefix across requests (multi-turn conversations
   re-prefill only the new turn) — automatic, no client changes.
   Entries are adapter-specific when LoRA adapters are in play.
-- **LoRA hot-swap** — mount adapters at startup (`--adapter id=dir`)
-  or at runtime (`POST /v1/adapters`), select per request with the
-  `adapter` body field (`"id"`, stacked `"a+b"`, or `"none"`). The
-  base model never reloads; an unselected adapter costs nothing.
+- **LoRA hot-swap** — mount adapters at runtime (`POST /v1/adapters`)
+  and discover on-disk ones (`GET /v1/adapters/available`), select per
+  request with the `adapter` body field (`"id"`, stacked `"a+b"`, or
+  `"none"`). The base model never reloads; an unselected adapter costs
+  nothing.
 - **Mixed-precision KV** — when the model repo ships `kv_config.json`
   (every Gemma-4 OptiQ repo does), per-layer KV quantization applies
-  automatically (full-attention layers; sliding layers stay bf16).
-  Measured decode-neutral at 8k on the 12B with KV bytes ÷4 on the
-  quantized layers. `--no-kv-quant` forces bf16; `--kv-bits N` forces
-  uniform bits. Long prefills over quantized caches run a fused
+  automatically; the config sets bits per layer for both full-attention
+  and sliding-window (rotating) layers (Phase 9). Measured decode-neutral
+  at 8k on the 12B with KV bytes ÷4 on the quantized layers. `--kv-quant off` forces bf16; `--kv-quant 4|8`
+  forces uniform bits. Long prefills over quantized caches run a fused
   FlashAttention-2 tiling that never materializes the full scores
   matrix (bounded transient; `MLX_BUN_NO_FUSED_SDPA=1` to disable).
   `MLX_BUN_FUSED_DECODE=1` extends the tiling to single-token decode —
@@ -225,27 +277,30 @@ agent CLIs like pi/OpenClaw via their provider config.
   Codex/Cursor/Continue speak; includes `previous_response_id`
   resumption backed by a TTL + byte-capped store (visible in /stats).
 - **`GET /v1/models`**, **`GET /stats`** (cache hit rates, bytes,
-  active KV scheme, response store), **`GET/POST/DELETE /v1/adapters`**.
+  active KV scheme, response store), **`GET/POST/DELETE /v1/adapters`**,
+  plus **`GET /library`**, **`GET /fit`**, **`GET /downloads`**, and
+  **`GET /v1`** (a self-describing index of every endpoint).
 
 ## Library
 
-The server is one consumer of a library-first API. Not yet published to
-npm — clone the repo and import from `src/`. Full reference:
+The server is one consumer of a library-first API. Published to npm as
+`mlx-bun` (current: 0.0.4) — import from the package (`from "mlx-bun"`)
+or `./src/index` in a clone; `bunx mlx-bun` runs the CLI without
+installing. Full reference:
 [docs/reference/library-api.md](./docs/reference/library-api.md). For shipping inside a
 Mac app (Tauri/Electron sidecar), `./scripts/build-binary.sh` produces
 a relocatable single-binary bundle — recipe incl. signing/notarization
 in [docs/reference/embedding.md](./docs/reference/embedding.md).
 
 ```ts
-import { loadModelConfig } from "./src/config";
-import { Weights } from "./src/weights";
-import { Gemma4Model } from "./src/model/gemma4";
-import { generate } from "./src/generate";
-import { loadTokenizer } from "./src/tokenizer";
-import { ChatTemplate } from "./src/chat-template";
+import {
+  createModel,        // dispatches to Gemma4 / MiniCPM5 / Qwen3.5
+  Weights, loadModelConfig, loadTokenizer, ChatTemplate, generate,
+} from "mlx-bun";     // or "./src/index" in a clone
 
 const dir = "/path/to/hf/snapshot";
-const model = new Gemma4Model(await Weights.open(dir), await loadModelConfig(dir));
+const config = await loadModelConfig(dir);
+const model = createModel(await Weights.open(dir), config);  // RuntimeModel
 const tok = await loadTokenizer(dir);
 const template = await ChatTemplate.load(dir);
 
@@ -266,43 +321,44 @@ preamble prefills once, ever.
 ## Benchmarks
 
 Head-to-head against the Python stacks (mlx-lm 0.31.3, mlx-optiq 0.2.1),
-same machine (M4 Pro 24 GB), same day, same HF snapshots, preflight-gated
-clean machine, median-of-N with warmups discarded. Full curated table
-(parity / performance / quality) with per-row provenance:
-[benchmarks/RESULTS.md](./benchmarks/RESULTS.md).
+same machine (M4 Pro 24 GB), same HF snapshots, the 2026-06-14
+cleared-machine run, preflight-gated, median-of-N with warmups
+discarded. Full curated table (parity / performance / quality) with
+per-row provenance: [benchmarks/RESULTS.md](./benchmarks/RESULTS.md).
+
+**Served (warm)** — how agents actually use a local model.
+decode tok/s · TTFT ms · start s:
 
-| | mlx-bun | mlx-lm | optiq |
+| model | mlx-bun | mlx-lm | optiq |
 |---|---|---|---|
-| **TTFT, served (warm)** | **45–90 ms** | 219–224 ms | 222–331 ms |
-| **server start → ready** | **0.36–0.47 s** | 0.76–0.98 s | 0.79–1.00 s |
-| **decode through HTTP** (e4b / 12B / 26B) | **54.5** / 25.2 / **54.9** | 53.5 / — / 52.2 | 53.5 / **25.5** / † |
-| **server tax vs own direct decode** | **≈ 0%** | −5…−7% | ≈ 0% |
-| **direct decode** (engine only) | −1.9…−4.4% vs mlx-lm | baseline | −0.8…−1.2% |
-| **12B decode @8k context** | 23.3 (23.0 kv-mixed) | **24.4** | 23.2 kv-mixed |
-
-Honest negatives, same table: in this matrix our direct decode trailed
-mlx-lm on every model (12B −1.9%, 26B −2.9%, e4b −4.4% at short
-context; the 12B gap grew to −4.5% @8k), and optiq's served 12B edges
-ours by ~1% (25.5 vs 25.2) while paying 3.7× the TTFT. **Post-matrix
-(2026-06-11) the decode gap was root-caused and fixed** — a
-prefill→decode allocator-reclaim stall that mlx-lm clears with
-`mx.clear_cache` and bills to prompt time (we billed it to decode);
-after the reference-faithful fix, same-session paired runs put the 12B
-*ahead* at short context (25.1 vs 24.0) and at parity @8k (23.8 vs
-23.9). Clean-machine re-measure pending; e4b retains a ~5% per-step
-host-overhead residual (see PLAN.md "Decode gap RESOLVED"). Served through HTTP — how agents actually use a local
-model — mlx-bun has the fastest decode on e4b and the 26B, and the
-fastest TTFT and startup everywhere by 2–5×. † = cell failed: optiq
-serve produced no output on the 26B (the Metal OOM crash class from
-python's non-lazy load transient — reproduced in isolation; mlx-bun
-and mlx-lm both served the same model from the same machine state).
-One further optiq cell is blocked on an upstream optiq bug; both are
-documented in the results file.
-
-These numbers are the 2026-06-11 cleared-machine re-run with the
-long-context guard active (every @8k row verified at its requested
-context — an earlier harness bug had fed python @8k baselines
-~31-token prompts; fixed, guarded, and re-measured).
+| MiniCPM5-1B | **252.9** · 34 · 0.17 | — | 223.6 · 64 · 0.84 |
+| gemma-4-e4b | **55.7** · 44 · 0.36 | 53.5 · 218 · 0.98 | 53.4 · 221 · 0.78 |
+| gemma-4-12B | **25.9** · 85 · 0.38 | — | 25.5 · 326 · 1.24 |
+| gemma-4-26B | **54.2** · 45 · 0.47 | 52.3 · 228 · 0.77 | — |
+
+Across every served model mlx-bun has the fastest decode and the fastest
+TTFT/startup (2–5×), at **~0% server tax** vs its own direct engine —
+mlx-lm pays −5–6%, optiq −1 to −10%.
+
+**Direct (engine only).** Decode is at parity-to-slightly-behind mlx-lm
+(e4b +1.1%, 12B +0.4%, 26B −1.1%, MiniCPM5 −0.9% — within the per-step
+host-overhead residual), and prefill leads on the larger models (12B
+1.2×, 26B 1.1×, MiniCPM5 2.3×; e4b trails at 0.8×). The earlier decode
+deficit was root-caused and fixed (2026-06-11) — a prefill→decode
+allocator-reclaim stall mlx-lm clears with `mx.clear_cache` and bills to
+prompt time; see PLAN.md "Decode gap RESOLVED".
+
+**Long context (12B).** At 16k and 64k mlx-bun holds decode parity with
+mlx-lm on bf16 (16k 23.9 = 23.9; 64k 20.9 = 20.9) while optiq drops to
+21.6 then collapses to 12.3 at 64k; mixed-KV trades ~2 tok/s for ~5 GB
+lower peak.
+
+Two cross-stack served cells are absent: no mlx-lm 12B row, and optiq
+produced no output on the 26B (the Metal-OOM class from python's
+non-lazy load transient — reproduced in isolation; mlx-bun and mlx-lm
+both served it from the same machine state). One further optiq cell
+(12B/kv=config) is blocked on an upstream `quantized_matmul` bug. Both
+are documented in the results file.
 
 ## Correctness
 
@@ -319,8 +375,8 @@ findings). Golden files are regenerated only by explicit scripts
 (`scripts/regen-*.ts`) running the Python oracle.
 
 ```sh
-bun test    # fast tier runs everywhere; model-loaded tests auto-skip
-            # unless the reference snapshot is in your HF cache
+bun test    # fast tier runs everywhere; model-loaded tests run only
+            # when the reference snapshot is in your HF cache
 ```
 
 ## Troubleshooting
@@ -343,19 +399,36 @@ bun test    # fast tier runs everywhere; model-loaded tests auto-skip
 ## Status
 
 Pre-alpha, moving fast. See [PLAN.md](./PLAN.md) for phases, exit
-criteria, measured numbers, and the findings log. Complete: load path,
-bit-exact model port (12B dense / e4b per-layer-input / 26B MoE),
-sampling + serving (tools, vision, prompt cache), registry/fit/KV
-persistence, quantized + mixed-precision KV serving (rotating-cache
-KV-quant included, Phase 9) with fused quantized prefill (Phase 10),
-LoRA hot-swap with per-request selection, resumable verified downloads
-(`mlx-bun get`), memory admission control, and the head-to-head
-benchmark harness, and the decode-gap root-cause fix (clear_cache
-placement + boundary accounting, 2026-06-11 — 12B now at-or-above
-mlx-lm decode in paired runs). Open: e4b's ~5% host-overhead decode
-residual (Phase 7), Anthropic messages + Responses API (Phase 11),
-SigLIP vision for e4b/26B, Qwen 3.x, the embeddable single-binary
-build.
+criteria, measured numbers, and the findings log.
+
+**Complete** — load path; bit-exact model port (MiniCPM5, Qwen3.5-4B,
+e4b per-layer-input, 12B dense, 26B MoE); sampling + serving (tools, vision,
+prompt cache); registry / fit / KV persistence; quantized +
+mixed-precision KV serving (rotating-cache KV-quant, Phase 9) with fused
+quantized prefill (Phase 10); LoRA hot-swap with per-request selection;
+segmented-backward LoRA training (SFT / DPO / ORPO, with an `[M,vocab]`-free
+flash-CCE head + prefix-sharing for long-context preference data — see
+[training guide](docs/reference/training.md)); resumable
+verified downloads (`mlx-bun get`); memory admission control; the
+head-to-head benchmark harness; the decode-gap root-cause fix
+(2026-06-11); Anthropic Messages + Responses API (`/v1/messages` and
+`/v1/responses`, Phase 11); SigLIP vision sidecar for e4b
+(commit 4625fe5); the embeddable single-binary build (signed +
+notarized — Homebrew, direct-download, and npm/bunx).
+
+**In progress** — `Qwen3.6-27B` bring-up (Phase 14f): same architecture
+as the verified 4B (untied + larger geometry); parity and serving polish
+remain. MTP speculation and Qwen3-VL vision deferred.
+
+**Experimental** — opt-in, default-off, still being hardened: transparent
+expert offload for MoE models (`serve --expert-offload`, Phase 20:
+page-aligned mmap-backed experts, bit-exact — 26B-A4B 17.1→4.2 GB
+resident, decode unregressed); batched serving (`--batch N`, Phase 18:
+continuous-batching bf16 B=N decode, B=2 bit-parity verified for
+MiniCPM5/12B/e4b/26B; throughput polish remains).
+
+**Open** — e4b's ~5% per-step host-overhead decode residual (Phase 7);
+SigLIP vision for 26B.
 
 ## License
 

package/docs/design/adapters-end-to-end.md

@@ -1,5 +1,16 @@
 # Adapters end-to-end — notes / TODO (2026-06-16)
 
+**Status legend:** ✅ DONE — shipped and verified in code | ⏳ PENDING — not yet implemented
+
+| Section | Status |
+| --- | --- |
+| §A Pi CLI extension (`extensions/mlx-bun-adapter.ts`) | ✅ DONE |
+| §B Web chat adapter selector (`src/web/app.html` + `before_provider_request` hook in `pi-web.ts`) | ✅ DONE |
+| §C `/v1/adapters/available` discovery (`src/server.ts:1182`, `src/lora.ts:139`) | ✅ DONE |
+| §D Overnight training from the web UI — monitor/stream endpoint + UI | ⏳ PENDING |
+| §E `--force-adapter` CLI flag (absent from `src/cli.ts`) | ⏳ PENDING |
+| Repo-id keying written into adapter configs at save time | ⏳ PENDING |
+
 Working notes for wiring trained LoRA adapters through the whole stack: Pi (CLI +
 web), the web Chat page UI, and the server's discovery surface. Captured from the
 2026-06-16 session (serving verified working; pi-extension mechanism confirmed from
@@ -31,16 +42,16 @@ API should let you move an adapter between these states explicitly.
 So **loaded** and **selected** are done. The gap is **available** (discovery) plus
 the two front-ends (Pi extension, web Chat selector).
 
-## TODO
+## Shipped (§A–§C) ✅
 
-### A. Pi CLI/terminal — adapter via extension  *(default none)*
+### A. Pi CLI/terminal — adapter via extension  *(default none)* ✅ DONE
 
-Mechanism (confirmed from the pi docs in-session): a Pi extension using the
-`before_provider_request` hook to inject the field into the outgoing
-OpenAI-compatible payload. `models.json` can't add arbitrary body fields — the hook
-is the right tool.
+Wired a Pi extension using the `before_provider_request` hook to inject the `adapter`
+field into the outgoing OpenAI-compatible payload. `models.json` can't add arbitrary
+body fields — the hook was the right tool.
 
-Create `~/.pi/agent/extensions/mlx-bun-adapter.ts`:
+Shipped as `~/.pi/agent/extensions/mlx-bun-adapter.ts` (copy in-repo + one-liner
+installer):
 
 ```ts
 import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
@@ -60,53 +71,44 @@ export default function (pi: ExtensionAPI) {
 - Selection via `PI_MLX_ADAPTER=chunk pi --provider mlx-bun --model local -p "..."`
   (no native `-p --adapter` flag; unknown CLI flags get rejected). Optional shell
   helper: `pi-chunk() { PI_MLX_ADAPTER=chunk pi --provider mlx-bun --model local -p "$@"; }`.
-- Put it in the global `~/.pi/agent/extensions/` dir so it loads for non-interactive
-  `-p` too. `/reload` to pick it up.
-- **Ship this** as part of mlx-bun (a copy in-repo + a one-liner installer), so users
-  don't hand-write it. Consider also reading the *selected* adapter from a small
-  state file the web UI / a `mlx-bun adapter use <id>` command writes, so CLI and web
-  share one selection.
+- Lives in the global `~/.pi/agent/extensions/` dir so it loads for non-interactive
+  `-p` too. `/reload` picks it up.
+- Also reads the *selected* adapter from a small state file the web UI / a
+  `mlx-bun adapter use <id>` command writes, so CLI and web share one selection.
 
-### B. Web Chat page — selector + load/unload  *(default none)*
+### B. Web Chat page — selector + load/unload  *(default none)* ✅ DONE
 
 Path: `src/web/app.html` (UI) ↔ `/ws/chat` (`src/pi-web.ts`) ↔ in-process Pi ↔ model.
 
 - **Selector** in `app.html`: a dropdown of *available* adapters with **"none"
   selected by default**. Changing it sets the active selection for subsequent
-  messages. Allow multi-select → `"a+b"` (resolveSpec already supports stacking).
+  messages. Multi-select → `"a+b"` (resolveSpec already supports stacking).
 - **Load / unload controls**: buttons that call `POST /v1/adapters` (load) and
-  `DELETE /v1/adapters/:id` (unload). Show each adapter's state (available / loaded /
-  selected) so the three concepts are visible. An adapter must be **loaded** before
-  it can be **selected** (selecting an unloaded id errors) — UI should either
-  auto-load on select or gate selection to loaded ones.
-- **Inject via the `before_provider_request` hook — the primary mechanism, identical
-  to the CLI (§A).** `pi-web.ts` ALREADY registers an inline extension this way
-  (`pi.on("tool_call", …)` wired into `createAgentSession`, pi-web.ts:469/605), so
-  `pi.on("before_provider_request", (e) => ({ ...e.payload, adapter }))` drops in the
-  same way and reads the current selection from the WS session state. **No custom
+  `DELETE /v1/adapters/:id` (unload). Each adapter's state (available / loaded /
+  selected) is shown so the three concepts are visible. Selecting an unloaded adapter
+  auto-loads it (fires `POST /v1/adapters`) before injecting the id.
+- **Injected via the `before_provider_request` hook — identical mechanism to the CLI
+  (§A).** `pi-web.ts` registers the inline extension in `createAgentSession`
+  (pi-web.ts:469/605), so `pi.on("before_provider_request", (e) => ({ ...e.payload,
+  adapter }))` reads the current selection from WS session state. **No custom
   provider plumbing, no new model-call wiring.** The hook is the spine for BOTH UIs.
-- **Selection → hook**: the dropdown just updates the session's selected-adapter value
-  that the hook reads — a tiny client→server message (or reuse the `set_thinking`-style
-  toggle path), NOT a heavyweight protocol. Default none → hook returns the payload
-  unchanged → base model.
-- **Mounting is the only thing the hook can't do**: the injected id must be mounted
-  server-side or `resolveSpec` rejects it. Auto-load-on-select handles it — selecting
-  fires one `POST /v1/adapters`, then the hook injects the id. Discovery (§C) and a
-  richer protocol are optional polish; the dropdown can start from `GET /v1/adapters`.
-
-### C. Server — adapter **discovery** (the missing "available" list)
-
-- Add a store the server scans for available adapters:
-  - `~/.cache/mlx-bun/adapters/` (the trainer already defaults saves here —
-    `src/server.ts:1653`) and/or a **sidecar next to the model dir**.
-- New endpoint, e.g. `GET /v1/adapters/available` →
+- **Selection → hook**: the dropdown updates the session's selected-adapter value via
+  a small client→server message (reusing the `set_thinking`-style toggle path). Default
+  none → hook returns the payload unchanged → base model.
+
+### C. Server — adapter **discovery** (`GET /v1/adapters/available`) ✅ DONE
+
+- Scans `~/.cache/mlx-bun-finetunes/` and `~/.cache/mlx-bun/adapters/`
+  (`src/server.ts:1192`) and any sidecar next to the model dir.
+- `GET /v1/adapters/available` (`src/server.ts:1182`, `src/lora.ts:139`) →
   `[{ id, path, source_repo, rank, scale, compatible }]`, where each entry is read
   from the adapter's `optiq_lora_config.json` (`source_model`, `rank`, `scale`).
-- **Filter by the served model.** Today `source_model` is a *machine/snapshot-specific
-  path* (`/Users/joshrossi/.cache/huggingface/.../snapshots/<hash>`), which won't
-  match across machines/snapshots. Key compatibility on the **repo id**
-  (`mlx-community/MiniCPM5-1B-OptiQ-4bit`) instead — see "repo-id keying" below. The
-  server already tracks `repoId` per model; `compatible = adapter.source_repo === served.repoId`.
+- **Filters by the served model** on **repo id**
+  (`mlx-community/MiniCPM5-1B-OptiQ-4bit`) rather than the fragile
+  machine/snapshot-specific path. The server already tracks `repoId` per model;
+  `compatible = adapter.source_repo === served.repoId`.
+
+## TODO (§D–§E)
 
 ### D. Overnight training from the web UI
 
@@ -117,8 +119,10 @@ Training must be fully drivable from the web UI — not just a script.
   (rank / scale / num_layers / LR / dropout / iters / seq / segmentSize — the
   configurable-params task, mlx-lm defaults), **start**, and leave it running.
 - **Monitor**: live loss / iter / ETA, and survives the browser closing (the run is
-  server-side; the page reconnects). `POST /v1/fit` already exists
-  (`src/server.ts:1652`) — needs a status/stream endpoint + a UI.
+  server-side; the page reconnects). The training backend already exists:
+  `POST /api/finetune/submit` (`src/server.ts:1781`). What §D still needs is a
+  status/stream endpoint + the overnight-monitoring UI. (Note: `GET /fit` is the
+  unrelated memory-fit solver in `src/fit.ts` — not a training endpoint.)
 - **Completion**: the finished adapter lands in the store and shows up as *available*
   (state A in the three-state model), ready to load/select or auto-serve.
 - This closes the loop with the chunk-task roadmap: iterate data → train overnight →

package/docs/design/compat-cli-surface-design.md

@@ -6,6 +6,8 @@ modes"). This doc is the surface contract: every verb, the flag
 vocabulary, how optiq is absorbed, the parity harness, and **the
 matrix of what we still lack**.
 
+**Update 2026-06-17:** Several items have shipped since the 2026-06-12 snapshot: **LoRA training** is live (`src/train/`, `scripts/chunk-finetune.ts`, segmented-backward Phase A MiniCPM5 + Phase B e4b both merged); the **web UI fine-tune tab** (`src/web/app.html`) is live with SFT/DPO workflow, loss curve, merge, and export; the **chat adapter selector** (web UI) is live; and **adapters end-to-end** (web selector + CLI `/adapter`) are done — the matrix below reflects these as ✅. The **Phase 17 compat parity harness itself** (the verb router, dotted aliases, flag-surface parity tests) remains **unbuilt**.
+
 ## Goal
 
 Design mlx-bun's entire CLI/flag surface from scratch (nothing is
@@ -54,7 +56,7 @@ shared subset; optiq-only features are additive (marked ⊕).
 | `chat` | mlx_lm.chat | engine ✅, web chat ✅; no terminal REPL (`tui.ts` is help-formatting) | build REPL frontend over `generate` |
 | `server` | mlx_lm.server / optiq serve | ✅ `serve` | rename flags; ⊕ `--kv-config`, `--anthropic` honored (default on) |
 | `convert` | mlx_lm.convert + optiq convert | ❌ | quant pipeline; ⊕ `--target-bpw/--candidate-bits/--reference` |
-| `lora` | mlx_lm.lora + optiq lora | serve/hot-swap ✅, train ❌ | `--train/--test`; ⊕ `--rank-scaling by_bits\|by_kl`; `lora info` |
+| `lora` | mlx_lm.lora + optiq lora | serve/hot-swap ✅, train ✅ | `--train/--test`; ⊕ `--rank-scaling by_bits\|by_kl`; `lora info` |
 | `fuse` | mlx_lm.fuse | ❌ | fuse adapters, dequant, GGUF export, upload |
 | `cache_prompt` | mlx_lm.cache_prompt | runtime ✅, CLI ❌ | precompute + save reusable KV cache file |
 | `benchmark` | mlx_lm.benchmark / optiq benchmark | ✅ `benchmark` | flag-name parity (`-p/-g/-b/-n`); ppl half ❌ |
@@ -83,7 +85,7 @@ mlx-lm" half.
 
 ### pi — the built-in agentic coding CLI
 
-(Phase 16; full plan in **docs/pi-builtin-investigation.md**.) pi is an
+(Phase 16; full plan in **docs/investigations/pi-builtin-investigation.md**.) pi is an
 MIT, Bun-native (`bun build --compile`) coding agent — the "CLI
 experience inside our package" that neither Python package has. Two
 commands, one for each audience:
@@ -111,8 +113,8 @@ split.
 The web UI's agent backbone is pi's RPC / `AgentSession.subscribe()`
 event stream (investigation Option 4) — the SAME events drive the
 embedded TUI and the browser, so tool calls / steering / progress come
-for free. Tiles: chat ✅; quantize / fine-tune / training-data gate on
-the capability matrix.
+for free. Tiles: chat ✅; fine-tune ✅ (shipped: SFT/DPO workflow, loss curve, merge,
+export); quantize / training-data gate on the capability matrix.
 
 ## Flag-vocabulary reconciliation (where we currently diverge)
 
@@ -150,7 +152,7 @@ column; the 🟥 column is the capability matrix that fills in behind it.
 | KV-cache quant: bf16 / uniform / **mixed per-layer** (kv_config.json) | `server` (`off`/`N`/`config`) | ✅ | done (Phase 9/10; config.ts reads, generate.ts applies per-layer) |
 | KV sensitivity *profiler* (authors a NEW kv_config.json) | `kv-cache` | 🟥 | model-prep, low pri (shipped artifacts already include one) |
 | TurboQuant — optional extra quant *method* (rotation VQ) | scheme, not a verb | 🟥 | Phase 13 (KV plumbing already exists) |
-| **LoRA / DoRA / full training** | `lora --train` | 🟥 | Training (new phase) |
+| **LoRA / DoRA / full training** | `lora --train` | ✅ | Training (done: Phase A MiniCPM5 + Phase B e4b) |
 | **Sensitivity-aware rank scaling** | `lora --rank-scaling` | 🟥 | Training |
 | **Fuse adapters (+GGUF/upload)** | `fuse` | 🟥 | Training |
 | **lm-eval-harness tasks** | `evaluate` | 🟥 | Eval (new phase) |
@@ -159,7 +161,7 @@ column; the 🟥 column is the capability matrix that fills in behind it.
 | **HF upload** | `upload` | 🟥 | Distribution |
 | **Distributed share** | `share` | 🟥 | Distribution (lowest pri) |
 | **Web UI: quantize** | `lab`/UI | 🟥 | gated on Quantize |
-| **Web UI: fine-tune** | `lab`/UI | 🟥 | gated on Training |
+| **Web UI: fine-tune** | `lab`/UI | ✅ | done (SFT/DPO workflow, loss curve, merge, export) |
 | **Web UI: training-data template + gen** | `lab`/UI | 🟥 | new (no engine dep) |
 | Web UI: chat | `/chat` | ✅ | done (publishing gate 2) |
 | **Native: pi embedded** (`mlx-bun pi`, in-process TUI) | — | ✅ (P3/P4) | Phase 16 (flagship) |
@@ -177,7 +179,7 @@ does. The real capability work splits along two axes:
   optional and not inference plumbing: a *profiler* to author a new
   kv_config.json (model-prep, low pri) and extra methods like TurboQuant
   (Phase 13).
-- **Training**: LoRA/DoRA/full + `fuse`.
+- **Training**: LoRA/DoRA/full training ✅ (shipped); `fuse` verb (adapter merge, GGUF export, upload) 🟥 remaining.
 - **Eval**: harness + `perplexity`.
 - **Distribution**: `upload`/`share`.
 
@@ -221,7 +223,7 @@ correctness diffs need no clean machine.
 3. Inference-time KV quant → already done (bf16 / uniform / mixed
    per-layer via kv_config.json, Phase 9/10). Optional later: a profiler
    to author new configs (low pri) + the TurboQuant method (Phase 13).
-4. Training → `lora --train`/`fuse` + web-UI fine-tune tile.
+4. Training → ✅ `lora --train` + web-UI fine-tune tile (both shipped); remaining: `fuse` verb (adapter merge, GGUF export, upload).
 5. Eval → `evaluate`/`perplexity`.
 6. Distribution → `upload`/`share`.
 7. Web UI training-data tile (independent of engine work).