mlx-bun 0.0.4

package/docs/investigations/lab-build-journal.md

@@ -0,0 +1,157 @@
+# Build journal — mlx-bun Lab (web UI + native quantize/train/dataset + pi web chat)
+
+*Branch `web-ui-and-native-lab`, 2026-06-13/14. Visual summary:
+[archive/mlx-bun-lab-report.html](../../archive/mlx-bun-lab-report.html). Evidence
+artifact: `benchmarks/lab-verification.json` (regenerable, gitignored — produced by
+`scripts/verify-lab.ts`).*
+
+## Goal
+
+Re-implement the whole mlx-bun web surface as one beautiful, keynote-grade SPA
+with five sections — a **full pi-agent chat**, plus **Quantize**, **Fine-tune**,
+and **Build Dataset** ported from the OptIQ "lab" — while keeping the status
+page. Compute is **native Bun/MLX (no Python)** per Josh's decision.
+
+## The decisions that shaped it
+
+1. **The big fork (asked up front).** Quantize + fine-tune compute could be (a)
+   native via mlx-c, (b) orchestrate the installed OptIQ Python, or (c) proxy
+   the running `optiq lab`. Josh chose **(a) native** — truest to mlx-bun's "the
+   buggy layer doesn't need Python" thesis. Chat = **full pi agent** (all tools).
+   Verification = **real tiny runs, pausing for go-ahead** before heavy jobs.
+
+2. **Gate the riskiest unknown first.** Native LoRA needs autograd across the
+   FFI boundary. Rather than discover mid-build that it's infeasible, the first
+   thing built was `spikes/phase-train-vag.ts`: a tiny LoRA-shaped loss
+   (`mean((x + (x@A)@B − y)²)`) differentiated by `mlx_value_and_grad` through a
+   `bun:ffi` JSCallback closure, checked against central finite differences.
+   **PASS (<1e-2), exactly 2 grads returned (base frozen).** Only then was the
+   trainer worth writing. This single check de-risked the entire subsystem.
+
+3. **One SPA, hash-routed, text-imported.** Kept the exact single-binary
+   embedding (`with { type: "text" }`) the status page already used — no
+   bundler, no asset-resolution to patch in `build-binary.sh`. Design tokens
+   lifted from `status-page.html` so the five sections feel like one product.
+
+4. **Mixed job-execution model by safety class.** Async Metal command-buffer
+   OOM is **uncatchable** (it `std::terminate`s). So GPU jobs (quantize,
+   finetune) run as **crash-isolated child `mlx-bun` subprocesses** behind a GPU
+   lease; dataset jobs (pure JS + loopback HTTP) run **in-process**. Both write
+   the same NDJSON event log + SQLite row, so the SSE/tail path is uniform.
+
+5. **Open-book ports, not inventions.** Each engine follows a readable source:
+   dataset → `optiq/lab/dataset_templates.py` (13 templates, field-for-field);
+   trainer → `optiq/lora/{trainer,dpo,config}.py` + mlx-lm `default_loss`;
+   quantize → mlx-lm `convert.py` rules + the on-disk OptIQ model format; jobs →
+   `optiq/lab/{jobs,db}.py`; chat → pi SDK `examples/sdk/12-full-control.ts`.
+
+## How it was built — parallel orchestration
+
+After laying the shared job contract (`src/jobs/types.ts`), six subsystems were
+fanned out to concurrent sub-agents on **non-overlapping files**, then
+integrated centrally:
+
+| subsystem | files | how verified |
+|---|---|---|
+| shared jobs | `src/jobs/*` | 12 tests (lifecycle, GPU lease, crash isolation, zombie, SSE) |
+| dataset | `src/dataset/*` | 32 tests (generators, CSV, 90/10 split, 13 templates) |
+| quantize | `src/quantize/*` | 6 tests + real-model e2e |
+| trainer | `src/train/*`, `src/mlx/autograd.ts` | 19 fast + real-model e2e |
+| web SPA | `src/web/app.html` | 7 in-process route tests |
+| pi embed | `src/pi-web.ts` | 9 unit + live chat turn |
+
+Shared/owned-by-me files were edited centrally to avoid conflicts:
+`src/server.ts` (websocket + Lab routes + serve the SPA), `src/mlx/{ffi,ops}.ts`
+(autograd/random/save bindings), `src/cli.ts`, `package.json` (pinned
+`@earendil-works/pi-coding-agent@0.79.3`).
+
+## Key technical findings
+
+- **`mlx_save_safetensors` exists** (mlx-c `io.h`) — the quantizer writes real
+  HF-layout shards natively; no hand-rolled byte writer, no dtype/alignment bugs.
+- **`mlx_value_and_grad` + `mlx_closure_new_func_payload`** are drivable from a
+  `bun:ffi` JSCallback (the repo already uses one for the mlx error handler).
+  The closure swaps the differentiated primals into `LoraWeights.{a,b}`, builds
+  the loss graph, and restores; mlx's graph keeps its own refs, so only A/B
+  differentiate and the quantized base stays frozen.
+- **No optimizer in mlx-c** → AdamW hand-rolled as array ops (m/v EMA, bias
+  correction, decoupled weight decay), each step swapping the leaf handle back
+  into the `LoraWeights` so the next forward sees the update.
+- **pi tool-approval gate** is a real pre-execution hook: an inline extension
+  via `DefaultResourceLoader.extensionFactories` registers `pi.on("tool_call")`
+  returning `{block:true}` to deny — read-only tools auto-allow, bash/edit/write
+  await a browser decision (120s auto-deny).
+
+## Bugs found and fixed (during the real-model runs)
+
+- **LoRA init key shape** — `mlx_random_normal` got a `[1,2]` key (a row slice of
+  the split-keys array) but mlx wants `[2]`. Reshaped the per-module subkey.
+- **Quantize e2e path** — the gated test passed a model *id* to
+  `quantizeModelDir` instead of a resolved snapshot *path* (the job runner
+  resolves via the registry; the direct call didn't). Pointed it at
+  `SNAPSHOT_MINICPM5`.
+- **Flaky chat test** — the agent sometimes takes a tool-call turn before
+  emitting text; the test resolved on the first `turn_end` and saw no tokens.
+  Fixed to settle on the first turn that actually streams text. The embed was
+  always correct (a debug run showed 25+ text deltas + a tool execution).
+
+## Measured evidence (M4 Pro, MiniCPM5-1B, on-disk)
+
+- **Quantize:** 170 modules → **4.50 bits/weight**, **0.62 GB** dir, **7.0 s**,
+  reopened correctly through the loader (quantized embed_tokens as `uint32`).
+- **Train:** 20 SFT iters, **loss 4.48 → 0.68** in **4.6 s**, 168 adapters
+  mounted. Behavior changed — on "Tell me about your day": base replies normally,
+  the UPPERCASE-fixture adapter replies `"MY DAY AHEAD. APPROVE AND PROPERLY…"`.
+- **Chat:** a real WebSocket turn streamed text + executed a tool to completion.
+- **Tests:** 85+ new, all green; `tsc --noEmit` clean; existing server suite
+  17/17 (no regressions).
+
+## Scope: shipped vs designed-seam
+
+- **Shipped (v1):** full SPA (5 sections), pi web chat with tools + approval
+  gate, native uniform-affine quantize, native SFT + DPO LoRA, 12/13 dataset
+  templates, shared job system, real tiny-run verification.
+- **Designed seams (follow-ups):** ~~OptIQ sensitivity+knapsack mixed-precision
+  quantization; batched (B>1) training; `hf_dataset_import`; HF push-to-hub;
+  folding pi's assets into the compiled single binary.~~ **ALL DONE in Round 2 —
+  see below.**
+
+## Round 2 — the leftover seams, ported 1:1 (2026-06-14)
+
+Josh: "all of those, port, one to one functionality." Built the five remaining
+items as faithful ports of the OptIQ / mlx-lm source (open book), in parallel on
+non-overlapping files, integrated centrally (server routes + one `app.html` UI
+pass), and verified.
+
+- **Mixed-precision quantization** (`src/quantize/{allocator,sensitivity,
+  calibration}.ts`) — `optimize_mixed_precision` ported **1:1** from
+  `optiq/core/optimizer.py` (greedy knapsack + block-aware floor + protected
+  layers + block-run guard + the uniform-vs-bf16 `_kl_reduction` sign detection).
+  Verified **bit-identical to OptIQ** on shared synthetic inputs by spawning the
+  venv Python in the test (4/4 cases). Per-layer KL sensitivity
+  (`analyze_sensitivity_exact`) runs on our graph: dequantize each layer's weight
+  to bf16, re-quantize at each candidate bit, swap into the running
+  `QuantizedLinear`, forward on a WikiText-2 calibration set, KL vs reference.
+  Real e2e: MiniCPM5-1B at target 5.0 bpw → 80×4-bit + 90×8-bit, **5.19 bpw, 32 s**.
+- **Batched (B>1) training** (`src/train/{dataset,forward,loss}.ts`) — ragged
+  padding + a `[B,1,L,L]` causal-AND-key-validity mask via a cache wrapper (no
+  model/op edits); B=1 stays a separate bit-identical path. Bug found: comparing a
+  length-4 reference to a length-6 batched row conflated mask correctness with
+  bf16 sequence-length rounding — the corrected shape-held parity test passes,
+  confirming the mask isolates rows.
+- **`hf_dataset_import`** (`src/dataset/generators.ts`) — the 13th template via
+  the HF datasets-server REST API (pagination, config resolve, label/min-chars/
+  cap filters, three output formats).
+- **HF push-to-hub** (`src/hf-push.ts`) — native Hub upload (create repo → LFS
+  preupload → S3 PUT → NDJSON commit) for model/adapter/dataset repos, token at
+  `~/.mlx-bun/hf.json`. Mock-server tested end-to-end (11 tests); real uploads
+  untested by design (would publish under the user's account).
+- **Single-binary web chat** (`scripts/build-binary.sh`) — pi's JS already
+  bundles under `bun build --compile`; only `photon_rs_bg.wasm` is sidecarred.
+  The compiled binary's headless pi session was verified to construct.
+
+UI: one `app.html` pass enabled the Uniform/Mixed quantize toggle + target-bpw
+slider, live merge/export/push on the fine-tune done step, push on quantize &
+dataset, and an HF-token settings gear. New server routes:
+`/api/finetune/{merge,export}`, `/api/{quantize,finetune,dataset}/push`,
+`/api/settings/hf-token`, + mixed-precision params on quantize submit.

package/docs/investigations/pi-builtin-investigation.md

@@ -0,0 +1,308 @@
+# Investigation: pi as a "built-in" terminal for mlx-bun
+
+*2026-06-12. Question (PRODUCT_ROADMAP "Harness UX" / PLAN direction A):
+can `mlx-bun pi` drop the user into a working pi agent session against
+the locally running model — and what's the best mechanism? Sources:
+https://pi.dev/docs/latest (sdk, tui, rpc, custom-provider pages) plus
+direct inspection of the installed packages
+(`~/.bun/install/global/node_modules/@earendil-works/*`, v0.79.1).*
+
+> **Outcome (2026-06-14, what actually shipped).** This investigation
+> proposed shipping the subprocess launcher first and promoting to an
+> embedded flagship later. We went straight to the embedded design and
+> then **dropped the subprocess path entirely** (Option 2 + the
+> `--external-pi` fallback in P4): `mlx-bun pi` runs pi's own
+> `InteractiveMode` **in-process** (src/pi-terminal.ts) with a custom
+> mlx-bun coding-agent prompt, full coding tools + web tools, and pi's
+> native approval prompts — nothing to install. A user who already runs
+> pi connects it to the local model with **`mlx-bun harness pi`**
+> (Option 1, src/harness-pi.ts) — that's the only "their pi" path, since
+> launching a pi they already have was pure duplication. The web-chat
+> half (Option 4) ships via src/pi-web.ts. The analysis below is the
+> point-in-time reasoning that led here.
+
+## TL;DR
+
+**Yes, and better than hoped.** pi is not just embeddable — it is a
+Bun-native, MIT-licensed npm package whose own release binary is built
+with `bun build --compile`, the exact toolchain mlx-bun's Phase 5
+single-binary bundle already uses. The full interactive TUI (editor,
+chat history, all built-in commands) is an exported SDK class
+(`InteractiveMode`), not CLI-only internals. Four integration options
+exist, they are not mutually exclusive, and the recommended path is:
+ship the cheap subprocess launcher now, promote to a true in-binary
+embed as the flagship. pi's RPC mode separately gives us the protocol
+for the built-in web chat UI, so this investigation pays for direction
+A's part 1 as well.
+
+## What pi is, technically
+
+- Four MIT-licensed packages, all v0.79.1, all from
+  `github.com/earendil-works/pi`:
+  `pi-coding-agent` (13 MB, the agent + CLI + SDK), `pi-ai` (4.9 MB,
+  provider/streaming layer), `pi-agent-core` (1.3 MB), `pi-tui`
+  (1.9 MB, differential-rendering TUI lib). ~21 MB total on disk
+  before tree-shaking — negligible next to model weights, modest next
+  to mlx-bun's current binary.
+- **Bun-compile native**: `pi-coding-agent`'s `build:binary` script is
+  literally `bun build --compile ./dist/bun/cli.js ...` — upstream
+  ships a Bun single-file executable themselves. Embedding it in our
+  Bun bundle is a supported configuration, not an experiment.
+- Engines: node ≥ 22.19 (pi-tui uses `/v`-flag regexes — fails under
+  node 18, fine under Bun ≥ 1.3.x, which mlx-bun already pins).
+- Assets that must ride along in a binary (upstream's
+  `copy-binary-assets` is the manifest): theme JSONs, a few PNGs,
+  export-html templates, `photon_rs_bg.wasm` (image ops), and pi-tui's
+  prebuilt `.node` natives (`darwin-arm64`/`darwin-x64` ship in the
+  package).
+- Local-model support is **already proven on this machine**:
+  `~/.pi/agent/models.json` has an `mlx` provider
+  (`api: "openai-completions"`, baseUrl + `sk-optiq-` key, compat
+  flags) serving Qwen quants. mlx-bun's OpenAI surface is what pi
+  expects.
+
+## The four integration options
+
+### Option 1 — `mlx-bun harness pi`: configure the user's existing pi
+
+Write/merge the provider entry into `~/.pi/agent/models.json` (or drop
+a tiny extension that calls `pi.registerProvider(...)` — the
+documented programmatic route, which can even discover models
+dynamically from our `/v1/models`). Validate with one tiny request,
+print the launch command.
+
+- **Effort**: hours. **Risk**: low (must merge, not clobber, user
+  config — the docs' extension route avoids touching models.json at
+  all). **Verdict**: do regardless; this is the harness-adapter
+  pattern from the roadmap and it's nearly free.
+
+**Pros**: nearly free; respects the user's pi setup, sessions, themes,
+extensions; zero version coupling; the dynamic-discovery extension
+keeps the model list in sync with our server automatically.
+**Cons**: requires pi to already be installed; not "built-in" — the
+appliance story still has two moving parts; config-merge bugs are the
+classic way to annoy users (the extension route mostly avoids this).
+
+### Option 2 — `mlx-bun pi` (v1): spawn the user's installed pi
+
+Start our server, then spawn `pi` with a session-scoped provider
+config (extension dir or env) pointing at it. If pi isn't installed,
+print the install one-liner.
+
+- **Effort**: a day with polish. **Risk**: version skew with whatever
+  pi the user has; not "built-in" — two installs.
+- **Verdict**: the right v1. Real UX win immediately, zero coupling.
+- **Bonus**: unrecognized flags pass straight through, so
+  `mlx-bun pi -p "summarize @notes.md"` (pi's non-interactive print
+  mode), `--mode json/rpc`, `--continue`, `--resume` etc. all work on
+  day one without us implementing anything.
+
+**Pros**: fast to ship; full pi feature surface via passthrough;
+upgrades to pi arrive for free; failure modes are simple (no pi →
+print install hint). **Cons**: still two installs; we can't control
+the pi version (an upstream flag/behavior change can break our
+launcher); per-run provider injection needs care so we don't pollute
+the user's global config; process lifecycle (server + child TUI)
+needs clean shutdown handling.
+
+### Option 3 — SDK embed (the real "built-in"): pi inside the mlx-bun binary
+
+This is the one the question was really about, and the SDK is
+explicitly designed for it:
+
+- `createAgentSession({...})` builds a session; the
+  `examples/sdk/12-full-control.ts` example shows **zero-discovery**
+  embedding: in-memory `SessionManager`/`SettingsManager`/
+  `ModelRegistry`, custom `ResourceLoader`, explicit tool list,
+  runtime API keys — nothing leaks from/to `~/.pi` unless we want it
+  to.
+- `InteractiveMode` is exported: "full TUI interactive mode with
+  editor, chat history, and all built-in commands", constructed from
+  an `AgentSessionRuntime`. We do not rebuild a chat TUI from pi-tui
+  primitives (the tui docs are explicit that components are for
+  extensions, not a full-app kit) — we run *pi's own* TUI in-process.
+- Provider wiring: register an `openai-completions` provider at our
+  own server's baseUrl (loopback HTTP into our already-running
+  `Bun.serve` — simplest and keeps one code path for external
+  clients), models listed from our registry with real context-window
+  numbers per device profile.
+- Flow for `mlx-bun pi`: detect profile → ensure model → start server
+  in-process → `createAgentSession` against `localhost` →
+  `new InteractiveMode(runtime).run()`.
+
+Open engineering questions (all look tractable, none look fatal):
+
+1. **One process, two workloads**: the TUI event loop and the
+   server/generation queue share the JS thread; mlx compute happens in
+   native code but graph-build is JS-side (~ms-scale per decode step).
+   TUI rendering is differential and cheap; needs a smoke test for
+   input latency during decode, not a redesign.
+2. **Bundle plumbing**: embed pi's asset list + pi-tui `.node`
+   prebuilds + photon wasm into the Phase 5 compile. Upstream's own
+   binary build proves each piece works under `bun build --compile`;
+   our work is making *their* asset-path resolution find files inside
+   *our* bundle.
+3. **Version pinning**: pi moves fast (0.79.x); pin exact, upgrade
+   deliberately, keep Option 2 as the fallback path when upstream
+   breaks SDK surface.
+4. **Identity**: embedded pi should read/write its own session dir
+   (`~/.mlx-bun/pi-sessions` or similar) and not fight a user's
+   standalone pi install. The full-control example shows every knob
+   needed (`agentDir`, custom storage paths).
+
+- **Effort**: ~1–2 weeks including bundle work and latency testing.
+- **Verdict**: feasible and high-value; this is the appliance moment —
+  one binary, one command, a real agent on a local model. `-p` and
+  `--mode rpc/json` remain supported natively: `runPrintMode` and
+  `runRpcMode` are SDK exports, so the embedded CLI keeps pi's
+  headless modes.
+
+**Pros**: the true appliance — one binary, no prior installs; we pin
+and test the exact pi version we ship; full control of defaults
+(session dir, tools, system prompt, model list per device profile);
+in-process events shared with the web UI; offline-clean.
+**Cons**: most engineering (asset bundling, latency validation); we
+own the upgrade treadmill for a fast-moving 0.x dependency; embedded
+pi diverges from the user's own pi config/extensions unless we
+deliberately opt into discovery; binary grows ~20 MB.
+
+### Option 4 — RPC / JSON event-stream mode (not for the terminal — for the web UI)
+
+`pi --mode rpc` speaks JSONL over stdin/stdout: `prompt`, `steer`,
+`abort`, model control, state/stats queries, plus a streamed event
+model (`message_update` deltas, `tool_execution_*`, `turn_*`) and even
+an extension-UI sub-protocol. The docs name "custom frontends (web
+dashboards…)" as a primary use case.
+
+- **Verdict**: wrong tool for `mlx-bun pi` (we'd be rebuilding pi's
+  TUI on top of pi), but the **right backbone for the built-in web
+  chat UI**: `mlx-bun` serves a page; the server bridges
+  WebSocket ↔ RPC events (or drives `AgentSession.subscribe()`
+  in-process — same event shapes, no subprocess). Tool calls,
+  steering, and progress come for free instead of us re-implementing
+  an agent loop in the browser.
+
+**Pros**: documented, versioned JSONL protocol; UI-agnostic (web,
+IDE, bots); steering/abort/stats built in; works with either the
+subprocess or the embedded session as the engine. **Cons**: not a
+terminal UX at all; strict framing rules to honor; the extension-UI
+sub-protocol must be bridged if extensions prompt the user.
+
+## Does this make sense as a feature?
+
+Yes — it is arguably the highest-leverage product move available:
+
+- It converts mlx-bun from "a server you point things at" into a
+  complete local-AI appliance, which is the PRODUCT_ROADMAP north
+  star verbatim.
+- The marginal cost is small because the hard parts are already done
+  on both sides: our OpenAI surface is SDK-verified, pi's SDK is
+  designed for embedding, and both projects compile with the same Bun
+  toolchain. MIT licensing removes the bundling question.
+- One investigation, both deliverables: the same `AgentSession` event
+  stream powers the embedded TUI *and* the web chat UI, so direction
+  A's two halves share a core.
+- Caveats worth stating: we take on pi's release cadence as a
+  dependency; the agent quality ceiling is the local model's (12B
+  OptiQ tool-calling is decent but this becomes the showcase — the
+  Lucien-direction evals would tell us exactly how good); and the
+  embedded TUI must not degrade decode throughput (measure, per house
+  rules).
+
+## Implementation plan
+
+To be explicit about the recommendation: **the locally installed pi is
+the v1 stepping stone, not the destination.** Users who have their own
+pi always get first-class support (P1); the flagship `mlx-bun pi` ends
+as the embedded, single-binary experience (P3/P4).
+
+### P1 — `mlx-bun harness pi` (hours–1 day)
+
+1. Detect pi (`which pi`, `~/.bun/bin/pi`, npm global); report version.
+2. Generate an extension file (e.g. `~/.pi/agent/extensions/mlx-bun.ts`)
+   that calls `pi.registerProvider("mlx-bun", ...)` with **dynamic
+   discovery**: fetch our `/v1/models` at pi startup so the model list
+   tracks the server. No models.json surgery; trivially reversible
+   (delete one file).
+3. Validate: tiny completion against the running server through pi's
+   provider path; print `pi --provider mlx-bun` launch hint.
+4. Failure modes: pi missing → print install one-liner; server down →
+   offer to start it.
+
+### P2 — `mlx-bun pi` v1: subprocess launcher (~1–2 days)
+
+1. Ensure server: reuse a running instance if healthy, else start
+   in-process and wait for readiness.
+2. Resolve the pi binary; if absent, offer the installer.
+3. Spawn pi with stdio inherited and a **session-scoped provider**: a
+   generated `-e <tmp-extension>.ts` registering the provider for this
+   run only (no global config writes), plus
+   `--provider mlx-bun --model <profile default>`.
+4. **Pass through all remaining argv** — this is where `mlx-bun pi -p
+   "..."`, `--mode rpc`, `--continue`, `@file` args work for free.
+5. Lifecycle: child exit → optionally keep the server warm (flag);
+   SIGINT goes to the child (pi handles double-ctrl-C), server shuts
+   down cleanly after.
+
+### P3 — embed spike (≤1 week, run via `bun run`, no bundling yet)
+
+1. Add pi packages as pinned deps (exact `0.79.x`).
+2. Wire `createAgentSession` full-control style: in-memory
+   registry/settings, our provider at the loopback baseUrl, session
+   dir under `~/.mlx-bun/`, default tools, our system prompt.
+3. Run `new InteractiveMode(runtime)` in the same process as the
+   server.
+4. **Gates** (house rule — numbers on this machine): editor latency
+   during 12B decode subjectively clean + decode tok/s within noise
+   of server-only; tool-call round-trip (read/bash/edit) works against
+   the 12B; clean teardown.
+5. Decide discovery policy: default isolated; `--pi-config` opt-in to
+   the user's `~/.pi` extensions/skills/themes.
+
+### P4 — single binary + shared event plumbing (~1 week)
+
+1. Fold pi's asset manifest (theme JSONs, PNGs, export-html templates,
+   photon wasm, pi-tui darwin `.node` prebuilds) into the Phase 5
+   `bun build --compile`; patch/verify pi's asset-path resolution
+   inside the bundle (upstream's own binary build is the existence
+   proof).
+2. CLI surface: `mlx-bun pi [pi-args]` → embedded `InteractiveMode`;
+   `-p` → `runPrintMode`; `--mode rpc` → `runRpcMode`. Subprocess
+   path (P2) remains as `--external-pi` fallback.
+3. Web chat UI rides the same `AgentSession.subscribe()` events over
+   WebSocket (Option 4 shape, in-process — no subprocess needed).
+4. Upgrade policy: bump pi deliberately per release, re-run the P3
+   gates as the regression suite.
+
+### First-run starter model (folds into P2/P4; model gated on Phase 14)
+
+The first-run experience must not be "watch a progress bar for 10
+minutes." Plan: a sub-gigabyte **starter model** downloads first and is
+chatting within ~1 minute — with a docs-grounded "tour guide" system
+prompt (README + server-api + pi guide preloaded, so a 1B doesn't
+hallucinate about mlx-bun) and a banner: *"the ideal model for your Mac
+is downloading in the background."* The profile model streams behind it
+(sequenced, not parallel, so downloads don't split bandwidth); swap
+when ready.
+
+Measured sizes (HF tree API, 2026-06-12):
+
+| Candidate | Download | Note |
+|---|---|---|
+| gemma-4-e2b-it-OptiQ-4bit | 5.26 GB | smallest Gemma — disqualifies the family for the starter slot |
+| gemma-4-e4b / 12B OptiQ-4bit | 7.0 / 8.4 GB | profile models, not starters |
+| **Qwen3.5-0.8B-OptiQ-4bit** | **0.89 GB** | **preferred**: rides the Phase 14 Qwen port; doubles as the first tier-a parity target |
+| MiniCPM5-1B-OptiQ-4bit | 0.92 GB | strong (52% MMLU, +4.44 OptiQ delta) but a third architecture family — A/B later |
+
+Decisions: starter is **downloaded, never bundled** (a ~900 MB binary
+kills brew install); it stays on disk afterward (future
+speculative-decoding draft + always-works fallback). Hot-swap (or
+offer-to-swap) when the profile model lands is the only new runtime
+work. Dependency: a runnable sub-GB model requires the Phase 14 Qwen
+bring-up — the starter feature is the reason the 0.8B goes first there.
+
+## Recommended sequence
+
+P1 + P2 now (days, immediate demo value, zero coupling) → P3 spike
+with its latency gate → P4 single-binary flagship, reusing the event
+plumbing for the built-in web UI.