mlx-bun 0.4.0 → 0.5.0

package/docs/reference/cli.md

@@ -29,6 +29,8 @@ each verb).
 | `harness pi` | connect your own pi install to the local server | — |
 | `generate` (`gen`) | one-shot generation, no server | `mlx_lm.generate` |
 | `embed` | text embeddings, no server | — |
+| `transcribe` | speech-to-text from an audio file, no server | — (`mlx_whisper` CLI is the nearest) |
+| `dictate` | push-to-talk microphone dictation → print / clipboard / typed keystrokes | — |
 | `get` | download a model (resumable, verified) | `mlx_lm.manage` (partial) |
 | `scan` | re-index the Hugging Face cache | `mlx_lm.manage --scan` |
 | `ls` | list downloaded models | `mlx_lm.manage --scan` |
@@ -68,7 +70,7 @@ Library requests support delayed uniform or per-layer affine conversion with
 qualified shared drafting providers, including rotating caches. CLI affine
 quantization starts at zero.
 
-The unreleased shared executor supports qualified Qwen MTP and start-zero
+The shared executor supports qualified Qwen MTP and start-zero
 TurboQuant at one or several active rows; `--batch 1` still selects the
 legacy serial control. No new CLI flag is required for the shared interfaces.
 Full-attention and rotating targets, including Llama, MiniCPM and Gemma, use shared bf16, uniform KV4/KV8, per-layer affine KV or TurboQuant
@@ -213,6 +215,65 @@ mlx-bun embed --instruct "Retrieve passages that answer the question" --text "wh
 | `[query]` / `--query <q>` | embedding model; omitted = the first downloaded embedding model (errors with a `get` suggestion when none) |
 | `--text "…"` | text to embed (omit to read one text per stdin line) |
 | `--instruct "…"` | Qwen3-Embedding query instruction; omit for plain document embeddings |
+
+### `transcribe` — speech-to-text
+
+Whisper transcription of one audio file with a local `mlx-community/whisper-*`
+checkpoint, no server. WAV decodes natively; other containers CoreAudio can
+read (mp3/m4a/aac/flac/ogg/aiff, mp4/mov audio…) decode in-process through
+AudioToolbox. Model support
+and the oracle contract: [models.md](models.md#supported-models).
+
+```sh
+mlx-bun transcribe meeting.m4a                       # first downloaded Whisper, language auto-detected
+mlx-bun transcribe talk.wav --language en --format srt > talk.srt
+mlx-bun transcribe note.wav --beam-size 5 --prompt "Sotto, SwiftUI, Metal" --verbose
+```
+
+| Flag | Meaning |
+|---|---|
+| `<audio-file>` | required positional |
+| `[query]` / `--model <path\|query>` | Whisper model; omitted = the first downloaded `whisper` checkpoint (errors with a `get` suggestion when none) |
+| `--language <code\|name>` | `en`, `japanese`, …; `auto` or omitted detects from the first 30 s |
+| `--task translate` | translate to English |
+| `--beam-size <n>` | beam search (default greedy) |
+| `--temperature <t>` | one temperature; default is mlx-whisper's `(0, 0.2, …, 1.0)` fallback ladder gated by compression-ratio / log-prob thresholds |
+| `--no-fallback` | temperature 0 only |
+| `--prompt "…"` | initial prompt (vocabulary hints, up to 223 tokens are kept) |
+| `--no-timestamps` | text-only decoding |
+| `--no-condition` | do not feed the previous window's text as the next prompt |
+| `--vad`, `--vad-threshold <t>`, `--vad-model <path>` | Silero VAD gate: no detected speech → empty output, Whisper never runs. Weights: `ggml-org/whisper-vad` in the HF cache (885 KB) or an explicit ggml path |
+| `--word-timestamps` | word-level timing in `verbose_json` (`words` per segment) |
+| `--faithful` | oracle-parity graph instead of the fast path (fused attention, compiled decoder step) |
+| `--audio-ctx <n>` | Lab: encode only n of the 1500 encoder positions (whisper.cpp `-ac`); degrades below ~1024 |
+| `--format <f>` | `text` (default) · `json` · `verbose_json` · `srt` · `vtt` |
+| `--verbose` | per-segment time ranges and a realtime-factor summary on stderr |
+
+### `dictate` — push-to-talk from the microphone
+
+Streams the mic (AVAudioEngine sidecar `mlx-bun-mic-capture`, built from
+`src/native/mic_capture.swift` on first use with swiftc, or shipped beside
+the binary) into a transcription session while you speak, so every
+finished 30 s window is transcribed during capture and the result lands
+about one window after you stop. Silence never runs Whisper (Silero VAD).
+macOS prompts for Microphone permission; `--hotkey` needs Input Monitoring
+and `--type` needs Accessibility for your terminal app.
+
+```sh
+mlx-bun dictate                              # Enter starts/stops a take; prints the text
+mlx-bun dictate --hotkey --copy              # hold Right Option, transcript → clipboard
+mlx-bun dictate --hotkey --type              # …typed into the frontmost app
+mlx-bun dictate --server http://localhost:8090 --vocabulary "Sotto,SwiftUI,Metal"
+```
+
+| Flag | Meaning |
+|---|---|
+| `[query]` / `--model <path\|query>` | Whisper model for in-process transcription (default: first downloaded `whisper` checkpoint) |
+| `--server <url>` | use a running server's `/v1/audio/sessions` instead |
+| `--hotkey [keycode]` | hold-to-talk on a macOS virtual keycode (default 61, Right Option) instead of Enter toggling |
+| `--language`, `--beam-size`, `--prompt`, `--vocabulary "a,b"`, `--no-vad` | decoding, as for `transcribe` |
+| `--idle-unload <s>` / `--resident` | release the weights after idle seconds (0 = after every take; default 30) or keep them loaded |
+| `--copy` / `--type` / `--type-delay <s>` | clipboard / System Events keystrokes into the frontmost app (delay defaults to 1 s in Enter mode so you can Cmd-Tab, 0 with `--hotkey`) |
 | `--json` | one OpenAI-style `{object:"list",data:[…]}` object instead of bare arrays |
 
 Server equivalent: `mlx-bun serve <embedding-model>` then

package/docs/reference/distribution.md

@@ -15,9 +15,33 @@ Three install channels ship from one release:
 | Direct download (`curl -fsSL https://mlx-bun.dev/install.sh \| sh`) | the same bundle, into `~/.mlx-bun` | one tarball, via curl |
 | npm / bunx (`bunx mlx-bun`, `bun install -g mlx-bun`) | a launcher + the TypeScript source, run under the user's Bun | the package, then the native runtime pack on first run |
 
-[docs/reference/distribution.md](./distribution.md) covers the *sidecar* use of the same bundle
-inside a Tauri/Electron app; the build and signing facts it relies on are
-documented here.
+The same bundle can run as a local worker inside a desktop application.
+See [Embedding in a desktop application](#embedding-in-a-desktop-application).
+
+## Embedding in a desktop application
+
+Include the complete release bundle in the application's resources and launch
+its executable by absolute path. Keep the native libraries, Metal library,
+helpers, and Pi assets together as listed below. Copying only the executable
+does not produce a self-contained installation.
+
+For a Bun host, the [library API](library-api.md) documents `openIsolatedHost`
+and its explicit executable command. A desktop host in another runtime can
+manage the worker process and use the [HTTP API](server-api.md). Keep transport
+and worker lifecycle in the application; request scheduling belongs to the
+engine. Stop the worker when the application no longer owns it.
+
+The model weights are separate from the executable bundle. Select an existing
+artifact or use the [model download commands](models.md) before starting the
+worker with that model. Follow the documented cache-location settings when
+the application manages its own storage.
+
+Repackaging within an application requires that application's signing and
+notarization workflow. The executable's hardened-runtime entitlements and
+native signing requirements are documented below. Include the third-party
+attributions with the distributed application. Test the installed application
+bundle, including worker launch and runtime assets, rather than relying only
+on a source-checkout run.
 
 ## What ships in the bundle
 
@@ -32,6 +56,7 @@ dir), ~80 MB compressed; `mlx.metallib` is the bulk of the installed size.
 | `libmlx.dylib`, `libjaccl.dylib`, `mlx.metallib` | MLX core, its distributed-comm dependency, the Metal kernels (libmlx loads the metallib from its own directory) | `/opt/homebrew/opt/mlx/lib` (override `MLX_LIB_DIR`) |
 | `libmlx_bun_expert_io.dylib` | bounded streamed-expert I/O for GLM-5.2 MoE serving | [scripts/build-expert-io.sh](../../scripts/build-expert-io.sh) |
 | `mlx-bun-frame-extract` | AVFoundation frame-extraction helper for video input (a Mach-O *executable*, not a dylib) | [scripts/build-frame-extract.sh](../../scripts/build-frame-extract.sh) → `swiftc -O src/native/frame_extract.swift` |
+| `mlx-bun-mic-capture` | AVAudioEngine microphone capture + CGEvent hold-key helper for `mlx-bun dictate` (Mach-O executable; the only native piece of the Whisper work — the web composer uses the browser mic) | [scripts/build-mic-capture.sh](../../scripts/build-mic-capture.sh) → `swiftc -O src/native/mic_capture.swift` |
 | `photon_rs_bg.wasm` | pi's image codec, resolved next to the executable by the web chat's `read`-on-image path (best-effort; pi degrades to a text note without it) | `node_modules/@silvia-odwyer/photon-node` (override `PHOTON_WASM`) |
 | `theme/*.json`, `assets/*.png`, `export-html/`, `package.json`, `CHANGELOG.md`, `native/darwin/prebuilds/<arch>/darwin-modifiers.node` | assets the embedded pi **terminal** (`mlx-bun pi`) resolves by path: themes, startup art, `/export` template, version banner + changelog, native modifier-key helper (degrades if absent) | `node_modules/@earendil-works/pi-coding-agent/dist`, `pi-tui` |
 
@@ -133,7 +158,7 @@ does, in order. Steps 1–3 are yours; the rest is the scripts.
    (`BUILD_DIR` overrides `dist`). Release builds are arm64-only; the script
    exits on any other host arch.
 5. **Sign nested code first** — every `*.dylib`, `*.node`, and the
-   `mlx-bun-frame-extract` helper, matched by name under the build dir, gets
+   `mlx-bun-frame-extract` / `mlx-bun-mic-capture` helpers, matched by name under the build dir, get
    `codesign --force --timestamp --options runtime -s "<identity>"`.
    Nested-before-container is required. The metallib, wasm, JSON, and PNG
    assets are data, not code, and are skipped.

package/docs/reference/environment.md

@@ -1,9 +1,8 @@
 # Reference environment and platform facts
 
-The canonical home for the oracle setup and the hard-won platform facts that
-agents and contributors need. CLAUDE.md points here; do not duplicate these
-facts elsewhere. Each fact is one paragraph with the evidence path that proves
-it — when you change the code, update the paragraph.
+Oracle configuration, native-runtime compatibility, and machine conditions
+for reproducing numerical and performance checks. Historical environment
+records below identify the versions used for their original comparisons.
 
 User-facing symptoms live in [troubleshooting.md](./troubleshooting.md).
 
@@ -21,8 +20,8 @@ Before asserting anything about RAM, OOM, or bandwidth, run
 reference `serve.sh` (below) hard-codes a 28 GB wired limit "of 32 GB", so it
 is written for the M1 Max.
 
-For the current M4 Pro optimization campaign, Josh specifies internal SSD
-storage for active models. The external SSD stores artifacts to copy when
+The M4 Pro optimization measurements use internal SSD storage for active
+models. The external SSD stores artifacts to copy when
 needed; external-drive loading performance is outside normal acceptance for
 the 12–17 GB models. Stage each needed artifact under `~/models/`, verify
 every file against its archive, and record that path in both benchmark arms.
@@ -34,10 +33,24 @@ user-local eval DB (`~/.cache/mlx-bun/evals.sqlite`) and promoted to
 `docs/reference/benchmarks.md` deliberately. `bun scripts/bench-serve.ts all`
 is the benchmark harness; it writes Markdown and raw JSON under `reports/`
 as gitignored outputs. Numbers on a loaded machine are
-garbage — run-to-run spread is the stability signal, and the harness retries
+diagnostic; run-to-run spread is the stability signal, and the harness retries
 unstable cells (`scripts/bench-serve.ts`; `benchmarks.md`, "Running the
 benchmark").
 
+A performance decision must account for concurrent work during the timed
+comparison. For unstable pairs, capture process activity, GPU utilization and
+VM/page-in changes before and during the repeat; keep timestamps that align
+those observations with each measured request or forward. A preflight snapshot
+alone cannot identify interference during a run. CPU usage or retained swap
+alone does not establish GPU contention or active paging. Preserve all original
+samples and label unresolved comparisons inconclusive rather than attributing
+an outlier to the candidate. GPU-wide counters and the last submitting PID are
+observations, not complete per-process GPU attribution. The serving decode
+samples use different nonce-bearing prompts; speculative acceptance and target
+call counts can therefore vary within an arm. Compare matching request hashes
+and work counters across arms before interpreting their TPS spread as temporal
+instability.
+
 ## External storage loading
 
 On the M4 Pro with MLX 0.32.2, two RTN4 first-request warmups from
@@ -66,6 +79,65 @@ deferred under the active-storage policy above. Evidence under `reports/qwen38-r
 `fill-append-http-final-partial-review.json`, `rtn4-storage-control-manifest.json`,
 `fill-append-storage-control-review.json` and `weights-loading-http.json`.
 
+## Reproduce the MiniCPM5 logit comparison
+
+From an Apple Silicon Mac running macOS 14 or later, install Bun 1.4.2 or
+newer and [uv](https://docs.astral.sh/uv/getting-started/installation/), then
+run these commands from a checkout of this repository:
+
+```sh
+bun install --frozen-lockfile && \
+  oracle_env="$(bash scripts/oracle/setup.sh)" && \
+  eval "$oracle_env" && \
+  bash scripts/oracle/compare-minicpm5.sh --download
+```
+
+The setup script creates an external Python 3.13.5 environment, installs the
+complete [reference lock](../../scripts/oracle/requirements.lock), checks its
+dependencies, and prints the `MLX_BUN_ORACLE_VENV` export. Its default location
+is `~/.cache/mlx-bun/oracle-mlx-0.32.2-py3.13.5`. Pass a directory to `setup.sh`
+to choose another location. Re-running it synchronizes that environment to
+the lock, removing packages outside the pinned set. It does not install model
+weights or run inference.
+
+The lock pins MLX 0.32.2, mlx-lm 0.31.3, mlx-optiq 0.2.7 and their installed
+dependencies. MLX-Metal uses the explicit macOS 14 wheel and SHA-256 from
+[the upstream release](https://pypi.org/project/mlx-metal/0.32.2/#files),
+matching native pack 0.4.0. Selecting only a package version could install a
+different Metal build on newer macOS releases.
+
+The comparison script downloads `mlx-community/MiniCPM5-1B-OptiQ-4bit` at
+revision `664aabaed233c653f82716d8dc822234d0091f78` when `--download` is supplied.
+Omit the flag to require a cached snapshot. Hugging Face cache environment
+settings are respected. Set `MLX_BUN_TEST_MINICPM5` to use an existing model
+directory instead; its config and weight hashes are recorded in the output.
+
+Each run creates a fresh fixture directory under
+`~/.cache/mlx-bun/comparisons/`, invokes the Python oracle through
+`scripts/regen/minicpm5.ts`, then runs
+`bun test tests/parity/minicpm5-parity.test.ts`. `MLX_BUN_GOLDEN_DIR` selects
+that directory for both generation and comparison, with no fallback to older
+fixtures. The manifest records the prompt, token IDs, Python/MLX/Metal/mlx-lm
+versions, device, model hashes, lock hash and per-vector hashes. The command
+prints the output directory and elapsed comparison time. Missing inputs,
+failed generation or mismatched logits produce a nonzero exit; skipped tests
+are not the acceptance result.
+
+This test compares the complete vocabulary logit vector before sampling at
+100 steps, plus all 100 greedy token IDs. It uses bf16 KV, one short prompt
+and single-row model forwards. It does not establish sampling-policy parity,
+batched scheduling correctness, mixed-KV parity or server performance. The
+[benchmark reference](benchmarks.md) links the separate
+coverage for those configurations.
+
+Fixtures must be generated on the comparison machine. The usual golden
+resolver selects `goldens/<machine-key>/` overrides; the reference machine
+uses the root `goldens/` directory. Metal kernels can differ across GPU
+families and runtime builds. In particular, the fast-SDPA dispatch boundary
+at sequence length L ≥ 16 can change floating-point operation order. This
+short-prompt recipe does not test that prefill boundary. Longer-prefill and
+batched comparisons need their corresponding same-machine oracle cases.
+
 ## The Python oracle
 
 Logit parity with mlx-lm is the correctness oracle; mixed-precision KV and the
@@ -85,6 +157,30 @@ bumped from 0.2.4 to 0.2.15 on 2026-07-06 and the mixed-KV goldens stayed
 byte-identical across the bump. The venv directory holds no project source —
 just the venv and `serve.sh`.
 
+**Whisper oracle.** mlx-whisper 0.4.3 lives in its own venv,
+`~/Code/mlx-whisper-oracle/.venv` (Python 3.12 via uv; `mlx==0.32.2` to match
+`MLX_CORE_VERSION`, plus torch and numba, which must never enter the pinned
+mlx-lm venv). Override with `MLX_BUN_WHISPER_ORACLE_VENV`. Regenerate with
+`bun scripts/regen.ts whisper`; the parity test compares against
+`goldens/whisper.json` and its untracked blobs. The mlx-community Whisper
+checkpoint ships no tokenizer: `openai/whisper-large-v3-turbo`'s
+`tokenizer.json` + `tokenizer_config.json` must be in the HF cache
+(`hf_hub_download` of those two files; ~2.5 MB). whisper.cpp comparisons use
+sotto's vendored v1.9.3 helper (`~/Code/sotto/build/server/helpers/sotto-engine`)
+with `ggml-large-v3-turbo.bin` — same weights, different runtime.
+
+Silero VAD parity uses the `silero-vad` 6.2.1 pip package (torch JIT model)
+in the same venv (`scripts/oracle/gen-silero-golden.py` → tracked
+`goldens/silero-vad.json`); the engine's weights are `ggml-org/whisper-vad`'s
+`ggml-silero-v6.2.0.bin` in the HF cache (byte-identical to sotto's copy,
+SHA-256 `2aa269b7…`). `coremltools` is installed there too for the Core ML
+encoder experiment (benchmarks.md).
+
+The repo's untracked `.env` may pin `MLX_BUN_ORACLE_VENV` to a directory that
+no longer exists (it pointed at `~/Code/mlx-lm-example/.venv` on the M1 Max
+on 2026-09-15); every parity test importing `tests/support/paths.ts` then
+throws at import. Fix the path or run with `bun --env-file=/dev/null test …`.
+
 The Qwen MTP compatibility test accepts explicit local artifact paths through
 `MLX_BUN_TEST_MTP_TARGET` and `MLX_BUN_TEST_MTP_DRAFT`, alongside
 `MLX_BUN_TEST_QWEN38_MTP=1`. It compares speculative and ordinary greedy
@@ -101,7 +197,7 @@ Qwen artifacts are also present under
 `/Volumes/MLX-Models/models/{Qwen,mjriii}/`; header-only inventories under
 `reports/qwen38-rd/` are machine-local and do not load the GPU.
 
-**Authorized M4 Pro runtime update, staged 2026-09-07:** MLX 0.32.2 with
+**M4 Pro runtime update, staged 2026-09-07:** MLX 0.32.2 with
 MLX-C `c74db5307cc8ce122f48d97ef951b30578674e7f` is built under
 `reports/qwen38-rd/mlx-upgrade-0.32.2/install/lib/`. Explicit
 `MLX_BUN_LIBMLXC=<that-directory>/libmlxc.dylib` selects it per process.

package/docs/reference/library-api.md

@@ -14,10 +14,13 @@ everything else under `src/` is internal and may change without notice.
 Every export is documented below, grouped the way `src/index.ts` groups
 them.
 
-One rule everywhere: **one generation at a time per process** (one GPU).
-The server serializes through a queue (or, under `--batch N`, its
-continuous scheduler — a server scheduling mechanism, not a library
-surface); direct library callers must do the same.
+The server uses continuous batching by default for eligible requests.
+Its scheduler coordinates work on shared model resources, including a single
+active request. Direct callers of `generate` must coordinate access to the
+same model and mutable state; launching independent generators does not
+automatically create a batch. Use the serving integration for its scheduling
+and request preparation, or compose the engine with explicit execution ports.
+See [server configuration](server-config.md) for supported combinations.
 
 ## Export map
 
@@ -672,3 +675,14 @@ When embedding the high-level library in your own compiled application, pass
 `command: ["/path/to/mlx-bun"]` to `openIsolatedHost`. Source/package consumers
 use the installed CLI source automatically; compiled callers supply the worker
 executable explicitly because their own executable is not the mlx-bun CLI.
+
+### Cache session affinity
+
+`GenerateOptions.cacheSessionId` carries application cache affinity through
+serving execution. The `PrefixCache` contract accepts the optional session ID
+on `take`, `prefetch` and `put`, and exposes optional `closeSession(id)`. The
+cache owns the association and both storage tiers; the ID does not change the
+numerical namespace or replace the request's full token history. Standalone
+`generate()` calls that supply their own cache retain caller-owned state; an
+ID alone does not create a process-wide cache. HTTP fields and lifecycle are
+documented in [server-api.md](server-api.md#cache-session-affinity).

package/docs/reference/memory.md

@@ -254,7 +254,7 @@ routes never touch the agent-tool surface above.
 
 ## Related
 
-- [cli.md](cli.md#memory--the-personal-wiki) — the `memory` command in the full CLI reference.
+- [cli.md](cli.md#memory-alias-setup--the-personal-wiki) — the `memory` command in the full CLI reference.
 - [server-api.md](server-api.md#get-apimemory--post-apimemoryinit) — `/api/memory/*` wire format.
 - [dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — the canonical pipeline design.
 - [docs/design/dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md), [docs/design/dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — synthesis internals and the local inference path.

package/docs/reference/models.md

@@ -4,6 +4,14 @@ The one supported-model list, plus how mlx-bun downloads, indexes, lists,
 and reclaims models. The store is the standard Hugging Face hub cache —
 nothing proprietary; `hf` and mlx-lm read/write the same tree.
 
+## Automatic model selection
+
+On a fresh install with no supported downloaded model, `serve` downloads and
+starts MiniCPM5-1B, then can download the device-recommended model in the
+background. Gemma e4b is the preferred general model once available and within
+the selection budget. `DEFAULT_REPO_ID` names that preference, not the
+first-download artifact. Explicit model selection bypasses this starter flow.
+
 ## Supported models
 
 The source of truth is code, not this table: `src/model/profile.ts`
@@ -16,25 +24,29 @@ is the bug.
 
 Scope is deliberate: a few families held to explicit oracle contracts
 (L1 = bit-exact vs mlx-lm, L2 = bit-exact vs mlx-optiq, L3 = measured, no
-oracle), plus a declared Tier-0 generic surface. There is no alias table —
+oracle), plus a declared Tier-0 generic surface. These are comparison
+contracts, not interchangeable claims about every artifact. The
+[coverage map](benchmarks.md#coverage-beyond-the-four-model-summary) identifies
+the test and remaining evidence limits for each roster row. There is no alias table —
 the "query" column is a substring that `resolve()` matches against
 already-downloaded repos (it must match exactly one repo; see
 [Query resolution rules](#query-resolution-rules-per-verb)).
 
 | `serve` query | Validated artifact (HF repo) | Family · load path · tier | Modalities | Draft sources | KV schemes | Notes |
 | --- | --- | --- | --- | --- | --- | --- |
-| `e4b` | `mlx-community/gemma-4-e4b-it-OptiQ-4bit` | Gemma 4 (`gemma4*`) · dedicated `gemma4` graph with a **generated** specialization keyed on the config fingerprint · L1 | text · **image** (SigLIP tower in the bf16 `optiq_vision.safetensors` sidecar) · **audio** (Conformer tower in the same sidecar) | two-model · Gemma `-assistant` (L2) · locally-trained DSpark (`dspark.json`) · ngram | bf16 · uniform 4/8 · `config` (kv_config.json, batches) · turbo · `--paged-kv` | **Default model** (`DEFAULT_REPO_ID`, src/fit.ts). The only audio-capable artifact. ~7.0 GB on disk (benchmarks.md legend, 2026-06-15). Trains (LoRA / ORPO / SFT). |
+| `e4b` | `mlx-community/gemma-4-e4b-it-OptiQ-4bit` | Gemma 4 (`gemma4*`) · dedicated `gemma4` graph with a **generated** specialization keyed on the config fingerprint · L1 | text · **image** (SigLIP tower in the bf16 `optiq_vision.safetensors` sidecar) · **audio** (Conformer tower in the same sidecar) | two-model · Gemma `-assistant` (L2) · locally-trained DSpark (`dspark.json`) · ngram | bf16 · uniform 4/8 · `config` (kv_config.json, batches) · turbo · `--paged-kv` | **Preferred model after download**; fresh installs first serve MiniCPM5. See [automatic selection](#automatic-model-selection). The only audio-capable artifact. ~7.0 GB on disk (benchmarks.md legend, 2026-06-15). Trains (LoRA / ORPO / SFT). |
 | `12B` | `mlx-community/gemma-4-12B-it-OptiQ-4bit` | Gemma 4 · generated specialization · L1 | text · **image** (encoder-free `gemma4_unified_vision` declared in config.json — no sidecar needed). No audio: the snapshot's sidecar is an audio **stub** (`embed_audio` only, no `audio_tower.*`) | two-model · `-assistant` · DSpark local · **DeepSpec released** (`deepseek-ai/dspark_gemma4_12b_block7`) · ngram | bf16 · uniform · `config` · turbo · `--paged-kv` | ~8.4 GB (2026-06-15). `largestRecommendedRepoId` pick for ≥ 24 GB RAM (src/fit.ts). Trains. |
 | `26B` | `mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit` | Gemma 4 MoE · generated specialization · L1 | text · image **only if** the snapshot ships the SigLIP sidecar (registry `has_vision_sidecar`; the tower is `gemma4_vision`). No `audio_config` → no audio | two-model · `-assistant` · ngram | bf16 · uniform · `config` · turbo · `--paged-kv` | ~18 GB (2026-06-15). `largestRecommendedRepoId` pick for ≥ 48 GB RAM. |
-| any other `gemma4*` config (1B, e2b, 31B, …) | — | Gemma 4 · the `Gemma4Model` monolith (no generated specialization) · L1 by family | per the registry's vision/audio flags | same as the family | same as the family | Loads by `model_type`; **no artifact is pinned in tests/support/goldens for these**, so treat as family-supported, artifact-unvalidated. |
+| any other `gemma4*` config (1B, e2b, 31B, …) | — | Gemma 4 · the `Gemma4Model` monolith (no generated specialization) · family support, artifact unvalidated | per the registry's vision/audio flags | same as the family | same as the family | Loads by `model_type`; **no artifact is pinned in tests/support/goldens for these**, so treat as family-supported, artifact-unvalidated. |
 | `MiniCPM5` | `mlx-community/MiniCPM5-1B-OptiQ-4bit` | MiniCPM5 · `model_type: llama` matched by exact shape (`isMiniCPM5Config`) → dedicated `minicpm5` graph · L1 / L2 | text | two-model · ngram | bf16 · uniform · `config` (batches — Phase 3.1) · turbo (quality curve measured 2026-07-06, benchmarks.md §3) | Sub-GB starter (benchmarks.md legend). Trains (segmented backward). Tool calling (XML). |
 | `Qwen3.5-4B` | `mlx-community/Qwen3.5-4B-OptiQ-4bit` | Qwen3.5 (`qwen3_5` / `qwen3_5_text`, dense hybrid gated-DeltaNet) · dedicated `qwen3.5` graph · L1 | text (image/video only when the artifact ships vision weights — see Qwen3.8) | two-model · ngram · `mtp` if a `qwen3_5_mtp` companion exists | bf16 (L1) · `config` (KV-ON parity passed 2026-06-15, PLAN 14e) | Thinking + tool calling. Batches on the SSM path (`MLX_BUN_BATCH_SSM=0` reverts). The MoE variant `qwen3_5_moe` is **not** supported. |
 | `Qwen3.8` | `mlx-community/Qwen3.8-27B-OptiQ-4bit` @ `b04599de95d7a9bfbd7f208d347c0f10d9432a42` | Qwen3.5 family · **exact artifact profile** `qwen3.8-27b-optiq-4bit` (safetensors + `qwen3.5` graph) · L1 | text · **image** · **video** (`image_url` / `video_url` parts; Qwen3-VL tower + mRoPE; video decodes via the AVFoundation sidecar `mlx-bun-frame-extract`, 2 fps, ≤ 768 frames). Video never combines with audio; no audio | **native MTP head** (`mlx-community/Qwen3.8-27B-MTP-bf16`, auto-detected `qwen3_5_mtp`; `--draft-kind mtp` with no `--draft-model` uses a bundled `mtp/` subfolder) · two-model · ngram | bf16 (L1) · `config` is **Lab** — policy copied from the same-topology Qwen3.6 artifact, no model-specific oracle (benchmarks.md, 2026-08-22) | `reasoning_effort` (xhigh/medium/low) + `preserve_thinking` + tool calling. `/v1/models` advertises vision. Shipped in v0.2.0. |
 | `Qwen3-Embedding` | `mlx-community/Qwen3-Embedding-4B-4bit-DWQ` | plain `qwen3` · dedicated `qwen3` graph · L1 | text | — | — | **Embeddings only**: `/v1/embeddings` + `mlx-bun embed`. Auto-picked by `embed` when no query is given; never the chat default. |
 | `Qwen3-30B` | `mlx-community/Qwen3-30B-A3B-Thinking-2507-4bit` | Qwen3-MoE (`qwen3_moe`) · dedicated `qwen3-moe` graph · L1 (opt-in slow parity test, `MLX_BUN_TEST_QWEN3_MOE=1`) | text | two-model · ngram | bf16 (the artifact ships no kv_config.json) | Sparse top-k SwitchGLU experts. |
 | `diffusiongemma` | `mlx-community/diffusiongemma-26B-A4B-it-OptiQ-4bit` | DiffusionGemma (`diffusion_gemma`) · dedicated graph + **denoising loop** (non-autoregressive) · L2 (optiq) | text · **image** (single image; its own inline-quantized SigLIP tower). No video / audio | — (no AR verify loop) | — (canvas model; the `--kv-quant` axis does not apply) | Serial lane always. Diffusion-native LoRA trains. |
+| `whisper` | `mlx-community/whisper-large-v3-turbo` (fp16, 1.6 GB, `weights.safetensors`) | Whisper (`whisper`) · dedicated encoder-decoder graph + transcription loop · **L1 vs mlx-whisper 0.4.3** (mel, encoder output and per-step decoder logits bit-exact; `tests/parity/whisper.test.ts`) | **audio in → text out** only: `/v1/audio/transcriptions`, `/v1/audio/translations`, `mlx-bun transcribe`. Never the chat model. | — | — | Tokenizer files are not in the mlx-community repo; the loader resolves `openai/whisper-large-v3-turbo`'s `tokenizer.json` from the HF cache (2.4 MB, `mlx-bun get` it once). Beam search (`beam_size`) is ours — mlx-whisper has none; checked against whisper.cpp's beam-5 transcripts. Word timestamps not yet. Serves as a companion beside any chat model or alone (transcription-only server). Any `mlx-community/whisper-*` fp16 checkpoint loads (80-mel v1/v2 vocab resolves `openai/whisper-large-v2`). |
 | `GLM-5.2-colibri` | `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` @ `3cc8db99b1b13fc79325d987ba3c1c430766b3b8` | GLM-5.2 (`glm_moe_dsa`) · **Colibri container loader** + dedicated `glm5.2` graph (streamed experts) · **exact artifact profile**, L3 | text | **checkpoint-native MTP** (`--mtp on`, the default; mutually exclusive with `--draft-model`) | checkpoint-native compressed **MLA/DSA** cache (its own scheme; SSD tier v3 persists it) | Fixed-context admission: `--context-length` (4096) and `--max-tokens` (128) are reserved by the header-only resource equation before weights open; `--memory-budget` defaults to min(25 GiB, RAM). `--mtp off` enables ordinary batching. Embeddings, vision/audio, adapters, and training are unsupported. See [below](#direct-container-artifacts-glm-5x). |
-| Tier-0 generic | validated manifest (`tests/support/universal-manifest.ts`): `Llama-3.2-1B-Instruct-4bit`, `Qwen2.5-0.5B-Instruct-4bit`, `Qwen3-0.6B-4bit`, `quantized-gemma-2b-it`, `gemma-2-2b-it-4bit`, `Phi-3.5-mini-instruct-4bit`, `GLM-4-9B-0414-4bit`, `granite-3.3-2b-instruct-4bit`, `starcoder2-3b-4bit`, `SmolLM3-3B-4bit` (all `mlx-community/`) and `ekryski/OLMo-2-0425-1B-Instruct-4bit` | 11 `model_type`s: `llama`, `smollm3`, `qwen2`, `qwen3`, `gemma`, `gemma2`, `phi3`, `olmo2`, `glm4`, `granite`, `starcoder2` (+ mlx-lm remaps `mistral`, `iquestcoder` → `llama`) · `universal-dense` graph · L1 | text | two-model · ngram | bf16; unreleased Llama full-attention uniform KV4/KV8 | `ls` labels these `supported (generic)`. Shared full-attention execution is qualified on Llama-3.2-3B at B1/B4, including speculative KV4/KV8 and generated RAM/SSD state; gemma2-family / sliding-window archs run serial. No compiled specialization. A `model_type` absent from the table refuses to load. |
+| Tier-0 generic | validated manifest (`tests/support/universal-manifest.ts`): `Llama-3.2-1B-Instruct-4bit`, `Qwen2.5-0.5B-Instruct-4bit`, `Qwen3-0.6B-4bit`, `quantized-gemma-2b-it`, `gemma-2-2b-it-4bit`, `Phi-3.5-mini-instruct-4bit`, `GLM-4-9B-0414-4bit`, `granite-3.3-2b-instruct-4bit`, `starcoder2-3b-4bit`, `SmolLM3-3B-4bit` (all `mlx-community/`) and `ekryski/OLMo-2-0425-1B-Instruct-4bit` | 11 `model_type`s: `llama`, `smollm3`, `qwen2`, `qwen3`, `gemma`, `gemma2`, `phi3`, `olmo2`, `glm4`, `granite`, `starcoder2` (+ mlx-lm remaps `mistral`, `iquestcoder` → `llama`) · `universal-dense` graph · L1 | text | two-model · ngram | bf16; Llama full-attention uniform KV4/KV8 | `ls` labels these `supported (generic)`. Shared full-attention execution is qualified on Llama-3.2-3B at B1/B4, including speculative KV4/KV8 and generated RAM/SSD state; gemma2-family / sliding-window archs run serial. No compiled specialization. A `model_type` absent from the table refuses to load. |
 
 Rules that apply across rows (all enforced in `src/server.ts`):
 
@@ -45,7 +57,7 @@ Rules that apply across rows (all enforced in `src/server.ts`):
   MTP is its own row. Qwen MTP companions may store dense or affine-quantized
   projections; the loader uses each module's quantization metadata. Target
   verification is unchanged, while draft acceptance and speed depend on the
-  companion. In the unreleased shared executor, qualified plain, uniform,
+  companion. In the shared executor, qualified plain, uniform,
   per-layer affine and TurboQuant layouts compose with drafting through the
   target/provider interfaces. Positive affine conversion thresholds are also
   supported by library requests. The loaded model and provider determine