mlx-bun 0.3.0 → 0.5.0

package/docs/reference/cli.md

@@ -29,6 +29,8 @@ each verb).
 | `harness pi` | connect your own pi install to the local server | — |
 | `generate` (`gen`) | one-shot generation, no server | `mlx_lm.generate` |
 | `embed` | text embeddings, no server | — |
+| `transcribe` | speech-to-text from an audio file, no server | — (`mlx_whisper` CLI is the nearest) |
+| `dictate` | push-to-talk microphone dictation → print / clipboard / typed keystrokes | — |
 | `get` | download a model (resumable, verified) | `mlx_lm.manage` (partial) |
 | `scan` | re-index the Hugging Face cache | `mlx_lm.manage --scan` |
 | `ls` | list downloaded models | `mlx_lm.manage --scan` |
@@ -64,6 +66,24 @@ mlx-bun serve GLM-5.2 --context-length 4096
 semantics, and which combinations compose live in
 **[server-config.md](server-config.md)** — this page only names them.
 
+Library requests support delayed uniform or per-layer affine conversion with
+qualified shared drafting providers, including rotating caches. CLI affine
+quantization starts at zero.
+
+The shared executor supports qualified Qwen MTP and start-zero
+TurboQuant at one or several active rows; `--batch 1` still selects the
+legacy serial control. No new CLI flag is required for the shared interfaces.
+Full-attention and rotating targets, including Llama, MiniCPM and Gemma, use shared bf16, uniform KV4/KV8, per-layer affine KV or TurboQuant
+prompt lookup and standalone drafting, including generated RAM/SSD state.
+Gemma assistant drafting also shares these target layouts and caches its last true target
+hidden alongside target KV through the same RAM/SSD interfaces.
+DeepSpec and DSpark use the same executor and persist their projected context through those
+interfaces. DeepSpec accepts bf16 or affine-quantized weights; DSpark retains
+its Markov/RNN head and checkpoint confidence policy.
+
+Shared Qwen MTP retains completed decode state through the existing
+`--prompt-cache` and `--ssd-cache` settings.
+
 | Group | Flags |
 |---|---|
 | Network | `--host`, `--port`, `--no-open`, `--allow-private-media` |
@@ -71,13 +91,17 @@ semantics, and which combinations compose live in
 | Process model | `--isolate`, `--model-pool`, `--unix` (internal — the engine half of `--isolate`) |
 | Scheduling | `--batch` (`--decode-concurrency` accepted as the mlx_lm.server alias) |
 | KV cache | `--kv-quant`, `--paged-kv`, `--paged-kv-block-size` |
-| Adapters | `--adapter` (`--adapter-path` alias) |
-| Speculative decoding | `--draft-model`, `--draft-kind`, `--num-draft-tokens`, `--ngram-max`, `--ngram-min` |
+| Adapters (compatible sets can batch) | `--adapter` (`--adapter-path` alias) |
+| Speculative decoding (Qwen methods and full-attention/rotating lookup/standalone and Gemma assistant/DeepSpec/DSpark drafting share execution) | `--draft-model`, `--draft-kind`, `--num-draft-tokens`, `--ngram-max`, `--ngram-min` |
 | GLM-5.2 | `--mtp`, `--context-length` |
 | Sampling defaults | `--temperature` (`--temp` alias), `--top-p`, `--top-k`, `--max-tokens`, `--thinking`, `--hlg-sampling`, `--hlg-width`, `--hlg-shoulder`, `--hlg-toe`, `--hlg-pivot-offset` |
 | Parity tier | `--l1`, `--l2` (`--l3` errors — removed) |
 | Kill switches | `--compiled-decode`, `--compiled-activations`, `--fused-sdpa`, `--force-wire`, `--expert-offload` |
 
+Serving treats memory estimates as advisory by default and attempts the request.
+`--memory-budget` opts into estimated admission limits; `--kv-budget` opts
+into aggregate KV limits. `--max-tokens` supplies an explicit output cap.
+
 Endpoints: `/v1/chat/completions`, `/v1/completions`, `/v1/messages`,
 `/v1/responses`, `/v1/embeddings`, `/v1/models`, `/v1/adapters`, `/health`,
 `/stats`, `/fit`, `/library`, `/downloads`; the web app at `/` (`/chat`,
@@ -191,6 +215,65 @@ mlx-bun embed --instruct "Retrieve passages that answer the question" --text "wh
 | `[query]` / `--query <q>` | embedding model; omitted = the first downloaded embedding model (errors with a `get` suggestion when none) |
 | `--text "…"` | text to embed (omit to read one text per stdin line) |
 | `--instruct "…"` | Qwen3-Embedding query instruction; omit for plain document embeddings |
+
+### `transcribe` — speech-to-text
+
+Whisper transcription of one audio file with a local `mlx-community/whisper-*`
+checkpoint, no server. WAV decodes natively; other containers CoreAudio can
+read (mp3/m4a/aac/flac/ogg/aiff, mp4/mov audio…) decode in-process through
+AudioToolbox. Model support
+and the oracle contract: [models.md](models.md#supported-models).
+
+```sh
+mlx-bun transcribe meeting.m4a                       # first downloaded Whisper, language auto-detected
+mlx-bun transcribe talk.wav --language en --format srt > talk.srt
+mlx-bun transcribe note.wav --beam-size 5 --prompt "Sotto, SwiftUI, Metal" --verbose
+```
+
+| Flag | Meaning |
+|---|---|
+| `<audio-file>` | required positional |
+| `[query]` / `--model <path\|query>` | Whisper model; omitted = the first downloaded `whisper` checkpoint (errors with a `get` suggestion when none) |
+| `--language <code\|name>` | `en`, `japanese`, …; `auto` or omitted detects from the first 30 s |
+| `--task translate` | translate to English |
+| `--beam-size <n>` | beam search (default greedy) |
+| `--temperature <t>` | one temperature; default is mlx-whisper's `(0, 0.2, …, 1.0)` fallback ladder gated by compression-ratio / log-prob thresholds |
+| `--no-fallback` | temperature 0 only |
+| `--prompt "…"` | initial prompt (vocabulary hints, up to 223 tokens are kept) |
+| `--no-timestamps` | text-only decoding |
+| `--no-condition` | do not feed the previous window's text as the next prompt |
+| `--vad`, `--vad-threshold <t>`, `--vad-model <path>` | Silero VAD gate: no detected speech → empty output, Whisper never runs. Weights: `ggml-org/whisper-vad` in the HF cache (885 KB) or an explicit ggml path |
+| `--word-timestamps` | word-level timing in `verbose_json` (`words` per segment) |
+| `--faithful` | oracle-parity graph instead of the fast path (fused attention, compiled decoder step) |
+| `--audio-ctx <n>` | Lab: encode only n of the 1500 encoder positions (whisper.cpp `-ac`); degrades below ~1024 |
+| `--format <f>` | `text` (default) · `json` · `verbose_json` · `srt` · `vtt` |
+| `--verbose` | per-segment time ranges and a realtime-factor summary on stderr |
+
+### `dictate` — push-to-talk from the microphone
+
+Streams the mic (AVAudioEngine sidecar `mlx-bun-mic-capture`, built from
+`src/native/mic_capture.swift` on first use with swiftc, or shipped beside
+the binary) into a transcription session while you speak, so every
+finished 30 s window is transcribed during capture and the result lands
+about one window after you stop. Silence never runs Whisper (Silero VAD).
+macOS prompts for Microphone permission; `--hotkey` needs Input Monitoring
+and `--type` needs Accessibility for your terminal app.
+
+```sh
+mlx-bun dictate                              # Enter starts/stops a take; prints the text
+mlx-bun dictate --hotkey --copy              # hold Right Option, transcript → clipboard
+mlx-bun dictate --hotkey --type              # …typed into the frontmost app
+mlx-bun dictate --server http://localhost:8090 --vocabulary "Sotto,SwiftUI,Metal"
+```
+
+| Flag | Meaning |
+|---|---|
+| `[query]` / `--model <path\|query>` | Whisper model for in-process transcription (default: first downloaded `whisper` checkpoint) |
+| `--server <url>` | use a running server's `/v1/audio/sessions` instead |
+| `--hotkey [keycode]` | hold-to-talk on a macOS virtual keycode (default 61, Right Option) instead of Enter toggling |
+| `--language`, `--beam-size`, `--prompt`, `--vocabulary "a,b"`, `--no-vad` | decoding, as for `transcribe` |
+| `--idle-unload <s>` / `--resident` | release the weights after idle seconds (0 = after every take; default 30) or keep them loaded |
+| `--copy` / `--type` / `--type-delay <s>` | clipboard / System Events keystrokes into the frontmost app (delay defaults to 1 s in Enter mode so you can Cmd-Tab, 0 with `--hotkey`) |
 | `--json` | one OpenAI-style `{object:"list",data:[…]}` object instead of bare arrays |
 
 Server equivalent: `mlx-bun serve <embedding-model>` then
@@ -469,6 +552,11 @@ and optiq reference servers and writes a dated report:
 ```sh
 bun scripts/bench-serve.ts all
 bun scripts/bench-serve.ts all --models cpm5,e4b --arms mlx-bun,mlx-lm --out report.md
+bun scripts/bench-serve.ts all --models qwen27b --arms mlx-bun-serial,mlx-bun --logprobs --top-logprobs 3
+# Same KV settings on both executors
+bun scripts/bench-serve.ts all --models qwen27b --arms mlx-bun-serial,mlx-bun --kv-quant 4
+# Same adapter settings on both execution arms:
+bun scripts/bench-serve.ts all --models e4b --arms mlx-bun-serial,mlx-bun --adapter fixtures/adapters/upper
 ```
 
 Curated results: [benchmarks.md](benchmarks.md).
@@ -544,3 +632,15 @@ behind each "not covered" row is in
 Native verbs with no mlx-lm counterpart — `pi`, `harness pi`, `embed`,
 `fit`, `train-watch`, `memory`, `evals` — are the surface that makes mlx-bun
 more than a drop-in.
+
+
+`--kv-quant turbo` composes with grouped lookup, standalone, Gemma assistant,
+DeepSpec and DSpark drafting on supported targets. Sliding layers stay bf16.
+See [batching design](../design/batching.md) for composition evidence and limits.
+
+Paged KV retains the configured `--batch` size. Gemma4 bf16 requests use the shared executor and bypass the prompt cache; quantized KV and draft combinations remain unavailable. See [server configuration](server-config.md).
+
+`--generation-checkpoint N` requires `--ssd-cache` and queues owned snapshots
+for eligible ordinary requests. Shared continuation supports eligible Qwen, Llama, MiniCPM and Gemma4 requests with or without adapters, excluding media, grammar, fill, paging, logprobs or speculation.
+Repeat the identical request to replay and resume; flush before shutdown for
+eventual queued snapshots to become durable.

package/docs/reference/distribution.md

@@ -15,9 +15,33 @@ Three install channels ship from one release:
 | Direct download (`curl -fsSL https://mlx-bun.dev/install.sh \| sh`) | the same bundle, into `~/.mlx-bun` | one tarball, via curl |
 | npm / bunx (`bunx mlx-bun`, `bun install -g mlx-bun`) | a launcher + the TypeScript source, run under the user's Bun | the package, then the native runtime pack on first run |
 
-[docs/reference/distribution.md](./distribution.md) covers the *sidecar* use of the same bundle
-inside a Tauri/Electron app; the build and signing facts it relies on are
-documented here.
+The same bundle can run as a local worker inside a desktop application.
+See [Embedding in a desktop application](#embedding-in-a-desktop-application).
+
+## Embedding in a desktop application
+
+Include the complete release bundle in the application's resources and launch
+its executable by absolute path. Keep the native libraries, Metal library,
+helpers, and Pi assets together as listed below. Copying only the executable
+does not produce a self-contained installation.
+
+For a Bun host, the [library API](library-api.md) documents `openIsolatedHost`
+and its explicit executable command. A desktop host in another runtime can
+manage the worker process and use the [HTTP API](server-api.md). Keep transport
+and worker lifecycle in the application; request scheduling belongs to the
+engine. Stop the worker when the application no longer owns it.
+
+The model weights are separate from the executable bundle. Select an existing
+artifact or use the [model download commands](models.md) before starting the
+worker with that model. Follow the documented cache-location settings when
+the application manages its own storage.
+
+Repackaging within an application requires that application's signing and
+notarization workflow. The executable's hardened-runtime entitlements and
+native signing requirements are documented below. Include the third-party
+attributions with the distributed application. Test the installed application
+bundle, including worker launch and runtime assets, rather than relying only
+on a source-checkout run.
 
 ## What ships in the bundle
 
@@ -32,6 +56,7 @@ dir), ~80 MB compressed; `mlx.metallib` is the bulk of the installed size.
 | `libmlx.dylib`, `libjaccl.dylib`, `mlx.metallib` | MLX core, its distributed-comm dependency, the Metal kernels (libmlx loads the metallib from its own directory) | `/opt/homebrew/opt/mlx/lib` (override `MLX_LIB_DIR`) |
 | `libmlx_bun_expert_io.dylib` | bounded streamed-expert I/O for GLM-5.2 MoE serving | [scripts/build-expert-io.sh](../../scripts/build-expert-io.sh) |
 | `mlx-bun-frame-extract` | AVFoundation frame-extraction helper for video input (a Mach-O *executable*, not a dylib) | [scripts/build-frame-extract.sh](../../scripts/build-frame-extract.sh) → `swiftc -O src/native/frame_extract.swift` |
+| `mlx-bun-mic-capture` | AVAudioEngine microphone capture + CGEvent hold-key helper for `mlx-bun dictate` (Mach-O executable; the only native piece of the Whisper work — the web composer uses the browser mic) | [scripts/build-mic-capture.sh](../../scripts/build-mic-capture.sh) → `swiftc -O src/native/mic_capture.swift` |
 | `photon_rs_bg.wasm` | pi's image codec, resolved next to the executable by the web chat's `read`-on-image path (best-effort; pi degrades to a text note without it) | `node_modules/@silvia-odwyer/photon-node` (override `PHOTON_WASM`) |
 | `theme/*.json`, `assets/*.png`, `export-html/`, `package.json`, `CHANGELOG.md`, `native/darwin/prebuilds/<arch>/darwin-modifiers.node` | assets the embedded pi **terminal** (`mlx-bun pi`) resolves by path: themes, startup art, `/export` template, version banner + changelog, native modifier-key helper (degrades if absent) | `node_modules/@earendil-works/pi-coding-agent/dist`, `pi-tui` |
 
@@ -133,7 +158,7 @@ does, in order. Steps 1–3 are yours; the rest is the scripts.
    (`BUILD_DIR` overrides `dist`). Release builds are arm64-only; the script
    exits on any other host arch.
 5. **Sign nested code first** — every `*.dylib`, `*.node`, and the
-   `mlx-bun-frame-extract` helper, matched by name under the build dir, gets
+   `mlx-bun-frame-extract` / `mlx-bun-mic-capture` helpers, matched by name under the build dir, get
    `codesign --force --timestamp --options runtime -s "<identity>"`.
    Nested-before-container is required. The metallib, wasm, JSON, and PNG
    assets are data, not code, and are skipped.
@@ -186,6 +211,13 @@ npm if the version is live. Env overrides: `MLX_BUN_SIGN_IDENTITY`,
 `NOTARY_PROFILE`, `BUILD_DIR`, `OUT_DIR` (release); `OUT_DIR`, `REPO`,
 `TAP_REPO` (publish).
 
+For an SSH-driven release, a local keychain unlock may still leave
+`notarytool` reporting `keychainLocked` in the SSH session. The v0.3.0
+release succeeded by running `release-binary.sh` in the M1 Max's local
+Terminal, then `publish-release.sh` over SSH. Both steps used the same clean
+merged source and `OUT_DIR`. Check `npm whoami` from the actual release
+checkout too: a project `.npmrc` can override the user's configuration.
+
 ### The notarization lesson (v0.2.0, 2026-08-24)
 
 The first v0.2.0 submission came back **Invalid**. The release added
@@ -311,7 +343,7 @@ sh scripts/build-native-pack.sh <ver> dist-native
 gh release create native-v<ver> \
   dist-native/mlx-bun-native-v<ver>-arm64.tar.gz \
   dist-native/mlx-bun-native-v<ver>-arm64.tar.gz.sha256 \
-  --title "mlx-bun native runtime v<ver>" --notes "<what changed>"
+  --latest=false --title "mlx-bun native runtime v<ver>" --notes "<what changed>"
 ```
 
 `build-native-pack.sh` stages the same six files with the same
@@ -323,13 +355,13 @@ package. Pre-publish smoke: a fresh-cache extraction plus a real MLX
 `dlopen` (and, when the helpers changed, an expert read or a video decode).
 The pack is ad-hoc signed only — `build-native-pack.sh` never touches a
 Developer ID and nothing notarizes it; it is `dlopen`ed by the user's own Bun
-process, not by our signed executable. The most recent pack
-(`native-v0.3.0`, published 2026-08-22) added `mlx-bun-frame-extract`.
+process, not by our signed executable. Use `--latest=false` so a native pack
+does not replace the application release behind the direct installer's URL.
 
-The working tree targets native pack 0.4.0 with MLX 0.32.2 and MLX-C
-`c74db5307cc8ce122f48d97ef951b30578674e7f`. Its six-file macOS 14 candidate
-has been built and verified locally. It is not published. Release the native
-asset before publishing any package that uses this manifest. Runtime bindings
+Native pack 0.4.0 was published on 2026-09-08 with MLX 0.32.2 and MLX-C
+`c74db5307cc8ce122f48d97ef951b30578674e7f`. Its six-file macOS 14 archive
+was uploaded from the M1 Max and downloaded again; its checksum matches the
+package manifest. It was published before mlx-bun v0.3.0. Runtime bindings
 match this one C API, and startup checks the linked core version. The old
 runtime and source controls remain separate for benchmark comparisons.
 
@@ -338,8 +370,8 @@ CI builds that pinned C wrapper with
 `MLX_BUN_LIBMLXC` to its `lib/libmlxc.dylib`. The script verifies the official
 macOS 14 MLX-Metal archive hash and checks out the exact MLX-C commit. It
 extracts native libraries and headers without running Python or compiling
-Metal sources. This also reproduces local native-pack build inputs before
-the candidate is published; Homebrew's C wrapper is not the CI runtime.
+Metal sources. This also reproduces the native-pack build inputs;
+Homebrew's C wrapper is not the CI runtime.
 
 Managed jobs in the compiled binary re-exec that binary through a private job
 entry, with the database/log paths passed in the environment. They do not depend

package/docs/reference/environment.md

@@ -1,9 +1,8 @@
 # Reference environment and platform facts
 
-The canonical home for the oracle setup and the hard-won platform facts that
-agents and contributors need. CLAUDE.md points here; do not duplicate these
-facts elsewhere. Each fact is one paragraph with the evidence path that proves
-it — when you change the code, update the paragraph.
+Oracle configuration, native-runtime compatibility, and machine conditions
+for reproducing numerical and performance checks. Historical environment
+records below identify the versions used for their original comparisons.
 
 User-facing symptoms live in [troubleshooting.md](./troubleshooting.md).
 
@@ -21,8 +20,8 @@ Before asserting anything about RAM, OOM, or bandwidth, run
 reference `serve.sh` (below) hard-codes a 28 GB wired limit "of 32 GB", so it
 is written for the M1 Max.
 
-For the current M4 Pro optimization campaign, Josh specifies internal SSD
-storage for active models. The external SSD stores artifacts to copy when
+The M4 Pro optimization measurements use internal SSD storage for active
+models. The external SSD stores artifacts to copy when
 needed; external-drive loading performance is outside normal acceptance for
 the 12–17 GB models. Stage each needed artifact under `~/models/`, verify
 every file against its archive, and record that path in both benchmark arms.
@@ -34,10 +33,24 @@ user-local eval DB (`~/.cache/mlx-bun/evals.sqlite`) and promoted to
 `docs/reference/benchmarks.md` deliberately. `bun scripts/bench-serve.ts all`
 is the benchmark harness; it writes Markdown and raw JSON under `reports/`
 as gitignored outputs. Numbers on a loaded machine are
-garbage — run-to-run spread is the stability signal, and the harness retries
+diagnostic; run-to-run spread is the stability signal, and the harness retries
 unstable cells (`scripts/bench-serve.ts`; `benchmarks.md`, "Running the
 benchmark").
 
+A performance decision must account for concurrent work during the timed
+comparison. For unstable pairs, capture process activity, GPU utilization and
+VM/page-in changes before and during the repeat; keep timestamps that align
+those observations with each measured request or forward. A preflight snapshot
+alone cannot identify interference during a run. CPU usage or retained swap
+alone does not establish GPU contention or active paging. Preserve all original
+samples and label unresolved comparisons inconclusive rather than attributing
+an outlier to the candidate. GPU-wide counters and the last submitting PID are
+observations, not complete per-process GPU attribution. The serving decode
+samples use different nonce-bearing prompts; speculative acceptance and target
+call counts can therefore vary within an arm. Compare matching request hashes
+and work counters across arms before interpreting their TPS spread as temporal
+instability.
+
 ## External storage loading
 
 On the M4 Pro with MLX 0.32.2, two RTN4 first-request warmups from
@@ -66,6 +79,65 @@ deferred under the active-storage policy above. Evidence under `reports/qwen38-r
 `fill-append-http-final-partial-review.json`, `rtn4-storage-control-manifest.json`,
 `fill-append-storage-control-review.json` and `weights-loading-http.json`.
 
+## Reproduce the MiniCPM5 logit comparison
+
+From an Apple Silicon Mac running macOS 14 or later, install Bun 1.4.2 or
+newer and [uv](https://docs.astral.sh/uv/getting-started/installation/), then
+run these commands from a checkout of this repository:
+
+```sh
+bun install --frozen-lockfile && \
+  oracle_env="$(bash scripts/oracle/setup.sh)" && \
+  eval "$oracle_env" && \
+  bash scripts/oracle/compare-minicpm5.sh --download
+```
+
+The setup script creates an external Python 3.13.5 environment, installs the
+complete [reference lock](../../scripts/oracle/requirements.lock), checks its
+dependencies, and prints the `MLX_BUN_ORACLE_VENV` export. Its default location
+is `~/.cache/mlx-bun/oracle-mlx-0.32.2-py3.13.5`. Pass a directory to `setup.sh`
+to choose another location. Re-running it synchronizes that environment to
+the lock, removing packages outside the pinned set. It does not install model
+weights or run inference.
+
+The lock pins MLX 0.32.2, mlx-lm 0.31.3, mlx-optiq 0.2.7 and their installed
+dependencies. MLX-Metal uses the explicit macOS 14 wheel and SHA-256 from
+[the upstream release](https://pypi.org/project/mlx-metal/0.32.2/#files),
+matching native pack 0.4.0. Selecting only a package version could install a
+different Metal build on newer macOS releases.
+
+The comparison script downloads `mlx-community/MiniCPM5-1B-OptiQ-4bit` at
+revision `664aabaed233c653f82716d8dc822234d0091f78` when `--download` is supplied.
+Omit the flag to require a cached snapshot. Hugging Face cache environment
+settings are respected. Set `MLX_BUN_TEST_MINICPM5` to use an existing model
+directory instead; its config and weight hashes are recorded in the output.
+
+Each run creates a fresh fixture directory under
+`~/.cache/mlx-bun/comparisons/`, invokes the Python oracle through
+`scripts/regen/minicpm5.ts`, then runs
+`bun test tests/parity/minicpm5-parity.test.ts`. `MLX_BUN_GOLDEN_DIR` selects
+that directory for both generation and comparison, with no fallback to older
+fixtures. The manifest records the prompt, token IDs, Python/MLX/Metal/mlx-lm
+versions, device, model hashes, lock hash and per-vector hashes. The command
+prints the output directory and elapsed comparison time. Missing inputs,
+failed generation or mismatched logits produce a nonzero exit; skipped tests
+are not the acceptance result.
+
+This test compares the complete vocabulary logit vector before sampling at
+100 steps, plus all 100 greedy token IDs. It uses bf16 KV, one short prompt
+and single-row model forwards. It does not establish sampling-policy parity,
+batched scheduling correctness, mixed-KV parity or server performance. The
+[benchmark reference](benchmarks.md) links the separate
+coverage for those configurations.
+
+Fixtures must be generated on the comparison machine. The usual golden
+resolver selects `goldens/<machine-key>/` overrides; the reference machine
+uses the root `goldens/` directory. Metal kernels can differ across GPU
+families and runtime builds. In particular, the fast-SDPA dispatch boundary
+at sequence length L ≥ 16 can change floating-point operation order. This
+short-prompt recipe does not test that prefill boundary. Longer-prefill and
+batched comparisons need their corresponding same-machine oracle cases.
+
 ## The Python oracle
 
 Logit parity with mlx-lm is the correctness oracle; mixed-precision KV and the
@@ -85,6 +157,30 @@ bumped from 0.2.4 to 0.2.15 on 2026-07-06 and the mixed-KV goldens stayed
 byte-identical across the bump. The venv directory holds no project source —
 just the venv and `serve.sh`.
 
+**Whisper oracle.** mlx-whisper 0.4.3 lives in its own venv,
+`~/Code/mlx-whisper-oracle/.venv` (Python 3.12 via uv; `mlx==0.32.2` to match
+`MLX_CORE_VERSION`, plus torch and numba, which must never enter the pinned
+mlx-lm venv). Override with `MLX_BUN_WHISPER_ORACLE_VENV`. Regenerate with
+`bun scripts/regen.ts whisper`; the parity test compares against
+`goldens/whisper.json` and its untracked blobs. The mlx-community Whisper
+checkpoint ships no tokenizer: `openai/whisper-large-v3-turbo`'s
+`tokenizer.json` + `tokenizer_config.json` must be in the HF cache
+(`hf_hub_download` of those two files; ~2.5 MB). whisper.cpp comparisons use
+sotto's vendored v1.9.3 helper (`~/Code/sotto/build/server/helpers/sotto-engine`)
+with `ggml-large-v3-turbo.bin` — same weights, different runtime.
+
+Silero VAD parity uses the `silero-vad` 6.2.1 pip package (torch JIT model)
+in the same venv (`scripts/oracle/gen-silero-golden.py` → tracked
+`goldens/silero-vad.json`); the engine's weights are `ggml-org/whisper-vad`'s
+`ggml-silero-v6.2.0.bin` in the HF cache (byte-identical to sotto's copy,
+SHA-256 `2aa269b7…`). `coremltools` is installed there too for the Core ML
+encoder experiment (benchmarks.md).
+
+The repo's untracked `.env` may pin `MLX_BUN_ORACLE_VENV` to a directory that
+no longer exists (it pointed at `~/Code/mlx-lm-example/.venv` on the M1 Max
+on 2026-09-15); every parity test importing `tests/support/paths.ts` then
+throws at import. Fix the path or run with `bun --env-file=/dev/null test …`.
+
 The Qwen MTP compatibility test accepts explicit local artifact paths through
 `MLX_BUN_TEST_MTP_TARGET` and `MLX_BUN_TEST_MTP_DRAFT`, alongside
 `MLX_BUN_TEST_QWEN38_MTP=1`. It compares speculative and ordinary greedy
@@ -101,7 +197,7 @@ Qwen artifacts are also present under
 `/Volumes/MLX-Models/models/{Qwen,mjriii}/`; header-only inventories under
 `reports/qwen38-rd/` are machine-local and do not load the GPU.
 
-**Authorized M4 Pro runtime update, staged 2026-09-07:** MLX 0.32.2 with
+**M4 Pro runtime update, staged 2026-09-07:** MLX 0.32.2 with
 MLX-C `c74db5307cc8ce122f48d97ef951b30578674e7f` is built under
 `reports/qwen38-rd/mlx-upgrade-0.32.2/install/lib/`. Explicit
 `MLX_BUN_LIBMLXC=<that-directory>/libmlxc.dylib` selects it per process.
@@ -134,10 +230,10 @@ eleven skips on the old core. The local native-pack
 candidate under `native-pack-candidate/` extracts into `native-pack-extracted/`;
 all five Mach-O files target 14.0, have valid ad-hoc signatures and load their
 non-system dependencies from siblings. These checks ran on macOS 27, not an
-actual macOS 14 host. The production native-pack manifest now targets this
-0.4.0 candidate and the verified local cache is
-`/Users/joshrossi/Library/Caches/mlx-bun/native-v0.4.0-arm64`. The candidate
-has not been published. The consolidated source passes 52 complete Qwen
+actual macOS 14 host. This build is published as native pack 0.4.0 and the
+verified local cache is
+`/Users/joshrossi/Library/Caches/mlx-bun/native-v0.4.0-arm64`. Publication
+details are in distribution.md. The consolidated source passes 52 complete Qwen
 forward/state/continuation cases and six Trellis generations, with fixed
 source/runtime/reference hashes and clean worker exits. The consolidated
 model-free suite passes 1,875 tests with ten skips; all typechecks pass.
@@ -179,6 +275,40 @@ against an M4 Pro output fixture. The test still requires exact output and
 state hashes; Python runs only the external reference. Trellis variant
 matrix tests allow 30 seconds for first-use kernel compilation on CI.
 
+The M1 Max override was replayed on 2026-09-08 with the same inputs using
+`/Users/joshrossi/Code/mlx-lm/.venv-mlx-0.32.2-macos14` (MLX/MLX-Metal
+0.32.2, MLX-LM 0.31.3). Bun matches its outputs and recurrent-state hashes
+exactly. The review file is
+`reports/qwen38-closeout/qwen-delta-m1-review.json`.
+
+Qwen3-Embedding tests also resolve `goldenPath("qwen3-embed")`, including
+machine overrides, and skip when their required binary blobs are absent.
+Regenerate with the matching Python and
+`scripts/oracle/gen-qwen3-embed-golden.py <artifact> --replay goldens/qwen3-embed/meta.json`;
+`--output-dir` selects a review directory. Replay preserves the text and token
+IDs and checks the tokenizer against them. Metadata records the device,
+runtime, artifact config, original metadata and output hashes. The M1 Max
+replay matches hidden states and pooled embeddings exactly. Cosine uses both
+vector norms: bf16 normalization does not guarantee a unit norm when the
+result is subsequently read as float32.
+
+The same M1 validation found June MiniCPM logit blobs without runtime
+provenance. Explicit `bun scripts/regen.ts minicpm5` with the matching venv
+preserves the artifact, prompt IDs and greedy trajectory; the regenerated
+logits match Bun exactly. The generator now records runtime/device/config
+provenance and all blob hashes. The old files and reviewed differences remain
+under `reports/qwen38-closeout/minicpm5-golden-before/` and
+`minicpm5-golden-review.json`.
+
+The Gemma bf16 and mixed-KV M1 replays also pass with unchanged prompt IDs
+on MLX 0.32.2. This current M1 venv contains **mlx-optiq 0.2.7** by installed
+distribution metadata, matching the documented M4 installation; it is not
+the historical 0.2.15 environment. OptiQ's module `__version__` incorrectly
+reports 0.2.5 there, so regeneration records distribution metadata. Old
+fixtures remain under `reports/qwen38-closeout/{parity,mixed-kv}-golden-before/`.
+These are current-stack parity checks, not a runtime-only A/B against the
+older mixed-KV fixture's 0.31.2/0.2.15 combination.
+
 Readable oracle source, in that `site-packages/`: `mlx_lm/models/gemma3.py`
 and `mlx_lm/server.py` for the port targets; `mlx_lm/models/cache.py` for the
 cache classes; `mlx_lm/tokenizer_utils.py` for prompt rendering;
@@ -250,7 +380,19 @@ servers DO validate: the Bearer token must start with `sk-optiq-`
 
 ## Bun and FFI facts
 
-**Bun ≥ 1.4.0.** `package.json` `engines.bun` is `>=1.4.0`. `Bun.Image`
+**Development and CI: Bun 1.4.2.** Build the committed frontend bundle with
+that version so its byte-for-byte freshness check uses the same compiler.
+Performance comparisons record the runtime and use the same version in both
+arms; historical results retain their original version.
+
+Non-login SSH on the M4 Pro can omit `/Users/joshrossi/.bun/bin` from `PATH`.
+Put that directory on the remote command's PATH when running the test suite:
+launching the parent Bun by absolute path alone does not let subprocess tests
+find `bun`. The quantized-row validation's initial seven command-entry/job
+failures disappear with that invocation corrected; no product code change was
+needed for those failures.
+
+**Runtime minimum: Bun ≥ 1.4.0.** `package.json` `engines.bun` is `>=1.4.0`. `Bun.Image`
 (native OS codecs via ImageIO, EXIF auto-orient, off-thread) is the vision
 decode path (`src/vision/preprocess.ts`); `Bun.Image.resize` is deliberately
 not used. The FFI ABI fix below is the other reason for the pin.
@@ -346,3 +488,7 @@ holds self-contained reproductions for upstream bug reports (the two Bun
 issues above, plus `optiq-mixed-kv-inert` and `vllm-metal-turboquant`); they
 are not part of the build. Committed absolute paths (the venv, the weights
 snapshot, `serve.sh`) are machine-specific on purpose — do not "correct" them.
+
+The quantized-KV golden producer (`bun scripts/regen.ts kvq`) passes an absolute output directory to Python, since the child runs from `scripts/`. This preserves the selected machine directory and existing prompt IDs. Missing M4 blobs were regenerated from the pinned oracle into the isolated `delayed-affine-m4-oracle` override; numerical expectations were not produced by the inference implementation. Evidence: `reports/qwen38-closeout/composition-baseline/oracle-generator-path/`.
+
+The M1 rotating-KV reference still identified MLX 0.31.2 and disagreed with both unchanged control and candidate on the KV8 greedy prefix. Regeneration with pinned MLX 0.32.2 retains the prompt IDs and passes exact logits plus both continuation checks. The matching M1 override and M4 reference caches are refreshed; previous manifests/blobs and provenance are retained with the generator-path evidence.

package/docs/reference/library-api.md

@@ -14,10 +14,13 @@ everything else under `src/` is internal and may change without notice.
 Every export is documented below, grouped the way `src/index.ts` groups
 them.
 
-One rule everywhere: **one generation at a time per process** (one GPU).
-The server serializes through a queue (or, under `--batch N`, its
-continuous scheduler — a server scheduling mechanism, not a library
-surface); direct library callers must do the same.
+The server uses continuous batching by default for eligible requests.
+Its scheduler coordinates work on shared model resources, including a single
+active request. Direct callers of `generate` must coordinate access to the
+same model and mutable state; launching independent generators does not
+automatically create a batch. Use the serving integration for its scheduling
+and request preparation, or compose the engine with explicit execution ports.
+See [server configuration](server-config.md) for supported combinations.
 
 ## Export map
 
@@ -672,3 +675,14 @@ When embedding the high-level library in your own compiled application, pass
 `command: ["/path/to/mlx-bun"]` to `openIsolatedHost`. Source/package consumers
 use the installed CLI source automatically; compiled callers supply the worker
 executable explicitly because their own executable is not the mlx-bun CLI.
+
+### Cache session affinity
+
+`GenerateOptions.cacheSessionId` carries application cache affinity through
+serving execution. The `PrefixCache` contract accepts the optional session ID
+on `take`, `prefetch` and `put`, and exposes optional `closeSession(id)`. The
+cache owns the association and both storage tiers; the ID does not change the
+numerical namespace or replace the request's full token history. Standalone
+`generate()` calls that supply their own cache retain caller-owned state; an
+ID alone does not create a process-wide cache. HTTP fields and lifecycle are
+documented in [server-api.md](server-api.md#cache-session-affinity).

package/docs/reference/memory.md

@@ -254,7 +254,7 @@ routes never touch the agent-tool surface above.
 
 ## Related
 
-- [cli.md](cli.md#memory--the-personal-wiki) — the `memory` command in the full CLI reference.
+- [cli.md](cli.md#memory-alias-setup--the-personal-wiki) — the `memory` command in the full CLI reference.
 - [server-api.md](server-api.md#get-apimemory--post-apimemoryinit) — `/api/memory/*` wire format.
 - [dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — the canonical pipeline design.
 - [docs/design/dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md), [docs/design/dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — synthesis internals and the local inference path.

package/docs/reference/models.md

@@ -4,6 +4,14 @@ The one supported-model list, plus how mlx-bun downloads, indexes, lists,
 and reclaims models. The store is the standard Hugging Face hub cache —
 nothing proprietary; `hf` and mlx-lm read/write the same tree.
 
+## Automatic model selection
+
+On a fresh install with no supported downloaded model, `serve` downloads and
+starts MiniCPM5-1B, then can download the device-recommended model in the
+background. Gemma e4b is the preferred general model once available and within
+the selection budget. `DEFAULT_REPO_ID` names that preference, not the
+first-download artifact. Explicit model selection bypasses this starter flow.
+
 ## Supported models
 
 The source of truth is code, not this table: `src/model/profile.ts`
@@ -16,25 +24,29 @@ is the bug.
 
 Scope is deliberate: a few families held to explicit oracle contracts
 (L1 = bit-exact vs mlx-lm, L2 = bit-exact vs mlx-optiq, L3 = measured, no
-oracle), plus a declared Tier-0 generic surface. There is no alias table —
+oracle), plus a declared Tier-0 generic surface. These are comparison
+contracts, not interchangeable claims about every artifact. The
+[coverage map](benchmarks.md#coverage-beyond-the-four-model-summary) identifies
+the test and remaining evidence limits for each roster row. There is no alias table —
 the "query" column is a substring that `resolve()` matches against
 already-downloaded repos (it must match exactly one repo; see
 [Query resolution rules](#query-resolution-rules-per-verb)).
 
 | `serve` query | Validated artifact (HF repo) | Family · load path · tier | Modalities | Draft sources | KV schemes | Notes |
 | --- | --- | --- | --- | --- | --- | --- |
-| `e4b` | `mlx-community/gemma-4-e4b-it-OptiQ-4bit` | Gemma 4 (`gemma4*`) · dedicated `gemma4` graph with a **generated** specialization keyed on the config fingerprint · L1 | text · **image** (SigLIP tower in the bf16 `optiq_vision.safetensors` sidecar) · **audio** (Conformer tower in the same sidecar) | two-model · Gemma `-assistant` (L2) · locally-trained DSpark (`dspark.json`) · ngram | bf16 · uniform 4/8 · `config` (kv_config.json, batches) · turbo · `--paged-kv` | **Default model** (`DEFAULT_REPO_ID`, src/fit.ts). The only audio-capable artifact. ~7.0 GB on disk (benchmarks.md legend, 2026-06-15). Trains (LoRA / ORPO / SFT). |
+| `e4b` | `mlx-community/gemma-4-e4b-it-OptiQ-4bit` | Gemma 4 (`gemma4*`) · dedicated `gemma4` graph with a **generated** specialization keyed on the config fingerprint · L1 | text · **image** (SigLIP tower in the bf16 `optiq_vision.safetensors` sidecar) · **audio** (Conformer tower in the same sidecar) | two-model · Gemma `-assistant` (L2) · locally-trained DSpark (`dspark.json`) · ngram | bf16 · uniform 4/8 · `config` (kv_config.json, batches) · turbo · `--paged-kv` | **Preferred model after download**; fresh installs first serve MiniCPM5. See [automatic selection](#automatic-model-selection). The only audio-capable artifact. ~7.0 GB on disk (benchmarks.md legend, 2026-06-15). Trains (LoRA / ORPO / SFT). |
 | `12B` | `mlx-community/gemma-4-12B-it-OptiQ-4bit` | Gemma 4 · generated specialization · L1 | text · **image** (encoder-free `gemma4_unified_vision` declared in config.json — no sidecar needed). No audio: the snapshot's sidecar is an audio **stub** (`embed_audio` only, no `audio_tower.*`) | two-model · `-assistant` · DSpark local · **DeepSpec released** (`deepseek-ai/dspark_gemma4_12b_block7`) · ngram | bf16 · uniform · `config` · turbo · `--paged-kv` | ~8.4 GB (2026-06-15). `largestRecommendedRepoId` pick for ≥ 24 GB RAM (src/fit.ts). Trains. |
 | `26B` | `mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit` | Gemma 4 MoE · generated specialization · L1 | text · image **only if** the snapshot ships the SigLIP sidecar (registry `has_vision_sidecar`; the tower is `gemma4_vision`). No `audio_config` → no audio | two-model · `-assistant` · ngram | bf16 · uniform · `config` · turbo · `--paged-kv` | ~18 GB (2026-06-15). `largestRecommendedRepoId` pick for ≥ 48 GB RAM. |
-| any other `gemma4*` config (1B, e2b, 31B, …) | — | Gemma 4 · the `Gemma4Model` monolith (no generated specialization) · L1 by family | per the registry's vision/audio flags | same as the family | same as the family | Loads by `model_type`; **no artifact is pinned in tests/support/goldens for these**, so treat as family-supported, artifact-unvalidated. |
+| any other `gemma4*` config (1B, e2b, 31B, …) | — | Gemma 4 · the `Gemma4Model` monolith (no generated specialization) · family support, artifact unvalidated | per the registry's vision/audio flags | same as the family | same as the family | Loads by `model_type`; **no artifact is pinned in tests/support/goldens for these**, so treat as family-supported, artifact-unvalidated. |
 | `MiniCPM5` | `mlx-community/MiniCPM5-1B-OptiQ-4bit` | MiniCPM5 · `model_type: llama` matched by exact shape (`isMiniCPM5Config`) → dedicated `minicpm5` graph · L1 / L2 | text | two-model · ngram | bf16 · uniform · `config` (batches — Phase 3.1) · turbo (quality curve measured 2026-07-06, benchmarks.md §3) | Sub-GB starter (benchmarks.md legend). Trains (segmented backward). Tool calling (XML). |
 | `Qwen3.5-4B` | `mlx-community/Qwen3.5-4B-OptiQ-4bit` | Qwen3.5 (`qwen3_5` / `qwen3_5_text`, dense hybrid gated-DeltaNet) · dedicated `qwen3.5` graph · L1 | text (image/video only when the artifact ships vision weights — see Qwen3.8) | two-model · ngram · `mtp` if a `qwen3_5_mtp` companion exists | bf16 (L1) · `config` (KV-ON parity passed 2026-06-15, PLAN 14e) | Thinking + tool calling. Batches on the SSM path (`MLX_BUN_BATCH_SSM=0` reverts). The MoE variant `qwen3_5_moe` is **not** supported. |
 | `Qwen3.8` | `mlx-community/Qwen3.8-27B-OptiQ-4bit` @ `b04599de95d7a9bfbd7f208d347c0f10d9432a42` | Qwen3.5 family · **exact artifact profile** `qwen3.8-27b-optiq-4bit` (safetensors + `qwen3.5` graph) · L1 | text · **image** · **video** (`image_url` / `video_url` parts; Qwen3-VL tower + mRoPE; video decodes via the AVFoundation sidecar `mlx-bun-frame-extract`, 2 fps, ≤ 768 frames). Video never combines with audio; no audio | **native MTP head** (`mlx-community/Qwen3.8-27B-MTP-bf16`, auto-detected `qwen3_5_mtp`; `--draft-kind mtp` with no `--draft-model` uses a bundled `mtp/` subfolder) · two-model · ngram | bf16 (L1) · `config` is **Lab** — policy copied from the same-topology Qwen3.6 artifact, no model-specific oracle (benchmarks.md, 2026-08-22) | `reasoning_effort` (xhigh/medium/low) + `preserve_thinking` + tool calling. `/v1/models` advertises vision. Shipped in v0.2.0. |
 | `Qwen3-Embedding` | `mlx-community/Qwen3-Embedding-4B-4bit-DWQ` | plain `qwen3` · dedicated `qwen3` graph · L1 | text | — | — | **Embeddings only**: `/v1/embeddings` + `mlx-bun embed`. Auto-picked by `embed` when no query is given; never the chat default. |
 | `Qwen3-30B` | `mlx-community/Qwen3-30B-A3B-Thinking-2507-4bit` | Qwen3-MoE (`qwen3_moe`) · dedicated `qwen3-moe` graph · L1 (opt-in slow parity test, `MLX_BUN_TEST_QWEN3_MOE=1`) | text | two-model · ngram | bf16 (the artifact ships no kv_config.json) | Sparse top-k SwitchGLU experts. |
 | `diffusiongemma` | `mlx-community/diffusiongemma-26B-A4B-it-OptiQ-4bit` | DiffusionGemma (`diffusion_gemma`) · dedicated graph + **denoising loop** (non-autoregressive) · L2 (optiq) | text · **image** (single image; its own inline-quantized SigLIP tower). No video / audio | — (no AR verify loop) | — (canvas model; the `--kv-quant` axis does not apply) | Serial lane always. Diffusion-native LoRA trains. |
+| `whisper` | `mlx-community/whisper-large-v3-turbo` (fp16, 1.6 GB, `weights.safetensors`) | Whisper (`whisper`) · dedicated encoder-decoder graph + transcription loop · **L1 vs mlx-whisper 0.4.3** (mel, encoder output and per-step decoder logits bit-exact; `tests/parity/whisper.test.ts`) | **audio in → text out** only: `/v1/audio/transcriptions`, `/v1/audio/translations`, `mlx-bun transcribe`. Never the chat model. | — | — | Tokenizer files are not in the mlx-community repo; the loader resolves `openai/whisper-large-v3-turbo`'s `tokenizer.json` from the HF cache (2.4 MB, `mlx-bun get` it once). Beam search (`beam_size`) is ours — mlx-whisper has none; checked against whisper.cpp's beam-5 transcripts. Word timestamps not yet. Serves as a companion beside any chat model or alone (transcription-only server). Any `mlx-community/whisper-*` fp16 checkpoint loads (80-mel v1/v2 vocab resolves `openai/whisper-large-v2`). |
 | `GLM-5.2-colibri` | `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` @ `3cc8db99b1b13fc79325d987ba3c1c430766b3b8` | GLM-5.2 (`glm_moe_dsa`) · **Colibri container loader** + dedicated `glm5.2` graph (streamed experts) · **exact artifact profile**, L3 | text | **checkpoint-native MTP** (`--mtp on`, the default; mutually exclusive with `--draft-model`) | checkpoint-native compressed **MLA/DSA** cache (its own scheme; SSD tier v3 persists it) | Fixed-context admission: `--context-length` (4096) and `--max-tokens` (128) are reserved by the header-only resource equation before weights open; `--memory-budget` defaults to min(25 GiB, RAM). `--mtp off` enables ordinary batching. Embeddings, vision/audio, adapters, and training are unsupported. See [below](#direct-container-artifacts-glm-5x). |
-| Tier-0 generic | validated manifest (`tests/support/universal-manifest.ts`): `Llama-3.2-1B-Instruct-4bit`, `Qwen2.5-0.5B-Instruct-4bit`, `Qwen3-0.6B-4bit`, `quantized-gemma-2b-it`, `gemma-2-2b-it-4bit`, `Phi-3.5-mini-instruct-4bit`, `GLM-4-9B-0414-4bit`, `granite-3.3-2b-instruct-4bit`, `starcoder2-3b-4bit`, `SmolLM3-3B-4bit` (all `mlx-community/`) and `ekryski/OLMo-2-0425-1B-Instruct-4bit` | 11 `model_type`s: `llama`, `smollm3`, `qwen2`, `qwen3`, `gemma`, `gemma2`, `phi3`, `olmo2`, `glm4`, `granite`, `starcoder2` (+ mlx-lm remaps `mistral`, `iquestcoder` → `llama`) · `universal-dense` graph · L1 | text | two-model · ngram | bf16 (the validated cell) | `ls` labels these `supported (generic)`. Batch only for plain full-attention archs (gated B=2 on Llama-3.2-3B); gemma2-family / sliding-window archs run serial. No compiled specialization. A `model_type` absent from the table refuses to load. |
+| Tier-0 generic | validated manifest (`tests/support/universal-manifest.ts`): `Llama-3.2-1B-Instruct-4bit`, `Qwen2.5-0.5B-Instruct-4bit`, `Qwen3-0.6B-4bit`, `quantized-gemma-2b-it`, `gemma-2-2b-it-4bit`, `Phi-3.5-mini-instruct-4bit`, `GLM-4-9B-0414-4bit`, `granite-3.3-2b-instruct-4bit`, `starcoder2-3b-4bit`, `SmolLM3-3B-4bit` (all `mlx-community/`) and `ekryski/OLMo-2-0425-1B-Instruct-4bit` | 11 `model_type`s: `llama`, `smollm3`, `qwen2`, `qwen3`, `gemma`, `gemma2`, `phi3`, `olmo2`, `glm4`, `granite`, `starcoder2` (+ mlx-lm remaps `mistral`, `iquestcoder` → `llama`) · `universal-dense` graph · L1 | text | two-model · ngram | bf16; Llama full-attention uniform KV4/KV8 | `ls` labels these `supported (generic)`. Shared full-attention execution is qualified on Llama-3.2-3B at B1/B4, including speculative KV4/KV8 and generated RAM/SSD state; gemma2-family / sliding-window archs run serial. No compiled specialization. A `model_type` absent from the table refuses to load. |
 
 Rules that apply across rows (all enforced in `src/server.ts`):
 
@@ -45,8 +57,11 @@ Rules that apply across rows (all enforced in `src/server.ts`):
   MTP is its own row. Qwen MTP companions may store dense or affine-quantized
   projections; the loader uses each module's quantization metadata. Target
   verification is unchanged, while draft acceptance and speed depend on the
-  companion. Any quantized KV scheme excludes the speculative lane
-  (requests keep the scheme and decode serially without speculation).
+  companion. In the shared executor, qualified plain, uniform,
+  per-layer affine and TurboQuant layouts compose with drafting through the
+  target/provider interfaces. Positive affine conversion thresholds are also
+  supported by library requests. The loaded model and provider determine
+  which combinations can execute.
 - **Qwen3.5-family checkpoint generations.** Both ship the same graph and both
   load: the mlx-lm-converted naming (`language_model.model.*`,
   `language_model.lm_head`, `vision_tower.*`) and the transformers-5.8 export
@@ -60,7 +75,9 @@ Rules that apply across rows (all enforced in `src/server.ts`):
 - **KV schemes.** `--kv-quant 4|8` (uniform, L1) and `config` (per-layer
   `kv_config.json`, L2) apply to the autoregressive families that ship the
   file; `turbo` additionally requires a full-attention `head_dim` in
-  {64, 128, 256, 512} (refused at startup otherwise) and is solo-only;
+  {64, 128, 256, 512} (refused at startup otherwise). Qualified TurboQuant
+  targets support shared ordinary and speculative execution; sliding layers
+  remain bf16;
   `--paged-kv` is Gemma 4 only, bf16, serial. Flag detail:
   [server-config.md](server-config.md).
 - **Modalities.** Image requests reach a Gemma 4 model only when
@@ -323,3 +340,7 @@ Safety rails:
 
 After a real deletion, gc re-scans so the registry drops the reaped
 rows immediately.
+
+
+Non-Qwen grouped TurboQuant drafting, including assistant donor attention, uses
+the shared cache interfaces. See [composition coverage](../design/batching.md#turboquant-donor-attention-and-grouped-draft-composition).