mlx-bun 0.2.0 → 0.4.0

package/docs/design/generic-model-support.md

@@ -1,384 +1,705 @@
-# Generic model support — run anything mlx-lm runs
+---
+status: active
+axis: ON
+canonical-for: model-support
+plan-anchor: "Serving architecture consolidation `[~]` (opened 2026-08-21)"
+last-verified: 2026-08-23
+---
+
+# Model support — run anything mlx-lm runs, and the ports we target
 
-**Status: Phase 1 + 1.5 BUILT (2026-07-01, commit 9bd9f1b)** — `src/model/universal/`
-(UniversalDense + descriptor table for 11 archs, rope factory bit-exact vs oracle
-fixtures, DenseLinear/bf16 checkpoints, weight audit, dispatch ladder in
-factory/support, manifest-driven parity harness). Parity gates light per-model
-once the small checkpoints are downloaded (`scripts/regen-universal-goldens.ts` →
-`MLX_BUN_TEST_UNIVERSAL=1`). Phases 2 (UniversalMoE), 3 (SSM/MLA one-offs),
-4 (`mlx-bun port` scaffold) remain design.
+**This is the one design doc for model support.** New model ports become
+SECTIONS under §6 (Ports) of this file, not new files in `docs/design/`.
+Per-port phase logs, gate results, and dates belong in PLAN.md (anchors are
+named per section); this doc keeps the design and the oracle each port is
+held to.
 
 Direction (Josh): *"we should be able to run any model that mlx-lm can run.
 Run any model generically; for the specific models we target, optimize them
 further — push the quality/performance border outwards. Like the gemma
 generator methodology."*
 
-The architecture already embodies the two-tier idea for one family:
-`src/model/gemma4-base.ts` (shared machinery) + `gemma4.ts` (config-driven
-monolith) + `src/model/generated/*.ts` (codegen'd specializations via
-`scripts/gen-model.ts`, gated bit-exact by `tests/generated-parity.test.ts`).
-This doc generalizes that shape to the whole mlx-lm zoo:
+Two tiers, one contract:
 
 - **Tier-0 GENERIC** — one config-driven "universal" module per architecture
-  *family* (dense llama-family first, MoE second). Runs any covered checkpoint
-  at **L1 parity** (bit-exact vs mlx-lm on this machine's GPU). Monolith path
-  only: no compiled decode, no kv-quant. Slow, never broken.
-- **Tier-1 TARGETED** — a model *graduates*: dedicated/generated forward,
-  optiq-quant **L2** parity, Lab experiments where warranted, benchmark
-  entries. Exactly what gemma4 / minicpm5 / qwen3_5 already are.
-
-The fidelity contract is unchanged (three-level tree): **L1 = mlx-lm
-bit-exact is the gate for ANY new model**, generic or targeted. A model the
-generic tier cannot hold to L1 parity is not "supported with caveats" — it is
-unsupported until fixed.
+  *family*. Runs any covered checkpoint at **L1 parity** (bit-exact vs mlx-lm
+  on this machine's GPU). Monolith path only, without compiled decode.
+  Standard attention now uses the shared plain/affine KV port; the softcap
+  branch retains plain KV. Qualified full-attention targets compose delayed
+  affine conversion with standalone and lookup drafting through the same
+  cache transaction interface; rotating layouts use that interface for
+  per-layer conversion and assistant donors. Built: `src/model/universal/` (dense
+  llama-family). Design only: UniversalMoE.
+- **Tier-1 TARGETED** — a model *graduates*: dedicated or generated forward,
+  L2 (mlx-optiq) composition where an optiq artifact exists, Lab experiments
+  where warranted, benchmark entries. The ports in §6.
+
+**L1 = mlx-lm bit-exact is the gate for ANY new model**, generic or targeted.
+Where no mlx-lm arm exists (audio, vision towers, DiffusionGemma, GLM-5.2's
+Colibri container) the oracle is whoever ships the capability — optiq's
+internal machinery, mlx-vlm, or the pinned Colibri engine — and the profile
+declares that fidelity honestly (§3). A model the generic tier cannot hold to
+L1 parity is not "supported with caveats"; it is unsupported until fixed.
 
 ---
 
-## 1. The mlx-lm model zoo, inventoried
+## 1. The mlx-lm zoo, inventoried (mlx-lm 0.31.3, the oracle venv)
 
-Source: `mlx-lm 0.31.3` (the oracle venv),
-`site-packages/mlx_lm/models/*.py`. **108 model files** after excluding
+108 model files in `site-packages/mlx_lm/models/` after excluding
 infrastructure (`base`, `cache`, `rope_utils`, `activations`,
 `switch_layers`, `ssm`, `gated_delta`, `mla`, `bitlinear_layers`,
-`pipeline`). `utils.py MODEL_REMAPPING` adds free aliases on top
-(`mistral→llama`, `kimi_k2→deepseek_v3`, `qwen2_5_vl→qwen2_vl`,
-`minimax_m2→minimax`, `iquestcoder→llama`, `falcon_mamba→mamba`, …).
-
-### Buckets (by reading llama.py in full + feature-grep + sampling ~20 files)
-
-**A. Dense llama-family — ~36 files, one parameterized module covers them.**
-All are: embed → N × (norm → GQA-attention(+rope) → residual → norm →
-MLP → residual) → norm → head. Every file is a ~170–240-line permutation of
-the same dozen knobs (verified deltas listed in §3.1):
-
-> llama (+mistral, iquestcoder via remap), qwen2, qwen3, gemma, gemma2,
-> helium, olmo2, olmo3, smollm3, starcoder2, stablelm, phi3, internlm2,
-> internlm3, granite, exaone, exaone4, glm, glm4, nemotron, minicpm, mimo,
-> ministral3, seed_oss, hunyuan_v1_dense, ernie4_5, cohere, cohere2,
-> apertus, nanochat, lille-130m, telechat3, solar_open, olmo, phi, plamo
-
-(`smollm3` is literally a 75-line subclass of `llama` that swaps some
-layers' rope for a no-op; `solar_open` is 38 lines. mlx-lm itself treats
-this family as one module with per-file ModelArgs.)
-
-**B. MoE llama-family (SwitchGLU experts) — ~17 files, a second module.**
-Same block, MLP replaced by router + `SwitchGLU` (+ optional shared expert):
-
-> mixtral, qwen2_moe, qwen3_moe, olmoe, phimoe, dots1, granitemoe,
-> ernie4_5_moe, glm4_moe, hunyuan, Klear, bailing_moe, exaone_moe, deepseek
-> (v1: MoE without MLA), step3p5, mimo_v2_flash, afmoe
-> (llama4_text / gpt_oss are near-family but carry extra deltas —
-> chunked-attention/NoPE, attention sinks.)
-
-**C. MLA lineage (DeepSeek latent attention, usually + MoE + yarn) — ~11:**
-> deepseek_v2, deepseek_v3 (+kimi_k2 remap), deepseek_v32, glm4_moe_lite,
-> glm_moe_dsa, kimi_k25, kimi_vl, longcat_flash(+ngram), minicpm3, youtu_llm
-
-**D. SSM / hybrid / linear-attention — ~17, individually shaped:**
-> mamba, mamba2 (+falcon_mamba remap), falcon_h1, jamba, granitemoehybrid,
-> nemotron_h, plamo2, lfm2, lfm2_moe, qwen3_next, kimi_linear, minimax,
-> recurrent_gemma, rwkv7, baichuan_m1, **qwen3_5 (already ported)**,
-> qwen3_5_moe
-
-**E. Legacy / exotic one-offs — ~13, deprioritized:**
-> gpt2, gpt_bigcode, gpt_neox, qwen (v1), openelm (per-layer varying dims),
-> phi3small (blocksparse attn), phixtral, dbrx, nemotron-nas (variable
-> blocks), bitnet (bitlinear), afm7, baichuan_m1 (also in D), lille-130m-ish
-> oddities
-
-**F. Vision/text wrappers — ~10, ~free once the text base exists:**
-> gemma3, gemma4 (ours already), qwen2_vl (+qwen2_5_vl), qwen3_vl,
-> qwen3_vl_moe, kimi_vl, lfm2-vl, mistral3 (+llava remap), pixtral, llama4
-> — mlx-lm runs only their *text* stack; each wrapper is ~50–120 lines of
-> config unwrapping + weight-prefix stripping.
-
-### Coverage math
-
-| Bucket | Files | Unlocked by |
-|---|---|---|
-| A. dense llama-family | ~36 (33%) | **one** UniversalDense module (Phase 1) |
-| B. MoE llama-family | ~17 (16%) | **one** UniversalMoE module (Phase 2) |
-| F. vision wrappers | ~10 (9%) | thin config/prefix shims over A/B (Phase 1–2) |
-| C. MLA lineage | ~11 (10%) | one MLA attention module (Phase 3, demand-driven — most are >24 GB even at 4-bit) |
-| D. SSM/hybrid | ~17 (16%) | individual ports, qwen3_5 pattern (Phase 3) |
-| E. legacy/exotic | ~13 (12%) | individual ports, likely never |
-| already in mlx-bun | 5 | gemma4(+text), qwen3, qwen3_5, minicpm5(llama-typed), diffusion_gemma |
-
-**Two generic modules + the wrapper shims ≈ 60% of the zoo by file count,
-and >90% of what actually gets downloaded** (Llama 3.x, Mistral/Ministral,
-Qwen 2.5/3 dense+MoE, Gemma 1–3, Phi-3/4, SmolLM3, OLMo-2/3, GLM-4,
-Granite, Mixtral, OLMoE are all in A/B/F).
+`pipeline`). `utils.py MODEL_REMAPPING` adds free aliases (`mistral→llama`,
+`kimi_k2→deepseek_v3`, `qwen2_5_vl→qwen2_vl`, `falcon_mamba→mamba`, …).
+
+| Bucket | ~Files | Shape | Unlocked by |
+|---|---|---|---|
+| A. dense llama-family | ~36 | embed → N × (norm → GQA attn(+rope) → residual → norm → MLP → residual) → norm → head; every file a permutation of a dozen knobs | **UniversalDense** (built) |
+| B. MoE llama-family | ~17 | same block, MLP → router + SwitchGLU (+ shared expert) | **UniversalMoE** (design) |
+| F. vision/text wrappers | ~10 | mlx-lm runs only the text stack; ~50–120 lines of config unwrapping + prefix stripping | thin shims over A/B |
+| C. MLA lineage | ~11 | DeepSeek latent attention, usually + MoE + yarn | one MLA module, demand-driven (most exceed local RAM at 4-bit) |
+| D. SSM / hybrid / linear-attention | ~17 | individually shaped (mamba, jamba, lfm2, qwen3_next, rwkv7, qwen3_5 …) | individual ports, the qwen3_5 pattern |
+| E. legacy / exotic | ~13 | gpt2, openelm, phi3small, dbrx, bitnet … | individual ports, likely never |
+
+Two generic modules + the wrapper shims cover roughly 60% of the zoo by
+file count and the large majority of what actually gets downloaded (Llama
+3.x, Mistral/Ministral, Qwen 2.5/3 dense+MoE, Gemma 1–3, Phi-3/4, SmolLM3,
+OLMo-2/3, GLM-4, Granite, Mixtral, OLMoE are all in A/B/F).
 
 ---
 
-## 2. What mlx-bun already has to build on
-
-Audited 2026-07-01; file references are load-bearing.
+## 2. Building blocks (what every tier is made of)
 
-- **Config parsing** — `src/config.ts loadModelConfig`: already a *normalized*
-  config (`TextConfig`) with per-model_type quirk handling inline (gemma
-  nested `text_config`, qwen3_5 flat rope dict, llama defaults, diffusion
-  backfill). Already carries: layerTypes/slidingWindow, headDim fallback,
-  tie, softcap, partialRotaryFactor, `ropeParameters` map, per-layer quant
-  map (`quantFor`), kv_config. **This is 70% of the normalization layer;**
-  it needs the delta fields from §3.1 and a table-driven (not if-chain)
-  quirk mechanism before it drowns.
+- **Config parsing** — `src/config.ts loadModelConfig` produces a normalized
+  `TextConfig` (layerTypes/slidingWindow, headDim fallback, tie, softcap,
+  partialRotaryFactor, `ropeParameters`, per-layer quant map via `quantFor`,
+  kv_config, sidecar detection). Per-arch quirks for the generic tier live in
+  the descriptor table (§4.1), not in this if-chain.
 - **Primitives** — `src/model/gemma4-base.ts`: `QuantizedLinear` (+LoRA
   residual), `QuantizedEmbedding` (+tied `asLinear`), `RMSNorm`,
-  `QuantizedSwitchLinear` (MoE experts — already used by 26B),
-  `KVCache`/`RotatingKVCache`(+quantized variants), mask machinery,
-  `logitSoftcap`, `disposing`. `src/model/qwen3.ts` (276 lines) is the
-  proof-of-shape: a complete dense port using only these primitives.
-  `minicpm5.ts`/`qwen3_5.ts` show the "dedicated module conforming to
-  RuntimeModel" pattern.
-- **Gating/dispatch** — `src/model/support.ts` (allowlist by model_type +
-  config predicates), `src/model/factory.ts` (specific classes →
-  fingerprint-matched generated class → monolith; *rejects* unknown types),
-  `src/model/fingerprint.ts` (structural hash for generated dispatch).
-- **Generator** — `scripts/gen-model.ts`: emits unrolled, branch-resolved
-  `forwardLayers` overrides *of Gemma4Model specifically* (reads the shard
-  index for layout facts, folds kv-quant constants, guards cache signature,
-  falls back to monolith). Gated by `tests/generated-parity.test.ts`
-  (bit-exact vs monolith, fingerprint asserted, fast-path-use asserted).
-- **Parity harness** — the `scripts/regen-*-goldens.ts` +
-  `tests/*-parity.test.ts` pattern (e.g. `regen-qwen-parity-goldens.ts`):
-  oracle venv emits `<prefix>-parity.json` (explicit prompt ids + greedy
-  ids) + per-step raw logit `.bin`s; the test replays ids (no cross-stack
-  tokenizer dependency) and compares bit-exact on the same GPU. Plus
-  `scripts/check-minicpm5-port.ts` (hard, no-skip port check) and
-  `scripts/inspect-model.ts` (tensor name/shape/dtype dump — the seed of
-  the weight audit).
-- **Tokenizer** — `src/tokenizer.ts`: `@huggingface/tokenizers` (pure JS)
-  reads stock `tokenizer.json` + `tokenizer_config.json`; already
-  generic across BPE/SentencePiece-as-tokenizer.json. Known landmine class:
-  BOS conventions (the double-BOS bug cost a debug session — see memory);
-  chat-template rendering rides Pi, with `/v1` as the boundary.
-
-### Gaps the generic tier must close (found in this audit)
-
-1. **No unquantized Linear.** `QuantizedLinear.load` throws without a
-   `.scales` tensor (`gemma4-base.ts:94`). Tier-0 v1 can inherit this
-   constraint (target mlx-community 4/8-bit checkpoints — what users pull
-   anyway), but a plain `DenseLinear` (bf16/fp16 `ops.matmul`) is a required
-   fast-follow: many small models ship only fp16.
-2. **No additive linear bias.** `QuantizedLinear`'s `biases` are quant
-   zero-points, not a bias *term*. qwen2 (attention qkv bias), stablelm,
-   starcoder2, phi et al. need an optional post-matmul `+ bias` in the
-   linear primitive.
-3. **Rope variants.** Only default/proportional exist in the runtime.
-   mlx-lm `rope_utils.py` also ships: `linear`, `llama3` (Llama3RoPE),
-   `yarn` (3 spellings), `longrope`/`su` (SuScaledRoPE), `mrope`. llama3 +
-   yarn + longrope are hard requirements for bucket A (Llama 3.x, Phi-3,
-   DeepSeek-lineage, various long-context finetunes).
-4. **Attention-score softcap** (gemma2): needs the manual
-   (q·kᵀ → tanh-cap → softmax) path — `ops.sdpa` can't express it.
-5. **factory.ts rejects unknown types** rather than attempting a generic
-   fallback (by design today; changes in §3.3).
+  `QuantizedSwitchLinear` (MoE experts), `KVCache`/`RotatingKVCache` (+
+  quantized and TurboQuant variants), mask machinery, `logitSoftcap`,
+  `disposing`. `src/model/universal/modules.ts` adds the dense
+  (unquantized) `nn.Linear`/`nn.Embedding` ports and the load-time
+  `WeightAudit`.
+- **Generator** — `scripts/gen-model.ts` emits unrolled, branch-resolved
+  `forwardLayers` overrides of `Gemma4Model` into `src/model/generated/`
+  (e4b, 12B, 26B), dispatched by config fingerprint, gated bit-exact vs the
+  monolith by `tests/generated-parity.test.ts`.
+- **Parity harness** — `scripts/regen.ts <job>` (jobs: `parity`,
+  `parity-26b`, `kvq`, `rotating-kvq`, `mixed-kv`, `lora`, `fused-sdpa`,
+  `universal`, `qwen-parity`, `minicpm5`, `minicpm5-kv`, `turboquant`,
+  `audio-fixtures`) drives the oracle venv
+  (`/Users/joshrossi/Code/mlx-lm/.venv/bin/python`) to emit
+  `<prefix>-parity.json` (explicit prompt ids + greedy ids) plus per-step raw
+  logit `.bin`s; `tests/*-parity.test.ts` replay the ids (no cross-stack
+  tokenizer dependency) and compare bit-exact on the same GPU. Manifests are
+  tracked; `.bin` blobs are machine-specific and untracked
+  (goldens/README.md).
+- **Tokenizer** — `src/tokenizer.ts` (`@huggingface/tokenizers`, pure JS)
+  reads stock `tokenizer.json` + `tokenizer_config.json`. Known landmine:
+  BOS conventions (the double-BOS bug). Chat-template rendering rides Pi;
+  `/v1` is the boundary.
 
 ---
 
-## 3. Design
-
-### 3.1 Tier-0: `UniversalDense` (src/model/universal/)
+## 3. Model profiles — capability declaration (`src/model/profile.ts`)
 
-One module, shaped exactly like `qwen3.ts` but every delta a config field.
-**The normalized config is the union of mlx-lm's per-arch ModelArgs**, and a
-declarative **arch descriptor table** supplies per-model_type defaults +
-config-key aliases — mirroring mlx-lm's structure (shared base module,
-per-arch args) without one file per arch:
+Model loading resolves ONE immutable profile before weights are opened.
+A profile declares **construction only**: request-level methods (MTP, KV
+schemes, adapters, grammar, sampling) are resolved elsewhere and cannot be
+rewritten by profile selection.
 
 ```ts
-// src/model/universal/archs.ts — ~15–25 lines per arch vs mlx-lm's ~200
-export interface ArchDescriptor {
-  // block structure
-  block: "sequential" | "sandwich" | "parallel";  // llama | gemma2/glm4 | cohere/phi
-  norm: { kind: "rmsnorm" | "layernorm"; plusOne?: boolean; bias?: boolean };
-  qkNorm: "none" | "head-rms" | "layernorm-2d";   // qwen3/olmo2 | cohere
-  mlp: { kind: "gated" | "plain"; act: "silu" | "gelu" | "gelu_tanh" | "relu2" };
-  fusedQkv?: boolean; fusedGateUp?: boolean;      // phi3 — split in sanitize
-  // scalars (all read from config.json when present; descriptor = default)
-  attnLogitSoftcap?: number;   // gemma2
-  logitScale?: ConfigPath;     // cohere (multiply), granite logits_scaling (divide)
-  embedScale?: "sqrt-hidden" | "mup" | null;      // gemma family, minicpm
-  residualMultiplier?: ConfigPath;                // granite
-  bias: { qkv?: boolean; o?: boolean; mlp?: boolean };
-  nope?: "interval" | "list";  // smollm3, llama4-style no-rope layers
-  // weights
-  weightMap?: Partial<Record<CanonicalName, string>>;  // starcoder2 c_fc/c_proj, gpt2 h.N.*
-  sanitize?: (names: string[]) => Rename[];       // fused-proj splits, inv_freq drops
-  configAliases?: Record<string, string>;         // norm_epsilon→rms_norm_eps etc.
+interface ModelProfile {
+  id: string;
+  artifactFingerprint?: string;   // exact external identity (hf:<repo>@<rev>); family profiles omit
+  configFingerprint?: string;     // structural guard for an exact artifact declaration
+  fidelity: FidelityTarget;       // {l1, mlx-lm, bit-exact} | {l2, mlx-optiq, bit-exact} | {l3, null, measured}
+  requiredCapabilities: EngineCapability[];
+  execution: { loader: "safetensors" | "colibri";
+               graph: "gemma4" | "minicpm5" | "qwen3.5" | "qwen3" | "qwen3-moe"
+                    | "diffusion-gemma" | "glm5.2" | "universal-dense";
+               loop: "autoregressive" | "diffusion";
+               specialization: "artifact" | "dedicated" | "generated" | "generic" };
 }
-export const ARCHS: Record<string /* model_type */, ArchDescriptor> = { ... };
 ```
 
+Engine capabilities (`ENGINE_CAPABILITIES`): `autoregressive`,
+`colibri-container`, `diffusion`, `diffusion-gemma-graph`, `gemma4-graph`,
+`generated-graph`, `glm5.2-graph`, `minicpm5-graph`, `mixed-precision-kv`,
+`native-mtp`, `qwen3-graph`, `qwen3-moe-graph`, `qwen3.5-graph`,
+`recurrent-state`, `safetensors`, `streamed-experts`,
+`universal-dense-graph`, `vision-sidecar`.
+
+Resolution (`resolveModelProfile`):
+
+1. **Exact artifact profiles outrank family profiles.** The external
+   fingerprint is derived only from an immutable HF snapshot path
+   (`models--<org>--<repo>/snapshots/<40–64 hex>`); relocation keeps the
+   identity, mutable aliases and arbitrary local dirs return null. A matched
+   exact profile whose config fingerprint or graph acceptance mismatches
+   REFUSES — it never falls back. Built-in exact declarations today:
+   `qwen3.8-27b-optiq-4bit` (L1) and `glm5.2-colibri-int4-int8-mtp` (L3).
+2. **Family profile** by config predicate (`src/model/support.ts`), in this
+   order: `glm_moe_dsa` → `diffusion_gemma` → MiniCPM5 (exact-shape `llama`
+   predicate) → `qwen3_5`/`qwen3_5_text` (dense hybrid only; the MoE variant
+   is excluded) → `qwen3_moe` → `qwen3` → `gemma4*` (generated class if
+   `GENERATED.has(fingerprint)`, else monolith) → universal-dense if
+   `genericArgsFor(config)` resolves → throw naming the targeted list and
+   the generic descriptor keys.
+3. `validateProfile` enforces the fidelity/oracle pairing, loader↔graph
+   (`glm5.2` ⇒ colibri), loop↔graph (`diffusion-gemma` ⇒ diffusion), and that
+   every capability the execution composition implies is declared. Missing
+   engine capabilities refuse; nothing downgrades.
+
+**Factory** (`src/model/factory.ts`): `openModel(dir)` resolves the profile,
+routes `loader: "colibri"` through `openGlm52RuntimeModel` (header-only
+memory plan first, then `Glm52Model.openStreamed`; the Colibri snapshot has no
+`model.safetensors.index.json` so it bypasses `Weights.open`), and otherwise
+`createModel(weights, config, profile)` switches on `execution.graph`.
+`RuntimeModel` = `Gemma4Model | MiniCPM5Model | Qwen35Model | Qwen3Model |
+Qwen3MoeModel | DiffusionGemmaModel | Glm52Model | UniversalDenseModel`.
+
+**Support tiers in UX** (`supportTier(modelType, repoId)` →
+`"targeted" | "generic" | null`): consumed by `mlx-bun ls`
+(src/cli.ts), `src/hub-rest.ts`, and the serve discovery routes. Drafters
+(`*_assistant`) are never selectable on their own. Note the MiniCPM5
+*record*-level check keys on the repo id containing
+`minicpm5-1b-optiq-4bit`; the config-level check is the exact-shape predicate.
+
+Reference mirror of this section: docs/reference/models.md "Declared model
+profiles" (keep in sync).
+
+---
+
+## 4. Tier-0 design
+
+### 4.1 `UniversalDense` (`src/model/universal/`)
+
+One module (`dense.ts`), shaped like `qwen3.ts`, with every arch delta
+selected by a `UniversalArgs` descriptor. `archs.ts` holds an explicit
+table keyed by `model_type` (mlx-lm's `MODEL_REMAPPING` vendored in) — no
+config sniffing to *identify* an arch; fingerprints remain a generated-
+dispatch guard only. Descriptors in the table today (11):
+**llama, smollm3, qwen2, qwen3, gemma, gemma2, phi3, olmo2, glm4, granite,
+starcoder2**. `GENERIC_MODEL_TYPES` = the table's keys.
+
 Key choices:
 
-- **Descriptor keyed by `model_type` + mlx-lm's MODEL_REMAPPING table
-  vendored in.** No config sniffing/fingerprinting to *identify* an arch —
-  fingerprints stay what they are today: a dispatch guard for generated
-  specializations. Explicit table = explicit support surface = honest
-  `mlx-bun models` output.
-- **Transcribe mlx-lm op-for-op** inside the universal forward, per
-  descriptor branch — same discipline as every existing port (q/k-norm
-  before vs after transpose, rope order, cast points). The descriptor
-  selects branches; it never approximates.
-- **Rope factory** `src/model/universal/rope.ts` ports
-  `rope_utils.initialize_rope` verbatim: default/linear (native
-  `ops.rope` with scale), llama3 (precomputed freqs), yarn, longrope/su.
-  This is the single riskiest chunk of Phase 1 — rope bugs are
-  off-by-a-little and only caught by logit goldens, which is why the parity
-  gate feeds explicit ids and compares raw logits per step.
-- **KV cache**: reuse `KVCache`/`RotatingKVCache` unchanged;
-  `layer_types`/`sliding_window` already normalized. No kv-quant at Tier-0
-  (that's an L2/graduation concern).
-- **Quantization**: v1 requires MLX-quantized checkpoints (`.scales`
-  present), same as the rest of the runtime; `DenseLinear` for
-  fp16/bf16 checkpoints is Phase 1.5. Add the additive-bias term to both.
-- **RuntimeModel surface**: implement the same minimal contract
-  `Qwen3Model` does (config, weightsBytes, loraState, makeCache,
-  forwardHidden, logitsFromHidden, forward, generate, loraTargets) so
-  serving/eval/LoRA hot-swap work day one. `forwardEmbeddings` throws
-  (vision is per-arch work).
-
-### 3.2 Tier-0.5: `UniversalMoE`
+- **Transcribe mlx-lm op-for-op** inside each descriptor branch (q/k-norm
+  before vs after reshape, rope order, scale points, softcap, mask kinds).
+  The descriptor selects branches; it never approximates.
+- **Rope factory** (`rope.ts`) — faithful port of `rope_utils.initialize_rope`:
+  default / linear / llama3 / yarn / longrope (SuScaledRoPE) / proportional.
+  Frequency tables are computed with the same mlx float32 ops the oracle uses
+  at module init so the bytes match; host double math only where the oracle
+  uses python floats. Gated by `tests/universal-rope.test.ts` against
+  machine-keyed oracle fixtures (`tests/fixtures/universal-rope/`).
+- **Dense checkpoints** (`modules.ts`) — verbatim `mlx.nn.Linear`
+  (`addmm(bias, x, W.T)` / `x @ W.T`) and `nn.Embedding`; quantized paths
+  delegate to gemma4-base. Additive bias is a first-class field (qwen2 qkv,
+  starcoder2 all-bias).
+- **Weight audit at load** — `WeightAudit.finish` diffs consumed tensors
+  against the shard index minus the arch's sanitize-drop rules; unconsumed
+  or missing tensors are a LOAD ERROR naming them, never a silently-wrong
+  model.
+- **KV cache** — `KVCache`/`RotatingKVCache` unchanged; `layer_types` /
+  `sliding_window` already normalized. Standard attention consumes the shared
+  cache-attention interface; qualified Llama uniform KV4/KV8 serving and
+  generated-cache coverage are described in batching.md.
+- **RuntimeModel surface** — the minimal contract `Qwen3Model` implements
+  (config, weightsBytes, loraState, makeCache, forwardHidden,
+  logitsFromHidden, forward, generate, loraTargets) so serving/eval/LoRA
+  hot-swap work day one. `forwardEmbeddings` is a stub (vision is per-arch).
+
+### 4.2 The parity gate for a new generic arch
+
+`tests/universal-manifest.ts` — one entry per launch arch pointing at the
+SMALLEST mlx-community 4-bit checkpoint of that arch, skip-if-absent, with
+`expectClass: "universal" | "dedicated"` (the `qwen3` entry expects
+`Qwen3Model` — generic never shadows targeted, the gate still runs the
+oracle bar). Flow: user downloads → `bun scripts/regen.ts universal [<prefix>|all]`
+→ `MLX_BUN_TEST_UNIVERSAL=1 bun test tests/universal-parity.test.ts`.
+Asserts: fingerprint-free construction, weight audit clean, greedy
+trajectory match, per-step logits bit-exact.
+
+**Definition of supported-generic = manifest entry green on the current
+machine.** Adding a descriptor without its manifest goldens is the one
+anti-pattern this design forbids.
+
+### 4.3 `UniversalMoE` (design only)
 
 Same block with the MLP swapped for router + `QuantizedSwitchLinear`
-(already exists and is serving 26B-A4B). Deltas to parameterize: top-k,
+(already serving 26B-A4B and Qwen3-MoE). Deltas to parameterize: top-k,
 `norm_topk_probs`, routing score fn (softmax/sigmoid), routing scale,
 shared-expert branch (+ optional shared-expert gate — qwen2_moe),
 first-k-dense-replace (deepseek/glm4_moe layouts), grouped routing
 (deepseek_v3-style, defer). gpt_oss (attention sinks) and llama4 (chunked
-attention) are *not* claimed by the generic MoE — they graduate to
-one-offs if wanted.
-
-Memory reality check on the reference machines (24–32 GB): Mixtral-8x7B
-4-bit ≈ 25 GB — no. OLMoE-1B-7B, Qwen3-30B-A3B (≈17 GB @4bit),
-GLM-4 lite variants, granitemoe small — yes. The generic MoE tier is
-worth building, but expectations belong in `mlx-bun models` output
-(weightsBytes vs RAM), not in marketing.
-
-### 3.3 Dispatch changes (support.ts / factory.ts)
-
-Order matters — **generic never shadows a dedicated port**:
+attention) are not claimed by the generic MoE. RAM honesty belongs in the
+`ls`/fit output (weightsBytes vs RAM), not in marketing.
+
+### 4.4 Graduation to Tier-1
+
+Criteria (all three): (a) L1 generic parity green and stable across real
+use; (b) it is a daily driver / benchmark / product target; (c) a measured
+perf or memory gap vs a dedicated path worth the maintenance (number in
+docs/reference/benchmarks.md).
+
+Process, in order, stop at any step: (1) L2 — optiq goldens
+(`mixed-kv` pattern) if an optiq artifact/kv_config exists; (2) dedicated
+or generated forward — extend `gen-model.ts` if gemma4-base-shaped,
+otherwise a hand port (`qwen3_5.ts` pattern; hand ports are proven and
+cheap); (3) perf levers behind their existing flags (compiled decode,
+kv-quant, fused sdpa) — output-changing experiments are Lab items
+(unified-engine-frontier-plan.md §6); (4) benchmark + docs entries, tier
+label flips to `targeted`. Graduation never deletes the generic path — it
+stays the permanent correctness fallback, as the monolith does for generated
+gemma classes.
+
+### 4.5 Remaining phases
+
+| Phase | Scope | State |
+|---|---|---|
+| 1 | UniversalDense + descriptor table + rope factory + bias + manifest harness | built |
+| 1.5 | dense (bf16/fp16) checkpoints, no `.scales` | built (`modules.ts`) |
+| 2 | UniversalMoE: mixtral, qwen2_moe, olmoe, glm4_moe, dots1, granitemoe, ernie4_5_moe, phimoe (qwen3_moe is already a dedicated port) | design |
+| 3 | demand-driven one-offs (SSM/hybrid: lfm2, qwen3_next, mamba2, nemotron_h; MLA if a runnable target appears) — each a real port with its own goldens and cache types | design |
+| 4 | `mlx-bun port <hf-repo>`: download smallest snapshot → descriptor coverage check (config keys + weight-audit dry run) → auto-regen goldens vs the oracle venv → run the gate → print verdict + manifest entry. Optionally generalize `gen-model.ts` to emit from UniversalDense | design (no CLI command exists) |
 
-```
-createModel(weights, config):
-  1. dedicated classes (diffusion, minicpm5, qwen3_5, qwen3)      — unchanged
-  2. gemma4*: GENERATED.get(fingerprint) ?? Gemma4Model            — unchanged
-  3. ARCHS[remap(config.modelType)] → new UniversalDense/MoE(...)  — NEW
-  4. throw "unsupported model_type X (nearest: …)"                 — unchanged shape
-```
+---
 
-- `support.ts isSupportedModelRecord`: add `GENERIC_MODEL_TYPES` (the
-  descriptor table's keys, post-remap). Keep the drafter exclusion.
-  `mlx-bun models` labels the tier: `supported (targeted)` vs
-  `supported (generic)` — sets perf expectations and doubles as the
-  graduation worklist.
-- The current hard throw for non-MiniCPM5 `llama` configs is *replaced* by
-  the generic path (that throw exists only because nothing generic existed).
-  `isMiniCPM5Config`'s exact-shape predicate keeps MiniCPM5 on its
-  dedicated class ahead of the generic.
-
-### 3.4 Weight-name mapping + load audit
-
-Bucket A is already ~uniform in HF land
-(`model.layers.N.self_attn.{q,k,v,o}_proj`, `mlp.{gate,up,down}_proj`,
-`input_layernorm`, `post_attention_layernorm`, `model.norm`,
-`model.embed_tokens`, optional `lm_head`). The descriptor handles the
-exceptions: rename maps (starcoder2 `c_fc/c_proj`), fused-proj split rules
-(phi3 `qkv_proj`, `gate_up_proj` — split at load into views/copies),
-prefix strips (vision wrappers `language_model.model.*` — gemma4 already
-does this via `prefixBase`), and drop rules (`rotary_emb.inv_freq`,
-mlx-lm `sanitize`).
-
-**Every generic load runs a weight audit** (grow `scripts/inspect-model.ts`
-into `src/model/universal/audit.ts`): after module construction, diff
-{tensors in shard index} vs {tensors consumed}. Unconsumed or missing
-tensors ⇒ load error naming them, not a silent wrong-answer model. This is
-the cheap tripwire that catches descriptor mistakes before parity does
-(the "verify the foundation" rule, mechanized).
-
-### 3.5 The parity gate for a NEW arch (cost: ~2–4 h/arch, mostly waiting)
-
-Pattern already proven three times (gemma/minicpm5/qwen). Parameterize it
-once:
-
-1. `scripts/regen-generic-goldens.ts <snapshot> <prefix>` — oracle venv
-   (`/Users/joshrossi/Code/mlx-lm/.venv/bin/python`, mlx-lm `load()` +
-   `make_prompt_cache`) emits `<prefix>-parity.json` (explicit prompt ids,
-   greedy ids) + 12 step-logit `.bin`s. Same-machine rule applies (logit
-   goldens are GPU-specific — label goldens with host).
-2. `tests/generic-parity.test.ts` — iterates a manifest
-   (`tests/generic-manifest.ts`: prefix → snapshot path → model_type),
-   skipIf-gated on snapshot presence like every parity test. Asserts:
-   fingerprint-free construction, weight audit clean, greedy trajectory
-   match, per-step logits bit-exact.
-3. Checkpoint choice: **smallest mlx-community 4-bit of the arch**
-   (0.5B–3B exists for nearly all of bucket A) — keeps the gate runnable
-   and the download small. Downloads stay a user action (ground rules).
-
-Definition of supported-generic = manifest entry green on the current
-machine. No entry, no support claim.
-
-### 3.6 Graduation to Tier-1 (targeted)
-
-**Criteria** (all three): (a) L1 generic parity green and stable across a
-few sessions of real use; (b) it's a daily driver / benchmark / product
-target (e.g. the Lucien pipeline models); (c) measured perf or memory gap
-vs a dedicated path that's worth the maintenance (number in
-`benchmarks/RESULTS.md`, per the every-perf-claim-gets-a-number rule).
-
-**Process** (the gemma methodology, in order — stop at any step):
-1. **L2**: optiq-quant goldens (`regen-*-kv-goldens` pattern, mixed-KV vs
-   the optiq oracle) if an optiq checkpoint/kv_config exists for it.
-2. **Dedicated or generated forward**: if the arch is gemma4-base-shaped,
-   extend `gen-model.ts`; otherwise hand-port a dedicated module
-   (`qwen3_5.ts` pattern). Note: today's `gen-model.ts` subclasses
-   `Gemma4Model` specifically — generalizing it to emit from
-   `UniversalDense` is Phase 4 work, and *optional*: hand ports are proven
-   and cheap ("faithful port took ~1 hr, not the huge effort claimed").
-3. **Perf levers**: compiled decode / kv-quant / fused-sdpa, each behind its
-   existing flag. (The former "L3 kernels" step — fused decode / flash paths
-   with frozen-oracle gates — was deleted 2026-07-05; output-changing
-   experiments are now Lab items, see
-   [unified-engine-frontier-plan.md §6](unified-engine-frontier-plan.md).)
-4. RESULTS.md + docs entries; `mlx-bun models` label flips to `targeted`.
-
-Nothing about graduation removes the generic path — it stays as the
-always-correct fallback, exactly like the monolith does for generated
-gemma classes today.
+## 5. Multimodal seams shared by the ports
+
+Every tower (vision or audio) returns language-space soft tokens
+`[1, soft, textHidden]` pre-divided by `embed_scale` (the language model
+re-multiplies), splices into the prompt through
+`src/vision/prompt.ts buildMultimodalPrompt` (images + audio in document
+order; `buildVisionPrompt` is a byte-identical wrapper), and enters the model
+through `forwardEmbeddings` (merged embeddings; per-layer-input models take
+a `multimodal` zeroing mask decoupled from the `bidir` attention overlay).
+Media requests route to the **serial lane**, skip the prompt cache and spec
+decode, and capability flags (`visionCapable`, `audioCapable` in
+`src/registry.ts`) check TENSORS, not just config. Non-WAV audio transcodes
+via macOS `afconvert`; video decodes via the AVFoundation sidecar
+`mlx-bun-frame-extract` — zero vendored codecs.
 
 ---
 
-## 4. Phased rollout
+## 6. Ports
+
+One subsection per family. Each states the graph, its oracle, and the
+design-relevant facts; phase logs live in PLAN.md at the named anchor.
+
+### 6.1 Gemma-4 (e4b / 12B / 26B-A4B) — `src/model/gemma4.ts`, `gemma4-base.ts`, `generated/`
+
+- **Graph**: line-for-line port of mlx-lm `gemma4_text.py` covering 12B
+  (dense), e4b (per-layer-input embeddings + KV-shared layers), 26B-A4B
+  (MoE: router + gather_qmm experts). SDPA scale 1.0 (q/k normalized);
+  full-attention layers use global_head_dim 512, one global KV head,
+  `attention_k_eq_v`; ProportionalRoPE rotates only
+  `partial_rotary_factor` dims; python-float scalars promote weakly; mlx
+  helper implementations replicated exactly (`x**3` is `mx.power`).
+  Sliding layers use a plain cache + window masks (numerically identical to
+  mlx-lm's RotatingKVCache).
+- **Specialization**: `scripts/gen-model.ts` → `generated/gemma4-{e4b,12b,26b}.ts`,
+  fingerprint-dispatched, bit-exact vs the monolith
+  (`tests/generated-parity.test.ts`).
+- **Oracle**: L1 mlx-lm (`goldens/parity.json` on the 12B snapshot,
+  `tests/parity.test.ts`; 26B `tests/parity-26b.test.ts`); L2 mlx-optiq for
+  the mixed-KV composition (26B `logits-26b-kvmix.bin`, `mixed-kv` job).
+  Profile: `gemma4-generated` / `gemma4-dedicated`, L1.
+- **Vision**: two towers behind one merge contract — `src/vision/siglip.ts`
+  (full SigLIP encoder shipped by e2b/e4b/26B/31B; port of optiq's vendored
+  mlx-vlm gemma4 encoder from the bf16 `optiq_vision.safetensors` sidecar)
+  and `src/vision/embedder.ts` (12B `gemma4_unified` encoder-free patch
+  embedder). Oracle = optiq internals (`goldens/e4b-vision.json`,
+  `tests/e4b-vision.test.ts`, `tests/vision.test.ts`): spliced ids and
+  soft-token count EXACT, greedy PREFIX match; full bit-exact greedy is not
+  asserted because the 16-layer bf16 encoder composition drifts sub-bf16 and
+  the scale-1.0 softmax amplifies it (siglip.ts header).
+- **Audio**: §6.6.
+- PLAN anchors: Phase 12 (SigLIP), Phase 2 findings (parity notes).
+
+### 6.2 MiniCPM5-1B — `src/model/minicpm5.ts`
+
+- **Graph**: `model_type: llama` (plain Llama block — no MiniCPM residual
+  scaling), 24 layers, hidden 1536, 16/2 heads @128, vocab 130560, untied,
+  `rope_theta` from `ropeParameters.full_attention`. Port target mlx-lm
+  `models/llama.py`; the expert swiglu is mx.compile-compiled like the
+  oracle's `activations.py` (`compiledSwiglu`, kill-switch
+  `MLX_BUN_COMPILED_SWIGLU=0`) — the one fusion that matched the oracle's
+  kernel set and brought CPM5 decode to parity with mlx-lm
+  (decode-speed-program.md).
+- **Dispatch**: the exact-shape `isMiniCPM5Config` predicate keeps it on
+  the dedicated class ahead of the generic `llama` descriptor.
+- **Oracle**: L1 mlx-lm (`goldens/minicpm5-parity.json`,
+  `tests/minicpm5-parity.test.ts`); L2 mlx-optiq for per-layer quantized KV
+  (`goldens/minicpm5-kv-parity.json`, `tests/minicpm5-kv-parity.test.ts`,
+  bit-exact via `QuantizedKVCache` + `quantizedSdpaUnfused`). Also the
+  TurboQuant KV testbed (`TurboQuantKVCache`). Profile
+  `minicpm5-dedicated`, L1 (the L2 composition is a request-level KV scheme,
+  not a profile property).
+- Training reference model for segmented backward and ORPO
+  (docs/design/orpo-training.md, orpo-training.md).
+
+#### 6.2.1 MiniCPM5-2B investigation
+
+Metadata and template inspection completed 2026-09-10. No weights were
+downloaded and no GPU inference was run. This checkpoint is a candidate,
+not an artifact with established parity or local performance results.
+
+Start with the official [OpenBMB MLX artifact](https://huggingface.co/openbmb/MiniCPM5-2B-MLX),
+revision `3d00c3da500debfe345e60e25a87ed2669f5b1ee`. Its config uses standard
+Llama blocks, 42 layers, hidden size 2048, intermediate size 6144, 16 query
+and 2 KV heads of dimension 128, untied embeddings, vocab 130560 and
+RoPE theta 5000000. Quantization is affine 4-bit with group size 64.
+Preserve both EOS IDs, `1` and `130073`, and the separate
+`chat_template.jinja`. The release is text-only and Apache-2.0.
+
+Current integration findings:
+
+- `isMiniCPM5Config` in `src/model/support.ts` matches the older 24-layer,
+  hidden-1536 artifact. The 2B config fails that predicate and qualifies for
+  generic Llama through `genericArgsFor`; `familyProfile` therefore selects
+  the universal graph. Do not widen the predicate before auditing dedicated
+  execution, training and cache assumptions against this shape.
+- A CPU-only smoke using the pinned config and template passed
+  `loadModelConfig`, `genericArgsFor`, `ChatTemplate.load` and rendering a
+  user → assistant tool call → tool result → user conversation. Explicit
+  thinking true/false both render correctly; the XML function call survives.
+  This checks rendering, not generated tool-call parsing or tokenizer parity.
+- `resolveEnableThinking` applies the MiniCPM no-thinking default through
+  the same exact-shape predicate. It does not apply to 2B. With no explicit
+  option, this artifact's prompt ends at the assistant header; true appends
+  an open think tag, false appends a closed empty think block. Set this
+  explicitly in comparisons. Template support for disabling thinking does
+  not establish no-thinking quality.
+
+Evaluation sequence:
+
+1. Pin an 8-bit or BF16 control alongside the official 4-bit candidate.
+   Establish tokenizer/prompt identity and same-machine logits against the
+   pinned mlx-lm oracle, then test multi-turn tool calls, reasoning splits,
+   EOS, cache reuse and batched continuation. Keep quantization quality
+   comparisons separate from same-artifact runtime parity.
+2. Compare with MiniCPM5-1B and Gemma e4b on held-out Dreaming segmentation,
+   extraction and source-grounded synthesis, plus bounded repository/tool
+   tasks. Score boundary/label accuracy and completed tasks, not just valid
+   JSON or XML. Use explicit thinking modes and equal token/time budgets.
+3. Record first-answer latency, complete task time, output/reasoning tokens,
+   repetition and truncation rates, memory and context scaling. Local
+   measurements belong only in `docs/reference/benchmarks.md`, with a named
+   quiet machine and the required serving benchmark procedure.
+4. Investigate the [released DSpark draft](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark)
+   after ordinary inference passes. Its config declares `Qwen3DSparkModel`,
+   five draft layers, block size seven, and target taps `[1,10,20,30,39]`.
+   Existing Gemma DSpark support does not establish compatibility. Audit
+   the draft's architecture, target taps, weights and verifier before a
+   paired task-time comparison.
+
+Evidence to use when designing the quality checks:
+[OpenBMB's model card](https://huggingface.co/openbmb/MiniCPM5-2B) distinguishes
+internally reproduced scores from Artificial Analysis results. The
+[independent September 7 evaluation](https://artificialanalysis.ai/articles/openbmb-releases-minicpm5-2b)
+supports strong capability for its size but reports substantial reasoning
+token use and weak absolute terminal-task success. These evaluations do not
+validate MLX 4-bit quality. [Upstream issue 374](https://github.com/OpenBMB/MiniCPM/issues/374)
+reports repetition sensitivity in GGUF coding runs. That is an unreproduced
+cross-runtime hypothesis for our sweep, not evidence of the same MLX failure
+or a reason to change repetition defaults.
+
+### 6.3 Qwen3 / Qwen3-MoE / Qwen3.5-family (3.6, 3.8) — `qwen3.ts`, `qwen3-moe.ts`, `qwen3_5.ts`
+
+- **Qwen3** (`model_type: qwen3`): dense GQA with per-head q/k RMSNorm
+  before RoPE, theta 1e6, tied embeddings. Primary use is the
+  text-embedding backbone (`embedPooled`: last-token hidden, L2-normalized).
+  Oracle L1 mlx-lm (`tests/qwen3-embed-parity.test.ts`). Shadows the
+  generic `qwen3` descriptor.
+- **Qwen3-MoE** (`qwen3_moe`): straight op-for-op transcription of
+  mlx-lm `qwen3_moe.py` + `switch_layers.py` — gate Linear → precise
+  softmax over all experts → argpartition top-k → renormalize → SwitchGLU
+  with compiled swiglu; `decoder_sparse_step` / `mlp_only_layers` dense
+  layers handled. Oracle L1 mlx-lm on the 30B-A3B
+  (`tests/qwen3-moe-parity.test.ts`; the cached checkpoint has no
+  kv_config, so no L2 bar).
+- **Qwen3.5 hybrid** (`qwen3_5` / `qwen3_5_text`; the `qwen3_5_moe` variant
+  is excluded by `isQwen35Config`): 64 layers, every
+  `full_attention_interval`-th (4) a gated softmax-attention layer (q-gate
+  split, per-head q/k norm, partial RoPE 0.25 @ base 1e7, GQA 24/4 @256,
+  `o_proj(out·σ(gate))`), the rest **gated DeltaNet** linear attention
+  (`in_proj_qkv/z/b/a`, depthwise causal conv1d k=4, `A_log`/`dt_bias`
+  float, `RMSNormGated`). The recurrence is the reference's custom Metal
+  kernel `gated_delta_step` ported VERBATIM (`src/model/qwen3-delta.ts`) —
+  bit-exact parity requires it; the ops fallback reduces in a different
+  order. `SSMCache` (conv + recurrent state) implements `Cache` and the
+  spec-round contract (`specRoundBegin/Commit/Rollback`: free snapshot,
+  replay of the kept window on partial reject — bit-exact by construction
+  because the kernel's per-thread loop is serial). Weights under
+  `language_model.`; `output_gate_type` other than `"swish"`/absent throws.
+  **Qwen3.8-27B is byte-for-byte the 3.6-27B text geometry** — zero new
+  graph work; the exact profile `qwen3.8-27b-optiq-4bit` pins the validated
+  revision.
+- **Oracles**: L1 mlx-lm `qwen3_5.py` with bf16 KV
+  (`goldens/qwen38-parity.json`, `tests/qwen-parity.test.ts`; earlier 4B/3.6
+  goldens via `regen.ts qwen-parity`); L2 mlx-optiq mixed-KV on the 3.5/3.6
+  OptiQ artifacts (`install_mixed_kv` per-layer bits keyed by cache index ≡
+  our `maybeQuantizeKv`). Qwen3.8's OptiQ repo ships no mixed-KV config →
+  no L2 bar; mixed-KV serving there is Lab-tier.
+- **Vision + video** (`src/vision/qwen3vl-{preprocess,tower,prompt}.ts`,
+  `video-frames.ts`, `src/model/qwen3-mrope.ts`): mlx-vlm `qwen3_vl`
+  vision tower at the qwen3_5 configuration (depth 27, hidden 1152, heads
+  16 @72, patch 16, temporal 2, merge 2, deepstack empty) from the bf16
+  sidecar `optiq/optiq_vision.safetensors`; PIL-exact fixed-point bicubic
+  smart_resize; conv3d patch embed (new `mlx_conv3d` binding);
+  `ensure_fused_sdpa`'s 72→80 head-dim pad replicated; **interleaved
+  mRoPE** via the reference's verbatim Metal apply kernel with
+  `get_rope_index` positions + delta exact, installed per request on the
+  serial lane (text-only keeps the untouched fast-rope path — equal
+  position streams degenerate mRoPE to plain RoPE). Video: AVFoundation
+  sidecar samples at 2 fps (max 768 frames), temporal-pair patchify, gridT>1
+  tower, transformers' `<t seconds>` per-frame-group prompt format with a
+  python-`%.1f` port. Oracle = **mlx-vlm 0.6.14 pinned to mlx 0.31.2**
+  (independent of mlx-lm, so no composite bit-match target exists): tower
+  bit-exact at small grids, calibrated envelope at 2304 patches; e2e greedy
+  token-exact on 2/3 image fixtures, step-0 argmax exact on all
+  (`tests/qwen38-vision*.test.ts`).
+- **Native MTP** (`src/spec/qwen-mtp-source.ts`, `--draft-kind mtp`): the
+  separately published `Qwen3.8-27B-MTP-*` head (`fc` over
+  concat(rms(embed(next)), rms(target_hidden)) → one full-attention layer
+  with its own KVCache → target lm_head, block_size 3, DeepSeek-V3 shape)
+  reads the target's PRE-final-norm hidden via `captureLayer`. Gate:
+  greedy speculation token-identical to non-spec greedy
+  (`tests/qwen38-mtp.test.ts`). Durable negative perf verdict on the quiet
+  M1 Max (MTP-on slower than off at γ=2 despite 61% acceptance) — stays
+  opt-in. Drafter logits verified against the mlx-vlm reference fed the
+  oracle's captured hidden grid.
+- Training: `loraTargets()` wired; the DeltaNet kernel is inference-only —
+  training needs mlx-lm's pure-mlx `gated_delta_ops` scan under training
+  (open).
+- PLAN anchors: Phase 14 (bring-up, 14g MTP, 14v vision, 14w video, 14y
+  YaRN open, 14z TurboQuant weights).
+
+### 6.4 GLM-5.2 (`glm_moe_dsa`) — the Colibri hierarchy, `src/model/glm52*.ts`
+
+Summary of docs/archive/investigations/colibri-glm52-port.md (frozen);
+full phase record G0–G8 in git (`git show 3199c75:PLAN-archive.md`,
+Phase 21, closed 2026-08-17). GLM-5.3+ would extend THIS section.
+
+- **Decision**: a native `Glm52Model` over the PUBLIC Colibri container in
+  place (no second copy, no Colibri child process, no `--backend`); the
+  pinned Colibri build (`JustVugg/colibri` @ `44e489b`) is an external
+  executable oracle for formats, logits/tokens, routing traces, memory and
+  speed. Its policy code is studied and reproduced, not shipped.
+- **Model**: 78 main layers (3 dense FFN + 75 sparse MoE, 256 routed
+  experts, top-8) + an MTP row with its own 256 experts (19,456 expert
+  instances). The dense spine (embeddings, lm-head, attention, norms, first
+  dense FFNs, routers, shared experts) stays resident at int4 (~9.9 GB); the
+  ~370 GB routed-expert pool stays on disk.
+- **Graph** (`glm52.ts`, `glm52-mla.ts`, `glm52-dsa.ts`, `glm52-moe.ts`):
+  MLA — q-a → norm → q-b; kv-a → latent norm; **compressed cache** of the
+  512-wide latent + 64 RoPE values per token (`MLACache`, ~57× smaller than
+  reconstructed K/V at this geometry), weight-absorption decode,
+  reconstructed-prefill fallback, partial interleaved RoPE. **DSA indexer**
+  — q/k/projection/norm, per-layer full/shared schedule, exact dense
+  fallback when the selected set covers the prefix (`DSAIndexCache` only on
+  layers that own it). MoE — sigmoid scores + correction bias for
+  selection, un-biased gate weights, true top-8, routed scale, shared expert
+  in parallel. Colibri arithmetic (offset-binary int4, per-row int8 scale
+  applied once, RMSNorm with multiplicative weight and double-accumulated
+  reduction) is mirrored in `glm52-reference.ts`. G2 favors an auditable
+  dequantize→f32 path; the Metal streamed kernel (`glm52-streamed-metal.ts`)
+  replaces the expert resolver without changing the graph.
+- **Expert residency / streaming** (`glm52-residency.ts`,
+  `glm52-streamed-experts.ts`, `glm52-expert-layout.ts`, `glm52-pilot.ts`,
+  `glm52-coupling.ts`, `glm52-atlas.ts`): per-layer bounded LRU slots
+  derived from the whole-process memory equation, pinned hot-store seeded
+  from persistent usage, LFRU repinning with hysteresis, batch-union of
+  unique (layer, expert) across rows so each expert is read once per
+  layer/step, `pread` worker pool with slot generations and completion
+  fences (no reuse while a lazy MLX command can retain the buffer), PILOT
+  (next-layer router on the current state) and COUPLE (cross-layer route
+  pairs) prefetch, RSS feedback shrinking only the evictable tier. Unified
+  memory removes a copy, not the 370 GB: the meaningful states are disk /
+  resident / pinned / loading / in-flight / evictable.
+- **Fixed-context admission** (`glm52-memory.ts`, `factory.ts
+  openGlm52RuntimeModel`): a header-only resource plan runs BEFORE any
+  resident tensor or expert slab is opened — process limit (default 25 GiB
+  preset or physical RAM, whichever is smaller), fixed context
+  (default 4096 tokens, 128 generation), allocator/Bun/safety reserves — and
+  refuses an impossible placement; the opened runtime's planned bytes must
+  equal the preflight equation or load fails. Serving clamps `max_tokens`
+  to the room under the fixed context (STATUS: "GLM fixed-context admission
+  correction", generalized to every model in v0.2.0).
+- **Native MTP** (`src/spec/glm52-mtp-source.ts`): the checkpoint's int8 MTP
+  row as an in-process draft source sharing the target; serial only —
+  batched rows decode single tokens (Colibri keeps MTP out of its mux path
+  too). Batched caches land via `makeEmptyBatch/mergeRows/extractRow/
+  filterRows/projectedBytes` on right-justified latent/RoPE/DSA tensors —
+  no full K/V at merge, admission, extraction or cancellation.
+- **Oracle / fidelity**: profile `glm5.2-colibri` / exact
+  `glm5.2-colibri-int4-int8-mtp`, **L3 (measured)** — Colibri is neither
+  mlx-lm nor mlx-optiq, so the tier system cannot call it bit-exact even
+  though the G2 gate is a teacher-forced trajectory from the pinned engine
+  on a tiny converted artifact (`tests/glm52-teacher-forcing.test.ts`,
+  `fixtures/colibri-glm52/`). Curated numbers live in code as operator
+  guidance (`GLM52_G5_MEASURED_WARM_DECODE_TPS`,
+  `GLM52_G5_DIRECT_ORACLE_WARM_DECODE_TPS`, M1 Max 32 GB, MTP on) and in
+  docs/reference/models.md "GLM-5.2 on a 32 GB Mac". The 24 GB M4 Pro is
+  below the one-slot-per-layer floor and out of scope.
+- **Not claimed**: embeddings, image/audio input, LoRA, training, adapters
+  — advertised false, never faked (archive N8).
+- PLAN anchor: Phase 21 (closed 2026-08-17); probes, gate tests, and
+  lab spikes for the port surface retired afterwards (commit fbafe44).
+
+### 6.5 DiffusionGemma-26B-A4B — `src/model/diffusion-gemma.ts`, `src/diffusion/`, `src/vision/diffusion-vision.ts`
+
+Summary of docs/design/generic-model-support.md (the D0 reference dossier
+there stays the porting contract; do not extend it).
+
+- **What it is**: the first non-autoregressive model — an encoder prefill
+  over the prompt builds a KV cache, then a decoder pass over a fixed
+  256-token canvas (initialized to uniform-random ids, NOT a mask token) is
+  denoised over up to 48 steps. 30 layers, hidden 2816, 16 heads; each
+  layer runs a parallel dense MLP + 128-expert top-8 MoE summed through 7
+  RMSNorms and a `layer_scalar`; attention scale 1.0, no attention softcap
+  (final-logit softcap 30 in fp32), QK/V-norm post-proj/pre-RoPE, full
+  layers (5,11,17,23,29) reuse k as v at hd 512 / kv 2 with partial rotary
+  0.25; tied 4-bit head via `QuantizedEmbedding.asLinear`; a
+  `SelfConditioning` MLP feeds soft embeddings from the previous step.
+  Bidirectional decoder masks (`_make_decoder_masks`) are the crux.
+- **Engine** (`diffusion-generate.ts`): linear temperature schedule
+  (`t_min 0.4 → t_max 0.8`, DIVIDES logits), both samplers
+  (confidence-threshold = optiq public default, entropy-bound = engine
+  default), self-conditioning feedback, stability/EOS, block loop. RNG
+  parity via bound `mlx_random_randint`/`mlx_random_seed` threading the
+  global key in the reference's draw order. Parity traps recorded in the
+  source doc: division not reciprocal-multiply; independent history copies;
+  the as-loaded oracle has `generation_config = None` (stable-stop OFF,
+  eos {1, 106}).
+- **Vision**: a DEDICATED tower (`diffusion-vision.ts`), not the e4b SigLIP
+  — inline-quantized `input_proj` truncates `2*(x-0.5)` patches to integers
+  (a trained-in quirk), head_dim 72 padded to 80 like the reference, plain
+  bf16 `down_proj`.
+- **Oracle**: **mlx-optiq IS the oracle** (stock mlx-lm cannot load it);
+  the mixed-precision quant is the native weight format, so matching optiq
+  = L2 = the floor. Profile `diffusion-gemma-dedicated`, L2. Gates:
+  single forward bit-exact (`tests/diffusion-parity.test.ts`), generation
+  token-for-token for both samplers (`diffusion-gen-parity`), image
+  token-for-token (`diffusion-vision`), serving (`diffusion-serving`), LoRA
+  with the denoising objective (`diffusion-lora`,
+  `src/train/diffusion-lora.ts`; MoE backward by stop-gradient on router
+  indices). All `MLX_BUN_TEST_DIFFUSION=1`.
+- **Entry points — honest state**: `src/generate.ts` routes by
+  `instanceof DiffusionGemmaModel` to `generateDiffusionInner`, and
+  `src/server.ts` has a diffusion vision lane; **`src/cli.ts` contains no
+  diffusion reference** (`grep -c diffusion src/cli.ts` = 0). There is no
+  diffusion-specific CLI surface; the model is reachable only through the
+  generic `serve` / `generate` path by profile + instanceof dispatch, and
+  streaming yields tokens after the engine completes. D4 (performance,
+  measured) is open.
+
+### 6.6 Audio input (gemma-4 audio tower) — `src/audio/`
+
+Summary of docs/design/generic-model-support.md. Scope: audio-in, text-out
+through the chat API; TTS/STT endpoints, streaming audio, >30 s, batched
+audio prefill, and 26B-A4B/DiffusionGemma audio (no `audio_config`) are
+non-goals.
+
+- **Oracle**: mlx-lm strips `audio_tower`/`embed_audio` at load and rejects
+  non-text content; optiq's internal gemma4 machinery is complete but its
+  serve frontend never exposes audio → the oracle is the optiq internal
+  model driven directly (`scripts/oracle/gen-e4b-audio-golden.py`,
+  `goldens/e4b-audio.json`). The e4b OptiQ-4bit sidecar already holds all
+  752 audio tensors (bf16, incl. clipped-linear min/max stats) plus
+  `audio_config` and token ids (`boa 256000`, `audio 258881`,
+  `eoa 258883`) — no download.
+- **Pipeline**: `decode.ts` (RIFF/WAVE PCM16/24/32/f32, mean mixdown, linear
+  resample to 16 kHz, PCM16/32768 like the oracle; >30 s truncates at
+  480 000 samples BEFORE features, the oracle extractor's default) →
+  `transcode.ts` (`afconvert` for non-WAV, content-sniffed) →
+  `features.ts` (USM log-mel, frame 320 / hop 160 / rfft 512 in f64, 128 HTK
+  bins, `log(x+1e-3)`, the oracle's exact f32 Hann values baked in because
+  numpy's vectorized f32 cos differs by 1 ulp and log-amplifies) →
+  `conformer.ts` (SSCP two Conv2d stride-2 stages (128, 32) → Linear 1024 →
+  12 Conformer blocks with chunked local attention (chunk 12, left 13,
+  softcap 50), depthwise Conv1d k=5, clipped linears → output_proj 1536 →
+  `embed_audio` RMSNormNoScale → Linear → /embed_scale). **Dtype contract
+  differs from siglip**: the oracle feeds f32 mel into bf16 weights and mlx
+  promotes, so activations stay f32 end-to-end — do not cast to weight
+  dtype. Clipped-linear stats are load-bearing (off diverges 90%).
+- **LM semantics (resolved against the oracle)**: audio tokens are strictly
+  CAUSAL; any audio in the prompt disables the vision bidir overlay
+  entirely; per-layer-input zeroing uses the union multimodal mask; USM
+  parameters are fixed constructor defaults (no per-bin normalization);
+  `<|audio|>` → `boa + audio×n + eoa` with `n = min(ceil(ms/40), 750)` is
+  splice-side. The merge divides by embed_scale AFTER the bf16 cast (weak
+  scalar) — `features(preDivide=false)` + astype + div mirrors it.
+- **Serve**: OpenAI `input_audio` plus optiq `audio`/`audio_url` parts;
+  lazy `getAudioTower` from the same sidecar, zero new flags; a request
+  with audio on a towerless model is an explicit 400; Anthropic
+  `/v1/messages` rejects audio blocks with a pointer to the OpenAI
+  endpoint; `audio` capability flag on `/v1/models`, `/library`, and the pi
+  `ready` frame (requires `audio_config` AND tower tensors — the 12B stub
+  sidecar reads false).
+- **Gates**: T0 mel model-free (`tests/audio-features.test.ts`, 1e-5),
+  T1 tower rel-RMSE (`e4b-audio-tower`, bit-exact result), T2 e2e greedy +
+  decoded text EXACT for both fixtures (`e4b-audio`), T3 HTTP
+  (`e4b-audio-serve`, `MLX_BUN_TEST_AUDIO_SERVE=1`).
+- **Open (A5)**: bench cells (tower ms, TTFT delta, RSS delta) into
+  docs/reference/benchmarks.md; 12B audio cell needs a sidecar rebuild via
+  optiq `build_vision_sidecar` (local 12B sidecar holds 1 audio tensor);
+  features-matrix note audio×batching = serial.
+- PLAN anchor: "Phase: audio input — gemma-4 audio tower, e4b first".
 
-| Phase | Scope | Unlocks | Effort | Key risks |
-|---|---|---|---|---|
-| **1** | `UniversalDense` + descriptor table + rope factory (llama3/yarn/longrope) + bias term + goldens harness. Launch archs: **llama (⇒ Llama 3.x + Mistral + Ministral), qwen2, qwen3(fold-in), gemma, gemma2, smollm3, olmo2, olmo3, phi3, glm4, granite, starcoder2** | ~20 files at launch; the rest of bucket A (~36 total) as S-sized descriptor+golden follow-ons; F-wrappers for covered bases | **L** (module M, rope M, harness S, 12×goldens S-each) | rope variants (llama3/yarn/longrope math), gemma2 softcap manual-attn path, phi3 fused-proj split, BOS/tokenizer conventions per family (the double-BOS class of bug), qwen2 qkv bias |
-| **1.5** | `DenseLinear` (bf16/fp16 checkpoints, no `.scales`) | fp16-only small models; removes the "MLX-quantized only" constraint | **S–M** | dtype/cast points vs mlx-lm (bf16 matmul accumulation — see e4b length-sensitivity memory) |
-| **2** | `UniversalMoE` (router + SwitchGLU + shared experts + first-k-dense) : **mixtral, qwen2_moe, qwen3_moe, olmoe, glm4_moe, dots1, granitemoe, ernie4_5_moe, phimoe** | ~12–15 of bucket B (gpt_oss/llama4 excluded) | **M** (SwitchGLU exists; routing variants are config) | routing-score parity (softmax vs sigmoid + norm_topk), RAM honesty on 24 GB |
-| **3** | Demand-driven one-offs, qwen3_5 pattern: SSM/hybrid (**lfm2, qwen3_next, mamba2, nemotron_h**…), MLA module if a runnable target appears (**minicpm3**; deepseek-lineage mostly exceeds local RAM) | bucket C/D picks, ~1 model per effort unit | **M–L each** (cumulatively XL; never a blocker for A/B) | new cache types (conv/ssm state) touch the Cache interface + serving slots; each is a real port with its own goldens |
-| **4** | Tooling: **`mlx-bun port <hf-repo>`** — download smallest snapshot → descriptor coverage check (config keys + weight-audit dry run) → auto-regen goldens vs oracle venv → run parity gate → print verdict + manifest entry. Optionally: generalize `gen-model.ts` to emit from UniversalDense for graduation | turns "new arch" into a ~1-command, ~1-hour operation; makes the coverage claim self-verifying | **M** (scaffold) + **L** (generator generalization, optional) | oracle-venv coupling (mlx-lm version drift changes goldens — pin + record versions in parity.json, already the pattern) |
-
-Sequencing note: Phase 1's descriptor table should be *born* with only the
-12 launch archs actually gated green — adding an arch without its manifest
-goldens is the one anti-pattern this design forbids.
+---
 
-## 5. Top design decisions (summary)
+## 7. Top design decisions
 
 1. **Explicit descriptor table keyed by model_type (+ vendored
    MODEL_REMAPPING), not config sniffing** — support is a declared,
    testable surface; fingerprints remain a generated-dispatch guard only.
-2. **One normalized config** — extend the existing `TextConfig` (it's
-   already 70% there) with the bucket-A delta union; move per-arch quirks
-   out of `loadModelConfig`'s if-chain into descriptor `configAliases`.
+2. **One profile per load, resolved before weights open** — exact artifact
+   > family > generic; mismatches refuse, never downgrade; profiles declare
+   construction only.
 3. **L1 bit-exact is the gate for generic too** — manifest-driven golden
-   harness (explicit ids, raw per-step logits, same-GPU), one regen script
-   + one test file for all generic archs.
-4. **Weight audit at load** — unconsumed/missing tensors are a load error,
-   never a silently-wrong model.
-5. **Generic never shadows targeted; graduation never deletes generic** —
-   the dispatch ladder is dedicated → generated → generic → reject, and the
-   generic path is the permanent correctness fallback.
-6. **Honest tiering in UX** — `mlx-bun models` shows `targeted` vs
-   `generic` (and RAM fit), which doubles as the graduation worklist.
+   harness (explicit ids, raw per-step logits, same GPU).
+4. **Weight audit at load** — unconsumed/missing tensors are a load error.
+5. **Generic never shadows targeted; graduation never deletes generic.**
+6. **The oracle for any capability is whoever ships it** — mlx-lm (L1),
+   mlx-optiq (L2), mlx-vlm / optiq internals / Colibri where those are the
+   only implementations — and the profile's fidelity label says which.
+7. **Honest tiering in UX** — `mlx-bun ls`, hub REST, and discovery routes
+   show `targeted` vs `generic`; media capability flags check tensors.
+8. **New ports are sections of this doc**, with their durable phase record
+   in PLAN.md.
+
+## History
+
+- 2026-06-24 — MiniCPM5 decode megakernel shelved (persistent single-dispatch
+  decode kernel, ~0.94× baseline at best; L2 ceiling 93/100 from quant
+  discontinuity amplifying 1-ULP GEMV drift). Premise removed by the
+  bandwidth-wall finding (decode-speed-program.md: the baseline decode path
+  sits at the memory-bandwidth wall; the real CPM5 waste was the unfused
+  swiglu, fixed by `compiledSwiglu`). Code shelved to scripts/experiments
+  2026-07-01 and deleted with that directory 2026-08-23 (git history); no
+  megakernel code remains in `src/`. The doc
+  `docs/design/generic-model-support.md` is superseded by this line.
+- 2026-06-24 — DiffusionGemma D0–D3 + D5 landed (bit-exact forward,
+  token-for-token generation, image, LoRA); D4 open.
+- 2026-06-15 → 2026-08-18 — Qwen3.5-family: 3.6 text parity both bars;
+  3.8 retarget (identical geometry), native MTP, vision, video landed.
+- 2026-07-01 — Generic Phase 1 + 1.5 built (commit 9bd9f1b):
+  UniversalDense, 11 descriptors, rope factory, dense checkpoints, weight
+  audit, manifest harness.
+- 2026-07-07 — Audio A0–A4 landed (e4b, oracle-exact); A5 open.
+- 2026-08-17 — GLM-5.2 Colibri port closed (Phase 21); surface retired,
+  design archived.
+- 2026-08-21 — Model profiles (`src/model/profile.ts`) introduced; exact
+  artifact declarations for Qwen3.8 OptiQ and GLM-5.2 Colibri.
+- 2026-08-23 — This doc consolidated: audio-input-plan,
+  diffusion-gemma-port, minicpm5-decode-megakernel, and the Colibri archive
+  summarized here; ports become sections.
+
+
+Non-Qwen grouped TurboQuant drafting, including assistant donor attention, uses
+the shared cache interfaces. See [composition coverage](batching.md#turboquant-donor-attention-and-grouped-draft-composition).