mlx-bun 0.2.0 → 0.4.0

package/docs/reference/memory.md

@@ -1,97 +1,260 @@
-# Memory
+# Memory — the personal wiki
 
-mlx-bun memory is a local Markdown wiki that the built-in pi agents can read as durable user context.
+`mlx-bun memory` is a local, durable memory for your assistant: a wiki of
+Markdown articles at `~/.mlx-bun/wiki/` that the built-in pi agent reads to
+remember your projects, people, decisions, and history across sessions. It is
+yours — git-tracked, editable in any tool (Obsidian opens it as a vault), and it
+never leaves the machine.
 
-Core rule:
+The one rule the whole feature is built around:
 
-> Chat agents read memory articles. `mlx-bun memory synthesize` is the only writer.
+> Chat-time agents **read** memory. `mlx-bun memory synthesize` is the only
+> thing that **writes** it.
 
-The current release includes the vault, read path, Obsidian open flow, scheduler, and a live synthesis engine: `mlx-bun memory synthesize` runs the full local DAG (and the nightly job runs it on a schedule). Synthesis uses the same local models you chat with, switched into an editorial pipeline; stage-specific LoRA adapters are the hardening roadmap below.
+Articles are durable priors; conversations are evidence; synthesis is the
+deliberate, auditable step that turns evidence into updated articles. Normal
+chat never mutates an article. Nightly synthesis is called **the Dreaming**;
+its engineering design lives in
+[dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — this
+page is the user-facing reference.
 
-## Storage
+## Quick start
 
-- Vault: `~/.mlx-bun/wiki/`
-- User articles: `~/.mlx-bun/wiki/articles/*.md`
-- Read-only reference docs: `~/.mlx-bun/wiki/Reference/*.md`
-- Meta pages: `~/.mlx-bun/wiki/Meta/*.md`
-- Talk pages: `~/.mlx-bun/wiki/Talk/`
-- Override: `MLX_BUN_WIKI=/path/to/wiki`
-- Derived synthesis DB: `~/.cache/mlx-bun/memory.sqlite`
+```bash
+mlx-bun memory init            # create the wiki; offers import + nightly job
+mlx-bun pi                     # the assistant now reads your memory automatically
+mlx-bun memory status          # where it is, how many articles, schedule state
+mlx-bun memory search <query>  # find something from the terminal
+mlx-bun memory open            # browse it in Obsidian
+```
+
+`init` is idempotent and safe to re-run. It never overwrites files you have
+edited: the README and `Meta/` pages are written only when missing. Once a
+wiki exists, every `mlx-bun pi` session and the web chat behind `mlx-bun serve`
+pick it up with no further configuration; if it does not exist, no memory
+tools or prompt hints are exposed at all.
+
+`mlx-bun setup` is a true alias for `mlx-bun memory` (same subcommands).
+
+## What the wiki contains
+
+Everything lives under one portable folder (copy it and your memory moves
+with you):
 
-The vault is plain Markdown, git-tracked, and opens directly in Obsidian. `Reference/` contains read-only symlinks to mlx-bun's own docs so memory has useful built-in residents on day one; synthesis must never write them.
+| Path | What it is |
+| --- | --- |
+| `~/.mlx-bun/wiki/` | The vault root. Override with `MLX_BUN_WIKI=/path/to/wiki`. |
+| `articles/*.md` | Your memory articles — one topic per file, `Topic_Name.md`. The only thing synthesis writes. |
+| `Reference/*.md` | Read-only symlinks to mlx-bun's own docs (README, server API/config, library API, embedding, distribution, training, ORPO quickstart, product roadmap), so the assistant can answer mlx-bun questions on day one. Synthesis never writes here. |
+| `Meta/*.md` | Operational pages the pipeline follows: editorial guidelines, article conventions, infobox schemas, category definitions, entities policy, summary style, bucketing/buckets, chunking, topics to ignore. Edit them to steer synthesis. |
+| `Talk/` | Per-article discussion and conflict notes. |
+| `README.md`, `.gitignore` | Orientation page; keeps `.DS_Store` and Obsidian workspace state out of git. |
+
+The vault is plain Markdown and a git repository (`init` runs `git init` and
+makes the initial commit). Every synthesis run commits its changes, so any
+article change is reviewable and revertible with ordinary git. You can edit
+any file directly; the next run respects your edits.
+
+Two things live **outside** the vault because they are derived or operational:
+
+- `~/.cache/mlx-bun/memory.sqlite` — the synthesis store (ingested
+  conversations, chunks, buckets, watermarks, and a rebuildable index of the
+  articles). The vault is truth; this DB is a cache plus pipeline ledger.
+- `~/.mlx-bun/logs/memory-synthesis.{out,err}.log` — output of the nightly job.
+
+Article names are **stems**: the filename without `.md`, underscores for
+spaces (`Archie_Project`). Every command and tool below takes a stem.
 
 ## CLI
 
+All subcommands are `mlx-bun memory <sub>`; with no subcommand, `status` runs.
+An unknown subcommand prints help and exits non-zero (so scripts and launchd
+jobs can detect typos).
+
+### Set up and inspect
+
 ```bash
-mlx-bun memory init              # create the wiki; optional import + schedule
-mlx-bun memory status            # path, article count, git, schedule, recent articles
-mlx-bun memory open              # open wiki in Obsidian/Finder
-mlx-bun memory open <article>    # open a specific article
-mlx-bun memory list              # list article stems + Reference/* docs
-mlx-bun memory search <query>    # search articles
-mlx-bun memory toc <article>     # headings + anchors
-mlx-bun memory section <article> <anchor>
-mlx-bun memory links <article>   # outbound/inbound wikilinks
-mlx-bun memory read <article>    # print article body
-mlx-bun memory synthesize --dry-run
-mlx-bun memory schedule --at 03:00
-mlx-bun memory unschedule
+mlx-bun memory init              # create the wiki (alias: setup)
+mlx-bun memory status            # path, article + reference counts, git, nightly state, recent articles
+mlx-bun memory open [article]    # open the wiki or one article (alias: browse)
 ```
 
-**`memory synthesize` writes to your vault.** It runs the full local synthesis DAG (segment → extract → route → create/patch → reconcile → link → wikify) and creates/patches articles under `articles/` (git-committed, so every change is reviewable and revertible). `--dry-run` is the safe, no-write mode: it reports what would happen without touching the vault. `--since` and `--model` scope/steer a run. The decomposed stage workers (`segment`/`extract`/`route`/`synthesize-stage`/`link`) run one stage each and are resumable.
+`init` walks through setup in a terminal: creates the vault, then offers to
+seed it by importing `articles/*.md` from an existing wiki you point it at
+(existing files are never overwritten), then offers to install the nightly
+synthesis job. Both offers are TTY-only and default to no import / no
+schedule, so a non-interactive `init` just creates the vault.
 
-## Agent tools
+`open` prefers Obsidian (`open -a Obsidian` for the vault, the `obsidian://`
+URL handler for a single article) and falls back to Finder / the default
+Markdown app.
 
-Both `mlx-bun pi` and the web chat expose these read-only tools, in the recommended call order — FIND the article, READ it small, follow the graph, fall back to reference docs and utilities, search only as a last resort:
+### Read from the terminal
 
-**FIND** (deterministic lookup, never a vector search):
-- `memory_resolve` — name/alias → the article
-- `memory_category` — category/type/series → members
+```bash
+mlx-bun memory list                        # article stems + Reference/* docs
+mlx-bun memory search <query>              # ranked article matches + sample lines
+mlx-bun memory toc <article>               # headings with their #anchors
+mlx-bun memory section <article> <anchor>  # one section's body
+mlx-bun memory links <article>             # resolved outbound + inbound [[wikilinks]]
+mlx-bun memory read <article>              # the full article
+```
 
-**READ** (TOC → one section, not a full-article dump):
-- `memory_read` — TOC + lead by default; `force=true` for the full article
-- `memory_section` — one section's body (the default granularity)
+These are the same filesystem helpers the agent tools use. `search` is a
+substring search over articles and reference docs, not a semantic one; results
+prefixed `Reference/` are mlx-bun docs, not your memory.
 
-**Follow the graph**:
-- `memory_links` — outbound/inbound wikilinks
-- `memory_infobox` — read an article's infobox fields (there is no infobox *query* tool — the infobox is content the model reads, not a filter)
+### Synthesize
 
-**Reference docs** (mlx-bun's own docs, mirrored read-only into the vault):
-- `reference_search`
-- `reference_read`
-- `reference_list`
+```bash
+mlx-bun memory synthesize [--dry-run]      # run the full pipeline now (aliases: pipeline, all)
+```
 
-**Utility**:
-- `memory_list` — user articles plus read-only `Reference/*` docs
-- `memory_status` — vault path, setup state, article count, git state, last synthesis run (from the vault's git log), schedule state, recent changed articles
+**This writes to your vault** (only `articles/`, committed to git). It runs
+the full local DAG — segment → extract → route → create/patch → link — then an
+editorial wikify sweep over every article. `--dry-run` is the safe mode: it
+lists the planned stages, makes no model calls, and writes nothing. The
+command also accepts `--since` and `--model`, but the current pipeline does
+not apply them — a run always covers the store's whole pending corpus with the
+default model.
 
-**Last resort**:
-- `memory_search` — substring fallback when FIND failed
+Synthesis runs on Gemma-4-e4b (the OptiQ 4-bit release) loaded from the
+Hugging Face hub cache; if it is not downloaded the command stops with the
+exact `hf download` line to run. If a trained `memory-chunk` adapter is present
+at `~/.cache/mlx-bun/adapters/memory-chunk` the segment stage mounts it; every
+other stage runs the base model with its policy prompt. Set
+`MLX_BUN_MEMORY_BATCH=<n>` to batch model calls (default 1).
+
+Synthesis operates on conversations already loaded into the synthesis store.
+The pipeline's ingest step reports "corpus already in the store" — it does not
+scan your pi sessions on its own today, so a fresh store yields an empty run.
+
+### Run one stage
 
-The tools are always registered. If no vault exists, they return a setup message instead of disappearing. Search/read results prefixed with `Reference/` are mlx-bun docs, not user memory articles.
+The DAG is decomposed into independent, resumable workers. Each pulls its
+eligible work from the store by state, walks oldest-conversation-first,
+persists, and exits — so you can run them separately, or as separate concurrent
+processes on different conversation slices (GPU and memory allowing):
 
-## Web chat REST surface
+```bash
+mlx-bun memory segment          [--limit N] [--convs a,b]   # conversations → topic chunks
+mlx-bun memory extract          [--limit N] [--convs a,b]   # chunks → entities
+mlx-bun memory route                       [--convs a,b]   # entities → create / capture decisions
+mlx-bun memory synthesize-stage [--limit N] [--convs a,b]   # create + patch articles (alias: stage-synthesize)
+mlx-bun memory link             [--limit N]                 # deterministic cross-linking, no model
+```
 
-The web chat's Memory panel talks to a separate, HTTP-only `/api/memory/*` route surface (status/list/search/article/links/history/diff + a `POST /api/memory/init` consent-card endpoint) — thin wrappers over the same vault read path above, documented in [server-api.md](server-api.md#get-apimemory--post-apimemoryinit). The agent-tool surface above is unaffected by it.
+`--limit` bounds the batch (segment, extract, synthesize-stage, link);
+`--convs` restricts a run to specific conversation ids. `link` inline-links
+first mentions of other articles and rebuilds each `## See also` from mentions
+and co-occurrence; it is idempotent and needs no model.
 
-## Obsidian
+### Schedule the Dreaming
 
 ```bash
-mlx-bun memory open
-mlx-bun memory open Archie_Project
+mlx-bun memory schedule [--at HH:MM]   # install the nightly job (default 03:00)
+mlx-bun memory unschedule              # remove it
 ```
 
-The first command opens the vault. The second opens one article. Obsidian is preferred; Finder/default Markdown app is the fallback.
+`schedule` writes a launchd agent (`~/Library/LaunchAgents/com.mlx-bun.memory.plist`,
+label `com.mlx-bun.memory`) that runs `mlx-bun memory synthesize` at the
+given local time and loads it. launchd survives reboots and runs a job missed
+while the machine was asleep on next wake. `status` shows whether the job is
+installed and loaded; if `launchctl load` fails the plist is still written and
+the command tells you to load it by hand.
+
+## The Dreaming — what nightly synthesis does
+
+From your side, the Dreaming is a nightly editor that reads what you talked
+about and keeps a wiki current. Each run:
+
+1. **Segments** each new conversation into single-topic chunks (pointer ranges
+   into the transcript, not copies).
+2. **Extracts** the subjects each chunk is about — people, projects, tools,
+   decisions — and resolves aliases so "the Lumix" and "Lumix S5" are one thing.
+3. **Routes** every subject: enough substance gets its own article; thin
+   subjects are captured so they stay searchable until they warrant one. There
+   is no notability filter — if you talk about it, it surfaces.
+4. **Creates or patches** articles: new subjects get an outline, per-section
+   drafts, and an infobox; existing articles get one section folded in at a
+   time, the rest left byte-identical. Corrections arrive as ordinary chunks,
+   and chronological processing means your latest position wins.
+5. **Cross-links** the vault (first mentions → `[[wikilinks]]`, co-occurrence →
+   See also) and commits.
+6. **Wikifies** — an editorial sweep over every article: tighten each section
+   without losing any citation or specific detail, refresh the infobox.
+
+Deterministic gates sit between every model step: citation survival, footnote
+integrity, word floors, wikilink resolution, article-structure checks. A weak
+model output is rejected and the article is left untouched — a bad night is a
+no-op, never a corrupted vault. Every article carries `conv:` citations back to
+the conversation that supported it.
+
+Everything runs locally on the same base model you chat with; no embeddings,
+no cloud. Architecture, stage contracts, and the gate definitions are in
+[dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md).
+
+## Agent tools (pi and web chat)
+
+When a vault exists, `mlx-bun pi` and the web chat register these read-only
+tools plus a `memory` skill that teaches the workflow: **FIND** the article,
+**READ** it small, follow the graph, use reference docs for mlx-bun questions,
+and search only as a last resort. The system prompt gets one soft hint that
+memory is on; memory is for user-specific continuity, not a first step for
+weather, public facts, or ordinary coding tasks.
+
+**FIND** (deterministic lookup, never a vector search):
+- `memory_resolve` — name or alias → the article (stem, title, kind, lead); offers near candidates on a miss.
+- `memory_category` — one of category / type / series → member articles.
+
+**READ** (TOC → one section, not a whole-article dump):
+- `memory_read` — TOC + lead by default; `force=true` for the full article (large articles still degrade to TOC + lead).
+- `memory_section` — one section by heading anchor; the default read granularity.
+
+**Follow the graph**:
+- `memory_links` — outbound `[[links]]` grouped by origin (infobox / series / see also / prose) plus inbound backlinks.
+- `memory_infobox` — an article's infobox as key:value facts to read. There is no infobox *query* tool; to find articles use `memory_resolve` or `memory_category`.
+
+**Reference docs** (mlx-bun's own docs, mirrored read-only under `Reference/`):
+- `reference_search`, `reference_read`, `reference_list`.
+
+**Utility**:
+- `memory_list` — your article stems (an overview/fallback, not a first move).
+- `memory_status` — vault path, setup state, article count, git state, schedule state.
+
+**Last resort**:
+- `memory_search` — substring search across articles; use only after the deterministic finders failed.
+
+All of these are read-only and auto-allowed; none can modify an article. If
+the vault is missing, the tools are not registered at all, and the skill and
+prompt hint are omitted.
+
+## Web chat: Memory panel and REST routes
+
+The web chat's Memory panel talks to a loopback-only HTTP surface that wraps
+the same vault helpers: `GET /api/memory/status`, `/list`, `/search`,
+`/article`, `/links`, `/history`, `/diff`, and `POST /api/memory/init` (the
+first-run consent card; the same `setupVault` the CLI uses, minus the
+interactive import and schedule prompts). A `GET /v1/memory/synthesize`
+route streams a synthesis run as server-sent events (`?dry=1` for a dry run).
+Wire details — parameters, response shapes, the no-vault response — are owned
+by [server-api.md](server-api.md#get-apimemory--post-apimemoryinit). These
+routes never touch the agent-tool surface above.
 
-## Synthesis roadmap
+## Consent and safety
 
-The pipeline (saved pi sessions → articles) runs today on the local model; the roadmap hardens each stage with four specialized LoRA adapters trained from existing Lucien-scale pipeline data and distributed from Hugging Face:
+- Creating the vault is explicit (`memory init` or the web consent card);
+  scheduling is explicit (`memory schedule` or the `init` prompt).
+- Chat-time tools are read-only. Article mutation by the agent during chat is
+  out of bounds by construction.
+- Opening Obsidian or Finder is you browsing and editing your own files; it is
+  not agent mutation.
+- Imports copy articles in; the source wiki is never modified.
+- Nothing — articles, conversations, synthesis — leaves the machine.
 
-1. deterministic ingest of saved web/terminal pi session JSONL;
-2. **memory-chunk LoRA** — conversation → topic chunks (already trained as a WIP artifact);
-3. **memory-bucket LoRA** — chunk → existing bucket or new article proposal;
-4. **memory-synthesis LoRA** — chunks + existing article → conservative article update candidate;
-5. **memory-editor LoRA** — candidate article → polished wiki Markdown;
-6. deterministic footnote/wikilink normalization, quality gates, and git commit.
+## Related
 
-The adapters propose; the pipeline validates and writes. The pipeline writes only `articles/`; `Reference/` is immutable context. Synthesis is live today (`mlx-bun memory synthesize` runs the full DAG; the nightly `memory schedule` job runs it automatically); imported/hand-written user articles and the built-in read-only mlx-bun reference docs coexist with synthesized ones.
+- [cli.md](cli.md#memory--the-personal-wiki) — the `memory` command in the full CLI reference.
+- [server-api.md](server-api.md#get-apimemory--post-apimemoryinit) — `/api/memory/*` wire format.
+- [dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — the canonical pipeline design.
+- [docs/design/dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md), [docs/design/dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — synthesis internals and the local inference path.

package/docs/reference/models.md

@@ -1,8 +1,93 @@
-# Model management: get / scan / ls / gc
-
-How mlx-bun downloads, indexes, lists, and reclaims models. The store is
-the standard Hugging Face hub cache — nothing proprietary; `hf` and
-mlx-lm read/write the same tree.
+# Models: supported roster + management (get / scan / ls / gc)
+
+The one supported-model list, plus how mlx-bun downloads, indexes, lists,
+and reclaims models. The store is the standard Hugging Face hub cache —
+nothing proprietary; `hf` and mlx-lm read/write the same tree.
+
+## Supported models
+
+The source of truth is code, not this table: `src/model/profile.ts`
+(which `model_type` loads via which graph), `src/model/support.ts` (the
+support tier `mlx-bun ls` prints), `src/model/universal/archs.ts` (the
+Tier-0 descriptor table), `src/spec/source.ts` + `src/server.ts` (draft
+sources), and the `instanceof` gates in `src/server.ts` for image / video /
+audio. If this table and the code disagree, the code wins and this table
+is the bug.
+
+Scope is deliberate: a few families held to explicit oracle contracts
+(L1 = bit-exact vs mlx-lm, L2 = bit-exact vs mlx-optiq, L3 = measured, no
+oracle), plus a declared Tier-0 generic surface. There is no alias table —
+the "query" column is a substring that `resolve()` matches against
+already-downloaded repos (it must match exactly one repo; see
+[Query resolution rules](#query-resolution-rules-per-verb)).
+
+| `serve` query | Validated artifact (HF repo) | Family · load path · tier | Modalities | Draft sources | KV schemes | Notes |
+| --- | --- | --- | --- | --- | --- | --- |
+| `e4b` | `mlx-community/gemma-4-e4b-it-OptiQ-4bit` | Gemma 4 (`gemma4*`) · dedicated `gemma4` graph with a **generated** specialization keyed on the config fingerprint · L1 | text · **image** (SigLIP tower in the bf16 `optiq_vision.safetensors` sidecar) · **audio** (Conformer tower in the same sidecar) | two-model · Gemma `-assistant` (L2) · locally-trained DSpark (`dspark.json`) · ngram | bf16 · uniform 4/8 · `config` (kv_config.json, batches) · turbo · `--paged-kv` | **Default model** (`DEFAULT_REPO_ID`, src/fit.ts). The only audio-capable artifact. ~7.0 GB on disk (benchmarks.md legend, 2026-06-15). Trains (LoRA / ORPO / SFT). |
+| `12B` | `mlx-community/gemma-4-12B-it-OptiQ-4bit` | Gemma 4 · generated specialization · L1 | text · **image** (encoder-free `gemma4_unified_vision` declared in config.json — no sidecar needed). No audio: the snapshot's sidecar is an audio **stub** (`embed_audio` only, no `audio_tower.*`) | two-model · `-assistant` · DSpark local · **DeepSpec released** (`deepseek-ai/dspark_gemma4_12b_block7`) · ngram | bf16 · uniform · `config` · turbo · `--paged-kv` | ~8.4 GB (2026-06-15). `largestRecommendedRepoId` pick for ≥ 24 GB RAM (src/fit.ts). Trains. |
+| `26B` | `mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit` | Gemma 4 MoE · generated specialization · L1 | text · image **only if** the snapshot ships the SigLIP sidecar (registry `has_vision_sidecar`; the tower is `gemma4_vision`). No `audio_config` → no audio | two-model · `-assistant` · ngram | bf16 · uniform · `config` · turbo · `--paged-kv` | ~18 GB (2026-06-15). `largestRecommendedRepoId` pick for ≥ 48 GB RAM. |
+| any other `gemma4*` config (1B, e2b, 31B, …) | — | Gemma 4 · the `Gemma4Model` monolith (no generated specialization) · L1 by family | per the registry's vision/audio flags | same as the family | same as the family | Loads by `model_type`; **no artifact is pinned in tests/support/goldens for these**, so treat as family-supported, artifact-unvalidated. |
+| `MiniCPM5` | `mlx-community/MiniCPM5-1B-OptiQ-4bit` | MiniCPM5 · `model_type: llama` matched by exact shape (`isMiniCPM5Config`) → dedicated `minicpm5` graph · L1 / L2 | text | two-model · ngram | bf16 · uniform · `config` (batches — Phase 3.1) · turbo (quality curve measured 2026-07-06, benchmarks.md §3) | Sub-GB starter (benchmarks.md legend). Trains (segmented backward). Tool calling (XML). |
+| `Qwen3.5-4B` | `mlx-community/Qwen3.5-4B-OptiQ-4bit` | Qwen3.5 (`qwen3_5` / `qwen3_5_text`, dense hybrid gated-DeltaNet) · dedicated `qwen3.5` graph · L1 | text (image/video only when the artifact ships vision weights — see Qwen3.8) | two-model · ngram · `mtp` if a `qwen3_5_mtp` companion exists | bf16 (L1) · `config` (KV-ON parity passed 2026-06-15, PLAN 14e) | Thinking + tool calling. Batches on the SSM path (`MLX_BUN_BATCH_SSM=0` reverts). The MoE variant `qwen3_5_moe` is **not** supported. |
+| `Qwen3.8` | `mlx-community/Qwen3.8-27B-OptiQ-4bit` @ `b04599de95d7a9bfbd7f208d347c0f10d9432a42` | Qwen3.5 family · **exact artifact profile** `qwen3.8-27b-optiq-4bit` (safetensors + `qwen3.5` graph) · L1 | text · **image** · **video** (`image_url` / `video_url` parts; Qwen3-VL tower + mRoPE; video decodes via the AVFoundation sidecar `mlx-bun-frame-extract`, 2 fps, ≤ 768 frames). Video never combines with audio; no audio | **native MTP head** (`mlx-community/Qwen3.8-27B-MTP-bf16`, auto-detected `qwen3_5_mtp`; `--draft-kind mtp` with no `--draft-model` uses a bundled `mtp/` subfolder) · two-model · ngram | bf16 (L1) · `config` is **Lab** — policy copied from the same-topology Qwen3.6 artifact, no model-specific oracle (benchmarks.md, 2026-08-22) | `reasoning_effort` (xhigh/medium/low) + `preserve_thinking` + tool calling. `/v1/models` advertises vision. Shipped in v0.2.0. |
+| `Qwen3-Embedding` | `mlx-community/Qwen3-Embedding-4B-4bit-DWQ` | plain `qwen3` · dedicated `qwen3` graph · L1 | text | — | — | **Embeddings only**: `/v1/embeddings` + `mlx-bun embed`. Auto-picked by `embed` when no query is given; never the chat default. |
+| `Qwen3-30B` | `mlx-community/Qwen3-30B-A3B-Thinking-2507-4bit` | Qwen3-MoE (`qwen3_moe`) · dedicated `qwen3-moe` graph · L1 (opt-in slow parity test, `MLX_BUN_TEST_QWEN3_MOE=1`) | text | two-model · ngram | bf16 (the artifact ships no kv_config.json) | Sparse top-k SwitchGLU experts. |
+| `diffusiongemma` | `mlx-community/diffusiongemma-26B-A4B-it-OptiQ-4bit` | DiffusionGemma (`diffusion_gemma`) · dedicated graph + **denoising loop** (non-autoregressive) · L2 (optiq) | text · **image** (single image; its own inline-quantized SigLIP tower). No video / audio | — (no AR verify loop) | — (canvas model; the `--kv-quant` axis does not apply) | Serial lane always. Diffusion-native LoRA trains. |
+| `GLM-5.2-colibri` | `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` @ `3cc8db99b1b13fc79325d987ba3c1c430766b3b8` | GLM-5.2 (`glm_moe_dsa`) · **Colibri container loader** + dedicated `glm5.2` graph (streamed experts) · **exact artifact profile**, L3 | text | **checkpoint-native MTP** (`--mtp on`, the default; mutually exclusive with `--draft-model`) | checkpoint-native compressed **MLA/DSA** cache (its own scheme; SSD tier v3 persists it) | Fixed-context admission: `--context-length` (4096) and `--max-tokens` (128) are reserved by the header-only resource equation before weights open; `--memory-budget` defaults to min(25 GiB, RAM). `--mtp off` enables ordinary batching. Embeddings, vision/audio, adapters, and training are unsupported. See [below](#direct-container-artifacts-glm-5x). |
+| Tier-0 generic | validated manifest (`tests/support/universal-manifest.ts`): `Llama-3.2-1B-Instruct-4bit`, `Qwen2.5-0.5B-Instruct-4bit`, `Qwen3-0.6B-4bit`, `quantized-gemma-2b-it`, `gemma-2-2b-it-4bit`, `Phi-3.5-mini-instruct-4bit`, `GLM-4-9B-0414-4bit`, `granite-3.3-2b-instruct-4bit`, `starcoder2-3b-4bit`, `SmolLM3-3B-4bit` (all `mlx-community/`) and `ekryski/OLMo-2-0425-1B-Instruct-4bit` | 11 `model_type`s: `llama`, `smollm3`, `qwen2`, `qwen3`, `gemma`, `gemma2`, `phi3`, `olmo2`, `glm4`, `granite`, `starcoder2` (+ mlx-lm remaps `mistral`, `iquestcoder` → `llama`) · `universal-dense` graph · L1 | text | two-model · ngram | bf16; unreleased Llama full-attention uniform KV4/KV8 | `ls` labels these `supported (generic)`. Shared full-attention execution is qualified on Llama-3.2-3B at B1/B4, including speculative KV4/KV8 and generated RAM/SSD state; gemma2-family / sliding-window archs run serial. No compiled specialization. A `model_type` absent from the table refuses to load. |
+
+Rules that apply across rows (all enforced in `src/server.ts`):
+
+- **Draft sources.** Two-model and ngram work on any autoregressive target
+  (two-model requires the same tokenizer family — probe-checked at startup).
+  The KV-borrowing drafters (`-assistant`, DSpark, DeepSpec) require a
+  Gemma 4 target; `qwen3_5_mtp` heads require a Qwen3.5-family target; GLM's
+  MTP is its own row. Qwen MTP companions may store dense or affine-quantized
+  projections; the loader uses each module's quantization metadata. Target
+  verification is unchanged, while draft acceptance and speed depend on the
+  companion. In the unreleased shared executor, qualified plain, uniform,
+  per-layer affine and TurboQuant layouts compose with drafting through the
+  target/provider interfaces. Positive affine conversion thresholds are also
+  supported by library requests. The loaded model and provider determine
+  which combinations can execute.
+- **Qwen3.5-family checkpoint generations.** Both ship the same graph and both
+  load: the mlx-lm-converted naming (`language_model.model.*`,
+  `language_model.lm_head`, `vision_tower.*`) and the transformers-5.8 export
+  (`model.language_model.*`, top-level `lm_head`, `model.visual.*`, in-repo
+  `mtp.*`, HF-layout `conv1d.weight` `[C,1,K]`, RMSNorm gains stored as γ−1).
+  The load seam (`src/model/qwen3_5-checkpoint.ts`) replicates mlx-lm's
+  `sanitize()` exactly: in-repo `mtp.*` and the vision tensors are dropped from
+  the text graph, and the +1.0 γ shift fires **only** when the artifact carries
+  `mtp.*` tensors **or** an unsanitized `conv1d.weight` — never on the naming
+  alone, so an old artifact can never be shifted.
+- **KV schemes.** `--kv-quant 4|8` (uniform, L1) and `config` (per-layer
+  `kv_config.json`, L2) apply to the autoregressive families that ship the
+  file; `turbo` additionally requires a full-attention `head_dim` in
+  {64, 128, 256, 512} (refused at startup otherwise). Qualified TurboQuant
+  targets support shared ordinary and speculative execution; sliding layers
+  remain bf16;
+  `--paged-kv` is Gemma 4 only, bf16, serial. Flag detail:
+  [server-config.md](server-config.md).
+- **Modalities.** Image requests reach a Gemma 4 model only when
+  `has_vision_sidecar` or a `*_unified_vision` config is present; audio only
+  when `audio_config` AND the sidecar's `audio_tower.*` tensors are both
+  present; video only on a Qwen3.5-family model. A request carrying media the
+  model cannot take is an explicit 400, never a silent text-only degrade.
+  Wire formats: [server-api.md](server-api.md).
+- **Memory.** Disk sizes above are the dated figures from
+  [benchmarks.md](benchmarks.md); for RAM, ask `mlx-bun fit <query> --ctx
+  <n>` rather than a table (src/fit.ts is the calculator).
+
+### How a new model gets here
+
+A `model_type` earns a row by landing a descriptor (Tier-0) or a dedicated
+graph (targeted) **plus a green parity manifest entry** against the oracle
+that already ships it — the process, cost per arch, and the graduation path
+from generic to targeted are in
+[docs/design/generic-model-support.md](../design/generic-model-support.md)
+(§3.1 descriptor table, §3.5 the parity gate, §3.6 graduation). Exact
+artifacts additionally get a declared profile (see
+[Declared model profiles](#declared-model-profiles)).
 
 ## The cache layout (and why duplicates happen)
 
@@ -65,7 +150,16 @@ existing prefix in chunks — no whole-file allocation):
 A mismatch deletes the partial (never resume corrupt bytes); a short
 read keeps the `.incomplete` for resume.
 
-## GLM-5.2 on a 32 GB Mac
+## Direct-container artifacts (GLM-5.x)
+
+Models that load through the Colibri container loader (`loader: "colibri"`
+in `src/model/profile.ts`) skip `Weights.open` entirely: a header-only
+resource equation runs before any resident tensor or expert slab opens, and
+an impossible plan fails without committing memory. One validated artifact
+today; a GLM-5.3+ artifact joins as a new row in the roster above plus a
+subsection here once it has a declared profile.
+
+### GLM-5.2 on a 32 GB Mac
 
 The validated direct-container artifact is
 `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` at revision
@@ -134,6 +228,15 @@ sampling. Those methods remain explicit flags/request fields or documented
 defaults. If an exact profile requires a capability the running engine does not
 have, loading refuses; it never downgrades to another model path.
 
+**Packed trellis artifacts.** A `quantization` entry with `mode: "trellis"`
+(TurboQuant Q2b, `mjriii/Qwen3.8-27B-q3-…-packed`) stores that module as a
+trellis-coded bit-stream + fp16 row scales and is served by the engine's own
+Metal kernels; stock mlx-lm cannot load it. Experimental k3/axis0 artifacts
+can store two-block interleaved codes as a 3D tensor; the loader identifies
+the layout from its shape and keeps one resident code copy. Format and kernels:
+`docs/design/turboquant.md` (Q campaign); fallback `MLX_BUN_TRELLIS=expand`:
+`server-config.md`.
+
 Fidelity labels mean L1 = bit-exact mlx-lm, L2 = bit-exact mlx-optiq, and L3 =
 measured evidence without a matching oracle. Model profiles do not imply that a
 quantized artifact retains the base model's benchmark score; quality evidence
@@ -225,3 +328,7 @@ Safety rails:
 
 After a real deletion, gc re-scans so the registry drops the reaped
 rows immediately.
+
+
+Non-Qwen grouped TurboQuant drafting, including assistant donor attention, uses
+the shared cache interfaces. See [composition coverage](../design/batching.md#turboquant-donor-attention-and-grouped-draft-composition).