mlx-bun 0.5.0 → 1.0.0

package/docs/reference/memory.md

@@ -1,260 +0,0 @@
-# Memory — the personal wiki
-
-`mlx-bun memory` is a local, durable memory for your assistant: a wiki of
-Markdown articles at `~/.mlx-bun/wiki/` that the built-in pi agent reads to
-remember your projects, people, decisions, and history across sessions. It is
-yours — git-tracked, editable in any tool (Obsidian opens it as a vault), and it
-never leaves the machine.
-
-The one rule the whole feature is built around:
-
-> Chat-time agents **read** memory. `mlx-bun memory synthesize` is the only
-> thing that **writes** it.
-
-Articles are durable priors; conversations are evidence; synthesis is the
-deliberate, auditable step that turns evidence into updated articles. Normal
-chat never mutates an article. Nightly synthesis is called **the Dreaming**;
-its engineering design lives in
-[dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — this
-page is the user-facing reference.
-
-## Quick start
-
-```bash
-mlx-bun memory init            # create the wiki; offers import + nightly job
-mlx-bun pi                     # the assistant now reads your memory automatically
-mlx-bun memory status          # where it is, how many articles, schedule state
-mlx-bun memory search <query>  # find something from the terminal
-mlx-bun memory open            # browse it in Obsidian
-```
-
-`init` is idempotent and safe to re-run. It never overwrites files you have
-edited: the README and `Meta/` pages are written only when missing. Once a
-wiki exists, every `mlx-bun pi` session and the web chat behind `mlx-bun serve`
-pick it up with no further configuration; if it does not exist, no memory
-tools or prompt hints are exposed at all.
-
-`mlx-bun setup` is a true alias for `mlx-bun memory` (same subcommands).
-
-## What the wiki contains
-
-Everything lives under one portable folder (copy it and your memory moves
-with you):
-
-| Path | What it is |
-| --- | --- |
-| `~/.mlx-bun/wiki/` | The vault root. Override with `MLX_BUN_WIKI=/path/to/wiki`. |
-| `articles/*.md` | Your memory articles — one topic per file, `Topic_Name.md`. The only thing synthesis writes. |
-| `Reference/*.md` | Read-only symlinks to mlx-bun's own docs (README, server API/config, library API, embedding, distribution, training, ORPO quickstart, product roadmap), so the assistant can answer mlx-bun questions on day one. Synthesis never writes here. |
-| `Meta/*.md` | Operational pages the pipeline follows: editorial guidelines, article conventions, infobox schemas, category definitions, entities policy, summary style, bucketing/buckets, chunking, topics to ignore. Edit them to steer synthesis. |
-| `Talk/` | Per-article discussion and conflict notes. |
-| `README.md`, `.gitignore` | Orientation page; keeps `.DS_Store` and Obsidian workspace state out of git. |
-
-The vault is plain Markdown and a git repository (`init` runs `git init` and
-makes the initial commit). Every synthesis run commits its changes, so any
-article change is reviewable and revertible with ordinary git. You can edit
-any file directly; the next run respects your edits.
-
-Two things live **outside** the vault because they are derived or operational:
-
-- `~/.cache/mlx-bun/memory.sqlite` — the synthesis store (ingested
-  conversations, chunks, buckets, watermarks, and a rebuildable index of the
-  articles). The vault is truth; this DB is a cache plus pipeline ledger.
-- `~/.mlx-bun/logs/memory-synthesis.{out,err}.log` — output of the nightly job.
-
-Article names are **stems**: the filename without `.md`, underscores for
-spaces (`Archie_Project`). Every command and tool below takes a stem.
-
-## CLI
-
-All subcommands are `mlx-bun memory <sub>`; with no subcommand, `status` runs.
-An unknown subcommand prints help and exits non-zero (so scripts and launchd
-jobs can detect typos).
-
-### Set up and inspect
-
-```bash
-mlx-bun memory init              # create the wiki (alias: setup)
-mlx-bun memory status            # path, article + reference counts, git, nightly state, recent articles
-mlx-bun memory open [article]    # open the wiki or one article (alias: browse)
-```
-
-`init` walks through setup in a terminal: creates the vault, then offers to
-seed it by importing `articles/*.md` from an existing wiki you point it at
-(existing files are never overwritten), then offers to install the nightly
-synthesis job. Both offers are TTY-only and default to no import / no
-schedule, so a non-interactive `init` just creates the vault.
-
-`open` prefers Obsidian (`open -a Obsidian` for the vault, the `obsidian://`
-URL handler for a single article) and falls back to Finder / the default
-Markdown app.
-
-### Read from the terminal
-
-```bash
-mlx-bun memory list                        # article stems + Reference/* docs
-mlx-bun memory search <query>              # ranked article matches + sample lines
-mlx-bun memory toc <article>               # headings with their #anchors
-mlx-bun memory section <article> <anchor>  # one section's body
-mlx-bun memory links <article>             # resolved outbound + inbound [[wikilinks]]
-mlx-bun memory read <article>              # the full article
-```
-
-These are the same filesystem helpers the agent tools use. `search` is a
-substring search over articles and reference docs, not a semantic one; results
-prefixed `Reference/` are mlx-bun docs, not your memory.
-
-### Synthesize
-
-```bash
-mlx-bun memory synthesize [--dry-run]      # run the full pipeline now (aliases: pipeline, all)
-```
-
-**This writes to your vault** (only `articles/`, committed to git). It runs
-the full local DAG — segment → extract → route → create/patch → link — then an
-editorial wikify sweep over every article. `--dry-run` is the safe mode: it
-lists the planned stages, makes no model calls, and writes nothing. The
-command also accepts `--since` and `--model`, but the current pipeline does
-not apply them — a run always covers the store's whole pending corpus with the
-default model.
-
-Synthesis runs on Gemma-4-e4b (the OptiQ 4-bit release) loaded from the
-Hugging Face hub cache; if it is not downloaded the command stops with the
-exact `hf download` line to run. If a trained `memory-chunk` adapter is present
-at `~/.cache/mlx-bun/adapters/memory-chunk` the segment stage mounts it; every
-other stage runs the base model with its policy prompt. Set
-`MLX_BUN_MEMORY_BATCH=<n>` to batch model calls (default 1).
-
-Synthesis operates on conversations already loaded into the synthesis store.
-The pipeline's ingest step reports "corpus already in the store" — it does not
-scan your pi sessions on its own today, so a fresh store yields an empty run.
-
-### Run one stage
-
-The DAG is decomposed into independent, resumable workers. Each pulls its
-eligible work from the store by state, walks oldest-conversation-first,
-persists, and exits — so you can run them separately, or as separate concurrent
-processes on different conversation slices (GPU and memory allowing):
-
-```bash
-mlx-bun memory segment          [--limit N] [--convs a,b]   # conversations → topic chunks
-mlx-bun memory extract          [--limit N] [--convs a,b]   # chunks → entities
-mlx-bun memory route                       [--convs a,b]   # entities → create / capture decisions
-mlx-bun memory synthesize-stage [--limit N] [--convs a,b]   # create + patch articles (alias: stage-synthesize)
-mlx-bun memory link             [--limit N]                 # deterministic cross-linking, no model
-```
-
-`--limit` bounds the batch (segment, extract, synthesize-stage, link);
-`--convs` restricts a run to specific conversation ids. `link` inline-links
-first mentions of other articles and rebuilds each `## See also` from mentions
-and co-occurrence; it is idempotent and needs no model.
-
-### Schedule the Dreaming
-
-```bash
-mlx-bun memory schedule [--at HH:MM]   # install the nightly job (default 03:00)
-mlx-bun memory unschedule              # remove it
-```
-
-`schedule` writes a launchd agent (`~/Library/LaunchAgents/com.mlx-bun.memory.plist`,
-label `com.mlx-bun.memory`) that runs `mlx-bun memory synthesize` at the
-given local time and loads it. launchd survives reboots and runs a job missed
-while the machine was asleep on next wake. `status` shows whether the job is
-installed and loaded; if `launchctl load` fails the plist is still written and
-the command tells you to load it by hand.
-
-## The Dreaming — what nightly synthesis does
-
-From your side, the Dreaming is a nightly editor that reads what you talked
-about and keeps a wiki current. Each run:
-
-1. **Segments** each new conversation into single-topic chunks (pointer ranges
-   into the transcript, not copies).
-2. **Extracts** the subjects each chunk is about — people, projects, tools,
-   decisions — and resolves aliases so "the Lumix" and "Lumix S5" are one thing.
-3. **Routes** every subject: enough substance gets its own article; thin
-   subjects are captured so they stay searchable until they warrant one. There
-   is no notability filter — if you talk about it, it surfaces.
-4. **Creates or patches** articles: new subjects get an outline, per-section
-   drafts, and an infobox; existing articles get one section folded in at a
-   time, the rest left byte-identical. Corrections arrive as ordinary chunks,
-   and chronological processing means your latest position wins.
-5. **Cross-links** the vault (first mentions → `[[wikilinks]]`, co-occurrence →
-   See also) and commits.
-6. **Wikifies** — an editorial sweep over every article: tighten each section
-   without losing any citation or specific detail, refresh the infobox.
-
-Deterministic gates sit between every model step: citation survival, footnote
-integrity, word floors, wikilink resolution, article-structure checks. A weak
-model output is rejected and the article is left untouched — a bad night is a
-no-op, never a corrupted vault. Every article carries `conv:` citations back to
-the conversation that supported it.
-
-Everything runs locally on the same base model you chat with; no embeddings,
-no cloud. Architecture, stage contracts, and the gate definitions are in
-[dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md).
-
-## Agent tools (pi and web chat)
-
-When a vault exists, `mlx-bun pi` and the web chat register these read-only
-tools plus a `memory` skill that teaches the workflow: **FIND** the article,
-**READ** it small, follow the graph, use reference docs for mlx-bun questions,
-and search only as a last resort. The system prompt gets one soft hint that
-memory is on; memory is for user-specific continuity, not a first step for
-weather, public facts, or ordinary coding tasks.
-
-**FIND** (deterministic lookup, never a vector search):
-- `memory_resolve` — name or alias → the article (stem, title, kind, lead); offers near candidates on a miss.
-- `memory_category` — one of category / type / series → member articles.
-
-**READ** (TOC → one section, not a whole-article dump):
-- `memory_read` — TOC + lead by default; `force=true` for the full article (large articles still degrade to TOC + lead).
-- `memory_section` — one section by heading anchor; the default read granularity.
-
-**Follow the graph**:
-- `memory_links` — outbound `[[links]]` grouped by origin (infobox / series / see also / prose) plus inbound backlinks.
-- `memory_infobox` — an article's infobox as key:value facts to read. There is no infobox *query* tool; to find articles use `memory_resolve` or `memory_category`.
-
-**Reference docs** (mlx-bun's own docs, mirrored read-only under `Reference/`):
-- `reference_search`, `reference_read`, `reference_list`.
-
-**Utility**:
-- `memory_list` — your article stems (an overview/fallback, not a first move).
-- `memory_status` — vault path, setup state, article count, git state, schedule state.
-
-**Last resort**:
-- `memory_search` — substring search across articles; use only after the deterministic finders failed.
-
-All of these are read-only and auto-allowed; none can modify an article. If
-the vault is missing, the tools are not registered at all, and the skill and
-prompt hint are omitted.
-
-## Web chat: Memory panel and REST routes
-
-The web chat's Memory panel talks to a loopback-only HTTP surface that wraps
-the same vault helpers: `GET /api/memory/status`, `/list`, `/search`,
-`/article`, `/links`, `/history`, `/diff`, and `POST /api/memory/init` (the
-first-run consent card; the same `setupVault` the CLI uses, minus the
-interactive import and schedule prompts). A `GET /v1/memory/synthesize`
-route streams a synthesis run as server-sent events (`?dry=1` for a dry run).
-Wire details — parameters, response shapes, the no-vault response — are owned
-by [server-api.md](server-api.md#get-apimemory--post-apimemoryinit). These
-routes never touch the agent-tool surface above.
-
-## Consent and safety
-
-- Creating the vault is explicit (`memory init` or the web consent card);
-  scheduling is explicit (`memory schedule` or the `init` prompt).
-- Chat-time tools are read-only. Article mutation by the agent during chat is
-  out of bounds by construction.
-- Opening Obsidian or Finder is you browsing and editing your own files; it is
-  not agent mutation.
-- Imports copy articles in; the source wiki is never modified.
-- Nothing — articles, conversations, synthesis — leaves the machine.
-
-## Related
-
-- [cli.md](cli.md#memory-alias-setup--the-personal-wiki) — the `memory` command in the full CLI reference.
-- [server-api.md](server-api.md#get-apimemory--post-apimemoryinit) — `/api/memory/*` wire format.
-- [dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — the canonical pipeline design.
-- [docs/design/dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md), [docs/design/dreaming-nightly-pipeline.md](../design/dreaming-nightly-pipeline.md) — synthesis internals and the local inference path.

package/docs/reference/models.md

@@ -1,346 +0,0 @@
-# Models: supported roster + management (get / scan / ls / gc)
-
-The one supported-model list, plus how mlx-bun downloads, indexes, lists,
-and reclaims models. The store is the standard Hugging Face hub cache —
-nothing proprietary; `hf` and mlx-lm read/write the same tree.
-
-## Automatic model selection
-
-On a fresh install with no supported downloaded model, `serve` downloads and
-starts MiniCPM5-1B, then can download the device-recommended model in the
-background. Gemma e4b is the preferred general model once available and within
-the selection budget. `DEFAULT_REPO_ID` names that preference, not the
-first-download artifact. Explicit model selection bypasses this starter flow.
-
-## Supported models
-
-The source of truth is code, not this table: `src/model/profile.ts`
-(which `model_type` loads via which graph), `src/model/support.ts` (the
-support tier `mlx-bun ls` prints), `src/model/universal/archs.ts` (the
-Tier-0 descriptor table), `src/spec/source.ts` + `src/server.ts` (draft
-sources), and the `instanceof` gates in `src/server.ts` for image / video /
-audio. If this table and the code disagree, the code wins and this table
-is the bug.
-
-Scope is deliberate: a few families held to explicit oracle contracts
-(L1 = bit-exact vs mlx-lm, L2 = bit-exact vs mlx-optiq, L3 = measured, no
-oracle), plus a declared Tier-0 generic surface. These are comparison
-contracts, not interchangeable claims about every artifact. The
-[coverage map](benchmarks.md#coverage-beyond-the-four-model-summary) identifies
-the test and remaining evidence limits for each roster row. There is no alias table —
-the "query" column is a substring that `resolve()` matches against
-already-downloaded repos (it must match exactly one repo; see
-[Query resolution rules](#query-resolution-rules-per-verb)).
-
-| `serve` query | Validated artifact (HF repo) | Family · load path · tier | Modalities | Draft sources | KV schemes | Notes |
-| --- | --- | --- | --- | --- | --- | --- |
-| `e4b` | `mlx-community/gemma-4-e4b-it-OptiQ-4bit` | Gemma 4 (`gemma4*`) · dedicated `gemma4` graph with a **generated** specialization keyed on the config fingerprint · L1 | text · **image** (SigLIP tower in the bf16 `optiq_vision.safetensors` sidecar) · **audio** (Conformer tower in the same sidecar) | two-model · Gemma `-assistant` (L2) · locally-trained DSpark (`dspark.json`) · ngram | bf16 · uniform 4/8 · `config` (kv_config.json, batches) · turbo · `--paged-kv` | **Preferred model after download**; fresh installs first serve MiniCPM5. See [automatic selection](#automatic-model-selection). The only audio-capable artifact. ~7.0 GB on disk (benchmarks.md legend, 2026-06-15). Trains (LoRA / ORPO / SFT). |
-| `12B` | `mlx-community/gemma-4-12B-it-OptiQ-4bit` | Gemma 4 · generated specialization · L1 | text · **image** (encoder-free `gemma4_unified_vision` declared in config.json — no sidecar needed). No audio: the snapshot's sidecar is an audio **stub** (`embed_audio` only, no `audio_tower.*`) | two-model · `-assistant` · DSpark local · **DeepSpec released** (`deepseek-ai/dspark_gemma4_12b_block7`) · ngram | bf16 · uniform · `config` · turbo · `--paged-kv` | ~8.4 GB (2026-06-15). `largestRecommendedRepoId` pick for ≥ 24 GB RAM (src/fit.ts). Trains. |
-| `26B` | `mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit` | Gemma 4 MoE · generated specialization · L1 | text · image **only if** the snapshot ships the SigLIP sidecar (registry `has_vision_sidecar`; the tower is `gemma4_vision`). No `audio_config` → no audio | two-model · `-assistant` · ngram | bf16 · uniform · `config` · turbo · `--paged-kv` | ~18 GB (2026-06-15). `largestRecommendedRepoId` pick for ≥ 48 GB RAM. |
-| any other `gemma4*` config (1B, e2b, 31B, …) | — | Gemma 4 · the `Gemma4Model` monolith (no generated specialization) · family support, artifact unvalidated | per the registry's vision/audio flags | same as the family | same as the family | Loads by `model_type`; **no artifact is pinned in tests/support/goldens for these**, so treat as family-supported, artifact-unvalidated. |
-| `MiniCPM5` | `mlx-community/MiniCPM5-1B-OptiQ-4bit` | MiniCPM5 · `model_type: llama` matched by exact shape (`isMiniCPM5Config`) → dedicated `minicpm5` graph · L1 / L2 | text | two-model · ngram | bf16 · uniform · `config` (batches — Phase 3.1) · turbo (quality curve measured 2026-07-06, benchmarks.md §3) | Sub-GB starter (benchmarks.md legend). Trains (segmented backward). Tool calling (XML). |
-| `Qwen3.5-4B` | `mlx-community/Qwen3.5-4B-OptiQ-4bit` | Qwen3.5 (`qwen3_5` / `qwen3_5_text`, dense hybrid gated-DeltaNet) · dedicated `qwen3.5` graph · L1 | text (image/video only when the artifact ships vision weights — see Qwen3.8) | two-model · ngram · `mtp` if a `qwen3_5_mtp` companion exists | bf16 (L1) · `config` (KV-ON parity passed 2026-06-15, PLAN 14e) | Thinking + tool calling. Batches on the SSM path (`MLX_BUN_BATCH_SSM=0` reverts). The MoE variant `qwen3_5_moe` is **not** supported. |
-| `Qwen3.8` | `mlx-community/Qwen3.8-27B-OptiQ-4bit` @ `b04599de95d7a9bfbd7f208d347c0f10d9432a42` | Qwen3.5 family · **exact artifact profile** `qwen3.8-27b-optiq-4bit` (safetensors + `qwen3.5` graph) · L1 | text · **image** · **video** (`image_url` / `video_url` parts; Qwen3-VL tower + mRoPE; video decodes via the AVFoundation sidecar `mlx-bun-frame-extract`, 2 fps, ≤ 768 frames). Video never combines with audio; no audio | **native MTP head** (`mlx-community/Qwen3.8-27B-MTP-bf16`, auto-detected `qwen3_5_mtp`; `--draft-kind mtp` with no `--draft-model` uses a bundled `mtp/` subfolder) · two-model · ngram | bf16 (L1) · `config` is **Lab** — policy copied from the same-topology Qwen3.6 artifact, no model-specific oracle (benchmarks.md, 2026-08-22) | `reasoning_effort` (xhigh/medium/low) + `preserve_thinking` + tool calling. `/v1/models` advertises vision. Shipped in v0.2.0. |
-| `Qwen3-Embedding` | `mlx-community/Qwen3-Embedding-4B-4bit-DWQ` | plain `qwen3` · dedicated `qwen3` graph · L1 | text | — | — | **Embeddings only**: `/v1/embeddings` + `mlx-bun embed`. Auto-picked by `embed` when no query is given; never the chat default. |
-| `Qwen3-30B` | `mlx-community/Qwen3-30B-A3B-Thinking-2507-4bit` | Qwen3-MoE (`qwen3_moe`) · dedicated `qwen3-moe` graph · L1 (opt-in slow parity test, `MLX_BUN_TEST_QWEN3_MOE=1`) | text | two-model · ngram | bf16 (the artifact ships no kv_config.json) | Sparse top-k SwitchGLU experts. |
-| `diffusiongemma` | `mlx-community/diffusiongemma-26B-A4B-it-OptiQ-4bit` | DiffusionGemma (`diffusion_gemma`) · dedicated graph + **denoising loop** (non-autoregressive) · L2 (optiq) | text · **image** (single image; its own inline-quantized SigLIP tower). No video / audio | — (no AR verify loop) | — (canvas model; the `--kv-quant` axis does not apply) | Serial lane always. Diffusion-native LoRA trains. |
-| `whisper` | `mlx-community/whisper-large-v3-turbo` (fp16, 1.6 GB, `weights.safetensors`) | Whisper (`whisper`) · dedicated encoder-decoder graph + transcription loop · **L1 vs mlx-whisper 0.4.3** (mel, encoder output and per-step decoder logits bit-exact; `tests/parity/whisper.test.ts`) | **audio in → text out** only: `/v1/audio/transcriptions`, `/v1/audio/translations`, `mlx-bun transcribe`. Never the chat model. | — | — | Tokenizer files are not in the mlx-community repo; the loader resolves `openai/whisper-large-v3-turbo`'s `tokenizer.json` from the HF cache (2.4 MB, `mlx-bun get` it once). Beam search (`beam_size`) is ours — mlx-whisper has none; checked against whisper.cpp's beam-5 transcripts. Word timestamps not yet. Serves as a companion beside any chat model or alone (transcription-only server). Any `mlx-community/whisper-*` fp16 checkpoint loads (80-mel v1/v2 vocab resolves `openai/whisper-large-v2`). |
-| `GLM-5.2-colibri` | `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` @ `3cc8db99b1b13fc79325d987ba3c1c430766b3b8` | GLM-5.2 (`glm_moe_dsa`) · **Colibri container loader** + dedicated `glm5.2` graph (streamed experts) · **exact artifact profile**, L3 | text | **checkpoint-native MTP** (`--mtp on`, the default; mutually exclusive with `--draft-model`) | checkpoint-native compressed **MLA/DSA** cache (its own scheme; SSD tier v3 persists it) | Fixed-context admission: `--context-length` (4096) and `--max-tokens` (128) are reserved by the header-only resource equation before weights open; `--memory-budget` defaults to min(25 GiB, RAM). `--mtp off` enables ordinary batching. Embeddings, vision/audio, adapters, and training are unsupported. See [below](#direct-container-artifacts-glm-5x). |
-| Tier-0 generic | validated manifest (`tests/support/universal-manifest.ts`): `Llama-3.2-1B-Instruct-4bit`, `Qwen2.5-0.5B-Instruct-4bit`, `Qwen3-0.6B-4bit`, `quantized-gemma-2b-it`, `gemma-2-2b-it-4bit`, `Phi-3.5-mini-instruct-4bit`, `GLM-4-9B-0414-4bit`, `granite-3.3-2b-instruct-4bit`, `starcoder2-3b-4bit`, `SmolLM3-3B-4bit` (all `mlx-community/`) and `ekryski/OLMo-2-0425-1B-Instruct-4bit` | 11 `model_type`s: `llama`, `smollm3`, `qwen2`, `qwen3`, `gemma`, `gemma2`, `phi3`, `olmo2`, `glm4`, `granite`, `starcoder2` (+ mlx-lm remaps `mistral`, `iquestcoder` → `llama`) · `universal-dense` graph · L1 | text | two-model · ngram | bf16; Llama full-attention uniform KV4/KV8 | `ls` labels these `supported (generic)`. Shared full-attention execution is qualified on Llama-3.2-3B at B1/B4, including speculative KV4/KV8 and generated RAM/SSD state; gemma2-family / sliding-window archs run serial. No compiled specialization. A `model_type` absent from the table refuses to load. |
-
-Rules that apply across rows (all enforced in `src/server.ts`):
-
-- **Draft sources.** Two-model and ngram work on any autoregressive target
-  (two-model requires the same tokenizer family — probe-checked at startup).
-  The KV-borrowing drafters (`-assistant`, DSpark, DeepSpec) require a
-  Gemma 4 target; `qwen3_5_mtp` heads require a Qwen3.5-family target; GLM's
-  MTP is its own row. Qwen MTP companions may store dense or affine-quantized
-  projections; the loader uses each module's quantization metadata. Target
-  verification is unchanged, while draft acceptance and speed depend on the
-  companion. In the shared executor, qualified plain, uniform,
-  per-layer affine and TurboQuant layouts compose with drafting through the
-  target/provider interfaces. Positive affine conversion thresholds are also
-  supported by library requests. The loaded model and provider determine
-  which combinations can execute.
-- **Qwen3.5-family checkpoint generations.** Both ship the same graph and both
-  load: the mlx-lm-converted naming (`language_model.model.*`,
-  `language_model.lm_head`, `vision_tower.*`) and the transformers-5.8 export
-  (`model.language_model.*`, top-level `lm_head`, `model.visual.*`, in-repo
-  `mtp.*`, HF-layout `conv1d.weight` `[C,1,K]`, RMSNorm gains stored as γ−1).
-  The load seam (`src/model/qwen3_5-checkpoint.ts`) replicates mlx-lm's
-  `sanitize()` exactly: in-repo `mtp.*` and the vision tensors are dropped from
-  the text graph, and the +1.0 γ shift fires **only** when the artifact carries
-  `mtp.*` tensors **or** an unsanitized `conv1d.weight` — never on the naming
-  alone, so an old artifact can never be shifted.
-- **KV schemes.** `--kv-quant 4|8` (uniform, L1) and `config` (per-layer
-  `kv_config.json`, L2) apply to the autoregressive families that ship the
-  file; `turbo` additionally requires a full-attention `head_dim` in
-  {64, 128, 256, 512} (refused at startup otherwise). Qualified TurboQuant
-  targets support shared ordinary and speculative execution; sliding layers
-  remain bf16;
-  `--paged-kv` is Gemma 4 only, bf16, serial. Flag detail:
-  [server-config.md](server-config.md).
-- **Modalities.** Image requests reach a Gemma 4 model only when
-  `has_vision_sidecar` or a `*_unified_vision` config is present; audio only
-  when `audio_config` AND the sidecar's `audio_tower.*` tensors are both
-  present; video only on a Qwen3.5-family model. A request carrying media the
-  model cannot take is an explicit 400, never a silent text-only degrade.
-  Wire formats: [server-api.md](server-api.md).
-- **Memory.** Disk sizes above are the dated figures from
-  [benchmarks.md](benchmarks.md); for RAM, ask `mlx-bun fit <query> --ctx
-  <n>` rather than a table (src/fit.ts is the calculator).
-
-### How a new model gets here
-
-A `model_type` earns a row by landing a descriptor (Tier-0) or a dedicated
-graph (targeted) **plus a green parity manifest entry** against the oracle
-that already ships it — the process, cost per arch, and the graduation path
-from generic to targeted are in
-[docs/design/generic-model-support.md](../design/generic-model-support.md)
-(§3.1 descriptor table, §3.5 the parity gate, §3.6 graduation). Exact
-artifacts additionally get a declared profile (see
-[Declared model profiles](#declared-model-profiles)).
-
-## The cache layout (and why duplicates happen)
-
-Default root: `~/.cache/huggingface/hub` (override with `HF_HUB_CACHE`,
-or `HF_HOME` — the root becomes `$HF_HOME/hub`). Per repo:
-
-```
-models--<org>--<name>/
-  blobs/<digest>                     # verified content, one file per blob
-  snapshots/<commit>/<file> → ../../blobs/<digest>   # symlink farm
-  refs/<revision>                    # e.g. refs/main = the commit sha
-```
-
-**Snapshot-per-commit semantics:** every revision you download gets its
-own `snapshots/<commit>` directory, and nothing ever deletes old ones.
-When upstream pushes a new commit and you `get` again, you gain a new
-snapshot; the previous one — and any blobs only it references, i.e. the
-old weights — stays on disk forever. That is why one repo can appear
-several times in a raw snapshot listing, and how tens of GB of dead
-blobs accumulate. `refs/main` always names the current (canonical)
-snapshot; the registry and every query verb resolve through it.
-
-## get — download (resumable, verified)
-
-```
-mlx-bun get <org/repo> [--revision main]
-mlx-bun get <substring>          # no "/" = registry query, re-gets the match
-```
-
-- Plain HTTPS resolve/CDN (no Xet), sequential files, `Range`-resume of
-  partial blobs (`<blob>.incomplete`). Auth: `HF_TOKEN` env, then the
-  token `hf auth login` writes. 401/403 answers get a "gated repo" hint.
-- Before the first payload request, mlx-bun totals the bytes still absent,
-  credits complete shared blobs and valid `.incomplete` prefixes, and checks
-  the target volume for that remainder plus a fixed 1 GiB safety reserve. An
-  impossible acquisition fails immediately with required/available GiB and a
-  rerun/resume instruction rather than filling the disk halfway through.
-- A substring argument (no `/`) resolves against already-downloaded
-  repos — `mlx-bun get 12B` refreshes the 12B to upstream's latest. An
-  unknown substring errors with a pointer to `mlx-bun ls <q>`.
-- Concurrent-writer safety: an `O_EXCL` lockfile (`<blob>.lock`, pid +
-  timestamp) serializes two processes downloading the same blob (e.g. a
-  foreground `get` racing the server's background auto-download). Locks
-  from dead pids, or older than ~1 h, are stolen; a live lock fails fast
-  with "another download … is in progress".
-- If the download created a new snapshot (upstream pushed), `get` says
-  so and estimates what `mlx-bun gc` would reclaim.
-
-### Verification story
-
-Every blob is checksummed **while streaming** (a resume re-hashes the
-existing prefix in chunks — no whole-file allocation):
-
-- **LFS files** (weights): sha256 must equal the API's current `lfs.sha256`
-  field (the older `lfs.oid` spelling is also accepted) — which is also the
-  blob's filename.
-- **Small files** (configs, tokenizer): git blob identity,
-  `sha1("blob <size>\0" + content)`, must equal the API's `blobId`.
-
-A mismatch deletes the partial (never resume corrupt bytes); a short
-read keeps the `.incomplete` for resume.
-
-## Direct-container artifacts (GLM-5.x)
-
-Models that load through the Colibri container loader (`loader: "colibri"`
-in `src/model/profile.ts`) skip `Weights.open` entirely: a header-only
-resource equation runs before any resident tensor or expert slab opens, and
-an impossible plan fails without committing memory. One validated artifact
-today; a GLM-5.3+ artifact joins as a new row in the roster above plus a
-subsection here once it has a declared profile.
-
-### GLM-5.2 on a 32 GB Mac
-
-The validated direct-container artifact is
-`mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` at revision
-`3cc8db99b1b13fc79325d987ba3c1c430766b3b8`. It is approximately 384 GB
-decimal / 357 GiB on disk, so choose the Hugging Face cache volume before the
-download. Fast internal NVMe is strongly preferred; an external volume also
-needs to sustain the expert-streaming workload after acquisition.
-
-```sh
-# Optional: choose a large fast volume before downloading.
-export HF_HUB_CACHE=/Volumes/FastSSD/huggingface/hub
-
-mlx-bun get mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp \
-  --revision 3cc8db99b1b13fc79325d987ba3c1c430766b3b8
-mlx-bun scan
-mlx-bun serve GLM-5.2-colibri \
-  --memory-budget 26.8435456 \
-  --context-length 4096 \
-  --max-tokens 128 \
-  --mtp on
-```
-
-`26.8435456` decimal GB is exactly 25 GiB. The validated plan reserves about
-19.9 GiB for the process and leaves the rest of the 32 GiB machine to macOS.
-Startup runs the exact header-only resource equation before opening resident
-weights or expert slabs. The server reports the resulting plan at
-`GET /stats` under `glm52`.
-
-If acquisition is interrupted, rerun the identical `get` command: mlx-bun
-rehashes each existing prefix in bounded chunks, sends a Range request for the
-remainder, and verifies the complete blob before publishing it. If the disk
-preflight refuses, free space or point `HF_HUB_CACHE` at another volume and
-rerun. No Python environment and no converted copy inside the mlx-bun checkout
-are involved.
-
-The artifact is an external MIT-licensed derivative of
-`zai-org/GLM-5.2-FP8`, converted with Colibri and extended with int8 MTP
-weights. Model weights are not bundled with mlx-bun; review the artifact model
-card before redistribution.
-
-## Declared model profiles
-
-Model loading resolves one immutable profile before weights are opened. A
-profile names four things: the external artifact fingerprint when one is
-known, its fidelity target, the engine capabilities it requires, and the
-composed loader/model-graph/generation-loop path.
-
-Exact artifact profiles outrank family profiles. The current exact declarations
-are:
-
-| profile | external artifact | fidelity | execution |
-| --- | --- | --- | --- |
-| `qwen3.8-27b-optiq-4bit` | `mlx-community/Qwen3.8-27B-OptiQ-4bit` revision `b04599de95d7a9bfbd7f208d347c0f10d9432a42` | L1: bit-exact mlx-lm | safetensors + dedicated Qwen3.5/3.8 graph + autoregressive loop |
-| `glm5.2-colibri-int4-int8-mtp` | `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` revision `3cc8db99b1b13fc79325d987ba3c1c430766b3b8` | L3: measured, no matching tier oracle | Colibri container + dedicated GLM-5.2 graph + autoregressive loop |
-
-The fingerprint is the Hugging Face repo plus immutable snapshot revision, so
-moving the cache does not change it. A mutable alias such as `snapshots/staged`
-or an arbitrary local directory is not treated as exact. Those artifacts use
-the dedicated model-family profile when one matches, then the universal dense
-profile when that architecture is supported. Unsupported architectures still
-refuse with the existing targeted/generic support list.
-
-Profiles select construction only. They do not enable, disable, or replace MTP,
-mixed-precision KV from `kv_config.json`, TurboQuant KV, adapters, grammar, or
-sampling. Those methods remain explicit flags/request fields or documented
-defaults. If an exact profile requires a capability the running engine does not
-have, loading refuses; it never downgrades to another model path.
-
-**Packed trellis artifacts.** A `quantization` entry with `mode: "trellis"`
-(TurboQuant Q2b, `mjriii/Qwen3.8-27B-q3-…-packed`) stores that module as a
-trellis-coded bit-stream + fp16 row scales and is served by the engine's own
-Metal kernels; stock mlx-lm cannot load it. Experimental k3/axis0 artifacts
-can store two-block interleaved codes as a 3D tensor; the loader identifies
-the layout from its shape and keeps one resident code copy. Format and kernels:
-`docs/design/turboquant.md` (Q campaign); fallback `MLX_BUN_TRELLIS=expand`:
-`server-config.md`.
-
-Fidelity labels mean L1 = bit-exact mlx-lm, L2 = bit-exact mlx-optiq, and L3 =
-measured evidence without a matching oracle. Model profiles do not imply that a
-quantized artifact retains the base model's benchmark score; quality evidence
-belongs to that exact artifact.
-
-## scan + the registry
-
-`mlx-bun scan` walks `models--*/snapshots/*` reading only `config.json`
-+ safetensors **headers** (never tensor bytes) into
-`~/.cache/mlx-bun/registry.sqlite`. The registry is a derived cache: on
-schema drift it rebuilds; rows whose snapshot dir vanished are reaped.
-
-Schema (table `models`, one row **per snapshot**, keyed on `path`):
-
-| column | meaning |
-| --- | --- |
-| `path` (PK) | `<hub>/models--…/snapshots/<commit>` |
-| `repo_id`, `model_type` | identity |
-| `param_count`, `num_layers`, `hidden_size`, `vocab_size` | shape |
-| `size_bytes` | language-model weights (sidecar excluded) |
-| `sidecar_bytes` | `optiq_vision.safetensors` (bf16 SigLIP sidecar) |
-| `experts_bytes` | `.experts.` tensor bytes (MoE fit math) |
-| `quant_bits`, `quant_group_size`, `quant_mode` | quantization |
-| `has_vision_sidecar` | sidecar file present |
-| `vision_config_type` | `config.json`'s `vision_config.model_type` when it names a vision tower (`*_vision`) |
-| `has_audio_config` | `config.json` declares an `audio_config` block |
-| `has_audio_tower` | sidecar header names `audio_tower.*` tensors |
-| `has_kv_config`, `has_tool_template`, `license` | capabilities/terms |
-| `scanned_at` | freshness (canonical tie-break) |
-
-**Vision capability** is `has_vision_sidecar OR vision_config_type =
-*unified_vision`: the SigLIP models (e2b/e4b/26B/31B,
-`gemma4_vision`) need the bf16 sidecar; the unified 12B
-(`gemma4_unified_vision`) declares its encoder-free tower in config.
-Presence of a `vision_config` key alone is **not** a signal — Qwen3.5
-nests a copy of its own text config there.
-
-**Audio capability** is `has_audio_config AND has_audio_tower`: both
-legs are required because the 12B OptiQ snapshot carries an
-`audio_config` but a stub sidecar whose only audio entry is
-`embed_audio.embedding_projection.weight` (no `audio_tower.*` Conformer
-tensors) — config presence alone would advertise audio that fails at
-request time. The check reads only the sidecar's safetensors header,
-never tensor bytes.
-
-## Query resolution rules (per verb)
-
-Every verb resolves fuzzy queries the same way, with revision collapsing
-so a stale snapshot never makes a repo "ambiguous":
-
-- **`ls [query]`** — substring match on repo id / model type; shows **one
-  row per repo** (the canonical revision). `--all-revisions` shows the
-  per-snapshot truth with the canonical marked `*`. Capabilities column
-  labels the support tier: `supported (targeted)` (dedicated forward,
-  L2/opt-in paths) vs `supported (generic)` (Tier-0 universal module).
-- **`fit` / `serve` / `train` / … (single-model verbs)** — `resolve()`:
-  the query must match exactly one **repo**; multiple cached revisions of
-  that repo collapse to the canonical snapshot (refs/main, else most
-  recently scanned). Drafter companions (`*_assistant`) never count.
-- **`get <substring>`** — same `resolve()`, then re-downloads that repo id.
-
-Canonical = the snapshot `refs/main` points at; if no `refs/main`, the
-most recently scanned snapshot (stable path tie-break).
-
-## gc — reclaim superseded snapshots + dead blobs
-
-```
-mlx-bun gc              # plan + per-repo reclaim summary, deletes nothing
-mlx-bun gc --yes        # actually delete (destructive)
-mlx-bun gc --dry-run    # never delete, even with --yes
-mlx-bun gc --yes --force  # also prune warned snapshots (see below)
-```
-
-Per repo: keep every snapshot a `refs/*` file points at, delete the
-rest, then delete blobs no surviving snapshot symlinks to. Prints
-keep/prune/skip counts and reclaimable bytes per repo.
-
-Safety rails:
-
-- **Deletion requires `--yes`.** The default run is a report.
-- **File-loss tripwire:** a superseded snapshot containing files the
-  kept snapshots *lack* is skipped with a warning naming those files —
-  deleting it would delete the machine's only copy (live example: a
-  stale gemma-4-12B snapshot holds an `optiq_vision.safetensors` that
-  the canonical revision dropped). `--force` overrides, deliberately.
-- Repos with no usable refs (no `refs/`, or refs naming a missing
-  snapshot) are left untouched.
-- `.incomplete` / `.lock` resume artifacts in `blobs/` are never touched.
-
-After a real deletion, gc re-scans so the registry drops the reaped
-rows immediately.
-
-
-Non-Qwen grouped TurboQuant drafting, including assistant donor attention, uses
-the shared cache interfaces. See [composition coverage](../design/batching.md#turboquant-donor-attention-and-grouped-draft-composition).