mlx-bun 0.0.7 → 0.0.9

package/docs/planning/pi-integration-review.md

@@ -0,0 +1,178 @@
+# pi integration review — internal embeds + `harness pi` (2026-07-01)
+
+Read-only audit of the pi surface: `mlx-bun pi` (src/pi-terminal.ts, in-process
+InteractiveMode), the shared provider wiring (src/pi-provider.ts), the web
+embed (src/pi-web.ts), `mlx-bun harness pi` (src/harness-pi.ts), and the docs
+that describe them. Every claim below was verified in the current source
+(post-8ee691d, working tree of 2026-07-01); the localhost/loopback behavior was
+verified empirically with Bun on this machine.
+
+Stale-memory corrections first (priors that are NO LONGER true):
+- `harness pi` does **not** write `~/.pi/agent/models.json` — it generates a
+  self-contained extension at `~/.pi/agent/extensions/mlx-bun-provider.ts`
+  that calls `pi.registerProvider()` with live `/v1/models` discovery.
+- Tool-call parsing is **not** Gemma-only anymore: src/tool-call.ts also parses
+  OpenAI-JSON `<tool_call>`, Qwen `<function=name><parameter=…>`, and MiniCPM5
+  `<function name=…><param name=…>` from decoded text (buffered-text mode).
+- The old `by_bits`/bitsMap and double-BOS issues are fixed and stayed fixed.
+
+## Ranked fix list
+
+### 1. HIGH — Web chat advertises memory tools it filtered out
+`src/pi-web.ts:568` passes `tools: [...WELCOME_TOOLS]` (= `["read","web_search"]`,
+line 68) to `createAgentSessionFromServices`, but `src/pi-web.ts:534` appends
+`surface.memoryHint` to the system prompt and `surface.skillPaths` includes the
+bundled `memory` skill (surface built at :528 with memory auto-detect). pi
+treats the `tools` list as an allowlist, so when a vault exists the prompt and
+skill instruct the model to call `memory_resolve` / `memory_read` /
+`memory_section` — tools pi filtered out of the session. Every such call fails.
+The intent that they be reachable is visible at `src/pi-web.ts:57`
+(`READ_ONLY_TOOLS` already auto-allows `MEMORY_TOOL_NAMES`/`REFERENCE_TOOL_NAMES`,
+imported at :47). The terminal embed does this correctly
+(`src/pi-terminal.ts:157` uses `surface.tools`).
+**Fix (pi-web.ts:568):**
+`tools: [...WELCOME_TOOLS, ...(surface.memoryEnabled ? [...MEMORY_TOOL_NAMES, ...REFERENCE_TOOL_NAMES] : [])]`
+— or, if the 1B welcome assistant must stay two-tool, build the surface with
+`memory: "off"` so the hint and skill disappear too. Add a pi-web test either
+way (tests/pi-web.test.ts has no memory coverage today).
+
+### 2. HIGH — `harness pi` maps EVERY /v1/models row to id `"local"` (duplicate models)
+`/v1/models` now returns the served model FIRST **plus every other servable
+registry model** (src/server.ts:1479–1499). Both consumers still assume the old
+single-entry shape:
+- `src/harness-pi.ts:76–81` (`fetchServerModels`) maps all rows;
+- the generated extension (`renderPiExtension`, src/harness-pi.ts:116–125) and
+  the baked fallback (`modelEntry`, :140–151) register one pi model per row,
+  **all with id `local`** — N duplicate ids, N−1 of which name models the
+  single-model server is not actually serving.
+Side effects: `installPiExtension`'s `bakedModels` and the CLI's
+"server live · …" line print the whole registry; `src/cli.ts:1615`
+("reusing running server at … (…)") lists every downloaded model as if served.
+`tests/harness-pi.test.ts` stubs a single-row /v1/models, so this is untested.
+**Fix:** in `fetchServerModels`, keep only the served entry — the first row is
+the served model and is the only one carrying the capability extras
+(`context_window`, `reasoning`); e.g. `body.data?.[0]` (or filter rows that
+have `context_window`). Same filter inside the generated extension's live
+fetch. Update cli.ts:1615/1668 to label `models[0]` as served. Extend the
+harness test with a multi-row stub.
+
+### 3. HIGH — Tier-0 generics (and plain Qwen3 chat) get Gemma's token sentinels
+`toolStreamMode` (src/server.ts:1175–1183) returns `"gemma-sentinel"` for every
+model that isn't MiniCPM5/Qwen3.5 — including the new universal-dense generics
+(llama, qwen2, qwen3, gemma1/2, phi3, olmo2, glm4, granite, starcoder2,
+smollm3 — src/model/universal/archs.ts, served via src/model/factory.ts:42–45)
+and plain Qwen3. In that mode `ToolAwareStream.push` (src/server.ts:500–509)
+unconditionally interprets token ids 48/49/100/101 as tool/channel sentinels.
+Those ids are Gemma-4 special tokens (src/tool-call.ts:9–23); on any other
+tokenizer they're ordinary low-id vocab entries, so a generic model emitting
+one silently reroutes all subsequent output into a phantom tool segment or
+"reasoning". It also means generics never reach `parseGeneratedToolCalls`
+(buffered-text), so tool calling is broken for them — and pi always sends
+tools, and `mlx-bun pi -q <generic>` is a supported path now.
+**Fix (src/server.ts:1180–1182):** make gemma-sentinel the *family-gated*
+branch, not the default:
+```ts
+if (ctx.model.config.modelType.startsWith("gemma4")) return "gemma-sentinel";
+return tools?.length ? "buffered-text" : "plain";
+```
+(`ctx.template.thinkingFormat === "gemma-channel"` is an alternative key, but
+DiffusionGemma forces it null while still being gemma-family, so prefer
+modelType.) Buffered-text already handles the OpenAI-JSON `<tool_call>` shape
+most generic templates emit; models whose format isn't covered fail soft
+(markup stays in content) instead of corrupting the stream.
+
+### 4. MEDIUM — vision capability never reaches the pi terminal or external pi
+`/v1/models` doesn't advertise vision (src/server.ts:1479–1485: only
+`context_window` + `reasoning`), `RunEmbeddedPiOptions` has no `vision` field
+and `src/cli.ts:1670` passes none, so `mlx-bun pi` on gemma-4-e4b — the
+*default* model, vision-capable — registers `input: ["text"]`
+(src/pi-provider.ts:100) and images are unusable in the TUI. The harness
+extension is hardcoded `input: ["text"]` too (src/harness-pi.ts:121, :146).
+Only pi-web wires it (src/pi-web.ts:491–495, from `ctx.vision || ctx.loadVision`
+via createServer). The `input` flag MUST match the real model (pi-ai
+content-parts vs string serialization — pi-provider.ts:44–49), so this needs
+the real signal, not a default.
+**Fix:** add `vision: !!(ctx.vision || ctx.loadVision)` to the served entry in
+/v1/models; carry it through `ServerModel`/`fetchServerModels`,
+`renderPiExtension`/`modelEntry` (`input: m.vision ? ["text","image"] : ["text"]`),
+`RunEmbeddedPiOptions`, and the cli `pi` case.
+
+### 5. MEDIUM — harness extension drifted from buildPiProvider on compat flags
+The internal provider sets `compat: { supportsDeveloperRole: false, … }`
+(src/pi-provider.ts:89–92); the generated extension sets only `thinkingFormat`
+for reasoning models (src/harness-pi.ts:120, :145). Currently masked because
+the server remaps `developer` → `system` (src/server.ts:743–760), but the two
+paths are documented as "kept aligned" (pi-provider.ts:20–22, harness-pi.ts
+header) and have drifted. **Fix:** emit the same
+`supportsDeveloperRole: false` block in `renderPiExtension`/`modelEntry`.
+
+### 6. MEDIUM — `memory_status` tells the built-in agent synthesis doesn't exist
+`src/memory/tools.ts:571–572` hardcodes
+`"last synthesis: not available yet (M1 synthesis is stubbed)"` and
+`"synthesis: M1 stub (manual/nightly writes are not implemented yet)"`.
+Synthesis is live: `mlx-bun memory synthesize` runs the full DAG
+(src/cli.ts:2508ff) and the nightly launchd job runs it
+(src/memory/schedule.ts:66–72). The pi session will confidently tell the user
+the feature doesn't exist. **Fix:** drop the stub lines; report real state
+(e.g. last synthesis run from the synthesis DB / vault git log, and
+"run: mlx-bun memory synthesize").
+
+### 7. MEDIUM — docs/reference/memory.md contradicts itself
+Line 9: "Synthesis is scaffolded but still a no-op M1 stub" vs line 41
+("**`memory synthesize` writes to your vault.** It runs the full local
+synthesis DAG …") and line 77 ("Synthesis is live today"). Fix line 9 (and the
+line-53 "last-synthesis placeholder" wording once #6 lands).
+
+### 8. LOW — server-api.md pi section omits `harness pi`
+docs/reference/server-api.md:462–477 documents only the manual
+`~/.pi/agent/models.json` route (which also bakes a concrete model id — the
+staleness problem the `local` id exists to solve). Add: the supported
+one-command path is `mlx-bun harness pi` (extension + live discovery +
+`--remove` undo), manual models.json as the fallback.
+
+### 9. LOW — three base-URL spellings for one loopback server
+pi-web uses `http://127.0.0.1:${port}/v1` (src/pi-web.ts:487); the cli `pi`
+probe/provider uses `http://localhost:${port}/v1` (src/cli.ts:1610, also the
+serve preflight at :1215); harness `DEFAULT_BASE_URL` is
+`http://localhost:8080/v1` (src/harness-pi.ts:25). Functionally coherent today
+— verified empirically: Bun's fetch to `localhost` reaches both a
+127.0.0.1-bound and a ::1-bound server — and the port sweep from 8ee691d is
+clean (no live `8090` outside historical artifacts). But the harness extension
+runs under the *user's pi* (node/jiti, not Bun), so standardize on
+`127.0.0.1` (the actual default bind) everywhere to be resolver-proof.
+
+### 10. LOW — `mlx-bun pi` arg edges (flag-strip itself is clean)
+Verified complete: `OURS_VAL`/`OURS_BOOL` (src/cli.ts:1575–1582) cover every
+flag read by `serverRuntimeFlags`/`applyDecodeRoute`/`mountStartupAdapter`/
+`resolveModelAuto` (cross-checked flag-by-flag); `opt()` only supports
+space-separated values repo-wide so there's no `--flag=value` bypass; the
+"ignoring …" notice on server reuse works; `harness pi` passes nothing through
+to pi (no leak path). Two edges remain:
+- `parsePiArgs` (src/pi-terminal.ts:237–242) drops an unknown flag but keeps
+  its **value** as message text: `mlx-bun pi --resume abc123` starts a chat
+  with the message "abc123". Either swallow the next token for unknown
+  `--flags` or warn that the flag was ignored.
+- `RunEmbeddedPiOptions.verbose` (src/pi-terminal.ts:97) is never settable —
+  parsePiArgs doesn't recognize `--verbose`. Wire it or delete it.
+
+## Verified-good (no action)
+- **Compat flags per model**: `buildPiProvider` derives `reasoning` →
+  `thinkingFormat: "qwen-chat-template"` from `ctx.template.supportsThinking`,
+  which is detected generically from the template source (`enable_thinking` +
+  `<think>`/`<|channel>`, src/chat-template.ts:143–159) — covers Gemma,
+  MiniCPM5, Qwen3.5, and Tier-0 generics with think-tag templates alike; the
+  server honors `chat_template_kwargs.enable_thinking` and `reasoning_effort`
+  with one precedence function (src/server.ts:1101–1108).
+- **Wire contract**: pi's OpenAI `tool_calls` shape is produced for both the
+  sentinel and buffered-text parsers (src/server.ts:548–575, 1916–1923,
+  1989–2003), `finish_reason: "tool_calls"` included; `developer` role remapped.
+- **Isolation**: both embeds use in-memory auth/registry (no ~/.pi cross-talk),
+  own agent dirs (`~/.mlx-bun/pi`, `~/.mlx-bun/pi-sessions`), `noExtensions/
+  noSkills/noPromptTemplates/noThemes`, curated skills only.
+- **Server ownership**: `pi`-started servers are `owner: "pi-session"` and the
+  reuse path warns when attaching to another session's server (cli.ts:1624–1631).
+- **Memory read tools vs current vault code**: all imports
+  (vault/query/article/schedule) exist and match; terminal surface exposes them
+  correctly; FIND→READ contract intact. Only the status text (#6) is stale.
+- **contextWindow fallbacks**: missing `max_position_embeddings` degrades to
+  32768 at every consumer (`?? 32768` / `?? DEFAULT_CONTEXT_WINDOW`).

package/docs/planning/release-notes-v0.0.9.md

@@ -0,0 +1,122 @@
+# v0.0.9 release notes (DRAFT — Josh edits + publishes)
+
+Everything since v0.0.8 (2026-06-24). The big themes: the mlx-lm drop-in
+surface is now materially complete, ORPO is paper-faithful, mlx-bun runs its
+first generically-supported architectures at verified bit-exact parity, and an
+adversarial review swept the whole engine — fixing 12 confirmed bugs and
+overturning a load-bearing performance assumption.
+
+## Drop-in compatibility with mlx_lm.server (the headline)
+
+- **Endpoints:** `POST /v1/completions` (raw text completion, stream +
+  non-stream), `GET /health` (byte-exact body), `GET /v1/models` now lists all
+  registry models (+ `/v1/models/<id>`).
+- **Request fields:** `min_p`, `xtc_probability`/`xtc_threshold`, `logit_bias`,
+  `presence_penalty`/`frequency_penalty` (+ `*_context_size` windows),
+  `logprobs`/`top_logprobs` with mlx-lm's EXACT semantics (same distribution,
+  same response shape, same [0,11] validation). All L1-faithful ports of
+  `sample_utils.py`.
+- **Flags/defaults:** port 8080 + loopback host (matching mlx_lm.server;
+  `--host 0.0.0.0` = LAN opt-in), `--temp` alias, `--max-tokens`,
+  serve `--adapter <dir>` mounts at startup.
+- **New CLI verbs:** `fuse` (fold LoRA into base — mlx-lm math, untouched
+  modules bit-identical), `convert` (wraps the mixed-precision quantizer,
+  `--target-bpw` exposed), `perplexity` (mlx_lm.perplexity methodology),
+  `upload` (native HF push).
+- Deliberately not ported: `role_mapping` (unreachable — every supported model
+  ships a chat template). Remaining: `cache_prompt`, `evaluate` (planned as an
+  lm-eval shim), `--draft-model`, awq/dwq/gptq (plan:
+  docs/design/mlx-lm-tool-parity-plan.md).
+
+## ORPO: paper-faithful by default
+
+- **`sft_scope: full | response`** (default `full`): the chosen-NLL is now the
+  token-mean CE over the full prompt+response — matching the ORPO paper, TRL,
+  and xfactlab/orpo — implemented across every path (naive/chunked/fused/
+  flash-CCE/prefix-shared/segmented). `response` reproduces old runs
+  BIT-EXACTLY (regression-pinned). The odds-ratio terms stay response-only in
+  both modes (matches TRL). CLI: `mlx-bun train --sft-scope`.
+- The rest of the ORPO stack was adversarially verified correct against
+  primary sources (odds-ratio math, gradients vs finite differences,
+  prefix-sharing gradient flow, tokenization/label-shift).
+
+## Generic model support (Tier-0)
+
+- **New:** `src/model/universal/` — a config-driven UniversalDense module with
+  explicit descriptors for llama, qwen2, qwen3, gemma, gemma2, phi3, olmo2,
+  glm4, granite, starcoder2, smollm3 (+ mlx-lm's remapping: mistral etc.).
+  bf16 (unquantized) checkpoints now load. Rope factory (llama3/yarn/longrope)
+  verified bit-exact vs the oracle.
+- **First three archs verified at L1 parity** (bit-exact per-step logits):
+  Llama-3.2-1B-Instruct-4bit, Qwen2.5-0.5B-Instruct-4bit, gemma-2-2b-it-4bit.
+- Dispatch ladder: dedicated → generated → generic → reject-with-helpful-error.
+  Targeted models keep their optimized paths; a model can graduate.
+  Design: docs/design/generic-model-support.md.
+
+## Fixed (the adversarial-review wave)
+
+- **`--l2` tier contract restored:** the L2 preset was enabling an
+  envelope-gated perf kernel (not bit-exact vs optiq). Bare `--l2` is now
+  genuinely optiq-bit-exact; the perf kernel is `--l3`/explicit opt-in.
+  `generate --l2/--l3` no longer silently degrades to L1.
+- Batched same-millisecond seed collision (identical concurrent completions);
+  streaming per-layer KV-quant conversion (the missing half of optiq's
+  tight-RAM fix); segmented compiled-decode mid-step failure KV corruption;
+  `MLX_BUN_TRAIN_ATTN=flash` now refused on Gemma (unrevalidated crash path);
+  eval-loss error swallowing; wrong tokens_per_sec metric; silent steel-kernel
+  fallbacks now warn; flash-attention tests extended (D=256, sliding-window,
+  non-tile T).
+- Memory (the Dreaming): batching default flipped to serial (measured 1.7–1.9×
+  faster for the real workload); `memory status` truth; third-person voice fix
+  merged.
+- CLI audit (14 findings): `fit --ctx` help matched code, `embed` auto-pick
+  fixed, pi flag leakage into first message fixed, `--l1/2/3` + `generate` +
+  `train-watch` documented, `setup` is a real alias, doc lies corrected.
+
+## Performance truth (research, groundwork for the next release)
+
+- The "decode is at the memory floor" claim was **overturned** with measured
+  rooflines: only the 12B is at the bandwidth wall (~92%); CPM5/e4b/26B sit at
+  58–70% with the host-side graph build as the top recoverable term. Ranked
+  fix plan in-repo; wins land next release.
+- The batching engine was audited (numerics solid, engine naive) — hotfixes +
+  containment landing; plan: docs/design/batching-v2-plan.md.
+- Registry duplicates root-caused (snapshot-per-commit accumulation);
+  dedupe + `gc` landing.
+
+## Infra
+
+- CI gate added (typecheck + model-free tests on push/PR) — previously nothing
+  ran on push.
+- Repo hygiene: scratch artifacts untracked, megakernel research relocated,
+  STATUS.md rewritten as a true current-state front door, doc map regenerated.
+
+## Late additions (landed after the first draft)
+
+- **Batching engine v2 (steps 1–3):** Qwen3.5/SSM models no longer fail under
+  `--batch` (capability gate + serial drain); per-row failure containment; no
+  serial-lane starvation; pipelined batched decode + sane cache cadence; the
+  huge admit transient fixed; all GPU work under one lock.
+- **Model management:** `ls` deduplicates to one row per repo (snapshot
+  history via `ls --all-revisions`); new `mlx-bun gc` reclaims superseded
+  snapshots (24.7 GB found on the dev machine) with a safety guard for
+  unique files; `get` accepts substring queries; download lockfile, streamed
+  resume verification, gated-repo auth hints; vision capability detected from
+  config (the unified-vision 12B now shows `vision`). New doc:
+  docs/reference/models.md.
+- **Pi integration:** web-chat memory tools actually callable; `harness pi`
+  registers exactly the served model; tool-calling fixed for Tier-0 generic
+  models (Gemma sentinels no longer applied to non-Gemma tokenizers); vision
+  advertised to pi; `mlx-bun pi` no longer leaks unknown flags into the first
+  message.
+- **Website + README:** six-goals restructure, drop-in guide, memory guide,
+  the lab page, full CLI reference from a tracked source; deploy now triggers
+  on reference-doc edits. NOTE the release sequence: push and publish in the
+  same sitting — the site deploys from the push.
+- **Test infra:** batched-serving goldens are machine-keyed (the M1-Max
+  "failures" were M4-Pro fixtures; all code exonerated — mlx-lm reproduces
+  mlx-bun token-for-token per machine); CI gate (typecheck + model-free tests).
+- **Research (the lab):** decode-roofline re-measurement overturned the "at
+  the floor" assumption (docs/investigations/decode-roofline-lookagain.md);
+  curve-sampler distinctness theorem + witness numbers + preregistered
+  protocol (docs/planning/curve-sampler-research-plan.md).

package/docs/planning/web-ui-pass-plan.md

@@ -0,0 +1,213 @@
+# Web UI pass — ranked fix list (2026-07-01 audit)
+
+Read-only audit of every web surface against the current server contract:
+`src/web/app.html` (unified SPA: chat / quantize / finetune / dataset /
+status+library / routes), `src/assets/curve-designer.html` (/curves),
+`src/pi-web.ts` (WS chat bridge), `src/server.ts` routes, plus the dead
+`src/status-page.html`. Method: diffed every fetch/WS consumer in the pages
+against the actual handler shapes in `server.ts`, `src/jobs/types.ts`,
+`src/train/{job,trainer}.ts`, `src/quantize/job.ts`, `src/download.ts`,
+`src/registry.ts`, `src/model/support.ts`.
+
+**Live click-through was skipped deliberately**: GPU-loading test suites
+(`bun test tests/server.test.ts …`, loads MiniCPM5) were running for the whole
+audit window, and starting a second model server would have contended with
+them. Every finding below is source-verified on both sides of the contract.
+
+Port sweep: no stale `8090` or hardcoded URLs remain in any html/js.
+`curve-designer.html:92` falls back to `http://localhost:8080` (matches the
+new default) and uses `location.origin` when served at `/curves`. ✓
+
+---
+
+## WRONG (bugs — fix first)
+
+1. **Fine-tune live stats never show LR or tok/s — metric field-name mismatch.**
+   `src/web/app.html:2317-2318` reads `e.lr` / `e.tps`, but the trainer emits
+   `learning_rate` / `tokens_per_sec` (`src/train/trainer.ts:559-560`, same at
+   :724-727 and :966-969; contract comment in `src/jobs/types.ts:23`). The
+   "learning rate" and "tok/s" chart stats stay `—` for the entire run.
+   Also update the protocol doc-comment at `app.html:1089` (`lr?`, `tps?`).
+   **Fix:** read `e.learning_rate` and `e.tokens_per_sec`.
+
+2. **Chat queue bar renders arrays as strings and empty tags.**
+   `pi-web.ts:183` sends `queue_update` with `steering: readonly string[]` and
+   `followUp: readonly string[]` (arrays). `app.html:1683-1684` does
+   `if (q.steering) … esc(String(q.steering))` — an empty array is truthy, so
+   every `queue_update` renders empty `steering:`/`queued:` pills, and multiple
+   queued messages render comma-joined.
+   **Fix:** guard with `.length` and render `q.followUp.map(...)` (or
+   `q.steering[q.steering.length-1]`).
+
+3. **Library table loads exactly once per app lifetime — new quants never appear.**
+   `app.html:2603` gates `loadLibrary()`/`loadFit()` behind `fitLoaded`, which
+   is never reset. The server explicitly invalidates its 30 s `/library` cache
+   when a quantize job completes so "the next poll rescans and shows it"
+   (`src/server.ts:2510-2513`) — but the UI never re-polls. A model quantized,
+   downloaded, or gc'd mid-session never shows until a full page reload.
+   **Fix:** refresh the library on its own interval (e.g. every 15 s inside
+   `tick()`), and force one refresh when a quantize job reaches `done`.
+
+4. **Adapter mount failure toasts "[object Object]".**
+   `POST /v1/adapters` errors use the OpenAI envelope `{error:{message}}`
+   (`src/server.ts:1600`), and the `api()` helper (`app.html:1080`) keeps
+   `data.error` as that object; `onSelectAdapter` (`app.html:1908-1909`) then
+   string-concatenates it. Any mount failure reads
+   `adapter: [object Object]`.
+   **Fix:** in `api()`, unwrap
+   `data.error && typeof data.error === "object" ? data.error.message : …`.
+
+5. **Quantize inspect result parsing targets a response shape the server never sends.**
+   `inspectModel` returns `support: boolean` (`src/quantize/job.ts:101`), but
+   `app.html:2135-2140` checks `d.support === "supported" | "ok" | "untested"`
+   (all dead branches) and, when `support` is false, prints
+   `esc(d.support || "unknown architecture")` → always "unknown architecture"
+   even when `arch` was resolved. `size_gb` is also rendered raw (unrounded
+   float, and `0` when a direct path bypasses the registry).
+   **Fix:** treat `support` as boolean; show `arch` + "not quantizable" on
+   false; `(+d.size_gb).toFixed(2)` and hide when 0.
+
+6. **Stale vision claim in the chat attach path.**
+   `app.html:1848`: "This model can't see images — switch to the **12B** for
+   vision." SigLIP landed: e2b/e4b/26B/31B are vision-capable too
+   (`src/registry.ts:213-219` `visionCapable`), so the 12B-only advice is
+   wrong.
+   **Fix:** model-agnostic copy: "This model can't see images — serve a
+   vision-capable model (Gemma vision builds show a *vision* tag in the
+   Library)."
+
+---
+
+## MISSING (current capabilities the UI doesn't expose)
+
+7. **`sft_scope` absent from the fine-tune form.**
+   The finetune job accepts `sft_scope: "full" | "response"` and validates it
+   at submit (`src/train/job.ts:54,74-76`); default is now `"full"`
+   (paper/TRL-faithful, commit d32fe32). The ORPO panel
+   (`app.html:791-800`) has λ + schedule but no scope control, and
+   `collectHP()` (`app.html:2276-2289`) never sends it.
+   **Fix:** add a segmented control (full · response) to `#f-orpo-extra` with
+   a hint ("full = paper/TRL chosen-NLL over the whole sequence; response =
+   pre-2026-07 response-only"), include `sft_scope` in `collectHP()` when
+   method === "orpo".
+
+8. **`min_p` / `xtc_probability` / `xtc_threshold` / `logprobs` not in any request-builder UI.**
+   Server supports them per-request (`src/server.ts:280-304`, wired at
+   :1138-1141). The chat sampling popover (`app.html:592-613`) only has
+   temperature/top_p/top_k — and the whole chain is width-limited: the
+   `set_sampling` WS frame (`pi-web.ts:160`), `SamplingOverrides`
+   (`pi-web.ts:408-412`), and `injectSampling` (`pi-web.ts:420-440`) all only
+   carry those three fields.
+   **Fix (scoped):** add `min_p` (slider 0–0.5) end-to-end: popover row →
+   `set_sampling` frame → `SamplingOverrides.min_p` → `injectSampling`. XTC and
+   logprobs are power-user/API features; document rather than add sliders.
+
+9. **No gc/reclaim story in Library or Downloads.**
+   `mlx-bun gc` exists (CLI `src/cli.ts:1100`; planner
+   `src/registry.ts:375-455` `planGc`/`executeGc`) and the download path
+   prints "gc reclaims ~X GB" (`cli.ts:1015-1018`) — but the web Library shows
+   superseded revisions' disk cost with no affordance. **Needs a new server
+   endpoint first** (e.g. `GET /api/gc/plan`, `POST /api/gc/execute {yes}`),
+   then a Library footer line: "N superseded snapshots · reclaim X GB —
+   [Clean up]". (Separate from the listCanonical wiring below, but pairs
+   naturally with it: dedupe hides the stale revisions, gc deletes them.)
+
+10. **Other accepted-but-hidden finetune/quantize knobs** (lower priority):
+    `grad_accumulation_steps`, `warm_start_adapter`, adapter naming (submit
+    always writes `adapter-<timestamp>`, `src/server.ts:2529-2530`) on the
+    finetune form; `reference`, `calibration_mix`, `n_calibration`
+    (`src/server.ts:2460-2461`) on the mixed-bpw quantize step. Add an
+    "Advanced" disclosure rather than more top-level fields.
+
+---
+
+## BLOCKED on the queued server.ts listCanonical / visionCapable / tier wiring
+
+The functions exist and are tested (`Registry.listCanonical`
+`src/registry.ts:181`, `visionCapable` :217, `supportTier`
+`src/model/support.ts:53`) but the server still serves the old view. UI
+consumption points to touch when the wiring lands:
+
+11. **Revision duplicates in /library and /v1/models.**
+    `src/server.ts:1263` (`/library`) and :1491 (`/v1/models`) iterate
+    `reg.list()` — one row per cached snapshot revision. Both should use
+    `reg.listCanonical()`. UI needs no change; the Library table and every
+    /v1/models-driven picker (external pi, CLI) currently show dupes.
+
+12. **`vision` flag understates capability.**
+    `src/server.ts:1278` sends `vision: m.hasVisionSidecar`, which misses the
+    12B's encoder-free `gemma4_unified_vision` (config-declared, no sidecar
+    file). Should be `visionCapable(m)`. UI consumption at `app.html:2663`
+    then needs no change.
+
+13. **`supported` is a boolean; tiers are invisible.**
+    `src/server.ts:1264` sends `supported: isSupportedModelRecord(...)`.
+    Generic Tier-0 models (universal-dense descriptors) now count as
+    supported — correct — but the Library can't distinguish "targeted
+    (L2/L3 paths)" from "generic (L1 monolith only)". When the server sends
+    `tier: "targeted" | "generic" | null` (from `supportTier`), update:
+    - `app.html:2659` status cell: keep `unsupported (model_type)` for null;
+      add a dim "generic" badge for Tier-0 rows.
+    - Keep `supported` as a boolean alongside `tier` for one release so the
+      UI change can land independently.
+
+---
+
+## CONFUSING / polish
+
+14. **Sampling popover "recommended" values are hardcoded to MiniCPM5.**
+    `app.html:1824-1825` (`SAMP_REC {topP:.95, topK:0}`, `recTemp()` 0.9/0.7)
+    claims to "mirror server toOptions defaultTemp", but the server resolves
+    defaults from each model's `generation_config.json`
+    (`src/server.ts:1120-1132`). For any other served model the dimmed "auto"
+    readouts lie. Fix: surface `genDefaults` (e.g. on the `ready` frame or
+    `/v1/models` extras) and seed the popover from it.
+
+15. **Adapter dropdown goes stale after training.**
+    `refreshAdapters()` runs only at chat init and on `ready` frames
+    (`app.html:1789-1792, 2037`). An adapter trained in the Fine-tune tab
+    doesn't appear until reconnect/reload. Refresh on finetune `done` and on
+    chat `enter()`. Also escape the interpolation at `app.html:1897-1898`
+    (`a.id`/`a.path` go into HTML unescaped).
+
+16. **Dead file: `src/status-page.html` (497 lines) + `scripts/status-page-stub.ts`.**
+    Nothing imports it since the SPA absorbed status (only
+    `scripts/status-page-stub.ts` references it). Delete both, or move the
+    stub note into the SPA docs.
+
+17. **Routes tab breaks in the compiled binary.**
+    `#/routes` iframes `/dag`, which `readFileSync`s
+    `../docs/dag/training-inference-map.html` (`src/server.ts:1231`) — absent
+    outside the repo checkout, so the tab shows a bare 404 string. Either
+    embed the artifact like app.html, or hide the tab when `/dag` 404s.
+
+18. **CLI banner vs landing route.** `src/cli.ts:64` says "status page at
+    http://localhost:8080/", but `/` lands on `#/chat`. Say "web app" or point
+    at `/#/status`.
+
+19. **Curves nav tab never highlights.** It's a full-page `href="/curves"`
+    (`app.html:520`), outside the hash router, so the active-tab logic
+    (`app.html:1390`) can't mark it. Cosmetic; acceptable, but a
+    `aria-current` on the curve page's back-link side would be tidier.
+
+20. **Library "too big" rows hide the useful number.** `app.html:2665-2666`
+    shows max context/decode only when `a.fits` (fit solved at 8192); a model
+    that fits at a smaller context shows `—`. Show `max_safe_context`
+    whenever it's > 0.
+
+21. **Downloads visibility is per-process.** `/downloads` reads a
+    process-local tracker (`src/download.ts:276-279`) — a `mlx-bun get` in
+    another terminal is invisible to the nav pill/status card. Note in the UI
+    copy or leave; fixing needs a shared store.
+
+---
+
+## Suggested landing order
+
+1. Bugs #1–#6 (one UI-only PR; no server changes).
+2. #7 sft_scope + #15 adapter refresh (small, independent).
+3. The queued server wiring (#11–#13) as its own PR: `listCanonical` +
+   `visionCapable` + `tier` in `/library` and `/v1/models`, then the
+   one-line UI badge.
+4. #8 min_p end-to-end, #9 gc endpoint + Library affordance.
+5. Cleanup: #16 dead files, #17 routes tab, #14 genDefaults surfacing.