mlx-bun 0.0.13 → 0.3.0

package/docs/design/bucketing-stage.md

@@ -1,197 +0,0 @@
-# Bucketing / Classify stage — local build plan
-
-Status: **design**, 2026-06-26. Companion to [memory-synthesis.md](./memory-synthesis.md).
-Grounded in a research+critique workflow over the real Lucien classify + the local
-foundation; re-centered on the governing principle below.
-
-## Governing principle — capacity by decomposition
-
-**A small local model's effective capacity scales inversely with task size. You don't
-make the model smarter; you make each task narrower.** Every stage is a *ladder of
-atomic, single-purpose tasks*, each one a single clear question. The full pipeline:
-
-```
-"what buckets should we have?"        ← taxonomy (cold-start)
-"which buckets need this chunk?"      ← classify / assignment
-"which section needs this info?"      ← section routing
-"integrate THIS chunk into THIS section"  ← synthesis patch (most atomic)
-```
-
-…and each rung decomposes *further* into the smallest decisive question. We never ask
-the local model to "assign a chunk among 12 buckets and emit JSON" — that is still too
-big. We ask **"does this chunk belong in bucket B? yes/no,"** one bucket at a time.
-
-**Retrieval (embeddings, lexical) exists only to NARROW the candidate set so the model
-makes the fewest, smallest, clearest calls. It never decides — the model decides, one
-tiny question at a time.** This is the production home for the embedding work: a
-candidate-narrower, not a classifier.
-
-**Why brute-force decomposition is affordable: the model is LOCAL, so inference is ~free.**
-A cloud model forces you to minimize calls (which is exactly why Lucien made one big
-"reason over everything" call). A local model flips the economics — you can run **M×N tiny
-binary checks** ("does chunk *i* belong in bucket *j*?" / "should chunk *i* update section
-*j*?") where a cloud model would have to make one expensive combined decision. We trade one
-big unreliable call for many tiny reliable ones. Local-ness is what makes radical
-decomposition viable; retrieval just keeps M×N from getting silly.
-
-This is adapting + improving Lucien, not duplicating it. Lucien put all 379 buckets in
-Opus's context and let it reason globally in one shot — a capability a 1B model does not
-have. We replace "one global decision" with "many tiny local decisions."
-
-## What we keep from Lucien
-
-- Classify on chunk **labels** (~49-char topic strings), not text or vectors.
-- The `Bucketing.md` policy governs behavior, inlined into the task prompt — editing it
-  re-tunes routing with no code change.
-- **Many-to-many** (a chunk → 0..3 buckets; empty is legal = the substance filter).
-- **Create buckets by judgment** when nothing fits and the topic is substantive.
-- **Existing-buckets-win** (re-scoped to global below — fix E).
-- Idempotent set-difference resumption (`NOT IN chunk_buckets`).
-
-## The bucketing stage, decomposed
-
-### "which buckets need this chunk?" — assignment
-
-For one chunk (its label):
-1. **Narrow (no model):** embedding shortlist — cosine(label, bucket centroid) → top-K
-   candidates; PLUS a **lexical floor** (any bucket whose name token-overlaps the label)
-   so an embedding miss can't drop an obvious match [fix C].
-2. **Decide (atomic model task, one per candidate):** *"Does this chunk — `<label>` —
-   belong in bucket `<name>: <description>`? Answer yes or no."* Run over the (few)
-   shortlisted candidates. No multi-item JSON, no batch indices → the JSON-fidelity and
-   batching failure modes (critique D) simply don't exist. A chunk can get "yes" on
-   several candidates (many-to-many falls out naturally) or none.
-3. **New bucket (atomic task, only if every candidate was "no" AND substantive):**
-   *"This topic fits no existing bucket. Propose one Wikipedia-style bucket name (2–6
-   words) + one-line description, or answer NONE if too thin."* Then the **global gate
-   [fix E — the #1 correctness fix]:** re-embed the proposed name against the FULL bucket
-   centroid cache; if the nearest existing bucket clears a similarity threshold, downgrade
-   to it. This restores Lucien's "none of 379 fit" discipline that a shortlist-only view
-   would break (→ taxonomy explosion).
-
-Each model call is a tiny yes/no or a single short proposal — the clear single tasks a
-1B model can do reliably. Cost is more calls, but each is fast and reliable; embeddings
-keep K small so it's bounded.
-
-### "what buckets should we have?" — cold-start taxonomy (from zero)
-
-There are **7308 distinct labels of 7316** — dedup collapses nothing, and a 1B model
-cannot globally compress thousands of labels into a coherent 30–60-bucket taxonomy
-(that is exactly Opus's global-reasoning ability). Decompose:
-1. **Group (geometry, no model):** embed all labels → k-means (k∈[30,60], pick by
-   silhouette). The geometry does the grouping the model can't do globally.
-2. **Purity-gate each group (atomic task) [fix B]:** *"Do these labels form ONE topic or
-   more than one?"* k-means optimizes cohesion and is blind to fusion (the ρ≈0.32 purity
-   blind spot we measured) — split any group the model flags before it becomes a bucket.
-3. **Name each group (atomic task):** *"Here are ~15 representative labels. Give ONE
-   Wikipedia-style bucket name (2–6 words) + a one-line description. Be specific, not
-   generic."*
-4. **Seed centroids from members [fix H]:** each new bucket's centroid = mean of the
-   cluster labels that birthed it (those vectors already exist) — so the first assignment
-   pass runs on strong member-based centroids, not weak name-only ones.
-
-### later rungs (designed here, built after classify)
-
-- **"which section needs this info?"** — the **M×N binary** pattern: for an article's
-  assigned bucket of *M* chunks and its *N* TOC sections, ask *"should chunk i update
-  section j? yes/no"* — M×N times. The "yes" pairs become atomic patches. A chunk that gets
-  **all "no" and is substantive → an atomic CREATE-NEW-SECTION task** (mirrors new-bucket).
-  (Embeddings can prune N, but with a local model you can also just run all M×N.)
-- **"integrate THIS chunk into THIS section"** — the atomic synthesis patch (memory-synthesis.md),
-  applied to each (chunk, section) "yes" pair.
-
-### The entities (distinct — do not conflate)
-
-- **chunk** — one topical segment of a conversation (pointer + label).
-- **bucket** — a single-topic GROUP OF CHUNKS (the classify output; the "Cameras and
-  Camera Gear" container you drop camera chunks into). **A bucket is NOT an article.**
-- **article** — a synthesized wiki page. A bucket RELATES TO **one or more** articles
-  (many-to-many): the reflecting-pool-renovation bucket relates to the *Lincoln Memorial
-  Reflecting Pool* article AND a *Trump* article.
-- **section** — a part of an article (its TOC entries).
-
-### Hierarchical pruning (coarse gates fine)
-
-Decompose top-down and **prune the subtree at every "no."** The levels are distinct:
-chunk → bucket → article(s) → section(s). The coarse gate is **"does this bucket relate to
-this article? yes/no"** — if no, skip *every* section of that article (a bucket can't
-relate to a section of an article it doesn't relate to). So the M×N section checks run
-*only inside the few articles a bucket relates to*, never "× every section in the wiki."
-Same idea recursively: each coarser binary check prunes the finer ones beneath it.
-Retrieval narrows candidates *within* a level; pruning eliminates whole levels.
-
-### The uniform growth rule (no-match → create)
-
-At *every* rung, "no existing candidate matched" branches on substance, it does not just
-drop: **substantive → CREATE, trivial → drop.** A chunk that fits no bucket → new bucket;
-a bucket that relates to no existing article → new article; a chunk that fits no section of
-a related article → new section. Only non-substantive trivia is filtered. So substantive
-content always lands *somewhere* — this is how the taxonomy grows new buckets, the wiki
-grows new articles, and articles grow new sections (the Wikipedia editor adds a page or a
-section for a fact that fits nowhere). Each create is its own atomic task, gated against
-the existing set (the new-bucket global gate, fix E).
-
-## Data model (`src/memory/db.ts`)
-
-Rewrite the copy-schema to the pointer design (memory-synthesis.md; no live data):
-`messages` (text once, `(conv, position)`), `chunks` (pointers + `label`, no text),
-`buckets` (writable), `chunk_buckets` (M:N — the classify output). `PRAGMA foreign_keys
-= ON` + `ON DELETE CASCADE` (fixes Lucien's 368 orphan edges / 570 dangling pointers).
-**Keep `synthesized_bucket_chunks` per-(bucket,chunk) [fix G]** — do not rename the
-working ledger for the unbuilt section feature; `chunk_sections` may exist as a
-future-proof empty table only.
-
-## Eval — embeddings as the test instrument
-
-`scripts/experiments/bucket-cohesion-signal.ts` (mirrors `chunk-embedding-signal.ts`):
-- **[measure FIRST — the #1 risk] shortlist recall ceiling:** is each gold bucket in its
-  chunk's embedding top-K? Report overall **and per bucket-size-bin** (122/379 buckets
-  have <5 chunks — the global number hides tail collapse [fix C]). If low, the shortlist
-  caps recall and must be re-tuned before building on it.
-- **bucket cohesion / separation / subcluster-gap** baseline over the 379 cloud buckets.
-- **purity (LLM judge)** on lowest-subcluster-gap buckets — embeddings can't grade purity
-  (ρ≈0.32); confirm the asymmetry so the judge stays load-bearing.
-- **decision accuracy:** since each task is binary, eval is per-(chunk,candidate-bucket)
-  yes/no accuracy + per-chunk Jaccard vs the 8441 valid gold edges. Report **oracle-shortlist**
-  F1 (force gold into candidates → pure model judgment) vs **real-shortlist** F1 (end-to-end);
-  the gap is the retrieval tax. **The LoRA decision keys off the oracle F1 [fix F].**
-
-## LoRA decision (staged)
-
-- **Stage 0 — embedding-only:** top-K cosine + threshold vs the 8809 gold edges → F1/Jaccard.
-  Might end it (classify could be embedding-shaped).
-- **Stage 1 — base MiniCPM5-1B, prompt-only, binary tasks + shortlist:** measure oracle-F1,
-  yes/no consistency. The binary framing is *much* easier for a small base model than JSON
-  multi-assignment — Stage 1 may clear the bar with no LoRA precisely because the tasks are atomic.
-- **Stage 2 — train `memory-bucket` LoRA, only if Stage 0/1 miss.** Data from lucien.db, one
-  example per (chunk, candidate-bucket) binary decision (gold = belongs?), shortlist regime =
-  serve regime. Recipe = `chunk-finetune.ts` (sft, rank 16, lr 1e-5, seq 8192,
-  no train-flash; no env-flag sanitization needed since the 2026-07-05 kernel
-  deletions); adapter →
-  `~/.cache/mlx-bun/adapters/memory-bucket`; pick checkpoints by the F1 judge, not val loss.
-  **GPU-gated.**
-
-## Build sequence
-
-Runnable now (no GPU training, no server):
-1. **[FIRST]** `bucket-cohesion-signal.ts` → shortlist recall ceiling (per size-bin) +
-   cohesion/sep/subcluster baseline. Verifies the shortlist foundation before any build.
-2. Rewrite `db.ts` to the pointer schema + `foreign_keys` + helpers; in-memory test.
-3. Stage-0 embedding-only baseline (oracle + real F1).
-4. Purity judge on lowest-subcluster-gap buckets; confirm the asymmetry.
-5. Implement `cluster.ts` as **atomic tasks**: shortlist → per-candidate yes/no → new-bucket
-   proposal + global gate; cold-start = cluster → purity-gate → name → member-seed.
-6. Stage-1 eval (base, no adapter) → apply LoRA criteria.
-
-GPU-gated (only if needed): 7. build binary SFT data · 8. train `memory-bucket` · 9. re-eval.
-Integration: 10. wire `clusterChunks` into `pipeline.ts`.
-
-**First command:** `bun scripts/experiments/bucket-cohesion-signal.ts`
-
-## Open risks
-
-- Shortlist recall ceiling (measure per size-bin first — step 1).
-- More model calls per chunk (K binary vs 1 JSON) — bounded by K; each call is tiny/fast.
-- Cold-start naming going generic — seed with distinctive labels + "specific not generic".
-- Distilled labels ≠ ground truth — don't over-trust exact-match F1; use reference-free
-  silhouette + purity on divergences.

package/docs/design/compat-cli-surface-design.md

@@ -1,230 +0,0 @@
-# Compat CLI surface + parity harness — design
-
-Status: design (2026-06-12). Engineering tracking lives in PLAN.md
-Phase 17. UX/DX rationale lives in docs/planning/PRODUCT_ROADMAP.md ("The three
-modes"). This doc is the surface contract: every verb, the flag
-vocabulary, how optiq is absorbed, the parity harness, and **the
-matrix of what we still lack**.
-
-**Update 2026-06-17:** Several items have shipped since the 2026-06-12 snapshot: **LoRA training** is live (`src/train/`, `scripts/chunk-finetune.ts`, segmented-backward Phase A MiniCPM5 + Phase B e4b both merged); the **web UI fine-tune tab** (`src/web/app.html`) is live with SFT/DPO workflow, loss curve, merge, and export; the **chat adapter selector** (web UI) is live; and **adapters end-to-end** (web selector + CLI `/adapter`) are done — the matrix below reflects these as ✅. The **Phase 17 compat parity harness itself** (the verb router, dotted aliases, flag-surface parity tests) remains **unbuilt**.
-
-## Goal
-
-Design mlx-bun's entire CLI/flag surface from scratch (nothing is
-published, nothing is set in stone) so that one coherent set of verbs
-serves three audiences at three depths — automatic / compat /
-stick-shift (see PRODUCT_ROADMAP). mlx-lm is the vocabulary we borrow,
-because it is the Apple-WWDC-referenced default and outweighs the
-alternative ~425× on monthly installs (1.64M vs 3.85k; mlx-lm 5.8k
-stars / 765 forks; mlx-optiq has no public repo — measured
-2026-06-12).
-
-## Scope boundary: surface parity ≠ architecture parity
-
-This is CLI/flag-**surface** parity. It is NOT a promise to run every
-model architecture mlx-lm supports — the design principle "scope is the
-survival strategy: target Gemma, Qwen, one MoE; not parity with mlx-lm's
-dozens" still holds. A compat verb invoked against an unsupported
-architecture returns a clear `architecture not supported by mlx-bun yet
-(supported: …)` error, never a silent failure or a wrong answer. The
-flag is accepted; the model may not be.
-
-## The three laws (from PRODUCT_ROADMAP, restated as surface rules)
-
-1. **One surface, mlx-lm vocabulary.** `mlx-bun.<verb>` is a pure alias
-   of `mlx-bun <verb>`. Adopt mlx-lm's flag name wherever it is the
-   standard; name a flag ourselves only where mlx-lm has no concept for
-   it (free to beat optiq's name there).
-2. **Superset, never intersection.** Every documented mlx-lm flag
-   behaves faithfully so scripts drop in unchanged; we never gate a free
-   capability to resemble an upstream (OpenAI+Anthropic+Responses all
-   stay on).
-3. **Gate only on conflict.** A capability is always on unless leaving
-   it on would degrade the automatic or compat experience (changes
-   output determinism, trades quality, surprises a script) — only then
-   does it move behind a stick-shift flag.
-
-## Verb map (the spine = mlx-lm)
-
-Each verb is reachable as `mlx-bun <verb>` and the dotted alias
-`mlx-bun.<verb>`. Flags follow `mlx_lm.<verb> --help` exactly for the
-shared subset; optiq-only features are additive (marked ⊕).
-
-| Verb | Maps to | mlx-bun engine today | Notes |
-|---|---|---|---|
-| `generate` | mlx_lm.generate | ✅ `generate.ts` (no CLI verb) | wire verb + full flag set |
-| `chat` | mlx_lm.chat | engine ✅, web chat ✅; no terminal REPL (`tui.ts` is help-formatting) | build REPL frontend over `generate` |
-| `server` | mlx_lm.server / optiq serve | ✅ `serve` | rename flags; ⊕ `--kv-config`, `--anthropic` honored (default on) |
-| `convert` | mlx_lm.convert + optiq convert | ✅ `convert` (wraps `quantize/quantizer.ts`) | -q/--q-bits/--q-group-size/--mlx-path/--upload-repo parity (upload rides `hf-push.ts`); ⊕ `--target-bpw/--candidate-bits/--calibration-mix/--n-calibration` (mixed precision); no `--dtype`/`--dequantize`/plain non-quantizing convert |
-| `lora` | mlx_lm.lora + optiq lora | serve/hot-swap ✅, train ✅ | `--train/--test`; ⊕ `--rank-scaling by_bits\|by_kl`; `lora info` |
-| `fuse` | mlx_lm.fuse | ✅ `fuse` (`train/fuse.ts`) | --adapter-path/--save-path parity; preserves per-module quant layout; dequant/GGUF/upload not supported (says so, exits 1) |
-| `cache_prompt` | mlx_lm.cache_prompt | runtime ✅, CLI ❌ | precompute + save reusable KV cache file |
-| `benchmark` | mlx_lm.benchmark / optiq benchmark | ✅ `benchmark` | flag-name parity (`-p/-g/-b/-n`); ppl half ❌ |
-| `evaluate` | mlx_lm.evaluate / optiq eval | ❌ (`evals` = our bench viewer) | task harness; ⊕ optiq task names |
-| `perplexity` | mlx_lm.perplexity | ✅ `perplexity` (`eval/perplexity.ts`) | reference methodology (pack → non-overlapping rows → f32 CE, delta-method SE); data source is a LOCAL .txt/.jsonl, not an HF dataset |
-| `upload` | mlx_lm.upload | ✅ `upload` (wraps `hf-push.ts`, the web push engine) | --path/--upload-repo parity; ⊕ `--private`; token from hf.json/$HF_TOKEN/hf auth login |
-| `manage` | mlx_lm.manage | ✅ `get`/`scan`/`ls` | add `--delete/--pattern/--scan` parity |
-| `kv-cache` | optiq kv-cache | runtime ✅, profiler ❌ | per-layer sensitivity → `kv_config.json` |
-| `latency` | optiq latency | ✅ `fit` | alias + `--calibrate` |
-| `awq` | mlx_lm.awq | ❌ | activation-aware quant |
-| `dwq` | mlx_lm.dwq | ❌ | distilled quant |
-| `dynamic_quant` | mlx_lm.dynamic_quant | ❌ | sensitivity mixed-bit (overlaps convert/TurboQuant) |
-| `gptq` | mlx_lm.gptq | ❌ | GPTQ |
-| `share` | mlx_lm.share | ❌ | distributed cache share (lowest priority) |
-
-Our own verbs stay (`fit`, `pi`, `harness`, `doctor`, no-args
-appliance) — they are the automatic/stick-shift surface and have no
-mlx-lm equivalent. See the native-surface section below.
-
-## mlx-bun-native surface (the differentiators)
-
-These have NO mlx-lm/optiq equivalent — they are not parity items, they
-are what makes mlx-bun *more* than a faster mlx-lm. The compat layer
-(Phase 17) is the "drop-in for mlx-lm" half; this is the "more than
-mlx-lm" half.
-
-### pi — the built-in agentic coding CLI
-
-(Phase 16; full plan in **docs/investigations/pi-builtin-investigation.md**.) pi is an
-MIT, Bun-native (`bun build --compile`) coding agent — the "CLI
-experience inside our package" that neither Python package has. Two
-commands, one for each audience:
-
-1. **`mlx-bun pi`** (the appliance, exists) — start the server (reuse if
-   healthy) and run pi's own `InteractiveMode` **in-process**: one
-   binary, one command, a real agent on a local model with nothing to
-   install (pi is a bundled dep). Custom mlx-bun coding-agent prompt,
-   full coding tools + web tools, pi's native approval prompts.
-   `-p`→`runPrintMode`, `--mode json/rpc`→`runPrintMode`/`runRpcMode`.
-2. **`mlx-bun harness pi`** (for people who already use pi, exists) —
-   point the user's OWN pi at mlx-bun's local model, via a generated
-   extension that dynamically discovers models from our `/v1/models`.
-   No models.json surgery; reversible. This is where the full pi flag
-   surface (`--continue`, `--resume`, `@file`, extensions, themes) lives.
-
-History: an earlier cut shipped `mlx-bun pi` as a subprocess launcher of
-the user's own pi (and briefly a `--external-pi` flag); both were dropped
-(2026-06-14) — a user who already has pi already knows how to run it, so
-it was pure duplication. The embedded agent + `harness pi` is the clean
-split.
-
-### Built-in web UI (OptIQ-Lab parity, pi-powered)
-
-The web UI's agent backbone is pi's RPC / `AgentSession.subscribe()`
-event stream (investigation Option 4) — the SAME events drive the
-embedded TUI and the browser, so tool calls / steering / progress come
-for free. Tiles: chat ✅; fine-tune ✅ (shipped: SFT/DPO workflow, loss curve, merge,
-export); quantize / training-data gate on the capability matrix.
-
-## Flag-vocabulary reconciliation (where we currently diverge)
-
-- `serve --kv-quant N` → adopt mlx-lm's `--kv-bits` + `--kv-group-size`
-  + `--quantized-kv-start` (keep `--kv-quant` as a deprecated alias).
-- `serve --prompt-cache` → mlx-lm's `--prompt-cache-size` (entries) +
-  `--prompt-cache-bytes` (our byte-cap is already the right model).
-- Our runtime levers (`--compiled-decode`, `--compiled-activations`,
-  `--fused-sdpa`, `--force-wire`) have no mlx-lm equivalent → keep, these
-  are stick-shift flags. (`--perf-kernel` and `--fused-decode` were deleted
-  2026-07-05 — docs/design/unified-engine-frontier-plan.md §6.)
-- Adopt mlx-lm's sampling names verbatim: `--temp`, `--top-p`,
-  `--top-k`, `--min-p`, `--max-tokens/-m`, `--seed`, `--xtc-*`.
-
-## What we still lack — capability gap matrix
-
-Legend: ✅ have · 🔌 wiring-only (engine exists, needs verb/flags) ·
-🟥 missing capability (real build). The compat **layer** is the 🔌
-column; the 🟥 column is the capability matrix that fills in behind it.
-
-| Capability | Verb(s) | Status | Bucket / owning phase |
-|---|---|---|---|
-| One-shot generate | `generate` | 🔌 | Phase 17 (compat) |
-| Chat REPL | `chat` | 🔌 | Phase 17 |
-| OpenAI/Anthropic/Responses serve | `server` | ✅ | done (Phase 4/11) |
-| Server flag parity | `server` | 🔌 | Phase 17 |
-| Benchmark (throughput) | `benchmark` | ✅ flags 🔌 | Phase 17 |
-| Latency prediction | `latency`/`fit` | ✅ | done; `--calibrate` 🔌 |
-| Model cache manage | `manage` | 🔌 | Phase 17 |
-| Precompute prompt cache | `cache_prompt` | 🔌 | Phase 17 |
-| LoRA serve / hot-swap | `lora` (serve) | ✅ | done (Phase 8) |
-| `lora info` reporting | `lora info` | 🔌 | Phase 17 |
-| **HF→MLX convert → new model artifact** | `convert` | ✅ | done (CLI verb over the native quantize pipeline) |
-| **Sensitivity / mixed-precision weights** | `convert`, `dynamic_quant` | ✅ (`convert --target-bpw`) | done via convert; `dynamic_quant` alias 🟥 |
-| **AWQ / DWQ / GPTQ (weights)** | `awq`/`dwq`/`gptq` | 🟥 | Model-quant |
-| KV-cache quant: bf16 / uniform / **mixed per-layer** (kv_config.json) | `server` (`off`/`N`/`config`) | ✅ | done (Phase 9/10; config.ts reads, generate.ts applies per-layer) |
-| KV sensitivity *profiler* (authors a NEW kv_config.json) | `kv-cache` | 🟥 | model-prep, low pri (shipped artifacts already include one) |
-| TurboQuant — optional extra quant *method* (rotation VQ) | scheme, not a verb | 🟥 | Phase 13 (KV plumbing already exists) |
-| **LoRA / DoRA / full training** | `lora --train` | ✅ | Training (done: Phase A MiniCPM5 + Phase B e4b) |
-| **Sensitivity-aware rank scaling** | `lora --rank-scaling` | 🟥 | Training |
-| **Fuse adapters (+GGUF/upload)** | `fuse` | ✅ fuse; GGUF/dequant/upload 🟥 | done (fold-into-base, quant layout preserved) |
-| **lm-eval-harness tasks** | `evaluate` | 🟥 | Eval (new phase) |
-| **optiq eval suite (kl/gsm8k/ifeval/bfcl/hashhop…)** | `evaluate` | 🟥 | Eval |
-| **Perplexity** | `perplexity` | ✅ | done (local text/jsonl datasets) |
-| **HF upload** | `upload` | ✅ | done (native `hf-push.ts`; also `convert --upload-repo`) |
-| **Distributed share** | `share` | 🟥 | Distribution (lowest pri) |
-| **Web UI: quantize** | `lab`/UI | 🟥 | gated on Quantize |
-| **Web UI: fine-tune** | `lab`/UI | ✅ | done (SFT/DPO workflow, loss curve, merge, export) |
-| **Web UI: training-data template + gen** | `lab`/UI | 🟥 | new (no engine dep) |
-| Web UI: chat | `/chat` | ✅ | done (publishing gate 2) |
-| **Native: pi embedded** (`mlx-bun pi`, in-process TUI) | — | ✅ (P3/P4) | Phase 16 (flagship) |
-| **Native: connect user's own pi** (`harness pi`) | — | ✅ (P1) | Phase 16 |
-
-Summary: serving and the read-only/measurement verbs are **🔌
-wiring-only** — the compat layer ships over what the engine already
-does. The real capability work splits along two axes:
-
-- **Model quantization** (offline, makes a new artifact): `convert`,
-  `awq`, `dwq`, `gptq`, `dynamic_quant`. Its own new phase.
-- **Inference-time KV-cache quant** (online, no artifact): bf16 /
-  uniform / **mixed per-layer** from kv_config.json all ✅ today
-  (Phase 9/10 — we read & apply shipped configs). The only KV gaps are
-  optional and not inference plumbing: a *profiler* to author a new
-  kv_config.json (model-prep, low pri) and extra methods like TurboQuant
-  (Phase 13).
-- **Training**: LoRA/DoRA/full training ✅ (shipped); `fuse` verb (adapter merge, GGUF export, upload) 🟥 remaining.
-- **Eval**: harness + `perplexity`.
-- **Distribution**: `upload` ✅ (CLI verb + `convert --upload-repo`); `share` 🟥.
-
-The OptIQ-Lab web-UI tiles gate on these: quantize ← model-quant,
-fine-tune ← training, training-data ← independent, chat ✅.
-
-Plus the **native half** (no parity equivalent — the differentiators):
-the pi agentic coding CLI (Phase 16: `mlx-bun pi` is the embedded
-in-process agent, `harness pi` connects the user's own pi) and the web
-UI, both riding pi's `AgentSession` event stream. Compat makes us a
-drop-in for mlx-lm;
-the native half makes us more than it.
-
-## Parity harness (turns "drop-in" into a passing test)
-
-Same discipline as logit parity. For each compat verb, a test diffs
-mlx-bun against the Python oracle in `/Users/joshrossi/Code/mlx-lm/.venv`:
-
-- **Flag-surface parity**: parse `mlx_lm.<verb> --help`, assert every
-  documented flag is accepted by `mlx-bun <verb>` (or explicitly
-  rejected with a clear message — never silently ignored).
-- **Behavior parity** where deterministic: `generate --seed --temp 0`
-  output, `manage --scan` listing shape, `cache_prompt` round-trip,
-  `benchmark` column schema.
-- **Gap honesty**: a not-yet-built verb (`convert`, `evaluate`, …)
-  exits non-zero with `not implemented in mlx-bun yet` — the harness
-  asserts the message, so gaps are visible, not silent.
-
-Gated by `benchmark.sh`-style preflight only where it measures speed;
-correctness diffs need no clean machine.
-
-## Build sequencing
-
-1. **Phase 17 — this layer**: verb router + dotted aliases + flag
-   vocabulary + reconcile existing commands + parity harness, over
-   existing capabilities (`generate`, `chat`, `server`, `benchmark`,
-   `manage`, `cache_prompt`, `lora info`, `latency`). Honest stubs for
-   🟥 verbs.
-2. Model quantization (new phase) → `convert`/`dynamic_quant`/`awq`/
-   `dwq`/`gptq` (new model artifacts) + web-UI quantize tile.
-3. Inference-time KV quant → already done (bf16 / uniform / mixed
-   per-layer via kv_config.json, Phase 9/10). Optional later: a profiler
-   to author new configs (low pri) + the TurboQuant method (Phase 13).
-4. Training → ✅ `lora --train` + web-UI fine-tune tile (both shipped); remaining: `fuse` verb (adapter merge, GGUF export, upload).
-5. Eval → `evaluate`/`perplexity`.
-6. Distribution → `upload` ✅ (2026-07-01); `share` remaining.
-7. Web UI training-data tile (independent of engine work).

package/docs/design/docs-reorg-plan.md

@@ -1,165 +0,0 @@
-# Docs & repo-organization cleanup plan
-
-**Status:** executed 2026-06-14 (P0 + P1 + P2 #11). Held: P2 #9/#10
-(src/ and scripts/ moves — they change import paths / script references
-and need a test run, so deferred to their own pass). Authored 2026-06-14.
-
-**Execution state (verified 2026-06-17):**
-- P0 #1 — **DONE**: `AGENTS.md` is a symlink → `CLAUDE.md`; drift risk resolved.
-- P0 #2 — **DONE**: `benchmarks/RESULTS.md` created; old committed benchmark artifacts `git rm`'d. Root untracked spew (`benchmarks-h2h-2026-06-13-*` etc.) is now cleaned — no untracked report files found at root.
-- P0 #3 — **DONE**: root HTML/md spew cleared (no `benchmarks-h2h-*.md`, `benchmark-report-*.html`, `kernel-fusion-report-*.html`, or `project-review.html` at root).
-- P0 #4 — **DONE**: hardcoded test count removed from CLAUDE.md (no "All 72 tests" present).
-- P1 #5 — **DONE**: `STATUS.md` exists at root.
-- P1 #6 — **DONE**: `docs/planning/` exists with `IDEAS.md`, `PRODUCT_ROADMAP.md`, `ResearchTopics.md`, `journal.md`.
-- P1 #7 — open (optimization_plan.md still lives in docs/design/ as a standalone).
-- P1 #8 — **RESOLVED (2026-07-01)**: the duplicate `archive/pi-builtin-investigation.html` is already gone; `docs/archive/` deliberately keeps 2 small demo artifacts (`mlx-bun-lab-report.html`, `wwdc-mlx-bun.html`, 76K total), tracked.
-- P2 #9 — **NOT DONE**: `harness-pi.ts`, `pi-provider.ts`, `pi-terminal.ts`, `pi-web.ts` still live flat in `src/` root (no `src/pi/` subdirectory).
-- P2 #10 — **PARTIALLY DONE**: ~57 one-off research/debug scripts moved to `scripts/experiments/`; production + tooling + bench-* + regen-* + parity-* + gen-model + eval/serve remain at `scripts/` root (the full bench/regen/build split was not applied).
-- P2 #11 — **DONE**: CLI vocabulary reconciled.
-
-**Scope & ethos.** Private repo, single author + agents, MLX-by-identity.
-No OSS/community scaffolding (CONTRIBUTING, issue templates, badges) —
-out of scope by design. **Prefer move / consolidate over delete.** The
-one exception is regenerable benchmark *output* (md/json/html report
-artifacts): those are noise, not experiments, and are explicitly
-deletable per Josh's direction. Losing *experiments* and code paths are
-never deleted (see memory: "don't delete optionality on one
-measurement").
-
----
-
-## The centerpiece: benchmark provenance (corrected)
-
-The original analysis mis-framed this. The correct policy:
-
-- Benchmark **runs** are date- and machine-stamped, regenerable, and
-  machine-specific. They do **not** belong in the durable repo. The
-  existing `.gitignore` rule `benchmarks-h2h-*` (ignore all raw runs) is
-  **correct and stays**.
-- The actual defect today: a handful of raw run files leaked into git
-  history (committed before that ignore rule existed), and the README's
-  headline performance claims cite one of them
-  (`benchmarks/benchmarks-h2h-2026-06-11-Joshs-MBP-2025.md`) as the
-  source of truth. The repo's central perf claim depends on exactly the
-  kind of ephemeral artifact the gitignore says shouldn't be tracked.
-
-**The fix:** one curated, intentionally-maintained record, structured
-around the **three distinct kinds of measurement** below; raw runs stay
-local/ignored; the eval DB remains the structured backing record.
-
-### The three kinds of benchmarks (the record's structure)
-
-These are categorically different — one is pass/fail correctness, one is
-a perf number, one is a quality delta — and `RESULTS.md` must keep them
-in separate sections rather than one undifferentiated table. They map to
-the project's three-tier parity model (L1/L2/L3).
-
-1. **Parity (porting correctness) — pass/fail, not a number.** Are we
-   bit-for-bit with the upstream oracle under like-for-like config?
-   - mlx-lm + standard (bf16) KV cache → bit-exact (L1).
-   - mlx-optiq + mixed-precision KV cache → bit-exact (L2).
-   Record as a matrix of model × mode → ✓/✗ (+ commit, oracle versions).
-   This is the gate the perf/quality numbers are only meaningful *under*.
-
-2. **Performance — like-for-like numbers, two comparison axes.** Under
-   the parity constraints above, measure tok/s, prefill/s, decode/s,
-   TTFT, startup, RSS, peak memory. Two axes:
-   - **vs the oracles** — mlx-bun vs mlx-lm / mlx-optiq, same model, same
-     mode, same machine, same day.
-   - **vs our own compat mode** — mlx-bun's NEW optimizations vs
-     mlx-bun's own compatible (bit-exact) path, to prove an optimization
-     actually beats the baseline it diverges from.
-
-3. **Quality — for our non-bit-exact optimizations only (L3).** When a
-   custom path trades exactness for speed, quantify what the trade costs:
-   - **mean score across the 6 eval tasks** (`src/eval/tasks/`: bfcl,
-     gsm8k, hashhop, humaneval, ifeval, mmlu), vs the compatible upstream.
-   - **KL divergence** vs the compatible distribution.
-   Purpose: enable statements like *"the optimized path gains 23% tok/s
-   while holding ±5% on the 6-test mean"* — the perf win is only worth
-   claiming alongside its measured quality cost.
-
-### B1. Create `benchmarks/RESULTS.md` (the new durable record)
-- Three top-level sections: **Parity** (model × mode → ✓/✗), **Performance**
-  (oracle comparison + our-optimized-vs-our-compat), **Quality** (6-test
-  mean + KL for non-exact paths). Each row carries provenance (machine,
-  date, commit, model, mode, oracle versions).
-- Named so it does **not** match `benchmarks-h2h-*`, so it's tracked
-  deliberately and future raw runs never auto-collide with it.
-- Migrate the headline numbers Josh wants to keep (currently the 06-11
-  matrix, plus the 06-13/06-14 follow-ups) into the right section
-  **before** removing the raw files, so README's claims never lose their
-  source.
-- Point README's "Full table" link at `benchmarks/RESULTS.md`.
-
-### B2. Remove the old committed benchmark artifacts
-`git rm` the leaked raw runs and verification dumps (numbers preserved in
-B1 + the eval DB):
-- `benchmarks/benchmarks-h2h-2026-06-10-Joshs-MBP-2025.md`
-- `benchmarks/benchmarks-h2h-2026-06-10-Joshs-MacBook-Pro.md`
-- `benchmarks/benchmarks-h2h-2026-06-10.md`
-- `benchmarks/benchmarks-h2h-2026-06-11-Joshs-MBP-2025.md`
-- `benchmarks/lab-verification.json`
-- `benchmarks/lab-verification-mixed.json`
-
-Also clear the loose **untracked** root spew (delete or move to an
-ignored `reports/`): `benchmarks-h2h-2026-06-13-*.md`,
-`benchmarks-h2h-2026-06-14-*.{md,html}`, `benchmark-report-*.html`,
-`kernel-fusion-report-*.html`, `project-review.html`.
-
-`scripts/verify-lab.ts` keeps writing `benchmarks/lab-verification.json`
-locally — it'll just be gitignored now (add `benchmarks/lab-verification*.json`
-to `.gitignore`).
-
-### B3. Eval DB — leave where it is (decided 2026-06-14)
-- `~/.cache/mlx-bun/evals.sqlite` ([src/evaldb.ts:8](../../src/evaldb.ts))
-  stays user-local — it is the structured backing record of every run.
-  **Not tracked in-repo** (binary SQLite in git doesn't diff, churns, and
-  grows history). `RESULTS.md` is the committed human-readable record; the
-  DB is its machine-local source. Decision confirmed by Josh — closed.
-
-### Do NOT touch
-`goldens/*.json` and `goldens/**` are the correctness oracle golden
-files, not benchmark output. Leave entirely alone.
-
----
-
-## P0 — Quick, safe wins (low risk)
-
-| # | Action | Why |
-|---|---|---|
-| 1 | Collapse `AGENTS.md` → `CLAUDE.md` (make AGENTS.md a 1-line pointer or symlink). | They're byte-identical except the title line — guaranteed to drift. |
-| 2 | B1–B2 above: create `RESULTS.md`, migrate numbers, `git rm` old artifacts, point README at it, gitignore `lab-verification*.json`. | Fixes benchmark provenance; declutters; keeps gitignore policy honest. |
-| 3 | Relocate untracked root HTML/md spew into an ignored `reports/` (or delete). | De-clutters the working-tree root. |
-| 4 | Drop the hardcoded "All 72 tests" in CLAUDE.md (say "the suite" / reference the count command). | 57 test files now — a baked-in number that's already wrong. |
-
-## P1 — Structural consolidation (the real navigability win)
-
-| # | Action | Why |
-|---|---|---|
-| 5 | Add `STATUS.md` (or a pinned top section in PLAN.md) holding only live handoff: current state + next action + blockers. Move PLAN.md's `NEXT UP` / `NEXT SESSION PICKUP` / `SESSION SWEEP` blocks into it. | Separates transient "do this next" from 2,475 lines of durable phase history; gives agents one entry point. |
-| 6 | Create `docs/planning/` and move `PRODUCT_ROADMAP.md`, `IDEAS.md`, `ResearchTopics.md`, `journal.md` under it. Leave only README + PLAN.md (+ STATUS.md) at root. Add a ~6-line "doc map" to README. | Root is currently a 6-file planning thicket; extends the taxonomy that already works in `docs/`. |
-| 7 | Pick one home for the optimization plan: fold `docs/design/optimization_plan.md` into PLAN.md Phase A–E, or have PLAN.md link out to it as canonical. | Two homes for the same plan = drift. |
-| 8 | Resolve the archive overlap: `archive/pi-builtin-investigation.html` duplicates [docs/investigations/pi-builtin-investigation.md](../investigations/pi-builtin-investigation.md). Decide whether `archive/` (tracked; 2 HTML files: `mlx-bun-lab-report.html` + `wwdc-mlx-bun.html`) should be tracked at all, or moved to an ignored location. Keep the `.md`, drop the duplicate. | Stops committing heavyweight generated HTML; one source per investigation. |
-
-## P2 — Polish (optional)
-
-| # | Action | Why |
-|---|---|---|
-| 9  | Group `src/pi/` (`harness-pi.ts`, `pi-provider.ts`, `pi-terminal.ts`, `pi-web.ts`). | `src/` root is 31 flat files; this is an obvious cohesive cluster. |
-| 10 | Sort `scripts/` into `scripts/{bench,regen,build}/` (40 files, clear prefixes). | Navigability only; low urgency. |
-| 11 | Reconcile CLI vocabulary: README quickstart uses the shipped `mlx-bun` verb (with a one-line "from a clone, `bun src/cli.ts` is the same"). | README/roadmap/package.json currently mix `bun src/cli.ts` and `mlx-bun`. |
-
----
-
-## Execution order (after slots work is pushed)
-
-1. P0 #1, #4 (trivial, isolated).
-2. Benchmark block B1 → B2 → README relink (do B1 fully before B2).
-3. P0 #3 (root spew).
-4. Confirm B3 eval-DB decision with Josh.
-5. P1 #5–#8 (the structural moves; one commit per item so history stays legible).
-6. P2 as appetite allows.
-
-Each step keeps the test suite green and changes no source behavior — this
-is docs + file layout only.