mlx-bun 0.4.0 → 1.0.0

package/docs/archive/investigations/trainer-validation-experiment.md

@@ -1,173 +0,0 @@
-# Trainer validation experiment
-
-**Status:** in progress (started 2026-06-20). **Goal:** prove the from-scratch
-SFT/DPO/ORPO trainer actually works before trusting any downstream result — general
-capability (Goal 1) or task-specific (Goal 2).
-
-## Why this exists — the smoking gun
-
-The margin/accuracy is recorded in `metrics.json` (`valTrajectory[].margin`,
-`bestMargin`), not the stdout log. The prior `cpm5-uf` ORPO run (λ=0.1) shows:
-
-| step | 100 | 200 | 300 | 400 | 500 | 600 | 700 | 800 |
-|---|---|---|---|---|---|---|---|---|
-| accuracy | .507 | .514 | .534 | .541 | .534 | .520 | .547 | .547 |
-| margin | .132 | .124 | .110 | .120 | .136 | .128 | .118 | .123 |
-
-**Preference accuracy is ~0.52 (near the 0.50 coin-flip) and the margin is flat.**
-ORPO's defining job is to push accuracy up and widen the margin. This is *not* a clean
-"it works" signal. Caveats: 800 steps ≈ 0.05 epoch (undertrained), provenance
-(from-base vs warm-start) unconfirmed. But it's enough to make validation mandatory:
-**we cannot currently claim our ORPO learns the preference at all.**
-
-## Method-decomposition (the validation key)
-
-`ORPO_loss = NLL(chosen) + λ·L_OR`. NLL is exactly the SFT objective; L_OR is the
-reference-free odds-ratio preference term. So we validate the building blocks, and ORPO
-is covered by its parts plus a small ORPO-specific surface (the OR term + λ).
-
-mlx-lm has **no DPO/ORPO** — so the oracle strategy is:
-- **SFT/NLL** → numeric parity vs **mlx-lm tuner** (oracle env exists).
-- **DPO + OR loss** → **numeric unit test vs the paper's equations** on a fixed tiny
-  input (hand-computed expected loss + grad). No external dep. Optional stronger
-  cross-check: **TRL** `DPOTrainer`/`ORPOTrainer` loss (PyTorch; multi-GB env →
-  approve-first, not default).
-
-Every positive control is designed so "weak data / weak lever" is **not** an available
-excuse — success must be unambiguous.
-
-## PILLAR 1 — Prove the trainer (gate for everything)
-
-- **1a · SFT positive control** *(infra GREEN, behavioral check WEAK)*. Train
-  `fixtures/adapters/data-upper` from scratch on MiniCPM5; assert generations are
-  actually uppercased (oracle: the repo's pre-trained `upper` adapter). Existing
-  `train-e2e.test.ts` only checks loss↓ — add the behavioral assert. *(seconds)*
-- **1b · DPO positive control** *(MISSING)*. Trivial preference (chosen uppercase,
-  rejected lowercase). Assert **margin increases** and **preference accuracy → ~1.0**.
-- **1c · ORPO positive control + the real question.** (i) Same trivial preference →
-  accuracy → ~1.0. (ii) **From-BASE ORPO on UF**, multiple epochs, watch margin widen
-  and accuracy climb well above chance — directly resolves the near-chance finding
-  (undertraining vs warm-start vs bug).
-- **1d · Loss-formula parity.** SFT NLL vs mlx-lm; DPO/ORPO loss + grad vs hand-computed
-  paper equations (unit test). The training-side analogue of the inference logit-parity
-  oracle — currently MISSING.
-- **1e · vs the paper.** Internal proxy first: preference accuracy robustly > chance on
-  UF. Then (approve-first, LLM-judge) AlpacaEval/MT-Bench **direction** vs base (not
-  absolute — paper used 7B). optiq suite (incl. IFEval) is an already-local downstream.
-
-> **Gate:** until 1a–1c are green, Pillars 2 & 3 sit on an unverified trainer.
-
-## PILLAR 2 — Measure general-capability impact (optiq suite)
-
-All 6 datasets are present locally (`~/.cache/mlx-bun/eval-data/`: mmlu, gsm8k, ifeval,
-bfcl, humaneval, hashhop); harness = `scripts/eval.ts capability`.
-- **2a · Eval parity:** run on base CPM5, reproduce optiq's **30.28** within tolerance
-  (validates the *measurement* before trusting deltas; confirm the README's open BFCL
-  "no-calls" / MMLU "at-chance" flags aren't broken on our side).
-- **2b · Delta:** trained checkpoint vs base, on the 6-task mean.
-- **2c · Targeting:** mean is floored by **BFCL 0.0 / GSM8K 2.7 / HashHop 4.0**; UF hits
-  the already-strong IFEval, so expect small mean movement. Highest-leverage Goal-1 lever
-  is **BFCL** (0.0 is format-trainable; 0→30 = +5 on the mean).
-
-## PILLAR 3 — Task-specific (separate; gated on Pillar 1)
-
-Chunk segmenter via `scripts/chunk-eval.ts` (boundary/label accuracy), using the
-hard-negative ORPO data (remove-a-cut / add-a-cut, balanced to kill the length
-shortcut). Measured by chunk accuracy, not val loss / capability. Credibility depends on
-Pillar 1.
-
-## Sequencing & gates
-
-1. Pillar 1 positive controls (1a→1b→1c-trivial) — fast, "does the machinery work."
-2. Pillar 1 real replication (1c from-base UF) + 1d loss-formula parity.
-3. Pillar 2a eval-parity (reproduce 30.28) → 2b deltas.
-4. Pillar 3 — once the trainer is trusted.
-
-## Execution log
-
-### 2026-06-20 — Pillar 1 quick validations: PASS
-
-**Machinery + gradient correctness — GREEN.** `MLX_BUN_TEST_TRAIN=1 bun test
-train-autograd train-e2e train-orpo-e2e` → 8 pass / 0 fail, real training on
-MiniCPM5 (11.3s). `train-autograd` confirms the backward matches autograd.
-
-**ORPO positive control (prefer-uppercase, 12 train / 4 val, 120 iters, lr 1e-3) — PASS.**
-Held-out preference accuracy **1.000 by step 10**, margin **5.5 → 4.0** (noisy at lr=1e-3
-but accuracy solid). Logged: `runs/orpo-pc.{json,csv}`.
-
-**DPO positive control (same data) — PASS, cleaner.** Accuracy **1.000 from step 10**,
-margin **monotonic 6.46 → 9.67**. Logged: `runs/dpo-pc.{json,csv}`.
-
-**Conclusion: the preference machinery is validated.** On a learnable preference both
-DPO and ORPO drive accuracy to 1.0 and the margin wide open (vs UF's flat ~0.12). A
-broken trainer could not. ⇒ **The UF near-chance result is the DATA, not a bug** — UF
-pairs are genuinely subtle for a 1B (both responses plausible); "prefer uppercase" is
-trivially separable. The smoking gun is explained: implementation sound, lever weak.
-
-**Gotcha found (recorded so we don't repeat it):** `AdapterManager.mount` (the *serving*
-hot-swap path) does **not** reflect into `branchLogpMean` (the *training* forward) — an
-external base-vs-adapter `orpoMetrics` measurement via mount returns bit-identical
-numbers (measured: 0.500/0.1463 both). Measure training-side metrics via the
-`trainLora` emit / loraState path (`scripts/experiments/pref-control.ts`), not mount.
-
-**Logging for plotting:** `runs/<stem>.{json,csv}` — per-step train `{loss, accuracy,
-margin}` + per-eval val `{loss, accuracy, margin}`. CSV columns: `phase,step,loss,
-accuracy,margin`. Tools: `scripts/experiments/pref-control.ts` (train+log),
-`scripts/experiments/measure-pref.ts` (standalone adapter eval — superseded by the
-emit path for training metrics; see gotcha).
-
-### 2026-06-20 — Pillar 2a: capability-harness parity. Two bugs found + fixed.
-
-**The first-measure step caught two harness bugs** (exactly its purpose — never trust a
-delta until the harness reproduces the published baseline):
-
-1. **Thinking mode.** The eval rendered the chat template's *default* (thinking-on for
-   MiniCPM5); optiq's numbers are **non-thinking** (HF card: IFEval 64.7 is non-thinking).
-   Symptom: IFEval **36.7%** vs 64.7. **Fix:** `src/eval/runner.ts` `generateText` now
-   renders `enableThinking: false` by default (env `MLX_BUN_EVAL_THINK=1` to re-enable).
-   → IFEval **73.3%** (n=30), optiq ballpark.
-2. **HumanEval oracle-python path.** `src/eval/tasks/humaneval.ts` hardcoded
-   `/Users/joshrossi/Code/mlx-lm/.venv/bin/python` — wrong on this machine (it's
-   `mlx-lm-example`), so every code-exec spawned a missing interpreter → **0%**. **Fix:**
-   env-overridable `MLX_BUN_ORACLE_PYTHON` (committed default = reference box; never
-   re-commit the path). → HumanEval **53.3%** (n=15), optiq 57.9.
-
-**Post-fix parity (base CPM5, non-thinking, n=20 — all ballpark):**
-
-| | MMLU | GSM8K | IFEval | BFCL | HumanEval | HashHop |
-|---|---|---|---|---|---|---|
-| ours | 55.0 | 10.0 | 75.0 | 0.0 | ~53 | 0.0 |
-| optiq | 52.4 | 2.7 | 64.7 | 0.0 | 57.9 | 4.0 |
-
-⇒ **harness reproduces optiq.** Small-n inflation on GSM8K/IFEval will settle toward
-optiq on the full set. **Exp-1 baseline harness VALIDATED.**
-
-### Ready-to-fire sequence (everything staged; only GPU time remains)
-
-All measurement tools validated. `scripts/eval.ts` now takes `--adapter <dir>` (mounts the
-trained LoRA → x2). `ORC=/Users/joshrossi/Code/mlx-lm-example/.venv/bin/python` below.
-
-1. **x1** (running): `MLX_BUN_ORACLE_PYTHON=$ORC bun scripts/eval.ts capability --candidate MiniCPM5`
-2. **Train** (user): from-base ORPO/UF → `./adapters/cpm5-uf-frombase` (command in chat).
-3. **x2 (Exp 1 after):** `MLX_BUN_ORACLE_PYTHON=$ORC bun scripts/eval.ts capability --candidate MiniCPM5 --adapter ./adapters/cpm5-uf-frombase` → compare to x1.
-4. **Exp 2 gen (GPU):** `bun scripts/experiments/prefbench.ts gen mtbench ./adapters/cpm5-uf-frombase mtbench-uf` (and `gen alpaca … alpaca-uf 200` for a subset).
-5. **Exp 2 judge (codex, anytime):** `bun scripts/experiments/prefbench.ts judge mtbench-uf` → TRAINED win-rate vs BASE.
-
-### Next
-- **Full baseline:** see step 1 (running). **(long → user-launched)**
-- **Train (the "do"):** from-base ORPO on UF (`pref-control.ts`, command in chat). **(long → user-launched)**
-- **Re-measure → x2**, compare to x1. Exp 1 done.
-- **Exp 2 judge — VALIDATED (keyless).** No OpenAI key needed: the judge shells out to
-  `codex exec -m gpt-5.4-mini` (codex's own auth), forcing structured `{"winner":...}` via
-  `--output-schema` and reading the clean verdict from `-o`. `scripts/experiments/llm-judge.ts`
-  — `judgeOnce` + `judgePair` (judges BOTH orders to cancel small-judge position bias).
-  Positive control: **3/3 obvious good-vs-bad pairs, good won both orders, zero flips.**
-  1bx vs 1bx1; consistency (same judge both sides), not absolute comparability to the paper's GPT-4.
-- **Exp 2 harness — STAGED.** Prompts downloaded: `~/.cache/mlx-bun/eval-data/{mt_bench.jsonl
-  (80), alpaca_eval.json (805)}`. Two-phase runner `scripts/experiments/prefbench.ts`:
-  `gen <mtbench|alpaca> <adapterDir> <stem> [limit]` (GPU — base then trained, non-thinking)
-  → `runs/<stem>.responses.jsonl`; `judge <stem> [limit]` (codex) → TRAINED win-rate vs BASE,
-  judging trained-vs-base **directly** (head-to-head, not vs a fixed reference). Only the `gen`
-  phase needs GPU; everything else is validated. Fire after the from-base training produces
-  `./adapters/cpm5-uf-frombase`.
-- **1c real / 1d parity** — still open (lower priority now the positive controls + harness parity are green).

package/docs/archive/planning/chat-ui-vision-memory-first.md

@@ -1,199 +0,0 @@
-# Chat UI Vision: The Model That Knows You
-
-**Stance: memory/personalization-first.** Every other axis in this
-doc — speed, tools, LoRA routing, privacy — is real, but it is in
-service of one idea: mlx-bun is the only local chat surface where the
-model visibly, auditably gets better at being *yours* over time. If we
-build a fast, private, tool-capable chat app that doesn't make that
-tangible on screen, we've built LM Studio with worse artifacts. That is
-not the plan.
-
-## The feeling
-
-Open the app. It's not a hero screen with generic suggestion chips —
-it's a hero screen that already knows something. Not in a creepy way:
-in the way a good assistant who's worked with you for months starts a
-Monday with "want me to pick up the Lisbon itinerary thing from
-Thursday?" instead of "How can I help you today?" It should feel like
-walking into a room where someone has been quietly paying attention,
-not opening a fresh terminal.
-
-The tell, mechanically: a chat with mlx-bun after a month of use is a
-*different product* from day one — not because we shipped a new model,
-but because the sidebar, the hero, and the answers themselves carry
-accumulated context the user can see, correct, and trust. Cloud
-chatbots fake a version of this behind a black box on someone else's
-server. Local competitors (LM Studio, Ollama, macMLX) don't attempt
-it — stateless between chats by design. Open WebUI and AnythingLLM
-attempt it as prompt-stuffing into a flat fact list. Odysseus gets
-closest (memory + skills) but on vLLM/llama.cpp infra with no adapter
-concept at all. **Nobody combines memory-as-training-artifact with
-weight-level LoRA routing and native MLX speed. That combination is the
-whole pitch, and today it's invisible in our own UI** — the vault, the
-memory tools, and the adapter runtime already exist server-side; none
-of it shows on screen.
-
-## The five things that make this unmistakable
-
-**1. Memory is a visible citizen of the chat, not a silent tool call.**
-Today `memory_search`/`memory_read` fire invisibly inside a generic
-tool-call card indistinguishable from a bash command. The redesign
-gives memory its own visual language: when the model consults the
-vault mid-answer, a small soft-colored chip appears inline — not a
-context dump, not an approval gate (it's read-only, never needs
-permission), just a quiet citation: **"grounded in: Photography,
-Lisbon-2026"**. Click it, it expands to the exact section read. This
-is the single highest-leverage UI change in the whole plan: it turns
-an architectural property ("the model is grounded in your vault") into
-a *felt* one, every single turn, for free, using tool-call
-infrastructure that already exists.
-
-**2. A real Memory surface — not a settings toggle, a workspace.** New
-8th tab (or better: reachable via one click from the sidebar, since
-memory is chat-adjacent, not admin-adjacent): a vault browser. Articles
-+ categories on the left, a reading pane on the right, a wikilink graph
-view (the backend already resolves `memory_links` — nobody has drawn
-it). Critically: a **git-history scrubber** on every article, because
-self-healing-through-correction (the wiki updates itself when you
-contradict a prior claim) is our best differentiator-in-the-flesh and
-today it happens with zero visible seam. Correct the model about your
-camera body in a chat on Tuesday; open the Photography article on
-Wednesday and see the diff, dated, with the source conversation linked.
-This is the "watch it happen" answer to ChatGPT/Claude's opaque
-"Manage Memories" list — we don't just show *what* is remembered, we
-show *how it changed and why*, because the whole pipeline runs on the
-user's own disk.
-
-**3. LoRA adapters are a routing story, not a filename dropdown.** The
-current `<select>` lists raw ids (`minicpm5-chunk-final · r16`). The
-redesign gives every adapter a name, a one-line specialty, and a
-*kind*: **memory adapters** (trained nightly from the vault — these are
-the "gets to know you" adapters) versus **task adapters** (coding,
-writing-voice, whatever a user trains deliberately). The composer shows
-a small persistent chip: **"grounded in your Writing adapter"** with a
-one-click swap, and — since the backend already supports it — a
-**stacking** affordance (memory adapter + task adapter together,
-visibly, not just via a hidden `a+b` string). This is where LoRA
-routing and memory literally become the same feature in the UI: a
-memory-LoRA *is* accumulated personalization made concrete in weights,
-and that's a claim no competitor's "persona" or "preset" system
-(LM Studio Presets, Msty Personas, Ollama Modelfiles) can make, because
-none of them adapt weights per user.
-
-**4. Speed is shown, not hidden, because it's the substrate memory runs
-on.** A live perf strip near the composer — tok/s, TTFT, prompt-cache
-hit (already computed server-side, never surfaced), a plain-English
-serving-lane badge ("batched ×3" / "spec-decode active"). This isn't a
-generic benchmarking flex: it's what makes "the model reads your whole
-vault before answering, every time, and it's still instant" credible
-instead of hopeful. Fast local retrieval is worthless if it feels slow.
-
-**5. Tool-call and approval UI as a trust surface for an agent with
-real disk access.** The approval gate is fully built (per-call
-allow/deny, diff preview) but dead in web chat today — the browser
-tool allowlist excludes bash/edit/write entirely. Ship a "Power" mode
-that widens the toolset with the approval cards already built, styled
-as inline code-review hunks (LM Studio's argument-editable pattern,
-Jan's non-blocking inline cards). This matters to the memory thesis
-specifically: a vault the model can *write* to, not just read, is the
-next step after this redesign, and users need to already trust
-approval before we get there.
-
-## Making it tangible in the surface
-
-**Layout.** Keep the two-pane shape (sidebar + thread) — it's not
-broken, and Msty/Claude/ChatGPT converge on it. Change what the sidebar
-*contains*: above the flat session list, a compact **Memory status
-strip** — vault article count, "last synthesis: 6h ago, 3 articles
-updated," a dot that goes amber if synthesis is overdue. This is
-`vaultStatus()` (already returns exactly this data) surfaced for the
-first time. One click opens the Memory workspace as a slide-over, not a
-full tab switch — memory should feel adjacent to chat, not a separate
-destination.
-
-**Composer.** Left of Send: the adapter chip (kind-colored — memory
-adapters warm, task adapters cool), click to open a compact router
-panel (available/loaded/selected, RAM cost per loaded adapter, stack
-toggle). Right of the textarea: the perf strip. Above the composer:
-typing `#` surfaces both attached-file context AND vault articles in
-one unified picker, because "attach a file" and "recall something I
-told you" are the same retrieval gesture from the user's point of view.
-Don't teach two mental models for "give the model context."
-
-**Messages.** Each assistant bubble gets, for the first time, a real
-action row (copy — currently missing outside code blocks, regenerate,
-edit-and-branch with the ChatGPT/Claude convention of a lightweight
-`< 1/2 >` sibling toggle rather than a full tree). The memory-citation
-chip attaches only when memory tools actually fired that turn — never
-fabricated, because false positives here destroy the trust the feature
-depends on.
-
-**Model & adapter switching.** The nav model label goes from a static
-span to a real switcher (currently zero affordance, a stark gap for a
-product built on per-model specialization). The adapter panel sits next
-to it, not buried in a settings modal.
-
-**Tool-call display.** Three visual tiers, not one generic card: (a)
-read-only tools (file read, web fetch) render minimally, collapsed,
-auto-allowed; (b) memory tools render as the soft citation chip from
-differentiator #1 — deliberately *not* styled like a tool call; (c)
-mutating tools (bash/edit/write, once Power mode exists) render as the
-inline approve/deny code-review-hunk card with visible diff. Same
-underlying `tool_call` event stream, three renderers — conflating "read
-your camera preferences" with "wants to delete a file" in one visual
-language is exactly what makes today's UI feel generic.
-
-**Onboarding.** The <1-minute install-to-chat promise gets a second
-clause: install-to-chat-*that-remembers*-under-two-minutes. First-run
-hero adds one honest, skippable memory-consent step ("Let mlx-bun keep
-a personal notes vault on your Mac? Read, edit, or delete it anytime —
-it's just Markdown files") in the same breath as the first message, not
-buried in a CLI-only `mlx-bun memory init`. Consent has to happen
-*before* the flywheel can start proving itself.
-
-## What we deliberately don't build
-
-- **No enterprise/multi-user layer** — no RBAC, no Open-WebUI-style
-  Channels, no admin console. mlx-bun is single-user by identity.
-- **No general-purpose agent-flow builder** (AnythingLLM's Agent Flows,
-  Msty's Turnstiles). Automation, if any, is scoped narrowly to what
-  the synthesis pipeline needs, not a drag-and-drop canvas.
-- **No cloud tier, no hosted anything** — no Ollama-style `:cloud`
-  hybrid. "Escalate to a bigger model" means local spec-decode against
-  a larger local target, never a paid API.
-- **No email/calendar/documents workspace clone** (Odysseus's scope
-  creep). Personal-context connectors, if they ever happen, are
-  read-only memory *feeders*, not a productivity suite to maintain.
-- **No full conversation-tree graph view.** The lightweight
-  sibling-toggle branch UI (ChatGPT/Claude's convention) is the right
-  amount of investment; neither incumbent ships a real tree either.
-- **No page-level PDF citation rendering as v1.** Filename + snippet
-  citation gets most of the trust benefit for a fraction of the
-  engineering; save PDF-aware rendering for later.
-- **No speech-to-speech voice mode as a v1 commitment** — a bolt-on
-  STT→LLM→TTS pipeline reads as worse than what users now expect;
-  scope as future work, not parity-by-next-release.
-- **No unsandboxed shell tool shipped quietly.** If Power mode widens
-  the toolset to bash/edit/write, the approval-gate trust surface
-  ships *with* it, not after — Odysseus's unsandboxed bash tool and
-  its 48-hour-patched CVEs are the cautionary tale.
-- **No competing on raw model-catalog breadth** (Ollama's library,
-  LM Studio's HF browser). MLX-only by identity; a focused, well-labeled
-  local model/adapter list beats a sprawling catalog we'd have to
-  maintain against a moving target.
-
-## Why this ordering, not another
-
-Speed, tool UX, and multi-adapter composition are all worth building —
-the research surfaces excellent, copyable patterns for each (Jan's
-inline approval cards, LM Studio's hardware-fit indicator, Msty's
-split-pane compare). But every one of those patterns is *matchable* by
-a well-resourced competitor within a release cycle. The memory flywheel
-is not: it requires a vault architecture, a synthesis pipeline, an
-adapter-namespaced KV runtime, and a product culture willing to treat
-"gets better at being yours" as the headline rather than a checkbox —
-all of which mlx-bun already has built and simply hasn't surfaced. The
-redesign's job is not to invent new backend capability. It's to stop
-hiding the one capability nothing else in this market has, and frame
-every other improvement (speed HUD, tool cards, adapter switching) as
-infrastructure *for* that capability, not a parallel feature track.

package/docs/archive/planning/chat-ui-vision.md

@@ -1,50 +0,0 @@
-# The mlx-bun chat surface: a vision
-
-*Draft vision, product/design lead pass — 2026-07-06. Bold stance: reimagine the chat surface from scratch. Companion to the market research and current-state audit that produced it; see `docs/archive/planning/web-ui-pass-plan.md` for the tactical bug list and `docs/design/adapters-end-to-end.md` / `docs/design/memory-system.md` for the backend contracts this vision assumes.*
-
-## The feeling
-
-Open mlx-bun and it should feel like sitting down with someone who already knows you — not a search box you interrogate, not a terminal you configure, not a dashboard you administer. Every other local-AI chat UI (Open WebUI, LM Studio, Jan, Ollama's app) is, underneath the polish, the **same product**: a stateless request/response window in front of a model file. They compete on how many knobs they expose. We are not building a better knob panel. We are building the only chat surface where "it remembers what I told it three weeks ago, and it's faster than the thing it's replacing" is literally, verifiably true — and where the UI's job is to make that trust legible turn by turn, not assert it in marketing copy.
-
-`src/web/app.html` has already solved the hard infrastructure problem (streaming, tool-calls, sessions, steering) and completely missed the product problem: memory is invisible, LoRA routing is a bare `<select>`, and the first thing a new user sees is marketing copy about privacy with zero proof of personalization on screen. The redesign's job is not "add more tabs." It's: collapse the surface to what a person actually touches every day (chat), and make the three things nobody else can build — memory that becomes weights, routed adapters, an engine fast enough to show its work — into things you can *see happen*.
-
-## Five things that make this unmistakably not-Open-WebUI
-
-**1. Memory has a face, and it's a citation, not a settings toggle.**
-Every competitor's "memory" (ChatGPT, AnythingLLM, Odysseus) is a flat fact-list re-injected into a system prompt — legible after the fact, invisible during the conversation. We already run `memory_search`/`memory_read` silently on every turn (`src/memory/tools.ts`, wired into `webChatToolAllowlist`) and render them with the same generic wrench icon as `bash`. Instead: when a memory tool fires, render a distinct, quieter card — a small "📖 Photography" chip inline with the response — that expands on click to show the exact vault section that grounded that sentence. No context dump, no "as an AI with access to your notes" narration, just a footnote that proves the flywheel is real. This costs almost nothing: the event stream already distinguishes memory calls; we just have to stop rendering them like `bash`.
-
-**2. Adapters are personas you route to, not files you pick.**
-Nobody else has hot-swappable LoRA adapters with adapter-namespaced KV — not LM Studio, not Ollama, not Msty (whose "Persona Studio" is prompt-only). We already have `/v1/adapters`, three-tier state, and per-request injection; the UI throws it away on a `<select>` full of raw filenames. Instead: a **persona rail** above the composer, one pill per named persona (adapter + system prompt + tool allowlist + memory scope bundled, a richer Ollama Modelfile), each with a one-line specialty, a resident/cold dot, and RAM cost on hover. Switching shows a "new context segment" micro-transition (genuine, not cosmetic — that's what adapter-namespaced KV means), and a Msty-style split-compare fires one prompt at two personas side by side to *prove* the adapter earns its keep.
-
-**3. Speed is a HUD, not a footnote.**
-Today tok/s is a dim gray number after the message finishes. Every other local tool either hides serving mechanics (Open WebUI) or shows only tok/s (Ollama `ps`, LM Studio). We control the whole stack down to spec-decode and batch slots. A persistent strip near the composer: live tok/s *during* the stream, TTFT, a context-fullness ring, and a lane badge — "draft+verify (dspark)" vs "batched ×3" vs "serial" — since mlx-lm's own architecture treats these as mutually exclusive lanes and users should see which one they're in. An internal engineering decision becomes a felt, screenshot-able proof point.
-
-**4. Tool-calls are a code-review hunk, not a permission dialog.**
-We already have per-call approve/deny in `pi-web.ts` with a diff-style card; Jan and LM Studio validate the pattern (inline, non-blocking, argument-editable), nobody's pushed it further. Ours should feel like reviewing a PR: syntax-highlighted args, a blast-radius line ("will write 40 lines to `src/foo.ts`"), three-state approval (once / always-this-session / deny) persisted to disk so it survives restarts (Claude Desktop and Cowork visibly get this wrong), and read-only tools pre-approved by default so fatigue only hits tools that deserve scrutiny.
-
-**5. The whole thing installs to a felt "it knows me" moment in under a minute.**
-Every competitor's onboarding ends at "chat works." Ours ends at "chat works, and it already asked, gently, whether it can start learning from you." First run: pick a hardware-fit-scored starter model, land in chat, and the hero screen's first chip isn't "what is mlx-bun?" — it's a dismissible memory-consent card ("Let mlx-bun remember what matters across chats? Nightly, local, git-tracked, editable any time.") where a philosophy statement used to be. Say yes and the vault seeds; say no and it never mentions it again — the one screen where "local-to-you" becomes an action instead of a claim.
-
-## Making the differentiators tangible, in one line each
-
-**Privacy** = a one-click "Temporary chat" pill, honestly stronger than ChatGPT/Claude's (theirs retains 30 days server-side; ours touches no third party, ever) — stated once in a tooltip, otherwise proven by the UI's total silence about accounts or "sync." **Memory** = the citation chips plus a **Memory panel** (slide-over, not a 6th tab): vault search, per-article git-diff so a user can *watch* an article self-correct, and a nightly synthesis changelog — the mechanism shown, not asserted. **LoRA routing** = the persona rail plus a routing table in settings (which surface defaults to which persona) with the three-tier lifecycle (available/loaded/selected) shown honestly, RAM cost visible before it's spent. **Speed** = the always-on HUD plus a Benchmark strip in Status (live prefill/decode tok/s, TTFT, peak RSS, spec-decode acceptance rate, cold-start — macMLX's pattern, extended with what only dspark can show). **Local tools** = the code-review-hunk approval cards plus a curated first-run toolset (file read, web fetch, our own `memory_search`) so day one isn't a JSON config exercise.
-
-## Concrete screens
-
-**Layout.** Kill the 7-tab flat nav. One primary surface (Chat), with Quantize / Fine-tune / Build Dataset / Curves / Routes / Status collapsed behind a single **Developer** toggle (LM Studio's User/Power-User/Developer pattern, flattened to two states). A new user's day-one mental model: sidebar, thread, composer, persona rail, speed HUD — not seven tabs including a fine-tuning dashboard before they've sent one message.
-
-**Sidebar.** Full-text session search (not just title — unlike Claude's well-known gap), and a real mobile treatment: a slide-over drawer behind a hamburger, not today's "delete the sidebar at 760px with no replacement."
-
-**Thread.** Actions that don't exist today: copy on any assistant message, edit-and-resend that forks non-destructively with an inline `< 1/2 >` branch switcher (a linear toggle, not a tree — both cloud incumbents prove that's the right ceiling), regenerate, and "regenerate with a different persona." Streaming gets fixed architecturally: block-level memoization (only the live tail re-parses) instead of today's `innerHTML = mdToHtml(fullText)` on every token — O(n²) over a response, and the thing most likely to make a spec-decode-accelerated model *feel* slow because the frontend can't keep up with its own backend. Code fences get real syntax highlighting; math gets KaTeX — both zero today.
-
-**Composer.** Autosize textarea, file/image attach (existing), a persona pill, live steering (existing — keep it), and the sampling popover expanded from three sliders to the full server-supported set (min_p, XTC, penalties, logit_bias) behind an "Advanced" disclosure — closing a plain parity gap against the mlx-lm oracle.
-
-**Memory & model switching.** A slide-over Memory panel (vault search, git-diff per article, synthesis changelog) plus a vault-status chip in the composer, and a real model switcher — today's `#nav-model` is a read-only label with no click handler, an odd miss given the pitch is fast tier-switching between e4b/12B/26B.
-
-**Tool-call display.** Inline, non-blocking approval cards (Jan's pattern) with argument diffs, blast-radius text, three-tier persistence, and pre-approved read-only defaults.
-
-## What we deliberately do not build
-
-No Channels or multi-user/Slack-style shared spaces (Open WebUI's most enterprise-flavored feature) — mlx-bun is single-user by identity. No RBAC, org admin, or plugin marketplace — there is no "org" to administer. No general-purpose no-code agent-flow canvas (AnythingLLM's Agent Flows, Msty's Turnstiles) — if one ever justifies itself, scope it to the memory/synthesis pipeline, not general automation. No speech-to-speech voice mode as a v1 commitment — it would visibly disappoint next to Advanced Voice Mode if half-built; an honest stretch goal, not table stakes. No paid cloud escalation tier (Ollama Cloud) — any "bigger model" story stays local (dspark spec-decode against a larger local target). No email/calendar/document-suite bloat (Odysseus), even reframed as "memory feeders," until the core loop is airtight. And no sandbox-theater for a shell tool we can't actually sandbox — if we ship tool execution, we say plainly it runs on your machine with your permissions.
-
-The through-line: every screen either makes local-to-you *provable* or it doesn't ship. Speed you can see. Memory you can read and correct. Adapters you can name, compare, and trust. Everything else is Open WebUI's problem to solve, not ours.

package/docs/archive/planning/chunk-task-roadmap.md

@@ -1,77 +0,0 @@
-# Chunk-task quality roadmap (2026-06-17)
-
-Where the MiniCPM5 chunking adapter stands and what raises real segmentation
-quality. Companion to the chunk-eval metric vs. accuracy finding (score = well-formedness, not gold accuracy) and
-[`docs/archive/investigations/chunk-finetune-experiment.md`](../investigations/chunk-finetune-experiment.md).
-
-## Where we are
-
-- Adapter **solved the output contract**: 0/25 invalid JSON (base model: 22/25
-  invalid — unusable). That's the entire 95.10 well-formedness score.
-- Adapter is **mediocre at the actual task**: vs-gold boundary ~35.6, label ~52.0,
-  all 25 cases in disagreement (`reports/chunk-eval-cpm-2026-06-17.html`).
-- Dominant failures: `under_split` 17/25, TAIL GAP / `missed_content` 9,
-  hallucinated anchors 4 — explained by the training-data bias below.
-
-## Step 1 (the lever): fix the training data
-
-The data (`chunk-data-le4000`) is **86% single-chunk** (626/730 examples = 1 chunk,
-mean 1.16, max 4). The model learned a strong "one short chunk" prior → it
-under-splits and truncates coverage. A 1B model on this diet isn't at its capacity
-ceiling, it's mimicking its data.
-
-- Rebuild the dataset so the **chunk-count distribution matches real operational
-  segmentation**, not a single-topic majority.
-- Teach **full-transcript coverage** (spans that reach the end — kill the tail gaps).
-- This is the #1 lever. Do it before touching capacity or iters.
-
-## Eval methodology: do NOT judge against codex's gold
-
-**Codex over-segments — it splits articles ~2× what we actually do in operations.**
-So the current gold is inflated: judging the adapter as `under_split` is partly an
-artifact of a gold that over-splits. The under-split signal is real but smaller than
-the verdict count suggests.
-
-- Recalibrate the gold to **operational granularity** (use real-ops segmentations as
-  the reference, or down-weight codex's extra splits) before trusting `bs`/`ls`.
-- Until the gold matches ops, the report **understates** the adapter.
-- Going forward, optimize against operational gold, not codex, and not the saturated
-  well-formedness score.
-
-## rank / scale: tune for a single narrow task
-
-We are telling CPM to do exactly **one** thing: take a sequence of messages and find
-the topic seams. That narrowness should drive the LoRA capacity knobs deliberately —
-revisit the inherited rank 16 / scale 1 rather than carrying old defaults. Folds into
-the "configurable training params with mlx-lm defaults" task (rank / scale / num_layers
-/ LR / dropout selectable at run start). The question to answer empirically: how little
-rank does this one transformation need, and what scale/LR pairs with it.
-
-## Streaming / windowed segmentation mode
-
-Whole-transcript-at-once is one framing; it likely isn't the operational one. Explore:
-
-- A **sliding message window** — segment incrementally as messages arrive, over a
-  bounded window, rather than ingesting the entire conversation.
-- "Lots of different ways to segment these conversations" — try multiple task
-  formulations (whole-transcript vs windowed vs streaming/online) and measure which
-  matches operations and trains best on a small model.
-- Implications for training data shape (windowed targets) and for serving (a streaming
-  endpoint / incremental decode).
-
-## Evaluate other base models
-
-CPM5-1B is tiny and was a memory-driven pick. Run a **bake-off**: score other base
-models on the chunk task (same gold, same harness) to find the best base for the
-chunking adapter. Capacity is the second lever after data — and the e4b +
-segmented-backward work exists precisely to train bigger models at long context. See
-[[e4b-lora-training-seqlen-ceiling]], [[segmented-backward-landed]].
-
-## Ordered plan
-
-1. **Fix the training data** (multi-chunk + full coverage, operational granularity).
-2. **Recalibrate the eval gold** off codex toward operational segmentation.
-3. **Tune rank/scale** for the single narrow task (with configurable params).
-4. **Prototype streaming/windowed** segmentation; pick the framing that fits ops.
-5. **Model bake-off**; move up in capacity (e4b via segmented backward) if data alone
-   plateaus.