mlx-bun 0.0.10 → 0.0.12

package/docs/investigations/openwiki-evaluation.md

@@ -0,0 +1,249 @@
+# OpenWiki — evaluation & fit analysis
+
+**Date:** 2026-07-06
+**Author:** investigation (agent), worktree `investigate/openwiki-memory`
+**Subject:** [langchain-ai/openwiki](https://github.com/langchain-ai/openwiki) — "a CLI that writes and
+maintains agent documentation for your codebase, built specifically for agents to consume."
+**Question asked:** how (if at all) does OpenWiki fit inside our memory system ("The Dreaming"), and
+is it a good fit anywhere else in mlx-bun?
+
+---
+
+## TL;DR — four verdicts
+
+| # | Question | Verdict | One-liner |
+|---|----------|---------|-----------|
+| 1 | Adopt OpenWiki **as the memory engine**? | ❌ **No** | Architecturally the mirror-image of The Dreaming; would throw away the hard-won decomposition + gate design and drag LangChain into a Bun/MLX-only repo. |
+| 2 | Borrow OpenWiki **patterns** into memory? | 🟡 **Marginal** | We already independently arrived at its good ideas, usually in stronger form. One minor borrow (the `_plan.md` durable-plan artifact). |
+| 3 | Run OpenWiki **on mlx-bun the repo** (its own agent docs)? | ✅ **Yes, good fit** | It targets exactly our named failure mode — reference-doc drift — and runs as an external dev tool, not a vendored dep. |
+| 4 | Point OpenWiki **at our own OpenAI-compatible server**? | ✅ **Yes — the strategic find** | DeepAgents-over-local-mlx-bun is an on-brand, zero-cloud dogfood showcase *and* a hard agentic-tool-use conformance test for the server. |
+
+The headline: **OpenWiki is not a fit *inside* the memory pipeline, but it is a fit *around* mlx-bun** —
+as a doc tool for our own repo and, more interestingly, as a real LangChain agent to run *against* our
+server to prove the "drop-in, local, agentic" thesis.
+
+---
+
+## 1. What OpenWiki is (full rundown)
+
+A CLI (`npm install -g openwiki`) from LangChain, released **v0.0.1 on 2026-07-05** (i.e. one day old at
+time of writing; ~7.7k stars, but 42 open PRs / 48 issues — treat as brand-new and unstable). MIT
+licensed. TypeScript/JavaScript.
+
+**Purpose.** Generate and maintain a repo's documentation *for coding agents*, not for humans. The
+generated wiki is "structured markdown optimized for LLM context — headings, cross-references, and
+summaries designed for an agent to quickly find relevant context." Motivation from the launch post:
+*"Agents write better code when they understand the repo they're working in,"* and stuffing the whole
+repo into one `CLAUDE.md` wastes context in large repos.
+
+**Surface.**
+```
+openwiki --init            # pick provider + API key, then generate the wiki
+openwiki                   # interactive
+openwiki "<request>"       # interactive, seeded
+openwiki -p "<request>"    # one-shot, non-interactive
+openwiki --update          # refresh from repo changes since last run
+```
+- Writes an `openwiki/` directory of markdown pages.
+- Patches `AGENTS.md` / `CLAUDE.md` with a pointer to the wiki **and a "when to use it" note**.
+- Ships a **GitHub Action** and **GitLab CI** template that open a PR/MR when docs need updating.
+- Config in `~/.openwiki/.env`: provider, API key, model id, optional LangSmith tracing.
+- Providers: OpenRouter (default, an open model), Fireworks, Baseten, OpenAI, Anthropic, and
+  **OpenAI-compatible** via `OPENAI_COMPATIBLE_BASE_URL` (also `ANTHROPIC_BASE_URL`). ← load-bearing
+  for verdict 4.
+
+## 2. How OpenWiki works (architecture)
+
+Built on LangChain's **DeepAgents** + **LangGraph**. The construction is deliberately thin:
+
+```ts
+const agent = createDeepAgent({
+  model,
+  tools: [],                         // NO custom tools
+  checkpointer,                      // thread memory
+  backend: new LocalShellBackend({ /* vfs → repo root */ }),
+  systemPrompt: createSystemPrompt(command),
+});
+```
+
+- **No bespoke tools.** It hands a general coding agent a `LocalShellBackend`: a virtual filesystem
+  mapped to the repo root exposing `ls / read_file / write_file / edit_file / glob / grep` plus a
+  sandboxed `execute` (120 s timeout, 100 KB output cap). All capability comes from the shell backend.
+- **All the IP is in the system prompt + the loop.** The prompt encodes:
+  - **Discovery → plan.** Before writing, the agent writes a temporary `openwiki/_plan.md` listing
+    *intended pages, source evidence per page, and remaining questions.*
+  - **Granularity rules.** ≤~10 primary source files → a `quickstart.md` + at most 1–2 supporting
+    pages. *"Do not create a directory unless it represents a real documentation area."*
+  - **One canonical page per concept.** *"Keep each concept in one canonical page… keep the detailed
+    explanation in the canonical page and make other mentions brief or link-only."* Inline source-file
+    references so a reader can verify.
+  - **Surgical, diff-budgeted updates.** On `--update` it *"checks which commits landed since the last
+    run, uses git diffs to understand what changed, then updates the wiki."* Soft budget: *"if fewer
+    than about 5 source files changed, update at most 1–2 wiki pages… Updates may be a no-op."*
+- **Change detection** by content snapshot (`createOpenWikiContentSnapshot`); persists run metadata
+  only when the wiki actually changed.
+- Streaming LangGraph invocation with a checkpointer keyed by `thread_id` (interactive Q&A over the
+  wiki + codebase reuses the thread).
+
+**Design bet:** a *strong single model* + a *good prompt* + a *free-roaming agent* with raw shell
+tools. Trust the model to hold the repo in its head and write files directly.
+
+## 3. The Dreaming, in one screen (for contrast)
+
+Our memory system ("The Dreaming", `src/memory/*`, ~12 k LoC, running end-to-end on real data):
+
+- **Corpus:** the user's conversation history (pointer-DB of `messages` → `chunks` as
+  `(conv,start,end)` ranges), not a code tree.
+- **Engine:** ONE local model (`gemma-4-e4b-it-OptiQ-4bit`) + per-stage LoRAs, loaded once. No cloud.
+- **Pipeline:** `SEGMENT → EXTRACT → ROUTE → SECTION-ROUTE → PATCH → NORMALIZE → LINK → WIKIFY →
+  RECONCILE → COMMIT`, every stage the *smallest atomic task* (binary yes/no preferred).
+- **Output:** entity-centric wiki — articles are *things*, with infoboxes, wikilinks, categories.
+- **Invariants enforced by code, not the model:** *"LoRAs propose, gates dispose"* — ≥70% prose floor,
+  every citation survives, weak output → NO-OP, deterministic footnote bijection + wikilink
+  canonicalization, one git commit per run.
+- **Single writer:** *"Chat agents read memory; `mlx-bun memory synthesize` is the only writer."* Reads
+  go through 7 read-only `memory_*` tools + a bundled skill.
+- **Self-healing:** chronological walk; latest statement wins; AI errors silently overwritten.
+
+**Design bet:** the exact opposite of OpenWiki — a *weak local model* + *heavy decomposition* +
+*deterministic scaffolding*. Assume the model **cannot** hold an article in its head or be trusted to
+write files freely, and engineer around that.
+
+## 4. Head-to-head: same shape, opposite philosophy
+
+Both are *"corpus → agentic synthesis → LLM-navigable markdown wiki → agent consumes it, kept fresh
+incrementally."* That surface similarity is why the question is worth asking. But under the hood they
+are mirror images:
+
+| Axis | OpenWiki | The Dreaming |
+|------|----------|--------------|
+| Corpus | Code repository | Personal conversation history |
+| Model | Strong cloud model (default OpenRouter) | Weak local e4b + LoRAs, no cloud |
+| Capacity model | Big context — model holds the repo | *Capacity by decomposition* — can't hold an article |
+| Harness | Thin (shell tools + one prompt) | Thick (10-stage DAG, ledgers, gates) |
+| Writes | Agent writes files freely | Single writer, gated, git-committed |
+| Dedup | Prompt rule ("one canonical page") | Deterministic entity name-match routing |
+| Update trigger | `git diff` since last commit | Chronological chunk cursor / new sessions |
+| Self-healing | Re-edit page on diff | Chronological latest-value reconcile |
+| Determinism | Trusts the model | Code enforces invariants |
+| Deps | LangChain + DeepAgents + LangGraph | Bun-native, MLX-only, minimal |
+
+Pithily: **OpenWiki = smart model, dumb harness. The Dreaming = dumb model, smart harness.** They solve
+the same *shape* of problem from opposite ends of the model-capability axis.
+
+## 5. The four fit verdicts, argued
+
+### Verdict 1 — OpenWiki as the memory engine: ❌ No
+
+- **Architectural inversion.** The Dreaming's entire thesis is that a local 4-bit e4b *cannot* do
+  OpenWiki's free-roaming, hold-it-in-context, write-files-directly loop — which is *why* the
+  decomposition + gate architecture exists. Adopting OpenWiki means discarding the exact design that
+  makes local synthesis work.
+- **Contract violation.** OpenWiki's agent writes files freely; The Dreaming has a single gated writer
+  with a git-commit-per-run audit trail and NO-OP-on-weak-output. These are incompatible write models.
+- **Corpus mismatch.** OpenWiki reads a code tree via shell/grep; The Dreaming reads a pointer-DB of
+  conversation chunks with chronological self-healing. Different input, different invariants.
+- **Identity/deps.** Pulling LangChain + LangGraph into a repo whose stated identity is Bun-native,
+  MLX-only, minimal-dep, private (see the `mlx-bun scope & framing` and `drop-in replacement` notes)
+  is a non-starter as a runtime dependency.
+- **Maturity.** v0.0.1, one day old. Nothing load-bearing should sit on it yet.
+
+### Verdict 2 — Borrow OpenWiki's patterns: 🟡 Marginal
+
+We have *convergently* arrived at most of OpenWiki's good ideas, usually stronger:
+
+| OpenWiki idea | Our equivalent | Who's ahead |
+|---------------|----------------|-------------|
+| `_plan.md` before writing | CREATE flow: sub-cluster → outline → draft | ~tie; their durable, reviewable *plan artifact* is a nice touch |
+| Soft diff budget, updates may be no-op | Idempotency ledger + conservative gate + NO-OP | **us** (ledger is stronger than a prompt heuristic) |
+| One canonical page (prompt rule) | Deterministic entity name-match routing | **us** (mechanism > instruction) |
+| Patch `AGENTS.md`/`CLAUDE.md` with pointer | 7 read-only `memory_*` tools + bundled skill | **us** (tools > prose pointer) |
+| git-diff-driven refresh | chronological chunk cursor | different corpus; both valid |
+
+**Only genuinely fresh borrow:** emit a durable, auditable **plan artifact** (their `_plan.md`) in the
+CREATE flow, so a synthesis run's *intended* article/section changes are reviewable in the git diff
+alongside the result. Low effort, small upside. Not worth a dedicated work item; fold in opportunistically.
+
+### Verdict 3 — Run OpenWiki on mlx-bun the repo: ✅ Good fit
+
+This is where OpenWiki was literally built to help, and mlx-bun is close to a canonical target:
+
+- **It attacks our named failure mode.** `CLAUDE.md` calls reference-doc drift *"the observed failure
+  mode"* (the 2026-07-03 sweep found undocumented flags, dead version pins). The doc-map even carries a
+  manual chore: *"keep these lists in sync with `ls docs/…`."* OpenWiki's diff-driven refresh + CI-PR is
+  exactly this job.
+- **Additive, not a replacement.** It would generate a *derived, agent-facing structural index of the
+  code* — it does **not** replace the curated, opinionated human docs (`PLAN.md` is "the engineering
+  contract"; `STATUS.md`, design docs). Think: an auto-maintained `openwiki/` map of `src/` that points
+  back into the hand-written design docs.
+- **Nice synergy with the vault.** The memory vault already symlinks mlx-bun docs into
+  `~/.mlx-bun/wiki/Reference/` (read-only). An auto-maintained repo wiki would flow straight into that
+  Reference surface — the personal vault gets a fresher code map for free.
+- **Runs as an external dev tool**, a global npm CLI in CI — *not* a vendored dependency. No LangChain
+  contamination of the shipped codebase. This sidesteps the Verdict-1 dependency objection entirely.
+
+**Caveat:** the one tension is "cloud API on a local/private repo." Verdict 4 dissolves it.
+
+### Verdict 4 — Point OpenWiki at our own OpenAI-compatible server: ✅ The strategic find
+
+OpenWiki speaks **OpenAI-compatible base URL**; mlx-bun *is* an OpenAI-compatible server (the whole
+"drop-in replacement for mlx-lm / serving speed is the user metric" thesis). Compose them:
+
+```
+OPENAI_COMPATIBLE_BASE_URL=http://localhost:<mlx-bun-port> \
+OPENAI_COMPATIBLE_API_KEY=… openwiki --init   # generate mlx-bun's own wiki, 100% local
+```
+
+Why this is the most interesting outcome of the investigation:
+
+1. **On-brand, zero-cloud.** Resolves the Verdict-3 tension: a real LangChain agent documents our repo
+   with *no* cloud call, on a Gemma running in mlx-bun. Directly demonstrates the `local-to-you` and
+   `install-to-chat` theses on an external, third-party agent.
+2. **A hard conformance test.** DeepAgents drives many tool calls + long contexts + streaming. Chat
+   parity is easy; *agentic tool-use conformance* is the bar that actually matters for "agents need the
+   server" (`serving speed is the user metric`). If OpenWiki runs unmodified end-to-end against our
+   server, that's a strong, concrete proof point for the README/website. If it *doesn't*, we've found a
+   real function-calling / prefill / KV gap under agentic load — also valuable.
+3. **Coverage/stress.** It exercises the server the way a coding agent does (glob/grep/read/write loops,
+   big prefills), complementing the batch/spec-decode perf work with a realistic agentic workload.
+
+**Caveat to measure, not assume:** DeepAgents presumes a *capable* model — the very assumption The
+Dreaming rejects for local e4b. A 4-bit e4b may drive the free-roaming loop poorly and produce a
+mediocre wiki. Mitigations: point it at a larger local model (the 12B parity target) via the same
+server, and/or judge output quality explicitly. Either way the *conformance* result (does the loop run?)
+is decoupled from the *quality* result (is the wiki good?), and both are worth knowing.
+
+## 6. Recommendation — what to actually do
+
+1. **Do not** put OpenWiki inside the memory pipeline. The Dreaming is more mature for its constraints
+   and philosophically opposite. (Verdicts 1 & 2.)
+2. **Spike Verdict 4 first — it's the highest-information, lowest-cost move.** Point OpenWiki at a
+   running mlx-bun server (Josh runs the server; no session starts it) and try `openwiki --init` on a
+   small repo, then on mlx-bun itself. Outcome is binary-useful: either a showcase demo of
+   "third-party LangChain agent runs locally on mlx-bun," or a concrete agentic-tool-use bug list for
+   the server. Bench it under the standard preflight if any perf number gets quoted
+   (`dirty-machine-numbers-are-garbage`).
+3. **If the spike is clean,** consider OpenWiki as an *external CI doc tool* for the repo (Verdict 3):
+   an auto-maintained `openwiki/` structural map that feeds the vault's `Reference/` surface. Keep it
+   clearly subordinate to the hand-written `PLAN.md`/design docs.
+4. **Opportunistically** add a durable `_plan.md`-style plan artifact to the memory CREATE flow for
+   reviewability (Verdict 2). Not a standalone work item.
+
+## 7. Risks & unknowns
+
+- OpenWiki is **v0.0.1, ~1 day old**, 42 open PRs / 48 issues — API and behavior will churn. Pin a
+  commit if used in CI.
+- **Local-model quality** driving DeepAgents is unproven (contradicts our own capacity-by-decomposition
+  finding). Measure; don't assume the wiki is good just because the loop runs.
+- **DeepAgents ⇄ our server function-calling compatibility** is the actual thing under test in Verdict
+  4 — tool-call formatting, streaming, and long-prefill behavior are where it could break.
+- Do **not** treat any OpenWiki-generated wiki as a source of truth over the curated docs; it's a
+  derived index.
+
+## Sources
+
+- [langchain-ai/openwiki](https://github.com/langchain-ai/openwiki) — repo, README, `src/agent/*`
+- [LangChain blog — Introducing OpenWiki](https://www.langchain.com/blog/introducing-openwiki-an-open-source-agent-for-repo-documentation)
+- Internal: `docs/design/memory-system.md`, `memory-synthesis.md`, `memory-inference-path.md`,
+  `write-pipeline-entity.md`, `the-dreaming-master-plan.md`, `the-dreaming-handoff.md`,
+  `docs/reference/memory.md`; `src/memory/*` (~12 k LoC).

package/docs/planning/IDEAS.md

@@ -53,3 +53,17 @@ persistent memory of *you*; mlx-bun is the local, private model runtime.
 Combined: a model that runs entirely on your machine **and** is continuously
 grounded in your own accumulated context — local in both senses, and yours in a
 way no hosted frontier model can be.
+
+
+## Library/app split (idea only — NOT planned, Josh 2026-07-05)
+
+Two pulls: "one executable, everything you want to do with local AI" (the
+product's charm) vs "some people just want a JS ML library — do they need
+the rest?" The runtime-isolation work (2026-07-05) drew the exact boundary
+either future needs: the engine child = the GPU stuff (src/model, src/mlx,
+caches, scheduler, generate); the parent = the app (serve/UI/registry/
+jobs/proxy). If a split ever happens it is a PACKAGING decision, not a
+refactor — e.g. npm subpath exports (mlx-bun/model vs the full CLI bundle)
+so library users import only the core. Library usage is unaffected by
+isolation today (in-process generate(), no server involved). Revisit only
+if Josh decides; both directions stay open from the current structure.

package/docs/planning/PRODUCT_ROADMAP.md

@@ -171,7 +171,6 @@ Each profile should eventually carry:
   kv: "mixed",
   memoryBudgetBytes: "...",
   compiledDecode: true,
-  perfKernel: false,
   expectedDecodeTps: 30,
   expectedTtftMs: 100,
   label: "Recommended for this Mac",
@@ -309,8 +308,8 @@ to text-only mode when no image is present, so the chat UI gains image support
 with no startup cost. Features are within ~1% rel-RMSE of the optiq oracle;
 every MLX primitive (rms_norm, gelu, matmul, SDPA, RoPE, pooling) is bit-exact.
 The residual is sub-bf16 floating-point accumulation across the deep encoder,
-not a porting gap. L1/L2/L3 paths all work; vision falls back to the monolith
-path on L1 and L2.
+not a porting gap. The L1 and L2 paths both work; vision falls back to the
+monolith path on L1 and L2.
 
 ### Adapter web selector and CLI /v1/adapters (shipped)
 

package/docs/planning/chat-ui-vision-memory-first.md

@@ -0,0 +1,199 @@
+# Chat UI Vision: The Model That Knows You
+
+**Stance: memory/personalization-first.** Every other axis in this
+doc — speed, tools, LoRA routing, privacy — is real, but it is in
+service of one idea: mlx-bun is the only local chat surface where the
+model visibly, auditably gets better at being *yours* over time. If we
+build a fast, private, tool-capable chat app that doesn't make that
+tangible on screen, we've built LM Studio with worse artifacts. That is
+not the plan.
+
+## The feeling
+
+Open the app. It's not a hero screen with generic suggestion chips —
+it's a hero screen that already knows something. Not in a creepy way:
+in the way a good assistant who's worked with you for months starts a
+Monday with "want me to pick up the Lisbon itinerary thing from
+Thursday?" instead of "How can I help you today?" It should feel like
+walking into a room where someone has been quietly paying attention,
+not opening a fresh terminal.
+
+The tell, mechanically: a chat with mlx-bun after a month of use is a
+*different product* from day one — not because we shipped a new model,
+but because the sidebar, the hero, and the answers themselves carry
+accumulated context the user can see, correct, and trust. Cloud
+chatbots fake a version of this behind a black box on someone else's
+server. Local competitors (LM Studio, Ollama, macMLX) don't attempt
+it — stateless between chats by design. Open WebUI and AnythingLLM
+attempt it as prompt-stuffing into a flat fact list. Odysseus gets
+closest (memory + skills) but on vLLM/llama.cpp infra with no adapter
+concept at all. **Nobody combines memory-as-training-artifact with
+weight-level LoRA routing and native MLX speed. That combination is the
+whole pitch, and today it's invisible in our own UI** — the vault, the
+memory tools, and the adapter runtime already exist server-side; none
+of it shows on screen.
+
+## The five things that make this unmistakable
+
+**1. Memory is a visible citizen of the chat, not a silent tool call.**
+Today `memory_search`/`memory_read` fire invisibly inside a generic
+tool-call card indistinguishable from a bash command. The redesign
+gives memory its own visual language: when the model consults the
+vault mid-answer, a small soft-colored chip appears inline — not a
+context dump, not an approval gate (it's read-only, never needs
+permission), just a quiet citation: **"grounded in: Photography,
+Lisbon-2026"**. Click it, it expands to the exact section read. This
+is the single highest-leverage UI change in the whole plan: it turns
+an architectural property ("the model is grounded in your vault") into
+a *felt* one, every single turn, for free, using tool-call
+infrastructure that already exists.
+
+**2. A real Memory surface — not a settings toggle, a workspace.** New
+8th tab (or better: reachable via one click from the sidebar, since
+memory is chat-adjacent, not admin-adjacent): a vault browser. Articles
++ categories on the left, a reading pane on the right, a wikilink graph
+view (the backend already resolves `memory_links` — nobody has drawn
+it). Critically: a **git-history scrubber** on every article, because
+self-healing-through-correction (the wiki updates itself when you
+contradict a prior claim) is our best differentiator-in-the-flesh and
+today it happens with zero visible seam. Correct the model about your
+camera body in a chat on Tuesday; open the Photography article on
+Wednesday and see the diff, dated, with the source conversation linked.
+This is the "watch it happen" answer to ChatGPT/Claude's opaque
+"Manage Memories" list — we don't just show *what* is remembered, we
+show *how it changed and why*, because the whole pipeline runs on the
+user's own disk.
+
+**3. LoRA adapters are a routing story, not a filename dropdown.** The
+current `<select>` lists raw ids (`minicpm5-chunk-final · r16`). The
+redesign gives every adapter a name, a one-line specialty, and a
+*kind*: **memory adapters** (trained nightly from the vault — these are
+the "gets to know you" adapters) versus **task adapters** (coding,
+writing-voice, whatever a user trains deliberately). The composer shows
+a small persistent chip: **"grounded in your Writing adapter"** with a
+one-click swap, and — since the backend already supports it — a
+**stacking** affordance (memory adapter + task adapter together,
+visibly, not just via a hidden `a+b` string). This is where LoRA
+routing and memory literally become the same feature in the UI: a
+memory-LoRA *is* accumulated personalization made concrete in weights,
+and that's a claim no competitor's "persona" or "preset" system
+(LM Studio Presets, Msty Personas, Ollama Modelfiles) can make, because
+none of them adapt weights per user.
+
+**4. Speed is shown, not hidden, because it's the substrate memory runs
+on.** A live perf strip near the composer — tok/s, TTFT, prompt-cache
+hit (already computed server-side, never surfaced), a plain-English
+serving-lane badge ("batched ×3" / "spec-decode active"). This isn't a
+generic benchmarking flex: it's what makes "the model reads your whole
+vault before answering, every time, and it's still instant" credible
+instead of hopeful. Fast local retrieval is worthless if it feels slow.
+
+**5. Tool-call and approval UI as a trust surface for an agent with
+real disk access.** The approval gate is fully built (per-call
+allow/deny, diff preview) but dead in web chat today — the browser
+tool allowlist excludes bash/edit/write entirely. Ship a "Power" mode
+that widens the toolset with the approval cards already built, styled
+as inline code-review hunks (LM Studio's argument-editable pattern,
+Jan's non-blocking inline cards). This matters to the memory thesis
+specifically: a vault the model can *write* to, not just read, is the
+next step after this redesign, and users need to already trust
+approval before we get there.
+
+## Making it tangible in the surface
+
+**Layout.** Keep the two-pane shape (sidebar + thread) — it's not
+broken, and Msty/Claude/ChatGPT converge on it. Change what the sidebar
+*contains*: above the flat session list, a compact **Memory status
+strip** — vault article count, "last synthesis: 6h ago, 3 articles
+updated," a dot that goes amber if synthesis is overdue. This is
+`vaultStatus()` (already returns exactly this data) surfaced for the
+first time. One click opens the Memory workspace as a slide-over, not a
+full tab switch — memory should feel adjacent to chat, not a separate
+destination.
+
+**Composer.** Left of Send: the adapter chip (kind-colored — memory
+adapters warm, task adapters cool), click to open a compact router
+panel (available/loaded/selected, RAM cost per loaded adapter, stack
+toggle). Right of the textarea: the perf strip. Above the composer:
+typing `#` surfaces both attached-file context AND vault articles in
+one unified picker, because "attach a file" and "recall something I
+told you" are the same retrieval gesture from the user's point of view.
+Don't teach two mental models for "give the model context."
+
+**Messages.** Each assistant bubble gets, for the first time, a real
+action row (copy — currently missing outside code blocks, regenerate,
+edit-and-branch with the ChatGPT/Claude convention of a lightweight
+`< 1/2 >` sibling toggle rather than a full tree). The memory-citation
+chip attaches only when memory tools actually fired that turn — never
+fabricated, because false positives here destroy the trust the feature
+depends on.
+
+**Model & adapter switching.** The nav model label goes from a static
+span to a real switcher (currently zero affordance, a stark gap for a
+product built on per-model specialization). The adapter panel sits next
+to it, not buried in a settings modal.
+
+**Tool-call display.** Three visual tiers, not one generic card: (a)
+read-only tools (file read, web fetch) render minimally, collapsed,
+auto-allowed; (b) memory tools render as the soft citation chip from
+differentiator #1 — deliberately *not* styled like a tool call; (c)
+mutating tools (bash/edit/write, once Power mode exists) render as the
+inline approve/deny code-review-hunk card with visible diff. Same
+underlying `tool_call` event stream, three renderers — conflating "read
+your camera preferences" with "wants to delete a file" in one visual
+language is exactly what makes today's UI feel generic.
+
+**Onboarding.** The <1-minute install-to-chat promise gets a second
+clause: install-to-chat-*that-remembers*-under-two-minutes. First-run
+hero adds one honest, skippable memory-consent step ("Let mlx-bun keep
+a personal notes vault on your Mac? Read, edit, or delete it anytime —
+it's just Markdown files") in the same breath as the first message, not
+buried in a CLI-only `mlx-bun memory init`. Consent has to happen
+*before* the flywheel can start proving itself.
+
+## What we deliberately don't build
+
+- **No enterprise/multi-user layer** — no RBAC, no Open-WebUI-style
+  Channels, no admin console. mlx-bun is single-user by identity.
+- **No general-purpose agent-flow builder** (AnythingLLM's Agent Flows,
+  Msty's Turnstiles). Automation, if any, is scoped narrowly to what
+  the synthesis pipeline needs, not a drag-and-drop canvas.
+- **No cloud tier, no hosted anything** — no Ollama-style `:cloud`
+  hybrid. "Escalate to a bigger model" means local spec-decode against
+  a larger local target, never a paid API.
+- **No email/calendar/documents workspace clone** (Odysseus's scope
+  creep). Personal-context connectors, if they ever happen, are
+  read-only memory *feeders*, not a productivity suite to maintain.
+- **No full conversation-tree graph view.** The lightweight
+  sibling-toggle branch UI (ChatGPT/Claude's convention) is the right
+  amount of investment; neither incumbent ships a real tree either.
+- **No page-level PDF citation rendering as v1.** Filename + snippet
+  citation gets most of the trust benefit for a fraction of the
+  engineering; save PDF-aware rendering for later.
+- **No speech-to-speech voice mode as a v1 commitment** — a bolt-on
+  STT→LLM→TTS pipeline reads as worse than what users now expect;
+  scope as future work, not parity-by-next-release.
+- **No unsandboxed shell tool shipped quietly.** If Power mode widens
+  the toolset to bash/edit/write, the approval-gate trust surface
+  ships *with* it, not after — Odysseus's unsandboxed bash tool and
+  its 48-hour-patched CVEs are the cautionary tale.
+- **No competing on raw model-catalog breadth** (Ollama's library,
+  LM Studio's HF browser). MLX-only by identity; a focused, well-labeled
+  local model/adapter list beats a sprawling catalog we'd have to
+  maintain against a moving target.
+
+## Why this ordering, not another
+
+Speed, tool UX, and multi-adapter composition are all worth building —
+the research surfaces excellent, copyable patterns for each (Jan's
+inline approval cards, LM Studio's hardware-fit indicator, Msty's
+split-pane compare). But every one of those patterns is *matchable* by
+a well-resourced competitor within a release cycle. The memory flywheel
+is not: it requires a vault architecture, a synthesis pipeline, an
+adapter-namespaced KV runtime, and a product culture willing to treat
+"gets better at being yours" as the headline rather than a checkbox —
+all of which mlx-bun already has built and simply hasn't surfaced. The
+redesign's job is not to invent new backend capability. It's to stop
+hiding the one capability nothing else in this market has, and frame
+every other improvement (speed HUD, tool cards, adapter switching) as
+infrastructure *for* that capability, not a parallel feature track.

package/docs/planning/chat-ui-vision.md

@@ -0,0 +1,50 @@
+# The mlx-bun chat surface: a vision
+
+*Draft vision, product/design lead pass — 2026-07-06. Bold stance: reimagine the chat surface from scratch. Companion to the market research and current-state audit that produced it; see `docs/planning/web-ui-pass-plan.md` for the tactical bug list and `docs/design/adapters-end-to-end.md` / `docs/design/memory-system.md` for the backend contracts this vision assumes.*
+
+## The feeling
+
+Open mlx-bun and it should feel like sitting down with someone who already knows you — not a search box you interrogate, not a terminal you configure, not a dashboard you administer. Every other local-AI chat UI (Open WebUI, LM Studio, Jan, Ollama's app) is, underneath the polish, the **same product**: a stateless request/response window in front of a model file. They compete on how many knobs they expose. We are not building a better knob panel. We are building the only chat surface where "it remembers what I told it three weeks ago, and it's faster than the thing it's replacing" is literally, verifiably true — and where the UI's job is to make that trust legible turn by turn, not assert it in marketing copy.
+
+`src/web/app.html` has already solved the hard infrastructure problem (streaming, tool-calls, sessions, steering) and completely missed the product problem: memory is invisible, LoRA routing is a bare `<select>`, and the first thing a new user sees is marketing copy about privacy with zero proof of personalization on screen. The redesign's job is not "add more tabs." It's: collapse the surface to what a person actually touches every day (chat), and make the three things nobody else can build — memory that becomes weights, routed adapters, an engine fast enough to show its work — into things you can *see happen*.
+
+## Five things that make this unmistakably not-Open-WebUI
+
+**1. Memory has a face, and it's a citation, not a settings toggle.**
+Every competitor's "memory" (ChatGPT, AnythingLLM, Odysseus) is a flat fact-list re-injected into a system prompt — legible after the fact, invisible during the conversation. We already run `memory_search`/`memory_read` silently on every turn (`src/memory/tools.ts`, wired into `webChatToolAllowlist`) and render them with the same generic wrench icon as `bash`. Instead: when a memory tool fires, render a distinct, quieter card — a small "📖 Photography" chip inline with the response — that expands on click to show the exact vault section that grounded that sentence. No context dump, no "as an AI with access to your notes" narration, just a footnote that proves the flywheel is real. This costs almost nothing: the event stream already distinguishes memory calls; we just have to stop rendering them like `bash`.
+
+**2. Adapters are personas you route to, not files you pick.**
+Nobody else has hot-swappable LoRA adapters with adapter-namespaced KV — not LM Studio, not Ollama, not Msty (whose "Persona Studio" is prompt-only). We already have `/v1/adapters`, three-tier state, and per-request injection; the UI throws it away on a `<select>` full of raw filenames. Instead: a **persona rail** above the composer, one pill per named persona (adapter + system prompt + tool allowlist + memory scope bundled, a richer Ollama Modelfile), each with a one-line specialty, a resident/cold dot, and RAM cost on hover. Switching shows a "new context segment" micro-transition (genuine, not cosmetic — that's what adapter-namespaced KV means), and a Msty-style split-compare fires one prompt at two personas side by side to *prove* the adapter earns its keep.
+
+**3. Speed is a HUD, not a footnote.**
+Today tok/s is a dim gray number after the message finishes. Every other local tool either hides serving mechanics (Open WebUI) or shows only tok/s (Ollama `ps`, LM Studio). We control the whole stack down to spec-decode and batch slots. A persistent strip near the composer: live tok/s *during* the stream, TTFT, a context-fullness ring, and a lane badge — "draft+verify (dspark)" vs "batched ×3" vs "serial" — since mlx-lm's own architecture treats these as mutually exclusive lanes and users should see which one they're in. An internal engineering decision becomes a felt, screenshot-able proof point.
+
+**4. Tool-calls are a code-review hunk, not a permission dialog.**
+We already have per-call approve/deny in `pi-web.ts` with a diff-style card; Jan and LM Studio validate the pattern (inline, non-blocking, argument-editable), nobody's pushed it further. Ours should feel like reviewing a PR: syntax-highlighted args, a blast-radius line ("will write 40 lines to `src/foo.ts`"), three-state approval (once / always-this-session / deny) persisted to disk so it survives restarts (Claude Desktop and Cowork visibly get this wrong), and read-only tools pre-approved by default so fatigue only hits tools that deserve scrutiny.
+
+**5. The whole thing installs to a felt "it knows me" moment in under a minute.**
+Every competitor's onboarding ends at "chat works." Ours ends at "chat works, and it already asked, gently, whether it can start learning from you." First run: pick a hardware-fit-scored starter model, land in chat, and the hero screen's first chip isn't "what is mlx-bun?" — it's a dismissible memory-consent card ("Let mlx-bun remember what matters across chats? Nightly, local, git-tracked, editable any time.") where a philosophy statement used to be. Say yes and the vault seeds; say no and it never mentions it again — the one screen where "local-to-you" becomes an action instead of a claim.
+
+## Making the differentiators tangible, in one line each
+
+**Privacy** = a one-click "Temporary chat" pill, honestly stronger than ChatGPT/Claude's (theirs retains 30 days server-side; ours touches no third party, ever) — stated once in a tooltip, otherwise proven by the UI's total silence about accounts or "sync." **Memory** = the citation chips plus a **Memory panel** (slide-over, not a 6th tab): vault search, per-article git-diff so a user can *watch* an article self-correct, and a nightly synthesis changelog — the mechanism shown, not asserted. **LoRA routing** = the persona rail plus a routing table in settings (which surface defaults to which persona) with the three-tier lifecycle (available/loaded/selected) shown honestly, RAM cost visible before it's spent. **Speed** = the always-on HUD plus a Benchmark strip in Status (live prefill/decode tok/s, TTFT, peak RSS, spec-decode acceptance rate, cold-start — macMLX's pattern, extended with what only dspark can show). **Local tools** = the code-review-hunk approval cards plus a curated first-run toolset (file read, web fetch, our own `memory_search`) so day one isn't a JSON config exercise.
+
+## Concrete screens
+
+**Layout.** Kill the 7-tab flat nav. One primary surface (Chat), with Quantize / Fine-tune / Build Dataset / Curves / Routes / Status collapsed behind a single **Developer** toggle (LM Studio's User/Power-User/Developer pattern, flattened to two states). A new user's day-one mental model: sidebar, thread, composer, persona rail, speed HUD — not seven tabs including a fine-tuning dashboard before they've sent one message.
+
+**Sidebar.** Full-text session search (not just title — unlike Claude's well-known gap), and a real mobile treatment: a slide-over drawer behind a hamburger, not today's "delete the sidebar at 760px with no replacement."
+
+**Thread.** Actions that don't exist today: copy on any assistant message, edit-and-resend that forks non-destructively with an inline `< 1/2 >` branch switcher (a linear toggle, not a tree — both cloud incumbents prove that's the right ceiling), regenerate, and "regenerate with a different persona." Streaming gets fixed architecturally: block-level memoization (only the live tail re-parses) instead of today's `innerHTML = mdToHtml(fullText)` on every token — O(n²) over a response, and the thing most likely to make a spec-decode-accelerated model *feel* slow because the frontend can't keep up with its own backend. Code fences get real syntax highlighting; math gets KaTeX — both zero today.
+
+**Composer.** Autosize textarea, file/image attach (existing), a persona pill, live steering (existing — keep it), and the sampling popover expanded from three sliders to the full server-supported set (min_p, XTC, penalties, logit_bias) behind an "Advanced" disclosure — closing a plain parity gap against the mlx-lm oracle.
+
+**Memory & model switching.** A slide-over Memory panel (vault search, git-diff per article, synthesis changelog) plus a vault-status chip in the composer, and a real model switcher — today's `#nav-model` is a read-only label with no click handler, an odd miss given the pitch is fast tier-switching between e4b/12B/26B.
+
+**Tool-call display.** Inline, non-blocking approval cards (Jan's pattern) with argument diffs, blast-radius text, three-tier persistence, and pre-approved read-only defaults.
+
+## What we deliberately do not build
+
+No Channels or multi-user/Slack-style shared spaces (Open WebUI's most enterprise-flavored feature) — mlx-bun is single-user by identity. No RBAC, org admin, or plugin marketplace — there is no "org" to administer. No general-purpose no-code agent-flow canvas (AnythingLLM's Agent Flows, Msty's Turnstiles) — if one ever justifies itself, scope it to the memory/synthesis pipeline, not general automation. No speech-to-speech voice mode as a v1 commitment — it would visibly disappoint next to Advanced Voice Mode if half-built; an honest stretch goal, not table stakes. No paid cloud escalation tier (Ollama Cloud) — any "bigger model" story stays local (dspark spec-decode against a larger local target). No email/calendar/document-suite bloat (Odysseus), even reframed as "memory feeders," until the core loop is airtight. And no sandbox-theater for a shell tool we can't actually sandbox — if we ship tool execution, we say plainly it runs on your machine with your permissions.
+
+The through-line: every screen either makes local-to-you *provable* or it doesn't ship. Speed you can see. Memory you can read and correct. Adapters you can name, compare, and trust. Everything else is Open WebUI's problem to solve, not ours.

package/docs/planning/memory-docs-and-dag-plan.md

@@ -83,6 +83,14 @@ one scope-fix + one hub table + reading order**, all cheap and link-stable:
 
 ## B. The DAG view (docs/dag/training-inference-map.html + parity-tier-dag.md)
 
+> **2026-07-05 annotation:** the Phase-1 deletion pass
+> ([unified-engine-frontier-plan.md §6](../design/unified-engine-frontier-plan.md))
+> deleted the perf-kernel, fused-decode, and fused-swiglu/geglu kernels and
+> removed `--l3` as a product mode (output-changing experiments are Lab items
+> now). The `perfKernelEnabled()` / `fusedDecodeSdpa` / `fusedSwiglu` route
+> predicates referenced in this section (and the map's n92/n95 branches) no
+> longer exist in the code; treat those mentions as the pre-deletion snapshot.
+
 ### B.1 What exists today
 
 - **parity-tier-dag.md** — the framework doc. **Already refreshed 2026-07-01**:
@@ -133,14 +141,14 @@ Keep it hand-authored, make it honest:
 **B3.2 Runtime route explainer (recommended first "derive from code" step,
 ~1 day).** Josh's ask is literally "for each model and config, how it is
 actually running" — the cheapest ground-truth answer is not static analysis
-but a CLI verb, e.g. `mlx-bun route <model> [--l1|--l2|--l3] [--kv-quant …]`,
+but a CLI verb, e.g. `mlx-bun route <model> [--l1|--l2] [--kv-quant …]`,
 that:
 - runs `applyDecodeRoute()` (cli.ts:723 — the TIERS preset table is already a
   machine-readable matrix), loads **config only** (no weights),
-- evaluates the actual exported predicates (`perfKernelEnabled`,
-  `fusedDecodeKernelSupported`, `fusedSwiglu/GeluEnabled+Supported`,
-  `NO_FUSED_SDPA`, `CompiledDecode.for` eligibility, kv-quant resolution,
-  factory ladder selection),
+- evaluates the actual exported predicates (`NO_FUSED_SDPA`,
+  `CompiledDecode.for` eligibility, kv-quant resolution, factory ladder
+  selection — the old perf-kernel/fused-decode/fused-swiglu predicates were
+  deleted 2026-07-05),
 - prints the resolved route as a tree/JSON: which model class, which sdpa
   path, which kernels, which tier each hop sits in, and **which flag would
   change each hop**.
@@ -164,7 +172,8 @@ What "derive the DAG from code" concretely means here:
   generator fails if a tagged node vanished (renames get caught).
 - **CI gate**: a workflow step asserts every **L1-tagged** node names an
   `oracleTest` file that exists and passes (`bun test <file>`); an L1 node
-  without a passing bit-exact test fails CI or must be re-tagged L3. This is
+  without a passing bit-exact test fails CI or must be re-tagged no-oracle
+  (Lab-gated). This is
   PLAN's "the oracle is the gate becomes a check" box — and it composes with
   STATUS finding #4 (no CI at all): land the `tsc + bun test` gate first,
   hang the L1-tag check off it.