mlx-bun 0.2.0 → 0.4.0

package/docs/planning/PRODUCT_ROADMAP.md

@@ -6,6 +6,22 @@ the evolving UI/UX/DX north star.
 
 ## Product north star
 
+Josh's guiding principle for the entire project is "The absolute BEST local
+AI experience on a mac". This governs installation, application development,
+everyday use and inference performance.
+
+Offer multiple simple installation methods without asking users to manage
+a Python virtual environment. Let developers import local AI into their
+applications without managing a separate inference sidecar. Make JavaScript
+and TypeScript first-class ways to build with local AI, without requiring
+Python. Make more capable models practical on the same Mac and reduce the
+time users wait for successful results.
+
+Every workstream must serve that experience. Evaluate engineering choices
+by the setup, integration and operating burden they remove, as well as the
+quality, reliability, speed and hardware efficiency they deliver. Research
+and publication can support the project; user benefit determines priority.
+
 Make the answer to "what is the best way to use local AI on my Mac?"
 boringly simple:
 
@@ -343,7 +359,7 @@ into its own persistent child process behind the existing
 API + routing, zero MLX calls); one GPU lease serializes all GPU consumers
 (chat, API, quantize, finetune) — reusing the spawn/lease pattern jobs
 already prove. Converges with model hot-swap (model lives off the main
-thread either way). Full design + phasing: `docs/design/runtime-isolation.md`.
+thread either way). Full design + phasing: `docs/reference/server-config.md`.
 
 ## Harness UX
 
@@ -543,3 +559,36 @@ reuse across pipeline stages.
 - **What is the right install path?** All of the above — shipped: npm
   publish (v0.0.4, `bunx mlx-bun`), Homebrew release pipeline (sign,
   notarize, `joshuarossi/homebrew-tap`), and GitHub release binary.
+
+## Vision notes (merged from IDEAS.md, 2026-08-23)
+
+### Local in the deeper sense
+
+Two phrases that capture the thesis — especially **mlx-bun × Lucien**:
+
+1. **"A truly local model in a deeper sense."** Not just local *compute*
+   (runs on your Mac, nothing leaves it) — local to *you*: grounded in your
+   own context and memory, not only your hardware.
+2. **"The best model at being yours."** Not the best model in the absolute /
+   benchmark sense — the best at *being yours*. A small local model,
+   fine-tuned and wired to your personal memory, can lose every general
+   benchmark and still win the only one that matters here.
+
+Lucien (the Dreaming) is the synthesized, persistent memory of *you*;
+mlx-bun is the local, private model runtime. Combined: a model that runs
+entirely on your machine **and** is continuously grounded in your own
+accumulated context — local in both senses, and yours in a way no hosted
+frontier model can be.
+
+### Library/app split (idea only — NOT planned, Josh 2026-07-05)
+
+Two pulls: "one executable, everything you want to do with local AI" (the
+product's charm) vs "some people just want a JS ML library — do they need
+the rest?" The runtime-isolation work (2026-07-05) drew the exact boundary
+either future needs: the engine child = the GPU stuff (src/model, src/mlx,
+caches, scheduler, generate); the parent = the app (serve/UI/registry/
+jobs/proxy). If a split ever happens it is a PACKAGING decision, not a
+refactor — e.g. npm subpath exports (mlx-bun/model vs the full CLI bundle)
+so library users import only the core. Library usage is unaffected by
+isolation today (in-process generate(), no server involved). Revisit only
+if Josh decides; both directions stay open from the current structure.

package/docs/planning/release-notes-v0.4.0.md

@@ -0,0 +1,45 @@
+# mlx-bun v0.4.0
+
+This release brings shared batched decode, generated-history cache reuse and
+resumable generation together through separate scheduling, inference, sampling
+and persistence interfaces.
+
+- Shared ordinary and speculative execution supports Qwen MTP, prompt lookup,
+  standalone, assistant, DeepSpec and DSpark/DFlash providers on qualified
+  model/request combinations. Single requests run at B=1 through the scheduler.
+- Full and rotating affine/TurboQuant cache layouts compose with delayed and
+  per-layer precision, speculative rollback and generated target/draft state.
+- Completed decode populates the shared RAM/SSD cache so later turns can reuse
+  generated history. SSD persistence is queued and runs during idle ownership.
+- Ordinary interrupted-generation resume now works through shared execution,
+  including compatible adapters and quantized caches. Restored sampler history,
+  pending tokens and usage accounting preserve the continuation contract.
+- Ordinary paged storage and adapter-qualified prompt lookup use the same state
+  and execution interfaces. GLM native MTP gains shared cache transactions and
+  row state; final real-artifact testing remains deferred.
+- Measured Trellis v13 is selected by default, and eligible Qwen KV4 speculation
+  is enabled. Workload-dependent lookup and other experimental settings remain
+  opt-in. Memory fit estimates are advisory unless users choose explicit limits.
+- Benchmark reporting retains request/source identities, paired comparisons,
+  failures and configuration differences. CI and the release build use Bun 1.4.2.
+
+Serving keeps its existing `--batch 8` default. A lone eligible request starts
+immediately at B=1; the cap does not wait for eight requests. `--batch 1` retains
+the serial executor, and unsupported compositions can still use it. Removing
+serial and completing every feature combination are follow-up work.
+
+Checkpoint capture is eventually persistent: queued snapshots can remain in RAM
+during uninterrupted traffic. A successful explicit flush is the durability
+boundary. Media, fill/grammar-jump and direct paged-attention work remain tracked.
+No DSpark/DFlash checkpoint has been trained for this campaign; execution fixtures
+do not establish trained-drafter performance.
+
+Measured performance and its workload limits are in the
+[benchmark record](https://github.com/joshuarossi/mlx-bun/blob/v0.4.0/docs/reference/benchmarks.md).
+This release does not claim a matched full-task Kanban speedup or completion of
+the entire optimization program. Supported settings are documented in the
+[server configuration reference](https://github.com/joshuarossi/mlx-bun/blob/v0.4.0/docs/reference/server-config.md).
+
+Requires Apple Silicon and macOS 14 or later. npm installs require Bun 1.4.0 or
+later; Bun 1.4.2 is the tested version. The existing native runtime pack 0.4.0
+provides MLX 0.32.2 and its matching C API.

package/docs/planning/video-series.md

@@ -292,7 +292,7 @@ mlx-bun          18.6 tok/s      17.0 tok/s        116 ms      19.5 tok/s
 mlx-lm           16.5 tok/s      14.8 tok/s        284 ms      18.6 tok/s
 ```
 
-Add a small source line: `benchmarks/RESULTS.md`.
+Add a small source line: `docs/reference/benchmarks.md`.
 
 **Dialogue**
 
@@ -399,7 +399,7 @@ font in the diagrams and screen recordings.
 - Confirm first-run behavior still matches the narration.
 - Confirm the API request and response fields against the current reference
   docs.
-- Confirm the benchmark card still matches `benchmarks/RESULTS.md`.
+- Confirm the benchmark card still matches `docs/reference/benchmarks.md`.
 - Record room tone and five seconds of clean terminal footage for edits.
 - Capture every important terminal action twice.