mlx-bun 0.2.0 → 0.3.0

package/docs/planning/PRODUCT_ROADMAP.md

@@ -343,7 +343,7 @@ into its own persistent child process behind the existing
 API + routing, zero MLX calls); one GPU lease serializes all GPU consumers
 (chat, API, quantize, finetune) — reusing the spawn/lease pattern jobs
 already prove. Converges with model hot-swap (model lives off the main
-thread either way). Full design + phasing: `docs/design/runtime-isolation.md`.
+thread either way). Full design + phasing: `docs/reference/server-config.md`.
 
 ## Harness UX
 
@@ -543,3 +543,36 @@ reuse across pipeline stages.
 - **What is the right install path?** All of the above — shipped: npm
   publish (v0.0.4, `bunx mlx-bun`), Homebrew release pipeline (sign,
   notarize, `joshuarossi/homebrew-tap`), and GitHub release binary.
+
+## Vision notes (merged from IDEAS.md, 2026-08-23)
+
+### Local in the deeper sense
+
+Two phrases that capture the thesis — especially **mlx-bun × Lucien**:
+
+1. **"A truly local model in a deeper sense."** Not just local *compute*
+   (runs on your Mac, nothing leaves it) — local to *you*: grounded in your
+   own context and memory, not only your hardware.
+2. **"The best model at being yours."** Not the best model in the absolute /
+   benchmark sense — the best at *being yours*. A small local model,
+   fine-tuned and wired to your personal memory, can lose every general
+   benchmark and still win the only one that matters here.
+
+Lucien (the Dreaming) is the synthesized, persistent memory of *you*;
+mlx-bun is the local, private model runtime. Combined: a model that runs
+entirely on your machine **and** is continuously grounded in your own
+accumulated context — local in both senses, and yours in a way no hosted
+frontier model can be.
+
+### Library/app split (idea only — NOT planned, Josh 2026-07-05)
+
+Two pulls: "one executable, everything you want to do with local AI" (the
+product's charm) vs "some people just want a JS ML library — do they need
+the rest?" The runtime-isolation work (2026-07-05) drew the exact boundary
+either future needs: the engine child = the GPU stuff (src/model, src/mlx,
+caches, scheduler, generate); the parent = the app (serve/UI/registry/
+jobs/proxy). If a split ever happens it is a PACKAGING decision, not a
+refactor — e.g. npm subpath exports (mlx-bun/model vs the full CLI bundle)
+so library users import only the core. Library usage is unaffected by
+isolation today (in-process generate(), no server involved). Revisit only
+if Josh decides; both directions stay open from the current structure.

package/docs/planning/release-notes-v0.3.0.md

@@ -0,0 +1,35 @@
+# mlx-bun v0.3.0
+
+This release adds the Qwen inference performance work, fixes long-generation
+memory retention, and makes configured serving benchmarks reproducible.
+
+- Packed Trellis decode and prefill kernels, interleaved weight readers,
+  joint TurboQuant K/V operations and inverse rotation optimizations.
+- Opt-in Qwen uniform KV4 speculation with MTP and bounded paired target/draft
+  RAM prefixes. Exact history, model identity and adapter namespace govern
+  reuse; paired speculative state does not persist across server restarts.
+- Model-owned structural token appends, serving ownership and parser fixes,
+  and a bounded range cache for sustained generation.
+- SSD flushes recognize prefixes covered by snapshots committed later in the
+  same flush, preserving errors for genuinely missing state.
+- Native runtime pack 0.4.0, bundling MLX 0.32.2 and the matching MLX-C API.
+  Persistent cache compatibility includes the runtime and GPU identity.
+- Serving benchmark controls for explicit draft, KV and prompt-cache settings,
+  with saved commands, request hashes, complete-request timing and raw results.
+
+The required packed Qwen target completes the fresh Luke Kanban task with
+thinking enabled and the published profile. Its configured KV4/MTP serving
+suite passes every phase on the M4 Pro. Full measurements, machine labels and
+limits are in the [benchmark record](https://github.com/joshuarossi/mlx-bun/blob/v0.3.0/docs/reference/benchmarks.md).
+No matched successful original/final Kanban timing is available, so this
+release claims no percentage reduction in full-task completion time.
+
+Optimization defaults remain gated. The standard-suite MiniCPM/Qwen timed
+output differences against the reference and the larger Qwen 4/8-bit
+artifact's prefill memory failures on the 24 GB Mac remain under investigation.
+The SSD restart failures from that suite are fixed and all four affected
+default/mixed serving repeats pass on their first attempt.
+
+The [server configuration reference](https://github.com/joshuarossi/mlx-bun/blob/v0.3.0/docs/reference/server-config.md)
+documents experimental switches and their supported combinations. Requires
+Apple Silicon, macOS 14 or later, and Bun 1.4.0 or later for npm installs.

package/docs/planning/video-series.md

@@ -292,7 +292,7 @@ mlx-bun          18.6 tok/s      17.0 tok/s        116 ms      19.5 tok/s
 mlx-lm           16.5 tok/s      14.8 tok/s        284 ms      18.6 tok/s
 ```
 
-Add a small source line: `benchmarks/RESULTS.md`.
+Add a small source line: `docs/reference/benchmarks.md`.
 
 **Dialogue**
 
@@ -399,7 +399,7 @@ font in the diagrams and screen recordings.
 - Confirm first-run behavior still matches the narration.
 - Confirm the API request and response fields against the current reference
   docs.
-- Confirm the benchmark card still matches `benchmarks/RESULTS.md`.
+- Confirm the benchmark card still matches `docs/reference/benchmarks.md`.
 - Record room tone and five seconds of clean terminal footage for edits.
 - Capture every important terminal action twice.