mlx-bun 0.3.0 → 0.5.0

package/docs/archive/planning/release-notes-v0.4.0.md

@@ -0,0 +1,45 @@
+# mlx-bun v0.4.0
+
+This release brings shared batched decode, generated-history cache reuse and
+resumable generation together through separate scheduling, inference, sampling
+and persistence interfaces.
+
+- Shared ordinary and speculative execution supports Qwen MTP, prompt lookup,
+  standalone, assistant, DeepSpec and DSpark/DFlash providers on qualified
+  model/request combinations. Single requests run at B=1 through the scheduler.
+- Full and rotating affine/TurboQuant cache layouts compose with delayed and
+  per-layer precision, speculative rollback and generated target/draft state.
+- Completed decode populates the shared RAM/SSD cache so later turns can reuse
+  generated history. SSD persistence is queued and runs during idle ownership.
+- Ordinary interrupted-generation resume now works through shared execution,
+  including compatible adapters and quantized caches. Restored sampler history,
+  pending tokens and usage accounting preserve the continuation contract.
+- Ordinary paged storage and adapter-qualified prompt lookup use the same state
+  and execution interfaces. GLM native MTP gains shared cache transactions and
+  row state; final real-artifact testing remains deferred.
+- Measured Trellis v13 is selected by default, and eligible Qwen KV4 speculation
+  is enabled. Workload-dependent lookup and other experimental settings remain
+  opt-in. Memory fit estimates are advisory unless users choose explicit limits.
+- Benchmark reporting retains request/source identities, paired comparisons,
+  failures and configuration differences. CI and the release build use Bun 1.4.2.
+
+Serving keeps its existing `--batch 8` default. A lone eligible request starts
+immediately at B=1; the cap does not wait for eight requests. `--batch 1` retains
+the serial executor, and unsupported compositions can still use it. Removing
+serial and completing every feature combination are follow-up work.
+
+Checkpoint capture is eventually persistent: queued snapshots can remain in RAM
+during uninterrupted traffic. A successful explicit flush is the durability
+boundary. Media, fill/grammar-jump and direct paged-attention work remain tracked.
+No DSpark/DFlash checkpoint has been trained for this campaign; execution fixtures
+do not establish trained-drafter performance.
+
+Measured performance and its workload limits are in the
+[benchmark record](https://github.com/joshuarossi/mlx-bun/blob/v0.4.0/docs/reference/benchmarks.md).
+This release does not claim a matched full-task Kanban speedup or completion of
+the entire optimization program. Supported settings are documented in the
+[server configuration reference](https://github.com/joshuarossi/mlx-bun/blob/v0.4.0/docs/reference/server-config.md).
+
+Requires Apple Silicon and macOS 14 or later. npm installs require Bun 1.4.0 or
+later; Bun 1.4.2 is the tested version. The existing native runtime pack 0.4.0
+provides MLX 0.32.2 and its matching C API.

package/docs/archive/planning/release-notes-v0.5.0.md

@@ -0,0 +1,37 @@
+# mlx-bun v0.5.0
+
+Whisper speech-to-text: transcription, dictation and voice input, bit-exact
+against mlx-whisper and faster than both mlx-whisper and whisper.cpp on the
+same clips.
+
+- `mlx-community/whisper-large-v3-turbo` (any fp16 `mlx-community/whisper-*`)
+  loads as a new encoder-decoder family. The oracle-parity graph matches
+  mlx-whisper 0.4.3 bit-exactly on the mel spectrogram, the encoder output and
+  every decoder logit; the default fast path (fused attention, fused cross-K/V,
+  a compiled decoder step with device-side logit filters, pipelined greedy,
+  in-graph beam top-k) is held to at most two differing tokens per clip. M1 Max,
+  11 s clip: 340 ms greedy (mlx-whisper 371), 424 ms beam-5 (whisper.cpp 499).
+- Decoding: greedy, sampling with the temperature fallback ladder, beam search
+  (mlx-whisper has none; matches whisper.cpp's beam-5 transcripts), prompts and
+  vocabulary hints fitted to the 223-token budget, language detection,
+  translation, >30 s seek loop, word-level timestamps.
+- `POST /v1/audio/transcriptions` and `/v1/audio/translations` (OpenAI shape,
+  multipart or JSON, json/verbose_json/text/srt/vtt, SSE streaming) on every
+  server, with Whisper loaded per take beside the chat model and released right
+  after it; `mlx-bun serve <whisper model>` runs a transcription-only server.
+- Streaming sessions (`/v1/audio/sessions`): audio uploaded while recording,
+  every finished 30 s window transcribed during capture — a 62 s take returns
+  327 ms after its last chunk.
+- Silero VAD v6.2 gate (ported from the reference graph, verified against the
+  pip package): silence returns an empty transcript without running Whisper;
+  runs on the stream inside sessions.
+- In-process audio decoding through AudioToolbox (3.7 ms for an 11 s mp3 vs
+  72 ms via afconvert); any CoreAudio-readable container.
+- `mlx-bun transcribe <file>` and `mlx-bun dictate` (push-to-talk from the
+  microphone, print / clipboard / typed keystrokes; new `mlx-bun-mic-capture`
+  sidecar), and a hold-to-talk mic in the web composer for any chat model.
+- Measured and shelved: a Core ML encoder on the Neural Engine (3.3× slower
+  than the GPU on the M1 Max); whisper.cpp-style encoder context truncation
+  ships as an off-by-default Lab option.
+
+Native pack `native-v0.5.0` adds the `mlx-bun-mic-capture` helper.