mlx-bun 0.5.0 → 1.0.0

package/docs/archive/planning/release-notes-v0.0.11.md

@@ -1,239 +0,0 @@
-# v0.0.11 release notes
-
-Everything since v0.0.10. The big themes: mlx-bun grows a second modality
-(audio), a constrained-decoding stack (structured output / grammar),
-speculative decoding, a novel KV-cache quantization scheme (TurboQuant),
-completed dynamic batching, crash-isolated serving, and a ground-up
-rewrite of the web chat UI. The decode-default story also resets: after
-benchmarking every custom performance kernel in a paired A/B against the
-bit-exact baseline and finding none of them won, they were deleted —
-naked `mlx-bun serve` now runs the faithful, mlx-lm-parity kernel set by
-default.
-
-## Audio input (Gemma e4b)
-
-- OpenAI-style `input_audio` parts are served end-to-end: WAV/AAC
-  transcode, a USM mel-spectrogram extractor, and a Conformer tower
-  ported bit-exact (rel-RMSE 2.4e-8) against the oracle.
-- Build-out ran phase-by-phase (A0–A5): a `bun:ffi` conv2d binding for
-  the front-end, the multimodal prompt builder wired to the LM, and
-  capability discovery for audio surfaced consistently across
-  `/v1/models`, `/library`, and the pi websocket handshake — mirroring
-  how vision capability is advertised.
-
-## Structured output / grammar-constrained generation
-
-- `response_format` (`json_object`, `json_schema`, `text`) and
-  `guided_grammar`/`guided_regex`/`guided_choice`/`structured_outputs`
-  are wired into both `/v1/chat/completions` and `/v1/completions`, built
-  on `xgrammar`: a `GrammarController` masks logits inside the decode
-  loop, with a per-`TokenizerInfo` compiler cache and a `MLX_BUN_GRAMMAR=0`
-  kill switch.
-- The batched lane got its own per-row matcher path so grammar-constrained
-  requests don't force a request onto the serial lane, and grammar
-  composes with `--draft-model` (the verify walk respects the mask).
-- Two defects found and fixed during the build: the oMLX-parity
-  JSON-system-prompt degrade path (used when a client can't do function
-  calling) had gone unreachable behind a dropped hint, and the WASM
-  grammar controllers were leaking on every early-400 reject path
-  (SSRF-blocked media, vision/audio validation failures, bad
-  `logit_bias`, etc.) — both fixed, with disposal now covering all paths.
-
-## Speculative decoding
-
-- `--draft-model` lands in serve: a two-model, serial-lane speculative
-  decoding path verified token-for-token against mlx-lm's own
-  `--draft-model` oracle (`DraftSource` seam, rewind rule, batched-verify
-  lm-head).
-- An ngram / jump-forward draft kind ships alongside the model-based
-  drafter for cases with no small model available.
-- A DeepSpec (DSpark) drafter-quant research track is underway
-  (Phase 1 landed) targeting the larger models; per
-  `benchmarks/RESULTS.md`, the first live pair (12B target +
-  bf16 DeepSpec drafter, 26–33% acceptance) measured net **slower**
-  than serial on a loaded box — spec decoding stays **off by default**
-  pending a quantized-drafter pair that clears the 1.3× bar.
-
-## TurboQuant KV-cache quantization
-
-- `--kv-quant turbo[:kXvY]` adds a rotation-based per-group KV
-  quantization scheme (sign-FWHT + Lloyd-Max for values), proven
-  bit-exact against the vendored vllm-metal reference
-  (`goldens/turboquant.json`).
-- Per `benchmarks/RESULTS.md`, the k8v3 default holds a lower KL-vs-bf16
-  than affine uniform kv4 while compressing KV **2.56×** (measured on
-  MiniCPM5-1B, paired quality/KL harness). It's a memory/context lever,
-  not a speed lever — v1 dequant-on-fetch is slower per decode step at
-  long context, so it stays **opt-in, not default**, per the project's
-  "levers must earn defaults" rule.
-
-## Batching: Phase D completed
-
-- Aggregate `--kv-budget <GB>` admission with FIFO queuing, vectorized
-  homogeneous sampling (penalties/`logit_bias`/`min_p`/XTC all run
-  per-row in the batch), and extend-join so a request can join an
-  in-flight batch mid-flight instead of waiting for a full drain.
-- Per-row RoPE lands for Tier-0 universal architectures, lifting the
-  batching gate for those models (e.g. Llama).
-- Batched mixed-precision quantized KV — the first stack to compose
-  per-row KV quantization with batching.
-- `--batch` default changes from 1 to 8 (see Breaking changes below).
-
-## Runtime isolation
-
-- `--isolate` runs the engine as a crash-isolated child process behind a
-  reactor parent, restarting with backoff on a crash — the product rule
-  is **"the AI may crash, the UI never may"** (`docs/design/runtime-isolation.md`).
-- `--model-pool` extends this to a child-per-model pool: LRU residency,
-  spawn-overlap switching (no dead time swapping models), and
-  `POST /admin/drain` for graceful eviction.
-
-## Optional paged KV cache
-
-- `--paged-kv` adds a vLLM-style block-table paged KV cache
-  (`PagedKVCache` + `BlockPool`) feeding the unchanged stock SDPA via
-  block gather. Default off; v1 scope is serial `--batch 1`, the
-  Gemma4 family, bf16 — it refuses to silently combine with
-  `--batch N>1`, `--kv-quant`, or `--draft-model`.
-
-## Prompt cache overhaul
-
-- Prefix-sharing v1: `take()` now returns non-consuming, ref-counted
-  clones instead of consuming/trimming the donor entry, so multiple
-  agents or sessions can reuse one prefill without cannibalizing each
-  other's cache.
-- The SSD cold tier is promoted from an add-on to a first-class property
-  of the cache store, with `--ssd-demote-idle <seconds>` (default 300,
-  requires `--ssd-cache`) freeing GPU memory by demoting idle entries;
-  `/stats` gains `ssd_cache.demotions`.
-- A stable-boundary snapshot fix restores prompt-cache hits for
-  multi-turn agent traffic beyond the sliding window — this was
-  previously a silent cache miss on turn 2+ of long conversations.
-- The SSD write-behind flush is now idle-gated, so background disk
-  writes stop contending with decode at long context; the earlier A7
-  RSS regression on the ssd-cache path closed with zero-copy write views
-  and a streamed copy-restore.
-
-## Web chat UI: ground-up redesign (Phases 0–3)
-
-- Streaming render rewritten to be block-memoized (no more O(n²)
-  re-parse of the growing message), with a typed module split.
-- New surfaces: a memory panel, adapter routing, a system-prompt editor,
-  an approval gate, client-side RAG over chat history (BM25, no server
-  round-trip), a Model Hub, Canvas, per-message sampling controls,
-  self-healing tool calls, a command palette, full-text search, and
-  chat export.
-- An app-aware assistant (v1) that can see and navigate its own UI —
-  spotlight/selector resolution now treats a model-invented or malformed
-  selector as a miss instead of throwing inside the websocket handler.
-- PWA support, plus a round of visual-QA fixes: the phone nav band,
-  light-theme code legibility, a composer placeholder regression, memory
-  panel markdown chrome, the theme-toggle Auto button, a clipped status
-  pill, and tab flex-shrink pinned to zero on phones.
-
-## Security: media-fetch SSRF hardening
-
-- `image_url`/`audio_url` fetches now refuse private/loopback/link-local/
-  CGNAT destinations and non-`http(s)` schemes by default, re-validate
-  through up to 5 redirect hops, enforce a 10s wall-clock timeout, and
-  cap bodies at 64MB.
-- LAN use needs the new escape hatch: `--allow-private-media` /
-  `MLX_BUN_ALLOW_PRIVATE_MEDIA=1`.
-
-## Fixed
-
-- 12B `/v1/completions` probe parity: adopted the oracle's step-0
-  prefill convention (drain to len−1, step 0 from an `L=1` forward of the
-  last prompt token). Per `benchmarks/RESULTS.md`, live HTTP probes are
-  now byte-identical to `mlx_lm.server`/optiq serve for MiniCPM5-1B,
-  gemma-4-e4b, and gemma-4-12B, on completion and chat probes, in both
-  the `--batch 8` default lane and `--batch 1`.
-- BPE detokenization now mirrors mlx-lm's bare-space hold-back and
-  never-finalize semantics exactly.
-- Cache invariant enforced on every tier; an FFI destructor deadlock
-  fixed; write-behind made non-blocking; per-row batch cache extraction
-  fixed.
-- `generate()` now yields a macrotask between prefill chunks, keeping the
-  isolation path (and the server generally) responsive during long
-  prefills.
-- Benchmark harness: the `--batch 1` serial control arm now runs by
-  default (it had been skippable, which undermined validating the new
-  batch-8 default); `--engine` is additive instead of mutually exclusive
-  with the serve pass; the batch-path test now asserts cumulative
-  `/stats` `submitted_rows` instead of a stale `cachedTokens=0` premise.
-- `any_whitespace` grammar knob landed; batch-grammar gates re-anchored
-  to compact separators.
-
-## Infra / benchmark harness redesign
-
-- `benchmark.sh` + `scripts/bench-modes.ts` now spawn real CLI paths
-  instead of bench-local server wrappers, folding bit-parity and
-  peak-memory checks into the same serve cells (one pass instead of
-  several). New columns: cold-start and restart-survival; new workload:
-  `xl` (~4k prefix); a feature-matrix benchmark mode.
-- Repo cleanup: `lab/` consolidation, a binary-tracking hygiene gate, and
-  a git history rewrite that took the repo from 182MB to 20MB, removing
-  the last multi-MB binaries from the index.
-- New design docs: `unified-engine-frontier-plan`, `dspark-serving-program`
-  (the DeepSpec/DSpark spec-decode research program), `structured-output`,
-  `web-chat-redesign` (+ app-aware-assistant, superset-doctrine, and
-  visual-QA-bar notes), `paged-kv-cache`, `batching-perf-path` /
-  `batching-v2-plan`, `runtime-isolation`.
-- Doc drift swept: env-lever table entries (`MLX_BUN_DSPARK_MINCONF`,
-  `MLX_BUN_BATCH_NO_PIPELINE`, `MLX_BUN_EXPERT_OFFLOAD`), ngram
-  draft-kind coverage in README/server-api, doc-map sync, an openwiki
-  investigation write-up. STATUS.md was rewritten to current-state-only
-  several times through the release to keep handoff notes accurate.
-- A new ground rule was recorded in CLAUDE.md: changes to the served
-  surface must update the user-facing reference docs in the same commit.
-- A direct optiq mixed-KV bit-parity golden replaced an indirect
-  ours-vs-ours comparison; a bit-exact L1 parity harness was added for
-  the Qwen3-30B-A3B MoE port, with uniform quantized KV confirmed
-  bit-exact against mlx-lm's own `--kv-bits` and a stock-mlx-lm baseline
-  added to the bench.
-- Dependency/CI bumps: pi, `actions/checkout`, and the site build moved
-  off a deprecated Node runtime.
-
-## Breaking / behavior changes
-
-- **`--batch` default changes from 1 to 8.** Serving now batches by
-  default whenever concurrent requests arrive; pass `--batch 1` to force
-  strict serial (needed if you require arrival-order-independent,
-  bit-identical numerics).
-- **The default decode path changed.** Naked, no-flag serving now runs
-  the faithful, bit-exact-vs-mlx-lm kernel set — what `--l1` used to
-  mean. `--l1`/`--l2`/`--l3` are now pure aliases over per-feature flags
-  rather than distinct performance tiers. Every custom-kernel path that
-  didn't beat this baseline in a paired A/B was deleted in the same
-  pass: the fused-decode kernel, fused-gelu/fused-swiglu/fused-mlp/
-  steel-linear Metal kernels, the flash-decode "perf" kernel and its
-  frozen-oracle test scaffolding, `--l3`, and `FaithfulMiniCPM5` /
-  `MLX_BUN_CPM5_FAITHFUL`. None of these flags or code paths exist
-  anymore. `MLX_BUN_FAITHFUL` is also gone (superseded by the faithful
-  kernels being the default) — the server's unset `--kv-quant` now
-  defaults to bf16 instead of `kv_config`, so getting the old default
-  composition requires an explicit `--kv-quant`.
-- **New CLI/API surface:** `--kv-quant turbo[:kXvY]` (TurboQuant),
-  `--paged-kv` (off by default; refuses to combine with `--batch N>1`,
-  `--kv-quant`, or `--draft-model`), `--draft-model` (speculative
-  decoding), `--isolate` and `--model-pool` (runtime isolation),
-  `--kv-budget <GB>` (aggregate KV admission), `--ssd-demote-idle
-  <seconds>` (default 300, requires `--ssd-cache`),
-  `--allow-private-media` / `MLX_BUN_ALLOW_PRIVATE_MEDIA=1`, and the
-  `response_format` / `guided_grammar` / `guided_regex` / `guided_choice`
-  / `structured_outputs` request fields (kill switch:
-  `MLX_BUN_GRAMMAR=0`).
-- **Media fetches are restricted by default.** `image_url`/`audio_url`
-  now refuse private/loopback/link-local/CGNAT destinations and
-  non-`http(s)` schemes — previously they fetched anything. LAN media
-  needs the new escape hatch above.
-- **`PromptCache.take()` semantics changed.** Cache hits now return
-  non-consuming, ref-counted clones instead of consuming/trimming the
-  donor entry — code holding onto the old consuming behavior should
-  re-check callers.
-
-Full details and measured numbers: STATUS.md, `benchmarks/RESULTS.md`,
-and the design docs referenced above. Credit where due: TurboQuant's
-codec is verified bit-exact against the vendored vllm-metal reference,
-and the speculative-decoding path is verified token-for-token against
-mlx-lm's own `--draft-model` implementation.

package/docs/archive/planning/release-notes-v0.0.12.md

@@ -1,80 +0,0 @@
-# v0.0.12 release notes
-
-mlx-bun 0.0.12 completes the native Colibri/GLM-5.2 program: a Python-free,
-direct-container runtime that can serve the approximately 357 GiB artifact on
-a 32 GiB Apple Silicon Mac by keeping a bounded working set in memory and
-streaming routed experts from fast storage.
-
-## GLM-5.2 direct-container serving
-
-- Loads `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` directly. No converted
-  copy and no Python process are involved in the serving path.
-- Preserves checkpoint precision and true top-8 routing. The port is gated
-  against pinned Colibri at tiny, production-component, real-model, MTP, cache,
-  and first-sparse DSA boundaries.
-- Native int8 MTP is enabled by default and uses the serial speculative lane.
-  It produced a measured 1.306x warm end-to-end win over MTP-off on the 32 GiB
-  reference machine while preserving all target tokens.
-- Compressed MLA/DSA/MTP cache persistence, restart restore, compressed-byte
-  admission, ordinary continuous batching, mixed-length join/leave/cancel, and
-  cross-row expert union are included. Per-row batched MTP remains a later
-  feature; MTP requests truthfully report `serial+spec`.
-- Chat completions, text completions, Anthropic Messages, OpenAI Responses,
-  SSE, tools, structured output, stops, sampling, usage, and serial logprobs
-  are supported. Embeddings, vision/audio, LoRA, and training are explicitly
-  reported unsupported for this model instead of being emulated.
-
-## Memory and acquisition contract
-
-- `mlx-bun get` now computes the exact remaining payload before the first
-  payload request, credits verified shared blobs and resumable prefixes, and
-  refuses an acquisition that cannot leave a fixed 1 GiB safety reserve.
-- The GLM header-only preflight runs before resident weights or expert slabs
-  open. With the stock DSA overlay installed, 4,096 context / 128 generation
-  tokens and MTP on produce a 19.89 GiB plan under a 25 GiB process ceiling,
-  leaving 5.11 GiB of process/macOS headroom on a 32 GiB machine.
-- `fit`, `/fit`, `/stats`, and the status page now distinguish the complete
-  artifact on disk from resident weights, main/MTP slabs, compressed KV, and
-  reserves. They also expose main/MTP residency, hit rate, SSD bytes, policy
-  telemetry, and repin events.
-
-## Performance: measured, not predicted
-
-- Warm quality-preserving throughput on the M1 Max 32 GiB reference machine is
-  0.1577 decode tok/s / 0.1487 end-to-end tok/s with MTP on. The same-machine
-  direct-Colibri control is approximately 0.27 tok/s.
-- The 2 tok/s figure remains an aspiration, not a release gate or `fit`
-  prediction. The measured warm end-to-end result is 13.45x below it.
-- Replicated telemetry measured 1,974,949,363,712 logical expert bytes per
-  128-token warm turn (15.429 decimal GB/token) and a 1.66% cache hit rate.
-  Startup auto-pin, live LFRU, PILOT hints, two-step prediction, and coupling
-  did not improve paired end-to-end performance and remain off by default.
-- The checkpoint's 21-full/57-shared DSA schedule is implemented and exact.
-  Paired 2K/8K tests did not establish an MTP-on product-speed win, so this
-  release makes no DSA speed claim. The public Colibri artifact does not ship
-  the stock DSA indexer overlay; without it the runtime follows the artifact's
-  available attention path.
-
-## Distribution and operator experience
-
-- First-run native pack v0.2.0 adds the bounded native expert-I/O helper. The
-  npm/bunx fresh-cache path downloads, verifies, and extracts all five runtime
-  files; the signed/Homebrew bundle includes the same helper beside the binary.
-- The pinned 32 GiB quickstart, volume selection, interrupted-download recovery,
-  model lineage, licenses, resource plan, measured performance, and limitations
-  are documented in README and the reference guides.
-- The interactive Atlas report reproduces Colibri's expert topic-affinity
-  result: 29/30 leave-one-prompt-out classifications (96.7%) and 1,065 strong
-  specialists among 13,236 replicated experts.
-
-## Stability fixes since v0.0.11
-
-- Closed pre-Colibri correctness and lifecycle findings, including expert-slab
-  post-close and lazy-graph reuse hazards, with forced native/MLX churn gates.
-- Fixed selected-logprob readback stalling and revalidated the audio splice
-  boundary against the oracle.
-- Corrected current Hugging Face LFS checksum-field handling and retained the
-  older spelling for compatibility.
-
-Full provenance and exact numbers are in `benchmarks/RESULTS.md`, `STATUS.md`,
-and `docs/design/colibri-glm52-port.md`.

package/docs/archive/planning/release-notes-v0.0.13.md

@@ -1,77 +0,0 @@
-# v0.0.13 release notes
-
-Everything since v0.0.12. One theme: **Qwen 3.8**. The current-generation
-Qwen (released August 2026) is brought up on the published OptiQ
-artifact at the repo's standard bar — bit-exact logit parity with
-mlx-lm — with its thinking controls served faithfully. Vision, video,
-and native Qwen MTP remain separately gated follow-ups.
-
-## Qwen3.8-27B (text, bit-exact)
-
-- `mlx-community/Qwen3.8-27B-OptiQ-4bit` serves on the existing
-  qwen3_5 hybrid graph (gated DeltaNet + gated attention) — the
-  architecture audit found byte-identical geometry to the verified
-  port, and parity confirmed it: **every logit at every prompt
-  position plus 32 greedy decode steps, bit-exact vs stock mlx-lm**
-  (the parity harness grew a full prefill-grid golden and
-  manifest-driven step counts in the process).
-- The config's new `output_gate_type: "swish"` field is inert in every
-  implementation (transformers ground truth included) — a load-time
-  guard now fails loudly if a future checkpoint carries a value we
-  haven't verified.
-- Fit reality: the artifact is 20.35 GB of shards. 32 GB machines
-  serve it comfortably; 24 GB machines need `--memory-budget` and
-  should prefer the uniform 4-bit conversion until the planned
-  ~4 bpw TurboQuant×weights artifact lands (PLAN 14z).
-
-## Thinking controls (Qwen 3.8's serving surface)
-
-- Thinking is on by default and streams as OpenAI `reasoning` deltas;
-  `chat_template_kwargs.enable_thinking: false` selects instruct mode.
-- `reasoning_effort` is now a real depth control on templates that read
-  it: OpenAI levels map onto Qwen's `low`/`medium`/`xhigh` (with
-  `xhigh` accepted as a first-class request value), and the variable is
-  only passed to templates that declare it — the Qwen 3.8 template
-  raises on unknown level names, so the server never forwards unmapped
-  strings.
-- `preserve_thinking` (new in Qwen 3.8, default on) is forwarded:
-  history keeps its think blocks, which also improves prompt-cache
-  reuse in agent loops.
-- Tool calling: Qwen 3.8's XML call format
-  (`<tool_call><function=name><parameter=…>`) parses through the
-  existing Qwen-style parser into OpenAI `tool_calls`, multiline
-  parameters included.
-
-## Qwen native MTP is deliberately not shipped yet
-
-- Qwen 3.8 ships a multi-step MTP head trained alongside the model;
-  mlx-community publishes it split out
-  (`mlx-community/Qwen3.8-27B-MTP-bf16`). Review found that the target's
-  48 recurrent DeltaNet caches cannot roll back rejected verify rows,
-  while the shared speculative loop requires exactly that operation.
-  The initial implementation therefore never reached a draft round.
-- MTP artifacts are detected and refused at startup with the exact reason,
-  before the 20 GB target is opened. Shipping a switch that silently disables
-  itself—or corrupts recurrent state when forced—would be worse than a clear
-  deferral. The follow-up needs recurrent-state snapshot/restore or a verify
-  schedule that never advances state beyond the accepted prefix.
-
-## GLM-5.2 client compatibility
-
-- Fixed-context GLM serving now treats request `max_tokens` as an upper bound:
-  when a client sends a family-wide value larger than the planned remaining
-  context, the server caps it instead of rejecting an otherwise valid first
-  prompt. The reported 2,788-prompt + 8,192-cap failure now admits a 1,308-token
-  completion ceiling within the 4,096-token plan. Prompts that consume the
-  whole context remain rejected before generation.
-
-## Also in this release
-
-- The parity harness's step-0 check now covers the full prefill logit
-  grid for all Qwen targets — masking/position bugs can no longer hide
-  behind a last-position comparison.
-- A GPU command-buffer failure mode on memory-saturated machines was
-  root-caused (Metal completion-thread exception, uncatchable by
-  design) and documented in PLAN 14g, along with the finding that some
-  mlx-c paths surface C++ errors as process panics instead of
-  catchable errors (hardening item).

package/docs/archive/planning/release-notes-v0.0.9.md

@@ -1,122 +0,0 @@
-# v0.0.9 release notes (DRAFT — Josh edits + publishes)
-
-Everything since v0.0.8 (2026-06-24). The big themes: the mlx-lm drop-in
-surface is now materially complete, ORPO is paper-faithful, mlx-bun runs its
-first generically-supported architectures at verified bit-exact parity, and an
-adversarial review swept the whole engine — fixing 12 confirmed bugs and
-overturning a load-bearing performance assumption.
-
-## Drop-in compatibility with mlx_lm.server (the headline)
-
-- **Endpoints:** `POST /v1/completions` (raw text completion, stream +
-  non-stream), `GET /health` (byte-exact body), `GET /v1/models` now lists all
-  registry models (+ `/v1/models/<id>`).
-- **Request fields:** `min_p`, `xtc_probability`/`xtc_threshold`, `logit_bias`,
-  `presence_penalty`/`frequency_penalty` (+ `*_context_size` windows),
-  `logprobs`/`top_logprobs` with mlx-lm's EXACT semantics (same distribution,
-  same response shape, same [0,11] validation). All L1-faithful ports of
-  `sample_utils.py`.
-- **Flags/defaults:** port 8080 + loopback host (matching mlx_lm.server;
-  `--host 0.0.0.0` = LAN opt-in), `--temp` alias, `--max-tokens`,
-  serve `--adapter <dir>` mounts at startup.
-- **New CLI verbs:** `fuse` (fold LoRA into base — mlx-lm math, untouched
-  modules bit-identical), `convert` (wraps the mixed-precision quantizer,
-  `--target-bpw` exposed), `perplexity` (mlx_lm.perplexity methodology),
-  `upload` (native HF push).
-- Deliberately not ported: `role_mapping` (unreachable — every supported model
-  ships a chat template). Remaining: `cache_prompt`, `evaluate` (planned as an
-  lm-eval shim), `--draft-model`, awq/dwq/gptq (plan:
-  docs/design/mlx-lm-tool-parity-plan.md).
-
-## ORPO: paper-faithful by default
-
-- **`sft_scope: full | response`** (default `full`): the chosen-NLL is now the
-  token-mean CE over the full prompt+response — matching the ORPO paper, TRL,
-  and xfactlab/orpo — implemented across every path (naive/chunked/fused/
-  flash-CCE/prefix-shared/segmented). `response` reproduces old runs
-  BIT-EXACTLY (regression-pinned). The odds-ratio terms stay response-only in
-  both modes (matches TRL). CLI: `mlx-bun train --sft-scope`.
-- The rest of the ORPO stack was adversarially verified correct against
-  primary sources (odds-ratio math, gradients vs finite differences,
-  prefix-sharing gradient flow, tokenization/label-shift).
-
-## Generic model support (Tier-0)
-
-- **New:** `src/model/universal/` — a config-driven UniversalDense module with
-  explicit descriptors for llama, qwen2, qwen3, gemma, gemma2, phi3, olmo2,
-  glm4, granite, starcoder2, smollm3 (+ mlx-lm's remapping: mistral etc.).
-  bf16 (unquantized) checkpoints now load. Rope factory (llama3/yarn/longrope)
-  verified bit-exact vs the oracle.
-- **First three archs verified at L1 parity** (bit-exact per-step logits):
-  Llama-3.2-1B-Instruct-4bit, Qwen2.5-0.5B-Instruct-4bit, gemma-2-2b-it-4bit.
-- Dispatch ladder: dedicated → generated → generic → reject-with-helpful-error.
-  Targeted models keep their optimized paths; a model can graduate.
-  Design: docs/design/generic-model-support.md.
-
-## Fixed (the adversarial-review wave)
-
-- **`--l2` tier contract restored:** the L2 preset was enabling an
-  envelope-gated perf kernel (not bit-exact vs optiq). Bare `--l2` is now
-  genuinely optiq-bit-exact; the perf kernel is `--l3`/explicit opt-in.
-  `generate --l2/--l3` no longer silently degrades to L1.
-- Batched same-millisecond seed collision (identical concurrent completions);
-  streaming per-layer KV-quant conversion (the missing half of optiq's
-  tight-RAM fix); segmented compiled-decode mid-step failure KV corruption;
-  `MLX_BUN_TRAIN_ATTN=flash` now refused on Gemma (unrevalidated crash path);
-  eval-loss error swallowing; wrong tokens_per_sec metric; silent steel-kernel
-  fallbacks now warn; flash-attention tests extended (D=256, sliding-window,
-  non-tile T).
-- Memory (the Dreaming): batching default flipped to serial (measured 1.7–1.9×
-  faster for the real workload); `memory status` truth; third-person voice fix
-  merged.
-- CLI audit (14 findings): `fit --ctx` help matched code, `embed` auto-pick
-  fixed, pi flag leakage into first message fixed, `--l1/2/3` + `generate` +
-  `train-watch` documented, `setup` is a real alias, doc lies corrected.
-
-## Performance truth (research, groundwork for the next release)
-
-- The "decode is at the memory floor" claim was **overturned** with measured
-  rooflines: only the 12B is at the bandwidth wall (~92%); CPM5/e4b/26B sit at
-  58–70% with the host-side graph build as the top recoverable term. Ranked
-  fix plan in-repo; wins land next release.
-- The batching engine was audited (numerics solid, engine naive) — hotfixes +
-  containment landing; plan: docs/design/batching-v2-plan.md.
-- Registry duplicates root-caused (snapshot-per-commit accumulation);
-  dedupe + `gc` landing.
-
-## Infra
-
-- CI gate added (typecheck + model-free tests on push/PR) — previously nothing
-  ran on push.
-- Repo hygiene: scratch artifacts untracked, megakernel research relocated,
-  STATUS.md rewritten as a true current-state front door, doc map regenerated.
-
-## Late additions (landed after the first draft)
-
-- **Batching engine v2 (steps 1–3):** Qwen3.5/SSM models no longer fail under
-  `--batch` (capability gate + serial drain); per-row failure containment; no
-  serial-lane starvation; pipelined batched decode + sane cache cadence; the
-  huge admit transient fixed; all GPU work under one lock.
-- **Model management:** `ls` deduplicates to one row per repo (snapshot
-  history via `ls --all-revisions`); new `mlx-bun gc` reclaims superseded
-  snapshots (24.7 GB found on the dev machine) with a safety guard for
-  unique files; `get` accepts substring queries; download lockfile, streamed
-  resume verification, gated-repo auth hints; vision capability detected from
-  config (the unified-vision 12B now shows `vision`). New doc:
-  docs/reference/models.md.
-- **Pi integration:** web-chat memory tools actually callable; `harness pi`
-  registers exactly the served model; tool-calling fixed for Tier-0 generic
-  models (Gemma sentinels no longer applied to non-Gemma tokenizers); vision
-  advertised to pi; `mlx-bun pi` no longer leaks unknown flags into the first
-  message.
-- **Website + README:** six-goals restructure, drop-in guide, memory guide,
-  the lab page, full CLI reference from a tracked source; deploy now triggers
-  on reference-doc edits. NOTE the release sequence: push and publish in the
-  same sitting — the site deploys from the push.
-- **Test infra:** batched-serving goldens are machine-keyed (the M1-Max
-  "failures" were M4-Pro fixtures; all code exonerated — mlx-lm reproduces
-  mlx-bun token-for-token per machine); CI gate (typecheck + model-free tests).
-- **Research (the lab):** decode-roofline re-measurement overturned the "at
-  the floor" assumption (docs/archive/investigations/decode-roofline-lookagain.md);
-  curve-sampler distinctness theorem + witness numbers + preregistered
-  protocol (docs/planning/curve-sampler-research-plan.md).

package/docs/archive/planning/release-notes-v0.2.0.md

@@ -1,49 +0,0 @@
-# mlx-bun v0.2.0
-
-Version 0.2.0 is the first release with the complete Qwen3.8 serving path:
-text, images, video, thinking controls, tool calls, and native MTP speculation.
-It also ships the serving and memory work completed since v0.0.13.
-
-## Highlights
-
-- Qwen3.8 image input now runs through the dedicated vision tower and mRoPE
-  path. Video content parts decode through the bundled AVFoundation helper,
-  sample frames at 2 fps, and use the gated Qwen video pipeline.
-- Qwen native MTP can mount from a separate companion or an artifact's `mtp/`
-  directory. Rejected drafts restore recurrent DeltaNet state correctly.
-- The default continuous scheduler now shares request planning, sampling,
-  usage accounting, cache selection, and protocol events with the strict
-  serial executor. `--batch 1` remains the explicit serial pin.
-- Memory admission treats `max_tokens` as a ceiling. If the prompt fits, the
-  server caps generation to the safe remaining context instead of rejecting a
-  broad client default.
-- `convert --rotate-weights` folds supported rotations before quantization.
-  The offline pipeline also gained streamed shard handling, 3-bit GPTQ, and
-  atomic output publication.
-
-## Reliability and performance fixes
-
-- Large Qwen models now enter the same wired-memory scope as mlx-lm during
-  serial and continuous execution. This removes the token-zero residency
-  stall on machines whose recommended working set grew beyond the old cutoff.
-- DeltaNet prefill and batch-row extraction now copy owned cache state instead
-  of pinning whole chunk buffers.
-- SSD prompt-cache snapshots stay dirty until atomic storage completes.
-  `POST /admin/cache/flush` and normal SIGINT/SIGTERM shutdown provide an
-  explicit durability boundary.
-- Rotating-cache batch joins now dispatch by cache capability and preserve the
-  correct rotating signature.
-- Qwen and GLM MTP sampling no longer passes a rank-3 logits tensor into the
-  rank-2 sampler.
-
-## Distribution
-
-- macOS on Apple Silicon remains the supported platform.
-- Source and npm installs require Bun 1.4.0 or newer.
-- Direct downloads and Homebrew use the signed, notarized self-contained
-  archive.
-- npm and `bunx` use native pack v0.3.0, which includes MLX, the bounded expert
-  I/O helper, and the AVFoundation frame extractor.
-
-Upgrade with `brew upgrade joshuarossi/tap/mlx-bun`, `bun update -g mlx-bun`,
-or rerun the direct installer.

package/docs/archive/planning/release-notes-v0.3.0.md

@@ -1,35 +0,0 @@
-# mlx-bun v0.3.0
-
-This release adds the Qwen inference performance work, fixes long-generation
-memory retention, and makes configured serving benchmarks reproducible.
-
-- Packed Trellis decode and prefill kernels, interleaved weight readers,
-  joint TurboQuant K/V operations and inverse rotation optimizations.
-- Opt-in Qwen uniform KV4 speculation with MTP and bounded paired target/draft
-  RAM prefixes. Exact history, model identity and adapter namespace govern
-  reuse; paired speculative state does not persist across server restarts.
-- Model-owned structural token appends, serving ownership and parser fixes,
-  and a bounded range cache for sustained generation.
-- SSD flushes recognize prefixes covered by snapshots committed later in the
-  same flush, preserving errors for genuinely missing state.
-- Native runtime pack 0.4.0, bundling MLX 0.32.2 and the matching MLX-C API.
-  Persistent cache compatibility includes the runtime and GPU identity.
-- Serving benchmark controls for explicit draft, KV and prompt-cache settings,
-  with saved commands, request hashes, complete-request timing and raw results.
-
-The required packed Qwen target completes the fresh Luke Kanban task with
-thinking enabled and the published profile. Its configured KV4/MTP serving
-suite passes every phase on the M4 Pro. Full measurements, machine labels and
-limits are in the [benchmark record](https://github.com/joshuarossi/mlx-bun/blob/v0.3.0/docs/reference/benchmarks.md).
-No matched successful original/final Kanban timing is available, so this
-release claims no percentage reduction in full-task completion time.
-
-Optimization defaults remain gated. The standard-suite MiniCPM/Qwen timed
-output differences against the reference and the larger Qwen 4/8-bit
-artifact's prefill memory failures on the 24 GB Mac remain under investigation.
-The SSD restart failures from that suite are fixed and all four affected
-default/mixed serving repeats pass on their first attempt.
-
-The [server configuration reference](https://github.com/joshuarossi/mlx-bun/blob/v0.3.0/docs/reference/server-config.md)
-documents experimental switches and their supported combinations. Requires
-Apple Silicon, macOS 14 or later, and Bun 1.4.0 or later for npm installs.

package/docs/archive/planning/release-notes-v0.4.0.md

@@ -1,45 +0,0 @@
-# mlx-bun v0.4.0
-
-This release brings shared batched decode, generated-history cache reuse and
-resumable generation together through separate scheduling, inference, sampling
-and persistence interfaces.
-
-- Shared ordinary and speculative execution supports Qwen MTP, prompt lookup,
-  standalone, assistant, DeepSpec and DSpark/DFlash providers on qualified
-  model/request combinations. Single requests run at B=1 through the scheduler.
-- Full and rotating affine/TurboQuant cache layouts compose with delayed and
-  per-layer precision, speculative rollback and generated target/draft state.
-- Completed decode populates the shared RAM/SSD cache so later turns can reuse
-  generated history. SSD persistence is queued and runs during idle ownership.
-- Ordinary interrupted-generation resume now works through shared execution,
-  including compatible adapters and quantized caches. Restored sampler history,
-  pending tokens and usage accounting preserve the continuation contract.
-- Ordinary paged storage and adapter-qualified prompt lookup use the same state
-  and execution interfaces. GLM native MTP gains shared cache transactions and
-  row state; final real-artifact testing remains deferred.
-- Measured Trellis v13 is selected by default, and eligible Qwen KV4 speculation
-  is enabled. Workload-dependent lookup and other experimental settings remain
-  opt-in. Memory fit estimates are advisory unless users choose explicit limits.
-- Benchmark reporting retains request/source identities, paired comparisons,
-  failures and configuration differences. CI and the release build use Bun 1.4.2.
-
-Serving keeps its existing `--batch 8` default. A lone eligible request starts
-immediately at B=1; the cap does not wait for eight requests. `--batch 1` retains
-the serial executor, and unsupported compositions can still use it. Removing
-serial and completing every feature combination are follow-up work.
-
-Checkpoint capture is eventually persistent: queued snapshots can remain in RAM
-during uninterrupted traffic. A successful explicit flush is the durability
-boundary. Media, fill/grammar-jump and direct paged-attention work remain tracked.
-No DSpark/DFlash checkpoint has been trained for this campaign; execution fixtures
-do not establish trained-drafter performance.
-
-Measured performance and its workload limits are in the
-[benchmark record](https://github.com/joshuarossi/mlx-bun/blob/v0.4.0/docs/reference/benchmarks.md).
-This release does not claim a matched full-task Kanban speedup or completion of
-the entire optimization program. Supported settings are documented in the
-[server configuration reference](https://github.com/joshuarossi/mlx-bun/blob/v0.4.0/docs/reference/server-config.md).
-
-Requires Apple Silicon and macOS 14 or later. npm installs require Bun 1.4.0 or
-later; Bun 1.4.2 is the tested version. The existing native runtime pack 0.4.0
-provides MLX 0.32.2 and its matching C API.