mlx-bun 0.3.0 → 0.4.0

package/docs/reference/cli.md

@@ -64,6 +64,24 @@ mlx-bun serve GLM-5.2 --context-length 4096
 semantics, and which combinations compose live in
 **[server-config.md](server-config.md)** — this page only names them.
 
+Library requests support delayed uniform or per-layer affine conversion with
+qualified shared drafting providers, including rotating caches. CLI affine
+quantization starts at zero.
+
+The unreleased shared executor supports qualified Qwen MTP and start-zero
+TurboQuant at one or several active rows; `--batch 1` still selects the
+legacy serial control. No new CLI flag is required for the shared interfaces.
+Full-attention and rotating targets, including Llama, MiniCPM and Gemma, use shared bf16, uniform KV4/KV8, per-layer affine KV or TurboQuant
+prompt lookup and standalone drafting, including generated RAM/SSD state.
+Gemma assistant drafting also shares these target layouts and caches its last true target
+hidden alongside target KV through the same RAM/SSD interfaces.
+DeepSpec and DSpark use the same executor and persist their projected context through those
+interfaces. DeepSpec accepts bf16 or affine-quantized weights; DSpark retains
+its Markov/RNN head and checkpoint confidence policy.
+
+Shared Qwen MTP retains completed decode state through the existing
+`--prompt-cache` and `--ssd-cache` settings.
+
 | Group | Flags |
 |---|---|
 | Network | `--host`, `--port`, `--no-open`, `--allow-private-media` |
@@ -71,13 +89,17 @@ semantics, and which combinations compose live in
 | Process model | `--isolate`, `--model-pool`, `--unix` (internal — the engine half of `--isolate`) |
 | Scheduling | `--batch` (`--decode-concurrency` accepted as the mlx_lm.server alias) |
 | KV cache | `--kv-quant`, `--paged-kv`, `--paged-kv-block-size` |
-| Adapters | `--adapter` (`--adapter-path` alias) |
-| Speculative decoding | `--draft-model`, `--draft-kind`, `--num-draft-tokens`, `--ngram-max`, `--ngram-min` |
+| Adapters (compatible sets can batch) | `--adapter` (`--adapter-path` alias) |
+| Speculative decoding (Qwen methods and full-attention/rotating lookup/standalone and Gemma assistant/DeepSpec/DSpark drafting share execution) | `--draft-model`, `--draft-kind`, `--num-draft-tokens`, `--ngram-max`, `--ngram-min` |
 | GLM-5.2 | `--mtp`, `--context-length` |
 | Sampling defaults | `--temperature` (`--temp` alias), `--top-p`, `--top-k`, `--max-tokens`, `--thinking`, `--hlg-sampling`, `--hlg-width`, `--hlg-shoulder`, `--hlg-toe`, `--hlg-pivot-offset` |
 | Parity tier | `--l1`, `--l2` (`--l3` errors — removed) |
 | Kill switches | `--compiled-decode`, `--compiled-activations`, `--fused-sdpa`, `--force-wire`, `--expert-offload` |
 
+Serving treats memory estimates as advisory by default and attempts the request.
+`--memory-budget` opts into estimated admission limits; `--kv-budget` opts
+into aggregate KV limits. `--max-tokens` supplies an explicit output cap.
+
 Endpoints: `/v1/chat/completions`, `/v1/completions`, `/v1/messages`,
 `/v1/responses`, `/v1/embeddings`, `/v1/models`, `/v1/adapters`, `/health`,
 `/stats`, `/fit`, `/library`, `/downloads`; the web app at `/` (`/chat`,
@@ -469,6 +491,11 @@ and optiq reference servers and writes a dated report:
 ```sh
 bun scripts/bench-serve.ts all
 bun scripts/bench-serve.ts all --models cpm5,e4b --arms mlx-bun,mlx-lm --out report.md
+bun scripts/bench-serve.ts all --models qwen27b --arms mlx-bun-serial,mlx-bun --logprobs --top-logprobs 3
+# Same KV settings on both executors
+bun scripts/bench-serve.ts all --models qwen27b --arms mlx-bun-serial,mlx-bun --kv-quant 4
+# Same adapter settings on both execution arms:
+bun scripts/bench-serve.ts all --models e4b --arms mlx-bun-serial,mlx-bun --adapter fixtures/adapters/upper
 ```
 
 Curated results: [benchmarks.md](benchmarks.md).
@@ -544,3 +571,15 @@ behind each "not covered" row is in
 Native verbs with no mlx-lm counterpart — `pi`, `harness pi`, `embed`,
 `fit`, `train-watch`, `memory`, `evals` — are the surface that makes mlx-bun
 more than a drop-in.
+
+
+`--kv-quant turbo` composes with grouped lookup, standalone, Gemma assistant,
+DeepSpec and DSpark drafting on supported targets. Sliding layers stay bf16.
+See [batching design](../design/batching.md) for composition evidence and limits.
+
+Paged KV retains the configured `--batch` size. Gemma4 bf16 requests use the shared executor and bypass the prompt cache; quantized KV and draft combinations remain unavailable. See [server configuration](server-config.md).
+
+`--generation-checkpoint N` requires `--ssd-cache` and queues owned snapshots
+for eligible ordinary requests. Shared continuation supports eligible Qwen, Llama, MiniCPM and Gemma4 requests with or without adapters, excluding media, grammar, fill, paging, logprobs or speculation.
+Repeat the identical request to replay and resume; flush before shutdown for
+eventual queued snapshots to become durable.

package/docs/reference/distribution.md

@@ -186,6 +186,13 @@ npm if the version is live. Env overrides: `MLX_BUN_SIGN_IDENTITY`,
 `NOTARY_PROFILE`, `BUILD_DIR`, `OUT_DIR` (release); `OUT_DIR`, `REPO`,
 `TAP_REPO` (publish).
 
+For an SSH-driven release, a local keychain unlock may still leave
+`notarytool` reporting `keychainLocked` in the SSH session. The v0.3.0
+release succeeded by running `release-binary.sh` in the M1 Max's local
+Terminal, then `publish-release.sh` over SSH. Both steps used the same clean
+merged source and `OUT_DIR`. Check `npm whoami` from the actual release
+checkout too: a project `.npmrc` can override the user's configuration.
+
 ### The notarization lesson (v0.2.0, 2026-08-24)
 
 The first v0.2.0 submission came back **Invalid**. The release added
@@ -311,7 +318,7 @@ sh scripts/build-native-pack.sh <ver> dist-native
 gh release create native-v<ver> \
   dist-native/mlx-bun-native-v<ver>-arm64.tar.gz \
   dist-native/mlx-bun-native-v<ver>-arm64.tar.gz.sha256 \
-  --title "mlx-bun native runtime v<ver>" --notes "<what changed>"
+  --latest=false --title "mlx-bun native runtime v<ver>" --notes "<what changed>"
 ```
 
 `build-native-pack.sh` stages the same six files with the same
@@ -323,13 +330,13 @@ package. Pre-publish smoke: a fresh-cache extraction plus a real MLX
 `dlopen` (and, when the helpers changed, an expert read or a video decode).
 The pack is ad-hoc signed only — `build-native-pack.sh` never touches a
 Developer ID and nothing notarizes it; it is `dlopen`ed by the user's own Bun
-process, not by our signed executable. The most recent pack
-(`native-v0.3.0`, published 2026-08-22) added `mlx-bun-frame-extract`.
+process, not by our signed executable. Use `--latest=false` so a native pack
+does not replace the application release behind the direct installer's URL.
 
-The working tree targets native pack 0.4.0 with MLX 0.32.2 and MLX-C
-`c74db5307cc8ce122f48d97ef951b30578674e7f`. Its six-file macOS 14 candidate
-has been built and verified locally. It is not published. Release the native
-asset before publishing any package that uses this manifest. Runtime bindings
+Native pack 0.4.0 was published on 2026-09-08 with MLX 0.32.2 and MLX-C
+`c74db5307cc8ce122f48d97ef951b30578674e7f`. Its six-file macOS 14 archive
+was uploaded from the M1 Max and downloaded again; its checksum matches the
+package manifest. It was published before mlx-bun v0.3.0. Runtime bindings
 match this one C API, and startup checks the linked core version. The old
 runtime and source controls remain separate for benchmark comparisons.
 
@@ -338,8 +345,8 @@ CI builds that pinned C wrapper with
 `MLX_BUN_LIBMLXC` to its `lib/libmlxc.dylib`. The script verifies the official
 macOS 14 MLX-Metal archive hash and checks out the exact MLX-C commit. It
 extracts native libraries and headers without running Python or compiling
-Metal sources. This also reproduces local native-pack build inputs before
-the candidate is published; Homebrew's C wrapper is not the CI runtime.
+Metal sources. This also reproduces the native-pack build inputs;
+Homebrew's C wrapper is not the CI runtime.
 
 Managed jobs in the compiled binary re-exec that binary through a private job
 entry, with the database/log paths passed in the environment. They do not depend

package/docs/reference/environment.md

@@ -134,10 +134,10 @@ eleven skips on the old core. The local native-pack
 candidate under `native-pack-candidate/` extracts into `native-pack-extracted/`;
 all five Mach-O files target 14.0, have valid ad-hoc signatures and load their
 non-system dependencies from siblings. These checks ran on macOS 27, not an
-actual macOS 14 host. The production native-pack manifest now targets this
-0.4.0 candidate and the verified local cache is
-`/Users/joshrossi/Library/Caches/mlx-bun/native-v0.4.0-arm64`. The candidate
-has not been published. The consolidated source passes 52 complete Qwen
+actual macOS 14 host. This build is published as native pack 0.4.0 and the
+verified local cache is
+`/Users/joshrossi/Library/Caches/mlx-bun/native-v0.4.0-arm64`. Publication
+details are in distribution.md. The consolidated source passes 52 complete Qwen
 forward/state/continuation cases and six Trellis generations, with fixed
 source/runtime/reference hashes and clean worker exits. The consolidated
 model-free suite passes 1,875 tests with ten skips; all typechecks pass.
@@ -179,6 +179,40 @@ against an M4 Pro output fixture. The test still requires exact output and
 state hashes; Python runs only the external reference. Trellis variant
 matrix tests allow 30 seconds for first-use kernel compilation on CI.
 
+The M1 Max override was replayed on 2026-09-08 with the same inputs using
+`/Users/joshrossi/Code/mlx-lm/.venv-mlx-0.32.2-macos14` (MLX/MLX-Metal
+0.32.2, MLX-LM 0.31.3). Bun matches its outputs and recurrent-state hashes
+exactly. The review file is
+`reports/qwen38-closeout/qwen-delta-m1-review.json`.
+
+Qwen3-Embedding tests also resolve `goldenPath("qwen3-embed")`, including
+machine overrides, and skip when their required binary blobs are absent.
+Regenerate with the matching Python and
+`scripts/oracle/gen-qwen3-embed-golden.py <artifact> --replay goldens/qwen3-embed/meta.json`;
+`--output-dir` selects a review directory. Replay preserves the text and token
+IDs and checks the tokenizer against them. Metadata records the device,
+runtime, artifact config, original metadata and output hashes. The M1 Max
+replay matches hidden states and pooled embeddings exactly. Cosine uses both
+vector norms: bf16 normalization does not guarantee a unit norm when the
+result is subsequently read as float32.
+
+The same M1 validation found June MiniCPM logit blobs without runtime
+provenance. Explicit `bun scripts/regen.ts minicpm5` with the matching venv
+preserves the artifact, prompt IDs and greedy trajectory; the regenerated
+logits match Bun exactly. The generator now records runtime/device/config
+provenance and all blob hashes. The old files and reviewed differences remain
+under `reports/qwen38-closeout/minicpm5-golden-before/` and
+`minicpm5-golden-review.json`.
+
+The Gemma bf16 and mixed-KV M1 replays also pass with unchanged prompt IDs
+on MLX 0.32.2. This current M1 venv contains **mlx-optiq 0.2.7** by installed
+distribution metadata, matching the documented M4 installation; it is not
+the historical 0.2.15 environment. OptiQ's module `__version__` incorrectly
+reports 0.2.5 there, so regeneration records distribution metadata. Old
+fixtures remain under `reports/qwen38-closeout/{parity,mixed-kv}-golden-before/`.
+These are current-stack parity checks, not a runtime-only A/B against the
+older mixed-KV fixture's 0.31.2/0.2.15 combination.
+
 Readable oracle source, in that `site-packages/`: `mlx_lm/models/gemma3.py`
 and `mlx_lm/server.py` for the port targets; `mlx_lm/models/cache.py` for the
 cache classes; `mlx_lm/tokenizer_utils.py` for prompt rendering;
@@ -250,7 +284,19 @@ servers DO validate: the Bearer token must start with `sk-optiq-`
 
 ## Bun and FFI facts
 
-**Bun ≥ 1.4.0.** `package.json` `engines.bun` is `>=1.4.0`. `Bun.Image`
+**Development and CI: Bun 1.4.2.** Build the committed frontend bundle with
+that version so its byte-for-byte freshness check uses the same compiler.
+Performance comparisons record the runtime and use the same version in both
+arms; historical results retain their original version.
+
+Non-login SSH on the M4 Pro can omit `/Users/joshrossi/.bun/bin` from `PATH`.
+Put that directory on the remote command's PATH when running the test suite:
+launching the parent Bun by absolute path alone does not let subprocess tests
+find `bun`. The quantized-row validation's initial seven command-entry/job
+failures disappear with that invocation corrected; no product code change was
+needed for those failures.
+
+**Runtime minimum: Bun ≥ 1.4.0.** `package.json` `engines.bun` is `>=1.4.0`. `Bun.Image`
 (native OS codecs via ImageIO, EXIF auto-orient, off-thread) is the vision
 decode path (`src/vision/preprocess.ts`); `Bun.Image.resize` is deliberately
 not used. The FFI ABI fix below is the other reason for the pin.
@@ -346,3 +392,7 @@ holds self-contained reproductions for upstream bug reports (the two Bun
 issues above, plus `optiq-mixed-kv-inert` and `vllm-metal-turboquant`); they
 are not part of the build. Committed absolute paths (the venv, the weights
 snapshot, `serve.sh`) are machine-specific on purpose — do not "correct" them.
+
+The quantized-KV golden producer (`bun scripts/regen.ts kvq`) passes an absolute output directory to Python, since the child runs from `scripts/`. This preserves the selected machine directory and existing prompt IDs. Missing M4 blobs were regenerated from the pinned oracle into the isolated `delayed-affine-m4-oracle` override; numerical expectations were not produced by the inference implementation. Evidence: `reports/qwen38-closeout/composition-baseline/oracle-generator-path/`.
+
+The M1 rotating-KV reference still identified MLX 0.31.2 and disagreed with both unchanged control and candidate on the KV8 greedy prefix. Regeneration with pinned MLX 0.32.2 retains the prompt IDs and passes exact logits plus both continuation checks. The matching M1 override and M4 reference caches are refreshed; previous manifests/blobs and provenance are retained with the generator-path evidence.

package/docs/reference/models.md

@@ -34,7 +34,7 @@ already-downloaded repos (it must match exactly one repo; see
 | `Qwen3-30B` | `mlx-community/Qwen3-30B-A3B-Thinking-2507-4bit` | Qwen3-MoE (`qwen3_moe`) · dedicated `qwen3-moe` graph · L1 (opt-in slow parity test, `MLX_BUN_TEST_QWEN3_MOE=1`) | text | two-model · ngram | bf16 (the artifact ships no kv_config.json) | Sparse top-k SwitchGLU experts. |
 | `diffusiongemma` | `mlx-community/diffusiongemma-26B-A4B-it-OptiQ-4bit` | DiffusionGemma (`diffusion_gemma`) · dedicated graph + **denoising loop** (non-autoregressive) · L2 (optiq) | text · **image** (single image; its own inline-quantized SigLIP tower). No video / audio | — (no AR verify loop) | — (canvas model; the `--kv-quant` axis does not apply) | Serial lane always. Diffusion-native LoRA trains. |
 | `GLM-5.2-colibri` | `mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp` @ `3cc8db99b1b13fc79325d987ba3c1c430766b3b8` | GLM-5.2 (`glm_moe_dsa`) · **Colibri container loader** + dedicated `glm5.2` graph (streamed experts) · **exact artifact profile**, L3 | text | **checkpoint-native MTP** (`--mtp on`, the default; mutually exclusive with `--draft-model`) | checkpoint-native compressed **MLA/DSA** cache (its own scheme; SSD tier v3 persists it) | Fixed-context admission: `--context-length` (4096) and `--max-tokens` (128) are reserved by the header-only resource equation before weights open; `--memory-budget` defaults to min(25 GiB, RAM). `--mtp off` enables ordinary batching. Embeddings, vision/audio, adapters, and training are unsupported. See [below](#direct-container-artifacts-glm-5x). |
-| Tier-0 generic | validated manifest (`tests/support/universal-manifest.ts`): `Llama-3.2-1B-Instruct-4bit`, `Qwen2.5-0.5B-Instruct-4bit`, `Qwen3-0.6B-4bit`, `quantized-gemma-2b-it`, `gemma-2-2b-it-4bit`, `Phi-3.5-mini-instruct-4bit`, `GLM-4-9B-0414-4bit`, `granite-3.3-2b-instruct-4bit`, `starcoder2-3b-4bit`, `SmolLM3-3B-4bit` (all `mlx-community/`) and `ekryski/OLMo-2-0425-1B-Instruct-4bit` | 11 `model_type`s: `llama`, `smollm3`, `qwen2`, `qwen3`, `gemma`, `gemma2`, `phi3`, `olmo2`, `glm4`, `granite`, `starcoder2` (+ mlx-lm remaps `mistral`, `iquestcoder` → `llama`) · `universal-dense` graph · L1 | text | two-model · ngram | bf16 (the validated cell) | `ls` labels these `supported (generic)`. Batch only for plain full-attention archs (gated B=2 on Llama-3.2-3B); gemma2-family / sliding-window archs run serial. No compiled specialization. A `model_type` absent from the table refuses to load. |
+| Tier-0 generic | validated manifest (`tests/support/universal-manifest.ts`): `Llama-3.2-1B-Instruct-4bit`, `Qwen2.5-0.5B-Instruct-4bit`, `Qwen3-0.6B-4bit`, `quantized-gemma-2b-it`, `gemma-2-2b-it-4bit`, `Phi-3.5-mini-instruct-4bit`, `GLM-4-9B-0414-4bit`, `granite-3.3-2b-instruct-4bit`, `starcoder2-3b-4bit`, `SmolLM3-3B-4bit` (all `mlx-community/`) and `ekryski/OLMo-2-0425-1B-Instruct-4bit` | 11 `model_type`s: `llama`, `smollm3`, `qwen2`, `qwen3`, `gemma`, `gemma2`, `phi3`, `olmo2`, `glm4`, `granite`, `starcoder2` (+ mlx-lm remaps `mistral`, `iquestcoder` → `llama`) · `universal-dense` graph · L1 | text | two-model · ngram | bf16; unreleased Llama full-attention uniform KV4/KV8 | `ls` labels these `supported (generic)`. Shared full-attention execution is qualified on Llama-3.2-3B at B1/B4, including speculative KV4/KV8 and generated RAM/SSD state; gemma2-family / sliding-window archs run serial. No compiled specialization. A `model_type` absent from the table refuses to load. |
 
 Rules that apply across rows (all enforced in `src/server.ts`):
 
@@ -45,8 +45,11 @@ Rules that apply across rows (all enforced in `src/server.ts`):
   MTP is its own row. Qwen MTP companions may store dense or affine-quantized
   projections; the loader uses each module's quantization metadata. Target
   verification is unchanged, while draft acceptance and speed depend on the
-  companion. Any quantized KV scheme excludes the speculative lane
-  (requests keep the scheme and decode serially without speculation).
+  companion. In the unreleased shared executor, qualified plain, uniform,
+  per-layer affine and TurboQuant layouts compose with drafting through the
+  target/provider interfaces. Positive affine conversion thresholds are also
+  supported by library requests. The loaded model and provider determine
+  which combinations can execute.
 - **Qwen3.5-family checkpoint generations.** Both ship the same graph and both
   load: the mlx-lm-converted naming (`language_model.model.*`,
   `language_model.lm_head`, `vision_tower.*`) and the transformers-5.8 export
@@ -60,7 +63,9 @@ Rules that apply across rows (all enforced in `src/server.ts`):
 - **KV schemes.** `--kv-quant 4|8` (uniform, L1) and `config` (per-layer
   `kv_config.json`, L2) apply to the autoregressive families that ship the
   file; `turbo` additionally requires a full-attention `head_dim` in
-  {64, 128, 256, 512} (refused at startup otherwise) and is solo-only;
+  {64, 128, 256, 512} (refused at startup otherwise). Qualified TurboQuant
+  targets support shared ordinary and speculative execution; sliding layers
+  remain bf16;
   `--paged-kv` is Gemma 4 only, bf16, serial. Flag detail:
   [server-config.md](server-config.md).
 - **Modalities.** Image requests reach a Gemma 4 model only when
@@ -323,3 +328,7 @@ Safety rails:
 
 After a real deletion, gc re-scans so the registry drops the reaped
 rows immediately.
+
+
+Non-Qwen grouped TurboQuant drafting, including assistant donor attention, uses
+the shared cache interfaces. See [composition coverage](../design/batching.md#turboquant-donor-attention-and-grouped-draft-composition).

package/docs/reference/server-api.md

@@ -117,8 +117,8 @@ Request body (OpenAI chat schema; unknown fields ignored):
   "stream": false,
   "max_tokens": 1024,            // or max_completion_tokens (wins); unset =
                                  // no cap — generate until EOS or the
-                                 // admitted context is exhausted (admission
-                                 // clamps; --max-tokens sets a server default)
+                                 // an explicit operator limit is reached;
+                                 // --max-tokens sets a server default
   "temperature": 0.7,            // 0 = greedy
   "top_p": 0, "top_k": 0,        // 0 = off
   "seed": 1234,                  // omit for time-derived
@@ -369,15 +369,34 @@ Speculative decoding is a **server-level mode** (`serve --draft-model`,
 or the model-free `serve --draft-kind ngram`, which mounts no draft
 model); there is no per-request draft field. The `speculation` usage
 extension appears on chat and text completions alike, non-streaming and on
-the final stream chunk. Spec-eligible requests are text-only on base weights
-(no adapter or logprobs capture, bf16 KV by default); experimental Qwen
-uniform KV4 start=0 can qualify through `MLX_BUN_QWEN_SPEC_KV4=1`.
-Ineligible requests decode normally and omit the field. The spec path bypasses
-the ordinary prompt cache. Qwen MTP with `MLX_BUN_MTP_PROMPT_CACHE=1` can reuse
-a paired target/draft RAM prefix, reported in `cached_tokens`; the default is
-zero. These prefixes do not persist across restarts. While a draft is mounted every request routes
-through the serial lane (mlx_lm.server parity: `is_batchable = draft is
-None`) — speculation and `--batch N` are different modes.
+the final stream chunk. Spec-eligible requests are text-only on base weights.
+In the unreleased shared executor, Qwen grouped MTP, prompt lookup and standalone drafting support logprobs and
+bf16, uniform KV4/KV8 or per-layer affine KV, or TurboQuant. Qwen MTP and prompt lookup also support
+positive library conversion thresholds for uniform KV4/KV8. Per-layer mixed KV and
+adapters remain incompatible with speculation. Strict legacy serial execution
+retains its narrower bf16/KV4 policy. The KV4 compatibility control is
+`MLX_BUN_QWEN_SPEC_KV4=0`.
+Ineligible requests decode normally and omit the field. Qwen MTP reuses paired
+target/draft prefill and completed decode state through the shared RAM/SSD cache by default in the
+unreleased source, reported in `cached_tokens`. Configured SSD persistence
+supports reuse after restart; `MLX_BUN_MTP_PROMPT_CACHE=0` disables reuse.
+Generated checkpoints cover processed tokens and use queued SSD persistence.
+Long-conversation performance acceptance remains open. Qwen prompt lookup also
+retains prefill and completed-output state in the common RAM/SSD cache, with
+its committed token history stored as a companion attachment. Standalone drafting
+on Qwen targets stores the draft model’s attention/recurrent state and pending
+last draft token as companion state, including restoration after restart. Qualified Qwen
+MTP, prompt lookup and standalone drafting use shared execution at one or several active rows and
+report `usage.lane: "batched"`. Supported full-attention and rotating targets, including
+Llama, MiniCPM and Gemma, use shared bf16, uniform KV4/KV8 or per-layer affine KV or TurboQuant lookup and standalone drafting, with
+logprobs and generated RAM/SSD checkpoints. Delayed and per-layer affine rotating speculation uses the same row lifecycle; positive conversion thresholds are a library setting, while served schemes default to zero. Gemma assistant drafting also shares execution with bf16, uniform KV4/KV8 or per-layer affine KV or TurboQuant target storage and stores its
+last true target hidden alongside generated RAM/SSD prefixes. Donor attention
+handles row validity independently of the draft graph. DeepSpec and DSpark also use these shared target layouts and the common RAM/SSD cache
+for projected context, with independent confidence-based proposal lengths.
+Other provider/target combinations still
+use the serial executor.
+Prompt-lookup concurrent performance acceptance remains open. Ineligible requests can use continuous ordinary execution
+when their features support it, even while a draft is mounted.
 
 `usage.fill` reports **token fast-forwarding** (`MLX_BUN_FILL=strict|echo`).
 Strict mode appends structural spans from the request's tools and chat
@@ -400,6 +419,13 @@ echo index proposed and `injected` is what survived. `wastedSamples` counts
 only ASSERT fills (a verify fill consumes the in-flight sample as its first
 position's check instead of discarding it).
 
+An explicit `seed` controls a request-local random stream and can use
+continuous execution. Repeating a request reproduces sampling at the same
+model arithmetic and batch composition. Different batch shapes can change
+floating-point results even with identical random draws; `--batch 1` retains
+the existing fixed-shape control. This differs from mlx-lm's policy of routing
+explicit seeds to its single-request generator.
+
 ### logprobs / top_logprobs
 
 With `"logprobs": true` and/or `"top_logprobs": k` (1–11), the
@@ -432,8 +458,8 @@ penalties), **before** the sampler's temperature/top-p/top-k/min-p/XTC.
 mlx-lm's top-k order is unspecified (argpartition); ours is sorted
 descending — the same set, so the entry is deterministically the argmax.
 Stream chunks never carry logprobs (mlx-lm's streaming responses don't
-either). Requests with logprobs run on the serial lane under `--batch N`
-(like the other mlx-lm sampler extensions). Invalid values are rejected
+either). Requests with logprobs can use ordinary continuous execution under
+`--batch N`; other feature exclusions still apply. Invalid values are rejected
 with mlx-lm's exact messages (see Errors).
 
 ### Streaming
@@ -544,11 +570,10 @@ Design and fidelity notes: [docs/reference/server-api.md](./server-api.md).
 
 ### Errors
 
-Runtime memory pressure can fail a request even after prompt-length admission.
-The serial memory guard first reclaims older prompt-cache entries, persisting
-them to the SSD tier when configured. If insufficient GPU headroom remains,
-the normal request/stream error path reports the failure. Saved in-flight
-generation checkpoints remain available for an identical retry.
+Memory estimates do not reject or shorten requests by default. Actual runtime
+failures use the request/stream error path when recoverable; native Metal
+allocation failure can terminate the process. Saved generation checkpoints
+remain available for an identical retry.
 
 All errors are `{ "error": { "message": …, ... } }`.
 
@@ -567,10 +592,12 @@ All errors are `{ "error": { "message": …, ... } }`.
   'low', 'medium', 'high', 'xhigh'`).
 - `400` with `"type": "memory_admission"`, `"code":
   "context_over_budget"` — the PROMPT itself leaves no generation slot
-  within the max safe context (`src/serve/request-plan.ts`). A prompt that
+  within an explicitly configured context/budget limit (`src/serve/request-plan.ts`). A prompt that
   fits is never rejected for a broad `max_tokens`: the upper bound is
   capped to the remaining room (`max_tokens` is a ceiling, not a promise)
-  and generation proceeds. The ceiling is visible at `/stats`.
+  and generation proceeds. This is opt-in via an operator budget/context cap
+  (or the fixed GLM-5.2 layout), never enforced from default fit estimates.
+  `/stats.admission.enforced_context_tokens` is null when no cap applies.
 - `429` with `"type": "resource_admission"`, `"code": "queue_full"` when
   a preparation or generation queue has 64 waiting requests. If SSE headers
   are already sent, the stream emits its terminal error instead.
@@ -590,7 +617,7 @@ and adapter selection as chat.
   "prompt": "Once upon a time",  // REQUIRED, non-empty string only (token
                                  // arrays rejected, matching mlx_lm.server)
   "max_tokens": 512,             // unset = no cap: generate until EOS or the
-                                 // admitted context (DEVIATION: mlx_lm.server
+                                 // an explicit operator limit (DEVIATION: mlx_lm.server
                                  // defaults 512 — pass --max-tokens 512 to
                                  // reproduce); max_completion_tokens wins
   "stream": false,
@@ -849,8 +876,9 @@ vision/audio, adapters, and training are not emulated by the serving port.
                  "pending_spill_bytes": 0, "dropped_spills": 0,
                  "failed_spills": 0, "longest_durable_prefix_tokens": 0 },
   "admission": {
-    "max_safe_context": 0,            // tokens; requests above this 400
-    "memory_budget_bytes": null,      // explicit budget, or null (machine default)
+    "max_safe_context": 0,            // advisory fit estimate, tokens
+    "enforced_context_tokens": null,  // explicit cap or fixed layout; null = none
+    "memory_budget_bytes": null,      // explicit budget, or null
     "usable_bytes": 0,
     "weights_bytes": 0
   },
@@ -924,7 +952,7 @@ generic all-resident SKU estimator is inapplicable
 ```jsonc
 {
   "machine": { "chip": "M4 Pro", "ram_bytes": 0, "bandwidth_gbs": 0.0 },
-  "context_tokens": 8192,          // current admission ceiling
+  "context_tokens": 8192,          // estimated context capacity
   "typical_context_tokens": 8192,  // min(8192, context_tokens)
   "typical_decode_tps": 0.0,       // predicted at typical_context_tokens
   "measured_decode_tps": null,     // real number from eval DB, or null
@@ -959,7 +987,11 @@ generic all-resident SKU estimator is inapplicable
 ## Adapters (LoRA hot-swap)
 
 Routes in `src/serve/model-admin-routes.ts`; the manager is
-`AdapterManager` in `src/lora.ts`.
+`AdapterManager` in `src/lora.ts`. Requests selecting the same ordered adapter
+set can share a continuous group. Different sets wait for the current group
+to finish and then use the same executor. Prefix-cache identity is resolved
+under the execution lease, so replacing an adapter while a request is queued
+cannot reuse state from its previous weights.
 
 - `GET /v1/adapters` — `{ adapters: [{ id, path, rank, scale, size_bytes,
   mounted_layers, ram_bytes }] }` — currently-mounted adapters only.
@@ -1719,3 +1751,25 @@ Under `--isolate`, the parent owns Responses continuation history and exposes it
 bounded store counters in `GET /engine` as `response_store`. Model-worker restart
 or eviction preserves that history; restarting the parent clears it. Direct
 serving retains the same process-local TTL and byte limits.
+
+
+Supported grouped TurboQuant drafting retains `usage.speculation`, seeded
+sampling, logprobs, structured output and generated RAM/SSD continuations.
+The cache codec does not select the scheduler or sampler implementation.
+Composition coverage: [batching design](../design/batching.md).
+
+With paged KV enabled, supported Gemma4 bf16 text requests report the batched lane when submitted through continuous scheduling. Seeded sampling, logprobs and grammar use the same interfaces. These requests bypass prompt-cache lookup/publication; paging does not provide SSD persistence.
+
+Ordinary adapter checkpoints are isolated by ordered adapter IDs and mounted
+content revisions. Base-model requests cannot restore or remove an adapter checkpoint.
+With generation checkpointing enabled, repeating an identical eligible request
+replays its saved completion prefix and resumes from the saved pending token.
+Cancellation retains queued snapshots; successful completion removes them.
+`POST /admin/cache/flush` drains both prompt spills and generation checkpoints,
+and its elapsed time and durability statistics include both queues. A hard
+process exit can lose snapshots that have not drained.
+
+For resumed generation checkpoints, `usage.prompt_tokens_details.cached_tokens`
+counts only the reused original prompt, bounded by `usage.prompt_tokens`. Saved
+completion tokens are replayed as completion output and do not increase that
+prompt-cache usage counter.