mlx-bun 0.0.13 → 0.2.0

package/README.md

@@ -83,7 +83,7 @@ runtime, so the only thing the first run fetches is the model.
 
 ### bunx (no install)
 
-Already have [Bun](https://bun.sh) ≥ 1.3.14? Run it straight from npm,
+Already have [Bun](https://bun.sh) ≥ 1.4.0? Run it straight from npm,
 nothing to install:
 
 ```sh
@@ -97,7 +97,7 @@ mlx-bun` under Node exits with a
 
 ### From source
 
-Needs [Bun](https://bun.sh) ≥ 1.3.14 — no Python, no native library to
+Needs [Bun](https://bun.sh) ≥ 1.4.0 — no Python, no native library to
 install by hand:
 
 ```sh
@@ -212,12 +212,12 @@ runtime that serves them:
   [quickstart](./docs/reference/orpo-quickstart.md)).
 - **The curve designer** — HLG tone-curve sampling with an interactive
   designer in the server UI at `/curves`
-  ([design](./docs/design/hlg-sampling.md)).
+  ([design](./docs/archive/hlg-sampling.md)).
 - **Speculative decoding research (DSpark)** — a KV-injection drafter
   ([design](./docs/design/dspark-speculative-decoding.md)).
 - The full research trail — including dead ends — is tracked in
   [docs/design/](./docs/design/) and
-  [docs/investigations/](./docs/investigations/).
+  [docs/archive/investigations/](./docs/archive/investigations/).
 
 On the site: [The lab](https://mlx-bun.dev/about/lab/).
 
@@ -234,7 +234,7 @@ the Qwen 3.x family, and the direct-container GLM-5.2 path:
 | [`mlx-community/gemma-4-e4b-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-e4b-it-OptiQ-4bit) | 7.0 GB | 16 GB | ✓ | **Recommended starter** (16 GB+); ~56 tok/s |
 | [`mlx-community/gemma-4-12B-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-12B-it-OptiQ-4bit) | 8.4 GB | 16 GB | ✓ | Vision + tool calling, both verified end-to-end |
 | [`mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit) | 18 GB | 24 GB | — | MoE (top-8 of 128 experts); ~54 tok/s |
-| [`mlx-community/Qwen3.8-27B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.8-27B-OptiQ-4bit) | 20.4 GB | 32 GB (24 GB with `--memory-budget`) | text-only for now | Current-gen Qwen; bf16-KV bit-exact parity (vs mlx-lm); thinking with `reasoning_effort` (xhigh/medium/low) + `preserve_thinking` + tool calling; MTP speculation, vision + video in bring-up |
+| [`mlx-community/Qwen3.8-27B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.8-27B-OptiQ-4bit) | 20.4 GB | 32 GB (24 GB with `--memory-budget`) | image + video | Current-gen Qwen; bf16-KV bit-exact parity (vs mlx-lm); thinking with `reasoning_effort` (xhigh/medium/low) + `preserve_thinking` + tool calling; native MTP head mounts as a `--draft-model` (lossless-by-verify, recurrent-cache rollback built in); **image AND video input serve** (`image_url` / `video_url` content parts — dedicated vision tower + mRoPE; video decodes via macOS-native AVFoundation, 2 fps ≤768 frames) |
 | [`mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp`](https://huggingface.co/mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp) | ~384 GB / 357 GiB | 32 GB + fast disk | — | Direct streamed Colibri container; true top-8, native MTP, bounded 25 GiB process plan; warm single-stream performance is disk-bound and below the aspirational 2 tok/s target |
 
 Not sure what fits your machine? `bun src/cli.ts fit <model> --ctx 8192`
@@ -381,7 +381,9 @@ agent CLIs like pi/OpenClaw via their provider config.
 - **Prompt caching** — a byte-capped LRU KV cache reuses the longest
   common token prefix across requests (multi-turn conversations
   re-prefill only the new turn) — automatic, no client changes.
-  Entries are adapter-specific when LoRA adapters are in play.
+  Entries are adapter-specific when LoRA adapters are in play. With
+  `--ssd-cache`, pending snapshots are flushed durably on `SIGINT`/`SIGTERM`;
+  `POST /admin/cache/flush` exposes the same boundary for automation.
 - **LoRA hot-swap** — mount adapters at runtime (`POST /v1/adapters`)
   and discover on-disk ones (`GET /v1/adapters/available`), select per
   request with the `adapter` body field (`"id"`, stacked `"a+b"`, or
@@ -416,12 +418,11 @@ agent CLIs like pi/OpenClaw via their provider config.
   targets, not small fast models. Serial lane only — with `--batch N` a
   mounted draft routes every request serial, like mlx_lm.server.
 - **Memory admission control** — `--memory-budget <GB>` refuses to load
-  a model that can't serve within the budget. Generic budgeted serving rejects
-  requests whose `prompt + max_tokens` exceed the safe context with a 400
-  (`type: "memory_admission"`) *before* generating. GLM-5.2 has a fixed,
-  pre-planned context instead: broad client `max_tokens` values are capped to
-  the remaining context, and only a prompt that leaves no generation slot is
-  rejected. The GPU
+  a model that can't serve within the budget. For requests, `max_tokens`
+  is a ceiling, not a promise: a broad client value is capped to the room
+  remaining under the safe context, and only a prompt that leaves no
+  generation slot at all is rejected with a 400
+  (`type: "memory_admission"`) *before* generating. The GPU
   out-of-memory crash it prevents is uncatchable by design — rejection
   up front is the only defense. Ceiling visible at `GET /stats`.
 - **Anthropic Messages API** (`POST /v1/messages`, on by default) —
@@ -478,6 +479,13 @@ for await (const t of gen) process.stdout.write(tok.decode([t.token], true));
 console.log(gen.stats); // prompt/decode tok/s, cached tokens, ...
 ```
 
+`openModel()` first resolves a declared model profile: exact immutable artifact
+profiles outrank dedicated family paths, which outrank the generic fallback.
+Profiles select the loader/model graph/generation loop and carry their fidelity
+contract; they never rewrite explicit or default-resolved KV, MTP, adapter,
+grammar, or sampling choices. `loadContext()` exposes the resolved declaration
+as `ctx.profile`, and direct library users can call `resolveModelProfile()`.
+
 KV caches can be persisted to disk (page-aligned files that reload as
 zero-copy GPU-safe mmaps — see `src/kv-store.ts`) so a standard agent
 preamble prefills once, ever.
@@ -566,7 +574,7 @@ On the site: [Correctness](https://mlx-bun.dev/about/correctness/) and
 - **Slow decode on a model near your RAM ceiling** — close memory-heavy
   apps; a model that doesn't fit the wired-memory budget pages weights
   every token (check with `fit`).
-- **Bun version** — requires ≥ 1.3.14 (`bun upgrade`); pinned for
+- **Bun version** — requires ≥ 1.4.0 (`bun upgrade`); pinned for
   `Bun.Image` and verified FFI behavior.
 
 ## Status
@@ -591,18 +599,20 @@ head-to-head benchmark harness; the decode-gap root-cause fix
 notarized — Homebrew, direct-download, and npm/bunx).
 
 **In progress** — `Qwen3.8-27B` full support (Phase 14 retarget): text
-parity is bit-exact and thinking controls (`reasoning_effort` depths,
-`preserve_thinking`) + tool calling are wired; native MTP speculation
-(the Qwen-trained head as a `--draft-model`), vision, and video input
-are in bring-up.
+parity is bit-exact, thinking controls (`reasoning_effort` depths,
+`preserve_thinking`) + tool calling are wired, and native MTP speculation
+(the Qwen-trained head as a `--draft-model`) mounts through the serve
+loop's recurrent-cache rollback contract; vision and video input are in
+bring-up.
 
 **Experimental** — opt-in, default-off, still being hardened: transparent
 expert offload for MoE models (`serve --expert-offload`, Phase 20:
 page-aligned mmap-backed experts, bit-exact — 26B-A4B 17.1→4.2 GB
-resident, decode unregressed); batched serving (`--batch N`, default 8
-since 2026-07-05 — a lone request runs the exact serial engine, so the cap
-engages only under real concurrency, e.g. coding sub-agents; `--batch 1`
-pins strict serial; Phase 18: continuous-batching bf16 decode at mlx-lm B=N parity,
+resident, decode unregressed); continuous serving (`--batch N`, default 8
+since 2026-07-05 — the scheduler's B=1 fast path uses serial-class caches and
+the parity-gated single-row graph, while the cap permits B=N under real
+concurrency such as coding sub-agents; `--batch 1` pins the preserved strict
+serial executor; Phase 18: bf16 decode at mlx-lm B=N parity,
 B=2 bit-parity verified for MiniCPM5/12B/e4b/26B; sampler extras,
 repetition penalty, and grammar-constrained requests batch; Qwen3.5's
 SSM caches batch, as do plain full-attention Tier-0 archs (e.g. Llama) —

package/bin/mlx-bun.mjs

@@ -4,7 +4,7 @@
 // (a guard inside cli.ts would never run under Node — its static imports
 // hoist and fail on the bun: scheme first).
 
-const MIN_BUN = "1.3.14"; // Bun.Image + verified FFI behavior (see README)
+const MIN_BUN = "1.4.0"; // fixed macOS arm64 FFI stack layout (see README)
 
 if (typeof Bun === "undefined") {
   console.error(

package/docs/{design → archive}/hlg-sampling.md

@@ -1,6 +1,6 @@
 # HLG sampling — a piecewise tone curve on the logits
 
-Status: **implemented, found neutral, superseded by Curve Designer** — `applyHlg` shipped in `src/sampler.ts`; investigation found HLG a wash vs plain temperature; the Curve Designer (`src/curve-sampler.ts`) is its v2 replacement. See [docs/investigations/hlg-sampling-investigation.md](../investigations/hlg-sampling-investigation.md) for findings. This doc remains as the design rationale.
+Status: **implemented, found neutral, superseded by Curve Designer** — `applyHlg` shipped in `src/sampler.ts`; investigation found HLG a wash vs plain temperature; the Curve Designer (`src/curve-sampler.ts`) is its v2 replacement. See [docs/archive/investigations/hlg-sampling-investigation.md](investigations/hlg-sampling-investigation.md) for findings. This doc remains as the design rationale.
 Owner: sampling layer (`src/sampler.ts`)
 Default: **off** (`--hlg-sampling off` = today's temperature sampler, untouched)
 
@@ -12,7 +12,7 @@ Default: **off** (`--hlg-sampling off` = today's temperature sampler, untouched)
 
 ## Outcome / Retrospective
 
-HLG was implemented through **Piece 2** (pure curve + neutrality tests + serial wiring, `applyHlg` live in `src/sampler.ts`). Evaluation found it **neutral vs plain temperature** — no measurable gain in diversity or quality at matched output entropy. As a result, HLG is **superseded by the Curve Designer** (`src/curve-sampler.ts`), which generalizes the same region-shaping idea with an editable control-point interface and cleaner UX. Full empirical findings are in [docs/investigations/hlg-sampling-investigation.md](../investigations/hlg-sampling-investigation.md).
+HLG was implemented through **Piece 2** (pure curve + neutrality tests + serial wiring, `applyHlg` live in `src/sampler.ts`). Evaluation found it **neutral vs plain temperature** — no measurable gain in diversity or quality at matched output entropy. As a result, HLG is **superseded by the Curve Designer** (`src/curve-sampler.ts`), which generalizes the same region-shaping idea with an editable control-point interface and cleaner UX. Full empirical findings are in [docs/archive/investigations/hlg-sampling-investigation.md](investigations/hlg-sampling-investigation.md).
 
 The sections below (Goal through Correctness) remain as design rationale. Pieces 3–6 of Phasing and the Open Questions were never executed; see the [Phasing](#phasing-the-piece-n-increments) and [Open Questions](#open-questions--risks) sections for historical context.
 
@@ -347,4 +347,4 @@ for it. That a transfer function is standard practice in another domain says
 nothing about whether it helps token sampling — that is decided only by
 measurement (the KL characterization, the capability guardrail, and the
 diversity lens in [Correctness](#correctness--evaluation)). The full empirical
-write-up is in [docs/investigations/hlg-sampling-investigation.md](../investigations/hlg-sampling-investigation.md).
+write-up is in [docs/archive/investigations/hlg-sampling-investigation.md](investigations/hlg-sampling-investigation.md).

package/docs/{investigations → archive/investigations}/hlg-sampling-investigation.md

@@ -3,7 +3,7 @@
 *2026-06-14. Question: can the structure of a piecewise nonlinear transfer
 function (HLG, the HDR transfer function) be transplanted onto a logit
 distribution as a sampling method, and if so, where do you place its pivot
-(middle grey)? Design contract: [docs/design/hlg-sampling.md](../design/hlg-sampling.md).
+(middle grey)? Design contract: [docs/archive/hlg-sampling.md](../design/hlg-sampling.md).
 Code: `src/sampler.ts` (`applyHlg`, `makeSampler`). Harness:
 `scripts/hlg-compare.ts`. Model under test: `gemma-4-e4b-it-OptiQ-4bit` on the
 M4 Pro.*

package/docs/archive/investigations/pareto-specialized-runtime-findings.md

@@ -0,0 +1,92 @@
+# Pareto-specialized runtime findings
+
+## Question
+
+Which parts of mlx-bun should converge on shared infrastructure, and which parts should remain deliberately specialized so cleanup does not turn the runtime into a lowest-common-denominator model host?
+
+This note compares three mature specialization clusters: the Colibri/GLM-5.2 runtime, the Qwen3.8 27B TurboQuant and MTP work, and ORPO training. The evidence says the same thing in all three places: share lifecycle and composition policy, but keep artifact layout and numerical work close to the model or method that owns them.
+
+## Shared control-plane infrastructure
+
+The shared layer decides what runs, owns resources, and records what happened. It does not rewrite the selected computation.
+
+- **Model construction.** `openModel()` recognizes the direct Colibri artifact before ordinary safetensors loading. `createModel()` prefers dedicated and fingerprint-generated model classes before the universal fallback, and explicitly says the generic path must not shadow a dedicated port. This is shared dispatch with specialized implementations, not one universal graph. [Factory dispatch](../../../src/model/factory.ts#L30-L150)
+- **Concurrency and request composition.** The target is one concurrency-driven engine. One active request uses the B=1 pipelined or compiled fast path; concurrent requests use the general step up to `--batch N`. B=1 is an internal specialization, not a permanent public lane. Features are stackable layers, and scheduling must not replace the selected KV scheme, adapter, draft method, grammar, or sampler. [Unified engine decision](../../design/unified-engine-frontier-plan.md#L333-L403) [Composition matrix](../../design/unified-engine-frontier-plan.md#L405-L424)
+- **Cache lifecycle and capability.** `Cache` owns update, masks, state, rollback, accounting, and disposal. `BatchableCache` adds row merge, extraction, filtering, and byte projection. The scheduler can depend on those capabilities without knowing whether a row contains ordinary KV, recurrent state, TurboQuant tensors, or compressed GLM state. [Cache interface](../../../src/model/gemma4-base.ts#L264-L303) [Batchable cache interface](../../../src/model/gemma4-base.ts#L347-L357)
+- **Resolved KV policy.** `KvScheme` preserves bf16, uniform affine, per-layer affine, and TurboQuant as distinct immutable schemes. It centralizes names, accounting, and current batch capability checks without pretending the schemes have identical storage or kernels. [KV scheme](../../../src/kv-scheme.ts#L84-L169)
+- **Draft lifecycle.** `DraftProvider` owns server-lifetime machinery and opens a request-scoped `DraftSource`. The source interface carries prompt mode, target taps, draft, commit, rollback-related cache state, and disposal. Assistant, DSpark, Qwen MTP, and native GLM MTP can share speculative orchestration while retaining different state transitions. [Draft source](../../../src/spec/source.ts#L31-L101) [Draft provider](../../../src/spec/source.ts#L103-L118)
+- **Expert residency policy.** The budget equation, stable expert union, lease state, backend contract, telemetry, and pressure correction are reusable. GLM supplies artifact geometry and byte segments to that policy. [Residency planning and union](../../../src/expert-residency.ts#L17-L155) [Residency backend](../../../src/expert-residency.ts#L157-L170)
+- **Offline transform orchestration.** `WeightTransform` separates name/config planning from lazy tensor application. Llama, the Qwen3.5 trunk, and Qwen MTP are real adapters over the same conversion seam. [Weight transform interface](../../../src/quantize/weight-transform.ts#L40-L51) [Qwen transform adapters](../../../src/quantize/weight-transform.ts#L244-L268)
+- **Training orchestration.** `TrainConfig` names independent techniques such as segmented backward, fused or flash CCE, prefix sharing, regularization, and ORPO semantics. `orpoLoop()` composes the selected techniques and chooses the model-specific backward implementation. [Training configuration](../../../src/train/trainer.ts#L42-L150) [ORPO composition](../../../src/train/trainer.ts#L821-L948)
+
+## Deliberately specialized data-plane code
+
+### Colibri and GLM-5.2
+
+GLM-5.2 is not a descriptor-table variation of a dense transformer. Its graph has MLA, DSA, shared plus routed MoE, native MTP, and compressed cache state. Its public artifact also has a dedicated container and streamed expert layout. The design therefore calls for a dedicated `Glm52Model`, direct artifact support, and a one-read routed-expert kernel rather than repacking into generic projection-major tensors. [Dedicated graph and artifact contract](../../design/colibri-glm52-port.md#L323-L357)
+
+The memory contract is also model and artifact specific. A header-only preflight reads the container, derives expert slot layouts, counts DSA and MTP state, and refuses startup when the exact resident weights, expert slabs, compressed KV, verification transients, allocator reserve, Bun reserve, and safety margin exceed the process limit. [GLM geometry and preflight](../../../src/model/glm52-memory.ts#L340-L500) The runtime then maps Colibri segments into generic residency leases, but retains GLM-specific slot layouts, MTP slabs, signed-int8 requirements, and routed-SwiGLU executors. [GLM residency binding](../../../src/model/glm52-residency.ts#L138-L230) [GLM executor construction](../../../src/model/glm52-residency.ts#L300-L459)
+
+That split is intentional. Bun owns residency policy and observability; the native helper owns aligned slabs, `pread`, wiring, and completion fences; the GLM model owns tensor semantics and kernels. [Ownership map](../../design/colibri-glm52-port.md#L580-L600)
+
+### Qwen3.8 27B, TurboQuant, and MTP
+
+Qwen3.5/3.8 keeps a dedicated graph because gated DeltaNet recurrence, full attention, MRoPE, output gates, and recurrent speculative rollback are numerical behavior, not configuration decoration. [Qwen graph](../../../src/model/qwen3_5.ts#L128-L322) [Qwen model interface](../../../src/model/qwen3_5.ts#L512-L673)
+
+Qwen MTP preserves its own artifact and state machine behind `DraftProvider` and `DraftSource`. The provider rejects a non-Qwen target or a hidden-size mismatch. The request source uses tapped pre-final-norm target state, maintains its own KV, samples through the target sampler, and rebuilds accepted rows from verified target hiddens. Those rules must not move into generic speculative orchestration. [Qwen MTP provider](../../../src/spec/qwen-mtp-source.ts#L193-L235) [Qwen MTP state machine](../../../src/spec/qwen-mtp-source.ts#L237-L430)
+
+TurboQuant KV demonstrates the right form of specialization. `TurboQuantKVCache` implements the shared `Cache` lifecycle, while keeping packed indices, scales, zero points, FWHT-domain values, supported head dimensions, and conversion rules inside the class. [TurboQuant cache](../../../src/model/gemma4-base.ts#L1539-L1800) The weight pipeline follows the same rule. `WeightTransform` is shared, but the Qwen corridor map preserves the output gate, recurrent internals, vision-to-residual seam, untied 27B head, and companion MTP basis. [Qwen transform seam](../../design/turboquant-weights.md#L80-L87) [Qwen corridor map](../../design/turboquant-weights.md#L125-L174)
+
+### ORPO training
+
+ORPO benefits from composition, not one generic backward. The trainer combines prefix sharing, segmented recomputation, fused or flash CCE, LoRA regularization, and objective semantics. The data plane remains specialized where the model topology demands it. Gemma4 segmented ORPO threads per-layer inputs and cross-segment donor KV cotangents; MiniCPM5 uses a simpler segment graph. [Composed ORPO paths](../../design/orpo-training.md#L178-L218) [Gemma4 segmented backward](../../../src/train/segmented.ts#L1137-L1471)
+
+Flash CCE is a shared quantized-head interface with specialized Metal kernels and explicit shape guards. Its implementation selects Steel, simdgroup, or scalar kernels from actual hidden, vocabulary, group, and tile geometry, and exposes exact-gradient escape hatches. Flattening this into a generic matrix-loss helper would lose the memory and watchdog constraints that justify it. [Flash CCE head](../../../src/train/flash-cce.ts#L42-L64) [Forward dispatch](../../../src/train/flash-cce.ts#L1033-L1132) [Backward dispatch](../../../src/train/flash-cce.ts#L1134-L1234)
+
+Dynamic lambda is not implemented. Its design belongs in the ORPO control plane: validation metrics update a live lambda source in `orpoLoop()`, while every monolithic and segmented loss reads the same value. It should not create another family of backward classes. [Dynamic-lambda status and signals](../../design/orpo-dynamic-lambda.md#L1-L27) [Proposed wiring](../../design/orpo-dynamic-lambda.md#L163-L180)
+
+## Interfaces that already preserve specialization
+
+| Interface | Shared invariant | Specialization retained behind it |
+|---|---|---|
+| `RuntimeModel` plus factory dispatch | Construction and serving can hold one model type | Dedicated GLM, Qwen, Gemma, diffusion, MoE, and generated graphs |
+| `Cache` / `BatchableCache` | Lifecycle, rollback, row operations, byte projection | Plain, rotating, quantized, TurboQuant, SSM, MLA, and DSA storage |
+| `KvScheme` | One resolved KV choice and stable identity | Per-scheme accounting, conversion, persistence, and kernels |
+| `DraftProvider` / `DraftSource` | Request lifecycle, target taps, draft and commit | Assistant, DSpark, Qwen MTP, and GLM MTP state machines |
+| `ExpertResidencyBackend` | Lease and I/O lifecycle | Colibri container offsets, slot packing, expert kernels, and prediction policy inputs |
+| `WeightTransform` | Plan first, apply lazily, record identity | Family-specific exact fold corridors and companion artifacts |
+| ORPO trainer configuration | One run composes selected techniques | Model-specific masks, donor topology, VJPs, and Metal kernels |
+
+## Declared profile seam (S2, 2026-08-21)
+
+Model construction now resolves one `ResolvedModelProfile` before opening
+weights. An exact artifact declaration binds an immutable external fingerprint,
+config fingerprint, fidelity contract, required engine capabilities, and a
+loader/graph/loop composition. Exact Qwen3.8 OptiQ and GLM-5.2 Colibri revisions
+are pinned; other supported artifacts retain dedicated family resolution and
+then the universal dense fallback.
+
+The factory consumes the resolved composition, and the serving context retains
+it for library callers. A matched exact declaration with a config or capability
+mismatch refuses rather than taking another route. The profile seam does not
+own request methods: MTP, KV schemes, adapters, grammar, and sampling remain
+independently resolved inputs to completion execution. This keeps the control
+plane declarative without turning a model profile into a feature preset.
+
+These interfaces preserve specialization because they describe lifecycle or capability, not tensor algebra. They also have multiple real implementations. That is the useful test for whether an abstraction belongs.
+
+## Rules for steady-state cleanup
+
+1. **Unify control, not math.** Centralize admission, concurrency, cancellation, ownership, accounting, persistence, telemetry, and technique composition. Keep tensor layout, recurrence, cache encoding, model topology, and kernel selection with their owning model, artifact, or method.
+2. **Keep one concurrency-driven engine.** B=1 and B>1 are internal step specializations in one engine. `--batch N` is a concurrency cap. Observed placement may be reported, but serial versus batch must not become a permanent public mode or feature-selection interface. Current feature-specific serial routing is capability debt.
+3. **Never weaken a resolved composition to fit a path.** If batching lacks TurboQuant, MTP, adapters, grammar, or another selected method, close that capability gap or reject the unsupported composition. Do not silently substitute bf16 KV, disable drafting, or change sampling.
+4. **Prefer capability interfaces over family checks.** Scheduling should ask caches whether they can merge or project bytes and draft sources what target state they require. Factory dispatch is the proper place for model identity checks.
+5. **Let dedicated ports outrank generic fallbacks.** The universal model path is valuable for ordinary architectures. It must not absorb GLM MLA/DSA, Qwen recurrence, artifact-specific MTP, or another optimized graph merely to reduce class count.
+6. **Treat artifacts as executable contracts.** Validate manifests, tensor families, layouts, quantization metadata, companion seeds, and memory equations before allocation. Keep artifact-specific loaders and transforms when repacking would add copies, lose reproducibility, or change arithmetic.
+7. **Generalize only after the invariant is visible in two implementations.** Extract lifecycle and capability seams such as `DraftSource` or `WeightTransform`. Do not extract a common tensor algorithm from code that only looks similar but has different masks, state, accumulation order, or oracle.
+8. **Compose optimizations at the orchestrator, then test the cross-product.** ORPO proves that prefix sharing, flash CCE, segmented backward, and regularization can stack while model-specific backward code remains intact. Serving needs the same discipline for model graph, KV scheme, MTP, adapters, grammar, sampling, and concurrency.
+9. **Bind every specialized path to its own oracle and measured purpose.** A specialized module earns its place through parity, memory, quality, or throughput gates on the artifact and hardware it targets. If the benefit disappears, delete the specialization instead of preserving it as architecture folklore.
+
+## Strongest conclusion
+
+The steady state is not a generic engine with scattered exceptions. It is a shared control plane that carries authoritative compositions through one concurrency-driven engine, plus specialized data planes selected by model, artifact, and numerical method. That structure preserves the project's Pareto advantage: common serving and training behavior where sameness is real, and unapologetically specific code where the result depends on it.

package/docs/{planning → archive/planning}/chat-ui-vision.md

@@ -1,6 +1,6 @@
 # The mlx-bun chat surface: a vision
 
-*Draft vision, product/design lead pass — 2026-07-06. Bold stance: reimagine the chat surface from scratch. Companion to the market research and current-state audit that produced it; see `docs/planning/web-ui-pass-plan.md` for the tactical bug list and `docs/design/adapters-end-to-end.md` / `docs/design/memory-system.md` for the backend contracts this vision assumes.*
+*Draft vision, product/design lead pass — 2026-07-06. Bold stance: reimagine the chat surface from scratch. Companion to the market research and current-state audit that produced it; see `docs/archive/planning/web-ui-pass-plan.md` for the tactical bug list and `docs/design/adapters-end-to-end.md` / `docs/design/memory-system.md` for the backend contracts this vision assumes.*
 
 ## The feeling
 

package/docs/{planning → archive/planning}/release-notes-v0.0.9.md

@@ -117,6 +117,6 @@ overturning a load-bearing performance assumption.
   "failures" were M4-Pro fixtures; all code exonerated — mlx-lm reproduces
   mlx-bun token-for-token per machine); CI gate (typecheck + model-free tests).
 - **Research (the lab):** decode-roofline re-measurement overturned the "at
-  the floor" assumption (docs/investigations/decode-roofline-lookagain.md);
+  the floor" assumption (docs/archive/investigations/decode-roofline-lookagain.md);
   curve-sampler distinctness theorem + witness numbers + preregistered
   protocol (docs/planning/curve-sampler-research-plan.md).

package/docs/{planning → archive/planning}/website-readme-pass-plan.md

@@ -20,7 +20,7 @@ Two coherent options:
 
 - **(A) Recommended: cut v0.0.9, then deploy the site.** All docs become
   true simultaneously; nothing below needs version annotations. The
-  release-notes draft already exists (`docs/planning/release-notes-v0.0.9.md`).
+  release-notes draft already exists (`docs/archive/planning/release-notes-v0.0.9.md`).
 - **(B)** If v0.0.9 is not imminent: hold site deploys, and treat every
   item marked *v0.0.9* below as blocked. (Reverting docs to 8090 is churn
   for nothing — don't.)

package/docs/design/colibri-glm52-port.md

@@ -692,7 +692,7 @@ batch.
 
 G0 inventory note (2026-07-21): the exact pin, artifact/layout contract, and
 manual checklist are recorded in
-[`../investigations/colibri-oracle-pin.md`](../investigations/colibri-oracle-pin.md).
+[`../archive/investigations/colibri-oracle-pin.md`](../archive/investigations/colibri-oracle-pin.md).
 Because the pinned Makefile writes ignored build artifacts and even creates a
 configuration stamp while parsing, model-free oracle targets must run from a
 temporary `git archive` extraction, not in the read-only checkout. The pinned

package/docs/design/compat-cli-surface-design.md

@@ -85,7 +85,7 @@ mlx-lm" half.
 
 ### pi — the built-in agentic coding CLI
 
-(Phase 16; full plan in **docs/investigations/pi-builtin-investigation.md**.) pi is an
+(Phase 16; full plan in **docs/archive/investigations/pi-builtin-investigation.md**.) pi is an
 MIT, Bun-native (`bun build --compile`) coding agent — the "CLI
 experience inside our package" that neither Python package has. Two
 commands, one for each audience:

package/docs/design/decode-speed-program.md

@@ -66,7 +66,7 @@ grammar composition, and `usage.speculation` telemetry ALL landed
   serve wiring: 27B/12B retarget (regen+train, Josh's shell), data scale
   (thousands of on-distribution generations vs 160), draft-loop
   tightening. At τ≈3 on the 27B agent workload: **~15 → ~35–45 tok/s**
-  (docs/investigations/dspark-handoff.md). Slots in as `DflashSource`,
+  (docs/archive/investigations/dspark-handoff.md). Slots in as `DflashSource`,
   no-oracle/KL-gated (a Lab item).
 
 ### 2. mlx bump — narrow 26B / spec-decode wins, NOT a general decode lever  [main-only, from-source]

package/docs/design/docs-reorg-plan.md

@@ -140,7 +140,7 @@ files, not benchmark output. Leave entirely alone.
 | 5 | Add `STATUS.md` (or a pinned top section in PLAN.md) holding only live handoff: current state + next action + blockers. Move PLAN.md's `NEXT UP` / `NEXT SESSION PICKUP` / `SESSION SWEEP` blocks into it. | Separates transient "do this next" from 2,475 lines of durable phase history; gives agents one entry point. |
 | 6 | Create `docs/planning/` and move `PRODUCT_ROADMAP.md`, `IDEAS.md`, `ResearchTopics.md`, `journal.md` under it. Leave only README + PLAN.md (+ STATUS.md) at root. Add a ~6-line "doc map" to README. | Root is currently a 6-file planning thicket; extends the taxonomy that already works in `docs/`. |
 | 7 | Pick one home for the optimization plan: fold `docs/design/optimization_plan.md` into PLAN.md Phase A–E, or have PLAN.md link out to it as canonical. | Two homes for the same plan = drift. |
-| 8 | Resolve the archive overlap: `archive/pi-builtin-investigation.html` duplicates [docs/investigations/pi-builtin-investigation.md](../investigations/pi-builtin-investigation.md). Decide whether `archive/` (tracked; 2 HTML files: `mlx-bun-lab-report.html` + `wwdc-mlx-bun.html`) should be tracked at all, or moved to an ignored location. Keep the `.md`, drop the duplicate. | Stops committing heavyweight generated HTML; one source per investigation. |
+| 8 | Resolve the archive overlap: `archive/pi-builtin-investigation.html` duplicates [docs/archive/investigations/pi-builtin-investigation.md](../archive/investigations/pi-builtin-investigation.md). Decide whether `archive/` (tracked; 2 HTML files: `mlx-bun-lab-report.html` + `wwdc-mlx-bun.html`) should be tracked at all, or moved to an ignored location. Keep the `.md`, drop the duplicate. | Stops committing heavyweight generated HTML; one source per investigation. |
 
 ## P2 — Polish (optional)
 

package/docs/design/dspark-serving-program.md

@@ -4,7 +4,7 @@ Status: **PLANNED** (2026-07-07). Trigger for Josh's pickup: TurboQuant merge
 but only Phase 5 actually depends on it; Phases 0–4 are start-anytime.
 Owner: decode / spec. Predecessors: PR #19 (DSpark paper code-complete +
 serve-integrated + DeepSpec oracle), the first-live-run findings in
-[dspark-handoff.md](../investigations/dspark-handoff.md) "FIRST LIVE RUN".
+[dspark-handoff.md](../archive/investigations/dspark-handoff.md) "FIRST LIVE RUN".
 
 ## The problem, in numbers (2026-07-07 first live run, loaded machine)
 

package/docs/design/dspark-speculative-decoding.md

@@ -247,6 +247,56 @@ regenerated by the target, 10 epochs, batch 512, lr 6e-4, position decay
 exp(−pos/4), ~38 TB cached hiddens — i.e. our "thousands of generations" data
 estimate was directionally right and their scale is far beyond a laptop run.
 
+## 27B program (2026-08-19): the TQ quant unlocks the real target
+
+The turboquant-weights campaign shipped rot-sens48 Qwen3.8-27B at 17 GB
+(docs/design/turboquant-weights.md) — and it changes both DSpark blockers
+this doc recorded:
+
+- **Target speed:** measured ~11.8 tok/s decode. This is the slow-target
+  regime where the doc's own analysis says τ≈3 nets 2–3× (≈25–35 tok/s) —
+  "the ideal target and the real motivation," now actually runnable.
+- **Training feasibility:** "27B memory-infeasible to train on 24 GB" (open
+  items below) assumed the bf16 trunk (54 GB). With the quantized trunk
+  FROZEN (inference-only: hidden taps + teacher logits) and only the small
+  drafter (~5 layers) holding grads/optimizer state in bf16, 27B-target
+  drafter training fits a 32 GB machine. The premise is dead; the line in
+  the gated item is superseded by this section.
+- **Rollback risk retired:** the stock Qwen-MTP spec path already serves
+  this model losslessly (68% per-token acceptance), so verify/rollback
+  across the 48 DeltaNet recurrent-state layers is proven engine behavior,
+  not new work.
+
+Two tracks, cheap-first:
+
+- **Track A — confidence-scheduled verification on the stock MTP head (no
+  drafter training).** Dynamic per-step block length instead of fixed
+  `--num-draft-tokens`: v0 confidence = the draft's own token probability
+  (no trained head), mapped through STS calibration
+  (`scripts/dspark-calibrate.ts` pattern) to expected acceptance; the
+  scheduler stops drafting when marginal expected-accept drops below the
+  verify amortization point. Calibration needs thousands of
+  (confidence, accepted?) pairs from served traffic — hours, not weeks.
+  Gate: paired A/B decode tok/s vs fixed-γ MTP on the same prompts;
+  lossless by construction (verify unchanged).
+- **Track B — true DSpark drafter for the 27B (multi-token single-pass).**
+  The faithful v2 (KV injection, Markov head) retargeted: `hiddenTap` added
+  to `qwen3_5.ts` forwardLayers (gemma4 has it; parity-safe no-op when
+  off), tap-layer selection for the 64-layer trunk (bias toward the 16
+  full-attention layers + post-finalNorm), data generated by TEACHER-FORCED
+  passes over corpus text with the frozen quantized trunk (every position
+  of a forward yields a sample — data scale is forward-passes over tokens,
+  not generate calls). Open idea: distill from / initialize against the
+  shipped MTP head to shortcut the acceptance floor. Gate: generalizing
+  per-position acceptance must beat the MTP head's 0.68 or Track B loses to
+  Track A + MTP and gets dropped.
+
+Sequencing: after the benchmark campaign closes (GPU is owned by the
+certification suite). Track A first — it's measurable in a day and its
+calibration data collection is also Track B's baseline instrumentation.
+Adapters/DSpark drafts trained on the TQ artifact are married to its
+rotation basis (fine for our bundle; note on the card if published).
+
 ## Open items / next
 
 - [x] Serve integration (Phase 1) · [x] Alg-1-shaped scheduler (matches the
@@ -258,11 +308,11 @@ estimate was directionally right and their scale is far beyond a laptop run.
   oracle scripts — the no-training path to the 12B speedup (in progress).
 - [ ] **Josh-gated GPU (the payoff):** real data scale (thousands of
   on-distribution generations) + **retarget to 12B** (`--model` +
-  `--tap-layers 5,17,29,41,46` (DeepSeek's trained taps); 27B memory-infeasible to train on 24 GB, kept
+  `--tap-layers 5,17,29,41,46` (DeepSeek's trained taps); ~~27B memory-infeasible to train on 24 GB~~ (superseded — the 17 GB TQ trunk makes it feasible; see "27B program" above), kept
   dim-generic) + train (`--seq-head` A/B optional) + **calibrate**
   (`scripts/dspark-calibrate.ts`) + live-τ measure. Recipe:
-  [docs/investigations/dspark-handoff.md](../investigations/dspark-handoff.md).
+  [docs/archive/investigations/dspark-handoff.md](../archive/investigations/dspark-handoff.md).
 - [ ] Verify the Eq 6 / §3.2.1 shapes against the actual paper when the PDF
   is available (both flagged in code).
 
-Full session handoff: `docs/investigations/dspark-handoff.md`.
+Full session handoff: `docs/archive/investigations/dspark-handoff.md`.

package/docs/design/grammar-spec-batching-integration.md

@@ -15,7 +15,7 @@ three workstreams so they compose deliberately instead of colliding:
   (docs/design/batching-perf-path.md).
 - **Speculative decoding** — research proven (assistant drafter bit-exact
   vs optiq, 12B γ=1 ≈ 1.09× win, docs/design/spec-decode-larger-targets.md;
-  DSpark architecture verified, docs/investigations/dspark-handoff.md) but
+  DSpark architecture verified, docs/archive/investigations/dspark-handoff.md) but
   **not served**: no `--draft-model`. Build plan already written:
   docs/design/mlx-lm-tool-parity-plan.md §7 (the `DraftSource` interface,
   two-model L1 source, serve loop, admission, prompt-cache composition).

package/docs/design/mlx-lm-serving-execution-seam.md

@@ -0,0 +1,81 @@
+# mlx-lm serving execution seam
+
+## Question
+
+What should mlx-bun borrow from the pinned mlx-lm oracle when deepening Completion execution around this flow?
+
+```text
+prepared prompt -> semantic events -> terminal summary
+```
+
+This note reads the local mlx-lm 0.31.3 source installed in the reference venv. The comparison uses the serving term defined in `CONTEXT.md`: a **Completion execution** includes output, resource lifetime, usage, and scheduling placement.
+
+mlx-lm is the correctness oracle only for behavior and numeric schemes it implements. It is not mlx-bun's architecture oracle or capability ceiling. mlx-bun also implements explicitly selected features such as mixed-precision KV, TurboQuant, and native draft methods. The execution path must preserve those resolved settings exactly.
+
+## Implementation status
+
+The first consolidation slice landed on 2026-08-21. `prepareCompletion()` performs admission before a streaming response starts and returns an opaque, single-use `PreparedCompletion`. `CompletionExecutor.execute()` then owns placement reporting, `CompletionSink`, logprobs, semantic event delivery, finish reason, usage, and pre-generation cleanup. [`completion-executor.ts:75`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/serve/completion-executor.ts:75) [`completion-executor.ts:219`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/serve/completion-executor.ts:219)
+
+Chat and raw-text adapters still declare their semantic components and format their own wire responses. Their streaming and non-streaming branches now call the same executor. `GenerationGateway` still owns placement and model execution. The change does not add or remove flags, alter defaults, or substitute one inference method for another.
+
+S3 completed the scheduling declaration at the same seam. A
+`GenerationPlacement` now names `serial` or `continuous`; it no longer calls a
+lone scheduler request "batched." The continuous scheduler chooses its B=1
+fast path or B=N step from active rows. The strict serial executor remains for
+`--batch 1` and for declared compositions the continuous scheduler does not yet
+implement. Existing parity evidence covers both the default scheduler's B=1
+path and strict serial. The recorded scheduler/serial decode ratios are
+0.992–0.996, and a fresh main-vs-branch serial gate after the sampler
+consolidation measured a 1.000 median-best ratio (about 265.0 tok/s on both
+sides, MiniCPM5, two interleaved 256-token repeats).
+
+Final acceptance after the GPQA pause exercised the seam through the complete
+two-shard suite and a real two-turn browser conversation. The suite passed
+2,064 tests with zero failures. The browser run verified streaming, tool
+events, context carryover, metrics, and turn recovery. A mainline-only queued
+DOM render race found by that run was fixed at the UI lifecycle boundary; no
+decode-loop work was added. The architecture changes likewise keep accounting
+and capability decisions outside numerical kernels. The fresh serial A/B plus
+the established scheduler B=1 parity and latency evidence are the performance
+gate.
+
+## What the oracle actually does
+
+The HTTP adapter creates a small `CompletionRequest` that still contains protocol-shaped chat messages or a raw text prompt. It separately builds `GenerationArguments`. Prompt preparation has not happened yet. [`server.py:182`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:182) [`server.py:198`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:198) [`server.py:1368`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1368)
+
+`ResponseGenerator.generate()` submits that pair to one worker thread through a queue. The worker applies the chat template, tokenizes, finds cache segments, chooses the initial reasoning state, constructs stop/tool/reasoning transitions, selects a lane, owns cache work, and emits per-token `Response` records. [`server.py:516`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:516) [`server.py:626`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:626) [`server.py:688`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:688) [`server.py:1026`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1026)
+
+That organization can be summarized as:
+
+```text
+HTTP request
+  -> CompletionRequest + GenerationArguments
+  -> ResponseGenerator queue
+  -> tokenize + state machine + lane + cache
+  -> token Response { text, token, state, match, logprob, finish_reason }
+  -> HTTP handler accumulates semantics and formats the protocol response
+```
+
+The model loader is separate. `utils.load()` returns the model and a tokenizer wrapper, while tokenizer loading chooses the streaming detokenizer and tool parser. This keeps model construction out of per-request execution. [`utils.py:453`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/utils.py:453) [`tokenizer_utils.py:578`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/tokenizer_utils.py:578)
+
+## Comparison by concern
+
+| Concern | Oracle organization | Consequence for the mlx-bun seam |
+| --- | --- | --- |
+| Prepared prompt | The worker owns chat templating, tokenization, initial reasoning state, and cache-oriented prompt segmentation. The HTTP adapter does not know those details. [`server.py:530`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:530) [`server.py:565`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:565) | A prepared-prompt value is a sound seam, but it should be richer than token ids. It must carry the generation inputs and owned media or grammar state needed by either lane. Protocol adapters should not assemble `RequestShape` or transfer resources piecemeal. |
+| Semantic events | The state machine classifies tokens as `normal`, `reasoning`, or `tool`. `_process_control_tokens` withholds matched control sequences. The HTTP handler then accumulates reasoning, tool text, visible text, and parses completed tool calls. [`server.py:236`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:236) [`server.py:654`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:654) [`server.py:1442`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1442) | Do not copy this split. mlx-bun's `CompletionSink` already emits content, reasoning, and tool-call events through one interface. [`completion-sink.ts:7`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/serve/completion-sink.ts:7) [`completion-sink.ts:90`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/serve/completion-sink.ts:90) The deepened module should own that semantic work so all protocol adapters consume the same events. |
+| Terminal summary | `generate.stream_generate()` puts timing, token counts, memory, and finish reason on every low-level response, but the server discards most of it. The HTTP handler counts emitted tokens, reads prompt and cache counts from `GenerationContext`, derives tool-call finish, and builds usage. [`generate.py:269`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/generate.py:269) [`generate.py:728`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/generate.py:728) [`server.py:1495`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1495) [`server.py:1517`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1517) | The oracle has no single terminal summary at the server seam. mlx-bun should add one. It should be the sole source of finish reason, prompt/generated/cached counts, final lane, speculation facts, and collected result. |
+| Streaming and non-streaming | Both modes consume the same response iterator and semantic accumulation loop. Streaming flushes deltas and optionally emits a final usage packet; non-streaming sends one accumulated response. [`server.py:1454`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1454) [`server.py:1507`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1507) [`server.py:1529`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1529) | Keep one Completion execution interface. Streaming and non-streaming are protocol adapters over the same semantic events and terminal summary, not separate execution paths. |
+| Scheduling placement | Model loading marks a model batchable only when no draft model exists and every cache supports merge. Per-request placement also excludes a fixed seed. The worker joins the active batch, creates a batch, runs serially, or drains an incompatible batch. [`server.py:370`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:370) [`server.py:685`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:685) [`server.py:731`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:731) [`server.py:801`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:801) [`server.py:831`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:831) | `RequestShape` describes settings already resolved from flags, request fields, model metadata, and defaults. `place()` asks only whether the continuous scheduler implements that exact composition; it never replaces MTP, a KV scheme, or another requested method. The frozen declaration names the `serial` or `continuous` mechanism, and both reporting and `run()` consume it. Active-row count selects B=1 or B=N inside the continuous mechanism. |
+| Cancellation and flow | Cancellation is cooperative. The handler's `finally` sets a flag; serial execution checks it after tokens, while batched execution checks it during prompt and generation work and removes the row. [`server.py:1008`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1008) [`server.py:861`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:861) [`server.py:911`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:911) [`server.py:1551`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1551) The response queue is unbounded, and socket writes block only the HTTP thread, so there is no end-to-end demand signal from the client to generation. [`server.py:444`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:444) [`server.py:1032`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1032) [`server.py:1489`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1489) | Do not regress to the oracle here. mlx-bun's token callback accepts a promise or `false`, and `GenerationGateway.run()` accepts an `AbortSignal`. [`generation-gateway.ts:94`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/serve/generation-gateway.ts:94) [`generation-gateway.ts:354`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/serve/generation-gateway.ts:354) Cancellation and async flow control are part of the interface, not HTTP details. |
+| Cache and resource lifetime | The worker fetches or creates the serial cache, generates, then inserts it. Batched execution fetches before admission, snapshots segment caches, inserts the final cache, and removes cancelled rows. [`server.py:963`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:963) [`server.py:1018`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1018) [`server.py:752`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:752) [`server.py:864`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:864) [`server.py:901`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:901) `LRUPromptCache` hides prefix matching, copying, trimming, and count/byte eviction behind a small interface. [`cache.py:1623`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/models/cache.py:1623) [`cache.py:1674`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/models/cache.py:1674) [`cache.py:1696`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/models/cache.py:1696) | `prepareCompletion()` now disposes owned media and grammar state on admission rejection. `execute()` disposes the same plan on placement or pre-generation failure, then transfers ownership exactly once before `GenerationGateway.run()`. Cache and in-generation disposal remain in the gateway and generation path. [`completion-executor.ts:75`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/serve/completion-executor.ts:75) [`completion-executor.ts:219`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/serve/completion-executor.ts:219) |
+| Errors | Errors before the response iterator exists become a JSON error, although the oracle uses status 404 for that whole class. Errors delivered later through the worker queue are raised by the iterator. The handler guarantees `ctx.stop()` but has no matching mid-stream error packet in this loop. [`server.py:1417`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1417) [`server.py:1424`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1424) [`server.py:1035`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1035) [`server.py:1454`](/Users/joshrossi/Code/mlx-lm/.venv/lib/python3.14/site-packages/mlx_lm/server.py:1454) | Preserve a typed distinction between rejection before execution and failure after events begin. Both must run the same ownership cleanup. Protocol adapters decide how to encode the failure after headers have been sent. |
+
+## Strongest conclusions
+
+1. **Keep resolved inference settings authoritative.** Serial and continuous execution are scheduling mechanisms for the already-selected composition. Consolidating placement may shrink the HTTP adapter, but it must not turn explicit features or opinionated defaults into inferred choices.
+2. **Do not borrow the oracle's semantic split.** Its worker emits state-tagged token records, then the HTTP handler performs semantic accumulation and tool parsing. mlx-bun already has a better starting point in `CompletionSink`. Move that module behind the Completion execution seam so protocol adapters see semantic events only.
+3. **Keep the terminal summary as the only completion result.** It now settles finish reason, usage, cached-token count, final lane, speculation, logprobs, and collected output once. The four HTTP branches only format that summary. [`completion-executor.ts:103`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/serve/completion-executor.ts:103) [`server.ts:3104`](/Users/joshrossi/.codex/worktrees/e408/mlx-bun/src/server.ts:3104)
+4. **Treat cancellation, async flow control, and ownership as interface requirements.** The oracle's queue makes cancellation and socket pressure indirect. mlx-bun has stronger mechanisms already. The deepening should hide their implementation without weakening their behavior.
+
+The deletion test is positive. If this Completion execution module disappeared, scheduling policy, cache participation, semantic parsing, finish rules, usage, and cleanup would spread back across four protocol branches. Concentrating those rules gives callers leverage and gives maintainers locality. Feature selection and fidelity policy remain outside this seam as resolved inputs.

package/docs/design/orpo-dynamic-lambda.md

@@ -158,7 +158,7 @@ than the absolute max-bearable (the edge is fragile).
 **Synergy with the diffusion frontier:** all of this is noise-robustness machinery for
 controlling off a noisy process variable — so it transfers *especially* well to a
 diffusion LM, where the likelihood is a stochastic ELBO (even noisier). See
-[orpo-base-uf-experiment-and-directions.md](../investigations/orpo-base-uf-experiment-and-directions.md) §6.7.
+[orpo-base-uf-experiment-and-directions.md](../archive/investigations/orpo-base-uf-experiment-and-directions.md) §6.7.
 
 ## Wiring
 

package/docs/design/orpo-future-enhancements.md

@@ -11,7 +11,7 @@ comfortable.
 > trainer)** shipped as `orpo_prefix_shared`, composed with the flash head AND the
 > segmented backward for MiniCPM5 **and** e4b. See [training.md](../reference/training.md),
 > [orpo-quickstart.md](../reference/orpo-quickstart.md), and the live record
-> [steel-flash-cce-handoff.md](../investigations/steel-flash-cce-handoff.md). The
+> [steel-flash-cce-handoff.md](../archive/investigations/steel-flash-cce-handoff.md). The
 > remaining sections (§2, §4–§9, the objective variants) are still genuinely future.
 
 ## Deferred levers — queued, do NOT start mid-run (2026-06-19)

package/docs/design/orpo-training.md

@@ -19,7 +19,7 @@ runs half the forwards per step.
 > The ✅/🔬 entries + optimization log below are the **design/build history**; for the
 > current usage, flags, and numbers see [training.md](../reference/training.md) +
 > [orpo-quickstart.md](../reference/orpo-quickstart.md), and the live engineering record
-> [steel-flash-cce-handoff.md](../investigations/steel-flash-cce-handoff.md).
+> [steel-flash-cce-handoff.md](../archive/investigations/steel-flash-cce-handoff.md).
 
 > Oracle note: neither `mlx_lm` nor `optiq` ships ORPO, so — unlike DPO
 > ([`loss.ts`](../../src/train/loss.ts) ported `optiq/lora/dpo.py`) — ORPO is
@@ -38,7 +38,7 @@ runs half the forwards per step.
 > hiddens before judging** (it's plausibly a free speedup to enable). Standing Lab
 > gates (filter-on-real-data, teacher-forced grad fidelity, end-to-end quality)
 > + the host-buffer pin-leak post-mortem:
-> [orpo-flash-cce-pin-leak.md](../investigations/orpo-flash-cce-pin-leak.md).
+> [orpo-flash-cce-pin-leak.md](../archive/investigations/orpo-flash-cce-pin-leak.md).
 
 ## Implementation status
 

package/docs/design/parity-tier-dag.md

@@ -26,7 +26,7 @@ exists AND it matches bit-exactly; otherwise it bubbles up. Note the subtlety th
 map already shows: an *optimization* that still matches the oracle stays low —
 `compiled-decode` is ours but is L1 because it replays the same ops bit-for-bit.
 Only nodes with **no** oracle (flash-CCE) are *forced* into the Lab. (Background +
-post-mortem: [orpo-flash-cce-pin-leak.md](../investigations/orpo-flash-cce-pin-leak.md).)
+post-mortem: [orpo-flash-cce-pin-leak.md](../archive/investigations/orpo-flash-cce-pin-leak.md).)
 
 ## Two axes: parity gates performance