mlx-bun 0.4.0 → 0.5.0

package/docs/design/decode-speed-program.md

@@ -98,15 +98,15 @@ What remains is drafters worth running, in this order:
   exist in `src/`. The measured motivation stands: acceptance dropped 51%→29%
   under grammar with a blind drafter (our own matrix). (iii) grammar-pruned
   draft TREES (XGrammar-2's `traverse_draft_tree`) — idea only.
-- **Per-slot drafting under batching** — the composition-matrix row in the
-  architecture doc; actual speculation still uses the serial verifier, while
-  supported ordinary requests can batch even with a draft mounted.
+- **Drafting under batching** — grouped provider execution is implemented at
+  B1/B>1, with method-owned candidate generation, verification and companion
+  state. Remaining compositions and evidence live in [batching](batching.md#consolidation-matrix).
 
 ### Lever 2 — native MLX updates against the pinned control
 
-We are pinned to the oracle stack in PLAN.md "Reference environment" (mlx
-0.31.2). The July audit below identified upstream changes absent from that
-pin. Recheck current releases and the native libraries actually loaded before
+The runtime and oracle now use MLX 0.32.2 / native pack 0.4.0; see §7.12 and
+[environment](../reference/environment.md). The historical July audit below
+identified upstream changes absent from its then-current MLX 0.31.2 pin. Recheck current releases and the native libraries actually loaded before
 an upgrade experiment. A new mlx/mlx-c build also needs matching oracles;
 retain the pinned installation and test the new stack in isolation.
 Upstream refs are full URLs on purpose (a bare `#NNNN` here resolves to our
@@ -652,7 +652,7 @@ ledger. A candidate's priority changes when the measured graph changes.
 | R13 | Attention and KV | Stock SDPA dispatch, tiling, grouped-query reuse, fused KV append/quantize, direct attention over compressed KV, mask specialization, RoPE fusion and paged versus contiguous storage. Measure quality, TTFT and decode by depth. |
 | R14 | Vocabulary head and sampler | Last-position-only projection, exact fused matvec/argmax for eligible greedy requests, GPU logits processors, reduced readbacks and structured-output singleton runs. Head elimination must respect logprobs and sampling contracts. |
 | R15 | Deterministic no-verification replay | Follow section 7.5. Improve proven coverage and span execution economics; report wall time and output identity independently. This is a first-class track ahead of speculative drafter training. |
-| R16 | MLX graph and loop | Use model-owned graph/method bindings for compiled regions, stable buffers/shapes, graph replay, cache donation, allocation removal, bounded device-side chaining and host/GPU overlap. Measure session/lease/readback costs without adding per-op evals. Preserve cancellation and ownership gates. |
+| R16 | MLX graph and loop | Use model-owned graph/method bindings for compiled regions, stable buffers/shapes, graph replay, cache donation, allocation removal, bounded device-side chaining and host/GPU overlap. Measure session/lease/readback costs without adding per-op evals. Preserve cancellation and ownership gates. | |  |  | Early-output native, HTTP, cancellation/overlap and cached/SSD acceptance is complete; see benchmarks.md for monitored repeats and activity limits. The MLX 0.32.2 submission screen is complete: retain upstream startup defaults, with the measured M4 Llama override available explicitly. The current native API has no model-scoped setter; revisit integration if that interface becomes available. See benchmarks.md for the cross-model results and activity records.
 | R17 | Hybrid prefix and recurrent state reuse | Exact prefixes, semantic anchors, page/block storage, compact active-state updates and checkpoint cadence. Test mixed prompt edits and cancellation; no KV-only hit that loses the corresponding recurrent state. |
 | R18 | Scheduler | Solo latency, actual B-wide prefill, continuous batching, chunked-prefill/decode interleaving and shape buckets. Report per-request latency and aggregate throughput; one cannot substitute for the other. |
 | R19 | Weight-quality allocation | Measured latency-aware allocation across MLP, attention, GDN, embedding and head; GPTQ/LDLQ calibration and damping; QTIP/YAQA rounding; group size, k, L, block length, rotations and sensitive-layer exceptions. Optimize quality/bytes/time together. |
@@ -1019,9 +1019,9 @@ ranges across layers. Keep exact range/mask values, outstanding-view ownership
 and the memory bound as gates; measure complete prefill and verification rounds
 before retaining an allocation or dispatch optimization.
 
-The pinned MLX 0.32.2 source also gives a specific native scheduling experiment
+The pinned MLX 0.32.2 source supplied a native scheduling experiment
 for the three-query KV4 verification window. Qwen's 24 query heads share four
-KV heads (GQA6), with D256. `quantizedSdpaUnfused` currently presents the heads
+KV heads (GQA6), with D256. `quantizedSdpaUnfused` originally presented the heads
 as separate broadcast batches. In `mlx/backend/metal/quantized.cpp`, the
 M4-eligible affine key multiplication selects `qmv_wide`: three vectors per
 threadgroup, eight reduction lanes, and a separate batch for each query head.
@@ -1050,6 +1050,15 @@ causal string currently rebuilds the boolean array in each unfused attention
 call. Keep the bf16 and OptiQ dispatch policies intact and measure the retained
 mask's lifetime as well as complete-forward timing before retaining that change.
 
+The screen and full-model/HTTP comparisons are now complete. Three-head key
+grouping is selected on M4 Pro for the tested B1/B2, L3, GQA6/D256,
+KV4/group64, bf16/f32, N≥8192 geometry. Other geometry retains the existing
+native selection. M1 grouped key scores differ, so its key geometry remains
+unchanged. The permanent regression compares strided KV against the separate
+head composition. Exactness, complete-forward and serving measurements are
+recorded in benchmarks.md under Three-query affine attention head grouping.
+The shared causal-mask experiment remains separate, unmeasured work.
+
 Do not apply that fold indiscriminately to the value multiplication. Its native
 matrix threshold is four query rows, so the same reshape would change the
 algorithm. At long context it uses `qvm_split_k`, with 32 partitions beyond
@@ -1133,13 +1142,17 @@ SSD boundary-before-transfer fix, Qwen tool-parameter parser,
 
 ### 7.9 Project kernel coverage
 
-The current AST census has 37 custom Metal kernel construction sites in 15
-files; frozen source controls retain the two old-core affine kernels the
+The current AST census has 44 custom Metal kernel construction sites in 18
+files, plus 12 direct compiled-function construction sites. The seven additions
+since the earlier census are three paged-attention kernels, two normalized
+greedy reductions, masked recurrent update and row-length-aware state replay.
+Frozen source controls retain the two old-core affine kernels the
 MLX 0.32.2 consolidation removed (§7.12). The initial inventory also counted
 355 calls into 162 distinct MLX C APIs; that count includes allocation,
 configuration and graph operations, not native GPU kernels. Runtime
 specializations still need shape, dtype, layout and stream coverage. Evidence:
-`reports/qwen38-rd/project-kernel-consolidated-census.json` (current), with
+`reports/prefill-observation/kernel-census.json` (current), with
+`reports/qwen38-rd/project-kernel-consolidated-census.json` and
 the earlier censuses in `project-kernel-inventory.json`,
 `project-kernel-coverage.json`, `project-kernel-vector-expand-addendum.json`,
 `project-kernel-turboquant-addendum.json`, `project-kernel-inverse-addendum.json`
@@ -1148,17 +1161,19 @@ and `project-kernel-wide-addendum.json`.
 | Owned kernel family | Sites | Current evidence and next coverage |
 |---|---:|---|
 | Packed grammar mask | 1 | Exact micro and HTTP outputs; serving timing inconclusive under external load. |
+| Normalized greedy partial and final reductions | 2 | Same-machine operator and actual-model comparisons pass on both Macs; MiniCPM standard AB/BA improves throughput while packed Qwen MTP3/TQ is flat. Metadata, stochastic and custom samplers retain their prior path. |
+| Direct paged-attention bf16, affine score and affine value | 3 | C5 operation/model/state and native HTTP checks pass. Recorded wins and regressions do not justify a default; block formats and attention remain optional through the cache/model interfaces. |
 | Joint TurboQuant K/V decoding and selected inverse rotation | 2 | Experimental opt-in; exact operation/lifetime and R6/RTN4 integrated model gates pass on MLX 0.32.2. Integrated inverse-rotation HTTP passes six pairs per quant with decode gains in every pair; R6 complete time improves and RTN4 complete time is inconclusive. Six deferred-consumer pairs per MiniCPM/Gemma also improve complete requests on consolidated source. Combined/pressure and quiet acceptance remain. Direct packed-key attention is exact but slower. Details live in turboquant.md. |
 | Flash attention forward, D, dKV and dQ | 4 | D256 alternative screen fails one KL gate; other attention shapes and backward costs remain. |
 | GLM sparse-attention rank and contract keys | 2 | Both keys now use uint32 with stable score sorting. Operation and complete tiny-model state gates pass; six balanced pairs per shape improve selection latency. Real Colibri serving remains unmeasured. |
 | GLM q4/q8 streamed expert gate/up and down | 4 | Synthetic resident-slot geometry screen and six balanced repeats preserve intermediate/output bytes and cleanup. BF16 timing is bimodal across identical process configurations; no geometry default retained. Attribute that variation before further timing claims. Real model and I/O overlap remain unmeasured. |
 | Qwen causal convolution, SiLU and copied tail | 1 | Exact operation/model/state gates; fusion has no compelling full-model speed win. |
-| Gated recurrent update | 1 | Unrolling, geometry, compilation and normalization fusion screened; full-model gains too small to retain. |
+| Gated recurrent update, masked update and prefix-state replay | 3 | Unrolling, geometry, compilation and normalization fusion screened; full-model gains too small to retain. Masked padding and row-length replay preserve inactive state and accepted-prefix state in composed native/serving gates. Uniform state-only replay did not establish a serving win and remains unselected. |
 | Qwen multimodal rotary embedding | 1 | Direct-stride fusion preserves operation/model state, but the complete-forward gain is too small to retain. Actual image/video HTTP coverage remains. |
 | Trellis reduce, scatter, expand and gate/up variants | 9 | Shared rows and fusion improve measured cells. Device codebook lookup loses. A scoped threadgroup integer table improves repeated M3/4 scatter, full forwards and MTP HTTP requests with exact outputs; M1 loses and stays procedural. Scheduling and prefill controls remain. Broader bit/axis/shape/model coverage remains. |
 | Trellis split-K projection and accumulation | 2 | Short-prefill model/serving acceptance; larger shapes and additional artifacts remain. |
 | Trellis tiled projection | 1 | Short axis-1 tiles accepted diagnostically; larger tiles, context and memory pressure remain. |
-| Trellis native-wide prefill | 1 | Reached only under `MLX_BUN_TRELLIS_VARIANT=11..13` (default 6 expands then uses native matmul). Matches MLX 0.32.2 small-prefill arithmetic on eligible GPUs. Operation, integrated R6 state/generation and six HTTP pairs per lane pass. Short-prefill TTFT improves; broader context and quiet acceptance remain. |
+| Trellis native-wide prefill | 1 | Reached under `MLX_BUN_TRELLIS_VARIANT=11..13`, including the current variant-13 default; variant 6 expands then uses native matmul. Matches MLX 0.32.2 small-prefill arithmetic on eligible GPUs. Operation, integrated R6 state/generation and six HTTP pairs per lane pass. Short-prefill TTFT improves; broader context and quiet acceptance remain. |
 | Trellis vectorized expansion | 1 | Exact operation, full-model prefill and HTTP responses. Shorter serial and default-scheduler HTTP prefills improve; existing-suite timing mostly flat. Broader context, pressure and quiet-machine gates remain. |
 | Cross-entropy forward/backward variants | 7 | All seven routes pass the vocabulary-partition screen, including 54 forced-fallback cases. The isolated fast-dispatch opportunity does not apply to the available models wired into this loss path; broader real-loss profiling remains. |
 
@@ -1597,14 +1612,30 @@ numbers are historical controls until the workload and version metadata match.
 
 ### 7.10 Final benchmark and HTML comparison report
 
+Request tracing separates the gateway’s request-slot wait (`engine.request_wait`)
+from execution-lock or row-admission wait (`engine.admission_wait`). Cache
+prefetch/restore preparation has its own `cache.prefetch` span before execution;
+later cache selection remains `cache.lookup_restore`. The prompt-response
+report sums the two queue spans and the two cache spans in their respective
+columns. Historical records without the new spans omit that work and must not
+be interpreted as complete queue attribution. The shared serving HTTP reader
+accepts a trace ID to correlate server phases with each scheduled request.
+
 The offline renderer and replay fixtures are implemented. It imports the
-existing native/serve formats, retains failures and provenance, and suppresses
+existing native/serve formats and saved Pi task results, retains failures and provenance, and suppresses
 speed ratios for incompatible work or unqualified runs. Its current metrics
 and limitations live in benchmarks.md. A bounded MiniCPM M1 diagnostic passes
 the real measurement path with the explicit reference control; concurrent
 output differences remain visible. Fixed-source block log-ratio intervals now
 use a deterministic bootstrap across distinct workload seeds, with duplicate
-seeds excluded and at least five blocks required. Quality inputs, the frozen
+seeds excluded and at least five blocks required. Task imports load adjacent
+`quality.json` checks with file hashes, report failures and blocked categories,
+and retain source/settings, per-turn request hashes, timing, usage and final
+SSD flush records. Task rows do not enter engine-speed pairing. The read-only
+quality-ledger exporter supplies portable model accuracy/KL/size records,
+with source rows, configuration and machine provenance retained. Legacy
+dataset/artifact identity limitations remain explicit; importing history
+does not establish current-artifact quality acceptance. The frozen
 final protocol and quiet final matrix remain acceptance work;
 rendering saved evidence does not close this gate.
 
@@ -2137,8 +2168,9 @@ six balanced pairs with exact outputs. The original rows below describe
 loaded-machine diagnostics. The closeout serial v6/v13 suite now passes six
 balanced pairs under the recorded audio-CPU allowance, with every paired
 request matching. Final-source native state and bf16/KV4 MTP-prefix gates
-also pass. Continuous and combined pressure acceptance remain; no default
-has changed. New evidence is in `reports/qwen38-closeout/` and
+also pass. Later acceptance promoted variant 13 and shared batching in v0.4.0; early
+first output is now also selected. Remaining combinations and measurement
+limits are identified per row. New evidence is in `reports/qwen38-closeout/` and
 [benchmarks.md](../reference/benchmarks.md). Historical evidence paths
 are relative to `reports/qwen38-rd/` on the measuring host; the narrative
 behind each row is the archived log
@@ -2150,30 +2182,30 @@ behind each row is the archived log
 | ID | Experiment | Outcome | Retained flags / defaults | Evidence files | Next discriminating step or missing resource | Reopen condition |
 |---|---|---|---|---|---|---|
 | R0 | Exact artifact and execution controls | **Inconclusive.** Isolated unmodified `6d45ca1`/`673b43f` checkouts: six native AB/BA pairs per flagship/compact/RTN4 match every ID at 6/128/512 prompt tokens with flat TTFT/complete/memory; six serial and six default-scheduler HTTP pairs preserve every response with flat or slightly lower complete time. Updated Bun vs MLX-LM native controls (six pairs per affine artifact, 64 tokens) emit identical IDs, complete time close, small median Bun advantage. Graph exports (v6/v8 at M=1/4/5), dispatch inventory, per-operation counter calibration and a schedule recorder exist; dispatch-level GPU sampling is unsupported on the M4 Pro, so per-kernel attribution stays open. The M4 DeltaNet golden is resolved (stale fixture; Bun and Python agree). The quiet serial/default/mlx-lm HTTP baseline never ran; the packed target has no stock mlx-lm arm. | None (controls only). | `refactor-fixed-kernel-audit-plan.json`, `refactor-native-paired{,-review}.json`, `refactor-http-paired-{serial,default}-review.json`, `native-affine-paired{,-review}.json`, `native-affine-paired/`, `native-memory-ledger.json`, `native-memory-chunk512-screen/`, `graph-captures/`, `variant8-attached.trace`, `attached-trace-summary.json`, `labelled-v8-analysis.json`, `labelled-v8-kernels.txt`, `variant8-labelled.trace`, `metal-counter-capabilities.json`, `mlx-counter-source-manifest.json`, `mlx-counter-build.patch`, `metal-counter-pooled-unit.stderr.txt`, `metal-counter-calibration-review.json`, `mlx-schedule-source-manifest.json`, `mlx-schedule-smoke-v2.log`, `top-p-schedule-screen{,-review}.json`, `empty-submit-r6-steady-trace{,-review}.json`, `mlx-empty-submit-steady-source-manifest.json` | Exclusive M4 time is now available. Finish the final-source serving/reference controls: `bun scripts/bench-serve.ts all --model-path ~/models/Qwen3.8-27B-q3-trellis-ldlq-k300-packed-interleave2-rd --arms mlx-bun-serial --context 4096 --tokens 192 --workload-seed <block>` alternated with the default arm across blocks, plus the RTN4 affine artifact with `--arms mlx-bun-serial,mlx-lm` (the only same-artifact oracle cell); quiet preflight passing; distinct `--out` per block. | Any change to the controlled path: native pack bump, artifact layout change, new kernel default. |
-| R1 | Packed prefill tile decode plus matmul | **Repeatable win (opt-in).** Variant 11 (tiled axis-1 gate/up, M=5..32) and variant 12 (split-K axis-0 down, M=5..8) reconstruct bf16 weights inside tiles with exact full-model logits/state/continuation; six native paired blocks improve eligible M; six raw-completion HTTP pairs per lane improve TTFT and complete time (six-token prompt, 8/64 output tokens); the second tile pass improves eight-request throughput and B=1 TTFT in six pairs. All 40 larger tile geometries at M=128/512 are exact but none beats expansion plus native matmul in six AB/BA pairs. On MLX 0.32.2/M3+ the M5..15 axis-1 path uses the wide-prefill kernel (native `gemv_wide` arithmetic); six HTTP pairs per lane improve TTFT in every pair, decode and RSS unchanged. | `MLX_BUN_TRELLIS_VARIANT=11..13`; default 6 untouched. | `variant11-full-model-review.json`, `trellis-v11-http{,-review}.json`, `variant12-full-model.json`, `trellis-splitk-{screen,tuning,full-model}-review.json`, `trellis-v12-combined-http{,-review}.json`, `trellis-v12-combined-native-review.json`, `trellis-v12-native-failure-replay.json`, `trellis-tile-layout-screen-review.json`, `trellis-tile-full-model-review.json`, `trellis-splitk-bm8-{other-bits,full-model}-review.json`, `trellis-tile-integrated-forward-review.json`, `trellis-tile-http-review.json`, `trellis-tile-seeded-serial-http-review.json`, `trellis-tile-native{,-review}.json`, `trellis-large-prefill-screen{,-review}.json`, `mlx-upgrade-0.32.2/trellis-wide-{screen,perf,full,integrated-full}`, `mlx-upgrade-0.32.2/trellis-wide-http{,-review}.json`, `mlx-upgrade-0.32.2/trellis-wide-continuous.json` | Quiet-machine six-pair `bench-serve all` serial, `MLX_BUN_TRELLIS_VARIANT=6` vs `13` on the packed artifact at 1k and 4k prompts, reporting cold TTFT and MLX peak allocation (the row's declared gate). | A tile geometry beating native matmul at M≥128 on real k3 matrices, or a GPU generation that changes native small-M dispatch. |
-| R2 | Bounded expansion scheduling | **Repeatable win (opt-in).** Unrestricted deferral (variant 9) and unrestricted async submission gain on short prompts but fail the saved-agent pressure gate (Metal insufficient memory at the 12,954-token request). The bounded policy (async only while MLX active allocation is below 75% of the recommended working set) passes serial and continuous pressure gates; integrated six HTTP pairs per lane: TTFT -4.00%/-3.96% at 128 prompt tokens (six wins), -1.43%/-1.40% at 512; wall -0.63..-0.88%; decode flat; native M16/128/512 forwards -2.09/-6.19/-2.80% with about +368/403 MB peak allocation. | `MLX_BUN_TRELLIS_ASYNC_EXPAND=1`, variant 13 only, default off. | `variant9-full-model.json`, `trellis-v13-boundary-full-model{,-review}.json`, `trellis-v13-boundary-http-serial{,-review}.json`, `qwen-long-boundary-serial{,-review}.json`, `qwen-long-async-serial{,-review}.json`, `trellis-v13-bounded-full-model{,-review}.json`, `qwen-long-bounded-{serial,default}{,-review}.json`, `trellis-v13-bounded-http-{serial,default}{,-review}.json`, `trellis-async-integrated-{typecheck,hygiene,model-free}.txt`, `trellis-async-production-full-model{,-review}.json`, `trellis-async-production-http-{serial,default}{,-review}.json`, `trellis-async-production-long-{serial,default}{,-review}.json` | Combined arm (variant 13 + async expand + fused TurboQuant KV + MTP) through the saved seven-request agent-boundary fixture on a quiet machine; the 75% threshold has one pressure pair per lane. | A change to the memory headroom guard or to per-layer cache evaluation boundaries (R16/R17). |
-| R3 | Axis-0 scatter layout and split-K | **Repeatable win (opt-in).** Variant 8 (balanced word-aligned 3-bit scatter, all 32 lanes; k3/T256/L≤12) and variant 10 (scatter decode shared across M=2..4) pass exact full-model gates and improve M=1/2/4 and M=2..4 in six native blocks; row-loop unrolling not retained. A threadgroup integer codebook restricted to the M3/4 k3 interleaved down projection: six pairs -1.44/-2.00% forwards, six MTP HTTP pairs -1.07/-1.27/-1.19% complete time (5/6, 5/6, 4/6 wins), six four-request cohorts -1.49% (4/6); M1 loses and stays procedural; folded into variant 13. A 2.59% M128 control slowdown in one sequence was not reproduced by two controls. Variant 6 unchanged. | Inside `MLX_BUN_TRELLIS_VARIANT=13`; default 6 untouched. | `balanced3-screen/`, `scatter-unroll-screen/`, `aligned3-screen/`, `variant8-paired/`, `variant8-full-model.json`, `shared-scatter-screen/`, `shared-scatter-all-screen/`, `variant10-full-model.json`, `trellis-threadgroup-codebook.json`, `trellis-threadgroup-scatter{,-m3}{,-review}.json`, `trellis-scatter-full{,-repeat}{,-review}.json`, `trellis-scatter-http{,-repeat}{,-review}.json`, `trellis-scatter-prefill{,-review}.json`, `trellis-scatter-sequence{,-review}.json`, `trellis-scatter-http-trace{,-review}.json`, `trellis-scatter-cohort-v2{,-review}.json`, `trellis-scatter-integration-source-change.json`, `trellis-scatter-integration-focused-tests-final.txt`, `trellis-scatter-integration-typecheck.txt`, `trellis-scatter-integrated-full{,-review}.json`, `trellis-scatter-integrated-http{,-review}.json`, `trellis-scatter-integrated-cohort{,-v2}{,-review}.json` | Quiet-machine six-pair v6 vs v13 single-request decode via `bench-serve all --arms mlx-bun-serial` on the packed artifact, to settle the M128 prefill control drift and M1 decode neutrality. | k2/k4 down projections (only k3 interleaved is qualified) or a new bit allocation from R19. |
-| R4 | Axis-1 reduce and gate/up fusion | **Repeatable win (opt-in).** Variant 7 shares each decoded gate/up weight across M=2..4 with fused gate/up, keeping the variant-6 accumulator order; exact matrix/MLP/full-model gates at M=1..5 and six AB/BA blocks retain the complete-MLP gain; the last-position append sweep confirms the M=4/5 cost discontinuity. Sharing activation loads across 2/4/8 output rows per SIMD group loses at k3 M=1; smaller threadgroups do not rescue it. The M=1 decode kernel is unchanged; the 64/128/256-thread and accumulator sweep with spill/occupancy inspection never ran because per-dispatch counters are unavailable. | Variants 7+ (`MLX_BUN_TRELLIS_VARIANT`); default 6 untouched. | `paired-shared-m/`, `variant7/`, `shared-m-full-model.json`, `shared-m-append-lengths.json`, `trellis-shared-rows-k3-r*-m1.json`, `trellis-shared-rows-k3-r*-s*-m1.json` | Missing resource: per-dispatch GPU counters or an Instruments shader profile (unsupported on the M4 Pro) to drive the M=1 thread/rows/accumulator sweep. | Kernel-level GPU timing becomes available, or a k mixture changes gate/up bit widths. |
+| R1 | Packed prefill tile decode plus matmul | **Adopted in the v0.4.0 default.** Variant 11 (tiled axis-1 gate/up, M=5..32) and variant 12 (split-K axis-0 down, M=5..8) reconstruct bf16 weights inside tiles with exact full-model logits/state/continuation; six native paired blocks improve eligible M; six raw-completion HTTP pairs per lane improve TTFT and complete time (six-token prompt, 8/64 output tokens); the second tile pass improves eight-request throughput and B=1 TTFT in six pairs. All 40 larger tile geometries at M=128/512 are exact but none beats expansion plus native matmul in six AB/BA pairs. On MLX 0.32.2/M3+ the M5..15 axis-1 path uses the wide-prefill kernel (native `gemv_wide` arithmetic); six HTTP pairs per lane improve TTFT in every pair, decode and RSS unchanged. | Trellis variant 13 is the shipped default; older variants remain explicit controls. | `variant11-full-model-review.json`, `trellis-v11-http{,-review}.json`, `variant12-full-model.json`, `trellis-splitk-{screen,tuning,full-model}-review.json`, `trellis-v12-combined-http{,-review}.json`, `trellis-v12-combined-native-review.json`, `trellis-v12-native-failure-replay.json`, `trellis-tile-layout-screen-review.json`, `trellis-tile-full-model-review.json`, `trellis-splitk-bm8-{other-bits,full-model}-review.json`, `trellis-tile-integrated-forward-review.json`, `trellis-tile-http-review.json`, `trellis-tile-seeded-serial-http-review.json`, `trellis-tile-native{,-review}.json`, `trellis-large-prefill-screen{,-review}.json`, `mlx-upgrade-0.32.2/trellis-wide-{screen,perf,full,integrated-full}`, `mlx-upgrade-0.32.2/trellis-wide-http{,-review}.json`, `mlx-upgrade-0.32.2/trellis-wide-continuous.json` | Existing six-pair serial/default/combined comparisons and final-source native gates are accepted. Reuse them; broader pressure and final report work remain separately tracked in Phase 6. | A tile geometry beating native matmul at M≥128 on real k3 matrices, or a GPU generation that changes native small-M dispatch. |
+| R2 | Bounded expansion scheduling | **Repeatable win (opt-in).** Unrestricted deferral (variant 9) and unrestricted async submission gain on short prompts but fail the saved-agent pressure gate (Metal insufficient memory at the 12,954-token request). The bounded policy (async only while MLX active allocation is below 75% of the recommended working set) passes serial and continuous pressure gates; integrated six HTTP pairs per lane: TTFT -4.00%/-3.96% at 128 prompt tokens (six wins), -1.43%/-1.40% at 512; wall -0.63..-0.88%; decode flat; native M16/128/512 forwards -2.09/-6.19/-2.80% with about +368/403 MB peak allocation. | `MLX_BUN_TRELLIS_ASYNC_EXPAND=1`, variant 13 only, default off. | `variant9-full-model.json`, `trellis-v13-boundary-full-model{,-review}.json`, `trellis-v13-boundary-http-serial{,-review}.json`, `qwen-long-boundary-serial{,-review}.json`, `qwen-long-async-serial{,-review}.json`, `trellis-v13-bounded-full-model{,-review}.json`, `qwen-long-bounded-{serial,default}{,-review}.json`, `trellis-v13-bounded-http-{serial,default}{,-review}.json`, `trellis-async-integrated-{typecheck,hygiene,model-free}.txt`, `trellis-async-production-full-model{,-review}.json`, `trellis-async-production-http-{serial,default}{,-review}.json`, `trellis-async-production-long-{serial,default}{,-review}.json` | The combined variant-13/fused-k8v3/MTP3 seven-request pressure gate passes on M4 after the existing M1 gate: all responses and complete usage match across roomy sync, constrained sync and constrained async arms; both threshold branches execute and persistence completes. See benchmarks.md and `reports/prefill-observation/combined-tq-mtp-pressure{,-review}.json`. Retain opt-in selection: the measured complete-request gain is modest and expansion overlap increases live allocation. Reopen for a changed model/barrier or a materially better timing/memory tradeoff; do not repeat the unchanged pressure gate. | A change to the memory headroom guard or to per-layer cache evaluation boundaries (R16/R17). |
+| R3 | Axis-0 scatter layout and split-K | **Adopted in the v0.4.0 default.** Variant 8 (balanced word-aligned 3-bit scatter, all 32 lanes; k3/T256/L≤12) and variant 10 (scatter decode shared across M=2..4) pass exact full-model gates and improve M=1/2/4 and M=2..4 in six native blocks; row-loop unrolling not retained. A threadgroup integer codebook restricted to the M3/4 k3 interleaved down projection: six pairs -1.44/-2.00% forwards, six MTP HTTP pairs -1.07/-1.27/-1.19% complete time (5/6, 5/6, 4/6 wins), six four-request cohorts -1.49% (4/6); M1 loses and stays procedural; folded into variant 13. A 2.59% M128 control slowdown in one sequence was not reproduced by two controls. Variant 6 unchanged. | Trellis variant 13 is the shipped default; older variants remain explicit controls. | `balanced3-screen/`, `scatter-unroll-screen/`, `aligned3-screen/`, `variant8-paired/`, `variant8-full-model.json`, `shared-scatter-screen/`, `shared-scatter-all-screen/`, `variant10-full-model.json`, `trellis-threadgroup-codebook.json`, `trellis-threadgroup-scatter{,-m3}{,-review}.json`, `trellis-scatter-full{,-repeat}{,-review}.json`, `trellis-scatter-http{,-repeat}{,-review}.json`, `trellis-scatter-prefill{,-review}.json`, `trellis-scatter-sequence{,-review}.json`, `trellis-scatter-http-trace{,-review}.json`, `trellis-scatter-cohort-v2{,-review}.json`, `trellis-scatter-integration-source-change.json`, `trellis-scatter-integration-focused-tests-final.txt`, `trellis-scatter-integration-typecheck.txt`, `trellis-scatter-integrated-full{,-review}.json`, `trellis-scatter-integrated-http{,-review}.json`, `trellis-scatter-integrated-cohort{,-v2}{,-review}.json` | Existing six-pair serial/default/combined comparisons and final-source native gates are accepted. Reuse them; broader pressure and final report work remain separately tracked in Phase 6. | k2/k4 down projections (only k3 interleaved is qualified) or a new bit allocation from R19. |
+| R4 | Axis-1 reduce and gate/up fusion | **Adopted in the v0.4.0 default.** Variant 7 shares each decoded gate/up weight across M=2..4 with fused gate/up, keeping the variant-6 accumulator order; exact matrix/MLP/full-model gates at M=1..5 and six AB/BA blocks retain the complete-MLP gain; the last-position append sweep confirms the M=4/5 cost discontinuity. Sharing activation loads across 2/4/8 output rows per SIMD group loses at k3 M=1; smaller threadgroups do not rescue it. The M=1 decode kernel is unchanged; the 64/128/256-thread and accumulator sweep with spill/occupancy inspection never ran because per-dispatch counters are unavailable. | Trellis variant 13 is the shipped default; older variants remain explicit controls. | `paired-shared-m/`, `variant7/`, `shared-m-full-model.json`, `shared-m-append-lengths.json`, `trellis-shared-rows-k3-r*-m1.json`, `trellis-shared-rows-k3-r*-s*-m1.json` | Missing resource: per-dispatch GPU counters or an Instruments shader profile (unsupported on the M4 Pro) to drive the M=1 thread/rows/accumulator sweep. | Kernel-level GPU timing becomes available, or a k mixture changes gate/up bit widths. |
 | R5 | Decode primitive | **Measured loss.** Every alternate decoder loses at M=1 (M4 Pro): device f32 LUT (all bit widths), constant int16 table (0.802 vs 0.331 ms reduce, 0.747 vs 0.443 ms scatter, 1.474 vs 0.615 ms gate/up), device/threadgroup fp16 tables, and a constant int16 table on MLX 0.32.2 (nearly 2x complete MLP). The integer y-mad rewrite is exact with no useful gain; the readable expression stays. The only retained table is the scoped M3/4 scatter codebook (R3). The activation audit finds the packed fused SwiGLU uses a float32 precise-exp sigmoid unlike compiled MLX: a Lab distinction, no default change. | None. | `lut6-*.json`, `trellis-codebook.json`, `trellis-codebook-scoped{,-review}.json`, `trellis-constant-short-k3-m1.json`, `trellis-device-half-k3-m{1,4}.json`, `trellis-threadgroup-half-k3-m{1,4}.json`, `mlx-upgrade-0.32.2/trellis-constant-{screen,review,invalid-attempt}.json`, `trellis-y-mad-manifest.json`, `trellis-y-mad-k*-m*.json`, `trellis-activation-audit.json` | None for exact rewrites; a changed code family (QTIP/YAQA computed codes) needs a new artifact plus the KL/MMLU/tGSM/rawGSM screen before any kernel. | A hardware generation with faster threadgroup/constant memory relative to ALU, or a new code family with quality evidence. |
-| R6 | Offline repacking without re-quantization | **Repeatable win (opt-in artifact).** Two-block interleaving of k3 axis-0 code words inverts to identical code bytes; prototype readers preserve 35 full-model cases. A separately written lossless artifact (quantizer `--interleave-codes`) read by every kernel from one resident code copy: six native pairs improve single-token/M4 forwards and 64-token generation in every pair with equal peak allocation; six serial and six default-scheduler HTTP pairs improve decode at 6/128/512 prompt tokens in every pair; TTFT small or mixed; RSS flat. | Not a default artifact change; served benchmarks use `Qwen3.8-27B-q3-trellis-ldlq-k300-packed-interleave2-rd`. | `trellis-block-interleave-screen-review.json`, `trellis-block-interleave-full-model-{g2,g10}-review.json`, `trellis-block-interleave-native-generation-g2-interface-review.json`, `trellis-block-interleave-http-{serial,default}-review.json`, `trellis-interleaved-readers-screen-review.json`, `trellis-interleaved-readers-full-model-count-review.json`, `trellis-interleaved-readers-order-full-model-review.json`, `trellis-interleaved-vector-schedule-screen-review.json`, `trellis-interleaved-readers-schedule-full-model-review.json`, `trellis-interleave-integrated-tests.txt`, `trellis-interleave-integrated-tail-tests.txt`, `trellis-interleave-model-free.txt`, `trellis-interleave-artifact-review.json`, `trellis-interleave-artifact-native-fd-review.json`, `trellis-interleave-encoder-tests.txt`, `trellis-interleave-artifact-http-{serial,default}-review.json` | Quiet-machine six-pair original vs interleaved artifact `bench-serve all` serial with prompt cache enabled (all artifact gates ran cache-disabled). | Expanded state-window index variant (bytes for ALU) untested; k2/k4 interleaving unqualified. |
-| R7 | Small-M packed kernel | **Repeatable win (opt-in).** Shared-M variants 7/8/10 reuse decoded weights across M=2..4 (R3/R4); the M=4/5 whole-model cost discontinuity is confirmed and the packed-to-expanded crossover is also a numerical boundary: the M≤8 crossover prototype wins MLP time but changes output bytes for every bit width, so it is Lab and not retained. The wide-prefill kernel (variants 11–13 on MLX 0.32.2/M3+) shares each decoded row across all M=5..15 vectors reproducing native `gemv_wide` arithmetic: 36 forward/state/continuation cases and six generations exact; six HTTP pairs per lane improve TTFT in every pair with decode/RSS unchanged. Fill spans and MTP verifies inherit these curves (R15/R20). | `MLX_BUN_TRELLIS_VARIANT=7..13`; default 6 unchanged. | `shared-m-append-lengths.json`, `m8-screen/`, `mlx-upgrade-0.32.2/trellis-runtime-full`, `mlx-upgrade-0.32.2/trellis-wide-{screen,perf,full,integrated-full}`, `mlx-upgrade-0.32.2/trellis-wide-http{,-review}.json`, `mlx-upgrade-0.32.2/trellis-wide-continuous.json` | Pre-M3 GPU qualification of the wide path (the gate keeps the earlier dispatch there) via the `trellis-wide` model gate on an M1/M2 host; the M≤8 crossover needs a KL/long-state quality gate if pursued as Lab. | Any MLX runtime bump that changes small-M dense dispatch (rerun `trellis-runtime-full`). |
+| R6 | Offline repacking without re-quantization | **Repeatable win (opt-in artifact).** Two-block interleaving of k3 axis-0 code words inverts to identical code bytes; prototype readers preserve 35 full-model cases. A separately written lossless artifact (quantizer `--interleave-codes`) read by every kernel from one resident code copy: six native pairs improve single-token/M4 forwards and 64-token generation in every pair with equal peak allocation; six serial and six default-scheduler HTTP pairs improve decode at 6/128/512 prompt tokens in every pair; TTFT small or mixed; RSS flat. | Not a default artifact change; served benchmarks use `Qwen3.8-27B-q3-trellis-ldlq-k300-packed-interleave2-rd`. | `trellis-block-interleave-screen-review.json`, `trellis-block-interleave-full-model-{g2,g10}-review.json`, `trellis-block-interleave-native-generation-g2-interface-review.json`, `trellis-block-interleave-http-{serial,default}-review.json`, `trellis-interleaved-readers-screen-review.json`, `trellis-interleaved-readers-full-model-count-review.json`, `trellis-interleaved-readers-order-full-model-review.json`, `trellis-interleaved-vector-schedule-screen-review.json`, `trellis-interleaved-readers-schedule-full-model-review.json`, `trellis-interleave-integrated-tests.txt`, `trellis-interleave-integrated-tail-tests.txt`, `trellis-interleave-model-free.txt`, `trellis-interleave-artifact-review.json`, `trellis-interleave-artifact-native-fd-review.json`, `trellis-interleave-encoder-tests.txt`, `trellis-interleave-artifact-http-{serial,default}-review.json`, `reports/prefill-observation/interleave-cache-shared-{0..11}.md.json`, `interleave-cache-shared-review.json` | **Accepted.** Six balanced original/interleaved pairs on M4 use the default shared scheduler, KV4/MTP2 and RAM/SSD prompt caching. All requests, complete usage and restart checks match, with decode and cached completion improvements in every pair. Machine activity and timing limits are recorded in benchmarks.md; no additional unchanged serial run is required. | Expanded state-window index variant (bytes for ALU) untested; k2/k4 interleaving unqualified. |
+| R7 | Small-M packed kernel | **Adopted in the v0.4.0 default.** Shared-M variants 7/8/10 reuse decoded weights across M=2..4 (R3/R4); the M=4/5 whole-model cost discontinuity is confirmed and the packed-to-expanded crossover is also a numerical boundary: the M≤8 crossover prototype wins MLP time but changes output bytes for every bit width, so it is Lab and not retained. The wide-prefill kernel (variants 11–13 on MLX 0.32.2/M3+) shares each decoded row across all M=5..15 vectors reproducing native `gemv_wide` arithmetic: 36 forward/state/continuation cases and six generations exact; six HTTP pairs per lane improve TTFT in every pair with decode/RSS unchanged. Fill spans and MTP verifies inherit these curves (R15/R20). | Trellis variant 13 is the shipped default; older variants remain explicit controls. | `shared-m-append-lengths.json`, `m8-screen/`, `mlx-upgrade-0.32.2/trellis-runtime-full`, `mlx-upgrade-0.32.2/trellis-wide-{screen,perf,full,integrated-full}`, `mlx-upgrade-0.32.2/trellis-wide-http{,-review}.json`, `mlx-upgrade-0.32.2/trellis-wide-continuous.json` | M1 qualification of the existing wide kernel fails exactness, including every full-width cell; retain earlier dispatch there. A different pre-M3 kernel or the M≤8 crossover requires its own numerical and quality gates. | Any MLX runtime bump that changes small-M dense dispatch (rerun `trellis-runtime-full`). |
 | R8 | Fused GDN conv/state/SiLU | **Measured loss.** A fused convolution/SiLU/copied-tail prototype matches bf16/f32 outputs and live tail at B=1/2 only with MLX's typed sigmoid expression; it passes full packed-model logits, live state, continuation and speculative rollback. Against an independently materialized baseline its full-model timing is approximately flat with similar peak memory; unpromoted. Side finding landed as default: `mlx_copy` aliasing retained the source buffer; compact state materialization now uses stock DynamicSlice. | None (DynamicSlice materialization is a memory fix, not a speed default). | `gdn-stage-probe.json`, `conv-probe-typed-sigmoid.json`, `qwen-conv-materialized-packed.json`, `materialize-final-packed-comparison.json` | None; the missing instrument is per-dispatch GPU timing of the conv stage. | Attribution shows conv/SiLU above 3% of the decode critical path. |
 | R9 | GDN norm, gates and recurrence | **Measured loss.** Threadgroup layouts, fixed sequence lengths, unrolling, compiled wrappers and typed sigmoid fusion pass all 114 shape cells after correcting a float32 sigmoid rewrite that broke bf16 recurrence. A compiled fixed-length candidate preserves full-model identity but its isolated gains mostly disappear in full forwards; a fused recurrence + output RMSNorm + gate kernel passes 60 isolated cells and small-M full-model checks with a gain too small to integrate. Fixed-shape CompiledFunction over the 16 SSM groups (R16) gives -0.062%/-0.202% median. Production dispatch unchanged. | None. | `gdn-specialization-typed.json`, `gdn-static-full-model-review.json`, `gdn-fused-norm-probe.json`, `gdn-fused-norm-full-model-review.json`, `mlx-upgrade-0.32.2/qwen-fixed-ssm-groups-r6.json`, `mlx-upgrade-0.32.2/qwen-fixed-ssm-groups-rtn4-internal.json`, `mlx-upgrade-0.32.2/qwen-ssm-hot-review.json` | None until prefill-regime attribution shows the recurrence dominates (decode M=1 was the measured target). | Prefill attribution, or a transposed-state layout with a measured M=1 win. |
-| R10 | Parallel/chunkwise GDN prefill | **Unmeasured (not started).** No chunkwise or scan-decomposition formulation was implemented or timed; only sequential-scan specializations were screened under R9. Stays Lab because a different reduction order is not exact. Source leads (vLLM Qwen GDN, FLA, the Gated Delta Networks paper) are pinned only. | None. | `upstream/sources.json` | Prefill-regime attribution at 1k/4k/8k first; then a Metal chunk formulation compared on teacher-forced logits and recurrent state across chunk boundaries versus the sequential scan. | Not applicable; never opened. |
+| R10 | Parallel/chunkwise GDN prefill | **Prefill attribution complete; no parallel rewrite retained.** Current packed-target controls and an operation observer preserve complete hidden/state hashes at 1k/4k/8k on both Macs. Gated-delta work is roughly 2.6–2.7% of M4 complete prefill, so it is not the principal bottleneck. M1 instrumentation perturbs larger runs. A chunkwise/scan formulation remains Lab because changing reduction order requires a separate quality contract. | None. | `upstream/sources.json` | Attribution is accepted; prioritize the measured expansion/projection/attention costs. A later chunkwise Metal arm needs same-input recurrent-state and held-out quality checks before any adoption. | Not applicable; never opened. |
 | R11 | Affine QMV/QMM and native builds | **Inconclusive.** Shared-M affine kernels won M4/8 (and M3/5/6/7 4-bit, compact 3/8-bit) in six full-model pairs and six HTTP pairs per configuration with exact outputs and flat B1; cross-model, Gemma 12B large projections win while MiniCPM/Llama regress; a compiled-call cache was closed. MLX 0.32.2 (native pack 0.4.0) superseded it: upstream `qmv_wide` implements the same reuse with different arithmetic, so the custom kernel and its flag were removed (§7.12); frozen source controls keep it. New-core affine Q3 gate/up operations beat the old shared ops in all six blocks, Q4 mixed, Q8 loses (operation level); RTN4 short decode flat, long-prefill timing order-dependent; twelve RTN4 oracle cells and 4K/8K prompts byte-exact on both cores. No end-to-end Qwen gain from the runtime change is established. | Native pack 0.4.0 (MLX 0.32.2) is the pinned runtime; no flag. | `affine-qmv-screen.json`, `affine-qmv-compiled.json`, `affine-shared-m-probe{,-initial}.json`, `affine-shared-m-debug-scale-bits.json`, `affine-shared-m-full-model-{packed,rtn4}-review.json`, `affine-shared-m-http-rtn4-review.json`, `affine-shared-m-production-full-model-rtn4-review.json`, `affine-shared-m-shapeless.json`, `affine-shared-m-integrated-unit.txt`, `affine-shared-m-integrated-http-rtn4-review.json`, `affine-shared-m-width-{rtn4,full-model-rtn4}-review.json`, `affine-shared-m-all-bits-*.json`, `affine-shared-m-broader-*`, `shared-affine-frontier-http-{rtn4,compact}-review.json`, `affine-compiled-call-cache-{full-model,http}-compact-review.json`, `affine-shared-cross-model-exactness.json`, `affine-shared-cross-model-{gemma12,gemmae4b,minicpm5,llama1,llama3}-review.json`, `affine-cross-model-matrix-{inventory,screen,screen-review}.json`, `affine-cross-model-matrix-gemma12-repeat.json`, `affine-head-batched-decode-mfirst-{llama1,llama3}-review.json`, `affine-shared-head-http-{llama1,llama3}-overlap-review.json`, `affine-large-projection-*.json`, `affine-nonfast-*-review.json`, `mlx-upgrade-0.32.2/affine-{old,new}-screen.json`, `mlx-upgrade-0.32.2/affine-runtime-repeat{,-review}.json`, `mlx-upgrade-0.32.2/runtime-oracle-rtn4{,-macos14}.json`, `mlx-upgrade-0.32.2/runtime-oracle-long-macos14.json`, `mlx-upgrade-0.32.2/native-runtime-screen{,-review}.json`, `mlx-upgrade-0.32.2/final-source-gates.json` | Quiet-machine six-pair old-core vs 0.32.2 `bench-serve all` on the RTN4 artifact with the same-artifact mlx-lm arm, to separate the runtime change from kernel changes. | A native pack bump, or MLX changing `qmv_wide` eligibility for GQA6/D256 or Q8 shapes. |
 | R12 | Norm/residual/activation/projection fusion | **Measured loss.** Residual-add/RMSNorm kernel (90 synthetic cases exact; six packed pairs at M1/4/8/128): M1 slightly slower, M4/8 flat, M128 varies. Fused affine gate/up/SwiGLU with the typed sigmoid (48 RTN4 cells exact): only M=1 wins in isolation, full-forward gain too small. Multimodal RoPE shared angles/direct strides (108 cases exact): M128 slightly slower, M512 flat, image encoding excluded. None integrated; helpers removed. Full-attention output-gate and QKV concatenation fusions were enumerated but not prototyped. | None. | `residual-rms-screen-scalar-review.json`, `residual-rms-full-model-packed-review.json`, `affine-gateup-debug{,-typed}.json`, `affine-gateup-typed.json`, `affine-gateup-full-model-review.json`, `mrope-reuse-screen-review.json`, `mrope-stride-full-model-packed-review.json` | None until per-kernel attribution shows elementwise/norm dispatches exceed 3% of the decode critical path. | Attribution evidence as above, or a real vision-request benchmark for mRoPE. |
-| R13 | Attention and KV | **Repeatable win (opt-in).** Joint TurboQuant K/V packed decode plus row-grouped inverse rotation at N≥8192: six 8,846-token HTTP blocks on R6 k8v3 improve decode 83.87% and complete time 12.77% versus the original codec, inverse rotation adds 5.39%/1.05%; on MLX 0.32.2 the inverse improves decode 4.69% (R6) and 5.98% (RTN4) in every pair, R6 complete -0.79% all six, RTN4 complete inconclusive; six MiniCPM and six Gemma deferred-consumer pairs improve complete time. Direct packed-key attention is exact but 6–19% slower than joint decode plus native SDPA (closed). Dense register-resident attention loses timing; Steel D256 fails one KL screen (Lab). KV append census: ordinary appends copy only the new row, rejecting the whole-cache-copy hypothesis; no append kernel. bf16-KV rows unaffected; TurboQuant KV remains serial-only. | `MLX_BUN_TURBOQUANT_FUSED_DECODE=1`, default off; details in turboquant.md. | `tq-value-*{,-review}.json`, `tq-kv-screen.json`, `tq-kv-independent.json`, `tq-kv-repeat{,-review}.json`, `tq-kv-full{,-review}.json`, `tq-kv-policy-http{,-review}.json`, `tq-kv-long-full{,-review}.json`, `tq-kv-rotation-{screen,repeat,rows-screen,rows-repeat,long,long-http}{,-review}.json`, `tq-kv-rtn4-full{,-review}.json`, `tq-kv-nonfinite.json`, `tq-kv-minicpm-full{,-review}.json`, `tq-kv-gemma-e4b-full{,-review}.json`, `tq-kv-integrated-*{,-review}.json`, `tq-kv-integrated-rtn4-http-acceptance.json`, `tq-kv-memory{,-review}.json`, `allocator-capacity-repro.json`, `tq-kv-inverse-integrated-r6-full{,-review}.json`, `mlx-upgrade-0.32.2/{r6,rtn4}-kv-inverse-full.json`, `mlx-upgrade-0.32.2/inverse-kv-http{,-replacement,-review,-isolation}.json`, `mlx-upgrade-0.32.2/deferred-kv-http{,-review}.json`, `tq-key-sdpa-screen.json`, `tq-key-sdpa-complete-repeat{,-review}.json`, `steel-attention-tune-manifest.json`, `attention-cpu-check.json`, `attention-oracle-check.json`, `attention-full-model-packed-128-512-review.json`, `dense-attention-memory-tune-review.json`, `kv-append-donation-probe{,-review}.json`, `kv-append-model-trace-2049{,-review}.json`, `kv-append-http-text-census{,-review}.json`, `kv-append-native-8193{,-review}.json` | Quiet-machine six-pair `bench-serve all --arms mlx-bun-serial --kv-quant turbo` flag off/on on both quants at 8k context; separately the unmeasured three-query KV4 verify-window head-pairing screen from §7.7. | MLX ships fused quantized SDPA for GQA6/D256, TurboQuant KV becomes batch-eligible, or the RTN4 complete-time cell resolves. |
-| R14 | Vocabulary head and sampler | **Inconclusive.** Compiled Metal grammar bitmask (`MLX_BUN_TOKEN_MASK=metal`) preserves mask bytes and sampled IDs at three vocabularies and beats the host loop in eight paired blocks, but six compact HTTP pairs at B1/B4 are inconclusive (B1 flat, B4 regresses under residual swap); host masking remains default. Top-p inverse-permutation removal is exact in 216/180 cases; Qwen packed serial six pairs +0.183% complete time (loss), Llama 1B -0.399%/-0.380% with six wins per lane; no production change. DSA sort keys reduced to uint32 are integrated as a shared-operation win (selection latency -10.47..-15.80%), not a Qwen result. Grouped greedy verify readback closed (R21). The exact fused matvec/argmax head was never prototyped. | `MLX_BUN_TOKEN_MASK=metal` opt-in, default `host`; DSA uint32 keys landed (shared operation). | `token-mask-probe.json`, `token-bitmask-unit.txt`, `token-bitmask-grammar-metal.txt`, `token-mask-sampling{,-review}.json`, `grammar-mask-http-compact-review.json`, `top-p-scatter-screen-v2{,-review}.json`, `top-p-scatter-compiled-screen{,-review}.json`, `top-p-lifetime-{source,native,scatter,metal}.json`, `top-p-sampler-screen-v2{,-review}.json`, `top-p-native-screen{,-review}.json`, `top-p-http-screen{,-review}.json`, `top-p-http-r6-serial-repeat{,-review}.json`, `top-p-http-llama-repeat{,-review}.json`, `dsa-order-u32-{screen,repeat}{,-review}.json`, `dsa-rank-u32-{screen,repeat}{,-review}.json`, `dsa-model-screen.json`, `dsa-rank-u32-model-tests-v2{.txt,.transform.json}`, `dsa-production-{unit-v2,typecheck-v2,model-free,hygiene}.txt` | Quiet-machine six-pair grammar-mask host vs metal at B1/B4 on a structured-output fixture (the only open candidate with a positive operation result). | Head attribution shows the vocabulary projection or readback material for greedy requests. |
-| R15 | Deterministic no-verification replay | **Repeatable win (opt-in).** Strict fill has parser-state guards (complete `</parameter>` delimiter, request-local tool-only context), a schema open-object compiler fix, and model-owned four-token append chunks that reproduce M1 arithmetic via broadcast `[M,1,K]` weight views and split at MLX attention transitions (KV 1023/1024/8192/32768/65536 on `applegpu_g16s`). Final internal-SSD comparison: twelve balanced pairs, 96 exact responses, zero verification; complete time -9.83%/-15.40% (R6 weather/bash) and -1.63%/-2.94% (RTN4), every pair improving; tool output is buffered so this is not a decode-rate result. Staged-body prototype flat, closed. Quantized-KV composition: TurboQuant k8v3 passes after a lease fix, affine KV4 fails at layer 4; guard retained. Held-out session corpus unmeasured. | `MLX_BUN_FILL=strict`, `MLX_BUN_FILL_APPEND_CHUNK_SIZE=0` (model limit), `MLX_BUN_FILL_MAX_SPAN=32`; default off. | `fill-parser-boundary-r6.json`, `fill-boundary-tokenizer-audit-v4.json`, `fill-boundary-cpu-tests-v8.log`, `fill-boundary-native{,-m1,-batch1,-batch1-v2-rtn4}-r6.json`, `fill-first-layer-{shape-probe,batched-m1}.json`, `fill-boundary-http-{screen,repeat,review}.json`, `fill-append-*.json`, `fill-append-final-model-free.log`, `fill-append-state-ownership-native-test.log`, `fill-schema-*`, `strict-fill-variant{6,7}.txt`, `strict-fill-v1{0,1}-cap{32,4}-served-eos.json`, `fill-adversarial-state-v12-eos-review.json`, `fill-state-v12-cap{32,4}-{eos,seed42}.json`, `fill-identical-input-append-review.json`, `fill-chunked-*-review.json`, `fill-integrated-{state-v12-seed42,http-packed}-review.json`, `fill-chunked-integrated-{unit,model-free,typecheck,hygiene}.txt`, `fill-staged-unit.txt`, `fill-staged-state-v13-seed42-review.json`, `fill-staged-complete-requests-review.json` | Frozen held-out real-session/adversarial corpus through `MLX_BUN_TEST_FILL_MODEL=<dir> MLX_BUN_FILL=strict bun test tests/parity/fill-strict.test.ts` on both quants (token IDs, stop decisions, subsequent logits/state), then a quiet six-pair HTTP repeat. | A Qwen tool-template or parser change, tool schemas with pattern properties, or a chunk-limit qualification on another GPU. |
-| R16 | MLX graph and loop | **Repeatable win (opt-in).** Early first token in the serial loop: six native pairs improve first-token latency at 6/128/512 tokens with exact tokens/state/usage; six chat HTTP pairs (84 requests, 42 exact) improve first visible output with complete time flat; raw prompts show no client gain because token 220 emits no event. The suffix screen finds a prefill-shape effect after a one-token return (seven packed/six RTN4 cases change state); explicit M=1 alignment restores exactness, so promotion is gated. Closed: shape-specific compiled-call cache, Metal launch-config cache (MiniCPM HTTP mismatch unresolved), untouched-finalize elision (-0.069%), submission/traversal limits for Qwen (Llama 25/25 gives -2.44% native and -1.86..-3.02% HTTP, diagnostic, not integrated), fixed-input proxies, per-layer graphs (RTN4 fails identity), whole-model graph (slower), SSM-group compilation (-0.06%/-0.20%). Shapeless custom kernels are an upstream MLX limitation. | `MLX_BUN_EARLY_FIRST_TOKEN=1`, default off. | `first-yield-*`, `first-token-production-long-{serial,continuous}{,-review}.json`, `first-token-lifecycle-{llama,qwen}{,-review}.json`, `affine-compiled-call-cache-{full-model,http}-compact-review.json`, `metal-config-cache-*`, `mlx-untouched-finalize-source-manifest.json`, `untouched-finalize-*`, `graph-schedule-screen-plan.json`, `graph-schedule-screen{,-review,-contamination}.json`, `graph-schedule-rtn4-rescreen-v2{,-review}.json`, `graph-schedule-llama-repeat{,-review}.json`, `graph-schedule-llama-http-repeat{,-review}.json`, `mlx-upgrade-0.32.2/custom-kernel-compile-probe.json`, `mlx-upgrade-0.32.2/fixed-custom-probe.json`, `mlx-upgrade-0.32.2/qwen-fixed-region-{r6,rtn4}-v{2,3,4}.json`, `mlx-upgrade-0.32.2/qwen-fixed-layer-{r6-full-v2,rtn4-full}.json`, `mlx-upgrade-0.32.2/compiled-region-probe.json`, `mlx-upgrade-0.32.2/qwen-fixed-hidden-regions-{r6,rtn4}.json` | The serial/native candidate now completes the missing M=1 step on non-aborted early return when the caller retains its cache. Original-source regression is reproduced; unit and M1 packed-Qwen/MiniCPM/Gemma state/continuation gates pass. Finish M4 packed/affine and server checks, then a quiet six-pair chat HTTP repeat; separately, scoped Llama submission-limit integration at the model-owned seam. | MLX adds shape-inferring CustomKernel or graph replay, or per-token host-overhead attribution changes. |
-| R17 | Hybrid prefix and recurrent state reuse | **Inconclusive.** The paired MTP prefix cache retains one aligned target/draft prefill boundary; native restores at 128/513/2,051 tokens are exact (also under Luke's sampling recipe), the HTTP gate returns eight identical responses, and one serving-suite diagnostic shows cached TTFT 91.9 ms versus 190.8 ms original. It captures prompt boundaries only. The working-tree producer now shares the existing RAM budget and SSD lifecycle, with native bf16/KV4 RAM and restart identity gates on both Macs; generated-history checkpoints and pressure/performance acceptance remain unfinished; in the completed Kanban run the compaction summary (90,214 tokens, 19m 42.7s) and first tool turn (8m 53.4s) had zero cached tokens. The SSD boundary-before-transfer fix landed as a durability correction. No balanced timing pairs exist for the prefix cache itself. | Paired prefill reuse enabled in candidate source; default-selection checks pending. SSD boundary fix landed; MTP output persistence remains open. | `mtp-cache-candidate-native.json`, `mtp-cache-http-gate.json`, `mtp-prefix-luke-sampling.json`, `luke-history-kv4-preflight.json`, `ssd-boundary{,-two}-{before,after}.json`, `ssd-boundary-tests.txt`, `serving-compact-{boundary-fixed,bun140-throw,bun142-throw,shared-guard}.review.json`, `agent-boundary-fixture.json`, `agent-boundary-current{,-review}.json`, `kanban-final/suite-driver.json`, `kanban-final/optimized-mtp-cache-suite.md.json` | Generated-history checkpoint (target KV, draft KV, pending hidden row aligned to the next rendered prefix) measured on the saved Kanban tool-turn requests; the 8m53s/19m42s zero-cache prefills are the target. | Open until generated boundaries are cached; also if Pi's compaction reserve can be tuned within the published profile. |
-| R18 | Scheduler | **Repeatable win (opt-in).** Continuous-scheduler early yield (same `MLX_BUN_EARLY_FIRST_TOKEN` flag): six AB/BA blocks, 84 responses, 42 exact, median TTFT -7.66% code / -7.90% explanation / -5.63% JSON (74–77 ms) with complete time +0.04..+0.15%; the integrated one-block check gives -68..-90 ms; 120 streaming cancellation/arrival requests pass on both quants and lanes. Concurrency evidence for kernels: v6 vs v12 four-request waves improve every pair and the tile pass improves eight-request throughput (R1). No B-wide prefill (Phase 18 S1a), no shape buckets, no fixed-arrival sustained-serving cell; bench-serve's four-request leg is serial queue throughput. | `MLX_BUN_EARLY_FIRST_TOKEN=1`, default off. | `first-prepare-scheduler-{control,candidate}.txt`, `first-prepare-http-screen-plan.json`, `first-prepare-http-{screen,repeat}{,-review}.json`, `first-prepare-production-{unit,model-free,typecheck,hygiene}.txt`, `first-prepare-production-http-r6-v2{,-review}.json`, `first-token-lifecycle-{llama,qwen}{,-review}.json`, `trellis-concurrent-http{,-review}.json`, `trellis-tile-http-review.json`, `trellis-scatter-integrated-cohort-v2{,-review}.json` | Fixed-arrival-schedule sustained-serving cell (concurrency 2/4/8; per-request queue/TTFT/latency and aggregate tok/s, verified lane and admitted rows) per §7.10 on a quiet machine. | B-wide prefill lands, or the R16 cache-boundary reconciliation changes scheduler behavior. |
-| R19 | Weight-quality allocation | **Unmeasured (not started).** No latency-aware allocation objective ran alongside the kernel work as §7.7 requires. Quality evidence predates §7 and lives in turboquant.md (Q3 KL 0.1553 at 3.55 bpw; Q2b packed 12.14 GiB, KL 0.1550; rotated down axis kept because `--down-axis in` costs rawGSM 29/50 vs 44/50). The Kanban task ran on the k300 packed artifact; R6 interleaving is lossless so it adds no quality delta. Q5 2.75-budget arm, q2a/q2b task columns and the rawGSM EOS-cliff root cause remain open in turboquant.md. | None. | turboquant.md "Q campaign" and "FINAL BOARD" sections | Allocation sweep weighting per-role bits by the measured M=1 kernel costs from R3/R4/R5, gated by the frozen KL/MMLU/tGSM/rawGSM screens; needs GPU time for quantization and evaluation. | Not applicable; never opened. |
-| R20 | MTP economics | **Repeatable win (opt-in).** Six balanced three-arm blocks versus native full-prefill on the packed artifact: fixed two drafts change complete time -43.22%/-38.81%/-44.92% (code/explanation/JSON) with every pair improving, TTFT -5.5..-8.4%, peak RSS +653 MiB; RTN4 -34.95%/-26.95%/-38.48%. Adaptive one-to-three drafts adds about 3% (ratios 0.9698/0.9782/0.9624) and stays research-only. Folded 4-bit/8-bit MTP heads: -1.48/-3.67/-2.95% and -0.65/-2.29/-1.86% with peak RSS 12.53/12.74 vs 13.04 GB; 1,184 vocabulary rows and 48 state samples byte-exact against serial replay. EOS and finish-reason accounting fixed. The quiet M1 Max control (0.821x at gamma 2, 2026-08-18) predates the kernel work. The shared-serving candidate now has a short-context depth-2/3/4 settings screen and native depth-3 coverage; see benchmarks.md. Depth 3 reaches the native matrix threshold for the KV4 value product, so late-context economics remain a separate open comparison. | `--draft-kind mtp --num-draft-tokens 2` opt-in; EOS/finish-reason fixes landed. | `mtp-v*-gamma*.json`, `mtp-corpus-screen/`, `mtp-screen-prompts.json`, `mtp-heldout-{plan,http,http-review}.json`, `mtp-heldout-native{,-review}.json`, `spec-token-counts-*.txt`, `mtp-post-eos-http{,-review}.json`, `spec-eos-serving-*.txt`, `mtp-eos-budget{,-integrated}-http{,-review}.json`, `spec-eos-continuous-*.txt`, `eos-budget-continuous{,-v2}-http{,-review}.json`, `mtp-quantized-draft-*-http{,-review}.json`, `mtp-draft-artifact-q4-{build,audit}.json`, `mtp-draft-quant-oracle.json`, `mtp-draft-artifact-q8-build.json`, `mtp-quantized-loader-*.txt`, `mtp-artifact-ownership{,-v2}.json`, `mtp-artifact-{screen,repeat}-http{,-review}.json`, `mtp-artifact-native{,-context}{,-review}.json`, `mtp-artifact-context-screen-http.json`, `mtp-matched-prefill-{r6,rtn4}-{screen-http,repeat}{,-review}.json`, `mtp-rtn4-screen-{plan,http,http-review}.json`, `mtp-rtn4-divergence-v2-audit{,-review}.json`, `mtp-rtn4-prefill-control{,-review}.json`, `mtp-v10-{planets,prose}-gamma{2,3}.json`, `mtp-provider-dispose-{before,after}.json`, `mtp-provider-ownership-tests.txt`, `mtp-post-dispose-http.json` | Draft depth 1/2/3 versus ordinary decode at late-task context depths (77k+ tokens, KV4, published profile): complete request time, acceptance and verify-window attention cost; then a quiet six-pair `bench-serve all` with the q4 head (command in benchmarks.md). | Any change to verify-window arithmetic (KV4 head pairing, fused KV) or a DFlash2 drafter (R21). |
-| R21 | Other exact speculative sources | **Blocked.** Qwen DFlash2 (the z-lab artifact behind the oMLX record the video cites; block size five, native MTP off) needs dynamic grouped causal convolutions around attention and MLP plus a top-16/rank-256 selector; the existing DFlash provider does not implement that architecture, so loading the checkpoint would not be DFlash2. Grouped greedy verify readback is closed: six three-arm blocks, 84 exact comparisons, concatenation improves only 0.33%/0.06%/0.23% and the evalAll variant regresses JSON 0.44%. Prompt-lookup, n-gram and suffix sources were not measured on Qwen in this campaign; SpecPrefill is approximate and belongs to a Lab arm. | None. | `video-dHK90xc9Q64/`, `video-dHK90xc9Q64/dflash-source-manifest.json`, `spec-grouped-greedy-screen.json`, `spec-grouped-greedy-eval-screen.json`, `spec-grouped-greedy-three-http{,-review}.json` | Missing resource: a DFlash2 architecture/selector port with target-state/rollback identity and draft residency, then a three-arm block (ordinary / fixed-two MTP / DFlash2) on the held-out prompts. | The port exists, or an exact n-gram source shows at least 3% on the saved Kanban history. |
-| R22 | Residency and shared memory | **Measured loss.** Page-aligned file-backed weights through Metal roots are exact in every payload/output case, but six native pairs give +0.25% (later +0.02%) on 64-token generation with no faster pair, six serial HTTP pairs +0.04..+0.14% complete and +0.14..+0.21% decode, and the balanced cold-start follow-up is worse in five of six pairs (+18.42% first token, +12.93% first generation). Peak RSS falls from 12,182,749,184 to 482,508,800 bytes as file-backed accounting only; physical RAM saving unproven. A read-only donation hazard was found and fixed in the prototype before closing; helpers removed. Retained as memory correctness, not speed: the 8 MiB bounded range cache in `mlx/ops.ts` (Kanban sustained run peak active -1,582,612,480 bytes, +2.171% time in one pair) and per-chunk draft-KV evaluation. | None for speed; bounded range cache and per-chunk draft-KV evaluation landed as memory fixes. | `mmap-page-window{,-pointer}-screen.json`, `mmap-weights-native-{llama1,packed}-screen.json`, `mmap-weights-native-packed-paired{,-review}.json`, `mmap-weights-http-serial{,-review}.json`, `mmap-weights-ownership-packed{,-mapping}.json`, `mmap-weights-single-consumer-lazy{,-mapping}.json`, `mmap-weights-whole-consumer-lazy.json`, `mmap-whole-consumer-source-integrity.json`, `mmap-donation-{ro,cow}-screen.json`, `mmap-managed-donation-screen.json`, `mmap-weights-managed-whole-consumer-lazy{,-mapping}.json`, `mmap-managed-submitted.json`, `mmap-weights-managed-native-{repeat,screen}{,-review}.json`, `mmap-initial-residency-attribution.json`, `mmap-lazy-view-ownership.json`, `mmap-weights-lazy-managed-native-screen{,-review}.json`, `mmap-prefault-native-screen.json`, `mmap-cold-native-repeat{,-review}.json`, `mmap-managed-shard-release{,-mapping}.json`, `mmap-managed-active-accounting-review.json`, `kanban-arange-{allocation-evidence,regression,micro-memory,reconstruction}.json`, `kanban-sustained-77k-owned-control.json`, `kanban-sustained-arange-comparison.json` | Reusable integer position-range prefix and native integer `arange` on Bun 1.4 against the bounded-cache baseline, gated on exact range/mask values, view ownership and the memory bound, measured on complete prefill and verification rounds. | A controlled page-cache-cold startup cell (§7.10) exists; mmap lost warm but cold was never controlled. |
-| R23 | Media and task completion | **Inconclusive.** The fresh Pi Kanban task on the 12.14 GiB packed target with Luke's `qwen3.8-27b-q3_k_xl-coding-128k` profile (131072 context, xhigh thinking, temperature 0.6, seed 42; KV4, MTP depth 2, paired prefixes, MLX 0.32.2) completed in 3h 33m 7.685s with 130,494 output tokens over 62 requests, 79.706% draft acceptance, 11.916 GiB peak combined RSS; the untouched app passes the functional browser checks. Earlier attempts failed on a tool-parser bug, dropped reasoning history, Metal OOM at the 85,238-token request and OOM during sustained decode. No matched successful original-source run exists, so no task-time speedup can be claimed. Image/video tower and encoder-cache cells were not measured. | Tool-parameter parser and `reasoning_content` alias landed as correctness fixes; no speed default. | `lukes-kanban/`, `kanban-final/optimized-mtp-invalid-32k-sleep/`, `kanban-final/luke-q3-128k-xhigh-{history-fixed,tool-values-fixed,mtp-memory-fixed,arange-bounded}/`, `kanban-final/comparison.html`, `kanban-capacity-110k-control.json`, `kanban-capacity-8k-final-comparison.json`, `kanban-capacity-110k-owned-final.json`, `kanban-memory-http-{owned,responsiveness}.json`; numbers in benchmarks.md Kanban sections | Missing resource: about four hours of exclusive GPU ownership on an idle machine for a matched original-source run (`673b43f`, variant 6, bf16 KV, no MTP) under the identical profile, prompt, seed and harness. | Open while no baseline exists; revisit with generated-history checkpoints (R17) and the compaction-reserve arm. |
-| R24 | Larger departures | **Blocked.** Only feasibility and operation screens exist. For the video's tq3-mini codebook format (eight-centroid indices, ten codes per uint32, 3.2 bits/value), the literal upstream `polar_qmv` through the Bun binding on synthetic matrices: explicit SIMD shuffles cut the 5120-to-17408 rotated operation 2.77–3.84% (every pair), and a fused fp16 expansion cuts large-prefill operation time 40.85–57.51% with about 765 MiB less temporary allocation. No production loader or kernel, no quality evidence; the checkpoint's reported agent failures are unresolved. Learned quantization, sparse residuals, activation quantization, pruning, layer skipping, low-rank, QAT/distillation and alternative drafters have no recorded kernel/quality/memory/benefit estimate. | None. | `video-dHK90xc9Q64/`, `polar-qmv-screen-v2.json`, `polar-qmv-variants.json`, `polar-qmv-geometry.json`, `polar-qmv-repeat{,-review}.json`, `polar-expand.json`, `polar-expand-repeat{,-review}.json` | Missing resource: a decision to build a loader for the tq3-mini format plus its KL/MMLU/tGSM/rawGSM screen on the actual checkpoint; without that the row is a feasibility bound only. | An artifact with quality evidence at or below Q2b bytes, or the operation gains reproduced whole-model. |
+| R13 | Attention and KV | **Adopted for TurboQuant decoding.** Joint TurboQuant K/V packed decode plus row-grouped inverse rotation at N≥8192: six 8,846-token HTTP blocks on R6 k8v3 improve decode 83.87% and complete time 12.77% versus the original codec, inverse rotation adds 5.39%/1.05%; on MLX 0.32.2 the inverse improves decode 4.69% (R6) and 5.98% (RTN4) in every pair, R6 complete -0.79% all six, RTN4 complete inconclusive; six MiniCPM and six Gemma deferred-consumer pairs improve complete time. Direct packed-key attention is exact but 6–19% slower than joint decode plus native SDPA (closed). Dense register-resident attention loses timing; Steel D256 fails one KL screen (Lab). KV append census: ordinary appends copy only the new row, rejecting the whole-cache-copy hypothesis; no append kernel. bf16-KV rows are unaffected. TurboQuant now composes with shared ordinary and speculative rows, including the accepted long cached/SSD run. | `MLX_BUN_TURBOQUANT_FUSED_DECODE` defaults on; `=0` retains the unfused control. Cache quantization remains an independent opt-in; details in turboquant.md. | `tq-value-*{,-review}.json`, `tq-kv-screen.json`, `tq-kv-independent.json`, `tq-kv-repeat{,-review}.json`, `tq-kv-full{,-review}.json`, `tq-kv-policy-http{,-review}.json`, `tq-kv-long-full{,-review}.json`, `tq-kv-rotation-{screen,repeat,rows-screen,rows-repeat,long,long-http}{,-review}.json`, `tq-kv-rtn4-full{,-review}.json`, `tq-kv-nonfinite.json`, `tq-kv-minicpm-full{,-review}.json`, `tq-kv-gemma-e4b-full{,-review}.json`, `tq-kv-integrated-*{,-review}.json`, `tq-kv-integrated-rtn4-http-acceptance.json`, `tq-kv-memory{,-review}.json`, `allocator-capacity-repro.json`, `tq-kv-inverse-integrated-r6-full{,-review}.json`, `mlx-upgrade-0.32.2/{r6,rtn4}-kv-inverse-full.json`, `mlx-upgrade-0.32.2/inverse-kv-http{,-replacement,-review,-isolation}.json`, `mlx-upgrade-0.32.2/deferred-kv-http{,-review}.json`, `tq-key-sdpa-screen.json`, `tq-key-sdpa-complete-repeat{,-review}.json`, `steel-attention-tune-manifest.json`, `attention-cpu-check.json`, `attention-oracle-check.json`, `attention-full-model-packed-128-512-review.json`, `dense-attention-memory-tune-review.json`, `kv-append-donation-probe{,-review}.json`, `kv-append-model-trace-2049{,-review}.json`, `kv-append-http-text-census{,-review}.json`, `kv-append-native-8193{,-review}.json` | The combined pressure gate is complete on both Macs. Current default shared off/default comparisons pass on M1 MiniCPM and M4 packed Qwen/MTP3; cached complete time improves in both orders, with mixed short-Qwen timing documented in benchmarks.md. The earlier native, six-pair serial and deferred-consumer gates remain accepted; no unchanged gate needs repeating. | MLX ships fused quantized SDPA for GQA6/D256, or a new codec shape/model exposes a numerical or complete-request regression. |
+| R14 | Vocabulary head and sampler | **Adopted normalized greedy kernel; other candidates remain characterized separately.** Same-input model/operation checks pass on both Macs. MiniCPM standard AB/BA throughput improves and packed Qwen MTP3/TQ stays effectively flat. **Inconclusive.** Compiled Metal grammar bitmask (`MLX_BUN_TOKEN_MASK=metal`) preserves mask bytes and sampled IDs at three vocabularies and beats the host loop in eight paired blocks, but six compact HTTP pairs at B1/B4 are inconclusive (B1 flat, B4 regresses under residual swap); host masking remains default. Top-p inverse-permutation removal is exact in 216/180 cases; Qwen packed serial six pairs +0.183% complete time (loss), Llama 1B -0.399%/-0.380% with six wins per lane; no production change. DSA sort keys reduced to uint32 are integrated as a shared-operation win (selection latency -10.47..-15.80%), not a Qwen result. Grouped greedy verify readback closed (R21). The exact fused matvec/argmax head was never prototyped. | Shared normalized greedy selection is the default. Grammar-mask Metal selection remains opt-in; DSA uint32 keys are integrated. | `token-mask-probe.json`, `token-bitmask-unit.txt`, `token-bitmask-grammar-metal.txt`, `token-mask-sampling{,-review}.json`, `grammar-mask-http-compact-review.json`, `top-p-scatter-screen-v2{,-review}.json`, `top-p-scatter-compiled-screen{,-review}.json`, `top-p-lifetime-{source,native,scatter,metal}.json`, `top-p-sampler-screen-v2{,-review}.json`, `top-p-native-screen{,-review}.json`, `top-p-http-screen{,-review}.json`, `top-p-http-r6-serial-repeat{,-review}.json`, `top-p-http-llama-repeat{,-review}.json`, `dsa-order-u32-{screen,repeat}{,-review}.json`, `dsa-rank-u32-{screen,repeat}{,-review}.json`, `dsa-model-screen.json`, `dsa-rank-u32-model-tests-v2{.txt,.transform.json}`, `dsa-production-{unit-v2,typecheck-v2,model-free,hygiene}.txt`; `reports/prefill-observation/sampler-*` | Quiet-machine six-pair grammar-mask host vs metal at B1/B4 on a structured-output fixture (the only open candidate with a positive operation result). | Head attribution shows the vocabulary projection or readback material for greedy requests. |
+| R15 | Deterministic no-verification replay | **Repeatable win (opt-in).** Strict fill has parser-state guards (complete `</parameter>` delimiter, request-local tool-only context), a schema open-object compiler fix, and model-owned four-token append chunks that reproduce M1 arithmetic via broadcast `[M,1,K]` weight views and split at MLX attention transitions (KV 1023/1024/8192/32768/65536 on `applegpu_g16s`). Final internal-SSD comparison: twelve balanced pairs, 96 exact responses, zero verification; complete time -9.83%/-15.40% (R6 weather/bash) and -1.63%/-2.94% (RTN4), every pair improving; tool output is buffered so this is not a decode-rate result. Staged-body prototype flat, closed. Quantized-KV composition: TurboQuant k8v3 passes after a lease fix, affine KV4/KV8 now preserve complete state after one-token attention is retained within shared projection appends. The qualified model binding declares KV4/KV8 and K8V3 support; delayed conversion passes real-generation state checks. Both quants retain exact HTTP responses in six-pair affine and TQ comparisons. Saved tool-history fixtures pass 20 affine pairs; held-out task quality and quantized verification remain separate. Shared strict fill now composes the common preparation, sampler, row state and RAM/SSD cache. Its qualified B1 append, delayed precision transitions and B2 retirement into B1 pass full hidden/cache/continuation checks; wider rows preserve one-position arithmetic. Six M4 HTTP arms complete 108 responses with exact paired outputs/counts/cache coverage and durable SSD restarts. The sequential fixture total improves 13.50%/13.57% against shared fill-off and remains within 0.13% of serial strict fill; concurrent latency tradeoffs remain. See benchmarks.md and reports/prefill-observation/shared-fill-http*.json. | `MLX_BUN_FILL=strict`, `MLX_BUN_FILL_APPEND_CHUNK_SIZE=0` (model limit), `MLX_BUN_FILL_MAX_SPAN=32`; default off. | `fill-parser-boundary-r6.json`, `fill-boundary-tokenizer-audit-v4.json`, `fill-boundary-cpu-tests-v8.log`, `fill-boundary-native{,-m1,-batch1,-batch1-v2-rtn4}-r6.json`, `fill-first-layer-{shape-probe,batched-m1}.json`, `fill-boundary-http-{screen,repeat,review}.json`, `fill-append-*.json`, `fill-append-final-model-free.log`, `fill-append-state-ownership-native-test.log`, `fill-schema-*`, `strict-fill-variant{6,7}.txt`, `strict-fill-v1{0,1}-cap{32,4}-served-eos.json`, `fill-adversarial-state-v12-eos-review.json`, `fill-state-v12-cap{32,4}-{eos,seed42}.json`, `fill-identical-input-append-review.json`, `fill-chunked-*-review.json`, `fill-integrated-{state-v12-seed42,http-packed}-review.json`, `fill-chunked-integrated-{unit,model-free,typecheck,hygiene}.txt`, `fill-staged-unit.txt`, `fill-staged-state-v13-seed42-review.json`, `fill-staged-complete-requests-review.json` | Frozen held-out real-session/adversarial corpus through `MLX_BUN_TEST_FILL_MODEL=<dir> MLX_BUN_FILL=strict bun test tests/parity/fill-strict.test.ts` on both quants (token IDs, stop decisions, subsequent logits/state), then a quiet six-pair HTTP repeat. | A Qwen tool-template or parser change, tool schemas with pattern properties, or a chunk-limit qualification on another GPU. |
+| R16 | MLX graph and loop | **Earlier output adopted.** Early first token in the serial loop: six native pairs improve first-token latency at 6/128/512 tokens with exact tokens/state/usage; six chat HTTP pairs (84 requests, 42 exact) improve first visible output with complete time flat; raw prompts show no client gain because token 220 emits no event. The suffix screen finds a prefill-shape effect after a one-token return (seven packed/six RTN4 cases change state); explicit M=1 alignment restores exactness. The final caller-owned alignment passes both-machine state/continuation checks and monitored M4 serving comparisons; first visible output improves with complete time close. Closed: shape-specific compiled-call cache, Metal launch-config cache (MiniCPM HTTP mismatch unresolved), untouched-finalize elision (-0.069%), submission/traversal limits for Qwen (Llama 25/25 gives -2.44% native and -1.86..-3.02% HTTP, diagnostic, not integrated), fixed-input proxies, per-layer graphs (RTN4 fails identity), whole-model graph (slower), SSM-group compilation (-0.06%/-0.20%). Shapeless custom kernels are an upstream MLX limitation. | `MLX_BUN_EARLY_FIRST_TOKEN` defaults on; `=0` retains the old output order. | `first-yield-*`, `first-token-production-long-{serial,continuous}{,-review}.json`, `first-token-lifecycle-{llama,qwen}{,-review}.json`, `affine-compiled-call-cache-{full-model,http}-compact-review.json`, `metal-config-cache-*`, `mlx-untouched-finalize-source-manifest.json`, `untouched-finalize-*`, `graph-schedule-screen-plan.json`, `graph-schedule-screen{,-review,-contamination}.json`, `graph-schedule-rtn4-rescreen-v2{,-review}.json`, `graph-schedule-llama-repeat{,-review}.json`, `graph-schedule-llama-http-repeat{,-review}.json`, `mlx-upgrade-0.32.2/custom-kernel-compile-probe.json`, `mlx-upgrade-0.32.2/fixed-custom-probe.json`, `mlx-upgrade-0.32.2/qwen-fixed-region-{r6,rtn4}-v{2,3,4}.json`, `mlx-upgrade-0.32.2/qwen-fixed-layer-{r6-full-v2,rtn4-full}.json`, `mlx-upgrade-0.32.2/compiled-region-probe.json`, `mlx-upgrade-0.32.2/qwen-fixed-hidden-regions-{r6,rtn4}.json` | Early-output native, HTTP, cancellation/overlap and cached/SSD acceptance is complete; see benchmarks.md for monitored repeats and activity limits. The MLX 0.32.2 submission screen is complete: retain upstream startup defaults, with the measured M4 Llama override available explicitly. The current native API has no model-scoped setter; revisit integration if that interface becomes available. See benchmarks.md for the cross-model results and activity records. | MLX adds shape-inferring CustomKernel or graph replay, or per-token host-overhead attribution changes. |
+| R17 | Hybrid prefix and recurrent state reuse | **Implemented and accepted for generated reuse.** One cache owns immutable target/recurrent state and draft companions, original generated token IDs, session/content lookup, RAM residency and queued SSD persistence. Native and HTTP checks cover continued mutation, cancellation, earlier-boundary fallback and fresh-process restoration on both Macs. The saved Kanban miss/OOM sequence passes with identical output; the completed session-cache Kanban run hits every follow-up and flushes all snapshots durably. Its app-quality defects remain separate from cache acceptance. | Shared RAM/SSD publication and generated-token provenance are enabled; whole-file SSD and LRU remain selected after measured alternatives. | `reports/kanban-cache-fixed-fresh/`, `reports/kanban-cache-fixed-repeat-r1/`, `reports/kanban-session-cache-r2/`; [cache implementation](kv-cache.md#54-background-persistence-and-ram-residency) and [measured task comparison](../reference/benchmarks.md#full-kanban-with-session-cache-and-queued-persistence). | No missing generated-history implementation. Reuse the completed native/HTTP and full-task evidence; do not rerun unchanged gates. | A reproduced history-alignment, retention, durability or reuse regression. |
+| R18 | Scheduler | **Early output and shared batching adopted.** Continuous-scheduler early yield (same `MLX_BUN_EARLY_FIRST_TOKEN` flag): six AB/BA blocks, 84 responses, 42 exact, median TTFT -7.66% code / -7.90% explanation / -5.63% JSON (74–77 ms) with complete time +0.04..+0.15%; the integrated one-block check gives -68..-90 ms; 120 streaming cancellation/arrival requests pass on both quants and lanes. Concurrency evidence for kernels: v6 vs v12 four-request waves improve every pair and the tile pass improves eight-request throughput (R1). The historical four-request kernel waves predate shared cohorts. B-wide prefill is now adopted, and current default serving uses shared ordinary/speculative execution; mixed iteration work is measured and remains off. Two opposite-order fixed-arrival repeated-wave comparisons now preserve all 84 paired outputs and improve aggregate throughput at loads two/four/eight. First-request completion and stream gaps retain a tradeoff; see benchmarks.md. Longer steady-state capacity remains separate. | Early first output defaults on; `MLX_BUN_EARLY_FIRST_TOKEN=0` retains the control. | `first-prepare-scheduler-{control,candidate}.txt`, `first-prepare-http-screen-plan.json`, `first-prepare-http-{screen,repeat}{,-review}.json`, `first-prepare-production-{unit,model-free,typecheck,hygiene}.txt`, `first-prepare-production-http-r6-v2{,-review}.json`, `first-token-lifecycle-{llama,qwen}{,-review}.json`, `trellis-concurrent-http{,-review}.json`, `trellis-tile-http-review.json`, `trellis-scatter-integrated-cohort-v2{,-review}.json` | The fixed-arrival repeated-wave cell is complete with preserved outputs and documented latency tradeoffs. Longer stationary capacity remains a distinct measurement; do not repeat the accepted wave test without a relevant source change. | A change to admission, mixed-work scheduling or output/cache boundaries. |
+| R19 | Weight-quality allocation | **MLP cost/allocation screen complete; quality acceptance open.** Actual k2/k3/k4 fused gate/up and down costs are measured at M1/M3/M4/M16 on both Macs in forward/reverse order. All 228 unchanged-reader checks match. A discrete size-constrained objective runs 64 machine/row/latency-weight cells; zero weight reproduces k300 exactly. Modest latency weight finds little estimated decode headroom. Aggressive M1-prefill proposals trade more external-codec sensitivity loss for time. Those losses are ranking proxies, not Trellis quality. No candidate is quantized or promoted. Existing Q3 and lossless-interleaving quality evidence remains unchanged. | Keep the existing artifact. | `reports/prefill-observation/r19-cost/`; benchmarks.md latency-aware MLP allocation screen | Teacher-based quantization and frozen KL/MMLU/tGSM/rawGSM screens; non-MLP allocation and calibration/rounding choices remain open. Checked primary model directories lack a complete original Qwen bf16 snapshot; promoted bits cannot be recovered from k300. | No measured full-model speed or quality claim from the proposal cost model. |
+| R20 | MTP economics | **Repeatable win (opt-in).** Six balanced three-arm blocks versus native full-prefill on the packed artifact: fixed two drafts change complete time -43.22%/-38.81%/-44.92% (code/explanation/JSON) with every pair improving, TTFT -5.5..-8.4%, peak RSS +653 MiB; RTN4 -34.95%/-26.95%/-38.48%. Adaptive one-to-three drafts adds about 3% (ratios 0.9698/0.9782/0.9624) and stays research-only. Folded 4-bit/8-bit MTP heads: -1.48/-3.67/-2.95% and -0.65/-2.29/-1.86% with peak RSS 12.53/12.74 vs 13.04 GB; 1,184 vocabulary rows and 48 state samples byte-exact against serial replay. EOS and finish-reason accounting fixed. The quiet M1 Max control (0.821x at gamma 2, 2026-08-18) predates the kernel work. Shared serving has short depth-2/3/4 comparisons, native depth-3 coverage and completed late-context MTP1/2/3 controls. Current KV4/MTP2 and fused K8V3/MTP3 both pass the 78,678-token cached/SSD workload. KV4/MTP2 remains the measured long-agent recommendation; depth 3 crosses a different KV4 value-product dispatch boundary. See benchmarks.md. | `--draft-kind mtp --num-draft-tokens 2` opt-in; EOS/finish-reason fixes landed. | `mtp-v*-gamma*.json`, `mtp-corpus-screen/`, `mtp-screen-prompts.json`, `mtp-heldout-{plan,http,http-review}.json`, `mtp-heldout-native{,-review}.json`, `spec-token-counts-*.txt`, `mtp-post-eos-http{,-review}.json`, `spec-eos-serving-*.txt`, `mtp-eos-budget{,-integrated}-http{,-review}.json`, `spec-eos-continuous-*.txt`, `eos-budget-continuous{,-v2}-http{,-review}.json`, `mtp-quantized-draft-*-http{,-review}.json`, `mtp-draft-artifact-q4-{build,audit}.json`, `mtp-draft-quant-oracle.json`, `mtp-draft-artifact-q8-build.json`, `mtp-quantized-loader-*.txt`, `mtp-artifact-ownership{,-v2}.json`, `mtp-artifact-{screen,repeat}-http{,-review}.json`, `mtp-artifact-native{,-context}{,-review}.json`, `mtp-artifact-context-screen-http.json`, `mtp-matched-prefill-{r6,rtn4}-{screen-http,repeat}{,-review}.json`, `mtp-rtn4-screen-{plan,http,http-review}.json`, `mtp-rtn4-divergence-v2-audit{,-review}.json`, `mtp-rtn4-prefill-control{,-review}.json`, `mtp-v10-{planets,prose}-gamma{2,3}.json`, `mtp-provider-dispose-{before,after}.json`, `mtp-provider-ownership-tests.txt`, `mtp-post-dispose-http.json` | Late-context KV4 depths 1/2/3 and ordinary controls are complete; the current ordinary cached-repeat failure is resolved. TQ/MTP3 passes the original context, RAM repeat and SSD restart. Current-source KV4/MTP2 and automatic prefill sizing pass; KV4/MTP2 remains the measured agent recommendation. | Any change to verify-window arithmetic (KV4 head pairing, fused KV) or a DFlash2 drafter (R21). |
+| R21 | Other exact speculative sources | **Shared lookup implemented; DFlash2 unimplemented.** Qwen DFlash2 (the z-lab artifact behind the oMLX record the video cites; block size five, native MTP off) needs dynamic grouped causal convolutions around attention and MLP plus a top-16/rank-256 selector; the existing DFlash provider does not implement that architecture, so loading the checkpoint would not be DFlash2. Grouped greedy verify readback is closed: six three-arm blocks, 84 exact comparisons, concatenation improves only 0.33%/0.06%/0.23% and the evalAll variant regresses JSON 0.44%. Prompt lookup is now adopted through the shared variable-proposal verifier, with both-machine bf16/KV4/KV8/TQ B4 serving, generated RAM/SSD and MTP regression checks. M4 arm orders show small single-request gains and lower aggregate throughput with changed concurrent outputs; it remains optional. The DSpark/DFlash execution provider has seeded-fixture coverage, but no trained checkpoint exists. This does not implement DFlash2. SpecPrefill remains a separate approximate Lab arm. | Shared prompt lookup is available; trained-drafter performance remains unmeasured. | `video-dHK90xc9Q64/`, `video-dHK90xc9Q64/dflash-source-manifest.json`, `spec-grouped-greedy-screen.json`, `spec-grouped-greedy-eval-screen.json`, `spec-grouped-greedy-three-http{,-review}.json` | Missing resource: a DFlash2 architecture/selector port with target-state/rollback identity and draft residency, then a three-arm block (ordinary / fixed-two MTP / DFlash2) on the held-out prompts. | The port exists, or an exact n-gram source shows at least 3% on the saved Kanban history. |
+| R22 | Residency and shared memory | **mmap loss; range-construction improvement.** Page-aligned file-backed weights through Metal roots are exact in every payload/output case, but six native pairs give +0.25% (later +0.02%) on 64-token generation with no faster pair, six serial HTTP pairs +0.04..+0.14% complete and +0.14..+0.21% decode, and the balanced cold-start follow-up is worse in five of six pairs (+18.42% first token, +12.93% first generation). Peak RSS falls from 12,182,749,184 to 482,508,800 bytes as file-backed accounting only; physical RAM saving unproven. A read-only donation hazard was found and fixed in the prototype before closing; helpers removed. Retained as memory correctness, not speed: the 8 MiB bounded range cache in `mlx/ops.ts` (Kanban sustained run peak active -1,582,612,480 bytes, +2.171% time in one pair) and per-chunk draft-KV evaluation. | Native long position ranges reduce operator/host-allocation work without a material model/serving speed claim; bounded range cache and per-chunk draft-KV evaluation remain memory fixes. | `mmap-page-window{,-pointer}-screen.json`, `mmap-weights-native-{llama1,packed}-screen.json`, `mmap-weights-native-packed-paired{,-review}.json`, `mmap-weights-http-serial{,-review}.json`, `mmap-weights-ownership-packed{,-mapping}.json`, `mmap-weights-single-consumer-lazy{,-mapping}.json`, `mmap-weights-whole-consumer-lazy.json`, `mmap-whole-consumer-source-integrity.json`, `mmap-donation-{ro,cow}-screen.json`, `mmap-managed-donation-screen.json`, `mmap-weights-managed-whole-consumer-lazy{,-mapping}.json`, `mmap-managed-submitted.json`, `mmap-weights-managed-native-{repeat,screen}{,-review}.json`, `mmap-initial-residency-attribution.json`, `mmap-lazy-view-ownership.json`, `mmap-weights-lazy-managed-native-screen{,-review}.json`, `mmap-prefault-native-screen.json`, `mmap-cold-native-repeat{,-review}.json`, `mmap-managed-shard-release{,-mapping}.json`, `mmap-managed-active-accounting-review.json`, `kanban-arange-{allocation-evidence,regression,micro-memory,reconstruction}.json`, `kanban-sustained-77k-owned-control.json`, `kanban-sustained-arange-comparison.json` | Native long int32 positions are accepted as an operator/host-allocation optimization after exact range/view/bound checks, six monitored M4 complete-forward pairs and two standard serving arm orders. Model and serving performance are effectively flat; the initial unmonitored regression is not reproduced. A reusable power-of-two prefix remains unselected after an inconsistent M1 screen. See benchmarks.md for activity records and limits. | A controlled page-cache-cold startup cell (§7.10) exists; mmap lost warm but cold was never controlled. |
+| R23 | Media and task completion | **Inconclusive.** The fresh Pi Kanban task on the 12.14 GiB packed target with Luke's `qwen3.8-27b-q3_k_xl-coding-128k` profile (131072 context, xhigh thinking, temperature 0.6, seed 42; KV4, MTP depth 2, paired prefixes, MLX 0.32.2) completed in 3h 33m 7.685s with 130,494 output tokens over 62 requests, 79.706% draft acceptance, 11.916 GiB peak combined RSS; the untouched app passes the functional browser checks. Earlier attempts failed on a tool-parser bug, dropped reasoning history, Metal OOM at the 85,238-token request and OOM during sustained decode. The later accepted R17 session-cache Kanban run completes in 83m57s with 76,031 output tokens, every follow-up cached and durable final persistence; its two application defects remain recorded. The automatic-prefill task misses the app-quality target, while the completed fixed-256 integrated control meets it and reproduces the earlier initial response. Keep the task-comparison profile fixed; results and browser evidence are in benchmarks.md. No matched successful original-source run exists, so these different trajectories cannot supply an engine-only task-time speedup. Gemma image/audio shared execution now passes both-machine native/HTTP and monitored M4 ABBA comparisons with all 78 paired responses/counts unchanged; concurrent gains and individual latency costs are recorded in benchmarks.md. Qwen image/video shared execution also passes both-machine source/group/HTTP and M4 ABBA checks; full usage matches, while changed concurrent geometry changes some text. Qwen encoder objects now reuse the shared RAM/SSD cache, with both-machine native/HTTP, fresh-process SSD and M4 source-controlled timing acceptance. First-use weight hashing costs more; repeated images improve with exact responses/usage. Gemma encoder reuse also passes both-machine identity/HTTP and M4 ABBA/fresh-process SSD acceptance, with all 168 paired responses/usage exact and repeated-request gains. Gemma and Qwen generated media KV reuse now pass both-machine native/HTTP, fresh-process SSD and M4 ABBA comparisons. The shared media-prefix option is selected by default; output and precision boundaries remain documented. | Tool-parameter parser and `reasoning_content` alias landed as correctness fixes; no speed default. | `lukes-kanban/`, `kanban-final/optimized-mtp-invalid-32k-sleep/`, `kanban-final/luke-q3-128k-xhigh-{history-fixed,tool-values-fixed,mtp-memory-fixed,arange-bounded}/`, `kanban-final/comparison.html`, `kanban-capacity-110k-control.json`, `kanban-capacity-8k-final-comparison.json`, `kanban-capacity-110k-owned-final.json`, `kanban-memory-http-{owned,responsiveness}.json`; numbers in benchmarks.md Kanban sections | The requested integrated fixed-profile task is complete. An original-source run is needed only for an engine-only task-time claim; no such ratio is claimed. Gemma image/audio shared serving acceptance is complete. Qwen image/video shared serving acceptance is complete with documented geometry and latency limits. Qwen encoder-cache acceptance is complete. Gemma encoder-cache acceptance is complete. Prepared Gemma/Qwen media-prefix reuse is accepted through the same cache interfaces. Chunked media prefill and wider configurations remain separate work. | No engine-only task-time ratio without a matched baseline. R17 generated-history acceptance is complete and does not need another unchanged task run. |
+| R24 | Larger departures | **Inconclusive.** Feasibility and operation screens exist. For the video's tq3-mini codebook format (eight-centroid indices, ten codes per uint32, 3.2 bits/value), the literal upstream `polar_qmv` through the Bun binding on synthetic matrices: explicit SIMD shuffles cut the 5120-to-17408 rotated operation 2.77–3.84% (every pair), and a fused fp16 expansion cuts large-prefill operation time 40.85–57.51% with about 765 MiB less temporary allocation. No production loader or kernel, no quality evidence; the checkpoint's reported agent failures are unresolved. Section 7.14 now records kernel, quality, memory and potential-benefit requirements for those candidates. A both-machine same-weight native activation-quantization screen is a measured loss; static size/training bounds are recorded. Actual-weight unweighted low-rank and sparse-residual screens now reject their simple variants. Packed forwards fail differentiation without a custom VJP; the expanded frozen-weight control succeeds. Calibrated quality and packed backwards remain open. | None. | `video-dHK90xc9Q64/`, `polar-qmv-screen-v2.json`, `polar-qmv-variants.json`, `polar-qmv-geometry.json`, `polar-qmv-repeat{,-review}.json`, `polar-expand.json`, `polar-expand-repeat{,-review}.json` | Next: activation-calibrated candidates and an explicit packed backward operation, if pursued after the measured simple-variant losses. The tq3-mini format still needs a loader and actual-checkpoint KL/MMLU/tGSM/rawGSM before a model claim. | An artifact with quality evidence at or below Q2b bytes, or the operation gains reproduced whole-model. |
 
 Josh's original direction is to "only ask the model to decode tokens that we
 can't reasonably already know." This includes reasoning snippets copied into
@@ -2275,33 +2307,29 @@ prefix. Both comparisons retokenize decoded reasoning; the SSE has no original
 sampled IDs, so the execution-state gate must record those IDs directly.
 Evidence: `reports/qwen38-closeout/r17-repo-prefix-audit.json`.
 
-#### Candidate defaults and remaining gates
-
-Defaults and semantics are owned by
-[server-config.md](../reference/server-config.md); this table lists only what
-still gates each campaign flag. The three default selections below are in the
-unreleased working tree; selection checks and final composed acceptance remain
-open. R17 output persistence and unified execution are not implemented by
-changing defaults. Phase 6 tracks those separately.
-
-| Flag | Default | Remaining gates |
-|---|---|---|
-| `MLX_BUN_TRELLIS` | `kernel` (`=expand` decodes trellis tensors at load into 8-bit g64 affine) | Production; `expand` is the fallback carrier for a machine where the kernels lose. No open gate. |
-| `MLX_BUN_TRELLIS_VARIANT` | `13` in candidate source | Six serial, six continuous and six combined KV4/MTP pairs pass with exact paired responses. Final-source native state gates pass. Open: default-selection regression checks and remaining composed/pressure acceptance. Other experimental variants retain their individual gates. |
-| `MLX_BUN_TRELLIS_ASYNC_EXPAND` | off (`=1`) | Variant 13 only. Open: broader callers/models, combined-optimization arm, quiet-machine acceptance; pressure gates have one pair per lane. |
-| `MLX_BUN_EARLY_FIRST_TOKEN` | off (`=1`) | Native caller-owned early-return alignment passes the initial M1 regression gates. Open: M4 packed/affine and server acceptance, cached/SSD pressure HTTP with streaming, quiet-machine repeat; raw prompts show no client gain. |
-| `MLX_BUN_TURBOQUANT_FUSED_DECODE` | off (`=1`; read via `process.env` when each TurboQuant cache is created, `src/model/gemma4-base.ts`) | Open: combined settings with MTP/KV4, pressure, quiet-machine acceptance; RTN4 complete-time cell inconclusive; serial-only because TurboQuant KV refuses continuous placement. |
-| `MLX_BUN_QWEN_SPEC_KV4` | on in candidate source, with configured draft and `--kv-quant 4` | Six combined M4 pairs and two completed tasks pass. Open: default-selection checks, broader composition/pressure; long tool-turn prefill memory/timing (unfused quantized attention materializes query-by-context scores); the three-query verify-window head-pairing screen is unmeasured. |
-| `MLX_BUN_MTP_PROMPT_CACHE` | on in candidate source | Open: balanced timing pairs for the shared cache, generated-boundary checkpoints and pressure acceptance; shared byte accounting and bf16/KV4 RAM/SSD restart correctness now pass. |
-| `MLX_BUN_FILL` | off (`=strict` \| `=echo`) | Open: held-out session/adversarial corpus, other models, quantized-KV and speculative composition (guarded), quiet-machine repeat; echo tier remains Lab. |
-| `MLX_BUN_FILL_APPEND_CHUNK_SIZE` | `0` (model limit; Qwen 27B on `applegpu_g16s` up to 4 positions, other GPUs 1) | Open: held-out, broader-model, other-GPU qualification, quiet acceptance. |
-| `MLX_BUN_FILL_MAX_SPAN` | `32` (floor 2) | Cap policy undecided: cap 4 restores exact state in the v12 audit and both caps improve request time; span policy must follow the selected variant's measured append cost. |
-| `MLX_BUN_FILL_TRACE` | off (`=1` \| `=<file.jsonl>`) | Diagnostic only; must be unset for any timing. |
-| `MLX_BUN_FILL_K` / `MLX_BUN_FILL_CANDIDATES` / `MLX_BUN_FILL_INDEX_MAX` | `8` (floor 2) / `24` / `131072` (floor 1024) | Echo tier (Lab, pre-campaign); paired A/B on task success and wall clock before any default. |
-| `MLX_BUN_RD_PREFILL_CHUNK` | `2048` unless the library caller supplies `prefillChunkSize` (Kanban/benchmark recipe uses 256) | Benchmark control, not a performance candidate; repeated-prefill cost at 256 vs 2048 on the saved Kanban requests is open. |
-| `MLX_BUN_RD_CONTEXT_LIMIT` | unset (explicit context cap; may exceed the advisory fit estimate) | Benchmark control only. |
-| `MLX_BUN_TOKEN_MASK` | `host` (`=metal`) | Open: controlled quiet B1/B4 structured-output session (six compact pairs inconclusive under residual swap). |
-| `MLX_BUN_PREFILL_TAIL_SPLIT` | on (`=0` reverts to full-chunk prefill) | Pre-campaign default; isolated as the cause of RTN4 native-vs-MTP trajectory differences; changing it needs its own oracle/quality gates. |
+#### Shipped defaults and remaining experiment gates
+
+[Server configuration](../reference/server-config.md) owns effective defaults,
+precedence and supported combinations. Trellis v13, eligible KV4 speculation
+and paired MTP prompt reuse shipped in v0.4.0. Shared batching is the default;
+R17 generated RAM/SSD persistence and long-agent setting selection are accepted.
+Serial removal and exhaustive feature migration are deferred by the current
+milestone. Remaining experiments do not reopen those completed gates.
+
+| Control | Accepted evidence | Remaining work |
+| --- | --- | --- |
+| `MLX_BUN_TRELLIS`, `MLX_BUN_TRELLIS_VARIANT` | Shipped selection; six serial, six continuous and six combined KV4/MTP pairs, plus native state gates. | Additional experimental variants retain their own numerical/performance requirements. |
+| `MLX_BUN_TRELLIS_ASYNC_EXPAND` | Native, repeated short serving and saved long-agent pressure checks pass, including fused k8v3/MTP3 on M4 and M1. | Keep opt-in for the measured small request-time gain and higher live allocation. Combined pressure is accepted; reopen only for new models/barriers or a better tradeoff. |
+| `MLX_BUN_EARLY_FIRST_TOKEN` | Streaming/cancellation/overlap and cached/SSD gates pass. Caller-owned M=1 alignment preserves state on M1 and M4. Monitored paired chat HTTP checks support earlier visible output; default on. | No open early-output acceptance gate. Invisible token zero need not improve raw-prompt latency; no decode-throughput gain claimed. |
+| `MLX_BUN_TURBOQUANT_FUSED_DECODE` | Adopted on by default for an explicitly selected TQ cache after native/state, repeated codec serving, long cached/SSD and combined pressure gates, plus current M1/M4 shared serving. | Explicit off control remains; distinguish codec-kernel gains from quantization-setting comparisons. |
+| `MLX_BUN_QWEN_SPEC_KV4` | Combined M4 pairs, completed tasks and the current 78,678-token cached/SSD workload pass. KV4/MTP2 remains the measured long-agent choice. | Three-query key-head grouping is integrated for the qualified M4 shape after operation, full-state and matched serving gates. Other geometries retain the existing dispatch. |
+| `MLX_BUN_MTP_PROMPT_CACHE` | Shared byte accounting, generated-boundary checkpoints, RAM/SSD restart and full Kanban session retention pass. | No open R17 acceptance gate; new methods retain their own state contracts. |
+| `MLX_BUN_FILL`, append chunk/span controls | Qualified Qwen committed appends preserve bf16/KV4/KV8/K8V3 state; balanced HTTP and saved tool-history fixture comparisons pass. | Held-out task quality, shared direct append, speculative/quantized verification and other model/GPU qualifications remain distinct cells. |
+| Fill trace and echo-index controls | Trace is diagnostic; echo is a separate Lab algorithm. | Disable trace for timing; measure echo task quality and whole-request cost before promotion. |
+| `MLX_BUN_RD_PREFILL_CHUNK` | Captured policy selects work size; explicit overrides remain. Automatic long hybrid prefill matches all 19 fixed-size responses through cached/SSD restart. | The completed fixed-256 integrated Kanban control reproduces the earlier first response and meets the app-quality target. Retain that task-comparison setting. The general automatic policy remains selected; one changed trajectory does not establish a general quality loss. |
+| `MLX_BUN_RD_CONTEXT_LIMIT` | Explicit caller constraint; fit estimates remain advisory. | Benchmark control, not a speed candidate. |
+| `MLX_BUN_TOKEN_MASK` | Exact structured-output responses; previous serving timing is inconclusive. | Controlled B1/B4 comparison if the mask implementation changes. |
+| `MLX_BUN_PREFILL_TAIL_SPLIT` | Existing policy explains recorded RTN4 ordinary/MTP trajectory differences. | Changing that policy requires a separate numerical/quality comparison. |
 
 #### Closed candidates
 
@@ -2332,7 +2360,7 @@ condition.
 - Multimodal RoPE shared angles/direct strides: M128 slightly slower, M512 flat; closed.
 - Metal launch-configuration cache: Qwen flat, MiniCPM fails the serving response gate (cause unresolved); closed.
 - Untouched-finalize elision in the Metal backend: -0.069%/+0.108%; closed without integration.
-- MLX submission/traversal/sync settings for Qwen: no compelling setting (0.94% traversal gain matched the 0.92% default drift); Llama 25/25 remains diagnostic, unintegrated.
+- MLX submission/traversal/sync settings for Qwen: no compelling setting (0.94% traversal gain matched the 0.92% default drift); Llama 25/25 remains an explicit process-start override after current-version cross-model and both-machine controls; no universal default is adopted. See benchmarks.md.
 - `MLX_METAL_FAST_SYNCH=1`: stalls the installed library's large bf16 three-stream case; not attributed to any prototype.
 - Top-p inverse-permutation removal (pure MLX and fused Metal): Qwen +0.183% complete; Llama -0.4%; no production sampler change.
 - Grouped greedy verify readback (concatenated and evalAll): ≤0.33% gain, JSON regression; closed.
@@ -2342,8 +2370,35 @@ condition.
 - KV append whole-cache-copy hypothesis: rejected by native/HTTP census (one-row copies); no append kernel.
 - mmap file-backed weight loader (immutable, managed, lazy, prefault variants): warm flat-to-slower, cold worse in 5/6; closed as a performance candidate.
 - Staged-body strict-fill prototype (skip the discarded in-flight sample): flat; closed.
-- Strict fill with affine KV4: fails recurrent state at layer 4; composition guard kept.
+- Multi-query affine attention inside strict append: recurrent state diverges at layer 4. Superseded by the qualified one-token attention calculation within shared projection appends; KV4/KV8 now pass state and serving gates.
 - Cross-entropy block-alignment partition: no wired model reaches a misaligned partition; closed without integration.
 - Streamed-expert threadgroup geometry: bf16 timing bimodal in all three library arms; no geometry default retained.
 - Proc-pid-rusage FFI collector: 0.0013 ms vs 1.09 ms per read, collector only; existing reports keep the ps collector.
 - Video tq3-mini polar format: operation wins on synthetic matrices; no loader, kernel or quality evidence; not adopted.
+
+
+### 7.14 Larger-departure feasibility decisions
+
+R24 now has an explicit initial screen. Resource calculations and the
+both-machine activation operation comparison live in
+[benchmarks](../reference/benchmarks.md#r24-activation-quantization-and-resource-bounds-2026-09-13).
+The following decisions apply to the current packed Qwen artifact and native
+pack. They do not close unmeasured candidate quality or claim the whole campaign
+is exhausted.
+
+| Candidate | Kernel and quality requirement | Training/calibration and next decision |
+|---|---|---|
+| Learned/codebook quantization | [AQLM](https://arxiv.org/abs/2401.06118) learns additive codebooks. Existing Trellis dispatch and the archived polar operation screen do not establish a loader or efficient decoder for that representation. New codes must pass the frozen quality suite. | Fit one matrix/block with calibration under the calculated memory budget before any whole-model conversion. A complete original bf16 Qwen snapshot was not found in the checked model directories; fitting the current packed reconstruction must be labeled as an additional approximation. |
+| Sparse residual/outlier correction | [SpQR](https://arxiv.org/abs/2306.03078) separates exceptional weights from the low-bit representation. The sparse correction must fit inside savings from a lower-bit base and fuse with the target's matrix operation. | Bounded matrix reconstruction is feasible in principle; measure error versus bytes and sparse gather/reduction time first. Do not install a second dense correction matrix or assume CUDA speed transfers to Metal. Original-weight recovery needs original weights. |
+| Activation quantization | [SmoothQuant](https://arxiv.org/abs/2211.10438) motivates jointly quantized weights/activations. The pinned native QQMM rounds then expands activations. The actual same-weight operation screen loses, especially at prefill sizes. | Current QQMM is rejected as a drop-in speed option. A direct compressed-activation kernel remains a separate implementation and quality experiment; INT8 input alone does not supply it. |
+| Structured pruning | Zero values in the existing packed format still consume codes and matrix work. A useful artifact must physically remove aligned channels/heads and update model dimensions. | Rank channels with calibration, then test one width-reduced MLP before a model conversion. Preserve matching gate/up selections and down input columns. No unstructured sparsity speedup is assumed. |
+| Layer skipping | [ShortGPT](https://arxiv.org/abs/2403.03853) ranks layer influence. A static reduced-layer artifact can remove work, but Qwen's recurrent/attention mix and cache topology must remain explicit. Dynamic token-dependent skips change recurrent evolution. | Start with static layer influence and held-out quality. Removing eight of 64 equally costly blocks would remove one eighth of block work only under that simplifying assumption; use measured layer costs before predicting time. No serving skip flag is selected. |
+| Low-rank approximation | [SVD-LLM](https://arxiv.org/abs/2403.07378) combines activation-aware factorization and correction. Two dense factor multiplies can use existing operations, but a loader/linear binding and compression-quality evidence are still needed. | The rank/byte screen rules out high-rank bf16 factors as a size win against W3. The actual layer-one gate spectrum rejects plain unweighted SVD at a size-feasible rank. Activation-aware error and task quality remain separate work. |
+| QAT/distillation | [QLoRA](https://arxiv.org/abs/2305.14314) shows why freezing a quantized base changes training memory. Full-resident 27B optimization does not fit either machine. `Qwen35Model.loraTargets()` explicitly excludes Trellis MLPs; existing adapter training does not establish packed-MLP recovery. | Layerwise QAT or adapter recovery needs a differentiable packed linear binding and measured activations. Stored teacher targets can separate teacher generation from student training. The quality gain and training duration remain unmeasured. |
+| Alternative drafters | Existing grouped providers already separate proposals from verification. A trained drafter must improve accepted tokens per total target/draft time and respect target memory needs. | No trained DSpark/DFlash checkpoint exists. Keep their untrained correctness checks distinct from performance acceptance. R20's measured native-MTP depths remain the available comparison. The specific DFlash2 architecture is separate open work. |
+
+Actual-weight screens now reject the simple unweighted low-rank and sparse
+variants. Differentiation confirms the need for a custom packed backward
+operation; expanded frozen weights already differentiate. Further work needs
+activation-calibrated candidates or that backward binding. No new deployment default,
+model-quality claim or training run follows from the paper results alone.