mlx-bun 0.3.0 → 0.5.0

package/docs/design/decode-speed-program.md

@@ -3,7 +3,7 @@ status: active
 axis: ON
 canonical-for: decode-speed-levers
 plan-anchor: "Phase 6 — Speed: change what gets computed `[~]`"
-last-verified: 2026-09-07
+last-verified: 2026-09-08
 ---
 
 # Inference performance program
@@ -12,9 +12,11 @@ This is the canonical performance research plan. The current priority is the
 Qwen3.8-27B program in section 7, covering quants, the complete execution graph,
 custom kernels, deterministic token replay, prefill, decode, and serving.
 Sections 1–6 retain the earlier cross-model research and the
-**port ledger vs oMLX** (§6, folded in from `docs/design/decode-speed-program.md` on
-2026-08-23). Status and run logs live in PLAN.md (Phase 6 and the serve
-h2h phases); the architecture the levers plug into is
+**port ledger vs oMLX** (§6, folded in from `docs/design/omlx-adoption-map.md`
+in `2b571ff`, 2026-08-24). Open status lives in PLAN.md (Phase 6); the
+campaign run log is
+[qwen38-27b-campaign-log.md](../archive/investigations/qwen38-27b-campaign-log.md)
+and its outcome ledger is §7.13; the architecture the levers plug into is
 `unified-engine-frontier-plan.md`; curated numbers are
 `docs/reference/benchmarks.md`. Every lever here has evidence, an expected
 win, and a trigger; pick them up in order. No number below is new — each
@@ -27,8 +29,8 @@ decode tok/s ≈ bandwidth ÷ bytes-touched-per-token. The earlier mode-matrix b
 (`scripts/bench-matrix.ts modes`) confirmed what the roofline work predicted:
 the then-tested L1 decode paths approached the memory-bandwidth wall
 at short context, so tier levers cannot buy single-stream decode speed
-(PLAN.md "Decision: naked default = --l1": decode parity with mlx-lm on every
-model; no output-changing lever beat it). Faster therefore means one of five
+(naked default = `--l1`, decided 2026-07-05, `git show 3199c75:PLAN-archive.md`:
+decode parity with mlx-lm on every model; no output-changing lever beat it). Faster therefore means one of five
 physical moves:
 
 1. read the weight bytes ONCE for several tokens — speculation, batching;
@@ -96,14 +98,15 @@ What remains is drafters worth running, in this order:
   exist in `src/`. The measured motivation stands: acceptance dropped 51%→29%
   under grammar with a blind drafter (our own matrix). (iii) grammar-pruned
   draft TREES (XGrammar-2's `traverse_draft_tree`) — idea only.
-- **Per-slot drafting under batching** — the composition-matrix row in the
-  architecture doc; a mounted draft routes every request serial today.
+- **Drafting under batching** — grouped provider execution is implemented at
+  B1/B>1, with method-owned candidate generation, verification and companion
+  state. Remaining compositions and evidence live in [batching](batching.md#consolidation-matrix).
 
 ### Lever 2 — native MLX updates against the pinned control
 
-We are pinned to the oracle stack in PLAN.md "Reference environment" (mlx
-0.31.2). The July audit below identified upstream changes absent from that
-pin. Recheck current releases and the native libraries actually loaded before
+The runtime and oracle now use MLX 0.32.2 / native pack 0.4.0; see §7.12 and
+[environment](../reference/environment.md). The historical July audit below
+identified upstream changes absent from its then-current MLX 0.31.2 pin. Recheck current releases and the native libraries actually loaded before
 an upgrade experiment. A new mlx/mlx-c build also needs matching oracles;
 retain the pinned installation and test the new stack in isolation.
 Upstream refs are full URLs on purpose (a bare `#NNNN` here resolves to our
@@ -322,15 +325,12 @@ contracts; oMLX is a Python appliance ON mlx-lm:
 
 Josh requested a complete inference optimization campaign on 2026-09-04.
 Optimize useful completed work at a measured quality and memory budget.
-The deliverable is a frontier of configurations for the M4 Pro, with a reason
-to keep or reject every investigated method. A smaller artifact, a faster
+The deliverable is a frontier of configurations, with a reason
+to keep or reject every investigated method; every number records the
+host/chip/RAM it was measured on. A smaller artifact, a faster
 microkernel, and a higher draft acceptance rate are intermediate results.
 Each must survive the full application measurement.
 
-Josh narrowed the remaining hardware scope to the M4 Pro on 2026-09-06.
-M1 Max runs are not required for this campaign. Existing M1 Max evidence
-remains historical; the final report format still supports other machines.
-
 Single-request performance is a primary acceptance target alongside aggregate
 serving. Use the existing `scripts/bench-serve.ts all` workload, which replaced
 the root `benchmark.sh` in `7d81462`: sustained decode, cold and cached TTFT,
@@ -636,7 +636,7 @@ ledger. A candidate's priority changes when the measured graph changes.
 
 | ID | experiment | measure and deciding condition |
 |---|---|---|
-| R0 | Exact artifact and execution controls for performance | Freeze `673b43f` plus the integrated local diff; matched token IDs, active graph/method/quant, raw timings and actual memory. Use relevant numerical and lifetime gates; the wider refactor acceptance program is separate. Resolve the M4 delta golden cell before L1 promotion. |
+| R0 | Exact artifact and execution controls for performance | Freeze `673b43f` plus the integrated local diff; matched token IDs, active graph/method/quant, raw timings and actual memory. Use relevant numerical and lifetime gates; the wider refactor acceptance program is separate. The M4 DeltaNet golden cell is resolved (§7.1; environment.md). |
 | R1 | Packed prefill tile decode plus matmul | Decode a tile into registers/threadgroup storage and consume it with supported Metal matrix operations. Compare with bounded dense expansion. Win must lower TTFT without a larger peak live working set. |
 | R2 | Bounded expansion scheduling | Compare per-projection, per-layer and bounded groups of projections; include cache outputs in dependencies. Test async evaluation and explicit memory budgets. A removed wait is a win only if memory lifetime stays bounded. |
 | R3 | Axis-0 scatter layout and split-K | Sweep splits 16/32/64/128/256, lane/output ownership, row tiling, vector loads and split-reduction fusion. Preserve the rotated coding axis. Include partial-buffer traffic and final reduction. |
@@ -652,7 +652,7 @@ ledger. A candidate's priority changes when the measured graph changes.
 | R13 | Attention and KV | Stock SDPA dispatch, tiling, grouped-query reuse, fused KV append/quantize, direct attention over compressed KV, mask specialization, RoPE fusion and paged versus contiguous storage. Measure quality, TTFT and decode by depth. |
 | R14 | Vocabulary head and sampler | Last-position-only projection, exact fused matvec/argmax for eligible greedy requests, GPU logits processors, reduced readbacks and structured-output singleton runs. Head elimination must respect logprobs and sampling contracts. |
 | R15 | Deterministic no-verification replay | Follow section 7.5. Improve proven coverage and span execution economics; report wall time and output identity independently. This is a first-class track ahead of speculative drafter training. |
-| R16 | MLX graph and loop | Use model-owned graph/method bindings for compiled regions, stable buffers/shapes, graph replay, cache donation, allocation removal, bounded device-side chaining and host/GPU overlap. Measure session/lease/readback costs without adding per-op evals. Preserve cancellation and ownership gates. |
+| R16 | MLX graph and loop | Use model-owned graph/method bindings for compiled regions, stable buffers/shapes, graph replay, cache donation, allocation removal, bounded device-side chaining and host/GPU overlap. Measure session/lease/readback costs without adding per-op evals. Preserve cancellation and ownership gates. | |  |  | Early-output native, HTTP, cancellation/overlap and cached/SSD acceptance is complete; see benchmarks.md for monitored repeats and activity limits. The MLX 0.32.2 submission screen is complete: retain upstream startup defaults, with the measured M4 Llama override available explicitly. The current native API has no model-scoped setter; revisit integration if that interface becomes available. See benchmarks.md for the cross-model results and activity records.
 | R17 | Hybrid prefix and recurrent state reuse | Exact prefixes, semantic anchors, page/block storage, compact active-state updates and checkpoint cadence. Test mixed prompt edits and cancellation; no KV-only hit that loses the corresponding recurrent state. |
 | R18 | Scheduler | Solo latency, actual B-wide prefill, continuous batching, chunked-prefill/decode interleaving and shape buckets. Report per-request latency and aggregate throughput; one cannot substitute for the other. |
 | R19 | Weight-quality allocation | Measured latency-aware allocation across MLP, attention, GDN, embedding and head; GPTQ/LDLQ calibration and damping; QTIP/YAQA rounding; group size, k, L, block length, rotations and sensitive-layer exceptions. Optimize quality/bytes/time together. |
@@ -778,7 +778,7 @@ One model-owning process at a time on these Macs; restart paired server arms
 sequentially when both do not fit. Control cold-cache and warm-cache trials
 explicitly. Preserve all samples and interruptions. Do not keep retrying until
 a favorable result appears. Recheck residency, swap activity and thermal state
-between cells, and repeat finalists in a separate session on the M4 Pro.
+between cells, and repeat finalists in a separate quiet-machine session.
 
 Estimate paired log-ratio confidence intervals with sessions/prompts as the
 independent units. A default speed win should exceed 3% and have a 95% lower
@@ -814,6 +814,9 @@ the combined and held-out gates, and a final source review finds no untested
 applicable candidate with meaningful remaining headroom. Blocked or
 unmeasured rows do not count as exhausted. This is a reproducible stopping
 rule for a finite search, not a claim that no future algorithm can be faster.
+§7.13 is the outcome ledger this rule requires: one row per R0–R24 with its
+outcome, retained flags, evidence, next discriminating step and reopen
+condition.
 
 The scope also includes every owned kernel and native operation used by the
 project's other model paths. Maintain an inventory of dispatch sites, generated
@@ -823,8 +826,8 @@ layers, embeddings, normalization, activation, sampling, masks, cache codecs,
 media paths and streamed experts, including graph construction and host work.
 Each inventory row needs an explicit result; a Qwen-only trace cannot close
 an operation used elsewhere. Keep unavailable-model or hardware cells visible.
-Run the final comparison on the M4 Pro. Pin source, runtime/library
-versions, artifacts, request settings and
+Run the final comparison on a quiet machine and record its host/chip/RAM.
+Pin source, runtime/library versions, artifacts, request settings and
 cache state so kernel gains are not confused with different test conditions.
 
 The final task benchmark uses
@@ -892,7 +895,7 @@ differ across Luke's coding profiles. The selected section is
 The task uses that exact prompt and request recipe. Its first attempt was
 stopped after a serving bug discarded the preceding thinking on tool turns.
 
-The MTP-prefix integration, now included in the PR #47 feature branch, permits Qwen uniform KV4 with start=0
+The MTP-prefix integration (merged in PR #47) permits Qwen uniform KV4 with start=0
 behind `MLX_BUN_QWEN_SPEC_KV4`. Prefill converts attention caches through
 `maybeQuantizeKv`; rollback binds after conversion, and prefix identity includes
 KV bits, group size and conversion policy. Recurrent state remains full precision.
@@ -911,7 +914,7 @@ size for MLX prefill, bounding long-context attention workspace.
 ceiling to exactly 131072, never enlarges it. Pi uses the same context and
 xhigh effort. Responses run until EOS or remaining context, with no invented
 16K cap. Task timing includes automatic tool use and retries; app scoring
-follows after inference exits. The completed retry is recorded later in this section.
+follows after inference exits. The completed retry is recorded below.
 Raw evidence and invalid runs remain in `reports/qwen38-rd/kanban-final/`.
 
 The paired cache currently captures prompt checkpoints only. Extending it to
@@ -1016,9 +1019,9 @@ ranges across layers. Keep exact range/mask values, outstanding-view ownership
 and the memory bound as gates; measure complete prefill and verification rounds
 before retaining an allocation or dispatch optimization.
 
-The pinned MLX 0.32.2 source also gives a specific native scheduling experiment
+The pinned MLX 0.32.2 source supplied a native scheduling experiment
 for the three-query KV4 verification window. Qwen's 24 query heads share four
-KV heads (GQA6), with D256. `quantizedSdpaUnfused` currently presents the heads
+KV heads (GQA6), with D256. `quantizedSdpaUnfused` originally presented the heads
 as separate broadcast batches. In `mlx/backend/metal/quantized.cpp`, the
 M4-eligible affine key multiplication selects `qmv_wide`: three vectors per
 threadgroup, eight reduction lanes, and a separate batch for each query head.
@@ -1047,6 +1050,15 @@ causal string currently rebuilds the boolean array in each unfused attention
 call. Keep the bf16 and OptiQ dispatch policies intact and measure the retained
 mask's lifetime as well as complete-forward timing before retaining that change.
 
+The screen and full-model/HTTP comparisons are now complete. Three-head key
+grouping is selected on M4 Pro for the tested B1/B2, L3, GQA6/D256,
+KV4/group64, bf16/f32, N≥8192 geometry. Other geometry retains the existing
+native selection. M1 grouped key scores differ, so its key geometry remains
+unchanged. The permanent regression compares strided KV against the separate
+head composition. Exactness, complete-forward and serving measurements are
+recorded in benchmarks.md under Three-query affine attention head grouping.
+The shared causal-mask experiment remains separate, unmeasured work.
+
 Do not apply that fold indiscriminately to the value multiplication. Its native
 matrix threshold is four query rows, so the same reshape would change the
 algorithm. At long context it uses `qvm_split_k`, with 32 partitions beyond
@@ -1111,1127 +1123,58 @@ This is a source-derived scheduling hypothesis, not a measured speed claim;
 each window's native arithmetic, rollback, cache reuse and completed response
 remain correctness and acceptance gates.
 
-### 7.8 Execution progress and next measurements
-
-Use the integrated memory safeguards and relevant numerical controls in
-section 7.1, then measure the expensive computation. The existing benchmark
-accepts an exact local artifact and can print its
-commands without loading a model. The first prepared jobs are the affine
-flagship on mlx-bun serial versus mlx-lm, followed by packed Q3 on mlx-bun
-serial. Start at 4k context; widen only after the measured memory row.
-The commands and preparation mode are documented in
-[benchmarks.md](../reference/benchmarks.md#running-the-benchmark).
-
-The harness now keeps all five decode samples, uses reproducible nonces with
-`--workload-seed`, pins thinking mode across arms, and saves request bodies,
-counts, finish reasons, errors and wall times in a JSON companion. It rejects
-missing token usage instead of counting SSE chunks as tokens. Each block
-still needs an explicit AB/BA order and a distinct output path; one harness
-pass is a screening block, not the full paired-session confidence interval.
-Identical decoded text and prompt counts remain a smoke check; engine logit
-and token-ID gates provide the stronger correctness evidence.
-
-The matched-prefill MTP control exposed a separate ordinary-generation
-latency cost: the serial loop constructs the next decode step before yielding
-token zero. A runtime-only source transform yields the first sampled token
-before that construction, retaining the later pipeline and excluding fill,
-grammar and resumed generation. Its initial packed-model pair preserves every
-token ID, live cache state, cache key, usage count and finish reason at raw
-prompt lengths 6/128/512. Native first-token latency improves while complete
-generation time is approximately flat. Six lifecycle tests, the 26 existing
-fill-loop tests and 15 binding tests pass with the prototype. One binding
-cancellation expectation needs the new event order: token zero may arrive
-before cancellation triggered inside the subsequent forward; no token arrives
-after cancellation. Consumer return at that first yield leaves a correctly
-described prompt-only cache. Those screens changed only the runtime-loaded module.
-
-The first serial HTTP pair preserves all seven warm/measured responses but
-does not reproduce the native TTFT improvement. Client-visible output can lag
-token zero through detokenizer buffering. A separate sink observer confirms
-that these raw prompts first generate bare-space token 220, which emits no
-event, followed by token 17 (`2`), which emits content and yields for network
-progress. Both arms preserve that sequence; earlier delivery of the hidden
-space cannot improve client TTFT. Keep this negative result and test chat
-fixtures before attributing a serving win or integrating the prototype.
-Both screens keep sources fixed and exit cleanly. Evidence:
-`first-yield-native-screen{,-review}.json`, `first-yield-http-screen.json`,
-`first-yield-http-observe.json` and its per-server `*.sink.json` records,
-`first-yield-smoke-candidate-v2.txt`, `first-yield-existing-loop-tests.txt` and
-`first-yield-binding-order-tests.txt` in the campaign report directory.
-
-The subsequent three-fixture chat HTTP screen preserves all seven paired
-responses and improves first visible output on each measured fixture, with
-approximately flat complete-request time. Evidence:
-`first-yield-chat-http-screen{,-review}.json`. A simplified version captures
-the current sample index before yielding and increments the normal generated
-count immediately, avoiding a second stats counter. Its control passes eight
-tests and the candidate passes 49 lifecycle, binding and fill-loop tests,
-including two- and three-token budgets. The first budget test expected an
-explicit native `length` reason, but the incumbent leaves budget completion
-implicit; the corrected assertion preserves that contract. Evidence:
-`first-yield-v2-{control-tests,tests,tests-corrected}.txt`. The simplified
-prototype then completes six fresh-process native pairs. All 36 warm/measured
-token, live-state, cache-key and usage comparisons agree; every measured pair
-improves first-token latency at all three prompt lengths. Complete generation
-time is approximately flat. All 12 children exit cleanly with fixed sources
-and the same remaining active allocation after explicit cleanup. Evidence:
-`first-yield-native-v2-repeat{,-review}.json`. Six chat server pairs then
-complete all 84 requests with 42 exact response/count/finish comparisons.
-Every measured pair improves first visible output, with approximately flat
-complete-request time. The sampled native audit also preserves token IDs,
-requested log probabilities and live cache state across two seeds and two
-prompt lengths, with penalties enabled. Both runs keep sources fixed and
-exit cleanly. Evidence: `first-yield-chat-http-v2-repeat{,-review}.json` and
-`first-yield-sampled-native.json`. Other artifacts and cached/pressure
-acceptance remain, including first-token consumer return and follow-up reuse.
-
-The serial change is now integrated behind `MLX_BUN_EARLY_FIRST_TOKEN=1`,
-default off. Its scheduling decision comes from the binding's captured
-runtime; request and execution interfaces are unchanged. Eight permanent
-binding tests cover both settings, captured-policy isolation, short budgets,
-early return, cancellation and initial EOS. The focused tests and enabled-fill
-regression pass, as do all three typechecks, hygiene and the complete
-model-free suite: 1,831 pass, 10 skip, zero failures. The first cancellation
-test incorrectly expected final stats after an exception; the corrected test
-preserves the existing null-stats contract for aborted runs. Orderly consumer
-return still reports the emitted token and exact cache key. Real-model and
-HTTP flag comparisons follow the prototype measurements before this thread
-can close. Evidence: `first-yield-production-unit{,-v2}.txt`,
-`first-yield-production-fill-enabled.txt`, `first-yield-production-typecheck.txt`,
-`first-yield-production-hygiene.txt` and `first-yield-production-model-free.txt`.
-The integrated packed-model native pair preserves all six warm/measured
-token/state/key/usage comparisons. Its serial HTTP pair completes 14 responses
-with all seven paired text/count/finish checks exact. Both reproduce earlier
-first-token delivery and keep sources fixed with clean child exits. A slow
-HTTP warmup is retained; these integration screens do not replace the six
-balanced prototype pairs for timing evidence. Evidence:
-`first-yield-production-native-r6{,-review}.json` and
-`first-yield-production-http-r6{,-review}.json`. Closed source-transform and
-duplicate test helpers are removed; permanent tests and raw reports remain.
-The first integrated RTN4 native pair also preserves all six token/state/key
-comparisons. Its apparent large complete-time difference is not accepted as
-a speed gain: step intervals change substantially within the control process,
-system swap grows and final preflight records substantial Chrome CPU use.
-Keep this correctness result and repeat timing before attribution. Evidence:
-`first-yield-production-native-rtn4{,-review}.json` and per-token arrival times
-in the same report. No unrelated application was stopped.
-
-The continuous scheduler has its own first-output boundary: an empty group
-finishes preparation and emits token zero, then advances decode before its
-event-loop yield. A separate prototype adds a yield only when preparation
-leaves exactly one active row and no queued arrivals. It preserves grouped
-short admissions and stops immediately if the group closes during that await.
-Nine scheduling tests pass for each control/candidate arm; an actual
-continuous HTTP screen must verify wire placement and client arrival time.
-Evidence: `first-prepare-scheduler-{control,candidate}.txt` and the frozen
-`first-prepare-http-screen-plan.json` in the campaign report directory.
-
-Six subsequent AB/BA blocks complete on the M4 Pro with 12 clean server exits,
-84 responses and 42 exact paired responses, including counts, finish reasons
-and cache usage. The actual continuous lane and submitted-row increment are
-checked for every request. Median paired TTFT changes are -7.66% for code,
--7.90% for explanation and -5.63% for JSON, about 74–77 ms earlier. Complete
-request time changes by +0.15%, +0.04% and +0.08%, respectively. One code pair
-has 2.20% worse TTFT and remains in the report. The machine begins with Chrome
-CPU activity and existing swap, so these are diagnostic measurements. Chrome
-is unrelated background load; a Bun HTTP client sends requests directly to
-the server without the chat UI. Source hashes remain fixed. Evidence is
-`first-prepare-http-repeat{,-review}.json`; scheduler lifecycle and broader
-request gates remain before integration.
-
-The continuous option is now integrated under `MLX_BUN_EARLY_FIRST_TOKEN`,
-default off. The MLX backend reads its captured runtime and passes a boolean
-to the generic scheduler. No request or execution-contract field is added.
-After the early yield, the scheduler restarts its policy loop so new arrivals,
-cancelled queued work, admission holds and shutdown are observed before the
-next step. Eight added unit cases cover those transitions, short-admission
-grouping, an existing active row and cleanup on a failed yield. All 1,839
-model-free tests, the three typechecks and hygiene pass. Integrated real HTTP,
-cache, pressure and quiet gates remain. The integrated native option also
-passes four sampled cases per artifact on packed Q3 and RTN4, preserving all
-32 token IDs, requested log probabilities, cache keys, live KV/recurrent state
-and usage. Each arm returns to the same measured native allocation baseline
-after disposal. These are correctness checks, not repeated timing claims.
-Evidence is `first-prepare-production-{unit,model-free,typecheck,hygiene}.txt`
-and `first-yield-production-sampled-{r6,rtn4}.json`.
-
-The first integrated continuous HTTP attempt is retained as a setup failure.
-Its driver set the new option after MLX imports had captured the runtime, so
-the candidate's execution counter remained zero and rejected the run. The
-corrected driver sets the flag in the child environment before imports. Both
-arms then execute the same instrumented production source, with seven early
-yields in the candidate and zero in the control. All seven paired responses
-and usage/finish checks pass, with fixed source and clean exits. TTFT improves
-by 68–90 ms across the three measured fixtures; this one-block integration
-check does not replace the six prototype timing pairs. Evidence is
-`first-prepare-production-http-r6-v2{,-review}.json`; the original unsuffixed
-report preserves the failed path-coverage check.
-
-Nine packed-Q3 native cache-lifecycle pairs also pass. At prompt lengths
-6/128/512, each arm either returns after one token, returns after two, or
-aborts after one, then reuses its own borrowed cache for a 16-token follow-up.
-An early first-token return/abort covers the prompt alone; the incumbent has
-already fed token zero. Clean-return cache keys reflect those exact positions,
-and an aborted generation retains the existing null-stats contract. A
-one-token follow-up suffix with ordinary tail splitting gives matched M=1
-forwards for any missing emitted token. All subsequent tokens, requested log
-probabilities, final live cache states and cache keys match. Both processes
-return to 16,390 native active bytes after disposal, without forced GC, and
-the source stays fixed. This proves the covered native reuse transitions;
-HTTP cache/SSD pressure and broader suffix shapes remain. Evidence is
-`first-yield-production-cache-r6.json`. The same nine pairs also pass on RTN4,
-including every follow-up token, requested log probability, cache key and live
-state. Both RTN4 arms return to six native active bytes after disposal. Its
-separate evidence is `first-yield-production-cache-rtn4.json`.
-
-The broader native suffix screen completes 110 follow-ups across both Qwen
-quants at suffix lengths 1/2/3/4/5/8/31/127/511/512/513. It does not pass
-universal identity. After returning at token one, the early arm's prompt-only
-cache makes the next prefill one token longer. Seven packed and six RTN4
-cases change log probabilities or live state; two packed cases and one RTN4
-case also change emitted IDs. Every two-token-return case remains exact.
-Feeding the missing emitted token separately at M=1 restores exact stopped
-state and all 22 aligned follow-ups, including requested log probabilities
-and cache keys. Both arms return to their original active-allocation baselines.
-This identifies a prefill-shape effect, not a seed mismatch or a corrupted
-snapshot. Keep the option experimental; reconciling the first-token cache
-boundary needs an implementation and performance gate before default
-promotion. Evidence is `first-yield-suffix-qwen{,-review}.json`. The preceding
-Llama smoke also records a natural 512-token suffix state/logprob difference.
-
-The integrated option also passes the saved seven-request agent-boundary/SSD
-gate in serial and actual continuous serving on the M4 Pro 24 GB. The first
-six one-token budgets retain their original path; each lane's counter proves
-the new option runs once on the final 512-token request. All 14 paired choices
-and complete usage records match. The longest prompt has 14,465 tokens and
-reuses the same 12,953-token prefix in every arm. All four final flushes report
-14 durable entries, no pending/missing/failed snapshots, and a longest durable
-prefix of 14,976 tokens. Sources stay fixed and all servers exit cleanly.
-These non-streaming responses establish the covered memory/cache lifecycle,
-not a TTFT result or an agent task success. Broader suffix shapes remain.
-Evidence is
-`first-token-production-long-{serial,continuous}{,-review}.json`.
-
-Actual streaming HTTP cancellation and arrival checks now pass 120 requests
-across eight Qwen servers, covering packed Q3 and RTN4, serial and continuous
-serving, and both option settings. Each server aborts twice at headers and
-twice at first output, then completes an independent recovery request after
-each abort. The server observes all 32 cancellations and settles before
-consuming the 512-token budget. Every recovery response, usage record and
-finish reason matches its reference. Final active MLX allocation equals the
-loaded-model baseline exactly, with no cache entries or active/pending rows.
-One-token budgets also remain exact. Both overlapping-arrival fixtures
-observe two active rows in continuous serving, with actual admission counts;
-serial serving remains serial. All sources stay fixed and servers exit
-cleanly. A preceding Llama smoke adds 60 requests and 16 cancellations.
-These instrumented checks establish lifecycle behavior, not speed. Concurrent
-trajectories are not compared across differing admission timing and batch
-shapes. Evidence is `first-token-lifecycle-{llama,qwen}{,-review}.json`.
-
-The first actual continuous HTTP pair then completes all 14 requests with
-seven exact response/count/finish comparisons and verified batched placement.
-First visible output improves on all three measured chat fixtures; complete
-time varies slightly. Both servers exit cleanly with fixed sources. This
-remains a prototype pending balanced repeats and broader scheduling gates.
-Evidence: `first-prepare-http-screen{,-review}.json`.
-
-The first performance work after the pull uses the packed artifact's actual
-k2/k3/k4 matrices. The M=1/2/3/4/5/8 screen identifies the gate/up crossover
-and the down-projection bit-width asymmetry. Shared-M reduce and fused gate/up
-kernels are retained as experimental variant 7 after exact matrix/MLP checks
-and six balanced diagnostic A/B blocks. Variant 8 adds a balanced and aligned
-3-bit down kernel; it improves the observed M=1/2/4 full-model forwards while
-preserving logits, live state and subsequent-token checks. A device LUT for the existing f32
-code values loses on this M4 and is recorded as a negative result. Kernel
-implementation, commands and evidence live in turboquant.md's packed-kernel section.
-
-The full packed 27B passes logits, live recurrent/KV state and subsequent-token
-identity at M=1..5. Six paired native-forward blocks also retain the gain at
-M=2/4, but they use a short synthetic prefix and project every output position.
-They are not HTTP decode, verified MTP or strict-fill throughput measurements.
-The existing exact-artifact graph/session/cache gate passes on the packed
-model as well. Variant 6 remains the default.
-
-The packed 27B strict-fill regression passes on variants 6 and 7: nonzero
-assert injection, no verification events and exactly matching emitted tokens.
-This is one tool-call fixture, not a held-out determinism or timing result.
-The last-position-only native append sweep covers M=1/2/3/4/5/8/9/16 with
-six paired blocks, preserving candidate/current logits and live state in every
-cell. It retains the small-M gain and confirms the M=4/5 whole-model cost
-discontinuity. Raw evidence: `reports/qwen38-rd/shared-m-append-lengths.json`.
-
-Variant 10 now shares scatter decode across small M as well. It passes the
-same exact full-model gates and improves M=2..4 against variant 8 in six
-paired native blocks. Variant 9 separately moves expanded-projection
-evaluation to the existing layer boundary. Its observed outputs agree, but
-peak memory rises. Extending the packed crossover to M=8 is a distinct Lab
-candidate with observed output differences. The packed-kernel section in
-turboquant.md owns these implementations and evidence.
-
-Integrated variant 11 adds direct tiled gate/up prefill for M=5..32, with
-the same full-model logit/state gates and paired native improvement against
-variant 10. The operation has no model-name dependency; its dispatch profile
-currently covers the measured Qwen MLP geometry. It also improves the strict
-fill fixture's complete generation time in six paired fill/no-fill blocks,
-with identical emitted tokens and no verification forwards. This remains
-fixture evidence, not a universal determinism proof. Evidence:
-`variant11-full-model-review.json` and
-`strict-fill-v11-cap32-served-eos.review.json`. Six paired raw-completion
-HTTP blocks per method improve first-token and complete-request latency,
-with identical output, through strict serial and continuous B=1 execution.
-They cover one six-token prompt and eight/64 output tokens; concurrency and
-broader-workload conclusions remain open. Evidence: `trellis-v11-http.json`
-and `trellis-v11-http-review.json`.
-
-Variant 12 adds tiled split-K down projection at M=5..8. The integrated
-kernel passes exact full-model logits/state/continuation checks and improves
-six native paired blocks without changing the paths outside that range.
-Its implementation and sweep evidence live in turboquant.md. Serving and
-combined-candidate measurements remain separate gates. The combined variant
-12 versus original variant 6 now passes six raw-completion HTTP pairs for
-each serial/default scheduling method, with matching text and actual token
-counts at eight/64 output tokens. Source hashes stay fixed throughout all
-24 bounded server processes. These one-prompt, B=1 diagnostics improve both
-TTFT and complete-request latency; they do not establish concurrent load or
-quiet-machine performance. Evidence: `trellis-v12-combined-http.json` and
-`trellis-v12-combined-http-review.json`. The exact-input native control also
-improves first-token and complete-request time in six paired blocks per
-output length, with identical emitted IDs. One candidate worker escaped
-through Bun with a C++ exception; its traced replay and the complete repeated
-pair succeed. The original failed block is retained and excluded from timing
-aggregation. This does not close the runtime reliability issue. Evidence:
-`trellis-v12-combined-native-review.json`, its original/resumed reports and
-`trellis-v12-native-failure-replay.json`.
-
-The packed serving comparison now also covers four simultaneous raw
-completions. Six fresh-process pairs compare variants 6 and 12, with a warm
-four-request wave followed by a measured wave. Every arm observes four active
-batch rows, all responses/counts/finish reasons agree, and the candidate
-improves complete-wave throughput in every pair. Prompt and SSD caches are
-disabled; the source snapshot stays fixed and all twelve servers exit cleanly.
-These diagnostics establish that the shared small-M kernels improve actual
-concurrent serving, alongside the earlier B=1 results. Mixed lengths, larger
-concurrency and quiet M4 Pro acceptance remain separate gates.
-Evidence: `trellis-concurrent-http.json` and
-`trellis-concurrent-http-review.json`.
-
-The operation trace led to smaller M=5..8 matrix tiles and deferred evaluation
-for direct tiles. Dense expansion retains its projection boundary. The
-integrated kernels pass targeted units and an old-source/new-source comparison
-at ten append lengths, including logits, live cache bytes and continuation.
-Six subsequent HTTP process pairs compare the previous and updated variant 12
-at B=1 and eight concurrent requests. Every wave observes the expected active
-row count; all paired warmup and measured responses, token counts and finish
-reasons agree. Eight-request throughput improves consistently; B=1 first-token
-latency improves, with a much smaller complete-request gain. All twelve servers
-exit cleanly and the source hashes remain fixed. The initial explicit-seed
-pilot routed through serial and is excluded from batching evidence. These
-remain M4 Pro diagnostics, with variant 6 unchanged. Evidence:
-`trellis-tile-integrated-forward-review.json`, `trellis-tile-http-review.json`
-and `trellis-tile-seeded-serial-http-review.json` in the campaign directory.
-Six old/new native process pairs also preserve all 64 emitted IDs and finish
-reasons in warmup and measurement. First-token latency improves in every pair;
-the complete native request improves slightly because single-row decode is
-unchanged. All twelve workers complete with fixed source hashes. Evidence:
-`trellis-tile-native.json` and `trellis-tile-native-review.json`.
-The tile and evaluation-boundary experiments are detailed in turboquant.md.
-
-The existing `bench-serve.ts all` suite also completes two AB/BA process pairs
-for packed variants 6 and 12 with the serial arm, fixed workload seed and 192
-decode tokens. Source snapshots match across all four runs. All 38 paired
-recorded requests and the separate raw/chat parity probes agree; every server
-exits cleanly, all SSD entries are durable, and no failed samples are dropped.
-Single-request decode improves consistently at both tested contexts. Cold
-prefill improves modestly; warm first-token latency varies between pairs and
-sampled RSS shows no reduction. The requested context target produces 2,679
-actual prompt tokens. The four-request aggregate leg is serial queue throughput,
-not batching evidence. This M4 Pro diagnostic does not promote a default or
-replace quiet-machine acceptance. Evidence:
-`single-request-suite-packed.json` and `single-request-suite-packed-review.json`,
-with all four raw suite reports in the campaign directory.
-
-Variant 13 adds vectorized packed-weight expansion after a larger fused-tile
-screen failed to beat native matmul. The operation and packing contract are
-documented in turboquant.md. Focused tests cover every codebook state, circular
-windows, scale dtypes, strided inputs and fallback. The integrated 12/13
-comparison preserves logits, live cache bytes and continuation at five append
-lengths across seven blocks, including six measured AB/BA pairs. Actual kernel
-calls confirm the eligible prefills improve while M=1/8 retain their existing
-path. All three typechecks, hygiene and 1,795 model-free tests pass. Evidence:
-`trellis-vector-integrated-full-model-review.json` and
-`trellis-vector-integrated-{unit,typecheck,hygiene,model-free}.txt` in the
-campaign report directory.
-
-Two subsequent AB/BA pairs through the existing serving suite preserve all 38
-paired requests, raw/chat parity and SSD restart durability. All four servers
-exit cleanly with fixed source snapshots and no failures. Cold TTFT at the
-smaller prefill improves modestly, while decode, long-context TTFT and RSS are
-approximately flat. Cached TTFT varies substantially, so it is not an accepted
-kernel gain. An external browser/build/test burst occurs near the first pair's
-transition and is retained in the observations. This result does not establish
-a broad serving improvement or promote variant 13. Quiet M4 Pro acceptance
-remains. Evidence: `vector-expand-suite-packed.json`,
-`vector-expand-suite-packed-review.json` and
-`vector-expand-suite-load-observations.json` in the campaign directory.
-
-Six further serial HTTP AB/BA pairs isolate 6/16/128/512-token prompts with
-64 generated tokens and prompt/SSD caching disabled. Each shape receives a
-separate warmup; all first-shape timings are retained. Every paired response,
-token count and finish reason matches across all 54 warmup/measured requests,
-sources remain fixed, and all twelve servers shut down cleanly. TTFT and
-complete-request time improve in every 16/128-token pair. The six-token
-control and decode speed are flat; the 512-token and sampled RSS results are
-less consistent. Six default-scheduler pairs repeat the improvement at
-16/128 tokens, with exact responses, fixed sources and clean shutdowns.
-Per-request telemetry proves scheduler admission. Existing swap usage and
-background CPU activity keep both M4 Pro series diagnostic; the 512-token
-effect remains less consistent. Evidence:
-`trellis-vector-short-http-paired-{serial,default}.json` and the corresponding
-`-review.json` files in the campaign directory. Quiet, second-machine and
-pressure acceptance remain before any default promotion.
-
-The updated affine controls now complete six alternating Bun/MLX-LM process
-pairs for each flagship, compact and RTN4 artifact. They use the same six raw
-prompt IDs, 64 output tokens, tokenizer/model EOS handling, production wiring
-decisions and fresh-cache cleanup. Every pair emits identical IDs. Complete
-native request time is close, with a small median Bun advantage; MLX-LM has
-slightly lower first-token latency. These diagnostic controls establish no
-new kernel win or HTTP conclusion. All 36 workers finish and the source
-snapshot stays fixed. Evidence: `native-affine-paired.json` and
-`native-affine-paired-review.json` in the campaign report directory.
-
-The shared grammar bitmask candidate compares host expansion, direct Metal,
-compiled Metal and XGrammar's compiled MLX bitmap-table approach. The compiled
-Metal operation preserves the additive mask's bytes for bf16/f16/f32,
-strided inputs, missing words, signed zero and nonfinite values. Its owned
-host snapshot survives matcher mutation before lazy evaluation. Repeated
-calls release their arrays after stream synchronization. Complete greedy,
-sampled and logprob-enabled sampling retains identical selected tokens and
-logprob bytes across changing masks at three vocabulary sizes, and improves
-eight paired blocks against the actual host loop. An earlier probe used a
-slower host callback expansion and is excluded from performance evidence.
-Grammar integration is opt-in pending actual serving measurements. Evidence:
-`token-mask-probe.json`, `token-bitmask-unit.txt` and
-`token-bitmask-grammar-metal.txt`, plus `token-mask-sampling.json` and
-`token-mask-sampling-review.json` in the report directory.
-
-The first dense attention prototype adapts vLLM-Metal's register-resident
-online softmax. It reduces temporary memory but loses the timing screen.
-Instantiating pinned MLX Steel attention at head dimension 256 is faster on
-some self-attention shapes, while the long-K/short-Q suffix case remains
-slower. Tiling must follow both lengths and the device's workspace limit.
-This changes numerics relative to the current unfused bf16 path and stays
-Lab pending model and task-quality checks. Evidence:
-`steel-attention-tune-manifest.json` and its per-tile reports. The initial
-probe's small-head error metrics used physical output order; the corrected
-contiguous readback agrees with the pinned oracle and CPU reference in
-`attention-cpu-check.json` and `attention-oracle-check.json`.
-The full packed-model Steel test is flat at 128/512 prompt tokens; one
-512-token cell exceeds its predeclared last-logit KL screen, despite matching
-the observed argmax. It has no model-level speed or quality acceptance.
-Moving Q into threadgroup storage and transposing K storage did not rescue
-the separate register-based prototype's timing. Evidence:
-`attention-full-model-packed-128-512-review.json` and
-`dense-attention-memory-tune-review.json`. Larger-context and exact-arithmetic
-attention candidates remain separate experiments.
-
-Graph export now records the existing evaluation boundaries for variants
-6/8 at M=1/4/5. Small-M forward has one final evaluation boundary. M=5 has
-192 projection evaluations, 64 layer evaluations and a final evaluation.
-The exported graphs identify the exercised operations, including affine
-projections, packed custom kernels, GDN, convolution, norms, RoPE, attention
-and casts. Counts are exported operation occurrences, not GPU dispatches.
-Raw graphs and the manifest are in `reports/qwen38-rd/graph-captures/`.
-
-An attached Metal System Trace captures the warmed Bun target and its GPU
-encoder intervals. The installed release MLX labels only command buffers
-and encoders, so individual-kernel duration attribution remains open. The
-capture covers part of prefill and decode; it is not a complete TTFT trace.
-The first launch-based capture contains no target GPU work and is excluded.
-Evidence: `variant8-attached.trace` and `attached-trace-summary.json` in the
-same report directory. A pinned MLX source checkout supports inspection of
-the actual dispatch and an isolated profiling build without changing the
-oracle or installed runtime.
-
-The isolated profiling build now records the kernel name, dispatch grid,
-threadgroup geometry and pipeline limits for each submitted dispatch. A second
-attached trace adds operation labels to encoder scopes. Those scopes contain
-multiple operations, including zero-work views, so their GPU duration cannot
-be attributed to the label's final operation. The selected Xcode counter
-profile is unsupported on this M4 and exports no shader or counter samples.
-Use the dispatch inventory to select isolated operation experiments while
-individual GPU timing remains open. Evidence: `labelled-v8-analysis.json`,
-`labelled-v8-kernels.txt` and `variant8-labelled.trace` in the report directory.
-
-A direct device capability query exposes timestamp counters only at compute
-stage boundaries on this M4 Pro. Dispatch-level sampling is unsupported.
-The isolated pinned runtime now samples each operation in a separate encoder,
-reserving two slots in a shared counter buffer and resolving them once per
-completed command buffer. A per-operation allocation first exhausted Metal's
-sample-buffer resources; a larger pool exceeded its documented-in-error size
-limit. Both failed attempts are retained, and the final pool fits that limit.
-The installed native library and Python oracle are unchanged.
-
-Calibration compares the release library, the isolated build without
-segmentation, segmentation alone, and counters. At B=1 and append lengths
-1/4/8/128 from a 32-token prefix, every arm preserves logits, live cache bytes
-and one continuation across one warmup and three measured forwards. The
-instrumented full-forward overhead is small in this screen. All recorded
-timestamps are valid, and sampled CPU/GPU clock pairs agree. These are
-operation-encoder intervals: operations with multiple dispatches stay grouped,
-intervals can overlap and include waits, and their sum is not GPU wall time
-or critical-path attribution. The small-M trace directs further work toward
-packed projections and affine matvecs; the eight-token trace exposes the
-cost of the tiled prefill path. Evidence: `metal-counter-capabilities.json`,
-`mlx-counter-source-manifest.json`, `mlx-counter-build.patch`,
-`metal-counter-pooled-unit.stderr.txt` and
-`metal-counter-calibration-review.json` in the report directory.
-
-A separate schedule recorder now preserves MLX's normal encoder and command
-buffer boundaries. It counts existing dispatches, barriers, fence waits and
-temporary references, and reads command-buffer GPU start/end times in the
-completion handler, following [Metal's timing contract](https://developer.apple.com/documentation/metal/mtlcommandbuffer/gpustarttime).
-It rejects simultaneous operation segmentation. The first smoke test exposes
-a unit mistake in the recorder: the pinned runtime's `buffer_sizes_` adds
-`data_size()`, which counts dtype elements. Correct the recorder to report
-that heuristic as elements and multiply by item size for temporary bytes.
-The effective M4 Pro submission defaults are 50 dispatches and 50 shifted
-element-count units. The setting's upstream name remains
-`MLX_MAX_MB_PER_BUFFER`; it is not a literal byte ceiling. Temporary-reference
-totals are also not peak resident memory.
-
-The corrected recorder passes 45 complete generations across Llama 1B and
-Qwen packed/RTN4, comparing the ordinary library, the isolated library without
-telemetry and the recorder with native/raw/compiled sampler arms. Every
-token, live state byte and usage record agrees. Warmed complete timings show
-no clear systematic instrumentation shift in this screen; repeated overhead
-calibration remains necessary for quantitative timing attribution. The
-64-token packed sampling trace loses four dispatches per token with fusion,
-but all three sampler arms still submit 3,905 buffers containing dispatches.
-Many additional buffers contain no kernel dispatches; they may still carry
-event waits or signals. Audit callers before attempting to remove them.
-Command-buffer spans include internal waits, so their union is not kernel
-active time or device utilization. Evidence is `mlx-schedule-source-manifest.json`,
-`mlx-schedule-smoke-v2.log` and `top-p-schedule-screen{,-review}.json`.
-
-A separate caller trace attributes 24 zero-dispatch submissions during steady
-packed greedy decode to the pressure branch in MLX's `eval_impl`, reached
-through `async_eval`. That branch finalizes open GPU streams before waiting
-for outstanding work. All three traced generations preserve emitted IDs,
-live cache bytes and usage against the calibrated recorder. Startup-only
-stacks had instead reached final evaluation and synchronization, so they
-could not establish the steady decode cause. The trace adds CPU logging and
-is not timing evidence. These observations do not establish that an empty
-dispatch list has no event work. Keep the validated recorder separate from
-the caller-trace build. Evidence is `empty-submit-r6-steady-trace{,-review}.json`
-and `mlx-empty-submit-steady-source-manifest.json`.
-
-The next stock-library screen varies MLX's command-buffer submission limits,
-bounded breadth-first graph traversal and optional Metal synchronization
-implementation. Measure complete warmed generation, with matching token IDs,
-requested log probabilities and live state, before changing native code or
-retaining a setting. New-shape prefill cells are correctness screens until
-their shapes are warmed in both arms. Positive cells require balanced native
-and actual serial/continuous HTTP repeats, plus memory and disposal checks.
-The frozen screen is `graph-schedule-screen-plan.json`.
-
-That first screen passes all 120 generations and 27 complete arm comparisons
-on Llama 1B and Qwen packed/RTN4. No packed-model decode setting has a
-compelling first-pass gain. The later RTN4 timings overlap with unrelated
-disburse TypeScript/Vite/Playwright jobs, including higher load and swap use.
-Exclude RTN4 timing from tuning decisions and repeat it after those workers
-exit; retain its exactness results. Llama's smaller-buffer observation needs
-balanced repeats. Evidence is `graph-schedule-screen{,-review}.json` and
-`graph-schedule-screen-contamination.json`. Retained tuning belongs at the
-existing model-owned execution seams; these experiments do not establish
-new global defaults.
-
-The RTN4 rescreen completes 40 generations with nine exact arm comparisons,
-including a return to the default settings at the end. The apparent 0.94%
-greedy gain with traversal width 64 matches the ending default's 0.92% drift;
-sampled generation is flat in both. Larger submission limits also fail to
-improve both decode fixtures. No Qwen setting advances from this screen.
-New-shape prefill timings remain excluded. Evidence is
-`graph-schedule-rtn4-rescreen-v2{,-review}.json`.
-
-Llama's six balanced native pairs preserve all 30 paired generations, including
-requested log probabilities and live state. Reducing both submission limits
-to 25 lowers complete warmed generation time by 2.44% for greedy and 2.30%
-for sampled output; all six pairs improve in each cell. TTFT is effectively
-flat. Peak MLX allocation falls in the measured cells, which is not a claim
-about physical RAM savings. Six actual HTTP pairs per lane then preserve all
-84 paired responses across 168 requests. Complete time improves by
-1.86–3.02% in serial and 2.22–3.00% in continuous serving across the three
-fixtures, with every pair faster. These M4 Pro results remain diagnostic.
-The controls are `graph-schedule-llama-repeat{,-review}.json` and
-`graph-schedule-llama-http-repeat{,-review}.json`. MLX caches these settings
-at device initialization; assigning environment variables from a later
-model forward would not apply the measured policy. Broader contexts,
-lifecycle coverage and scoped integration remain.
-
-A separate native prototype elides `finalize` only when the command buffer
-has not been touched since the preceding commit. Access to the buffer,
-encoder creation, output registration and temporary registration all make it
-ineligible. This preserves event-only and completion-handler work, and leaves
-explicit synchronization unchanged. The isolated build and its disabled arm
-each pass 84 stream/lifetime cases against the installed library, spanning
-three dtypes, two sizes, two memory limits, CPU/GPU and GPU/GPU dependencies,
-view-only events and disposal before asynchronous completion. All cases
-return active memory to zero. The subsequent 45 complete generations also
-preserve IDs, requested log probabilities, live state and usage across the
-three artifacts and both library controls. Packed generation is effectively
-flat. Six subsequent RTN4 AB/BA pairs preserve all 30 paired generations but
-find no reliable gain: complete greedy time changes by -0.069%, sampled by
-+0.108% and warmed prefill by +0.011%. Peak MLX allocation is flat. Close the
-prototype without production integration. Evidence
-is `mlx-untouched-finalize-source-manifest.json` and
-`untouched-finalize-stream-default{,-review}.json`, plus
-`untouched-finalize-native{,-review}.json` and
-`untouched-finalize-rtn4-repeat{,-review}.json`.
-
-The separate exact traces confirm that the prototype removes the intended
-submissions. For 64 packed sampled tokens, zero-dispatch buffers fall from
-3,311 to 194, while all 116,164 dispatches and 3,905 buffers containing
-dispatches remain. RTN4's corresponding zero-dispatch count falls from 4,083
-to two; its 115,947 dispatches remain. The sampled skip counters record
-3,113/4,072 omitted calls. Counts need not subtract exactly between runs
-because pressure callbacks and completion timing vary. This is mechanism
-evidence, not an end-to-end speed claim. Evidence is
-`untouched-finalize-trace{,-review}.json`.
-
-The optional upstream `MLX_METAL_FAST_SYNCH=1` mode stalls in the installed
-library's large bf16 three-stream case under the probe's small submission
-limits. A stack sample finds the host in `scheduler::wait_for_one` and a
-worker in `Fence::wait`. Stop only that probe after more than 100 seconds
-without progress, retain its partial results, and rerun all default-mode
-cases successfully. Do not attribute this stock-library failure to the new
-prototype or treat the unrun experimental-mode arms as passes. Evidence is
-`untouched-finalize-stream.json`, `untouched-finalize-stream-fast1-interruption.json`
-and `untouched-finalize-stream-fast1-default.sample.txt`.
-The initial interruption annotation mislabeled that case as f32; the partial
-report's dtype 12 identifies bf16. The corrected attribution is recorded in
-`untouched-finalize-stream-fast1-failure-review.json`.
-
-The existing GDN stage profiler has also been run at M=1/4/128/512 against
-unprofiled forwards from the same prefix. Observed logits agree. Its added
-stage/layer evaluations substantially inflate small-M time, so those stage
-durations cannot substitute for GPU kernel attribution. Evidence:
-`reports/qwen38-rd/gdn-stage-probe.json`. A fused convolution/SiLU/copied-tail
-prototype matches the tested bf16/f32 outputs and live tail at B=1/2. It
-requires MLX's typed sigmoid expression; a float32 rewrite changes bf16
-results. The opt-in operation now passes full packed-model logits, live
-state, continuation and speculative rollback checks. Its initial memory
-benefit exposed an aliasing baseline: `mlx_copy` retains the source buffer.
-The default path now uses a shared compact materialization operation backed
-by stock MLX DynamicSlice, with gradient and compiled-execution gates.
-Replacing the alias preserves all tested packed-model logits, live state,
-continuation and rollback bytes. Explicit disposal still owns handle cleanup;
-the copy change allows the large backing allocation to be released.
-Against an independently materialized baseline, convolution fusion has
-approximately flat full-model timing and similar peak memory. It remains an
-unpromoted R8 experiment. Evidence: `qwen-conv-materialized-packed.json`,
-`materialize-final-packed-comparison.json`, and `conv-probe-typed-sigmoid.json`.
-The final stock-copy rerun is a correctness gate; concurrent CPU trace exports
-exclude its diagnostic timings from performance comparisons.
-
-GDN specialization now covers threadgroup layouts, fixed sequence lengths,
-loop unrolling, compiled wrappers and typed sigmoid fusion. Correcting the
-initial float32 sigmoid rewrite restores exact bf16 recurrence and state.
-The subsequent shape sweep passes all 114 cells. A compiled fixed-length
-candidate preserves full-model logits, live state and continuation, but its
-isolated gains mostly disappear in full forwards. A separate fused recurrence,
-output RMS normalization and gate kernel passes 60 isolated shape/dtype cells
-and the small-M full-model checks. Its full-forward improvement is too small
-to justify integration at this point. Neither candidate changes production
-dispatch. Evidence: `gdn-specialization-typed.json`,
-`gdn-static-full-model-review.json`, `gdn-fused-norm-probe.json` and
-`gdn-fused-norm-full-model-review.json`.
-
-The affine matvec layout screen preserves pinned MLX's reduction arithmetic
-while varying output rows per SIMD group and groups per threadgroup. Actual
-packed-artifact head and GDN projection weights pass exact checks with both
-direct and compiled wrappers. The best isolated head cells improve only
-slightly, and the small-M projection results are mixed. There is no accepted
-whole-model gain. Combined projection dispatch remains a separate candidate.
-Evidence: `affine-qmv-screen.json` and `affine-qmv-compiled.json`.
-
-A second affine prototype shares packed weights and quantization metadata
-across four or eight input rows, preserving MLX's dot-product and SIMD
-reduction order. The corrected real-matrix screen passes all 50 cases; larger
-GDN projections improve in isolation, while vocabulary-head gains are smaller.
-An initial bf16 failure was large, not rounding noise: some shared-row layouts
-read incorrect scale/bias values. Explicit bf16 bit expansion avoids the
-observed failure without resident float32 metadata copies. Direct and compiled
-bf16/f32 debug cells then agree exactly. The underlying compiler cause is not
-established. Six full-model pairs preserve logits, live cache bytes and
-continuation on both packed and plain RTN4 artifacts. RTN4 improves consistently
-at M=4/8 with unchanged peak allocation and flat unused-shape controls. Packed
-improves little; its unused M=128 control also shifts, so the smaller packed
-timing difference needs an independent check. Six RTN4 HTTP process pairs
-then preserve all 108 paired warmup/measured responses, token counts and
-finish reasons. Every measured eight-request wave reaches eight active rows
-and improves throughput; the single-request control is approximately flat.
-All twelve servers exit cleanly with fixed source hashes. Evidence:
-`affine-shared-m-http-rtn4-review.json`, `affine-shared-m-probe.json`,
-`affine-shared-m-probe-initial.json` and
-`affine-shared-m-debug-scale-bits.json`, plus
-`affine-shared-m-full-model-{packed,rtn4}-review.json`.
-
-The retained shared operation is `src/mlx/affine-shared-m.ts`, opt-in via
-`MLX_BUN_AFFINE_SHARED_M=1`. Eligibility follows dtype, quantization, shape
-and stream rather than a model name. It retains native MLX for unsupported
-cells and during shapeless compile traces: the native CustomKernel primitive
-cannot infer output shapes there. The trace context is restored on return or
-exception. Shape-specific compiled calls remain eligible. Focused tests cover
-strided activations and metadata, leading dimensions, shape changes and
-fallbacks. A cleaned-up kernel passes another six RTN4 full-model pairs and
-retains the gain. Six final production-flag HTTP pairs preserve all 156 paired
-warmup/measured responses with fixed source hashes and clean server exits.
-Both four- and eight-request throughput improve in every pair; the
-single-request control is flat. Dispatch counters confirm M4 and M8 use the
-shared operation. Broader artifacts and quiet M4 Pro gates remain. Evidence:
-`affine-shared-m-production-full-model-rtn4-review.json`,
-`affine-shared-m-shapeless.json`, `affine-shared-m-integrated-unit.txt` and
-`affine-shared-m-integrated-http-rtn4-review.json`.
-
-The next row-width screen passes all 112 real RTN4 matrix cases at
-M=2/3/5/6/7. Six full-model pairs per width also preserve logits, live state
-and continuation. M2 loses and retains native dispatch. M3 improves slightly;
-M5/6/7 improve consistently without a material peak-memory increase. The large
-head needs separate geometry: its M6/7 shared-row cells lose even when smaller
-projections win. Wider integration therefore keeps those head cells native.
-Evidence: `affine-shared-m-width-rtn4-review.json` and
-`affine-shared-m-width-full-model-rtn4-review.json`.
-
-A broader-format prototype initially fails nine of 216 arithmetic cells,
-all bf16 M8 layouts with 3- or 5-bit weights. The errors are large and survive
-float32 output storage. Float32 activations remove the large error but also
-change the reference's typed arithmetic. Explicit bf16 input expansion plus
-rounding after each typed addition preserves that arithmetic and passes all
-216 dtype/bit/group-size cells. Evidence:
-`affine-shared-m-all-bits-correctness-review.json`,
-`affine-shared-m-all-bits-debug.json` and
-`affine-shared-m-all-bits-explicit-input-correctness-review.json`.
-
-The compact artifact supplies actual 3-bit head/MLP and 8-bit attention/GDN
-matrices. All 80 real-matrix cells agree exactly, including compiled calls.
-Reusing each packed load for two input rows at 3-bit and four at 8-bit wins
-the retained M4/8 cells. Both compute four output rows per SIMD group, with
-two SIMD groups per threadgroup. Six full-model pairs
-with the prototype and another six with the cleaned production operation
-preserve logits, live cache bytes and continuation; both row counts improve,
-unused-shape controls stay approximately flat and peak allocation is unchanged.
-The 3/8-bit implementation now lives in `src/mlx/affine-shared-quant.ts`, under
-the same opt-in flag and native fallback rules. Other bit widths and group
-sizes remain research-only. Evidence:
-`affine-shared-m-all-bits-explicit-compact-review.json`,
-`affine-shared-m-all-bits-full-model-compact-review.json` and
-`affine-shared-m-broader-full-model-compact-review.json`.
-
-The final 4-bit operation includes M3/5/6/7, retaining native M2 and the large
-head at M5/6/7. Six complete-model pairs per width preserve logits, state and
-continuation. Six production-flag HTTP pairs at B1/3/5/6/7 preserve all 264
-paired warmup/measured responses. Each measured wave reaches the requested
-active width; candidate dispatch is observed and all twelve servers exit
-cleanly. Concurrent throughput improves in every pair. Single-request decode
-is approximately flat. All pairs are retained, including noisier final runs:
-the machine has residual swap, XProtect is active at the initial preflight,
-and unrelated build/browser activity is observed after the run. This remains
-a diagnostic result. Wider model coverage remains open.
-Evidence: `affine-shared-m-broader-full-model-rtn4-review.json` and
-`shared-affine-frontier-http-rtn4-review.json`. Current focused tests, all
-three typechecks, hygiene and the complete model-free tier pass; the latter
-has 1,785 passes, ten skips and no failures. Evidence:
-`affine-shared-m-broader-{unit,typecheck,hygiene,model-free}.txt`.
-
-Six final compact production-flag HTTP pairs preserve all 156 paired warmup
-and measured responses, usage counts and stop reasons. Both B4 and B8 improve
-in every measured pair; the median B1 control is flat, with individual noisy
-outliers retained. Actual active widths and both 3/8-bit dispatches are
-observed, all twelve servers exit cleanly and source hashes remain fixed.
-Residual swap keeps this diagnostic. Evidence:
-`shared-affine-frontier-http-compact-review.json`.
-
-A subsequent host-overhead prototype caches shape-specific compiled calls
-around these shared-affine kernels. Every weight/scale/bias/activation remains
-a dynamic input, so one compiled function can serve distinct model layers.
-Six compact full-model pairs retain exact logits, state and continuation;
-24 compiled configurations serve all eligible projections without new traces
-in measured rows. The eligible M4/8 cells improve modestly. Six subsequent
-HTTP process pairs preserve all 156 paired warm/measured responses with clean
-server exits and fixed source hashes. Serving gains shrink to approximately
-flat at B1/4 and a small B8 difference. This does not justify integrating the
-extra compiled-function cache at present. Evidence:
-`affine-compiled-call-cache-full-model-compact-review.json` and
-`affine-compiled-call-cache-http-compact-review.json`.
-
-The grammar-mask operation now completes six compact HTTP process pairs at
-B1/4 with shared-affine enabled in both arms and grammar jumping disabled.
-All 60 paired warm/measured responses match and satisfy the frozen JSON
-schema; both arms apply the mask the same number of times. Every concurrent
-wave reaches four active rows, source hashes remain fixed and all servers
-exit cleanly. The timing is inconclusive for promotion: residual swap and
-overlapping unrelated build/browser work are recorded, the B1 median is
-approximately flat and the B4 median regresses. Preserve all samples and
-revisit on a controlled session. Host masking remains default. Evidence:
-`grammar-mask-http-compact-review.json`.
-
-A fused affine gate/up/SwiGLU prototype shares activation loads and replaces
-the two projections plus activation dispatch. Its first version preserves
-both projection outputs but fails the activation check: a float32 sigmoid
-expression differs from MLX's dtype-specific expression. The corrected
-kernel passes all 48 real RTN4 small-M cells, including complete MLP output.
-Only M=1 benefits in that isolated screen. Restricting the full-model prototype
-to M=1 preserves logits, live state and continuation across six paired blocks,
-but its full-forward gain is too small to justify integration. Longer-M
-forwards retain the original path. Evidence: `affine-gateup-debug.json`,
-`affine-gateup-debug-typed.json`, `affine-gateup-typed.json` and
-`affine-gateup-full-model-review.json`. The existing packed activation's
-separate arithmetic contract needs its own audit before reusing that code
-in an affine operation.
-
-A residual-add/RMSNorm kernel preserves the native reduction topology and
-the rounded residual before normalization. All 90 synthetic bf16/f16/f32
-cases preserve both outputs. Six packed-model pairs at M1/4/8/128 then
-preserve logits, live cache bytes and continuation, with 64 fused calls per
-forward. M1 is slightly slower, M4/8 are approximately flat and M128 varies.
-The complete path does not establish a compelling benefit, so this version
-is not integrated. Operation samples include wrapper/evaluation costs and are
-not GPU-only timestamps. Evidence: `residual-rms-screen-scalar-review.json`
-and `residual-rms-full-model-packed-review.json`. Earlier driver/binding
-failures occurred before timing and remain separate. Closed helpers are
-removed after recording the finding.
-
-The first real HTTP screen uses the compact artifact through bounded CLI
-servers. It retains request hashes, actual usage, arrival times, cache hits,
-restart durability, concurrent request completion and failures. Its cold
-context leg required a Bun restart; the retried nonce changed the prompt, so
-that context comparison is unmatched. The chat probe also diverged despite
-equal prompt counts. Frozen chat IDs produce identical direct Bun/Python
-tokens. A cold HTTP control reproduces the Python difference only in its
-seedless BatchGenerator path; `seed=0` selects its serial method and matches
-the direct text. Compare those execution methods separately. The Bun HTTP
-sample also omits a final space, so it has no byte-identical wire-output claim.
-The benchmark records failed-attempt
-stderr and process status, including failures recovered by retry. This screen
-is diagnostic and establishes no serving win. Evidence and exclusions:
-`serving-compact-screen-0.review.json` and
-`serving-primes-native-comparison.json` in the report directory.
-
-Repeated native-throw traces identify the continuous serving crash as Metal
-command-buffer out-of-memory. Bun 1.4.2 avoids it in one screen but misses
-one pending SSD snapshot at its first restart boundary; neither issue is
-closed. Continuous preparation and decode now share the serial memory
-guard, entered after cache acquisition under the execution lease. Ownership,
-refusal and guard restoration tests pass. The same context failure still
-reproduces: the affine graph's transient allocations occur at evaluation,
-after the per-layer construction checks. Investigate allocator residency
-and snapshot retention before declaring this fixed. Evidence:
-`serving-compact-bun140-throw.review.json`,
-`serving-compact-bun142-throw.review.json` and
-`serving-compact-shared-guard.review.json`. Source snapshots now include
-untracked kernels as well as tracked changes.
-
-The frozen compact request isolates a cache-consumption cause: after one
-warm repeat transfers the exact recurrent boundary out of RAM, a third
-identical prompt loses that prefix and prefills again. Stopping after two
-requests also reproduces a missing SSD snapshot followed by a false-success
-flush; restart cannot reuse the longer untrimmable descendant. Persisting the
-boundary before transfer preserves the third request's prefix. Durability
-records now remain dirty until actually covered, and already-durable records
-avoid another clone. Ownership/failure and coordinator regression gates pass.
-The three-request HTTP replay preserves output and cached tokens through
-repeat and restart, with no missing flush. The two-request replay also writes
-the consumed boundary before restart. This requires a synchronous SSD write
-on the first transfer when that boundary is not already durable. The complete
-compact serving diagnostic now finishes context repeats, SSD flush/restart
-and four concurrent streams without a recorded failure. One diagnostic pass
-does not close long-agent or repeated reliability acceptance. Evidence:
-`ssd-boundary-before.json`, `ssd-boundary-two-before.json`,
-`ssd-boundary-after.json`, `ssd-boundary-two-after.json`,
-`ssd-boundary-tests.txt` and `serving-compact-boundary-fixed.review.json`
-in the campaign report directory.
-
-The merged packed path now passes the seven saved Pi tool-result boundaries
-in separate serial and continuous server processes. Both arms use variant 12
-and the same frozen requests. Each completes the longest prompt, emits the
-requested 512 tokens, demotes old cache entries under pressure, and finishes
-with a durable SSD flush and no pending, missing or failed snapshots. Responses
-match across the two methods and the source snapshot stays fixed. The system
-prompt was reconstructed once with the installed Pi package and frozen;
-historical system-prompt byte identity is not established. This is a bounded
-memory/integration check, not a long-running agent success claim. No returned
-tools execute. Evidence: `agent-boundary-fixture.json`,
-`agent-boundary-current.json` and `agent-boundary-current-review.json`.
-
-The M4 DeltaNet fixture discrepancy is now traced to the cached reference,
-not a Bun/oracle disagreement. Both implementations match gate values, output
-bytes and complete recurrent state on the frozen inputs. The explicit
-replay changes one older expected output and adds oracle provenance/state
-hashes. The platform evidence and regeneration procedure live in
-[environment.md](../reference/environment.md).
-
-R22 now tests immutable file-backed weights through page-aligned Metal roots
-and ordinary MLX tensor views. The existing external-buffer entry point is
-sufficient for element-aligned tensors; each root includes the leading page
-padding and the tensor view carries the offset. This resembles
-[llama.cpp's mapped Metal buffer views](https://github.com/ggml-org/llama.cpp/blob/427291b5b34cd914a31b3fd3b61a68f6184f4b9f/ggml/src/ggml-metal/ggml-metal-device.m#L1999),
-which align the root pointer and split oversized mappings into overlapping
-views. The prototype uses no new artifact and no new production loader.
-Small metadata and unsupported element alignment retain native loading.
-All 32 operation cases match payload bytes and GPU add/matmul results;
-pointer equality proves that the candidate aliases the file, and file hashes
-stay unchanged. The Llama 1B artifact is ineligible because its large payloads
-start at odd byte offsets, so that initial native/native control establishes
-no candidate performance result.
-
-The first packed 27B native pair maps 758 tensors, 11,895,111,680 payload
-bytes, through 11,907,530,752 bytes of page windows. Prefix state, ten
-full-forward logit/live-state/continuation cases and both generation token
-sequences match. The subsequent six-pair repeat preserves all 60 full-forward
-logit/state/continuation cases, every prefix state and all 12 generation
-comparisons. Sources stay fixed and all 12 child processes exit cleanly.
-The measured 64-token generation is 0.25% slower by median paired wall-time
-ratio, with no faster pair; TTFT is approximately flat. M1 is about 0.6%
-faster, while M4/8/128/512 give no consistent useful improvement. Evidence:
-`mmap-weights-native-packed-paired{,-review}.json`. Readiness varies strongly
-with order and file-cache state, so it has no cold-start improvement claim. Process RSS and macOS
-physical-footprint accounting fall sharply because most private copied weights
-become file-backed mappings. That does not shrink model bytes or establish a
-corresponding reduction in total physical RAM: clean file pages and GPU
-residency require separate accounting. Preserve MLX active/peak bytes, RSS,
-`vmmap` output and system pressure together. Evidence:
-`reports/qwen38-rd/mmap-page-window{,-pointer}-screen.json` and
-`mmap-weights-native-{llama1,packed}-screen.json`.
-
-The serial HTTP follow-up completes six fresh-server pairs with all 42
-warm/measured response texts, usage counts and finish reasons exact. Sources
-remain fixed and all servers exit cleanly. Warm complete-response time is
-0.04% to 0.14% slower by median paired ratio; decode is 0.14% to 0.21% slower.
-Median per-server peak RSS changes from 12,182,749,184 to 482,508,800 bytes.
-The recorded physical-footprint summaries show the same private/file-backed
-accounting change, not a smaller tensor working set. Evidence:
-`mmap-weights-http-serial{,-review}.json`. Default scheduling, system-memory
-pressure and cold file-cache behavior remain separate gates.
-
-The initial prototype retains immutable roots and mapping owners for the
-process, matching the existing external-array lifetime requirement. A retained
-loader must also prove unload, outstanding GPU-reference ownership, repeated
-model replacement, pressure behavior and HTTP performance. No default or
-production lifetime change follows from this screen. A subsequent ownership
-prototype stores array/root handles with each `Weights` instance and releases
-them on disposal. Existing shard mappings remain mapped for the process. Three
-full-artifact cycles each load 2,019 tensors and release all 13,056,265,696
-active bytes. An outstanding GPU consumer survives weight disposal, produces
-the exact expected output and releases the remaining 2,048 bytes on disposal;
-active memory is zero before GC. Evidence:
-`mmap-weights-ownership-packed{,-mapping}.json`. Three additional cycles
-retain a single-input lazy negation, dispose the weights before submitting it,
-and then evaluate it. Outputs remain exact; the pending view retains
-39,747,584 bytes until its disposal, after which active memory returns to zero
-before GC. Evidence: `mmap-weights-single-consumer-lazy{,-mapping}.json`.
-The whole-tensor follow-up fails output identity on the first cycle after
-owner disposal, although active memory returns to zero and the source tensor
-still matches the untouched original artifact. The earlier small view did
-not qualify for unary buffer donation: MLX also bounds excess backing-buffer
-bytes. Thus those passing cases did not establish donation safety. Keep this
-read-only loader out of production; reduce the failure on scratch data and
-test writable private mappings with native lifetime ownership. Evidence:
-`mmap-weights-whole-consumer-lazy.json`, its log and
-`mmap-whole-consumer-source-integrity.json`. Full serving, default-scheduler
-and pressure gates remain for any corrected loader.
-
-Scratch files isolate the failure across float32/bfloat16 and aligned/offset
-views. With a retained root, all read-only cases are exact and output uses a
-separate buffer. After root release, all four donated read-only cases return
-the original input bytes instead of the negation. Private writable mappings
-preserve every output, including donation, and leave each file unchanged.
-Evidence: `mmap-donation-{ro,cow}-screen.json`. The corrected prototype gives
-each private page window a native payload destructor that unmaps it on the
-last MLX buffer release. It uses no JavaScript destructor or new production
-API. All eight scratch ownership cases preserve output and file bytes, retain
-the mapping through the live consumer, and release every mapping afterward.
-Three full-artifact cycles then preserve the whole-tensor lazy negation after
-weight disposal. Only its 1,654,784-byte mapping survives until the pending
-consumer is disposed; native mapping counts and MLX active bytes return to
-zero before GC. Evidence: `mmap-managed-donation-screen.json` and
-`mmap-weights-managed-whole-consumer-lazy{,-mapping}.json`. A further 64-cycle
-scratch gate submits chained operations asynchronously and disposes the
-input handles. Half the cycles check every output value; the others dispose
-the final output before synchronization. All release every native mapping and
-return MLX active memory to zero. Evidence: `mmap-managed-submitted.json`.
-Six subsequent private-mapping native blocks on the M4 Pro 24 GB preserve 60 full-forward
-logit/state/continuation cases and 12 8/64-token generation comparisons, with
-fixed sources, clean exits and zero remaining native mapping windows. The
-64-token warm generation median paired wall-time change is +0.02%; TTFT is
-+0.13%. Warm full forwards at M1/4/8/128/512 have no useful consistent speed
-gain. The opening-plus-first-hidden-forward phase is about 7.38 seconds slower
-by median paired difference, but that observation materializes different
-amounts of the model. After the hidden-only forward, native head tensors are
-still lazy while mapped roots already account for their full extent. The
-727,580,672-byte active-allocation difference is exactly the 715,161,600-byte
-head plus 12,419,072 bytes of page padding, in all six pairs. Large changes in
-RSS and `vmmap` physical-footprint accounting still do not establish how much
-system RAM is saved. Child wall time includes `vmmap` inspection and is not a
-performance metric. Evidence is
-`mmap-weights-managed-native-repeat{,-review}.json` and
-`mmap-initial-residency-attribution.json`.
-
-A separate lazy-view control removes per-tensor evaluation and pointer checks
-from loading. All 64 unsubmitted consumers and 64 submitted consumers remain
-valid after releasing their weight owners, then release every mapping and
-return active allocation to zero without forced GC. Its model screen preserves
-ten full-forward cases and both generations, but warm 64-token generation is
-+0.22% slower and the first hidden-forward phase does not improve. This rules
-out retaining the lazy change on that screen; it does not isolate the cause
-of the first-use cost. Evidence is
-`mmap-lazy-view-ownership.json` and
-`mmap-weights-lazy-managed-native-screen{,-review}.json`. Neither prototype is
-integrated.
-
-A subsequent four-arm screen measures opening the model through the first
-generated token, completing all 64 tokens, then a warm 64-token generation.
-Native loading, lazy private mappings, mapped pages with an OS read-ahead
-hint, and mapped pages read once per page preserve all token IDs, live state
-and usage, then release every mapping. On the M4 Pro 24 GB, native first-token
-time from opening is 14.09 seconds; lazy mapping is 13.62 seconds, read-ahead
-19.97 seconds and explicit reads 30.50 seconds. These timings include the
-preparation itself. Warm generation is approximately flat. The two CPU
-preparation methods lose this screen. Lazy mapping's small first-generation
-gain needs balanced repeats: this is one fixed-order block with unpurged
-filesystem cache. Neither a cold-storage result nor a physical-memory saving
-is established. Evidence is `mmap-prefault-native-screen.json`.
-
-The balanced follow-up rejects that apparent cold-generation gain. All six
-fresh-process pairs preserve both 64-token generations, live state and usage,
-with fixed sources, clean exits and all 758 mappings released. First-token
-time from opening is worse in five pairs: the median paired change is +18.42%
-or +2.24 seconds. Completing the first generation is +12.93% or +2.29 seconds;
-warm generation is approximately flat at +0.24%. One large native outlier
-remains in the report. The filesystem cache is unpurged and the machine is
-not quiet, so these are diagnostic results. RSS accounting falls sharply for
-file-backed pages but physical-memory savings are still unproven. Close this
-loader as a performance candidate on the measured workload; no integration
-or HTTP speed claim follows. Closed loader, page-preparation and ownership
-helpers are removed; raw plans and evidence remain in
-`mmap-cold-native-repeat{,-review}.json`.
-
-The first full-model pair with native private mappings preserves prefix state,
-all ten checked logit/live-state/continuation cases and both 8/64-token
-generation sequences. Sources remain fixed; both children exit cleanly and
-the mapped child releases all 758 native windows before exit. Its 64-token
-wall time is approximately flat. This single pair is a correctness screen,
-not a performance acceptance. Evidence:
-`mmap-weights-managed-native-screen{,-review}.json`. Repeated native/HTTP,
-and pressure gates still apply. A subsequent API ownership gate preserves
-pending-consumer output and rematerialized source bytes after `release`,
-`releaseShard` and `dispose`. It rejects a late view change and releases the
-mapping after an intentional tensor-fixup failure. All native mapping counts
-and MLX active bytes return to zero. Evidence:
-`mmap-managed-shard-release{,-mapping}.json`.
-
-Next: finish MTP and strict-fill request measurements after the kernel
-changes, broaden their correctness and state audits, and obtain individual
-kernel attribution. Continue shared GDN/convolution/affine/prefill work and
-the exact-artifact controls across quants. Quiet application A/Bs, pressure
-replay and quiet M4 Pro gates remain. Recheck process ownership and preflight
-before each GPU run; diagnostic measurements cannot promote a default.
-
-The native-owned managed mapping is visible to MLX allocation accounting.
-Across all ten full-model rows in its paired screen, mapped peak allocation
-exceeds the incumbent by exactly 12,419,072 bytes, the difference between
-page-aligned root and payload bytes. The low process RSS is not evidence of
-an unaccounted model in the async-scheduling threshold. Combining these
-experiments still needs a separate pressure gate. Evidence:
-`mmap-managed-active-accounting-review.json`.
-
-A dedicated KV append dispatch probe covers 18 bf16 cases with four KV heads
-and dimension 256, at depths 65/256/2049/2048/8193/8192. Every active prefix,
-new row and retained snapshot remains exact; explicit cleanup returns MLX
-active allocation to zero. With spare capacity and no retained view, each
-SliceUpdate dispatches only the new-row copy. Holding an evaluated temporal
-view adds a full-buffer copy; releasing it restores donation. At a 256-step
-growth boundary, the pair allocates padding, concatenates old/new storage
-and appends, totaling eight dispatches regardless of retained views. This
-rejects the assumption that ordinary cache append always copies the full
-buffer. The next gate is a real-generation census of retained views and
-growth; instrumented one-shot timings do not establish a speed win. Evidence:
-`kv-append-donation-probe{,-review}.json` and its native dispatch log.
-
-A real native-generation census on the interleaved v13 artifact uses a
-2,049-token synthetic prompt and 32 generated tokens. Without a prompt
-snapshot, all 1,024 observed one-row K/V appends dispatch only the new-row
-copy. Retaining a real `cloneKvCaches` snapshot produces 32 full-buffer copies,
-one per K/V buffer, then 992 new-row-only appends. Both runs emit identical
-IDs with fixed sources. This rejects a repeated whole-cache copy in the
-tested native path; serving, deeper contexts and other cache types remain
-separate coverage. The counter build is for attribution, not timing claims.
-Explicit cleanup leaves 16,390 active bytes, so this is not a zero-allocation
-claim. Evidence: `kv-append-model-trace-2049{,-review}.json` and its dispatch
-log under campaign reports.
-
-The actual HTTP census confirms the same append behavior at a verified
-2,049-token text prompt. Serial and continuous servers each complete two
-16-token warmups and a 32-token measured response. All three paired outputs,
-counts and finish reasons match, the continuous lane is verified, and both
-servers exit cleanly with fixed source. Each measured request has 1,024
-one-row K/V update encoders, all writing only the new row; neither performs
-a full-buffer copy for those appends. No matching update records fall outside
-the request phase markers. This rejects a repeated whole-cache-copy hypothesis
-for both tested HTTP lanes with prefix caching disabled. Deeper context,
-retained snapshots, growth boundaries and compressed caches remain separate
-regimes. The original token-array HTTP fixture was correctly rejected by the
-endpoint; the corrected text fixture is encoded and hashed by the Bun
-tokenizer before tracing. Evidence is
-`kv-append-http-text-census{,-review}.json` and its per-server dispatch logs.
-
-The native census also passes at an 8,193-token prompt with 32 generated
-tokens. Ordinary ownership again gives 1,024 new-row-only K/V writes. A held
-snapshot gives 32 whole-buffer copies followed by 992 new-row-only writes.
-The two runs emit identical IDs, cover the same 8,224 cached tokens, keep
-sources fixed and return to 16,390 active bytes after explicit cleanup.
-Each phase also contains 480 larger prefill updates. These counts classify
-`SliceUpdate` encoders only; allocation/growth and copy traffic in other
-operations remain separate measurements. Evidence is
-`kv-append-native-8193{,-review}.json` and its dispatch log. There is no timing
-claim or new append kernel from this attribution result.
+### 7.8 Execution log (archived)
+
+The chronological run log (2026-09-04 → 2026-09-08; harness preparation,
+early first token, shared-M/tile/split-K kernels, graph exports and counter
+calibration, MLX scheduling settings, shared affine kernels, GDN
+specializations, strict fill, grammar mask, mmap loader, KV append census)
+moved verbatim to
+[qwen38-27b-campaign-log.md](../archive/investigations/qwen38-27b-campaign-log.md).
+Every evidence filename it cites is kept there and resolves to
+`reports/qwen38-rd/` on the measuring host. Outcomes, retained flags, next
+steps and reopen conditions per row are in §7.13; §7.9–§7.12 stay live.
+Changes that landed by default from that log are correctness or memory fixes,
+not speed defaults: bounded 8 MiB range cache (§7.7), per-chunk draft-KV
+evaluation (§7.7), compact state materialization through stock DynamicSlice,
+SSD boundary-before-transfer fix, Qwen tool-parameter parser,
+`reasoning_content` alias, EOS/finish-reason accounting, native pack 0.4.0.
 
 ### 7.9 Project kernel coverage
 
-The initial AST inventory contains 35 custom Metal kernel construction sites
-and 355 calls into 162 distinct MLX C APIs. The API count includes allocation,
-configuration and graph operations; it is not a count of native GPU kernels.
-Runtime specializations still need shape, dtype, layout and stream coverage.
-The complete construction-site list, source hashes and per-site evidence are
-`reports/qwen38-rd/project-kernel-inventory.json` and
-`project-kernel-coverage.json`. Every site remains open for broader coverage.
-Variant 13 adds the 36th construction site; its source hash and evidence are
-in `project-kernel-vector-expand-addendum.json` alongside that initial census.
-Joint TurboQuant K/V decoding adds the 37th. A refreshed construction-site
-census and its evidence are in `project-kernel-turboquant-addendum.json`.
-The narrowly selected inverse-rotation kernel adds the 38th; the refreshed AST
-census is `project-kernel-inverse-addendum.json`.
-The MLX 0.32.2 Trellis small-prefill port adds the 39th. Its refreshed census
-and gates are in `project-kernel-wide-addendum.json` and §7.12.
-Consolidating the runtime removes the two old-core affine construction sites.
-The current AST census has 37 sites in 15 files; frozen controls retain the
-retired kernels. Evidence: `project-kernel-consolidated-census.json`.
+The current AST census has 44 custom Metal kernel construction sites in 18
+files, plus 12 direct compiled-function construction sites. The seven additions
+since the earlier census are three paged-attention kernels, two normalized
+greedy reductions, masked recurrent update and row-length-aware state replay.
+Frozen source controls retain the two old-core affine kernels the
+MLX 0.32.2 consolidation removed (§7.12). The initial inventory also counted
+355 calls into 162 distinct MLX C APIs; that count includes allocation,
+configuration and graph operations, not native GPU kernels. Runtime
+specializations still need shape, dtype, layout and stream coverage. Evidence:
+`reports/prefill-observation/kernel-census.json` (current), with
+`reports/qwen38-rd/project-kernel-consolidated-census.json` and
+the earlier censuses in `project-kernel-inventory.json`,
+`project-kernel-coverage.json`, `project-kernel-vector-expand-addendum.json`,
+`project-kernel-turboquant-addendum.json`, `project-kernel-inverse-addendum.json`
+and `project-kernel-wide-addendum.json`.
 
 | Owned kernel family | Sites | Current evidence and next coverage |
 |---|---:|---|
 | Packed grammar mask | 1 | Exact micro and HTTP outputs; serving timing inconclusive under external load. |
+| Normalized greedy partial and final reductions | 2 | Same-machine operator and actual-model comparisons pass on both Macs; MiniCPM standard AB/BA improves throughput while packed Qwen MTP3/TQ is flat. Metadata, stochastic and custom samplers retain their prior path. |
+| Direct paged-attention bf16, affine score and affine value | 3 | C5 operation/model/state and native HTTP checks pass. Recorded wins and regressions do not justify a default; block formats and attention remain optional through the cache/model interfaces. |
 | Joint TurboQuant K/V decoding and selected inverse rotation | 2 | Experimental opt-in; exact operation/lifetime and R6/RTN4 integrated model gates pass on MLX 0.32.2. Integrated inverse-rotation HTTP passes six pairs per quant with decode gains in every pair; R6 complete time improves and RTN4 complete time is inconclusive. Six deferred-consumer pairs per MiniCPM/Gemma also improve complete requests on consolidated source. Combined/pressure and quiet acceptance remain. Direct packed-key attention is exact but slower. Details live in turboquant.md. |
 | Flash attention forward, D, dKV and dQ | 4 | D256 alternative screen fails one KL gate; other attention shapes and backward costs remain. |
 | GLM sparse-attention rank and contract keys | 2 | Both keys now use uint32 with stable score sorting. Operation and complete tiny-model state gates pass; six balanced pairs per shape improve selection latency. Real Colibri serving remains unmeasured. |
 | GLM q4/q8 streamed expert gate/up and down | 4 | Synthetic resident-slot geometry screen and six balanced repeats preserve intermediate/output bytes and cleanup. BF16 timing is bimodal across identical process configurations; no geometry default retained. Attribute that variation before further timing claims. Real model and I/O overlap remain unmeasured. |
 | Qwen causal convolution, SiLU and copied tail | 1 | Exact operation/model/state gates; fusion has no compelling full-model speed win. |
-| Gated recurrent update | 1 | Unrolling, geometry, compilation and normalization fusion screened; full-model gains too small to retain. |
+| Gated recurrent update, masked update and prefix-state replay | 3 | Unrolling, geometry, compilation and normalization fusion screened; full-model gains too small to retain. Masked padding and row-length replay preserve inactive state and accepted-prefix state in composed native/serving gates. Uniform state-only replay did not establish a serving win and remains unselected. |
 | Qwen multimodal rotary embedding | 1 | Direct-stride fusion preserves operation/model state, but the complete-forward gain is too small to retain. Actual image/video HTTP coverage remains. |
 | Trellis reduce, scatter, expand and gate/up variants | 9 | Shared rows and fusion improve measured cells. Device codebook lookup loses. A scoped threadgroup integer table improves repeated M3/4 scatter, full forwards and MTP HTTP requests with exact outputs; M1 loses and stays procedural. Scheduling and prefill controls remain. Broader bit/axis/shape/model coverage remains. |
 | Trellis split-K projection and accumulation | 2 | Short-prefill model/serving acceptance; larger shapes and additional artifacts remain. |
 | Trellis tiled projection | 1 | Short axis-1 tiles accepted diagnostically; larger tiles, context and memory pressure remain. |
-| Trellis native-wide prefill | 1 | Matches MLX 0.32.2 small-prefill arithmetic on eligible GPUs. Operation, integrated R6 state/generation and six HTTP pairs per lane pass. Short-prefill TTFT improves; broader context and quiet acceptance remain. |
-| Trellis vectorized expansion | 1 | Exact operation, full-model prefill and HTTP responses. Shorter serial and default-scheduler HTTP prefills improve; existing-suite timing mostly flat. Broader context, pressure and quiet M4 Pro gates remain. |
+| Trellis native-wide prefill | 1 | Reached under `MLX_BUN_TRELLIS_VARIANT=11..13`, including the current variant-13 default; variant 6 expands then uses native matmul. Matches MLX 0.32.2 small-prefill arithmetic on eligible GPUs. Operation, integrated R6 state/generation and six HTTP pairs per lane pass. Short-prefill TTFT improves; broader context and quiet acceptance remain. |
+| Trellis vectorized expansion | 1 | Exact operation, full-model prefill and HTTP responses. Shorter serial and default-scheduler HTTP prefills improve; existing-suite timing mostly flat. Broader context, pressure and quiet-machine gates remain. |
 | Cross-entropy forward/backward variants | 7 | All seven routes pass the vocabulary-partition screen, including 54 forced-fallback cases. The isolated fast-dispatch opportunity does not apply to the available models wired into this loss path; broader real-loss profiling remains. |
 
 The native-operation audit covers quantized/dense/gather matmul, SDPA,
@@ -2292,7 +1235,7 @@ controls vary from about 0.29 to 0.58 ms; Q8 M1 controls vary from about 0.24 to
 0.51 ms. Some candidate/control pairs reverse by nearly 2x. Float32 is much
 steadier and generally flat across geometries. Attribute this bf16 timing
 variation before further optimization claims. No geometry default changes.
-These are M4 Pro 24 GB diagnostics. Real Colibri serving remains unavailable;
+These synthetic screens were measured on the M4 Pro 24 GB. Real Colibri serving remains unavailable;
 the local GLM-5.3-Flash artifact uses a different architecture. Evidence:
 `streamed-expert-screen{,-review}.json` and
 `streamed-expert-repeat{,-review}.json`.
@@ -2419,7 +1362,7 @@ unchanged and the continuous lane/submitted-row counters are asserted.
 The apparent packed serial improvement does not survive six balanced pairs:
 all 30 paired responses remain exact, but complete sampled request time is
 0.183% higher at the median and only one pair is faster. The untouched greedy
-control moves by +0.058%. These M4 Pro diagnostic results do not justify
+control moves by +0.058%. These loaded-machine results (M4 Pro) do not justify
 retaining the fusion for Qwen. Evidence is `top-p-http-screen{,-review}.json`
 and `top-p-http-r6-serial-repeat{,-review}.json`.
 
@@ -2534,8 +1477,8 @@ dispatch. Both arms disable the existing shared-affine flag, and the candidate
 pays its eligibility checks. The first two HTTP attempts failed overlap
 qualification and remain excluded. Evidence:
 `affine-head-batched-decode-mfirst-{llama1,llama3}-review.json` and
-`affine-shared-head-http-{llama1,llama3}-overlap-review.json`. These M4 Pro
-diagnostics do not replace quiet-machine acceptance. Closed head-only helpers
+`affine-shared-head-http-{llama1,llama3}-overlap-review.json`. These
+loaded-machine diagnostics (M4 Pro) do not replace quiet-machine acceptance. Closed head-only helpers
 are removed; the broader operation's eligibility work remains open.
 
 A bf16-only selection uses large projections and leaves small matrices on
@@ -2579,9 +1522,10 @@ Six native pairs per append length preserve all logits, live states and
 continuations. Both M4 and M8 improve in every pair; M1 and M128 controls are
 approximately flat. The candidate exercises 105 additional projections at
 M4/8. Evidence: `affine-nonfast-large-full-model-gemma12-review.json`.
-The existing opt-in 4-bit kernel now selects native eight- or sixteen-value
-lane arithmetic for those shapes. The flag remains off by default, and the
-3/8-bit selection remains restricted to its original shapes. Large nonfast
+The then-existing opt-in 4-bit kernel selected native eight- or sixteen-value
+lane arithmetic for those shapes, off by default, with the 3/8-bit selection
+restricted to its original shapes; that kernel and its flag were removed in
+the MLX 0.32.2 consolidation (§7.12) and survive only in frozen source controls. Large nonfast
 output heads with N≥65536 remain native because no such head has passed the
 model/serving performance gate. This does not change the measured Gemma
 projection selection. Focused tests
@@ -2659,16 +1603,45 @@ fast RoPE. Evidence: `mrope-reuse-screen-review.json` and
 `mrope-stride-full-model-packed-review.json`. Closed helpers are removed; raw
 reports and source hashes remain.
 
-The final M4 Pro bundle must record incumbent and candidate source hashes,
-local diff, Bun/native/oracle versions, model hashes, flags and workload files.
-Run the existing serving suite and native generation separately on the M4 Pro,
+The final quiet-machine bundle must record host/chip/RAM, incumbent and
+candidate source hashes, local diff, Bun/native/oracle versions, model hashes,
+flags and workload files.
+Run the existing serving suite and native generation separately on a quiet machine,
 with fixed cache conditions and all paired samples retained. Its existing
 numbers are historical controls until the workload and version metadata match.
 
 ### 7.10 Final benchmark and HTML comparison report
 
+Request tracing separates the gateway’s request-slot wait (`engine.request_wait`)
+from execution-lock or row-admission wait (`engine.admission_wait`). Cache
+prefetch/restore preparation has its own `cache.prefetch` span before execution;
+later cache selection remains `cache.lookup_restore`. The prompt-response
+report sums the two queue spans and the two cache spans in their respective
+columns. Historical records without the new spans omit that work and must not
+be interpreted as complete queue attribution. The shared serving HTTP reader
+accepts a trace ID to correlate server phases with each scheduled request.
+
+The offline renderer and replay fixtures are implemented. It imports the
+existing native/serve formats and saved Pi task results, retains failures and provenance, and suppresses
+speed ratios for incompatible work or unqualified runs. Its current metrics
+and limitations live in benchmarks.md. A bounded MiniCPM M1 diagnostic passes
+the real measurement path with the explicit reference control; concurrent
+output differences remain visible. Fixed-source block log-ratio intervals now
+use a deterministic bootstrap across distinct workload seeds, with duplicate
+seeds excluded and at least five blocks required. Task imports load adjacent
+`quality.json` checks with file hashes, report failures and blocked categories,
+and retain source/settings, per-turn request hashes, timing, usage and final
+SSD flush records. Task rows do not enter engine-speed pairing. The read-only
+quality-ledger exporter supplies portable model accuracy/KL/size records,
+with source rows, configuration and machine provenance retained. Legacy
+dataset/artifact identity limitations remain explicit; importing history
+does not establish current-artifact quality acceptance. The frozen
+final protocol and quiet final matrix remain acceptance work;
+rendering saved evidence does not close this gate.
+
 After the optimization gates close, build and run a repeatable benchmark for
-the incumbent, combined finalists and retained quantizations on the M4 Pro.
+the incumbent, combined finalists and retained quantizations on a quiet
+machine, recording its host/chip/RAM.
 The benchmark must preserve the existing suite's measurements while making
 native generation, serving, memory and quality comparisons easier to inspect.
 Its protocol is frozen before final runs; the held-out Kanban task in section
@@ -2758,7 +1731,7 @@ them. Embed the report's required assets so the exported file works offline.
 Exit requires a model-free replay fixture proving schema validation, unit
 definitions, pairing, failure handling and renderer escaping; a bounded live
 smoke through the real measurement paths; then the frozen final matrix on
-the M4 Pro. Compare the old suite and new suite on shared cells to
+a quiet machine. Compare the old suite and new suite on shared cells to
 detect measurement changes before attributing a difference to the engine.
 Publish qualified numbers only in the canonical benchmark reference, with
 raw JSON and HTML under `reports/`. Machine access or unsupported cells that
@@ -2842,8 +1815,8 @@ ordinary first-token reads. All servers exit cleanly with equal active
 capacity, and all 547 source hashes remain fixed. Complete-request paired
 medians for concatenation improve only 0.33%, 0.06% and 0.23% across the code,
 explanation and JSON fixtures. The `evalAll` alternative improves 0.26% and
-0.04% but regresses 0.44% on JSON. Losses remain in both results. These M4 Pro
-diagnostics establish no broad serving benefit worth another sampler path.
+0.04% but regresses 0.44% on JSON. Losses remain in both results. These
+diagnostics (M4 Pro) establish no broad serving benefit worth another sampler path.
 Evidence: spec-grouped-greedy-screen.json, spec-grouped-greedy-eval-screen.json
 and spec-grouped-greedy-three-http{,-review}.json. This tests target verification
 readback only, not DFlash2's architecture or the separate forced-token method.
@@ -2862,7 +1835,7 @@ Priority is the bounded M1 scalar operation screen, current-artifact long-agent
 memory/context coverage, then Qwen DFlash2 feasibility. Only positive operation
 economics and quality evidence justify a new full-model loader or artifact.
 Approximate prompt reduction follows the exact-path work. No external speed
-or memory figure changes our baseline; remaining runs stay on the M4 Pro.
+or memory figure changes our baseline.
 
 The bounded Bun/Metal M1 screen now preserves all 54 fixture cases, including
 online rotation, dtype promotion and incomplete packed rows. Six fresh-process
@@ -2886,30 +1859,18 @@ zero/subnormal/overflow scale values. Six fresh processes then preserve all 84
 large-projection comparisons at M257/512, including rotation and native matmul,
 with identical cross-process hashes and zero final active allocation. Median
 complete-operation time falls 40.85–57.51%; peak temporary allocation falls
-765.00–765.05 MiB. Every pair improves. These are M4 Pro diagnostics on synthetic
+765.00–765.05 MiB. Every pair improves. These were measured on the M4 Pro on synthetic
 resident matrices. They do not measure real model quality, RSS or whole-model
 speed, and they do not replace the separate small-M QMM arithmetic. Evidence:
 `polar-expand.json` and `polar-expand-repeat{,-review}.json`.
 
 Our existing `TurboQuantKVCache.#decode` has the same unpack/gather/scale
-pattern. The byte-packed KV decoder passes operation repeats, long-context
-Qwen state, RTN4 and deferred D128/D512 consumer gates. It is integrated behind
-an experimental opt-in, with exact operation/fallback/lifetime checks, six
-HTTP pairs per R6/RTN4 quant and the first integrated 8K state gate. Allocation
-diagnostics distinguish live-array ownership from Metal buffer capacity.
-Six long-context three-arm blocks preserve responses and improve complete time;
-the narrower inverse-rotation operation is now integrated. Its R6 and RTN4
-model and repeated HTTP response gates pass on the new core. Decode improves
-on both; R6 complete time improves while RTN4 complete time remains inconclusive.
-Six HTTP
-pairs each on MiniCPM and Gemma deferred consumers pass response identity and
-improve complete time. Pressure and quiet acceptance remain. Direct packed-key
-attention preserves the native arithmetic but loses
-against the integrated decoder in complete codec/attention repeats; that
-additional speed candidate is closed.
-Its mechanism and evidence
-live in turboquant.md, "Packed-value decode fusion investigation". The video's
-weight format still has no production loader or kernel here.
+pattern. The byte-packed KV decoder and the narrower inverse-rotation
+operation are integrated behind `MLX_BUN_TURBOQUANT_FUSED_DECODE=1`; their
+gates, numbers and the closed direct packed-key attention arm live in
+turboquant.md, "Packed-value decode fusion investigation", and the ledger row
+is §7.13 R13. The video's weight format still has no production loader or
+kernel here.
 
 ### 7.12 Custom-kernel graph reuse and the MLX runtime update
 
@@ -2980,8 +1941,8 @@ continuation. All twelve cells are byte-exact against the matching MLX-LM
 reference. The two new-core builds also agree with each other in those cells.
 Source hashes stay fixed and every worker exits cleanly. Evidence:
 runtime-oracle-rtn4.json and runtime-oracle-rtn4-macos14.json.
-The working tree now selects the verified macOS 14 native-pack candidate
-0.4.0 from a local cache. It has not been published. Final-source acceptance
+The package selects the verified macOS 14 native pack 0.4.0, now published;
+distribution.md records the release. Final-source acceptance
 passes 16 RTN4 same-version reference cases (including actual 4K/8K prompts),
 36 Trellis forward/state/continuation cases and six Trellis generations.
 The Trellis control is the accepted new-core implementation; this is not a
@@ -3019,7 +1980,10 @@ for every tested 2/3/4-bit cell. A full-model prototype comparison passes all
 Those timings are screening evidence, not serving results.
 
 The integrated implementation keeps geometry and arithmetic in
-`trellis-wide-prefill.ts`. Qwen supplies an explicit layout proof from its
+`trellis-wide-prefill.ts`. It is dispatched only under
+`MLX_BUN_TRELLIS_VARIANT=11..13` (`TrellisLinear.forward` in
+`src/model/trellis-linear.ts`); the default variant 6 still expands and uses
+native matmul, so this path is opt-in, not production. Qwen supplies an explicit layout proof from its
 RMSNorm output through the existing MLP and linear calls. Unproven layouts
 return to native expansion/matmul for the affected sizes. Reading lazy-array
 strides before evaluation cannot establish this proof, and inserting an eval
@@ -3064,8 +2028,8 @@ Both append lengths, one and eight tokens, pass at both contexts. Full
 logits, every live cache array and the fixed continuation are byte-exact;
 both workers exit cleanly with fixed source hashes. Evidence:
 runtime-oracle-long-macos14.json and
-native-runtime-screen{,-review}.json. These runs used the loaded M4 Pro and
-are not canonical benchmark rows.
+native-runtime-screen{,-review}.json. These runs used a loaded machine (M4 Pro)
+and are not canonical benchmark rows.
 
 A native proxy prototype establishes a narrower graph-reuse option. It holds
 the original custom operation, delegates evaluation to MLX's existing encoder,
@@ -3176,6 +2140,265 @@ Their reported gains are not our measurements. New [GQA8 attention reuse](https:
 has a narrow D64/128 dispatch; Qwen3.8-27B uses GQA6/D256, so it is an algorithm
 lead rather than a direct dispatch win here. The [optional fused full-attention
 path](https://github.com/ml-explore/mlx/pull/4185) explicitly trades throughput
-for bounded temporary memory. Evaluate that tradeoff on the M4 Pro's actual
+for bounded temporary memory. Evaluate that tradeoff on the actual
 eligible path. Reference numerical changes must be explained, never hidden by
 loosening the oracle tolerance.
+
+Qwen layer evaluation bounds deferred prefill work. The exact oracle and
+retained-history results, including superseded reservation trials, are in
+[benchmarks.md](../reference/benchmarks.md). Josh's closeout machine policy
+permits the recorded audio-daemon CPU activity with exclusive GPU ownership,
+available memory and stable alternating pairs. Earlier diagnostic evidence
+keeps its original classification.
+
+Josh subsequently chose request execution over predicted-memory refusal. The
+reservation prototypes remain historical diagnostics. Default fit estimates
+must neither reject prompts nor clamp completion caps; only explicit operator
+limits and fixed representation capacities apply. The Qwen layer evaluation
+fix stays, while reservation callbacks and their hot-path checks are removed.
+Actual failures remain recorded in reliability and timing results.
+
+### 7.13 Outcome ledger
+
+One row per §7.4 experiment, in the §7.7 vocabulary (repeatable win, measured
+loss, inconclusive, blocked, unmeasured). Every campaign number was measured
+on Joshs-MBP-2025.local (Apple M4 Pro, 24 GB) unless a row says otherwise;
+the host is provenance, not scope. All "repeatable win" rows have at least
+six balanced pairs with exact outputs. The original rows below describe
+loaded-machine diagnostics. The closeout serial v6/v13 suite now passes six
+balanced pairs under the recorded audio-CPU allowance, with every paired
+request matching. Final-source native state and bf16/KV4 MTP-prefix gates
+also pass. Later acceptance promoted variant 13 and shared batching in v0.4.0; early
+first output is now also selected. Remaining combinations and measurement
+limits are identified per row. New evidence is in `reports/qwen38-closeout/` and
+[benchmarks.md](../reference/benchmarks.md). Historical evidence paths
+are relative to `reports/qwen38-rd/` on the measuring host; the narrative
+behind each row is the archived log
+([qwen38-27b-campaign-log.md](../archive/investigations/qwen38-27b-campaign-log.md)),
+§7.9–§7.12, [turboquant.md](turboquant.md) "Packed kernel experiments" and
+"Packed-value decode fusion investigation", and
+[speculative-decoding.md](speculative-decoding.md) §4.7/§7.4.
+
+| ID | Experiment | Outcome | Retained flags / defaults | Evidence files | Next discriminating step or missing resource | Reopen condition |
+|---|---|---|---|---|---|---|
+| R0 | Exact artifact and execution controls | **Inconclusive.** Isolated unmodified `6d45ca1`/`673b43f` checkouts: six native AB/BA pairs per flagship/compact/RTN4 match every ID at 6/128/512 prompt tokens with flat TTFT/complete/memory; six serial and six default-scheduler HTTP pairs preserve every response with flat or slightly lower complete time. Updated Bun vs MLX-LM native controls (six pairs per affine artifact, 64 tokens) emit identical IDs, complete time close, small median Bun advantage. Graph exports (v6/v8 at M=1/4/5), dispatch inventory, per-operation counter calibration and a schedule recorder exist; dispatch-level GPU sampling is unsupported on the M4 Pro, so per-kernel attribution stays open. The M4 DeltaNet golden is resolved (stale fixture; Bun and Python agree). The quiet serial/default/mlx-lm HTTP baseline never ran; the packed target has no stock mlx-lm arm. | None (controls only). | `refactor-fixed-kernel-audit-plan.json`, `refactor-native-paired{,-review}.json`, `refactor-http-paired-{serial,default}-review.json`, `native-affine-paired{,-review}.json`, `native-affine-paired/`, `native-memory-ledger.json`, `native-memory-chunk512-screen/`, `graph-captures/`, `variant8-attached.trace`, `attached-trace-summary.json`, `labelled-v8-analysis.json`, `labelled-v8-kernels.txt`, `variant8-labelled.trace`, `metal-counter-capabilities.json`, `mlx-counter-source-manifest.json`, `mlx-counter-build.patch`, `metal-counter-pooled-unit.stderr.txt`, `metal-counter-calibration-review.json`, `mlx-schedule-source-manifest.json`, `mlx-schedule-smoke-v2.log`, `top-p-schedule-screen{,-review}.json`, `empty-submit-r6-steady-trace{,-review}.json`, `mlx-empty-submit-steady-source-manifest.json` | Exclusive M4 time is now available. Finish the final-source serving/reference controls: `bun scripts/bench-serve.ts all --model-path ~/models/Qwen3.8-27B-q3-trellis-ldlq-k300-packed-interleave2-rd --arms mlx-bun-serial --context 4096 --tokens 192 --workload-seed <block>` alternated with the default arm across blocks, plus the RTN4 affine artifact with `--arms mlx-bun-serial,mlx-lm` (the only same-artifact oracle cell); quiet preflight passing; distinct `--out` per block. | Any change to the controlled path: native pack bump, artifact layout change, new kernel default. |
+| R1 | Packed prefill tile decode plus matmul | **Adopted in the v0.4.0 default.** Variant 11 (tiled axis-1 gate/up, M=5..32) and variant 12 (split-K axis-0 down, M=5..8) reconstruct bf16 weights inside tiles with exact full-model logits/state/continuation; six native paired blocks improve eligible M; six raw-completion HTTP pairs per lane improve TTFT and complete time (six-token prompt, 8/64 output tokens); the second tile pass improves eight-request throughput and B=1 TTFT in six pairs. All 40 larger tile geometries at M=128/512 are exact but none beats expansion plus native matmul in six AB/BA pairs. On MLX 0.32.2/M3+ the M5..15 axis-1 path uses the wide-prefill kernel (native `gemv_wide` arithmetic); six HTTP pairs per lane improve TTFT in every pair, decode and RSS unchanged. | Trellis variant 13 is the shipped default; older variants remain explicit controls. | `variant11-full-model-review.json`, `trellis-v11-http{,-review}.json`, `variant12-full-model.json`, `trellis-splitk-{screen,tuning,full-model}-review.json`, `trellis-v12-combined-http{,-review}.json`, `trellis-v12-combined-native-review.json`, `trellis-v12-native-failure-replay.json`, `trellis-tile-layout-screen-review.json`, `trellis-tile-full-model-review.json`, `trellis-splitk-bm8-{other-bits,full-model}-review.json`, `trellis-tile-integrated-forward-review.json`, `trellis-tile-http-review.json`, `trellis-tile-seeded-serial-http-review.json`, `trellis-tile-native{,-review}.json`, `trellis-large-prefill-screen{,-review}.json`, `mlx-upgrade-0.32.2/trellis-wide-{screen,perf,full,integrated-full}`, `mlx-upgrade-0.32.2/trellis-wide-http{,-review}.json`, `mlx-upgrade-0.32.2/trellis-wide-continuous.json` | Existing six-pair serial/default/combined comparisons and final-source native gates are accepted. Reuse them; broader pressure and final report work remain separately tracked in Phase 6. | A tile geometry beating native matmul at M≥128 on real k3 matrices, or a GPU generation that changes native small-M dispatch. |
+| R2 | Bounded expansion scheduling | **Repeatable win (opt-in).** Unrestricted deferral (variant 9) and unrestricted async submission gain on short prompts but fail the saved-agent pressure gate (Metal insufficient memory at the 12,954-token request). The bounded policy (async only while MLX active allocation is below 75% of the recommended working set) passes serial and continuous pressure gates; integrated six HTTP pairs per lane: TTFT -4.00%/-3.96% at 128 prompt tokens (six wins), -1.43%/-1.40% at 512; wall -0.63..-0.88%; decode flat; native M16/128/512 forwards -2.09/-6.19/-2.80% with about +368/403 MB peak allocation. | `MLX_BUN_TRELLIS_ASYNC_EXPAND=1`, variant 13 only, default off. | `variant9-full-model.json`, `trellis-v13-boundary-full-model{,-review}.json`, `trellis-v13-boundary-http-serial{,-review}.json`, `qwen-long-boundary-serial{,-review}.json`, `qwen-long-async-serial{,-review}.json`, `trellis-v13-bounded-full-model{,-review}.json`, `qwen-long-bounded-{serial,default}{,-review}.json`, `trellis-v13-bounded-http-{serial,default}{,-review}.json`, `trellis-async-integrated-{typecheck,hygiene,model-free}.txt`, `trellis-async-production-full-model{,-review}.json`, `trellis-async-production-http-{serial,default}{,-review}.json`, `trellis-async-production-long-{serial,default}{,-review}.json` | The combined variant-13/fused-k8v3/MTP3 seven-request pressure gate passes on M4 after the existing M1 gate: all responses and complete usage match across roomy sync, constrained sync and constrained async arms; both threshold branches execute and persistence completes. See benchmarks.md and `reports/prefill-observation/combined-tq-mtp-pressure{,-review}.json`. Retain opt-in selection: the measured complete-request gain is modest and expansion overlap increases live allocation. Reopen for a changed model/barrier or a materially better timing/memory tradeoff; do not repeat the unchanged pressure gate. | A change to the memory headroom guard or to per-layer cache evaluation boundaries (R16/R17). |
+| R3 | Axis-0 scatter layout and split-K | **Adopted in the v0.4.0 default.** Variant 8 (balanced word-aligned 3-bit scatter, all 32 lanes; k3/T256/L≤12) and variant 10 (scatter decode shared across M=2..4) pass exact full-model gates and improve M=1/2/4 and M=2..4 in six native blocks; row-loop unrolling not retained. A threadgroup integer codebook restricted to the M3/4 k3 interleaved down projection: six pairs -1.44/-2.00% forwards, six MTP HTTP pairs -1.07/-1.27/-1.19% complete time (5/6, 5/6, 4/6 wins), six four-request cohorts -1.49% (4/6); M1 loses and stays procedural; folded into variant 13. A 2.59% M128 control slowdown in one sequence was not reproduced by two controls. Variant 6 unchanged. | Trellis variant 13 is the shipped default; older variants remain explicit controls. | `balanced3-screen/`, `scatter-unroll-screen/`, `aligned3-screen/`, `variant8-paired/`, `variant8-full-model.json`, `shared-scatter-screen/`, `shared-scatter-all-screen/`, `variant10-full-model.json`, `trellis-threadgroup-codebook.json`, `trellis-threadgroup-scatter{,-m3}{,-review}.json`, `trellis-scatter-full{,-repeat}{,-review}.json`, `trellis-scatter-http{,-repeat}{,-review}.json`, `trellis-scatter-prefill{,-review}.json`, `trellis-scatter-sequence{,-review}.json`, `trellis-scatter-http-trace{,-review}.json`, `trellis-scatter-cohort-v2{,-review}.json`, `trellis-scatter-integration-source-change.json`, `trellis-scatter-integration-focused-tests-final.txt`, `trellis-scatter-integration-typecheck.txt`, `trellis-scatter-integrated-full{,-review}.json`, `trellis-scatter-integrated-http{,-review}.json`, `trellis-scatter-integrated-cohort{,-v2}{,-review}.json` | Existing six-pair serial/default/combined comparisons and final-source native gates are accepted. Reuse them; broader pressure and final report work remain separately tracked in Phase 6. | k2/k4 down projections (only k3 interleaved is qualified) or a new bit allocation from R19. |
+| R4 | Axis-1 reduce and gate/up fusion | **Adopted in the v0.4.0 default.** Variant 7 shares each decoded gate/up weight across M=2..4 with fused gate/up, keeping the variant-6 accumulator order; exact matrix/MLP/full-model gates at M=1..5 and six AB/BA blocks retain the complete-MLP gain; the last-position append sweep confirms the M=4/5 cost discontinuity. Sharing activation loads across 2/4/8 output rows per SIMD group loses at k3 M=1; smaller threadgroups do not rescue it. The M=1 decode kernel is unchanged; the 64/128/256-thread and accumulator sweep with spill/occupancy inspection never ran because per-dispatch counters are unavailable. | Trellis variant 13 is the shipped default; older variants remain explicit controls. | `paired-shared-m/`, `variant7/`, `shared-m-full-model.json`, `shared-m-append-lengths.json`, `trellis-shared-rows-k3-r*-m1.json`, `trellis-shared-rows-k3-r*-s*-m1.json` | Missing resource: per-dispatch GPU counters or an Instruments shader profile (unsupported on the M4 Pro) to drive the M=1 thread/rows/accumulator sweep. | Kernel-level GPU timing becomes available, or a k mixture changes gate/up bit widths. |
+| R5 | Decode primitive | **Measured loss.** Every alternate decoder loses at M=1 (M4 Pro): device f32 LUT (all bit widths), constant int16 table (0.802 vs 0.331 ms reduce, 0.747 vs 0.443 ms scatter, 1.474 vs 0.615 ms gate/up), device/threadgroup fp16 tables, and a constant int16 table on MLX 0.32.2 (nearly 2x complete MLP). The integer y-mad rewrite is exact with no useful gain; the readable expression stays. The only retained table is the scoped M3/4 scatter codebook (R3). The activation audit finds the packed fused SwiGLU uses a float32 precise-exp sigmoid unlike compiled MLX: a Lab distinction, no default change. | None. | `lut6-*.json`, `trellis-codebook.json`, `trellis-codebook-scoped{,-review}.json`, `trellis-constant-short-k3-m1.json`, `trellis-device-half-k3-m{1,4}.json`, `trellis-threadgroup-half-k3-m{1,4}.json`, `mlx-upgrade-0.32.2/trellis-constant-{screen,review,invalid-attempt}.json`, `trellis-y-mad-manifest.json`, `trellis-y-mad-k*-m*.json`, `trellis-activation-audit.json` | None for exact rewrites; a changed code family (QTIP/YAQA computed codes) needs a new artifact plus the KL/MMLU/tGSM/rawGSM screen before any kernel. | A hardware generation with faster threadgroup/constant memory relative to ALU, or a new code family with quality evidence. |
+| R6 | Offline repacking without re-quantization | **Repeatable win (opt-in artifact).** Two-block interleaving of k3 axis-0 code words inverts to identical code bytes; prototype readers preserve 35 full-model cases. A separately written lossless artifact (quantizer `--interleave-codes`) read by every kernel from one resident code copy: six native pairs improve single-token/M4 forwards and 64-token generation in every pair with equal peak allocation; six serial and six default-scheduler HTTP pairs improve decode at 6/128/512 prompt tokens in every pair; TTFT small or mixed; RSS flat. | Not a default artifact change; served benchmarks use `Qwen3.8-27B-q3-trellis-ldlq-k300-packed-interleave2-rd`. | `trellis-block-interleave-screen-review.json`, `trellis-block-interleave-full-model-{g2,g10}-review.json`, `trellis-block-interleave-native-generation-g2-interface-review.json`, `trellis-block-interleave-http-{serial,default}-review.json`, `trellis-interleaved-readers-screen-review.json`, `trellis-interleaved-readers-full-model-count-review.json`, `trellis-interleaved-readers-order-full-model-review.json`, `trellis-interleaved-vector-schedule-screen-review.json`, `trellis-interleaved-readers-schedule-full-model-review.json`, `trellis-interleave-integrated-tests.txt`, `trellis-interleave-integrated-tail-tests.txt`, `trellis-interleave-model-free.txt`, `trellis-interleave-artifact-review.json`, `trellis-interleave-artifact-native-fd-review.json`, `trellis-interleave-encoder-tests.txt`, `trellis-interleave-artifact-http-{serial,default}-review.json`, `reports/prefill-observation/interleave-cache-shared-{0..11}.md.json`, `interleave-cache-shared-review.json` | **Accepted.** Six balanced original/interleaved pairs on M4 use the default shared scheduler, KV4/MTP2 and RAM/SSD prompt caching. All requests, complete usage and restart checks match, with decode and cached completion improvements in every pair. Machine activity and timing limits are recorded in benchmarks.md; no additional unchanged serial run is required. | Expanded state-window index variant (bytes for ALU) untested; k2/k4 interleaving unqualified. |
+| R7 | Small-M packed kernel | **Adopted in the v0.4.0 default.** Shared-M variants 7/8/10 reuse decoded weights across M=2..4 (R3/R4); the M=4/5 whole-model cost discontinuity is confirmed and the packed-to-expanded crossover is also a numerical boundary: the M≤8 crossover prototype wins MLP time but changes output bytes for every bit width, so it is Lab and not retained. The wide-prefill kernel (variants 11–13 on MLX 0.32.2/M3+) shares each decoded row across all M=5..15 vectors reproducing native `gemv_wide` arithmetic: 36 forward/state/continuation cases and six generations exact; six HTTP pairs per lane improve TTFT in every pair with decode/RSS unchanged. Fill spans and MTP verifies inherit these curves (R15/R20). | Trellis variant 13 is the shipped default; older variants remain explicit controls. | `shared-m-append-lengths.json`, `m8-screen/`, `mlx-upgrade-0.32.2/trellis-runtime-full`, `mlx-upgrade-0.32.2/trellis-wide-{screen,perf,full,integrated-full}`, `mlx-upgrade-0.32.2/trellis-wide-http{,-review}.json`, `mlx-upgrade-0.32.2/trellis-wide-continuous.json` | M1 qualification of the existing wide kernel fails exactness, including every full-width cell; retain earlier dispatch there. A different pre-M3 kernel or the M≤8 crossover requires its own numerical and quality gates. | Any MLX runtime bump that changes small-M dense dispatch (rerun `trellis-runtime-full`). |
+| R8 | Fused GDN conv/state/SiLU | **Measured loss.** A fused convolution/SiLU/copied-tail prototype matches bf16/f32 outputs and live tail at B=1/2 only with MLX's typed sigmoid expression; it passes full packed-model logits, live state, continuation and speculative rollback. Against an independently materialized baseline its full-model timing is approximately flat with similar peak memory; unpromoted. Side finding landed as default: `mlx_copy` aliasing retained the source buffer; compact state materialization now uses stock DynamicSlice. | None (DynamicSlice materialization is a memory fix, not a speed default). | `gdn-stage-probe.json`, `conv-probe-typed-sigmoid.json`, `qwen-conv-materialized-packed.json`, `materialize-final-packed-comparison.json` | None; the missing instrument is per-dispatch GPU timing of the conv stage. | Attribution shows conv/SiLU above 3% of the decode critical path. |
+| R9 | GDN norm, gates and recurrence | **Measured loss.** Threadgroup layouts, fixed sequence lengths, unrolling, compiled wrappers and typed sigmoid fusion pass all 114 shape cells after correcting a float32 sigmoid rewrite that broke bf16 recurrence. A compiled fixed-length candidate preserves full-model identity but its isolated gains mostly disappear in full forwards; a fused recurrence + output RMSNorm + gate kernel passes 60 isolated cells and small-M full-model checks with a gain too small to integrate. Fixed-shape CompiledFunction over the 16 SSM groups (R16) gives -0.062%/-0.202% median. Production dispatch unchanged. | None. | `gdn-specialization-typed.json`, `gdn-static-full-model-review.json`, `gdn-fused-norm-probe.json`, `gdn-fused-norm-full-model-review.json`, `mlx-upgrade-0.32.2/qwen-fixed-ssm-groups-r6.json`, `mlx-upgrade-0.32.2/qwen-fixed-ssm-groups-rtn4-internal.json`, `mlx-upgrade-0.32.2/qwen-ssm-hot-review.json` | None until prefill-regime attribution shows the recurrence dominates (decode M=1 was the measured target). | Prefill attribution, or a transposed-state layout with a measured M=1 win. |
+| R10 | Parallel/chunkwise GDN prefill | **Prefill attribution complete; no parallel rewrite retained.** Current packed-target controls and an operation observer preserve complete hidden/state hashes at 1k/4k/8k on both Macs. Gated-delta work is roughly 2.6–2.7% of M4 complete prefill, so it is not the principal bottleneck. M1 instrumentation perturbs larger runs. A chunkwise/scan formulation remains Lab because changing reduction order requires a separate quality contract. | None. | `upstream/sources.json` | Attribution is accepted; prioritize the measured expansion/projection/attention costs. A later chunkwise Metal arm needs same-input recurrent-state and held-out quality checks before any adoption. | Not applicable; never opened. |
+| R11 | Affine QMV/QMM and native builds | **Inconclusive.** Shared-M affine kernels won M4/8 (and M3/5/6/7 4-bit, compact 3/8-bit) in six full-model pairs and six HTTP pairs per configuration with exact outputs and flat B1; cross-model, Gemma 12B large projections win while MiniCPM/Llama regress; a compiled-call cache was closed. MLX 0.32.2 (native pack 0.4.0) superseded it: upstream `qmv_wide` implements the same reuse with different arithmetic, so the custom kernel and its flag were removed (§7.12); frozen source controls keep it. New-core affine Q3 gate/up operations beat the old shared ops in all six blocks, Q4 mixed, Q8 loses (operation level); RTN4 short decode flat, long-prefill timing order-dependent; twelve RTN4 oracle cells and 4K/8K prompts byte-exact on both cores. No end-to-end Qwen gain from the runtime change is established. | Native pack 0.4.0 (MLX 0.32.2) is the pinned runtime; no flag. | `affine-qmv-screen.json`, `affine-qmv-compiled.json`, `affine-shared-m-probe{,-initial}.json`, `affine-shared-m-debug-scale-bits.json`, `affine-shared-m-full-model-{packed,rtn4}-review.json`, `affine-shared-m-http-rtn4-review.json`, `affine-shared-m-production-full-model-rtn4-review.json`, `affine-shared-m-shapeless.json`, `affine-shared-m-integrated-unit.txt`, `affine-shared-m-integrated-http-rtn4-review.json`, `affine-shared-m-width-{rtn4,full-model-rtn4}-review.json`, `affine-shared-m-all-bits-*.json`, `affine-shared-m-broader-*`, `shared-affine-frontier-http-{rtn4,compact}-review.json`, `affine-compiled-call-cache-{full-model,http}-compact-review.json`, `affine-shared-cross-model-exactness.json`, `affine-shared-cross-model-{gemma12,gemmae4b,minicpm5,llama1,llama3}-review.json`, `affine-cross-model-matrix-{inventory,screen,screen-review}.json`, `affine-cross-model-matrix-gemma12-repeat.json`, `affine-head-batched-decode-mfirst-{llama1,llama3}-review.json`, `affine-shared-head-http-{llama1,llama3}-overlap-review.json`, `affine-large-projection-*.json`, `affine-nonfast-*-review.json`, `mlx-upgrade-0.32.2/affine-{old,new}-screen.json`, `mlx-upgrade-0.32.2/affine-runtime-repeat{,-review}.json`, `mlx-upgrade-0.32.2/runtime-oracle-rtn4{,-macos14}.json`, `mlx-upgrade-0.32.2/runtime-oracle-long-macos14.json`, `mlx-upgrade-0.32.2/native-runtime-screen{,-review}.json`, `mlx-upgrade-0.32.2/final-source-gates.json` | Quiet-machine six-pair old-core vs 0.32.2 `bench-serve all` on the RTN4 artifact with the same-artifact mlx-lm arm, to separate the runtime change from kernel changes. | A native pack bump, or MLX changing `qmv_wide` eligibility for GQA6/D256 or Q8 shapes. |
+| R12 | Norm/residual/activation/projection fusion | **Measured loss.** Residual-add/RMSNorm kernel (90 synthetic cases exact; six packed pairs at M1/4/8/128): M1 slightly slower, M4/8 flat, M128 varies. Fused affine gate/up/SwiGLU with the typed sigmoid (48 RTN4 cells exact): only M=1 wins in isolation, full-forward gain too small. Multimodal RoPE shared angles/direct strides (108 cases exact): M128 slightly slower, M512 flat, image encoding excluded. None integrated; helpers removed. Full-attention output-gate and QKV concatenation fusions were enumerated but not prototyped. | None. | `residual-rms-screen-scalar-review.json`, `residual-rms-full-model-packed-review.json`, `affine-gateup-debug{,-typed}.json`, `affine-gateup-typed.json`, `affine-gateup-full-model-review.json`, `mrope-reuse-screen-review.json`, `mrope-stride-full-model-packed-review.json` | None until per-kernel attribution shows elementwise/norm dispatches exceed 3% of the decode critical path. | Attribution evidence as above, or a real vision-request benchmark for mRoPE. |
+| R13 | Attention and KV | **Adopted for TurboQuant decoding.** Joint TurboQuant K/V packed decode plus row-grouped inverse rotation at N≥8192: six 8,846-token HTTP blocks on R6 k8v3 improve decode 83.87% and complete time 12.77% versus the original codec, inverse rotation adds 5.39%/1.05%; on MLX 0.32.2 the inverse improves decode 4.69% (R6) and 5.98% (RTN4) in every pair, R6 complete -0.79% all six, RTN4 complete inconclusive; six MiniCPM and six Gemma deferred-consumer pairs improve complete time. Direct packed-key attention is exact but 6–19% slower than joint decode plus native SDPA (closed). Dense register-resident attention loses timing; Steel D256 fails one KL screen (Lab). KV append census: ordinary appends copy only the new row, rejecting the whole-cache-copy hypothesis; no append kernel. bf16-KV rows are unaffected. TurboQuant now composes with shared ordinary and speculative rows, including the accepted long cached/SSD run. | `MLX_BUN_TURBOQUANT_FUSED_DECODE` defaults on; `=0` retains the unfused control. Cache quantization remains an independent opt-in; details in turboquant.md. | `tq-value-*{,-review}.json`, `tq-kv-screen.json`, `tq-kv-independent.json`, `tq-kv-repeat{,-review}.json`, `tq-kv-full{,-review}.json`, `tq-kv-policy-http{,-review}.json`, `tq-kv-long-full{,-review}.json`, `tq-kv-rotation-{screen,repeat,rows-screen,rows-repeat,long,long-http}{,-review}.json`, `tq-kv-rtn4-full{,-review}.json`, `tq-kv-nonfinite.json`, `tq-kv-minicpm-full{,-review}.json`, `tq-kv-gemma-e4b-full{,-review}.json`, `tq-kv-integrated-*{,-review}.json`, `tq-kv-integrated-rtn4-http-acceptance.json`, `tq-kv-memory{,-review}.json`, `allocator-capacity-repro.json`, `tq-kv-inverse-integrated-r6-full{,-review}.json`, `mlx-upgrade-0.32.2/{r6,rtn4}-kv-inverse-full.json`, `mlx-upgrade-0.32.2/inverse-kv-http{,-replacement,-review,-isolation}.json`, `mlx-upgrade-0.32.2/deferred-kv-http{,-review}.json`, `tq-key-sdpa-screen.json`, `tq-key-sdpa-complete-repeat{,-review}.json`, `steel-attention-tune-manifest.json`, `attention-cpu-check.json`, `attention-oracle-check.json`, `attention-full-model-packed-128-512-review.json`, `dense-attention-memory-tune-review.json`, `kv-append-donation-probe{,-review}.json`, `kv-append-model-trace-2049{,-review}.json`, `kv-append-http-text-census{,-review}.json`, `kv-append-native-8193{,-review}.json` | The combined pressure gate is complete on both Macs. Current default shared off/default comparisons pass on M1 MiniCPM and M4 packed Qwen/MTP3; cached complete time improves in both orders, with mixed short-Qwen timing documented in benchmarks.md. The earlier native, six-pair serial and deferred-consumer gates remain accepted; no unchanged gate needs repeating. | MLX ships fused quantized SDPA for GQA6/D256, or a new codec shape/model exposes a numerical or complete-request regression. |
+| R14 | Vocabulary head and sampler | **Adopted normalized greedy kernel; other candidates remain characterized separately.** Same-input model/operation checks pass on both Macs. MiniCPM standard AB/BA throughput improves and packed Qwen MTP3/TQ stays effectively flat. **Inconclusive.** Compiled Metal grammar bitmask (`MLX_BUN_TOKEN_MASK=metal`) preserves mask bytes and sampled IDs at three vocabularies and beats the host loop in eight paired blocks, but six compact HTTP pairs at B1/B4 are inconclusive (B1 flat, B4 regresses under residual swap); host masking remains default. Top-p inverse-permutation removal is exact in 216/180 cases; Qwen packed serial six pairs +0.183% complete time (loss), Llama 1B -0.399%/-0.380% with six wins per lane; no production change. DSA sort keys reduced to uint32 are integrated as a shared-operation win (selection latency -10.47..-15.80%), not a Qwen result. Grouped greedy verify readback closed (R21). The exact fused matvec/argmax head was never prototyped. | Shared normalized greedy selection is the default. Grammar-mask Metal selection remains opt-in; DSA uint32 keys are integrated. | `token-mask-probe.json`, `token-bitmask-unit.txt`, `token-bitmask-grammar-metal.txt`, `token-mask-sampling{,-review}.json`, `grammar-mask-http-compact-review.json`, `top-p-scatter-screen-v2{,-review}.json`, `top-p-scatter-compiled-screen{,-review}.json`, `top-p-lifetime-{source,native,scatter,metal}.json`, `top-p-sampler-screen-v2{,-review}.json`, `top-p-native-screen{,-review}.json`, `top-p-http-screen{,-review}.json`, `top-p-http-r6-serial-repeat{,-review}.json`, `top-p-http-llama-repeat{,-review}.json`, `dsa-order-u32-{screen,repeat}{,-review}.json`, `dsa-rank-u32-{screen,repeat}{,-review}.json`, `dsa-model-screen.json`, `dsa-rank-u32-model-tests-v2{.txt,.transform.json}`, `dsa-production-{unit-v2,typecheck-v2,model-free,hygiene}.txt`; `reports/prefill-observation/sampler-*` | Quiet-machine six-pair grammar-mask host vs metal at B1/B4 on a structured-output fixture (the only open candidate with a positive operation result). | Head attribution shows the vocabulary projection or readback material for greedy requests. |
+| R15 | Deterministic no-verification replay | **Repeatable win (opt-in).** Strict fill has parser-state guards (complete `</parameter>` delimiter, request-local tool-only context), a schema open-object compiler fix, and model-owned four-token append chunks that reproduce M1 arithmetic via broadcast `[M,1,K]` weight views and split at MLX attention transitions (KV 1023/1024/8192/32768/65536 on `applegpu_g16s`). Final internal-SSD comparison: twelve balanced pairs, 96 exact responses, zero verification; complete time -9.83%/-15.40% (R6 weather/bash) and -1.63%/-2.94% (RTN4), every pair improving; tool output is buffered so this is not a decode-rate result. Staged-body prototype flat, closed. Quantized-KV composition: TurboQuant k8v3 passes after a lease fix, affine KV4/KV8 now preserve complete state after one-token attention is retained within shared projection appends. The qualified model binding declares KV4/KV8 and K8V3 support; delayed conversion passes real-generation state checks. Both quants retain exact HTTP responses in six-pair affine and TQ comparisons. Saved tool-history fixtures pass 20 affine pairs; held-out task quality and quantized verification remain separate. Shared strict fill now composes the common preparation, sampler, row state and RAM/SSD cache. Its qualified B1 append, delayed precision transitions and B2 retirement into B1 pass full hidden/cache/continuation checks; wider rows preserve one-position arithmetic. Six M4 HTTP arms complete 108 responses with exact paired outputs/counts/cache coverage and durable SSD restarts. The sequential fixture total improves 13.50%/13.57% against shared fill-off and remains within 0.13% of serial strict fill; concurrent latency tradeoffs remain. See benchmarks.md and reports/prefill-observation/shared-fill-http*.json. | `MLX_BUN_FILL=strict`, `MLX_BUN_FILL_APPEND_CHUNK_SIZE=0` (model limit), `MLX_BUN_FILL_MAX_SPAN=32`; default off. | `fill-parser-boundary-r6.json`, `fill-boundary-tokenizer-audit-v4.json`, `fill-boundary-cpu-tests-v8.log`, `fill-boundary-native{,-m1,-batch1,-batch1-v2-rtn4}-r6.json`, `fill-first-layer-{shape-probe,batched-m1}.json`, `fill-boundary-http-{screen,repeat,review}.json`, `fill-append-*.json`, `fill-append-final-model-free.log`, `fill-append-state-ownership-native-test.log`, `fill-schema-*`, `strict-fill-variant{6,7}.txt`, `strict-fill-v1{0,1}-cap{32,4}-served-eos.json`, `fill-adversarial-state-v12-eos-review.json`, `fill-state-v12-cap{32,4}-{eos,seed42}.json`, `fill-identical-input-append-review.json`, `fill-chunked-*-review.json`, `fill-integrated-{state-v12-seed42,http-packed}-review.json`, `fill-chunked-integrated-{unit,model-free,typecheck,hygiene}.txt`, `fill-staged-unit.txt`, `fill-staged-state-v13-seed42-review.json`, `fill-staged-complete-requests-review.json` | Frozen held-out real-session/adversarial corpus through `MLX_BUN_TEST_FILL_MODEL=<dir> MLX_BUN_FILL=strict bun test tests/parity/fill-strict.test.ts` on both quants (token IDs, stop decisions, subsequent logits/state), then a quiet six-pair HTTP repeat. | A Qwen tool-template or parser change, tool schemas with pattern properties, or a chunk-limit qualification on another GPU. |
+| R16 | MLX graph and loop | **Earlier output adopted.** Early first token in the serial loop: six native pairs improve first-token latency at 6/128/512 tokens with exact tokens/state/usage; six chat HTTP pairs (84 requests, 42 exact) improve first visible output with complete time flat; raw prompts show no client gain because token 220 emits no event. The suffix screen finds a prefill-shape effect after a one-token return (seven packed/six RTN4 cases change state); explicit M=1 alignment restores exactness. The final caller-owned alignment passes both-machine state/continuation checks and monitored M4 serving comparisons; first visible output improves with complete time close. Closed: shape-specific compiled-call cache, Metal launch-config cache (MiniCPM HTTP mismatch unresolved), untouched-finalize elision (-0.069%), submission/traversal limits for Qwen (Llama 25/25 gives -2.44% native and -1.86..-3.02% HTTP, diagnostic, not integrated), fixed-input proxies, per-layer graphs (RTN4 fails identity), whole-model graph (slower), SSM-group compilation (-0.06%/-0.20%). Shapeless custom kernels are an upstream MLX limitation. | `MLX_BUN_EARLY_FIRST_TOKEN` defaults on; `=0` retains the old output order. | `first-yield-*`, `first-token-production-long-{serial,continuous}{,-review}.json`, `first-token-lifecycle-{llama,qwen}{,-review}.json`, `affine-compiled-call-cache-{full-model,http}-compact-review.json`, `metal-config-cache-*`, `mlx-untouched-finalize-source-manifest.json`, `untouched-finalize-*`, `graph-schedule-screen-plan.json`, `graph-schedule-screen{,-review,-contamination}.json`, `graph-schedule-rtn4-rescreen-v2{,-review}.json`, `graph-schedule-llama-repeat{,-review}.json`, `graph-schedule-llama-http-repeat{,-review}.json`, `mlx-upgrade-0.32.2/custom-kernel-compile-probe.json`, `mlx-upgrade-0.32.2/fixed-custom-probe.json`, `mlx-upgrade-0.32.2/qwen-fixed-region-{r6,rtn4}-v{2,3,4}.json`, `mlx-upgrade-0.32.2/qwen-fixed-layer-{r6-full-v2,rtn4-full}.json`, `mlx-upgrade-0.32.2/compiled-region-probe.json`, `mlx-upgrade-0.32.2/qwen-fixed-hidden-regions-{r6,rtn4}.json` | Early-output native, HTTP, cancellation/overlap and cached/SSD acceptance is complete; see benchmarks.md for monitored repeats and activity limits. The MLX 0.32.2 submission screen is complete: retain upstream startup defaults, with the measured M4 Llama override available explicitly. The current native API has no model-scoped setter; revisit integration if that interface becomes available. See benchmarks.md for the cross-model results and activity records. | MLX adds shape-inferring CustomKernel or graph replay, or per-token host-overhead attribution changes. |
+| R17 | Hybrid prefix and recurrent state reuse | **Implemented and accepted for generated reuse.** One cache owns immutable target/recurrent state and draft companions, original generated token IDs, session/content lookup, RAM residency and queued SSD persistence. Native and HTTP checks cover continued mutation, cancellation, earlier-boundary fallback and fresh-process restoration on both Macs. The saved Kanban miss/OOM sequence passes with identical output; the completed session-cache Kanban run hits every follow-up and flushes all snapshots durably. Its app-quality defects remain separate from cache acceptance. | Shared RAM/SSD publication and generated-token provenance are enabled; whole-file SSD and LRU remain selected after measured alternatives. | `reports/kanban-cache-fixed-fresh/`, `reports/kanban-cache-fixed-repeat-r1/`, `reports/kanban-session-cache-r2/`; [cache implementation](kv-cache.md#54-background-persistence-and-ram-residency) and [measured task comparison](../reference/benchmarks.md#full-kanban-with-session-cache-and-queued-persistence). | No missing generated-history implementation. Reuse the completed native/HTTP and full-task evidence; do not rerun unchanged gates. | A reproduced history-alignment, retention, durability or reuse regression. |
+| R18 | Scheduler | **Early output and shared batching adopted.** Continuous-scheduler early yield (same `MLX_BUN_EARLY_FIRST_TOKEN` flag): six AB/BA blocks, 84 responses, 42 exact, median TTFT -7.66% code / -7.90% explanation / -5.63% JSON (74–77 ms) with complete time +0.04..+0.15%; the integrated one-block check gives -68..-90 ms; 120 streaming cancellation/arrival requests pass on both quants and lanes. Concurrency evidence for kernels: v6 vs v12 four-request waves improve every pair and the tile pass improves eight-request throughput (R1). The historical four-request kernel waves predate shared cohorts. B-wide prefill is now adopted, and current default serving uses shared ordinary/speculative execution; mixed iteration work is measured and remains off. Two opposite-order fixed-arrival repeated-wave comparisons now preserve all 84 paired outputs and improve aggregate throughput at loads two/four/eight. First-request completion and stream gaps retain a tradeoff; see benchmarks.md. Longer steady-state capacity remains separate. | Early first output defaults on; `MLX_BUN_EARLY_FIRST_TOKEN=0` retains the control. | `first-prepare-scheduler-{control,candidate}.txt`, `first-prepare-http-screen-plan.json`, `first-prepare-http-{screen,repeat}{,-review}.json`, `first-prepare-production-{unit,model-free,typecheck,hygiene}.txt`, `first-prepare-production-http-r6-v2{,-review}.json`, `first-token-lifecycle-{llama,qwen}{,-review}.json`, `trellis-concurrent-http{,-review}.json`, `trellis-tile-http-review.json`, `trellis-scatter-integrated-cohort-v2{,-review}.json` | The fixed-arrival repeated-wave cell is complete with preserved outputs and documented latency tradeoffs. Longer stationary capacity remains a distinct measurement; do not repeat the accepted wave test without a relevant source change. | A change to admission, mixed-work scheduling or output/cache boundaries. |
+| R19 | Weight-quality allocation | **MLP cost/allocation screen complete; quality acceptance open.** Actual k2/k3/k4 fused gate/up and down costs are measured at M1/M3/M4/M16 on both Macs in forward/reverse order. All 228 unchanged-reader checks match. A discrete size-constrained objective runs 64 machine/row/latency-weight cells; zero weight reproduces k300 exactly. Modest latency weight finds little estimated decode headroom. Aggressive M1-prefill proposals trade more external-codec sensitivity loss for time. Those losses are ranking proxies, not Trellis quality. No candidate is quantized or promoted. Existing Q3 and lossless-interleaving quality evidence remains unchanged. | Keep the existing artifact. | `reports/prefill-observation/r19-cost/`; benchmarks.md latency-aware MLP allocation screen | Teacher-based quantization and frozen KL/MMLU/tGSM/rawGSM screens; non-MLP allocation and calibration/rounding choices remain open. Checked primary model directories lack a complete original Qwen bf16 snapshot; promoted bits cannot be recovered from k300. | No measured full-model speed or quality claim from the proposal cost model. |
+| R20 | MTP economics | **Repeatable win (opt-in).** Six balanced three-arm blocks versus native full-prefill on the packed artifact: fixed two drafts change complete time -43.22%/-38.81%/-44.92% (code/explanation/JSON) with every pair improving, TTFT -5.5..-8.4%, peak RSS +653 MiB; RTN4 -34.95%/-26.95%/-38.48%. Adaptive one-to-three drafts adds about 3% (ratios 0.9698/0.9782/0.9624) and stays research-only. Folded 4-bit/8-bit MTP heads: -1.48/-3.67/-2.95% and -0.65/-2.29/-1.86% with peak RSS 12.53/12.74 vs 13.04 GB; 1,184 vocabulary rows and 48 state samples byte-exact against serial replay. EOS and finish-reason accounting fixed. The quiet M1 Max control (0.821x at gamma 2, 2026-08-18) predates the kernel work. Shared serving has short depth-2/3/4 comparisons, native depth-3 coverage and completed late-context MTP1/2/3 controls. Current KV4/MTP2 and fused K8V3/MTP3 both pass the 78,678-token cached/SSD workload. KV4/MTP2 remains the measured long-agent recommendation; depth 3 crosses a different KV4 value-product dispatch boundary. See benchmarks.md. | `--draft-kind mtp --num-draft-tokens 2` opt-in; EOS/finish-reason fixes landed. | `mtp-v*-gamma*.json`, `mtp-corpus-screen/`, `mtp-screen-prompts.json`, `mtp-heldout-{plan,http,http-review}.json`, `mtp-heldout-native{,-review}.json`, `spec-token-counts-*.txt`, `mtp-post-eos-http{,-review}.json`, `spec-eos-serving-*.txt`, `mtp-eos-budget{,-integrated}-http{,-review}.json`, `spec-eos-continuous-*.txt`, `eos-budget-continuous{,-v2}-http{,-review}.json`, `mtp-quantized-draft-*-http{,-review}.json`, `mtp-draft-artifact-q4-{build,audit}.json`, `mtp-draft-quant-oracle.json`, `mtp-draft-artifact-q8-build.json`, `mtp-quantized-loader-*.txt`, `mtp-artifact-ownership{,-v2}.json`, `mtp-artifact-{screen,repeat}-http{,-review}.json`, `mtp-artifact-native{,-context}{,-review}.json`, `mtp-artifact-context-screen-http.json`, `mtp-matched-prefill-{r6,rtn4}-{screen-http,repeat}{,-review}.json`, `mtp-rtn4-screen-{plan,http,http-review}.json`, `mtp-rtn4-divergence-v2-audit{,-review}.json`, `mtp-rtn4-prefill-control{,-review}.json`, `mtp-v10-{planets,prose}-gamma{2,3}.json`, `mtp-provider-dispose-{before,after}.json`, `mtp-provider-ownership-tests.txt`, `mtp-post-dispose-http.json` | Late-context KV4 depths 1/2/3 and ordinary controls are complete; the current ordinary cached-repeat failure is resolved. TQ/MTP3 passes the original context, RAM repeat and SSD restart. Current-source KV4/MTP2 and automatic prefill sizing pass; KV4/MTP2 remains the measured agent recommendation. | Any change to verify-window arithmetic (KV4 head pairing, fused KV) or a DFlash2 drafter (R21). |
+| R21 | Other exact speculative sources | **Shared lookup implemented; DFlash2 unimplemented.** Qwen DFlash2 (the z-lab artifact behind the oMLX record the video cites; block size five, native MTP off) needs dynamic grouped causal convolutions around attention and MLP plus a top-16/rank-256 selector; the existing DFlash provider does not implement that architecture, so loading the checkpoint would not be DFlash2. Grouped greedy verify readback is closed: six three-arm blocks, 84 exact comparisons, concatenation improves only 0.33%/0.06%/0.23% and the evalAll variant regresses JSON 0.44%. Prompt lookup is now adopted through the shared variable-proposal verifier, with both-machine bf16/KV4/KV8/TQ B4 serving, generated RAM/SSD and MTP regression checks. M4 arm orders show small single-request gains and lower aggregate throughput with changed concurrent outputs; it remains optional. The DSpark/DFlash execution provider has seeded-fixture coverage, but no trained checkpoint exists. This does not implement DFlash2. SpecPrefill remains a separate approximate Lab arm. | Shared prompt lookup is available; trained-drafter performance remains unmeasured. | `video-dHK90xc9Q64/`, `video-dHK90xc9Q64/dflash-source-manifest.json`, `spec-grouped-greedy-screen.json`, `spec-grouped-greedy-eval-screen.json`, `spec-grouped-greedy-three-http{,-review}.json` | Missing resource: a DFlash2 architecture/selector port with target-state/rollback identity and draft residency, then a three-arm block (ordinary / fixed-two MTP / DFlash2) on the held-out prompts. | The port exists, or an exact n-gram source shows at least 3% on the saved Kanban history. |
+| R22 | Residency and shared memory | **mmap loss; range-construction improvement.** Page-aligned file-backed weights through Metal roots are exact in every payload/output case, but six native pairs give +0.25% (later +0.02%) on 64-token generation with no faster pair, six serial HTTP pairs +0.04..+0.14% complete and +0.14..+0.21% decode, and the balanced cold-start follow-up is worse in five of six pairs (+18.42% first token, +12.93% first generation). Peak RSS falls from 12,182,749,184 to 482,508,800 bytes as file-backed accounting only; physical RAM saving unproven. A read-only donation hazard was found and fixed in the prototype before closing; helpers removed. Retained as memory correctness, not speed: the 8 MiB bounded range cache in `mlx/ops.ts` (Kanban sustained run peak active -1,582,612,480 bytes, +2.171% time in one pair) and per-chunk draft-KV evaluation. | Native long position ranges reduce operator/host-allocation work without a material model/serving speed claim; bounded range cache and per-chunk draft-KV evaluation remain memory fixes. | `mmap-page-window{,-pointer}-screen.json`, `mmap-weights-native-{llama1,packed}-screen.json`, `mmap-weights-native-packed-paired{,-review}.json`, `mmap-weights-http-serial{,-review}.json`, `mmap-weights-ownership-packed{,-mapping}.json`, `mmap-weights-single-consumer-lazy{,-mapping}.json`, `mmap-weights-whole-consumer-lazy.json`, `mmap-whole-consumer-source-integrity.json`, `mmap-donation-{ro,cow}-screen.json`, `mmap-managed-donation-screen.json`, `mmap-weights-managed-whole-consumer-lazy{,-mapping}.json`, `mmap-managed-submitted.json`, `mmap-weights-managed-native-{repeat,screen}{,-review}.json`, `mmap-initial-residency-attribution.json`, `mmap-lazy-view-ownership.json`, `mmap-weights-lazy-managed-native-screen{,-review}.json`, `mmap-prefault-native-screen.json`, `mmap-cold-native-repeat{,-review}.json`, `mmap-managed-shard-release{,-mapping}.json`, `mmap-managed-active-accounting-review.json`, `kanban-arange-{allocation-evidence,regression,micro-memory,reconstruction}.json`, `kanban-sustained-77k-owned-control.json`, `kanban-sustained-arange-comparison.json` | Native long int32 positions are accepted as an operator/host-allocation optimization after exact range/view/bound checks, six monitored M4 complete-forward pairs and two standard serving arm orders. Model and serving performance are effectively flat; the initial unmonitored regression is not reproduced. A reusable power-of-two prefix remains unselected after an inconsistent M1 screen. See benchmarks.md for activity records and limits. | A controlled page-cache-cold startup cell (§7.10) exists; mmap lost warm but cold was never controlled. |
+| R23 | Media and task completion | **Inconclusive.** The fresh Pi Kanban task on the 12.14 GiB packed target with Luke's `qwen3.8-27b-q3_k_xl-coding-128k` profile (131072 context, xhigh thinking, temperature 0.6, seed 42; KV4, MTP depth 2, paired prefixes, MLX 0.32.2) completed in 3h 33m 7.685s with 130,494 output tokens over 62 requests, 79.706% draft acceptance, 11.916 GiB peak combined RSS; the untouched app passes the functional browser checks. Earlier attempts failed on a tool-parser bug, dropped reasoning history, Metal OOM at the 85,238-token request and OOM during sustained decode. The later accepted R17 session-cache Kanban run completes in 83m57s with 76,031 output tokens, every follow-up cached and durable final persistence; its two application defects remain recorded. The automatic-prefill task misses the app-quality target, while the completed fixed-256 integrated control meets it and reproduces the earlier initial response. Keep the task-comparison profile fixed; results and browser evidence are in benchmarks.md. No matched successful original-source run exists, so these different trajectories cannot supply an engine-only task-time speedup. Gemma image/audio shared execution now passes both-machine native/HTTP and monitored M4 ABBA comparisons with all 78 paired responses/counts unchanged; concurrent gains and individual latency costs are recorded in benchmarks.md. Qwen image/video shared execution also passes both-machine source/group/HTTP and M4 ABBA checks; full usage matches, while changed concurrent geometry changes some text. Qwen encoder objects now reuse the shared RAM/SSD cache, with both-machine native/HTTP, fresh-process SSD and M4 source-controlled timing acceptance. First-use weight hashing costs more; repeated images improve with exact responses/usage. Gemma encoder reuse also passes both-machine identity/HTTP and M4 ABBA/fresh-process SSD acceptance, with all 168 paired responses/usage exact and repeated-request gains. Gemma and Qwen generated media KV reuse now pass both-machine native/HTTP, fresh-process SSD and M4 ABBA comparisons. The shared media-prefix option is selected by default; output and precision boundaries remain documented. | Tool-parameter parser and `reasoning_content` alias landed as correctness fixes; no speed default. | `lukes-kanban/`, `kanban-final/optimized-mtp-invalid-32k-sleep/`, `kanban-final/luke-q3-128k-xhigh-{history-fixed,tool-values-fixed,mtp-memory-fixed,arange-bounded}/`, `kanban-final/comparison.html`, `kanban-capacity-110k-control.json`, `kanban-capacity-8k-final-comparison.json`, `kanban-capacity-110k-owned-final.json`, `kanban-memory-http-{owned,responsiveness}.json`; numbers in benchmarks.md Kanban sections | The requested integrated fixed-profile task is complete. An original-source run is needed only for an engine-only task-time claim; no such ratio is claimed. Gemma image/audio shared serving acceptance is complete. Qwen image/video shared serving acceptance is complete with documented geometry and latency limits. Qwen encoder-cache acceptance is complete. Gemma encoder-cache acceptance is complete. Prepared Gemma/Qwen media-prefix reuse is accepted through the same cache interfaces. Chunked media prefill and wider configurations remain separate work. | No engine-only task-time ratio without a matched baseline. R17 generated-history acceptance is complete and does not need another unchanged task run. |
+| R24 | Larger departures | **Inconclusive.** Feasibility and operation screens exist. For the video's tq3-mini codebook format (eight-centroid indices, ten codes per uint32, 3.2 bits/value), the literal upstream `polar_qmv` through the Bun binding on synthetic matrices: explicit SIMD shuffles cut the 5120-to-17408 rotated operation 2.77–3.84% (every pair), and a fused fp16 expansion cuts large-prefill operation time 40.85–57.51% with about 765 MiB less temporary allocation. No production loader or kernel, no quality evidence; the checkpoint's reported agent failures are unresolved. Section 7.14 now records kernel, quality, memory and potential-benefit requirements for those candidates. A both-machine same-weight native activation-quantization screen is a measured loss; static size/training bounds are recorded. Actual-weight unweighted low-rank and sparse-residual screens now reject their simple variants. Packed forwards fail differentiation without a custom VJP; the expanded frozen-weight control succeeds. Calibrated quality and packed backwards remain open. | None. | `video-dHK90xc9Q64/`, `polar-qmv-screen-v2.json`, `polar-qmv-variants.json`, `polar-qmv-geometry.json`, `polar-qmv-repeat{,-review}.json`, `polar-expand.json`, `polar-expand-repeat{,-review}.json` | Next: activation-calibrated candidates and an explicit packed backward operation, if pursued after the measured simple-variant losses. The tq3-mini format still needs a loader and actual-checkpoint KL/MMLU/tGSM/rawGSM before a model claim. | An artifact with quality evidence at or below Q2b bytes, or the operation gains reproduced whole-model. |
+
+Josh's original direction is to "only ask the model to decode tokens that we
+can't reasonably already know." This includes reasoning snippets copied into
+tool arguments, repeated atomic values such as URLs, and predictable tool JSON
+structure and field order. The goal is deterministic continuation wherever the
+application can establish the text, with known-text proposals where the model's
+choice remains uncertain. Prefix-cache reuse avoids rereading history; it does
+not by itself avoid generating the same text again. Strict appends and verified
+copy proposals have different behavior contracts and must be measured as such.
+Josh deferred the broader deterministic-continuation investigation. Generated
+state reuse is active R17 work and comes first; it is part of the original
+cache objective, not completion of the broader continuation direction.
+
+The R17 closeout audit confirms a reusable generated prefix in the saved
+first tool turn: the first response's reasoning text tokenizes as an exact
+extension of the original prompt inside the next rendered request. This is
+textual opportunity evidence only, not an accepted recurrent-state snapshot
+or a timing result. See `reports/qwen38-closeout/r17-first-tool-prefix-opportunity.json`.
+A candidate must retain aligned target KV/recurrent state, draft KV and the
+pending true hidden row at a completed decode boundary before tool formatting
+can diverge. Preserve the original prompt snapshot as a fallback within the
+existing cache budget. Test with independent prompts and verify snapshot
+identity after further decode; saved task timings cannot substitute for that
+state test. Quantities and original task timings belong in benchmarks.md.
+
+R17 uses the existing tiered cache. `createMlxSerialExecutor` now passes its
+shared cache service into speculative execution. `QwenMtpProvider` supplies a
+checkpoint codec and retains no prefix store of its own. Prefill publishes
+paired target/draft state through `PromptCache.take`/`put`; processed output
+publication remains unfinished. `PromptCache` owns RAM/SSD prefix selection,
+eviction and write-behind persistence. The execution-state interface must capture, restore and serialize a
+complete token-prefix checkpoint, regardless of whether those tokens came
+from a prompt or generated output. Qwen MTP supplies its target KV/recurrent
+state, draft KV and pending hidden row through that interface; it must not
+grow a second storage policy.
+
+The first implementation step now gives ordinary and batched consumers the
+same `PrefixCache` contract. MLX companion tensors flow through the existing
+RAM budget, spill queue, streamed SSD writer/reader and durability coordinator.
+The new storage tests pass on M1 Max and M4 Pro, including a fresh-process
+restore; layout and ownership details belong in [kv-cache method companion state](kv-cache.md#method-companion-state).
+Qwen MTP prefill migration now passes native bf16/KV4 RAM and SSD restart
+continuation gates on both Macs. Live target/draft/hidden hashes, emitted IDs
+and acceptance decisions match after reuse; the cache/provider restart also
+checks stable draft identity. Raw gates are in
+`reports/qwen38-closeout/composition-baseline/{m1,m4}-shared-mtp-cache-*.log`.
+Generated-ID alignment, native fresh-process continuation and saved-request
+performance acceptance remain unfinished.
+
+A subsequent isolated output-checkpoint candidate resolves the final verified
+round and publishes only processed content. Synthetic stop/EOS/callback cases
+pass. Generated-prefix SSD continuation matches fresh recomputation on M1 Max
+in bf16/KV4 and on M4 Pro in KV4. M4 bf16 preserves checkpoint bytes but changes
+the next argmax; a fresh split-prefill control also changes logits. The cause
+is not yet established against a matched execution/oracle control, so this
+candidate is not adopted. Evidence: `composition-baseline/m*-mtp-generated-reuse*.log`
+under `reports/qwen38-closeout/`. Shared execution work continues independently.
+
+A subsequent shared-method implementation publishes target-row and draft-provider
+snapshots through the common cache port before retirement. M1/M4 bf16/KV4/fused
+TQ native tests preserve actual processed-ID coverage and RAM/SSD continuation,
+including early stops and immutable donors after sibling/follow-up work.
+Production parser/template replay also preserves the sampled prefix of bounded
+tool turns with thinking off/on. Production HTTP RAM reuse and fresh-process
+SSD restoration now also pass on both Macs in bf16/KV4/fused TQ with thinking off/on;
+responses, logprobs and acceptance match within each restart pair. These
+results do not replace the long Kanban or performance gates. The five-file
+change is adopted; final native/HTTP composition with the prefill-policy fix
+and its matched short M4 benchmark now pass. Long Kanban and pressure
+acceptance remain. Details belong to the method-companion
+section in kv-cache.md; evidence is in the separate
+`generated-prefix-composition-validation` checkout's `reports/generated-prefix/`.
+
+The pre-refactor source and measured-evidence hashes are preserved under
+`reports/qwen38-closeout/composition-baseline/manifest.json`. It distinguishes
+the current unmeasured working tree from the frozen source used for the M4
+comparisons. Baseline numbers and limitations remain in benchmarks.md. Keep
+interface refactors, kernel experiments and the MTP depth comparison as
+separate changes before measuring their composition.
+
+Acceptance must cover actual generated IDs at a rendered-stable boundary,
+continued generation after capture without changing the retained state, RAM
+reuse, eviction to SSD, a fresh-process SSD restore, and an exact-prefix miss
+that still selects a usable earlier checkpoint. Retain shorter recurrent
+boundaries because a later state cannot be trimmed back. Account for all
+companion tensors in the existing cache budgets and persistence lifecycle.
+Measure complete next-request time, including capture/write/restore cost;
+the saved work is not literally free. The current Kanban candidate remains
+frozen while this change is developed and supplies the pre-R17 task evidence.
+
+The private `joshuarossi/kanban-board` archive was audited at
+`c34577fecbf5e1b4c48c13b14c7a7f92b92b6ba9`. Its first request and first SSE
+response are byte-identical to the original saved evidence. The archived
+second request changes two literal escape sequences inside reasoning from
+`\u0000` to `\^@`; this explains its earlier reconstructed-prefix divergence.
+Do not change either evidence copy or treat that archive difference as a Pi
+serialization defect. The original saved request retains the full reasoning
+prefix. Both comparisons retokenize decoded reasoning; the SSE has no original
+sampled IDs, so the execution-state gate must record those IDs directly.
+Evidence: `reports/qwen38-closeout/r17-repo-prefix-audit.json`.
+
+#### Shipped defaults and remaining experiment gates
+
+[Server configuration](../reference/server-config.md) owns effective defaults,
+precedence and supported combinations. Trellis v13, eligible KV4 speculation
+and paired MTP prompt reuse shipped in v0.4.0. Shared batching is the default;
+R17 generated RAM/SSD persistence and long-agent setting selection are accepted.
+Serial removal and exhaustive feature migration are deferred by the current
+milestone. Remaining experiments do not reopen those completed gates.
+
+| Control | Accepted evidence | Remaining work |
+| --- | --- | --- |
+| `MLX_BUN_TRELLIS`, `MLX_BUN_TRELLIS_VARIANT` | Shipped selection; six serial, six continuous and six combined KV4/MTP pairs, plus native state gates. | Additional experimental variants retain their own numerical/performance requirements. |
+| `MLX_BUN_TRELLIS_ASYNC_EXPAND` | Native, repeated short serving and saved long-agent pressure checks pass, including fused k8v3/MTP3 on M4 and M1. | Keep opt-in for the measured small request-time gain and higher live allocation. Combined pressure is accepted; reopen only for new models/barriers or a better tradeoff. |
+| `MLX_BUN_EARLY_FIRST_TOKEN` | Streaming/cancellation/overlap and cached/SSD gates pass. Caller-owned M=1 alignment preserves state on M1 and M4. Monitored paired chat HTTP checks support earlier visible output; default on. | No open early-output acceptance gate. Invisible token zero need not improve raw-prompt latency; no decode-throughput gain claimed. |
+| `MLX_BUN_TURBOQUANT_FUSED_DECODE` | Adopted on by default for an explicitly selected TQ cache after native/state, repeated codec serving, long cached/SSD and combined pressure gates, plus current M1/M4 shared serving. | Explicit off control remains; distinguish codec-kernel gains from quantization-setting comparisons. |
+| `MLX_BUN_QWEN_SPEC_KV4` | Combined M4 pairs, completed tasks and the current 78,678-token cached/SSD workload pass. KV4/MTP2 remains the measured long-agent choice. | Three-query key-head grouping is integrated for the qualified M4 shape after operation, full-state and matched serving gates. Other geometries retain the existing dispatch. |
+| `MLX_BUN_MTP_PROMPT_CACHE` | Shared byte accounting, generated-boundary checkpoints, RAM/SSD restart and full Kanban session retention pass. | No open R17 acceptance gate; new methods retain their own state contracts. |
+| `MLX_BUN_FILL`, append chunk/span controls | Qualified Qwen committed appends preserve bf16/KV4/KV8/K8V3 state; balanced HTTP and saved tool-history fixture comparisons pass. | Held-out task quality, shared direct append, speculative/quantized verification and other model/GPU qualifications remain distinct cells. |
+| Fill trace and echo-index controls | Trace is diagnostic; echo is a separate Lab algorithm. | Disable trace for timing; measure echo task quality and whole-request cost before promotion. |
+| `MLX_BUN_RD_PREFILL_CHUNK` | Captured policy selects work size; explicit overrides remain. Automatic long hybrid prefill matches all 19 fixed-size responses through cached/SSD restart. | The completed fixed-256 integrated Kanban control reproduces the earlier first response and meets the app-quality target. Retain that task-comparison setting. The general automatic policy remains selected; one changed trajectory does not establish a general quality loss. |
+| `MLX_BUN_RD_CONTEXT_LIMIT` | Explicit caller constraint; fit estimates remain advisory. | Benchmark control, not a speed candidate. |
+| `MLX_BUN_TOKEN_MASK` | Exact structured-output responses; previous serving timing is inconclusive. | Controlled B1/B4 comparison if the mask implementation changes. |
+| `MLX_BUN_PREFILL_TAIL_SPLIT` | Existing policy explains recorded RTN4 ordinary/MTP trajectory differences. | Changing that policy requires a separate numerical/quality comparison. |
+
+#### Closed candidates
+
+Do not re-open without new evidence of the kind named in the row's reopen
+condition.
+
+- Device f32 code LUT for variant-6 decode: slower at M=1 for every bit width (`lut6-*.json`).
+- Constant-address-space int16 codebook table: 0.802 vs 0.331 ms reduce; nearly 2x complete MLP on MLX 0.32.2 (`trellis-codebook*`, `trellis-constant-*`).
+- Device/threadgroup fp16 code tables at M=1/4: lose; threadgroup table retained only for M3/4 k3 scatter inside variant 13.
+- Integer y-mad final-sum rewrite: exact, no useful gain; readable expression kept.
+- Shared activation loads across 2/4/8 gate/up rows per SIMD group: loses at k3 M=1.
+- Row-loop unrolling of the aligned 3-bit scatter: no material gain.
+- Larger fused prefill tiles at M=128/512 (40 geometries): all exact, none beats expansion plus native matmul.
+- Contiguous packed reads with transposed threadgroup writes and threadgroup padding: slower; not retained.
+- Variant 9 unrestricted deferred expansion and unrestricted async submission: fail the saved-agent pressure gate (Metal insufficient memory); superseded by the bounded 75% policy.
+- M≤8 packed/expanded crossover: MLP win but changes output bytes; Lab, not in production dispatch.
+- fp16 storage as a Steel GEMM throughput shortcut: source audit shows float accumulation regardless; rejected.
+- Fused GDN convolution/SiLU/copied tail (R8): flat full-model timing; unpromoted.
+- GDN compiled fixed-length recurrence and fused recurrence/RMSNorm/gate kernels (R9): gains too small; not integrated.
+- Fixed-shape CompiledFunction over 16 SSM groups: -0.062%/-0.202%; no integration.
+- Fixed-input custom-kernel proxies, 64 per-layer cached graphs (RTN4 fails identity) and whole-model graph (slower): no production tracer manipulation.
+- Affine QMV layout screen (rows per SIMD group / groups per threadgroup): no accepted whole-model gain.
+- Shared-M affine kernels and `MLX_BUN_AFFINE_SHARED_M` (4-bit M3..8, compact 3/8-bit, nonfast large projections): superseded by MLX 0.32.2 `qmv_wide`; kernel and flag removed, frozen source controls retain them.
+- Shape-specific compiled-call cache around shared-affine kernels: HTTP gains shrink to flat; not integrated.
+- Broader cross-model 4-bit shared-affine prototype: MiniCPM/Llama regress; float16 head-only B8 variant <1% on 3B; not integrated.
+- Fused affine gate/up/SwiGLU: only M=1 wins in isolation; full-forward gain too small.
+- Residual-add/RMSNorm fused kernel: M1 slightly slower, M4/8 flat; not integrated.
+- Multimodal RoPE shared angles/direct strides: M128 slightly slower, M512 flat; closed.
+- Metal launch-configuration cache: Qwen flat, MiniCPM fails the serving response gate (cause unresolved); closed.
+- Untouched-finalize elision in the Metal backend: -0.069%/+0.108%; closed without integration.
+- MLX submission/traversal/sync settings for Qwen: no compelling setting (0.94% traversal gain matched the 0.92% default drift); Llama 25/25 remains an explicit process-start override after current-version cross-model and both-machine controls; no universal default is adopted. See benchmarks.md.
+- `MLX_METAL_FAST_SYNCH=1`: stalls the installed library's large bf16 three-stream case; not attributed to any prototype.
+- Top-p inverse-permutation removal (pure MLX and fused Metal): Qwen +0.183% complete; Llama -0.4%; no production sampler change.
+- Grouped greedy verify readback (concatenated and evalAll): ≤0.33% gain, JSON regression; closed.
+- Direct packed-key TurboQuant attention: exact, 6–19% slower than joint decode plus native SDPA; closed.
+- Fused inverse-Hadamard one-row layout inside the joint decoder: +12.76..+24.67% slower; only the row-grouped N≥8192 variant retained.
+- Dense register-resident attention (vLLM-Metal port): loses timing; Steel D256 fails one 512-token KL screen; Lab, unintegrated.
+- KV append whole-cache-copy hypothesis: rejected by native/HTTP census (one-row copies); no append kernel.
+- mmap file-backed weight loader (immutable, managed, lazy, prefault variants): warm flat-to-slower, cold worse in 5/6; closed as a performance candidate.
+- Staged-body strict-fill prototype (skip the discarded in-flight sample): flat; closed.
+- Multi-query affine attention inside strict append: recurrent state diverges at layer 4. Superseded by the qualified one-token attention calculation within shared projection appends; KV4/KV8 now pass state and serving gates.
+- Cross-entropy block-alignment partition: no wired model reaches a misaligned partition; closed without integration.
+- Streamed-expert threadgroup geometry: bf16 timing bimodal in all three library arms; no geometry default retained.
+- Proc-pid-rusage FFI collector: 0.0013 ms vs 1.09 ms per read, collector only; existing reports keep the ps collector.
+- Video tq3-mini polar format: operation wins on synthetic matrices; no loader, kernel or quality evidence; not adopted.
+
+
+### 7.14 Larger-departure feasibility decisions
+
+R24 now has an explicit initial screen. Resource calculations and the
+both-machine activation operation comparison live in
+[benchmarks](../reference/benchmarks.md#r24-activation-quantization-and-resource-bounds-2026-09-13).
+The following decisions apply to the current packed Qwen artifact and native
+pack. They do not close unmeasured candidate quality or claim the whole campaign
+is exhausted.
+
+| Candidate | Kernel and quality requirement | Training/calibration and next decision |
+|---|---|---|
+| Learned/codebook quantization | [AQLM](https://arxiv.org/abs/2401.06118) learns additive codebooks. Existing Trellis dispatch and the archived polar operation screen do not establish a loader or efficient decoder for that representation. New codes must pass the frozen quality suite. | Fit one matrix/block with calibration under the calculated memory budget before any whole-model conversion. A complete original bf16 Qwen snapshot was not found in the checked model directories; fitting the current packed reconstruction must be labeled as an additional approximation. |
+| Sparse residual/outlier correction | [SpQR](https://arxiv.org/abs/2306.03078) separates exceptional weights from the low-bit representation. The sparse correction must fit inside savings from a lower-bit base and fuse with the target's matrix operation. | Bounded matrix reconstruction is feasible in principle; measure error versus bytes and sparse gather/reduction time first. Do not install a second dense correction matrix or assume CUDA speed transfers to Metal. Original-weight recovery needs original weights. |
+| Activation quantization | [SmoothQuant](https://arxiv.org/abs/2211.10438) motivates jointly quantized weights/activations. The pinned native QQMM rounds then expands activations. The actual same-weight operation screen loses, especially at prefill sizes. | Current QQMM is rejected as a drop-in speed option. A direct compressed-activation kernel remains a separate implementation and quality experiment; INT8 input alone does not supply it. |
+| Structured pruning | Zero values in the existing packed format still consume codes and matrix work. A useful artifact must physically remove aligned channels/heads and update model dimensions. | Rank channels with calibration, then test one width-reduced MLP before a model conversion. Preserve matching gate/up selections and down input columns. No unstructured sparsity speedup is assumed. |
+| Layer skipping | [ShortGPT](https://arxiv.org/abs/2403.03853) ranks layer influence. A static reduced-layer artifact can remove work, but Qwen's recurrent/attention mix and cache topology must remain explicit. Dynamic token-dependent skips change recurrent evolution. | Start with static layer influence and held-out quality. Removing eight of 64 equally costly blocks would remove one eighth of block work only under that simplifying assumption; use measured layer costs before predicting time. No serving skip flag is selected. |
+| Low-rank approximation | [SVD-LLM](https://arxiv.org/abs/2403.07378) combines activation-aware factorization and correction. Two dense factor multiplies can use existing operations, but a loader/linear binding and compression-quality evidence are still needed. | The rank/byte screen rules out high-rank bf16 factors as a size win against W3. The actual layer-one gate spectrum rejects plain unweighted SVD at a size-feasible rank. Activation-aware error and task quality remain separate work. |
+| QAT/distillation | [QLoRA](https://arxiv.org/abs/2305.14314) shows why freezing a quantized base changes training memory. Full-resident 27B optimization does not fit either machine. `Qwen35Model.loraTargets()` explicitly excludes Trellis MLPs; existing adapter training does not establish packed-MLP recovery. | Layerwise QAT or adapter recovery needs a differentiable packed linear binding and measured activations. Stored teacher targets can separate teacher generation from student training. The quality gain and training duration remain unmeasured. |
+| Alternative drafters | Existing grouped providers already separate proposals from verification. A trained drafter must improve accepted tokens per total target/draft time and respect target memory needs. | No trained DSpark/DFlash checkpoint exists. Keep their untrained correctness checks distinct from performance acceptance. R20's measured native-MTP depths remain the available comparison. The specific DFlash2 architecture is separate open work. |
+
+Actual-weight screens now reject the simple unweighted low-rank and sparse
+variants. Differentiation confirms the need for a custom packed backward
+operation; expanded frozen weights already differentiate. Further work needs
+activation-calibrated candidates or that backward binding. No new deployment default,
+model-quality claim or training run follows from the paper results alone.