mlx-bun 0.0.13 → 0.3.0

package/docs/design/dspark-serving-program.md

@@ -1,371 +0,0 @@
-# DSpark serving program — flip spec wall-clock positive + design the product surface
-
-Status: **PLANNED** (2026-07-07). Trigger for Josh's pickup: TurboQuant merge —
-but only Phase 5 actually depends on it; Phases 0–4 are start-anytime.
-Owner: decode / spec. Predecessors: PR #19 (DSpark paper code-complete +
-serve-integrated + DeepSpec oracle), the first-live-run findings in
-[dspark-handoff.md](../investigations/dspark-handoff.md) "FIRST LIVE RUN".
-
-## The problem, in numbers (2026-07-07 first live run, loaded machine)
-
-`gemma-4-12B-it-OptiQ-4bit` + `deepseek-ai/dspark_gemma4_12b_block7` (γ=7):
-
-- **τ ≈ 2.8** committed tokens per target forward (2.7× fewer target calls) —
-  the algorithm WORKS, at 26–33% per-token acceptance (the predicted
-  OptiQ-4bit-target degradation vs their bf16 training).
-- **Wall-clock LOST ~3.4×** (spec 14.6 vs serial 49.8 agg tok/s). The tax is
-  the drafter itself: 6.9 GB bf16 → ~3 GB of weight reads per draft round
-  (2 GB untied lm_head + 7 × 134 MB markov_w2 + layers), ~14 host syncs per
-  round in the transcription-faithful `draftBlock`, tapped verifies falling
-  back to the unoptimized monolith, and ~14 GB of two-model weights
-  squeezing the 24 GB box.
-- The 20 s bench TTFT was conc-4 queueing on the serial-forced spec lane
-  (documented routing), not prefill — but per-request slowdown is real.
-
-**Goal:** spec ≥ 1.3× serial decode on 12B at the best config, clean-machine
-paired A/B — then (and only then) design when it's on by default. Every phase
-has its own gate; the drafter-quantization thesis
-([[drafter-quantization-thesis]]) makes the risky-looking phase (aggressive
-quant) the SAFEST: drafter numerics only move acceptance, never correctness,
-so the entire quality gate is one acceptance A/B.
-
-**THE OBJECTIVE, stated right (Josh, 2026-07-07): 26–33% acceptance with the
-bf16 drafter is the WORKABLE BASELINE — τ≈2.8 already pays. The question
-quantization must answer is PRESERVATION: do we still get that number at
-4-bit?** The point of quantizing (and of mixed precision specifically) is
-never compression for its own sake — it's cutting the drafter's cost 4×
-WITHOUT making it terrible at predicting tokens.
-
-**Sensitivity is MEASURED, never guessed — and THE TOOL ALREADY EXISTS
-(Josh): optiq's sensitivity method** (mlx-optiq.com/docs/sensitivity —
-per-group simulate-quant → KL vs reference on a curated calibration mix →
-greedy knapsack allocation to target BPW, emitted as a quant predicate).
-Phase 1e adopts it for the drafter rather than inventing anything; the
-drafter's target-hiddens-in forward even makes it cheaper (cached hiddens →
-no 12B in the sweep loop). Allocation comes from the measured table;
-hypotheses (lm_head argmax-proximal, chain compounding, buffered layers)
-are recorded for falsification only. Phase 1.5 (fine-tune) is an OPTIONAL
-UPSIDE lever on top of the workable baseline, not a premise that acceptance
-is broken.
-
-**And TurboQuant's role, stated right (Josh): TQ is the better-quant-at-
-EQUAL-bits instrument — the preservation tool, not a compression ladder.**
-Mechanism: distribute the weight vectors Gaussian, then place the
-quantization levels non-uniformly with FINER step sizes around the peak
-where the mass lives — affine 4-bit spends half its levels on tails TQ
-barely pays for. Expected: TQ-4bit preserves acceptance better than
-affine-4bit at the same size; the Phase-5 headline comparison is TQ-4 vs
-affine-4 AT EQUAL BPW, with lower-bpw rungs only after that's won.
-
-## Phase 0 — baseline + attribution (no code; Josh's shell for GPU runs)
-
-The scoreboard everything else is measured against. All directional runs
-fine on a loaded box; the FINAL pre/post pair must be clean-machine.
-
-- [ ] **0a. Oracle gate the bf16 port ONCE** (before any optimization, so
-  later acceptance deltas are attributable to quant/tuning, not port bugs):
-  torch venv + DeepSpec install (header of
-  `scripts/oracle-dspark-deepspec.py` documents setup), dump the temp-0
-  trace (threshold 0 and 0.5 arms), run
-  `scripts/dspark-deepspec-compare.ts` — target vs their reference on the
-  **bf16 HF 12B** (`--target` the bf16 snapshot; the OptiQ-4bit arm is
-  `--acceptance-only`, a measurement not a bit gate). Fixture per
-  goldens/README.md conventions. Exit: round-for-round PASS, or a filed
-  divergence with the first-divergent round attached.
-- [ ] **0b. γ sweep, conc 1** (no code): serve with `--num-draft-tokens`
-  ∈ {2, 3, 5, 7} × `bench-feature-matrix.ts --concurrency 1 --cells
-  serial,spec`. Record per-request tok/s + acceptance + TTFT per γ. Exit: a
-  table; the best-γ config is the Phase 1 comparison base.
-- [ ] **0c. Time attribution**: xctrace (or coarse: MLX_BUN_LANE_DEBUG +
-  timers around `source.draft` / verify forward / commit in a scratch
-  branch) on ~20 rounds. Split: draft-block GPU, draft host syncs, verify
-  forward, tap/commit overhead. Plus the paging check: RSS + `vm_stat`
-  pageouts during a run (is the 14 GB pairing thrashing?). Exit: a
-  percentage breakdown — it ORDERS phases 1–3 by measured value, replacing
-  the estimates below.
-
-## Phase 1 — quantize the drafter (uniform 4-bit baseline; NOT TurboQuant yet)
-
-The expected big lever: 6.9 GB → ~1.8 GB kills bandwidth AND memory pressure.
-mlx-native group quantization, no research dependency.
-
-- [x] **1a. `scripts/dspark-quantize-drafter.ts`** *(DONE 2026-07-07 — thin
-  driver over the house quantizer: `src/spec/dspark/quantize-drafter.ts`
-  adds a `quantizePredicate` overlay (confidence_head kept bf16, recorded
-  as the mlx `false` convention); load smoke = DeepspecDrafter reload +
-  one quantized fc forward)* — reads a DeepSpec-format
-  checkpoint dir, writes a quantized sibling (same config.json + arch stamp,
-  plus the house `quantization: {bits, group_size}` block). Per-tensor
-  policy:
-  - QUANTIZE (group-64, 4-bit default, flag-tunable): every 2-D matmul
-    weight — `layers.*.self_attn.{q,k,o}_proj`, `layers.*.mlp.{gate,up,
-    down}_proj`, `fc`, `lm_head`, `markov_head.markov_w2`, and
-    `embed_tokens` + `markov_head.markov_w1` as quantized-gather lookups
-    (mlx QuantizedEmbedding semantics — mlx-lm quantizes embeddings; ours
-    must match its gather-dequant numerics).
-  - KEEP bf16: all norms, `layers.*.layer_scalar`, `confidence_head.proj.*`
-    (tiny), biases.
-  - Output verified loadable before exit (load smoke).
-- [x] **1b. Quantized forward in `src/spec/dspark/deepspec-module.ts`**
-  *(DONE 2026-07-07 — MatW/TableW weight representation: `.scales` sibling
-  detection, quantized_matmul transpose=true for every Linear, mlx
-  QuantizedEmbedding gather-dequant for embed_tokens/markov_w1; bf16 path
-  keeps the lazy transpose views untouched; synthetic quantized-checkpoint
-  describe block added — 8-bit tracks bf16 token-for-token)*:
-  detect quantization at load (scales/biases sibling keys — the house
-  detection pattern), route matmuls through `quantized_matmul` and lookups
-  through the quantized-gather path; bf16 path untouched (both formats load
-  through the same class — kind detection unchanged). Synthetic-format test
-  extended with a tiny quantized checkpoint (same file, new describe block).
-- [x] **1c. The acceptance A/B harness** *(DONE 2026-07-07 — runner drives
-  `specServeRun` per arm (target loaded once, drafters sequential); spec
-  telemetry extended with per-position draftedByPos/acceptedByPos; the
-  stats/verdict math is model-free in `src/spec/dspark/ab-stats.ts`,
-  unit-tested in tests/dspark-ab-stats.test.ts; chat-template rendering
-  pins enable_thinking OFF)* — `scripts/dspark-drafter-ab.ts`:
-  same target, same prompt set (≥32 prompts, temp 0), drafter A vs drafter
-  B; reports per-position acceptance, τ, tok/s, and a paired verdict. This
-  is THE gate for every quantization experiment from here on (Phase 5
-  reuses it unchanged). Model-free logic unit-tested; the run itself is
-  GPU/Josh.
-- [ ] **1d. Run it**: 4-bit vs bf16 drafter at best-γ from 0b. Exit criteria:
-  acceptance drop ≤ 3 points absolute AND wall-clock strictly improves;
-  record both. If 4-bit alone flips spec past serial — say so loudly in the
-  handoff, that's the headline.
-- [ ] **1e. The MEASURED sensitivity map — ADOPT OPTIQ'S METHOD, don't
-  invent** (mlx-optiq.com/docs/sensitivity; source readable in the oracle
-  venv's `site-packages/optiq`). Their loop, verbatim: reference logits on
-  calibration data → simulate-quantize ONE group (quantize→dequantize
-  round-trip) → KL divergence on the full output distribution → restore →
-  next (group, bits); then their greedy knapsack (max KL-reduction per
-  extra bit) allocates to a target BPW. **Applied to the drafter it gets
-  CHEAPER than optiq's own runs**: the drafter isn't mlx_lm-loadable
-  (its forward needs target hiddens), so `optiq convert` can't run on it
-  directly — but that same fact removes the 12B from the loop entirely.
-  Cache tapped hiddens ONCE over an optiq-style 6-domain calibration mix
-  (our PATH-B shard format stores exactly this), then the whole sweep is
-  drafter-forward-only KL: 3B × ~10 groups × candidate bits = minutes.
-  The knapsack's output config gets ONE end-to-end acceptance A/B (1c
-  harness) as confirmation — KL is the search signal, acceptance is the
-  ship gate. Their protected-tensor defaults (lm_head/embed highest bits,
-  first/last blocks) are noted as the static prior their own exact method
-  exists to replace — we run the exact method; the drafter is small.
-
-## Phase 1.5 — acceptance upside (OPTIONAL; the baseline is already workable)
-
-26–33% (τ≈2.8) with the bf16 drafter pays as-is — this phase is the
-opportunistic numerator work on top, NOT a precondition: the gap vs their
-bf16-target numbers is distribution shift with a known mechanism, and every
-point recovered multiplies the cost work. Run it if Phase 1's preserved
-acceptance + cost win lands short of the ≥1.3× goal, or whenever the upside
-is worth the run.
-
-- [ ] **1.5a. Adaptation fine-tune on OUR target's hiddens** — their
-  checkpoint as init (the expensive pretraining is done; this is short
-  distribution adaptation, not their 10-epoch/38 TB run). The data comes
-  from OUR stack by construction: tap `[5,17,29,41,46]` hidden dumps from
-  the OptiQ-4bit 12B over on-distribution generations (the PATH-B regen
-  tooling extracts exactly this). OPEN DESIGN (resolve first): where the
-  gradient runs — (a) export our tapped-hidden shards into DeepSpec's
-  torch trainer format and fine-tune with THEIR code (MIT; least new code,
-  needs a torch box/venv and a shard-format bridge), vs (b) a training
-  path for the deepspec module in our stack (autograd through the port —
-  more work, fully local). Gate: acceptance A/B (1c harness) pre/post
-  fine-tune on held-out prompts; expected: a large chunk of the
-  bf16-vs-OptiQ gap closes.
-- [ ] **1.5b. Confidence-threshold activation** — at 26% acceptance the
-  scheduler is finally in its design regime: sweep the raw threshold
-  (their knob) and/or our calibrated thresholds; objective =
-  tokens-per-second, not τ (pruning doomed positions trades a little τ
-  for less wasted draft+verify work). Free: no code, the machinery
-  shipped in PR #19.
-- [ ] **1.5c. Quantify the target-fidelity axis** (measurement, not a
-  build): acceptance vs target quant — e.g. the same drafter against a
-  higher-fidelity 12B variant if one fits (mixed/8-bit tap-layer
-  experiments), to split "drafter needs adapting" from "target hiddens
-  are too degraded". Informs whether 1.5a alone suffices.
-
-## Phase 2 — tighten `draftBlock` (replay the proven tightening)
-
-`Gemma4DSparkModel.draftBlock` kept the reference's per-position structure
-(transcription fidelity): 7 argmax `itemUint32` syncs + 7 confidence
-`toFloat32` syncs per round. We already solved this exact shape once
-(module-dflash.ts, 2026-07-06 — bit-identity-pinned).
-
-- [ ] **2a. On-device token chaining**: the sequential Markov loop keeps the
-  argmax result as a device `[1]` index feeding the next position's
-  `takeAxis(markov_w1, ·)`; ONE concat + ONE host read after the loop.
-  (Post-1b the bias matmul is quantized too.)
-- [ ] **2b. Confidence: compute NOTHING at threshold ≤ 0** (their default —
-  today we compute + sync 7 confidences that are then unused); when a
-  threshold IS set, per-position reads are inherent to the truncation
-  decision (same reasoning as the dflash loop — document it).
-- [ ] **2c. `collectLogits`-style skip** for `baseLogits` if the serve path
-  still materializes anything it discards (audit `DeepspecSource.draft` —
-  it disposes `block.baseLogits` unread; make the module skip building it,
-  mirroring `DflashDraftOpts.collectLogits`).
-- [ ] **Gate**: bit-identity — pinned tokens on the deterministic synthetic
-  checkpoint pre/post rewrite (the tests/dspark-infer-loop.test.ts
-  pattern), plus real-checkpoint determinism (same prompt twice → same
-  drafts). Tightening is a pure optimization; any token change is a bug.
-
-## Phase 3 — `captureLayer` in the generated gemma forwards
-
-Tapped spec forwards (prefill + EVERY verify round) currently fall back to
-the monolith (the 2026-07-07 guard fix — correct-but-unspecialized). The
-verify forward is the spec path's biggest single GPU cost; give it back the
-generated fast path. Independent of Phases 1–2; parallelizable.
-
-- [ ] **3a. `scripts/gen-model.ts`**: emit the tap exactly as the monolith
-  does — `captureLayer(i, h)` after each layer's residual
-  (gemma4.ts:970 semantics) + the post-finalNorm sentinel
-  (index = nLayers, gemma4.ts:989) — and REMOVE `this.hiddenTap !== null`
-  from the emitted fallback guard.
-- [ ] **3b. Regenerate all three** (12b / 26b / e4b; regen commands in each
-  file's header) — the hand-patched guards are superseded by generator
-  output.
-- [ ] **Gate**: (i) generated-parity suite green (untapped behavior
-  untouched); (ii) NEW tap-parity check — tapped rows bit-equal
-  monolith-vs-generated on the same input (model-gated test, e4b);
-  (iii) spec bench pre/post for the verify-round win.
-
-## Phase 4 — serving UX: flags, defaults, composition (the product pass)
-
-Decisions, made on Phase 0–3 DATA, then implemented. The doctrine rails:
-drop-in-replacement (a lone request must never get slower because a flag
-exists), dont-delete-optionality (losing configs stay documented
-default-off), spec = opt-in until it WINS a clean-machine paired A/B.
-
-- [ ] **4a. γ default per drafter kind**: the checkpoint's trained γ is the
-  cap, not necessarily the default — if 0b says γ=3 beats γ=7 at our
-  acceptance, `numDraftTokens` defaults to the measured-best (per-kind
-  table in code, overridable), and the handoff documents why.
-- [ ] **4b. Quantized-drafter UX**: `mlx-bun get` the bf16 →
-  `scripts/dspark-quantize-drafter.ts` is a manual step today; decide
-  whether `serve --draft-model <bf16-dir>` should offer/auto-run the
-  4-bit conversion (like the starter-model flow) or just print the
-  command. Also: registry listing shows drafters distinctly (they are not
-  servable models — `/v1/models` must NOT list them; verify).
-- [ ] **4c. Spec × prompt-cache composition** (parity-plan §7.6 — currently
-  BYPASSED on the spec path, v1 note in serve-loop.ts): multi-turn agents
-  re-prefill every turn when a draft is mounted. Scope it here or
-  explicitly re-defer with the measured cost (12B multi-turn TTFT with
-  spec on). At minimum: the boundary-snapshot machinery must not regress
-  when spec is mounted.
-- [ ] **4d. Surface audit**: `--draft-kind` (keep), `--num-draft-tokens`
-  semantics doc, `/stats` shows mounted drafter + kind + γ, `usage.
-  speculation` already ships. server-config.md gets a "speculative
-  serving" section consolidating the four kinds + the when-does-it-pay
-  table (from Phase 0/1 data). features-matrix + cli.md + README rows.
-- [ ] **4e. The default decision, written down**: for each (model, drafter)
-  pair measured, spec default on/off per the clean-machine A/B — recorded
-  in RESULTS.md "composition" + the features matrix. (Prediction to test:
-  ON for 12B+quantized drafter, OFF for e4b anything.)
-
-## Phase 5 — TurboQuant application (the trigger event)
-
-Trigger, precisely: **TQ WEIGHT quantization** in the convert tooling — PR
-#20 (2026-07-07) landed TurboQuant for KV; the rotation-based weight-side
-is the noted follow-up (turboquant-kv.md), and it's what the drafter
-needs. The drafter is its lowest-risk first customer — one-number gate,
-no KL/eval battery. **TQ's HYPOTHESIS here is PRESERVATION AT EQUAL BITS**
-(Gaussian-distributed vectors + finer step sizes around the peak, where
-affine spends half its levels on tails) — a hypothesis 5b MEASURES, never
-a default: TurboQuant earns its place the same way everything in this
-repo does.
-
-- [ ] **5a.** Extend `dspark-quantize-drafter.ts` with the TurboQuant
-  scheme.
-- [ ] **5b. THE HEADLINE: TQ-4bit vs affine-4bit AT EQUAL BPW** through the
-  Phase-1c harness — same size, whose acceptance is closer to bf16? If
-  affine-4 bled in 1d, this is the expected fix; if affine-4 held, TQ's
-  margin becomes headroom for lower rungs. Only AFTER TQ-4 ≥ affine-4:
-  the lower-bpw ladder (3.5 / 3.0 / 2.5) → acceptance-per-byte curve,
-  knee adopted (dont-delete-optionality for the rest).
-- [ ] **5b″. TQ IS HOW, SENSITIVITY-AWARE IS WHAT (Josh)** — two orthogonal
-  axes, never mixed: TQ vs affine is a METHOD question (decided by 5b's
-  equal-bpw measurement — a method that wins at equal bits wins for every
-  tensor, so there is no reason to mix METHODS within an artifact);
-  sensitivity is the bit ALLOCATION question (the 1e knapsack, whatever
-  the method — candidate set {method-2, method-3, method-4, method-8} per
-  group, measured KL cost each, greedy to target BPW). IF 5b confirms TQ,
-  the artifact is TQ-uniform at knapsack-allocated bits; if it doesn't,
-  affine-uniform at knapsack-allocated bits — either way the OUTCOME of
-  the measurement, not a premise. Nothing here is "on by default";
-  defaults are Phase 4e/6 decisions made on clean-machine numbers.
-- [ ] **5b′. PER-TENSOR MIXED PRECISION rungs** (Josh, 2026-07-07). The
-  intuition "only 5 layers = few knobs" undercounts: the drafter's mass is
-  ~10 TENSOR GROUPS, and two of them are 58% of the bytes — `lm_head`
-  (~2 GB, read FULLY every round: the bandwidth whale) and `embed_tokens`
-  (~2 GB, gather-only: ~free to read, pure resident-memory win), plus
-  `markov_w2` (134 MB read ×7/round ≈ 940 MB). Both whales only shape the
-  DRAFT distribution → acceptance-only risk. Mixed rungs to sweep: 3-bit
-  lm_head + 3-bit markov_w2 + 4-bit layers; 2-bit embed variants; k_proj
-  (2 M params) 8-bit for free. The ~10-group space is small enough for a
-  near-exhaustive sweep through the SAME one-number harness — unlike
-  sensitivity search on a full model. **Allocation objective (the point):
-  PRESERVE the bf16 acceptance at the smallest cost, not minimize bytes —
-  and allocation follows the MEASURED sensitivity table from 1e, never a
-  prior.** The mixed rungs are built FROM the table: give the measured-
-  sensitive groups more bits (or TQ them, post-Phase-5), squeeze the
-  measured-insensitive ones. The "only 5 layers" worry dissolves either
-  way — the tensor-group space (~10 knobs, two of them 58% of the bytes)
-  is the real granularity, and 1e prices every knob empirically.
-- [ ] **5c.** If TurboQuant wins: fold the recipe into 4b's UX (the
-  conversion the server offers is the TurboQuant one).
-
-## Phase 6 — quotable results + promotion
-
-- [ ] Clean-machine protocol (reboot + `sudo purge`), then: bench-feature-
-  matrix conc-1 AND agg×4, serial vs best-spec-config, on 12B (and e4b as
-  the expected-negative control). Promote to `benchmarks/RESULTS.md`
-  "composition"; update decode-speed-program.md (lever 1c: measured),
-  STATUS, and the handoff's FIRST LIVE RUN section with the closing
-  numbers.
-- [ ] Loose ends carried from PR #19, closed or re-deferred with reasons:
-  the paper-faithful gated RNN head (seqHead:"gru", currently variant-
-  documented), paper-Alg-1/STS survival-product scheduler (Lab item),
-  per-slot spec under batching (unified-engine frontier row — NOT this
-  program).
-
-## Dependency graph + who does what
-
-```
-0a (oracle gate, Josh: venv+GPU) ──┐
-0b (γ sweep, Josh: GPU)  ──────────┼─→ 1a→1b→1c→1d (quant baseline; 1d Josh GPU)
-0c (attribution, Josh: GPU) ───────┘        │
-                                            ├─→ 1.5 (OPTIONAL acceptance
-                                            │    upside: fine-tune on our
-                                            │    target's hiddens + threshold
-                                            │    sweep; run if 1's preserved
-                                            │    acceptance falls short)
-                                            ├─→ 2a-2c (tightening; CPU-gated)
-3a-3b (generator tap; parallel anytime) ────┤
-                                            └─→ 4 (UX, needs 0–3 data)
-TurboQuant merge ─→ 5a-5c (reuses 1c harness) ─→ 6 (clean machine, Josh)
-```
-Priority note: Phase 1's PRESERVATION question ("do we keep 26–33% at
-4-bit?") is the program's hinge — one A/B answers it; if affine-4 bleeds,
-the answers are MEASURED (1e sensitivity table → mixed allocation) and
-ENGINEERED (Phase 5: TQ-4 at equal bpw — the better quant is the designed
-fix, not a fallback).
-
-Code phases (1a/1b/2/3/4-impl) are agent-runnable sessions; every GPU
-measurement is Josh's shell. Nothing except Phase 5 waits on TurboQuant —
-Phases 0–4 can land first so TurboQuant drops into a finished harness.
-
-## Exit criteria for the whole program
-
-1. Spec wall-clock ≥ 1.3× serial on 12B at the recommended config,
-   clean-machine paired — or a written verdict that it can't be reached and
-   why (with the config left as a characterized default-off lever).
-2. **Acceptance PRESERVED and understood**: the quantized drafter holds the
-   bf16 baseline's 26–33% (or the mixed rung that does is adopted, or the
-   loss is measured and accepted with numbers); the optional 1.5 upside
-   levers are run-or-explicitly-deferred — the drafter predicts tokens;
-   small was never the goal.
-3. The acceptance-per-byte curve exists (TurboQuant vs 4-bit vs mixed vs
-   bf16), with acceptance-at-fixed-budget allocation swept (5b′).
-4. The serving surface is documented, defaults decided per-pair and
-   recorded, and a fresh `mlx-bun get` → serve → speedup flow works without
-   reading any design doc.

package/docs/design/dspark-speculative-decoding.md

@@ -1,268 +0,0 @@
-# DSpark speculative decoding for mlx-bun
-
-Implementation of DeepSeek's **DSpark** ("Confidence-Scheduled Speculative
-Decoding with Semi-Autoregressive Generation", `DSpark_paper.pdf`) against this
-repo's models. DSpark = a **DFlash** parallel backbone (KV-injected, all γ draft
-tokens in one pass) **+ a lightweight sequential head** (Markov, Eq 5, or RNN,
-Eq 6) that adds intra-block dependency (the *semi-autoregressive* part) **+ a
-confidence head → STS calibration → hardware-aware prefix scheduler**. Decode
-becomes draft → verify → accept; output is lossless (bit-identical to the target
-at any temperature); only decode speed (τ) changes.
-
-> **DFlash vs DSpark:** DFlash is *only* the parallel backbone (prior work) and
-> suffers suffix decay. DSpark adds the sequential head + confidence-scheduled
-> verification. The sequential Markov head is what makes this DSpark.
-
-## Two implementations (flag-selected, both in `src/spec/dspark/`)
-
-| | `module.ts` (v1) | `module-dflash.ts` (v2, **the real one**) |
-|---|---|---|
-| context | **single vector** (final hidden at the anchor) fused as one token | faithful **multi-layer KV injection** (Eq 2–3): H_ctx = RMSNorm(W_c·[H^{l1};…;H^{lm}]) over the FULL context, re-projected into every draft layer's K/V |
-| layers | 2 | 5 |
-| status | **superseded** — dropped the paper's core idea (kept as baseline) | the actual DSpark model (Markov variant) |
-
-v1 was a shortcut that removed the load-bearing mechanism; it never tested the
-paper. v2 is faithful. Checkpoints stamp `variant` in `dspark.json`; the runtime
-loads the matching module. (`variant:"dflash"` should be renamed to `"dspark"` —
-see Open items.)
-
-## The mechanism (v2, paper Eq 2–3)
-
-- **Eq 2 (context):** tap the target's hidden states at layers `{l1…lm}` (e4b
-  default {20,31,41,42}, the last = post-finalNorm), concat on the feature axis,
-  project to the draft width: `H_ctx ∈ [L_ctx, d]`. Extraction via a parity-safe
-  `model.hiddenTap` (no-op when off) added to `gemma4.ts` `forwardLayers`.
-- **Eq 3 (injection):** every draft layer forms `K_i=[W_i^K H_ctx; W_i^K H_d]`,
-  `V_i` likewise, `Q_i=W_i^Q H_d`; the γ-block queries attend bidirectionally
-  over `[context ++ block]`. Context is read-only memory re-projected fresh each
-  layer. The draft projects the target's HIDDEN STATES with its OWN Wk/Wv (not
-  the target's K/V — that's the separate `GemmaAssistantDrafter` baseline).
-- **Markov head (Eq 5):** `B_k = W1[x_{k-1}]·W2`, r=256, W2=0 init → starts as
-  pure DFlash, τ climbs as intra-block dependency is learned.
-- **Confidence head (Eq 7):** `c_k = σ(wᵀ[h_k; W1[x_{k-1}]])`.
-- **Loss (Eq 9–12):** `0.1·CE + 0.9·TV + 1.0·BCE`, position-weighted.
-- **Lossless verify** (`generate-dflash.ts`): greedy = longest-prefix match;
-  temp>0 = `min(1,p/q)` accept + residual `norm(relu(p−q))` resample. Growing
-  multi-layer H_ctx buffer; rollback via `trim(n, bypass=true)` (physically
-  slices the rejected tips past the sliding window — see `gemma4-base.ts`).
-
-## Results & findings (measured, M1 Max 32GB, e4b OptiQ-4bit)
-
-**The architecture is CORRECT — proven by an overfit test.** Trained on 3
-articles, eval on the same 3: per-position acceptance **~0.75**, τ **3.24** —
-squarely paper-range (0.6–0.9 / τ 3–4). So the faithful DFlash, correctly
-implemented, reaches the paper's numbers.
-
-**Getting there required finding a real bug — an off-by-one in the p^t target
-alignment** (the TV loss, weight 0.9). Block position k predicts `x_{t+1+k}`,
-whose target distribution is `softmax(LM_head(h_{t+k}))`; the data gathered the
-hidden at `t+k+1` (shifted forward one). Present in **both v1 and v2** — so every
-τ number before the fix was trained against a shifted target. Fixed in
-`data-dflash.ts` (`blockIdx = g+k`, k=0..γ-1). Before fix: per-pos ~0.08. After:
-~0.17 generalizing, ~0.75 overfit.
-
-**Two things stand between "correct architecture" and "net speedup," now
-separated:**
-1. **Data.** 160 articles → generalizes only to per-pos ~0.17 (plateaus, doesn't
-   climb); overfit on 3 proves capacity. The paper uses 1.3M samples × 10 epochs.
-   → needs real data scale.
-2. **Target speed (the decisive one).** Spec decode amortizes the fixed draft
-   overhead only when the *target forward* is expensive. Measured decode:
-   **e4b 45.9 tok/s, 12b 27.5 tok/s, 27B ~15 tok/s.** On fast e4b the draft
-   overhead dominates → even τ≈3 nets ~0.4–0.5× (SLOWER). On a slow target (27B,
-   ~67 ms/forward) that overhead is a small fraction → the same τ≈3 could land
-   ~2–3× (15 → ~35–45 tok/s). **e4b is close to the worst case for spec decode;
-   the 27B agentic workload is the ideal target and the real motivation.**
-
-Also confirmed: width is NOT the ceiling (`dDraft=1024` and `2560` both plateau
-~0.17 pre-scale); the parity-safe `hiddenTap` doesn't change target numerics;
-long-context spec rollback works via the `trim(n, bypass)` physical-slice.
-
-## Serve integration (Phase 1, 2026-07-06)
-
-DSpark is now serve-loadable behind the same `--draft-model` seam as every
-other drafter — one verify/accept executor, no lane routing. The seam
-(`src/spec/source.ts`) was extended from its two-model-only shape to carry the
-target state KV-borrowing drafters need (it previously could NOT host these,
-despite the header claiming so):
-
-- `DraftProvider.open({sampler, target:{model,caches}})` — the source gets the
-  target model + its live caches.
-- `DraftSource.tapLayers?` — when set (DSpark), the serve loop taps the
-  target's prefill + every verify forward and hands the captured `[1,L,m*H]`
-  context to `prefill(ids, ctxML)` / `commit(n, kAccept, vCtxML)` (ownership
-  transfers). `draft(feed, n, stepBase, anchorHidden?)` — the assistant borrows
-  the target anchor hidden; DSpark drafts from its growing `H_ctx`.
-
-Provider selection is by artifact kind (`src/server.ts` `detectDraftKind`):
-`dspark.json` → **DflashProvider** (`src/spec/dflash-source.ts`), a `*_assistant`
-config → **AssistantProvider** (`src/spec/assistant-source.ts`, the optiq
-KV-borrowing Gemma drafter — ships a real ~1.09× γ=1 decode win with no
-training), else **TwoModelProvider**. `--draft-kind` overrides. The server pins
-`--num-draft-tokens` to the DSpark checkpoint's trained block width
-(`cfg.gamma`) so the serve loop never asks for more positions than were trained.
-
-**Unified-engine alignment (per-slot readiness).** This is the
-`DraftSource` seam the unified-engine plan names as the substrate for its
-frontier "per-slot drafting behind the `DraftSource` seam" row
-([docs/design/unified-engine-frontier-plan.md](unified-engine-frontier-plan.md)).
-Spec still forces the serial lane today (upstream `is_batchable = draft is
-None`); Phase 1 did NOT change that — it made the seam *carry per-source target
-state* (each source binds its own `TargetView{model,caches}`, anchor hidden,
-and tapped H_ctx), which is exactly what a batched executor would need to open
-one `DraftSource` per row. So the seam is forward-compatible with per-slot
-drafting; the hard part that remains is the executor, not the seam — variable
-accept-length per slot breaks a uniform-B step (see
-[spec-decode-larger-targets.md](spec-decode-larger-targets.md) caveats), so
-per-slot spec is a real batched-scheduler project, not free composition.
-
-**v1 acceptance caveat:** the serve loop verifies with TOKEN-MATCH acceptance
-(mlx-lm/optiq style, lossless at any temperature), NOT the paper's
-distribution-level rejection-sampling verify. Greedy is identical; temp>0 is
-still lossless but lower-acceptance. The richer rejection-sampling verify stays
-in the standalone `generate-dflash.ts` (the measure script). See
-[[dspark-seam-kv-borrowing]].
-
-## Files
-- `src/spec/source.ts` — the extended DraftSource/DraftProvider/TargetView seam.
-- `src/spec/assistant-source.ts` / `dflash-source.ts` — the two KV-borrowing providers.
-- `src/spec/serve-loop.ts` — the shared verify/accept executor (now tap-aware).
-- `src/model/gemma4.ts` — `hiddenTap`/`captureLayer` (parity-safe multi-layer tap).
-- `src/model/gemma4-base.ts` — `trim(n, bypass)` on rotating caches (spec rollback past the window).
-- `src/spec/dspark/module-dflash.ts` — faithful DFlash+Markov+confidence module.
-- `src/spec/dspark/data-dflash.ts` — multi-layer shards, variable-length prefix context, left-pad masking.
-- `src/spec/dspark/generate-dflash.ts` — growing-H_ctx lossless decode loop.
-- `src/spec/dspark/{loss,sample}.ts` — loss (Eq 9–12) + temp>0 sampling (shared with v1).
-- `src/spec/dspark/{module,data,generate}.ts` — v1 single-vector (superseded baseline).
-- `scripts/dspark-regen-dflash.ts` / `dspark-train-dflash.ts` (`--resume`) / `dspark-measure-dflash.ts`.
-- `scripts/dspark-dflash-smoke.ts` — CPU smoke (16/16). `scripts/dspark-smoke.ts` — v1 smoke (33/33).
-
-## Run sequence (faithful DFlash; GPU = Josh runs)
-```
-bun scripts/dspark-regen-dflash.ts --topics <topics.txt> --out <data> --max-resp 320
-bun scripts/dspark-train-dflash.ts --data <data> --out <ckpt> --iters 8000 --batch 8 [--resume] [--ddraft 2560]
-bun scripts/dspark-measure-dflash.ts --drafter <ckpt> --data <prompts.jsonl>   # τ + tok/s vs vanilla
-```
-
-## Paper components — Phase 2 (2026-07-06): the paper is CODE-COMPLETE
-
-All remaining components landed the same day (multi-agent build, adversarially
-reviewed; smoke 21/21, dspark test files 17/17, real-weights serve gate 3/3):
-
-- **Alg 1, single-user form — confidence-scheduled draft-length pruning.**
-  `forwardInfer` drops position k (and truncates the block) when
-  `c_k < thresholds[k] ?? minConf`; position 0 always survives. Activation is
-  checkpoint-driven: STS thresholds in `dspark.json` (`cfg.sts`) or the
-  `MLX_BUN_DSPARK_MINCONF` env override — **uncalibrated checkpoints draft
-  fixed-γ exactly as before** (zero behavior change). Losslessness invariant:
-  pruning changes how many positions the target verifies, never what's emitted
-  (gated: truncation-never-redraw, prefix-identical to the unpruned block).
-- **Variable-length draft contract.** `DraftSource.draft()` returns 1..n —
-  the RETURN LENGTH is authoritative; the serve loop and `dflashGenerate`
-  verify/trim/re-feed over `d = drafts.length`. Two-model/assistant return
-  exactly n (unchanged).
-- **STS calibration (§3.2.1).** `src/spec/dspark/calibration.ts`
-  (`fitStsThresholds`: per position, smallest τ whose Laplace-smoothed
-  `P(accepted | conf ≥ τ)` ≥ target; pos 0 → 0, under-sampled → 0 (don't
-  prune on thin evidence), unreachable target → 1.0) +
-  `scripts/dspark-calibrate.ts` (GPU: greedy unpruned rounds via
-  `dflashGenerate`'s `onRound` hook → fit → patch `config.sts` into
-  dspark.json in place). ⚠ estimator shape is our reading — paper PDF absent.
-- **RNN sequential head (Eq 6 — ⚠ design-doc-faithful shape, paper PDF
-  absent, flagged in code).** `cfg.seqHead: "markov"|"rnn"`; Elman recurrence
-  `s_k = tanh(s_{k-1}·wH + E[x_{k-1}] + bH)`, `B_k = s_k·wO`, `wO` zero-init
-  (starts as pure DFlash, like markov.w2); shares `markov.w1` as the token
-  embedding (also feeds the confidence head). Same loss, autograd-gated;
-  init-equivalence gate: rnn ≡ markov token-for-token at init.
-  Train with `scripts/dspark-train-dflash.ts --seq-head rnn`.
-- **Draft loop TIGHTENED** (handoff item 3): greedy tokens chain ON-DEVICE
-  (one concat + one host read after the loop, was γ syncs); confidence reads
-  deferred+batched when pruning is inactive (per-position only when Alg 1
-  needs the answer mid-loop — inherent); `collectLogits: false` on the serve
-  path skips the per-position draft-logits materialization entirely
-  (`draftLogits` now optional). Gate: bit-identical to the pre-tightening
-  loop (pinned reference tokens, tests/dspark-infer-loop.test.ts).
-- **Rename + central loader.** Canonical variant is now `"dspark"`
-  (`save()` stamps it; `load()` accepts legacy `"dflash"`);
-  `src/spec/dspark/loader.ts` `loadDsparkDrafter()` dispatches by variant and
-  refuses v1 single-vector checkpoints with a pointer at the v2 trainer. The
-  serve path and the bench (`bench-feature-matrix.ts` via the server's
-  exported `detectDraftKind`) both go through it.
-
-## DeepSpec ground truth (2026-07-06 audit — paper arXiv:2607.05147 + source)
-
-DeepSeek open-sourced the reference implementation (**DeepSpec**, MIT,
-github.com/deepseek-ai/DeepSpec) + trained drafters incl.
-**`deepseek-ai/dspark_gemma4_12b_block7`** (for google/gemma-4-12B-it — our
-primary target; 6.86 GB bf16 single safetensors, γ=7). Two audit agents
-verified our build against the paper AND the source. Verdicts:
-
-- **MATCH the paper exactly:** Eq 2-3 (context + KV injection), the
-  anchor++masks block, Eq 5 Markov (r=256), Eq 7 confidence (we add a
-  zero-init bias — superset), **Eq 8-12 loss to the letter** (0.1/0.9/1.0,
-  w_k=exp(−(k−1)/γ), analytic c*). Our defaults = their production config
-  (γ=5, 5 layers, Markov head).
-- **Eq 6 RNN differs:** paper is a gated cell over
-  `z_k=[s_{k−1}; W₁[x_{k−1}]; h_k]`, fused (2r+d)→3r projection,
-  `B_k=W₂ᵀtanh(W_o z_k)` (conditioned on the backbone hidden). Ours is
-  ungated Elman without h_k — kept as variant `"rnn"` (paper calls the gains
-  "marginal"; production uses Markov); DeepSpec also ships an RNNHead of the
-  gated form if we ever want the faithful one.
-- **§3.2.1 STS + Alg 1: the paper's machinery is NOT in the released code.**
-  STS = Sequential Temperature Scaling (per-position temperatures by ECE grid
-  search on cumulative survival products); Alg 1 = expected-throughput
-  maximization over survival products with a profiled SPS(B) cost table —
-  both live in their UNRELEASED production serving layer. The released
-  reference ships **per-position threshold truncation on sigmoid confidence
-  (default 0 = off), ℓ=0 allowed** — i.e. the released reference is shaped
-  like OUR scheduler. Our threshold fit (`calibration.ts`) is therefore a
-  reasonable calibrator for the reference-shaped scheduler, NOT the paper's
-  STS; renamed framing in code comments. Paper-faithful STS/Alg-1 = a future
-  Lab item if the survival-product scheduler ever earns a default.
-- **The reference drafter is a DIFFERENT module than ours:** a full
-  Gemma4-shaped 5-layer transformer — hidden 3840, 16 Q heads × head_dim 512,
-  **1 KV head with K≡V (`attention_k_eq_v`, no v_proj tensor; scale-less
-  v_norm)**, attention scale 1.0 (QK-norm), RoPE partial_rotary 0.25 θ=1e6,
-  per-layer learned `layer_scalar`, scaled embed (×√3840, not stored), UNTIED
-  lm_head, final softcap 30, and an **incremental context-KV cache** (stores
-  projected target-context K/V; crops the block's noise K/V each round).
-  Sequential Markov sampling + confidence head (input [h; w1[prev]] = 4096)
-  as in the paper. Checkpoint: 74 tensors, bf16, **no key prefix**;
-  config.json `architectures: ["Gemma4DSparkModel"]` (loader detection key).
-  **12B tap layers: `target_layer_ids [5,17,29,41,46]`** (their layer_id+1
-  output convention == our tapLayers index; no final-norm sentinel).
-- **THE ORACLE:** their verify is leaky rejection sampling that at
-  temperature 0 **degenerates to exact argmax token-match — RNG-free,
-  token-for-token deterministic**. Their eval emits per-round
-  proposal/accepted/next-token traces. Protocol: run their eval at temp 0
-  (+ a threshold arm) on fixed prompts → JSONL round fixtures → our port
-  must match round-for-round. Same discipline as the D2 diffusion oracle,
-  minus RNG parity entirely.
-
-**Consequence:** DSpark is no longer oracle-less L3. The port of their module
-(the `deepspec` variant) + their checkpoint gives a bit-exact-gateable path to
-a REAL 12B speedup with zero training. Our own trainable module remains the
-research path for custom targets. Training recipe reference: open-perfectblend
-regenerated by the target, 10 epochs, batch 512, lr 6e-4, position decay
-exp(−pos/4), ~38 TB cached hiddens — i.e. our "thousands of generations" data
-estimate was directionally right and their scale is far beyond a laptop run.
-
-## Open items / next
-
-- [x] Serve integration (Phase 1) · [x] Alg-1-shaped scheduler (matches the
-  RELEASED reference; paper Alg-1 unreleased) · [x] threshold calibration ·
-  [x] RNN head (Elman variant; paper's gated cell differs — documented above)
-  · [x] loop tightening · [x] `dspark` rename + loader —
-  **everything buildable without a GPU is DONE.**
-- [ ] **DeepSpec-variant port** (`Gemma4DSparkModel`) + loader detection +
-  oracle scripts — the no-training path to the 12B speedup (in progress).
-- [ ] **Josh-gated GPU (the payoff):** real data scale (thousands of
-  on-distribution generations) + **retarget to 12B** (`--model` +
-  `--tap-layers 5,17,29,41,46` (DeepSeek's trained taps); 27B memory-infeasible to train on 24 GB, kept
-  dim-generic) + train (`--seq-head` A/B optional) + **calibrate**
-  (`scripts/dspark-calibrate.ts`) + live-τ measure. Recipe:
-  [docs/investigations/dspark-handoff.md](../investigations/dspark-handoff.md).
-- [ ] Verify the Eq 6 / §3.2.1 shapes against the actual paper when the PDF
-  is available (both flagged in code).
-
-Full session handoff: `docs/investigations/dspark-handoff.md`.