@tryhamster/gerbil 1.0.0-rc.8 → 1.0.0

package/docs/gpu-engine/paper.md

@@ -0,0 +1,2109 @@
+# Gerbil WebGPU Inference Engine
+
+**A WebGPU-native transformer inference engine for browser-based LLM execution.**
+
+*Living technical document. Last updated: June 2026 (native audio — Moonshine STT + Kani-TTS-2 decoder — Gemma 4 E2B, on-device memory/RAG, and the text+ViT autoresearch campaign).*
+
+---
+
+## Table of Contents
+
+1. [Introduction & Motivation](#1-introduction--motivation)
+2. [Previous Approach & Its Shortcomings](#2-previous-approach--its-shortcomings)
+3. [Architecture Overview](#3-architecture-overview)
+4. [Intermediate Representation](#4-intermediate-representation)
+5. [Safetensors Parser](#5-safetensors-parser)
+6. [WebGPU Device Layer](#6-webgpu-device-layer)
+7. [WGSL Kernel Library](#7-wgsl-kernel-library)
+8. [Tokenizer](#8-tokenizer)
+9. [Sampler](#9-sampler)
+10. [Architecture Registry & Graph Generators](#10-architecture-registry--graph-generators)
+11. [KV Cache](#11-kv-cache)
+12. [Executor](#12-executor)
+13. [Model Loading Pipeline](#13-model-loading-pipeline)
+14. [Public API (WebGPUEngine)](#14-public-api-webgpuengine)
+15. [What's Built vs. What's Planned](#15-whats-built-vs-whats-planned)
+16. [Key Design Decisions](#16-key-design-decisions)
+17. [The Four-Failure-Mode Diagnosis (June 2026)](#17-the-four-failure-mode-diagnosis-june-2026)
+18. [The Mobile Fix Campaign](#18-the-mobile-fix-campaign)
+19. [Mobile Results & Submit-Granularity Scaling](#19-mobile-results--submit-granularity-scaling)
+20. [Autoresearch: Profile-Guided Throughput Optimization](#20-autoresearch-profile-guided-throughput-optimization-june-2026)
+21. [Native Text Embeddings](#21-native-text-embeddings-june-2026)
+22. [Native Vision Encoder (Qwen3.5 ViT)](#22-native-vision-encoder-qwen35-vit-june-2026)
+23. [The Native-Only Architecture Decision](#23-the-native-only-architecture-decision)
+24. [iOS Model-Caching Reality](#24-ios-model-caching-reality)
+25. [EmbeddingGemma-300M: a Second Embedding Family](#25-embeddinggemma-300m-a-second-embedding-family-validated-on-device-june-2026)
+26. [The SentencePiece Tokenizer Fix](#26-the-sentencepiece-tokenizer-fix-a-cross-family-lesson)
+27. [MLX-4bit Loading and the DWQ Trap](#27-mlx-4bit-loading-broadened-detection-and-the-dwq-trap)
+28. [The Progress-Reporting Fix](#28-the-progress-reporting-fix-killing-the-stuck-at-10-freeze)
+29. [Cross-Device Multi-Modal Parity](#29-cross-device-multi-modal-parity-june-2026)
+30. [Model-Zoo Growth and Kernel-Library Saturation](#30-model-zoo-growth-and-the-saturation-of-the-kernel-library)
+31. [Native STT: Moonshine](#31-native-stt-moonshine-june-2026)
+32. [Native TTS: Kani-TTS-2 and the NanoCodec Decoder](#32-native-tts-kani-tts-2-and-the-nanocodec-decoder-june-2026)
+33. [Gemma 4 E2B: PLE, KV-Sharing, and Logit Softcap](#33-gemma-4-e2b-ple-kv-sharing-and-logit-softcap-june-2026)
+34. [On-Device Memory / RAG](#34-on-device-memory-rag-june-2026)
+35. [The June-2026 Autoresearch Campaign (Text, LFM2, and ViT)](#35-the-june-2026-autoresearch-campaign-text-lfm2-and-vit)
+- [Appendix A: File Map](#appendix-a-file-map)
+- [Appendix B: WebGPU Browser Compatibility](#appendix-b-webgpu-browser-compatibility)
+
+---
+
+## 1. Introduction & Motivation
+
+Gerbil is an on-device AI library for JavaScript. Its GPU engine is a WebGPU-native transformer inference engine that runs large language models directly in the browser with zero server-side dependencies. Given any HuggingFace model repository, the engine:
+
+1. Downloads the model's `config.json` and determines its architecture
+2. Generates a fine-grained computation graph (the IR) at runtime
+3. Downloads safetensors weight files and uploads them to GPU buffers
+4. Dispatches WGSL compute shaders for each operation in topological order
+5. Streams generated tokens back to the caller
+
+The engine was built to replace an earlier approach based on transformers.js and ONNX Runtime Web, which suffered from fundamental architectural limitations. Those limitations -- iOS memory crashes, 50MB bundle sizes, webpack-within-webpack collisions, ONNX format lock-in, and three divergent inference paths -- motivated a complete rewrite with WebGPU as the single execution target.
+
+The core philosophy is: **own every layer of the stack**. No ONNX runtime, no transformers.js, no external WASM blobs. Just TypeScript orchestration code and hand-written WGSL kernels, speaking directly to the WebGPU API. As of this cycle that single engine is no longer text-only: **native text embeddings** (Section 21) and the **native Qwen3.5 vision encoder** (Section 22, bit-exact vs HuggingFace) run through the same IR and kernel registry, and the architecture decision is explicitly **native-only** across text, vision, and embeddings — no fallback lane (Section 23).
+
+As of June 2026 the engine runs Qwen3.5-0.8B (MLX 4-bit) with byte-identical greedy output on desktop Dawn (node, Chrome) and WebKit WebGPU (iPad), at **145 tok/s on M4 Max** and **31.7-35.9 tok/s on iPad (iPadOS 26.5)** — the first correct, crash-free on-device mobile results in the project's history. Getting there required diagnosing what had been treated as "one mobile bug" into four independent failure modes (Section 17) and a fix campaign spanning memory layout, submit architecture, a WGSL data race, and platform detection (Section 18).
+
+---
+
+## 2. Previous Approach & Its Shortcomings
+
+Understanding why the GPU engine was built requires understanding what came before it and why that approach was unsustainable.
+
+### 2.1 The transformers.js + ort-web Architecture
+
+The original gerbil browser inference path used [transformers.js](https://github.com/huggingface/transformers.js) (from HuggingFace) as its runtime. The architecture was:
+
+```
+User Code
+  -> gerbil browser SDK (createGerbilWorker)
+    -> Web Worker (blob URL, IIFE bundle)
+      -> transformers.js (AutoModelForCausalLM, AutoTokenizer)
+        -> ONNX Runtime Web (ort-web)
+          -> WebGPU backend (or WASM fallback)
+            -> ONNX model files (.onnx)
+```
+
+transformers.js provided a high-level API (`AutoModelForCausalLM.from_pretrained`) that handled model loading, tokenization, and generation. Under the hood, it delegated all tensor math to ONNX Runtime Web (ort-web), which could target either WebGPU or WASM backends.
+
+This worked for desktop browsers. On mobile -- particularly iOS -- it was catastrophic.
+
+### 2.2 The iOS Crisis
+
+The single most painful failure mode: **models would download to 100% and then crash the page**.
+
+iOS Safari and iOS Chrome (which uses WKWebView under the hood) impose a hard memory limit of approximately 300-400MB per web page. The ort-web WASM runtime alone consumed significant memory, and once model weights were loaded on top of that, the total exceeded the budget. The page would be silently terminated by the OS with no error, no exception, no callback -- just a blank page.
+
+Every mitigation was attempted and failed:
+
+- **Running in a Web Worker**: WKWebView imposes even stricter memory limits on workers than the main thread. Moving inference to a worker made things worse.
+- **Main-thread inference**: Avoided worker memory limits but froze the UI entirely during generation. Still crashed on models above ~600MB.
+- **KV cache disposal**: Aggressively disposing past_key_values after each generation freed some memory but not enough. The WASM runtime's own memory footprint was the floor, and it was already above 200MB.
+- **Model fallback chains**: Automatically trying smaller models when larger ones failed. This helped with availability but didn't solve the crash-then-reload loop.
+- **Session phase tracking**: Using `sessionStorage` to detect crash-and-reload cycles (`detectMemoryCrash()` in `device-guards.ts`). This only diagnosed the problem; it couldn't prevent it.
+- **Breadcrumb logging to localStorage**: Writing debug state before each critical step so that after a crash, the reload could read what step killed the page. This produced excellent diagnostics but no fix.
+
+The iOS code path grew increasingly complex. `createIOSMainThreadWorker()` in `worker.ts` is 270+ lines of iOS-specific logic including CDN dynamic imports, breadcrumb logging, 128-token generation caps, and manual KV cache disposal -- all of which existed solely to work around ort-web's memory consumption.
+
+### 2.3 The Bundle Size Problem
+
+The browser build bundled transformers.js and its dependencies (including kokoro-js for TTS) into a single ESM file via rolldown with `noExternal: ["@huggingface/transformers", "kokoro-js"]` and `inlineDynamicImports: true`. The resulting `dist/browser/index.js` was approximately 50MB.
+
+This meant:
+
+- Slow initial page loads even with caching
+- Massive JavaScript parse times on mobile devices
+- The browser had to allocate memory for the entire bundle before inference even began
+
+### 2.4 The Webpack-Within-Webpack Crash
+
+This was perhaps the most bizarre bug in the entire project. The `@huggingface/transformers` package ships as a **pre-compiled webpack bundle**. When rolldown inlined this bundle into gerbil's ESM output, a subtle scoping issue emerged:
+
+The transformers.js webpack runtime declares variables like `var __webpack_exports__`, `var __webpack_modules__`, `var __webpack_module_cache__`, and `var __webpack_require__` inside arrow function callbacks. Arrow functions in JavaScript **do not create a `var` scope** -- `var` declarations inside arrow functions are hoisted to the enclosing function scope (or module scope).
+
+When Next.js webpack processed gerbil's bundle (which now contained these inlined declarations), it evaluated the code inside `eval()` contexts where its own `__webpack_exports__` was a parameter. The inlined `var __webpack_exports__` hoisted through the arrow function and **shadowed** webpack's own parameter with `undefined`. This caused:
+
+```
+__webpack_require__.r(undefined)
+  -> Object.defineProperty(undefined, '__esModule', ...)
+    -> "Properties can only be defined on Objects"
+```
+
+The fix was a `renderChunk` plugin in `tsdown.config.ts` that renamed all inner webpack runtime variables:
+
+```typescript
+result = result.replaceAll("__webpack_exports__", "__gerbil_tf_exports__");
+result = result.replaceAll("__webpack_modules__", "__gerbil_tf_modules__");
+result = result.replaceAll("__webpack_module_cache__", "__gerbil_tf_cache__");
+result = result.replaceAll("__webpack_require__", "__gerbil_tf_require__");
+```
+
+Additionally, the plugin stripped `//# sourceMappingURL` comments to prevent the Next.js dev overlay from attempting to parse the 8MB source map file, which caused a stack overflow.
+
+This hack worked, but it was fragile. Every transformers.js update risked breaking it, and it highlighted a fundamental problem: the dependency chain was too deep and too opaque.
+
+### 2.5 The ONNX Dependency
+
+transformers.js required models to be pre-converted to ONNX format. This meant:
+
+- Only models that had been converted and uploaded to HuggingFace in ONNX format could be used
+- The conversion process itself was error-prone and model-specific
+- Quantization was limited to what the ONNX ecosystem supported
+- Users couldn't point at any arbitrary HuggingFace model repo and run it
+
+### 2.6 The Blob URL Worker Complexity
+
+iOS WKWebView cannot load ES module workers (`new Worker(url, { type: "module" })`). The workaround was an IIFE blob URL worker:
+
+1. `worker-entry.ts` -- real TypeScript worker source with static transformers.js imports
+2. `scripts/build-worker.mjs` -- esbuild bundles worker-entry.ts as IIFE into `worker-code.generated.ts`
+3. `worker.ts` -- creates a classic (non-module) Blob worker using the `WORKER_CODE` string constant
+
+This created a chain of build tooling issues:
+- The IIFE bundle couldn't use `import.meta.url`, which ort-web relied on internally for locating its WASM files
+- CDN paths for the WASM runtime (`ort.bundle.min.mjs`, `ort-wasm-simd-threaded.jsep.wasm`) had to be explicitly configured via `env.backends.onnx.wasm.wasmPaths`
+- These CDN files had to be aliased to `false` in Next.js's webpack config to prevent webpack from trying to resolve them as modules
+
+### 2.7 Three Separate Inference Paths
+
+The original gerbil maintained three completely different inference paths:
+
+1. **Browser WebGPU** via transformers.js (desktop Chrome, Edge, Safari 18+)
+2. **Browser WASM** via transformers.js (iOS, older browsers, low-GPU-memory devices)
+3. **Node.js** via server-side transformers.js with different model loading
+
+Each path had different bugs, different performance characteristics, and different maintenance burdens. The `backend-selector.ts` file alone was over 200 lines of tiered decision logic (`selectBackend()`, `getModelFallbackChain()`, `getDeviceContextLimit()`) to route users to the right path.
+
+### 2.8 Summary: Why a Rewrite Was Necessary
+
+| Problem | Root Cause |
+|---------|-----------|
+| iOS crashes after download | ort-web WASM runtime baseline memory too high for WKWebView |
+| 50MB bundle | Bundling transformers.js + ort-web + kokoro-js |
+| Webpack crash | var hoisting through arrow functions in pre-compiled webpack bundle |
+| Limited model support | ONNX format requirement |
+| Worker complexity | iOS WKWebView module worker limitations + ort WASM path resolution |
+| Maintenance burden | Three divergent inference code paths |
+
+The GPU engine eliminates all of these problems by owning the entire inference stack: a custom IR, custom safetensors parser, custom WGSL kernels, custom tokenizer, and custom sampler -- all speaking directly to the WebGPU API with zero intermediate runtimes.
+
+---
+
+## 3. Architecture Overview
+
+The GPU engine is structured as a pipeline of cooperating components:
+
+```
+                 HuggingFace Hub
+                       |
+                  model-loader.ts
+                 /       |       \
+           config.json  tokenizer.json  *.safetensors
+               |            |               |
+    architectures/     tokenizer.ts    safetensors.ts
+       qwen2.ts            |               |
+           |          Tokenizer         weight data
+           |                               |
+       ModelGraph (IR)                     |
+           |                               |
+       executor.ts  <----------------------+
+       /    |    \
+  device.ts | kv-cache.ts
+      |     |
+  WGSL kernels          sampler.ts
+  (12 shaders)     (CPU-side sampling)
+      |                    |
+  GPU dispatch        token selection
+      |                    |
+   logits  ----------->  next token
+```
+
+### Component Responsibilities
+
+| Component | File | Role |
+|-----------|------|------|
+| **IR** | `ir.ts` | Type definitions for the computation graph: OpType, TensorDesc, OpNode, ModelGraph |
+| **Architecture Registry** | `architectures/index.ts` | Maps HF architecture strings to graph generators |
+| **Graph Generator** | `architectures/qwen2.ts` | Builds a complete ModelGraph from a Qwen config.json |
+| **Safetensors Parser** | `safetensors.ts` | Parses HF safetensors binary format into typed array views |
+| **Device Layer** | `device.ts` | WebGPU initialization, buffer management, pipeline compilation, readback |
+| **KV Cache** | `kv-cache.ts` | Pre-allocated GPU buffers for autoregressive key/value storage |
+| **Executor** | `executor.ts` | Allocates buffers, dispatches ops, manages forward passes |
+| **WGSL Kernels** | `kernels/wgsl/*.wgsl` | 12 hand-written compute shaders for all tensor operations |
+| **Tokenizer** | `tokenizer.ts` | Pure JS BPE tokenizer from HF tokenizer.json |
+| **Sampler** | `sampler.ts` | CPU-side temperature/top-k/top-p/repetition penalty sampling |
+| **Model Loader** | `model-loader.ts` | HF Hub integration: fetch config, tokenizer, weights |
+
+### Data Flow
+
+1. **Model Load**: `model-loader.ts` fetches `config.json` from HuggingFace, determines the architecture (e.g., `Qwen2ForCausalLM`), and calls the matching graph generator. The generator produces a `ModelGraph` -- the complete IR specifying every tensor and every operation. The loader then downloads safetensors files, maps HF keys to canonical names, and returns the weights alongside the graph and tokenizer.
+
+2. **Initialization**: The `Executor` takes the `ModelGraph`, allocates a liveness-pooled set of GPU buffers for activation tensors (Section 18.1), creates the KV cache, and uploads weight data to GPU storage buffers.
+
+3. **Forward Pass**: Given input token IDs, the executor walks the `executionOrder` array, dispatching each operation's WGSL kernel with the appropriate buffers and uniforms. On Dawn (Chrome, node) all dispatches are batched into a single command encoder; on WebKit they are grouped into command buffers of a configurable size with one buffer in flight (Section 18.3).
+
+4. **Logit Readback**: After the forward pass, the `[1, vocab]` logits buffer (only the last position's logits are ever computed — Section 18.2) is copied to a MAP_READ staging buffer and read back to CPU as a `Float32Array`.
+
+5. **Sampling**: `sampler.ts` applies the sampling pipeline (repetition penalty, temperature, top-k, top-p, random selection) to produce the next token ID.
+
+6. **Decode Loop**: Steps 3-5 repeat with T=1 (single token) until an EOS token or max length is reached.
+
+---
+
+## 4. Intermediate Representation
+
+The IR (`ir.ts`) is the contract that every component builds on. It defines what the model computes, how tensors are shaped, and in what order operations execute.
+
+### Design Philosophy
+
+The IR is **fine-grained** and **runtime-generated**. Rather than compiling a model to an opaque binary format (like ONNX), the engine generates the IR at runtime from a model's `config.json`. This means:
+
+- **Any HuggingFace model** with a supported architecture can be used -- no pre-conversion step
+- **Config changes** (different hidden sizes, layer counts, head counts) are handled automatically
+- **The IR is inspectable**: it's a plain TypeScript object that can be logged, debugged, and visualized
+
+### OpType Taxonomy
+
+Every computation the engine can perform is represented as an `OpType`:
+
+**Core ops (implemented):**
+- `Embedding` -- Lookup rows from embedding weight matrix by token ID
+- `MatMul` -- Full-precision (f32) tiled matrix multiplication
+- `MatMulInt4` -- Fused INT4 dequantize + matrix multiplication
+- `Add` -- Element-wise addition (residual connections)
+- `Mul` -- Element-wise multiplication (SwiGLU gate)
+- `RMSNorm` -- Root mean square layer normalization
+- `LayerNorm` -- Standard layer normalization (mean + variance)
+- `RoPE` -- Rotary position embeddings
+- `Attention` -- Scaled dot-product attention with causal mask and GQA
+- `Softmax` -- Row-wise softmax
+- `SiLU` -- Sigmoid linear unit activation
+- `GELU` -- Gaussian error linear unit activation (tanh approximation, `gelu_pytorch_tanh`)
+- `GeluErf` -- Exact (erf-based) GELU, used by the ViT merger
+- `AddBias` -- Row-broadcast bias add (ViT projections carry bias)
+- `ApplyRotaryEmb` -- Precomputed-cos/sin `rotate_half` rotary (ViT 2D RoPE)
+- `SliceCols` -- Extract a column range (splits fused QKV in the ViT)
+- `L2Norm` -- Row-wise L2 normalization (embedding tail)
+- `SliceLastRow` -- Extract the last position's row (last-token pooling / lm_head)
+
+**Structural ops (defined, not yet kernel-backed):**
+- `Gather`, `Reshape`, `Transpose`, `Concat`
+
+**Future ops (stubbed):**
+- `MoERouter`, `ExpertMatMul` -- Mixture of Experts (when first MoE model is added)
+- `Conv2d`, `AvgPool2d`, `CrossAttention` -- audio / encoder-decoder (future)
+
+### TensorDesc
+
+Every tensor in the graph is described by a `TensorDesc`:
+
+```typescript
+interface TensorDesc {
+  name: string;                    // Unique name, e.g. "layers.0.self_attn.q_proj.weight"
+  shape: (number | string)[];     // Concrete or symbolic dimensions
+  dtype: DType;                   // "f32" | "f16" | "i32" | "u32" | "i4"
+  storage: TensorStorage;         // "constant" | "activation" | "kv_cache"
+  safetensorsKey?: string;        // Key in safetensors file (constants only)
+}
+```
+
+**Symbolic dimensions** are strings like `"T"` (current sequence length) and `"L_max"` (maximum cache length). These are resolved to concrete numbers at execution time. This allows the graph to be generated once and used for both prefill (T=prompt length) and decode (T=1) passes.
+
+**Storage types:**
+- `constant` -- Weight tensors loaded from safetensors. Immutable after upload.
+- `activation` -- Intermediate computation results. Buffers are pre-allocated at max size and reused.
+- `kv_cache` -- Key/value cache tensors managed by the KV cache module.
+
+### OpNode
+
+Each operation is represented as an `OpNode`:
+
+```typescript
+interface OpNode {
+  id: string;                           // Unique node ID, e.g. "layer0_norm1"
+  opType: OpType;                       // Which operation to perform
+  inputs: string[];                     // Input tensor names (order = kernel binding order)
+  outputs: string[];                    // Output tensor names
+  attributes: Record<string, unknown>;  // Op-specific params (hidden_size, eps, etc.)
+}
+```
+
+The `attributes` dictionary carries operation-specific parameters that the kernel needs. For example, an `RMSNorm` node carries `hidden_size` and `eps`; a `MatMul` node carries `M_tensor`, `K`, and `N`.
+
+### ModelGraph
+
+The top-level container:
+
+```typescript
+interface ModelGraph {
+  architecture: string;                    // e.g. "Qwen2ForCausalLM"
+  config: ModelArchConfig;                 // Resolved dimensions
+  capabilities: ModelCapabilities;         // { text: true, vision: false, moe: false }
+  tensors: Record<string, TensorDesc>;     // All tensors, keyed by name
+  nodes: OpNode[];                         // All computation nodes
+  executionOrder: string[];                // Topologically-sorted node IDs
+  inputs: string[];                        // Graph input names (["input_ids"])
+  outputs: string[];                       // Graph output names (["logits"])
+}
+```
+
+### Canonical Tensor Naming
+
+The IR uses a canonical naming convention for weight tensors:
+
+```
+embed_tokens.weight                          -- Token embedding matrix
+layers.{i}.input_layernorm.weight            -- Pre-attention norm
+layers.{i}.self_attn.q_proj.weight           -- Q projection
+layers.{i}.self_attn.k_proj.weight           -- K projection
+layers.{i}.self_attn.v_proj.weight           -- V projection
+layers.{i}.self_attn.o_proj.weight           -- Output projection
+layers.{i}.post_attention_layernorm.weight   -- Post-attention norm
+layers.{i}.mlp.gate_proj.weight              -- MLP gate (SwiGLU)
+layers.{i}.mlp.up_proj.weight                -- MLP up projection
+layers.{i}.mlp.down_proj.weight              -- MLP down projection
+norm.weight                                  -- Final norm
+lm_head.weight                               -- Language model head
+```
+
+This convention is defined in `CANONICAL_KEYS` and matches the common LLaMA/Qwen naming scheme. The `HFKeyMapper` (default: strip the `"model."` prefix) converts HuggingFace safetensors keys to these canonical names.
+
+For more details, see [ir.md](./ir.md).
+
+---
+
+## 5. Safetensors Parser
+
+The safetensors parser (`safetensors.ts`) reads HuggingFace's binary safetensors format into typed array views over the raw buffer.
+
+### Binary Format
+
+```
+[8 bytes: header_length (little-endian u64)]
+[header_length bytes: JSON header]
+[remaining bytes: raw tensor data, contiguous]
+```
+
+The JSON header maps tensor names to their metadata:
+
+```json
+{
+  "model.layers.0.self_attn.q_proj.weight": {
+    "dtype": "F32",
+    "shape": [896, 896],
+    "data_offsets": [0, 3211264]
+  },
+  "__metadata__": {
+    "format": "pt"
+  }
+}
+```
+
+### Zero-Copy Design
+
+`getTensorData()` creates typed array views directly into the original `ArrayBuffer` when byte alignment allows. For example, an F32 tensor at a 4-byte-aligned offset returns a `Float32Array` view with no data copy. When alignment requirements aren't met (rare in practice), the function copies the relevant slice into a new properly-aligned buffer.
+
+### Streaming Support
+
+`parseSafetensorsFromResponse()` accepts a `Response` object and reads it into an `ArrayBuffer`. The header can be parsed independently of the tensor data, enabling a two-phase approach for large models: parse the header first to learn tensor sizes and offsets, then fetch tensor data by range request.
+
+For more details, see [safetensors.md](./safetensors.md).
+
+---
+
+## 6. WebGPU Device Layer
+
+The device layer (`device.ts`) wraps the WebGPU API with helpers for buffer management, pipeline compilation, and data readback.
+
+### Device Initialization
+
+`initGPU()` requests a high-performance GPU adapter and device with:
+
+- Maximum buffer size and storage buffer binding size from the adapter
+- 256x256 max compute workgroup size
+- `shader-f16` feature when available
+- A device loss handler that logs the reason
+
+```typescript
+const adapter = await navigator.gpu.requestAdapter({
+  powerPreference: "high-performance",
+});
+```
+
+If `shader-f16` is available, the pipeline compiler automatically prepends `enable f16;` to WGSL source code.
+
+### WebKit Implementation Detection (`isWebKitWebGPU`)
+
+`initGPU()` classifies the WebGPU *implementation*, not the GPU hardware. This distinction matters: Dawn running on Apple Silicon (node-dawn, Chrome on macOS) reports `adapter.info.vendor === "apple"` exactly like Safari does, so adapter info **cannot** distinguish Dawn-on-Metal from WebKit-on-Metal. An earlier predicate that keyed on vendor/architecture routed desktop Dawn through every Safari workaround, collapsing desktop decode from ~161 to 19.7 tok/s (Section 17.4).
+
+The detection is now purely user-agent based (`device.ts`): `AppleWebKit` in the UA, with iOS/iPadOS detected via `/iPhone|iPad|iPod/` or `maxTouchPoints > 1` (iPadOS masquerades as macOS), and macOS Safari as `AppleWebKit && !Chrome/`. Node has no `navigator.userAgent`, so node-dawn resolves to `false`. Only `isWebKitWebGPU === true` enables the grouped-submit path and other WebKit workarounds.
+
+### Uncaptured Error Surfacing
+
+`initGPU()` registers `device.onuncapturederror`. Without it, failed buffer allocations and invalid bind groups on Safari are completely silent — the affected dispatches no-op and the symptom is simply zero logits. `WebGPUEngine.create()` additionally wraps executor construction, weight upload, and bind-group creation in paired `pushErrorScope("out-of-memory")` / `pushErrorScope("validation")` scopes and throws a descriptive error if either pops non-null. Before this, every iPad failure was observed blind (Section 18.6).
+
+### Buffer Types
+
+| Buffer Type | Usage Flags | Alignment | Purpose |
+|------------|-------------|-----------|---------|
+| Storage | STORAGE \| COPY_SRC \| COPY_DST | 4 bytes | Weights, activations, KV cache |
+| Uniform | UNIFORM \| COPY_DST | 16 bytes | Kernel parameters |
+| Readback | MAP_READ \| COPY_DST | 4 bytes | Reading logits back to CPU |
+
+### Pipeline Compilation and Caching
+
+`getOrCreatePipeline()` compiles WGSL shader code into a `GPUComputePipeline` and caches it by a key of `shaderCode::entryPoint`. This ensures each unique kernel is compiled exactly once per session. The cache can be cleared with `clearPipelineCache()` when switching models or recovering from device loss.
+
+### Logit Readback
+
+The primary CPU-GPU synchronization point is logit readback. `readbackFloats()` copies data from a storage buffer to a MAP_READ staging buffer, maps it, and returns a `Float32Array` copy. This is used after each forward pass to get the logit distribution for sampling.
+
+### GPU Diagnostics Suite (A-P)
+
+`verifyGPU()` runs 17 progressive diagnostics testing increasingly complex GPU patterns. This isolates Safari/Metal-specific bugs without needing a full model load:
+
+| Test | What It Verifies |
+|------|-----------------|
+| A-E | Basic compute, input binding, workgroups, shared memory, separate encoders |
+| F-G | pack2x16float / unpack2x16float (packed-f16 KV path) |
+| H | 256-thread tree reduction (RMSNorm/softmax pattern) |
+| I | INT4 dequant (nibble extraction + scale/zero) |
+| J | Multi-dispatch chain (3 sequential dispatches in one pass) |
+| K | exp() clamp safety (Metal fast-math NaN check) |
+| L | Large buffer integrity (1 MB upload + readback) |
+| M | Offset buffer integrity (view with non-zero byteOffset, simulates safetensors views) |
+| N | Real RMSNorm kernel (actual WGSL from registry, hidden=1024) |
+| O | Real MatVec kernel (actual WGSL, K-parallel layout) |
+| P | 300 dispatches in one compute pass (model does ~300/forward) |
+
+Available via `WebGPUEngine.quickDiagnose()` (no model needed) or `engine.diagnose()`.
+
+### Integrity Checker
+
+`integrityCheck()` reads back weight tensors from the GPU and runs a single forward pass, producing checksums that can be compared between Dawn (known-good) and Safari. This pinpoints WHERE corruption occurs:
+
+- **Weights match, logits differ** → Kernel computation bug on Metal (WGSL→MSL miscompilation)
+- **Weights match, embed_out all zeros** → writeBuffer/dispatch synchronization bug (Safari reads stale uniform params)
+- **Weights don't match** → Data transfer/download corruption (Safari fetch or writeBuffer bug)
+
+Weight checksums are compared against hardcoded Dawn reference values. Logits are validated for corruption signals (NaN, Inf, all-same values). Automatically runs after engine loads (all browsers, not just Safari — results shown in Playground UI for devices without console access).
+
+---
+
+## 7. WGSL Kernel Library
+
+The engine includes 12 hand-written WGSL compute shaders covering all operations needed for transformer inference. Each kernel is in `src/gpu/kernels/wgsl/`.
+
+For a comprehensive reference of each kernel's algorithm, bindings, uniform layout, dispatch formula, and optimization opportunities, see [kernels.md](./kernels.md).
+
+### Kernel Summary
+
+| Kernel | File | Workgroup Size | Algorithm |
+|--------|------|---------------|-----------|
+| Embedding | `embedding.wgsl` | (256, 1, 1) | Parallel row gather from weight matrix |
+| MatMul | `matmul.wgsl` | (16, 16, 1) | 16x16 tiled multiply with shared memory |
+| MatMulInt4 | `matmul_int4.wgsl` | (16, 16, 1) | Fused INT4 dequantize + multiply |
+| RMSNorm | `rmsnorm.wgsl` | (256, 1, 1) | One workgroup per row, tree reduction |
+| LayerNorm | `layernorm.wgsl` | (256, 1, 1) | Two-pass (mean, variance), tree reduction |
+| RoPE | `rope.wgsl` | (256, 1, 1) | Parallel pair rotation on Q and K |
+| Attention | `attention.wgsl` | (256, 1, 1) | Per-head causal attention with GQA |
+| Softmax | `softmax.wgsl` | (256, 1, 1) | Three-pass (max, sum, normalize) with tree reduction |
+| SiLU | `silu.wgsl` | (256, 1, 1) | Element-wise x / (1 + exp(-x)) |
+| GELU | `gelu.wgsl` | (256, 1, 1) | Approximate GELU with tanh |
+| Add | `add.wgsl` | (256, 1, 1) | Element-wise addition |
+| Mul | `mul.wgsl` | (256, 1, 1) | Element-wise multiplication |
+
+### Design Notes
+
+The kernels use a performance-oriented design. Key optimizations:
+
+- **Attention**: Tiled online-softmax kernel (Flash-Attention style). Processes KV cache in TILE_S=16 tiles with cooperative loading into shared memory. 256 threads split into groups of 16 for parallel Q·K dot products with vec4 loads, parallel tree reductions for max/sum, and per-dimension V accumulation. Uses exactly 16 KB shared memory (minimum WebGPU guarantee) for iOS compatibility. Score registers are saved before V tile load to allow full shared memory reuse. The Q·K score reduction is **two-phase**: leader threads reduce their group's partials into a local variable, hit a `workgroupBarrier()` in uniform control flow, and only then write `smem[pos_in_tile]`. The earlier single-phase version was a genuine WGSL data race — leader 0 read `smem[1..15]` while leaders 1..15 wrote `smem[0..15]` after the same barrier (Section 17.3). The barrier is a race fix, not a performance knob.
+- **GEMV (decode)**: K-parallel matrix-vector multiply with 8 output columns × 32 K-threads per column, vec4 dot products, single-barrier reduction. Optimized for Apple Silicon SIMD width 32.
+- **MatMul (prefill)**: 16×16 tiling with shared memory tiles.
+- **SwiGLU**: Fused SiLU + Mul in a single kernel (saves 2 dispatches per MLP block).
+- **SwiGLU MatVec**: Fused gate_proj + up_proj + SwiGLU for M=1 decode. Reads input vector once from L1 cache, computes both weight matrix projections, and applies SiLU gating — all in a single dispatch. Saves 2 dispatches per MLP block vs separate gate + up + SwiGLU. The executor detects the gate_proj → up_proj → SwiGLU pattern and automatically substitutes the fused kernel for decode.
+- **ResidualRMSNorm**: Fused residual add + RMS normalization (saves 2 dispatches per layer).
+
+Safari/Metal compatibility: the attention kernel avoids WGSL `select()` (Safari has correctness bugs) and clamps `exp()` arguments to -80 (Metal returns NaN for `exp(-1e30)` instead of 0). Additionally, Safari uses packed f16 KV cache (`array<u32>` + `pack2x16float`/`unpack2x16float`) instead of native `array<f16>`, which WebKit's WGSL compiler miscompiles. (The packed-f16 path was A/B-verified against `?kvf32=1` on iPadOS 26.5: both produce byte-identical greedy output — Section 19.) Submit granularity on WebKit is handled by the executor's grouped-submit path, not by kernel changes (Section 18.3).
+
+The matmul kernel uses a straightforward 16x16 tiling strategy without register blocking.
+
+### Optimization Roadmap
+
+Remaining optimizations, prioritized by expected impact. Research from 63 sources covering WebGPU spec, Apple Silicon architecture docs, Chrome/Safari implementation details, and Metal compute best practices.
+
+#### Priority 1: f16 KV Cache — DONE
+
+Halves memory traffic through the attention kernel during decode (192 MB vs 384 MB for Qwen3.5-0.8B at maxSeqLen=4096).
+
+**Three kernel strategies** via `KvMode`:
+- **`native-f16`** (Chrome/Dawn): `enable f16;` + `array<f16>` for K/V buffers. Direct `f16()` cast on write, `f32()` on read. Best performance on platforms with correct f16 WGSL support.
+- **`packed-f16`** (Safari): `array<u32>` + `pack2x16float()`/`unpack2x16float()`. No `enable f16` directive — avoids WebKit's WGSL compiler miscompilation of native f16 types. Same memory savings, slightly more ALU for pack/unpack.
+- **`f32`** (fallback): Standard f32 buffers for devices without shader-f16.
+
+**Auto-detection** in `WebGPUEngine.create()`: Safari → packed-f16, Chrome/Dawn → native-f16, no f16 → f32. Override via `?kvf32=1` or `?maxseq=N` URL params.
+
+**Safari/WebKit bug**: Native `f16` types (`array<f16>`, `f16()` casts) cause garbage output on WebKit's WGSL compiler (confirmed iPad Safari, likely Metal shader compiler). The packed approach sidesteps this entirely by only using `u32` and `f32` types with standard pack/unpack built-ins.
+
+#### Priority 2: Subgroup Operations (est. 1.1-1.3×)
+
+The stable directive is `enable subgroups;` (shipped Chrome 134). Apple Silicon subgroupSize is always 32. The current 256-thread max reduction drops from 8 barriers + 1024 bytes of shared memory to 1 barrier + 32 bytes: each subgroup does a hardware `subgroupMax()`, leaders write to a tiny scratch array, one barrier, then subgroup 0 reduces the 8 values.
+
+**Critical**: Subgroup ops must be in uniform control flow by default (`subgroup_uniformity` diagnostic errors otherwise). Cross-subgroup `if (sg_index == 0u)` blocks ARE uniform within each subgroup, so they pass analysis. Safari 26 does NOT expose subgroups — maintain the shared-memory fallback. Feature detection required.
+
+#### Priority 3: Dynamic Uniform Buffer Offsets (est. 1.1-1.3×)
+
+`minUniformBufferOffsetAlignment` = 256 bytes everywhere. Pack all ~441 op params at 256-byte stride into a single ~113 KB buffer. Can mix dynamic and static bindings in the same bind group. During dispatch, `pass.setBindGroup(0, bg, [i * 256])` selects each op's params. Only update the ~24 slots with changing `seq_pos` per step via targeted `writeBuffer` calls.
+
+#### Priority 4: Fused GEMV + Residual
+
+Thread 0 of the K-parallel reduction adds `residual[row]` before writing output — single extra read, 48 dispatches saved ≈ ~1.4ms at 30μs per dispatch overhead.
+
+#### Priority 5: Fused KV Append
+
+Combine K+V cache writes into one kernel. Trivial, saves 12 dispatches. Best done alongside f16 KV cache work.
+
+#### Priority 6: Advanced GEMV (Register Tiling)
+
+Register tiling (4 output rows per thread) is the biggest GEMV win — amortizes input vector reads 4×, increasing arithmetic intensity. Apple GPU has 128 GPRs per SIMD-group, current kernels use ~20-40, so 4 extra accumulators fit easily. Double-buffering is overkill for single-token GEMV where the input vector (4 KB) fits in L1.
+
+#### Priority 7: Timestamp Profiling
+
+Per-pass timestamps via `timestampWrites` on pass descriptor. `writeTimestamp()` was removed from WebGPU spec. Safari 26 does NOT support `timestamp-query`. Chrome quantizes to 100μs by default; `chrome://flags/#enable-webgpu-developer-features` for full precision. Essential for finding real bottlenecks before investing in lower-impact optimizations.
+
+#### Not Worth Pursuing
+
+- **Indirect dispatch** (`dispatchWorkgroupsIndirect()`): Slower than direct dispatch due to Chrome's validation overhead. Not useful for fixed architectures.
+- **Double-buffering input**: Overkill for decode where the input vector is 4 KB and fits in L1.
+
+#### Platform Capability Matrix
+
+| Feature | Chrome 134+ | Safari 26 |
+|---------|-------------|-----------|
+| `shader-f16` | ✅ | ✅ |
+| `subgroups` | ✅ | ❌ |
+| `timestamp-query` | ✅ (100μs quant) | ❌ |
+| `maxComputeWorkgroupStorageSize` | 32768 (requestable) | 16384 (spec minimum) |
+
+#### Apple Silicon Occupancy
+
+Current 256-thread workgroups with 16 KiB shared memory give ~50% occupancy (2 threadgroups per compute unit sharing the 32 KiB threadgroup memory). This is good for memory-bound kernels where DRAM bandwidth (68-100 GB/s), not compute, is the bottleneck. Smaller workgroups would just increase reduction overhead. Using f16 types reduces register pressure, potentially enabling higher occupancy as a side benefit.
+
+#### CPU-GPU Sync Notes
+
+`mapAsync()` already guarantees prior work completion — no need for `onSubmittedWorkDone()`. `createCommandEncoder()` cost is <1μs and cannot be avoided (command buffers are single-use in WebGPU). The 441 dispatches cost ~3.5-5.5ms of CPU encoding time. Double-buffered staging buffers could overlap readback with the next step's encoding.
+
+**Current baselines (2026-06-13, Qwen3.5-0.8B MLX 4-bit, greedy; see `docs/research/tps-baselines.md`)**: M4 Max node-dawn **207 tok/s** (cooled-stable, after the autoresearch optimization campaign in §20; 145 was the post-detection-fix starting point); iPad (iPadOS 26.5) **31.7 tok/s** batch-all, up to **51.7 tok/s** sustained.
+
+**Goals**: desktop **180+ tok/s** via kernel tuning (K_THREADS/N_TILE/workgroup sweeps), fused KV append (−24 dispatches), fused GEMV+residual (−48 dispatches); iPad **50+ tok/s** via single-compute-pass batching on iPadOS 26.5+ plus dispatch-count reduction — every dispatch eliminated helps mobile 2-3× more than desktop because mobile is dispatch-overhead-bound (Section 19).
+
+**Mobile invariants any optimization must respect**: iPad `maxComputeWorkgroupStorageSize` = 16384 (the attention kernel sits exactly at this limit); iPad default `maxBufferSize` = 256MB and `maxStorageBufferBindingSize` = 128MB (the INT4 embedding is ~127MB — no headroom for larger vocabularies without sharding); activation buffers are liveness-pooled, so fused kernels must never read and write the same pooled buffer in one dispatch; the two-phase attention reduction barrier is a race fix, not a perf knob.
+
+---
+
+## 8. Tokenizer
+
+The tokenizer (`tokenizer.ts`) is a pure JavaScript BPE implementation that reads HuggingFace `tokenizer.json` files directly. No WASM dependencies, no external libraries.
+
+### Encoding Pipeline
+
+1. **Added token splitting**: Text is split around ALL added tokens (not just `special: true` ones) using a regex built from the sorted token list. This ensures tokens like `<think>` and `</think>` (which HF marks as non-special) are correctly recognized during encoding
+2. **Pre-tokenization**: Non-special segments are split using a GPT-style regex pattern that separates contractions, words, numbers, and punctuation
+3. **Byte-level encoding**: Characters are converted to the HF byte representation (space becomes `\u0120`, control characters are offset by 256)
+4. **BPE merge**: Character sequences are iteratively merged according to the merge priority table until no more merges are possible
+5. **Byte fallback**: Characters not in the vocabulary are encoded as `<0xHH>` byte-level tokens
+
+### Decoding
+
+Decoding reverses the process: token IDs are mapped back to strings, joined, and post-processed to restore spaces (from `\u0120`) and byte-level tokens (from `<0xHH>` format).
+
+### Chat Template
+
+The tokenizer implements ChatML format for chat-style models:
+
+```
+<|im_start|>system
+{system message}<|im_end|>
+<|im_start|>user
+{user message}<|im_end|>
+<|im_start|>assistant
+```
+
+A TODO exists for parsing Jinja2 templates from `tokenizer_config.json` for full generality across model families.
+
+For a detailed walkthrough with a worked example, see [tokenizer.md](./tokenizer.md).
+
+---
+
+## 9. Sampler
+
+Token sampling is performed on CPU (`sampler.ts`). The rationale is straightforward: for a vocabulary of ~150K tokens, the logits array is ~600KB -- well within CPU memory bandwidth. Moving sampling to GPU would add dispatch overhead and a synchronization point without meaningful benefit.
+
+### Sampling Pipeline
+
+Given a `Float32Array` of logits:
+
+1. **Greedy check**: If temperature < 1e-6, return `argmax(logits)` immediately
+2. **Copy**: Work on a copy to avoid mutating the source
+3. **Repetition penalty**: For each previously generated token, divide positive logits by the penalty factor and multiply negative logits by it
+4. **Temperature**: Divide all scores by temperature
+5. **Top-k**: Sort scores descending, keep only the top K candidates
+6. **Softmax**: Compute `exp(score - max)` and normalize to get probabilities
+7. **Top-p (nucleus)**: Keep the smallest set of tokens whose cumulative probability exceeds `topP`, then re-normalize
+8. **Weighted random sample**: Draw a random number and walk the cumulative distribution
+
+### Default Parameters
+
+```typescript
+temperature: 0.7
+topK: 50
+topP: 0.9
+repetitionPenalty: 1.0  // disabled
+```
+
+---
+
+## 10. Architecture Registry & Graph Generators
+
+The architecture registry (`architectures/index.ts`) maps HuggingFace model architecture strings (from `config.architectures[0]`) to graph generator functions.
+
+### How It Works
+
+```typescript
+const ARCHITECTURES: Record<string, GraphGenerator> = {
+  Qwen2ForCausalLM: generateQwen2Graph,
+  Qwen3ForCausalLM: generateQwen2Graph,
+  // LlamaForCausalLM: generateLlamaGraph,  // TODO
+};
+```
+
+When a model is loaded, the engine reads `config.architectures[0]` and looks it up in this registry. The matched generator receives the raw `config.json` object and returns a complete `ModelGraph`.
+
+### Qwen2 Graph Generator
+
+`architectures/qwen2.ts` generates the graph for Qwen2/Qwen3 models. Each transformer layer produces 12 operation nodes:
+
+1. `RMSNorm` -- input_layernorm
+2. `MatMul` -- Q projection
+3. `MatMul` -- K projection
+4. `MatMul` -- V projection
+5. `RoPE` -- rotate Q and K
+6. `Attention` -- scaled dot-product with causal mask and GQA
+7. `MatMul` -- output projection
+8. `Add` -- residual connection 1
+9. `RMSNorm` -- post_attention_layernorm
+10. `MatMul` -- gate projection (SwiGLU)
+11. `MatMul` -- up projection
+12. `SiLU` -- activation on gate output
+13. `Mul` -- gate * up (SwiGLU combine)
+14. `MatMul` -- down projection
+15. `Add` -- residual connection 2
+
+Plus 4 global nodes: `Embedding`, `RMSNorm` (final), `SliceLastRow` (extract the last position of `final_norm_out`), and `MatMul` (lm_head, M=1). Both graph generators (`qwen2.ts`, `qwen3_5.ts`) emit the `SliceLastRow` → `lm_head` pair so that logits are computed only for the last position (Section 18.2).
+
+For a Qwen3.5-0.8B model (24 layers), this produces:
+- ~360 operation nodes (15 per layer * 24 + 3)
+- ~500+ tensor descriptors
+
+### Adding a New Architecture
+
+See [architectures.md](./architectures.md) for a step-by-step guide.
+
+---
+
+## 11. KV Cache
+
+The KV cache (`kv-cache.ts`) stores previously computed key and value vectors in GPU memory, enabling autoregressive generation without recomputing attention over the full sequence at each step.
+
+### LHSd Layout
+
+The cache uses LHSd (Layer, Head, Sequence, head_dim) layout:
+
+```
+Per layer:
+  K buffer: [num_kv_heads, max_seq_len, head_dim]   f32
+  V buffer: [num_kv_heads, max_seq_len, head_dim]   f32
+```
+
+Each layer has its own pair of K and V GPU storage buffers.
+
+### Pre-allocation Strategy
+
+Buffers are allocated at `max_seq_len` capacity when the executor is created. This avoids dynamic reallocation during generation. The tradeoff is higher initial memory usage, but it eliminates allocation jitter and potential out-of-memory errors mid-generation.
+
+### Memory Budget Calculation
+
+For a model with `L` layers, `H_kv` KV heads, `d` head dimension, and max sequence length `S`:
+
+```
+KV cache bytes = L * H_kv * S * d * 4 (f32) * 2 (K + V)
+```
+
+Example for Qwen3.5-0.8B (24 layers, 4 KV heads, 64 head_dim, 2048 max seq):
+```
+24 * 4 * 2048 * 64 * 4 * 2 = 100,663,296 bytes (~96 MB)
+```
+
+### Prefill vs. Decode
+
+- **Prefill** (T = prompt length): All T tokens' K/V entries are written at once at positions [0, T-1]. The attention kernel reads and writes the full range.
+- **Decode** (T = 1): A single K/V entry is written at position `seqPos`. The attention kernel reads positions [0, seqPos] and writes position [seqPos].
+
+The `seqPos` counter in the `KVCache` struct tracks how many tokens have been cached. `advanceKVCache()` increments it after each forward pass. `resetKVCache()` sets it to 0 for a new generation without reallocating buffers.
+
+---
+
+## 12. Executor
+
+The executor (`executor.ts`) is the core runtime that orchestrates buffer allocation, weight upload, and compute dispatch.
+
+### Buffer Allocation Strategy
+
+The executor allocates activation buffers once at construction, sized for `maxSeqLen` — there is no dynamic allocation during generation and memory usage is deterministic and front-loaded. But buffers are **not** one-per-tensor: `allocateActivationBuffers()` runs a last-use liveness analysis over `graph.executionOrder` and recycles buffers through a size-keyed free pool, so tensors that are never live simultaneously share storage. For Qwen3.5-0.8B this collapses 431 activation tensors to 20 physical buffers (37 MB at T=256) — versus ~2.3 GB for the naive per-tensor scheme that was jetsam-killing iPads (Section 17.1). Graph outputs and tensors that are read before they are written (cross-forward state) are excluded from pooling and keep dedicated buffers. See Section 18.1 for the full algorithm.
+
+Weight buffers are allocated on demand during `uploadWeights()` and are sized to exactly match the data.
+
+### Forward Pass
+
+```typescript
+async forward(inputIds: Uint32Array): Promise<ForwardResult>
+```
+
+The forward pass uses a **two-phase dispatch pattern**:
+
+**Phase 1 — Uniform buffer updates** (before any dispatch):
+
+1. Writes token IDs to the pre-allocated `input_ids` buffer
+2. Resolves all symbolic shapes (`"T"` -> actual T, `"L_max"` -> seqPos + T)
+3. For each operation: builds uniform params, compares against cached bytes, and calls `device.queue.writeBuffer()` only if changed
+
+**Phase 2 — Compute dispatch** (no writeBuffer calls), with a per-implementation submit strategy:
+
+- **Dawn (Chrome, node)**: a single `GPUCommandEncoder` with one compute pass containing all ~440 dispatches, plus the logits copy, submitted in one batch.
+- **WebKit** (`needsMultiEncoder === true`): dispatches are grouped into command buffers of `webkitGroupSize` (one compute pass per dispatch), each submitted and **awaited via `onSubmittedWorkDone()` before the next is encoded** — exactly one command buffer in flight. The logits copy goes in its own final submit. See Section 18.3 for why, and Section 19 for the throughput-vs-group-size curve.
+
+Either way the pass then:
+
+4. Copies the `[1, vocab]` logits buffer to the readback buffer (full buffer, offset 0 — `SliceLastRow` already restricted lm_head to the last position)
+5. Maps the readback buffer and returns the logits as `Float32Array`
+6. Advances the KV cache position by T
+
+`forwardArgmax()` (greedy decode without logits readback) follows the same structure, with the argmax dispatch and its 4-byte readback copy in separate submits on WebKit, mirroring `forward()`.
+
+> **History — the "Safari writeBuffer bug" callout, revised**: An earlier version of this section blamed iPad gibberish on WebKit failing to synchronize `writeBuffer()` calls made *during* an active compute pass: stale zero params → all threads exit early → all-zero output → degenerate text at ~276 tok/s (zero-work dispatches are nearly free, which is why the "throughput" was 2-4× desktop). The throughput signature and the "zeros, not garbage" character of that diagnosis were correct — the recorded gibberish *was* zero logits in disguise. But the June 2026 investigation **refuted stale params as the mechanism** (Section 17.2): the kernels' write guards mean zeroed params would have written *nothing*, yet probes showed zeros actively overwriting pre-seeded data, and the failure flipped correct↔zeros purely on submit granularity with byte-identical CPU-side writes. The real cause was a within-submission storage-visibility bug in WebKit, since fixed upstream (does not reproduce on iPadOS 26.5). The two-phase write-then-dispatch pattern is retained as cheap, spec-clean hygiene, but it was never the fix.
+
+### Prefill vs. Decode Dispatch
+
+The same forward pass code handles both prefill and decode -- the only difference is the value of T:
+
+- **Prefill**: T = prompt token count (e.g., 50). Matmul dispatch sizes are larger, attention processes T queries against T keys.
+- **Decode**: T = 1. Single-token forward pass. Matmul dispatch is minimal, attention processes 1 query against seqPos keys.
+
+The kernel dispatch sizing functions (in the kernel registry) use the resolved shapes to compute appropriate workgroup counts for either case.
+
+---
+
+## 13. Model Loading Pipeline
+
+The model loader (`model-loader.ts`) orchestrates the entire process of downloading and preparing a model from HuggingFace Hub.
+
+### Loading Steps
+
+1. **Fetch config.json**: Determine architecture, extract dimensions
+2. **Generate IR**: Call the matching graph generator to produce the `ModelGraph`
+3. **Fetch tokenizer**: Download `tokenizer.json` and `tokenizer_config.json` in parallel, construct the `Tokenizer`
+4. **Discover safetensors files**: Try `model.safetensors.index.json` first (for sharded models), fall back to single `model.safetensors`
+5. **Download weight files**: Stream each safetensors file with progress callbacks
+6. **Parse and map weights**: Parse safetensors headers, extract typed array views, map HF keys to canonical names via `HFKeyMapper`
+7. **Verify**: Check that all expected weight tensors are present; warn about missing ones
+
+### HuggingFace Hub Integration
+
+The loader resolves repo strings to HF Hub URLs:
+
+```
+"Qwen/Qwen3.5-0.8B" -> "https://huggingface.co/Qwen/Qwen3.5-0.8B/resolve/main"
+```
+
+It supports:
+- Gated models via `hfToken` authentication
+- Custom revisions/branches
+- Full URL pass-through (for self-hosted models)
+- Multi-shard safetensors models (via the index file)
+
+### Progress Tracking
+
+The loader provides normalized progress (0-100) to the caller:
+- 0-5%: Fetching config
+- 5-10%: Fetching tokenizer
+- 10-95%: Downloading weight files (distributed evenly across shards)
+- 95-100%: Final verification
+
+---
+
+## 14. Public API (WebGPUEngine)
+
+The engine is designed to be used through a high-level API surface. Based on the current codebase, the intended usage pattern is:
+
+```typescript
+import { initGPU } from "./device.js";
+import { loadModel } from "./model-loader.js";
+import { Executor } from "./executor.js";
+import { sampleToken } from "./sampler.js";
+
+// 1. Initialize WebGPU
+const ctx = await initGPU();
+
+// 2. Load model from HuggingFace
+const { graph, tokenizer, weights } = await loadModel({
+  repo: "Qwen/Qwen3.5-0.8B",
+  onProgress: (loaded, total, msg) => console.log(`${loaded}/${total}: ${msg}`),
+});
+
+// 3. Create executor and upload weights
+const executor = new Executor(ctx, graph, { maxSeqLen: 2048 });
+executor.uploadWeights(weights);
+
+// 4. Encode prompt
+const inputIds = new Uint32Array(
+  tokenizer.encodeChat([
+    { role: "system", content: "You are a helpful assistant." },
+    { role: "user", content: "Hello!" },
+  ])
+);
+
+// 5. Prefill
+let result = await executor.forward(inputIds);
+let nextToken = sampleToken(result.logits, { temperature: 0.7, topK: 50, topP: 0.9 });
+
+// 6. Decode loop
+const maxTokens = 256;
+const generated: number[] = [nextToken];
+
+while (generated.length < maxTokens && nextToken !== tokenizer.config.eosTokenId) {
+  result = await executor.forward(new Uint32Array([nextToken]));
+  nextToken = sampleToken(result.logits, { temperature: 0.7 }, generated);
+  generated.push(nextToken);
+  process.stdout.write(tokenizer.decode([nextToken]));
+}
+
+// 7. Cleanup
+executor.destroy();
+```
+
+### Key Methods
+
+| Method | Description |
+|--------|-------------|
+| `initGPU()` | Request WebGPU adapter and device |
+| `loadModel(options)` | Download model from HF Hub, generate IR, build tokenizer |
+| `new Executor(ctx, graph, opts)` | Create executor with KV cache allocation |
+| `executor.uploadWeights(weights)` | Upload safetensors data to GPU |
+| `executor.forward(inputIds)` | Run one forward pass, return logits |
+| `executor.reset()` | Clear KV cache for new conversation |
+| `executor.destroy()` | Free all GPU resources |
+| `sampleToken(logits, params, history?)` | Sample next token from logit distribution |
+
+---
+
+## 15. What's Built vs. What's Planned
+
+| Component | Status | Notes |
+|-----------|--------|-------|
+| IR type system | Built | OpType, TensorDesc, OpNode, ModelGraph |
+| Canonical key mapping | Built | Default HF key mapper (strip "model." prefix) |
+| Safetensors parser | Built | Zero-copy typed views, streaming support |
+| WebGPU device layer | Built | Buffer management, pipeline caching, readback |
+| Embedding kernel | Built | Simple parallel gather |
+| MatMul kernel (f32) | Built | 16x16 tiled with shared memory |
+| MatMulInt4 kernel | Built | Fused dequant+matmul, not yet wired to graph generators |
+| RMSNorm kernel | Built | Tree reduction, one workgroup per row |
+| LayerNorm kernel | Built | Two-pass mean+variance, tree reduction |
+| RoPE kernel | Built | Handles GQA (separate Q/K head counts) |
+| Attention kernel | Built | Tiled online-softmax, 256-thread parallel, 16 KB shared memory, two-phase score reduction (race-free) |
+| Softmax kernel | Built | Three-pass with tree reduction |
+| SiLU kernel | Built | Element-wise |
+| GELU kernel | Built | Approximate version |
+| Add kernel | Built | Element-wise |
+| Mul kernel | Built | Element-wise |
+| BPE Tokenizer | Built | Pure JS, HF tokenizer.json compatible |
+| CPU Sampler | Built | temp/top-k/top-p/repetition penalty |
+| Qwen2/3 graph generator | Built | Full layer structure with SwiGLU MLP |
+| KV Cache | Built | LHSd layout, pre-allocated |
+| Executor | Built | Single-encoder batching (Dawn) + grouped submits with one CB in flight (WebKit, `?group=N`) |
+| Activation liveness pooling | Built | 431 tensors → 20 buffers (37 MB at T=256), size-keyed free pool |
+| SliceLastRow + [1, vocab] logits | Built | Last-position lm_head only; saves 485 MB at T=512 and the full-vocab prefill matmul |
+| Model Loader | Built | HF Hub, multi-shard support |
+| Kernel Registry | Built | Mapping OpType -> kernel spec with dispatch/params helpers |
+| GEMV decode kernels | Built | K-parallel MatVec (f32 + INT4), 8 cols × 32 K-threads per workgroup |
+| SwiGLU MatVec fusion | Built | Fused gate+up+SwiGLU decode kernel, auto-detected by executor |
+| EmbeddingInt4 kernel | Built | INT4 dequant lookup, tied lm_head reuse |
+| INT4 quantization | Built | On-the-fly F32→INT4, GPTQ repack, MLX adapter |
+| WebKit/Safari compat | Built | UA-based `isWebKitWebGPU`, grouped submits, no select(), clamped exp(), 16 KB smem, packed-f16 KV, Cache-API write skip >64MB |
+| GPU error surfacing | Built | `onuncapturederror` + error scopes around alloc/upload/bind-group setup |
+| GPU diagnostics (A-P) | Built | 17-test suite: buffer integrity, compute, shared mem, f16, tree reduce, INT4, multi-dispatch, exp safety, real kernels, 300-dispatch stress |
+| Integrity checker | Built | Weight/logits checksum comparison between Dawn and Safari, auto-runs on Safari load |
+| Qwen3.5 graph gen | Built | Hybrid Mamba SSM + full attention with Gated Delta Net |
+| LLaMA/Mistral graph gen | Planned | Same base architecture, different key mappings |
+| Phi graph gen | Planned | Different MLP structure (fc1/fc2 vs gate/up/down) |
+| f16 KV cache | Built | Native f16 (Chrome) + packed u32 (Safari), auto-detected |
+| f16 compute path | Planned | Use shader-f16 for faster matmul |
+| Native text embeddings | Built | Qwen3-Embedding-0.6B (`Qwen3ForCausalLM`): last-token EOS pooling + `L2Norm` tail. dim 1024, unit norm, cos(similar)=0.81 > cos(unrelated)=0.56 (Section 21) |
+| L2Norm kernel | Built | Row-wise L2 normalization, one workgroup/row |
+| Vision encoder (ViT) | Built | Qwen3.5's own 12-layer ViT from the same checkpoint; bit-exact vs HF transformers 5.12 (per-token cosine 1.000000, max abs err ~5e-6). `engine.encodeImage(patches, gridTHW)` (Section 22) |
+| Vision ops (AddBias, GeluErf, ApplyRotaryEmb, SliceCols) | Built | ViT bias adds, exact-erf merger GELU, 2D rotary, fused-QKV split |
+| Bidirectional attention (`is_causal` flag) | Built | One-line uniform on the existing parallel attention kernel; text stays causal by default, ViT runs non-causal (Section 22) |
+| Vision LM integration (M-RoPE, token splice, image preprocessing) | In progress | Phase 2 — the encoder is done; splicing image tokens into the text stream is the remaining work |
+| MoE support | Planned | MoERouter, ExpertMatMul kernels |
+| Streaming API | Planned | High-level `stream()` method with async iteration |
+| Jinja2 template parser | Planned | Full chat template support beyond ChatML |
+
+### 15.1 Exploration backlog (open directions, June 2026)
+
+Candidates under evaluation — not commitments. The first item gates several others.
+
+| Direction | Source / rationale | Status |
+|-----------|--------------------|--------|
+| **Engine consolidation decision** | Two engines (native WGSL text-only; transformers.js/ONNX breadth) is untenable long-term. Only transformers.js covers all modalities (vision/TTS/STT/embeddings). The decisive test: does transformers.js run on iOS 26.5 (post our environmental fixes + ORT 1.25 + IIFE worker)? If yes → consolidate on it; native WGSL kept only if a head-to-head shows a decisive text-speed win. | **Decision gate — test pending** |
+| **KVSwap: KV-cache memory tiering** | son-of-ole/infinite-edge-agent (MIT). KV cache as VRAM→RAM→IndexedDB hierarchy with pinning/eviction/prefetch. Lifts the mobile context cap (the next wall after activation pooling). Applies to any WebGPU text path. | Explore — high value |
+| **SSA: sparse/sub-quadratic attention** | Same repo. Block-summary → top-K block routing → gather → sparse attention. Cheaper long context, but changes attention semantics (approximate) — a quality/throughput research bet, not drop-in. | Explore — research |
+| **Evaluate WebLLM/MLC as the text backend** | Mature compiled-WebGPU runtime (TVM-based); the most-used browser LLM engine. Text-only (not multimodal). Relevant only if we decide a separate fast text path is worth maintaining — as a more-supported alternative to the native WGSL engine. | Evaluate |
+| **Gemma 4 E2B support** | Smallest current on-device Gemma (~3.35 GB q4). Tier 2: needs sliding-window attention + GeGLU + logit soft-cap (`docs/research/qwen36-gemma4-targets.md`). | Planned target |
+| **Qwen3-0.6B (dense)** | Tier-1 proof of the `add-model-family` process; ~0.32 GB q4. | Planned target |
+| **Agent-runtime feature layer** | Persistent local memory + context reconstruction (cf. same repo's Memory OS). A *feature set* packaged AI-SDK-style, NOT an engine concern. | Explore — product layer |
+
+---
+
+## 16. Key Design Decisions
+
+### Why Runtime IR (Not Offline Converter)
+
+An offline converter (like ONNX export) adds a mandatory pre-processing step. Users must find or create converted model files, keep them in sync with upstream model updates, and deal with conversion bugs. By generating the IR at runtime from `config.json`, gerbil can point at any HuggingFace repo that uses a supported architecture and just run it. The cost is a few milliseconds of graph generation at load time -- negligible compared to weight download.
+
+### Why f32 First (Not f16)
+
+f16 compute requires the `shader-f16` WebGPU feature, which is not universally available. Starting with f32 ensures correctness across all WebGPU-capable browsers. f16 is being added incrementally: the KV cache uses f16 storage with f32 compute (halving attention memory traffic), and f16 matmul will follow. Feature detection ensures automatic fallback to f32 on devices without `shader-f16`.
+
+### Why CPU Sampling
+
+The logits array for a typical vocabulary (~150K tokens) is ~600KB as f32. Copying this to CPU and sampling there takes microseconds. GPU-side sampling would require:
+1. An additional kernel for argmax/multinomial sampling
+2. A readback of a single u32 (the selected token ID)
+3. Synchronization to get the result
+
+The overhead of kernel dispatch + sync would exceed the CPU sampling time. The CPU also has access to `Math.random()` and can implement complex sampling strategies (repetition penalty over arbitrary history) more naturally.
+
+### Why No WASM Fallback
+
+The previous approach maintained three inference paths (WebGPU, WASM, CPU), each with different bugs and performance profiles. The GPU engine targets WebGPU exclusively. If WebGPU is unavailable, the engine throws a clear error rather than silently degrading to a 10x slower path. This keeps the codebase focused and testable.
+
+Browser WebGPU coverage (Chrome 113+, Safari 18+, Firefox 141+) is sufficient for the target audience. The old transformers.js path remains available for legacy browser support if needed.
+
+### Why Canonical Tensor Naming
+
+Different model families use different key prefixes in their safetensors files (`model.layers.0.self_attn.q_proj.weight` vs `transformer.h.0.attn.c_attn.weight`). By mapping to canonical names, the graph generators and executor code don't need to know about HF-specific naming. Adding a new model family only requires writing a key mapper function, not modifying the core engine.
+
+### Why LHSd KV Layout
+
+LHSd (Layer, Head, Sequence, head_dim) puts the sequence dimension third, which means appending a new token's K/V vectors is a simple write at offset `seqPos * head_dim`. The attention kernel reads contiguous head_dim-sized chunks, which is cache-friendly for the dot product computation. This layout matches what most transformer implementations use internally.
+
+### Why Detect the Implementation, Not the Hardware
+
+Workarounds in this engine exist for *WebGPU implementations* (WebKit's), not for GPUs. Apple Silicon is the hardware under both Safari and desktop Chrome/node-dawn, and `adapter.info` reports `vendor: "apple"` in all three — so any hardware-keyed predicate inevitably drags a correct, fast implementation through workarounds built for a broken one (Section 17.4 documents the 161 → 19.7 tok/s cost of getting this wrong). The user agent identifies the implementation unambiguously: all iOS/iPadOS browsers are WebKit by platform mandate, macOS Safari is `AppleWebKit` without `Chrome/`, and node has no UA at all.
+
+### Why Liveness Pooling Instead of Per-Tensor Buffers
+
+Per-tensor allocation at `maxSeqLen` is simple and was fine on a 128 GB desktop, but it scales with *graph size*, not with *concurrent liveness* — and a 431-tensor graph at T=512 wants ~2.3 GB while only ~20 tensors are ever alive at once. The pooled allocator keeps every desirable property of pre-allocation (no allocation during generation, deterministic footprint, buffers reused across forwards) at ~1.6% of the memory. The one obligation it creates: fused kernels must never read and write the same pooled buffer in a single dispatch, and mid-graph debug readbacks are only meaningful before a later op reuses the buffer.
+
+---
+
+## 17. The Four-Failure-Mode Diagnosis (June 2026)
+
+Through early 2026 the engine's mobile story was a single undifferentiated narrative: "it crashes or produces garbage on iPad, probably Metal coherence." On 2026-06-12 a multi-agent investigation (full report: `docs/mobile-failure-diagnosis.md`; running log: `docs/metal-safari-intel.md`) took the four observed symptoms apart and found **four independent root causes** — two of them local bugs in this codebase, one a genuine (and since-fixed) WebKit bug, and one a detection bug that had silently poisoned every desktop benchmark for months. Conflating them is precisely what made the problem look intractable.
+
+### 17.1 jetsam-crash: a local memory bug, not Metal
+
+The iPad tab kills ("download reaches 100%, then blank page") were attributed to dispatch-strategy overhead — the engineering log explicitly blamed "400 roundtrips + Promise overhead" for the DRAIN_EVERY=1 OOM. That explanation was wrong: a fully drained loop bounds in-flight command buffers to ~1, so roundtrip count cannot dominate memory. The actual cause was the executor's buffer allocator.
+
+`allocateActivationBuffers()` created **one dedicated buffer per activation tensor at full `maxSeqLen`**, with zero reuse. For Qwen3.5-0.8B (431 activation tensors, vocab 248,320) at `maxSeqLen=512` the math is unforgiving:
+
+| Component | Bytes |
+|---|---|
+| Logits buffer alone: `[T, vocab]` f32 = 512 × 248,320 × 4 | **508.6 MB (≈485 MiB)** — of which only the last row (993 KB) was ever read |
+| All ~430 per-tensor activation buffers | **~2.30 GB** |
+| INT4 weights | ~0.44 GB |
+| **Total GPU footprint, INT4 @ T=512** | **~2.77 GB** |
+
+Against an iOS web-content jetsam budget of ~1.5-2 GB, **every configuration ever run on the iPad was memory-doomed before the first dispatch executed** — INT4 @ T=256 was 1.61 GB, the engine's then-default `maxSeqLen=4096` requested 18.9 GB, and the diagnostic page's default URL silently loaded the BF16 repo (6.08 GB F32 graph) under an "INT4" label. The drained per-dispatch experiments "failed" for memory reasons, not because draining was wrong — which sent the investigation chasing submit strategies when the bug was in allocation.
+
+Fixes: liveness pooling (18.1), `SliceLastRow` (18.2), iOS `maxSeqLen` clamping, diagnostic-page default model fix, and load-transient trims. Post-fix footprint at T=512: ~0.44 GB weights + ~0.04 GB pooled activations + ~0.03 GB KV/SSM state ≈ **0.6-0.7 GB — comfortably inside the budget for the first time in the project's history**.
+
+### 17.2 zero-logits: a real WebKit bug — and an OS-version-dependent one
+
+The historical core mystery: a full forward pass (~440 dispatches in one command buffer) produced all-zero logits from dispatch entry 2 onward on iPad Safari, while every per-dispatch-submit probe was correct and the identical code was correct on Dawn. Three things are now established:
+
+1. **It was not a local bug.** Stale-params and silent-allocation-failure theories were refuted: CPU-side state is byte-identical between the batched and per-dispatch paths (only submit granularity differs, and that alone flipped correct↔zeros); the kernels' `col < params.N` write guards mean zeroed params would write *nothing*, yet probes showed zeros actively **overwriting pre-seeded nonzero data** — a read-side visibility failure, with entry 2 computing real output from a stale (zero) view of entry 1's freshly written input.
+
+2. **It was a genuine WebKit within-submission storage-visibility bug, threshold-dependent.** The WebGPU spec guarantees cross-dispatch storage visibility within a submission (gpuweb #4433/#4434). Every passing diagnostic was tiny (Test J: 2 pipelines, 16-byte buffers; Test P: 300 dispatches but one pipeline); production combined ~441 dispatches, 15+ distinct pipelines, 6-9 bindings, and MB-scale buffers in one submission — a scale no diagnostic ever reached.
+
+3. **KEY FINDING (2026-06-12): the bug does not reproduce on iPadOS 26.5.** The exact historical zero-logits configuration — the entire ~440-dispatch forward in a single command buffer — now produces **byte-identical output to the desktop Dawn reference** on the test iPad (WebKit WebGPU via Chrome-iOS WKWebView over HTTPS). The bug is OS-version-dependent and was fixed upstream in WebKit (cf. WebKit bug 311598, where llama.cpp reports ~64 passes/CB stable on iOS 26.4). The grouped-submit machinery (18.3) is therefore retained as a *compatibility dial for older WebKit*, not as the permanent execution model.
+
+### 17.3 gibberish: zeros in disguise, plus a real data race waiting underneath
+
+The recorded gibberish event (plausible-but-wrong tokens at ~276 tok/s on iPad, 2026-03) had been attributed to an "unknown Safari WGSL→MSL compiler bug." Two separate findings replace that:
+
+**(a) The recorded event was zero-logits in disguise.** ~276 tok/s — 2-4× faster than an M4 Max — is the signature of zero-work dispatches: all threads early-exit, argmax over an all-zero logits buffer returns token 0 or whatever degenerate pattern the sampler makes of it, and "generation" races ahead. The miscompilation theory was additionally contradicted by Tests N/O (the real RMSNorm and MatVec kernels pass in isolation on the same iPad) and by the fact that the packed-f16 kernels only ever saw already-zero inputs in failing runs.
+
+**(b) A genuine WGSL data race existed in all three attention variants.** In the Q·K score reduction, after the publish barrier, each dot-group leader read `smem[tid..tid+15]` and wrote `smem[pos_in_tile]` (slots 0..15) in the same block — leader 0's read range is exactly the write target of leaders 1..15, with no barrier between cross-group reads and writes. This is spec-level UB. It happened to be benign on Tint/M-series scheduling, which is why desktop never showed it, but WebKit's independent WGSL→MSL compiler and A-series scheduling carry no such guarantee; it would corrupt the first key position of each 16-position tile — exactly the "plausible but wrong tokens" phenotype. Fixed unconditionally with the two-phase reduction (18.4) since the cost is one barrier per KV tile.
+
+### 17.4 desktop-regression: conflating Apple-GPU hardware with the WebKit implementation
+
+The Safari workaround gate was `isMetalBackend = vendor === "apple" || arch.startsWith("common")` — a *hardware* test. Dawn running on the M4 Max reports `vendor: "apple", arch: "metal-3"`, so node-dawn and Chrome-on-macOS took the full WebKit workaround path: per-dispatch multi-encoder submits plus the (already-disproven, yet still active) shader variant alternation that compiled ~850-900 per-node pipelines instead of ~25. Result: the recorded **161.8 → 19.7 tok/s collapse** on desktop, and every desktop performance number recorded between commit `2f0cabc` and the fix is poisoned. The fix is the UA-based `isWebKitWebGPU` predicate (18.5); post-fix desktop baseline is 145 tok/s.
+
+### 17.5 Corrections to earlier claims in this document
+
+The June 2026 findings falsify several statements this paper (and the engineering log) previously made. For the record:
+
+- **"Metal provides no cross-dispatch coherence by design"** — wrong. The WWDC25 quote about command-buffer boundaries explains why CB boundaries are *expensive*, not why they are required for coherence. The WebGPU spec mandates within-submission visibility; Dawn-on-Metal delivers it; WebKit on iPadOS 26.5 delivers it. The older-WebKit zeros were a bug, not a design property to architect around.
+- **Shader variant alternation** (commit `2f0cabc`) — never necessary. Disproven by the project's own Test Q, yet it remained active and was a major contributor to the desktop regression. Deleted; do not reintroduce.
+- **Per-dispatch submit without await (fire-and-forget**, commit `38bc674`**)** — an anti-pattern, not a fix. It queues ~400 unbounded in-flight Metal command buffers per token, the documented WebKit resource-exhaustion pattern, and never produced a recorded on-device result. Replaced by grouped submits with exactly one CB in flight (18.3).
+- **"DRAIN_EVERY=1/5 OOMs prove roundtrips exhaust memory"** — wrong attribution; the buffer footprint (17.1) was the cause. Drained loops are memory-bounded by construction.
+- **The two-phase writeBuffer pattern as the gibberish fix** — the pattern is harmless and retained, but stale params were refuted as the zero-logits mechanism (17.2); the recorded gibberish was zeros in disguise (17.3a).
+- **Validity boundary**: the single-CB zeros behavior was real on the older iPadOS version where it was recorded and presumably remains real on pre-26.5 WebKit; statements about it are *historical, version-scoped facts*, not current behavior.
+
+---
+
+## 18. The Mobile Fix Campaign
+
+All fixes landed 2026-06-12. Each is small; together they took the iPad from "never completed a recorded run" to byte-correct 35.9 tok/s.
+
+### 18.1 Liveness-based activation pooling
+
+`executor.ts allocateActivationBuffers()` replaces per-tensor allocation with a classic last-use liveness scheme over the topologically ordered graph:
+
+1. **Liveness pass**: walk `graph.executionOrder` once, recording each activation tensor's first defining node and last reading node. Tensors read before they are ever written carry state across forward passes (or are written in place) — they join a `persistent` set along with all `graph.outputs`, and are never pooled.
+2. **Allocation pass**: walk the order again with a **size-keyed free pool** (`Map<byteSize, GPUBuffer[]>`). For each node, *acquire outputs first* (popping an exact-size buffer from the pool or creating one) so a node never writes the buffer of a tensor it also reads; then release every input/output whose last use is this node back into the pool.
+3. Anything persistent or untouched by the execution order gets a dedicated buffer.
+
+Reuse is safe because dispatches execute in `executionOrder` on every path (single-pass Dawn, grouped WebKit) and WebGPU synchronizes write-after-read hazards between dispatches. The executor logs the result: `431 tensors → 20 buffers (37 MB)` at T=256. Two standing obligations follow (also listed in Section 7's mobile invariants): fused kernels must not read+write one pooled buffer in a single dispatch, and `debugReadBuffer()` on an intermediate tensor is only meaningful before a later op reuses it.
+
+### 18.2 SliceLastRow and the [1, vocab] logits buffer
+
+Only the last position's logits are ever consumed (the sampler operates on one row). Previously both graph generators emitted `lm_head` over the full `[T, hidden]` normed activations into a `[T, vocab]` logits tensor — 508.6 MB at T=512 (512 × 248,320 × 4) of which 993 KB was read. Now a `SliceLastRow` op (`kernels/registry.ts`, a trivial 256-wide copy kernel with `{width, last_row_offset}` params) extracts `final_norm_out`'s last row into `final_norm_last [1, hidden]`, and `lm_head` runs with M=1 into `logits [1, vocab]`. This saves the 485 MiB buffer **and** removes the full-vocab matmul over all prefill rows — the dominant prefill compute — so it is a pure win on desktop too. Readback offsets collapse to 0 in `forward()`/`forwardArgmax()`.
+
+### 18.3 Grouped-submit architecture (WebKit)
+
+The WebKit execution path in `forward()`/`forwardArgmax()` is now a parameterized grouped loop: `webkitGroupSize` dispatches per command buffer (one compute pass per dispatch), `queue.submit()` then `await queue.onSubmittedWorkDone()` per group — **exactly one command buffer in flight at all times**. The logits copy (and in `forwardArgmax()`, the argmax dispatch and its 4-byte readback copy) go in separate final submits. This replaces the fire-and-forget anti-pattern of ~400 unbounded in-flight CBs per token. `group=1` is the proven-correct floor for older WebKit; the size is sweepable at runtime via the `?group=N` URL parameter, which produced the scaling curve in Section 19. On Dawn nothing changed: one encoder, one compute pass, one submit.
+
+### 18.4 Two-phase attention score reduction
+
+In all three attention kernel variants (f32, native-f16, packed-f16), the leader-thread score reduction now accumulates into a local variable, executes a `workgroupBarrier()` in **uniform control flow** (the barrier cannot live inside the guarded block — non-uniform barriers are a WGSL validation error), and only then writes `smem[pos_in_tile]` in a second guarded block. Cost: one extra barrier per 16-position KV tile. This closes the race described in 17.3(b).
+
+### 18.5 `isWebKitWebGPU` detection
+
+Described in Section 6. The hardware-keyed `isMetalBackend` predicate is gone; the executor's `needsMultiEncoder` is set solely from the UA-based `isWebKitWebGPU`, and shader variant alternation is deleted outright (~850-900 pipelines → ~25 on the workaround path's worst case).
+
+### 18.6 Error scopes and uncaptured-error surfacing
+
+`device.onuncapturederror` is registered at init, and `WebGPUEngine.create()` brackets executor construction, weight upload, and bind-group creation with `pushErrorScope("out-of-memory")` + `pushErrorScope("validation")`, throwing a descriptive error (with a crash-phase breadcrumb in `localStorage`) if either pops. WebKit reports failed allocations asynchronously and otherwise *silently no-ops the affected dispatches* — before this change, every mobile observation was collected blind. Device limits (`maxBufferSize`, `maxStorageBufferBindingSize`, `maxComputeWorkgroupStorageSize`) are logged on WebKit at startup.
+
+### 18.7 WebKit Cache-API large-write skip and memory-policy fixes
+
+- `model-loader.ts browserCacheWrite()`: the defensive `data.slice(0)` needed for `cache.put()` doubles a weight shard in memory at the worst moment; on WebKit, writes over 64 MB are skipped entirely (the HTTP cache still serves warm reloads).
+- iOS `maxSeqLen` policy (`gpu/index.ts`): WebKit defaults to 512 and is hard-clamped to 2048 (the old default of 4096 was an 18.9 GB request); `?kvf32=1` on Safari caps at 1024; desktop caps at 4096. `?maxseq=N` overrides within `context_length`.
+- The iPad diagnostic page's default model is now the MLX 4-bit repo, never the BF16 one.
+
+---
+
+## 19. Mobile Results & Submit-Granularity Scaling
+
+All measurements 2026-06-12 on iPad, iPadOS 26.5, WebKit WebGPU via Chrome-iOS WKWebView over HTTPS. Model: Qwen3.5-0.8B MLX 4-bit, greedy (temperature 0). **Every configuration below produced output byte-identical to the desktop Dawn reference** — these are correctness results first, throughput results second. (Canonical numbers: `docs/research/tps-baselines.md`.)
+
+Device limits (default adapter): `maxBufferSize` 256 MB, `maxStorageBufferBindingSize` 128 MB, `maxComputeWorkgroupStorageSize` 16384 bytes.
+
+### Throughput vs. group size (24-token generations)
+
+| `?group=N` (dispatches/CB, one CB in flight, awaited) | Decode tok/s |
+|---|---|
+| 1 | 6.4-7.7 |
+| 8 | 19.7 |
+| 32 | 24.6 |
+| 64 | 28.8 |
+| 128 | 29.7 |
+| batch-all (entire forward in one CB) | **31.7** |
+| sustained 120-token run, group=64, maxseq=512 | **35.9** |
+
+**Interpretation**: the curve is round-trip-bound until ~32 dispatches/CB — each group costs a full submit + `onSubmittedWorkDone()` JS↔GPU round trip, so throughput scales nearly linearly with group size at first (1→8 is a 3× jump). Past 32-64 the round trips amortize away and the engine becomes **dispatch-bound**: 64→all is only 28.8→31.7, because ~440 individual compute passes per token dominate regardless of CB packaging. This is why dispatch-count reduction (kernel fusion) is worth 2-3× more on mobile than on desktop, and why the iPad goal path (50+ tok/s) runs through a single compute pass per CB plus fusion rather than through submit tuning.
+
+Other results from the same session:
+
+- **maxseq survival**: at group=1, both `maxseq=64` and `maxseq=512` complete — pre-fix, every configuration was jetsam-killed (17.1). The memory fix and the correctness fix are independently confirmed.
+- **KV A/B**: packed-f16 KV vs `?kvf32=1` both byte-correct — the packed-f16 path is cleared on-device (it had never run against correct inputs before; see 17.3a).
+- **Within-submission visibility**: the historical zero-logits configuration (batch-all) is correct on iPadOS 26.5 — the decisive evidence for 17.2's key finding.
+- **Desktop post-fix baseline**: node-dawn on M4 Max at **145 tok/s** (143.9 / 144.1 / 147.1), confirming the 19.7 tok/s regression was entirely the detection predicate.
+
+### Production submit strategy
+
+The strategy is OS-gated, chosen at engine init:
+
+- **iPadOS/iOS 26.5+**: batch-all (single CB per forward) — the upstream WebKit fix makes the fast path correct.
+- **Older 26.x WebKit**: run a **startup coherence probe** — a small batched dependent-chain dispatch whose result is compared against the same chain run per-dispatch. Probe passes → `group=64`; probe fails → `group=1` awaited (the proven-correct floor, 6.4-7.7 tok/s).
+- **Dawn (Chrome, node)**: single encoder, single compute pass, unchanged.
+
+Until the probe ships, `webkitGroupSize` defaults to 1 (correctness floor) and `?group=N` overrides it.
+
+### Update (2026-06-15): a third device class — batching *crashes* (iPhone)
+
+A `?group=N` sweep on an **iPhone (iOS 18.7, WebKit/Safari 26.5)** revealed a failure mode distinct from both the iPad-26.5 class (batch-all correct) and the older-WebKit class (batch-all → zero logits): **any `group > 1` hard-crashes the GPU process** — the page dies ("A problem repeatedly occurred"), confirmed at `group=4`, `32`, and `4096`. `group=1` runs correctly at ~**4.7 tok/s** (the round-trip-bound floor predicted by the curve above). This is the single most important mobile fact: on this device the submit-granularity lever — the whole basis of the 6.4→31.7 tok/s scaling on iPad — is **unavailable**, so the *only* remaining levers are dispatch-count reduction (fusion) and memory reduction.
+
+Both signs point at **memory**, not pure miscompilation:
+
+- The crash is at the **first grouped compute (prefill)**, immediately after load, and scales with group size — consistent with the command buffers / intermediate state held in flight before the single `onSubmittedWorkDone()` drain exceeding the iPhone's (smaller-than-iPad) jetsam budget.
+- The same device **hard-crashes at model *load*** with the ~596 MB vision-enabled Qwen checkpoint but loads the ~404 MB text-only one. It is operating right at its memory ceiling, so the extra working set a batched submit needs is enough to push it over. (Action taken: the site's default chat engine now loads the text-only checkpoint; the vision tower is built only on the Vision tab.)
+
+**Revised production strategy** — the startup probe must be *crash-surviving*, not merely output-comparing (the original design in the previous subsection cannot detect a class that kills the page before returning a result):
+
+1. Persist the candidate group size to `localStorage` **before** running the probe (reuse the existing crash breadcrumb in `device-guards.ts` / `setDownloadPhase`).
+2. On reload, if the breadcrumb for a group size is still set, that size crashed → step down (batch-all → 64 → 8 → 1) and record the safe ceiling for the device.
+3. Promote a group size to "known-good" only after a probe both completes **and** returns byte-correct output.
+
+This self-calibrates across all three classes (correct-batched / wrong-batched / crash-batched) with no hardcoded device table, and survives the crash class because the breadcrumb outlives the page kill. **This — shipping the crash-surviving probe — is the decided path; the rabbit hole was hand-sweeping `?group` on a device that crashes the page on every batched value, which a self-calibrating probe is specifically designed to avoid.**
+
+**Caching note (same session):** on iOS Safari in a plain tab the CacheStorage model cache is **evicted between visits** (`navigator.storage.persist()` is not granted outside an installed PWA — `model-loader.ts:requestPersistence` is best-effort), so every visit re-downloads the full model regardless of throughput. The durable fix is OPFS via a Worker (`createSyncAccessHandle` in chunks), tracked as a separate task; "add to Home Screen" (PWA) also earns persistent storage as an interim path.
+
+---
+
+## 20. Autoresearch: Profile-Guided Throughput Optimization (June 2026)
+
+Once mobile correctness was secured, an autonomous optimization loop
+(`scripts/engine/optimize.mjs --mode=autoresearch`, driven by the model itself)
+took **desktop decode throughput from 145 → 207 tok/s** (M4 Max, node-dawn,
+Qwen3.5-0.8B INT4), cooled-stable, with every round verified byte-coherent. The
+loop's contract: read engine code → make one focused edit → build → benchmark
+twice (lower mean, to defeat thermal lies) → keep only if >1.5–2% over the
+current best, else `git checkout` the touched files → log to `results.jsonl`.
+
+### The arc — and the change of angle
+
+The interesting result is not the number but the *shape* of how it was reached:
+
+| Round | tok/s | Kept | Edit |
+|---|---|---|---|
+| baseline | 145 | — | — |
+| r1 | 145.9 | ✗ | fuse down_proj + residual |
+| r2 | 152.8 | ✓ | vec4 INT4 weight loads (32 nibbles/iter) |
+| r3 | 158 | ✓ | vec4 loads in the SwiGLU matvec |
+| r4 | 158 | ✗ | subgroup-shuffle reduction |
+| **r5** | **188** | ✓ | **GPU-driven pipelined decode** |
+| **r6** | **213** | ✓ | **fuse MambaSSM (profiled #2 hotspot)** |
+| r7 | 210 | ✗ | SSM occupancy split |
+| **r8** | **222** | ✓ | **f16 SSM-state storage** |
+
+Rounds 1–4 are a *sideways* plateau: local kernel-load micro-optimizations
+(145→158), half reverted. The breakthroughs (r5, r6, r8) came when the loop
+**stopped optimizing the kernel in front of it and changed its analysis**:
+
+- **r5** is not a kernel tweak but a *control-flow* restructuring — keep the
+  argmax result on the GPU and feed it back as the next token's input without a
+  per-token CPU round-trip (158→188).
+- **r6/r8** are *profile-guided*: the loop identified the Mamba-2 SSM as the #2
+  hotspot (≈1982 µs/token, 37% of decode), then attacked it structurally —
+  fusing four per-head passes (188→213), then halving its dominant memory
+  traffic by storing SSM state in f16 while keeping f32 compute (213→222).
+
+This transition — from "make this matmul faster" to "profile the whole forward,
+find the real bottleneck, and rewrite its data flow" — is the qualitative jump
+that broke the plateau. It is also the clearest signal that the search was
+reasoning about angles it had not tried before, not enumerating variations of
+the same one.
+
+### A safety lesson: reverts can clobber un-tracked work
+
+One round's failed-experiment revert ran `git checkout -- executor.ts
+registry.ts`, which silently reset *uncommitted* mobile-fix edits in those files
+to the last commit, then re-layered only the optimizer's own changes — producing
+a tree that built (esbuild strips types) but failed `tsc` and would have lost the
+mobile work. The incident motivates two rules now documented for the loop: revert
+only the files a round actually touched, and commit known-good baselines before
+launching autonomous edit sessions. The kept kernel wins were recovered intact
+from the loop's own per-round `.bak` snapshots.
+
+### Open throughput frontiers
+
+The decode path is now bandwidth- and dispatch-bound rather than matmul-bound.
+Remaining candidates (tracked in `scripts/engine/backlog.md`): fused KV-append,
+RMSNorm/RoPE fusion, further dispatch-count reduction (which helps mobile 2–3×
+more than desktop, since mobile is round-trip-bound below ~32 dispatches/CB), and
+investigating reported large wins from fusing the linear-attention/SSM layers
+end-to-end.
+
+#### The two-regime model (recalibrated, June 2026)
+
+A profiling pass (env-gated timestamp-query profiler, `scripts/engine/test-profile-decode.mjs`)
+plus a 5-agent research synthesis (`docs/research/dispatch-reduction-hivemind.md`)
+resolved an apparent contradiction in this campaign. Decode has **two distinct
+overhead regimes**:
+
+- **Mobile (Safari/WebKit):** each dispatch is its own command-buffer submit +
+  `onSubmittedWorkDone()` drain, so per-dispatch cost is ~32–71 µs (Metal; arXiv
+  2604.02344). At ~287 dispatches/token this regime **is** dispatch-count-bound —
+  any dispatch cut helps, and the lever is fewer dispatches.
+- **Desktop (node-Dawn, the primary `test-benchmark.mjs` metric):** the whole
+  decode step is **one** `beginComputePass` / **one** submit; measured per-dispatch
+  overhead is ~2–3 µs. Decode is **not** dispatch-count-bound here. The metric
+  moves only on **bytes-moved reductions over WIDE tensors.**
+
+This is load-bearing and confirmed by the engine's own history: pure dispatch-count
+cuts were flat/reverted on Dawn (ResidualRMSNorm Add+Norm −23 dispatches = +0.25%;
+RMSNorm-into-MambaSSM −18 dispatches = +0.3%; software-prefetch −1.2%), while every
+kept Dawn win removed a **wide** global round-trip (MambaSSM single-pass state
+fusion ~+13%, f16 SSM state +4%, SiLU-into-conv +1.8%, matvec A-reuse +4.9%). True
+WebGPU megakernels are **not** viable (no portable inter-workgroup sync / forward
+progress); the achievable analog is aggressive per-block fusion. The ranked plan
+(epilogue/prologue Mamba megakernels that erase 2048/6144-wide round-trips) lives
+in `docs/research/dispatch-reduction-hivemind.md`.
+
+**Measurement rule:** dispatch-count cuts must be validated on the **iPad runner**
+(mobile regime), not the desktop benchmark, or they read as "flat" and get reverted
+despite being real mobile wins.
+
+---
+
+## 21. Native Text Embeddings (June 2026)
+
+The second modality to go native after text generation is **text embeddings**.
+The target model is **Qwen3-Embedding-0.6B**, and the key fact that makes it cheap
+is that it ships as architecture `Qwen3ForCausalLM` — the *same* graph generator
+the engine already runs for text. An embedding model is not a separate engine; it
+is the existing causal-LM forward pass with a different tail.
+
+### What changes vs. text generation
+
+The graph generator (`architectures/qwen2.ts`) takes an `embedding` flag. When
+set, it replaces the `SliceLastRow → lm_head → logits` tail with a two-op
+**pooling tail**:
+
+1. **`SliceLastRow`** on `final_norm_out` — Qwen3-Embedding uses **last-token
+   (EOS-position) pooling**: the pooled vector is the final hidden state at the
+   last input position. This reuses the exact op the LM path already emits to
+   restrict lm_head to the last row, so no new pooling kernel is needed.
+2. **`L2Norm`** — a new row-wise L2-normalization kernel (`WGSL_L2NORM`,
+   `kernels/registry.ts`: one workgroup per row, `{rows, width}` params) that
+   divides the pooled vector by its Euclidean norm, producing a unit-length
+   embedding in tensor `embedding`.
+
+The lm_head matmul (and the entire ~248K-row vocab projection) is dropped
+entirely — an embedding forward pass is strictly cheaper than a generation step.
+`graph.outputs` becomes `["embedding"]`, and `WebGPUEngine` exposes the modality
+through `engine.embed(text, options)` (with `engine.isEmbedding` as the guard).
+
+### The pooling-token subtlety
+
+Last-token pooling reads the final position, so *which* token sits there matters.
+Qwen3-Embedding was trained to pool at the literal `<|endoftext|>` token
+(`eos_token_id`), **not** the chat `<|im_end|>` token the tokenizer reports as its
+generic `eos_token`. `engine.embed()` (`gpu/index.ts`) resolves the literal
+`<|endoftext|>` id and appends it if absent, truncating to the budget while
+keeping that pooling token last. Query embeddings additionally take the
+Qwen3-Embedding instruction prefix (`"Instruct: {task}\nQuery:{text}"`); document
+embeddings omit it.
+
+### Validation
+
+The native embeddings were validated against the reference: output **dim 1024**,
+**unit L2 norm** confirmed, and the semantic geometry is correct —
+**cos(similar) = 0.81 > cos(unrelated) = 0.56**. The path is non-autoregressive
+(prefill only), so it inherits the existing memory and submit machinery with no
+mobile-specific work.
+
+---
+
+## 22. Native Vision Encoder (Qwen3.5 ViT) (June 2026)
+
+Qwen3.5 is **natively multimodal**: the same checkpoint that the engine already
+runs for text ships a **12-layer Vision Transformer** in its weights — a ~192 MB
+tower the engine had previously been dropping on load. This cycle implements that
+ViT natively and validates it **bit-exact against HuggingFace transformers 5.12**.
+
+### Why this was tractable
+
+The vision feasibility was re-assessed (Section 5 / `docs/PROJECT-STATE.md`) and
+found materially easier than an earlier strategy pass had concluded. That pass
+believed native vision was blocked behind a *single-threaded attention kernel
+rewrite* — but it had read `attention.wgsl`, a **dead reference file imported
+nowhere** (the live kernels are embedded strings in `registry.ts`). The live
+attention kernel was already the tiled, online-softmax, fully parallel
+flash-attention-style kernel of Section 7. Making it bidirectional for the ViT was
+**one uniform flag**, not a rewrite (below). With that correction, the encoder was
+a bounded amount of ordinary graph-generator work.
+
+### Architecture
+
+The ViT generator (`architectures/qwen3_5_vision.ts`) produces a non-autoregressive
+graph over a symbolic patch count `N`, validated line-for-line against
+`modeling_qwen3_5.py`:
+
+- **Unfold-free patch embed.** Patches arrive **already flattened** to `[N, 1536]`
+  (`patch_dim = in_channels · temporal_patch_size · patch_size² = 3·2·16²`), so the
+  5-D Conv3d patch embedding collapses to a **plain `MatMul`** + `AddBias`. No
+  `Conv3d`/`unfold` kernel is needed — the host image processor delivers patches in
+  the right layout. A bilinear-interpolated learned position embedding is then
+  added.
+- **12 pre-norm, bidirectional blocks.** Each block is `LayerNorm → fused-QKV
+  MatMul+AddBias → SliceCols (split Q/K/V) → ApplyRotaryEmb(Q,K) → Attention
+  (bidirectional) → proj MatMul+AddBias → residual → LayerNorm → fc1 MatMul+AddBias
+  → GELU(tanh) → fc2 MatMul+AddBias → residual`.
+- **Bidirectional attention via one flag.** The attention kernel computes
+  `causal_limit = q_pos + position_offset + 1; S_eff = is_causal ? min(S, causal_limit) : S`.
+  An `is_causal` uniform (default `1`) keeps text decoding causal and unchanged; the
+  ViT passes `causal: false`, setting `is_causal = 0` so every patch attends to all
+  patches. This is the entire non-causal change.
+- **2D rotary.** The ViT uses a 2D (row, col) rotary embedding. The cos/sin tables
+  and the bilinear-interpolated position embeddings are functions of the image grid
+  `(t, h, w)` **only** — not of weights or pixels — so they are precomputed on the
+  host (`vision-preprocess.ts`, porting `get_vision_bilinear_indices_and_weights`,
+  `get_vision_position_ids`, and `Qwen3_5VisionRotaryEmbedding`) and fed in as input
+  activations. `ApplyRotaryEmb` then applies them with a `rotate_half`. This keeps
+  the GPU graph to the weight-dependent math while staying byte-identical to HF.
+- **Two distinct GELUs.** The transformer blocks use `gelu_pytorch_tanh` (the
+  `GELU` op); the merger uses the **exact erf** GELU (the new `GeluErf` op). Getting
+  both exactly right was load-bearing for the bit-exact match.
+- **Spatial-merge-2 → 1024-dim tokens.** The merger's `LayerNorm` over the 768-dim
+  block output is followed by a free row-major reshape `[N, 768] → [N/4, 3072]`
+  (a spatial 2×2 merge — patches arrive pre-grouped into merge-blocks so no gather
+  is needed), then `fc1 → GeluErf → fc2` to produce merged image tokens of dim
+  **1024**, matching the LM hidden size, as `[N/4, 1024]`.
+
+The encoder runs in its own `VisionExecutor` (`vision-executor.ts`) — kept
+separate from the autoregressive text `Executor` because it is a single prefill
+over `N` patches with no KV cache, one buffer per tensor, dispatched in
+`executionOrder`. It reuses the shared kernel registry and device helpers, so the
+kernel math is identical to the text path and inherits the same WebKit
+grouped-submit behavior.
+
+### A real bug the validation caught
+
+Achieving the bit-exact match surfaced a genuine kernel bug: **`WGSL_GELU`
+returned NaN for large arguments on Metal/Dawn**. The MLP can produce `|x| > 30`,
+and `x³` then pushes the tanh argument into the thousands, where Metal's fast-math
+`tanh`/`exp` returns NaN instead of saturating. The fix clamps the inner argument
+to a safe `±15` (`SQRT_2_OVER_PI · (x + GELU_COEFF·x³)`), which is numerically
+indistinguishable from the true GELU on the saturated tails. This is the same
+class of Metal fast-math hazard already documented for `exp()` in the attention
+kernel (Section 7).
+
+### Validation and the public surface
+
+Validated **bit-exact vs HF transformers 5.12**: **per-token cosine = 1.000000**,
+**max absolute error ~5e-6**. The encoder is exposed as
+`engine.encodeImage(patches, gridTHW)` → merged image tokens `[rows, 1024]`,
+behind `enableVision: true` at load (the engine snapshots the raw
+`visual.pos_embed.weight` table before `uploadWeights` consumes it, for the host
+bilinear interpolation).
+
+### What is *not* done yet (phase 2)
+
+`encodeImage()` returns image tokens; it does **not** splice them into a text
+sequence. The remaining LM-side integration — **M-RoPE** position assignment for
+interleaved image/text tokens, **token splicing** of image embeddings into the
+input stream at the image placeholder positions, and the host-side **image
+preprocessing** (pixels → ordered patches) — is the in-progress phase 2. The
+encoder being bit-exact means that integration is plumbing over a verified
+numerical core, not open research.
+
+---
+
+## 23. The Native-Only Architecture Decision
+
+The earlier framing of this project as **"two lanes"** — a native WGSL engine for
+text plus a transformers.js/ONNX fallback lane for breadth — has been retired. The
+decided architecture (owner decision, see `docs/PROJECT-STATE.md`) is **one native
+WebGPU engine**:
+
+- **Launch set = text + vision + embeddings, all native, in one engine.** Text
+  generation, the Qwen3.5 ViT encoder (Section 22), and Qwen3-Embedding
+  (Section 21) all run through the same IR, kernel registry, and device layer. One
+  model per modality by default, expandable to other families via the
+  add-model-family process (`docs/adding-a-model-family.md`).
+- **A permanent fallback lane is rejected** — it "assumes defeat to begin with."
+  `transformers.js`/ONNX (`chrome-backend.ts`) is at most temporary dev scaffolding
+  to keep desktop demos alive during the native build, and is being **removed, not
+  kept** as a lane.
+- **Audio is deferred to native small models, not delegated to a second engine.**
+  Launching without audio is acceptable; a permanent second runtime is not. The
+  candidates are native: **OmniVoice** for TTS (a Qwen3 backbone + a codec decoder —
+  i.e. mostly the text path the engine already runs, plus a decoder) and
+  **Moonshine** for STT (a lean encoder-decoder with a raw-waveform Conv1d
+  frontend, no log-mel Conv2d). A thin onnxruntime-web bridge remains only a
+  break-glass option for a model with no extractable weights *and* no native
+  alternative.
+
+### Corrections to earlier "two-lane / blocked-vision" language
+
+Two claims that appeared in earlier roadmap text are now corrected for the record:
+
+- **"Native vision is blocked on a parallel attention kernel."** False — the
+  attention kernel was **already parallel** (Section 7). The blocking claim came
+  from reading `attention.wgsl`, a dead file imported nowhere; the live kernel is a
+  tiled flash-attention-style kernel. Non-causal attention is a one-line uniform
+  (`is_causal`), not a new kernel. **Vision is done at the encoder level**
+  (Section 22), bit-exact.
+- **"tfjs is a kept breadth lane."** It is not. The engine is native-only across
+  all launch modalities; `chrome-backend.ts` is slated for deletion.
+
+---
+
+## 24. iOS Model-Caching Reality
+
+A persistent finding this cycle, with consequences for how the engine ships on
+iOS: **durably caching a ~400 MB model in iOS/iPadOS Safari is not achievable from
+a plain browser tab.** The full investigation is in
+`docs/research/ios-safari-model-caching.md`; the load-bearing facts:
+
+- **Persistence requires a PWA.** WebKit's only documented positive heuristic for
+  granting `navigator.storage.persist()` is "opened as a Home-Screen Web App." A
+  plain tab gets `false` (confirmed by on-device probe). Only a persistence grant
+  excludes an origin from eviction — so durable model caching means shipping an
+  Add-to-Home-Screen PWA.
+- **Eviction is quota-pressure-driven, not reload-driven.** Best-effort data
+  survives reloads under normal conditions; what kills the cache on a near-full
+  device is WebKit's **origin-wide LRU eviction under storage pressure**. The
+  probed iPad had a ~1 GB origin quota (Safari 17+ sets it to ~60% of *free* disk,
+  and the device was nearly full) with ~444 MB already consumed by foreign caches —
+  a 400 MB write lands near the cap and gets evicted.
+- **Switching Cache API → IndexedDB does not help.** On iOS they share one
+  best-effort origin pool, evicted together under the same quota and 7-day-ITP
+  policy. The migration buys no durability.
+- **Main-thread OPFS is broken on iOS.** `createWritable()` on the main thread
+  throws OOM on the test device; the only viable large-write path is a **Worker
+  using OPFS `createSyncAccessHandle()`** in small chunks. This finding drove the
+  removal of the engine's OPFS write path in favor of a Cache-API-only loader
+  (`model-loader.ts`); a main-thread OPFS attempt left unclearable junk that filled
+  the quota and evicted everything.
+
+The practical consequence: as a plain tab the engine treats re-download as
+unavoidable and minimizes its cost (smaller/streamed model, HTTP cache, fast CDN);
+durable on-device caching is a PWA feature, gated behind the persistence grant,
+foreign-cache cleanup, and Worker-OPFS chunked writes.
+
+---
+
+## 25. EmbeddingGemma-300M: a Second Embedding Family, Validated On-Device (June 2026)
+
+§21 shipped native embeddings by reusing the *text* graph — Qwen3-Embedding is
+`Qwen3ForCausalLM` with a last-token-pool + L2Norm tail. That model proved the
+modality but not the engine's **generality across embedding families**, and it had a
+fatal mobile problem: the only weights that existed were BF16 (~1.2 GB, which OOMs
+the iPad) or a broken MLX-DWQ convert. This cycle adds **EmbeddingGemma-300M**
+(`mlx-community/embeddinggemma-300m-4bit`, ~173 MB at MLX-4bit) — the first
+**non-Qwen** embedding model, a genuinely different architecture, **and the first
+embedding model confirmed running on iPad Safari on-device** (owner-confirmed).
+
+### A real encoder, not a re-skinned causal LM
+
+EmbeddingGemma is a **bidirectional Gemma3 encoder**, so unlike §21 it needed its own
+graph generator (`architectures/gemma3_encoder.ts`, `generateGemma3EncoderGraph`).
+The architecture, read line-for-line from `config.json` (the generator hardcodes
+nothing — block count, dims and head config all come from the config; the 24-block /
+768-hidden / 3072-intermediate shape below is that model's config):
+
+- **24 pre-norm blocks, bidirectional.** The attention op is emitted with
+  `causal: false` — the same one-flag `is_causal` mechanism the ViT introduced
+  (§22), reused for an encoder. This is the second consumer of that flag and the
+  payoff of having made non-causal a uniform rather than a kernel.
+- **GQA 3 q / 1 kv, head_dim 256.** `q_dim = num_heads·head_dim`,
+  `kv_dim = num_kv_heads·head_dim`; for this model 3·256 = 768 query, 1·256 kv.
+- **Per-head q-norm / k-norm.** Each block emits `q_norm` and `k_norm` as `RMSNorm`
+  over `head_dim` before RoPE — the Gemma3 per-head QK normalization.
+- **Dual-theta RoPE, selected per layer from `layer_types`.** Gemma3 interleaves
+  sliding-window and full-attention layers, and they use *different* rotary bases.
+  The generator reads `layer_types[i]`: a `"full_attention"` layer takes
+  `rope_theta` (1e6); every other (local/sliding) layer takes `rope_local_base_freq`
+  (10000). The full head_dim is rotated. (This is a per-layer `layer_types` lookup,
+  not a fixed "every 6th" rule — the config happens to make every 6th layer global,
+  but the code keys off the type string.)
+- **GeGLU MLP.** `down(gelu_tanh(gate) · up)` — separate gate/up projections, a
+  `GELU` (tanh-approx) op on the gate, a `Mul`, then the down projection. Not the
+  SiLU-based fused SwiGLU of the Qwen text path.
+- **Gemma's four-norm "sandwich."** Each block carries four norms, not two:
+  `input_layernorm` (pre-attn), `post_attention_layernorm`, `pre_feedforward_layernorm`,
+  `post_feedforward_layernorm`. The two *post* norms are applied to the sublayer
+  output **before** the residual add — Gemma's distinguishing sandwich layout.
+- **Embedding scale ×√768.** The token embeddings are multiplied by
+  `√hidden_size` via a new `Scale` op before the blocks.
+- **Tail: MeanPool → Dense0 (768→3072) → Dense1 (3072→768) → L2Norm.** Unlike
+  Qwen3-Embedding's *last-token* EOS pool, EmbeddingGemma uses **mean pooling** over
+  all tokens (a new `MeanPool` kernel), followed by the model's two learned Dense
+  projection heads and a final L2 normalization to a unit-length 768-dim vector.
+
+### Two new kernels — and that is all
+
+The only genuinely new GPU kernels this model required are **`MeanPool`** and
+**`Scale`** (`kernels/registry.ts`):
+
+- **`MeanPool`** — `output[c] = (1/T)·Σ_t src[t·width + c]`: the column-mean of a
+  `[T, width]` activation into `[1, width]`. One thread per output channel,
+  workgroup_size 256, params `{ seq_len, width }`.
+- **`Scale`** — `output[i] = input[i]·scale`, the embedding normalizer. Params
+  `{ count, scale_bits }` (the f32 scale passed as a bit pattern and `bitcast` back).
+
+Everything else — RMSNorm, the bidirectional attention, RoPE, GELU, Mul, the INT4
+matmul, L2Norm — was already in the registry. A whole new embedding family cost two
+small reduction/elementwise kernels.
+
+### The (1+weight) norm absorption — baked by the loader, for MLX too
+
+Gemma's RMSNorm is `(1 + weight)·normalized`, not `weight·normalized`. Rather than
+fork the kernel, the **loader bakes the +1 into every Gemma norm weight** at load
+(`model-loader.ts`), so the standard RMSNorm kernel stays correct. The subtle part:
+this runs **even for MLX-4bit** Gemma. mlx-lm pre-absorbs the +1 for Qwen3.5 but
+**does not** for Gemma — so the Gemma branch deliberately omits the `&& !isMLX`
+guard the Qwen branch carries, and adds +1 to `input/post_attention/pre_feedforward/
+post_feedforward_layernorm`, `q_norm`, `k_norm`, and the final `norm`.
+
+### Validation: bit-faithful, then semantic
+
+Correctness was pinned against an **independent NumPy reference**
+(`scripts/engine/test-embedding-gemma-reference.py`) that re-implements the MLX
+affine dequant (`scale·nibble + bias`, group_size 64), the `(1+w)` Gemma norm, the
+`query_pre_attn_scalar^-0.5` attention scale, GQA repeat, dual-theta RoPE and the two
+Dense heads from raw safetensors. The reference asserts engine-vs-NumPy
+`cosine > 0.95` per probe; the measured result was **cos = 1.00000** (the commit's
+own headline). On top of that, a semantic test (`test-embedding-gemma.mjs`) confirms
+the geometry is *useful*, not merely reproducible: a "Red Planet" query embeds
+**closer to two Mars documents than to an unrelated sourdough-bread document by a
+>0.1 cosine margin**, all vectors are unit-norm at dim 768, and none are NaN or
+degenerate. The path is prefill-only (non-autoregressive), so it inherits the
+existing memory/submit machinery with no mobile-specific work — which is why, once
+the size dropped from 1.2 GB to 173 MB, **it simply ran on the iPad**.
+
+### Why this one matters
+
+Qwen3-Embedding proved the *tail*. EmbeddingGemma proves the *engine*: a different
+vendor, a different block structure (four-norm sandwich, dual-theta RoPE, per-head
+QK-norm, mean-pool, Dense heads), validated bit-faithful and then run on-device on
+the hardware that previously crashed. It is the difference between "we support an
+embedding model" and "the engine generalizes across embedding families." The
+abandoned 1.2 GB Qwen3-Embedding-on-iPad and the MLX-DWQ-garbage trap (§27) are the
+two dead ends this model routes around.
+
+---
+
+## 26. The SentencePiece Tokenizer Fix (a Cross-Family Lesson)
+
+EmbeddingGemma surfaced a tokenizer bug that had been **silently destroying
+semantics**, and the fix is a reusable lesson for any future non-GPT-lineage model.
+
+Gemma's `tokenizer.json` declares `"type": "BPE"` — but it is **SentencePiece-flavored
+BPE**, not the byte-level (GPT-2 / `Ġ`) BPE the engine was built for. The two
+disagree on nearly everything that matters:
+
+| | Byte-level BPE (Qwen, LFM2) | SentencePiece BPE (Gemma) |
+|---|---|---|
+| Space marker | `Ġ` (`Ġ`) | `▁` (U+2581) |
+| Token bytes | byte-to-unicode remapped | raw UTF-8 |
+| `merges` form | `"a b"` strings | `["a","b"]` arrays |
+| Unknown bytes | — | `<0xHH>` byte-fallback |
+
+Feeding a SentencePiece vocab through the byte-level path **char-split every word**:
+the pre-tokenizer's byte-to-unicode remap turned the raw-UTF-8 ▁-prefixed Gemma
+tokens into unmatchable garbage, the merges never fired, and the embeddings that came
+out were numerically "valid" (unit norm, no NaN) but **semantically meaningless** —
+exactly the kind of failure that passes a smoke test and fails a margin test.
+
+The fix (`tokenizer.ts`) **auto-detects SPM mode structurally**, with no model-name
+list. A `spmMode` flag is set true when either: the `normalizer` is (or contains, in
+a `Sequence`) a `Replace` node mapping `" " → "▁"`; or the model has
+`byte_fallback: true` **and** the vocab literally contains the ▁-prefixed token
+`"▁the"`. In SPM mode the tokenizer:
+
+- **encodes** by replacing spaces with ▁ and splitting on ▁ (keeping it attached to
+  the following piece) — *no* byte-to-unicode remap, BPE runs on raw UTF-8;
+- **decodes** by fusing runs of consecutive `<0xHH>` byte-fallback tokens into a
+  single UTF-8 decode (so a multi-byte codepoint split across byte tokens
+  reassembles) and finally mapping ▁ → space;
+- **parses merges** from both array and string form into one space-joined key.
+
+Crucially the detection is *structural*, so **Qwen and LFM2 fall through to the
+byte-level path unchanged** — they have no `" "→"▁"` normalizer and are not
+byte_fallback SPM vocabs, so `spmMode` is false and the `Ġ` machinery is untouched.
+The lesson, recorded for the add-model-family process: **`type: "BPE"` in a HF
+tokenizer is not a guarantee of byte-level BPE.** Any SentencePiece-lineage family
+(Gemma, Llama, Mistral) needs the SPM path, and the engine now picks it automatically.
+
+---
+
+## 27. MLX-4bit Loading: Broadened Detection, and the DWQ Trap
+
+Shipping a `mlx-community` checkpoint forced the loader to get precise about what
+"MLX-4bit" means (`model-loader.ts`). Three changes:
+
+1. **Detection broadened to mode-less configs.** Standard mlx-lm converts often emit
+   `{ bits: 4, group_size: N }` with **no `mode` field**, where earlier code only
+   recognized an explicit `mode: "affine"`. The loader now treats a config as
+   MLX-shaped when `bits === 4` and either `mode === "affine"` **or**
+   (`mode` is absent **and** `group_size` is a number).
+
+2. **A DWQ reject, because DWQ is config-indistinguishable.** Distillation-quantized
+   (DWQ) MLX repos carry the *same* `{bits:4, group_size}` config as a standard
+   affine convert but pack weights the engine's dequant can't read — they produce
+   garbage, not an error. Since the config can't tell them apart, the loader rejects
+   by **repo name**: any repo whose lowercased id contains `"dwq"` can never be
+   treated as a verified MLX repo.
+
+3. **A `VERIFIED_MLX_REPOS` allowlist.** A mode-less config only loads if its repo is
+   on a hardcoded allowlist of repos confirmed to be standard affine MLX-4bit (case-
+   insensitive substring match, currently exactly
+   `["mlx-community/embeddinggemma-300m-4bit"]`). The full gate is
+   `isMLX = hasMlxShape && (mode === "affine" || isVerifiedMlxRepo)` — so an explicit
+   `affine` config loads anywhere, but a bare `{bits:4, group_size}` only loads from a
+   vetted repo, and never from a DWQ one. This is the codified form of the
+   "MLX-DWQ-garbage trap" that wasted a cycle on the Qwen embedding model.
+
+---
+
+## 28. The Progress-Reporting Fix: Killing the "Stuck at 10%" Freeze
+
+A long-standing, every-model UX bug was fixed this cycle (commit `682a09b`,
+`model-loader.ts`): the download bar **froze at "10% — discovering weight files"**
+for several seconds on every load, looking like a hang. The cause was a *dead zone*
+of latency-heavy network round-trips between the "discovering" progress emit and the
+first emit that the download code produced — during which nothing was reported:
+
+- the **index probe** (`fetchJSON("model.safetensors.index.json")`),
+- **two header range-requests** per shard (`fetchRange(url, 0, 8)` to read the header
+  length, then a second range to read the full header), and
+- the **first-byte latency** of the first data Range request.
+
+The fix brackets that dead zone with two descriptive emits *before* the first chunk:
+a `"Reading {filename} header (i/N)…"` message right before the header fetch, and a
+`"Downloading {filename} (0/{totalMB} MB)"` message that shows the size up front so
+the first (latency-heavy) chunk doesn't read as a freeze. No throughput changed; the
+bar now narrates the round-trips it was previously silent through. It is a small fix
+with broad reach — it affected **every model the engine loads**.
+
+---
+
+## 29. Cross-Device Multi-Modal Parity (June 2026)
+
+The milestone this cycle is not any single model but their *intersection*: **text,
+vision, and embeddings now all run natively on iPad Safari/WebKit**, on the same
+device that crashed at the start of the campaign (§17). Concretely, on iPad
+(iPadOS 26.5, WebKit), through the one native WGSL engine:
+
+| Modality | Native model on iPad | Status |
+|---|---|---|
+| Text | Qwen3.5-0.8B INT4 | ✅ ~51 tok/s sustained (200-tok run), bit-correct |
+| Text (alt) | LFM2.5-350M | ✅ ~46 tok/s on-device; faster/smaller alternative |
+| Vision | Qwen3.5 ViT (`describeImage`) | ✅ runs; encoder bit-exact vs HF (§22) |
+| Embeddings | EmbeddingGemma-300M | ✅ runs on-device (§25), 173 MB |
+
+The desktop numbers remain higher (Qwen3.5 ~207 tok/s, §20; LFM2.5 ~600 tok/s on
+M4 Max), but the *parity* claim is about mobile: the native engine has reached the
+**same modality coverage on iPad that the transformers.js/ONNX path offered on the
+modalities that matter — without the mobile crashes, and faster** (native mobile
+decode is ~5× the transformers.js path, §19/PROJECT-STATE §4).
+
+The honest gap is **audio**. Neither STT nor TTS runs natively yet:
+
+- **TTS via OmniVoice** — in progress; a Qwen3 backbone (mostly the text path the
+  engine already runs) plus a codec decoder.
+- **STT via Moonshine** — not started; a lean encoder-decoder with a raw-waveform
+  Conv1d frontend (no log-mel Conv2d), needing a parallel CrossAttention kernel.
+
+Until those land, audio is the one launch modality that still requires the
+transformers.js path (Kokoro/Supertonic TTS, Whisper STT). The native engine is at
+**multi-modal parity minus audio**.
+
+---
+
+## 30. Model-Zoo Growth and the Saturation of the Kernel Library
+
+Two model-zoo facts close out the cycle, and together they describe a qualitative
+shift in what "add a model" costs.
+
+**LFM2.5-350M shipped** (`architectures/lfm2.ts`, `Lfm2ForCausalLM`) — a hybrid
+conv/attention text model, ~199 MB at q4 (half Qwen3.5's footprint), faster on both
+desktop (~600 tok/s, ~2.8× Qwen3.5) and mobile (~46 tok/s). It needed **no new
+kernels**. Two general fixes fell out of it and were kept: (1) LFM2's effective FF
+dim is the `block_auto_adjust_ff_dim`-rounded value, not the config's raw
+`intermediate_size`; (2) its "garbage output" was a **chat-template** problem, not a
+graph problem — LFM2.5 ships its template as a `chat_template.jinja` sidecar absent
+from `tokenizer_config.json`, so the engine fell back to Qwen ChatML and injected an
+empty `<think>` loop. Fetching the `.jinja` sidecar and gating think-injection on the
+template actually emitting `<think>` fixes **any** model with a jinja sidecar.
+
+**Adding a new *text* family is now usually Tier-1: a generator, no new kernels.**
+The kernel library has effectively **saturated** for standard transformers — Llama,
+Mistral, and Gemma-text all reduce to ops already in the registry (RMSNorm,
+GQA attention, RoPE, SwiGLU/GeGLU, INT4 matmul). New kernels are needed only for
+genuinely novel computation: a new norm, an SSM/Mamba path, PLE, a cross-attention
+for STT. The effort tiers (`docs/adding-a-model-family.md`) make this concrete —
+Tier 1 (hours, generator-only) covers most of the HF text zoo; Tier 2 (one novel op)
+and Tier 3 (SSM/MoE/new-executor) are now the exception. EmbeddingGemma was a
+Tier-2-ish outlier *only* because of MeanPool/Scale and the SPM tokenizer; the next
+text family will most likely cost a config→IR generator and nothing else.
+
+---
+
+## 31. Native STT: Moonshine (June 2026)
+
+§29 named audio as the one launch modality still on the transformers.js path. This
+cycle closes the **speech-to-text** half of that gap natively. The model is
+**Moonshine** (`architectures/moonshine.ts`, `moonshine-executor.ts`,
+`moonshine-stt.ts`) — a lean encoder-decoder ASR model chosen over Whisper
+precisely because it avoids the two things the native engine did not want to build:
+a log-mel front-end (a Conv2d/FFT pipeline) and a generic spectrogram path.
+
+### A raw-waveform Conv1d front-end (no FFT, no log-mel)
+
+Moonshine consumes **16 kHz PCM directly**. The front-end
+(`architectures/moonshine.ts`) is three strided 1-D convolutions, not a
+spectrogram:
+
+1. `Conv1d(1→H, kernel 127, stride 64)` + `tanh`,
+2. `GroupNorm(num_groups=1)` over the H channels,
+3. `Conv1d(H→2H, kernel 7, stride 3)` + GELU,
+4. `Conv1d(2H→H, kernel 3, stride 2)` + GELU,
+5. a `Transpose` from `[H, frames]` to `[frames, H]` for the transformer.
+
+The total downsample is 64·3·2 = **384×**, i.e. ~41.6 frames/second at 16 kHz.
+This needed three genuinely new kernels — **`Conv1dFull`**, **`GroupNorm`**, and
+**`Tanh`** (plus a `Transpose`) in `kernels/registry.ts` — and **no FFT and no
+2-D convolution**, which is the whole reason Moonshine was picked over Whisper.
+
+### The CrossAttention kernel — the one real new attention primitive
+
+The decoder attends to the frozen encoder output, so it needed a **cross-attention**
+kernel distinct from the engine's self-attention. `WGSL_CROSS_ATTENTION`
+(`kernels/registry.ts`, `crossAttentionSpec`) is a tiled, online-softmax
+cross-attention: decoder queries stream over the encoder sequence in tiles of 16
+with a 256-thread workgroup, running-max/running-sum softmax, V accumulation — and
+the same mobile-safe discipline the self-attention kernel uses (no `select()`,
+`exp()` clamped, ≤16 KB workgroup memory). It was validated against an independent
+NumPy reference (`scripts/engine/test-crossattention.mjs`): **max absolute error
+< 2e-4 and cosine ≥ 0.9999** — bit-exact within f32 rounding.
+
+### The dual-graph executor: encode once, freeze K/V, then decode
+
+The runtime is **two graphs, not one** (`moonshine-executor.ts`,
+`moonshine-stt.ts`):
+
+- `MoonshineEncoderExecutor.encode(pcm)` runs the Conv1d front-end and the
+  bidirectional encoder transformer **once** over the whole utterance, then
+  projects and caches the **per-decoder-layer K/V** from the encoder output.
+- `MoonshineSTT.transcribe(pcm)` binds that **frozen encoder K/V** into the decoder
+  and runs a normal greedy autoregressive loop, where each step does self-attention
+  over the generated text *plus* cross-attention into the frozen encoder K/V.
+
+This is the first encoder-decoder shape in the engine; the design choice — compute
+the encoder K/V exactly once and treat it as a constant during decode — is what
+keeps cross-attention cheap per token.
+
+### Interleaved RoPE
+
+Moonshine's rotary embedding is **interleaved**, not split-half. Where the engine's
+default RoPE (HF Llama `rotate_half`) pairs dim *i* with dim *i*+½·rope_dim,
+Moonshine pairs **adjacent** dims (2p, 2p+1) under a single frequency `inv_freq[p]`.
+This is a separate kernel, `WGSL_ROPE_INTERLEAVED` (`ROPE_INTERLEAVED_SPEC`),
+selected by an `interleaved: true` attribute on the encoder/decoder RoPE nodes;
+the comment records it as verified against HF's `MoonshineRotaryEmbedding`
+(`cos[:half].repeat_interleave(2)` + interleaved `rotate_half`).
+
+### Validation and status
+
+End-to-end transcription is exercised by `scripts/engine/test-moonshine-transcribe.mjs`,
+which asserts the transcript **contains the expected ground-truth substrings** for
+standard HF Moonshine reference clips (e.g. "stew for dinner", "his belly counsel"),
+and reports real-time-factor and the 4-bit size projection as informational output
+(both computed dynamically from the run, not hardcoded). The architecture comment
+records **encoder cosine ≈ 0.990 vs HF transformers** on Dawn. (The crisper
+"6/6 verbatim / RTF ~40× / ~31 MB at 4-bit" framing from the working notes is
+consistent with these checks, but the *enforced* gates in the committed tests are
+the substring-match transcript assertions and the bit-exact CrossAttention numbers
+above; quote those when precision matters.) **Whisper stays as the multilingual /
+no-WebGPU fallback** — `WhisperSTT` (`src/core/stt.ts`) is a separate
+transformers.js/ONNX path, untouched by and independent of the native Moonshine
+engine.
+
+---
+
+## 32. Native TTS: Kani-TTS-2 and the NanoCodec Decoder (June 2026)
+
+The **text-to-speech** half of the audio gap is **partly** landed: the hard,
+novel piece — the **NanoCodec audio decoder** — is implemented and validated
+bit-exact, while the codec-LM backbone's autoregressive driver is scaffolded and
+deliberately not yet runnable end-to-end. The model is **Kani-TTS-2**
+(`architectures/kani_tts.ts`).
+
+### The backbone: an LFM2-350M codec-LM
+
+Kani-TTS-2's backbone is **LFM2-350M** (arch string `KaniTTS2ForCausalLM`,
+model_type `lfm2`) reusing `generateLfm2Graph` almost verbatim — the LFM2 block
+math (RMSNorm, short-conv with conv-state cache, GQA, SwiGLU) shipped already in
+§30. It autoregressively emits **NanoCodec audio tokens, 4 per frame**, into a
+vocab that extends *above* the text vocab (`vocab_size` 80538 vs `text_vocab_size`
+64400; audio token IDs start at 64410). The backbone-specific additions are
+frame-level position IDs (audio tokens within a frame share a position; text tokens
+advance by one), a **learnable per-layer RoPE**
+(`α^(l) = alpha_min + (alpha_max−alpha_min)·sigmoid(alpha_weight^(l))`), the
+4-token-frame decode loop, and an optional speaker-embedding projection.
+`generateKaniTtsGraph` currently **throws a descriptive error** rather than emit a
+half-wired graph: it parses and reports the config, confirms the decoder is done,
+and lists the remaining position/RoPE/decode-driver glue.
+
+### The NanoCodec decoder — FSQ + causal HiFi-GAN, the novel part
+
+`generateNanoCodecDecoderGraph` is the genuinely new computation, and it is
+**complete and validated**. Two stages:
+
+- **FSQ dequant.** NanoCodec uses **finite scalar quantization**: 4 groups × 4
+  dims, per-group levels `[9, 8, 8, 7]` (codebook 4032), mixed-radix base
+  `[1, 9, 72, 576]`. Each audio code is unpacked by
+  `nonneg = (idx // base[d]) % levels[d]; code = (nonneg − L/2)/(L/2)` — a new
+  **`FSQDequant`** kernel.
+- **Causal HiFi-GAN vocoder.** A `CausalConv1d(16→864, k7)`, then 5 upsample
+  stages with rates `[7, 7, 6, 3, 2]` (each: `HalfSnake` → depthwise
+  `CausalConvTranspose1d(k = 2·rate)` → a HiFi-GAN residual layer averaging
+  kernels `[3, 7, 11]` over dilations `[1, 3, 5]`), then `HalfSnake` → a final
+  `CausalConv1d(27→1, k3)` and a clamp to [-1, 1]. The hop is 1764, so PCM length
+  = `frames · 1764` at 22050 Hz.
+
+This needed two more new kernels beyond `FSQDequant`: **`HalfSnake1d`** (snake
+activation on the first half of the channels, leaky-ReLU on the rest) and
+**`ConvTranspose1dDepthwise`** (causal depthwise transposed convolution) — all
+three registered in `kernels/registry.ts`.
+
+### Validation and the license note
+
+The full decoder is checked by `scripts/engine/test-nanocodec-decode.mjs` against a
+real MLX reference. The committed assertion gate is **`err < 1e-3`** (and matching
+PCM length); the run prints the *actual* measured error, which the code's own
+headers record as **max|err| ≈ 4.2e-6 vs the MLX reference** — i.e. bit-exact
+within f32 rounding, with comfortable margin under the 1e-3 gate. **Status:**
+NanoCodec decoder + Kani scaffold landed and validated; the codec-LM backbone's
+frame-position / learnable-RoPE / 4-token-frame AR loop is the remaining glue
+(most of the block math is reused from `lfm2.ts`).
+
+**Licensing** (recorded in the source header): the shipping **kani-tts-2-en** is
+**LFM1.0 (Liquid AI), `license: other`** — it fine-tunes LFM2-350M, so it is *not*
+Apache; the NanoCodec weights are under the **NVIDIA Open Model License**. The
+older `kani-tts-450m-0.2-ft` variant is **Apache-2.0** with the same architecture.
+
+---
+
+## 33. Gemma 4 E2B: PLE, KV-Sharing, and Logit Softcap (June 2026)
+
+Gemma 4 E2B is the smallest Gemma 4 and the engine's first **Tier-2** text decoder
+beyond the Gemma3 encoder of §25 — a *text-only* model with several Gemma-4-specific
+ops, but **no MatFormer / AltUp / LAuReL** (those belong to Gemma-3n, not E2B). Its
+graph generator is `architectures/gemma4.ts`, building on the Gemma machinery from
+`gemma3_encoder.ts` with a causal LM tail. The decode graph is **structurally
+complete and validated**; the gate to running it on real weights is embedding
+sharding (below).
+
+### What is Gemma-4-specific
+
+- **Per-Layer Embeddings (PLE).** A *second* embedding table is gathered per token,
+  projected once (`per_layer_model_projection` + `per_layer_projection_norm`), and
+  then injected at **every** layer:
+  `h = h + post_norm( per_layer_projection( gelu(gate(h)) · ple_i ) )` — a per-layer
+  gate, GELU, elementwise multiply with that layer's PLE slice, projection, norm,
+  and residual.
+- **KV-cache sharing.** The last `num_kv_shared_layers` layers reuse the K/V cache of
+  the matching `layer_type` from before the shared region — a *graph-level rewire*
+  with no kernel change and no K/V projection emitted for the shared layers. For E2B
+  that is **35 layers total, the last 20 shared** (layers 15–34 reuse layers 0–14 of
+  matching type).
+- **Proportional / dual-theta RoPE.** Full-attention layers rotate the first
+  `partial_rotary_factor·head_dim` dims (0.25·256 = 64) but compute `inv_freq` over
+  the **full** `head_dim` denominator (the `rope_denom` attribute) — distinct from
+  Qwen3.5's partial RoPE, which divides by the *rotated* dim. Full layers use
+  `rope_theta` 1e6, sliding layers 1e4 — the same dual-theta selection as the Gemma3
+  encoder.
+- **GeGLU MLP** (`down(gelu_tanh(gate) · up)`) and Gemma's four-norm sandwich,
+  reused from §25.
+- **Final logit softcap via a new `Softcap` kernel.** `WGSL_SOFTCAP`
+  (`kernels/registry.ts`, `softcapSpec`) computes `out = cap · tanh(in / cap)` with
+  the cap passed as a bit-pattern; for E2B `cap = 30`, squashing the final logits to
+  ±30. It is wired into the graph tail (`logit_softcap` node) when
+  `final_logit_softcapping > 0`, and validated by `test-gemma4-softcap.mjs`
+  (GPU vs CPU reference, max err < 1e-4; saturation behavior confirmed).
+
+### Validation and the gating item
+
+`scripts/engine/test-gemma4-graph.mjs` is a structural validation of the generated
+graph — embedding+PLE pipeline, per-layer PLE injection chain, the sandwich norms,
+QK-norm + GeGLU, causal attention, proportional RoPE per layer-type, the KV-share
+rewire (own K/V for layers 0–14, shared for 15–34), the LM head and the softcap —
+and it passes **62/62** (893 nodes, 1920 tensors).
+
+The open item is **embedding sharding**. At 4-bit the PLE nibble buffer is
+**~1.17 GB** (≈1174 MB) and the main embedding is **~201 MB**, both over the
+per-binding cap, so the loader / `EmbeddingInt4` op must shard the quantized tables
+across multiple storage buffers before the decode path can run on real weights. The
+decode graph (last-row slice → tied LM head → Softcap) is otherwise complete and
+verified; sharding is loader-level work, in progress.
+
+---
+
+## 34. On-Device Memory / RAG (June 2026)
+
+The native embedding stack (§21, §25) makes a small **retrieval-augmented memory**
+layer cheap to build entirely on-device, with no server and no external vector DB.
+It ships as `@tryhamster/gerbil/memory` (`src/memory/`, exported in `package.json`)
+and reuses the **native EmbeddingGemma** path through a thin adapter.
+
+### Shape
+
+- **Pluggable vector stores.** Three backends behind one interface
+  (`src/memory/stores/`): `InMemoryStore` (default, process lifetime),
+  `IndexedDBStore` (browser, durable across sessions), and `FileStore` (Node, a
+  durable JSON file on disk) — via `createInMemoryStore()` / `createIndexedDBStore()`
+  / `createFileStore(path)`.
+- **Native-embedder adapter.** `createGerbilEmbedder(engine)`
+  (`src/memory/gerbil-embedder.ts`) wraps anything with an `embedBatch` — a Gerbil
+  instance, the one-liner `embedBatch`, or the browser `useEmbedding().embedBatch` —
+  so a memory is built with `createMemory({ embed: createGerbilEmbedder(g) })` after
+  `g.loadModel("embeddinggemma-300m")`. The whole pipeline (embed → store → recall)
+  runs on the native WGSL engine.
+- **Chunking.** `add(text, { chunk: true })` splits long documents into overlapping
+  character windows (`src/memory/chunking.ts`, defaults 1000 chars / 200 overlap),
+  one record per chunk, so retrieval targets relevant passages.
+- **Redaction on write.** `applyRedaction` (`src/memory/redaction.ts`) accepts a
+  `RegExp` (matches replaced with `[REDACTED]`) or a function, applied **before**
+  the text is embedded and stored, so sensitive data never lands in the index.
+- **Token-budgeted recall.** `recall(query, options)` (`src/memory/memory.ts`)
+  searches the store, then **greedily packs the highest-scoring records under a
+  token budget** (default 1024, via a ~4-chars/token estimate, accounting for the
+  separator), returning `{ context, records, tokensUsed }` — a ready-to-inject
+  context string sized to fit a prompt budget.
+
+### Validation
+
+`src/memory/memory.test.ts` is **12 tests** covering relevance ranking, metadata
+filtering, budget-aware packing (including the empty-context edge), chunking,
+import/export round-trip, regex and predicate redaction, and durability for both the
+IndexedDB and File stores. The module is a straightforward, fully-tested consumer of
+the native embedding modality — no new kernels, no GPU work of its own.
+
+---
+
+## 35. The June-2026 Autoresearch Campaign (Text, LFM2, and ViT)
+
+§20 documented the first autoresearch run (Qwen3.5 decode 145→207 on M4 Max). After
+LFM2.5 (§30) and the ViT (§22) landed, the same loop
+(`scripts/engine/optimize.mjs --mode=autoresearch`; results in
+`scripts/engine/results.jsonl`, chart at `scripts/engine/chart.html`) was run over
+**three more batches** — two on text, one on vision — all on M4 Max / node-dawn. The
+numbers are smaller than the first run's, and **that is the finding**: the tuned
+kernels are now near the bandwidth floor, so only a specific *class* of edit still
+wins.
+
+### The three batches
+
+| Batch | Target | Baseline | Best | Kept | Reverted |
+|---|---|---|---|---|---|
+| Text (b1) | Qwen3.5-0.8B Q4 decode | 219.1 | ~223 | 3 | 5 |
+| Text (b1) | LFM2.5-350M Q4 decode | 624.1 | ~649 | (same batch) | |
+| Text (b2) | Qwen3.5 / LFM2.5 decode | 220.8 / 652.6 | **234.4 / 672.2** | several | several |
+| Vision (b3/b4) | Qwen3.5 ViT encode | 581.8 ms | **~502 ms** (−~14%) | 3 + 1 | 3 |
+| Vision (b3/b4) | `describeImage` decode | 37.0 tok/s | **42.0 tok/s** (+13.5%) | (same batch) | |
+
+The text wins were **fusions that also kill a wide-tensor round-trip**: fuse SiLU
+into Qwen's CausalConv1d (+1.8%), fuse LFM2's post-gate `C·conv` (+2.0%) and its
+pre-gate `B·x`/`B/x` slices via a `MulCols` (+1.9%); the second batch pushed Qwen to
+~234 and LFM2 to ~672. The vision wins were all in the **shared f32 `MatMul`** that
+dominates ViT encode — 2×2 register blocking (−6.5%), vec4 global-tile loads
+(−3.0%), 4×2 register blocking (−1.9%), plus a `MatMul`+`AddBias` → `MatMulBias`
+structural fusion (desktop ~flat but removes ~86 dispatches and wide round-trips per
+encode). Text decode was *unaffected* by the vision batch (it uses `MatMulInt4`,
+confirmed steady at ~233 tok/s throughout), and every kept ViT change stayed
+bit-exact (merged cosine 1.0, e2e 7/7, description matches HF).
+
+### The lesson, sharpened
+
+The first campaign's lesson ("stop optimizing the kernel in front of you; profile
+the whole forward") generalized into a sharper rule, recorded in the batch summaries:
+
+> **On desktop Dawn, dispatch-count cuts only win when they also eliminate a wide
+> round-trip on a poorly-occupied kernel.** Pure dispatch/barrier reduction on an
+> already-tuned kernel is *noise*.
+
+Concretely, the desktop wins came from eliminating **large, wide reads on
+poorly-occupied kernels** — fused conv+activation, register-blocked + f16-mixed ViT
+matmuls — while the tuned INT4 matmuls and the Mamba SSM sit at the **bandwidth
+floor**, where butterfly reductions, subgroup shuffles, and bigger N-tiles came back
+flat or negative and were reverted. The honest read is that **the remaining headroom
+is mobile, not desktop**: several reverted-on-desktop fusions (the `MatMul`+`AddBias`
+merge, residual-Add fusion) are *predicted mobile wins* because mobile is
+round-trip-bound below ~32 dispatches per command buffer (§19) — which is exactly
+why the autoresearch loop's next leg is a mobile-validation pass rather than more
+desktop rounds.
+
+---
+
+## Appendix A: File Map
+
+```
+src/gpu/
+  ir.ts                          -- IR types: OpType, TensorDesc, OpNode, ModelGraph, CANONICAL_KEYS
+  safetensors.ts                 -- Safetensors binary parser with zero-copy typed views
+  device.ts                      -- WebGPU device init, buffer helpers, pipeline cache, readback
+  tokenizer.ts                   -- Pure JS BPE tokenizer from HF tokenizer.json
+  sampler.ts                     -- CPU-side token sampling (temp/top-k/top-p/rep penalty)
+  executor.ts                    -- Graph executor: buffer allocation, op dispatch, forward pass
+  kv-cache.ts                    -- GPU-resident KV cache: allocation, advance, reset, destroy
+  model-loader.ts                -- HF Hub integration: fetch config/tokenizer/weights, generate IR (Cache-API-only; OPFS removed)
+  vision-executor.ts             -- VisionExecutor: runs the ViT graph (single prefill over N patches)
+  vision-preprocess.ts           -- Host-side ViT pos-embeds + 2D rotary cos/sin (grid-only, bit-exact vs HF)
+  moonshine-executor.ts          -- MoonshineEncoderExecutor: raw-PCM Conv1d front-end + bidirectional encoder, frozen per-layer K/V
+  moonshine-stt.ts               -- MoonshineSTT.transcribe(): AR decoder with self- + cross-attention into frozen encoder K/V
+  architectures/
+    index.ts                     -- Architecture registry: maps HF strings to graph generators
+    qwen2.ts                     -- Qwen2/3/3.5 graph generator (SwiGLU MLP, GQA attention; embedding tail for Qwen3-Embedding)
+    qwen3_5_vision.ts            -- Qwen3.5 12-layer ViT encoder graph (bidirectional, 2D rotary, spatial-merge-2)
+    gemma3_encoder.ts            -- EmbeddingGemma-300M bidirectional encoder (4-norm sandwich, dual-theta RoPE, per-head QK-norm, GeGLU, mean-pool + 2 Dense heads + L2Norm)
+    lfm2.ts                      -- LFM2.5 hybrid conv/attention text generator (Lfm2ForCausalLM; Tier-1, no new kernels)
+    moonshine.ts                 -- Moonshine STT encoder-decoder graph (raw-waveform Conv1d front-end, interleaved RoPE, cross-attention)
+    kani_tts.ts                  -- Kani-TTS-2: LFM2-350M codec-LM scaffold + NanoCodec decoder graph (FSQ + causal HiFi-GAN); decoder validated, backbone AR-loop pending
+    gemma4.ts                    -- Gemma 4 E2B Tier-2 text decoder (PLE, KV-cache sharing, proportional/dual-theta RoPE, GeGLU, final logit Softcap)
+  kernels/
+    registry.ts                  -- KernelSpec registry: WGSL sources, bindings, dispatch sizing, uniform builders (incl. SliceLastRow, fused decode kernels, MeanPool, Scale, L2Norm, CrossAttention, ROPE_INTERLEAVED, Conv1dFull/GroupNorm/Tanh/Transpose, FSQDequant/HalfSnake1d/ConvTranspose1dDepthwise, Softcap)
+    wgsl/
+      embedding.wgsl             -- Embedding lookup (gather rows by token ID)
+      matmul.wgsl                -- Tiled f32 matrix multiply (16x16 shared memory)
+      matmul_int4.wgsl           -- Fused INT4 dequantize + matmul
+      rmsnorm.wgsl               -- RMS normalization (tree reduction)
+      layernorm.wgsl             -- Layer normalization (mean + variance, tree reduction)
+      rope.wgsl                  -- Rotary position embeddings (GQA-aware)
+      attention.wgsl             -- Scaled dot-product attention (causal, GQA)
+      softmax.wgsl               -- Row-wise softmax (three-pass, tree reduction)
+      silu.wgsl                  -- SiLU activation (x * sigmoid(x))
+      gelu.wgsl                  -- Approximate GELU activation
+      add.wgsl                   -- Element-wise addition
+      mul.wgsl                   -- Element-wise multiplication
+```
+
+---
+
+## Appendix B: WebGPU Browser Compatibility
+
+| Browser | Version | Status |
+|---------|---------|--------|
+| Chrome | 113+ (May 2023) | Stable support |
+| Edge | 113+ (May 2023) | Stable support (Chromium-based) |
+| Safari | 18+ (Sep 2024) | Stable support (macOS + iOS) |
+| Firefox | 141+ (Jan 2025) | Stable support |
+| Chrome Android | 113+ | Stable support |
+| Safari iOS | 18+ | Stable support (via WKWebView) |
+| Samsung Internet | 25+ | Stable support |
+
+### Notable Limitations
+
+- **iOS WKWebView memory**: the jetsam budget for a web-content process is ~1.5-2 GB. Post-fix (Sections 17.1, 18.1-18.2), Qwen3.5-0.8B INT4 at `maxSeqLen=512` runs at ~0.6-0.7 GB total GPU footprint — comfortable headroom, versus the 2.77 GB the per-tensor allocator used to request. The engine clamps iOS `maxSeqLen` to 2048 (default 512) so an oversized request can never reach the device.
+- **iPad WebGPU limits** (measured, iPadOS 26.5 defaults): `maxBufferSize` 256 MB, `maxStorageBufferBindingSize` 128 MB, `maxComputeWorkgroupStorageSize` 16384 bytes. The ~127 MB INT4 embedding sits just under the binding limit — larger vocabularies need sharding.
+- **WebKit submit granularity**: on WebKit older than iPadOS 26.5, batching many dispatches into one command buffer can zero storage reads mid-chain (Section 17.2); the engine's grouped-submit path (`?group=N`, Section 18.3) is the compatibility dial. iPadOS 26.5+ is correct at batch-all.
+- **Firefox**: WebGPU support arrived later than Chromium and Safari. Feature coverage is complete but performance may vary.
+- **shader-f16**: Not available on all GPUs. The engine detects this at initialization and adapts accordingly (currently all kernels use f32).
+- **maxBufferSize**: Varies by device. The engine requests the adapter's maximum, but some mobile GPUs have limits below 256MB which constrains model size.