@tryhamster/gerbil 1.1.2 → 1.1.3

package/docs/gpu-engine/safetensors.md

@@ -1,312 +0,0 @@
-# Safetensors Parser Deep Dive
-
-The safetensors parser (`src/gpu/safetensors.ts`) reads HuggingFace's binary safetensors format and provides zero-copy typed array views into the raw data. This document covers the binary format, the parser's design, alignment handling, and streaming support.
-
----
-
-## Binary Format
-
-The safetensors format is intentionally simple. A file consists of three contiguous sections:
-
-```
-+--------------------------------------------------+
-| 8 bytes: header_length (little-endian uint64)     |
-+--------------------------------------------------+
-| header_length bytes: JSON header (UTF-8)          |
-+--------------------------------------------------+
-| remaining bytes: raw tensor data (contiguous)     |
-+--------------------------------------------------+
-```
-
-### Section 1: Header Length (8 bytes)
-
-The first 8 bytes encode the JSON header length as a little-endian unsigned 64-bit integer. In practice, headers are always well under 4GB, so only the lower 32 bits are meaningful. The parser uses `DataView.getBigUint64(0, true)` and converts to a JavaScript `Number`.
-
-### Section 2: JSON Header
-
-The header is a JSON object mapping tensor names to their metadata:
-
-```json
-{
-  "model.embed_tokens.weight": {
-    "dtype": "F32",
-    "shape": [151936, 896],
-    "data_offsets": [0, 544534528]
-  },
-  "model.layers.0.input_layernorm.weight": {
-    "dtype": "F32",
-    "shape": [896],
-    "data_offsets": [544534528, 544538112]
-  },
-  "__metadata__": {
-    "format": "pt"
-  }
-}
-```
-
-Each tensor entry contains:
-- `dtype`: Data type string (see dtype table below)
-- `shape`: Array of dimension sizes
-- `data_offsets`: `[start, end]` byte offsets relative to the beginning of the data section
-
-The special `__metadata__` key is optional and contains file-level metadata (e.g., the framework that produced the file).
-
-### Section 3: Tensor Data
-
-Raw tensor data, stored contiguously. Each tensor's data occupies `data_offsets[1] - data_offsets[0]` bytes starting at `data_start + data_offsets[0]`, where `data_start = 8 + header_length`.
-
-Tensors are stored in row-major order (C contiguous). Multi-dimensional tensors are flattened: element `[i, j, k]` of a tensor with shape `[D0, D1, D2]` is at flat index `i * D1 * D2 + j * D2 + k`.
-
----
-
-## Layout Diagram
-
-For a file containing two tensors, "A" (F32, shape [2, 3]) and "B" (F32, shape [4]):
-
-```
-Byte 0                                          Byte N
-|<-- 8 -->|<------ header_length ------->|<--- tensor data --->|
-+=========+============================+=======================+
-| len=120 | {"A":{"dtype":"F32",       | A: 24 bytes (6 f32)   |
-|  (u64)  |  "shape":[2,3],            | B: 16 bytes (4 f32)   |
-|         |  "data_offsets":[0,24]},    |                       |
-|         | "B":{"dtype":"F32",         |                       |
-|         |  "shape":[4],               |                       |
-|         |  "data_offsets":[24,40]}}    |                       |
-+=========+============================+=======================+
-```
-
-Offsets:
-- Header starts at byte 8
-- Data section starts at byte 8 + header_length = 128
-- Tensor A data: bytes 128 to 151 (24 bytes)
-- Tensor B data: bytes 152 to 167 (16 bytes)
-
----
-
-## DType Mapping
-
-| Safetensors DType | Bytes | Alignment | JS TypedArray | Notes |
-|-------------------|-------|-----------|---------------|-------|
-| `F32` | 4 | 4 | `Float32Array` | Most common for model weights |
-| `F16` | 2 | 2 | `Uint16Array` | No native f16 typed array; bitwise representation |
-| `BF16` | 2 | 2 | `Uint16Array` | Brain float 16; bitwise representation |
-| `F64` | 8 | 8 | `Float64Array` | Rare in ML models |
-| `I32` | 4 | 4 | `Int32Array` | |
-| `U32` | 4 | 4 | `Uint32Array` | |
-| `I16` | 2 | 2 | `Int16Array` | |
-| `U16` | 2 | 2 | `Uint16Array` | |
-| `I8` | 1 | 1 | `Int8Array` | Used in some quantization schemes |
-| `U8` | 1 | 1 | `Uint8Array` | |
-| `I64` | 8 | 8 | `BigInt64Array` | |
-| `U64` | 8 | 8 | `BigUint64Array` | |
-| `BOOL` | 1 | 1 | `Uint8Array` | |
-
-Note on F16 and BF16: JavaScript has no native `Float16Array`. The parser returns `Uint16Array` views containing the raw bit patterns. To use F16 data with WebGPU, the bits can be uploaded directly to a GPU buffer typed as `f16` in WGSL. To use them on CPU, manual conversion to f32 is required:
-
-```typescript
-// F16 to F32 conversion (for CPU use)
-function f16ToF32(bits: number): number {
-  const sign = (bits >> 15) & 1;
-  const exp = (bits >> 10) & 0x1f;
-  const frac = bits & 0x3ff;
-  if (exp === 0) return (sign ? -1 : 1) * 2 ** -14 * (frac / 1024);
-  if (exp === 31) return frac ? NaN : (sign ? -Infinity : Infinity);
-  return (sign ? -1 : 1) * 2 ** (exp - 15) * (1 + frac / 1024);
-}
-```
-
----
-
-## Zero-Copy Design
-
-The parser's core principle is to avoid copying data whenever possible. `getTensorData()` returns a typed array **view** into the original `ArrayBuffer`:
-
-```typescript
-export function getTensorData(
-  buffer: ArrayBuffer,
-  file: SafetensorsFile,
-  entry: SafetensorEntry,
-): ArrayBufferView {
-  const offset = file.dataStart + entry.dataOffset;
-  return makeTypedView(buffer, offset, entry.dataLength, entry.dtype);
-}
-```
-
-### When Zero-Copy Works
-
-A typed array view requires that the byte offset is aligned to the element size. For example, a `Float32Array` requires 4-byte alignment. Since safetensors data is stored contiguously and most tensors are F32, alignment is almost always satisfied:
-
-```typescript
-// Zero-copy path (aligned):
-const src = buffer;        // Original buffer
-const base = offset;       // Offset into original buffer
-return new Float32Array(src, base, byteLength / 4);
-```
-
-### When Copying Is Required
-
-If the offset is not aligned (e.g., an F32 tensor starts at a byte offset that is not a multiple of 4), the parser copies the relevant slice:
-
-```typescript
-// Copy path (misaligned):
-const src = buffer.slice(offset, offset + byteLength);  // New aligned buffer
-const base = 0;
-return new Float32Array(src, base, byteLength / 4);
-```
-
-In practice, misalignment is rare because safetensors writers typically align tensor data. It only occurs when small tensors of mixed dtypes are packed in unusual ways.
-
----
-
-## Parser API
-
-### `parseSafetensorsHeader(buffer: ArrayBuffer): SafetensorsFile`
-
-Parses the header from a buffer. Can be called with just the header portion (first 8 + header_length bytes) or the entire file.
-
-Returns:
-```typescript
-interface SafetensorsFile {
-  headerLength: number;                    // Length of JSON header in bytes
-  dataStart: number;                       // Byte offset where tensor data begins (8 + headerLength)
-  entries: SafetensorEntry[];              // All tensor entries, sorted by offset
-  metadata: Record<string, string> | null; // Optional __metadata__
-}
-```
-
-Entries are sorted by `dataOffset` for sequential access patterns during loading.
-
-### `getTensorData(buffer, file, entry): ArrayBufferView`
-
-Returns a typed array view for a specific tensor entry. Zero-copy when alignment allows.
-
-### `findTensor(file, name): SafetensorEntry | undefined`
-
-Find a tensor entry by exact name match. Linear scan; suitable for the typical case of 100-500 tensors per file.
-
-### `parseSafetensorsFromResponse(response: Response): Promise<{file, fullBuffer}>`
-
-Convenience function that reads an entire HTTP response into an `ArrayBuffer` and parses the header. Returns both the parsed header and the full buffer for subsequent tensor extraction.
-
-### `totalTensorBytes(file): number`
-
-Sums the `dataLength` of all tensor entries. Useful for progress bars and memory budget estimation.
-
----
-
-## Streaming Support
-
-The parser supports a two-phase loading strategy for large models:
-
-### Phase 1: Header Only
-
-Download just the first 8 + header_length bytes using an HTTP Range request. Parse the header to discover tensor names, shapes, and offsets.
-
-```typescript
-// Fetch just the first 64KB (more than enough for most headers)
-const headerResponse = await fetch(url, {
-  headers: { Range: "bytes=0-65535" }
-});
-const headerBuffer = await headerResponse.arrayBuffer();
-const file = parseSafetensorsHeader(headerBuffer);
-
-// Now we know all tensor names, shapes, and byte offsets
-for (const entry of file.entries) {
-  console.log(`${entry.name}: ${entry.dtype} ${entry.shape} @ offset ${entry.dataOffset}`);
-}
-```
-
-### Phase 2: Selective Tensor Download
-
-Download specific tensors by offset using Range requests:
-
-```typescript
-const start = file.dataStart + entry.dataOffset;
-const end = start + entry.dataLength - 1;
-const dataResponse = await fetch(url, {
-  headers: { Range: `bytes=${start}-${end}` }
-});
-```
-
-This is useful when:
-- You only need a subset of tensors (e.g., loading a single layer for testing)
-- Memory is constrained and you want to upload tensors to GPU one at a time, freeing CPU memory between downloads
-- The model uses sharding and you want to parallelize downloads of independent shards
-
----
-
-## Multi-Shard Support
-
-Large models split their weights across multiple safetensors files. The split is described in `model.safetensors.index.json`:
-
-```json
-{
-  "metadata": { "total_size": 1234567890 },
-  "weight_map": {
-    "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
-    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
-    "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
-    "lm_head.weight": "model-00003-of-00003.safetensors"
-  }
-}
-```
-
-The model loader (`model-loader.ts`) handles this automatically:
-1. Tries to fetch `model.safetensors.index.json`
-2. If found, extracts the unique filenames from `weight_map`
-3. Downloads each shard and parses it independently
-4. Maps all tensors from all shards to canonical names
-
----
-
-## Example: Parsing a Safetensors File
-
-```typescript
-import { parseSafetensorsHeader, getTensorData, findTensor } from "./safetensors.js";
-
-// Assume `buffer` is an ArrayBuffer from fetch()
-const file = parseSafetensorsHeader(buffer);
-
-console.log(`Header: ${file.headerLength} bytes`);
-console.log(`Data starts at: ${file.dataStart}`);
-console.log(`Tensors: ${file.entries.length}`);
-
-// Find a specific tensor
-const embedding = findTensor(file, "model.embed_tokens.weight");
-if (embedding) {
-  console.log(`Embedding: ${embedding.dtype} ${embedding.shape}`);
-  console.log(`  Offset: ${embedding.dataOffset}, Length: ${embedding.dataLength}`);
-
-  // Get the data as a typed array
-  const data = getTensorData(buffer, file, embedding);
-  console.log(`  Type: ${data.constructor.name}`);
-  console.log(`  Elements: ${data.byteLength / 4}`);
-
-  // For F32, we can read values directly
-  const floats = data as Float32Array;
-  console.log(`  First 5 values: ${Array.from(floats.slice(0, 5))}`);
-}
-```
-
----
-
-## Memory Considerations
-
-A full model's weights must fit in memory twice during loading:
-1. The raw `ArrayBuffer` from the HTTP response
-2. The GPU buffer after upload
-
-After GPU upload, the CPU-side `ArrayBuffer` can be released (garbage collected) since the typed array views no longer need the backing buffer. The model loader handles this by processing one safetensors shard at a time.
-
-For a 1.7B parameter model in F32:
-```
-Weight bytes = 1.7e9 * 4 = 6.8 GB (would not fit in browser memory)
-```
-
-This is why quantized models are preferred for browser inference. In INT4 (with group scales):
-```
-Weight bytes ~ 1.7e9 * 0.5 + overhead = ~1 GB (feasible)
-```
-
-The safetensors parser itself is lightweight -- it only parses the header (a few KB) and creates views into the existing buffer. The dominant memory cost is the buffer itself, not the parser.

package/docs/gpu-engine/tokenizer.md

@@ -1,302 +0,0 @@
-# BPE Tokenizer Deep Dive
-
-The tokenizer (`src/gpu/tokenizer.ts`) is a pure JavaScript Byte Pair Encoding (BPE) implementation that reads HuggingFace `tokenizer.json` files. No WASM, no native dependencies, no external libraries.
-
----
-
-## Overview
-
-The `Tokenizer` class provides:
-- `encode(text)` -- Convert text to token IDs
-- `decode(ids)` -- Convert token IDs back to text
-- `applyChatTemplate(messages)` -- Format chat messages (ChatML)
-- `encodeChat(messages)` -- Template + encode in one call
-- `Tokenizer.fromJSON(tokenizerJSON, configJSON)` -- Factory from HF files
-
----
-
-## Internal Data Structures
-
-The tokenizer maintains five core maps built from `tokenizer.json`:
-
-| Map | Type | Source | Purpose |
-|-----|------|--------|---------|
-| `vocab` | `string -> number` | `model.vocab` | Token string to ID |
-| `vocabReverse` | `number -> string` | (inverse of vocab) | ID to token string |
-| `merges` | `string -> number` | `model.merges` | Merge pair to priority (lower = higher priority) |
-| `specialTokens` | `string -> number` | `added_tokens` (where `special: true`) | Special token detection |
-| `byteFallback` | `number -> string` | Derived from vocab | Byte value to `<0xHH>` token string |
-
----
-
-## Encoding Pipeline
-
-The encoding pipeline transforms text into a sequence of token IDs through five stages:
-
-### Worked Example: `"Hello world"`
-
-#### Stage 1: Special Token Splitting
-
-The text is split around any special tokens (like `<|im_start|>`, `<|endoftext|>`). Regular text segments are marked `special: false`.
-
-```
-Input:  "Hello world"
-Output: [{ text: "Hello world", special: false }]
-```
-
-If the text contained special tokens:
-```
-Input:  "<|im_start|>user\nHello<|im_end|>"
-Output: [
-  { text: "<|im_start|>", special: true },
-  { text: "user\nHello", special: false },
-  { text: "<|im_end|>", special: true },
-]
-```
-
-Special tokens are matched using a regex built from the sorted (longest-first) special token list, ensuring greedy matching.
-
-#### Stage 2: Pre-tokenization
-
-Non-special text is split into chunks using a GPT-style regex:
-
-```
-/'s|'t|'re|'ve|'m|'ll|'d| ?\p{L}+| ?\p{N}+| ?[^\s\p{L}\p{N}]+|\s+/gu
-```
-
-This splits on:
-- Contractions: `'s`, `'t`, `'re`, `'ve`, `'m`, `'ll`, `'d`
-- Words (with optional leading space): ` Hello`, ` world`
-- Numbers (with optional leading space): ` 42`
-- Punctuation (with optional leading space): ` !`
-- Whitespace runs
-
-```
-Input:  "Hello world"
-Output: ["Hello", " world"]
-```
-
-#### Stage 3: Byte-Level Encoding
-
-Each chunk is converted to the byte-level representation used in HF vocabularies. The key transformation is the space-to-`\u0120` (character) mapping:
-
-| Character | Code Point | Representation |
-|-----------|-----------|----------------|
-| Space (` `) | 32 | `\u0120` (latin capital G with dot above) |
-| Newline (`\n`) | 10 | `\u010A` (offset by 256) |
-| Tab (`\t`) | 9 | `\u0109` (offset by 256) |
-| Regular printable | 33-126 | Unchanged |
-
-```
-Input:  ["Hello", " world"]
-Output: ["Hello", "\u0120world"]    (the space becomes the special character)
-```
-
-This is the convention used by GPT-2 and all derivative tokenizers. The `\u0120` character serves as an in-band marker for "this token starts with a space."
-
-#### Stage 4: BPE Merge
-
-Each byte-encoded chunk undergoes iterative pair merging:
-
-1. Start with individual characters: `["\u0120", "w", "o", "r", "l", "d"]`
-2. Find the pair with the lowest merge rank in the merge table
-3. Merge that pair into one token
-4. Repeat until no more merges are possible
-
-```
-"\u0120world" merge trace (hypothetical ranks):
-  Step 0: ["\u0120", "w", "o", "r", "l", "d"]
-  Step 1: ["\u0120w", "o", "r", "l", "d"]        (merge "\u0120" + "w", rank 42)
-  Step 2: ["\u0120w", "or", "l", "d"]             (merge "o" + "r", rank 87)
-  Step 3: ["\u0120w", "orl", "d"]                 (merge "or" + "l", rank 203)
-  Step 4: ["\u0120world"]                          (merge "orl" + "d", then "\u0120w" + "orld")
-```
-
-(Actual merge orders depend on the specific tokenizer's merge table.)
-
-The algorithm always selects the pair with the **lowest rank** (highest priority). This greedy strategy produces the optimal BPE encoding for any given text.
-
-If a complete chunk is already in the vocabulary as a single token, the BPE step is skipped and the chunk maps directly to its token ID.
-
-#### Stage 5: Byte Fallback
-
-If any character or merged symbol isn't in the vocabulary, it's encoded as a sequence of raw byte tokens using the `<0xHH>` format:
-
-```
-Unknown character "ñ" (UTF-8 bytes: 0xC3, 0xB1):
-  -> ["<0xC3>", "<0xB1>"]
-  -> [token_id_for_0xC3, token_id_for_0xB1]
-```
-
-This ensures every possible input can be encoded, even if it contains characters not in the training data.
-
-### Complete Example Result
-
-```
-"Hello world"
-  -> pre-tokenize: ["Hello", " world"]
-  -> byte encode:  ["Hello", "\u0120world"]
-  -> BPE merge:    ["Hello"] -> [token_id_15496]
-                   ["\u0120world"] -> [token_id_1917]
-  -> final IDs:    [15496, 1917]
-```
-
-(Token IDs are model-specific; these are illustrative.)
-
----
-
-## Decoding Pipeline
-
-Decoding reverses the encoding:
-
-1. Map each token ID to its string representation via `vocabReverse`
-2. Optionally skip special tokens (BOS, EOS, etc.)
-3. Join all token strings
-4. Replace `\u0120` back to space
-5. Replace `<0xHH>` patterns back to raw bytes
-
-```typescript
-decode(ids: number[], skipSpecialTokens: boolean = true): string
-```
-
----
-
-## Chat Template Support
-
-The tokenizer implements ChatML format for chat-style conversation encoding:
-
-```
-<|im_start|>system
-You are a helpful assistant.<|im_end|>
-<|im_start|>user
-What is 2+2?<|im_end|>
-<|im_start|>assistant
-```
-
-### API
-
-```typescript
-const tokenizer = Tokenizer.fromJSON(tokenizerJSON, configJSON);
-
-const text = tokenizer.applyChatTemplate([
-  { role: "system", content: "You are a helpful assistant." },
-  { role: "user", content: "What is 2+2?" },
-], { addGenerationPrompt: true });
-
-// Returns the formatted string:
-// "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n
-//  <|im_start|>user\nWhat is 2+2?<|im_end|>\n
-//  <|im_start|>assistant\n"
-
-const ids = tokenizer.encodeChat([
-  { role: "system", content: "You are a helpful assistant." },
-  { role: "user", content: "What is 2+2?" },
-]);
-// Returns the token IDs directly
-```
-
-### Message Types
-
-```typescript
-interface ChatMessage {
-  role: "system" | "user" | "assistant";
-  content: string;
-}
-```
-
-### Limitations
-
-The current implementation hardcodes the ChatML format. HuggingFace tokenizers store a Jinja2 template in `tokenizer_config.json` under the `chat_template` field. A future improvement would parse and evaluate this template to support non-ChatML formats (e.g., Llama's `[INST]...[/INST]` format, Phi's `<|user|>...<|end|>` format).
-
----
-
-## Configuration
-
-The tokenizer reads configuration from `tokenizer_config.json`:
-
-```typescript
-interface TokenizerConfig {
-  bosToken: string | null;      // e.g. "<|endoftext|>"
-  eosToken: string | null;      // e.g. "<|im_end|>" or "<|endoftext|>"
-  bosTokenId: number | null;    // Resolved from vocab
-  eosTokenId: number | null;    // Resolved from vocab
-  chatTemplate: string | null;  // Jinja2 template (stored but not yet parsed)
-  addBosToken: boolean;         // Whether to prepend BOS to encoded text
-  addEosToken: boolean;         // Whether to append EOS to encoded text
-}
-```
-
-When `addBosToken` is true, `encode()` prepends the BOS token ID. When `addEosToken` is true, it appends the EOS token ID. These settings come from the model's tokenizer config.
-
----
-
-## HuggingFace tokenizer.json Format
-
-The tokenizer reads the standard HF `tokenizer.json` format:
-
-```json
-{
-  "model": {
-    "type": "BPE",
-    "vocab": {
-      "Hello": 15496,
-      "\u0120world": 1917,
-      "<0x0A>": 198,
-      ...
-    },
-    "merges": [
-      "\u0120 t",
-      "i n",
-      "e r",
-      ...
-    ]
-  },
-  "added_tokens": [
-    { "id": 151643, "content": "<|endoftext|>", "special": true },
-    { "id": 151644, "content": "<|im_start|>", "special": true },
-    { "id": 151645, "content": "<|im_end|>", "special": true }
-  ]
-}
-```
-
-Key fields:
-- `model.type`: Must be `"BPE"` (the only supported type)
-- `model.vocab`: Complete vocabulary mapping token strings to IDs
-- `model.merges`: Ordered list of merge pairs (index = priority)
-- `added_tokens`: Special tokens with their IDs and flags
-
----
-
-## Performance Characteristics
-
-The BPE algorithm has quadratic worst-case complexity in the length of a single chunk (O(n^2) where n is the number of characters). In practice, this is fast because:
-
-1. Pre-tokenization breaks text into small chunks (typically words)
-2. Most words are 5-15 characters, so the inner merge loop is small
-3. Common words are in the vocabulary directly, skipping BPE entirely
-
-For a typical prompt of 200 words:
-- Pre-tokenization: ~200 chunks
-- BPE per chunk: ~10-15 merge iterations
-- Total: ~3000 operations, well under 1ms
-
-The dominant cost for long prompts is the regex pre-tokenization pass, which is a single linear scan using the built-in regex engine.
-
----
-
-## Vocabulary Size
-
-Common vocabulary sizes for models the engine targets:
-
-| Model Family | Vocab Size | Notable |
-|-------------|-----------|---------|
-| Qwen2/3 | 151,936 | Large vocab with extensive CJK coverage |
-| LLaMA 2 | 32,000 | |
-| LLaMA 3 | 128,256 | Significantly expanded |
-| Phi-3 | 32,064 | |
-| SmolLM2 | 49,152 | |
-
-The vocab size directly affects:
-- LM head matmul cost (hidden_size x vocab_size)
-- Logit readback size (vocab_size * 4 bytes)
-- Sampler sorting cost (O(vocab_size * log(vocab_size)) for top-k)

package/docs/kernel-research-queue.md

@@ -1,85 +0,0 @@
-# Kernel optimization research queue — data-backed from the Gemma-4 challenge corpus
-
-Source of truth: `scripts/research/out/gemma-results.jsonl` (532 experiments harvested
-from the HF `gemma-challenge` bucket; `harvest-gemma-bucket.mjs` re-runs it). 241
-legit rows (ppl 1.5–2.31). Analyzed by a multi-agent workflow that separated
-WebGPU-portable wins from vLLM/CUDA server noise.
-
-**Headline reframe:** the 412-TPS leaderboard top is largely **server-only** —
-its biggest delta is `feopt2` (orjson + HTTP/JSON serialization; no browser
-analog) on a trained vLLM speculative drafter. Stripping serving noise, the
-portable wins attack Gerbil's two confirmed bottlenecks exactly: **dispatch-bound
-decode** and the **270 MB vocab OOM**. IGNORE the 412/506/511 headlines as porting
-targets (server serialization + FA3 + ppl spent past Gerbil's 2.31 guardrail).
-
-## THE CONVERGENCE (do these first — they fix the iOS OOM *and* boost TPS)
-
-**#1 · vocabprune (lm_head row-prune + logits scatter)** — directly-portable, gemma4.
-Prune the LM head 262144 → 32k–128k emittable rows, run the INT4 GEMV/argmax over
-the small head, scatter logits back to full vocab (−inf complement) before sampling.
-*Corpus proof:* `braiam-fable onegraph-pck04-vocabprune` 262144→32768 = **+21.3 TPS
-(311→332) AND ppl improved 2.0266→2.0091**; `dixie pck04b 16k` verify bytes
-**~335MB→~21MB/step**. (16k fails MMLU-Pro 0.596; 128k clears 0.688 → keep ~128k
-for a general assistant; still halves the head.) Pure load-time data transform,
-**no new WGSL**: slice lm_head q/scales/zeros to keep-rows, set `config.vocab_size`
-(argmax `executor.ts:1016` + matmul_int4 N already read it), add scatter-to-full.
-*Test:* vocab ∈ {full, 131072, 32768} × {decode TPS, resident MB, ppl, MMLU-Pro/GSM8K} on M-series + iPad.
-
-**#2 · Extend the prune to embed_tokens + the Gemma-4 PLE table** — directly-portable, gemma4.
-NOT in the corpus (the competition ran on servers with no load-time memory limit;
-they only pruned the head for *speed*). **Gerbil's OOM is at model LOAD**, and the
-Gemma-4 `embed_tokens_per_layer` (PLE) table is **~1.17 GB at 4-bit — the single
-largest resident tensor** (`model-loader.ts:87, ~1916`). Apply the same keep-set
-row-slice to `embed_tokens` + the PLE table so the *resident* weight actually
-shrinks. **Highest-leverage iOS-OOM action.** *Test:* peak resident MB at load with
-embed/PLE pruned to {full,131072,32768} on iPad; confirm no OOM at the risky tier.
-
-**#4 · Untie + INT4 the lm_head** — directly-portable, gemma4. Prerequisite for #1
-(a tied head can't be row-pruned independently). `model-loader.ts:1844-1847`
-currently does the tied fallback (lm_head→embed_tokens). Untie + `quantizeInt4`
-(`quantize.ts:32`, DEFAULT_GROUP_SIZE=128 already matches the corpus g128 sweet spot).
-*Corpus proof:* `foffee int4-lmhead` **118.3 TPS / ppl 2.0067 — 2.68× over bf16**
-(kills the ~1.34GB bf16 head read/step); `senpai int4_g128_lmhead` 126.4 / 2.0057,
-MMLU-Pro 0.668 / GSM8K 0.850.
-
-## Whole-engine, portable (after the OOM convergence)
-
-**#3 · onegraph-analog (fewer submits, no per-token readback on WebKit)** —
-maps-to-webgpu-analog. Gerbil already built the Dawn version: `forwardArgmax`
-(`executor.ts:971-1024`, argmax stays on GPU), `submitGreedyDecodeStep`
-(`executor.ts:1109`, PIPELINE_DEPTH=2, submits without awaiting). **Port that
-no-readback shape to the Safari/iOS path**; raise `webkitGroupSize` toward
-llama.cpp's ~64/CB; no host-JS branch in the hot loop (the `lffn` trap: a host
-`os.path.exists` in the graph cost 420→368 TPS). *Test:* sweep `?group=N` ∈ {1,8,32,64} for correctness + TPS on iPad.
-
-**#5 · More decode-op fusion** — directly-portable. Gerbil already fuses
-(`fuseDualMatVecDecode`/`fuseDualKVCacheAppend`/`fuseGatedOProj`/`fuseDualRMSNorm`,
-`executor.ts:507-517`). Next: fuse final RMSNorm→lm_head, and argmax into the
-lm_head tail. On WebKit each removed dispatch = a removed submit+drain. *Test:*
-diff `[executor] decode: N dispatches/token` + parity per fusion.
-
-**#6 · W4A16 QAT checkpoint** — directly-portable, gemma4. QAT int4 is a quality
-GAIN, not a cost: stock bf16 44.0 / 2.302 → W4A16 QAT **~95.6 / 2.0056** (ppl-guard,
-steve, speed-demon-ne, infinityoptimizer all reproduce). **PREFERRED:** requant
-`google/gemma-4-E4B-it-qat-q4_0-unquantized` through the **existing** `quantizeInt4`
-(g128) — zero new loader code, reuses the safetensors path. **ALT:** add a
-compressed-tensors/pack-quantized adapter like `repackGPTQ` (`gptq-adapter.ts:41`)
-to consume the official `…-qat-w4a16-ct` directly (`model-loader.ts:1074` only knows
-`quant_method=='gptq'` today). *Test:* current post-hoc int4 vs QAT-requant int4 at
-identical g128: ppl + MMLU-Pro/GSM8K (expect QAT lower ppl), TPS unchanged.
-
-**#7 · splitkv (KV layout f16/packed/head-split)** — maps-to-webgpu-analog.
-`deja-vu splitkv-honest-16k` 377.8 / 2.002 (best ppl-2.00 run) but n=1, terse body.
-`executor.ts allocateKVCacheBuffers ~1652` + `DualKVCacheAppend ~1936` + `softmax.wgsl`.
-
-**#8 · Speculative decoding** — DEFER until #1–#3 land. Biggest raw multiplier
-(~2–3.5×) but needs a **second model in memory** (fights the OOM #1/#2 just relieved)
-and a batched verify that depends on the #3 onegraph-analog (per-token submit+drain
-eats the acceptance win). No draft/verify infra exists in `src/gpu` today. Add
-relaxed/typical acceptance (eps=0.5, +~6%) only as an OPT-IN sampler flag (it changes
-emitted tokens; off by default for an honest on-device assistant).
-
-## Sequence
-1 → 4 → 2 (the OOM convergence — load-time data transforms, no new WGSL, fixes iOS
-OOM + boosts TPS) → 6 (QAT requant, parallel) → 3 (WebKit onegraph-analog) → 5
-(fusion) → 7 (KV) → 8 (spec decode, last). Each gated on-device (ppl + MMLU-Pro/GSM8K).