@tryhamster/gerbil 1.1.2 → 1.2.0

package/docs/adding-a-model-family.md

@@ -1,280 +0,0 @@
-# Adding a Model Family to the Gerbil WebGPU Engine
-
-This is the repeatable process for teaching the native WebGPU engine a new model
-architecture. The engine is **not** Qwen-specific — it's a registry of graph
-generators over a family-agnostic IR and kernel library. Adding a family is
-usually "write one generator + register it," and only occasionally "write a new
-kernel."
-
-> TL;DR effort tiers:
-> - **Tier 1 (hours):** the model is Llama-like (standard transformer). Reuse existing ops; write a config→IR generator. Llama, Mistral, Gemma-text, Phi, Qwen all live here.
-> - **Tier 2 (days):** the model has one novel op (a new norm, sliding-window attention, a gate) OR new per-node params on existing kernels. Write the generator + 1–2 WGSL kernels.
-> - **Tier 3 (weeks):** the model has a fundamentally new computation (SSM/Mamba, MoE routing, dual-graph encoder-decoder, codec decoder). New kernels + executor support. Qwen3.5's Mamba-2 path was Tier 3.
-
-> **Repeatable SOP:** for the step-by-step loop use the `add-model-family` skill
-> (`.claude/skills/add-model-family/SKILL.md`). This doc is the reference; the
-> skill is the procedure. Both encode the lessons in "Lessons from production"
-> below — read that section first.
-
----
-
-## Lessons from production (read first)
-
-A wave of families was added against this framework — Qwen3.5 (Mamba-2 SSM + ViT),
-LFM2.5 (hybrid conv/attn), EmbeddingGemma (bidirectional Gemma3 encoder), Moonshine
-STT (CrossAttention + dual-graph), Kani-TTS (LFM2 codec-LM + NanoCodec decoder),
-Gemma 4 E2B (PLE CPU-streaming + proportional RoPE + double-wide MLP + value-norm +
-head_dim-512). What actually mattered:
-
-| Family | Arch string | Tier | What was new |
-|---|---|---|---|
-| Qwen2/3 | `Qwen2/3ForCausalLM` | 1 | baseline standard transformer (the template) |
-| Qwen3-Embedding | `Qwen3ForCausalLM` (embedding flag) | 1 | `SliceLastRow`→`L2Norm` pooling tail, no new kernels |
-| Qwen3.5 | `Qwen3_5ForConditionalGeneration` | 3 | Mamba-2 SSM kernels + ViT encoder (`VisionExecutor`) |
-| LFM2.5 | `Lfm2ForCausalLM` | 2 | short-conv/attention hybrid; `out_proj`→`o_proj` rename |
-| EmbeddingGemma | `Gemma3TextModel`/`Gemma3Model` | 2 | bidirectional encoder (`causal:false`) + mean-pool + dense tail |
-| Moonshine STT | `MoonshineForConditionalGeneration` | 3 | new `CrossAttention` kernel; dual-graph (encoder+decoder) executor |
-| Kani-TTS | `KaniTTS2ForCausalLM` | 3 | LFM2 codec-LM backbone + NanoCodec decoder (FSQ + causal HiFi-GAN) |
-| Gemma 4 E2B | `Gemma4ForConditionalGeneration` | 2 | PLE CPU-streaming, proportional/partial RoPE, value-norm, head_dim-512, double-wide MLP, per-layer scalar — all via per-node attrs + loader hooks |
-
-1. **VERIFY FIRST against the live checkpoint.** Fetch the live `config.json` and
-   range-read the safetensors header (keys/dtypes/shapes) BEFORE writing anything.
-   Confirm the arch string, dims, quant format, and classify every op. Do NOT
-   trust assumptions from sibling models: Gemma4 had no MatFormer though Gemma3n
-   did; OmniVoice's codec is diffusion not AR; MLX repos ship the ViT under a
-   different key prefix (`vision_tower.*`) than HF bf16 (`model.visual.*`), and the
-   LM under `language_model.model.*` (MLX) vs `model.language_model.*` (GPTQ/HF).
-   Watch for **nested config** — Qwen3.5/Gemma4 put the text tower under
-   `text_config`.
-2. **Prefer per-node params over new kernels.** Most "novel" behavior is a new
-   parameter on an existing kernel. Add it to the WGSL `Params` struct and read it
-   in `buildParams` with a **default that reproduces the old behavior
-   byte-identically** (`attn_scale ?? 1/sqrt(head_dim)`), so every existing caller
-   is unchanged. Gemma4's attn_scale=1.0, partial RoPE
-   (`rope_dim/rope_half/rope_denom/rope_active_pairs`), and per-layer `Scale`
-   (loader-patched from `layer_scalar_key`) all landed this way — zero new attention
-   kernels.
-3. **Validate with a LOAD-ONCE reference harness — the make-or-break lesson.** Dump
-   the HF/MLX reference activations to disk ONCE
-   (`{tokens, per_layer_last[L][hidden], argmax, logits_top}`), then iterate
-   engine-only in SECONDS via a single `engine.create()` + `rawForward()` +
-   per-layer `debugReadBuffer` cosine loop (`test-gemma4-perlayer.mjs`). Reloading
-   the 2.5GB model every cycle made Gemma4 cost an hour per iteration. Localize
-   divergence to the exact layer/op. Set `GERBIL_NO_ACT_POOL=1` when reading early
-   activations (the pool aliases buffers). Gate on coherent generation — bit-exact
-   is not the bar for lossy quants.
-4. **No-regression suite is mandatory for shared-kernel changes.** Keep all
-   existing models bit-exact: `test-q4-generate`, `test-vision-e2e`,
-   `test-crossattention`, `test-nanocodec-decode`, the embedding test, and the fast
-   `test-gemma4-perlayer`. The byte-identical-default pattern (lesson 2) is what
-   keeps these green.
-5. **Big tensors: shard or stream.** Per-buffer caps are ~256MB
-   (`maxBufferSize`) / ~128MB (`maxStorageBufferBindingSize`) on iPad. Shard the
-   embedding if vocab×hidden exceeds the cap; for vocab-scale auxiliary tables keep
-   them CPU-resident and stream per-layer slices. Gemma4's ~1.17GB PLE table has
-   **0 MB GPU residency** — the loader builds a `PleSource`, diverts it from the
-   weights map, and the executor uploads only the needed `[T, width]` rows per step.
-6. **Merge additively, never commit node_modules.** Registry/IR/index conflicts are
-   always additive — keep ALL families/ops from both sides. Use targeted `git add`,
-   never `git add -A`.
-
----
-
-## The architecture
-
-When a model loads, `model-loader.ts` reads `config.json` → `architectures[0]`
-(e.g. `"LlamaForCausalLM"`) and looks it up in the registry:
-
-```
-src/gpu/architectures/index.ts   →  ARCHITECTURES: Record<string, GraphGenerator>
-```
-
-A `GraphGenerator` turns the raw HF config into a `ModelGraph` (the IR):
-
-```ts
-type GraphGenerator = (
-  config: Record<string, unknown>,
-  dtype?: "f32" | "q4",      // q4 = on-the-fly INT4 quantization
-  groupSize?: number,        // INT4 group size (default 128)
-  kvDtype?: "f32" | "f16",   // KV cache precision
-) => ModelGraph;
-```
-
-The `ModelGraph` is `{ tensors, nodes, executionOrder, inputs, outputs, config, capabilities, architecture }`.
-The executor (`executor.ts`) consumes it generically: it allocates buffers
-(liveness-pooled), compiles one pipeline per node from `KERNEL_REGISTRY`, and
-dispatches in `executionOrder`. **The generator never touches WebGPU** — it only
-describes the computation.
-
----
-
-## Step-by-step
-
-### 1. Identify the family and check op coverage
-
-Read the model's `config.json` and its modeling code on HuggingFace. Classify
-every layer's ops and check them against what the engine already has
-(`KERNEL_REGISTRY` in `src/gpu/kernels/registry.ts`):
-
-**Implemented ops (reuse freely):** `Embedding`, `EmbeddingInt4`, `MatMul`,
-`MatMulInt4`, `Add`, `Mul`, `Scale` (per-element, loader-patchable scalar),
-`RMSNorm`, `LayerNorm`, `RoPE` (with optional partial-rotary attrs
-`rope_dim/rope_half/rope_denom/rope_active_pairs`), `Attention` (causal *and*
-bidirectional via the `causal` flag — set `causal: false` for an encoder/ViT;
-takes a per-node `attn_scale` defaulting to `1/sqrt(head_dim)`, plus
-`sliding_window`), `CrossAttention` (encoder-decoder), `Softmax`, `SiLU`,
-`SwiGLU`, `GELU` (tanh approx), `GeluErf` (exact erf), `AddBias` (row-broadcast
-bias), `ApplyRotaryEmb` (precomputed-cos/sin `rotate_half`), `SliceCols`
-(column-range extract, e.g. split a fused QKV), `L2Norm` (row-wise, embedding
-tail), `ResidualRMSNorm`, `KVCacheAppend`, `MambaSSM`, `CausalConv1d`,
-`SigmoidGate`, `ConvStateUpdate`, `SliceLastRow`, and codec ops
-(`Conv1d`/`ConvTranspose1d`/`Snake1d`, FSQ) for audio decoders.
-
-If the model only uses these → **Tier 1**. If it needs something new → note it;
-you'll write a kernel in Step 5.
-
-**Embedding models** are Tier 1: an embedding model (e.g. Qwen3-Embedding-0.6B,
-which is `Qwen3ForCausalLM`) is the causal-LM forward pass with a different tail —
-pass the `embedding` flag to `generateQwen2Graph` to swap `lm_head` for the
-`SliceLastRow` (last-token EOS pool) → `L2Norm` tail. No new kernels (paper §21).
-
-**Vision-capable checkpoints** (e.g. Qwen3.5) ship a ViT in the same weights; the
-encoder is a separate graph (`qwen3_5_vision.ts`) run by `VisionExecutor`, already
-built and bit-exact vs HF (paper §22). Patches arrive pre-flattened, so the
-patch-embed is a plain `MatMul` + `AddBias` — no Conv3d kernel.
-
-**Exotic features that need new work (flag early):**
-- Sliding-window / local attention (needs an attention kernel variant)
-- Mixture-of-Experts routing (`MoERouter`/`ExpertMatMul` are stubbed, not built)
-- Novel normalization (e.g. QK-norm, per-head norm) — new kernel
-- MatFormer / elastic parameters (Gemma 3n) — needs slicing logic + a loader story
-- Attention logit soft-capping (Gemma 2/3) — small kernel tweak
-- Non-RoPE position encodings
-
-### 2. Map config → dimensions
-
-Every generator starts by pulling dimensions from the raw config. Copy the block
-from `src/gpu/architectures/qwen2.ts` and adjust names to the new model's config:
-
-```ts
-const hidden_size = rawConfig.hidden_size as number;
-const num_layers = rawConfig.num_hidden_layers as number;
-const num_heads = rawConfig.num_attention_heads as number;
-const num_kv_heads = (rawConfig.num_key_value_heads as number) ?? num_heads;  // GQA
-const intermediate_size = rawConfig.intermediate_size as number;
-const vocab_size = rawConfig.vocab_size as number;
-const context_length = (rawConfig.max_position_embeddings as number) ?? 32768;
-const rms_norm_eps = (rawConfig.rms_norm_eps as number) ?? 1e-6;
-const rope_base = (rawConfig.rope_theta as number) ?? 1_000_000.0;
-const head_dim = (rawConfig.head_dim as number) ?? Math.floor(hidden_size / num_heads);
-const tieWordEmbeddings = (rawConfig.tie_word_embeddings as boolean) ?? false;
-```
-
-Watch for family differences: **head_dim is often NOT hidden_size/num_heads**
-(Gemma sets it explicitly); GQA means `num_kv_heads < num_heads`; some families
-have **QKV projection bias** (Qwen2 does, Llama/Gemma don't).
-
-### 3. Write the generator
-
-Create `src/gpu/architectures/<family>.ts` exporting
-`generate<Family>Graph(...)`. Use the local `addTensor`/`addNode` helpers (they
-also push to `executionOrder`). The per-layer skeleton, following qwen2.ts:
-
-1. **Embedding** — `Embedding`/`EmbeddingInt4` reading `input_ids` + `CANONICAL_KEYS.EMBED`.
-2. **Per layer** (loop `num_layers`):
-   - input RMSNorm → Q/K/V projections (`MatMulInt4`) → RoPE → `KVCacheAppend` → `Attention` → output projection → residual `Add`
-   - post-attn RMSNorm → gate/up projections → `SwiGLU` → down projection → residual `Add`
-   - (use `ResidualRMSNorm` to fuse the residual+norm where the family allows)
-3. **Final norm** (`RMSNorm`).
-4. **`SliceLastRow`** on the final hidden state → **lm_head** (`MatMulInt4`) → `logits`.
-   The `[1, vocab]` logits + SliceLastRow are mandatory (saves ~485MB at long
-   context and skips the full-vocab prefill matmul — see paper §18). Honor
-   `tieWordEmbeddings` (reuse embed weights for lm_head).
-
-Declare intermediate tensors with `storage: "activation"` — the executor pools
-them automatically; don't manage buffers yourself. Set `capabilities`
-(`text`, `vision`, `moe`) and return the graph.
-
-### 4. Register it
-
-In `src/gpu/architectures/index.ts`:
-
-```ts
-import { generateLlamaGraph } from "./llama.js";
-export const ARCHITECTURES = {
-  // ...existing...
-  LlamaForCausalLM: generateLlamaGraph,
-  MistralForCausalLM: generateLlamaGraph,   // Mistral == Llama arch
-};
-```
-
-One generator can serve a whole family — Llama, Mistral, and many fine-tunes
-share `LlamaForCausalLM`.
-
-### 5. Weight key mapping
-
-The loader maps HF safetensors keys → canonical names via a `HFKeyMapper`
-(`createDefaultHFKeyMapper` strips the `model.` prefix). If the new family names
-weights differently, supply a mapper or set `safetensorsKey` on each `addTensor`.
-`CANONICAL_KEYS` has helpers (`qProj(i)`, `layerInputNorm(i)`, `EMBED`,
-`LM_HEAD`, …) — reuse them so the loader and generator agree.
-
-### 6. Write any new kernel (only if Step 1 flagged one)
-
-Add to `src/gpu/kernels/registry.ts`: a WGSL string + a `KernelSpec`
-(`shaderCode`, `entryPoint`, `bindings`, `getDispatchSize`, `buildParams`), then
-register it in `KERNEL_REGISTRY` under your new `OpType` (add the op to the union
-in `ir.ts`). **Mobile constraints (WebKit/iPad):** workgroup memory ≤ 16 KB
-(`maxComputeWorkgroupStorageSize`); no `enable f16` (use packed `pack2x16float`);
-clamp `exp()` for Metal fast-math; avoid `select()` (use if/else). See the
-attention kernels for the safe patterns and the two-phase reduction.
-
-### 7. Validate correctness (do not skip)
-
-1. **Reference activations:** adapt `scripts/engine/test-reference.py` to dump the
-   new model's logits + key intermediate activations from HF transformers.
-2. **Compare:** run `engine.integrityCheck()` (weights + isolated dispatches +
-   pipeline probes) and diff the engine's logits against the reference. Match the
-   argmax and check sums within tolerance.
-3. **Coherence:** generate from a fixed prompt at temperature 0 and confirm
-   sensible text (the `scripts/engine/test-q4-generate.mjs` pattern).
-4. **Cross-platform:** the desktop (Dawn) and mobile (WebKit) outputs must be
-   byte-identical — queue an iPad run via the harness (`scripts/engine/ipad-queue.json`).
-
-### 8. Mobile / download budget checks
-
-- **Download size at q4** is the headline mobile metric — report it.
-- **Per-buffer limit:** iPad `maxBufferSize` defaults to 256MB and
-  `maxStorageBufferBindingSize` to 128MB. A single weight tensor (often the
-  embedding) must fit — shard it if the model's vocab×hidden exceeds the cap.
-- **maxSeqLen policy:** the engine clamps iOS to 512 by default; large contexts
-  multiply KV-cache memory.
-
----
-
-## Worked example: the smallest possible new family (Llama)
-
-Llama is Qwen2 minus QKV bias. The fastest path:
-1. Copy `qwen2.ts` → `llama.ts`, rename the export.
-2. Remove the QKV-bias tensors/handling (Llama has none).
-3. Confirm head_dim, GQA (`num_key_value_heads`), and `tie_word_embeddings` from config.
-4. Register `LlamaForCausalLM`/`MistralForCausalLM` → `generateLlamaGraph`.
-5. Validate against a reference (Step 7). No new kernels needed — Tier 1.
-
-This single generator unlocks Llama, Mistral, and every model that ships as
-`LlamaForCausalLM` on HuggingFace.
-
----
-
-## Checklist
-
-- [ ] Classified every op; new ops listed (or confirmed none)
-- [ ] `generate<Family>Graph` written, dimensions mapped from config
-- [ ] GQA / head_dim / QKV-bias / tied-embeddings handled
-- [ ] `SliceLastRow` + `[1, vocab]` logits in place
-- [ ] Registered in `ARCHITECTURES`
-- [ ] Key mapping verified (loader finds every weight)
-- [ ] New kernels (if any) respect the 16 KB / no-`enable f16` / clamped-`exp` mobile rules
-- [ ] Validated vs HF reference (logits argmax + coherence)
-- [ ] Desktop and iPad outputs byte-identical
-- [ ] Download size at q4 reported; largest weight fits the 256MB/128MB buffer caps

package/docs/ai-sdk.md

@@ -1,205 +0,0 @@
-# Gerbil + AI SDK
-
-Gerbil works as a [Vercel AI SDK v5](https://sdk.vercel.ai/) provider, supporting text generation, embeddings, speech synthesis (TTS), and transcription (STT).
-
-> **Pre-1.0 note.** The AI SDK provider routes through the `Gerbil` class, which now runs on
-> the native WebGPU engine (no ONNX / transformers.js). TTS uses Kani-TTS-2, STT uses
-> Moonshine, and embeddings use EmbeddingGemma regardless of the model-id string you pass —
-> legacy ids like `kokoro-82m` / `whisper-tiny.en` are vestigial labels and the device must
-> have WebGPU (there is no CPU/WASM fallback). The first-class surface for the engine is
-> `WebGPUEngine` / `useEngine` (see the [README](../README.md), [TTS](./tts.md),
-> [STT](./stt.md), [Embeddings](./embeddings.md) docs).
-
-## Setup
-
-```typescript
-import { generateText, streamText } from "ai";
-import { gerbil } from "@tryhamster/gerbil/ai";
-```
-
-## Text Generation
-
-### Generate Text
-
-```typescript
-const { text } = await generateText({
-  model: gerbil("qwen3-0.6b"),
-  prompt: "Write a commit message for adding dark mode",
-});
-```
-
-### Stream Text
-
-```typescript
-const stream = streamText({
-  model: gerbil("qwen3-0.6b"),
-  prompt: "Explain React hooks",
-});
-
-for await (const chunk of stream.textStream) {
-  process.stdout.write(chunk);
-}
-```
-
-### With System Prompt
-
-```typescript
-const { text } = await generateText({
-  model: gerbil("qwen3-0.6b"),
-  system: "You are a helpful coding assistant.",
-  prompt: "How do I handle errors in async/await?",
-});
-```
-
-### Thinking Mode
-
-```typescript
-import { createGerbil } from "@tryhamster/gerbil/ai";
-
-const local = createGerbil({ device: "gpu" });
-
-const { text } = await generateText({
-  model: local("qwen3-0.6b", { thinking: true }),
-  prompt: "What is 127 × 43?",
-});
-```
-
-## Embeddings
-
-> **Native.** `gerbil.embedding()` runs native EmbeddingGemma-300M on the WebGPU engine
-> (768-dim; the `all-MiniLM-L6-v2` default id is a vestigial label — the old ONNX
-> MiniLM/BGE/GTE lane has been removed). For direct control use `engine.embed()` — see
-> [Embeddings docs](./embeddings.md). Requires WebGPU.
-
-Generate text embeddings for semantic search, similarity, and RAG:
-
-```typescript
-import { embed, embedMany } from "ai";
-import { gerbil } from "@tryhamster/gerbil/ai";
-
-// Single embedding (768-dim EmbeddingGemma vector)
-const { embedding } = await embed({
-  model: gerbil.embedding(),
-  value: "Hello world",
-});
-
-// Multiple embeddings
-const { embeddings } = await embedMany({
-  model: gerbil.embedding(),
-  values: ["Hello", "World", "How are you?"],
-});
-```
-
-## Speech Generation (TTS)
-
-> **Native.** `gerbil.speech()` runs Kani-TTS-2 on the native engine (the `kokoro-82m`
-> default id is a vestigial label). For direct control use `engine.speak()` — see
-> [TTS docs](./tts.md). Requires WebGPU.
-
-Generate speech from text:
-
-```typescript
-import { experimental_generateSpeech as generateSpeech } from "ai";
-import { gerbil } from "@tryhamster/gerbil/ai";
-
-const result = await generateSpeech({
-  model: gerbil.speech(),        // native Kani-TTS-2
-  text: "Hello, welcome to Gerbil!",
-});
-
-// result.audio is the synthesized PCM clip
-await writeFile("output.wav", result.audio);
-```
-
-> Voice/speed selection from the old Kokoro lane no longer applies — Kani-TTS-2 uses its own
-> default voice. For full control over the native speech path, use `engine.speak()` directly
-> (see [TTS docs](./tts.md)).
-
-## Transcription (STT)
-
-> **Native.** `gerbil.transcription()` runs Moonshine on the native engine (the
-> `whisper-tiny.en` default id is a vestigial label; English only). For direct control use
-> `MoonshineSTT` — see [STT docs](./stt.md). Requires WebGPU.
-
-Transcribe audio to text:
-
-```typescript
-import { experimental_transcribe as transcribe } from "ai";
-import { gerbil } from "@tryhamster/gerbil/ai";
-import { readFile } from "fs/promises";
-
-const result = await transcribe({
-  model: gerbil.transcription(),   // native Moonshine (English)
-  audio: await readFile("audio.wav"),
-});
-
-console.log(result.text);              // "Hello world"
-console.log(result.language);          // "en"
-```
-
-> The Whisper model family (multilingual variants, timestamped segments) has been removed.
-> The native path is Moonshine, English-only, and does not produce timestamps. For full
-> control use `MoonshineSTT` directly (see [STT docs](./stt.md)).
-
-## Custom Provider
-
-```typescript
-import { createGerbil } from "@tryhamster/gerbil/ai";
-
-const local = createGerbil({
-  device: "gpu",
-  dtype: "q4",
-});
-
-// Text generation
-const { text } = await generateText({
-  model: local("qwen3-0.6b"),
-  prompt: "Hello",
-});
-
-// Speech
-const speech = await generateSpeech({
-  model: local.speech(),
-  text: "Hello",
-});
-
-// Transcription
-const transcript = await transcribe({
-  model: local.transcription(),
-  audio: audioData,
-});
-```
-
-## Model Preloading
-
-Download models ahead of time via the provider:
-
-```typescript
-import { gerbil } from "@tryhamster/gerbil/ai";
-
-// Check if cached
-if (!(await gerbil.isCached("qwen3-0.6b"))) {
-  // Preload during app init
-  await gerbil.preload("qwen3-0.6b", {
-    onProgress: (p) => console.log(p.status, p.progress),
-  });
-}
-
-// Later: generateText loads instantly from cache
-```
-
-## Specification
-
-Gerbil implements the following AI SDK v5 interfaces:
-
-| Interface | Purpose | Method |
-|-----------|---------|--------|
-| `LanguageModelV2` | Text generation | `gerbil(modelId)` |
-| `SpeechModelV2` | Text-to-Speech | `gerbil.speech()` |
-| `TranscriptionModelV2` | Speech-to-Text | `gerbil.transcription()` |
-
-All models support:
-- `specificationVersion: "v2"`
-- Proper warning reporting
-- Request/response metadata
-

package/docs/architecture/README.md

@@ -1,84 +0,0 @@
-# Gerbil Architecture
-
-Technical deep-dive into how Gerbil works under the hood.
-
-## Contents
-
-| Document | Description |
-|----------|-------------|
-| [Overview](./overview.md) | High-level architecture and design decisions |
-| [Inference Pipeline](./inference.md) | ONNX Runtime, transformers.js, quantization |
-| [WebGPU](./webgpu.md) | GPU acceleration in browser and Node.js |
-| [Streaming](./streaming.md) | Web Worker architecture and token streaming |
-| [Caching](./caching.md) | Model caching strategies |
-
-## Quick Overview
-
-```
-┌───────────────────────────────────────────────────────────────────┐
-│                         Your Application                          │
-├───────────────────────────────────────────────────────────────────┤
-│                                                                   │
-│  ┌─────────────┐    ┌─────────────┐    ┌─────────────────────┐    │
-│  │   Browser   │    │   Node.js   │    │      CLI/REPL       │    │
-│  │  (WebGPU)   │    │ (CPU/WebGPU)│    │    (CPU/WebGPU)     │    │
-│  └──────┬──────┘    └──────┬──────┘    └──────────┬──────────┘    │
-│         │                  │                       │              │
-│         ▼                  ▼                       ▼              │
-│  ┌─────────────────────────────────────────────────────────────┐  │
-│  │                     Gerbil Core                             │  │
-│  │  ┌─────────┐  ┌──────────┐  ┌────────┐  ┌───────────────┐   │  │
-│  │  │ Models  │  │ Generate │  │ Stream │  │ JSON/Embed    │   │  │
-│  │  └────┬────┘  └────┬─────┘  └───┬────┘  └───────┬───────┘   │  │
-│  └───────┼────────────┼────────────┼───────────────┼───────────┘  │
-│          │            │            │               │              │
-│          ▼            ▼            ▼               ▼              │
-│  ┌─────────────────────────────────────────────────────────────┐  │
-│  │                   transformers.js                           │  │
-│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │  │
-│  │  │  Tokenizer  │  │    Model    │  │   TextStreamer      │  │  │
-│  │  └─────────────┘  └──────┬──────┘  └─────────────────────┘  │  │
-│  └──────────────────────────┼──────────────────────────────────┘  │
-│                             │                                     │
-│                             ▼                                     │
-│  ┌─────────────────────────────────────────────────────────────┐  │
-│  │                    ONNX Runtime                             │  │
-│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │  │
-│  │  │   WebGPU    │  │     CPU     │  │       WASM          │  │  │
-│  │  │  (Browser)  │  │   (Node.js) │  │   (Fallback)        │  │  │
-│  │  └─────────────┘  └─────────────┘  └─────────────────────┘  │  │
-│  └─────────────────────────────────────────────────────────────┘  │
-│                                                                   │
-└───────────────────────────────────────────────────────────────────┘
-```
-
-## Key Design Decisions
-
-### 1. transformers.js as the Foundation
-
-We use [Hugging Face transformers.js](https://huggingface.co/docs/transformers.js) which provides:
-- Pre-converted ONNX models from Hugging Face Hub
-- Tokenizers that match the original models exactly
-- Unified API across CPU/GPU/WASM backends
-
-### 2. WebGPU First
-
-WebGPU provides 5-10x speedup over CPU for inference:
-- **Browser**: Native WebGPU via `navigator.gpu`
-- **Node.js**: Headless Chrome as a WebGPU accelerator (ChromeGPUBackend)
-
-### 3. Quantization for Speed
-
-All models use quantized weights:
-- **q4f16**: 4-bit weights, fp16 compute (WebGPU)
-- **q4**: 4-bit weights, fp32 compute (CPU/WASM)
-
-This reduces model size by ~4x and improves inference speed.
-
-### 4. Streaming via Web Workers
-
-Browser inference runs in a Web Worker to:
-- Keep the UI responsive during model loading
-- Stream tokens in real-time without blocking
-- Isolate GPU memory from the main thread
-