volvoxai 0.1.0

package/docs/textbook/03-efficientdet-vision-model.md

@@ -0,0 +1,281 @@
+# Chapter 3 — A Vision Model, op by op (EfficientDet-Lite0)
+
+*Goal: follow a **320×320 photo** through an object detector until it outputs labeled boxes
+("dog at (x0,y0,x1,y1)"). Different ops than Chapter 2 — but the same idea: a graph of small
+kernels on tensors.*
+
+The model lives in `models/efficientdet_lite0_*/`. **EfficientDet-Lite0** is a compact
+[object detector](https://arxiv.org/abs/1911.09070): given an image, it finds *what* objects are
+present and *where*. It ships here in three numeric precisions — **fp32, fp16, int8** — that
+compute the same thing at different size/speed trade-offs. This chapter traces the **fp32**
+version (ops named `Conv2D`); Chapter 4 explains how int8 swaps in `QConv2D`.
+
+## 3.1 What the model consumes and produces
+
+```
+INPUT   input0 : shape [1, 320, 320, 3]   one 320×320 RGB image (NHWC)
+        (int8 variant takes uint8 pixels 0–255; fp32 takes normalized floats)
+
+OUTPUT  scores : shape [1, 19206, 90]     for each of 19206 candidate boxes, 90 class scores
+        boxes  : shape [1, 19206, 4]       for each candidate box, 4 coordinates
+```
+
+The network proposes **19,206 candidate boxes** covering the image at many positions and sizes,
+scores each against **90 object classes** (the COCO label set — person, car, dog, …), and you
+keep the few confident, non-overlapping ones. Where does 19,206 come from? Five detection grids
+of decreasing resolution, 9 candidate boxes ("anchors") per cell:
+
+```
+grid 40×40 × 9 = 14400   (finds small objects)
+grid 20×20 × 9 =  3600
+grid 10×10 × 9 =   900
+grid  5× 5 × 9 =   225
+grid  3× 3 × 9 =    81   (finds big objects)
+                 ─────
+          total  19206
+```
+
+The whole graph is **262 nodes** (265 for int8). Inventory:
+
+```
+182 Conv2D   42 Add   14 MaxPool2D   12 ResizeNearest2D   10 Reshape   2 Concat
+   └─ of the convs: 102 are "pointwise/regular", 80 are "depthwise" (see §3.3)
+```
+
+## 3.2 The pipeline at a glance
+
+```mermaid
+flowchart TD
+    IMG["image [1,320,320,3]"] --> BB
+    subgraph BB[1 · Backbone · EfficientNet-Lite0]
+      direction TB
+      STEM[stem Conv2D<br/>320→160, 3→32 ch] --> MB[16 × MBConv blocks<br/>depthwise + pointwise + residual]
+    end
+    BB -->|features at 5 scales<br/>40,20,10,5,3| FPN
+    subgraph FPN[2 · BiFPN · multi-scale feature fusion]
+      direction TB
+      FUSE["Resize ↑ / MaxPool ↓ then weighted Add<br/>(repeated a few times)"]
+    end
+    FPN --> HEADS
+    subgraph HEADS[3 · Detection heads]
+      direction TB
+      CLS[class head Conv2D<br/>→ 90 scores per anchor]
+      BOX[box head Conv2D<br/>→ 4 coords per anchor]
+    end
+    HEADS --> DEC[Reshape + Concat<br/>flatten 5 grids into one list]
+    DEC --> OUT["scores [1,19206,90]<br/>boxes [1,19206,4]"]
+    OUT --> POST[Postprocess<br/>sigmoid · decode vs anchors · NMS]
+    POST --> RES["final: few labeled boxes"]
+```
+
+Three learned stages (backbone → BiFPN → heads) produce raw numbers; a fixed postprocess turns
+them into boxes you can draw. Let's take them in order — but first, the one op that dominates:
+**convolution**.
+
+---
+
+## 3.3 The core op: convolution (`Conv2D`)
+
+Where the language model leans on `MatMul`, a vision model leans on `Conv2D`. A convolution
+slides a small **filter** (a little grid of weights, e.g. 3×3) across the image. At each
+position it multiplies the filter by the pixels underneath and sums — a **dot product** — to
+produce one output value. Slide it everywhere and you get a new image ("feature map") that lights
+up wherever that filter's pattern (an edge, a texture, an eye) appears.
+
+```
+   input patch        filter (3×3)      one output pixel
+   ┌──────────┐       ┌──────────┐
+   │ a  b  c  │       │ w1 w2 w3 │      out = a·w1 + b·w2 + c·w3
+   │ d  e  f  │   ⊙   │ w4 w5 w6 │  =       + d·w4 + e·w5 + f·w6
+   │ g  h  i  │       │ w7 w8 w9 │          + g·w7 + h·w8 + i·w9
+   └──────────┘       └──────────┘        (then slide right by `stride` and repeat)
+```
+
+The real reference kernel (`js/ops/conv2D.js`) is just those slides written as nested loops —
+batch, output-row, output-col, output-channel, then the filter taps:
+
+```javascript
+for (let oh = 0; oh < out_h; oh++)
+ for (let ow = 0; ow < out_w; ow++)
+  for (let oc = 0; oc < out_c; oc++) {
+    let sum = 0;
+    for (let ic = 0; ic < in_c; ic++)          // over input channels
+     for (let kh = 0; kh < k_h; kh++)          // over filter height
+      for (let kw = 0; kw < k_w; kw++) {        // over filter width
+        const ih = oh*stride_y + kh*dil_y - pad; // which input pixel
+        const iw = ow*stride_x + kw*dil_x - pad;
+        if (in-bounds) sum += input[…ih,iw,ic…] * weight[…kh,kw,ic,oc…];
+      }
+    if (bias) sum += bias[oc];
+    if (relu) sum = clamp(sum, 0, 6);           // fused ReLU6 activation
+    output[…oh,ow,oc…] = sum;
+  }
+```
+
+Two flavors of convolution appear, and their combination is the whole efficiency trick of this
+model family:
+
+| | **Pointwise** (1×1) | **Depthwise** (3×3, `groups = channels`) |
+|---|---|---|
+| Filter | 1×1, mixes **channels** | 3×3, mixes **space**, each channel separately |
+| Job | "recombine features" | "look at local patterns" |
+| Cost | cheap per pixel, but all-to-all channels | very cheap — no channel mixing |
+
+A regular conv does both at once (expensive). **Depthwise-separable** convolution splits it into
+a depthwise (spatial) + pointwise (channel) pair that costs a fraction as much for nearly the
+same power. That pairing is the `MBConv` block, the backbone's Lego brick — and why 80 of the
+182 convs are depthwise.
+
+> **`groups`** in the code: `groups = in_c` means "each channel is convolved by its own filter"
+> (depthwise). `groups = 1` means "every output channel sees every input channel" (regular). The
+> same kernel handles both by looping over the right channel range.
+
+---
+
+## 3.4 Stage 1 — Backbone: image → features
+
+The **backbone** (an *EfficientNet-Lite0*) is a feature extractor. It repeatedly:
+
+1. **Shrinks** the spatial size (via stride-2 convs and pooling) — 320→160→80→40→20→10→…
+2. **Grows** the channel count — 3→32→…→320+ — trading "where" for "what."
+
+Early layers detect edges and colors; middle layers detect textures and parts (an eye, a wheel);
+late layers detect whole objects. This hierarchy is *learned*, not programmed.
+
+```
+stem:      [1,320,320,  3]  --Conv2D stride2-->  [1,160,160, 32]
+MBConv ×16: … depthwise + pointwise + residual Add …   (channels grow, size shrinks)
+outputs 5 feature maps at strides 8,16,32,64,128:
+   P3 [1,40,40,C]   P4 [1,20,20,C]   P5 [1,10,10,C]   P6 [1,5,5,C]   P7 [1,3,3,C]
+```
+
+The **residual `Add`** inside each MBConv is the same trick as the transformer: add the block's
+output back onto its input so deep stacks stay trainable. Same idea, different domain.
+
+Why five feature maps instead of one? **Scale.** A 40×40 map has fine detail (good for small
+objects); a 3×3 map sees huge receptive fields (good for big objects). Detecting at multiple
+scales is how one network finds both a distant bird and a close-up bus.
+
+---
+
+## 3.5 Stage 2 — BiFPN: mix the scales together
+
+A small feature map knows *"there's an object here"* but is spatially coarse; a large one is
+precise but semantically shallow. The **BiFPN** (Bi-directional Feature Pyramid Network) lets the
+five scales exchange information, top-down and bottom-up, so every scale gets both fine detail
+and high-level meaning. It uses exactly three ops you already understand:
+
+```mermaid
+flowchart TB
+    P7b[P7 3×3] -->|Resize ↑| u6
+    P6b[P6 5×5] --> u6((weighted Add)) -->|Resize ↑| u5
+    P5b[P5 10×10] --> u5((weighted Add)) -->|Resize ↑| u4
+    P4b[P4 20×20] --> u4((weighted Add)) -->|Resize ↑| u3
+    P3b[P3 40×40] --> u3((weighted Add))
+    u3 -->|MaxPool ↓| d4((weighted Add))
+    u4 --> d4 -->|MaxPool ↓| d5((weighted Add))
+    u5 --> d5 -->|MaxPool ↓| d6
+    style u6 fill:#eef
+```
+
+- **`ResizeNearest2D`** upsamples a small map to a bigger one (top-down path). *12 of these.*
+- **`MaxPool2D`** downsamples a big map to a smaller one (bottom-up path). *14 of these.* The
+  kernel (`js/ops/maxPool2D.js`) just keeps the max value in each window.
+- **`Add`** (often *weighted* — learnable importance per input) fuses two aligned maps. *42
+  Adds* across the model do this fusion and the backbone residuals.
+
+That's it — BiFPN is "resize until two maps are the same size, then add them," repeated. No new
+math.
+
+---
+
+## 3.6 Stage 3 — Heads: features → per-anchor predictions
+
+Two small conv stacks (shared across the five scales) read the fused features and, at **every**
+grid cell, output predictions for that cell's **9 anchors** (9 reference box shapes of different
+sizes/aspect ratios centered on the cell):
+
+- **Class head** → `90` numbers per anchor: a raw score for each object class.
+- **Box head** → `4` numbers per anchor: adjustments (dx, dy, dw, dh) to the anchor's position
+  and size.
+
+> **Anchors** are the clever bit. Rather than predict boxes from nothing, the model predicts
+> small *corrections* to a fixed grid of prior boxes. Predicting "shift this reference box a bit"
+> is far easier to learn than "invent a box at (173, 92, 240, 210) from scratch."
+
+---
+
+## 3.7 Stage 4 — Reshape + Concat: flatten five grids into one list
+
+Each of the 5 scales produced predictions in its own grid shape. `Reshape` flattens each grid to
+a plain list of anchors, and `Concat` stacks all five lists into one (this is the tail of the
+graph — see the real node shapes):
+
+```
+class predictions:  [1,40,40,…] → [1,14400,90]  ┐
+                    [1,20,20,…] → [1, 3600,90]  │
+                    [1,10,10,…] → [1,  900,90]  ├─Concat─▶ scores [1,19206,90]
+                    [1, 5, 5,…] → [1,  225,90]  │
+                    [1, 3, 3,…] → [1,   81,90]  ┘
+box predictions:    … same five grids …        ─Concat─▶ boxes  [1,19206,4]
+```
+
+`Reshape` doesn't move numbers around in memory at all — it just reinterprets the same flat
+buffer with a new shape (recall §1.2). In this repo it's a copy-through op. **The network's job
+is now done:** two tensors, 19,206 scored candidate boxes.
+
+---
+
+## 3.8 Stage 5 — Postprocess: 19,206 candidates → a few boxes
+
+The raw outputs aren't drawable yet. Three fixed (non-learned) steps finish the job:
+
+1. **Sigmoid** the class scores → probabilities in `[0,1]`. (In the int8 export this `Sigmoid`
+   was folded away for speed, so `scores` are raw logits; apply sigmoid yourself, or just compare
+   them — bigger is still more confident.)
+2. **Decode boxes**: turn each anchor's 4 deltas into real pixel corners `(x0,y0,x1,y1)` by
+   applying them to that anchor's reference box.
+3. **Non-Max Suppression (NMS)**: the same object usually fires several overlapping anchors. NMS
+   keeps the highest-scoring box and deletes others that overlap it too much (high *IoU*,
+   intersection-over-union), per class. VolvoxAI has this as an op — `js/ops/nonMaxSuppression.js`.
+
+```
+before NMS:  ▢▢▢  three overlapping "dog" boxes, scores 0.91, 0.88, 0.72
+after  NMS:  ▢    keep the 0.91; suppress the two that overlap it > 50%
+```
+
+The native `detect` command (`native/main.c`, `print_detections`) uses a **simplified** version
+of step 3 for the demo: it takes the top-`max_det` boxes by their best class score and prints
+them as a ranked table (label lookup via `labels.txt`):
+
+```
+rank  index  score   class  x0     y0     x1     y1
+1     4213   0.91    17     0.31   0.44   0.62   0.88     ← "dog"
+2     991    0.86     2     0.05   0.10   0.40   0.95     ← "bicycle"
+```
+
+Draw those rectangles on the original photo and you have object detection.
+
+---
+
+## 3.9 The two models, side by side
+
+You've now traced both worlds. Notice how much they share:
+
+| | TinyStories (language) | EfficientDet-Lite0 (vision) |
+|---|---|---|
+| Input tensor | tokens `[1,256]` | image `[1,320,320,3]` |
+| Dominant op | `MatMul` | `Conv2D` |
+| "Mixing" mechanism | attention (`SDPA`) across tokens | convolution across pixels |
+| Nonlinearity | `GELU` | `ReLU6` (fused into conv) |
+| Deep-stack trick | residual `Add` + `LayerNorm` | residual `Add` (in MBConv) |
+| Output | logits `[…,50257]` → next word | scores/boxes `[…,19206,…]` → objects |
+| Postprocess | argmax / sampling | sigmoid + decode + NMS |
+
+Same skeleton — *a graph of small tensor ops with learned weights* — solving text and pixels.
+That transfer is the whole point: learn the skeleton once and every model becomes readable.
+
+The last big question is the one the three EfficientDet folders raise: **fp32 vs fp16 vs int8.**
+What are those, and why ship the same model three times? That's Chapter 4.
+
+**Next:** [Chapter 4 — Precision & Quantization →](04-precision-and-quantization.md)

package/docs/textbook/04-precision-and-quantization.md

@@ -0,0 +1,208 @@
+# Chapter 4 — Precision & Quantization (fp32 / fp16 / int8)
+
+*Goal: understand why the same detector ships in three folders, how numbers are stored as bits,
+and the exact integer math that makes the int8 model 4× smaller — with almost no accuracy loss.*
+
+Look at the three EfficientDet folders. Same architecture (Chapter 3), same 262-ish nodes,
+**very different file sizes**:
+
+| Folder | Weights store each number as… | `model.safetensors` | Relative |
+|---|---|---|---|
+| `efficientdet_lite0_fp32` | 32-bit float | **12.67 MB** | 1.0× |
+| `efficientdet_lite0_fp16` | 16-bit float | **6.34 MB** | 0.50× |
+| `efficientdet_lite0_int8` | 8-bit integer | **3.39 MB** | 0.27× |
+
+The *model* is identical. Only the **number format** — the **precision** — changed. Smaller
+numbers → smaller downloads, less memory, and (with the right hardware) faster math. This chapter
+is about that trade.
+
+---
+
+## 4.1 How a computer stores a number
+
+A neural net weight like `0.10125` has to become a fixed pattern of bits. There are two families.
+
+### Floating-point (fp32, fp16): "scientific notation in binary"
+
+A float splits its bits into a **sign**, an **exponent** (how big), and a **mantissa** (the
+precise digits). More bits → more precision and more range.
+
+```
+fp32  (4 bytes)  [S][ 8-bit exponent ][      23-bit mantissa      ]   ~7 decimal digits
+fp16  (2 bytes)  [S][ 5-bit exp ][   10-bit mantissa   ]              ~3 decimal digits
+```
+
+- **fp32** ("single precision") is the default everywhere — huge range, ~7 digits of precision.
+  It's what training uses and what VolvoxAI uses for activations.
+- **fp16** ("half precision") halves the storage. Plenty precise for inference, but its range is
+  smaller (big/small values can overflow/underflow), and — crucially — a GPU must *support*
+  fp16 math to get a speedup. (See §4.5 for why Volvox is cautious with it in browsers.)
+
+### Integer (int8): "a ruler with 256 evenly-spaced ticks"
+
+An **int8** stores a whole number from **−128 to 127** — just 256 possible values, 1 byte. That's
+far too coarse to hold `0.10125` directly. The trick — **quantization** — is to store a cheap
+integer plus a *recipe* for turning it back into a float.
+
+---
+
+## 4.2 Quantization: the scale + zero-point recipe
+
+Real weights in a layer live in some range, say `−0.4 … +0.4`. Quantization stretches the int8
+ruler (`−128 … 127`) across that range with two numbers:
+
+- **`scale`** — the size of one tick (real units per integer step).
+- **`zero_point`** — which integer represents real `0.0`.
+
+```
+real value  r  ≈  (q − zero_point) × scale          ← DEQUANTIZE (int → float)
+integer     q  =  round(r / scale) + zero_point      ← QUANTIZE   (float → int)
+
+  real:  -0.4         -0.2          0.0          0.2          0.4
+          │            │            │            │            │
+  int8: -128         -64            0           64          127     (scale ≈ 0.4/127)
+```
+
+Both formulas are *in the codebase, verbatim*. Dequantize (`js/ops/dequantizeLinear.js`):
+
+```javascript
+out[i] = (in[i] - zero_point) * scale;     // int8 → float
+```
+
+Quantize (`native/quant_cpu_opt.c`, `quantize_scalar_i8`):
+
+```c
+q = clamp_i8( lrintf(x / scale) + zero_point );   // float → int8, clamped to [-128,127]
+```
+
+That's the whole idea. An int8 weight is a *ticket*; `scale` and `zero_point` tell you what it's
+worth. Storing the ticket costs 1 byte; the recipe is shared across a whole channel, so it's
+nearly free.
+
+> **Per-channel scales.** A single scale for a whole layer would be crude — one big weight would
+> stretch the ruler and crush everyone else's precision. So each output **channel** gets its own
+> `weight_scale` (you saw `weight_scale: "w1"` as a *tensor* input to `QConv2D`). This
+> "per-channel quantization" is what keeps int8 accuracy within ~1% of fp32.
+
+---
+
+## 4.3 A worked example (real numbers from the model)
+
+The int8 EfficientDet's first node is `QuantizeLinear` with `input_scale = 0.0078125` (which is
+exactly `1/128`) — it turns incoming pixels into int8. Then the first `QConv2D` has
+`output_scale = 0.0235294`, `output_zero_point = -128`. Let's quantize one weight and one output.
+
+```
+Quantize a weight  r = 0.101,  scale = 0.008,  zero_point = 0:
+   q = round(0.101 / 0.008) + 0 = round(12.625) = 13        → stored as the byte 13
+
+Later, recover it:
+   r ≈ (13 - 0) × 0.008 = 0.104                              → 0.104 vs 0.101, error 0.003
+```
+
+The 0.003 error is **quantization noise**. Spread across a big dot product, these tiny rounding
+errors mostly cancel — which is why an 8-bit model still detects dogs correctly.
+
+---
+
+## 4.4 How int8 convolution actually runs (`QConv2D`)
+
+Here's the payoff. A quantized conv does its heavy multiply-accumulate loop in **cheap integer
+arithmetic**, and only converts back to a real number once at the very end. The pipeline for one
+output value (`native/quant_cpu_opt.c`):
+
+```mermaid
+flowchart LR
+    A["int8 inputs<br/>(−128…127)"] --> B["int32 accumulate<br/>Σ (in_q − in_zp) × w_q<br/>(all integer math)"]
+    B --> C["to float:<br/>v = acc × in_scale × w_scale + bias"]
+    C --> D["ReLU6 clamp<br/>(fused activation)"]
+    D --> E["requantize:<br/>q = round(v / out_scale) + out_zp"]
+    E --> F["int8 output<br/>feeds next layer"]
+```
+
+1. **Integer accumulate.** Multiply int8×int8, sum into an **int32** accumulator. Integer
+   multiply-add is fast and cheap on every CPU (and vectorizes 8–32 lanes wide with AVX2/NEON).
+2. **Requantize.** Convert the int32 sum back to the layer's int8 scale in one step. The real
+   code composes all the scales into one multiply:
+
+   ```c
+   // acc (int32) → float → relu6 → int8, for one output element:
+   float v = acc * (input_scale * weight_scale) + bias;   // combined rescale
+   int8   q = requantize_i8(v, output_scale, output_zp, relu);
+   //        = clamp_i8( round( relu6(v) / output_scale ) + output_zp );
+   ```
+
+The key insight: **activations stay int8 from layer to layer** ("a quantized island"), so the
+whole backbone runs in bytes. Only at the very end does `DequantizeLinear` turn the final
+`scores`/`boxes` back into floats you can read. This is exactly what VolvoxAI's native CPU path
+does (`native/quant_cpu_opt.c`); the browser tiers instead fold int8 conv weights back to fp32 at
+load time (simpler, still small on disk).
+
+---
+
+## 4.5 Why each format exists — the trade-off
+
+```
+        SMALLER / FASTER  ◀───────────────────────────────▶  MORE ACCURATE / SIMPLER
+             int8                      fp16                      fp32
+        1 byte/weight              2 bytes/weight            4 bytes/weight
+        integer math              needs fp16 HW             works everywhere
+        ~4× smaller               ~2× smaller               reference accuracy
+        tiny accuracy dip         ~lossless                 lossless
+```
+
+| Question | fp32 | fp16 | int8 |
+|---|---|---|---|
+| Disk / memory | biggest | half | quarter |
+| Accuracy vs fp32 | reference | ~identical | usually within ~1% |
+| Needs special hardware? | no | **yes** (fp16 units) | no (integer is universal) |
+| Best when… | max accuracy, or you'll quantize later | GPU has fp16 & you want easy 2× | edge/mobile/browser, size & speed matter |
+
+**Why VolvoxAI leans on fp32 + int8 and is wary of fp16 (from the README):**
+
+- **fp16** needs the WebGPU `shader-f16` extension, which isn't universal on consumer devices —
+  so a browser engine can't rely on it. (The fp16 folder here is mainly for platforms/formats
+  that do support it.)
+- **int8** gives a 4× size cut with near-zero accuracy loss, and integer math runs fast on *any*
+  CPU/GPU — the sweet spot for portable inference. Volvox skips **int4** because 4-bit needs
+  fiddly bit-unpacking that hurts low-end mobile GPUs.
+
+---
+
+## 4.6 The three configs, side by side
+
+Because precision is a *storage* choice, the graphs are nearly identical — only the conv op and a
+little quant bookkeeping differ:
+
+```
+fp32 / fp16 graph:            int8 graph:
+  input (float)                 input (uint8)
+     │                             │  QuantizeLinear   ← float/uint8 → int8 (once)
+  Conv2D ─┐                     QConv2D ─┐             ← integer conv, int8 in/out
+  Conv2D  │ 182 float convs     QConv2D  │ 182 int8 convs (stays int8 the whole way)
+   …      │                      …       │
+  Add / MaxPool / Resize        Add / MaxPool / Resize (int8-aware)
+     │                             │  DequantizeLinear ← int8 → float (twice, at the end)
+  scores, boxes (float)         scores, boxes (float)
+```
+
+That's why the int8 config has **3 extra nodes** (1 `QuantizeLinear` + 2 `DequantizeLinear`) and
+its 182 convs are `QConv2D` instead of `Conv2D`. Same detector, three sizes — you pick the point
+on the curve your device needs.
+
+---
+
+## 4.7 What you just learned
+
+- **Precision** is how many bits each number gets: fp32 (4 B), fp16 (2 B), int8 (1 B).
+- **Quantization** maps floats onto the 256-value int8 ruler with a `scale` + `zero_point`; the
+  formulas `(q−zp)×scale` and `round(r/scale)+zp` are the whole trick, and they're in the repo.
+- **int8 conv** accumulates in cheap int32 and requantizes once — activations stay int8 layer to
+  layer, giving ~4× smaller + faster with ~1% accuracy cost.
+- You **choose** the format per deployment: fp32 for fidelity, int8 for the edge, fp16 when the
+  hardware supports it.
+
+Next: how VolvoxAI takes any of these graphs and runs it *fast* — the four hardware tiers, the
+jump from a naive kernel to an optimized one, and operator fusion.
+
+**Next:** [Chapter 5 — Inside the Engine →](05-inside-the-engine.md)

package/docs/textbook/05-inside-the-engine.md

@@ -0,0 +1,155 @@
+# Chapter 5 — Inside the Engine
+
+*Goal: understand how VolvoxAI turns "a list of ops" into something that runs **fast** on real
+hardware — the four tiers, the leap from a naive kernel to an optimized one, and operator fusion.*
+
+You now know *what* the models compute. This chapter is about *making it fast* — the engineering
+that separates a textbook implementation from a shippable one. This is where a lot of real AI
+systems work lives.
+
+---
+
+## 5.1 One graph, four engines (the tiers)
+
+Recall from Chapter 1: VolvoxAI can run the same graph four ways in the browser (plus a native
+binary). They differ only in **who does the arithmetic** and **where the tensors live**.
+
+```mermaid
+flowchart TD
+    G["Graph + weights"] --> I["VolvoxAI.init(backend)"]
+    I --> T1["Tier 1 · WebNN<br/>hand graph to browser ML API → NPU/GPU/CPU"]
+    I --> T2["Tier 2 · WebGPU<br/>one compute pipeline per node, all on GPU"]
+    I --> T3["Tier 3 · WASM SIMD<br/>compiled C kernels over linear memory"]
+    I --> T4["Tier 4 · Pure JS<br/>reference kernels — slow, always correct"]
+    T1 -.fallback.-> T3
+    T3 -.fallback.-> T4
+```
+
+- **Tier 4 — Pure JS** (`js/ops/*.js`): the naive kernels we read all book. Slow but simple and
+  dependency-free. **It is the ground truth**: every faster tier is checked against it.
+- **Tier 3 — WASM** (`js/WasmEngine.js` + `native/kernels/*.c`): the *same* ops compiled to
+  WebAssembly with SIMD. A bump allocator drops every tensor into one flat block of linear
+  memory; `execute()` calls a compiled C kernel per node. Often 5–50× faster than pure JS.
+- **Tier 2 — WebGPU** (`js/GraphExecutor.js` + `shaders/*.wgsl`): each op becomes a GPU **compute
+  shader**. At compile time it uploads all weights to VRAM and builds *one pipeline per node*;
+  `execute()` replays them in a single command stream, **with no CPU round-trip between nodes**,
+  so the whole model stays resident on the GPU. If a needed shader is missing, the current
+  executor warns and skips that node; use WASM/CPU for models that require unsupported ops.
+- **Tier 1 — WebNN** (`js/WebNNEngine.js`): hands the graph to the browser's own neural-network
+  API, which may dispatch to a dedicated **NPU**. Falls through to a lower tier for any op it
+  doesn't support.
+
+The design principle: **choose a viable backend up front.** WebNN can fall through when graph
+compilation fails, and WASM can fall back to pure JS helpers. WebGPU should be used for graphs
+whose ops are covered by shaders.
+
+---
+
+## 5.2 Why the naive kernel is slow
+
+Reread the naive `Conv2D` from Chapter 3: seven nested loops, one multiply at a time. It is
+*correct*, but a modern CPU hates it, for two reasons:
+
+1. **It wastes the SIMD units.** A CPU core can multiply 8 (AVX2) or more floats in a *single*
+   instruction. The naive loop does them one… at… a… time, using a fraction of the silicon.
+2. **It thrashes the cache.** RAM is ~100× slower than the CPU. Cores keep recently used data in
+   a tiny fast **cache**. The naive conv jumps all over memory (strided image reads), so it
+   constantly waits on RAM instead of computing.
+
+The result: the naive kernel might use **2–5%** of the chip's real throughput. Optimization is
+about feeding the SIMD units and respecting the cache.
+
+---
+
+## 5.3 From naive to fast: the same math, rearranged
+
+Optimized kernels never change *what* is computed (the output is identical to Tier 4) — they
+change *the order and layout* of the work. VolvoxAI's hot paths live in `native/conv_f32_opt.c`
+(fp32) and `native/quant_cpu_opt.c` (int8). The main techniques:
+
+| Technique | Idea | Payoff |
+|---|---|---|
+| **im2col + GEMM** | Unfold conv patches into a big matrix, then call a fast matrix-multiply | reuses decades of tuned matmul; feeds SIMD |
+| **Pointwise GEMM** | 1×1 convs *are* a matrix multiply — treat them as one directly | biggest single win (most convs are 1×1) |
+| **Weight prepacking** | Re-lay-out weights once at load into the exact order the inner loop reads | turns cache-miss reads into sequential reads |
+| **Blocking / tiling** | Process small tiles that fit in cache before moving on | data is reused while it's still hot |
+| **SIMD (AVX2/NEON)** | Do 8–32 multiply-adds per instruction | ~8–32× the arithmetic per cycle |
+| **Multithreading** | Split output rows across CPU cores | ~Ncores× on top of everything |
+
+```
+naive conv                      optimized conv (im2col + GEMM)
+ for each output pixel:          [1] unfold input patches → one big matrix  (once)
+   for each filter tap:          [2] one big, cache-friendly, SIMD, threaded
+     one scalar multiply             matrix-multiply against prepacked weights
+ (scattered memory, 1 lane)      (sequential memory, many lanes, many cores)
+```
+
+> **A real result from this repo.** The memory notes for this project record that adding an
+> *arena buffer-reuse planner* (reusing scratch memory instead of allocating fresh per node) cut
+> peak memory from **88.8 MB → 18.8 MB** and, combined with a pointwise-GEMM path, sped the
+> EfficientDet forward pass up meaningfully — closing the gap with Google's TFLite/XNNPACK to
+> ~1.16× (warm). Same numbers out; dramatically less memory and time. *That* is kernel
+> engineering.
+
+The lesson: **correctness lives in the naive kernel; performance lives in memory layout.**
+Read `js/ops/conv2D.js`, then diff it against `native/conv_f32_opt.c` to see the two
+halves of the craft.
+
+---
+
+## 5.4 Operator fusion: stop touching memory so much
+
+Between ops, results get written to memory and read back by the next op. For big feature maps,
+*moving* the data can cost more than the math. **Operator fusion** merges adjacent ops so the
+data is touched once. VolvoxAI runs a compile-time fusion pass (`native/graph_opt_fusion.c`):
+
+```
+before fusion:  Conv2D ─▶ [write feature map] ─▶ ReLU6 ─▶ [write again]
+after  fusion:  Conv2D+ReLU6 ─▶ [write once, already activated]
+```
+
+Fusion patterns it applies (see `docs/operator_fusion_patterns.md`):
+
+- **Conv + ReLU6** → clamp right inside the conv's requantize step (you saw `relu` folded into the
+  kernel in Chapters 3–4).
+- **Chained `Add`** → sum several residuals in one pass.
+- **Depthwise → Pointwise** → run the MBConv pair back-to-back without spilling the intermediate.
+- **Concat + Sigmoid**, **alias elision** (drop no-op copies like some `Reshape`s).
+
+Each fused pair is one fewer full read+write of a big tensor. Across 262 nodes, that adds up.
+
+---
+
+## 5.5 Memory: tensors are temporary
+
+A subtle but important point: most tensors in a forward pass are **scratch** — needed briefly,
+then never again (e.g. `ln1_0` is dead once the attention MatMul consumed it). A naive engine
+allocates a fresh buffer for every node (simple, but wasteful). A smart one **plans** which
+buffers can share memory, because their lifetimes don't overlap — the **arena buffer-reuse
+planner** mentioned above. This is why VolvoxAI can run a model whose tensors *sum* to hundreds of
+MB in a fraction of that peak RAM: the same physical bytes are recycled node after node.
+
+```
+node lifetimes (─ = alive):     buffer reuse:
+  A: ───                          A and C never overlap → give them the SAME buffer
+  B:   ─────                      B and D never overlap → share a buffer
+  C:      ────
+  D:          ───                 4 tensors, 2 physical buffers
+```
+
+---
+
+## 5.6 The whole engine, in one sentence
+
+> **VolvoxAI walks the graph's node list, dispatches each node to the fastest available backend's
+> kernel, and does so while reusing memory buffers and fusing adjacent ops — producing the exact
+> same numbers as the naive reference, just far faster and smaller.**
+
+That's the entire system. Everything else is one more op, one more backend, or one more
+optimization on this skeleton.
+
+> **This chapter covered the four *browser* tiers.** VolvoxAI also ships a full **native** engine
+> (a freestanding C binary spanning CPU + Vulkan / OpenGL / Metal / Android NNAPI) that runs the
+> *same blueprint*. That's the whole next chapter.
+
+**Next:** [Chapter 6 — The Native Engine →](06-native-engine-architecture.md)