volvoxai 0.1.0

package/docs/textbook/01-foundations.md

@@ -0,0 +1,233 @@
+# Chapter 1 — Foundations
+
+*Goal: after this chapter you can look at any model and see it as a **graph of operations on
+tensors**, and you understand the pieces VolvoxAI uses to run one.*
+
+---
+
+## 1.1 What "running an AI" actually means
+
+When you use an AI model, three things happen:
+
+1. Your input (text, an image, audio) is turned into **numbers**.
+2. Those numbers flow through a long list of **arithmetic operations**. Each operation also
+   uses a big table of pre-computed numbers called **weights**.
+3. The final numbers are turned back into something meaningful (a word, a box, a label).
+
+Step 2 is the model. Running it once is called a **forward pass**, or **inference**. That is
+all inference is: a pipeline of multiply-and-add, arranged in a specific order that someone
+*trained* to be useful.
+
+> **Training vs. inference.** *Training* is the expensive process that searches for good weight
+> values by showing the network millions of examples and nudging the weights to reduce mistakes.
+> *Inference* just uses the finished weights. **VolvoxAI is an inference engine** — it never
+> changes a weight. Everything in this book is about inference.
+
+---
+
+## 1.2 The tensor: the only data structure you need
+
+Every number moving through a network lives in a **tensor**. A tensor is just a
+multi-dimensional array (a grid of numbers) plus a **shape** that says how big each dimension is.
+
+```
+scalar   3.14                         shape []            (a single number)
+vector   [3.14, 2.71, 1.62]           shape [3]           (a list)
+matrix   [[1, 2, 3],                   shape [2, 3]        (a table: 2 rows, 3 cols)
+          [4, 5, 6]]
+tensor   a batch of 224x224 RGB images shape [8, 224, 224, 3]
+```
+
+That last shape, `[8, 224, 224, 3]`, reads as: **8** images, each **224** pixels tall, **224**
+wide, with **3** color channels (red, green, blue). Four numbers fully describe millions of
+values.
+
+**In this repo**, a tensor is a tiny object (`js/Tensor.js`) — a name, a shape, a data type,
+and a flat buffer of numbers:
+
+```javascript
+// js/Tensor.js (paraphrased)
+class Tensor {
+  name;      // e.g. "hidden_0"
+  shape;     // e.g. [1, 256, 64]
+  dtype;     // "float32" | "int8" | "int32"
+  buffer;    // a flat Float32Array / Int8Array holding shape-product numbers
+  isWeight;  // true if it was learned during training (read-only at inference)
+}
+```
+
+### Tensors are stored flat
+
+A computer's memory is one long line of bytes — it has no idea about "rows" and "columns." A
+tensor with shape `[2, 3]` is stored as **6 numbers in a row**, and we compute *where* element
+`[row, col]` lives with index math:
+
+```
+logical view          flat memory (what actually exists)
+[[a, b, c],           [ a, b, c, d, e, f ]
+ [d, e, f]]             0  1  2  3  4  5
+
+  element [row, col]  →  flat index = row * 3 + col
+  element [1, 2] = 'f' →  1 * 3 + 2 = 5   ✓
+```
+
+You will see this exact pattern — `((b * H + y) * W + x) * C + c` — all over the kernels. It is
+how a 4-D image tensor is addressed inside a 1-D array. Memorize the shape, and the index math
+follows.
+
+> **NHWC vs NCHW.** The *order* of the dimensions matters. VolvoxAI's vision models use
+> **NHWC** (batch, height, width, channels) — the color channels of one pixel sit next to each
+> other in memory. PyTorch usually uses **NCHW**. Same data, different memory layout; kernels
+> must agree on which one they're reading. (This repo's converter records `"internal_layout":
+> "NHWC"` in every vision `config.json`.)
+
+---
+
+## 1.3 The operation: one small, well-defined job
+
+An **operation** (or **op**, or **layer**) takes one or more input tensors, does a fixed piece
+of math, and writes one or more output tensors. Examples you will meet:
+
+| Op | In words | Used by |
+|---|---|---|
+| `MatMul` | Matrix multiply — the core "mixing" of features | both models |
+| `Conv2D` | Slide a small filter over an image | the detector |
+| `Add` | Element-wise add two tensors | both |
+| `LayerNorm` | Re-center and re-scale a vector to be well-behaved | the LM |
+| `SDPA` | Scaled-dot-product attention — "which words look at which" | the LM |
+| `GELU` / `ReLU` | A nonlinear squashing function | both |
+| `MaxPool2D` | Shrink an image by keeping the biggest value in each patch | the detector |
+
+Every op in VolvoxAI has a plain-English **reference implementation** in `js/ops/`, one small
+file each. Here is the *entire* `Add` op — this is not a simplification, it is the real code:
+
+```javascript
+// js/ops/add.js — element-wise addition with broadcasting
+for (let i = 0; i < out.length; i++) {
+  out[i] = a[i] + b[i % b.length];   // b.length may be smaller ("broadcast")
+}
+```
+
+That is the whole secret: a model is thousands of operations like this, each trivial, chained
+together. **There is no step where something inexplicable happens.**
+
+---
+
+## 1.4 The graph: operations wired together
+
+A model is a **graph** — a list of ops where each op's outputs feed later ops' inputs. VolvoxAI
+stores this as a `Graph` object (`js/Graph.js`): a set of **tensors** and a list of **nodes**
+(op + which tensors are its inputs/outputs + its parameters).
+
+Because every op declares its inputs and outputs *by name*, the graph is just bookkeeping:
+
+```
+tokens ─┐
+        ├─▶ [Embedding] ─▶ emb_tok ─┐
+wte  ───┘                           ├─▶ [Add] ─▶ hidden_0 ─▶ [LayerNorm] ─▶ ...
+positions ─▶ [Embedding] ─▶ emb_pos ┘
+```
+
+To **run** the graph, VolvoxAI simply walks the node list top to bottom and executes each op.
+The whole executor loop is this readable (`js/CPUEngine.js`):
+
+```javascript
+// js/CPUEngine.js — the heart of the engine
+for (const node of graph.nodes) {
+  this._runNode(node);      // dispatch on node.opType → the matching kernel
+}
+```
+
+`_runNode` is a big `switch` on the op type (`"MatMul"` → matmul kernel, `"Conv2D"` → conv
+kernel, …). That's it. **A neural network engine is a `for` loop over a list of function calls.**
+Everything else is making those functions fast (Chapter 5) and numerically small (Chapter 4).
+
+---
+
+## 1.5 The blueprint: how a model is stored
+
+A VolvoxAI model on disk is two files:
+
+```
+model/
+  config.json          # the GRAPH: a list of op nodes (topology + params + shapes)
+  model.safetensors    # the WEIGHTS: the learned numbers, in a standard binary format
+```
+
+- **`config.json`** is the *blueprint*: an ordered list of nodes. Each node names its op, its
+  input/output tensors, its parameters, and the exact output shape (pre-computed by the
+  exporter so the engine never has to guess). Here is one real node from TinyStories:
+
+  ```json
+  {
+    "op": "LayerNorm",
+    "inputs":  { "input": "hidden_0", "weight": "h.0.ln_1.weight", "bias": "h.0.ln_1.bias" },
+    "outputs": { "out": "ln1_0" },
+    "outputs_shape": { "out": [1, 256, 64] },
+    "params": { "eps": 1e-05, "d_model": 64 }
+  }
+  ```
+
+- **`model.safetensors`** holds the raw weight tensors (`h.0.ln_1.weight`, `wte.weight`, …) in
+  [safetensors](https://github.com/huggingface/safetensors) format — a simple, safe, standard
+  layout used across the ML world.
+
+`js/GraphLoader.js` reads both, builds the `Graph`, and hands it to the engine. **You train in
+PyTorch, export to this blueprint, and Volvox runs it** — with no PyTorch, no ONNX Runtime, no
+dependencies at inference time.
+
+---
+
+## 1.6 One model, four ways to run it (the "tiers")
+
+The same graph can be executed on very different hardware. VolvoxAI picks the best available
+**tier** automatically, and every tier computes the *same* result:
+
+```mermaid
+flowchart TD
+    G[Graph + weights] --> SEL{VolvoxAI.init<br/>picks best available}
+    SEL -->|browser NPU/GPU| T1[Tier 1 · WebNN]
+    SEL -->|browser GPU| T2[Tier 2 · WebGPU<br/>WGSL compute shaders]
+    SEL -->|any CPU, fast| T3[Tier 3 · WASM SIMD<br/>compiled C kernels]
+    SEL -->|any CPU, always works| T4[Tier 4 · Pure JS<br/>reference kernels]
+    N[Native binary · C<br/>Vulkan/OpenGL/Metal/CPU] -.same blueprint.-> G
+```
+
+- **Tier 4 (Pure JS, `js/ops/*.js`)** is the *reference*: slow but obviously-correct, and the
+  ground truth every other tier is checked against. **We use it as our teaching text** because
+  it is the most readable.
+- **Tier 3 (WASM)** runs the same math as compiled C for a big speedup.
+- **Tier 2 (WebGPU)** re-expresses each op as a GPU compute shader (`shaders/*.wgsl`).
+- **Tier 1 (WebNN)** hands the graph to the browser's own neural-network API (can hit an NPU).
+- **Native** (`native/`) is a standalone C program that runs the *same* blueprint on a desktop
+  or phone, optionally on Vulkan/OpenGL/Metal.
+
+For the rest of the book, when we "trace an op," we read the pure-JS or portable-C version,
+because they say most directly *what the math is*.
+
+---
+
+## 1.7 The mental model, assembled
+
+Put it together and you have the whole engine in one picture:
+
+```
+  INPUT ──encode──▶ TENSORS ──┐
+                              │   for each node in graph.nodes:
+   WEIGHTS (from .safetensors)─┼──▶   op(inputs, params) → output tensor
+                              │
+                        (repeat for all ~85 or ~262 nodes)
+                              │
+                              ▼
+                        OUTPUT TENSOR ──decode──▶ ANSWER
+```
+
+Two questions define any model:
+
+1. **What are the ops, and in what order?** (the graph / `config.json`)
+2. **What do the weights make each op do?** (the `.safetensors`)
+
+In the next two chapters we answer both questions for two real models — and you'll see that a
+"language model" and an "image detector" are the *same idea* with different ops in the list.
+
+**Next:** [Chapter 2 — A Language Model, op by op →](02-tinystories-language-model.md)

package/docs/textbook/02-tinystories-language-model.md

@@ -0,0 +1,300 @@
+# Chapter 2 — A Language Model, op by op (TinyStories)
+
+*Goal: follow the words **"Once upon a time, Lily"** through a real GPT-style transformer and
+watch it predict the next word. Every op here is one of the small kernels from `js/ops/`.*
+
+The model lives in `models/tinystories_1m/`. It is a tiny GPT (a **decoder-only transformer**)
+trained on the [TinyStories](https://arxiv.org/abs/2305.07759) dataset of simple children's
+stories. "Tiny" is real: its internal vector width is **64**, it has **8** layers, and yet it
+writes coherent little stories. Studying it teaches you the *exact* architecture behind
+GPT-2/3/4, LLaMA, and Mistral — those are this graph, scaled up.
+
+## 2.1 The model's dimensions (read them off the blueprint)
+
+From `config.json`, one number at a time:
+
+| Symbol | Value | Meaning |
+|---|---|---|
+| `d_model` | **64** | Width of the "thought vector" carried per token. |
+| `n_layers` | **8** | Number of transformer blocks stacked. |
+| `n_heads` | **16** | Attention heads per block (so `head_dim = 64/16 = 4`). |
+| `d_mlp` | **256** | Width of the feed-forward hidden layer (`4 × d_model`). |
+| `vocab` | **50257** | Number of distinct tokens it knows (GPT-2 vocabulary). |
+| `context` | **256** | Max tokens it can look at in one pass. |
+
+> The folder is called `tinystories_1m` (~1M transformer parameters), but `model.safetensors`
+> is ~27 MB. Why? The **token embedding table** is `50257 × 64 ≈ 3.2M` numbers, and there are
+> two such tables (input + output). In tiny LMs, the *vocabulary*, not the *layers*, dominates
+> the file. That's a real lesson: model size ≠ model depth.
+
+The whole graph is **85 nodes**. Their inventory:
+
+```
+33 MatMul   17 LayerNorm   17 Add   8 SDPA   8 GELU   2 Embedding
+= 2 embeddings + 8 blocks × (2 LayerNorm + 4 MatMul + 1 SDPA + 1 GELU + 2 Add) + final LayerNorm + 1 lm_head
+```
+
+## 2.2 The pipeline at a glance
+
+```mermaid
+flowchart TD
+    P["prompt: 'Once upon a time, Lily'"] --> TOK[Tokenizer BPE<br/>text → token ids]
+    TOK --> EMB
+    subgraph EMB[Input embedding]
+      T[token ids] --> WTE[Embedding · wte]
+      POSN[positions 0,1,2…] --> WPE[Embedding · wpe]
+      WTE --> ADD0[Add]
+      WPE --> ADD0
+    end
+    ADD0 --> BLK
+    subgraph BLK["× 8 transformer blocks"]
+      direction TB
+      L1[LayerNorm] --> QKV[MatMul: qkv_proj] --> SDPA[SDPA · causal] --> OP[MatMul: out_proj] --> R1((+ residual))
+      R1 --> L2[LayerNorm] --> FC[MatMul: c_fc] --> G[GELU] --> PR[MatMul: c_proj] --> R2((+ residual))
+    end
+    BLK --> LNF[LayerNorm · final] --> HEAD[MatMul: lm_head] --> LOG[logits: 50257 scores]
+    LOG --> ARGMAX[pick highest score] --> NEXT[next token] --> DETOK[decode → text]
+    NEXT -. append & repeat .-> TOK
+```
+
+Now we walk it, stage by stage, with the real kernels.
+
+---
+
+## 2.3 Stage 0 — Tokenize: text becomes integers
+
+A neural net cannot read letters; it reads numbers. The **tokenizer** (`js/Tokenizer.js`,
+`native/tokenizer.c`) chops text into **tokens** (common word-pieces) and maps each to an
+integer ID using a vocabulary + a list of **merge rules** (this is **Byte-Pair Encoding**, BPE).
+
+```
+"Once upon a time, Lily"
+   │  regex splits into word-ish chunks, then BPE merges frequent byte-pairs
+   ▼
+[ "Once", " upon", " a", " time", ",", " Lily" ]     (illustrative)
+   │  each chunk → an integer id from the 50257-word vocab
+   ▼
+tokens    = [7454, 2402, 257, 640, 11, 20037, …]     ← the model's real input
+positions = [   0,    1,   2,   3,  4,     5, …]     ← "which slot am I?"
+```
+
+Two integer tensors go into the graph: **`tokens`** (what the words are) and **`positions`**
+(their order, `0,1,2,…`). Both are shape `[1, 256]` — the sequence is padded to the 256-token
+context window.
+
+> **Why positions?** The attention math (below) is order-blind by itself — it would treat
+> "dog bites man" and "man bites dog" identically. Feeding in an explicit position number lets
+> the model learn word order.
+
+---
+
+## 2.4 Stage 1 — Embedding: integers become vectors
+
+An ID like `20037` ("Lily") is meaningless as a *number* (it isn't 20037× anything). We replace
+it with a learned **vector** of 64 numbers — its **embedding** — that encodes meaning. The
+`Embedding` op is pure table lookup. Here is the entire kernel (`js/ops/embedding.js`):
+
+```javascript
+for (let i = 0; i < seq_len; i++) {
+  const token_id = tokens[i];
+  for (let j = 0; j < d_model; j++) {
+    out[i * d_model + j] = weight[token_id * d_model + j];   // copy row `token_id`
+  }
+}
+```
+
+The weight `wte.weight` is the `[50257, 64]` table; row `token_id` *is* that token's meaning
+vector. It runs **twice**:
+
+- `Embedding(tokens, wte)` → `emb_tok` `[1,256,64]` — *what* each token is.
+- `Embedding(positions, wpe)` → `emb_pos` `[1,256,64]` — *where* it is.
+
+Then `Add` fuses them: `hidden_0 = emb_tok + emb_pos`. Now every one of the 256 slots holds a
+64-number vector that mixes *word identity* and *position*. This tensor, `hidden_0 [1,256,64]`,
+is the **residual stream** — the "conveyor belt" that every block reads from and writes back to.
+
+```
+hidden_0:  256 rows (one per token position), each a 64-number vector
+
+ pos 0 "Once"  [ 0.12, -0.4, ...(64) ]
+ pos 1 " upon" [-0.03,  0.9, ...(64) ]
+ pos 2 " a"    [ ...                 ]
+   ⋮
+```
+
+---
+
+## 2.5 Stage 2 — A transformer block (this happens 8×)
+
+Each block refines the residual stream with two sub-steps: **attention** (tokens share
+information) and a **feed-forward MLP** (each token thinks on its own). Both are wrapped in the
+**pre-norm + residual** pattern that makes deep networks trainable.
+
+### 2.5a LayerNorm — keep the numbers sane
+
+Before each sub-step, `LayerNorm` rescales each token's 64-vector to have mean 0 and variance 1,
+then applies a learned scale (`weight`) and shift (`bias`). This stops values from exploding or
+vanishing across 8 layers. The real kernel (`js/ops/layerNorm.js`), per token row:
+
+```javascript
+const mean = sum / d_model;
+const variance = sq_sum / d_model - mean * mean;
+const inv_std = 1 / Math.sqrt(variance + 1e-5);          // eps guards ÷0
+out[j] = (in[j] - mean) * inv_std * weight[j] + bias[j]; // normalize, then re-scale/shift
+```
+
+### 2.5b Attention — "which earlier words matter to me?"
+
+This is the heart of a transformer. First a single `MatMul` projects each 64-vector up to **192**
+numbers (`qkv_proj`, shape `[1,256,192]`). Those 192 are three 64-vectors glued together: the
+**Query**, **Key**, and **Value** (Q, K, V). Intuition:
+
+- **Query** = "what am I looking for?"
+- **Key** = "what do I offer?"
+- **Value** = "what I'll hand over if you pick me."
+
+Then `SDPA` (**Scaled Dot-Product Attention**) does the actual looking. For each token *q*, it
+compares its Query to every earlier token's Key (a dot product = similarity), turns the
+similarities into weights with **softmax**, and returns a weighted blend of those tokens' Values.
+The real causal kernel (`js/ops/sDPA.js`), lightly annotated:
+
+```javascript
+for (let h = 0; h < num_heads; h++) {                 // 16 independent heads
+  for (let q = 0; q < seq_len; q++) {                 // for each query position
+    for (let k = 0; k <= q; k++) {                    // ← only look at k ≤ q  (CAUSAL)
+      score = dot(Q[q,h], K[k,h]) * scale;            // similarity of q to k
+    }
+    softmax(scores);                                  // turn scores into weights that sum to 1
+    out[q,h] = Σ_k  weight[k] * V[k,h];               // blended value
+  }
+}
+```
+
+Two ideas worth pausing on:
+
+- **Causal masking** (`k <= q`): a token may only attend to itself and *earlier* tokens, never
+  future ones. That's what makes it a left-to-right text *generator* — position 3 can't cheat by
+  peeking at position 4.
+- **Multi-head** (`h`): the 64 dims are split into 16 groups of 4. Each "head" learns a different
+  kind of relationship (e.g. one tracks subjects, another tracks punctuation) in parallel.
+
+```
+Attention for the token " Lily" (illustrative weights after softmax):
+
+  " Lily" attends to →   "Once"  " upon"  " a"  " time"  ","   " Lily"
+  weight                  0.05    0.05   0.05   0.30   0.05   0.50
+                                                  ▲             ▲
+                                       "time" is relevant   mostly itself
+  output = 0.05·V(Once) + … + 0.30·V(time) + 0.50·V(Lily)
+```
+
+A final `MatMul` (`out_proj`, with bias) mixes the 16 heads' outputs back into a 64-vector, and a
+**residual `Add`** adds it onto the stream: `add1 = hidden + attention_output`. "Residual" means
+we *add* the block's result instead of replacing the stream — so information is never lost and
+gradients flow during training.
+
+### 2.5c Feed-forward MLP — each token thinks
+
+After tokens have shared info, each one is transformed on its own by a 2-layer MLP:
+
+```
+c_fc  : MatMul 64 → 256   (+bias)     "expand: give it room to compute"
+GELU  : nonlinearity                  "let it make nonlinear decisions"
+c_proj: MatMul 256 → 64   (+bias)     "compress back to stream width"
+Add   : residual                      hidden_next = add1 + mlp_output
+```
+
+`GELU` (`js/ops/gELU.js`) is the nonlinearity — a smooth gate that lets small negatives leak and
+passes positives. Without a nonlinearity like this, stacking MatMuls would collapse into a single
+MatMul and the network could only learn straight-line relationships:
+
+```javascript
+out[i] = 0.5 * x * (1 + tanh(0.7978845608 * (x + 0.044715 * x*x*x)));   // the GELU curve
+```
+
+The block's output `hidden_next [1,256,64]` has the same shape as its input — which is exactly
+why we can stack **8** of them. Each block reads the stream and writes a slightly smarter version
+back.
+
+---
+
+## 2.6 Stage 3 — Head: vectors become word-scores
+
+After the 8th block, one last `LayerNorm` (`ln_f`) cleans up the stream. Then the **language-model
+head** — a single `MatMul` by `lm_head.weight [50257, 64]` — turns each 64-vector into **50257
+scores**, one per vocabulary word:
+
+```
+final_norm [1,256,64]  ──MatMul lm_head──▶  logits [1,256,50257]
+```
+
+These raw scores are called **logits**. `logits[0, p, w]` = "how strongly the model, having read
+tokens `0..p`, expects word `w` to come next." We only care about the row for the **last real
+token** — that's the prediction for what comes after the prompt.
+
+---
+
+## 2.7 Stage 4 — Sample: scores become the next word
+
+We now have 50257 scores for the next token. This repo's generator (`native/main.c`,
+`command_generate`) uses the simplest rule, **greedy / argmax** — just take the highest:
+
+```c
+int best_id = 0; float best_val = -1e30f;
+for (int i = 0; i < vocab_count; i++)
+    if (logits[i] > best_val) { best_val = logits[i]; best_id = i; }   // argmax
+// best_id is the next token; decode it back to text:
+printf("%s", tokenizer_decode(tok, best_id));
+```
+
+> **Real generators add randomness** — *temperature* (flatten/sharpen the scores), *top-k* /
+> *top-p* (sample only from the most likely few). Those turn logits into a probability
+> distribution with `softmax` and roll a weighted die, which is why ChatGPT gives different
+> answers each time. Greedy is the deterministic special case; it's perfect for a textbook.
+
+---
+
+## 2.8 The loop — one word at a time (autoregression)
+
+A transformer predicts **one** token per forward pass. To write a sentence, you append the new
+token and run again. This is **autoregressive generation**:
+
+```mermaid
+flowchart LR
+    A["tokens so far"] --> B[forward pass<br/>85 ops] --> C[logits] --> D[argmax → next token]
+    D --> E{stop?<br/>max length or EOS}
+    E -- no --> A
+    E -- yes --> F[done: full story]
+```
+
+```
+step 0:  "Once upon a time, Lily"           → " was"
+step 1:  "Once upon a time, Lily was"       → " a"
+step 2:  "Once upon a time, Lily was a"     → " little"
+step 3:  … → " girl" → " who" → " loved" → " to" → " play" …
+```
+
+That is literally how ChatGPT types word-by-word: it is running this loop, each new token fed
+back in as input.
+
+> **KV-cache (an optimization you'll hear about).** Naively, step *N* recomputes attention over
+> all *N* tokens from scratch — wasteful. Production engines *cache* each token's Key and Value
+> so each step only computes the new token's. VolvoxAI's native runner exposes this split as
+> `engine_prefill()` (process the whole prompt once) and `engine_decode()` (one new token at a
+> time). The math is identical; the cache just avoids repeating work.
+
+---
+
+## 2.9 What you just learned
+
+- A language model is: **tokenize → embed → (LayerNorm, attention, MLP) × N → head → sample →
+  loop.** Nothing more.
+- **Attention** lets tokens share information ("which earlier words matter to me?"); the **MLP**
+  lets each token compute on its own; **residuals + LayerNorm** make the stack trainable and deep.
+- Every op is one small kernel in `js/ops/` — `MatMul`, `SDPA`, `LayerNorm`, `GELU`, `Add`,
+  `Embedding`. GPT-2/3/4 and LLaMA are **this exact graph, wider and deeper**.
+
+Next we switch domains entirely — from text to pixels — and you'll see the *same skeleton*
+(a graph of small ops on tensors) solve a completely different problem.
+
+**Next:** [Chapter 3 — A Vision Model, op by op →](03-efficientdet-vision-model.md)