toy 0.8.0

data/README.md

@@ -0,0 +1,154 @@
+# toy
+
+<p align="center">
+  <img src="toy_logo.png" alt="toy" width="240" />
+</p>
+
+**v0.8.0** · first published gem · pre-1.0, not API-stable
+&nbsp;·&nbsp; [CHANGELOG](CHANGELOG.md)
+&nbsp;·&nbsp; [docs](docs/architecture.md)
+&nbsp;·&nbsp; [framework guide](docs/framework.md)
+
+**Readable machine learning.** toy is a transformer LM framework in
+Ruby, [Spinel](https://github.com/matz/spinel)-compiled to native
+binaries. The whole forward pass fits on one screen, every shape is
+annotated, and the building blocks are named after the math — and it
+still runs real HuggingFace models (SmolLM2, Llama 3, Qwen 2.5/3,
+Mistral, Gemma 2, OLMoE) with output matched against PyTorch, on CPU,
+CUDA, and Metal.
+
+```ruby
+# One transformer block (GPT-2 family — Llama swaps LN→RMSNorm and
+# adds RoPE inside self_attention; same one-screen shape).
+def transformer_block(x, block)
+  h  = layer_norm(x, block.ln1_gamma, block.ln1_beta)
+  x.add!(self_attention(h, block))
+  h2 = layer_norm(x, block.ln2_gamma, block.ln2_beta)
+  x.add!(feed_forward(h2, block))
+  x
+end
+```
+
+This is not pseudocode next to the real implementation — it **is** the
+implementation. Every model also carries an `algorithm_card` emitting
+Phuong–Hutter-style pseudocode (arXiv:2207.09238) with shape
+annotations, and the round-trip closes: `toy describe <model>` renders
+the card from a GGUF's metadata, and the card parses back into the Ruby
+that constructs the model.
+
+Inference (KV-cache decode, F32/Q8, zero-copy mmap), training
+(from-scratch, warm-start, LoRA), eval (per-token logprobs), and an
+OpenAI-compatible server — each an end-to-end single native binary.
+
+## Five minutes of play
+
+Requires Ruby, [Spinel](https://github.com/matz/spinel), and a C
+compiler. The CLI is plain MRI Ruby; the native compute runners are
+built on demand.
+
+```sh
+toy install                                  # build/verify the CPU backend
+toy fetch ggml-org/models \
+    tinyllamas/stories15M-q4_0.gguf          # grab a tiny model into ./data
+toy infer data/stories15M-q4_0.gguf \
+    --prompt "Once upon a time"              # greedy decode
+```
+
+Then train a small transformer from scratch, inspect it, serve it:
+
+```sh
+toy train from-scratch --steps 20 --seed 0   # runs/<id>/: weights, events, loss curve
+toy eval data/SmolLM2-135M-Instruct-Q8_0.gguf --top-k 5
+toy serve data/SmolLM2-135M-Instruct-Q8_0.gguf --port 4567 --name smol
+toy list                                     # finds GGUFs in HF / Ollama / LM Studio caches
+toy describe data/SmolLM2-135M-Instruct-Q8_0.gguf   # the algorithm card, from metadata
+```
+
+`toy --help` shows all 9 commands (`new`, `install`, `list`,
+`describe`, `fetch`, `infer`, `train`, `eval`, `serve`);
+[`docs/cli.md`](docs/cli.md) has flags, exit codes, and the
+machine-readable `--manifest` contract. If `toy list` shows nothing,
+any of `toy fetch`, `huggingface-cli download`, `ollama pull`, or LM
+Studio will populate a cache it sees.
+
+## Using toy as a framework
+
+The CLI is the front door; the framework is the house. Everything is a
+layered stack — **primitives → blocks → archs → engines → recipes** —
+each layer plain Ruby, each gated bit-identical against a reference,
+all of it loaded with one require:
+
+```ruby
+require "toy/compute"
+
+cfg  = Toy::SmolLM2Config.tiny
+opts = Toy::LLM::RecipeOptions.new
+opts.t_seq = 32
+opts.seed  = 42
+
+recipe = Toy::LLM::Recipes::FromScratch.new
+recipe.realize!(cfg, opts)
+steps.times do |step|
+  loss = recipe.step!(batch.seq_ids, batch.positions, batch.labels,
+                      batch.hp, step == 0)
+end
+```
+
+`realize!` builds the entire forward + loss + backward + AdamW graph
+natively; `step!` drives one training step; every knob is a named
+setter. That's the whole training contract — the same one toy's own
+gates use. Start your own project with `toy new mylab` (an experiment
+tree with ENV-driven hyperparameters — one compile, many runs) or
+`toy new mylib --lib` (a library consuming toy as a gem, native
+vendoring and a multi-arch `build.sh` included; devices are chosen at
+compile time).
+
+**[The framework guide](docs/framework.md)** is the tour;
+[`docs/authoring.md`](docs/authoring.md) shows how to add your own
+primitive, block, arch, or recipe; [`docs/consuming-toy.md`](docs/consuming-toy.md)
+is the full dependency story.
+
+## Models and backends
+
+Seventeen checkpoints run today — across GPT-2, SmolLM2, TinyLlama,
+Llama 3.2, Mistral, Qwen 2.5/3, Gemma 2, and OLMoE (MoE) — in F32 and
+Q8_0, with three tokenizer flavors and RoPE scaling auto-detected from
+the GGUF. CPU is the gated reference
+backend; CUDA and Metal mirror it, held bit-identical by
+`make verify-mirrors`.
+
+The honest per-model/per-backend matrix (including the footnotes —
+what's validated vs. expected-to-work vs. not wired) lives in
+[`docs/models.md`](docs/models.md); per-op coverage vs PyTorch in
+[`docs/coverage.md`](docs/coverage.md).
+
+## Documentation
+
+- [`docs/framework.md`](docs/framework.md) — **start here to build with
+  toy**: the stack, recipes, `toy/compute`, project scaffolds.
+- [`docs/architecture.md`](docs/architecture.md) — the five-layer
+  algorithm stack and how the CLI shells to compute runners.
+- [`docs/cli.md`](docs/cli.md) — the 9 commands, flags, exit codes,
+  and the manifest contract.
+- [`docs/authoring.md`](docs/authoring.md) — adding a primitive, block,
+  arch, or recipe; the algorithm-card round-trip.
+- [`docs/models.md`](docs/models.md) — the supported-models matrix,
+  tokenizers, RoPE, opt-in performance knobs.
+- [`docs/consuming-toy.md`](docs/consuming-toy.md) — toy as a gem
+  dependency: vendoring, native builds, CUDA/Metal opt-ins.
+- [`docs/gating.md`](docs/gating.md) — the bit-identical
+  reproducibility gates that hold all of the above together.
+- [`docs/events.md`](docs/events.md) — the `toy/v1` event schema.
+- [`docs/roadmap.md`](docs/roadmap.md) — deferred work and live
+  research directions.
+- [`examples/`](examples/README.md) — the narrated teaching set: seven
+  single-file examples (train, warm-start, LoRA, generate, logprobs,
+  run-log compare, ViT), each one `make` target away.
+
+## Acknowledgments
+
+A heartfelt thank-you to **Ninoslav Milenović** for graciously handing over
+the [`toy`](https://rubygems.org/gems/toy) gem name on RubyGems. A good name
+is a gift, and giving one up for someone else's project is the kind of quiet
+generosity the Ruby community runs on. We don't take it lightly — thank you,
+Ninoslav. 🙏

data/bin/toy

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+# bin/toy — the `toy` CLI entrypoint.
+#
+# Plain MRI Ruby (NOT Spinel-compiled). A thin shebang stub: load the
+# CRuby CLI shell under lib/toy/core/ and dispatch. The shell NEVER pulls
+# in the Spinel-compiled libs (tinynn/arch/transformer/...), so this runs
+# under system ruby with no native build.
+require_relative "../lib/toy/core/cli"
+exit Toy::Core::CLI.run(ARGV)

data/lib/toy/compute.rb

@@ -0,0 +1,135 @@
+# lib/toy/compute.rb — the full Toy compute surface, in one require.
+#
+# WHY THIS EXISTS (toy#42). A library consumer composes models out of Toy's
+# engines / primitives / recipes / loaders. Before this file they had to
+# hand-`require_relative` each one (the generated `vendor/spinel/deps.rb` only
+# pulls the top-level `toy.rb`, which is the MRI-safe CLI surface — Mat + Card +
+# version, no FFI). This file is the SPINEL-ONLY compute entrypoint: require it
+# once and the whole public composition API is loaded.
+#
+#   require_relative "vendor/spinel/toy/lib/toy/compute"   # one require
+#   cfg    = Toy::SmolLM2Config.new(627, 64, 4, 4, 128, 2, 32, 10000.0, 1.0e-5)
+#   engine = Toy::LLM::Engine::LlamaSeqEngine.new
+#   engine.realize_for_random_init(cfg, 32, 1, 0, false, false, 0, 1.0)
+#   # … build_training_step / KV decode / GGUF load …
+#
+# MRI-SAFETY BOUNDARY. Do NOT require this from `toy.rb` or anything under
+# `lib/toy/core/` (the CRuby CLI): it pulls `tinynn` (FFI `ffi_lib`s) and the
+# engines, which only load under a Spinel build. `toy.rb` must stay loadable by
+# plain MRI so `bin/toy` can require it. This file is for Spinel consumers only.
+#
+# MINIMAL-SURFACE NOTE. Spinel has no tree-shaking, so this pulls the COMMON
+# composition surface (all three engines + recipes + loaders). A consumer that
+# only ever builds, say, a LlamaSeqEngine can still require the individual files
+# instead to compile less; this is the convenience "everything" entry. The
+# backend mirrors (`*_cuda` / `*_metal`) are NOT required here — a consumer
+# selects its backend by requiring the matching engine variant.
+
+# FFI compute layer (Mat, sessions, tnn_* ops) + the MRI-safe sugar.
+require_relative "ffi/tinynn"
+require_relative "../toy"
+
+# Models + configs (Llama/SmolLM2, ViT) and the GGUF→engine loader.
+require_relative "models/toy_smollm2"
+require_relative "io/loaders/toy_smollm2_loader"
+require_relative "models/toy_vit"
+
+# Engines — the L4 forward/train drivers. Each pulls its L1 primitives,
+# L2 block, and L3 arch transitively.
+require_relative "llm/engine/llama_seq_engine"
+require_relative "llm/engine/vit_tiny_engine"
+require_relative "llm/engine/gpt2_seq_engine"
+
+# KV-cache decode engine (inference).
+require_relative "llm/engine/llama_kv_engine"
+
+# Recipes — the named training/init compositions. Realize-path orchestration
+# over the engines.
+#
+# HISTORY (toy#52, closed 2026-06-11): `recipes/lora` was excluded from this
+# surface for three days — an UNCALLED LoRA#realize! forwarding `cfg` into a
+# constructor slot poisoned the shared SmolLM2/RoPE ctor to poly program-wide
+# (spinel-dev#12, the constructor-slot sibling of spinel-dev#11). The Phase-A
+# RecipeOptions reshape of realize! (toy#64) dissolved the poison path: the
+# re-add compiles + trains byte-identically even on pre-#12-fix Spinel revs
+# (probed at 08a189c, 2026-06-11, toy#69). The compute-surface smoke keeps an
+# instantiated-but-UNCALLED LoRA as a tripwire so any regression re-breaks
+# that gate's C compile loudly.
+require_relative "llm/recipe_options"
+require_relative "llm/recipes/from_scratch"
+require_relative "llm/recipes/warm_start"
+require_relative "llm/recipes/lora"
+require_relative "llm/recipes/vit_tiny"
+
+# Inference I/O: GGUF model loading + the BPE tokenizer.
+require_relative "io/gguf_load"
+require_relative "io/tokenizer"
+
+# Training I/O + schedule (toy#73 item 2): the streaming token-corpus
+# loader, the ViT image/label loader, and the cosine LR schedule. The
+# loaders are plain file I/O through C helpers in libtinynn_ggml.a
+# (backend-agnostic; every backend links it) and ToyLR is pure Ruby, so
+# all three are SHARED across the CPU/CUDA/Metal entries — no mirrors.
+require_relative "io/toy_corpus_loader"
+require_relative "io/toy_image_loader"
+require_relative "train/toy_lr_schedule"
+
+# AdamW hyper-parameter value object + training labels (used when driving
+# build_training_step directly) + the validating per-step batch wrapper.
+require_relative "llm/adamw"
+require_relative "llm/labels"
+require_relative "llm/training_batch"
+require_relative "llm/classify_batch"
+
+# ── Toy::Device — the device-agnostic construction seam (toy#64 item 8) ──
+#
+# DEVICE IS CHOSEN AT COMPILE TIME by which compute entry you require:
+#   lib/toy/compute.rb        → CPU     (this file)
+#   lib/toy/compute_cuda.rb   → CUDA    (GB10/sm_121 etc.)
+#   lib/toy/compute_metal.rb  → Metal   (macOS only)
+# Spinel does NOT resolve a require_relative inside a conditional — an
+# `if ENV[...]`-guarded require is SILENTLY compiled to 0 (probed on
+# a699cf9: "cannot resolve call to 'require_relative'... emitting 0"),
+# so a TOY_DEVICE-switching single entry is impossible; per-device
+# entry files are the mechanism.
+#
+# User source stays DEVICE-AGNOSTIC by constructing through these
+# factories instead of naming the backend classes: each entry defines
+# the SAME Toy::Device surface returning its backend's types, so the
+# same experiment body compiles against any entry (the `toy new --lib`
+# scaffold's build.sh loops devices over per-device main_<dev>.rb
+# shims). Backends without a given engine/recipe variant simply do not
+# ship it here: vit + warm-start recipes are CPU/CUDA-uneven (vit has
+# no GPU engine; warm-start has no Metal recipe) — consumers that need
+# them require the variant file directly.
+module Toy
+  module Device
+    def self.name
+      "cpu"
+    end
+
+    def self.llama_engine
+      Toy::LLM::Engine::LlamaSeqEngine.new
+    end
+
+    def self.gpt2_engine
+      Toy::LLM::Engine::GPT2SeqEngine.new
+    end
+
+    def self.from_scratch_recipe
+      Toy::LLM::Recipes::FromScratch.new
+    end
+
+    def self.warm_start_recipe
+      Toy::LLM::Recipes::WarmStart.new
+    end
+  end
+end
+
+# Run-bundle writer (toy#73 item 1): runs/<id>/events.jsonl in the toy/v1
+# schema + the weights/ checkpoint-dir convention. SHARED (not mirrored):
+# the events C symbols live in libtinynn_ggml.a, linked by every backend.
+# Required AFTER the Toy::Device module above — run_start! stamps
+# backend{kind: Toy::Device.name} (toy#73 A.3), and Spinel needs the
+# constant defined before a file referencing it compiles.
+require_relative "io/run_bundle"

data/lib/toy/compute_cuda.rb

@@ -0,0 +1,104 @@
+# lib/toy/compute_cuda.rb — the Toy compute surface, CUDA entry.
+#
+# ── LOCKSTEP WARNING ── hand-maintained twin of lib/toy/compute.rb
+# (NOT mechanically mirrorable: the CPU entry pulls vit + warm-start +
+# loader files whose CUDA story is uneven — see the exclusions below —
+# so prep/gen_cuda_mirror.rb's uniform substitution cannot produce this
+# file). When compute.rb changes, update this file (and
+# compute_metal.rb) in the same commit.
+#
+# DEVICE IS CHOSEN AT COMPILE TIME (toy#64 item 8): require THIS file
+# instead of lib/toy/compute.rb and the same Toy::Device-driven
+# experiment body compiles into a CUDA binary. Spinel cannot switch the
+# require on ENV (a conditional require_relative silently compiles to
+# 0 — probed on a699cf9), hence per-device entries. Link with the CUDA
+# archives + force-link flag, exactly like every cuda target:
+#   spinel --cc='cc -Wl,-u,tnn_cuda_force_link' <entry> -o <bin>
+#   (deps: tinynn/libtinynn_ggml.a tinynn/libtinynn_ggml_cuda.a)
+#
+# SINGLE-TYPE BINARY discipline (landmine #16): TinyNNCuda is the
+# compute module; the CPU TinyNN is still DEFINED (transformer.rb
+# requires tinynn) for the checkpoint write/fuse seam, exactly like
+# the hand-written cuda runners. Same MRI-safety boundary as
+# compute.rb: Spinel-only, never require from the CRuby CLI.
+
+# FFI compute layer: CUDA TinyNN + the MRI-safe sugar (Mat + Card).
+require_relative "ffi/tinynn_cuda"
+require_relative "../toy"
+
+# Models + configs (shared — pure model/config code, no backend calls).
+require_relative "models/toy_smollm2"
+require_relative "io/loaders/toy_smollm2_loader"
+require_relative "models/toy_vit"
+
+# Engines — the CUDA mirrors (generated from the CPU files by
+# prep/gen_cuda_mirror.rb; `make` pattern rules keep them fresh).
+# NO ViT engine here: vit_tiny_engine has no CUDA mirror (absent from
+# MIRRORABLE — vit training is CPU-only this arc).
+require_relative "llm/engine/llama_seq_engine_cuda"
+require_relative "llm/engine/gpt2_seq_engine_cuda"
+
+# KV-cache decode engine (inference), CUDA mirror.
+require_relative "llm/engine/llama_kv_engine_cuda"
+
+# Recipes — the generated CUDA twins. lora_cuda included since toy#52
+# closed (the spinel-dev#12 constructor-slot facet was dissolved by the
+# toy#64 RecipeOptions reshape; see compute.rb's HISTORY note). Backend
+# gap that remains: vit_tiny has no CUDA recipe (no CUDA vit engine).
+require_relative "llm/recipe_options"
+require_relative "llm/recipes/from_scratch_cuda"
+require_relative "llm/recipes/warm_start_cuda"
+require_relative "llm/recipes/lora_cuda"
+
+# Inference I/O (shared): GGUF model loading + the BPE tokenizer.
+require_relative "io/gguf_load"
+require_relative "io/tokenizer"
+
+# Training I/O + schedule (toy#73 item 2): corpus loader + image loader +
+# cosine LR schedule. SHARED with the CPU entry (file I/O through C helpers
+# in libtinynn_ggml.a + pure Ruby) — see compute.rb's note. The image
+# loader rides along even though vit training stays CPU-only this arc
+# (the loader is backend-free; the missing piece is the engine/recipe).
+require_relative "io/toy_corpus_loader"
+require_relative "io/toy_image_loader"
+require_relative "train/toy_lr_schedule"
+
+# AdamW + labels + the validating batch wrapper (shared, pure Ruby).
+require_relative "llm/adamw"
+require_relative "llm/labels"
+require_relative "llm/training_batch"
+require_relative "llm/classify_batch"
+
+# ── Toy::Device, CUDA arm — see the compute.rb doc block. Same
+# surface, CUDA types; user source stays device-agnostic.
+module Toy
+  module Device
+    def self.name
+      "cuda"
+    end
+
+    def self.llama_engine
+      Toy::LLM::Engine::LlamaSeqEngineCuda.new
+    end
+
+    def self.gpt2_engine
+      Toy::LLM::Engine::GPT2SeqEngineCuda.new
+    end
+
+    def self.from_scratch_recipe
+      Toy::LLM::Recipes::FromScratchCuda.new
+    end
+
+    def self.warm_start_recipe
+      Toy::LLM::Recipes::WarmStartCuda.new
+    end
+  end
+end
+
+# Run-bundle writer (toy#73 item 1): runs/<id>/events.jsonl in the toy/v1
+# schema + the weights/ checkpoint-dir convention. SHARED (not mirrored):
+# the events C symbols live in libtinynn_ggml.a, linked by every backend
+# (the CPU TinyNN module is defined in this binary too — see the
+# checkpoint-seam note above). Required AFTER the Toy::Device module —
+# run_start! stamps backend{kind: Toy::Device.name} (toy#73 A.3).
+require_relative "io/run_bundle"

data/lib/toy/compute_metal.rb

@@ -0,0 +1,97 @@
+# lib/toy/compute_metal.rb — the Toy compute surface, Metal entry
+# (macOS ONLY).
+#
+# ── LOCKSTEP WARNING ── hand-maintained twin of lib/toy/compute.rb /
+# compute_cuda.rb (see compute_cuda.rb's header for why this is not
+# mechanically mirrorable). When compute.rb changes, update all three
+# entries in the same commit.
+#
+# DEVICE IS CHOSEN AT COMPILE TIME (toy#64 item 8): require THIS file
+# instead of lib/toy/compute.rb on a Mac and the same Toy::Device-
+# driven experiment body compiles into a Metal binary. Build with the
+# Apple frameworks + force-link, exactly like every metal target:
+#   spinel --cc='cc -Wl,-u,_tnn_metal_force_link -framework Foundation \
+#                -framework Metal -framework MetalKit' <entry> -o <bin>
+#   (deps: tinynn/libtinynn_ggml.a tinynn/libtinynn_ggml_metal.a)
+#
+# gx10 RUNTIME-UNVERIFIED (Linux cannot build or run Metal — same
+# status as every *_metal target; validate on the Mac via gate-metal's
+# discipline). SINGLE-TYPE BINARY: TinyNNMetal is the compute module;
+# CPU TinyNN stays defined for the checkpoint write/fuse seam.
+#
+# Backend gaps (consumers needing more require the files directly):
+#   - warm-start: NO Metal recipe exists (from-scratch only, matching
+#     libexec/toy-train-metal) — Device.warm_start_recipe is absent.
+#   - vit: no Metal engine. lora: no Metal recipe (a backend gap only —
+#     the toy#52 exclusion is history, see compute.rb's HISTORY note).
+
+# FFI compute layer: Metal TinyNN + the MRI-safe sugar (Mat + Card).
+require_relative "ffi/tinynn_metal"
+require_relative "../toy"
+
+# Models + configs (shared).
+require_relative "models/toy_smollm2"
+require_relative "io/loaders/toy_smollm2_loader"
+require_relative "models/toy_vit"
+
+# Engines — the Metal mirrors (generated by prep/gen_cuda_mirror.rb).
+require_relative "llm/engine/llama_seq_engine_metal"
+require_relative "llm/engine/gpt2_seq_engine_metal"
+
+# KV-cache decode engine (inference), Metal mirror.
+require_relative "llm/engine/llama_kv_engine_metal"
+
+# Recipes — from-scratch only (see the backend-gaps note above).
+require_relative "llm/recipe_options"
+require_relative "llm/recipes/from_scratch_metal"
+
+# Inference I/O (shared): GGUF model loading + the BPE tokenizer.
+require_relative "io/gguf_load"
+require_relative "io/tokenizer"
+
+# Training I/O + schedule (toy#73 item 2): corpus loader + image loader +
+# cosine LR schedule. SHARED with the CPU entry (file I/O through C helpers
+# in libtinynn_ggml.a + pure Ruby) — see compute.rb's note. The image
+# loader rides along even though vit training stays CPU-only this arc
+# (the loader is backend-free; the missing piece is the engine/recipe).
+require_relative "io/toy_corpus_loader"
+require_relative "io/toy_image_loader"
+require_relative "train/toy_lr_schedule"
+
+# AdamW + labels + the validating batch wrapper (shared, pure Ruby).
+require_relative "llm/adamw"
+require_relative "llm/labels"
+require_relative "llm/training_batch"
+require_relative "llm/classify_batch"
+
+# ── Toy::Device, Metal arm — see the compute.rb doc block. NOTE: no
+# warm_start_recipe here (no Metal recipe exists); a device-agnostic
+# body that uses it will fail to compile against this entry — loudly,
+# at compile time, which is the correct failure mode.
+module Toy
+  module Device
+    def self.name
+      "metal"
+    end
+
+    def self.llama_engine
+      Toy::LLM::Engine::LlamaSeqEngineMetal.new
+    end
+
+    def self.gpt2_engine
+      Toy::LLM::Engine::GPT2SeqEngineMetal.new
+    end
+
+    def self.from_scratch_recipe
+      Toy::LLM::Recipes::FromScratchMetal.new
+    end
+  end
+end
+
+# Run-bundle writer (toy#73 item 1): runs/<id>/events.jsonl in the toy/v1
+# schema + the weights/ checkpoint-dir convention. SHARED (not mirrored):
+# the events C symbols live in libtinynn_ggml.a, linked by every backend
+# (the CPU TinyNN module is defined in this binary too — see the
+# checkpoint-seam note above). Required AFTER the Toy::Device module —
+# run_start! stamps backend{kind: Toy::Device.name} (toy#73 A.3).
+require_relative "io/run_bundle"