torch-type-nn 0.1.0.dev0__tar.gz → 0.2.0__tar.gz

torch_type_nn-0.2.0/BOARD.CUDA.md

@@ -0,0 +1,312 @@
+# CUDA board
+
+Numbers on this page come from `benchmarks/board.py --write-cuda`. They are
+**not** bit-identical to C `bench.c`; C stays the number-for-number backend
+on [BOARD.md](BOARD.md). This file is the ragged device store
+(`CudaTypeNN`: packed `W (R, n_in)`, fused `X @ W.T`).
+
+`nix flake check --impure` skips `checks.cuda` unless the Nix daemon enables
+the `cuda` system feature. A GPU machine without that feature still runs the
+suite on the host:
+
+```sh
+nix run .#test-cuda          # pytest + xor on the device (prints the probe)
+nix run .#board-cuda         # 5-seed CUDA board + scale, then this file
+```
+
+or step by step:
+
+```sh
+nix run .#bench-cuda -- all cuda-type-nn,cuda-type-nn-overfit,cuda-graph-type-nn,mlp \
+    --seeds 5 --device cuda --jobs 0 \
+    > benchmarks/results/board-cuda.jsonl
+nix run .#scale-cuda -- all \
+    --models cuda-type-nn,cuda-type-nn-overfit,cuda-graph-type-nn,mlp-16,mlp-64 \
+    --seeds 5 --device cuda --jobs 0 \
+    > benchmarks/results/scale-cuda.jsonl
+nix run .#board -- \
+    --board benchmarks/results/board-cuda.jsonl benchmarks/results/board-per-sample.jsonl \
+    --scale benchmarks/results/scale-cuda.jsonl benchmarks/results/scale-batch32.jsonl \
+    --write-cuda BOARD.CUDA.md
+```
+
+Look for `======== torch-type-nn CUDA probe ========` and
+`torch-type-nn device: CUDA …` in the log. A green run with only
+`torch-type-nn device: CPU` did not use a GPU.
+
+## Why C rows disagreed across the two boards
+
+Two different gaps, not one:
+
+1. **C type-nn on BOARD.md vs C type-nn on BOARD.CUDA.md.**
+   `board.py --write-cuda` used to merge `benchmarks/reference/c-type-nn-7333bf7-30seeds.jsonl`
+   (C binary, 30 seeds) into this file. BOARD.md's C rows are
+   **5-seed NativeTypeNN**. Those are not the same experiment. The CUDA
+   writer no longer reads that 30-seed file. Pass
+   `benchmarks/results/board-per-sample.jsonl` next to `board-cuda.jsonl`
+   so both markdown files show the same 5-seed C cells.
+
+2. **C type-nn vs cuda-type-nn (and the overfit pair).**
+   `CudaTypeNN` is the C *layout* (ragged `W (R, n_in)`, fused GEMM) with
+   a *thinner scaler*. It is not `type_nn_scale.c` / `type_nn_overfit_scale.c`.
+   Missing or simplified relative to C: pair-cache BIC, width probes
+   (new unit + consumer column at 0), the fold when a depth probe is
+   inserted, and C's exact identity-layer metric. So CUDA stays closer to
+   birth depth, grows fewer params, and can lose on xor-train / Friedman
+   while C grows. Forward+Adam on a *fixed* graph match C; the structure
+   trajectory does not. C remains the number-for-number backend.
+
+## Per-sample tasks
+
+Same tasks, split, epochs and lr as [BOARD.md](BOARD.md). CUDA rows are
+per-sample Adam on the device (C protocol, fused GEMV). MLP rows in the
+same jsonl were trained with `--device cuda`.
+
+<!-- board:per-sample -->
+Generated by `benchmarks/board.py --write-cuda` from `benchmarks/results/board-cuda.jsonl`, `benchmarks/results/board-per-sample.jsonl` (5 seeds, batch 1, cpu; 5 seeds, batch 1, cuda; cuda type-nn rows from CudaTypeNN).
+
+| task | model | hold MSE | ± sd | hold acc | train MSE | params | ± sd | layers | train s | µs/inf |
+|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| xor | C type-nn | n/a | n/a | n/a | 5.87e-09 | 26.4 | 3.6 | 1→2.0 | 0.09 | 55.4 |
+| xor | C type-nn-overfit | n/a | n/a | n/a | 1.39e-07 | 107.6 | 73.4 | 1→3.0 | 0.09 | 79.4 |
+| xor | torch ref-type-nn | n/a | n/a | n/a | 8.40e-06 | 24.8 | 1.8 | 1→2.0 | 45.49 | 419.0 |
+| xor | torch ref-type-nn-overfit | n/a | n/a | n/a | 4.90e-06 | 56.0 | 17.0 | 1→2.4 | 52.21 | 313.7 |
+| xor | **torch type-nn** | n/a | n/a | n/a | 5.11e-06 | 63.0 | 23.7 | 1→2.4 | 41.93 | 541.8 |
+| xor | torch type-nn-bic | n/a | n/a | n/a | 8.40e-06 | 24.8 | 1.8 | 1→2.0 | 39.84 | 677.9 |
+| xor | torch MLP | n/a | n/a | n/a | 7.81e-10 | 33.0 | 0.0 | 2→2.0 | 24.91 | 352.5 |
+| xor | torch MLP, scaled | n/a | n/a | n/a | 8.34e-02 | 39.0 | 37.1 | 2→3.2 | 31.02 | 196.0 |
+| xor | cuda type-nn | n/a | n/a | n/a | 6.74e-06 | 24.8 | 1.8 | 1→2.0 | 412.81 | 4432.8 |
+| xor | cuda type-nn-overfit | n/a | n/a | n/a | 1.04e-06 | 73.6 | 36.0 | 1→2.6 | 322.93 | 7841.3 |
+| xor | C graph-type-nn | n/a | n/a | n/a | 3.11e-07 | 24.8 | 1.8 | 1→2.0 | 0.20 | 61.9 |
+| xor | **torch graph-type-nn** | n/a | n/a | n/a | 5.00e-02 | 20.0 | 8.9 | 1→1.8 | 80.70 | 1391.5 |
+| xor | cuda graph-type-nn | n/a | n/a | n/a | 5.00e-02 | 20.0 | 8.9 | 1→1.8 | 314.00 | 7123.7 |
+| iris | C type-nn | 0.0215 | 0.0069 | 0.969 | 8.43e-04 | 125.4 | 52.2 | 3→3.8 | 0.96 | 26.6 |
+| iris | C type-nn-overfit | 0.0249 | 0.0044 | 0.960 | 1.81e-03 | 174.6 | 29.8 | 3→4.0 | 0.30 | 60.2 |
+| iris | torch ref-type-nn | 0.0250 | 0.0063 | 0.960 | 8.26e-05 | 139.0 | 25.9 | 3→4.0 | 263.82 | 1475.9 |
+| iris | torch ref-type-nn-overfit | 0.0261 | 0.0075 | 0.956 | 2.64e-03 | 167.6 | 19.3 | 3→4.0 | 269.55 | 1087.1 |
+| iris | **torch type-nn** | 0.0250 | 0.0038 | 0.956 | 1.08e-04 | 190.0 | 18.4 | 3→4.0 | 252.28 | 871.5 |
+| iris | torch type-nn-bic | 0.0377 | 0.0303 | 0.956 | 1.38e-02 | 147.4 | 72.9 | 3→4.0 | 270.28 | 805.0 |
+| iris | torch MLP | 0.0309 | 0.0043 | 0.956 | 1.48e-02 | 67.0 | 0.0 | 2→2.0 | 36.66 | 153.0 |
+| iris | torch MLP, scaled | 0.0262 | 0.0054 | 0.956 | 5.89e-03 | 71.8 | 16.3 | 2→3.0 | 102.53 | 203.6 |
+| iris | cuda type-nn | 0.0244 | 0.0049 | 0.960 | 1.57e-03 | 145.2 | 35.0 | 3→4.0 | 1613.89 | 17826.8 |
+| iris | cuda type-nn-overfit | 0.0404 | 0.0354 | 0.956 | 1.57e-02 | 189.8 | 38.3 | 3→4.0 | 1487.48 | 22645.5 |
+| iris | C graph-type-nn | 0.0218 | 0.0065 | 0.969 | 8.47e-04 | 80.6 | 16.6 | 3→3.8 | 0.94 | 33.3 |
+| iris | **torch graph-type-nn** | 0.0216 | 0.0054 | 0.964 | 2.66e-03 | 102.0 | 36.5 | 3→3.8 | 347.44 | 2088.9 |
+| iris | cuda graph-type-nn | 0.0263 | 0.0029 | 0.956 | 3.23e-03 | 81.6 | 24.9 | 3→3.8 | 1067.25 | 11004.1 |
+| wine | C type-nn | 0.0277 | 0.0048 | 0.956 | 7.85e-05 | 169.8 | 52.3 | 4→5.0 | 1.45 | 45.8 |
+| wine | C type-nn-overfit | 0.0209 | 0.0105 | 0.967 | 1.61e-04 | 392.6 | 43.7 | 4→5.0 | 0.28 | 38.4 |
+| wine | torch ref-type-nn | 0.0130 | 0.0064 | 0.978 | 2.35e-05 | 211.6 | 32.9 | 4→5.0 | 306.89 | 1004.1 |
+| wine | torch ref-type-nn-overfit | 0.0322 | 0.0112 | 0.944 | 4.01e-05 | 358.6 | 27.6 | 4→5.0 | 301.92 | 1205.0 |
+| wine | **torch type-nn** | 0.0242 | 0.0069 | 0.959 | 2.85e-05 | 469.6 | 35.7 | 4→5.0 | 296.13 | 1686.3 |
+| wine | torch type-nn-bic | 0.0408 | 0.0415 | 0.941 | 1.44e-02 | 152.8 | 103.7 | 4→5.0 | 304.39 | 858.0 |
+| wine | torch MLP | 0.0259 | 0.0032 | 0.956 | 3.87e-03 | 275.0 | 0.0 | 2→2.0 | 33.32 | 148.6 |
+| wine | torch MLP, scaled | 0.0145 | 0.0045 | 0.974 | 2.85e-04 | 124.6 | 32.7 | 2→3.0 | 89.95 | 228.6 |
+| wine | cuda type-nn | 0.0208 | 0.0043 | 0.959 | 2.67e-05 | 214.6 | 44.8 | 4→5.0 | 1952.68 | 19216.9 |
+| wine | cuda type-nn-overfit | 0.0220 | 0.0094 | 0.963 | 1.21e-05 | 470.2 | 52.3 | 4→5.0 | 1761.65 | 14747.7 |
+| wine | C graph-type-nn | 0.0184 | 0.0039 | 0.967 | 1.08e-04 | 165.8 | 49.1 | 4→5.4 | 2.48 | 74.7 |
+| wine | **torch graph-type-nn** | 0.0148 | 0.0068 | 0.974 | 3.69e-04 | 145.8 | 47.8 | 4→4.8 | 426.05 | 2604.5 |
+| wine | cuda graph-type-nn | 0.0191 | 0.0075 | 0.963 | 3.46e-04 | 122.8 | 50.8 | 4→4.8 | 1340.38 | 14599.9 |
+| wdbc | C type-nn | 0.0448 | 0.0048 | 0.951 | 1.01e-04 | 162.8 | 28.7 | 3→4.0 | 0.38 | 49.4 |
+| wdbc | C type-nn-overfit | 0.0459 | 0.0030 | 0.950 | 4.22e-04 | 162.6 | 31.2 | 3→3.2 | 0.11 | 33.4 |
+| wdbc | torch ref-type-nn | 0.0475 | 0.0065 | 0.945 | 2.61e-04 | 175.8 | 36.3 | 3→4.0 | 309.73 | 830.8 |
+| wdbc | torch ref-type-nn-overfit | 0.0454 | 0.0080 | 0.949 | 7.91e-04 | 171.6 | 57.4 | 3→3.6 | 303.12 | 1184.8 |
+| wdbc | **torch type-nn** | 0.0465 | 0.0044 | 0.947 | 3.69e-04 | 233.4 | 32.9 | 3→3.6 | 296.80 | 758.3 |
+| wdbc | torch type-nn-bic | 0.0467 | 0.0059 | 0.947 | 1.64e-03 | 211.0 | 84.7 | 3→4.0 | 323.76 | 1019.8 |
+| wdbc | torch MLP | 0.0471 | 0.0040 | 0.944 | 5.70e-03 | 513.0 | 0.0 | 2→2.0 | 46.26 | 173.4 |
+| wdbc | torch MLP, scaled | 0.0539 | 0.0079 | 0.943 | 1.84e-03 | 132.0 | 23.3 | 2→3.0 | 120.17 | 214.2 |
+| wdbc | cuda type-nn | 0.0435 | 0.0028 | 0.952 | 9.64e-05 | 207.8 | 34.3 | 3→4.0 | 1279.90 | 3551.4 |
+| wdbc | cuda type-nn-overfit | 0.0475 | 0.0035 | 0.945 | 4.42e-04 | 211.2 | 41.6 | 3→3.4 | 1059.34 | 2187.8 |
+| wdbc | C graph-type-nn | 0.0398 | 0.0035 | 0.954 | 7.75e-04 | 128.4 | 55.0 | 3→4.0 | 0.78 | 31.0 |
+| wdbc | **torch graph-type-nn** | 0.0421 | 0.0044 | 0.956 | 4.02e-03 | 90.8 | 56.2 | 3→4.0 | 387.83 | 897.5 |
+| wdbc | cuda graph-type-nn | 0.0495 | 0.0076 | 0.950 | 9.35e-04 | 147.0 | 62.6 | 3→4.0 | 567.67 | 2806.1 |
+| diabetes | C type-nn | 0.0329 | 0.0010 | n/a | 2.73e-02 | 20.4 | 7.2 | 2→2.4 | 0.10 | 33.5 |
+| diabetes | C type-nn-overfit | 0.0371 | 0.0015 | n/a | 2.40e-02 | 33.0 | 8.5 | 2→2.2 | 0.09 | 38.7 |
+| diabetes | torch ref-type-nn | 0.0328 | 0.0005 | n/a | 2.73e-02 | 18.6 | 4.9 | 2→2.2 | 322.97 | 272.7 |
+| diabetes | torch ref-type-nn-overfit | 0.0366 | 0.0030 | n/a | 2.43e-02 | 31.8 | 7.8 | 2→2.4 | 328.22 | 629.4 |
+| diabetes | **torch type-nn** | 0.0501 | 0.0145 | n/a | 1.75e-02 | 100.0 | 42.6 | 2→3.0 | 353.05 | 657.9 |
+| diabetes | torch type-nn-bic | 0.0330 | 0.0006 | n/a | 2.78e-02 | 18.0 | 2.1 | 2→2.6 | 338.44 | 261.5 |
+| diabetes | torch MLP | 0.0478 | 0.0038 | n/a | 1.47e-02 | 193.0 | 0.0 | 2→2.0 | 66.53 | 164.3 |
+| diabetes | torch MLP, scaled | 0.0410 | 0.0017 | n/a | 2.05e-02 | 61.6 | 10.3 | 2→3.0 | 172.25 | 97.0 |
+| diabetes | cuda type-nn | 0.0328 | 0.0006 | n/a | 2.78e-02 | 17.4 | 1.3 | 2→2.8 | 1938.75 | 10158.2 |
+| diabetes | cuda type-nn-overfit | 0.0453 | 0.0053 | n/a | 1.74e-02 | 98.4 | 22.1 | 2→3.0 | 1966.89 | 9561.5 |
+| diabetes | C graph-type-nn | 0.0332 | 0.0012 | n/a | 2.73e-02 | 18.8 | 6.9 | 2→2.2 | 0.22 | 83.5 |
+| diabetes | **torch graph-type-nn** | 0.0332 | 0.0012 | n/a | 2.73e-02 | 18.8 | 6.9 | 2→2.2 | 393.13 | 1256.7 |
+| diabetes | cuda graph-type-nn | 0.0332 | 0.0012 | n/a | 2.73e-02 | 18.8 | 6.9 | 2→2.2 | 1184.15 | 4112.5 |
+| ionosphere | C type-nn | 0.0886 | 0.0111 | 0.904 | 9.70e-03 | 153.8 | 65.0 | 4→4.4 | 0.80 | 93.3 |
+| ionosphere | C type-nn-overfit | 0.0813 | 0.0119 | 0.913 | 4.33e-03 | 298.0 | 60.5 | 4→4.4 | 0.38 | 68.7 |
+| ionosphere | torch ref-type-nn | 0.1060 | 0.0412 | 0.891 | 3.56e-03 | 192.0 | 17.0 | 4→5.2 | 358.87 | 1182.7 |
+| ionosphere | torch ref-type-nn-overfit | 0.0855 | 0.0108 | 0.906 | 2.44e-03 | 314.8 | 65.4 | 4→4.6 | 333.98 | 1213.1 |
+| ionosphere | **torch type-nn** | 0.0767 | 0.0087 | 0.911 | 5.62e-05 | 389.0 | 63.8 | 4→4.8 | 338.53 | 699.1 |
+| ionosphere | torch type-nn-bic | 0.1096 | 0.0176 | 0.875 | 1.53e-02 | 178.8 | 81.5 | 4→4.4 | 337.81 | 545.1 |
+| ionosphere | torch MLP | 0.1356 | 0.0128 | 0.908 | 1.57e-02 | 577.0 | 0.0 | 2→2.0 | 43.09 | 158.7 |
+| ionosphere | torch MLP, scaled | 0.1142 | 0.0235 | 0.885 | 8.43e-03 | 167.8 | 17.1 | 2→3.0 | 107.52 | 96.4 |
+| ionosphere | cuda type-nn | 0.1001 | 0.0113 | 0.877 | 2.13e-02 | 167.2 | 124.5 | 4→4.2 | 1729.76 | 8618.0 |
+| ionosphere | cuda type-nn-overfit | 0.1028 | 0.0333 | 0.887 | 1.24e-02 | 336.2 | 35.9 | 4→4.8 | 1550.01 | 15940.8 |
+| ionosphere | C graph-type-nn | 0.0862 | 0.0209 | 0.902 | 7.79e-03 | 221.4 | 133.0 | 4→4.8 | 1.56 | 56.7 |
+| ionosphere | **torch graph-type-nn** | 0.0967 | 0.0244 | 0.885 | 2.42e-02 | 128.2 | 146.5 | 4→3.6 | 433.01 | 1059.0 |
+| ionosphere | cuda graph-type-nn | 0.0978 | 0.0403 | 0.879 | 1.46e-02 | 239.0 | 163.6 | 4→4.2 | 699.41 | 7323.0 |
+<!-- /board:per-sample -->
+
+## Comparisons
+
+Hold-out gap is **clear** when it exceeds two standard errors of the
+difference, `2 sqrt(sd_a²/n_a + sd_b²/n_b)`.
+
+<!-- board:pairs -->
+| task | comparison | Δ hold MSE | 2se | verdict | params ratio |
+|---|---|---:|---:|---|---:|
+| xor | cuda type-nn vs torch MLP | n/a | n/a | both fit (no hold-out) | 0.75× |
+| xor | cuda type-nn-overfit vs torch MLP | n/a | n/a | both fit (no hold-out) | 2.23× |
+| xor | cuda type-nn vs cuda type-nn-overfit | n/a | n/a | both fit (no hold-out) | 0.34× |
+| xor | cuda type-nn vs C type-nn | n/a | n/a | both fit (no hold-out) | 0.94× |
+| xor | cuda type-nn-overfit vs C type-nn-overfit | n/a | n/a | both fit (no hold-out) | 0.68× |
+| iris | cuda type-nn vs torch MLP | -0.0065 | 0.0059 | lower, **clear** | 2.17× |
+| iris | cuda type-nn-overfit vs torch MLP | +0.0095 | 0.0319 | higher, within noise | 2.83× |
+| iris | cuda type-nn vs cuda type-nn-overfit | -0.0160 | 0.0320 | lower, within noise | 0.77× |
+| iris | cuda type-nn vs C type-nn | +0.0029 | 0.0076 | higher, within noise | 1.16× |
+| iris | cuda type-nn-overfit vs C type-nn-overfit | +0.0156 | 0.0319 | higher, within noise | 1.09× |
+| wine | cuda type-nn vs torch MLP | -0.0050 | 0.0048 | lower, **clear** | 0.78× |
+| wine | cuda type-nn-overfit vs torch MLP | -0.0039 | 0.0089 | lower, within noise | 1.71× |
+| wine | cuda type-nn vs cuda type-nn-overfit | -0.0011 | 0.0092 | lower, within noise | 0.46× |
+| wine | cuda type-nn vs C type-nn | -0.0068 | 0.0058 | lower, **clear** | 1.26× |
+| wine | cuda type-nn-overfit vs C type-nn-overfit | +0.0011 | 0.0126 | higher, within noise | 1.20× |
+| wdbc | cuda type-nn vs torch MLP | -0.0037 | 0.0044 | lower, within noise | 0.41× |
+| wdbc | cuda type-nn-overfit vs torch MLP | +0.0003 | 0.0047 | higher, within noise | 0.41× |
+| wdbc | cuda type-nn vs cuda type-nn-overfit | -0.0040 | 0.0040 | lower, within noise | 0.98× |
+| wdbc | cuda type-nn vs C type-nn | -0.0014 | 0.0050 | lower, within noise | 1.28× |
+| wdbc | cuda type-nn-overfit vs C type-nn-overfit | +0.0016 | 0.0041 | higher, within noise | 1.30× |
+| diabetes | cuda type-nn vs torch MLP | -0.0150 | 0.0034 | lower, **clear** | 0.09× |
+| diabetes | cuda type-nn-overfit vs torch MLP | -0.0024 | 0.0058 | lower, within noise | 0.51× |
+| diabetes | cuda type-nn vs cuda type-nn-overfit | -0.0125 | 0.0047 | lower, **clear** | 0.18× |
+| diabetes | cuda type-nn vs C type-nn | -0.0002 | 0.0010 | lower, within noise | 0.85× |
+| diabetes | cuda type-nn-overfit vs C type-nn-overfit | +0.0082 | 0.0049 | higher, **clear** | 2.98× |
+| ionosphere | cuda type-nn vs torch MLP | -0.0355 | 0.0153 | lower, **clear** | 0.29× |
+| ionosphere | cuda type-nn-overfit vs torch MLP | -0.0328 | 0.0319 | lower, **clear** | 0.58× |
+| ionosphere | cuda type-nn vs cuda type-nn-overfit | -0.0027 | 0.0315 | lower, within noise | 0.50× |
+| ionosphere | cuda type-nn vs C type-nn | +0.0116 | 0.0142 | higher, within noise | 1.09× |
+| ionosphere | cuda type-nn-overfit vs C type-nn-overfit | +0.0216 | 0.0316 | higher, within noise | 1.13× |
+<!-- /board:pairs -->
+
+## graph-type-nn
+
+The graph variant has three implementations of one rule (type-nn's
+`type_nn_graph.c`): **C graph-type-nn** is the C code itself
+(`NativeGraphTypeNN`), **torch graph-type-nn** is `GraphTypeNN` (AndOr nodes,
+autograd, `TypeAdam`) and **cuda graph-type-nn** is `CudaGraphTypeNN` (packed
+nodes, hand-written backward, on the device). All three draw from C's
+xorshift32 stream in C's order, so they are born with the same weights and,
+up to floating-point rounding, make the same structural decisions. The port
+rows test that; "the goal" row is the graph against the chain (C type-nn):
+the same hold-out with fewer parameters.
+
+<!-- board:graph -->
+Generated by `benchmarks/board.py --write-cuda` from `benchmarks/results/board-cuda.jsonl`, `benchmarks/results/board-per-sample.jsonl`.
+
+| task | model | hold MSE | ± sd | hold acc | train MSE | params | ± sd | layers | edges | train s | µs/inf |
+|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| xor | C graph-type-nn | n/a | n/a | n/a | 3.11e-07 | 24.8 | 1.8 | 1→2.0 | 2.0 | 0.20 | 61.9 |
+| xor | **torch graph-type-nn** | n/a | n/a | n/a | 5.00e-02 | 20.0 | 8.9 | 1→1.8 | 1.8 | 80.70 | 1391.5 |
+| xor | cuda graph-type-nn | n/a | n/a | n/a | 5.00e-02 | 20.0 | 8.9 | 1→1.8 | 1.8 | 314.00 | 7123.7 |
+| iris | C graph-type-nn | 0.0218 | 0.0065 | 0.969 | 8.47e-04 | 80.6 | 16.6 | 3→3.8 | 5.2 | 0.94 | 33.3 |
+| iris | **torch graph-type-nn** | 0.0216 | 0.0054 | 0.964 | 2.66e-03 | 102.0 | 36.5 | 3→3.8 | 5.2 | 347.44 | 2088.9 |
+| iris | cuda graph-type-nn | 0.0263 | 0.0029 | 0.956 | 3.23e-03 | 81.6 | 24.9 | 3→3.8 | 5.0 | 1067.25 | 11004.1 |
+| wine | C graph-type-nn | 0.0184 | 0.0039 | 0.967 | 1.08e-04 | 165.8 | 49.1 | 4→5.4 | 8.2 | 2.48 | 74.7 |
+| wine | **torch graph-type-nn** | 0.0148 | 0.0068 | 0.974 | 3.69e-04 | 145.8 | 47.8 | 4→4.8 | 6.6 | 426.05 | 2604.5 |
+| wine | cuda graph-type-nn | 0.0191 | 0.0075 | 0.963 | 3.46e-04 | 122.8 | 50.8 | 4→4.8 | 6.6 | 1340.38 | 14599.9 |
+| wdbc | C graph-type-nn | 0.0398 | 0.0035 | 0.954 | 7.75e-04 | 128.4 | 55.0 | 3→4.0 | 5.2 | 0.78 | 31.0 |
+| wdbc | **torch graph-type-nn** | 0.0421 | 0.0044 | 0.956 | 4.02e-03 | 90.8 | 56.2 | 3→4.0 | 4.8 | 387.83 | 897.5 |
+| wdbc | cuda graph-type-nn | 0.0495 | 0.0076 | 0.950 | 9.35e-04 | 147.0 | 62.6 | 3→4.0 | 5.6 | 567.67 | 2806.1 |
+| diabetes | C graph-type-nn | 0.0332 | 0.0012 | n/a | 2.73e-02 | 18.8 | 6.9 | 2→2.2 | 2.4 | 0.22 | 83.5 |
+| diabetes | **torch graph-type-nn** | 0.0332 | 0.0012 | n/a | 2.73e-02 | 18.8 | 6.9 | 2→2.2 | 2.4 | 393.13 | 1256.7 |
+| diabetes | cuda graph-type-nn | 0.0332 | 0.0012 | n/a | 2.73e-02 | 18.8 | 6.9 | 2→2.2 | 2.4 | 1184.15 | 4112.5 |
+| ionosphere | C graph-type-nn | 0.0862 | 0.0209 | 0.902 | 7.79e-03 | 221.4 | 133.0 | 4→4.8 | 7.4 | 1.56 | 56.7 |
+| ionosphere | **torch graph-type-nn** | 0.0967 | 0.0244 | 0.885 | 2.42e-02 | 128.2 | 146.5 | 4→3.6 | 4.8 | 433.01 | 1059.0 |
+| ionosphere | cuda graph-type-nn | 0.0978 | 0.0403 | 0.879 | 1.46e-02 | 239.0 | 163.6 | 4→4.2 | 6.0 | 699.41 | 7323.0 |
+
+| task | comparison | Δ hold MSE | 2se | verdict | params ratio |
+|---|---|---:|---:|---|---:|
+| xor | **torch graph-type-nn** vs C graph-type-nn | n/a | n/a | train acc 0.95 vs 1.00 (no hold-out) | 0.81× |
+| xor | cuda graph-type-nn vs C graph-type-nn | n/a | n/a | train acc 0.95 vs 1.00 (no hold-out) | 0.81× |
+| xor | C graph-type-nn vs C type-nn | n/a | n/a | both fit (no hold-out) | 0.94× |
+| xor | **torch graph-type-nn** vs torch ref-type-nn | n/a | n/a | train acc 0.95 vs 1.00 (no hold-out) | 0.81× |
+| xor | **torch graph-type-nn** vs torch MLP | n/a | n/a | train acc 0.95 vs 1.00 (no hold-out) | 0.61× |
+| iris | **torch graph-type-nn** vs C graph-type-nn | -0.0002 | 0.0075 | lower, within noise | 1.27× |
+| iris | cuda graph-type-nn vs C graph-type-nn | +0.0045 | 0.0064 | higher, within noise | 1.01× |
+| iris | C graph-type-nn vs C type-nn | +0.0003 | 0.0085 | higher, within noise | 0.64× |
+| iris | **torch graph-type-nn** vs torch ref-type-nn | -0.0034 | 0.0074 | lower, within noise | 0.73× |
+| iris | **torch graph-type-nn** vs torch MLP | -0.0093 | 0.0062 | lower, **clear** | 1.52× |
+| wine | **torch graph-type-nn** vs C graph-type-nn | -0.0036 | 0.0070 | lower, within noise | 0.88× |
+| wine | cuda graph-type-nn vs C graph-type-nn | +0.0007 | 0.0075 | higher, within noise | 0.74× |
+| wine | C graph-type-nn vs C type-nn | -0.0093 | 0.0055 | lower, **clear** | 0.98× |
+| wine | **torch graph-type-nn** vs torch ref-type-nn | +0.0018 | 0.0084 | higher, within noise | 0.69× |
+| wine | **torch graph-type-nn** vs torch MLP | -0.0111 | 0.0067 | lower, **clear** | 0.53× |
+| wdbc | **torch graph-type-nn** vs C graph-type-nn | +0.0023 | 0.0050 | higher, within noise | 0.71× |
+| wdbc | cuda graph-type-nn vs C graph-type-nn | +0.0097 | 0.0074 | higher, **clear** | 1.14× |
+| wdbc | C graph-type-nn vs C type-nn | -0.0051 | 0.0053 | lower, within noise | 0.79× |
+| wdbc | **torch graph-type-nn** vs torch ref-type-nn | -0.0054 | 0.0071 | lower, within noise | 0.52× |
+| wdbc | **torch graph-type-nn** vs torch MLP | -0.0051 | 0.0053 | lower, within noise | 0.18× |
+| diabetes | **torch graph-type-nn** vs C graph-type-nn | -0.0000 | 0.0016 | lower, within noise | 1.00× |
+| diabetes | cuda graph-type-nn vs C graph-type-nn | -0.0000 | 0.0016 | lower, within noise | 1.00× |
+| diabetes | C graph-type-nn vs C type-nn | +0.0003 | 0.0014 | higher, within noise | 0.92× |
+| diabetes | **torch graph-type-nn** vs torch ref-type-nn | +0.0004 | 0.0012 | higher, within noise | 1.01× |
+| diabetes | **torch graph-type-nn** vs torch MLP | -0.0145 | 0.0035 | lower, **clear** | 0.10× |
+| ionosphere | **torch graph-type-nn** vs C graph-type-nn | +0.0105 | 0.0288 | higher, within noise | 0.58× |
+| ionosphere | cuda graph-type-nn vs C graph-type-nn | +0.0116 | 0.0406 | higher, within noise | 1.08× |
+| ionosphere | C graph-type-nn vs C type-nn | -0.0023 | 0.0212 | lower, within noise | 1.44× |
+| ionosphere | **torch graph-type-nn** vs torch ref-type-nn | -0.0093 | 0.0428 | lower, within noise | 0.67× |
+| ionosphere | **torch graph-type-nn** vs torch MLP | -0.0389 | 0.0246 | lower, **clear** | 0.22× |
+<!-- /board:graph -->
+
+## Larger data
+
+Same `scale.py` tasks as [BOARD.md](BOARD.md) (digits, Friedman #1). CUDA
+rows use `CudaTypeNN.epoch` (per-sample on the device). MLP-16 / MLP-64 in
+this file are the batched baseline on the same `--device`.
+
+<!-- board:scale -->
+Generated by `benchmarks/board.py` from `benchmarks/results/scale-cuda.jsonl`, `benchmarks/results/scale-batch32.jsonl` (5 seeds, batch 1, cpu; 5 seeds, batch 1, cuda; 5 seeds, batch 32, cpu; 5 seeds, batch 32, cuda).
+
+| task | model | hold MSE | ± sd | × noise floor | hold acc | train MSE | params | layers | train s |
+|---|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| digits | **type-nn** | 0.00620 | 0.00131 | n/a | 0.963 | 0.00023 | 5509 | 7.0 | 47.1 |
+| digits | type-nn-bic | 0.00696 | 0.00028 | n/a | 0.957 | 0.00014 | 3352 | 7.0 | 190.9 |
+| digits | MLP, scaled | 0.00779 | 0.00053 | n/a | 0.964 | 0.00170 | 1323 | 3.0 | 14.9 |
+| digits | MLP-16 | 0.01110 | 0.00074 | n/a | 0.959 | 0.00490 | 1210 | 2.0 | 4.2 |
+| digits | MLP-64 | 0.01089 | 0.00022 | n/a | 0.974 | 0.00343 | 4810 | 2.0 | 3.4 |
+| digits | C type-nn | 0.00708 | 0.00086 | n/a | 0.956 | 0.00006 | 3289 | 7.0 | 198.8 |
+| digits | C type-nn-overfit | 0.00610 | 0.00165 | n/a | 0.964 | 0.00008 | 3712 | 6.6 | 8.7 |
+| digits | cuda type-nn | 0.00711 | 0.00100 | n/a | 0.957 | 0.00019 | 3247 | 7.0 | 5501.9 |
+| digits | cuda type-nn-overfit | 0.00688 | 0.00125 | n/a | 0.956 | 0.00010 | 3662 | 6.0 | 5004.0 |
+| digits | C graph-type-nn | 0.00623 | 0.00086 | n/a | 0.959 | 0.00025 | 2453 | 7.0 | 192.7 |
+| digits | **graph-type-nn** | 0.07058 | 0.04228 | n/a | 0.416 | 0.06746 | 542 | 4.4 | 193.9 |
+| digits | cuda graph-type-nn | 0.00647 | 0.00068 | n/a | 0.957 | 0.00027 | 2392 | 7.0 | 2675.8 |
+| friedman | **type-nn** | 0.00660 | 0.00037 | 6.65 | n/a | 0.00666 | 18 | 2.0 | 247.5 |
+| friedman | type-nn-bic | 0.00195 | 0.00089 | 1.97 | n/a | 0.00194 | 80 | 3.0 | 286.6 |
+| friedman | MLP, scaled | 0.01377 | 0.01039 | 13.88 | n/a | 0.01362 | 17 | 2.6 | 127.6 |
+| friedman | MLP-16 | 0.00121 | 0.00013 | 1.22 | n/a | 0.00118 | 193 | 2.0 | 51.6 |
+| friedman | MLP-64 | 0.00114 | 0.00005 | 1.15 | n/a | 0.00110 | 769 | 2.0 | 48.5 |
+| friedman | C type-nn | 0.00532 | 0.00070 | 5.37 | n/a | 0.00536 | 36 | 3.0 | 5.8 |
+| friedman | C type-nn-overfit | 0.00669 | 0.00003 | 6.74 | n/a | 0.00676 | 15 | 2.0 | 2.7 |
+| friedman | cuda type-nn | 0.00152 | 0.00032 | 1.53 | n/a | 0.00151 | 79 | 3.0 | 27321.4 |
+| friedman | cuda type-nn-overfit | 0.00670 | 0.00003 | 6.75 | n/a | 0.00677 | 15 | 2.0 | 24903.1 |
+| friedman | C graph-type-nn | 0.00125 | 0.00016 | 1.26 | n/a | 0.00124 | 83 | 3.0 | 15.4 |
+| friedman | **graph-type-nn** | 0.00115 | 0.00007 | 1.16 | n/a | 0.00114 | 114 | 3.0 | 177.1 |
+| friedman | cuda graph-type-nn | 0.00125 | 0.00016 | 1.26 | n/a | 0.00124 | 83 | 3.0 | 14542.3 |
+
+friedman: noise floor (best reachable hold-out MSE, `1 / span²`) = 0.00099
+
+| task | comparison | Δ hold MSE | 2se | verdict | params ratio |
+|---|---|---:|---:|---|---:|
+| digits | cuda type-nn vs MLP-16 | -0.0040 | 0.0011 | lower, **clear** | 2.68× |
+| digits | cuda type-nn-overfit vs MLP-16 | -0.0042 | 0.0013 | lower, **clear** | 3.03× |
+| digits | cuda type-nn vs cuda type-nn-overfit | +0.0002 | 0.0014 | higher, within noise | 0.89× |
+| digits | cuda type-nn vs C type-nn | +0.0000 | 0.0012 | higher, within noise | 0.99× |
+| digits | cuda graph-type-nn vs C graph-type-nn | +0.0002 | 0.0010 | higher, within noise | 0.98× |
+| digits | cuda graph-type-nn vs MLP-16 | -0.0046 | 0.0009 | lower, **clear** | 1.98× |
+| friedman | cuda type-nn vs MLP-16 | +0.0003 | 0.0003 | higher, **clear** | 0.41× |
+| friedman | cuda type-nn-overfit vs MLP-16 | +0.0055 | 0.0001 | higher, **clear** | 0.08× |
+| friedman | cuda type-nn vs cuda type-nn-overfit | -0.0052 | 0.0003 | lower, **clear** | 5.27× |
+| friedman | cuda type-nn vs C type-nn | -0.0038 | 0.0007 | lower, **clear** | 2.19× |
+| friedman | cuda graph-type-nn vs C graph-type-nn | +0.0000 | 0.0002 | higher, within noise | 1.00× |
+| friedman | cuda graph-type-nn vs MLP-16 | +0.0000 | 0.0002 | higher, within noise | 0.43× |
+<!-- /board:scale -->