diskformer 0.5.1 → 0.5.2

package/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.5.2 (2026-10-14)
+- `diskformer/llama` works when imported from a CDN: a `Llama` on another origin starts its worker from a same-origin
+  stub that imports `worker.mjs`, because browsers refuse a cross-origin worker URL. Checked from a localhost page
+  importing only from jsDelivr: download, load and a chat reply from Granite 4.0 H Tiny.
+- README: a quick start that runs as pasted; the llama.cpp path leads.
+- docs/coverage.md: diffusion MoE models are not supported. Paging is exact for them, but each denoising step reads
+  nearly all experts (about 3.9 GB per step for LLaDA-MoE 7B-A1B); llama.cpp's WebGPU output for it degrades (open).
+
 ## 0.5.1 (2026-10-14)
 - The browser build (`diskformer/llama`) is rebuilt on the fork after merging upstream llama.cpp (fork `377491df7`, 167
   upstream commits including the MUL_MAT_ID rewrite #30451); checksums in `ggml/dist/BUILD.md`. Every gated model was

package/README.md

@@ -1,10 +1,11 @@
 <h1 align="center">diskformer.js</h1>
 
-**Run a model larger than your GPU memory in a browser tab.** The weights stay on disk in the browser's private
-storage (OPFS); WebGPU holds only what each step needs. A disk tier for in-browser inference, with two reference
-engines that use it.
+**Run a mixture-of-experts model larger than your GPU memory in a browser tab, from one import.** llama.cpp runs on
+WebGPU in a worker; the GGUF stays on disk in the browser's private storage (OPFS), and only the experts each token
+needs are paged into GPU memory. 16 architectures are checked against llama.cpp, from Qwen3.6 35B to gpt-oss 20B.
 
-Plain ES modules, no dependencies, no build step. Chromium with WebGPU. Weights stay in the user's own browser storage.
+Plain ES modules, no dependencies, no build step. Chrome or Edge with WebGPU, on any origin (CDN or self-hosted).
+Weights stay in the user's own browser storage. The disk tier underneath (`src/`) also runs two hand-written engines.
 
 [![npm](https://img.shields.io/npm/v/diskformer?style=flat-square&color=555)](https://www.npmjs.com/package/diskformer)
 [![license](https://img.shields.io/badge/license-MIT-555?style=flat-square)](LICENSE)
@@ -17,9 +18,27 @@ Plain ES modules, no dependencies, no build step. Chromium with WebGPU. Weights
 | How | Command |
 |---|---|
 | npm | `npm install diskformer` |
-| CDN | `import { Llama } from 'https://cdn.jsdelivr.net/npm/diskformer@0.5/ggml/llama.js'` |
+| CDN | `import { Llama } from 'https://cdn.jsdelivr.net/npm/diskformer@0.5.2/ggml/llama.js'` |
 | Copy | `src/`, `engines/` and `index.js`; nothing else is needed |
 
+## Quick start
+
+Paste into any page served over http(s), in Chrome or Edge. The first run downloads the 4.2 GB GGUF from Hugging Face into
+OPFS (resumable); later loads take seconds. Any MoE GGUF on a [checked architecture](docs/coverage.md) loads the same way.
+```html
+<script type="module">
+import { Llama } from 'https://cdn.jsdelivr.net/npm/diskformer@0.5.2/ggml/llama.js';
+const llama = new Llama();
+const name = 'granite-4.0-h-tiny-Q4_K_M.gguf';
+await llama.download({ name, size: 4230976352,
+  url: `https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/resolve/main/${name}` });
+await llama.load({ name, budget: 3e9 });                     // GPU memory in bytes; the rest pages in from OPFS
+const { text } = await llama.chat([{ role: 'user', content: 'Name three primary colours.' }]);
+console.log(text);
+</script>
+```
+`download` takes a progress callback, `chat` an `onPiece` callback for streaming, and `abort()` stops either.
+
 ## Try it: an 80B model (48 GB file) in under 5 GB of GPU memory
 
 https://github.com/user-attachments/assets/4a23b43e-995a-4ba8-b1df-5ffae0222b4e
@@ -37,45 +56,29 @@ once. Measured in Chrome on a MacBook Pro M4 Pro, 24 GB, 2026-10-13 (the recordi
 16 architectures pass both checks (paging exact natively, then Chrome against llama.cpp): Gemma 4, Qwen3, Qwen3-Next, Qwen3.6,
 Qwen3-VL, GLM-4.7-Flash, Nemotron 3, ERNIE 4.5, gpt-oss, North, LFM2/2.5, Mellum2, Ling 2.0/3.0, Granite 4.0 H, OLMoE. [docs/coverage.md](docs/coverage.md).
 
-The hand-written engines, measured on a MacBook M4 Pro with 24 GB, Chrome 154, on 2026-10-05:
-
-| Model | On disk | GPU budget | GPU memory used | Decode | Output |
-|---|---|---|---|---|---|
-| Gemma 4 26B-A4B, QAT Q4_0 | 14.4 GB | 2.5 GB | 2.02 GB | 12.4 tok/s | 9/9 replies identical to llama.cpp |
-| Gemma 4 26B-A4B, QAT Q4_0 | 14.4 GB | 4.3 GB | 3.93 GB | 14.2 tok/s | 9/9 |
-| Gemma 4 26B-A4B, QAT Q4_0 | 14.4 GB | 7 GB | 6.63 GB | 17.9 tok/s | 9/9 |
-| Qwen3.6 35B-A3B, Q8_0 | 36.9 GB | 4 GB | 3.52 GB | 9.2 tok/s | not compared |
-
-"Identical": the reply matches llama.cpp b9830 Metal character for character on the same GGUF (9 conversations, greedy,
-64 tokens; Qwen3.6 does not fit Metal here). Each row is `node examples/run-demo.mjs chat --budget <GB>`, decode the
-median. Only Apple silicon is tested; discrete GPUs and 8–16 GB machines are not.
-
-In your own page (the GGUF downloads into OPFS once; later loads take seconds). The hand-written engines run at
-[/classic/](https://diskformer.naklitechie.com/classic/) and load the same way: `Gemma4MoeSsd.load(null, { gpuBudgetBytes })`.
-```js
-import { Llama } from 'diskformer/llama';                   // llama.cpp path, any MoE GGUF on a gated architecture
-const llama = new Llama();
-await llama.download({ url, name: 'model.gguf', size });    // resumable; a GGUF on disk works with a local URL
-await llama.load({ name: 'model.gguf', budget: 4e9 });      // GPU memory in bytes; the experts page in from OPFS
-const { text } = await llama.chat([{ role: 'user', content: 'Hello' }], { onPiece: show });
-```
-
 ## Why
 
 Your model does not fit on the GPU, or fits only by crowding out everything else. Browser engines load every weight
-at load time, though a mixture-of-experts model uses under a tenth of its weights per token.
-
-diskformer keeps those weights in OPFS, read through sync access handles in workers. Only what each step needs
-moves into a GPU cache. The core is the store and the residency layer: your engine keeps its own kernels and decides
-what a step needs. `engines/` shows two complete engines built on it.
+at load time, though a mixture-of-experts model uses under a tenth of its weights per token. diskformer keeps those
+weights in OPFS, read through sync access handles in workers, and moves only what each step needs into a GPU cache.
 
 ## The engines
 
-`engines/gemma4_moe_ssd.js` runs Gemma 4 26B-A4B. `engines/qwen35_moe_ssd.js` runs Qwen3.6 35B-A3B, and
-`engines/qwen3_moe_ssd.js` runs Qwen3-30B-A3B as their base. They are from-scratch WebGPU engines. The dense layers and
-KV cache live on the GPU. The experts stream from OPFS into a GPU cache, sized by `gpuBudgetBytes`. A budget that
-cannot hold the dense part and two tokens' experts is refused with the numbers. Decoding is greedy, one `generate()`
-at a time per engine. Kernels: gemma4-webgpu (Apache-2.0, `engines/NOTICE`); files kept in step with LocalMind.
+The disk tier also stands alone: your engine keeps its own kernels and decides what a step needs. `engines/` holds three
+from-scratch WebGPU engines on it: Gemma 4 26B-A4B, Qwen3.6 35B-A3B and Qwen3-30B-A3B (at
+[/classic/](https://diskformer.naklitechie.com/classic/); `Gemma4MoeSsd.load(null, { gpuBudgetBytes })`). A budget that
+cannot hold the dense part and two tokens' experts is refused with the numbers. Kernels: gemma4-webgpu (Apache-2.0,
+`engines/NOTICE`). Measured on a MacBook M4 Pro, 24 GB, Chrome 154, 2026-10-05, decode the median of
+`node examples/run-demo.mjs chat --budget <GB>`:
+
+| Model | On disk | GPU memory used | Decode | Output |
+|---|---|---|---|---|
+| Gemma 4 26B-A4B, QAT Q4_0 | 14.4 GB | 2.02 GB (budget 2.5) | 12.4 tok/s | 9/9 replies identical to llama.cpp b9830 Metal |
+| Gemma 4 26B-A4B, QAT Q4_0 | 14.4 GB | 6.63 GB (budget 7) | 17.9 tok/s | 9/9 |
+| Qwen3.6 35B-A3B, Q8_0 | 36.9 GB | 3.52 GB (budget 4) | 9.2 tok/s | not compared (does not fit Metal here) |
+
+"Identical": character for character on the same GGUF, 9 conversations, greedy, 64 tokens. Only Apple silicon is
+tested; discrete GPUs and 8–16 GB machines are not.
 
 ## A table on disk
 
@@ -94,9 +97,7 @@ through a mapped staging ring.
 
 `listStores()` lists the stored models with size and state; a store with zero-byte or short files (a crash) is
 `invalid` and is ingested again, never loaded. `removeStore(key)` removes one and keeps the rest. Errors carry a `code`
-(`LOAD_ABORTED`, `BUDGET_TOO_SMALL`, … in `src/errors.js`), checked with `isDiskformerError(e, code)`.
-
-`measure({ path, recordBytes })` reports sequential MB/s and random record reads; OPFS speed varies by machine.
+(`LOAD_ABORTED`, `BUDGET_TOO_SMALL`, … in `src/errors.js`), checked with `isDiskformerError(e, code)`. `measure({ path, recordBytes })` reports OPFS read speed.
 
 ## Commands
 
@@ -105,9 +106,7 @@ npm test                                     # Node unit tests: record pool, row
 node examples/run-demo.mjs rows              # headless Chrome: a 64 MB table on disk, 2,048 rows on the GPU, every byte checked
 node examples/run-demo.mjs embedding         # lookup → slots → a WGSL gather kernel, every output value checked
 node examples/run-demo.mjs chat --budget 2.5 # downloads Gemma once (14.4 GB) to a fixed profile, replays 9 llama.cpp replies, fails on any difference
-node examples/run-demo.mjs test/browser/ingest-resume.html --models <dir> --file <gguf>   # abort, resume, compare bytes
-node examples/run-demo.mjs test/browser/greedy.html --models <dir> --file <gguf> --ref test/refs/<ref>.json   # token by token vs llama.cpp
-node examples/run-demo.mjs test/browser/states.html --models <dir> --file g4-6l.gguf   # per-layer states; then test/state-check.py
+node examples/run-demo.mjs test/browser/<page>.html --models <dir> --file <gguf>   # ingest resume, token-by-token, per-layer checks
 npm run sync                                 # the files shared with LocalMind match their sources
 ```
 

package/ggml/llama.js

@@ -9,9 +9,18 @@
 //
 // The worker's messages are documented at the top of ggml/worker.mjs; this file only gives them promises.
 
+// The worker script, started from a blob when it lives on another origin (a CDN): browsers refuse a cross-origin
+// worker URL but let a module worker import a cross-origin module that allows CORS.
+function startWorker() {
+  const url = new URL('./worker.mjs', import.meta.url);
+  if (url.origin === location.origin) return new Worker(url, { type: 'module' });
+  const stub = URL.createObjectURL(new Blob([`import ${JSON.stringify(url.href)};`], { type: 'text/javascript' }));
+  return new Worker(stub, { type: 'module' });
+}
+
 export class Llama {
   constructor({ worker } = {}) {
-    this.worker = worker || new Worker(new URL('./worker.mjs', import.meta.url), { type: 'module' });
+    this.worker = worker || startWorker();
     this.busy = Promise.resolve();
     this.onLog = null;  // (line) => {}: llama.cpp's log
   }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "diskformer",
-  "version": "0.5.1",
+  "version": "0.5.2",
   "description": "Run a model larger than your GPU memory in a browser tab: weights stay in OPFS, WebGPU holds only what each step needs. llama.cpp with paged MoE experts (diskformer/llama), a disk tier, and hand-written MoE engines.",
   "type": "module",
   "license": "MIT",