anyformer 0.1.0 → 0.3.2

package/CHANGELOG.md

@@ -1,5 +1,111 @@
 # Changelog
 
+## 0.3.2 — 2026-10-23
+
+### Changed
+- Nothing loads from the LocalMind repo any more. diskformer v0.6.4 (095ab20) holds LocalMind's whole-model
+  engines too (LFM2.5, Gemma 4 E2B/E4B, Ternary Bonsai 2 27B; MIT, confirmed by their author), and the `localmind`
+  back end loads them from diskformer's `engines/`. The id stays `localmind`. `PINS.localmind` and
+  `SOURCES.localmind` are gone.
+- LFM2.5 230M through the `localmind` back end from diskformer's copy: the same reply as from LocalMind's commit
+  203288a (the engine file is byte-identical), on topic.
+
+## 0.3.1 — 2026-10-23
+
+### Added
+- MiniCPM-V 4.7 35B-A3B (openbmb/MiniCPM-V-4.7-35B-A3B-gguf, Q4_0, 19.7 GB) on the `diskformer` back end, through
+  diskformer's `MiniCpmV47Ssd`: text, and images through `ensureVision()` and `{ type: 'image', rgb, width, height }`
+  blocks. Floor 4 GB (Qwen3.6's; the smallest working budget is unmeasured); 14.1 tok/s over short continuations at a
+  4 GiB pool, ~6.1 GB on the GPU (LocalMind's gate). It ranks before `diskformer-llama`, which has no image path.
+  Not yet run through anyformer in a browser; LocalMind's live check is that check.
+
+## 0.3.0 — 2026-10-23
+
+What LocalMind's worker uses from its disk-tier engines, so its models can move onto anyformer with nothing lost.
+
+### Added
+- `generate()` chunks on the `diskformer` back end carry `token`, the engine's token id.
+- `session.think`: `{ openId, closeId, inPrompt, preamble, always }`, where a model's reasoning starts and ends as
+  token ids (Gemma 4's thought channel, Qwen3.6 and Ling's `<think>`); null where the back end has none.
+- `session.prime(system)`, where the engine has it (Qwen3.6 35B): checkpoints a system prompt after load.
+- `session.ensureVision({ onProgress })`, where the engine has it; image blocks `{ type: 'image', rgb, width, height }`
+  pass through to such an engine. No vision model is in `DISKFORMER_MODELS` yet.
+- `load(repo, { poolBytes })`: the expert cache size, passed to the engine; the budget then only admits the model.
+- `load(repo, { localFile })`: a GGUF the user picked, ingested instead of downloaded.
+
+## 0.2.5 — 2026-10-23
+
+### Changed
+- diskformer pinned to v0.6.3 (1d8b418): Ling 3.0 Flash loads a complete store without reading the parts' headers
+  from Hugging Face; the browser llama.cpp build is rebuilt (split GGUFs still refused there). Granite 4.0 H Tiny
+  through `diskformer-llama` on the new build: 9/9, chat path = reference. The hand engines anyformer gated are
+  unchanged (`qwen3_moe_ssd.js` only adds an export).
+
+## 0.2.4 — 2026-10-23
+
+### Changed
+- The `diskformer` back end's `generate()` honours `thinking`, as `diskformer-llama` does since 0.2.1: off by default,
+  `{ thinking: true }` lets Gemma 4, Qwen3.6 or Ling 3.0 Flash reason first (the engines' `enableThinking`).
+
+## 0.2.3 — 2026-10-23
+
+### Added
+- Ling 3.0 Flash (inclusionAI/Ling-3.0-flash-GGUF, two-part Q4_K_M, 77 GB) on the `diskformer` back end, through
+  diskformer's `LingMoeSsd` hand engine: from a 6.8 GB budget, 5.53 tok/s and an 80.7 GB OPFS store (LocalMind's gate).
+  Not yet run through anyformer in a browser: LocalMind's live check through anyformer is that check.
+- `describe()` keeps the Hub's last answer per repo in Cache Storage (`anyformer-describe-v1`) and uses it when the
+  network fails, so a downloaded model loads offline.
+
+### Changed
+- diskformer pinned to v0.6.2 (f5f1177): adds `ling_moe_ssd.js` and two quant formats to `iq_quants.js`, which no
+  engine gated earlier imports; the earlier gates stand.
+
+## 0.2.2 — 2026-10-23
+
+### Added
+- `load(repo, { file })` and `plan(repo, { file })`: load a chosen GGUF from the repo; a back end that cannot load
+  that file refuses with code `model`.
+
+### Changed
+- `diskformer-llama` loads the file diskformer checked when the repo has it (`CHECKED_FILES`, from diskformer's
+  catalog and coverage table), else the preferred quant. GLM-4.7-Flash now defaults to Q3_K_M (14.6 GB, gated 8/9 by
+  diskformer), not Q4_K_M (18.3 GB, not gated). Granite 4.0 H Tiny and Gemma 4 26B keep the files their gates used.
+
+## 0.2.1 — 2026-10-23
+
+### Changed
+- diskformer pinned to v0.6.1 (5bccf32): its llama.cpp build takes a thinking switch. `diskformer-llama`'s
+  `generate()` now turns thinking off by default (`thinking: true` turns it on), so a model that reasons first answers
+  directly, as on the other back ends.
+
+### Measured (Chrome, M4 Pro; correctness only)
+- Granite 4.0 H Tiny via `diskformer-llama`: 9/9; the chat path matches the reference.
+- Gemma 4 26B-A4B via `diskformer-llama`: 8/9 (miss: system-pirate, as diskformer's own run); the chat path matches the
+  reference, with no thought block. The hand engines are unchanged in v0.6.1, so their 9/9 at 0.2.0 stands.
+
+## 0.2.0 — 2026-10-22
+
+### Added
+- `diskformer-llama` back end: mixture-of-experts GGUFs through diskformer's llama.cpp path (`diskformer/llama`,
+  llama.cpp with WebGPU in a worker, experts paged from OPFS), for the 17 architectures diskformer has checked
+  (`LLAMA_ARCHITECTURES`). A diskformer hand-written engine still ranks first where it covers the model.
+- `probe()` reports `jspi` (WebAssembly JSPI), which the llama.cpp build needs.
+- `stored()` and `remove()` cover the llama.cpp path's GGUFs in OPFS (`gguf/`).
+
+### Changed
+- diskformer pinned to v0.6.0 (439ad4e) for both its hand engines and the llama.cpp path.
+
+### Measured (Chrome, M4 Pro)
+- Granite 4.0 H Tiny through `diskformer-llama`: 9/9 replies identical to diskformer's reference
+  (`npm run test:browser -- llama-gate`).
+- Gemma 4 26B-A4B through `diskformer-llama`: 8/9 identical to llama.cpp b9830, the same 8/9 (miss: system-pirate)
+  diskformer records for its own Chrome run. The resolver still sends this repo to the hand engine.
+- Gemma 4 26B-A4B on the hand engine at the new pin: 9/9 identical (`diskformer-chat`).
+
+### Known gaps
+- `diskformer-llama`'s chat path does not turn thinking off: a model that reasons first (Gemma 4, Qwen3.6, gpt-oss)
+  begins its reply with its thought block. Closed in 0.2.1.
+
 ## 0.1.0 — 2026-10-14
 
 The first release with code. `anyformer@0.0.1` reserved the name and held none.

package/README.md

@@ -58,7 +58,8 @@ back end's answer: yes with an estimate (GPU bytes, disk bytes, tok/s when measu
 | `kohra` | kohra, text-diffusion over onnxruntime-web | MDLM and BD3LM repos tagged `text-diffusion` |
 | `localmind` | LocalMind's hand-written WebGPU kernels, the whole model on the GPU | any LFM2 / LFM2.5 GGUF; Gemma 4 E2B / E4B QAT-mobile; Ternary Bonsai 2 |
 | `wllama` | llama.cpp in WebAssembly: WebGPU when the budget holds the model, the CPU otherwise (no WebGPU needed) | any GGUF up to about 3.6 GB that no engine above covers; files over 2 GB are split into OPFS shards once |
-| `diskformer` | diskformer engines: weights in OPFS, only what a step needs on the GPU | Gemma 4 26B-A4B (from 2.5 GB), Qwen3.6 35B-A3B (from 4 GB) |
+| `diskformer` | diskformer engines: weights in OPFS, only what a step needs on the GPU | Gemma 4 26B-A4B (from 2.5 GB), Qwen3.6 35B-A3B (from 4 GB), Ling 3.0 Flash 124B (from 6.8 GB), MiniCPM-V 4.7 with images (from 4 GB) |
+| `diskformer-llama` | diskformer's llama.cpp build with WebGPU, mixture-of-experts layers paged in from OPFS | GGUFs of the 17 MoE architectures diskformer has checked (Granite 4.0 H, gpt-oss, GLM-4.7-Flash, LFM2 MoE…); a hand engine above wins where it covers the model |
 
 Each runtime loads from an exact version or commit in [src/sources.js](src/sources.js); set a back end's `.source`
 (for example `kohraBackend.source`) to load another copy. `BACKENDS` lists the back ends `load()` asks by default: each joins after its browser test passes.
@@ -66,8 +67,7 @@ Each runtime loads from an exact version or commit in [src/sources.js](src/sourc
 ## When no back end can run it
 
 `load()` throws an `AnyformerError` with code `NO_BACKEND` and the plan, naming each back end, its code and its
-reason. Codes are a closed set, each with one
-remedy in `REFUSALS`: `format`, `model`, `task`, `webgpu`, `f16`, `opfs`, `budget` (with `need`, the budget that would pass)
+reason. Codes are a closed set, each with one remedy in `REFUSALS`: `format`, `model`, `task`, `webgpu`, `f16`, `opfs`, `budget` (with `need`, the budget that would pass)
 and `threw`. Every other failure is an `AnyformerError` too, with a code from `ERROR_CODES` (`LOAD_FAILED`,
 `BUDGET_TOO_SMALL`, `ABORTED`, `EXECUTION_FAILED`, `INVALID_ARGUMENT`, `DISPOSED`); check it with
 `isAnyformerError(e, code)`. A session runs one `generate()` at a time, in call order.
@@ -76,7 +76,7 @@ and `threw`. Every other failure is an `AnyformerError` too, with a code from `E
 
 ```js
 plan(repo, { budget, task })      // what would run it, and why the others would not; loads nothing
-load(repo, { budget, task, maxCtx, onProgress, signal })  // → { backend, task, plan, run(input), generate?, dispose() }
+load(repo, { budget, task, file, maxCtx, poolBytes, localFile, onProgress, signal })  // → { backend, task, plan, run(input), generate?, think?, prime?, dispose() }
 pipeline(task, repo, { budget })  // transformers.js's call shape: gen(messages) → [{ generated_text }]
 probe()                           // WebGPU, shader-f16, subgroups, limits, OPFS quota, memory64
 describe(repo)                    // Hub repo → formats, architecture, task, tags, files
@@ -107,6 +107,7 @@ npm run test:browser -- store             # list a download, remove it
 npm run test:browser -- chat --query "repo=LiquidAI/LFM2.5-1.2B-Instruct-GGUF"   # a LocalMind engine
 npm run test:browser -- vision --query "repo=onnx-community/Qwen3.5-4B-ONNX-OPT"   # names the two cats
 npm run test:browser -- diskformer-chat    # Gemma 4 26B-A4B at 2.5 GB: 9/9 replies identical to llama.cpp b9830
+npm run test:browser -- llama-gate --query "repo=ibm-granite/granite-4.0-h-tiny-GGUF&refs=diskformer-granite-refs"   # 9/9 via diskformer-llama
 ```
 
 The browser tests fail a reply that is off topic or repeats itself, and `diskformer-chat` fails any reply that differs

package/index.js

@@ -7,6 +7,7 @@ export { resolve, REFUSALS } from './src/resolve.js';
 export { transformersBackend } from './src/backends/transformers.js';
 export { kohraBackend } from './src/backends/kohra.js';
 export { diskformerBackend, DISKFORMER_MODELS } from './src/backends/diskformer.js';
+export { diskformerLlamaBackend, LLAMA_ARCHITECTURES } from './src/backends/diskformer-llama.js';
 export { TOOLS, call, registerTools, journal, toolOptions } from './src/tools.js';
 export { MODELS } from './src/models.js';
 export { stored, remove } from './src/store.js';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "anyformer",
-  "version": "0.1.0",
+  "version": "0.3.2",
   "description": "One call to run a model in a browser tab: anyformer reads the Hub repo, probes the browser, and picks the back end (transformers.js, LocalMind's WebGPU kernels, diskformer, kohra) that fits the model's format, architecture and your GPU budget.",
   "type": "module",
   "main": "./index.js",

package/src/backends/diskformer-llama.js

@@ -0,0 +1,161 @@
+// diskformer-llama: mixture-of-experts GGUFs through diskformer's llama.cpp path (diskformer/llama: llama.cpp with
+// WebGPU in a worker, the experts paged in from OPFS on demand). One engine for every architecture diskformer has
+// checked in Chrome, so a new MoE model of a covered architecture needs no anyformer code. Where diskformer also has a
+// hand-written engine for the model (the diskformer back end), that one carries a measured speed and ranks first.
+// llama.js loads from the CDN at load() time, at the commit src/sources.js pins; set diskformerLlamaBackend.source to
+// load another copy.
+
+import { AnyformerError } from '../errors.js';
+import { SOURCES } from '../sources.js';
+
+// Architectures that pass diskformer's two checks (docs/coverage.md at the pinned commit): paging is exact natively,
+// and the Chrome replies are compared with llama.cpp. Other models of the same architecture are expected to run.
+// Left out: hy_v3 and afmoe (run, below diskformer's bar), laguna (unknown tokens), dense architectures (no experts
+// to page), and Mixtral-style `llama` MoE (runs, too slow to use).
+export const LLAMA_ARCHITECTURES = [
+  'gemma4', 'qwen3moe', 'qwen3next', 'qwen35moe', 'qwen3vlmoe', 'deepseek2', 'nemotron_h_moe', 'ernie4_5-moe',
+  'gpt-oss', 'cohere2moe', 'lfm2moe', 'mellum', 'bailingmoe2', 'granitehybrid', 'bailingmoe3', 'flint', 'olmoe',
+];
+
+// Quant preference by the token in the file name; the first one the repo has wins. The files diskformer checked are
+// Q4_K_M, Q4_0, Q4_1, MXFP4 and Q3_K_M.
+const QUANTS = ['Q4_K_M', 'Q4_0', 'Q4_1', 'MXFP4', 'Q3_K_M', 'Q4_K_S', 'Q5_K_M', 'Q5_K_S', 'Q6_K', 'Q8_0'];
+
+// The exact files diskformer checked (ggml/chat.html CATALOG and docs/coverage.md at the pinned commit). When a repo
+// holds one, it is the default: the coverage numbers were measured on that file.
+export const CHECKED_FILES = [
+  'gemma-4-26B_q4_0-it.gguf', 'Qwen3-30B-A3B-Instruct-2507-Q3_K_M.gguf', 'Qwen3-Next-80B-A3B-Instruct-Q4_K_M.gguf',
+  'Qwen3.6-35B-A3B-UD-Q3_K_M.gguf', 'Qwen3-VL-30B-A3B-Instruct-Q3_K_M.gguf', 'GLM-4.7-Flash-Q3_K_M.gguf',
+  'NVIDIA-Nemotron-3-Nano-30B-A3B-Q4_K_M.gguf', 'ERNIE-4.5-21B-A3B-Thinking-Q4_K_M.gguf', 'gpt-oss-20b-MXFP4.gguf',
+  'North-Mini-Code-1.0-UD-Q3_K_M.gguf', 'LFM2.5-24B-A2B-QAD-Q4_0.gguf', 'LFM2-8B-A1B-Q4_K_M.gguf',
+  'Mellum2-12B-A2.5B-Instruct-Q4_K_M.gguf', 'Ling-mini-2.0-Q4_K_M.gguf', 'granite-4.0-h-tiny-Q4_K_M.gguf',
+  'Ling-3.0-tiny-Q4_K_M.gguf', 'flint-int4-q4_1.gguf', 'OLMoE-1B-7B-0125-Instruct-Q4_K_M.gguf',
+];
+
+// The GGUF to load: the language model (no mmproj vision tower), one file (no split sets). The caller's file when
+// given; else the file diskformer checked; else the preferred quant.
+export function pickGguf(model, file = null) {
+  const lms = model.files.gguf.filter((f) => !/mmproj/i.test(f.name) && !/-\d{5}-of-\d{5}\.gguf$/.test(f.name));
+  if (file) return lms.find((f) => f.name === file) || null;
+  const checked = lms.find((f) => CHECKED_FILES.includes(f.name.split('/').pop()));
+  if (checked) return checked;
+  for (const q of QUANTS) {
+    const f = lms.find((x) => new RegExp(`(^|[-_.])${q}([-_.]|$)`, 'i').test(x.name.replace(/\.gguf$/i, '')));
+    if (f) return f;
+  }
+  return null;
+}
+
+// OPFS keeps llama.cpp's GGUFs by file name (gguf/<name>), with no repo. anyformer records name → repo here, so
+// stored() can say which repo a file belongs to.
+export const LLAMA_INDEX = 'anyformer-llama.json';
+async function readIndex() {
+  try { return JSON.parse(await (await (await (await navigator.storage.getDirectory()).getFileHandle(LLAMA_INDEX)).getFile()).text()); } catch { return {}; }
+}
+async function writeIndex(index) {
+  const h = await (await navigator.storage.getDirectory()).getFileHandle(LLAMA_INDEX, { create: true });
+  const w = await h.createWritable();
+  await w.write(JSON.stringify(index));
+  await w.close();
+}
+export { readIndex as readLlamaIndex };
+
+// Chat content: a string, or text blocks joined. llama.cpp's chat here takes text only.
+function textOnly(messages) {
+  return messages.map((m) => {
+    if (typeof m.content === 'string') return m;
+    const other = m.content.find((b) => b.type !== 'text');
+    if (other) throw new AnyformerError('INVALID_ARGUMENT', `diskformer-llama runs text only; got a ${other.type} block`);
+    return { ...m, content: m.content.map((b) => b.text).join('\n') };
+  });
+}
+
+export const diskformerLlamaBackend = {
+  id: 'diskformer-llama',
+  tasks: ['text-generation', 'image-text-to-text'],
+  source: SOURCES.diskformerLlama,
+  // Where a GGUF downloads from; a test points it at a local server.
+  url: (model, file) => `${model.endpoint || 'https://huggingface.co'}/${model.repo}/resolve/${model.revision || 'main'}/${file.name.split('/').map(encodeURIComponent).join('/')}`,
+
+  canRun(model, caps, { budget, task, file = null }) {
+    if (!model.formats.includes('gguf')) return { ok: false, code: 'format', reason: 'no GGUF in the repo' };
+    if (task && !diskformerLlamaBackend.tasks.includes(task)) return { ok: false, code: 'task', reason: `task ${task}; diskformer's llama.cpp path runs text generation only` };
+    if (!LLAMA_ARCHITECTURES.includes(model.architecture)) return { ok: false, code: 'model', reason: `architecture ${model.architecture || 'unknown'} is not one diskformer's llama.cpp path has checked (${LLAMA_ARCHITECTURES.join(', ')})` };
+    const f = pickGguf(model, file);
+    if (!f) return { ok: false, code: 'model', reason: file ? `${file} is not a single-file language-model GGUF in the repo` : `no single-file GGUF at ${QUANTS.join(', ')} in the repo` };
+    if (!caps.webgpu) return { ok: false, code: 'webgpu', reason: 'no WebGPU adapter' };
+    if (!caps.memory64 || !caps.jspi) return { ok: false, code: 'webgpu', reason: 'the llama.cpp build needs wasm memory64 and JSPI (Chrome or Edge 137+)' };
+    if (!caps.opfs) return { ok: false, code: 'opfs', reason: 'no OPFS (navigator.storage.getDirectory)' };
+    // The budget is not checked here: llama.cpp plans its expert slots from it at load time, and refuses there.
+    return { ok: true, estimate: { gpuBytes: budget, diskBytes: f.size, tokPerSec: null } };
+  },
+
+  // maxCtx sizes the KV cache; 1024 is the context diskformer's gates run at.
+  async load(model, { budget, file = null, maxCtx = 1024, onProgress, signal }) {
+    const f = pickGguf(model, file);
+    const { Llama } = await import(diskformerLlamaBackend.source);
+    const llama = new Llama();
+    const say = (e) => onProgress?.({ backend: 'diskformer-llama', file: f.name, ...e });
+    const stop = () => llama.abort();
+    signal?.addEventListener('abort', stop, { once: true });
+    let info;
+    try {
+      const have = (await llama.list()).find((m) => m.name === f.name && !m.partial);
+      if (!have || have.size !== f.size) {
+        say({ status: 'download', loaded: 0, total: f.size });
+        await llama.download({ url: diskformerLlamaBackend.url(model, f), name: f.name, size: f.size },
+          (p) => say({ status: 'download', loaded: p.copied, total: p.total }));
+        signal?.throwIfAborted();
+      }
+      await writeIndex({ ...(await readIndex()), [f.name]: model.repo });
+      say({ status: 'load' });
+      info = await llama.load({ name: f.name, budget, ctx: maxCtx }, (p) => say({ status: 'plan', plan: p }));
+      signal?.throwIfAborted();
+    } catch (e) {
+      llama.terminate();
+      throw e;
+    } finally {
+      signal?.removeEventListener('abort', stop);
+    }
+    say({ status: 'ready', slots: info.slots });
+    return {
+      task: 'text-generation',
+      llama,
+      info,
+      // llama.cpp hands back text pieces, not token ids, so there are no think ids to report.
+      think: null,
+      // thinking: false (the default, as the other engines here) renders the template with thinking off, so a model that
+      // reasons first answers directly; true lets it think.
+      async *generate(messages, { maxNewTokens = 512, signal, thinking = false } = {}) {
+        const chunks = [];
+        let wake = null, finished = false, failed = null, result = null, text = '', tokens = 0;
+        const nudge = () => { wake?.(); wake = null; };
+        const abort = () => llama.abort();
+        signal?.addEventListener('abort', abort, { once: true });
+        const job = llama.chat(textOnly(messages), {
+          n: maxNewTokens, thinking,
+          onPiece: (s) => { text += s; tokens++; chunks.push({ text, tokens }); nudge(); },
+        }).then((r) => { result = r; }, (e) => { failed = e; }).finally(() => { finished = true; nudge(); });
+        try {
+          for (;;) {
+            while (chunks.length) yield chunks.shift();
+            if (finished) break;
+            await new Promise((r) => { wake = r; });
+          }
+          signal?.throwIfAborted();
+          if (failed) throw failed;
+          const s = result.stats || {};
+          const decode = s.decode_tokens && s.decode_ms ? s.decode_tokens / (s.decode_ms / 1000) : null;
+          yield { text: result.text, tokens: s.decode_tokens ?? tokens, ...(decode && { tokPerSec: decode }) };
+        } finally {
+          signal?.removeEventListener('abort', abort);
+          // The caller stopped reading early: stop the reply so the next call does not queue behind it.
+          if (!finished) { llama.abort(); await job; }
+        }
+      },
+      // A continuation of an already formatted prompt, greedy: what diskformer's gate replays.
+      complete: (prompt, { maxNewTokens = 64 } = {}) => llama.complete(prompt, { n: maxNewTokens }),
+      dispose: () => llama.terminate(),
+    };
+  },
+};

package/src/backends/diskformer.js

@@ -18,6 +18,28 @@ export const DISKFORMER_MODELS = [
     engine: 'qwen35_moe_ssd.js', Engine: 'Qwen35MoeSsd', minBudget: 4e9,
     speeds: [[4e9, 9.2]],
   },
+  // Qwen3.6's language model with a vision encoder (mmproj, ~1.1 GB, loaded on the first image by ensureVision).
+  // From LocalMind's gate (plan/2026-10-14-minicpm-harness, 4 GiB pool, context 512): ~6.1 GB on the GPU (dense 1.59,
+  // pool 4.29), 14.1 tok/s over short greedy continuations. The smallest working budget is unmeasured; the floor is
+  // Qwen3.6's, the same language-model shape. Only this engine reads its images, and it ranks before diskformer-llama.
+  {
+    repo: 'openbmb/MiniCPM-V-4.7-35B-A3B-gguf', file: 'MiniCPM-V-4_7-35B-A3B-Q4_0.gguf', size: 19714550688,
+    engine: 'qwen35_moe_ssd.js', Engine: 'MiniCpmV47Ssd', minBudget: 4e9,
+    speeds: [[6.1e9, 14.1]],
+  },
+  // Two parts, ingested as one file by the engine's splitFetch (engines/ling_moe_ssd.js LING_FLASH, rev 29edce5).
+  // Budget and speed from LocalMind's gate (plan/2026-10-22-ling-harness/results.md): 2.51 GB of dense weights on
+  // the GPU plus a 4.29 GB expert pool, 5.53 tok/s decode; the OPFS store is 80.7 GB.
+  {
+    repo: 'inclusionAI/Ling-3.0-flash-GGUF',
+    parts: [
+      { file: 'Q4_K_M/Ling-3.0-flash-Q4_K_M-00001-of-00002.gguf', size: 49995534208 },
+      { file: 'Q4_K_M/Ling-3.0-flash-Q4_K_M-00002-of-00002.gguf', size: 27014610464 },
+    ],
+    size: 49995534208 + 27014610464, storeBytes: 80.7e9,
+    engine: 'ling_moe_ssd.js', Engine: 'LingMoeSsd', minBudget: 6.8e9,
+    speeds: [[6.8e9, 5.53]],
+  },
 ];
 
 // diskformer's engines refuse a budget with a plain Error ("GPU budget 2.50 GB is too small: … need 3.04 GB, and the
@@ -27,8 +49,9 @@ function budgetError(e) {
   return m && new AnyformerError('BUDGET_TOO_SMALL', `diskformer: ${e.message}`, { cause: e, need: Math.ceil((+m[1] + +m[2]) * 1e9) });
 }
 
+const has = (model, file, size) => model.files.gguf.some((f) => f.name === file && f.size === size);
 const match = (model) => DISKFORMER_MODELS.find((m) => m.repo === model.repo &&
-  model.files.gguf.some((f) => f.name === m.file && f.size === m.size));
+  (m.parts ? m.parts.every((p) => has(model, p.file, p.size)) : has(model, m.file, m.size)));
 
 // The speed measured at the largest budget not above this one.
 const speedAt = (m, budget) => m.speeds.filter(([b]) => b <= budget).at(-1)?.[1] ?? null;
@@ -40,40 +63,61 @@ export const diskformerBackend = {
   // Gemma 4's repo is tagged image-text-to-text; the engines run its text turns.
   tasks: ['text-generation', 'image-text-to-text'],
   source: SOURCES.diskformer,
+  // Where the engine downloads its GGUF from; null is Hugging Face. A test points it at a local server.
+  url: null,
 
   canRun(model, caps, { budget, task }) {
     if (!model.formats.includes('gguf')) return { ok: false, code: 'format', reason: 'no GGUF in the repo' };
     if (task && !diskformerBackend.tasks.includes(task)) return { ok: false, code: 'task', reason: `task ${task}; diskformer engines run text generation only` };
     const m = match(model);
-    if (!m) return { ok: false, code: 'model', reason: `no diskformer engine for this GGUF; engines cover ${DISKFORMER_MODELS.map((x) => x.file).join(', ')}` };
+    if (!m) return { ok: false, code: 'model', reason: `no diskformer engine for this GGUF; engines cover ${DISKFORMER_MODELS.map((x) => x.repo).join(', ')}` };
     if (!caps.webgpu) return { ok: false, code: 'webgpu', reason: 'no WebGPU adapter' };
     if (!caps.f16) return { ok: false, code: 'f16', reason: 'the GPU lacks shader-f16' };
     if (!caps.opfs) return { ok: false, code: 'opfs', reason: 'no OPFS (navigator.storage.getDirectory)' };
     if (budget < m.minBudget) return { ok: false, code: 'budget', need: m.minBudget, reason: `a ${(budget / 1e9).toFixed(1)} GB budget is under the ${m.minBudget / 1e9} GB this model has passed its gate at` };
     // Storage quota is not checked: Chrome raises it as the download writes, while the disk has room.
-    return { ok: true, estimate: { gpuBytes: budget, diskBytes: m.size, tokPerSec: speedAt(m, budget) } };
+    return { ok: true, estimate: { gpuBytes: budget, diskBytes: m.storeBytes ?? m.size, tokPerSec: speedAt(m, budget) } };
   },
 
   // maxCtx sizes the KV cache, which stays on the GPU: the 2.5 GB minimum and the speeds above hold at 1024 tokens,
   // the context diskformer's chat gate runs at. A longer context needs a larger budget.
-  async load(model, { budget, maxCtx = 1024, onProgress, signal }) {
+  // poolBytes: the expert cache size, passed to the engine; the budget then only admits the model (the engine sizes
+  //   its pool from gpuBudgetBytes when given one, so it gets one or the other).
+  // localFile: the GGUF as a File the user picked; the engine ingests it instead of downloading, and refuses a file
+  //   whose size differs from the one it is pinned to.
+  async load(model, { budget, maxCtx = 1024, poolBytes, localFile, onProgress, signal }) {
     const m = match(model);
     const mod = await import(`${diskformerBackend.source}/${m.engine}`);
     let engine;
     try {
       engine = await mod[m.Engine].load(null, {
         // The engines' own default root, named so the store module (src/store.js) lists the same folder.
-        gpuBudgetBytes: budget, maxCtx, signal, root: 'localmind-ssd',
+        ...(poolBytes ? { poolBytes } : { gpuBudgetBytes: budget }), maxCtx, signal, root: 'localmind-ssd',
+        ...(localFile && { localFile }),
+        ...(diskformerBackend.url && (m.parts ? { urls: m.parts.map((p) => diskformerBackend.url(model, { ...m, file: p.file })) } : { url: diskformerBackend.url(model, m) })),
         onProgress: onProgress && ((e) => onProgress({ backend: 'diskformer', ...e })),
       });
     } catch (e) { throw budgetError(e) || e; }
+    // Where the model's reasoning starts and ends, as token ids, so a caller can split it out without parsing text.
+    const open = engine.thinkOpenTokenId ?? null;
     return {
       task: 'text-generation',
       engine,
-      async *generate(messages, { maxNewTokens = 512, signal } = {}) {
-        for await (const r of engine.generate(messages, { maxNewTokens, signal, enableThinking: false }))
-          yield { text: r.text, tokens: r.tokens };
+      think: open === null ? null : {
+        openId: open, closeId: engine.thinkCloseTokenId ?? null, inPrompt: engine.thinkInPrompt ?? null,
+        preamble: engine.thinkPreamble ?? null, always: engine.alwaysThinks ?? null,
+      },
+      // thinking: false (the default, as on every back end here) answers directly; true lets a model that reasons
+      // (Gemma 4, Qwen3.6, Ling) think first. Image blocks ({ type: 'image', rgb, width, height }) pass through to
+      // an engine that reads them.
+      async *generate(messages, { maxNewTokens = 512, signal, thinking = false } = {}) {
+        for await (const r of engine.generate(messages, { maxNewTokens, signal, enableThinking: thinking === true }))
+          yield { text: r.text, token: r.token, tokens: r.tokens };
       },
+      // Checkpoints a system prompt after load, so the first question does not wait for it (Qwen3.6, MiniCPM-V).
+      ...(typeof engine.prime === 'function' && { prime: (system) => engine.prime(system) }),
+      // Loads the vision tower, for an engine that reads images (MiniCPM-V).
+      ...(typeof engine.ensureVision === 'function' && { ensureVision: (o) => engine.ensureVision(o) }),
       dispose: () => engine.dispose(),
     };
   },

package/src/backends/localmind.js

@@ -1,7 +1,8 @@
 // LocalMind's hand-written WebGPU engines: whole model resident on the GPU, kernels written per architecture. Each
 // engine reads its config from the model file, so it runs any checkpoint of its architecture in a format it reads,
-// not only the one LocalMind ships. They load from the CDN at load() time, at the commit src/sources.js pins; set
-// localmindBackend.source to load another copy.
+// not only the one LocalMind ships. They live in diskformer (engines/, since v0.6.4) and load from the CDN at load()
+// time, at the commit src/sources.js pins; set localmindBackend.source to load another copy. The id stays 'localmind',
+// the name callers and stored() already use, though no code loads from LocalMind's repo any more.
 //
 //   lfm2     src/lfm2_5.js           GGUF, general.architecture 'lfm2' (dense LFM2 / LFM2.5), Q4_0 Q8_0 Q4_K Q5_K Q6_K
 //   gemma4   src/gemma-4-e2b.js      Google's QAT-mobile safetensors export of Gemma 4 (E2B, E4B): config.json-driven
@@ -58,7 +59,7 @@ function match(model) {
 export const localmindBackend = {
   id: 'localmind',
   tasks: ['text-generation'],
-  source: SOURCES.localmind,
+  source: SOURCES.diskformer,
 
   canRun(model, caps, { budget, task }) {
     const m = match(model);

package/src/describe.js

@@ -3,11 +3,24 @@
 
 const HF = 'https://huggingface.co';
 
-export async function describe(repo, { fetch = globalThis.fetch, revision = 'main', endpoint = HF } = {}) {
+// The last answer the Hub gave for each URL, kept in Cache Storage so a model already downloaded still loads offline.
+export const DESCRIBE_CACHE = 'anyformer-describe-v1';
+
+export async function describe(repo, { fetch = globalThis.fetch, revision = 'main', endpoint = HF, caches = globalThis.caches } = {}) {
   const url = `${endpoint}/api/models/${repo}${revision === 'main' ? '' : `/revision/${revision}`}?blobs=true`;
-  const res = await fetch(url);
-  if (!res.ok) throw new Error(`describe ${repo}: ${url} returned ${res.status}`);
-  return fromHubInfo(await res.json(), { endpoint });
+  const store = caches ? await caches.open(DESCRIBE_CACHE).catch(() => null) : null;
+  let res, failed;
+  try { res = await fetch(url); } catch (e) { failed = e; }
+  if (res?.ok) {
+    const info = await res.json();
+    await store?.put(url, new Response(JSON.stringify(info), { headers: { 'Content-Type': 'application/json' } })).catch(() => {});
+    return fromHubInfo(info, { endpoint });
+  }
+  // No network, or the Hub failed: the cached answer, when this browser has one.
+  const hit = await store?.match(url).catch(() => null);
+  if (hit) return fromHubInfo(await hit.json(), { endpoint });
+  throw failed ? new Error(`describe ${repo}: ${url} failed: ${failed.message}`, { cause: failed })
+    : new Error(`describe ${repo}: ${url} returned ${res.status}`);
 }
 
 // Pure: Hub API JSON → descriptor. Exported for tests and for callers that already hold the JSON.

package/src/load.js

@@ -4,12 +4,15 @@ import { resolve } from './resolve.js';
 import { transformersBackend } from './backends/transformers.js';
 import { kohraBackend } from './backends/kohra.js';
 import { diskformerBackend } from './backends/diskformer.js';
+import { diskformerLlamaBackend } from './backends/diskformer-llama.js';
 import { localmindBackend } from './backends/localmind.js';
 import { wllamaBackend } from './backends/wllama.js';
 import { AnyformerError, wrap } from './errors.js';
 
-// Back ends land here as each one passes its browser test (test/browser/). The list order breaks ties.
-export const BACKENDS = [transformersBackend, kohraBackend, diskformerBackend, localmindBackend, wllamaBackend];
+// Back ends land here as each one passes its browser test (test/browser/). The list order breaks ties: a
+// diskformer hand engine carries a measured speed and outranks diskformer-llama for the same model; diskformer-llama
+// comes before wllama because it pages experts instead of holding the whole model on the GPU.
+export const BACKENDS = [transformersBackend, kohraBackend, diskformerBackend, localmindBackend, diskformerLlamaBackend, wllamaBackend];
 
 // 2.5 GB: the smallest budget diskformer's chat gate passes at. Browsers do not report GPU memory, so a caller that
 // knows more passes { budget }.
@@ -23,28 +26,29 @@ function noBackend(plan) {
 
 // What load() would do, without loading: the descriptor, the caps, and every back end's answer.
 // task: a Hugging Face pipeline task ('text-generation', 'feature-extraction', …); null runs the repo's own.
+// file: a weights file in the repo to load (a GGUF name); null lets each back end pick.
 // repo: a Hub id, or an entry from MODELS (its task and budget apply unless opts set them).
 const entry = (repo, opts) => (typeof repo === 'object' && repo?.repo
   ? [repo.repo, { ...(repo.task && { task: repo.task }), ...(repo.budget && { budget: repo.budget }), ...opts }]
   : [repo, opts]);
 
 export async function plan(repoOrEntry, opts = {}) {
-  let [repo, { budget = DEFAULT_BUDGET, task = null, backends = BACKENDS, caps, model, fetch }] = entry(repoOrEntry, opts);
+  let [repo, { budget = DEFAULT_BUDGET, task = null, file = null, backends = BACKENDS, caps, model, fetch }] = entry(repoOrEntry, opts);
   caps ??= await probe();
   model ??= await describe(repo, fetch ? { fetch } : {});
-  return { model, caps, budget, task, ...(await resolve(model, caps, backends, { budget, task })) };
+  return { model, caps, budget, task, file, ...(await resolve(model, caps, backends, { budget, task, file })) };
 }
 
 export async function load(repoOrEntry, options = {}) {
   const [repo, opts] = entry(repoOrEntry, options);
-  const { backends = BACKENDS, maxCtx, dtype, onProgress, signal } = opts;
+  const { backends = BACKENDS, maxCtx, dtype, poolBytes, localFile, onProgress, signal } = opts;
   let p;
   try { p = await plan(repo, opts); } catch (e) { throw wrap(e, 'LOAD_FAILED', signal, `${repo}: `); }
   if (!p.chosen) throw noBackend(p);
   const backend = backends.find((b) => b.id === p.chosen);
   let session;
   try {
-    session = await backend.load(p.model, { budget: p.budget, task: p.task, caps: p.caps, maxCtx, dtype, onProgress, signal });
+    session = await backend.load(p.model, { budget: p.budget, task: p.task, file: p.file, caps: p.caps, maxCtx, dtype, poolBytes, localFile, onProgress, signal });
   } catch (e) { throw wrap(e, 'LOAD_FAILED', signal, `${backend.id}: `); }
   return { backend: backend.id, task: session.task || p.task || p.model.task || null, ...(session.device && { device: session.device }), plan: p, ...serialize(session) };
 }
@@ -80,8 +84,25 @@ function serialize(session) {
       release();
     }
   };
+  // A call that waits its turn like generate(): prime and ensureVision share the engine with it.
+  const queued = (fn) => async (...args) => {
+    if (disposed) throw gone();
+    const release = await turn();
+    try {
+      if (disposed) throw gone();
+      return await fn(...args);
+    } catch (e) {
+      throw wrap(e, 'EXECUTION_FAILED');
+    } finally {
+      release();
+    }
+  };
   return {
     ...(generate && { generate }),
+    // Where reasoning starts and ends as token ids ({ openId, closeId, inPrompt, preamble, always }), or null.
+    ...('think' in session && { think: session.think }),
+    ...(session.prime && { prime: queued(session.prime) }),
+    ...(session.ensureVision && { ensureVision: queued(session.ensureVision) }),
     // The one call every task has. For text generation it is the last chunk of generate().
     async run(input, o = {}) {
       if (generate) { let last; for await (const c of generate(input, o)) last = c; return last; }

package/src/probe.js

@@ -14,6 +14,8 @@ export async function probe({ gpu = globalThis.navigator?.gpu, storage = globalT
     maxBufferSize: 0, maxStorageBufferBindingSize: 0, adapter: null,
     opfs: false, quotaBytes: 0, usageBytes: 0,
     memory64: memory64(),
+    // JSPI: wasm that suspends on a promise (diskformer's llama.cpp build reads experts from OPFS this way).
+    jspi: typeof globalThis.WebAssembly?.Suspending === 'function',
   };
   const adapter = gpu ? await gpu.requestAdapter({ powerPreference: 'high-performance' }).catch(() => null) : null;
   if (adapter) {

package/src/resolve.js

@@ -3,7 +3,8 @@
 // Back-end contract (each back end is a plain object):
 //   id        string, stable, e.g. 'diskformer'
 //   tasks     the Hugging Face pipeline tasks it runs, e.g. ['text-generation']
-//   canRun(model, caps, { budget, task }) → { ok: true, estimate } | { ok: false, code, reason, need? }
+//   canRun(model, caps, { budget, task, file }) → { ok: true, estimate } | { ok: false, code, reason, need? }
+//             file: a weights file the caller chose (a GGUF name); a back end that cannot load that file refuses.
 //             estimate = { gpuBytes, diskBytes, tokPerSec | null }. No network, no GPU allocation: it reads the
 //             descriptor and caps only. code is one of REFUSALS below; need is the budget that would pass, with
 //             code 'budget'. A throw counts as code 'threw' with the message as the reason.
@@ -26,11 +27,11 @@ export const REFUSALS = {
   threw: 'the back end crashed deciding; a bug in that back end, report it with the reason',
 };
 
-export async function resolve(model, caps, backends, { budget, task = null } = {}) {
+export async function resolve(model, caps, backends, { budget, task = null, file = null } = {}) {
   const candidates = [];
   for (const [order, b] of backends.entries()) {
     let r;
-    try { r = await b.canRun(model, caps, { budget, task }); } catch (e) { r = { ok: false, code: 'threw', reason: `canRun threw: ${e.message}` }; }
+    try { r = await b.canRun(model, caps, { budget, task, file }); } catch (e) { r = { ok: false, code: 'threw', reason: `canRun threw: ${e.message}` }; }
     candidates.push(r.ok ? { backend: b.id, order, ok: true, estimate: r.estimate } : { backend: b.id, order, ok: false, code: r.code || 'model', reason: r.reason, ...(r.need && { need: r.need }) });
   }
   const ranked = candidates.filter((c) => c.ok).sort(rank);

package/src/sources.js

@@ -1,5 +1,5 @@
 // Where each back end's runtime loads from: exact versions and commits, so a given anyformer behaves the same tomorrow.
-// kohra, diskformer and LocalMind move fast on their main branches; `node scripts/pin-sources.mjs` moves these pins
+// kohra and diskformer move fast on their main branches; `node scripts/pin-sources.mjs` moves these pins
 // to each repo's current main, and the browser tests then prove the new pins (test/browser/README.md).
 // transformers.js and wllama move only by hand, in step with LocalMind's pins (plan/history.md, 2026-10-08).
 
@@ -7,8 +7,9 @@ export const PINS = {
   transformers: '4.3.1',
   wllama: '3.4.1',      // LocalMind's pin; its stream-drain workaround (src/backends/wllama.js) is for this version
   kohra: 'f678fac7342c79b58b640540b8b3a7d078b7ab1a',
-  diskformer: '524bc50e2edd186403403090dd83d13f66a8f73f',
-  localmind: '203288a0f0e23919b43c76bff8dc5d49f48ed300',
+  // v0.6.4: every engine (the disk-tier MoE engines, Ling 3.0 Flash, and LocalMind's whole-model engines LFM2.5,
+  // Gemma 4 E2B/E4B and Ternary Bonsai 2) under engines/, and the llama.cpp path under ggml/.
+  diskformer: '095ab2058be9de56841c51b9369ba556d625392b',
 };
 
 const gh = (repo, sha, path) => `https://cdn.jsdelivr.net/gh/NakliTechie/${repo}@${sha}/${path}`;
@@ -18,5 +19,5 @@ export const SOURCES = {
   wllama: `https://cdn.jsdelivr.net/npm/@wllama/wllama@${PINS.wllama}/esm`,
   kohra: gh('kohra', PINS.kohra, 'kohra.js'),
   diskformer: gh('diskformer.js', PINS.diskformer, 'engines'),
-  localmind: gh('LocalMind', PINS.localmind, 'src'),
+  diskformerLlama: gh('diskformer.js', PINS.diskformer, 'ggml/llama.js'),
 };

package/src/store.js

@@ -10,10 +10,13 @@
 //                   system-prompt snapshot).
 //   OPFS            'localmind-ssd/<key>/' (diskformer's MoE stores, Gemma 4's PLE table) with manifest.json naming the
 //                   source; 'cache/' (wllama: '__metadata__<name>' names each file's URL); 'anyformer-gguf/<key>/'
-//                   (GGUFs over 2 GB as shards, meta.json or done.json naming the repo; src/gguf-split.js).
+//                   (GGUFs over 2 GB as shards, meta.json or done.json naming the repo; src/gguf-split.js);
+//                   'gguf/<name>' (diskformer-llama: llama.cpp's GGUFs, with <name>.part and .part.json while downloading;
+//                   anyformer-llama.json maps each name to its repo).
 
 import { DISKFORMER_MODELS } from './backends/diskformer.js';
 import { SPLIT_ROOT } from './gguf-split.js';
+import { readLlamaIndex, LLAMA_INDEX } from './backends/diskformer-llama.js';
 
 const CACHE_BACKEND = { 'transformers-cache': 'transformers', 'gguf-v1': 'localmind', 'gguf-v1-headers': 'localmind', 'bonsai-pipeline-v1': 'localmind' };
 const IDB_STORES = ['gguf-cache-v1', 'safetensors-cache-v1'];
@@ -86,7 +89,7 @@ async function inventory() {
   if (ssd) for await (const [key, h] of ssd.entries()) {
     if (h.kind !== 'directory') continue;
     const src = (await readJson(h, 'manifest.json'))?.source;
-    const repo = src?.repo || repoOf(src?.url) || DISKFORMER_MODELS.find((m) => m.file.toLowerCase().startsWith(key))?.repo;
+    const repo = src?.repo || repoOf(src?.url) || DISKFORMER_MODELS.find((m) => m.file?.toLowerCase().startsWith(key))?.repo;
     add(DISKFORMER_MODELS.some((m) => m.repo === repo) ? 'diskformer' : 'localmind', repo, await dirBytes(h), () => ssd.removeEntry(key, { recursive: true }));
   }
   const wcache = await opfsDir(['cache']);
@@ -103,6 +106,22 @@ async function inventory() {
     const repo = (await readJson(h, 'meta.json'))?.repo || (await readJson(h, 'done.json'))?.repo;
     add('wllama', repo, await dirBytes(h), () => split.removeEntry(key, { recursive: true }));
   }
+  const llama = await opfsDir(['gguf']);
+  if (llama) {
+    const index = await readLlamaIndex();
+    for await (const [name, h] of llama.entries()) {
+      if (h.kind !== 'file' || name.endsWith('.part.json') || name.endsWith('.crswap')) continue;
+      const base = name.replace(/\.part$/, '');
+      add('diskformer-llama', index[base], (await h.getFile()).size, async () => {
+        for (const n of [base, `${base}.part`, `${base}.part.json`]) await llama.removeEntry(n).catch(() => {});
+        const rest = await readLlamaIndex();
+        delete rest[base];
+        const w = await (await (await navigator.storage.getDirectory()).getFileHandle(LLAMA_INDEX, { create: true })).createWritable();
+        await w.write(JSON.stringify(rest));
+        await w.close();
+      });
+    }
+  }
   return out;
 }