anyformer 0.4.4 → 0.4.8

package/CHANGELOG.md

@@ -1,5 +1,63 @@
 # Changelog
 
+## 0.4.8 — 2026-10-24
+
+Skips 0.4.7 on npm: this release carries its Nemotron-Labs-Diffusion-3B entry too.
+
+### Changed
+- kohra pinned to v0.3.0 (85e01f0); kohra.js is identical to 0.4.7's pin, 14e430c.
+- Nemotron-Labs-Diffusion-3B decodes with threshold 0.9 by default (Fast-dLLM: unmask every position at that
+  confidence per step); `generate(messages, { threshold: null })` runs fixed steps. kohra measured 34 forward passes
+  against 64 with the same answer.
+
+### Verified
+- Browser (Chrome, M4 Pro, GPU shared with another session): one page, the CPU prompt, 32 tokens, threshold
+  null / 0.9 / null / 0.9: identical text in every run; 7.1 s and 5.8 s at fixed steps, 4.9 s at 0.9. npm test 89/89.
+
+## 0.4.7 — 2026-10-24
+
+### Added
+- Nemotron-Labs-Diffusion-3B (`MODELS.chat.NEMOTRON_DIFFUSION_3B`, naklitechie/Nemotron-Labs-Diffusion-3B-ONNX) on
+  `kohra`: the q4f16 seeded block-KV graph (onnx/model_kv_q4f16.onnx, 2.73 GB with its two data files), mask id 100,
+  EOS 11, and Nemotron's chat template over the whole conversation (a system turn, an empty think block before each
+  assistant reply). Content blocks are joined as text.
+
+### Changed
+- kohra pinned to main 14e430c (after v0.2.1; kohra.js changed in afee5f4): seeded KV graphs, `chatFormat`, graphs
+  whose data is split into `_data`, `_data_1`.
+
+### Verified
+- Browser (Chrome, M4 Pro): `npm run test:browser -- kohra --query "repo=naklitechie/Nemotron-Labs-Diffusion-3B-ONNX"`
+  answered on topic (32 tokens, 5.0 tok/s, load 156 s with the download); `npm run test:browser -- kohra` (Qwen3-0.6B
+  MDLM) on the new pin answered on topic. npm test 89/89.
+
+## 0.4.6 — 2026-10-24
+
+Skips 0.4.5 on npm: this release carries its diskformer v0.6.8 pin too.
+
+### Added
+- Nemotron 3.5 Bolt E4B (`MODELS.chat.NEMOTRON_3_5_BOLT_E4B`) on `transformers` at a 5 GB budget:
+  danyalmalik/NVIDIA-Nemotron-3.5-Bolt-E4B-ONNX-WebGPU, a community q4f16 export (Bolt's weights in onnx-community's
+  Gemma 4 E4B graph), text only, 4.91 GB.
+- `transformers`: a Gemma 4 repo in the multimodal layout with no vision or audio encoder loads through
+  `Gemma4ForCausalLM` and the tokenizer. Its `use_external_data_format` comes from the repo's `_data` files, because
+  that repo's config.json has no transformers.js_config (without it: "Module.MountedFiles is not available"). Image
+  and audio blocks are refused with `INVALID_ARGUMENT`.
+
+### Verified
+- Browser (Chrome, M4 Pro): `npm run test:browser -- transformers --query "repo=danyalmalik/NVIDIA-Nemotron-3.5-Bolt-E4B-ONNX-WebGPU&budget=5.3"`
+  answered on topic, 34.9 tok/s, load 231 s with the download. npm test 88/88.
+
+## 0.4.5 — 2026-10-24
+
+### Changed
+- diskformer pinned to v0.6.8 (3aad103). Its llama.cpp path (ggml/) reads past 2 GiB in a browser, adds head-512
+  attention and loads drafter GGUFs (`draft`). The hand engines (engines/, src/) are unchanged since v0.6.7.
+
+### Verified
+- `llama-gate` on Granite 4.0 H Tiny (Chrome, M4 Pro, v0.6.8 pin): 9/9 identical to diskformer's refs, chat path
+  equal, 19.1 tok/s median. npm test 87/87. Not rerun: the hand-engine gates, because engines/ did not change.
+
 ## 0.4.4 — 2026-10-24
 
 ### Changed

package/README.md

@@ -14,7 +14,7 @@ browser storage; no account, no server, no telemetry.
 | How | Command |
 |---|---|
 | npm | `npm install anyformer` |
-| CDN | `import { load } from 'https://cdn.jsdelivr.net/npm/anyformer@0.4.4/+esm';` |
+| CDN | `import { load } from 'https://cdn.jsdelivr.net/npm/anyformer@0.4.8/+esm';` |
 | Copy | `index.js` and `src/`; nothing else is needed |
 
 ```js
@@ -54,8 +54,8 @@ back end's answer: yes with an estimate (GPU bytes, disk bytes, tok/s when measu
 
 | Back end | Runs | Models |
 |---|---|---|
-| `transformers` | transformers.js 4.3.1: text generation, image and audio chat (Gemma 4, Qwen3.5), embeddings, speech to text, detection, classification, depth, segmentation | ONNX repos in the onnx-community / Xenova layout |
-| `kohra` | kohra, text-diffusion over onnxruntime-web | MDLM and BD3LM repos tagged `text-diffusion` |
+| `transformers` | transformers.js 4.3.1: text generation (Nemotron 3.5 Bolt E4B among them), image and audio chat (Gemma 4, Qwen3.5), embeddings, speech to text, detection, classification, depth, segmentation | ONNX repos in the onnx-community / Xenova layout |
+| `kohra` | kohra, text-diffusion over onnxruntime-web | MDLM and BD3LM repos tagged `text-diffusion`; Nemotron-Labs-Diffusion-3B |
 | `localmind` | LocalMind's hand-written WebGPU kernels (shipped in diskformer), the whole model on the GPU | any LFM2 / LFM2.5 GGUF; Gemma 4 E2B / E4B QAT-mobile; Ternary Bonsai 2 |
 | `wllama` | llama.cpp in WebAssembly: WebGPU when the budget holds the model, the CPU otherwise (no WebGPU needed) | any GGUF up to about 3.6 GB that no engine above covers; files over 2 GB are split into OPFS shards once |
 | `diskformer` | diskformer engines: weights in OPFS, only what a step needs on the GPU | Gemma 4 26B-A4B (from 2.5 GB), Qwen3.6 35B-A3B (from 4 GB), Ling 3.0 Flash 124B (from 6.8 GB), MiniCPM-V 4.7 with images, Flint 40B, LFM2.5 24B (from 4 GB) |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "anyformer",
-  "version": "0.4.4",
+  "version": "0.4.8",
   "description": "One call to run a model in a browser tab: anyformer reads the Hub repo, probes the browser, and picks the back end (transformers.js, LocalMind's WebGPU kernels, diskformer, kohra) that fits the model's format, architecture and your GPU budget.",
   "type": "module",
   "main": "./index.js",

package/src/backends/kohra.js

@@ -9,8 +9,29 @@ const FP16 = 'onnx/model_fp16_fused.onnx';
 const Q4 = 'onnx/model_q4f16_rtn_sym.onnx';
 const Q4_ORT = '1.26.0-dev.20260416-b7804b056c';
 
+// Models with their own graph and tokens, keyed by config.model_type (kohra index.html REGISTRY at the pinned commit).
+// Nemotron-Labs-Diffusion-3B: a seeded block-KV graph (kohra.js finds its attention_mask input and denoises blocks
+// from the prompt end), mask id 100, EOS 11, and Nemotron's template: a system turn, always, and an empty think block
+// before each assistant reply.
+const SPECIAL = {
+  nemotron_labs_diffusion: {
+    graph: 'onnx/model_kv_q4f16.onnx', opts: { kvCache: true, maskId: 100n, eosId: 11n }, blockCausal: false,
+    // Fast-dLLM decoding: unmask every position at confidence 0.9 or more per step. Same answer in 34 forward passes
+    // where fixed steps took 64 (kohra, primes prompt).
+    threshold: 0.9,
+    template: (messages) => {
+      const [system, rest] = messages[0]?.role === 'system' ? [messages[0].content, messages.slice(1)] : ['', messages];
+      return `<|im_start|>system\n${system}<|im_end|>\n` + rest.map((m) => (m.role === 'assistant'
+        ? `<|im_start|>assistant\n<think></think>${m.content.trim()}<|im_end|>\n`
+        : `<|im_start|>${m.role}\n${m.content}<|im_end|>\n`)).join('') + '<|im_start|>assistant\n<think></think>';
+    },
+  },
+};
+
 const pickGraph = (model, caps) => {
   const have = new Set(model.files.onnx.map((f) => f.name));
+  const special = SPECIAL[model.architecture];
+  if (special) return have.has(special.graph) ? special.graph : null;
   return caps.f16 && have.has(FP16) ? FP16 : have.has(Q4) ? Q4 : null;
 };
 
@@ -18,6 +39,9 @@ const pickGraph = (model, caps) => {
 const chatml = (messages) =>
   messages.map((m) => `<|im_start|>${m.role}\n${m.content}<|im_end|>\n`).join('') + '<|im_start|>assistant\n';
 
+// Text blocks joined: kohra reads text only.
+const text = (messages) => messages.map((m) => (typeof m.content === 'string' ? m : { ...m, content: m.content.map((b) => b.text ?? '').join('\n') }));
+
 export const kohraBackend = {
   id: 'kohra',
   tasks: ['text-generation'],
@@ -28,7 +52,7 @@ export const kohraBackend = {
     if (task && task !== 'text-generation') return { ok: false, code: 'task', reason: `task ${task}; kohra runs text-generation only` };
     if (!caps.webgpu) return { ok: false, code: 'webgpu', reason: 'no WebGPU adapter' };
     const graph = pickGraph(model, caps);
-    if (!graph) return { ok: false, code: 'format', reason: `no kohra graph (${FP16} or ${Q4}) in the repo` };
+    if (!graph) return { ok: false, code: 'format', reason: `no kohra graph (${SPECIAL[model.architecture]?.graph || `${FP16} or ${Q4}`}) in the repo` };
     const bytes = graphBytes(model, graph);
     return { ok: true, estimate: { gpuBytes: bytes, diskBytes: bytes, tokPerSec: null } };
   },
@@ -38,18 +62,21 @@ export const kohraBackend = {
     const { DiffusionLM } = await import(kohraBackend.source);
     onProgress?.({ backend: 'kohra', status: 'loading', file: graph });
     const base = `https://huggingface.co/${model.repo}/resolve/${model.revision || 'main'}`;
+    const special = SPECIAL[model.architecture];
     const lm = await DiffusionLM.from_pretrained({
-      model: `${base}/${graph}`, tokenizer: model.repo, ...(graph === Q4 && { ortVersion: Q4_ORT }),
+      model: `${base}/${graph}`, tokenizer: model.repo, ...(graph === Q4 && { ortVersion: Q4_ORT }), ...special?.opts,
     });
     signal?.throwIfAborted();
-    const blockCausal = model.tags.includes('block-diffusion');
+    const blockCausal = special ? special.blockCausal : model.tags.includes('block-diffusion');
+    const prompt = special?.template || chatml;
     return {
       task: 'text-generation',
       // Diffusion fills the whole block at once, so the reply arrives as one chunk.
-      async *generate(messages, { maxNewTokens = 128, steps, signal } = {}) {
+      // threshold: unmask every position at that confidence per step (Fast-dLLM); null runs fixed steps.
+      async *generate(messages, { maxNewTokens = 128, steps, threshold = special?.threshold ?? null, signal } = {}) {
         signal?.throwIfAborted();
-        const out = await lm.generate(chatml(messages), {
-          maxNewTokens, steps: steps ?? maxNewTokens, blockCausal, chatTemplate: false, stripThink: true,
+        const out = await lm.generate(prompt(text(messages)), {
+          maxNewTokens, steps: steps ?? maxNewTokens, threshold, blockCausal, chatTemplate: false, stripThink: true,
         });
         yield { text: out.text, tokens: out.tokens, tokPerSec: out.tokensPerSecond };
       },

package/src/backends/transformers.js

@@ -2,6 +2,7 @@
 // importing anyformer costs nothing until this back end is chosen.
 
 
+import { AnyformerError } from '../errors.js';
 import { SOURCES } from '../sources.js';
 
 // dtype → the ONNX file transformers.js reads for it (decoder-only repos keep one graph per dtype under onnx/).
@@ -31,6 +32,18 @@ function pickMmDtype(model, caps) {
   return order.find((d) => have.has(`onnx/decoder_model_merged${MM_SUFFIX[d]}.onnx`)) || null;
 }
 const mmBytes = (model, dtype) => mmModules(model).reduce((n, m) => n + graphBytes(model, `onnx/${m}${MM_SUFFIX[dtype]}.onnx`), 0);
+// A text-only export in the multimodal layout (embed_tokens + decoder_model_merged, no vision or audio encoder), e.g.
+// danyalmalik/NVIDIA-Nemotron-3.5-Bolt-E4B-ONNX-WebGPU: loads through the causal-LM class and the tokenizer, as
+// LocalMind does, since there is no processor.
+const CAUSAL = { gemma4: 'Gemma4ForCausalLM' };
+const textOnly = (model) => model.architecture in CAUSAL && !mmModules(model).some((m) => /^(vision|audio)_encoder$/.test(m));
+// How many external-data files each graph has (<graph>_data, <graph>_data_1 …), keyed as transformers.js_config's
+// use_external_data_format is: a repo whose config.json lacks that block fails to mount the data without it. The
+// counts equal the config's in onnx-community's Gemma 4 E2B and Qwen3.5 4B repos.
+const externalData = (model, dtype) => Object.fromEntries(mmModules(model).map((m) => {
+  const file = `onnx/${m}${MM_SUFFIX[dtype]}.onnx`;
+  return [file.slice(5), model.files.all.filter((f) => f.name.startsWith(`${file}_data`)).length];
+}).filter(([, n]) => n > 0));
 const isMultimodal = (model) => model.architecture in MULTIMODAL && model.files.onnx.some((f) => f.name.startsWith('onnx/decoder_model_merged'));
 // The chat tasks a multimodal repo is tagged with on the Hub; all run here as chat.
 const MM_TASKS = ['text-generation', 'image-text-to-text', 'any-to-any'];
@@ -100,6 +113,7 @@ export const transformersBackend = {
     const want = task || model.task || 'text-generation';
     const lib = await import(transformersBackend.source);
     const progress_callback = onProgress && ((p) => onProgress({ backend: 'transformers', ...p }));
+    if (isMultimodal(model) && textOnly(model)) return loadCausal(lib, model, forced || pickMmDtype(model, caps), { progress_callback, signal });
     if (isMultimodal(model)) return loadMultimodal(lib, model, forced || pickMmDtype(model, caps), { progress_callback, signal });
     if (want !== 'text-generation') return loadPipeline(lib, model, want, { progress_callback, signal });
     const dtype = forced || pickDtype(model, caps);
@@ -156,6 +170,34 @@ async function* stream(lib, tokenizer, signal, run) {
   if (failed) throw failed;
 }
 
+// A text-only model in the multimodal layout: the causal-LM class over embed_tokens + decoder_model_merged.
+async function loadCausal(lib, model, dtype, { progress_callback, signal }) {
+  const opts = { revision: model.revision || 'main', progress_callback };
+  const tokenizer = await lib.AutoTokenizer.from_pretrained(model.repo, opts);
+  const net = await lib[CAUSAL[model.architecture]].from_pretrained(model.repo, {
+    ...opts, dtype, device: 'webgpu', use_external_data_format: externalData(model, dtype),
+  });
+  signal?.throwIfAborted();
+  return {
+    task: 'text-generation',
+    generate: (messages, { maxNewTokens = 256, repetitionPenalty, signal } = {}) => stream(lib, tokenizer, signal, (streamer, stopping_criteria) => {
+      const inputs = tokenizer.apply_chat_template(textMessages(messages), { add_generation_prompt: true, enable_thinking: false, return_dict: true });
+      return net.generate({ ...inputs, max_new_tokens: maxNewTokens, do_sample: false, ...(repetitionPenalty && { repetition_penalty: repetitionPenalty }), streamer, stopping_criteria });
+    }),
+    dispose: () => net.dispose?.(),
+  };
+}
+
+// Text blocks joined; a text-only model refuses any other block.
+function textMessages(messages) {
+  return messages.map((m) => {
+    if (typeof m.content === 'string') return m;
+    const other = m.content.find((b) => b.type !== 'text');
+    if (other) throw new AnyformerError('INVALID_ARGUMENT', `this model reads text only; got a ${other.type} block`);
+    return { ...m, content: m.content.map((b) => b.text).join('\n') };
+  });
+}
+
 // A multimodal chat model. Message content is a string or an array of blocks: { type: 'text', text },
 // { type: 'image', image } (ImageData, RawImage or a URL) and, for Gemma 4, { type: 'audio', audio } (16 kHz
 // Float32Array). The template sees placeholders; the processor gets the media, in order.

package/src/models.js

@@ -16,7 +16,9 @@ const GEMMA4_E2B_ONNX = M('onnx-community/gemma-4-E2B-it-ONNX', 'transformers',
 const GEMMA4_E4B_ONNX = M('onnx-community/gemma-4-E4B-it-ONNX', 'transformers', 'text-generation', 'transformers', { budget: 5.3e9, downloadBytes: 5.18e9 });
 const MINICPM5_2B = M('openbmb/MiniCPM5-2B-GGUF', 'wllama', 'text-generation', 'chat', { downloadBytes: 1.56e9 });
 const QWEN3_0_6B_MDLM = M('naklitechie/Qwen3-0.6B-diffusion-mdlm-ONNX', 'kohra', 'text-generation', 'kohra', { downloadBytes: 1.54e9 });
+const NEMOTRON_DIFFUSION_3B = M('naklitechie/Nemotron-Labs-Diffusion-3B-ONNX', 'kohra', 'text-generation', 'kohra', { downloadBytes: 2.77e9 });
 const GEMMA4_26B_A4B = M('google/gemma-4-26B-A4B-it-qat-q4_0-gguf', 'diskformer', 'text-generation', 'diskformer-chat', { budget: 2.5e9, downloadBytes: 14.44e9 });
+const NEMOTRON_3_5_BOLT_E4B = M('danyalmalik/NVIDIA-Nemotron-3.5-Bolt-E4B-ONNX-WebGPU', 'transformers', 'text-generation', 'transformers', { budget: 5e9, downloadBytes: 4.94e9 });
 const MINILM_L6 = M('Xenova/all-MiniLM-L6-v2', 'transformers', 'feature-extraction', 'embed', { downloadBytes: 90e6 });
 const DETR_RESNET_50 = M('Xenova/detr-resnet-50', 'transformers', 'object-detection', 'detect', { downloadBytes: 167e6 });
 
@@ -40,8 +42,12 @@ export const MODELS = {
     MINICPM5_2B,
     /** Text diffusion: the whole reply denoises at once (kohra). */
     QWEN3_0_6B_MDLM,
+    /** NVIDIA's Nemotron-Labs-Diffusion-3B, block diffusion with a KV cache, q4 (kohra). */
+    NEMOTRON_DIFFUSION_3B,
     /** 26B parameters in 2.5 GB of GPU memory; 14.4 GB on disk (diskformer). */
     GEMMA4_26B_A4B,
+    /** NVIDIA's Nemotron 3.5 Bolt E4B (dense, text only), a community q4f16 ONNX export in Gemma 4 E4B's graph. */
+    NEMOTRON_3_5_BOLT_E4B,
   },
   /** Chat with images in the messages: content [{ type: 'image', image }, { type: 'text', text }]. Gemma 4 takes audio too. */
   vision: { DEFAULT: QWEN3_5_4B, QWEN3_5_4B, GEMMA4_E2B_ONNX, GEMMA4_E4B_ONNX },

package/src/sources.js

@@ -6,11 +6,13 @@
 export const PINS = {
   transformers: '4.3.1',
   wllama: '3.4.1',      // LocalMind's pin; its stream-drain workaround (src/backends/wllama.js) is for this version
-  kohra: 'f678fac7342c79b58b640540b8b3a7d078b7ab1a',
-  // v0.6.7: every engine (the disk-tier MoE engines, Ling 3.0 Flash, and LocalMind's whole-model engines LFM2.5,
-  // Gemma 4 E2B/E4B and Ternary Bonsai 2) under engines/, and the llama.cpp path under ggml/, which loads split GGUFs.
-  // The hand engines' ingest streams the GGUF header with progress and resumes a dropped range.
-  diskformer: '663b2b01cfbed95ae8d319ce845d8aea759d60cf',
+  // v0.3.0: seeded block-KV graphs (Nemotron-Labs-Diffusion-3B), chatFormat, split external data.
+  kohra: '85e01f0193163b0f2e9696d4f6f5e12b68a277e9',
+  // v0.6.8: every engine (the disk-tier MoE engines, Ling 3.0 Flash, and LocalMind's whole-model engines LFM2.5,
+  // Gemma 4 E2B/E4B and Ternary Bonsai 2) under engines/, and the llama.cpp path under ggml/, which loads split GGUFs
+  // and drafters, and reads past 2 GiB (Nemotron 3.5 Bolt E4B needs v0.6.8 or later). The hand engines' ingest
+  // streams the GGUF header with progress and resumes a dropped range.
+  diskformer: '3aad103e47cac76c1471e92a438edfe1c80dd86e',
 };
 
 const gh = (repo, sha, path) => `https://cdn.jsdelivr.net/gh/NakliTechie/${repo}@${sha}/${path}`;