anyformer 0.4.10 → 0.4.12

package/CHANGELOG.md

@@ -1,5 +1,45 @@
 # Changelog
 
+## 0.4.12 — 2026-10-29
+
+### Changed
+- transformers.js pin 4.3.1 → 4.3.2 (in step with LocalMind), which adds `model_type` `lfm3` and `lfm3_vl`.
+- transformers back end: a graph whose last-position input is named `logits_to_keep` (LiquidAI's LFM3 ONNX export)
+  gets it fed at the session's run; transformers.js 4.3.2 feeds only `num_logits_to_keep` and stopped with "Missing
+  the following inputs". `LiquidAI/LFM3-1.7B-ONNX` now loads and generates through `chat.html` (64 tokens at
+  67 tok/s, Chrome, this Mac, 2026-10-28).
+- transformers back end, text pipeline path: `thinking: false` by default, as on every other back end. The template
+  renders with `enable_thinking` off (through `tokenizer_encode_kwargs`), so LFM3 answers directly; `thinking: true`
+  per request turns it on. Templates without the switch ignore it.
+- `test/browser/chat.html` takes `?dtype=` and fails a reply that contains `<think>` text.
+
+### Added
+- `MODELS.chat.LFM3_1_7B`: `LiquidAI/LFM3-1.7B-ONNX` on `transformers` (asked for by LocalMind for Chirag). Browser
+  chat check on 2026-10-29, Chrome, this Mac: on topic, no `<think>` text, on q4 (anyformer's pick, 1.63 GB; 82.4
+  tok/s) and on q4f16 (`dtype: 'q4f16'`, LocalMind's precision, 1.38 GB; 76.6 tok/s), the same 61-token reply.
+
+## 0.4.11 — 2026-10-24
+
+### Added
+- `MODELS.chat.DEEPSEEK_V4_1_FLASH`: DeepSeek V4.1 Flash (763B MoE, ISTA-DASLab GSQ-RCO IQ1_M) on `diskformer-llama`,
+  budget 8 GB, `experimental: true`, so LocalMind shows it as an opt-in tech demo (Chirag's approval, relayed by the
+  LocalMind session). 231,316,411,072 bytes on disk: 4 parts plus the 110.6 GB Engram file. Very slow: diskformer
+  measured about 0.4 tok/s natively from a USB SSD.
+- `diskformer-llama`: side files (`SIDE_FILES`). A model's side file downloads with its parts under its own name,
+  counts toward the disk bytes, goes to llama.cpp as `load({ extra })`, and is never picked as the model. The
+  `deepseek41-engram` architecture is accepted for the demo file only; another quant in the repo is refused.
+
+### Changed
+- diskformer pinned to v0.6.9 (53c976e), which adds `load({ extra })`. diskformer reports Granite and Bolt 9/9 on it.
+
+### Verified
+- npm test 92/92 (fixture of the repo at cb02422c: the IQ1_M set with its side file, no Engram file alone, a missing
+  side file refuses, `extra` reaches the fake llama.js). `plan()` against the live Hub picks `diskformer-llama` at 8 GB
+  with 231,316,411,072 disk bytes.
+- Not run, on purpose (Chirag: no live runs or benchmarks of this model): any browser load through anyformer.
+  diskformer checked one hand-written prompt through `complete()` in Chrome; the chat-template path that `generate()`
+  uses, and so LocalMind's chat, has not run in any browser.
+
 ## 0.4.10 — 2026-10-24
 
 ### Changed
@@ -9,8 +49,10 @@
 
 ### Verified
 - npm test 90/90. kohra's own Chrome check on v0.3.1 (kohra session): the first Nemotron load cached all 3 files with
-  their content-length; a reload was ready in 7.4 s with no graph fetches; output unchanged. anyformer's own browser
-  check is owed (the GPU is with Quench), and the npm publish waits for it.
+  their content-length; a reload was ready in 7.4 s with no graph fetches; output unchanged. Through anyformer
+  (`npm run test:browser -- kohra-cache`, Chrome, GPU shared, after the npm publish): the first load sent 35,416
+  progress events to `onProgress` and answered on topic in 200 s; `stored()` listed 2,735,424,112 bytes under kohra;
+  the second load took 5.1 s with the same reply; `remove()` freed 2,752,678,776 bytes (graph and tokenizer).
 
 ## 0.4.9 — 2026-10-24
 

package/README.md

@@ -14,7 +14,7 @@ browser storage; no account, no server, no telemetry.
 | How | Command |
 |---|---|
 | npm | `npm install anyformer` |
-| CDN | `import { load } from 'https://cdn.jsdelivr.net/npm/anyformer@0.4.10/+esm';` |
+| CDN | `import { load } from 'https://cdn.jsdelivr.net/npm/anyformer@0.4.11/+esm';` |
 | Copy | `index.js` and `src/`; nothing else is needed |
 
 ```js
@@ -54,12 +54,12 @@ back end's answer: yes with an estimate (GPU bytes, disk bytes, tok/s when measu
 
 | Back end | Runs | Models |
 |---|---|---|
-| `transformers` | transformers.js 4.3.1: text generation (Nemotron 3.5 Bolt E4B among them), image and audio chat (Gemma 4, Qwen3.5), embeddings, speech to text, detection, classification, depth, segmentation | ONNX repos in the onnx-community / Xenova layout |
+| `transformers` | transformers.js 4.3.2: text generation (Nemotron 3.5 Bolt E4B and LFM3 1.7B among them), image and audio chat (Gemma 4, Qwen3.5), embeddings, speech to text, detection, classification, depth, segmentation | ONNX repos in the onnx-community / Xenova layout |
 | `kohra` | kohra, text-diffusion over onnxruntime-web | MDLM and BD3LM repos tagged `text-diffusion`; Nemotron-Labs-Diffusion-3B |
 | `localmind` | LocalMind's hand-written WebGPU kernels (shipped in diskformer), the whole model on the GPU | any LFM2 / LFM2.5 GGUF; Gemma 4 E2B / E4B QAT-mobile; Ternary Bonsai 2 |
 | `wllama` | llama.cpp in WebAssembly: WebGPU when the budget holds the model, the CPU otherwise (no WebGPU needed) | any GGUF up to about 3.6 GB that no engine above covers; files over 2 GB are split into OPFS shards once |
 | `diskformer` | diskformer engines: weights in OPFS, only what a step needs on the GPU | Gemma 4 26B-A4B (from 2.5 GB), Qwen3.6 35B-A3B (from 4 GB), Ling 3.0 Flash 124B (from 6.8 GB), MiniCPM-V 4.7 with images, Flint 40B, LFM2.5 24B (from 4 GB) |
-| `diskformer-llama` | diskformer's llama.cpp build with WebGPU, mixture-of-experts layers paged in from OPFS | GGUFs of the 17 MoE architectures diskformer has checked (Granite 4.0 H, gpt-oss, GLM-4.7-Flash, LFM2 MoE…), split sets included; a hand engine above wins where it covers the model |
+| `diskformer-llama` | diskformer's llama.cpp build with WebGPU, mixture-of-experts layers paged in from OPFS | GGUFs of the 17 MoE architectures diskformer has checked (Granite 4.0 H, gpt-oss, GLM-4.7-Flash, LFM2 MoE…), split sets included; a hand engine above wins where it covers the model; DeepSeek V4.1 Flash as an experimental tech demo (231 GB, side file included) |
 
 Each runtime loads from an exact version or commit in [src/sources.js](src/sources.js); set a back end's `.source`
 (for example `kohraBackend.source`) to load another copy. `BACKENDS` lists the back ends `load()` asks by default: each joins after its browser test passes.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "anyformer",
-  "version": "0.4.10",
+  "version": "0.4.12",
   "description": "One call to run a model in a browser tab: anyformer reads the Hub repo, probes the browser, and picks the back end (transformers.js, LocalMind's WebGPU kernels, diskformer, kohra) that fits the model's format, architecture and your GPU budget.",
   "type": "module",
   "main": "./index.js",

package/src/backends/diskformer-llama.js

@@ -35,11 +35,29 @@ export const CHECKED_FILES = [
 
 const SPLIT = /-(\d{5})-of-(\d{5})\.gguf$/i;
 
-// The GGUF to load, as { name, size, parts: [{ name, size }] }: the language model (no mmproj vision tower), one file or
-// every part of a split set (name-00001-of-0000n.gguf, named by its first part; size is the sum). The caller's file
-// when given; else the file diskformer checked; else the preferred quant, a single file before a split set.
+// Side files a model opens beside its GGUF (llama.load({ extra }), diskformer v0.6.9+), keyed by the GGUF's file name:
+// they download with its parts, count toward its disk bytes, and are never picked as the model itself. DeepSeek V4.1
+// Flash is a tech demo in diskformer (docs/coverage.md: one prompt identical to native in Chrome, about 0.4 tok/s, the
+// chat template not checked), so its architecture is accepted for this file only, not added to LLAMA_ARCHITECTURES.
+export const SIDE_FILES = {
+  'DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ1_M-00001-of-00004.gguf': { architecture: 'deepseek41-engram', files: ['DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf'] },
+};
+const SIDE_NAMES = new Set(Object.values(SIDE_FILES).flatMap((s) => s.files));
+
+// The GGUF to load, as { name, size, parts: [{ name, size }], extra: [{ name, size }] }: the language model (no mmproj
+// vision tower), one file or every part of a split set (name-00001-of-0000n.gguf, named by its first part), and its
+// side files (SIDE_FILES); size is the sum of all of them. The caller's file when given; else the file diskformer
+// checked; else the preferred quant, a single file before a split set. A model missing a side file cannot load.
 export function pickGguf(model, file = null) {
-  const lms = model.files.gguf.filter((f) => !/mmproj/i.test(f.name));
+  const lms = model.files.gguf.filter((f) => !/mmproj/i.test(f.name) && !SIDE_NAMES.has(opfsName(f)));
+  const f = pickModel(lms, file);
+  if (!f) return null;
+  const extra = (SIDE_FILES[opfsName(f)]?.files || []).map((n) => model.files.gguf.find((x) => opfsName(x) === n));
+  if (extra.some((x) => !x)) return null;
+  return { ...f, size: f.size + extra.reduce((n, x) => n + x.size, 0), extra };
+}
+
+function pickModel(lms, file) {
   const sets = new Map();
   for (const f of lms.filter((x) => SPLIT.test(x.name))) {
     const base = f.name.replace(SPLIT, '');
@@ -54,7 +72,7 @@ export function pickGguf(model, file = null) {
     }),
   ];
   if (file) return candidates.find((f) => f.name === file) || null;
-  const checked = candidates.find((f) => CHECKED_FILES.includes(opfsName(f)));
+  const checked = candidates.find((f) => CHECKED_FILES.includes(opfsName(f)) || SIDE_FILES[opfsName(f)]);
   if (checked) return checked;
   for (const q of QUANTS) {
     const f = candidates.find((x) => new RegExp(`(^|[-_.])${q}([-_.]|$)`, 'i').test(x.name.replace(SPLIT, '').replace(/\.gguf$/i, '')));
@@ -101,8 +119,9 @@ export const diskformerLlamaBackend = {
   canRun(model, caps, { budget, task, file = null }) {
     if (!model.formats.includes('gguf')) return { ok: false, code: 'format', reason: 'no GGUF in the repo' };
     if (task && !diskformerLlamaBackend.tasks.includes(task)) return { ok: false, code: 'task', reason: `task ${task}; diskformer's llama.cpp path runs text generation only` };
-    if (!LLAMA_ARCHITECTURES.includes(model.architecture)) return { ok: false, code: 'model', reason: `architecture ${model.architecture || 'unknown'} is not one diskformer's llama.cpp path has checked (${LLAMA_ARCHITECTURES.join(', ')})` };
     const f = pickGguf(model, file);
+    const demo = f && SIDE_FILES[opfsName(f)]?.architecture === model.architecture;
+    if (!LLAMA_ARCHITECTURES.includes(model.architecture) && !demo) return { ok: false, code: 'model', reason: `architecture ${model.architecture || 'unknown'} is not one diskformer's llama.cpp path has checked (${LLAMA_ARCHITECTURES.join(', ')})` };
     if (!f) return { ok: false, code: 'model', reason: file ? `${file} is not a language-model GGUF in the repo (a split set is named by its first part, with every part present)` : `no GGUF at ${QUANTS.join(', ')} in the repo (a split set needs every part)` };
     if (!caps.webgpu) return { ok: false, code: 'webgpu', reason: 'no WebGPU adapter' };
     if (!caps.memory64 || !caps.jspi) return { ok: false, code: 'webgpu', reason: 'the llama.cpp build needs wasm memory64 and JSPI (Chrome or Edge 137+)' };
@@ -121,9 +140,10 @@ export const diskformerLlamaBackend = {
     signal?.addEventListener('abort', stop, { once: true });
     let info;
     try {
-      // Each part downloads under its own name; progress counts bytes over the whole set.
+      // Each part and side file downloads under its own name; progress counts bytes over all of them.
+      const files = [...f.parts, ...f.extra];
       let before = 0;
-      for (const part of f.parts) {
+      for (const part of files) {
         const name = opfsName(part);
         const have = (await llama.list()).find((m) => m.name === name && !m.partial);
         if (!have || have.size !== part.size) {
@@ -134,9 +154,9 @@ export const diskformerLlamaBackend = {
         }
         before += part.size;
       }
-      await writeIndex({ ...(await readIndex()), ...Object.fromEntries(f.parts.map((p) => [opfsName(p), model.repo])) });
+      await writeIndex({ ...(await readIndex()), ...Object.fromEntries(files.map((p) => [opfsName(p), model.repo])) });
       say({ status: 'load' });
-      info = await llama.load({ name: opfsName(f), budget, ctx: maxCtx }, (p) => say({ status: 'plan', plan: p }));
+      info = await llama.load({ name: opfsName(f), budget, ctx: maxCtx, extra: f.extra.map(opfsName) }, (p) => say({ status: 'plan', plan: p }));
       signal?.throwIfAborted();
     } catch (e) {
       llama.terminate();

package/src/backends/transformers.js

@@ -120,16 +120,33 @@ export const transformersBackend = {
     const gen = await lib.pipeline('text-generation', model.repo, {
       device: 'webgpu', dtype, revision: model.revision || 'main', progress_callback,
     });
+    feedLogitsToKeep(lib, gen.model);
     signal?.throwIfAborted();
     return {
       task: 'text-generation',
-      generate: (messages, { maxNewTokens = 256, repetitionPenalty, signal } = {}) => stream(lib, gen.tokenizer, signal,
-        (streamer, stopping_criteria) => gen(messages, { max_new_tokens: maxNewTokens, do_sample: false, ...(repetitionPenalty && { repetition_penalty: repetitionPenalty }), streamer, stopping_criteria })),
+      // thinking: false (the default, as on every back end here) renders the template with enable_thinking off, so a
+      // model that reasons by default (LFM3) answers directly; templates without the switch ignore it.
+      generate: (messages, { maxNewTokens = 256, repetitionPenalty, signal, thinking = false } = {}) => stream(lib, gen.tokenizer, signal,
+        (streamer, stopping_criteria) => gen(messages, { max_new_tokens: maxNewTokens, do_sample: false, ...(repetitionPenalty && { repetition_penalty: repetitionPenalty }), tokenizer_encode_kwargs: { enable_thinking: thinking === true }, streamer, stopping_criteria })),
       dispose: () => gen.dispose?.(),
     };
   },
 };
 
+// LiquidAI's LFM3 ONNX export names its last-position input logits_to_keep (shape [1]); transformers.js 4.3.2 feeds
+// only num_logits_to_keep and stops with "Missing the following inputs: logits_to_keep". Hide the input from
+// transformers.js's check and add it at the session's run, set to 1: generate() reads only the last position's logits.
+// LocalMind carries the same shim (index.html feedLogitsToKeep).
+function feedLogitsToKeep(lib, net) {
+  const sess = net?.sessions?.model;
+  if (!sess?.inputNames.includes('logits_to_keep')) return;
+  const names = sess.inputNames.filter((n) => n !== 'logits_to_keep');
+  const one = new lib.Tensor('int64', [1n], [1]).ort_tensor;
+  const run = sess.run.bind(sess);
+  Object.defineProperty(sess, 'inputNames', { value: names });
+  sess.run = (feeds, opts) => run({ ...feeds, logits_to_keep: one }, opts);
+}
+
 // A non-generation task: one pipeline, run(input, opts) calls it. ImageData (from a canvas) becomes a RawImage; URLs,
 // RawImage and, for speech, 16 kHz Float32Array samples pass through as transformers.js takes them.
 async function loadPipeline(lib, model, task, { progress_callback, signal }) {

package/src/models.js

@@ -20,7 +20,12 @@ const MINICPM5_2B = M('openbmb/MiniCPM5-2B-GGUF', 'wllama', 'text-generation', '
 const QWEN3_0_6B_MDLM = M('naklitechie/Qwen3-0.6B-diffusion-mdlm-ONNX', 'kohra', 'text-generation', 'kohra', { downloadBytes: 1.54e9, diffusion: true });
 const NEMOTRON_DIFFUSION_3B = M('naklitechie/Nemotron-Labs-Diffusion-3B-ONNX', 'kohra', 'text-generation', 'kohra', { downloadBytes: 2.77e9, diffusion: true, experimental: true });
 const GEMMA4_26B_A4B = M('google/gemma-4-26B-A4B-it-qat-q4_0-gguf', 'diskformer', 'text-generation', 'diskformer-chat', { budget: 2.5e9, downloadBytes: 14.44e9 });
+const LFM3_1_7B = M('LiquidAI/LFM3-1.7B-ONNX', 'transformers', 'text-generation', 'chat', { downloadBytes: 1.63e9 });
 const NEMOTRON_3_5_BOLT_E4B = M('danyalmalik/NVIDIA-Nemotron-3.5-Bolt-E4B-ONNX-WebGPU', 'transformers', 'text-generation', 'transformers', { budget: 5e9, downloadBytes: 4.94e9 });
+// A tech demo, not a gated model: 231 GB on disk (4 IQ1_M parts plus the 110.6 GB Engram side file), about 0.4 tok/s.
+// diskformer checked one hand-written prompt through complete(); generate() renders the GGUF's chat template, which no
+// browser run has checked.
+const DEEPSEEK_V4_1_FLASH = M('ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF', 'diskformer-llama', 'text-generation', 'llama-gate', { budget: 8e9, downloadBytes: 231316411072, experimental: true });
 const MINILM_L6 = M('Xenova/all-MiniLM-L6-v2', 'transformers', 'feature-extraction', 'embed', { downloadBytes: 90e6 });
 const DETR_RESNET_50 = M('Xenova/detr-resnet-50', 'transformers', 'object-detection', 'detect', { downloadBytes: 167e6 });
 
@@ -50,6 +55,10 @@ export const MODELS = {
     GEMMA4_26B_A4B,
     /** NVIDIA's Nemotron 3.5 Bolt E4B (dense, text only), a community q4f16 ONNX export in Gemma 4 E4B's graph. */
     NEMOTRON_3_5_BOLT_E4B,
+    /** LiquidAI's LFM3 1.7B (dense), LiquidAI's own ONNX export; thinking off unless a request asks for it. */
+    LFM3_1_7B,
+    /** DeepSeek V4.1 Flash (763B MoE), a tech demo on diskformer's llama.cpp path: 231 GB on disk, very slow. */
+    DEEPSEEK_V4_1_FLASH,
   },
   /** Chat with images in the messages: content [{ type: 'image', image }, { type: 'text', text }]. Gemma 4 takes audio too. */
   vision: { DEFAULT: QWEN3_5_4B, QWEN3_5_4B, GEMMA4_E2B_ONNX, GEMMA4_E4B_ONNX },

package/src/sources.js

@@ -4,16 +4,17 @@
 // transformers.js and wllama move only by hand, in step with LocalMind's pins (plan/history.md, 2026-10-08).
 
 export const PINS = {
-  transformers: '4.3.1',
+  transformers: '4.3.2', // 4.3.2 adds model_type lfm3 (LFM3 1.7B)
   wllama: '3.4.1',      // LocalMind's pin; its stream-drain workaround (src/backends/wllama.js) is for this version
   // v0.3.1: seeded block-KV graphs (Nemotron-Labs-Diffusion-3B), chatFormat, split external data; download progress
   // (progress_callback) and the graph files cached in Cache Storage 'kohra-models'.
   kohra: 'bf95d44c3e3cfc001201baa03fa121c2bf9c7d73',
+  // v0.6.9: llama.load({ extra }) mounts side files beside the GGUF (DeepSeek V4.1 Flash's Engram tables).
   // v0.6.8: every engine (the disk-tier MoE engines, Ling 3.0 Flash, and LocalMind's whole-model engines LFM2.5,
   // Gemma 4 E2B/E4B and Ternary Bonsai 2) under engines/, and the llama.cpp path under ggml/, which loads split GGUFs
   // and drafters, and reads past 2 GiB (Nemotron 3.5 Bolt E4B needs v0.6.8 or later). The hand engines' ingest
   // streams the GGUF header with progress and resumes a dropped range.
-  diskformer: '3aad103e47cac76c1471e92a438edfe1c80dd86e',
+  diskformer: '53c976e43d34ac2d2601a731124de81cb07fdab4',
 };
 
 const gh = (repo, sha, path) => `https://cdn.jsdelivr.net/gh/NakliTechie/${repo}@${sha}/${path}`;