anyformer 0.0.0-stage → 0.1.0

package/CHANGELOG.md

@@ -0,0 +1,36 @@
+# Changelog
+
+## 0.1.0 — 2026-10-14
+
+The first release with code. `anyformer@0.0.1` reserved the name and held none.
+
+### Added
+- `plan(repo)` and `load(repo)`: read the Hub repo, probe the browser, ask every back end, run the best yes. A refusal
+  carries a closed code (`format`, `model`, `task`, `webgpu`, `f16`, `opfs`, `budget`, `threw`) and one remedy in `REFUSALS`.
+- Five back ends:
+  - `transformers`: transformers.js 4.3.1. Text generation, image and audio chat (Gemma 4, Qwen3.5), embeddings,
+    speech to text, detection, classification, depth, segmentation.
+  - `kohra`: text diffusion (MDLM, BD3LM) over onnxruntime-web.
+  - `localmind`: LocalMind's hand-written WebGPU kernels (LFM2 / LFM2.5, Gemma 4 E2B / E4B, Ternary Bonsai 2).
+  - `wllama`: llama.cpp in WebAssembly for any other GGUF up to about 3.6 GB, on WebGPU or the CPU; files over 2 GB
+    split into OPFS shards once.
+  - `diskformer`: weights in OPFS, only what a step needs on the GPU (Gemma 4 26B-A4B from 2.5 GB, Qwen3.6 35B-A3B from 4 GB).
+- `MODELS`: models that pass a browser test, loadable by entry.
+- `pipeline(task, repo)`: transformers.js's call shape over any back end.
+- `stored()` and `remove(repo)`: what is downloaded in this browser, across every back end.
+- Coded errors: `AnyformerError` with a code from `ERROR_CODES`, checked with `isAnyformerError(e, code)`.
+- One `generate()` at a time per session, in call order.
+- Options: `load()` takes `task`, `maxCtx`, `dtype` and `signal`; `generate()` takes `maxNewTokens`, `repetitionPenalty`
+  and `signal`; each back end's `.source` (for example `kohraBackend.source`) overrides where its runtime loads from.
+- Pinned runtimes in `src/sources.js` (exact versions and commits); `npm run pin` moves them.
+- Agent face: 9 tools over one manifest, through WebMCP's `navigator.modelContext` or `window.anyformer.tools`.
+- The workbench (`examples/chat/`): pick a model, see the plan, then chat, compare embeddings or detect objects.
+
+### Measured (Chrome, M4 Pro, 24 GB)
+- 13 of 13 LocalMind Hub models plan at an 8 GB budget; 12 ran in a browser test.
+- 10 of 10 replies byte-identical with LocalMind direct on the same transformers.js pin.
+- Speed 0.99–1.02× LocalMind direct on the steady back ends.
+
+### Not tested yet
+Audio input blocks, abort mid-reply, speech / classification / depth / segmentation in a browser, any machine other
+than the M4 Pro, and Qwen3.6 35B-A3B (it needs more than this machine's 24 GB).

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Chirag Patnaik
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,119 @@
-# Temporary Holding Version
+<h1 align="center">anyformer.js</h1>
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**One call to run a model in a browser tab: anyformer reads the repo, probes the browser, and picks the back end that
+fits the model's format, architecture and your GPU budget.**
+
+Plain ES modules, no build step, no dependencies to install. Chromium with WebGPU (GGUF models also run on the CPU). Weights stay in the user's own
+browser storage; no account, no server, no telemetry.
+
+[![license](https://img.shields.io/badge/license-MIT-555?style=flat-square)](LICENSE)
+[![dependencies](https://img.shields.io/badge/dependencies-none-555?style=flat-square)](package.json)
+
+## Install
+
+| How | Command |
+|---|---|
+| npm | `npm install anyformer` |
+| CDN | `import { load } from 'https://cdn.jsdelivr.net/npm/anyformer@0.1.0/+esm';` |
+| Copy | `index.js` and `src/`; nothing else is needed |
+
+```js
+import { load } from 'anyformer';
+
+const chat = await load('onnx-community/Qwen2.5-0.5B-Instruct', { budget: 2.5e9, onProgress: console.log });
+for await (const { text } of chat.generate([{ role: 'user', content: 'What does a CPU do?' }], { maxNewTokens: 64 }))
+  show(text);
+```
+
+Every session has `run(input)`: chat messages for text generation (content a string, or blocks such as
+`[{ type: 'image', image }, { type: 'text', text }]` for Gemma 4 and Qwen3.5), a string or strings for embeddings, ImageData for
+vision, 16 kHz samples for speech. `load(repo, { task })` picks a task when a repo has more than one. The first load
+downloads the weights into the browser; later loads read them from there. Each back end's runtime loads
+from jsDelivr only when that back end is chosen.
+
+## Try it
+
+Serve the repo and open the workbench: pick a model, see which back end anyformer chose and why every other one
+refused, then chat (with an image for the vision models), compare embeddings, or detect objects; remove downloads.
+
+```bash
+python3 -m http.server 8000      # then open http://localhost:8000/examples/chat/
+```
+
+## Why
+
+You have a model id and a browser tab. Whether it runs depends on its format (ONNX, GGUF), its architecture, whether it
+is autoregressive or diffusion, and how much GPU memory the tab can take. Each runtime covers part of that, and you
+find out which part by failing to load.
+
+anyformer asks every back end first. `plan(repo)` returns the model's description, the browser's capabilities and each
+back end's answer: yes with an estimate (GPU bytes, disk bytes, tok/s when measured), or no with a code and a reason.
+`load(repo)` runs the best yes.
+
+## Back ends
+
+| Back end | Runs | Models |
+|---|---|---|
+| `transformers` | transformers.js 4.3.1: text generation, image and audio chat (Gemma 4, Qwen3.5), embeddings, speech to text, detection, classification, depth, segmentation | ONNX repos in the onnx-community / Xenova layout |
+| `kohra` | kohra, text-diffusion over onnxruntime-web | MDLM and BD3LM repos tagged `text-diffusion` |
+| `localmind` | LocalMind's hand-written WebGPU kernels, the whole model on the GPU | any LFM2 / LFM2.5 GGUF; Gemma 4 E2B / E4B QAT-mobile; Ternary Bonsai 2 |
+| `wllama` | llama.cpp in WebAssembly: WebGPU when the budget holds the model, the CPU otherwise (no WebGPU needed) | any GGUF up to about 3.6 GB that no engine above covers; files over 2 GB are split into OPFS shards once |
+| `diskformer` | diskformer engines: weights in OPFS, only what a step needs on the GPU | Gemma 4 26B-A4B (from 2.5 GB), Qwen3.6 35B-A3B (from 4 GB) |
+
+Each runtime loads from an exact version or commit in [src/sources.js](src/sources.js); set a back end's `.source`
+(for example `kohraBackend.source`) to load another copy. `BACKENDS` lists the back ends `load()` asks by default: each joins after its browser test passes.
+
+## When no back end can run it
+
+`load()` throws an `AnyformerError` with code `NO_BACKEND` and the plan, naming each back end, its code and its
+reason. Codes are a closed set, each with one
+remedy in `REFUSALS`: `format`, `model`, `task`, `webgpu`, `f16`, `opfs`, `budget` (with `need`, the budget that would pass)
+and `threw`. Every other failure is an `AnyformerError` too, with a code from `ERROR_CODES` (`LOAD_FAILED`,
+`BUDGET_TOO_SMALL`, `ABORTED`, `EXECUTION_FAILED`, `INVALID_ARGUMENT`, `DISPOSED`); check it with
+`isAnyformerError(e, code)`. A session runs one `generate()` at a time, in call order.
+
+## Commands
+
+```js
+plan(repo, { budget, task })      // what would run it, and why the others would not; loads nothing
+load(repo, { budget, task, maxCtx, onProgress, signal })  // → { backend, task, plan, run(input), generate?, dispose() }
+pipeline(task, repo, { budget })  // transformers.js's call shape: gen(messages) → [{ generated_text }]
+probe()                           // WebGPU, shader-f16, subgroups, limits, OPFS quota, memory64
+describe(repo)                    // Hub repo → formats, architecture, task, tags, files
+resolve(model, caps, backends, { budget })  // rank back ends for a described model
+stored() · remove(repo)           // what is downloaded in this browser; delete one
+MODELS.chat.DEFAULT               // models that pass a browser test; load(MODELS.chat.GEMMA4_26B_A4B) works too
+registerTools()                   // publish the agent face (below)
+```
+
+Agents get the same calls as tools (`anyformer_models`, `_probe`, `_plan`, `_load`, `_generate`, `_sessions`, `_unload`,
+`_stored`, `_remove`) through
+WebMCP's `navigator.modelContext` where the browser has it and `window.anyformer.tools` otherwise; `anyformer_load`
+loads nothing until the caller accepts the plan's disk bytes, and `anyformer_remove` deletes nothing without
+`confirm: true`.
+
+## Verify it yourself
+
+```bash
+npm test                                   # Node: probe, describe, resolver, back-end verdicts, agent face
+npm run test:browser -- probe              # headless Chrome: WebGPU, f16, OPFS
+npm run test:browser -- transformers       # Qwen2.5-0.5B answers on topic
+npm run test:browser -- kohra              # Qwen3-0.6B MDLM answers on topic
+npm run test:browser -- embed             # MiniLM embeddings rank a paraphrase above an unrelated sentence
+npm run test:browser -- detect            # DETR finds both cats in cats.jpg
+npm run test:browser -- examples/chat/index.html --query "auto=1"   # the workbench loads and answers
+npm run test:browser -- pipeline --port 8192 --profile ~/.cache/anyformer-chat-profile   # pipeline() on a GGUF
+npm run test:browser -- store             # list a download, remove it
+npm run test:browser -- chat --query "repo=LiquidAI/LFM2.5-1.2B-Instruct-GGUF"   # a LocalMind engine
+npm run test:browser -- vision --query "repo=onnx-community/Qwen3.5-4B-ONNX-OPT"   # names the two cats
+npm run test:browser -- diskformer-chat    # Gemma 4 26B-A4B at 2.5 GB: 9/9 replies identical to llama.cpp b9830
+```
+
+The browser tests fail a reply that is off topic or repeats itself, and `diskformer-chat` fails any reply that differs
+from llama.cpp's by one character. Only Apple silicon (M4 Pro, Chrome) has run them. Not tested yet: audio input blocks, abort mid-reply, speech / classification / depth / segmentation in a browser, any
+machine other than the M4 Pro, and Qwen3.6 35B-A3B (it needs more than that machine's 24 GB).
+
+## License
+
+MIT. Spec and agent contract: [SPEC.md](SPEC.md) · depends on [diskformer.js](https://github.com/NakliTechie/diskformer.js)
+· [kohra](https://github.com/NakliTechie/kohra) · [LocalMind](https://github.com/NakliTechie/LocalMind)

package/SPEC.md

@@ -0,0 +1,68 @@
+# anyformer.js — spec
+
+Tier: Product (several back ends, a resolver, one store). anyformer depends on diskformer and kohra; neither depends on
+anyformer.
+
+## §0 Agent contract (DRIVER v0.1 pass, 2026-10-08)
+
+The driver: an agent in a browser tab (or a script) that must run a model it names by repo id, with no idea which
+runtime fits, how much GPU memory the tab may take, or whether a download is 0.5 GB or 36 GB. It is context-poor, may
+be killed mid-turn, and must not burn a user's disk or bandwidth by guessing.
+
+1. **One perception act: `anyformer_plan` (`plan()`).** One call returns the model's formats, architecture and task,
+   the browser's caps, the budget, the chosen back end, and every back end's verdict. The agent never probes or
+   fetches separately before deciding.
+2. **Closed vocabularies.** Errors from load() and a session are `AnyformerError` with `code` in `ERROR_CODES`
+   (NO_BACKEND · LOAD_FAILED · BUDGET_TOO_SMALL+need · ABORTED · EXECUTION_FAILED · INVALID_ARGUMENT · DISPOSED),
+   checked by name and code (`isAnyformerError`), so two copies of the library agree. A refusal is `{code, reason, need?}` with `code` in `REFUSALS`: format · model · task · webgpu ·
+   f16 · opfs · budget · threw. Each code has one remedy (`REFUSALS[code]`); `budget` carries `need`, the bytes that
+   would pass. Tool consequences are `read` · `stages` · `mutates`. Tool errors are `{error}` data, never throws.
+3. **One verdict per next action.** `chosen` set → load. `chosen` null → read the codes: format/model → another repo;
+   webgpu/f16/opfs → another browser or GPU (nothing in this tab fixes it); budget → re-plan with `need`; threw → a bug.
+4. **Bounded output.** `anyformer_plan` grows with the number of back ends, never with the repo's file count (the file
+   list stays out). The journal holds the last 50 calls. `generate` returns one reply.
+5. **Every failure names its remedy.** NO_BACKEND lists each back end with its code and reason; `REFUSALS[code]`
+   is the next move.
+6. **Crash-safe and idempotent.** A session runs one generate at a time; dispose() waits for it and is idempotent. `plan` and `probe` have no side effects. Weights land in OPFS through diskformer's
+   resumable ingest (an interrupted download resumes); transformers.js and kohra use the browser HTTP cache. Loading
+   twice loads twice; it never corrupts a store.
+7. **The tool holds the memory.** `anyformer_sessions` renders the open sessions and the last 50 tool calls with the
+   door each came through (`direct`, `window`, `modelContext`).
+8. **Accretion.** A back end joins `BACKENDS` only after its browser test passes (test/registry.test.js pins the list);
+   every escaped wrong answer becomes a browser-test check (the transformers.js q4f16 word salad became `onTopic`).
+9. **The tower.** One line per layer; each reads only the one below.
+   - `probe` (browser caps) · `describe` (Hub repo → descriptor)
+   - back ends: `canRun` (pure, descriptor + caps) · `load` → session
+   - `resolve` (rank the verdicts) → `plan` (probe + describe + resolve) → `load` (plan + back-end load)
+   - agent face: `TOOLS` manifest → `call` → WebMCP `registerTool` / `window.anyformer.tools`
+10. **The evaluator stays outside.** The browser tests (test/browser/) judge back ends against fixed references:
+    llama.cpp b9830 replies for diskformer, an on-topic check for transformers.js and kohra. A page that times out
+    exits 2 (indeterminate), never 0. Staging is enforced in the tool, not left to the agent: `anyformer_load` loads
+    nothing unless `acceptDiskBytes` covers the plan's disk bytes.
+
+Self-check: perception act ✓ · closed vocabularies ✓ · output bound ✓ · crash-safety ✓ · trajectory ✓ · accretion ✓ ·
+tower ✓ · evaluator boundary ✓.
+
+## What shipped (Batch A)
+
+- `load(repo, {budget, maxCtx, backends, onProgress, signal})` → `{backend, plan, generate(messages, {maxNewTokens, signal}),
+  dispose()}`; `generate` yields `{text, tokens}`, `text` cumulative.
+- Tasks: `plan/load(repo, { task })`, default the repo's pipeline tag; every session has `run(input)`, text generation
+  also `generate()`. Back ends list their `tasks`.
+- Back ends: transformers.js 4.3.1 (ONNX text-generation, q4 first), kohra @main (text-diffusion ONNX), diskformer
+  @main (pinned Gemma 4 26B-A4B and Qwen3.6 35B-A3B GGUFs). transformers.js also runs feature-extraction, speech recognition, object
+  detection, image classification, depth, segmentation, token and text classification on fp32 graphs. kohra and diskformer load from their main branches;
+  `<backend>.source` pins another copy.
+- transformers.js also runs Gemma 4 and Qwen3.5 ONNX as chat with image (and, for Gemma 4, audio) blocks, through
+  their conditional-generation classes at q4f16, as LocalMind does.
+- localmind @main: LocalMind's hand-written engines, matched by architecture and format (lfm2 GGUF, Gemma 4
+  QAT-mobile, qwen35 Prism ternary); resident on the GPU, so budget ≥ file size.
+- wllama 3.4.1 (LocalMind's pin): any GGUF ≤ ~3.6 GB no architecture engine covers (a repo's split set as is; a
+  single file over 2 GB split once into OPFS shards, src/gguf-split.js); WebGPU when the budget
+  holds it, CPU otherwise. Thinking off (chat_template_kwargs). Ranks last.
+- Pins: src/sources.js holds exact versions and commits; `npm run pin` moves the git pins to each repo's main.
+- Agent face: 9 tools (models, probe, plan, load, generate, sessions, unload, stored, remove). Person-only acts: none.
+- `MODELS`: tested entries by family; `load(entry)`. `stored()` / `remove(repo)` over the transformers.js cache and
+  diskformer's OPFS stores.
+- Gaps: no wasm fallback (every back end needs WebGPU); one store per back end, not the shared OPFS store yet (kohra's graph is in the HTTP cache, unlisted);
+  llama.cpp-webgpu not started; speed estimates exist for diskformer only.

package/index.js

@@ -0,0 +1,16 @@
+// anyformer — one call to run any model in a browser tab. Plain ESM, no build step.
+export { load, plan, BACKENDS, DEFAULT_BUDGET } from './src/load.js';
+export { AnyformerError, isAnyformerError, ERROR_CODES } from './src/errors.js';
+export { probe, memory64 } from './src/probe.js';
+export { describe, fromHubInfo } from './src/describe.js';
+export { resolve, REFUSALS } from './src/resolve.js';
+export { transformersBackend } from './src/backends/transformers.js';
+export { kohraBackend } from './src/backends/kohra.js';
+export { diskformerBackend, DISKFORMER_MODELS } from './src/backends/diskformer.js';
+export { TOOLS, call, registerTools, journal, toolOptions } from './src/tools.js';
+export { MODELS } from './src/models.js';
+export { stored, remove } from './src/store.js';
+export { pipeline } from './src/pipeline.js';
+export { localmindBackend } from './src/backends/localmind.js';
+export { wllamaBackend } from './src/backends/wllama.js';
+export { PINS, SOURCES } from './src/sources.js';

package/package.json

@@ -1,6 +1,48 @@
 {
   "name": "anyformer",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.0",
+  "description": "One call to run a model in a browser tab: anyformer reads the Hub repo, probes the browser, and picks the back end (transformers.js, LocalMind's WebGPU kernels, diskformer, kohra) that fits the model's format, architecture and your GPU budget.",
+  "type": "module",
+  "main": "./index.js",
+  "exports": {
+    ".": "./index.js"
+  },
+  "files": [
+    "index.js",
+    "src/",
+    "README.md",
+    "LICENSE",
+    "CHANGELOG.md",
+    "SPEC.md"
+  ],
+  "sideEffects": false,
+  "scripts": {
+    "test": "node --test 'test/*.test.js'",
+    "test:browser": "node test/browser/run.mjs",
+    "pin": "node scripts/pin-sources.mjs"
+  },
+  "keywords": [
+    "agent",
+    "browser",
+    "diffusion",
+    "diskformer",
+    "gguf",
+    "inference",
+    "llm",
+    "localmind",
+    "onnx",
+    "transformers",
+    "webgpu",
+    "webmcp"
+  ],
+  "license": "MIT",
+  "author": "Chirag Patnaik",
+  "homepage": "https://github.com/NakliTechie/anyformer.js#readme",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/NakliTechie/anyformer.js.git"
+  },
+  "bugs": {
+    "url": "https://github.com/NakliTechie/anyformer.js/issues"
+  }
+}

package/src/backends/diskformer.js

@@ -0,0 +1,80 @@
+// diskformer: MoE models larger than GPU memory. Weights stay in OPFS; the GPU holds the dense layers, the KV cache
+// and an expert cache sized by the budget. Engines load from the CDN at load() time, at the commit src/sources.js pins;
+// set diskformerBackend.source to load another copy.
+
+import { AnyformerError } from '../errors.js';
+import { SOURCES } from '../sources.js';
+
+// The GGUFs diskformer's engines are pinned to (GEMMA4_26B_A4B, QWEN36_35B_A3B in engines/*.js), with the smallest budget each
+// has passed its gate at and the measured decode speeds (diskformer README, M4 Pro, 2026-10-05).
+export const DISKFORMER_MODELS = [
+  {
+    repo: 'google/gemma-4-26B-A4B-it-qat-q4_0-gguf', file: 'gemma-4-26B_q4_0-it.gguf', size: 14439363584,
+    engine: 'gemma4_moe_ssd.js', Engine: 'Gemma4MoeSsd', minBudget: 2.5e9,
+    speeds: [[2.5e9, 12.4], [4.3e9, 14.2], [7e9, 17.9]],
+  },
+  {
+    repo: 'unsloth/Qwen3.6-35B-A3B-GGUF', file: 'Qwen3.6-35B-A3B-Q8_0.gguf', size: 36903140320,
+    engine: 'qwen35_moe_ssd.js', Engine: 'Qwen35MoeSsd', minBudget: 4e9,
+    speeds: [[4e9, 9.2]],
+  },
+];
+
+// diskformer's engines refuse a budget with a plain Error ("GPU budget 2.50 GB is too small: … need 3.04 GB, and the
+// expert cache at least 0.05 GB more"). Until they carry a code, read the two numbers to give BUDGET_TOO_SMALL a need.
+function budgetError(e) {
+  const m = /GPU budget [\d.]+ GB is too small\b.*?need ([\d.]+) GB.*?at least ([\d.]+) GB more/s.exec(e?.message || '');
+  return m && new AnyformerError('BUDGET_TOO_SMALL', `diskformer: ${e.message}`, { cause: e, need: Math.ceil((+m[1] + +m[2]) * 1e9) });
+}
+
+const match = (model) => DISKFORMER_MODELS.find((m) => m.repo === model.repo &&
+  model.files.gguf.some((f) => f.name === m.file && f.size === m.size));
+
+// The speed measured at the largest budget not above this one.
+const speedAt = (m, budget) => m.speeds.filter(([b]) => b <= budget).at(-1)?.[1] ?? null;
+
+export { budgetError };
+
+export const diskformerBackend = {
+  id: 'diskformer',
+  // Gemma 4's repo is tagged image-text-to-text; the engines run its text turns.
+  tasks: ['text-generation', 'image-text-to-text'],
+  source: SOURCES.diskformer,
+
+  canRun(model, caps, { budget, task }) {
+    if (!model.formats.includes('gguf')) return { ok: false, code: 'format', reason: 'no GGUF in the repo' };
+    if (task && !diskformerBackend.tasks.includes(task)) return { ok: false, code: 'task', reason: `task ${task}; diskformer engines run text generation only` };
+    const m = match(model);
+    if (!m) return { ok: false, code: 'model', reason: `no diskformer engine for this GGUF; engines cover ${DISKFORMER_MODELS.map((x) => x.file).join(', ')}` };
+    if (!caps.webgpu) return { ok: false, code: 'webgpu', reason: 'no WebGPU adapter' };
+    if (!caps.f16) return { ok: false, code: 'f16', reason: 'the GPU lacks shader-f16' };
+    if (!caps.opfs) return { ok: false, code: 'opfs', reason: 'no OPFS (navigator.storage.getDirectory)' };
+    if (budget < m.minBudget) return { ok: false, code: 'budget', need: m.minBudget, reason: `a ${(budget / 1e9).toFixed(1)} GB budget is under the ${m.minBudget / 1e9} GB this model has passed its gate at` };
+    // Storage quota is not checked: Chrome raises it as the download writes, while the disk has room.
+    return { ok: true, estimate: { gpuBytes: budget, diskBytes: m.size, tokPerSec: speedAt(m, budget) } };
+  },
+
+  // maxCtx sizes the KV cache, which stays on the GPU: the 2.5 GB minimum and the speeds above hold at 1024 tokens,
+  // the context diskformer's chat gate runs at. A longer context needs a larger budget.
+  async load(model, { budget, maxCtx = 1024, onProgress, signal }) {
+    const m = match(model);
+    const mod = await import(`${diskformerBackend.source}/${m.engine}`);
+    let engine;
+    try {
+      engine = await mod[m.Engine].load(null, {
+        // The engines' own default root, named so the store module (src/store.js) lists the same folder.
+        gpuBudgetBytes: budget, maxCtx, signal, root: 'localmind-ssd',
+        onProgress: onProgress && ((e) => onProgress({ backend: 'diskformer', ...e })),
+      });
+    } catch (e) { throw budgetError(e) || e; }
+    return {
+      task: 'text-generation',
+      engine,
+      async *generate(messages, { maxNewTokens = 512, signal } = {}) {
+        for await (const r of engine.generate(messages, { maxNewTokens, signal, enableThinking: false }))
+          yield { text: r.text, tokens: r.tokens };
+      },
+      dispose: () => engine.dispose(),
+    };
+  },
+};

package/src/backends/kohra.js

@@ -0,0 +1,59 @@
+// kohra: text-diffusion LMs (MDLM, BD3LM). kohra.js runs the denoising loop over onnxruntime-web; it loads from the
+// CDN at load() time, at the commit src/sources.js pins; set kohraBackend.source to load another copy.
+
+import { graphBytes } from './transformers.js';
+import { SOURCES } from '../sources.js';
+
+const FP16 = 'onnx/model_fp16_fused.onnx';
+// The q4 graph decodes on WebGPU only with RTN packing and this ORT-web dev build (kohra README, "Known constraints").
+const Q4 = 'onnx/model_q4f16_rtn_sym.onnx';
+const Q4_ORT = '1.26.0-dev.20260416-b7804b056c';
+
+const pickGraph = (model, caps) => {
+  const have = new Set(model.files.onnx.map((f) => f.name));
+  return caps.f16 && have.has(FP16) ? FP16 : have.has(Q4) ? Q4 : null;
+};
+
+// kohra's own template covers one user turn; this is the same ChatML over the whole conversation.
+const chatml = (messages) =>
+  messages.map((m) => `<|im_start|>${m.role}\n${m.content}<|im_end|>\n`).join('') + '<|im_start|>assistant\n';
+
+export const kohraBackend = {
+  id: 'kohra',
+  tasks: ['text-generation'],
+  source: SOURCES.kohra,
+
+  canRun(model, caps, { task } = {}) {
+    if (!model.tags?.includes('text-diffusion')) return { ok: false, code: 'model', reason: 'not a text-diffusion model' };
+    if (task && task !== 'text-generation') return { ok: false, code: 'task', reason: `task ${task}; kohra runs text-generation only` };
+    if (!caps.webgpu) return { ok: false, code: 'webgpu', reason: 'no WebGPU adapter' };
+    const graph = pickGraph(model, caps);
+    if (!graph) return { ok: false, code: 'format', reason: `no kohra graph (${FP16} or ${Q4}) in the repo` };
+    const bytes = graphBytes(model, graph);
+    return { ok: true, estimate: { gpuBytes: bytes, diskBytes: bytes, tokPerSec: null } };
+  },
+
+  async load(model, { caps, onProgress, signal }) {
+    const graph = pickGraph(model, caps);
+    const { DiffusionLM } = await import(kohraBackend.source);
+    onProgress?.({ backend: 'kohra', status: 'loading', file: graph });
+    const base = `https://huggingface.co/${model.repo}/resolve/${model.revision || 'main'}`;
+    const lm = await DiffusionLM.from_pretrained({
+      model: `${base}/${graph}`, tokenizer: model.repo, ...(graph === Q4 && { ortVersion: Q4_ORT }),
+    });
+    signal?.throwIfAborted();
+    const blockCausal = model.tags.includes('block-diffusion');
+    return {
+      task: 'text-generation',
+      // Diffusion fills the whole block at once, so the reply arrives as one chunk.
+      async *generate(messages, { maxNewTokens = 128, steps, signal } = {}) {
+        signal?.throwIfAborted();
+        const out = await lm.generate(chatml(messages), {
+          maxNewTokens, steps: steps ?? maxNewTokens, blockCausal, chatTemplate: false, stripThink: true,
+        });
+        yield { text: out.text, tokens: out.tokens, tokPerSec: out.tokensPerSecond };
+      },
+      dispose: () => lm.session?.release?.(),
+    };
+  },
+};

package/src/backends/localmind.js

@@ -0,0 +1,115 @@
+// LocalMind's hand-written WebGPU engines: whole model resident on the GPU, kernels written per architecture. Each
+// engine reads its config from the model file, so it runs any checkpoint of its architecture in a format it reads,
+// not only the one LocalMind ships. They load from the CDN at load() time, at the commit src/sources.js pins; set
+// localmindBackend.source to load another copy.
+//
+//   lfm2     src/lfm2_5.js           GGUF, general.architecture 'lfm2' (dense LFM2 / LFM2.5), Q4_0 Q8_0 Q4_K Q5_K Q6_K
+//   gemma4   src/gemma-4-e2b.js      Google's QAT-mobile safetensors export of Gemma 4 (E2B, E4B): config.json-driven
+//   qwen35   src/ternary_bonsai_2_27b.js  GGUF, 'qwen35' with Prism's ternary types (PTQ1_0, PQ2_0) only: a standard
+//            K-quant (unsloth/Qwen3.5-4B-GGUF Q4_K_M) fails with "This feature is not included in this bundle".
+// Speeds: LocalMind's quiet-machine engine bench (plan/bench-engines-quiet-2026-09-24.md, M4 Pro, 256 tokens, greedy).
+
+import { SOURCES } from '../sources.js';
+
+const ENGINES = {
+  lfm2: { file: 'lfm2_5.js', Engine: 'Lfm2Mobile' },
+  gemma4: { file: 'gemma-4-e2b.js', Engine: 'Gemma4Mobile' },
+  qwen35: { file: 'ternary_bonsai_2_27b.js', Engine: 'TernaryBonsai2' },
+};
+
+// Quant preference per engine, by the GGUF file-name suffix. Plain Q4_0 first: it is LocalMind's default.
+const LFM2_QUANTS = ['Q4_0', 'Q4_K_M', 'Q4_K_S', 'Q5_K_M', 'Q5_K_S', 'Q6_K', 'Q8_0'];
+const QWEN35_QUANTS = ['PTQ1_0', 'PQ2_0'];
+const MEASURED = {
+  'LiquidAI/LFM2.5-230M-GGUF': 1062.5,
+  'google/gemma-4-E2B-it-qat-mobile-transformers': 171.3,
+  'prism-ml/Ternary-Bonsai-2-27B-gguf': 27.2,
+};
+
+// The GGUF to load: the first quant in preference order. Projector (mmproj) files are vision towers, never the LM.
+function pickGguf(model, quants) {
+  const lms = model.files.gguf.filter((f) => !/mmproj/i.test(f.name));
+  for (const q of quants) {
+    const f = lms.find((x) => new RegExp(`[-_.]${q}\\.gguf$`, 'i').test(x.name) && !/QAD/i.test(x.name));
+    if (f) return f;
+  }
+  return null;
+}
+
+// Which engine reads this repo, and the file and bytes it would load; null when none does.
+function match(model) {
+  if (model.formats.includes('gguf') && model.architecture === 'lfm2') {
+    const f = pickGguf(model, LFM2_QUANTS);
+    return f && { engine: 'lfm2', file: f.name, bytes: f.size };
+  }
+  if (model.formats.includes('gguf') && model.architecture === 'qwen35') {
+    const f = pickGguf(model, QWEN35_QUANTS);
+    return f && { engine: 'qwen35', file: f.name, bytes: f.size };
+  }
+  // The QAT-mobile export's tensor names (weight_scale, embedding_quantized …) are what the engine reads; a plain
+  // Gemma 4 safetensors checkpoint has model_type gemma4 too but will not load, so the repo name decides.
+  if (model.architecture === 'gemma4' && /-qat-mobile-transformers$/.test(model.repo)) {
+    const f = model.files.safetensors.find((x) => x.name === 'model.safetensors');
+    return f && { engine: 'gemma4', file: f.name, bytes: f.size };
+  }
+  return null;
+}
+
+export const localmindBackend = {
+  id: 'localmind',
+  tasks: ['text-generation'],
+  source: SOURCES.localmind,
+
+  canRun(model, caps, { budget, task }) {
+    const m = match(model);
+    if (!m) return { ok: false, code: 'model', reason: 'no LocalMind engine reads this repo (lfm2 GGUF, qwen35 Prism GGUF, Gemma 4 QAT-mobile)' };
+    if (task && task !== 'text-generation') return { ok: false, code: 'task', reason: `task ${task}; LocalMind's engines run text generation` };
+    if (!caps.webgpu) return { ok: false, code: 'webgpu', reason: 'no WebGPU adapter' };
+    if (m.engine === 'qwen35' && !(caps.f16 && caps.subgroups)) return { ok: false, code: 'f16', reason: 'the ternary kernels need shader-f16 and subgroups' };
+    // The whole model stays on the GPU, so the budget must hold the file.
+    if (budget < m.bytes) return { ok: false, code: 'budget', need: m.bytes, reason: `${m.file} stays on the GPU whole (${(m.bytes / 1e9).toFixed(2)} GB) and the budget is ${(budget / 1e9).toFixed(2)} GB` };
+    return { ok: true, estimate: { gpuBytes: m.bytes, diskBytes: m.bytes, tokPerSec: MEASURED[model.repo] ?? null } };
+  },
+
+  async load(model, { maxCtx, onProgress, signal }) {
+    const m = match(model);
+    const e = ENGINES[m.engine];
+    const mod = await import(`${localmindBackend.source}/${e.file}`);
+    const opts = {
+      revision: model.revision || 'main', signal,
+      onProgress: onProgress && ((p) => onProgress({ backend: 'localmind', engine: m.engine, ...p })),
+      ...(m.engine !== 'gemma4' && { file: m.file }),
+      ...(maxCtx && m.engine !== 'gemma4' && { maxLength: maxCtx }),
+    };
+    const engine = await mod[e.Engine].load(model.repo, opts);
+    signal?.throwIfAborted();
+    // Thinking off, as the diskformer back end does: a chat reply, not the model's reasoning. (The qwen35 engine
+    // takes template arguments as a field; LocalMind sets the same one.)
+    if ('chatTemplateArgs' in engine) engine.chatTemplateArgs = { ...engine.chatTemplateArgs, enable_thinking: false };
+    // Compile the kernels now, as LocalMind's workers do, so the first reply is not paying for it.
+    onProgress?.({ backend: 'localmind', engine: m.engine, status: 'warmup' });
+    await engine.warmup?.();
+    // Gemma's template has no system turn; LocalMind folds the system prompt into the first user turn the same way.
+    const prepare = m.engine !== 'gemma4' ? (msgs) => msgs : (msgs) => {
+      const sys = msgs.filter((x) => x.role === 'system').map((x) => x.content).join('\n\n');
+      const rest = msgs.filter((x) => x.role !== 'system');
+      const i = rest.findIndex((x) => x.role === 'user');
+      return sys && i >= 0 ? rest.map((x, k) => (k === i ? { ...x, content: `${sys}\n\n${x.content}` } : x)) : rest;
+    };
+    let tokens = 0;
+    return {
+      task: 'text-generation',
+      engine,
+      async *generate(messages, { maxNewTokens = 512, signal } = {}) {
+        tokens = 0;
+        for await (const c of engine.generate(prepare(messages), { maxNewTokens, signal })) {
+          if (c.token != null) tokens++;
+          yield { text: c.text, tokens };
+        }
+      },
+      dispose: () => engine.dispose?.(),
+    };
+  },
+};
+
+export { match as localmindMatch };