decidekit 0.1.0 → 0.1.1

package/CHANGELOG.md

@@ -1,5 +1,26 @@
 # Changelog
 
+## 0.1.1 (2026-11-02)
+
+### Models
+
+- The default model id is `debuffjp/decide-small` (Decide Small). Before 0.1.1 it was `Xenova/bge-small-en-v1.5`. The
+  weights are identical, so decisions don't change. For other languages the id is `debuffjp/decide-multilingual`
+  (Decide Multilingual), identical to `Xenova/multilingual-e5-small`.
+- The old ids and heads recording them keep working. They resolve to the new ids, so `d.model` returns the new id. A
+  head trained on 0.1.1 records the new id, so a 0.1.0 decider needs that id passed as `model` to use it.
+- Downloads of both models are pinned to a revision.
+- New option `modelHost` serves model files from `<modelHost>/<model>/` instead of Hugging Face. It applies to the
+  whole page or process.
+
+### Docs
+
+- Label names count as 1 example in every doc, and the advice to turn them off with bge-small is gone: it applied to the
+  old weight of 3.
+- Calibration, out-of-scope and zero-shot numbers are now bge-small's or labelled with the model they were measured on.
+- `decide()` documents that a described label scores by the better of its name and description.
+- The API reference covers `decidekit/typesafe`.
+
 ## 0.1.0 (2026-11-02)
 
 The first release. Numbers are on public test splits unless noted; the scripts that produced them are in `eval/`.

package/README.md

@@ -2,7 +2,7 @@
 
 [![npm](https://img.shields.io/npm/v/decidekit?color=cffb2d&labelColor=11140d)](https://www.npmjs.com/package/decidekit) [![ci](https://github.com/debuffjp/decidekit/actions/workflows/ci.yml/badge.svg)](https://github.com/debuffjp/decidekit/actions/workflows/ci.yml) [![MIT](https://img.shields.io/badge/licence-MIT-cffb2d?labelColor=11140d)](LICENSE) [![Discord](https://img.shields.io/badge/discord-debuff-a78bfa?labelColor=11140d)](https://discord.gg/jshbDHSpCY)
 
-**[Try it in your browser](https://debuff.jp/decidekit/playground/)** · [Watch it learn from an LLM](https://debuff.jp/decidekit/lanes/) · [Launch post](https://debuff.jp/blog/introducing-decidekit) · [Research: when the LLM is wrong](https://debuff.jp/research/learning-from-a-noisy-llm)
+**[Try it in your browser](https://debuff.jp/decide/playground/)** · [Watch it learn from an LLM](https://debuff.jp/decide/lanes/) · [Launch post](https://debuff.jp/blog/introducing-decide) · [Research: when the LLM is wrong](https://debuff.jp/research/learning-from-a-noisy-llm)
 
 A decision model for the browser and Node: it picks one of your labels for a piece of text, with confidence you can trust, and learns from an LLM when it isn't sure. Your labels are the type. No server, no API key.
 
@@ -29,8 +29,8 @@ await d.decide(text, ["question", "bug_report", "praise"]);
 await d.decide(text, { refund: "wants their money back", bug: "something is broken" });
 ```
 
-Zero-shot is for trying things out: 61% on BANKING77 and 70% on CLINC150 (dev splits), against 90-94% trained.
-Its confidence came out close to calibrated on those datasets (error 0.005-0.045 with 2 to 150 options), but check
+Zero-shot is for trying things out: 66% on BANKING77 and 73% on CLINC150 (dev splits), against 90-94% trained.
+Its confidence came out close to calibrated on those datasets (error 0.03-0.04), but check
 it on your own labels before relying on a threshold.
 
 ## Train a head
@@ -146,20 +146,24 @@ That lets wasm use several threads: in our runs with all-MiniLM-L6-v2 it cut p95
 
 Download on first visit: the model (34 MB) and onnxruntime's wasm (23.6 MB, 5.8 MB gzipped), both cached after.
 
-## Models
-
-decidekit runs the Decide models, each a sentence encoder plus a calibrated head:
-
-| Model | Encoder | Download | Reads |
-|---|---|---|---|
-| Decide Small (default) | `Xenova/bge-small-en-v1.5` | 34 MB | English |
-| Decide Multilingual | `Xenova/multilingual-e5-small` | 118 MB | 76% on Japanese and Chinese intents |
-
-Pass the encoder id as `model` to `createDecider()`.
-
+## Models
+
+decidekit runs the Decide models, each a sentence encoder plus a calibrated head:
+
+| Model | Id | Download | Reads | Upstream |
+|---|---|---|---|---|
+| Decide Small (default) | [`debuffjp/decide-small`](https://huggingface.co/debuffjp/decide-small) | 34 MB | English | BAAI's [bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5), identical to `Xenova/bge-small-en-v1.5` |
+| Decide Multilingual | [`debuffjp/decide-multilingual`](https://huggingface.co/debuffjp/decide-multilingual) | about 118 MB | 76% on Japanese and Chinese intents | intfloat's [multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small), identical to `Xenova/multilingual-e5-small` |
+
+Pass the id as `model` to `createDecider()`. The files download from Hugging Face on first use, pinned to a revision, and
+are then cached. To serve them yourself, pass `modelHost`: see [self-hosting](docs/guide-browser.md#host-the-model-files-yourself).
+
+Before 0.1.1 the default id was `Xenova/bge-small-en-v1.5`. The old ids and heads recording them keep working: they
+resolve to the new ids, so `d.model` returns the new id.
+
 ## Numbers
 
-bge-small-en-v1.5 (the default) at q8, Ryzen 7 260 laptop, Node 24 and headless Chrome 156, 2026-11-01. Accuracy
+Decide Small (bge-small-en-v1.5, the default) at q8, without label names, Ryzen 7 260 laptop, Node 24 and headless Chrome 156, 2026-11-01. Accuracy
 and calibration error (ECE, lower is better) on each dataset's test split, mean of 3 seeds (one run for "all"):
 [eval/results-bge-small.json](eval/results-bge-small.json).
 
@@ -196,16 +200,16 @@ in-scope examples only, with `minConfidence` 0.93 picked on the validation split
 
 **The smaller model.** `createDecider({ model: "Xenova/all-MiniLM-L6-v2" })` (23.7 MB, the pre-release default) is
 about 1.8x faster per decision in Chrome and 3 to 5 points less accurate with few examples, about the same with all
-the data. Pass it to keep using heads trained on it. Before this default changed, the README suggested
-`{ labelNames: false }` with bge-small: label names slightly hurt it on the dev splits.
+the data. Pass it to keep using heads trained on it. Before 0.1.0 a label name counted as 3 examples and the README
+suggested `{ labelNames: false }` with bge-small. It now counts as 1, which helps.
 
 Reproduce: `npm run eval:suite` (Node), `npm run bench` then `npm run bench:run` (Chrome).
 
 ## Playground
 
-Try it at [debuff.jp/decidekit](https://debuff.jp/decidekit/). To run it locally: `npm run playground`, then serve `playground/dist` cross-origin isolated (`node bench/serve.mjs` style headers).
+Try it at [debuff.jp/decide](https://debuff.jp/decide/). To run it locally: `npm run playground`, then serve `playground/dist` cross-origin isolated (`node bench/serve.mjs` style headers).
 Try zero-shot labels, the ready-made heads, a head trained in the page and "none of these".
-[Two lanes](https://debuff.jp/decidekit/lanes/) shows `withFallback` live: decidekit starts from twelve label names,
+[Two lanes](https://debuff.jp/decide/lanes/) shows `withFallback` live: decidekit starts from twelve label names,
 asks a small LLM when it's unsure, learns from each answer, and replays the first 20 messages at the end.
 
 ## Ready-made heads
@@ -231,14 +235,9 @@ and offline apps, and text that can't leave the device. Details, code and poor f
 | MediaPipe text classifier | yes | no, offline | no | no |
 | LLM structured output (AI SDK `Output.choice`) | no, API | n/a | yes | no |
 
-jimothy reports 92.4% on BANKING77, above our 90.7%. Pick it if you train in Node and want the higher number. Pick
-decidekit if training has to happen in the page, from the user's own data.
-
-[system-one](https://github.com/lukeramsden/system-one) is a different kind of tool: a typed client for hosted System One models (Jev and Laya). It runs no model itself.
-
 ## Docs
 
-[docs/README.md](docs/README.md) indexes the guides (training, confidence and thresholds, the browser), the
+[debuff.jp/decide/docs](https://debuff.jp/decide/docs/) (also in [docs/README.md](docs/README.md)) indexes the guides (training, confidence and thresholds, the browser), the
 API reference, how it works and the FAQ.
 
 ## Community
@@ -249,6 +248,6 @@ issue template for them. See [CONTRIBUTING.md](CONTRIBUTING.md).
 
 ## Status
 
-0.1. English with the default model, other languages with `multilingual-e5-small` ([FAQ](docs/faq.md#which-languages-does-it-support)); short texts (about a paragraph). Not done yet: WebGPU in the worker and fine-tuning the encoder (the likely route past about 91%).
+0.1. English with the default model, other languages with Decide Multilingual ([FAQ](docs/faq.md#which-languages-does-it-support)); short texts (about a paragraph). Not done yet: WebGPU in the worker and fine-tuning the encoder (the likely route past about 91%).
 
 Made by the [debuff](https://debuff.jp) team. MIT.

package/dist/index.d.ts

@@ -72,13 +72,19 @@ export interface TrainOptions<L extends string = string> {
 }
 export interface DeciderOptions {
     /**
-     * A Hugging Face sentence-embedding model with ONNX weights. Default `Xenova/bge-small-en-v1.5` (34 MB at q8).
+     * A Hugging Face sentence-embedding model with ONNX weights. Default `debuffjp/decide-small` (Decide Small, 34 MB at q8).
+     * Before 0.1.1 the default id was `Xenova/bge-small-en-v1.5`, the same weights; that id and heads recording it still work.
      * Pre-release builds defaulted to `Xenova/all-MiniLM-L6-v2` (23 MB): smaller and 1.6x faster, 3 to 5 points less
      * accurate. Pass it to keep using heads trained on it.
      */
     model?: string;
     /** Defaults to transformers.js' choice: wasm in browsers, cpu in Node. */
     device?: "wasm" | "webgpu" | "cpu";
+    /**
+     * Serve model files from your own host instead of Hugging Face, as `<modelHost>/<model>/`, for example
+     * `https://cdn.example.com/models/debuffjp/decide-small/onnx/model_quantized.onnx`. Applies to the whole page or process.
+     */
+    modelHost?: string;
     /** Weight precision. Default `q8`. */
     dtype?: "q8" | "fp16" | "fp32";
     /** Called as model files download (first load only; later loads come from cache). */
@@ -98,7 +104,7 @@ export interface DeciderOptions {
  * `train()`. In a browser, prefer `createWorkerDecider()` from `decidekit/browser`: the encoder blocks the thread
  * it runs on.
  */
-export declare function createDecider({ model, device, dtype, temperature, onProgress }?: DeciderOptions): Promise<{
+export declare function createDecider({ model: requested, device, dtype, temperature, onProgress, modelHost }?: DeciderOptions): Promise<{
     model: string;
     decide: {
         <const L extends string>(text: string, using: Choices<L>, options?: {

package/dist/index.js

@@ -1,7 +1,7 @@
 // decide(text, options): typed, calibrated decisions on device. Zero-shot compares the text with each option's
 // embedding; a trained head (a softmax layer over the same embeddings, 156 KB for 77 labels) is far more
 // accurate. Both return the option literals as the choice type.
-import { pipeline } from "@huggingface/transformers";
+import { env, pipeline } from "@huggingface/transformers";
 import { fitHead, logits, softmax } from "./head.js";
 function isHead(c) {
     if (Array.isArray(c) || !("weights" in c))
@@ -23,9 +23,15 @@ const LABEL_NAME_WEIGHT = 1;
 // multilingual-e5-small's similarities sit close together, so it needs far lower temperatures: on MASSIVE in English
 // and Japanese, 0.0105 for label names (calibration error 0.03 to 0.04; descriptions not yet measured) and a floor of
 // 0.4 (eval/e5-settings.ts). It also expects "query: " before every text.
-const SMALL_HEAD_FLOOR = { "Xenova/bge-small-en-v1.5": 0.5, "Xenova/multilingual-e5-small": 0.4 };
-const TEMPERATURES = { "Xenova/bge-small-en-v1.5": [0.03, 0.025], "Xenova/all-MiniLM-L6-v2": [0.05, 0.05], "Xenova/multilingual-e5-small": [0.0105, 0.0105] };
-const PREFIX = { "Xenova/multilingual-e5-small": "query: " };
+// The Decide models are unmodified copies of these upstream ids, so the old ids and heads trained on them still work.
+const SMALL = "debuffjp/decide-small", MULTILINGUAL = "debuffjp/decide-multilingual";
+const SAME_MODEL = { "Xenova/bge-small-en-v1.5": SMALL, "Xenova/multilingual-e5-small": MULTILINGUAL };
+const canonical = (model) => SAME_MODEL[model] ?? model;
+// Pinned so a later change on Hugging Face can't change what an installed version decides.
+const REVISIONS = { [SMALL]: "188d690f86190863e3b4e44e29c88348b10230c4", [MULTILINGUAL]: "a3e353d26d846b2163db8acad65ef3d43893592e" };
+const SMALL_HEAD_FLOOR = { [SMALL]: 0.5, [MULTILINGUAL]: 0.4 };
+const TEMPERATURES = { [SMALL]: [0.03, 0.025], "Xenova/all-MiniLM-L6-v2": [0.05, 0.05], [MULTILINGUAL]: [0.0105, 0.0105] };
+const PREFIX = { [MULTILINGUAL]: "query: " };
 function toDecision(labels, p) {
     let best = 0;
     for (let i = 1; i < p.length; i++)
@@ -39,9 +45,14 @@ function toDecision(labels, p) {
  * `train()`. In a browser, prefer `createWorkerDecider()` from `decidekit/browser`: the encoder blocks the thread
  * it runs on.
  */
-export async function createDecider({ model = "Xenova/bge-small-en-v1.5", device, dtype = "q8", temperature, onProgress } = {}) {
+export async function createDecider({ model: requested = SMALL, device, dtype = "q8", temperature, onProgress, modelHost } = {}) {
+    const model = canonical(requested);
+    if (modelHost) {
+        env.remoteHost = modelHost.endsWith("/") ? modelHost : modelHost + "/";
+        env.remotePathTemplate = "{model}/";
+    }
     const embed = await pipeline("feature-extraction", model, {
-        device, dtype,
+        device, dtype, revision: REVISIONS[model],
         progress_callback: onProgress && ((p) => p.status === "progress" && onProgress({ file: p.file, loaded: p.loaded, total: p.total })),
     });
     const optionCache = new Map(), prefix = PREFIX[model] ?? "";
@@ -58,7 +69,7 @@ export async function createDecider({ model = "Xenova/bge-small-en-v1.5", device
     async function score(q, using, minConfidence) {
         let d;
         if (isHead(using)) {
-            if (using.model !== model)
+            if (canonical(using.model) !== model)
                 throw new Error(`decidekit: this head was trained on ${using.model}, but the decider uses ${model}. Pass { model: "${using.model}" } to createDecider.`);
             d = toDecision(using.labels, softmax(logits(using, q), using.temperature));
         }

package/package.json

@@ -1,93 +1,93 @@
-{
-  "name": "decidekit",
-  "version": "0.1.0",
-  "description": "A System One model in the browser: typed, calibrated text decisions on the user's device. Learns from an LLM when it isn't sure.",
-  "license": "MIT",
-  "repository": "github:debuffjp/decidekit",
-  "keywords": [
-    "decision model",
-    "system one",
-    "classification",
-    "intent",
-    "zero-shot",
-    "calibration",
-    "llm",
-    "embeddings",
-    "browser",
-    "typescript",
-    "text classification",
-    "transformers.js",
-    "on-device",
-    "llm fallback"
-  ],
-  "type": "module",
-  "exports": {
-    ".": {
-      "types": "./dist/index.d.ts",
-      "default": "./dist/index.js"
-    },
-    "./browser": {
-      "types": "./dist/browser.d.ts",
-      "default": "./dist/browser.js"
-    },
-    "./react": {
-      "types": "./dist/react.d.ts",
-      "default": "./dist/react.js"
-    },
-    "./switch": {
-      "types": "./dist/switch.d.ts",
-      "default": "./dist/switch.js"
-    },
-    "./typesafe": {
-      "types": "./dist/typesafe.d.ts",
-      "default": "./dist/typesafe.js"
-    }
-  },
-  "files": [
-    "dist",
-    "CHANGELOG.md"
-  ],
-  "scripts": {
-    "build": "tsc -p tsconfig.build.json",
-    "prepack": "npm run build",
-    "typecheck": "tsc --noEmit",
-    "eval": "node --experimental-strip-types eval/banking77.ts",
-    "bench": "esbuild src/worker.ts src/browser.ts --bundle --format=esm --platform=browser --outdir=bench --alias:onnxruntime-web/webgpu=onnxruntime-web/wasm && node bench/serve.mjs",
-    "playground": "node playground/build.mjs",
-    "heads": "node --experimental-strip-types heads/build.ts",
-    "bench:run": "node bench/run.mjs",
-    "eval:suite": "node --experimental-strip-types eval/suite.ts",
-    "eval:hill": "node --experimental-strip-types eval/hill.ts",
-    "test": "node --test --experimental-strip-types \"test/*.test.ts\"",
-    "lint:prose": "node scripts/lint-prose.mjs",
-    "smoke": "node scripts/smoke.mjs"
-  },
-  "dependencies": {
-    "@huggingface/transformers": "4.2.0"
-  },
-  "devDependencies": {
-    "@types/node": "22.15.3",
-    "@types/react": "19.3.0",
-    "esbuild": "0.25.10",
-    "playwright-core": "1.64.1",
-    "react": "19.3.1",
-    "spryte": "0.11.0",
-    "typescript": "5.9.3"
-  },
-  "engines": {
-    "node": ">=22.18"
-  },
-  "homepage": "https://debuff.jp/decidekit/",
-  "bugs": "https://github.com/debuffjp/decidekit/issues",
-  "bin": {
-    "decidekit": "./dist/cli.js"
-  },
-  "peerDependencies": {
-    "react": ">=18"
-  },
-  "peerDependenciesMeta": {
-    "react": {
-      "optional": true
-    }
-  }
-}
+{
+  "name": "decidekit",
+  "version": "0.1.1",
+  "description": "A System One model in the browser: typed, calibrated text decisions on the user's device. Learns from an LLM when it isn't sure.",
+  "license": "MIT",
+  "repository": "github:debuffjp/decidekit",
+  "keywords": [
+    "decision model",
+    "system one",
+    "classification",
+    "intent",
+    "zero-shot",
+    "calibration",
+    "llm",
+    "embeddings",
+    "browser",
+    "typescript",
+    "text classification",
+    "transformers.js",
+    "on-device",
+    "llm fallback"
+  ],
+  "type": "module",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "default": "./dist/index.js"
+    },
+    "./browser": {
+      "types": "./dist/browser.d.ts",
+      "default": "./dist/browser.js"
+    },
+    "./react": {
+      "types": "./dist/react.d.ts",
+      "default": "./dist/react.js"
+    },
+    "./switch": {
+      "types": "./dist/switch.d.ts",
+      "default": "./dist/switch.js"
+    },
+    "./typesafe": {
+      "types": "./dist/typesafe.d.ts",
+      "default": "./dist/typesafe.js"
+    }
+  },
+  "files": [
+    "dist",
+    "CHANGELOG.md"
+  ],
+  "scripts": {
+    "build": "tsc -p tsconfig.build.json",
+    "prepack": "npm run build",
+    "typecheck": "tsc --noEmit",
+    "eval": "node --experimental-strip-types eval/banking77.ts",
+    "bench": "esbuild src/worker.ts src/browser.ts --bundle --format=esm --platform=browser --outdir=bench --alias:onnxruntime-web/webgpu=onnxruntime-web/wasm && node bench/serve.mjs",
+    "playground": "node playground/build.mjs",
+    "heads": "node --experimental-strip-types heads/build.ts",
+    "bench:run": "node bench/run.mjs",
+    "eval:suite": "node --experimental-strip-types eval/suite.ts",
+    "eval:hill": "node --experimental-strip-types eval/hill.ts",
+    "test": "node --test --experimental-strip-types \"test/*.test.ts\"",
+    "lint:prose": "node scripts/lint-prose.mjs",
+    "smoke": "node scripts/smoke.mjs"
+  },
+  "dependencies": {
+    "@huggingface/transformers": "4.2.0"
+  },
+  "devDependencies": {
+    "@types/node": "22.15.3",
+    "@types/react": "19.3.0",
+    "esbuild": "0.25.10",
+    "playwright-core": "1.64.1",
+    "react": "19.3.1",
+    "spryte": "0.11.0",
+    "typescript": "5.9.3"
+  },
+  "engines": {
+    "node": ">=22.18"
+  },
+  "homepage": "https://debuff.jp/decide/",
+  "bugs": "https://github.com/debuffjp/decidekit/issues",
+  "bin": {
+    "decidekit": "./dist/cli.js"
+  },
+  "peerDependencies": {
+    "react": ">=18"
+  },
+  "peerDependenciesMeta": {
+    "react": {
+      "optional": true
+    }
+  }
+}