decidekit 0.1.1 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,77 +1,93 @@
1
- # Changelog
2
-
3
- ## 0.1.1 (2026-11-02)
4
-
5
- ### Models
6
-
7
- - The default model id is `debuffjp/decide-small` (Decide Small). Before 0.1.1 it was `Xenova/bge-small-en-v1.5`. The
8
- weights are identical, so decisions don't change. For other languages the id is `debuffjp/decide-multilingual`
9
- (Decide Multilingual), identical to `Xenova/multilingual-e5-small`.
10
- - The old ids and heads recording them keep working. They resolve to the new ids, so `d.model` returns the new id. A
11
- head trained on 0.1.1 records the new id, so a 0.1.0 decider needs that id passed as `model` to use it.
12
- - Downloads of both models are pinned to a revision.
13
- - New option `modelHost` serves model files from `<modelHost>/<model>/` instead of Hugging Face. It applies to the
14
- whole page or process.
15
-
16
- ### Docs
17
-
18
- - Label names count as 1 example in every doc, and the advice to turn them off with bge-small is gone: it applied to the
19
- old weight of 3.
20
- - Calibration, out-of-scope and zero-shot numbers are now bge-small's or labelled with the model they were measured on.
21
- - `decide()` documents that a described label scores by the better of its name and description.
22
- - The API reference covers `decidekit/typesafe`.
23
-
24
- ## 0.1.0 (2026-11-02)
25
-
26
- The first release. Numbers are on public test splits unless noted; the scripts that produced them are in `eval/`.
27
-
28
- ### Deciding
29
-
30
- - `createDecider()` picks one of your labels for a text, in Node or the browser, with `confidence` and `probabilities`. Labels
31
- are the type: `choice` has the literal type of the labels you pass.
32
- - Zero-shot from label names, or from described labels (`{ card_arrival: "My card hasn't arrived." }`), which score by
33
- the better of name and description. Zero-shot decisions report `similarity`, the best cosine similarity to any
34
- option, which is low for off-topic text.
35
- - `minConfidence` returns `null` below a threshold. `decideMany()` for batches.
36
- - Default encoder: `Xenova/bge-small-en-v1.5` at 8-bit (34 MB). Any transformers.js feature-extraction model can be
37
- passed as `model`.
38
- - Supported encoders with measured settings: `Xenova/bge-small-en-v1.5` (default) and `Xenova/multilingual-e5-small`.
39
- The temperature fit now searches 0.01 to 10, so encoders with close similarities calibrate.
40
- - `describeLabels()` asks an LLM once to write label descriptions (7 to 10 points better zero-shot on the dev splits).
41
-
42
- ### Training
43
-
44
- - `train()` fits one softmax layer on frozen embeddings: 90.7% on BANKING77 and 94.4% on CLINC150 with all examples,
45
- 82.3% and 89.6% with 5 per label. HWU64, never used for tuning: 90.2%.
46
- - Calibrated: the temperature is fitted on held-out examples (5 folds for small sets), with a per-model floor for
47
- small heads. Calibration error 0.01 to 0.03 for heads with 10 or more examples per label.
48
- - `labels` adds labels without examples yet (they start from their names). `calibrateOn` fits the temperature on
49
- examples known to be right, for training data from a source that is sometimes wrong.
50
- - `saveHead()` and `loadHead()`. Ready-made heads for BANKING77 and CLINC150.
51
-
52
- ### Learning from an LLM
53
-
54
- - `withFallback()` answers locally when confident and calls your `ask` otherwise; each answer becomes a training
55
- example and `retrainEvery` retrains in the background. It can start from labels alone.
56
- - Built for an LLM that is sometimes wrong: starting from labels, it fits confidence on zero-shot's own confident
57
- answers rather than the LLM's, and learns `null` answers as the label `NONE` (`learnNone`) when the text is far
58
- from every label (`noneBelow`), so it can answer `null` itself for off-topic text.
59
- - `route.saved()` returns the learned answers and the calibration set as plain JSON; pass them back as `restore` after
60
- a reload. With real Llama 3.1 8B answers on
61
- 12 labels, the share sent fell from 30% (zero-shot) to 19%, and off-topic text given a wrong label from 6.7% to 2.4%.
62
-
63
- ### Browser and tools
64
-
65
- - `decidekit/typesafe`: `typesafeClient()` gives a decider the shape of TypeSafe's SDK (`systemOne({ state, questions })`,
66
- `choice()`, `probabilities`, their `confidence` formula). Choice questions only.
67
- - `decidekit/browser`: `createWorkerDecider()` runs the encoder in a Web Worker (training BANKING77 froze the page for
68
- 171 s on the main thread; the longest stall in a worker was 18 ms).
69
- - `decidekit/react`: `useDecider()` and `useDecision()`.
70
- - `decidekit/switch`: a decider for the switch agent harness.
71
- - A CLI: `decidekit train` and `decidekit decide`.
72
-
73
- ### Known limits
74
-
75
- - The default encoder reads English. `Xenova/multilingual-e5-small` is supported for other languages (76% on Japanese
76
- and Chinese MASSIVE intents, calibrated), but `withFallback`'s off-topic gate doesn't fire with it.
77
- - Weak at tone: 61% on tweet sentiment and 49% on emotion with 32 examples per label. See the FAQ.
1
+ # Changelog
2
+
3
+ ## 0.2.1 (2026-11-02)
4
+
5
+ ### Browser
6
+
7
+ - `createWorkerDecider()` rejects when the worker fails to load, crashes or sends an unreadable message. Before, the call never settled. Calls after a failure or after `terminate()` reject too.
8
+ - The wasm download is 12.9 MB (3.3 MB gzipped) unless you pass `device: "webgpu"`. Earlier docs said an alias got this size, but in Chrome with an esbuild build, 0.2.0 fetched the 23.6 MB file with or without it. The alias is now optional and only trims about 14.5 KB gzipped of worker JS. Paths you set in `wasmPaths` are left alone.
9
+
10
+ ## 0.2.0 (2026-11-02)
11
+
12
+ ### Training
13
+
14
+ - From 8 examples per label (mean across labels), `train()` fits the head with full-batch Adam, starting from the same centroid and label-name point. Test accuracy with all data: BANKING77 90.6% to 93.1%, CLINC150 94.6% to 96.1%, HWU64 90.15% to 91.17% (one seed, on cached embeddings; through train() in eval/results-h14.json, BANKING77 93.1% and CLINC150 96.3%). At 5 per label nothing changes. At 10 per label: BANKING77 84.4% to 87.5%, CLINC150 92.2% to 92.5%, HWU64 85.50% to 85.38%. Adam's settings come from jimothy's `src/linear.ts` (MIT, Andrew Prifer).
15
+ - New option `solver`. Setting `learningRate` or `l2` without it keeps SGD, so tuned calls behave as before. The default `epochs` is 100 with Adam.
16
+ - Saved heads keep working unchanged. Retrained heads can score differently, so pick `minConfidence` thresholds again: the CLINC150 out-of-scope threshold moved from 0.9 to 0.98.
17
+ - Training takes longer: in Node, BANKING77 went from 84 s to 123 s on cached embeddings. In Chrome, BANKING77 took 208 s with a longest page stall of 19 ms.
18
+
19
+ ## 0.1.1 (2026-11-02)
20
+
21
+ ### Models
22
+
23
+ - The default model id is `debuffjp/decide-small` (Decide Small). Before 0.1.1 it was `Xenova/bge-small-en-v1.5`. The
24
+ weights are identical, so decisions don't change. For other languages the id is `debuffjp/decide-multilingual`
25
+ (Decide Multilingual), identical to `Xenova/multilingual-e5-small`.
26
+ - The old ids and heads recording them keep working. They resolve to the new ids, so `d.model` returns the new id. A
27
+ head trained on 0.1.1 records the new id, so a 0.1.0 decider needs that id passed as `model` to use it.
28
+ - Downloads of both models are pinned to a revision.
29
+ - New option `modelHost` serves model files from `<modelHost>/<model>/` instead of Hugging Face. It applies to the
30
+ whole page or process.
31
+
32
+ ### Docs
33
+
34
+ - Label names count as 1 example in every doc, and the advice to turn them off with bge-small is gone: it applied to the
35
+ old weight of 3.
36
+ - Calibration, out-of-scope and zero-shot numbers are now bge-small's or labelled with the model they were measured on.
37
+ - `decide()` documents that a described label scores by the better of its name and description.
38
+ - The API reference covers `decidekit/typesafe`.
39
+
40
+ ## 0.1.0 (2026-11-02)
41
+
42
+ The first release. Numbers are on public test splits unless noted; the scripts that produced them are in `eval/`.
43
+
44
+ ### Deciding
45
+
46
+ - `createDecider()` picks one of your labels for a text, in Node or the browser, with `confidence` and `probabilities`. Labels
47
+ are the type: `choice` has the literal type of the labels you pass.
48
+ - Zero-shot from label names, or from described labels (`{ card_arrival: "My card hasn't arrived." }`), which score by
49
+ the better of name and description. Zero-shot decisions report `similarity`, the best cosine similarity to any
50
+ option, which is low for off-topic text.
51
+ - `minConfidence` returns `null` below a threshold. `decideMany()` for batches.
52
+ - Default encoder: `Xenova/bge-small-en-v1.5` at 8-bit (34 MB). Any transformers.js feature-extraction model can be
53
+ passed as `model`.
54
+ - Supported encoders with measured settings: `Xenova/bge-small-en-v1.5` (default) and `Xenova/multilingual-e5-small`.
55
+ The temperature fit now searches 0.01 to 10, so encoders with close similarities calibrate.
56
+ - `describeLabels()` asks an LLM once to write label descriptions (7 to 10 points better zero-shot on the dev splits).
57
+
58
+ ### Training
59
+
60
+ - `train()` fits one softmax layer on frozen embeddings: 90.7% on BANKING77 and 94.4% on CLINC150 with all examples,
61
+ 82.3% and 89.6% with 5 per label. HWU64, never used for tuning: 90.2%.
62
+ - Calibrated: the temperature is fitted on held-out examples (5 folds for small sets), with a per-model floor for
63
+ small heads. Calibration error 0.01 to 0.03 for heads with 10 or more examples per label.
64
+ - `labels` adds labels without examples yet (they start from their names). `calibrateOn` fits the temperature on
65
+ examples known to be right, for training data from a source that is sometimes wrong.
66
+ - `saveHead()` and `loadHead()`. Ready-made heads for BANKING77 and CLINC150.
67
+
68
+ ### Learning from an LLM
69
+
70
+ - `withFallback()` answers locally when confident and calls your `ask` otherwise; each answer becomes a training
71
+ example and `retrainEvery` retrains in the background. It can start from labels alone.
72
+ - Built for an LLM that is sometimes wrong: starting from labels, it fits confidence on zero-shot's own confident
73
+ answers rather than the LLM's, and learns `null` answers as the label `NONE` (`learnNone`) when the text is far
74
+ from every label (`noneBelow`), so it can answer `null` itself for off-topic text.
75
+ - `route.saved()` returns the learned answers and the calibration set as plain JSON; pass them back as `restore` after
76
+ a reload. With real Llama 3.1 8B answers on
77
+ 12 labels, the share sent fell from 30% (zero-shot) to 19%, and off-topic text given a wrong label from 6.7% to 2.4%.
78
+
79
+ ### Browser and tools
80
+
81
+ - `decidekit/typesafe`: `typesafeClient()` gives a decider the shape of TypeSafe's SDK (`systemOne({ state, questions })`,
82
+ `choice()`, `probabilities`, their `confidence` formula). Choice questions only.
83
+ - `decidekit/browser`: `createWorkerDecider()` runs the encoder in a Web Worker (training BANKING77 froze the page for
84
+ 171 s on the main thread; the longest stall in a worker was 18 ms).
85
+ - `decidekit/react`: `useDecider()` and `useDecision()`.
86
+ - `decidekit/switch`: a decider for the switch agent harness.
87
+ - A CLI: `decidekit train` and `decidekit decide`.
88
+
89
+ ### Known limits
90
+
91
+ - The default encoder reads English. `Xenova/multilingual-e5-small` is supported for other languages (76% on Japanese
92
+ and Chinese MASSIVE intents, calibrated), but `withFallback`'s off-topic gate doesn't fire with it.
93
+ - Weak at tone: 61% on tweet sentiment and 49% on emotion with 32 examples per label. See the FAQ.
package/README.md CHANGED
@@ -16,8 +16,8 @@ const r = await d.decide("my card never arrived", ["card_arrival", "refund", "ot
16
16
  r.choice; // "card_arrival" | "refund" | "other"
17
17
  ```
18
18
 
19
- Give it 5 examples per label and it reaches 82% on BANKING77 (77 intents); give it the full training set and it
20
- reaches 90.7%, with each answer taking about 8 ms (median) in Chrome. Training runs in the user's tab, so their data never
19
+ Give it 5 examples per label and it reaches 84% on BANKING77 (77 intents); give it the full training set and it
20
+ reaches 93.1%, with each answer taking about 8 ms (median) in Chrome. Training runs in the user's tab, so their data never
21
21
  leaves the device.
22
22
 
23
23
  ## Zero-shot
@@ -134,8 +134,7 @@ const r = await d.decide(text, head);
134
134
 
135
135
  Import `loadHead` from `decidekit/browser` too, not from `decidekit`: that keeps the encoder out of your page bundle.
136
136
 
137
- Tested from npm in Vite 7, webpack 5 and Next.js 16 apps; see [docs/bundlers.md](docs/bundlers.md) for the config, including a one-line alias
138
- that halves the wasm download. Serve the page cross-origin isolated if you can:
137
+ Tested from npm in Vite 7, webpack 5 and Next.js 16 apps; see [docs/bundlers.md](docs/bundlers.md) for the config. Serve the page cross-origin isolated if you can:
139
138
 
140
139
  ```
141
140
  Cross-Origin-Opener-Policy: same-origin
@@ -144,7 +143,7 @@ Cross-Origin-Embedder-Policy: credentialless
144
143
 
145
144
  That lets wasm use several threads: in our runs with all-MiniLM-L6-v2 it cut p95 decision time from 14.3 ms to 7.8 ms. With the default bge-small, isolated, p95 is 13.8 ms.
146
145
 
147
- Download on first visit: the model (34 MB) and onnxruntime's wasm (23.6 MB, 5.8 MB gzipped), both cached after.
146
+ Download on first visit: the model (34 MB) and onnxruntime's wasm (12.9 MB, 3.3 MB gzipped; 23.6 MB before 0.2.1), both cached after.
148
147
 
149
148
  ## Models
150
149
 
@@ -167,6 +166,15 @@ Decide Small (bge-small-en-v1.5, the default) at q8, without label names, Ryzen
167
166
  and calibration error (ECE, lower is better) on each dataset's test split, mean of 3 seeds (one run for "all"):
168
167
  [eval/results-bge-small.json](eval/results-bge-small.json).
169
168
 
169
+ | Examples per label | 5 | 10 | 25 | all |
170
+ |---|---|---|---|---|
171
+ | BANKING77 (77 labels) | 83.7% | 87.5% | 90.2% | 93.1% |
172
+ | CLINC150 (150 labels) | 90.2% | 92.8% | 94.3% | 96.3% |
173
+ | ECE, BANKING77 | 0.050 | 0.015 | 0.013 | 0.013 |
174
+ | ECE, CLINC150 | 0.020 | 0.015 | 0.016 | 0.008 |
175
+
176
+ Those are 0.2, with label names, from [eval/results-h14.json](eval/results-h14.json). HWU64 with all data goes from 90.15% to 91.17%. Before 0.2 (SGD at every size, no label names, `eval/results-bge-small.json`):
177
+
170
178
  | Examples per label | 5 | 10 | 25 | all |
171
179
  |---|---|---|---|---|
172
180
  | BANKING77 (77 labels) | 82.3% | 83.8% | 88.7% | 90.7% |
@@ -189,7 +197,7 @@ queries, [eval/tiny.ts](eval/tiny.ts), measured with all-MiniLM-L6-v2), 1 exampl
189
197
  a threshold there.
190
198
 
191
199
  **In Chrome** (BANKING77, full training set, worker, cross-origin isolated, 16 threads): 7.7 ms median and 13.8 ms
192
- p95 per decision, 228 s to train on 10,003 examples with no page stall longer than 18 ms. all-MiniLM-L6-v2 in the
200
+ p95 per decision before 0.2, 228 s to train on 10,003 examples with no page stall longer than 18 ms. A 0.2 run on the same machine took 208 s, with a longest stall of 19 ms and 93.05% test accuracy (runs on this laptop vary: another 0.1 run took 147 s). all-MiniLM-L6-v2 in the
193
201
  same run: 4.2 ms median, 7.7 ms p95. In Node: 3.2 ms median for a single text, 1.1 ms per text batched (MiniLM 2.0
194
202
  and 0.7). Earlier MiniLM run: [bench/results.json](bench/results.json).
195
203
 
package/dist/browser.js CHANGED
@@ -18,7 +18,20 @@ export async function createWorkerDecider({ onProgress, ...options } = {}, worke
18
18
  pending.delete(data.id);
19
19
  data.error ? p.reject(new Error(data.error)) : p.resolve(data.value);
20
20
  };
21
+ // After this the worker never answers, so every call, now or later, has to reject.
22
+ let dead;
23
+ const fail = (message) => {
24
+ dead = new Error(message);
25
+ for (const p of pending.values())
26
+ p.reject(dead);
27
+ pending.clear();
28
+ };
29
+ // A failed load gives an empty message, and a wrong worker URL from the bundler is its usual cause.
30
+ worker.onerror = (e) => fail(`decidekit: the worker failed: ${e.message || "it did not load, check the worker URL your bundler produced"}`);
31
+ worker.onmessageerror = () => fail("decidekit: the worker failed: it sent a message the page can't read");
21
32
  const call = (op, payload, progress) => new Promise((resolve, reject) => {
33
+ if (dead)
34
+ return reject(dead);
22
35
  const id = next++;
23
36
  pending.set(id, { resolve, reject, progress });
24
37
  worker.postMessage({ id, op, ...payload });
@@ -35,9 +48,7 @@ export async function createWorkerDecider({ onProgress, ...options } = {}, worke
35
48
  train: (examples, { onEpoch, ...opts } = {}) => call("train", { args: [examples, opts] }, (p) => onEpoch?.(p.epoch, p.accuracy)),
36
49
  terminate: () => {
37
50
  worker.terminate();
38
- for (const p of pending.values())
39
- p.reject(new Error("decidekit: the worker was terminated"));
40
- pending.clear();
51
+ fail("decidekit: the worker was terminated");
41
52
  },
42
53
  };
43
54
  }
package/dist/head.js CHANGED
@@ -1,5 +1,10 @@
1
1
  const BATCH = 64;
2
2
  const CENTROID_SCALE = 20;
3
+ // At 5 per label Adam lost 0.29 points on CLINC150, at 10 it gained on BANKING77 and CLINC150 (eval/spike). The cutoff
4
+ // between them wasn't measured on its own.
5
+ const ADAM_FROM = 8;
6
+ // Without a better holdout loss for this many epochs, stop the probe run.
7
+ const ADAM_PATIENCE = 30;
3
8
  // Lets a tab paint and handle input between training batches. A message-channel task, unlike setTimeout, isn't
4
9
  // throttled to once a second (or minute) when the tab is in the background.
5
10
  function yieldToPage() {
@@ -56,11 +61,23 @@ function batchGradient(ids, X, y, W, B, gW, gb, z, C, dim) {
56
61
  }
57
62
  }
58
63
  }
64
+ // One Adam step from the summed gradient. Adam's settings follow jimothy's src/linear.ts (MIT, Andrew Prifer).
65
+ function adamStep(p, g, m, v, rows, l2, lr, epoch) {
66
+ const c1 = 1 - 0.9 ** epoch, c2 = 1 - 0.999 ** epoch;
67
+ for (let k = 0; k < p.length; k++) {
68
+ const d = g[k] / rows + l2 * p[k];
69
+ m[k] = 0.9 * m[k] + 0.1 * d;
70
+ v[k] = 0.999 * v[k] + 0.001 * d * d;
71
+ p[k] -= (lr * m[k]) / c1 / (Math.sqrt(v[k] / c2) + 1e-8);
72
+ }
73
+ }
59
74
  /** Fits weights, biases and temperature to embeddings `X` with label indices `y`. */
60
75
  export async function fitHead(X, y, labels, opts = {}, priors, priorWeight = 1, centroidScale = CENTROID_SCALE, smallFloor = 1, calibration) {
61
76
  // smallFloor: the lowest temperature a head with under 50 held-out answers may get; createDecider picks it per model.
62
77
  // calibration: labelled examples known to be right to fit the temperature on instead of the training examples.
63
- const { epochs = 60, learningRate = 2, l2 = 1e-4, holdout = 0.1, seed = 7, onEpoch } = opts;
78
+ // A set learningRate or l2 was tuned for SGD, so it keeps SGD unless the solver is named.
79
+ const adam = opts.solver ? opts.solver === "adam" : opts.learningRate === undefined && opts.l2 === undefined && X.length / labels.length >= ADAM_FROM;
80
+ const { epochs = adam ? 100 : 60, learningRate = adam ? 0.05 : 2, l2 = adam ? 0 : 1e-4, holdout = 0.1, seed = 7, onEpoch } = opts;
64
81
  const dim = X[0].length, C = labels.length;
65
82
  // One flat buffer, so the inner loops read typed memory instead of an array of arrays.
66
83
  const Xf = new Float64Array(X.length * dim);
@@ -92,6 +109,33 @@ export async function fitHead(X, y, labels, opts = {}, priors, priorWeight = 1,
92
109
  let best = { acc: check ? acc() : 0, weights: head.weights.slice(), bias: head.bias.slice(), epochs: 0 };
93
110
  const gW = new Float64Array(C * dim), gb = new Float64Array(C), z = new Float64Array(C);
94
111
  const W = head.weights, B = head.bias;
112
+ if (adam) {
113
+ const mW = new Float64Array(C * dim), vW = new Float64Array(C * dim), mB = new Float64Array(C), vB = new Float64Array(C);
114
+ let bestLoss = Infinity;
115
+ for (let ep = 0; ep < nEpochs; ep++) {
116
+ gW.fill(0);
117
+ gb.fill(0);
118
+ for (let b = 0; b < rows.length; b += BATCH) {
119
+ batchGradient(rows.slice(b, b + BATCH), Xf, y, W, B, gW, gb, z, C, dim);
120
+ await yieldToPage();
121
+ }
122
+ adamStep(W, gW, mW, vW, rows.length, l2, learningRate, ep + 1);
123
+ adamStep(B, gb, mB, vB, rows.length, 0, learningRate, ep + 1);
124
+ if (!check)
125
+ continue;
126
+ const loss = check.reduce((sum, i) => sum - Math.log(softmax(logits(head, X[i]), 1)[y[i]] + 1e-300), 0);
127
+ if (onEpoch)
128
+ onEpoch(ep, acc());
129
+ if (loss < bestLoss * (1 - 1e-7)) {
130
+ bestLoss = loss;
131
+ best = { acc: 0, weights: W.slice(), bias: B.slice(), epochs: ep + 1 };
132
+ }
133
+ if (ep + 1 - best.epochs >= ADAM_PATIENCE)
134
+ break;
135
+ await yieldToPage();
136
+ }
137
+ return check ? { ...head, weights: best.weights, bias: best.bias, epochs: best.epochs } : { ...head, epochs: nEpochs };
138
+ }
95
139
  const order = [...rows];
96
140
  for (let ep = 0; ep < nEpochs; ep++) {
97
141
  order.sort(() => rand() - 0.5);
package/dist/index.d.ts CHANGED
@@ -51,11 +51,16 @@ export interface TrainOptions<L extends string = string> {
51
51
  * can still be chosen. Needs `labelNames`.
52
52
  */
53
53
  labels?: readonly L[];
54
- /** Passes over the examples. Default 60; the best holdout epoch is kept. */
54
+ /**
55
+ * How to fit the head. By default `adam` from 8 examples per label (mean across labels), `sgd` below that. Setting
56
+ * `learningRate` or `l2` without this also picks `sgd`.
57
+ */
58
+ solver?: "adam" | "sgd";
59
+ /** Most passes over the examples. Default 100 with `adam`, 60 with `sgd`; the best holdout epoch is kept. */
55
60
  epochs?: number;
56
- /** Default 2. */
61
+ /** Default 0.05 with `adam`, 2 with `sgd`. */
57
62
  learningRate?: number;
58
- /** Weight decay. Default 1e-4. */
63
+ /** Weight decay. Default 0 with `adam`, 1e-4 with `sgd`. */
59
64
  l2?: number;
60
65
  /** Share of examples held out for early stopping and temperature fitting. Default 0.1. */
61
66
  holdout?: number;
@@ -74,7 +79,7 @@ export interface DeciderOptions {
74
79
  /**
75
80
  * A Hugging Face sentence-embedding model with ONNX weights. Default `debuffjp/decide-small` (Decide Small, 34 MB at q8).
76
81
  * Before 0.1.1 the default id was `Xenova/bge-small-en-v1.5`, the same weights; that id and heads recording it still work.
77
- * Pre-release builds defaulted to `Xenova/all-MiniLM-L6-v2` (23 MB): smaller and 1.6x faster, 3 to 5 points less
82
+ * Pre-release builds defaulted to `Xenova/all-MiniLM-L6-v2` (23.7 MB): smaller and about 1.8x faster in Chrome, 3 to 5 points less
78
83
  * accurate. Pass it to keep using heads trained on it.
79
84
  */
80
85
  model?: string;
package/dist/index.js CHANGED
@@ -40,6 +40,15 @@ function toDecision(labels, p) {
40
40
  const probabilities = Object.fromEntries(labels.map((l, i) => [l, p[i]]));
41
41
  return { choice: labels[best], confidence: p[best], probabilities };
42
42
  }
43
+ function usePlainWasm() {
44
+ // transformers.js points browsers other than Safari at the 23.6 MB WebGPU-capable build whatever JS is bundled.
45
+ // Leaves paths the app set itself alone.
46
+ const paths = env.backends.onnx.wasm?.wasmPaths;
47
+ if (!paths || typeof paths === "string" || !paths.wasm?.startsWith("https://cdn.jsdelivr.net/npm/onnxruntime-web@"))
48
+ return;
49
+ paths.wasm = paths.wasm.replace(".asyncify.", ".");
50
+ paths.mjs = paths.mjs?.replace(".asyncify.", ".");
51
+ }
43
52
  /**
44
53
  * Loads the encoder (downloaded once, then cached by the browser or transformers.js) and returns `decide()` and
45
54
  * `train()`. In a browser, prefer `createWorkerDecider()` from `decidekit/browser`: the encoder blocks the thread
@@ -51,6 +60,8 @@ export async function createDecider({ model: requested = SMALL, device, dtype =
51
60
  env.remoteHost = modelHost.endsWith("/") ? modelHost : modelHost + "/";
52
61
  env.remotePathTemplate = "{model}/";
53
62
  }
63
+ if (device !== "webgpu")
64
+ usePlainWasm();
54
65
  const embed = await pipeline("feature-extraction", model, {
55
66
  device, dtype, revision: REVISIONS[model],
56
67
  progress_callback: onProgress && ((p) => p.status === "progress" && onProgress({ file: p.file, loaded: p.loaded, total: p.total })),
@@ -103,8 +114,8 @@ export async function createDecider({ model: requested = SMALL, device, dtype =
103
114
  /**
104
115
  * Trains a head on labelled examples: mini-batch gradient descent on the embeddings, keeping the best holdout
105
116
  * epoch, then a temperature fitted on the holdout so `confidence` is calibrated. Every label needs at least a few
106
- * examples. Each label starts at the mean of its examples and its name, so 5 per label already gave 79% on BANKING77 and
107
- * 87% on CLINC150 (see the README).
117
+ * examples. Each label starts at the mean of its examples and its name, so 5 per label already gave 82.3% on BANKING77 and
118
+ * 89.6% on CLINC150 (see the README).
108
119
  */
109
120
  async function train(examples, opts = {}) {
110
121
  const bad = examples.findIndex((e) => typeof e?.text !== "string" || typeof e?.label !== "string" || !e.label);
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "decidekit",
3
- "version": "0.1.1",
3
+ "version": "0.2.1",
4
4
  "description": "A System One model in the browser: typed, calibrated text decisions on the user's device. Learns from an LLM when it isn't sure.",
5
5
  "license": "MIT",
6
6
  "repository": "github:debuffjp/decidekit",