jev-web 0.7.2 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,503 +1,626 @@
1
- # jev-web
2
-
3
- [![npm](https://img.shields.io/npm/v/jev-web.svg)](https://www.npmjs.com/package/jev-web)
4
- [![license](https://img.shields.io/npm/l/jev-web.svg)](./LICENSE)
5
- [![CI](https://github.com/warsang/jev-web/actions/workflows/test.yml/badge.svg)](https://github.com/warsang/jev-web/actions/workflows/test.yml)
6
-
7
- Run **typed-decision models** in the browser. One *state* text
8
- plus any number of typed *questions* (`choice` / `score` / `noul`) go in; a
9
- calibrated probability distribution per question comes back from a **single
10
- forward pass**. No server, no API keys, no network calls after the weights are
11
- cached.
12
-
13
- Three model families ship today — open-jev (fused DeBERTa graph), Laya
14
- (ModernBERT encoder + typed head), Strands Decider 2B (Qwen torso + pointer
15
- head), Bekko System One v0 (Ettin cross-encoder), Decision 2.0 (Qwen3 +
16
- candidate head), and Kev (Qwen3 + pointer head) — behind one registry:
17
- `createDecisionRuntime({ family, fallback })`.
18
-
19
- This is a generic runtime, not a classifier for any particular domain. You bring
20
- the state text and the questions; the model answers by choosing among the options
21
- you give it.
22
-
23
- ## ▶ Live demo
24
-
25
- **https://warsang.github.io/jev-web/**
26
-
27
- An interactive playground that runs this exact package: type a state, add typed
28
- questions, and watch the model and a hand-written keyword baseline answer
29
- side by side. It also counts your actual outbound requests, so the
30
- "offline / private / no server" claim is checkable rather than asserted.
31
-
32
- ![jev-web demo: typed decisions vs raw text parsing](docs/demo.gif)
33
-
34
- <sub>The frames are captured from the live demo at
35
- [warsang.github.io/jev-web](https://warsang.github.io/jev-web/) — site → typed
36
- decisions with latency → the negation case where the rules and the model
37
- disagree → the comparison matrix.</sub>
38
-
39
- Three questions, one forward pass, and a hard label from the rule parser that
40
- could not answer the third question at all:
41
-
42
- ![three typed decisions with full distributions and a measured latency, beside a keyword baseline that returns hard labels and no answer](docs/demo-typed.png)
43
-
44
- ## Quickstart
45
-
46
- ```js
47
- import { createDecider } from "jev-web";
48
-
49
- const decider = await createDecider(); // ~340 MB of weights, once
50
- const { answers } = await decider.decide(
51
- "I was charged twice for the same order.",
52
- [{ type: "noul", instructions: "The customer is asking for a refund." }],
53
- );
54
-
55
- answers[0].noul; // 0.92 — p(yes)
56
- answers[0].probabilities; // { no: 0.08, yes: 0.92 }
57
- answers[0].confidence; // 0.92
58
- ```
59
-
60
- That is the whole integration. The first call downloads the weights and the
61
- browser caches them; every later call — and every later visit, including
62
- offline — is local. **Zero dependencies**: `@huggingface/transformers` is an
63
- optional peer, imported lazily.
64
-
65
- ## What comes back
66
-
67
- ```js
68
- const { answers } = await decider.decide(
69
- "I was charged twice for the same order and nobody answers my emails. I want my money back now.",
70
- [
71
- { type: "choice", instructions: "Which product area is the message about?",
72
- options: ["fees & charges", "refund & dispute", "card", "other"] },
73
- { type: "noul", instructions: "The customer is asking for a refund." },
74
- { type: "score", instructions: "How negative is the message?",
75
- options: ["very negative", "negative", "neutral", "positive", "very positive"] },
76
- ],
77
- );
78
-
79
- // [
80
- // { type: "choice", choice: "fees & charges", index: 0,
81
- // probabilities: { "fees & charges": 0.796, "refund & dispute": 0.169,
82
- // card: 0.019, other: 0.016 }, confidence: 0.796 },
83
- // { type: "noul", noul: 0.927,
84
- // probabilities: { no: 0.073, yes: 0.927 }, confidence: 0.927 },
85
- // { type: "score", score: 0.866, level: 1,
86
- // probabilities: { "very negative": 0.336, negative: 0.478, neutral: 0.175,
87
- // positive: 0.008, "very positive": 0.003 }, confidence: 0.478 },
88
- // ]
89
- // timings: { totalMs, inferMs } · length, truncated, prompts
90
- ```
91
-
92
- Three questions, **one** forward pass, and every question carries the full
93
- distribution — not just a label. That last part is the point: a hard label
94
- cannot be thresholded, routed by confidence, or second-guessed, and the
95
- `choice` answer above is a genuine 79.6 / 16.9 split rather than a coin flip you
96
- find out about later.
97
-
98
- ## jev-web vs. calling a hosted LLM API with `fetch()`
99
-
100
- | | jev-web | Hosted LLM API via `fetch()` |
101
- |---|---|---|
102
- | **Cost per decision** | **$0.00** — the weights are static files | per-token, forever; a busy form multiplies it by every visitor |
103
- | **Latency** | one local forward pass, measured live on the demo page | typically 0.4–3 s, plus a network round trip |
104
- | **Network at answer time** | **0 requests** — works in airplane mode | one round trip per call, minimum |
105
- | **Output shape** | **calibrated distribution per question, every question in one pass** | prose you then have to regex back into a distribution |
106
- | **Determinism** | same input → same probabilities, always | temperature 0 + structured outputs helps; still not guaranteed |
107
- | **Privacy** | your text never leaves the device | your users' text is sent to a third party on every call |
108
- | **Ops burden** | static files on a CDN, no key, no rate limit | key management, quotas, rate limits, uptime, spend alerts |
109
- | **Ceiling** | only *picks among options you supply* — never writes, never explains | general purpose; will happily invent an option you never offered |
110
-
111
- **The honest trade.** jev-web is a classifier, not a chatbot. If you need
112
- free-form answers you need a generator. If you need a routing decision, a score,
113
- or a yes/no with a confidence you can threshold on, this is the cheaper, faster
114
- and private one — and it is also *smaller*: a 0.5 B model plus your options
115
- beats a frontier model plus a parsing layer, on every axis except generality.
116
-
117
- ## Try it before you install it
118
-
119
- The live demo covers the interesting cases, and the presets are chosen to break
120
- keyword parsing rather than to flatter the model:
121
-
122
- | preset | what it shows |
123
- |---|---|
124
- | **Banking triage** | easy text — a fair rule can match, the model adds a confidence |
125
- | **Negation trap** | *"This is not about a refund. My card was stolen…"* — the keyword rule routes to `refund & dispute`; the model routes to `card` (53.3%) and answers the refund question **no** (0.101) |
126
- | **Contradictory review** | praise and complaint in one sentence; keyword polarity reads the praise |
127
- | **Mixed intent** | two intents in one state — a hard label must pick one, a distribution does not have to |
128
- | **Out of domain** | no in-domain signal at all; the rule parser has nothing to say |
129
-
130
- The negation case is the one worth looking at. The state is *"This is not about
131
- a refund. My card was stolen in Madrid and the bank refuses to help. I never
132
- received the money back I asked about."*
133
-
134
- ![negation: the keyword baseline routes to refund and dispute, the model routes to card](docs/demo-negation.png)
135
-
136
- The rule parser finds the word `refund` and routes to **refund & dispute** — the
137
- exact opposite of what the customer meant. The model routes to **card** (53.3%)
138
- and answers *"the customer is asking for a refund"* with **no** (0.101). Both
139
- the bad routing and the inversion come from the same clause, and neither is
140
- visible in a hard label.
141
-
142
- The demo also tracks outbound requests through the Performance API: you will see
143
- ~14 requests while the weights load, then **0** for every decision after that.
144
-
145
- ## Install
146
-
147
- ```bash
148
- npm i jev-web @huggingface/transformers
149
- ```
150
-
151
- `@huggingface/transformers` is an optional peer dependency (it is imported
152
- lazily). Hosts that already ship their own copy can inject it via
153
- `createDecider({ transformers })`.
154
-
155
- ## Use
156
-
157
- ```js
158
- import { createDecider, DEFAULT_MODEL, DEFAULT_REVISION } from "jev-web";
159
-
160
- const decider = await createDecider({
161
- // defaults to the reference open-jev DeBERTa-v3-large ONNX export,
162
- // pinned by revision; any repo with the same typed-decision graph works
163
- onProgress: (p) => console.log(p.phase, p.file, p.progress),
164
- });
165
-
166
- const { answers } = await decider.decide(
167
- "I was charged twice for the same order and nobody answers my emails.",
168
- [
169
- {
170
- type: "choice",
171
- instructions: "Which product area is the message about?",
172
- options: ["fees & charges", "refund & dispute", "card", "other"],
173
- },
174
- { type: "noul", instructions: "The customer is asking for a refund." },
175
- {
176
- type: "score",
177
- instructions: "How negative is the message?",
178
- options: ["very negative", "negative", "neutral", "positive", "very positive"],
179
- },
180
- ],
181
- );
182
- ```
183
-
184
- ## Question types
185
-
186
- | type | options | answer |
187
- |---|---|---|
188
- | `choice` | 2–255 labels | `choice` (argmax), `probabilities`, `confidence` |
189
- | `score` | 2–10 ordered levels | `score` (expected level index, may fall between levels) |
190
- | `noul` | exactly 2 (defaults `["no","yes"]`) | `noul` (p of the second option = p(yes)), `confidence` |
191
-
192
- Every answer carries the full distribution. `decide()` also returns
193
- `{ truncated, length }` — the state is capped (256 tokens by default) and the
194
- flag tells you when it was cut.
195
-
196
- ## Backends and quantization
197
-
198
- `device` / `dtype` default to `auto`: WebGPU → `q4f16`, otherwise WASM → `q4`.
199
- Override explicitly (`fp16`, `fp32`, `q8`, …) when accuracy matters more than
200
- download size. First load downloads the weights once; transformers.js caches
201
- them (Cache API / IndexedDB) so later visits are offline-fast.
202
-
203
- `decider.info` reports the resolved `{ model, revision, device, dtype,
204
- temperature, maxStateTokens, maxLen, pool, configSource }`.
205
-
206
- ## Model config
207
-
208
- Defaults come from the model repo's `open_jev_config.json` when present
209
- (temperature, marker tokens, state cap), else the published reference values.
210
- To use a different typed-decision export:
211
-
212
- ```js
213
- createDecider({ model: "your-org/your-typed-decision-ONNX", revision: "main" });
214
- ```
215
-
216
- The export must add the `[STATE]`, `[Q]`, `[OPT]` markers to its tokenizer,
217
- take `input_ids, attention_mask, seg, pair_q, pair_opt`, and return per-pair
218
- `logits`. `typedMarkerIds()` fails loudly when a repo is missing them.
219
-
220
- ## Multiple families (registry)
221
-
222
- `jev-web` is a multi-family runtime: `createDecisionRuntime({ family, fallback })`
223
- tries the requested family and then any fallbacks, returning the shared contract
224
- `{ info, decide(state, questions), dispose? }`:
225
-
226
- ```js
227
- import { createDecisionRuntime, listDecisionFamilies } from "jev-web";
228
-
229
- listDecisionFamilies(); // ["open-jev", "laya", "strands-decider", "bekko", "decision2", "kev"]
230
- const runtime = await createDecisionRuntime({
231
- family: "strands-decider",
232
- fallback: ["laya", "open-jev"],
233
- onFallback: ({ family, error }) => console.warn(family, error),
234
- });
235
- const { answers, truncated, length, prompts, timings } = await runtime.decide(state, questions);
236
- ```
237
-
238
- Third-party families plug in without touching core:
239
-
240
- ```js
241
- registerDecisionFamily("my-model", {
242
- defaults: { model: "org/my-typed-decision-ONNX", revision: "main" },
243
- create: async (opts) => ({ info, decide, dispose }),
244
- });
245
- ```
246
-
247
- `decide()` returns `prompts` (decoded input sequence(s), for audit/debug) and
248
- `timings` (per-stage ms) alongside the answers.
249
-
250
- ## Laya family (ModernBERT encoder + typed head)
251
-
252
- `createLayaDecider(options)` runs the [Laya](https://huggingface.co/convaiinnovations/laya)
253
- System-1 decision models, whose browser export is a **two-file q8 ONNX split**
254
- (encoder + head) rather than one fused graph:
255
-
256
- ```js
257
- import { createLayaDecider } from "jev-web";
258
-
259
- const decider = await createLayaDecider({
260
- // defaults to alfred361/laya-typed-decisions-web-q8 @ pinned revision
261
- wasmPaths: "/assets/", // self-hosted ORT wasm (optional)
262
- onProgress: (p) => console.log(p.phase, p.loaded, p.total),
263
- });
264
- const { answers } = await decider.decide(stateText, questions);
265
- ```
266
-
267
- Protocol implemented here (mirrors the reference `laya` Python runtime):
268
-
269
- - sequence `[CLS] <type> question: <instructions> [SEP] [MASK]opt0 … [SEP] <state> [SEP]`,
270
- `marker_pos` at each option's `[MASK]`, `qtype` 0 choice / 1 score / 2 noul;
271
- - encoder `input_ids, attention_mask → hidden`; head
272
- `hidden, attention_mask, marker_pos, marker_mask, qtype → logits`;
273
- - per-question calibration from `rl_agent_config.json`
274
- (`temperature` + `temperature_by_options` buckets, clamped to [0.5, 5]).
275
-
276
- Notes:
277
- - the q8 web head is **batch-1 only**, so `decide()` batches the encoder and
278
- runs the head once per question;
279
- - `choice` supports at most `max_prefixes` options (6 for the typed-decisions
280
- export);
281
- - weights download once (~520 MB q8) and are browser-cached.
282
-
283
- Lower-level helpers are exported for custom runtimes:
284
- `buildLayaSequence`, `collateLayaItems`, `layaAnswersFromLogits`,
285
- `layaTokenizerAdapter`, `renderLayaOptions`, `layaTempBucket`, `clampTemperature`.
286
-
287
- ## Strands Decider family (Qwen torso + pointer head)
288
-
289
- `createStrandsDecider(options)` runs
290
- [Strands Decider 2B](https://strandsagents.com/blog/introducing-strands-decider/),
291
- a 2B-parameter decision model: a Qwen3.5-2B torso (LoRA merged) with its LM head
292
- replaced by a pointer head that scores each option from its own hidden state
293
- against the `<answer>` position. One fused ONNX graph, one forward pass for the
294
- whole batch:
295
-
296
- ```js
297
- import { createStrandsDecider } from "jev-web";
298
-
299
- const decider = await createStrandsDecider({
300
- // defaults to onnx-community/strands-decider-2B-hobson-v19-ONNX @ pinned
301
- // revision, q8 (onnx/model_quantized.onnx)
302
- onProgress: (p) => console.log(p.phase, p.loaded, p.total),
303
- });
304
- const { answers } = await decider.decide(stateText, questions);
305
- ```
306
-
307
- Protocol implemented here (mirrors the reference Python runtime):
308
-
309
- - prompt `<state>…</state><question type="{noul|choice|score}">…<options>\n1. …\n</options></question><answer>`;
310
- - graph `input_ids, attention_mask, answer_pos, option_pos → logits`, where
311
- `answer_pos` is each row's last real token (the `<answer>` pooling position)
312
- and `option_pos` carries each option line's last-token position (padded with
313
- 0); the prompt is tokenised in chunks at pre-tokeniser piece boundaries so
314
- those positions are exact with no offset mapping;
315
- - choice questions accept `criteria` as `{label: description}` (mirroring the
316
- reference schema); noul slots are fixed `false`/`true` with overridable
317
- rubric descriptions;
318
- - the question gets first claim on the 4096-token window (up to 75%), the state
319
- is truncated from the right into what remains, and an over-long question is
320
- truncated from the front, keeping the options and `<answer>`;
321
- - per-kind calibration from the checkpoint (`noul` 0.911 / `choice` 0.734 /
322
- `score` 1.328); `score` confidence is ordinal (spread-based, with the
323
- training smoothing floor), not max-probability.
324
-
325
- Lower-level helpers are exported for custom runtimes:
326
- `renderStrandsQuestion`, `renderStrandsState`, `renderStrandsContent`,
327
- `buildStrandsBatch`, `collateStrandsItems`, `strandsOptionTokenIndices`,
328
- `strandsAnswersFromLogits`, `strandsChoiceConfidence`, `strandsScoreConfidence`.
329
-
330
- ## Bekko family (Ettin cross-encoder)
331
-
332
- `createBekkoDecider(options)` runs
333
- [Bekko System One v0](https://github.com/hotchpotch/bekko-system-one), a
334
- cross-encoder that scores each (query, candidate) pair: the query renders as
335
- `Instruction: {instruction}\nState: {state}` and every option becomes a
336
- `Candidate: {id}: {description}` document. One ONNX graph
337
- (`prefix_ids, prefix_mask, doc_ids, doc_mask, owners → logits [N, tasks]`),
338
- one forward per question, with candidates batched by the reference
339
- attention budget:
340
-
341
- ```js
342
- import { createBekkoDecider } from "jev-web";
343
-
344
- const decider = await createBekkoDecider({
345
- // defaults to hotchpotch/bekko-system-one-v0-17m @ pinned revision,
346
- // onnx_browser/ (29 MB)
347
- onProgress: (p) => console.log(p.phase, p.loaded, p.total),
348
- });
349
- // or pick a larger sibling:
350
- // await createBekkoDecider({ variant: "68m" }); // 196 MB
351
- // await createBekkoDecider({ variant: "400m" }); // 1.43 GB, #2 on the S1MB leaderboard
352
- const { answers } = await decider.decide(stateText, questions);
353
- ```
354
-
355
- Protocol implemented here (mirrors the reference browser runtime,
356
- `browser/src/core.js` + `decision.js`, MIT):
357
-
358
- - query `[cls] Instruction: …\nState: … [sep]`; instruction and state split the
359
- 4096-token budget 50/50 with spillover; candidates truncate to 2048 tokens;
360
- - choice questions accept `criteria` as `{label: description}`; noul slots are
361
- fixed `true`/`false` with authored meanings (overridable via criteria);
362
- - score levels map to numeric candidate values `0..n-1`; the answer is the
363
- expected value plus the argmax level and a normalised score.
364
-
365
- Larger siblings are available via the `variant` option (`"17m"`, `"68m"`,
366
- `"400m"`) or the exported model/revision constants: `BEKKO_68M_MODEL` /
367
- `BEKKO_68M_REVISION` and `BEKKO_400M_MODEL` / `BEKKO_400M_REVISION` (same
368
- `onnx_browser/` layout). The `BEKKO_VARIANTS` map carries each size's pinned
369
- revision and download size.
370
-
371
- Lower-level helpers are exported for custom runtimes:
372
- `renderBekkoContent`, `renderBekkoRequest`, `tokenizeBekkoRequest`,
373
- `collateBekkoDocs`, `bekkoSoftmax`, `bekkoAnswerFromLogits`.
374
-
375
- ## Decision 2.0 family (Qwen3 + candidate head)
376
-
377
- `createDecision2Decider(options)` runs
378
- [vLLM Semantic Router Decision 2.0](https://huggingface.co/collections/vllm-sr/decision-20),
379
- a Qwen3 fine-tune with a candidate head: one prompt per question, each option
380
- rendered as a `{"description": …, "key": …}` JSON segment. One ONNX graph
381
- (`input_ids, attention_mask, answer_pos, option_pos → logits [B, K]`), one
382
- forward pass for the whole batch:
383
-
384
- ```js
385
- import { createDecision2Decider } from "jev-web";
386
-
387
- const decider = await createDecision2Decider({
388
- // defaults to onnx-community/Decision-2.0-Kai-0.6B-ONNX @ pinned revision,
389
- // q8 (onnx/model_quantized.onnx, 0.61 GB)
390
- onProgress: (p) => console.log(p.phase, p.loaded, p.total),
391
- });
392
- // or pick a larger sibling:
393
- // await createDecision2Decider({ variant: "eos" }); // 0.8B, 16k context
394
- // await createDecision2Decider({ variant: "sol" }); // 2B, 16k context
395
- const { answers } = await decider.decide(stateText, questions);
396
- ```
397
-
398
- Protocol implemented here (mirrors the reference `decision2` runtime):
399
-
400
- - segments `Context:` / `Task type:` / `Question:` / `Options:` / one
401
- `<option>` JSON segment per option / the `Decision:` suffix, each tokenized
402
- separately so `option_pos` (last token of each `</option>`) and `answer_pos`
403
- (last token) are exact — validated 11/11 against the reference fixtures;
404
- - choice questions accept `criteria` as `{label: description}` (empty
405
- descriptions encode as `null`); noul slots are fixed `false`/`true`;
406
- - score questions add the per-level `score_bias` from `config.json` before the
407
- softmax (temperature 1).
408
-
409
- Larger siblings are available via the `variant` option (`"kai"`, `"eos"`,
410
- `"sol"`) or the exported model/revision constants: `DECISION2_EOS_MODEL` /
411
- `DECISION2_EOS_REVISION` (0.8B) and `DECISION2_SOL_MODEL` /
412
- `DECISION2_SOL_REVISION` (2B). The `DECISION2_VARIANTS` map carries each
413
- variant's pinned revision and context size.
414
-
415
- Lower-level helpers are exported for custom runtimes:
416
- `renderDecision2Content`, `renderDecision2Options`, `buildDecision2Segments`,
417
- `tokenizeDecision2Segments`, `decision2Softmax`, `decision2AnswerFromLogits`.
418
-
419
- ## Kev family (Qwen3 + pointer head)
420
-
421
- `createKevDecider(options)` runs
422
- [Kev](https://huggingface.co/jaredpalmer/kev-0.6b) (Apache-2.0), a typed
423
- decision model: one state and any number of typed questions are packed into a
424
- single sequence with five delimiter tokens; a block-causal mask derived
425
- in-graph lets every question see the state and itself only, and a pointer head
426
- scores each option's `</opt>` token against its question's `<decide>` token.
427
- One ONNX graph (`input_ids, attention_mask → logits [B, seq]`), one forward
428
- pass for the whole batch — read the logit at each option's `</opt>` position
429
- and softmax within the question:
430
-
431
- ```js
432
- import { createKevDecider } from "jev-web";
433
-
434
- const decider = await createKevDecider({
435
- // defaults to onnx-community/kev-0.6b-ONNX @ pinned revision,
436
- // onnx/model_q4f16.onnx (335 MB)
437
- onProgress: (p) => console.log(p.phase, p.loaded, p.total),
438
- });
439
- // or pick the larger sibling:
440
- // await createKevDecider({ variant: "4b" }); // 1.07 GB, #29 on the Decision Index
441
- const { answers } = await decider.decide(stateText, questions);
442
- ```
443
-
444
- Protocol implemented here (mirrors the reference JS in the model README):
445
-
446
- - delimiters `<|fim_prefix|>` (state), `<|fim_middle|>` (question),
447
- `<|box_start|>` / `<|box_end|>` (option), `<|fim_suffix|>` (decide); ids
448
- derived through the tokenizer, limits from `config.json`'s `kev` block
449
- (8192 state / 8192 branch tokens, 255 options);
450
- - caller text is escaped (`<|name|>` → `<¦name¦>`) so it can never forge a
451
- delimiter;
452
- - choice questions accept `criteria` as `{label: description}`, rendered as
453
- `label: description`; noul defaults to `no` / `yes`.
454
-
455
- The `KEV_VARIANTS` map (`"0.6b"`, `"4b"`) carries each size's pinned revision
456
- and download size. Lower-level helpers are exported for custom runtimes:
457
- `escapeKevText`, `packKevSequence`.
458
-
459
- ## Worker / thread
460
-
461
- `jev-web` does not bundle a worker, so hosts can create one the way their
462
- bundler prefers:
463
-
464
- ```js
465
- // host code
466
- const worker = new Worker(new URL("./my-worker.mjs", import.meta.url), { type: "module" });
467
- // my-worker.mjs
468
- import { createDecider } from "jev-web";
469
- ```
470
-
471
- ## API
472
-
473
- - `createDecider(options)` → open-jev family: `{ info, decide, decideMany, dispose }`
474
- - `createLayaDecider(options)` → Laya family: `{ info, decide, dispose }` (same `decide` contract)
475
- - `createStrandsDecider(options)` → Strands Decider family: `{ info, decide, dispose }` (same `decide` contract)
476
- - `normalizeQuestions`, `buildDecisionInput`, `typedMarkerIds`, `answersFromScores`,
477
- `softmaxWithTemperature`, `resolveDevice`, `resolveDtype`, `detectWebGPU`,
478
- `loadModelConfig`, defaults (model/revision/temperature/caps).
479
-
480
- ## Demos
481
-
482
- - **Live demo** — <https://warsang.github.io/jev-web/> (built from `demo/`, deployed on every push)
483
- - **Local playground** — `npm run dev` → <http://localhost:5188>. Same ideas, plus
484
- COOP/COEP headers that unlock `SharedArrayBuffer` for the threaded
485
- onnxruntime-web build (GitHub Pages cannot set those headers, so the deployed
486
- demo runs single-threaded).
487
- - **Verify a change visually** — `node tools/shot.mjs jobs.json` drives headless
488
- Chrome over CDP, waits on a real DOM condition, and reports the page's own
489
- console errors. The GIF in this README was built with it. It exists because a
490
- screenshot alone cannot tell you a promise is silently rejecting.
491
-
492
- ## Limits
493
-
494
- - Typed-decision models answer **only** by picking among your options; they do
495
- not explain or generate text.
496
- - Accuracy is domain-bound: questions and states unlike the model's training
497
- domain are out of distribution. Measure before relying on any answer.
498
- - English, 512-token sequence (state capped), one state per forward pass.
499
-
500
- ## License
501
-
502
- MIT (runtime code). Model weights keep their own license (the reference export
1
+ # jev-web
2
+
3
+ [![npm](https://img.shields.io/npm/v/jev-web.svg)](https://www.npmjs.com/package/jev-web)
4
+ [![license](https://img.shields.io/npm/l/jev-web.svg)](./LICENSE)
5
+ [![CI](https://github.com/warsang/jev-web/actions/workflows/test.yml/badge.svg)](https://github.com/warsang/jev-web/actions/workflows/test.yml)
6
+
7
+ Run **typed-decision models** in the browser. One *state* text
8
+ plus any number of typed *questions* (`choice` / `score` / `noul`) go in; a
9
+ calibrated probability distribution per question comes back from a **single
10
+ forward pass**. No server, no API keys, no network calls after the weights are
11
+ cached.
12
+
13
+ Seven model families ship today — open-jev (fused DeBERTa graph), Laya
14
+ (ModernBERT encoder + typed head), Strands Decider 2B (Qwen torso + pointer
15
+ head), Bekko System One v0 (Ettin cross-encoder), Decision 2.0 (Qwen3 +
16
+ candidate head), Kev (Qwen3 + pointer head), and Vega 0.8B (frozen Qwen3.5
17
+ torso + latent decision-physics engine) — behind one registry:
18
+ `createDecisionRuntime({ family, fallback })`.
19
+
20
+ This is a generic runtime, not a classifier for any particular domain. You bring
21
+ the state text and the questions; the model answers by choosing among the options
22
+ you give it.
23
+
24
+ ## ▶ Live demo
25
+
26
+ **https://warsang.github.io/jev-web/**
27
+
28
+ An interactive playground that runs this exact package: type a state, add typed
29
+ questions, and watch the model and a hand-written keyword baseline answer
30
+ side by side. It also counts your actual outbound requests, so the
31
+ "offline / private / no server" claim is checkable rather than asserted.
32
+
33
+ ![jev-web demo: typed decisions vs raw text parsing](docs/demo.gif)
34
+
35
+ <sub>The frames are captured from the live demo at
36
+ [warsang.github.io/jev-web](https://warsang.github.io/jev-web/) — site → typed
37
+ decisions with latency → the negation case where the rules and the model
38
+ disagree → the comparison matrix.</sub>
39
+
40
+ Three questions, one forward pass, and a hard label from the rule parser that
41
+ could not answer the third question at all:
42
+
43
+ ![three typed decisions with full distributions and a measured latency, beside a keyword baseline that returns hard labels and no answer](docs/demo-typed.png)
44
+
45
+ ## Quickstart
46
+
47
+ ```js
48
+ import { createDecider } from "jev-web";
49
+
50
+ const decider = await createDecider(); // ~340 MB of weights, once
51
+ const { answers } = await decider.decide(
52
+ "I was charged twice for the same order.",
53
+ [{ type: "noul", instructions: "The customer is asking for a refund." }],
54
+ );
55
+
56
+ answers[0].noul; // 0.92 — p(yes)
57
+ answers[0].probabilities; // { no: 0.08, yes: 0.92 }
58
+ answers[0].confidence; // 0.92
59
+ ```
60
+
61
+ That is the whole integration. The first call downloads the weights and the
62
+ browser caches them; every later call — and every later visit, including
63
+ offline — is local. **Zero dependencies**: `@huggingface/transformers` is an
64
+ optional peer, imported lazily.
65
+
66
+ ## What comes back
67
+
68
+ ```js
69
+ const { answers } = await decider.decide(
70
+ "I was charged twice for the same order and nobody answers my emails. I want my money back now.",
71
+ [
72
+ { type: "choice", instructions: "Which product area is the message about?",
73
+ options: ["fees & charges", "refund & dispute", "card", "other"] },
74
+ { type: "noul", instructions: "The customer is asking for a refund." },
75
+ { type: "score", instructions: "How negative is the message?",
76
+ options: ["very negative", "negative", "neutral", "positive", "very positive"] },
77
+ ],
78
+ );
79
+
80
+ // [
81
+ // { type: "choice", choice: "fees & charges", index: 0,
82
+ // probabilities: { "fees & charges": 0.796, "refund & dispute": 0.169,
83
+ // card: 0.019, other: 0.016 }, confidence: 0.796 },
84
+ // { type: "noul", noul: 0.927,
85
+ // probabilities: { no: 0.073, yes: 0.927 }, confidence: 0.927 },
86
+ // { type: "score", score: 0.866, level: 1,
87
+ // probabilities: { "very negative": 0.336, negative: 0.478, neutral: 0.175,
88
+ // positive: 0.008, "very positive": 0.003 }, confidence: 0.478 },
89
+ // ]
90
+ // timings: { totalMs, inferMs } · length, truncated, prompts
91
+ ```
92
+
93
+ Three questions, **one** forward pass, and every question carries the full
94
+ distribution — not just a label. That last part is the point: a hard label
95
+ cannot be thresholded, routed by confidence, or second-guessed, and the
96
+ `choice` answer above is a genuine 79.6 / 16.9 split rather than a coin flip you
97
+ find out about later.
98
+
99
+ ## jev-web vs. calling a hosted LLM API with `fetch()`
100
+
101
+ | | jev-web | Hosted LLM API via `fetch()` |
102
+ |---|---|---|
103
+ | **Cost per decision** | **$0.00** — the weights are static files | per-token, forever; a busy form multiplies it by every visitor |
104
+ | **Latency** | one local forward pass, measured live on the demo page | typically 0.4–3 s, plus a network round trip |
105
+ | **Network at answer time** | **0 requests** — works in airplane mode | one round trip per call, minimum |
106
+ | **Output shape** | **calibrated distribution per question, every question in one pass** | prose you then have to regex back into a distribution |
107
+ | **Determinism** | same input → same probabilities, always | temperature 0 + structured outputs helps; still not guaranteed |
108
+ | **Privacy** | your text never leaves the device | your users' text is sent to a third party on every call |
109
+ | **Ops burden** | static files on a CDN, no key, no rate limit | key management, quotas, rate limits, uptime, spend alerts |
110
+ | **Ceiling** | only *picks among options you supply* — never writes, never explains | general purpose; will happily invent an option you never offered |
111
+
112
+ **The honest trade.** jev-web is a classifier, not a chatbot. If you need
113
+ free-form answers you need a generator. If you need a routing decision, a score,
114
+ or a yes/no with a confidence you can threshold on, this is the cheaper, faster
115
+ and private one — and it is also *smaller*: a 0.5 B model plus your options
116
+ beats a frontier model plus a parsing layer, on every axis except generality.
117
+
118
+ ## Try it before you install it
119
+
120
+ The live demo covers the interesting cases, and the presets are chosen to break
121
+ keyword parsing rather than to flatter the model:
122
+
123
+ | preset | what it shows |
124
+ |---|---|
125
+ | **Banking triage** | easy text — a fair rule can match, the model adds a confidence |
126
+ | **Negation trap** | *"This is not about a refund. My card was stolen…"* — the keyword rule routes to `refund & dispute`; the model routes to `card` (53.3%) and answers the refund question **no** (0.101) |
127
+ | **Contradictory review** | praise and complaint in one sentence; keyword polarity reads the praise |
128
+ | **Mixed intent** | two intents in one state — a hard label must pick one, a distribution does not have to |
129
+ | **Out of domain** | no in-domain signal at all; the rule parser has nothing to say |
130
+
131
+ The negation case is the one worth looking at. The state is *"This is not about
132
+ a refund. My card was stolen in Madrid and the bank refuses to help. I never
133
+ received the money back I asked about."*
134
+
135
+ ![negation: the keyword baseline routes to refund and dispute, the model routes to card](docs/demo-negation.png)
136
+
137
+ The rule parser finds the word `refund` and routes to **refund & dispute** — the
138
+ exact opposite of what the customer meant. The model routes to **card** (53.3%)
139
+ and answers *"the customer is asking for a refund"* with **no** (0.101). Both
140
+ the bad routing and the inversion come from the same clause, and neither is
141
+ visible in a hard label.
142
+
143
+ The demo also tracks outbound requests through the Performance API: you will see
144
+ ~14 requests while the weights load, then **0** for every decision after that.
145
+
146
+ ## Install
147
+
148
+ ```bash
149
+ npm i jev-web @huggingface/transformers
150
+ ```
151
+
152
+ `@huggingface/transformers` is an optional peer dependency (it is imported
153
+ lazily). Hosts that already ship their own copy can inject it via
154
+ `createDecider({ transformers })`.
155
+
156
+ ## Use
157
+
158
+ ```js
159
+ import { createDecider, DEFAULT_MODEL, DEFAULT_REVISION } from "jev-web";
160
+
161
+ const decider = await createDecider({
162
+ // defaults to the reference open-jev DeBERTa-v3-large ONNX export,
163
+ // pinned by revision; any repo with the same typed-decision graph works
164
+ onProgress: (p) => console.log(p.phase, p.file, p.progress),
165
+ });
166
+
167
+ const { answers } = await decider.decide(
168
+ "I was charged twice for the same order and nobody answers my emails.",
169
+ [
170
+ {
171
+ type: "choice",
172
+ instructions: "Which product area is the message about?",
173
+ options: ["fees & charges", "refund & dispute", "card", "other"],
174
+ },
175
+ { type: "noul", instructions: "The customer is asking for a refund." },
176
+ {
177
+ type: "score",
178
+ instructions: "How negative is the message?",
179
+ options: ["very negative", "negative", "neutral", "positive", "very positive"],
180
+ },
181
+ ],
182
+ );
183
+ ```
184
+
185
+ ## Question types
186
+
187
+ | type | options | answer |
188
+ |---|---|---|
189
+ | `choice` | 2–255 labels | `choice` (argmax), `probabilities`, `confidence` |
190
+ | `score` | 2–10 ordered levels | `score` (expected level index, may fall between levels) |
191
+ | `noul` | exactly 2 (defaults `["no","yes"]`) | `noul` (p of the second option = p(yes)), `confidence` |
192
+
193
+ Every answer carries the full distribution. `decide()` also returns
194
+ `{ truncated, length }` — the state is capped (256 tokens by default) and the
195
+ flag tells you when it was cut.
196
+
197
+ ## Backends and quantization
198
+
199
+ `device` / `dtype` default to `auto`: WebGPU → `q4f16`, otherwise WASM → `q4`.
200
+ Override explicitly (`fp16`, `fp32`, `q8`, …) when accuracy matters more than
201
+ download size. First load downloads the weights once; transformers.js caches
202
+ them (Cache API / IndexedDB) so later visits are offline-fast.
203
+
204
+ `decider.info` reports the resolved `{ model, revision, device, dtype,
205
+ temperature, maxStateTokens, maxLen, pool, configSource }`.
206
+
207
+ ## Model config
208
+
209
+ Defaults come from the model repo's `open_jev_config.json` when present
210
+ (temperature, marker tokens, state cap), else the published reference values.
211
+ To use a different typed-decision export:
212
+
213
+ ```js
214
+ createDecider({ model: "your-org/your-typed-decision-ONNX", revision: "main" });
215
+ ```
216
+
217
+ The export must add the `[STATE]`, `[Q]`, `[OPT]` markers to its tokenizer,
218
+ take `input_ids, attention_mask, seg, pair_q, pair_opt`, and return per-pair
219
+ `logits`. `typedMarkerIds()` fails loudly when a repo is missing them.
220
+
221
+ ## Multiple families (registry)
222
+
223
+ `jev-web` is a multi-family runtime: `createDecisionRuntime({ family, fallback })`
224
+ tries the requested family and then any fallbacks, returning the shared contract
225
+ `{ info, decide(state, questions), dispose? }`:
226
+
227
+ ```js
228
+ import { createDecisionRuntime, listDecisionFamilies } from "jev-web";
229
+
230
+ listDecisionFamilies(); // ["open-jev", "laya", "strands-decider", "bekko", "decision2", "kev", "vega"]
231
+ const runtime = await createDecisionRuntime({
232
+ family: "strands-decider",
233
+ fallback: ["laya", "open-jev"],
234
+ onFallback: ({ family, error }) => console.warn(family, error),
235
+ });
236
+ const { answers, truncated, length, prompts, timings } = await runtime.decide(state, questions);
237
+ ```
238
+
239
+ Third-party families plug in without touching core:
240
+
241
+ ```js
242
+ registerDecisionFamily("my-model", {
243
+ defaults: { model: "org/my-typed-decision-ONNX", revision: "main" },
244
+ create: async (opts) => ({ info, decide, dispose }),
245
+ });
246
+ ```
247
+
248
+ `decide()` returns `prompts` (decoded input sequence(s), for audit/debug) and
249
+ `timings` (per-stage ms) alongside the answers.
250
+
251
+ ## Laya family (ModernBERT encoder + typed head)
252
+
253
+ `createLayaDecider(options)` runs the [Laya](https://huggingface.co/convaiinnovations/laya)
254
+ System-1 decision models, whose browser export is a **two-file q8 ONNX split**
255
+ (encoder + head) rather than one fused graph:
256
+
257
+ ```js
258
+ import { createLayaDecider } from "jev-web";
259
+
260
+ const decider = await createLayaDecider({
261
+ // defaults to alfred361/laya-typed-decisions-web-q8 @ pinned revision
262
+ wasmPaths: "/assets/", // self-hosted ORT wasm (optional)
263
+ onProgress: (p) => console.log(p.phase, p.loaded, p.total),
264
+ });
265
+ const { answers } = await decider.decide(stateText, questions);
266
+ ```
267
+
268
+ Protocol implemented here (mirrors the reference `laya` Python runtime):
269
+
270
+ - sequence `[CLS] <type> question: <instructions> [SEP] [MASK]opt0 … [SEP] <state> [SEP]`,
271
+ `marker_pos` at each option's `[MASK]`, `qtype` 0 choice / 1 score / 2 noul;
272
+ - encoder `input_ids, attention_mask → hidden`; head
273
+ `hidden, attention_mask, marker_pos, marker_mask, qtype → logits`;
274
+ - per-question calibration from `rl_agent_config.json`
275
+ (`temperature` + `temperature_by_options` buckets, clamped to [0.5, 5]).
276
+
277
+ Notes:
278
+ - the q8 web head is **batch-1 only**, so `decide()` batches the encoder and
279
+ runs the head once per question;
280
+ - `choice` supports at most `max_prefixes` options (6 for the typed-decisions
281
+ export);
282
+ - weights download once (~520 MB q8) and are browser-cached.
283
+
284
+ Lower-level helpers are exported for custom runtimes:
285
+ `buildLayaSequence`, `collateLayaItems`, `layaAnswersFromLogits`,
286
+ `layaTokenizerAdapter`, `renderLayaOptions`, `layaTempBucket`, `clampTemperature`.
287
+
288
+ ## Strands Decider family (pointer head over a Qwen3.5 or Gemma 4 torso)
289
+
290
+ `createStrandsDecider(options)` runs
291
+ [Strands Decider 2B](https://strandsagents.com/blog/introducing-strands-decider/),
292
+ a decision model: a Qwen3.5 or Gemma 4 torso (LoRA merged) with its LM head
293
+ replaced by a pointer head that scores each option from its own hidden state
294
+ against the `<answer>` position. One fused ONNX graph, one forward pass for the
295
+ whole batch:
296
+
297
+ ```js
298
+ import { createStrandsDecider } from "jev-web";
299
+
300
+ const decider = await createStrandsDecider({
301
+ // defaults to onnx-community/strands-decider-2B-hobson-v19-ONNX @ pinned
302
+ // revision, q8 (onnx/model_quantized.onnx)
303
+ onProgress: (p) => console.log(p.phase, p.loaded, p.total),
304
+ });
305
+ const { answers } = await decider.decide(stateText, questions);
306
+ ```
307
+
308
+ Protocol implemented here (mirrors the reference Python runtime):
309
+
310
+ - prompt `<state>…</state><question type="{noul|choice|score}">…<options>\n1. …\n</options></question><answer>`;
311
+ - graph `input_ids, attention_mask, answer_pos, option_pos → logits`, where
312
+ `answer_pos` is each row's last real token (the `<answer>` pooling position)
313
+ and `option_pos` carries each option line's last-token position (padded with
314
+ -1); the prompt is tokenised in chunks at pre-tokeniser piece boundaries so
315
+ those positions are exact with no offset mapping;
316
+ - choice questions accept `criteria` as `{label: description}` (mirroring the
317
+ reference schema); noul slots are fixed `false`/`true` with overridable
318
+ rubric descriptions;
319
+ - the question gets first claim on the 4096-token window (up to 75%), the state
320
+ is truncated from the right into what remains, and an over-long question is
321
+ truncated from the front, keeping the options and `<answer>`;
322
+ - **calibration comes from the checkpoint**, not from this package: the loader
323
+ reads `strands_decider_config.json` (`hobson_config.json` on v19) from the
324
+ model repo at start-up and applies its `temperature` / `temperature_by_kind`
325
+ / `ordinal_smoothing` / `max_length`. A repo that ships no config falls back
326
+ to the pinned reference values (the current pin's temperatures: `noul` 0.911
327
+ / `choice` 0.734 / `score` 1.328). `decider.info.configSource` names the file
328
+ that was used, or `null` for the pinned defaults;
329
+ - explicit `temperature` / `temperaturesByKind` / `ordinalSmoothing` / `maxLen`
330
+ options still override the checkpoint, which overrides the pinned values;
331
+ `calibrationModel` points the config load at another repo (e.g. the source
332
+ checkpoint when your own export predates config shipping);
333
+ - `score` confidence is ordinal (spread-based, with the training smoothing
334
+ floor), not max-probability.
335
+
336
+ Checkpoint compatibility (verified against the 2026-10-09 drop):
337
+
338
+ | checkpoint | params / q8 ONNX | browser? |
339
+ |---|---|---|
340
+ | `2B-hobson-v19` (current pin) | ~2.0B / 1.8 GB | ✅ shipped |
341
+ | `2B-qwen3.5-v1-2610`, `E2B-gemma4-v1-2610` | ~2.1B / ~2.1 GB | ✅ export-ready (E2B: see caveat below) |
342
+ | `E4B-gemma4-v1-2610` | ~5.7B / ~5.7 GB | ⚠️ opt-in heavy only |
343
+ | `12B-gemma4-v1-2610` | ~13.3B / ~13 GB | ❌ too heavy |
344
+ | `26B-A4B-gemma4-v1-2610` (MoE, 128 experts) | ~26B / ~26 GB | ❌ too heavy |
345
+
346
+ A new browser-eligible checkpoint needs an ONNX export plus a model/revision
347
+ repin — no runtime code change (`export/strands-decider/` documents the
348
+ pipeline and its browser-feasibility gate). Named checkpoints are addressed
349
+ through `STRANDS_VARIANTS` — temperatures are never baked into a variant,
350
+ they ship with the checkpoint:
351
+
352
+ ```js
353
+ const decider = await createStrandsDecider({ variant: "qwen3.5-v1" }); // the newer Qwen torso
354
+ const decider2 = await createStrandsDecider(); // hobson-v19, the default
355
+ ```
356
+
357
+ `qwen3.5-v1` currently ships the exact fp16 graph (~3.7 GB) because dynamic
358
+ q8 fails shape-inference on this graph's export metadata and the weight-only
359
+ path drifts on the GatedDeltaNet delta rule (see
360
+ `export/strands-decider/RESUME.md`), so it stays an opt-in until a quantized
361
+ build lands. Caveat: the E2B model card reports
362
+ a faulty GPU kernel path for *multi-question* yes/no answers before an upstream
363
+ attention fix; this runtime batches every question into one forward pass, so
364
+ validate multi-question parity before pinning E2B as a default.
365
+
366
+ Lower-level helpers are exported for custom runtimes:
367
+ `renderStrandsQuestion`, `renderStrandsState`, `renderStrandsContent`,
368
+ `buildStrandsBatch`, `collateStrandsItems`, `strandsOptionTokenIndices`,
369
+ `strandsAnswersFromLogits`, `strandsChoiceConfidence`, `strandsScoreConfidence`,
370
+ `parseStrandsCalibration`, `loadStrandsCalibration`.
371
+
372
+ ## Bekko family (Ettin cross-encoder)
373
+
374
+ `createBekkoDecider(options)` runs
375
+ [Bekko System One v0](https://github.com/hotchpotch/bekko-system-one), a
376
+ cross-encoder that scores each (query, candidate) pair: the query renders as
377
+ `Instruction: {instruction}\nState: {state}` and every option becomes a
378
+ `Candidate: {id}: {description}` document. One ONNX graph
379
+ (`prefix_ids, prefix_mask, doc_ids, doc_mask, owners → logits [N, tasks]`),
380
+ one forward per question, with candidates batched by the reference
381
+ attention budget:
382
+
383
+ ```js
384
+ import { createBekkoDecider } from "jev-web";
385
+
386
+ const decider = await createBekkoDecider({
387
+ // defaults to hotchpotch/bekko-system-one-v0-17m @ pinned revision,
388
+ // onnx_browser/ (29 MB)
389
+ onProgress: (p) => console.log(p.phase, p.loaded, p.total),
390
+ });
391
+ // or pick a larger sibling:
392
+ // await createBekkoDecider({ variant: "68m" }); // 196 MB
393
+ // await createBekkoDecider({ variant: "400m" }); // 1.43 GB, #2 on the S1MB leaderboard
394
+ const { answers } = await decider.decide(stateText, questions);
395
+ ```
396
+
397
+ Protocol implemented here (mirrors the reference browser runtime,
398
+ `browser/src/core.js` + `decision.js`, MIT):
399
+
400
+ - query `[cls] Instruction: …\nState: … [sep]`; instruction and state split the
401
+ 4096-token budget 50/50 with spillover; candidates truncate to 2048 tokens;
402
+ - choice questions accept `criteria` as `{label: description}`; noul slots are
403
+ fixed `true`/`false` with authored meanings (overridable via criteria);
404
+ - score levels map to numeric candidate values `0..n-1`; the answer is the
405
+ expected value plus the argmax level and a normalised score.
406
+
407
+ Larger siblings are available via the `variant` option (`"17m"`, `"68m"`,
408
+ `"400m"`) or the exported model/revision constants: `BEKKO_68M_MODEL` /
409
+ `BEKKO_68M_REVISION` and `BEKKO_400M_MODEL` / `BEKKO_400M_REVISION` (same
410
+ `onnx_browser/` layout). The `BEKKO_VARIANTS` map carries each size's pinned
411
+ revision and download size.
412
+
413
+ Lower-level helpers are exported for custom runtimes:
414
+ `renderBekkoContent`, `renderBekkoRequest`, `tokenizeBekkoRequest`,
415
+ `collateBekkoDocs`, `bekkoSoftmax`, `bekkoAnswerFromLogits`.
416
+
417
+ ## Decision 2.0 family (Qwen3 + candidate head)
418
+
419
+ `createDecision2Decider(options)` runs
420
+ [vLLM Semantic Router Decision 2.0](https://huggingface.co/collections/vllm-sr/decision-20),
421
+ a Qwen3 fine-tune with a candidate head: one prompt per question, each option
422
+ rendered as a `{"description": …, "key": …}` JSON segment. One ONNX graph
423
+ (`input_ids, attention_mask, answer_pos, option_pos → logits [B, K]`), one
424
+ forward pass for the whole batch:
425
+
426
+ ```js
427
+ import { createDecision2Decider } from "jev-web";
428
+
429
+ const decider = await createDecision2Decider({
430
+ // defaults to onnx-community/Decision-2.0-Kai-0.6B-ONNX @ pinned revision,
431
+ // q8 (onnx/model_quantized.onnx, 0.61 GB)
432
+ onProgress: (p) => console.log(p.phase, p.loaded, p.total),
433
+ });
434
+ // or pick a larger sibling:
435
+ // await createDecision2Decider({ variant: "eos" }); // 0.8B, 16k context
436
+ // await createDecision2Decider({ variant: "sol" }); // 2B, 16k context
437
+ const { answers } = await decider.decide(stateText, questions);
438
+ ```
439
+
440
+ Protocol implemented here (mirrors the reference `decision2` runtime):
441
+
442
+ - segments `Context:` / `Task type:` / `Question:` / `Options:` / one
443
+ `<option>` JSON segment per option / the `Decision:` suffix, each tokenized
444
+ separately so `option_pos` (last token of each `</option>`) and `answer_pos`
445
+ (last token) are exact — validated 11/11 against the reference fixtures;
446
+ - choice questions accept `criteria` as `{label: description}` (empty
447
+ descriptions encode as `null`); noul slots are fixed `false`/`true`;
448
+ - score questions add the per-level `score_bias` from `config.json` before the
449
+ softmax (temperature 1).
450
+
451
+ Larger siblings are available via the `variant` option (`"kai"`, `"eos"`,
452
+ `"sol"`) or the exported model/revision constants: `DECISION2_EOS_MODEL` /
453
+ `DECISION2_EOS_REVISION` (0.8B) and `DECISION2_SOL_MODEL` /
454
+ `DECISION2_SOL_REVISION` (2B). The `DECISION2_VARIANTS` map carries each
455
+ variant's pinned revision and context size.
456
+
457
+ Lower-level helpers are exported for custom runtimes:
458
+ `renderDecision2Content`, `renderDecision2Options`, `buildDecision2Segments`,
459
+ `tokenizeDecision2Segments`, `decision2Softmax`, `decision2AnswerFromLogits`.
460
+
461
+ ## Kev family (Qwen3 + pointer head)
462
+
463
+ `createKevDecider(options)` runs
464
+ [Kev](https://huggingface.co/jaredpalmer/kev-0.6b) (Apache-2.0), a typed
465
+ decision model: one state and any number of typed questions are packed into a
466
+ single sequence with five delimiter tokens; a block-causal mask derived
467
+ in-graph lets every question see the state and itself only, and a pointer head
468
+ scores each option's `</opt>` token against its question's `<decide>` token.
469
+ One ONNX graph (`input_ids, attention_mask → logits [B, seq]`), one forward
470
+ pass for the whole batch — read the logit at each option's `</opt>` position
471
+ and softmax within the question:
472
+
473
+ ```js
474
+ import { createKevDecider } from "jev-web";
475
+
476
+ const decider = await createKevDecider({
477
+ // defaults to onnx-community/kev-0.6b-ONNX @ pinned revision,
478
+ // onnx/model_q4f16.onnx (335 MB)
479
+ onProgress: (p) => console.log(p.phase, p.loaded, p.total),
480
+ });
481
+ // or pick the larger sibling:
482
+ // await createKevDecider({ variant: "4b" }); // 1.07 GB, #29 on the Decision Index
483
+ const { answers } = await decider.decide(stateText, questions);
484
+ ```
485
+
486
+ Protocol implemented here (mirrors the reference JS in the model README):
487
+
488
+ - delimiters `<|fim_prefix|>` (state), `<|fim_middle|>` (question),
489
+ `<|box_start|>` / `<|box_end|>` (option), `<|fim_suffix|>` (decide); ids
490
+ derived through the tokenizer, limits from `config.json`'s `kev` block
491
+ (8192 state / 8192 branch tokens, 255 options);
492
+ - caller text is escaped (`<|name|>` → `<¦name¦>`) so it can never forge a
493
+ delimiter;
494
+ - choice questions accept `criteria` as `{label: description}`, rendered as
495
+ `label: description`; noul defaults to `no` / `yes`.
496
+
497
+ The `KEV_VARIANTS` map (`"0.6b"`, `"4b"`) carries each size's pinned revision
498
+ and download size. Lower-level helpers are exported for custom runtimes:
499
+ `escapeKevText`, `packKevSequence`.
500
+
501
+ ## Vega family (frozen Qwen3.5 backbone + latent decision-physics engine)
502
+
503
+ `createVegaDecider(options)` runs [Vega 0.8B](https://huggingface.co/nandakishorm/vega-08b-public-intents)
504
+ (by the Laya author): the language model is frozen and never trained — a 0.8B
505
+ Qwen3.5 torso is read once (layers 13 and 19, span means pooled in-graph) and
506
+ a 14M-parameter physics engine on top does the deciding. Each question drops a
507
+ particle into a latent space whose landscape holds one Gaussian well per
508
+ option; a damped symplectic integrator with a selective-SSM thermostat runs
509
+ up to 12 steps (settled particles freeze), and the Boltzmann occupancy of the
510
+ wells is the answer distribution:
511
+
512
+ ```js
513
+ import { createVegaDecider } from "jev-web";
514
+
515
+ const decider = await createVegaDecider({
516
+ // defaults to warsang/vega-08b-web (revision pinned once the export repo
517
+ // lands — export/vega/05_upload.py prints the sha for VEGA_DEFAULT_REVISION)
518
+ onProgress: (p) => console.log(p.phase, p.loaded, p.total),
519
+ });
520
+ const { answers } = await decider.decide(stateText, questions);
521
+ ```
522
+
523
+ Protocol implemented here (mirrors the reference `vega_common.py` / `vega_api.py`):
524
+
525
+ - prompt `[BOS] Observation:\n<state>\n\nMeasurement (choice|score|boolean): <instructions>\nPossible outcomes:\n* <option> …\nOutcome:` — the state comes first so its span never sees the question
526
+ (causal attention), and short states get a query-first prefix so the frozen
527
+ model reads knowing what is asked;
528
+ - two ONNX graphs: the frozen torso emits per-segment span means for the
529
+ state/question/final-token slots, every option, and 16 equal state chunks
530
+ (the engine's memory reader) from the one causal read; the engine graph
531
+ (unrolled integrator; the settle-freeze math is identical to the
532
+ reference's early exit) emits raw logits plus the particle physics;
533
+ - calibration is **thermal and checkpoint-driven**: `vega_config.json`'s
534
+ `calibration.theta` maps the particle's physical state (residual kinetic
535
+ energy, distance to the nearest well, steps used, option count) to a
536
+ temperature per question type; `decider.info.calibrationSource` names the
537
+ file used;
538
+ - answers carry the full distribution plus Vega's reliability machinery:
539
+ a split-conformal `predictionSet`, an `abstain` flag at the chosen risk
540
+ (`alpha` conformal level, `risk` abstention level, both per question type),
541
+ an `unbound` out-of-distribution flag (particle ended far from every
542
+ well), `settleSteps`, and the final `energy`;
543
+ - jev-web's `noul` maps to Vega's `boolean` (fixed `no` / `yes` slots);
544
+ `choice` accepts `criteria` as `{label: description}`, and `score` uses the
545
+ option texts as levels.
546
+
547
+ Caveats: the checkpoint's gated task adapters are not shipped in v1 — the
548
+ runtime uses the base engine, which is the source repo's default; the torso
549
+ ships as the exact fp16 graph (~1.5 GB, the weight class of the strands 2B q8
550
+ default) because q8 on this graph drifts on the GatedDeltaNet delta rule,
551
+ with the measured q8 experiment available as an opt-in file; and the prompt
552
+ window is the trained 73,728 tokens, so pass `maxLen` to cap a browser's
553
+ read. The browser export repo (`warsang/vega-08b-web`, ~1.7 GB) is uploaded
554
+ by `export/vega/05_upload.py`; the revision lands in `VEGA_DEFAULT_REVISION`.
555
+
556
+ Lower-level helpers are exported for custom runtimes:
557
+ `buildVegaPrompt`, `packVegaBatch`, `packVegaChunks`, `vegaQuestionDef`,
558
+ `vegaOptionTexts`, `vegaThermalFeatures`, `vegaThermalProbabilities`,
559
+ `vegaAnswers`, `vegaOptionLabels`.
560
+
561
+ ## Worker / thread
562
+
563
+ `jev-web` does not bundle a worker, so hosts can create one the way their
564
+ bundler prefers:
565
+
566
+ ```js
567
+ // host code
568
+ const worker = new Worker(new URL("./my-worker.mjs", import.meta.url), { type: "module" });
569
+ // my-worker.mjs
570
+ import { createDecider } from "jev-web";
571
+ ```
572
+
573
+ ## API
574
+
575
+ - `createDecider(options)` → open-jev family: `{ info, decide, decideMany, dispose }`
576
+ - `createLayaDecider(options)` → Laya family: `{ info, decide, dispose }` (same `decide` contract)
577
+ - `createStrandsDecider(options)` → Strands Decider family: `{ info, decide, dispose }` (same `decide` contract)
578
+ - `normalizeQuestions`, `buildDecisionInput`, `typedMarkerIds`, `answersFromScores`,
579
+ `softmaxWithTemperature`, `resolveDevice`, `resolveDtype`, `detectWebGPU`,
580
+ `loadModelConfig`, defaults (model/revision/temperature/caps).
581
+
582
+ ## Demos
583
+
584
+ - **Live demo** — <https://warsang.github.io/jev-web/> (built from `demo/`, deployed on every push)
585
+ - **Local playground** — `npm run dev` → <http://localhost:5188>. Same ideas, plus
586
+ COOP/COEP headers that unlock `SharedArrayBuffer` for the threaded
587
+ onnxruntime-web build (GitHub Pages cannot set those headers, so the deployed
588
+ demo runs single-threaded).
589
+ - **Verify a change visually** — `node tools/shot.mjs jobs.json` drives headless
590
+ Chrome over CDP, waits on a real DOM condition, and reports the page's own
591
+ console errors. The GIF in this README was built with it. It exists because a
592
+ screenshot alone cannot tell you a promise is silently rejecting.
593
+
594
+ ## Limits
595
+
596
+ - Typed-decision models answer **only** by picking among your options; they do
597
+ not explain or generate text.
598
+ - Accuracy is domain-bound: questions and states unlike the model's training
599
+ domain are out of distribution. Measure before relying on any answer.
600
+ - English, 512-token sequence (state capped), one state per forward pass.
601
+ - **WebGPU support is uneven.** A `navigator.gpu` adapter existing is not
602
+ evidence that inference will run: onnxruntime-web compiles WGSL per subgraph,
603
+ and Firefox currently rejects the reference DeBERTa-v3 export's `Clip`
604
+ subgraph (`Failed to create a WebGPU compute pipeline: ShaderModule with
605
+ 'Clip' label is invalid`, surfaced as `failed to call OrtRun()`). The live
606
+ demo detects that signature and rebuilds on WASM, which works but is much
607
+ slower. If you hit it in your own app, fall back explicitly rather than
608
+ trusting adapter detection:
609
+
610
+ ```js
611
+ let decider;
612
+ try {
613
+ decider = await createDecider({ device: "webgpu" });
614
+ await decider.decide("warmup", [{ type: "noul", instructions: "Ready?" }]);
615
+ } catch {
616
+ decider = await createDecider({ device: "wasm", dtype: "q4" });
617
+ }
618
+ ```
619
+
620
+ That warm-up call is the point: probe with **inference**, not with
621
+ `requestAdapter()`.
622
+
623
+ ## License
624
+
625
+ MIT (runtime code). Model weights keep their own license (the reference export
503
626
  is Apache-2.0).