jev-web 0.3.3 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +413 -201
- package/docs/demo-negation.png +0 -0
- package/docs/demo-typed.png +0 -0
- package/docs/demo.gif +0 -0
- package/package.json +60 -55
- package/src/answers.mjs +36 -36
- package/src/bekko.mjs +391 -0
- package/src/config.mjs +65 -65
- package/src/encode.mjs +83 -83
- package/src/index.mjs +83 -42
- package/src/laya.mjs +630 -625
- package/src/questions.mjs +39 -39
- package/src/registry.mjs +129 -102
- package/src/session.mjs +151 -151
- package/src/strands.mjs +535 -0
package/README.md
CHANGED
|
@@ -1,201 +1,413 @@
|
|
|
1
|
-
# jev-web
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
##
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
);
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
//
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
1
|
+
# jev-web
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/jev-web)
|
|
4
|
+
[](./LICENSE)
|
|
5
|
+
[](https://github.com/warsang/jev-web/actions/workflows/test.yml)
|
|
6
|
+
|
|
7
|
+
Run **typed-decision models** in the browser. One *state* text
|
|
8
|
+
plus any number of typed *questions* (`choice` / `score` / `noul`) go in; a
|
|
9
|
+
calibrated probability distribution per question comes back from a **single
|
|
10
|
+
forward pass**. No server, no API keys, no network calls after the weights are
|
|
11
|
+
cached.
|
|
12
|
+
|
|
13
|
+
Three model families ship today — open-jev (fused DeBERTa graph), Laya
|
|
14
|
+
(ModernBERT encoder + typed head), Strands Decider 2B (Qwen torso + pointer
|
|
15
|
+
head), and Bekko System One v0 (Ettin cross-encoder) — behind one registry:
|
|
16
|
+
`createDecisionRuntime({ family, fallback })`.
|
|
17
|
+
|
|
18
|
+
This is a generic runtime, not a classifier for any particular domain. You bring
|
|
19
|
+
the state text and the questions; the model answers by choosing among the options
|
|
20
|
+
you give it.
|
|
21
|
+
|
|
22
|
+
## ▶ Live demo
|
|
23
|
+
|
|
24
|
+
**https://warsang.github.io/jev-web/**
|
|
25
|
+
|
|
26
|
+
An interactive playground that runs this exact package: type a state, add typed
|
|
27
|
+
questions, and watch the model and a hand-written keyword baseline answer
|
|
28
|
+
side by side. It also counts your actual outbound requests, so the
|
|
29
|
+
"offline / private / no server" claim is checkable rather than asserted.
|
|
30
|
+
|
|
31
|
+

|
|
32
|
+
|
|
33
|
+
<sub>The frames are captured from the live demo at
|
|
34
|
+
[warsang.github.io/jev-web](https://warsang.github.io/jev-web/) — site → typed
|
|
35
|
+
decisions with latency → the negation case where the rules and the model
|
|
36
|
+
disagree → the comparison matrix.</sub>
|
|
37
|
+
|
|
38
|
+
Three questions, one forward pass, and a hard label from the rule parser that
|
|
39
|
+
could not answer the third question at all:
|
|
40
|
+
|
|
41
|
+

|
|
42
|
+
|
|
43
|
+
## Quickstart
|
|
44
|
+
|
|
45
|
+
```js
|
|
46
|
+
import { createDecider } from "jev-web";
|
|
47
|
+
|
|
48
|
+
const decider = await createDecider(); // ~340 MB of weights, once
|
|
49
|
+
const { answers } = await decider.decide(
|
|
50
|
+
"I was charged twice for the same order.",
|
|
51
|
+
[{ type: "noul", instructions: "The customer is asking for a refund." }],
|
|
52
|
+
);
|
|
53
|
+
|
|
54
|
+
answers[0].noul; // 0.92 — p(yes)
|
|
55
|
+
answers[0].probabilities; // { no: 0.08, yes: 0.92 }
|
|
56
|
+
answers[0].confidence; // 0.92
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
That is the whole integration. The first call downloads the weights and the
|
|
60
|
+
browser caches them; every later call — and every later visit, including
|
|
61
|
+
offline — is local. **Zero dependencies**: `@huggingface/transformers` is an
|
|
62
|
+
optional peer, imported lazily.
|
|
63
|
+
|
|
64
|
+
## What comes back
|
|
65
|
+
|
|
66
|
+
```js
|
|
67
|
+
const { answers } = await decider.decide(
|
|
68
|
+
"I was charged twice for the same order and nobody answers my emails. I want my money back now.",
|
|
69
|
+
[
|
|
70
|
+
{ type: "choice", instructions: "Which product area is the message about?",
|
|
71
|
+
options: ["fees & charges", "refund & dispute", "card", "other"] },
|
|
72
|
+
{ type: "noul", instructions: "The customer is asking for a refund." },
|
|
73
|
+
{ type: "score", instructions: "How negative is the message?",
|
|
74
|
+
options: ["very negative", "negative", "neutral", "positive", "very positive"] },
|
|
75
|
+
],
|
|
76
|
+
);
|
|
77
|
+
|
|
78
|
+
// [
|
|
79
|
+
// { type: "choice", choice: "fees & charges", index: 0,
|
|
80
|
+
// probabilities: { "fees & charges": 0.796, "refund & dispute": 0.169,
|
|
81
|
+
// card: 0.019, other: 0.016 }, confidence: 0.796 },
|
|
82
|
+
// { type: "noul", noul: 0.927,
|
|
83
|
+
// probabilities: { no: 0.073, yes: 0.927 }, confidence: 0.927 },
|
|
84
|
+
// { type: "score", score: 0.866, level: 1,
|
|
85
|
+
// probabilities: { "very negative": 0.336, negative: 0.478, neutral: 0.175,
|
|
86
|
+
// positive: 0.008, "very positive": 0.003 }, confidence: 0.478 },
|
|
87
|
+
// ]
|
|
88
|
+
// timings: { totalMs, inferMs } · length, truncated, prompts
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Three questions, **one** forward pass, and every question carries the full
|
|
92
|
+
distribution — not just a label. That last part is the point: a hard label
|
|
93
|
+
cannot be thresholded, routed by confidence, or second-guessed, and the
|
|
94
|
+
`choice` answer above is a genuine 79.6 / 16.9 split rather than a coin flip you
|
|
95
|
+
find out about later.
|
|
96
|
+
|
|
97
|
+
## jev-web vs. calling a hosted LLM API with `fetch()`
|
|
98
|
+
|
|
99
|
+
| | jev-web | Hosted LLM API via `fetch()` |
|
|
100
|
+
|---|---|---|
|
|
101
|
+
| **Cost per decision** | **$0.00** — the weights are static files | per-token, forever; a busy form multiplies it by every visitor |
|
|
102
|
+
| **Latency** | one local forward pass, measured live on the demo page | typically 0.4–3 s, plus a network round trip |
|
|
103
|
+
| **Network at answer time** | **0 requests** — works in airplane mode | one round trip per call, minimum |
|
|
104
|
+
| **Output shape** | **calibrated distribution per question, every question in one pass** | prose you then have to regex back into a distribution |
|
|
105
|
+
| **Determinism** | same input → same probabilities, always | temperature 0 + structured outputs helps; still not guaranteed |
|
|
106
|
+
| **Privacy** | your text never leaves the device | your users' text is sent to a third party on every call |
|
|
107
|
+
| **Ops burden** | static files on a CDN, no key, no rate limit | key management, quotas, rate limits, uptime, spend alerts |
|
|
108
|
+
| **Ceiling** | only *picks among options you supply* — never writes, never explains | general purpose; will happily invent an option you never offered |
|
|
109
|
+
|
|
110
|
+
**The honest trade.** jev-web is a classifier, not a chatbot. If you need
|
|
111
|
+
free-form answers you need a generator. If you need a routing decision, a score,
|
|
112
|
+
or a yes/no with a confidence you can threshold on, this is the cheaper, faster
|
|
113
|
+
and private one — and it is also *smaller*: a 0.5 B model plus your options
|
|
114
|
+
beats a frontier model plus a parsing layer, on every axis except generality.
|
|
115
|
+
|
|
116
|
+
## Try it before you install it
|
|
117
|
+
|
|
118
|
+
The live demo covers the interesting cases, and the presets are chosen to break
|
|
119
|
+
keyword parsing rather than to flatter the model:
|
|
120
|
+
|
|
121
|
+
| preset | what it shows |
|
|
122
|
+
|---|---|
|
|
123
|
+
| **Banking triage** | easy text — a fair rule can match, the model adds a confidence |
|
|
124
|
+
| **Negation trap** | *"This is not about a refund. My card was stolen…"* — the keyword rule routes to `refund & dispute`; the model routes to `card` (53.3%) and answers the refund question **no** (0.101) |
|
|
125
|
+
| **Contradictory review** | praise and complaint in one sentence; keyword polarity reads the praise |
|
|
126
|
+
| **Mixed intent** | two intents in one state — a hard label must pick one, a distribution does not have to |
|
|
127
|
+
| **Out of domain** | no in-domain signal at all; the rule parser has nothing to say |
|
|
128
|
+
|
|
129
|
+
The negation case is the one worth looking at. The state is *"This is not about
|
|
130
|
+
a refund. My card was stolen in Madrid and the bank refuses to help. I never
|
|
131
|
+
received the money back I asked about."*
|
|
132
|
+
|
|
133
|
+

|
|
134
|
+
|
|
135
|
+
The rule parser finds the word `refund` and routes to **refund & dispute** — the
|
|
136
|
+
exact opposite of what the customer meant. The model routes to **card** (53.3%)
|
|
137
|
+
and answers *"the customer is asking for a refund"* with **no** (0.101). Both
|
|
138
|
+
the bad routing and the inversion come from the same clause, and neither is
|
|
139
|
+
visible in a hard label.
|
|
140
|
+
|
|
141
|
+
The demo also tracks outbound requests through the Performance API: you will see
|
|
142
|
+
~14 requests while the weights load, then **0** for every decision after that.
|
|
143
|
+
|
|
144
|
+
## Install
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
npm i jev-web @huggingface/transformers
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
`@huggingface/transformers` is an optional peer dependency (it is imported
|
|
151
|
+
lazily). Hosts that already ship their own copy can inject it via
|
|
152
|
+
`createDecider({ transformers })`.
|
|
153
|
+
|
|
154
|
+
## Use
|
|
155
|
+
|
|
156
|
+
```js
|
|
157
|
+
import { createDecider, DEFAULT_MODEL, DEFAULT_REVISION } from "jev-web";
|
|
158
|
+
|
|
159
|
+
const decider = await createDecider({
|
|
160
|
+
// defaults to the reference open-jev DeBERTa-v3-large ONNX export,
|
|
161
|
+
// pinned by revision; any repo with the same typed-decision graph works
|
|
162
|
+
onProgress: (p) => console.log(p.phase, p.file, p.progress),
|
|
163
|
+
});
|
|
164
|
+
|
|
165
|
+
const { answers } = await decider.decide(
|
|
166
|
+
"I was charged twice for the same order and nobody answers my emails.",
|
|
167
|
+
[
|
|
168
|
+
{
|
|
169
|
+
type: "choice",
|
|
170
|
+
instructions: "Which product area is the message about?",
|
|
171
|
+
options: ["fees & charges", "refund & dispute", "card", "other"],
|
|
172
|
+
},
|
|
173
|
+
{ type: "noul", instructions: "The customer is asking for a refund." },
|
|
174
|
+
{
|
|
175
|
+
type: "score",
|
|
176
|
+
instructions: "How negative is the message?",
|
|
177
|
+
options: ["very negative", "negative", "neutral", "positive", "very positive"],
|
|
178
|
+
},
|
|
179
|
+
],
|
|
180
|
+
);
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
## Question types
|
|
184
|
+
|
|
185
|
+
| type | options | answer |
|
|
186
|
+
|---|---|---|
|
|
187
|
+
| `choice` | 2–255 labels | `choice` (argmax), `probabilities`, `confidence` |
|
|
188
|
+
| `score` | 2–10 ordered levels | `score` (expected level index, may fall between levels) |
|
|
189
|
+
| `noul` | exactly 2 (defaults `["no","yes"]`) | `noul` (p of the second option = p(yes)), `confidence` |
|
|
190
|
+
|
|
191
|
+
Every answer carries the full distribution. `decide()` also returns
|
|
192
|
+
`{ truncated, length }` — the state is capped (256 tokens by default) and the
|
|
193
|
+
flag tells you when it was cut.
|
|
194
|
+
|
|
195
|
+
## Backends and quantization
|
|
196
|
+
|
|
197
|
+
`device` / `dtype` default to `auto`: WebGPU → `q4f16`, otherwise WASM → `q4`.
|
|
198
|
+
Override explicitly (`fp16`, `fp32`, `q8`, …) when accuracy matters more than
|
|
199
|
+
download size. First load downloads the weights once; transformers.js caches
|
|
200
|
+
them (Cache API / IndexedDB) so later visits are offline-fast.
|
|
201
|
+
|
|
202
|
+
`decider.info` reports the resolved `{ model, revision, device, dtype,
|
|
203
|
+
temperature, maxStateTokens, maxLen, pool, configSource }`.
|
|
204
|
+
|
|
205
|
+
## Model config
|
|
206
|
+
|
|
207
|
+
Defaults come from the model repo's `open_jev_config.json` when present
|
|
208
|
+
(temperature, marker tokens, state cap), else the published reference values.
|
|
209
|
+
To use a different typed-decision export:
|
|
210
|
+
|
|
211
|
+
```js
|
|
212
|
+
createDecider({ model: "your-org/your-typed-decision-ONNX", revision: "main" });
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
The export must add the `[STATE]`, `[Q]`, `[OPT]` markers to its tokenizer,
|
|
216
|
+
take `input_ids, attention_mask, seg, pair_q, pair_opt`, and return per-pair
|
|
217
|
+
`logits`. `typedMarkerIds()` fails loudly when a repo is missing them.
|
|
218
|
+
|
|
219
|
+
## Multiple families (registry)
|
|
220
|
+
|
|
221
|
+
`jev-web` is a multi-family runtime: `createDecisionRuntime({ family, fallback })`
|
|
222
|
+
tries the requested family and then any fallbacks, returning the shared contract
|
|
223
|
+
`{ info, decide(state, questions), dispose? }`:
|
|
224
|
+
|
|
225
|
+
```js
|
|
226
|
+
import { createDecisionRuntime, listDecisionFamilies } from "jev-web";
|
|
227
|
+
|
|
228
|
+
listDecisionFamilies(); // ["open-jev", "laya", "strands-decider", "bekko"]
|
|
229
|
+
const runtime = await createDecisionRuntime({
|
|
230
|
+
family: "strands-decider",
|
|
231
|
+
fallback: ["laya", "open-jev"],
|
|
232
|
+
onFallback: ({ family, error }) => console.warn(family, error),
|
|
233
|
+
});
|
|
234
|
+
const { answers, truncated, length, prompts, timings } = await runtime.decide(state, questions);
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
Third-party families plug in without touching core:
|
|
238
|
+
|
|
239
|
+
```js
|
|
240
|
+
registerDecisionFamily("my-model", {
|
|
241
|
+
defaults: { model: "org/my-typed-decision-ONNX", revision: "main" },
|
|
242
|
+
create: async (opts) => ({ info, decide, dispose }),
|
|
243
|
+
});
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
`decide()` returns `prompts` (decoded input sequence(s), for audit/debug) and
|
|
247
|
+
`timings` (per-stage ms) alongside the answers.
|
|
248
|
+
|
|
249
|
+
## Laya family (ModernBERT encoder + typed head)
|
|
250
|
+
|
|
251
|
+
`createLayaDecider(options)` runs the [Laya](https://huggingface.co/convaiinnovations/laya)
|
|
252
|
+
System-1 decision models, whose browser export is a **two-file q8 ONNX split**
|
|
253
|
+
(encoder + head) rather than one fused graph:
|
|
254
|
+
|
|
255
|
+
```js
|
|
256
|
+
import { createLayaDecider } from "jev-web";
|
|
257
|
+
|
|
258
|
+
const decider = await createLayaDecider({
|
|
259
|
+
// defaults to alfred361/laya-typed-decisions-web-q8 @ pinned revision
|
|
260
|
+
wasmPaths: "/assets/", // self-hosted ORT wasm (optional)
|
|
261
|
+
onProgress: (p) => console.log(p.phase, p.loaded, p.total),
|
|
262
|
+
});
|
|
263
|
+
const { answers } = await decider.decide(stateText, questions);
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
Protocol implemented here (mirrors the reference `laya` Python runtime):
|
|
267
|
+
|
|
268
|
+
- sequence `[CLS] <type> question: <instructions> [SEP] [MASK]opt0 … [SEP] <state> [SEP]`,
|
|
269
|
+
`marker_pos` at each option's `[MASK]`, `qtype` 0 choice / 1 score / 2 noul;
|
|
270
|
+
- encoder `input_ids, attention_mask → hidden`; head
|
|
271
|
+
`hidden, attention_mask, marker_pos, marker_mask, qtype → logits`;
|
|
272
|
+
- per-question calibration from `rl_agent_config.json`
|
|
273
|
+
(`temperature` + `temperature_by_options` buckets, clamped to [0.5, 5]).
|
|
274
|
+
|
|
275
|
+
Notes:
|
|
276
|
+
- the q8 web head is **batch-1 only**, so `decide()` batches the encoder and
|
|
277
|
+
runs the head once per question;
|
|
278
|
+
- `choice` supports at most `max_prefixes` options (6 for the typed-decisions
|
|
279
|
+
export);
|
|
280
|
+
- weights download once (~520 MB q8) and are browser-cached.
|
|
281
|
+
|
|
282
|
+
Lower-level helpers are exported for custom runtimes:
|
|
283
|
+
`buildLayaSequence`, `collateLayaItems`, `layaAnswersFromLogits`,
|
|
284
|
+
`layaTokenizerAdapter`, `renderLayaOptions`, `layaTempBucket`, `clampTemperature`.
|
|
285
|
+
|
|
286
|
+
## Strands Decider family (Qwen torso + pointer head)
|
|
287
|
+
|
|
288
|
+
`createStrandsDecider(options)` runs
|
|
289
|
+
[Strands Decider 2B](https://strandsagents.com/blog/introducing-strands-decider/),
|
|
290
|
+
a 2B-parameter decision model: a Qwen3.5-2B torso (LoRA merged) with its LM head
|
|
291
|
+
replaced by a pointer head that scores each option from its own hidden state
|
|
292
|
+
against the `<answer>` position. One fused ONNX graph, one forward pass for the
|
|
293
|
+
whole batch:
|
|
294
|
+
|
|
295
|
+
```js
|
|
296
|
+
import { createStrandsDecider } from "jev-web";
|
|
297
|
+
|
|
298
|
+
const decider = await createStrandsDecider({
|
|
299
|
+
// defaults to onnx-community/strands-decider-2B-hobson-v19-ONNX @ pinned
|
|
300
|
+
// revision, q8 (onnx/model_quantized.onnx)
|
|
301
|
+
onProgress: (p) => console.log(p.phase, p.loaded, p.total),
|
|
302
|
+
});
|
|
303
|
+
const { answers } = await decider.decide(stateText, questions);
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
Protocol implemented here (mirrors the reference Python runtime):
|
|
307
|
+
|
|
308
|
+
- prompt `<state>…</state><question type="{noul|choice|score}">…<options>\n1. …\n</options></question><answer>`;
|
|
309
|
+
- graph `input_ids, attention_mask, answer_pos, option_pos → logits`, where
|
|
310
|
+
`answer_pos` is each row's last real token (the `<answer>` pooling position)
|
|
311
|
+
and `option_pos` carries each option line's last-token position (padded with
|
|
312
|
+
0); the prompt is tokenised in chunks at pre-tokeniser piece boundaries so
|
|
313
|
+
those positions are exact with no offset mapping;
|
|
314
|
+
- choice questions accept `criteria` as `{label: description}` (mirroring the
|
|
315
|
+
reference schema); noul slots are fixed `false`/`true` with overridable
|
|
316
|
+
rubric descriptions;
|
|
317
|
+
- the question gets first claim on the 4096-token window (up to 75%), the state
|
|
318
|
+
is truncated from the right into what remains, and an over-long question is
|
|
319
|
+
truncated from the front, keeping the options and `<answer>`;
|
|
320
|
+
- per-kind calibration from the checkpoint (`noul` 0.911 / `choice` 0.734 /
|
|
321
|
+
`score` 1.328); `score` confidence is ordinal (spread-based, with the
|
|
322
|
+
training smoothing floor), not max-probability.
|
|
323
|
+
|
|
324
|
+
Lower-level helpers are exported for custom runtimes:
|
|
325
|
+
`renderStrandsQuestion`, `renderStrandsState`, `renderStrandsContent`,
|
|
326
|
+
`buildStrandsBatch`, `collateStrandsItems`, `strandsOptionTokenIndices`,
|
|
327
|
+
`strandsAnswersFromLogits`, `strandsChoiceConfidence`, `strandsScoreConfidence`.
|
|
328
|
+
|
|
329
|
+
## Bekko family (Ettin cross-encoder)
|
|
330
|
+
|
|
331
|
+
`createBekkoDecider(options)` runs
|
|
332
|
+
[Bekko System One v0](https://github.com/hotchpotch/bekko-system-one), a
|
|
333
|
+
cross-encoder that scores each (query, candidate) pair: the query renders as
|
|
334
|
+
`Instruction: {instruction}\nState: {state}` and every option becomes a
|
|
335
|
+
`Candidate: {id}: {description}` document. One ONNX graph
|
|
336
|
+
(`prefix_ids, prefix_mask, doc_ids, doc_mask, owners → logits [N, tasks]`),
|
|
337
|
+
one forward per question, with candidates batched by the reference
|
|
338
|
+
attention budget:
|
|
339
|
+
|
|
340
|
+
```js
|
|
341
|
+
import { createBekkoDecider } from "jev-web";
|
|
342
|
+
|
|
343
|
+
const decider = await createBekkoDecider({
|
|
344
|
+
// defaults to hotchpotch/bekko-system-one-v0-17m @ pinned revision,
|
|
345
|
+
// onnx_browser/ (29 MB)
|
|
346
|
+
onProgress: (p) => console.log(p.phase, p.loaded, p.total),
|
|
347
|
+
});
|
|
348
|
+
const { answers } = await decider.decide(stateText, questions);
|
|
349
|
+
```
|
|
350
|
+
|
|
351
|
+
Protocol implemented here (mirrors the reference browser runtime,
|
|
352
|
+
`browser/src/core.js` + `decision.js`, MIT):
|
|
353
|
+
|
|
354
|
+
- query `[cls] Instruction: …\nState: … [sep]`; instruction and state split the
|
|
355
|
+
4096-token budget 50/50 with spillover; candidates truncate to 2048 tokens;
|
|
356
|
+
- choice questions accept `criteria` as `{label: description}`; noul slots are
|
|
357
|
+
fixed `true`/`false` with authored meanings (overridable via criteria);
|
|
358
|
+
- score levels map to numeric candidate values `0..n-1`; the answer is the
|
|
359
|
+
expected value plus the argmax level and a normalised score.
|
|
360
|
+
|
|
361
|
+
Larger siblings are available via the exported model/revision constants:
|
|
362
|
+
`BEKKO_68M_MODEL` / `BEKKO_68M_REVISION` and `BEKKO_400M_MODEL` /
|
|
363
|
+
`BEKKO_400M_REVISION` (68M and 400M, same `onnx_browser/` layout).
|
|
364
|
+
|
|
365
|
+
Lower-level helpers are exported for custom runtimes:
|
|
366
|
+
`renderBekkoContent`, `renderBekkoRequest`, `tokenizeBekkoRequest`,
|
|
367
|
+
`collateBekkoDocs`, `bekkoSoftmax`, `bekkoAnswerFromLogits`.
|
|
368
|
+
|
|
369
|
+
## Worker / thread
|
|
370
|
+
|
|
371
|
+
`jev-web` does not bundle a worker, so hosts can create one the way their
|
|
372
|
+
bundler prefers:
|
|
373
|
+
|
|
374
|
+
```js
|
|
375
|
+
// host code
|
|
376
|
+
const worker = new Worker(new URL("./my-worker.mjs", import.meta.url), { type: "module" });
|
|
377
|
+
// my-worker.mjs
|
|
378
|
+
import { createDecider } from "jev-web";
|
|
379
|
+
```
|
|
380
|
+
|
|
381
|
+
## API
|
|
382
|
+
|
|
383
|
+
- `createDecider(options)` → open-jev family: `{ info, decide, decideMany, dispose }`
|
|
384
|
+
- `createLayaDecider(options)` → Laya family: `{ info, decide, dispose }` (same `decide` contract)
|
|
385
|
+
- `createStrandsDecider(options)` → Strands Decider family: `{ info, decide, dispose }` (same `decide` contract)
|
|
386
|
+
- `normalizeQuestions`, `buildDecisionInput`, `typedMarkerIds`, `answersFromScores`,
|
|
387
|
+
`softmaxWithTemperature`, `resolveDevice`, `resolveDtype`, `detectWebGPU`,
|
|
388
|
+
`loadModelConfig`, defaults (model/revision/temperature/caps).
|
|
389
|
+
|
|
390
|
+
## Demos
|
|
391
|
+
|
|
392
|
+
- **Live demo** — <https://warsang.github.io/jev-web/> (built from `demo/`, deployed on every push)
|
|
393
|
+
- **Local playground** — `npm run dev` → <http://localhost:5188>. Same ideas, plus
|
|
394
|
+
COOP/COEP headers that unlock `SharedArrayBuffer` for the threaded
|
|
395
|
+
onnxruntime-web build (GitHub Pages cannot set those headers, so the deployed
|
|
396
|
+
demo runs single-threaded).
|
|
397
|
+
- **Verify a change visually** — `node tools/shot.mjs jobs.json` drives headless
|
|
398
|
+
Chrome over CDP, waits on a real DOM condition, and reports the page's own
|
|
399
|
+
console errors. The GIF in this README was built with it. It exists because a
|
|
400
|
+
screenshot alone cannot tell you a promise is silently rejecting.
|
|
401
|
+
|
|
402
|
+
## Limits
|
|
403
|
+
|
|
404
|
+
- Typed-decision models answer **only** by picking among your options; they do
|
|
405
|
+
not explain or generate text.
|
|
406
|
+
- Accuracy is domain-bound: questions and states unlike the model's training
|
|
407
|
+
domain are out of distribution. Measure before relying on any answer.
|
|
408
|
+
- English, 512-token sequence (state capped), one state per forward pass.
|
|
409
|
+
|
|
410
|
+
## License
|
|
411
|
+
|
|
412
|
+
MIT (runtime code). Model weights keep their own license (the reference export
|
|
413
|
+
is Apache-2.0).
|
|
Binary file
|
|
Binary file
|
package/docs/demo.gif
ADDED
|
Binary file
|