laya-system-one 1.1.0-alpha.1 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. package/README.md +393 -221
  2. package/models/model.manifest.json +15 -14
  3. package/models/rl_agent_config.json +26 -26
  4. package/package.json +25 -23
  5. package/src/agent.js +2 -0
  6. package/src/laya-native.js +31 -6
  7. package/src/server.js +194 -179
package/README.md CHANGED
@@ -1,299 +1,471 @@
1
- # Laya System-One
1
+ # Laya System-One ⚡
2
2
 
3
- **System 1 decision engine** — a multilingual INT8 transformer that answers typed questions (choice / score / yes-no) about any text, in milliseconds, offline.
3
+ [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
4
+ [![Runtime](https://img.shields.io/badge/Runtime-Node.js%20%7C%20Bun%20%7C%20Browser-green.svg)]()
5
+ [![TypeSafe Jev](https://img.shields.io/badge/Wire%20Protocol-TypeSafe%20Jev%20Compatible-orange.svg)]()
6
+ [![Dependencies](https://img.shields.io/badge/dependencies-0-brightgreen.svg)]()
7
+ [![Built on Laya](https://img.shields.io/badge/Built%20on-Laya%20by%20Convai%20Innovations-8A2BE2.svg)](https://github.com/NandhaKishorM/laya)
4
8
 
5
- Wire-compatible with the **TypeSafe Jev** `/v1/systemone` protocol: send state + typed questions, get structured decisions back.
9
+ > **A fast, self-contained decision engine. Give it any text and a set of typed questions, and it answers them — offline, in milliseconds, in over 100 languages. Drop-in compatible with the TypeSafe Jev API (`POST /v1/systemone`).**
10
+
11
+ Runs entirely on your machine. No Python, no PyTorch, no API keys, no cloud
12
+ calls at inference time. One `npm install` and it works.
13
+
14
+ > **Built on [Laya](https://github.com/NandhaKishorM/laya)** by
15
+ > [Convai Innovations](https://huggingface.co/convaiinnovations) — a
16
+ > community project. The model is theirs; this package makes it run in
17
+ > Node.js, Bun and the browser with no Python in the loop. See
18
+ > [Credits](#-credits).
19
+
20
+ ---
21
+
22
+ ## 🌟 Why Laya System-One?
23
+
24
+ - 🔒 **100% Offline:** Nothing leaves your machine. Ideal for corporate
25
+ intranets, edge servers and privacy-sensitive workflows.
26
+ - ⚡ **Fast:** ~40 ms per question on a warm engine, measured on every platform
27
+ we ship for.
28
+ - 🔄 **TypeSafe Jev Compatible:** Drop-in `POST /v1/systemone`. Point an
29
+ existing Jev client at it and it just works.
30
+ - 🌍 **Multilingual:** Understands English, Portuguese, Spanish, German,
31
+ French, Chinese, Japanese and 100+ more, out of the box.
32
+ - 💻 **Node.js, Bun and Browsers:** Native binary on Node and Bun, WebAssembly
33
+ in the browser.
34
+ - 📦 **Zero Dependencies:** `dependencies` is empty. Nothing to compile,
35
+ nothing to install system-wide, nothing to keep patched.
36
+ - 🧩 **Two Ways to Run It:** As a local HTTP service via the CLI, or in-process
37
+ for zero network overhead.
38
+
39
+ ---
40
+
41
+ ## 📦 Installation
6
42
 
7
43
  ```bash
44
+ # npm
8
45
  npm install laya-system-one
46
+
47
+ # bun
48
+ bun add laya-system-one
49
+
50
+ # pnpm
51
+ pnpm add laya-system-one
9
52
  ```
10
53
 
11
- ```js
12
- import { Laya } from 'laya-system-one';
54
+ The right engine for your machine is installed automatically. The model
55
+ (~324 MB) is fetched once on first use and cached.
13
56
 
14
- const laya = await Laya.load(); // model is acquired on first use (once)
15
- const out = await laya.predict(
16
- 'We were billed twice on the March invoice and want a refund.',
17
- {
18
- department: {
19
- type: 'choice',
20
- instructions: 'Which department should handle this?',
21
- criteria: { billing: 'refunds and invoices', tech: 'bugs', sales: 'upgrades' }
22
- },
23
- churn: { type: 'noul', instructions: 'Is the user at churn risk?', threshold: 0.5 },
24
- severity: { type: 'score', instructions: 'Urgency?', criteria: ['low', 'mid', 'high'] }
25
- }
26
- );
57
+ ---
27
58
 
28
- console.log(out.answers.department.choice); // → "billing"
29
- ```
59
+ ## 🚀 Quick Start
30
60
 
31
- Or run it as a service:
61
+ ### 1. Launch the HTTP service
32
62
 
33
63
  ```bash
34
64
  npx laya-system-one --port 8080
35
- curl -X POST http://localhost:8080/v1/systemone \
36
- -H "Content-Type: application/json" \
37
- -d '{"state":"I was charged twice and need a refund.","questions":{"dept":{"type":"choice","instructions":"Which department?","criteria":{"billing":"refunds","tech":"bugs"}}}}'
38
65
  ```
39
66
 
40
- ---
67
+ | Flag | Env | Default | Description |
68
+ | :--- | :--- | :--- | :--- |
69
+ | `--port <number>` | `PORT` | `8080` | Port to bind |
70
+ | `--host <string>` | `HOST` | `0.0.0.0` | Address to bind |
71
+ | `--backend <type>` | `LAYA_BACKEND` | `native` | `native` or `wasm` |
72
+ | `--api-key <token>` | `LAYA_API_KEY` | *(none)* | Require Bearer auth on `/v1/systemone` |
41
73
 
42
- ## What it actually is
74
+ With authentication:
43
75
 
44
- A single ONNX checkpoint (`model.onnx`, ~324 MB, INT8) trained to score answer options for three question types, plus a small runtime that:
76
+ ```bash
77
+ npx laya-system-one --port 8080 --api-key secret-token-xyz
78
+ ```
45
79
 
46
- 1. renders the state and the question options into a prompt,
47
- 2. runs one forward pass,
48
- 3. turns the logits into calibrated probabilities.
80
+ ### 2. Use it in-process (zero network overhead)
49
81
 
50
- The runtime is JavaScript (Node/Bun/browser) and executes that forward pass in exactly two ways — pick one with `--backend` or `LAYA_BACKEND`:
82
+ ```javascript
83
+ import { Laya } from 'laya-system-one';
51
84
 
52
- | backend | what it is | when to use |
53
- |---|---|---|
54
- | `native` **(default)** | the `laya-serve` binary bundled in the package (Rust: Axum + tokenizers + ONNX Runtime, statically linked) | fastest path, zero system dependencies, any OS |
55
- | `wasm` | pure-Rust `tract` compiled to WASM, bundled in the package | browsers and extreme portability |
85
+ // 1. Initialize the engine
86
+ const laya = await Laya.load();
87
+
88
+ // 2. Define the state (string, object, or array)
89
+ const state = {
90
+ customer_id: 'cust_9821',
91
+ message: 'We were charged twice on our March invoice. Please refund the duplicate amount or we will cancel our plan.'
92
+ };
93
+
94
+ // 3. Define typed questions
95
+ const questions = {
96
+ department: {
97
+ type: 'choice',
98
+ instructions: 'Which team should resolve this customer inquiry?',
99
+ criteria: {
100
+ billing: 'Invoices, refunds, and duplicate charges',
101
+ tech_support: 'Software bugs, outages, and error messages',
102
+ sales: 'Upgrades, plan changes, and enterprise contracts'
103
+ }
104
+ },
105
+ urgency: {
106
+ type: 'score',
107
+ instructions: 'Assess the urgency level of this inquiry.',
108
+ criteria: ['Low / routine', 'Moderate', 'Critical / blocking / angry']
109
+ },
110
+ churn_risk: {
111
+ type: 'noul',
112
+ instructions: 'Does this message present an explicit risk of customer churn?',
113
+ threshold: 0.5
114
+ }
115
+ };
56
116
 
57
- There are **no external runtime dependencies**: no `onnxruntime-*`, no `@huggingface/transformers`, nothing to download besides the model. The tokenizer itself is a pure-JS BPE implementation (`src/bpe-tokenizer.js`) verified token-for-token against the reference `tokenizers` crate that the native binary links — so both backends see exactly the same input ids.
117
+ // 4. Evaluate
118
+ const result = await laya.predict(state, questions);
58
119
 
59
- > **Note on WebGPU:** earlier versions advertised WebGPU acceleration. Measured reality: ONNX Runtime's Node WebGPU execution provider falls back to CPU per-op with large overhead — it is consistently *slower* than the native backend. WebGPU is not used server-side.
120
+ console.log(result.answers.department.choice); // -> "billing"
121
+ console.log(result.answers.department.confidence); // -> 1.0
122
+ console.log(result.answers.urgency.score); // -> 1.95
123
+ console.log(result.answers.churn_risk.noul); // -> 0.968
124
+ console.log(result.answers.churn_risk.decision); // -> true
125
+ ```
60
126
 
61
- ---
127
+ ### 3. Serve it from inside your app
62
128
 
63
- ## How the heavy pieces arrive
64
-
65
- The entry package is small on purpose (**~8.5 MB**): source, tokenizer and the
66
- wasm fallback. The binaries (150+ MB for every platform) and the model
67
- (324 MB) are published as separate `@sys-one` packages and installed as
68
- `optionalDependencies`, so npm fetches only what the machine needs:
69
-
70
- | package | selected by | size |
71
- | --- | --- | --- |
72
- | `@sys-one/laya-serve-<os>-<arch>` | npm's `os`/`cpu` fields | 8–26 MB |
73
- | `@sys-one/laya-serve-universal` | fallback for exotic platforms | ~76 MB |
74
- | `@sys-one/laya-model-chunk-00` … `-12` | always (all 13) | ~24 MB each |
75
-
76
- A `linux/x64` machine installs `laya-serve-linux-x64` and nothing else;
77
- a `linux/ppc64` machine matches none of the specific packages and gets the
78
- universal one. No package installs all of them.
79
-
80
- Two things worth knowing about how the selection works, because both were
81
- verified by experiment rather than assumed:
82
-
83
- - **`libc` cannot select.** The field exists but is unreliable (undocumented
84
- shorthand, and real packages have shipped bugs where `--libc=glibc` pulls
85
- the musl build too). So the glibc and musl builds of a Linux arch travel
86
- **together** in one package, and the loader picks at runtime.
87
- - **Exclusions are AND-ed.** `os: ["!darwin", "!win32"]` means "not both",
88
- not "either", so the complement of the specific packages cannot be spelled
89
- out. The universal package lists the exotic cpus and OSes explicitly
90
- instead: skipped wherever a specific package applies, selected everywhere
91
- else.
92
-
93
- The model is assembled from its chunk packages on first use and verified
94
- against `models/model.manifest.json` (sha256 of the model *and* of every
95
- chunk), written atomically — a killed process cannot leave a corrupt model
96
- behind, the next run retries. The order is:
97
-
98
- 1. `LAYA_MODEL_PATH` — explicit path to a `model.onnx` (file or directory);
99
- 2. a `model.onnx` already present in the package's `models/` directory;
100
- 3. `~/.cache/laya-system-one/model.onnx` — previously assembled copy;
101
- 4. the installed `@sys-one/laya-model-chunk-*` packages (the normal path);
102
- 5. the npm registry, if the chunks were not installed as dependencies;
103
- 6. the GitHub Release asset — **opt-in only** (`LAYA_ALLOW_GITHUB_FALLBACK=1`),
104
- kept for compatibility with 1.0.0 installs.
105
-
106
- ### Offline / air-gapped installs
129
+ ```javascript
130
+ import { serve } from 'laya-system-one';
107
131
 
108
- ```bash
109
- LAYA_PREFETCH_MODEL=1 npm install laya-system-one # fetch during install
110
- # or point at a model you already have:
111
- LAYA_MODEL_PATH=/opt/models/model.onnx
112
- # or drop chunk files somewhere and point at them:
113
- LAYA_MODEL_CHUNKS_DIR=/opt/models/chunks
114
- ```
132
+ const srv = await serve({ host: '127.0.0.1', port: 8080, apiKey: 'optional-key' });
115
133
 
116
- ---
134
+ console.log(`Laya server running at ${srv.url}/v1/systemone`);
117
135
 
118
- ## API
136
+ // later:
137
+ await srv.close();
138
+ ```
119
139
 
120
- ### `Laya.load(options)` → `laya`
140
+ ### `Laya.load(options)` options
121
141
 
122
- | option | default | description |
123
- |---|---|---|
124
- | `backend` | `'native'` | `native` \| `wasm` (or env `LAYA_BACKEND`) |
125
- | `modelDir` | `<package>/models` | where `model.onnx` and `tokenizer.json` live |
126
- | `apiKey` | `null` | Bearer token required by the HTTP layer |
142
+ | Option | Default | Description |
143
+ | :--- | :--- | :--- |
144
+ | `backend` | `'native'` | `native` (bundled Rust server) or `wasm` |
145
+ | `modelDir` | the package's `models/` | where `model.onnx` and `tokenizer.json` live |
146
+ | `maxLen` | `2048` | token budget for the state — raise it for long documents (max 8192, see [Long inputs](#long-inputs)) |
147
+ | `apiKey` | `null` | require a Bearer token on the HTTP layer |
127
148
  | `port` / `host` | `0` / `127.0.0.1` | where the native server binds |
149
+ | `threads` | `0` | inference threads (`0` = runtime default) |
128
150
 
129
- ### `laya.predict(state, questions, model?)` → `Promise<Answer>`
151
+ ---
130
152
 
131
- `state` is a string, object or array (serialized as JSON). `questions` is a map of question definitions:
153
+ ## 📡 HTTP API Reference (TypeSafe Jev compatible)
132
154
 
133
- | type | `criteria` | answer |
134
- |---|---|---|
135
- | `choice` | object (label → meaning) or array | `{ type: 'choice', choice, probabilities, confidence }` |
136
- | `score` | array of ordered levels | `{ type: 'score', score, legend, probabilities, confidence }` |
137
- | `noul` | optional `{false, true}` text | `{ type: 'noul', noul, confidence, threshold?, decision? }` |
155
+ ```http
156
+ POST /v1/systemone
157
+ Host: localhost:8080
158
+ Content-Type: application/json
159
+ Authorization: Bearer <API_KEY> [optional unless configured]
160
+ ```
138
161
 
139
- `noul` returns `noul` ∈ [0,1] (probability of *true*). With a `threshold`, a boolean `decision` is added. All fields round to 4 decimals.
162
+ | Parameter | Type | Required | Description |
163
+ | :--- | :--- | :--- | :--- |
164
+ | `state` | `string` \| `object` \| `array` | **Yes** | The context or text being evaluated. |
165
+ | `questions` | `Record<string, Question>` | **Yes** | Map of question keys to typed questions. |
166
+ | `model` | `string` | No | Model name (defaults to `laya-multilingual`, echoed back). |
140
167
 
141
- ### HTTP server
168
+ ### Question types
142
169
 
143
- ```js
144
- import { serve } from 'laya-system-one';
145
- const srv = await serve({ port: 8080, apiKey: process.env.LAYA_API_KEY });
146
- console.log(srv.url); // http://localhost:8080
147
- await srv.close(); // releases the engine and any child process
170
+ **`choice`** — pick one of several options:
171
+
172
+ ```json
173
+ {
174
+ "type": "choice",
175
+ "instructions": "Which department should handle this ticket?",
176
+ "criteria": {
177
+ "billing": "Invoices and credit card transactions",
178
+ "technical": "Software bugs and service disruptions"
179
+ }
180
+ }
148
181
  ```
149
182
 
150
- | endpoint | method | body | response |
151
- |---|---|---|---|
152
- | `/v1/systemone` | `POST` | `{ state, questions, model? }` | `{ model, answers, usage }` |
153
- | `/health` | `GET` | – | `{ status, model, backend, protocol }` |
183
+ **`score`** — place on an ordered scale:
154
184
 
155
- Errors: `401` missing/invalid API key, `422` invalid payload, `404` unknown route, `413` body over 4 MB.
185
+ ```json
186
+ {
187
+ "type": "score",
188
+ "instructions": "Rate the severity of the issue.",
189
+ "criteria": ["Minor cosmetic issue", "Degraded functionality", "Critical full service outage"]
190
+ }
191
+ ```
156
192
 
157
- > The `native` backend runs its own internal HTTP server for the engine;
158
- > `serve()` keeps it on a private loopback port and never exposes it.
193
+ **`noul`** — calibrated yes/no probability:
159
194
 
160
- ---
195
+ ```json
196
+ {
197
+ "type": "noul",
198
+ "instructions": "Does the user explicitly request a refund?",
199
+ "threshold": 0.6
200
+ }
201
+ ```
161
202
 
162
- ## CLI
203
+ ### Example request
163
204
 
164
205
  ```bash
165
- npx laya-system-one --port 8080 --backend native
206
+ curl -X POST http://localhost:8080/v1/systemone \
207
+ -H "Content-Type: application/json" \
208
+ -d '{
209
+ "state": { "text": "Fui cobrado duas vezes na minha fatura. Reembolsem imediatamente." },
210
+ "questions": {
211
+ "dept": {
212
+ "type": "choice",
213
+ "instructions": "Which department should respond?",
214
+ "criteria": { "billing": "Refunds, invoices, and payments", "support": "Technical and product questions" }
215
+ },
216
+ "urgency": {
217
+ "type": "score",
218
+ "instructions": "Urgency rating",
219
+ "criteria": ["Low", "Medium", "High"]
220
+ },
221
+ "refund_demanded": {
222
+ "type": "noul",
223
+ "instructions": "Is the customer requesting a refund?",
224
+ "threshold": 0.5
225
+ }
226
+ }
227
+ }'
228
+ ```
229
+
230
+ ### Example response
231
+
232
+ ```json
233
+ {
234
+ "model": "laya-multilingual",
235
+ "answers": {
236
+ "dept": {
237
+ "type": "choice",
238
+ "choice": "billing",
239
+ "probabilities": { "billing": 1.0, "support": 0.0 },
240
+ "confidence": 1.0
241
+ },
242
+ "urgency": {
243
+ "type": "score",
244
+ "score": 1.9482,
245
+ "legend": { "0": "Low", "1": "Medium", "2": "High" },
246
+ "probabilities": { "0": 0.0011, "1": 0.0496, "2": 0.9493 },
247
+ "confidence": 0.9493
248
+ },
249
+ "refund_demanded": {
250
+ "type": "noul",
251
+ "noul": 0.9852,
252
+ "confidence": 0.9852,
253
+ "threshold": 0.5,
254
+ "decision": true
255
+ }
256
+ },
257
+ "usage": { "input_tokens": 82, "output_tokens": 12 }
258
+ }
166
259
  ```
167
260
 
261
+ ### Healthcheck
262
+
263
+ ```http
264
+ GET /health
168
265
  ```
169
- --port <number> HTTP port (default 8080, or PORT env)
170
- --host <string> bind address (default 0.0.0.0, or HOST env)
171
- --backend <type> native | wasm (default native)
172
- --api-key <string> require Bearer token auth
173
- --help / --version
266
+
267
+ ```json
268
+ {
269
+ "status": "ok",
270
+ "model": "laya-multilingual",
271
+ "version": "1.1.0",
272
+ "protocol": "TypeSafe Jev /v1/systemone compatible"
273
+ }
174
274
  ```
175
275
 
276
+ ### Error codes
277
+
278
+ - `401 Unauthorized` — API key configured, header missing or wrong.
279
+ - `422 Unprocessable Entity` — invalid JSON, or `state`/`questions` missing.
280
+ - `404 Not Found` — unknown route.
281
+ - `413 Payload Too Large` — body over 4 MB.
282
+
176
283
  ---
177
284
 
178
- ## Environment variables
285
+ ## ⚙️ Backends
286
+
287
+ | Backend | How it runs | When to use |
288
+ | :--- | :--- | :--- |
289
+ | **`native`** *(default)* | A self-contained Rust server bundled with the package | The normal choice. Fastest, nothing to install. |
290
+ | **`wasm`** | Pure Rust compiled to WebAssembly, also bundled | Browsers, or platforms with no native build. |
179
291
 
180
- | variable | effect |
181
- |---|---|
182
- | `LAYA_BACKEND` | `native` \| `wasm` |
183
- | `LAYA_MODEL_PATH` | explicit `model.onnx` (file or directory) |
184
- | `LAYA_MODEL_CHUNKS_DIR` | directory with chunk files / chunk packages |
185
- | `LAYA_MODEL_URL` | override the download URL of the model asset |
186
- | `LAYA_CACHE_DIR` | where downloaded models are cached |
187
- | `LAYA_PREFETCH_MODEL` | `1` = fetch the model during `npm install` |
188
- | `LAYA_SKIP_MODEL_DOWNLOAD` | `1` = never download, never hint |
189
- | `LAYA_API_KEY` / `API_KEY` | require `Authorization: Bearer <key>` |
190
- | `LAYA_SERVE_BIN` | explicit path to the `laya-serve` binary |
292
+ Both ship inside the package — nothing is compiled or downloaded at install
293
+ time. Switch with `--backend wasm` or `LAYA_BACKEND=wasm`.
191
294
 
192
295
  ---
193
296
 
194
- ## Performance
195
-
196
- `npm run bench` measures it on your own hardware and writes a JSON report.
197
-
198
- Measured by CI on every platform we ship a binary for (the bundled
199
- `laya-serve` binary, 4 questions per call). Shared GitHub runners vary
200
- ~20% between runs, so treat these as orders of magnitude, not promises -
201
- run `npm run bench` on your own hardware for exact numbers:
202
-
203
- | platform | init | cold question | warm avg (4 q) | warm p50 / p95 | 1 q per call | throughput |
204
- |---|---|---|---|---|---|---|
205
- | **linux-arm64** (musl) | 1.9 s | 247 ms | 154 ms | 143 / 205 | 36 ms | 28.1 q/s |
206
- | **linux-arm64** (glibc) | 1.6 s | 184 ms | 145 ms | 142 / 163 | 40 ms | 24.9 q/s |
207
- | **win-x64** | 1.7 s | 175 ms | 149 ms | 142 / 175 | 40 ms | 24.8 q/s |
208
- | **win-arm64** | 1.4 s | 231 ms | 178 ms | 174 / 204 | 47 ms | 21.2 q/s |
209
- | **mac-arm64** | 1.3 s | 268 ms | 220 ms | 218 / 250 | 51 ms | 19.5 q/s |
210
- | **linux-x64** (musl) | 2.3 s | 387 ms | 253 ms | 239 / 311 | 59 ms | 16.9 q/s |
211
- | **linux-x64** (glibc) | 2.5 s | 331 ms | 264 ms | 259 / 322 | 63 ms | 15.8 q/s |
212
- | **mac-x64** (Intel, ORT 1.23) | 2.9 s | 387 ms | 360 ms | 356 / 398 | 84 ms | 11.9 q/s |
213
-
214
- `init` is loading the model, `cold` is the very first question (warmup and
215
- arena allocation), and the warm numbers are the sustained latency.
216
-
217
- The `wasm` fallback is ~100x slower by design: `tract` specializes the whole
218
- model per input shape, so it runs a fixed padded shape (see
219
- `LayaEngine.padForWasm`) and pays for every position - about 6 s per question
220
- in the small bucket, 1.8 s to load. It exists for browsers and exotic
221
- platforms, not for throughput.
222
-
223
- **If you are on a platform we ship a binary for and you see the wasm backend
224
- being used, that is a bug** - the install is broken. The tests fail on
225
- purpose in that situation (`LAYA_ALLOW_WASM_FALLBACK=1` is the only way to
226
- accept the fallback).
227
-
228
- ## Size
229
-
230
- | component | size |
231
- |---|---|
232
- | npm package | ~88 MB (native binaries for 6 platforms + WASM + tokenizer) |
233
- | model asset | ~324 MB (acquired once, cached, checksum-verified) |
234
- | disk after install + model | ~560 MB |
235
- | Docker (Debian slim + Bun + package) | ~450 MB image |
297
+ ## 📊 Performance
236
298
 
237
- ---
299
+ Measured on real hardware, on every platform we ship a binary for, with the
300
+ default `native` backend. Run `npm run bench` to measure your own machine.
238
301
 
239
- ## Requirements
302
+ | Platform | Load | First answer | Warm (4 q/call) | Per question |
303
+ | :--- | ---: | ---: | ---: | ---: |
304
+ | macOS arm64 | 1.3 s | 268 ms | 220 ms | 51 ms |
305
+ | Windows arm64 | 1.4 s | 231 ms | 178 ms | 47 ms |
306
+ | Linux arm64 | 1.6 s | 184 ms | 145 ms | 40 ms |
307
+ | Windows x64 | 1.7 s | 175 ms | 149 ms | 40 ms |
308
+ | Linux arm64 (musl) | 1.9 s | 247 ms | 154 ms | 36 ms |
309
+ | Linux x64 (musl) | 2.3 s | 387 ms | 253 ms | 59 ms |
310
+ | Linux x64 | 2.5 s | 331 ms | 264 ms | 63 ms |
311
+ | macOS x64 | 2.9 s | 387 ms | 360 ms | 84 ms |
240
312
 
241
- | | |
242
- |---|---|
243
- | **Node.js** | ≥ 18.17 (zero runtime dependencies) |
244
- | **Bun** | ≥ 1.0 (fully supported) |
245
- | **Browsers** | WASM backend (no install required) |
246
- | **OS** | Linux (glibc + musl/Alpine), macOS (arm64), Windows (x64 + arm64) |
247
- | **Docker** | works on Debian slim, Ubuntu, Alpine |
313
+ `Load` is reading the model into memory. `First answer` includes warmup. The
314
+ warm numbers are sustained latency. Shared CI runners vary by ~20% between
315
+ runs, so treat these as orders of magnitude rather than exact figures.
248
316
 
249
- The `native` backend needs nothing installed: the binary is statically linked and ships in the package (for musl/Alpine it ships as a single self-extracting bundle with its own `lib/`).
317
+ The `wasm` backend is roughly 100x slower — it exists so browsers and unusual
318
+ platforms work at all, not for throughput.
250
319
 
251
- ---
320
+ ### Long inputs
252
321
 
253
- ## Storage
322
+ The model reads up to **8,192 tokens**, but it ships with a conservative
323
+ **2,048-token** budget so it stays usable on weak machines. The budget is a
324
+ cap, not a cost: **short inputs are unaffected by raising it** — a 74-token
325
+ question answers in ~90 ms whatever the limit is, because the work follows the
326
+ input's real length.
254
327
 
255
- | what | where | size |
256
- |---|---|---|
257
- | package (code + binaries) | `node_modules/laya-system-one` | ~88 MB unpacked |
258
- | model asset | `models/model.onnx` or `~/.cache/laya-system-one/` | ~324 MB |
259
- | chunk packages | `dist/release/model-chunks/` (release artifacts) | ~324 MB |
328
+ Raise it when your inputs are long documents:
260
329
 
261
- The model is written to the package directory when it is writable, otherwise to the user cache — so global installs (`npm i -g`) and read-only containers work out of the box.
330
+ ```bash
331
+ LAYA_MAX_LEN=8192 npx laya-system-one --port 8080
332
+ ```
333
+
334
+ ```js
335
+ const laya = await Laya.load({ maxLen: 8192 });
336
+ ```
337
+
338
+ Measured on one machine (Windows arm64, `native` backend), by input length:
339
+
340
+ | tokens | default (2048) | `maxLen: 8192` |
341
+ | ---: | ---: | ---: |
342
+ | 74 | 88 ms | 88 ms |
343
+ | 1,000 | 0.9 s | 0.9 s |
344
+ | 2,000 | 6.4 s | 6.4 s |
345
+ | 4,000 | 9.2 s *(truncated)* | 21.3 s |
346
+ | 8,000 | 9.2 s *(truncated)* | 190 s |
347
+
348
+ Two things worth knowing before you raise it:
349
+
350
+ - **Accuracy degrades with length.** Upstream measured 16–18 of 20 requests
351
+ correct up to about 4,000 tokens, and 8–17 of 20 beyond that. Check your own
352
+ data — long-document accuracy is not something to assume.
353
+ - **Cost grows steeply.** Past ~2,000 tokens the time climbs faster than the
354
+ input does (attention is quadratic). 8,000 tokens is minutes, not seconds,
355
+ on a CPU. If you routinely handle documents that long, truncate them
356
+ yourself to the part that matters, or run the upstream Python package on a
357
+ GPU.
358
+
359
+ Truncation is the real risk of leaving it at the default: a long message gets
360
+ cut off and the answer can be wrong rather than slow. On a ~3,000-token input
361
+ the shipped default answered `sales` where the full text answers `billing`.
262
362
 
263
363
  ---
264
364
 
265
- ## Development
365
+ ## 🧾 Environment variables
366
+
367
+ | Variable | Effect |
368
+ | :--- | :--- |
369
+ | `LAYA_BACKEND` | `native` or `wasm` |
370
+ | `LAYA_MAX_LEN` | token budget for the state (default 2048, max 8192) |
371
+ | `LAYA_MODEL_PATH` | Use a `model.onnx` you already have (file or directory) |
372
+ | `LAYA_MODEL_CHUNKS_DIR` | Directory holding the model chunks |
373
+ | `LAYA_MODEL_URL` | Override where the model is downloaded from |
374
+ | `LAYA_CACHE_DIR` | Where the model is cached |
375
+ | `LAYA_PREFETCH_MODEL` | `1` = download the model during `npm install` |
376
+ | `LAYA_SKIP_MODEL_DOWNLOAD` | `1` = never download, never prompt |
377
+ | `LAYA_API_KEY` / `API_KEY` | Require `Authorization: Bearer <key>` |
378
+ | `LAYA_SERVE_BIN` | Use a specific `laya-serve` binary |
379
+
380
+ **Offline or air-gapped:**
266
381
 
267
382
  ```bash
268
- npm install
269
- npm test # unit + packaging (fast, offline)
270
- npm run test:integration # real model + tokenizer
271
- npm run test:e2e # HTTP protocol + CLI + lifecycle
272
- npm run test:install # pack + install into a clean dir + run
273
- npm run smoke # one-shot human-readable verification
274
- npm run lint # syntax + packaging + docs consistency gate
275
- npm run tokenizer:diff # prove the JS tokenizer == the native binary's
383
+ LAYA_PREFETCH_MODEL=1 npm install laya-system-one # fetch during install
384
+ LAYA_MODEL_PATH=/opt/models/model.onnx # or bring your own copy
385
+ LAYA_MODEL_CHUNKS_DIR=/opt/models/chunks # or a directory of chunks
276
386
  ```
277
387
 
278
- The model-distribution pipeline (the reason the 324 MB asset can live on npm):
388
+ ---
389
+
390
+ ## 💻 Requirements
391
+
392
+ | | |
393
+ | :--- | :--- |
394
+ | **Node.js** | ≥ 18.17 |
395
+ | **Bun** | ≥ 1.0 |
396
+ | **Browsers** | The `wasm` backend |
397
+ | **OS** | Linux (glibc and musl/Alpine), macOS (arm64 and x64), Windows (x64 and arm64) |
398
+ | **Docker** | Debian, Ubuntu, Alpine |
399
+
400
+ No runtime dependencies. The right native binary for your machine is installed
401
+ automatically — nothing to compile, no system packages to add.
402
+
403
+ ---
404
+
405
+ ## 📥 What gets installed
406
+
407
+ The package itself is small; the heavy parts arrive as dependencies npm picks
408
+ for your platform, so you only download what you can run.
409
+
410
+ | | Size |
411
+ | :--- | ---: |
412
+ | `laya-system-one` (code, tokenizer, wasm engine) | ~8.5 MB |
413
+ | The one native binary for your platform | 8–26 MB |
414
+ | The 13 model chunks | ~235 MB total |
415
+ | The model on disk, after the first run | ~324 MB |
416
+
417
+ The model is written next to the package when that directory is writable, and
418
+ to your user cache otherwise — so `npm i -g` and read-only containers work
419
+ without extra configuration. Every copy is checksum-verified, and a run that
420
+ is killed mid-download leaves nothing corrupt behind.
421
+
422
+ ---
423
+
424
+ ## 🛠️ Development
279
425
 
280
426
  ```bash
281
- npm run model:chunks # split models/model.onnx into chunk packages
282
- npm run model:assemble # reassemble from the chunks (byte-identical)
283
- npm run model:verify # checksum the model against the manifest
284
- npm run model:publish # publish the chunk packages to npm
427
+ npm install
428
+ npm run check # lint, unit, packaging, integration, e2e + install rehearsal
429
+ npm run check:quick # the same minus the model-backed suites
430
+ npm run test:rehearsal # install from a local registry and use it, Node + Bun
431
+ npm run bench # measure on this machine
432
+ npm run lint # syntax + packaging + docs consistency
285
433
  ```
286
434
 
287
- CI runs the whole matrix (OS × Node) plus model-backed integration tests on every push — see `.github/workflows/ci.yml`.
288
-
289
- ## Migrating from 1.0.0
435
+ CI builds every platform, proves each binary answers 10 questions, and uploads
436
+ the packages as artifacts. `verify-published.yml` installs a published version
437
+ from the real registry on every platform — Node and Bun, including Alpine for
438
+ musl — and runs a real inference.
290
439
 
291
- 1.0.0 on npm was a different codebase (ONNX Runtime only, model embedded, single language). The `/v1/systemone` wire protocol is unchanged, so HTTP clients keep working. Node API changes:
440
+ ---
292
441
 
293
- - `predict(state, questions)` takes a *map* of questions (1.0.0 took a single question and returned a bare value);
294
- - `server.js`/`client.js` were replaced by `serve()` + `Laya.load()`;
295
- - the model is no longer embedded in the package — it is acquired on first use (see above).
442
+ ## 🙏 Credits
296
443
 
297
- ## License
444
+ **This package would not exist without
445
+ [Laya](https://github.com/NandhaKishorM/laya).** It is a community project by
446
+ [Convai Innovations](https://huggingface.co/convaiinnovations) — the model,
447
+ the architecture, the training method and the wire protocol are all theirs.
448
+ What this package adds is a way to run it where Python is not an option:
449
+ Node.js, Bun and the browser.
298
450
 
299
- Apache-2.0 — see [LICENSE](LICENSE).
451
+ | | |
452
+ | :--- | :--- |
453
+ | **Upstream project** | [NandhaKishorM/laya](https://github.com/NandhaKishorM/laya) |
454
+ | **Model** | [`convaiinnovations/laya-multilingual`](https://huggingface.co/convaiinnovations/laya-multilingual) (mmBERT-base, 322M params) |
455
+ | **Other checkpoints** | [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) (English), [`laya-typed-decisions`](https://huggingface.co/convaiinnovations/laya-typed-decisions) |
456
+ | **Demo** | [Hugging Face Space](https://huggingface.co/spaces/convaiinnovations/laya-demo) |
457
+ | **Method** | RLCD — reinforcement learning against strictly proper scoring rules |
458
+ | **License** | Apache-2.0 (upstream and this package alike) |
459
+
460
+ If you find this useful, the credit belongs upstream — star
461
+ [their repository](https://github.com/NandhaKishorM/laya) and consider
462
+ [supporting the author](https://www.buymeacoffee.com/nandakishorm).
463
+
464
+ ## 📄 License
465
+
466
+ [Apache-2.0](LICENSE) — the same license as the upstream project.
467
+
468
+ - **This package:** [Italo Almeida](https://github.com/italoalmeida0) —
469
+ [laya-system-one](https://github.com/italoalmeida0/laya-system-one)
470
+ - **Model & upstream:** Convai Innovations —
471
+ [NandhaKishorM/laya](https://github.com/NandhaKishorM/laya)