laya-system-one 1.1.0-alpha.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,299 +1,384 @@
1
- # Laya System-One
2
-
3
- **System 1 decision engine** — a multilingual INT8 transformer that answers typed questions (choice / score / yes-no) about any text, in milliseconds, offline.
4
-
5
- Wire-compatible with the **TypeSafe Jev** `/v1/systemone` protocol: send state + typed questions, get structured decisions back.
6
-
7
- ```bash
8
- npm install laya-system-one
9
- ```
10
-
11
- ```js
12
- import { Laya } from 'laya-system-one';
13
-
14
- const laya = await Laya.load(); // model is acquired on first use (once)
15
- const out = await laya.predict(
16
- 'We were billed twice on the March invoice and want a refund.',
17
- {
18
- department: {
19
- type: 'choice',
20
- instructions: 'Which department should handle this?',
21
- criteria: { billing: 'refunds and invoices', tech: 'bugs', sales: 'upgrades' }
22
- },
23
- churn: { type: 'noul', instructions: 'Is the user at churn risk?', threshold: 0.5 },
24
- severity: { type: 'score', instructions: 'Urgency?', criteria: ['low', 'mid', 'high'] }
25
- }
26
- );
27
-
28
- console.log(out.answers.department.choice); // → "billing"
29
- ```
30
-
31
- Or run it as a service:
32
-
33
- ```bash
34
- npx laya-system-one --port 8080
35
- curl -X POST http://localhost:8080/v1/systemone \
36
- -H "Content-Type: application/json" \
37
- -d '{"state":"I was charged twice and need a refund.","questions":{"dept":{"type":"choice","instructions":"Which department?","criteria":{"billing":"refunds","tech":"bugs"}}}}'
38
- ```
39
-
40
- ---
41
-
42
- ## What it actually is
43
-
44
- A single ONNX checkpoint (`model.onnx`, ~324 MB, INT8) trained to score answer options for three question types, plus a small runtime that:
45
-
46
- 1. renders the state and the question options into a prompt,
47
- 2. runs one forward pass,
48
- 3. turns the logits into calibrated probabilities.
49
-
50
- The runtime is JavaScript (Node/Bun/browser) and executes that forward pass in exactly two ways — pick one with `--backend` or `LAYA_BACKEND`:
51
-
52
- | backend | what it is | when to use |
53
- |---|---|---|
54
- | `native` **(default)** | the `laya-serve` binary bundled in the package (Rust: Axum + tokenizers + ONNX Runtime, statically linked) | fastest path, zero system dependencies, any OS |
55
- | `wasm` | pure-Rust `tract` compiled to WASM, bundled in the package | browsers and extreme portability |
56
-
57
- There are **no external runtime dependencies**: no `onnxruntime-*`, no `@huggingface/transformers`, nothing to download besides the model. The tokenizer itself is a pure-JS BPE implementation (`src/bpe-tokenizer.js`) verified token-for-token against the reference `tokenizers` crate that the native binary links — so both backends see exactly the same input ids.
58
-
59
- > **Note on WebGPU:** earlier versions advertised WebGPU acceleration. Measured reality: ONNX Runtime's Node WebGPU execution provider falls back to CPU per-op with large overhead — it is consistently *slower* than the native backend. WebGPU is not used server-side.
60
-
61
- ---
62
-
63
- ## How the heavy pieces arrive
64
-
65
- The entry package is small on purpose (**~8.5 MB**): source, tokenizer and the
66
- wasm fallback. The binaries (150+ MB for every platform) and the model
67
- (324 MB) are published as separate `@sys-one` packages and installed as
68
- `optionalDependencies`, so npm fetches only what the machine needs:
69
-
70
- | package | selected by | size |
71
- | --- | --- | --- |
72
- | `@sys-one/laya-serve-<os>-<arch>` | npm's `os`/`cpu` fields | 8–26 MB |
73
- | `@sys-one/laya-serve-universal` | fallback for exotic platforms | ~76 MB |
74
- | `@sys-one/laya-model-chunk-00` … `-12` | always (all 13) | ~24 MB each |
75
-
76
- A `linux/x64` machine installs `laya-serve-linux-x64` and nothing else;
77
- a `linux/ppc64` machine matches none of the specific packages and gets the
78
- universal one. No package installs all of them.
79
-
80
- Two things worth knowing about how the selection works, because both were
81
- verified by experiment rather than assumed:
82
-
83
- - **`libc` cannot select.** The field exists but is unreliable (undocumented
84
- shorthand, and real packages have shipped bugs where `--libc=glibc` pulls
85
- the musl build too). So the glibc and musl builds of a Linux arch travel
86
- **together** in one package, and the loader picks at runtime.
87
- - **Exclusions are AND-ed.** `os: ["!darwin", "!win32"]` means "not both",
88
- not "either", so the complement of the specific packages cannot be spelled
89
- out. The universal package lists the exotic cpus and OSes explicitly
90
- instead: skipped wherever a specific package applies, selected everywhere
91
- else.
92
-
93
- The model is assembled from its chunk packages on first use and verified
94
- against `models/model.manifest.json` (sha256 of the model *and* of every
95
- chunk), written atomically — a killed process cannot leave a corrupt model
96
- behind, the next run retries. The order is:
97
-
98
- 1. `LAYA_MODEL_PATH` — explicit path to a `model.onnx` (file or directory);
99
- 2. a `model.onnx` already present in the package's `models/` directory;
100
- 3. `~/.cache/laya-system-one/model.onnx` — previously assembled copy;
101
- 4. the installed `@sys-one/laya-model-chunk-*` packages (the normal path);
102
- 5. the npm registry, if the chunks were not installed as dependencies;
103
- 6. the GitHub Release asset — **opt-in only** (`LAYA_ALLOW_GITHUB_FALLBACK=1`),
104
- kept for compatibility with 1.0.0 installs.
105
-
106
- ### Offline / air-gapped installs
107
-
108
- ```bash
109
- LAYA_PREFETCH_MODEL=1 npm install laya-system-one # fetch during install
110
- # or point at a model you already have:
111
- LAYA_MODEL_PATH=/opt/models/model.onnx
112
- # or drop chunk files somewhere and point at them:
113
- LAYA_MODEL_CHUNKS_DIR=/opt/models/chunks
114
- ```
115
-
116
- ---
117
-
118
- ## API
119
-
120
- ### `Laya.load(options)` → `laya`
121
-
122
- | option | default | description |
123
- |---|---|---|
124
- | `backend` | `'native'` | `native` \| `wasm` (or env `LAYA_BACKEND`) |
125
- | `modelDir` | `<package>/models` | where `model.onnx` and `tokenizer.json` live |
126
- | `apiKey` | `null` | Bearer token required by the HTTP layer |
127
- | `port` / `host` | `0` / `127.0.0.1` | where the native server binds |
128
-
129
- ### `laya.predict(state, questions, model?)` → `Promise<Answer>`
130
-
131
- `state` is a string, object or array (serialized as JSON). `questions` is a map of question definitions:
132
-
133
- | type | `criteria` | answer |
134
- |---|---|---|
135
- | `choice` | object (label → meaning) or array | `{ type: 'choice', choice, probabilities, confidence }` |
136
- | `score` | array of ordered levels | `{ type: 'score', score, legend, probabilities, confidence }` |
137
- | `noul` | optional `{false, true}` text | `{ type: 'noul', noul, confidence, threshold?, decision? }` |
138
-
139
- `noul` returns `noul` ∈ [0,1] (probability of *true*). With a `threshold`, a boolean `decision` is added. All fields round to 4 decimals.
140
-
141
- ### HTTP server
142
-
143
- ```js
144
- import { serve } from 'laya-system-one';
145
- const srv = await serve({ port: 8080, apiKey: process.env.LAYA_API_KEY });
146
- console.log(srv.url); // http://localhost:8080
147
- await srv.close(); // releases the engine and any child process
148
- ```
149
-
150
- | endpoint | method | body | response |
151
- |---|---|---|---|
152
- | `/v1/systemone` | `POST` | `{ state, questions, model? }` | `{ model, answers, usage }` |
153
- | `/health` | `GET` | – | `{ status, model, backend, protocol }` |
154
-
155
- Errors: `401` missing/invalid API key, `422` invalid payload, `404` unknown route, `413` body over 4 MB.
156
-
157
- > The `native` backend runs its own internal HTTP server for the engine;
158
- > `serve()` keeps it on a private loopback port and never exposes it.
159
-
160
- ---
161
-
162
- ## CLI
163
-
164
- ```bash
165
- npx laya-system-one --port 8080 --backend native
166
- ```
167
-
168
- ```
169
- --port <number> HTTP port (default 8080, or PORT env)
170
- --host <string> bind address (default 0.0.0.0, or HOST env)
171
- --backend <type> native | wasm (default native)
172
- --api-key <string> require Bearer token auth
173
- --help / --version
174
- ```
175
-
176
- ---
177
-
178
- ## Environment variables
179
-
180
- | variable | effect |
181
- |---|---|
182
- | `LAYA_BACKEND` | `native` \| `wasm` |
183
- | `LAYA_MODEL_PATH` | explicit `model.onnx` (file or directory) |
184
- | `LAYA_MODEL_CHUNKS_DIR` | directory with chunk files / chunk packages |
185
- | `LAYA_MODEL_URL` | override the download URL of the model asset |
186
- | `LAYA_CACHE_DIR` | where downloaded models are cached |
187
- | `LAYA_PREFETCH_MODEL` | `1` = fetch the model during `npm install` |
188
- | `LAYA_SKIP_MODEL_DOWNLOAD` | `1` = never download, never hint |
189
- | `LAYA_API_KEY` / `API_KEY` | require `Authorization: Bearer <key>` |
190
- | `LAYA_SERVE_BIN` | explicit path to the `laya-serve` binary |
191
-
192
- ---
193
-
194
- ## Performance
195
-
196
- `npm run bench` measures it on your own hardware and writes a JSON report.
197
-
198
- Measured by CI on every platform we ship a binary for (the bundled
199
- `laya-serve` binary, 4 questions per call). Shared GitHub runners vary
200
- ~20% between runs, so treat these as orders of magnitude, not promises -
201
- run `npm run bench` on your own hardware for exact numbers:
202
-
203
- | platform | init | cold question | warm avg (4 q) | warm p50 / p95 | 1 q per call | throughput |
204
- |---|---|---|---|---|---|---|
205
- | **linux-arm64** (musl) | 1.9 s | 247 ms | 154 ms | 143 / 205 | 36 ms | 28.1 q/s |
206
- | **linux-arm64** (glibc) | 1.6 s | 184 ms | 145 ms | 142 / 163 | 40 ms | 24.9 q/s |
207
- | **win-x64** | 1.7 s | 175 ms | 149 ms | 142 / 175 | 40 ms | 24.8 q/s |
208
- | **win-arm64** | 1.4 s | 231 ms | 178 ms | 174 / 204 | 47 ms | 21.2 q/s |
209
- | **mac-arm64** | 1.3 s | 268 ms | 220 ms | 218 / 250 | 51 ms | 19.5 q/s |
210
- | **linux-x64** (musl) | 2.3 s | 387 ms | 253 ms | 239 / 311 | 59 ms | 16.9 q/s |
211
- | **linux-x64** (glibc) | 2.5 s | 331 ms | 264 ms | 259 / 322 | 63 ms | 15.8 q/s |
212
- | **mac-x64** (Intel, ORT 1.23) | 2.9 s | 387 ms | 360 ms | 356 / 398 | 84 ms | 11.9 q/s |
213
-
214
- `init` is loading the model, `cold` is the very first question (warmup and
215
- arena allocation), and the warm numbers are the sustained latency.
216
-
217
- The `wasm` fallback is ~100x slower by design: `tract` specializes the whole
218
- model per input shape, so it runs a fixed padded shape (see
219
- `LayaEngine.padForWasm`) and pays for every position - about 6 s per question
220
- in the small bucket, 1.8 s to load. It exists for browsers and exotic
221
- platforms, not for throughput.
222
-
223
- **If you are on a platform we ship a binary for and you see the wasm backend
224
- being used, that is a bug** - the install is broken. The tests fail on
225
- purpose in that situation (`LAYA_ALLOW_WASM_FALLBACK=1` is the only way to
226
- accept the fallback).
227
-
228
- ## Size
229
-
230
- | component | size |
231
- |---|---|
232
- | npm package | ~88 MB (native binaries for 6 platforms + WASM + tokenizer) |
233
- | model asset | ~324 MB (acquired once, cached, checksum-verified) |
234
- | disk after install + model | ~560 MB |
235
- | Docker (Debian slim + Bun + package) | ~450 MB image |
236
-
237
- ---
238
-
239
- ## Requirements
240
-
241
- | | |
242
- |---|---|
243
- | **Node.js** | ≥ 18.17 (zero runtime dependencies) |
244
- | **Bun** | ≥ 1.0 (fully supported) |
245
- | **Browsers** | WASM backend (no install required) |
246
- | **OS** | Linux (glibc + musl/Alpine), macOS (arm64), Windows (x64 + arm64) |
247
- | **Docker** | works on Debian slim, Ubuntu, Alpine |
248
-
249
- The `native` backend needs nothing installed: the binary is statically linked and ships in the package (for musl/Alpine it ships as a single self-extracting bundle with its own `lib/`).
250
-
251
- ---
252
-
253
- ## Storage
254
-
255
- | what | where | size |
256
- |---|---|---|
257
- | package (code + binaries) | `node_modules/laya-system-one` | ~88 MB unpacked |
258
- | model asset | `models/model.onnx` or `~/.cache/laya-system-one/` | ~324 MB |
259
- | chunk packages | `dist/release/model-chunks/` (release artifacts) | ~324 MB |
260
-
261
- The model is written to the package directory when it is writable, otherwise to the user cache — so global installs (`npm i -g`) and read-only containers work out of the box.
262
-
263
- ---
264
-
265
- ## Development
266
-
267
- ```bash
268
- npm install
269
- npm test # unit + packaging (fast, offline)
270
- npm run test:integration # real model + tokenizer
271
- npm run test:e2e # HTTP protocol + CLI + lifecycle
272
- npm run test:install # pack + install into a clean dir + run
273
- npm run smoke # one-shot human-readable verification
274
- npm run lint # syntax + packaging + docs consistency gate
275
- npm run tokenizer:diff # prove the JS tokenizer == the native binary's
276
- ```
277
-
278
- The model-distribution pipeline (the reason the 324 MB asset can live on npm):
279
-
280
- ```bash
281
- npm run model:chunks # split models/model.onnx into chunk packages
282
- npm run model:assemble # reassemble from the chunks (byte-identical)
283
- npm run model:verify # checksum the model against the manifest
284
- npm run model:publish # publish the chunk packages to npm
285
- ```
286
-
287
- CI runs the whole matrix (OS × Node) plus model-backed integration tests on every push — see `.github/workflows/ci.yml`.
288
-
289
- ## Migrating from 1.0.0
290
-
291
- 1.0.0 on npm was a different codebase (ONNX Runtime only, model embedded, single language). The `/v1/systemone` wire protocol is unchanged, so HTTP clients keep working. Node API changes:
292
-
293
- - `predict(state, questions)` takes a *map* of questions (1.0.0 took a single question and returned a bare value);
294
- - `server.js`/`client.js` were replaced by `serve()` + `Laya.load()`;
295
- - the model is no longer embedded in the package — it is acquired on first use (see above).
296
-
297
- ## License
298
-
299
- Apache-2.0 — see [LICENSE](LICENSE).
1
+ # Laya System-One ⚡
2
+
3
+ [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
4
+ [![Runtime](https://img.shields.io/badge/Runtime-Node.js%20%7C%20Bun%20%7C%20Browser-green.svg)]()
5
+ [![TypeSafe Jev](https://img.shields.io/badge/Wire%20Protocol-TypeSafe%20Jev%20Compatible-orange.svg)]()
6
+ [![Dependencies](https://img.shields.io/badge/dependencies-0-brightgreen.svg)]()
7
+
8
+ > **A fast, self-contained decision engine. Give it any text and a set of typed questions, and it answers them — offline, in milliseconds, in over 100 languages. Drop-in compatible with the TypeSafe Jev API (`POST /v1/systemone`).**
9
+
10
+ Runs entirely on your machine. No Python, no PyTorch, no API keys, no cloud
11
+ calls at inference time. One `npm install` and it works.
12
+
13
+ ---
14
+
15
+ ## 🌟 Why Laya System-One?
16
+
17
+ - 🔒 **100% Offline:** Nothing leaves your machine. Ideal for corporate
18
+ intranets, edge servers and privacy-sensitive workflows.
19
+ - ⚡ **Fast:** ~40 ms per question on a warm engine, measured on every platform
20
+ we ship for.
21
+ - 🔄 **TypeSafe Jev Compatible:** Drop-in `POST /v1/systemone`. Point an
22
+ existing Jev client at it and it just works.
23
+ - 🌍 **Multilingual:** Understands English, Portuguese, Spanish, German,
24
+ French, Chinese, Japanese and 100+ more, out of the box.
25
+ - 💻 **Node.js, Bun and Browsers:** Native binary on Node and Bun, WebAssembly
26
+ in the browser.
27
+ - 📦 **Zero Dependencies:** `dependencies` is empty. Nothing to compile,
28
+ nothing to install system-wide, nothing to keep patched.
29
+ - 🧩 **Two Ways to Run It:** As a local HTTP service via the CLI, or in-process
30
+ for zero network overhead.
31
+
32
+ ---
33
+
34
+ ## 📦 Installation
35
+
36
+ ```bash
37
+ # npm
38
+ npm install laya-system-one
39
+
40
+ # bun
41
+ bun add laya-system-one
42
+
43
+ # pnpm
44
+ pnpm add laya-system-one
45
+ ```
46
+
47
+ The right engine for your machine is installed automatically. The model
48
+ (~324 MB) is fetched once on first use and cached.
49
+
50
+ ---
51
+
52
+ ## 🚀 Quick Start
53
+
54
+ ### 1. Launch the HTTP service
55
+
56
+ ```bash
57
+ npx laya-system-one --port 8080
58
+ ```
59
+
60
+ | Flag | Env | Default | Description |
61
+ | :--- | :--- | :--- | :--- |
62
+ | `--port <number>` | `PORT` | `8080` | Port to bind |
63
+ | `--host <string>` | `HOST` | `0.0.0.0` | Address to bind |
64
+ | `--backend <type>` | `LAYA_BACKEND` | `native` | `native` or `wasm` |
65
+ | `--api-key <token>` | `LAYA_API_KEY` | *(none)* | Require Bearer auth on `/v1/systemone` |
66
+
67
+ With authentication:
68
+
69
+ ```bash
70
+ npx laya-system-one --port 8080 --api-key secret-token-xyz
71
+ ```
72
+
73
+ ### 2. Use it in-process (zero network overhead)
74
+
75
+ ```javascript
76
+ import { Laya } from 'laya-system-one';
77
+
78
+ // 1. Initialize the engine
79
+ const laya = await Laya.load();
80
+
81
+ // 2. Define the state (string, object, or array)
82
+ const state = {
83
+ customer_id: 'cust_9821',
84
+ message: 'We were charged twice on our March invoice. Please refund the duplicate amount or we will cancel our plan.'
85
+ };
86
+
87
+ // 3. Define typed questions
88
+ const questions = {
89
+ department: {
90
+ type: 'choice',
91
+ instructions: 'Which team should resolve this customer inquiry?',
92
+ criteria: {
93
+ billing: 'Invoices, refunds, and duplicate charges',
94
+ tech_support: 'Software bugs, outages, and error messages',
95
+ sales: 'Upgrades, plan changes, and enterprise contracts'
96
+ }
97
+ },
98
+ urgency: {
99
+ type: 'score',
100
+ instructions: 'Assess the urgency level of this inquiry.',
101
+ criteria: ['Low / routine', 'Moderate', 'Critical / blocking / angry']
102
+ },
103
+ churn_risk: {
104
+ type: 'noul',
105
+ instructions: 'Does this message present an explicit risk of customer churn?',
106
+ threshold: 0.5
107
+ }
108
+ };
109
+
110
+ // 4. Evaluate
111
+ const result = await laya.predict(state, questions);
112
+
113
+ console.log(result.answers.department.choice); // -> "billing"
114
+ console.log(result.answers.department.confidence); // -> 1.0
115
+ console.log(result.answers.urgency.score); // -> 1.95
116
+ console.log(result.answers.churn_risk.noul); // -> 0.968
117
+ console.log(result.answers.churn_risk.decision); // -> true
118
+ ```
119
+
120
+ ### 3. Serve it from inside your app
121
+
122
+ ```javascript
123
+ import { serve } from 'laya-system-one';
124
+
125
+ const srv = await serve({ host: '127.0.0.1', port: 8080, apiKey: 'optional-key' });
126
+
127
+ console.log(`Laya server running at ${srv.url}/v1/systemone`);
128
+
129
+ // later:
130
+ await srv.close();
131
+ ```
132
+
133
+ ---
134
+
135
+ ## 📡 HTTP API Reference (TypeSafe Jev compatible)
136
+
137
+ ```http
138
+ POST /v1/systemone
139
+ Host: localhost:8080
140
+ Content-Type: application/json
141
+ Authorization: Bearer <API_KEY> [optional unless configured]
142
+ ```
143
+
144
+ | Parameter | Type | Required | Description |
145
+ | :--- | :--- | :--- | :--- |
146
+ | `state` | `string` \| `object` \| `array` | **Yes** | The context or text being evaluated. |
147
+ | `questions` | `Record<string, Question>` | **Yes** | Map of question keys to typed questions. |
148
+ | `model` | `string` | No | Model name (defaults to `laya-multilingual`, echoed back). |
149
+
150
+ ### Question types
151
+
152
+ **`choice`** — pick one of several options:
153
+
154
+ ```json
155
+ {
156
+ "type": "choice",
157
+ "instructions": "Which department should handle this ticket?",
158
+ "criteria": {
159
+ "billing": "Invoices and credit card transactions",
160
+ "technical": "Software bugs and service disruptions"
161
+ }
162
+ }
163
+ ```
164
+
165
+ **`score`** — place on an ordered scale:
166
+
167
+ ```json
168
+ {
169
+ "type": "score",
170
+ "instructions": "Rate the severity of the issue.",
171
+ "criteria": ["Minor cosmetic issue", "Degraded functionality", "Critical full service outage"]
172
+ }
173
+ ```
174
+
175
+ **`noul`** — calibrated yes/no probability:
176
+
177
+ ```json
178
+ {
179
+ "type": "noul",
180
+ "instructions": "Does the user explicitly request a refund?",
181
+ "threshold": 0.6
182
+ }
183
+ ```
184
+
185
+ ### Example request
186
+
187
+ ```bash
188
+ curl -X POST http://localhost:8080/v1/systemone \
189
+ -H "Content-Type: application/json" \
190
+ -d '{
191
+ "state": { "text": "Fui cobrado duas vezes na minha fatura. Reembolsem imediatamente." },
192
+ "questions": {
193
+ "dept": {
194
+ "type": "choice",
195
+ "instructions": "Which department should respond?",
196
+ "criteria": { "billing": "Refunds, invoices, and payments", "support": "Technical and product questions" }
197
+ },
198
+ "urgency": {
199
+ "type": "score",
200
+ "instructions": "Urgency rating",
201
+ "criteria": ["Low", "Medium", "High"]
202
+ },
203
+ "refund_demanded": {
204
+ "type": "noul",
205
+ "instructions": "Is the customer requesting a refund?",
206
+ "threshold": 0.5
207
+ }
208
+ }
209
+ }'
210
+ ```
211
+
212
+ ### Example response
213
+
214
+ ```json
215
+ {
216
+ "model": "laya-multilingual",
217
+ "answers": {
218
+ "dept": {
219
+ "type": "choice",
220
+ "choice": "billing",
221
+ "probabilities": { "billing": 1.0, "support": 0.0 },
222
+ "confidence": 1.0
223
+ },
224
+ "urgency": {
225
+ "type": "score",
226
+ "score": 1.9482,
227
+ "legend": { "0": "Low", "1": "Medium", "2": "High" },
228
+ "probabilities": { "0": 0.0011, "1": 0.0496, "2": 0.9493 },
229
+ "confidence": 0.9493
230
+ },
231
+ "refund_demanded": {
232
+ "type": "noul",
233
+ "noul": 0.9852,
234
+ "confidence": 0.9852,
235
+ "threshold": 0.5,
236
+ "decision": true
237
+ }
238
+ },
239
+ "usage": { "input_tokens": 82, "output_tokens": 12 }
240
+ }
241
+ ```
242
+
243
+ ### Healthcheck
244
+
245
+ ```http
246
+ GET /health
247
+ ```
248
+
249
+ ```json
250
+ {
251
+ "status": "ok",
252
+ "model": "laya-multilingual",
253
+ "version": "1.1.0",
254
+ "protocol": "TypeSafe Jev /v1/systemone compatible"
255
+ }
256
+ ```
257
+
258
+ ### Error codes
259
+
260
+ - `401 Unauthorized` — API key configured, header missing or wrong.
261
+ - `422 Unprocessable Entity` — invalid JSON, or `state`/`questions` missing.
262
+ - `404 Not Found` — unknown route.
263
+ - `413 Payload Too Large` — body over 4 MB.
264
+
265
+ ---
266
+
267
+ ## ⚙️ Backends
268
+
269
+ | Backend | How it runs | When to use |
270
+ | :--- | :--- | :--- |
271
+ | **`native`** *(default)* | A self-contained Rust server bundled with the package | The normal choice. Fastest, nothing to install. |
272
+ | **`wasm`** | Pure Rust compiled to WebAssembly, also bundled | Browsers, or platforms with no native build. |
273
+
274
+ Both ship inside the package — nothing is compiled or downloaded at install
275
+ time. Switch with `--backend wasm` or `LAYA_BACKEND=wasm`.
276
+
277
+ ---
278
+
279
+ ## 📊 Performance
280
+
281
+ Measured on real hardware, on every platform we ship a binary for, with the
282
+ default `native` backend. Run `npm run bench` to measure your own machine.
283
+
284
+ | Platform | Load | First answer | Warm (4 q/call) | Per question |
285
+ | :--- | ---: | ---: | ---: | ---: |
286
+ | macOS arm64 | 1.3 s | 268 ms | 220 ms | 51 ms |
287
+ | Windows arm64 | 1.4 s | 231 ms | 178 ms | 47 ms |
288
+ | Linux arm64 | 1.6 s | 184 ms | 145 ms | 40 ms |
289
+ | Windows x64 | 1.7 s | 175 ms | 149 ms | 40 ms |
290
+ | Linux arm64 (musl) | 1.9 s | 247 ms | 154 ms | 36 ms |
291
+ | Linux x64 (musl) | 2.3 s | 387 ms | 253 ms | 59 ms |
292
+ | Linux x64 | 2.5 s | 331 ms | 264 ms | 63 ms |
293
+ | macOS x64 | 2.9 s | 387 ms | 360 ms | 84 ms |
294
+
295
+ `Load` is reading the model into memory. `First answer` includes warmup. The
296
+ warm numbers are sustained latency. Shared CI runners vary by ~20% between
297
+ runs, so treat these as orders of magnitude rather than exact figures.
298
+
299
+ The `wasm` backend is roughly 100x slower — it exists so browsers and unusual
300
+ platforms work at all, not for throughput.
301
+
302
+ ---
303
+
304
+ ## 🧾 Environment variables
305
+
306
+ | Variable | Effect |
307
+ | :--- | :--- |
308
+ | `LAYA_BACKEND` | `native` or `wasm` |
309
+ | `LAYA_MODEL_PATH` | Use a `model.onnx` you already have (file or directory) |
310
+ | `LAYA_MODEL_CHUNKS_DIR` | Directory holding the model chunks |
311
+ | `LAYA_MODEL_URL` | Override where the model is downloaded from |
312
+ | `LAYA_CACHE_DIR` | Where the model is cached |
313
+ | `LAYA_PREFETCH_MODEL` | `1` = download the model during `npm install` |
314
+ | `LAYA_SKIP_MODEL_DOWNLOAD` | `1` = never download, never prompt |
315
+ | `LAYA_API_KEY` / `API_KEY` | Require `Authorization: Bearer <key>` |
316
+ | `LAYA_SERVE_BIN` | Use a specific `laya-serve` binary |
317
+
318
+ **Offline or air-gapped:**
319
+
320
+ ```bash
321
+ LAYA_PREFETCH_MODEL=1 npm install laya-system-one # fetch during install
322
+ LAYA_MODEL_PATH=/opt/models/model.onnx # or bring your own copy
323
+ LAYA_MODEL_CHUNKS_DIR=/opt/models/chunks # or a directory of chunks
324
+ ```
325
+
326
+ ---
327
+
328
+ ## 💻 Requirements
329
+
330
+ | | |
331
+ | :--- | :--- |
332
+ | **Node.js** | ≥ 18.17 |
333
+ | **Bun** | ≥ 1.0 |
334
+ | **Browsers** | The `wasm` backend |
335
+ | **OS** | Linux (glibc and musl/Alpine), macOS (arm64 and x64), Windows (x64 and arm64) |
336
+ | **Docker** | Debian, Ubuntu, Alpine |
337
+
338
+ No runtime dependencies. The right native binary for your machine is installed
339
+ automatically — nothing to compile, no system packages to add.
340
+
341
+ ---
342
+
343
+ ## 📥 What gets installed
344
+
345
+ The package itself is small; the heavy parts arrive as dependencies npm picks
346
+ for your platform, so you only download what you can run.
347
+
348
+ | | Size |
349
+ | :--- | ---: |
350
+ | `laya-system-one` (code, tokenizer, wasm engine) | ~8.5 MB |
351
+ | The one native binary for your platform | 8–26 MB |
352
+ | The 13 model chunks | ~235 MB total |
353
+ | The model on disk, after the first run | ~324 MB |
354
+
355
+ The model is written next to the package when that directory is writable, and
356
+ to your user cache otherwise — so `npm i -g` and read-only containers work
357
+ without extra configuration. Every copy is checksum-verified, and a run that
358
+ is killed mid-download leaves nothing corrupt behind.
359
+
360
+ ---
361
+
362
+ ## 🛠️ Development
363
+
364
+ ```bash
365
+ npm install
366
+ npm run check # lint, unit, packaging, integration, e2e + install rehearsal
367
+ npm run check:quick # the same minus the model-backed suites
368
+ npm run test:rehearsal # install from a local registry and use it, Node + Bun
369
+ npm run bench # measure on this machine
370
+ npm run lint # syntax + packaging + docs consistency
371
+ ```
372
+
373
+ CI builds every platform, proves each binary answers 10 questions, and uploads
374
+ the packages as artifacts. `verify-published.yml` installs a published version
375
+ from the real registry on every platform — Node and Bun, including Alpine for
376
+ musl — and runs a real inference.
377
+
378
+ ---
379
+
380
+ ## 📄 License & Attribution
381
+
382
+ - **License:** [Apache-2.0](LICENSE)
383
+ - **Author:** [Italo Almeida](https://github.com/italoalmeida0)
384
+ - **Repository:** [https://github.com/italoalmeida0/laya-system-one](https://github.com/italoalmeida0/laya-system-one)
@@ -10,91 +10,91 @@
10
10
  {
11
11
  "index": 0,
12
12
  "package": "@sys-one/laya-model-chunk-00",
13
- "version": "1.1.0-alpha.0",
13
+ "version": "1.1.0",
14
14
  "bytes": 25165824,
15
15
  "sha256": "ef5fb331bbc913691ddb94c65efd4139bbad581870f00ceec697e27a1a277d68"
16
16
  },
17
17
  {
18
18
  "index": 1,
19
19
  "package": "@sys-one/laya-model-chunk-01",
20
- "version": "1.1.0-alpha.0",
20
+ "version": "1.1.0",
21
21
  "bytes": 25165824,
22
22
  "sha256": "197cf82966fa114b1c5480d67dfb0c69e2a29b26d9cdd260a15d34ed7e94bc3d"
23
23
  },
24
24
  {
25
25
  "index": 2,
26
26
  "package": "@sys-one/laya-model-chunk-02",
27
- "version": "1.1.0-alpha.0",
27
+ "version": "1.1.0",
28
28
  "bytes": 25165824,
29
29
  "sha256": "a72e60a93dfa4e93eedc45eb0bfc0c0ab48010db35d93acd613c2312e59c35e1"
30
30
  },
31
31
  {
32
32
  "index": 3,
33
33
  "package": "@sys-one/laya-model-chunk-03",
34
- "version": "1.1.0-alpha.0",
34
+ "version": "1.1.0",
35
35
  "bytes": 25165824,
36
36
  "sha256": "7db3119f56cc6a4874b0c300effc06ae0f2e4587486d4c6673d74375157426b6"
37
37
  },
38
38
  {
39
39
  "index": 4,
40
40
  "package": "@sys-one/laya-model-chunk-04",
41
- "version": "1.1.0-alpha.0",
41
+ "version": "1.1.0",
42
42
  "bytes": 25165824,
43
43
  "sha256": "b3c66e9ae3210f4a6b0e89e235e880b9eea6636afe33c90e21b07c4260dfea91"
44
44
  },
45
45
  {
46
46
  "index": 5,
47
47
  "package": "@sys-one/laya-model-chunk-05",
48
- "version": "1.1.0-alpha.0",
48
+ "version": "1.1.0",
49
49
  "bytes": 25165824,
50
50
  "sha256": "eb659ce78d4deb34107eff20b86339d77f5fd4e2ae3ccad625775528cd376b6c"
51
51
  },
52
52
  {
53
53
  "index": 6,
54
54
  "package": "@sys-one/laya-model-chunk-06",
55
- "version": "1.1.0-alpha.0",
55
+ "version": "1.1.0",
56
56
  "bytes": 25165824,
57
57
  "sha256": "9ac3fd26e588debb86e2368093e58272b9a318515b9eaebf07da8d1429dc5b62"
58
58
  },
59
59
  {
60
60
  "index": 7,
61
61
  "package": "@sys-one/laya-model-chunk-07",
62
- "version": "1.1.0-alpha.0",
62
+ "version": "1.1.0",
63
63
  "bytes": 25165824,
64
64
  "sha256": "40c158c978ba8285a69b41282308f108d24390734660935be69c6b58ea186b6a"
65
65
  },
66
66
  {
67
67
  "index": 8,
68
68
  "package": "@sys-one/laya-model-chunk-08",
69
- "version": "1.1.0-alpha.0",
69
+ "version": "1.1.0",
70
70
  "bytes": 25165824,
71
71
  "sha256": "d8ce88b2cc1c2f0bcb6f50414fb56741a21d3d2221db19c9fff8879f5e58668e"
72
72
  },
73
73
  {
74
74
  "index": 9,
75
75
  "package": "@sys-one/laya-model-chunk-09",
76
- "version": "1.1.0-alpha.0",
76
+ "version": "1.1.0",
77
77
  "bytes": 25165824,
78
78
  "sha256": "a83210739f94a9ffbf99c32aedd6e8ac68bcbe8be42d57485fff9534e2d8011b"
79
79
  },
80
80
  {
81
81
  "index": 10,
82
82
  "package": "@sys-one/laya-model-chunk-10",
83
- "version": "1.1.0-alpha.0",
83
+ "version": "1.1.0",
84
84
  "bytes": 25165824,
85
85
  "sha256": "c06ed434512a4a8d610d27f52a91bdfff46001eab2b586f11d65c506140753f0"
86
86
  },
87
87
  {
88
88
  "index": 11,
89
89
  "package": "@sys-one/laya-model-chunk-11",
90
- "version": "1.1.0-alpha.0",
90
+ "version": "1.1.0",
91
91
  "bytes": 25165824,
92
92
  "sha256": "44a0a9ad824ab26b8a1c5edb2d71c7283341e4bee5e1bfa4de2a4a9a813e9081"
93
93
  },
94
94
  {
95
95
  "index": 12,
96
96
  "package": "@sys-one/laya-model-chunk-12",
97
- "version": "1.1.0-alpha.0",
97
+ "version": "1.1.0",
98
98
  "bytes": 22135720,
99
99
  "sha256": "35e2c872c113e76456e6790f08c9714f2b83f8a049a550814efd5355640df5ce"
100
100
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "laya-system-one",
3
- "version": "1.1.0-alpha.0",
3
+ "version": "1.1.0",
4
4
  "description": "Self-contained System 1 decision engine: multilingual INT8 transformer served by a zero-dependency native binary (or the bundled pure-Rust WASM engine). TypeSafe Jev wire-compatible (/v1/systemone). Runs offline in Node.js, Bun and browsers.",
5
5
  "main": "./src/index.js",
6
6
  "module": "./src/index.js",
@@ -79,25 +79,25 @@
79
79
  "homepage": "https://github.com/italoalmeida0/laya-system-one#readme",
80
80
  "dependencies": {},
81
81
  "optionalDependencies": {
82
- "@sys-one/laya-serve-darwin-arm64": "1.1.0-alpha.0",
83
- "@sys-one/laya-serve-darwin-x64": "1.1.0-alpha.0",
84
- "@sys-one/laya-serve-win32-x64": "1.1.0-alpha.0",
85
- "@sys-one/laya-serve-win32-arm64": "1.1.0-alpha.0",
86
- "@sys-one/laya-serve-linux-x64": "1.1.0-alpha.0",
87
- "@sys-one/laya-serve-linux-arm64": "1.1.0-alpha.0",
88
- "@sys-one/laya-serve-universal": "1.1.0-alpha.0",
89
- "@sys-one/laya-model-chunk-00": "1.1.0-alpha.0",
90
- "@sys-one/laya-model-chunk-01": "1.1.0-alpha.0",
91
- "@sys-one/laya-model-chunk-02": "1.1.0-alpha.0",
92
- "@sys-one/laya-model-chunk-03": "1.1.0-alpha.0",
93
- "@sys-one/laya-model-chunk-04": "1.1.0-alpha.0",
94
- "@sys-one/laya-model-chunk-05": "1.1.0-alpha.0",
95
- "@sys-one/laya-model-chunk-06": "1.1.0-alpha.0",
96
- "@sys-one/laya-model-chunk-07": "1.1.0-alpha.0",
97
- "@sys-one/laya-model-chunk-08": "1.1.0-alpha.0",
98
- "@sys-one/laya-model-chunk-09": "1.1.0-alpha.0",
99
- "@sys-one/laya-model-chunk-10": "1.1.0-alpha.0",
100
- "@sys-one/laya-model-chunk-11": "1.1.0-alpha.0",
101
- "@sys-one/laya-model-chunk-12": "1.1.0-alpha.0"
82
+ "@sys-one/laya-serve-darwin-arm64": "1.1.0",
83
+ "@sys-one/laya-serve-darwin-x64": "1.1.0",
84
+ "@sys-one/laya-serve-win32-x64": "1.1.0",
85
+ "@sys-one/laya-serve-win32-arm64": "1.1.0",
86
+ "@sys-one/laya-serve-linux-x64": "1.1.0",
87
+ "@sys-one/laya-serve-linux-arm64": "1.1.0",
88
+ "@sys-one/laya-serve-universal": "1.1.0",
89
+ "@sys-one/laya-model-chunk-00": "1.1.0",
90
+ "@sys-one/laya-model-chunk-01": "1.1.0",
91
+ "@sys-one/laya-model-chunk-02": "1.1.0",
92
+ "@sys-one/laya-model-chunk-03": "1.1.0",
93
+ "@sys-one/laya-model-chunk-04": "1.1.0",
94
+ "@sys-one/laya-model-chunk-05": "1.1.0",
95
+ "@sys-one/laya-model-chunk-06": "1.1.0",
96
+ "@sys-one/laya-model-chunk-07": "1.1.0",
97
+ "@sys-one/laya-model-chunk-08": "1.1.0",
98
+ "@sys-one/laya-model-chunk-09": "1.1.0",
99
+ "@sys-one/laya-model-chunk-10": "1.1.0",
100
+ "@sys-one/laya-model-chunk-11": "1.1.0",
101
+ "@sys-one/laya-model-chunk-12": "1.1.0"
102
102
  }
103
103
  }
@@ -122,12 +122,17 @@ function binaryCandidates() {
122
122
  const exe = plat === 'win32' ? 'laya-serve.exe' : 'laya-serve';
123
123
  const slots = [];
124
124
  for (const arch of arches) {
125
- if (plat === 'linux') slots.push(`linux-${arch}-musl`);
126
- if (musl) continue; // never fall back to a glibc binary on musl
127
- slots.push(`${plat}-${arch}`);
125
+ // Order matters, and libc decides it: on musl only the musl bundle can
126
+ // load (the glibc binary has no interpreter), and on glibc the musl
127
+ // bundle is the wrong build. Never probe both - the earlier version
128
+ // pushed the musl slot first on every Linux, so a glibc machine picked
129
+ // the musl bundle and died with "exited before ready".
130
+ if (plat === 'linux') {
131
+ slots.push(musl ? `linux-${arch}-musl` : `linux-${arch}`);
132
+ } else {
133
+ slots.push(`${plat}-${arch}`);
134
+ }
128
135
  }
129
- // the universal package stores glibc builds under the plain slot too, so a
130
- // non-musl loop above already covers it; nothing extra to add here.
131
136
  return { exe, slots };
132
137
  }
133
138
 
package/src/server.js CHANGED
@@ -1,179 +1,194 @@
1
- import http from 'node:http';
2
- import { Laya } from './agent.js';
3
-
4
- /**
5
- * Start HTTP server exposing the TypeSafe Jev /v1/systemone wire protocol.
6
- * @param {Object} options - Server options:
7
- * - host: string (default: '0.0.0.0' or process.env.HOST)
8
- * - port: number (default: 8080 or process.env.PORT)
9
- * - apiKey: string (optional, or process.env.LAYA_API_KEY / process.env.API_KEY)
10
- * - laya: preloaded Laya instance (optional)
11
- * - device: 'auto' | 'webgpu' | 'wasm' | 'cpu' (default: 'auto')
12
- * @returns {Promise<{ server: http.Server, url: string, laya: object, close: Function }>}
13
- */
14
- export async function serve(options = {}) {
15
- const host = options.host || process.env.HOST || '0.0.0.0';
16
- // `port: 0` means "pick a free port" — it must not be treated as unset.
17
- const portRaw = options.port ?? process.env.PORT;
18
- const parsed = Number.parseInt(portRaw ?? '8080', 10);
19
- const port = Number.isFinite(parsed) ? parsed : 8080;
20
- const apiKey = options.apiKey || process.env.LAYA_API_KEY || process.env.API_KEY || null;
21
- // When we create the engine ourselves we also own its lifecycle: close()
22
- // must release it (the native backend spawns a laya-serve child process —
23
- // leaving it alive keeps the Node event loop busy and the process hangs
24
- // forever after the HTTP server is done).
25
- const ownsLaya = !options.laya;
26
- // The engine runs its own internal HTTP server (native backend). It must
27
- // never share the public port: on Windows two sockets CAN bind the same
28
- // address (SO_REUSEADDR), so requests would reach the wrong server and
29
- // shutdown would look broken. Keep it on a private loopback port.
30
- const laya = options.laya || (await Laya.load({ ...options, host: '127.0.0.1', port: 0 }));
31
-
32
- // End-to-end warmup: first real predict() pays tokenizer-cache fill +
33
- // any remaining lazy init. Do it once at startup (best-effort) so the
34
- // first Tetris piece doesn't eat the cold-start cost.
35
- if (!options.laya && options.warmup !== false) {
36
- try {
37
- await laya.predict('warmup', { w: { type: 'noul', instructions: 'warmup probe' } });
38
- } catch { /* best-effort */ }
39
- }
40
-
41
- const server = http.createServer((req, res) => {
42
- // Standard CORS headers
43
- res.setHeader('Access-Control-Allow-Origin', '*');
44
- res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
45
- res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Authorization');
46
-
47
- // Handle preflight OPTIONS request
48
- if (req.method === 'OPTIONS') {
49
- res.writeHead(204);
50
- res.end();
51
- return;
52
- }
53
-
54
- const url = new URL(req.url, `http://${req.headers.host || 'localhost'}`);
55
-
56
- // Healthcheck endpoint
57
- if (req.method === 'GET' && (url.pathname === '/' || url.pathname === '/health')) {
58
- res.writeHead(200, { 'Content-Type': 'application/json' });
59
- res.end(JSON.stringify({
60
- status: 'ok',
61
- model: 'laya-multilingual',
62
- version: '1.0.0',
63
- protocol: 'TypeSafe Jev /v1/systemone compatible'
64
- }));
65
- return;
66
- }
67
-
68
- // TypeSafe Jev evaluation endpoint
69
- if (req.method === 'POST' && url.pathname === '/v1/systemone') {
70
- // Optional bearer token authentication
71
- if (apiKey) {
72
- const authHeader = req.headers['authorization'] || '';
73
- if (authHeader !== `Bearer ${apiKey}`) {
74
- res.writeHead(401, { 'Content-Type': 'application/json' });
75
- res.end(JSON.stringify({
76
- error: 'Unauthorized: missing or invalid bearer token in Authorization header.'
77
- }));
78
- return;
79
- }
80
- }
81
-
82
- const chunks = [];
83
- let size = 0;
84
- req.on('data', chunk => {
85
- chunks.push(chunk);
86
- size += chunk.length;
87
- if (size > 4 * 1024 * 1024) { // 4MB guard
88
- res.writeHead(413, { 'Content-Type': 'application/json' });
89
- res.end(JSON.stringify({ error: 'Payload too large (max 4MB).' }));
90
- req.destroy();
91
- }
92
- });
93
-
94
- req.on('end', async () => {
95
- let payload;
96
- try {
97
- payload = JSON.parse(Buffer.concat(chunks).toString('utf8'));
98
- } catch (e) {
99
- res.writeHead(422, { 'Content-Type': 'application/json' });
100
- res.end(JSON.stringify({
101
- error: 'Unprocessable Entity: invalid JSON payload.'
102
- }));
103
- return;
104
- }
105
-
106
- const state = payload.state;
107
- const questions = payload.questions;
108
- const requestedModel = payload.model || null;
109
-
110
- if (state === undefined || state === null) {
111
- res.writeHead(422, { 'Content-Type': 'application/json' });
112
- res.end(JSON.stringify({
113
- error: "Unprocessable Entity: missing required 'state' field."
114
- }));
115
- return;
116
- }
117
-
118
- if (!questions || typeof questions !== 'object' || Array.isArray(questions)) {
119
- res.writeHead(422, { 'Content-Type': 'application/json' });
120
- res.end(JSON.stringify({
121
- error: "Unprocessable Entity: 'questions' must be an object map of typed questions."
122
- }));
123
- return;
124
- }
125
-
126
- try {
127
- const result = await laya.predict(state, questions, requestedModel);
128
- res.writeHead(200, { 'Content-Type': 'application/json' });
129
- res.end(JSON.stringify(result));
130
- } catch (err) {
131
- res.writeHead(422, { 'Content-Type': 'application/json' });
132
- res.end(JSON.stringify({
133
- error: `Evaluation failed: ${err.message || err}`
134
- }));
135
- }
136
- });
137
- return;
138
- }
139
-
140
- // 404 handler
141
- res.writeHead(404, { 'Content-Type': 'application/json' });
142
- res.end(JSON.stringify({
143
- error: `Not found: ${req.method} ${url.pathname}. Expected POST /v1/systemone`
144
- }));
145
- });
146
-
147
- // Keep-alive tuning: game clients (Tetris) fire one request per piece on
148
- // the same connection. Long keep-alive avoids TCP+handshake per move.
149
- server.keepAliveTimeout = 65000;
150
- server.headersTimeout = 66000;
151
- server.requestTimeout = 0;
152
- server.maxRequestsPerSocket = 0;
153
-
154
- return new Promise((resolve, reject) => {
155
- server.listen(port, host, () => {
156
- const displayHost = host === '0.0.0.0' ? 'localhost' : host;
157
- // use the port actually bound (0 means "any free port")
158
- const boundPort = server.address()?.port ?? port;
159
- const url = `http://${displayHost}:${boundPort}`;
160
- resolve({
161
- server,
162
- url,
163
- laya,
164
- close: async () => {
165
- await new Promise((resolve) => {
166
- server.close(() => resolve());
167
- // Node keeps idle keep-alive sockets open (undici pools them for
168
- // seconds), which would block server.close() — force them shut so
169
- // shutdown is immediate and deterministic.
170
- server.closeIdleConnections?.();
171
- server.closeAllConnections?.();
172
- });
173
- if (ownsLaya && typeof laya.close === 'function') await laya.close();
174
- }
175
- });
176
- });
177
- server.on('error', reject);
178
- });
179
- }
1
+ import http from 'node:http';
2
+ import fs from 'node:fs';
3
+ import path from 'node:path';
4
+ import { fileURLToPath } from 'node:url';
5
+ import { Laya } from './agent.js';
6
+
7
+ const __dirname = path.dirname(fileURLToPath(import.meta.url));
8
+
9
+ // Report the version we actually are, not a literal that goes stale: /health
10
+ // is what a deployment checks, and it said 1.0.0 while the package was 1.1.0.
11
+ const PKG_VERSION = (() => {
12
+ try {
13
+ return JSON.parse(fs.readFileSync(path.join(__dirname, '..', 'package.json'), 'utf8')).version;
14
+ } catch {
15
+ return 'unknown';
16
+ }
17
+ })();
18
+
19
+ /**
20
+ * Start HTTP server exposing the TypeSafe Jev /v1/systemone wire protocol.
21
+ * @param {Object} options - Server options:
22
+ * - host: string (default: '0.0.0.0' or process.env.HOST)
23
+ * - port: number (default: 8080 or process.env.PORT)
24
+ * - apiKey: string (optional, or process.env.LAYA_API_KEY / process.env.API_KEY)
25
+ * - laya: preloaded Laya instance (optional)
26
+ * - device: 'auto' | 'webgpu' | 'wasm' | 'cpu' (default: 'auto')
27
+ * @returns {Promise<{ server: http.Server, url: string, laya: object, close: Function }>}
28
+ */
29
+ export async function serve(options = {}) {
30
+ const host = options.host || process.env.HOST || '0.0.0.0';
31
+ // `port: 0` means "pick a free port" — it must not be treated as unset.
32
+ const portRaw = options.port ?? process.env.PORT;
33
+ const parsed = Number.parseInt(portRaw ?? '8080', 10);
34
+ const port = Number.isFinite(parsed) ? parsed : 8080;
35
+ const apiKey = options.apiKey || process.env.LAYA_API_KEY || process.env.API_KEY || null;
36
+ // When we create the engine ourselves we also own its lifecycle: close()
37
+ // must release it (the native backend spawns a laya-serve child process —
38
+ // leaving it alive keeps the Node event loop busy and the process hangs
39
+ // forever after the HTTP server is done).
40
+ const ownsLaya = !options.laya;
41
+ // The engine runs its own internal HTTP server (native backend). It must
42
+ // never share the public port: on Windows two sockets CAN bind the same
43
+ // address (SO_REUSEADDR), so requests would reach the wrong server and
44
+ // shutdown would look broken. Keep it on a private loopback port.
45
+ const laya = options.laya || (await Laya.load({ ...options, host: '127.0.0.1', port: 0 }));
46
+
47
+ // End-to-end warmup: first real predict() pays tokenizer-cache fill +
48
+ // any remaining lazy init. Do it once at startup (best-effort) so the
49
+ // first Tetris piece doesn't eat the cold-start cost.
50
+ if (!options.laya && options.warmup !== false) {
51
+ try {
52
+ await laya.predict('warmup', { w: { type: 'noul', instructions: 'warmup probe' } });
53
+ } catch { /* best-effort */ }
54
+ }
55
+
56
+ const server = http.createServer((req, res) => {
57
+ // Standard CORS headers
58
+ res.setHeader('Access-Control-Allow-Origin', '*');
59
+ res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
60
+ res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Authorization');
61
+
62
+ // Handle preflight OPTIONS request
63
+ if (req.method === 'OPTIONS') {
64
+ res.writeHead(204);
65
+ res.end();
66
+ return;
67
+ }
68
+
69
+ const url = new URL(req.url, `http://${req.headers.host || 'localhost'}`);
70
+
71
+ // Healthcheck endpoint
72
+ if (req.method === 'GET' && (url.pathname === '/' || url.pathname === '/health')) {
73
+ res.writeHead(200, { 'Content-Type': 'application/json' });
74
+ res.end(JSON.stringify({
75
+ status: 'ok',
76
+ model: 'laya-multilingual',
77
+ version: PKG_VERSION,
78
+ protocol: 'TypeSafe Jev /v1/systemone compatible'
79
+ }));
80
+ return;
81
+ }
82
+
83
+ // TypeSafe Jev evaluation endpoint
84
+ if (req.method === 'POST' && url.pathname === '/v1/systemone') {
85
+ // Optional bearer token authentication
86
+ if (apiKey) {
87
+ const authHeader = req.headers['authorization'] || '';
88
+ if (authHeader !== `Bearer ${apiKey}`) {
89
+ res.writeHead(401, { 'Content-Type': 'application/json' });
90
+ res.end(JSON.stringify({
91
+ error: 'Unauthorized: missing or invalid bearer token in Authorization header.'
92
+ }));
93
+ return;
94
+ }
95
+ }
96
+
97
+ const chunks = [];
98
+ let size = 0;
99
+ req.on('data', chunk => {
100
+ chunks.push(chunk);
101
+ size += chunk.length;
102
+ if (size > 4 * 1024 * 1024) { // 4MB guard
103
+ res.writeHead(413, { 'Content-Type': 'application/json' });
104
+ res.end(JSON.stringify({ error: 'Payload too large (max 4MB).' }));
105
+ req.destroy();
106
+ }
107
+ });
108
+
109
+ req.on('end', async () => {
110
+ let payload;
111
+ try {
112
+ payload = JSON.parse(Buffer.concat(chunks).toString('utf8'));
113
+ } catch (e) {
114
+ res.writeHead(422, { 'Content-Type': 'application/json' });
115
+ res.end(JSON.stringify({
116
+ error: 'Unprocessable Entity: invalid JSON payload.'
117
+ }));
118
+ return;
119
+ }
120
+
121
+ const state = payload.state;
122
+ const questions = payload.questions;
123
+ const requestedModel = payload.model || null;
124
+
125
+ if (state === undefined || state === null) {
126
+ res.writeHead(422, { 'Content-Type': 'application/json' });
127
+ res.end(JSON.stringify({
128
+ error: "Unprocessable Entity: missing required 'state' field."
129
+ }));
130
+ return;
131
+ }
132
+
133
+ if (!questions || typeof questions !== 'object' || Array.isArray(questions)) {
134
+ res.writeHead(422, { 'Content-Type': 'application/json' });
135
+ res.end(JSON.stringify({
136
+ error: "Unprocessable Entity: 'questions' must be an object map of typed questions."
137
+ }));
138
+ return;
139
+ }
140
+
141
+ try {
142
+ const result = await laya.predict(state, questions, requestedModel);
143
+ res.writeHead(200, { 'Content-Type': 'application/json' });
144
+ res.end(JSON.stringify(result));
145
+ } catch (err) {
146
+ res.writeHead(422, { 'Content-Type': 'application/json' });
147
+ res.end(JSON.stringify({
148
+ error: `Evaluation failed: ${err.message || err}`
149
+ }));
150
+ }
151
+ });
152
+ return;
153
+ }
154
+
155
+ // 404 handler
156
+ res.writeHead(404, { 'Content-Type': 'application/json' });
157
+ res.end(JSON.stringify({
158
+ error: `Not found: ${req.method} ${url.pathname}. Expected POST /v1/systemone`
159
+ }));
160
+ });
161
+
162
+ // Keep-alive tuning: game clients (Tetris) fire one request per piece on
163
+ // the same connection. Long keep-alive avoids TCP+handshake per move.
164
+ server.keepAliveTimeout = 65000;
165
+ server.headersTimeout = 66000;
166
+ server.requestTimeout = 0;
167
+ server.maxRequestsPerSocket = 0;
168
+
169
+ return new Promise((resolve, reject) => {
170
+ server.listen(port, host, () => {
171
+ const displayHost = host === '0.0.0.0' ? 'localhost' : host;
172
+ // use the port actually bound (0 means "any free port")
173
+ const boundPort = server.address()?.port ?? port;
174
+ const url = `http://${displayHost}:${boundPort}`;
175
+ resolve({
176
+ server,
177
+ url,
178
+ laya,
179
+ close: async () => {
180
+ await new Promise((resolve) => {
181
+ server.close(() => resolve());
182
+ // Node keeps idle keep-alive sockets open (undici pools them for
183
+ // seconds), which would block server.close() — force them shut so
184
+ // shutdown is immediate and deterministic.
185
+ server.closeIdleConnections?.();
186
+ server.closeAllConnections?.();
187
+ });
188
+ if (ownsLaya && typeof laya.close === 'function') await laya.close();
189
+ }
190
+ });
191
+ });
192
+ server.on('error', reject);
193
+ });
194
+ }