arcane-os 0.65.0 → 0.67.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +47 -0
- package/README.md +1 -1
- package/browser-runtime/ai/browser-decisions.mjs +3 -2
- package/browser-runtime/ai/core-image.mjs +84 -34
- package/browser-runtime/ai/core-local.mjs +195 -43
- package/browser-runtime/ai/core-onnx.mjs +71 -12
- package/browser-runtime/ai/decision-runtime.mjs +14 -24
- package/docs/publishing.md +39 -0
- package/docs/reference/README.md +8 -1
- package/docs/reference/ai/browser-decisions.md +7 -4
- package/docs/reference/core-native-packaging.md +11 -4
- package/docs/reference/core-repositories.md +214 -11
- package/docs/reference/core-runtime.md +32 -5
- package/docs/reference/git-text-snapshot.md +15 -1
- package/docs/reference/inventory/package-api.json +41 -13
- package/docs/reference/inventory/runtime-modules.json +5 -4
- package/docs/reference/local-ai.md +119 -2
- package/docs/reference/local-image-generation.md +15 -4
- package/docs/reference/local-whisper.md +7 -1
- package/docs/reference/mcp-stdio.md +1 -1
- package/docs/reference/model-assets.md +50 -3
- package/docs/reference/native-decisions.md +181 -0
- package/docs/reference/native-speech.md +6 -0
- package/docs/reference/sdk-api.md +151 -16
- package/package.json +5 -2
- package/src/core/repositories.mjs +37 -14
- package/src/core/runtime.mjs +11 -0
- package/src/core/services/decisions.mjs +144 -0
- package/src/core/services/local-ai.mjs +31 -9
- package/src/core/services/model-assets.mjs +98 -13
- package/src/git-identity.mjs +96 -0
- package/src/git-text-snapshot.mjs +5 -3
- package/src/local-ai/decision-tokenizer-worker.mjs +31 -0
- package/src/local-ai/decision-tokenizer.mjs +109 -0
- package/src/local-ai/decisions.mjs +273 -0
- package/src/local-ai/dev.mjs +8 -2
- package/src/local-ai/onnx-worker.mjs +67 -7
- package/src/local-ai/onnx.mjs +23 -12
- package/src/native/portable-layout.mjs +8 -3
- package/src/repository.mjs +144 -4
|
@@ -70,6 +70,13 @@ assembly compose it through the shared speech service without changing the
|
|
|
70
70
|
browser's selected STT route. See [native Whisper](local-whisper.md) for the
|
|
71
71
|
complete configuration, platform, media and shutdown contracts.
|
|
72
72
|
|
|
73
|
+
Adding `encoder: 'openvino-npu'` selects the optional Intel NPU audio encoder
|
|
74
|
+
with a separately selected CUDA or CPU decoder. The Windows x64 precompiled
|
|
75
|
+
runtime uses paired GGML/XML/BIN model resources and the computer's installed
|
|
76
|
+
compatible Intel NPU driver. See the [Intel NPU Whisper guide](../guides/native-whisper-openvino.md)
|
|
77
|
+
for preparation, observed initialization, cancellation/reload and custom
|
|
78
|
+
runtime production. Omitting the encoder retains ordinary Whisper preparation.
|
|
79
|
+
|
|
73
80
|
The selected model stays loaded across recordings. An eligible native
|
|
74
81
|
accelerated failure may drain that helper and retry the same complete recording
|
|
75
82
|
once on the prepared CPU backend. That accepted request retains its actual
|
|
@@ -167,6 +174,34 @@ This factory accepts chat/ONNX requirements; the separate
|
|
|
167
174
|
owns image requirements. The development and generated native composition
|
|
168
175
|
split the shared application record accordingly.
|
|
169
176
|
|
|
177
|
+
### Native service owners
|
|
178
|
+
|
|
179
|
+
A native application service uses its Core lifecycle or request context to
|
|
180
|
+
obtain the existing selected owner:
|
|
181
|
+
|
|
182
|
+
```js
|
|
183
|
+
const localAI = await context.getService('local-ai');
|
|
184
|
+
const onnx = localAI.getONNXRuntime();
|
|
185
|
+
const modelAssets = await context.getService('model-assets');
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
`getService()` awaits only that service's existing startup. The synchronous
|
|
189
|
+
`getONNXRuntime()` returns its actual `load/run/unload/current/subscribe/close`
|
|
190
|
+
owner; it neither prepares another runtime nor loads a model. An unselected
|
|
191
|
+
engine reports `LOCAL_AI_RUNTIME_NOT_SELECTED`; an unavailable or recovering
|
|
192
|
+
owner reports `LOCAL_AI_RUNTIME_UNAVAILABLE`, and an aborted service lifetime
|
|
193
|
+
retains its abort reason. Development and generated native compositions register
|
|
194
|
+
one shared model-assets service when ONNX or image generation is selected.
|
|
195
|
+
|
|
196
|
+
Reacquire `getONNXRuntime()` on each explicit model load because runtime recovery
|
|
197
|
+
replaces that owner. Retain the acquired handle for that session's cleanup and
|
|
198
|
+
await its `unload({id})` without an already-aborted request signal before releasing
|
|
199
|
+
the corresponding model-assets retain handle. The native ONNX owner joins actual
|
|
200
|
+
worker exit; a cancelled request's rejection alone does not establish that exit.
|
|
201
|
+
Core closes services concurrently, and the host can cancel in-flight native work
|
|
202
|
+
at shutdown. These native members require no renderer readiness and introduce no
|
|
203
|
+
second inference engine. See [native Core composition](core-runtime.md#native-service-composition).
|
|
204
|
+
|
|
170
205
|
`localai.status` returns `{runtimes}` and, when Ollama is selected, the existing
|
|
171
206
|
`ollama` and `models.ollama` catalog fields. Runtime records distinguish
|
|
172
207
|
`installed`, `available`, `state`, `models`, ownership and errors. A listening
|
|
@@ -191,6 +226,14 @@ background health polling.
|
|
|
191
226
|
| `llama.unload` | `{model}` | Current state after release |
|
|
192
227
|
| `llama.chat` | `{model,payload,stream?,streamId?}` | Complete OpenAI-shaped completion |
|
|
193
228
|
|
|
229
|
+
`llama.status` and `llama.models` observe the selected service without loading,
|
|
230
|
+
restarting or waiting for it. Starting, loading, stopped and failed services
|
|
231
|
+
return their retained state/catalog; an available service supplies a fresh
|
|
232
|
+
catalog observation. The llama.cpp runtime record includes `released: true`
|
|
233
|
+
only after Core successfully closes its owned single-model server. Explicit
|
|
234
|
+
load, inference or recovery can resume that engine; observation cannot.
|
|
235
|
+
Status observation adds no background polling.
|
|
236
|
+
|
|
194
237
|
The `payload` is the complete request. SDK option names are translated only at
|
|
195
238
|
the upstream protocol boundary; messages, tool definitions and documents remain
|
|
196
239
|
unchanged. An explicit `payload.model` selects that request's model. Core waits
|
|
@@ -206,7 +249,8 @@ observe load, sleep and unload changes. Direct servers expose readiness at
|
|
|
206
249
|
request boundaries; they do not offer the router's passive model event stream.
|
|
207
250
|
|
|
208
251
|
Unloading a model from an owned single-model server closes that owned process;
|
|
209
|
-
a subsequent load can start it again.
|
|
252
|
+
a subsequent load can start it again. Repeating that unload returns the released
|
|
253
|
+
state without restarting the process. Releasing an externally owned direct
|
|
210
254
|
server returns `released: true, unloaded: false`, retaining the real catalog
|
|
211
255
|
and leaving that server running.
|
|
212
256
|
|
|
@@ -282,6 +326,38 @@ does not install software or select a model. A missing Core connection reports
|
|
|
282
326
|
unavailability, leaving browser Wllama/ONNX routes available for the application
|
|
283
327
|
to select.
|
|
284
328
|
|
|
329
|
+
Omit `client` to follow the installed Core client through
|
|
330
|
+
[`subscribeCoreClient`](core-client.md): construction may precede installation.
|
|
331
|
+
Each installation reads `llama.status` without selecting, downloading or loading
|
|
332
|
+
a model. Retirement or replacement synchronously revokes this provider's
|
|
333
|
+
`status().loaded`, cancels its owned operations and detaches the old lifecycle
|
|
334
|
+
listener. Late results cannot restore the retired selection. Explicitly load
|
|
335
|
+
the selection against the replacement client before inference. Passing a
|
|
336
|
+
`client`, including `null`, keeps that caller-supplied connection fixed.
|
|
337
|
+
|
|
338
|
+
`status()` is this provider's current state. The `arcane-ai-provider/2` contract
|
|
339
|
+
does not provide a provider-state subscription, so this installation handling
|
|
340
|
+
alone does not update an idle AI runtime's retained role state. The provider
|
|
341
|
+
runtime still reconciles the provider at its existing operation boundaries.
|
|
342
|
+
`inspect()` reports whether an explicit load can proceed, separately from
|
|
343
|
+
`status().loaded`. A genuine `starting` or `loading` service without an error
|
|
344
|
+
permits that load preflight even before its catalog arrives; Core's `llama.load`
|
|
345
|
+
waits for startup and verifies the exact requested model. An owned, successfully
|
|
346
|
+
released service permits loading only a model retained in its actual catalog.
|
|
347
|
+
An available service still requires that catalog membership. These observations
|
|
348
|
+
do not fabricate model records or readiness, and unavailable external services,
|
|
349
|
+
closed services and actual errors remain unavailable.
|
|
350
|
+
|
|
351
|
+
`dispose()` removes installation and service subscriptions and cancels owned
|
|
352
|
+
operations synchronously, then attempts its actually retained model's release
|
|
353
|
+
through its captured client. A configured selection alone does not own a native
|
|
354
|
+
model: disposing before load or after successful unload sends no native release,
|
|
355
|
+
including when the caller explicitly supplied `client: null`. A failed release
|
|
356
|
+
rejects with the actual error even though the provider is disposed; disposal
|
|
357
|
+
does not establish successful native unload.
|
|
358
|
+
It never unloads a retired selection through a replacement client or closes the
|
|
359
|
+
shared Core client.
|
|
360
|
+
|
|
285
361
|
If a router evicts the primary model while serving a request-specific model,
|
|
286
362
|
reload the primary selection before its next request. The provider does not
|
|
287
363
|
silently change the application's saved model choice. Release registration
|
|
@@ -309,13 +385,45 @@ try {
|
|
|
309
385
|
```
|
|
310
386
|
|
|
311
387
|
The factory accepts `{modulePath,signal?,onEvent?}` and starts no worker until
|
|
312
|
-
`load({id,model,sessionOptions?,signal?})`. The model path is absolute for this
|
|
388
|
+
`load({id,model,sessionOptions?,executionPreference?,signal?})`. The model path is absolute for this
|
|
313
389
|
direct Node API. Session creation defaults to the CPU execution provider;
|
|
314
390
|
explicit native session options remain caller-owned. The returned session
|
|
315
391
|
record contains `id`, `model`, `state`, `loaded`, `error`, input/output names and
|
|
316
392
|
metadata. A live ID requires explicit unload before replacement. An ID may be
|
|
317
393
|
loaded again once its previous worker exits after a failed load or unload.
|
|
318
394
|
A run error retains the loaded session so a corrected request can use it.
|
|
395
|
+
`current().sessions[].stopping` becomes `true` when terminal shutdown begins,
|
|
396
|
+
including worker failure or explicit cancellation/unload. The session is then
|
|
397
|
+
unavailable for new runs even while `loaded` remains `true` until actual exit.
|
|
398
|
+
A recoverable run error keeps `stopping:false` and the loaded session usable.
|
|
399
|
+
`current().sessions[].exited` reports actual worker exit, independently of a
|
|
400
|
+
rejected load or inference. Unload joins worker exit and output delivery even
|
|
401
|
+
when requesting termination fails; complete cleanup errors remain observable.
|
|
402
|
+
|
|
403
|
+
Select `executionPreference: 'gpu'` to try the installed runtime's advertised
|
|
404
|
+
CUDA, TensorRT, DirectML, CoreML and WebGPU providers in that order, followed by
|
|
405
|
+
CPU when no accelerator session can be created. Discovery runs once per load;
|
|
406
|
+
each advertised candidate receives one actual session-creation attempt with CPU
|
|
407
|
+
available for graph nodes that the accelerator cannot handle. No provider is
|
|
408
|
+
downloaded by selection. The upstream `bundled` flag describes packaging and
|
|
409
|
+
does not exclude separately installed CUDA or TensorRT libraries. An explicit
|
|
410
|
+
`sessionOptions.executionProviders`, including an empty array, takes precedence
|
|
411
|
+
and is passed unchanged without automatic discovery or retries. Automatic
|
|
412
|
+
DirectML attempts default `enableMemPattern` to `false` and `executionMode` to
|
|
413
|
+
`'sequential'` as required by that provider; explicit caller values remain
|
|
414
|
+
unchanged, including on the final CPU attempt.
|
|
415
|
+
|
|
416
|
+
Automatic GPU selection adds an `execution` record to the loaded session and
|
|
417
|
+
its ordinary state events: `preference`, complete `supportedBackends`,
|
|
418
|
+
`selectedProviders`, ordered `attempts` with their full errors, `fallback`, and
|
|
419
|
+
`discoveryError`. Each attempt records `executionProviders`, `status`
|
|
420
|
+
(`configured` or `failed`), and `error`. `fallback: true` means the SDK created
|
|
421
|
+
the final CPU session after GPU preference, including when no GPU candidate was
|
|
422
|
+
advertised. Successful creation establishes an accepted provider configuration;
|
|
423
|
+
it does not establish that a GPU executed any graph nodes. If every attempt
|
|
424
|
+
fails, the load error retains the complete attempt errors and `execution`
|
|
425
|
+
record. Inference failures retain their existing behavior and do not replay
|
|
426
|
+
inference on another provider.
|
|
319
427
|
|
|
320
428
|
`run({id,feeds,fetches?,runOptions?,signal?})` returns the complete output-name
|
|
321
429
|
map of `{type,dims,data}` tensors. Input records use that same shape with native
|
|
@@ -376,6 +484,15 @@ Closing the accessor cancels its requests and subscriptions; model unloading
|
|
|
376
484
|
is explicit because other accessors may share the Core session. It never
|
|
377
485
|
closes the shared Core client.
|
|
378
486
|
|
|
487
|
+
Omitting `client` follows Core installation events with current-client replay,
|
|
488
|
+
including installation after construction. Each new connection reads
|
|
489
|
+
`onnx.status`; it does not install a runtime or load a model. Retirement or
|
|
490
|
+
replacement immediately clears retained session readiness and cancels this
|
|
491
|
+
accessor's operations. Late old-client state and results are ignored. Passing
|
|
492
|
+
an explicit `client`, including `null`, retains caller ownership and does not
|
|
493
|
+
follow global installation changes. `close()` also removes the installation
|
|
494
|
+
subscription.
|
|
495
|
+
|
|
379
496
|
Construction neither opens a connection nor installs a runtime. A browser needs
|
|
380
497
|
an already available native Core service or the explicit development bridge.
|
|
381
498
|
Unavailable native ONNX leaves browser ONNX/Wllama under the application's
|
|
@@ -199,14 +199,16 @@ ordered `Blob` or `File` members after its resumable download. Range support
|
|
|
199
199
|
allows concurrent transfer within a member; those transfer parts remain the
|
|
200
200
|
store's concern. Use `offline: true` to require an already complete stored model.
|
|
201
201
|
The store does not select or run the image engine. In source development,
|
|
202
|
-
|
|
203
|
-
in [Local AI through Core](local-ai.md#development)
|
|
204
|
-
|
|
202
|
+
use the development Core bootstrap described
|
|
203
|
+
in [Local AI through Core](local-ai.md#development). The accessor can be
|
|
204
|
+
constructed before that client is installed; perform model operations once
|
|
205
|
+
the service is available. Native hosts install their Core client through the
|
|
206
|
+
existing host path.
|
|
205
207
|
An ordinary browser without Core reports local image generation as unavailable.
|
|
206
208
|
|
|
207
209
|
| Member | Contract |
|
|
208
210
|
|---|---|
|
|
209
|
-
| `createCoreImageRuntime({client?, signal?, onEvent?})` |
|
|
211
|
+
| `createCoreImageRuntime({client?, signal?, onEvent?})` | Follows Core installation and lifecycle events by default, asynchronously reading status for each client. An explicit `client`, including `null`, remains caller-owned. It returns immediately. |
|
|
210
212
|
| `load({model, context?, assetProjectionId?, resourcePaths?, signal?})` | Loads the selected model and returns authoritative service state after native completion. `context` supplies native context options. |
|
|
211
213
|
| `generate({model, prompt, parameters?, assetProjectionId?, resourcePaths?, signal?, onProgress?})` | Returns the complete result with `images: [{blob, mediaType, width, height}]`. Additional runtime result metadata remains present. |
|
|
212
214
|
| `unload({signal?})` | Explicitly unloads the shared selected native context and returns its resulting state. |
|
|
@@ -229,6 +231,15 @@ model definitions, including their resource metadata before engine preparation
|
|
|
229
231
|
finishes. Installed files do not imply a loaded model. A late status
|
|
230
232
|
response never overwrites a newer lifecycle event.
|
|
231
233
|
|
|
234
|
+
With `client` omitted, the accessor consumes the shared
|
|
235
|
+
[`subscribeCoreClient`](core-client.md) installation stream. A late installation
|
|
236
|
+
attaches the service listener and reads current status; it never installs an
|
|
237
|
+
engine or loads a model. Retirement or replacement synchronously clears
|
|
238
|
+
readiness and transient `Thinking`, cancels owned requests and detaches the
|
|
239
|
+
retired service listener. Old state, progress and results cannot update the
|
|
240
|
+
replacement connection. `close()` also removes the installation subscription;
|
|
241
|
+
it does not claim that the shared native model was unloaded.
|
|
242
|
+
|
|
232
243
|
Prompts and parameters pass unchanged to the native runtime. Native option
|
|
233
244
|
names, supported model operations and defaults belong to that runtime and the
|
|
234
245
|
selected model definition. A prompt containing U+0000 or an unpaired UTF-16
|
|
@@ -7,6 +7,12 @@ the model or recording files are reused. The browser's existing
|
|
|
7
7
|
`AI.fetchSTT(audioFile, signal)` and Core `speech.transcribe` routes retain the
|
|
8
8
|
application's selected model and cancellation signal.
|
|
9
9
|
|
|
10
|
+
For the optional Intel NPU encoder, add `encoder: "openvino-npu"` to the
|
|
11
|
+
runtime record. The [Intel NPU guide](../guides/native-whisper-openvino.md)
|
|
12
|
+
covers the Windows x64 precompiled distribution, paired GGML/XML/BIN model
|
|
13
|
+
resources, independent CUDA/CPU decoding and joined cancellation. Omitting
|
|
14
|
+
`encoder` preserves the ordinary runtime and model selection below.
|
|
15
|
+
|
|
10
16
|
## Select the runtime and model
|
|
11
17
|
|
|
12
18
|
An application's `arcane-app.json` selects native speech alongside its other
|
|
@@ -130,7 +136,7 @@ prepared descriptor returned by `ensureLocalAIRuntimes`; alternatively,
|
|
|
130
136
|
|
|
131
137
|
| Member | Contract |
|
|
132
138
|
| --- | --- |
|
|
133
|
-
| `current()` | Returns `providerId`, selected `modelId`, lifecycle `state`, `loaded`, `busy`, correlated `requestId` or null,
|
|
139
|
+
| `current()` | Returns `providerId`, selected `modelId`, lifecycle `state`, `loaded`, `busy`, correlated `requestId` or null, `requestedBackend`, `observedBackend`, `backendEvidence`, `requestedEncoder`, `observedEncoder`, `encoderEvidence` and any error. Encoder initialization evidence is separate from decoder evidence. |
|
|
134
140
|
| `subscribe(listener, {replay = true, signal} = {})` | Immediately replays current state by default and returns an unsubscribe function. |
|
|
135
141
|
| `load({modelId, signal} = {})` | Prepares the runtime if necessary, then waits for actual native model readiness. With no selected model it remains unloaded. |
|
|
136
142
|
| `transcribe(request, {signal, onProgress, requestId} = {})` | Consumes the complete `audioBase64` recording. Optional `language` defaults to native automatic detection; `translate: true` selects Whisper's translation operation. `model` must match the loaded model. Optional `requestId` carries the caller's existing operation identity through lifecycle snapshots as separate control information; it does not alter the recording or request payload. |
|
|
@@ -216,7 +216,7 @@ normal close or EOF and rejects with an `AggregateError` when transport or
|
|
|
216
216
|
diagnostic delivery failed. Observe it even when the initiating caller continues
|
|
217
217
|
other work. Request failures alone do not close the session.
|
|
218
218
|
|
|
219
|
-
## MCP_PROTOCOL_VERSION
|
|
219
|
+
## `MCP_PROTOCOL_VERSION`
|
|
220
220
|
|
|
221
221
|
`MCP_PROTOCOL_VERSION` is exactly `'2025-11-25'`. Initialization always returns
|
|
222
222
|
that supported version. A client proposing another version must accept this
|
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# Native working files from
|
|
1
|
+
# Native working files from model assets
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Browser preparation and native retain/release ownership were first published in `arcane-os@0.61.0`. Browser
|
|
4
4
|
preparation is exported by `arcane-os/ai/core-model-assets`, and the native
|
|
5
5
|
service by `arcane-os/core/model-assets`. The image example below uses
|
|
6
6
|
`arcane-os/ai/core-image`, published in the same release.
|
|
@@ -87,6 +87,53 @@ cleanup failures remain observable.
|
|
|
87
87
|
|
|
88
88
|
## Native ownership
|
|
89
89
|
|
|
90
|
+
Native services can prepare explicitly selected upstream members without a
|
|
91
|
+
renderer or browser store. Obtain the registered shared service through
|
|
92
|
+
`await context.getService('model-assets')`, then use its native preparation API:
|
|
93
|
+
|
|
94
|
+
```javascript
|
|
95
|
+
const modelAssets = await context.getService('model-assets');
|
|
96
|
+
const projection = await modelAssets.prepare({
|
|
97
|
+
id: selectedOperationId,
|
|
98
|
+
workingDirectory: selectedWorkingDirectory,
|
|
99
|
+
members: selectedMembers, // [{path: 'onnx/model.onnx', url: selectedURL}, ...]
|
|
100
|
+
signal,
|
|
101
|
+
onProgress: renderPreparationProgress
|
|
102
|
+
});
|
|
103
|
+
const use = modelAssets.retain(projection.id);
|
|
104
|
+
await modelAssets.release(projection.id);
|
|
105
|
+
try {
|
|
106
|
+
await ownNativeModelLifetime(use.members);
|
|
107
|
+
} finally {
|
|
108
|
+
// Join actual native unload/worker exit before releasing these files.
|
|
109
|
+
await use.release();
|
|
110
|
+
}
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
`prepare({id,workingDirectory,members,signal,onProgress})` performs standard
|
|
114
|
+
fetches only when explicitly called. Every member has its original relative
|
|
115
|
+
`path` and selected `url`; the caller supplies the complete companion-file set.
|
|
116
|
+
Independent members download concurrently. Each complete response stream is
|
|
117
|
+
written directly through the existing ordered native file owner, without a
|
|
118
|
+
whole-model buffer or base64 conversion. No model or runtime is selected or
|
|
119
|
+
downloaded by service startup. The result is the ready projection snapshot with
|
|
120
|
+
ordered `members: [{path,nativePath}]`.
|
|
121
|
+
|
|
122
|
+
Progress uses `open`, `download`, `complete` and `ready` phases, with
|
|
123
|
+
`completed`, `total` and `unit:'files'`; member progress additionally includes
|
|
124
|
+
`memberIndex` and `path`. Progress callbacks may return a promise, which the
|
|
125
|
+
preparation observes. They must not await cleanup of the preparation currently
|
|
126
|
+
calling them. Callback failures remain preparation failures. HTTP failures
|
|
127
|
+
retain `url`, `status` and the complete response text in `response`.
|
|
128
|
+
|
|
129
|
+
`release(id)` relinquishes preparation ownership without a request signal.
|
|
130
|
+
For active native downloads, release or disposal aborts their fetches and joins
|
|
131
|
+
the transfer tasks and native writes before deleting only their owned child
|
|
132
|
+
directory. Preparation cancellation and failure follow that same cleanup path;
|
|
133
|
+
the original failures and cleanup failures remain observable. A native retain
|
|
134
|
+
handle continues to protect completed files through actual engine release.
|
|
135
|
+
Native preparation leaves browser DBOPFS originals and unrelated files alone.
|
|
136
|
+
|
|
90
137
|
```javascript
|
|
91
138
|
import {createModelAssetService} from 'arcane-os/core/model-assets';
|
|
92
139
|
|
|
@@ -103,7 +150,7 @@ try {
|
|
|
103
150
|
```
|
|
104
151
|
|
|
105
152
|
`createModelAssetService({appRoot})` returns a Core service with `current()`,
|
|
106
|
-
`retain(id)` and `dispose()`. `retain()` accepts only a completed projection
|
|
153
|
+
`prepare(options)`, `retain(id)`, `release(id)` and `dispose()`. `retain()` accepts only a completed projection
|
|
107
154
|
and returns `{id,directory,members,release}`. A native engine takes this handle
|
|
108
155
|
before loading and releases it after actual unload, failed-load cleanup or
|
|
109
156
|
worker exit. A rejected inference/load promise alone does not establish native
|
|
@@ -0,0 +1,181 @@
|
|
|
1
|
+
# Native Laya typed decisions
|
|
2
|
+
|
|
3
|
+
`arcane-os/local-ai/decisions` and `arcane-os/core/decisions` run the selected
|
|
4
|
+
Laya FP32 graph through the SDK's existing native ONNX owner. Tokenization runs
|
|
5
|
+
in its own Node Worker. A Core service can load and evaluate decisions without
|
|
6
|
+
a renderer, browser session, WebGPU browser context or chat provider.
|
|
7
|
+
|
|
8
|
+
The application owns the model selection, original state text, questions,
|
|
9
|
+
options and interpretation. The SDK owns model preparation, token framing,
|
|
10
|
+
inference and cleanup. This is option scoring, not a conversational LLM, a
|
|
11
|
+
replacement for a configured chat model or the cloud System One API.
|
|
12
|
+
|
|
13
|
+
## Compose with existing Core services
|
|
14
|
+
|
|
15
|
+
Register a decision service alongside the selected local-AI and model-assets
|
|
16
|
+
services in the application's existing Core runtime:
|
|
17
|
+
|
|
18
|
+
```js
|
|
19
|
+
import {createNativeDecisionService} from 'arcane-os/core/decisions';
|
|
20
|
+
|
|
21
|
+
const decisions = createNativeDecisionService({
|
|
22
|
+
model: 'onnx-community/laya-typed-decisions-ONNX',
|
|
23
|
+
dtype: 'fp32',
|
|
24
|
+
workingDirectory: '.models/laya'
|
|
25
|
+
}, {appRoot});
|
|
26
|
+
|
|
27
|
+
// Include decisions in the existing createCoreRuntime({services}) selection.
|
|
28
|
+
// That runtime already owns local-ai with ONNX selected and model-assets.
|
|
29
|
+
await runtime.getService('decisions');
|
|
30
|
+
await decisions.load(); // Explicit activation; no model download at registration.
|
|
31
|
+
try {
|
|
32
|
+
const result = await decisions.evaluate([{
|
|
33
|
+
state: 'The moon chef has one kettle and seventeen impatient dragons.',
|
|
34
|
+
question: 'What should the chef prepare?',
|
|
35
|
+
options: ['Tea', 'An inflatable castle']
|
|
36
|
+
}]);
|
|
37
|
+
console.log(result.decisions[0].value);
|
|
38
|
+
} finally {
|
|
39
|
+
await decisions.unload();
|
|
40
|
+
}
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Here `runtime` and `appRoot` are the application's existing Core runtime and
|
|
44
|
+
root. Add the service before starting that runtime; the example does not create
|
|
45
|
+
a second Core host or engine. Development and packaged native compositions
|
|
46
|
+
already register one model-assets service when ONNX is selected. The decision
|
|
47
|
+
service obtains those exact owners with `context.getService('local-ai')`,
|
|
48
|
+
`localAI.getONNXRuntime()` and `context.getService('model-assets')` on explicit
|
|
49
|
+
load. It reacquires the current ONNX owner after a later runtime recovery.
|
|
50
|
+
See [Core composition](core-runtime.md#native-service-composition) and
|
|
51
|
+
[native service owners](local-ai.md#native-service-owners).
|
|
52
|
+
|
|
53
|
+
`createNativeDecisionService(configuration, {appRoot})` defaults its service
|
|
54
|
+
name to `decisions`. It exposes `load`, `evaluate`, `classify` (an alias for
|
|
55
|
+
`evaluate`), `current`, `subscribe`, `unload` and `dispose`, plus Core lifecycle
|
|
56
|
+
`start(context)`. Its RPC methods are:
|
|
57
|
+
|
|
58
|
+
| Method | Parameters and result |
|
|
59
|
+
| --- | --- |
|
|
60
|
+
| `decisions.status` | Current lifecycle snapshot; does not load anything. |
|
|
61
|
+
| `decisions.load` | Explicit activation; returns the ready lifecycle snapshot. The request signal cancels activation. |
|
|
62
|
+
| `decisions.evaluate` | `{rows, runOptions?}`; returns `{decisions, outputs}`. The request signal cancels the active model operation. |
|
|
63
|
+
| `decisions.unload` | Releases the activation. Service-lifetime cleanup continues after the caller disconnects. |
|
|
64
|
+
|
|
65
|
+
RPC evaluation uses the SDK's existing complete JSON tensor encoding for
|
|
66
|
+
`outputs` at the transport boundary. Direct native evaluation returns native
|
|
67
|
+
typed tensor records. Service notifications use `decisions.state`; direct
|
|
68
|
+
`subscribe(listener, {emitCurrent:true, signal})` replays current state by
|
|
69
|
+
default and returns an unsubscribe function. Startup stores the service
|
|
70
|
+
context only: importing or registering this service starts no tokenizer,
|
|
71
|
+
model download or inference.
|
|
72
|
+
|
|
73
|
+
## Direct native model
|
|
74
|
+
|
|
75
|
+
`createNativeDecisionModel(options)` from `arcane-os/local-ai/decisions` returns
|
|
76
|
+
`{load, evaluate, classify, current, status, subscribe, unload, dispose}`.
|
|
77
|
+
`status` aliases `current`; `classify` aliases `evaluate`. Supply the existing
|
|
78
|
+
native `onnx` owner, plus either the existing `modelAssets` owner and selected
|
|
79
|
+
`workingDirectory`, or caller-owned native files:
|
|
80
|
+
|
|
81
|
+
```js
|
|
82
|
+
import {createNativeDecisionModel} from 'arcane-os/local-ai/decisions';
|
|
83
|
+
|
|
84
|
+
const decisions = createNativeDecisionModel({
|
|
85
|
+
onnx: localAI.getONNXRuntime(),
|
|
86
|
+
paths: {
|
|
87
|
+
model: selectedGraphPath,
|
|
88
|
+
tokenizer: selectedTokenizerPath,
|
|
89
|
+
tokenizerConfig: selectedTokenizerConfigPath
|
|
90
|
+
}
|
|
91
|
+
});
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
These direct paths are native filesystem paths. Keep the graph's external data
|
|
95
|
+
beside it under the exact filename required by the graph. The Core service
|
|
96
|
+
resolves configured `paths` relative to `appRoot`; it does not copy or remove
|
|
97
|
+
caller-owned files. For SDK-prepared files, the default selected upstream is
|
|
98
|
+
`onnx-community/laya-typed-decisions-ONNX`, revision `main`, with exactly:
|
|
99
|
+
|
|
100
|
+
- `onnx/model.onnx` and `onnx/model.onnx_data`;
|
|
101
|
+
- `tokenizer.json` and `tokenizer_config.json`.
|
|
102
|
+
|
|
103
|
+
`model` selects the upstream Hugging Face repository and `revision` its
|
|
104
|
+
revision. Native preparation performs normal upstream fetches only during
|
|
105
|
+
explicit `load()`. It streams complete members through the existing working
|
|
106
|
+
file owner; it neither vendors the weights nor creates a second persistent
|
|
107
|
+
model store. The approved `@huggingface/tokenizers@0.2.0` dependency reads the
|
|
108
|
+
selected tokenizer files in its Worker. No Transformers or second ONNX runtime
|
|
109
|
+
is loaded by this API.
|
|
110
|
+
|
|
111
|
+
This graph selection requires `dtype:'fp32'`, which is also the default; another
|
|
112
|
+
dtype reports the incompatible selection rather than substituting a graph.
|
|
113
|
+
`sessionOptions` and `runOptions` pass to the existing native ONNX API unchanged.
|
|
114
|
+
`executionPreference` defaults to `gpu` here: actual advertised native GPU
|
|
115
|
+
provider creation is attempted, followed by an honest CPU fallback. Explicit
|
|
116
|
+
`sessionOptions.executionProviders` takes precedence. The underlying generic
|
|
117
|
+
ONNX factory continues to default to CPU. Accepted provider configuration is
|
|
118
|
+
not evidence that GPU nodes executed. Full provider-attempt diagnostics remain
|
|
119
|
+
in `execution`; see [native ONNX sessions](local-ai.md#native-onnx-sessions).
|
|
120
|
+
|
|
121
|
+
## Complete inputs and outputs
|
|
122
|
+
|
|
123
|
+
`evaluate(rows, {signal, runOptions})` requires an explicitly loaded or loading
|
|
124
|
+
activation; it does not silently load an inactive model. Rows retain their
|
|
125
|
+
caller order, complete `state`, `question` and string-array `options`, plus
|
|
126
|
+
optional `type` (`choice`, `score` or `noul`). Additional row fields remain in
|
|
127
|
+
the returned row. Strings are not trimmed, rewritten or clipped. The shared
|
|
128
|
+
browser/native encoding owner constructs only the model's necessary token
|
|
129
|
+
framing; repeated identical state is tokenized once per batch. Tokenization
|
|
130
|
+
and ONNX execution run outside Core's foreground thread.
|
|
131
|
+
|
|
132
|
+
The result is `{decisions, outputs}`. Each decision contains its original
|
|
133
|
+
`row`, complete option `logits`, unrounded raw-softmax `probabilities`,
|
|
134
|
+
`answerIndex` and `value`. `choice` returns the selected complete option;
|
|
135
|
+
`score` returns the probability-weighted zero-based option index; `noul`
|
|
136
|
+
requires `[falseOption, trueOption]` and returns the probability of option 1.
|
|
137
|
+
When the graph actually supplies `act_logits`, action logits, probabilities
|
|
138
|
+
and `actProbability` are also returned. Every named graph output is retained
|
|
139
|
+
as `{type, dims, data}`, including extra tensors and padding. Raw model scores
|
|
140
|
+
are not claimed calibrated confidence. Results do not enter chat history,
|
|
141
|
+
memory extraction or DBOPFS automatically.
|
|
142
|
+
|
|
143
|
+
The model's real graph/context limits still apply. There is no SDK token cap,
|
|
144
|
+
silent chunking or fabricated result aggregation. Native/browser token-ID or
|
|
145
|
+
numerical parity requires execution evidence for the selected tokenizer and
|
|
146
|
+
runtime; sharing the encoding implementation alone does not establish it.
|
|
147
|
+
|
|
148
|
+
## Lifetime and cancellation
|
|
149
|
+
|
|
150
|
+
`current()` returns `{family, model, revision, dtype, state, loaded, busy,
|
|
151
|
+
activeRequests, progress, error, execution}`. States include `unloaded`,
|
|
152
|
+
`loading`, `ready`, `unloading`, `disposing`, `disposed` and `error`. Semantic
|
|
153
|
+
progress includes model preparation, tokenization and evaluation; downloaded
|
|
154
|
+
members report files, not byte progress. Complete technical errors belong in
|
|
155
|
+
developer diagnostics, while the application owns its user-facing status.
|
|
156
|
+
|
|
157
|
+
Concurrent callers share one explicit activation. Cancelling a load or
|
|
158
|
+
evaluation cancels that activation and its outstanding operations, because the
|
|
159
|
+
native session and tokenizer have one owner. Use separately owned models when
|
|
160
|
+
independent cancellation is required. The construction `signal` governs the
|
|
161
|
+
whole model lifetime. `unload()` permits another explicit load after successful
|
|
162
|
+
cleanup; `dispose()` is terminal. Readiness is revoked as disposal begins.
|
|
163
|
+
|
|
164
|
+
Retained model files remain owned until the ONNX and tokenizer Workers really
|
|
165
|
+
exit. A rejected load, inference or termination request alone does not permit
|
|
166
|
+
file deletion. Cleanup joins active operations and native output delivery,
|
|
167
|
+
preserves cleanup errors, then releases only its working projection. It never
|
|
168
|
+
deletes supplied `paths`. A failed ordinary row does not destroy a healthy
|
|
169
|
+
loaded session or cancel sibling evaluations. The ONNX owner's `stopping:true`
|
|
170
|
+
retires decision readiness immediately on terminal Worker failure or shutdown,
|
|
171
|
+
even while its physical `loaded` state remains true pending actual exit. Cancelled or
|
|
172
|
+
retired operations cannot return a successful late result.
|
|
173
|
+
|
|
174
|
+
Shared Node/Worker/filesystem code is portable across Windows, Linux and macOS;
|
|
175
|
+
the selected ONNX Runtime build and provider determine actual platform/device
|
|
176
|
+
availability. Android requires the host's supported native adaptation. Static
|
|
177
|
+
source review and synthetic authored fixtures are not model execution or a
|
|
178
|
+
claim of tested platform, precision or accelerator behavior.
|
|
179
|
+
|
|
180
|
+
Upstream selection: [Laya FP32 model files](https://huggingface.co/onnx-community/laya-typed-decisions-ONNX/tree/main),
|
|
181
|
+
[Tokenizer package](https://www.npmjs.com/package/@huggingface/tokenizers).
|
|
@@ -26,6 +26,12 @@ selection, supported formats, and native process ownership remain with the
|
|
|
26
26
|
selected engine and its host composition. Browser-only operation remains
|
|
27
27
|
independent of this native service.
|
|
28
28
|
|
|
29
|
+
The SDK's [retained Whisper engine](local-whisper.md) supplies native STT.
|
|
30
|
+
Its optional [Intel NPU encoder](../guides/native-whisper-openvino.md) uses the
|
|
31
|
+
same service, readiness events, complete request/result contract and cancellation
|
|
32
|
+
path. The host selects that encoder and its paired model resources; the browser
|
|
33
|
+
continues calling `speech.transcribe` without loading native libraries itself.
|
|
34
|
+
|
|
29
35
|
## Startup and status
|
|
30
36
|
|
|
31
37
|
`start(context)` subscribes to each configured engine and starts their loads
|