arcane-os 0.92.0 → 0.94.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +32 -0
- package/README.md +1 -1
- package/browser-runtime/ai/twin-cloud.mjs +109 -46
- package/browser-runtime/core/client.mjs +20 -0
- package/browser-runtime/core/contracts.mjs +20 -0
- package/browser-runtime/core/native-diagnostics.mjs +217 -0
- package/docs/reference/ai/twin-cloud.md +35 -14
- package/docs/reference/availability-and-normalization.md +1 -1
- package/docs/reference/core/arcane-api.md +32 -0
- package/docs/reference/core/arcane-events.md +25 -0
- package/docs/reference/core-client.md +99 -0
- package/docs/reference/inventory/package-api.json +97 -13
- package/docs/reference/local-ai.md +81 -5
- package/docs/reference/native-diagnostics.md +167 -0
- package/docs/reference/native-speech.md +220 -11
- package/docs/reference/runtime-modules.md +21 -18
- package/docs/reference/sdk-api.md +152 -11
- package/package.json +4 -1
- package/schemas/arcane-app.schema.json +2 -2
- package/schemas/arcane-package.schema.json +2 -2
- package/src/core/app-control.mjs +1 -0
- package/src/core/hosts/windows/ArcaneAppControl.cs +1 -1
- package/src/core/hosts/windows/ArcaneHost.cs +167 -0
- package/src/core/hosts/windows/ArcaneLauncher.cs +1 -0
- package/src/core/hosts/windows/ArcaneNativeDiagnostics.cs +137 -0
- package/src/core/hosts/windows/ArcaneNotifications.cs +822 -0
- package/src/core/hosts/windows/ArcaneWindowControl.cs +3 -2
- package/src/core/services/speech.mjs +85 -23
- package/src/core-native-diagnostics-source.mjs +13 -0
- package/src/import-map.mjs +1 -0
- package/src/local-ai/config.mjs +12 -1
- package/src/local-ai/dev.mjs +71 -4
- package/src/local-ai/install.mjs +113 -5
- package/src/local-ai/kokoro/frontend.mjs +219 -0
- package/src/local-ai/kokoro/helper.mjs +168 -0
- package/src/local-ai/kokoro/index.mjs +645 -0
- package/src/local-ai/kokoro/native/CMakeLists.txt +45 -0
- package/src/local-ai/kokoro/native/main.cpp +249 -0
- package/src/native/portable-layout.mjs +67 -9
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
# Native application diagnostics
|
|
2
|
+
|
|
3
|
+
Native diagnostics retain errors from the actual packaged application's top-level
|
|
4
|
+
document in its native host session. Read them through the existing app-control
|
|
5
|
+
connection, without foregrounding a window, desktop input, or arbitrary script
|
|
6
|
+
execution. This is a developer inspection surface, separate from ordinary product
|
|
7
|
+
status and saved conversation history.
|
|
8
|
+
|
|
9
|
+
The Windows WebView2 host installs the fixed observer at document creation, before
|
|
10
|
+
the ordinary Core client and application scripts. It captures `window.error` and
|
|
11
|
+
`unhandledrejection` events and accepts explicit reports of application-caught
|
|
12
|
+
errors. It leaves the browser's ordinary error behavior intact. It does not
|
|
13
|
+
intercept console output, inspect application state, collect resource/request
|
|
14
|
+
bodies, or capture errors from other applications or frame documents.
|
|
15
|
+
|
|
16
|
+
## Report a caught application error
|
|
17
|
+
|
|
18
|
+
```js
|
|
19
|
+
import {reportNativeError} from 'arcane-os/core/native-diagnostics';
|
|
20
|
+
|
|
21
|
+
const receipt = await reportNativeError(error, {
|
|
22
|
+
details: {operation: 'source.load', sourceId}
|
|
23
|
+
});
|
|
24
|
+
// receipt: {accepted: true, sessionId, sequence}
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
`error` is the actual caught value, including non-Error rejection values.
|
|
28
|
+
`details` is optional application-selected diagnostic context. Supply diagnostic
|
|
29
|
+
information only; credentials, secrets, user documents and provider/tool payloads
|
|
30
|
+
do not belong in this report. The SDK adds document/session metadata outside the
|
|
31
|
+
reported values and does not rewrite their text. Reporting does not change the
|
|
32
|
+
application's own handling of the original failure.
|
|
33
|
+
|
|
34
|
+
The promise resolves after the native host retains the record. A missing document
|
|
35
|
+
observer or native handler rejects with
|
|
36
|
+
`ARCANE_NATIVE_DIAGNOSTICS_UNAVAILABLE`. A negative host acknowledgement rejects
|
|
37
|
+
with `ARCANE_NATIVE_DIAGNOSTIC_REJECTED`, preserving the complete host reply in
|
|
38
|
+
`details` and its error in `cause`. Reflection, encoding and native transport
|
|
39
|
+
failures reject with their actual error. A lost reply can leave the caller's
|
|
40
|
+
outcome unknown; the SDK does not automatically retry and duplicate the report.
|
|
41
|
+
|
|
42
|
+
Automatic capture observes its own reporting failures through the existing
|
|
43
|
+
console, with the original error, without recursively manufacturing an unhandled
|
|
44
|
+
rejection. Console output remains ordinary console output; it is not retained
|
|
45
|
+
diagnostic history. Applications must observe explicit reporting promises.
|
|
46
|
+
|
|
47
|
+
## Retrieve retained errors
|
|
48
|
+
|
|
49
|
+
Use the endpoint of the selected running app from its existing app-control launch
|
|
50
|
+
configuration. Connecting does not launch the application.
|
|
51
|
+
|
|
52
|
+
```js
|
|
53
|
+
import {connectAppControl} from 'arcane-os/core/app-control';
|
|
54
|
+
|
|
55
|
+
const control = await connectAppControl({endpoint});
|
|
56
|
+
try {
|
|
57
|
+
const snapshot = await control.diagnostics();
|
|
58
|
+
console.dir(snapshot, {depth: null});
|
|
59
|
+
} finally {
|
|
60
|
+
await control.close();
|
|
61
|
+
}
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
`control.diagnostics({afterSequence?, documentId?}, {signal?})` invokes
|
|
65
|
+
`app.control.diagnostics`. Omitted filters return every retained error.
|
|
66
|
+
`afterSequence` selects records with a greater session sequence;
|
|
67
|
+
`documentId` selects one actual document. Retrieval is non-consuming: independent
|
|
68
|
+
inspectors can read the same complete records. Connection/cancellation behavior is
|
|
69
|
+
the existing app-control contract. Retrieval does not wait for renderer, Core,
|
|
70
|
+
model or application readiness and does not evaluate a script in the document.
|
|
71
|
+
|
|
72
|
+
The result is:
|
|
73
|
+
|
|
74
|
+
```js
|
|
75
|
+
{
|
|
76
|
+
sessionId,
|
|
77
|
+
captureStartedAt,
|
|
78
|
+
captureInstalled,
|
|
79
|
+
latestSequence,
|
|
80
|
+
documents: [{documentId, documentUrl, time, receivedAt}],
|
|
81
|
+
records: [{
|
|
82
|
+
sequence, receivedAt, documentId, documentUrl, time,
|
|
83
|
+
kind, error, details, valueFormat, values, location, message
|
|
84
|
+
}]
|
|
85
|
+
}
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
`sessionId` identifies this native window/host lifetime. `captureStartedAt` is the
|
|
89
|
+
time its in-memory ledger was created. `captureInstalled` means document-start
|
|
90
|
+
script registration completed; it does not prove that any application failure
|
|
91
|
+
occurred or that a particular operation succeeded. `documents` records actual
|
|
92
|
+
ingress from observed documents. Each document has one identity, shared with app
|
|
93
|
+
control, independent of Core bridge generations. A cancelled navigation that
|
|
94
|
+
leaves the same document alive keeps that identity.
|
|
95
|
+
|
|
96
|
+
`sequence` is assigned when an error is retained, in host arrival order.
|
|
97
|
+
`latestSequence` describes the whole session even when filters select fewer
|
|
98
|
+
records. `time` comes from document capture; `receivedAt` comes from the host.
|
|
99
|
+
`kind` is `window.error`, `unhandledrejection`, or `application`. Window errors
|
|
100
|
+
include `location: {file, line, column}` when the browser exposes those fields.
|
|
101
|
+
Their browser `message` is retained separately from `error`, preserving a null
|
|
102
|
+
or undefined error value rather than replacing it with that message.
|
|
103
|
+
`details` and `location` are otherwise omitted.
|
|
104
|
+
|
|
105
|
+
Records survive document navigation and Core transport failure within the host
|
|
106
|
+
session. Closing the inspector leaves them retained. Closing the native host ends
|
|
107
|
+
the session; this API does not persist, migrate, prune, or recover records across
|
|
108
|
+
host restarts. An empty snapshot means no matching records were retained in this
|
|
109
|
+
session, not that earlier failures never occurred. Errors before capture existed,
|
|
110
|
+
handled failures never explicitly reported, errors the browser does not expose,
|
|
111
|
+
and vanished documents whose pending reports never reached the host remain
|
|
112
|
+
unavailable. Earlier never-retained failures cannot be reconstructed through this API.
|
|
113
|
+
|
|
114
|
+
## Complete error property representation
|
|
115
|
+
|
|
116
|
+
Each record uses `valueFormat: 'arcane.diagnostic-value/1'`. `error` and optional
|
|
117
|
+
`details` are roots into the single `values` graph. JSON-native primitives remain
|
|
118
|
+
their original values. Undefined, bigint, non-finite numbers and negative zero
|
|
119
|
+
have explicit typed representations. `{ref: id}` identifies a graph node.
|
|
120
|
+
Repeated references and cycles retain the same identity within the record.
|
|
121
|
+
|
|
122
|
+
Object nodes retain every own property descriptor, including non-enumerable and
|
|
123
|
+
symbol-keyed fields. This includes full Error messages and stacks, `cause`,
|
|
124
|
+
AggregateError members and custom diagnostic details. Standard error fields
|
|
125
|
+
inherited from a prototype appear in `inherited`. Data descriptors contain
|
|
126
|
+
`value`; accessor descriptors contain references for `get` and `set` without
|
|
127
|
+
invoking either function. An accessor supplied as the `details` option is likewise
|
|
128
|
+
represented rather than called. No `toJSON` method on reported content is called.
|
|
129
|
+
Maps and sets retain their entries, dates their numeric time, and regular
|
|
130
|
+
expressions their source and flags, alongside their own properties.
|
|
131
|
+
|
|
132
|
+
This is a diagnostic property representation, not an executable object snapshot:
|
|
133
|
+
functions retain their property identity, not executable closures; platform-private
|
|
134
|
+
internal state and private class fields are not observable properties. The host
|
|
135
|
+
stores and returns the graph without reconstructing or executing those objects.
|
|
136
|
+
An object that rejects reflection causes an observable reporting failure rather
|
|
137
|
+
than a silently incomplete success. Inspect the complete graph in developer
|
|
138
|
+
diagnostics; do not flatten it into ordinary chat or user-facing status.
|
|
139
|
+
|
|
140
|
+
## Host ownership and availability
|
|
141
|
+
|
|
142
|
+
`installNativeDiagnostics(globalThis)` installs one observer per document and
|
|
143
|
+
returns its owner `{documentId, ready, report}`. `ready` resolves after the document
|
|
144
|
+
announcement is accepted, and `report` has the same report contract. Importing
|
|
145
|
+
the ESM module alone installs no observer. The packaged native host owns
|
|
146
|
+
installation and lifetime; ordinary applications use `reportNativeError`.
|
|
147
|
+
|
|
148
|
+
`createNativeDiagnosticsSource()` from
|
|
149
|
+
`arcane-os/core/native-diagnostics-source` projects that same canonical helper
|
|
150
|
+
into the top-level document-start script. It adds no second observer or serializer.
|
|
151
|
+
Windows binds the script to the dedicated `arcaneDiagnostics.Send` host object.
|
|
152
|
+
This route is independent of `arcaneBridge` and the Core subprocess. Host close
|
|
153
|
+
stops new ingress and removes the object/script during existing shutdown.
|
|
154
|
+
|
|
155
|
+
The implementation here supplies Windows WebView2 capture and app-control
|
|
156
|
+
retrieval. macOS, Linux and Android require their native host ingress/session and
|
|
157
|
+
app-control adapters; this source change does not claim those adapters exist.
|
|
158
|
+
The helper recognizes the same fixed-name WebKit reply handler when a host
|
|
159
|
+
provides it; that transport seam is not platform availability. A browser preview
|
|
160
|
+
cannot establish actual native acceptance.
|
|
161
|
+
|
|
162
|
+
Native delivery requires the corresponding rebuilt SDK Windows host asset and
|
|
163
|
+
consumer adoption through the published SDK package. Source inspection, authored
|
|
164
|
+
fixtures and configured capture are distinct from executed built-application
|
|
165
|
+
verification. Existing host stderr/log diagnostics, live Core events, readiness
|
|
166
|
+
replay and `Arcane.diagnostics.recentErrors/get` retain their existing semantics;
|
|
167
|
+
this API does not turn them into a historical error service.
|
|
@@ -6,8 +6,9 @@ from `arcane-os/core/speech`.
|
|
|
6
6
|
|
|
7
7
|
The host supplies an STT engine, a TTS engine, or both. Construction performs no
|
|
8
8
|
installation, model download, model loading, or inference. Core registration
|
|
9
|
-
exposes `speech.status`, `speech.
|
|
10
|
-
operation whose engine was omitted rejects with
|
|
9
|
+
exposes `speech.status`, `speech.load`, `speech.unload`, `speech.transcribe`,
|
|
10
|
+
and `speech.synthesize`; an operation whose engine was omitted rejects with
|
|
11
|
+
`SPEECH_ENGINE_UNAVAILABLE`.
|
|
11
12
|
The optional `signal` controls the service lifetime.
|
|
12
13
|
|
|
13
14
|
```js
|
|
@@ -20,11 +21,11 @@ const core = createCoreRuntime({services: [speech]});
|
|
|
20
21
|
core.start();
|
|
21
22
|
```
|
|
22
23
|
|
|
23
|
-
This service is the shared integration layer.
|
|
24
|
-
Kokoro
|
|
25
|
-
selection, supported formats, and
|
|
26
|
-
selected engine and its host
|
|
27
|
-
independent of this native service.
|
|
24
|
+
This service is the shared integration layer. The SDK separately exports the
|
|
25
|
+
[native Kokoro engine](#native-kokoro-engine) from `arcane-os/local-ai/kokoro`.
|
|
26
|
+
Engine preparation, model resources, device selection, supported formats, and
|
|
27
|
+
native process ownership remain with the selected engine and its host
|
|
28
|
+
composition. Browser-only operation remains independent of this native service.
|
|
28
29
|
|
|
29
30
|
The SDK's [retained Whisper engine](local-whisper.md) supplies native STT.
|
|
30
31
|
Its optional [Intel NPU encoder](../guides/native-whisper-openvino.md) uses the
|
|
@@ -59,6 +60,24 @@ model readiness. Existing readiness consumers additionally recognize
|
|
|
59
60
|
the engine IDs `whisper.cpp` and `kokoro-onnx`; another engine ID requires the
|
|
60
61
|
corresponding consumer integration.
|
|
61
62
|
|
|
63
|
+
## Explicit role activation
|
|
64
|
+
|
|
65
|
+
`speech.load({role, ...selection})` and `speech.unload({role, ...selection})`
|
|
66
|
+
route only to the selected `stt` or `tts` engine. The service separates the
|
|
67
|
+
`role` routing field and forwards the remaining selection fields unchanged,
|
|
68
|
+
with the operation's signal. Each call returns that engine's complete result.
|
|
69
|
+
An invalid role reports `SPEECH_ROLE_INVALID`; an engine without the requested
|
|
70
|
+
method reports `SPEECH_OPERATION_UNAVAILABLE`. These calls do not alter the
|
|
71
|
+
other speech role or select a browser provider.
|
|
72
|
+
|
|
73
|
+
The browser facade is `Arcane.speech.load(request, {signal, onRequest})` and
|
|
74
|
+
`Arcane.speech.unload(request, {signal, onRequest})`. Both use an unlimited Core
|
|
75
|
+
request duration (`timeoutMs: 0`). Synthesis retains its existing
|
|
76
|
+
`Arcane.speech.synthesize(request, {signal})` facade and 180-second timeout.
|
|
77
|
+
Call `client.invoke('speech.synthesize', request, {signal, timeoutMs: 0, onRequest})`
|
|
78
|
+
when the owning application explicitly needs an unlimited request duration or
|
|
79
|
+
the actual request ID. The synthesis payload is the same through either path.
|
|
80
|
+
|
|
62
81
|
## Events and subscriptions
|
|
63
82
|
|
|
64
83
|
`subscribe(listener, {replay = true, signal} = {})` immediately delivers the
|
|
@@ -78,6 +97,8 @@ that initial request is pending.
|
|
|
78
97
|
The service emits `progress: {phase: 'accepted'}` synchronously when accepting
|
|
79
98
|
a speech request. Subsequent progress is the selected engine's actual report;
|
|
80
99
|
cancelled requests publish no later progress.
|
|
100
|
+
Role load/unload acknowledgements additionally carry `operation: 'load'` or
|
|
101
|
+
`'unload'` in that progress record.
|
|
81
102
|
|
|
82
103
|
## Engine interface
|
|
83
104
|
|
|
@@ -86,15 +107,18 @@ Each supplied engine implements:
|
|
|
86
107
|
- `current()` returning the actual selected-model state.
|
|
87
108
|
- `subscribe(listener)` with immediate current-state replay and an unsubscribe
|
|
88
109
|
function.
|
|
89
|
-
- `load({signal})`,
|
|
90
|
-
|
|
110
|
+
- `load({signal, ...selection})`, applying that engine's explicit selection
|
|
111
|
+
contract. Independent engines may load concurrently. Kokoro without selected
|
|
112
|
+
assets resolves to its ordinary unloaded state.
|
|
113
|
+
- Optional `unload(selection)`, releasing the selected engine's model through
|
|
114
|
+
`speech.unload`; engines without this method report it unavailable.
|
|
91
115
|
- `transcribe(request, {signal, onProgress, requestId})` for STT, or
|
|
92
116
|
`synthesize(request, {signal, onProgress, requestId})` for TTS. `requestId` is
|
|
93
117
|
the actual Core request's control metadata, separate from the unchanged payload.
|
|
94
118
|
- `close()`, cancelling and joining its native work and releasing its resources.
|
|
95
119
|
|
|
96
|
-
The service forwards each complete request and result
|
|
97
|
-
no input trimming, content limits, model or voice substitution, transcript
|
|
120
|
+
The service forwards each complete transcription/synthesis request and result
|
|
121
|
+
unchanged. It applies no input trimming, content limits, model or voice substitution, transcript
|
|
98
122
|
rewriting, or audio conversion. The engine owns any format decoding,
|
|
99
123
|
phonemization, required model segmentation, output encoding, and supported
|
|
100
124
|
model selection. Necessary processing must preserve complete ordered input
|
|
@@ -120,6 +144,191 @@ share one shutdown promise. Actual engine cleanup failures reject with an
|
|
|
120
144
|
releases the service's event subscriptions. New work after shutdown begins
|
|
121
145
|
rejects with `CORE_CLOSING`.
|
|
122
146
|
|
|
147
|
+
## Native Kokoro engine
|
|
148
|
+
|
|
149
|
+
`createNativeKokoroRuntime` is the named and default export from
|
|
150
|
+
`arcane-os/local-ai/kokoro`. It uses the existing native ONNX owner for model
|
|
151
|
+
execution and one retained C++ helper for eSpeak phonemization and Ogg Opus
|
|
152
|
+
encoding. The Kokoro route requires no Python, Sherpa-ONNX or FFmpeg runtime.
|
|
153
|
+
|
|
154
|
+
```js
|
|
155
|
+
import {createNativeKokoroRuntime} from 'arcane-os/local-ai/kokoro';
|
|
156
|
+
import {createSpeechService} from 'arcane-os/core/speech';
|
|
157
|
+
|
|
158
|
+
const tts = createNativeKokoroRuntime({
|
|
159
|
+
onnx, // The existing Core local-ai owner's getONNXRuntime() result.
|
|
160
|
+
modelAssets, // The existing Core model-assets service.
|
|
161
|
+
runtime, // Prepared helperExecutable, espeakDataDirectory, libraryDirectory.
|
|
162
|
+
executionPreference: 'cpu',
|
|
163
|
+
sessionOptions: {executionProviders: ['cpu']}
|
|
164
|
+
});
|
|
165
|
+
const speech = createSpeechService({tts});
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
The constructor accepts `{onnx, modelAssets, runtime, prepare, modelId, model,
|
|
169
|
+
revision, dtype, paths, assetProjectionId, resourcePaths, sessionOptions,
|
|
170
|
+
executionPreference, executionTarget, signal, onEvent}`. At least `onnx` or
|
|
171
|
+
`prepare` must be supplied. Optional `prepare({signal})` returns `{onnx, runtime}`
|
|
172
|
+
and runs only when an activation has selected assets and either its ONNX owner
|
|
173
|
+
is absent or closed, or its helper descriptor is absent. It returns the host's
|
|
174
|
+
existing ONNX owner; it does not construct another one. Construction performs
|
|
175
|
+
no acquisition or loading.
|
|
176
|
+
The helper data path is the **parent** containing `espeak-ng-data`.
|
|
177
|
+
|
|
178
|
+
The provider ID is `kokoro-onnx`; the public `modelId` alias defaults to `kokoro`.
|
|
179
|
+
The implemented model is `onnx-community/Kokoro-82M-v1.0-ONNX`, revision
|
|
180
|
+
`1939ad2a8e416c0acfeecc08a694d14ef25f2231`, with `dtype: 'fp32'`. Another model,
|
|
181
|
+
revision or dtype reports incompatibility. The module also exports
|
|
182
|
+
`KOKORO_MODEL`, `KOKORO_REVISION`, `KOKORO_SAMPLE_RATE` (`24000`) and
|
|
183
|
+
`KOKORO_VOICES`.
|
|
184
|
+
|
|
185
|
+
### Resources and loading
|
|
186
|
+
|
|
187
|
+
Supply either prepared native `paths` or an existing completed
|
|
188
|
+
[`model-assets` projection](model-assets.md). Both mappings have
|
|
189
|
+
`{model, tokenizer, tokenizerConfig, voices: {[voiceId]: path}}`: native paths
|
|
190
|
+
identify prepared files, while `resourcePaths` identifies projection members.
|
|
191
|
+
Projection loading requires the existing `modelAssets` service. Direct-path
|
|
192
|
+
callers keep their prepared files available until engine cleanup completes.
|
|
193
|
+
The complete selection contains `onnx/model.onnx`, `tokenizer.json`,
|
|
194
|
+
`tokenizer_config.json`, and all 28 `voices/<voiceId>.bin` files from the same
|
|
195
|
+
revision. The engine does not download models or create another model store.
|
|
196
|
+
|
|
197
|
+
`load({modelId, assetProjectionId, resourcePaths, executionTarget, signal})`
|
|
198
|
+
activates that selection. Omitted selection fields retain the previous choice;
|
|
199
|
+
an omitted or undefined target retains the previous target, and `null` requests
|
|
200
|
+
automatic selection. `modelId`, when supplied, must match this engine's alias.
|
|
201
|
+
With neither constructor paths nor a projection selected, loading resolves
|
|
202
|
+
unloaded without preparing the helper. Actual readiness requires the frontend
|
|
203
|
+
resources, live helper and loaded ONNX session.
|
|
204
|
+
|
|
205
|
+
For a complete DBOPFS-to-Core projection already prepared by the application:
|
|
206
|
+
|
|
207
|
+
```js
|
|
208
|
+
await Arcane.speech.load({
|
|
209
|
+
role: 'tts',
|
|
210
|
+
modelId: 'kokoro',
|
|
211
|
+
assetProjectionId: projection.id,
|
|
212
|
+
resourcePaths,
|
|
213
|
+
executionTarget: {deviceId: 'cpu'}
|
|
214
|
+
}, {signal});
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
The engine retains the incoming projection before retiring a previous
|
|
218
|
+
activation, including replacement using the same projection. It releases that
|
|
219
|
+
retain only after frontend loading settles and the actual ONNX worker and
|
|
220
|
+
helper exit. The preparation owner retains its own independent release duty.
|
|
221
|
+
Cancelling a caller's `load` signal stops that caller's wait; explicit unload,
|
|
222
|
+
replacement or engine close owns cancellation of the shared activation.
|
|
223
|
+
After `localai.services.recover({runtimes: ['onnx']})` completes, a subsequent
|
|
224
|
+
explicit `speech.load` uses the composition's `prepare` callback to reacquire
|
|
225
|
+
the current ONNX owner after the previous activation's cleanup has joined.
|
|
226
|
+
|
|
227
|
+
`executionPreference` defaults to `'gpu'` and accepts the existing ONNX CPU/GPU
|
|
228
|
+
selection contract. `sessionOptions` and `executionTarget` reach that same
|
|
229
|
+
owner unchanged. Explicit providers take precedence, and an explicit physical
|
|
230
|
+
target is never retried on another device or provider. The automatic route's
|
|
231
|
+
provider attempts and possible CPU selection remain visible in `execution`.
|
|
232
|
+
STT keeps its own independent device selection. See
|
|
233
|
+
[native ONNX execution targets](local-ai.md#physical-execution-targets);
|
|
234
|
+
accepted configuration and actual physical execution are separate facts.
|
|
235
|
+
|
|
236
|
+
### Synthesis, voices and state
|
|
237
|
+
|
|
238
|
+
`synthesize(request, {signal, onProgress, requestId} = {})` accepts
|
|
239
|
+
`{input, model?, voice?, speed?, responseFormat?}`. `input` is the complete
|
|
240
|
+
string; optional `model` must match the alias. Defaults are voice `af_heart`,
|
|
241
|
+
speed `1`, and format `wav`. Speed must be a positive finite float32 value.
|
|
242
|
+
|
|
243
|
+
| English voice set | Exact supported IDs |
|
|
244
|
+
| --- | --- |
|
|
245
|
+
| US female | `af_heart`, `af_alloy`, `af_aoede`, `af_bella`, `af_jessica`, `af_kore`, `af_nicole`, `af_nova`, `af_river`, `af_sarah`, `af_sky` |
|
|
246
|
+
| US male | `am_adam`, `am_echo`, `am_eric`, `am_fenrir`, `am_liam`, `am_michael`, `am_onyx`, `am_puck`, `am_santa` |
|
|
247
|
+
| GB female | `bf_emma`, `bf_isabella`, `bf_alice`, `bf_lily` |
|
|
248
|
+
| GB male | `bm_george`, `bm_lewis`, `bm_daniel`, `bm_fable` |
|
|
249
|
+
|
|
250
|
+
The selected voice determines US or GB phonemization. Unsupported voices,
|
|
251
|
+
formats and emitted phonemes report their actual errors; there is no silent
|
|
252
|
+
voice substitution or phoneme omission. An embedded NUL or unpaired Unicode
|
|
253
|
+
surrogate reports the public eSpeak interface's inability to consume that
|
|
254
|
+
complete string. The engine preserves source text, consumes all returned
|
|
255
|
+
clauses and divides model tokens into ordered segments according to the model's
|
|
256
|
+
context. Each segment's complete waveform contributes to the final audio.
|
|
257
|
+
|
|
258
|
+
The result is `{audioBase64, contentType, sampleRate: 24000, channels: 1,
|
|
259
|
+
model, voice, speed}`. `wav` returns IEEE float32 WAV with `audio/wav`;
|
|
260
|
+
`opus` and `ogg` both return finalized Ogg Opus with
|
|
261
|
+
`audio/ogg; codecs=opus`. The result contains the complete encoded audio rather
|
|
262
|
+
than a playback side effect. Synthesis requires a ready activation and proceeds
|
|
263
|
+
in FIFO order. Progress reports the actual `phonemizing`, `synthesizing` and
|
|
264
|
+
`encoding` phases; segment progress is not streamed audio.
|
|
265
|
+
|
|
266
|
+
`current()` returns the selected model/provider, `state`, `loaded`, `busy`,
|
|
267
|
+
active `requestId` (or `null` when omitted), `queuedRequests`, `pendingActivation`, `execution`,
|
|
268
|
+
complete `progress` and serialized `error`, plus `defaultVoice` and `voices`.
|
|
269
|
+
`subscribe(listener, {replay = true, signal} = {})` returns an unsubscribe
|
|
270
|
+
function and replays that current state. Through Core, it appears in
|
|
271
|
+
`speech.status().roles.tts` and `speech.state`.
|
|
272
|
+
|
|
273
|
+
Cancelling a queued synthesis removes only that request. Cancelling active
|
|
274
|
+
synthesis retires its native session/helper and joins their actual exit before
|
|
275
|
+
settlement. Surviving queued requests retain their complete input and order;
|
|
276
|
+
the next request reopens the same activation with the accepted provider options
|
|
277
|
+
and the resolved device when known. It does not replay the cancelled synthesis or search for
|
|
278
|
+
another provider. With no queued work, explicitly `load()` before new synthesis.
|
|
279
|
+
Native ONNX cancellation can wait for an in-flight native call to return.
|
|
280
|
+
|
|
281
|
+
`unload()` cancels the activation and its queued work, joins native cleanup and
|
|
282
|
+
releases retained model assets. `close()` also disposes engine subscriptions;
|
|
283
|
+
both preserve the host's shared ONNX owner and other engines. Cleanup errors
|
|
284
|
+
remain complete and observable. The Core equivalent is
|
|
285
|
+
`Arcane.speech.unload({role: 'tts'})`.
|
|
286
|
+
|
|
287
|
+
### Native helper production and availability
|
|
288
|
+
|
|
289
|
+
Windows x64 has a selected SDK helper archive, `arcane-kokoro-windows-x64.tar.gz`.
|
|
290
|
+
[Runtime preparation](local-ai.md#native-kokoro-selection) resolves it from the
|
|
291
|
+
installed SDK version's release. Linux and macOS use the shared JavaScript
|
|
292
|
+
engine and native CMake source with an explicitly prepared, platform-matching
|
|
293
|
+
complete helper tree or archive. Automatic Linux/macOS helper distributions
|
|
294
|
+
are not provided; execution on those hosts remains unverified.
|
|
295
|
+
|
|
296
|
+
SDK selected-output verification on October 7, 2026 used Node `26.7.0` and
|
|
297
|
+
ONNX Runtime Node `1.30.0` with explicit CPU selection for the selected Windows
|
|
298
|
+
helper and FP32 model. It covered US `af_heart` and GB `bf_emma` synthesis to WAV and Ogg Opus,
|
|
299
|
+
including queued/active cancellation and a surviving queued request. This
|
|
300
|
+
establishes that selected CPU output boundary, not GPU execution, other voices'
|
|
301
|
+
pronunciation, listening acceptance or consumer application behavior.
|
|
302
|
+
The subsequent Core-recovery reacquisition correction received source review
|
|
303
|
+
only and was not part of those execution runs.
|
|
304
|
+
|
|
305
|
+
The repository's [`tools/build-kokoro-runtime-windows.ps1`](https://github.com/TheWizardNexus/arcane-os-sdk/blob/main/tools/build-kokoro-runtime-windows.ps1)
|
|
306
|
+
builds from prepared `espeak-ng`, `opus` and `libopusenc` source directories.
|
|
307
|
+
Its selected inputs are the `csukuangfj/espeak-ng` fork at
|
|
308
|
+
`ed530aa113046142eb5115cf2fc9157854d0ffe1` with its matching UCD and language data,
|
|
309
|
+
libopus `1.6.1`, and libopusenc `0.3`. It uses installed CMake and Visual Studio
|
|
310
|
+
2022 Build Tools, defaulting to MSVC `14.44.35207` and Windows SDK
|
|
311
|
+
`10.0.26100.0`; it acquires no dependencies or model files.
|
|
312
|
+
|
|
313
|
+
```powershell
|
|
314
|
+
& ./tools/build-kokoro-runtime-windows.ps1 `
|
|
315
|
+
-SourceRoot ./.arcane/kokoro-source `
|
|
316
|
+
-OutputDirectory ./.arcane/kokoro-build `
|
|
317
|
+
-Component espeak
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
Run `espeak`, `opus` and `opusenc` components independently, concurrently when
|
|
321
|
+
desired, using the same output directory. After all three complete, run
|
|
322
|
+
`helper`, then `stage`. Each component retains complete command/result logs;
|
|
323
|
+
do not run the same component concurrently in that output. The output must
|
|
324
|
+
belong to `ProjectDirectory` and be separate from the prepared source tree.
|
|
325
|
+
The staged helper, matching complete data, corresponding source and included
|
|
326
|
+
terms travel together. The recipe statically links its selected libraries and
|
|
327
|
+
retains their upstream notices alongside the SDK license. Model/voice assets
|
|
328
|
+
and the separately selected ONNX runtime remain outside the helper package.
|
|
329
|
+
This reusable recipe was source-reviewed; its publication does not claim a
|
|
330
|
+
fresh execution of every component or a build on another operating system.
|
|
331
|
+
|
|
123
332
|
## AI built-in native speech readiness
|
|
124
333
|
|
|
125
334
|
The `AI` module's `LOCAL_SPEACH` adapter observes `speech.state` before reading
|
|
@@ -210,30 +210,33 @@ Built-in cloud chat decodes an HTTP error body once as JSON or text and rejects
|
|
|
210
210
|
with that complete value unchanged. It does not reconstruct an Error, replace
|
|
211
211
|
the message, or add `providerMessage`, `status`, or an SDK failure code to the
|
|
212
212
|
provider body. Network errors pass through after the retry policy below;
|
|
213
|
-
decoding errors pass through immediately. Cancellation
|
|
213
|
+
successful-response decoding errors pass through immediately. Cancellation
|
|
214
214
|
retains the existing `ARCANE_AI_REQUEST_ABORTED` contract.
|
|
215
215
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
216
|
+
Rejected Fetch calls and all HTTP `429`/`529` responses share three retries.
|
|
217
|
+
Quota messages need no `overload` keyword. A readable `Retry-After` controls a
|
|
218
|
+
`429` wait; otherwise its upcoming retry number selects `3000`, `6000`, or
|
|
219
|
+
`12000` milliseconds. Fetch rejection and `529` retain `3000` milliseconds.
|
|
220
|
+
After a `429`, active requests to the same endpoint using the same credential
|
|
221
|
+
share its cooldown and dispatch three seconds apart; other endpoints and
|
|
222
|
+
credentials remain independent. The group is released when its last HTTP
|
|
223
|
+
operation settles. See [TWiN recovery](ai/twin-cloud.md#shared-request-behavior)
|
|
224
|
+
for the scope and lifetime.
|
|
225
|
+
|
|
226
|
+
Every attempt uses the same destination, headers, complete serialized body,
|
|
227
|
+
and signal. `onRequest` runs once. Cancellation stops waits and prevents another
|
|
228
|
+
attempt. Recovery precedes successful-body consumption, so partial streams and
|
|
229
|
+
tool callbacks are never replayed. Complete errors, including provider
|
|
230
|
+
`status_code` and nested records, pass through unchanged on exhaustion.
|
|
231
|
+
Diagnostic-body reading failures for `429`/`529` participate in recovery;
|
|
232
|
+
other HTTP errors, successful-body reads, decoding, and callbacks do not.
|
|
231
233
|
|
|
232
234
|
`fetchRequest()` and `streamRequest()` accept
|
|
233
235
|
`onRetry({phase,attempt,delayMs,status,error})`. The observer receives
|
|
234
236
|
`phase:'waiting'` before the abortable delay and `phase:'requesting'` immediately
|
|
235
|
-
before the next Fetch. `attempt` is the one-based upcoming retry number
|
|
236
|
-
|
|
237
|
+
before the next Fetch. `attempt` is the one-based upcoming retry number, or zero
|
|
238
|
+
for an initial request waiting behind a shared quota cooldown. `delayMs` is
|
|
239
|
+
the initially selected wait; shared pacing can extend it. `status` is the HTTP status or `null`
|
|
237
240
|
for rejected Fetch; `error` is the complete original failure value. The SDK
|
|
238
241
|
invokes this observational callback synchronously and observes a returned
|
|
239
242
|
promise without awaiting it. Callback throws and rejections are logged through
|