arcane-os 0.93.0 → 0.94.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +15 -0
- package/README.md +1 -1
- package/browser-runtime/ai/twin-cloud.mjs +109 -46
- package/browser-runtime/core/client.mjs +2 -0
- package/docs/reference/ai/twin-cloud.md +35 -14
- package/docs/reference/availability-and-normalization.md +1 -1
- package/docs/reference/inventory/package-api.json +95 -11
- package/docs/reference/local-ai.md +81 -5
- package/docs/reference/native-speech.md +220 -11
- package/docs/reference/runtime-modules.md +21 -18
- package/docs/reference/sdk-api.md +152 -11
- package/package.json +2 -1
- package/schemas/arcane-app.schema.json +2 -2
- package/schemas/arcane-package.schema.json +2 -2
- package/src/core/services/speech.mjs +85 -23
- package/src/local-ai/config.mjs +12 -1
- package/src/local-ai/dev.mjs +71 -4
- package/src/local-ai/install.mjs +113 -5
- package/src/local-ai/kokoro/frontend.mjs +219 -0
- package/src/local-ai/kokoro/helper.mjs +168 -0
- package/src/local-ai/kokoro/index.mjs +645 -0
- package/src/local-ai/kokoro/native/CMakeLists.txt +45 -0
- package/src/local-ai/kokoro/native/main.cpp +249 -0
- package/src/native/portable-layout.mjs +67 -9
|
@@ -1,13 +1,16 @@
|
|
|
1
1
|
# Local AI through Core
|
|
2
2
|
|
|
3
3
|
Core owns native llama.cpp and Ollama processes, ONNX worker sessions,
|
|
4
|
-
selected local image generation
|
|
4
|
+
selected local image generation, retained Whisper transcription and native
|
|
5
|
+
Kokoro synthesis. The browser retains its own
|
|
5
6
|
Wllama and ONNX implementations, and can also use an explicitly connected Core
|
|
6
7
|
when that system service is available. Runtime installation, model loading and
|
|
7
8
|
inference are separate operations.
|
|
8
9
|
|
|
9
10
|
Native ONNX runs caller-selected graphs with complete named input/output tensors.
|
|
10
|
-
Applications own preprocessing, tokenization and model-specific pipelines
|
|
11
|
+
Applications own preprocessing, tokenization and model-specific pipelines unless
|
|
12
|
+
they explicitly select an SDK engine such as native Kokoro, which owns its
|
|
13
|
+
versioned speech frontend.
|
|
11
14
|
Browser ONNX continues through its existing browser implementation.
|
|
12
15
|
|
|
13
16
|
## Application requirements
|
|
@@ -113,6 +116,71 @@ the SDK does not guess a companion location. Installation records distinguish
|
|
|
113
116
|
the distribution's `backend` from its `requestedBackend` execution mode.
|
|
114
117
|
Selecting an unavailable distribution reports its actual error.
|
|
115
118
|
|
|
119
|
+
### Native Kokoro selection
|
|
120
|
+
|
|
121
|
+
Select both the existing ONNX runtime and the native speech helper:
|
|
122
|
+
|
|
123
|
+
```json
|
|
124
|
+
{
|
|
125
|
+
"runtimes": [
|
|
126
|
+
{"id": "onnx"},
|
|
127
|
+
{"id": "kokoro-native", "modelId": "kokoro"}
|
|
128
|
+
]
|
|
129
|
+
}
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
`kokoro-native` is the installable helper ID, `kokoro-onnx` is the speech
|
|
133
|
+
provider ID, and `kokoro` is the default model alias. The requirement also
|
|
134
|
+
preserves `model`, `revision`, `dtype`, prepared `paths`, `assetProjectionId`,
|
|
135
|
+
`resourcePaths`, `sessionOptions`, `executionPreference` and `executionTarget`
|
|
136
|
+
for the engine. The implemented model/revision, complete resource mapping,
|
|
137
|
+
voices, synthesis formats and cancellation contract are in
|
|
138
|
+
[native speech](native-speech.md#native-kokoro-engine).
|
|
139
|
+
|
|
140
|
+
With no `helperRoot` or `url`, Windows x64 preparation selects
|
|
141
|
+
`arcane-kokoro-windows-x64.tar.gz` from the installed SDK's numeric release:
|
|
142
|
+
`https://github.com/TheWizardNexus/arcane-os-sdk/releases/download/<sdk-version>/arcane-kokoro-windows-x64.tar.gz`.
|
|
143
|
+
That SDK version selects the default helper release; an authored runtime
|
|
144
|
+
`version` alone does not select another SDK release's helper.
|
|
145
|
+
An explicit `url` selects a complete platform-matching archive, with optional
|
|
146
|
+
`version` metadata. Alternatively, `helperRoot` supplies an already prepared
|
|
147
|
+
complete tree copied into the new installation. Select one of these sources.
|
|
148
|
+
Linux and macOS currently require that explicit archive or prepared tree;
|
|
149
|
+
the native CMake source is available, with no claim of an exercised helper
|
|
150
|
+
build on those platforms.
|
|
151
|
+
|
|
152
|
+
The helper record accepts `helperExecutable`, `espeakDataDirectory` and
|
|
153
|
+
`libraryDirectory` overrides within the selected tree. Defaults are
|
|
154
|
+
`bin/arcane-kokoro.exe` on Windows (`bin/arcane-kokoro` elsewhere), `share`, and
|
|
155
|
+
`bin`. The data field identifies the parent containing `espeak-ng-data`.
|
|
156
|
+
Overrides are relative to the selected tree or absolute locations inside
|
|
157
|
+
`helperRoot`; all three paths relocate with that complete tree. Installation
|
|
158
|
+
prepares the helper and its matching data, independently of ONNX and without
|
|
159
|
+
downloading model weights, tokenizer files or voices.
|
|
160
|
+
|
|
161
|
+
Development and generated native composition register this engine as the TTS
|
|
162
|
+
role of `createSpeechService`, sharing the existing model-assets service and
|
|
163
|
+
the selected local-ai ONNX owner. Whisper, when selected, retains the separate
|
|
164
|
+
STT role and device choice. Construction and no-assets startup leave Kokoro
|
|
165
|
+
unloaded without helper preparation. An explicit activation with prepared
|
|
166
|
+
resources acquires the helper in development and waits only for that existing
|
|
167
|
+
ONNX owner. Native packaging has already bundled the helper. These waits stay
|
|
168
|
+
inside speech activation; page rendering and independent services continue.
|
|
169
|
+
Omitting the explicit `onnx` requirement reports unavailability on activation.
|
|
170
|
+
After an explicit `localai.services.recover({runtimes: ['onnx']})` completes,
|
|
171
|
+
the next explicit `speech.load({role: 'tts', ...selection})` reacquires the
|
|
172
|
+
current ONNX owner through existing preparation, after prior activation cleanup.
|
|
173
|
+
|
|
174
|
+
For saved browser assets, prepare the complete projection through the existing
|
|
175
|
+
[`model-assets` owner](model-assets.md), then call
|
|
176
|
+
`Arcane.speech.load({role: 'tts', modelId: 'kokoro', assetProjectionId,
|
|
177
|
+
resourcePaths, executionTarget}, {signal})`. Core retains the incoming
|
|
178
|
+
projection before replacing its activation and releases it only after native
|
|
179
|
+
exit. Preparing/loading a model remains separate from installing the helper.
|
|
180
|
+
Raw constructor `paths` are caller-prepared native paths; portable assembly
|
|
181
|
+
does not turn external model paths or a prior process's projection ID into
|
|
182
|
+
packaged resources. Prepare the intended model resources at their actual host.
|
|
183
|
+
|
|
116
184
|
## Development
|
|
117
185
|
|
|
118
186
|
```sh
|
|
@@ -125,6 +193,8 @@ configuration. Subsequent `arcane dev` runs reuse the saved requirements.
|
|
|
125
193
|
application selects its `models` in the runtime record before requesting a
|
|
126
194
|
load or generation. The development owner composes separate image and
|
|
127
195
|
model-assets services alongside the existing chat/ONNX service.
|
|
196
|
+
`--local-ai onnx,kokoro-native` records the native TTS requirements above;
|
|
197
|
+
its model activation remains explicit when no assets are configured.
|
|
128
198
|
Development checks selected endpoints and existing executable paths, then
|
|
129
199
|
installs missing runtimes under the application's `.arcane/local-ai/runtimes`.
|
|
130
200
|
Runtime preparation runs alongside HTTP serving. Browser rendering continues
|
|
@@ -174,8 +244,8 @@ processes. An already running external service remains running.
|
|
|
174
244
|
handle with `ready`, `current()`, `handler()` and `close()`. Its `ready` promise
|
|
175
245
|
means Core composition has started, not that installation or a selected model
|
|
176
246
|
is ready. Runtime preparation continues at each owning service. `current()`
|
|
177
|
-
includes separate `core`, `localAI` and optional `image` state;
|
|
178
|
-
remains callable during preparation. Composition failures reject `ready`;
|
|
247
|
+
includes separate `core`, `localAI` and optional `image` and `speech` state;
|
|
248
|
+
image status remains callable during preparation. Composition failures reject `ready`;
|
|
179
249
|
later service preparation failures remain on Core lifecycle and diagnostic
|
|
180
250
|
events. Page rendering waits for neither.
|
|
181
251
|
|
|
@@ -227,7 +297,8 @@ owner; it neither prepares another runtime nor loads a model. An unselected
|
|
|
227
297
|
engine reports `LOCAL_AI_RUNTIME_NOT_SELECTED`; an unavailable or recovering
|
|
228
298
|
owner reports `LOCAL_AI_RUNTIME_UNAVAILABLE`, and an aborted service lifetime
|
|
229
299
|
retains its abort reason. Development and generated native compositions register
|
|
230
|
-
one shared model-assets service when ONNX
|
|
300
|
+
one shared model-assets service when ONNX, llama.cpp, image generation or
|
|
301
|
+
native Kokoro is selected.
|
|
231
302
|
|
|
232
303
|
Reacquire `getONNXRuntime()` on each explicit model load because runtime recovery
|
|
233
304
|
replaces that owner. Retain the acquired handle for that session's cleanup and
|
|
@@ -757,6 +828,11 @@ retains that metadata with the complete native libraries. Image records include
|
|
|
757
828
|
`libraryPath`,
|
|
758
829
|
`bindingModulePath`, requested `backend`, installed `variants` and selected
|
|
759
830
|
`models` metadata. Variant records carry backend, runtime root and library path.
|
|
831
|
+
Kokoro records include `helperExecutable`, `espeakDataDirectory` and
|
|
832
|
+
`libraryDirectory`; bundling preserves the complete helper tree under
|
|
833
|
+
`runtime/local-ai/kokoro-native` and makes all three paths artifact-relative.
|
|
834
|
+
The generated entry resolves them on the destination host and obtains the
|
|
835
|
+
existing ONNX owner when the selected TTS activation needs it.
|
|
760
836
|
Bundle returns `{runtimes,files}`,
|
|
761
837
|
with artifact-relative runtime paths and a complete emitted file inventory.
|
|
762
838
|
Bundling requires a fresh native staging destination, preserving existing
|
|
@@ -6,8 +6,9 @@ from `arcane-os/core/speech`.
|
|
|
6
6
|
|
|
7
7
|
The host supplies an STT engine, a TTS engine, or both. Construction performs no
|
|
8
8
|
installation, model download, model loading, or inference. Core registration
|
|
9
|
-
exposes `speech.status`, `speech.
|
|
10
|
-
operation whose engine was omitted rejects with
|
|
9
|
+
exposes `speech.status`, `speech.load`, `speech.unload`, `speech.transcribe`,
|
|
10
|
+
and `speech.synthesize`; an operation whose engine was omitted rejects with
|
|
11
|
+
`SPEECH_ENGINE_UNAVAILABLE`.
|
|
11
12
|
The optional `signal` controls the service lifetime.
|
|
12
13
|
|
|
13
14
|
```js
|
|
@@ -20,11 +21,11 @@ const core = createCoreRuntime({services: [speech]});
|
|
|
20
21
|
core.start();
|
|
21
22
|
```
|
|
22
23
|
|
|
23
|
-
This service is the shared integration layer.
|
|
24
|
-
Kokoro
|
|
25
|
-
selection, supported formats, and
|
|
26
|
-
selected engine and its host
|
|
27
|
-
independent of this native service.
|
|
24
|
+
This service is the shared integration layer. The SDK separately exports the
|
|
25
|
+
[native Kokoro engine](#native-kokoro-engine) from `arcane-os/local-ai/kokoro`.
|
|
26
|
+
Engine preparation, model resources, device selection, supported formats, and
|
|
27
|
+
native process ownership remain with the selected engine and its host
|
|
28
|
+
composition. Browser-only operation remains independent of this native service.
|
|
28
29
|
|
|
29
30
|
The SDK's [retained Whisper engine](local-whisper.md) supplies native STT.
|
|
30
31
|
Its optional [Intel NPU encoder](../guides/native-whisper-openvino.md) uses the
|
|
@@ -59,6 +60,24 @@ model readiness. Existing readiness consumers additionally recognize
|
|
|
59
60
|
the engine IDs `whisper.cpp` and `kokoro-onnx`; another engine ID requires the
|
|
60
61
|
corresponding consumer integration.
|
|
61
62
|
|
|
63
|
+
## Explicit role activation
|
|
64
|
+
|
|
65
|
+
`speech.load({role, ...selection})` and `speech.unload({role, ...selection})`
|
|
66
|
+
route only to the selected `stt` or `tts` engine. The service separates the
|
|
67
|
+
`role` routing field and forwards the remaining selection fields unchanged,
|
|
68
|
+
with the operation's signal. Each call returns that engine's complete result.
|
|
69
|
+
An invalid role reports `SPEECH_ROLE_INVALID`; an engine without the requested
|
|
70
|
+
method reports `SPEECH_OPERATION_UNAVAILABLE`. These calls do not alter the
|
|
71
|
+
other speech role or select a browser provider.
|
|
72
|
+
|
|
73
|
+
The browser facade is `Arcane.speech.load(request, {signal, onRequest})` and
|
|
74
|
+
`Arcane.speech.unload(request, {signal, onRequest})`. Both use an unlimited Core
|
|
75
|
+
request duration (`timeoutMs: 0`). Synthesis retains its existing
|
|
76
|
+
`Arcane.speech.synthesize(request, {signal})` facade and 180-second timeout.
|
|
77
|
+
Call `client.invoke('speech.synthesize', request, {signal, timeoutMs: 0, onRequest})`
|
|
78
|
+
when the owning application explicitly needs an unlimited request duration or
|
|
79
|
+
the actual request ID. The synthesis payload is the same through either path.
|
|
80
|
+
|
|
62
81
|
## Events and subscriptions
|
|
63
82
|
|
|
64
83
|
`subscribe(listener, {replay = true, signal} = {})` immediately delivers the
|
|
@@ -78,6 +97,8 @@ that initial request is pending.
|
|
|
78
97
|
The service emits `progress: {phase: 'accepted'}` synchronously when accepting
|
|
79
98
|
a speech request. Subsequent progress is the selected engine's actual report;
|
|
80
99
|
cancelled requests publish no later progress.
|
|
100
|
+
Role load/unload acknowledgements additionally carry `operation: 'load'` or
|
|
101
|
+
`'unload'` in that progress record.
|
|
81
102
|
|
|
82
103
|
## Engine interface
|
|
83
104
|
|
|
@@ -86,15 +107,18 @@ Each supplied engine implements:
|
|
|
86
107
|
- `current()` returning the actual selected-model state.
|
|
87
108
|
- `subscribe(listener)` with immediate current-state replay and an unsubscribe
|
|
88
109
|
function.
|
|
89
|
-
- `load({signal})`,
|
|
90
|
-
|
|
110
|
+
- `load({signal, ...selection})`, applying that engine's explicit selection
|
|
111
|
+
contract. Independent engines may load concurrently. Kokoro without selected
|
|
112
|
+
assets resolves to its ordinary unloaded state.
|
|
113
|
+
- Optional `unload(selection)`, releasing the selected engine's model through
|
|
114
|
+
`speech.unload`; engines without this method report it unavailable.
|
|
91
115
|
- `transcribe(request, {signal, onProgress, requestId})` for STT, or
|
|
92
116
|
`synthesize(request, {signal, onProgress, requestId})` for TTS. `requestId` is
|
|
93
117
|
the actual Core request's control metadata, separate from the unchanged payload.
|
|
94
118
|
- `close()`, cancelling and joining its native work and releasing its resources.
|
|
95
119
|
|
|
96
|
-
The service forwards each complete request and result
|
|
97
|
-
no input trimming, content limits, model or voice substitution, transcript
|
|
120
|
+
The service forwards each complete transcription/synthesis request and result
|
|
121
|
+
unchanged. It applies no input trimming, content limits, model or voice substitution, transcript
|
|
98
122
|
rewriting, or audio conversion. The engine owns any format decoding,
|
|
99
123
|
phonemization, required model segmentation, output encoding, and supported
|
|
100
124
|
model selection. Necessary processing must preserve complete ordered input
|
|
@@ -120,6 +144,191 @@ share one shutdown promise. Actual engine cleanup failures reject with an
|
|
|
120
144
|
releases the service's event subscriptions. New work after shutdown begins
|
|
121
145
|
rejects with `CORE_CLOSING`.
|
|
122
146
|
|
|
147
|
+
## Native Kokoro engine
|
|
148
|
+
|
|
149
|
+
`createNativeKokoroRuntime` is the named and default export from
|
|
150
|
+
`arcane-os/local-ai/kokoro`. It uses the existing native ONNX owner for model
|
|
151
|
+
execution and one retained C++ helper for eSpeak phonemization and Ogg Opus
|
|
152
|
+
encoding. The Kokoro route requires no Python, Sherpa-ONNX or FFmpeg runtime.
|
|
153
|
+
|
|
154
|
+
```js
|
|
155
|
+
import {createNativeKokoroRuntime} from 'arcane-os/local-ai/kokoro';
|
|
156
|
+
import {createSpeechService} from 'arcane-os/core/speech';
|
|
157
|
+
|
|
158
|
+
const tts = createNativeKokoroRuntime({
|
|
159
|
+
onnx, // The existing Core local-ai owner's getONNXRuntime() result.
|
|
160
|
+
modelAssets, // The existing Core model-assets service.
|
|
161
|
+
runtime, // Prepared helperExecutable, espeakDataDirectory, libraryDirectory.
|
|
162
|
+
executionPreference: 'cpu',
|
|
163
|
+
sessionOptions: {executionProviders: ['cpu']}
|
|
164
|
+
});
|
|
165
|
+
const speech = createSpeechService({tts});
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
The constructor accepts `{onnx, modelAssets, runtime, prepare, modelId, model,
|
|
169
|
+
revision, dtype, paths, assetProjectionId, resourcePaths, sessionOptions,
|
|
170
|
+
executionPreference, executionTarget, signal, onEvent}`. At least `onnx` or
|
|
171
|
+
`prepare` must be supplied. Optional `prepare({signal})` returns `{onnx, runtime}`
|
|
172
|
+
and runs only when an activation has selected assets and either its ONNX owner
|
|
173
|
+
is absent or closed, or its helper descriptor is absent. It returns the host's
|
|
174
|
+
existing ONNX owner; it does not construct another one. Construction performs
|
|
175
|
+
no acquisition or loading.
|
|
176
|
+
The helper data path is the **parent** containing `espeak-ng-data`.
|
|
177
|
+
|
|
178
|
+
The provider ID is `kokoro-onnx`; the public `modelId` alias defaults to `kokoro`.
|
|
179
|
+
The implemented model is `onnx-community/Kokoro-82M-v1.0-ONNX`, revision
|
|
180
|
+
`1939ad2a8e416c0acfeecc08a694d14ef25f2231`, with `dtype: 'fp32'`. Another model,
|
|
181
|
+
revision or dtype reports incompatibility. The module also exports
|
|
182
|
+
`KOKORO_MODEL`, `KOKORO_REVISION`, `KOKORO_SAMPLE_RATE` (`24000`) and
|
|
183
|
+
`KOKORO_VOICES`.
|
|
184
|
+
|
|
185
|
+
### Resources and loading
|
|
186
|
+
|
|
187
|
+
Supply either prepared native `paths` or an existing completed
|
|
188
|
+
[`model-assets` projection](model-assets.md). Both mappings have
|
|
189
|
+
`{model, tokenizer, tokenizerConfig, voices: {[voiceId]: path}}`: native paths
|
|
190
|
+
identify prepared files, while `resourcePaths` identifies projection members.
|
|
191
|
+
Projection loading requires the existing `modelAssets` service. Direct-path
|
|
192
|
+
callers keep their prepared files available until engine cleanup completes.
|
|
193
|
+
The complete selection contains `onnx/model.onnx`, `tokenizer.json`,
|
|
194
|
+
`tokenizer_config.json`, and all 28 `voices/<voiceId>.bin` files from the same
|
|
195
|
+
revision. The engine does not download models or create another model store.
|
|
196
|
+
|
|
197
|
+
`load({modelId, assetProjectionId, resourcePaths, executionTarget, signal})`
|
|
198
|
+
activates that selection. Omitted selection fields retain the previous choice;
|
|
199
|
+
an omitted or undefined target retains the previous target, and `null` requests
|
|
200
|
+
automatic selection. `modelId`, when supplied, must match this engine's alias.
|
|
201
|
+
With neither constructor paths nor a projection selected, loading resolves
|
|
202
|
+
unloaded without preparing the helper. Actual readiness requires the frontend
|
|
203
|
+
resources, live helper and loaded ONNX session.
|
|
204
|
+
|
|
205
|
+
For a complete DBOPFS-to-Core projection already prepared by the application:
|
|
206
|
+
|
|
207
|
+
```js
|
|
208
|
+
await Arcane.speech.load({
|
|
209
|
+
role: 'tts',
|
|
210
|
+
modelId: 'kokoro',
|
|
211
|
+
assetProjectionId: projection.id,
|
|
212
|
+
resourcePaths,
|
|
213
|
+
executionTarget: {deviceId: 'cpu'}
|
|
214
|
+
}, {signal});
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
The engine retains the incoming projection before retiring a previous
|
|
218
|
+
activation, including replacement using the same projection. It releases that
|
|
219
|
+
retain only after frontend loading settles and the actual ONNX worker and
|
|
220
|
+
helper exit. The preparation owner retains its own independent release duty.
|
|
221
|
+
Cancelling a caller's `load` signal stops that caller's wait; explicit unload,
|
|
222
|
+
replacement or engine close owns cancellation of the shared activation.
|
|
223
|
+
After `localai.services.recover({runtimes: ['onnx']})` completes, a subsequent
|
|
224
|
+
explicit `speech.load` uses the composition's `prepare` callback to reacquire
|
|
225
|
+
the current ONNX owner after the previous activation's cleanup has joined.
|
|
226
|
+
|
|
227
|
+
`executionPreference` defaults to `'gpu'` and accepts the existing ONNX CPU/GPU
|
|
228
|
+
selection contract. `sessionOptions` and `executionTarget` reach that same
|
|
229
|
+
owner unchanged. Explicit providers take precedence, and an explicit physical
|
|
230
|
+
target is never retried on another device or provider. The automatic route's
|
|
231
|
+
provider attempts and possible CPU selection remain visible in `execution`.
|
|
232
|
+
STT keeps its own independent device selection. See
|
|
233
|
+
[native ONNX execution targets](local-ai.md#physical-execution-targets);
|
|
234
|
+
accepted configuration and actual physical execution are separate facts.
|
|
235
|
+
|
|
236
|
+
### Synthesis, voices and state
|
|
237
|
+
|
|
238
|
+
`synthesize(request, {signal, onProgress, requestId} = {})` accepts
|
|
239
|
+
`{input, model?, voice?, speed?, responseFormat?}`. `input` is the complete
|
|
240
|
+
string; optional `model` must match the alias. Defaults are voice `af_heart`,
|
|
241
|
+
speed `1`, and format `wav`. Speed must be a positive finite float32 value.
|
|
242
|
+
|
|
243
|
+
| English voice set | Exact supported IDs |
|
|
244
|
+
| --- | --- |
|
|
245
|
+
| US female | `af_heart`, `af_alloy`, `af_aoede`, `af_bella`, `af_jessica`, `af_kore`, `af_nicole`, `af_nova`, `af_river`, `af_sarah`, `af_sky` |
|
|
246
|
+
| US male | `am_adam`, `am_echo`, `am_eric`, `am_fenrir`, `am_liam`, `am_michael`, `am_onyx`, `am_puck`, `am_santa` |
|
|
247
|
+
| GB female | `bf_emma`, `bf_isabella`, `bf_alice`, `bf_lily` |
|
|
248
|
+
| GB male | `bm_george`, `bm_lewis`, `bm_daniel`, `bm_fable` |
|
|
249
|
+
|
|
250
|
+
The selected voice determines US or GB phonemization. Unsupported voices,
|
|
251
|
+
formats and emitted phonemes report their actual errors; there is no silent
|
|
252
|
+
voice substitution or phoneme omission. An embedded NUL or unpaired Unicode
|
|
253
|
+
surrogate reports the public eSpeak interface's inability to consume that
|
|
254
|
+
complete string. The engine preserves source text, consumes all returned
|
|
255
|
+
clauses and divides model tokens into ordered segments according to the model's
|
|
256
|
+
context. Each segment's complete waveform contributes to the final audio.
|
|
257
|
+
|
|
258
|
+
The result is `{audioBase64, contentType, sampleRate: 24000, channels: 1,
|
|
259
|
+
model, voice, speed}`. `wav` returns IEEE float32 WAV with `audio/wav`;
|
|
260
|
+
`opus` and `ogg` both return finalized Ogg Opus with
|
|
261
|
+
`audio/ogg; codecs=opus`. The result contains the complete encoded audio rather
|
|
262
|
+
than a playback side effect. Synthesis requires a ready activation and proceeds
|
|
263
|
+
in FIFO order. Progress reports the actual `phonemizing`, `synthesizing` and
|
|
264
|
+
`encoding` phases; segment progress is not streamed audio.
|
|
265
|
+
|
|
266
|
+
`current()` returns the selected model/provider, `state`, `loaded`, `busy`,
|
|
267
|
+
active `requestId` (or `null` when omitted), `queuedRequests`, `pendingActivation`, `execution`,
|
|
268
|
+
complete `progress` and serialized `error`, plus `defaultVoice` and `voices`.
|
|
269
|
+
`subscribe(listener, {replay = true, signal} = {})` returns an unsubscribe
|
|
270
|
+
function and replays that current state. Through Core, it appears in
|
|
271
|
+
`speech.status().roles.tts` and `speech.state`.
|
|
272
|
+
|
|
273
|
+
Cancelling a queued synthesis removes only that request. Cancelling active
|
|
274
|
+
synthesis retires its native session/helper and joins their actual exit before
|
|
275
|
+
settlement. Surviving queued requests retain their complete input and order;
|
|
276
|
+
the next request reopens the same activation with the accepted provider options
|
|
277
|
+
and the resolved device when known. It does not replay the cancelled synthesis or search for
|
|
278
|
+
another provider. With no queued work, explicitly `load()` before new synthesis.
|
|
279
|
+
Native ONNX cancellation can wait for an in-flight native call to return.
|
|
280
|
+
|
|
281
|
+
`unload()` cancels the activation and its queued work, joins native cleanup and
|
|
282
|
+
releases retained model assets. `close()` also disposes engine subscriptions;
|
|
283
|
+
both preserve the host's shared ONNX owner and other engines. Cleanup errors
|
|
284
|
+
remain complete and observable. The Core equivalent is
|
|
285
|
+
`Arcane.speech.unload({role: 'tts'})`.
|
|
286
|
+
|
|
287
|
+
### Native helper production and availability
|
|
288
|
+
|
|
289
|
+
Windows x64 has a selected SDK helper archive, `arcane-kokoro-windows-x64.tar.gz`.
|
|
290
|
+
[Runtime preparation](local-ai.md#native-kokoro-selection) resolves it from the
|
|
291
|
+
installed SDK version's release. Linux and macOS use the shared JavaScript
|
|
292
|
+
engine and native CMake source with an explicitly prepared, platform-matching
|
|
293
|
+
complete helper tree or archive. Automatic Linux/macOS helper distributions
|
|
294
|
+
are not provided; execution on those hosts remains unverified.
|
|
295
|
+
|
|
296
|
+
SDK selected-output verification on October 7, 2026 used Node `26.7.0` and
|
|
297
|
+
ONNX Runtime Node `1.30.0` with explicit CPU selection for the selected Windows
|
|
298
|
+
helper and FP32 model. It covered US `af_heart` and GB `bf_emma` synthesis to WAV and Ogg Opus,
|
|
299
|
+
including queued/active cancellation and a surviving queued request. This
|
|
300
|
+
establishes that selected CPU output boundary, not GPU execution, other voices'
|
|
301
|
+
pronunciation, listening acceptance or consumer application behavior.
|
|
302
|
+
The subsequent Core-recovery reacquisition correction received source review
|
|
303
|
+
only and was not part of those execution runs.
|
|
304
|
+
|
|
305
|
+
The repository's [`tools/build-kokoro-runtime-windows.ps1`](https://github.com/TheWizardNexus/arcane-os-sdk/blob/main/tools/build-kokoro-runtime-windows.ps1)
|
|
306
|
+
builds from prepared `espeak-ng`, `opus` and `libopusenc` source directories.
|
|
307
|
+
Its selected inputs are the `csukuangfj/espeak-ng` fork at
|
|
308
|
+
`ed530aa113046142eb5115cf2fc9157854d0ffe1` with its matching UCD and language data,
|
|
309
|
+
libopus `1.6.1`, and libopusenc `0.3`. It uses installed CMake and Visual Studio
|
|
310
|
+
2022 Build Tools, defaulting to MSVC `14.44.35207` and Windows SDK
|
|
311
|
+
`10.0.26100.0`; it acquires no dependencies or model files.
|
|
312
|
+
|
|
313
|
+
```powershell
|
|
314
|
+
& ./tools/build-kokoro-runtime-windows.ps1 `
|
|
315
|
+
-SourceRoot ./.arcane/kokoro-source `
|
|
316
|
+
-OutputDirectory ./.arcane/kokoro-build `
|
|
317
|
+
-Component espeak
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
Run `espeak`, `opus` and `opusenc` components independently, concurrently when
|
|
321
|
+
desired, using the same output directory. After all three complete, run
|
|
322
|
+
`helper`, then `stage`. Each component retains complete command/result logs;
|
|
323
|
+
do not run the same component concurrently in that output. The output must
|
|
324
|
+
belong to `ProjectDirectory` and be separate from the prepared source tree.
|
|
325
|
+
The staged helper, matching complete data, corresponding source and included
|
|
326
|
+
terms travel together. The recipe statically links its selected libraries and
|
|
327
|
+
retains their upstream notices alongside the SDK license. Model/voice assets
|
|
328
|
+
and the separately selected ONNX runtime remain outside the helper package.
|
|
329
|
+
This reusable recipe was source-reviewed; its publication does not claim a
|
|
330
|
+
fresh execution of every component or a build on another operating system.
|
|
331
|
+
|
|
123
332
|
## AI built-in native speech readiness
|
|
124
333
|
|
|
125
334
|
The `AI` module's `LOCAL_SPEACH` adapter observes `speech.state` before reading
|
|
@@ -210,30 +210,33 @@ Built-in cloud chat decodes an HTTP error body once as JSON or text and rejects
|
|
|
210
210
|
with that complete value unchanged. It does not reconstruct an Error, replace
|
|
211
211
|
the message, or add `providerMessage`, `status`, or an SDK failure code to the
|
|
212
212
|
provider body. Network errors pass through after the retry policy below;
|
|
213
|
-
decoding errors pass through immediately. Cancellation
|
|
213
|
+
successful-response decoding errors pass through immediately. Cancellation
|
|
214
214
|
retains the existing `ARCANE_AI_REQUEST_ABORTED` contract.
|
|
215
215
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
216
|
+
Rejected Fetch calls and all HTTP `429`/`529` responses share three retries.
|
|
217
|
+
Quota messages need no `overload` keyword. A readable `Retry-After` controls a
|
|
218
|
+
`429` wait; otherwise its upcoming retry number selects `3000`, `6000`, or
|
|
219
|
+
`12000` milliseconds. Fetch rejection and `529` retain `3000` milliseconds.
|
|
220
|
+
After a `429`, active requests to the same endpoint using the same credential
|
|
221
|
+
share its cooldown and dispatch three seconds apart; other endpoints and
|
|
222
|
+
credentials remain independent. The group is released when its last HTTP
|
|
223
|
+
operation settles. See [TWiN recovery](ai/twin-cloud.md#shared-request-behavior)
|
|
224
|
+
for the scope and lifetime.
|
|
225
|
+
|
|
226
|
+
Every attempt uses the same destination, headers, complete serialized body,
|
|
227
|
+
and signal. `onRequest` runs once. Cancellation stops waits and prevents another
|
|
228
|
+
attempt. Recovery precedes successful-body consumption, so partial streams and
|
|
229
|
+
tool callbacks are never replayed. Complete errors, including provider
|
|
230
|
+
`status_code` and nested records, pass through unchanged on exhaustion.
|
|
231
|
+
Diagnostic-body reading failures for `429`/`529` participate in recovery;
|
|
232
|
+
other HTTP errors, successful-body reads, decoding, and callbacks do not.
|
|
231
233
|
|
|
232
234
|
`fetchRequest()` and `streamRequest()` accept
|
|
233
235
|
`onRetry({phase,attempt,delayMs,status,error})`. The observer receives
|
|
234
236
|
`phase:'waiting'` before the abortable delay and `phase:'requesting'` immediately
|
|
235
|
-
before the next Fetch. `attempt` is the one-based upcoming retry number
|
|
236
|
-
|
|
237
|
+
before the next Fetch. `attempt` is the one-based upcoming retry number, or zero
|
|
238
|
+
for an initial request waiting behind a shared quota cooldown. `delayMs` is
|
|
239
|
+
the initially selected wait; shared pacing can extend it. `status` is the HTTP status or `null`
|
|
237
240
|
for rejected Fetch; `error` is the complete original failure value. The SDK
|
|
238
241
|
invokes this observational callback synchronously and observes a returned
|
|
239
242
|
promise without awaiting it. Callback throws and rejections are logged through
|