arcane-os 0.94.0 → 0.96.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +36 -0
- package/README.md +1 -1
- package/browser-runtime/ai/core-model-assets.mjs +74 -0
- package/browser-runtime/core/client.mjs +195 -0
- package/docs/reference/README.md +9 -0
- package/docs/reference/ai/browser-speech.md +61 -10
- package/docs/reference/core-client.md +35 -0
- package/docs/reference/core-native-packaging.md +38 -6
- package/docs/reference/inventory/package-api.json +133 -5
- package/docs/reference/inventory/runtime-modules.json +39 -13
- package/docs/reference/model-assets.md +164 -3
- package/docs/reference/native-magpie.md +98 -0
- package/docs/reference/native-nemotron.md +204 -0
- package/docs/reference/native-speech.md +250 -5
- package/docs/reference/runtime-modules.md +157 -11
- package/docs/reference/sdk-api.md +247 -4
- package/package.json +7 -3
- package/runtime/arcane/modules/AI.js +114 -14
- package/runtime/arcane/modules/AIProviderRuntime.js +32 -15
- package/runtime/arcane/modules/ContinuousVoiceCapture.js +118 -21
- package/runtime/arcane/modules/NativeSpeechCapture.js +198 -0
- package/runtime/arcane/modules/NativeSpeechPlayback.js +304 -0
- package/runtime/arcane/modules/VoiceCaptureWorklet.js +62 -2
- package/src/core/host.mjs +1 -0
- package/src/core/services/local-ai.mjs +46 -12
- package/src/core/services/model-assets.mjs +491 -59
- package/src/core/services/speech.mjs +237 -4
- package/src/local-ai/magpie/build.mjs +36 -0
- package/src/local-ai/magpie/helper.mjs +254 -0
- package/src/local-ai/magpie/index.mjs +488 -0
- package/src/local-ai/magpie/native/CMakeLists.txt +10 -0
- package/src/local-ai/magpie/native/main.cpp +270 -0
- package/src/local-ai/nemo-models.mjs +160 -0
- package/src/local-ai/nemotron/build.mjs +53 -0
- package/src/local-ai/nemotron/helper.mjs +308 -0
- package/src/local-ai/nemotron/index.mjs +693 -0
- package/src/local-ai/nemotron/native/CMakeLists.txt +17 -0
- package/src/local-ai/nemotron/native/main.cpp +282 -0
- package/src/native/macos-provider.mjs +14 -7
- package/src/native/portable-assembly.mjs +180 -0
- package/src/native/portable-provider.mjs +17 -161
- package/src/native/publish-output.mjs +101 -0
- package/src/native/windows-provider.mjs +12 -6
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
{
|
|
2
2
|
"schemaVersion": 1,
|
|
3
|
-
"sdkVersion": "0.
|
|
3
|
+
"sdkVersion": "0.95.0",
|
|
4
4
|
"environment": {
|
|
5
5
|
"runtime": "Node.js for Node entrypoints; browser for browser-only entrypoints",
|
|
6
6
|
"minimumVersion": "22.23.2 for Node entrypoints",
|
|
7
7
|
"moduleSystem": "ESM"
|
|
8
8
|
},
|
|
9
|
-
"memberCount":
|
|
9
|
+
"memberCount": 322,
|
|
10
10
|
"runtimeSubpathPatterns": {
|
|
11
11
|
"arcane-os/modules/*": "./runtime/arcane/modules/*",
|
|
12
12
|
"arcane-os/entities/*": "./runtime/arcane/entities/*"
|
|
@@ -250,6 +250,134 @@
|
|
|
250
250
|
"protocol": "Native speech voice selection",
|
|
251
251
|
"normalization": "Complete ordered voice array exported by the frontend; af_heart is the omitted voice default. An unsupported explicit voice reports its actual incompatibility without substitution."
|
|
252
252
|
},
|
|
253
|
+
{
|
|
254
|
+
"id": "local-ai-nemotron:createNemotronRuntime",
|
|
255
|
+
"name": "createNemotronRuntime",
|
|
256
|
+
"displayName": "createNemotronRuntime()",
|
|
257
|
+
"kind": "function",
|
|
258
|
+
"signature": "createNemotronRuntime({runtime,prepare,modelId='nemotron-3.5-asr-streaming-0.6b',modelPath,diarizationModelPath,gpu,onEvent}={})",
|
|
259
|
+
"entrypoints": [
|
|
260
|
+
"arcane-os/local-ai/nemotron"
|
|
261
|
+
],
|
|
262
|
+
"primaryImport": "arcane-os/local-ai/nemotron",
|
|
263
|
+
"group": "Core native speech",
|
|
264
|
+
"summary": "Retains one native recognizer and optional diarizer across explicit utterance endpoints.",
|
|
265
|
+
"availability": "Node with a matching first-party helper, selected NeMo ASR runtime and models; Windows/Linux/macOS source, Android host adaptation",
|
|
266
|
+
"protocol": "Native ASR C ABI through one retained helper and replaying nemotron.state",
|
|
267
|
+
"normalization": "Returns current/subscribe/load/unload/transcribe/openStream/close. Exact ordered Float32 or f32le input uses the actual declared sample rate; complete snapshots retain alternatives, words, timestamps and speaker scope. forceEndpoint preserves the native stream; finish drains terminally. Callback joins reject NEMOTRON_CALLBACK_WAIT. Cancellation joins cleanup and exposes resetRequired when an in-flight native call requires helper termination; explicit load is required afterward. No microphone, semantic completion or fallback."
|
|
268
|
+
},
|
|
269
|
+
{
|
|
270
|
+
"id": "local-ai-nemotron:default",
|
|
271
|
+
"name": "default",
|
|
272
|
+
"displayName": "createNemotronRuntime default export",
|
|
273
|
+
"kind": "function",
|
|
274
|
+
"signature": "default as createNemotronRuntime",
|
|
275
|
+
"entrypoints": [
|
|
276
|
+
"arcane-os/local-ai/nemotron"
|
|
277
|
+
],
|
|
278
|
+
"primaryImport": "arcane-os/local-ai/nemotron",
|
|
279
|
+
"group": "Core native speech",
|
|
280
|
+
"summary": "Default binding for the same createNemotronRuntime function.",
|
|
281
|
+
"availability": "Node with a matching first-party helper, selected NeMo ASR runtime and models; Windows/Linux/macOS source, Android host adaptation",
|
|
282
|
+
"protocol": "Native ASR C ABI through one retained helper and replaying nemotron.state",
|
|
283
|
+
"normalization": "Binding identity equals the named createNemotronRuntime export"
|
|
284
|
+
},
|
|
285
|
+
{
|
|
286
|
+
"id": "local-ai-nemotron-build:buildNemotronHelper",
|
|
287
|
+
"name": "buildNemotronHelper",
|
|
288
|
+
"displayName": "buildNemotronHelper()",
|
|
289
|
+
"kind": "function",
|
|
290
|
+
"signature": "async buildNemotronHelper({runtime,outputRoot,signal,onEvent}={})",
|
|
291
|
+
"entrypoints": [
|
|
292
|
+
"arcane-os/local-ai/nemotron/build"
|
|
293
|
+
],
|
|
294
|
+
"primaryImport": "arcane-os/local-ai/nemotron/build",
|
|
295
|
+
"group": "Core native speech",
|
|
296
|
+
"summary": "Builds the first-party retained Nemotron ASR helper against a caller-selected NeMo installation.",
|
|
297
|
+
"availability": "Node with CMake 3.26+, C++17 toolchain and platform-matching public NeMo ASR headers/libraries/CMake package",
|
|
298
|
+
"protocol": "Owned configure/build/install process lifecycle",
|
|
299
|
+
"normalization": "runtime.cmakeDirectory and outputRoot select the prepared upstream installation and owned output. Returns platform/architecture/root/helperExecutable/executable and complete configure/compile/install diagnostics. Preserves cancellation and process failures; installs no dependency, model or toolchain and performs no inference."
|
|
300
|
+
},
|
|
301
|
+
{
|
|
302
|
+
"id": "local-ai-nemotron-build:default",
|
|
303
|
+
"name": "default",
|
|
304
|
+
"displayName": "buildNemotronHelper default export",
|
|
305
|
+
"kind": "function",
|
|
306
|
+
"signature": "default as buildNemotronHelper",
|
|
307
|
+
"entrypoints": [
|
|
308
|
+
"arcane-os/local-ai/nemotron/build"
|
|
309
|
+
],
|
|
310
|
+
"primaryImport": "arcane-os/local-ai/nemotron/build",
|
|
311
|
+
"group": "Core native speech",
|
|
312
|
+
"summary": "Default binding for the same buildNemotronHelper function.",
|
|
313
|
+
"availability": "Node with CMake 3.26+, C++17 toolchain and platform-matching public NeMo ASR headers/libraries/CMake package",
|
|
314
|
+
"protocol": "Owned configure/build/install process lifecycle",
|
|
315
|
+
"normalization": "Binding identity equals the named buildNemotronHelper export"
|
|
316
|
+
},
|
|
317
|
+
{
|
|
318
|
+
"id": "local-ai-magpie:createMagpieRuntime",
|
|
319
|
+
"name": "createMagpieRuntime",
|
|
320
|
+
"displayName": "createMagpieRuntime()",
|
|
321
|
+
"kind": "function",
|
|
322
|
+
"signature": "createMagpieRuntime({runtime,prepare,paths,modelAssets,assetProjectionId,resourcePaths,modelId='magpie',voice='Aria',languageCode='en-US',executionTarget,signal,onEvent}={})",
|
|
323
|
+
"entrypoints": [
|
|
324
|
+
"arcane-os/local-ai/magpie"
|
|
325
|
+
],
|
|
326
|
+
"primaryImport": "arcane-os/local-ai/magpie",
|
|
327
|
+
"group": "Core native speech",
|
|
328
|
+
"summary": "Retains native Magpie and NanoCodec for complete audio or ordered PCM synthesis.",
|
|
329
|
+
"availability": "Node with a matching first-party helper, selected NeMo TTS runtime, Magpie/tokenizer and NanoCodec assets; Windows/Linux/macOS source, Android host adaptation",
|
|
330
|
+
"protocol": "Native TTS C ABI through one retained helper and replaying magpie.state",
|
|
331
|
+
"normalization": "Returns current/subscribe/load/unload/synthesize/synthesizeStream/stop/close. Explicit CPU/CUDA selection, model-owned voice map and per-request language remain separate from exact input. Complete wav/pcm export and awaited ordered s16le onAudio delivery share FIFO request ownership. Native speed is 1. Stop targets a requestId and joins native/callback cleanup; self-joins reject MAGPIE_CALLBACK_WAIT. Playback and conversational meaning remain caller-owned; no download or fallback."
|
|
332
|
+
},
|
|
333
|
+
{
|
|
334
|
+
"id": "local-ai-magpie:default",
|
|
335
|
+
"name": "default",
|
|
336
|
+
"displayName": "createMagpieRuntime default export",
|
|
337
|
+
"kind": "function",
|
|
338
|
+
"signature": "default as createMagpieRuntime",
|
|
339
|
+
"entrypoints": [
|
|
340
|
+
"arcane-os/local-ai/magpie"
|
|
341
|
+
],
|
|
342
|
+
"primaryImport": "arcane-os/local-ai/magpie",
|
|
343
|
+
"group": "Core native speech",
|
|
344
|
+
"summary": "Default binding for the same createMagpieRuntime function.",
|
|
345
|
+
"availability": "Node with a matching first-party helper, selected NeMo TTS runtime, Magpie/tokenizer and NanoCodec assets; Windows/Linux/macOS source, Android host adaptation",
|
|
346
|
+
"protocol": "Native TTS C ABI through one retained helper and replaying magpie.state",
|
|
347
|
+
"normalization": "Binding identity equals the named createMagpieRuntime export"
|
|
348
|
+
},
|
|
349
|
+
{
|
|
350
|
+
"id": "local-ai-magpie-build:buildMagpieHelper",
|
|
351
|
+
"name": "buildMagpieHelper",
|
|
352
|
+
"displayName": "buildMagpieHelper()",
|
|
353
|
+
"kind": "function",
|
|
354
|
+
"signature": "async buildMagpieHelper({runtime,outputRoot,signal,onEvent}={})",
|
|
355
|
+
"entrypoints": [
|
|
356
|
+
"arcane-os/local-ai/magpie/build"
|
|
357
|
+
],
|
|
358
|
+
"primaryImport": "arcane-os/local-ai/magpie/build",
|
|
359
|
+
"group": "Core native speech",
|
|
360
|
+
"summary": "Builds the first-party Magpie synthesis helper against a caller-selected NeMo installation.",
|
|
361
|
+
"availability": "Node with CMake 3.26+, C++17 toolchain, platform threads and matching public NeMo TTS headers/libraries/CMake package",
|
|
362
|
+
"protocol": "Owned configure/build/install process lifecycle",
|
|
363
|
+
"normalization": "runtime.cmakeDirectory and outputRoot select the prepared upstream installation and owned output. Returns platform/architecture/root/helperExecutable and complete configure/compile/install diagnostics. Preserves cancellation and process failures; installs no dependency, model or toolchain and performs no inference."
|
|
364
|
+
},
|
|
365
|
+
{
|
|
366
|
+
"id": "local-ai-magpie-build:default",
|
|
367
|
+
"name": "default",
|
|
368
|
+
"displayName": "buildMagpieHelper default export",
|
|
369
|
+
"kind": "function",
|
|
370
|
+
"signature": "default as buildMagpieHelper",
|
|
371
|
+
"entrypoints": [
|
|
372
|
+
"arcane-os/local-ai/magpie/build"
|
|
373
|
+
],
|
|
374
|
+
"primaryImport": "arcane-os/local-ai/magpie/build",
|
|
375
|
+
"group": "Core native speech",
|
|
376
|
+
"summary": "Default binding for the same buildMagpieHelper function.",
|
|
377
|
+
"availability": "Node with CMake 3.26+, C++17 toolchain, platform threads and matching public NeMo TTS headers/libraries/CMake package",
|
|
378
|
+
"protocol": "Owned configure/build/install process lifecycle",
|
|
379
|
+
"normalization": "Binding identity equals the named buildMagpieHelper export"
|
|
380
|
+
},
|
|
253
381
|
{
|
|
254
382
|
"id": "local-ai-whisper:createWhisperRuntime",
|
|
255
383
|
"name": "createWhisperRuntime",
|
|
@@ -429,8 +557,8 @@
|
|
|
429
557
|
"group": "Core native speech",
|
|
430
558
|
"summary": "Composes host-supplied transcription and synthesis engines with independent loading and role readiness.",
|
|
431
559
|
"availability": "Node Core with explicitly supplied engines; engine platform, model and format support remain engine-owned",
|
|
432
|
-
"protocol": "arcane/1 speech.status/load/unload/transcribe/synthesize with speech.state/progress",
|
|
433
|
-
"normalization": "Returns current/subscribe/close/start/methods/drain/dispose; forwards complete requests and results unchanged; replaying service state, synchronous accepted Thinking, combined request/lifetime cancellation and concurrent joined engine shutdown. Explicit load/unload require role stt or tts and forward the remaining activation fields and complete result through that role's configured engine; supported preparation remains with that engine. Startup waiting is independently owned from caller cancellation; no automatic provider selection is introduced."
|
|
560
|
+
"protocol": "arcane/1 speech.status/load/unload/transcribe/synthesize; optional open/push/endpoint/finish/cancel/synthesizeStream/audioConsumed/cancelSynthesis with speech.state/progress/opened/transcription/audio",
|
|
561
|
+
"normalization": "Returns current/subscribe/close/start/methods/drain/dispose; forwards complete requests and results unchanged; replaying service state, synchronous accepted Thinking, combined request/lifetime cancellation and concurrent joined engine shutdown. Explicit load/unload require role stt or tts and forward the remaining activation fields and complete result through that role's configured engine; supported preparation remains with that engine. Per-role supportsStreaming reflects actual engine methods independently of readiness. Request-scoped streaming retains exact PCM and unchanged snapshots; endpoint preserves stream scope, terminal finish drains, and cancellation joins native ownership. Awaited TTS audio acknowledgements apply producer backpressure without blocking control traffic. ASR finals do not decide conversation completion. Startup waiting is independently owned from caller cancellation; no automatic provider selection is introduced."
|
|
434
562
|
},
|
|
435
563
|
{
|
|
436
564
|
"id": "core-speech:default",
|
|
@@ -4444,7 +4572,7 @@
|
|
|
4444
4572
|
"summary": "Projects a Core client into the established application-facing Arcane namespace.",
|
|
4445
4573
|
"availability": "Browser and native WebView JavaScript hosts with a Core client",
|
|
4446
4574
|
"protocol": "arcane/1 compatibility facade",
|
|
4447
|
-
"normalization": "Routes namespace methods through client.invoke while retaining operation-specific parameters, correlation, streaming callbacks, timeouts and cancellation; window.state({signal}?) reads the requesting window and window.setState({state},{signal}?) selects normal, maximized or fullscreen. The matching Windows host returns {platform:'windows',supported:true,state}, including observed minimized state, restores normal bounds/frame controls on leaving fullscreen and prevents activation/focus assignment during visible state transitions without changing TopMost, Core, origin or profile. Saved Settings remain application-owned; cancellation does not undo an applied change and unavailable hosts retain their actual errors without fallback. window.setTheme(presentation,{signal}?) forwards the complete per-window backgroundColor/textColor record, with omitted fields unchanged and null reset, and returns the host's actual applied/unsupported result without changing system appearance. notifications.status/show/state/close forward complete caller requests and standard invoke options; status normalizes only METHOD_NOT_ALLOWED with core-namespace-unavailable while retaining the complete original error. Matching native window hosts retain complete notification records and ordered revisions across renderer navigation, with notifications.state events and subscribe-before-snapshot recovery. Submitted means native Show returned, not displayed or read; cancellation after submission does not remove an accepted notice, and explicit close retains its record. Recipient, text, trigger, deduplication and navigation policy remains application-owned. speech.load(request,options) and speech.unload(request,options) forward complete activation requests, cancellation and request observers with no elapsed-time cutoff; selected engines own preparation and native retirement."
|
|
4575
|
+
"normalization": "speech.openStream(request,options) returns streamId/ready/completion/current/push/forceEndpoint/finish/cancel with ordered exact Float32 input and complete snapshot callbacks; endpoint/finish delivery joins must occur outside their own callbacks. Explicit stream cancel joins native cleanup, distinct from ordinary signal cancellation. speech.synthesizeStream(request,{signal,onAudio,onProgress,onRequest}) awaits required ordered Uint8Array PCM callbacks and exact consumption acknowledgements, joins cancellation, and returns complete terminal metadata without playback. Both streams have no fixed timeout. Routes namespace methods through client.invoke while retaining operation-specific parameters, correlation, streaming callbacks, timeouts and cancellation; window.state({signal}?) reads the requesting window and window.setState({state},{signal}?) selects normal, maximized or fullscreen. The matching Windows host returns {platform:'windows',supported:true,state}, including observed minimized state, restores normal bounds/frame controls on leaving fullscreen and prevents activation/focus assignment during visible state transitions without changing TopMost, Core, origin or profile. Saved Settings remain application-owned; cancellation does not undo an applied change and unavailable hosts retain their actual errors without fallback. window.setTheme(presentation,{signal}?) forwards the complete per-window backgroundColor/textColor record, with omitted fields unchanged and null reset, and returns the host's actual applied/unsupported result without changing system appearance. notifications.status/show/state/close forward complete caller requests and standard invoke options; status normalizes only METHOD_NOT_ALLOWED with core-namespace-unavailable while retaining the complete original error. Matching native window hosts retain complete notification records and ordered revisions across renderer navigation, with notifications.state events and subscribe-before-snapshot recovery. Submitted means native Show returned, not displayed or read; cancellation after submission does not remove an accepted notice, and explicit close retains its record. Recipient, text, trigger, deduplication and navigation policy remains application-owned. speech.load(request,options) and speech.unload(request,options) forward complete activation requests, cancellation and request observers with no elapsed-time cutoff; selected engines own preparation and native retirement."
|
|
4448
4576
|
},
|
|
4449
4577
|
{
|
|
4450
4578
|
"id": "core-client:installCoreClient",
|
|
@@ -5,12 +5,12 @@
|
|
|
5
5
|
"repository": "https://github.com/TheWizardNexus/arcane-os-sdk.git",
|
|
6
6
|
"branch": "main",
|
|
7
7
|
"path": "runtime/arcane",
|
|
8
|
-
"sdkVersion": "0.
|
|
8
|
+
"sdkVersion": "0.95.0",
|
|
9
9
|
"protocol": "arcane/1"
|
|
10
10
|
},
|
|
11
|
-
"artifactCount":
|
|
12
|
-
"javascriptArtifactCount":
|
|
13
|
-
"esmExportCount":
|
|
11
|
+
"artifactCount": 95,
|
|
12
|
+
"javascriptArtifactCount": 93,
|
|
13
|
+
"esmExportCount": 377,
|
|
14
14
|
"artifacts": [
|
|
15
15
|
{
|
|
16
16
|
"file": "runtime/arcane/modules/AI.js",
|
|
@@ -27,11 +27,11 @@
|
|
|
27
27
|
"AI_READY_EVENT",
|
|
28
28
|
"default"
|
|
29
29
|
],
|
|
30
|
-
"summary": "Owns provider-selectable chat and the
|
|
30
|
+
"summary": "Owns provider-selectable chat and the caller-selected browser/native STT/TTS configuration, lifecycle, synthesis, transcription, and playback boundary.",
|
|
31
31
|
"availability": "Browser + native bridge + TWiN Cloud",
|
|
32
32
|
"protocol": "arcane-ai-browser-speech-configuration/1, AIProviderRuntime arcane-ai-provider/2 routes, globalThis.arcaneEvents, TWiN Cloud HTTPS, Arcane.ollama, Arcane.speech, Android WebView bridge",
|
|
33
33
|
"normalization": "Complete mutable caller-owned browser speech authority, SDK-owned provider registration/replacement/disposal, explicit STT/TTS activation, normalized Whisper STT and Kokoro TTS execution with default capacities 1 and 4 respectively, explicit provider-neutral execution snapshots through status(role,{execution:true}), sticky readiness, canonical window.user readiness without compatibility-event object-identity admission or polling, exact ordered structural calls with required arguments.message, atomic nonblank all-ID tool-result sequencing, complete all-choice streaming/data validation, native Ollama structural adaptation, complete-response then all validated tool callbacks then completion ordering, observed async callbacks, Blob/File transcription, default repeated-formatting-mark removal from cloned outbound TTS input, explicit textFormat:'plain' preservation for complete synthesis/preparation while streaming cleanup remains automatic and caller/visible content stays exact, immediate exact-segment synthesis, per-call voice/speed capture, optional final-segment pause on the existing audio clock, optional per-call terminal playback completion, detached preparation with complete semantic DBOPFS audio reuse and same-owner pending request sharing; each preparation owns its pending activation interest, final-interest cancellation aborts that activation while other interests preserve it, complete cache hits start no model, and cancellation after readiness leaves the ready model loaded; independent prepared playback without a model load for cached audio, ordered audio-clock scheduling, playable audio, active-generation TTS operation-failure routing, mute, and cancellation are normalized; provider/model/runtime/voice policy remains caller-owned. Both speech roles default execution.device to auto: webnn-npu when navigator.ml.createContext is exposed, then webgpu when navigator.gpu is exposed, then wasm for CPU execution. Failed backend loads clean up the pool before trying fresh Workers with the same prepared model and dtype; explicit devices never fall back. STT capacity only accepts 1; TTS accepts 1 through 4. Execution snapshots expose requestedDevice, selectedDevice, maxConcurrentRequests, and activeRequestCount for both roles; selectedDevice is null while unloaded and names the successful upstream session backend request when loaded. It does not prove every operation used the physical NPU: WebNN unsupported operations may use WASM, and compatibility depends on the browser, driver, hardware, and graph. High-level LLM selection defaults to eager activation; startLanguageModel:false preserves the selected tuple and registered adapters without preloading through later readiness callbacks. Caller cancellation waits for accepted old-role cleanup and prevents later configuration/preload, or reaches the first owned load after selection commits. Newer accepted selections supersede older continuations; deferred or cancelled activation stays inactive until explicit load/startup, and a caller abort after successful settlement never unloads the ready model. Separate callers joining an existing runtime load retain their current cancellation ownership.",
|
|
34
|
-
"surface": "Native fetchTTS(payload,signal) and fetchSTT(audio,signal) forward caller cancellation separately from their complete Core speech payloads, normalize aborts to AbortError and suppress late results; actual engine interruption remains service-owned. prepareTTSPlayback(payload,signal) silently returns a real Blob or native-speech descriptor; native controls expose finished/released/state/error/pause/resume/stop, and actual release owns the playback lane. fetchTTS and durable prepareTTS report ARCANE_AI_TTS_AUDIO_EXPORT_UNAVAILABLE for native voices. Request-local model overrides remain exact and never mutate the configured model. Browser-speech protocol/event/error/reason constants; default `AI`; read-only `providerRuntime`, `browserSpeechConfiguration`, and `browserSpeechDescriptor`; explicit `providerRuntime.status(role,{execution:true})` snapshots; `streamTTS(text,end,options={})` with optional voice, speed, pauseAfterMs, and waitForPlayback; default speech-input cleanup with SDK-internal preparation metadata and explicit complete-call textFormat:'plain' consumed before engine dispatch; `prepareTTS({parts,storage,identity,textFormat,signal,onState})` returning ordered segments, state, ready, getAudio(index), and cancel(); async `readPreparedTTS({parts,textFormat,storage,identity,signal,onState})` returning a complete ready handle or null through the same semantic matcher, with no activation, unmute, synthesis, playback, storage writes, or partial repair; manifest, audio availability, and returned-handle reads use DBOPFS createTable:false so absent tables remain absent; supplied adapters own that read-existing behavior; missing matches/audio return null, malformed metadata and storage failures reject, and omitted storage sees only completed matching same-AI in-memory audio; hits replay ready state, misses emit nothing, and the lookup signal detaches on settlement while returned-handle cancellation stays independent; `prepareAudioOutput()` returning the persistent WebAudio handle {kind,state,contextState,error,subscribe,resume,dispose} without resume, unmute, model load, sources, sound, or gesture listeners; current-state replay and output-specific canonical `AI_AUDIO_OUTPUT_STATE_EVENT` (`ai-audio-output-state`), with ready only for an actually running context; explicit owner disposal observes context-bound cleanup and close; `playPreparedTTS(prepared,{signal,onState,audioOutput})` returning state, error, finished, pause(), resume(), and stop() with one playback lane per AI and observational state/error callbacks; explicit live same-AI audioOutput borrows its context until owner disposal, while omission or null preserves per-playback terminal close; configure/dispose speech, route lifecycle, declaration-validated chat/stream requests with exact ordered calls; optional streamRequest toolText:{name,field} and onToolText(text,call,displayId) for decoded selected argument text with actual call identity, distinct from ordinary prose and terminal tool execution; synthesis/transcription and playback controls; initializes from current canonical user readiness, installs `window.ai`, and projects `ai-ready` plus active-generation `ai-tts-failure`. `setAI(llmService,sttService,ttsService,model,modelTTS,modelSTT,options)` and `transitionAI(llmService,sttService,ttsService,model,modelTTS,modelSTT,options)` preserve six positional preferences; `configureProviders(selections,options)` and `transitionProviders(selections,options)` accept the same optional {signal:null,startLanguageModel:true} defaults. setAI retains its synchronous boolean and configureProviders its synchronous route result. Explicit `providerRuntime.load('llm',{signal})` or `startProviders({startLanguageModel:true,signal})` activates a deferred selected route."
|
|
34
|
+
"surface": "Explicit null LLM service and null model constructor arguments leave LLM routes unselected; omitted/empty arguments retain defaults, and an existing window.ai is reused. supportsTranscriptionCapture() and createTranscriptionCapture(options) expose selected-provider capture without starting it. Native capture forwards exact PCM, callbacks and external endpoint/stop/cancel joins; finals are recognition segments, never automatic chat. Streaming native TTS preparation is inert, play consumes ordered PCM and released joins native cleanup. Per-utterance languageCode is forwarded separately from text. voice/speed/languageCode persist with an unfinished streamTTS utterance through finishTTS; explicit language change queues the previous remainder under its prior selection. Native fetchTTS(payload,signal) and fetchSTT(audio,signal) forward caller cancellation separately from their complete Core speech payloads, normalize aborts to AbortError and suppress late results; actual engine interruption remains service-owned. prepareTTSPlayback(payload,signal) silently returns a real Blob or native-speech descriptor; native controls expose finished/released/state/error/pause/resume/stop, and actual release owns the playback lane. fetchTTS and durable prepareTTS report ARCANE_AI_TTS_AUDIO_EXPORT_UNAVAILABLE for browser-native voices without audio export; Magpie retains full-audio export. Request-local model overrides remain exact and never mutate the configured model. Browser-speech protocol/event/error/reason constants; default `AI`; read-only `providerRuntime`, `browserSpeechConfiguration`, and `browserSpeechDescriptor`; explicit `providerRuntime.status(role,{execution:true})` snapshots; `streamTTS(text,end,options={})` with optional voice, speed, languageCode, pauseAfterMs, and waitForPlayback; default speech-input cleanup with SDK-internal preparation metadata and explicit complete-call textFormat:'plain' consumed before engine dispatch; `prepareTTS({parts,storage,identity,textFormat,signal,onState})` returning ordered segments, state, ready, getAudio(index), and cancel(); async `readPreparedTTS({parts,textFormat,storage,identity,signal,onState})` returning a complete ready handle or null through the same semantic matcher, with no activation, unmute, synthesis, playback, storage writes, or partial repair; manifest, audio availability, and returned-handle reads use DBOPFS createTable:false so absent tables remain absent; supplied adapters own that read-existing behavior; missing matches/audio return null, malformed metadata and storage failures reject, and omitted storage sees only completed matching same-AI in-memory audio; hits replay ready state, misses emit nothing, and the lookup signal detaches on settlement while returned-handle cancellation stays independent; `prepareAudioOutput()` returning the persistent WebAudio handle {kind,state,contextState,error,subscribe,resume,dispose} without resume, unmute, model load, sources, sound, or gesture listeners; current-state replay and output-specific canonical `AI_AUDIO_OUTPUT_STATE_EVENT` (`ai-audio-output-state`), with ready only for an actually running context; explicit owner disposal observes context-bound cleanup and close; `playPreparedTTS(prepared,{signal,onState,audioOutput})` returning state, error, finished, pause(), resume(), and stop() with one playback lane per AI and observational state/error callbacks; explicit live same-AI audioOutput borrows its context until owner disposal, while omission or null preserves per-playback terminal close; configure/dispose speech, route lifecycle, declaration-validated chat/stream requests with exact ordered calls; optional streamRequest toolText:{name,field} and onToolText(text,call,displayId) for decoded selected argument text with actual call identity, distinct from ordinary prose and terminal tool execution; synthesis/transcription and playback controls; initializes from current canonical user readiness, installs `window.ai`, and projects `ai-ready` plus active-generation `ai-tts-failure`. `setAI(llmService,sttService,ttsService,model,modelTTS,modelSTT,options)` and `transitionAI(llmService,sttService,ttsService,model,modelTTS,modelSTT,options)` preserve six positional preferences; `configureProviders(selections,options)` and `transitionProviders(selections,options)` accept the same optional {signal:null,startLanguageModel:true} defaults. setAI retains its synchronous boolean and configureProviders its synchronous route result. Explicit `providerRuntime.load('llm',{signal})` or `startProviders({startLanguageModel:true,signal})` activates a deferred selected route."
|
|
35
35
|
},
|
|
36
36
|
{
|
|
37
37
|
"file": "runtime/arcane/modules/AIModelSelectionController.js",
|
|
@@ -92,7 +92,7 @@
|
|
|
92
92
|
"availability": "Cross-host in-process runtime; registered providers remain browser, native, or cloud specific",
|
|
93
93
|
"protocol": "arcane-ai-runtime/2, arcane-ai-provider/2, arcane-ai-model-authority/1",
|
|
94
94
|
"normalization": "Normalizes mutable complete per-role routes, lifecycle/status, cancellation, streaming cleanup, capacity-1 FIFO LLM/STT lanes, bounded provider-declared parallel TTS requests with FIFO overflow, default repeated-formatting-mark removal from cloned direct TTS payloads or explicit textFormat:'plain' preservation with the control consumed before engine dispatch, local-only selection, direct LLM history/declaration/terminal structural contracts, complete all-choice content/reasoning FIFO iteration, private automatic provider draining with result-first buffering, terminal-only structural calls, exact per-choice observed/terminal correlation, and a separate complete validated terminal result without creating a fallback.",
|
|
95
|
-
"surface": "Protocol constants; singleton-only `AIProviderRuntime`; `aiProviderRuntime`; `getAIProviderRuntime()`; provider registration/identity/selection, closed three-role and STT/TTS-only validation/configuration/replacement, status/catalog/inspection, startup, per-role load/unload/dispose/cancel, dispose-all, request/stream/speech aliases, SDK-internal speech-input preparation metadata, and setSpeechMuted(muted,{signal}={}) controls. Each pending unmute caller owns its abortable interest; the final interest aborts the owned activation while other preparation/playback or joining direct TTS-load interests preserve it. A pre-aborted unmute starts no activation. Explicit mute retains global cancellation and joined unload even with an aborted observer signal."
|
|
95
|
+
"surface": "supportsTranscriptionCapture(providerId=null) reports the provider capture capability; createTranscriptionCapture(options) returns null or an owned start/stop/cancel/destroy/done handle with optional forceEndpoint. Start requires an idle ready selected STT role, remains in the initiating user gesture and is never queued. Capture holds that role until done joins cleanup; cancellation/retirement uses the same role owner. Native streaming TTS uses the existing request lane with ordered awaited PCM callbacks. Protocol constants; singleton-only `AIProviderRuntime`; `aiProviderRuntime`; `getAIProviderRuntime()`; provider registration/identity/selection, closed three-role and STT/TTS-only validation/configuration/replacement, status/catalog/inspection, startup, per-role load/unload/dispose/cancel, dispose-all, request/stream/speech aliases, SDK-internal speech-input preparation metadata, and setSpeechMuted(muted,{signal}={}) controls. Each pending unmute caller owns its abortable interest; the final interest aborts the owned activation while other preparation/playback or joining direct TTS-load interests preserve it. A pre-aborted unmute starts no activation. Explicit mute retains global cancellation and joined unload even with an aborted observer signal."
|
|
96
96
|
},
|
|
97
97
|
{
|
|
98
98
|
"file": "runtime/arcane/modules/AIResponseURLPolicy.js",
|
|
@@ -442,11 +442,11 @@
|
|
|
442
442
|
"exports": [
|
|
443
443
|
"default"
|
|
444
444
|
],
|
|
445
|
-
"summary": "Owns continuous microphone capture with rolling pre-roll, activity-triggered pause/periodic segments, complete WAV delivery, final flush, and cancellation.",
|
|
445
|
+
"summary": "Owns continuous microphone capture with complete ordered PCM or rolling pre-roll, activity-triggered pause/periodic segments, complete WAV delivery, final flush, and cancellation.",
|
|
446
446
|
"availability": "Browser / native WebView with getUserMedia, AudioContext, and AudioWorklet",
|
|
447
447
|
"protocol": "Native microphone and Web Audio lifecycle; internal VoiceCaptureWorklet MessagePort",
|
|
448
|
-
"normalization": "Complete mono Float32 WAV Blob segments with session-local sequence, pause/periodic/stop reason, and durationMs. Sound amplitude drives segmentation without semantic speech recognition. Inactive audio produces no segment, and periodic segments do not reuse samples. Stop releases microphone tracks immediately and waits for final flush; cancel releases capture and suppresses late callbacks.",
|
|
449
|
-
"surface": "default ContinuousVoiceCapture; constructor({preRollMs=1500,quietMs=2000,chunkMs=30000,activityThreshold=0.02,onSegment,onError,onState}); start({mediaConstraints={audio:true},signal}={}) resolves boolean; stop() resolves after final flush; cancel() and destroy() release the owned session."
|
|
448
|
+
"normalization": "mode:pcm transfers every sample in ordered Float32Array records with actual sampleRate/channels/sequence; amplitude activity reports no semantic speech decision and never gates PCM. flush joins earlier PCM delivery without closing capture. Stop/cancel/destroy join accepted PCM and actual AudioContext close, surfacing failures. In the default segments mode: Complete mono Float32 WAV Blob segments with session-local sequence, pause/periodic/stop reason, and durationMs. Sound amplitude drives segmentation without semantic speech recognition. Inactive audio produces no segment, and periodic segments do not reuse samples. Stop releases microphone tracks immediately and waits for final flush; cancel releases capture and suppresses late callbacks.",
|
|
449
|
+
"surface": "default ContinuousVoiceCapture; constructor({preRollMs=1500,quietMs=2000,chunkMs=30000,activityThreshold=0.02,mode=segments,pcmChunkMs=100,onSegment,onPCM,onActivity,onError,onState}); start({mediaConstraints={audio:true},sampleRate,signal}={}) resolves boolean; flush() resolves after pending PCM delivery; stop() resolves after final flush and close; cancel() and destroy() release the owned session."
|
|
450
450
|
},
|
|
451
451
|
{
|
|
452
452
|
"file": "runtime/arcane/modules/ConversationActionItems.js",
|
|
@@ -991,6 +991,32 @@
|
|
|
991
991
|
"normalization": "Complete mutable definition data with coded syntax errors for malformed input.",
|
|
992
992
|
"surface": "`parseModelDefinition()`, `loadModelDefinitionSystemPrompt()`."
|
|
993
993
|
},
|
|
994
|
+
{
|
|
995
|
+
"file": "runtime/arcane/modules/NativeSpeechCapture.js",
|
|
996
|
+
"name": "NativeSpeechCapture.js",
|
|
997
|
+
"kind": "esm",
|
|
998
|
+
"exports": [
|
|
999
|
+
"default"
|
|
1000
|
+
],
|
|
1001
|
+
"summary": "Connects one microphone lifetime to one retained Core transcription stream without deciding conversational turns.",
|
|
1002
|
+
"availability": "Browser / native WebView with microphone, AudioWorklet and selected streaming Core speech",
|
|
1003
|
+
"protocol": "ContinuousVoiceCapture PCM and Arcane.speech.openStream",
|
|
1004
|
+
"normalization": "Exact ordered mono PCM at the actual declared rate; complete provisional/final snapshots and callback controls. forceEndpoint flushes audio and recognition while preserving capture and stream scope; stop drains terminally; cancel joins capture/native cleanup without claiming a complete transcript.",
|
|
1005
|
+
"surface": "default createNativeSpeechCapture({speech,model,sampleRate=16000,pcmChunkMs=100,activityThreshold=0.02,streamOptions,mediaConstraints,onUpdate,onInterim,onSegment,onActivity,onState,onError,onProgress,onRequest}); returns start({signal,mediaConstraints}), forceEndpoint(), stop(), requestStop(), cancel(reason), destroy(reason), done and current(). Recognition callbacks receive (result,{requestStop}); request a stop then return, and join externally. No implicit model load, conversation reply or fallback."
|
|
1006
|
+
},
|
|
1007
|
+
{
|
|
1008
|
+
"file": "runtime/arcane/modules/NativeSpeechPlayback.js",
|
|
1009
|
+
"name": "NativeSpeechPlayback.js",
|
|
1010
|
+
"kind": "esm",
|
|
1011
|
+
"exports": [
|
|
1012
|
+
"default"
|
|
1013
|
+
],
|
|
1014
|
+
"summary": "Prepares an inert native speech descriptor and plays ordered PCM through the existing shared Web Audio owner.",
|
|
1015
|
+
"availability": "Browser / native WebView with Web Audio and selected streaming speech",
|
|
1016
|
+
"protocol": "speech.synthesizeStream with awaited PCM consumption",
|
|
1017
|
+
"normalization": "Preparation starts no synthesis or playback. play schedules every supplied s16le sample in original order with audio-clock backpressure. Pause retains playback position; stop cancels only that utterance. finished and released distinguish playback termination from joined native cleanup; errors remain observable.",
|
|
1018
|
+
"surface": "default prepareNativeSpeechPlayback({speech,request,getAudioContext,signal=null}); returns {kind:native-speech,input,voice,language,speed,play}. play({signal,onState}) returns finished/released/state/error/pause/resume/stop. The caller owns its AudioContext; this module never closes unrelated output."
|
|
1019
|
+
},
|
|
994
1020
|
{
|
|
995
1021
|
"file": "runtime/arcane/modules/Ollama.js",
|
|
996
1022
|
"name": "Ollama.js",
|
|
@@ -1462,11 +1488,11 @@
|
|
|
1462
1488
|
"name": "VoiceCaptureWorklet.js",
|
|
1463
1489
|
"kind": "worker",
|
|
1464
1490
|
"exports": [],
|
|
1465
|
-
"summary": "
|
|
1491
|
+
"summary": "Transfers continuous PCM or segments microphone samples on the audio rendering thread and transfers complete mono Float32 WAV clips to ContinuousVoiceCapture.",
|
|
1466
1492
|
"availability": "AudioWorkletGlobalScope",
|
|
1467
1493
|
"protocol": "AudioWorklet processor arcane-continuous-voice-capture and its owner-local MessagePort",
|
|
1468
|
-
"normalization": "
|
|
1469
|
-
"surface": "No ESM exports. ContinuousVoiceCapture loads this static worklet, supplies processorOptions, receives segment messages, and requests stop followed by the stopped acknowledgement."
|
|
1494
|
+
"normalization": "mode:pcm emits every original sample in ordered PCM frames at the actual worklet sample rate, plus activity observations; pcmChunkMs defines transport scheduling, not a recording cap. flush returns pending PCM and a correlated flushed acknowledgement without closing the processor. The default segments mode retains rolling pre-roll, fixed 20 ms RMS activity windows independent of callback length, audio-frame quiet duration, periodic boundaries, sequence, complete original WAV audio, and stop acknowledgement. activityThreshold is RMS amplitude, not peak amplitude. Stop classifies a partial final window using its actual samples and retains softer trailing audio in the active clip. Inactive capture emits no clips; adjacent periodic clips have no repeated samples.",
|
|
1495
|
+
"surface": "No ESM exports. ContinuousVoiceCapture loads this static worklet, supplies processorOptions, receives segment or pcm/activity/flushed messages, and requests stop followed by the stopped acknowledgement."
|
|
1470
1496
|
},
|
|
1471
1497
|
{
|
|
1472
1498
|
"file": "runtime/arcane/modules/WaitForComponent.js",
|
|
@@ -110,11 +110,11 @@ try {
|
|
|
110
110
|
}
|
|
111
111
|
```
|
|
112
112
|
|
|
113
|
-
`prepare({id,workingDirectory,members,signal,onProgress,refresh=false})` reuses
|
|
113
|
+
`prepare({id,workingDirectory,members,archives,signal,onProgress,refresh=false})` reuses
|
|
114
114
|
a complete native acquisition or fetches the explicitly selected members.
|
|
115
115
|
Every member has its original relative `path` and selected `url`; the caller
|
|
116
116
|
supplies the complete companion-file set. Independent members download
|
|
117
|
-
concurrently. Each complete response stream is written directly to its native
|
|
117
|
+
concurrently, with up to four independent source transfers active. Each complete response stream is written directly to its native
|
|
118
118
|
file, without a whole-model buffer or base64 conversion. No model or runtime
|
|
119
119
|
is selected or downloaded by service startup. The result remains the ready
|
|
120
120
|
projection snapshot with ordered `members: [{path,nativePath}]`.
|
|
@@ -144,6 +144,158 @@ callback does not hold another caller's transfer or result. Callbacks must not
|
|
|
144
144
|
await cleanup of the preparation currently calling them. Callback failures
|
|
145
145
|
reject that caller's preparation. HTTP failures
|
|
146
146
|
retain `url`, `status` and the complete response text in `response`.
|
|
147
|
+
During an active member, repeated activity notifications are emitted at most
|
|
148
|
+
once per second; opening, member completion and terminal phases remain distinct.
|
|
149
|
+
Progress describes files and phases, never transfer volume.
|
|
150
|
+
|
|
151
|
+
### Selected archive companions
|
|
152
|
+
|
|
153
|
+
An optional `archives` array acquires exact companion members from an upstream
|
|
154
|
+
uncompressed TAR source through this same acquisition owner:
|
|
155
|
+
|
|
156
|
+
```javascript
|
|
157
|
+
const projection = await modelAssets.prepare({
|
|
158
|
+
id: selectedOperationId,
|
|
159
|
+
workingDirectory: selectedModelDirectory,
|
|
160
|
+
members: selectedModelFiles,
|
|
161
|
+
archives: [{
|
|
162
|
+
url: selectedArchiveURL,
|
|
163
|
+
format: 'tar',
|
|
164
|
+
members: [
|
|
165
|
+
{path: 'tokenizer/model_config.yaml', member: 'model_config.yaml'},
|
|
166
|
+
{path: 'tokenizer/speakers.json', member: 'speakers.json'}
|
|
167
|
+
]
|
|
168
|
+
}],
|
|
169
|
+
signal,
|
|
170
|
+
onProgress
|
|
171
|
+
});
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
The result orders ordinary members first, followed by each archive's declared
|
|
175
|
+
members. Extraction preserves each complete selected file. TAR names and PAX
|
|
176
|
+
path records select members; a leading `./` is archive path framing. Missing
|
|
177
|
+
selected members reject preparation and identify the missing names. Gzip and
|
|
178
|
+
other archive formats are outside this reader's contract.
|
|
179
|
+
|
|
180
|
+
When the upstream catalog supplies aligned TAR segments, an archive may include
|
|
181
|
+
`ranges:[{start,end,stopBefore?}]`. The reader requests each inclusive HTTP range
|
|
182
|
+
once and parses its TAR framing; `stopBefore` ends that segment before the
|
|
183
|
+
specified member. Offsets and TAR lengths are solely external transport framing.
|
|
184
|
+
They are never download progress, model readiness criteria or content identity.
|
|
185
|
+
Ranges require the matching HTTP partial-content response; no complete-archive
|
|
186
|
+
fallback is silently substituted. Native reuse identifies archive members by
|
|
187
|
+
their selected URL, archive member name and output path, alongside ordinary
|
|
188
|
+
members, without re-fetching an already completed set.
|
|
189
|
+
|
|
190
|
+
## Discover and open installed files
|
|
191
|
+
|
|
192
|
+
`discover({directories,catalog=[],recursive=true,signal})` inspects only the
|
|
193
|
+
explicit directories, without downloading, copying, loading or selecting a
|
|
194
|
+
model. Paths resolve from `appRoot`. It returns:
|
|
195
|
+
|
|
196
|
+
```javascript
|
|
197
|
+
{
|
|
198
|
+
directories: [{directory, exists, error}], // error is present only on failure
|
|
199
|
+
models: [{
|
|
200
|
+
directory,
|
|
201
|
+
entryPath,
|
|
202
|
+
members: [{path, nativePath, available}],
|
|
203
|
+
missingMembers: [],
|
|
204
|
+
complete: true,
|
|
205
|
+
source: 'directory', // or 'catalog' or 'acquisition'
|
|
206
|
+
catalogId // present for a matching caller-owned catalog entry
|
|
207
|
+
}]
|
|
208
|
+
}
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
Ordinary `.gguf` files are candidates. Split names such as
|
|
212
|
+
`moon-00001-of-00003.gguf` form one candidate with all expected members in numeric
|
|
213
|
+
order, including explicit missing members. The first shard is `entryPath`,
|
|
214
|
+
including when a completed acquisition declared only later shards or listed
|
|
215
|
+
them out of order. Acquisition candidates retain all their ordinary and
|
|
216
|
+
companion members alongside these required shards; separate filename groups
|
|
217
|
+
are never inferred to be one model. Symlinked model files and directories are
|
|
218
|
+
followed, with already visited directory targets reused rather than rescanned.
|
|
219
|
+
Discovery does not establish a model's role, runtime compatibility or successful
|
|
220
|
+
inference. Those selections remain with the application and engine owners.
|
|
221
|
+
|
|
222
|
+
Catalog entries are `{id,members:[{path,url?}],archives?,entryPath?}`. They describe
|
|
223
|
+
an exact companion set and can include nested tokenizer files or multiple model
|
|
224
|
+
files. A discovered catalog candidate reports every expected member, including
|
|
225
|
+
missing companions. Completed native acquisition records are also discoverable;
|
|
226
|
+
when the catalog supplies URLs or archive member names, acquisition matching
|
|
227
|
+
also preserves those recorded source selections. A different pinned source with
|
|
228
|
+
the same filenames is not relabeled as that catalog selection. Path-only catalogs
|
|
229
|
+
and ordinary installed-directory candidates make no source-revision claim.
|
|
230
|
+
Unfinished acquisition attempts are ignored. Discovery never removes either.
|
|
231
|
+
Set `recursive:false` for a directory-only inspection. Missing root directories
|
|
232
|
+
are reported with `exists:false`; other root access failures include their
|
|
233
|
+
complete serialized error. A failure while reading a descendant is surfaced.
|
|
234
|
+
|
|
235
|
+
`openExisting({id,directory,members:[{path}],signal})` opens a complete selected
|
|
236
|
+
set as an ordinary ready projection. It rechecks every member and reports
|
|
237
|
+
`MODEL_ASSET_MEMBERS_MISSING` with `members` when files are absent. It performs
|
|
238
|
+
no copy or network request. Engines use the same `retain(id)` lifecycle.
|
|
239
|
+
Preparation release, engine unload and service disposal never delete borrowed
|
|
240
|
+
originals. External edits or deletion remain controlled by the files' owner.
|
|
241
|
+
|
|
242
|
+
The application supplies persistent model directories separately from disposable
|
|
243
|
+
build output. The native launch context exposes its platform-specific
|
|
244
|
+
`modelsDirectory`; apps may additionally inspect selected application directories.
|
|
245
|
+
The model-assets service creates no platform directory policy of its own.
|
|
246
|
+
|
|
247
|
+
`resolveNativeLaunchContext({appId,context})` defaults `modelsDirectory` to
|
|
248
|
+
`Models` beneath the resolved persistent `stateRoot`. It preserves an explicitly
|
|
249
|
+
supplied `modelsDirectory` unchanged and creates or migrates no files:
|
|
250
|
+
|
|
251
|
+
| Host | Default application state root |
|
|
252
|
+
| --- | --- |
|
|
253
|
+
| Windows | `%LOCALAPPDATA%/Arcane/<appId>`; the user's `AppData/Local` when that environment variable is absent |
|
|
254
|
+
| macOS | `~/Library/Application Support/Arcane/<appId>` |
|
|
255
|
+
| Linux | `$XDG_DATA_HOME/Arcane/<appId>` when absolute, otherwise `~/.local/share/Arcane/<appId>` |
|
|
256
|
+
| Android or another host adaptation | The host supplies its explicit app-private `stateRoot` |
|
|
257
|
+
|
|
258
|
+
Pass this `modelsDirectory` as the acquisition `workingDirectory` and as one
|
|
259
|
+
discovery directory. Completed files then live beneath its `model-assets`
|
|
260
|
+
subdirectory, separate from replaceable application builds. Additional selected
|
|
261
|
+
application directories can participate in the same discovery call.
|
|
262
|
+
|
|
263
|
+
### Prepare a selected local runtime on demand
|
|
264
|
+
|
|
265
|
+
`createLocalAIService(configuration,{autoStart:false,...options})` registers the
|
|
266
|
+
existing local-AI owner without preparing or starting its selected runtimes at
|
|
267
|
+
Core startup. The default remains `autoStart:true`, preserving existing startup
|
|
268
|
+
behavior. After the user explicitly requests preparation, invoke
|
|
269
|
+
`client.invoke('localai.prepare', {}, {signal,timeoutMs:0})`. The operation uses
|
|
270
|
+
the configured existing runtime preparation callback and returns the current
|
|
271
|
+
local-AI state; it does not invent a runtime or model selection.
|
|
272
|
+
|
|
273
|
+
Preparation callback events are published as `localai.preparation` with their
|
|
274
|
+
complete original event payload. Subscribe through the existing Core event
|
|
275
|
+
owner to display actual preparation phases. Concurrent preparation requests
|
|
276
|
+
reuse the owner's in-flight work, and subsequent completed calls reuse its
|
|
277
|
+
prepared state. Runtime and model selection, explicit model acquisition and
|
|
278
|
+
microphone activation remain app-owned; preparation is not an inference or
|
|
279
|
+
microphone operation. Rendering the interface and
|
|
280
|
+
discovering installed files need not wait for runtime preparation.
|
|
281
|
+
|
|
282
|
+
### Browser discovery and explicit download
|
|
283
|
+
|
|
284
|
+
`arcane-os/ai/core-model-assets` additionally exports
|
|
285
|
+
`discoverCoreModelAssets({client,directories,catalog,recursive,signal})`,
|
|
286
|
+
`openExistingCoreModelAssets({client,directory,members,signal})`, and
|
|
287
|
+
`downloadCoreModelAssets({client,workingDirectory,members,archives,refresh,signal,onProgress})`.
|
|
288
|
+
`client` defaults to the installed Core client. Discovery returns the native
|
|
289
|
+
record above; opening and downloading return `{id,directory,members,release}`.
|
|
290
|
+
The existing Blob-based `prepareCoreModelAssets` keeps its original contract.
|
|
291
|
+
|
|
292
|
+
Download progress callbacks receive the complete semantic progress record,
|
|
293
|
+
correlated to this preparation and observed in order. Callback failure cancels
|
|
294
|
+
the operation. Cancellation joins explicit native cleanup and local callback
|
|
295
|
+
delivery before the helper settles; cleanup failures remain observable.
|
|
296
|
+
Callbacks must not await their own enclosing preparation or cancellation join.
|
|
297
|
+
Applications initiate downloads explicitly and display actual phases, selected
|
|
298
|
+
member paths, completed-file counts and errors in the appropriate surface.
|
|
147
299
|
|
|
148
300
|
`release(id)` relinquishes preparation ownership without a request signal.
|
|
149
301
|
Same-process callers sharing the actual store, complete source selection and
|
|
@@ -186,7 +338,8 @@ try {
|
|
|
186
338
|
```
|
|
187
339
|
|
|
188
340
|
`createModelAssetService({appRoot})` returns a Core service with `current()`,
|
|
189
|
-
`
|
|
341
|
+
`discover(options)`, `openExisting(options)`, `prepare(options)`, `retain(id)`,
|
|
342
|
+
`release(id)` and `dispose()`. `retain()` accepts only a completed projection
|
|
190
343
|
and returns `{id,directory,members,release}`. A native engine takes this handle
|
|
191
344
|
before loading and releases it after actual unload, failed-load cleanup or
|
|
192
345
|
worker exit. A rejected inference/load promise alone does not establish native
|
|
@@ -249,6 +402,10 @@ for source selection, cancellation and complete row/output contracts.
|
|
|
249
402
|
| Method | Parameters | Result |
|
|
250
403
|
| --- | --- | --- |
|
|
251
404
|
| `modelAssets.open` | `{id,workingDirectory,members:[{path}]}` | Preparing projection and native member paths |
|
|
405
|
+
| `modelAssets.discover` | `{directories,catalog?,recursive?}` | Directory status and complete/incomplete model candidates |
|
|
406
|
+
| `modelAssets.openExisting` | `{id,directory,members:[{path}]}` | Ready retained projection of existing originals |
|
|
407
|
+
| `modelAssets.prepare` | `{id,workingDirectory,members,archives?,refresh?}` | Ready persistent acquisition; request-scoped `modelAssets.progress` emits `{id,progress}` |
|
|
408
|
+
| `modelAssets.cancelPrepare` | `{id}` | Cancel and join the matching native preparation, then release its preparation ownership |
|
|
252
409
|
| `modelAssets.write` | `{id,memberIndex,contentBase64}` | `{id,memberIndex,written:true}` after the native write |
|
|
253
410
|
| `modelAssets.complete` | `{id}` | Ready projection after all files close |
|
|
254
411
|
| `modelAssets.release` | `{id}` | Released preparation ownership; native uses may retain the files |
|
|
@@ -258,6 +415,10 @@ The browser helper owns this transport sequence. Correlation IDs identify
|
|
|
258
415
|
operations; they are unrelated to content identity. Original model files remain
|
|
259
416
|
separate and complete. A retry creates a new working projection from the stored
|
|
260
417
|
assets; it does not redownload them or replace the authoritative DBOPFS record.
|
|
418
|
+
`cancelPrepare` and `release` have service lifetime so cancellation of the
|
|
419
|
+
original request cannot cancel its cleanup control. Native preparation follows
|
|
420
|
+
the original request signal, including connection cancellation; raw RPC callers
|
|
421
|
+
join `cancelPrepare` when cancelling and release any successful unused projection.
|
|
261
422
|
|
|
262
423
|
The implementation uses browser streams, the existing portable Core JSON
|
|
263
424
|
transport and Node filesystem operations. Native paths and cleanup follow the
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Native Magpie speech
|
|
2
|
+
|
|
3
|
+
`arcane-os/local-ai/magpie` exports `createMagpieRuntime` as a named and default export. It retains the selected Magpie model and NanoCodec decoder in one owned native process. It produces complete encoded audio or ordered PCM callbacks; the caller owns playback. Existing Kokoro, browser speech, and complete-audio APIs remain separate capabilities.
|
|
4
|
+
|
|
5
|
+
## Prepare and select the runtime
|
|
6
|
+
|
|
7
|
+
Supply a prepared native executable and the external NeMo shared-library directory:
|
|
8
|
+
|
|
9
|
+
```js
|
|
10
|
+
import {createMagpieRuntime} from 'arcane-os/local-ai/magpie';
|
|
11
|
+
|
|
12
|
+
const speech = createMagpieRuntime({
|
|
13
|
+
runtime: {helperExecutable, libraryDirectory},
|
|
14
|
+
paths: {
|
|
15
|
+
model: magpieModel,
|
|
16
|
+
codec: nanoCodecModel,
|
|
17
|
+
tokenizerDirectory
|
|
18
|
+
},
|
|
19
|
+
executionTarget: {backend: 'cuda', cudaVisibleDevices: selectedCudaUuid},
|
|
20
|
+
voice: 'Aria',
|
|
21
|
+
languageCode: 'en-US'
|
|
22
|
+
});
|
|
23
|
+
await speech.load();
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
The paths identify an already selected Magpie GGUF, matching tokenizer assets, and matching NanoCodec decoder. Nothing is downloaded or copied by this factory. `executionTarget.backend` must explicitly select `cuda` or `cpu`. For CUDA, the native device owner supplies its selected visible device or UUID through `cudaVisibleDevices`; the child receives that process-local environment before loading CUDA. The adapter does not invent a GPU ordinal, select another device, or fall back to another provider.
|
|
27
|
+
|
|
28
|
+
Instead of `runtime`, supply `prepare({signal})`, returning the prepared descriptor directly: `{helperExecutable, libraryDirectory, ...}`. Preparation runs when loading needs the executable, alongside model metadata preparation. It must return the descriptor, not a `{runtime}` wrapper. Without `libraryDirectory`, the executable's directory is used.
|
|
29
|
+
|
|
30
|
+
Applications using model-assets ownership may replace `paths` with `modelAssets`, `assetProjectionId`, and `resourcePaths: {model, codec, tokenizerConfig}`. The resource paths identify members of the retained projection; the tokenizer directory is derived from the selected configuration member. The projection stays retained until its native helper closes.
|
|
31
|
+
|
|
32
|
+
Other factory options are `modelId` (default `magpie`), a lifetime `signal`, and `onEvent` for the existing process/event diagnostics. Factory defaults are voice `Aria` and language `en-US`; both are exposed by `current()`.
|
|
33
|
+
|
|
34
|
+
## State and lifecycle
|
|
35
|
+
|
|
36
|
+
- `current()` returns provider/model identity, `state`, `loaded`, `busy`, active `requestId`, `queuedRequests`, `defaultVoice`, `languageCode`, model-owned `voices`, configured execution information, `sampleRate`, progress, and a serialized error. Its speech catalog is `{streaming: true, responseFormats: ['wav', 'pcm'], defaultResponseFormat: 'wav'}`. Configured execution is not evidence of actual GPU inference.
|
|
37
|
+
- `subscribe(listener, {replay: true, signal})` observes lifecycle state and replays the current state by default; it returns an unsubscribe function.
|
|
38
|
+
- `load({paths, assetProjectionId, resourcePaths, executionTarget, modelId, signal})` loads or replaces the selected resources. Reusing the same selection reuses its loaded engine. The optional signal cancels that caller's load wait; the factory lifetime signal owns engine cancellation. Load the engine before requesting speech.
|
|
39
|
+
- `unload()` cancels owned work, joins native and callback cleanup, releases retained assets, and leaves the factory reusable.
|
|
40
|
+
- `close()` joins unloading and disposes the factory. Repeated close calls share completion.
|
|
41
|
+
|
|
42
|
+
Requests accepted by one engine are processed in order. Unloading or replacing that engine cancels its owned work; it does not cancel another engine or audio owner.
|
|
43
|
+
|
|
44
|
+
## Exact requests, voices, and languages
|
|
45
|
+
|
|
46
|
+
Both synthesis methods accept the same request:
|
|
47
|
+
|
|
48
|
+
```js
|
|
49
|
+
const request = {
|
|
50
|
+
input: 'The moon is closed for dragon maintenance.',
|
|
51
|
+
voice: 'Aria',
|
|
52
|
+
languageCode: 'en-US'
|
|
53
|
+
};
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
`input` is passed unchanged: no trimming, sentence splitting, prompt injection, tempo stage, or output cap. A NUL or an unpaired surrogate cannot be represented by the native UTF-8 C-string interface and produces an explicit incompatibility error.
|
|
57
|
+
|
|
58
|
+
Optional `model` must match the configured `modelId`. `voice` defaults to the factory selection and is resolved case-insensitively against the matching tokenizer's `model_config.yaml` speaker-map reference and its model-owned JSON IDs. Embedded GGUF voice labels are not used as the selection map.
|
|
59
|
+
|
|
60
|
+
`languageCode` (or `language_code`) is separate per-request control information; it never changes the authored input. Omission uses the configured language. The application/model owns language selection. Installed tokenizer support determines whether a language can run; a language listed in configuration is not proof of runtime support. Native errors are returned rather than switching languages or engines. The public native ABI supplies the natural speaking rate only; an optional `speed` must be `1`.
|
|
61
|
+
|
|
62
|
+
## Complete audio and streaming audio
|
|
63
|
+
|
|
64
|
+
`synthesize(request, {signal, requestId, onProgress})` resolves with complete `audioBase64`, `contentType`, `format`, `sampleFormat: 's16le'`, sample rate, channel count, model, voice, language code, speed, and request ID. Set `request.responseFormat` to `wav` (default) or `pcm`. WAV uses the external RIFF framing; when a complete result cannot fit that format, select PCM or streaming instead of truncating it.
|
|
65
|
+
|
|
66
|
+
`synthesizeStream(request, {signal, requestId, onProgress, onAudio})` requires `onAudio` and awaits each callback in original order:
|
|
67
|
+
|
|
68
|
+
```js
|
|
69
|
+
const controller = new AbortController();
|
|
70
|
+
const completion = speech.synthesizeStream(request, {
|
|
71
|
+
signal: controller.signal,
|
|
72
|
+
requestId: 'dragon-announcement',
|
|
73
|
+
async onAudio({pcm, format, sampleRate, channels, requestId}) {
|
|
74
|
+
await audioOutput.write({pcm, format, sampleRate, channels, requestId});
|
|
75
|
+
}
|
|
76
|
+
});
|
|
77
|
+
const result = await completion;
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
`pcm` is a Node `Buffer` of signed 16-bit little-endian interleaved PCM; the current native engine returns mono audio. The callback includes `format: 's16le'`, `sampleRate`, `channels`, and `requestId`. Awaited callback delivery applies backpressure to the native helper. The streaming terminal result contains metadata and `complete: true`, without accumulating or returning the audio again. Browser/Core transport and playback have their own delivery and cleanup owners.
|
|
81
|
+
|
|
82
|
+
`onProgress` receives synthesis progress. Callback failures cancel the matching utterance and remain observable. A caller-supplied `requestId` identifies its active or queued request; otherwise the runtime creates one.
|
|
83
|
+
|
|
84
|
+
## Cancellation and callback ownership
|
|
85
|
+
|
|
86
|
+
`await speech.stop({requestId})` stops only that active or queued utterance and joins native completion and callback cleanup. It returns `{requestId, stopped: true}`, or `stopped: false` when the ID is no longer owned. Cancellation of an active native call is observed at a PCM callback boundary: this ABI has no asynchronous pre-first-PCM interrupt. Model creation can be cancelled by retiring its owned process.
|
|
87
|
+
|
|
88
|
+
An output callback must return before joining its own stop, unload, close, or selection replacement. Those self-joining calls reject with `MAGPIE_CALLBACK_WAIT`; their return types do not change inside callbacks. To cancel from `onAudio` or `onProgress`, synchronously call the caller-owned `controller.abort()` and return. The external owner then awaits the synthesis completion and cleanup. Likewise, do not await a new queued synthesis from a callback that the current synthesis is awaiting.
|
|
89
|
+
|
|
90
|
+
The synthesis promise rejects on cancellation or failure. Native errors, callback failures, and cleanup failures remain observable; combined failures are preserved in aggregates. Complete audio is never reported for an interrupted request.
|
|
91
|
+
|
|
92
|
+
## Build the first-party helper
|
|
93
|
+
|
|
94
|
+
`arcane-os/local-ai/magpie/build` exports named/default `buildMagpieHelper({runtime, outputRoot, signal, onEvent})`. The selected upstream installation supplies `runtime.cmakeDirectory`, containing `NeMoSpeechConfig.cmake`, its public TTS headers, and its platform-matching libraries. Build prerequisites are CMake 3.26 or newer, a C++17 compiler, platform threads support, and that prepared NeMo TTS installation.
|
|
95
|
+
|
|
96
|
+
The builder configures, builds, and installs the SDK's `arcane-magpie` executable under `outputRoot`. It returns `{platform, architecture, root, helperExecutable, diagnostics: {configure, compile, install}}`. Combine its executable with the selected upstream `libraryDirectory` for the runtime descriptor. The SDK does not vendor NeMo libraries, models, tokenizer assets, or voices.
|
|
97
|
+
|
|
98
|
+
The source targets Windows, Linux, and macOS; Android requires its host adaptation and compatible upstream runtime. On October 17, 2026, the selected Windows x64 helper configured, compiled, and installed successfully against installed NeMo-Speech.cpp 0.2.0 using CMake 4.4.3, Visual Studio 2022, MSVC 19.44.35217, and Windows SDK 10.0.26100. Each operation exited 0, with complete diagnostics retained in the SDK's ignored selected-output directory. The helper was not launched: model inference, microphone capture, and audio playback remain unexecuted for this adapter. Linux and macOS compilation and execution remain unverified. Build success, actual selected-device inference, and audible playback are separate evidence boundaries.
|