arcane-os 0.95.0 → 0.96.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +18 -0
- package/README.md +1 -1
- package/browser-runtime/ai/core-model-assets.mjs +74 -0
- package/docs/reference/README.md +9 -0
- package/docs/reference/ai/browser-speech.md +61 -10
- package/docs/reference/core-client.md +35 -0
- package/docs/reference/core-native-packaging.md +38 -6
- package/docs/reference/inventory/package-api.json +133 -5
- package/docs/reference/inventory/runtime-modules.json +39 -13
- package/docs/reference/model-assets.md +164 -3
- package/docs/reference/native-speech.md +105 -4
- package/docs/reference/runtime-modules.md +157 -11
- package/docs/reference/sdk-api.md +247 -4
- package/package.json +3 -3
- package/src/core/host.mjs +1 -0
- package/src/core/services/local-ai.mjs +46 -12
- package/src/core/services/model-assets.mjs +491 -59
- package/src/core/services/speech.mjs +2 -2
- package/src/local-ai/nemo-models.mjs +160 -0
- package/src/local-ai/nemotron/index.mjs +63 -7
- package/src/native/macos-provider.mjs +14 -7
- package/src/native/portable-assembly.mjs +180 -0
- package/src/native/portable-provider.mjs +17 -161
- package/src/native/publish-output.mjs +101 -0
- package/src/native/windows-provider.mjs +12 -6
|
@@ -5,12 +5,12 @@
|
|
|
5
5
|
"repository": "https://github.com/TheWizardNexus/arcane-os-sdk.git",
|
|
6
6
|
"branch": "main",
|
|
7
7
|
"path": "runtime/arcane",
|
|
8
|
-
"sdkVersion": "0.
|
|
8
|
+
"sdkVersion": "0.95.0",
|
|
9
9
|
"protocol": "arcane/1"
|
|
10
10
|
},
|
|
11
|
-
"artifactCount":
|
|
12
|
-
"javascriptArtifactCount":
|
|
13
|
-
"esmExportCount":
|
|
11
|
+
"artifactCount": 95,
|
|
12
|
+
"javascriptArtifactCount": 93,
|
|
13
|
+
"esmExportCount": 377,
|
|
14
14
|
"artifacts": [
|
|
15
15
|
{
|
|
16
16
|
"file": "runtime/arcane/modules/AI.js",
|
|
@@ -27,11 +27,11 @@
|
|
|
27
27
|
"AI_READY_EVENT",
|
|
28
28
|
"default"
|
|
29
29
|
],
|
|
30
|
-
"summary": "Owns provider-selectable chat and the
|
|
30
|
+
"summary": "Owns provider-selectable chat and the caller-selected browser/native STT/TTS configuration, lifecycle, synthesis, transcription, and playback boundary.",
|
|
31
31
|
"availability": "Browser + native bridge + TWiN Cloud",
|
|
32
32
|
"protocol": "arcane-ai-browser-speech-configuration/1, AIProviderRuntime arcane-ai-provider/2 routes, globalThis.arcaneEvents, TWiN Cloud HTTPS, Arcane.ollama, Arcane.speech, Android WebView bridge",
|
|
33
33
|
"normalization": "Complete mutable caller-owned browser speech authority, SDK-owned provider registration/replacement/disposal, explicit STT/TTS activation, normalized Whisper STT and Kokoro TTS execution with default capacities 1 and 4 respectively, explicit provider-neutral execution snapshots through status(role,{execution:true}), sticky readiness, canonical window.user readiness without compatibility-event object-identity admission or polling, exact ordered structural calls with required arguments.message, atomic nonblank all-ID tool-result sequencing, complete all-choice streaming/data validation, native Ollama structural adaptation, complete-response then all validated tool callbacks then completion ordering, observed async callbacks, Blob/File transcription, default repeated-formatting-mark removal from cloned outbound TTS input, explicit textFormat:'plain' preservation for complete synthesis/preparation while streaming cleanup remains automatic and caller/visible content stays exact, immediate exact-segment synthesis, per-call voice/speed capture, optional final-segment pause on the existing audio clock, optional per-call terminal playback completion, detached preparation with complete semantic DBOPFS audio reuse and same-owner pending request sharing; each preparation owns its pending activation interest, final-interest cancellation aborts that activation while other interests preserve it, complete cache hits start no model, and cancellation after readiness leaves the ready model loaded; independent prepared playback without a model load for cached audio, ordered audio-clock scheduling, playable audio, active-generation TTS operation-failure routing, mute, and cancellation are normalized; provider/model/runtime/voice policy remains caller-owned. Both speech roles default execution.device to auto: webnn-npu when navigator.ml.createContext is exposed, then webgpu when navigator.gpu is exposed, then wasm for CPU execution. Failed backend loads clean up the pool before trying fresh Workers with the same prepared model and dtype; explicit devices never fall back. STT capacity only accepts 1; TTS accepts 1 through 4. Execution snapshots expose requestedDevice, selectedDevice, maxConcurrentRequests, and activeRequestCount for both roles; selectedDevice is null while unloaded and names the successful upstream session backend request when loaded. It does not prove every operation used the physical NPU: WebNN unsupported operations may use WASM, and compatibility depends on the browser, driver, hardware, and graph. High-level LLM selection defaults to eager activation; startLanguageModel:false preserves the selected tuple and registered adapters without preloading through later readiness callbacks. Caller cancellation waits for accepted old-role cleanup and prevents later configuration/preload, or reaches the first owned load after selection commits. Newer accepted selections supersede older continuations; deferred or cancelled activation stays inactive until explicit load/startup, and a caller abort after successful settlement never unloads the ready model. Separate callers joining an existing runtime load retain their current cancellation ownership.",
|
|
34
|
-
"surface": "Native fetchTTS(payload,signal) and fetchSTT(audio,signal) forward caller cancellation separately from their complete Core speech payloads, normalize aborts to AbortError and suppress late results; actual engine interruption remains service-owned. prepareTTSPlayback(payload,signal) silently returns a real Blob or native-speech descriptor; native controls expose finished/released/state/error/pause/resume/stop, and actual release owns the playback lane. fetchTTS and durable prepareTTS report ARCANE_AI_TTS_AUDIO_EXPORT_UNAVAILABLE for native voices. Request-local model overrides remain exact and never mutate the configured model. Browser-speech protocol/event/error/reason constants; default `AI`; read-only `providerRuntime`, `browserSpeechConfiguration`, and `browserSpeechDescriptor`; explicit `providerRuntime.status(role,{execution:true})` snapshots; `streamTTS(text,end,options={})` with optional voice, speed, pauseAfterMs, and waitForPlayback; default speech-input cleanup with SDK-internal preparation metadata and explicit complete-call textFormat:'plain' consumed before engine dispatch; `prepareTTS({parts,storage,identity,textFormat,signal,onState})` returning ordered segments, state, ready, getAudio(index), and cancel(); async `readPreparedTTS({parts,textFormat,storage,identity,signal,onState})` returning a complete ready handle or null through the same semantic matcher, with no activation, unmute, synthesis, playback, storage writes, or partial repair; manifest, audio availability, and returned-handle reads use DBOPFS createTable:false so absent tables remain absent; supplied adapters own that read-existing behavior; missing matches/audio return null, malformed metadata and storage failures reject, and omitted storage sees only completed matching same-AI in-memory audio; hits replay ready state, misses emit nothing, and the lookup signal detaches on settlement while returned-handle cancellation stays independent; `prepareAudioOutput()` returning the persistent WebAudio handle {kind,state,contextState,error,subscribe,resume,dispose} without resume, unmute, model load, sources, sound, or gesture listeners; current-state replay and output-specific canonical `AI_AUDIO_OUTPUT_STATE_EVENT` (`ai-audio-output-state`), with ready only for an actually running context; explicit owner disposal observes context-bound cleanup and close; `playPreparedTTS(prepared,{signal,onState,audioOutput})` returning state, error, finished, pause(), resume(), and stop() with one playback lane per AI and observational state/error callbacks; explicit live same-AI audioOutput borrows its context until owner disposal, while omission or null preserves per-playback terminal close; configure/dispose speech, route lifecycle, declaration-validated chat/stream requests with exact ordered calls; optional streamRequest toolText:{name,field} and onToolText(text,call,displayId) for decoded selected argument text with actual call identity, distinct from ordinary prose and terminal tool execution; synthesis/transcription and playback controls; initializes from current canonical user readiness, installs `window.ai`, and projects `ai-ready` plus active-generation `ai-tts-failure`. `setAI(llmService,sttService,ttsService,model,modelTTS,modelSTT,options)` and `transitionAI(llmService,sttService,ttsService,model,modelTTS,modelSTT,options)` preserve six positional preferences; `configureProviders(selections,options)` and `transitionProviders(selections,options)` accept the same optional {signal:null,startLanguageModel:true} defaults. setAI retains its synchronous boolean and configureProviders its synchronous route result. Explicit `providerRuntime.load('llm',{signal})` or `startProviders({startLanguageModel:true,signal})` activates a deferred selected route."
|
|
34
|
+
"surface": "Explicit null LLM service and null model constructor arguments leave LLM routes unselected; omitted/empty arguments retain defaults, and an existing window.ai is reused. supportsTranscriptionCapture() and createTranscriptionCapture(options) expose selected-provider capture without starting it. Native capture forwards exact PCM, callbacks and external endpoint/stop/cancel joins; finals are recognition segments, never automatic chat. Streaming native TTS preparation is inert, play consumes ordered PCM and released joins native cleanup. Per-utterance languageCode is forwarded separately from text. voice/speed/languageCode persist with an unfinished streamTTS utterance through finishTTS; explicit language change queues the previous remainder under its prior selection. Native fetchTTS(payload,signal) and fetchSTT(audio,signal) forward caller cancellation separately from their complete Core speech payloads, normalize aborts to AbortError and suppress late results; actual engine interruption remains service-owned. prepareTTSPlayback(payload,signal) silently returns a real Blob or native-speech descriptor; native controls expose finished/released/state/error/pause/resume/stop, and actual release owns the playback lane. fetchTTS and durable prepareTTS report ARCANE_AI_TTS_AUDIO_EXPORT_UNAVAILABLE for browser-native voices without audio export; Magpie retains full-audio export. Request-local model overrides remain exact and never mutate the configured model. Browser-speech protocol/event/error/reason constants; default `AI`; read-only `providerRuntime`, `browserSpeechConfiguration`, and `browserSpeechDescriptor`; explicit `providerRuntime.status(role,{execution:true})` snapshots; `streamTTS(text,end,options={})` with optional voice, speed, languageCode, pauseAfterMs, and waitForPlayback; default speech-input cleanup with SDK-internal preparation metadata and explicit complete-call textFormat:'plain' consumed before engine dispatch; `prepareTTS({parts,storage,identity,textFormat,signal,onState})` returning ordered segments, state, ready, getAudio(index), and cancel(); async `readPreparedTTS({parts,textFormat,storage,identity,signal,onState})` returning a complete ready handle or null through the same semantic matcher, with no activation, unmute, synthesis, playback, storage writes, or partial repair; manifest, audio availability, and returned-handle reads use DBOPFS createTable:false so absent tables remain absent; supplied adapters own that read-existing behavior; missing matches/audio return null, malformed metadata and storage failures reject, and omitted storage sees only completed matching same-AI in-memory audio; hits replay ready state, misses emit nothing, and the lookup signal detaches on settlement while returned-handle cancellation stays independent; `prepareAudioOutput()` returning the persistent WebAudio handle {kind,state,contextState,error,subscribe,resume,dispose} without resume, unmute, model load, sources, sound, or gesture listeners; current-state replay and output-specific canonical `AI_AUDIO_OUTPUT_STATE_EVENT` (`ai-audio-output-state`), with ready only for an actually running context; explicit owner disposal observes context-bound cleanup and close; `playPreparedTTS(prepared,{signal,onState,audioOutput})` returning state, error, finished, pause(), resume(), and stop() with one playback lane per AI and observational state/error callbacks; explicit live same-AI audioOutput borrows its context until owner disposal, while omission or null preserves per-playback terminal close; configure/dispose speech, route lifecycle, declaration-validated chat/stream requests with exact ordered calls; optional streamRequest toolText:{name,field} and onToolText(text,call,displayId) for decoded selected argument text with actual call identity, distinct from ordinary prose and terminal tool execution; synthesis/transcription and playback controls; initializes from current canonical user readiness, installs `window.ai`, and projects `ai-ready` plus active-generation `ai-tts-failure`. `setAI(llmService,sttService,ttsService,model,modelTTS,modelSTT,options)` and `transitionAI(llmService,sttService,ttsService,model,modelTTS,modelSTT,options)` preserve six positional preferences; `configureProviders(selections,options)` and `transitionProviders(selections,options)` accept the same optional {signal:null,startLanguageModel:true} defaults. setAI retains its synchronous boolean and configureProviders its synchronous route result. Explicit `providerRuntime.load('llm',{signal})` or `startProviders({startLanguageModel:true,signal})` activates a deferred selected route."
|
|
35
35
|
},
|
|
36
36
|
{
|
|
37
37
|
"file": "runtime/arcane/modules/AIModelSelectionController.js",
|
|
@@ -92,7 +92,7 @@
|
|
|
92
92
|
"availability": "Cross-host in-process runtime; registered providers remain browser, native, or cloud specific",
|
|
93
93
|
"protocol": "arcane-ai-runtime/2, arcane-ai-provider/2, arcane-ai-model-authority/1",
|
|
94
94
|
"normalization": "Normalizes mutable complete per-role routes, lifecycle/status, cancellation, streaming cleanup, capacity-1 FIFO LLM/STT lanes, bounded provider-declared parallel TTS requests with FIFO overflow, default repeated-formatting-mark removal from cloned direct TTS payloads or explicit textFormat:'plain' preservation with the control consumed before engine dispatch, local-only selection, direct LLM history/declaration/terminal structural contracts, complete all-choice content/reasoning FIFO iteration, private automatic provider draining with result-first buffering, terminal-only structural calls, exact per-choice observed/terminal correlation, and a separate complete validated terminal result without creating a fallback.",
|
|
95
|
-
"surface": "Protocol constants; singleton-only `AIProviderRuntime`; `aiProviderRuntime`; `getAIProviderRuntime()`; provider registration/identity/selection, closed three-role and STT/TTS-only validation/configuration/replacement, status/catalog/inspection, startup, per-role load/unload/dispose/cancel, dispose-all, request/stream/speech aliases, SDK-internal speech-input preparation metadata, and setSpeechMuted(muted,{signal}={}) controls. Each pending unmute caller owns its abortable interest; the final interest aborts the owned activation while other preparation/playback or joining direct TTS-load interests preserve it. A pre-aborted unmute starts no activation. Explicit mute retains global cancellation and joined unload even with an aborted observer signal."
|
|
95
|
+
"surface": "supportsTranscriptionCapture(providerId=null) reports the provider capture capability; createTranscriptionCapture(options) returns null or an owned start/stop/cancel/destroy/done handle with optional forceEndpoint. Start requires an idle ready selected STT role, remains in the initiating user gesture and is never queued. Capture holds that role until done joins cleanup; cancellation/retirement uses the same role owner. Native streaming TTS uses the existing request lane with ordered awaited PCM callbacks. Protocol constants; singleton-only `AIProviderRuntime`; `aiProviderRuntime`; `getAIProviderRuntime()`; provider registration/identity/selection, closed three-role and STT/TTS-only validation/configuration/replacement, status/catalog/inspection, startup, per-role load/unload/dispose/cancel, dispose-all, request/stream/speech aliases, SDK-internal speech-input preparation metadata, and setSpeechMuted(muted,{signal}={}) controls. Each pending unmute caller owns its abortable interest; the final interest aborts the owned activation while other preparation/playback or joining direct TTS-load interests preserve it. A pre-aborted unmute starts no activation. Explicit mute retains global cancellation and joined unload even with an aborted observer signal."
|
|
96
96
|
},
|
|
97
97
|
{
|
|
98
98
|
"file": "runtime/arcane/modules/AIResponseURLPolicy.js",
|
|
@@ -442,11 +442,11 @@
|
|
|
442
442
|
"exports": [
|
|
443
443
|
"default"
|
|
444
444
|
],
|
|
445
|
-
"summary": "Owns continuous microphone capture with rolling pre-roll, activity-triggered pause/periodic segments, complete WAV delivery, final flush, and cancellation.",
|
|
445
|
+
"summary": "Owns continuous microphone capture with complete ordered PCM or rolling pre-roll, activity-triggered pause/periodic segments, complete WAV delivery, final flush, and cancellation.",
|
|
446
446
|
"availability": "Browser / native WebView with getUserMedia, AudioContext, and AudioWorklet",
|
|
447
447
|
"protocol": "Native microphone and Web Audio lifecycle; internal VoiceCaptureWorklet MessagePort",
|
|
448
|
-
"normalization": "Complete mono Float32 WAV Blob segments with session-local sequence, pause/periodic/stop reason, and durationMs. Sound amplitude drives segmentation without semantic speech recognition. Inactive audio produces no segment, and periodic segments do not reuse samples. Stop releases microphone tracks immediately and waits for final flush; cancel releases capture and suppresses late callbacks.",
|
|
449
|
-
"surface": "default ContinuousVoiceCapture; constructor({preRollMs=1500,quietMs=2000,chunkMs=30000,activityThreshold=0.02,onSegment,onError,onState}); start({mediaConstraints={audio:true},signal}={}) resolves boolean; stop() resolves after final flush; cancel() and destroy() release the owned session."
|
|
448
|
+
"normalization": "mode:pcm transfers every sample in ordered Float32Array records with actual sampleRate/channels/sequence; amplitude activity reports no semantic speech decision and never gates PCM. flush joins earlier PCM delivery without closing capture. Stop/cancel/destroy join accepted PCM and actual AudioContext close, surfacing failures. In the default segments mode: Complete mono Float32 WAV Blob segments with session-local sequence, pause/periodic/stop reason, and durationMs. Sound amplitude drives segmentation without semantic speech recognition. Inactive audio produces no segment, and periodic segments do not reuse samples. Stop releases microphone tracks immediately and waits for final flush; cancel releases capture and suppresses late callbacks.",
|
|
449
|
+
"surface": "default ContinuousVoiceCapture; constructor({preRollMs=1500,quietMs=2000,chunkMs=30000,activityThreshold=0.02,mode=segments,pcmChunkMs=100,onSegment,onPCM,onActivity,onError,onState}); start({mediaConstraints={audio:true},sampleRate,signal}={}) resolves boolean; flush() resolves after pending PCM delivery; stop() resolves after final flush and close; cancel() and destroy() release the owned session."
|
|
450
450
|
},
|
|
451
451
|
{
|
|
452
452
|
"file": "runtime/arcane/modules/ConversationActionItems.js",
|
|
@@ -991,6 +991,32 @@
|
|
|
991
991
|
"normalization": "Complete mutable definition data with coded syntax errors for malformed input.",
|
|
992
992
|
"surface": "`parseModelDefinition()`, `loadModelDefinitionSystemPrompt()`."
|
|
993
993
|
},
|
|
994
|
+
{
|
|
995
|
+
"file": "runtime/arcane/modules/NativeSpeechCapture.js",
|
|
996
|
+
"name": "NativeSpeechCapture.js",
|
|
997
|
+
"kind": "esm",
|
|
998
|
+
"exports": [
|
|
999
|
+
"default"
|
|
1000
|
+
],
|
|
1001
|
+
"summary": "Connects one microphone lifetime to one retained Core transcription stream without deciding conversational turns.",
|
|
1002
|
+
"availability": "Browser / native WebView with microphone, AudioWorklet and selected streaming Core speech",
|
|
1003
|
+
"protocol": "ContinuousVoiceCapture PCM and Arcane.speech.openStream",
|
|
1004
|
+
"normalization": "Exact ordered mono PCM at the actual declared rate; complete provisional/final snapshots and callback controls. forceEndpoint flushes audio and recognition while preserving capture and stream scope; stop drains terminally; cancel joins capture/native cleanup without claiming a complete transcript.",
|
|
1005
|
+
"surface": "default createNativeSpeechCapture({speech,model,sampleRate=16000,pcmChunkMs=100,activityThreshold=0.02,streamOptions,mediaConstraints,onUpdate,onInterim,onSegment,onActivity,onState,onError,onProgress,onRequest}); returns start({signal,mediaConstraints}), forceEndpoint(), stop(), requestStop(), cancel(reason), destroy(reason), done and current(). Recognition callbacks receive (result,{requestStop}); request a stop then return, and join externally. No implicit model load, conversation reply or fallback."
|
|
1006
|
+
},
|
|
1007
|
+
{
|
|
1008
|
+
"file": "runtime/arcane/modules/NativeSpeechPlayback.js",
|
|
1009
|
+
"name": "NativeSpeechPlayback.js",
|
|
1010
|
+
"kind": "esm",
|
|
1011
|
+
"exports": [
|
|
1012
|
+
"default"
|
|
1013
|
+
],
|
|
1014
|
+
"summary": "Prepares an inert native speech descriptor and plays ordered PCM through the existing shared Web Audio owner.",
|
|
1015
|
+
"availability": "Browser / native WebView with Web Audio and selected streaming speech",
|
|
1016
|
+
"protocol": "speech.synthesizeStream with awaited PCM consumption",
|
|
1017
|
+
"normalization": "Preparation starts no synthesis or playback. play schedules every supplied s16le sample in original order with audio-clock backpressure. Pause retains playback position; stop cancels only that utterance. finished and released distinguish playback termination from joined native cleanup; errors remain observable.",
|
|
1018
|
+
"surface": "default prepareNativeSpeechPlayback({speech,request,getAudioContext,signal=null}); returns {kind:native-speech,input,voice,language,speed,play}. play({signal,onState}) returns finished/released/state/error/pause/resume/stop. The caller owns its AudioContext; this module never closes unrelated output."
|
|
1019
|
+
},
|
|
994
1020
|
{
|
|
995
1021
|
"file": "runtime/arcane/modules/Ollama.js",
|
|
996
1022
|
"name": "Ollama.js",
|
|
@@ -1462,11 +1488,11 @@
|
|
|
1462
1488
|
"name": "VoiceCaptureWorklet.js",
|
|
1463
1489
|
"kind": "worker",
|
|
1464
1490
|
"exports": [],
|
|
1465
|
-
"summary": "
|
|
1491
|
+
"summary": "Transfers continuous PCM or segments microphone samples on the audio rendering thread and transfers complete mono Float32 WAV clips to ContinuousVoiceCapture.",
|
|
1466
1492
|
"availability": "AudioWorkletGlobalScope",
|
|
1467
1493
|
"protocol": "AudioWorklet processor arcane-continuous-voice-capture and its owner-local MessagePort",
|
|
1468
|
-
"normalization": "
|
|
1469
|
-
"surface": "No ESM exports. ContinuousVoiceCapture loads this static worklet, supplies processorOptions, receives segment messages, and requests stop followed by the stopped acknowledgement."
|
|
1494
|
+
"normalization": "mode:pcm emits every original sample in ordered PCM frames at the actual worklet sample rate, plus activity observations; pcmChunkMs defines transport scheduling, not a recording cap. flush returns pending PCM and a correlated flushed acknowledgement without closing the processor. The default segments mode retains rolling pre-roll, fixed 20 ms RMS activity windows independent of callback length, audio-frame quiet duration, periodic boundaries, sequence, complete original WAV audio, and stop acknowledgement. activityThreshold is RMS amplitude, not peak amplitude. Stop classifies a partial final window using its actual samples and retains softer trailing audio in the active clip. Inactive capture emits no clips; adjacent periodic clips have no repeated samples.",
|
|
1495
|
+
"surface": "No ESM exports. ContinuousVoiceCapture loads this static worklet, supplies processorOptions, receives segment or pcm/activity/flushed messages, and requests stop followed by the stopped acknowledgement."
|
|
1470
1496
|
},
|
|
1471
1497
|
{
|
|
1472
1498
|
"file": "runtime/arcane/modules/WaitForComponent.js",
|
|
@@ -110,11 +110,11 @@ try {
|
|
|
110
110
|
}
|
|
111
111
|
```
|
|
112
112
|
|
|
113
|
-
`prepare({id,workingDirectory,members,signal,onProgress,refresh=false})` reuses
|
|
113
|
+
`prepare({id,workingDirectory,members,archives,signal,onProgress,refresh=false})` reuses
|
|
114
114
|
a complete native acquisition or fetches the explicitly selected members.
|
|
115
115
|
Every member has its original relative `path` and selected `url`; the caller
|
|
116
116
|
supplies the complete companion-file set. Independent members download
|
|
117
|
-
concurrently. Each complete response stream is written directly to its native
|
|
117
|
+
concurrently, with up to four independent source transfers active. Each complete response stream is written directly to its native
|
|
118
118
|
file, without a whole-model buffer or base64 conversion. No model or runtime
|
|
119
119
|
is selected or downloaded by service startup. The result remains the ready
|
|
120
120
|
projection snapshot with ordered `members: [{path,nativePath}]`.
|
|
@@ -144,6 +144,158 @@ callback does not hold another caller's transfer or result. Callbacks must not
|
|
|
144
144
|
await cleanup of the preparation currently calling them. Callback failures
|
|
145
145
|
reject that caller's preparation. HTTP failures
|
|
146
146
|
retain `url`, `status` and the complete response text in `response`.
|
|
147
|
+
During an active member, repeated activity notifications are emitted at most
|
|
148
|
+
once per second; opening, member completion and terminal phases remain distinct.
|
|
149
|
+
Progress describes files and phases, never transfer volume.
|
|
150
|
+
|
|
151
|
+
### Selected archive companions
|
|
152
|
+
|
|
153
|
+
An optional `archives` array acquires exact companion members from an upstream
|
|
154
|
+
uncompressed TAR source through this same acquisition owner:
|
|
155
|
+
|
|
156
|
+
```javascript
|
|
157
|
+
const projection = await modelAssets.prepare({
|
|
158
|
+
id: selectedOperationId,
|
|
159
|
+
workingDirectory: selectedModelDirectory,
|
|
160
|
+
members: selectedModelFiles,
|
|
161
|
+
archives: [{
|
|
162
|
+
url: selectedArchiveURL,
|
|
163
|
+
format: 'tar',
|
|
164
|
+
members: [
|
|
165
|
+
{path: 'tokenizer/model_config.yaml', member: 'model_config.yaml'},
|
|
166
|
+
{path: 'tokenizer/speakers.json', member: 'speakers.json'}
|
|
167
|
+
]
|
|
168
|
+
}],
|
|
169
|
+
signal,
|
|
170
|
+
onProgress
|
|
171
|
+
});
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
The result orders ordinary members first, followed by each archive's declared
|
|
175
|
+
members. Extraction preserves each complete selected file. TAR names and PAX
|
|
176
|
+
path records select members; a leading `./` is archive path framing. Missing
|
|
177
|
+
selected members reject preparation and identify the missing names. Gzip and
|
|
178
|
+
other archive formats are outside this reader's contract.
|
|
179
|
+
|
|
180
|
+
When the upstream catalog supplies aligned TAR segments, an archive may include
|
|
181
|
+
`ranges:[{start,end,stopBefore?}]`. The reader requests each inclusive HTTP range
|
|
182
|
+
once and parses its TAR framing; `stopBefore` ends that segment before the
|
|
183
|
+
specified member. Offsets and TAR lengths are solely external transport framing.
|
|
184
|
+
They are never download progress, model readiness criteria or content identity.
|
|
185
|
+
Ranges require the matching HTTP partial-content response; no complete-archive
|
|
186
|
+
fallback is silently substituted. Native reuse identifies archive members by
|
|
187
|
+
their selected URL, archive member name and output path, alongside ordinary
|
|
188
|
+
members, without re-fetching an already completed set.
|
|
189
|
+
|
|
190
|
+
## Discover and open installed files
|
|
191
|
+
|
|
192
|
+
`discover({directories,catalog=[],recursive=true,signal})` inspects only the
|
|
193
|
+
explicit directories, without downloading, copying, loading or selecting a
|
|
194
|
+
model. Paths resolve from `appRoot`. It returns:
|
|
195
|
+
|
|
196
|
+
```javascript
|
|
197
|
+
{
|
|
198
|
+
directories: [{directory, exists, error}], // error is present only on failure
|
|
199
|
+
models: [{
|
|
200
|
+
directory,
|
|
201
|
+
entryPath,
|
|
202
|
+
members: [{path, nativePath, available}],
|
|
203
|
+
missingMembers: [],
|
|
204
|
+
complete: true,
|
|
205
|
+
source: 'directory', // or 'catalog' or 'acquisition'
|
|
206
|
+
catalogId // present for a matching caller-owned catalog entry
|
|
207
|
+
}]
|
|
208
|
+
}
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
Ordinary `.gguf` files are candidates. Split names such as
|
|
212
|
+
`moon-00001-of-00003.gguf` form one candidate with all expected members in numeric
|
|
213
|
+
order, including explicit missing members. The first shard is `entryPath`,
|
|
214
|
+
including when a completed acquisition declared only later shards or listed
|
|
215
|
+
them out of order. Acquisition candidates retain all their ordinary and
|
|
216
|
+
companion members alongside these required shards; separate filename groups
|
|
217
|
+
are never inferred to be one model. Symlinked model files and directories are
|
|
218
|
+
followed, with already visited directory targets reused rather than rescanned.
|
|
219
|
+
Discovery does not establish a model's role, runtime compatibility or successful
|
|
220
|
+
inference. Those selections remain with the application and engine owners.
|
|
221
|
+
|
|
222
|
+
Catalog entries are `{id,members:[{path,url?}],archives?,entryPath?}`. They describe
|
|
223
|
+
an exact companion set and can include nested tokenizer files or multiple model
|
|
224
|
+
files. A discovered catalog candidate reports every expected member, including
|
|
225
|
+
missing companions. Completed native acquisition records are also discoverable;
|
|
226
|
+
when the catalog supplies URLs or archive member names, acquisition matching
|
|
227
|
+
also preserves those recorded source selections. A different pinned source with
|
|
228
|
+
the same filenames is not relabeled as that catalog selection. Path-only catalogs
|
|
229
|
+
and ordinary installed-directory candidates make no source-revision claim.
|
|
230
|
+
Unfinished acquisition attempts are ignored. Discovery never removes either.
|
|
231
|
+
Set `recursive:false` for a directory-only inspection. Missing root directories
|
|
232
|
+
are reported with `exists:false`; other root access failures include their
|
|
233
|
+
complete serialized error. A failure while reading a descendant is surfaced.
|
|
234
|
+
|
|
235
|
+
`openExisting({id,directory,members:[{path}],signal})` opens a complete selected
|
|
236
|
+
set as an ordinary ready projection. It rechecks every member and reports
|
|
237
|
+
`MODEL_ASSET_MEMBERS_MISSING` with `members` when files are absent. It performs
|
|
238
|
+
no copy or network request. Engines use the same `retain(id)` lifecycle.
|
|
239
|
+
Preparation release, engine unload and service disposal never delete borrowed
|
|
240
|
+
originals. External edits or deletion remain controlled by the files' owner.
|
|
241
|
+
|
|
242
|
+
The application supplies persistent model directories separately from disposable
|
|
243
|
+
build output. The native launch context exposes its platform-specific
|
|
244
|
+
`modelsDirectory`; apps may additionally inspect selected application directories.
|
|
245
|
+
The model-assets service creates no platform directory policy of its own.
|
|
246
|
+
|
|
247
|
+
`resolveNativeLaunchContext({appId,context})` defaults `modelsDirectory` to
|
|
248
|
+
`Models` beneath the resolved persistent `stateRoot`. It preserves an explicitly
|
|
249
|
+
supplied `modelsDirectory` unchanged and creates or migrates no files:
|
|
250
|
+
|
|
251
|
+
| Host | Default application state root |
|
|
252
|
+
| --- | --- |
|
|
253
|
+
| Windows | `%LOCALAPPDATA%/Arcane/<appId>`; the user's `AppData/Local` when that environment variable is absent |
|
|
254
|
+
| macOS | `~/Library/Application Support/Arcane/<appId>` |
|
|
255
|
+
| Linux | `$XDG_DATA_HOME/Arcane/<appId>` when absolute, otherwise `~/.local/share/Arcane/<appId>` |
|
|
256
|
+
| Android or another host adaptation | The host supplies its explicit app-private `stateRoot` |
|
|
257
|
+
|
|
258
|
+
Pass this `modelsDirectory` as the acquisition `workingDirectory` and as one
|
|
259
|
+
discovery directory. Completed files then live beneath its `model-assets`
|
|
260
|
+
subdirectory, separate from replaceable application builds. Additional selected
|
|
261
|
+
application directories can participate in the same discovery call.
|
|
262
|
+
|
|
263
|
+
### Prepare a selected local runtime on demand
|
|
264
|
+
|
|
265
|
+
`createLocalAIService(configuration,{autoStart:false,...options})` registers the
|
|
266
|
+
existing local-AI owner without preparing or starting its selected runtimes at
|
|
267
|
+
Core startup. The default remains `autoStart:true`, preserving existing startup
|
|
268
|
+
behavior. After the user explicitly requests preparation, invoke
|
|
269
|
+
`client.invoke('localai.prepare', {}, {signal,timeoutMs:0})`. The operation uses
|
|
270
|
+
the configured existing runtime preparation callback and returns the current
|
|
271
|
+
local-AI state; it does not invent a runtime or model selection.
|
|
272
|
+
|
|
273
|
+
Preparation callback events are published as `localai.preparation` with their
|
|
274
|
+
complete original event payload. Subscribe through the existing Core event
|
|
275
|
+
owner to display actual preparation phases. Concurrent preparation requests
|
|
276
|
+
reuse the owner's in-flight work, and subsequent completed calls reuse its
|
|
277
|
+
prepared state. Runtime and model selection, explicit model acquisition and
|
|
278
|
+
microphone activation remain app-owned; preparation is not an inference or
|
|
279
|
+
microphone operation. Rendering the interface and
|
|
280
|
+
discovering installed files need not wait for runtime preparation.
|
|
281
|
+
|
|
282
|
+
### Browser discovery and explicit download
|
|
283
|
+
|
|
284
|
+
`arcane-os/ai/core-model-assets` additionally exports
|
|
285
|
+
`discoverCoreModelAssets({client,directories,catalog,recursive,signal})`,
|
|
286
|
+
`openExistingCoreModelAssets({client,directory,members,signal})`, and
|
|
287
|
+
`downloadCoreModelAssets({client,workingDirectory,members,archives,refresh,signal,onProgress})`.
|
|
288
|
+
`client` defaults to the installed Core client. Discovery returns the native
|
|
289
|
+
record above; opening and downloading return `{id,directory,members,release}`.
|
|
290
|
+
The existing Blob-based `prepareCoreModelAssets` keeps its original contract.
|
|
291
|
+
|
|
292
|
+
Download progress callbacks receive the complete semantic progress record,
|
|
293
|
+
correlated to this preparation and observed in order. Callback failure cancels
|
|
294
|
+
the operation. Cancellation joins explicit native cleanup and local callback
|
|
295
|
+
delivery before the helper settles; cleanup failures remain observable.
|
|
296
|
+
Callbacks must not await their own enclosing preparation or cancellation join.
|
|
297
|
+
Applications initiate downloads explicitly and display actual phases, selected
|
|
298
|
+
member paths, completed-file counts and errors in the appropriate surface.
|
|
147
299
|
|
|
148
300
|
`release(id)` relinquishes preparation ownership without a request signal.
|
|
149
301
|
Same-process callers sharing the actual store, complete source selection and
|
|
@@ -186,7 +338,8 @@ try {
|
|
|
186
338
|
```
|
|
187
339
|
|
|
188
340
|
`createModelAssetService({appRoot})` returns a Core service with `current()`,
|
|
189
|
-
`
|
|
341
|
+
`discover(options)`, `openExisting(options)`, `prepare(options)`, `retain(id)`,
|
|
342
|
+
`release(id)` and `dispose()`. `retain()` accepts only a completed projection
|
|
190
343
|
and returns `{id,directory,members,release}`. A native engine takes this handle
|
|
191
344
|
before loading and releases it after actual unload, failed-load cleanup or
|
|
192
345
|
worker exit. A rejected inference/load promise alone does not establish native
|
|
@@ -249,6 +402,10 @@ for source selection, cancellation and complete row/output contracts.
|
|
|
249
402
|
| Method | Parameters | Result |
|
|
250
403
|
| --- | --- | --- |
|
|
251
404
|
| `modelAssets.open` | `{id,workingDirectory,members:[{path}]}` | Preparing projection and native member paths |
|
|
405
|
+
| `modelAssets.discover` | `{directories,catalog?,recursive?}` | Directory status and complete/incomplete model candidates |
|
|
406
|
+
| `modelAssets.openExisting` | `{id,directory,members:[{path}]}` | Ready retained projection of existing originals |
|
|
407
|
+
| `modelAssets.prepare` | `{id,workingDirectory,members,archives?,refresh?}` | Ready persistent acquisition; request-scoped `modelAssets.progress` emits `{id,progress}` |
|
|
408
|
+
| `modelAssets.cancelPrepare` | `{id}` | Cancel and join the matching native preparation, then release its preparation ownership |
|
|
252
409
|
| `modelAssets.write` | `{id,memberIndex,contentBase64}` | `{id,memberIndex,written:true}` after the native write |
|
|
253
410
|
| `modelAssets.complete` | `{id}` | Ready projection after all files close |
|
|
254
411
|
| `modelAssets.release` | `{id}` | Released preparation ownership; native uses may retain the files |
|
|
@@ -258,6 +415,10 @@ The browser helper owns this transport sequence. Correlation IDs identify
|
|
|
258
415
|
operations; they are unrelated to content identity. Original model files remain
|
|
259
416
|
separate and complete. A retry creates a new working projection from the stored
|
|
260
417
|
assets; it does not redownload them or replace the authoritative DBOPFS record.
|
|
418
|
+
`cancelPrepare` and `release` have service lifetime so cancellation of the
|
|
419
|
+
original request cannot cancel its cleanup control. Native preparation follows
|
|
420
|
+
the original request signal, including connection cancellation; raw RPC callers
|
|
421
|
+
join `cancelPrepare` when cancelling and release any successful unused projection.
|
|
261
422
|
|
|
262
423
|
The implementation uses browser streams, the existing portable Core JSON
|
|
263
424
|
transport and Node filesystem operations. Native paths and cleanup follow the
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Native speech service
|
|
2
2
|
|
|
3
|
-
`createSpeechService({stt, tts, signal} = {})` composes independently selected
|
|
3
|
+
`createSpeechService({stt, tts, signal, autoLoad = true} = {})` composes independently selected
|
|
4
4
|
native speech engines into one Core service. It is the named and default export
|
|
5
5
|
from `arcane-os/core/speech`.
|
|
6
6
|
|
|
@@ -50,6 +50,13 @@ independently. It returns immediately, allowing Core and `speech.status` to
|
|
|
50
50
|
respond while models load. A failure in one role leaves the other role usable.
|
|
51
51
|
Each engine publishes the actual state of its selected model.
|
|
52
52
|
|
|
53
|
+
Use `autoLoad: false` when the host first needs the user to select or download
|
|
54
|
+
model files. Startup still subscribes and exposes the ordinary unloaded state;
|
|
55
|
+
it does not call either engine's `load()`. Explicit `speech.load({role,
|
|
56
|
+
...selection})` activates the prepared role afterward. The default remains
|
|
57
|
+
`true` for existing hosts. This option changes service startup, not the selected
|
|
58
|
+
engine's missing-model errors or inference readiness requirement.
|
|
59
|
+
|
|
53
60
|
`current()` and `speech.status` return the same snapshot:
|
|
54
61
|
|
|
55
62
|
| Field | Meaning |
|
|
@@ -66,9 +73,10 @@ An engine's state includes `providerId`, `modelId`, `state`, `loaded`, and
|
|
|
66
73
|
`busy`, with its complete error or progress information when present. An
|
|
67
74
|
omitted role has `state: 'unavailable'`, `loaded: false`, and `busy: false`.
|
|
68
75
|
File presence, a spawned process, and a callable bridge do not establish loaded
|
|
69
|
-
model readiness.
|
|
70
|
-
|
|
71
|
-
|
|
76
|
+
model readiness. The shared AI adapter follows the selected engine identity
|
|
77
|
+
and model in these per-role snapshots, including Nemotron and Magpie. Only the
|
|
78
|
+
older aggregate-only fallback retains fixed `whisper.cpp`/`whisper-small` and
|
|
79
|
+
`kokoro-onnx`/`kokoro` expectations.
|
|
72
80
|
|
|
73
81
|
Each role also reports `supportsStreaming`: STT requires the configured
|
|
74
82
|
engine's `openStream()` method, and TTS requires its `synthesizeStream()` method.
|
|
@@ -288,6 +296,99 @@ share one shutdown promise. Actual engine cleanup failures reject with an
|
|
|
288
296
|
releases the service's event subscriptions. New work after shutdown begins
|
|
289
297
|
rejects with `CORE_CLOSING`.
|
|
290
298
|
|
|
299
|
+
## Selected NeMo model acquisition
|
|
300
|
+
|
|
301
|
+
`NEMO_SPEECH_MODELS`, `prepareNemotronModels`, and `prepareMagpieModels` are
|
|
302
|
+
named exports from `arcane-os/local-ai/nemotron`. Importing them performs no
|
|
303
|
+
network access, installation, model loading, or inference. The catalog records
|
|
304
|
+
NVIDIA's selected [NeMo-Speech.cpp v0.2.0 model index](https://github.com/NVIDIA/NeMo-Speech.cpp/blob/v0.2.0/models/index.json):
|
|
305
|
+
|
|
306
|
+
| Role | Complete default selection |
|
|
307
|
+
|---|---|
|
|
308
|
+
| `stt` | Nemotron 3.5 streaming ASR Q8_0 and Nemotron 3 Diarization Q8_0. |
|
|
309
|
+
| `tts` | Magpie multilingual v2607 F16, NanoCodec 22 kHz F16 decoder, and all ten matching tokenizer/configuration/speaker-map members from the same Magpie revision. |
|
|
310
|
+
|
|
311
|
+
Each role exposes `{modelId, providerId, name, members, resourcePaths}`.
|
|
312
|
+
`members` contains ordered `{path, url}` file sources. TTS also exposes
|
|
313
|
+
`archives`, with the upstream uncompressed `.nemo` TAR source, its selected
|
|
314
|
+
transport ranges, and exact `{path, member}` tokenizer entries. The shared
|
|
315
|
+
[model-assets service](model-assets.md) owns discovery, acquisition, progress,
|
|
316
|
+
cancellation, persistence, and projection lifetime; these helpers do not create
|
|
317
|
+
another downloader or store. Tokenizer files retain their complete original
|
|
318
|
+
contents. The selected tokenizer configuration and its referenced JSON speaker
|
|
319
|
+
map continue to own voice IDs.
|
|
320
|
+
|
|
321
|
+
```js
|
|
322
|
+
import {createNemotronRuntime, prepareNemotronModels} from 'arcane-os/local-ai/nemotron';
|
|
323
|
+
import {createSpeechService} from 'arcane-os/core/speech';
|
|
324
|
+
|
|
325
|
+
const stt = createNemotronRuntime({runtime, modelAssets, gpu: -1});
|
|
326
|
+
const speech = createSpeechService({stt, autoLoad: false});
|
|
327
|
+
// Run this only for the user's explicit model preparation action.
|
|
328
|
+
const prepared = await prepareNemotronModels({
|
|
329
|
+
modelAssets,
|
|
330
|
+
workingDirectory: modelsDirectory,
|
|
331
|
+
directories: [modelsDirectory, appModelsDirectory],
|
|
332
|
+
signal,
|
|
333
|
+
onProgress: showModelProgress
|
|
334
|
+
});
|
|
335
|
+
try {
|
|
336
|
+
await stt.load({
|
|
337
|
+
assetProjectionId: prepared.assetProjectionId,
|
|
338
|
+
resourcePaths: prepared.resourcePaths,
|
|
339
|
+
signal
|
|
340
|
+
});
|
|
341
|
+
} finally {
|
|
342
|
+
await prepared.release();
|
|
343
|
+
}
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
Both preparation helpers accept `{modelAssets, workingDirectory, directories,
|
|
347
|
+
id, signal, onProgress, refresh = false}`. With `directories`, they ask the
|
|
348
|
+
shared owner for the first complete matching catalog selection in those roots
|
|
349
|
+
before downloading. `openExisting()` borrows its original files and never
|
|
350
|
+
deletes them. Incomplete selections stay visible through the owner's discovery
|
|
351
|
+
results and are not mistaken for a complete model. When no complete match is
|
|
352
|
+
available, the existing acquisition path prepares the complete selected set;
|
|
353
|
+
its completed downloads remain reusable. Explicit `refresh: true` requests a
|
|
354
|
+
fresh acquisition. The host chooses the roots: its portable native launch
|
|
355
|
+
context supplies `modelsDirectory` under the resolved per-user `stateRoot` by
|
|
356
|
+
default, while explicit directory overrides remain supported. An application
|
|
357
|
+
may include its own `models` directory among the discovery roots.
|
|
358
|
+
|
|
359
|
+
The result is `{assetProjectionId, directory, members, resourcePaths, release}`.
|
|
360
|
+
`release()` is idempotent and releases preparation ownership. Each engine
|
|
361
|
+
retains the projection synchronously when `load()` accepts it, so the caller
|
|
362
|
+
may release its preparation after starting that load. Nemotron accepts
|
|
363
|
+
`modelAssets`, `assetProjectionId`, and `resourcePaths` in its constructor and
|
|
364
|
+
`assetProjectionId`/`resourcePaths` in `load()`, with roles `{model, diarization}`.
|
|
365
|
+
Magpie uses `{model, codec, tokenizerConfig}`. Native file paths still work
|
|
366
|
+
through the engines' existing options. Nemotron requires unloading before
|
|
367
|
+
changing an active selection; its retained files are released only after
|
|
368
|
+
native work and helper exit have joined. Load failures also join asset cleanup,
|
|
369
|
+
and cleanup failures remain observable.
|
|
370
|
+
|
|
371
|
+
`onProgress` receives ordered, awaited shared-owner phase and file-completion
|
|
372
|
+
events, including reuse and ready for installed selections. Signal cancellation
|
|
373
|
+
stops that preparation's work; an abort after projection creation releases it
|
|
374
|
+
before rejecting. Engine load cancellation remains separate: cancelling the
|
|
375
|
+
caller's wait does not cancel a shared activation; explicit engine unload/close
|
|
376
|
+
owns that cancellation.
|
|
377
|
+
|
|
378
|
+
The default GGUF selections are individual complete upstream files. The shared
|
|
379
|
+
owner also discovers ordered GGUF shard sets and reports their missing members;
|
|
380
|
+
HTTP range transfers and tokenizer TAR ranges are transport mechanisms, not
|
|
381
|
+
additional model shards. A discovered shard set alone does not establish that a
|
|
382
|
+
selected native engine can load that model. The engine remains the authority
|
|
383
|
+
for model compatibility and actual readiness.
|
|
384
|
+
|
|
385
|
+
The [upstream native runtime](https://github.com/NVIDIA/NeMo-Speech.cpp/blob/v0.2.0/docs/install.md)
|
|
386
|
+
provides Windows, Linux, and macOS distributions. Model files and that runtime
|
|
387
|
+
remain separate upstream assets, not vendored SDK payloads. This acquisition
|
|
388
|
+
integration received source review only; no download, inference, or new platform
|
|
389
|
+
execution is claimed here. See the [Nemotron](native-nemotron.md) and
|
|
390
|
+
[Magpie](native-magpie.md) contracts for native preparation and execution evidence.
|
|
391
|
+
|
|
291
392
|
## Native Kokoro engine
|
|
292
393
|
|
|
293
394
|
`createNativeKokoroRuntime` is the named and default export from
|