arcane-os 0.63.0 → 0.65.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +48 -0
- package/README.md +1 -1
- package/browser-runtime/core/client.mjs +69 -7
- package/docs/guides/native-whisper-openvino.md +334 -0
- package/docs/reference/README.md +6 -0
- package/docs/reference/arcane-ollama.md +1 -1
- package/docs/reference/codex-app-server.md +269 -0
- package/docs/reference/core-client.md +71 -4
- package/docs/reference/core-development.md +157 -0
- package/docs/reference/core-native-packaging.md +48 -6
- package/docs/reference/core-repositories.md +258 -0
- package/docs/reference/inventory/package-api.json +199 -3
- package/docs/reference/inventory/runtime-modules.json +3 -2
- package/docs/reference/local-ai.md +77 -4
- package/docs/reference/local-image-flux.md +1 -1
- package/docs/reference/local-image-generation.md +11 -0
- package/docs/reference/local-image-sdxl.md +139 -0
- package/docs/reference/local-whisper.md +219 -0
- package/docs/reference/mcp-stdio.md +263 -0
- package/docs/reference/native-speech.md +51 -2
- package/docs/reference/runtime-components.md +18 -1
- package/docs/reference/runtime-modules.md +26 -4
- package/docs/reference/sdk-api.md +412 -2
- package/package.json +9 -2
- package/runtime/arcane/components/speech.html +15 -3
- package/runtime/arcane/components/voice-transcription.html +19 -3
- package/runtime/arcane/modules/AI.js +406 -40
- package/runtime/arcane/modules/AIProviderRuntime.js +20 -3
- package/runtime/arcane/modules/AIRuntimeState.js +15 -2
- package/runtime/arcane/modules/ComponentContracts.js +3 -0
- package/runtime/arcane/modules/Ollama.js +1 -1
- package/runtime/arcane/modules/OllamaModelIdentifier.js +13 -0
- package/schemas/arcane-app.schema.json +60 -2
- package/schemas/arcane-package.schema.json +60 -2
- package/src/codex/app-server.mjs +845 -0
- package/src/core/development.mjs +457 -0
- package/src/core/repositories.mjs +375 -0
- package/src/core/services/local-ai.mjs +80 -10
- package/src/core/services/speech.mjs +1 -0
- package/src/dev-server.mjs +3 -2
- package/src/index.mjs +1 -0
- package/src/local-ai/config.mjs +7 -3
- package/src/local-ai/dev.mjs +85 -401
- package/src/local-ai/image-library.mjs +9 -0
- package/src/local-ai/image-models/sdxl.mjs +18 -0
- package/src/local-ai/install.mjs +26 -7
- package/src/local-ai/whisper/build.mjs +48 -0
- package/src/local-ai/whisper/config.mjs +89 -0
- package/src/local-ai/whisper/index.mjs +572 -0
- package/src/local-ai/whisper/install.mjs +332 -0
- package/src/local-ai/whisper/native/CMakeLists.txt +98 -0
- package/src/local-ai/whisper/native/NOTICE +18 -0
- package/src/local-ai/whisper/native/README.md +104 -0
- package/src/local-ai/whisper/native/main.cpp +520 -0
- package/src/local-ai/whisper/openvino-build.mjs +138 -0
- package/src/mcp/stdio-server.mjs +579 -0
- package/src/native/portable-layout.mjs +39 -7
- package/src/toolchain.mjs +19 -8
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,53 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.65.0
|
|
4
|
+
|
|
5
|
+
- Add optional precompiled Windows x64 Intel NPU encoding for native Whisper
|
|
6
|
+
with `encoder: "openvino-npu"`, paired small-model resources, separate
|
|
7
|
+
encoder/decoder state, and existing GPU-first decoding with CPU recovery.
|
|
8
|
+
Preserve ordinary Whisper, complete transcripts and joined cancellation.
|
|
9
|
+
- Add `createRepositoryWorkspace(...).write({files, message, signal})` for
|
|
10
|
+
complete selected text, literal-path staging/commit and ordinary push, with
|
|
11
|
+
complete diagnostics and explicit confirmed or uncertain partial outcomes.
|
|
12
|
+
- Add retained native Codex App Server sessions and an explicit MCP STDIO
|
|
13
|
+
server through `arcane-os/codex/app-server` and `arcane-os/mcp/stdio`.
|
|
14
|
+
Preserve complete native payloads, origin-bound server-request replies,
|
|
15
|
+
ordered live results, cooperative cancellation and owned write settlement.
|
|
16
|
+
- Add application Core composition during source development without requiring
|
|
17
|
+
local AI, `/arcane-core.js`, and replayed client installation/retirement
|
|
18
|
+
observation. Preserve the existing local-AI bootstrap, early-close cleanup
|
|
19
|
+
and ordered reentrant installation notifications.
|
|
20
|
+
- Tie Ollama readiness to actual selected-model residency observations, with
|
|
21
|
+
ordinary model-name alias comparison, operation cancellation and stale
|
|
22
|
+
observation/engine ownership. Preserve original outbound model names and
|
|
23
|
+
complete payloads; no background polling or atomic residency claim.
|
|
24
|
+
- Keep the npm dependency tree unchanged. Native runtime producers and
|
|
25
|
+
consumers use the documented upstream resources and compatible helper
|
|
26
|
+
assets attached to this numeric release. Git identity selection and native
|
|
27
|
+
Laya fp32 integration remain separate subsequent work.
|
|
28
|
+
|
|
29
|
+
## 0.64.0
|
|
30
|
+
|
|
31
|
+
- Add retained native Whisper transcription through the existing Core speech
|
|
32
|
+
service and AI API, with actual model readiness, complete recordings and
|
|
33
|
+
transcripts, native progress, joined cancellation, and one eligible
|
|
34
|
+
accelerated-inference recovery on the prepared CPU runtime.
|
|
35
|
+
- Preserve the same accepted transcription through internal recovery using
|
|
36
|
+
its actual Core request identity. Shared AI and speech controls report
|
|
37
|
+
recovering/error states honestly without discarding the original request
|
|
38
|
+
or turning its terminal native failure into cancellation.
|
|
39
|
+
- Add the SDXL Base 1.0 text-to-image descriptor and correct the shared native
|
|
40
|
+
image binding's asynchronous worker stack allocation while preserving
|
|
41
|
+
existing model selection, complete inputs and native lifecycle.
|
|
42
|
+
- Add `arcane-os/core/repositories` with per-user `ArcaneData/Repos` defaults
|
|
43
|
+
and connected working-repository open/status/pull/push/drain ownership.
|
|
44
|
+
Explicit paths and existing data remain unchanged; applications own
|
|
45
|
+
connection selection, remotes and product records. No data migration.
|
|
46
|
+
- Keep optional NPU, repository file-write orchestration, generic development
|
|
47
|
+
Core composition, Codex/MCP and Ollama lifecycle drafts outside this release.
|
|
48
|
+
The npm dependency tree is unchanged; native resources use their documented
|
|
49
|
+
upstream installation and matching numeric-release helper assets.
|
|
50
|
+
|
|
3
51
|
## 0.63.0
|
|
4
52
|
|
|
5
53
|
- Add the approved FLUX.2 Klein 4B text-to-image model descriptor to the
|
package/README.md
CHANGED
|
@@ -20,7 +20,7 @@ selected SDK projection or an explicit live SDK source mount. Both profiles
|
|
|
20
20
|
share the theme, packaging, event, cancellation,
|
|
21
21
|
and browser run contracts while retaining their selected application layout.
|
|
22
22
|
|
|
23
|
-
This checkout defines the `0.
|
|
23
|
+
This checkout defines the `0.65.0` SDK contract. Development applications declare
|
|
24
24
|
`arcane-os` as `latest` and retain their lockfile. Exact SDK pins begin when Roshi
|
|
25
25
|
designates a production build. Follow [Consumer updates](docs/consumer-updates.md)
|
|
26
26
|
to refresh and compare the installed SDK and lockfile with the published channel;
|
|
@@ -3,9 +3,34 @@ import {CORE_PROTOCOL,CORE_READY_EVENTS,CoreError,serializeCoreError} from './co
|
|
|
3
3
|
|
|
4
4
|
const CORE_CLIENT_KEY=Symbol.for('arcane-os.core.client');
|
|
5
5
|
const CORE_CLIENT_INSTALLATION=Symbol.for('arcane-os.core.client.installation');
|
|
6
|
+
const CORE_CLIENT_OBSERVATION=Symbol.for('arcane-os.core.client.observation');
|
|
7
|
+
const CORE_CLIENT_INSTALLATION_EVENT='core.client.installation';
|
|
6
8
|
const CORE_EVENT='core.rpc.event';
|
|
7
9
|
const LONG_OPERATION_TIMEOUT=50*60*1000;
|
|
8
10
|
|
|
11
|
+
function publishCoreClientInstallation(global,previousClient,reason,error=null){
|
|
12
|
+
// A classic document-created installation may precede the ESM event owner.
|
|
13
|
+
// The live installation already owns replay; retain no retired-client record.
|
|
14
|
+
const observation=global[CORE_CLIENT_OBSERVATION];
|
|
15
|
+
if(!observation?.source)return;
|
|
16
|
+
observation.notifications.push({client:getInstalledCoreClient(global),previousClient,reason,error});
|
|
17
|
+
if(observation.publishing)return;
|
|
18
|
+
observation.publishing=true;
|
|
19
|
+
let delivered=0;
|
|
20
|
+
try{
|
|
21
|
+
// A listener may change the installation. Finish notifying this
|
|
22
|
+
// transition's observers before publishing that successor transition.
|
|
23
|
+
while(delivered<observation.notifications.length){
|
|
24
|
+
const notification=observation.notifications[delivered];
|
|
25
|
+
observation.notifications[delivered++]=null;
|
|
26
|
+
observation.source.dispatch(CORE_CLIENT_INSTALLATION_EVENT,notification);
|
|
27
|
+
}
|
|
28
|
+
}finally{
|
|
29
|
+
observation.notifications.splice(0,delivered);
|
|
30
|
+
observation.publishing=false;
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
|
|
9
34
|
/** One RPC client. Native execution and service policy belong to the host. */
|
|
10
35
|
export function createCoreClient({
|
|
11
36
|
global=globalThis,
|
|
@@ -459,7 +484,10 @@ export function createCoreClient({
|
|
|
459
484
|
if(eventOwnerFailure)return Promise.reject(eventOwnerFailure);
|
|
460
485
|
let selected;
|
|
461
486
|
try{selected=connect();}catch(error){return Promise.reject(error);}
|
|
462
|
-
const {signal,timeoutMs=10*60*1000}=options??{};
|
|
487
|
+
const {signal,timeoutMs=10*60*1000,onRequest}=options??{};
|
|
488
|
+
if(onRequest!==undefined&&typeof onRequest!=='function'){
|
|
489
|
+
return Promise.reject(new TypeError('Arcane request observer must be a function.'));
|
|
490
|
+
}
|
|
463
491
|
if(signal&&(typeof signal.aborted!=='boolean'
|
|
464
492
|
||typeof signal.addEventListener!=='function'
|
|
465
493
|
||typeof signal.removeEventListener!=='function')){
|
|
@@ -483,6 +511,8 @@ export function createCoreClient({
|
|
|
483
511
|
if(!settle(id,failure))report(failure);
|
|
484
512
|
}
|
|
485
513
|
try{
|
|
514
|
+
if(onRequest)Promise.resolve(onRequest({requestId:id})).catch(report);
|
|
515
|
+
if(!pending.has(id))return promise;
|
|
486
516
|
Promise.resolve(selected.send(frame)).catch(sendFailed);
|
|
487
517
|
}catch(error){sendFailed(error);}
|
|
488
518
|
return promise;
|
|
@@ -569,7 +599,7 @@ export function createCoreFacade(client){
|
|
|
569
599
|
mail:{send:request=>invoke('mail.send',request??{},{timeoutMs:450000})},
|
|
570
600
|
speech:{status:()=>invoke('speech.status',{},{timeoutMs:10000}),
|
|
571
601
|
synthesize:(request,options)=>invoke('speech.synthesize',request??{},{timeoutMs:180000,signal:options?.signal}),
|
|
572
|
-
transcribe:(request,options)=>invoke('speech.transcribe',request??{},{timeoutMs:180000,signal:options?.signal})},
|
|
602
|
+
transcribe:(request,options)=>invoke('speech.transcribe',request??{},{timeoutMs:180000,signal:options?.signal,onRequest:options?.onRequest})},
|
|
573
603
|
localAI:{status:()=>invoke('localai.status',{},{timeoutMs:15000}),
|
|
574
604
|
ensurePlatform:()=>invoke('localai.platform.ensure',{},long),recover:request=>invoke('localai.services.recover',request??{},long),
|
|
575
605
|
setParallelRequests:request=>invoke('localai.parallel.requests.set',request??{},long),
|
|
@@ -582,7 +612,7 @@ export function createCoreFacade(client){
|
|
|
582
612
|
return invoke('localai.isolated.question',{...request,operationId},long).finally(unsubscribe);
|
|
583
613
|
}},
|
|
584
614
|
ollama:{version:()=>invoke('ollama.version'),models:()=>invoke('ollama.models'),list:()=>invoke('ollama.models'),
|
|
585
|
-
running:()
|
|
615
|
+
running:function running(options={}){return invoke('ollama.running',{},options);},show:(model,options={})=>invoke('ollama.show',{...options,model}),
|
|
586
616
|
generate:(request,options)=>ollamaInvoke('generate',request,options),chat:(request,options)=>ollamaInvoke('chat',request,options),
|
|
587
617
|
embed:request=>invoke('ollama.embed',request??{}),
|
|
588
618
|
pull:(model,options={},streamOptions)=>ollamaInvoke('pull',{...options,model},streamOptions),
|
|
@@ -639,6 +669,28 @@ export function getInstalledCoreClient(global=globalThis){
|
|
|
639
669
|
return installation?.global===global&&installation.active?client:null;
|
|
640
670
|
}
|
|
641
671
|
|
|
672
|
+
/** Observe this global's exact installed client without installing a transport. */
|
|
673
|
+
export function subscribeCoreClient(listener,{global=globalThis,emitCurrent=true,signal}={}){
|
|
674
|
+
if(typeof listener!=='function')throw new TypeError('The Core client installation listener must be a function.');
|
|
675
|
+
if(signal?.aborted)return function alreadyAborted(){return false;};
|
|
676
|
+
const observation=global[CORE_CLIENT_OBSERVATION]??={source:null,notifications:[],publishing:false};
|
|
677
|
+
observation.source??=arcaneEvents.createSource(observation,{
|
|
678
|
+
source:'core-client-installation',eventTypes:[CORE_CLIENT_INSTALLATION_EVENT]
|
|
679
|
+
});
|
|
680
|
+
// Source subscriptions receive the compatibility view, whose shallow detail
|
|
681
|
+
// preserves these live references; global subscriptions receive snapshots.
|
|
682
|
+
const unsubscribe=observation.source.on(
|
|
683
|
+
CORE_CLIENT_INSTALLATION_EVENT,
|
|
684
|
+
function installedClientChanged(occurrence){listener(occurrence.detail);},
|
|
685
|
+
{signal}
|
|
686
|
+
);
|
|
687
|
+
if(emitCurrent&&!signal?.aborted){
|
|
688
|
+
try{listener({client:getInstalledCoreClient(global),previousClient:null,reason:'current',error:null});}
|
|
689
|
+
catch(error){unsubscribe();throw error;}
|
|
690
|
+
}
|
|
691
|
+
return unsubscribe;
|
|
692
|
+
}
|
|
693
|
+
|
|
642
694
|
/** Installs the same synchronous facade used by classic native-host scripts. */
|
|
643
695
|
export function installCoreClient(global=globalThis,options={}){
|
|
644
696
|
if(global[CORE_CLIENT_KEY])return global[CORE_CLIENT_KEY];
|
|
@@ -661,6 +713,7 @@ export function installCoreClient(global=globalThis,options={}){
|
|
|
661
713
|
const failTransport=client.failTransport;
|
|
662
714
|
let installedTransportFailure=null;
|
|
663
715
|
function releaseInstalledClient(){
|
|
716
|
+
if(!installation.active)return false;
|
|
664
717
|
installation.active=false;
|
|
665
718
|
if(global[CORE_CLIENT_KEY]===client)delete global[CORE_CLIENT_KEY];
|
|
666
719
|
if(global.Arcane===facade){
|
|
@@ -675,21 +728,30 @@ export function installCoreClient(global=globalThis,options={}){
|
|
|
675
728
|
if(previousFailure===undefined)delete global.__arcaneTransportFailed;
|
|
676
729
|
else global.__arcaneTransportFailed=previousFailure;
|
|
677
730
|
}
|
|
731
|
+
return true;
|
|
678
732
|
}
|
|
679
733
|
client.close=function closeInstalledClient(){
|
|
680
|
-
|
|
681
|
-
try{return close();}finally{
|
|
734
|
+
const released=releaseInstalledClient();
|
|
735
|
+
try{return close();}finally{
|
|
736
|
+
if(released)publishCoreClientInstallation(global,client,'closed');
|
|
737
|
+
}
|
|
682
738
|
};
|
|
683
739
|
client.failTransport=function failInstalledTransport(error){
|
|
684
740
|
if(installation.active)installedTransportFailure=error instanceof CoreError?error:new CoreError(error);
|
|
685
|
-
releaseInstalledClient();
|
|
686
|
-
return failTransport(installedTransportFailure??error);
|
|
741
|
+
const released=releaseInstalledClient();
|
|
742
|
+
try{return failTransport(installedTransportFailure??error);}finally{
|
|
743
|
+
if(released)publishCoreClientInstallation(global,client,'transport-failed',installedTransportFailure);
|
|
744
|
+
}
|
|
687
745
|
};
|
|
688
746
|
try{client.connect();}catch(error){
|
|
689
747
|
if(error!==installedTransportFailure&&error.code!=='ARCANE_TRANSPORT_UNAVAILABLE'){
|
|
690
748
|
if(options.onError)options.onError(error);
|
|
691
749
|
else global.console?.error('Arcane Core transport initialization failed.',error);
|
|
692
750
|
}
|
|
751
|
+
}finally{
|
|
752
|
+
if(getInstalledCoreClient(global)===client){
|
|
753
|
+
publishCoreClientInstallation(global,null,'installed');
|
|
754
|
+
}
|
|
693
755
|
}
|
|
694
756
|
return client;
|
|
695
757
|
}
|
|
@@ -0,0 +1,334 @@
|
|
|
1
|
+
# Native Whisper with an Intel NPU encoder
|
|
2
|
+
|
|
3
|
+
Select `encoder: "openvino-npu"` to run Whisper's audio encoder through
|
|
4
|
+
OpenVINO on the Intel NPU. The decoder retains its separate `backend`
|
|
5
|
+
selection. Omitting `encoder` preserves the ordinary native Whisper path.
|
|
6
|
+
|
|
7
|
+
The initial precompiled distribution and producer target Windows x64 team
|
|
8
|
+
computers with Intel AI Boost and a compatible installed Intel NPU driver.
|
|
9
|
+
The selected runtime uses Whisper 1.9.4 and OpenVINO 2026.3.1. Application
|
|
10
|
+
preparation downloads native resources; it does not install a driver,
|
|
11
|
+
compiler, Python environment, or model converter. Other platform distributions
|
|
12
|
+
require their own complete compatible native resources; the Windows asset
|
|
13
|
+
does not establish NPU support on Linux, macOS, or Android.
|
|
14
|
+
|
|
15
|
+
## Declare the application requirement
|
|
16
|
+
|
|
17
|
+
Add the runtime to the application's `arcane-app.json`, retaining its other
|
|
18
|
+
descriptor fields:
|
|
19
|
+
|
|
20
|
+
```json
|
|
21
|
+
{
|
|
22
|
+
"native": {
|
|
23
|
+
"localAI": {
|
|
24
|
+
"runtimes": [
|
|
25
|
+
{
|
|
26
|
+
"id": "whisper.cpp",
|
|
27
|
+
"encoder": "openvino-npu",
|
|
28
|
+
"backend": "auto",
|
|
29
|
+
"models": ["whisper-small"]
|
|
30
|
+
}
|
|
31
|
+
]
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
For a package descriptor, use the equivalent `arcane-package.json` `localAI`
|
|
38
|
+
field. Selecting only the runtime leaves `models` empty. A single prepared
|
|
39
|
+
model becomes the selected `modelId`; with several models, supply a `modelId`
|
|
40
|
+
that names one of them.
|
|
41
|
+
|
|
42
|
+
The default optional asset contains its matching helper, OpenVINO runtime,
|
|
43
|
+
NPU plugin and compiler libraries, and CUDA and CPU decoder libraries. An
|
|
44
|
+
`auto` installation extracts that complete tree once and creates CUDA and CPU
|
|
45
|
+
records referring to it.
|
|
46
|
+
|
|
47
|
+
| `backend` | Decoder selection with `encoder: "openvino-npu"` |
|
|
48
|
+
| --- | --- |
|
|
49
|
+
| `auto` | Try CUDA first, with the prepared CPU decoder available for loading or inference recovery. |
|
|
50
|
+
| `cuda` | Select the CUDA variant without Arcane's automatic CPU retry. |
|
|
51
|
+
| `cpu` | Select the CPU decoder. |
|
|
52
|
+
| `metal` | Unavailable in the default Windows distribution. |
|
|
53
|
+
|
|
54
|
+
Both auto-selected records retain the NPU encoder. A reported
|
|
55
|
+
`WHISPER_ENCODER_UNAVAILABLE` failure ends loading after the helper is drained;
|
|
56
|
+
changing the decoder cannot repair the same failed encoder initialization.
|
|
57
|
+
The helper requests the literal OpenVINO device `NPU` and does not replace it
|
|
58
|
+
with a CPU encoder. Decoder selection is a request; inspect observed metadata
|
|
59
|
+
for native initialization evidence.
|
|
60
|
+
|
|
61
|
+
## Keep the model files together
|
|
62
|
+
|
|
63
|
+
Under this encoder selection, the built-in `whisper-small` descriptor selects
|
|
64
|
+
Intel's multilingual small [paired model archive](https://huggingface.co/Intel/whisper.cpp-openvino-models/blob/main/ggml-small-models.zip):
|
|
65
|
+
|
|
66
|
+
| File | Purpose |
|
|
67
|
+
| --- | --- |
|
|
68
|
+
| `ggml-small.bin` | Whisper model used by the native context. |
|
|
69
|
+
| `ggml-small-encoder-openvino.xml` | OpenVINO encoder description. |
|
|
70
|
+
| `ggml-small-encoder-openvino.bin` | Matching OpenVINO encoder data. |
|
|
71
|
+
|
|
72
|
+
Preparation extracts the three selected files from that archive. Matching
|
|
73
|
+
prepared models can be reused when another model selection changes. The
|
|
74
|
+
ordinary `whisper-small` selection without `encoder` continues to use its
|
|
75
|
+
existing single GGML download.
|
|
76
|
+
|
|
77
|
+
A custom local model supplies all three paths:
|
|
78
|
+
|
|
79
|
+
```js
|
|
80
|
+
const model = {
|
|
81
|
+
id: 'moon-base-small',
|
|
82
|
+
path: '/application-owned/models/ggml-small.bin',
|
|
83
|
+
encoderPath: '/application-owned/models/ggml-small-encoder-openvino.xml',
|
|
84
|
+
encoderDataPath: '/application-owned/models/ggml-small-encoder-openvino.bin'
|
|
85
|
+
};
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Use real paths on the target host. The installer copies the files together,
|
|
89
|
+
preserving their filenames so the XML can locate its matching BIN. A supplied
|
|
90
|
+
`path` selects this local-file route even when the descriptor also retains
|
|
91
|
+
archive metadata. Keep the GGML, XML and BIN from the same selected model
|
|
92
|
+
bundle.
|
|
93
|
+
|
|
94
|
+
An archive descriptor names each selected member:
|
|
95
|
+
|
|
96
|
+
```js
|
|
97
|
+
const model = {
|
|
98
|
+
id: 'whisper-small',
|
|
99
|
+
archiveUrl: 'https://huggingface.co/Intel/whisper.cpp-openvino-models/resolve/main/ggml-small-models.zip',
|
|
100
|
+
filename: 'ggml-small.bin',
|
|
101
|
+
encoderFilename: 'ggml-small-encoder-openvino.xml',
|
|
102
|
+
encoderDataFilename: 'ggml-small-encoder-openvino.bin'
|
|
103
|
+
};
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Put the descriptor in `models`. A standalone GGML `url` does not supply the
|
|
107
|
+
encoder files required by this selection.
|
|
108
|
+
|
|
109
|
+
## Development, native hosts and bundles
|
|
110
|
+
|
|
111
|
+
With the declarative requirement, `npm exec -- arcane dev` prepares the
|
|
112
|
+
application's native speech service alongside page serving. Its runtime
|
|
113
|
+
installation is under `.arcane/local-ai/runtimes`; Whisper operation files
|
|
114
|
+
and the OpenVINO encoder cache belong under `.arcane/speech/whisper`.
|
|
115
|
+
Preparation and model loading proceed independently of the page and other AI
|
|
116
|
+
services. Transcription follows the speech role's actual readiness.
|
|
117
|
+
|
|
118
|
+
A native host can prepare the same requirement explicitly:
|
|
119
|
+
|
|
120
|
+
```js
|
|
121
|
+
import path from 'node:path';
|
|
122
|
+
import {ensureLocalAIRuntimes, bundleLocalAIRuntimes} from 'arcane-os/local-ai';
|
|
123
|
+
import {createWhisperRuntime} from 'arcane-os/local-ai/whisper';
|
|
124
|
+
|
|
125
|
+
const appRoot = process.cwd();
|
|
126
|
+
const directory = path.join(appRoot, '.arcane', 'local-ai', 'runtimes');
|
|
127
|
+
const runtimes = [{
|
|
128
|
+
id: 'whisper.cpp',
|
|
129
|
+
encoder: 'openvino-npu',
|
|
130
|
+
backend: 'auto',
|
|
131
|
+
models: ['whisper-small']
|
|
132
|
+
}];
|
|
133
|
+
|
|
134
|
+
const [runtime] = await ensureLocalAIRuntimes({runtimes, directory});
|
|
135
|
+
const stt = createWhisperRuntime({
|
|
136
|
+
runtime,
|
|
137
|
+
temporaryDirectory: path.join(appRoot, '.arcane', 'speech', 'whisper')
|
|
138
|
+
});
|
|
139
|
+
|
|
140
|
+
await stt.load();
|
|
141
|
+
console.log(stt.current());
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
`ensureLocalAIRuntimes` returns prepared descriptors with absolute paths.
|
|
145
|
+
The host owns the engine's lifetime and calls `await stt.close()` at shutdown.
|
|
146
|
+
Use `subscribe(listener, {replay: true, signal})` for current-state replay and
|
|
147
|
+
later state changes. Preparation and loading accept `signal` and the
|
|
148
|
+
preparation APIs and engine constructor accept `onEvent` for diagnostics.
|
|
149
|
+
|
|
150
|
+
The application's existing native build path is
|
|
151
|
+
`npm exec -- arcane build --target windows-x64`, with the native target declared
|
|
152
|
+
in its descriptor. For an explicitly composed bundle, the same selection can
|
|
153
|
+
be copied into an application-owned output:
|
|
154
|
+
|
|
155
|
+
```js
|
|
156
|
+
const bundle = await bundleLocalAIRuntimes({
|
|
157
|
+
runtimes,
|
|
158
|
+
directory,
|
|
159
|
+
outputRoot: path.join(appRoot, '.arcane', 'native-output'),
|
|
160
|
+
platform: 'win32',
|
|
161
|
+
architecture: 'x64'
|
|
162
|
+
});
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Use a fresh bundle destination. This returns `{runtimes, files}` with paths
|
|
166
|
+
relative to the output root and places Whisper resources under
|
|
167
|
+
`runtime/local-ai/whisper.cpp`. The generated native Core entry resolves the
|
|
168
|
+
helper, runtime libraries, decoder and all three model paths from the relocated
|
|
169
|
+
bundle. Its operation files and `encoder-cache` use the native application's
|
|
170
|
+
state root. Native resources remain beside the application executable in the
|
|
171
|
+
bundle; application startup uses the prepared runtime.
|
|
172
|
+
|
|
173
|
+
## Connect a browser through native speech
|
|
174
|
+
|
|
175
|
+
The browser accesses this capability through its connected native Core speech
|
|
176
|
+
service. Browser JavaScript does not load OpenVINO DLLs or initialize the NPU.
|
|
177
|
+
Development uses the existing [local AI bootstrap](../reference/local-ai.md);
|
|
178
|
+
native packages compose the same speech service in their generated Core entry.
|
|
179
|
+
|
|
180
|
+
With an already connected Core client:
|
|
181
|
+
|
|
182
|
+
```js
|
|
183
|
+
const transcript = await client.speech.transcribe({
|
|
184
|
+
audioBase64,
|
|
185
|
+
mimeType: 'audio/webm',
|
|
186
|
+
model: 'whisper-small'
|
|
187
|
+
}, {signal});
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
The existing `AI.fetchSTT(audioFile, signal)` route remains available. Observe
|
|
191
|
+
`speech.state` and `speech.status` through the
|
|
192
|
+
[native speech contract](../reference/native-speech.md); `roles.stt` and
|
|
193
|
+
`transcriptionAvailable` expose STT readiness independently of TTS.
|
|
194
|
+
|
|
195
|
+
Requests retain the complete recording and existing language and translation
|
|
196
|
+
options. Results retain `{text, language, duration, segments}`, with segment
|
|
197
|
+
`start` and `end` in seconds. Native media decoding, transcript content and
|
|
198
|
+
segment delivery follow the [Whisper contract](../reference/local-whisper.md).
|
|
199
|
+
|
|
200
|
+
## Read initialization evidence and handle failures
|
|
201
|
+
|
|
202
|
+
The engine's `current()` and state subscriptions separate the encoder from the
|
|
203
|
+
decoder:
|
|
204
|
+
|
|
205
|
+
| Field | Meaning |
|
|
206
|
+
| --- | --- |
|
|
207
|
+
| `requestedEncoder` | `NPU` for the selected OpenVINO NPU requirement; otherwise `null`. |
|
|
208
|
+
| `observedEncoder` | `NPU` after the public OpenVINO encoder initialization call succeeds; otherwise `null`. |
|
|
209
|
+
| `encoderEvidence` | `openvino-initialization` after that success; otherwise `null`. |
|
|
210
|
+
| `requestedBackend` | The selected decoder variant, or the configured preference before a helper is selected. |
|
|
211
|
+
| `observedBackend` | Backend observed in native initialization logs, when available. |
|
|
212
|
+
| `backendEvidence` | `runtime-log` when that observation is available. |
|
|
213
|
+
|
|
214
|
+
Before encoder initialization succeeds, observed encoder fields remain
|
|
215
|
+
`null`. The helper emits `ready` only after both its Whisper context and
|
|
216
|
+
selected encoder initialize. This proves initialization for that context;
|
|
217
|
+
it does not prove that a recording has been transcribed, that every operation
|
|
218
|
+
runs on the NPU or GPU, or that either device improves performance.
|
|
219
|
+
|
|
220
|
+
When native encoder initialization returns a nonzero result,
|
|
221
|
+
`WHISPER_ENCODER_UNAVAILABLE` includes that return code in its message. The
|
|
222
|
+
failure retains the joined helper failure in
|
|
223
|
+
`cause` when present, with captured process output in that failure's `details`.
|
|
224
|
+
For NPU `WHISPER_INFERENCE_FAILED`, Arcane shuts down and
|
|
225
|
+
joins the failed helper before returning the error: a successful shutdown
|
|
226
|
+
record is available in `error.data`, including complete `stderr`; a separate
|
|
227
|
+
exit failure is retained in `error.cause`. When multiple attempts fail,
|
|
228
|
+
inspect the complete `AggregateError.errors` as well. `onEvent` can deliver
|
|
229
|
+
diagnostics while work proceeds. Keep these complete records in developer
|
|
230
|
+
diagnostics; ordinary application status can use the concise lifecycle error.
|
|
231
|
+
|
|
232
|
+
With `backend: "auto"`, a recoverable accelerated inference failure can retry
|
|
233
|
+
the complete decoded recording once using the CPU decoder and the same NPU
|
|
234
|
+
encoder selection. The engine reports `recovering` while replacing the
|
|
235
|
+
context, retains the accepted request, and accepts no concurrent transcription.
|
|
236
|
+
|
|
237
|
+
OpenVINO's synchronous encoder call has no per-request cancellation seam in
|
|
238
|
+
Whisper. Cancelling an active NPU inference therefore retires the owned
|
|
239
|
+
helper process and waits for its exit before removing recording files. The
|
|
240
|
+
model becomes unloaded. After cancellation settles, call `await stt.load()`
|
|
241
|
+
with a fresh, unaborted signal before the next transcription. `unload()` also
|
|
242
|
+
cancels active work and drains the helper; `close()` ends the engine's lifetime.
|
|
243
|
+
Cancellation before native inference need not unload an already ready model;
|
|
244
|
+
follow the resulting authoritative state.
|
|
245
|
+
|
|
246
|
+
## Supply a complete custom runtime
|
|
247
|
+
|
|
248
|
+
For an existing compatible build, select a complete variant rather than
|
|
249
|
+
individual DLLs:
|
|
250
|
+
|
|
251
|
+
```js
|
|
252
|
+
const requirement = {
|
|
253
|
+
id: 'whisper.cpp',
|
|
254
|
+
encoder: 'openvino-npu',
|
|
255
|
+
backend: 'cpu',
|
|
256
|
+
models: ['whisper-small'],
|
|
257
|
+
variants: [{
|
|
258
|
+
backend: 'cpu',
|
|
259
|
+
encoder: 'openvino-npu',
|
|
260
|
+
root: nativeRuntimeDirectory,
|
|
261
|
+
libraryDirectory: path.join(nativeRuntimeDirectory, 'bin')
|
|
262
|
+
}]
|
|
263
|
+
};
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
Each variant supplies a local `root` or an archive `url`. A supplied
|
|
267
|
+
`libraryDirectory` is relative to or inside that selected tree; otherwise it
|
|
268
|
+
is found from the native Whisper library. Each optional complete tree contains
|
|
269
|
+
its own matching `arcane-whisper.exe` beside the runtime DLLs. Preparation
|
|
270
|
+
retains that variant's executable, so multiple custom builds keep their
|
|
271
|
+
matching helpers. Include all selected OpenVINO, NPU plugin/compiler, TBB,
|
|
272
|
+
Whisper, ggml and compiler-runtime libraries, plus the CUDA libraries when
|
|
273
|
+
selecting CUDA. A CPU variant still needs the OpenVINO NPU resources.
|
|
274
|
+
|
|
275
|
+
For custom automatic decoder recovery, declare both CUDA and CPU variants
|
|
276
|
+
with `encoder: "openvino-npu"`. Custom variants are prepared as explicitly
|
|
277
|
+
supplied trees. Library search configuration belongs to the launched helper
|
|
278
|
+
process and does not change the machine's global `PATH`. The general Whisper
|
|
279
|
+
helper and FFmpeg decoder overrides remain available through the existing
|
|
280
|
+
[runtime descriptor](../reference/local-whisper.md).
|
|
281
|
+
|
|
282
|
+
## Produce the optional native runtime
|
|
283
|
+
|
|
284
|
+
Runtime producers can use the public
|
|
285
|
+
`arcane-os/local-ai/whisper/openvino-build` export with the selected installed
|
|
286
|
+
Whisper 1.9.4 source, OpenVINO 2026.3.1 Windows development tree and an existing
|
|
287
|
+
compatible CMake/MSVC environment. `cudaDirectory`, when supplied, names the
|
|
288
|
+
matching upstream b5130 CUDA distribution's library directory. It supplies
|
|
289
|
+
the precompiled CUDA backend; omitting it produces the NPU-plus-CPU runtime.
|
|
290
|
+
|
|
291
|
+
```js
|
|
292
|
+
import {cp} from 'node:fs/promises';
|
|
293
|
+
import {buildWhisperOpenVinoRuntime} from 'arcane-os/local-ai/whisper/openvino-build';
|
|
294
|
+
import {buildWhisperHelper} from 'arcane-os/local-ai/whisper/build';
|
|
295
|
+
|
|
296
|
+
const nativeRuntime = await buildWhisperOpenVinoRuntime({
|
|
297
|
+
sourceDirectory,
|
|
298
|
+
openvinoDirectory,
|
|
299
|
+
cudaDirectory,
|
|
300
|
+
outputRoot: path.join(appRoot, '.arcane', 'native-build', 'openvino'),
|
|
301
|
+
env,
|
|
302
|
+
signal,
|
|
303
|
+
onEvent
|
|
304
|
+
});
|
|
305
|
+
|
|
306
|
+
const helper = await buildWhisperHelper({
|
|
307
|
+
runtime: nativeRuntime,
|
|
308
|
+
outputRoot: path.join(appRoot, '.arcane', 'native-build', 'helper'),
|
|
309
|
+
env,
|
|
310
|
+
signal,
|
|
311
|
+
onEvent
|
|
312
|
+
});
|
|
313
|
+
|
|
314
|
+
await cp(path.join(helper.root, 'bin'), nativeRuntime.libraryDirectory, {
|
|
315
|
+
recursive: true
|
|
316
|
+
});
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
The variables for source directories, build environment, cancellation and
|
|
320
|
+
diagnostics belong to the producer. Both functions also accept `cmake`,
|
|
321
|
+
`generator` and `cmakeArgs` for that existing build environment.
|
|
322
|
+
|
|
323
|
+
The runtime producer returns `root`, `directory`, `libraryDirectory`,
|
|
324
|
+
`sourceDirectory`, selected `backends` and complete build `diagnostics`. Its
|
|
325
|
+
output keeps the copied source and build directory separate from the runtime
|
|
326
|
+
tree. It applies the encoder return-value propagation and exception-to-stderr
|
|
327
|
+
corrections in that copied source. The helper build uses the matching headers
|
|
328
|
+
and runtime and installs its required release compiler-runtime libraries;
|
|
329
|
+
copying its complete installed `bin` directory places them with the optional
|
|
330
|
+
runtime. Use `nativeRuntime.root` as the custom variant root and retain its
|
|
331
|
+
packaging notices.
|
|
332
|
+
|
|
333
|
+
This producer path builds a native distribution. Applications using the
|
|
334
|
+
precompiled asset follow the preparation and bundling paths above.
|
package/docs/reference/README.md
CHANGED
|
@@ -56,11 +56,16 @@ alone does not make bare module names resolve in a browser.
|
|
|
56
56
|
| Compose native Core dispatch, service lifecycle, and stdio hosting | [Native Core runtime](core-runtime.md) |
|
|
57
57
|
| Run application-selected llama.cpp, Ollama or ONNX through Core | [Local AI through Core](local-ai.md) |
|
|
58
58
|
| Generate complete local PNG images with a retained native model | [Local image generation](local-image-generation.md) |
|
|
59
|
+
| Select FLUX.2 Klein 4B and its three complementary model resources | [FLUX.2 Klein 4B through Core](local-image-flux.md) |
|
|
60
|
+
| Select the complete SDXL Base 1.0 checkpoint for text-to-image generation | [SDXL Base 1.0 through Core](local-image-sdxl.md) |
|
|
61
|
+
| Compose independently loaded transcription and synthesis engines | [Native speech service](native-speech.md) |
|
|
62
|
+
| Keep a native Whisper model loaded across complete recordings | [Native Whisper transcription](local-whisper.md) |
|
|
59
63
|
| Edit an original PNG with a complete prompt and explicit strength | [Local image editing](local-image-editing.md) |
|
|
60
64
|
| Project complete stored model files into native working files | [Model assets through Core](model-assets.md) |
|
|
61
65
|
| Retain a Nemotron model and diarize caller-fed audio streams | [Native speaker diarization](diarization.md) |
|
|
62
66
|
| Persist application-selected preferences through Core | [Core preferences service](core-preferences.md) |
|
|
63
67
|
| Read complete selected Git text without changing a checkout | [Git text snapshots](git-text-snapshot.md) |
|
|
68
|
+
| Own a native working checkout under per-user ArcaneData/Repos | [Native repository workspaces](core-repositories.md) |
|
|
64
69
|
| Build an installed-SDK portable Core payload | [Portable Core packaging](core-native-packaging.md) |
|
|
65
70
|
| Pass host-selected state defaults and unchanged explicit launch values to services | [Launch-time locations](core-native-packaging.md#launch-time-locations) |
|
|
66
71
|
| Build a Windows executable without an OS checkout | [Windows executable packaging](core-native-packaging.md#windows-executable) |
|
|
@@ -225,6 +230,7 @@ Ollama, Wllama, Whisper, Kokoro, native, or cloud transport:
|
|
|
225
230
|
| Use Core-normalized chat | [`globalThis.Arcane.ai`](core/arcane-ai-contracts.md) | Native/Core only when separately admitted. |
|
|
226
231
|
| Run a caller-selected GGUF LLM locally | [`arcane-os/ai/browser-wasm`](ai/browser-wasm.md) | Browser secure context with WebAssembly and OPFS/DBOPFS; explicit CPU selection or WebGPU with full offload and no automatic CPU fallback. |
|
|
227
232
|
| Run application-selected native llama.cpp/Ollama chat | [`arcane-os/ai/core-local`, `arcane-os/core/local-ai`](local-ai.md) | Explicit Core connection and selected native runtimes; installation, availability and loaded-model readiness remain distinct. |
|
|
233
|
+
| Transcribe complete recordings with a retained native Whisper model | [`arcane-os/local-ai/whisper`](local-whisper.md) and [Core speech](native-speech.md) | Matching native helper, Whisper libraries, decoder and selected model; Windows defaults and explicit other-platform distributions have distinct preparation requirements. |
|
|
228
234
|
| Run a local ONNX graph with named tensors | [`arcane-os/local-ai/onnx`, `arcane-os/ai/core-onnx`](local-ai.md#native-onnx-sessions) | Retained native workers; browser access requires an available Core ONNX service. |
|
|
229
235
|
| Generate local images from complete prompts and selected model resources | [`arcane-os/local-ai/image`, `arcane-os/ai/core-image`](local-image-generation.md) | Prepared native image runtime; browser access requires its Core service. CPU Windows/Linux and Mac Metal/CPU paths have distinct actual host requirements. |
|
|
230
236
|
| Supply complete stored model members to a native engine | [`arcane-os/ai/core-model-assets`, `arcane-os/core/model-assets`](model-assets.md) | Available Core service; DBOPFS originals stay authoritative while native owners retain working files. |
|
|
@@ -120,7 +120,7 @@ non-writable `globalThis.arcaneOllama` convenience and emits
|
|
|
120
120
|
| `version()` | `Arcane.ollama.version()` | Raw service version diagnostic. | [version](core/reference/arcane-api/ai-and-ollama.md#arcaneollamaversion) |
|
|
121
121
|
| `models()` | `Arcane.ollama.models()` | Raw installed-model diagnostic. | [models](core/reference/arcane-api/ai-and-ollama.md#arcaneollamamodels) |
|
|
122
122
|
| `list()` | Calls `Arcane.ollama.models()` | Module alias for `models()`; it does not call the bridge's separate `list` alias. | [list](core/reference/arcane-api/ai-and-ollama.md#arcaneollamalist) |
|
|
123
|
-
| `running()` | `Arcane.ollama.running()` | Raw resident-model diagnostic. | [running](core/reference/arcane-api/ai-and-ollama.md#arcaneollamarunning) |
|
|
123
|
+
| `running(options?)` | `Arcane.ollama.running(options?)` | Raw resident-model diagnostic; optional `{signal, timeoutMs, onRequest}` uses the Core request lifetime. No-argument calls remain supported. | [running](core/reference/arcane-api/ai-and-ollama.md#arcaneollamarunning) |
|
|
124
124
|
| `show(model, options)` | `Arcane.ollama.show(...)` | Raw bounded model metadata. | [show](core/reference/arcane-api/ai-and-ollama.md#arcaneollamashow) |
|
|
125
125
|
| `generate(request, options)` | `Arcane.ollama.generate(...)` | Admitted generation; optional chunk callback/signal/timeout. | [generate](core/reference/arcane-api/ai-and-ollama.md#arcaneollamagenerate) |
|
|
126
126
|
| `chat(request, options)` | `Arcane.ollama.chat(...)` | Admitted chat/tools; optional chunk callback/signal/timeout. | [chat](core/reference/arcane-api/ai-and-ollama.md#arcaneollamachat) |
|