arcane-os 0.64.0 → 0.66.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +50 -0
- package/README.md +1 -1
- package/browser-runtime/ai/browser-decisions.mjs +3 -2
- package/browser-runtime/ai/decision-runtime.mjs +14 -24
- package/browser-runtime/core/client.mjs +62 -5
- package/docs/guides/native-whisper-openvino.md +334 -0
- package/docs/publishing.md +39 -0
- package/docs/reference/README.md +11 -1
- package/docs/reference/ai/browser-decisions.md +7 -4
- package/docs/reference/arcane-ollama.md +1 -1
- package/docs/reference/codex-app-server.md +269 -0
- package/docs/reference/core-client.md +60 -2
- package/docs/reference/core-development.md +157 -0
- package/docs/reference/core-native-packaging.md +13 -0
- package/docs/reference/core-repositories.md +232 -4
- package/docs/reference/git-text-snapshot.md +15 -1
- package/docs/reference/inventory/package-api.json +174 -6
- package/docs/reference/inventory/runtime-modules.json +8 -6
- package/docs/reference/local-ai.md +80 -3
- package/docs/reference/local-image-generation.md +5 -0
- package/docs/reference/local-image-sdxl.md +1 -1
- package/docs/reference/local-whisper.md +7 -1
- package/docs/reference/mcp-stdio.md +263 -0
- package/docs/reference/native-speech.md +6 -0
- package/docs/reference/runtime-components.md +18 -1
- package/docs/reference/runtime-modules.md +10 -2
- package/docs/reference/sdk-api.md +346 -4
- package/package.json +6 -2
- package/runtime/arcane/modules/AI.js +156 -13
- package/runtime/arcane/modules/Ollama.js +1 -1
- package/runtime/arcane/modules/OllamaModelIdentifier.js +13 -0
- package/schemas/arcane-app.schema.json +8 -1
- package/schemas/arcane-package.schema.json +8 -1
- package/src/codex/app-server.mjs +845 -0
- package/src/core/development.mjs +457 -0
- package/src/core/repositories.mjs +178 -9
- package/src/core/services/local-ai.mjs +80 -10
- package/src/dev-server.mjs +3 -2
- package/src/git-identity.mjs +132 -0
- package/src/git-text-snapshot.mjs +5 -3
- package/src/index.mjs +1 -0
- package/src/local-ai/dev.mjs +80 -426
- package/src/local-ai/whisper/config.mjs +31 -3
- package/src/local-ai/whisper/index.mjs +62 -5
- package/src/local-ai/whisper/install.mjs +72 -14
- package/src/local-ai/whisper/native/main.cpp +33 -2
- package/src/local-ai/whisper/openvino-build.mjs +138 -0
- package/src/mcp/stdio-server.mjs +579 -0
- package/src/native/portable-layout.mjs +3 -1
- package/src/toolchain.mjs +19 -8
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,55 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.66.0
|
|
4
|
+
|
|
5
|
+
- Add `readGitIdentity` from `arcane-os/core/repositories` for separate global,
|
|
6
|
+
repository-local and effective Git configuration observations. Preserve full
|
|
7
|
+
name/email/githubUser values, empty versus unset fields, configured includes,
|
|
8
|
+
joined cancellation and complete process/observer failures.
|
|
9
|
+
- Add optional construction-time `gitIdentity: {name, email, username}` to
|
|
10
|
+
repository workspaces and bare text snapshots. Explicit author fields apply
|
|
11
|
+
only to child Git configuration/environment; omitted fields retain ordinary
|
|
12
|
+
inheritance. Username is a credential hint, not authenticated-account proof.
|
|
13
|
+
- Preserve complete selected text, existing clone/pull/commit/push behavior,
|
|
14
|
+
ordered workspace operations and shared snapshot acquisition. No Git config
|
|
15
|
+
file, remote, credentials, application preferences or data is rewritten.
|
|
16
|
+
- Preserve original process errors when an observer failure is already included,
|
|
17
|
+
and report non-UTF-8 Git identity output with its complete raw diagnostics
|
|
18
|
+
instead of returning substituted identity text.
|
|
19
|
+
- Honor an explicit browser decision `dtype` while preserving the existing
|
|
20
|
+
Laya FP16 and Julia FP32 defaults. Share the complete tensor-record preparation
|
|
21
|
+
with the separately developing native owner; this does not deliver native Laya.
|
|
22
|
+
- Keep the dependency tree unchanged. Native Laya and target-bound repository
|
|
23
|
+
publication remain separate work. Retained compatible native assets continue
|
|
24
|
+
to use matching numeric-release routes; optional NPU upload follow-through
|
|
25
|
+
remains separately tracked.
|
|
26
|
+
|
|
27
|
+
## 0.65.0
|
|
28
|
+
|
|
29
|
+
- Add optional precompiled Windows x64 Intel NPU encoding for native Whisper
|
|
30
|
+
with `encoder: "openvino-npu"`, paired small-model resources, separate
|
|
31
|
+
encoder/decoder state, and existing GPU-first decoding with CPU recovery.
|
|
32
|
+
Preserve ordinary Whisper, complete transcripts and joined cancellation.
|
|
33
|
+
- Add `createRepositoryWorkspace(...).write({files, message, signal})` for
|
|
34
|
+
complete selected text, literal-path staging/commit and ordinary push, with
|
|
35
|
+
complete diagnostics and explicit confirmed or uncertain partial outcomes.
|
|
36
|
+
- Add retained native Codex App Server sessions and an explicit MCP STDIO
|
|
37
|
+
server through `arcane-os/codex/app-server` and `arcane-os/mcp/stdio`.
|
|
38
|
+
Preserve complete native payloads, origin-bound server-request replies,
|
|
39
|
+
ordered live results, cooperative cancellation and owned write settlement.
|
|
40
|
+
- Add application Core composition during source development without requiring
|
|
41
|
+
local AI, `/arcane-core.js`, and replayed client installation/retirement
|
|
42
|
+
observation. Preserve the existing local-AI bootstrap, early-close cleanup
|
|
43
|
+
and ordered reentrant installation notifications.
|
|
44
|
+
- Tie Ollama readiness to actual selected-model residency observations, with
|
|
45
|
+
ordinary model-name alias comparison, operation cancellation and stale
|
|
46
|
+
observation/engine ownership. Preserve original outbound model names and
|
|
47
|
+
complete payloads; no background polling or atomic residency claim.
|
|
48
|
+
- Keep the npm dependency tree unchanged. Native runtime producers and
|
|
49
|
+
consumers use the documented upstream resources and compatible helper
|
|
50
|
+
assets attached to this numeric release. Git identity selection and native
|
|
51
|
+
Laya fp32 integration remain separate subsequent work.
|
|
52
|
+
|
|
3
53
|
## 0.64.0
|
|
4
54
|
|
|
5
55
|
- Add retained native Whisper transcription through the existing Core speech
|
package/README.md
CHANGED
|
@@ -20,7 +20,7 @@ selected SDK projection or an explicit live SDK source mount. Both profiles
|
|
|
20
20
|
share the theme, packaging, event, cancellation,
|
|
21
21
|
and browser run contracts while retaining their selected application layout.
|
|
22
22
|
|
|
23
|
-
This checkout defines the `0.
|
|
23
|
+
This checkout defines the `0.66.0` SDK contract. Development applications declare
|
|
24
24
|
`arcane-os` as `latest` and retain their lockfile. Exact SDK pins begin when Roshi
|
|
25
25
|
designates a production build. Follow [Consumer updates](docs/consumer-updates.md)
|
|
26
26
|
to refresh and compare the installed SDK and lockfile with the published channel;
|
|
@@ -74,7 +74,8 @@ class BrowserDecisionModel {
|
|
|
74
74
|
#error = null;
|
|
75
75
|
#disposed = false;
|
|
76
76
|
|
|
77
|
-
constructor({family, model, revision = 'main', device = 'webgpu',
|
|
77
|
+
constructor({family, model, revision = 'main', device = 'webgpu',
|
|
78
|
+
dtype = family === 'laya' ? 'fp16' : 'fp32', runtime = {}} = {}) {
|
|
78
79
|
if (family !== 'laya' && family !== 'julia') {
|
|
79
80
|
throw new TypeError('family must be "laya" or "julia".');
|
|
80
81
|
}
|
|
@@ -89,7 +90,7 @@ class BrowserDecisionModel {
|
|
|
89
90
|
model: requiredText(model, 'model'),
|
|
90
91
|
revision: requiredText(revision, 'revision'),
|
|
91
92
|
device: requiredText(device, 'device'),
|
|
92
|
-
dtype:
|
|
93
|
+
dtype: requiredText(dtype, 'dtype'),
|
|
93
94
|
runtime: {...runtime, moduleUrl: requiredText(moduleUrl, 'runtime.moduleUrl')}
|
|
94
95
|
};
|
|
95
96
|
this.#events = createArcaneEventSource(
|
|
@@ -82,7 +82,8 @@ export function encodeDecisionRows(tokenizer, rows, family) {
|
|
|
82
82
|
return {encoded, pad};
|
|
83
83
|
}
|
|
84
84
|
|
|
85
|
-
|
|
85
|
+
/** Plain tensor records shared by browser and native ONNX execution owners. */
|
|
86
|
+
export function createDecisionInputs(encoded, pad) {
|
|
86
87
|
let sequenceLength = 0;
|
|
87
88
|
let optionCount = 0;
|
|
88
89
|
for (const row of encoded) {
|
|
@@ -108,26 +109,11 @@ function decisionInputs(Tensor, encoded, pad) {
|
|
|
108
109
|
types[rowIndex] = BigInt(row.qtype);
|
|
109
110
|
}
|
|
110
111
|
return {
|
|
111
|
-
input_ids:
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
'int64', attention,
|
|
117
|
-
[batch, sequenceLength]
|
|
118
|
-
),
|
|
119
|
-
marker_pos: new Tensor(
|
|
120
|
-
'int64', positions,
|
|
121
|
-
[batch, optionCount]
|
|
122
|
-
),
|
|
123
|
-
marker_mask: new Tensor(
|
|
124
|
-
'bool', mask,
|
|
125
|
-
[batch, optionCount]
|
|
126
|
-
),
|
|
127
|
-
qtype: new Tensor(
|
|
128
|
-
'int64', types,
|
|
129
|
-
[batch]
|
|
130
|
-
)
|
|
112
|
+
input_ids: {type: 'int64', data: ids, dims: [batch, sequenceLength]},
|
|
113
|
+
attention_mask: {type: 'int64', data: attention, dims: [batch, sequenceLength]},
|
|
114
|
+
marker_pos: {type: 'int64', data: positions, dims: [batch, optionCount]},
|
|
115
|
+
marker_mask: {type: 'bool', data: mask, dims: [batch, optionCount]},
|
|
116
|
+
qtype: {type: 'int64', data: types, dims: [batch]}
|
|
131
117
|
};
|
|
132
118
|
}
|
|
133
119
|
|
|
@@ -210,7 +196,8 @@ export function decodeDecisionOutputs(rows, outputs) {
|
|
|
210
196
|
}
|
|
211
197
|
|
|
212
198
|
export async function loadDecisionRuntime(configuration, report) {
|
|
213
|
-
const {family, model: repository, revision = 'main', device = 'webgpu',
|
|
199
|
+
const {family, model: repository, revision = 'main', device = 'webgpu',
|
|
200
|
+
dtype = family === 'laya' ? 'fp16' : 'fp32', runtime} = configuration;
|
|
214
201
|
report(
|
|
215
202
|
{phase: 'loading-runtime'}
|
|
216
203
|
);
|
|
@@ -219,7 +206,7 @@ export async function loadDecisionRuntime(configuration, report) {
|
|
|
219
206
|
namespace.env.backends.onnx.wasm.wasmPaths = runtime.wasmPaths;
|
|
220
207
|
}
|
|
221
208
|
const {AutoTokenizer, AutoModel, PreTrainedModel, Tensor} = namespace;
|
|
222
|
-
const options = {revision, device, dtype
|
|
209
|
+
const options = {revision, device, dtype};
|
|
223
210
|
// No progress_callback: upstream 4.3 otherwise performs file-size
|
|
224
211
|
// aggregation. This owner reports actual semantic lifecycle transitions.
|
|
225
212
|
report(
|
|
@@ -261,7 +248,10 @@ export async function loadDecisionRuntime(configuration, report) {
|
|
|
261
248
|
async function evaluate(rows) {
|
|
262
249
|
const {encoded, pad} = encodeDecisionRows(tokenizer, rows, family);
|
|
263
250
|
if (encoded.length === 0) return {decisions: [], outputs: {}};
|
|
264
|
-
const inputs =
|
|
251
|
+
const inputs = {};
|
|
252
|
+
for (const [name, record] of Object.entries(createDecisionInputs(encoded, pad))) {
|
|
253
|
+
inputs[name] = new Tensor(record.type, record.data, record.dims);
|
|
254
|
+
}
|
|
265
255
|
let output;
|
|
266
256
|
try {
|
|
267
257
|
output = await model(inputs);
|
|
@@ -3,9 +3,34 @@ import {CORE_PROTOCOL,CORE_READY_EVENTS,CoreError,serializeCoreError} from './co
|
|
|
3
3
|
|
|
4
4
|
const CORE_CLIENT_KEY=Symbol.for('arcane-os.core.client');
|
|
5
5
|
const CORE_CLIENT_INSTALLATION=Symbol.for('arcane-os.core.client.installation');
|
|
6
|
+
const CORE_CLIENT_OBSERVATION=Symbol.for('arcane-os.core.client.observation');
|
|
7
|
+
const CORE_CLIENT_INSTALLATION_EVENT='core.client.installation';
|
|
6
8
|
const CORE_EVENT='core.rpc.event';
|
|
7
9
|
const LONG_OPERATION_TIMEOUT=50*60*1000;
|
|
8
10
|
|
|
11
|
+
function publishCoreClientInstallation(global,previousClient,reason,error=null){
|
|
12
|
+
// A classic document-created installation may precede the ESM event owner.
|
|
13
|
+
// The live installation already owns replay; retain no retired-client record.
|
|
14
|
+
const observation=global[CORE_CLIENT_OBSERVATION];
|
|
15
|
+
if(!observation?.source)return;
|
|
16
|
+
observation.notifications.push({client:getInstalledCoreClient(global),previousClient,reason,error});
|
|
17
|
+
if(observation.publishing)return;
|
|
18
|
+
observation.publishing=true;
|
|
19
|
+
let delivered=0;
|
|
20
|
+
try{
|
|
21
|
+
// A listener may change the installation. Finish notifying this
|
|
22
|
+
// transition's observers before publishing that successor transition.
|
|
23
|
+
while(delivered<observation.notifications.length){
|
|
24
|
+
const notification=observation.notifications[delivered];
|
|
25
|
+
observation.notifications[delivered++]=null;
|
|
26
|
+
observation.source.dispatch(CORE_CLIENT_INSTALLATION_EVENT,notification);
|
|
27
|
+
}
|
|
28
|
+
}finally{
|
|
29
|
+
observation.notifications.splice(0,delivered);
|
|
30
|
+
observation.publishing=false;
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
|
|
9
34
|
/** One RPC client. Native execution and service policy belong to the host. */
|
|
10
35
|
export function createCoreClient({
|
|
11
36
|
global=globalThis,
|
|
@@ -587,7 +612,7 @@ export function createCoreFacade(client){
|
|
|
587
612
|
return invoke('localai.isolated.question',{...request,operationId},long).finally(unsubscribe);
|
|
588
613
|
}},
|
|
589
614
|
ollama:{version:()=>invoke('ollama.version'),models:()=>invoke('ollama.models'),list:()=>invoke('ollama.models'),
|
|
590
|
-
running:()
|
|
615
|
+
running:function running(options={}){return invoke('ollama.running',{},options);},show:(model,options={})=>invoke('ollama.show',{...options,model}),
|
|
591
616
|
generate:(request,options)=>ollamaInvoke('generate',request,options),chat:(request,options)=>ollamaInvoke('chat',request,options),
|
|
592
617
|
embed:request=>invoke('ollama.embed',request??{}),
|
|
593
618
|
pull:(model,options={},streamOptions)=>ollamaInvoke('pull',{...options,model},streamOptions),
|
|
@@ -644,6 +669,28 @@ export function getInstalledCoreClient(global=globalThis){
|
|
|
644
669
|
return installation?.global===global&&installation.active?client:null;
|
|
645
670
|
}
|
|
646
671
|
|
|
672
|
+
/** Observe this global's exact installed client without installing a transport. */
|
|
673
|
+
export function subscribeCoreClient(listener,{global=globalThis,emitCurrent=true,signal}={}){
|
|
674
|
+
if(typeof listener!=='function')throw new TypeError('The Core client installation listener must be a function.');
|
|
675
|
+
if(signal?.aborted)return function alreadyAborted(){return false;};
|
|
676
|
+
const observation=global[CORE_CLIENT_OBSERVATION]??={source:null,notifications:[],publishing:false};
|
|
677
|
+
observation.source??=arcaneEvents.createSource(observation,{
|
|
678
|
+
source:'core-client-installation',eventTypes:[CORE_CLIENT_INSTALLATION_EVENT]
|
|
679
|
+
});
|
|
680
|
+
// Source subscriptions receive the compatibility view, whose shallow detail
|
|
681
|
+
// preserves these live references; global subscriptions receive snapshots.
|
|
682
|
+
const unsubscribe=observation.source.on(
|
|
683
|
+
CORE_CLIENT_INSTALLATION_EVENT,
|
|
684
|
+
function installedClientChanged(occurrence){listener(occurrence.detail);},
|
|
685
|
+
{signal}
|
|
686
|
+
);
|
|
687
|
+
if(emitCurrent&&!signal?.aborted){
|
|
688
|
+
try{listener({client:getInstalledCoreClient(global),previousClient:null,reason:'current',error:null});}
|
|
689
|
+
catch(error){unsubscribe();throw error;}
|
|
690
|
+
}
|
|
691
|
+
return unsubscribe;
|
|
692
|
+
}
|
|
693
|
+
|
|
647
694
|
/** Installs the same synchronous facade used by classic native-host scripts. */
|
|
648
695
|
export function installCoreClient(global=globalThis,options={}){
|
|
649
696
|
if(global[CORE_CLIENT_KEY])return global[CORE_CLIENT_KEY];
|
|
@@ -666,6 +713,7 @@ export function installCoreClient(global=globalThis,options={}){
|
|
|
666
713
|
const failTransport=client.failTransport;
|
|
667
714
|
let installedTransportFailure=null;
|
|
668
715
|
function releaseInstalledClient(){
|
|
716
|
+
if(!installation.active)return false;
|
|
669
717
|
installation.active=false;
|
|
670
718
|
if(global[CORE_CLIENT_KEY]===client)delete global[CORE_CLIENT_KEY];
|
|
671
719
|
if(global.Arcane===facade){
|
|
@@ -680,21 +728,30 @@ export function installCoreClient(global=globalThis,options={}){
|
|
|
680
728
|
if(previousFailure===undefined)delete global.__arcaneTransportFailed;
|
|
681
729
|
else global.__arcaneTransportFailed=previousFailure;
|
|
682
730
|
}
|
|
731
|
+
return true;
|
|
683
732
|
}
|
|
684
733
|
client.close=function closeInstalledClient(){
|
|
685
|
-
|
|
686
|
-
try{return close();}finally{
|
|
734
|
+
const released=releaseInstalledClient();
|
|
735
|
+
try{return close();}finally{
|
|
736
|
+
if(released)publishCoreClientInstallation(global,client,'closed');
|
|
737
|
+
}
|
|
687
738
|
};
|
|
688
739
|
client.failTransport=function failInstalledTransport(error){
|
|
689
740
|
if(installation.active)installedTransportFailure=error instanceof CoreError?error:new CoreError(error);
|
|
690
|
-
releaseInstalledClient();
|
|
691
|
-
return failTransport(installedTransportFailure??error);
|
|
741
|
+
const released=releaseInstalledClient();
|
|
742
|
+
try{return failTransport(installedTransportFailure??error);}finally{
|
|
743
|
+
if(released)publishCoreClientInstallation(global,client,'transport-failed',installedTransportFailure);
|
|
744
|
+
}
|
|
692
745
|
};
|
|
693
746
|
try{client.connect();}catch(error){
|
|
694
747
|
if(error!==installedTransportFailure&&error.code!=='ARCANE_TRANSPORT_UNAVAILABLE'){
|
|
695
748
|
if(options.onError)options.onError(error);
|
|
696
749
|
else global.console?.error('Arcane Core transport initialization failed.',error);
|
|
697
750
|
}
|
|
751
|
+
}finally{
|
|
752
|
+
if(getInstalledCoreClient(global)===client){
|
|
753
|
+
publishCoreClientInstallation(global,null,'installed');
|
|
754
|
+
}
|
|
698
755
|
}
|
|
699
756
|
return client;
|
|
700
757
|
}
|
|
@@ -0,0 +1,334 @@
|
|
|
1
|
+
# Native Whisper with an Intel NPU encoder
|
|
2
|
+
|
|
3
|
+
Select `encoder: "openvino-npu"` to run Whisper's audio encoder through
|
|
4
|
+
OpenVINO on the Intel NPU. The decoder retains its separate `backend`
|
|
5
|
+
selection. Omitting `encoder` preserves the ordinary native Whisper path.
|
|
6
|
+
|
|
7
|
+
The initial precompiled distribution and producer target Windows x64 team
|
|
8
|
+
computers with Intel AI Boost and a compatible installed Intel NPU driver.
|
|
9
|
+
The selected runtime uses Whisper 1.9.4 and OpenVINO 2026.3.1. Application
|
|
10
|
+
preparation downloads native resources; it does not install a driver,
|
|
11
|
+
compiler, Python environment, or model converter. Other platform distributions
|
|
12
|
+
require their own complete compatible native resources; the Windows asset
|
|
13
|
+
does not establish NPU support on Linux, macOS, or Android.
|
|
14
|
+
|
|
15
|
+
## Declare the application requirement
|
|
16
|
+
|
|
17
|
+
Add the runtime to the application's `arcane-app.json`, retaining its other
|
|
18
|
+
descriptor fields:
|
|
19
|
+
|
|
20
|
+
```json
|
|
21
|
+
{
|
|
22
|
+
"native": {
|
|
23
|
+
"localAI": {
|
|
24
|
+
"runtimes": [
|
|
25
|
+
{
|
|
26
|
+
"id": "whisper.cpp",
|
|
27
|
+
"encoder": "openvino-npu",
|
|
28
|
+
"backend": "auto",
|
|
29
|
+
"models": ["whisper-small"]
|
|
30
|
+
}
|
|
31
|
+
]
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
For a package descriptor, use the equivalent `arcane-package.json` `localAI`
|
|
38
|
+
field. Selecting only the runtime leaves `models` empty. A single prepared
|
|
39
|
+
model becomes the selected `modelId`; with several models, supply a `modelId`
|
|
40
|
+
that names one of them.
|
|
41
|
+
|
|
42
|
+
The default optional asset contains its matching helper, OpenVINO runtime,
|
|
43
|
+
NPU plugin and compiler libraries, and CUDA and CPU decoder libraries. An
|
|
44
|
+
`auto` installation extracts that complete tree once and creates CUDA and CPU
|
|
45
|
+
records referring to it.
|
|
46
|
+
|
|
47
|
+
| `backend` | Decoder selection with `encoder: "openvino-npu"` |
|
|
48
|
+
| --- | --- |
|
|
49
|
+
| `auto` | Try CUDA first, with the prepared CPU decoder available for loading or inference recovery. |
|
|
50
|
+
| `cuda` | Select the CUDA variant without Arcane's automatic CPU retry. |
|
|
51
|
+
| `cpu` | Select the CPU decoder. |
|
|
52
|
+
| `metal` | Unavailable in the default Windows distribution. |
|
|
53
|
+
|
|
54
|
+
Both auto-selected records retain the NPU encoder. A reported
|
|
55
|
+
`WHISPER_ENCODER_UNAVAILABLE` failure ends loading after the helper is drained;
|
|
56
|
+
changing the decoder cannot repair the same failed encoder initialization.
|
|
57
|
+
The helper requests the literal OpenVINO device `NPU` and does not replace it
|
|
58
|
+
with a CPU encoder. Decoder selection is a request; inspect observed metadata
|
|
59
|
+
for native initialization evidence.
|
|
60
|
+
|
|
61
|
+
## Keep the model files together
|
|
62
|
+
|
|
63
|
+
Under this encoder selection, the built-in `whisper-small` descriptor selects
|
|
64
|
+
Intel's multilingual small [paired model archive](https://huggingface.co/Intel/whisper.cpp-openvino-models/blob/main/ggml-small-models.zip):
|
|
65
|
+
|
|
66
|
+
| File | Purpose |
|
|
67
|
+
| --- | --- |
|
|
68
|
+
| `ggml-small.bin` | Whisper model used by the native context. |
|
|
69
|
+
| `ggml-small-encoder-openvino.xml` | OpenVINO encoder description. |
|
|
70
|
+
| `ggml-small-encoder-openvino.bin` | Matching OpenVINO encoder data. |
|
|
71
|
+
|
|
72
|
+
Preparation extracts the three selected files from that archive. Matching
|
|
73
|
+
prepared models can be reused when another model selection changes. The
|
|
74
|
+
ordinary `whisper-small` selection without `encoder` continues to use its
|
|
75
|
+
existing single GGML download.
|
|
76
|
+
|
|
77
|
+
A custom local model supplies all three paths:
|
|
78
|
+
|
|
79
|
+
```js
|
|
80
|
+
const model = {
|
|
81
|
+
id: 'moon-base-small',
|
|
82
|
+
path: '/application-owned/models/ggml-small.bin',
|
|
83
|
+
encoderPath: '/application-owned/models/ggml-small-encoder-openvino.xml',
|
|
84
|
+
encoderDataPath: '/application-owned/models/ggml-small-encoder-openvino.bin'
|
|
85
|
+
};
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Use real paths on the target host. The installer copies the files together,
|
|
89
|
+
preserving their filenames so the XML can locate its matching BIN. A supplied
|
|
90
|
+
`path` selects this local-file route even when the descriptor also retains
|
|
91
|
+
archive metadata. Keep the GGML, XML and BIN from the same selected model
|
|
92
|
+
bundle.
|
|
93
|
+
|
|
94
|
+
An archive descriptor names each selected member:
|
|
95
|
+
|
|
96
|
+
```js
|
|
97
|
+
const model = {
|
|
98
|
+
id: 'whisper-small',
|
|
99
|
+
archiveUrl: 'https://huggingface.co/Intel/whisper.cpp-openvino-models/resolve/main/ggml-small-models.zip',
|
|
100
|
+
filename: 'ggml-small.bin',
|
|
101
|
+
encoderFilename: 'ggml-small-encoder-openvino.xml',
|
|
102
|
+
encoderDataFilename: 'ggml-small-encoder-openvino.bin'
|
|
103
|
+
};
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Put the descriptor in `models`. A standalone GGML `url` does not supply the
|
|
107
|
+
encoder files required by this selection.
|
|
108
|
+
|
|
109
|
+
## Development, native hosts and bundles
|
|
110
|
+
|
|
111
|
+
With the declarative requirement, `npm exec -- arcane dev` prepares the
|
|
112
|
+
application's native speech service alongside page serving. Its runtime
|
|
113
|
+
installation is under `.arcane/local-ai/runtimes`; Whisper operation files
|
|
114
|
+
and the OpenVINO encoder cache belong under `.arcane/speech/whisper`.
|
|
115
|
+
Preparation and model loading proceed independently of the page and other AI
|
|
116
|
+
services. Transcription follows the speech role's actual readiness.
|
|
117
|
+
|
|
118
|
+
A native host can prepare the same requirement explicitly:
|
|
119
|
+
|
|
120
|
+
```js
|
|
121
|
+
import path from 'node:path';
|
|
122
|
+
import {ensureLocalAIRuntimes, bundleLocalAIRuntimes} from 'arcane-os/local-ai';
|
|
123
|
+
import {createWhisperRuntime} from 'arcane-os/local-ai/whisper';
|
|
124
|
+
|
|
125
|
+
const appRoot = process.cwd();
|
|
126
|
+
const directory = path.join(appRoot, '.arcane', 'local-ai', 'runtimes');
|
|
127
|
+
const runtimes = [{
|
|
128
|
+
id: 'whisper.cpp',
|
|
129
|
+
encoder: 'openvino-npu',
|
|
130
|
+
backend: 'auto',
|
|
131
|
+
models: ['whisper-small']
|
|
132
|
+
}];
|
|
133
|
+
|
|
134
|
+
const [runtime] = await ensureLocalAIRuntimes({runtimes, directory});
|
|
135
|
+
const stt = createWhisperRuntime({
|
|
136
|
+
runtime,
|
|
137
|
+
temporaryDirectory: path.join(appRoot, '.arcane', 'speech', 'whisper')
|
|
138
|
+
});
|
|
139
|
+
|
|
140
|
+
await stt.load();
|
|
141
|
+
console.log(stt.current());
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
`ensureLocalAIRuntimes` returns prepared descriptors with absolute paths.
|
|
145
|
+
The host owns the engine's lifetime and calls `await stt.close()` at shutdown.
|
|
146
|
+
Use `subscribe(listener, {replay: true, signal})` for current-state replay and
|
|
147
|
+
later state changes. Preparation and loading accept `signal` and the
|
|
148
|
+
preparation APIs and engine constructor accept `onEvent` for diagnostics.
|
|
149
|
+
|
|
150
|
+
The application's existing native build path is
|
|
151
|
+
`npm exec -- arcane build --target windows-x64`, with the native target declared
|
|
152
|
+
in its descriptor. For an explicitly composed bundle, the same selection can
|
|
153
|
+
be copied into an application-owned output:
|
|
154
|
+
|
|
155
|
+
```js
|
|
156
|
+
const bundle = await bundleLocalAIRuntimes({
|
|
157
|
+
runtimes,
|
|
158
|
+
directory,
|
|
159
|
+
outputRoot: path.join(appRoot, '.arcane', 'native-output'),
|
|
160
|
+
platform: 'win32',
|
|
161
|
+
architecture: 'x64'
|
|
162
|
+
});
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Use a fresh bundle destination. This returns `{runtimes, files}` with paths
|
|
166
|
+
relative to the output root and places Whisper resources under
|
|
167
|
+
`runtime/local-ai/whisper.cpp`. The generated native Core entry resolves the
|
|
168
|
+
helper, runtime libraries, decoder and all three model paths from the relocated
|
|
169
|
+
bundle. Its operation files and `encoder-cache` use the native application's
|
|
170
|
+
state root. Native resources remain beside the application executable in the
|
|
171
|
+
bundle; application startup uses the prepared runtime.
|
|
172
|
+
|
|
173
|
+
## Connect a browser through native speech
|
|
174
|
+
|
|
175
|
+
The browser accesses this capability through its connected native Core speech
|
|
176
|
+
service. Browser JavaScript does not load OpenVINO DLLs or initialize the NPU.
|
|
177
|
+
Development uses the existing [local AI bootstrap](../reference/local-ai.md);
|
|
178
|
+
native packages compose the same speech service in their generated Core entry.
|
|
179
|
+
|
|
180
|
+
With an already connected Core client:
|
|
181
|
+
|
|
182
|
+
```js
|
|
183
|
+
const transcript = await client.speech.transcribe({
|
|
184
|
+
audioBase64,
|
|
185
|
+
mimeType: 'audio/webm',
|
|
186
|
+
model: 'whisper-small'
|
|
187
|
+
}, {signal});
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
The existing `AI.fetchSTT(audioFile, signal)` route remains available. Observe
|
|
191
|
+
`speech.state` and `speech.status` through the
|
|
192
|
+
[native speech contract](../reference/native-speech.md); `roles.stt` and
|
|
193
|
+
`transcriptionAvailable` expose STT readiness independently of TTS.
|
|
194
|
+
|
|
195
|
+
Requests retain the complete recording and existing language and translation
|
|
196
|
+
options. Results retain `{text, language, duration, segments}`, with segment
|
|
197
|
+
`start` and `end` in seconds. Native media decoding, transcript content and
|
|
198
|
+
segment delivery follow the [Whisper contract](../reference/local-whisper.md).
|
|
199
|
+
|
|
200
|
+
## Read initialization evidence and handle failures
|
|
201
|
+
|
|
202
|
+
The engine's `current()` and state subscriptions separate the encoder from the
|
|
203
|
+
decoder:
|
|
204
|
+
|
|
205
|
+
| Field | Meaning |
|
|
206
|
+
| --- | --- |
|
|
207
|
+
| `requestedEncoder` | `NPU` for the selected OpenVINO NPU requirement; otherwise `null`. |
|
|
208
|
+
| `observedEncoder` | `NPU` after the public OpenVINO encoder initialization call succeeds; otherwise `null`. |
|
|
209
|
+
| `encoderEvidence` | `openvino-initialization` after that success; otherwise `null`. |
|
|
210
|
+
| `requestedBackend` | The selected decoder variant, or the configured preference before a helper is selected. |
|
|
211
|
+
| `observedBackend` | Backend observed in native initialization logs, when available. |
|
|
212
|
+
| `backendEvidence` | `runtime-log` when that observation is available. |
|
|
213
|
+
|
|
214
|
+
Before encoder initialization succeeds, observed encoder fields remain
|
|
215
|
+
`null`. The helper emits `ready` only after both its Whisper context and
|
|
216
|
+
selected encoder initialize. This proves initialization for that context;
|
|
217
|
+
it does not prove that a recording has been transcribed, that every operation
|
|
218
|
+
runs on the NPU or GPU, or that either device improves performance.
|
|
219
|
+
|
|
220
|
+
When native encoder initialization returns a nonzero result,
|
|
221
|
+
`WHISPER_ENCODER_UNAVAILABLE` includes that return code in its message. The
|
|
222
|
+
failure retains the joined helper failure in
|
|
223
|
+
`cause` when present, with captured process output in that failure's `details`.
|
|
224
|
+
For NPU `WHISPER_INFERENCE_FAILED`, Arcane shuts down and
|
|
225
|
+
joins the failed helper before returning the error: a successful shutdown
|
|
226
|
+
record is available in `error.data`, including complete `stderr`; a separate
|
|
227
|
+
exit failure is retained in `error.cause`. When multiple attempts fail,
|
|
228
|
+
inspect the complete `AggregateError.errors` as well. `onEvent` can deliver
|
|
229
|
+
diagnostics while work proceeds. Keep these complete records in developer
|
|
230
|
+
diagnostics; ordinary application status can use the concise lifecycle error.
|
|
231
|
+
|
|
232
|
+
With `backend: "auto"`, a recoverable accelerated inference failure can retry
|
|
233
|
+
the complete decoded recording once using the CPU decoder and the same NPU
|
|
234
|
+
encoder selection. The engine reports `recovering` while replacing the
|
|
235
|
+
context, retains the accepted request, and accepts no concurrent transcription.
|
|
236
|
+
|
|
237
|
+
OpenVINO's synchronous encoder call has no per-request cancellation seam in
|
|
238
|
+
Whisper. Cancelling an active NPU inference therefore retires the owned
|
|
239
|
+
helper process and waits for its exit before removing recording files. The
|
|
240
|
+
model becomes unloaded. After cancellation settles, call `await stt.load()`
|
|
241
|
+
with a fresh, unaborted signal before the next transcription. `unload()` also
|
|
242
|
+
cancels active work and drains the helper; `close()` ends the engine's lifetime.
|
|
243
|
+
Cancellation before native inference need not unload an already ready model;
|
|
244
|
+
follow the resulting authoritative state.
|
|
245
|
+
|
|
246
|
+
## Supply a complete custom runtime
|
|
247
|
+
|
|
248
|
+
For an existing compatible build, select a complete variant rather than
|
|
249
|
+
individual DLLs:
|
|
250
|
+
|
|
251
|
+
```js
|
|
252
|
+
const requirement = {
|
|
253
|
+
id: 'whisper.cpp',
|
|
254
|
+
encoder: 'openvino-npu',
|
|
255
|
+
backend: 'cpu',
|
|
256
|
+
models: ['whisper-small'],
|
|
257
|
+
variants: [{
|
|
258
|
+
backend: 'cpu',
|
|
259
|
+
encoder: 'openvino-npu',
|
|
260
|
+
root: nativeRuntimeDirectory,
|
|
261
|
+
libraryDirectory: path.join(nativeRuntimeDirectory, 'bin')
|
|
262
|
+
}]
|
|
263
|
+
};
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
Each variant supplies a local `root` or an archive `url`. A supplied
|
|
267
|
+
`libraryDirectory` is relative to or inside that selected tree; otherwise it
|
|
268
|
+
is found from the native Whisper library. Each optional complete tree contains
|
|
269
|
+
its own matching `arcane-whisper.exe` beside the runtime DLLs. Preparation
|
|
270
|
+
retains that variant's executable, so multiple custom builds keep their
|
|
271
|
+
matching helpers. Include all selected OpenVINO, NPU plugin/compiler, TBB,
|
|
272
|
+
Whisper, ggml and compiler-runtime libraries, plus the CUDA libraries when
|
|
273
|
+
selecting CUDA. A CPU variant still needs the OpenVINO NPU resources.
|
|
274
|
+
|
|
275
|
+
For custom automatic decoder recovery, declare both CUDA and CPU variants
|
|
276
|
+
with `encoder: "openvino-npu"`. Custom variants are prepared as explicitly
|
|
277
|
+
supplied trees. Library search configuration belongs to the launched helper
|
|
278
|
+
process and does not change the machine's global `PATH`. The general Whisper
|
|
279
|
+
helper and FFmpeg decoder overrides remain available through the existing
|
|
280
|
+
[runtime descriptor](../reference/local-whisper.md).
|
|
281
|
+
|
|
282
|
+
## Produce the optional native runtime
|
|
283
|
+
|
|
284
|
+
Runtime producers can use the public
|
|
285
|
+
`arcane-os/local-ai/whisper/openvino-build` export with the selected installed
|
|
286
|
+
Whisper 1.9.4 source, OpenVINO 2026.3.1 Windows development tree and an existing
|
|
287
|
+
compatible CMake/MSVC environment. `cudaDirectory`, when supplied, names the
|
|
288
|
+
matching upstream b5130 CUDA distribution's library directory. It supplies
|
|
289
|
+
the precompiled CUDA backend; omitting it produces the NPU-plus-CPU runtime.
|
|
290
|
+
|
|
291
|
+
```js
|
|
292
|
+
import {cp} from 'node:fs/promises';
|
|
293
|
+
import {buildWhisperOpenVinoRuntime} from 'arcane-os/local-ai/whisper/openvino-build';
|
|
294
|
+
import {buildWhisperHelper} from 'arcane-os/local-ai/whisper/build';
|
|
295
|
+
|
|
296
|
+
const nativeRuntime = await buildWhisperOpenVinoRuntime({
|
|
297
|
+
sourceDirectory,
|
|
298
|
+
openvinoDirectory,
|
|
299
|
+
cudaDirectory,
|
|
300
|
+
outputRoot: path.join(appRoot, '.arcane', 'native-build', 'openvino'),
|
|
301
|
+
env,
|
|
302
|
+
signal,
|
|
303
|
+
onEvent
|
|
304
|
+
});
|
|
305
|
+
|
|
306
|
+
const helper = await buildWhisperHelper({
|
|
307
|
+
runtime: nativeRuntime,
|
|
308
|
+
outputRoot: path.join(appRoot, '.arcane', 'native-build', 'helper'),
|
|
309
|
+
env,
|
|
310
|
+
signal,
|
|
311
|
+
onEvent
|
|
312
|
+
});
|
|
313
|
+
|
|
314
|
+
await cp(path.join(helper.root, 'bin'), nativeRuntime.libraryDirectory, {
|
|
315
|
+
recursive: true
|
|
316
|
+
});
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
The variables for source directories, build environment, cancellation and
|
|
320
|
+
diagnostics belong to the producer. Both functions also accept `cmake`,
|
|
321
|
+
`generator` and `cmakeArgs` for that existing build environment.
|
|
322
|
+
|
|
323
|
+
The runtime producer returns `root`, `directory`, `libraryDirectory`,
|
|
324
|
+
`sourceDirectory`, selected `backends` and complete build `diagnostics`. Its
|
|
325
|
+
output keeps the copied source and build directory separate from the runtime
|
|
326
|
+
tree. It applies the encoder return-value propagation and exception-to-stderr
|
|
327
|
+
corrections in that copied source. The helper build uses the matching headers
|
|
328
|
+
and runtime and installs its required release compiler-runtime libraries;
|
|
329
|
+
copying its complete installed `bin` directory places them with the optional
|
|
330
|
+
runtime. Use `nativeRuntime.root` as the custom variant root and retain its
|
|
331
|
+
packaging notices.
|
|
332
|
+
|
|
333
|
+
This producer path builds a native distribution. Applications using the
|
|
334
|
+
precompiled asset follow the preparation and bundling paths above.
|
package/docs/publishing.md
CHANGED
|
@@ -116,6 +116,45 @@ tag and GitHub release title must both be the same bare numeric
|
|
|
116
116
|
`MAJOR.MINOR.PATCH`. Prerelease versions do not get a misleading numeric
|
|
117
117
|
GitHub release, and no release creates a Git branch for an npm dist-tag.
|
|
118
118
|
|
|
119
|
+
## Native runtime assets on numeric releases
|
|
120
|
+
|
|
121
|
+
Default SDK-hosted native asset URLs use the installed SDK's version and its
|
|
122
|
+
matching numeric GitHub release. Publishing npm alone does not make those
|
|
123
|
+
downloads available. Every applicable numeric release therefore includes its
|
|
124
|
+
compatible retained SDK-hosted assets as release follow-through:
|
|
125
|
+
|
|
126
|
+
| Selected capability | Windows x64 release asset |
|
|
127
|
+
| --- | --- |
|
|
128
|
+
| Native application host | `arcane-native-windows-x64.tar.gz` |
|
|
129
|
+
| Ordinary native Whisper helper | `arcane-whisper-windows-x64.tar.gz` |
|
|
130
|
+
| Optional Intel NPU Whisper encoder | `arcane-whisper-openvino-windows-x64.tar.gz` |
|
|
131
|
+
|
|
132
|
+
Attach each asset used by the selected package's default SDK-hosted download
|
|
133
|
+
paths, including later releases whose native implementation is unchanged.
|
|
134
|
+
Include the optional NPU asset when that release supplies `encoder:
|
|
135
|
+
"openvino-npu"`; applications still opt in, and the ordinary Whisper asset
|
|
136
|
+
remains available. Upstream runtime and model versions retain their separate
|
|
137
|
+
upstream distribution URLs; they are not renamed to the SDK version.
|
|
138
|
+
|
|
139
|
+
Reuse the producer's retained compatible archive and its recorded selected
|
|
140
|
+
evidence. Do not rebuild or redownload an unchanged archive for each release.
|
|
141
|
+
An actual native change belongs to its producer under the existing build and
|
|
142
|
+
dependency authority. Preserve the complete runtime closure and its packaging
|
|
143
|
+
notices together. Models continue through their upstream acquisition path
|
|
144
|
+
unless their redistribution is separately requested.
|
|
145
|
+
|
|
146
|
+
The release owner assigns each upload, observes its outcome and records the
|
|
147
|
+
exact asset name and public release URL in the durable release handoff. If an
|
|
148
|
+
upload result is uncertain, inspect that release's existing asset before
|
|
149
|
+
retrying. Retain the exact pending operation, executing owner and last result
|
|
150
|
+
until delivery finishes; reassign idle delegated work rather than treating
|
|
151
|
+
delegation as completion.
|
|
152
|
+
|
|
153
|
+
Source delivery, selected package verification, npm publication, native asset
|
|
154
|
+
delivery, requester notices and documentation/site delivery are distinct.
|
|
155
|
+
Native archive follow-through does not gate npm publication or require a new
|
|
156
|
+
package build. Consumer adoption remains with each application owner.
|
|
157
|
+
|
|
119
158
|
## Documentation publication
|
|
120
159
|
|
|
121
160
|
After every successful npm release and its applicable requester notices, the
|