arcane-os 0.59.1 → 0.61.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +43 -0
- package/README.md +1 -1
- package/browser-runtime/ai/core-image.mjs +360 -0
- package/browser-runtime/ai/core-model-assets.mjs +152 -0
- package/browser-runtime/core/client.mjs +8 -0
- package/browser-runtime/core/contracts.mjs +5 -0
- package/docs/reference/core/arcane-ai-contracts.md +17 -1
- package/docs/reference/core/arcane-api.md +12 -1
- package/docs/reference/core/reference/arcane-api/ai-and-ollama.md +20 -11
- package/docs/reference/core/reference/arcane-api/namespaces.md +14 -0
- package/docs/reference/core-client.md +67 -0
- package/docs/reference/core-native-packaging.md +66 -2
- package/docs/reference/inventory/package-api.json +2 -2
- package/docs/reference/inventory/runtime-modules.json +2 -2
- package/docs/reference/local-image-editing.md +79 -0
- package/docs/reference/local-image-generation.md +267 -0
- package/docs/reference/model-assets.md +147 -0
- package/docs/reference/runtime-modules.md +37 -5
- package/docs/reference/sdk-api.md +41 -5
- package/package.json +8 -3
- package/runtime/arcane/modules/ThemeBootstrap.js +217 -2
- package/schemas/arcane-app.schema.json +2 -2
- package/schemas/arcane-package.schema.json +2 -2
- package/src/core/hosts/windows/ArcaneHost.cs +211 -0
- package/src/core/hosts/windows/ArcaneLauncher.cs +5 -0
- package/src/core/services/image.mjs +563 -0
- package/src/core/services/local-ai.mjs +8 -3
- package/src/core/services/model-assets.mjs +267 -0
- package/src/import-map.mjs +2 -0
- package/src/local-ai/config.mjs +41 -1
- package/src/local-ai/dev.mjs +46 -9
- package/src/local-ai/image-editing.mjs +159 -0
- package/src/local-ai/image-library.mjs +985 -0
- package/src/local-ai/image-models/sd14.mjs +15 -0
- package/src/local-ai/image-png.mjs +74 -0
- package/src/local-ai/image-runtime.mjs +336 -0
- package/src/local-ai/image-worker.mjs +218 -0
- package/src/local-ai/install.mjs +326 -10
- package/src/native/portable-layout.mjs +28 -2
- package/src/native/windows-icon.mjs +582 -0
- package/src/native/windows-provider.mjs +60 -2
|
@@ -646,7 +646,7 @@ console.info({
|
|
|
646
646
|
|
|
647
647
|
### Overview
|
|
648
648
|
|
|
649
|
-
`Arcane.speech.synthesize(request)` converts bounded text to local speech. Use
|
|
649
|
+
`Arcane.speech.synthesize(request, options?)` converts bounded text to local speech. Use
|
|
650
650
|
it only after checking synthesis readiness, and create an audio URL from the
|
|
651
651
|
returned bytes without logging the base64 payload.
|
|
652
652
|
|
|
@@ -683,20 +683,29 @@ owned active operation before retrying.
|
|
|
683
683
|
### Streaming, cancellation, and events
|
|
684
684
|
|
|
685
685
|
Synthesis returns one complete audio result and emits no method event. The
|
|
686
|
-
wrapper timeout
|
|
687
|
-
|
|
686
|
+
wrapper timeout remains 180 seconds. Optional `{signal}` controls the Core
|
|
687
|
+
request lifetime separately from the complete synthesis payload. Pre-abort
|
|
688
|
+
sends no request; in-flight abort sends `request.cancel`, rejects with
|
|
689
|
+
`ARCANE_REQUEST_ABORTED` and suppresses late responses. Unrelated requests
|
|
690
|
+
continue. The registered native service owns actual synthesis interruption and
|
|
691
|
+
resource release; the facade supplies no engine. `AI.fetchTTS(payload, signal)`
|
|
692
|
+
forwards its signal through this call and normalizes cancellation to
|
|
693
|
+
`AbortError` with `ARCANE_AI_REQUEST_ABORTED`.
|
|
688
694
|
|
|
689
695
|
### Example
|
|
690
696
|
|
|
691
697
|
```javascript
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
696
|
-
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
|
|
698
|
+
async function synthesizeMoonEncore(input, signal) {
|
|
699
|
+
return globalThis.Arcane.speech.synthesize(
|
|
700
|
+
{
|
|
701
|
+
input,
|
|
702
|
+
voice: 'af_heart',
|
|
703
|
+
responseFormat: 'opus',
|
|
704
|
+
speed: 1
|
|
705
|
+
},
|
|
706
|
+
{signal}
|
|
707
|
+
);
|
|
708
|
+
}
|
|
700
709
|
```
|
|
701
710
|
|
|
702
711
|
## Arcane.speech.transcribe()
|
|
@@ -1038,6 +1038,20 @@ const canReadPreferences = access.methods.includes('preferences.list');
|
|
|
1038
1038
|
console.log('Preference inventory admitted:', canReadPreferences);
|
|
1039
1039
|
```
|
|
1040
1040
|
|
|
1041
|
+
## Arcane.window
|
|
1042
|
+
|
|
1043
|
+
The SDK client exposes `Arcane.window.setTheme(presentation, {signal}?)` for
|
|
1044
|
+
the window hosting the calling document. The optional `backgroundColor` and
|
|
1045
|
+
`textColor` accept numeric RGBA records or `null` for platform-default reset.
|
|
1046
|
+
The host reports each applied or unsupported field independently. Rendering
|
|
1047
|
+
and browser theme changes continue without waiting for this request.
|
|
1048
|
+
|
|
1049
|
+
Shared ThemeBootstrap forwards the app's computed semantic colors through
|
|
1050
|
+
this method when a native host is present. It does not change OS-user
|
|
1051
|
+
appearance. See the [current-window contract](../../../core-client.md#current-window-theme)
|
|
1052
|
+
for complete usage, accepted channel values, cancellation and current adapter
|
|
1053
|
+
availability. A facade method alone does not establish host support.
|
|
1054
|
+
|
|
1041
1055
|
## Arcane.appearance
|
|
1042
1056
|
|
|
1043
1057
|
### Overview
|
|
@@ -83,6 +83,9 @@ the 180,000 ms timeout and pass the complete request unchanged. A pre-aborted
|
|
|
83
83
|
signal sends no synthesis request; an in-flight abort sends `request.cancel`
|
|
84
84
|
and suppresses late responses. Actual synthesis interruption depends on the
|
|
85
85
|
registered speech service and its engine; the facade supplies no speech engine.
|
|
86
|
+
`AI.fetchTTS(payload, signal)` carries its caller-owned signal into this native
|
|
87
|
+
call and normalizes cancellation to `AbortError` with
|
|
88
|
+
`ARCANE_AI_REQUEST_ABORTED`, preserving the original error as its cause.
|
|
86
89
|
|
|
87
90
|
`Arcane.speech.transcribe(request, {signal})` forwards cancellation through the
|
|
88
91
|
same request lifetime and retains its 180,000 ms timeout and complete request.
|
|
@@ -91,6 +94,70 @@ pre-aborted signal sends no transcription request; an in-flight abort sends
|
|
|
91
94
|
`request.cancel` and suppresses late responses. The registered transcription
|
|
92
95
|
service owns interruption and release of its actual engine operation.
|
|
93
96
|
|
|
97
|
+
## Current-window theme
|
|
98
|
+
|
|
99
|
+
`Arcane.window.setTheme(presentation, {signal}?)` changes only the window
|
|
100
|
+
hosting the calling document. It uses the existing `window.setTheme` request
|
|
101
|
+
and response correlation; it neither calls `appearance.apply` nor changes the
|
|
102
|
+
operating-system user's appearance. The complete presentation object reaches
|
|
103
|
+
the host unchanged.
|
|
104
|
+
|
|
105
|
+
Each optional `backgroundColor` and `textColor` is either `null` to restore the
|
|
106
|
+
platform default or `{red, green, blue, alpha}`. All four channels are numbers:
|
|
107
|
+
finite sRGB red/green/blue from 0 through 255 and alpha from 0 through 1.
|
|
108
|
+
Omitting a field leaves its current value unchanged. There is no scheme field.
|
|
109
|
+
|
|
110
|
+
```js
|
|
111
|
+
const host=globalThis.Arcane;
|
|
112
|
+
if(host?.runtime?.current().native&&host.window?.setTheme){
|
|
113
|
+
host.window.setTheme(
|
|
114
|
+
{
|
|
115
|
+
backgroundColor:{red:29,green:22,blue:19,alpha:1},
|
|
116
|
+
textColor:{red:248,green:239,blue:229,alpha:1}
|
|
117
|
+
}
|
|
118
|
+
).then(
|
|
119
|
+
function reportWindowColors(result){
|
|
120
|
+
console.log('Moon observatory window colors:',result);
|
|
121
|
+
}
|
|
122
|
+
).catch(
|
|
123
|
+
function reportWindowFailure(error){
|
|
124
|
+
console.error('Window color update failed.',error);
|
|
125
|
+
}
|
|
126
|
+
);
|
|
127
|
+
}
|
|
128
|
+
// Page rendering continues independently of the host request.
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
The result is `{platform, supported, applied, unsupported}`. `applied` contains
|
|
132
|
+
only requested fields the platform accepted in this call, including `null`
|
|
133
|
+
for an accepted default reset. `unsupported` lists requested fields the adapter
|
|
134
|
+
cannot apply. `supported` is true when at least one field was applied. These
|
|
135
|
+
are accepted settings, not measured pixels, and the operation is not atomic.
|
|
136
|
+
Malformed arguments reject with `INVALID_ARGUMENT`; native API failures reject
|
|
137
|
+
with `ARCANE_WINDOW_THEME_FAILED`, preserving the native error and
|
|
138
|
+
`details.applied`/`details.unsupported` for earlier results in this call.
|
|
139
|
+
|
|
140
|
+
The Windows host maps background/text to DWM caption/text attributes supported
|
|
141
|
+
on Windows 11 build 22000 and later, retaining normal system window controls.
|
|
142
|
+
It rounds RGB channels to the nearest integer, with midpoint values rounded
|
|
143
|
+
up, and returns those actual accepted values. Translucent colors remain
|
|
144
|
+
unsupported without changing the existing field. Older Windows reports those
|
|
145
|
+
fields unsupported. This contract makes no promise about caption-button colors.
|
|
146
|
+
The current macOS and Linux hosts have no implementation of this method;
|
|
147
|
+
their ordinary unavailable-method response remains observable. A new facade
|
|
148
|
+
alone does not add the operation to an older native executable. Use the
|
|
149
|
+
matching published host artifact for the selected SDK version.
|
|
150
|
+
|
|
151
|
+
Cancellation uses the normal request lifetime and suppresses late responses;
|
|
152
|
+
it does not roll back a setting the host already accepted. Window settings
|
|
153
|
+
survive document navigation until a subsequent document changes them or the
|
|
154
|
+
window closes. Responses belong to their originating document.
|
|
155
|
+
|
|
156
|
+
The shared [ThemeBootstrap](runtime-modules.md#themebootstrapjs) automatically
|
|
157
|
+
forwards the app's computed `--background` and `--text-color` in native hosts.
|
|
158
|
+
Applications using that owner need no duplicate theme listener. Ordinary
|
|
159
|
+
browsers keep their CSS presentation without making a native request.
|
|
160
|
+
|
|
94
161
|
## Events and request lifetime
|
|
95
162
|
|
|
96
163
|
`events.on(name, listener)`, `once(name, listener)` and
|
|
@@ -303,6 +303,70 @@ observes complete diagnostics and exit. Cancelling that operation closes its
|
|
|
303
303
|
input, allowing the window and accepted Core service work to drain without a
|
|
304
304
|
forced process termination.
|
|
305
305
|
|
|
306
|
+
### Application icon and window colors
|
|
307
|
+
|
|
308
|
+
`appDescriptor.native.icon` selects the application's existing app-relative
|
|
309
|
+
image. Windows assembly converts PNG images, including a 512-pixel source,
|
|
310
|
+
into a derived `runtime/arcane-app.ico` with 16, 32, 48 and 256-pixel images.
|
|
311
|
+
Aspect ratio and transparency are preserved. An ICO selection keeps its
|
|
312
|
+
authored image set. The original application asset remains unchanged.
|
|
313
|
+
|
|
314
|
+
The provider embeds the selected images in the copied `Arcane.exe` and records
|
|
315
|
+
`manifest.host.icon` and `manifest.host.executableIcon:true`. It replaces the
|
|
316
|
+
executable's main icon group while preserving other groups, languages, resources
|
|
317
|
+
and file content. Conversion uses Node standard capabilities on every assembly
|
|
318
|
+
platform; it adds no application compiler, image library or Windows-only build
|
|
319
|
+
step. The selected host cache and Core executable remain unchanged.
|
|
320
|
+
|
|
321
|
+
JPEG/JPG, WebP and other existing descriptor formats retain ordinary assembly.
|
|
322
|
+
They currently report `native.icon.unsupported` through the builder's `onEvent`
|
|
323
|
+
callback, as does a selected executable whose PE layout cannot accommodate the
|
|
324
|
+
derived icon. The event identifies the app, icon, target, code and complete
|
|
325
|
+
message; `manifest.host.executableIcon:false` reports that the executable keeps
|
|
326
|
+
its prior icon. `host.icon` then points to the original selected image. This is
|
|
327
|
+
an explicit unsupported branding result, not a claim that a generic icon was
|
|
328
|
+
replaced. Unreadable files and malformed supported image data report their
|
|
329
|
+
ordinary filesystem or image-parser error.
|
|
330
|
+
|
|
331
|
+
The matching Windows launcher reads `host.icon` and loads it independently of
|
|
332
|
+
Core and page startup. ICO and standard Windows image codecs supply the window
|
|
333
|
+
and taskbar icon; JPEG/JPG can therefore work at runtime even while executable
|
|
334
|
+
embedding is unsupported. WebP is outside the host's standard image codecs;
|
|
335
|
+
runtime support is not promised. A runtime icon error preserves ordinary startup
|
|
336
|
+
and reaches complete developer diagnostics as `ARCANE_WINDOW_ICON_UNAVAILABLE`.
|
|
337
|
+
Hosts built before this option was introduced do not read the new icon field.
|
|
338
|
+
|
|
339
|
+
`Arcane.window.setTheme(presentation,{signal}?)` changes only the requesting
|
|
340
|
+
native window's chrome. The shared ThemeBootstrap supplies computed application
|
|
341
|
+
background and text colors without delaying application CSS or rendering.
|
|
342
|
+
Direct callers may supply `backgroundColor` and `textColor`, each either
|
|
343
|
+
`{red,green,blue,alpha}` or `null` to restore the platform default. RGB channels
|
|
344
|
+
are finite numbers from 0 through 255 and alpha is from 0 through 1. Omitted
|
|
345
|
+
fields remain unchanged. The complete presentation follows the existing Core
|
|
346
|
+
request transport; the window host handles `window.setTheme` before forwarding
|
|
347
|
+
other methods to Core. No system appearance setting changes.
|
|
348
|
+
|
|
349
|
+
Windows 11 build 22000 and later support titlebar background and text colors
|
|
350
|
+
through DWM. The normal titlebar and window controls remain in place. DWM accepts
|
|
351
|
+
opaque colors, so alpha values other than exactly 1 report that field as
|
|
352
|
+
unsupported. RGB values round to the nearest integer, with midpoint values
|
|
353
|
+
rounded upward. Older Windows versions report the requested fields unsupported.
|
|
354
|
+
|
|
355
|
+
The result is `{platform:'windows',supported,applied,unsupported}`. `applied`
|
|
356
|
+
contains only fields accepted during that call, including rounded channels or
|
|
357
|
+
`null` for a successful default reset. `supported` is true when at least one
|
|
358
|
+
field applied; `unsupported` lists requested fields the platform could not
|
|
359
|
+
apply. These results report platform acceptance rather than measured pixels.
|
|
360
|
+
An invalid argument uses `INVALID_ARGUMENT`; a platform API failure uses
|
|
361
|
+
`ARCANE_WINDOW_THEME_FAILED`, preserving complete native details and any
|
|
362
|
+
earlier changes in `error.details.applied` and `error.details.unsupported`.
|
|
363
|
+
The call is not atomic across fields. Existing request cancellation and document
|
|
364
|
+
lifetime semantics remain in effect; an already accepted native color change
|
|
365
|
+
is not rolled back by later cancellation. See [the Core client](core-client.md)
|
|
366
|
+
for the facade and browser behavior.
|
|
367
|
+
|
|
368
|
+
### Launch configuration
|
|
369
|
+
|
|
306
370
|
For explicit user-selected workspace/state locations, launch:
|
|
307
371
|
|
|
308
372
|
```sh
|
|
@@ -331,8 +395,8 @@ exit. It does not kill the child on an ordinary close.
|
|
|
331
395
|
`ArcaneHost.cs` composes that process with a WebView2 window through
|
|
332
396
|
`ArcaneHost.Run(options,onDiagnostic,onError)` or `ArcaneHostForm`. Options
|
|
333
397
|
select the app root/start path, stable virtual HTTPS origin, profile directory,
|
|
334
|
-
title, canonical classic-client source and Core process
|
|
335
|
-
`Ready`, `Completion` and `CloseAsync()`. Page navigation, transport connection,
|
|
398
|
+
title, optional `IconPath`, canonical classic-client source and Core process
|
|
399
|
+
inputs. The form exposes `Ready`, `Completion` and `CloseAsync()`. Page navigation, transport connection,
|
|
336
400
|
Core readiness and service readiness are separate states. Accepted work drains
|
|
337
401
|
before the window closes; complete engineering errors go to the supplied
|
|
338
402
|
diagnostic/error callbacks. Callbacks must not synchronously wait for the
|
|
@@ -3898,7 +3898,7 @@
|
|
|
3898
3898
|
"summary": "Projects a Core client into the established application-facing Arcane namespace.",
|
|
3899
3899
|
"availability": "Browser and native WebView JavaScript hosts with a Core client",
|
|
3900
3900
|
"protocol": "arcane/1 compatibility facade",
|
|
3901
|
-
"normalization": "Routes
|
|
3901
|
+
"normalization": "Routes namespace methods through client.invoke while retaining operation-specific parameters, correlation, streaming callbacks, timeouts and cancellation; window.setTheme(presentation,{signal}?) forwards the complete per-window backgroundColor/textColor record, with omitted fields unchanged and null reset, and returns the host's actual applied/unsupported result without changing system appearance"
|
|
3902
3902
|
},
|
|
3903
3903
|
{
|
|
3904
3904
|
"id": "core-client:installCoreClient",
|
|
@@ -4052,7 +4052,7 @@
|
|
|
4052
4052
|
"summary": "Assembles selected application services and browser content with the SDK-owned Windows x64 WebView2 and SEA host.",
|
|
4053
4053
|
"availability": "Node for assembly; Windows x64 with WebView2 Runtime for execution",
|
|
4054
4054
|
"protocol": "arcane-native-builder/1 windows-x64 provider",
|
|
4055
|
-
"normalization": "Uses a selected hostDirectory or the exact installed SDK version's numeric release asset; preserves complete payloads; run cancellation closes stdin and waits for host-owned graceful shutdown"
|
|
4055
|
+
"normalization": "Uses a selected hostDirectory or the exact installed SDK version's numeric release asset; preserves complete payloads; native.icon PNG/ICO derives the copied executable and host.icon without an app compiler, while other formats retain assembly with native.icon.unsupported and executableIcon:false; the matching host loads the selected window/taskbar icon and handles window.setTheme with per-field applied/unsupported results; run cancellation closes stdin and waits for host-owned graceful shutdown"
|
|
4056
4056
|
},
|
|
4057
4057
|
{
|
|
4058
4058
|
"id": "native-windows-provider:arcaneNativeBuilderProvider",
|
|
@@ -30,7 +30,7 @@
|
|
|
30
30
|
"availability": "Browser + native bridge + TWiN Cloud",
|
|
31
31
|
"protocol": "arcane-ai-browser-speech-configuration/1, AIProviderRuntime arcane-ai-provider/2 routes, globalThis.arcaneEvents, TWiN Cloud HTTPS, Arcane.ollama, Arcane.speech, Android WebView bridge",
|
|
32
32
|
"normalization": "Complete mutable caller-owned browser speech authority, SDK-owned provider registration/replacement/disposal, explicit STT/TTS activation, normalized Whisper STT and Kokoro TTS execution with default capacities 1 and 4 respectively, explicit provider-neutral execution snapshots through status(role,{execution:true}), sticky readiness, canonical window.user readiness without compatibility-event object-identity admission or polling, exact ordered structural calls with required arguments.message, atomic nonblank all-ID tool-result sequencing, complete all-choice streaming/data validation, native Ollama structural adaptation, complete-response then all validated tool callbacks then completion ordering, observed async callbacks, Blob/File transcription, automatic repeated-formatting-mark removal from cloned outbound TTS input while caller and visible content stays exact, immediate exact-segment synthesis, per-call voice/speed capture, optional final-segment pause on the existing audio clock, optional per-call terminal playback completion, detached preparation with complete semantic DBOPFS audio reuse and same-owner pending request sharing, independent prepared playback without a model load for cached audio, ordered audio-clock scheduling, playable audio, active-generation TTS operation-failure routing, mute, and cancellation are normalized; provider/model/runtime/voice policy remains caller-owned. Both speech roles default execution.device to auto: webnn-npu when navigator.ml.createContext is exposed, then webgpu when navigator.gpu is exposed, then wasm for CPU execution. Failed backend loads clean up the pool before trying fresh Workers with the same prepared model and dtype; explicit devices never fall back. STT capacity only accepts 1; TTS accepts 1 through 4. Execution snapshots expose requestedDevice, selectedDevice, maxConcurrentRequests, and activeRequestCount for both roles; selectedDevice is null while unloaded and names the successful upstream session backend request when loaded. It does not prove every operation used the physical NPU: WebNN unsupported operations may use WASM, and compatibility depends on the browser, driver, hardware, and graph.",
|
|
33
|
-
"surface": "prepareTTSPlayback(payload,signal) silently returns a real Blob or native-speech descriptor; native controls expose finished/released/state/error/pause/resume/stop, and actual release owns the playback lane. fetchTTS and durable prepareTTS report ARCANE_AI_TTS_AUDIO_EXPORT_UNAVAILABLE for native voices. Request-local model overrides remain exact and never mutate the configured model. Browser-speech protocol/event/error/reason constants; default `AI`; read-only `providerRuntime`, `browserSpeechConfiguration`, and `browserSpeechDescriptor`; explicit `providerRuntime.status(role,{execution:true})` snapshots; `streamTTS(text,end,options={})` with optional voice, speed, pauseAfterMs, and waitForPlayback; automatic speech-input cleanup with SDK-internal preparation metadata; `prepareTTS({parts,storage,identity,signal,onState})` returning ordered segments, state, ready, getAudio(index), and cancel(); `playPreparedTTS(prepared,{signal,onState})` returning state, error, finished, pause(), resume(), and stop() with one playback lane per AI and observational state/error callbacks; configure/dispose speech, route lifecycle, declaration-validated chat/stream requests with exact ordered calls; optional streamRequest toolText:{name,field} and onToolText(text,call,displayId) for decoded selected argument text with actual call identity, distinct from ordinary prose and terminal tool execution; synthesis/transcription and playback controls; initializes from current canonical user readiness, installs `window.ai`, and projects `ai-ready` plus active-generation `ai-tts-failure`."
|
|
33
|
+
"surface": "Native fetchTTS(payload,signal) and fetchSTT(audio,signal) forward caller cancellation separately from their complete Core speech payloads, normalize aborts to AbortError and suppress late results; actual engine interruption remains service-owned. prepareTTSPlayback(payload,signal) silently returns a real Blob or native-speech descriptor; native controls expose finished/released/state/error/pause/resume/stop, and actual release owns the playback lane. fetchTTS and durable prepareTTS report ARCANE_AI_TTS_AUDIO_EXPORT_UNAVAILABLE for native voices. Request-local model overrides remain exact and never mutate the configured model. Browser-speech protocol/event/error/reason constants; default `AI`; read-only `providerRuntime`, `browserSpeechConfiguration`, and `browserSpeechDescriptor`; explicit `providerRuntime.status(role,{execution:true})` snapshots; `streamTTS(text,end,options={})` with optional voice, speed, pauseAfterMs, and waitForPlayback; automatic speech-input cleanup with SDK-internal preparation metadata; `prepareTTS({parts,storage,identity,signal,onState})` returning ordered segments, state, ready, getAudio(index), and cancel(); `playPreparedTTS(prepared,{signal,onState})` returning state, error, finished, pause(), resume(), and stop() with one playback lane per AI and observational state/error callbacks; configure/dispose speech, route lifecycle, declaration-validated chat/stream requests with exact ordered calls; optional streamRequest toolText:{name,field} and onToolText(text,call,displayId) for decoded selected argument text with actual call identity, distinct from ordinary prose and terminal tool execution; synthesis/transcription and playback controls; initializes from current canonical user readiness, installs `window.ai`, and projects `ai-ready` plus active-generation `ai-tts-failure`."
|
|
34
34
|
},
|
|
35
35
|
{
|
|
36
36
|
"file": "runtime/arcane/modules/AIModelSelectionController.js",
|
|
@@ -1364,7 +1364,7 @@
|
|
|
1364
1364
|
"availability": "Browser/native hybrid",
|
|
1365
1365
|
"protocol": "ThemeManager + host Arcane.events projected once through the per-realm globalThis.arcaneEvents authority",
|
|
1366
1366
|
"normalization": "Theme state normalized; storage/native errors mixed.",
|
|
1367
|
-
"surface": "`bootstrapArcaneTheme()`, `disposeArcaneThemeBootstrap()`, `arcaneThemeReady`, and default ready promise. Shared bootstrap replays an already-ready User and
|
|
1367
|
+
"surface": "`bootstrapArcaneTheme()`, `disposeArcaneThemeBootstrap()`, `arcaneThemeReady`, and default ready promise. Shared bootstrap replays an already-ready User. In native hosts it independently forwards computed --background/--text-color through Arcane.window.setTheme, observing shared theme events, root/body presentation attributes, stylesheets, device scheme and page restoration; browser CSS resolution retains alpha and converts modern colors to sRGB. Disposal releases appearance/user/theme subscriptions, native sampling observers and listeners, and pending window requests. Ordinary browsers remain silent. Presentation cache is nonauthoritative; use classic ThemePresentation after app-id and before CSS for early restoration."
|
|
1368
1368
|
},
|
|
1369
1369
|
{
|
|
1370
1370
|
"file": "runtime/arcane/modules/ThemeManager.js",
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# Local image edit inputs
|
|
2
|
+
|
|
3
|
+
The SDK input decoder prepares a PNG raster for the shared native image
|
|
4
|
+
runtime. Decoding does not load a model or perform inference. The runtime owns
|
|
5
|
+
model selection, supported editing operations, native calls and their complete
|
|
6
|
+
results. Applications retain the original image and decide whether to save a
|
|
7
|
+
newly generated result.
|
|
8
|
+
|
|
9
|
+
## Decoder contract
|
|
10
|
+
|
|
11
|
+
`src/local-ai/image-editing.mjs` exports the internal worker function
|
|
12
|
+
`decodeImage({data, mediaType, signal})`.
|
|
13
|
+
|
|
14
|
+
| Field | Contract |
|
|
15
|
+
| --- | --- |
|
|
16
|
+
| `data` | Complete encoded PNG in a `Uint8Array`; a Node `Buffer` is also accepted. Only the supplied view is read. |
|
|
17
|
+
| `mediaType` | `image/png`, an empty string, or omitted. The PNG signature is read from the actual input. Other declared formats report unsupported input. |
|
|
18
|
+
| `signal` | Optional `AbortSignal` belonging to the image request. |
|
|
19
|
+
|
|
20
|
+
The promise returns `{width, height, channel, data}`. Dimensions come from the
|
|
21
|
+
source raster; `channel` is `3` for RGB or `4` for RGBA. Returned `data` is a
|
|
22
|
+
separate contiguous `Uint8Array` containing interleaved samples, with pixels
|
|
23
|
+
ordered left to right and rows ordered top to bottom. The native request owner
|
|
24
|
+
retains this allocation until the native operation actually completes,
|
|
25
|
+
including after a cancellation request.
|
|
26
|
+
|
|
27
|
+
Input decoding accepts static, non-interlaced, 8-bit RGB and RGBA PNGs, all
|
|
28
|
+
five PNG row filters, and image data spread across multiple `IDAT` chunks.
|
|
29
|
+
RGB transparent-color information is represented as RGBA. An optional suggested
|
|
30
|
+
palette on an RGB or RGBA image does not change its color type.
|
|
31
|
+
|
|
32
|
+
Indexed-color, grayscale, 16-bit, interlaced and animated PNGs currently report
|
|
33
|
+
unsupported input. JPEG and WebP are also unsupported by this decoder. The
|
|
34
|
+
decoder does not silently convert those formats or select one animation frame.
|
|
35
|
+
|
|
36
|
+
The original encoded image is copied for the operation and never modified or
|
|
37
|
+
detached. Decoding preserves stored color samples and straight alpha, including
|
|
38
|
+
RGB values underneath transparent pixels. It does not composite onto a chosen
|
|
39
|
+
background, rotate pixels from metadata, apply a color-profile conversion,
|
|
40
|
+
resize, crop or create a file. Metadata remains in the original image; the
|
|
41
|
+
native raster record contains pixels and dimensions only.
|
|
42
|
+
|
|
43
|
+
## Execution and cancellation
|
|
44
|
+
|
|
45
|
+
Invoke the decoder inside the shared image runtime's existing worker.
|
|
46
|
+
Inflation uses asynchronous Node zlib. Row reconstruction yields between
|
|
47
|
+
scanlines so the worker can receive cancellation. An abort during inflation is
|
|
48
|
+
observed after that native operation settles; it does not claim instant
|
|
49
|
+
interruption. A cancelled decode returns no raster for inference.
|
|
50
|
+
|
|
51
|
+
`ARCANE_IMAGE_INPUT_UNSUPPORTED` describes an unavailable input format or
|
|
52
|
+
encoding. `ARCANE_IMAGE_DECODE_FAILED` describes unreadable PNG structure or
|
|
53
|
+
scanlines. Node zlib failures retain their complete original errors, and
|
|
54
|
+
cancellation retains the owning signal's reason or Node `AbortError`. A decode
|
|
55
|
+
failure never returns partial imagery or substitutes a different image.
|
|
56
|
+
|
|
57
|
+
## Editing semantics at the native owner
|
|
58
|
+
|
|
59
|
+
Whole-image img2img uses an initial image, prompt and strength. Masked editing
|
|
60
|
+
adds a separate mask; instruction/reference editing uses a model that supports
|
|
61
|
+
reference conditioning. These are distinct operations and must be advertised
|
|
62
|
+
only when the selected model and native path implement them. Ordinary img2img
|
|
63
|
+
does not require LoRA training. The candidate engine documents this directly
|
|
64
|
+
with its base SD1.4 model in the
|
|
65
|
+
[tagged img2img example](https://github.com/leejet/stable-diffusion.cpp/blob/master-929-3f8527a/docs/sd.md#img2img-example).
|
|
66
|
+
|
|
67
|
+
Source dimensions are decoder output, not an implicit request to change the
|
|
68
|
+
generation dimensions. The model owner documents its actual canvas adaptation
|
|
69
|
+
and reports actual output dimensions. In the candidate engine, temporary
|
|
70
|
+
initial images use model preprocessing while caller images remain unchanged;
|
|
71
|
+
even `mode=none` does not suppress all later model adaptation. See the
|
|
72
|
+
[native preprocessing contract](https://github.com/leejet/stable-diffusion.cpp/blob/master-929-3f8527a/docs/image_preprocessing.md).
|
|
73
|
+
|
|
74
|
+
An RGB-only model such as the selected SD1.4 candidate consumes the stored RGB
|
|
75
|
+
and does not preserve alpha. The SDK must not silently choose a compositing
|
|
76
|
+
background or claim that passing RGBA preserves its displayed appearance.
|
|
77
|
+
Strength zero is also not an exact-copy operation: native img2img still uses
|
|
78
|
+
the model's encode, sampling and decode path. These model behaviors are
|
|
79
|
+
separate from the decoder preserving the original input.
|
|
@@ -0,0 +1,267 @@
|
|
|
1
|
+
# Local image generation through Core
|
|
2
|
+
|
|
3
|
+
The SDK's local image service runs selected stable-diffusion.cpp models in a
|
|
4
|
+
retained native context. The browser uses Core for model lifecycle and complete
|
|
5
|
+
PNG results. Applications own image prompts, style choices, character identity,
|
|
6
|
+
associations and saved results.
|
|
7
|
+
|
|
8
|
+
Runtime installation, model asset preparation, model loading and image
|
|
9
|
+
generation are separate operations. The service starts promptly and retains
|
|
10
|
+
runtime preparation in the background. `image.status` remains callable during
|
|
11
|
+
preparation; rendering and unrelated Core services continue independently.
|
|
12
|
+
|
|
13
|
+
## Selection and model assets
|
|
14
|
+
|
|
15
|
+
Select the runtime in `arcane-app.json` under `native.localAI`, or in the
|
|
16
|
+
corresponding package projection's `localAI` record:
|
|
17
|
+
|
|
18
|
+
```json
|
|
19
|
+
{
|
|
20
|
+
"runtimes": [
|
|
21
|
+
{
|
|
22
|
+
"id": "stable-diffusion.cpp",
|
|
23
|
+
"version": "master-929-3f8527a",
|
|
24
|
+
"backend": "auto",
|
|
25
|
+
"models": ["sd14"]
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
}
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
`sd14` selects the Stable Diffusion 1.4 definition and its official
|
|
32
|
+
`sd-v1-4.ckpt` resource. A model definition contains `id`, `name`, `family`,
|
|
33
|
+
`resources`, `context`, `defaults` and `operations`. Selection supplies the
|
|
34
|
+
model's resource metadata; runtime installation does not independently download
|
|
35
|
+
or bundle model files. The shared model asset owner prepares selected browser
|
|
36
|
+
assets and supplies a Core working projection for native path-based loading.
|
|
37
|
+
|
|
38
|
+
The first published distribution path selects CPU on Windows and Linux.
|
|
39
|
+
On macOS, `auto` selects the same upstream universal runtime's built-in Metal
|
|
40
|
+
backend, with CPU fallback; explicit `cpu` remains available. A caller-supplied
|
|
41
|
+
runtime archive with `backend: 'auto'` uses its available accelerated device
|
|
42
|
+
with CPU fallback. Built-in Windows CUDA distribution preparation is a separate
|
|
43
|
+
pending addition. Actual backend/device availability and failures
|
|
44
|
+
come from the runtime, separately from hardware metadata or an installed archive.
|
|
45
|
+
Cancellation, invalid model input and inference failures retain their actual
|
|
46
|
+
errors. Model loading has no native cancellation handle before its constructor
|
|
47
|
+
returns; cancellation joins that call and releases its returned context before
|
|
48
|
+
the operation settles.
|
|
49
|
+
|
|
50
|
+
The complete prepared projection has an `id`, a `directory`, a `members` list
|
|
51
|
+
and `release()`. `resourcePaths` maps native model resource roles to exact
|
|
52
|
+
projection member paths. For the selected checkpoint, the role is `model`.
|
|
53
|
+
The member name must match the preparation result. See
|
|
54
|
+
[Model assets through Core](model-assets.md) for the shared preparation
|
|
55
|
+
lifecycle. Core acquires its own
|
|
56
|
+
retain before loading. The caller can release its preparation ownership after
|
|
57
|
+
successful `load()` acknowledgement: Core retains the files while its native
|
|
58
|
+
context needs them. Release caller ownership when loading fails too; native
|
|
59
|
+
cleanup keeps its own retain through actual completion. A later
|
|
60
|
+
`generate({model: 'sd14', ...})` reuses that prepared selection. Replacing or
|
|
61
|
+
unloading the context releases displaced projections
|
|
62
|
+
after actual native operations settle. Core shutdown joins owned operations
|
|
63
|
+
before releasing retained model files.
|
|
64
|
+
|
|
65
|
+
`model` also accepts a complete prepared model descriptor. Native callers can
|
|
66
|
+
supply actual resource paths directly. The browser projection arguments supply
|
|
67
|
+
the native paths through the shared asset service and preserve the remaining
|
|
68
|
+
descriptor fields. The image service creates no second model downloader or
|
|
69
|
+
durable model store.
|
|
70
|
+
|
|
71
|
+
## Browser accessor
|
|
72
|
+
|
|
73
|
+
```js
|
|
74
|
+
import {createCoreImageRuntime} from 'arcane-os/ai/core-image';
|
|
75
|
+
import {prepareCoreModelAssets} from 'arcane-os/ai/core-model-assets';
|
|
76
|
+
import {createBrowserModelSource, createDbopfsModelStore} from 'arcane-os/ai/browser-wasm';
|
|
77
|
+
|
|
78
|
+
const page = new AbortController();
|
|
79
|
+
window.addEventListener(
|
|
80
|
+
'pagehide',
|
|
81
|
+
function detachImagePage() {
|
|
82
|
+
page.abort();
|
|
83
|
+
},
|
|
84
|
+
{once: true}
|
|
85
|
+
);
|
|
86
|
+
|
|
87
|
+
const images = createCoreImageRuntime(
|
|
88
|
+
{signal: page.signal}
|
|
89
|
+
);
|
|
90
|
+
|
|
91
|
+
images.subscribe(
|
|
92
|
+
function showImageStatus(snapshot) {
|
|
93
|
+
statusElement.textContent = snapshot.status ?? snapshot.state;
|
|
94
|
+
}
|
|
95
|
+
);
|
|
96
|
+
|
|
97
|
+
const selected = (await images.inspect()).models.find(
|
|
98
|
+
function selectedCheckpoint(model) {
|
|
99
|
+
return model.id === 'sd14';
|
|
100
|
+
}
|
|
101
|
+
);
|
|
102
|
+
const resource = selected.resources.model;
|
|
103
|
+
const source = createBrowserModelSource(
|
|
104
|
+
{
|
|
105
|
+
id: selected.id,
|
|
106
|
+
files: [
|
|
107
|
+
{name: resource.filename, url: resource.url}
|
|
108
|
+
]
|
|
109
|
+
}
|
|
110
|
+
);
|
|
111
|
+
const store = createDbopfsModelStore(
|
|
112
|
+
{dbopfs, downloadConcurrency: 4}
|
|
113
|
+
);
|
|
114
|
+
const stored = await store.ensure(
|
|
115
|
+
source,
|
|
116
|
+
{signal: page.signal, offline: false}
|
|
117
|
+
);
|
|
118
|
+
const projection = await prepareCoreModelAssets(
|
|
119
|
+
{
|
|
120
|
+
workingDirectory: '.arcane/model-working',
|
|
121
|
+
members: source.files.map(
|
|
122
|
+
function preparedCheckpoint(member, index) {
|
|
123
|
+
return {path: member.name, file: stored.files[index]};
|
|
124
|
+
}
|
|
125
|
+
),
|
|
126
|
+
signal: page.signal
|
|
127
|
+
}
|
|
128
|
+
);
|
|
129
|
+
|
|
130
|
+
try {
|
|
131
|
+
await images.load(
|
|
132
|
+
{
|
|
133
|
+
model: 'sd14',
|
|
134
|
+
assetProjectionId: projection.id,
|
|
135
|
+
resourcePaths: {
|
|
136
|
+
model: 'sd-v1-4.ckpt'
|
|
137
|
+
},
|
|
138
|
+
signal: page.signal
|
|
139
|
+
}
|
|
140
|
+
);
|
|
141
|
+
} finally {
|
|
142
|
+
await projection.release();
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
const result = await images.generate(
|
|
146
|
+
{
|
|
147
|
+
model: 'sd14',
|
|
148
|
+
prompt: 'Portrait of a dignified raccoon astronaut holding a tiny moon cactus.',
|
|
149
|
+
signal: page.signal,
|
|
150
|
+
onProgress(progress) {
|
|
151
|
+
console.log(progress);
|
|
152
|
+
}
|
|
153
|
+
}
|
|
154
|
+
);
|
|
155
|
+
|
|
156
|
+
for (const image of result.images) {
|
|
157
|
+
const url = URL.createObjectURL(image.blob);
|
|
158
|
+
const preview = new Image();
|
|
159
|
+
preview.alt = 'Generated raccoon astronaut portrait';
|
|
160
|
+
function releasePreviewURL() {
|
|
161
|
+
URL.revokeObjectURL(url);
|
|
162
|
+
}
|
|
163
|
+
preview.addEventListener(
|
|
164
|
+
'load',
|
|
165
|
+
releasePreviewURL,
|
|
166
|
+
{once: true}
|
|
167
|
+
);
|
|
168
|
+
preview.addEventListener(
|
|
169
|
+
'error',
|
|
170
|
+
releasePreviewURL,
|
|
171
|
+
{once: true}
|
|
172
|
+
);
|
|
173
|
+
preview.src = url;
|
|
174
|
+
imageContainer.append(preview);
|
|
175
|
+
}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
The example uses the application's existing `dbopfs` instance and existing
|
|
179
|
+
`statusElement` and `imageContainer` elements. The published model store accepts
|
|
180
|
+
the selected checkpoint as an ordinary resource: `ensure()` returns complete
|
|
181
|
+
ordered `Blob` or `File` members after its resumable download. Range support
|
|
182
|
+
allows concurrent transfer within a member; those transfer parts remain the
|
|
183
|
+
store's concern. Use `offline: true` to require an already complete stored model.
|
|
184
|
+
The store does not select or run the image engine. In source development,
|
|
185
|
+
load the development Core bootstrap described
|
|
186
|
+
in [Local AI through Core](local-ai.md#development) before constructing the
|
|
187
|
+
accessor. Native hosts install their Core client through the existing host path.
|
|
188
|
+
An ordinary browser without Core reports local image generation as unavailable.
|
|
189
|
+
|
|
190
|
+
| Member | Contract |
|
|
191
|
+
|---|---|
|
|
192
|
+
| `createCoreImageRuntime({client?, signal?, onEvent?})` | Uses the installed Core client by default, subscribes to lifecycle events and asynchronously reads one initial status snapshot. It returns immediately. |
|
|
193
|
+
| `load({model, context?, assetProjectionId?, resourcePaths?, signal?})` | Loads the selected model and returns authoritative service state after native completion. `context` supplies native context options. |
|
|
194
|
+
| `generate({model, prompt, parameters?, assetProjectionId?, resourcePaths?, signal?, onProgress?})` | Returns the complete result with `images: [{blob, mediaType, width, height}]`. Additional runtime result metadata remains present. |
|
|
195
|
+
| `unload({signal?})` | Explicitly unloads the shared selected native context and returns its resulting state. |
|
|
196
|
+
| `inspect({signal?})` | Reads current Core status without waiting for model or runtime preparation. |
|
|
197
|
+
| `current()` | Synchronously returns the retained lifecycle snapshot and this accessor's transient request status. |
|
|
198
|
+
| `subscribe(listener, {replay = true, signal?})` | Observes `image.state` and immediately replays the current snapshot by default; returns an unsubscribe function. |
|
|
199
|
+
| `close()` | Cancels this accessor's requests, removes subscriptions and joins its request promises. It leaves the shared Core client and selected model under their existing owners. |
|
|
200
|
+
|
|
201
|
+
Generation publishes `status: 'Thinking'` synchronously when accepted, before
|
|
202
|
+
transport, preparation, model loading or inference waits. This remains transient
|
|
203
|
+
status until the complete image result arrives. Native sampling progress is
|
|
204
|
+
reported as progress, with no fabricated image chunks or conversation turns.
|
|
205
|
+
The accessor's `requests` records contain method, stream ID, status and progress;
|
|
206
|
+
they contain no saved prompt or image history.
|
|
207
|
+
|
|
208
|
+
The authoritative lifecycle snapshot includes `selectedModel`, `loaded`,
|
|
209
|
+
`state`, `backend`, `progress`, `error`, native resource paths and device state.
|
|
210
|
+
Service metadata adds runtime `id`, installation/availability state and selected
|
|
211
|
+
model definitions, including their resource metadata before engine preparation
|
|
212
|
+
finishes. Installed files do not imply a loaded model. A late status
|
|
213
|
+
response never overwrites a newer lifecycle event.
|
|
214
|
+
|
|
215
|
+
Prompts and parameters pass unchanged to the native runtime. Native option
|
|
216
|
+
names, supported model operations and defaults belong to that runtime and the
|
|
217
|
+
selected model definition. A prompt containing U+0000 or an unpaired UTF-16
|
|
218
|
+
surrogate cannot pass unchanged through the native UTF-8 C-string boundary and
|
|
219
|
+
reports that incompatibility.
|
|
220
|
+
|
|
221
|
+
The request signal propagates through Core to the owning native operation.
|
|
222
|
+
Aborted requests produce no late image result or progress callback. Request
|
|
223
|
+
invocations use `timeoutMs: 0`; there is no arbitrary image-generation timeout.
|
|
224
|
+
Errors retain their actual details on rejected promises and diagnostic events.
|
|
225
|
+
Applications choose concise ordinary status text and keep complete engineering
|
|
226
|
+
details in their diagnostic surface, outside durable chat history.
|
|
227
|
+
|
|
228
|
+
`onEvent` observes `image.state`, `image.progress`, `image.request`,
|
|
229
|
+
`image.result` and `image.error` records. Observer failures are reported to the
|
|
230
|
+
developer console and do not replace inference results. Observers own their
|
|
231
|
+
asynchronous callback lifetimes. The SDK persists no prompt, result, transcript
|
|
232
|
+
or application-owned image association.
|
|
233
|
+
|
|
234
|
+
## Core service and transport
|
|
235
|
+
|
|
236
|
+
`createLocalImageService(configuration, options)` is exported from
|
|
237
|
+
`arcane-os/core/image`. `configuration` is the selected `localAI` record.
|
|
238
|
+
Options are `appRoot`, installed `runtimes`, `signal`, `onEvent`, an optional
|
|
239
|
+
`prepare` callback and the existing `modelAssets` service instance.
|
|
240
|
+
|
|
241
|
+
`prepare({appRoot, configuration, requirement, signal, onEvent})` returns one
|
|
242
|
+
installed runtime record. Preparation runs once per service. That record carries
|
|
243
|
+
`libraryPath`, `bindingModulePath`, `backend`, `variants` and model definitions.
|
|
244
|
+
Runtime variants preserve their native backend, library path and runtime root.
|
|
245
|
+
The service retains one native runtime and subscribes to its current-state
|
|
246
|
+
replay. Expensive model execution belongs to that runtime's worker.
|
|
247
|
+
|
|
248
|
+
| Method/event | Parameters or result |
|
|
249
|
+
|---|---|
|
|
250
|
+
| `image.status` | Immediate lifecycle snapshot. |
|
|
251
|
+
| `image.load` | `{model, context?, assetProjectionId?, resourcePaths?}`; returns lifecycle snapshot. |
|
|
252
|
+
| `image.generate` | `{model, prompt, parameters?, streamId?, assetProjectionId?, resourcePaths?}`; returns complete encoded images. |
|
|
253
|
+
| `image.unload` | Empty parameters; returns lifecycle snapshot after unload. |
|
|
254
|
+
| `image.state` | Complete service lifecycle snapshot. Service and browser `subscribe()` provide current-state replay. |
|
|
255
|
+
| `image.progress` | `{streamId, requestId, status: 'Thinking', progress}` for the corresponding generation request. |
|
|
256
|
+
|
|
257
|
+
Core's JSON transport represents each complete PNG as
|
|
258
|
+
`{data, encoding: 'base64', mediaType: 'image/png', width, height}`. The browser
|
|
259
|
+
decodes that complete representation into a `Blob` and preserves its metadata.
|
|
260
|
+
There is no result resizing, clipping, storage write or application policy at
|
|
261
|
+
this transport boundary.
|
|
262
|
+
|
|
263
|
+
One service prepares one selected runtime, retains one selected native context
|
|
264
|
+
and performs one native generation per request. The native owner serializes
|
|
265
|
+
only competing operations on that context. Independent Core services and page
|
|
266
|
+
rendering remain concurrent. Native platform execution evidence and package
|
|
267
|
+
publication are separate from this source/API description.
|