arcane-os 0.51.0 → 0.51.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +20 -0
- package/README.md +81 -44
- package/browser-runtime/ai/twin-cloud.mjs +4 -0
- package/docs/architecture.md +33 -17
- package/docs/platform-targets.md +15 -2
- package/docs/reference/README.md +31 -26
- package/docs/reference/ai/browser-speech.md +106 -31
- package/docs/reference/ai/browser-wasm.md +27 -24
- package/docs/reference/ai/twin-cloud.md +30 -11
- package/docs/reference/asset-versioning.md +34 -37
- package/docs/reference/availability-and-normalization.md +31 -20
- package/docs/reference/cli.md +30 -17
- package/docs/reference/inventory/package-api.json +34 -6
- package/docs/reference/inventory/runtime-components.json +25 -11
- package/docs/reference/inventory/runtime-entities.json +6 -3
- package/docs/reference/inventory/runtime-modules.json +63 -8
- package/docs/reference/protocols.md +62 -56
- package/docs/reference/pwa.md +20 -13
- package/docs/reference/runtime-components.md +9 -2
- package/docs/reference/runtime-entities.md +14 -2
- package/docs/reference/runtime-modules.md +59 -2
- package/docs/reference/sdk-api.md +231 -65
- package/docs/roadmap.md +32 -26
- package/package.json +1 -1
- package/runtime/arcane/modules/AI.js +20 -6
- package/src/dev-server.mjs +3 -30
|
@@ -1,11 +1,26 @@
|
|
|
1
1
|
# Browser speech providers
|
|
2
2
|
|
|
3
|
-
`arcane-os/ai/browser-speech` supplies caller-selected Whisper
|
|
4
|
-
|
|
3
|
+
`arcane-os/ai/browser-speech` supplies caller-selected Whisper transcription,
|
|
4
|
+
native browser recognition and synthesis, Kokoro text-to-speech, and TWiN Cloud
|
|
5
|
+
text-to-speech providers. Its local model path
|
|
5
6
|
provides artifact storage, live module routing, role Workers, provider/2
|
|
6
7
|
adapters, bounded parallel TTS synthesis, audio normalization, cancellation,
|
|
7
8
|
and cleanup.
|
|
8
9
|
|
|
10
|
+
## Choose a speech provider
|
|
11
|
+
|
|
12
|
+
| Need | Factory | Input or output | Runtime and locality |
|
|
13
|
+
| --- | --- | --- | --- |
|
|
14
|
+
| Transcribe a recorded clip | `createBrowserWhisperProvider` | Audio input, complete transcript output | Caller-selected local model and Worker runtime |
|
|
15
|
+
| Recognize live microphone speech | `createBrowserSpeechRecognitionProvider` | Final segments and transient interim text | Native browser service; may use a remote service |
|
|
16
|
+
| Speak with installed browser voices | `createBrowserSpeechSynthesisProvider` | Silent native playback descriptor, then explicit playback | Native voice inventory; individual voices may be remote |
|
|
17
|
+
| Generate reusable audio locally | `createBrowserKokoroProvider` | Synthesized audio | Caller-selected local model, voice and Worker runtime |
|
|
18
|
+
| Generate audio through TWiN Cloud | `createTwinCloudTTSProvider` | Completed remote synthesis returned as audio | Application-owned key reader and selected FAL model |
|
|
19
|
+
|
|
20
|
+
STT and TTS have independent selections and lifecycles. Applications choose
|
|
21
|
+
their defaults, recognition language, voice, and activation controls. Native
|
|
22
|
+
browser providers need no SDK model download or DBOPFS artifact store.
|
|
23
|
+
|
|
9
24
|
## Native browser speech recognition
|
|
10
25
|
|
|
11
26
|
Select the browser's `SpeechRecognition` or `webkitSpeechRecognition` service
|
|
@@ -44,6 +59,36 @@ component `transcribe(file,context)` callback retains the existing clip path.
|
|
|
44
59
|
Final segments waiting behind an application save remain visible in that
|
|
45
60
|
transient surface until the ordered queue appends them to the main transcript.
|
|
46
61
|
|
|
62
|
+
### Recognition language and editable Chat drafts
|
|
63
|
+
|
|
64
|
+
The shared speech control exposes `recognitionLanguage` as an editable BCP47
|
|
65
|
+
tag. Its visible picker is available before live capture, shows every suggested
|
|
66
|
+
language when opened, and filters after the user changes its text. Suggestions
|
|
67
|
+
retain native language and region labels; callers can also supply an exact
|
|
68
|
+
custom tag. The picker is disabled during capture and outside live-recognition
|
|
69
|
+
mode. Recognition language is independent of Chat's conversation language.
|
|
70
|
+
|
|
71
|
+
For an initialized shared Chat component, choose whether the completed
|
|
72
|
+
transcription submits immediately or remains editable:
|
|
73
|
+
|
|
74
|
+
```javascript
|
|
75
|
+
// chat is the application's ready shared Chat component.
|
|
76
|
+
chat.transcriptionMode = 'draft';
|
|
77
|
+
chat.recognitionLanguage = 'es-MX';
|
|
78
|
+
chat.appendDraft('Captain, the moon-powered toaster reports: ');
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
`'submit'` is the existing default. In `'draft'` mode, final and interim native
|
|
82
|
+
recognition appear immediately in the composer alongside typed text. Completion
|
|
83
|
+
leaves Send to the user. Interim replacement preserves user edits; cancellation
|
|
84
|
+
retires only the unedited interim range and retains confirmed text. The shared
|
|
85
|
+
speech component emits cumulative `speech-transcription-progress` detail with
|
|
86
|
+
`text`, `interim`, and `operationId`; confirmed completion keeps the existing
|
|
87
|
+
`speech-transcription-complete` event. Interim text remains transient and never
|
|
88
|
+
enters a completion payload or durable history.
|
|
89
|
+
|
|
90
|
+
### Direct capture
|
|
91
|
+
|
|
47
92
|
For direct use, `ai.supportsTranscriptionCapture()` detects the selected
|
|
48
93
|
provider's capture capability. `ai.createTranscriptionCapture(options)` returns
|
|
49
94
|
`null` for a file-only provider or a single-use capture handle with
|
|
@@ -107,6 +152,11 @@ const unsubscribe = voicePreview.subscribeCatalog(function showVoices(models) {
|
|
|
107
152
|
function playIntroduction(input, voice) {
|
|
108
153
|
return voicePreview.play({input, voice});
|
|
109
154
|
}
|
|
155
|
+
|
|
156
|
+
// voiceURI comes from the real catalog or the application's saved selection.
|
|
157
|
+
function selectDefaultVoice(voiceURI) {
|
|
158
|
+
return voicePreview.setDefaultVoice(voiceURI);
|
|
159
|
+
}
|
|
110
160
|
// The owning view unsubscribes and disposes voicePreview when it closes.
|
|
111
161
|
```
|
|
112
162
|
|
|
@@ -173,6 +223,16 @@ when the returned descriptor is played. `AI.fetchTTS()` and durable
|
|
|
173
223
|
`ARCANE_AI_TTS_AUDIO_EXPORT_UNAVAILABLE` for native synthesis, which exposes no
|
|
174
224
|
downloadable audio file.
|
|
175
225
|
|
|
226
|
+
Choose preparation according to the result the application needs:
|
|
227
|
+
|
|
228
|
+
| API | Result | Playback and retention |
|
|
229
|
+
| --- | --- | --- |
|
|
230
|
+
| `provider.prepare(payload,{signal})` | Native speech descriptor | Silent until its explicit `play()` call; keeps the selected voice |
|
|
231
|
+
| `ai.prepareTTSPlayback(payload,signal)` | Promise resolving a native descriptor or real audio Blob | Silent, provider-neutral preparation; no durable storage |
|
|
232
|
+
| `ai.streamTTS(text,end,options)` | Promise resolving a preparation or playback boolean | Shared segmentation and playback in original order |
|
|
233
|
+
| `ai.fetchTTS(payload,signal)` | Promise resolving a real audio Blob | Audio-file providers only; does not play |
|
|
234
|
+
| `ai.prepareTTS(options)` | Preparation handle with `ready` and `getAudio(index)` | Audio-file providers only; optional caller-owned durable storage |
|
|
235
|
+
|
|
176
236
|
<a id="digitalocean-fal-text-to-speech"></a>
|
|
177
237
|
|
|
178
238
|
## TWiN Cloud text-to-speech
|
|
@@ -314,18 +374,18 @@ and [multilingual-v2 model schema](https://fal.ai/models/fal-ai/elevenlabs/tts/m
|
|
|
314
374
|
## Quick start: say one sentence
|
|
315
375
|
|
|
316
376
|
Use this in a browser application served by `arcane dev`, where the generated
|
|
317
|
-
import map resolves `arcane/
|
|
318
|
-
|
|
377
|
+
import map resolves `arcane-os/ai` and `arcane-os/modules/DBOPFS.js`. These
|
|
378
|
+
browser modules require a browser host. To create an application:
|
|
319
379
|
|
|
320
380
|
```bash
|
|
321
|
-
npx arcane-os@
|
|
381
|
+
npx arcane-os@latest new hello-speech --path ./hello-speech --target browser
|
|
322
382
|
cd hello-speech
|
|
323
383
|
npm install
|
|
324
384
|
npm run dev
|
|
325
385
|
```
|
|
326
386
|
|
|
327
387
|
Keep the generated page's Arcane theme and import map. The examples below go in
|
|
328
|
-
`
|
|
388
|
+
`modules/App.js` at the application root and the adjacent `speech-selection.js`.
|
|
329
389
|
Follow the development server's printed URL. Installing the SDK supplies its
|
|
330
390
|
provider, storage, and Worker code; it does not install a speech model or choose
|
|
331
391
|
an upstream speech runtime for your application.
|
|
@@ -374,9 +434,9 @@ instance. Configuration selects the provider without loading it; the button
|
|
|
374
434
|
explicitly loads and unmutes TTS before requesting speech.
|
|
375
435
|
|
|
376
436
|
```javascript
|
|
377
|
-
import arcaneThemeReady from 'arcane/ThemeBootstrap';
|
|
378
|
-
import AI, { AI_BROWSER_SPEECH_CONFIGURATION_PROTOCOL } from 'arcane/
|
|
379
|
-
import DBOPFS from 'arcane/DBOPFS';
|
|
437
|
+
import arcaneThemeReady from 'arcane-os/modules/ThemeBootstrap.js';
|
|
438
|
+
import AI, { AI_BROWSER_SPEECH_CONFIGURATION_PROTOCOL } from 'arcane-os/ai';
|
|
439
|
+
import DBOPFS from 'arcane-os/modules/DBOPFS.js';
|
|
380
440
|
import { speechSelection } from './speech-selection.js';
|
|
381
441
|
|
|
382
442
|
await arcaneThemeReady;
|
|
@@ -445,7 +505,7 @@ name. It does not load a model or create a GPU device or WebNN context:
|
|
|
445
505
|
```html
|
|
446
506
|
<html-import
|
|
447
507
|
id="browserAISetup"
|
|
448
|
-
href="/arcane/components/browser-ai-setup.html">
|
|
508
|
+
href="/node_modules/arcane-os/runtime/arcane/components/browser-ai-setup.html">
|
|
449
509
|
</html-import>
|
|
450
510
|
```
|
|
451
511
|
|
|
@@ -491,13 +551,11 @@ runtime's evidence to determine actual accelerator execution.
|
|
|
491
551
|
|
|
492
552
|
## Developer diagnostics
|
|
493
553
|
|
|
494
|
-
The shared logging API and speech traces are available in SDK `0.5.14`.
|
|
495
|
-
|
|
496
554
|
Arcane uses the existing shared `user.developer` preference for diagnostic
|
|
497
555
|
logging. Enable **developer mode** in the application's profile settings; the
|
|
498
556
|
logger reads that preference on every emission after the shared user is ready.
|
|
499
|
-
There is no separate speech verbosity
|
|
500
|
-
|
|
557
|
+
There is no separate speech verbosity setting. Ordinary warnings and errors
|
|
558
|
+
remain visible with developer mode disabled.
|
|
501
559
|
|
|
502
560
|
Applications can use the same owner for their complete AI requests and parsed
|
|
503
561
|
responses:
|
|
@@ -534,10 +592,12 @@ that the buffer finished; use its `playback.ended` event.
|
|
|
534
592
|
Diagnostics do not change text, voice, speed, language selection, segmentation,
|
|
535
593
|
generation capacity, or playback scheduling. They stay outside chat history.
|
|
536
594
|
|
|
537
|
-
|
|
595
|
+
<a id="four-synthesis-slots-and-exact-order-playback"></a>
|
|
596
|
+
|
|
597
|
+
## Local synthesis capacity and exact-order playback
|
|
538
598
|
|
|
539
|
-
|
|
540
|
-
wait in the SDK's FIFO queue; they are not dropped. Synthesis may finish out of
|
|
599
|
+
The default Kokoro capacity of 4 means up to four segments synthesize at once.
|
|
600
|
+
Segment 5 and later wait in the SDK's FIFO queue; they are not dropped. Synthesis may finish out of
|
|
541
601
|
order, but playback waits for earlier segments and plays exact input order.
|
|
542
602
|
Each slot owns a Worker/model session, so raising capacity trades memory for
|
|
543
603
|
latency.
|
|
@@ -554,7 +614,7 @@ Use this when your application already has complete segments in an array. The
|
|
|
554
614
|
configured `ai` below is the same instance created in the quick start.
|
|
555
615
|
|
|
556
616
|
```javascript
|
|
557
|
-
import SpeechPlayback from 'arcane/
|
|
617
|
+
import SpeechPlayback from 'arcane-os/speech-playback';
|
|
558
618
|
|
|
559
619
|
const audio = document.body.appendChild(document.createElement('audio'));
|
|
560
620
|
audio.controls = true;
|
|
@@ -575,16 +635,23 @@ speakAll.addEventListener('click', async function speakAllSegments() {
|
|
|
575
635
|
```
|
|
576
636
|
|
|
577
637
|
`prepare()` submits all three parts immediately because this `ai` exposes
|
|
578
|
-
`
|
|
579
|
-
the provider admits up to four synthesis requests and keeps later requests in
|
|
638
|
+
`prepareTTSPlayback` and advertises TTS execution capacity. With the Kokoro
|
|
639
|
+
configuration above, the provider admits up to four synthesis requests and keeps later requests in
|
|
580
640
|
its FIFO queue. Completed audio remains indexed, so playback is always first,
|
|
581
641
|
second, third even if the third synthesis finishes first.
|
|
582
642
|
|
|
643
|
+
`SpeechPlayback` prefers `prepareTTSPlayback()` when present and also supports
|
|
644
|
+
existing audio-only `fetchTTS()` clients. Native browser descriptors and real
|
|
645
|
+
audio Blobs remain in the same ordered sequence, including requested pauses.
|
|
646
|
+
Native segments advance only after actual utterance completion and resource
|
|
647
|
+
release; they are never converted into placeholder audio files.
|
|
648
|
+
|
|
583
649
|
This eager path is capability-driven. A native `Arcane.speech.synthesize`
|
|
584
650
|
client, or a custom client without a positive advertised TTS execution capacity,
|
|
585
651
|
retains serialized synthesis with one lookahead segment. `togglePause()` pauses
|
|
586
|
-
and resumes the same `audio` element.
|
|
587
|
-
playback. `replay()` keeps completed
|
|
652
|
+
and resumes the active native utterance or the same `audio` element.
|
|
653
|
+
`stop()` cancels all requests owned by the playback. `replay()` keeps completed
|
|
654
|
+
and pending provider segments and retries
|
|
588
655
|
only failed missing segments.
|
|
589
656
|
|
|
590
657
|
The default `{device:'auto',maxConcurrentRequests:4}` attempts the full ONNX
|
|
@@ -597,8 +664,6 @@ selected execution device or assess the generated audio.
|
|
|
597
664
|
|
|
598
665
|
## Queue complete passages and wait for playback
|
|
599
666
|
|
|
600
|
-
These options are available in SDK `0.5.12`.
|
|
601
|
-
|
|
602
667
|
For a complete page or passage, call
|
|
603
668
|
`ai.streamTTS(text, true, {voice, speed, pauseAfterMs, waitForPlayback:true})`.
|
|
604
669
|
The existing AI queue owns segmentation, concurrent synthesis, ordered
|
|
@@ -911,8 +976,8 @@ speech method retain their existing behavior.
|
|
|
911
976
|
|
|
912
977
|
## Automatic speech-input formatting cleanup
|
|
913
978
|
|
|
914
|
-
|
|
915
|
-
speech-input copy automatically
|
|
979
|
+
The shared streaming, audio-file, and `SpeechPlayback` paths remove repeated
|
|
980
|
+
same formatting marks from the outbound speech-input copy automatically:
|
|
916
981
|
|
|
917
982
|
```javascript
|
|
918
983
|
ai.streamTTS('## Heading\n**Hello');
|
|
@@ -937,6 +1002,12 @@ Applications omit that internal metadata. Displayed messages, saved history,
|
|
|
937
1002
|
model input, caller payload objects, language, voice, synthesis capacity, and
|
|
938
1003
|
playback timing are not changed by the filter.
|
|
939
1004
|
|
|
1005
|
+
Direct native `provider.prepare()` and `provider.play()` preserve their complete
|
|
1006
|
+
`input` as supplied. The native route of `ai.prepareTTSPlayback()` also preserves
|
|
1007
|
+
that input; shared streaming and `SpeechPlayback` perform their existing cleanup
|
|
1008
|
+
before calling it. Choose the high-level narration path when that speech-only
|
|
1009
|
+
formatting cleanup is desired.
|
|
1010
|
+
|
|
940
1011
|
The shared helper is public for code that needs the speech-only
|
|
941
1012
|
transformation directly:
|
|
942
1013
|
|
|
@@ -954,8 +1025,9 @@ console.log(speechText.append('ing', true)); // ing
|
|
|
954
1025
|
|
|
955
1026
|
## Choose a device or reduce memory use
|
|
956
1027
|
|
|
957
|
-
|
|
958
|
-
`
|
|
1028
|
+
Local Whisper and Kokoro roles accept
|
|
1029
|
+
`execution:{device,maxConcurrentRequests}`. Omitting `stt.execution` selects
|
|
1030
|
+
`{device:'auto',maxConcurrentRequests:1}`; omitting
|
|
959
1031
|
`tts.execution` selects `{device:'auto',maxConcurrentRequests:4}`. Whisper keeps
|
|
960
1032
|
one transcription slot. Kokoro accepts capacities 1 through 4.
|
|
961
1033
|
|
|
@@ -1147,11 +1219,12 @@ records; this contract does not freeze them or shorten their content.
|
|
|
1147
1219
|
|
|
1148
1220
|
| Host | Availability | Notes |
|
|
1149
1221
|
| --- | --- | --- |
|
|
1150
|
-
|
|
|
1222
|
+
| Local browser models | Shipped | Require Workers, Fetch, Blob/File, object URLs, DBOPFS/OPFS, and Web Locks. Blob/File STT requests also require the browser audio decoder. |
|
|
1151
1223
|
| Native WebView | Conditional | Available when the WebView exposes the browser APIs above. It does not invoke Core speech. |
|
|
1152
1224
|
| Node | Remote adapter available; local execution unavailable | TWiN Cloud uses Fetch and Blob. The local providers have no SDK Node speech-storage, Worker, or audio-decoder host. |
|
|
1153
1225
|
| Remote TTS | Explicitly selected | TWiN Cloud uses standard Fetch and Blob with an application-owned key; no Worker, OPFS, or local model is needed for this adapter. |
|
|
1154
1226
|
| Native browser STT | Explicitly selected, conditional | Requires `SpeechRecognition` or `webkitSpeechRecognition` and native microphone/service availability. No Worker, OPFS, or SDK model download; locality is not promised. |
|
|
1227
|
+
| Native browser TTS | Explicitly selected, conditional | Requires `speechSynthesis` and `SpeechSynthesisUtterance`. Uses the native voice catalog and playback lifecycle; no Worker, OPFS, or SDK model download. Native voices expose no audio export. |
|
|
1155
1228
|
|
|
1156
1229
|
STT and TTS own independent provider lifecycles. A failure or cancellation in
|
|
1157
1230
|
one role does not disable the other role or authorize a fallback provider.
|
|
@@ -1164,12 +1237,14 @@ import {
|
|
|
1164
1237
|
BROWSER_SPEECH_ARTIFACT_PROTOCOL,
|
|
1165
1238
|
createBrowserKokoroProvider,
|
|
1166
1239
|
createBrowserSpeechRecognitionProvider,
|
|
1240
|
+
createBrowserSpeechSynthesisProvider,
|
|
1167
1241
|
createBrowserSpeechArtifactGraph,
|
|
1168
1242
|
createBrowserSpeechAuthority,
|
|
1169
1243
|
createBrowserWhisperProvider,
|
|
1170
1244
|
createDbopfsSpeechArtifactStore,
|
|
1171
1245
|
createDigitalOceanFalTTSProvider,
|
|
1172
|
-
createTwinCloudTTSProvider
|
|
1246
|
+
createTwinCloudTTSProvider,
|
|
1247
|
+
removeBrowserSpeechModelCache
|
|
1173
1248
|
} from 'arcane-os/ai/browser-speech';
|
|
1174
1249
|
```
|
|
1175
1250
|
|
|
@@ -1489,7 +1564,7 @@ materialized URLs. `dispose()` performs final teardown and prevents later use.
|
|
|
1489
1564
|
|
|
1490
1565
|
`createBrowserSpeechAuthority({providerId,role,model,runtime,security})` creates
|
|
1491
1566
|
the ordinary single-entrypoint authority. The model descriptor is
|
|
1492
|
-
`{id,repository,revision,dtype?,defaultVoice?,files?}`. The runtime descriptor is
|
|
1567
|
+
`{id,repository,revision,dtype?,defaultVoice?,voices?,files?}`. The runtime descriptor is
|
|
1493
1568
|
`{adapter,version,revision,entry,wasmPaths?,files}`. A file is
|
|
1494
1569
|
`{path,url,mediaType?}`. Model files may be omitted so the selected upstream
|
|
1495
1570
|
provider performs its normal model and voice downloads after explicit use.
|
|
@@ -3,19 +3,19 @@
|
|
|
3
3
|
Use this browser-only entrypoint when an application deliberately owns a local
|
|
4
4
|
GGUF model authority and wants provider-neutral LLM lifecycle, chat, streaming,
|
|
5
5
|
cancellation, and structural tool-call results without an Arcane Core host.
|
|
6
|
-
For
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
6
|
+
For a remote request with an explicit model and key, use
|
|
7
|
+
[TWiN Cloud](twin-cloud.md). Applications using an Arcane Core host can use
|
|
8
|
+
the [native AI contracts](../core/arcane-ai-contracts.md) through
|
|
9
|
+
`globalThis.Arcane.ai`. This page covers local inference in the browser.
|
|
10
10
|
|
|
11
11
|
The wiring example assumes a scaffolded or materialized Arcane application
|
|
12
|
-
using
|
|
13
|
-
`arcane
|
|
12
|
+
using its installed SDK and managed browser import map. Browser modules use
|
|
13
|
+
the published `arcane-os` exports below; DBOPFS still requires a browser host.
|
|
14
14
|
See [browser runtime delivery](../protocols.md#browser-runtime-delivery) before
|
|
15
15
|
using the example in a custom host or bundler.
|
|
16
16
|
|
|
17
17
|
```javascript
|
|
18
|
-
import DBOPFS from 'arcane/DBOPFS';
|
|
18
|
+
import DBOPFS from 'arcane-os/modules/DBOPFS.js';
|
|
19
19
|
import {
|
|
20
20
|
createArcaneAI,
|
|
21
21
|
createBrowserModelSource,
|
|
@@ -106,7 +106,7 @@ selection; a failed GPU load does not silently switch to CPU.
|
|
|
106
106
|
| `ai.llm.chat(request)` / `ai.fetchRequest(request)` | Validated OpenAI-like completion. |
|
|
107
107
|
| `ai.llm.stream(request)` | Mutable async-iterator handle with `result` and `cancel(reason)`. |
|
|
108
108
|
| `ai.streamRequest(request)` | Consumes the stream, publishes every choice's ordinary content/reasoning in provider order, and returns complete JSON text for a multi-choice terminal completion, one structural-call array for selected tool output, or ordinary single-choice text. |
|
|
109
|
-
| `ai.createChatSession(options)` | Asynchronously resolves `Promise<PersistentAIChatSession>` bound to this exact controller; caller-supplied `chat`
|
|
109
|
+
| `ai.createChatSession(options)` | Asynchronously resolves `Promise<PersistentAIChatSession>` bound to this exact controller; caller-supplied `ai` and `chat` are rejected. |
|
|
110
110
|
| `ai.unload()` | Cancels active work, releases the Wllama session, and returns flat unloaded status; the DBOPFS cache remains. |
|
|
111
111
|
| `ai.dispose()` | Permanently disposes the controller; explicit `store.remove(source)` is required to delete cached model content. |
|
|
112
112
|
|
|
@@ -212,7 +212,7 @@ cached entry or rejects with `ARCANE_AI_MODEL_OFFLINE_MISS`.
|
|
|
212
212
|
|
|
213
213
|
```javascript
|
|
214
214
|
const {security, cache} = ai.status().llm;
|
|
215
|
-
console.log(security
|
|
215
|
+
console.log(security?.secure === true); // false when the optional field is absent
|
|
216
216
|
console.log(cache);
|
|
217
217
|
```
|
|
218
218
|
|
|
@@ -254,8 +254,11 @@ the selected hardware. Chromium's optional adapter `type` is a
|
|
|
254
254
|
[developer feature](https://developer.chrome.com/docs/web-platform/webgpu/developer-features);
|
|
255
255
|
the standard API does not always expose integrated or discrete classification.
|
|
256
256
|
The Profile says **Already using the performance GPU.** for an explicitly
|
|
257
|
-
reported discrete GPU
|
|
258
|
-
|
|
257
|
+
reported discrete GPU. Missing classification shows **GPU available.** and
|
|
258
|
+
remains unconfirmed. On supported Windows desktop Chromium browsers, Profile
|
|
259
|
+
keeps its **Copy GPU flag address** control available regardless of adapter
|
|
260
|
+
classification or detection outcome. This Profile control is separate from
|
|
261
|
+
the provider's conditional one-time alert described above.
|
|
259
262
|
The browser does not expose the flag's current value or a reliable machine-wide
|
|
260
263
|
GPU count, and a discrete adapter does not prove it is the fastest available.
|
|
261
264
|
See [Chrome's GPU selection guidance](https://developer.chrome.com/docs/web-platform/webgpu/troubleshooting-tips#webgpu_is_slower_than_webgl).
|
|
@@ -349,7 +352,7 @@ connect its existing display operations without parsing provider protocol:
|
|
|
349
352
|
```javascript
|
|
350
353
|
import {
|
|
351
354
|
formatConversationClosingReportText
|
|
352
|
-
} from '
|
|
355
|
+
} from 'arcane-os/conversation-closing-report';
|
|
353
356
|
|
|
354
357
|
async function streamClosingReport(ai, messages, closingTool, view, signal) {
|
|
355
358
|
return ai.streamRequest(
|
|
@@ -415,10 +418,10 @@ explain why load is available, blocked, or not yet measured without guessing.
|
|
|
415
418
|
`capabilities()` reports browser observations such as WebAssembly, OPFS,
|
|
416
419
|
WebGPU API presence, WebGPU operation, secure context, cross-origin
|
|
417
420
|
isolation, and hardware concurrency. `navigator.gpu` alone is not operational
|
|
418
|
-
evidence. The authoritative runtime can operate without cross-origin isolation
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
operations only while unloaded; it does not download a model.
|
|
421
|
+
evidence. The authoritative runtime can operate without cross-origin isolation.
|
|
422
|
+
Secure context remains required; WebGPU/full-offload support applies to the
|
|
423
|
+
GPU route, while explicit `gpuLayers:0` uses CPU. `probe()` exercises packaged
|
|
424
|
+
Wllama backend operations only while unloaded; it does not download a model.
|
|
422
425
|
|
|
423
426
|
## BROWSER_WASM_RUNTIME_AUTHORITY
|
|
424
427
|
|
|
@@ -427,9 +430,9 @@ operations only while unloaded; it does not download a model.
|
|
|
427
430
|
Mutable metadata for the shipped browser runtime. Its protocol is
|
|
428
431
|
`arcane-ai-browser-wasm/2`; the direct provider uses
|
|
429
432
|
`arcane-ai-adapter/1` and `adaptV1LlmProvider()` projects it into
|
|
430
|
-
`arcane-ai-provider/2`. It records Wllama `3.6.0`,
|
|
431
|
-
|
|
432
|
-
remote-model-helper policy.
|
|
433
|
+
`arcane-ai-provider/2`. It records Wllama `3.6.0`, upstream license metadata,
|
|
434
|
+
packaged runtime asset URLs, execution policy, and the disabled
|
|
435
|
+
compatibility-runtime and remote-model-helper policy.
|
|
433
436
|
|
|
434
437
|
### Value and import
|
|
435
438
|
|
|
@@ -541,9 +544,9 @@ async function loadCachedModelAfterUserChoice() {
|
|
|
541
544
|
`await ai.createChatSession(options)` dynamically creates a
|
|
542
545
|
[`PersistentAIChatSession`](../runtime-modules.md#persistentaichatsessionjs)
|
|
543
546
|
whose AI API is permanently bound to this controller. `options` must
|
|
544
|
-
be a plain object and must not contain `chat`; this prevents a session
|
|
545
|
-
claiming the controller's lifecycle while sending its turns through
|
|
546
|
-
provider.
|
|
547
|
+
be a plain object and must not contain `ai` or `chat`; this prevents a session
|
|
548
|
+
from claiming the controller's lifecycle while sending its turns through
|
|
549
|
+
another provider.
|
|
547
550
|
|
|
548
551
|
## createBrowserModelSource()
|
|
549
552
|
|
|
@@ -728,7 +731,7 @@ console.
|
|
|
728
731
|
### Availability and normalization
|
|
729
732
|
|
|
730
733
|
The adapter is available anywhere the caller can supply a compatible
|
|
731
|
-
`arcane-ai-
|
|
734
|
+
`arcane-ai-adapter/1` provider object. It performs only the versioned
|
|
732
735
|
provider-shape normalization into `arcane-ai-provider/2`. The v1 ingress,
|
|
733
736
|
terminal result, cancellation, and ordinary iterator projection use the same
|
|
734
737
|
structural-message contract; the wrapper does not create a
|
|
@@ -741,7 +744,7 @@ provider and are forwarded through the normalized role contract.
|
|
|
741
744
|
|
|
742
745
|
```javascript
|
|
743
746
|
import {adaptV1LlmProvider} from 'arcane-os/ai/browser-wasm';
|
|
744
|
-
import {getAIProviderRuntime} from 'arcane/
|
|
747
|
+
import {getAIProviderRuntime} from 'arcane-os/ai-provider-runtime';
|
|
745
748
|
|
|
746
749
|
const runtime = getAIProviderRuntime();
|
|
747
750
|
const release = runtime.register(adaptV1LlmProvider(provider));
|
|
@@ -1,12 +1,24 @@
|
|
|
1
|
-
# TWiN Cloud
|
|
1
|
+
# TWiN Cloud requests, images, and state evaluation
|
|
2
2
|
|
|
3
3
|
Use `fetchRequest` from `arcane-os/ai/twin-cloud` for a complete TWiN Cloud
|
|
4
4
|
request in Node or a browser with an explicit key and model. It imports no
|
|
5
5
|
browser profile, DOM, user singleton, or storage, and starts no work on import.
|
|
6
6
|
The existing browser `AI.js` interface remains available for applications that
|
|
7
7
|
already use its provider selection, lifecycle, and speech. TWiN Cloud is that
|
|
8
|
-
interface's default remote LLM service, named `TWIN
|
|
9
|
-
|
|
8
|
+
interface's default remote LLM service, named `TWIN`. Speech is selected
|
|
9
|
+
independently: applications can use local models, native browser speech, or
|
|
10
|
+
[TWiN Cloud TTS](browser-speech.md#twin-cloud-text-to-speech) with an
|
|
11
|
+
application-owned key reader.
|
|
12
|
+
|
|
13
|
+
| Operation | Public API | Caller supplies | Result |
|
|
14
|
+
| --- | --- | --- | --- |
|
|
15
|
+
| Chat completion | `fetchRequest(options)` | Key, model, complete messages, and optional tools or output schema | Complete parsed provider JSON |
|
|
16
|
+
| State and questions | `fetchSystemOneRequest(options)` | Key, a System One model, complete state, and questions | Complete parsed native System One response |
|
|
17
|
+
| Image generation | `generateImages(options)` | Model, prompt, provider parameters, and key or key reader | Every generated image as a Blob in provider order |
|
|
18
|
+
|
|
19
|
+
These functions share no retained conversation. Choose the operation supported
|
|
20
|
+
by the selected model; each section below describes its own transport,
|
|
21
|
+
callbacks, cancellation, and retry behavior.
|
|
10
22
|
|
|
11
23
|
## Node: explicit key, model, and structured result
|
|
12
24
|
|
|
@@ -57,7 +69,7 @@ strict:true, schema:...}}`. `structuredOutput:true` or `'json'` instead selects
|
|
|
57
69
|
## Shared request behavior
|
|
58
70
|
|
|
59
71
|
The focused API accepts complete `messages`, optional `tools`, `toolChoice`,
|
|
60
|
-
`parallelToolCalls`, and `
|
|
72
|
+
`parallelToolCalls`, `reasoningEffort`, and `temperature` in addition to the explicit
|
|
61
73
|
`twinKey` and `model`. Tool options use the existing chat-completion wire fields
|
|
62
74
|
`tools`, `tool_choice`, and `parallel_tool_calls` when `tools` is nonempty.
|
|
63
75
|
A supplied nonempty `reasoningEffort` uses the provider's `reasoning_effort`
|
|
@@ -65,6 +77,13 @@ field; omission preserves its default. No output limit is added by this API.
|
|
|
65
77
|
The function neither executes tools nor adds provider-response envelope
|
|
66
78
|
validation.
|
|
67
79
|
|
|
80
|
+
An optional caller-selected `temperature` is forwarded unchanged, including
|
|
81
|
+
`0`. Omitting it or passing `undefined` leaves the field absent so the selected
|
|
82
|
+
provider uses its own default. The SDK does not clamp it, persist it as a
|
|
83
|
+
preference, or add an output limit. Each request and its retries keep that
|
|
84
|
+
request's value. The browser `AI.fetchRequest` and `AI.streamRequest` object
|
|
85
|
+
APIs support the same option on their built-in TWiN route.
|
|
86
|
+
|
|
68
87
|
Optional `id`, `onRequest(request,id,metadata)`, and
|
|
69
88
|
`onResponse(response,id,false)` follow the complete-response `AI.fetchRequest`
|
|
70
89
|
callback shape. The request callback runs before dispatch; the response
|
|
@@ -331,17 +350,17 @@ The following browser example uses the same managed imports as the
|
|
|
331
350
|
Create an application and start its source server:
|
|
332
351
|
|
|
333
352
|
```bash
|
|
334
|
-
npx arcane-os@
|
|
353
|
+
npx arcane-os@latest new hello-twin --path ./hello-twin --target browser
|
|
335
354
|
cd hello-twin
|
|
336
355
|
npm install
|
|
337
356
|
npm run dev
|
|
338
357
|
```
|
|
339
358
|
|
|
340
359
|
Keep the generated Arcane theme and import map. Place the JavaScript below in
|
|
341
|
-
`
|
|
360
|
+
`modules/App.js` at the application root. Run it in the served browser page.
|
|
342
361
|
This first example is for the new application created above. An existing
|
|
343
362
|
application must complete the saved-preference migration below before importing
|
|
344
|
-
`arcane/
|
|
363
|
+
`arcane-os/ai` or any module that imports it.
|
|
345
364
|
|
|
346
365
|
## Supply the key at runtime and display the response
|
|
347
366
|
|
|
@@ -352,12 +371,12 @@ configuration source; do not put a real key in this module, Git, or diagnostics.
|
|
|
352
371
|
The SDK also reads `globalThis.arcane.config.twinCloud.accessKey` when present.
|
|
353
372
|
|
|
354
373
|
```javascript
|
|
355
|
-
import arcaneThemeReady from 'arcane/ThemeBootstrap';
|
|
374
|
+
import arcaneThemeReady from 'arcane-os/modules/ThemeBootstrap.js';
|
|
356
375
|
|
|
357
376
|
await arcaneThemeReady;
|
|
358
377
|
// In an upgrade bootstrap, the existing preference owner's migration must
|
|
359
378
|
// already be complete before this dynamic import evaluates AI.js.
|
|
360
|
-
const { default: AI } = await import('arcane/
|
|
379
|
+
const { default: AI } = await import('arcane-os/ai');
|
|
361
380
|
const applicationRuntime = globalThis.applicationRuntime;
|
|
362
381
|
const ai = new AI();
|
|
363
382
|
ai.twinKey = applicationRuntime.twinKey;
|
|
@@ -470,7 +489,7 @@ This ordering matters during module evaluation: `AI.js` installs its
|
|
|
470
489
|
user-readiness handler immediately. If `window.user.ready` is already true, it
|
|
471
490
|
can immediately read that user's preference tuple and construct `window.ai`.
|
|
472
491
|
Waiting until a later `setAI()`, provider-startup call, or button click is too
|
|
473
|
-
late. A static `import AI from 'arcane/
|
|
492
|
+
late. A static `import AI from 'arcane-os/ai'` evaluates before the surrounding
|
|
474
493
|
module body, even if its text appears below migration code. Keep AI and its
|
|
475
494
|
importing modules out of the bootstrap's static import graph, finish the
|
|
476
495
|
existing owner's migration, then cross the dynamic-import boundary:
|
|
@@ -478,7 +497,7 @@ existing owner's migration, then cross the dynamic-import boundary:
|
|
|
478
497
|
```javascript
|
|
479
498
|
// Place these lines after the application's existing preference migration
|
|
480
499
|
// has finished writing and exposing migratedTuple, not before that operation.
|
|
481
|
-
const { default: AI } = await import('arcane/
|
|
500
|
+
const { default: AI } = await import('arcane-os/ai');
|
|
482
501
|
const ai = new AI(...migratedTuple);
|
|
483
502
|
```
|
|
484
503
|
|
|
@@ -1,50 +1,46 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Clean browser resource URLs
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
packages. Workspace runtime materialization remains usable by either target;
|
|
9
|
-
the selected browser delivery applies its PWA URL policy.
|
|
3
|
+
The SDK's managed import maps, source server and application packager use clean
|
|
4
|
+
local resource URLs for PWA, non-PWA and native delivery. Their shared
|
|
5
|
+
transformation removes the SDK-owned `arcaneVersion` field. It never appends a
|
|
6
|
+
release version to those URLs. Enabled [PWA delivery](pwa.md) records the
|
|
7
|
+
application and SDK versions in its generated offline manifest and worker.
|
|
10
8
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
For example, an existing `./arcane/modules/HTMLImport.js?v=6#module` reference
|
|
18
|
-
becomes `./arcane/modules/HTMLImport.js?v=6&arcaneVersion=${version}#module`.
|
|
19
|
-
`arcaneVersion` is the SDK's resource version field: transformation updates its
|
|
20
|
-
first existing value and removes duplicate `arcaneVersion` fields, or appends it
|
|
21
|
-
when absent. Regenerating for another SDK release replaces only that SDK value.
|
|
9
|
+
For example,
|
|
10
|
+
`./node_modules/arcane-os/runtime/arcane/modules/HTMLImport.js?v=6&arcaneVersion=0.51.1#module`
|
|
11
|
+
becomes
|
|
12
|
+
`./node_modules/arcane-os/runtime/arcane/modules/HTMLImport.js?v=6#module`.
|
|
13
|
+
Every SDK-owned field is removed, including duplicate and encoded spellings.
|
|
22
14
|
Authored fields, including `v`, encoded keys and values, repeated or empty query
|
|
23
|
-
segments, their source spelling, and fragments remain intact.
|
|
15
|
+
segments, their source spelling, and fragments remain intact. Remote URLs and
|
|
16
|
+
fragment-only references retain their existing value.
|
|
24
17
|
|
|
25
18
|
## Public tooling
|
|
26
19
|
|
|
27
20
|
- Run the installed SDK's `materializeInstalledSdkRuntime()` to refresh the
|
|
28
|
-
workspace runtime from that installation.
|
|
29
|
-
|
|
21
|
+
physical workspace runtime from that installation. It copies the complete
|
|
22
|
+
selected runtime; direct installed-package workspaces read their npm routes
|
|
23
|
+
without requiring this copy.
|
|
30
24
|
- Run `arcane import-map` through the installed SDK to regenerate the managed
|
|
31
25
|
import map and its application HTML. Named SDK imports and URL-shaped module
|
|
32
|
-
entries point to
|
|
26
|
+
entries point to the selected clean resource URLs.
|
|
33
27
|
- `arcane dev` applies the same reference transformation to served source
|
|
34
|
-
without editing the source files. An explicit live SDK source mount
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
- The application packager applies
|
|
38
|
-
adapter finishes. The resulting application can be served by an ordinary
|
|
28
|
+
without editing the source files. An explicit live SDK source mount changes
|
|
29
|
+
the source of those resources. PWA release metadata still identifies the
|
|
30
|
+
selected source or installed SDK version.
|
|
31
|
+
- The application packager applies the clean-URL transformation after its
|
|
32
|
+
selected adapter finishes. The resulting application can be served by an ordinary
|
|
39
33
|
static host without a JavaScript transformation service.
|
|
40
34
|
|
|
41
35
|
The shared transformation edits actual local module imports, resource URL
|
|
42
36
|
construction, HTML resource attributes and CSS resource references. It does not
|
|
43
37
|
change displayed text, prompts, application data, downloaded model content,
|
|
44
38
|
remote provider URLs, or arbitrary strings that happen to resemble filenames.
|
|
45
|
-
Worker entry references and their local module imports are
|
|
46
|
-
|
|
47
|
-
|
|
39
|
+
Worker entry references and their local module imports are handled at their
|
|
40
|
+
own resource boundaries because workers do not inherit a document's import map.
|
|
41
|
+
`HTMLImport` removes the SDK-owned field from same-origin component resource
|
|
42
|
+
references and resolves SDK component resources from the component's actual
|
|
43
|
+
runtime root, including direct installed-package paths.
|
|
48
44
|
|
|
49
45
|
Computed application URLs that have no statically identifiable resource
|
|
50
46
|
reference remain application-owned expressions. The SDK does not intercept
|
|
@@ -55,7 +51,7 @@ an exact matching computed module URL, but is not a wildcard query rule.
|
|
|
55
51
|
|
|
56
52
|
The SDK server sends `Cache-Control: no-cache` for application entry/managed
|
|
57
53
|
HTML and managed import-map JSON. This allows storage but requests revalidation, so an ordinary navigation
|
|
58
|
-
or refresh obtains the current entry document and
|
|
54
|
+
or refresh obtains the current entry document and clean resource URLs. Other assets
|
|
59
55
|
retain ordinary caching; no cache or user storage is cleared.
|
|
60
56
|
|
|
61
57
|
For PWA delivery the SDK server revalidates all served resources, including
|
|
@@ -64,12 +60,13 @@ revalidate stable resource URLs. The service worker maintains its own selected
|
|
|
64
60
|
offline resource cache independently of the HTTP cache.
|
|
65
61
|
|
|
66
62
|
An independently configured static host must likewise revalidate entry HTML
|
|
67
|
-
and managed import-map JSON
|
|
68
|
-
|
|
63
|
+
and managed import-map JSON, and set resource caching appropriate to its delivery
|
|
64
|
+
workflow. The generated package supplies clean resource references; it cannot
|
|
65
|
+
configure another server's HTTP headers.
|
|
69
66
|
|
|
70
|
-
An already-open document keeps modules it has evaluated.
|
|
71
|
-
|
|
72
|
-
instances, force a reload, restart a model, or erase a conversation. The SDK does
|
|
67
|
+
An already-open document keeps modules it has evaluated. Changed files become
|
|
68
|
+
available through ordinary navigation and cache revalidation; they do not replace
|
|
69
|
+
live module instances, force a reload, restart a model, or erase a conversation. The SDK does
|
|
73
70
|
not retain earlier installed runtime trees after materialization.
|
|
74
71
|
|
|
75
72
|
`HTMLImport` registers its element once per browser custom-element registry.
|