arcane-os 0.46.0 → 0.48.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +42 -0
- package/README.md +1 -1
- package/browser-runtime/ai/browser-speech-recognition.mjs +495 -0
- package/browser-runtime/ai/browser-speech.mjs +1 -0
- package/docs/reference/ai/browser-speech.md +80 -2
- package/docs/reference/availability-and-normalization.md +2 -1
- package/docs/reference/inventory/package-api.json +14 -0
- package/docs/reference/runtime-components.md +89 -14
- package/docs/reference/runtime-modules.md +55 -12
- package/docs/reference/sdk-api.md +57 -0
- package/package.json +1 -1
- package/runtime/arcane/components/chat.html +223 -2
- package/runtime/arcane/components/speech.html +263 -14
- package/runtime/arcane/components/voice-transcription.html +77 -10
- package/runtime/arcane/css/document-site.css +2 -35
- package/runtime/arcane/css/print.css +235 -0
- package/runtime/arcane/css/theme.css +2 -40
- package/runtime/arcane/modules/AI.js +15 -4
- package/runtime/arcane/modules/AIProviderRuntime.js +118 -0
- package/runtime/arcane/modules/PrintView.js +19 -55
|
@@ -72,6 +72,7 @@ version; WebKitGTK availability must not be generalized to macOS.
|
|
|
72
72
|
| Select and observe independent LLM/STT/TTS roles | `/arcane/modules/AIProviderRuntime.js` and `AIRuntimeState.js` | **Cross-host** controller/state; registered providers retain their own host requirements | Required/projected provider members, route/configuration records, and status fields; per-role lifecycle, cancellation, stream cleanup, sticky state, and startup barriers are normalized. `localOnly` creates no fallback. |
|
|
73
73
|
| Run a caller-selected local LLM entirely in a browser renderer | `arcane-os/ai/browser-wasm` through `createArcaneAI()` | **Browser** only; secure context, WebAssembly, and OPFS/DBOPFS; full WebGPU offload by default or explicit CPU with `gpuLayers:0` | The public AI API module normalizes multi-model lifecycle, status, complete all-choice streaming, cancellation, exact ordered structural tool-call visibility, and session persistence. Model sources are canonical ordered file descriptors; licenses and model choice remain application policy. |
|
|
74
74
|
| Run caller-selected Whisper or Kokoro in a browser renderer | `arcane-os/ai/browser-speech` registered with `AIProviderRuntime` | **Browser** only; DBOPFS, Web Locks, Workers, Fetch/object URLs, and a caller-supplied self-contained runtime/model closure are required | STT/TTS use independent provider/2 lifecycle and status. Kokoro adds bounded Worker/session concurrency and explicit `auto`, `webgpu`, or `wasm` execution. Complete model/runtime selection, offline behavior, cancellation, Worker teardown, and request/result shapes are normalized. No runtime/model content or cloud fallback is supplied. |
|
|
75
|
+
| Capture speech with the explicitly selected native browser service | `createBrowserSpeechRecognitionProvider` from `arcane-os/ai/browser-speech`, configured as STT | **Browser / native WebView**, conditional on `SpeechRecognition` or `webkitSpeechRecognition`; no SDK model, Worker, or storage required | `localOnly:false` because the browser may use a remote service. Factory/catalog remain usable without native support; inspect/load report availability. User-gesture start, exact final results, transient interim text, stop/drain and abort are owned by the provider/runtime and shared controls. No provider fallback. |
|
|
75
76
|
| Prepare ordered speech playback or reuse the speech-input formatting filter | `arcane-os/speech-playback` and `arcane-os/speech-text` | **Node** with injected media adapters, or **Browser / Native WebView** media; the text filter itself is **Cross-host** | Stored and caller-owned text stays exact. Only the outbound synthesis copy automatically loses repeated same formatting marks. A capacity-advertising provider receives complete segments immediately while retaining indexed playback; native/custom synthesis stays serialized. |
|
|
76
77
|
| Preserve complete chat history and memory | `/arcane/modules/PersistentAIChatSession.js` | **Browser / native WebView** with ChatEntity/DBOPFS and a configured chat function | Existing DBOPFS names and memory semantics are preserved. Live-context commit is atomic; durable persistence is explicit and coherent across user/assistant turns and atomic all-ID tool-result batches. |
|
|
77
78
|
| Search an app-owned document corpus for explicit chat context | `/arcane/modules/DBOPFSDocumentLibrary.js` | **Browser** or compatible injected DBOPFS adapter | Generation/manifest completion, complete lexical search, partial read failures, and untrusted context labels are normalized. Construction does not search; an explicitly wired context builder performs retrieval for each prepared chat send. |
|
|
@@ -79,7 +80,7 @@ version; WebKitGTK availability must not be generalized to macOS.
|
|
|
79
80
|
| Use local AI without coupling app code to Ollama HTTP | `Arcane.localAI`, `Arcane.ai`, or `/arcane/modules/Ollama.js` | Primarily **Native**; Android exposes a narrower admitted inference projection | Admission, errors, and managed-operation events are normalized. Direct Ollama response envelopes remain **Provider-native**. |
|
|
80
81
|
| Use TWiN Cloud from the renderer profile | `/arcane/modules/AI.js` | **Cloud** from an allowed browser/native renderer | High-level chat behavior is normalized by the module. The TWiN access key authenticates remote LLM chat; raw provider diagnostics remain provider-specific. No automatic cloud fallback is inferred from local failure. |
|
|
81
82
|
| Send one TWiN Cloud request with an explicit key and model | `fetchRequest` from `arcane-os/ai/twin-cloud` | **Node** and **Browser**, using standard Fetch and a remote HTTPS provider | Keeps complete messages and returns the full parsed provider JSON. Rejected Fetch and HTTP 529 share three retries after 3000 ms; HTTP 429 overload retains unlimited three-second retries. Optional `onRetry` reports waiting/requesting phases without blocking recovery. Shared structured-output mapping and cancellation match browser TWiN transport. No browser profile, AI/user singleton, DBOPFS, or retained request history is created. |
|
|
82
|
-
| Use speech through one application helper | `/arcane/modules/AI.js` and `Arcane.speech` | **Browser** or **Native** |
|
|
83
|
+
| Use speech through one application helper | `/arcane/modules/AI.js` and `Arcane.speech` | **Browser** or **Native** | Whisper and Kokoro keep their on-device paths; explicitly selected Web Speech STT and TWiN Cloud TTS declare their non-local behavior. The helper automatically cleans only the outbound synthesis-input copy and normalizes application-facing audio/text behavior while request/response plumbing differs below that boundary. |
|
|
83
84
|
| Inspect or manage raw Ollama models | `Arcane.ollama` or `/arcane/modules/Ollama.js` | **Native** desktop Core for management; narrower Android inference only | Wrapper method names, errors, streaming correlation, and admission are Arcane-controlled. Direct Ollama success envelopes are intentionally provider-native. |
|
|
84
85
|
| Use native terminal, installation, user, provisioning, or machine controls | matching `Arcane.*` namespace | **Native** and app/capability restricted | Calls and errors use the common bridge contract. Platform results can be host-specific and are marked in the method guide. |
|
|
85
86
|
|
|
@@ -3134,6 +3134,20 @@
|
|
|
3134
3134
|
"protocol": "arcane-ai-provider/2 tts/synthesize",
|
|
3135
3135
|
"normalization": "Construction and activation make no adapter network requests. SDK initial POSTs bypass the provider-local js-queue; follow-up status/result/audio GETs use followUpQueue capacity and cadence through complete body consumption. Retry-After affects only its job. Whole-job maxConcurrentRequests remains independent. Local Kokoro and Whisper paths are unchanged. Applications own model, voice, credentials, activation, and queue settings."
|
|
3136
3136
|
},
|
|
3137
|
+
{
|
|
3138
|
+
"id": "browser-speech:createBrowserSpeechRecognitionProvider",
|
|
3139
|
+
"name": "createBrowserSpeechRecognitionProvider",
|
|
3140
|
+
"displayName": "createBrowserSpeechRecognitionProvider()",
|
|
3141
|
+
"kind": "function",
|
|
3142
|
+
"signature": "createBrowserSpeechRecognitionProvider({id,model,language}={})",
|
|
3143
|
+
"entrypoints": ["arcane-os/ai/browser-speech"],
|
|
3144
|
+
"primaryImport": "arcane-os/ai/browser-speech",
|
|
3145
|
+
"group": "Browser speech providers",
|
|
3146
|
+
"summary": "Creates an explicitly selected native browser speech recognition provider with gesture-started live capture.",
|
|
3147
|
+
"availability": "Browser or WebView exposing SpeechRecognition or webkitSpeechRecognition; construction remains available on unsupported hosts",
|
|
3148
|
+
"protocol": "arcane-ai-provider/2 stt with optional createCapture",
|
|
3149
|
+
"normalization": "Non-local service declaration, independent STT lifecycle, exact final segments, transient interim results, graceful stop/drain and native abort. No model download, storage, microphone on activation, file-transcription emulation, or automatic fallback."
|
|
3150
|
+
},
|
|
3137
3151
|
{
|
|
3138
3152
|
"id": "browser-speech:createBrowserKokoroProvider",
|
|
3139
3153
|
"name": "createBrowserKokoroProvider",
|
|
@@ -56,13 +56,14 @@ appropriate.
|
|
|
56
56
|
|
|
57
57
|
## Shared print presentation
|
|
58
58
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
scoped
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
59
|
+
The shared `theme.css` and `document-site.css` import `print.css`, the single
|
|
60
|
+
print-style owner. It provides a light paper palette and `1in` (`25.4mm`) page
|
|
61
|
+
margins. During print, root/body/scoped palette tokens override screen colors;
|
|
62
|
+
the shared header/sidebar shell is hidden and its main scroller expands into
|
|
63
|
+
document flow. Screen layout, preferences, fonts, and media remain unchanged.
|
|
64
|
+
[`PrintView.js`](runtime-modules.md#printviewjs) loads the same stylesheet for
|
|
65
|
+
rendered snapshots of Chat, the editor, and file previews. Its temporary
|
|
66
|
+
snapshot remains owned until the browser's `afterprint` event.
|
|
66
67
|
|
|
67
68
|
## Canonical inventory
|
|
68
69
|
|
|
@@ -73,7 +74,7 @@ owner described with the editor and file preview below.
|
|
|
73
74
|
| [`browser-ai-setup.html`](#browser-ai-setuphtml) | Browser API availability, reported GPU class, and copyable NPU/GPU settings addresses. | `refresh()`<br>`checkGpu()`<br>`open()`<br>`destroy()`<br>`ready` | `browser-ai-setup-ready` | Browser-reported adapter class; browser flag state and model execution remain browser-owned |
|
|
74
75
|
| [`calculator.html`](#calculatorhtml) | Calculator keypad and result/error event surface backed by CalculatorEngine. | `calculate()`<br>`destroy()` | `calculator-ready`<br>`calculation-complete`<br>`calculation-error` | Normalized Calculation/error events |
|
|
75
76
|
| [`chart.html`](#charthtml) | Accessible uPlot line, area, or point chart with normalized options and rows. | `configure()`<br>`populate()`<br>`setData()`<br>`addData()`<br>`update()`<br>`destroy()` | `chart-ready`<br>`chart-remove` | Options/rows normalized; uPlot rendering is vendor-native |
|
|
76
|
-
| [`chat.html`](#chathtml) | Shared chat, visible selected-model activation request, file upload, streaming, structural tool settlement, speech, language, availability, and conversation-timebox surface. | `streamMessage()`<br>`setMessageProgress()`<br>`setAIAvailability()`<br>`setInitialSpeechMuted()`<br>`setConversationComplete()`<br>`bindConversationTimebox()`<br>`bindSession()`<br>`submitMessage()`<br>`submitToolResult()`<br>`submitToolResults()`<br>`sendMessage()`<br>`languageChanged()`<br>`requestAIActivation()`<br>`destroy()` | `chat-ready`<br>`chat-session-bound`<br>`chat-session-message`<br>`chat-session-error`<br>`chat-send-message`<br>`chat-send-error`<br>`chat-file-uploaded`<br>`chat-file-upload-error`<br>`chat-language-changed`<br>`chat-language-change-error`<br>`chat-ai-activation-request`<br>`chat-ai-activation-error`<br>`chat-speech-synthesis-error`<br>`conversation-timebox-error` | UI/runtime state, explicit user activation intent, and honest structural-call settlement normalized; AI/storage/media behavior mixed |
|
|
77
|
+
| [`chat.html`](#chathtml) | Shared chat, visible selected-model activation request, file upload, streaming, structural tool settlement, speech, language, availability, and conversation-timebox surface. | `streamMessage()`<br>`setMessageProgress()`<br>`setAIAvailability()`<br>`setInitialSpeechMuted()`<br>`setConversationComplete()`<br>`bindConversationTimebox()`<br>`bindSession()`<br>`submitMessage()`<br>`submitToolResult()`<br>`submitToolResults()`<br>`sendMessage()`<br>`appendDraft()`<br>`print()`<br>`languageChanged()`<br>`requestAIActivation()`<br>`destroy()` | `chat-ready`<br>`chat-session-bound`<br>`chat-session-message`<br>`chat-session-error`<br>`chat-send-message`<br>`chat-send-error`<br>`chat-file-uploaded`<br>`chat-file-upload-error`<br>`chat-language-changed`<br>`chat-language-change-error`<br>`chat-ai-activation-request`<br>`chat-ai-activation-error`<br>`chat-speech-synthesis-error`<br>`conversation-timebox-error` | UI/runtime state, explicit user activation intent, and honest structural-call settlement normalized; AI/storage/media behavior mixed |
|
|
77
78
|
| [`conversation-view.html`](#conversation-viewhtml) | Provider-neutral conversation display, advisory actions, composer, busy state, and status. | `setConversation()`<br>`setBusy()`<br>`setStatus()`<br>`clearComposer()`<br>`destroy()` | `conversation-view-ready`<br>`communication-send`<br>`communication-advisory-action` | DOM-normalized |
|
|
78
79
|
| [`dashboard-config.html`](#dashboard-confightml) | Selects which normalized chart definitions are visible on a dashboard. | `configure()`<br>`setDefinitions()`<br>`setVisibility()`<br>`getChartOptions()`<br>`getEffectiveVisibility()`<br>`open()`<br>`close()`<br>`destroy()` | `dashboard-config-ready`<br>`dashboard-config-opened`<br>`dashboard-config-closed`<br>`dashboard-config-change` | Fully normalized definitions and visibility |
|
|
79
80
|
| [`data-maintenance.html`](#data-maintenancehtml) | Runs destructive cleanup of empty chats and memories inside the current app data scope. | `open()`<br>`destroy()` | `data-maintenance-ready`<br>`data-maintenance-complete` | Normalized counts; DBOPFS failures mixed |
|
|
@@ -98,7 +99,7 @@ owner described with the editor and file preview below.
|
|
|
98
99
|
| [`screen-capture.html`](#screen-capturehtml) | Presents image, video, or GIF display-capture workflow. | `capture` (`ScreenCapture` instance)<br>`destroy()` | `screen-capture-ready`<br>`screen-capture-result` | State/result normalized; media permission/codec failures mixed |
|
|
99
100
|
| [`source-code-viewer.html`](#source-code-viewerhtml) | Renders complete line-addressable source code with load, error, focus, and state behavior. | `configure()`<br>`load()`<br>`render()`<br>`clear()`<br>`fail()`<br>`focus()`<br>`focusLine()`<br>`destroy()` | `source-code-viewer-ready`<br>`source-code-viewer-state`<br>`source-code-viewer-state-loading`<br>`source-code-viewer-state-ready`<br>`source-code-viewer-state-empty`<br>`source-code-viewer-state-error` | Complete mutable source/state |
|
|
100
101
|
| [`source-explanation.html`](#source-explanationhtml) | Presents an evidence finding, source selection, explanation, and save state. | `showFinding()`<br>`populate()`<br>`selectSource()`<br>`markSaved()`<br>`destroy()` | `source-explanation-ready`<br>`source-explanation-save`<br>`source-explanation-source-selected` | DOM-normalized |
|
|
101
|
-
| [`speech.html`](#speechhtml) | Coordinates explicit STT activation, speech controls, transcription completion, mute state, and microphone availability. | `configure()`<br>`setMuted()`<br>`reportTTSError()`<br>`requestSTTActivation()`<br>`destroy()`<br>`availability`<br>`muted`<br>`initialMuted`<br>`componentReady` | `speech-ready`<br>`speech-transcription-complete`<br>`speech-transcription-error`<br>`speech-transcription-cancelled`<br>`speech-microphone-unavailable`<br>`speech-stt-activation-request`<br>`speech-stt-activation-error`<br>`speech-tts-lifecycle-error`<br>`speech-synthesis-error` | Sticky runtime speech readiness, explicit STT activation, request cancellation, TTS mute lifecycle intent, and exact TTS operation failures normalized; provider/model authority remains external |
|
|
102
|
+
| [`speech.html`](#speechhtml) | Coordinates explicit STT activation, speech controls, transcription completion, mute state, and microphone availability. | `configure()`<br>`setMuted()`<br>`reportTTSError()`<br>`requestSTTActivation()`<br>`destroy()`<br>`availability`<br>`muted`<br>`initialMuted`<br>`recognitionLanguage`<br>`displayTranscription`<br>`componentReady` | `speech-ready`<br>`speech-transcription-progress`<br>`speech-transcription-complete`<br>`speech-transcription-error`<br>`speech-transcription-cancelled`<br>`speech-microphone-unavailable`<br>`speech-stt-activation-request`<br>`speech-stt-activation-error`<br>`speech-tts-lifecycle-error`<br>`speech-synthesis-error` | Sticky runtime speech readiness, explicit STT activation, request cancellation, TTS mute lifecycle intent, and exact TTS operation failures normalized; provider/model authority remains external |
|
|
102
103
|
| [`summary-strip.html`](#summary-striphtml) | Displays compact selectable KPI or summary items. | `configure()`<br>`setItems()`<br>`updateItem()`<br>`clear()`<br>`destroy()` | `summary-strip-ready`<br>`summary-strip-change`<br>`summary-strip-select` | DOM-normalized |
|
|
103
104
|
| [`table.html`](#tablehtml) | Builds and updates a simple header/body table. | `buildHeader()`<br>`buildTable()`<br>`destroy()` | `table-ready`<br>`header-update`<br>`body-update` | DOM-normalized |
|
|
104
105
|
| [`task-progress.html`](#task-progresshtml) | Runs and displays a task list with started/change/complete/error state. | `configure()`<br>`setTasks()`<br>`updateTask()`<br>`runTasks()`<br>`clear()`<br>`destroy()` | `task-progress-ready`<br>`task-progress-started`<br>`task-progress-change`<br>`task-progress-complete`<br>`task-progress-error` | Task state normalized; injected task results mixed |
|
|
@@ -317,7 +318,8 @@ Methods/properties: `streamMessage()`, `setMessageProgress()`,
|
|
|
317
318
|
`submitMessage()`, `submitToolResult()`, `submitToolResults()`, `sendMessage()`,
|
|
318
319
|
`languageChanged()`, `requestAIActivation()`, `session`, `sessionStatus`,
|
|
319
320
|
`pendingTool`, `pendingTools`, `pendingToolCall`, `pendingToolCalls`, `modelName`,
|
|
320
|
-
|
|
321
|
+
`transcriptionMode`, `recognitionLanguage`, `appendDraft()`, `print()`,
|
|
322
|
+
`printTitle`, and `destroy()`.
|
|
321
323
|
|
|
322
324
|
`sendMessage(text)` and `languageChanged(text)` are host-overridable async
|
|
323
325
|
extension callbacks. The component installs warning-only defaults when the host
|
|
@@ -331,6 +333,38 @@ editing. The visible Send button remains available on every device. Input
|
|
|
331
333
|
changes resize the composer without submitting; held desktop Enter does not
|
|
332
334
|
submit repeatedly. All input listeners follow the component's existing cleanup.
|
|
333
335
|
|
|
336
|
+
`transcriptionMode` defaults to `'submit'`, preserving completed-transcription
|
|
337
|
+
submission. Set it to `'draft'` for editable voice input with manual Send.
|
|
338
|
+
Native final words and interim hypotheses appear immediately in the existing
|
|
339
|
+
composer; later hypotheses replace only the recognition-owned interim range.
|
|
340
|
+
Typing elsewhere is preserved, and editing that range makes its text user-owned.
|
|
341
|
+
Completion does not append the same final text twice. Cancellation removes only
|
|
342
|
+
an unedited interim hypothesis; final words and user edits remain in the draft.
|
|
343
|
+
The shared Speech status stops repeating transcript text in draft mode.
|
|
344
|
+
`appendDraft(text)` appends complete text without submitting, preserves existing
|
|
345
|
+
draft text and selection, and emits the normal input event. It adds a newline
|
|
346
|
+
between entries only when neither boundary already has whitespace. It returns
|
|
347
|
+
`false` for empty input or a destroyed component.
|
|
348
|
+
|
|
349
|
+
`recognitionLanguage` forwards the Speech component's native BCP47 language
|
|
350
|
+
selection. If omitted, Speech uses the browser language. It is independent of
|
|
351
|
+
the existing conversation-language label and is latched when capture starts.
|
|
352
|
+
|
|
353
|
+
`await print()` delegates the current rendered transcript to shared PrintView;
|
|
354
|
+
`printTitle` optionally supplies the title (default `Conversation`). Controls,
|
|
355
|
+
status, transient thinking, and tool-inspection details are outside the printed
|
|
356
|
+
conversation; complete human-facing tool messages remain. Print uses
|
|
357
|
+
a document-flow transcript layout instead of left/right screen bubbles;
|
|
358
|
+
timestamps keep their own alignment and table headings can repeat across pages.
|
|
359
|
+
Native Print uses
|
|
360
|
+
the same registered view while it is visible and has transcript content. The
|
|
361
|
+
method resolves `true` when the browser print dialog is requested, not when a
|
|
362
|
+
PDF or paper output is produced; it resolves `false` when unavailable or
|
|
363
|
+
cancelled during preparation and rejects preparation failures. Complete errors
|
|
364
|
+
reach developer diagnostics and a concise recovery status stays outside the
|
|
365
|
+
transcript. Destruction cancels pending preparation, while an already requested
|
|
366
|
+
print retains its snapshot through `afterprint`.
|
|
367
|
+
|
|
334
368
|
Before any session binding begins, callback-mode `sessionStatus` follows the
|
|
335
369
|
selected sticky LLM role and the existing application-owned `modelName` label
|
|
336
370
|
(or the selected model ID). Assigning `modelName` refreshes that status even
|
|
@@ -950,7 +984,7 @@ preview ahead of the editor. Printing uses the shared
|
|
|
950
984
|
[`PrintView.js`](runtime-modules.md#printviewjs) owner; accessible HTML frames
|
|
951
985
|
are copied from their current rendered document, never substituted with source
|
|
952
986
|
text. Explicit printing waits for images/fonts and HTML frame readiness.
|
|
953
|
-
The shared print surface uses a light paper palette and `
|
|
987
|
+
The shared print surface uses a light paper palette and `1in` (`25.4mm`) page margins
|
|
954
988
|
while preserving image/SVG colors and leaving the screen theme unchanged.
|
|
955
989
|
Native PDF and inaccessible embedded viewers require their own print command.
|
|
956
990
|
`true` means the browser print dialog was requested, not that paper or a PDF
|
|
@@ -1153,7 +1187,7 @@ Markdown with `entryTitle`, independently of saving. Native browser Print uses
|
|
|
1153
1187
|
that same registered view unless an open file preview takes precedence.
|
|
1154
1188
|
The SDK copies current rendered DOM/styles into a temporary print-only view,
|
|
1155
1189
|
leaving the editor's source, controls, and screen layout in place. Both explicit
|
|
1156
|
-
and native print use the shared light paper palette and `
|
|
1190
|
+
and native print use the shared light paper palette and `1in` (`25.4mm`) page margins;
|
|
1157
1191
|
image pixels and SVG artwork keep their colors. Explicit printing waits for
|
|
1158
1192
|
resource readiness; native `beforeprint` is synchronous
|
|
1159
1193
|
and uses the currently rendered content. `print()` returns `false` when the
|
|
@@ -1629,6 +1663,30 @@ Events: `source-explanation-ready`, `source-explanation-save`, `source-explanati
|
|
|
1629
1663
|
|
|
1630
1664
|
## speech.html
|
|
1631
1665
|
|
|
1666
|
+
An explicitly selected native Web Speech provider uses gesture-started live
|
|
1667
|
+
capture instead of MediaRecorder. The shared control retains every final
|
|
1668
|
+
result and emits its existing completion event once with the exact combined
|
|
1669
|
+
text when native capture ends. The `speech-transcription-progress` event carries
|
|
1670
|
+
cumulative complete final `text` and the current `interim` hypothesis, with the
|
|
1671
|
+
capture's `operationId`. Final segments replace their interim hypothesis before
|
|
1672
|
+
publishing; capture completion clears remaining interim text. The live display
|
|
1673
|
+
retains both while `displayTranscription` is true (the default). Set it false
|
|
1674
|
+
when a containing Chat owns the editable draft; status still reports listening,
|
|
1675
|
+
loading, and errors. Interim hypotheses stay transient. Release/Stop
|
|
1676
|
+
finishes native recognition; cancellation, replacement and destruction abort
|
|
1677
|
+
it and suppress late submission. A persisted page hide stops live native capture
|
|
1678
|
+
without restarting it when the page returns. Existing Whisper and TTS paths
|
|
1679
|
+
remain unchanged.
|
|
1680
|
+
|
|
1681
|
+
On desktop native capture, an accessible editable BCP47 language control occupies
|
|
1682
|
+
the feedback area instead of idle transcription/voice readiness boilerplate.
|
|
1683
|
+
`recognitionLanguage` defaults to `navigator.language`, accepts an explicit
|
|
1684
|
+
language tag without restricting it to the suggested choices, and is passed to
|
|
1685
|
+
the capture owner at start. The control is disabled during active capture so
|
|
1686
|
+
changes affect the next capture. Native voice loading, errors, and speaking
|
|
1687
|
+
remain visible. Model-based transcription retains its existing status, and
|
|
1688
|
+
mobile speech retains its existing voice-output-only presentation.
|
|
1689
|
+
|
|
1632
1690
|
### Overview
|
|
1633
1691
|
|
|
1634
1692
|
Coordinates explicit speech-to-text activation, speech controls, transcription
|
|
@@ -1638,9 +1696,11 @@ completion, mute state, and microphone availability.
|
|
|
1638
1696
|
|
|
1639
1697
|
Methods/properties: `configure()`, `setMuted()`,
|
|
1640
1698
|
`reportTTSError(error,boundary='synthesis')`, `requestSTTActivation()`,
|
|
1641
|
-
`destroy()`, `availability`, `muted`, `initialMuted`,
|
|
1699
|
+
`destroy()`, `availability`, `muted`, `initialMuted`, `recognitionLanguage`,
|
|
1700
|
+
`displayTranscription`, and `componentReady`. The two new properties may also
|
|
1701
|
+
be supplied through `configure()`.
|
|
1642
1702
|
|
|
1643
|
-
Events: `speech-ready`, `speech-transcription-complete`,
|
|
1703
|
+
Events: `speech-ready`, `speech-transcription-progress`, `speech-transcription-complete`,
|
|
1644
1704
|
`speech-transcription-error`, `speech-transcription-cancelled`,
|
|
1645
1705
|
`speech-microphone-unavailable`, `speech-stt-activation-request`,
|
|
1646
1706
|
`speech-stt-activation-error`, `speech-tts-lifecycle-error`, and
|
|
@@ -2083,6 +2143,21 @@ Native methods remain subject to the bound app's capabilities. [Deep protocol de
|
|
|
2083
2143
|
|
|
2084
2144
|
### Continuous capture and controls
|
|
2085
2145
|
|
|
2146
|
+
An explicitly selected native Web Speech provider uses the same ordered
|
|
2147
|
+
transcript/save queue in either capture mode. Each complete final text segment
|
|
2148
|
+
skips file transcription and enters the existing append/save stages. The existing
|
|
2149
|
+
transcript output immediately displays canonical text, queued final segments,
|
|
2150
|
+
and the current interim hypothesis in order. There is no separate interim pane.
|
|
2151
|
+
Interim text never reaches `value`, a save, or a success event. Moving an accepted
|
|
2152
|
+
final segment into canonical text does not display it twice; saved text and
|
|
2153
|
+
retry payloads remain owned by the ordered queue. Native `stopRecording()` returns the drain promise, including final
|
|
2154
|
+
results emitted during native stop. Browser `end` stops capture without an
|
|
2155
|
+
automatic restart. Provider replacement stops native capture while already
|
|
2156
|
+
accepted final segments retain their ordered saves. Explicit Cancel, transcript
|
|
2157
|
+
replacement and destruction retain the existing supersession behavior.
|
|
2158
|
+
This path does not require AudioWorklet or MediaRecorder. A supplied custom
|
|
2159
|
+
`transcribe(file,context)` callback keeps the existing audio-clip route.
|
|
2160
|
+
|
|
2086
2161
|
`configure({capture:{mode:'continuous'}})` opts into the SDK's
|
|
2087
2162
|
[`ContinuousVoiceCapture`](runtime-modules.md#continuousvoicecapturejs) owner.
|
|
2088
2163
|
The default capture settings are `preRollMs:1500`, `quietMs:2000`,
|
|
@@ -275,8 +275,9 @@ tool declarations, emitted tool calls, or callback ordering.
|
|
|
275
275
|
`configureSpeechProviders({stt,tts})` commits only the two speech routes and
|
|
276
276
|
leaves the current LLM route and sticky lifecycle record unchanged. Both speech
|
|
277
277
|
roles must be unloaded and own no request, load, unload, or dispose operation.
|
|
278
|
-
STT remains local-only (`AI_STT_DEVICE_ONLY` for
|
|
279
|
-
|
|
278
|
+
File-based STT remains local-only (`AI_STT_DEVICE_ONLY` for an unsupported
|
|
279
|
+
remote file selection). An explicitly selected native live-capture STT provider
|
|
280
|
+
or TTS provider may declare `localOnly:false`.
|
|
280
281
|
`transitionSpeechProviders({stt,tts})` stops queued audio, explicitly unloads
|
|
281
282
|
only STT and TTS, then commits that same closed speech route record. Neither
|
|
282
283
|
method loads a model, selects a fallback, or changes caller-owned model or voice
|
|
@@ -302,6 +303,13 @@ cancels that pending unmute. The application keeps model, voice, and credential
|
|
|
302
303
|
selection. See [DigitalOcean FAL speech](ai/browser-speech.md#digitalocean-fal-text-to-speech)
|
|
303
304
|
for the public remote adapter and completed-job audio transport.
|
|
304
305
|
|
|
306
|
+
`supportsTranscriptionCapture()` reports whether the selected STT provider
|
|
307
|
+
offers native live capture. `createTranscriptionCapture(options={})` delegates
|
|
308
|
+
to the provider runtime's owned capture boundary below and returns `null` for
|
|
309
|
+
file-only providers. Shared speech controls select it automatically; apps keep
|
|
310
|
+
using their existing controls and completion/save callbacks. See
|
|
311
|
+
[native browser recognition](ai/browser-speech.md#native-browser-speech-recognition).
|
|
312
|
+
|
|
305
313
|
`startProviders({startLanguageModel=true,startMuted=true,startTranscription=false,signal=null}={})`
|
|
306
314
|
starts provider-owned text chat without requesting an STT load by default.
|
|
307
315
|
Callers selecting a browser-WASM LLM pass `startLanguageModel:false` so it
|
|
@@ -961,6 +969,8 @@ The singleton exposes read-only `protocol`, `configured`, and `speechMuted`;
|
|
|
961
969
|
`request(role,options={},preparation={})`;
|
|
962
970
|
`chat(payload,options={})`; `stream(payload,options={})`;
|
|
963
971
|
`transcribe(payload,options={})`;
|
|
972
|
+
`supportsTranscriptionCapture(providerId=null)`;
|
|
973
|
+
`createTranscriptionCapture(options={})`;
|
|
964
974
|
`synthesize(payload,options={},preparation={})`; and
|
|
965
975
|
`setSpeechMuted(muted)`. Provider payloads must be data-only; callbacks,
|
|
966
976
|
accessors, symbols, and cycles are rejected at the provider boundary.
|
|
@@ -1099,6 +1109,24 @@ cleanup are normalized, while the
|
|
|
1099
1109
|
selected provider retains its own capability, permission, download, and model
|
|
1100
1110
|
requirements. [Deep protocol details](protocols.md#portable-ai-provider-runtime).
|
|
1101
1111
|
|
|
1112
|
+
STT providers may additionally implement `createCapture(options)`. Registration
|
|
1113
|
+
retains that optional method without changing the required provider/2 methods.
|
|
1114
|
+
`supportsTranscriptionCapture(providerId=null)` checks the current selection by
|
|
1115
|
+
default or a specified registered STT provider without starting it.
|
|
1116
|
+
`createTranscriptionCapture({language,continuous,onSegment,onInterim,onState,onError})`
|
|
1117
|
+
returns `null` for a provider without that method; otherwise it returns
|
|
1118
|
+
`{start,stop,cancel,destroy,done}`. Creation has no microphone side effect.
|
|
1119
|
+
`start({signal})` requires the same selected ready provider with an idle STT
|
|
1120
|
+
lane and invokes its capture start synchronously in the caller's gesture.
|
|
1121
|
+
Capture occupies one active STT request until its `done` promise settles;
|
|
1122
|
+
ordinary file requests remain queued, and unload/dispose/cancel use the same
|
|
1123
|
+
owned request cancellation lifecycle. Stop drains native final recognition;
|
|
1124
|
+
cancel suppresses late callbacks. Provider callbacks carry exact final
|
|
1125
|
+
`{text,sequence}` segments, transient `{text}` interim results, capture state,
|
|
1126
|
+
or complete errors. The browser provider's `done` resolves a success boolean;
|
|
1127
|
+
runtime cleanup errors remain observable as rejection. No callback is inserted
|
|
1128
|
+
into a data-only transcription request payload.
|
|
1129
|
+
|
|
1102
1130
|
### Example
|
|
1103
1131
|
|
|
1104
1132
|
```javascript
|
|
@@ -3366,12 +3394,14 @@ console.log(typeof prepareSpeech); // function
|
|
|
3366
3394
|
|
|
3367
3395
|
### Overview
|
|
3368
3396
|
|
|
3369
|
-
Shared rendered-print owner used by Markdown Editor and File Manager. It
|
|
3397
|
+
Shared rendered-print owner used by Chat, Markdown Editor, and File Manager. It
|
|
3370
3398
|
captures the current DOM and computed styles, including accessible rendered
|
|
3371
3399
|
HTML frames, complete text, selected images, and the current title. A temporary
|
|
3372
3400
|
print-only surface expands scrollable content without editing the live screen.
|
|
3373
|
-
|
|
3374
|
-
|
|
3401
|
+
The shared `/arcane/css/print.css` owns print presentation for this snapshot,
|
|
3402
|
+
direct themed pages, and documentation sites. It supplies the existing Arcane
|
|
3403
|
+
light palette and `1in` (`25.4mm`) page margins. The snapshot retains rendered
|
|
3404
|
+
fonts, font sizing, line height, emphasis, and complete
|
|
3375
3405
|
content while removing screen-theme text/shadow paint from ordinary HTML and
|
|
3376
3406
|
pseudo-elements. Links are underlined and table cells receive visible borders.
|
|
3377
3407
|
Image pixels, canvas snapshots, and SVG artwork keep their own colors. These
|
|
@@ -3406,6 +3436,13 @@ the developer console receives complete preparation failures.
|
|
|
3406
3436
|
Native printing never invokes the asynchronous `prepare` callback; prepared
|
|
3407
3437
|
views still receive the synchronous content/title read at capture.
|
|
3408
3438
|
|
|
3439
|
+
Import fetches the complete shared stylesheet through a module-relative URL
|
|
3440
|
+
and resolves after that text is available. The editor, file preview, and chat
|
|
3441
|
+
load this module independently of rendering; stylesheet readiness delays only
|
|
3442
|
+
print availability. Every registered view can therefore capture synchronously
|
|
3443
|
+
during native `beforeprint`, with no new stylesheet request at print time.
|
|
3444
|
+
Loading failures reject the module import and remain observable to its caller.
|
|
3445
|
+
|
|
3409
3446
|
`destroy()` unregisters the view and cancels pending preparation. Once the
|
|
3410
3447
|
dialog has been requested, its snapshot and retained media remain until
|
|
3411
3448
|
`afterprint`. Import alone creates no listener, provider, storage, or print job.
|
|
@@ -3415,7 +3452,8 @@ Exact exports: `createPrintView`.
|
|
|
3415
3452
|
### Availability and normalization
|
|
3416
3453
|
|
|
3417
3454
|
**Browser and native WebViews with a print implementation.** Requires DOM,
|
|
3418
|
-
CSSOM, FontFace/Image readiness, and
|
|
3455
|
+
CSSOM, Fetch for initial stylesheet loading, FontFace/Image readiness, and
|
|
3456
|
+
browser print events. No Core capability.
|
|
3419
3457
|
Cross-origin/inaccessible frames, native PDF frames, and object/embed viewers
|
|
3420
3458
|
must use their owning viewer's print command; the helper does not replace their
|
|
3421
3459
|
content with raw source or claim an unrendered document was printed. Font
|
|
@@ -4060,13 +4098,18 @@ The bare root retains the generic neutral light/dark baseline; the prior
|
|
|
4060
4098
|
layout default palette is selected explicitly through `data-user-skin="default"`,
|
|
4061
4099
|
`body.default` compatibility, or the default swatch.
|
|
4062
4100
|
|
|
4063
|
-
|
|
4064
|
-
|
|
4101
|
+
`theme.css` and `document-site.css` import the shared `/arcane/css/print.css`.
|
|
4102
|
+
During printing, that owner applies the existing base light palette, with a
|
|
4103
|
+
white paper surface and `1in` (`25.4mm`) page margins, to the root, body, and scoped
|
|
4065
4104
|
`data-arcane-palette` elements. These print-only tokens override screen/custom
|
|
4066
|
-
palette tokens.
|
|
4067
|
-
|
|
4068
|
-
|
|
4069
|
-
|
|
4105
|
+
palette tokens. During print, it resets root/body margins, hides the shared
|
|
4106
|
+
header/sidebar shell, and expands main-content scrolling into document flow.
|
|
4107
|
+
Saved preferences, screen presentation, fonts, and media remain unchanged.
|
|
4108
|
+
[`PrintView.js`](#printviewjs) loads that same stylesheet before registering
|
|
4109
|
+
views and applies it to its expanded rendered-content snapshot. Snapshot rules
|
|
4110
|
+
are scoped to owned print nodes; documentation-site print rules remain scoped
|
|
4111
|
+
to that site's surface. Screen styles and saved theme preferences remain with
|
|
4112
|
+
their existing owners.
|
|
4070
4113
|
|
|
4071
4114
|
A scoped `data-arcane-palette="warm"` element exposes the same palette variables
|
|
4072
4115
|
for a chooser preview without changing the page's selection. Use any of the
|
|
@@ -169,6 +169,7 @@ browser map are cataloged separately in [Runtime modules](runtime-modules.md).
|
|
|
169
169
|
| `createBrowserSpeechAuthority()` | function | `arcane-os/ai/browser-speech` | Browser speech providers | Browser descriptor construction; use requires the selected storage and provider Web APIs |
|
|
170
170
|
| `createBrowserWasmLlmProvider()` | function | `arcane-os/ai/browser-wasm` | Browser-WASM local AI | Browser context with WebAssembly and OPFS/DBOPFS; WebGPU by default or explicit CPU with gpuLayers:0 |
|
|
171
171
|
| `createBrowserWhisperProvider()` | function | `arcane-os/ai/browser-speech` | Browser speech providers | Browser with Workers, object URLs, caller-selected Whisper runtime/model artifacts, and an SDK DBOPFS speech store |
|
|
172
|
+
| `createBrowserSpeechRecognitionProvider()` | function | `arcane-os/ai/browser-speech` | Browser speech providers | Browser or WebView with native SpeechRecognition; construction and catalog remain available on unsupported hosts |
|
|
172
173
|
| `createCanonicalUstarHeader()` | function | `arcane-os` | Packaging and release bundles | Node |
|
|
173
174
|
| `createDbopfsModelStore()` | function | `arcane-os/ai/browser-wasm` | Browser-WASM local AI | Browser with a ready DBOPFS instance and OPFS |
|
|
174
175
|
| `getBrowserDeviceClass()` | function | `arcane-os/browser-device` | Browser device settings | Node and Browser; synchronous identity hint with no model, storage, or GPU operation |
|
|
@@ -7185,6 +7186,62 @@ async function transcribeAfterUserChoice(audioBlob) {
|
|
|
7185
7186
|
}
|
|
7186
7187
|
```
|
|
7187
7188
|
|
|
7189
|
+
## createBrowserSpeechRecognitionProvider()
|
|
7190
|
+
|
|
7191
|
+
### Overview
|
|
7192
|
+
|
|
7193
|
+
Creates an explicitly selected provider/2 STT adapter for the browser's native
|
|
7194
|
+
`SpeechRecognition` or `webkitSpeechRecognition` service. It declares
|
|
7195
|
+
`localOnly:false` because recognition may use a remote browser service. It does
|
|
7196
|
+
not change Whisper, TTS, typing, stored preferences, or any default selection.
|
|
7197
|
+
|
|
7198
|
+
### Signature and result
|
|
7199
|
+
|
|
7200
|
+
```text
|
|
7201
|
+
createBrowserSpeechRecognitionProvider({id,model,language}={})
|
|
7202
|
+
```
|
|
7203
|
+
|
|
7204
|
+
The application supplies its provider `id` and `model:{id}`; optional `language`
|
|
7205
|
+
is passed to native recognition without changing transcript text. The provider
|
|
7206
|
+
implements the required `catalog`, `inspect`, `status`, `load`, `request`,
|
|
7207
|
+
`unload`, and `dispose` methods plus `createCapture(options)`. There is no SDK
|
|
7208
|
+
model download or storage. Construction/catalog do not require native support;
|
|
7209
|
+
inspect/load report actual browser availability without opening a microphone.
|
|
7210
|
+
File transcription requests are unsupported for this provider.
|
|
7211
|
+
|
|
7212
|
+
`createCapture({language,continuous,onSegment,onInterim,onState,onError})`
|
|
7213
|
+
returns a single-use `{start,stop,cancel,destroy,done}` handle. Start must be
|
|
7214
|
+
called directly from a user gesture and returns a promise for startup success.
|
|
7215
|
+
Final callbacks carry exact `{text,sequence}` values once per final result;
|
|
7216
|
+
interim `{text}` stays transient. Stop preserves pending native final results
|
|
7217
|
+
through `end`; cancellation aborts recognition and suppresses late text.
|
|
7218
|
+
`done` resolves `true` for normal native end and `false` for cancellation or
|
|
7219
|
+
failure. Unload/dispose cancel owned capture.
|
|
7220
|
+
The shared runtime's `createTranscriptionCapture()` additionally owns STT busy
|
|
7221
|
+
state and lifecycle replacement. See the
|
|
7222
|
+
[complete native capture contract](ai/browser-speech.md#native-browser-speech-recognition).
|
|
7223
|
+
|
|
7224
|
+
### Availability and normalization
|
|
7225
|
+
|
|
7226
|
+
Conditional browser/native-WebView support on Windows, Linux, macOS and
|
|
7227
|
+
Android, determined by the actual browser API and permission/service outcome,
|
|
7228
|
+
not by operating-system name. Missing API, denied microphone permission and
|
|
7229
|
+
service errors remain honest STT failures without disabling TTS or typing.
|
|
7230
|
+
No Core, Whisper, or cloud-provider fallback is inferred.
|
|
7231
|
+
|
|
7232
|
+
### Example
|
|
7233
|
+
|
|
7234
|
+
```javascript
|
|
7235
|
+
import {createBrowserSpeechRecognitionProvider} from 'arcane-os/ai/browser-speech';
|
|
7236
|
+
|
|
7237
|
+
async function selectNativeRecognition(ai, signal) {
|
|
7238
|
+
const provider = createBrowserSpeechRecognitionProvider(
|
|
7239
|
+
{id: 'WEB_SPEECH', model: {id: 'WEB_SPEECH'}}
|
|
7240
|
+
);
|
|
7241
|
+
return ai.configureSpeechProvider('stt', provider, {modelId: 'WEB_SPEECH', signal});
|
|
7242
|
+
}
|
|
7243
|
+
```
|
|
7244
|
+
|
|
7188
7245
|
## createTwinCloudTTSProvider()
|
|
7189
7246
|
|
|
7190
7247
|
### Overview
|
package/package.json
CHANGED