arcane-os 0.74.0 → 0.76.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +28 -0
- package/README.md +5 -4
- package/docs/reference/README.md +2 -2
- package/docs/reference/ai/browser-speech.md +8 -4
- package/docs/reference/inventory/package-api.json +1 -1
- package/docs/reference/inventory/runtime-modules.json +3 -3
- package/docs/reference/local-image-editing.md +6 -3
- package/docs/reference/local-image-generation.md +2 -1
- package/docs/reference/local-image-sdxl.md +44 -7
- package/docs/reference/runtime-modules.md +25 -7
- package/docs/reference/sdk-api.md +2 -1
- package/package.json +1 -1
- package/runtime/arcane/modules/SpeechPlayback.js +18 -4
- package/src/local-ai/image-models/sdxl.mjs +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,33 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.76.0
|
|
4
|
+
|
|
5
|
+
- Add public `textFormat:'plain'` to the `SpeechPlayback` constructor and
|
|
6
|
+
`prepare()` options. Preserve every prepared part's exact input through
|
|
7
|
+
audio-file, native and custom speech clients, including queued narration;
|
|
8
|
+
omitted selection retains the existing formatting cleanup.
|
|
9
|
+
- Distinguish a current native utterance's non-error stopped completion from
|
|
10
|
+
initial preparation with `reason:'native-playback-stopped'` on the existing
|
|
11
|
+
`ready` event. Natural final completion remains `ended`; an error does not
|
|
12
|
+
establish native resource release.
|
|
13
|
+
- Preserve per-part voices, ordered playback, cancellation, replay and silent
|
|
14
|
+
preparation. No model, dependency, native host or activation changes are
|
|
15
|
+
included. Focused source fixtures are authored separately from the selected
|
|
16
|
+
package verification; no live speech result is claimed.
|
|
17
|
+
|
|
18
|
+
## 0.75.0
|
|
19
|
+
|
|
20
|
+
- Advertise whole-image `img2img` alongside `txt2img` for the built-in
|
|
21
|
+
`sdxl-base-1.0` selection. Use the existing public `images.edit()` operation
|
|
22
|
+
with the original PNG, complete prompt and explicit strength.
|
|
23
|
+
- Reuse the complete SDXL Base checkpoint, embedded VAE and existing retained
|
|
24
|
+
native context. Preserve 1024-by-1024 defaults, caller-selected canvas,
|
|
25
|
+
progress, cancellation and complete separate PNG results. No additional
|
|
26
|
+
model resource, runtime, adapter or dependency is introduced.
|
|
27
|
+
- Document the pinned engine's encoding/sampling path and whole-image editing
|
|
28
|
+
semantics. Masked/reference editing and pixel or output-quality guarantees
|
|
29
|
+
remain outside this selection; source review is not native execution proof.
|
|
30
|
+
|
|
3
31
|
## 0.74.0
|
|
4
32
|
|
|
5
33
|
- Add optional `longPaths` to `createRepositoryWorkspace()` for newly cloned
|
package/README.md
CHANGED
|
@@ -20,7 +20,7 @@ selected SDK projection or an explicit live SDK source mount. Both profiles
|
|
|
20
20
|
share the theme, packaging, event, cancellation,
|
|
21
21
|
and browser run contracts while retaining their selected application layout.
|
|
22
22
|
|
|
23
|
-
This checkout defines the `0.
|
|
23
|
+
This checkout defines the `0.76.0` SDK contract. Development applications declare
|
|
24
24
|
`arcane-os` as `latest` and retain their lockfile. Exact SDK pins begin when Roshi
|
|
25
25
|
designates a production build. Follow [Consumer updates](docs/consumer-updates.md)
|
|
26
26
|
to refresh and compare the installed SDK and lockfile with the published channel;
|
|
@@ -476,9 +476,10 @@ ordinary repeated punctuation remain literal. Complete synthesis calls may
|
|
|
476
476
|
select `textFormat:'plain'` to preserve exact input: supply it in `prepareTTS`
|
|
477
477
|
options or the payload of `fetchTTS`, provider-runtime TTS requests, or direct
|
|
478
478
|
Kokoro requests. The SDK consumes this control before engine/host dispatch;
|
|
479
|
-
omission keeps the default cleanup.
|
|
480
|
-
|
|
481
|
-
|
|
479
|
+
omission keeps the default cleanup. `SpeechPlayback` accepts the same option in
|
|
480
|
+
its constructor or `prepare()` call and forwards each part's exact input.
|
|
481
|
+
Streaming narration retains its existing formatting behavior. This option does
|
|
482
|
+
not change the selected engine's input capacity or establish complete upstream synthesis. Original
|
|
482
483
|
messages and caller payloads remain unchanged for display, storage, and model
|
|
483
484
|
input. The `arcane-os/speech-text` entrypoint exports the same
|
|
484
485
|
`MarkdownSpeech` streaming class and `stripSpeechFormatting()` one-pass helper
|
package/docs/reference/README.md
CHANGED
|
@@ -72,7 +72,7 @@ alone does not make bare module names resolve in a browser.
|
|
|
72
72
|
| Run application-selected llama.cpp, Ollama or ONNX through Core | [Local AI through Core](local-ai.md) |
|
|
73
73
|
| Generate complete local PNG images with a retained native model | [Local image generation](local-image-generation.md) |
|
|
74
74
|
| Select FLUX.2 Klein 4B and its three complementary model resources | [FLUX.2 Klein 4B through Core](local-image-flux.md) |
|
|
75
|
-
| Select the complete SDXL Base 1.0 checkpoint for
|
|
75
|
+
| Select the complete SDXL Base 1.0 checkpoint for generation and whole-image editing | [SDXL Base 1.0 through Core](local-image-sdxl.md) |
|
|
76
76
|
| Compose independently loaded transcription and synthesis engines | [Native speech service](native-speech.md) |
|
|
77
77
|
| Keep a native Whisper model loaded across complete recordings | [Native Whisper transcription](local-whisper.md) |
|
|
78
78
|
| Select the optional Intel NPU encoder with independent CUDA or CPU decoding | [Intel NPU Whisper](../guides/native-whisper-openvino.md) |
|
|
@@ -82,7 +82,7 @@ alone does not make bare module names resolve in a browser.
|
|
|
82
82
|
| Retain a Nemotron model and diarize caller-fed audio streams | [Native speaker diarization](diarization.md) |
|
|
83
83
|
| Persist application-selected preferences through Core | [Core preferences service](core-preferences.md) |
|
|
84
84
|
| Read complete selected Git text without changing a checkout | [Git text snapshots](git-text-snapshot.md) |
|
|
85
|
-
| Own a native working checkout under per-user ArcaneData/Repos | [Native repository workspaces](core-repositories.md) |
|
|
85
|
+
| Own a native working checkout under per-user ArcaneData/Repos, with optional Windows long paths for new clones | [Native repository workspaces](core-repositories.md) and [Windows clone configuration](core-repositories.md#windows-long-paths-for-a-new-checkout) |
|
|
86
86
|
| Read Git identity defaults or select child-only author and credential hints | [Git identity configuration](core-repositories.md#git-identity-configuration) |
|
|
87
87
|
| Write, commit and push complete selected repository text | [Selected-text repository writing](core-repositories.md#write-commit-and-push-selected-text) |
|
|
88
88
|
| Observe existing checkout configuration and capture an explicit remote/ref | [Repository configuration and selected targets](core-repositories.md#existing-checkout-configuration-and-selected-targets) |
|
|
@@ -1008,8 +1008,11 @@ waits for its next character or the final flush.
|
|
|
1008
1008
|
state. `fetchTTS()`, provider-runtime TTS requests, direct Kokoro provider
|
|
1009
1009
|
requests, and `SpeechPlayback` apply the same default cleanup to complete input.
|
|
1010
1010
|
Complete-input APIs can explicitly select `textFormat:'plain'` as described
|
|
1011
|
-
below.
|
|
1012
|
-
|
|
1011
|
+
below. `SpeechPlayback` accepts `textFormat:'plain'` in its constructor or
|
|
1012
|
+
`prepare()` options and forwards each prepared part's exact input through every
|
|
1013
|
+
supported speech client. Omitted options reuse its current selection; explicit
|
|
1014
|
+
`null` restores default cleanup. The streaming `streamTTS()` options still ignore
|
|
1015
|
+
`textFormat` and streaming narration keeps its existing cleanup.
|
|
1013
1016
|
SDK-internal delegation carries `{speechInputPrepared:true}` outside the speech
|
|
1014
1017
|
payload after its owner selects text treatment, preventing a second
|
|
1015
1018
|
non-idempotent cleanup pass or cleanup of explicitly plain input.
|
|
@@ -1019,8 +1022,9 @@ playback timing are not changed by the filter.
|
|
|
1019
1022
|
|
|
1020
1023
|
Direct native `provider.prepare()` and `provider.play()` preserve their complete
|
|
1021
1024
|
`input` as supplied. The native route of `ai.prepareTTSPlayback()` also preserves
|
|
1022
|
-
that input; shared streaming and `SpeechPlayback` perform
|
|
1023
|
-
|
|
1025
|
+
that input; shared streaming and default `SpeechPlayback` perform cleanup before
|
|
1026
|
+
calling it. Plain `SpeechPlayback` preserves the complete input instead.
|
|
1027
|
+
Choose the high-level narration path when that speech-only
|
|
1024
1028
|
formatting cleanup is desired.
|
|
1025
1029
|
|
|
1026
1030
|
The shared helper is public for code that needs the speech-only
|
|
@@ -1262,11 +1262,11 @@
|
|
|
1262
1262
|
"default",
|
|
1263
1263
|
"splitSpeechText"
|
|
1264
1264
|
],
|
|
1265
|
-
"summary": "Preserves exact nonblank stored segment text; removes repeated formatting marks from
|
|
1265
|
+
"summary": "Preserves exact nonblank stored segment text; removes repeated formatting marks from the outbound synthesis copy by default or forwards exact input with constructor/prepare textFormat:'plain'; eagerly submits complete parts to a capacity-advertising fetchTTS provider while keeping playback indexed; and retains serialized one-segment lookahead for other clients.",
|
|
1266
1266
|
"availability": "Browser + compatible AI/native bridge",
|
|
1267
1267
|
"protocol": "AI.prepareTTSPlayback or AI.fetchTTS plus providerRuntime TTS execution capacity, native provider.prepare, or compatible serialized Arcane.speech.synthesize; globalThis.arcaneEvents, Blob URLs/audio element, and native playback controls",
|
|
1268
|
-
"normalization": "Exact nonblank split/part input strings are preserved in mutable records. Each synthesis request
|
|
1269
|
-
"surface": "prepareTTSPlayback or direct native provider.prepare may return silent native-speech descriptors. autoplay:false stays silent; actual completion advances ordered parts and failed cancellation retains controls until released. Replay preserves completed segments and retries only missing failed generation. SpeechPlayback class/default, mutable `SPEECH_VOICE_OPTIONS` records, mutable `SPEECH_VOICE_ALIASES` membership, `SPEECH_PLAYBACK_STATE_EVENT`, optional constructor `onState(detail)`, `splitSpeechText()`, canonical playback lifecycle events, cancellation, and destroy APIs."
|
|
1268
|
+
"normalization": "Exact nonblank split/part input strings are preserved in mutable records. Each synthesis request copies that input, preserves it exactly for constructor/prepare textFormat:'plain' or applies default formatting cleanup, and carries SDK-internal preparation metadata to prevent a second cleanup pass. A capable provider receives complete segments immediately and owns bounded FIFO admission; URLs and playback stay in exact input order. Native/custom clients remain serialized. Canonical state dispatch precedes the optional synchronous onState callback. Cancellation, Replay recovery, Blob results, and lifecycle state are normalized; provider failures remain external.",
|
|
1269
|
+
"surface": "prepareTTSPlayback or direct native provider.prepare may return silent native-speech descriptors. autoplay:false stays silent; actual completion advances ordered parts and failed cancellation retains controls until released. Current native finished:false without error reports ready with reason native-playback-stopped; natural final completion reports ended. Replay preserves completed segments and retries only missing failed generation. SpeechPlayback class/default, mutable `SPEECH_VOICE_OPTIONS` records, mutable `SPEECH_VOICE_ALIASES` membership, `SPEECH_PLAYBACK_STATE_EVENT`, optional constructor `onState(detail)`, `splitSpeechText()`, canonical playback lifecycle events, cancellation, and destroy APIs."
|
|
1270
1270
|
},
|
|
1271
1271
|
{
|
|
1272
1272
|
"file": "runtime/arcane/modules/StaticDocumentCatalog.js",
|
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
# Local image editing through Core
|
|
2
2
|
|
|
3
3
|
`createCoreImageRuntime()` exposes whole-image img2img through `edit()`, using
|
|
4
|
-
the same retained native context as generation. The
|
|
5
|
-
|
|
4
|
+
the same retained native context as generation. The built-in SD1.4 and
|
|
5
|
+
[SDXL Base 1.0](local-image-sdxl.md#edit-a-whole-image) models advertise
|
|
6
|
+
`txt2img` and `img2img`. Applications supply the original PNG, the
|
|
6
7
|
complete prompt and explicit strength, retain the original, and decide whether
|
|
7
8
|
to save the separate generated result.
|
|
8
9
|
|
|
@@ -149,6 +150,8 @@ own model-specific contracts and are outside this PNG img2img operation.
|
|
|
149
150
|
Ordinary img2img does not require LoRA training. The selected engine documents
|
|
150
151
|
this directly with its base SD1.4 model in the
|
|
151
152
|
[tagged img2img example](https://github.com/leejet/stable-diffusion.cpp/blob/master-929-3f8527a/docs/sd.md#img2img-example).
|
|
153
|
+
The complete SDXL Base checkpoint uses its embedded VAE through the same
|
|
154
|
+
image pipeline; see the [SDXL model and operation guide](local-image-sdxl.md).
|
|
152
155
|
|
|
153
156
|
Source dimensions are decoder output, not an implicit request to change the
|
|
154
157
|
generation dimensions. The engine center-crops and resizes temporary initial
|
|
@@ -157,7 +160,7 @@ Caller images remain unchanged;
|
|
|
157
160
|
even `mode=none` does not suppress all later model adaptation. See the
|
|
158
161
|
[native preprocessing contract](https://github.com/leejet/stable-diffusion.cpp/blob/master-929-3f8527a/docs/image_preprocessing.md).
|
|
159
162
|
|
|
160
|
-
An RGB-only model such as the selected SD1.4 consumes the stored RGB
|
|
163
|
+
An RGB-only model such as the selected SD1.4 or SDXL consumes the stored RGB
|
|
161
164
|
and does not preserve alpha. The SDK must not silently choose a compositing
|
|
162
165
|
background or claim that passing RGBA preserves its displayed appearance.
|
|
163
166
|
Strength zero is also not an exact-copy operation: native img2img still uses
|
|
@@ -50,7 +50,8 @@ editing or imply the same input path as SD1.4 img2img.
|
|
|
50
50
|
`sdxl-base-1.0` selects the complete SDXL Base checkpoint, including its text
|
|
51
51
|
encoders and original VAE. Its [family guide](local-image-sdxl.md) covers the
|
|
52
52
|
single model resource, 1024-by-1024 default canvas and native sampling defaults.
|
|
53
|
-
It advertises `txt2img
|
|
53
|
+
It advertises `txt2img` and whole-image `img2img` through the same public
|
|
54
|
+
`edit()` operation, using the checkpoint's embedded VAE and retained context.
|
|
54
55
|
|
|
55
56
|
The first published distribution path selects CPU on Windows and Linux.
|
|
56
57
|
On macOS, `auto` selects the same upstream universal runtime's built-in Metal
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# SDXL Base 1.0 through Core
|
|
2
2
|
|
|
3
3
|
The `sdxl-base-1.0` model definition selects Stability AI's complete SDXL Base
|
|
4
|
-
1.0 checkpoint for text-to-image generation through the
|
|
5
|
-
[local image runtime](local-image-generation.md). Applications supply their
|
|
4
|
+
1.0 checkpoint for text-to-image generation and whole-image img2img through the
|
|
5
|
+
shared [local image runtime](local-image-generation.md). Applications supply their
|
|
6
6
|
own complete prompts and own the resulting images and saved associations.
|
|
7
7
|
The model uses the shared retained native context, progress, cancellation
|
|
8
8
|
and complete PNG result contract.
|
|
@@ -40,8 +40,8 @@ selection alone does not establish actual execution on any device.
|
|
|
40
40
|
The single `model` resource is the official
|
|
41
41
|
[`sd_xl_base_1.0.safetensors` checkpoint](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/462165984030d82259a11f4367a4eed129e94a7b/sd_xl_base_1.0.safetensors)
|
|
42
42
|
at revision `462165984030d82259a11f4367a4eed129e94a7b`. It includes the text
|
|
43
|
-
encoders, denoiser and original VAE.
|
|
44
|
-
resource; a refiner, separate VAE, LoRA or Python converter is unnecessary for
|
|
43
|
+
encoders, denoiser and original VAE. Generation and whole-image editing use this
|
|
44
|
+
one resource; a refiner, separate VAE, LoRA or Python converter is unnecessary for
|
|
45
45
|
this selection. The upstream model carries the
|
|
46
46
|
[CreativeML Open RAIL++-M license](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/462165984030d82259a11f4367a4eed129e94a7b/LICENSE.md).
|
|
47
47
|
|
|
@@ -107,6 +107,39 @@ Repeated generation reuses the loaded context; unload it when the application
|
|
|
107
107
|
no longer needs it. Follow the shared lifecycle contract for request
|
|
108
108
|
cancellation, state subscriptions, unloading and accessor closure.
|
|
109
109
|
|
|
110
|
+
## Edit a whole image
|
|
111
|
+
|
|
112
|
+
After loading the same model, pass an original PNG and an explicit strength to
|
|
113
|
+
the existing public accessor. `originalPNG` is the application's PNG Blob or
|
|
114
|
+
File; the application retains it and decides whether to save the new result.
|
|
115
|
+
|
|
116
|
+
```javascript
|
|
117
|
+
const edited = await images.edit({
|
|
118
|
+
model: 'sdxl-base-1.0',
|
|
119
|
+
image: originalPNG,
|
|
120
|
+
prompt: 'The octopus librarian now wears a purple knitted diving helmet.',
|
|
121
|
+
strength: 0.6,
|
|
122
|
+
signal
|
|
123
|
+
});
|
|
124
|
+
// Display edited.images using the same complete PNG Blob result contract.
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
This is whole-image img2img, using the checkpoint's embedded VAE encoder,
|
|
128
|
+
denoiser and decoder. It needs no additional model resource, adapter or engine.
|
|
129
|
+
The operation reuses the same loaded context and progress/cancellation owners
|
|
130
|
+
as generation. The SDK forwards the complete prompt and original PNG; it does
|
|
131
|
+
not add style instructions or save either the request or result.
|
|
132
|
+
|
|
133
|
+
The model still defaults to a 1024-by-1024 canvas. Supply
|
|
134
|
+
`parameters: {width, height}` when choosing another output canvas; input
|
|
135
|
+
dimensions do not silently replace that selection. The native engine adapts
|
|
136
|
+
the initial raster to the selected canvas and denoises the whole image.
|
|
137
|
+
Framing, texture, identity and individual pixels are model results, not
|
|
138
|
+
preservation guarantees. Strength zero is not an exact-copy operation, and
|
|
139
|
+
the RGB model does not preserve alpha. See the shared
|
|
140
|
+
[editing contract](local-image-editing.md) for supported PNG encodings,
|
|
141
|
+
native preprocessing, full-result handling and cancellation.
|
|
142
|
+
|
|
110
143
|
## Defaults and operation scope
|
|
111
144
|
|
|
112
145
|
Only canvas dimensions are model-specific defaults here. Sampling parameters
|
|
@@ -127,9 +160,9 @@ applies its own text conditioning and prompt-weighting syntax; see the shared
|
|
|
127
160
|
guide for native string-boundary limitations. The SDK adds no avatar wording,
|
|
128
161
|
style instructions or application-specific content.
|
|
129
162
|
|
|
130
|
-
This definition advertises `txt2img
|
|
131
|
-
|
|
132
|
-
|
|
163
|
+
This definition advertises `txt2img` and `img2img`. Masked and reference-image
|
|
164
|
+
editing are separate operations with their own model and runtime requirements;
|
|
165
|
+
this base selection does not advertise them.
|
|
133
166
|
|
|
134
167
|
The model resource and standalone use follow the
|
|
135
168
|
[official model card](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/462165984030d82259a11f4367a4eed129e94a7b/README.md).
|
|
@@ -137,3 +170,7 @@ Native defaults and embedded-VAE behavior follow the selected engine's
|
|
|
137
170
|
[request implementation](https://github.com/leejet/stable-diffusion.cpp/blob/master-929-3f8527a/src/pipeline/request.cpp),
|
|
138
171
|
[parameter initialization](https://github.com/leejet/stable-diffusion.cpp/blob/master-929-3f8527a/src/stable-diffusion.cpp)
|
|
139
172
|
and [model construction](https://github.com/leejet/stable-diffusion.cpp/blob/master-929-3f8527a/src/pipeline/model_builders.cpp).
|
|
173
|
+
The pinned [image pipeline](https://github.com/leejet/stable-diffusion.cpp/blob/3f8527a/src/pipeline/image.cpp)
|
|
174
|
+
encodes the initial image, samples its latent and decodes the result. That
|
|
175
|
+
implementation establishes the operation path; source inspection alone does
|
|
176
|
+
not establish execution, output quality or device performance.
|
|
@@ -132,7 +132,7 @@ own asynchronous work, cancellation, and backpressure.
|
|
|
132
132
|
| [`ScamRiskPolicy.js`](#scamriskpolicyjs) | esm | Combines deterministic scam signals with optional Arcane blocked-domain evidence and safety guidance. | Cross-host | Complete mutable results; blocked-domain policy requires `secure:true`. |
|
|
133
133
|
| [`ScopedOPFSCache.js`](#scopedopfscachejs) | esm | Provides a narrow exact-key JSON cache inside one app-owned OPFS namespace. | Browser / native WebView | Filename-safe keys, complete JSON values, and malformed-cache cleanup normalized; storage errors mixed. |
|
|
134
134
|
| [`ScreenCapture.js`](#screencapturejs) | esm | Captures a display surface as image, video, or GIF with explicit lifecycle events. | Browser / native WebView | State/events normalized; permission and codec errors mixed. |
|
|
135
|
-
| [`SpeechPlayback.js`](#speechplaybackjs) | esm | Admits complete speech segments to a capacity-advertising provider immediately, retains serialized native/custom lookahead, and plays every result in exact indexed order. | Browser + native bridge | Stored part text stays exact;
|
|
135
|
+
| [`SpeechPlayback.js`](#speechplaybackjs) | esm | Admits complete speech segments to a capacity-advertising provider immediately, retains serialized native/custom lookahead, and plays every result in exact indexed order. | Browser + native bridge | Stored part text stays exact; outbound input receives default formatting cleanup or explicit plain-text preservation; provider/media failures remain mixed. |
|
|
136
136
|
| [`StaticDocumentCatalog.js`](#staticdocumentcatalogjs) | esm | Loads a positive static document inventory with cache, search, and complete context. | Browser / native WebView / server with fetch | Mutable complete catalog/content normalization; malformed data and transport failures remain visible. |
|
|
137
137
|
| [`SystemAppearance.js`](#systemappearancejs) | esm | Reads or applies native appearance, returning an explicit unsupported result when the bridge or optional namespace is absent. | Browser/native hybrid | Exact absent optional namespace normalized with its complete error; partial-method and genuine native errors preserved. |
|
|
138
138
|
| [`SystemPlatformPresentation.js`](#systemplatformpresentationjs) | classic-script | Maps kernel names to presentation labels/classes without granting platform authority. | Browser / native WebView classic script | Fully normalized presentation only. |
|
|
@@ -3973,8 +3973,8 @@ console.log(Object.keys(module));
|
|
|
3973
3973
|
|
|
3974
3974
|
Preserves exact nonblank text, admits complete speech segments according to the
|
|
3975
3975
|
selected client's advertised capacity, and plays indexed HTML audio in exact
|
|
3976
|
-
input order. Stored parts remain exact;
|
|
3977
|
-
|
|
3976
|
+
input order. Stored parts remain exact; each outbound synthesis payload copy
|
|
3977
|
+
receives default formatting-mark cleanup or explicit plain-text preservation.
|
|
3978
3978
|
|
|
3979
3979
|
### Public surface
|
|
3980
3980
|
|
|
@@ -3992,6 +3992,7 @@ new SpeechPlayback({
|
|
|
3992
3992
|
model=null,
|
|
3993
3993
|
voice=null,
|
|
3994
3994
|
responseFormat=null,
|
|
3995
|
+
textFormat=null,
|
|
3995
3996
|
speed=1,
|
|
3996
3997
|
onState=()=>{},
|
|
3997
3998
|
createObjectURL,
|
|
@@ -4006,7 +4007,7 @@ native descriptor, a native provider's silent `prepare(payload,{signal})`,
|
|
|
4006
4007
|
`fetchTTS(payload, signal)`, or `synthesize(payload, {signal})`.
|
|
4007
4008
|
The playback-oriented method is preferred. `SpeechPlayback` also supplies a third
|
|
4008
4009
|
SDK-internal preparation object; existing two-argument clients may ignore it.
|
|
4009
|
-
`prepare({key,parts,model,voice,responseFormat,
|
|
4010
|
+
`prepare({key,parts,model,voice,responseFormat,textFormat,
|
|
4010
4011
|
speed,autoplay=true})` uses only caller-supplied model, voice, and response-format
|
|
4011
4012
|
values; those three omitted values remain omitted so the selected AI/model
|
|
4012
4013
|
catalog may provide its documented defaults. Speed defaults to `1`, is normalized
|
|
@@ -4020,13 +4021,25 @@ There is no hard-coded model, response format, voice, or cloud/browser fallback.
|
|
|
4020
4021
|
the caller's exact string in one mutable array without trimming, splitting, or
|
|
4021
4022
|
freezing it. `prepare()` likewise preserves each nonblank part's exact `input`
|
|
4022
4023
|
string while normalizing its other playback fields into a new mutable record.
|
|
4023
|
-
At synthesis time, `requestSpeech()` copies that record, removes
|
|
4024
|
-
formatting marks from only the outbound `input`, and delegates with the
|
|
4024
|
+
At synthesis time, default `requestSpeech()` behavior copies that record, removes
|
|
4025
|
+
repeated same formatting marks from only the outbound `input`, and delegates with the
|
|
4025
4026
|
SDK-internal `{speechInputPrepared:true}` argument so downstream SDK boundaries
|
|
4026
4027
|
do not apply the non-idempotent filter again. Original part objects, stored
|
|
4027
4028
|
parts, displayed text, and all non-input payload fields remain unchanged. The
|
|
4028
4029
|
class applies no part-count, character-count, pause, or input upper cap.
|
|
4029
4030
|
|
|
4031
|
+
Set `textFormat:'plain'` in the constructor or `prepare()` options to forward
|
|
4032
|
+
every part's exact `input` without formatting cleanup. The selection is captured
|
|
4033
|
+
with the prepared parts, including queued and lookahead requests. A `prepare()`
|
|
4034
|
+
override becomes the instance's selection for later omitted options, like its
|
|
4035
|
+
voice and speed options; explicit `null` or another non-`undefined` value other
|
|
4036
|
+
than `'plain'` selects the existing cleanup. Plain requests carry `textFormat:'plain'` to every
|
|
4037
|
+
supported speech client along with the existing internal preparation marker;
|
|
4038
|
+
default requests keep their existing payload shape. Original part objects stay
|
|
4039
|
+
unchanged, and per-part voices, ordering, cancellation, and silent
|
|
4040
|
+
`autoplay:false` preparation retain their existing behavior. This controls SDK
|
|
4041
|
+
text treatment, not the selected speech engine's pronunciation or input capacity.
|
|
4042
|
+
|
|
4030
4043
|
### Admission and playback order
|
|
4031
4044
|
|
|
4032
4045
|
When `speech` exposes both `fetchTTS(payload, signal)` and
|
|
@@ -4053,6 +4066,11 @@ Replay use the native control; only a successful current native `finished`
|
|
|
4053
4066
|
advances the next part. Native errors remain observable, and a failed native
|
|
4054
4067
|
control stays owned until its optional `released` promise confirms cleanup.
|
|
4055
4068
|
Stale completion after stop/restart/disposal cannot advance replacement content.
|
|
4069
|
+
When current native `finished` resolves `false` without an error, the existing
|
|
4070
|
+
`ready` event carries `reason:'native-playback-stopped'` and its operation ID;
|
|
4071
|
+
initial prepared `ready` has no such reason. Only natural completion of the last
|
|
4072
|
+
part emits `ended`. Errors never establish native resource release; failed
|
|
4073
|
+
controls remain retained through their existing `released` boundary.
|
|
4056
4074
|
Selecting a native part hides and clears the HTML audio source while preserving
|
|
4057
4075
|
the same shared playback state events and complete original parts.
|
|
4058
4076
|
|
|
@@ -4087,7 +4105,7 @@ Exact lifecycle reasons are `playback-replaced`, `playback-stopped`,
|
|
|
4087
4105
|
`playback-destroyed`, `speech-playback-cancelled`,
|
|
4088
4106
|
`speech-synthesis-superseded`, `speech-synthesis-cancelled`,
|
|
4089
4107
|
`speech-synthesizer-unavailable`, `synthesized-audio-contract-mismatch`,
|
|
4090
|
-
`audio-playback-rejected`, `audio-autoplay-rejected`,
|
|
4108
|
+
`audio-playback-rejected`, `audio-autoplay-rejected`, `native-playback-stopped`,
|
|
4091
4109
|
`speech-playback-request-contract-mismatch`, and
|
|
4092
4110
|
`speech-synthesis-rejected`, as applicable to the emitted state.
|
|
4093
4111
|
|
|
@@ -8357,7 +8357,8 @@ mediaType}` and copies the supplied view at acceptance. Its explicit finite
|
|
|
8357
8357
|
`strength` is from `0` through `1`, takes precedence over model/parameter
|
|
8358
8358
|
defaults and is never clamped. It decodes in the existing worker and uses the
|
|
8359
8359
|
same selected context, queue, progress, cancellation and complete result
|
|
8360
|
-
contract. SD1.4
|
|
8360
|
+
contract. SD1.4 and [SDXL Base 1.0](local-image-sdxl.md#edit-a-whole-image)
|
|
8361
|
+
advertise `txt2img` and whole-image `img2img`. See the
|
|
8361
8362
|
[editing contract](local-image-editing.md#core-and-native-request) for supported
|
|
8362
8363
|
PNG input, native preprocessing, alpha and strength-zero semantics; the
|
|
8363
8364
|
original file remains unchanged.
|
package/package.json
CHANGED
|
@@ -42,7 +42,7 @@ function splitSpeechText(value=''){
|
|
|
42
42
|
return text.trim()?[text]:[];
|
|
43
43
|
}
|
|
44
44
|
|
|
45
|
-
function normalizeParts(parts,defaultVoice=null,defaultSpeed=1){
|
|
45
|
+
function normalizeParts(parts,defaultVoice=null,defaultSpeed=1,textFormat=null){
|
|
46
46
|
if(!is.array(parts)||!parts.length){
|
|
47
47
|
throw new TypeError('Narration text cannot be blank. The full visual content remains available.');
|
|
48
48
|
}
|
|
@@ -63,6 +63,7 @@ function normalizeParts(parts,defaultVoice=null,defaultSpeed=1){
|
|
|
63
63
|
if(!is.finite(speed)||speed<=0)throw new RangeError('Every speech segment requires a positive speech speed.');
|
|
64
64
|
return {
|
|
65
65
|
input,
|
|
66
|
+
...(textFormat==='plain'?{textFormat:'plain'}:{}),
|
|
66
67
|
pauseAfterMs,
|
|
67
68
|
role:String(candidate?.role||'narration'),
|
|
68
69
|
speed,
|
|
@@ -425,6 +426,7 @@ class SpeechPlayback{
|
|
|
425
426
|
model=null,
|
|
426
427
|
voice=null,
|
|
427
428
|
responseFormat=null,
|
|
429
|
+
textFormat=null,
|
|
428
430
|
speed=1,
|
|
429
431
|
onState=function noop(){},
|
|
430
432
|
createObjectURL,
|
|
@@ -485,6 +487,7 @@ class SpeechPlayback{
|
|
|
485
487
|
this.responseFormat=responseFormat===null||responseFormat===undefined
|
|
486
488
|
?null
|
|
487
489
|
:String(responseFormat);
|
|
490
|
+
this.textFormat=textFormat==='plain'?'plain':null;
|
|
488
491
|
this.speed=normalizedSpeed;
|
|
489
492
|
this.key='';
|
|
490
493
|
this.state='idle';
|
|
@@ -622,10 +625,13 @@ class SpeechPlayback{
|
|
|
622
625
|
}
|
|
623
626
|
|
|
624
627
|
async requestSpeech(part,signal){
|
|
625
|
-
const input=
|
|
628
|
+
const input=part.textFormat==='plain'
|
|
629
|
+
?part.input
|
|
630
|
+
:stripSpeechFormatting(part.input);
|
|
626
631
|
const preparation={speechInputPrepared:true};
|
|
627
632
|
const payload={
|
|
628
633
|
input,
|
|
634
|
+
...(part.textFormat==='plain'?{textFormat:'plain'}:{}),
|
|
629
635
|
speed:part.speed,
|
|
630
636
|
...(this.model?{model:this.model}:{}),
|
|
631
637
|
...(part.voice?{voice:part.voice}:{}),
|
|
@@ -799,6 +805,7 @@ class SpeechPlayback{
|
|
|
799
805
|
model=this.model,
|
|
800
806
|
voice=this.voice,
|
|
801
807
|
responseFormat=this.responseFormat,
|
|
808
|
+
textFormat=this.textFormat,
|
|
802
809
|
speed=this.speed,
|
|
803
810
|
autoplay=true
|
|
804
811
|
}={}){
|
|
@@ -843,10 +850,12 @@ class SpeechPlayback{
|
|
|
843
850
|
this.model=selectedModel;
|
|
844
851
|
this.voice=selectedVoice;
|
|
845
852
|
this.responseFormat=selectedResponseFormat;
|
|
853
|
+
this.textFormat=textFormat==='plain'?'plain':null;
|
|
846
854
|
normalized=normalizeParts(
|
|
847
855
|
is.function(parts)?parts():parts,
|
|
848
856
|
selectedVoice,
|
|
849
|
-
numericSpeed
|
|
857
|
+
numericSpeed,
|
|
858
|
+
this.textFormat
|
|
850
859
|
);
|
|
851
860
|
this.parts=normalized;
|
|
852
861
|
}catch(error){
|
|
@@ -1036,7 +1045,12 @@ class SpeechPlayback{
|
|
|
1036
1045
|
playback.abortControllers.delete(active.controller);
|
|
1037
1046
|
if(completed===true)return await playback.advance();
|
|
1038
1047
|
if(active.control.error)playback.fail(active.control.error);
|
|
1039
|
-
else playback.emit(
|
|
1048
|
+
else playback.emit(
|
|
1049
|
+
'ready',
|
|
1050
|
+
playback.message('stopped'),
|
|
1051
|
+
playback.key,
|
|
1052
|
+
{reason:'native-playback-stopped'}
|
|
1053
|
+
);
|
|
1040
1054
|
return false;
|
|
1041
1055
|
}catch(error){
|
|
1042
1056
|
console.error('[Arcane speech playback] Native speech failed.',error);
|