@qvac/tts-ggml 0.9.1 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. package/README.md +342 -40
  2. package/docs/pocket-tts.md +201 -0
  3. package/examples/moss-dialogue-tts.js +113 -0
  4. package/examples/moss-tts.js +154 -0
  5. package/examples/pocket-tts.js +51 -0
  6. package/examples/supertonic-chunk-stream-tts.js +156 -0
  7. package/examples/supertonic-sentence-stream-tts.js +4 -4
  8. package/index.d.ts +352 -29
  9. package/index.js +698 -52
  10. package/lib/fit.d.ts +127 -0
  11. package/lib/fit.js +29 -0
  12. package/lib/pocketConfig.d.ts +10 -0
  13. package/lib/pocketConfig.js +145 -0
  14. package/package.json +14 -11
  15. package/test/integration/addon.test.js +5 -0
  16. package/test/integration/audio8.test.js +7 -0
  17. package/test/integration/chatterbox-mtl.test.js +6 -0
  18. package/test/integration/chatterbox-speed.test.js +13 -2
  19. package/test/integration/cosyvoice3-clone.test.js +16 -1
  20. package/test/integration/cosyvoice3-lavasr.test.js +6 -0
  21. package/test/integration/cosyvoice3.test.js +19 -0
  22. package/test/integration/fit.test.js +174 -0
  23. package/test/integration/gpu-smoke.test.js +13 -0
  24. package/test/integration/lavasr-enhancer.test.js +104 -3
  25. package/test/integration/moss.test.js +282 -0
  26. package/test/integration/multiple-runs.test.js +18 -2
  27. package/test/integration/output-sample-rate.test.js +6 -0
  28. package/test/integration/parler-wer.test.js +7 -1
  29. package/test/integration/parler.test.js +27 -3
  30. package/test/integration/pocket.integration.test.js +156 -0
  31. package/test/integration/supertonic-mtl.test.js +2 -0
  32. package/test/integration/supertonic.test.js +70 -0
  33. package/test/integration/supertonic3-quant.test.js +2 -0
  34. package/test/integration/voice-controls.test.js +56 -0
  35. package/test/mobile/integration.auto.cjs +16 -1
  36. package/test/mobile/pocket-worklet-host.cpp +92 -0
  37. package/test/mobile/pocket-worklet.cjs +47 -0
  38. package/test/mobile/test-groups.json +8 -4
  39. package/test/utils/runCosyvoiceTTS.js +1 -0
  40. package/test/utils/runTTS.js +6 -11
  41. package/tts.d.ts +13 -2
  42. package/tts.js +3 -2
package/README.md CHANGED
@@ -7,8 +7,10 @@ v1/v2 still loadable), **Parler** (mini/large English + indic 21-language,
7
7
  description-conditioned with voice/emotion templates), **CosyVoice3**
8
8
  (Fun-CosyVoice3-0.5B, instruct-conditioned, 24 kHz, CPU with opt-in
9
9
  Metal on Apple, CUDA/Vulkan on Linux, Vulkan on Windows, and OpenCL/Adreno
10
- GPU offload on Android), and **Audio8**
11
- (DualAR + neural codec, in-process voice cloning, desktop GPU), plus optional
10
+ GPU offload on Android), **Audio8**
11
+ (DualAR + neural codec, in-process voice cloning, desktop GPU), and **MOSS**
12
+ (MOSS-TTS Delay, 24 kHz, in-process voice cloning, native chunk streaming,
13
+ desktop), plus optional
12
14
  LavaSR neural denoise + 48 kHz bandwidth-extension enhancement. Unsure
13
15
  which checkpoint to stage? Start with [Choosing a model](#choosing-a-model).
14
16
 
@@ -24,7 +26,8 @@ backend per its own per-vendor allowlist (Adreno → OpenCL,
24
26
  Xclipse/Mali → Vulkan). Parler supports Apple/Metal, linux CUDA, and the
25
27
  validated Android paths, including Vulkan on ARM Mali (see
26
28
  [Backends & GPU acceleration](#backends--gpu-acceleration)). Audio8 supports
27
- CUDA/Vulkan offload on Linux and Vulkan on Windows.
29
+ CUDA/Vulkan offload on Linux and Vulkan on Windows. MOSS GPU offload is
30
+ validated on Apple/Metal.
28
31
 
29
32
  [qvac-tts-cpp]: https://github.com/tetherto/qvac-fabric-speech.cpp/tree/master/engines/tts
30
33
 
@@ -44,6 +47,9 @@ CUDA/Vulkan offload on Linux and Vulkan on Windows.
44
47
  `config.useGPU: true` on GPU-capable hosts — including Android, where
45
48
  `tts-cpp` selects the GPU backend per its per-vendor allowlist (see
46
49
  [Backends & GPU acceleration](#backends--gpu-acceleration)).
50
+ - **Apple Core ML sidecars** for the Supertonic vocoder and the Audio8 codec
51
+ on macOS / iOS, picked up when staged next to the model (see
52
+ [Core ML sidecars on Apple](#core-ml-sidecars-on-apple)).
47
53
  - **Dynamic backend loading on Android** — per-arch CPU + Vulkan +
48
54
  OpenCL `.so` files ship under `prebuilds/<bare-target>/qvac__tts-ggml/`
49
55
  and are picked up at runtime via the new `backendsDir` option (see
@@ -66,6 +72,8 @@ obvious for each job.
66
72
  | English + voice cloning + low first-audio latency | `chatterbox-t3-turbo.gguf` + `chatterbox-s3gen.gguf` | Reference-wav / voice-profile cloning; native `streamChunkTokens` chunk streaming. Native 24 kHz. |
67
73
  | Multilingual + voice cloning (EU / CJK) | `chatterbox-t3-mtl.gguf` + `chatterbox-s3gen-mtl.gguf` | en/es/fr/de/pt/it/zh/ja/ko/…; same cloning + streaming surface as Turbo. |
68
74
  | Voice cloning at 44.1 kHz with nothing pre-baked | `audio8-lm-q8_0.gguf` + `audio8-codec-decoder-q8_0.gguf` (+ `audio8-codec-encoder-q8_0.gguf` to clone) | Clones from a reference wav **and its transcript**, encoded in-process — no enrolment step, no voice profile. Whole-utterance only (no native chunk streaming). |
75
+ | Multilingual speech with native chunk streaming and cloning on a desktop | `moss-tts-delay-f16.gguf` + `moss-codec-decoder-f16.gguf` (+ `moss-codec-encoder-f16.gguf` to clone) | 8B backbone, desktop only. Clones from a reference wav encoded in-process (no transcript needed); native `streamChunkTokens` chunk streaming. Native 24 kHz. |
76
+ | Multi-speaker dialogue that continues each speaker's reference voice | `moss-ttsd-f16.gguf` + `moss-codec-decoder-f16.gguf` + `moss-codec-encoder-f16.gguf` | 8B backbone, desktop only. One 24 kHz reference per speaker; the text opens with their transcripts. Native 24 kHz. |
69
77
  | Indic languages | `parler-indic-q8_0.gguf` | 21 Indic languages; voice / emotion templates. Emotion is officially tested on 10 languages — see [Parler descriptions & emotions](#parler-descriptions--emotions) (or the upstream model card). Native 44.1 kHz. |
70
78
  | Chinese dialects (Cantonese, Sichuan, Shanghai, …) | CosyVoice3 dir (`cosyvoice3-llm-*.gguf` + flow / hift / `voice.gguf`) | Instruct-conditioned; 17 dialects via `instruct: { dialect: '…' }`. CPU, with opt-in GPU offload (Metal on Apple, Vulkan on Linux/Windows, OpenCL/Adreno on Android); native 24 kHz. |
71
79
  | Zero-shot / cross-lingual cloning (multilingual) | CosyVoice3 dir + `cosyvoice3-s3tok-*.gguf` + `cosyvoice3-campplus-*.gguf` | Clones from a reference wav; its transcript (`promptText`) selects zero-shot, omitting it selects cross-lingual (timbre only, any target language). Composes with `instruct`. |
@@ -88,6 +96,8 @@ here, so this guide does not go stale when backends change.
88
96
  | `parler-mini-v1-q8_0` | English | ~1.2 GB | 44.1 kHz | Description / templates | Sentence streaming |
89
97
  | CosyVoice3 (`cosyvoice3/`) | Instruct-led (strong on Chinese + dialects) | ~2.3 GB dir (+ ~300 MB to clone) | 24 kHz | Reference wav (zero-shot / cross-lingual) + `instruct` (dialect / emotion / speed / volume / style) | Native chunk opts |
90
98
  | `audio8-lm-q8_0` + codec halves | Text-led (no `language` option) | ~0.7 GB (+ ~120 MB to clone) | 44.1 kHz | Reference wav + transcript, per call too | Sentence streaming |
99
+ | `moss-tts-delay-f16` + codec halves | Multilingual (`language` hint) | ~18.8 GB (+ ~1.8 GB to clone) | 24 kHz | Reference wav, fixed per instance | Sentence + native chunk |
100
+ | `moss-ttsd-f16` + codec halves | Multilingual (`language` hint) | ~20.5 GB | 24 kHz | One reference wav per speaker, fixed per instance | Native chunk |
91
101
 
92
102
  ### Legacy Supertonic v1 / v2
93
103
 
@@ -119,8 +129,9 @@ npm install @qvac/tts-ggml
119
129
  Requires [Bare](https://github.com/holepunchto/bare) `>=1.19.0`.
120
130
 
121
131
  `@qvac/tts-ggml` is a meta package that ships the JavaScript wrapper only.
122
- The native prebuild for each host lives in a version-locked platform package
123
- selected at install time through `os`/`cpu` filtered `optionalDependencies`:
132
+ The native prebuild for each desktop host lives in a version-locked platform
133
+ package selected at install time through `os`/`cpu` filtered
134
+ `optionalDependencies`:
124
135
 
125
136
  | Host | Package |
126
137
  | --- | --- |
@@ -129,12 +140,10 @@ selected at install time through `os`/`cpu` filtered `optionalDependencies`:
129
140
  | darwin-arm64 | `@qvac/tts-ggml-darwin-arm64` |
130
141
  | darwin-x64 | `@qvac/tts-ggml-darwin-x64` |
131
142
  | win32-x64 | `@qvac/tts-ggml-win32-x64` |
132
- | android-arm64 | `@qvac/tts-ggml-android-arm64` |
133
- | ios (device + simulators) | `@qvac/tts-ggml-ios` |
134
143
 
135
- Do not depend on platform packages directly. Supported installers are npm 7+,
136
- pnpm, bun, and Yarn Berry. Yarn v1 and `--omit=optional` installs skip the
137
- platform package and fail at require time with an error naming the missing
144
+ Do not depend on desktop platform packages directly. Supported installers are
145
+ npm 7+, pnpm, bun, and Yarn Berry. Yarn v1 and `--omit=optional` installs skip
146
+ the platform package and fail at require time with an error naming the missing
138
147
  package; a locally built `prebuilds/` directory in the package root always
139
148
  takes precedence. Use `require('@qvac/tts-ggml').resolveBackendsDir()` to
140
149
  locate the directory holding the host's prebuilt binaries and dynamically
@@ -142,10 +151,29 @@ loaded ggml backends. Unsupported targets must
142
151
  [build from source](#build-from-source); installation does not automatically
143
152
  compile a local addon.
144
153
 
154
+ Mobile targets are cross-built, so no install host ever matches their `os`,
155
+ and `optionalDependencies` filtering can never select them. Mobile
156
+ applications must declare the target's platform package as a direct
157
+ dependency, pinned to the exact `@qvac/tts-ggml` version:
158
+
159
+ | Target | Package |
160
+ | --- | --- |
161
+ | android-arm64 | `@qvac/tts-ggml-android-arm64` |
162
+ | ios (device + simulators) | `@qvac/tts-ggml-ios` |
163
+
164
+ ```json
165
+ {
166
+ "dependencies": {
167
+ "@qvac/tts-ggml": "x.y.z",
168
+ "@qvac/tts-ggml-android-arm64": "x.y.z"
169
+ }
170
+ }
171
+ ```
172
+
145
173
  ## Model files
146
174
 
147
- Five engine families are wrapped (Chatterbox, Supertonic, Parler,
148
- CosyVoice3, Audio8), each with its own GGUF layout under `models/`:
175
+ Six engine families are wrapped (Chatterbox, Supertonic, Parler,
176
+ CosyVoice3, Audio8, MOSS), each with its own GGUF layout under `models/`:
149
177
 
150
178
  ```
151
179
  # Chatterbox turbo (English)
@@ -159,6 +187,9 @@ chatterbox-s3gen-mtl.gguf (~1.0 GB)
159
187
  # Supertonic 3 (Supertone/supertonic-3; 31 languages) — preferred Supertonic
160
188
  # checkpoint; published per quant tier (auto-detected from modelDir)
161
189
  supertonic3-q4_0.gguf (~80 MB; also -q8_0 ~126 MB / -f16 ~191 MB / -f32)
190
+ supertonic3-vocoder.mlmodelc/ (optional, macOS / iOS: Apple Core ML sidecar for
191
+ the vocoder, used by the 8-bit and wider tiers; see
192
+ Core ML sidecars on Apple)
162
193
 
163
194
  # Legacy Supertonic (not recommended for new integrations — see Choosing a model)
164
195
  supertonic.gguf (~263 MB) — v1 English only
@@ -177,8 +208,9 @@ parler-indic-q8_0.gguf (~1.3 GB; also -f16 / -f32; 21 Indic languages)
177
208
  # from the cosyvoice3-llm-*.gguf file
178
209
  cosyvoice3/
179
210
  cosyvoice3-llm-*.gguf (~973 MB q8_0 — Qwen2.5 speech LM)
180
- cosyvoice3-flow-*.gguf (~1.3 GB f32 — DiT conditional-flow-matching)
181
- cosyvoice3-hift-*.gguf (~83 MB f32 — CausalHiFT vocoder)
211
+ cosyvoice3-flow-*.gguf (~1.3 GB f32 — DiT conditional-flow-matching; also
212
+ -f16 / -bf16 / -q8_0 / -q4_0)
213
+ cosyvoice3-hift-*.gguf (~83 MB f32 — CausalHiFT vocoder; also -f16)
182
214
  voice.gguf (baked default voice: timbre + prompt tensors)
183
215
  vocab.json merges.txt (Qwen2 BPE tokenizer)
184
216
  cosyvoice3-s3tok-*.gguf (~275 MB q8_0 / ~497 MB f16 — speech tokenizer;
@@ -190,6 +222,17 @@ cosyvoice3/
190
222
  audio8-lm-q8_0.gguf (~0.6 GB; also -f16 / -q4_0 / -f32)
191
223
  audio8-codec-decoder-q8_0.gguf (~110 MB; also -f16 / -f32)
192
224
  audio8-codec-encoder-q8_0.gguf (~120 MB; also -f16 / -f32; cloning only)
225
+ audio8-codec-decoder.mlmodelc/ (optional, macOS / iOS: Apple Core ML sidecar for
226
+ the codec synthesis stack, picked up when it sits
227
+ next to the decoder GGUF; one export serves every
228
+ quant tier)
229
+
230
+ # MOSS (OpenMOSS MOSS-TTS v1.5 Delay; 24 kHz, 8B backbone + RVQ codec); the
231
+ # encoder is only needed to clone a voice or run a dialogue
232
+ moss-tts-delay-f16.gguf (~17 GB)
233
+ moss-ttsd-f16.gguf (~17 GB; MOSS-TTSD dialogue backbone, optional)
234
+ moss-codec-decoder-f16.gguf (~1.8 GB)
235
+ moss-codec-encoder-f16.gguf (~1.8 GB; cloning and dialogue only)
193
236
  ```
194
237
 
195
238
  Download the registry-published Chatterbox, Supertonic, and Parler models into
@@ -201,10 +244,15 @@ npm run download-models:registry -- --group chatterbox,supertonic3
201
244
  npm run download-models:registry -- --output /path/to/models
202
245
  ```
203
246
 
204
- CosyVoice3 and Audio8 are not currently included in that registry command.
205
- Stage their layouts shown above from local converted artifacts. The package
206
- converts upstream Chatterbox, Supertonic, and Parler checkpoints via a Python
207
- venv pipeline:
247
+ CosyVoice3, Audio8 and MOSS are not currently included in that registry command.
248
+ Stage their layouts shown above from local converted artifacts. CosyVoice3
249
+ resolves each component by filename prefix and does not rank quantizations, so
250
+ stage one file per component or name it explicitly with
251
+ `files.cosyvoiceFlowModelPath` / `cosyvoiceHiftModelPath`. Which flow tier is
252
+ fastest depends on the backend — `q8_0` on Vulkan / CUDA, `f16` on Metal,
253
+ `bf16` on AVX512-BF16 CPUs and `f16` on other CPUs — and the `f16` HiFT is the
254
+ recommendation everywhere. The package converts upstream Chatterbox,
255
+ Supertonic, and Parler checkpoints via a Python venv pipeline:
208
256
 
209
257
  ```bash
210
258
  npm run setup-models # creates ./venv, installs requirements.txt, runs convert-models.sh
@@ -220,7 +268,9 @@ npm run convert-models
220
268
  The Audio8 GGUFs are produced by the converters in
221
269
  qvac-fabric-speech.cpp (`engines/tts/scripts/convert-audio8-lm-to-gguf.py`
222
270
  and `convert-audio8-codec-to-gguf.py`) until they are published to the model
223
- registry alongside the other engines.
271
+ registry alongside the other engines. The MOSS GGUFs come from
272
+ `convert-moss-delay-to-gguf.py` and `convert-moss-codec-to-gguf.py` in the same
273
+ directory.
224
274
 
225
275
  Point the addon at a custom location via `files.modelDir` (engine
226
276
  auto-detected from the gguf filenames present), or pass explicit
@@ -229,7 +279,9 @@ auto-detected from the gguf filenames present), or pass explicit
229
279
  `files.cosyvoiceModelDir` (CosyVoice3 — a directory, see
230
280
  [CosyVoice3 instruct](#cosyvoice3-instruct)) /
231
281
  `files.audio8Lm` + `files.audio8CodecDecoder` (+ `files.audio8CodecEncoder`
232
- to clone) (Audio8).
282
+ to clone) (Audio8) /
283
+ `files.mossBackbone` + `files.mossCodecDecoder` (+ `files.mossCodecEncoder`
284
+ to clone) (MOSS).
233
285
 
234
286
  ## Quick start
235
287
 
@@ -302,7 +354,8 @@ Full runnable demo (with streaming playback):
302
354
 
303
355
  Use when you want the fastest possible first-audio-out **within a
304
356
  single utterance**. The C++ engine splits each synthesis into chunks
305
- of `streamChunkTokens` speech tokens (25 ≈ 1 s of audio) and emits
357
+ of `streamChunkTokens` speech tokens (25 ≈ 1 s of audio; on MOSS the unit
358
+ is a codec frame, so 25 ≈ 2 s) and emits
306
359
  audio per chunk, keeping HiFT's source cache phase-continuous across
307
360
  seams so the joins are inaudible.
308
361
 
@@ -327,6 +380,28 @@ await response.onUpdate(data => {
327
380
  Full runnable demo (with gapless playback via `sox` or `ffplay`):
328
381
  `bare examples/chatterbox-chunk-stream-tts.js`
329
382
 
383
+ Long utterances re-synthesize the whole emitted prefix for every chunk by
384
+ default; `streamLeftContextTokens` (typically 25-50) keeps only that many
385
+ tokens of left context per chunk, which bounds the per-chunk cost.
386
+
387
+ Supertonic streams the same way, but its chunks are measured in text tokens
388
+ (Unicode code points; about 50 for English, 25-30 for CJK). The engine snaps
389
+ each boundary to a sentence end, clause or space within
390
+ `streamChunkTolerancePct` of the target and never emits a chunk shorter than
391
+ `streamMinChunkTokens`. Supertonic native streaming cannot be combined with
392
+ the LavaSR enhancer or denoiser; use sentence-level streaming for enhanced
393
+ output.
394
+
395
+ ```js
396
+ const model = new TTSGgml({
397
+ files: { supertonicModel: './models/supertonic3-q8_0.gguf' },
398
+ streamChunkTokens: 50, // text tokens per chunk
399
+ streamFirstChunkTokens: 20 // smaller first chunk
400
+ })
401
+ ```
402
+
403
+ Full runnable demo (with streaming playback): `bare examples/supertonic-chunk-stream-tts.js`
404
+
330
405
  ## Voice cloning
331
406
 
332
407
  ### Chatterbox
@@ -397,6 +472,14 @@ across calls skips the encoder. Per-call fields also ride on the
397
472
  `runStream` / `runStreaming` options, pinned for the whole response so the
398
473
  cache stays hot across chunks.
399
474
 
475
+ ### MOSS
476
+
477
+ MOSS clones from `referenceAudio` alone, with no transcript. The recording
478
+ must be sampled at 24 kHz (there is no resampling; multichannel input is
479
+ downmixed), needs `files.mossCodecEncoder`, and is fixed for the instance.
480
+ For a multi-speaker dialogue, pass one recording per speaker in
481
+ `dialogueReferences` instead. See [MOSS](#moss).
482
+
400
483
  ### CosyVoice3
401
484
 
402
485
  CosyVoice3 clones **zero-shot or cross-lingual**, selected by whether the
@@ -561,6 +644,10 @@ Notes:
561
644
  - Denoiser + native chunk streaming (`streamChunkTokens > 0`) is rejected up
562
645
  front for every engine — a stateful streaming denoiser is the follow-up. Use
563
646
  batch synthesis, or drop the denoiser for streaming.
647
+ - The denoiser follows the engine's resolved device, like the enhancer: a GPU
648
+ engine runs its ggml graph on the GPU (falling back to the ggml CPU backend
649
+ when no GPU backend initialises), a CPU engine keeps the scalar core.
650
+ `denoiserBackendDevice` / `denoiserBackendId` in the stats report where it ran.
564
651
  - The tts-cpp UL-UNAS forward is implemented in
565
652
  [qvac-fabric-speech.cpp#78](https://github.com/tetherto/qvac-fabric-speech.cpp/pull/78)
566
653
  (scalar CPU port, validated bit-close to the ONNX reference); it requires a
@@ -587,6 +674,46 @@ On hosts where more than one backend is usable, `TTS_CPP_GPU_BACKEND`
587
674
  and fails loudly when that backend cannot be resolved; unset (or empty)
588
675
  keeps the automatic preference above.
589
676
 
677
+ ### Core ML sidecars on Apple
678
+
679
+ The macOS / iOS prebuilds carry the Apple Core ML (Neural Engine) sidecars
680
+ for the Supertonic vocoder and the Audio8 codec. They are presence-driven:
681
+ each stage runs on a compiled `.mlmodelc` found next to its model file
682
+ (`supertonic3-q8_0.gguf` -> `supertonic3-vocoder.mlmodelc`) and falls back to
683
+ the ggml graph when it is absent, so a model directory without sidecars
684
+ behaves exactly as before. The sidecar name drops the quantization suffix, so
685
+ one sidecar serves every tier of a model. Sidecars are not part of the
686
+ published model set yet; supply your own to opt in.
687
+
688
+ | Model | Sidecar next to the GGUF | Stage on Core ML | Runs on ggml instead | Force ggml |
689
+ | --- | --- | --- | --- | --- |
690
+ | Supertonic 1 / 2 / 3 | `<model>-vocoder.mlmodelc` | vocoder, in 64-latent-frame windows | GGUFs whose vocoder weights are stored below 8 bits (`q4_0`) | `SUPERTONIC_COREML_DISABLE=1` |
691
+ | Audio8 | `audio8-codec-decoder.mlmodelc`, beside the codec decoder GGUF | codec synthesis stack (upsampling + DAC decoder), in 64-post-frame windows; the language model stays on the ggml backend | a call that fails on the sidecar, which also retires it for every later call on that instance | `AUDIO8_COREML_DISABLE=1` |
692
+ | Chatterbox, Parler, CosyVoice3, MOSS, LavaSR | none | — | always | — |
693
+
694
+ Set the force-ggml variables in the process environment before `load()`.
695
+ Audio8 reports its codec path in `response.stats`: `codecSidecarLoaded` is 1
696
+ while the sidecar is attached and `codecOnCoreml` is 1 when that synthesis
697
+ ran its codec on it (see [Response shape](#response-shape)). The Supertonic
698
+ vocoder path is not reported in the stats.
699
+
700
+ Worth it where the GPU is consumer-class: on an Apple M4 the Supertonic
701
+ vocoder runs 1.6-2.9x faster on the Neural Engine than on Metal (1.06-1.13x
702
+ end to end). On workstation parts the GPU wins — an M3 Ultra is 0.6-0.9x —
703
+ so do not stage a sidecar there. `q4_0` models ignore the vocoder sidecar:
704
+ it carries full-precision weights and would substitute a different vocoder
705
+ rather than accelerate the quantized one.
706
+
707
+ Export the sidecars from the model GGUFs with
708
+ `engines/tts/scripts/export-supertonic-coreml.py` and
709
+ `engines/tts/scripts/export-audio8-codec-coreml.py` from the
710
+ [`qvac-fabric-speech.cpp`](https://github.com/tetherto/qvac-fabric-speech.cpp)
711
+ tree at the ref `speech-cpp` pins. Its
712
+ [Supertonic](https://github.com/tetherto/qvac-fabric-speech.cpp/blob/master/engines/tts/docs/supertonic.md#core-ml-vocoder-sidecar)
713
+ and
714
+ [Audio8](https://github.com/tetherto/qvac-fabric-speech.cpp/blob/master/engines/tts/docs/audio8.md#core-ml-codec-sidecar)
715
+ guides cover export, placement, and measurements.
716
+
590
717
  When the addon is built with `ENABLE_CUDA` — on in the published linux-x64
591
718
  prebuilds, opt-in on linux-arm64 and win32-x64 (`npm run build:cuda` or
592
719
  `bare-make generate -D ENABLE_CUDA=ON`) — the CUDA backend ships as a
@@ -658,7 +785,9 @@ and they are set the same way: the constructor and `reload()` everywhere, plus
658
785
  per call on the engines that can change them per call. The vocabulary is owned
659
786
  by tts-cpp (`include/tts-cpp/voice_controls.h`) and mirrored here; each engine
660
787
  declares the subset it supports, and an unsupported value throws naming that
661
- engine's set.
788
+ engine's set. `TTSGgml.getVoiceControls()` returns tts-cpp's vocabulary and
789
+ each engine's subset straight from the native library, without loading a model
790
+ (`{ emotions, paces, engines: { parler: { emotions, paces }, … } }`).
662
791
 
663
792
  | engine | `emotion` | `pace` | per call | exact rate knob |
664
793
  |---|---|---|---|---|
@@ -667,6 +796,7 @@ engine's set.
667
796
  | Supertonic | not supported | slow / moderate / fast | no | `speed` |
668
797
  | Chatterbox | not supported | not supported | — | `speed` |
669
798
  | Audio8 | not supported | not supported | — | — |
799
+ | MOSS | not supported | not supported | — | — |
670
800
 
671
801
  The 12 canonical emotions (case-insensitive): `command`, `anger`, `narration`,
672
802
  `conversation`, `disgust`, `fear`, `happy`, `neutral`, `proper noun`, `news`,
@@ -801,6 +931,89 @@ available, the engine falls back to CPU and sets
801
931
  `response.stats.gpuUnsupported`. Voice cloning uses the same backend and adds
802
932
  a one-off encode when a new reference recording is supplied.
803
933
 
934
+ ## MOSS
935
+
936
+ MOSS Delay is an autoregressive model that predicts 32 RVQ codebooks per 80 ms
937
+ frame on a delay pattern, and a transformer codec turns those codes back into
938
+ 24 kHz audio. The backbone and the codec's synthesis half run on every
939
+ synthesis, the analysis half only to clone a voice or run a dialogue, so a
940
+ text-only deployment can omit `files.mossCodecEncoder`. The backbones have 8B
941
+ parameters, so MOSS targets desktop hosts; mobile is not supported.
942
+
943
+ ```js
944
+ const model = new TTSGgml({
945
+ engine: TTSGgml.ENGINE_MOSS,
946
+ files: { modelDir: './models' },
947
+ referenceAudio: './voice.wav', // optional; 24 kHz, needs moss-codec-encoder-*.gguf
948
+ streamChunkTokens: 25, // optional; codec frames per chunk (2 s)
949
+ config: { language: 'en' },
950
+ opts: { stats: true }
951
+ })
952
+ await model.load()
953
+ await model.run({ input: 'Hello from a fully on-device pipeline.' })
954
+ ```
955
+
956
+ `config.language` is passed to the model as a prompt hint. The reference
957
+ recording is encoded once at load and stays fixed for the instance, so
958
+ changing voices means a new instance. `streamChunkTokens > 0` delivers audio in
959
+ chunks of that many codec frames (12.5 per second) while the backbone is still
960
+ generating; `streamFirstChunkTokens` is not supported. MOSS emits 24 kHz only,
961
+ so `config.outputSampleRate` is rejected unless it equals `24000`, and the
962
+ LavaSR enhancer / denoiser are not supported.
963
+
964
+ The speech is directable from the text itself: `[pause 2.0s]` markers insert
965
+ a silence of roughly that length, and inline Pinyin (`ni3 hao3`) or IPA
966
+ (`/həloʊ/`) steers pronunciation. `durationTokens` asks for a target length
967
+ in codec frames (12.5 per second, so `38` is about 3 s; `0` keeps the length
968
+ free). It is reloadable, and the engine rejects a target that does not fit in
969
+ its generation budget.
970
+
971
+ ```js
972
+ const model = new TTSGgml({
973
+ engine: TTSGgml.ENGINE_MOSS,
974
+ files: { modelDir: './models' },
975
+ durationTokens: 38,
976
+ config: { language: 'en' }
977
+ })
978
+ await model.load()
979
+ await model.run({ input: 'Hold on [pause 1.0s] here it comes.' })
980
+ ```
981
+
982
+ Dialogue needs the MOSS-TTSD backbone: with `dialogueReferences` set, a
983
+ `modelDir` must hold `moss-ttsd-*.gguf` (or name the file with
984
+ `files.mossBackbone`). A `modelDir` holding only the TTSD backbone also serves
985
+ plain synthesis. `dialogueReferences` takes one 24 kHz recording per
986
+ speaker, in the order the text tags them with `[S1]`, `[S2]`, and so on. The
987
+ model continues the references, so the text must open with what each
988
+ recording says, under its speaker tag, followed by the lines to generate; only
989
+ the new lines come out as audio. For the same reason a dialogue cannot be split
990
+ into sentences: `runStream()`, `runStreaming()` and `run({ streamOutput: true })`
991
+ are rejected; use `run()`, with `streamChunkTokens` for chunked audio. The references need
992
+ `files.mossCodecEncoder`, exclude `referenceAudio`, and are fixed for the
993
+ instance. They stay in the codec as causal history, so the first generated
994
+ words continue the reference voices without a seam.
995
+
996
+ ```js
997
+ const dialogue = new TTSGgml({
998
+ engine: TTSGgml.ENGINE_MOSS,
999
+ files: { modelDir: './models' },
1000
+ dialogueReferences: ['./alice.wav', './bob.wav'],
1001
+ config: { language: 'en' }
1002
+ })
1003
+ await dialogue.load()
1004
+ await dialogue.run({
1005
+ input:
1006
+ '[S1] What alice.wav says. [S2] What bob.wav says. ' +
1007
+ '[S1] Did the build finish? [S2] Yes, every test passed.'
1008
+ })
1009
+ ```
1010
+
1011
+ MOSS runs on CPU by default. `config.useGPU: true` (or a non-zero
1012
+ `nGpuLayers`) asks the engine for a GPU backend; when none is available it
1013
+ falls back to CPU and sets `response.stats.gpuUnsupported`. `backendsDir`
1014
+ reaches MOSS like every other engine, so builds that load ggml backends at
1015
+ runtime find them.
1016
+
804
1017
  ## API overview
805
1018
 
806
1019
  ### Constructor — `new TTSGgml(options)`
@@ -816,47 +1029,65 @@ a one-off encode when a new reference recording is supplied.
816
1029
  | `files.cosyvoiceS3tokModel` | string | — | CosyVoice3 speech_tokenizer_v3 GGUF; needed only with `referenceAudio` (auto-discovered in the model dir as `cosyvoice3-s3tok-*.gguf`) |
817
1030
  | `files.cosyvoiceCampplusModel` | string | — | CosyVoice3 CAM++ speaker-encoder GGUF; needed only with `referenceAudio` (auto-discovered as `cosyvoice3-campplus-*.gguf`) |
818
1031
  | `files.cosyvoiceLlmModelPath` / `cosyvoiceFlowModelPath` / `cosyvoiceHiftModelPath` | string | — | Per-component overrides for the CosyVoice3 model dir |
1032
+ | `files.cosyvoiceVocab` / `cosyvoiceMerges` / `cosyvoiceVoiceModel` | string | — | CosyVoice3 text frontend (`vocab.json`, `merges.txt`) and baked voice (`voice.gguf`) overrides; each resolves under the model dir when unset. `cosyvoiceVoiceModel` selects a different baked voice without a separate model dir (`*Path` aliases accepted) |
819
1033
  | `files.audio8Lm` | string | — | Audio8 DualAR language model GGUF (overrides `modelDir`) |
820
1034
  | `files.audio8CodecDecoder`| string | — | Audio8 codec synthesis half — codes to wav (overrides `modelDir`) |
821
1035
  | `files.audio8CodecEncoder`| string | — | Audio8 codec analysis half — wav to codes; only needed to clone a voice |
1036
+ | `files.mossBackbone` | string | — | MOSS Delay backbone GGUF: MOSS-TTS or the MOSS-TTSD dialogue checkpoint (overrides `modelDir`) |
1037
+ | `files.mossCodecDecoder` | string | — | MOSS codec synthesis half — codes to wav (overrides `modelDir`) |
1038
+ | `files.mossCodecEncoder` | string | — | MOSS codec analysis half — wav to codes; only needed to clone a voice or for `dialogueReferences` |
822
1039
  | `files.lavasrEnhancer` | string | — | LavaSR enhancer GGUF — supplying it turns on 48 kHz enhancement |
823
1040
  | `files.lavasrDenoiser` | string | — | LavaSR denoiser GGUF — supplying it turns on denoising (batch only) |
824
- | `engine` | string | auto | Force `'chatterbox'`, `'supertonic'`, `'cosyvoice3'`, `'parler'` or `'audio8'` (`TTSGgml.ENGINE_CHATTERBOX` / `ENGINE_SUPERTONIC` / `ENGINE_COSYVOICE3` / `ENGINE_PARLER` / `ENGINE_AUDIO8`); auto-detected from the GGUFs present otherwise |
825
- | `referenceAudio` | string | — | Wav to clone (Chatterbox: mono, ≥ 5 s; CosyVoice3: 0.5-30 s, multichannel downmixed to mono, needs the s3tok + campplus GGUFs; Audio8: also needs `referenceText`). Audio8 accepts it per call too |
1041
+ | `engine` | string | auto | Force `'chatterbox'`, `'supertonic'`, `'cosyvoice3'`, `'parler'`, `'audio8'` or `'moss'` (`TTSGgml.ENGINE_CHATTERBOX` / `ENGINE_SUPERTONIC` / `ENGINE_COSYVOICE3` / `ENGINE_PARLER` / `ENGINE_AUDIO8` / `ENGINE_MOSS`); auto-detected from the GGUFs present otherwise |
1042
+ | `referenceAudio` | string | — | Wav to clone (Chatterbox: mono, ≥ 5 s; CosyVoice3: 0.5-30 s, multichannel downmixed to mono, needs the s3tok + campplus GGUFs; Audio8: also needs `referenceText`; MOSS: needs `files.mossCodecEncoder`). Audio8 accepts it per call too |
826
1043
  | `referenceText` | string | — | Audio8-only: what `referenceAudio` says, verbatim. Required whenever a reference is set; accepted per call |
1044
+ | `dialogueReferences` | string[] | — | MOSS-only: one 24 kHz recording per speaker (`[S1]`, `[S2]`, ...) for dialogue synthesis; the text must open with each recording's transcript under its tag; needs `files.mossCodecEncoder`, excludes `referenceAudio`, fixed per instance |
1045
+ | `durationTokens` | number | `0` | MOSS-only: target length in codec frames (12.5 per second); `0` keeps the length free; reloadable |
827
1046
  | `voiceDir` | string | — | Pre-baked voice profile |
828
- | `seed` | number | 42 | RNG seed (CFM noise + sampling) |
1047
+ | `seed` | number | 42 | RNG seed (CFM noise + sampling); MOSS defaults to its engine's 1234 |
829
1048
  | `nGpuLayers` | number | 0 | Layers offloaded to GPU (mirrors `useGPU`; pass `99` to offload all) |
830
1049
  | `nCtx` | number | 4096 | Chatterbox T3 context limit for the prompt plus generated speech tokens (25 tokens ≈ 1 s of audio). The KV cache is allocated up front at this length, so the 4096-token default directly bounds memory. Pass `0` to use the GGUF metadata value |
831
1050
  | `kvCacheType` | string | `f16` | T3 KV-cache dtype: `f32` \| `f16` \| `q8_0`. `f16` (~50% of f32) is the safe cross-backend default. `q8_0` stores the cache at ~27% of f32 and decodes 20-30% faster on Metal, but only works on backends with a q8_0 CONT op (CPU, CUDA) — it hard-aborts the multilingual model on Metal, so it is opt-in. Turbo greedy decoding is byte-identical across all three (upstream-validated). Pass `f32` for bit-exact pre-quantisation behaviour |
832
- | `threads` | number | hw.concurrency capped at 4 | |
833
- | `streamChunkTokens` | number | 0 | **>0 enables native chunk streaming** |
1051
+ | `threads` | number | hw.concurrency capped at 4 | MOSS: engine default of 4; `0` keeps it |
1052
+ | `streamChunkTokens` | number | 0 | **>0 enables native chunk streaming** (speech tokens per chunk; Supertonic: text tokens; MOSS: codec frames per chunk, 12.5 per second) |
834
1053
  | `streamFirstChunkTokens` | number | = streamChunkTokens | Smaller first chunk for low first-audio-out |
835
- | `cfmSteps` | number | 2 | Chatterbox: 1 = faster (halved CFM cost) |
1054
+ | `streamChunkTolerancePct` | number | 20 | Supertonic-only: boundary-snap window for clause / space chunk boundaries, in percent of the chunk target |
1055
+ | `streamMinChunkTokens` | number | 30 | Supertonic-only: floor on every streamed chunk's size in text tokens; shorter trailing chunks merge into the previous one |
1056
+ | `streamLeftContextTokens` | number | 0 | Chatterbox: left-context speech tokens S3Gen keeps per streamed chunk (typically 25-50), bounding per-chunk cost on long utterances; `0` = the whole emitted prefix. CosyVoice3: reserved — the pinned engine accepts but does not read it |
1057
+ | `cfmSteps` | number | 2 | Chatterbox native-streaming CFM steps: 1 = faster (halved CFM cost) |
1058
+ | `batchCfmSteps` | number | engine default | Chatterbox-only batch CFM steps (`0` = 2-step meanflow on Turbo, the model's full schedule on multilingual). The multilingual engine does not floor it, so values well below its schedule under-integrate |
1059
+ | `nPredict` | number | 1000 | Chatterbox-only cap on speech tokens per synthesis (25 ≈ 1 s, so ~40 s by default). Longer single calls also need `nCtx` to fit, or use `maxSentenceChars` |
1060
+ | `maxSentenceChars` | number | 0 | Chatterbox-only sentence auto-split: T3 + S3Gen run per segment of at most this many bytes (the reference CLI uses 180), so `nPredict` applies per segment |
1061
+ | `crossfadeMs` | number | 30 | Chatterbox-only raised-cosine crossfade between auto-split batch segments; native streaming stays gapless |
1062
+ | `repeatPenalty` | number | 1.2 | Chatterbox-only T3 repetition penalty (`1` = off) |
1063
+ | `exaggeration` / `cfgWeight` / `minP` | number | 0.5 / 0.5 / 0 | Chatterbox multilingual only (Turbo ignores them): prosody intensity (> 0 more expressive, < 0 flatter), T3 classifier-free guidance scale (`0` = off; `cfgRate` is the separate S3Gen rate), and the min-p sampling threshold |
836
1064
  | `speed` | number | 1.0 | Speaking-rate multiplier, bounded `[0.25, 4.0]` (`< 1` slower, `> 1` faster). Both engines |
837
1065
  | `voice` / `voiceName` | string | — | Supertonic voice id (e.g. `'F1'`, `'M1'`); Parler template speaker name (e.g. `'Laura'`, `'Rohit'`) |
1066
+ | `voiceJsonPath` | string | — | Supertonic-only: external voice JSON (`{ style_ttl, style_dp }`, e.g. a cloned voice) that overrides `voice`; the engine checks the tensor sizes against the model |
1067
+ | `prewarmText` | string | — | Supertonic-only: text synthesized once at load so GPU pipelines compile, and a Core ML vocoder sidecar specializes, before the first `run()` (skipped on a plain CPU run); use a representative length. Wins over the `vulkanCacheDir` default pre-warm |
1068
+ | `vulkanDevice` | number | 0 | Supertonic / CosyVoice3 Vulkan adapter, also used by their LavaSR enhancer: `0` = first, `N` = the Nth, `-1` = auto-pick by free VRAM preferring a discrete GPU |
1069
+ | `flowCutPrompt` | boolean | `false` | CosyVoice3-only: attention-only prompt frames in the flow — faster, with output that deviates slightly from the reference |
838
1070
  | `steps` / `numInferenceSteps` | number | GGUF default | Supertonic vector-estimator CFM steps (`0` = GGUF default) |
839
1071
  | `noiseNpyPath` | string | — | Supertonic: optional fixed CFM noise `.npy` for reproducibility |
840
1072
  | `description` / `voiceDescription` | string | fallback caption | Parler-only: full free-text voice description (mutually exclusive with the template fields) |
841
1073
  | `emotion`, `pace` | string | — | Cross-engine conditioning (see [Emotion & pace](#emotion--pace-cross-engine)); an unsupported value errors listing what that engine supports |
842
1074
  | `pitch`, `expressivity`, `noise`, `reverb`, `quality` | string | — | Parler-only template fields (see [Parler descriptions](#parler-descriptions)); invalid values error listing the valid set |
843
- | `temperature` / `topK` / `topP` | number | engine defaults | Parler + Audio8 sampling knobs (omit for the engine's own defaults — Parler temp 1.0 / top-k 50, Audio8 temp 0.7 / top-k 50 / top-p 0.9; `topP` in `(0, 1]`) |
1075
+ | `temperature` / `topK` / `topP` | number | engine defaults | Parler, Audio8 and Chatterbox sampling knobs (omit for the engine's own defaults — Parler temp 1.0 / top-k 50, Audio8 temp 0.7 / top-k 50 / top-p 0.9, Chatterbox temp 0.8 / top-k 1000 / top-p 0.95 where temp `0` is greedy and top-k `0` disables the cutoff; `topP` in `(0, 1]`) |
844
1076
  | `maxFrames` | number | engine max | Parler + Audio8 generation cap in frames (Parler ~86/s of audio, Audio8 ~21.5/s); `0` = model default, Parler rejects 1–9 |
845
1077
  | `greedy` | boolean | `false` | Audio8-only: take the argmax instead of sampling; ignores `temperature` / `topK` / `topP` |
846
1078
  | `minNewTokens` | number | GGUF default | Parler-only minimum tokens before EOS (`-1` = model default) |
847
1079
  | `normalizeNumbers` | boolean | `true` | Parler-only: expand digits before tokenization (English words; script-native digits on indic) — parler voices raw digits badly |
848
- | `instruct` | object \| string | — | CosyVoice3-only: instruction controls (`dialect` / `volume` / `style`, resolved by precedence in that order) or a raw instruction string; an unknown key or invalid value throws, and it counts toward the one-instruction rule (see [CosyVoice3 instruct](#cosyvoice3-instruct)) |
1080
+ | `instruct` | object \| string | — | CosyVoice3-only: instruction controls (`dialect` / `volume` / `style`, resolved by precedence in that order) or a raw instruction string; an unknown key or invalid value throws, and it counts toward the one-instruction rule (see [CosyVoice3 instruct](#cosyvoice3-instruct)). Also accepted per call, where it replaces the configured conditioning like a per-call `emotion` / `pace` |
849
1081
  | `promptText` | string | — | CosyVoice3-only: verbatim transcript of `referenceAudio` — set for zero-shot cloning, omit for cross-lingual; without a reference it overrides the baked voice's transcript |
850
- | `streamLeftContextTokens` | number | — | CosyVoice3-only: intended native chunk-streaming left-context tokens. Reserved / not yet effective — the pinned engine accepts but does not read it |
851
1082
  | `mecabDictDir` | string | — | Chatterbox MTL Japanese (`ja`): compiled MeCab/IPAdic dictionary directory |
852
1083
  | `cangjieTsvPath` | string | — | Chatterbox MTL Chinese (`zh`): `Cangjie5_TC` TSV path |
853
1084
  | `backendsDir` | string | `resolveBackendsDir()` | Root dir the addon scans for dynamically-loaded ggml backend `.so` files. Defaults to the package's own `prebuilds/` when present, otherwise the installed platform package. Required on Android when backends ship elsewhere (e.g. inside the APK); ignored on platforms that statically link the backend |
854
1085
  | `openclCacheDir` | string | unset | Android-only: directory where the OpenCL backend persists its compiled program-binary cache. Setting it across runs avoids re-JITing the kernels on every fresh process |
855
1086
  | `vulkanCacheDir` | string | unset | Supertonic + `useGPU: true` only: writable directory where the Vulkan backend persists its compiled pipeline cache (`GGML_VK_PIPELINE_CACHE_DIR`). Moves the one-time first-dispatch pipeline-compile cost (seconds on Mali) off the first `run()` — paid once per install instead of once per process — and enables a load-time pre-warm. Fully opt-in: unset -> no cross-process cache, no pre-warm, behaviour unchanged |
856
- | `config.language` | string | `"en"` | Chatterbox MTL accepts `es/fr/de/pt/it/zh/ja/ko/...`; turbo & Supertonic are English |
857
- | `config.useGPU` | boolean | `false` | Set to `true` to route through Metal / CUDA / Vulkan / OpenCL if available. Honored for Chatterbox/Supertonic on GPU-capable hosts (including Android, per `tts-cpp`'s per-vendor allowlist); Parler is validated on Apple/Metal, linux CUDA, and Android/ARM Mali Vulkan; CosyVoice3 and Audio8 offload on Apple/Metal, desktop linux CUDA/Vulkan, Windows Vulkan, and Android OpenCL/Adreno. Unsupported backends fall back to CPU. See [Backends & GPU acceleration](#backends--gpu-acceleration) |
858
- | `config.outputSampleRate` | number | — (engine-native) | Resample the output to this rate (8000–192000 Hz). Omit to keep the engine-native rate (Chatterbox 24 kHz, Supertonic / Parler / Audio8 44.1 kHz, CosyVoice3 24 kHz, enhancer 48 kHz). Parler native chunk streaming accepts a non-native rate only with the enhancer active |
859
- | `opts.stats` | boolean | `false` | Populate `response.stats` with RTF, `backendDevice` (0=CPU, 1=GPU), `backendId` (0=CPU, 1=Metal, 2=CUDA, 3=Vulkan, 4=OpenCL, 99=other), and — when an enhancer is active — `enhancerBackendDevice` / `enhancerBackendId` |
1087
+ | `config.language` | string | `"en"` | Chatterbox MTL accepts `es/fr/de/pt/it/zh/ja/ko/...`; turbo & Supertonic are English; MOSS takes it as a prompt hint (`en`, `zh`, ...) |
1088
+ | `config.useGPU` | boolean | `false` | Set to `true` to route through Metal / CUDA / Vulkan / OpenCL if available. Honored for Chatterbox/Supertonic on GPU-capable hosts (including Android, per `tts-cpp`'s per-vendor allowlist); Parler is validated on Apple/Metal, linux CUDA, and Android/ARM Mali Vulkan; CosyVoice3 and Audio8 offload on Apple/Metal, desktop linux CUDA/Vulkan, Windows Vulkan, and Android OpenCL/Adreno; MOSS GPU offload is validated on Apple/Metal. Unsupported backends fall back to CPU. See [Backends & GPU acceleration](#backends--gpu-acceleration) |
1089
+ | `config.outputSampleRate` | number | — (engine-native) | Resample the output to this rate (8000–192000 Hz). Omit to keep the engine-native rate (Chatterbox 24 kHz, Supertonic / Parler / Audio8 44.1 kHz, CosyVoice3 24 kHz, MOSS 24 kHz, enhancer 48 kHz). Parler native chunk streaming accepts a non-native rate only with the enhancer active; MOSS accepts only its native rate |
1090
+ | `opts.stats` | boolean | `false` | Populate `response.stats` with RTF, `backendDevice` (0=CPU, 1=GPU), `backendId` (0=CPU, 1=Metal, 2=CUDA, 3=Vulkan, 4=OpenCL, 99=other), `enhancerBackendDevice` / `enhancerBackendId` and `denoiserBackendDevice` / `denoiserBackendId` (`-1` when that LavaSR stage is off), and the engine's own stage stats: Chatterbox `t3Ms` / `s3genMs` / `t3Tokens`; CosyVoice3 per-stage ms (`lmPrefillMs` … `hiftDecodeMs`, `stageTotalMs`) and work counters (`speechTokens`, `decodeSteps`, …); Audio8 per-stage ms (`prefillMs`, `fastDecodeMs`, `codecSynthMs`, …, `stageTotalMs`) |
860
1091
  | `exclusiveRun` | boolean | `false` | **Top-level** option (not under `opts`): serialize overlapping streaming runs |
861
1092
 
862
1093
  ### Methods
@@ -869,6 +1100,8 @@ a one-off encode when a new reference recording is supplied.
869
1100
  config (`language`, `useGPU`, `outputSampleRate`, …).
870
1101
  - `await model.destroy()` — `unload()` + mark this instance dead.
871
1102
  - `await model.cancel()` — best-effort cancel of any in-flight run.
1103
+ - `TTSGgml.getVoiceControls()` — static: tts-cpp's emotion / pace vocabulary
1104
+ and each engine's supported subset (see [Emotion & pace](#emotion--pace-cross-engine)).
872
1105
  - `model.run({ input, type: 'text' })` → `QvacResponse`.
873
1106
  - `model.run({ input, streamOutput: true })` → sentence-chunked
874
1107
  synthesis driven by the JS-side sentence splitter (see
@@ -895,7 +1128,7 @@ All `run*` methods return a `QvacResponse` (from `@qvac/infer-base`):
895
1128
  ```js
896
1129
  response.onUpdate(data => {
897
1130
  data.outputArray // Int16Array — mono PCM
898
- data.sampleRate // actual rate: Chatterbox 24000, Supertonic/Parler/Audio8 44100, enhancer 48000
1131
+ data.sampleRate // actual rate: Chatterbox/CosyVoice3/MOSS 24000, Supertonic/Parler/Audio8 44100, enhancer 48000
899
1132
  data.chunkIndex // present on sentence-streaming events only
900
1133
  data.sentenceChunk // present on sentence-streaming events only
901
1134
  })
@@ -906,8 +1139,10 @@ response.stats.totalTime // seconds
906
1139
  response.stats.realTimeFactor // synthesis time / audio duration
907
1140
  response.stats.audioDurationMs
908
1141
  response.stats.totalSamples
909
- response.stats.tokensPerSecond // Audio8 counts codec frames, the others characters
910
- response.stats.generatedFrames // Audio8 only: codec frames, on a fixed 46 ms grid
1142
+ response.stats.tokensPerSecond // Audio8 and MOSS count codec frames, the others characters
1143
+ response.stats.generatedFrames // Audio8 and MOSS only: codec frames (Audio8 every 46 ms, MOSS every 80 ms)
1144
+ response.stats.codecSidecarLoaded // Audio8 only, macOS / iOS: 1 while the Apple Core ML codec sidecar is attached
1145
+ response.stats.codecOnCoreml // Audio8 only: 1 when this synthesis ran the codec on that sidecar, 0 when on the ggml backend
911
1146
  response.stats.backendDevice // 0=CPU, 1=GPU
912
1147
  response.stats.backendId // 0=CPU, 1=Metal, 2=CUDA, 3=Vulkan, 4=OpenCL, 99=other
913
1148
  // present when a LavaSR enhancer is active:
@@ -915,6 +1150,11 @@ response.stats.enhancerBackendDevice // -1 none, 0 CPU, 1 GPU
915
1150
  response.stats.enhancerBackendId
916
1151
  ```
917
1152
 
1153
+ For `runStream()` and `runStreaming()`, `codecSidecarLoaded` and
1154
+ `codecOnCoreml` retain the last chunk that reported each field, including zero
1155
+ when a sidecar is retired or synthesis falls back to ggml. They are not summed;
1156
+ if no chunk reports a field, it remains absent.
1157
+
918
1158
  ### Text helpers
919
1159
 
920
1160
  The sentence splitter is available from
@@ -948,6 +1188,57 @@ const {
948
1188
  | 13010 | `FAILED_TO_STOP` |
949
1189
  | 13011 | `JOB_ALREADY_RUNNING` |
950
1190
 
1191
+ ## Assessing fit
1192
+
1193
+ `assessFit` projects a load against the memory free right now. It reads GGUF metadata and never weight data, so the registry's weightless copy of each file answers the same as the file itself and the projection can run before anything is downloaded. It is a module export, not an instance method: nothing is loaded to call it.
1194
+
1195
+ A request is a `createInstance` config plus the workload, so it carries the same keys the load takes and the engine's own config builder reads it.
1196
+
1197
+ ```js
1198
+ const TTSGgml = require('@qvac/tts-ggml')
1199
+
1200
+ const fit = TTSGgml.assessFit({
1201
+ engineType: 'supertonic',
1202
+ supertonicModelPath: '/models/supertonic.gguf',
1203
+ useGPU: true,
1204
+ textTokens: 256,
1205
+ audioSeconds: 30
1206
+ })
1207
+
1208
+ fit.status // 'fits' | 'does-not-fit' | 'error'
1209
+ fit.reason // the engine's own wording, e.g. 'model-unreadable', 'workload-too-large'
1210
+ fit.modelVariant // which pipeline was projected, e.g. 'chatterbox-t3-turbo'
1211
+ fit.deviceName
1212
+ fit.deviceBytes
1213
+ fit.weightsBytes
1214
+ fit.stateBytes
1215
+ fit.lmComputeBytes // 0 for a pipeline with no language-model stage, such as supertonic
1216
+ fit.codecComputeBytes // 0 for a pipeline with no separate codec stage
1217
+ fit.hostBytes
1218
+ fit.lavasrFileBytes // the LavaSR stages' size on disk, 0 when none was named
1219
+ fit.report
1220
+ ```
1221
+
1222
+ `engineType` picks the fitter, the same key a load uses, and is required: a fit request carries none of the file keys a load is inferred from. Each engine takes the load's own file keys, plus a workload the fit adds:
1223
+
1224
+ | `engineType` | Files | Workload |
1225
+ | --- | --- | --- |
1226
+ | `supertonic` | `supertonicModelPath` | `textTokens`, `audioSeconds`, `precision`, `f16Weights` |
1227
+ | `parler` | `parlerModelPath` | `descriptionTokens`, `promptTokens`, `maxFrames` |
1228
+ | `chatterbox` | `t3ModelPath`, `s3genModelPath` | `textTokens`, `predictTokens` |
1229
+ | `audio8` | `audio8LmPath`, `audio8CodecDecoderPath`, `audio8CodecEncoderPath` | `promptTokens`, `maxFrames`, `referenceSeconds` |
1230
+ | `cosyvoice3` | `cosyvoiceLlmModelPath`, `cosyvoiceFlowModelPath`, `cosyvoiceHiftModelPath`, `cosyvoiceVoiceModelPath` | `textTokens`, `speechTokens` |
1231
+
1232
+ Everything else comes from the load config: `nGpuLayers` and `useGPU` carry the offload intent, `nCtx` and `kvCacheType` size the chatterbox cache, `steps` takes the GGUF's own default at 0, and `vulkanDevice` and `backendsDir` place the backend. Supplying `audio8CodecEncoderPath` projects voice cloning, which the decoder alone cannot do. `marginBytes` sets the free memory that must remain for the projection to count as fitting.
1233
+
1234
+ `lavasrEnhancerPath` and `lavasrDenoiserPath` are reported as `lavasrFileBytes`, their size on disk. Those stages have no fitter, so the figure is a size on disk. It is counted in neither `deviceBytes` nor `hostBytes`.
1235
+
1236
+ `deviceSharesHostMemory` reports that the device pool is system RAM, so host bytes compete with device bytes.
1237
+
1238
+ A model the engine cannot read is `status: "error"`, as is a voice with no fitter, which reports `reason: "unsupported-engine"`. MOSS is the one voice in that state. A broken request, or a host with no native binding, throws.
1239
+
1240
+ The supertonic fitter covers the fused graph path: a validated GPU, or a CPU without the Accelerate pointwise kernels. Elsewhere it answers `compute-path-not-supported` and projects nothing.
1241
+
951
1242
  ## Examples
952
1243
 
953
1244
  Runnable demos under `examples/`:
@@ -965,12 +1256,15 @@ Runnable demos under `examples/`:
965
1256
  | `supertonic-mtl-tts.js` | Multilingual Supertonic synthesis |
966
1257
  | `supertonic-mtl-sweep-tts.js` | Multilingual Supertonic sweep across languages |
967
1258
  | `supertonic-sentence-stream-tts.js` | Supertonic sentence-level streaming |
1259
+ | `supertonic-chunk-stream-tts.js` | Supertonic native chunk streaming via `streamChunkTokens`, with streaming playback |
968
1260
  | `supertonic-enhanced.js` | Supertonic + LavaSR 48 kHz enhancement. `bare examples/supertonic-enhanced.js "Hello"` |
969
1261
  | `parler-tts.js` | Parler batch synth with voice/emotion templates. `bare examples/parler-tts.js "Hello" Laura happy` |
970
1262
  | `parler-enhanced.js` | Parler + LavaSR 48 kHz enhancement. `bare examples/parler-enhanced.js "Hello" Laura happy` |
971
1263
  | `cosyvoice-tts.js` | CosyVoice3 batch synth with the cross-engine emotion option (24 kHz; CPU by default, `--gpu` opts into Metal / desktop Vulkan / Android GPU). `bare examples/cosyvoice-tts.js --gpu "Hello" happy` |
972
1264
  | `cosyvoice-enhanced.js` | CosyVoice3 + LavaSR 48 kHz enhancement (add `--denoise` for the denoiser). `bare examples/cosyvoice-enhanced.js "Hello"` |
973
1265
  | `audio8-tts.js` | Audio8 batch synth, optionally cloning a reference. Set `QVAC_TTS_AUDIO8_GPU=1` for the desktop GPU backend (CUDA or Vulkan on linux, Vulkan on Windows). `bare examples/audio8-tts.js "Hello" voice.wav "What it says."` |
1266
+ | `moss-tts.js` | MOSS batch synth, optionally cloning a reference. Set `QVAC_TTS_MOSS_STREAM_FRAMES=25` for native chunk streaming, `QVAC_TTS_MOSS_DURATION=38` for a target length and `QVAC_TTS_MOSS_GPU=1` for the GPU backend. `bare examples/moss-tts.js "Hello" voice.wav` |
1267
+ | `moss-dialogue-tts.js` | MOSS-TTSD multi-speaker dialogue from one 24 kHz reference per speaker; the text opens with each reference's transcript. Set `QVAC_TTS_MOSS_GPU=1` for the GPU backend. `bare examples/moss-dialogue-tts.js "[S1] What alice.wav says. [S2] What bob.wav says. [S1] Hi. [S2] Hello." alice.wav bob.wav` |
974
1268
 
975
1269
  The two streaming examples feed PCM into a single long-running
976
1270
  `sox play` / `ffplay` process so chunks play back-to-back without any
@@ -993,8 +1287,7 @@ files are absent. They cover, across the engines:
993
1287
  * batch synthesis with full RuntimeStats,
994
1288
  * sentence-level streaming (`runStream` / `run({ streamOutput: true })`
995
1289
  / `runStreaming` over async iterators),
996
- * native sub-sentence chunk streaming (Chatterbox-only via
997
- `streamChunkTokens`),
1290
+ * native sub-sentence chunk streaming via `streamChunkTokens`,
998
1291
  * sequential-run / fresh-instance / reload-stability behaviour,
999
1292
  * strict GPU-backend assertion via `response.stats.backendDevice` +
1000
1293
  `backendId` (set `NO_GPU=true` to skip on CPU-only runners,
@@ -1005,6 +1298,11 @@ files are absent. They cover, across the engines:
1005
1298
  for WER against the synthesized audio (whisper-small via the
1006
1299
  `@qvac/asr-ggml` development dependency).
1007
1300
 
1301
+ The MOSS integration test (`moss.test.js`) does not download its GGUFs:
1302
+ set `QVAC_TEST_MOSS_MODEL_DIR` to a directory holding them, otherwise it is
1303
+ skipped. The C++ suite runs its MOSS synthesis cases when
1304
+ `QVAC_TEST_MOSS_BACKBONE_GGUF` and `QVAC_TEST_MOSS_DECODER_GGUF` are set.
1305
+
1008
1306
  To stress-test long inputs, set `INPUT_SENTENCES=medium` (or `long`)
1009
1307
  and re-run the integration suite — `addon.test.js` reads the env var to
1010
1308
  pick its sentence corpus from `test/data/sentences-{medium,long}.js`.
@@ -1071,3 +1369,7 @@ OpenCL; Xclipse and Mali use Vulkan).
1071
1369
  ## License
1072
1370
 
1073
1371
  Apache-2.0. See [LICENSE](./LICENSE).
1372
+
1373
+ ## Pocket TTS
1374
+
1375
+ See [Pocket TTS](docs/pocket-tts.md) for model conversion, addon/SDK usage, supported controls and validation commands.