@qvac/tts-ggml 0.9.2 → 0.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. package/README.md +330 -36
  2. package/docs/pocket-tts.md +201 -0
  3. package/examples/moss-dialogue-tts.js +113 -0
  4. package/examples/moss-tts.js +154 -0
  5. package/examples/pocket-tts.js +51 -0
  6. package/examples/supertonic-chunk-stream-tts.js +156 -0
  7. package/examples/supertonic-sentence-stream-tts.js +4 -4
  8. package/index.d.ts +352 -29
  9. package/index.js +698 -52
  10. package/lib/fit.d.ts +127 -0
  11. package/lib/fit.js +29 -0
  12. package/lib/pocketConfig.d.ts +10 -0
  13. package/lib/pocketConfig.js +145 -0
  14. package/package.json +16 -9
  15. package/test/integration/addon.test.js +13 -7
  16. package/test/integration/audio8.test.js +7 -0
  17. package/test/integration/chatterbox-mtl.test.js +20 -9
  18. package/test/integration/chatterbox-speed.test.js +16 -3
  19. package/test/integration/cosyvoice3-clone.test.js +17 -2
  20. package/test/integration/cosyvoice3-lavasr.test.js +21 -11
  21. package/test/integration/cosyvoice3.test.js +33 -8
  22. package/test/integration/fit.test.js +174 -0
  23. package/test/integration/gpu-smoke.test.js +13 -0
  24. package/test/integration/lavasr-enhancer.test.js +107 -5
  25. package/test/integration/moss.test.js +287 -0
  26. package/test/integration/multiple-runs.test.js +84 -57
  27. package/test/integration/output-sample-rate.test.js +10 -2
  28. package/test/integration/parler-wer.test.js +14 -9
  29. package/test/integration/parler.test.js +54 -24
  30. package/test/integration/pocket.integration.test.js +156 -0
  31. package/test/integration/supertonic-mtl.test.js +2 -0
  32. package/test/integration/supertonic.test.js +70 -0
  33. package/test/integration/supertonic3-quant.test.js +2 -0
  34. package/test/integration/voice-controls.test.js +56 -0
  35. package/test/mobile/integration.auto.cjs +16 -1
  36. package/test/mobile/pocket-worklet-host.cpp +92 -0
  37. package/test/mobile/pocket-worklet.cjs +47 -0
  38. package/test/mobile/test-groups.json +8 -4
  39. package/test/utils/runCosyvoiceTTS.js +1 -0
  40. package/test/utils/runTTS.js +6 -11
  41. package/tts.d.ts +13 -2
  42. package/tts.js +3 -2
package/README.md CHANGED
@@ -7,8 +7,10 @@ v1/v2 still loadable), **Parler** (mini/large English + indic 21-language,
7
7
  description-conditioned with voice/emotion templates), **CosyVoice3**
8
8
  (Fun-CosyVoice3-0.5B, instruct-conditioned, 24 kHz, CPU with opt-in
9
9
  Metal on Apple, CUDA/Vulkan on Linux, Vulkan on Windows, and OpenCL/Adreno
10
- GPU offload on Android), and **Audio8**
11
- (DualAR + neural codec, in-process voice cloning, desktop GPU), plus optional
10
+ GPU offload on Android), **Audio8**
11
+ (DualAR + neural codec, in-process voice cloning, desktop GPU), and **MOSS**
12
+ (MOSS-TTS Delay, 24 kHz, in-process voice cloning, native chunk streaming,
13
+ desktop), plus optional
12
14
  LavaSR neural denoise + 48 kHz bandwidth-extension enhancement. Unsure
13
15
  which checkpoint to stage? Start with [Choosing a model](#choosing-a-model).
14
16
 
@@ -24,7 +26,8 @@ backend per its own per-vendor allowlist (Adreno → OpenCL,
24
26
  Xclipse/Mali → Vulkan). Parler supports Apple/Metal, linux CUDA, and the
25
27
  validated Android paths, including Vulkan on ARM Mali (see
26
28
  [Backends & GPU acceleration](#backends--gpu-acceleration)). Audio8 supports
27
- CUDA/Vulkan offload on Linux and Vulkan on Windows.
29
+ CUDA/Vulkan offload on Linux and Vulkan on Windows. MOSS GPU offload is
30
+ validated on Apple/Metal.
28
31
 
29
32
  [qvac-tts-cpp]: https://github.com/tetherto/qvac-fabric-speech.cpp/tree/master/engines/tts
30
33
 
@@ -44,6 +47,9 @@ CUDA/Vulkan offload on Linux and Vulkan on Windows.
44
47
  `config.useGPU: true` on GPU-capable hosts — including Android, where
45
48
  `tts-cpp` selects the GPU backend per its per-vendor allowlist (see
46
49
  [Backends & GPU acceleration](#backends--gpu-acceleration)).
50
+ - **Apple Core ML sidecars** for the Supertonic vocoder and the Audio8 codec
51
+ on macOS / iOS, picked up when staged next to the model (see
52
+ [Core ML sidecars on Apple](#core-ml-sidecars-on-apple)).
47
53
  - **Dynamic backend loading on Android** — per-arch CPU + Vulkan +
48
54
  OpenCL `.so` files ship under `prebuilds/<bare-target>/qvac__tts-ggml/`
49
55
  and are picked up at runtime via the new `backendsDir` option (see
@@ -66,6 +72,8 @@ obvious for each job.
66
72
  | English + voice cloning + low first-audio latency | `chatterbox-t3-turbo.gguf` + `chatterbox-s3gen.gguf` | Reference-wav / voice-profile cloning; native `streamChunkTokens` chunk streaming. Native 24 kHz. |
67
73
  | Multilingual + voice cloning (EU / CJK) | `chatterbox-t3-mtl.gguf` + `chatterbox-s3gen-mtl.gguf` | en/es/fr/de/pt/it/zh/ja/ko/…; same cloning + streaming surface as Turbo. |
68
74
  | Voice cloning at 44.1 kHz with nothing pre-baked | `audio8-lm-q8_0.gguf` + `audio8-codec-decoder-q8_0.gguf` (+ `audio8-codec-encoder-q8_0.gguf` to clone) | Clones from a reference wav **and its transcript**, encoded in-process — no enrolment step, no voice profile. Whole-utterance only (no native chunk streaming). |
75
+ | Multilingual speech with native chunk streaming and cloning on a desktop | `moss-tts-delay-f16.gguf` + `moss-codec-decoder-f16.gguf` (+ `moss-codec-encoder-f16.gguf` to clone) | 8B backbone, desktop only. Clones from a reference wav encoded in-process (no transcript needed); native `streamChunkTokens` chunk streaming. Native 24 kHz. |
76
+ | Multi-speaker dialogue that continues each speaker's reference voice | `moss-ttsd-f16.gguf` + `moss-codec-decoder-f16.gguf` + `moss-codec-encoder-f16.gguf` | 8B backbone, desktop only. One 24 kHz reference per speaker; the text opens with their transcripts. Native 24 kHz. |
69
77
  | Indic languages | `parler-indic-q8_0.gguf` | 21 Indic languages; voice / emotion templates. Emotion is officially tested on 10 languages — see [Parler descriptions & emotions](#parler-descriptions--emotions) (or the upstream model card). Native 44.1 kHz. |
70
78
  | Chinese dialects (Cantonese, Sichuan, Shanghai, …) | CosyVoice3 dir (`cosyvoice3-llm-*.gguf` + flow / hift / `voice.gguf`) | Instruct-conditioned; 17 dialects via `instruct: { dialect: '…' }`. CPU, with opt-in GPU offload (Metal on Apple, Vulkan on Linux/Windows, OpenCL/Adreno on Android); native 24 kHz. |
71
79
  | Zero-shot / cross-lingual cloning (multilingual) | CosyVoice3 dir + `cosyvoice3-s3tok-*.gguf` + `cosyvoice3-campplus-*.gguf` | Clones from a reference wav; its transcript (`promptText`) selects zero-shot, omitting it selects cross-lingual (timbre only, any target language). Composes with `instruct`. |
@@ -88,6 +96,8 @@ here, so this guide does not go stale when backends change.
88
96
  | `parler-mini-v1-q8_0` | English | ~1.2 GB | 44.1 kHz | Description / templates | Sentence streaming |
89
97
  | CosyVoice3 (`cosyvoice3/`) | Instruct-led (strong on Chinese + dialects) | ~2.3 GB dir (+ ~300 MB to clone) | 24 kHz | Reference wav (zero-shot / cross-lingual) + `instruct` (dialect / emotion / speed / volume / style) | Native chunk opts |
90
98
  | `audio8-lm-q8_0` + codec halves | Text-led (no `language` option) | ~0.7 GB (+ ~120 MB to clone) | 44.1 kHz | Reference wav + transcript, per call too | Sentence streaming |
99
+ | `moss-tts-delay-f16` + codec halves | Multilingual (`language` hint) | ~18.8 GB (+ ~1.8 GB to clone) | 24 kHz | Reference wav, fixed per instance | Sentence + native chunk |
100
+ | `moss-ttsd-f16` + codec halves | Multilingual (`language` hint) | ~20.5 GB | 24 kHz | One reference wav per speaker, fixed per instance | Native chunk |
91
101
 
92
102
  ### Legacy Supertonic v1 / v2
93
103
 
@@ -162,8 +172,8 @@ dependency, pinned to the exact `@qvac/tts-ggml` version:
162
172
 
163
173
  ## Model files
164
174
 
165
- Five engine families are wrapped (Chatterbox, Supertonic, Parler,
166
- CosyVoice3, Audio8), each with its own GGUF layout under `models/`:
175
+ Six engine families are wrapped (Chatterbox, Supertonic, Parler,
176
+ CosyVoice3, Audio8, MOSS), each with its own GGUF layout under `models/`:
167
177
 
168
178
  ```
169
179
  # Chatterbox turbo (English)
@@ -177,6 +187,9 @@ chatterbox-s3gen-mtl.gguf (~1.0 GB)
177
187
  # Supertonic 3 (Supertone/supertonic-3; 31 languages) — preferred Supertonic
178
188
  # checkpoint; published per quant tier (auto-detected from modelDir)
179
189
  supertonic3-q4_0.gguf (~80 MB; also -q8_0 ~126 MB / -f16 ~191 MB / -f32)
190
+ supertonic3-vocoder.mlmodelc/ (optional, macOS / iOS: Apple Core ML sidecar for
191
+ the vocoder, used by the 8-bit and wider tiers; see
192
+ Core ML sidecars on Apple)
180
193
 
181
194
  # Legacy Supertonic (not recommended for new integrations — see Choosing a model)
182
195
  supertonic.gguf (~263 MB) — v1 English only
@@ -195,8 +208,9 @@ parler-indic-q8_0.gguf (~1.3 GB; also -f16 / -f32; 21 Indic languages)
195
208
  # from the cosyvoice3-llm-*.gguf file
196
209
  cosyvoice3/
197
210
  cosyvoice3-llm-*.gguf (~973 MB q8_0 — Qwen2.5 speech LM)
198
- cosyvoice3-flow-*.gguf (~1.3 GB f32 — DiT conditional-flow-matching)
199
- cosyvoice3-hift-*.gguf (~83 MB f32 — CausalHiFT vocoder)
211
+ cosyvoice3-flow-*.gguf (~1.3 GB f32 — DiT conditional-flow-matching; also
212
+ -f16 / -bf16 / -q8_0 / -q4_0)
213
+ cosyvoice3-hift-*.gguf (~83 MB f32 — CausalHiFT vocoder; also -f16)
200
214
  voice.gguf (baked default voice: timbre + prompt tensors)
201
215
  vocab.json merges.txt (Qwen2 BPE tokenizer)
202
216
  cosyvoice3-s3tok-*.gguf (~275 MB q8_0 / ~497 MB f16 — speech tokenizer;
@@ -208,6 +222,17 @@ cosyvoice3/
208
222
  audio8-lm-q8_0.gguf (~0.6 GB; also -f16 / -q4_0 / -f32)
209
223
  audio8-codec-decoder-q8_0.gguf (~110 MB; also -f16 / -f32)
210
224
  audio8-codec-encoder-q8_0.gguf (~120 MB; also -f16 / -f32; cloning only)
225
+ audio8-codec-decoder.mlmodelc/ (optional, macOS / iOS: Apple Core ML sidecar for
226
+ the codec synthesis stack, picked up when it sits
227
+ next to the decoder GGUF; one export serves every
228
+ quant tier)
229
+
230
+ # MOSS (OpenMOSS MOSS-TTS v1.5 Delay; 24 kHz, 8B backbone + RVQ codec); the
231
+ # encoder is only needed to clone a voice or run a dialogue
232
+ moss-tts-delay-f16.gguf (~17 GB)
233
+ moss-ttsd-f16.gguf (~17 GB; MOSS-TTSD dialogue backbone, optional)
234
+ moss-codec-decoder-f16.gguf (~1.8 GB)
235
+ moss-codec-encoder-f16.gguf (~1.8 GB; cloning and dialogue only)
211
236
  ```
212
237
 
213
238
  Download the registry-published Chatterbox, Supertonic, and Parler models into
@@ -219,10 +244,15 @@ npm run download-models:registry -- --group chatterbox,supertonic3
219
244
  npm run download-models:registry -- --output /path/to/models
220
245
  ```
221
246
 
222
- CosyVoice3 and Audio8 are not currently included in that registry command.
223
- Stage their layouts shown above from local converted artifacts. The package
224
- converts upstream Chatterbox, Supertonic, and Parler checkpoints via a Python
225
- venv pipeline:
247
+ CosyVoice3, Audio8 and MOSS are not currently included in that registry command.
248
+ Stage their layouts shown above from local converted artifacts. CosyVoice3
249
+ resolves each component by filename prefix and does not rank quantizations, so
250
+ stage one file per component or name it explicitly with
251
+ `files.cosyvoiceFlowModelPath` / `cosyvoiceHiftModelPath`. Which flow tier is
252
+ fastest depends on the backend — `q8_0` on Vulkan / CUDA, `f16` on Metal,
253
+ `bf16` on AVX512-BF16 CPUs and `f16` on other CPUs — and the `f16` HiFT is the
254
+ recommendation everywhere. The package converts upstream Chatterbox,
255
+ Supertonic, and Parler checkpoints via a Python venv pipeline:
226
256
 
227
257
  ```bash
228
258
  npm run setup-models # creates ./venv, installs requirements.txt, runs convert-models.sh
@@ -238,7 +268,9 @@ npm run convert-models
238
268
  The Audio8 GGUFs are produced by the converters in
239
269
  qvac-fabric-speech.cpp (`engines/tts/scripts/convert-audio8-lm-to-gguf.py`
240
270
  and `convert-audio8-codec-to-gguf.py`) until they are published to the model
241
- registry alongside the other engines.
271
+ registry alongside the other engines. The MOSS GGUFs come from
272
+ `convert-moss-delay-to-gguf.py` and `convert-moss-codec-to-gguf.py` in the same
273
+ directory.
242
274
 
243
275
  Point the addon at a custom location via `files.modelDir` (engine
244
276
  auto-detected from the gguf filenames present), or pass explicit
@@ -247,7 +279,9 @@ auto-detected from the gguf filenames present), or pass explicit
247
279
  `files.cosyvoiceModelDir` (CosyVoice3 — a directory, see
248
280
  [CosyVoice3 instruct](#cosyvoice3-instruct)) /
249
281
  `files.audio8Lm` + `files.audio8CodecDecoder` (+ `files.audio8CodecEncoder`
250
- to clone) (Audio8).
282
+ to clone) (Audio8) /
283
+ `files.mossBackbone` + `files.mossCodecDecoder` (+ `files.mossCodecEncoder`
284
+ to clone) (MOSS).
251
285
 
252
286
  ## Quick start
253
287
 
@@ -320,7 +354,8 @@ Full runnable demo (with streaming playback):
320
354
 
321
355
  Use when you want the fastest possible first-audio-out **within a
322
356
  single utterance**. The C++ engine splits each synthesis into chunks
323
- of `streamChunkTokens` speech tokens (25 ≈ 1 s of audio) and emits
357
+ of `streamChunkTokens` speech tokens (25 ≈ 1 s of audio; on MOSS the unit
358
+ is a codec frame, so 25 ≈ 2 s) and emits
324
359
  audio per chunk, keeping HiFT's source cache phase-continuous across
325
360
  seams so the joins are inaudible.
326
361
 
@@ -345,6 +380,28 @@ await response.onUpdate(data => {
345
380
  Full runnable demo (with gapless playback via `sox` or `ffplay`):
346
381
  `bare examples/chatterbox-chunk-stream-tts.js`
347
382
 
383
+ Long utterances re-synthesize the whole emitted prefix for every chunk by
384
+ default; `streamLeftContextTokens` (typically 25-50) keeps only that many
385
+ tokens of left context per chunk, which bounds the per-chunk cost.
386
+
387
+ Supertonic streams the same way, but its chunks are measured in text tokens
388
+ (Unicode code points; about 50 for English, 25-30 for CJK). The engine snaps
389
+ each boundary to a sentence end, clause or space within
390
+ `streamChunkTolerancePct` of the target and never emits a chunk shorter than
391
+ `streamMinChunkTokens`. Supertonic native streaming cannot be combined with
392
+ the LavaSR enhancer or denoiser; use sentence-level streaming for enhanced
393
+ output.
394
+
395
+ ```js
396
+ const model = new TTSGgml({
397
+ files: { supertonicModel: './models/supertonic3-q8_0.gguf' },
398
+ streamChunkTokens: 50, // text tokens per chunk
399
+ streamFirstChunkTokens: 20 // smaller first chunk
400
+ })
401
+ ```
402
+
403
+ Full runnable demo (with streaming playback): `bare examples/supertonic-chunk-stream-tts.js`
404
+
348
405
  ## Voice cloning
349
406
 
350
407
  ### Chatterbox
@@ -415,6 +472,14 @@ across calls skips the encoder. Per-call fields also ride on the
415
472
  `runStream` / `runStreaming` options, pinned for the whole response so the
416
473
  cache stays hot across chunks.
417
474
 
475
+ ### MOSS
476
+
477
+ MOSS clones from `referenceAudio` alone, with no transcript. The recording
478
+ must be sampled at 24 kHz (there is no resampling; multichannel input is
479
+ downmixed), needs `files.mossCodecEncoder`, and is fixed for the instance.
480
+ For a multi-speaker dialogue, pass one recording per speaker in
481
+ `dialogueReferences` instead. See [MOSS](#moss).
482
+
418
483
  ### CosyVoice3
419
484
 
420
485
  CosyVoice3 clones **zero-shot or cross-lingual**, selected by whether the
@@ -579,6 +644,10 @@ Notes:
579
644
  - Denoiser + native chunk streaming (`streamChunkTokens > 0`) is rejected up
580
645
  front for every engine — a stateful streaming denoiser is the follow-up. Use
581
646
  batch synthesis, or drop the denoiser for streaming.
647
+ - The denoiser follows the engine's resolved device, like the enhancer: a GPU
648
+ engine runs its ggml graph on the GPU (falling back to the ggml CPU backend
649
+ when no GPU backend initialises), a CPU engine keeps the scalar core.
650
+ `denoiserBackendDevice` / `denoiserBackendId` in the stats report where it ran.
582
651
  - The tts-cpp UL-UNAS forward is implemented in
583
652
  [qvac-fabric-speech.cpp#78](https://github.com/tetherto/qvac-fabric-speech.cpp/pull/78)
584
653
  (scalar CPU port, validated bit-close to the ONNX reference); it requires a
@@ -605,6 +674,46 @@ On hosts where more than one backend is usable, `TTS_CPP_GPU_BACKEND`
605
674
  and fails loudly when that backend cannot be resolved; unset (or empty)
606
675
  keeps the automatic preference above.
607
676
 
677
+ ### Core ML sidecars on Apple
678
+
679
+ The macOS / iOS prebuilds carry the Apple Core ML (Neural Engine) sidecars
680
+ for the Supertonic vocoder and the Audio8 codec. They are presence-driven:
681
+ each stage runs on a compiled `.mlmodelc` found next to its model file
682
+ (`supertonic3-q8_0.gguf` -> `supertonic3-vocoder.mlmodelc`) and falls back to
683
+ the ggml graph when it is absent, so a model directory without sidecars
684
+ behaves exactly as before. The sidecar name drops the quantization suffix, so
685
+ one sidecar serves every tier of a model. Sidecars are not part of the
686
+ published model set yet; supply your own to opt in.
687
+
688
+ | Model | Sidecar next to the GGUF | Stage on Core ML | Runs on ggml instead | Force ggml |
689
+ | --- | --- | --- | --- | --- |
690
+ | Supertonic 1 / 2 / 3 | `<model>-vocoder.mlmodelc` | vocoder, in 64-latent-frame windows | GGUFs whose vocoder weights are stored below 8 bits (`q4_0`) | `SUPERTONIC_COREML_DISABLE=1` |
691
+ | Audio8 | `audio8-codec-decoder.mlmodelc`, beside the codec decoder GGUF | codec synthesis stack (upsampling + DAC decoder), in 64-post-frame windows; the language model stays on the ggml backend | a call that fails on the sidecar, which also retires it for every later call on that instance | `AUDIO8_COREML_DISABLE=1` |
692
+ | Chatterbox, Parler, CosyVoice3, MOSS, LavaSR | none | — | always | — |
693
+
694
+ Set the force-ggml variables in the process environment before `load()`.
695
+ Audio8 reports its codec path in `response.stats`: `codecSidecarLoaded` is 1
696
+ while the sidecar is attached and `codecOnCoreml` is 1 when that synthesis
697
+ ran its codec on it (see [Response shape](#response-shape)). The Supertonic
698
+ vocoder path is not reported in the stats.
699
+
700
+ Worth it where the GPU is consumer-class: on an Apple M4 the Supertonic
701
+ vocoder runs 1.6-2.9x faster on the Neural Engine than on Metal (1.06-1.13x
702
+ end to end). On workstation parts the GPU wins — an M3 Ultra is 0.6-0.9x —
703
+ so do not stage a sidecar there. `q4_0` models ignore the vocoder sidecar:
704
+ it carries full-precision weights and would substitute a different vocoder
705
+ rather than accelerate the quantized one.
706
+
707
+ Export the sidecars from the model GGUFs with
708
+ `engines/tts/scripts/export-supertonic-coreml.py` and
709
+ `engines/tts/scripts/export-audio8-codec-coreml.py` from the
710
+ [`qvac-fabric-speech.cpp`](https://github.com/tetherto/qvac-fabric-speech.cpp)
711
+ tree at the ref `speech-cpp` pins. Its
712
+ [Supertonic](https://github.com/tetherto/qvac-fabric-speech.cpp/blob/master/engines/tts/docs/supertonic.md#core-ml-vocoder-sidecar)
713
+ and
714
+ [Audio8](https://github.com/tetherto/qvac-fabric-speech.cpp/blob/master/engines/tts/docs/audio8.md#core-ml-codec-sidecar)
715
+ guides cover export, placement, and measurements.
716
+
608
717
  When the addon is built with `ENABLE_CUDA` — on in the published linux-x64
609
718
  prebuilds, opt-in on linux-arm64 and win32-x64 (`npm run build:cuda` or
610
719
  `bare-make generate -D ENABLE_CUDA=ON`) — the CUDA backend ships as a
@@ -676,7 +785,9 @@ and they are set the same way: the constructor and `reload()` everywhere, plus
676
785
  per call on the engines that can change them per call. The vocabulary is owned
677
786
  by tts-cpp (`include/tts-cpp/voice_controls.h`) and mirrored here; each engine
678
787
  declares the subset it supports, and an unsupported value throws naming that
679
- engine's set.
788
+ engine's set. `TTSGgml.getVoiceControls()` returns tts-cpp's vocabulary and
789
+ each engine's subset straight from the native library, without loading a model
790
+ (`{ emotions, paces, engines: { parler: { emotions, paces }, … } }`).
680
791
 
681
792
  | engine | `emotion` | `pace` | per call | exact rate knob |
682
793
  |---|---|---|---|---|
@@ -685,6 +796,7 @@ engine's set.
685
796
  | Supertonic | not supported | slow / moderate / fast | no | `speed` |
686
797
  | Chatterbox | not supported | not supported | — | `speed` |
687
798
  | Audio8 | not supported | not supported | — | — |
799
+ | MOSS | not supported | not supported | — | — |
688
800
 
689
801
  The 12 canonical emotions (case-insensitive): `command`, `anger`, `narration`,
690
802
  `conversation`, `disgust`, `fear`, `happy`, `neutral`, `proper noun`, `news`,
@@ -819,6 +931,89 @@ available, the engine falls back to CPU and sets
819
931
  `response.stats.gpuUnsupported`. Voice cloning uses the same backend and adds
820
932
  a one-off encode when a new reference recording is supplied.
821
933
 
934
+ ## MOSS
935
+
936
+ MOSS Delay is an autoregressive model that predicts 32 RVQ codebooks per 80 ms
937
+ frame on a delay pattern, and a transformer codec turns those codes back into
938
+ 24 kHz audio. The backbone and the codec's synthesis half run on every
939
+ synthesis, the analysis half only to clone a voice or run a dialogue, so a
940
+ text-only deployment can omit `files.mossCodecEncoder`. The backbones have 8B
941
+ parameters, so MOSS targets desktop hosts; mobile is not supported.
942
+
943
+ ```js
944
+ const model = new TTSGgml({
945
+ engine: TTSGgml.ENGINE_MOSS,
946
+ files: { modelDir: './models' },
947
+ referenceAudio: './voice.wav', // optional; 24 kHz, needs moss-codec-encoder-*.gguf
948
+ streamChunkTokens: 25, // optional; codec frames per chunk (2 s)
949
+ config: { language: 'en' },
950
+ opts: { stats: true }
951
+ })
952
+ await model.load()
953
+ await model.run({ input: 'Hello from a fully on-device pipeline.' })
954
+ ```
955
+
956
+ `config.language` is passed to the model as a prompt hint. The reference
957
+ recording is encoded once at load and stays fixed for the instance, so
958
+ changing voices means a new instance. `streamChunkTokens > 0` delivers audio in
959
+ chunks of that many codec frames (12.5 per second) while the backbone is still
960
+ generating; `streamFirstChunkTokens` is not supported. MOSS emits 24 kHz only,
961
+ so `config.outputSampleRate` is rejected unless it equals `24000`, and the
962
+ LavaSR enhancer / denoiser are not supported.
963
+
964
+ The speech is directable from the text itself: `[pause 2.0s]` markers insert
965
+ a silence of roughly that length, and inline Pinyin (`ni3 hao3`) or IPA
966
+ (`/həloʊ/`) steers pronunciation. `durationTokens` asks for a target length
967
+ in codec frames (12.5 per second, so `38` is about 3 s; `0` keeps the length
968
+ free). It is reloadable, and the engine rejects a target that does not fit in
969
+ its generation budget.
970
+
971
+ ```js
972
+ const model = new TTSGgml({
973
+ engine: TTSGgml.ENGINE_MOSS,
974
+ files: { modelDir: './models' },
975
+ durationTokens: 38,
976
+ config: { language: 'en' }
977
+ })
978
+ await model.load()
979
+ await model.run({ input: 'Hold on [pause 1.0s] here it comes.' })
980
+ ```
981
+
982
+ Dialogue needs the MOSS-TTSD backbone: with `dialogueReferences` set, a
983
+ `modelDir` must hold `moss-ttsd-*.gguf` (or name the file with
984
+ `files.mossBackbone`). A `modelDir` holding only the TTSD backbone also serves
985
+ plain synthesis. `dialogueReferences` takes one 24 kHz recording per
986
+ speaker, in the order the text tags them with `[S1]`, `[S2]`, and so on. The
987
+ model continues the references, so the text must open with what each
988
+ recording says, under its speaker tag, followed by the lines to generate; only
989
+ the new lines come out as audio. For the same reason a dialogue cannot be split
990
+ into sentences: `runStream()`, `runStreaming()` and `run({ streamOutput: true })`
991
+ are rejected; use `run()`, with `streamChunkTokens` for chunked audio. The references need
992
+ `files.mossCodecEncoder`, exclude `referenceAudio`, and are fixed for the
993
+ instance. They stay in the codec as causal history, so the first generated
994
+ words continue the reference voices without a seam.
995
+
996
+ ```js
997
+ const dialogue = new TTSGgml({
998
+ engine: TTSGgml.ENGINE_MOSS,
999
+ files: { modelDir: './models' },
1000
+ dialogueReferences: ['./alice.wav', './bob.wav'],
1001
+ config: { language: 'en' }
1002
+ })
1003
+ await dialogue.load()
1004
+ await dialogue.run({
1005
+ input:
1006
+ '[S1] What alice.wav says. [S2] What bob.wav says. ' +
1007
+ '[S1] Did the build finish? [S2] Yes, every test passed.'
1008
+ })
1009
+ ```
1010
+
1011
+ MOSS runs on CPU by default. `config.useGPU: true` (or a non-zero
1012
+ `nGpuLayers`) asks the engine for a GPU backend; when none is available it
1013
+ falls back to CPU and sets `response.stats.gpuUnsupported`. `backendsDir`
1014
+ reaches MOSS like every other engine, so builds that load ggml backends at
1015
+ runtime find them.
1016
+
822
1017
  ## API overview
823
1018
 
824
1019
  ### Constructor — `new TTSGgml(options)`
@@ -834,47 +1029,65 @@ a one-off encode when a new reference recording is supplied.
834
1029
  | `files.cosyvoiceS3tokModel` | string | — | CosyVoice3 speech_tokenizer_v3 GGUF; needed only with `referenceAudio` (auto-discovered in the model dir as `cosyvoice3-s3tok-*.gguf`) |
835
1030
  | `files.cosyvoiceCampplusModel` | string | — | CosyVoice3 CAM++ speaker-encoder GGUF; needed only with `referenceAudio` (auto-discovered as `cosyvoice3-campplus-*.gguf`) |
836
1031
  | `files.cosyvoiceLlmModelPath` / `cosyvoiceFlowModelPath` / `cosyvoiceHiftModelPath` | string | — | Per-component overrides for the CosyVoice3 model dir |
1032
+ | `files.cosyvoiceVocab` / `cosyvoiceMerges` / `cosyvoiceVoiceModel` | string | — | CosyVoice3 text frontend (`vocab.json`, `merges.txt`) and baked voice (`voice.gguf`) overrides; each resolves under the model dir when unset. `cosyvoiceVoiceModel` selects a different baked voice without a separate model dir (`*Path` aliases accepted) |
837
1033
  | `files.audio8Lm` | string | — | Audio8 DualAR language model GGUF (overrides `modelDir`) |
838
1034
  | `files.audio8CodecDecoder`| string | — | Audio8 codec synthesis half — codes to wav (overrides `modelDir`) |
839
1035
  | `files.audio8CodecEncoder`| string | — | Audio8 codec analysis half — wav to codes; only needed to clone a voice |
1036
+ | `files.mossBackbone` | string | — | MOSS Delay backbone GGUF: MOSS-TTS or the MOSS-TTSD dialogue checkpoint (overrides `modelDir`) |
1037
+ | `files.mossCodecDecoder` | string | — | MOSS codec synthesis half — codes to wav (overrides `modelDir`) |
1038
+ | `files.mossCodecEncoder` | string | — | MOSS codec analysis half — wav to codes; only needed to clone a voice or for `dialogueReferences` |
840
1039
  | `files.lavasrEnhancer` | string | — | LavaSR enhancer GGUF — supplying it turns on 48 kHz enhancement |
841
1040
  | `files.lavasrDenoiser` | string | — | LavaSR denoiser GGUF — supplying it turns on denoising (batch only) |
842
- | `engine` | string | auto | Force `'chatterbox'`, `'supertonic'`, `'cosyvoice3'`, `'parler'` or `'audio8'` (`TTSGgml.ENGINE_CHATTERBOX` / `ENGINE_SUPERTONIC` / `ENGINE_COSYVOICE3` / `ENGINE_PARLER` / `ENGINE_AUDIO8`); auto-detected from the GGUFs present otherwise |
843
- | `referenceAudio` | string | — | Wav to clone (Chatterbox: mono, ≥ 5 s; CosyVoice3: 0.5-30 s, multichannel downmixed to mono, needs the s3tok + campplus GGUFs; Audio8: also needs `referenceText`). Audio8 accepts it per call too |
1041
+ | `engine` | string | auto | Force `'chatterbox'`, `'supertonic'`, `'cosyvoice3'`, `'parler'`, `'audio8'` or `'moss'` (`TTSGgml.ENGINE_CHATTERBOX` / `ENGINE_SUPERTONIC` / `ENGINE_COSYVOICE3` / `ENGINE_PARLER` / `ENGINE_AUDIO8` / `ENGINE_MOSS`); auto-detected from the GGUFs present otherwise |
1042
+ | `referenceAudio` | string | — | Wav to clone (Chatterbox: mono, ≥ 5 s; CosyVoice3: 0.5-30 s, multichannel downmixed to mono, needs the s3tok + campplus GGUFs; Audio8: also needs `referenceText`; MOSS: needs `files.mossCodecEncoder`). Audio8 accepts it per call too |
844
1043
  | `referenceText` | string | — | Audio8-only: what `referenceAudio` says, verbatim. Required whenever a reference is set; accepted per call |
1044
+ | `dialogueReferences` | string[] | — | MOSS-only: one 24 kHz recording per speaker (`[S1]`, `[S2]`, ...) for dialogue synthesis; the text must open with each recording's transcript under its tag; needs `files.mossCodecEncoder`, excludes `referenceAudio`, fixed per instance |
1045
+ | `durationTokens` | number | `0` | MOSS-only: target length in codec frames (12.5 per second); `0` keeps the length free; reloadable |
845
1046
  | `voiceDir` | string | — | Pre-baked voice profile |
846
- | `seed` | number | 42 | RNG seed (CFM noise + sampling) |
1047
+ | `seed` | number | 42 | RNG seed (CFM noise + sampling); MOSS defaults to its engine's 1234 |
847
1048
  | `nGpuLayers` | number | 0 | Layers offloaded to GPU (mirrors `useGPU`; pass `99` to offload all) |
848
1049
  | `nCtx` | number | 4096 | Chatterbox T3 context limit for the prompt plus generated speech tokens (25 tokens ≈ 1 s of audio). The KV cache is allocated up front at this length, so the 4096-token default directly bounds memory. Pass `0` to use the GGUF metadata value |
849
1050
  | `kvCacheType` | string | `f16` | T3 KV-cache dtype: `f32` \| `f16` \| `q8_0`. `f16` (~50% of f32) is the safe cross-backend default. `q8_0` stores the cache at ~27% of f32 and decodes 20-30% faster on Metal, but only works on backends with a q8_0 CONT op (CPU, CUDA) — it hard-aborts the multilingual model on Metal, so it is opt-in. Turbo greedy decoding is byte-identical across all three (upstream-validated). Pass `f32` for bit-exact pre-quantisation behaviour |
850
- | `threads` | number | hw.concurrency capped at 4 | |
851
- | `streamChunkTokens` | number | 0 | **>0 enables native chunk streaming** |
1051
+ | `threads` | number | hw.concurrency capped at 4 | MOSS: engine default of 4; `0` keeps it |
1052
+ | `streamChunkTokens` | number | 0 | **>0 enables native chunk streaming** (speech tokens per chunk; Supertonic: text tokens; MOSS: codec frames per chunk, 12.5 per second) |
852
1053
  | `streamFirstChunkTokens` | number | = streamChunkTokens | Smaller first chunk for low first-audio-out |
853
- | `cfmSteps` | number | 2 | Chatterbox: 1 = faster (halved CFM cost) |
1054
+ | `streamChunkTolerancePct` | number | 20 | Supertonic-only: boundary-snap window for clause / space chunk boundaries, in percent of the chunk target |
1055
+ | `streamMinChunkTokens` | number | 30 | Supertonic-only: floor on every streamed chunk's size in text tokens; shorter trailing chunks merge into the previous one |
1056
+ | `streamLeftContextTokens` | number | 0 | Chatterbox: left-context speech tokens S3Gen keeps per streamed chunk (typically 25-50), bounding per-chunk cost on long utterances; `0` = the whole emitted prefix. CosyVoice3: reserved — the pinned engine accepts but does not read it |
1057
+ | `cfmSteps` | number | 2 | Chatterbox native-streaming CFM steps: 1 = faster (halved CFM cost) |
1058
+ | `batchCfmSteps` | number | engine default | Chatterbox-only batch CFM steps (`0` = 2-step meanflow on Turbo, the model's full schedule on multilingual). The multilingual engine does not floor it, so values well below its schedule under-integrate |
1059
+ | `nPredict` | number | 1000 | Chatterbox-only cap on speech tokens per synthesis (25 ≈ 1 s, so ~40 s by default). Longer single calls also need `nCtx` to fit, or use `maxSentenceChars` |
1060
+ | `maxSentenceChars` | number | 0 | Chatterbox-only sentence auto-split: T3 + S3Gen run per segment of at most this many bytes (the reference CLI uses 180), so `nPredict` applies per segment |
1061
+ | `crossfadeMs` | number | 30 | Chatterbox-only raised-cosine crossfade between auto-split batch segments; native streaming stays gapless |
1062
+ | `repeatPenalty` | number | 1.2 | Chatterbox-only T3 repetition penalty (`1` = off) |
1063
+ | `exaggeration` / `cfgWeight` / `minP` | number | 0.5 / 0.5 / 0 | Chatterbox multilingual only (Turbo ignores them): prosody intensity (> 0 more expressive, < 0 flatter), T3 classifier-free guidance scale (`0` = off; `cfgRate` is the separate S3Gen rate), and the min-p sampling threshold |
854
1064
  | `speed` | number | 1.0 | Speaking-rate multiplier, bounded `[0.25, 4.0]` (`< 1` slower, `> 1` faster). Both engines |
855
1065
  | `voice` / `voiceName` | string | — | Supertonic voice id (e.g. `'F1'`, `'M1'`); Parler template speaker name (e.g. `'Laura'`, `'Rohit'`) |
1066
+ | `voiceJsonPath` | string | — | Supertonic-only: external voice JSON (`{ style_ttl, style_dp }`, e.g. a cloned voice) that overrides `voice`; the engine checks the tensor sizes against the model |
1067
+ | `prewarmText` | string | — | Supertonic-only: text synthesized once at load so GPU pipelines compile, and a Core ML vocoder sidecar specializes, before the first `run()` (skipped on a plain CPU run); use a representative length. Wins over the `vulkanCacheDir` default pre-warm |
1068
+ | `vulkanDevice` | number | 0 | Supertonic / CosyVoice3 Vulkan adapter, also used by their LavaSR enhancer: `0` = first, `N` = the Nth, `-1` = auto-pick by free VRAM preferring a discrete GPU |
1069
+ | `flowCutPrompt` | boolean | `false` | CosyVoice3-only: attention-only prompt frames in the flow — faster, with output that deviates slightly from the reference |
856
1070
  | `steps` / `numInferenceSteps` | number | GGUF default | Supertonic vector-estimator CFM steps (`0` = GGUF default) |
857
1071
  | `noiseNpyPath` | string | — | Supertonic: optional fixed CFM noise `.npy` for reproducibility |
858
1072
  | `description` / `voiceDescription` | string | fallback caption | Parler-only: full free-text voice description (mutually exclusive with the template fields) |
859
1073
  | `emotion`, `pace` | string | — | Cross-engine conditioning (see [Emotion & pace](#emotion--pace-cross-engine)); an unsupported value errors listing what that engine supports |
860
1074
  | `pitch`, `expressivity`, `noise`, `reverb`, `quality` | string | — | Parler-only template fields (see [Parler descriptions](#parler-descriptions)); invalid values error listing the valid set |
861
- | `temperature` / `topK` / `topP` | number | engine defaults | Parler + Audio8 sampling knobs (omit for the engine's own defaults — Parler temp 1.0 / top-k 50, Audio8 temp 0.7 / top-k 50 / top-p 0.9; `topP` in `(0, 1]`) |
1075
+ | `temperature` / `topK` / `topP` | number | engine defaults | Parler, Audio8 and Chatterbox sampling knobs (omit for the engine's own defaults — Parler temp 1.0 / top-k 50, Audio8 temp 0.7 / top-k 50 / top-p 0.9, Chatterbox temp 0.8 / top-k 1000 / top-p 0.95 where temp `0` is greedy and top-k `0` disables the cutoff; `topP` in `(0, 1]`) |
862
1076
  | `maxFrames` | number | engine max | Parler + Audio8 generation cap in frames (Parler ~86/s of audio, Audio8 ~21.5/s); `0` = model default, Parler rejects 1–9 |
863
1077
  | `greedy` | boolean | `false` | Audio8-only: take the argmax instead of sampling; ignores `temperature` / `topK` / `topP` |
864
1078
  | `minNewTokens` | number | GGUF default | Parler-only minimum tokens before EOS (`-1` = model default) |
865
1079
  | `normalizeNumbers` | boolean | `true` | Parler-only: expand digits before tokenization (English words; script-native digits on indic) — parler voices raw digits badly |
866
- | `instruct` | object \| string | — | CosyVoice3-only: instruction controls (`dialect` / `volume` / `style`, resolved by precedence in that order) or a raw instruction string; an unknown key or invalid value throws, and it counts toward the one-instruction rule (see [CosyVoice3 instruct](#cosyvoice3-instruct)) |
1080
+ | `instruct` | object \| string | — | CosyVoice3-only: instruction controls (`dialect` / `volume` / `style`, resolved by precedence in that order) or a raw instruction string; an unknown key or invalid value throws, and it counts toward the one-instruction rule (see [CosyVoice3 instruct](#cosyvoice3-instruct)). Also accepted per call, where it replaces the configured conditioning like a per-call `emotion` / `pace` |
867
1081
  | `promptText` | string | — | CosyVoice3-only: verbatim transcript of `referenceAudio` — set for zero-shot cloning, omit for cross-lingual; without a reference it overrides the baked voice's transcript |
868
- | `streamLeftContextTokens` | number | — | CosyVoice3-only: intended native chunk-streaming left-context tokens. Reserved / not yet effective — the pinned engine accepts but does not read it |
869
1082
  | `mecabDictDir` | string | — | Chatterbox MTL Japanese (`ja`): compiled MeCab/IPAdic dictionary directory |
870
1083
  | `cangjieTsvPath` | string | — | Chatterbox MTL Chinese (`zh`): `Cangjie5_TC` TSV path |
871
1084
  | `backendsDir` | string | `resolveBackendsDir()` | Root dir the addon scans for dynamically-loaded ggml backend `.so` files. Defaults to the package's own `prebuilds/` when present, otherwise the installed platform package. Required on Android when backends ship elsewhere (e.g. inside the APK); ignored on platforms that statically link the backend |
872
1085
  | `openclCacheDir` | string | unset | Android-only: directory where the OpenCL backend persists its compiled program-binary cache. Setting it across runs avoids re-JITing the kernels on every fresh process |
873
1086
  | `vulkanCacheDir` | string | unset | Supertonic + `useGPU: true` only: writable directory where the Vulkan backend persists its compiled pipeline cache (`GGML_VK_PIPELINE_CACHE_DIR`). Moves the one-time first-dispatch pipeline-compile cost (seconds on Mali) off the first `run()` — paid once per install instead of once per process — and enables a load-time pre-warm. Fully opt-in: unset -> no cross-process cache, no pre-warm, behaviour unchanged |
874
- | `config.language` | string | `"en"` | Chatterbox MTL accepts `es/fr/de/pt/it/zh/ja/ko/...`; turbo & Supertonic are English |
875
- | `config.useGPU` | boolean | `false` | Set to `true` to route through Metal / CUDA / Vulkan / OpenCL if available. Honored for Chatterbox/Supertonic on GPU-capable hosts (including Android, per `tts-cpp`'s per-vendor allowlist); Parler is validated on Apple/Metal, linux CUDA, and Android/ARM Mali Vulkan; CosyVoice3 and Audio8 offload on Apple/Metal, desktop linux CUDA/Vulkan, Windows Vulkan, and Android OpenCL/Adreno. Unsupported backends fall back to CPU. See [Backends & GPU acceleration](#backends--gpu-acceleration) |
876
- | `config.outputSampleRate` | number | — (engine-native) | Resample the output to this rate (8000–192000 Hz). Omit to keep the engine-native rate (Chatterbox 24 kHz, Supertonic / Parler / Audio8 44.1 kHz, CosyVoice3 24 kHz, enhancer 48 kHz). Parler native chunk streaming accepts a non-native rate only with the enhancer active |
877
- | `opts.stats` | boolean | `false` | Populate `response.stats` with RTF, `backendDevice` (0=CPU, 1=GPU), `backendId` (0=CPU, 1=Metal, 2=CUDA, 3=Vulkan, 4=OpenCL, 99=other), and — when an enhancer is active — `enhancerBackendDevice` / `enhancerBackendId` |
1087
+ | `config.language` | string | `"en"` | Chatterbox MTL accepts `es/fr/de/pt/it/zh/ja/ko/...`; turbo & Supertonic are English; MOSS takes it as a prompt hint (`en`, `zh`, ...) |
1088
+ | `config.useGPU` | boolean | `false` | Set to `true` to route through Metal / CUDA / Vulkan / OpenCL if available. Honored for Chatterbox/Supertonic on GPU-capable hosts (including Android, per `tts-cpp`'s per-vendor allowlist); Parler is validated on Apple/Metal, linux CUDA, and Android/ARM Mali Vulkan; CosyVoice3 and Audio8 offload on Apple/Metal, desktop linux CUDA/Vulkan, Windows Vulkan, and Android OpenCL/Adreno; MOSS GPU offload is validated on Apple/Metal. Unsupported backends fall back to CPU. See [Backends & GPU acceleration](#backends--gpu-acceleration) |
1089
+ | `config.outputSampleRate` | number | — (engine-native) | Resample the output to this rate (8000–192000 Hz). Omit to keep the engine-native rate (Chatterbox 24 kHz, Supertonic / Parler / Audio8 44.1 kHz, CosyVoice3 24 kHz, MOSS 24 kHz, enhancer 48 kHz). Parler native chunk streaming accepts a non-native rate only with the enhancer active; MOSS accepts only its native rate |
1090
+ | `opts.stats` | boolean | `false` | Populate `response.stats` with RTF, `backendDevice` (0=CPU, 1=GPU), `backendId` (0=CPU, 1=Metal, 2=CUDA, 3=Vulkan, 4=OpenCL, 99=other), `enhancerBackendDevice` / `enhancerBackendId` and `denoiserBackendDevice` / `denoiserBackendId` (`-1` when that LavaSR stage is off), and the engine's own stage stats: Chatterbox `t3Ms` / `s3genMs` / `t3Tokens`; CosyVoice3 per-stage ms (`lmPrefillMs` … `hiftDecodeMs`, `stageTotalMs`) and work counters (`speechTokens`, `decodeSteps`, …); Audio8 per-stage ms (`prefillMs`, `fastDecodeMs`, `codecSynthMs`, …, `stageTotalMs`) |
878
1091
  | `exclusiveRun` | boolean | `false` | **Top-level** option (not under `opts`): serialize overlapping streaming runs |
879
1092
 
880
1093
  ### Methods
@@ -887,6 +1100,8 @@ a one-off encode when a new reference recording is supplied.
887
1100
  config (`language`, `useGPU`, `outputSampleRate`, …).
888
1101
  - `await model.destroy()` — `unload()` + mark this instance dead.
889
1102
  - `await model.cancel()` — best-effort cancel of any in-flight run.
1103
+ - `TTSGgml.getVoiceControls()` — static: tts-cpp's emotion / pace vocabulary
1104
+ and each engine's supported subset (see [Emotion & pace](#emotion--pace-cross-engine)).
890
1105
  - `model.run({ input, type: 'text' })` → `QvacResponse`.
891
1106
  - `model.run({ input, streamOutput: true })` → sentence-chunked
892
1107
  synthesis driven by the JS-side sentence splitter (see
@@ -913,7 +1128,7 @@ All `run*` methods return a `QvacResponse` (from `@qvac/infer-base`):
913
1128
  ```js
914
1129
  response.onUpdate(data => {
915
1130
  data.outputArray // Int16Array — mono PCM
916
- data.sampleRate // actual rate: Chatterbox 24000, Supertonic/Parler/Audio8 44100, enhancer 48000
1131
+ data.sampleRate // actual rate: Chatterbox/CosyVoice3/MOSS 24000, Supertonic/Parler/Audio8 44100, enhancer 48000
917
1132
  data.chunkIndex // present on sentence-streaming events only
918
1133
  data.sentenceChunk // present on sentence-streaming events only
919
1134
  })
@@ -924,8 +1139,10 @@ response.stats.totalTime // seconds
924
1139
  response.stats.realTimeFactor // synthesis time / audio duration
925
1140
  response.stats.audioDurationMs
926
1141
  response.stats.totalSamples
927
- response.stats.tokensPerSecond // Audio8 counts codec frames, the others characters
928
- response.stats.generatedFrames // Audio8 only: codec frames, on a fixed 46 ms grid
1142
+ response.stats.tokensPerSecond // Audio8 and MOSS count codec frames, the others characters
1143
+ response.stats.generatedFrames // Audio8 and MOSS only: codec frames (Audio8 every 46 ms, MOSS every 80 ms)
1144
+ response.stats.codecSidecarLoaded // Audio8 only, macOS / iOS: 1 while the Apple Core ML codec sidecar is attached
1145
+ response.stats.codecOnCoreml // Audio8 only: 1 when this synthesis ran the codec on that sidecar, 0 when on the ggml backend
929
1146
  response.stats.backendDevice // 0=CPU, 1=GPU
930
1147
  response.stats.backendId // 0=CPU, 1=Metal, 2=CUDA, 3=Vulkan, 4=OpenCL, 99=other
931
1148
  // present when a LavaSR enhancer is active:
@@ -933,6 +1150,11 @@ response.stats.enhancerBackendDevice // -1 none, 0 CPU, 1 GPU
933
1150
  response.stats.enhancerBackendId
934
1151
  ```
935
1152
 
1153
+ For `runStream()` and `runStreaming()`, `codecSidecarLoaded` and
1154
+ `codecOnCoreml` retain the last chunk that reported each field, including zero
1155
+ when a sidecar is retired or synthesis falls back to ggml. They are not summed;
1156
+ if no chunk reports a field, it remains absent.
1157
+
936
1158
  ### Text helpers
937
1159
 
938
1160
  The sentence splitter is available from
@@ -966,6 +1188,57 @@ const {
966
1188
  | 13010 | `FAILED_TO_STOP` |
967
1189
  | 13011 | `JOB_ALREADY_RUNNING` |
968
1190
 
1191
+ ## Assessing fit
1192
+
1193
+ `assessFit` projects a load against the memory free right now. It reads GGUF metadata and never weight data, so the registry's weightless copy of each file answers the same as the file itself and the projection can run before anything is downloaded. It is a module export, not an instance method: nothing is loaded to call it.
1194
+
1195
+ A request is a `createInstance` config plus the workload, so it carries the same keys the load takes and the engine's own config builder reads it.
1196
+
1197
+ ```js
1198
+ const TTSGgml = require('@qvac/tts-ggml')
1199
+
1200
+ const fit = TTSGgml.assessFit({
1201
+ engineType: 'supertonic',
1202
+ supertonicModelPath: '/models/supertonic.gguf',
1203
+ useGPU: true,
1204
+ textTokens: 256,
1205
+ audioSeconds: 30
1206
+ })
1207
+
1208
+ fit.status // 'fits' | 'does-not-fit' | 'error'
1209
+ fit.reason // the engine's own wording, e.g. 'model-unreadable', 'workload-too-large'
1210
+ fit.modelVariant // which pipeline was projected, e.g. 'chatterbox-t3-turbo'
1211
+ fit.deviceName
1212
+ fit.deviceBytes
1213
+ fit.weightsBytes
1214
+ fit.stateBytes
1215
+ fit.lmComputeBytes // 0 for a pipeline with no language-model stage, such as supertonic
1216
+ fit.codecComputeBytes // 0 for a pipeline with no separate codec stage
1217
+ fit.hostBytes
1218
+ fit.lavasrFileBytes // the LavaSR stages' size on disk, 0 when none was named
1219
+ fit.report
1220
+ ```
1221
+
1222
+ `engineType` picks the fitter, the same key a load uses, and is required: a fit request carries none of the file keys a load is inferred from. Each engine takes the load's own file keys, plus a workload the fit adds:
1223
+
1224
+ | `engineType` | Files | Workload |
1225
+ | --- | --- | --- |
1226
+ | `supertonic` | `supertonicModelPath` | `textTokens`, `audioSeconds`, `precision`, `f16Weights` |
1227
+ | `parler` | `parlerModelPath` | `descriptionTokens`, `promptTokens`, `maxFrames` |
1228
+ | `chatterbox` | `t3ModelPath`, `s3genModelPath` | `textTokens`, `predictTokens` |
1229
+ | `audio8` | `audio8LmPath`, `audio8CodecDecoderPath`, `audio8CodecEncoderPath` | `promptTokens`, `maxFrames`, `referenceSeconds` |
1230
+ | `cosyvoice3` | `cosyvoiceLlmModelPath`, `cosyvoiceFlowModelPath`, `cosyvoiceHiftModelPath`, `cosyvoiceVoiceModelPath` | `textTokens`, `speechTokens` |
1231
+
1232
+ Everything else comes from the load config: `nGpuLayers` and `useGPU` carry the offload intent, `nCtx` and `kvCacheType` size the chatterbox cache, `steps` takes the GGUF's own default at 0, and `vulkanDevice` and `backendsDir` place the backend. Supplying `audio8CodecEncoderPath` projects voice cloning, which the decoder alone cannot do. `marginBytes` sets the free memory that must remain for the projection to count as fitting.
1233
+
1234
+ `lavasrEnhancerPath` and `lavasrDenoiserPath` are reported as `lavasrFileBytes`, their size on disk. Those stages have no fitter, so the figure is a size on disk. It is counted in neither `deviceBytes` nor `hostBytes`.
1235
+
1236
+ `deviceSharesHostMemory` reports that the device pool is system RAM, so host bytes compete with device bytes.
1237
+
1238
+ A model the engine cannot read is `status: "error"`, as is a voice with no fitter, which reports `reason: "unsupported-engine"`. MOSS is the one voice in that state. A broken request, or a host with no native binding, throws.
1239
+
1240
+ The supertonic fitter covers the fused graph path: a validated GPU, or a CPU without the Accelerate pointwise kernels. Elsewhere it answers `compute-path-not-supported` and projects nothing.
1241
+
969
1242
  ## Examples
970
1243
 
971
1244
  Runnable demos under `examples/`:
@@ -983,12 +1256,15 @@ Runnable demos under `examples/`:
983
1256
  | `supertonic-mtl-tts.js` | Multilingual Supertonic synthesis |
984
1257
  | `supertonic-mtl-sweep-tts.js` | Multilingual Supertonic sweep across languages |
985
1258
  | `supertonic-sentence-stream-tts.js` | Supertonic sentence-level streaming |
1259
+ | `supertonic-chunk-stream-tts.js` | Supertonic native chunk streaming via `streamChunkTokens`, with streaming playback |
986
1260
  | `supertonic-enhanced.js` | Supertonic + LavaSR 48 kHz enhancement. `bare examples/supertonic-enhanced.js "Hello"` |
987
1261
  | `parler-tts.js` | Parler batch synth with voice/emotion templates. `bare examples/parler-tts.js "Hello" Laura happy` |
988
1262
  | `parler-enhanced.js` | Parler + LavaSR 48 kHz enhancement. `bare examples/parler-enhanced.js "Hello" Laura happy` |
989
1263
  | `cosyvoice-tts.js` | CosyVoice3 batch synth with the cross-engine emotion option (24 kHz; CPU by default, `--gpu` opts into Metal / desktop Vulkan / Android GPU). `bare examples/cosyvoice-tts.js --gpu "Hello" happy` |
990
1264
  | `cosyvoice-enhanced.js` | CosyVoice3 + LavaSR 48 kHz enhancement (add `--denoise` for the denoiser). `bare examples/cosyvoice-enhanced.js "Hello"` |
991
1265
  | `audio8-tts.js` | Audio8 batch synth, optionally cloning a reference. Set `QVAC_TTS_AUDIO8_GPU=1` for the desktop GPU backend (CUDA or Vulkan on linux, Vulkan on Windows). `bare examples/audio8-tts.js "Hello" voice.wav "What it says."` |
1266
+ | `moss-tts.js` | MOSS batch synth, optionally cloning a reference. Set `QVAC_TTS_MOSS_STREAM_FRAMES=25` for native chunk streaming, `QVAC_TTS_MOSS_DURATION=38` for a target length and `QVAC_TTS_MOSS_GPU=1` for the GPU backend. `bare examples/moss-tts.js "Hello" voice.wav` |
1267
+ | `moss-dialogue-tts.js` | MOSS-TTSD multi-speaker dialogue from one 24 kHz reference per speaker; the text opens with each reference's transcript. Set `QVAC_TTS_MOSS_GPU=1` for the GPU backend. `bare examples/moss-dialogue-tts.js "[S1] What alice.wav says. [S2] What bob.wav says. [S1] Hi. [S2] Hello." alice.wav bob.wav` |
992
1268
 
993
1269
  The two streaming examples feed PCM into a single long-running
994
1270
  `sox play` / `ffplay` process so chunks play back-to-back without any
@@ -999,9 +1275,10 @@ the demos still run and write the concatenated wav.
999
1275
  ## Testing
1000
1276
 
1001
1277
  ```bash
1002
- npm run test:unit # mocked binding; fast
1003
- npm run test:integration # spins up the real engine; needs models
1004
- npm run test # both
1278
+ npm run test:unit # mocked binding; fast
1279
+ npm run test:integration # spins up the real engine; needs models
1280
+ npm run test:integration:smoke # Supertonic suite only; the darwin-x64 CI leg runs this
1281
+ npm run test # both
1005
1282
  ```
1006
1283
 
1007
1284
  Integration tests scan a few candidate `models/` directories for the
@@ -1011,8 +1288,7 @@ files are absent. They cover, across the engines:
1011
1288
  * batch synthesis with full RuntimeStats,
1012
1289
  * sentence-level streaming (`runStream` / `run({ streamOutput: true })`
1013
1290
  / `runStreaming` over async iterators),
1014
- * native sub-sentence chunk streaming (Chatterbox-only via
1015
- `streamChunkTokens`),
1291
+ * native sub-sentence chunk streaming via `streamChunkTokens`,
1016
1292
  * sequential-run / fresh-instance / reload-stability behaviour,
1017
1293
  * strict GPU-backend assertion via `response.stats.backendDevice` +
1018
1294
  `backendId` (set `NO_GPU=true` to skip on CPU-only runners,
@@ -1023,6 +1299,20 @@ files are absent. They cover, across the engines:
1023
1299
  for WER against the synthesized audio (whisper-small via the
1024
1300
  `@qvac/asr-ggml` development dependency).
1025
1301
 
1302
+ On CPU the suite keeps to brief inference: single short syntheses, plus
1303
+ Supertonic's sub-second multi-sentence checks and a few capped CPU-backend
1304
+ checks (Audio8's 16-frame runs, the LavaSR denoiser composition). Tests that
1305
+ synthesize more (several runs or loads, longer text, WER) run on the GPU and
1306
+ are skipped when `NO_GPU=true`. So are the CosyVoice3, Chatterbox MTL and
1307
+ Parler f16/large tests, where one CPU synthesis already takes about a minute;
1308
+ the CPU smokes in `gpu-smoke.test.js` still run everywhere. The multi-run
1309
+ Parler tests need Metal, so off Apple they are skipped too.
1310
+
1311
+ The MOSS integration test (`moss.test.js`) does not download its GGUFs:
1312
+ set `QVAC_TEST_MOSS_MODEL_DIR` to a directory holding them, otherwise it is
1313
+ skipped. The C++ suite runs its MOSS synthesis cases when
1314
+ `QVAC_TEST_MOSS_BACKBONE_GGUF` and `QVAC_TEST_MOSS_DECODER_GGUF` are set.
1315
+
1026
1316
  To stress-test long inputs, set `INPUT_SENTENCES=medium` (or `long`)
1027
1317
  and re-run the integration suite — `addon.test.js` reads the env var to
1028
1318
  pick its sentence corpus from `test/data/sentences-{medium,long}.js`.
@@ -1089,3 +1379,7 @@ OpenCL; Xclipse and Mali use Vulkan).
1089
1379
  ## License
1090
1380
 
1091
1381
  Apache-2.0. See [LICENSE](./LICENSE).
1382
+
1383
+ ## Pocket TTS
1384
+
1385
+ See [Pocket TTS](docs/pocket-tts.md) for model conversion, addon/SDK usage, supported controls and validation commands.