@qvac/tts-ggml 0.10.1 → 0.11.0

package/README.md

@@ -10,7 +10,9 @@ Metal on Apple, CUDA/Vulkan on Linux, Vulkan on Windows, and OpenCL/Adreno
 GPU offload on Android), **Audio8**
 (DualAR + neural codec, in-process voice cloning, desktop GPU), and **MOSS**
 (MOSS-TTS Delay, 24 kHz, in-process voice cloning, native chunk streaming,
-desktop), plus optional
+desktop), **MOSS-SoundEffect** (text-to-sound-effects, 48 kHz, up to 30 s,
+desktop), and **MOSS-Speech** (speech-to-speech: a spoken question in, a
+spoken answer out, 24 kHz, desktop), plus optional
 LavaSR neural denoise + 48 kHz bandwidth-extension enhancement.  Unsure
 which checkpoint to stage? Start with [Choosing a model](#choosing-a-model).
 
@@ -74,6 +76,8 @@ obvious for each job.
 | Voice cloning at 44.1 kHz with nothing pre-baked | `audio8-lm-q8_0.gguf` + `audio8-codec-decoder-q8_0.gguf` (+ `audio8-codec-encoder-q8_0.gguf` to clone) | Clones from a reference wav **and its transcript**, encoded in-process — no enrolment step, no voice profile. Whole-utterance only (no native chunk streaming). |
 | Multilingual speech with native chunk streaming and cloning on a desktop | `moss-tts-delay-f16.gguf` + `moss-codec-decoder-f16.gguf` (+ `moss-codec-encoder-f16.gguf` to clone) | 8B backbone, desktop only. Clones from a reference wav encoded in-process (no transcript needed); native `streamChunkTokens` chunk streaming. Native 24 kHz. |
 | Multi-speaker dialogue that continues each speaker's reference voice | `moss-ttsd-f16.gguf` + `moss-codec-decoder-f16.gguf` + `moss-codec-encoder-f16.gguf` | 8B backbone, desktop only. One 24 kHz reference per speaker; the text opens with their transcripts. Native 24 kHz. |
+| Sound effects from a text description (not speech) | `moss-sfx-v2-q8_0.gguf` (or `moss-sfx-v2-f16.gguf`) | Diffusion model, desktop GPU recommended. Up to 30 s per clip, `seconds` per call; no streaming. Native 48 kHz. |
+| Answer a spoken question with speech (voice assistant) | `moss-speech-q8_0.gguf` + `moss-speech-codec-f16.gguf` | 9B model, desktop GPU. Audio in, 24 kHz audio (plus the reply text) out; conversation history, reply voice and reply length per call; no streaming. |
 | Indic languages | `parler-indic-q8_0.gguf` | 21 Indic languages; voice / emotion templates. Emotion is officially tested on 10 languages — see [Parler descriptions & emotions](#parler-descriptions--emotions) (or the upstream model card). Native 44.1 kHz. |
 | Chinese dialects (Cantonese, Sichuan, Shanghai, …) | CosyVoice3 dir (`cosyvoice3-llm-*.gguf` + flow / hift / `voice.gguf`) | Instruct-conditioned; 17 dialects via `instruct: { dialect: '…' }`. CPU, with opt-in GPU offload (Metal on Apple, Vulkan on Linux/Windows, OpenCL/Adreno on Android); native 24 kHz. |
 | Zero-shot / cross-lingual cloning (multilingual) | CosyVoice3 dir + `cosyvoice3-s3tok-*.gguf` + `cosyvoice3-campplus-*.gguf` | Clones from a reference wav; its transcript (`promptText`) selects zero-shot, omitting it selects cross-lingual (timbre only, any target language). Composes with `instruct`. |
@@ -98,6 +102,8 @@ here, so this guide does not go stale when backends change.
 | `audio8-lm-q8_0` + codec halves | Text-led (no `language` option) | ~0.7 GB (+ ~120 MB to clone) | 44.1 kHz | Reference wav + transcript, per call too | Sentence streaming |
 | `moss-tts-delay-f16` + codec halves | Multilingual (`language` hint) | ~18.8 GB (+ ~1.8 GB to clone) | 24 kHz | Reference wav, fixed per instance | Sentence + native chunk |
 | `moss-ttsd-f16` + codec halves | Multilingual (`language` hint) | ~20.5 GB | 24 kHz | One reference wav per speaker, fixed per instance | Native chunk |
+| `moss-sfx-v2-q8_0` (or `-f16`) | Sound effects from a text prompt | ~3.5 GB (`q8_0`) / ~6.4 GB (`f16`) | 48 kHz | — | None |
+| `moss-speech-q8_0` (or `-bf16`) + `moss-speech-codec-f16` | Speech-to-speech replies | ~9.7 GB (`q8_0`) / ~18.2 GB (`bf16`), + 1.3 GB codec | 24 kHz | Codec default, or a per-call reference | None |
 
 ### Legacy Supertonic v1 / v2
 
@@ -172,8 +178,9 @@ dependency, pinned to the exact `@qvac/tts-ggml` version:
 
 ## Model files
 
-Six engine families are wrapped (Chatterbox, Supertonic, Parler,
-CosyVoice3, Audio8, MOSS), each with its own GGUF layout under `models/`:
+Eight engine families are wrapped (Chatterbox, Supertonic, Parler,
+CosyVoice3, Audio8, MOSS, MOSS-SoundEffect, MOSS-Speech), each with its own GGUF layout
+under `models/`:
 
 ```
 # Chatterbox turbo (English)
@@ -233,6 +240,13 @@ moss-tts-delay-f16.gguf          (~17 GB)
 moss-ttsd-f16.gguf               (~17 GB; MOSS-TTSD dialogue backbone, optional)
 moss-codec-decoder-f16.gguf      (~1.8 GB)
 moss-codec-encoder-f16.gguf      (~1.8 GB; cloning and dialogue only)
+
+# MOSS-SoundEffect (OpenMOSS MOSS-SoundEffect-v2; 48 kHz text-to-sound-effects)
+moss-sfx-v2-q8_0.gguf            (~3.5 GB; or moss-sfx-v2-f16.gguf, ~6.4 GB)
+
+# MOSS-Speech (fnlp/MOSS-Speech; 24 kHz speech-to-speech)
+moss-speech-q8_0.gguf            (~9.7 GB; or moss-speech-bf16.gguf, ~18.2 GB)
+moss-speech-codec-f16.gguf       (~1.3 GB; carries the default reply voice)
 ```
 
 Download the registry-published Chatterbox, Supertonic, and Parler models into
@@ -244,7 +258,7 @@ npm run download-models:registry -- --group chatterbox,supertonic3
 npm run download-models:registry -- --output /path/to/models
 ```
 
-CosyVoice3, Audio8 and MOSS are not currently included in that registry command.
+CosyVoice3, Audio8, MOSS, MOSS-SoundEffect and MOSS-Speech are not currently included in that registry command.
 Stage their layouts shown above from local converted artifacts. CosyVoice3
 resolves each component by filename prefix and does not rank quantizations, so
 stage one file per component or name it explicitly with
@@ -270,7 +284,9 @@ qvac-fabric-speech.cpp (`engines/tts/scripts/convert-audio8-lm-to-gguf.py`
 and `convert-audio8-codec-to-gguf.py`) until they are published to the model
 registry alongside the other engines. The MOSS GGUFs come from
 `convert-moss-delay-to-gguf.py` and `convert-moss-codec-to-gguf.py` in the same
-directory.
+directory, the MOSS-SoundEffect GGUF from `convert-moss-sfx-to-gguf.py`, and the
+MOSS-Speech GGUFs from `convert-moss-speech-to-gguf.py` and
+`convert-moss-speech-codec-to-gguf.py`.
 
 Point the addon at a custom location via `files.modelDir` (engine
 auto-detected from the gguf filenames present), or pass explicit
@@ -281,7 +297,9 @@ auto-detected from the gguf filenames present), or pass explicit
 `files.audio8Lm` + `files.audio8CodecDecoder` (+ `files.audio8CodecEncoder`
 to clone) (Audio8) /
 `files.mossBackbone` + `files.mossCodecDecoder` (+ `files.mossCodecEncoder`
-to clone) (MOSS).
+to clone) (MOSS) /
+`files.mossSoundEffect` (MOSS-SoundEffect) /
+`files.mossSpeechModel` + `files.mossSpeechCodec` (MOSS-Speech).
 
 ## Quick start
 
@@ -688,8 +706,8 @@ published model set yet; supply your own to opt in.
 | Model | Sidecar next to the GGUF | Stage on Core ML | Runs on ggml instead | Force ggml |
 | --- | --- | --- | --- | --- |
 | Supertonic 1 / 2 / 3 | `<model>-vocoder.mlmodelc` | vocoder, in 64-latent-frame windows | GGUFs whose vocoder weights are stored below 8 bits (`q4_0`) | `SUPERTONIC_COREML_DISABLE=1` |
-| Audio8 | `audio8-codec-decoder.mlmodelc`, beside the codec decoder GGUF | codec synthesis stack (upsampling + DAC decoder), in 64-post-frame windows; the language model stays on the ggml backend | a call that fails on the sidecar, which also retires it for every later call on that instance | `AUDIO8_COREML_DISABLE=1` |
-| Chatterbox, Parler, CosyVoice3, MOSS, LavaSR | none | — | always | — |
+| Audio8 | `audio8-codec-decoder.mlmodelc`, beside the codec decoder GGUF | codec synthesis stack (upsampling + DAC decoder), in 64-post-frame windows synthesised while the language model is still generating, so only the last one is left after it; the language model stays on the ggml backend | a call that fails on the sidecar, which also retires it for every later call on that instance | `AUDIO8_COREML_DISABLE=1` |
+| Chatterbox, Parler, CosyVoice3, MOSS, MOSS-SoundEffect, MOSS-Speech, LavaSR | none | — | always | — |
 
 Set the force-ggml variables in the process environment before `load()`.
 Audio8 reports its codec path in `response.stats`: `codecSidecarLoaded` is 1
@@ -797,6 +815,8 @@ each engine's subset straight from the native library, without loading a model
 | Chatterbox | not supported | not supported | — | `speed` |
 | Audio8 | not supported | not supported | — | — |
 | MOSS | not supported | not supported | — | — |
+| MOSS-SoundEffect | not supported | not supported | — | — |
+| MOSS-Speech | not supported | not supported | — | — |
 
 The 12 canonical emotions (case-insensitive): `command`, `anger`, `narration`,
 `conversation`, `disgust`, `fear`, `happy`, `neutral`, `proper noun`, `news`,
@@ -1014,6 +1034,104 @@ falls back to CPU and sets `response.stats.gpuUnsupported`.  `backendsDir`
 reaches MOSS like every other engine, so builds that load ggml backends at
 runtime find them.
 
+## MOSS-SoundEffect
+
+MOSS-SoundEffect-v2 generates a sound effect from a text description: a Qwen3
+text encoder reads the prompt, a DiT denoises a latent over a number of
+flow-matching steps, and a DAC decoder turns it into 48 kHz mono audio of up
+to 30 seconds.  It is not a speech model, and it ships as one GGUF.  Select it
+with `engine: TTSGgml.ENGINE_MOSS_SFX` or by staging `moss-sfx-*.gguf` in the
+`modelDir`.
+
+```js
+const sfx = new TTSGgml({
+  engine: TTSGgml.ENGINE_MOSS_SFX,
+  files: { modelDir: './models' },
+  config: { useGPU: true },
+  opts: { stats: true }
+})
+await sfx.load()
+const response = await sfx.run({
+  input: 'Heavy rain falling on a tin roof with distant thunder.',
+  seconds: 10,              // (0, 30], rounded to 0.1 s; defaults to 10
+  negativePrompt: 'music',  // optional
+  steps: 100,               // optional; model default 100
+  guidance: 4,              // optional; model default 4, 1 skips the negative branch
+  shift: 5                  // optional; model default 5
+})
+```
+
+The generation controls are per call and moss-sfx-only (`steps` in the
+constructor is rejected); the output lasts `seconds` rounded to 0.1 s.  `seed` (constructor) makes a clip reproducible.  Every clip
+costs about the same whatever its length, because the model always denoises a
+30-second latent and crops it: about two minutes for 100 steps on an Apple M1
+Ultra GPU, and much longer on CPU, so fewer `steps` trade quality for time.
+There is no streaming (`runStream()`, `runStreaming()` and
+`run({ streamOutput: true })` are rejected), `config.outputSampleRate` must be
+unset or `48000`, and the LavaSR stages and voice options are not supported.
+
+## MOSS-Speech
+
+MOSS-Speech answers a spoken question with speech, with no text step in
+between: a 9B language model with separate text and audio branches reads the
+question's speech tokens and writes the reply's, and the codec speaks them in a
+24 kHz voice.  It ships as two GGUFs, the language model and the codec (which
+also carries the default reply voice).  Select it with
+`engine: TTSGgml.ENGINE_MOSS_SPEECH` or by staging `moss-speech-*.gguf` and
+`moss-speech-codec-*.gguf` in the `modelDir`; `q8_0` is preferred over `bf16`
+when both are there.
+
+```js
+const assistant = new TTSGgml({
+  engine: TTSGgml.ENGINE_MOSS_SPEECH,
+  files: { modelDir: './models' },
+  config: { useGPU: true },
+  opts: { stats: true }
+})
+await assistant.load()
+const response = await assistant.run({
+  input: '',                    // or a typed question instead of `audio`
+  audio: questionPcm,           // Int16Array or Float32Array, mono
+  sampleRate: 16000,            // of `audio`, 8000..192000
+  messages: [                   // optional earlier turns, oldest first
+    { role: 'user', text: 'Hi, I am planning a trip.' },
+    { role: 'assistant', text: 'Great, where to?' }
+  ],
+  systemPrompt: 'Answer briefly.', // optional; replaces the default system turn
+  replyVoice: voicePcm,         // optional reply voice (up to 60 s of mono PCM)
+  replyVoiceSampleRate: 24000,
+  maxReplySeconds: 20,          // optional cut of the spoken reply
+  maxNewTokens: 1000,           // optional, 1..4096
+  textReply: false,             // true: answer in text only, no audio
+  greedy: false, temperature: 0.7, topP: 0.95, topK: 20 // optional sampling
+})
+await response.onUpdate((data) => {
+  // data.outputArray: 24 kHz Int16 PCM of the spoken reply
+  // data.text: the reply's text channel (the whole answer with textReply)
+}).await()
+```
+
+When `maxNewTokens` is omitted, MOSS-Speech uses the remaining model context
+after the prompt instead of an implicit 1000-token budget. The JavaScript API
+currently accepts explicit budgets from 1 to 4096; the native engine also
+checks that the requested budget fits the remaining context. `maxReplySeconds`
+is optional and defaults to no time cut.
+
+The user turn is either `audio` (with `sampleRate`) or the run `input` text,
+not both; each entry of `messages` carries exactly one of `text` or `audio`
+(with its `sampleRate`).  Every call is one exchange: pass the earlier turns in
+`messages` to keep a conversation going.  `seed` (constructor) makes sampled
+replies reproducible.  The reply comes back in one piece: there is no streaming
+(`runStream()`, `runStreaming()`, `run({ streamOutput: true })` and
+`streamChunkTokens` are rejected), `config.outputSampleRate` must be unset or
+`24000`, and the LavaSR stages and `referenceAudio` are not supported.  On an
+Apple M1 Ultra GPU the model generates about 47 reply tokens per second in
+`q8_0` (32 in `bf16`) against the 12.5 tokens per second of speech it produces;
+preparing the default voice adds a few seconds to the first reply of an
+instance.  Runtime stats add `promptTokens`, `generatedTokens`, `replyTokens`,
+`truncated` and the per-stage `encodeMs` / `prefillMs` / `generateMs` /
+`decodeMs`.
+
 ## API overview
 
 ### Constructor — `new TTSGgml(options)`
@@ -1036,15 +1154,18 @@ runtime find them.
 | `files.mossBackbone`      | string     | —          | MOSS Delay backbone GGUF: MOSS-TTS or the MOSS-TTSD dialogue checkpoint (overrides `modelDir`) |
 | `files.mossCodecDecoder`  | string     | —          | MOSS codec synthesis half — codes to wav (overrides `modelDir`) |
 | `files.mossCodecEncoder`  | string     | —          | MOSS codec analysis half — wav to codes; only needed to clone a voice or for `dialogueReferences` |
+| `files.mossSoundEffect`   | string     | —          | MOSS-SoundEffect GGUF (`moss-sfx-*.gguf`); routes to the `moss-sfx` engine |
+| `files.mossSpeechModel`   | string     | —          | MOSS-Speech language model GGUF (`moss-speech-*.gguf`); routes to the `moss-speech` engine |
+| `files.mossSpeechCodec`   | string     | —          | MOSS-Speech codec GGUF (`moss-speech-codec-*.gguf`), required with `files.mossSpeechModel` |
 | `files.lavasrEnhancer`    | string     | —          | LavaSR enhancer GGUF — supplying it turns on 48 kHz enhancement |
 | `files.lavasrDenoiser`    | string     | —          | LavaSR denoiser GGUF — supplying it turns on denoising (batch only) |
-| `engine`                  | string     | auto       | Force `'chatterbox'`, `'supertonic'`, `'cosyvoice3'`, `'parler'`, `'audio8'` or `'moss'` (`TTSGgml.ENGINE_CHATTERBOX` / `ENGINE_SUPERTONIC` / `ENGINE_COSYVOICE3` / `ENGINE_PARLER` / `ENGINE_AUDIO8` / `ENGINE_MOSS`); auto-detected from the GGUFs present otherwise |
+| `engine`                  | string     | auto       | Force `'chatterbox'`, `'supertonic'`, `'cosyvoice3'`, `'parler'`, `'audio8'`, `'moss'`, `'moss-sfx'` or `'moss-speech'` (`TTSGgml.ENGINE_CHATTERBOX` / `ENGINE_SUPERTONIC` / `ENGINE_COSYVOICE3` / `ENGINE_PARLER` / `ENGINE_AUDIO8` / `ENGINE_MOSS` / `ENGINE_MOSS_SFX` / `ENGINE_MOSS_SPEECH`); auto-detected from the GGUFs present otherwise |
 | `referenceAudio`          | string     | —          | Wav to clone (Chatterbox: mono, ≥ 5 s; CosyVoice3: 0.5-30 s, multichannel downmixed to mono, needs the s3tok + campplus GGUFs; Audio8: also needs `referenceText`; MOSS: needs `files.mossCodecEncoder`).  Audio8 accepts it per call too |
 | `referenceText`           | string     | —          | Audio8-only: what `referenceAudio` says, verbatim.  Required whenever a reference is set; accepted per call |
 | `dialogueReferences`      | string[]   | —          | MOSS-only: one 24 kHz recording per speaker (`[S1]`, `[S2]`, ...) for dialogue synthesis; the text must open with each recording's transcript under its tag; needs `files.mossCodecEncoder`, excludes `referenceAudio`, fixed per instance |
 | `durationTokens`          | number     | `0`        | MOSS-only: target length in codec frames (12.5 per second); `0` keeps the length free; reloadable |
 | `voiceDir`                | string     | —          | Pre-baked voice profile |
-| `seed`                    | number     | 42         | RNG seed (CFM noise + sampling); MOSS defaults to its engine's 1234 |
+| `seed`                    | number     | 42         | RNG seed (CFM noise + sampling); MOSS defaults to its engine's 1234, MOSS-SoundEffect and MOSS-Speech to 0, so an unseeded prompt gives the same clip or reply every call |
 | `nGpuLayers`              | number     | 0          | Layers offloaded to GPU (mirrors `useGPU`; pass `99` to offload all) |
 | `nCtx`                    | number     | 4096       | Chatterbox T3 context limit for the prompt plus generated speech tokens (25 tokens ≈ 1 s of audio). The KV cache is allocated up front at this length, so the 4096-token default directly bounds memory. Pass `0` to use the GGUF metadata value |
 | `kvCacheType`             | string     | `f16`      | T3 KV-cache dtype: `f32` \| `f16` \| `q8_0`.  `f16` (~50% of f32) is the safe cross-backend default.  `q8_0` stores the cache at ~27% of f32 and decodes 20-30% faster on Metal, but only works on backends with a q8_0 CONT op (CPU, CUDA) — it hard-aborts the multilingual model on Metal, so it is opt-in.  Turbo greedy decoding is byte-identical across all three (upstream-validated).  Pass `f32` for bit-exact pre-quantisation behaviour |
@@ -1065,7 +1186,7 @@ runtime find them.
 | `voice` / `voiceName`     | string     | —          | Supertonic voice id (e.g. `'F1'`, `'M1'`); Parler template speaker name (e.g. `'Laura'`, `'Rohit'`) |
 | `voiceJsonPath`           | string     | —          | Supertonic-only: external voice JSON (`{ style_ttl, style_dp }`, e.g. a cloned voice) that overrides `voice`; the engine checks the tensor sizes against the model |
 | `prewarmText`             | string     | —          | Supertonic-only: text synthesized once at load so GPU pipelines compile, and a Core ML vocoder sidecar specializes, before the first `run()` (skipped on a plain CPU run); use a representative length. Wins over the `vulkanCacheDir` default pre-warm |
-| `vulkanDevice`            | number     | 0          | Supertonic / CosyVoice3 Vulkan adapter, also used by their LavaSR enhancer: `0` = first, `N` = the Nth, `-1` = auto-pick by free VRAM preferring a discrete GPU |
+| `vulkanDevice`            | number     | 0          | Supertonic / CosyVoice3 Vulkan adapter, also used by their LavaSR enhancer: `0` = first, `N` = the Nth, `-1` = auto-pick by free VRAM preferring a discrete GPU. Audio8 always auto-picks |
 | `flowCutPrompt`           | boolean    | `false`    | CosyVoice3-only: attention-only prompt frames in the flow — faster, with output that deviates slightly from the reference |
 | `steps` / `numInferenceSteps` | number | GGUF default | Supertonic vector-estimator CFM steps (`0` = GGUF default) |
 | `noiseNpyPath`            | string     | —          | Supertonic: optional fixed CFM noise `.npy` for reproducibility |
@@ -1085,8 +1206,8 @@ runtime find them.
 | `openclCacheDir`          | string     | unset      | Android-only: directory where the OpenCL backend persists its compiled program-binary cache.  Setting it across runs avoids re-JITing the kernels on every fresh process |
 | `vulkanCacheDir`          | string     | unset      | Supertonic + `useGPU: true` only: writable directory where the Vulkan backend persists its compiled pipeline cache (`GGML_VK_PIPELINE_CACHE_DIR`).  Moves the one-time first-dispatch pipeline-compile cost (seconds on Mali) off the first `run()` — paid once per install instead of once per process — and enables a load-time pre-warm.  Fully opt-in: unset -> no cross-process cache, no pre-warm, behaviour unchanged |
 | `config.language`         | string     | `"en"`     | Chatterbox MTL accepts `es/fr/de/pt/it/zh/ja/ko/...`; turbo & Supertonic are English; MOSS takes it as a prompt hint (`en`, `zh`, ...) |
-| `config.useGPU`           | boolean    | `false`    | Set to `true` to route through Metal / CUDA / Vulkan / OpenCL if available. Honored for Chatterbox/Supertonic on GPU-capable hosts (including Android, per `tts-cpp`'s per-vendor allowlist); Parler is validated on Apple/Metal, linux CUDA, and Android/ARM Mali Vulkan; CosyVoice3 and Audio8 offload on Apple/Metal, desktop linux CUDA/Vulkan, Windows Vulkan, and Android OpenCL/Adreno; MOSS GPU offload is validated on Apple/Metal. Unsupported backends fall back to CPU. See [Backends & GPU acceleration](#backends--gpu-acceleration) |
-| `config.outputSampleRate` | number     | — (engine-native) | Resample the output to this rate (8000–192000 Hz). Omit to keep the engine-native rate (Chatterbox 24 kHz, Supertonic / Parler / Audio8 44.1 kHz, CosyVoice3 24 kHz, MOSS 24 kHz, enhancer 48 kHz). Parler native chunk streaming accepts a non-native rate only with the enhancer active; MOSS accepts only its native rate |
+| `config.useGPU`           | boolean    | `false`    | Set to `true` to route through Metal / CUDA / Vulkan / OpenCL if available. Honored for Chatterbox/Supertonic on GPU-capable hosts (including Android, per `tts-cpp`'s per-vendor allowlist); Parler is validated on Apple/Metal, linux CUDA, and Android/ARM Mali Vulkan; CosyVoice3 and Audio8 offload on Apple/Metal, desktop linux CUDA/Vulkan, Windows Vulkan, and Android OpenCL/Adreno; MOSS, MOSS-SoundEffect and MOSS-Speech GPU offload is validated on Apple/Metal. Unsupported backends fall back to CPU. See [Backends & GPU acceleration](#backends--gpu-acceleration) |
+| `config.outputSampleRate` | number     | — (engine-native) | Resample the output to this rate (8000–192000 Hz). Omit to keep the engine-native rate (Chatterbox 24 kHz, Supertonic / Parler / Audio8 44.1 kHz, CosyVoice3 24 kHz, MOSS 24 kHz, MOSS-SoundEffect 48 kHz, MOSS-Speech 24 kHz, enhancer 48 kHz). Parler native chunk streaming accepts a non-native rate only with the enhancer active; MOSS, MOSS-SoundEffect and MOSS-Speech accept only their native rate |
 | `opts.stats`              | boolean    | `false`    | Populate `response.stats` with RTF, `backendDevice` (0=CPU, 1=GPU), `backendId` (0=CPU, 1=Metal, 2=CUDA, 3=Vulkan, 4=OpenCL, 99=other), `enhancerBackendDevice` / `enhancerBackendId` and `denoiserBackendDevice` / `denoiserBackendId` (`-1` when that LavaSR stage is off), and the engine's own stage stats: Chatterbox `t3Ms` / `s3genMs` / `t3Tokens`; CosyVoice3 per-stage ms (`lmPrefillMs` … `hiftDecodeMs`, `stageTotalMs`) and work counters (`speechTokens`, `decodeSteps`, …); Audio8 per-stage ms (`prefillMs`, `fastDecodeMs`, `codecSynthMs`, …, `stageTotalMs`) |
 | `exclusiveRun`            | boolean    | `false`    | **Top-level** option (not under `opts`): serialize overlapping streaming runs |
 
@@ -1103,6 +1224,14 @@ runtime find them.
 - `TTSGgml.getVoiceControls()` — static: tts-cpp's emotion / pace vocabulary
   and each engine's supported subset (see [Emotion & pace](#emotion--pace-cross-engine)).
 - `model.run({ input, type: 'text' })` → `QvacResponse`.
+- `model.run({ input, seconds, negativePrompt, steps, guidance, shift })` →
+  moss-sfx: one sound effect of `seconds` length; the controls are optional
+  and moss-sfx-only (see [MOSS-SoundEffect](#moss-soundeffect)).
+- `model.run({ input, audio, sampleRate, messages, systemPrompt, replyVoice,
+  replyVoiceSampleRate, maxReplySeconds, maxNewTokens, textReply, greedy,
+  temperature, topP, topK })` → moss-speech: one spoken (or text) reply with
+  its text in `data.text`; the fields are moss-speech-only (see
+  [MOSS-Speech](#moss-speech)).
 - `model.run({ input, streamOutput: true })` → sentence-chunked
   synthesis driven by the JS-side sentence splitter (see
   `lib/textChunker.js`).  Equivalent to `runStream(input)`.
@@ -1128,7 +1257,8 @@ All `run*` methods return a `QvacResponse` (from `@qvac/infer-base`):
 ```js
 response.onUpdate(data => {
   data.outputArray   // Int16Array — mono PCM
-  data.sampleRate    // actual rate: Chatterbox/CosyVoice3/MOSS 24000, Supertonic/Parler/Audio8 44100, enhancer 48000
+  data.sampleRate    // actual rate: Chatterbox/CosyVoice3/MOSS/MOSS-Speech 24000, Supertonic/Parler/Audio8 44100, MOSS-SoundEffect and enhancer 48000
+  data.text          // MOSS-Speech only: the reply's text
   data.chunkIndex    // present on sentence-streaming events only
   data.sentenceChunk // present on sentence-streaming events only
 })
@@ -1228,6 +1358,8 @@ fit.report
 | `chatterbox` | `t3ModelPath`, `s3genModelPath` | `textTokens`, `predictTokens` |
 | `audio8` | `audio8LmPath`, `audio8CodecDecoderPath`, `audio8CodecEncoderPath` | `promptTokens`, `maxFrames`, `referenceSeconds` |
 | `cosyvoice3` | `cosyvoiceLlmModelPath`, `cosyvoiceFlowModelPath`, `cosyvoiceHiftModelPath`, `cosyvoiceVoiceModelPath` | `textTokens`, `speechTokens` |
+| `moss` | `mossBackbonePath`, `mossCodecDecoderPath`, optional `mossCodecEncoderPath` | Required `promptRows`, `referenceSamples`, `streaming`; load controls `durationTokens`, `streamChunkTokens`, `threads` |
+| `moss-sfx` | Required `mossSoundEffectPath` | Required `prompt`, `seconds`; optional `negativePrompt`, `steps`, `guidance`, `shift`, `threads` |
 
 Everything else comes from the load config: `nGpuLayers` and `useGPU` carry the offload intent, `nCtx` and `kvCacheType` size the chatterbox cache, `steps` takes the GGUF's own default at 0, and `vulkanDevice` and `backendsDir` place the backend. Supplying `audio8CodecEncoderPath` projects voice cloning, which the decoder alone cannot do. `marginBytes` sets the free memory that must remain for the projection to count as fitting.
 
@@ -1235,10 +1367,60 @@ Everything else comes from the load config: `nGpuLayers` and `useGPU` carry the
 
 `deviceSharesHostMemory` reports that the device pool is system RAM, so host bytes compete with device bytes.
 
-A model the engine cannot read is `status: "error"`, as is a voice with no fitter, which reports `reason: "unsupported-engine"`. MOSS is the one voice in that state. A broken request, or a host with no native binding, throws.
+A model the engine cannot read is `status: "error"`, as is a voice with no fitter, which reports `reason: "unsupported-engine"`. MOSS-TTS, MOSS-TTSD and MOSS-SoundEffect are supported. MOSS-Speech has no SDK fit projection. A broken request, or a host with no native binding, throws.
+
+MOSS-TTS and MOSS-TTSD share the `moss` fitter. `promptRows` is the complete
+native prompt length, including special tokens, encoded speaker references and
+TTSD continuation rows. `referenceSamples` is the total mono reference sample
+count (0 without a reference); a positive count requires the encoder path.
+`streaming` explicitly selects native chunk streaming or batch synthesis.
+The fitter does not open reference recordings. Use the TTSD backbone to assess
+dialogue, supplying its complete prompt and total reference workload.
+
+```js
+const fit = TTSGgml.assessFit({
+  engineType: 'moss',
+  mossBackbonePath: './moss-tts-delay-f16.gguf',
+  mossCodecDecoderPath: './moss-codec-decoder-f16.gguf',
+  promptRows: 128,
+  referenceSamples: 0,
+  streaming: false,
+  useGPU: true
+})
+```
+
+The result includes LM weights and KV state, codec weights and compute, streaming
+caches when applicable, and host staging/audio memory. It sums component peaks
+as a conservative upper bound, including the reference encoder when requested;
+this may reject a workload whose actual peak is smaller. The fit follows the
+load's generation settings and does not change synthesis behavior.
 
 The supertonic fitter covers the fused graph path: a validated GPU, or a CPU without the Accelerate pointwise kernels. Elsewhere it answers `compute-path-not-supported` and projects nothing.
 
+### MOSS-SoundEffect fit
+
+```js
+const fit = TTSGgml.assessFit({
+  engineType: 'moss-sfx',
+  mossSoundEffectPath: './moss-sfx-v2-q8_0.gguf',
+  prompt: 'Rain falling on a tin roof.',
+  seconds: 8,
+  useGPU: false
+})
+console.log(fit.report)
+```
+
+The fitter uses generation's tokenizer, request validation and graph builders,
+without loading weights or generating audio. It supports full and metadata-only
+GGUF files. `negativePrompt`, `steps`, `guidance` and `shift` keep their generation
+semantics; `marginBytes` keeps the shared 256 MiB default. DiT processes the model's
+full latent duration even for a short output. Requested seconds size decoder
+windows and audio buffers. The shared compute arena is counted at its peak:
+`lmComputeBytes` covers text/DiT, `codecComputeBytes` any additional VAE demand.
+Host memory includes conditioning, temporary arrays and CPU fallback buffers.
+
+Run `bare examples/moss-sfx-fit.js ./moss-sfx-v2-q8_0.gguf "Rain on a roof." 8`.
+
 ## Examples
 
 Runnable demos under `examples/`:
@@ -1264,16 +1446,25 @@ Runnable demos under `examples/`:
 | `cosyvoice-enhanced.js` | CosyVoice3 + LavaSR 48 kHz enhancement (add `--denoise` for the denoiser). `bare examples/cosyvoice-enhanced.js "Hello"` |
 | `audio8-tts.js` | Audio8 batch synth, optionally cloning a reference. Set `QVAC_TTS_AUDIO8_GPU=1` for the desktop GPU backend (CUDA or Vulkan on linux, Vulkan on Windows). `bare examples/audio8-tts.js "Hello" voice.wav "What it says."` |
 | `moss-tts.js` | MOSS batch synth, optionally cloning a reference. Set `QVAC_TTS_MOSS_STREAM_FRAMES=25` for native chunk streaming, `QVAC_TTS_MOSS_DURATION=38` for a target length and `QVAC_TTS_MOSS_GPU=1` for the GPU backend. `bare examples/moss-tts.js "Hello" voice.wav` |
+| `moss-sound-effect.js` | MOSS-SoundEffect text-to-sound-effects at 48 kHz. Set `QVAC_TTS_MOSS_SFX_GPU=1` for the GPU backend and `QVAC_TTS_MOSS_SFX_STEPS` to trade quality for time. `bare examples/moss-sound-effect.js "Rain on a tin roof." 5` |
+| `moss-speech.js` | MOSS-Speech speech-to-speech: answers a spoken question WAV with a 24 kHz spoken reply (optionally in the voice of a second WAV). Set `QVAC_TTS_MOSS_SPEECH_GPU=1` for the GPU backend and `QVAC_TTS_MOSS_SPEECH_TEXT=1` for a text-only answer. `bare examples/moss-speech.js test/reference-audio/jfk.wav` |
 | `moss-dialogue-tts.js` | MOSS-TTSD multi-speaker dialogue from one 24 kHz reference per speaker; the text opens with each reference's transcript. Set `QVAC_TTS_MOSS_GPU=1` for the GPU backend. `bare examples/moss-dialogue-tts.js "[S1] What alice.wav says. [S2] What bob.wav says. [S1] Hi. [S2] Hello." alice.wav bob.wav` |
 
-The two streaming examples feed PCM into a single long-running
+The streaming examples feed PCM into a single long-running
 `sox play` / `ffplay` process so chunks play back-to-back without any
-per-chunk spawn gaps — install one of them (`brew install sox` or
-`brew install ffmpeg` on macOS) to enable playback.  Absent a player
-the demos still run and write the concatenated wav.
+per-chunk spawn gaps. Playback needs one of those players (`brew install sox`
+or `brew install ffmpeg` on macOS) and `bare-subprocess`, a development
+dependency: run `npm install bare-subprocess` when you run the examples from
+an installed package. Without them the demos still run and write the
+concatenated wav.
 
 ## Testing
 
+The consolidated C++ CI lane uses persistent vcpkg binaries, a package-specific
+compiler cache, and two build workers. The deterministic C++ tier remains a
+required gate. See [C++ CI configuration](../../docs/ci/nx-ci-consolidation.md#optionsci-cheat-sheet)
+for cache warming and resource settings.
+
 ```bash
 npm run test:unit               # mocked binding; fast
 npm run test:integration        # spins up the real engine; needs models
@@ -1311,12 +1502,39 @@ Parler tests need Metal, so off Apple they are skipped too.
 The MOSS integration test (`moss.test.js`) does not download its GGUFs:
 set `QVAC_TEST_MOSS_MODEL_DIR` to a directory holding them, otherwise it is
 skipped.  The C++ suite runs its MOSS synthesis cases when
-`QVAC_TEST_MOSS_BACKBONE_GGUF` and `QVAC_TEST_MOSS_DECODER_GGUF` are set.
+`QVAC_TEST_MOSS_BACKBONE_GGUF` and `QVAC_TEST_MOSS_DECODER_GGUF` are set.  The
+MOSS-SoundEffect integration test (`moss-sfx.test.js`) likewise needs
+`QVAC_TEST_MOSS_SFX_MODEL_DIR`, and the C++ suite generates a short clip when
+`QVAC_TEST_MOSS_SFX_GGUF` is set.  The MOSS-Speech integration test
+(`moss-speech.test.js`) needs `QVAC_TEST_MOSS_SPEECH_MODEL_DIR` (the model and
+its codec), and the C++ suite answers a short spoken turn when
+`QVAC_TEST_MOSS_SPEECH_GGUF` and `QVAC_TEST_MOSS_SPEECH_CODEC_GGUF` are set.
 
 To stress-test long inputs, set `INPUT_SENTENCES=medium` (or `long`)
 and re-run the integration suite — `addon.test.js` reads the env var to
 pick its sentence corpus from `test/data/sentences-{medium,long}.js`.
 
+### Fuzzing
+
+The JS-adapter config string parsers have a [Google FuzzTest][fuzztest]
+target, `tts-config-parse-fuzz`. Fuzzing is Linux-only: it needs clang with
+libFuzzer and AddressSanitizer, plus the [build-from-source](#build-from-source)
+prerequisites. The target does not link tts-cpp, so it runs with full ASan and
+LeakSanitizer.
+
+```bash
+npm run fuzz                     # bounded run of every FUZZ_TEST, as Linux CI does
+npm run fuzz:continuous          # coverage-guided, one FUZZ_TEST at a time
+npm run fuzz:continuous -- TtsConfigParseFuzz.ParseFloatNeverCrashes --fuzz_for=30m
+```
+
+Flags other than `--continuous` and `--build-dir` go to the fuzz binary. See
+[`docs/architecture/ADDON-FUZZING.md`](../../docs/architecture/ADDON-FUZZING.md)
+for the fuzzing design, the vcpkg-supplied FuzzTest stack, and how to add a
+target.
+
+[fuzztest]: https://github.com/google/fuzztest
+
 ## Build from source
 
 Prerequisites: `clang` with C++20 support, CMake ≥ 3.25,

package/examples/moss-sfx-fit.js

@@ -0,0 +1,18 @@
+'use strict'
+
+const process = require('bare-process')
+const TTSGgml = require('@qvac/tts-ggml')
+
+const [, , model, prompt, seconds] = process.argv
+if (!model || !prompt || seconds === undefined) {
+  throw new Error('Usage: bare examples/moss-sfx-fit.js <model.gguf> <prompt> <seconds>')
+}
+
+const fit = TTSGgml.assessFit({
+  engineType: 'moss-sfx',
+  mossSoundEffectPath: model,
+  prompt,
+  seconds: Number(seconds),
+  useGPU: false
+})
+console.log(fit.report || `${fit.status}: ${fit.reason}`)

package/examples/moss-sound-effect.js

@@ -0,0 +1,125 @@
+'use strict'
+
+/**
+ * MOSS-SoundEffect text-to-sound-effects generation for @qvac/tts-ggml.
+ *
+ * MOSS-SoundEffect-v2 is a flow-matching diffusion model: a Qwen3 text
+ * encoder reads the prompt, a DiT denoises a latent over a fixed number of
+ * steps, and a DAC decoder turns it into 48 kHz mono audio of up to 30
+ * seconds.  It is not a speech model: the prompt describes a sound.
+ *
+ * Usage:
+ *   bare examples/moss-sound-effect.js "description of the sound" [seconds]
+ *
+ * Examples:
+ *   bare examples/moss-sound-effect.js "Heavy rain on a tin roof with distant thunder." 10
+ *   QVAC_TTS_MOSS_SFX_GPU=1 bare examples/moss-sound-effect.js "A dog barking twice." 5
+ *   QVAC_TTS_MOSS_SFX_STEPS=50 bare examples/moss-sound-effect.js "Fast typing on a keyboard." 5
+ *
+ * Expects the MOSS-SoundEffect GGUF (moss-sfx-v2-f16.gguf or
+ * moss-sfx-v2-q8_0.gguf) under:
+ *   models/
+ * Produce it with engines/tts/scripts/convert-moss-sfx-to-gguf.py in
+ * qvac-fabric-speech.cpp until it is published to the model registry.  Every
+ * clip costs the same whatever its length (the model always denoises a
+ * 30-second latent); use a GPU for the default 100 steps.
+ */
+
+const path = require('bare-path')
+const proc = require('bare-process')
+const TTSGgml = require('../')
+const { createWav } = require('./wav-helper')
+const { setLogger, releaseLogger } = require('../addonLogging')
+
+const SFX_SAMPLE_RATE = 48000
+const DEFAULT_SECONDS = 10
+
+const argv = global.Bare ? global.Bare.argv : process.argv
+const promptArg = argv[2]
+const seconds = Number(argv[3] || DEFAULT_SECONDS)
+const steps = Number(proc.env.QVAC_TTS_MOSS_SFX_STEPS || 0)
+
+function fail(message) {
+  console.error(message)
+  if (global.Bare) global.Bare.exit(1)
+  else process.exit(1)
+}
+
+if (!promptArg || typeof promptArg !== 'string' || promptArg.trim().length === 0) {
+  fail('Usage: moss-sound-effect.js "<description of the sound>" [seconds]')
+}
+
+const pkgRoot = path.join(__dirname, '..')
+const modelDir = path.join(pkgRoot, 'models')
+
+function buildModel() {
+  return new TTSGgml({
+    engine: TTSGgml.ENGINE_MOSS_SFX,
+    files: { modelDir },
+    config: { useGPU: proc.env.QVAC_TTS_MOSS_SFX_GPU === '1' },
+    logger: console,
+    opts: { stats: true }
+  })
+}
+
+function reportStats(stats) {
+  if (!stats) return
+  console.log(
+    `Generation stats: totalTime=${stats.totalTime.toFixed(2)}s, ` +
+      `realTimeFactor=${stats.realTimeFactor.toFixed(3)}, ` +
+      `audioDuration=${stats.audioDurationMs}ms, totalSamples=${stats.totalSamples}, ` +
+      `backendDevice=${stats.backendDevice}, backendId=${stats.backendId}`
+  )
+}
+
+async function collectPcm(response) {
+  let buffer = []
+  await response
+    .onUpdate((data) => {
+      if (data && data.outputArray) buffer = buffer.concat(Array.from(data.outputArray))
+    })
+    .await()
+  return buffer
+}
+
+async function main() {
+  setLogger((priority, message) => {
+    if (priority > 1) return
+    const names = { 0: 'ERROR', 1: 'WARNING', 2: 'INFO', 3: 'DEBUG', 4: 'OFF' }
+    const name = names[priority] || 'UNKNOWN'
+    console.log(`[${new Date().toISOString()}] [C++ log] [${name}]: ${message}`)
+  })
+
+  const outputFile = path.join(__dirname, 'moss-sound-effect.wav')
+  const model = buildModel()
+
+  try {
+    console.log('Loading MOSS-SoundEffect model...')
+    await model.load()
+    console.log('Model loaded.')
+
+    console.log(`Generating ${seconds}s of: "${promptArg}"`)
+    const response = await model.run({
+      input: promptArg,
+      type: 'text',
+      seconds,
+      ...(steps > 0 ? { steps } : {})
+    })
+    const buffer = await collectPcm(response)
+
+    console.log('Generation finished!')
+    reportStats(response.stats)
+
+    createWav(buffer, SFX_SAMPLE_RATE, outputFile)
+    console.log(`Finished writing to ${outputFile}`)
+  } finally {
+    await model.unload()
+    releaseLogger()
+  }
+}
+
+main().catch((err) => {
+  console.error(err)
+  if (global.Bare) global.Bare.exit(1)
+  else process.exit(1)
+})