@qvac/tts-ggml 0.12.0 → 0.14.0

package/README.md

@@ -17,6 +17,22 @@ LavaSR neural denoise + 48 kHz bandwidth-extension enhancement.  Unsure
 which checkpoint to stage? Start with [Choosing a model](#choosing-a-model).
 
 
+### CosyVoice3 Core ML sidecars
+
+Apple builds support optional CosyVoice3 Core ML sidecars through the native
+`speech-cpp` 2026-10-19 dependency. Export and placement details are in the
+[CosyVoice3 Core ML guide](https://github.com/tetherto/qvac-fabric-speech.cpp/blob/ea128498e23d20cd122729275e92361b11a361df/engines/tts/docs/cosyvoice3.md#core-ml-sidecars).
+Re-export existing CosyVoice3 LM sidecars with the updated exporter to include
+the cache feedback correction for consecutive predictions.
+
+The LM, flow frontend/decoder, F0 predictor and HiFT vocoder use resident
+sidecars when present and fall back to ggml when unavailable. HiFT exports
+use FP32 for waveform parity. Core ML chooses operation placement, so a
+sidecar does not guarantee ANE execution or a speed improvement.
+When a sidecar is present, `assessFit` returns an error with reason
+`coreml-memory-estimate-unavailable`; its memory usage is not projected.
+
+
 Runs in-process with a persistent native engine — the GGUFs, the S3Gen
 preload, the ggml backend, and any voice-conditioning tensors are
 loaded once and reused across every synthesis call.  GPU acceleration
@@ -29,7 +45,8 @@ Xclipse/Mali → Vulkan). Parler supports Apple/Metal, linux CUDA, and the
 validated Android paths, including Vulkan on ARM Mali (see
 [Backends & GPU acceleration](#backends--gpu-acceleration)). Audio8 supports
 CUDA/Vulkan offload on Linux and Vulkan on Windows. MOSS GPU offload is
-validated on Apple/Metal.
+validated on Apple/Metal; desktop CUDA/Vulkan validation requires the native
+and registry prerequisites described under [MOSS](#moss).
 
 [qvac-tts-cpp]: https://github.com/tetherto/qvac-fabric-speech.cpp/tree/master/engines/tts
 
@@ -674,6 +691,8 @@ Notes:
   engine runs its ggml graph on the GPU (falling back to the ggml CPU backend
   when no GPU backend initialises), a CPU engine keeps the scalar core.
   `denoiserBackendDevice` / `denoiserBackendId` in the stats report where it ran.
+- Android ARM Mali devices can use Vulkan for denoising with `config.useGPU: true`.
+  This path is validated on Pixel 9 Pro XL (Mali-G715).
 - The tts-cpp UL-UNAS forward is implemented in
   [qvac-fabric-speech.cpp#78](https://github.com/tetherto/qvac-fabric-speech.cpp/pull/78)
   (scalar CPU port, validated bit-close to the ONNX reference); it requires a
@@ -1060,6 +1079,100 @@ falls back to CPU and sets `response.stats.gpuUnsupported`.  `backendsDir`
 reaches MOSS like every other engine, so builds that load ggml backends at
 runtime find them.
 
+The same controls apply to MOSS-TTSD and MOSS-Speech. `nGpuLayers` is a
+boolean GPU request for these engines (zero = CPU, non-zero = GPU), not a
+partial layer count. The addon forwards these controls to the native engine;
+it does not choose a different backend or change the CPU default.
+`response.stats.backendId` identifies the language-model backend (2 = CUDA,
+3 = Vulkan); it is not a measurement of every codec operation.
+
+The combined desktop CUDA/Vulkan speech changes in
+[Vulkan #308](https://github.com/tetherto/qvac-fabric-speech.cpp/pull/308),
+including [CUDA #321](https://github.com/tetherto/qvac-fabric-speech.cpp/pull/321),
+merged to native master at `84b02694001cfea863167776dbc185474a816656`.
+The exact native dependency versions are declared in [vcpkg.json](vcpkg.json),
+and the registry configuration is declared in
+[vcpkg-configuration.json](vcpkg-configuration.json). The earlier native `f4d4c257`
+passed ten smaller-model CUDA/Vulkan cases; those results do not establish GPU
+validation of the final native revision or the packaged addon. Rebuild the addon
+before running its GPU tests. Native desktop results also do not establish
+Android support or speech-quality signoff.
+
+Packaged-addon CUDA/Vulkan inference still requires validation on provisioned
+GPU hosts before claiming a validated release.
+
+From `packages/tts-ggml`, resolve the checked-in manifest without building or
+downloading models (with `VCPKG_ROOT` set):
+
+```sh
+vcpkg install --dry-run --triplet x64-linux --x-feature=cuda --x-feature=vulkan
+```
+
+On a macOS host without a Linux cross-compiler, prefix that dry-run command
+with `VCPKG_FEATURE_FLAGS=-compilertracking` to check registry resolution only.
+For a bounded native build on a suitably provisioned Linux GPU host:
+
+```sh
+CMAKE_BUILD_PARALLEL_LEVEL=2 VCPKG_MAX_CONCURRENCY=2 npm run build:cuda
+```
+
+The normal PR workflow requires approval of the `fork-ci` environment for
+external forks. Obtain that approval through the existing gate; a manual
+workflow dispatch is not a substitute for fork approval.
+
+On a desktop with those prerequisites, stage the TTS and TTSD backbones,
+codec decoder/encoder and the bundled `test/reference-audio/jfk.wav`, plus
+the Speech model/codec. Set
+`QVAC_TEST_MOSS_MODEL_DIR` and `QVAC_TEST_MOSS_SPEECH_MODEL_DIR` to their
+respective directories. Run each available backend in a fresh process:
+
+```sh
+QVAC_TEST_MOSS_FULL_GPU=true TTS_CPP_GPU_BACKEND=cuda npx --no-install brittle-bare test/integration/moss.test.js test/integration/moss-speech.test.js
+QVAC_TEST_MOSS_FULL_GPU=true TTS_CPP_GPU_BACKEND=vulkan npx --no-install brittle-bare test/integration/moss.test.js test/integration/moss-speech.test.js
+```
+
+The tests check the exact pinned backend, `backendDevice` and
+`gpuUnsupported` for TTS batch/streaming/cloning, TTSD dialogue, and Speech
+text/audio replies. CPU control cases still require CPU even with a GPU pin.
+`QVAC_TEST_MOSS_FULL_GPU=true` fails before model loading unless both model
+directories, the TTS/TTSD/Speech backbones, all three codec GGUFs, and the
+reference WAV exist. It also requires an explicit backend pin (case-insensitive),
+rejects `NO_GPU=true` and mobile CPU-only cases, and checks that every required
+case completes. Run both files together without test filters: a successful full
+run has 12 passing tests, including the two completion checks, and no skips.
+Preflight only checks asset presence; actual model loading and inference must
+also pass. These commands cover TTS, TTSD and Speech, not MOSS-SoundEffect (SFX).
+To validate the fresh packaged addons on the provisioned Linux GPU runner,
+dispatch the existing integration workflow on the branch being tested:
+
+```sh
+gh workflow run integration-test-nx.yml --repo tetherto/qvac --ref <branch> -f moss_packaged_gpu=true
+```
+
+This opt-in mode retains the `release` environment gate and builds both addons
+from the dispatch SHA with build concurrency capped at two. It stages the fresh
+platform packages through `link:platform`, verifies that both CUDA and Vulkan
+plugins are present, then runs CUDA and Vulkan sequentially. Each backend must
+pass the 12 TTS/TTSD/Speech tests above, two SFX tests (including exact backend
+and no-fallback assertions for both GPU clips), and the focused ASR Spanish
+transcription test with default and hotword prompts. Fixture skips fail the run.
+CPU control cases remain part of the TTS/SFX suites.
+
+The runner stages q8_0 TTS/TTSD/Speech/SFX backbones and q5_0 Transcribe, with
+the required codec precision preserved. TTS/TTSD q8_0 are generated on the
+runner from the source checkpoints using the pinned native quantizer. The
+artifacts retain source pins, model and prebuild hashes, the fresh prebuilds,
+and per-backend logs/results; model weights are removed and never uploaded.
+This mode does not publish. A dispatched or building run is not GPU validation:
+only passing case logs from both backends establish packaged-addon coverage.
+
+Without strict mode, developer behavior is unchanged: an unpinned run accepts
+the normal GPU cascade, missing model directories skip tests, and missing
+encoder/TTSD assets omit cloning/dialogue cases. Such runs are not full GPU
+validation. `npm run test:node` covers backend selection and strict validation
+failures without loading models. Do not load these multi-GB models on
+memory-constrained hosts.
+
 ## MOSS-SoundEffect
 
 MOSS-SoundEffect-v2 generates a sound effect from a text description: a Qwen3
@@ -1348,12 +1461,14 @@ const {
 
 `assessFit` projects a load against the memory free right now. It reads GGUF metadata and never weight data, so the registry's weightless copy of each file answers the same as the file itself and the projection can run before anything is downloaded. It is a module export, not an instance method: nothing is loaded to call it.
 
+`assessFit` returns a Promise. Fits run one at a time within this addon.
+
 A request is a `createInstance` config plus the workload, so it carries the same keys the load takes and the engine's own config builder reads it.
 
 ```js
 const TTSGgml = require('@qvac/tts-ggml')
 
-const fit = TTSGgml.assessFit({
+const fit = await TTSGgml.assessFit({
   engineType: 'supertonic',
   supertonicModelPath: '/models/supertonic.gguf',
   useGPU: true,
@@ -1393,7 +1508,7 @@ Everything else comes from the load config: `nGpuLayers` and `useGPU` carry the
 
 `deviceSharesHostMemory` reports that the device pool is system RAM, so host bytes compete with device bytes.
 
-A model the engine cannot read is `status: "error"`, as is a voice with no fitter, which reports `reason: "unsupported-engine"`. MOSS-TTS, MOSS-TTSD and MOSS-SoundEffect are supported. MOSS-Speech has no SDK fit projection. A broken request, or a host with no native binding, throws.
+A model the engine cannot read resolves to `status: "error"`, as does a voice with no fitter, which reports `reason: "unsupported-engine"`. MOSS-TTS, MOSS-TTSD and MOSS-SoundEffect are supported. MOSS-Speech has no SDK fit projection. A broken request, or a host with no native binding, rejects.
 
 MOSS-TTS and MOSS-TTSD share the `moss` fitter. `promptRows` is the complete
 native prompt length, including special tokens, encoded speaker references and
@@ -1404,7 +1519,7 @@ The fitter does not open reference recordings. Use the TTSD backbone to assess
 dialogue, supplying its complete prompt and total reference workload.
 
 ```js
-const fit = TTSGgml.assessFit({
+const fit = await TTSGgml.assessFit({
   engineType: 'moss',
   mossBackbonePath: './moss-tts-delay-f16.gguf',
   mossCodecDecoderPath: './moss-codec-decoder-f16.gguf',
@@ -1426,7 +1541,7 @@ The supertonic fitter covers the fused graph path: a validated GPU, or a CPU wit
 ### MOSS-SoundEffect fit
 
 ```js
-const fit = TTSGgml.assessFit({
+const fit = await TTSGgml.assessFit({
   engineType: 'moss-sfx',
   mossSoundEffectPath: './moss-sfx-v2-q8_0.gguf',
   prompt: 'Rain falling on a tin roof.',
@@ -1581,7 +1696,8 @@ build becomes loadable once `npm run link:platform` (run by
 
 The vcpkg port is hosted in
 [`tetherto/qvac-registry-vcpkg`][registry] and pulls
-[`qvac-tts.cpp`][qvac-tts-cpp] at a pinned REF.  See
+[`qvac-fabric-speech.cpp`](https://github.com/tetherto/qvac-fabric-speech.cpp)
+(`engines/tts`) at a pinned REF.  See
 [`vcpkg-configuration.json`](./vcpkg-configuration.json) for the
 baseline commit.
 
@@ -1640,4 +1756,9 @@ Apache-2.0.  See [LICENSE](./LICENSE).
 
 ## Pocket TTS
 
+The native `speech-cpp` dependency includes Pocket Metal, CUDA, Vulkan and
+OpenCL support in the native library. The JavaScript Pocket API continues
+to require `config.useGPU: false`; updating the native dependency does not
+remove that addon validation.
+
 See [Pocket TTS](docs/pocket-tts.md) for model conversion, addon/SDK usage, supported controls and validation commands.

package/docs/pocket-tts.md

@@ -1,7 +1,9 @@
 # Pocket TTS
 
-Pocket is an English, CPU-only TTS engine implemented in the Fabric speech
-library. It streams PCM16 audio through the `@qvac/tts-ggml` addon. Python
+Pocket is an English TTS engine implemented in the Fabric speech library.
+The native library supports CPU, Metal, CUDA, Vulkan and OpenCL; the
+`@qvac/tts-ggml` JavaScript API currently accepts CPU inference only.
+It streams PCM16 audio through the addon. Python
 is used only to convert weights and benchmark the upstream reference; it is
 not needed for native synthesis. Inference plugin and public SDK integration
 will follow in a separate change after the addon release.
@@ -139,22 +141,22 @@ audio and broad subjective quality evaluations remain outside this coverage.
 
 ## Native dependencies and measured performance
 
-The manifest requires speech-cpp and ggml-speech `2026-09-15` without changing
-`vcpkg-configuration.json`. These registered versions select the merged source
-commits `a976c4d63195601fcff8bfc0c2cdf3ef36fba539` (speech-cpp) and
-`59a0ca2c2bcd7de36c27d930f79e0c94cc0c2ff3` (ggml-speech).
-[Registry #367](https://github.com/tetherto/qvac-registry-vcpkg/pull/367)
-contains these packages. Pocket resolves CPU memory planning through the
+The exact native dependency versions are declared in [vcpkg.json](../vcpkg.json).
+The Pocket GPU implementation comes from the merged
+[Fabric #338](https://github.com/tetherto/qvac-fabric-speech.cpp/pull/338)
+change, registered in
+[registry #433](https://github.com/tetherto/qvac-registry-vcpkg/pull/433).
+Pocket resolves CPU memory planning through the
 backend registry, fixing the Linux/Android unresolved `ggml_graph_plan` import.
 The native fixes are merged in [ggml #92](https://github.com/tetherto/qvac-ext-ggml/pull/92)
 and [speech #251](https://github.com/tetherto/qvac-fabric-speech.cpp/pull/251).
 
-Before the addon-only review revision, these exact native port trees passed
+Before the addon-only review revision, the original CPU-only native dependency versions passed
 [the full prebuild and integration run](https://github.com/tetherto/qvac/actions/runs/34975293366)
 at Fabric `49d864d2e1a2070a4cb9872cd0f62319fdb9e839`: nine platform prebuilds
 and seven desktop integration lanes covering CPU, Metal, Vulkan and CUDA.
 That run validates the native dependencies; it does not validate subsequent
-JavaScript review changes. The same native pins also passed all 10 Pocket
+JavaScript review changes. Those original native pins also passed all 10 Pocket
 CTests with upstream numerical fixtures, static/dynamic CPU planner tests,
 and 116 selected Supertonic Metal operations against CPU. Six paired
 old-versus-merged native audio cases produced bit-identical PCM; this is

package/examples/moss-sfx-fit.js

@@ -8,11 +8,18 @@ if (!model || !prompt || seconds === undefined) {
   throw new Error('Usage: bare examples/moss-sfx-fit.js <model.gguf> <prompt> <seconds>')
 }
 
-const fit = TTSGgml.assessFit({
-  engineType: 'moss-sfx',
-  mossSoundEffectPath: model,
-  prompt,
-  seconds: Number(seconds),
-  useGPU: false
+async function main() {
+  const fit = await TTSGgml.assessFit({
+    engineType: 'moss-sfx',
+    mossSoundEffectPath: model,
+    prompt,
+    seconds: Number(seconds),
+    useGPU: false
+  })
+  console.log(fit.report || `${fit.status}: ${fit.reason}`)
+}
+
+main().catch((err) => {
+  console.error(err)
+  process.exit(1)
 })
-console.log(fit.report || `${fit.status}: ${fit.reason}`)