@qvac/audiogen-ggml 0.8.0 → 0.9.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,67 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [0.9.1] - 2026-10-28
9
+
10
+ ### Added
11
+
12
+ - Expose the full MiniMax Core ML runtime through `coremlProfileDir` and
13
+ `coremlComputeUnits` on macOS 15+, with four explicit Apple device policies.
14
+ The compiled profile stays resident across generations. Native generation
15
+ keeps its existing behavior when these options are absent
16
+ ([#5389](https://github.com/tetherto/qvac/pull/5389)).
17
+
18
+ ### Changed
19
+
20
+ - Align the shared native `speech-cpp` source to `2026-10-27` across the speech
21
+ addons ([#5397](https://github.com/tetherto/qvac/pull/5397)).
22
+ - The shared native `ggml-speech` moves to `2026-10-27` across the speech
23
+ addons, which also speeds up single-query Metal softmax on long KV caches
24
+ ([#5392](https://github.com/tetherto/qvac/pull/5392)).
25
+
26
+ ### Fixed
27
+
28
+ - Adreno GPUs (Android, OpenCL): matrix products on every quantized weight
29
+ type whose sums exceed the fp16 range no longer come back as infinity;
30
+ `ggml-speech` `2026-10-23` fixed q8_0 only
31
+ ([#5392](https://github.com/tetherto/qvac/pull/5392)).
32
+ - Metal: a fused SwiGLU mat-vec whose output overlaps its inputs, and copies
33
+ into strided destinations, no longer produce wrong values; quantized matrix
34
+ products keep F32 activations
35
+ ([#5392](https://github.com/tetherto/qvac/pull/5392)).
36
+ - Vulkan: tensor views not aligned for the speech ops fall back instead of
37
+ returning wrong data ([#5392](https://github.com/tetherto/qvac/pull/5392)).
38
+
39
+ ## [0.9.0] - 2026-10-23
40
+
41
+ ### Added
42
+
43
+ - Bounded FuzzTest coverage for the JS-adapter number parsers
44
+ (`parseInteger`, `parseFloat`, `checkedInteger`, `checkedFloat`)
45
+ ([#4606](https://github.com/tetherto/qvac/pull/4606)). Linux C++ CI runs the
46
+ suite after the native build's caches are saved; this addon has no
47
+ addon-level C++ unit tests yet. The parsers compile without audiogen-cpp, so
48
+ ASan and LeakSanitizer stay at full strength. No public addon API changes.
49
+
50
+ ### Changed
51
+
52
+ - Align the shared native `speech-cpp` source pin to `2026-10-20#1` across
53
+ the speech addons so they use the same source revision
54
+ ([#5062](https://github.com/tetherto/qvac/pull/5062),
55
+ [#5194](https://github.com/tetherto/qvac/pull/5194),
56
+ [#5245](https://github.com/tetherto/qvac/pull/5245)).
57
+ - Move the shared native `speech-cpp` source pin to `2026-10-23#2` across
58
+ the speech addons so they use the same source revision
59
+ ([#5326](https://github.com/tetherto/qvac/pull/5326)).
60
+
61
+ ### Fixed
62
+
63
+ - Adreno GPUs (Android, OpenCL): q8_0 matrix products whose sums exceed
64
+ the fp16 range no longer come back as infinity, and attention over 32 or
65
+ more tokens with unpadded lengths is computed correctly. The shared native
66
+ `ggml-speech` moves to `2026-10-23` across the speech addons
67
+ ([#5326](https://github.com/tetherto/qvac/pull/5326)).
68
+
8
69
  ## [0.8.0] - 2026-10-14
9
70
 
10
71
  ### Breaking
@@ -203,7 +264,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
203
264
  cause for clearer diagnostics. Without this the wrong module was returned
204
265
  verbatim and the first symptom was `createInstance is not a function` deep in
205
266
  a model load. Local source-build behaviour is unchanged.
206
-
267
+
207
268
  ### Fixed
208
269
 
209
270
  - Mobile platform packages (`@qvac/audiogen-ggml-android-arm64`,
package/README.md CHANGED
@@ -56,13 +56,13 @@ only. The native prebuild for each desktop host lives in a version-locked
56
56
  platform package selected at install time through `os`/`cpu` filtered
57
57
  `optionalDependencies`:
58
58
 
59
- | Host | Package |
60
- | --- | --- |
61
- | linux-x64 (glibc) | `@qvac/audiogen-ggml-linux-x64` |
62
- | linux-arm64 (glibc) | `@qvac/audiogen-ggml-linux-arm64` |
63
- | darwin-arm64 | `@qvac/audiogen-ggml-darwin-arm64` |
64
- | darwin-x64 | `@qvac/audiogen-ggml-darwin-x64` |
65
- | win32-x64 | `@qvac/audiogen-ggml-win32-x64` |
59
+ | Host | Package |
60
+ | ------------------- | ---------------------------------- |
61
+ | linux-x64 (glibc) | `@qvac/audiogen-ggml-linux-x64` |
62
+ | linux-arm64 (glibc) | `@qvac/audiogen-ggml-linux-arm64` |
63
+ | darwin-arm64 | `@qvac/audiogen-ggml-darwin-arm64` |
64
+ | darwin-x64 | `@qvac/audiogen-ggml-darwin-x64` |
65
+ | win32-x64 | `@qvac/audiogen-ggml-win32-x64` |
66
66
 
67
67
  Do not depend on desktop platform packages directly. Supported installers are
68
68
  npm 7+, pnpm, bun, and Yarn Berry. Yarn v1 and `--omit=optional` installs skip
@@ -78,10 +78,10 @@ and `optionalDependencies` filtering can never select them. Mobile
78
78
  applications must declare the target's platform package as a direct
79
79
  dependency, pinned to the exact `@qvac/audiogen-ggml` version:
80
80
 
81
- | Target | Package |
82
- | --- | --- |
83
- | android-arm64 | `@qvac/audiogen-ggml-android-arm64` |
84
- | ios (device + simulators) | `@qvac/audiogen-ggml-ios` |
81
+ | Target | Package |
82
+ | ------------------------- | ----------------------------------- |
83
+ | android-arm64 | `@qvac/audiogen-ggml-android-arm64` |
84
+ | ios (device + simulators) | `@qvac/audiogen-ggml-ios` |
85
85
 
86
86
  ```json
87
87
  {
@@ -165,6 +165,43 @@ the engine time into `arMs`, `conditionMs`, `flowMs` and `vocoderMs`, and
165
165
  the song ended on its own. See
166
166
  [`examples/generate-music-minimax.js`](examples/generate-music-minimax.js).
167
167
 
168
+ ### MiniMax full Core ML on macOS
169
+
170
+ On macOS 15+, `speech-cpp[audiogen]` includes the full Core ML runtime. Use
171
+ the original F16 GGUF pair and its matching compiled profile:
172
+
173
+ ```js
174
+ const generator = new AudioGen({
175
+ engine: ENGINE_MINIMAX,
176
+ files: { lmModel: '/models/mm3-lm-f16.gguf', synthModel: '/models/mm3-synth-f16.gguf' },
177
+ config: { coremlProfileDir: '/models/minimax-coreml', coremlComputeUnits: 'CPU_AND_NE' }
178
+ })
179
+ ```
180
+
181
+ Set both options explicitly. `CPU_ONLY`, `CPU_AND_GPU`, `CPU_AND_NE`, and
182
+ `ALL` select Apple's allowed compute units; they do not guarantee that Apple
183
+ places every operation on a particular device. They take precedence over the
184
+ native `device`/`useGPU` selection for this route. Without the pair, MiniMax
185
+ uses its existing native backend. An explicit Core ML request fails if the
186
+ build, OS, or profile cannot support it; it does not silently switch backends.
187
+
188
+ The profile directory belongs to the inference host and contains `manifest.json`
189
+ and all compiled LM, depth, conditioner, DiT, and vocoder programs. Keep its
190
+ weights and dimensions matched to the GGUF pair. The compiled LM cache profile
191
+ must accommodate the prompt and generation; no request is truncated to fit it.
192
+ The programs stay resident between generations and are released on unload.
193
+ Core ML returns `backendId: 99` (`other`); native-only `backendDevice` and
194
+ `gpuFallbackReason` are omitted. Memory preflight returns
195
+ `coreml-memory-estimate-unavailable` for this route.
196
+
197
+ CPU/ANE initial-logit parity against the native reference is still under
198
+ validation; the measured device policies are not a claim of identical sampled
199
+ music or faster generation. See the native engine's
200
+ [full Core ML validation guide](https://github.com/tetherto/qvac-fabric-speech.cpp/blob/master/engines/audiogen/docs/minimax-coreml-validation.md).
201
+
202
+ The example accepts `AUDIOGEN_COREML_PROFILE_DIR` and
203
+ `AUDIOGEN_COREML_COMPUTE_UNITS` with the same explicit pair.
204
+
168
205
  ### 1. Simplest case — an instrumental
169
206
 
170
207
  ```js
@@ -210,7 +247,7 @@ await gen.destroy()
210
247
  > Progress items arrive live during generation. The current native engine emits
211
248
  > one PCM item after generation, not incremental audio chunks. `await()` resolves
212
249
  > with terminal stats. `backendDevice` /
213
- > `backendId` report the backend the engine *resolved to*, not the one requested,
250
+ > `backendId` report the backend the engine _resolved to_, not the one requested,
214
251
  > so a `useGPU: true` run that fell back to the CPU is detectable. Use
215
252
  > `audiogenBackendName(stats.backendId)` rather than copying the code table
216
253
  > above; it returns `undefined` for an id this version does not know.
@@ -245,13 +282,13 @@ Pass `lyrics` for vocals, and steer the LM with musical hints — `bpm`,
245
282
 
246
283
  ```js
247
284
  const response = await gen.run('energetic cumbia, brass stabs, live percussion, party vibe', {
248
- vocalLanguage: 'es', // language the vocals are sung in
249
- bpm: 98, // tempo
250
- keyscale: 'A minor', // key / scale
251
- timesignature: '4/4', // time signature
285
+ vocalLanguage: 'es', // language the vocals are sung in
286
+ bpm: 98, // tempo
287
+ keyscale: 'A minor', // key / scale
288
+ timesignature: '4/4', // time signature
252
289
  augmentCaptionWithMetadata: true, // reinforce these hints in the conditioning caption
253
- duration: 150, // target length in seconds (omit => the LM decides)
254
- seed: 42, // reproducible run
290
+ duration: 150, // target length in seconds (omit => the LM decides)
291
+ seed: 42, // reproducible run
255
292
  lyrics: `[verse]
256
293
  Suena el tambor y el barrio se enciende
257
294
  la noche es joven y la gente lo entiende
@@ -358,10 +395,11 @@ set is kept. Leave `lyrics` unset for LM-written vocals, or pass
358
395
  `'[Instrumental]'` for an instrumental song.
359
396
 
360
397
  ```js
361
- const response = await gen.run(
362
- 'a romantic modern salsa with male lead vocals for a wedding',
363
- { simpleMode: true, duration: 0, seed: 4242 }
364
- )
398
+ const response = await gen.run('a romantic modern salsa with male lead vocals for a wedding', {
399
+ simpleMode: true,
400
+ duration: 0,
401
+ seed: 4242
402
+ })
365
403
  ```
366
404
 
367
405
  Or end to end from the repo:
@@ -543,12 +581,14 @@ const fs = require('fs')
543
581
  const pcmChunks = []
544
582
  for await (const item of response.iterate()) {
545
583
  if (!('outputArray' in item)) continue
546
- pcmChunks.push(Buffer.from(
547
- item.outputArray.buffer.slice(
548
- item.outputArray.byteOffset,
549
- item.outputArray.byteOffset + item.outputArray.byteLength
584
+ pcmChunks.push(
585
+ Buffer.from(
586
+ item.outputArray.buffer.slice(
587
+ item.outputArray.byteOffset,
588
+ item.outputArray.byteOffset + item.outputArray.byteLength
589
+ )
550
590
  )
551
- ))
591
+ )
552
592
  }
553
593
  const pcmBuffer = Buffer.concat(pcmChunks)
554
594
 
@@ -571,27 +611,27 @@ for (const f of files) fs.writeFileSync(`song.${f.extension}`, f.data)
571
611
  vendored for `@qvac/decoder-audio`), so the matching encoder must be compiled
572
612
  into that build.
573
613
 
574
- | `format` | Container / codec | Extension | MIME | Notes |
575
- |----------|-------------------|-----------|------|-------|
576
- | `pcm` | raw interleaved Int16 | `pcm` | `audio/L16` | no container, dependency-free |
577
- | `wav` | WAV / PCM s16le | `wav` | `audio/wav` | dependency-free (pure-JS header) |
578
- | `flac` | FLAC | `flac` | `audio/flac` | lossless |
579
- | `alac` | MP4 / ALAC | `m4a` | `audio/mp4` | Apple Lossless (shares the `.m4a` extension) |
580
- | `aiff` | AIFF / PCM s16be | `aiff` | `audio/aiff` | uncompressed |
581
- | `caf` | CAF / PCM s16le | `caf` | `audio/x-caf` | Apple Core Audio Format, uncompressed |
582
- | `m4a` | MP4 / AAC | `m4a` | `audio/mp4` | AAC in an MP4/M4A container |
583
- | `aac` | ADTS / AAC | `aac` | `audio/aac` | raw AAC stream |
584
- | `opus` | Ogg / Opus | `opus` | `audio/opus` | resampled to 48 kHz (libopus requirement) |
585
- | `ogg` | Ogg / Vorbis | `ogg` | `audio/ogg` | Vorbis |
586
- | `ac3` | AC-3 | `ac3` | `audio/ac3` | Dolby Digital |
587
- | `wma` | ASF / WMA v2 | `wma` | `audio/x-ms-wma` | Windows Media Audio |
588
- | `mp2` | MPEG-1 Layer II | `mp2` | `audio/mpeg` | legacy MPEG audio |
614
+ | `format` | Container / codec | Extension | MIME | Notes |
615
+ | -------- | --------------------- | --------- | ---------------- | -------------------------------------------- |
616
+ | `pcm` | raw interleaved Int16 | `pcm` | `audio/L16` | no container, dependency-free |
617
+ | `wav` | WAV / PCM s16le | `wav` | `audio/wav` | dependency-free (pure-JS header) |
618
+ | `flac` | FLAC | `flac` | `audio/flac` | lossless |
619
+ | `alac` | MP4 / ALAC | `m4a` | `audio/mp4` | Apple Lossless (shares the `.m4a` extension) |
620
+ | `aiff` | AIFF / PCM s16be | `aiff` | `audio/aiff` | uncompressed |
621
+ | `caf` | CAF / PCM s16le | `caf` | `audio/x-caf` | Apple Core Audio Format, uncompressed |
622
+ | `m4a` | MP4 / AAC | `m4a` | `audio/mp4` | AAC in an MP4/M4A container |
623
+ | `aac` | ADTS / AAC | `aac` | `audio/aac` | raw AAC stream |
624
+ | `opus` | Ogg / Opus | `opus` | `audio/opus` | resampled to 48 kHz (libopus requirement) |
625
+ | `ogg` | Ogg / Vorbis | `ogg` | `audio/ogg` | Vorbis |
626
+ | `ac3` | AC-3 | `ac3` | `audio/ac3` | Dolby Digital |
627
+ | `wma` | ASF / WMA v2 | `wma` | `audio/x-ms-wma` | Windows Media Audio |
628
+ | `mp2` | MPEG-1 Layer II | `mp2` | `audio/mpeg` | legacy MPEG audio |
589
629
 
590
630
  > `mp3` is intentionally **not** listed: the vendored FFmpeg build ships no MP3
591
631
  > encoder (`libmp3lame`). Unknown/unsupported formats throw.
592
632
  >
593
633
  > `AudioGen.encode(pcm, formats, opts)` returns `{ format, data, extension,
594
- > mimeType }` for a single format, or an array of those (input order) for an
634
+ mimeType }` for a single format, or an array of those (input order) for an
595
635
  > array. `OUTPUT_FORMATS` exports the full allowed list.
596
636
 
597
637
  See [`examples/generate-music.js`](examples/generate-music.js) for a full,
@@ -606,60 +646,61 @@ runnable end-to-end script (`npm run example`).
606
646
 
607
647
  `files` — model paths:
608
648
 
609
- | Option | Meaning |
610
- |--------|---------|
611
- | `modelDir` | Folder holding the GGUFs (stages auto-classified by name). |
612
- | `ditVariant` | Which DiT to load from `modelDir`: `turbo-q4` \| `turbo-q8` \| `sft`. |
613
- | `textEncModel` / `lmModel` / `ditModel` / `vaeModel` | Explicit per-stage paths (override `modelDir`). |
614
- | `lmModel` / `synthModel` | Explicit MiniMax-Music3 pair (override `modelDir`). |
649
+ | Option | Meaning |
650
+ | ---------------------------------------------------- | --------------------------------------------------------------------- |
651
+ | `modelDir` | Folder holding the GGUFs (stages auto-classified by name). |
652
+ | `ditVariant` | Which DiT to load from `modelDir`: `turbo-q4` \| `turbo-q8` \| `sft`. |
653
+ | `textEncModel` / `lmModel` / `ditModel` / `vaeModel` | Explicit per-stage paths (override `modelDir`). |
654
+ | `lmModel` / `synthModel` | Explicit MiniMax-Music3 pair (override `modelDir`). |
615
655
 
616
656
  `config` — runtime knobs:
617
657
 
618
- | Option | Meaning |
619
- |--------|---------|
620
- | `useGPU` | Run on GPU (Metal / CUDA / Vulkan, including Android Mali); falls back to CPU. |
621
- | `device` | MiniMax only, instead of `useGPU`: `gpu` fails `load()` without a usable GPU, `auto` falls back to the CPU, `cpu` never uses a GPU. |
622
- | `inferenceSteps` / `shift` | Advanced; leave unset to auto-tune per DiT. `run()` can override both per call. |
623
- | `cfgScale` | Default MiniMax flow guidance scale; `0` uses the model default. |
624
- | `nGpuLayers` | GPU layers to offload when `useGPU` is set (99 = all). |
625
- | `threads` | CPU thread count (0 / unset = hardware default). |
626
- | `backendsDir` | Advanced; directory scanned, as given, for dlopen'd ggml backend modules. Unset scans the addon's own, next to its native module, which is where the platform package and `bare-link` put them. |
658
+ | Option | Meaning |
659
+ | ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
660
+ | `useGPU` | Run on GPU (Metal / CUDA / Vulkan, including Android Mali); falls back to CPU. |
661
+ | `coremlProfileDir` / `coremlComputeUnits` | MiniMax full Core ML profile and explicit Apple device policy; both required together, macOS 15+. |
662
+ | `device` | MiniMax only, instead of `useGPU`: `gpu` fails `load()` without a usable GPU, `auto` falls back to the CPU, `cpu` never uses a GPU. |
663
+ | `inferenceSteps` / `shift` | Advanced; leave unset to auto-tune per DiT. `run()` can override both per call. |
664
+ | `cfgScale` | Default MiniMax flow guidance scale; `0` uses the model default. |
665
+ | `nGpuLayers` | GPU layers to offload when `useGPU` is set (99 = all). |
666
+ | `threads` | CPU thread count (0 / unset = hardware default). |
667
+ | `backendsDir` | Advanced; directory scanned, as given, for dlopen'd ggml backend modules. Unset scans the addon's own, next to its native module, which is where the platform package and `bare-link` put them. |
627
668
 
628
669
  `logger` — an optional object implementing `error`/`warn`/`info`/`debug`,
629
670
  wrapped by a level-gated `QvacLogger`.
630
671
 
631
672
  **`run(caption, opts)`** returns a `QvacResponse`.
632
673
 
633
- | Option | Meaning |
634
- |--------|---------|
635
- | `lyrics` | Lyrics text; use `[Instrumental]` for no vocals. |
636
- | `vocalLanguage` | Vocal language hint. |
637
- | `bpm` | Tempo in beats per minute. |
638
- | `keyscale` | Key and scale, such as `C minor`. |
639
- | `timesignature` | Time signature, such as `4/4`. |
640
- | `augmentCaptionWithMetadata` | Append BPM/tempo, time signature, and key guidance to the internal conditioning caption; defaults to `false`. |
641
- | `duration` | Target length in seconds; omit to let the LM decide. |
642
- | `maxFrames` | MiniMax semantic-frame cap; cannot be combined with `duration`. |
643
- | `inferenceSteps` | Per-run diffusion steps (`0`–`1000`); omit to keep `config.inferenceSteps`. |
644
- | `shift` | ACE-Step per-run DiT timestep shift; omit to keep `config.shift`. |
645
- | `cfgScale` | Per-run MiniMax flow guidance scale. |
646
- | `seed` | RNG seed for reproducible generation. |
647
- | `lmTemperature` / `lmTopP` / `lmTopK` / `lmCfgScale` | LM sampling controls. |
648
- | `lmPhase1` | Allow the LM to infer missing metadata before generating semantic codes. |
649
- | `simpleMode` | Expand the caption query into a full request (caption, lyrics, unset metadata) before synthesis. |
650
- | `rewriteQuery` | LM FORMAT pass: rewrite the caption into a detailed description, preserving the lyric content. Requires real `lyrics` and `taskType: 'text2music'`; mutually exclusive with `simpleMode`. |
651
- | `normalizeLoudness` | Percentile loudness normalization of generated audio (default `true`); edits are never normalized. |
652
- | `generateLrc` | Synchronized lyric timestamps: `stats.lrc` (LRC text) + `stats.lyricsScore`. Requires lyrics and `taskType: 'text2music'`. |
653
- | `computeQualityScore` | Teacher-forced LM quality score of the generated codes; `stats.qualityScore` in `[0, 1]`. Requires `taskType: 'text2music'`. |
654
- | `dcwEnabled` / `dcwScaler` / `dcwHighScaler` | Haar DCW correction controls. |
655
- | `audioCodes` | Frozen ACE-Step semantic codes as an `Int32Array`; skips the LM. |
656
- | `referenceAudio` | Optional finite, normalized, interleaved stereo 48 kHz `Float32Array` used for timbre conditioning. |
657
- | `sourceAudio` | Source PCM in the same format; required by cover and lego tasks. |
658
- | `taskType` | `text2music` (default), `cover-nofsq`, `lego`, or reserved `cover`. |
659
- | `track` | Lego target layer; required when `taskType` is `lego`. |
660
- | `guidanceScale` | DiT classifier-free guidance; `0` (default) auto-resolves to `1.0` on turbo and `7.0` on base/sft. |
661
- | `audioCoverStrength` | Fraction of the run that follows the source, from `0` to `1` (default `1`). |
662
- | `coverNoiseStrength` | Initial source/noise blend from `0` to `1`. |
674
+ | Option | Meaning |
675
+ | ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
676
+ | `lyrics` | Lyrics text; use `[Instrumental]` for no vocals. |
677
+ | `vocalLanguage` | Vocal language hint. |
678
+ | `bpm` | Tempo in beats per minute. |
679
+ | `keyscale` | Key and scale, such as `C minor`. |
680
+ | `timesignature` | Time signature, such as `4/4`. |
681
+ | `augmentCaptionWithMetadata` | Append BPM/tempo, time signature, and key guidance to the internal conditioning caption; defaults to `false`. |
682
+ | `duration` | Target length in seconds; omit to let the LM decide. |
683
+ | `maxFrames` | MiniMax semantic-frame cap; cannot be combined with `duration`. |
684
+ | `inferenceSteps` | Per-run diffusion steps (`0`–`1000`); omit to keep `config.inferenceSteps`. |
685
+ | `shift` | ACE-Step per-run DiT timestep shift; omit to keep `config.shift`. |
686
+ | `cfgScale` | Per-run MiniMax flow guidance scale. |
687
+ | `seed` | RNG seed for reproducible generation. |
688
+ | `lmTemperature` / `lmTopP` / `lmTopK` / `lmCfgScale` | LM sampling controls. |
689
+ | `lmPhase1` | Allow the LM to infer missing metadata before generating semantic codes. |
690
+ | `simpleMode` | Expand the caption query into a full request (caption, lyrics, unset metadata) before synthesis. |
691
+ | `rewriteQuery` | LM FORMAT pass: rewrite the caption into a detailed description, preserving the lyric content. Requires real `lyrics` and `taskType: 'text2music'`; mutually exclusive with `simpleMode`. |
692
+ | `normalizeLoudness` | Percentile loudness normalization of generated audio (default `true`); edits are never normalized. |
693
+ | `generateLrc` | Synchronized lyric timestamps: `stats.lrc` (LRC text) + `stats.lyricsScore`. Requires lyrics and `taskType: 'text2music'`. |
694
+ | `computeQualityScore` | Teacher-forced LM quality score of the generated codes; `stats.qualityScore` in `[0, 1]`. Requires `taskType: 'text2music'`. |
695
+ | `dcwEnabled` / `dcwScaler` / `dcwHighScaler` | Haar DCW correction controls. |
696
+ | `audioCodes` | Frozen ACE-Step semantic codes as an `Int32Array`; skips the LM. |
697
+ | `referenceAudio` | Optional finite, normalized, interleaved stereo 48 kHz `Float32Array` used for timbre conditioning. |
698
+ | `sourceAudio` | Source PCM in the same format; required by cover and lego tasks. |
699
+ | `taskType` | `text2music` (default), `cover-nofsq`, `lego`, or reserved `cover`. |
700
+ | `track` | Lego target layer; required when `taskType` is `lego`. |
701
+ | `guidanceScale` | DiT classifier-free guidance; `0` (default) auto-resolves to `1.0` on turbo and `7.0` on base/sft. |
702
+ | `audioCoverStrength` | Fraction of the run that follows the source, from `0` to `1` (default `1`). |
703
+ | `coverNoiseStrength` | Initial source/noise blend from `0` to `1`. |
663
704
 
664
705
  ## Models
665
706
 
@@ -676,14 +717,14 @@ MiniMax-Music3 Community License.
676
717
  Four stages. Three are fixed; only the DiT changes, so you pick it with
677
718
  `ditVariant`.
678
719
 
679
- | Stage | GGUF | Size | Notes |
680
- |------|------|------|-------|
681
- | text-enc | Qwen3-Embedding-0.6B-Q8_0 | 748 MB | fixed |
682
- | LM | acestep-5Hz-lm-0.6B-Q8_0 | 677 MB | fixed |
683
- | VAE | vae-BF16 | 322 MB | fixed |
684
- | DiT | acestep-v15-turbo-Q4_K_M | 1.35 GB | `ditVariant: 'turbo-q4'` (fastest / smallest) |
685
- | DiT | acestep-v15-turbo-Q8_0 | 2.37 GB | `ditVariant: 'turbo-q8'` (higher precision) |
686
- | DiT | acestep-v15-sft-Q8_0 | 2.37 GB | `ditVariant: 'sft'` (50-step, non-turbo) |
720
+ | Stage | GGUF | Size | Notes |
721
+ | -------- | ------------------------- | ------- | --------------------------------------------- |
722
+ | text-enc | Qwen3-Embedding-0.6B-Q8_0 | 748 MB | fixed |
723
+ | LM | acestep-5Hz-lm-0.6B-Q8_0 | 677 MB | fixed |
724
+ | VAE | vae-BF16 | 322 MB | fixed |
725
+ | DiT | acestep-v15-turbo-Q4_K_M | 1.35 GB | `ditVariant: 'turbo-q4'` (fastest / smallest) |
726
+ | DiT | acestep-v15-turbo-Q8_0 | 2.37 GB | `ditVariant: 'turbo-q8'` (higher precision) |
727
+ | DiT | acestep-v15-sft-Q8_0 | 2.37 GB | `ditVariant: 'sft'` (50-step, non-turbo) |
687
728
 
688
729
  Downloading the GGUFs is the caller's job (the qvac SDK's `resolveModelPath`, a
689
730
  download script, etc.) — the addon only ever receives a local path. `models.js`
@@ -736,18 +777,18 @@ fit.report
736
777
  `modelsDir` holds the four stage GGUFs; `textEncoderPath`, `lmPath`, `ditPath`
737
778
  and `vaePath` name them individually and win over it.
738
779
 
739
- | Option | Description |
740
- | --- | --- |
741
- | `durationSeconds` | Longest single generation the projection must accommodate. |
742
- | `textTokens`, `lyricTokens` | The prompt the projection is sized for. |
743
- | `lmPromptTokens` | 0 derives it from the text and lyric budgets. |
744
- | `lmMaxNewTokens` | 0 derives it from the duration, as the pipeline does. |
745
- | `lmCfgScale`, `guidanceScale` | 0 picks CFG or no CFG from the checkpoint. |
746
- | `withSourceAudio` | Projects the extra VAE-encoder phase a cover or reference request loads. |
747
- | `keepStages` | `-1` mirrors the engine, `0` forces the staged projection, `1` all-resident. |
748
- | `gpuLayers` | Greater than 0 requests the GPU stack, with the fallbacks a real load applies. |
749
- | `threads`, `backendsDir` | As the load takes them. |
750
- | `marginBytes` | Free memory that must remain for the projection to count as fitting. Defaults to 256 MiB. |
780
+ | Option | Description |
781
+ | ----------------------------- | ----------------------------------------------------------------------------------------- |
782
+ | `durationSeconds` | Longest single generation the projection must accommodate. |
783
+ | `textTokens`, `lyricTokens` | The prompt the projection is sized for. |
784
+ | `lmPromptTokens` | 0 derives it from the text and lyric budgets. |
785
+ | `lmMaxNewTokens` | 0 derives it from the duration, as the pipeline does. |
786
+ | `lmCfgScale`, `guidanceScale` | 0 picks CFG or no CFG from the checkpoint. |
787
+ | `withSourceAudio` | Projects the extra VAE-encoder phase a cover or reference request loads. |
788
+ | `keepStages` | `-1` mirrors the engine, `0` forces the staged projection, `1` all-resident. |
789
+ | `gpuLayers` | Greater than 0 requests the GPU stack, with the fallbacks a real load applies. |
790
+ | `threads`, `backendsDir` | As the load takes them. |
791
+ | `marginBytes` | Free memory that must remain for the projection to count as fitting. Defaults to 256 MiB. |
751
792
 
752
793
  `deviceSharesHostMemory` reports that the device pool is system RAM, so host
753
794
  bytes compete with device bytes.
@@ -771,12 +812,12 @@ const fit = await assessFit({
771
812
  })
772
813
  ```
773
814
 
774
- | Option | Description |
775
- | --- | --- |
776
- | `device` | `'cpu'`, `'gpu'` or `'auto'`, as `config.device` takes it. Without it, `gpuLayers` greater than 0 projects `'auto'` and anything else `'cpu'`. |
777
- | `durationSeconds`, `maxFrames` | The generation length, as `run()` takes it (25 frames per second); one or the other. Defaults to 300 frames. |
778
- | `promptTokens` | Tokenized prompt length: caption, lyrics and template. Defaults to 1024. A prompt above the checkpoint's limit, or one that leaves too little context for the frames, is `workload-too-large`. |
779
- | `threads`, `backendsDir`, `marginBytes` | As for ACE-Step. |
815
+ | Option | Description |
816
+ | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
817
+ | `device` | `'cpu'`, `'gpu'` or `'auto'`, as `config.device` takes it. Without it, `gpuLayers` greater than 0 projects `'auto'` and anything else `'cpu'`. |
818
+ | `durationSeconds`, `maxFrames` | The generation length, as `run()` takes it (25 frames per second); one or the other. Defaults to 300 frames. |
819
+ | `promptTokens` | Tokenized prompt length: caption, lyrics and template. Defaults to 1024. A prompt above the checkpoint's limit, or one that leaves too little context for the frames, is `workload-too-large`. |
820
+ | `threads`, `backendsDir`, `marginBytes` | As for ACE-Step. |
780
821
 
781
822
  On Android and iOS, where MiniMax-Music3 is unavailable, `engine: 'minimax'`
782
823
  comes back as `status: "error"` with `reason: "unsupported-engine"`.
@@ -827,14 +868,35 @@ dispatches to the corresponding `audiogen-cpp` engine.
827
868
  - `addon/src/model-interface/minimax/` — desktop-only `MinimaxModel`.
828
869
  - Built with `cmake-bare` + `cmake-vcpkg`; `vcpkg.json` depends on the
829
870
  `speech-cpp[audiogen]` port.
830
- - `ggml-speech` and `speech-cpp` are pinned with vcpkg `overrides` to the same
831
- versions in every speech addon (`@qvac/asr-ggml`, `@qvac/audiogen-ggml`,
832
- `@qvac/bci-whispercpp`, `@qvac/tts-ggml`), and `package.json` records the
833
- `ggml-speech` build as `qvac["ggml-speech"]`. The speech addons ship their
871
+ - Native dependency versions are pinned with vcpkg `overrides` in
872
+ [vcpkg.json](vcpkg.json), and `package.json` records the `ggml-speech`
873
+ build as `qvac["ggml-speech"]`. The speech addons ship their
834
874
  ggml backends under the same file names, so an app that bundles several of
835
875
  them keeps one copy of each: use releases that record the same
836
876
  `ggml-speech`.
837
877
 
878
+ ## Fuzzing
879
+
880
+ The JS-adapter number parsers have a [Google FuzzTest][fuzztest] target,
881
+ `audiogen-config-parse-fuzz`. Fuzzing is Linux-only: it needs clang with
882
+ libFuzzer and AddressSanitizer, CMake 3.25 or newer, [vcpkg](https://vcpkg.io/)
883
+ (set `VCPKG_ROOT`) and `bare-make`. The target does not link audiogen-cpp, so
884
+ it runs with full ASan and LeakSanitizer.
885
+
886
+ ```bash
887
+ npm install
888
+ npm run fuzz # bounded run of every FUZZ_TEST, as Linux CI does
889
+ npm run fuzz:continuous # coverage-guided, one FUZZ_TEST at a time
890
+ npm run fuzz:continuous -- AudiogenConfigParseFuzz.ParseFloatNeverCrashes --fuzz_for=30m
891
+ ```
892
+
893
+ Flags other than `--continuous` and `--build-dir` go to the fuzz binary. See
894
+ [`docs/architecture/ADDON-FUZZING.md`](../../docs/architecture/ADDON-FUZZING.md)
895
+ for the fuzzing design, the vcpkg-supplied FuzzTest stack, and how to add a
896
+ target.
897
+
898
+ [fuzztest]: https://github.com/google/fuzztest
899
+
838
900
  ## Benchmarking
839
901
 
840
902
  The package ships the Real-Time Factor benchmark it is measured with, so the
package/audiogen.d.ts CHANGED
@@ -11,6 +11,8 @@ export interface AudioGenConfigurationParams {
11
11
  useGPU?: boolean;
12
12
  /** MiniMax compute device ('cpu' | 'gpu' | 'auto'); overrides useGPU. */
13
13
  device?: string;
14
+ coremlProfileDir?: string;
15
+ coremlComputeUnits?: string;
14
16
  nGpuLayers?: number;
15
17
  threads?: number;
16
18
  /**
@@ -48,7 +48,11 @@ async function main() {
48
48
  const generator = new AudioGen({
49
49
  engine: ENGINE_MINIMAX,
50
50
  files: { modelDir: requiredModelDir() },
51
- config: { threads: numberFromEnv('AUDIOGEN_THREADS') }
51
+ config: {
52
+ threads: numberFromEnv('AUDIOGEN_THREADS'),
53
+ coremlProfileDir: process.env.AUDIOGEN_COREML_PROFILE_DIR,
54
+ coremlComputeUnits: process.env.AUDIOGEN_COREML_COMPUTE_UNITS
55
+ }
52
56
  })
53
57
  try {
54
58
  await generator.load()
package/index.d.ts CHANGED
@@ -10,6 +10,8 @@ export declare const MINIMAX_DEFAULT_MAX_FRAMES = 300;
10
10
  export type AudioGenEngine = typeof ENGINE_ACESTEP | typeof ENGINE_MINIMAX;
11
11
  /** MiniMax compute device, see `AudioGenRuntimeConfig.device`. */
12
12
  export type AudiogenMinimaxDevice = 'cpu' | 'gpu' | 'auto';
13
+ /** Allowed devices for Apple's full MiniMax Core ML runtime. */
14
+ export type AudiogenMinimaxCoreMLComputeUnits = 'CPU_ONLY' | 'CPU_AND_GPU' | 'CPU_AND_NE' | 'ALL';
13
15
  /** Model file paths for ACE-Step or MiniMax-Music3. */
14
16
  export interface AudioGenFiles {
15
17
  /** Directory holding the four ACE-Step GGUFs (engine auto-classifies them). */
@@ -47,6 +49,10 @@ export interface AudioGenRuntimeConfig {
47
49
  * CPU, `'cpu'` never touches a GPU.
48
50
  */
49
51
  device?: AudiogenMinimaxDevice;
52
+ /** MiniMax only: compiled full Core ML profile directory on the inference host (macOS 15+). */
53
+ coremlProfileDir?: string;
54
+ /** MiniMax only: explicit Core ML device policy; requires coremlProfileDir. */
55
+ coremlComputeUnits?: AudiogenMinimaxCoreMLComputeUnits;
50
56
  /** ACE-Step only: GPU layers to offload when `useGPU` is set (99 = all). */
51
57
  nGpuLayers?: number;
52
58
  /** 0 = engine auto-picks. */
package/index.js CHANGED
@@ -35,6 +35,12 @@ const INT32_MAX = 2147483647;
35
35
  const FLOAT32_MAX = 3.4028234663852886e38;
36
36
  const FLOAT32_MIN_POSITIVE = 1.401298464324817e-45;
37
37
  const MINIMAX_DEVICES = ['cpu', 'gpu', 'auto'];
38
+ const MINIMAX_COREML_COMPUTE_UNITS = [
39
+ 'CPU_ONLY',
40
+ 'CPU_AND_GPU',
41
+ 'CPU_AND_NE',
42
+ 'ALL'
43
+ ];
38
44
  /** `AudiogenStats.backendId` codes, named. Codes match @qvac/tts-ggml. */
39
45
  exports.AUDIOGEN_BACKEND_NAMES = {
40
46
  0: 'cpu',
@@ -359,8 +365,26 @@ function validateAcestepOptions(files, config) {
359
365
  if (config.device !== undefined) {
360
366
  throw invalidInput('ACE-Step does not accept device (use useGPU and nGpuLayers)');
361
367
  }
368
+ if (config.coremlProfileDir !== undefined || config.coremlComputeUnits !== undefined) {
369
+ throw invalidInput('coremlProfileDir and coremlComputeUnits are MiniMax-only options');
370
+ }
371
+ }
372
+ function validateMinimaxCoreMLConfig(config) {
373
+ if (config.coremlProfileDir === undefined && config.coremlComputeUnits === undefined)
374
+ return;
375
+ if (typeof config.coremlProfileDir !== 'string' ||
376
+ config.coremlProfileDir.length === 0 ||
377
+ config.coremlProfileDir.includes('\0') ||
378
+ typeof config.coremlComputeUnits !== 'string' ||
379
+ !MINIMAX_COREML_COMPUTE_UNITS.includes(config.coremlComputeUnits)) {
380
+ throw invalidInput('MiniMax Core ML requires coremlProfileDir and coremlComputeUnits (CPU_ONLY, CPU_AND_GPU, CPU_AND_NE or ALL)');
381
+ }
382
+ if (os.platform() !== 'darwin') {
383
+ throw invalidInput('MiniMax Core ML requires macOS 15 or later');
384
+ }
362
385
  }
363
386
  function validateMinimaxConfig(config) {
387
+ validateMinimaxCoreMLConfig(config);
364
388
  if (config.useGPU !== undefined && typeof config.useGPU !== 'boolean') {
365
389
  throw invalidInput('useGPU must be a boolean');
366
390
  }
@@ -586,6 +610,8 @@ class AudioGen {
586
610
  threads,
587
611
  useGPU: config.useGPU ?? false,
588
612
  device: config.device,
613
+ coremlProfileDir: config.coremlProfileDir,
614
+ coremlComputeUnits: config.coremlComputeUnits,
589
615
  backendsDir
590
616
  };
591
617
  }
package/lib/fit.d.ts CHANGED
@@ -26,6 +26,9 @@ export interface AudiogenFitRequest {
26
26
  * > 0 projects `auto` (a GPU when one is usable) and anything else `cpu`.
27
27
  */
28
28
  device?: 'cpu' | 'gpu' | 'auto'
29
+ /** MiniMax full Core ML profile. Projection returns coreml-memory-estimate-unavailable. */
30
+ coremlProfileDir?: string
31
+ coremlComputeUnits?: 'CPU_ONLY' | 'CPU_AND_GPU' | 'CPU_AND_NE' | 'ALL'
29
32
  threads?: number
30
33
  backendsDir?: string
31
34
 
package/package.json CHANGED
@@ -1,9 +1,9 @@
1
1
  {
2
2
  "name": "@qvac/audiogen-ggml",
3
- "version": "0.8.0",
3
+ "version": "0.9.1",
4
4
  "description": "Audio generation (music) addon for qvac",
5
5
  "qvac": {
6
- "ggml-speech": "2026-10-07#1"
6
+ "ggml-speech": "2026-10-27"
7
7
  },
8
8
  "engines": {
9
9
  "bare": ">=1.19.0"
@@ -34,8 +34,8 @@
34
34
  },
35
35
  "scripts": {
36
36
  "build": "npm run build:ts && npm run build:native",
37
- "build:native": "bare-make generate && bare-make build && bare-make install && npm run link:platform",
38
- "build:native:cuda": "bare-make generate -D ENABLE_CUDA=ON && bare-make build && bare-make install && npm run link:platform",
37
+ "build:native": "bare-make generate -D BUILD_FUZZING=OFF && bare-make build && bare-make install && npm run link:platform",
38
+ "build:native:cuda": "bare-make generate -D ENABLE_CUDA=ON -D BUILD_FUZZING=OFF && bare-make build && bare-make install && npm run link:platform",
39
39
  "link:platform": "node ../../scripts/ci/slice-platform-packages.mjs --link-local --workdir .",
40
40
  "build:cuda": "npm run build:ts && npm run build:native:cuda",
41
41
  "build:ts": "tsc -p tsconfig.build.json",
@@ -59,6 +59,11 @@
59
59
  "test:integration": "npm run build:ts && brittle-bare test/integration/**/*.test.js",
60
60
  "test:benchmark:rtf": "npm run build:ts && brittle-bare test/benchmark/rtf-benchmark.test.js",
61
61
  "test:benchmark:rtf:matrix": "node scripts/run-rtf-benchmark-matrix.js",
62
+ "fuzz:build": "bare-make generate -D BUILD_FUZZING=ON -D BUILD_TESTING=OFF -D FUZZTEST_FUZZING_MODE=OFF && bare-make build --target audiogen-config-parse-fuzz",
63
+ "fuzz:build:continuous": "bare-make generate -D BUILD_FUZZING=ON -D BUILD_TESTING=OFF -D FUZZTEST_FUZZING_MODE=ON && bare-make build --target audiogen-config-parse-fuzz",
64
+ "fuzz:run": "node scripts/run-cpp-fuzz.js",
65
+ "fuzz": "npm run fuzz:build && npm run fuzz:run",
66
+ "fuzz:continuous": "npm run fuzz:build:continuous && node scripts/run-cpp-fuzz.js --continuous",
62
67
  "test": "npm run test:types && npm run test:unit && npm run test:integration",
63
68
  "lint": "npm run lint:js && npm run lint:ts && npm run test:types",
64
69
  "lint:js": "prettier --check \"addon-unavailable.js\" \"binding.js\" \"examples/*.js\" \"lib/*.js\" \"scripts/**/*.{js,mjs}\" \"test/**/*.js\" && lunte \"addon-unavailable.js\" \"binding.js\" \"examples/*.js\" \"lib/*.js\" \"scripts/**/*.{js,mjs}\" \"test/**/*.js\"",
@@ -126,7 +131,7 @@
126
131
  ],
127
132
  "devDependencies": {
128
133
  "@qvac/registry-client": "^0.6.1",
129
- "bare-pack": "^2.2.3",
134
+ "bare-pack": "^3.0.0",
130
135
  "brittle": "^3.17.0",
131
136
  "cmake-bare": "^1.7.5",
132
137
  "cmake-vcpkg": "^1.1.0",
@@ -151,19 +156,19 @@
151
156
  "bare-process": "^4.2.2"
152
157
  },
153
158
  "optionalDependencies": {
154
- "@qvac/audiogen-ggml-linux-x64": "0.8.0",
155
- "@qvac/audiogen-ggml-linux-arm64": "0.8.0",
156
- "@qvac/audiogen-ggml-darwin-arm64": "0.8.0",
157
- "@qvac/audiogen-ggml-darwin-x64": "0.8.0",
158
- "@qvac/audiogen-ggml-win32-x64": "0.8.0"
159
+ "@qvac/audiogen-ggml-linux-x64": "0.9.1",
160
+ "@qvac/audiogen-ggml-linux-arm64": "0.9.1",
161
+ "@qvac/audiogen-ggml-darwin-arm64": "0.9.1",
162
+ "@qvac/audiogen-ggml-darwin-x64": "0.9.1",
163
+ "@qvac/audiogen-ggml-win32-x64": "0.9.1"
159
164
  },
160
165
  "overrides": {
161
166
  "bare-module-lexer": "1.6.3"
162
167
  },
163
168
  "types": "index.d.ts",
164
169
  "peerDependencies": {
165
- "@qvac/audiogen-ggml-android-arm64": "0.8.0",
166
- "@qvac/audiogen-ggml-ios": "0.8.0"
170
+ "@qvac/audiogen-ggml-android-arm64": "0.9.1",
171
+ "@qvac/audiogen-ggml-ios": "0.9.1"
167
172
  },
168
173
  "peerDependenciesMeta": {
169
174
  "@qvac/audiogen-ggml-android-arm64": {
@@ -47,6 +47,15 @@ async function runAudioGen(gen, { caption, opts = {} } = {}) {
47
47
  }
48
48
 
49
49
  async function collectAudioGenResponse(response) {
50
+ const [{ chunks, stages, stageTotals, sampleRate, channels }, stats] = await Promise.all([
51
+ collectAudioGenOutput(response),
52
+ response.await()
53
+ ])
54
+
55
+ return summarizeAudioGenOutput({ chunks, stages, stageTotals, sampleRate, channels, stats })
56
+ }
57
+
58
+ async function collectAudioGenOutput(response) {
50
59
  const chunks = []
51
60
  const stages = new Set()
52
61
  const stageTotals = {}
@@ -68,8 +77,10 @@ async function collectAudioGenResponse(response) {
68
77
  }
69
78
  }
70
79
 
71
- const stats = await response.await()
80
+ return { chunks, stages, stageTotals, sampleRate, channels }
81
+ }
72
82
 
83
+ function summarizeAudioGenOutput({ chunks, stages, stageTotals, sampleRate, channels, stats }) {
73
84
  let sampleCount = 0
74
85
  for (const chunk of chunks) sampleCount += chunk.length
75
86