@qvac/audiogen-ggml 0.9.0 → 0.9.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,37 @@ All notable changes to this project will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [0.9.1] - 2026-10-28
9
+
10
+ ### Added
11
+
12
+ - Expose the full MiniMax Core ML runtime through `coremlProfileDir` and
13
+ `coremlComputeUnits` on macOS 15+, with four explicit Apple device policies.
14
+ The compiled profile stays resident across generations. Native generation
15
+ keeps its existing behavior when these options are absent
16
+ ([#5389](https://github.com/tetherto/qvac/pull/5389)).
17
+
18
+ ### Changed
19
+
20
+ - Align the shared native `speech-cpp` source to `2026-10-27` across the speech
21
+ addons ([#5397](https://github.com/tetherto/qvac/pull/5397)).
22
+ - The shared native `ggml-speech` moves to `2026-10-27` across the speech
23
+ addons, which also speeds up single-query Metal softmax on long KV caches
24
+ ([#5392](https://github.com/tetherto/qvac/pull/5392)).
25
+
26
+ ### Fixed
27
+
28
+ - Adreno GPUs (Android, OpenCL): matrix products on every quantized weight
29
+ type whose sums exceed the fp16 range no longer come back as infinity;
30
+ `ggml-speech` `2026-10-23` fixed q8_0 only
31
+ ([#5392](https://github.com/tetherto/qvac/pull/5392)).
32
+ - Metal: a fused SwiGLU mat-vec whose output overlaps its inputs, and copies
33
+ into strided destinations, no longer produce wrong values; quantized matrix
34
+ products keep F32 activations
35
+ ([#5392](https://github.com/tetherto/qvac/pull/5392)).
36
+ - Vulkan: tensor views not aligned for the speech ops fall back instead of
37
+ returning wrong data ([#5392](https://github.com/tetherto/qvac/pull/5392)).
38
+
8
39
  ## [0.9.0] - 2026-10-23
9
40
 
10
41
  ### Added
@@ -233,7 +264,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
233
264
  cause for clearer diagnostics. Without this the wrong module was returned
234
265
  verbatim and the first symptom was `createInstance is not a function` deep in
235
266
  a model load. Local source-build behaviour is unchanged.
236
-
267
+
237
268
  ### Fixed
238
269
 
239
270
  - Mobile platform packages (`@qvac/audiogen-ggml-android-arm64`,
package/README.md CHANGED
@@ -56,13 +56,13 @@ only. The native prebuild for each desktop host lives in a version-locked
56
56
  platform package selected at install time through `os`/`cpu` filtered
57
57
  `optionalDependencies`:
58
58
 
59
- | Host | Package |
60
- | --- | --- |
61
- | linux-x64 (glibc) | `@qvac/audiogen-ggml-linux-x64` |
62
- | linux-arm64 (glibc) | `@qvac/audiogen-ggml-linux-arm64` |
63
- | darwin-arm64 | `@qvac/audiogen-ggml-darwin-arm64` |
64
- | darwin-x64 | `@qvac/audiogen-ggml-darwin-x64` |
65
- | win32-x64 | `@qvac/audiogen-ggml-win32-x64` |
59
+ | Host | Package |
60
+ | ------------------- | ---------------------------------- |
61
+ | linux-x64 (glibc) | `@qvac/audiogen-ggml-linux-x64` |
62
+ | linux-arm64 (glibc) | `@qvac/audiogen-ggml-linux-arm64` |
63
+ | darwin-arm64 | `@qvac/audiogen-ggml-darwin-arm64` |
64
+ | darwin-x64 | `@qvac/audiogen-ggml-darwin-x64` |
65
+ | win32-x64 | `@qvac/audiogen-ggml-win32-x64` |
66
66
 
67
67
  Do not depend on desktop platform packages directly. Supported installers are
68
68
  npm 7+, pnpm, bun, and Yarn Berry. Yarn v1 and `--omit=optional` installs skip
@@ -78,10 +78,10 @@ and `optionalDependencies` filtering can never select them. Mobile
78
78
  applications must declare the target's platform package as a direct
79
79
  dependency, pinned to the exact `@qvac/audiogen-ggml` version:
80
80
 
81
- | Target | Package |
82
- | --- | --- |
83
- | android-arm64 | `@qvac/audiogen-ggml-android-arm64` |
84
- | ios (device + simulators) | `@qvac/audiogen-ggml-ios` |
81
+ | Target | Package |
82
+ | ------------------------- | ----------------------------------- |
83
+ | android-arm64 | `@qvac/audiogen-ggml-android-arm64` |
84
+ | ios (device + simulators) | `@qvac/audiogen-ggml-ios` |
85
85
 
86
86
  ```json
87
87
  {
@@ -165,6 +165,43 @@ the engine time into `arMs`, `conditionMs`, `flowMs` and `vocoderMs`, and
165
165
  the song ended on its own. See
166
166
  [`examples/generate-music-minimax.js`](examples/generate-music-minimax.js).
167
167
 
168
+ ### MiniMax full Core ML on macOS
169
+
170
+ On macOS 15+, `speech-cpp[audiogen]` includes the full Core ML runtime. Use
171
+ the original F16 GGUF pair and its matching compiled profile:
172
+
173
+ ```js
174
+ const generator = new AudioGen({
175
+ engine: ENGINE_MINIMAX,
176
+ files: { lmModel: '/models/mm3-lm-f16.gguf', synthModel: '/models/mm3-synth-f16.gguf' },
177
+ config: { coremlProfileDir: '/models/minimax-coreml', coremlComputeUnits: 'CPU_AND_NE' }
178
+ })
179
+ ```
180
+
181
+ Set both options explicitly. `CPU_ONLY`, `CPU_AND_GPU`, `CPU_AND_NE`, and
182
+ `ALL` select Apple's allowed compute units; they do not guarantee that Apple
183
+ places every operation on a particular device. They take precedence over the
184
+ native `device`/`useGPU` selection for this route. Without the pair, MiniMax
185
+ uses its existing native backend. An explicit Core ML request fails if the
186
+ build, OS, or profile cannot support it; it does not silently switch backends.
187
+
188
+ The profile directory belongs to the inference host and contains `manifest.json`
189
+ and all compiled LM, depth, conditioner, DiT, and vocoder programs. Keep its
190
+ weights and dimensions matched to the GGUF pair. The compiled LM cache profile
191
+ must accommodate the prompt and generation; no request is truncated to fit it.
192
+ The programs stay resident between generations and are released on unload.
193
+ Core ML returns `backendId: 99` (`other`); native-only `backendDevice` and
194
+ `gpuFallbackReason` are omitted. Memory preflight returns
195
+ `coreml-memory-estimate-unavailable` for this route.
196
+
197
+ CPU/ANE initial-logit parity against the native reference is still under
198
+ validation; the measured device policies are not a claim of identical sampled
199
+ music or faster generation. See the native engine's
200
+ [full Core ML validation guide](https://github.com/tetherto/qvac-fabric-speech.cpp/blob/master/engines/audiogen/docs/minimax-coreml-validation.md).
201
+
202
+ The example accepts `AUDIOGEN_COREML_PROFILE_DIR` and
203
+ `AUDIOGEN_COREML_COMPUTE_UNITS` with the same explicit pair.
204
+
168
205
  ### 1. Simplest case — an instrumental
169
206
 
170
207
  ```js
@@ -210,7 +247,7 @@ await gen.destroy()
210
247
  > Progress items arrive live during generation. The current native engine emits
211
248
  > one PCM item after generation, not incremental audio chunks. `await()` resolves
212
249
  > with terminal stats. `backendDevice` /
213
- > `backendId` report the backend the engine *resolved to*, not the one requested,
250
+ > `backendId` report the backend the engine _resolved to_, not the one requested,
214
251
  > so a `useGPU: true` run that fell back to the CPU is detectable. Use
215
252
  > `audiogenBackendName(stats.backendId)` rather than copying the code table
216
253
  > above; it returns `undefined` for an id this version does not know.
@@ -245,13 +282,13 @@ Pass `lyrics` for vocals, and steer the LM with musical hints — `bpm`,
245
282
 
246
283
  ```js
247
284
  const response = await gen.run('energetic cumbia, brass stabs, live percussion, party vibe', {
248
- vocalLanguage: 'es', // language the vocals are sung in
249
- bpm: 98, // tempo
250
- keyscale: 'A minor', // key / scale
251
- timesignature: '4/4', // time signature
285
+ vocalLanguage: 'es', // language the vocals are sung in
286
+ bpm: 98, // tempo
287
+ keyscale: 'A minor', // key / scale
288
+ timesignature: '4/4', // time signature
252
289
  augmentCaptionWithMetadata: true, // reinforce these hints in the conditioning caption
253
- duration: 150, // target length in seconds (omit => the LM decides)
254
- seed: 42, // reproducible run
290
+ duration: 150, // target length in seconds (omit => the LM decides)
291
+ seed: 42, // reproducible run
255
292
  lyrics: `[verse]
256
293
  Suena el tambor y el barrio se enciende
257
294
  la noche es joven y la gente lo entiende
@@ -358,10 +395,11 @@ set is kept. Leave `lyrics` unset for LM-written vocals, or pass
358
395
  `'[Instrumental]'` for an instrumental song.
359
396
 
360
397
  ```js
361
- const response = await gen.run(
362
- 'a romantic modern salsa with male lead vocals for a wedding',
363
- { simpleMode: true, duration: 0, seed: 4242 }
364
- )
398
+ const response = await gen.run('a romantic modern salsa with male lead vocals for a wedding', {
399
+ simpleMode: true,
400
+ duration: 0,
401
+ seed: 4242
402
+ })
365
403
  ```
366
404
 
367
405
  Or end to end from the repo:
@@ -543,12 +581,14 @@ const fs = require('fs')
543
581
  const pcmChunks = []
544
582
  for await (const item of response.iterate()) {
545
583
  if (!('outputArray' in item)) continue
546
- pcmChunks.push(Buffer.from(
547
- item.outputArray.buffer.slice(
548
- item.outputArray.byteOffset,
549
- item.outputArray.byteOffset + item.outputArray.byteLength
584
+ pcmChunks.push(
585
+ Buffer.from(
586
+ item.outputArray.buffer.slice(
587
+ item.outputArray.byteOffset,
588
+ item.outputArray.byteOffset + item.outputArray.byteLength
589
+ )
550
590
  )
551
- ))
591
+ )
552
592
  }
553
593
  const pcmBuffer = Buffer.concat(pcmChunks)
554
594
 
@@ -571,27 +611,27 @@ for (const f of files) fs.writeFileSync(`song.${f.extension}`, f.data)
571
611
  vendored for `@qvac/decoder-audio`), so the matching encoder must be compiled
572
612
  into that build.
573
613
 
574
- | `format` | Container / codec | Extension | MIME | Notes |
575
- |----------|-------------------|-----------|------|-------|
576
- | `pcm` | raw interleaved Int16 | `pcm` | `audio/L16` | no container, dependency-free |
577
- | `wav` | WAV / PCM s16le | `wav` | `audio/wav` | dependency-free (pure-JS header) |
578
- | `flac` | FLAC | `flac` | `audio/flac` | lossless |
579
- | `alac` | MP4 / ALAC | `m4a` | `audio/mp4` | Apple Lossless (shares the `.m4a` extension) |
580
- | `aiff` | AIFF / PCM s16be | `aiff` | `audio/aiff` | uncompressed |
581
- | `caf` | CAF / PCM s16le | `caf` | `audio/x-caf` | Apple Core Audio Format, uncompressed |
582
- | `m4a` | MP4 / AAC | `m4a` | `audio/mp4` | AAC in an MP4/M4A container |
583
- | `aac` | ADTS / AAC | `aac` | `audio/aac` | raw AAC stream |
584
- | `opus` | Ogg / Opus | `opus` | `audio/opus` | resampled to 48 kHz (libopus requirement) |
585
- | `ogg` | Ogg / Vorbis | `ogg` | `audio/ogg` | Vorbis |
586
- | `ac3` | AC-3 | `ac3` | `audio/ac3` | Dolby Digital |
587
- | `wma` | ASF / WMA v2 | `wma` | `audio/x-ms-wma` | Windows Media Audio |
588
- | `mp2` | MPEG-1 Layer II | `mp2` | `audio/mpeg` | legacy MPEG audio |
614
+ | `format` | Container / codec | Extension | MIME | Notes |
615
+ | -------- | --------------------- | --------- | ---------------- | -------------------------------------------- |
616
+ | `pcm` | raw interleaved Int16 | `pcm` | `audio/L16` | no container, dependency-free |
617
+ | `wav` | WAV / PCM s16le | `wav` | `audio/wav` | dependency-free (pure-JS header) |
618
+ | `flac` | FLAC | `flac` | `audio/flac` | lossless |
619
+ | `alac` | MP4 / ALAC | `m4a` | `audio/mp4` | Apple Lossless (shares the `.m4a` extension) |
620
+ | `aiff` | AIFF / PCM s16be | `aiff` | `audio/aiff` | uncompressed |
621
+ | `caf` | CAF / PCM s16le | `caf` | `audio/x-caf` | Apple Core Audio Format, uncompressed |
622
+ | `m4a` | MP4 / AAC | `m4a` | `audio/mp4` | AAC in an MP4/M4A container |
623
+ | `aac` | ADTS / AAC | `aac` | `audio/aac` | raw AAC stream |
624
+ | `opus` | Ogg / Opus | `opus` | `audio/opus` | resampled to 48 kHz (libopus requirement) |
625
+ | `ogg` | Ogg / Vorbis | `ogg` | `audio/ogg` | Vorbis |
626
+ | `ac3` | AC-3 | `ac3` | `audio/ac3` | Dolby Digital |
627
+ | `wma` | ASF / WMA v2 | `wma` | `audio/x-ms-wma` | Windows Media Audio |
628
+ | `mp2` | MPEG-1 Layer II | `mp2` | `audio/mpeg` | legacy MPEG audio |
589
629
 
590
630
  > `mp3` is intentionally **not** listed: the vendored FFmpeg build ships no MP3
591
631
  > encoder (`libmp3lame`). Unknown/unsupported formats throw.
592
632
  >
593
633
  > `AudioGen.encode(pcm, formats, opts)` returns `{ format, data, extension,
594
- > mimeType }` for a single format, or an array of those (input order) for an
634
+ mimeType }` for a single format, or an array of those (input order) for an
595
635
  > array. `OUTPUT_FORMATS` exports the full allowed list.
596
636
 
597
637
  See [`examples/generate-music.js`](examples/generate-music.js) for a full,
@@ -606,60 +646,61 @@ runnable end-to-end script (`npm run example`).
606
646
 
607
647
  `files` — model paths:
608
648
 
609
- | Option | Meaning |
610
- |--------|---------|
611
- | `modelDir` | Folder holding the GGUFs (stages auto-classified by name). |
612
- | `ditVariant` | Which DiT to load from `modelDir`: `turbo-q4` \| `turbo-q8` \| `sft`. |
613
- | `textEncModel` / `lmModel` / `ditModel` / `vaeModel` | Explicit per-stage paths (override `modelDir`). |
614
- | `lmModel` / `synthModel` | Explicit MiniMax-Music3 pair (override `modelDir`). |
649
+ | Option | Meaning |
650
+ | ---------------------------------------------------- | --------------------------------------------------------------------- |
651
+ | `modelDir` | Folder holding the GGUFs (stages auto-classified by name). |
652
+ | `ditVariant` | Which DiT to load from `modelDir`: `turbo-q4` \| `turbo-q8` \| `sft`. |
653
+ | `textEncModel` / `lmModel` / `ditModel` / `vaeModel` | Explicit per-stage paths (override `modelDir`). |
654
+ | `lmModel` / `synthModel` | Explicit MiniMax-Music3 pair (override `modelDir`). |
615
655
 
616
656
  `config` — runtime knobs:
617
657
 
618
- | Option | Meaning |
619
- |--------|---------|
620
- | `useGPU` | Run on GPU (Metal / CUDA / Vulkan, including Android Mali); falls back to CPU. |
621
- | `device` | MiniMax only, instead of `useGPU`: `gpu` fails `load()` without a usable GPU, `auto` falls back to the CPU, `cpu` never uses a GPU. |
622
- | `inferenceSteps` / `shift` | Advanced; leave unset to auto-tune per DiT. `run()` can override both per call. |
623
- | `cfgScale` | Default MiniMax flow guidance scale; `0` uses the model default. |
624
- | `nGpuLayers` | GPU layers to offload when `useGPU` is set (99 = all). |
625
- | `threads` | CPU thread count (0 / unset = hardware default). |
626
- | `backendsDir` | Advanced; directory scanned, as given, for dlopen'd ggml backend modules. Unset scans the addon's own, next to its native module, which is where the platform package and `bare-link` put them. |
658
+ | Option | Meaning |
659
+ | ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
660
+ | `useGPU` | Run on GPU (Metal / CUDA / Vulkan, including Android Mali); falls back to CPU. |
661
+ | `coremlProfileDir` / `coremlComputeUnits` | MiniMax full Core ML profile and explicit Apple device policy; both required together, macOS 15+. |
662
+ | `device` | MiniMax only, instead of `useGPU`: `gpu` fails `load()` without a usable GPU, `auto` falls back to the CPU, `cpu` never uses a GPU. |
663
+ | `inferenceSteps` / `shift` | Advanced; leave unset to auto-tune per DiT. `run()` can override both per call. |
664
+ | `cfgScale` | Default MiniMax flow guidance scale; `0` uses the model default. |
665
+ | `nGpuLayers` | GPU layers to offload when `useGPU` is set (99 = all). |
666
+ | `threads` | CPU thread count (0 / unset = hardware default). |
667
+ | `backendsDir` | Advanced; directory scanned, as given, for dlopen'd ggml backend modules. Unset scans the addon's own, next to its native module, which is where the platform package and `bare-link` put them. |
627
668
 
628
669
  `logger` — an optional object implementing `error`/`warn`/`info`/`debug`,
629
670
  wrapped by a level-gated `QvacLogger`.
630
671
 
631
672
  **`run(caption, opts)`** returns a `QvacResponse`.
632
673
 
633
- | Option | Meaning |
634
- |--------|---------|
635
- | `lyrics` | Lyrics text; use `[Instrumental]` for no vocals. |
636
- | `vocalLanguage` | Vocal language hint. |
637
- | `bpm` | Tempo in beats per minute. |
638
- | `keyscale` | Key and scale, such as `C minor`. |
639
- | `timesignature` | Time signature, such as `4/4`. |
640
- | `augmentCaptionWithMetadata` | Append BPM/tempo, time signature, and key guidance to the internal conditioning caption; defaults to `false`. |
641
- | `duration` | Target length in seconds; omit to let the LM decide. |
642
- | `maxFrames` | MiniMax semantic-frame cap; cannot be combined with `duration`. |
643
- | `inferenceSteps` | Per-run diffusion steps (`0`–`1000`); omit to keep `config.inferenceSteps`. |
644
- | `shift` | ACE-Step per-run DiT timestep shift; omit to keep `config.shift`. |
645
- | `cfgScale` | Per-run MiniMax flow guidance scale. |
646
- | `seed` | RNG seed for reproducible generation. |
647
- | `lmTemperature` / `lmTopP` / `lmTopK` / `lmCfgScale` | LM sampling controls. |
648
- | `lmPhase1` | Allow the LM to infer missing metadata before generating semantic codes. |
649
- | `simpleMode` | Expand the caption query into a full request (caption, lyrics, unset metadata) before synthesis. |
650
- | `rewriteQuery` | LM FORMAT pass: rewrite the caption into a detailed description, preserving the lyric content. Requires real `lyrics` and `taskType: 'text2music'`; mutually exclusive with `simpleMode`. |
651
- | `normalizeLoudness` | Percentile loudness normalization of generated audio (default `true`); edits are never normalized. |
652
- | `generateLrc` | Synchronized lyric timestamps: `stats.lrc` (LRC text) + `stats.lyricsScore`. Requires lyrics and `taskType: 'text2music'`. |
653
- | `computeQualityScore` | Teacher-forced LM quality score of the generated codes; `stats.qualityScore` in `[0, 1]`. Requires `taskType: 'text2music'`. |
654
- | `dcwEnabled` / `dcwScaler` / `dcwHighScaler` | Haar DCW correction controls. |
655
- | `audioCodes` | Frozen ACE-Step semantic codes as an `Int32Array`; skips the LM. |
656
- | `referenceAudio` | Optional finite, normalized, interleaved stereo 48 kHz `Float32Array` used for timbre conditioning. |
657
- | `sourceAudio` | Source PCM in the same format; required by cover and lego tasks. |
658
- | `taskType` | `text2music` (default), `cover-nofsq`, `lego`, or reserved `cover`. |
659
- | `track` | Lego target layer; required when `taskType` is `lego`. |
660
- | `guidanceScale` | DiT classifier-free guidance; `0` (default) auto-resolves to `1.0` on turbo and `7.0` on base/sft. |
661
- | `audioCoverStrength` | Fraction of the run that follows the source, from `0` to `1` (default `1`). |
662
- | `coverNoiseStrength` | Initial source/noise blend from `0` to `1`. |
674
+ | Option | Meaning |
675
+ | ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
676
+ | `lyrics` | Lyrics text; use `[Instrumental]` for no vocals. |
677
+ | `vocalLanguage` | Vocal language hint. |
678
+ | `bpm` | Tempo in beats per minute. |
679
+ | `keyscale` | Key and scale, such as `C minor`. |
680
+ | `timesignature` | Time signature, such as `4/4`. |
681
+ | `augmentCaptionWithMetadata` | Append BPM/tempo, time signature, and key guidance to the internal conditioning caption; defaults to `false`. |
682
+ | `duration` | Target length in seconds; omit to let the LM decide. |
683
+ | `maxFrames` | MiniMax semantic-frame cap; cannot be combined with `duration`. |
684
+ | `inferenceSteps` | Per-run diffusion steps (`0`–`1000`); omit to keep `config.inferenceSteps`. |
685
+ | `shift` | ACE-Step per-run DiT timestep shift; omit to keep `config.shift`. |
686
+ | `cfgScale` | Per-run MiniMax flow guidance scale. |
687
+ | `seed` | RNG seed for reproducible generation. |
688
+ | `lmTemperature` / `lmTopP` / `lmTopK` / `lmCfgScale` | LM sampling controls. |
689
+ | `lmPhase1` | Allow the LM to infer missing metadata before generating semantic codes. |
690
+ | `simpleMode` | Expand the caption query into a full request (caption, lyrics, unset metadata) before synthesis. |
691
+ | `rewriteQuery` | LM FORMAT pass: rewrite the caption into a detailed description, preserving the lyric content. Requires real `lyrics` and `taskType: 'text2music'`; mutually exclusive with `simpleMode`. |
692
+ | `normalizeLoudness` | Percentile loudness normalization of generated audio (default `true`); edits are never normalized. |
693
+ | `generateLrc` | Synchronized lyric timestamps: `stats.lrc` (LRC text) + `stats.lyricsScore`. Requires lyrics and `taskType: 'text2music'`. |
694
+ | `computeQualityScore` | Teacher-forced LM quality score of the generated codes; `stats.qualityScore` in `[0, 1]`. Requires `taskType: 'text2music'`. |
695
+ | `dcwEnabled` / `dcwScaler` / `dcwHighScaler` | Haar DCW correction controls. |
696
+ | `audioCodes` | Frozen ACE-Step semantic codes as an `Int32Array`; skips the LM. |
697
+ | `referenceAudio` | Optional finite, normalized, interleaved stereo 48 kHz `Float32Array` used for timbre conditioning. |
698
+ | `sourceAudio` | Source PCM in the same format; required by cover and lego tasks. |
699
+ | `taskType` | `text2music` (default), `cover-nofsq`, `lego`, or reserved `cover`. |
700
+ | `track` | Lego target layer; required when `taskType` is `lego`. |
701
+ | `guidanceScale` | DiT classifier-free guidance; `0` (default) auto-resolves to `1.0` on turbo and `7.0` on base/sft. |
702
+ | `audioCoverStrength` | Fraction of the run that follows the source, from `0` to `1` (default `1`). |
703
+ | `coverNoiseStrength` | Initial source/noise blend from `0` to `1`. |
663
704
 
664
705
  ## Models
665
706
 
@@ -676,14 +717,14 @@ MiniMax-Music3 Community License.
676
717
  Four stages. Three are fixed; only the DiT changes, so you pick it with
677
718
  `ditVariant`.
678
719
 
679
- | Stage | GGUF | Size | Notes |
680
- |------|------|------|-------|
681
- | text-enc | Qwen3-Embedding-0.6B-Q8_0 | 748 MB | fixed |
682
- | LM | acestep-5Hz-lm-0.6B-Q8_0 | 677 MB | fixed |
683
- | VAE | vae-BF16 | 322 MB | fixed |
684
- | DiT | acestep-v15-turbo-Q4_K_M | 1.35 GB | `ditVariant: 'turbo-q4'` (fastest / smallest) |
685
- | DiT | acestep-v15-turbo-Q8_0 | 2.37 GB | `ditVariant: 'turbo-q8'` (higher precision) |
686
- | DiT | acestep-v15-sft-Q8_0 | 2.37 GB | `ditVariant: 'sft'` (50-step, non-turbo) |
720
+ | Stage | GGUF | Size | Notes |
721
+ | -------- | ------------------------- | ------- | --------------------------------------------- |
722
+ | text-enc | Qwen3-Embedding-0.6B-Q8_0 | 748 MB | fixed |
723
+ | LM | acestep-5Hz-lm-0.6B-Q8_0 | 677 MB | fixed |
724
+ | VAE | vae-BF16 | 322 MB | fixed |
725
+ | DiT | acestep-v15-turbo-Q4_K_M | 1.35 GB | `ditVariant: 'turbo-q4'` (fastest / smallest) |
726
+ | DiT | acestep-v15-turbo-Q8_0 | 2.37 GB | `ditVariant: 'turbo-q8'` (higher precision) |
727
+ | DiT | acestep-v15-sft-Q8_0 | 2.37 GB | `ditVariant: 'sft'` (50-step, non-turbo) |
687
728
 
688
729
  Downloading the GGUFs is the caller's job (the qvac SDK's `resolveModelPath`, a
689
730
  download script, etc.) — the addon only ever receives a local path. `models.js`
@@ -736,18 +777,18 @@ fit.report
736
777
  `modelsDir` holds the four stage GGUFs; `textEncoderPath`, `lmPath`, `ditPath`
737
778
  and `vaePath` name them individually and win over it.
738
779
 
739
- | Option | Description |
740
- | --- | --- |
741
- | `durationSeconds` | Longest single generation the projection must accommodate. |
742
- | `textTokens`, `lyricTokens` | The prompt the projection is sized for. |
743
- | `lmPromptTokens` | 0 derives it from the text and lyric budgets. |
744
- | `lmMaxNewTokens` | 0 derives it from the duration, as the pipeline does. |
745
- | `lmCfgScale`, `guidanceScale` | 0 picks CFG or no CFG from the checkpoint. |
746
- | `withSourceAudio` | Projects the extra VAE-encoder phase a cover or reference request loads. |
747
- | `keepStages` | `-1` mirrors the engine, `0` forces the staged projection, `1` all-resident. |
748
- | `gpuLayers` | Greater than 0 requests the GPU stack, with the fallbacks a real load applies. |
749
- | `threads`, `backendsDir` | As the load takes them. |
750
- | `marginBytes` | Free memory that must remain for the projection to count as fitting. Defaults to 256 MiB. |
780
+ | Option | Description |
781
+ | ----------------------------- | ----------------------------------------------------------------------------------------- |
782
+ | `durationSeconds` | Longest single generation the projection must accommodate. |
783
+ | `textTokens`, `lyricTokens` | The prompt the projection is sized for. |
784
+ | `lmPromptTokens` | 0 derives it from the text and lyric budgets. |
785
+ | `lmMaxNewTokens` | 0 derives it from the duration, as the pipeline does. |
786
+ | `lmCfgScale`, `guidanceScale` | 0 picks CFG or no CFG from the checkpoint. |
787
+ | `withSourceAudio` | Projects the extra VAE-encoder phase a cover or reference request loads. |
788
+ | `keepStages` | `-1` mirrors the engine, `0` forces the staged projection, `1` all-resident. |
789
+ | `gpuLayers` | Greater than 0 requests the GPU stack, with the fallbacks a real load applies. |
790
+ | `threads`, `backendsDir` | As the load takes them. |
791
+ | `marginBytes` | Free memory that must remain for the projection to count as fitting. Defaults to 256 MiB. |
751
792
 
752
793
  `deviceSharesHostMemory` reports that the device pool is system RAM, so host
753
794
  bytes compete with device bytes.
@@ -771,12 +812,12 @@ const fit = await assessFit({
771
812
  })
772
813
  ```
773
814
 
774
- | Option | Description |
775
- | --- | --- |
776
- | `device` | `'cpu'`, `'gpu'` or `'auto'`, as `config.device` takes it. Without it, `gpuLayers` greater than 0 projects `'auto'` and anything else `'cpu'`. |
777
- | `durationSeconds`, `maxFrames` | The generation length, as `run()` takes it (25 frames per second); one or the other. Defaults to 300 frames. |
778
- | `promptTokens` | Tokenized prompt length: caption, lyrics and template. Defaults to 1024. A prompt above the checkpoint's limit, or one that leaves too little context for the frames, is `workload-too-large`. |
779
- | `threads`, `backendsDir`, `marginBytes` | As for ACE-Step. |
815
+ | Option | Description |
816
+ | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
817
+ | `device` | `'cpu'`, `'gpu'` or `'auto'`, as `config.device` takes it. Without it, `gpuLayers` greater than 0 projects `'auto'` and anything else `'cpu'`. |
818
+ | `durationSeconds`, `maxFrames` | The generation length, as `run()` takes it (25 frames per second); one or the other. Defaults to 300 frames. |
819
+ | `promptTokens` | Tokenized prompt length: caption, lyrics and template. Defaults to 1024. A prompt above the checkpoint's limit, or one that leaves too little context for the frames, is `workload-too-large`. |
820
+ | `threads`, `backendsDir`, `marginBytes` | As for ACE-Step. |
780
821
 
781
822
  On Android and iOS, where MiniMax-Music3 is unavailable, `engine: 'minimax'`
782
823
  comes back as `status: "error"` with `reason: "unsupported-engine"`.
@@ -827,10 +868,9 @@ dispatches to the corresponding `audiogen-cpp` engine.
827
868
  - `addon/src/model-interface/minimax/` — desktop-only `MinimaxModel`.
828
869
  - Built with `cmake-bare` + `cmake-vcpkg`; `vcpkg.json` depends on the
829
870
  `speech-cpp[audiogen]` port.
830
- - `ggml-speech` and `speech-cpp` are pinned with vcpkg `overrides` to the same
831
- versions in every speech addon (`@qvac/asr-ggml`, `@qvac/audiogen-ggml`,
832
- `@qvac/bci-whispercpp`, `@qvac/tts-ggml`), and `package.json` records the
833
- `ggml-speech` build as `qvac["ggml-speech"]`. The speech addons ship their
871
+ - Native dependency versions are pinned with vcpkg `overrides` in
872
+ [vcpkg.json](vcpkg.json), and `package.json` records the `ggml-speech`
873
+ build as `qvac["ggml-speech"]`. The speech addons ship their
834
874
  ggml backends under the same file names, so an app that bundles several of
835
875
  them keeps one copy of each: use releases that record the same
836
876
  `ggml-speech`.
package/audiogen.d.ts CHANGED
@@ -11,6 +11,8 @@ export interface AudioGenConfigurationParams {
11
11
  useGPU?: boolean;
12
12
  /** MiniMax compute device ('cpu' | 'gpu' | 'auto'); overrides useGPU. */
13
13
  device?: string;
14
+ coremlProfileDir?: string;
15
+ coremlComputeUnits?: string;
14
16
  nGpuLayers?: number;
15
17
  threads?: number;
16
18
  /**
@@ -48,7 +48,11 @@ async function main() {
48
48
  const generator = new AudioGen({
49
49
  engine: ENGINE_MINIMAX,
50
50
  files: { modelDir: requiredModelDir() },
51
- config: { threads: numberFromEnv('AUDIOGEN_THREADS') }
51
+ config: {
52
+ threads: numberFromEnv('AUDIOGEN_THREADS'),
53
+ coremlProfileDir: process.env.AUDIOGEN_COREML_PROFILE_DIR,
54
+ coremlComputeUnits: process.env.AUDIOGEN_COREML_COMPUTE_UNITS
55
+ }
52
56
  })
53
57
  try {
54
58
  await generator.load()
package/index.d.ts CHANGED
@@ -10,6 +10,8 @@ export declare const MINIMAX_DEFAULT_MAX_FRAMES = 300;
10
10
  export type AudioGenEngine = typeof ENGINE_ACESTEP | typeof ENGINE_MINIMAX;
11
11
  /** MiniMax compute device, see `AudioGenRuntimeConfig.device`. */
12
12
  export type AudiogenMinimaxDevice = 'cpu' | 'gpu' | 'auto';
13
+ /** Allowed devices for Apple's full MiniMax Core ML runtime. */
14
+ export type AudiogenMinimaxCoreMLComputeUnits = 'CPU_ONLY' | 'CPU_AND_GPU' | 'CPU_AND_NE' | 'ALL';
13
15
  /** Model file paths for ACE-Step or MiniMax-Music3. */
14
16
  export interface AudioGenFiles {
15
17
  /** Directory holding the four ACE-Step GGUFs (engine auto-classifies them). */
@@ -47,6 +49,10 @@ export interface AudioGenRuntimeConfig {
47
49
  * CPU, `'cpu'` never touches a GPU.
48
50
  */
49
51
  device?: AudiogenMinimaxDevice;
52
+ /** MiniMax only: compiled full Core ML profile directory on the inference host (macOS 15+). */
53
+ coremlProfileDir?: string;
54
+ /** MiniMax only: explicit Core ML device policy; requires coremlProfileDir. */
55
+ coremlComputeUnits?: AudiogenMinimaxCoreMLComputeUnits;
50
56
  /** ACE-Step only: GPU layers to offload when `useGPU` is set (99 = all). */
51
57
  nGpuLayers?: number;
52
58
  /** 0 = engine auto-picks. */
package/index.js CHANGED
@@ -35,6 +35,12 @@ const INT32_MAX = 2147483647;
35
35
  const FLOAT32_MAX = 3.4028234663852886e38;
36
36
  const FLOAT32_MIN_POSITIVE = 1.401298464324817e-45;
37
37
  const MINIMAX_DEVICES = ['cpu', 'gpu', 'auto'];
38
+ const MINIMAX_COREML_COMPUTE_UNITS = [
39
+ 'CPU_ONLY',
40
+ 'CPU_AND_GPU',
41
+ 'CPU_AND_NE',
42
+ 'ALL'
43
+ ];
38
44
  /** `AudiogenStats.backendId` codes, named. Codes match @qvac/tts-ggml. */
39
45
  exports.AUDIOGEN_BACKEND_NAMES = {
40
46
  0: 'cpu',
@@ -359,8 +365,26 @@ function validateAcestepOptions(files, config) {
359
365
  if (config.device !== undefined) {
360
366
  throw invalidInput('ACE-Step does not accept device (use useGPU and nGpuLayers)');
361
367
  }
368
+ if (config.coremlProfileDir !== undefined || config.coremlComputeUnits !== undefined) {
369
+ throw invalidInput('coremlProfileDir and coremlComputeUnits are MiniMax-only options');
370
+ }
371
+ }
372
+ function validateMinimaxCoreMLConfig(config) {
373
+ if (config.coremlProfileDir === undefined && config.coremlComputeUnits === undefined)
374
+ return;
375
+ if (typeof config.coremlProfileDir !== 'string' ||
376
+ config.coremlProfileDir.length === 0 ||
377
+ config.coremlProfileDir.includes('\0') ||
378
+ typeof config.coremlComputeUnits !== 'string' ||
379
+ !MINIMAX_COREML_COMPUTE_UNITS.includes(config.coremlComputeUnits)) {
380
+ throw invalidInput('MiniMax Core ML requires coremlProfileDir and coremlComputeUnits (CPU_ONLY, CPU_AND_GPU, CPU_AND_NE or ALL)');
381
+ }
382
+ if (os.platform() !== 'darwin') {
383
+ throw invalidInput('MiniMax Core ML requires macOS 15 or later');
384
+ }
362
385
  }
363
386
  function validateMinimaxConfig(config) {
387
+ validateMinimaxCoreMLConfig(config);
364
388
  if (config.useGPU !== undefined && typeof config.useGPU !== 'boolean') {
365
389
  throw invalidInput('useGPU must be a boolean');
366
390
  }
@@ -586,6 +610,8 @@ class AudioGen {
586
610
  threads,
587
611
  useGPU: config.useGPU ?? false,
588
612
  device: config.device,
613
+ coremlProfileDir: config.coremlProfileDir,
614
+ coremlComputeUnits: config.coremlComputeUnits,
589
615
  backendsDir
590
616
  };
591
617
  }
package/lib/fit.d.ts CHANGED
@@ -26,6 +26,9 @@ export interface AudiogenFitRequest {
26
26
  * > 0 projects `auto` (a GPU when one is usable) and anything else `cpu`.
27
27
  */
28
28
  device?: 'cpu' | 'gpu' | 'auto'
29
+ /** MiniMax full Core ML profile. Projection returns coreml-memory-estimate-unavailable. */
30
+ coremlProfileDir?: string
31
+ coremlComputeUnits?: 'CPU_ONLY' | 'CPU_AND_GPU' | 'CPU_AND_NE' | 'ALL'
29
32
  threads?: number
30
33
  backendsDir?: string
31
34
 
package/package.json CHANGED
@@ -1,9 +1,9 @@
1
1
  {
2
2
  "name": "@qvac/audiogen-ggml",
3
- "version": "0.9.0",
3
+ "version": "0.9.1",
4
4
  "description": "Audio generation (music) addon for qvac",
5
5
  "qvac": {
6
- "ggml-speech": "2026-10-23"
6
+ "ggml-speech": "2026-10-27"
7
7
  },
8
8
  "engines": {
9
9
  "bare": ">=1.19.0"
@@ -156,19 +156,19 @@
156
156
  "bare-process": "^4.2.2"
157
157
  },
158
158
  "optionalDependencies": {
159
- "@qvac/audiogen-ggml-linux-x64": "0.9.0",
160
- "@qvac/audiogen-ggml-linux-arm64": "0.9.0",
161
- "@qvac/audiogen-ggml-darwin-arm64": "0.9.0",
162
- "@qvac/audiogen-ggml-darwin-x64": "0.9.0",
163
- "@qvac/audiogen-ggml-win32-x64": "0.9.0"
159
+ "@qvac/audiogen-ggml-linux-x64": "0.9.1",
160
+ "@qvac/audiogen-ggml-linux-arm64": "0.9.1",
161
+ "@qvac/audiogen-ggml-darwin-arm64": "0.9.1",
162
+ "@qvac/audiogen-ggml-darwin-x64": "0.9.1",
163
+ "@qvac/audiogen-ggml-win32-x64": "0.9.1"
164
164
  },
165
165
  "overrides": {
166
166
  "bare-module-lexer": "1.6.3"
167
167
  },
168
168
  "types": "index.d.ts",
169
169
  "peerDependencies": {
170
- "@qvac/audiogen-ggml-android-arm64": "0.9.0",
171
- "@qvac/audiogen-ggml-ios": "0.9.0"
170
+ "@qvac/audiogen-ggml-android-arm64": "0.9.1",
171
+ "@qvac/audiogen-ggml-ios": "0.9.1"
172
172
  },
173
173
  "peerDependenciesMeta": {
174
174
  "@qvac/audiogen-ggml-android-arm64": {
@@ -47,6 +47,15 @@ async function runAudioGen(gen, { caption, opts = {} } = {}) {
47
47
  }
48
48
 
49
49
  async function collectAudioGenResponse(response) {
50
+ const [{ chunks, stages, stageTotals, sampleRate, channels }, stats] = await Promise.all([
51
+ collectAudioGenOutput(response),
52
+ response.await()
53
+ ])
54
+
55
+ return summarizeAudioGenOutput({ chunks, stages, stageTotals, sampleRate, channels, stats })
56
+ }
57
+
58
+ async function collectAudioGenOutput(response) {
50
59
  const chunks = []
51
60
  const stages = new Set()
52
61
  const stageTotals = {}
@@ -68,8 +77,10 @@ async function collectAudioGenResponse(response) {
68
77
  }
69
78
  }
70
79
 
71
- const stats = await response.await()
80
+ return { chunks, stages, stageTotals, sampleRate, channels }
81
+ }
72
82
 
83
+ function summarizeAudioGenOutput({ chunks, stages, stageTotals, sampleRate, channels, stats }) {
73
84
  let sampleCount = 0
74
85
  for (const chunk of chunks) sampleCount += chunk.length
75
86