@qvac/audiogen-ggml 0.9.1 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +28 -0
- package/README.md +34 -25
- package/index.d.ts +1 -1
- package/lib/fit.d.ts +24 -48
- package/lib/fit.js +4 -5
- package/package.json +11 -9
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,34 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.10.0] - 2026-11-05
|
|
11
|
+
|
|
12
|
+
### Breaking
|
|
13
|
+
|
|
14
|
+
- `assessFit` returns `{ status, devices, stats }`: each device the pipeline
|
|
15
|
+
uses, then the host CPU. `status` is `fits`, `does-not-fit` or `unknown`. An
|
|
16
|
+
unreadable model or an invalid request rejects
|
|
17
|
+
([#5474](https://github.com/tetherto/qvac/pull/5474)).
|
|
18
|
+
- On an iOS device, `assessFit` checks the load and its margin against the
|
|
19
|
+
app's memory limit ([#5474](https://github.com/tetherto/qvac/pull/5474)).
|
|
20
|
+
|
|
21
|
+
### Changed
|
|
22
|
+
|
|
23
|
+
- Update the native ACE-Step dependency and reject unusable automatic GPU backends.
|
|
24
|
+
|
|
25
|
+
- Update the shared `speech-cpp` native dependency to `2026-10-29`
|
|
26
|
+
([#5420](https://github.com/tetherto/qvac/pull/5420),
|
|
27
|
+
[#5447](https://github.com/tetherto/qvac/pull/5447)).
|
|
28
|
+
- Update the shared `speech-cpp` native dependency to `2026-11-03`
|
|
29
|
+
([registry #474](https://github.com/tetherto/qvac-registry-vcpkg/pull/474)).
|
|
30
|
+
- `assessFit` names the device and each stage's device by its ggml device name
|
|
31
|
+
(`CPU`, `CUDA0`, `MTL0`, `GPUOpenCL`), counts Android's OpenCL GPU as sharing
|
|
32
|
+
host memory, and gives a device that reports no memory of its own the
|
|
33
|
+
host's free and total figures
|
|
34
|
+
([#5447](https://github.com/tetherto/qvac/pull/5447)).
|
|
35
|
+
|
|
8
36
|
## [0.9.1] - 2026-10-28
|
|
9
37
|
|
|
10
38
|
### Added
|
package/README.md
CHANGED
|
@@ -762,18 +762,30 @@ const fit = await assessFit({
|
|
|
762
762
|
lyricTokens: 256
|
|
763
763
|
})
|
|
764
764
|
|
|
765
|
-
fit.status // 'fits' | 'does-not-fit' | '
|
|
766
|
-
fit.
|
|
767
|
-
fit.
|
|
768
|
-
fit.isTurbo
|
|
769
|
-
fit.deviceName
|
|
770
|
-
fit.deviceBytes // peak across the pipeline phases under the projected residency mode
|
|
771
|
-
fit.hostBytes
|
|
772
|
-
fit.hostFreeBytes // host capacity, a budget of its own where the device has its own memory
|
|
773
|
-
fit.stagesResident
|
|
774
|
-
fit.report
|
|
765
|
+
fit.status // 'fits' | 'does-not-fit' | 'unknown'
|
|
766
|
+
fit.devices // [{ id, name, unified, usedBytes, freeBytes, totalBytes }]
|
|
767
|
+
fit.stats // {}
|
|
775
768
|
```
|
|
776
769
|
|
|
770
|
+
`devices` holds each device the pipeline uses, then the host CPU, which holds
|
|
771
|
+
the CPU-placed stages, the mapped weights and the host-side buffers. A load on
|
|
772
|
+
the CPU is one row. `usedBytes` is the load's peak on that device across the
|
|
773
|
+
pipeline phases, under the projected residency mode. `id` is the PCI bus ID when
|
|
774
|
+
the backend reports one, so the same GPU matches across addons built on
|
|
775
|
+
different backends, and the device name otherwise. `unified` is true for memory
|
|
776
|
+
the host owns: the CPU, an integrated GPU, a GPU that shares the host's memory,
|
|
777
|
+
Android's OpenCL GPU, and a device that reports no memory of its own, which
|
|
778
|
+
takes the host's figures.
|
|
779
|
+
|
|
780
|
+
On an iOS device every unified row shares the app's own memory limit,
|
|
781
|
+
`os_proc_available_memory()`. Each such row's free memory is capped at that
|
|
782
|
+
limit, and a load whose unified rows together take more than the limit less
|
|
783
|
+
the margin does not fit. A limit of 0 means the app has none left.
|
|
784
|
+
|
|
785
|
+
`status` is `unknown`, with no rows, for a load the fitter cannot size:
|
|
786
|
+
MiniMax-Music3 with Core ML stages, which have no memory estimate, or a load
|
|
787
|
+
whose memory the engine fails to measure.
|
|
788
|
+
|
|
777
789
|
`modelsDir` holds the four stage GGUFs; `textEncoderPath`, `lmPath`, `ditPath`
|
|
778
790
|
and `vaePath` name them individually and win over it.
|
|
779
791
|
|
|
@@ -790,17 +802,13 @@ and `vaePath` name them individually and win over it.
|
|
|
790
802
|
| `threads`, `backendsDir` | As the load takes them. |
|
|
791
803
|
| `marginBytes` | Free memory that must remain for the projection to count as fitting. Defaults to 256 MiB. |
|
|
792
804
|
|
|
793
|
-
`deviceSharesHostMemory` reports that the device pool is system RAM, so host
|
|
794
|
-
bytes compete with device bytes.
|
|
795
|
-
|
|
796
805
|
### MiniMax-Music3
|
|
797
806
|
|
|
798
807
|
`engine: 'minimax'` projects a MiniMax-Music3 pair on desktop builds, with the
|
|
799
808
|
same result shape. `modelsDir` holds the pair; `lmPath` and `synthPath` name the
|
|
800
809
|
two GGUFs and win over it. The weights and the synthesis graphs stay resident
|
|
801
|
-
between runs and the LM's cache is rebuilt by each run, so `
|
|
802
|
-
peak of repeated generations
|
|
803
|
-
`dit` and `vocoder`.
|
|
810
|
+
between runs and the LM's cache is rebuilt by each run, so `usedBytes` is the
|
|
811
|
+
peak of repeated generations.
|
|
804
812
|
|
|
805
813
|
```js
|
|
806
814
|
const fit = await assessFit({
|
|
@@ -812,18 +820,19 @@ const fit = await assessFit({
|
|
|
812
820
|
})
|
|
813
821
|
```
|
|
814
822
|
|
|
815
|
-
| Option | Description
|
|
816
|
-
| --------------------------------------- |
|
|
817
|
-
| `device` | `'cpu'`, `'gpu'` or `'auto'`, as `config.device` takes it. Without it, `gpuLayers` greater than 0 projects `'auto'` and anything else `'cpu'`.
|
|
818
|
-
| `durationSeconds`, `maxFrames` | The generation length, as `run()` takes it (25 frames per second); one or the other. Defaults to 300 frames.
|
|
819
|
-
| `promptTokens` | Tokenized prompt length: caption, lyrics and template. Defaults to 1024. A prompt above the checkpoint's limit, or one that leaves too little context for the frames,
|
|
820
|
-
| `threads`, `backendsDir`, `marginBytes` | As for ACE-Step.
|
|
823
|
+
| Option | Description |
|
|
824
|
+
| --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
825
|
+
| `device` | `'cpu'`, `'gpu'` or `'auto'`, as `config.device` takes it. Without it, `gpuLayers` greater than 0 projects `'auto'` and anything else `'cpu'`. |
|
|
826
|
+
| `durationSeconds`, `maxFrames` | The generation length, as `run()` takes it (25 frames per second); one or the other. Defaults to 300 frames. |
|
|
827
|
+
| `promptTokens` | Tokenized prompt length: caption, lyrics and template. Defaults to 1024. A prompt above the checkpoint's limit, or one that leaves too little context for the frames, rejects with `workload-too-large`. |
|
|
828
|
+
| `threads`, `backendsDir`, `marginBytes` | As for ACE-Step. |
|
|
821
829
|
|
|
822
830
|
On Android and iOS, where MiniMax-Music3 is unavailable, `engine: 'minimax'`
|
|
823
|
-
|
|
831
|
+
rejects.
|
|
824
832
|
|
|
825
|
-
A model the engine cannot read
|
|
826
|
-
host with no native binding
|
|
833
|
+
A model the engine cannot read, an engine with no fitter, or a broken request
|
|
834
|
+
rejects with an `InvalidArgument` error, and a host with no native binding
|
|
835
|
+
rejects as well.
|
|
827
836
|
|
|
828
837
|
## Lifecycle
|
|
829
838
|
|
package/index.d.ts
CHANGED
|
@@ -560,7 +560,7 @@ export type { DitVariant, ModelManifest, ModelSources, ResolveDitModelPathOption
|
|
|
560
560
|
export { encodePcm, pcmToWav, SUPPORTED_FORMATS as OUTPUT_FORMATS } from './lib/audio-format';
|
|
561
561
|
export type { OutputFormat, EncodeOptions, EncodedAudio } from './lib/audio-format';
|
|
562
562
|
export { assessFit } from './lib/fit';
|
|
563
|
-
export type { AudiogenFitRequest, AudiogenFitResult, AudiogenFitStatus } from './lib/fit';
|
|
563
|
+
export type { AudiogenFitRequest, AudiogenFitResult, AudiogenFitStatus, AudiogenFitDevice, AudiogenFitStats } from './lib/fit';
|
|
564
564
|
export { ERR_CODE_RANGE, ERR_CODES, QvacErrorAudioGen } from './error';
|
|
565
565
|
export { AudioEditOperationType, RepaintMode } from './audiogen';
|
|
566
566
|
export type { AudioGenConfigurationParams, AudioGenJobData, AudioGenBinding, AudioGenOutputCallback } from './audiogen';
|
package/lib/fit.d.ts
CHANGED
|
@@ -3,7 +3,7 @@ export type AudiogenFitEngine = 'acestep' | 'minimax'
|
|
|
3
3
|
export interface AudiogenFitRequest {
|
|
4
4
|
/**
|
|
5
5
|
* Engine that would run the load. Defaults to `acestep`. `minimax` is
|
|
6
|
-
* projected on desktop builds; on Android and iOS it
|
|
6
|
+
* projected on desktop builds; on Android and iOS it rejects.
|
|
7
7
|
*/
|
|
8
8
|
engine?: AudiogenFitEngine
|
|
9
9
|
/**
|
|
@@ -58,62 +58,38 @@ export interface AudiogenFitRequest {
|
|
|
58
58
|
marginBytes?: number
|
|
59
59
|
}
|
|
60
60
|
|
|
61
|
-
|
|
61
|
+
/**
|
|
62
|
+
* `unknown` is a load the fitter cannot size: MiniMax with Core ML stages, or
|
|
63
|
+
* a failed measurement.
|
|
64
|
+
*/
|
|
65
|
+
export type AudiogenFitStatus = 'fits' | 'does-not-fit' | 'unknown'
|
|
62
66
|
|
|
63
|
-
|
|
64
|
-
|
|
67
|
+
export interface AudiogenFitDevice {
|
|
68
|
+
/** The PCI bus ID when the backend reports one, the device name otherwise. */
|
|
69
|
+
id: string
|
|
70
|
+
name: string
|
|
71
|
+
/** Memory the host owns: the host CPU, and a GPU that shares the host's memory. */
|
|
72
|
+
unified: boolean
|
|
65
73
|
/**
|
|
66
|
-
*
|
|
67
|
-
*
|
|
74
|
+
* The load's peak on this device across the pipeline phases. MiniMax keeps
|
|
75
|
+
* its stages resident, so its peak is the steady state of repeated runs.
|
|
68
76
|
*/
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
onGpu: boolean
|
|
73
|
-
weightsBytes: number
|
|
74
|
-
/** The portion mapped in place from the file, on the host. */
|
|
75
|
-
weightsMmapBytes: number
|
|
76
|
-
/** The KV cache of the LM, or of the MiniMax depth decoder. */
|
|
77
|
-
stateBytes: number
|
|
78
|
-
/** Largest compute arena the stage holds at once. */
|
|
79
|
-
computeBytes: number
|
|
80
|
-
/** Stage-phase host buffers: masks, latents, PCM, staged graph inputs. */
|
|
81
|
-
hostBytes: number
|
|
77
|
+
usedBytes: number
|
|
78
|
+
freeBytes: number
|
|
79
|
+
totalBytes: number
|
|
82
80
|
}
|
|
83
81
|
|
|
82
|
+
export type AudiogenFitStats = Record<string, never>
|
|
83
|
+
|
|
84
84
|
export interface AudiogenFitResult {
|
|
85
85
|
status: AudiogenFitStatus
|
|
86
86
|
/**
|
|
87
|
-
*
|
|
88
|
-
*
|
|
89
|
-
*
|
|
90
|
-
*/
|
|
91
|
-
reason: string
|
|
92
|
-
modelName: string
|
|
93
|
-
/** ACE-Step turbo checkpoint; always false for MiniMax. */
|
|
94
|
-
isTurbo: boolean
|
|
95
|
-
deviceName: string
|
|
96
|
-
deviceIsCpu: boolean
|
|
97
|
-
/** The device pool is system RAM, so host bytes compete with device bytes. */
|
|
98
|
-
deviceSharesHostMemory: boolean
|
|
99
|
-
deviceFreeBytes: number
|
|
100
|
-
deviceTotalBytes: number
|
|
101
|
-
/**
|
|
102
|
-
* Peak across the pipeline phases under the projected residency mode. MiniMax
|
|
103
|
-
* keeps its stages resident, so this is the steady state of repeated runs.
|
|
104
|
-
*/
|
|
105
|
-
deviceBytes: number
|
|
106
|
-
hostBytes: number
|
|
107
|
-
/** Host capacity, which is a budget of its own where the device has its own memory. */
|
|
108
|
-
hostFreeBytes: number
|
|
109
|
-
hostTotalBytes: number
|
|
110
|
-
stagesResident: boolean
|
|
111
|
-
/**
|
|
112
|
-
* One row per pipeline stage. `deviceBytes` is a peak across phases, so the
|
|
113
|
-
* rows are where the weights, state and compute split apart.
|
|
87
|
+
* Each device the pipeline uses, then the host CPU, which holds the
|
|
88
|
+
* CPU-placed stages, the mapped weights and the host-side buffers. One CPU
|
|
89
|
+
* row for a load on the CPU. Empty when `unknown`.
|
|
114
90
|
*/
|
|
115
|
-
|
|
116
|
-
|
|
91
|
+
devices: AudiogenFitDevice[]
|
|
92
|
+
stats: AudiogenFitStats
|
|
117
93
|
}
|
|
118
94
|
|
|
119
95
|
export function assessFit(request: AudiogenFitRequest): Promise<AudiogenFitResult>
|
package/lib/fit.js
CHANGED
|
@@ -7,12 +7,11 @@
|
|
|
7
7
|
* downloaded.
|
|
8
8
|
*
|
|
9
9
|
* `engine` defaults to `acestep`; `minimax` projects a MiniMax-Music3 pair on
|
|
10
|
-
* desktop builds.
|
|
11
|
-
* `status: "error"` with `reason: "unsupported-engine"`.
|
|
10
|
+
* desktop builds.
|
|
12
11
|
*
|
|
13
|
-
*
|
|
14
|
-
*
|
|
15
|
-
*
|
|
12
|
+
* An engine the build carries no fitter for, a model set the engine cannot
|
|
13
|
+
* read, a broken request, or a host with no native binding rejects. Fits run
|
|
14
|
+
* one at a time within the addon.
|
|
16
15
|
*/
|
|
17
16
|
async function assessFit(request) {
|
|
18
17
|
const binding = require('../binding.js')
|
package/package.json
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@qvac/audiogen-ggml",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.10.0",
|
|
4
4
|
"description": "Audio generation (music) addon for qvac",
|
|
5
5
|
"qvac": {
|
|
6
|
-
"ggml-speech": "2026-10-27"
|
|
6
|
+
"ggml-speech": "2026-10-27#1"
|
|
7
7
|
},
|
|
8
8
|
"engines": {
|
|
9
9
|
"bare": ">=1.19.0"
|
|
@@ -57,6 +57,8 @@
|
|
|
57
57
|
"test:package": "npm run build:ts && brittle test/package/package-contract.test.js",
|
|
58
58
|
"test:node": "node --test scripts/__tests__/*.test.js",
|
|
59
59
|
"test:integration": "npm run build:ts && brittle-bare test/integration/**/*.test.js",
|
|
60
|
+
"test:cpp:build": "bare-make generate -D BUILD_TESTING=ON -D BUILD_FUZZING=OFF && bare-make build --target audiogen_ggml_tests",
|
|
61
|
+
"test:cpp": "npm run test:cpp:build && ./build/audiogen_ggml_tests",
|
|
60
62
|
"test:benchmark:rtf": "npm run build:ts && brittle-bare test/benchmark/rtf-benchmark.test.js",
|
|
61
63
|
"test:benchmark:rtf:matrix": "node scripts/run-rtf-benchmark-matrix.js",
|
|
62
64
|
"fuzz:build": "bare-make generate -D BUILD_FUZZING=ON -D BUILD_TESTING=OFF -D FUZZTEST_FUZZING_MODE=OFF && bare-make build --target audiogen-config-parse-fuzz",
|
|
@@ -156,19 +158,19 @@
|
|
|
156
158
|
"bare-process": "^4.2.2"
|
|
157
159
|
},
|
|
158
160
|
"optionalDependencies": {
|
|
159
|
-
"@qvac/audiogen-ggml-linux-x64": "0.
|
|
160
|
-
"@qvac/audiogen-ggml-linux-arm64": "0.
|
|
161
|
-
"@qvac/audiogen-ggml-darwin-arm64": "0.
|
|
162
|
-
"@qvac/audiogen-ggml-darwin-x64": "0.
|
|
163
|
-
"@qvac/audiogen-ggml-win32-x64": "0.
|
|
161
|
+
"@qvac/audiogen-ggml-linux-x64": "0.10.0",
|
|
162
|
+
"@qvac/audiogen-ggml-linux-arm64": "0.10.0",
|
|
163
|
+
"@qvac/audiogen-ggml-darwin-arm64": "0.10.0",
|
|
164
|
+
"@qvac/audiogen-ggml-darwin-x64": "0.10.0",
|
|
165
|
+
"@qvac/audiogen-ggml-win32-x64": "0.10.0"
|
|
164
166
|
},
|
|
165
167
|
"overrides": {
|
|
166
168
|
"bare-module-lexer": "1.6.3"
|
|
167
169
|
},
|
|
168
170
|
"types": "index.d.ts",
|
|
169
171
|
"peerDependencies": {
|
|
170
|
-
"@qvac/audiogen-ggml-android-arm64": "0.
|
|
171
|
-
"@qvac/audiogen-ggml-ios": "0.
|
|
172
|
+
"@qvac/audiogen-ggml-android-arm64": "0.10.0",
|
|
173
|
+
"@qvac/audiogen-ggml-ios": "0.10.0"
|
|
172
174
|
},
|
|
173
175
|
"peerDependenciesMeta": {
|
|
174
176
|
"@qvac/audiogen-ggml-android-arm64": {
|