@qvac/audiogen-ggml 0.4.1 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +23 -0
- package/README.md +52 -0
- package/index.d.ts +3 -1
- package/index.js +3 -1
- package/lib/fit.d.ts +88 -0
- package/lib/fit.js +30 -0
- package/package.json +6 -6
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.5.0] - 2026-09-25
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- Raise the `ggml-speech` floor to `2026-09-23` and the `speech-cpp` floor to
|
|
15
|
+
`2026-09-23#1`. The speech ggml now tracks upstream ggml 0.20.2 (was 0.10.2),
|
|
16
|
+
and its Vulkan backend no longer crashes on NVIDIA GPUs that report
|
|
17
|
+
cooperative-matrix2 support when a matmul is asked for f32 precision. The
|
|
18
|
+
ACE-Step memory-fit preflight now returns a verdict instead of aborting the
|
|
19
|
+
process: it priced the VAE decode/encode windows through a graph scheduler
|
|
20
|
+
that cannot place a metadata-only model's weights, so every ACE-Step model
|
|
21
|
+
hit a ggml assertion on CPU and GPU alike. Generation itself is unchanged.
|
|
22
|
+
- Raise the `speech-cpp` floor to `2026-09-18`. MiniMax-Music3 generates faster
|
|
23
|
+
with the model files you already have: the RVQ depth decoder keeps a K/V cache
|
|
24
|
+
instead of recomputing its prefix on every codebook step, and the flow default
|
|
25
|
+
is 20 steps instead of 30 (`inferenceSteps` still overrides it). On an AMD
|
|
26
|
+
Strix Halo a 10 s track goes from 57.0 s to 39.1 s on Vulkan. Synth files
|
|
27
|
+
re-converted with the new engine also quantize the depth decoder with the
|
|
28
|
+
chosen tier, worth a further 2.6 s there.
|
|
29
|
+
- Raise the `speech-cpp` floor to `2026-09-16`, keeping the speech packages on
|
|
30
|
+
one engine stack. Nothing in the window touches the audiogen engine, so
|
|
31
|
+
published behavior is unchanged.
|
|
32
|
+
|
|
10
33
|
## [0.4.1] - 2026-09-16
|
|
11
34
|
|
|
12
35
|
### Changed
|
package/README.md
CHANGED
|
@@ -658,6 +658,58 @@ npx qvac-audiogen-download-models --output ./models/audiogen --variant turbo-q4
|
|
|
658
658
|
`sft`, or `all`; it defaults to `turbo-q4`. Use `--help` to print the flags.
|
|
659
659
|
The command does not write into the installed package.
|
|
660
660
|
|
|
661
|
+
## Assessing fit
|
|
662
|
+
|
|
663
|
+
`assessFit` projects a load against the memory free right now. It reads GGUF
|
|
664
|
+
metadata and never weight data, so the registry's weightless copy of each stage
|
|
665
|
+
answers the same as the file itself and the projection can run before anything
|
|
666
|
+
is downloaded. It is a module export, not an instance method — nothing is
|
|
667
|
+
loaded to call it.
|
|
668
|
+
|
|
669
|
+
```js
|
|
670
|
+
const { assessFit } = require('@qvac/audiogen-ggml')
|
|
671
|
+
|
|
672
|
+
const fit = assessFit({
|
|
673
|
+
modelsDir: '/models/ace-step',
|
|
674
|
+
durationSeconds: 30,
|
|
675
|
+
textTokens: 256,
|
|
676
|
+
lyricTokens: 256
|
|
677
|
+
})
|
|
678
|
+
|
|
679
|
+
fit.status // 'fits' | 'does-not-fit' | 'error'
|
|
680
|
+
fit.reason // the engine's own wording, e.g. 'model-unreadable', 'workload-too-large'
|
|
681
|
+
fit.modelName
|
|
682
|
+
fit.isTurbo
|
|
683
|
+
fit.deviceName
|
|
684
|
+
fit.deviceBytes // peak across the pipeline phases under the projected residency mode
|
|
685
|
+
fit.hostBytes
|
|
686
|
+
fit.hostFreeBytes // host capacity, a budget of its own where the device has its own memory
|
|
687
|
+
fit.stagesResident
|
|
688
|
+
fit.report
|
|
689
|
+
```
|
|
690
|
+
|
|
691
|
+
`modelsDir` holds the four stage GGUFs; `textEncoderPath`, `lmPath`, `ditPath`
|
|
692
|
+
and `vaePath` name them individually and win over it.
|
|
693
|
+
|
|
694
|
+
| Option | Description |
|
|
695
|
+
| --- | --- |
|
|
696
|
+
| `durationSeconds` | Longest single generation the projection must accommodate. |
|
|
697
|
+
| `textTokens`, `lyricTokens` | The prompt the projection is sized for. |
|
|
698
|
+
| `lmPromptTokens` | 0 derives it from the text and lyric budgets. |
|
|
699
|
+
| `lmMaxNewTokens` | 0 derives it from the duration, as the pipeline does. |
|
|
700
|
+
| `lmCfgScale`, `guidanceScale` | 0 picks CFG or no CFG from the checkpoint. |
|
|
701
|
+
| `withSourceAudio` | Projects the extra VAE-encoder phase a cover or reference request loads. |
|
|
702
|
+
| `keepStages` | `-1` mirrors the engine, `0` forces the staged projection, `1` all-resident. |
|
|
703
|
+
| `gpuLayers` | Greater than 0 requests the GPU stack, with the fallbacks a real load applies. |
|
|
704
|
+
| `threads`, `backendsDir` | As the load takes them. |
|
|
705
|
+
| `marginBytes` | Free memory that must remain for the projection to count as fitting. Defaults to 256 MiB. |
|
|
706
|
+
|
|
707
|
+
`deviceSharesHostMemory` reports that the device pool is system RAM, so host
|
|
708
|
+
bytes compete with device bytes.
|
|
709
|
+
|
|
710
|
+
A model the engine cannot read is `status: "error"`; a broken request, or a
|
|
711
|
+
host with no native binding, throws.
|
|
712
|
+
|
|
661
713
|
## Lifecycle
|
|
662
714
|
|
|
663
715
|
Call `load()` before `run()`. Overlapping `run()` calls are admitted in order;
|
package/index.d.ts
CHANGED
|
@@ -79,7 +79,7 @@ export interface GenerateOptions {
|
|
|
79
79
|
duration?: number;
|
|
80
80
|
/** MiniMax semantic-frame cap. Cannot be combined with `duration`. */
|
|
81
81
|
maxFrames?: number;
|
|
82
|
-
/** MiniMax flow steps for this generation; 0 uses the
|
|
82
|
+
/** MiniMax flow steps for this generation; 0 uses the engine default (20). */
|
|
83
83
|
inferenceSteps?: number;
|
|
84
84
|
/** MiniMax flow classifier-free guidance scale for this generation. */
|
|
85
85
|
cfgScale?: number;
|
|
@@ -459,6 +459,8 @@ export type { DitVariant, ModelManifest, ModelSources, ResolveDitModelPathOption
|
|
|
459
459
|
export { encodePcm, pcmToWav, SUPPORTED_FORMATS as OUTPUT_FORMATS } from './lib/audio-format';
|
|
460
460
|
export type { OutputFormat, EncodeOptions, EncodedAudio } from './lib/audio-format';
|
|
461
461
|
export { resolveBackendsDir } from './lib/backends';
|
|
462
|
+
export { assessFit } from './lib/fit';
|
|
463
|
+
export type { AudiogenFitRequest, AudiogenFitResult, AudiogenFitStatus } from './lib/fit';
|
|
462
464
|
export { ERR_CODE_RANGE, ERR_CODES, QvacErrorAudioGen } from './error';
|
|
463
465
|
export { AudioEditOperationType, RepaintMode } from './audiogen';
|
|
464
466
|
export type { AudioGenConfigurationParams, AudioGenJobData, AudioGenBinding, AudioGenOutputCallback } from './audiogen';
|
package/index.js
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
// streams the engine's output (progress ticks + one interleaved-Int16 PCM
|
|
12
12
|
// chunk) and resolves with the run stats.
|
|
13
13
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
14
|
-
exports.RepaintMode = exports.AudioEditOperationType = exports.QvacErrorAudioGen = exports.ERR_CODES = exports.ERR_CODE_RANGE = exports.resolveBackendsDir = exports.OUTPUT_FORMATS = exports.pcmToWav = exports.encodePcm = exports.allRegistryPaths = exports.resolveDitModelPath = exports.modelSources = exports.modelManifest = exports.modelFilenames = exports.registryPath = exports.ditFilename = exports.ditVariants = exports.DEFAULT_DIT_VARIANT = exports.DIT_VARIANTS = exports.FIXED_MODELS = exports.REGISTRY_PREFIX = exports.REGISTRY_SOURCE = exports.AudioGen = exports.AudioEditSession = exports.AUDIOGEN_GPU_FALLBACK_REASONS = exports.AUDIOGEN_BACKEND_NAMES = exports.MINIMAX_DEFAULT_MAX_FRAMES = exports.MINIMAX_FRAMES_PER_SECOND = exports.ENGINE_MINIMAX = exports.ENGINE_ACESTEP = void 0;
|
|
14
|
+
exports.RepaintMode = exports.AudioEditOperationType = exports.QvacErrorAudioGen = exports.ERR_CODES = exports.ERR_CODE_RANGE = exports.assessFit = exports.resolveBackendsDir = exports.OUTPUT_FORMATS = exports.pcmToWav = exports.encodePcm = exports.allRegistryPaths = exports.resolveDitModelPath = exports.modelSources = exports.modelManifest = exports.modelFilenames = exports.registryPath = exports.ditFilename = exports.ditVariants = exports.DEFAULT_DIT_VARIANT = exports.DIT_VARIANTS = exports.FIXED_MODELS = exports.REGISTRY_PREFIX = exports.REGISTRY_SOURCE = exports.AudioGen = exports.AudioEditSession = exports.AUDIOGEN_GPU_FALLBACK_REASONS = exports.AUDIOGEN_BACKEND_NAMES = exports.MINIMAX_DEFAULT_MAX_FRAMES = exports.MINIMAX_FRAMES_PER_SECOND = exports.ENGINE_MINIMAX = exports.ENGINE_ACESTEP = void 0;
|
|
15
15
|
exports.audiogenBackendName = audiogenBackendName;
|
|
16
16
|
exports.audiogenGpuFallbackReason = audiogenGpuFallbackReason;
|
|
17
17
|
exports.detectEngineType = detectEngineType;
|
|
@@ -1079,6 +1079,8 @@ Object.defineProperty(exports, "pcmToWav", { enumerable: true, get: function ()
|
|
|
1079
1079
|
Object.defineProperty(exports, "OUTPUT_FORMATS", { enumerable: true, get: function () { return audio_format_2.SUPPORTED_FORMATS; } });
|
|
1080
1080
|
var backends_2 = require("./lib/backends");
|
|
1081
1081
|
Object.defineProperty(exports, "resolveBackendsDir", { enumerable: true, get: function () { return backends_2.resolveBackendsDir; } });
|
|
1082
|
+
var fit_1 = require("./lib/fit");
|
|
1083
|
+
Object.defineProperty(exports, "assessFit", { enumerable: true, get: function () { return fit_1.assessFit; } });
|
|
1082
1084
|
var error_2 = require("./error");
|
|
1083
1085
|
Object.defineProperty(exports, "ERR_CODE_RANGE", { enumerable: true, get: function () { return error_2.ERR_CODE_RANGE; } });
|
|
1084
1086
|
Object.defineProperty(exports, "ERR_CODES", { enumerable: true, get: function () { return error_2.ERR_CODES; } });
|
package/lib/fit.d.ts
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
export type AudiogenFitEngine = 'acestep' | 'minimax'
|
|
2
|
+
|
|
3
|
+
export interface AudiogenFitRequest {
|
|
4
|
+
/** Engine that would run the load. Defaults to `acestep`. */
|
|
5
|
+
engine?: AudiogenFitEngine
|
|
6
|
+
/** Directory holding the four stage GGUFs; explicit paths win over it. */
|
|
7
|
+
modelsDir?: string
|
|
8
|
+
textEncoderPath?: string
|
|
9
|
+
lmPath?: string
|
|
10
|
+
ditPath?: string
|
|
11
|
+
vaePath?: string
|
|
12
|
+
|
|
13
|
+
/** > 0 requests the GPU stack, with the fallbacks a real load applies. */
|
|
14
|
+
gpuLayers?: number
|
|
15
|
+
threads?: number
|
|
16
|
+
backendsDir?: string
|
|
17
|
+
|
|
18
|
+
/** Longest single generation the projection must accommodate. */
|
|
19
|
+
durationSeconds?: number
|
|
20
|
+
textTokens?: number
|
|
21
|
+
lyricTokens?: number
|
|
22
|
+
/** 0 derives it from the text and lyric budgets. */
|
|
23
|
+
lmPromptTokens?: number
|
|
24
|
+
/** 0 derives it from the duration, as the pipeline does. */
|
|
25
|
+
lmMaxNewTokens?: number
|
|
26
|
+
lmCfgScale?: number
|
|
27
|
+
/** 0 picks CFG or no CFG from the checkpoint. */
|
|
28
|
+
guidanceScale?: number
|
|
29
|
+
/** Projects the extra VAE-encoder phase a cover or reference request loads. */
|
|
30
|
+
withSourceAudio?: boolean
|
|
31
|
+
/** -1 mirrors the engine, 0 forces the staged projection, 1 all-resident. */
|
|
32
|
+
keepStages?: number
|
|
33
|
+
/** Free memory that must remain for the projection to count as fitting. Defaults to 256 MiB. */
|
|
34
|
+
marginBytes?: number
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
export type AudiogenFitStatus = 'fits' | 'does-not-fit' | 'error'
|
|
38
|
+
|
|
39
|
+
/** One pipeline stage, as the fitter placed it. */
|
|
40
|
+
export interface AudiogenFitStage {
|
|
41
|
+
/** `textenc` | `lm` | `cond` | `detok` | `dit` | `vae`. */
|
|
42
|
+
name: string
|
|
43
|
+
/** Backend the stage loads on, after placement. */
|
|
44
|
+
deviceName: string
|
|
45
|
+
onGpu: boolean
|
|
46
|
+
weightsBytes: number
|
|
47
|
+
/** The portion mapped in place from the file, on the host. */
|
|
48
|
+
weightsMmapBytes: number
|
|
49
|
+
/** The LM KV cache. */
|
|
50
|
+
stateBytes: number
|
|
51
|
+
/** Largest compute arena the stage holds at once. */
|
|
52
|
+
computeBytes: number
|
|
53
|
+
/** Stage-phase host buffers: masks, latents, PCM. */
|
|
54
|
+
hostBytes: number
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
export interface AudiogenFitResult {
|
|
58
|
+
status: AudiogenFitStatus
|
|
59
|
+
/**
|
|
60
|
+
* The engine's own wording, e.g. `model-unreadable`, `workload-too-large`.
|
|
61
|
+
* `unsupported-engine` names an engine the addon loads and the fitter does
|
|
62
|
+
* not cover.
|
|
63
|
+
*/
|
|
64
|
+
reason: string
|
|
65
|
+
modelName: string
|
|
66
|
+
isTurbo: boolean
|
|
67
|
+
deviceName: string
|
|
68
|
+
deviceIsCpu: boolean
|
|
69
|
+
/** The device pool is system RAM, so host bytes compete with device bytes. */
|
|
70
|
+
deviceSharesHostMemory: boolean
|
|
71
|
+
deviceFreeBytes: number
|
|
72
|
+
deviceTotalBytes: number
|
|
73
|
+
/** Peak across the pipeline phases under the projected residency mode. */
|
|
74
|
+
deviceBytes: number
|
|
75
|
+
hostBytes: number
|
|
76
|
+
/** Host capacity, which is a budget of its own where the device has its own memory. */
|
|
77
|
+
hostFreeBytes: number
|
|
78
|
+
hostTotalBytes: number
|
|
79
|
+
stagesResident: boolean
|
|
80
|
+
/**
|
|
81
|
+
* One row per pipeline stage. `deviceBytes` is a peak across phases, so the
|
|
82
|
+
* rows are where the weights, state and compute split apart.
|
|
83
|
+
*/
|
|
84
|
+
stages: AudiogenFitStage[]
|
|
85
|
+
report: string
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
export function assessFit(request: AudiogenFitRequest): AudiogenFitResult
|
package/lib/fit.js
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
'use strict'
|
|
2
|
+
|
|
3
|
+
const { resolveBackendsDir } = require('./backends')
|
|
4
|
+
|
|
5
|
+
/**
|
|
6
|
+
* Projects one model set against the memory free right now, reading GGUF
|
|
7
|
+
* metadata and never weight data. The registry's weightless copy of each stage
|
|
8
|
+
* answers the same as the stage itself, so this can run before anything is
|
|
9
|
+
* downloaded.
|
|
10
|
+
*
|
|
11
|
+
* `engine` defaults to `acestep`, the one engine audiogen-cpp ships a fitter
|
|
12
|
+
* for. Any other engine comes back as `status: "error"` with
|
|
13
|
+
* `reason: "unsupported-engine"`.
|
|
14
|
+
*
|
|
15
|
+
* A model set the engine cannot read comes back as `status: "error"` with its
|
|
16
|
+
* own reason; a broken request, or a host with no native binding, throws.
|
|
17
|
+
*/
|
|
18
|
+
function assessFit(request) {
|
|
19
|
+
const binding = require('../binding.js')
|
|
20
|
+
|
|
21
|
+
return binding.assessFit({
|
|
22
|
+
...request,
|
|
23
|
+
backendsDir:
|
|
24
|
+
typeof request.backendsDir === 'string' && request.backendsDir.length > 0
|
|
25
|
+
? request.backendsDir
|
|
26
|
+
: resolveBackendsDir()
|
|
27
|
+
})
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
module.exports = { assessFit }
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@qvac/audiogen-ggml",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0",
|
|
4
4
|
"description": "Audio generation (music) addon for qvac",
|
|
5
5
|
"addon": true,
|
|
6
6
|
"engines": {
|
|
@@ -173,10 +173,10 @@
|
|
|
173
173
|
},
|
|
174
174
|
"types": "index.d.ts",
|
|
175
175
|
"optionalDependencies": {
|
|
176
|
-
"@qvac/audiogen-ggml-linux-x64": "0.
|
|
177
|
-
"@qvac/audiogen-ggml-linux-arm64": "0.
|
|
178
|
-
"@qvac/audiogen-ggml-darwin-arm64": "0.
|
|
179
|
-
"@qvac/audiogen-ggml-darwin-x64": "0.
|
|
180
|
-
"@qvac/audiogen-ggml-win32-x64": "0.
|
|
176
|
+
"@qvac/audiogen-ggml-linux-x64": "0.5.0",
|
|
177
|
+
"@qvac/audiogen-ggml-linux-arm64": "0.5.0",
|
|
178
|
+
"@qvac/audiogen-ggml-darwin-arm64": "0.5.0",
|
|
179
|
+
"@qvac/audiogen-ggml-darwin-x64": "0.5.0",
|
|
180
|
+
"@qvac/audiogen-ggml-win32-x64": "0.5.0"
|
|
181
181
|
}
|
|
182
182
|
}
|