@libraz/libsonare 1.5.2 → 1.5.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/README.md +69 -725
  2. package/dist/index.d.ts +1155 -183
  3. package/dist/index.js +4924 -3935
  4. package/dist/index.js.map +1 -1
  5. package/dist/sonare.js +1 -1
  6. package/dist/sonare.wasm +0 -0
  7. package/dist/worklet.d.ts +29 -1
  8. package/dist/worklet.js +1141 -970
  9. package/dist/worklet.js.map +1 -1
  10. package/package.json +15 -3
  11. package/src/_chain_config.ts +46 -0
  12. package/src/audio.ts +24 -4
  13. package/src/codes.ts +27 -4
  14. package/src/effects_mastering.ts +25 -1
  15. package/src/effects_transform.ts +292 -46
  16. package/src/effects_voice_change.ts +46 -53
  17. package/src/feature_core.ts +316 -12
  18. package/src/feature_music.ts +371 -4
  19. package/src/feature_pitch.ts +61 -0
  20. package/src/feature_resample.ts +17 -2
  21. package/src/feature_spectral.ts +349 -2
  22. package/src/feature_spectrogram.ts +336 -0
  23. package/src/index.ts +130 -1
  24. package/src/mastering_chain.ts +305 -50
  25. package/src/mastering_core.ts +232 -16
  26. package/src/mastering_dynamics.ts +66 -10
  27. package/src/mastering_repair.ts +119 -7
  28. package/src/metering.ts +370 -83
  29. package/src/mixer.ts +9 -2
  30. package/src/mixing_oneshot.ts +25 -5
  31. package/src/module_state.ts +1 -1
  32. package/src/project_class.ts +30 -6
  33. package/src/project_internal.ts +12 -2
  34. package/src/project_types.ts +25 -2
  35. package/src/public_types_mastering.ts +114 -4
  36. package/src/public_types_spectral.ts +7 -0
  37. package/src/quick_analysis.ts +325 -117
  38. package/src/realtime_engine.ts +8 -0
  39. package/src/realtime_voice_changer.ts +79 -3
  40. package/src/sonare.js.d.ts +35 -0
  41. package/src/stream_analyzer.ts +20 -1
  42. package/src/stream_types.ts +12 -1
  43. package/src/validation.ts +17 -2
  44. package/src/worklet/engine-clips.ts +67 -2
  45. package/src/worklet/engine-processor.ts +33 -5
  46. package/src/worklet/engine.ts +7 -3
  47. package/src/worklet/guards.ts +3 -0
  48. package/src/worklet/messages.ts +27 -0
package/README.md CHANGED
@@ -5,27 +5,25 @@
5
5
  [![npm downloads](https://img.shields.io/npm/dm/@libraz/libsonare)](https://www.npmjs.com/package/@libraz/libsonare)
6
6
  [![types](https://img.shields.io/npm/types/@libraz/libsonare)](https://www.npmjs.com/package/@libraz/libsonare)
7
7
  [![License](https://img.shields.io/github/license/libraz/libsonare)](https://github.com/libraz/libsonare/blob/main/LICENSE)
8
+ [![Docs](https://img.shields.io/badge/docs-libsonare.libraz.net-2563eb)](https://libsonare.libraz.net)
8
9
  [![PyPI](https://img.shields.io/pypi/v/libsonare?label=PyPI)](https://pypi.org/project/libsonare/)
9
10
 
10
11
  **Turn audio into data and back — entirely in the browser.** Analyze songs
11
12
  (BPM, key, chords, loudness), master and mix to broadcast loudness, and render
12
13
  MIDI through built-in instruments, all client-side via WebAssembly — the same
13
14
  C++ engine that runs natively, with zero dependencies and no Python or model
14
- weights. 66 named mastering DSP processors implemented against published
15
+ weights. 76 named mastering DSP processors implemented against published
15
16
  references (ITU-R BS.1770-4 true-peak limiting, Linkwitz-Riley crossovers,
16
17
  Vicanek matched-Z biquads, ADAA-antialiased saturation); analysis defaults match
17
18
  librosa where the two overlap.
18
19
 
19
- > **Audio input:** This package expects already-decoded `Float32Array` mono
20
- > samples (it does not bundle a file decoder). Use the Web Audio API in the
21
- > browser or `node:wasi` / a JS audio decoder in Node to obtain samples.
22
- > If you need to read WAV/MP3/M4A files directly in Node, use the native
23
- > N-API package [`@libraz/libsonare-native`](https://github.com/libraz/libsonare/tree/main/bindings/node) instead.
20
+ ## Try it in the browser
24
21
 
25
- > **Platform constraints:** the WebAssembly build is single-threaded (analysis
26
- > runs to completion on the calling thread — there is no non-blocking variant),
27
- > has no host filesystem access, and expects pre-decoded `Float32Array` sample
28
- > buffers. Drive long-running calls from a Web Worker to keep the UI responsive.
22
+ Everything runs client-side — no server, nothing uploaded.
23
+
24
+ - 🎧 **[Live demos](https://libsonare.libraz.net/demos)** — analyze a song (BPM / key / chords), master to a target loudness, mix, and render MIDI through the built-in instruments, all in the page.
25
+ - 🎛️ **[sonare studio](https://sonare-studio.libraz.net)** — a full browser DAW (multi-track sequencing, piano roll, mixer, mastering, WAV / MP3 / MIDI / MusicXML export) built entirely on this WASM engine. It shows how far one Apache-2.0 engine reaches, from analysis to a playable, exportable arrangement.
26
+ - 📖 **[Documentation & getting started](https://libsonare.libraz.net/docs/getting-started)**
29
27
 
30
28
  ## Installation
31
29
 
@@ -33,766 +31,109 @@ librosa where the two overlap.
33
31
  npm install @libraz/libsonare
34
32
  ```
35
33
 
36
- ## Usage
37
-
38
- ```typescript
39
- import { init, detectBpm, detectKey, analyze, Audio } from '@libraz/libsonare';
40
-
41
- await init();
42
-
43
- // Function API
44
- const bpm = detectBpm(samples, sampleRate);
45
- const key = detectKey(samples, sampleRate);
46
- const result = analyze(samples, sampleRate);
47
- console.log(`BPM: ${result.bpm}, Key: ${result.key.name}`);
48
-
49
- // Advanced key options are opt-in; defaults preserve existing behavior.
50
- const keyWithOptions = detectKey(samples, sampleRate, {
51
- useHpss: true,
52
- loudnessWeighted: true,
53
- highPassHz: 80,
54
- nFft: 4096,
55
- hopLength: 512,
56
- });
57
-
58
- // Audio class API
59
- const audio = Audio.fromBuffer(samples, sampleRate);
60
- console.log(`BPM: ${audio.detectBpm()}`);
61
- console.log(`Key: ${audio.detectKey().name}`);
62
- const audioKeyWithOptions = audio.detectKey({ useHpss: true, highPassHz: 80 });
63
- ```
64
-
65
- ### Pitch, timbre, and spectral APIs
66
-
67
- Pitch tracking keeps unvoiced `f0` frames as `NaN` by default. Pass
68
- `fillNa: true` when downstream code needs finite values and should treat
69
- unvoiced frames as `0`. Timbre analysis returns aggregate metrics plus
70
- `timbreOverTime`.
71
-
72
- ```typescript
73
- import {
74
- analyzeTimbre,
75
- decompose,
76
- ebur128LoudnessRange,
77
- estimateTuning,
78
- hpssWithResidual,
79
- init,
80
- lufsInterleaved,
81
- nnFilter,
82
- phaseVocoder,
83
- pitchPyin,
84
- pitchTuning,
85
- pitchYin,
86
- polyFeatures,
87
- remix,
88
- spectralContrast,
89
- zeroCrossings,
90
- } from '@libraz/libsonare';
91
-
92
- await init();
93
-
94
- const yin = pitchYin(samples, sampleRate, 2048, 512, 65, 2093, 0.3, true);
95
- const pyin = pitchPyin(samples, sampleRate, 2048, 512, 65, 2093, 0.3, true);
96
-
97
- const timbre = analyzeTimbre(samples, sampleRate);
98
- console.log(timbre.brightness, timbre.timbreOverTime[0]?.brightness);
99
-
100
- const contrast = spectralContrast(samples, sampleRate); // Matrix2d result
101
- const poly = polyFeatures(samples, sampleRate); // Matrix2d result
102
- const crossings = zeroCrossings(samples); // Int32Array
103
- const tuning = estimateTuning(samples, sampleRate);
104
- const offset = pitchTuning(yin.f0);
105
-
106
- const { w, h } = decompose(spectrogram, nFeatures, nFrames, 8);
107
- const filtered = nnFilter(spectrogram, nFeatures, nFrames);
108
- const remixed = remix(samples, Int32Array.from([0, sampleRate, sampleRate, 2 * sampleRate]));
109
- const stretched = phaseVocoder(samples, 1.5, sampleRate);
110
- const hpss = hpssWithResidual(samples, sampleRate);
111
-
112
- const multi = lufsInterleaved(interleaved, 2, sampleRate);
113
- const lra = ebur128LoudnessRange(samples, sampleRate);
114
- ```
115
-
116
- ### Room acoustics
117
-
118
- Use `detectAcoustic` for blind RT60/EDT estimation from ordinary audio.
119
- Use `analyzeImpulseResponse` when you have a measured impulse response and need
120
- clarity metrics (`c50`, `c80`, `d50`). Blind mode returns `NaN` for clarity
121
- metrics because they are not reliable without an impulse response.
122
-
123
- ```typescript
124
- import { init, analyzeImpulseResponse, detectAcoustic } from '@libraz/libsonare';
125
-
126
- await init();
127
-
128
- // Blind estimation from ordinary audio; tuning options (nOctaveBands, …) are optional.
129
- const blind = detectAcoustic(samples, sampleRate);
130
- const room = analyzeImpulseResponse(irSamples, sampleRate);
131
- console.log(blind.rt60, room.c50);
132
- ```
133
-
134
- Acoustic simulation adds `synthesizeRir` (synthesize a shoebox-room impulse
135
- response from geometry), `estimateRoom` (recover an equivalent room from a
136
- recording or IR), and `roomMorph` (creatively re-reverberate audio toward a
137
- target room — not dereverberation).
138
-
139
- ```typescript
140
- import { init, synthesizeRir, estimateRoom, roomMorph } from '@libraz/libsonare';
141
-
142
- await init();
143
-
144
- const { rir, hasError } = synthesizeRir({
145
- lengthM: 6,
146
- widthM: 4,
147
- heightM: 3,
148
- absorption: 0.2,
149
- sampleRate: 48000,
150
- });
151
-
152
- const room = estimateRoom(samples, 48000); // { volume, length, width, height, ... }
153
- const morphed = roomMorph(samples, sampleRate, { lengthM: 12, widthM: 9, wet: 0.5 });
154
- ```
155
-
156
- ### Error handling
157
-
158
- Native (C++) failures are thrown as a `SonareError` carrying a numeric `code`
159
- (an `ErrorCode` value) and its canonical `codeName`, so you can branch on the
160
- cause instead of matching message text. Use the `isSonareError` type guard in a
161
- `catch`:
162
-
163
- ```typescript
164
- import { init, analyze, ErrorCode, isSonareError } from '@libraz/libsonare';
165
-
166
- await init();
167
-
168
- try {
169
- const result = analyze(samples, sampleRate);
170
- } catch (error) {
171
- if (isSonareError(error)) {
172
- console.error(`${error.codeName} (${error.code}): ${error.message}`);
173
- if (error.code === ErrorCode.InvalidParameter) {
174
- // recover...
175
- }
176
- } else {
177
- throw error;
178
- }
179
- }
180
- ```
181
-
182
- ### Decoding files in the browser
183
-
184
- ```typescript
185
- import { init, analyze } from '@libraz/libsonare';
186
-
187
- await init();
188
-
189
- const arrayBuffer = await fetch('song.m4a').then((r) => r.arrayBuffer());
190
- const audioCtx = new AudioContext();
191
- const decoded = await audioCtx.decodeAudioData(arrayBuffer);
192
- // Mono downmix for libsonare:
193
- const samples = decoded.getChannelData(0);
194
- const result = analyze(samples, decoded.sampleRate);
195
- ```
196
-
197
- Web Audio's `decodeAudioData` handles whatever codecs the browser ships with
198
- (WAV/MP3/M4A/AAC/Opus/FLAC on most modern browsers).
199
-
200
- ### Browser (CDN)
201
-
202
- ```html
203
- <script type="module">
204
- import { init, analyze } from 'https://esm.sh/@libraz/libsonare';
205
-
206
- await init();
207
- // ... use with Web Audio API
208
- </script>
209
- ```
210
-
211
- ### Bundlers (Vite, webpack, Next.js, etc.)
212
-
213
- If your bundler doesn't automatically resolve the `.wasm` file, specify its path:
214
-
215
- ```typescript
216
- import wasmUrl from '@libraz/libsonare/wasm?url'; // Vite
217
- import { init } from '@libraz/libsonare';
218
-
219
- // `locateFile(path, prefix)` is called by the Emscripten loader to resolve the
220
- // `.wasm` file; return your bundler-provided URL for it.
221
- await init({ locateFile: (path) => (path.endsWith('.wasm') ? wasmUrl : path) });
222
- ```
223
-
224
- ### Real-time Streaming
225
-
226
- ```typescript
227
- import { init, StreamAnalyzer } from '@libraz/libsonare';
228
-
229
- await init();
230
-
231
- const analyzer = new StreamAnalyzer({ sampleRate: 44100 });
232
-
233
- // In audio processing callback
234
- analyzer.process(audioChunk);
235
-
236
- const stats = analyzer.stats();
237
- console.log(`BPM: ${stats.estimate.bpm}, Key: ${stats.estimate.key}`);
238
- ```
34
+ ## Quick Start
239
35
 
240
- ### Mastering (WASM)
36
+ `init()` loads the WASM module once; every API is available afterwards. Top-level
37
+ one-shot functions accept a request object (recommended) or positional arguments.
241
38
 
242
- The npm package ships mastering DSP in the default WebAssembly build. Pass
243
- decoded `Float32Array` samples directly:
39
+ > **Audio input:** analysis works on decoded `Float32Array` mono samples.
40
+ > `Audio.fromMemory` decodes WAV from an in-memory buffer, but the WebAssembly
41
+ > build bundles no decoder for compressed formats — use the Web Audio API
42
+ > (`decodeAudioData`) for MP3 / M4A / AAC / Opus / FLAC, or the native N-API
43
+ > package [`@libraz/libsonare-native`](https://github.com/libraz/libsonare/tree/main/bindings/node)
44
+ > to read files from disk.
244
45
 
245
- ```typescript
246
- import { init, masteringChain, masteringChainStereo } from '@libraz/libsonare';
247
-
248
- await init();
249
-
250
- // Config is a tree of processor sections; set only what you want to change.
251
- const mastered = masteringChain(samples, sampleRate, {
252
- dynamics: { compressor: { thresholdDb: -24, ratio: 1.5 } },
253
- loudness: { targetLufs: -14, ceilingDb: -1 },
254
- });
255
-
256
- const stereo = masteringChainStereo(left, right, sampleRate, {
257
- stereo: { imager: { width: 1.1 } },
258
- loudness: { targetLufs: -14, ceilingDb: -1 },
259
- });
260
- ```
261
-
262
- Named mastering processors use the same names and behavior as the native,
263
- Python, C, and CLI APIs:
264
-
265
- ```typescript
266
- import {
267
- masteringPairAnalyze,
268
- masteringPairProcess,
269
- masteringPairProcessorNames,
270
- masteringProcess,
271
- masteringProcessStereo,
272
- masteringProcessorNames,
273
- masteringStereoAnalyze,
274
- } from '@libraz/libsonare';
275
-
276
- const names = masteringProcessorNames(); // e.g. "dynamics.compressor"
277
- const compressed = masteringProcess('dynamics.compressor', samples, sampleRate, {
278
- thresholdDb: -24,
279
- ratio: 1.5,
280
- });
281
-
282
- const widened = masteringProcessStereo('stereo.imager', left, right, sampleRate, {
283
- width: 1.1,
284
- });
285
-
286
- const pairNames = masteringPairProcessorNames(); // e.g. "match.abCrossfade"
287
- const crossfaded = masteringPairProcess('match.abCrossfade', source, reference, sampleRate, {
288
- mix: 0.25,
289
- });
290
-
291
- const loudnessJson = masteringPairAnalyze(
292
- 'match.referenceLoudness',
293
- source,
294
- reference,
295
- sampleRate,
296
- );
297
- const monoCompatJson = masteringStereoAnalyze(
298
- 'stereo.monoCompatCheck',
299
- left,
300
- right,
301
- sampleRate,
302
- );
303
- ```
304
-
305
- ### Mastering presets
46
+ > **Platform constraints:** the WebAssembly build is single-threaded (analysis
47
+ > runs to completion on the calling thread — there is no non-blocking variant)
48
+ > and has no host filesystem access. Drive long-running calls from a Web Worker
49
+ > to keep the UI responsive.
306
50
 
307
51
  ```typescript
308
- import { init, masterAudio, masteringPresetNames } from '@libraz/libsonare';
52
+ import { init, analyze, masterAudio } from '@libraz/libsonare';
309
53
 
310
54
  await init();
311
55
 
312
- masteringPresetNames(); // ['pop', 'edm', 'acoustic', 'hipHop', 'aiMusic', 'speech', 'streaming', 'youtube', 'broadcast', 'podcast', 'audiobook', 'cinema', 'jpop', 'ambient', 'lofi', 'classical', 'drumAndBass', 'techno', 'metal', 'trap', 'rnb', 'jazz', 'kpop', 'trance', 'gameOst']
56
+ // Analyze decoded Float32Array mono samples
57
+ const { bpm, key } = analyze({ samples, sampleRate });
58
+ console.log(`BPM: ${bpm} Key: ${key.name}`);
313
59
 
314
- const result = masterAudio(samples, sampleRate, 'aiMusic', {
315
- // optional flat overrides applied on top of the preset (dot notation)
316
- 'loudness.targetLufs': -13,
317
- });
318
- console.log(result.outputLufs, result.appliedGainDb);
60
+ // Master toward a target loudness with a named preset
61
+ const mastered = masterAudio({ samples, sampleRate, preset: 'streaming' });
62
+ console.log(mastered.outputLufs, mastered.appliedGainDb);
319
63
  ```
320
64
 
321
- ### Mixing
322
-
323
- ```typescript
324
- import { init, Mixer, mixStereo, mixingScenePresetJson } from '@libraz/libsonare';
325
-
326
- await init();
327
-
328
- const sceneJson = mixingScenePresetJson('vocalReverbSend');
329
- const offline = mixStereo([vocalL, musicL], [vocalR, musicR], sampleRate, {
330
- inputTrimDb: [3, 0],
331
- faderDb: [-3, -12],
332
- pan: [0, -0.2],
333
- width: [1, 0.9],
334
- });
335
-
336
- const mixer = Mixer.fromSceneJson(sceneJson, sampleRate, 512);
337
- const block = mixer.processStereo([vocalBlockL, returnBlockL], [vocalBlockR, returnBlockR]);
338
- console.log(offline.meters[0].maxTruePeakDb, block.left.length);
339
-
340
- const outL = new Float32Array(512);
341
- const outR = new Float32Array(512);
342
- mixer.processStereoInto([vocalBlockL, returnBlockL], [vocalBlockR, returnBlockR], outL, outR);
343
-
344
- const realtime = mixer.createRealtimeBuffer();
345
- realtime.leftInputs[0].set(vocalBlockL);
346
- realtime.rightInputs[0].set(vocalBlockR);
347
- realtime.leftInputs[1].set(returnBlockL);
348
- realtime.rightInputs[1].set(returnBlockR);
349
- realtime.process();
350
- console.log(realtime.outLeft[0], realtime.outRight[0]);
351
- mixer.delete();
352
- ```
353
-
354
- ### Headless DAW project
355
-
356
- `Project` is a headless arrangement model: audio & MIDI tracks and clips, MIDI
357
- sequencing, SMF / MIDI 2.0 Clip File I/O, deterministic JSON save/load, and an
358
- offline `bounce`. Every mutation routes through an undoable history, and musical
359
- positions are PPQ (quarter notes). Call `delete()` (or wrap in `try/finally`) to
360
- release the WASM object — the embind handle is not garbage-collected.
65
+ Render a MIDI arrangement through a built-in instrument with the headless
66
+ `Project`. The embind handle is not garbage-collected — call `delete()` when done.
361
67
 
362
68
  ```typescript
363
69
  import { init, Project } from '@libraz/libsonare';
364
70
 
365
71
  await init();
366
72
 
367
- const project = new Project();
368
- try {
369
- project.setSampleRate(48000);
370
-
371
- const { clipId } = project.addMidiClip(0, 4); // { trackId, clipId }
372
- project.setMidiEvents(clipId, [
373
- Project.midiNoteOn(0, 0, 0, 60, 100), // ppq, group, channel, note, velocity
374
- Project.midiNoteOff(1, 0, 0, 60),
375
- ]);
376
-
377
- const json = project.toJson(); // deterministic, byte-stable within a build
378
- const smf = project.exportSmf(); // Uint8Array — Standard MIDI File
379
- const midi2 = project.exportClipFile(); // Uint8Array — MIDI 2.0 Clip File (lossless)
380
-
381
- const { hasTimeline, diagnostics } = project.compile();
382
- const audio = project.bounce({ numChannels: 2 }); // interleaved Float32Array
383
- } finally {
384
- project.delete();
385
- }
386
- ```
387
-
388
- #### Clip warp
389
-
390
- A clip can be time-warped during an offline `bounce`. `setClipWarpMode` selects
391
- the playback mode (`ProjectWarpMode`: `'off'` | `'repitch'` | `'tempo-sync'`),
392
- `setClipWarpRef` binds it to a warp map, and `setWarpMap` registers a first-class
393
- warp map (anchors mapping warp-timeline samples to source samples).
394
-
395
- ```typescript
396
- project.setWarpMap({
397
- id: 1,
398
- name: 'main',
399
- anchors: [
400
- { warpSample: 0, sourceSample: 0 },
401
- { warpSample: 48000, sourceSample: 24000 },
402
- ],
403
- });
404
- project.setClipWarpRef(clipId, 1);
405
- project.setClipWarpMode(clipId, 'tempo-sync');
406
- ```
407
-
408
- > Warp is an offline `Project.bounce` feature only. Realtime warp playback is
409
- > **not** available in `RealtimeEngine`.
410
-
411
- ### Instruments and synthesis
412
-
413
- MIDI tracks bounce silently unless an instrument is bound. `Project` offers
414
- three instrument backends, each as a `bounceWith…` variant that takes a binding
415
- (or array of bindings) plus the usual `ProjectBounceOptions`:
416
-
417
- - `bounceWithBuiltinInstrument(binding?, options?)` — simple built-in oscillator
418
- synth (`BuiltinSynthConfig` / `BuiltinSynthBinding`: waveform + ADSR + gain).
419
- - `bounceWithSynthInstrument(patchOrName?, options?)` — patch-driven NativeSynth
420
- (`SynthPatch`, or a preset-name string like `'saw-lead'`).
421
- - `bounceWithSf2Instrument(config?, options?)` — GS-compatible SoundFont player
422
- (`Sf2InstrumentConfig`), fed by `loadSoundFont()`.
423
-
424
- Discover NativeSynth presets with `synthPresetNames()` and fetch one as an
425
- editable patch with `synthPresetPatch(name)`.
426
-
427
- ```typescript
428
- import { init, Project, synthPresetNames, synthPresetPatch } from '@libraz/libsonare';
429
-
430
- await init();
431
-
432
73
  const project = new Project();
433
74
  try {
434
75
  const { clipId } = project.addMidiClip(0, 4);
435
76
  project.setMidiEvents(clipId, [
436
- Project.midiNoteOn(0, 0, 0, 60, 100),
77
+ Project.midiNoteOn(0, 0, 0, 60, 100), // ppq, group, channel, note, velocity
437
78
  Project.midiNoteOff(1, 0, 0, 60),
438
79
  ]);
439
-
440
- // Built-in oscillator synth.
441
- const a = project.bounceWithBuiltinInstrument({ waveform: 'saw' }, { numChannels: 2 });
442
-
443
- // NativeSynth from a named preset, tweaked.
444
- const patch = synthPresetPatch(synthPresetNames()[0]);
445
- patch.cutoffHz = 4000;
446
- const b = project.bounceWithSynthInstrument(patch, { numChannels: 2 });
447
-
448
- // SoundFont player (requires loadSoundFont first).
449
- project.loadSoundFont(sf2Bytes);
450
- const c = project.bounceWithSf2Instrument({ gain: 0.6 }, { numChannels: 2 });
80
+ const audio = project.bounceWithSynthInstrument('saw-lead', { numChannels: 2 });
451
81
  } finally {
452
82
  project.delete();
453
83
  }
454
84
  ```
455
85
 
456
- ### Real-time engine
457
-
458
- `RealtimeEngine` is a control-thread-driven transport + render engine: it plays
459
- a clip/automation timeline, hosts MIDI instruments, accepts live MIDI, and
460
- renders blocks (or bounces offline). Bind it to an AudioWorklet for browser
461
- playback — see the [AudioWorklet bridge](#audioworklet-bridge) below; the engine
462
- is the offline/headless half, the worklet is the audio-thread half.
463
-
464
- ```typescript
465
- import { init, RealtimeEngine } from '@libraz/libsonare';
466
-
467
- await init();
468
-
469
- const engine = new RealtimeEngine(48000, 128); // sampleRate, maxBlockSize
470
- try {
471
- engine.setSynthInstrument('saw-lead', 0); // patch (or name), destinationId
472
- engine.play();
473
- engine.pushMidiNoteOn(0, 0, 0, 60, 100); // destination, group, channel, note, velocity
474
- engine.pushMidiNoteOff(0, 0, 0, 60);
475
-
476
- const blockL = new Float32Array(128);
477
- const blockR = new Float32Array(128);
478
- const out = engine.process([blockL, blockR]); // Float32Array[] per channel
479
- const telemetry = engine.drainTelemetry();
480
- } finally {
481
- engine.destroy();
482
- }
483
- ```
484
-
485
- Capabilities:
486
-
487
- - **Transport**: `play` / `stop` / `seekSample` / `seekPpq` / `setTempo` /
488
- `setTimeSignature` / `setLoop`, plus `getTransportState`.
489
- - **Instruments**: `setBuiltinInstrument` / `setSynthInstrument` /
490
- `setSf2Instrument` (+ `loadSoundFont`) per MIDI destination id.
491
- - **Live MIDI**: `pushMidiNoteOn` / `pushMidiNoteOff` / `pushMidiCc` /
492
- `pushMidiPanic`, and `bindMidiCc(channel, controller, paramId, options?)` to
493
- map a CC to an automation parameter.
494
- - **Process / bounce**: `process` (real-time blocks), the allocation-free
495
- `prepareChannels` + `getChannelBuffer` + `processPrepared` worklet path,
496
- `renderOffline`, `bounceOffline`, `freezeOffline`.
497
- - **Clip page providers**: `createClipPageProvider` + `supplyClipPage` for
498
- streaming large clip audio in pages; pair with the OPFS helpers
499
- (`createOpfsClipPageProvider`) and the bounded-memory `ClipPageStreamer`
500
- (see below) so a long multitrack arrangement never holds its full PCM in
501
- WASM memory.
502
- - **Telemetry**: `drainTelemetry` / `drainMeterTelemetry`. Inspect runtime
503
- capabilities (ABI compatibility, SharedArrayBuffer/Atomics) via
504
- `engineCapabilities()`.
505
-
506
- #### Bounded-memory clip streaming
507
-
508
- `ClipPageStreamer` keeps OPFS-paged clips fed within a sliding window around the
509
- playback position, evicting pages that fall outside it. Resident memory per clip
510
- is bounded to `retainBehindPages + readAheadPages + 1` pages regardless of clip
511
- length. `attachOpfsClipStream` wires the provider, primes the leading page, and
512
- registers it in one call; `setClips` schedules the returned `provider`, and
513
- `pump()` (called on your control-thread tick) services page misses.
514
-
515
- ```typescript
516
- import { ClipPageStreamer, attachOpfsClipStream, RealtimeEngine } from '@libraz/libsonare';
517
-
518
- const engine = new RealtimeEngine(48000, 128);
519
- const streamer = new ClipPageStreamer(engine, { readAheadPages: 2, retainBehindPages: 1 });
520
-
521
- const { provider } = await attachOpfsClipStream(streamer, engine, {
522
- path: 'clips/vocal.f32',
523
- clipId: 1,
524
- numChannels: 2,
525
- numSamples: 26_460_000, // ~10 min at 44.1 kHz, never fully resident
526
- pageFrames: 65_536,
527
- });
528
- engine.setClips([{ id: 1, pageProvider: provider, startPpq: 0 }]);
529
- engine.play();
530
-
531
- // On each animation frame / control tick:
532
- await streamer.pump(); // fetch upcoming pages, evict old ones
533
- ```
534
-
535
- ### Real-time voice changer
536
-
537
- `RealtimeVoiceChanger` runs a block-by-block voice transformation chain (retune,
538
- formant shaping, EQ, gate, compressor). Construct it from a preset id (see
539
- `realtimeVoiceChangerPresetNames()`) or a full config object, then process blocks.
540
-
541
- ```typescript
542
- import { init, RealtimeVoiceChanger, voiceChangeRealtime } from '@libraz/libsonare';
543
-
544
- await init();
545
-
546
- const changer = new RealtimeVoiceChanger('bright-idol');
547
- try {
548
- changer.prepare(48000, 128, 1); // sampleRate, maxBlockSize, channels
549
- const out = changer.processMono(block);
550
- } finally {
551
- changer.delete();
552
- }
553
-
554
- // Whole-buffer convenience wrapper (constructs/prepares/disposes internally).
555
- const processed = voiceChangeRealtime(samples, 48000, 'deep-narrator');
556
- ```
557
-
558
- For a simple offline pitch + formant shift without the full chain, use
559
- `voiceChange(samples, sampleRate, { pitchSemitones: -2, formantFactor: 1.1 })`.
560
- Inspect a preset with `realtimeVoiceChangerPresetJson(name)`.
86
+ ### Decoding audio in the browser
561
87
 
562
- ### AudioWorklet bridge
563
-
564
- The package exposes an optional worklet entry that uses the same `sonare.wasm`
565
- as the offline API. The bridge processes fixed 128-sample render quanta and
566
- treats each AudioWorklet input as one stereo mixer strip.
88
+ The WASM build takes decoded samples, so decode with the Web Audio API first:
567
89
 
568
90
  ```typescript
569
- // worklet.ts, loaded with audioContext.audioWorklet.addModule(...)
570
- import { init, mixingScenePresetJson } from '@libraz/libsonare';
571
- import { registerSonareWorkletProcessor } from '@libraz/libsonare/worklet';
572
-
573
- await init();
574
- registerSonareWorkletProcessor();
91
+ const buf = await fetch('song.m4a').then((r) => r.arrayBuffer());
92
+ const decoded = await new AudioContext().decodeAudioData(buf);
93
+ const { bpm, key } = analyze({ samples: decoded.getChannelData(0), sampleRate: decoded.sampleRate });
575
94
  ```
576
95
 
577
- ```typescript
578
- // main thread
579
- import { mixingScenePresetJson } from '@libraz/libsonare';
580
-
581
- await audioContext.audioWorklet.addModule('/worklet.js');
582
- const sceneJson = mixingScenePresetJson('vocalReverbSend');
583
- const node = new AudioWorkletNode(audioContext, 'sonare-worklet-processor', {
584
- numberOfInputs: 2,
585
- numberOfOutputs: 1,
586
- outputChannelCount: [2],
587
- processorOptions: {
588
- sceneJson,
589
- sampleRate: audioContext.sampleRate,
590
- blockSize: 128,
591
- spectrumIntervalFrames: 2048,
592
- spectrumBands: 16,
593
- },
594
- });
595
-
596
- node.port.postMessage({
597
- type: 'scheduleInsertAutomation',
598
- stripIndex: 0,
599
- insertIndex: 0,
600
- paramId: 0,
601
- samplePos: 0,
602
- value: 0,
603
- curve: 'linear',
604
- });
605
-
606
- node.port.onmessage = (event) => {
607
- if (event.data?.type === 'meter') {
608
- console.log(event.data.peakDbL, event.data.rmsDbL, event.data.correlation);
609
- } else if (event.data?.type === 'spectrum') {
610
- console.log(event.data.frame, event.data.bands);
611
- }
612
- };
613
- ```
614
-
615
- For cross-origin-isolated pages, meters and spectrum snapshots can use optional
616
- SharedArrayBuffer rings instead of per-message `postMessage`:
617
-
618
- ```typescript
619
- import {
620
- createSonareMeterRingBuffer,
621
- createSonareSpectrumRingBuffer,
622
- readSonareMeterRingBuffer,
623
- readSonareSpectrumRingBuffer,
624
- } from '@libraz/libsonare/worklet';
625
-
626
- const meterRing = createSonareMeterRingBuffer(128);
627
- const spectrumRing = createSonareSpectrumRingBuffer(64, 16);
628
- const node = new AudioWorkletNode(audioContext, 'sonare-worklet-processor', {
629
- numberOfInputs: 2,
630
- numberOfOutputs: 1,
631
- outputChannelCount: [2],
632
- processorOptions: {
633
- sceneJson,
634
- sampleRate: audioContext.sampleRate,
635
- blockSize: 128,
636
- meterSharedBuffer: meterRing.sharedBuffer,
637
- spectrumIntervalFrames: 2048,
638
- spectrumSharedBuffer: spectrumRing.sharedBuffer,
639
- },
640
- });
641
-
642
- let nextMeterRead = 0;
643
- let nextSpectrumRead = 0;
644
- function readMeters() {
645
- const result = readSonareMeterRingBuffer(meterRing, nextMeterRead);
646
- nextMeterRead = result.nextReadIndex;
647
- for (const meter of result.meters) {
648
- console.log(meter.frame, meter.peakDbL, meter.rmsDbL);
649
- }
650
- const spectra = readSonareSpectrumRingBuffer(spectrumRing, nextSpectrumRead);
651
- nextSpectrumRead = spectra.nextReadIndex;
652
- for (const spectrum of spectra.spectra) {
653
- console.log(spectrum.frame, spectrum.bands);
654
- }
655
- }
656
- ```
657
-
658
- #### RealtimeEngine AudioWorklet facade
659
-
660
- For browser DAW-style playback, `SonareEngine` keeps a main-thread
661
- `RealtimeEngine` mirror for offline renders and synchronizes the live worklet
662
- engine through control messages. There is a single runtime: the full-featured
663
- embind engine runs both the main-thread mirror and the AudioWorklet.
664
-
665
- ```typescript
666
- import { init } from '@libraz/libsonare';
667
- import { init as initWorklet, SonareEngine } from '@libraz/libsonare/worklet';
668
-
669
- await init();
670
- await audioContext.audioWorklet.addModule('/worklet.js');
671
- await initWorklet();
672
-
673
- const engine = await SonareEngine.create(audioContext, { mode: 'sab' });
674
- engine.setTrackLanes([{ trackId: 1 }]);
675
- engine.setTrackStripJson(1, trackSceneJson);
676
- engine.addClip(1, [clipL, clipR], 0);
677
- engine.setTempoSegments([{ startPpq: 0, bpm: 120 }]);
678
- engine.setTimeSignatureSegments([{ startPpq: 0, numerator: 4, denominator: 4 }]);
679
- engine.transport.play();
680
- ```
681
-
682
- | Need | Facade/API |
683
- |---|---|
684
- | Track routing, fader, pan, solo/mute | `setTrackLanes`, `setStripGain`, `setStripPan`, `setSoloMute` |
685
- | Track/master inserts and EQ | `setTrackStripJson`, `setMasterStripJson`, `setTrackStripEqBand`, `setMasterStripEqBand`, insert bypass methods |
686
- | Sends and buses | `setSends`, `setBusGain`, `setBusStripJson` |
687
- | MIDI clips and live MIDI | `setMidiClips`, `pushMidiNoteOn`, `pushMidiNoteOff`, `pushMidiCc`, `pushMidiPanic` |
688
- | Instruments | `setBuiltinInstrument`, `setSynthInstrument`, `loadSoundFont`, `setSf2Instrument` |
689
- | Recording and monitoring | `configureCapture` (incl. the `inputMonitor` option), `armRecord`, `punch`, `capturedAudio`, `captureStatus` |
690
- | Transport, tempo, markers | `getTransportState`, `cachedTransportState`, `setTempoSegments`, `setTimeSignatureSegments`, marker methods, `setLoopFromMarkers` |
691
- | Clip updates | `addClip`, `removeClip`; the facade sends clip deltas while the processor keeps full-sync compatibility |
692
- | Meters | `onMeter` messages or the meter SharedArrayBuffer ring; records include master, lane, bus, and input target ids |
693
-
694
- Studio integration notes:
695
-
696
- - If a host worklet entry filters message names before forwarding, add every
697
- `sync*`, `captureRequest`, and `transportRequest` message used above to that
698
- allowlist; otherwise the host may silently drop the new control messages.
699
- - If a host vendors built worklet bundles, regenerate and reimport
700
- `worklet.js`, `worklet.d.ts`, `sonare.js`, and `sonare.wasm` together.
701
-
702
- ### Progress callback
96
+ ### Loading the `.wasm` file
703
97
 
704
- `masteringChainWithProgress` (and its stereo variant) is `masteringChain` with
705
- an extra `(progress, stage) => void` callback invoked after each enabled stage:
98
+ Bundlers that don't auto-resolve the `.wasm` asset need its URL. Pass a
99
+ `locateFile` resolver to `init()`:
706
100
 
707
101
  ```typescript
708
- import { init, masteringChainWithProgress } from '@libraz/libsonare';
102
+ import wasmUrl from '@libraz/libsonare/wasm?url'; // Vite; adapt per bundler
709
103
 
710
- await init();
711
-
712
- masteringChainWithProgress(
713
- samples,
714
- sampleRate,
715
- { dynamics: { compressor: { thresholdDb: -24 } } },
716
- (progress, stage) => console.log(`${stage}: ${(progress * 100).toFixed(0)}%`),
717
- );
718
- ```
719
-
720
- ### Streaming mastering chain
721
-
722
- `StreamingMasteringChain` processes blocks while maintaining per-stage state
723
- across calls. It only supports modules whose state depends solely on the
724
- sample rate — it cannot include `repair.denoise` or `loudness` (those require
725
- offline / look-ahead analysis) and throws at construction if they are enabled.
726
-
727
- ```typescript
728
- import { init, StreamingMasteringChain } from '@libraz/libsonare';
729
-
730
- await init();
731
-
732
- const chain = new StreamingMasteringChain({
733
- eq: { tiltDb: 0.5 },
734
- dynamics: { compressor: { thresholdDb: -20 } },
735
- });
736
- chain.prepare(48000, 512, 2);
737
-
738
- // Mono block
739
- const monoBlock = new Float32Array(512);
740
- const processedMono = chain.processMono(monoBlock);
741
-
742
- // Stereo block (separate L/R Float32Arrays)
743
- const left = new Float32Array(512);
744
- const right = new Float32Array(512);
745
- const { left: outL, right: outR } = chain.processStereo(left, right);
746
-
747
- chain.reset();
748
- chain.delete(); // release WASM memory
104
+ await init({ locateFile: (path) => (path.endsWith('.wasm') ? wasmUrl : path) });
749
105
  ```
750
106
 
751
- ### Streaming equalizer and retune
107
+ From a CDN, `import { init } from 'https://esm.sh/@libraz/libsonare'` resolves the
108
+ `.wasm` automatically. See the
109
+ [getting-started guide](https://libsonare.libraz.net/docs/getting-started) for
110
+ per-bundler setup and the AudioWorklet bridge.
752
111
 
753
- `StreamingEqualizer` wraps the unified `EqualizerProcessor` (up to 24 bands,
754
- RBJ/Vicanek biquads, dynamic EQ, linear-phase FIR, mid/side, auto-gain) with
755
- state maintained across calls. `StreamingRetune` is a block-by-block mono voice
756
- retune / pitch shifter.
112
+ ## Capabilities
757
113
 
758
- ```typescript
759
- import { init, StreamingEqualizer, StreamingRetune } from '@libraz/libsonare';
114
+ Every area below has runnable examples and the full API in the
115
+ [documentation](https://libsonare.libraz.net/docs/wasm).
760
116
 
761
- await init();
117
+ - **Analysis** — BPM, key (+ candidates), chords, downbeats, sections, melody, tuning; pitch (YIN / pYIN), timbre, and the full spectral feature set (STFT, mel, MFCC, chroma, CQT/VQT, spectral contrast); metering (true-peak, LUFS, correlation, vectorscope, waveform peaks). → [API](https://libsonare.libraz.net/docs/wasm)
118
+ - **Mastering** — 76 named DSP processors, the configurable `masteringChain`, 25 named presets via `masterAudio`, and reference-matching. → [Mastering processors](https://libsonare.libraz.net/docs/mastering-processors)
119
+ - **Mixing** — offline `mixStereo` and the block-based `Mixer` with scene presets. → [Mixing](https://libsonare.libraz.net/docs/mixing)
120
+ - **Editing DSP** — time-stretch, pitch-shift, HPSS (+ residual), phase vocoder, normalize, trim, remix. → [Editing DSP](https://libsonare.libraz.net/docs/editing-dsp)
121
+ - **Room acoustics** — blind RT60 / EDT, impulse-response clarity metrics, RIR synthesis, room estimation and morphing. → [Room acoustics](https://libsonare.libraz.net/docs/acoustic-analysis)
122
+ - **Realtime & streaming** — `RealtimeEngine` (transport / MIDI / render, bounded-memory clip streaming), `StreamingMasteringChain` / `StreamingEqualizer` / `StreamingRetune`, `RealtimeVoiceChanger`, and the AudioWorklet bridge. → [Realtime & streaming](https://libsonare.libraz.net/docs/realtime-streaming)
123
+ - **Instruments & synthesis** — built-in oscillator synth, patch-driven NativeSynth (15 synthesis engines, incl. physically-modeled piano / strings / winds — being tuned over time), and a GS-compatible SoundFont (SF2) player. → [API](https://libsonare.libraz.net/docs/wasm)
124
+ - **Headless DAW** — `Project` arrangement model: audio / MIDI tracks and clips, undo/redo, clip warp, SMF / MIDI 2.0 Clip File I/O, deterministic JSON, offline `bounce`. → [API](https://libsonare.libraz.net/docs/wasm)
125
+ - **Conversions** — Hz / mel / MIDI / note, frames / time, resample.
762
126
 
763
- const eq = new StreamingEqualizer({ sampleRate: 48000, maxBlockSize: 512 });
764
- try {
765
- eq.setBand(0, { type: 'HighShelf', frequencyHz: 8000, gainDb: 6, enabled: true });
766
- const { left: eqL, right: eqR } = eq.processStereo(left, right);
767
- } finally {
768
- eq.delete();
769
- }
127
+ Native failures throw a `SonareError` carrying a numeric `code` (an `ErrorCode`
128
+ value) and its `codeName`; narrow with the `isSonareError` type guard.
770
129
 
771
- const retune = new StreamingRetune({ semitones: 2, mix: 1.0 });
772
- try {
773
- retune.prepare(48000, 512); // sampleRate, maxBlockSize
774
- const shifted = retune.processMono(monoBlock);
775
- } finally {
776
- retune.delete();
777
- }
778
- ```
130
+ ## Documentation
779
131
 
780
- ## Features
781
-
782
- - **Detection**: `detectBeats`, `detectOnsets`, `detectDownbeats`, `detectChords`, `detectKey`, `detectKeyCandidates`, `chordFunctionalAnalysis`, sections
783
- - **Analysis**: `analyze`, `analyzeWithProgress`, `analyzeBpm`, `analyzeRhythm`, `analyzeDynamics`, `analyzeTimbre`; `hasFfmpegSupport` capability check
784
- - **Effects**: HPSS, HPSS with residual, time stretch, phase vocoder, pitch shift, normalize, trim, remix
785
- - **Mastering**: EQ, compressor, tape/exciter, air band, stereo imaging,
786
- true-peak limiting, loudness optimization
787
- - **Features**: STFT, mel spectrogram, MFCC, chroma, CQT/VQT, spectral contrast, poly features, zero crossings
788
- - **Pitch**: YIN, pYIN algorithms with optional `fillNa`
789
- - **Decomposition & loudness**: NMF decomposition, nearest-neighbour filtering, multichannel LUFS, EBU R128 LRA
790
- - **Streaming**: Real-time analysis with progressive estimates; streaming mastering chain, equalizer, and retune
791
- - **Instruments**: built-in synth, patch-driven NativeSynth (12 synthesis engines, incl. physically-modeled piano/strings/winds — being tuned over time), SoundFont (SF2) player — bound to `Project` bounces or the `RealtimeEngine`
792
- - **Real-time**: `RealtimeEngine` transport/MIDI/render, `RealtimeVoiceChanger`, AudioWorklet bridge
793
- - **Room acoustics**: blind RT60/EDT, impulse-response clarity metrics, RIR synthesis, room estimation, room morphing
794
- - **Headless DAW**: `Project` arrangement model — audio/MIDI tracks & clips, undo/redo, MIDI sequencing, clip warp, SMF / MIDI 2.0 Clip File I/O, deterministic JSON, offline `bounce`
795
- - **Conversions**: Hz/mel/MIDI/note, frames/time, resample
132
+ Full API reference, guides, and browser-local demos live at
133
+ **[libsonare.libraz.net](https://libsonare.libraz.net)**
134
+ ([getting started](https://libsonare.libraz.net/docs/getting-started) ·
135
+ [browser / WASM API](https://libsonare.libraz.net/docs/wasm) ·
136
+ [demos](https://libsonare.libraz.net/demos)).
796
137
 
797
138
  ## Also available
798
139
 
@@ -800,6 +141,9 @@ try {
800
141
  pip install libsonare # Python bindings with CLI
801
142
  ```
802
143
 
144
+ The native Node.js N-API binding (reads files from disk) lives at
145
+ [`bindings/node`](https://github.com/libraz/libsonare/tree/main/bindings/node).
146
+
803
147
  ## License
804
148
 
805
149
  [Apache License 2.0](https://github.com/libraz/libsonare/blob/main/LICENSE)