node-web-audio-api 0.21.0 → 0.21.2

package/CHANGELOG.md

@@ -1,6 +1,20 @@
+# CHANGELOG
+
+## v0.21.2 (20/09/2024)
+
+- - Update upstream crate to [v1.0.1](https://github.com/orottier/web-audio-api-rs/blob/main/CHANGELOG.md#version-101-2024-09-18)
+- Fix: Make sure `AudioBuffer` returned by `OfflineContext` is valid
+- Fix: Allow contexts to be properly garbage collected
+
+## v0.21.1 (10/06/2024)
+
+- Feat: Buffer pool for AudioWorketProcessor
+- Fix: Propagate `addModule` errors to main thread
+- Fix: Memory leak due to `onended` events
+
 ## v0.21.0 (17/05/2024)
 
-- Feat: implement AudioWorkletNode
+- Feat: Implement AudioWorkletNode
 
 ## v0.20.0 (29/04/2024)
 
@@ -86,7 +100,7 @@
 ## v0.5.0 (19/12/2022)
 
 - Implement AudioParam#setValueCurveAtTime
-- Offline context constructor 
+- Offline context constructor
 
 ## v0.4.0 (07/11/2022)
 

package/README.md

@@ -65,7 +65,7 @@ node examples/granular-scrub.mjs
 
 ## Caveats
 
-- `Streams`: only a minimial audio input stream and the `MediaStreamSourceNode` are provided. All other `MediaStream` features are left on the side for now as they principaly concern a different API specification, which is not a trivial problem.
+- `Streams`: only a minimal audio input stream and the `MediaStreamSourceNode` are provided. All other `MediaStream` features are left on the side for now as they principally concern a different API specification, which is not a trivial problem.
 
 ## Supported Platforms
 
@@ -79,24 +79,7 @@ node examples/granular-scrub.mjs
 | Linux arm gnueabihf (RPi)    | ✓        | ✓      |
 | Linux arm64 gnu (RPi)        | ✓        | ✓      |
 
-
-## Notes for Linux users
-
-Using the library on Linux with the ALSA backend might lead to unexpected cranky sound with the default render size (i.e. 128 frames). In such cases, a simple workaround is to pass the `playback` latency hint when creating the audio context, which will increase the render size to 1024 frames:
-
-```js
-const audioContext = new AudioContext({ latencyHint: 'playback' });
-```
-
-For real-time and interactive applications where low latency is crucial, you should instead rely on the JACK backend provided by `cpal`. By default the audio context will use that backend if a running JACK server is found.
-
-If you don't have JACK installed, you can still pass the `WEB_AUDIO_LATENCY=playback` env variable to all examples to create the audio context with the playback latency hint, e.g.:
-
-```sh
-WEB_AUDIO_LATENCY=playback node examples/amplitude-modulation.mjs
-```
-
-### Manual Build
+## Manual Build
 
 If prebuilt binaries are not shippped for your platform, you will need to:
 
@@ -119,15 +102,51 @@ The package will be built on your machine, which might take some time.
 
 Be aware that the package won't be listed on your `package.json` file, and that it won't be re-installed if running `npm install` again. A possible workaround would be to include the above in a postinstall script.
 
+## Notes for Linux users
+
+### Build
+
+To build the library, you will need to manually install the `libasound2-dev` package:
+
+```sh
+sudo apt install libasound2-dev
+```
+
+Optionally, if you use the Jack Audio Backend, the `libjack-jackd2-dev` package:
+
+```sh
+sudo apt install libjack-jackd2-dev
+```
+
+In such case, you can use the `npm run build:jack` script to enable the Jack feature.
+
+### Audio backend and latency
+
+Using the library on Linux with the ALSA backend might lead to unexpected cranky sound with the default render size (i.e. 128 frames). In such cases, a simple workaround is to pass the `playback` latency hint when creating the audio context, which will increase the render size to 1024 frames:
+
+```js
+const audioContext = new AudioContext({ latencyHint: 'playback' });
+```
+
+For real-time and interactive applications where low latency is crucial, you should instead rely on the JACK backend provided by `cpal`. By default the audio context will use that backend if a running JACK server is found.
+
+If you don't have JACK installed, you can still pass the `WEB_AUDIO_LATENCY=playback` environment variable to all examples to create the audio context with the playback latency hint, e.g.:
+
+```sh
+WEB_AUDIO_LATENCY=playback node examples/amplitude-modulation.mjs
+```
+
 ## Development notes
 
+### Synchronize versioning
+
 The npm `postversion` script rely on [`cargo-bump`](https://crates.io/crates/cargo-bump) to maintain versions synced between the `package.json` and the `Cargo.toml` files. Therefore, you will need to install `cargo-bump` on your machine
 
 ```
 cargo install cargo-bump
 ```
 
-## Running the web-platform-test suite
+### Running the web-platform-test suite
 
 Follow the steps for 'Manual Build' first. Then checkout the web-platform-tests submodule with:
 

package/all-checks.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+
+
+echo "-----------------------------------------------"
+echo "> cargo fmt -- --check --color always"
+echo "-----------------------------------------------"
+cargo fmt -- --check --color always
+
+echo "-----------------------------------------------"
+echo "> cargo clippy --all-targets --features cpal -- -D warnings"
+echo "-----------------------------------------------"
+cargo clippy --all-targets --features cpal -- -D warnings
+
+echo "-----------------------------------------------"
+echo "> Run js tests"
+echo "-----------------------------------------------"
+npm run test
+

package/js/AudioContext.js

@@ -89,30 +89,22 @@ module.exports = function(jsExport, nativeBinding) {
       });
 
       // Add function to Napi object to bridge from Rust events to JS EventTarget
-      this[kNapiObj][kOnStateChange] = (err, rawEvent) => {
-        if (typeof rawEvent !== 'object' && !('type' in rawEvent)) {
-          throw new TypeError('Invalid [kOnStateChange] Invocation: rawEvent should have a type property');
-        }
-
+      this[kNapiObj][kOnStateChange] = (function(err, rawEvent) {
         const event = new Event(rawEvent.type);
         propagateEvent(this, event);
-      };
-
-      this[kNapiObj][kOnSinkChange] = (err, rawEvent) => {
-        if (typeof rawEvent !== 'object' && !('type' in rawEvent)) {
-          throw new TypeError('Invalid [kOnSinkChange] Invocation: rawEvent should have a type property');
-        }
+      }).bind(this);
 
+      this[kNapiObj][kOnSinkChange] = (function(err, rawEvent) {
         const event = new Event(rawEvent.type);
         propagateEvent(this, event);
-      };
+      }).bind(this);
 
       // Workaround to bind the `sinkchange` and `statechange` events to EventTarget.
       // This must be called from JS facade ctor as the JS handler are added to the Napi
       // object after its instantiation, and that we don't have any initial `resume` call.
       this[kNapiObj].listen_to_events();
 
-      // @todo - check if this is still required
+      // @todo - This is probably not requested anymore as the event listeners
       // prevent garbage collection and process exit
       const id = contextId++;
       // store in process to prevent garbage collection
@@ -125,7 +117,6 @@ module.exports = function(jsExport, nativeBinding) {
       });
       // keep process awake until context is closed
       const keepAwakeId = setInterval(() => {}, 10 * 1000);
-
       // clear on close
       this.addEventListener('statechange', () => {
         if (this.state === 'closed') {
@@ -134,6 +125,11 @@ module.exports = function(jsExport, nativeBinding) {
           clearTimeout(keepAwakeId);
         }
       });
+
+      // for wpt tests, see ./.scripts/wpt_harness.mjs for informations
+      if (process.WPT_TEST_RUNNER) {
+        process.WPT_TEST_RUNNER.once('cleanup', () => this.close());
+      }
     }
 
     get baseLatency() {

package/js/AudioRenderCapacity.js

@@ -31,10 +31,10 @@ class AudioRenderCapacity extends EventTarget {
 
     this[kNapiObj] = options[kNapiObj];
 
-    this[kNapiObj][kOnUpdate] = (err, rawEvent) => {
+    this[kNapiObj][kOnUpdate] = (function(err, rawEvent) {
       const event = new AudioRenderCapacityEvent('update', rawEvent);
       propagateEvent(this, event);
-    };
+    }).bind(this);
 
     this[kNapiObj].listen_to_events();
   }

package/js/AudioScheduledSourceNode.js

@@ -33,14 +33,15 @@ class AudioScheduledSourceNode extends AudioNode {
 
     // Add function to Napi object to bridge from Rust events to JS EventTarget
     // It will be effectively registered on rust side when `start` is called
-    this[kNapiObj][kOnEnded] = (err, rawEvent) => {
-      if (typeof rawEvent !== 'object' && !('type' in rawEvent)) {
-        throw new TypeError('Invalid [kOnEnded] Invocation: rawEvent should have a type property');
-      }
-
+    //
+    // Note 2024-06-05 - We use bind instead of arrow function because arrow function
+    // prevent the node to be collected by Scavenge step of GC, which can lead to
+    // oversized graphs and performance issues.
+    // cf. https://github.com/ircam-ismm/node-web-audio-api/tree/fix/118
+    this[kNapiObj][kOnEnded] = (function(_err, rawEvent) {
       const event = new Event(rawEvent.type);
       propagateEvent(this, event);
-    };
+    }).bind(this);
   }
 
   get onended() {

package/js/AudioWorklet.js

@@ -64,7 +64,7 @@ const resolveModule = async (moduleUrl) => {
     // get caller site from error stack trace
     const callerSite = caller(2);
 
-    if (callerSite.startsWith('http')) {
+    if (callerSite.startsWith('http')) { // this branch exists for wpt where caller site is an url
       let url;
       // handle origin relative and caller path relative URLs
       if (moduleUrl.startsWith('/')) {
@@ -134,6 +134,14 @@ class AudioWorklet {
           resolve();
           break;
         }
+        case 'node-web-audio-api:worklet:add-module-failed': {
+          const { promiseId, ctor, name, message } = event;
+          const { reject } = this.#idPromiseMap.get(promiseId);
+          this.#idPromiseMap.delete(promiseId);
+          const err = new globalThis[ctor](message, name);
+          reject(err);
+          break;
+        }
         case 'node-web-audio-api:worlet:processor-registered': {
           const { name, parameterDescriptors } = event;
           this.#workletParamDescriptorsMap.set(name, parameterDescriptors);
@@ -188,7 +196,6 @@ class AudioWorklet {
   // For OfflineAudioContext only, check that all processors have been properly
   // created before actual `startRendering`
   async [kCheckProcessorsCreated]() {
-    // console.log(this.#pendingCreateProcessors);
     return new Promise(async resolve => {
       while (this.#pendingCreateProcessors.size !== 0) {
         // we need a microtask to ensure message can be received

package/js/AudioWorkletGlobalScope.js

@@ -1,10 +1,12 @@
 const {
   parentPort,
   workerData,
+  markAsUntransferable,
 } = require('node:worker_threads');
 
 const conversions = require('webidl-conversions');
 
+// these are defined in rust side
 const {
   exit_audio_worklet_global_scope,
   run_audio_worklet_global_scope,
@@ -21,14 +23,72 @@ const kWorkletInputs = Symbol.for('node-web-audio-api:worklet-inputs');
 const kWorkletOutputs = Symbol.for('node-web-audio-api:worklet-outputs');
 const kWorkletParams = Symbol.for('node-web-audio-api:worklet-params');
 const kWorkletParamsCache = Symbol.for('node-web-audio-api:worklet-params-cache');
+const kWorkletGetBuffer = Symbol.for('node-web-audio-api:worklet-get-buffer');
+const kWorkletRecycleBuffer = Symbol.for('node-web-audio-api:worklet-recycle-buffer');
+const kWorkletRecycleBuffer1 = Symbol.for('node-web-audio-api:worklet-recycle-buffer-1');
+const kWorkletMarkAsUntransferable = Symbol.for('node-web-audio-api:worklet-mark-as-untransferable');
 // const kWorkletOrderedParamNames = Symbol.for('node-web-audio-api:worklet-ordered-param-names');
 
+
 const nameProcessorCtorMap = new Map();
 const processors = {};
 let pendingProcessorConstructionData = null;
 let loopStarted = false;
 let runLoopImmediateId = null;
 
+class BufferPool {
+  #bufferSize;
+  #pool;
+
+  constructor(bufferSize, initialPoolSize) {
+    this.#bufferSize = bufferSize;
+    this.#pool = new Array(initialPoolSize);
+
+    for (let i = 0; i < this.#pool.length; i++) {
+      this.#pool[i] = this.#allocate();
+    }
+  }
+
+  #allocate() {
+    const float32 = new Float32Array(this.#bufferSize);
+    markAsUntransferable(float32);
+    // Mark underlying buffer as untransfrable too, this will fail one of
+    // the task in `audioworkletprocessor-process-frozen-array.https.html`
+    // but prevent segmentation fault
+    markAsUntransferable(float32.buffer);
+
+    return float32;
+  }
+
+  get() {
+    if (this.#pool.length === 0) {
+      return this.#allocate();
+    }
+
+    return this.#pool.pop();
+  }
+
+  recycle(buffer) {
+    // make sure we cannot polute our pool
+    if (buffer.length === this.#bufferSize) {
+      this.#pool.push(buffer);
+    }
+  }
+}
+
+const renderQuantumSize = 128;
+
+const pool128 = new BufferPool(renderQuantumSize, 256);
+const pool1 = new BufferPool(1, 64);
+// allow rust to access some methods required when io layout change
+globalThis[kWorkletGetBuffer] = () => pool128.get();
+globalThis[kWorkletRecycleBuffer] = buffer => pool128.recycle(buffer);
+globalThis[kWorkletRecycleBuffer1] = buffer => pool1.recycle(buffer);
+globalThis[kWorkletMarkAsUntransferable] = obj => {
+  markAsUntransferable(obj);
+  return obj;
+}
+
 function isIterable(obj) {
   // checks for null and undefined
   if (obj === null || obj === undefined) {
@@ -54,12 +114,11 @@ function runLoop() {
   runLoopImmediateId = setImmediate(runLoop);
 }
 
-// s
 globalThis.currentTime = 0
 globalThis.currentFrame = 0;
 globalThis.sampleRate = sampleRate;
 // @todo - implement in upstream crate
-// globalThis.renderQuantumSize = 128;
+globalThis.renderQuantumSize = renderQuantumSize;
 
 globalThis.AudioWorkletProcessor = class AudioWorkletProcessor {
   static get parameterDescriptors() {
@@ -76,13 +135,14 @@ globalThis.AudioWorkletProcessor = class AudioWorkletProcessor {
       parameterDescriptors,
     } = pendingProcessorConstructionData;
 
-    // @todo - Mark [[callable process]] as true, set to false in render quantum
+    // Mark [[callable process]] as true, set to false in render quantum
     // either "process" doese not exists, either it throws an error
     this[kWorkletCallableProcess] = true;
-    // @todo - reuse Float32Arrays between calls + freeze arrays
+
+    // Populate with dummy values which will be replaced in first render call
     this[kWorkletInputs] = new Array(numberOfInputs).fill([]);
-    // @todo - use `outputChannelCount`
     this[kWorkletOutputs] = new Array(numberOfOutputs).fill([]);
+
     // Object to be reused as `process` parameters argument
     this[kWorkletParams] = {};
     // Cache of 2 Float32Array (of length 128 and 1) for each param, to be reused on
@@ -91,8 +151,8 @@ globalThis.AudioWorkletProcessor = class AudioWorkletProcessor {
 
     parameterDescriptors.forEach(desc => {
       this[kWorkletParamsCache][desc.name] = [
-        new Float32Array(128), // should be globalThis.renderQuantumSize
-        new Float32Array(1),
+        pool128.get(), // should be globalThis.renderQuantumSize
+        pool1.get(),
       ]
     });
 
@@ -234,13 +294,11 @@ globalThis.registerProcessor = function registerProcessor(name, processorCtor) {
 // NOTE: Authors that register an event listener on the "message" event of this
 // port should call close on either end of the MessageChannel (either in the
 // AudioWorklet or the AudioWorkletGlobalScope side) to allow for resources to be collected.
-parentPort.on('exit', () => {
-  process.stdout.write('closing worklet');
-});
+// parentPort.on('exit', () => {
+//   process.stdout.write('closing worklet');
+// });
 
 parentPort.on('message', event => {
-  console.log(event.cmd + '\n');
-
   switch (event.cmd) {
     case 'node-web-audio-api:worklet:init': {
       const { workletId, processors, promiseId } = event;
@@ -257,13 +315,23 @@ parentPort.on('message', event => {
     case 'node-web-audio-api:worklet:add-module': {
       const { code, promiseId } = event;
       const func = new Function('AudioWorkletProcessor', 'registerProcessor', code);
-      func(AudioWorkletProcessor, registerProcessor);
 
-      // send registered param descriptors on main thread and resolve Promise
-      parentPort.postMessage({
-        cmd: 'node-web-audio-api:worklet:module-added',
-        promiseId,
-      });
+      try {
+        func(AudioWorkletProcessor, registerProcessor);
+        // send registered param descriptors on main thread and resolve Promise
+        parentPort.postMessage({
+          cmd: 'node-web-audio-api:worklet:module-added',
+          promiseId,
+        });
+      } catch (err) {
+        parentPort.postMessage({
+          cmd: 'node-web-audio-api:worklet:add-module-failed',
+          promiseId,
+          ctor: err.constructor.name,
+          name: err.name,
+          message: err.message,
+        });
+      }
       break;
     }
     case 'node-web-audio-api:worklet:create-processor': {

package/js/AudioWorkletNode.js

@@ -98,14 +98,16 @@ module.exports = (jsExport, nativeBinding) => {
           throw new DOMException(`Failed to construct 'AudioWorkletNode': Invalid 'outputChannelCount' property from AudioWorkletNodeOptions: 'outputChannelCount' length (${parsedOptions.outputChannelCount.length}) does not equal 'numberOfOutputs' (${parsedOptions.numberOfOutputs})`, 'IndexSizeError');
         }
       } else {
-        // If outputChannelCount does not exists,
         // - If both numberOfInputs and numberOfOutputs are 1, set the initial channel count of the node output to 1 and return.
         //   NOTE: For this case, the output chanel count will change to computedNumberOfChannels dynamically based on the input and the channelCountMode at runtime.
-        // - Otherwise set the channel count of each output of the node to 1 and return.
-
-        // @note - not sure what this means, let's go simple
-        parsedOptions.outputChannelCount = new Uint32Array(parsedOptions.numberOfOutputs);
-        parsedOptions.outputChannelCount.fill(1);
+        if (parsedOptions.numberOfInputs === 1 && parsedOptions.numberOfOutputs === 1) {
+          // rust waits for an empty Vec as the special case value
+          parsedOptions.outputChannelCount = new Uint32Array(0);
+        } else {
+          // - Otherwise set the channel count of each output of the node to 1 and return.
+          parsedOptions.outputChannelCount = new Uint32Array(parsedOptions.numberOfOutputs);
+          parsedOptions.outputChannelCount.fill(1);
+        }
       }
 
       // @todo

package/js/OfflineAudioContext.js

@@ -82,33 +82,26 @@ module.exports = function patchOfflineAudioContext(jsExport, nativeBinding) {
 
       // Add function to Napi object to bridge from Rust events to JS EventTarget
       // They will be effectively registered on rust side when `startRendering` is called
-      this[kNapiObj][kOnStateChange] = (err, rawEvent) => {
-        if (typeof rawEvent !== 'object' && !('type' in rawEvent)) {
-          throw new TypeError('Invalid [kOnStateChange] Invocation: rawEvent should have a type property');
-        }
-
+      this[kNapiObj][kOnStateChange] = (function(_err, rawEvent) {
         const event = new Event(rawEvent.type);
         propagateEvent(this, event);
-      };
+      }).bind(this);
 
       // This event is, per spec, the last trigerred one
-      this[kNapiObj][kOnComplete] = (err, rawEvent) => {
-        if (typeof rawEvent !== 'object' && !('type' in rawEvent)) {
-          throw new TypeError('Invalid [kOnComplete] Invocation: rawEvent should have a type property');
-        }
-
-        // @fixme: workaround the fact that this event seems to be triggered before
+      this[kNapiObj][kOnComplete] = (function(err, rawEvent) {
+        // workaround the fact that the oncomplete event is triggered before
         // startRendering fulfills and that we want to return the exact same instance
-        if (this.#renderedBuffer === null) {
-          this.#renderedBuffer = new jsExport.AudioBuffer({ [kNapiObj]: rawEvent.renderedBuffer });
-        }
+        this.#renderedBuffer = new jsExport.AudioBuffer({ [kNapiObj]: rawEvent.renderedBuffer });
 
         const event = new jsExport.OfflineAudioCompletionEvent(rawEvent.type, {
           renderedBuffer: this.#renderedBuffer,
         });
 
-        propagateEvent(this, event);
-      };
+        // delay event propagation to next tick that it is executed after startRendering fulfills
+        setImmediate(() => {
+          propagateEvent(this, event);
+        }, 0);
+      }).bind(this);
     }
 
     get length() {
@@ -153,15 +146,9 @@ module.exports = function patchOfflineAudioContext(jsExport, nativeBinding) {
         throwSanitizedError(err);
       }
 
-      // release audio worklet, if any
+      // release audio worklets
       await this.audioWorklet[kWorkletRelease]();
 
-      // workaround the fact that this event seems to be triggered before
-      // startRendering fulfills and that we want to return the exact same instance
-      if (this.#renderedBuffer === null) {
-        this.#renderedBuffer = new jsExport.AudioBuffer({ [kNapiObj]: nativeAudioBuffer });
-      }
-
       return this.#renderedBuffer;
     }
 

package/js/ScriptProcessorNode.js

@@ -107,11 +107,7 @@ module.exports = (jsExport, nativeBinding) => {
         [kNapiObj]: napiObj,
       });
 
-      this[kNapiObj][kOnAudioProcess] = (err, rawEvent) => {
-        if (typeof rawEvent !== 'object' && !('type' in rawEvent)) {
-          throw new TypeError('Invalid [kOnStateChange] Invocation: rawEvent should have a type property');
-        }
-
+      this[kNapiObj][kOnAudioProcess] = (function(err, rawEvent) {
         const audioProcessingEventInit = {
           playbackTime: rawEvent.playbackTime,
           inputBuffer: new jsExport.AudioBuffer({ [kNapiObj]: rawEvent.inputBuffer }),
@@ -120,7 +116,7 @@ module.exports = (jsExport, nativeBinding) => {
 
         const event = new jsExport.AudioProcessingEvent('audioprocess', audioProcessingEventInit);
         propagateEvent(this, event);
-      };
+      }).bind(this);
 
       this[kNapiObj].listen_to_events();
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "node-web-audio-api",
-  "version": "0.21.0",
+  "version": "0.21.2",
   "author": "Benjamin Matuszewski",
   "description": "Node.js bindings for web-audio-api-rs using napi-rs",
   "exports": {
@@ -44,13 +44,15 @@
     "lint": "npx eslint index.cjs index.mjs && npx eslint js/*.js && npx eslint examples/*.mjs",
     "preversion": "yarn install && npm run generate",
     "postversion": "cargo bump $npm_package_version && git commit -am \"v$npm_package_version\" && node .scripts/check-changelog.mjs",
-    "test": "mocha tests",
+    "test": "mocha tests/*.spec.mjs",
+    "test:only": "mocha",
     "wpt": "npm run build && node ./.scripts/wpt-harness.mjs",
     "wpt:only": "node ./.scripts/wpt-harness.mjs"
   },
   "devDependencies": {
     "@ircam/eslint-config": "^1.3.0",
     "@ircam/sc-gettime": "^1.0.0",
+    "@ircam/sc-scheduling": "^0.1.7",
     "@ircam/sc-utils": "^1.3.3",
     "@sindresorhus/slugify": "^2.1.1",
     "camelcase": "^7.0.1",
@@ -66,7 +68,6 @@
     "octokit": "^2.0.11",
     "ping": "^0.4.2",
     "template-literal": "^1.0.4",
-    "waves-masters": "^2.3.1",
     "webidl2": "^24.2.0",
     "wpt-runner": "^5.0.0"
   },