@pexip/media-processor 21.0.0 → 22.1.0

package/CHANGELOG.md

@@ -1,5 +1,85 @@
 # Changelog
 
+## 22.1.0
+
+### Patch Changes
+
+- 136aa70: Fix race on close
+- ad47e6a: Log Segmenter with different level
+- Updated dependencies [44ad859]
+- Updated dependencies [514c523]
+  - @pexip/utils@17.4.0
+
+## 22.0.0
+
+### Major Changes
+
+- 498ff3b: A full architectural overhaul of the video processing pipeline:
+
+  BREAKING CHANGES:
+
+  - **`createCanvasTransform` removed**
+  - **`createVideoTrackProcessor` / `createVideoTrackProcessorWithFallback`
+    removed**
+  - **`createVideoProcessor` redesigned** — now accepts a `Segmenter` +
+    `InputProcessAdaptor` instead of an array of `Transform`s. Exposes
+    `update()`, `mute()`, `status`, `renderOptions`, and processing dimensions.
+  - **New `InputProcessAdaptor` system**:
+    - `createMediaStreamTrackInputProcessAdaptor` — uses native
+      `MediaStreamTrack` transfer (Safari v18+ compatible).
+    - `createReadWritableStreamInputProcessAdaptor` — uses
+      `MediaStreamTrackProcessor` + `MediaStreamTrackGenerator`.
+    - `createReadWritableStreamFallbackInputProcessAdaptor` — canvas/video
+      element fallback.
+    - `getDefaultInputProcessAdaptor` — picks the best path via feature
+      detection (`MediaStreamTrackTransferable`).
+  - **`createSegmenter` updated**
+    - Now processes `MediaStreamTrack` or `ReadWritableStream` directly (no more
+      `VideoFrameLike`).
+    - Added `mute()` method.
+    - Added `stats` callback for frame timing data.
+    - Worker is no longer re-created on every `open()`.
+  - **Worker rewritten**:
+    - Processing now uses `TransformStream` to pipe `VideoFrame`s through
+      segmentation.
+    - Supports both track-based
+      (`VideoTrackGenerator`/`MediaStreamTrackProcessor`) and ReadWritable
+      stream processing.
+    - Added frame timing stats (`FrameTransformStats`) batched and posted back
+      to main thread.
+    - Removes direct canvas-copy output (`directOutput`/`output`); renderer now
+      writes to `processCanvas` and creates a new `VideoFrame`.
+    - `close` and `suspend` separated; resource cleanup improved.
+    - Background image fetching is now abortable.
+  - **`Renderer.render()` signature simplified** — accepts `OffscreenCanvas` +
+    `VideoFrame` directly (no `VideoFrameLike` wrapper).
+  - **WebGL/WebGPU texture renderers** updated to accept
+    `VideoFrame | ImageBitmap | OffscreenCanvas` natively.
+  - **WebGL program release guard** added to prevent double-free.
+  - **`benchUtils`** extended to accept explicit timestamps in
+    `begin()`/`end()` + tests added.
+  - **`MediaStreamTrackTransferable`** feature-detection helper exported.
+  - **Default frame rate raised** from 20 → 30 fps.
+  - **Default backend changed** to `webgpu`.
+
+- fe0a0a4: Set a max width of 300px of in meeting selfview.
+
+  Rename openEffectsModal prop on InMeetingSelfview to openEffects
+
+  Make selfview muted indicator look more like an indicator
+
+  Show tooltips for background effects button and upload background effect
+
+### Patch Changes
+
+- f8bd1c5: Fix VideoProcessor and Segment open/update behavior
+- 925cce6: Fix f16 issue
+- fb41d4d: Changes related to Typescript V6 Upgrade
+- Updated dependencies [498ff3b]
+- Updated dependencies [fb41d4d]
+- Updated dependencies [b99c070]
+  - @pexip/utils@17.3.0
+
 ## 21.0.0
 
 ### Major Changes

package/README.md

@@ -106,10 +106,12 @@ await audioGraph.release();
 
 ```typescript
 import {
+  createMediaStreamTrackInputProcessAdaptor,
+  createReadWritableStreamFallbackInputProcessAdaptor,
+  createReadWritableStreamInputProcessAdaptor,
   createSegmenter,
-  createCanvasTransform,
-  createVideoTrackProcessor,
-  createVideoTrackProcessorWithFallback,
+  createVideoProcessor,
+  getDefaultInputProcessAdaptor,
 } from '@pexip/media-processor';
 
 // Grab the user's camera stream
@@ -129,17 +131,22 @@ const modelAsset = {
   modelName: 'selfie' as const,
 };
 
-const segmenter = createSegmenter(tasksVisionBasePath, {modelAssets});
-// Create a processing transformer and set the effects to `blur`
-const transformer = createCanvasTransform(segmenter, {effects: 'blur'});
-const processor = createVideoProcessor([transformer]);
+const segmenter = createSegmenter(tasksVisionBasePath, {modelAsset});
+// Choose the appropriate input process adaptor based on browser capabilities
+const inputProcessAdaptor = getDefaultInputProcessAdaptor(true);
+// Create a processor and set the effects to `blur`
+const processor = createVideoProcessor(segmenter, inputProcessAdaptor);
 
 // Start the processor
 await videoProcessor.open();
 
-// Passing the raw MediaStream to apply the effects
-// Then, use the output stream for whatever purpose
-const processedStream = await videoProcessor.process(stream);
+// Start the processor with blur effects
+await processor.open({renderOptions: {effects: 'blur'}});
+
+// Process the video track and create a new stream
+const videoTrack = stream.getVideoTracks()[0];
+const processedTrack = await processor.process(videoTrack);
+const processedStream = new MediaStream([processedTrack]);
 ```
 
 ## Profiling Web Audio

package/api-docs/README.mdx

@@ -106,10 +106,12 @@ await audioGraph.release();
 
 ```typescript
 import {
+  createMediaStreamTrackInputProcessAdaptor,
+  createReadWritableStreamFallbackInputProcessAdaptor,
+  createReadWritableStreamInputProcessAdaptor,
   createSegmenter,
-  createCanvasTransform,
-  createVideoTrackProcessor,
-  createVideoTrackProcessorWithFallback,
+  createVideoProcessor,
+  getDefaultInputProcessAdaptor,
 } from '@pexip/media-processor';
 
 // Grab the user's camera stream
@@ -129,17 +131,22 @@ const modelAsset = {
   modelName: 'selfie' as const,
 };
 
-const segmenter = createSegmenter(tasksVisionBasePath, {modelAssets});
-// Create a processing transformer and set the effects to `blur`
-const transformer = createCanvasTransform(segmenter, {effects: 'blur'});
-const processor = createVideoProcessor([transformer]);
+const segmenter = createSegmenter(tasksVisionBasePath, {modelAsset});
+// Choose the appropriate input process adaptor based on browser capabilities
+const inputProcessAdaptor = getDefaultInputProcessAdaptor(true);
+// Create a processor and set the effects to `blur`
+const processor = createVideoProcessor(segmenter, inputProcessAdaptor);
 
 // Start the processor
 await videoProcessor.open();
 
-// Passing the raw MediaStream to apply the effects
-// Then, use the output stream for whatever purpose
-const processedStream = await videoProcessor.process(stream);
+// Start the processor with blur effects
+await processor.open({renderOptions: {effects: 'blur'}});
+
+// Process the video track and create a new stream
+const videoTrack = stream.getVideoTracks()[0];
+const processedTrack = await processor.process(videoTrack);
+const processedStream = new MediaStream([processedTrack]);
 ```
 
 ## Profiling Web Audio
@@ -193,11 +200,13 @@ A library for media analysis using Web APIs.
 | Interface | Description |
 | ------ | ------ |
 | [Segmentation](interfaces/Segmentation.mdx) | - |
+| [ReadWritableStream](interfaces/ReadWritableStream.mdx) | - |
 | [VideoFrameLike](interfaces/VideoFrameLike.mdx) | - |
 | [ImageRecord](interfaces/ImageRecord.mdx) | - |
 | [Rect](interfaces/Rect.mdx) | - |
 | [ProcessEventOpen](interfaces/ProcessEventOpen.mdx) | Message event to initializing the Processor |
 | [ProcessEventProcess](interfaces/ProcessEventProcess.mdx) | Message event to process the video frame |
+| [ProcessEventMute](interfaces/ProcessEventMute.mdx) | - |
 | [ProcessEventUpdate](interfaces/ProcessEventUpdate.mdx) | Message event to update processor options |
 | [ProcessEventClose](interfaces/ProcessEventClose.mdx) | Message event to stop the processing |
 | [ProcessEventDestroy](interfaces/ProcessEventDestroy.mdx) | Message event to release all the resources |
@@ -205,8 +214,11 @@ A library for media analysis using Web APIs.
 | [ProcessWorkerEventProcessed](interfaces/ProcessWorkerEventProcessed.mdx) | - |
 | [ProcessWorkerEventOpened](interfaces/ProcessWorkerEventOpened.mdx) | - |
 | [ProcessWorkerEventUpdated](interfaces/ProcessWorkerEventUpdated.mdx) | - |
+| [ProcessWorkerEventClosed](interfaces/ProcessWorkerEventClosed.mdx) | - |
 | [ProcessWorkerEvent](interfaces/ProcessWorkerEvent.mdx) | - |
 | [ProcessWorkerEventDebug](interfaces/ProcessWorkerEventDebug.mdx) | - |
+| [FrameTransformStats](interfaces/FrameTransformStats.mdx) | Frame transform statistic, i.e. the timing data for processing each frame |
+| [ProcessWorkerEventStats](interfaces/ProcessWorkerEventStats.mdx) | - |
 | [ProcessorOptions](interfaces/ProcessorOptions.mdx) | - |
 | [ProcessorProcessOptions](interfaces/ProcessorProcessOptions.mdx) | - |
 | [ProcessWorkerDebugMessage](interfaces/ProcessWorkerDebugMessage.mdx) | - |
@@ -242,13 +254,12 @@ A library for media analysis using Web APIs.
 | [Clock](interfaces/Clock.mdx) | Clock interface to get the current time with now method, |
 | [ThrottleOptions](interfaces/ThrottleOptions.mdx) | Limit the rate of flow in terms of millisecond, and provided Clock |
 | [Runner](interfaces/Runner.mdx) | - |
-| [Transform](interfaces/Transform.mdx) | - |
 | [AsyncAssets](interfaces/AsyncAssets.mdx) | - |
 | [Process](interfaces/Process.mdx) | - |
 | [BackgroundImage](interfaces/BackgroundImage.mdx) | - |
 | [Segmenter](interfaces/Segmenter.mdx) | - |
-| [Detector](interfaces/Detector.mdx) | - |
-| [SegmentationParams](interfaces/SegmentationParams.mdx) | Options controlling the rendering and effects for person/background segmentation. All values are stable per-frame and can be safely updated in real-time. |
+| [InputProcessAdaptor](interfaces/InputProcessAdaptor.mdx) | - |
+| [ProcessorUpdateOptions](interfaces/ProcessorUpdateOptions.mdx) | - |
 | [VideoProcessor](interfaces/VideoProcessor.mdx) | - |
 
 ## Type Aliases
@@ -266,6 +277,7 @@ A library for media analysis using Web APIs.
 | [ProcessWorkerEvents](type-aliases/ProcessWorkerEvents.mdx) | Message events returned from the processor |
 | [ProcessorEvents](type-aliases/ProcessorEvents.mdx) | - |
 | [ProcessorWorkerEvents](type-aliases/ProcessorWorkerEvents.mdx) | - |
+| [Struct](type-aliases/Struct.mdx) | - |
 | [SegmentationModel](type-aliases/SegmentationModel.mdx) | - |
 | [Delegate](type-aliases/Delegate.mdx) | - |
 | [ExtractMessageEventType](type-aliases/ExtractMessageEventType.mdx) | - |
@@ -307,10 +319,9 @@ A library for media analysis using Web APIs.
 | [Callback](type-aliases/Callback.mdx) | - |
 | [RunnerCreator](type-aliases/RunnerCreator.mdx) | - |
 | [WasmPaths](type-aliases/WasmPaths.mdx) | - |
+| [Logger](type-aliases/Logger.mdx) | - |
 | [Color](type-aliases/Color.mdx) | - |
 | [InputFrame](type-aliases/InputFrame.mdx) | - |
-| [SegmentationTransform](type-aliases/SegmentationTransform.mdx) | - |
-| [ProcessVideoTrack](type-aliases/ProcessVideoTrack.mdx) | - |
 
 ## Variables
 
@@ -342,18 +353,24 @@ A library for media analysis using Web APIs.
 | [RENDER\_EFFECT](variables/RENDER_EFFECT.mdx) | - |
 | [RENDER\_BACKEND](variables/RENDER_BACKEND.mdx) | - |
 | [RENDERING\_EVENTS](variables/RENDERING_EVENTS.mdx) | - |
+| [isValidGpuAPI](variables/isValidGpuAPI.mdx) | - |
+| [isRenderOptionsKey](variables/isRenderOptionsKey.mdx) | - |
+| [RENDERER\_OPTIONS\_STRUCT](variables/RENDERER_OPTIONS_STRUCT.mdx) | - |
 | [SEGMENTATION\_MODELS](variables/SEGMENTATION_MODELS.mdx) | - |
 | [isSegmentationModel](variables/isSegmentationModel.mdx) | - |
 | [isRenderingEvents](variables/isRenderingEvents.mdx) | - |
 | [toRenderingEvents](variables/toRenderingEvents.mdx) | - |
 | [getErrorMessage](variables/getErrorMessage.mdx) | - |
 | [createObjectUpdater](variables/createObjectUpdater.mdx) | - |
+| [ImageFetchError](variables/ImageFetchError.mdx) | - |
 | [createRemoteImageBitmap](variables/createRemoteImageBitmap.mdx) | - |
 | [clamping](variables/clamping.mdx) | - |
 | [calculateMaxBlurPass](variables/calculateMaxBlurPass.mdx) | - |
 | [resize](variables/resize.mdx) | Calculate the coordinates and size of the source and destination to fit the target aspect ratio. |
 | [createLazyProps](variables/createLazyProps.mdx) | - |
 | [handleWebGLContextLoss](variables/handleWebGLContextLoss.mdx) | - |
+| [cloneImageRecord](variables/cloneImageRecord.mdx) | - |
+| [compareImageRecords](variables/compareImageRecords.mdx) | Compare provided ImageRecord |
 | [urls](variables/urls.mdx) | - |
 | [SILENT\_THRESHOLD](variables/SILENT_THRESHOLD.mdx) | Default silent threshold At least one LSB 16-bit data (compare is on absolute value). |
 | [MONO\_THRESHOLD](variables/MONO_THRESHOLD.mdx) | Default mono detection threshold Data must be identical within one LSB 16-bit to be identified as mono. |
@@ -361,6 +378,7 @@ A library for media analysis using Web APIs.
 | [CLIP\_THRESHOLD](variables/CLIP_THRESHOLD.mdx) | Default clipping detection threshold |
 | [VOICE\_PROBABILITY\_THRESHOLD](variables/VOICE_PROBABILITY_THRESHOLD.mdx) | Default Voice probability threshold |
 | [CLIP\_COUNT\_THRESHOLD](variables/CLIP_COUNT_THRESHOLD.mdx) | Default clipping count threshold Number of consecutive clipThreshold level samples that indicate clipping. |
+| [MediaStreamTrackTransferable](variables/MediaStreamTrackTransferable.mdx) | Result of the feature detection if `MediaStreamTrack` is `Transferable` |
 | [FRAME\_RATE](variables/FRAME_RATE.mdx) | - |
 | [PLAY\_VIDEO\_TIMEOUT](variables/PLAY_VIDEO_TIMEOUT.mdx) | - |
 
@@ -368,7 +386,7 @@ A library for media analysis using Web APIs.
 
 | Function | Description |
 | ------ | ------ |
-| [nearestPowerOfTwo](functions/nearestPowerOfTwo.mdx) | Coerce any positive number to the nearest power of two within [minPow, maxPow] Example: nearestPowerOfTwo(13) => 16; nearestPowerOfTwo(3.2) => 4 |
+| [nearestPowerOfTwo](functions/nearestPowerOfTwo.mdx) | Coerce any positive number to the nearest power of two within [minPow, maxPow] Example: nearestPowerOfTwo(13) =\> 16; nearestPowerOfTwo(3.2) =\> 4 |
 | [iou](functions/iou.mdx) | Intersection over Union (IoU), how much two masks overlap as a fraction of either's total. 1.0 = perfect match, 0 = no overlap. Used for stable, robust tracking of the main person between frames, especially with bystander removal and temporal instance switching. |
 | [labelComponents](functions/labelComponents.mdx) | - |
 | [scoreComponent](functions/scoreComponent.mdx) | - |
@@ -419,18 +437,21 @@ A library for media analysis using Web APIs.
 | [isAudioNodeInit](functions/isAudioNodeInit.mdx) | - |
 | [isAnalyzerNodeInit](functions/isAnalyzerNodeInit.mdx) | - |
 | [createAsyncCallbackLoop](functions/createAsyncCallbackLoop.mdx) | Create an async callback loop to be called recursively with delay based on the `frameRate` |
-| [createCanvasTransform](functions/createCanvasTransform.mdx) | - |
+| [createReadWritableStreamInputProcessAdaptor](functions/createReadWritableStreamInputProcessAdaptor.mdx) | An implementation of `InputProcessAdaptor` which uses `MediaStreamTrackGenerator` and `MediaStreamTrackProcessor` APIs to create `ReadableStream` & `WritableStream` |
+| [createReadWritableStreamFallbackInputProcessAdaptor](functions/createReadWritableStreamFallbackInputProcessAdaptor.mdx) | An implementation of `InputProcessAdaptor` which uses `HTMLCanvasElement` or `OffscreenCanvas` API to create `ReadableStream` & `WritableStream` |
+| [createMediaStreamTrackInputProcessAdaptor](functions/createMediaStreamTrackInputProcessAdaptor.mdx) | An implementation of `InputProcessAdaptor` which adapts `MediaStreamTrack` directly as specified from the spec https://w3c.github.io/mediacapture-transform |
+| [getDefaultInputProcessAdaptor](functions/getDefaultInputProcessAdaptor.mdx) | Get the supported InputProcessAdaptor based on feature detection and preference provided |
 | [loadScript](functions/loadScript.mdx) | - |
 | [loadWasms](functions/loadWasms.mdx) | - |
 | [createSegmenter](functions/createSegmenter.mdx) | - |
 | [isRenderEffects](functions/isRenderEffects.mdx) | - |
+| [isMediaStreamVideoTrack](functions/isMediaStreamVideoTrack.mdx) | - |
+| [assertMediaStreamVideoTrack](functions/assertMediaStreamVideoTrack.mdx) | - |
 | [createCanvas](functions/createCanvas.mdx) | Create a Canvas element with provided width and height |
 | [setVideoElementSrc](functions/setVideoElementSrc.mdx) | - |
 | [playVideo](functions/playVideo.mdx) | - |
 | [createFrameCallbackRequest](functions/createFrameCallbackRequest.mdx) | Create a callback loop for video frame processing using `requestVideoFrameCallback` under-the-hood when available otherwise our fallback implementation based on `setTimeout`. |
 | [createVideoProcessor](functions/createVideoProcessor.mdx) | - |
-| [createVideoTrackProcessor](functions/createVideoTrackProcessor.mdx) | Video track processor using MediaStreamTrackProcessor API |
-| [createVideoTrackProcessorWithFallback](functions/createVideoTrackProcessorWithFallback.mdx) | Video track processor using Canvas[captureStream] API |
 | [calculateDistance](functions/calculateDistance.mdx) | Calculate the distance between two Points |
 | [getBezierCurveControlPoints](functions/getBezierCurveControlPoints.mdx) | Spline Interpolation for Bezier Curve |
 | [line](functions/line.mdx) | Create a straight line path command |

package/api-docs/functions/assertMediaStreamVideoTrack.mdx

@@ -0,0 +1,13 @@
+```ts
+function assertMediaStreamVideoTrack(t): asserts t is MediaStreamVideoTrack;
+```
+
+## Parameters
+
+| Parameter | Type |
+| ------ | ------ |
+| `t` | `MediaStreamTrack` |
+
+## Returns
+
+`asserts t is MediaStreamVideoTrack`

package/api-docs/functions/avg.mdx

@@ -8,7 +8,7 @@ Average an array of numbers
 
 | Parameter | Type | Description |
 | ------ | ------ | ------ |
-| `nums` | `number`[] | An array of numbers |
+| `nums` | `number[]` | An array of numbers |
 
 ## Returns
 

package/api-docs/functions/createAnalyzerGraphNode.mdx

@@ -12,7 +12,9 @@ Create an AnalyzerNodeInit
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`AnalyserNode`, [`Analyzer`](../interfaces/Analyzer.mdx)\>
+```ts
+AudioNodeInit<AnalyserNode, Analyzer>
+```
 
 ## See
 

package/api-docs/functions/createAnalyzerSubscribableGraphNode.mdx

@@ -12,4 +12,6 @@ Create an analyzer node with push-based subscription
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`AnalyserNode`, [`Analyzer`](../interfaces/Analyzer.mdx)\>
+```ts
+AudioNodeInit<AnalyserNode, Analyzer>
+```

package/api-docs/functions/createAsyncCallbackLoop.mdx

@@ -12,16 +12,16 @@ the `frameRate`
 
 | Type Parameter |
 | ------ |
-| `P` *extends* `unknown`[] |
-| `R` *extends* `Promise`\<`unknown`\> |
+| `P extends unknown[]` |
+| `R extends Promise<unknown>` |
 
 ## Parameters
 
 | Parameter | Type | Description |
 | ------ | ------ | ------ |
-| `callback` | [`Callback`](../type-aliases/Callback.mdx)\<`R`, `P`\> | The callback to be invoked |
+| `callback` | [`Callback<R, P>`](../type-aliases/Callback.mdx) | The callback to be invoked |
 | `frameRate` | `number` | The rate to be expected to invoke the `callback` |
-| `__namedParameters` | `Partial`\<`AsyncCallbackLoopOptions`\> | - |
+| `__namedParameters` | `Partial<AsyncCallbackLoopOptions>` | - |
 
 ## Returns
 
@@ -29,7 +29,7 @@ the `frameRate`
 
 | Name | Type |
 | ------ | ------ |
-| `start()` | (...`params`) => `Promise`\<`void`\> |
-| `stop()` | () => `void` |
+| `start()` | `(...params) => Promise<void>` |
+| `stop()` | `() => void` |
 | `get frameRate` | `number` |
 | `set frameRate` | `void` |

package/api-docs/functions/createAudioDestinationGraphNode.mdx

@@ -6,4 +6,4 @@ Create an `AudioDestinationNode`
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`AudioDestinationNode`, `AudioDestinationNode`\>
+[`AudioNodeInit<AudioDestinationNode, AudioDestinationNode>`](../interfaces/AudioNodeInit.mdx)

package/api-docs/functions/createAudioSignalDetector.mdx

@@ -18,8 +18,8 @@ lastCheck | silent | should call onSignalDetected
 
 | Parameter | Type |
 | ------ | ------ |
-| `shouldDetect` | () => `boolean` |
-| `onDetected` | (`silent`) => `void` |
+| `shouldDetect` | `() => boolean` |
+| `onDetected` | `(silent) => void` |
 
 ## Returns
 
@@ -31,7 +31,7 @@ lastCheck | silent | should call onSignalDetected
 
 | Parameter | Type |
 | ------ | ------ |
-| `buffer` | `Queue`\<`number`[]\> |
+| `buffer` | `Queue<number[]>` |
 | `threshold?` | `number` |
 
 ### Returns
@@ -44,7 +44,7 @@ lastCheck | silent | should call onSignalDetected
 
 | Parameter | Type |
 | ------ | ------ |
-| `samples` | `number`[] |
+| `samples` | `number[]` |
 
 #### Returns
 

package/api-docs/functions/createAudioStats.mdx

@@ -9,7 +9,7 @@ AudioStats builder
 | Parameter | Type | Description |
 | ------ | ------ | ------ |
 | `stats` | `Partial`\<[`AudioStats`](../interfaces/AudioStats.mdx)\> | overwrite the default attributes |
-| `options` | \{ `silentThreshold?`: `number`; `lowVolumeThreshold?`: `number`; `clipCountThreshold?`: `number`; \} | `silentThreshold`, `lowVolumeThreshold` and `clipCountThreshold` |
+| `options` | `{ silentThreshold?: number; lowVolumeThreshold?: number; clipCountThreshold?: number; }` | `silentThreshold`, `lowVolumeThreshold` and `clipCountThreshold` |
 | `options.silentThreshold?` | `number` | - |
 | `options.lowVolumeThreshold?` | `number` | - |
 | `options.clipCountThreshold?` | `number` | - |

package/api-docs/functions/createChannelMergerGraphNode.mdx

@@ -12,7 +12,7 @@ Create a ChannelMergerNode
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`ChannelMergerNode`, `ChannelMergerNode`\>
+[`AudioNodeInit<ChannelMergerNode, ChannelMergerNode>`](../interfaces/AudioNodeInit.mdx)
 
 ## See
 

package/api-docs/functions/createChannelSplitterGraphNode.mdx

@@ -12,7 +12,7 @@ Create a ChannelSplitterNode
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`ChannelSplitterNode`, `ChannelSplitterNode`\>
+[`AudioNodeInit<ChannelSplitterNode, ChannelSplitterNode>`](../interfaces/AudioNodeInit.mdx)
 
 ## See
 

package/api-docs/functions/createDelayGraphNode.mdx

@@ -12,7 +12,7 @@ Create a `DelayNode`
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`DelayNode`, `DelayNode`\>
+[`AudioNodeInit<DelayNode, DelayNode>`](../interfaces/AudioNodeInit.mdx)
 
 ## See
 

package/api-docs/functions/createDenoiseWorkletGraphNode.mdx

@@ -9,8 +9,8 @@ Create a noise suppression node
 | Parameter | Type | Description |
 | ------ | ------ | ------ |
 | `data` | `BufferSource` | WebAssembly source |
-| `messageHandler?` | (`vads`) => `void` | - |
+| `messageHandler?` | `(vads) => void` | - |
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`AudioWorkletNode`, `AudioWorkletNode`\>
+[`AudioNodeInit<AudioWorkletNode, AudioWorkletNode>`](../interfaces/AudioNodeInit.mdx)

package/api-docs/functions/createFrameCallbackRequest.mdx

@@ -1,5 +1,5 @@
 ```ts
-function createFrameCallbackRequest(
+function createFrameCallbackRequest<T>(
    callback, 
    frameRate, 
    __namedParameters): object;
@@ -9,11 +9,17 @@ Create a callback loop for video frame processing using
 `requestVideoFrameCallback` under-the-hood when available otherwise our
 fallback implementation based on `setTimeout`.
 
+## Type Parameters
+
+| Type Parameter |
+| ------ |
+| `T` |
+
 ## Parameters
 
 | Parameter | Type | Description |
 | ------ | ------ | ------ |
-| `callback` | [`Callback`](../type-aliases/Callback.mdx)\<`Promise`\<`void`\>, \[[`ProcessInputType`](../type-aliases/ProcessInputType.mdx)\]\> | To be called by the loop |
+| `callback` | [`Callback`](../type-aliases/Callback.mdx)\<`Promise<void>`, \[`T`\]\> | To be called by the loop |
 | `frameRate` | `number` | A fallback frame rate when we are not able to get the rate from API |
 | `__namedParameters` | `FrameCallbackRequestOptions` | - |
 
@@ -23,7 +29,7 @@ fallback implementation based on `setTimeout`.
 
 | Name | Type |
 | ------ | ------ |
-| `start()` | (`input`) => `Promise`\<`void`\> |
-| `stop()` | () => `void` |
+| `start()` | `(input) => Promise<void>` |
+| `stop()` | `() => void` |
 | `get frameRate` | `number` |
 | `set frameRate` | `void` |

package/api-docs/functions/createGainGraphNode.mdx

@@ -12,4 +12,6 @@ Create a GainNodeInit
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`GainNode`, [`Gain`](../interfaces/Gain.mdx)\>
+```ts
+AudioNodeInit<GainNode, Gain>
+```

package/api-docs/functions/createMediaElementSourceGraphNode.mdx

@@ -12,4 +12,4 @@ Create a MediaStreamAudioSourceNodeInit
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`MediaElementAudioSourceNode`, `MediaElementAudioSourceNode`\>
+[`AudioNodeInit<MediaElementAudioSourceNode, MediaElementAudioSourceNode>`](../interfaces/AudioNodeInit.mdx)

package/api-docs/functions/createMediaStreamTrackInputProcessAdaptor.mdx

@@ -0,0 +1,10 @@
+```ts
+function createMediaStreamTrackInputProcessAdaptor(): InputProcessAdaptor;
+```
+
+An implementation of `InputProcessAdaptor` which adapts `MediaStreamTrack`
+directly as specified from the spec https://w3c.github.io/mediacapture-transform
+
+## Returns
+
+[`InputProcessAdaptor`](../interfaces/InputProcessAdaptor.mdx)

package/api-docs/functions/createReadWritableStreamFallbackInputProcessAdaptor.mdx

@@ -0,0 +1,17 @@
+```ts
+function createReadWritableStreamFallbackInputProcessAdaptor(__namedParameters): InputProcessAdaptor;
+```
+
+An implementation of `InputProcessAdaptor` which uses `HTMLCanvasElement` or
+`OffscreenCanvas` API to create `ReadableStream` & `WritableStream`
+
+## Parameters
+
+| Parameter | Type | Description |
+| ------ | ------ | ------ |
+| `__namedParameters` | `{ frameRate: number; }` | - |
+| `__namedParameters.frameRate` | `number` | Fallback frame rate |
+
+## Returns
+
+[`InputProcessAdaptor`](../interfaces/InputProcessAdaptor.mdx)

package/api-docs/functions/createReadWritableStreamInputProcessAdaptor.mdx

@@ -0,0 +1,10 @@
+```ts
+function createReadWritableStreamInputProcessAdaptor(): InputProcessAdaptor;
+```
+
+An implementation of `InputProcessAdaptor` which uses `MediaStreamTrackGenerator` and
+`MediaStreamTrackProcessor` APIs to create `ReadableStream` & `WritableStream`
+
+## Returns
+
+[`InputProcessAdaptor`](../interfaces/InputProcessAdaptor.mdx)

package/api-docs/functions/createSegmenter.mdx

@@ -6,7 +6,7 @@ function createSegmenter(basePath, __namedParameters): Segmenter;
 
 | Parameter | Type | Default value | Description |
 | ------ | ------ | ------ | ------ |
-| `basePath` | `string` \| `undefined` | `'/'` | A base path to specify the directory the Wasm files should be loaded from. If not provided, it will be loaded from the host's root directory. **See** [FileResolver](https://ai.google.dev/edge/api/mediapipe/js/tasks-vision.filesetresolver#filesetresolverforvisiontasks) |
+| `basePath` | `string \| undefined` | `'/'` | A base path to specify the directory the Wasm files should be loaded from. If not provided, it will be loaded from the host's root directory. **See** [FileResolver](https://ai.google.dev/edge/api/mediapipe/js/tasks-vision.filesetresolver#filesetresolverforvisiontasks) |
 | `__namedParameters` | `Options` | `...` | - |
 
 ## Returns

package/api-docs/functions/createStreamDestinationGraphNode.mdx

@@ -12,4 +12,4 @@ Create a MediaStreamAudioDestinationNodeInit
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`MediaStreamAudioDestinationNode`, `MediaStreamAudioDestinationNode`\>
+[`AudioNodeInit<MediaStreamAudioDestinationNode, MediaStreamAudioDestinationNode>`](../interfaces/AudioNodeInit.mdx)

package/api-docs/functions/createStreamSourceGraphNode.mdx

@@ -13,4 +13,4 @@ Create a MediaStreamAudioSourceNodeInit
 
 ## Returns
 
-[`AudioNodeInit`](../interfaces/AudioNodeInit.mdx)\<`MediaStreamAudioSourceNode`, `MediaStreamAudioSourceNode`\>
+[`AudioNodeInit<MediaStreamAudioSourceNode, MediaStreamAudioSourceNode>`](../interfaces/AudioNodeInit.mdx)

package/api-docs/functions/createVADetector.mdx

@@ -11,8 +11,8 @@ Create a voice detector based on provided params
 
 | Parameter | Type | Description |
 | ------ | ------ | ------ |
-| `onDetected` | () => `void` | When there is voice activity, this callback will be called |
-| `shouldDetect` | () => `boolean` | When return `true`, voice activity will function, otherwise, not function |
+| `onDetected` | `() => void` | When there is voice activity, this callback will be called |
+| `shouldDetect` | `() => boolean` | When return `true`, voice activity will function, otherwise, not function |
 | `options?` | [`ThrottleOptions`](../interfaces/ThrottleOptions.mdx) |  |
 
 ## Returns
@@ -31,7 +31,7 @@ Create a voice detector based on provided params
 
 | Parameter | Type |
 | ------ | ------ |
-| `isVoice` | [`IsVoice`](../type-aliases/IsVoice.mdx)\<`T`\> |
+| `isVoice` | [`IsVoice<T>`](../type-aliases/IsVoice.mdx) |
 
 ### Returns
 

package/api-docs/functions/createVideoProcessor.mdx

@@ -1,13 +1,13 @@
 ```ts
-function createVideoProcessor(transformers, processTrack): VideoProcessor;
+function createVideoProcessor(segmenter, inputProcessAdaptor): VideoProcessor;
 ```
 
 ## Parameters
 
 | Parameter | Type |
 | ------ | ------ |
-| `transformers` | [`Transform`](../interfaces/Transform.mdx)\<[`InputFrame`](../type-aliases/InputFrame.mdx), [`InputFrame`](../type-aliases/InputFrame.mdx)\>[] |
-| `processTrack` | [`ProcessVideoTrack`](../type-aliases/ProcessVideoTrack.mdx) |
+| `segmenter` | [`Segmenter`](../interfaces/Segmenter.mdx) |
+| `inputProcessAdaptor` | [`InputProcessAdaptor`](../interfaces/InputProcessAdaptor.mdx) |
 
 ## Returns
 

package/api-docs/functions/createVoiceDetectorFromProbability.mdx

@@ -12,4 +12,4 @@ A function to check the provided probability is considered as voice activity
 
 ## Returns
 
-[`IsVoice`](../type-aliases/IsVoice.mdx)\<`number`\>
+[`IsVoice<number>`](../type-aliases/IsVoice.mdx)