@pexip/media 22.1.0 → 22.3.0

package/CHANGELOG.md

@@ -1,5 +1,67 @@
 # @pexip/media
 
+## 22.3.0
+
+### Minor Changes
+
+- 5d1bb40: Add `createMediaSession` to present an ongoing call to the platform's
+  media controls via the Media Session API, and to keep audio capture alive
+  while the page is in the background on mobile. The supported actions are
+  extended with the conferencing specific `hangup`, `togglecamera`,
+  `togglemicrophone`, `togglescreenshare` and `voiceactivity`, along with
+  `enterpictureinpicture`.
+
+  The controller also exposes `setPositionState` and `setScreenshareActive`, and
+  `destroy` is now terminal, so a slower owner cannot resurrect the platform
+  controls after the call has ended.
+
+  The capture state is only reported to platforms that treat it as such. WebKit
+  implements `setMicrophoneActive`, `setCameraActive` and `setScreenshareActive`
+  as commands mutating the real capture state, and rejects activating one
+  outside a user gesture, which would otherwise leave the microphone muted with
+  no way back. Pass `applyImperativeCaptureState` to opt in when every call is
+  made from a user gesture handler.
+
+### Patch Changes
+
+- Updated dependencies [b666166]
+- Updated dependencies [53af4ef]
+- Updated dependencies [2f3c6b7]
+  - @pexip/media-processor@22.3.0
+  - @pexip/media-control@22.3.0
+
+## 22.2.0
+
+### Patch Changes
+
+- 72f8dde: Temporarily remove updating the backend in the pipeline
+- 746f6b6: Add Canvas2d rendering backend support: alpha mask converter,
+  Canvas2d mask renderer, canvas dimension-aware rendering, and proper backend
+  propagation through the video processor pipeline.
+- 265c898: Migrate test runner from Jest to Vitest.
+- a5e8026: Fix "We can't find your microphone" error when changing the
+  microphone while sharing content without audio. The audio mixing processor no
+  longer tries to create an audio source node from a presentation stream that
+  has no audio track, and audio nodes belonging to a stale `AudioContext` are
+  recreated instead of being reused.
+- 0878b3b: Fix race conditions in the video processing pipeline so effect
+  changes made during processor startup are correctly applied and worker option
+  diffs are computed against the previous state.
+- Updated dependencies [5a17991]
+- Updated dependencies [f4af45a]
+- Updated dependencies [746f6b6]
+- Updated dependencies [c101d02]
+- Updated dependencies [476acbf]
+- Updated dependencies [7c99e29]
+- Updated dependencies [1b82fd5]
+- Updated dependencies [265c898]
+- Updated dependencies [a5e8026]
+- Updated dependencies [0878b3b]
+  - @pexip/utils@17.4.1
+  - @pexip/media-processor@22.2.0
+  - @pexip/media-control@22.2.0
+  - @pexip/signal@16.9.6
+
 ## 22.1.0
 
 ### Minor Changes

package/api-docs/README.mdx

@@ -282,6 +282,11 @@ await mediaController.media.applyConstraints({
 
 | Interface | Description |
 | ------ | ------ |
+| [MediaSessionEnterPictureInPictureActionDetails](interfaces/MediaSessionEnterPictureInPictureActionDetails.mdx) | Details passed to the `enterpictureinpicture` action handler, which are not part of the `MediaSessionActionDetails` type shipped with TypeScript's DOM library. |
+| [MediaSessionLike](interfaces/MediaSessionLike.mdx) | The subset of the `navigator.mediaSession` API used by [createMediaSession](functions/createMediaSession.mdx), so that an equivalent implementation can be injected, e.g. in tests. |
+| [MediaSessionMetadataInit](interfaces/MediaSessionMetadataInit.mdx) | Descriptive information presented by the platform's media controls, e.g. the lock screen or the notification shade. |
+| [MediaSessionInit](interfaces/MediaSessionInit.mdx) | Options to create a media session. |
+| [MediaSessionController](interfaces/MediaSessionController.mdx) | Controls an active media session. |
 | [PreviewEventHandler](interfaces/PreviewEventHandler.mdx) | - |
 | [PreviewStreamParams](interfaces/PreviewStreamParams.mdx) | - |
 | [PreviewControllerProps](interfaces/PreviewControllerProps.mdx) | - |
@@ -307,6 +312,8 @@ await mediaController.media.applyConstraints({
 | Type Alias | Description |
 | ------ | ------ |
 | [GetDisplayMedia](type-aliases/GetDisplayMedia.mdx) | - |
+| [ExtendedMediaSessionAction](type-aliases/ExtendedMediaSessionAction.mdx) | Media session actions, including the conferencing specific ones which are not part of the `MediaSessionAction` type shipped with TypeScript's DOM library. |
+| [MediaSessionEnterPictureInPictureReason](type-aliases/MediaSessionEnterPictureInPictureReason.mdx) | Why the user agent asked the page to enter picture in picture. |
 | [PreviewInput](type-aliases/PreviewInput.mdx) | - |
 | [CreatePreviewStreamController](type-aliases/CreatePreviewStreamController.mdx) | - |
 | [GeneratedMediaSignals](type-aliases/GeneratedMediaSignals.mdx) | - |
@@ -373,6 +380,7 @@ await mediaController.media.applyConstraints({
 | [createGetDisplayMedia](functions/createGetDisplayMedia.mdx) | - |
 | [setLogger](functions/setLogger.mdx) | - |
 | [createMedia](functions/createMedia.mdx) | Create an object to interact with the media scream, which is usually used for our main stream. |
+| [createMediaSession](functions/createMediaSession.mdx) | Create a media session, presenting the ongoing call to the platform's media controls, e.g. the lock screen, the notification shade or headset buttons. |
 | [createPreviewStreamController](functions/createPreviewStreamController.mdx) | - |
 | [createMediaSignal](functions/createMediaSignal.mdx) | Create a general signal with consistent scoped name |
 | [createMediaSignals](functions/createMediaSignals.mdx) | Create and return all required and optional (if specified with `more`), signals for media to work |

package/api-docs/functions/createMediaSession.mdx

@@ -0,0 +1,37 @@
+```ts
+function createMediaSession(__namedParameters): MediaSessionController;
+```
+
+Create a media session, presenting the ongoing call to the platform's media
+controls, e.g. the lock screen, the notification shade or headset buttons.
+
+Declaring a media session also hints the browser that the page is playing
+media, which mobile browsers use to relax the throttling applied to
+backgrounded pages and to keep the audio capture alive.
+
+Every method is a no-op when the Media Session API is unavailable, so the
+caller does not need to feature detect it.
+
+## Parameters
+
+| Parameter | Type |
+| ------ | ------ |
+| `__namedParameters` | [`MediaSessionInit`](../interfaces/MediaSessionInit.mdx) |
+
+## Returns
+
+[`MediaSessionController`](../interfaces/MediaSessionController.mdx)
+
+## Example
+
+```ts
+const session = createMediaSession({
+    metadata: {title: 'Weekly sync', artist: 'Pexip'},
+    handlers: {
+        hangup: () => leaveMeeting(),
+        togglemicrophone: () => toggleAudioMuted(),
+    },
+});
+await session.setMicrophoneActive(true);
+session.destroy();
+```

package/api-docs/interfaces/MediaSessionController.mdx

@@ -0,0 +1,50 @@
+Controls an active media session.
+
+## Properties
+
+| Property | Modifier | Type | Description |
+| ------ | ------ | ------ | ------ |
+| <a id="issupported"></a> `isSupported` | `readonly` | `boolean` | Whether the media session API is available in the current environment. Every other method is a no-op when it is `false`. |
+| <a id="setmetadata"></a> `setMetadata` | `public` | `(metadata?) => void` | Update, or clear when omitted, the presented metadata. |
+| <a id="setplaybackstate"></a> `setPlaybackState` | `public` | `(playbackState) => void` | Update the presented playback state. |
+| <a id="setmicrophoneactive"></a> `setMicrophoneActive` | `public` | `(active) => Promise<void>` | Update whether the platform should present the microphone as active. A no-op on platforms that implement this as a command mutating the real tracks, unless `applyImperativeCaptureState` was set. |
+| <a id="setcameraactive"></a> `setCameraActive` | `public` | `(active) => Promise<void>` | Update whether the platform should present the camera as active, with the same caveat as [MediaSessionController.setMicrophoneActive](#setmicrophoneactive). |
+| <a id="setscreenshareactive"></a> `setScreenshareActive` | `public` | `(active) => Promise<void>` | Update whether the platform should present the screen as shared, with the same caveat as [MediaSessionController.setMicrophoneActive](#setmicrophoneactive). Only implemented by WebKit so far. |
+| <a id="setpositionstate"></a> `setPositionState` | `public` | `(state?) => void` | Update, or reset when omitted, the presented playback position. Pass `duration: Infinity` for a live stream such as a call, which makes the platform present an elapsed clock rather than a seekable scrub bar. States the user agent rejects are logged and otherwise ignored. |
+| <a id="destroy"></a> `destroy` | `public` | `() => void` | Unregister every handler registered by this session and reset the metadata, the playback state and the position state. Terminal and idempotent: every other method becomes a no-op afterwards, so a slower owner cannot resurrect the platform's media controls. |
+
+## Methods
+
+### registerActions()
+
+```ts
+registerActions(handlers): void;
+```
+
+#### Parameters
+
+| Parameter | Type |
+| ------ | ------ |
+| `handlers` | `Partial`\<`Record`\<[`ExtendedMediaSessionAction`](../type-aliases/ExtendedMediaSessionAction.mdx), `MediaSessionActionHandler`\>\> |
+
+#### Returns
+
+`void`
+
+***
+
+### deregisterActions()
+
+```ts
+deregisterActions(actions): void;
+```
+
+#### Parameters
+
+| Parameter | Type |
+| ------ | ------ |
+| `actions` | [`ExtendedMediaSessionAction`](../type-aliases/ExtendedMediaSessionAction.mdx)[] |
+
+#### Returns
+
+`void`

package/api-docs/interfaces/MediaSessionEnterPictureInPictureActionDetails.mdx

@@ -0,0 +1,17 @@
+Details passed to the `enterpictureinpicture` action handler, which are not
+part of the `MediaSessionActionDetails` type shipped with TypeScript's DOM
+library.
+
+## Extends
+
+- `MediaSessionActionDetails`
+
+## Properties
+
+| Property | Type | Inherited from |
+| ------ | ------ | ------ |
+| <a id="enterpictureinpicturereason"></a> `enterPictureInPictureReason?` | [`MediaSessionEnterPictureInPictureReason`](../type-aliases/MediaSessionEnterPictureInPictureReason.mdx) | - |
+| <a id="action"></a> `action` | `MediaSessionAction` | `MediaSessionActionDetails.action` |
+| <a id="fastseek"></a> `fastSeek?` | `boolean` | `MediaSessionActionDetails.fastSeek` |
+| <a id="seekoffset"></a> `seekOffset?` | `number` | `MediaSessionActionDetails.seekOffset` |
+| <a id="seektime"></a> `seekTime?` | `number` | `MediaSessionActionDetails.seekTime` |

package/api-docs/interfaces/MediaSessionInit.mdx

@@ -0,0 +1,13 @@
+Options to create a media session.
+
+## Properties
+
+| Property | Type | Description |
+| ------ | ------ | ------ |
+| <a id="handlers"></a> `handlers?` | `Partial`\<`Record`\<[`ExtendedMediaSessionAction`](../type-aliases/ExtendedMediaSessionAction.mdx), `MediaSessionActionHandler`\>\> | Handlers to register, keyed by the action they handle. Actions without a handler are left untouched, allowing the platform to fall back to its default behavior. |
+| <a id="metadata"></a> `metadata?` | [`MediaSessionMetadataInit`](MediaSessionMetadataInit.mdx) | Initial metadata to present. |
+| <a id="playbackstate"></a> `playbackState?` | `MediaSessionPlaybackState` | Initial playback state, defaults to `playing`. |
+| <a id="positionstate"></a> `positionState?` | `MediaPositionState` | Initial playback position to present. |
+| <a id="applyimperativecapturestate"></a> `applyImperativeCaptureState?` | `boolean` | Apply the capture state even when the platform implements it as a command that mutes the real tracks, defaults to `false`. Only enable this when every call is made from a user gesture handler, otherwise unmuting is rejected and the capture stays muted. |
+| <a id="mediasession"></a> `mediaSession?` | [`MediaSessionLike`](MediaSessionLike.mdx) | The media session implementation to use, defaults to `navigator.mediaSession` when available. |
+| <a id="createmetadata"></a> `createMetadata?` | `(init) => MediaMetadata` | Factory building the metadata object, defaults to the `MediaMetadata` constructor when available. |

package/api-docs/interfaces/MediaSessionLike.mdx

@@ -0,0 +1,122 @@
+The subset of the `navigator.mediaSession` API used by
+[createMediaSession](../functions/createMediaSession.mdx), so that an equivalent implementation can be
+injected, e.g. in tests.
+
+## Properties
+
+| Property | Type | Description |
+| ------ | ------ | ------ |
+| <a id="metadata"></a> `metadata` | `MediaMetadata \| null` | Descriptive information about the currently played media. |
+| <a id="playbackstate"></a> `playbackState` | `MediaSessionPlaybackState` | Whether the media session is currently playing or paused. |
+
+## Methods
+
+### setActionHandler()
+
+```ts
+setActionHandler(action, handler): void;
+```
+
+Register, or unregister when the handler is `null`, a handler for the
+given media session action.
+
+#### Parameters
+
+| Parameter | Type |
+| ------ | ------ |
+| `action` | [`ExtendedMediaSessionAction`](../type-aliases/ExtendedMediaSessionAction.mdx) |
+| `handler` | `MediaSessionActionHandler \| null` |
+
+#### Returns
+
+`void`
+
+***
+
+### setCameraActive()?
+
+```ts
+optional setCameraActive(active): Promise<void>;
+```
+
+Tell the user agent whether the camera is considered to be active.
+
+P.S. Safari's impl. is an imperative action to change the track's
+muted stated whereas Chromium-based browsers indicate to the user
+agent the microphone capture state desired by the page
+
+#### Parameters
+
+| Parameter | Type |
+| ------ | ------ |
+| `active` | `boolean` |
+
+#### Returns
+
+`Promise<void>`
+
+***
+
+### setMicrophoneActive()?
+
+```ts
+optional setMicrophoneActive(active): Promise<void>;
+```
+
+Tell the user agent whether the microphone is considered to be active.
+
+P.S. Safari's impl. is an imperative action to change the track's
+muted stated whereas Chromium-based browsers indicate to the user
+agent the microphone capture state desired by the page
+
+#### Parameters
+
+| Parameter | Type |
+| ------ | ------ |
+| `active` | `boolean` |
+
+#### Returns
+
+`Promise<void>`
+
+***
+
+### setScreenshareActive()?
+
+```ts
+optional setScreenshareActive(active): Promise<void>;
+```
+
+Tell the user agent whether the screen is considered to be shared. Only
+specified so far, no browser implements it yet.
+
+#### Parameters
+
+| Parameter | Type |
+| ------ | ------ |
+| `active` | `boolean` |
+
+#### Returns
+
+`Promise<void>`
+
+***
+
+### setPositionState()?
+
+```ts
+optional setPositionState(state?): void;
+```
+
+Present the playback position, or reset it when called without a state.
+Throws a `TypeError` for a state the user agent considers invalid.
+
+#### Parameters
+
+| Parameter | Type |
+| ------ | ------ |
+| `state?` | `MediaPositionState` |
+
+#### Returns
+
+`void`

package/api-docs/interfaces/MediaSessionMetadataInit.mdx

@@ -0,0 +1,11 @@
+Descriptive information presented by the platform's media controls, e.g. the
+lock screen or the notification shade.
+
+## Properties
+
+| Property | Type | Description |
+| ------ | ------ | ------ |
+| <a id="title"></a> `title` | `string` | Main label, e.g. the name of the meeting. |
+| <a id="artist"></a> `artist?` | `string` | Secondary label, e.g. the name of the brand. |
+| <a id="album"></a> `album?` | `string` | Tertiary label. |
+| <a id="artwork"></a> `artwork?` | `MediaImage[]` | Images associated with the session, e.g. the brand logo. |

package/api-docs/type-aliases/ExtendedMediaSessionAction.mdx

@@ -0,0 +1,28 @@
+```ts
+type ExtendedMediaSessionAction = 
+  | MediaSessionAction
+  | "enterpictureinpicture"
+  | "hangup"
+  | "togglecamera"
+  | "togglemicrophone"
+  | "togglescreenshare"
+  | "voiceactivity";
+```
+
+Media session actions, including the conferencing specific ones which are
+not part of the `MediaSessionAction` type shipped with TypeScript's DOM
+library.
+
+`hangup`, `togglecamera`, `togglemicrophone` and `enterpictureinpicture` are
+implemented by Chromium based browsers. `togglescreenshare` and
+`voiceactivity` are only specified so far, and are registered as a no-op by
+every current browser, which [createMediaSession](../functions/createMediaSession.mdx) handles gracefully.
+
+Note that `voiceactivity` runs the other way around to the rest: the user
+agent invokes the handler when it detects voice on a live microphone track,
+e.g. to let a page nudge a muted user who is talking.
+
+## See
+
+ - [https://w3c.github.io/mediasession/#enumdef-mediasessionaction](https://w3c.github.io/mediasession/#enumdef-mediasessionaction)
+ - [https://developer.mozilla.org/en-US/docs/Web/API/MediaSession/setActionHandler](https://developer.mozilla.org/en-US/docs/Web/API/MediaSession/setActionHandler)

package/api-docs/type-aliases/MediaSessionEnterPictureInPictureReason.mdx

@@ -0,0 +1,9 @@
+```ts
+type MediaSessionEnterPictureInPictureReason = "other" | "useraction" | "contentoccluded";
+```
+
+Why the user agent asked the page to enter picture in picture.
+
+## See
+
+[https://developer.mozilla.org/en-US/docs/Web/API/MediaSession/setActionHandler](https://developer.mozilla.org/en-US/docs/Web/API/MediaSession/setActionHandler)

package/dist/audioMixingProcessor.d.ts

@@ -5,6 +5,7 @@ interface AudioStreamProcessorProps {
     mixWithAdditionalMedia?: boolean;
     merger?: AudioNodeInit<ChannelMergerNode, ChannelMergerNode>;
     displaySource?: AudioNodeInit<MediaStreamAudioSourceNode, MediaStreamAudioSourceNode>;
+    displayStream?: MediaStream;
     audioGraph?: AudioGraph;
 }
 declare const FEATURE_KEYS: {
@@ -13,6 +14,20 @@ declare const FEATURE_KEYS: {
 type FeaturePropKeys = keyof typeof FEATURE_KEYS;
 type FeatureProps = Pick<AudioStreamProcessorProps, FeaturePropKeys>;
 export declare const updateFeatureProps: (constraints: MediaDeviceRequest["audio"], props: FeatureProps) => FeatureProps;
+/**
+ * Extract a `MediaStream` containing only the audio tracks of the provided
+ * stream, or `undefined` when there is no audio track to mix, e.g. sharing a
+ * screen without audio.
+ *
+ * @param stream - The stream to extract the audio tracks from
+ *
+ * @example
+ *
+ * ```typescript
+ * const audioOnly = toAudioOnlyStream(displayStream);
+ * ```
+ */
+export declare const toAudioOnlyStream: (stream?: MediaStream) => MediaStream | undefined;
 /**
  * Create an Audio Mixing Processor and will own the stream passed-in
  */

package/dist/audioMixingProcessor.js

@@ -28,6 +28,26 @@ export const updateFeatureProps = (constraints, props) => {
     }
     return features;
 };
+/**
+ * Extract a `MediaStream` containing only the audio tracks of the provided
+ * stream, or `undefined` when there is no audio track to mix, e.g. sharing a
+ * screen without audio.
+ *
+ * @param stream - The stream to extract the audio tracks from
+ *
+ * @example
+ *
+ * ```typescript
+ * const audioOnly = toAudioOnlyStream(displayStream);
+ * ```
+ */
+export const toAudioOnlyStream = (stream) => {
+    const audioTracks = stream?.getAudioTracks?.() ?? [];
+    if (audioTracks.length === 0) {
+        return undefined;
+    }
+    return new MediaStream(audioTracks);
+};
 /**
  * Create an Audio Mixing Processor and will own the stream passed-in
  */
@@ -36,7 +56,8 @@ export const createAudioMixingProcess = (getCurrentMedia, signals, label = PROCE
     return async (prevMediaTrack) => {
         updateFeatureProps(prevMediaTrack.getConstraints(), props);
         const displayStream = getCurrentMedia();
-        const displayTrack = displayStream?.getAudioTracks?.().at(0);
+        const displayAudioStream = toAudioOnlyStream(displayStream);
+        const displayTrack = displayAudioStream?.getAudioTracks().at(0);
         const applyConstraints = async (constraints) => {
             if (isEmpty(constraints)) {
                 return;
@@ -50,8 +71,8 @@ export const createAudioMixingProcess = (getCurrentMedia, signals, label = PROCE
             }
             if (features.mixWithAdditionalMedia) {
                 const newStream = getCurrentMedia();
-                const [newTrack] = getCurrentMedia()?.getAudioTracks() ?? [];
-                if (!newTrack) {
+                const newAudioStream = toAudioOnlyStream(newStream);
+                if (!newAudioStream) {
                     return;
                 }
                 if (!props.audioGraph || !props.merger) {
@@ -59,8 +80,7 @@ export const createAudioMixingProcess = (getCurrentMedia, signals, label = PROCE
                     return;
                 }
                 if (props.displaySource && props.merger) {
-                    if (newStream ===
-                        props.displaySource.audioNode?.mediaStream) {
+                    if (newStream === props.displayStream) {
                         return;
                     }
                     props.audioGraph.disconnect([
@@ -68,14 +88,13 @@ export const createAudioMixingProcess = (getCurrentMedia, signals, label = PROCE
                         props.merger,
                     ]);
                 }
-                if (newStream) {
-                    props.displaySource =
-                        createStreamSourceGraphNode(newStream);
-                    props.audioGraph.connect([
-                        props.displaySource,
-                        props.merger,
-                    ]);
-                }
+                props.displayStream = newStream;
+                props.displaySource =
+                    createStreamSourceGraphNode(newAudioStream);
+                props.audioGraph.connect([
+                    props.displaySource,
+                    props.merger,
+                ]);
             }
             else if (features.mixWithAdditionalMedia === false) {
                 // Unmixing
@@ -83,6 +102,7 @@ export const createAudioMixingProcess = (getCurrentMedia, signals, label = PROCE
                     props.displaySource.release();
                     props.audioGraph?.disconnect([props.displaySource]);
                     props.displaySource = undefined;
+                    props.displayStream = undefined;
                 }
             }
             return Promise.resolve();
@@ -118,8 +138,10 @@ export const createAudioMixingProcess = (getCurrentMedia, signals, label = PROCE
         }
         assert(prevMediaTrack.track, 'Main track should be available');
         const mainSource = createStreamSourceGraphNode(new MediaStream([prevMediaTrack.track]));
-        props.displaySource =
-            displayStream && createStreamSourceGraphNode(displayStream);
+        props.displaySource = displayAudioStream
+            ? createStreamSourceGraphNode(displayAudioStream)
+            : undefined;
+        props.displayStream = displayAudioStream ? displayStream : undefined;
         props.merger = createChannelMergerGraphNode();
         const destination = createStreamDestinationGraphNode({
             channelCount: 1,
@@ -151,10 +173,13 @@ export const createAudioMixingProcess = (getCurrentMedia, signals, label = PROCE
             logger.debug({ label }, 'Release Media');
             unsubscribe();
             // Release Props
-            await props.audioGraph?.release();
-            props.audioGraph = undefined;
-            props.merger = undefined;
-            props.displaySource = undefined;
+            await audioGraph.release();
+            if (props.audioGraph === audioGraph) {
+                props.audioGraph = undefined;
+                props.merger = undefined;
+                props.displaySource = undefined;
+                props.displayStream = undefined;
+            }
         };
         return Promise.resolve(createMediaTrack({
             label,

package/dist/audioProcessor.js

@@ -173,6 +173,14 @@ label = PROCESSOR_LABELS.AudioProcessor, }) => {
             [source, analyzer],
         ];
         logger.debug({ initialAudioNodeConnection, label }, 'Initial AudioNodeConnection');
+        if (props.denoiseNode) {
+            props.denoiseNode.release();
+            props.denoiseNode = undefined;
+        }
+        if (props.analyzer) {
+            props.analyzer.release();
+            props.analyzer = undefined;
+        }
         const audioGraph = createAudioGraphProxy(createAudioGraph(initialAudioNodeConnection, props.audioGraphOptions), {
             connect: (target, args) => {
                 logger.debug({ label, target, args }, 'connect nodes');
@@ -199,9 +207,11 @@ label = PROCESSOR_LABELS.AudioProcessor, }) => {
             track.stop();
             // Release Props
             await audioGraph.release();
-            props.denoiseNode = undefined;
-            props.analyzer = undefined;
-            props.audioGraph = undefined;
+            if (props.audioGraph === audioGraph) {
+                props.denoiseNode = undefined;
+                props.analyzer = undefined;
+                props.audioGraph = undefined;
+            }
         };
         return createMediaTrack({
             label,

package/dist/index.d.ts

@@ -1,4 +1,5 @@
 export * from './displayMedia';
+export * from './mediaSession';
 export * from './previewController';
 export * from './signals';
 export * from './status';

package/dist/index.js

@@ -1,4 +1,5 @@
 export * from './displayMedia';
+export * from './mediaSession';
 export * from './previewController';
 export * from './signals';
 export * from './status';

package/dist/mediaSession.d.ts

@@ -0,0 +1,221 @@
+/**
+ * Media session actions, including the conferencing specific ones which are
+ * not part of the `MediaSessionAction` type shipped with TypeScript's DOM
+ * library.
+ *
+ * `hangup`, `togglecamera`, `togglemicrophone` and `enterpictureinpicture` are
+ * implemented by Chromium based browsers. `togglescreenshare` and
+ * `voiceactivity` are only specified so far, and are registered as a no-op by
+ * every current browser, which {@link createMediaSession} handles gracefully.
+ *
+ * Note that `voiceactivity` runs the other way around to the rest: the user
+ * agent invokes the handler when it detects voice on a live microphone track,
+ * e.g. to let a page nudge a muted user who is talking.
+ *
+ * @see {@link https://w3c.github.io/mediasession/#enumdef-mediasessionaction}
+ * @see {@link https://developer.mozilla.org/en-US/docs/Web/API/MediaSession/setActionHandler}
+ */
+export type ExtendedMediaSessionAction = MediaSessionAction | 'enterpictureinpicture' | 'hangup' | 'togglecamera' | 'togglemicrophone' | 'togglescreenshare' | 'voiceactivity';
+/**
+ * Why the user agent asked the page to enter picture in picture.
+ *
+ * @see {@link https://developer.mozilla.org/en-US/docs/Web/API/MediaSession/setActionHandler}
+ */
+export type MediaSessionEnterPictureInPictureReason = 'other' | 'useraction' | 'contentoccluded';
+/**
+ * Details passed to the `enterpictureinpicture` action handler, which are not
+ * part of the `MediaSessionActionDetails` type shipped with TypeScript's DOM
+ * library.
+ */
+export interface MediaSessionEnterPictureInPictureActionDetails extends MediaSessionActionDetails {
+    enterPictureInPictureReason?: MediaSessionEnterPictureInPictureReason;
+}
+/**
+ * The subset of the `navigator.mediaSession` API used by
+ * {@link createMediaSession}, so that an equivalent implementation can be
+ * injected, e.g. in tests.
+ */
+export interface MediaSessionLike {
+    /**
+     * Descriptive information about the currently played media.
+     */
+    metadata: MediaMetadata | null;
+    /**
+     * Whether the media session is currently playing or paused.
+     */
+    playbackState: MediaSessionPlaybackState;
+    /**
+     * Register, or unregister when the handler is `null`, a handler for the
+     * given media session action.
+     */
+    setActionHandler(action: ExtendedMediaSessionAction, handler: MediaSessionActionHandler | null): void;
+    /**
+     * Tell the user agent whether the camera is considered to be active.
+     *
+     * P.S. Safari's impl. is an imperative action to change the track's
+     * muted stated whereas Chromium-based browsers indicate to the user
+     * agent the microphone capture state desired by the page
+     */
+    setCameraActive?(active: boolean): Promise<void>;
+    /**
+     * Tell the user agent whether the microphone is considered to be active.
+     *
+     * P.S. Safari's impl. is an imperative action to change the track's
+     * muted stated whereas Chromium-based browsers indicate to the user
+     * agent the microphone capture state desired by the page
+     */
+    setMicrophoneActive?(active: boolean): Promise<void>;
+    /**
+     * Tell the user agent whether the screen is considered to be shared. Only
+     * specified so far, no browser implements it yet.
+     */
+    setScreenshareActive?(active: boolean): Promise<void>;
+    /**
+     * Present the playback position, or reset it when called without a state.
+     * Throws a `TypeError` for a state the user agent considers invalid.
+     */
+    setPositionState?(state?: MediaPositionState): void;
+}
+/**
+ * Descriptive information presented by the platform's media controls, e.g. the
+ * lock screen or the notification shade.
+ */
+export interface MediaSessionMetadataInit {
+    /**
+     * Main label, e.g. the name of the meeting.
+     */
+    title: string;
+    /**
+     * Secondary label, e.g. the name of the brand.
+     */
+    artist?: string;
+    /**
+     * Tertiary label.
+     */
+    album?: string;
+    /**
+     * Images associated with the session, e.g. the brand logo.
+     */
+    artwork?: MediaImage[];
+}
+/**
+ * Options to create a media session.
+ */
+export interface MediaSessionInit {
+    /**
+     * Handlers to register, keyed by the action they handle. Actions without a
+     * handler are left untouched, allowing the platform to fall back to its
+     * default behavior.
+     */
+    handlers?: Partial<Record<ExtendedMediaSessionAction, MediaSessionActionHandler>>;
+    /**
+     * Initial metadata to present.
+     */
+    metadata?: MediaSessionMetadataInit;
+    /**
+     * Initial playback state, defaults to `playing`.
+     */
+    playbackState?: MediaSessionPlaybackState;
+    /**
+     * Initial playback position to present.
+     */
+    positionState?: MediaPositionState;
+    /**
+     * Apply the capture state even when the platform implements it as a
+     * command that mutes the real tracks, defaults to `false`.
+     *
+     * Only enable this when every call is made from a user gesture handler,
+     * otherwise unmuting is rejected and the capture stays muted.
+     */
+    applyImperativeCaptureState?: boolean;
+    /**
+     * The media session implementation to use, defaults to
+     * `navigator.mediaSession` when available.
+     */
+    mediaSession?: MediaSessionLike;
+    /**
+     * Factory building the metadata object, defaults to the `MediaMetadata`
+     * constructor when available.
+     */
+    createMetadata?: (init: MediaSessionMetadataInit) => MediaMetadata;
+}
+/**
+ * Controls an active media session.
+ */
+export interface MediaSessionController {
+    /**
+     * Whether the media session API is available in the current environment.
+     * Every other method is a no-op when it is `false`.
+     */
+    readonly isSupported: boolean;
+    /**
+     * Update, or clear when omitted, the presented metadata.
+     */
+    setMetadata: (metadata?: MediaSessionMetadataInit) => void;
+    /**
+     * Update the presented playback state.
+     */
+    setPlaybackState: (playbackState: MediaSessionPlaybackState) => void;
+    /**
+     * Update whether the platform should present the microphone as active.
+     *
+     * A no-op on platforms that implement this as a command mutating the real
+     * tracks, unless `applyImperativeCaptureState` was set.
+     */
+    setMicrophoneActive: (active: boolean) => Promise<void>;
+    /**
+     * Update whether the platform should present the camera as active, with
+     * the same caveat as {@link MediaSessionController.setMicrophoneActive}.
+     */
+    setCameraActive: (active: boolean) => Promise<void>;
+    /**
+     * Update whether the platform should present the screen as shared, with
+     * the same caveat as {@link MediaSessionController.setMicrophoneActive}.
+     *
+     * Only implemented by WebKit so far.
+     */
+    setScreenshareActive: (active: boolean) => Promise<void>;
+    /**
+     * Update, or reset when omitted, the presented playback position.
+     *
+     * Pass `duration: Infinity` for a live stream such as a call, which makes
+     * the platform present an elapsed clock rather than a seekable scrub bar.
+     * States the user agent rejects are logged and otherwise ignored.
+     */
+    setPositionState: (state?: MediaPositionState) => void;
+    registerActions(handlers: Partial<Record<ExtendedMediaSessionAction, MediaSessionActionHandler>>): void;
+    deregisterActions(actions: ExtendedMediaSessionAction[]): void;
+    /**
+     * Unregister every handler registered by this session and reset the
+     * metadata, the playback state and the position state.
+     *
+     * Terminal and idempotent: every other method becomes a no-op afterwards,
+     * so a slower owner cannot resurrect the platform's media controls.
+     */
+    destroy: () => void;
+}
+/**
+ * Create a media session, presenting the ongoing call to the platform's media
+ * controls, e.g. the lock screen, the notification shade or headset buttons.
+ *
+ * Declaring a media session also hints the browser that the page is playing
+ * media, which mobile browsers use to relax the throttling applied to
+ * backgrounded pages and to keep the audio capture alive.
+ *
+ * Every method is a no-op when the Media Session API is unavailable, so the
+ * caller does not need to feature detect it.
+ *
+ * @example
+ * ```ts
+ * const session = createMediaSession({
+ *     metadata: {title: 'Weekly sync', artist: 'Pexip'},
+ *     handlers: {
+ *         hangup: () => leaveMeeting(),
+ *         togglemicrophone: () => toggleAudioMuted(),
+ *     },
+ * });
+ * await session.setMicrophoneActive(true);
+ * session.destroy();
+ * ```
+ */
+export declare const createMediaSession: ({ handlers, metadata, playbackState, positionState, applyImperativeCaptureState, mediaSession, createMetadata, }?: MediaSessionInit) => MediaSessionController;

package/dist/mediaSession.js

@@ -0,0 +1,213 @@
+import { logger } from './logger';
+const getDefaultMediaSession = () => typeof navigator !== 'undefined' && 'mediaSession' in navigator
+    ? navigator.mediaSession
+    : undefined;
+const getDefaultCreateMetadata = () => typeof MediaMetadata === 'function'
+    ? init => new MediaMetadata(init)
+    : undefined;
+/**
+ * Whether the platform treats the capture state setters as a command rather
+ * than as a report.
+ *
+ * Chromium's `setMicrophoneActive`/`setCameraActive` are purely declarative:
+ * they only choose which glyph the media control draws. WebKit's are
+ * imperative, and route through `UserMediaController::updateCaptureState` to
+ * the page level capture muted state, which mutes the real tracks. Activating
+ * capture there additionally requires a visible document and a user gesture,
+ * and can raise a permission prompt.
+ *
+ * That asymmetry is a trap for a state driven caller: muting always succeeds,
+ * while the matching unmute is rejected outside a gesture, leaving the
+ * microphone muted at the page level with no way back from a signal handler.
+ *
+ * `setScreenshareActive` distinguishes the two, as only WebKit implements it.
+ * WebKit exposes all three under a single `MediaSessionCaptureToggleAPIEnabled`
+ * setting, so they appear and disappear together, which makes the probe
+ * reliable: when the setting is off none of them exist and there is nothing to
+ * call, and Chromium only ever exposes the microphone and the camera ones.
+ */
+const hasImperativeCaptureState = (mediaSession) => typeof mediaSession.setScreenshareActive === 'function';
+const createUnsupportedMediaSession = () => ({
+    isSupported: false,
+    setMetadata: () => undefined,
+    setPlaybackState: () => undefined,
+    setMicrophoneActive: () => Promise.resolve(),
+    setCameraActive: () => Promise.resolve(),
+    setScreenshareActive: () => Promise.resolve(),
+    setPositionState: () => undefined,
+    registerActions: () => undefined,
+    deregisterActions: () => undefined,
+    destroy: () => undefined,
+});
+/**
+ * Create a media session, presenting the ongoing call to the platform's media
+ * controls, e.g. the lock screen, the notification shade or headset buttons.
+ *
+ * Declaring a media session also hints the browser that the page is playing
+ * media, which mobile browsers use to relax the throttling applied to
+ * backgrounded pages and to keep the audio capture alive.
+ *
+ * Every method is a no-op when the Media Session API is unavailable, so the
+ * caller does not need to feature detect it.
+ *
+ * @example
+ * ```ts
+ * const session = createMediaSession({
+ *     metadata: {title: 'Weekly sync', artist: 'Pexip'},
+ *     handlers: {
+ *         hangup: () => leaveMeeting(),
+ *         togglemicrophone: () => toggleAudioMuted(),
+ *     },
+ * });
+ * await session.setMicrophoneActive(true);
+ * session.destroy();
+ * ```
+ */
+export const createMediaSession = ({ handlers = {}, metadata, playbackState = 'playing', positionState, applyImperativeCaptureState = false, mediaSession = getDefaultMediaSession(), createMetadata = getDefaultCreateMetadata(), } = {}) => {
+    if (!mediaSession) {
+        logger.debug({ context: 'MediaSession' }, 'Media Session API is not supported');
+        return createUnsupportedMediaSession();
+    }
+    const registeredActions = new Set();
+    let destroyed = false;
+    const skipCaptureState = !applyImperativeCaptureState && hasImperativeCaptureState(mediaSession);
+    if (skipCaptureState) {
+        logger.debug({ context: 'MediaSession' }, 'Skipping the media session capture state, the platform applies it to the real tracks');
+    }
+    const setActionHandler = (action, handler) => {
+        try {
+            mediaSession.setActionHandler(action, handler);
+            return true;
+        }
+        catch (error) {
+            logger.debug({ context: 'MediaSession', action, error }, 'Media session action is not supported');
+            return false;
+        }
+    };
+    const setMetadata = (updatedMetadata) => {
+        if (destroyed) {
+            return;
+        }
+        if (!updatedMetadata || !createMetadata) {
+            mediaSession.metadata = null;
+            return;
+        }
+        mediaSession.metadata = createMetadata(updatedMetadata);
+    };
+    const setPlaybackState = (updatedPlaybackState) => {
+        if (destroyed) {
+            return;
+        }
+        mediaSession.playbackState = updatedPlaybackState;
+    };
+    const setMicrophoneActive = async (active) => {
+        if (destroyed ||
+            skipCaptureState ||
+            !mediaSession.setMicrophoneActive) {
+            return;
+        }
+        try {
+            await mediaSession.setMicrophoneActive(active);
+        }
+        catch (error) {
+            logger.debug({ context: 'MediaSession', active, error }, 'Failed to update the media session microphone state');
+        }
+    };
+    const setCameraActive = async (active) => {
+        if (destroyed || skipCaptureState || !mediaSession.setCameraActive) {
+            return;
+        }
+        try {
+            await mediaSession.setCameraActive(active);
+        }
+        catch (error) {
+            logger.debug({ context: 'MediaSession', active, error }, 'Failed to update the media session camera state');
+        }
+    };
+    const setScreenshareActive = async (active) => {
+        if (destroyed ||
+            skipCaptureState ||
+            !mediaSession.setScreenshareActive) {
+            return;
+        }
+        try {
+            await mediaSession.setScreenshareActive(active);
+        }
+        catch (error) {
+            logger.debug({ context: 'MediaSession', active, error }, 'Failed to update the media session screenshare state');
+        }
+    };
+    const setPositionState = (state) => {
+        if (destroyed || !mediaSession.setPositionState) {
+            return;
+        }
+        try {
+            // Blink rejects a missing, `NaN` or negative duration, a negative
+            // position, a position past the duration and a zero playback rate.
+            mediaSession.setPositionState(state);
+        }
+        catch (error) {
+            logger.debug({ context: 'MediaSession', state, error }, 'Failed to update the media session position state');
+        }
+    };
+    const destroy = () => {
+        if (destroyed) {
+            return;
+        }
+        for (const action of registeredActions) {
+            setActionHandler(action, null);
+        }
+        registeredActions.clear();
+        setMetadata();
+        setPlaybackState('none');
+        setPositionState();
+        // The session is a view onto the global `navigator.mediaSession`, which
+        // outlives it, so every mutator has to stop writing to it once the
+        // session is gone. Owners tear down on different clocks, e.g. the view
+        // layer only unbinds its media element listeners on the next React
+        // passive cleanup, well after the service has ended the call.
+        destroyed = true;
+    };
+    const registerActions = handlers => {
+        if (destroyed) {
+            return;
+        }
+        for (const [action, handler] of Object.entries(handlers)) {
+            if (!handler) {
+                continue;
+            }
+            if (setActionHandler(action, handler)) {
+                registeredActions.add(action);
+            }
+        }
+    };
+    const deregisterActions = actions => {
+        if (destroyed) {
+            return;
+        }
+        for (const action of actions) {
+            if (registeredActions.delete(action)) {
+                setActionHandler(action, null);
+            }
+        }
+    };
+    // Register initial actions
+    registerActions(handlers);
+    setMetadata(metadata);
+    mediaSession.playbackState = playbackState;
+    if (positionState) {
+        setPositionState(positionState);
+    }
+    return {
+        isSupported: true,
+        setMetadata,
+        setPlaybackState,
+        setMicrophoneActive,
+        setCameraActive,
+        setScreenshareActive,
+        setPositionState,
+        registerActions,
+        deregisterActions,
+        destroy,
+    };
+};

package/dist/userMedia.d.ts

@@ -11,10 +11,7 @@ export declare const toSameDeviceStatus: ({ audio, video, }: {
     audio: boolean;
     video: boolean;
 }) => UserMediaStatus.PermissionsGranted | UserMediaStatus.PermissionsGrantedFallback | UserMediaStatus.PermissionsGrantedFallbackAudioinput | UserMediaStatus.PermissionsGrantedFallbackVideoinput;
-export declare const toOnlyDeviceStatus: (kind: "audioinput" | "videoinput", streamedRequestedDevices: {
-    audio: boolean;
-    video: boolean;
-}, devices: IndexedDevices) => UserMediaStatus.PermissionsRejectedAudioInput | UserMediaStatus.PermissionsRejectedVideoInput | UserMediaStatus.PermissionsOnlyAudioinput | UserMediaStatus.PermissionsOnlyAudioinputNoVideoDevices | UserMediaStatus.PermissionsOnlyAudioinputFallback | UserMediaStatus.PermissionsOnlyAudioinputFallbackNoVideoDevices | UserMediaStatus.PermissionsOnlyVideoinput | UserMediaStatus.PermissionsOnlyVideoinputNoAudioDevices | UserMediaStatus.PermissionsOnlyVideoinputFallback | UserMediaStatus.PermissionsOnlyVideoinputFallbackNoAudioDevices;
+export declare const toOnlyDeviceStatus: (kind: "audioinput" | "videoinput", matched: boolean, devices: IndexedDevices) => UserMediaStatus.PermissionsRejectedAudioInput | UserMediaStatus.PermissionsRejectedVideoInput | UserMediaStatus.PermissionsOnlyAudioinput | UserMediaStatus.PermissionsOnlyAudioinputNoVideoDevices | UserMediaStatus.PermissionsOnlyAudioinputFallback | UserMediaStatus.PermissionsOnlyAudioinputFallbackNoVideoDevices | UserMediaStatus.PermissionsOnlyVideoinput | UserMediaStatus.PermissionsOnlyVideoinputNoAudioDevices | UserMediaStatus.PermissionsOnlyVideoinputFallback | UserMediaStatus.PermissionsOnlyVideoinputFallbackNoAudioDevices;
 /**
  * Check if there is any track match with the `kind`
  */

package/dist/userMedia.js

@@ -71,16 +71,13 @@ export const toSameDeviceStatus = ({ audio, video, }) => {
     }
     return UserMediaStatus.PermissionsGrantedFallback;
 };
-export const toOnlyDeviceStatus = (kind, streamedRequestedDevices, devices) => {
-    const [self, other] = kind === 'audioinput' ? ['audio', 'video'] : ['video', 'audio'];
+export const toOnlyDeviceStatus = (kind, matched, devices) => {
     const otherKind = kind === 'audioinput' ? 'videoinput' : 'audioinput';
-    const matched = streamedRequestedDevices[self];
-    const anotherDeviceRequested = streamedRequestedDevices[other];
     const hasAnotherTypeOfAuthorizedDevice = devices.anyAuthorizedDevice(otherKind);
     const hasAnotherTypeOfDevice = devices.size(otherKind) > 0;
     if (matched) {
         if (hasAnotherTypeOfDevice) {
-            if (hasAnotherTypeOfAuthorizedDevice || !anotherDeviceRequested) {
+            if (hasAnotherTypeOfAuthorizedDevice) {
                 return kind === 'audioinput'
                     ? UserMediaStatus.PermissionsOnlyAudioinput
                     : UserMediaStatus.PermissionsOnlyVideoinput;
@@ -94,7 +91,7 @@ export const toOnlyDeviceStatus = (kind, streamedRequestedDevices, devices) => {
             : UserMediaStatus.PermissionsOnlyVideoinputNoAudioDevices;
     }
     if (hasAnotherTypeOfDevice) {
-        if (hasAnotherTypeOfAuthorizedDevice || !anotherDeviceRequested) {
+        if (hasAnotherTypeOfAuthorizedDevice) {
             return kind === 'audioinput'
                 ? UserMediaStatus.PermissionsOnlyAudioinputFallback
                 : UserMediaStatus.PermissionsOnlyVideoinputFallback;
@@ -259,6 +256,7 @@ export const mergeStatus = (prevStatus, nextStatus, prevTracks = [], nextTracks
                     return nextStatus;
             }
         }
+        return nextStatus;
     }
     // Failed to get a new track
     if (prevValidAudio) {
@@ -426,8 +424,8 @@ export const requestUserMedia = (getCurrentDevices, getMedia = getUserMedia) =>
                 : false,
         });
         const streamedRequestedDevices = isStreamingRequestedDevices(constraints, stream, devices);
-        const onlyAudioStatus = toOnlyDeviceStatus('audioinput', streamedRequestedDevices, devices);
-        const onlyVideoStatus = toOnlyDeviceStatus('videoinput', streamedRequestedDevices, devices);
+        const onlyAudioStatus = toOnlyDeviceStatus('audioinput', streamedRequestedDevices.audio, devices);
+        const onlyVideoStatus = toOnlyDeviceStatus('videoinput', streamedRequestedDevices.video, devices);
         const status = deriveDeviceStatus(onlyAudioStatus, onlyVideoStatus, toSameDeviceStatus(streamedRequestedDevices));
         return [stream, status];
     }

package/dist/videoProcessor.d.ts

@@ -22,6 +22,7 @@ interface VideoStreamProcessProps extends Partial<VideoRenderParams> {
     contentHint?: VideoContentHint;
 }
 declare const FEATURE_KEYS: {
+    readonly backend: "string";
     readonly backgroundBlurAmount: "number";
     readonly backgroundImageUrl: "string";
     readonly backgroundThreshold: "number";

package/dist/videoProcessor.js

@@ -6,6 +6,7 @@ import { logger } from './logger';
 import { isVideoContentHint } from './typeGuard';
 import { PROCESSOR_LABELS } from './constants';
 const FEATURE_KEYS = {
+    backend: 'string',
     backgroundBlurAmount: 'number',
     backgroundImageUrl: 'string',
     backgroundThreshold: 'number',
@@ -217,21 +218,26 @@ const applyFeatures = async (processor, features, referenceProcessingHeight) =>
             case 'downSampleFactor':
             case 'backgroundThreshold': {
                 const value = features[k];
-                if (value !== undefined) {
+                if (value !== undefined &&
+                    value !== processor.renderOptions[k]) {
                     renderOptions[k] = value;
                 }
                 break;
             }
             case 'excludeBystanders': {
                 const value = features[k];
-                if (value !== undefined) {
+                if (value !== undefined &&
+                    value !== processor.renderOptions[k]) {
                     renderOptions[k] = value;
                 }
                 break;
             }
             case 'personCenterX': {
                 const value = features[k];
-                if (value !== undefined) {
+                if (value !== undefined &&
+                    value !== processor.renderOptions.personCenter[0]) {
+                    // Keep the other coordinate from the local draft or existing
+                    // processor state so X/Y single-field updates stay paired.
                     renderOptions.personCenter = [
                         value,
                         renderOptions.personCenter?.[1] ??
@@ -242,7 +248,10 @@ const applyFeatures = async (processor, features, referenceProcessingHeight) =>
             }
             case 'personCenterY': {
                 const value = features[k];
-                if (value !== undefined) {
+                if (value !== undefined &&
+                    value !== processor.renderOptions.personCenter[1]) {
+                    // Keep the other coordinate from the local draft or existing
+                    // processor state so X/Y single-field updates stay paired.
                     renderOptions.personCenter = [
                         renderOptions.personCenter?.[0] ??
                             processor.renderOptions.personCenter[0],
@@ -255,14 +264,20 @@ const applyFeatures = async (processor, features, referenceProcessingHeight) =>
             case 'backgroundBlurAmount': {
                 const value = features[k];
                 if (value !== undefined) {
-                    renderOptions[k] = getBlurKernelSize(value, referenceProcessingHeight ?? processor.processingHeight, processor.processingHeight);
+                    const actualValue = getBlurKernelSize(value, referenceProcessingHeight ?? processor.processingHeight, processor.processingHeight);
+                    if (actualValue !== processor.renderOptions[k]) {
+                        renderOptions[k] = actualValue;
+                    }
                 }
                 break;
             }
             case 'edgeBlurAmount': {
                 const value = features[k];
                 if (value !== undefined) {
-                    renderOptions[k] = getTentBlurSize(value, referenceProcessingHeight ?? processor.processingHeight, processor.processingHeight);
+                    const actualValue = getTentBlurSize(value, referenceProcessingHeight ?? processor.processingHeight, processor.processingHeight);
+                    if (actualValue !== processor.renderOptions[k]) {
+                        renderOptions[k] = actualValue;
+                    }
                 }
                 break;
             }
@@ -273,6 +288,13 @@ const applyFeatures = async (processor, features, referenceProcessingHeight) =>
                 }
                 break;
             }
+            case 'backend': {
+                const value = features[k];
+                if (value && value !== processor.renderOptions.backend) {
+                    renderOptions.backend = value;
+                }
+                break;
+            }
             case 'backgroundImageUrl': {
                 const value = features[k];
                 if (value &&
@@ -356,9 +378,23 @@ label = PROCESSOR_LABELS.VideoProcessor, referenceProcessingHeight, maskCombineR
             });
         }
         else {
-            await applyFeatures(videoProcessor, { ...features, videoSegmentation }, referenceProcessingHeight);
+            await applyFeatures(videoProcessor, {
+                backend: gpuAPI(),
+                ...renderOptions,
+                videoSegmentation,
+            }, referenceProcessingHeight);
         }
         const track = await videoProcessor.process(prevMediaTrack.track);
+        // The source track constraints may have changed while `process()` was
+        // awaiting the segmentation model to load (e.g. the user toggled blur
+        // in pre-flight before the processor was ready). The features computed
+        // above are already baked into the freshly opened processor, so
+        // re-apply anything that changed in the meantime to avoid silently
+        // dropping the effect.
+        const latestFeatures = updateFeatureProps(prevMediaTrack.getConstraints(), props);
+        if (!isEmpty(latestFeatures)) {
+            await applyFeatures(videoProcessor, latestFeatures, referenceProcessingHeight);
+        }
         // Inherit contentHint from previous track
         track.contentHint = prevMediaTrack.track.contentHint;
         let trackReleaseTimeoutID = 0;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@pexip/media",
-  "version": "22.1.0",
+  "version": "22.3.0",
   "description": "Home for media related stuff",
   "homepage": "https://developer.pexip.com",
   "bugs": "https://gitlab.com/pexip/zoo/issues",
@@ -39,7 +39,7 @@
     "clean": "rm -fr dist api-docs",
     "dev": "pexip-bundler dev",
     "prepack": "yarn clean && yarn build && yarn docs",
-    "test": "jest",
+    "test": "pexip-bundler test",
     "check-format": "prettier --ignore-path=../../.prettierignore --check .",
     "format": "prettier --ignore-path=../../.prettierignore --write .",
     "lint": "yarn run -T biome lint",
@@ -47,20 +47,15 @@
     "docs": "typedoc"
   },
   "dependencies": {
-    "@pexip/media-control": "22.1.0",
-    "@pexip/media-processor": "22.1.0",
-    "@pexip/signal": "16.9.5",
-    "@pexip/utils": "17.4.0"
+    "@pexip/media-control": "22.3.0",
+    "@pexip/media-processor": "22.3.0",
+    "@pexip/signal": "16.9.6",
+    "@pexip/utils": "17.4.1"
   },
   "devDependencies": {
-    "@jest/globals": "^29.5.12",
-    "@pexip/bundler": "18.2.1",
-    "@swc/core": "^1.15.33",
-    "@swc/jest": "^0.2.39",
-    "jest": "^29.5.12",
-    "jest-junit": "^16.0.0",
+    "@pexip/bundler": "18.3.0",
     "prettier": "^3.2.5",
-    "typedoc": "^0.28.18",
+    "typedoc": "^0.28.20",
     "typedoc-plugin-markdown": "^4.11.0",
     "typescript": "~6.0.2"
   },