@pexip/media-control 20.3.5 → 21.0.0

package/CHANGELOG.md

@@ -1,5 +1,52 @@
 # Changelog
 
+## 21.0.0
+
+### Minor Changes
+
+- 1786b2d: `isStreamingRequestingDevice` returns `false` when request is `false`
+  Added an early return when request === false, so that a device kind explicitly
+  set to false in the constraints is correctly reported as not
+  streaming/requesting that device. Previously this case fell through to the
+  request === undefined check which returned true.
+- a004e3c: Introduce new `renderParams` that can be applied via the video
+  constraints:
+  - `backgroundThreshold` - Segmentation threshold for classifying pixels as
+    background (lower values include more background).
+  - `downSampleFactor` - Factor for downsampling the video frame before
+    processing (higher values improve performance but reduce quality).
+  - `lightWrapBlurAmount` - Amount of blur applied to the light wrap effect
+    around the subject's edges.
+  - `lightWrapEdgeBand` - Width of the edge band in pixels where the light wrap
+    effect is applied.
+  - `lightWrapIntensity` - Intensity/strength of the light wrap effect (0.0 to
+    1.0).
+  - `lightWrapTightness` - Controls how tightly the light wrap effect hugs the
+    subject's edges.
+  - `morphDilateRadiusPx` - Radius in pixels for morphological dilation
+    operation on the segmentation mask.
+  - `morphErodeRadiusPx` - Radius in pixels for morphological erosion operation
+    on the segmentation mask.
+  - `morphPass` - Number of morphological filter passes to apply for mask
+    refinement.
+  - `personCenterX` - X-coordinate (0.0 to 1.0) of the expected center position
+    of the person in the frame for segmentation optimization.
+  - `personCenterY` - Y-coordinate (0.0 to 1.0) of the expected center position
+    of the person in the frame for segmentation optimization.
+  - `sigmaRangeHi` - Bilateral filter: upper bound of sigma in color space for
+    edge-preserving smoothing.
+  - `sigmaRangeLo` - Bilateral filter: lower bound of sigma in color space for
+    edge-preserving smoothing.
+  - `sigmaSpace` - Bilateral filter: sigma in pixel space for mask edge
+    snapping.
+
+### Patch Changes
+
+- Updated dependencies [1fff975]
+- Updated dependencies [3963c2e]
+- Updated dependencies [762ebd2]
+  - @pexip/utils@17.2.0
+
 ## 20.3.5
 
 ### Patch Changes

package/api-docs/functions/extractConstraintsWithKeys.mdx

@@ -8,7 +8,7 @@ Extract the constraints with provided keys
 
 | Type Parameter |
 | ------ |
-| `T` *extends* \| `"deviceId"` \| `"groupId"` \| `"device"` \| `"videoSegmentation"` \| `"videoSegmentationModel"` \| `"denoise"` \| `"vad"` \| `"asd"` \| `"mixWithAdditionalMedia"` \| `"backgroundImageUrl"` \| `"resizeMode"` \| `"pan"` \| `"tilt"` \| `"zoom"` \| `"contentHint"` \| `"facingMode"` \| `"echoCancellation"` \| `"autoGainControl"` \| `"noiseSuppression"` \| `ConstrainNumberKeys` |
+| `T` *extends* \| `"deviceId"` \| `"groupId"` \| `"device"` \| `"videoSegmentation"` \| `"videoSegmentationModel"` \| `"denoise"` \| `"vad"` \| `"asd"` \| `"mixWithAdditionalMedia"` \| `"backgroundImageUrl"` \| `"excludeBystanders"` \| `"resizeMode"` \| `"pan"` \| `"tilt"` \| `"zoom"` \| `"contentHint"` \| `"facingMode"` \| `"echoCancellation"` \| `"autoGainControl"` \| `"noiseSuppression"` \| `ConstrainNumberKeys` |
 
 ## Parameters
 

package/api-docs/interfaces/InputConstraintSet.mdx

@@ -17,7 +17,22 @@
 | <a id="edgebluramount"></a> `edgeBlurAmount?` | `ConstrainULong` | Blur amount applied to the segmented person's edge | - |
 | <a id="foregroundthreshold"></a> `foregroundThreshold?` | `ConstrainDouble` | Erode level for edge smoothing when using video segmentation | - |
 | <a id="backgroundimageurl"></a> `backgroundImageUrl?` | `ConstrainDOMString` | Image Url that is being used for video overlay effects | - |
-| <a id="maskcombineratio"></a> `maskCombineRatio?` | `ConstrainDouble` | The ratio to be used for smoothing segmentation mask | - |
+| <a id="maskcombineratio"></a> `maskCombineRatio?` | `ConstrainDouble` | The ratio to be used for smoothing segmentation mask. How much uncertainty enables temporal blend, [0,1]. Higher = more frame-to-frame blending, more stability but more trailing ghosts. Typical: 0.6–0.85. | - |
+| <a id="backgroundthreshold"></a> `backgroundThreshold?` | `ConstrainDouble` | [0,1] Lower mask confidence: all mask pixels <= this are considered definite background. Adjust to make mask edges tighter or more permissive for background inclusion. Example: 0.3 ("mask < 0.3 = sure background"). | - |
+| <a id="sigmaspace"></a> `sigmaSpace?` | `ConstrainDouble` | Bilateral filter: sigma in pixel space for mask edge snapping. Controls the neighborhood/pixel distance—higher is larger window (5–10 typical). | - |
+| <a id="sigmarangelo"></a> `sigmaRangeLo?` | `ConstrainDouble` | Bilateral filter: minimum sigma in color/luma (for mask interiors), [0,1]. Controls how little color difference is needed to cross-mask regions away from edge. Lower = sharper edge preservation in mask interiors. Example: 0.2 | - |
+| <a id="sigmarangehi"></a> `sigmaRangeHi?` | `ConstrainDouble` | Bilateral filter: maximum sigma in color/luma (for mask uncertain zones), [0,1]. Controls how much color similarity is required at uncertain/edge mask locations. Higher = more mask smoothing at edges. Example: 0.8 | - |
+| <a id="excludebystanders"></a> `excludeBystanders?` | `ConstrainBoolean` | Whether to exclude bystanders/connected people from the output mask. The main person instance (closest to personCenter OR largest blob) is kept in the output mask. All other bystanders/connected people are suppressed (bystander removal). | - |
+| <a id="personcenterx"></a> `personCenterX?` | `ConstrainDouble` | Person center: normalized in [0,1] image coordinates for main subject selection. Used by the CCL/bias filter to select the main person blob. | - |
+| <a id="personcentery"></a> `personCenterY?` | `ConstrainDouble` | Person center: normalized in [0,1] image coordinates for main subject selection. Used by the CCL/bias filter to select the main person blob. | - |
+| <a id="morpheroderadiuspx"></a> `morphErodeRadiusPx?` | `ConstrainULong` | Morphological erosion: radius in pixels [1,12]. Sets kernel size for bridge-breaking. Larger values break thicker bridges but may erode fine details. | - |
+| <a id="morphpass"></a> `morphPass?` | `ConstrainULong` | Morphological pass: number of separable passes [1,10]. More passes increase the effective span. | - |
+| <a id="morphdilateradiuspx"></a> `morphDilateRadiusPx?` | `ConstrainULong` | Geodesic dilation: radius in pixels [1,12]. Must match erosion for good reconstruction. | - |
+| <a id="lightwrapintensity"></a> `lightWrapIntensity?` | `ConstrainDouble` | [0,1] Light wrap amount (halo intensity) at mask edge. Wraps blurred background color just outside the subject edge onto the subject boundary for realism. | - |
+| <a id="lightwraptightness"></a> `lightWrapTightness?` | `ConstrainDouble` | [0.5,4.0] Controls power/tightness of the wrap band falloff. Higher = tighter/narrower wrap. | - |
+| <a id="lightwrapedgeband"></a> `lightWrapEdgeBand?` | `ConstrainDouble` | Width of the wrap band (feather size, in normalized mask units) between the backgroundThreshold and foregroundThreshold in [0.0, (foregroundThreshold - backgroundThreshold) / 2] Higher = wider soft halo. | - |
+| <a id="lightwrapbluramount"></a> `lightWrapBlurAmount?` | `ConstrainULong` | Blur amount used for background color in the wrap band (units=blur passes). | - |
+| <a id="downsamplefactor"></a> `downSampleFactor?` | `ConstrainULong` | Downsampling factor for mask/blurring, must be [2,4]. 2 = half resolution, 4 = quarter resolution for faster filtering. Use for performance tuning vs. quality. | - |
 | <a id="resizemode"></a> `resizeMode?` | `ConstrainDOMString` | Image Url that is being used for video overlay effects | - |
 | <a id="pan"></a> `pan?` | `boolean` | Current pan, tilt and zoom level for a PTZ controllable camera on web | - |
 | <a id="tilt"></a> `tilt?` | `boolean` | - | - |

package/dist/devices.js

@@ -372,6 +372,9 @@ export const resolveInputDevice = (tracksOrDevice, findInput) => {
     return undefined;
 };
 export const isStreamingRequestingDevice = (request, trackOrDevice, devices) => {
+    if (request === false) {
+        return false;
+    }
     if (request === undefined) {
         return true;
     }

package/dist/typeGuards.d.ts

@@ -21,11 +21,11 @@ export declare const EXTENDED_CONSTRAIN_STRING_KEYS: readonly ["videoSegmentatio
 export type ExtendedConstrainStringKeys = (typeof EXTENDED_CONSTRAIN_STRING_KEYS)[number];
 export type ConstrainStringKeys = (typeof CONSTRAIN_STRING_KEYS)[number];
 export declare const CONSTRAIN_U_LONG_KEYS: readonly ["width", "height", "sampleRate", "sampleSize", "channelCount"];
-export declare const EXTENDED_CONSTRAIN_U_LONG_KEYS: readonly ["backgroundBlurAmount", "edgeBlurAmount"];
+export declare const EXTENDED_CONSTRAIN_U_LONG_KEYS: readonly ["backgroundBlurAmount", "edgeBlurAmount", "morphErodeRadiusPx", "morphPass", "morphDilateRadiusPx", "lightWrapBlurAmount", "downSampleFactor"];
 export type ExtendedConstrainULongKeys = (typeof EXTENDED_CONSTRAIN_U_LONG_KEYS)[number];
 export type ConstrainULongKeys = (typeof CONSTRAIN_U_LONG_KEYS)[number];
 export declare const CONSTRAIN_DOUBLE_KEYS: readonly ["aspectRatio", "frameRate", "latency"];
-export declare const EXTENDED_CONSTRAIN_DOUBLE_KEYS: readonly ["foregroundThreshold", "maskCombineRatio"];
+export declare const EXTENDED_CONSTRAIN_DOUBLE_KEYS: readonly ["foregroundThreshold", "backgroundThreshold", "sigmaSpace", "sigmaRangeLo", "sigmaRangeHi", "maskCombineRatio", "personCenterX", "personCenterY", "lightWrapIntensity", "lightWrapTightness", "lightWrapEdgeBand"];
 export type ExtendedConstrainDoubleKeys = (typeof EXTENDED_CONSTRAIN_DOUBLE_KEYS)[number];
 export type ConstrainDoubleKeys = (typeof CONSTRAIN_DOUBLE_KEYS)[number];
 export declare const CONSTRAIN_BOOLEAN_KEYS: readonly ["echoCancellation", "autoGainControl", "noiseSuppression", "pan", "tilt", "zoom"];
@@ -33,7 +33,7 @@ export declare const CONSTRAIN_BOOLEAN_KEYS: readonly ["echoCancellation", "auto
  * Extends boolean constraint keys for our own implementation of the media
  * feature
  */
-export declare const EXTENDED_CONSTRAIN_BOOLEAN_KEYS: readonly ["vad", "asd", "mixWithAdditionalMedia", "denoise"];
+export declare const EXTENDED_CONSTRAIN_BOOLEAN_KEYS: readonly ["vad", "asd", "mixWithAdditionalMedia", "denoise", "excludeBystanders"];
 export type ExtendedConstrainBooleanKeys = (typeof EXTENDED_CONSTRAIN_BOOLEAN_KEYS)[number];
 export type ConstrainBooleanKeys = (typeof CONSTRAIN_BOOLEAN_KEYS)[number];
 /**

package/dist/typeGuards.js

@@ -39,6 +39,11 @@ export const CONSTRAIN_U_LONG_KEYS = [
 export const EXTENDED_CONSTRAIN_U_LONG_KEYS = [
     'backgroundBlurAmount',
     'edgeBlurAmount',
+    'morphErodeRadiusPx',
+    'morphPass',
+    'morphDilateRadiusPx',
+    'lightWrapBlurAmount',
+    'downSampleFactor',
 ];
 export const CONSTRAIN_DOUBLE_KEYS = [
     'aspectRatio',
@@ -47,7 +52,16 @@ export const CONSTRAIN_DOUBLE_KEYS = [
 ];
 export const EXTENDED_CONSTRAIN_DOUBLE_KEYS = [
     'foregroundThreshold',
+    'backgroundThreshold',
+    'sigmaSpace',
+    'sigmaRangeLo',
+    'sigmaRangeHi',
     'maskCombineRatio',
+    'personCenterX',
+    'personCenterY',
+    'lightWrapIntensity',
+    'lightWrapTightness',
+    'lightWrapEdgeBand',
 ];
 export const CONSTRAIN_BOOLEAN_KEYS = [
     'echoCancellation',
@@ -70,6 +84,8 @@ export const EXTENDED_CONSTRAIN_BOOLEAN_KEYS = [
     'mixWithAdditionalMedia',
     // Noise Suppression using our own impl
     'denoise',
+    // Bystander removal
+    'excludeBystanders',
 ];
 /**
  * The keys from the `MediaTrackConstraintSet`

package/dist/types.d.ts

@@ -107,9 +107,93 @@ export interface InputConstraintSet extends MediaTrackConstraints {
      */
     backgroundImageUrl?: ConstrainDOMString;
     /**
-     * The ratio to be used for smoothing segmentation mask
+     * The ratio to be used for smoothing segmentation mask.
+     * How much uncertainty enables temporal blend, [0,1].
+     * Higher = more frame-to-frame blending, more stability but more trailing ghosts.
+     * Typical: 0.6–0.85.
      */
     maskCombineRatio?: ConstrainDouble;
+    /**
+     * [0,1] Lower mask confidence: all mask pixels <= this are considered definite background.
+     * Adjust to make mask edges tighter or more permissive for background inclusion.
+     * Example: 0.3 ("mask < 0.3 = sure background").
+     */
+    backgroundThreshold?: ConstrainDouble;
+    /**
+     * Bilateral filter: sigma in pixel space for mask edge snapping.
+     * Controls the neighborhood/pixel distance—higher is larger window (5–10 typical).
+     */
+    sigmaSpace?: ConstrainDouble;
+    /**
+     * Bilateral filter: minimum sigma in color/luma (for mask interiors), [0,1].
+     * Controls how little color difference is needed to cross-mask regions away from edge.
+     * Lower = sharper edge preservation in mask interiors.
+     * Example: 0.2
+     */
+    sigmaRangeLo?: ConstrainDouble;
+    /**
+     * Bilateral filter: maximum sigma in color/luma (for mask uncertain zones), [0,1].
+     * Controls how much color similarity is required at uncertain/edge mask locations.
+     * Higher = more mask smoothing at edges.
+     * Example: 0.8
+     */
+    sigmaRangeHi?: ConstrainDouble;
+    /**
+     * Whether to exclude bystanders/connected people from the output mask.
+     * The main person instance (closest to personCenter OR largest blob) is kept in
+     * the output mask.
+     * All other bystanders/connected people are suppressed (bystander removal).
+     */
+    excludeBystanders?: ConstrainBoolean;
+    /**
+     * Person center: normalized in [0,1] image coordinates for main subject selection.
+     * Used by the CCL/bias filter to select the main person blob.
+     */
+    personCenterX?: ConstrainDouble;
+    /**
+     * Person center: normalized in [0,1] image coordinates for main subject selection.
+     * Used by the CCL/bias filter to select the main person blob.
+     */
+    personCenterY?: ConstrainDouble;
+    /**
+     * Morphological erosion: radius in pixels [1,12]. Sets kernel size for bridge-breaking.
+     * Larger values break thicker bridges but may erode fine details.
+     */
+    morphErodeRadiusPx?: ConstrainULong;
+    /**
+     * Morphological pass: number of separable passes [1,10].
+     * More passes increase the effective span.
+     */
+    morphPass?: ConstrainULong;
+    /**
+     * Geodesic dilation: radius in pixels [1,12]. Must match erosion for good reconstruction.
+     */
+    morphDilateRadiusPx?: ConstrainULong;
+    /**
+     * [0,1] Light wrap amount (halo intensity) at mask edge.
+     * Wraps blurred background color just outside the subject edge onto the subject boundary for realism.
+     */
+    lightWrapIntensity?: ConstrainDouble;
+    /**
+     * [0.5,4.0] Controls power/tightness of the wrap band falloff. Higher = tighter/narrower wrap.
+     */
+    lightWrapTightness?: ConstrainDouble;
+    /**
+     * Width of the wrap band (feather size, in normalized mask units) between the backgroundThreshold
+     * and foregroundThreshold in [0.0, (foregroundThreshold - backgroundThreshold) / 2]
+     * Higher = wider soft halo.
+     */
+    lightWrapEdgeBand?: ConstrainDouble;
+    /**
+     * Blur amount used for background color in the wrap band (units=blur passes).
+     */
+    lightWrapBlurAmount?: ConstrainULong;
+    /**
+     * Downsampling factor for mask/blurring, must be [2,4].
+     * 2 = half resolution, 4 = quarter resolution for faster filtering.
+     * Use for performance tuning vs. quality.
+     */
+    downSampleFactor?: ConstrainULong;
     /**
      * Image Url that is being used for video overlay effects
      */

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@pexip/media-control",
-  "version": "20.3.5",
-  "homepage": "https://gitlab.com/pexip/zoo",
+  "version": "21.0.0",
+  "homepage": "https://developer.pexip.com",
   "bugs": "https://gitlab.com/pexip/zoo/issues",
   "repository": {
     "type": "git",
@@ -35,11 +35,11 @@
     "api-docs"
   ],
   "scripts": {
-    "build": "pexip-bundler build && yarn docs",
-    "build:sourcemap": "pexip-bundler build --sourcemap && yarn docs",
+    "build": "pexip-bundler build",
+    "build:sourcemap": "pexip-bundler build --sourcemap",
     "clean": "rm -rf dist api-docs",
     "dev": "pexip-bundler dev",
-    "prepack": "yarn clean && yarn build",
+    "prepack": "yarn clean && yarn build && yarn docs",
     "test": "jest",
     "check-format": "prettier --ignore-path=../../.prettierignore --check .",
     "format": "prettier --ignore-path=../../.prettierignore --write .",
@@ -48,7 +48,7 @@
     "docs": "typedoc"
   },
   "dependencies": {
-    "@pexip/utils": "17.1.3"
+    "@pexip/utils": "17.2.0"
   },
   "devDependencies": {
     "@jest/globals": "^30.2.0",