@pexip/media 16.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,1079 @@
1
+ # @pexip/media
2
+
3
+ ## 16.7.1
4
+
5
+ ### Patch Changes
6
+
7
+ - Updated dependencies [94b7f753]
8
+ - @pexip/utils@16.6.0
9
+ - @pexip/media-control@16.7.1
10
+ - @pexip/media-processor@16.7.1
11
+
12
+ ## 16.7.0
13
+
14
+ ### Minor Changes
15
+
16
+ - 39ceb6d0: Take `MediaStreamTrack['muted']` into account for
17
+ `Media['audioMuted']` and `Media['videoMuted']`
18
+ - cf498a4a: Remove @sentry/hub
19
+ - 20a44927: Resume AudioContext used in the audio mixer
20
+ - 0f95c333: Refactor to standardize error logging
21
+
22
+ ### Patch Changes
23
+
24
+ - Updated dependencies [05ef9099]
25
+ - Updated dependencies [20a44927]
26
+ - Updated dependencies [456c06c6]
27
+ - @pexip/media-control@16.7.0
28
+ - @pexip/media-processor@16.7.0
29
+ - @pexip/utils@16.5.0
30
+
31
+ ## 16.6.0
32
+
33
+ ### Minor Changes
34
+
35
+ - 454eda4: Derive media status from previous status to avoid going back to
36
+ initial
37
+
38
+ Firefox Android always revoking the permission by emptying the device label,
39
+ this causing some unexpected behavior. With previous status, we have more
40
+ information to derive the correct initial status.
41
+
42
+ - f8d4190238: Sync Media['devices'] on `devicechange`
43
+ - d2ad3cd13f: Fix isGranted logic to include only error
44
+ - f8d4190238: Split isGranted into isGrantedVideo and isGrantedAudio
45
+
46
+ ### Patch Changes
47
+
48
+ - Updated dependencies [3ba403f688]
49
+ - Updated dependencies [f8d4190238]
50
+ - Updated dependencies [f8d4190238]
51
+ - Updated dependencies [f8d4190238]
52
+ - @pexip/utils@16.4.1
53
+ - @pexip/media-control@16.6.0
54
+ - @pexip/media-processor@16.6.0
55
+
56
+ ## 16.5.0
57
+
58
+ ### Minor Changes
59
+
60
+ - 6225612: Remove @pexip/logger dependency
61
+
62
+ ### Patch Changes
63
+
64
+ - Updated dependencies [e6ef1f2]
65
+ - Updated dependencies [6225612]
66
+ - @pexip/utils@16.4.0
67
+ - @pexip/media-control@16.5.0
68
+ - @pexip/signal@16.5.0
69
+ - @pexip/media-processor@16.5.0
70
+
71
+ ## 16.4.0
72
+
73
+ ### Minor Changes
74
+
75
+ - 0a57d4d: Expose Video processor processing API flag
76
+ - 6496a9d: Extract no device status
77
+
78
+ ### Patch Changes
79
+
80
+ - Updated dependencies [6534b17]
81
+ - Updated dependencies [6496a9d]
82
+ - Updated dependencies [c1c3767]
83
+ - @pexip/signal@16.4.0
84
+ - @pexip/media-control@16.4.0
85
+ - @pexip/media-processor@16.4.0
86
+
87
+ ## 16.3.1
88
+
89
+ ### Patch Changes
90
+
91
+ - Updated dependencies [4763e5b4]
92
+ - @pexip/utils@16.3.1
93
+ - @pexip/media-control@16.3.1
94
+ - @pexip/media-processor@16.3.1
95
+
96
+ ## 16.3.0
97
+
98
+ ### Patch Changes
99
+
100
+ - @pexip/media-control@16.3.0
101
+ - @pexip/media-processor@16.3.0
102
+ - @pexip/signal@16.3.0
103
+ - @pexip/utils@16.3.0
104
+
105
+ ## 16.2.0
106
+
107
+ ### Patch Changes
108
+
109
+ - 3ec61611: Fix for video mute when videoProcessing is off.
110
+ - @pexip/media-control@16.2.0
111
+ - @pexip/media-processor@16.2.0
112
+ - @pexip/signal@16.2.0
113
+ - @pexip/utils@16.2.0
114
+
115
+ ## 16.1.0
116
+
117
+ ### Patch Changes
118
+
119
+ - @pexip/media-control@16.1.0
120
+ - @pexip/media-processor@16.1.0
121
+ - @pexip/signal@16.1.0
122
+ - @pexip/utils@16.1.0
123
+
124
+ ## 16.0.0
125
+
126
+ ### Patch Changes
127
+
128
+ - @pexip/media-control@16.0.0
129
+ - @pexip/media-processor@16.0.0
130
+ - @pexip/signal@16.0.0
131
+ - @pexip/utils@16.0.0
132
+
133
+ ## 15.0.0
134
+
135
+ ### Patch Changes
136
+
137
+ - @pexip/media-control@15.0.0
138
+ - @pexip/media-processor@15.0.0
139
+ - @pexip/signal@15.0.0
140
+ - @pexip/utils@15.0.0
141
+
142
+ ## 14.1.0
143
+
144
+ ### Minor Changes
145
+
146
+ - 9ac21215: Improve blur param with boundary
147
+ - 53d63e31: Force to down-mix to mono for audio mixing process
148
+
149
+ ### Patch Changes
150
+
151
+ - @pexip/media-control@14.1.0
152
+ - @pexip/media-processor@14.1.0
153
+ - @pexip/signal@14.1.0
154
+ - @pexip/utils@14.1.0
155
+
156
+ ## 14.0.0
157
+
158
+ ### Patch Changes
159
+
160
+ - Updated dependencies [c85b88f8]
161
+ - @pexip/media-control@14.0.0
162
+ - @pexip/media-processor@14.0.0
163
+ - @pexip/signal@14.0.0
164
+ - @pexip/utils@14.0.0
165
+
166
+ ## 13.2.0
167
+
168
+ ### Patch Changes
169
+
170
+ - @pexip/media-processor@13.2.0
171
+ - @pexip/media-control@13.2.0
172
+ - @pexip/signal@13.2.0
173
+ - @pexip/utils@13.2.0
174
+
175
+ ## 13.1.0
176
+
177
+ ### Patch Changes
178
+
179
+ - @pexip/media-control@13.1.0
180
+ - @pexip/media-processor@13.1.0
181
+ - @pexip/signal@13.1.0
182
+ - @pexip/utils@13.1.0
183
+
184
+ ## 13.0.0
185
+
186
+ ### Patch Changes
187
+
188
+ - Updated dependencies [3750d84b]
189
+ - @pexip/media-control@13.0.0
190
+ - @pexip/media-processor@13.0.0
191
+ - @pexip/signal@13.0.0
192
+ - @pexip/utils@13.0.0
193
+
194
+ ## 12.0.0
195
+
196
+ ### Major Changes
197
+
198
+ - 8caca489: Remove Personify model for security reasons
199
+
200
+ ### Patch Changes
201
+
202
+ - Updated dependencies [8caca489]
203
+ - Updated dependencies [52cd2cdb]
204
+ - @pexip/media-processor@12.0.0
205
+ - @pexip/signal@12.0.0
206
+ - @pexip/media-control@12.0.0
207
+ - @pexip/utils@12.0.0
208
+
209
+ ## 11.0.0
210
+
211
+ ### Patch Changes
212
+
213
+ - @pexip/media-control@11.0.0
214
+ - @pexip/media-processor@11.0.0
215
+ - @pexip/signal@11.0.0
216
+ - @pexip/utils@11.0.0
217
+
218
+ ## 10.1.0
219
+
220
+ ### Patch Changes
221
+
222
+ - @pexip/media-control@10.1.0
223
+ - @pexip/media-processor@10.1.0
224
+ - @pexip/signal@10.1.0
225
+ - @pexip/utils@10.1.0
226
+
227
+ ## 10.0.0
228
+
229
+ ### Patch Changes
230
+
231
+ - @pexip/media-control@10.0.0
232
+ - @pexip/media-processor@10.0.0
233
+ - @pexip/signal@10.0.0
234
+ - @pexip/utils@10.0.0
235
+
236
+ ## 9.0.0
237
+
238
+ ### Minor Changes
239
+
240
+ - 07fab35b: ## `edgeBlurAmount` and `backgroundBlurAmount` are a percentage of
241
+ the processing height
242
+
243
+ - The render parameters: `edgeBlurAmount` and `backgroundBlurAmount` are now a
244
+ percentage of the processing height.
245
+
246
+ - 5a852b05: No unnecessary reqeust on blocked devices
247
+
248
+ ### Patch Changes
249
+
250
+ - Updated dependencies [5a852b05]
251
+ - @pexip/media-control@9.0.0
252
+ - @pexip/media-processor@9.0.0
253
+ - @pexip/signal@9.0.0
254
+ - @pexip/utils@9.0.0
255
+
256
+ ## 8.0.0
257
+
258
+ ### Patch Changes
259
+
260
+ - @pexip/media-control@8.0.0
261
+ - @pexip/media-processor@8.0.0
262
+ - @pexip/signal@8.0.0
263
+ - @pexip/utils@8.0.0
264
+
265
+ ## 7.0.0
266
+
267
+ ### Minor Changes
268
+
269
+ - af8233c9: Fix the logic in tryAndGetUserMedia and
270
+ isInitialPermissionsNotGranted. Add isInitialPermissionsGranted.
271
+ - 5d824c12: No unnecessary gUM for irrelevant device change
272
+
273
+ ## '@pexip/media': minor
274
+
275
+ - No unnecessary gUM for irrelevant device change
276
+
277
+ closes https://gitlab.com/pexip/zoo/-/issues/3129
278
+
279
+ ## '@pexip/media-control': minor
280
+
281
+ - Add 2 new util functions
282
+ - hasRequestingDevice,
283
+ - shouldRequestDevice,
284
+
285
+ - efc7d2a8: Better media logging
286
+
287
+ ## @pexip/logger
288
+
289
+ - Add more AudioNode to log
290
+
291
+ ## @pexip/media
292
+
293
+ - Truncate unnecessary fields when logging with JSON
294
+
295
+ ## @pexip/media-control
296
+
297
+ - Put MediaDeviceInfoKind['settings'] fields into toJSON for logging
298
+
299
+ ## @pexip/media-processor
300
+
301
+ - Drop logger dependency and related usages
302
+
303
+ - dead15ae: Fix unplug and plug media fallback issue
304
+
305
+ closes https://gitlab.com/pexip/zoo/-/issues/2986
306
+
307
+ ## @pexip/media
308
+
309
+ - Use the correct media status after release
310
+
311
+ ## @pexip/media-control
312
+
313
+ - Ignore ended track when validating the current device and the new constraint
314
+
315
+ ### Patch Changes
316
+
317
+ - Updated dependencies [16854bc7]
318
+ - Updated dependencies [5d824c12]
319
+ - Updated dependencies [efc7d2a8]
320
+ - Updated dependencies [dead15ae]
321
+ - Updated dependencies [cbe3ccf1]
322
+ - @pexip/utils@7.0.0
323
+ - @pexip/media-control@7.0.0
324
+ - @pexip/media-processor@7.0.0
325
+ - @pexip/signal@7.0.0
326
+
327
+ ## 6.0.0
328
+
329
+ ### Patch Changes
330
+
331
+ - Updated dependencies [0522557c]
332
+ - @pexip/media-processor@6.0.0
333
+ - @pexip/media-control@6.0.0
334
+ - @pexip/signal@6.0.0
335
+ - @pexip/utils@6.0.0
336
+
337
+ ## 5.0.0
338
+
339
+ ### Minor Changes
340
+
341
+ - 9f68146d: # Presentation with audio.
342
+
343
+ Be able to share presentation with audio.
344
+
345
+ ## Changes on `media`
346
+
347
+ Introduce `audioMixingProcessor` to mix audio track with additional one.
348
+
349
+ ## Changes on `media-control`
350
+
351
+ Introduce `mixWithAdditionalMedia` constraint.
352
+
353
+ ## Changes on `media-components`
354
+
355
+ Pass a cleanup `fn` to `usePresentation` hook.
356
+
357
+ ## Changes on `media-processor`
358
+
359
+ Add `merger` node constructor.
360
+
361
+ ### Patch Changes
362
+
363
+ - Updated dependencies [2e36be80]
364
+ - Updated dependencies [9f68146d]
365
+ - @pexip/utils@5.0.0
366
+ - @pexip/media-control@5.0.0
367
+ - @pexip/media-processor@5.0.0
368
+ - @pexip/signal@5.0.0
369
+
370
+ ## 4.0.0
371
+
372
+ ### Minor Changes
373
+
374
+ - cf8bec29b6: # Use Streams API
375
+
376
+ ## Changes on `media-processor`
377
+
378
+ - Use Streams API for video processing, fallback to
379
+ requestVideoFrameCallback/setTimeout when the API is not available
380
+ - Turn off alpha when producing the final drawing on canvas
381
+ - closes https://gitlab.com/pexip/zoo/-/issues/2186
382
+ - Use desynchronized hint for canvas
383
+ - Fix edge issue when composing blur background
384
+ - Fix process `opened` state issue on async function
385
+
386
+ ### Commits
387
+
388
+ - fc398f4b38 perf(media-processor): use desynchronized hint
389
+ - ed2d95c653 feat(media-processor): use stream transform
390
+ - da53143a49 fix(media-processor): dom-webcodecs types
391
+ - e271047b0f feat(media-processor): VideoProcessor accept transformers
392
+ - eeb8992f97 feat(media-processor): add transformer creator
393
+ - 20d085aac2 feat(media-processor): add MediaStreamTrack generator & processor
394
+ - 096f1adc77 perf(media-processor): turn off alpha for none effect
395
+ - a1a282c34a refactor(media-processor): move canvas related into
396
+ canvasRenderer
397
+
398
+ ## Changes on `media`
399
+
400
+ - Use the latest APIs from `media-processor`
401
+ - Fix types overridden by `dom-webcodecs`
402
+
403
+ ### Commits
404
+
405
+ - 0156012aab fix(media): use streams API from media-processor
406
+ - e61ca85375 fix(media): type overrides
407
+
408
+ ## Changes on `media-control`
409
+
410
+ - Fix types overridden by `dom-webcodecs`
411
+
412
+ ### Commit
413
+
414
+ - 0a63010d20 fix(media-control): type overrides
415
+
416
+ ## Changes on Apps
417
+
418
+ - Use the latest APIs
419
+
420
+ ### Commits
421
+
422
+ - b7a768a350 feat(infinity-connect): turn on video processing for Safari
423
+ - d05dc48b8f fix(media-processor-demo): remove comments
424
+ - 6d58d0c987 fix(media-processor-demo): add missing await
425
+ - 4859029594 fix(infinity-connect): apply latest changes from media
426
+ - 3c30502ee7 fix(aquila): use latest changes from media
427
+ - 179359a95d fix(media-demo): update settings as well
428
+ - e3ac0cdd0e fix(media-demo): apply latest changes from media
429
+ - f251c75f16 feat(media-processor-demo): use createCanvasTransform
430
+
431
+ ## Others
432
+
433
+ - Add types/dom-webcodecs
434
+
435
+ ### Commit
436
+
437
+ - 2458690b46 build: add types/dom-webcodecs
438
+
439
+ ### Patch Changes
440
+
441
+ - Updated dependencies [d429f07df2]
442
+ - Updated dependencies [cf8bec29b6]
443
+ - @pexip/media-processor@4.0.0
444
+ - @pexip/media-control@4.0.0
445
+ - @pexip/signal@4.0.0
446
+ - @pexip/utils@4.0.0
447
+
448
+ ## 3.0.0
449
+
450
+ ### Major Changes
451
+
452
+ - 8e3e3b1f9: Refactor media package to follow the media constraint pattern
453
+
454
+ https://developer.mozilla.org/en-US/docs/Web/API/Media_Streams_API/Constraints
455
+
456
+ - Move and extend the media features based on MediaTrackConstraints by
457
+ interpreting the provided constraints
458
+ - Voice Activity Detection - `vad` of type `boolean`
459
+ - Audio Signal Detection - `asd` of type `boolean`
460
+ - Noise suppression - `denoise` (to avoid the conflict of the built-in
461
+ `noiseSupression`) of type `boolean`, `true` means using our own impl,
462
+ remember to disable the built-in `noiseSupression` as you don't want to
463
+ use them both
464
+ - Blur level - `blurLevel` of type `number`
465
+ - Erode param - `erode` of type `number`
466
+ - Video segmentation effects - `videoSegmentation` of type `none`, `blur`,
467
+ `overlay` or `remove` (`remove` will be added in another MR)
468
+ - Ad-hoc change of the features should be done via the `Media` object with the
469
+ new `applyConstraints` API
470
+ - `getUserMedia` is always getting a new media while applying the features
471
+ based on the constraints when there is change to the constraints
472
+
473
+ Example:
474
+
475
+ ```typescript
476
+ const constriants = {
477
+ audio: {
478
+ noiseSuppression: false, // Disable the built-in noise suppression
479
+ denoise: true, // Use our own impl of noise suppression
480
+ vad: true, // Use Voice Activity Detection
481
+ asd: true, // Use Audio Signal Detection to check if the microphone is malfunctioned
482
+ },
483
+ video: {
484
+ videoSegmentation: 'blur', // Use background blur
485
+ erode: 1,
486
+ blurLevel: 21,
487
+ },
488
+ };
489
+ ```
490
+
491
+ The new `MediaController` interface (removed feature-related props):
492
+
493
+ ```typescript
494
+ export interface MediaController {
495
+ /**
496
+ * Current Media if any
497
+ */
498
+ media: Media;
499
+ /**
500
+ * Current device list
501
+ */
502
+ devices: MediaDeviceInfoLike[];
503
+ /**
504
+ * Execute the media pipeline immediately with provided constraints
505
+ *
506
+ * @param constraints - @see MediaDeviceRequest
507
+ */
508
+ getUserMedia: (constraints: MediaDeviceRequest) => void;
509
+ /**
510
+ * Execute the media pipeline immediately with provided constraints. An
511
+ * explicit version of `getUserMedia` to make it possible to chain other
512
+ * async actions
513
+ *
514
+ * @param constraints - @see MediaDeviceRequest
515
+ */
516
+ getUserMediaAsync: (constraints: MediaDeviceRequest) => Promise<void>;
517
+ /**
518
+ * Cross-check PermissionState and available devices before requesting user
519
+ * media, the request will be skipped if there is no granted device.
520
+ */
521
+ tryAndGetUserMedia: (constraints: MediaDeviceRequest) => void;
522
+ }
523
+ ```
524
+
525
+ The new `Media` interface (added `applyConstraints`):
526
+
527
+ ```typescript
528
+ export interface Media {
529
+ /**
530
+ * The constraints used to request the media
531
+ */
532
+ constraints?: MediaDeviceRequest;
533
+ /**
534
+ * The devices used for the media
535
+ */
536
+ devices: MediaDeviceInfoLike[];
537
+ /**
538
+ * Media stream for the media
539
+ */
540
+ stream?: MediaStream;
541
+ /**
542
+ * The raw stream obtained from the `getUserMedia` API
543
+ */
544
+ rawStream?: MediaStream;
545
+ /**
546
+ * Audio input device is used for the audio track from the current stream
547
+ */
548
+ audioInput?: MediaDeviceInfoLike;
549
+ /**
550
+ * Video input device is used for the video track from the current stream
551
+ */
552
+ videoInput?: MediaDeviceInfoLike;
553
+ /**
554
+ * The audio input device which is expected to be used for the current
555
+ * stream based on the provided constraints
556
+ */
557
+ expectedAudioInput?: MediaDeviceInfoLike;
558
+ /**
559
+ * The video input device which is expected to be used for the current
560
+ * stream based on the provided constraints
561
+ */
562
+ expectedVideoInput?: MediaDeviceInfoLike;
563
+ /**
564
+ * The status of the media
565
+ */
566
+ status: UserMediaStatus;
567
+ /**
568
+ * Current mute state of audio track
569
+ * `undefined` means there is no such track from the stream
570
+ */
571
+ audioMuted: boolean | undefined;
572
+ /**
573
+ * Current mute state of video track
574
+ * `undefined` means there is no such track fromt the stream
575
+ */
576
+ videoMuted: boolean | undefined;
577
+ /**
578
+ * mute/unmute the audio track
579
+ */
580
+ muteAudio(mute: boolean): void;
581
+ /**
582
+ * mute/unmute the video track
583
+ */
584
+ muteVideo(mute: boolean): void;
585
+ /**
586
+ * Apply the constraints to the current media
587
+ */
588
+ applyConstraints(constraints: MediaDeviceRequest): Promise<void>;
589
+ /**
590
+ * Release the media resources, e.g. camera/microphone
591
+ */
592
+ release(): Promise<void>;
593
+ }
594
+ ```
595
+
596
+ ### Minor Changes
597
+
598
+ - acedce8d8: Decouple resolution check from device check
599
+
600
+ ## '@pexip/media-control'
601
+
602
+ - Decouple resolution check from `isRequestedInputDevice` and add a new
603
+ function to verify the resolution `isRequestedResolution`
604
+
605
+ ## '@pexip/media'
606
+
607
+ - Use `isRequestedResolution` to check the resolution to decide if we should
608
+ request the media again
609
+
610
+ - 19b7323d5: Add background blur feature
611
+
612
+ ## @pexip/aquila
613
+
614
+ - Add the following flags to `config` to control the features
615
+ - `videoSegmentation`: Enable/disable the video segmentation feature.
616
+ Default is `false`
617
+ - `blurLevel`: Control the blur level. Default is `7`.
618
+ - `segmentationEffects`: The segmentation effects to use when processing the
619
+ video. Available options are `none`, `blur` and `overlay`. Default is
620
+ `none`, aka no effects.
621
+ - Use the latest API from `media` package to add video segmentation feature
622
+ - Use 640x360 size for processing
623
+ - Keep an instance of segmentation for preview controller to avoid re-creation
624
+
625
+ ## @pexip/bundler
626
+
627
+ - Disable content hashing for wasm files since the way Personify SDK loads the
628
+ wasm file is hard-coded
629
+ - Add "babel-plugin-transform-import-meta" to fix babel-jest transform issue
630
+ of using `import.meta`
631
+
632
+ ## @pexip/media-demo
633
+
634
+ - Add video segmentation feature toggle to settings for switching on/off the
635
+ feature
636
+ - Add video segmentation feature
637
+
638
+ ## @pexip/media-processor-demo
639
+
640
+ - Add image loader to load image for background overlay and effects toggle
641
+ button
642
+ - Add blur level selector
643
+ - Add video segmentation feature
644
+
645
+ ## @pexip/bg-blur
646
+
647
+ - Add type definition
648
+ - Add @pexip/bg-blur package to host Personify SDK
649
+
650
+ ## @pexip/media
651
+
652
+ - Add `videoProcessor` to media processing pipeline
653
+ - Add lazy loading of the process for media pipeline
654
+ - Add `blurLevel`, `frameRate`, `psySegWasmPaths`, `segmentationSize` and
655
+ `getFeatureFlags` to `MediaOptions`
656
+ - Expose `segmentation` for `MediaController` and `PreviewStreamController`
657
+ - Add `getSegmentationOptions` and `videoSegmentationEnabled` to
658
+ `PreviewStreamParams`
659
+
660
+ ## @pexip/media-processor
661
+
662
+ - Add video processing related modules
663
+
664
+ - 89f90433a: Upgrade Personify SDK to v1.0.3
665
+
666
+ ## '@pexip/aquila'
667
+
668
+ - Upgrade `@pexip/bg-blur` version
669
+ - Add `erode` param to config
670
+
671
+ ## '@pexip/media-demo'
672
+
673
+ - Update accordingly
674
+
675
+ ## '@pexip/media-processor-demo'
676
+
677
+ - Update accordingly
678
+
679
+ ## '@pexip/bg-blur'
680
+
681
+ - Use Personify JS SDK v1.0.3
682
+ - Use npm package `psyseg-js-sdk` `v1.0.6`
683
+ - Add a workaround for yarn for `tensorflowjs` dependencies issue
684
+
685
+ ## '@pexip/media'
686
+
687
+ - Add `erode` property
688
+ - Allow explicitly skipping the script loading for `PreviewController`
689
+
690
+ ## '@pexip/media-processor'
691
+
692
+ - Upgrade according to `@pexip/bg-blur` changes
693
+
694
+ - 3b3883832: Deprecate `AudioStatsWorklet` and consolidate VAD
695
+
696
+ - Move `createVoiceActivityDetector` to @pexip/media-processor as
697
+ `createVADetector`
698
+ - Move `createAudioSignalDetector` to @pexip/media-processor
699
+ - Renamed `isSpeakingWhileMuted` to `isVoiceActivity`
700
+ - Remove AudioStatsWorklet and related
701
+ - Use VAD either from denoise or analyzer, but not both
702
+
703
+ - b3cbb8a59: Fallback input device notification
704
+
705
+ ## '@pexip/aquila'
706
+
707
+ - Add `useFallbackDeviceNotifier` hook and use it in Preflight
708
+ - Log the global `Notifyer` whenever there is a new toast.
709
+ - Fix an unnecessary update of the selected inputs when media is set to
710
+ `undefined`
711
+
712
+ ## '@pexip/media'
713
+
714
+ - Extract the device info and put it to
715
+ `expectedVideoInput`/`expectedAudioInput` when we cannot find the device
716
+ from the current device list for completeness.
717
+ - Use `isStreamingRequestedDevices` instead of
718
+ `isStreamingRequestedDevicesBase` to have more info about the track
719
+
720
+ ## '@pexip/media-control'
721
+
722
+ - Add a new utility function, `extractDeviceFromConstraints`, to extract the
723
+ device info from the provided constraints
724
+ - Update `isStreamingRequestedDevices` to allow passing `undefined` stream
725
+ param.
726
+
727
+ - 57eb593c0: Be more forgiveness when comparing input devices
728
+
729
+ When checking against the requested input device with the constraints
730
+ provided, it is better to use the label to verify them to avoid stale
731
+ `deviceId` issue.
732
+
733
+ ## '@pexip/media'
734
+
735
+ - Should always use permission granted devices to do the comparison
736
+
737
+ ## '@pexip/media-control'
738
+
739
+ - Use device's label to compare device when possible
740
+
741
+ - 07c09b3e2: Add background blur UIs
742
+
743
+ ## @pexip/aquila
744
+
745
+ - Add background blur input controls to Preflight and InMeeting The UIs are
746
+ only be shown when the `videoSegmentation` flag is set to `true` and there
747
+ is video track
748
+ - When the video is muted, the segmentation effects will be set to `none` to
749
+ avoid unnecessary processing, and the UI will be disabled
750
+
751
+ ## @pexip/media
752
+
753
+ - Set segmentation effects to `none` when the video is muted
754
+
755
+ - 00f9c290c: ## Support modular routing for Web Audio
756
+
757
+ From the spec of Web Audio API, we are allowed to do
758
+ [modular routing](https://webaudio.github.io/web-audio-api/#ModularRouting)
759
+ which allows arbitrary connections between different `AudioNode` objects.
760
+ However, in the initial design of our `AudioGraph`, this is not part of the
761
+ consideration. This MR aims to fill the gap between the Web Audio API and our
762
+ wrapper.
763
+
764
+ With the new API, we can declare the intended connections between any
765
+ `AudioNode`, e.g.
766
+
767
+ ```typescript
768
+ // Assuming we have a stereo audio input and would like visualize each channel independently
769
+ const stream = navigator.mediaDevices.getUserMedia({audio: true});
770
+ const source = createStreamSourceGraphNode(stream);
771
+ const splitter = createChannelSplitterGraphNode();
772
+ const left = createAnalyzerGraphNode();
773
+ const right = createAnalyzerGraphNode();
774
+ const destination = createStreamDestinationGraphNode();
775
+
776
+ // source |-- destination
777
+ // |-- splitter |-- left
778
+ // |-- right
779
+ // Note: Connection Param tuple: `[AudioNodeInit, outputIndex, inputIndex]`
780
+ const graph = createAudioGraph([
781
+ [source, destination],
782
+ [source, splitter],
783
+ [[splitter, 0, 0], left], // Connect the output#0 of the splitter to the `left` analyzer
784
+ [[splitter, 1, 0], right], // Connect the output#1 of the splitter to the `right` analyzer
785
+ ]);
786
+
787
+ // Later if you want to disconnect the analyzers
788
+ graph.disconnect([source, splitter]);
789
+ // Or connect the splitter again
790
+ graph.connect([source, splitter]);
791
+ // Check if there is any connection between 2 nodes
792
+ source.hasConnectedTo(splitter); // Expect `true` if they are connected, otherwise `false`
793
+ ```
794
+
795
+ ## Other notable changes
796
+
797
+ - Add `createChannelSplitterGraphNode`
798
+ - Add `AudioNodeInitConnectParam` for declaring the connection
799
+ - Remove `workletModules` from `AudioGraphOptions`
800
+ - Expose `AudioGraph['addWorklet']`
801
+
802
+ - 7e7dcfa30: Properly update the cache when the resolution changes
803
+
804
+ ### Patch Changes
805
+
806
+ - Updated dependencies [0a054710b]
807
+ - Updated dependencies [acedce8d8]
808
+ - Updated dependencies [19b7323d5]
809
+ - Updated dependencies [89f90433a]
810
+ - Updated dependencies [3b3883832]
811
+ - Updated dependencies [b3cbb8a59]
812
+ - Updated dependencies [57eb593c0]
813
+ - Updated dependencies [8e3e3b1f9]
814
+ - Updated dependencies [d62f2a28d]
815
+ - Updated dependencies [00f9c290c]
816
+ - Updated dependencies [730082765]
817
+ - Updated dependencies [048e384e7]
818
+ - @pexip/media-control@2.1.0
819
+ - @pexip/media-processor@3.0.0
820
+ - @pexip/utils@2.1.0
821
+ - @pexip/signal@2.0.1
822
+
823
+ ## 2.0.0
824
+
825
+ ### Minor Changes
826
+
827
+ - 0430bbad2: ## '@pexip/media'
828
+
829
+ Add mute state and related events
830
+
831
+ - Add `audioMuted` and `videoMuted`
832
+ - Connect `MediaStreamTrack` native events to signals
833
+ - Add and connect `onStreamTrackEnabled` signal
834
+ - Refactor `buildMedia`
835
+
836
+ ## '@pexip/media-demo'
837
+
838
+ Add mute state and related logics
839
+
840
+ ## '@pexip/media-control'
841
+
842
+ Change `createStreamTrackEventSubscriptions` to accept track instead of
843
+ stream.
844
+
845
+ - 7b2228e7b: Fix Settings flash issue caused by empty device ID
846
+ - 6fca55cbe: Fix media leak issues from Settings
847
+ - 0430bbad2: Allow mute state to be undefined
848
+ - cf2493ea9: Fix blocked device selection issues
849
+
850
+ ## @pexip/aquila
851
+
852
+ - Add i18n to `DeviceSelect` component
853
+ - Use `MissDeviceAlert` component only when there is denied device or no
854
+ device available for selection
855
+
856
+ ## @pexip/components
857
+
858
+ - Add `errorTextId` prop to `Select` component for testing
859
+
860
+ ## @pexip/media
861
+
862
+ - Add more props to Media
863
+
864
+ Add `status`, `expectedAudioInput`, `expectedVideoInput`, `constraints` to
865
+ `Media`
866
+
867
+ ## @pexip/media-control
868
+
869
+ - Add `findDeviceFromDeviceConstraints` util
870
+
871
+ Find device from the device list with provided constraints, i.e.
872
+ `InputDeviceConstraint` -> `MediaDeviceInfoLike | true | undefined`. It
873
+ returns `true` meaning that it can be any devices, `undefined` means device
874
+ not found, otherwise, the matched device will be returned
875
+
876
+ - 07caf0c47: Fix behavior signal `onDeviceChanged` issue
877
+
878
+ ## General
879
+
880
+ The behavior signal of `onDevicesChanged` can
881
+ [cause problem](https://gitlab.com/pexip/zoo/-/issues/1780) when there is no
882
+ device at all.
883
+
884
+ Indeed, there is no need to use a behaving signal, a generic one is good
885
+ enough for our usecase of `onDevicesChanged`. What we need to do before we
886
+ initiate the initial `getUserMedia` is to cross-check with the
887
+ `PermissionState` and the `devices`, and thus `tryAndGetUserMedia` API is
888
+ introduced to do the job.
889
+
890
+ ## Changes
891
+
892
+ ### fix(aquila): use preflight page to trigger media request
893
+
894
+ - Change `onDevicesChanged` to a `generic` signal, instead of `behavior`
895
+ - Use Preflight to trigger media request
896
+
897
+ closes https://gitlab.com/pexip/zoo/-/issues/1780
898
+
899
+ ### feat(aquila): add useGetUserMedia hook
900
+
901
+ - Add `useGetUserMedia` hook to trigger permission validation and initiate
902
+ media request
903
+
904
+ ### feat(media): new tryAndGetUserMedia API
905
+
906
+ - Add `tryAndGetUserMedia`
907
+ - Check if there is any actual changes of devices
908
+
909
+ ### refactor(media-control): for re-usability
910
+
911
+ - Refactor `createTrackDevicesChanges` to allow passing initial state
912
+
913
+ - 055c1d42f: # Fix incorrect selected device in Preflight device selector
914
+
915
+ When the media input device stored in the `localStorage` is stale, the
916
+ selected device from the `DeviceSelector` failed to show the correct selected
917
+ input. This is because we was passing the input config from the `localStorage`
918
+ directly to the `DeviceSelector` component and under some situation the
919
+ persisted input device preferences are no longer valid. e.g. Buy a new device
920
+ of the same model, OS/System update causing the change of `deviceId`, etc.
921
+
922
+ ## Steps to reproduce the issue
923
+
924
+ 1. Save your preference of the device from the App Settings/Preflight.
925
+ 1. Change the deviceId from the `localStorage`:
926
+ `aquila:videoInpiut`/`aquila:audioInput` to simulate the change of
927
+ `deviceId`
928
+ 1. Refresh the page, and then check the device selectors
929
+
930
+ ## Expected behavior
931
+
932
+ The Device selector should correctly show the device in-use
933
+
934
+ # Fix stale device list issue in Firefox
935
+
936
+ In Firefox, the life of granted devices can only live as long as the media
937
+ stream session, when the stream is released, all the granted devices are
938
+ revoked, i.e. the label will be empty again. However, the media package does
939
+ not capture this behavior. This causes inconsistent UIs from the Selfview and
940
+ the Device selection list.
941
+
942
+ ## Steps to reproduce the issue
943
+
944
+ 1. Go to Preflight
945
+ 1. Click "Request permissions" button
946
+ 1. Go back to dashboard, and then start a new meeting.
947
+
948
+ ## Expected behavior
949
+
950
+ The Device selector should show "Waiting for permissions..." when the
951
+ permission is not granted.
952
+
953
+ # Changes
954
+
955
+ ## @pexip/media
956
+
957
+ - Update `devices` prop when the status is `InitialPermissionNotGranted` to
958
+ avoid caching the outdated device list
959
+ - Move `resolveInput` from `aquila` into `media` package
960
+
961
+ ## @pexip/aquila
962
+
963
+ - Fix incorrect selected device in Preflight device selector
964
+ - Fix stale device list issue in Firefox
965
+
966
+ ## @pexip/media-control
967
+
968
+ - Enhance `mergeConstraints` utility to accept `InputDeviceConstraint`
969
+
970
+ - 6fca55cbe: Fix a future media release issue
971
+ - b0406fb6f: Mute with MediaStreamTrack['enabled'] instead of GainNode
972
+ - 329125c70: Fix active input device and new request constraints comparison
973
+
974
+ - Always use the latest device list when accessing the
975
+ `audioInput`/`videoInput` props from `Media`
976
+ - Remove fallback to `MediaStreamTrack` when accessing
977
+ `audioInput`/`videoInput` props from `Media` when doing comparison between
978
+ the active stream and new request
979
+
980
+ - 0430bbad2: Log muteAudio and muteVideo with track info
981
+ - 02d093bd3: Invalidate input cache when the related track is not live
982
+ - a0b8d9a6f: ## fix(aquila): avoid muting something which does not exists
983
+
984
+ Verify before we call the media mute functions
985
+
986
+ ## fix(media): the track will be enabled from the processor
987
+
988
+ Only mute when there is a track to mute to
989
+
990
+ - ed6b2d90e: Fix guessing error by elimination
991
+
992
+ We guess the media request error by eliminating the audio and video
993
+ constraints, if we cannot get a stream with the provided constraints. e.g.
994
+ instead of getting `PermissionsOnlyAudioinput` when there is an error with the
995
+ video input when requesting the device, the status for this will be
996
+ `VideoDeviceInUse`, if the error is `NotReadableError`.
997
+
998
+ Add the following `UserMediaStatus`
999
+
1000
+ - `Overconstrained`
1001
+ - `VideoOverconstrained`
1002
+ - `AudioOverconstrained`
1003
+ - `InvalidConstraints`
1004
+ - `InvalidVideoConstraints`
1005
+ - `InvalidAudioConstraints`
1006
+ - `NotSupportedError`
1007
+ - `NotSupportedErrorOnlyVideoInput`
1008
+ - `NotSupportedErrorOnlyAudioInput`
1009
+
1010
+ - 329125c70: Use exact constraints for preview controller
1011
+ - c1bf7f05c: Remove duplicated devices before checking changes
1012
+
1013
+ - Extract the logic to track device changes, and add unit test cases
1014
+ - Remove duplicated devices when tracking changes from `devicechange` event
1015
+ - Use `MediaDeviceInfoLike` instead `MediaDeviceInfo` for related events
1016
+
1017
+ A bug from Firefox (v88.0.1) 64-bit or above, there is chance that the devices
1018
+ we get from the API `enumerateDevices` are not unique with particular devices,
1019
+ e.g. Logitech BRIO. Since we are relying on the uniqueness for tracking the
1020
+ changes of device to fire `devices:found` or `devices:lost` events, this
1021
+ causes all the operations relying on these events, e.g. false positive
1022
+ `devices:found` event.
1023
+
1024
+ See https://gitlab.com/pexip/zoo/-/issues/1743
1025
+
1026
+ - f3a8057f4: Add rawStream to Media and usee raw MediaStream when available for
1027
+ preview controller
1028
+ - 29741a2d4: Log improvement and audio click fix
1029
+
1030
+ ## '@pexip/logger'
1031
+
1032
+ - Add what to log AnalyserNode
1033
+
1034
+ ## '@pexip/media'
1035
+
1036
+ - Remove unnecessary sync of audio mute
1037
+ - Add more logs
1038
+ - Add docs
1039
+
1040
+ ## '@pexip/media-processor'
1041
+
1042
+ - Reverse the release order to avoid click
1043
+
1044
+ ### Patch Changes
1045
+
1046
+ - 94a86d3c7: add states for no video devices during audio detection
1047
+ - 839436f8d: ## @pexip/media
1048
+
1049
+ - Only use `error` level when on the following errors when requesting media,
1050
+ otherwise, `warn` level
1051
+
1052
+ - `TypeError`
1053
+ - `NotSupportedError`
1054
+ - `UnknownError`
1055
+ - `OverconstrainedError` (when it is not `exact` device request)
1056
+
1057
+ ## @pexip/media-control
1058
+
1059
+ - Add `isExactDeviceConstraint` utility to check if the provided constraint is
1060
+ an exact constraint
1061
+
1062
+ - Updated dependencies [0430bbad2]
1063
+ - Updated dependencies [281f3afbe]
1064
+ - Updated dependencies [2655f552f]
1065
+ - Updated dependencies [649e27941]
1066
+ - Updated dependencies [cf2493ea9]
1067
+ - Updated dependencies [07caf0c47]
1068
+ - Updated dependencies [31c667453]
1069
+ - Updated dependencies [6ddadd2e5]
1070
+ - Updated dependencies [89ef2b1aa]
1071
+ - Updated dependencies [839436f8d]
1072
+ - Updated dependencies [029f5fb8a]
1073
+ - Updated dependencies [5ad1a1e92]
1074
+ - Updated dependencies [c1bf7f05c]
1075
+ - Updated dependencies [29741a2d4]
1076
+ - Updated dependencies [72641f2cd]
1077
+ - @pexip/media-control@2.0.0
1078
+ - @pexip/media-processor@2.0.0
1079
+ - @pexip/signal@2.0.0