@pexip/media-processor 16.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,1047 @@
1
+ # Changelog
2
+
3
+ ## 16.7.0
4
+
5
+ ### Minor Changes
6
+
7
+ - 20a44927: React on unmute track to resume suspended AudioContext
8
+
9
+ ### Patch Changes
10
+
11
+ - Updated dependencies [456c06c6]
12
+ - @pexip/utils@16.5.0
13
+
14
+ ## 16.6.0
15
+
16
+ ### Patch Changes
17
+
18
+ - Updated dependencies [3ba403f688]
19
+ - @pexip/utils@16.4.1
20
+
21
+ ## 16.5.0
22
+
23
+ ### Patch Changes
24
+
25
+ - Updated dependencies [e6ef1f2]
26
+ - @pexip/utils@16.4.0
27
+
28
+ ## 16.4.0
29
+
30
+ ### Minor Changes
31
+
32
+ - c1c3767: Avoid error when there is no MediaSource available
33
+
34
+ ### Patch Changes
35
+
36
+ - Updated dependencies [c71c68f]
37
+ - @pexip/bg-blur@16.4.0
38
+
39
+ ## 16.3.1
40
+
41
+ ### Patch Changes
42
+
43
+ - Updated dependencies [4763e5b4]
44
+ - @pexip/utils@16.3.1
45
+
46
+ ## 16.3.0
47
+
48
+ ### Patch Changes
49
+
50
+ - @pexip/bg-blur@16.3.0
51
+ - @pexip/denoise@16.3.0
52
+ - @pexip/utils@16.3.0
53
+
54
+ ## 16.2.0
55
+
56
+ ### Patch Changes
57
+
58
+ - @pexip/bg-blur@16.2.0
59
+ - @pexip/denoise@16.2.0
60
+ - @pexip/utils@16.2.0
61
+
62
+ ## 16.1.0
63
+
64
+ ### Patch Changes
65
+
66
+ - @pexip/bg-blur@16.1.0
67
+ - @pexip/denoise@16.1.0
68
+ - @pexip/utils@16.1.0
69
+
70
+ ## 16.0.0
71
+
72
+ ### Patch Changes
73
+
74
+ - @pexip/bg-blur@16.0.0
75
+ - @pexip/denoise@16.0.0
76
+ - @pexip/utils@16.0.0
77
+
78
+ ## 15.0.0
79
+
80
+ ### Patch Changes
81
+
82
+ - @pexip/bg-blur@15.0.0
83
+ - @pexip/denoise@15.0.0
84
+ - @pexip/utils@15.0.0
85
+
86
+ ## 14.1.0
87
+
88
+ ### Patch Changes
89
+
90
+ - @pexip/bg-blur@14.1.0
91
+ - @pexip/denoise@14.1.0
92
+ - @pexip/utils@14.1.0
93
+
94
+ ## 14.0.0
95
+
96
+ ### Patch Changes
97
+
98
+ - @pexip/bg-blur@14.0.0
99
+ - @pexip/denoise@14.0.0
100
+ - @pexip/utils@14.0.0
101
+
102
+ ## 13.2.0
103
+
104
+ ### Patch Changes
105
+
106
+ - Updated dependencies [f450282b]
107
+ - @pexip/bg-blur@13.2.0
108
+ - @pexip/denoise@13.2.0
109
+ - @pexip/utils@13.2.0
110
+
111
+ ## 13.1.0
112
+
113
+ ### Patch Changes
114
+
115
+ - @pexip/bg-blur@13.1.0
116
+ - @pexip/denoise@13.1.0
117
+ - @pexip/utils@13.1.0
118
+
119
+ ## 13.0.0
120
+
121
+ ### Patch Changes
122
+
123
+ - @pexip/bg-blur@13.0.0
124
+ - @pexip/denoise@13.0.0
125
+ - @pexip/utils@13.0.0
126
+
127
+ ## 12.0.0
128
+
129
+ ### Major Changes
130
+
131
+ - 8caca489: Remove Personify model for security reasons
132
+
133
+ ### Patch Changes
134
+
135
+ - Updated dependencies [8caca489]
136
+ - @pexip/bg-blur@12.0.0
137
+ - @pexip/denoise@12.0.0
138
+ - @pexip/utils@12.0.0
139
+
140
+ ## 11.0.0
141
+
142
+ ### Patch Changes
143
+
144
+ - @pexip/bg-blur@11.0.0
145
+ - @pexip/denoise@11.0.0
146
+ - @pexip/utils@11.0.0
147
+
148
+ ## 10.1.0
149
+
150
+ ### Patch Changes
151
+
152
+ - @pexip/bg-blur@10.1.0
153
+ - @pexip/denoise@10.1.0
154
+ - @pexip/utils@10.1.0
155
+
156
+ ## 10.0.0
157
+
158
+ ### Patch Changes
159
+
160
+ - @pexip/bg-blur@10.0.0
161
+ - @pexip/denoise@10.0.0
162
+ - @pexip/utils@10.0.0
163
+
164
+ ## 9.0.0
165
+
166
+ ### Patch Changes
167
+
168
+ - @pexip/bg-blur@9.0.0
169
+ - @pexip/denoise@9.0.0
170
+ - @pexip/utils@9.0.0
171
+
172
+ ## 8.0.0
173
+
174
+ ### Patch Changes
175
+
176
+ - @pexip/bg-blur@8.0.0
177
+ - @pexip/denoise@8.0.0
178
+ - @pexip/utils@8.0.0
179
+
180
+ ## 7.0.0
181
+
182
+ ### Minor Changes
183
+
184
+ - efc7d2a8: Better media logging
185
+
186
+ ## @pexip/logger
187
+
188
+ - Add more AudioNode to log
189
+
190
+ ## @pexip/media
191
+
192
+ - Truncate unnecessary fields when logging with JSON
193
+
194
+ ## @pexip/media-control
195
+
196
+ - Put MediaDeviceInfoKind['settings'] fields into toJSON for logging
197
+
198
+ ## @pexip/media-processor
199
+
200
+ - Drop logger dependency and related usages
201
+
202
+ - cbe3ccf1: Ensure to resolve play video promise
203
+
204
+ ### Patch Changes
205
+
206
+ - Updated dependencies [16854bc7]
207
+ - @pexip/utils@7.0.0
208
+ - @pexip/bg-blur@7.0.0
209
+ - @pexip/denoise@7.0.0
210
+
211
+ ## 6.0.0
212
+
213
+ ### Minor Changes
214
+
215
+ - 0522557c: Fix older version AbortSignal
216
+
217
+ AbortSignal['reason'] is supported from Chromium v98 and Firefox v97. Before
218
+ that, there is no way to identify the abort reason so we should simply ignore
219
+ this abort error. Ref.
220
+ https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal#browser_compatibility
221
+ closes #3158
222
+
223
+ ### Patch Changes
224
+
225
+ - @pexip/bg-blur@6.0.0
226
+ - @pexip/denoise@6.0.0
227
+ - @pexip/logger@6.0.0
228
+ - @pexip/utils@6.0.0
229
+
230
+ ## 5.0.0
231
+
232
+ ### Minor Changes
233
+
234
+ - 9f68146d: # Presentation with audio.
235
+
236
+ Be able to share presentation with audio.
237
+
238
+ ## Changes on `media`
239
+
240
+ Introduce `audioMixingProcessor` to mix audio track with additional one.
241
+
242
+ ## Changes on `media-control`
243
+
244
+ Introduce `mixWithAdditionalMedia` constraint.
245
+
246
+ ## Changes on `media-components`
247
+
248
+ Pass a cleanup `fn` to `usePresentation` hook.
249
+
250
+ ## Changes on `media-processor`
251
+
252
+ Add `merger` node constructor.
253
+
254
+ ### Patch Changes
255
+
256
+ - Updated dependencies [2e36be80]
257
+ - @pexip/utils@5.0.0
258
+ - @pexip/bg-blur@5.0.0
259
+ - @pexip/denoise@5.0.0
260
+ - @pexip/logger@5.0.0
261
+
262
+ ## 4.0.0
263
+
264
+ ### Major Changes
265
+
266
+ - cf8bec29b6: # Use Streams API
267
+
268
+ ## Changes on `media-processor`
269
+
270
+ - Use Streams API for video processing, fallback to
271
+ requestVideoFrameCallback/setTimeout when the API is not available
272
+ - Turn off alpha when producing the final drawing on canvas
273
+ - closes https://gitlab.com/pexip/zoo/-/issues/2186
274
+ - Use desynchronized hint for canvas
275
+ - Fix edge issue when composing blur background
276
+ - Fix process `opened` state issue on async function
277
+
278
+ ### Commits
279
+
280
+ - fc398f4b38 perf(media-processor): use desynchronized hint
281
+ - ed2d95c653 feat(media-processor): use stream transform
282
+ - da53143a49 fix(media-processor): dom-webcodecs types
283
+ - e271047b0f feat(media-processor): VideoProcessor accept transformers
284
+ - eeb8992f97 feat(media-processor): add transformer creator
285
+ - 20d085aac2 feat(media-processor): add MediaStreamTrack generator & processor
286
+ - 096f1adc77 perf(media-processor): turn off alpha for none effect
287
+ - a1a282c34a refactor(media-processor): move canvas related into
288
+ canvasRenderer
289
+
290
+ ## Changes on `media`
291
+
292
+ - Use the latest APIs from `media-processor`
293
+ - Fix types overridden by `dom-webcodecs`
294
+
295
+ ### Commits
296
+
297
+ - 0156012aab fix(media): use streams API from media-processor
298
+ - e61ca85375 fix(media): type overrides
299
+
300
+ ## Changes on `media-control`
301
+
302
+ - Fix types overridden by `dom-webcodecs`
303
+
304
+ ### Commit
305
+
306
+ - 0a63010d20 fix(media-control): type overrides
307
+
308
+ ## Changes on Apps
309
+
310
+ - Use the latest APIs
311
+
312
+ ### Commits
313
+
314
+ - b7a768a350 feat(infinity-connect): turn on video processing for Safari
315
+ - d05dc48b8f fix(media-processor-demo): remove comments
316
+ - 6d58d0c987 fix(media-processor-demo): add missing await
317
+ - 4859029594 fix(infinity-connect): apply latest changes from media
318
+ - 3c30502ee7 fix(aquila): use latest changes from media
319
+ - 179359a95d fix(media-demo): update settings as well
320
+ - e3ac0cdd0e fix(media-demo): apply latest changes from media
321
+ - f251c75f16 feat(media-processor-demo): use createCanvasTransform
322
+
323
+ ## Others
324
+
325
+ - Add types/dom-webcodecs
326
+
327
+ ### Commit
328
+
329
+ - 2458690b46 build: add types/dom-webcodecs
330
+
331
+ ### Patch Changes
332
+
333
+ - d429f07df2: ### Changes
334
+
335
+ 733b6c099f chore(deps): update react monorepo to v18 (major) d59cd19c89
336
+ perf(media-processor): use image size d642488120 perf(media-processor): more
337
+ canvas tweaks 348e0ebbb2 perf(media-processor): skip edge blurring
338
+ optimization 59d6fa8cce feat(media-processor): use requestVideoFrameCallback
339
+ e97aeec800 fix(media-processor): use input resolution on canvas when
340
+ processing is not applied 972f4b8109 fix(deps): update rust crate nnnoiseless
341
+ to 0.4.0 df6972a4f4 Merge branch 'renovate/getrandom-0.x' into 'main'
342
+ 5527c14276 fix(deps): update rust crate getrandom to 0.2.6 11a4eb2a00
343
+ chore(deps): update rust crate wasm-bindgen-test to 0.3.30 6a2e06cdb4
344
+ fix(media): bgImageUrl constraint feature d285413fcb
345
+ feat(media,media-processor): add facemesh 0b8a98a71b chore(deps): update
346
+ dependency typescript to ~4.6.0 8fba551b62 build: bump @pexip/ 0ecad34992
347
+ refactor(aquila): move Settings logic to useSettingsModal and move to
348
+ media-components d6a8a63dcc fix(deps): update rust crate rand_distr to 0.4.3
349
+ 923f4a571d chore(deps): lock file maintenance 4c482279eb fix(deps): update
350
+ dependency typescript to ~4.5.5 dc4778a8e0 Merge branch
351
+ 'renovate/wasm-bindgen-test-0.x' into 'main' 1bd8cc1728 Merge branch
352
+ 'renovate/rand-0.x' into 'main' 0a314afb50 Merge branch
353
+ 'renovate/getrandom-0.x' into 'main' f143902ee5 Selfie segmentation
354
+ integration eaab7b6bff build: build workspace with turbo 764ce1b085 fix(deps):
355
+ update rust crate getrandom to 0.2.5 e2b67c02bf chore(deps): lock file
356
+ maintenance 2176590e3a fix(deps): update rust crate rand to 0.8.5 87444b046c
357
+ chore(deps): lock file maintenance 9dc8d65577 fix(media-processor): AudioNode
358
+ related issues 1ed12aba0b chore: update, normalize, and sort all package.json
359
+ files b64030565a feat(infinity-connect): one time flow first cut 00f9c290c0
360
+ feat(media-processor): support multi connect audio node 09dd205f0c
361
+ chore(deps): update rust crate wasm-bindgen-test to 0.3.29 e1ada64e65
362
+ fix(deps): update rust crate console_error_panic_hook to 0.1.7 6856a69269
363
+ fix(media-processor): band-aid to fix storybook deb6609b59 feat(media): Add
364
+ AudioDestionationNodeInit and DelayNodeInit a96bd188e9 feat(media): add
365
+ background overlay and removal 8e3e3b1f98 refactor(media): follow the follow
366
+ media constraints pattern b0b1e1bce2 build(media-processor): upgrade
367
+ nnnoiseless to v0.3.3 e5002561db build(aquila): build and dev with vite
368
+
369
+ - @pexip/bg-blur@4.0.0
370
+ - @pexip/denoise@4.0.0
371
+ - @pexip/logger@4.0.0
372
+ - @pexip/utils@4.0.0
373
+
374
+ ## 3.0.0
375
+
376
+ ### Major Changes
377
+
378
+ - 3b3883832: Deprecate `AudioStatsWorklet` and consolidate VAD
379
+
380
+ - Move `createVoiceActivityDetector` to @pexip/media-processor as
381
+ `createVADetector`
382
+ - Move `createAudioSignalDetector` to @pexip/media-processor
383
+ - Renamed `isSpeakingWhileMuted` to `isVoiceActivity`
384
+ - Remove AudioStatsWorklet and related
385
+ - Use VAD either from denoise or analyzer, but not both
386
+
387
+ - 00f9c290c: ## Support modular routing for Web Audio
388
+
389
+ From the spec of Web Audio API, we are allowed to do
390
+ [modular routing](https://webaudio.github.io/web-audio-api/#ModularRouting)
391
+ which allows arbitrary connections between different `AudioNode` objects.
392
+ However, in the initial design of our `AudioGraph`, this is not part of the
393
+ consideration. This MR aims to fill the gap between the Web Audio API and our
394
+ wrapper.
395
+
396
+ With the new API, we can declare the intended connections between any
397
+ `AudioNode`, e.g.
398
+
399
+ ```typescript
400
+ // Assuming we have a stereo audio input and would like visualize each channel independently
401
+ const stream = navigator.mediaDevices.getUserMedia({audio: true});
402
+ const source = createStreamSourceGraphNode(stream);
403
+ const splitter = createChannelSplitterGraphNode();
404
+ const left = createAnalyzerGraphNode();
405
+ const right = createAnalyzerGraphNode();
406
+ const destination = createStreamDestinationGraphNode();
407
+
408
+ // source |-- destination
409
+ // |-- splitter |-- left
410
+ // |-- right
411
+ // Note: Connection Param tuple: `[AudioNodeInit, outputIndex, inputIndex]`
412
+ const graph = createAudioGraph([
413
+ [source, destination],
414
+ [source, splitter],
415
+ [[splitter, 0, 0], left], // Connect the output#0 of the splitter to the `left` analyzer
416
+ [[splitter, 1, 0], right], // Connect the output#1 of the splitter to the `right` analyzer
417
+ ]);
418
+
419
+ // Later if you want to disconnect the analyzers
420
+ graph.disconnect([source, splitter]);
421
+ // Or connect the splitter again
422
+ graph.connect([source, splitter]);
423
+ // Check if there is any connection between 2 nodes
424
+ source.hasConnectedTo(splitter); // Expect `true` if they are connected, otherwise `false`
425
+ ```
426
+
427
+ ## Other notable changes
428
+
429
+ - Add `createChannelSplitterGraphNode`
430
+ - Add `AudioNodeInitConnectParam` for declaring the connection
431
+ - Remove `workletModules` from `AudioGraphOptions`
432
+ - Expose `AudioGraph['addWorklet']`
433
+
434
+ ### Minor Changes
435
+
436
+ - 19b7323d5: Add background blur feature
437
+
438
+ ## @pexip/aquila
439
+
440
+ - Add the following flags to `config` to control the features
441
+ - `videoSegmentation`: Enable/disable the video segmentation feature.
442
+ Default is `false`
443
+ - `blurLevel`: Control the blur level. Default is `7`.
444
+ - `segmentationEffects`: The segmentation effects to use when processing the
445
+ video. Available options are `none`, `blur` and `overlay`. Default is
446
+ `none`, aka no effects.
447
+ - Use the latest API from `media` package to add video segmentation feature
448
+ - Use 640x360 size for processing
449
+ - Keep an instance of segmentation for preview controller to avoid re-creation
450
+
451
+ ## @pexip/bundler
452
+
453
+ - Disable content hashing for wasm files since the way Personify SDK loads the
454
+ wasm file is hard-coded
455
+ - Add "babel-plugin-transform-import-meta" to fix babel-jest transform issue
456
+ of using `import.meta`
457
+
458
+ ## @pexip/media-demo
459
+
460
+ - Add video segmentation feature toggle to settings for switching on/off the
461
+ feature
462
+ - Add video segmentation feature
463
+
464
+ ## @pexip/media-processor-demo
465
+
466
+ - Add image loader to load image for background overlay and effects toggle
467
+ button
468
+ - Add blur level selector
469
+ - Add video segmentation feature
470
+
471
+ ## @pexip/bg-blur
472
+
473
+ - Add type definition
474
+ - Add @pexip/bg-blur package to host Personify SDK
475
+
476
+ ## @pexip/media
477
+
478
+ - Add `videoProcessor` to media processing pipeline
479
+ - Add lazy loading of the process for media pipeline
480
+ - Add `blurLevel`, `frameRate`, `psySegWasmPaths`, `segmentationSize` and
481
+ `getFeatureFlags` to `MediaOptions`
482
+ - Expose `segmentation` for `MediaController` and `PreviewStreamController`
483
+ - Add `getSegmentationOptions` and `videoSegmentationEnabled` to
484
+ `PreviewStreamParams`
485
+
486
+ ## @pexip/media-processor
487
+
488
+ - Add video processing related modules
489
+
490
+ - 89f90433a: Upgrade Personify SDK to v1.0.3
491
+
492
+ ## '@pexip/aquila'
493
+
494
+ - Upgrade `@pexip/bg-blur` version
495
+ - Add `erode` param to config
496
+
497
+ ## '@pexip/media-demo'
498
+
499
+ - Update accordingly
500
+
501
+ ## '@pexip/media-processor-demo'
502
+
503
+ - Update accordingly
504
+
505
+ ## '@pexip/bg-blur'
506
+
507
+ - Use Personify JS SDK v1.0.3
508
+ - Use npm package `psyseg-js-sdk` `v1.0.6`
509
+ - Add a workaround for yarn for `tensorflowjs` dependencies issue
510
+
511
+ ## '@pexip/media'
512
+
513
+ - Add `erode` property
514
+ - Allow explicitly skipping the script loading for `PreviewController`
515
+
516
+ ## '@pexip/media-processor'
517
+
518
+ - Upgrade according to `@pexip/bg-blur` changes
519
+
520
+ - 8e3e3b1f9: Refactor media package to follow the media constraint pattern
521
+
522
+ https://developer.mozilla.org/en-US/docs/Web/API/Media_Streams_API/Constraints
523
+
524
+ - Move and extend the media features based on MediaTrackConstraints by
525
+ interpreting the provided constraints
526
+ - Voice Activity Detection - `vad` of type `boolean`
527
+ - Audio Signal Detection - `asd` of type `boolean`
528
+ - Noise suppression - `denoise` (to avoid the conflict of the built-in
529
+ `noiseSupression`) of type `boolean`, `true` means using our own impl,
530
+ remember to disable the built-in `noiseSupression` as you don't want to
531
+ use them both
532
+ - Blur level - `blurLevel` of type `number`
533
+ - Erode param - `erode` of type `number`
534
+ - Video segmentation effects - `videoSegmentation` of type `none`, `blur`,
535
+ `overlay` or `remove` (`remove` will be added in another MR)
536
+ - Ad-hoc change of the features should be done via the `Media` object with the
537
+ new `applyConstraints` API
538
+ - `getUserMedia` is always getting a new media while applying the features
539
+ based on the constraints when there is change to the constraints
540
+
541
+ Example:
542
+
543
+ ```typescript
544
+ const constriants = {
545
+ audio: {
546
+ noiseSuppression: false, // Disable the built-in noise suppression
547
+ denoise: true, // Use our own impl of noise suppression
548
+ vad: true, // Use Voice Activity Detection
549
+ asd: true, // Use Audio Signal Detection to check if the microphone is malfunctioned
550
+ },
551
+ video: {
552
+ videoSegmentation: 'blur', // Use background blur
553
+ erode: 1,
554
+ blurLevel: 21,
555
+ },
556
+ };
557
+ ```
558
+
559
+ The new `MediaController` interface (removed feature-related props):
560
+
561
+ ```typescript
562
+ export interface MediaController {
563
+ /**
564
+ * Current Media if any
565
+ */
566
+ media: Media;
567
+ /**
568
+ * Current device list
569
+ */
570
+ devices: MediaDeviceInfoLike[];
571
+ /**
572
+ * Execute the media pipeline immediately with provided constraints
573
+ *
574
+ * @param constraints - @see MediaDeviceRequest
575
+ */
576
+ getUserMedia: (constraints: MediaDeviceRequest) => void;
577
+ /**
578
+ * Execute the media pipeline immediately with provided constraints. An
579
+ * explicit version of `getUserMedia` to make it possible to chain other
580
+ * async actions
581
+ *
582
+ * @param constraints - @see MediaDeviceRequest
583
+ */
584
+ getUserMediaAsync: (constraints: MediaDeviceRequest) => Promise<void>;
585
+ /**
586
+ * Cross-check PermissionState and available devices before requesting user
587
+ * media, the request will be skipped if there is no granted device.
588
+ */
589
+ tryAndGetUserMedia: (constraints: MediaDeviceRequest) => void;
590
+ }
591
+ ```
592
+
593
+ The new `Media` interface (added `applyConstraints`):
594
+
595
+ ```typescript
596
+ export interface Media {
597
+ /**
598
+ * The constraints used to request the media
599
+ */
600
+ constraints?: MediaDeviceRequest;
601
+ /**
602
+ * The devices used for the media
603
+ */
604
+ devices: MediaDeviceInfoLike[];
605
+ /**
606
+ * Media stream for the media
607
+ */
608
+ stream?: MediaStream;
609
+ /**
610
+ * The raw stream obtained from the `getUserMedia` API
611
+ */
612
+ rawStream?: MediaStream;
613
+ /**
614
+ * Audio input device is used for the audio track from the current stream
615
+ */
616
+ audioInput?: MediaDeviceInfoLike;
617
+ /**
618
+ * Video input device is used for the video track from the current stream
619
+ */
620
+ videoInput?: MediaDeviceInfoLike;
621
+ /**
622
+ * The audio input device which is expected to be used for the current
623
+ * stream based on the provided constraints
624
+ */
625
+ expectedAudioInput?: MediaDeviceInfoLike;
626
+ /**
627
+ * The video input device which is expected to be used for the current
628
+ * stream based on the provided constraints
629
+ */
630
+ expectedVideoInput?: MediaDeviceInfoLike;
631
+ /**
632
+ * The status of the media
633
+ */
634
+ status: UserMediaStatus;
635
+ /**
636
+ * Current mute state of audio track
637
+ * `undefined` means there is no such track from the stream
638
+ */
639
+ audioMuted: boolean | undefined;
640
+ /**
641
+ * Current mute state of video track
642
+ * `undefined` means there is no such track fromt the stream
643
+ */
644
+ videoMuted: boolean | undefined;
645
+ /**
646
+ * mute/unmute the audio track
647
+ */
648
+ muteAudio(mute: boolean): void;
649
+ /**
650
+ * mute/unmute the video track
651
+ */
652
+ muteVideo(mute: boolean): void;
653
+ /**
654
+ * Apply the constraints to the current media
655
+ */
656
+ applyConstraints(constraints: MediaDeviceRequest): Promise<void>;
657
+ /**
658
+ * Release the media resources, e.g. camera/microphone
659
+ */
660
+ release(): Promise<void>;
661
+ }
662
+ ```
663
+
664
+ - d62f2a28d: Fix buffer type def
665
+
666
+ ## '@pexip/bg-blur'
667
+
668
+ - PsySegBuf['data'] can be `null`
669
+
670
+ ## '@pexip/media-processor'
671
+
672
+ - Avoid drawing something doesn't exist
673
+
674
+ When there is no data is written into the output buffer, (possibly a bug
675
+ from the Personify), we should avoid drawing, and just log the incidence.
676
+
677
+ - 730082765: Enable video segmentation
678
+
679
+ ## @pexip/aquila
680
+
681
+ - Enable video segmentation for Chrome (not running from mobile)
682
+ - Persist blur effects when it is in Preflight
683
+
684
+ ## @pexip/media-processor-demo
685
+
686
+ - Cleanup
687
+
688
+ ## @pexip/media-processor
689
+
690
+ - Refactor and add more logs
691
+
692
+ ### Patch Changes
693
+
694
+ - Updated dependencies [19b7323d5]
695
+ - Updated dependencies [89f90433a]
696
+ - Updated dependencies [3b3883832]
697
+ - Updated dependencies [d62f2a28d]
698
+ - Updated dependencies [46b70ba43]
699
+ - @pexip/bg-blur@1.1.0
700
+ - @pexip/utils@2.1.0
701
+ - @pexip/logger@2.1.0
702
+
703
+ ## 2.0.0
704
+
705
+ ### Minor Changes
706
+
707
+ - 2655f552f: Add timeout delay constant to Analyzer timeout subscription
708
+ - 29741a2d4: Log improvement and audio click fix
709
+
710
+ ## '@pexip/logger'
711
+
712
+ - Add what to log AnalyserNode
713
+
714
+ ## '@pexip/media'
715
+
716
+ - Remove unnecessary sync of audio mute
717
+ - Add more logs
718
+ - Add docs
719
+
720
+ ## '@pexip/media-processor'
721
+
722
+ - Reverse the release order to avoid click
723
+
724
+ ### Patch Changes
725
+
726
+ - Updated dependencies [29741a2d4]
727
+ - Updated dependencies [958250434]
728
+ - Updated dependencies [04752bc9d]
729
+ - Updated dependencies [945ca1989]
730
+ - Updated dependencies [29741a2d4]
731
+ - @pexip/logger@2.0.0
732
+ - @pexip/utils@2.0.0
733
+
734
+ ## 1.4.0
735
+
736
+ ### Minor Changes
737
+
738
+ - 5c9120efd: Track changes using changesets
739
+
740
+ ### Patch Changes
741
+
742
+ - Updated dependencies [5c9120efd]
743
+ - @pexip/denoise@1.4.0
744
+ - @pexip/logger@1.4.0
745
+ - @pexip/utils@1.4.0
746
+
747
+ ## [0.10.7] - 2021-03-11
748
+
749
+ ### Fixed
750
+
751
+ - Add a guard to avoid `TypeError: The provided value is non-finite`
752
+ - A potential a node is removed but not disconnected
753
+
754
+ ## [0.10.6] - 2021-03-08
755
+
756
+ ### Changed
757
+
758
+ - Separate states per channel for denoise processing
759
+
760
+ ### Added
761
+
762
+ - Add `build:denoise` script to build denoise wasm
763
+
764
+ ## [0.10.5] - 2021-03-07
765
+
766
+ ## Fixed
767
+
768
+ - don't use stale audio context state in onstatechange [#3750]
769
+
770
+ ## [0.10.4] - 2021-03-04
771
+
772
+ ## Fixed
773
+
774
+ - Add guard in async `insertNode` to avoid `InvalidAccessError`
775
+
776
+ ## [0.10.3] - 2021-03-03
777
+
778
+ ## Fixed
779
+
780
+ - Should not include `AudioWorkletProcessor` in the `hasAudioWorklet()` check
781
+
782
+ ## [0.10.2] - 2021-03-03
783
+
784
+ ## Fixed
785
+
786
+ - Log Proxied method only when it is called
787
+ - Return `false` to set trap to avoid change to the target properties
788
+
789
+ ## [0.10.1] - 2021-03-03
790
+
791
+ ## Fixed
792
+
793
+ - Logging the lifecycle
794
+
795
+ ## Added
796
+
797
+ - Add an option to pass `AudioContext` to `AudioGraph`
798
+ - Export `createAudioContext` utility
799
+
800
+ ## [0.10.0] - 2021-03-02
801
+
802
+ ### Fixed
803
+
804
+ - `AudioWorklet` browser compatibility issues
805
+
806
+ ### Added
807
+
808
+ - `AudioGraph['insertNode']`
809
+
810
+ `AudioGraph` is changed to be built in 2 phases if there is any `Worklet` node
811
+ needed:
812
+
813
+ - Phase 1: Build and connect Native `AudioNode`
814
+ - Phase 2: Add `AudioWorklet` module and then insert the `AudioWorkletNode`
815
+ into the graph when the `Worklet` is ready (`async`)
816
+
817
+ - `createAudioStatsGraphNode`: A wrapper to either create an
818
+ `AudioStatsWorkletNodeInit` or `AnalyzerNodeInit`
819
+
820
+ ### Removed
821
+
822
+ - **Breaking Change**: `AudioGraph['addNode']` is removed, use
823
+ `AudioGraph['insertNode']` instead
824
+
825
+ ### Changed
826
+
827
+ - Use Proxy for `AudioNodeInit` creation
828
+ - Rename `createAudioStatsGraphNode` to `createAudioStatsAnalyzerGraphNode`
829
+
830
+ ## [0.9.1] - 2021-02-26
831
+
832
+ ### Fixed
833
+
834
+ - Fix accessor issues for AudioNodeInit creation
835
+
836
+ ## [0.9.0] - 2021-02-24
837
+
838
+ ### Changed
839
+
840
+ - Rename to `@pexip/media-processor`
841
+ - Remove `api-extractor` and related
842
+ - Refactor the package for scalability and flexibility
843
+
844
+ Changed **from**
845
+
846
+ ```typescript
847
+ const fftSize = 64;
848
+ const analyser = createAudioAnalyzer(stream, {controlGain: true, fftSize});
849
+ analyzer.setMute(true); // to mute
850
+ analyzer.mute; // to get the mute state
851
+
852
+ analyzer.stream; // to get the stream
853
+
854
+ const buffer = new Float32Array(fftSize);
855
+ const volume = analyzer.getAverageVolume(buffer); // to get the average volume
856
+
857
+ await analyzer.release();
858
+ ```
859
+
860
+ **to**
861
+
862
+ ```typescript
863
+ const fftSize = 64;
864
+ const source = createStreamSourceGraphNode(stream);
865
+ const analyzer = createAnalyzerGraphNode({fftSize});
866
+ const gain = createGainGraphNode(false);
867
+ const destination = createStreamDestinationGraphNode();
868
+ const graph = createAudioGraph([source, analyzer, gain, destination]);
869
+
870
+ if (gain.node) {
871
+ gain.node.mute = true; // to mute
872
+ }
873
+ gain.node?.mute; // get the mute state
874
+
875
+ destination.node.stream; // to get the stream
876
+
877
+ const buffer = new Float32Array(fftSize);
878
+ const volume = analyzer.node?.getAverageVolume(buffer); // to get the average volume
879
+
880
+ await graph.release();
881
+ ```
882
+
883
+ ### Added
884
+
885
+ - Denoise wasm module and AudioWorklet for noise suppression
886
+
887
+ - Add a sub folder, `denoise` to host the source code of denoise WebAssembly
888
+
889
+ - APIs to create `AudioNodeInit` for building the `AudioGraph`
890
+
891
+ - `createStreamSourceGraphNode(mediaStream: MediaStream): MediaStreamAudioSourceNodeInit`
892
+ - `createAudioStatsGraphNode(options: StatsProcessorOptions & AnalyserOptions): AnalyzerNodeInit`
893
+ - `createAudioStatsWorkletGraphNode(StatsProcessorOptions & AnalyserOptions): AudioStatsWorkletNodeInit`
894
+ - `createDenoiseWorkletGraphNode(data: BufferSource): DenoiseWorkletNodeInit`
895
+ - `createGainGraphNode(mute: boolean): GainNodeInit`
896
+ - `createAnalyzerGraphNode(options?: AnalyserOptions): AnalyzerNodeInit`
897
+ - `createStreamDestinationGraphNode(options?: AudioNodeOptions): MediaStreamAudioDestinationNodeInit`
898
+
899
+ - API to create `AudioGraph`
900
+ - `createAudioGraph(inits: AudioNodeInit[], options?: AudioGraphOptions): AudioGraph`
901
+
902
+ ### Removed
903
+
904
+ - Breaking Change: `createAudioAnalyzer` and related are removed.
905
+
906
+ ## [0.8.4] - 2021-02-16
907
+
908
+ ### Fixed
909
+
910
+ - Subscribe to the AudioContext's `onstatechange` to undo Safari's stream
911
+ interruptions [#3205](https://github.com/pexip/aquila-research/issues/3205)
912
+
913
+ ## [0.8.3] - 2021-02-02
914
+
915
+ ### Fixed
916
+
917
+ - Add safe guard to avoid unnecessary calling AudioContext['close'] when it is
918
+ in-process to fix
919
+ [#3172](https://github.com/pexip/aquila-research/issues/3172)
920
+
921
+ ## [0.8.2] - 2020-11-18
922
+
923
+ ### Fixed
924
+
925
+ - Gain node error on safari [#2215]
926
+
927
+ ## [0.8.1] - 2020-11-02
928
+
929
+ ### Fixed
930
+
931
+ - Create `GainNode` to control volume with disabled track [#2119]
932
+
933
+ ### Changed
934
+
935
+ - Remove `onAudioStats` message debug log to be less noisy [#2119]
936
+
937
+ ## [0.8.0] - 2020-10-30
938
+
939
+ ### Added
940
+
941
+ - Add debug logging using @pexip/logger [#2089]
942
+
943
+ ### Fixed
944
+
945
+ - Fix infinite timeout call when using fallback mode for `onAudioStats` function
946
+ [#2089]
947
+
948
+ ## [0.7.3] - 2020-10-30
949
+
950
+ ### Changed
951
+
952
+ - Remove hard link between `AudioContext['createMediaStreamSource']` and the
953
+ pass-in `MediaStream` to avoid any side-effect done to the original stream by
954
+ cloning [#2086]
955
+
956
+ ## [0.7.2] - 2020-10-29
957
+
958
+ ### Added
959
+
960
+ - Support Safari and possibly old browsers with webkit prefix
961
+ [#2071](https://github.com/pexip/aquila-research/pull/2071)
962
+
963
+ ## [0.7.1] - 2020-10-28
964
+
965
+ ### Changed
966
+
967
+ - Stop source stream when releasing audio context
968
+ [#2047](https://github.com/pexip/aquila-research/pull/2047)
969
+
970
+ ## [0.7.0] - 2020-10-28
971
+
972
+ ### Changed
973
+
974
+ - Remove the side effect of changing the audio track from the pass-in media
975
+ stream. [#2036](https://github.com/pexip/aquila-research/pull/2036)
976
+
977
+ ### Removed
978
+
979
+ - Remove `setAudioStreamMute`
980
+
981
+ ## [0.6.2] - 2020-10-27
982
+
983
+ ### Changed
984
+
985
+ - Sync audio track `enabled` state when constructing `GainNode`
986
+ [#2003](https://github.com/pexip/aquila-research/pull/2003)
987
+
988
+ ### Removed
989
+
990
+ - Remove `shouldInitMuted` option
991
+
992
+ ## [0.6.1] - 2020-10-27
993
+
994
+ ### Changed
995
+
996
+ - Stop the tracks from the destination stream when `release` is called
997
+ [#2002](https://github.com/pexip/aquila-research/pull/2002)
998
+
999
+ ## [0.6.0] - 2020-10-27
1000
+
1001
+ ### Changed
1002
+
1003
+ - Skip `AudioWorkletProcessor` for browsers which do not support
1004
+ `AudioWorklet`[#1997](https://github.com/pexip/aquila-research/pull/1997)
1005
+
1006
+ - **BREAKING**: Remove suspend/resume as they are no longer needed
1007
+ [#1997](https://github.com/pexip/aquila-research/pull/1997)
1008
+ - **BREAKING**: Rename `beforeAlter` to `beforeGain` for `getAverageVolume`
1009
+ options [#1997](https://github.com/pexip/aquila-research/pull/1997)
1010
+
1011
+ ### Fixed
1012
+
1013
+ - Audio graph disconnection issue
1014
+ [#1997](https://github.com/pexip/aquila-research/pull/1997)
1015
+
1016
+ ## [0.5.0] - 2020-10-21
1017
+
1018
+ ### Added
1019
+
1020
+ - Expose suspend/resume/state from `AudioContext`
1021
+ [#1899](https://github.com/pexip/aquila-research/pull/1899)
1022
+ - Sync mute state with suspend/resume
1023
+ [#1907](https://github.com/pexip/aquila-research/pull/1907)
1024
+
1025
+ ### Changed
1026
+
1027
+ - Expose `running` instead of `state`
1028
+ [#1907](https://github.com/pexip/aquila-research/pull/1907)
1029
+
1030
+ ### Fixed
1031
+
1032
+ - Fix multiple subscriptions issue for port message
1033
+ [#1907](https://github.com/pexip/aquila-research/pull/1907)
1034
+
1035
+ ## [0.4.1] - 2020-10-16
1036
+
1037
+ ### Changed
1038
+
1039
+ - Rename `worklet` files
1040
+ [#1846](https://github.com/pexip/aquila-research/pull/1846)
1041
+ - Improve asynchronous processing
1042
+ [#1816](https://github.com/pexip/aquila-research/pull/1816)
1043
+
1044
+ ### Fixed
1045
+
1046
+ - Eslint import type issues
1047
+ [#1786](https://github.com/pexip/aquila-research/pull/1786)