@cloudflare/realtimekit-virtual-background 0.0.2-staging.1 → 0.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -4,10 +4,10 @@
|
|
|
4
4
|
<img src="https://docs.realtime.cloudflare.com/logo/cf.svg" alt="Logo" width="120">
|
|
5
5
|
</a>
|
|
6
6
|
|
|
7
|
-
<h3 align="center">RealtimeKit Video Background
|
|
7
|
+
<h3 align="center">RealtimeKit Video Background Transformer</h3>
|
|
8
8
|
|
|
9
9
|
<p align="center">
|
|
10
|
-
A
|
|
10
|
+
A package that adds video background & blur to participant video feeds in your RealtimeKit Meetings.
|
|
11
11
|
<br />
|
|
12
12
|
<a href="https://docs.realtime.cloudflare.com"><strong>Explore the docs »</strong></a>
|
|
13
13
|
<br />
|
|
@@ -38,7 +38,9 @@
|
|
|
38
38
|
<!-- ABOUT THE PROJECT -->
|
|
39
39
|
## About The Project
|
|
40
40
|
|
|
41
|
-
|
|
41
|
+
This package provides the foundation to add video background and blur to a participant's video feed in your RealtimeKit Meeting with your custom UI implementation.
|
|
42
|
+
|
|
43
|
+
If you are using the `rtk-meeting` component directly and prefer abstraction over this package, refer to [UI Kit Addons](https://www.npmjs.com/package/@cloudflare/realtimekit-ui-addons).
|
|
42
44
|
|
|
43
45
|
### Built With
|
|
44
46
|
|
|
@@ -49,24 +51,15 @@ The goal of Video Background Transformer is to create RealtimeKit Meetings' Vide
|
|
|
49
51
|
<!-- GETTING STARTED -->
|
|
50
52
|
## Getting Started
|
|
51
53
|
|
|
52
|
-
### Prerequisites
|
|
53
|
-
* npm
|
|
54
|
-
* Web Core Version >= 2.0 (Web Core, React Web Core, Angular Web Core etc.)
|
|
55
|
-
* UI Kit Version >= 2.0 (UI Kit, React UI Kit, Angular UI Kit etc.)
|
|
56
|
-
|
|
57
54
|
### Installation
|
|
58
55
|
|
|
59
56
|
```sh
|
|
60
57
|
npm install @cloudflare/realtimekit-virtual-background
|
|
61
58
|
```
|
|
62
59
|
|
|
63
|
-
|
|
64
|
-
|
|
65
60
|
<!-- USAGE EXAMPLES -->
|
|
66
61
|
## Usage
|
|
67
62
|
|
|
68
|
-
**_Note:_** Make sure that you are using the 2.x versions of Web Core & UI Kit.
|
|
69
|
-
|
|
70
63
|
Disable the default per frame rendering of video middleware to improve speed and quality by letting this middleware control it on its own.
|
|
71
64
|
|
|
72
65
|
```ts
|
|
@@ -90,10 +83,11 @@ meeting.self.addVideoMiddleware(
|
|
|
90
83
|
await videoBackgroundTransformer.createStaticBackgroundVideoMiddleware(imageUrl)
|
|
91
84
|
);
|
|
92
85
|
```
|
|
93
|
-
|
|
86
|
+
|
|
87
|
+
2. ```createBackgroundBlurVideoMiddleware``` expects ```blurStrength``` (0-100) as a parameter (50% by default) and blurs the background of the user by the given blurStrength.
|
|
94
88
|
```ts
|
|
95
89
|
meeting.self.addVideoMiddleware(
|
|
96
|
-
await videoBackgroundTransformer.createBackgroundBlurVideoMiddleware(
|
|
90
|
+
await videoBackgroundTransformer.createBackgroundBlurVideoMiddleware(50)
|
|
97
91
|
);
|
|
98
92
|
```
|
|
99
93
|
|
|
@@ -125,25 +119,4 @@ const videoBackgroundTransformer = await RealtimeKitVideoBackgroundTransformer.i
|
|
|
125
119
|
targetFps: 35,
|
|
126
120
|
}
|
|
127
121
|
});
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
<!-- CONTRIBUTING -->
|
|
131
|
-
## Contributing
|
|
132
|
-
|
|
133
|
-
Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are **greatly appreciated**.
|
|
134
|
-
|
|
135
|
-
1. Fork the Project
|
|
136
|
-
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
|
|
137
|
-
3. Commit your Changes (`git commit -m 'feat: Add some AmazingFeature'`)
|
|
138
|
-
4. Push to the Branch (`git push -u origin feature/AmazingFeature`)
|
|
139
|
-
5. Open a Pull Request
|
|
140
|
-
|
|
141
|
-
<!-- LICENSE -->
|
|
142
|
-
## License
|
|
143
|
-
|
|
144
|
-
Distributed under the MIT License. See [`LICENSE`](./LICENSE) for more information.
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
<!-- MARKDOWN LINKS & IMAGES -->
|
|
149
|
-
<!-- https://www.markdownguide.org/basic-syntax/#reference-style-links -->
|
|
122
|
+
```
|
package/package.json
CHANGED
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cloudflare/realtimekit-virtual-background",
|
|
3
|
-
"version": "0.0.2
|
|
4
|
-
"description": "A Package that allows ease of use for RealtimeKit's video middlewares",
|
|
3
|
+
"version": "0.0.2",
|
|
5
4
|
"main": "./dist/index.cjs.js",
|
|
6
5
|
"module": "./dist/index.es.js",
|
|
7
6
|
"exports": {
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import
|
|
1
|
+
import RTKMeeting from '@cloudflare/realtimekit';
|
|
2
2
|
import { SegmentationConfig } from '../core/helpers/segmentationHelper';
|
|
3
3
|
import { TFLite } from '../core/hooks/useTFLite';
|
|
4
4
|
export type BlendMode = 'screen' | 'linearDodge';
|
|
@@ -14,7 +14,7 @@ export type PostProcessingConfig = {
|
|
|
14
14
|
blendMode: BlendMode;
|
|
15
15
|
};
|
|
16
16
|
export interface InitParams {
|
|
17
|
-
meeting:
|
|
17
|
+
meeting: RTKMeeting;
|
|
18
18
|
segmentationConfig: SegmentationConfig;
|
|
19
19
|
postProcessingConfig: PostProcessingConfig;
|
|
20
20
|
tflite: TFLite;
|
|
@@ -28,7 +28,7 @@ export default class RealtimeKitVideoBackgroundTransformer {
|
|
|
28
28
|
* which can be used to create video middlewares for RealtimeKit Meetings.
|
|
29
29
|
*/
|
|
30
30
|
static init({ meeting, segmentationConfig: passedSegmentationConfig, postProcessingConfig: passedPostProcessingConfig, }: {
|
|
31
|
-
meeting:
|
|
31
|
+
meeting: RTKMeeting;
|
|
32
32
|
segmentationConfig?: Partial<SegmentationConfig>;
|
|
33
33
|
postProcessingConfig?: Partial<PostProcessingConfig>;
|
|
34
34
|
}): Promise<RealtimeKitVideoBackgroundTransformer>;
|
package/README_DEV_HELP.md
DELETED
|
@@ -1,33 +0,0 @@
|
|
|
1
|
-
### How it is made?
|
|
2
|
-
|
|
3
|
-
We are using https://github.com/Volcomix/virtual-background (Demo: https://volcomix.github.io/virtual-background/) WebGL pipeline for Chrome & Firefox, whereas using 2d canvas rendering pipeline for Safari and other such Webkit based browsers.
|
|
4
|
-
|
|
5
|
-
Commit at which RealtimeKit has branched from this repo is: `99dfe4a912573a1440236a9762dcb415b86d216a`, of main branch.
|
|
6
|
-
|
|
7
|
-
We have 2 pipelines because WebGL pipeline is performant whereas 2d Canvas rendering is comparatively not performant, however 2d canvas rendering based pipeline is able to draw to canvas even when tab is backgrounded in Safari which is not the case with WebGL thus making WebGL pipeline useless in Safari once user's tab is backgrounded.
|
|
8
|
-
|
|
9
|
-
There are very minimal changes made to the core of Volcomix repo. If you want to check the difference between our files and theirs, diff can be checked at https://www.diffchecker.com/. We have kept the file names mostly the same for ease of use.
|
|
10
|
-
|
|
11
|
-
We have mostly ignored the linting in Volcomix repo files to ensure minimum diff if we ever want to import more changes. React code was stripped to ensure compatibility with all JS frameworks.
|
|
12
|
-
|
|
13
|
-
Repo's tflite models are hosted at `https://dyte-plugins.s3.ap-south-1.amazonaws.com/tf-assets/models/*`.
|
|
14
|
-
|
|
15
|
-
### Caveats
|
|
16
|
-
|
|
17
|
-
1. For WebKit based browsers, we would be using 2d canvas rendering instead of webgl because Safari doesn't support WebGL canvas rendering in the backgrounded tab.
|
|
18
|
-
|
|
19
|
-
2. For local development, please uncomment the following line in index.html.
|
|
20
|
-
```
|
|
21
|
-
<!-- <script type="module" src="/src/index.ts"></script> // For local -->
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
And comment out the following.
|
|
25
|
-
|
|
26
|
-
```
|
|
27
|
-
<script src="./dist/index.iife.js"></script>
|
|
28
|
-
```
|
|
29
|
-
|
|
30
|
-
This was done so that the repo could be deployed to Vercel without any issues.
|
|
31
|
-
|
|
32
|
-
If you don't want to change index.html, you will have to perform `npm run build` after every code change.
|
|
33
|
-
|