deepar 4.1.0 → 5.0.0-alpha-1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (24) hide show
  1. package/README.md +119 -131
  2. package/effects/aviators +0 -0
  3. package/effects/dalmatian +0 -0
  4. package/effects/koala +0 -0
  5. package/effects/lion +0 -0
  6. package/js/deepar.esm.js +1 -1
  7. package/js/deepar.js +1 -1
  8. package/js/types/DeepAR.d.ts +293 -0
  9. package/js/types/callbacks.d.ts +42 -0
  10. package/js/types/faceData.d.ts +33 -0
  11. package/js/types/footData.d.ts +9 -0
  12. package/js/types/footTrackingConfig.d.ts +21 -0
  13. package/js/types/index.d.ts +7 -0
  14. package/js/types/initParams.d.ts +101 -0
  15. package/js/types/logType.d.ts +18 -0
  16. package/js/types/platformDetect.d.ts +17 -0
  17. package/js/types/touchType.d.ts +34 -0
  18. package/js/types/version.d.ts +5 -0
  19. package/js/types/videoRecordingOptions.d.ts +14 -0
  20. package/js/types/wrapper.d.ts +10 -0
  21. package/package.json +9 -3
  22. package/wasm/deepar.wasm +0 -0
  23. package/js/deepar.d.ts +0 -492
  24. package/js/deepar.esm.d.ts +0 -492
package/README.md CHANGED
@@ -1,22 +1,32 @@
1
1
  # deepar
2
2
 
3
- DeepAR SDK for Web is an augmented reality SDK that allows users to integrate advanced, Snapchat-like face lenses in the browser environment. It supports face masks, effects, multiple face tracking, natural image tracking.
4
- The SDK requires an internet connection.
3
+ DeepAR Web is an augmented reality SDK
4
+ that allows users to integrate advanced, Snapchat-like
5
+ face lenses in the browser environment.
5
6
 
6
- ## Documentation
7
+ DeepAR Web supports:
8
+ - Face filters and masks.
9
+ - Background replacement.
10
+ - Background blur.
11
+ - Shoe try-on.
12
+ - AR mini-games.
13
+
14
+ > ❗ DeepAR Web requires an internet connection.
7
15
 
8
- Visit the official DeepAR docs for Web SDK here: https://docs.deepar.ai/category/deepar-sdk-for-web
9
- See the official example here: https://github.com/DeepARSDK/quickstart-web-js-npm
16
+ ## Documentation
10
17
 
11
- ## Prerequisites
18
+ - Visit the official DeepAR docs for Web SDK here: https://docs.deepar.ai/category/deepar-sdk-for-web
19
+ - See the official example here: https://github.com/DeepARSDK/quickstart-web-js-npm
12
20
 
13
- > ⚠️ Note that these steps are only needed when deploying to production (non-localhost) domain.
21
+ ## License key
14
22
 
15
23
  In order to use the DeepAR Web SDK you need to set up a license key for your web app on [developer.deepar.ai](https://developer.deepar.ai).
16
24
  1. Create an account: https://developer.deepar.ai/signup.
17
25
  2. Create a project: https://developer.deepar.ai/projects.
18
26
  3. Add a web app to the project. Note that you need to specify the domain name which you plan to use for hosting the app.
19
27
 
28
+ > ⚠️ Note that license key is only required when deploying to production (non-localhost) domain.
29
+
20
30
  ## Installation
21
31
 
22
32
  Using `npm`:
@@ -33,7 +43,7 @@ $ yarn add deepar
33
43
 
34
44
  ## Bundler setup
35
45
 
36
- We recommend using a bundler to correctly include assets like models, effects and WebAssembly files.
46
+ We recommend using a bundler to correctly include assets like DeepAR effects files.
37
47
 
38
48
  For example, if using Webpack, add this to your `webpack.config.js`:
39
49
 
@@ -42,13 +52,6 @@ module.exports = {
42
52
  // ...
43
53
  module: {
44
54
  rules: [
45
- {
46
- test: /\.(wasm)|(bin)|(obj)$/i,
47
- include: [
48
- path.resolve(__dirname, 'node_modules/deepar/'),
49
- ],
50
- type: 'asset/resource',
51
- },
52
55
  {
53
56
  include: [
54
57
  path.resolve(__dirname, 'effects/'),
@@ -62,168 +65,153 @@ module.exports = {
62
65
 
63
66
  ## Canvas
64
67
 
65
- DeepAR requires a `canvas` element for the preview of camera, masks, filters and effects. You can add it directly in the HTML.
68
+ DeepAR requires a [canvas](https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API) element for the preview of camera, masks, filters and effects. You can add it directly in the HTML.
66
69
 
67
70
  ```html
68
- <!DOCTYPE HTML>
69
- <html>
70
- <head>
71
- <title>DeepAR</title>
72
- </head>
73
- <body>
74
- <canvas width="1280" height="720" id="deepar-canvas"></canvas>
75
- </body>
76
- </html>
71
+ <canvas width="1280" height="720"></canvas>
77
72
  ```
78
73
 
79
74
  Or you can create it in Javascript.
80
75
  ```javascript
81
- let canvas = document.createElement("canvas");
76
+ const canvas = document.createElement("canvas");
77
+ canvas.width = 1280;
78
+ canvas.height = 720;
82
79
  ```
83
80
 
84
- > **Note:** Be sure to set `width` and `height` properties of the `canvas`!
81
+ > ⚠️ **Note:** Be sure to set `width` and `height` properties of the `canvas`!
85
82
 
86
- ## Initialize DeepAR
87
-
88
- Import DeepAR module and DeepAR WebAssembly file.
89
- If you wish to use an effect that uses face tracking, import the face tracking model and the effect.
90
83
 
84
+ Or you can create it in Javascript.
91
85
  ```javascript
92
- import { DeepAR } from 'deepar';
93
- import deeparWasmPath from 'deepar/wasm/deepar.wasm';
94
-
95
- import faceTrackingModelPath from 'deepar/models/face/models-68-extreme.bin';
96
- import someEffect from './path/to/effect_file';
86
+ let canvas = document.createElement("canvas");
97
87
  ```
98
88
 
99
- Then initialize DeepAR.
89
+ > **Note:** Be sure to set `width` and `height` properties of the `canvas`!
100
90
 
101
- ```javascript
102
- const deepAR = new DeepAR({
103
- licenseKey: 'your_license_key_here',
104
- canvas: document.getElementById('deepar-canvas'),
105
- deeparWasmPath,
106
- callbacks: {
107
- onInitialize: function() {
108
- deepAR.startVideo(true);
109
- deepAR.switchEffect(0, 'mask', someEffect);
110
- },
111
- },
112
- });
91
+ ## Getting started
92
+ There are two main ways to get deepar.js in your JavaScript project:
93
+ via [script tags](https://developer.mozilla.org/en-US/docs/Learn/HTML/Howto/Use_JavaScript_within_a_webpage)
94
+ or by installing it from [NPM](https://www.npmjs.com/package/deepar)
95
+ and using a build tool like
96
+ [Parcel](https://parceljs.org/),
97
+ [WebPack](https://webpack.js.org/),
98
+ or [Rollup](https://rollupjs.org/guide/en).
99
+
100
+ ### via Script tag
101
+ Add the following code to an HTML file:
102
+ ```html
103
+ <html>
104
+ <head>
105
+ <!-- Load deepar.js -->
106
+ <script src="https://cdn.jsdelivr.net/npm/deepar/js/deepar.js"> </script>
107
+ <script>
108
+ const deepAR = await deepar.initialize({
109
+ licenseKey: 'your_license_key_here',
110
+ canvas: document.getElementById('deepar-canvas'),
111
+ effect: 'https://cdn.jsdelivr.net/npm/deepar/effects/aviators'
112
+ });
113
+ </script>
114
+ </head>
113
115
 
114
- // Download the face tracking model. This is requred in order to track face.
115
- deepAR.downloadFaceTrackingModel(faceTrackingModelPath);
116
+ <body>
117
+ <canvas width="1280" height="720" id="deepar-canvas"></canvas>
118
+ </body>
119
+ </html>>
116
120
  ```
117
121
 
118
- ## Callbacks
119
-
120
- DeepAR will call specified callbacks on certain events. List of all callbacks can be found in API reference.
122
+ ### via NPM
123
+ Add deepar.js to your project using [yarn](https://yarnpkg.com/en/) or
124
+ [npm](https://docs.npmjs.com/cli/npm).
121
125
 
122
- > **IMPORTANT** You always need to provide `onInitialize` callback since most of the DeepAR methods will not work
123
- > until SDK has fully initialized.
126
+ **Note**: Because we use ES2017 syntax (such as import),
127
+ this workflow assumes you are using a modern browser or a
128
+ bundler/transpiler to convert your code to something older browsers
129
+ understand. However, you are free to use any build tool that you prefer.
124
130
 
125
- You can provide callbacks in the constructor of the `DeepAR` class in the `callbacks` parameter.
126
131
  ```javascript
127
- const deepAR = new DeepAR({
128
- callbacks: {
129
- onInitialize: function() {
130
- // This is where you start camera preview and start loading effects
131
- },
132
- onScreenshotTaken: function(imageUrl) {
133
- // Show and/or save the screenshot
134
- },
135
- onFaceTracked: function(faceData) {
136
- // Inspect the face tracking features
137
- }
138
- },
139
- // other parameters ...
140
- });
141
- ```
142
- Add or change callbacks via `DeepAR.callbacks` property.
132
+ import * as deepar from 'deepar';
133
+ import deeparEffect from './effects/effectFile'
143
134
 
144
- ```javascript
145
- deepAR.callbacks.onScreenshotTaken = (url) => {
146
- // download or show the image from url
147
- }
148
- ```
149
-
150
- To remove certain callback:
151
- ```javascript
152
- deepAR.callbacks.onScreenshotTaken = undefined;
135
+ const deepAR = await deepar.initialize({
136
+ licenseKey: 'your_license_key_here',
137
+ canvas: document.getElementById('deepar-canvas'),
138
+ effect: deeparEffect
139
+ });
153
140
  ```
154
141
 
155
142
  ## Switch effects
156
143
 
157
- All masks, filters, background removal, etc. are represented by effect files in DeepAR. You can load them to preview the effect.
158
- You can download a free filter pack here: https://docs.deepar.ai/deep-ar-studio/free-filter-pack.
144
+ All filters are represented by effect files in DeepAR. You can load them to preview the effect.
145
+
146
+ Places you can get DeepAR effects:
147
+ - Download a free filter pack: https://docs.deepar.ai/deep-ar-studio/free-filter-pack.
148
+ - Visit DeepAR asset store: https://www.store.deepar.ai/
149
+ - Create your own filters with [DeepAR Studio](https://www.deepar.ai/creator-studio).
159
150
 
160
151
  Load an effect using the `switchEffect` method:
161
152
  ```javascript
162
- import alienEffect from './effects/alien';
163
-
164
- // ...
165
-
166
- deepAR.switchEffect(0, 'slot', alienEffect);
153
+ deepAR.switchEffect('path/to/effect/alien');
167
154
  ```
168
155
 
169
- Load different effects on different persons' faces:
170
- ```javascript
171
- import alienEffect from './effects/alien';
172
- import lionEffect from './effects/lion';
173
-
174
- // ...
156
+ ## Take screenshot or video
175
157
 
176
- deepAR.switchEffect(0, 'slot', alienEffect);
177
- deepAR.switchEffect(1, 'slot', lionEffect);
158
+ Take a screenshot.
159
+ ```javascript
160
+ const image = await deepAR.takeScreenshot();
178
161
  ```
179
162
 
180
- Load a background removal effect:
163
+ Shoot a video.
181
164
  ```javascript
182
- import segmentationEffect from './effects/background_segmentation';
183
-
184
- // ...
185
-
186
- deepAR.switchEffect(0, 'slot', segmentationEffect);
165
+ deepAR.startVideoRecording();
166
+ // Video is now recording...
167
+ // When user is ready to stop recording.
168
+ const video = await deepAR.finishVideoRecording();
187
169
  ```
188
170
 
189
- ## Background removal or blur
171
+ ## Different video sources
190
172
 
191
- To use background segmentation DeepAR needs to initialize the segmentation model.
173
+ DeepAR will by default use the user facing camera. It will also ask the
174
+ camera permission from the user.
192
175
 
176
+ Use the back camera on the phones:
193
177
  ```javascript
194
- import segmentationModelPath from 'deepar/models/segmentation/segmentation-160x160-opt.bin';
195
-
196
- // ...
197
-
198
- const deepAR = new DeepAR({
199
- segmentationConfig: {
200
- modelPath: segmentationModelPath,
201
- },
202
- // other params ...
178
+ const deepAR = await deepar.initialize({
179
+ // ...
180
+ additionalOptions: {
181
+ cameraConfig: {
182
+ facingMode: 'environment'
183
+ }
184
+ }
203
185
  });
204
186
  ```
205
187
 
206
- ## Shoe try-on
188
+ Set up your own camera or custom video source:
189
+ ```javascript
190
+ const deepAR = await deepar.initialize({
191
+ // ...
192
+ additionalOptions: {
193
+ disableDefaultCamera: true
194
+ }
195
+ });
196
+
197
+ // HTMLVideoElement that can contain camera or any video.
198
+ const video = ...;
207
199
 
208
- To use shoe try-on feature DeepAR needs to initialize foot tracking. All the `footTrackingConfig` parameters are required.
200
+ deepAR.setVideoElement(video, true);
201
+ ```
209
202
 
203
+ Initialize DeepAR but start the camera later.
204
+ This is used when you do not want to ask the camera permission right away.
210
205
  ```javascript
211
- import poseEstimationWasmPath from 'deepar/wasm/libxzimgPoseEstimation.wasm';
212
- import footDetectorPath from 'deepar/models/foot/foot-detection-96x96x6.bin';
213
- import footTrackerPath from 'deepar/models/foot/foot-tracker-96x96x18-test.bin'; // or foot-tracker-96x96x13-test.bin
214
- import footObjPath from 'deepar/models/foot/foot-model.obj';
215
-
216
- // ...
217
-
218
- const deepAR = new DeepAR({
219
- footTrackingConfig: {
220
- poseEstimationWasmPath,
221
- detectorPath: footDetectorPath,
222
- trackerPath: footTrackerPath,
223
- objPath: footObjPath,
224
- },
225
- // other params ...
206
+ const deepAR = await deepar.initialize({
207
+ // ...
208
+ additionalOptions: {
209
+ disableDefaultCamera: true
210
+ }
226
211
  });
212
+
213
+ // When you want to ask the camera permission and start the camera.
214
+ deepAR.startCamera();
227
215
  ```
228
216
 
229
217
  ## License
package/effects/aviators ADDED
Binary file
package/effects/dalmatian ADDED
Binary file
package/effects/koala ADDED
Binary file
package/effects/lion ADDED
Binary file