@capacitor-mlkit/selfie-segmentation 8.1.0 → 8.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Capacitor ML Kit Selfie Segmentation Plugin
|
|
2
2
|
|
|
3
3
|
Unofficial Capacitor plugin for [ML Kit Selfie Segmentation](https://developers.google.com/ml-kit/vision/selfie-segmentation).[^1]
|
|
4
4
|
|
|
@@ -8,9 +8,13 @@ Unofficial Capacitor plugin for [ML Kit Selfie Segmentation](https://developers.
|
|
|
8
8
|
</a>
|
|
9
9
|
</div>
|
|
10
10
|
|
|
11
|
-
##
|
|
11
|
+
## Use Cases
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
The Selfie Segmentation plugin is typically used to separate a person from the background of a photo, for example:
|
|
14
|
+
|
|
15
|
+
- **Background removal**: Remove or replace the background of a selfie, for example for profile pictures or avatars.
|
|
16
|
+
- **Photo effects**: Build editing features that combine the segmented image with new backgrounds or overlays.
|
|
17
|
+
- **Sticker creation**: Turn selfies into cutouts or stickers that users can share in chats and posts.
|
|
14
18
|
|
|
15
19
|
## Compatibility
|
|
16
20
|
|
|
@@ -78,6 +82,12 @@ A working example can be found here: [robingenz/capacitor-mlkit-plugin-demo](htt
|
|
|
78
82
|
|
|
79
83
|
## Usage
|
|
80
84
|
|
|
85
|
+
The following example shows how to segment a person from the background.
|
|
86
|
+
|
|
87
|
+
### Segment a person from the background
|
|
88
|
+
|
|
89
|
+
Pass the local path of an image file to `processImage(...)` to perform the segmentation. You can optionally scale the image using the `width` and `height` options and adjust the confidence threshold. The result contains the path to the segmented image file along with its width and height. Only available on Android and iOS:
|
|
90
|
+
|
|
81
91
|
```typescript
|
|
82
92
|
import { SelfieSegmentation } from '@capacitor-mlkit/selfie-segmentation';
|
|
83
93
|
|
|
@@ -146,6 +156,34 @@ Only available on Android and iOS.
|
|
|
146
156
|
|
|
147
157
|
</docgen-api>
|
|
148
158
|
|
|
159
|
+
## FAQ
|
|
160
|
+
|
|
161
|
+
### Which platforms are supported by this plugin?
|
|
162
|
+
|
|
163
|
+
The plugin is available on Android and iOS. The `processImage(...)` method is only available on Android and iOS, so there is no web implementation.
|
|
164
|
+
|
|
165
|
+
### Can I install this plugin using Swift Package Manager?
|
|
166
|
+
|
|
167
|
+
No, this plugin only supports CocoaPods for iOS dependency management because the ML Kit SDK itself does not support Swift Package Manager. Also make sure to set the deployment target in your `ios/App/Podfile` to at least `15.5` (see [Installation](#installation)).
|
|
168
|
+
|
|
169
|
+
### How can I adjust the accuracy of the segmentation?
|
|
170
|
+
|
|
171
|
+
You can set the confidence threshold using the `confidence` option of the `processImage(...)` method. The default value is `0.9`. Experiment with different values to find the best result for your images.
|
|
172
|
+
|
|
173
|
+
### How can I scale the image before it is processed?
|
|
174
|
+
|
|
175
|
+
Use the `width` and `height` options of the `processImage(...)` method to scale the image. If only one of the two values is given, the aspect ratio of the image is respected.
|
|
176
|
+
|
|
177
|
+
### Can I use this plugin with Ionic, React, Vue or Angular?
|
|
178
|
+
|
|
179
|
+
Yes, the plugin is framework-agnostic. It works in any Capacitor app regardless of the web framework, including Ionic with Angular, React, or Vue, as well as plain JavaScript projects.
|
|
180
|
+
|
|
181
|
+
## Related Plugins
|
|
182
|
+
|
|
183
|
+
- [Subject Segmentation](https://capawesome.io/docs/sdks/capacitor/mlkit/subject-segmentation/): Unofficial Capacitor plugin for ML Kit Subject Segmentation.
|
|
184
|
+
- [Face Detection](https://capawesome.io/docs/sdks/capacitor/mlkit/face-detection/): Unofficial Capacitor plugin for ML Kit Face Detection.
|
|
185
|
+
- [Face Mesh Detection](https://capawesome.io/docs/sdks/capacitor/mlkit/face-mesh-detection/): Unofficial Capacitor plugin for ML Kit Face Mesh Detection.
|
|
186
|
+
|
|
149
187
|
## Terms & Privacy
|
|
150
188
|
|
|
151
189
|
This plugin uses the [Google ML Kit](https://developers.google.com/ml-kit):
|
|
@@ -154,6 +192,10 @@ This plugin uses the [Google ML Kit](https://developers.google.com/ml-kit):
|
|
|
154
192
|
- [Android Data Disclosure](https://developers.google.com/ml-kit/android-data-disclosure)
|
|
155
193
|
- [iOS Data Disclosure](https://developers.google.com/ml-kit/ios-data-disclosure)
|
|
156
194
|
|
|
195
|
+
## Newsletter
|
|
196
|
+
|
|
197
|
+
Stay up to date with the latest news and updates about the Capawesome, Capacitor, and Ionic ecosystem by subscribing to our [Capawesome Newsletter](https://cloud.capawesome.io/newsletter/).
|
|
198
|
+
|
|
157
199
|
## Changelog
|
|
158
200
|
|
|
159
201
|
See [CHANGELOG.md](https://github.com/capawesome-team/capacitor-mlkit/blob/main/packages/selfie-segmentation/CHANGELOG.md).
|
|
@@ -48,78 +48,72 @@ public class SelfieSegmentation {
|
|
|
48
48
|
|
|
49
49
|
final Segmenter segmenter = Segmentation.getClient(selfieSegmenterOptions);
|
|
50
50
|
|
|
51
|
-
plugin
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
.
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
} else {
|
|
86
|
-
pixels.putInt(i << 2, 0x00000000); // transparent
|
|
87
|
-
}
|
|
51
|
+
plugin.getActivity().runOnUiThread(() ->
|
|
52
|
+
segmenter
|
|
53
|
+
.process(inputImage)
|
|
54
|
+
.addOnSuccessListener(segmentationMask -> {
|
|
55
|
+
segmenter.close();
|
|
56
|
+
|
|
57
|
+
ByteBuffer mask = segmentationMask.getBuffer();
|
|
58
|
+
|
|
59
|
+
Bitmap bitmap = inputImage.getBitmapInternal();
|
|
60
|
+
Objects.requireNonNull(bitmap).setHasAlpha(true);
|
|
61
|
+
|
|
62
|
+
ByteBuffer pixels = ByteBuffer.allocateDirect(bitmap.getAllocationByteCount());
|
|
63
|
+
bitmap.copyPixelsToBuffer(pixels);
|
|
64
|
+
|
|
65
|
+
final boolean bigEndian = pixels.order() == ByteOrder.BIG_ENDIAN;
|
|
66
|
+
final int ALPHA = bigEndian ? 3 : 0;
|
|
67
|
+
final int RED = bigEndian ? 2 : 1;
|
|
68
|
+
final int GREEN = bigEndian ? 1 : 2;
|
|
69
|
+
final int BLUE = bigEndian ? 0 : 3;
|
|
70
|
+
|
|
71
|
+
for (int i = 0; i < pixels.capacity() >> 2; i++) {
|
|
72
|
+
float confidence = mask.getFloat();
|
|
73
|
+
|
|
74
|
+
if (confidence >= threshold) {
|
|
75
|
+
byte red = pixels.get((i << 2) + RED);
|
|
76
|
+
byte green = pixels.get((i << 2) + GREEN);
|
|
77
|
+
byte blue = pixels.get((i << 2) + BLUE);
|
|
78
|
+
|
|
79
|
+
pixels.put((i << 2) + ALPHA, (byte) (0xff));
|
|
80
|
+
pixels.put((i << 2) + RED, (byte) (red * confidence));
|
|
81
|
+
pixels.put((i << 2) + GREEN, (byte) (green * confidence));
|
|
82
|
+
pixels.put((i << 2) + BLUE, (byte) (blue * confidence));
|
|
83
|
+
} else {
|
|
84
|
+
pixels.putInt(i << 2, 0x00000000); // transparent
|
|
88
85
|
}
|
|
86
|
+
}
|
|
89
87
|
|
|
90
|
-
|
|
88
|
+
bitmap.copyPixelsFromBuffer(pixels.rewind());
|
|
91
89
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
90
|
+
// Create an image file name
|
|
91
|
+
@SuppressLint("SimpleDateFormat")
|
|
92
|
+
String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
|
|
93
|
+
String imageFileName = "PNG_" + timeStamp + "_";
|
|
96
94
|
|
|
97
|
-
|
|
98
|
-
|
|
95
|
+
try {
|
|
96
|
+
File image = File.createTempFile(imageFileName, ".png");
|
|
99
97
|
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
98
|
+
OutputStream stream = new FileOutputStream(image);
|
|
99
|
+
bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
|
|
100
|
+
stream.close();
|
|
103
101
|
|
|
104
|
-
|
|
105
|
-
image.getAbsolutePath(),
|
|
106
|
-
bitmap.getWidth(),
|
|
107
|
-
bitmap.getHeight()
|
|
108
|
-
);
|
|
102
|
+
ProcessImageResult result = new ProcessImageResult(image.getAbsolutePath(), bitmap.getWidth(), bitmap.getHeight());
|
|
109
103
|
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
callback.error(exception);
|
|
113
|
-
}
|
|
114
|
-
})
|
|
115
|
-
.addOnCanceledListener(() -> {
|
|
116
|
-
segmenter.close();
|
|
117
|
-
callback.cancel();
|
|
118
|
-
})
|
|
119
|
-
.addOnFailureListener(exception -> {
|
|
120
|
-
segmenter.close();
|
|
104
|
+
callback.success(result);
|
|
105
|
+
} catch (Exception exception) {
|
|
121
106
|
callback.error(exception);
|
|
122
|
-
}
|
|
123
|
-
|
|
107
|
+
}
|
|
108
|
+
})
|
|
109
|
+
.addOnCanceledListener(() -> {
|
|
110
|
+
segmenter.close();
|
|
111
|
+
callback.cancel();
|
|
112
|
+
})
|
|
113
|
+
.addOnFailureListener(exception -> {
|
|
114
|
+
segmenter.close();
|
|
115
|
+
callback.error(exception);
|
|
116
|
+
})
|
|
117
|
+
);
|
|
124
118
|
}
|
|
125
119
|
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@capacitor-mlkit/selfie-segmentation",
|
|
3
|
-
"version": "8.1.
|
|
4
|
-
"description": "Capacitor plugin for ML Kit Selfie Segmentation.",
|
|
3
|
+
"version": "8.1.1",
|
|
4
|
+
"description": "Capacitor plugin for ML Kit Selfie Segmentation on Android and iOS.",
|
|
5
5
|
"main": "dist/plugin.cjs.js",
|
|
6
6
|
"module": "dist/esm/index.js",
|
|
7
7
|
"types": "dist/esm/index.d.ts",
|
|
@@ -32,11 +32,20 @@
|
|
|
32
32
|
"url": "https://opencollective.com/capawesome"
|
|
33
33
|
}
|
|
34
34
|
],
|
|
35
|
-
"homepage": "https://capawesome.io/
|
|
35
|
+
"homepage": "https://capawesome.io/docs/sdks/capacitor/mlkit/selfie-segmentation/",
|
|
36
36
|
"keywords": [
|
|
37
37
|
"capacitor",
|
|
38
38
|
"plugin",
|
|
39
|
-
"native"
|
|
39
|
+
"native",
|
|
40
|
+
"capacitor-plugin",
|
|
41
|
+
"mlkit",
|
|
42
|
+
"ml kit",
|
|
43
|
+
"selfie segmentation",
|
|
44
|
+
"image segmentation",
|
|
45
|
+
"person segmentation",
|
|
46
|
+
"background removal",
|
|
47
|
+
"background remover",
|
|
48
|
+
"remove background"
|
|
40
49
|
],
|
|
41
50
|
"scripts": {
|
|
42
51
|
"verify": "npm run verify:ios && npm run verify:android && npm run verify:web",
|
|
@@ -57,16 +66,15 @@
|
|
|
57
66
|
},
|
|
58
67
|
"devDependencies": {
|
|
59
68
|
"@capacitor/android": "8.0.0",
|
|
60
|
-
"@capacitor/cli": "8.
|
|
69
|
+
"@capacitor/cli": "8.5.0",
|
|
61
70
|
"@capacitor/core": "8.0.0",
|
|
62
71
|
"@capacitor/docgen": "0.3.1",
|
|
63
72
|
"@capacitor/ios": "8.0.0",
|
|
64
73
|
"@ionic/eslint-config": "0.4.0",
|
|
65
74
|
"eslint": "8.57.0",
|
|
66
|
-
"prettier": "
|
|
67
|
-
"prettier-plugin-java": "2.6.7",
|
|
75
|
+
"prettier-plugin-java": "2.9.7",
|
|
68
76
|
"rimraf": "6.1.2",
|
|
69
|
-
"rollup": "4.
|
|
77
|
+
"rollup": "4.62.3",
|
|
70
78
|
"swiftlint": "2.0.0",
|
|
71
79
|
"typescript": "5.9.3"
|
|
72
80
|
},
|