@capacitor-mlkit/selfie-segmentation 8.0.1 → 8.1.1

package/README.md

@@ -1,4 +1,4 @@
-# @capacitor-mlkit/selfie-segmentation
+# Capacitor ML Kit Selfie Segmentation Plugin
 
 Unofficial Capacitor plugin for [ML Kit Selfie Segmentation](https://developers.google.com/ml-kit/vision/selfie-segmentation).[^1]
 
@@ -8,6 +8,14 @@ Unofficial Capacitor plugin for [ML Kit Selfie Segmentation](https://developers.
   </a>
 </div>
 
+## Use Cases
+
+The Selfie Segmentation plugin is typically used to separate a person from the background of a photo, for example:
+
+- **Background removal**: Remove or replace the background of a selfie, for example for profile pictures or avatars.
+- **Photo effects**: Build editing features that combine the segmented image with new backgrounds or overlays.
+- **Sticker creation**: Turn selfies into cutouts or stickers that users can share in chats and posts.
+
 ## Compatibility
 
 | Plugin Version | Capacitor Version | Status         |
@@ -18,6 +26,21 @@ Unofficial Capacitor plugin for [ML Kit Selfie Segmentation](https://developers.
 
 ## Installation
 
+You can use our **AI-Assisted Setup** to install the plugin.
+Add the [Capawesome Skills](https://github.com/capawesome-team/skills) to your AI tool using the following command:
+
+```bash
+npx skills add capawesome-team/skills --skill capacitor-plugins
+```
+
+Then use the following prompt:
+
+```
+Use the `capacitor-plugins` skill from `capawesome-team/skills` to install the `@capacitor-mlkit/selfie-segmentation` plugin in my project.
+```
+
+If you prefer **Manual Setup**, install the plugin by running the following commands and follow the platform-specific instructions below:
+
 ```bash
 npm install @capacitor-mlkit/selfie-segmentation
 npx cap sync
@@ -55,10 +78,16 @@ A working example can be found here: [robingenz/capacitor-mlkit-plugin-demo](htt
 
 | Android                                                                                                                           | iOS                                                                                                                               |
 | --------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
-| <img src="https://github.com/capawesome-team/capacitor-mlkit/assets/13857929/1cef9ea9-570e-47d9-a751-d778535ffdff" width="324" /> | <img src="https://github.com/capawesome-team/capacitor-mlkit/assets/13857929/b9caa88c-31b9-4a25-a731-961625207af4" width="324" /> |
+| <img src="https://github.com/capawesome-team/capacitor-mlkit/assets/13857929/1cef9ea9-570e-47d9-a751-d778535ffdff" width="324" alt="Android demo of the Selfie Segmentation plugin" /> | <img src="https://github.com/capawesome-team/capacitor-mlkit/assets/13857929/b9caa88c-31b9-4a25-a731-961625207af4" width="324" alt="iOS demo of the Selfie Segmentation plugin" /> |
 
 ## Usage
 
+The following example shows how to segment a person from the background.
+
+### Segment a person from the background
+
+Pass the local path of an image file to `processImage(...)` to perform the segmentation. You can optionally scale the image using the `width` and `height` options and adjust the confidence threshold. The result contains the path to the segmented image file along with its width and height. Only available on Android and iOS:
+
 ```typescript
 import { SelfieSegmentation } from '@capacitor-mlkit/selfie-segmentation';
 
@@ -127,6 +156,34 @@ Only available on Android and iOS.
 
 </docgen-api>
 
+## FAQ
+
+### Which platforms are supported by this plugin?
+
+The plugin is available on Android and iOS. The `processImage(...)` method is only available on Android and iOS, so there is no web implementation.
+
+### Can I install this plugin using Swift Package Manager?
+
+No, this plugin only supports CocoaPods for iOS dependency management because the ML Kit SDK itself does not support Swift Package Manager. Also make sure to set the deployment target in your `ios/App/Podfile` to at least `15.5` (see [Installation](#installation)).
+
+### How can I adjust the accuracy of the segmentation?
+
+You can set the confidence threshold using the `confidence` option of the `processImage(...)` method. The default value is `0.9`. Experiment with different values to find the best result for your images.
+
+### How can I scale the image before it is processed?
+
+Use the `width` and `height` options of the `processImage(...)` method to scale the image. If only one of the two values is given, the aspect ratio of the image is respected.
+
+### Can I use this plugin with Ionic, React, Vue or Angular?
+
+Yes, the plugin is framework-agnostic. It works in any Capacitor app regardless of the web framework, including Ionic with Angular, React, or Vue, as well as plain JavaScript projects.
+
+## Related Plugins
+
+- [Subject Segmentation](https://capawesome.io/docs/sdks/capacitor/mlkit/subject-segmentation/): Unofficial Capacitor plugin for ML Kit Subject Segmentation.
+- [Face Detection](https://capawesome.io/docs/sdks/capacitor/mlkit/face-detection/): Unofficial Capacitor plugin for ML Kit Face Detection.
+- [Face Mesh Detection](https://capawesome.io/docs/sdks/capacitor/mlkit/face-mesh-detection/): Unofficial Capacitor plugin for ML Kit Face Mesh Detection.
+
 ## Terms & Privacy
 
 This plugin uses the [Google ML Kit](https://developers.google.com/ml-kit):
@@ -135,6 +192,10 @@ This plugin uses the [Google ML Kit](https://developers.google.com/ml-kit):
 - [Android Data Disclosure](https://developers.google.com/ml-kit/android-data-disclosure)
 - [iOS Data Disclosure](https://developers.google.com/ml-kit/ios-data-disclosure)
 
+## Newsletter
+
+Stay up to date with the latest news and updates about the Capawesome, Capacitor, and Ionic ecosystem by subscribing to our [Capawesome Newsletter](https://cloud.capawesome.io/newsletter/).
+
 ## Changelog
 
 See [CHANGELOG.md](https://github.com/capawesome-team/capacitor-mlkit/blob/main/packages/selfie-segmentation/CHANGELOG.md).

package/android/src/main/java/io/capawesome/capacitorjs/plugins/mlkit/selfiesegmentation/SelfieSegmentation.java

@@ -48,78 +48,72 @@ public class SelfieSegmentation {
 
         final Segmenter segmenter = Segmentation.getClient(selfieSegmenterOptions);
 
-        plugin
-            .getActivity()
-            .runOnUiThread(() ->
-                segmenter
-                    .process(inputImage)
-                    .addOnSuccessListener(segmentationMask -> {
-                        segmenter.close();
-
-                        ByteBuffer mask = segmentationMask.getBuffer();
-
-                        Bitmap bitmap = inputImage.getBitmapInternal();
-                        Objects.requireNonNull(bitmap).setHasAlpha(true);
-
-                        ByteBuffer pixels = ByteBuffer.allocateDirect(bitmap.getAllocationByteCount());
-                        bitmap.copyPixelsToBuffer(pixels);
-
-                        final boolean bigEndian = pixels.order() == ByteOrder.BIG_ENDIAN;
-                        final int ALPHA = bigEndian ? 3 : 0;
-                        final int RED = bigEndian ? 2 : 1;
-                        final int GREEN = bigEndian ? 1 : 2;
-                        final int BLUE = bigEndian ? 0 : 3;
-
-                        for (int i = 0; i < pixels.capacity() >> 2; i++) {
-                            float confidence = mask.getFloat();
-
-                            if (confidence >= threshold) {
-                                byte red = pixels.get((i << 2) + RED);
-                                byte green = pixels.get((i << 2) + GREEN);
-                                byte blue = pixels.get((i << 2) + BLUE);
-
-                                pixels.put((i << 2) + ALPHA, (byte) (0xff));
-                                pixels.put((i << 2) + RED, (byte) (red * confidence));
-                                pixels.put((i << 2) + GREEN, (byte) (green * confidence));
-                                pixels.put((i << 2) + BLUE, (byte) (blue * confidence));
-                            } else {
-                                pixels.putInt(i << 2, 0x00000000); // transparent
-                            }
+        plugin.getActivity().runOnUiThread(() ->
+            segmenter
+                .process(inputImage)
+                .addOnSuccessListener(segmentationMask -> {
+                    segmenter.close();
+
+                    ByteBuffer mask = segmentationMask.getBuffer();
+
+                    Bitmap bitmap = inputImage.getBitmapInternal();
+                    Objects.requireNonNull(bitmap).setHasAlpha(true);
+
+                    ByteBuffer pixels = ByteBuffer.allocateDirect(bitmap.getAllocationByteCount());
+                    bitmap.copyPixelsToBuffer(pixels);
+
+                    final boolean bigEndian = pixels.order() == ByteOrder.BIG_ENDIAN;
+                    final int ALPHA = bigEndian ? 3 : 0;
+                    final int RED = bigEndian ? 2 : 1;
+                    final int GREEN = bigEndian ? 1 : 2;
+                    final int BLUE = bigEndian ? 0 : 3;
+
+                    for (int i = 0; i < pixels.capacity() >> 2; i++) {
+                        float confidence = mask.getFloat();
+
+                        if (confidence >= threshold) {
+                            byte red = pixels.get((i << 2) + RED);
+                            byte green = pixels.get((i << 2) + GREEN);
+                            byte blue = pixels.get((i << 2) + BLUE);
+
+                            pixels.put((i << 2) + ALPHA, (byte) (0xff));
+                            pixels.put((i << 2) + RED, (byte) (red * confidence));
+                            pixels.put((i << 2) + GREEN, (byte) (green * confidence));
+                            pixels.put((i << 2) + BLUE, (byte) (blue * confidence));
+                        } else {
+                            pixels.putInt(i << 2, 0x00000000); // transparent
                         }
+                    }
 
-                        bitmap.copyPixelsFromBuffer(pixels.rewind());
+                    bitmap.copyPixelsFromBuffer(pixels.rewind());
 
-                        // Create an image file name
-                        @SuppressLint("SimpleDateFormat")
-                        String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
-                        String imageFileName = "PNG_" + timeStamp + "_";
+                    // Create an image file name
+                    @SuppressLint("SimpleDateFormat")
+                    String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
+                    String imageFileName = "PNG_" + timeStamp + "_";
 
-                        try {
-                            File image = File.createTempFile(imageFileName, ".png");
+                    try {
+                        File image = File.createTempFile(imageFileName, ".png");
 
-                            OutputStream stream = new FileOutputStream(image);
-                            bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
-                            stream.close();
+                        OutputStream stream = new FileOutputStream(image);
+                        bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
+                        stream.close();
 
-                            ProcessImageResult result = new ProcessImageResult(
-                                image.getAbsolutePath(),
-                                bitmap.getWidth(),
-                                bitmap.getHeight()
-                            );
+                        ProcessImageResult result = new ProcessImageResult(image.getAbsolutePath(), bitmap.getWidth(), bitmap.getHeight());
 
-                            callback.success(result);
-                        } catch (Exception exception) {
-                            callback.error(exception);
-                        }
-                    })
-                    .addOnCanceledListener(() -> {
-                        segmenter.close();
-                        callback.cancel();
-                    })
-                    .addOnFailureListener(exception -> {
-                        segmenter.close();
+                        callback.success(result);
+                    } catch (Exception exception) {
                         callback.error(exception);
-                    })
-            );
+                    }
+                })
+                .addOnCanceledListener(() -> {
+                    segmenter.close();
+                    callback.cancel();
+                })
+                .addOnFailureListener(exception -> {
+                    segmenter.close();
+                    callback.error(exception);
+                })
+        );
     }
 }

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@capacitor-mlkit/selfie-segmentation",
-  "version": "8.0.1",
-  "description": "Capacitor plugin for ML Kit Selfie Segmentation.",
+  "version": "8.1.1",
+  "description": "Capacitor plugin for ML Kit Selfie Segmentation on Android and iOS.",
   "main": "dist/plugin.cjs.js",
   "module": "dist/esm/index.js",
   "types": "dist/esm/index.d.ts",
@@ -32,11 +32,20 @@
       "url": "https://opencollective.com/capawesome"
     }
   ],
-  "homepage": "https://capawesome.io/plugins/mlkit/selfie-segmentation/",
+  "homepage": "https://capawesome.io/docs/sdks/capacitor/mlkit/selfie-segmentation/",
   "keywords": [
     "capacitor",
     "plugin",
-    "native"
+    "native",
+    "capacitor-plugin",
+    "mlkit",
+    "ml kit",
+    "selfie segmentation",
+    "image segmentation",
+    "person segmentation",
+    "background removal",
+    "background remover",
+    "remove background"
   ],
   "scripts": {
     "verify": "npm run verify:ios && npm run verify:android && npm run verify:web",
@@ -57,24 +66,21 @@
   },
   "devDependencies": {
     "@capacitor/android": "8.0.0",
-    "@capacitor/cli": "8.0.0",
+    "@capacitor/cli": "8.5.0",
     "@capacitor/core": "8.0.0",
     "@capacitor/docgen": "0.3.1",
     "@capacitor/ios": "8.0.0",
     "@ionic/eslint-config": "0.4.0",
-    "@ionic/swiftlint-config": "2.0.0",
     "eslint": "8.57.0",
-    "prettier": "3.4.2",
-    "prettier-plugin-java": "2.6.7",
+    "prettier-plugin-java": "2.9.7",
     "rimraf": "6.1.2",
-    "rollup": "4.53.3",
+    "rollup": "4.62.3",
     "swiftlint": "2.0.0",
     "typescript": "5.9.3"
   },
   "peerDependencies": {
     "@capacitor/core": ">=8.0.0"
   },
-  "swiftlint": "@ionic/swiftlint-config",
   "eslintConfig": {
     "extends": "@ionic/eslint-config/recommended"
   },