react-native-nitro-pose-exercises 1.0.9 → 1.1.2

package/NitroPoseExercises.podspec

@@ -19,13 +19,14 @@ Pod::Spec.new do |s|
     "cpp/**/*.{hpp,cpp}",
   ]
 
+  s.frameworks = ["AVFoundation", "Vision"]
+
   s.dependency 'React-jsi'
   s.dependency 'React-callinvoker'
 
   load 'nitrogen/generated/ios/NitroPoseExercises+autolinking.rb'
   add_nitrogen_files(s)
 
-  s.dependency 'MediaPipeTasksVision', '~> 0.10.0'
   s.dependency "VisionCamera"
 
   install_modules_dependencies(s)

package/README.md

@@ -6,21 +6,22 @@
 
 # react-native-nitro-pose-exercises
 
-A **React Native Nitro Module** for real-time, on-device exercise tracking using pose estimation. Built on **MediaPipe Pose Landmarker** and **VisionCamera v5**.
+A **React Native Nitro Module** for real-time, on-device exercise tracking using pose estimation. Uses **OS-native pose detection** — Apple Vision on iOS and Google ML Kit on Android — with **VisionCamera v5**.
 
-- 🏋️ **Rep Counting** — Automatic rep detection with configurable state machines
-- 🧘 **Hold Tracking** — Duration and stability tracking for planks, yoga poses, and isometric holds
-- 📐 **Form Validation** — Real-time form feedback with configurable angle-based rules
-- 💀 **Skeleton Overlay** — Optional Skia-powered skeleton rendering over the camera feed
-- ⚡ **Fully Native** — MediaPipe runs on-device via Nitro Modules, zero JS bridge overhead
+* 🏋️ **Rep Counting** — Automatic rep detection with configurable state machines
+* 🧘 **Hold Tracking** — Duration and stability tracking for planks, yoga poses, and isometric holds
+* 📐 **Form Validation** — Real-time form feedback with configurable angle-based rules
+* 💀 **Skeleton Overlay** — Optional Skia-powered skeleton rendering over the camera feed
+* ⚡ **Fully Native** — OS-level pose detection via Nitro Modules, zero JS bridge overhead
+* 📦 **Zero Model Bundling** — No ML model files to download or ship with your app
 
 ---
 
 > [!IMPORTANT]
 >
-> - Requires React Native **0.76+** with Nitro Modules and VisionCamera **v5**.
-> - Must be tested on a **physical device** — camera + ML inference don't work on simulators.
-> - MediaPipe Pose Landmarker model file (`pose_landmarker_lite.task`) must be bundled with the app.
+> * Requires React Native **0.76+** with Nitro Modules and VisionCamera **v5**.
+> * Must be tested on a **physical device** — camera + ML inference don't work on simulators.
+> * iOS requires **iOS 14+** (Vision body pose API). Android requires **API 23+** (ML Kit).
 
 ---
 
@@ -44,9 +45,10 @@ cd ios && pod install
 ```
 
 > [!NOTE]
-> This package uses **MediaPipe Pose Landmarker** natively on both platforms.
-> iOS uses `MediaPipeTasksVision` via CocoaPods.
-> Android uses `com.google.mediapipe:tasks-vision` via Gradle.
+> This package uses **OS-native pose detection** on both platforms.
+> iOS uses Apple's **Vision framework** (`VNDetectHumanBodyPoseRequest`) — built into iOS, no extra dependencies.
+> Android uses **Google ML Kit Pose Detection** (`com.google.mlkit:pose-detection:18.0.0-beta5`) — model auto-managed via Play Services.
+> **No model files to bundle, no extra downloads, no color format conversions.**
 
 ---
 
@@ -71,30 +73,25 @@ cd ios && pod install
 
 ## 🧠 Overview
 
-| Feature                  | Description                                                               |
-| ------------------------ | ------------------------------------------------------------------------- |
-| **Rep-Based Exercises**  | Cyclic state machine (UP → DOWN → UP = 1 rep). Push-ups, squats, curls.   |
+| Feature | Description |
+| --- | --- |
+| **Rep-Based Exercises** | Cyclic state machine (UP → DOWN → UP = 1 rep). Push-ups, squats, curls. |
 | **Hold-Based Exercises** | Single target pose with duration tracking. Planks, wall sits, yoga poses. |
-| **Flow-Based Exercises** | Ordered sequence of poses. Sun salutation, yoga flows. _(coming soon)_    |
-| **Form Feedback**        | Angle-based rules with throttled real-time callbacks.                     |
-| **Skeleton Overlay**     | 33-point body skeleton drawn over camera via Skia.                        |
-| **Bilateral Tracking**   | Left and right side angles tracked independently.                         |
+| **Flow-Based Exercises** | Ordered sequence of poses. Sun salutation, yoga flows. *(coming soon)* |
+| **Form Feedback** | Angle-based rules with throttled real-time callbacks. |
+| **Skeleton Overlay** | Body skeleton drawn over camera via Skia (19 joints iOS, 33 joints Android). |
+| **Bilateral Tracking** | Left and right side angles tracked independently. |
 
 ---
 
 ## 🔧 Setup
 
-### Model File
+### No Model File Needed
 
-Download the MediaPipe Pose Landmarker model:
+Unlike MediaPipe-based solutions, this library uses OS-native APIs. There is **no model file to download or bundle**.
 
-```
-https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_lite/float16/latest/pose_landmarker_lite.task
-```
-
-**iOS:** Drag `pose_landmarker_lite.task` into your Xcode project (Copy items if needed, add to app target).
-
-**Android:** Place at `android/app/src/main/assets/pose_landmarker_lite.task`
+* **iOS:** Apple Vision is a system framework — it's already on every iPhone running iOS 14+.
+* **Android:** ML Kit manages its own model via Google Play Services — it downloads and updates automatically.
 
 ### Permissions
 
@@ -125,6 +122,26 @@ module.exports = {
 };
 ```
 
+### Podspec (for library authors)
+
+The iOS podspec needs the Vision and AVFoundation system frameworks:
+
+```ruby
+ s.frameworks = ["AVFoundation", "Vision"]
+```
+
+No CocoaPods dependencies required — Vision is built into iOS.
+
+### Android Gradle (for library authors)
+
+Add ML Kit Pose Detection to `android/build.gradle`:
+
+```groovy
+dependencies {
+    implementation 'com.google.mlkit:pose-detection:18.0.0-beta5'
+}
+```
+
 ---
 
 ## ⚙️ Usage
@@ -159,10 +176,10 @@ export default function App() {
     if (!hasPermission) requestPermission();
   }, [hasPermission]);
 
-  // Initialize pose engine
+  // Initialize pose engine — modelPath is ignored (OS-native, no model file)
   useEffect(() => {
     async function init() {
-      await nitroPoseExercises.initialize('pose_landmarker_lite.task');
+      await nitroPoseExercises.initialize('');
       nitroPoseExercises.loadExercise(PUSHUP_CONFIG);
 
       nitroPoseExercises.onRepComplete = (data: RepData) => {
@@ -310,7 +327,7 @@ const SKELETON_CONNECTIONS: [number, number][] = [
 ### Lifecycle
 
 ```ts
-// Initialize MediaPipe with model file path
+// Initialize the pose engine (modelPath is ignored — OS-native, no model file needed)
 initialize(modelPath: string): Promise<void>
 
 // Clean up resources
@@ -346,7 +363,7 @@ processFrame(frame: Frame): void
 readonly status: SessionStatus        // 'idle' | 'countdown' | 'active' | 'paused' | 'completed'
 readonly currentPhase: ExercisePhase   // 'up' | 'down' | 'hold' | 'transition' | 'unknown'
 readonly repCount: number
-readonly landmarks: Landmark[]         // 33 body landmarks from MediaPipe
+readonly landmarks: Landmark[]         // Body landmarks (mapped to MediaPipe indices)
 ```
 
 ### Callbacks
@@ -380,9 +397,9 @@ onSessionComplete: ((result: SessionResult) => void) | undefined
 
 ```ts
 {
-  ruleName: string; // e.g. 'hipSag'
-  message: string; // e.g. 'Keep your hips up'
-  severity: FormSeverity; // 'info' | 'warning' | 'error'
+  ruleName: string        // e.g. 'hipSag'
+  message: string         // e.g. 'Keep your hips up'
+  severity: FormSeverity  // 'info' | 'warning' | 'error'
 }
 ```
 
@@ -403,62 +420,88 @@ onSessionComplete: ((result: SessionResult) => void) | undefined
 
 ## 🏋️ Built-In Exercise Configs
 
-### Push-Up (`PUSHUP_CONFIG`)
+### Rep-Based
 
-| Parameter     | Value                                 |
-| ------------- | ------------------------------------- |
-| Type          | `rep`                                 |
-| Primary Angle | Left elbow (shoulder → elbow → wrist) |
-| UP Phase      | Elbow angle 140°–180°                 |
-| DOWN Phase    | Elbow angle 30°–110°                  |
-| Rep Sequence  | UP → DOWN → UP                        |
-| Form Rules    | Hip sag detection, hip pike detection |
+| Config | Exercise | Primary Angle | Form Rules |
+| --- | --- | --- | --- |
+| `PUSHUP_CONFIG` | Push-Up | Elbow (140°–180° up, 30°–110° down) | Hip sag, hip pike |
+| `SQUAT_CONFIG` | Squat | Knee (155°–180° up, 50°–105° down) | Knees caving, leaning forward |
+| `BICEP_CURL_CONFIG` | Bicep Curl | Elbow (150°–180° down, 25°–70° up) | Elbow flare, swinging |
+| `SHOULDER_PRESS_CONFIG` | Shoulder Press | Elbow (155°–180° up, 60°–100° down) | Back arch |
+| `LUNGE_CONFIG` | Lunge | Front knee (155°–180° up, 70°–110° down) | Knee over toe, torso lean |
+| `SITUP_CONFIG` | Sit-Up | Hip (130°–180° down, 40°–90° up) | Neck strain |
+| `TRICEP_DIP_CONFIG` | Tricep Dip | Elbow (150°–180° up, 60°–100° down) | Going too deep |
+
+### Hold-Based
+
+| Config | Exercise | Hold Angle | Duration | Form Rules |
+| --- | --- | --- | --- | --- |
+| `PLANK_CONFIG` | Plank | Hip 155°–180° | 60s | Hip sag, hip pike |
+| `WALL_SIT_CONFIG` | Wall Sit | Knee 80°–110° | 45s | Too high, leaning forward |
+
+### Yoga Poses
+
+| Config | Exercise | Hold Angle | Duration | Form Rules |
+| --- | --- | --- | --- | --- |
+| `TREE_POSE_CONFIG` | Tree Pose (Vrksasana) | Standing leg 165°–180° | 30s | Standing leg bent, leaning torso |
+| `WARRIOR_I_CONFIG` | Warrior I (Virabhadrasana I) | Front knee 80°–110° | 30s | Knee too straight, back leg bent, arms not extended, torso leaning |
+| `WARRIOR_II_CONFIG` | Warrior II (Virabhadrasana II) | Front knee 80°–110° | 30s | Knee too straight, back leg bent, arms drooping |
+| `DOWNWARD_DOG_CONFIG` | Downward Dog (Adho Mukha Svanasana) | Hip 55°–100° | 30s | Arms bent, legs bent, hips too low |
+| `CHAIR_POSE_CONFIG` | Chair Pose (Utkatasana) | Knee 90°–130° | 30s | Knees too straight, leaning forward, arms not up |
+| `COBRA_POSE_CONFIG` | Cobra Pose (Bhujangasana) | Hip extension 120°–170° | 30s | Shoulders tensed, legs bending |
 
 ### Custom Exercise Config
 
 ```ts
 import type { ExerciseConfig } from 'react-native-nitro-pose-exercises';
 
-const SQUAT_CONFIG: ExerciseConfig = {
-  name: 'Squat',
-  type: 'rep',
+const MY_EXERCISE: ExerciseConfig = {
+  name: 'Custom Exercise',
+  type: 'rep',  // 'rep' | 'hold'
   angles: [
-    { name: 'leftKnee', landmarkA: 23, landmarkB: 25, landmarkC: 27 },
-    { name: 'rightKnee', landmarkA: 24, landmarkB: 26, landmarkC: 28 },
+    { name: 'myAngle', landmarkA: 11, landmarkB: 13, landmarkC: 15 },
   ],
   phases: [
-    { phase: 'up', angleName: 'leftKnee', minAngle: 160, maxAngle: 180 },
-    { phase: 'down', angleName: 'leftKnee', minAngle: 50, maxAngle: 110 },
+    { phase: 'up', angleName: 'myAngle', minAngle: 150, maxAngle: 180 },
+    { phase: 'down', angleName: 'myAngle', minAngle: 30, maxAngle: 100 },
   ],
   repSequence: ['up', 'down', 'up'],
-  formRules: [
-    {
-      name: 'kneesCaving',
-      message: 'Push your knees out over your toes',
-      severity: 'warning',
-      angleName: 'leftKnee',
-      minAngle: 50,
-      maxAngle: 180,
-    },
-  ],
+  formRules: [],
   holdDurationMs: 0,
 };
 ```
 
 ---
 
-## 📐 MediaPipe Landmark Index Reference
+## 📐 Landmark Index Reference
+
+Landmarks are mapped to MediaPipe-compatible indices on both platforms. iOS Vision provides 19 joints (all exercise-critical joints covered), Android ML Kit provides the full 33.
 
-| Index | Landmark       | Index | Landmark    |
-| ----- | -------------- | ----- | ----------- |
-| 0     | Nose           | 16    | Right wrist |
-| 11    | Left shoulder  | 23    | Left hip    |
-| 12    | Right shoulder | 24    | Right hip   |
-| 13    | Left elbow     | 25    | Left knee   |
-| 14    | Right elbow    | 26    | Right knee  |
-| 15    | Left wrist     | 27    | Left ankle  |
+| Index | Landmark | Index | Landmark |
+| --- | --- | --- | --- |
+| 0 | Nose | 16 | Right wrist |
+| 11 | Left shoulder | 23 | Left hip |
+| 12 | Right shoulder | 24 | Right hip |
+| 13 | Left elbow | 25 | Left knee |
+| 14 | Right elbow | 26 | Right knee |
+| 15 | Left wrist | 27 | Left ankle |
 
-Full 33-point reference: [MediaPipe Pose Landmarks](https://ai.google.dev/edge/mediapipe/solutions/vision/pose_landmarker#pose_landmarker_model)
+**iOS note:** Vision provides 19 joints. Indices not available from Vision (face details 1-10, hands 17-22, feet 29-32) are filled with `visibility: 0` and skipped by the skeleton overlay.
+
+**Android note:** ML Kit provides all 33 landmarks matching MediaPipe indices exactly.
+
+---
+
+## 🏗️ Architecture — OS-Native vs MediaPipe
+
+| | OS-Native (current) | MediaPipe (previous) |
+| --- | --- | --- |
+| **iOS** | Apple Vision framework (built-in) | MediaPipeTasksVision (CocoaPod) |
+| **Android** | Google ML Kit (Play Services) | com.google.mediapipe:tasks-vision |
+| **Model file** | None needed | ~3 MB bundled `.task` file |
+| **Color conversion** | None — takes CVPixelBuffer/ImageProxy directly | BGRA required (iOS), NV21→RGB (Android) |
+| **App size impact** | ~200 KB (Nitro module code only) | ~11-15 MB (SDK + model) |
+| **Updates** | OS/Play Services updates | Manual model file replacement |
 
 ---
 
@@ -466,33 +509,35 @@ Full 33-point reference: [MediaPipe Pose Landmarks](https://ai.google.dev/edge/m
 
 For best results, the camera should see the exerciser from a **side profile**:
 
-| ✅ Good                            | ❌ Bad                   |
-| ---------------------------------- | ------------------------ |
-| Side view, full body visible       | Front-facing view        |
-| Phone at waist height, 6-8 ft away | Ground-level angle       |
-| Well-lit environment               | Heavy glare or backlight |
+| ✅ Good | ❌ Bad |
+| --- | --- |
+| Side view, full body visible | Front-facing view |
+| Phone at waist height, 6-8 ft away | Ground-level angle |
+| Well-lit environment | Heavy glare or backlight |
 
 ---
 
 ## 🧩 Supported Platforms
 
-| Platform             | Status           | Notes                             |
-| -------------------- | ---------------- | --------------------------------- |
-| **iOS**              | ✅ Supported     | Requires physical device, iOS 14+ |
-| **Android**          | ✅ Supported     | Min SDK 24 (Android 7.0)          |
-| **iOS Simulator**    | ❌ Not supported | No camera access                  |
-| **Android Emulator** | ❌ Not supported | No real camera feed               |
+| Platform | Status | Notes |
+| --- | --- | --- |
+| **iOS** | ✅ Supported | Requires physical device, iOS 14+ (Vision body pose) |
+| **Android** | ✅ Supported | API 23+ (ML Kit), Google Play Services required |
+| **iOS Simulator** | ❌ Not supported | No camera access |
+| **Android Emulator** | ❌ Not supported | No real camera feed |
 
 ---
 
 ## 📊 App Size Impact
 
-| Component                    | Size          |
-| ---------------------------- | ------------- |
-| Pose model (Lite)            | ~3 MB         |
-| MediaPipe SDK (per platform) | ~8–12 MB      |
-| Nitro module code            | ~200 KB       |
-| **Total new addition**       | **~11–15 MB** |
+| Component | Size |
+| --- | --- |
+| Nitro module code (Swift + Kotlin) | ~200 KB |
+| ML Kit (Android, via Play Services) | ~0 KB (managed externally) |
+| Vision framework (iOS, built-in) | ~0 KB (system framework) |
+| **Total new addition** | **~200 KB** |
+
+Compared to the MediaPipe approach (~11-15 MB), the OS-native approach adds virtually zero app size.
 
 ---
 
@@ -500,9 +545,9 @@ For best results, the camera should see the exerciser from a **side profile**:
 
 PRs welcome!
 
-- [Development Workflow](CONTRIBUTING.md#development-workflow)
-- [Sending a PR](CONTRIBUTING.md#sending-a-pull-request)
-- [Code of Conduct](CODE_OF_CONDUCT.md)
+* [Development Workflow](CONTRIBUTING.md#development-workflow)
+* [Sending a PR](CONTRIBUTING.md#sending-a-pull-request)
+* [Code of Conduct](CODE_OF_CONDUCT.md)
 
 ---
 
@@ -512,4 +557,4 @@ MIT © [**Gautham Vijayan**](https://gauthamvijay.com)
 
 ---
 
-Made with ❤️ and [**Nitro Modules**](https://nitro.margelo.com) + [**VisionCamera**](https://visioncamera.margelo.com) + [**MediaPipe**](https://ai.google.dev/edge/mediapipe)
+Made with ❤️ and [**Nitro Modules**](https://nitro.margelo.com) + [**VisionCamera**](https://visioncamera.margelo.com) + [**Apple Vision**](https://developer.apple.com/documentation/vision) + [**ML Kit**](https://developers.google.com/ml-kit/vision/pose-detection)

package/android/build.gradle

@@ -120,7 +120,7 @@ repositories {
 dependencies {
   implementation "com.facebook.react:react-android"
   implementation project(":react-native-nitro-modules")
-  implementation 'com.google.mediapipe:tasks-vision:0.10.29'
+  implementation 'com.google.mlkit:pose-detection:18.0.0-beta5'
   implementation project(":react-native-vision-camera")
 }
 

package/android/src/main/java/com/margelo/nitro/nitroposeexercises/NitroPoseExercises.kt

@@ -1,17 +1,18 @@
 package com.margelo.nitro.nitroposeexercises
 
 import android.graphics.Bitmap
+import android.graphics.Matrix
+import android.media.Image
 import androidx.annotation.Keep
 import com.facebook.proguard.annotations.DoNotStrip
-import com.google.mediapipe.framework.image.BitmapImageBuilder
-import com.google.mediapipe.tasks.core.BaseOptions
-import com.google.mediapipe.tasks.vision.core.RunningMode
-import com.google.mediapipe.tasks.vision.poselandmarker.PoseLandmarker
-import com.google.mediapipe.tasks.vision.poselandmarker.PoseLandmarker.PoseLandmarkerOptions
+import com.google.mlkit.vision.common.InputImage
+import com.google.mlkit.vision.pose.PoseDetection
+import com.google.mlkit.vision.pose.PoseDetector
+import com.google.mlkit.vision.pose.PoseLandmark
+import com.google.mlkit.vision.pose.defaults.PoseDetectorOptions
 import com.margelo.nitro.NitroModules
 import com.margelo.nitro.core.Promise
 import com.margelo.nitro.camera.HybridFrameSpec
-import com.margelo.nitro.camera.public.NativeFrame
 import kotlin.math.acos
 import kotlin.math.max
 import kotlin.math.min
@@ -19,12 +20,54 @@ import kotlin.math.sqrt
 
 @Keep
 @DoNotStrip
-class NitroPoseExercises : HybridNitroPoseExercisesSpec() {
+class HybridPoseExercise : HybridNitroPoseExercisesSpec() {
 
-  // ─── MediaPipe ──────────────────────────────────────────────
-  private var poseLandmarker: PoseLandmarker? = null
+  // ─── ML Kit ─────────────────────────────────────────────────
+  private var poseDetector: PoseDetector? = null
   private var isInitialized = false
 
+  // ─── Cached Landmarks (ML Kit is async, we cache last result) ──
+  private var cachedLandmarks: Array<Landmark> = emptyArray()
+  private val landmarkLock = Any()
+
+  // ─── Landmark Index Mapping ─────────────────────────────────
+  // ML Kit PoseLandmark type → MediaPipe index that JS configs expect
+  private val mlKitToMediaPipeMap = mapOf(
+    PoseLandmark.NOSE to 0,
+    PoseLandmark.LEFT_EYE_INNER to 1,
+    PoseLandmark.LEFT_EYE to 2,
+    PoseLandmark.LEFT_EYE_OUTER to 3,
+    PoseLandmark.RIGHT_EYE_INNER to 4,
+    PoseLandmark.RIGHT_EYE to 5,
+    PoseLandmark.RIGHT_EYE_OUTER to 6,
+    PoseLandmark.LEFT_EAR to 7,
+    PoseLandmark.RIGHT_EAR to 8,
+    PoseLandmark.LEFT_MOUTH to 9,
+    PoseLandmark.RIGHT_MOUTH to 10,
+    PoseLandmark.LEFT_SHOULDER to 11,
+    PoseLandmark.RIGHT_SHOULDER to 12,
+    PoseLandmark.LEFT_ELBOW to 13,
+    PoseLandmark.RIGHT_ELBOW to 14,
+    PoseLandmark.LEFT_WRIST to 15,
+    PoseLandmark.RIGHT_WRIST to 16,
+    PoseLandmark.LEFT_PINKY to 17,
+    PoseLandmark.RIGHT_PINKY to 18,
+    PoseLandmark.LEFT_INDEX to 19,
+    PoseLandmark.RIGHT_INDEX to 20,
+    PoseLandmark.LEFT_THUMB to 21,
+    PoseLandmark.RIGHT_THUMB to 22,
+    PoseLandmark.LEFT_HIP to 23,
+    PoseLandmark.RIGHT_HIP to 24,
+    PoseLandmark.LEFT_KNEE to 25,
+    PoseLandmark.RIGHT_KNEE to 26,
+    PoseLandmark.LEFT_ANKLE to 27,
+    PoseLandmark.RIGHT_ANKLE to 28,
+    PoseLandmark.LEFT_HEEL to 29,
+    PoseLandmark.RIGHT_HEEL to 30,
+    PoseLandmark.LEFT_FOOT_INDEX to 31,
+    PoseLandmark.RIGHT_FOOT_INDEX to 32,
+  )
+
   // ─── Exercise Config ────────────────────────────────────────
   private var exerciseConfig: ExerciseConfig? = null
 
@@ -59,7 +102,7 @@ class NitroPoseExercises : HybridNitroPoseExercisesSpec() {
   // ─── Pose Tracking ──────────────────────────────────────────
   private var poseWasLost = false
 
-  // ─── Frame Skip ─────────────────────────────────────────────
+  // ─── Frame Throttle ─────────────────────────────────────────
   private var frameCount: Int = 0
   private val processEveryNFrames: Int = 3
 
@@ -80,31 +123,21 @@ class NitroPoseExercises : HybridNitroPoseExercisesSpec() {
   // ═══════════════════════════════════════════════════════════
 
   override fun initialize(modelPath: String): Promise<Unit> {
+    // No model file needed — ML Kit downloads/bundles its own model
     return Promise.async {
-      val context = NitroModules.applicationContext
-        ?: throw Error("No ApplicationContext set!")
-
-      val baseOptions = BaseOptions.builder()
-        .setModelAssetPath(modelPath)
+      val options = PoseDetectorOptions.Builder()
+        .setDetectorMode(PoseDetectorOptions.STREAM_MODE)
         .build()
 
-      val options = PoseLandmarkerOptions.builder()
-        .setBaseOptions(baseOptions)
-        .setRunningMode(RunningMode.IMAGE)
-        .setNumPoses(1)
-        .setMinPoseDetectionConfidence(0.5f)
-        .setMinPosePresenceConfidence(0.5f)
-        .setMinTrackingConfidence(0.5f)
-        .build()
-
-      poseLandmarker = PoseLandmarker.createFromOptions(context, options)
+      poseDetector = PoseDetection.getClient(options)
       isInitialized = true
+      println("[PoseExercise] Initialized with ML Kit Pose Detection (no model file needed)")
     }
   }
 
   override fun release() {
-    poseLandmarker?.close()
-    poseLandmarker = null
+    poseDetector?.close()
+    poseDetector = null
     isInitialized = false
     _status = SessionStatus.IDLE
     resetSession()
@@ -154,61 +187,98 @@ class NitroPoseExercises : HybridNitroPoseExercisesSpec() {
   }
 
   // ═══════════════════════════════════════════════════════════
-  // Frame Processing
+  // Frame Processing (ML Kit — async with cached results)
   // ═══════════════════════════════════════════════════════════
 
-  override fun processFrame(frame: HybridFrameSpec) {
+ override fun processFrame(frame: HybridFrameSpec) {
     if (_status != SessionStatus.ACTIVE && _status != SessionStatus.COUNTDOWN) return
-    if (!isInitialized || poseLandmarker == null) return
+    if (!isInitialized || poseDetector == null) return
 
+    // Frame throttle
     frameCount++
     if (frameCount % processEveryNFrames != 0) return
 
     try {
-      // Get the native buffer from VisionCamera frame
+      // Get raw buffer from VisionCamera frame
       val nativeBuffer = frame.getNativeBuffer()
-      val w = frame.width.toInt()
-      val h = frame.height.toInt()
-
-      // On Android, the pointer is an AHardwareBuffer*
-      // Create a Bitmap and use MediaPipe's BitmapImageBuilder
-      val bitmap = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888)
-      val mpImage = BitmapImageBuilder(bitmap).build()
-      val result = poseLandmarker!!.detect(mpImage)
-
-      if (result.landmarks().isNotEmpty()) {
-        val poseLandmarks = result.landmarks()[0]
-
-        if (poseWasLost) {
-          poseWasLost = false
-          onPoseRegained?.invoke()
+      val width = frame.getWidth()
+      val height = frame.getHeight()
+      val bytesPerRow = frame.getBytesPerRow()
+
+      // Create Bitmap from the native buffer
+      val buffer = java.nio.ByteBuffer.allocateDirect(height * bytesPerRow)
+      val pointer = nativeBuffer.pointer
+      // Copy from native pointer to ByteBuffer
+      NativeBufferHelper.copyToByteBuffer(pointer, buffer, height * bytesPerRow)
+
+      val bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
+      buffer.rewind()
+      bitmap.copyPixelsFromBuffer(buffer)
+
+      val inputImage = InputImage.fromBitmap(bitmap, 0)
+
+      // ML Kit is async — fire detection, cache result
+      poseDetector!!.process(inputImage)
+        .addOnSuccessListener { pose ->
+          val poseLandmarks = pose.allPoseLandmarks
+
+          if (poseLandmarks.isNotEmpty()) {
+            if (poseWasLost) {
+              poseWasLost = false
+              onPoseRegained?.invoke()
+            }
+
+            val landmarkArray = Array(34) { Landmark(x = 0.0, y = 0.0, z = 0.0, visibility = 0.0) }
+
+            val imageWidth = width.toDouble()
+            val imageHeight = height.toDouble()
+
+            for (poseLandmark in poseLandmarks) {
+              val mediaPipeIndex = mlKitToMediaPipeMap[poseLandmark.landmarkType] ?: continue
+              if (mediaPipeIndex >= 34) continue
+
+              landmarkArray[mediaPipeIndex] = Landmark(
+                x = poseLandmark.position.x.toDouble() / imageWidth,
+                y = poseLandmark.position.y.toDouble() / imageHeight,
+                z = poseLandmark.position3D.z.toDouble(),
+                visibility = poseLandmark.inFrameLikelihood.toDouble()
+              )
+            }
+
+            synchronized(landmarkLock) {
+              cachedLandmarks = landmarkArray
+            }
+          } else {
+            if (!poseWasLost) {
+              poseWasLost = true
+              onPoseLost?.invoke()
+            }
+            synchronized(landmarkLock) {
+              cachedLandmarks = emptyArray()
+            }
+          }
+
+          bitmap.recycle()
         }
-
-        _landmarks = poseLandmarks.map { lm ->
-          Landmark(
-            x = lm.x().toDouble(),
-            y = lm.y().toDouble(),
-            z = lm.z().toDouble(),
-            visibility = (lm.visibility().orElse(0f)).toDouble()
-          )
-        }.toTypedArray()
-
-        if (_status == SessionStatus.ACTIVE) {
-          processExerciseLogic()
+        .addOnFailureListener { e ->
+          println("[PoseExercise] ML Kit error: ${e.message}")
+          bitmap.recycle()
         }
-      } else {
-        if (!poseWasLost) {
-          poseWasLost = true
-          onPoseLost?.invoke()
-        }
-        _landmarks = emptyArray()
+
+      // Use cached landmarks for exercise logic
+      val currentLandmarks: Array<Landmark>
+      synchronized(landmarkLock) {
+        currentLandmarks = cachedLandmarks.copyOf()
       }
 
-      bitmap.recycle()
-      nativeBuffer.release()
+      _landmarks = currentLandmarks
+
+      if (currentLandmarks.isNotEmpty() && _status == SessionStatus.ACTIVE) {
+        processExerciseLogic()
+      }
 
     } catch (e: Exception) {
-      // MediaPipe detection failed — skip this frame
+      println("[PoseExercise] Frame processing error: ${e.message}")
     }
   }
 
@@ -230,6 +300,9 @@ class NitroPoseExercises : HybridNitroPoseExercisesSpec() {
 
       if (a >= _landmarks.size || b >= _landmarks.size || c >= _landmarks.size) continue
 
+      // Only calculate if all three landmarks have reasonable confidence
+      if (_landmarks[a].visibility < 0.3 || _landmarks[b].visibility < 0.3 || _landmarks[c].visibility < 0.3) continue
+
       val angle = calculateAngle(_landmarks[a], _landmarks[b], _landmarks[c])
       currentAngles[angleDef.name] = angle
       angleSnapshots.add(AngleSnapshot(name = angleDef.name, value = angle))
@@ -365,6 +438,7 @@ class NitroPoseExercises : HybridNitroPoseExercisesSpec() {
 
     for (rule in config.formRules) {
       val angle = currentAngles[rule.angleName] ?: continue
+
       val isViolating = angle < rule.minAngle || angle > rule.maxAngle
 
       if (isViolating) {
@@ -407,18 +481,24 @@ class NitroPoseExercises : HybridNitroPoseExercisesSpec() {
     }
 
     if (inPosition) {
-      if (holdStartTime == null) holdStartTime = System.currentTimeMillis()
+      if (holdStartTime == null) {
+        holdStartTime = System.currentTimeMillis()
+      }
 
       val elapsed = (System.currentTimeMillis() - holdStartTime!!).toDouble()
       val stability = min(100.0, max(0.0, repFormScore))
 
-      onHoldProgress?.invoke(HoldProgress(
+      val progress = HoldProgress(
         elapsedMs = elapsed,
         targetMs = config.holdDurationMs,
         stability = stability
-      ))
+      )
 
-      if (elapsed >= config.holdDurationMs) completeSession()
+      onHoldProgress?.invoke(progress)
+
+      if (elapsed >= config.holdDurationMs) {
+        completeSession()
+      }
     } else {
       holdStartTime = null
     }
@@ -435,14 +515,16 @@ class NitroPoseExercises : HybridNitroPoseExercisesSpec() {
     val avgRepDuration = if (allRepDurations.isEmpty()) 0.0 else allRepDurations.average()
     val avgFormScore = if (allRepFormScores.isEmpty()) 100.0 else allRepFormScores.average()
 
-    onSessionComplete?.invoke(SessionResult(
+    val result = SessionResult(
       totalReps = _repCount,
       totalDurationMs = totalDuration,
       averageRepDurationMs = avgRepDuration,
       averageFormScore = avgFormScore,
       formViolations = sessionFormViolations.toTypedArray(),
       angleHistory = repAngleSnapshots
-    ))
+    )
+
+    onSessionComplete?.invoke(result)
   }
 
   // ═══════════════════════════════════════════════════════════
@@ -482,5 +564,9 @@ class NitroPoseExercises : HybridNitroPoseExercisesSpec() {
     poseWasLost = false
     targetReps = 0.0
     countdownSeconds = 0.0
+    frameCount = 0
+    synchronized(landmarkLock) {
+      cachedLandmarks = emptyArray()
+    }
   }
 }

package/ios/NitroPoseExercises.swift

@@ -1,15 +1,43 @@
+// ios/HybridPoseExercise.swift
+
 import Foundation
 import NitroModules
-import MediaPipeTasksVision
 import VisionCamera
 import AVFoundation
+import Vision
 
-class NitroPoseExercises: HybridNitroPoseExercisesSpec {
+class HybridPoseExercise: HybridNitroPoseExercisesSpec {
 
-  // ─── MediaPipe ──────────────────────────────────────────────
-  private var poseLandmarker: PoseLandmarker?
+  // ─── Vision Framework ───────────────────────────────────────
   private var isInitialized = false
 
+  // ─── Landmark Index Mapping ─────────────────────────────────
+  // Maps Apple Vision joint names to MediaPipe landmark indices
+  // that our JS configs expect
+  private static let visionToMediaPipeMap: [(VNHumanBodyPoseObservation.JointName, Int)] = [
+    (.nose, 0),
+    (.leftShoulder, 11),
+    (.rightShoulder, 12),
+    (.leftElbow, 13),
+    (.rightElbow, 14),
+    (.leftWrist, 15),
+    (.rightWrist, 16),
+    (.leftHip, 23),
+    (.rightHip, 24),
+    (.leftKnee, 25),
+    (.rightKnee, 26),
+    (.leftAnkle, 27),
+    (.rightAnkle, 28),
+    // Vision also provides these but they map to non-standard indices
+    // We include them for skeleton drawing
+    (.neck, 10),         // approximate — MediaPipe doesn't have neck
+    (.root, 33),         // hip center — not in MediaPipe, we skip
+    (.leftEar, 7),
+    (.rightEar, 8),
+    (.leftEye, 2),
+    (.rightEye, 5),
+  ]
+
   // ─── Exercise Config ────────────────────────────────────────
   private var exerciseConfig: ExerciseConfig?
 
@@ -34,9 +62,6 @@ class NitroPoseExercises: HybridNitroPoseExercisesSpec {
   private var countdownSeconds: Double = 0
   private var countdownTimer: Timer?
 
-  private var frameCount: Int = 0
-  private let processEveryNFrames: Int = 3  // Only process every 3rd frame
-
   // ─── Form Tracking ──────────────────────────────────────────
   private var lastFormFeedbackTime: [String: Date] = [:]
   private var sessionFormViolations: [FormFeedback] = []
@@ -48,6 +73,10 @@ class NitroPoseExercises: HybridNitroPoseExercisesSpec {
   // ─── Pose Tracking ──────────────────────────────────────────
   private var poseWasLost = false
 
+  // ─── Frame Throttle ─────────────────────────────────────────
+  private var frameCount: Int = 0
+  private let processEveryNFrames: Int = 3
+
   // ─── Callbacks ──────────────────────────────────────────────
   var onRepComplete: ((_ data: RepData) -> Void)?
   var onPhaseChange: ((_ phase: ExercisePhase) -> Void)?
@@ -65,28 +94,15 @@ class NitroPoseExercises: HybridNitroPoseExercisesSpec {
   // ═══════════════════════════════════════════════════════════
 
   func initialize(modelPath: String) throws -> Promise<Void> {
-    print("[PoseExercise] initialize called with path: \(modelPath)")
-
+    // No model loading needed — Vision framework is built into iOS
     return Promise.async { [weak self] in
       guard let self = self else { return }
-
-      let options = PoseLandmarkerOptions()
-      options.baseOptions.modelAssetPath = modelPath
-      options.runningMode = .image
-      options.numPoses = 1
-      options.minPoseDetectionConfidence = 0.5
-      options.minPosePresenceConfidence = 0.5
-      options.minTrackingConfidence = 0.5
-
-      self.poseLandmarker = try PoseLandmarker(options: options)
       self.isInitialized = true
-
-      print("[PoseExercise] MediaPipe initialized successfully")
+      print("[PoseExercise] Initialized with Apple Vision (no model file needed)")
     }
   }
 
   func release() throws {
-    poseLandmarker = nil
     isInitialized = false
     _status = .idle
     resetSession()
@@ -106,8 +122,6 @@ class NitroPoseExercises: HybridNitroPoseExercisesSpec {
   // ═══════════════════════════════════════════════════════════
 
   func startSession(targetReps: Double, countdownSeconds: Double) throws {
-    print("[PoseExercise] startSession called - target: \(targetReps), countdown: \(countdownSeconds)")
-
     resetSession()
     self.targetReps = targetReps
     self.countdownSeconds = countdownSeconds
@@ -138,75 +152,90 @@ class NitroPoseExercises: HybridNitroPoseExercisesSpec {
   }
 
   // ═══════════════════════════════════════════════════════════
-  // MARK: - Frame Processing
+  // MARK: - Frame Processing (Apple Vision)
   // ═══════════════════════════════════════════════════════════
 
-func processFrame(frame: any HybridFrameSpec) throws {
-  print("[PoseExercise] processFrame called, status: \(_status)")
+  func processFrame(frame: any HybridFrameSpec) throws {
+    guard _status == .active || _status == .countdown else { return }
+    guard isInitialized else { return }
 
-  frameCount += 1
-  if frameCount % processEveryNFrames != 0 { return }
+    // Frame throttle
+    frameCount += 1
+    if frameCount % processEveryNFrames != 0 { return }
 
-  guard _status == .active || _status == .countdown else {
-    print("[PoseExercise] Skipping - status is \(_status)")
-    return
-  }
-  guard isInitialized, let landmarker = poseLandmarker else {
-    print("[PoseExercise] Skipping - not initialized: \(isInitialized)")
-    return
-  }
+    // Get CMSampleBuffer from VisionCamera frame
+    guard let nativeFrame = frame as? any NativeFrame,
+          let sampleBuffer = nativeFrame.sampleBuffer else { return }
 
-  guard let nativeFrame = frame as? any NativeFrame else {
-    print("[PoseExercise] Failed to cast to NativeFrame")
-    return
-  }
-  guard let sampleBuffer = nativeFrame.sampleBuffer else {
-    print("[PoseExercise] sampleBuffer is nil")
-    return
-  }
+    // Get pixel buffer — Vision takes CVPixelBuffer directly, no color conversion needed
+    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
 
-  print("[PoseExercise] Got sampleBuffer, running detection...")
+    // Create Vision request
+    let request = VNDetectHumanBodyPoseRequest()
 
-  do {
-    let mpImage = try MPImage(sampleBuffer: sampleBuffer)
-    let result = try landmarker.detect(image: mpImage)
+    // Run synchronously on this frame processor thread
+    let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:])
 
-    print("[PoseExercise] Detection done, landmarks count: \(result.landmarks.count)")
+    do {
+      try handler.perform([request])
+
+      guard let observation = request.results?.first else {
+        // No pose detected
+        if !poseWasLost {
+          poseWasLost = true
+          onPoseLost?()
+        }
+        _landmarks = []
+        return
+      }
 
-    if let poseLandmarks = result.landmarks.first {
+      // Pose detected
       if poseWasLost {
         poseWasLost = false
         onPoseRegained?()
       }
 
-      _landmarks = poseLandmarks.map { lm in
-        Landmark(
-          x: Double(lm.x),
-          y: Double(lm.y),
-          z: Double(lm.z),
-          visibility: Double(lm.visibility ?? 0)
-        )
+      // Map Vision joints to MediaPipe landmark array (34 slots, indices 0-33)
+      // Fill all slots with zero-visibility first
+      var landmarkArray = [Landmark](repeating: Landmark(x: 0, y: 0, z: 0, visibility: 0), count: 34)
+
+      for (jointName, mediaPipeIndex) in HybridPoseExercise.visionToMediaPipeMap {
+        guard mediaPipeIndex < 34 else { continue }
+
+        do {
+          let point = try observation.recognizedPoint(jointName)
+
+          // Vision uses bottom-left origin (0,0 = bottom-left)
+          // MediaPipe uses top-left origin (0,0 = top-left)
+          // Flip Y axis
+          let confidence = Double(point.confidence)
+
+          landmarkArray[mediaPipeIndex] = Landmark(
+            x: Double(point.location.x),
+            y: 1.0 - Double(point.location.y),  // flip Y
+            z: 0,  // Vision doesn't provide Z depth
+            visibility: confidence
+          )
+
+          // Debug logging — uncomment to verify mapping
+          // print("[PoseExercise] \(jointName.rawValue.rawValue) → index \(mediaPipeIndex): x=\(String(format: "%.3f", point.location.x)) y=\(String(format: "%.3f", 1.0 - Double(point.location.y))) conf=\(String(format: "%.2f", confidence))")
+
+        } catch {
+          // Joint not detected — leave as zero visibility
+          continue
+        }
       }
 
-      print("[PoseExercise] Landmarks detected: \(_landmarks.count)")
+      _landmarks = landmarkArray
 
       if _status == .active {
         processExerciseLogic()
       }
 
-    } else {
-      print("[PoseExercise] No pose detected in frame")
-      if !poseWasLost {
-        poseWasLost = true
-        onPoseLost?()
-      }
-      _landmarks = []
+    } catch {
+      print("[PoseExercise] Vision error: \(error.localizedDescription)")
     }
-
-  } catch {
-    print("[PoseExercise] MediaPipe error: \(error.localizedDescription)")
   }
-}
 
   // ═══════════════════════════════════════════════════════════
   // MARK: - Exercise Logic Engine
@@ -216,7 +245,6 @@ func processFrame(frame: any HybridFrameSpec) throws {
     guard let config = exerciseConfig else { return }
     guard !_landmarks.isEmpty else { return }
 
-    // 1. Calculate all angles defined in the config
     var currentAngles: [String: Double] = [:]
     var angleSnapshots: [AngleSnapshot] = []
 
@@ -227,6 +255,11 @@ func processFrame(frame: any HybridFrameSpec) throws {
 
       guard a < _landmarks.count, b < _landmarks.count, c < _landmarks.count else { continue }
 
+      // Only calculate if all three landmarks have reasonable confidence
+      guard _landmarks[a].visibility > 0.3,
+            _landmarks[b].visibility > 0.3,
+            _landmarks[c].visibility > 0.3 else { continue }
+
       let angle = calculateAngle(
         pointA: _landmarks[a],
         vertex: _landmarks[b],
@@ -239,31 +272,25 @@ func processFrame(frame: any HybridFrameSpec) throws {
 
     repAngleSnapshots = angleSnapshots
 
-    // 2. Determine current phase from angle thresholds
+    // Debug logging — uncomment to see angles
+    // for (name, angle) in currentAngles {
+    //   print("[PoseExercise] Angle \(name): \(String(format: "%.1f", angle))°")
+    // }
+
     let detectedPhase = determinePhase(from: currentAngles, config: config)
 
     if detectedPhase != _currentPhase && detectedPhase != .unknown {
       let previousPhase = _currentPhase
       _currentPhase = detectedPhase
       onPhaseChange?(detectedPhase)
-
-      // 3. Update phase history for rep counting
       handlePhaseTransition(from: previousPhase, to: detectedPhase, config: config)
     }
 
-    // 4. Check form rules
     checkFormRules(currentAngles: currentAngles, config: config)
 
-    // 5. Handle hold-based exercises
     if config.type == .hold {
       handleHoldProgress(currentAngles: currentAngles, config: config)
     }
-
-    // Temporary debug logging — remove after testing
-for (name, angle) in currentAngles {
-  print("[PoseExercise] Angle \(name): \(String(format: "%.1f", angle))°")
-}
-print("[PoseExercise] Detected phase: \(detectedPhase), Current phase: \(_currentPhase)")
   }
 
   // ═══════════════════════════════════════════════════════════
@@ -284,9 +311,7 @@ print("[PoseExercise] Detected phase: \(detectedPhase), Current phase: \(_curren
 
     let cosAngle = max(-1.0, min(1.0, dot / (magA * magC)))
     let angleRad = acos(cosAngle)
-    let angleDeg = angleRad * (180.0 / .pi)
-
-    return angleDeg
+    return angleRad * (180.0 / .pi)
   }
 
   // ═══════════════════════════════════════════════════════════
@@ -296,7 +321,6 @@ print("[PoseExercise] Detected phase: \(detectedPhase), Current phase: \(_curren
   private func determinePhase(from angles: [String: Double], config: ExerciseConfig) -> ExercisePhase {
     for phaseThreshold in config.phases {
       guard let angle = angles[phaseThreshold.angleName] else { continue }
-
       if angle >= phaseThreshold.minAngle && angle <= phaseThreshold.maxAngle {
         return phaseThreshold.phase
       }
@@ -308,7 +332,7 @@ print("[PoseExercise] Detected phase: \(detectedPhase), Current phase: \(_curren
   // MARK: - Rep Counting State Machine
   // ═══════════════════════════════════════════════════════════
 
-private func handlePhaseTransition(from previousPhase: ExercisePhase, to newPhase: ExercisePhase, config: ExerciseConfig) {
+  private func handlePhaseTransition(from previousPhase: ExercisePhase, to newPhase: ExercisePhase, config: ExerciseConfig) {
     guard config.type == .rep else { return }
 
     phaseHistory.append(newPhase)
@@ -523,5 +547,6 @@ private func handlePhaseTransition(from previousPhase: ExercisePhase, to newPhas
     countdownSeconds = 0
     countdownTimer?.invalidate()
     countdownTimer = nil
+    frameCount = 0
   }
-}
+}

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "react-native-nitro-pose-exercises",
-  "version": "1.0.9",
-  "description": "Real-time on-device exercise tracking for React Native. Rep counting, form validation, and skeleton overlay powered by MediaPipe Pose Landmarker and VisionCamera v5 via Nitro Modules.",
+  "version": "1.1.2",
+  "description": "Real-time on-device exercise tracking for React Native. Rep counting, form validation, and skeleton overlay powered by Apple Vision (iOS) and Google ML Kit (Android) with VisionCamera v5 via Nitro Modules.",
   "main": "./lib/module/index.js",
   "types": "./lib/typescript/src/index.d.ts",
   "exports": {
@@ -69,9 +69,9 @@
     "@eslint/eslintrc": "^3.3.1",
     "@eslint/js": "^9.35.0",
     "@jest/globals": "^30.0.0",
-    "@react-native/babel-preset": "0.85.0",
-    "@react-native/eslint-config": "0.85.0",
-    "@react-native/jest-preset": "0.85.0",
+    "@react-native/babel-preset": "0.85.3",
+    "@react-native/eslint-config": "0.85.3",
+    "@react-native/jest-preset": "0.85.3",
     "@release-it/conventional-changelog": "^10.0.6",
     "@types/react": "^19.2.0",
     "commitlint": "^20.5.0",
@@ -82,16 +82,16 @@
     "eslint-plugin-prettier": "^5.5.4",
     "jest": "^30.3.0",
     "lefthook": "^2.1.4",
-    "nitrogen": "^0.35.7",
+    "nitrogen": "^0.35.9",
     "prettier": "^3.8.1",
     "react": "19.2.3",
-    "react-native": "0.85.0",
+    "react-native": "0.85.3",
     "react-native-builder-bob": "^0.41.0",
-    "react-native-nitro-modules": "^0.35.7",
+    "react-native-nitro-modules": "^0.35.9",
     "react-native-svg": "^15.15.5",
-    "react-native-vision-camera": "^5.0.10",
-    "react-native-vision-camera-worklets": "^5.0.10",
-    "react-native-worklets": "^0.8.3",
+    "react-native-vision-camera": "^5.0.11",
+    "react-native-vision-camera-worklets": "^5.0.11",
+    "react-native-worklets": "^0.9.1",
     "release-it": "^19.2.4",
     "turbo": "^2.8.21",
     "typescript": "^6.0.2"