react-native-ai-debugger 1.0.7 → 1.0.8

package/README.md

@@ -17,7 +17,7 @@ An MCP (Model Context Protocol) server for AI-powered React Native debugging. En
 -   **Android device control** - screenshots, tap, swipe, text input, key events via ADB
 -   **iOS simulator control** - screenshots, app management, URL handling via simctl
 -   **iOS UI automation** - tap, swipe, text input, button presses via IDB (optional)
--   **iOS accessibility inspection** - get UI element tree and element info at coordinates via IDB
+-   **Element-based UI automation** - find and wait for elements by text/label without screenshots (faster, cheaper)
 
 ## Requirements
 
@@ -133,55 +133,35 @@ Requires VS Code 1.102+ with Copilot ([docs](https://code.visualstudio.com/docs/
 
 ### Android (ADB)
 
-| Tool                      | Description                                          |
-| ------------------------- | ---------------------------------------------------- |
-| `list_android_devices`    | List connected Android devices and emulators via ADB |
-| `android_screenshot`      | Take a screenshot from an Android device/emulator    |
-| `android_install_app`     | Install an APK on an Android device/emulator         |
-| `android_launch_app`      | Launch an app by package name                        |
-| `android_list_packages`   | List installed packages (with optional filter)       |
-| `android_tap`             | Tap at specific coordinates on screen                |
-| `android_long_press`      | Long press at specific coordinates                   |
-| `android_swipe`           | Swipe from one point to another                      |
-| `android_input_text`      | Type text at current focus point                     |
-| `android_key_event`       | Send key events (HOME, BACK, ENTER, etc.)            |
-| `android_get_screen_size` | Get device screen resolution                         |
-
-### Android Accessibility (UI Hierarchy)
-
-| Tool                    | Description                                           |
-| ----------------------- | ----------------------------------------------------- |
-| `android_describe_all`  | Get full UI hierarchy tree using uiautomator          |
-| `android_describe_point`| Get UI element info at specific coordinates           |
-| `android_tap_element`   | Tap element by text, content-desc, or resource-id     |
+| Tool                        | Description                                                   |
+| --------------------------- | ------------------------------------------------------------- |
+| `list_android_devices`      | List connected Android devices and emulators via ADB          |
+| `android_screenshot`        | Take a screenshot from an Android device/emulator             |
+| `android_install_app`       | Install an APK on an Android device/emulator                  |
+| `android_launch_app`        | Launch an app by package name                                 |
+| `android_list_packages`     | List installed packages (with optional filter)                |
+| `android_tap`               | Tap at specific coordinates on screen                         |
+| `android_long_press`        | Long press at specific coordinates                            |
+| `android_swipe`             | Swipe from one point to another                               |
+| `android_input_text`        | Type text at current focus point                              |
+| `android_key_event`         | Send key events (HOME, BACK, ENTER, etc.)                     |
+| `android_get_screen_size`   | Get device screen resolution                                  |
+| `android_find_element`      | Find element by text/contentDesc/resourceId (no screenshot)   |
+| `android_wait_for_element`  | Wait for element to appear (useful for screen transitions)    |
 
 ### iOS (Simulator)
 
-| Tool                  | Description                                 |
-| --------------------- | ------------------------------------------- |
-| `list_ios_simulators` | List available iOS simulators               |
-| `ios_screenshot`      | Take a screenshot from an iOS simulator     |
-| `ios_install_app`     | Install an app bundle (.app) on a simulator |
-| `ios_launch_app`      | Launch an app by bundle ID                  |
-| `ios_open_url`        | Open a URL (deep links or web URLs)         |
-| `ios_terminate_app`   | Terminate a running app                     |
-| `ios_boot_simulator`  | Boot a simulator by UDID                    |
-
-### iOS UI Interaction (requires IDB)
-
-These tools require [Facebook IDB](https://fbidb.io/) to be installed: `brew install idb-companion`
-
-| Tool                | Description                                           |
-| ------------------- | ----------------------------------------------------- |
-| `ios_tap`           | Tap at specific coordinates on screen                 |
-| `ios_tap_element`   | Tap an element by its accessibility label             |
-| `ios_swipe`         | Swipe from one point to another                       |
-| `ios_input_text`    | Type text into the active input field                 |
-| `ios_button`        | Press hardware buttons (HOME, LOCK, SIRI, etc.)       |
-| `ios_key_event`     | Send a key event by keycode                           |
-| `ios_key_sequence`  | Send multiple key events in sequence                  |
-| `ios_describe_all`  | Get accessibility tree for entire screen              |
-| `ios_describe_point`| Get accessibility info for element at specific point  |
+| Tool                    | Description                                                     |
+| ----------------------- | --------------------------------------------------------------- |
+| `list_ios_simulators`   | List available iOS simulators                                   |
+| `ios_screenshot`        | Take a screenshot from an iOS simulator                         |
+| `ios_install_app`       | Install an app bundle (.app) on a simulator                     |
+| `ios_launch_app`        | Launch an app by bundle ID                                      |
+| `ios_open_url`          | Open a URL (deep links or web URLs)                             |
+| `ios_terminate_app`     | Terminate a running app                                         |
+| `ios_boot_simulator`    | Boot a simulator by UDID                                        |
+| `ios_find_element`      | Find element by label/value (requires IDB, no screenshot)       |
+| `ios_wait_for_element`  | Wait for element to appear (requires IDB)                       |
 
 ## Usage
 
@@ -411,20 +391,6 @@ Set `awaitPromise=false` for synchronous execution only.
 
 ## Device Interaction
 
-> **💡 Best Practice: Use Element-Based Tapping**
->
-> Prefer `android_tap_element` and `ios_tap_element` over coordinate-based tapping (`android_tap`, `ios_tap`).
->
-> **Why?**
-> - More reliable - elements are found by text/label, not fragile coordinates
-> - Self-documenting - `tap_element(text="Settings")` is clearer than `tap(x=540, y=1200)`
-> - Resolution-independent - works across different screen sizes
->
-> **Workflow:**
-> 1. Use `android_describe_all` or `ios_describe_all` to see available elements
-> 2. Use `android_tap_element` or `ios_tap_element` to interact by text/label
-> 3. Fall back to coordinate-based tapping only when elements lack accessible text
-
 ### Android (requires ADB)
 
 List connected devices:
@@ -466,53 +432,6 @@ android_key_event with key="HOME"
 android_key_event with key="ENTER"
 ```
 
-### Android UI Automation (Accessibility)
-
-Get the full UI hierarchy:
-
-```
-android_describe_all
-```
-
-Example output:
-```
-[FrameLayout] frame=(0, 0, 1080x2340) tap=(540, 1170)
-  [LinearLayout] frame=(0, 63, 1080x147) tap=(540, 136)
-    [TextView] "Settings" frame=(48, 77, 200x63) tap=(148, 108)
-  [RecyclerView] frame=(0, 210, 1080x2130) tap=(540, 1275)
-    [Button] "Save" frame=(800, 2200, 200x80) tap=(900, 2240)
-```
-
-Get element info at coordinates:
-
-```
-android_describe_point with x=540 y=1170
-```
-
-Tap an element by text:
-
-```
-android_tap_element with text="Settings"
-```
-
-Tap using partial text match:
-
-```
-android_tap_element with textContains="Save"
-```
-
-Tap by resource ID:
-
-```
-android_tap_element with resourceId="save_button"
-```
-
-Tap by content description:
-
-```
-android_tap_element with contentDesc="Navigate up"
-```
-
 ### iOS Simulator (requires Xcode)
 
 List available simulators:
@@ -545,78 +464,91 @@ Open a deep link:
 ios_open_url with url="myapp://settings"
 ```
 
-### iOS UI Automation (requires IDB)
-
-Install IDB first: `brew install idb-companion`
+## Efficient UI Automation (No Screenshots)
 
-**Important: Coordinate System**
-- iOS IDB uses **points** (logical coordinates), not pixels
-- For 2x Retina displays: 1 point = 2 pixels
-- Example: 1640x2360 pixel screenshot = 820x1180 points
-- Use `ios_describe_all` to get exact element coordinates in points
+For action triggering without layout debugging, use element-based tools instead of screenshots. This is **2-3x faster** and uses fewer tokens.
 
-Tap on screen (coordinates in points):
+### Android - Find and Tap by Text
 
 ```
-ios_tap with x=200 y=400
-```
+# Wait for screen to load
+android_wait_for_element with text="Login"
 
-Long press (hold for 2 seconds):
+# Find element (returns tap coordinates)
+android_find_element with textContains="submit"
 
-```
-ios_tap with x=200 y=400 duration=2
+# Tap the element (use coordinates from find_element)
+android_tap with x=540 y=960
 ```
 
-Swipe gesture:
+Search options:
+- `text` - exact text match
+- `textContains` - partial text (case-insensitive)
+- `contentDesc` - accessibility content description
+- `contentDescContains` - partial content description
+- `resourceId` - resource ID (e.g., "button" or "com.app:id/button")
 
-```
-ios_swipe with startX=200 startY=600 endX=200 endY=200
-```
+### iOS - Find and Tap by Label (requires IDB)
 
-Type text (tap input field first):
-
-```
-ios_tap with x=200 y=300
-ios_input_text with text="hello@example.com"
+```bash
+# Install IDB first
+brew install idb-companion
 ```
 
-Press hardware buttons:
-
 ```
-ios_button with button="HOME"
-ios_button with button="LOCK"
-ios_button with button="SIRI"
+# Wait for element
+ios_wait_for_element with label="Sign In"
+
+# Find element by partial label
+ios_find_element with labelContains="welcome"
 ```
 
-Get accessibility info for the screen:
+Search options:
+- `label` - exact accessibility label
+- `labelContains` - partial label (case-insensitive)
+- `value` - accessibility value
+- `valueContains` - partial value
+- `type` - element type (e.g., "Button", "TextField")
 
-```
-ios_describe_all
-```
+### Wait for Screen Transitions
 
-Get accessibility info at a specific point:
+Both platforms support waiting with timeout:
 
 ```
-ios_describe_point with x=200 y=400
+android_wait_for_element with text="Dashboard" timeoutMs=15000 pollIntervalMs=500
+ios_wait_for_element with label="Home" timeoutMs=10000
 ```
 
-Tap an element by accessibility label:
+### Recommended Workflow (Priority Order)
 
-```
-ios_tap_element with label="Settings"
-```
+**Always try accessibility tools first, fall back to screenshots only when needed:**
 
-Tap using partial label match:
+1. **Wait for screen** → Use `wait_for_element` with expected text/label
+2. **Find target** → Use `find_element` to get tap coordinates
+3. **Tap** → Use `tap` with coordinates from step 2
+4. **Fallback** → If element not in accessibility tree, use `screenshot`
 
 ```
-ios_tap_element with labelContains="Sign"
+# Example: Tap "Submit" button after screen loads
+android_wait_for_element with text="Submit"     # Step 1: Wait
+android_find_element with text="Submit"         # Step 2: Find (returns center coordinates)
+android_tap with x=540 y=1200                   # Step 3: Tap (use returned coordinates)
 ```
 
-When multiple elements match, use index (0-based):
+**Why this order?**
+- `find_element`: ~100-200 tokens, <100ms
+- `screenshot`: ~400-500 tokens, 200-500ms
 
-```
-ios_tap_element with labelContains="Button" index=1
-```
+### When to Use Screenshots vs Element Tools
+
+| Use Case | Recommended Tool |
+|----------|------------------|
+| Trigger button taps | `find_element` + `tap` |
+| Wait for screen load | `wait_for_element` |
+| Navigate through flow | `wait_for_element` + `tap` |
+| Debug layout issues | `screenshot` |
+| Verify visual appearance | `screenshot` |
+| Find elements without labels | `screenshot` |
 
 ## Supported React Native Versions
 

package/build/core/android.d.ts

@@ -96,6 +96,81 @@ export declare function androidInputText(text: string, deviceId?: string): Promi
  * Send a key event to an Android device
  */
 export declare function androidKeyEvent(keyCode: number | keyof typeof ANDROID_KEY_EVENTS, deviceId?: string): Promise<AdbResult>;
+/**
+ * UI Element from accessibility tree
+ */
+export interface AndroidUIElement {
+    text: string;
+    contentDesc: string;
+    resourceId: string;
+    className: string;
+    bounds: {
+        left: number;
+        top: number;
+        right: number;
+        bottom: number;
+        width: number;
+        height: number;
+    };
+    center: {
+        x: number;
+        y: number;
+    };
+    clickable: boolean;
+    enabled: boolean;
+    focused: boolean;
+    scrollable: boolean;
+    selected: boolean;
+}
+/**
+ * Result of element find operations
+ */
+export interface FindElementResult {
+    success: boolean;
+    found: boolean;
+    element?: AndroidUIElement;
+    allMatches?: AndroidUIElement[];
+    matchCount?: number;
+    error?: string;
+}
+/**
+ * Result of wait for element operations
+ */
+export interface WaitForElementResult extends FindElementResult {
+    elapsedMs?: number;
+    timedOut?: boolean;
+}
+/**
+ * Options for finding elements
+ */
+export interface FindElementOptions {
+    text?: string;
+    textContains?: string;
+    contentDesc?: string;
+    contentDescContains?: string;
+    resourceId?: string;
+    index?: number;
+}
+/**
+ * Get UI accessibility tree from Android device using uiautomator
+ */
+export declare function androidGetUITree(deviceId?: string): Promise<{
+    success: boolean;
+    elements?: AndroidUIElement[];
+    rawXml?: string;
+    error?: string;
+}>;
+/**
+ * Find element(s) in the UI tree matching the given criteria
+ */
+export declare function androidFindElement(options: FindElementOptions, deviceId?: string): Promise<FindElementResult>;
+/**
+ * Wait for element to appear on screen with polling
+ */
+export declare function androidWaitForElement(options: FindElementOptions & {
+    timeoutMs?: number;
+    pollIntervalMs?: number;
+}, deviceId?: string): Promise<WaitForElementResult>;
 /**
  * Get device screen size
  */

package/build/core/android.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"android.d.ts","sourceRoot":"","sources":["../../src/core/android.ts"],"names":[],"mappings":"AAgBA,MAAM,WAAW,aAAa;IAC1B,EAAE,EAAE,MAAM,CAAC;IACX,MAAM,EAAE,QAAQ,GAAG,SAAS,GAAG,cAAc,GAAG,gBAAgB,GAAG,MAAM,CAAC;IAC1E,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,WAAW,CAAC,EAAE,MAAM,CAAC;CACxB;AAGD,MAAM,WAAW,SAAS;IACtB,OAAO,EAAE,OAAO,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,IAAI,CAAC,EAAE,MAAM,CAAC;IAEd,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,cAAc,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED;;GAEG;AACH,wBAAsB,cAAc,IAAI,OAAO,CAAC,OAAO,CAAC,CAOvD;AAED;;GAEG;AACH,wBAAsB,kBAAkB,IAAI,OAAO,CAAC,SAAS,CAAC,CA6D7D;AAED;;GAEG;AACH,wBAAsB,uBAAuB,IAAI,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,CAiBtE;AASD;;GAEG;AACH,wBAAsB,iBAAiB,CACnC,UAAU,CAAC,EAAE,MAAM,EACnB,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAqFpB;AAED;;GAEG;AACH,wBAAsB,iBAAiB,CACnC,OAAO,EAAE,MAAM,EACf,QAAQ,CAAC,EAAE,MAAM,EACjB,OAAO,CAAC,EAAE;IAAE,OAAO,CAAC,EAAE,OAAO,CAAC;IAAC,gBAAgB,CAAC,EAAE,OAAO,CAAA;CAAE,GAC5D,OAAO,CAAC,SAAS,CAAC,CA0DpB;AAED;;GAEG;AACH,wBAAsB,gBAAgB,CAClC,WAAW,EAAE,MAAM,EACnB,YAAY,CAAC,EAAE,MAAM,EACrB,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAmDpB;AAED;;GAEG;AACH,wBAAsB,mBAAmB,CACrC,QAAQ,CAAC,EAAE,MAAM,EACjB,MAAM,CAAC,EAAE,MAAM,GAChB,OAAO,CAAC,SAAS,CAAC,CAoDpB;AAMD;;GAEG;AACH,eAAO,MAAM,kBAAkB;;;;;;;;;;;;;;;;;;;;;;;CAuBrB,CAAC;AAEX;;GAEG;AACH,wBAAsB,UAAU,CAC5B,CAAC,EAAE,MAAM,EACT,CAAC,EAAE,MAAM,EACT,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAkCpB;AAED;;GAEG;AACH,wBAAsB,gBAAgB,CAClC,CAAC,EAAE,MAAM,EACT,CAAC,EAAE,MAAM,EACT,UAAU,GAAE,MAAa,EACzB,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAuCpB;AAED;;GAEG;AACH,wBAAsB,YAAY,CAC9B,MAAM,EAAE,MAAM,EACd,MAAM,EAAE,MAAM,EACd,IAAI,EAAE,MAAM,EACZ,IAAI,EAAE,MAAM,EACZ,UAAU,GAAE,MAAY,EACxB,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAwCpB;AAED;;;;;GAKG;AACH,wBAAsB,gBAAgB,CAClC,IAAI,EAAE,MAAM,EACZ,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CA+GpB;AAED;;GAEG;AACH,wBAAsB,eAAe,CACjC,OAAO,EAAE,MAAM,GAAG,MAAM,OAAO,kBAAkB,EACjD,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAoDpB;AAED;;GAEG;AACH,wBAAsB,oBAAoB,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;IACnE,OAAO,EAAE,OAAO,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,KAAK,CAAC,EAAE,MAAM,CAAC;CAClB,CAAC,CA4CD;AAMD;;GAEG;AACH,MAAM,WAAW,2BAA2B;IACxC,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,MAAM,EAAE;QACJ,IAAI,EAAE,MAAM,CAAC;QACb,GAAG,EAAE,MAAM,CAAC;QACZ,KAAK,EAAE,MAAM,CAAC;QACd,MAAM,EAAE,MAAM,CAAC;KAClB,CAAC;IACF,KAAK,EAAE;QACH,CAAC,EAAE,MAAM,CAAC;QACV,CAAC,EAAE,MAAM,CAAC;QACV,KAAK,EAAE,MAAM,CAAC;QACd,MAAM,EAAE,MAAM,CAAC;KAClB,CAAC;IACF,GAAG,EAAE;QACD,CAAC,EAAE,MAAM,CAAC;QACV,CAAC,EAAE,MAAM,CAAC;KACb,CAAC;IACF,QAAQ,EAAE,2BAA2B,EAAE,CAAC;IAExC,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,UAAU,CAAC,EAAE,OAAO,CAAC;IACrB,QAAQ,CAAC,EAAE,OAAO,CAAC;CACtB;AAED;;GAEG;AACH,MAAM,WAAW,qBAAqB;IAClC,OAAO,EAAE,OAAO,CAAC;IACjB,QAAQ,CAAC,EAAE,2BAA2B,EAAE,CAAC;IACzC,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,KAAK,CAAC,EAAE,MAAM,CAAC;CAClB;AAmKD;;GAEG;AACH,wBAAsB,kBAAkB,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,qBAAqB,CAAC,CAuE1F;AAED;;GAEG;AACH,wBAAsB,oBAAoB,CACtC,CAAC,EAAE,MAAM,EACT,CAAC,EAAE,MAAM,EACT,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,qBAAqB,CAAC,CA2EhC;AAED;;GAEG;AACH,wBAAsB,iBAAiB,CACnC,OAAO,EAAE;IACL,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,mBAAmB,CAAC,EAAE,MAAM,CAAC;IAC7B,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,QAAQ,CAAC,EAAE,MAAM,CAAC;CACrB,GACF,OAAO,CAAC,SAAS,CAAC,CAqFpB"}
+{"version":3,"file":"android.d.ts","sourceRoot":"","sources":["../../src/core/android.ts"],"names":[],"mappings":"AAgBA,MAAM,WAAW,aAAa;IAC1B,EAAE,EAAE,MAAM,CAAC;IACX,MAAM,EAAE,QAAQ,GAAG,SAAS,GAAG,cAAc,GAAG,gBAAgB,GAAG,MAAM,CAAC;IAC1E,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,WAAW,CAAC,EAAE,MAAM,CAAC;CACxB;AAGD,MAAM,WAAW,SAAS;IACtB,OAAO,EAAE,OAAO,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,IAAI,CAAC,EAAE,MAAM,CAAC;IAEd,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,cAAc,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED;;GAEG;AACH,wBAAsB,cAAc,IAAI,OAAO,CAAC,OAAO,CAAC,CAOvD;AAED;;GAEG;AACH,wBAAsB,kBAAkB,IAAI,OAAO,CAAC,SAAS,CAAC,CA6D7D;AAED;;GAEG;AACH,wBAAsB,uBAAuB,IAAI,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,CAiBtE;AASD;;GAEG;AACH,wBAAsB,iBAAiB,CACnC,UAAU,CAAC,EAAE,MAAM,EACnB,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAqFpB;AAED;;GAEG;AACH,wBAAsB,iBAAiB,CACnC,OAAO,EAAE,MAAM,EACf,QAAQ,CAAC,EAAE,MAAM,EACjB,OAAO,CAAC,EAAE;IAAE,OAAO,CAAC,EAAE,OAAO,CAAC;IAAC,gBAAgB,CAAC,EAAE,OAAO,CAAA;CAAE,GAC5D,OAAO,CAAC,SAAS,CAAC,CA0DpB;AAED;;GAEG;AACH,wBAAsB,gBAAgB,CAClC,WAAW,EAAE,MAAM,EACnB,YAAY,CAAC,EAAE,MAAM,EACrB,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAmDpB;AAED;;GAEG;AACH,wBAAsB,mBAAmB,CACrC,QAAQ,CAAC,EAAE,MAAM,EACjB,MAAM,CAAC,EAAE,MAAM,GAChB,OAAO,CAAC,SAAS,CAAC,CAoDpB;AAMD;;GAEG;AACH,eAAO,MAAM,kBAAkB;;;;;;;;;;;;;;;;;;;;;;;CAuBrB,CAAC;AAEX;;GAEG;AACH,wBAAsB,UAAU,CAC5B,CAAC,EAAE,MAAM,EACT,CAAC,EAAE,MAAM,EACT,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAkCpB;AAED;;GAEG;AACH,wBAAsB,gBAAgB,CAClC,CAAC,EAAE,MAAM,EACT,CAAC,EAAE,MAAM,EACT,UAAU,GAAE,MAAa,EACzB,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAuCpB;AAED;;GAEG;AACH,wBAAsB,YAAY,CAC9B,MAAM,EAAE,MAAM,EACd,MAAM,EAAE,MAAM,EACd,IAAI,EAAE,MAAM,EACZ,IAAI,EAAE,MAAM,EACZ,UAAU,GAAE,MAAY,EACxB,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAwCpB;AAED;;;;;GAKG;AACH,wBAAsB,gBAAgB,CAClC,IAAI,EAAE,MAAM,EACZ,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CA+GpB;AAED;;GAEG;AACH,wBAAsB,eAAe,CACjC,OAAO,EAAE,MAAM,GAAG,MAAM,OAAO,kBAAkB,EACjD,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,SAAS,CAAC,CAoDpB;AAMD;;GAEG;AACH,MAAM,WAAW,gBAAgB;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,WAAW,EAAE,MAAM,CAAC;IACpB,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,MAAM,CAAC;IAClB,MAAM,EAAE;QACJ,IAAI,EAAE,MAAM,CAAC;QACb,GAAG,EAAE,MAAM,CAAC;QACZ,KAAK,EAAE,MAAM,CAAC;QACd,MAAM,EAAE,MAAM,CAAC;QACf,KAAK,EAAE,MAAM,CAAC;QACd,MAAM,EAAE,MAAM,CAAC;KAClB,CAAC;IACF,MAAM,EAAE;QAAE,CAAC,EAAE,MAAM,CAAC;QAAC,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IACjC,SAAS,EAAE,OAAO,CAAC;IACnB,OAAO,EAAE,OAAO,CAAC;IACjB,OAAO,EAAE,OAAO,CAAC;IACjB,UAAU,EAAE,OAAO,CAAC;IACpB,QAAQ,EAAE,OAAO,CAAC;CACrB;AAED;;GAEG;AACH,MAAM,WAAW,iBAAiB;IAC9B,OAAO,EAAE,OAAO,CAAC;IACjB,KAAK,EAAE,OAAO,CAAC;IACf,OAAO,CAAC,EAAE,gBAAgB,CAAC;IAC3B,UAAU,CAAC,EAAE,gBAAgB,EAAE,CAAC;IAChC,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,KAAK,CAAC,EAAE,MAAM,CAAC;CAClB;AAED;;GAEG;AACH,MAAM,WAAW,oBAAqB,SAAQ,iBAAiB;IAC3D,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,QAAQ,CAAC,EAAE,OAAO,CAAC;CACtB;AAED;;GAEG;AACH,MAAM,WAAW,kBAAkB;IAC/B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,mBAAmB,CAAC,EAAE,MAAM,CAAC;IAC7B,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,KAAK,CAAC,EAAE,MAAM,CAAC;CAClB;AA+FD;;GAEG;AACH,wBAAsB,gBAAgB,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;IAC/D,OAAO,EAAE,OAAO,CAAC;IACjB,QAAQ,CAAC,EAAE,gBAAgB,EAAE,CAAC;IAC9B,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,KAAK,CAAC,EAAE,MAAM,CAAC;CAClB,CAAC,CAiDD;AAED;;GAEG;AACH,wBAAsB,kBAAkB,CACpC,OAAO,EAAE,kBAAkB,EAC3B,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,iBAAiB,CAAC,CA2D5B;AAED;;GAEG;AACH,wBAAsB,qBAAqB,CACvC,OAAO,EAAE,kBAAkB,GAAG;IAC1B,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,cAAc,CAAC,EAAE,MAAM,CAAC;CAC3B,EACD,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,oBAAoB,CAAC,CA+C/B;AAED;;GAEG;AACH,wBAAsB,oBAAoB,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;IACnE,OAAO,EAAE,OAAO,CAAC;IACjB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,KAAK,CAAC,EAAE,MAAM,CAAC;CAClB,CAAC,CA4CD;AAMD;;GAEG;AACH,MAAM,WAAW,2BAA2B;IACxC,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,MAAM,EAAE;QACJ,IAAI,EAAE,MAAM,CAAC;QACb,GAAG,EAAE,MAAM,CAAC;QACZ,KAAK,EAAE,MAAM,CAAC;QACd,MAAM,EAAE,MAAM,CAAC;KAClB,CAAC;IACF,KAAK,EAAE;QACH,CAAC,EAAE,MAAM,CAAC;QACV,CAAC,EAAE,MAAM,CAAC;QACV,KAAK,EAAE,MAAM,CAAC;QACd,MAAM,EAAE,MAAM,CAAC;KAClB,CAAC;IACF,GAAG,EAAE;QACD,CAAC,EAAE,MAAM,CAAC;QACV,CAAC,EAAE,MAAM,CAAC;KACb,CAAC;IACF,QAAQ,EAAE,2BAA2B,EAAE,CAAC;IAExC,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,UAAU,CAAC,EAAE,OAAO,CAAC;IACrB,QAAQ,CAAC,EAAE,OAAO,CAAC;CACtB;AAED;;GAEG;AACH,MAAM,WAAW,qBAAqB;IAClC,OAAO,EAAE,OAAO,CAAC;IACjB,QAAQ,CAAC,EAAE,2BAA2B,EAAE,CAAC;IACzC,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,KAAK,CAAC,EAAE,MAAM,CAAC;CAClB;AAmKD;;GAEG;AACH,wBAAsB,kBAAkB,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,qBAAqB,CAAC,CAuE1F;AAED;;GAEG;AACH,wBAAsB,oBAAoB,CACtC,CAAC,EAAE,MAAM,EACT,CAAC,EAAE,MAAM,EACT,QAAQ,CAAC,EAAE,MAAM,GAClB,OAAO,CAAC,qBAAqB,CAAC,CA2EhC;AAED;;GAEG;AACH,wBAAsB,iBAAiB,CACnC,OAAO,EAAE;IACL,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,mBAAmB,CAAC,EAAE,MAAM,CAAC;IAC7B,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,QAAQ,CAAC,EAAE,MAAM,CAAC;CACrB,GACF,OAAO,CAAC,SAAS,CAAC,CAqFpB"}

package/build/core/android.js

@@ -641,6 +641,245 @@ export async function androidKeyEvent(keyCode, deviceId) {
         };
     }
 }
+/**
+ * Parse bounds string like "[0,0][1080,1920]" to AndroidUIElement bounds
+ */
+function parseBoundsForUIElement(boundsStr) {
+    const match = boundsStr.match(/\[(\d+),(\d+)\]\[(\d+),(\d+)\]/);
+    if (!match)
+        return null;
+    const left = parseInt(match[1], 10);
+    const top = parseInt(match[2], 10);
+    const right = parseInt(match[3], 10);
+    const bottom = parseInt(match[4], 10);
+    return {
+        left,
+        top,
+        right,
+        bottom,
+        width: right - left,
+        height: bottom - top
+    };
+}
+/**
+ * Parse uiautomator XML dump into element array
+ */
+function parseUIAutomatorXML(xml) {
+    const elements = [];
+    // Match all node elements with their attributes
+    const nodeRegex = /<node\s+([^>]+)\/?>|<node\s+([^>]+)>/g;
+    let match;
+    while ((match = nodeRegex.exec(xml)) !== null) {
+        const attrStr = match[1] || match[2];
+        if (!attrStr)
+            continue;
+        // Extract attributes
+        const getAttr = (name) => {
+            const attrMatch = attrStr.match(new RegExp(`${name}="([^"]*)"`));
+            return attrMatch ? attrMatch[1] : "";
+        };
+        const boundsStr = getAttr("bounds");
+        const bounds = parseBoundsForUIElement(boundsStr);
+        if (!bounds)
+            continue;
+        const element = {
+            text: getAttr("text"),
+            contentDesc: getAttr("content-desc"),
+            resourceId: getAttr("resource-id"),
+            className: getAttr("class"),
+            bounds,
+            center: {
+                x: Math.round((bounds.left + bounds.right) / 2),
+                y: Math.round((bounds.top + bounds.bottom) / 2)
+            },
+            clickable: getAttr("clickable") === "true",
+            enabled: getAttr("enabled") === "true",
+            focused: getAttr("focused") === "true",
+            scrollable: getAttr("scrollable") === "true",
+            selected: getAttr("selected") === "true"
+        };
+        elements.push(element);
+    }
+    return elements;
+}
+/**
+ * Match element against find options
+ */
+function matchesElement(element, options) {
+    if (options.text !== undefined) {
+        if (element.text !== options.text)
+            return false;
+    }
+    if (options.textContains !== undefined) {
+        if (!element.text.toLowerCase().includes(options.textContains.toLowerCase()))
+            return false;
+    }
+    if (options.contentDesc !== undefined) {
+        if (element.contentDesc !== options.contentDesc)
+            return false;
+    }
+    if (options.contentDescContains !== undefined) {
+        if (!element.contentDesc.toLowerCase().includes(options.contentDescContains.toLowerCase()))
+            return false;
+    }
+    if (options.resourceId !== undefined) {
+        // Support both full "com.app:id/button" and short "button" forms
+        const shortId = element.resourceId.split("/").pop() || "";
+        if (element.resourceId !== options.resourceId && shortId !== options.resourceId)
+            return false;
+    }
+    return true;
+}
+/**
+ * Get UI accessibility tree from Android device using uiautomator
+ */
+export async function androidGetUITree(deviceId) {
+    try {
+        const adbAvailable = await isAdbAvailable();
+        if (!adbAvailable) {
+            return {
+                success: false,
+                error: "ADB is not installed or not in PATH. Install Android SDK Platform Tools."
+            };
+        }
+        const deviceArg = buildDeviceArg(deviceId);
+        const device = deviceId || (await getDefaultAndroidDevice());
+        if (!device) {
+            return {
+                success: false,
+                error: "No Android device connected. Connect a device or start an emulator."
+            };
+        }
+        // Dump UI hierarchy to device
+        const remotePath = "/sdcard/ui_dump.xml";
+        await execAsync(`adb ${deviceArg} shell uiautomator dump ${remotePath}`, {
+            timeout: ADB_TIMEOUT
+        });
+        // Read the XML content
+        const { stdout } = await execAsync(`adb ${deviceArg} shell cat ${remotePath}`, {
+            timeout: ADB_TIMEOUT
+        });
+        // Clean up remote file
+        await execAsync(`adb ${deviceArg} shell rm ${remotePath}`, {
+            timeout: ADB_TIMEOUT
+        }).catch(() => { });
+        const elements = parseUIAutomatorXML(stdout);
+        return {
+            success: true,
+            elements,
+            rawXml: stdout
+        };
+    }
+    catch (error) {
+        return {
+            success: false,
+            error: `Failed to get UI tree: ${error instanceof Error ? error.message : String(error)}`
+        };
+    }
+}
+/**
+ * Find element(s) in the UI tree matching the given criteria
+ */
+export async function androidFindElement(options, deviceId) {
+    try {
+        // Validate that at least one search criteria is provided
+        if (!options.text && !options.textContains && !options.contentDesc &&
+            !options.contentDescContains && !options.resourceId) {
+            return {
+                success: false,
+                found: false,
+                error: "At least one search criteria (text, textContains, contentDesc, contentDescContains, or resourceId) must be provided"
+            };
+        }
+        const treeResult = await androidGetUITree(deviceId);
+        if (!treeResult.success || !treeResult.elements) {
+            return {
+                success: false,
+                found: false,
+                error: treeResult.error
+            };
+        }
+        // Find matching elements
+        const matches = treeResult.elements.filter(el => matchesElement(el, options));
+        if (matches.length === 0) {
+            return {
+                success: true,
+                found: false,
+                matchCount: 0
+            };
+        }
+        // Select the element at the specified index (default 0)
+        const index = options.index ?? 0;
+        const selectedElement = matches[index];
+        if (!selectedElement) {
+            return {
+                success: true,
+                found: false,
+                matchCount: matches.length,
+                error: `Index ${index} out of bounds. Found ${matches.length} matching element(s).`
+            };
+        }
+        return {
+            success: true,
+            found: true,
+            element: selectedElement,
+            allMatches: matches,
+            matchCount: matches.length
+        };
+    }
+    catch (error) {
+        return {
+            success: false,
+            found: false,
+            error: `Failed to find element: ${error instanceof Error ? error.message : String(error)}`
+        };
+    }
+}
+/**
+ * Wait for element to appear on screen with polling
+ */
+export async function androidWaitForElement(options, deviceId) {
+    const timeoutMs = options.timeoutMs ?? 10000;
+    const pollIntervalMs = options.pollIntervalMs ?? 500;
+    const startTime = Date.now();
+    // Validate that at least one search criteria is provided
+    if (!options.text && !options.textContains && !options.contentDesc &&
+        !options.contentDescContains && !options.resourceId) {
+        return {
+            success: false,
+            found: false,
+            timedOut: false,
+            error: "At least one search criteria (text, textContains, contentDesc, contentDescContains, or resourceId) must be provided"
+        };
+    }
+    while (Date.now() - startTime < timeoutMs) {
+        const result = await androidFindElement(options, deviceId);
+        if (result.found && result.element) {
+            return {
+                ...result,
+                elapsedMs: Date.now() - startTime,
+                timedOut: false
+            };
+        }
+        // If there was an error (not just "not found"), return it
+        if (!result.success) {
+            return {
+                ...result,
+                elapsedMs: Date.now() - startTime,
+                timedOut: false
+            };
+        }
+        // Wait before next poll
+        await new Promise(resolve => setTimeout(resolve, pollIntervalMs));
+    }
+    return {
+        success: true,
+        found: false,
+        elapsedMs: Date.now() - startTime,
+        timedOut: true,
+        error: `Timed out after ${timeoutMs}ms waiting for element`
+    };
+}
 /**
  * Get device screen size
  */