mobile-debug-mcp 0.5.0 → 0.7.0

package/README.md

@@ -26,6 +26,7 @@ This server is designed with security in mind, using strict argument handling to
 - Node.js >= 18
 - Android SDK (`adb` in PATH) for Android support
 - Xcode command-line tools (`xcrun simctl`) for iOS support
+- **iOS Device Bridge (`idb`)** for iOS UI tree support
 - Booted iOS simulator for iOS testing
 
 ---
@@ -34,7 +35,18 @@ This server is designed with security in mind, using strict argument handling to
 
 You can install and use **Mobile Debug MCP** in one of two ways:
 
-### 1. Clone the repository for local development
+### 1. Install Dependencies
+
+**iOS Prerequisite (`idb`):**
+To use the `get_ui_tree` tool on iOS, you must install Facebook's `idb`:
+
+```bash
+brew tap facebook/fb
+brew install idb-companion
+pip3 install fb-idb
+```
+
+### 2. Clone the repository for local development
 
 ```bash
 git clone https://github.com/YOUR_USERNAME/mobile-debug-mcp.git
@@ -45,7 +57,7 @@ npm run build
 
 This option is suitable if you want to modify or contribute to the code.
 
-### 2. Install via npm for standard use
+### 3. Install via npm for standard use
 
 ```bash
 npm install -g mobile-debug-mcp
@@ -55,27 +67,7 @@ This option installs the package globally for easy use without cloning the repo.
 
 ---
 
-## Testing
-33. 
-34. The repository includes a smoke test script to verify end-to-end functionality on real devices or simulators.
-35. 
-36. ```bash
-37. # Run smoke test for Android
-38. npx tsx smoke-test.ts android com.example.package
-39. 
-40. # Run smoke test for iOS
-41. npx tsx smoke-test.ts ios com.example.bundleid
-42. ```
-43. 
-44. The smoke test performs the following sequence:
-45. 1. Starts the app
-46. 2. Captures a screenshot
-47. 3. Fetches logs
-48. 4. Terminates the app
-49. 
-50. ---
-51. 
-52. ## MCP Server Configuration
+## MCP Server Configuration
 
 Example WebUI MCP config using `npx --yes` and environment variables:
 
@@ -110,7 +102,7 @@ All tools accept a JSON input payload and return a structured JSON response. **E
 Launch a mobile app.
 
 **Input:**
-```json
+```jsonc
 {
   "platform": "android" | "ios",
   "appId": "com.example.app", // Android package or iOS bundle ID (Required)
@@ -131,7 +123,7 @@ Launch a mobile app.
 Fetch recent logs from the app or device.
 
 **Input:**
-```json
+```jsonc
 {
   "platform": "android" | "ios",
   "appId": "com.example.app", // Optional: filter logs by app
@@ -155,7 +147,7 @@ Returns two content blocks:
 Capture a screenshot of the current device screen.
 
 **Input:**
-```json
+```jsonc
 {
   "platform": "android" | "ios",
   "deviceId": "emulator-5554" // Optional: target specific device
@@ -177,7 +169,7 @@ Returns two content blocks:
 Terminate a running app.
 
 **Input:**
-```json
+```jsonc
 {
   "platform": "android" | "ios",
   "appId": "com.example.app", // Android package or iOS bundle ID (Required)
@@ -197,7 +189,7 @@ Terminate a running app.
 Restart an app (terminate then launch).
 
 **Input:**
-```json
+```jsonc
 {
   "platform": "android" | "ios",
   "appId": "com.example.app", // Android package or iOS bundle ID (Required)
@@ -218,7 +210,7 @@ Restart an app (terminate then launch).
 Clear app storage (reset to fresh install state).
 
 **Input:**
-```json
+```jsonc
 {
   "platform": "android" | "ios",
   "appId": "com.example.app", // Android package or iOS bundle ID (Required)
@@ -234,6 +226,206 @@ Clear app storage (reset to fresh install state).
 }
 ```
 
+### get_ui_tree
+Get the current UI hierarchy from the device. Returns a structured JSON representation of the screen content.
+
+**Input:**
+```jsonc
+{
+  "platform": "android" | "ios",
+  "deviceId": "emulator-5554" // Optional
+}
+```
+
+**Response:**
+```json
+{
+  "device": { /* device info */ },
+  "screen": "",
+  "resolution": { "width": 1080, "height": 1920 },
+  "elements": [
+    {
+      "text": "Login",
+      "contentDescription": null,
+      "type": "android.widget.Button",
+      "resourceId": "com.example:id/login_button",
+      "clickable": true,
+      "enabled": true,
+      "visible": true,
+      "bounds": [120,400,280,450],
+      "center": [200, 425],
+      "depth": 1,
+      "parentId": 0,
+      "children": []
+    }
+  ]
+}
+```
+
+### get_current_screen
+Get the currently visible activity on an Android device.
+
+**Input:**
+```jsonc
+{
+  "deviceId": "emulator-5554" // Optional: target specific device
+}
+```
+
+**Response:**
+```json
+{
+  "device": { /* device info */ },
+  "package": "com.example.app",
+  "activity": "com.example.app.LoginActivity",
+  "shortActivity": "LoginActivity"
+}
+```
+
+### wait_for_element
+Wait until a UI element with matching text appears on screen or timeout is reached. Useful for handling loading states or transitions.
+
+**Input:**
+```jsonc
+{
+  "platform": "android" | "ios",
+  "text": "Home", // Text to wait for
+  "timeout": 5000, // Max wait time in ms (default 10000)
+  "deviceId": "emulator-5554" // Optional
+}
+```
+
+**Response:**
+```json
+{
+  "device": { /* device info */ },
+  "found": true,
+  "element": { /* UIElement object if found */ }
+}
+```
+
+If the element is not found within the timeout, `found` will be `false`. If a system error occurs (e.g., ADB failure), an `error` field will be present.
+
+```json
+{
+  "device": { /* device info */ },
+  "found": false,
+  "error": "Optional error message"
+}
+```
+
+### tap
+Simulate a finger tap on the device screen at specific coordinates.
+
+Platform support and constraints:
+- Android: Implemented via `adb shell input tap` and works when `adb` is available in PATH or configured via `ADB_PATH`.
+- iOS: Requires Facebook's `idb` tooling. The iOS implementation uses `idb` to deliver UI events and is simulator-oriented (works reliably on a booted simulator). Physical device support depends on `idb` capabilities and a running `idb_companion` on the target device; it may not work in all environments.
+
+Prerequisites for iOS (if you intend to use tap on iOS):
+```bash
+brew tap facebook/fb
+brew install idb-companion
+pip3 install fb-idb
+```
+Ensure `idb` and `idb_companion` are in your PATH. If you use non-standard tool locations, set `XCRUN_PATH` and/or `ADB_PATH` environment variables as appropriate.
+
+Behavior notes:
+- The tool is a primitive input: it only sends a tap at the provided coordinates. It does not inspect or interpret the UI.
+- If `idb` is missing or the simulator/device is not available, the tool will return an error explaining the failure.
+
+**Input:**
+```jsonc
+{
+  "platform": "android" | "ios", // Optional, defaults to "android"
+  "x": 200, // X coordinate (Required)
+  "y": 400, // Y coordinate (Required)
+  "deviceId": "emulator-5554" // Optional
+}
+```
+
+**Response:**
+```json
+{
+  "device": { /* device info */ },
+  "success": true,
+  "x": 200,
+  "y": 400
+}
+```
+
+If the tap fails (e.g., missing `adb`/`idb`, device not found), `success` will be `false` and an `error` field will be present.
+
+### swipe
+Simulate a swipe gesture on an Android device.
+
+**Input:**
+```jsonc
+{
+  "platform": "android", // Optional, defaults to "android"
+  "x1": 500, // Start X (Required)
+  "y1": 1500, // Start Y (Required)
+  "x2": 500, // End X (Required)
+  "y2": 500, // End Y (Required)
+  "duration": 300, // Duration in ms (Required)
+  "deviceId": "emulator-5554" // Optional
+}
+```
+
+**Response:**
+```json
+{
+  "device": { /* device info */ },
+  "success": true,
+  "start": [500, 1500],
+  "end": [500, 500],
+  "duration": 300
+}
+```
+
+If the swipe fails, `success` will be `false` and an `error` field will be present.
+
+### type_text
+Type text into the currently focused input field on an Android device.
+
+**Input:**
+```jsonc
+{
+  "platform": "android", // Optional, defaults to "android"
+  "text": "hello world", // Text to type (Required)
+  "deviceId": "emulator-5554" // Optional
+}
+```
+
+**Response:**
+```json
+{
+  "device": { /* device info */ },
+  "success": true,
+  "text": "hello world"
+}
+```
+
+If the command fails, `success` will be `false` and an `error` field will be present.
+
+### press_back
+Simulate pressing the Android Back button.
+
+**Input:**
+```jsonc
+{
+  "platform": "android", // Optional
+  "deviceId": "emulator-5554" // Optional
+}
+```
+
+**Response:**
+```json
+{
+  "device": { /* device info */ },
+  "success": true
+}
+```
+
 ---
 
 ## Recommended Workflow
@@ -242,8 +434,9 @@ Clear app storage (reset to fresh install state).
 2. Use `start_app` to launch the app.
 3. Use `get_logs` to read the latest logs.
 4. Use `capture_screenshot` to visually inspect the app if needed.
-5. Use `reset_app_data` to clear state if debugging fresh install scenarios.
-6. Use `restart_app` to quickly reboot the app during development cycles.
+5. Use `wait_for_element` to ensure the app is in the expected state before proceeding (e.g., after login).
+6. Use `reset_app_data` to clear state if debugging fresh install scenarios.
+7. Use `restart_app` to quickly reboot the app during development cycles.
 
 ---
 
@@ -255,6 +448,26 @@ Clear app storage (reset to fresh install state).
 
 ---
 
+## Testing
+
+The repository includes a smoke test script to verify end-to-end functionality on real devices or simulators.
+
+```bash
+# Run smoke test for Android
+npx tsx smoke-test.ts android com.example.package
+
+# Run smoke test for iOS
+npx tsx smoke-test.ts ios com.example.bundleid
+```
+
+The smoke test performs the following sequence:
+1. Starts the app
+2. Captures a screenshot
+3. Fetches logs
+4. Terminates the app
+
+---
+
 ## License
 
 MIT License

package/dist/android/interact.js

@@ -0,0 +1,106 @@
+import { execAdb, getAndroidDeviceMetadata, getDeviceInfo } from "./utils.js";
+import { AndroidObserve } from "./observe.js";
+export class AndroidInteract {
+    observe = new AndroidObserve();
+    async waitForElement(text, timeout, deviceId) {
+        const metadata = await getAndroidDeviceMetadata("", deviceId);
+        const deviceInfo = getDeviceInfo(deviceId || 'default', metadata);
+        const startTime = Date.now();
+        while (Date.now() - startTime < timeout) {
+            try {
+                const tree = await this.observe.getUITree(deviceId);
+                if (tree.error) {
+                    return { device: deviceInfo, found: false, error: tree.error };
+                }
+                const element = tree.elements.find(e => e.text === text);
+                if (element) {
+                    return { device: deviceInfo, found: true, element };
+                }
+            }
+            catch (e) {
+                // Ignore errors during polling and retry
+                console.error("Error polling UI tree:", e);
+            }
+            const elapsed = Date.now() - startTime;
+            const remaining = timeout - elapsed;
+            if (remaining <= 0)
+                break;
+            await new Promise(resolve => setTimeout(resolve, Math.min(500, remaining)));
+        }
+        return { device: deviceInfo, found: false };
+    }
+    async tap(x, y, deviceId) {
+        const metadata = await getAndroidDeviceMetadata("", deviceId);
+        const deviceInfo = getDeviceInfo(deviceId || 'default', metadata);
+        try {
+            await execAdb(['shell', 'input', 'tap', x.toString(), y.toString()], deviceId);
+            return { device: deviceInfo, success: true, x, y };
+        }
+        catch (e) {
+            return { device: deviceInfo, success: false, x, y, error: e instanceof Error ? e.message : String(e) };
+        }
+    }
+    async swipe(x1, y1, x2, y2, duration, deviceId) {
+        const metadata = await getAndroidDeviceMetadata("", deviceId);
+        const deviceInfo = getDeviceInfo(deviceId || 'default', metadata);
+        try {
+            await execAdb(['shell', 'input', 'swipe', x1.toString(), y1.toString(), x2.toString(), y2.toString(), duration.toString()], deviceId);
+            return { device: deviceInfo, success: true, start: [x1, y1], end: [x2, y2], duration };
+        }
+        catch (e) {
+            return { device: deviceInfo, success: false, start: [x1, y1], end: [x2, y2], duration, error: e instanceof Error ? e.message : String(e) };
+        }
+    }
+    async typeText(text, deviceId) {
+        const metadata = await getAndroidDeviceMetadata("", deviceId);
+        const deviceInfo = getDeviceInfo(deviceId || 'default', metadata);
+        try {
+            // Encode spaces as %s to ensure proper input handling by adb shell input text
+            const encodedText = text.replace(/\s/g, '%s');
+            // Note: 'input text' might fail with some characters or if keyboard isn't ready, but it's the standard ADB way.
+            await execAdb(['shell', 'input', 'text', encodedText], deviceId);
+            return { device: deviceInfo, success: true, text };
+        }
+        catch (e) {
+            return { device: deviceInfo, success: false, text, error: e instanceof Error ? e.message : String(e) };
+        }
+    }
+    async pressBack(deviceId) {
+        const metadata = await getAndroidDeviceMetadata("", deviceId);
+        const deviceInfo = getDeviceInfo(deviceId || 'default', metadata);
+        try {
+            await execAdb(['shell', 'input', 'keyevent', '4'], deviceId);
+            return { device: deviceInfo, success: true };
+        }
+        catch (e) {
+            return { device: deviceInfo, success: false, error: e instanceof Error ? e.message : String(e) };
+        }
+    }
+    async startApp(appId, deviceId) {
+        const metadata = await getAndroidDeviceMetadata(appId, deviceId);
+        const deviceInfo = getDeviceInfo(deviceId || 'default', metadata);
+        await execAdb(['shell', 'monkey', '-p', appId, '-c', 'android.intent.category.LAUNCHER', '1'], deviceId);
+        return { device: deviceInfo, appStarted: true, launchTimeMs: 1000 };
+    }
+    async terminateApp(appId, deviceId) {
+        const metadata = await getAndroidDeviceMetadata(appId, deviceId);
+        const deviceInfo = getDeviceInfo(deviceId || 'default', metadata);
+        await execAdb(['shell', 'am', 'force-stop', appId], deviceId);
+        return { device: deviceInfo, appTerminated: true };
+    }
+    async restartApp(appId, deviceId) {
+        await this.terminateApp(appId, deviceId);
+        const startResult = await this.startApp(appId, deviceId);
+        return {
+            device: startResult.device,
+            appRestarted: startResult.appStarted,
+            launchTimeMs: startResult.launchTimeMs
+        };
+    }
+    async resetAppData(appId, deviceId) {
+        const metadata = await getAndroidDeviceMetadata(appId, deviceId);
+        const deviceInfo = getDeviceInfo(deviceId || 'default', metadata);
+        const output = await execAdb(['shell', 'pm', 'clear', appId], deviceId);
+        return { device: deviceInfo, dataCleared: output === 'Success' };
+    }
+}