mcp-mobile-interaction 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Pablo Ortiz
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,163 @@
+# mcp-mobile-interaction
+
+An MCP (Model Context Protocol) server that lets Claude interact with Android and iOS devices/emulators. Take screenshots, tap, swipe, type, inspect UI elements, and more — no Appium required.
+
+## Prerequisites
+
+### Android
+- [Android SDK](https://developer.android.com/studio) with `adb` in your PATH
+- An Android emulator running or a physical device connected via USB with ADB debugging enabled
+
+### iOS
+- macOS with [Xcode](https://developer.apple.com/xcode/) installed (provides `xcrun simctl`)
+- For physical devices: [idb](https://fbidb.io/) (`brew install idb-companion && pip install fb-idb`)
+- A booted iOS simulator or connected physical device
+
+## Installation
+
+### With Claude Code
+
+```bash
+claude mcp add mobile -- npx -y mcp-mobile-interaction
+```
+
+Or add `.mcp.json` to your project root (shared with your team):
+
+```json
+{
+  "mcpServers": {
+    "mobile": {
+      "command": "npx",
+      "args": ["-y", "mcp-mobile-interaction"]
+    }
+  }
+}
+```
+
+### With Claude Desktop
+
+Add to your `claude_desktop_config.json`:
+
+```json
+{
+  "mcpServers": {
+    "mobile": {
+      "command": "npx",
+      "args": ["-y", "mcp-mobile-interaction"]
+    }
+  }
+}
+```
+
+### Manual
+
+```bash
+npm install -g mcp-mobile-interaction
+```
+
+## Tools
+
+All tools accept a `platform` parameter (`"android"` or `"ios"`) and an optional `device_id` (defaults to the first connected device).
+
+### Core Tools
+
+| Tool | Description |
+|------|-------------|
+| `list_devices` | List connected devices and emulators/simulators |
+| `screenshot` | Capture a screenshot (returns base64 JPEG) |
+| `get_ui_tree` | Get a flat list of UI elements with bounds and center coordinates |
+| `get_screen_info` | Get screen dimensions, density, and orientation |
+| `get_screen_state` | Get UI tree + screenshot in a single call (saves a round-trip) |
+
+### Action Tools
+
+| Tool | Description |
+|------|-------------|
+| `tap` | Tap at (x, y) coordinates |
+| `double_tap` | Double-tap at (x, y) coordinates |
+| `long_press` | Long-press at (x, y) with configurable duration |
+| `swipe` | Swipe between coordinates or by direction (up/down/left/right) |
+| `type_text` | Type text into the focused input field |
+| `press_key` | Press a key (home, back, enter, delete, volume_up, volume_down, power, tab, recent_apps) |
+| `launch_app` | Launch an app by package name / bundle ID |
+| `open_url` | Open a URL or deep link |
+| `tap_element` | Find an element by text and tap it (combines get_ui_tree + tap) |
+
+### Waiting Tools
+
+| Tool | Description |
+|------|-------------|
+| `wait_for_element` | Poll until an element matching text/type criteria appears on screen |
+| `wait_for_stable` | Poll until the screen stops changing (two consecutive UI snapshots match) |
+
+## Observe Mode
+
+All 8 action tools (`tap`, `double_tap`, `long_press`, `swipe`, `type_text`, `press_key`, `launch_app`, `open_url`) support optional **observe** parameters that capture the screen state after the action completes — returning the result in a single round-trip instead of two:
+
+| Parameter | Description |
+|-----------|-------------|
+| `observe` | `"none"` (default), `"ui_tree"`, `"screenshot"`, or `"both"` |
+| `observe_delay_ms` | Milliseconds to wait before capturing (default: 500) |
+| `observe_stabilize` | If `true`, wait for UI to stop changing instead of a fixed delay |
+
+### Example: before vs after
+
+**Before** (2 calls):
+```
+tap(x=540, y=960) → get_ui_tree()
+```
+
+**After** (1 call):
+```
+tap(x=540, y=960, observe="ui_tree")
+```
+
+For a 5-step test flow, this cuts round-trips roughly in half.
+
+## Examples
+
+### Take a screenshot
+
+```
+"Take a screenshot of my Android emulator"
+```
+
+Claude will call `screenshot` with `platform: "android"` and display the image.
+
+### Navigate an app
+
+```
+"Open Settings on my iOS simulator, then scroll down and tap General"
+```
+
+Claude will use `launch_app`, `tap_element`, and `swipe` to navigate.
+
+### Run a test flow efficiently
+
+```
+"Tap 'Picking Flow', wait for the sessions to load, tap the first session, fill in the value, and submit"
+```
+
+Claude will use `tap_element` with `observe_stabilize: true` and `wait_for_element` to handle loading states server-side.
+
+### Inspect the UI
+
+```
+"What buttons are visible on the screen?"
+```
+
+Claude will use `get_ui_tree` to list all visible elements with their types, text, and coordinates.
+
+## How It Works
+
+- **Android**: Uses `adb` commands directly (screencap, input, uiautomator, am, wm)
+- **iOS Simulators**: Uses `xcrun simctl` (screenshot, io, launch, openurl)
+- **iOS Physical Devices**: Uses `idb` (Facebook's iOS Development Bridge)
+
+Screenshots are compressed with [sharp](https://sharp.pixelplumbing.com/) (resized + JPEG quality) to stay under Claude's 1MB image limit.
+
+UI elements include `type`, `text`, `bounds`, `center_x`/`center_y` (for tapping), `clickable`, `resource_id`, `enabled`, and `focused`.
+
+## License
+
+MIT

package/dist/index.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export {};

package/dist/index.js

@@ -0,0 +1,50 @@
+#!/usr/bin/env node
+import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
+import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import { registerListDevicesTool } from "./tools/list-devices.js";
+import { registerScreenshotTool } from "./tools/screenshot.js";
+import { registerTapTool } from "./tools/tap.js";
+import { registerDoubleTapTool } from "./tools/double-tap.js";
+import { registerLongPressTool } from "./tools/long-press.js";
+import { registerSwipeTool } from "./tools/swipe.js";
+import { registerTypeTextTool } from "./tools/type-text.js";
+import { registerPressKeyTool } from "./tools/press-key.js";
+import { registerGetUiTreeTool } from "./tools/get-ui-tree.js";
+import { registerGetScreenInfoTool } from "./tools/get-screen-info.js";
+import { registerLaunchAppTool } from "./tools/launch-app.js";
+import { registerOpenUrlTool } from "./tools/open-url.js";
+import { registerWaitForElementTool } from "./tools/wait-for-element.js";
+import { registerWaitForStableTool } from "./tools/wait-for-stable.js";
+import { registerTapElementTool } from "./tools/tap-element.js";
+import { registerGetScreenStateTool } from "./tools/get-screen-state.js";
+const server = new McpServer({
+    name: "mcp-mobile-interaction",
+    version: "1.0.0",
+});
+// Register all 16 tools
+registerListDevicesTool(server);
+registerScreenshotTool(server);
+registerTapTool(server);
+registerDoubleTapTool(server);
+registerLongPressTool(server);
+registerSwipeTool(server);
+registerTypeTextTool(server);
+registerPressKeyTool(server);
+registerGetUiTreeTool(server);
+registerGetScreenInfoTool(server);
+registerLaunchAppTool(server);
+registerOpenUrlTool(server);
+registerWaitForElementTool(server);
+registerWaitForStableTool(server);
+registerTapElementTool(server);
+registerGetScreenStateTool(server);
+async function main() {
+    const transport = new StdioServerTransport();
+    await server.connect(transport);
+    console.error("mcp-mobile-interaction server running on stdio");
+}
+main().catch((error) => {
+    console.error("Fatal error:", error);
+    process.exit(1);
+});
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAEA,OAAO,EAAE,SAAS,EAAE,MAAM,yCAAyC,CAAC;AACpE,OAAO,EAAE,oBAAoB,EAAE,MAAM,2CAA2C,CAAC;AAEjF,OAAO,EAAE,uBAAuB,EAAE,MAAM,yBAAyB,CAAC;AAClE,OAAO,EAAE,sBAAsB,EAAE,MAAM,uBAAuB,CAAC;AAC/D,OAAO,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AACjD,OAAO,EAAE,qBAAqB,EAAE,MAAM,uBAAuB,CAAC;AAC9D,OAAO,EAAE,qBAAqB,EAAE,MAAM,uBAAuB,CAAC;AAC9D,OAAO,EAAE,iBAAiB,EAAE,MAAM,kBAAkB,CAAC;AACrD,OAAO,EAAE,oBAAoB,EAAE,MAAM,sBAAsB,CAAC;AAC5D,OAAO,EAAE,oBAAoB,EAAE,MAAM,sBAAsB,CAAC;AAC5D,OAAO,EAAE,qBAAqB,EAAE,MAAM,wBAAwB,CAAC;AAC/D,OAAO,EAAE,yBAAyB,EAAE,MAAM,4BAA4B,CAAC;AACvE,OAAO,EAAE,qBAAqB,EAAE,MAAM,uBAAuB,CAAC;AAC9D,OAAO,EAAE,mBAAmB,EAAE,MAAM,qBAAqB,CAAC;AAC1D,OAAO,EAAE,0BAA0B,EAAE,MAAM,6BAA6B,CAAC;AACzE,OAAO,EAAE,yBAAyB,EAAE,MAAM,4BAA4B,CAAC;AACvE,OAAO,EAAE,sBAAsB,EAAE,MAAM,wBAAwB,CAAC;AAChE,OAAO,EAAE,0BAA0B,EAAE,MAAM,6BAA6B,CAAC;AAEzE,MAAM,MAAM,GAAG,IAAI,SAAS,CAAC;IAC3B,IAAI,EAAE,wBAAwB;IAC9B,OAAO,EAAE,OAAO;CACjB,CAAC,CAAC;AAEH,wBAAwB;AACxB,uBAAuB,CAAC,MAAM,CAAC,CAAC;AAChC,sBAAsB,CAAC,MAAM,CAAC,CAAC;AAC/B,eAAe,CAAC,MAAM,CAAC,CAAC;AACxB,qBAAqB,CAAC,MAAM,CAAC,CAAC;AAC9B,qBAAqB,CAAC,MAAM,CAAC,CAAC;AAC9B,iBAAiB,CAAC,MAAM,CAAC,CAAC;AAC1B,oBAAoB,CAAC,MAAM,CAAC,CAAC;AAC7B,oBAAoB,CAAC,MAAM,CAAC,CAAC;AAC7B,qBAAqB,CAAC,MAAM,CAAC,CAAC;AAC9B,yBAAyB,CAAC,MAAM,CAAC,CAAC;AAClC,qBAAqB,CAAC,MAAM,CAAC,CAAC;AAC9B,mBAAmB,CAAC,MAAM,CAAC,CAAC;AAC5B,0BAA0B,CAAC,MAAM,CAAC,CAAC;AACnC,yBAAyB,CAAC,MAAM,CAAC,CAAC;AAClC,sBAAsB,CAAC,MAAM,CAAC,CAAC;AAC/B,0BAA0B,CAAC,MAAM,CAAC,CAAC;AAEnC,KAAK,UAAU,IAAI;IACjB,MAAM,SAAS,GAAG,IAAI,oBAAoB,EAAE,CAAC;IAC7C,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;IAChC,OAAO,CAAC,KAAK,CAAC,gDAAgD,CAAC,CAAC;AAClE,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,KAAK,EAAE,EAAE;IACrB,OAAO,CAAC,KAAK,CAAC,cAAc,EAAE,KAAK,CAAC,CAAC;IACrC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}

package/dist/platforms/android.d.ts

@@ -0,0 +1,14 @@
+import type { Device, UiElement, ScreenInfo } from "../types.js";
+export declare function listDevices(): Promise<Device[]>;
+export declare function getFirstDeviceId(): Promise<string>;
+export declare function screenshot(deviceId?: string): Promise<Buffer>;
+export declare function tap(x: number, y: number, deviceId?: string): Promise<void>;
+export declare function doubleTap(x: number, y: number, deviceId?: string): Promise<void>;
+export declare function longPress(x: number, y: number, durationMs?: number, deviceId?: string): Promise<void>;
+export declare function swipe(startX: number, startY: number, endX: number, endY: number, durationMs?: number, deviceId?: string): Promise<void>;
+export declare function typeText(text: string, deviceId?: string): Promise<void>;
+export declare function pressKey(key: string, deviceId?: string): Promise<void>;
+export declare function getUiTree(deviceId?: string): Promise<UiElement[]>;
+export declare function getScreenInfo(deviceId?: string): Promise<ScreenInfo>;
+export declare function launchApp(packageName: string, deviceId?: string): Promise<void>;
+export declare function openUrl(url: string, deviceId?: string): Promise<void>;

package/dist/platforms/android.js

@@ -0,0 +1,230 @@
+import { exec, execBuffer } from "../utils/exec.js";
+function adb(deviceId, cmd) {
+    return `adb -s ${deviceId} ${cmd}`;
+}
+export async function listDevices() {
+    const output = await exec("adb devices -l");
+    const lines = output.trim().split("\n").slice(1); // skip header
+    const devices = [];
+    for (const line of lines) {
+        const trimmed = line.trim();
+        if (!trimmed || trimmed.startsWith("*"))
+            continue;
+        const parts = trimmed.split(/\s+/);
+        const id = parts[0];
+        const status = parts[1];
+        // Extract model name from "model:XXX" token
+        const modelToken = parts.find((p) => p.startsWith("model:"));
+        const name = modelToken ? modelToken.split(":")[1] : id;
+        devices.push({ id, name, platform: "android", status });
+    }
+    return devices;
+}
+export async function getFirstDeviceId() {
+    const devices = await listDevices();
+    const connected = devices.filter((d) => d.status === "device");
+    if (connected.length === 0) {
+        throw new Error("No connected Android devices found. Make sure an emulator is running or a device is connected via USB with ADB debugging enabled.");
+    }
+    return connected[0].id;
+}
+async function resolveDevice(deviceId) {
+    return deviceId ?? (await getFirstDeviceId());
+}
+export async function screenshot(deviceId) {
+    const id = await resolveDevice(deviceId);
+    return execBuffer(adb(id, "exec-out screencap -p"), { timeout: 30_000 });
+}
+export async function tap(x, y, deviceId) {
+    const id = await resolveDevice(deviceId);
+    await exec(adb(id, `shell input tap ${x} ${y}`));
+}
+export async function doubleTap(x, y, deviceId) {
+    const id = await resolveDevice(deviceId);
+    // Two rapid taps with minimal delay
+    await exec(adb(id, `shell "input tap ${x} ${y} && sleep 0.05 && input tap ${x} ${y}"`));
+}
+export async function longPress(x, y, durationMs = 1000, deviceId) {
+    const id = await resolveDevice(deviceId);
+    // Swipe from point to same point = long press
+    await exec(adb(id, `shell input swipe ${x} ${y} ${x} ${y} ${durationMs}`));
+}
+export async function swipe(startX, startY, endX, endY, durationMs = 300, deviceId) {
+    const id = await resolveDevice(deviceId);
+    await exec(adb(id, `shell input swipe ${startX} ${startY} ${endX} ${endY} ${durationMs}`));
+}
+export async function typeText(text, deviceId) {
+    const id = await resolveDevice(deviceId);
+    // Escape special characters for ADB shell
+    const escaped = text
+        .replace(/\\/g, "\\\\")
+        .replace(/"/g, '\\"')
+        .replace(/'/g, "\\'")
+        .replace(/ /g, "%s")
+        .replace(/&/g, "\\&")
+        .replace(/</g, "\\<")
+        .replace(/>/g, "\\>")
+        .replace(/\|/g, "\\|")
+        .replace(/;/g, "\\;")
+        .replace(/\(/g, "\\(")
+        .replace(/\)/g, "\\)")
+        .replace(/\$/g, "\\$")
+        .replace(/`/g, "\\`");
+    await exec(adb(id, `shell input text "${escaped}"`));
+}
+const KEYCODE_MAP = {
+    home: 3,
+    back: 4,
+    enter: 66,
+    delete: 67,
+    volume_up: 24,
+    volume_down: 25,
+    power: 26,
+    tab: 61,
+    recent_apps: 187,
+};
+export async function pressKey(key, deviceId) {
+    const id = await resolveDevice(deviceId);
+    const keycode = KEYCODE_MAP[key];
+    if (keycode === undefined) {
+        throw new Error(`Unknown key: ${key}. Supported keys: ${Object.keys(KEYCODE_MAP).join(", ")}`);
+    }
+    await exec(adb(id, `shell input keyevent ${keycode}`));
+}
+export async function getUiTree(deviceId) {
+    const id = await resolveDevice(deviceId);
+    // Strategy: tty → file → wait+tty → wait+file
+    const strategies = [
+        () => dumpUiViaTty(id),
+        () => dumpUiViaFile(id),
+        async () => { await delay(800); return dumpUiViaTty(id); },
+        async () => { await delay(800); return dumpUiViaFile(id); },
+    ];
+    for (const strategy of strategies) {
+        try {
+            const xml = await strategy();
+            if (xml) {
+                const elements = parseUiXml(xml);
+                if (elements.length > 0)
+                    return elements;
+            }
+        }
+        catch {
+            // Try next strategy
+        }
+    }
+    throw new Error("Failed to parse UI tree XML from uiautomator dump after 4 attempts. The screen may be in transition — try again after a short delay.");
+}
+function delay(ms) {
+    return new Promise((resolve) => setTimeout(resolve, ms));
+}
+async function dumpUiViaTty(deviceId) {
+    const output = await exec(adb(deviceId, "exec-out uiautomator dump /dev/tty"), {
+        timeout: 30_000,
+    });
+    // Strip null bytes and other binary artifacts
+    const cleaned = output.replace(/\0/g, "").trim();
+    return extractXml(cleaned);
+}
+async function dumpUiViaFile(deviceId) {
+    const remotePath = "/sdcard/window_dump.xml";
+    await exec(adb(deviceId, `shell uiautomator dump ${remotePath}`), {
+        timeout: 30_000,
+    });
+    const output = await exec(adb(deviceId, `shell cat ${remotePath}`), {
+        timeout: 10_000,
+    });
+    // Clean up remote file
+    exec(adb(deviceId, `shell rm -f ${remotePath}`)).catch(() => { });
+    const cleaned = output.replace(/\0/g, "").trim();
+    return extractXml(cleaned);
+}
+function extractXml(output) {
+    const xmlMatch = output.match(/<\?xml[\s\S]*<\/hierarchy>/);
+    if (xmlMatch)
+        return xmlMatch[0];
+    const altMatch = output.match(/<hierarchy[\s\S]*<\/hierarchy>/);
+    if (altMatch)
+        return altMatch[0];
+    return null;
+}
+function parseUiXml(xml) {
+    const elements = [];
+    // Match both self-closing <node ... /> and opening <node ...> tags
+    const nodeRegex = /<node\s+([^>]+?)\/?>/g;
+    let match;
+    let index = 0;
+    while ((match = nodeRegex.exec(xml)) !== null) {
+        const attrs = match[1];
+        const type = extractAttr(attrs, "class")?.split(".").pop() ?? "Unknown";
+        const text = extractAttr(attrs, "text") ?? "";
+        const contentDesc = extractAttr(attrs, "content-desc") ?? "";
+        const clickable = extractAttr(attrs, "clickable") === "true";
+        const boundsStr = extractAttr(attrs, "bounds") ?? "";
+        const rawResourceId = extractAttr(attrs, "resource-id") ?? "";
+        const enabled = extractAttr(attrs, "enabled") === "true";
+        const focused = extractAttr(attrs, "focused") === "true";
+        // Parse bounds "[x1,y1][x2,y2]"
+        const boundsMatch = boundsStr.match(/\[(\d+),(\d+)\]\[(\d+),(\d+)\]/);
+        if (!boundsMatch)
+            continue;
+        const x1 = parseInt(boundsMatch[1], 10);
+        const y1 = parseInt(boundsMatch[2], 10);
+        const x2 = parseInt(boundsMatch[3], 10);
+        const y2 = parseInt(boundsMatch[4], 10);
+        const displayText = text || contentDesc;
+        // Strip package prefix from resource-id (e.g. "com.example:id/btn" → "btn")
+        const resourceId = rawResourceId
+            ? rawResourceId.replace(/^[^:]+:id\//, "")
+            : undefined;
+        elements.push({
+            index,
+            type,
+            text: displayText,
+            bounds: {
+                x: x1,
+                y: y1,
+                width: x2 - x1,
+                height: y2 - y1,
+            },
+            center_x: Math.round((x1 + x2) / 2),
+            center_y: Math.round((y1 + y2) / 2),
+            clickable,
+            resource_id: resourceId,
+            enabled,
+            focused,
+        });
+        index++;
+    }
+    return elements;
+}
+function extractAttr(attrs, name) {
+    const regex = new RegExp(`${name}="([^"]*)"`);
+    const match = attrs.match(regex);
+    return match ? match[1] : undefined;
+}
+export async function getScreenInfo(deviceId) {
+    const id = await resolveDevice(deviceId);
+    const [sizeOutput, densityOutput] = await Promise.all([
+        exec(adb(id, "shell wm size")),
+        exec(adb(id, "shell wm density")),
+    ]);
+    // Parse "Physical size: 1080x1920"
+    const sizeMatch = sizeOutput.match(/(\d+)x(\d+)/);
+    const width = sizeMatch ? parseInt(sizeMatch[1], 10) : 0;
+    const height = sizeMatch ? parseInt(sizeMatch[2], 10) : 0;
+    // Parse "Physical density: 420"
+    const densityMatch = densityOutput.match(/(\d+)/);
+    const density = densityMatch ? parseInt(densityMatch[1], 10) : 0;
+    const orientation = width > height ? "landscape" : "portrait";
+    return { width, height, density, orientation };
+}
+export async function launchApp(packageName, deviceId) {
+    const id = await resolveDevice(deviceId);
+    await exec(adb(id, `shell monkey -p ${packageName} -c android.intent.category.LAUNCHER 1`));
+}
+export async function openUrl(url, deviceId) {
+    const id = await resolveDevice(deviceId);
+    await exec(adb(id, `shell am start -a android.intent.action.VIEW -d "${url}"`));
+}
+//# sourceMappingURL=android.js.map

package/dist/platforms/android.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"android.js","sourceRoot":"","sources":["../../src/platforms/android.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAGpD,SAAS,GAAG,CAAC,QAAgB,EAAE,GAAW;IACxC,OAAO,UAAU,QAAQ,IAAI,GAAG,EAAE,CAAC;AACrC,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,WAAW;IAC/B,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,gBAAgB,CAAC,CAAC;IAC5C,MAAM,KAAK,GAAG,MAAM,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,cAAc;IAEhE,MAAM,OAAO,GAAa,EAAE,CAAC;IAC7B,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;QACzB,MAAM,OAAO,GAAG,IAAI,CAAC,IAAI,EAAE,CAAC;QAC5B,IAAI,CAAC,OAAO,IAAI,OAAO,CAAC,UAAU,CAAC,GAAG,CAAC;YAAE,SAAS;QAElD,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;QACnC,MAAM,EAAE,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;QACpB,MAAM,MAAM,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;QAExB,4CAA4C;QAC5C,MAAM,UAAU,GAAG,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAC,CAAC;QAC7D,MAAM,IAAI,GAAG,UAAU,CAAC,CAAC,CAAC,UAAU,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;QAExD,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,SAAS,EAAE,MAAM,EAAE,CAAC,CAAC;IAC1D,CAAC;IAED,OAAO,OAAO,CAAC;AACjB,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,gBAAgB;IACpC,MAAM,OAAO,GAAG,MAAM,WAAW,EAAE,CAAC;IACpC,MAAM,SAAS,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,KAAK,QAAQ,CAAC,CAAC;IAC/D,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC3B,MAAM,IAAI,KAAK,CACb,mIAAmI,CACpI,CAAC;IACJ,CAAC;IACD,OAAO,SAAS,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;AACzB,CAAC;AAED,KAAK,UAAU,aAAa,CAAC,QAAiB;IAC5C,OAAO,QAAQ,IAAI,CAAC,MAAM,gBAAgB,EAAE,CAAC,CAAC;AAChD,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,UAAU,CAAC,QAAiB;IAChD,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IACzC,OAAO,UAAU,CAAC,GAAG,CAAC,EAAE,EAAE,uBAAuB,CAAC,EAAE,EAAE,OAAO,EAAE,MAAM,EAAE,CAAC,CAAC;AAC3E,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,GAAG,CACvB,CAAS,EACT,CAAS,EACT,QAAiB;IAEjB,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IACzC,MAAM,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,mBAAmB,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;AACnD,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,SAAS,CAC7B,CAAS,EACT,CAAS,EACT,QAAiB;IAEjB,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IACzC,oCAAoC;IACpC,MAAM,IAAI,CACR,GAAG,CAAC,EAAE,EAAE,oBAAoB,CAAC,IAAI,CAAC,+BAA+B,CAAC,IAAI,CAAC,GAAG,CAAC,CAC5E,CAAC;AACJ,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,SAAS,CAC7B,CAAS,EACT,CAAS,EACT,aAAqB,IAAI,EACzB,QAAiB;IAEjB,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IACzC,8CAA8C;IAC9C,MAAM,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,qBAAqB,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,UAAU,EAAE,CAAC,CAAC,CAAC;AAC7E,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,KAAK,CACzB,MAAc,EACd,MAAc,EACd,IAAY,EACZ,IAAY,EACZ,aAAqB,GAAG,EACxB,QAAiB;IAEjB,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IACzC,MAAM,IAAI,CACR,GAAG,CAAC,EAAE,EAAE,qBAAqB,MAAM,IAAI,MAAM,IAAI,IAAI,IAAI,IAAI,IAAI,UAAU,EAAE,CAAC,CAC/E,CAAC;AACJ,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,QAAQ,CAC5B,IAAY,EACZ,QAAiB;IAEjB,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IACzC,0CAA0C;IAC1C,MAAM,OAAO,GAAG,IAAI;SACjB,OAAO,CAAC,KAAK,EAAE,MAAM,CAAC;SACtB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC;SACpB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC;SACpB,OAAO,CAAC,IAAI,EAAE,IAAI,CAAC;SACnB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC;SACpB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC;SACpB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC;SACpB,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC;SACrB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC;SACpB,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC;SACrB,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC;SACrB,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC;SACrB,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;IACxB,MAAM,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,qBAAqB,OAAO,GAAG,CAAC,CAAC,CAAC;AACvD,CAAC;AAED,MAAM,WAAW,GAA2B;IAC1C,IAAI,EAAE,CAAC;IACP,IAAI,EAAE,CAAC;IACP,KAAK,EAAE,EAAE;IACT,MAAM,EAAE,EAAE;IACV,SAAS,EAAE,EAAE;IACb,WAAW,EAAE,EAAE;IACf,KAAK,EAAE,EAAE;IACT,GAAG,EAAE,EAAE;IACP,WAAW,EAAE,GAAG;CACjB,CAAC;AAEF,MAAM,CAAC,KAAK,UAAU,QAAQ,CAC5B,GAAW,EACX,QAAiB;IAEjB,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IACzC,MAAM,OAAO,GAAG,WAAW,CAAC,GAAG,CAAC,CAAC;IACjC,IAAI,OAAO,KAAK,SAAS,EAAE,CAAC;QAC1B,MAAM,IAAI,KAAK,CACb,gBAAgB,GAAG,qBAAqB,MAAM,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAC9E,CAAC;IACJ,CAAC;IACD,MAAM,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,wBAAwB,OAAO,EAAE,CAAC,CAAC,CAAC;AACzD,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,SAAS,CAAC,QAAiB;IAC/C,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IAEzC,8CAA8C;IAC9C,MAAM,UAAU,GAAwC;QACtD,GAAG,EAAE,CAAC,YAAY,CAAC,EAAE,CAAC;QACtB,GAAG,EAAE,CAAC,aAAa,CAAC,EAAE,CAAC;QACvB,KAAK,IAAI,EAAE,GAAG,MAAM,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,OAAO,YAAY,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;QAC1D,KAAK,IAAI,EAAE,GAAG,MAAM,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,OAAO,aAAa,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;KAC5D,CAAC;IAEF,KAAK,MAAM,QAAQ,IAAI,UAAU,EAAE,CAAC;QAClC,IAAI,CAAC;YACH,MAAM,GAAG,GAAG,MAAM,QAAQ,EAAE,CAAC;YAC7B,IAAI,GAAG,EAAE,CAAC;gBACR,MAAM,QAAQ,GAAG,UAAU,CAAC,GAAG,CAAC,CAAC;gBACjC,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC;oBAAE,OAAO,QAAQ,CAAC;YAC3C,CAAC;QACH,CAAC;QAAC,MAAM,CAAC;YACP,oBAAoB;QACtB,CAAC;IACH,CAAC;IAED,MAAM,IAAI,KAAK,CACb,sIAAsI,CACvI,CAAC;AACJ,CAAC;AAED,SAAS,KAAK,CAAC,EAAU;IACvB,OAAO,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,UAAU,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,CAAC;AAC3D,CAAC;AAED,KAAK,UAAU,YAAY,CAAC,QAAgB;IAC1C,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,EAAE,oCAAoC,CAAC,EAAE;QAC7E,OAAO,EAAE,MAAM;KAChB,CAAC,CAAC;IAEH,8CAA8C;IAC9C,MAAM,OAAO,GAAG,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;IACjD,OAAO,UAAU,CAAC,OAAO,CAAC,CAAC;AAC7B,CAAC;AAED,KAAK,UAAU,aAAa,CAAC,QAAgB;IAC3C,MAAM,UAAU,GAAG,yBAAyB,CAAC;IAC7C,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,EAAE,0BAA0B,UAAU,EAAE,CAAC,EAAE;QAChE,OAAO,EAAE,MAAM;KAChB,CAAC,CAAC;IACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,EAAE,aAAa,UAAU,EAAE,CAAC,EAAE;QAClE,OAAO,EAAE,MAAM;KAChB,CAAC,CAAC;IACH,uBAAuB;IACvB,IAAI,CAAC,GAAG,CAAC,QAAQ,EAAE,eAAe,UAAU,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;IAEjE,MAAM,OAAO,GAAG,MAAM,CAAC,OAAO,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;IACjD,OAAO,UAAU,CAAC,OAAO,CAAC,CAAC;AAC7B,CAAC;AAED,SAAS,UAAU,CAAC,MAAc;IAChC,MAAM,QAAQ,GAAG,MAAM,CAAC,KAAK,CAAC,4BAA4B,CAAC,CAAC;IAC5D,IAAI,QAAQ;QAAE,OAAO,QAAQ,CAAC,CAAC,CAAC,CAAC;IAEjC,MAAM,QAAQ,GAAG,MAAM,CAAC,KAAK,CAAC,gCAAgC,CAAC,CAAC;IAChE,IAAI,QAAQ;QAAE,OAAO,QAAQ,CAAC,CAAC,CAAC,CAAC;IAEjC,OAAO,IAAI,CAAC;AACd,CAAC;AAED,SAAS,UAAU,CAAC,GAAW;IAC7B,MAAM,QAAQ,GAAgB,EAAE,CAAC;IACjC,mEAAmE;IACnE,MAAM,SAAS,GAAG,uBAAuB,CAAC;IAC1C,IAAI,KAA6B,CAAC;IAClC,IAAI,KAAK,GAAG,CAAC,CAAC;IAEd,OAAO,CAAC,KAAK,GAAG,SAAS,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;QAC9C,MAAM,KAAK,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC;QAEvB,MAAM,IAAI,GAAG,WAAW,CAAC,KAAK,EAAE,OAAO,CAAC,EAAE,KAAK,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,IAAI,SAAS,CAAC;QACxE,MAAM,IAAI,GAAG,WAAW,CAAC,KAAK,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC;QAC9C,MAAM,WAAW,GAAG,WAAW,CAAC,KAAK,EAAE,cAAc,CAAC,IAAI,EAAE,CAAC;QAC7D,MAAM,SAAS,GAAG,WAAW,CAAC,KAAK,EAAE,WAAW,CAAC,KAAK,MAAM,CAAC;QAC7D,MAAM,SAAS,GAAG,WAAW,CAAC,KAAK,EAAE,QAAQ,CAAC,IAAI,EAAE,CAAC;QACrD,MAAM,aAAa,GAAG,WAAW,CAAC,KAAK,EAAE,aAAa,CAAC,IAAI,EAAE,CAAC;QAC9D,MAAM,OAAO,GAAG,WAAW,CAAC,KAAK,EAAE,SAAS,CAAC,KAAK,MAAM,CAAC;QACzD,MAAM,OAAO,GAAG,WAAW,CAAC,KAAK,EAAE,SAAS,CAAC,KAAK,MAAM,CAAC;QAEzD,gCAAgC;QAChC,MAAM,WAAW,GAAG,SAAS,CAAC,KAAK,CAAC,gCAAgC,CAAC,CAAC;QACtE,IAAI,CAAC,WAAW;YAAE,SAAS;QAE3B,MAAM,EAAE,GAAG,QAAQ,CAAC,WAAW,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;QACxC,MAAM,EAAE,GAAG,QAAQ,CAAC,WAAW,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;QACxC,MAAM,EAAE,GAAG,QAAQ,CAAC,WAAW,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;QACxC,MAAM,EAAE,GAAG,QAAQ,CAAC,WAAW,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;QAExC,MAAM,WAAW,GAAG,IAAI,IAAI,WAAW,CAAC;QAExC,4EAA4E;QAC5E,MAAM,UAAU,GAAG,aAAa;YAC9B,CAAC,CAAC,aAAa,CAAC,OAAO,CAAC,aAAa,EAAE,EAAE,CAAC;YAC1C,CAAC,CAAC,SAAS,CAAC;QAEd,QAAQ,CAAC,IAAI,CAAC;YACZ,KAAK;YACL,IAAI;YACJ,IAAI,EAAE,WAAW;YACjB,MAAM,EAAE;gBACN,CAAC,EAAE,EAAE;gBACL,CAAC,EAAE,EAAE;gBACL,KAAK,EAAE,EAAE,GAAG,EAAE;gBACd,MAAM,EAAE,EAAE,GAAG,EAAE;aAChB;YACD,QAAQ,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC;YACnC,QAAQ,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC;YACnC,SAAS;YACT,WAAW,EAAE,UAAU;YACvB,OAAO;YACP,OAAO;SACR,CAAC,CAAC;QACH,KAAK,EAAE,CAAC;IACV,CAAC;IAED,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED,SAAS,WAAW,CAAC,KAAa,EAAE,IAAY;IAC9C,MAAM,KAAK,GAAG,IAAI,MAAM,CAAC,GAAG,IAAI,YAAY,CAAC,CAAC;IAC9C,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;IACjC,OAAO,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;AACtC,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,aAAa,CACjC,QAAiB;IAEjB,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IAEzC,MAAM,CAAC,UAAU,EAAE,aAAa,CAAC,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC;QACpD,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,eAAe,CAAC,CAAC;QAC9B,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,kBAAkB,CAAC,CAAC;KAClC,CAAC,CAAC;IAEH,mCAAmC;IACnC,MAAM,SAAS,GAAG,UAAU,CAAC,KAAK,CAAC,aAAa,CAAC,CAAC;IAClD,MAAM,KAAK,GAAG,SAAS,CAAC,CAAC,CAAC,QAAQ,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IACzD,MAAM,MAAM,GAAG,SAAS,CAAC,CAAC,CAAC,QAAQ,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAE1D,gCAAgC;IAChC,MAAM,YAAY,GAAG,aAAa,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;IAClD,MAAM,OAAO,GAAG,YAAY,CAAC,CAAC,CAAC,QAAQ,CAAC,YAAY,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAEjE,MAAM,WAAW,GAAG,KAAK,GAAG,MAAM,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,UAAU,CAAC;IAE9D,OAAO,EAAE,KAAK,EAAE,MAAM,EAAE,OAAO,EAAE,WAAW,EAAE,CAAC;AACjD,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,SAAS,CAC7B,WAAmB,EACnB,QAAiB;IAEjB,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IACzC,MAAM,IAAI,CACR,GAAG,CAAC,EAAE,EAAE,mBAAmB,WAAW,wCAAwC,CAAC,CAChF,CAAC;AACJ,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,OAAO,CAC3B,GAAW,EACX,QAAiB;IAEjB,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;IACzC,MAAM,IAAI,CACR,GAAG,CAAC,EAAE,EAAE,oDAAoD,GAAG,GAAG,CAAC,CACpE,CAAC;AACJ,CAAC"}

package/dist/platforms/ios.d.ts

@@ -0,0 +1,14 @@
+import type { Device, UiElement, ScreenInfo } from "../types.js";
+export declare function listDevices(): Promise<Device[]>;
+export declare function getFirstDeviceId(): Promise<string>;
+export declare function screenshot(deviceId?: string): Promise<Buffer>;
+export declare function tap(x: number, y: number, deviceId?: string): Promise<void>;
+export declare function doubleTap(x: number, y: number, deviceId?: string): Promise<void>;
+export declare function longPress(x: number, y: number, durationMs?: number, deviceId?: string): Promise<void>;
+export declare function swipe(startX: number, startY: number, endX: number, endY: number, durationMs?: number, deviceId?: string): Promise<void>;
+export declare function typeText(text: string, deviceId?: string): Promise<void>;
+export declare function pressKey(key: string, deviceId?: string): Promise<void>;
+export declare function getUiTree(deviceId?: string): Promise<UiElement[]>;
+export declare function getScreenInfo(deviceId?: string): Promise<ScreenInfo>;
+export declare function launchApp(bundleId: string, deviceId?: string): Promise<void>;
+export declare function openUrl(url: string, deviceId?: string): Promise<void>;