@mobilenext/mobile-mcp 0.0.23 → 0.0.25

package/README.md

@@ -1,38 +1,40 @@
-# Mobile Next - MCP server for Mobile Development and Automation  | iOS, Android, Simulator, Emulator, and physical devices
+# Mobile Next - MCP server for Mobile Development and Automation | iOS, Android, Simulator, Emulator, and Real Devices
 
-This is a [Model Context Protocol (MCP) server](https://github.com/modelcontextprotocol) that enables scalable mobile automation, development through a platform-agnostic interface, eliminating the need for distinct iOS or Android knowledge. You can run it on emulators, simulators, and physical devices (iOS and Android).
+This is a [Model Context Protocol (MCP) server](https://github.com/modelcontextprotocol) that enables scalable mobile automation, development through a platform-agnostic interface, eliminating the need for distinct iOS or Android knowledge. You can run it on emulators, simulators, and real devices (iOS and Android).
 This server allows Agents and LLMs to interact with native iOS/Android applications and devices through structured accessibility snapshots or coordinate-based taps based on screenshots.
 
 <h4 align="center">
-<a href="https://github.com/mobile-next/mobile-mcp">
+  <a href="https://github.com/mobile-next/mobile-mcp">
     <img src="https://img.shields.io/github/stars/mobile-next/mobile-mcp" alt="Mobile Next Stars" />
   </a>
- <a href="https://github.com/mobile-next/mobile-mcp">
+  <a href="https://github.com/mobile-next/mobile-mcp">
     <img src="https://img.shields.io/github/contributors/mobile-next/mobile-mcp?color=green" alt="Mobile Next Downloads" />
   </a>
   <a href="https://www.npmjs.com/package/@mobilenext/mobile-mcp">
-    <img src="https://img.shields.io/npm/dm/@mobilenext/mobile-mcp?logo=npm&style=flat&color=red" alt="npm">
+    <img src="https://img.shields.io/npm/dm/@mobilenext/mobile-mcp?logo=npm&style=flat&color=red" alt="npm" />
   </a>
-<a href="https://github.com/mobile-next/mobile-mcp/releases">
-    <img src="https://img.shields.io/github/release/mobile-next/mobile-mcp">
+  <a href="https://github.com/mobile-next/mobile-mcp/releases">
+    <img src="https://img.shields.io/github/release/mobile-next/mobile-mcp" />
   </a>
-<a href="https://github.com/mobile-next/mobile-mcp/blob/main/LICENSE">
-    <img src="https://img.shields.io/badge/license-Apache 2.0-blue.svg" alt="Mobile MCP is released under the Apache-2.0 License">
+  <a href="https://github.com/mobile-next/mobile-mcp/blob/main/LICENSE">
+    <img src="https://img.shields.io/badge/license-Apache 2.0-blue.svg" alt="Mobile MCP is released under the Apache-2.0 License" />
   </a>
-
-</p>
+  <a href="https://insiders.vscode.dev/redirect?url=vscode%3Amcp%2Finstall%3F%7B%22name%22%3A%22mobile-mcp%22%2C%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40mobilenext%2Fmobile-mcp%40latest%22%5D%7D">
+    <img src="https://img.shields.io/badge/VS_Code-VS_Code?style=flat-square&label=Install%20Server&color=0098FF" alt="Install in VS Code" />
+  </a>
+</h4>
 
 <h4 align="center">
-<a href="http://mobilenexthq.com/join-slack">
+  <a href="http://mobilenexthq.com/join-slack">
     <img src="https://img.shields.io/badge/join-Slack-blueviolet?logo=slack&style=flat" alt="Slack community channel" />
-</a>
-</p>
+  </a>
+</h4>
 
 https://github.com/user-attachments/assets/c4e89c4f-cc71-4424-8184-bdbc8c638fa1
 
 <p align="center">
     <a href="https://github.com/mobile-next/">
-        <img alt="mobile-mcp" src="https://raw.githubusercontent.com/mobile-next/mobile-next-assets/refs/heads/main/mobile-mcp-banner.png" width="600">
+        <img alt="mobile-mcp" src="https://raw.githubusercontent.com/mobile-next/mobile-next-assets/refs/heads/main/mobile-mcp-banner.png" width="600" />
     </a>
 </p>
 
@@ -49,7 +51,7 @@ Check out our detailed roadmap to see upcoming features, improvements, and miles
 How we help to scale mobile automation:
 
 - 📲 Native app automation (iOS and Android) for testing or data-entry scenarios.
-- 📝 Scripted flows and form interactions without manually controlling simulators/emulators or physical devices (iPhone, Samsung, Google Pixel etc)
+- 📝 Scripted flows and form interactions without manually controlling simulators/emulators or real devices (iPhone, Samsung, Google Pixel etc)
 - 🧭 Automating multi-step user journeys driven by an LLM
 - 👆 General-purpose mobile application interaction for agent-based frameworks
 - 🤖 Enables agent-to-agent communication for mobile automation usecases, data extraction
@@ -178,18 +180,18 @@ What you will need to connect MCP with your agent and mobile devices:
 - [node.js](https://nodejs.org/en/download/) v22+
 - [MCP](https://modelcontextprotocol.io/introduction) supported foundational models or agents, like [Claude MCP](https://modelcontextprotocol.io/quickstart/server), [OpenAI Agent SDK](https://openai.github.io/openai-agents-python/mcp/), [Copilot Studio](https://www.microsoft.com/en-us/microsoft-copilot/blog/copilot-studio/introducing-model-context-protocol-mcp-in-copilot-studio-simplified-integration-with-ai-apps-and-agents/)
 
-### Simulators, Emulators, and Physical Devices
+### Simulators, Emulators, and Real Devices
 
 When launched, Mobile MCP can connect to:
 - iOS Simulators on macOS/Linux
 - Android Emulators on Linux/Windows/macOS
-- Physical iOS or Android devices (requires proper platform tools and drivers)
+- iOS or Android real devices (requires proper platform tools and drivers)
 
 Make sure you have your mobile platform SDKs (Xcode, Android SDK) installed and configured properly before running Mobile Next Mobile MCP.
 
 ### Running in "headless" mode on Simulators/Emulators
 
-When you do not have a physical phone connected to your machine, you can run Mobile MCP with an emulator or simulator in the background.
+When you do not have a real device connected to your machine, you can run Mobile MCP with an emulator or simulator in the background.
 
 For example, on Android:
 1. Start an emulator (avdmanager / emulator command).

package/lib/android.js

@@ -39,14 +39,18 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.AndroidDeviceManager = exports.AndroidRobot = void 0;
 const node_path_1 = __importDefault(require("node:path"));
 const node_child_process_1 = require("node:child_process");
+const node_fs_1 = require("node:fs");
 const xml = __importStar(require("fast-xml-parser"));
 const robot_1 = require("./robot");
 const getAdbPath = () => {
-    let executable = "adb";
     if (process.env.ANDROID_HOME) {
-        executable = node_path_1.default.join(process.env.ANDROID_HOME, "platform-tools", "adb");
+        return node_path_1.default.join(process.env.ANDROID_HOME, "platform-tools", "adb");
     }
-    return executable;
+    const defaultAndroidSdk = node_path_1.default.join(process.env.HOME || "", "Library", "Android", "sdk", "platform-tools", "adb");
+    if ((0, node_fs_1.existsSync)(defaultAndroidSdk)) {
+        return defaultAndroidSdk;
+    }
+    return "adb";
 };
 const BUTTON_MAP = {
     "BACK": "KEYCODE_BACK",
@@ -73,11 +77,6 @@ class AndroidRobot {
             timeout: TIMEOUT,
         });
     }
-    getFirstDisplayId() {
-        const output = this.adb("shell", "dumpsys", "SurfaceFlinger", "--display-id").toString();
-        const match = output.match(/Display (\d+) \(/);
-        return match ? match[1] : null;
-    }
     getSystemFeatures() {
         return this.adb("shell", "pm", "list", "features")
             .toString()
@@ -195,16 +194,42 @@ class AndroidRobot {
         }
         this.adb("shell", "input", "swipe", `${x0}`, `${y0}`, `${x1}`, `${y1}`, "1000");
     }
-    async getScreenshot() {
-        const displayId = this.getFirstDisplayId();
-        if (displayId !== null) {
-            // always good to provide displayId. required for multi-display devices such as fold
-            return this.adb("exec-out", "screencap", "-p", "-d", displayId);
+    getDisplayCount() {
+        return this.adb("shell", "dumpsys", "SurfaceFlinger", "--display-id")
+            .toString()
+            .split("\n")
+            .filter(s => s.startsWith("Display "))
+            .length;
+    }
+    getFirstDisplayId() {
+        const displays = this.adb("shell", "cmd", "display", "get-displays")
+            .toString()
+            .split("\n")
+            .filter(s => s.startsWith("Display id "))
+            // filter for state ON even though get-displays only returns turned on displays
+            .filter(s => s.indexOf(", state ON,") >= 0)
+            // another paranoia check
+            .filter(s => s.indexOf(", uniqueId ") >= 0);
+        if (displays.length > 0) {
+            const m = displays[0].match(/uniqueId \"([^\"]+)\"/);
+            if (m !== null) {
+                const displayId = m[1];
+                if (displayId.indexOf("local:") === 0) {
+                    return displayId.split(":")[1];
+                }
+                return displayId;
+            }
         }
-        else {
-            // backward compatibility for android 10 and below
+        return null;
+    }
+    async getScreenshot() {
+        if (this.getDisplayCount() <= 1) {
+            // backward compatibility for android 10 and below, and for single display devices
             return this.adb("exec-out", "screencap", "-p");
         }
+        // find the first display that is turned on, and capture that one
+        const displayId = this.getFirstDisplayId();
+        return this.adb("exec-out", "screencap", "-p", "-d", `${displayId}`);
     }
     collectElements(node) {
         const elements = [];
@@ -254,6 +279,10 @@ class AndroidRobot {
     isAscii(text) {
         return /^[\x00-\x7F]*$/.test(text);
     }
+    escapeShellText(text) {
+        // escape all shell special characters that could be used for injection
+        return text.replace(/[\\'"` \t\n\r|&;()<>{}[\]$*?]/g, "\\$&");
+    }
     async isDeviceKitInstalled() {
         const packages = await this.listPackages();
         return packages.includes("com.mobilenext.devicekit");
@@ -267,7 +296,7 @@ class AndroidRobot {
         if (this.isAscii(text)) {
             // adb shell input only supports ascii characters. and
             // some of the keys have to be escaped.
-            const _text = text.replace(/ /g, "\\ ");
+            const _text = this.escapeShellText(text);
             this.adb("shell", "input", "text", _text);
         }
         else if (await this.isDeviceKitInstalled()) {
@@ -287,16 +316,21 @@ class AndroidRobot {
         if (!BUTTON_MAP[button]) {
             throw new robot_1.ActionableError(`Button "${button}" is not supported`);
         }
-        this.adb("shell", "input", "keyevent", BUTTON_MAP[button]);
+        const mapped = BUTTON_MAP[button];
+        this.adb("shell", "input", "keyevent", mapped);
     }
     async tap(x, y) {
         this.adb("shell", "input", "tap", `${x}`, `${y}`);
     }
+    async longPress(x, y) {
+        // a long press is a swipe with no movement and a long duration
+        this.adb("shell", "input", "swipe", `${x}`, `${y}`, `${x}`, `${y}`, "500");
+    }
     async setOrientation(orientation) {
-        const orientationValue = orientation === "portrait" ? 0 : 1;
+        const value = orientation === "portrait" ? 0 : 1;
         // disable auto-rotation prior to setting the orientation
         this.adb("shell", "settings", "put", "system", "accelerometer_rotation", "0");
-        this.adb("shell", "content", "insert", "--uri", "content://settings/system", "--bind", "name:s:user_rotation", "--bind", `value:i:${orientationValue}`);
+        this.adb("shell", "content", "insert", "--uri", "content://settings/system", "--bind", "name:s:user_rotation", "--bind", `value:i:${value}`);
     }
     async getOrientation() {
         const rotation = this.adb("shell", "settings", "get", "system", "user_rotation").toString().trim();

package/lib/ios.js

@@ -117,6 +117,10 @@ class IosRobot {
         const wda = await this.wda();
         await wda.tap(x, y);
     }
+    async longPress(x, y) {
+        const wda = await this.wda();
+        await wda.longPress(x, y);
+    }
     async getElementsOnScreen() {
         const wda = await this.wda();
         return await wda.getElementsOnScreen();

package/lib/iphone-simulator.js

@@ -104,6 +104,10 @@ class Simctl {
         const wda = await this.wda();
         return wda.tap(x, y);
     }
+    async longPress(x, y) {
+        const wda = await this.wda();
+        return wda.longPress(x, y);
+    }
     async pressButton(button) {
         const wda = await this.wda();
         return wda.pressButton(button);

package/lib/server.js

@@ -229,6 +229,14 @@ const createMcpServer = () => {
         await robot.tap(x, y);
         return `Clicked on screen at coordinates: ${x}, ${y}`;
     });
+    tool("mobile_long_press_on_screen_at_coordinates", "Long press on the screen at given x,y coordinates. If long pressing on an element, use the list_elements_on_screen tool to find the coordinates.", {
+        x: zod_1.z.number().describe("The x coordinate to long press on the screen, in pixels"),
+        y: zod_1.z.number().describe("The y coordinate to long press on the screen, in pixels"),
+    }, async ({ x, y }) => {
+        requireRobot();
+        await robot.longPress(x, y);
+        return `Long pressed on screen at coordinates: ${x}, ${y}`;
+    });
     tool("mobile_list_elements_on_screen", "List elements on screen and their coordinates, with display text or accessibility label. Do not cache this result.", {
         noParams
     }, async ({}) => {

package/lib/webdriver-agent.js

@@ -142,6 +142,32 @@ class WebDriverAgent {
             });
         });
     }
+    async longPress(x, y) {
+        await this.withinSession(async (sessionUrl) => {
+            const url = `${sessionUrl}/actions`;
+            await fetch(url, {
+                method: "POST",
+                headers: {
+                    "Content-Type": "application/json",
+                },
+                body: JSON.stringify({
+                    actions: [
+                        {
+                            type: "pointer",
+                            id: "finger1",
+                            parameters: { pointerType: "touch" },
+                            actions: [
+                                { type: "pointerMove", duration: 0, x, y },
+                                { type: "pointerDown", button: 0 },
+                                { type: "pause", duration: 500 },
+                                { type: "pointerUp", button: 0 }
+                            ]
+                        }
+                    ]
+                }),
+            });
+        });
+    }
     isVisible(rect) {
         return rect.x >= 0 && rect.y >= 0;
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mobilenext/mobile-mcp",
-  "version": "0.0.23",
+  "version": "0.0.25",
   "description": "Mobile MCP",
   "repository": {
     "type": "git",
@@ -13,6 +13,7 @@
   "scripts": {
     "build": "tsc && chmod +x lib/index.js",
     "lint": "eslint .",
+    "fixlint": "eslint . --fix",
     "test": "nyc mocha --require ts-node/register test/*.ts",
     "watch": "tsc --watch",
     "clean": "rm -rf lib",