@mobilenext/mobile-mcp 0.0.17 → 0.0.19

package/README.md

@@ -1,30 +1,44 @@
 # Mobile Next - MCP server for Mobile Development and Automation  | iOS, Android, Simulator, Emulator, and physical devices
 
 This is a [Model Context Protocol (MCP) server](https://github.com/modelcontextprotocol) that enables scalable mobile automation, development through a platform-agnostic interface, eliminating the need for distinct iOS or Android knowledge. You can run it on emulators, simulators, and physical devices (iOS and Android).
-This server allows Agents and LLMs to interact with native iOS/Android applications and devices through structured accessibility snapshots or coordinate-based taps based on screenshots. 
+This server allows Agents and LLMs to interact with native iOS/Android applications and devices through structured accessibility snapshots or coordinate-based taps based on screenshots.
 
-https://github.com/user-attachments/assets/c4e89c4f-cc71-4424-8184-bdbc8c638fa1
-
-
-<p align="center">
+<h4 align="center">
+<a href="https://github.com/mobile-next/mobile-mcp">
+    <img src="https://img.shields.io/github/stars/mobile-next/mobile-mcp" alt="Mobile Next Stars" />
+  </a>
+ <a href="https://github.com/mobile-next/mobile-mcp">
+    <img src="https://img.shields.io/github/contributors/mobile-next/mobile-mcp?color=green" alt="Mobile Next Downloads" />
+  </a>
   <a href="https://www.npmjs.com/package/@mobilenext/mobile-mcp">
-    <img src="https://img.shields.io/badge/npm-@mobilenext/mobile--mcp-red" alt="npm">
+    <img src="https://img.shields.io/npm/dm/@mobilenext/mobile-mcp?logo=npm&style=flat&color=red" alt="npm">
+  </a>
+<a href="https://github.com/mobile-next/mobile-mcp/releases">
+    <img src="https://img.shields.io/github/release/mobile-next/mobile-mcp">
   </a>
-  <a href="https://github.com/mobile-next/mobile-mcp">
-    <img src="https://img.shields.io/badge/github-repo-black" alt="GitHub repo">
+<a href="https://github.com/mobile-next/mobile-mcp/blob/main/LICENSE">
+    <img src="https://img.shields.io/badge/license-Apache 2.0-blue.svg" alt="Mobile MCP is released under the Apache-2.0 License">
   </a>
+
+</p>
+
+<h4 align="center">
+<a href="http://mobilenexthq.com/join-slack">
+    <img src="https://img.shields.io/badge/join-Slack-blueviolet?logo=slack&style=flat" alt="Slack community channel" />
+</a>
 </p>
 
+https://github.com/user-attachments/assets/c4e89c4f-cc71-4424-8184-bdbc8c638fa1
+
 <p align="center">
     <a href="https://github.com/mobile-next/">
         <img alt="mobile-mcp" src="https://raw.githubusercontent.com/mobile-next/mobile-next-assets/refs/heads/main/mobile-mcp-banner.png" width="600">
     </a>
 </p>
 
-
 ### 🚀 Mobile MCP Roadmap: Building the Future of Mobile
 
-Join us on our journey as we continuously enhance Mobile MCP! 
+Join us on our journey as we continuously enhance Mobile MCP!
 Check out our detailed roadmap to see upcoming features, improvements, and milestones. Your feedback is invaluable in shaping the future of mobile automation.
 
 👉 [Explore the Roadmap](https://github.com/orgs/mobile-next/projects/3)
@@ -34,7 +48,7 @@ Check out our detailed roadmap to see upcoming features, improvements, and miles
 
 How we help to scale mobile automation:
 
-- 📲 Native app automation (iOS and Android) for testing or data-entry scenarios. 
+- 📲 Native app automation (iOS and Android) for testing or data-entry scenarios.
 - 📝 Scripted flows and form interactions without manually controlling simulators/emulators or physical devices (iPhone, Samsung, Google Pixel etc)
 - 🧭 Automating multi-step user journeys driven by an LLM
 - 👆 General-purpose mobile application interaction for agent-based frameworks
@@ -42,11 +56,11 @@ How we help to scale mobile automation:
 
 ## Main Features
 
-- 🚀 **Fast and lightweight**: Uses native accessibility trees for most interactions, or screenshot based coordinates where a11y labels are not available. 
+- 🚀 **Fast and lightweight**: Uses native accessibility trees for most interactions, or screenshot based coordinates where a11y labels are not available.
 - 🤖 **LLM-friendly**: No computer vision model required in Accessibility (Snapshot).
 - 🧿 **Visual Sense**: Evaluates and analyses what’s actually rendered on screen to decide the next action. If accessibility data or view-hierarchy coordinates are unavailable, it falls back to screenshot-based analysis.
 - 📊 **Deterministic tool application**: Reduces ambiguity found in purely screenshot-based approaches by relying on structured data whenever possible.
-- 📺 **Extract structured data**: Enables you to extract structred data from anything visible on screen. 
+- 📺 **Extract structured data**: Enables you to extract structred data from anything visible on screen.
 
 ## 🏗️ Mobile MCP Architecture
 
@@ -57,14 +71,14 @@ How we help to scale mobile automation:
 </p>
 
 
-## 📚 Wiki page 
+## 📚 Wiki page
 
 More details in our [wiki page](https://github.com/mobile-next/mobile-mcp/wiki) for setup, configuration and debugging related questions.
 
 
 ## Installation and configuration
 
-Setup our MCP with Cursor, Claude, VS Code, Github Copilot:
+Setup our MCP with Cline, Cursor, Claude, VS Code, Github Copilot:
 
 ```json
 {
@@ -77,11 +91,13 @@ Setup our MCP with Cursor, Claude, VS Code, Github Copilot:
 }
 
 ```
+[Cline:](https://docs.cline.bot/mcp/configuring-mcp-servers) To setup Cline, just add the json above to your MCP settings file.
+[More in our wiki](https://github.com/mobile-next/mobile-mcp/wiki/Cline)
 
 [Claude Code:](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview)
 
 ```
-claude mcp add mobile -- npx -y @mobilenext/mobile-mcp@latest ⁠
+claude mcp add mobile -- npx -y @mobilenext/mobile-mcp@latest
 ```
 
 [Read more in our wiki](https://github.com/mobile-next/mobile-mcp/wiki)! 🚀
@@ -89,7 +105,7 @@ claude mcp add mobile -- npx -y @mobilenext/mobile-mcp@latest ⁠
 
 ### 🛠️ How to Use 📝
 
-After adding the MCP server to your IDE/Client, you can instruct your AI assistant to use the available tools. 
+After adding the MCP server to your IDE/Client, you can instruct your AI assistant to use the available tools.
 For example, in Cursor's agent mode, you could use the prompts below to quickly validate, test and iterate on UI intereactions, read information from screen, go through complex workflows.
 Be descriptive, straight to the point.
 
@@ -101,48 +117,57 @@ You can specifiy detailed workflows in a single prompt, verify business logic, s
 
 **Search for a video, comment, like and share it.**
 ```
-Find the video called " Beginner Recipe for Tonkotsu Ramen" by Way of Ramen, click on like video, after liking write a comment " this was delicious, will make it next Friday", share the video with the first contact in your whatsapp list. 
+Find the video called " Beginner Recipe for Tonkotsu Ramen" by Way of
+Ramen, click on like video, after liking write a comment " this was
+delicious, will make it next Friday", share the video with the first
+contact in your whatsapp list.
 ```
 
-**Download a successful step counter app, register, setup workout and 5 start the app**
+**Download a successful step counter app, register, setup workout and 5-star the app**
 ```
-Find and Download a free "Pomodoro" app that has more thank 1k stars. 
-Launch the app, register with my email, after registration find how to start a pomodoro timer. 
-When the pomodoro timer started, go back to the app store and rate the app 5 stars, 
-and leave a comment how useful the app is. 
+Find and Download a free "Pomodoro" app that has more than 1k stars.
+Launch the app, register with my email, after registration find how to
+start a pomodoro timer. When the pomodoro timer started, go back to the
+app store and rate the app 5 stars, and leave a comment how useful the
+app is.
 ```
 
 **Search in Substack, read, highlight, comment and save an article**
 ```
-Open Substack website, search for "Latest trends in AI automation 2025", open the first article, 
-highlight the section titled "Emerging AI trends", and save article to reading list for later review, 
-comment a random paragraph summary.
+Open Substack website, search for "Latest trends in AI automation 2025",
+open the first article, highlight the section titled "Emerging AI trends",
+and save article to reading list for later review, comment a random
+paragraph summary.
 ```
 
 **Reserve a workout class, set timer**
 ```
-Open ClassPass, search for yoga classes tomorrow morning within 2 miles, 
+Open ClassPass, search for yoga classes tomorrow morning within 2 miles,
 book the highest-rated class at 7 AM, confirm reservation,
- setup a timer for the booked slot in the phone
+setup a timer for the booked slot in the phone
 ```
 
 **Find a local event, setup calendar event**
 ```
-Open Eventbrite, search for AI startup meetup events happening this weekend in "Austin, TX", 
-select the most popular one, register and RSVP yes to the even, setup a calendar event as a reminder.
+Open Eventbrite, search for AI startup meetup events happening this
+weekend in "Austin, TX", select the most popular one, register and RSVP
+yes to the event, setup a calendar event as a reminder.
 ```
 
 **Check weather forecast and send a Whatsapp/Telegram/Slack message**
 ```
-Open Weather app, check tomorrow's weather forecast for "Berlin", and send the summary 
-via Whatsapp/Telegram/Slack to contact "Lauren Trown", thumbs up their response.
+Open Weather app, check tomorrow's weather forecast for "Berlin", and
+send the summary via Whatsapp/Telegram/Slack to contact "Lauren Trown",
+thumbs up their response.
 ```
 
 - **Schedule a meeting in Zoom and share invite via email**
 ```
-Open Zoom app, schedule a meeting titled "AI Hackathon" for tomorrow at 10 AM with a duration of 1 hour,
-copy the invitation link, and send it via Gmail to contacts "team@example.com".
+Open Zoom app, schedule a meeting titled "AI Hackathon" for tomorrow at
+10AM with a duration of 1 hour, copy the invitation link, and send it via
+Gmail to contacts "team@example.com".
 ```
+[More prompt examples can be found here.](https://github.com/mobile-next/mobile-mcp/wiki/Prompt-Example-repo-list)
 
 ## Prerequisites
 
@@ -176,7 +201,7 @@ On iOS, you'll need Xcode and to run the Simulator before using Mobile MCP with
 
 # Thanks to all contributors ❤️
 
-### We appreciate everyone who has helped improve this project. 
+### We appreciate everyone who has helped improve this project.
 
   <a href = "https://github.com/mobile-next/mobile-mcp/graphs/contributors">
    <img src = "https://contrib.rocks/image?repo=mobile-next/mobile-mcp"/>

package/lib/android.js

@@ -120,7 +120,6 @@ class AndroidRobot {
     async swipe(direction) {
         const screenSize = await this.getScreenSize();
         const centerX = screenSize.width >> 1;
-        // const centerY = screenSize[1] >> 1;
         let x0, y0, x1, y1;
         switch (direction) {
             case "up":
@@ -133,6 +132,50 @@ class AndroidRobot {
                 y0 = Math.floor(screenSize.height * 0.20);
                 y1 = Math.floor(screenSize.height * 0.80);
                 break;
+            case "left":
+                x0 = Math.floor(screenSize.width * 0.80);
+                x1 = Math.floor(screenSize.width * 0.20);
+                y0 = y1 = Math.floor(screenSize.height * 0.50);
+                break;
+            case "right":
+                x0 = Math.floor(screenSize.width * 0.20);
+                x1 = Math.floor(screenSize.width * 0.80);
+                y0 = y1 = Math.floor(screenSize.height * 0.50);
+                break;
+            default:
+                throw new robot_1.ActionableError(`Swipe direction "${direction}" is not supported`);
+        }
+        this.adb("shell", "input", "swipe", `${x0}`, `${y0}`, `${x1}`, `${y1}`, "1000");
+    }
+    async swipeFromCoordinate(x, y, direction, distance) {
+        const screenSize = await this.getScreenSize();
+        let x0, y0, x1, y1;
+        // Use provided distance or default to 30% of screen dimension
+        const defaultDistanceY = Math.floor(screenSize.height * 0.3);
+        const defaultDistanceX = Math.floor(screenSize.width * 0.3);
+        const swipeDistanceY = distance || defaultDistanceY;
+        const swipeDistanceX = distance || defaultDistanceX;
+        switch (direction) {
+            case "up":
+                x0 = x1 = x;
+                y0 = y;
+                y1 = Math.max(0, y - swipeDistanceY);
+                break;
+            case "down":
+                x0 = x1 = x;
+                y0 = y;
+                y1 = Math.min(screenSize.height, y + swipeDistanceY);
+                break;
+            case "left":
+                x0 = x;
+                x1 = Math.max(0, x - swipeDistanceX);
+                y0 = y1 = y;
+                break;
+            case "right":
+                x0 = x;
+                x1 = Math.min(screenSize.width, x + swipeDistanceX);
+                y0 = y1 = y;
+                break;
             default:
                 throw new robot_1.ActionableError(`Swipe direction "${direction}" is not supported`);
         }
@@ -164,6 +207,10 @@ class AndroidRobot {
                 // only provide it if it's true, otherwise don't confuse llm
                 element.focused = true;
             }
+            const resourceId = node["resource-id"];
+            if (resourceId !== null && resourceId !== "") {
+                element.identifier = resourceId;
+            }
             if (element.rect.width > 0 && element.rect.height > 0) {
                 elements.push(element);
             }
@@ -215,7 +262,7 @@ class AndroidRobot {
                 // console.error("Failed to get UIAutomator XML. Here's a screenshot: " + screenshot.toString("base64"));
                 continue;
             }
-            return dump;
+            return dump.substring(dump.indexOf("<?xml"));
         }
         throw new robot_1.ActionableError("Failed to get UIAutomator XML");
     }

package/lib/index.js

@@ -1,17 +1,60 @@
 #!/usr/bin/env node
 "use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
 Object.defineProperty(exports, "__esModule", { value: true });
+const sse_js_1 = require("@modelcontextprotocol/sdk/server/sse.js");
 const stdio_js_1 = require("@modelcontextprotocol/sdk/server/stdio.js");
 const server_1 = require("./server");
 const logger_1 = require("./logger");
-async function main() {
-    const transport = new stdio_js_1.StdioServerTransport();
+const express_1 = __importDefault(require("express"));
+const commander_1 = require("commander");
+const startSseServer = async (port) => {
+    const app = (0, express_1.default)();
     const server = (0, server_1.createMcpServer)();
-    await server.connect(transport);
-    (0, logger_1.error)("mobile-mcp server running on stdio");
-}
-main().catch(err => {
-    console.error("Fatal error in main():", err);
-    (0, logger_1.error)("Fatal error in main(): " + JSON.stringify(err.stack));
-    process.exit(1);
-});
+    let transport = null;
+    app.post("/mcp", (req, res) => {
+        if (transport) {
+            transport.handlePostMessage(req, res);
+        }
+    });
+    app.get("/mcp", (req, res) => {
+        if (transport) {
+            transport.close();
+        }
+        transport = new sse_js_1.SSEServerTransport("/mcp", res);
+        server.connect(transport);
+    });
+    app.listen(port, () => {
+        (0, logger_1.error)(`mobile-mcp ${(0, server_1.getAgentVersion)()} sse server listening on http://localhost:${port}/mcp`);
+    });
+};
+const startStdioServer = async () => {
+    try {
+        const transport = new stdio_js_1.StdioServerTransport();
+        const server = (0, server_1.createMcpServer)();
+        await server.connect(transport);
+        (0, logger_1.error)("mobile-mcp server running on stdio");
+    }
+    catch (err) {
+        console.error("Fatal error in main():", err);
+        (0, logger_1.error)("Fatal error in main(): " + JSON.stringify(err.stack));
+        process.exit(1);
+    }
+};
+const main = async () => {
+    commander_1.program
+        .version((0, server_1.getAgentVersion)())
+        .option("--port <port>", "Start SSE server on this port")
+        .option("--stdio", "Start stdio server (default)")
+        .parse(process.argv);
+    const options = commander_1.program.opts();
+    if (options.port) {
+        await startSseServer(+options.port);
+    }
+    else {
+        await startStdioServer();
+    }
+};
+main().then();

package/lib/ios.js

@@ -83,6 +83,10 @@ class IosRobot {
         const wda = await this.wda();
         await wda.swipe(direction);
     }
+    async swipeFromCoordinate(x, y, direction, distance) {
+        const wda = await this.wda();
+        await wda.swipeFromCoordinate(x, y, direction, distance);
+    }
     async listApps() {
         await this.assertTunnelRunning();
         const output = await this.ios("apps", "--all", "--list");

package/lib/iphone-simulator.js

@@ -39,75 +39,13 @@ class Simctl {
     async terminateApp(packageName) {
         this.simctl("terminate", this.simulatorUuid, packageName);
     }
-    static parseIOSAppData(inputText) {
-        const result = [];
-        let ParseState;
-        (function (ParseState) {
-            ParseState[ParseState["LOOKING_FOR_APP"] = 0] = "LOOKING_FOR_APP";
-            ParseState[ParseState["IN_APP"] = 1] = "IN_APP";
-            ParseState[ParseState["IN_PROPERTY"] = 2] = "IN_PROPERTY";
-        })(ParseState || (ParseState = {}));
-        let state = ParseState.LOOKING_FOR_APP;
-        let currentApp = {};
-        let appIdentifier = "";
-        const lines = inputText.split("\n");
-        for (let line of lines) {
-            line = line.trim();
-            if (line === "") {
-                continue;
-            }
-            switch (state) {
-                case ParseState.LOOKING_FOR_APP:
-                    // look for app identifier pattern: "com.example.app" = {
-                    const appMatch = line.match(/^"?([^"=]+)"?\s*=\s*\{/);
-                    if (appMatch) {
-                        appIdentifier = appMatch[1].trim();
-                        currentApp = {
-                            CFBundleIdentifier: appIdentifier,
-                        };
-                        state = ParseState.IN_APP;
-                    }
-                    break;
-                case ParseState.IN_APP:
-                    if (line === "};") {
-                        result.push(currentApp);
-                        currentApp = {};
-                        state = ParseState.LOOKING_FOR_APP;
-                    }
-                    else {
-                        // look for property: PropertyName = Value;
-                        const propertyMatch = line.match(/^([^=]+)\s*=\s*(.+?);\s*$/);
-                        if (propertyMatch) {
-                            const propName = propertyMatch[1].trim();
-                            let propValue = propertyMatch[2].trim();
-                            // remove quotes if present (they're optional)
-                            if (propValue.startsWith('"') && propValue.endsWith('"')) {
-                                propValue = propValue.substring(1, propValue.length - 1);
-                            }
-                            // add property to current app
-                            currentApp[propName] = propValue;
-                        }
-                        else if (line.endsWith("{")) {
-                            // nested property like GroupContainers = {
-                            state = ParseState.IN_PROPERTY;
-                        }
-                    }
-                    break;
-                case ParseState.IN_PROPERTY:
-                    if (line === "};") {
-                        // end of nested property
-                        state = ParseState.IN_APP;
-                    }
-                    // skip content of nested properties, we don't care of those right now
-                    break;
-            }
-        }
-        return result;
-    }
     async listApps() {
         const text = this.simctl("listapps", this.simulatorUuid).toString();
-        const apps = Simctl.parseIOSAppData(text);
-        return apps.map(app => ({
+        const result = (0, child_process_1.execFileSync)("plutil", ["-convert", "json", "-o", "-", "-r", "-"], {
+            input: text,
+        });
+        const output = JSON.parse(result.toString());
+        return Object.values(output).map(app => ({
             packageName: app.CFBundleIdentifier,
             appName: app.CFBundleDisplayName,
         }));
@@ -124,6 +62,10 @@ class Simctl {
         const wda = await this.wda();
         return wda.swipe(direction);
     }
+    async swipeFromCoordinate(x, y, direction, distance) {
+        const wda = await this.wda();
+        return wda.swipeFromCoordinate(x, y, direction, distance);
+    }
     async tap(x, y) {
         const wda = await this.wda();
         return wda.tap(x, y);

package/lib/server.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.createMcpServer = void 0;
+exports.createMcpServer = exports.getAgentVersion = void 0;
 const mcp_js_1 = require("@modelcontextprotocol/sdk/server/mcp.js");
 const zod_1 = require("zod");
 const logger_1 = require("./logger");
@@ -14,6 +14,7 @@ const getAgentVersion = () => {
     const json = require("../package.json");
     return json.version;
 };
+exports.getAgentVersion = getAgentVersion;
 const getLatestAgentVersion = async () => {
     const response = await fetch("https://api.github.com/repos/mobile-next/mobile-mcp/tags?per_page=1");
     const json = await response.json();
@@ -22,7 +23,7 @@ const getLatestAgentVersion = async () => {
 const checkForLatestAgentVersion = async () => {
     try {
         const latestVersion = await getLatestAgentVersion();
-        const currentVersion = getAgentVersion();
+        const currentVersion = (0, exports.getAgentVersion)();
         if (latestVersion !== currentVersion) {
             (0, logger_1.trace)(`You are running an older version of the agent. Please update to the latest version: ${latestVersion}.`);
         }
@@ -34,12 +35,13 @@ const checkForLatestAgentVersion = async () => {
 const createMcpServer = () => {
     const server = new mcp_js_1.McpServer({
         name: "mobile-mcp",
-        version: getAgentVersion(),
+        version: (0, exports.getAgentVersion)(),
         capabilities: {
             resources: {},
             tools: {},
         },
     });
+    const noParams = zod_1.z.object({});
     const tool = (name, description, paramsSchema, cb) => {
         const wrappedCb = async (args) => {
             try {
@@ -75,7 +77,9 @@ const createMcpServer = () => {
             throw new robot_1.ActionableError("No device selected. Use the mobile_use_device tool to select a device.");
         }
     };
-    tool("mobile_list_available_devices", "List all available devices. This includes both physical devices and simulators. If there is more than one device returned, you need to let the user select one of them.", {}, async ({}) => {
+    tool("mobile_list_available_devices", "List all available devices. This includes both physical devices and simulators. If there is more than one device returned, you need to let the user select one of them.", {
+        noParams
+    }, async ({}) => {
         const iosManager = new ios_1.IosManager();
         const androidManager = new android_1.AndroidDeviceManager();
         const devices = simulatorManager.listBootedSimulators();
@@ -117,7 +121,9 @@ const createMcpServer = () => {
         }
         return `Selected device: ${device}`;
     });
-    tool("mobile_list_apps", "List all the installed apps on the device", {}, async ({}) => {
+    tool("mobile_list_apps", "List all the installed apps on the device", {
+        noParams
+    }, async ({}) => {
         requireRobot();
         const result = await robot.listApps();
         return `Found these apps on device: ${result.map(app => `${app.appName} (${app.packageName})`).join(", ")}`;
@@ -136,7 +142,9 @@ const createMcpServer = () => {
         await robot.terminateApp(packageName);
         return `Terminated app ${packageName}`;
     });
-    tool("mobile_get_screen_size", "Get the screen size of the mobile device in pixels", {}, async ({}) => {
+    tool("mobile_get_screen_size", "Get the screen size of the mobile device in pixels", {
+        noParams
+    }, async ({}) => {
         requireRobot();
         const screenSize = await robot.getScreenSize();
         return `Screen size is ${screenSize.width}x${screenSize.height} pixels`;
@@ -149,7 +157,9 @@ const createMcpServer = () => {
         await robot.tap(x, y);
         return `Clicked on screen at coordinates: ${x}, ${y}`;
     });
-    tool("mobile_list_elements_on_screen", "List elements on screen and their coordinates, with display text or accessibility label. Do not cache this result.", {}, async ({}) => {
+    tool("mobile_list_elements_on_screen", "List elements on screen and their coordinates, with display text or accessibility label. Do not cache this result.", {
+        noParams
+    }, async ({}) => {
         requireRobot();
         const elements = await robot.getElementsOnScreen();
         const result = elements.map(element => {
@@ -159,6 +169,7 @@ const createMcpServer = () => {
                 label: element.label,
                 name: element.name,
                 value: element.value,
+                identifier: element.identifier,
                 coordinates: {
                     x: element.rect.x,
                     y: element.rect.y,
@@ -188,11 +199,23 @@ const createMcpServer = () => {
         return `Opened URL: ${url}`;
     });
     tool("swipe_on_screen", "Swipe on the screen", {
-        direction: zod_1.z.enum(["up", "down"]).describe("The direction to swipe"),
-    }, async ({ direction }) => {
+        direction: zod_1.z.enum(["up", "down", "left", "right"]).describe("The direction to swipe"),
+        x: zod_1.z.number().optional().describe("The x coordinate to start the swipe from, in pixels. If not provided, uses center of screen"),
+        y: zod_1.z.number().optional().describe("The y coordinate to start the swipe from, in pixels. If not provided, uses center of screen"),
+        distance: zod_1.z.number().optional().describe("The distance to swipe in pixels. Defaults to 400 pixels for iOS or 30% of screen dimension for Android"),
+    }, async ({ direction, x, y, distance }) => {
         requireRobot();
-        await robot.swipe(direction);
-        return `Swiped ${direction} on screen`;
+        if (x !== undefined && y !== undefined) {
+            // Use coordinate-based swipe
+            await robot.swipeFromCoordinate(x, y, direction, distance);
+            const distanceText = distance ? ` ${distance} pixels` : "";
+            return `Swiped ${direction}${distanceText} from coordinates: ${x}, ${y}`;
+        }
+        else {
+            // Use center-based swipe
+            await robot.swipe(direction);
+            return `Swiped ${direction} on screen`;
+        }
     });
     tool("mobile_type_keys", "Type text into the focused element", {
         text: zod_1.z.string().describe("The text to type"),
@@ -205,7 +228,9 @@ const createMcpServer = () => {
         }
         return `Typed text: ${text}`;
     });
-    server.tool("mobile_take_screenshot", "Take a screenshot of the mobile device. Use this to understand what's on screen, if you need to press an element that is available through view hierarchy then you must list elements on screen instead. Do not cache this result.", {}, async ({}) => {
+    server.tool("mobile_take_screenshot", "Take a screenshot of the mobile device. Use this to understand what's on screen, if you need to press an element that is available through view hierarchy then you must list elements on screen instead. Do not cache this result.", {
+        noParams
+    }, async ({}) => {
         requireRobot();
         try {
             const screenSize = await robot.getScreenSize();
@@ -249,7 +274,9 @@ const createMcpServer = () => {
         await robot.setOrientation(orientation);
         return `Changed device orientation to ${orientation}`;
     });
-    tool("mobile_get_orientation", "Get the current screen orientation of the device", {}, async () => {
+    tool("mobile_get_orientation", "Get the current screen orientation of the device", {
+        noParams
+    }, async () => {
         requireRobot();
         const orientation = await robot.getOrientation();
         return `Current device orientation is ${orientation}`;

package/lib/webdriver-agent.js

@@ -29,7 +29,14 @@ class WebDriverAgent {
             },
             body: JSON.stringify({ capabilities: { alwaysMatch: { platformName: "iOS" } } }),
         });
+        if (!response.ok) {
+            const errorText = await response.text();
+            throw new robot_1.ActionableError(`Failed to create WebDriver session: ${response.status} ${errorText}`);
+        }
         const json = await response.json();
+        if (!json.value || !json.value.sessionId) {
+            throw new robot_1.ActionableError(`Invalid session response: ${JSON.stringify(json)}`);
+        }
         return json.value.sessionId;
     }
     async deleteSession(sessionId) {
@@ -44,8 +51,8 @@ class WebDriverAgent {
         await this.deleteSession(sessionId);
         return result;
     }
-    async getScreenSize() {
-        return this.withinSession(async (sessionUrl) => {
+    async getScreenSize(sessionUrl) {
+        if (sessionUrl) {
             const url = `${sessionUrl}/wda/screen`;
             const response = await fetch(url);
             const json = await response.json();
@@ -54,7 +61,19 @@ class WebDriverAgent {
                 height: json.value.screenSize.height,
                 scale: json.value.scale || 1,
             };
-        });
+        }
+        else {
+            return this.withinSession(async (sessionUrlInner) => {
+                const url = `${sessionUrlInner}/wda/screen`;
+                const response = await fetch(url);
+                const json = await response.json();
+                return {
+                    width: json.value.screenSize.width,
+                    height: json.value.screenSize.height,
+                    scale: json.value.scale || 1,
+                };
+            });
+        }
     }
     async sendKeys(keys) {
         await this.withinSession(async (sessionUrl) => {
@@ -130,12 +149,13 @@ class WebDriverAgent {
         const acceptedTypes = ["TextField", "Button", "Switch", "Icon", "SearchField", "StaticText", "Image"];
         if (acceptedTypes.includes(source.type)) {
             if (source.isVisible === "1" && this.isVisible(source.rect)) {
-                if (source.label !== null || source.name !== null) {
+                if (source.label !== null || source.name !== null || source.rawIdentifier !== null) {
                     output.push({
                         type: source.type,
                         label: source.label,
                         name: source.name,
                         value: source.value,
+                        identifier: source.rawIdentifier,
                         rect: {
                             x: source.rect.x,
                             y: source.rect.y,
@@ -173,17 +193,95 @@ class WebDriverAgent {
     }
     async swipe(direction) {
         await this.withinSession(async (sessionUrl) => {
-            const x0 = 200;
-            let y0 = 600;
-            const x1 = 200;
-            let y1 = 200;
-            if (direction === "up") {
-                const tmp = y0;
-                y0 = y1;
-                y1 = tmp;
+            const screenSize = await this.getScreenSize(sessionUrl);
+            let x0, y0, x1, y1;
+            // Use 60% of the width/height for swipe distance
+            const verticalDistance = Math.floor(screenSize.height * 0.6);
+            const horizontalDistance = Math.floor(screenSize.width * 0.6);
+            const centerX = Math.floor(screenSize.width / 2);
+            const centerY = Math.floor(screenSize.height / 2);
+            switch (direction) {
+                case "up":
+                    x0 = x1 = centerX;
+                    y0 = centerY + Math.floor(verticalDistance / 2);
+                    y1 = centerY - Math.floor(verticalDistance / 2);
+                    break;
+                case "down":
+                    x0 = x1 = centerX;
+                    y0 = centerY - Math.floor(verticalDistance / 2);
+                    y1 = centerY + Math.floor(verticalDistance / 2);
+                    break;
+                case "left":
+                    y0 = y1 = centerY;
+                    x0 = centerX + Math.floor(horizontalDistance / 2);
+                    x1 = centerX - Math.floor(horizontalDistance / 2);
+                    break;
+                case "right":
+                    y0 = y1 = centerY;
+                    x0 = centerX - Math.floor(horizontalDistance / 2);
+                    x1 = centerX + Math.floor(horizontalDistance / 2);
+                    break;
+                default:
+                    throw new robot_1.ActionableError(`Swipe direction "${direction}" is not supported`);
             }
             const url = `${sessionUrl}/actions`;
-            await fetch(url, {
+            const response = await fetch(url, {
+                method: "POST",
+                headers: {
+                    "Content-Type": "application/json",
+                },
+                body: JSON.stringify({
+                    actions: [
+                        {
+                            type: "pointer",
+                            id: "finger1",
+                            parameters: { pointerType: "touch" },
+                            actions: [
+                                { type: "pointerMove", duration: 0, x: x0, y: y0 },
+                                { type: "pointerDown", button: 0 },
+                                { type: "pointerMove", duration: 1000, x: x1, y: y1 },
+                                { type: "pointerUp", button: 0 }
+                            ]
+                        }
+                    ]
+                }),
+            });
+            if (!response.ok) {
+                const errorText = await response.text();
+                throw new robot_1.ActionableError(`WebDriver actions request failed: ${response.status} ${errorText}`);
+            }
+            // Clear actions to ensure they complete
+            await fetch(`${sessionUrl}/actions`, {
+                method: "DELETE",
+            });
+        });
+    }
+    async swipeFromCoordinate(x, y, direction, distance = 400) {
+        await this.withinSession(async (sessionUrl) => {
+            // Use simple coordinates like the working swipe method
+            const x0 = x;
+            const y0 = y;
+            let x1 = x;
+            let y1 = y;
+            // Calculate target position based on direction and distance
+            switch (direction) {
+                case "up":
+                    y1 = y - distance; // Move up by specified distance
+                    break;
+                case "down":
+                    y1 = y + distance; // Move down by specified distance
+                    break;
+                case "left":
+                    x1 = x - distance; // Move left by specified distance
+                    break;
+                case "right":
+                    x1 = x + distance; // Move right by specified distance
+                    break;
+                default:
+                    throw new robot_1.ActionableError(`Swipe direction "${direction}" is not supported`);
+            }
+            const url = `${sessionUrl}/actions`;
+            const response = await fetch(url, {
                 method: "POST",
                 headers: {
                     "Content-Type": "application/json",
@@ -197,14 +295,21 @@ class WebDriverAgent {
                             actions: [
                                 { type: "pointerMove", duration: 0, x: x0, y: y0 },
                                 { type: "pointerDown", button: 0 },
-                                { type: "pointerMove", duration: 0, x: x1, y: y1 },
-                                { type: "pause", duration: 1000 },
+                                { type: "pointerMove", duration: 1000, x: x1, y: y1 },
                                 { type: "pointerUp", button: 0 }
                             ]
                         }
                     ]
                 }),
             });
+            if (!response.ok) {
+                const errorText = await response.text();
+                throw new robot_1.ActionableError(`WebDriver actions request failed: ${response.status} ${errorText}`);
+            }
+            // Clear actions to ensure they complete
+            await fetch(`${sessionUrl}/actions`, {
+                method: "DELETE",
+            });
         });
     }
     async setOrientation(orientation) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mobilenext/mobile-mcp",
-  "version": "0.0.17",
+  "version": "0.0.19",
   "description": "Mobile MCP",
   "repository": {
     "type": "git",
@@ -23,6 +23,10 @@
   ],
   "dependencies": {
     "@modelcontextprotocol/sdk": "^1.6.1",
+    "@types/commander": "^2.12.0",
+    "@types/express": "^5.0.3",
+    "commander": "^14.0.0",
+    "express": "^5.1.0",
     "fast-xml-parser": "^5.0.9",
     "zod-to-json-schema": "^3.24.4"
   },
@@ -40,8 +44,8 @@
     "eslint-plugin-import": "^2.31.0",
     "eslint-plugin-notice": "^1.0.0",
     "husky": "^9.1.7",
-    "nyc": "^17.1.0",
     "mocha": "^11.1.0",
+    "nyc": "^17.1.0",
     "ts-node": "^10.9.2",
     "typescript": "^5.8.2"
   },