native-devtools-mcp 0.3.6 → 0.4.0

package/README.md

@@ -4,16 +4,18 @@
 
 ![Version](https://img.shields.io/npm/v/native-devtools-mcp?style=flat-square)
 ![License](https://img.shields.io/npm/l/native-devtools-mcp?style=flat-square)
-![Platform](https://img.shields.io/badge/platform-macOS%20%7C%20Windows-blue?style=flat-square)
+![Platform](https://img.shields.io/badge/platform-macOS%20%7C%20Windows%20%7C%20Android-blue?style=flat-square)
 ![Downloads](https://img.shields.io/npm/dt/native-devtools-mcp?style=flat-square)
 
-**Give your AI agent "eyes" and "hands" for native desktop applications.**
+**Give your AI agent "eyes" and "hands" for native desktop and mobile applications.**
 
-A Model Context Protocol (MCP) server that provides **Computer Use** capabilities: screenshots, OCR, input simulation, and window management.
+A Model Context Protocol (MCP) server that provides **Computer Use** capabilities: screenshots, OCR, input simulation, and window management — for **native desktop apps** and **Android devices**, not just browsers.
 
-[//]: # "Search keywords: MCP, Model Context Protocol, computer use, desktop automation, UI automation, RPA, screenshots, OCR, mouse, keyboard, screen reading, macOS, Windows, native-devtools-mcp"
+**Works with:** [Claude Desktop](https://claude.ai/download) • [Claude Code](https://docs.anthropic.com/en/docs/claude-code) • [Cursor](https://cursor.com) • Any MCP-compatible client
 
-[Features](#-features) • [Installation](#-installation) • [For AI Agents](#-for-ai-agents-llms) • [Permissions](#-required-permissions-macos)
+[//]: # "Search keywords: MCP, MCP server, Model Context Protocol, computer use, desktop automation, UI automation, native app testing, test automation, e2e testing, RPA, screenshots, OCR, template matching, accessibility, mouse, keyboard, screen reading, macOS, Windows, Android, ADB, mobile testing, Claude, Claude Code, Cursor, AI agent, native-devtools-mcp"
+
+[Features](#-features) • [Installation](#-installation) • [For AI Agents](#-for-ai-agents-llms) • [Android](#-android-support) • [Permissions](#-required-permissions-macos)
 
 <table>
 <tr>
@@ -30,10 +32,6 @@ A Model Context Protocol (MCP) server that provides **Computer Use** capabilitie
 
 ---
 
-## 🔍 Search Keywords
-
-MCP, Model Context Protocol, computer use, desktop automation, UI automation, RPA, screenshots, OCR, screen reading, mouse, keyboard, macOS, Windows, native-devtools-mcp.
-
 ## 🚀 Features
 
 - **👀 Computer Vision:** Capture screenshots of screens, windows, or specific regions. Includes built-in OCR (text recognition) to "read" the screen.
@@ -41,6 +39,7 @@ MCP, Model Context Protocol, computer use, desktop automation, UI automation, RP
 - **🪟 Window Management:** List open windows, find applications, and bring them to focus.
 - **🧩 Template Matching:** Find non-text UI elements (icons, shapes) using `load_image` + `find_image`, returning precise click coordinates.
 - **🔒 Local & Private:** 100% local execution. No screenshots or data are ever sent to external servers.
+- **📱 Android Support:** Connect to Android devices over ADB for screenshots, input simulation, UI element search, and app management — all from the same MCP server.
 - **🔌 Dual-Mode Interaction:**
     1.  **Visual/Native:** Works with *any* app via screenshots & coordinates (Universal).
     2.  **AppDebugKit:** Deep integration for supported apps to inspect the UI tree (DOM-like structure).
@@ -57,7 +56,7 @@ This MCP server is designed to be **highly discoverable and usable** by AI model
 3.  `find_text`: A shortcut to find text on screen and get its coordinates immediately. Uses the platform **accessibility API** (macOS Accessibility / Windows UI Automation) for precise element-level matching, with OCR fallback.
 4.  `load_image` / `find_image`: Template matching for non-text UI elements (icons, shapes), returning screen coordinates for clicking.
 
-## 📦 Installation (macOS + Windows)
+## 📦 Installation
 
 The install steps are identical on macOS and Windows.
 
@@ -84,6 +83,12 @@ cd native-devtools-mcp
 cargo build --release
 # Binary: ./target/release/native-devtools-mcp
 ```
+
+To include Android device support, enable the `android` feature flag:
+
+```bash
+cargo build --release --features android
+```
 </details>
 
 ## ⚙️ Configuration
@@ -179,6 +184,77 @@ Use `find_image` when the target is **not text** (icons, toggles, custom control
 
 Optional inputs like `mask_id`, `search_region`, `scales`, and `rotations` can improve precision and performance.
 
+## 📱 Android Support
+
+Android support is available as an optional feature flag. It lets the MCP server communicate with Android devices over ADB (USB or Wi-Fi), providing screenshots, input simulation, UI element search, and app management.
+
+### Prerequisites
+
+1. **ADB installed** on the host machine (`brew install android-platform-tools` on macOS, or install via [Android SDK](https://developer.android.com/tools/releases/platform-tools))
+2. **USB debugging enabled** on the Android device (Settings > Developer options > USB debugging)
+3. **ADB server running** — starts automatically when you run `adb devices`
+
+### Building with Android support
+
+```bash
+cargo build --release --features android
+```
+
+### Android tools
+
+All Android tools are prefixed with `android_` and appear dynamically after connecting to a device:
+
+| Tool | Description |
+|------|-------------|
+| `android_list_devices` | List all ADB-connected devices (always available) |
+| `android_connect` | Connect to a device by serial number |
+| `android_disconnect` | Disconnect from the current device |
+| `android_screenshot` | Capture the device screen |
+| `android_find_text` | Find UI elements by text (via uiautomator) |
+| `android_click` | Tap at screen coordinates |
+| `android_swipe` | Swipe between two points |
+| `android_type_text` | Type text on the device |
+| `android_press_key` | Press a key (e.g., `KEYCODE_HOME`, `KEYCODE_BACK`) |
+| `android_launch_app` | Launch an app by package name |
+| `android_list_apps` | List installed packages |
+| `android_get_display_info` | Get screen resolution and density |
+| `android_get_current_activity` | Get the current foreground activity |
+
+### Typical workflow
+
+```
+android_list_devices          → find your device serial
+android_connect(serial="...")  → connect (unlocks android_* tools)
+android_screenshot            → see what's on screen
+android_find_text(text="OK")  → locate a button
+android_click(x=..., y=...)   → tap it
+```
+
+### Known issues
+
+> **MIUI / HyperOS (Xiaomi, Redmi, POCO devices):** Input injection (`android_click`, `android_type_text`, `android_press_key`, `android_swipe`) and `android_find_text` (via uiautomator) require an additional security toggle:
+>
+> **Settings > Developer options > USB debugging (Security settings)** — enable this toggle. MIUI may require you to sign in with a Mi account to enable it.
+>
+> Without this, you'll see `INJECT_EVENTS permission` errors for input tools and `could not get idle state` errors for `android_find_text`. Screenshot and device info tools work without this toggle.
+
+> **Wireless ADB:** To connect without a USB cable, first connect via USB and run:
+> ```bash
+> adb tcpip 5555
+> adb connect <phone-ip>:5555
+> ```
+> Then use the `<phone-ip>:5555` serial in `android_connect`.
+
+### Smoke tests
+
+Smoke tests verify all Android tools against a real connected device. They are `#[ignore]`d by default and must be run explicitly:
+
+```bash
+cargo test --features android --test android_smoke_tests -- --ignored --test-threads=1
+```
+
+Tests must run sequentially (`--test-threads=1`) since they share a single physical device. The device must be unlocked and awake.
+
 ## 🏗️ Architecture
 
 ```mermaid
@@ -186,6 +262,7 @@ graph TD
     Client[Claude / LLM Client] <-->|JSON-RPC 2.0| Server[native-devtools-mcp]
     Server -->|Direct API| Sys[System APIs]
     Server -->|WebSocket| Debug[AppDebugKit]
+    Server -->|ADB Protocol| Android[Android Device]
 
     subgraph "Your Machine"
         Sys -->|Screen/OCR| macOS[CoreGraphics / Vision]
@@ -193,6 +270,12 @@ graph TD
         Sys -->|Text Search| UIA[UI Automation]
         Debug -.->|Inspect| App[Target App]
     end
+
+    subgraph "Android Device (USB/Wi-Fi)"
+        Android -->|screencap| Screen[Screenshots]
+        Android -->|input| Input[Tap / Swipe / Type]
+        Android -->|uiautomator| UITree[UI Hierarchy]
+    end
 ```
 
 <details>
@@ -208,6 +291,10 @@ graph TD
 | | Input | `SendInput` (Win32) |
 | | Text Search (`find_text`) | `UI Automation` (primary), WinRT OCR (fallback) |
 | | OCR | `Windows.Media.Ocr` (WinRT) |
+| **Android** | Screenshots | `screencap` / ADB framebuffer |
+| | Input | `adb shell input` (tap, swipe, text, keyevent) |
+| | Text Search (`find_text`) | `uiautomator dump` (accessibility tree) |
+| | Device Communication | `adb_client` crate (native Rust ADB protocol) |
 
 ### Screenshot Coordinate Precision
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "native-devtools-mcp",
-  "version": "0.3.6",
-  "description": "MCP server for computer-use / desktop automation of native apps (screenshots, OCR, input)",
+  "version": "0.4.0",
+  "description": "MCP server for native desktop app testing — screenshot, OCR, click, type, find_text, template matching. macOS & Windows.",
   "license": "MIT",
   "repository": {
     "type": "git",
@@ -10,19 +10,28 @@
   "homepage": "https://github.com/sh3ll3x3c/native-devtools-mcp",
   "keywords": [
     "mcp",
+    "mcp-server",
     "model-context-protocol",
     "computer-use",
     "desktop-automation",
     "ui-automation",
+    "native-app",
+    "test-automation",
+    "e2e-testing",
     "rpa",
     "ocr",
     "screenshot",
+    "template-matching",
+    "accessibility",
+    "find-text",
     "screen-reading",
     "mouse",
     "keyboard",
     "ai-agent",
     "llm",
     "claude",
+    "claude-code",
+    "cursor",
     "gemini",
     "gpt",
     "devtools",
@@ -39,8 +48,8 @@
     "bin"
   ],
   "optionalDependencies": {
-    "@sh3ll3x3c/native-devtools-mcp-darwin-arm64": "0.3.6",
-    "@sh3ll3x3c/native-devtools-mcp-win32-x64": "0.3.6"
+    "@sh3ll3x3c/native-devtools-mcp-darwin-arm64": "0.4.0",
+    "@sh3ll3x3c/native-devtools-mcp-win32-x64": "0.4.0"
   },
   "engines": {
     "node": ">=18"