native-devtools-mcp 0.5.0 → 0.5.1

package/README.md

@@ -1,22 +1,33 @@
 # native-devtools-mcp
 
-<div align="center">
+`native-devtools-mcp` is a Model Context Protocol (MCP) server for computer use on macOS, Windows, and Android. It gives AI agents and MCP clients direct control over native desktop apps and Android devices through screenshots, OCR, accessibility-based text lookup, input simulation, window management, and ADB.
 
-![Version](https://img.shields.io/npm/v/native-devtools-mcp?style=flat-square)
-![License](https://img.shields.io/npm/l/native-devtools-mcp?style=flat-square)
-![Platform](https://img.shields.io/badge/platform-macOS%20%7C%20Windows%20%7C%20Android-blue?style=flat-square)
-![Downloads](https://img.shields.io/npm/dt/native-devtools-mcp?style=flat-square)
+Use it when browser-only automation is not enough: Electron apps, system dialogs, desktop tools, native app testing, and Android device workflows. It works with [Claude Desktop](https://claude.ai/download), [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Cursor](https://cursor.com), and other MCP-compatible clients.
 
-**Give your AI agent "eyes" and "hands" for native desktop and mobile applications.**
+Useful for MCP-based computer use, desktop automation, UI automation, native app testing, e2e testing, RPA, screen reading, mouse and keyboard control, and Android device automation.
 
-A Model Context Protocol (MCP) server that provides **Computer Use** capabilities: screenshots, OCR, input simulation, and window management — for **native desktop apps** and **Android devices**, not just browsers.
+```bash
+npx -y native-devtools-mcp
+```
 
-**Works with:** [Claude Desktop](https://claude.ai/download) • [Claude Code](https://docs.anthropic.com/en/docs/claude-code) • [Cursor](https://cursor.com) • Any MCP-compatible client
+**Core capabilities**
+- Screenshots, OCR, and accessibility-first `find_text`
+- `click`, `type_text`, `scroll`, `launch_app`, `quit_app`, and window management
+- `element_at_point` for inspecting accessible UI elements at screen coordinates
+- `load_image` + `find_image` for non-text UI elements such as icons and custom controls
+- Android screenshots, text lookup, input, and app control over ADB
+- Local execution: screenshots and input stay on the machine
 
-[//]: # "Search keywords: MCP, MCP server, Model Context Protocol, computer use, desktop automation, UI automation, native app testing, test automation, e2e testing, RPA, screenshots, OCR, template matching, accessibility, mouse, keyboard, screen reading, macOS, Windows, Android, ADB, mobile testing, Claude, Claude Code, Cursor, AI agent, native-devtools-mcp"
+**For AI agents:** Read [`AGENTS.md`](./AGENTS.md) for tool definitions, workflow patterns, and machine-readable usage guidance.
 
-[Features](#-features) • [Installation](#-installation) • [Getting Started](#-getting-started) • [Security & Trust](#-security--trust) • [For AI Agents](#-for-ai-agents-llms) • [Android](#-android-support)
+![Version](https://img.shields.io/npm/v/native-devtools-mcp?style=flat-square)
+![License](https://img.shields.io/npm/l/native-devtools-mcp?style=flat-square)
+![Platform](https://img.shields.io/badge/platform-macOS%20%7C%20Windows%20%7C%20Android-blue?style=flat-square)
+![Downloads](https://img.shields.io/npm/dt/native-devtools-mcp?style=flat-square)
+
+[Features](#-features) • [Installation](#-installation) • [Getting Started](#-getting-started) • [Recipes](#-recipes-and-examples) • [Security & Trust](#-security--trust) • [For AI Agents](#-for-ai-agents-llms) • [Android](#-android-support)
 
+<div align="center">
 <table>
 <tr>
 <td align="center"><strong>macOS</strong></td>
@@ -27,7 +38,6 @@ A Model Context Protocol (MCP) server that provides **Computer Use** capabilitie
 <td><img src="windows-demo-1.gif" width="450" alt="Windows Demo"></td>
 </tr>
 </table>
-
 </div>
 
 ---
@@ -40,6 +50,7 @@ A Model Context Protocol (MCP) server that provides **Computer Use** capabilitie
 - **🧩 Template Matching:** Find non-text UI elements (icons, shapes) using `load_image` + `find_image`, returning precise click coordinates.
 - **🔒 Local & Private:** 100% local execution. No screenshots or data are ever sent to external servers.
 - **📱 Android Support:** Connect to Android devices over ADB for screenshots, input simulation, UI element search, and app management — all from the same MCP server.
+- **🔍 Hover Tracking:** Track cursor hover transitions across UI elements in real-time. Configurable dwell threshold filters pass-through noise — designed for LLMs observing user navigation patterns. macOS only.
 - **🔌 Dual-Mode Interaction:**
     1.  **Visual/Native:** Works with *any* app via screenshots & coordinates (Universal).
     2.  **AppDebugKit:** Deep integration for supported apps to inspect the UI tree (DOM-like structure).
@@ -56,6 +67,8 @@ This MCP server is designed to be **highly discoverable and usable** by AI model
 3.  `find_text`: A shortcut to find text on screen and get its coordinates immediately. Uses the platform **accessibility API** (macOS Accessibility / Windows UI Automation) for precise element-level matching, with OCR fallback.
 4.  `element_at_point`: Inspect the accessibility element at given screen coordinates — returns name, role, label, value, bounds, pid, and app_name. Note: privacy-focused Electron apps (e.g. Signal) may restrict their AX tree, returning only a container — use `take_screenshot` with OCR as a fallback.
 5.  `load_image` / `find_image`: Template matching for non-text UI elements (icons, shapes), returning screen coordinates for clicking.
+6.  `start_hover_tracking` / `get_hover_events` / `stop_hover_tracking`: Track cursor hover transitions across UI elements. Configurable dwell threshold filters pass-throughs. macOS only.
+7.  `launch_app` / `quit_app`: Launch apps with optional CLI args, or gracefully/forcefully quit them.
 
 ## 📦 Installation
 
@@ -119,6 +132,18 @@ Then restart your MCP client and you're ready to go.
 > { "permissions": { "allow": ["mcp__native-devtools__*"] } }
 > ```
 
+## 📚 Recipes and Examples
+
+- [Recipes and Examples Index](./examples/README.md)
+- [Claude Desktop Setup](./examples/claude-desktop-setup.md)
+- [Claude Code Setup](./examples/claude-code-setup.md)
+- [Cursor Setup](./examples/cursor-setup.md)
+- [End-to-End Desktop Flow](./examples/end-to-end-desktop-flow.md)
+- [Native App Click Flow](./examples/native-app-click-flow.md)
+- [OCR Fallback and Element Inspection](./examples/ocr-fallback-and-element-inspection.md)
+- [Template Matching Flow](./examples/template-matching-flow.md)
+- [Android Quickstart](./examples/android-quickstart.md)
+
 <details>
 <summary><strong>Manual configuration (without setup)</strong></summary>
 
@@ -319,6 +344,7 @@ graph TD
 | | Input | `CGEvent` (CoreGraphics) |
 | | Text Search (`find_text`) | `Accessibility API` (primary), Vision OCR (fallback) |
 | | Element Inspection (`element_at_point`) | `AXUIElementCopyElementAtPosition` + AX tree walk fallback (Accessibility API) |
+| | Hover Tracking (`start_hover_tracking`) | `CGEvent` cursor + Accessibility API polling (macOS only) |
 | | OCR | `VNRecognizeTextRequest` (Vision Framework) |
 | **Windows** | Screenshots | `BitBlt` (GDI) |
 | | Input | `SendInput` (Win32) |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "native-devtools-mcp",
-  "version": "0.5.0",
+  "version": "0.5.1",
   "mcpName": "io.github.sh3ll3x3c/native-devtools",
   "description": "MCP server for native app testing — screenshot, OCR, click, type, find_text, template matching. macOS, Windows & Android.",
   "license": "MIT",
@@ -53,8 +53,8 @@
     "bin"
   ],
   "optionalDependencies": {
-    "@sh3ll3x3c/native-devtools-mcp-darwin-arm64": "0.5.0",
-    "@sh3ll3x3c/native-devtools-mcp-win32-x64": "0.5.0"
+    "@sh3ll3x3c/native-devtools-mcp-darwin-arm64": "0.5.1",
+    "@sh3ll3x3c/native-devtools-mcp-win32-x64": "0.5.1"
   },
   "engines": {
     "node": ">=18"