minitap-mcp 0.5.2__tar.gz → 0.6.0__tar.gz

{minitap_mcp-0.5.2 → minitap_mcp-0.6.0}/PKG-INFO

@@ -1,17 +1,18 @@
 Metadata-Version: 2.3
 Name: minitap-mcp
-Version: 0.5.2
+Version: 0.6.0
 Summary: Model Context Protocol server for controlling Android & iOS devices with natural language
 Author: Pierre-Louis Favreau, Jean-Pierre Lo, Clément Guiguet
 Requires-Dist: fastmcp>=2.12.4
 Requires-Dist: python-dotenv>=1.1.1
 Requires-Dist: pydantic>=2.12.0
 Requires-Dist: pydantic-settings>=2.10.1
-Requires-Dist: minitap-mobile-use>=2.9.2
+Requires-Dist: minitap-mobile-use>=3.0.0
 Requires-Dist: jinja2>=3.1.6
 Requires-Dist: langchain-core>=0.3.75
 Requires-Dist: pillow>=11.1.0
 Requires-Dist: structlog>=24.4.0
+Requires-Dist: aiohttp>=3.9.0
 Requires-Dist: ruff==0.5.3 ; extra == 'dev'
 Requires-Dist: pytest==8.4.1 ; extra == 'dev'
 Requires-Dist: pytest-cov==5.0.0 ; extra == 'dev'
@@ -40,11 +41,19 @@ Before running the MCP server, ensure you have the required mobile automation to
 - **For Android devices:**
 
   - [ADB (Android Debug Bridge)](https://developer.android.com/tools/adb) - For device communication
-  - [Maestro](https://maestro.mobile.dev/) - For mobile automation
 
 - **For iOS devices (macOS only):**
   - Xcode Command Line Tools with `xcrun`
-  - [Maestro](https://maestro.mobile.dev/) - For mobile automation
+  - **[fb-idb](https://fbidb.io/docs/installation/)**: Facebook's iOS Development Bridge for device automation.
+
+```bash
+# Install via Homebrew (macOS)
+brew tap facebook/fb
+brew install idb-companion
+```
+
+> [!NOTE]
+> `idb_companion` is required to communicate with iOS simulators. Make sure it's in your PATH after installation.
 
 For detailed setup instructions, see the [mobile-use repository](https://github.com/minitap-ai/mobile-use).
 
@@ -244,6 +253,50 @@ The tool will:
 3. Compare both screenshots using vision AI
 4. Return a detailed analysis highlighting differences
 
+## Cloud Mobile Mode
+
+Run the MCP server with cloud-hosted mobile devices instead of requiring a local device. This enables:
+
+- **Zero local setup**: No ADB or physical device required
+- **Remote development**: Control cloud mobiles from anywhere
+- **Scalable automation**: Access multiple cloud devices
+
+### Setting Up Cloud Mobile Mode
+
+1. **Create a Cloud Mobile** on [Minitap Platform](https://platform.minitap.ai/cloud-mobiles):
+   - Click **Create New Device**
+   - Choose platform (currently Android v11 / API level 30)
+   - Set a **Reference Name** (e.g., `my-dev-device`)
+
+2. **Configure the environment variable**:
+
+```bash
+# Using reference name (recommended)
+export CLOUD_MOBILE_NAME="my-dev-device"
+```
+
+3. **Start the server** (no local device needed):
+
+```bash
+minitap-mcp --server --api-key your_minitap_api_key
+```
+
+The server will:
+- Connect to your cloud mobile on startup
+- Maintain a keep-alive connection while running
+- Automatically disconnect when the server stops
+
+> ⚠️ **Important**: Cloud mobiles are billed while connected. The MCP server automatically stops the connection when you close your IDE or stop the server. Make sure to properly shut down the server to avoid unexpected charges.
+
+### Cloud vs Local Mode
+
+| Feature | Local Mode | Cloud Mode |
+|---------|------------|------------|
+| Device requirement | Physical/emulator | None |
+| Setup complexity | ADB setup required | Low (env var only) |
+| `CLOUD_MOBILE_NAME` | Not set | Set to device name/UUID |
+| Billing | None | Per-minute usage |
+
 ## Advanced Configuration
 
 ### Custom ADB Server

{minitap_mcp-0.5.2 → minitap_mcp-0.6.0}/PYPI_README.md

@@ -17,11 +17,19 @@ Before running the MCP server, ensure you have the required mobile automation to
 - **For Android devices:**
 
   - [ADB (Android Debug Bridge)](https://developer.android.com/tools/adb) - For device communication
-  - [Maestro](https://maestro.mobile.dev/) - For mobile automation
 
 - **For iOS devices (macOS only):**
   - Xcode Command Line Tools with `xcrun`
-  - [Maestro](https://maestro.mobile.dev/) - For mobile automation
+  - **[fb-idb](https://fbidb.io/docs/installation/)**: Facebook's iOS Development Bridge for device automation.
+
+```bash
+# Install via Homebrew (macOS)
+brew tap facebook/fb
+brew install idb-companion
+```
+
+> [!NOTE]
+> `idb_companion` is required to communicate with iOS simulators. Make sure it's in your PATH after installation.
 
 For detailed setup instructions, see the [mobile-use repository](https://github.com/minitap-ai/mobile-use).
 
@@ -221,6 +229,50 @@ The tool will:
 3. Compare both screenshots using vision AI
 4. Return a detailed analysis highlighting differences
 
+## Cloud Mobile Mode
+
+Run the MCP server with cloud-hosted mobile devices instead of requiring a local device. This enables:
+
+- **Zero local setup**: No ADB or physical device required
+- **Remote development**: Control cloud mobiles from anywhere
+- **Scalable automation**: Access multiple cloud devices
+
+### Setting Up Cloud Mobile Mode
+
+1. **Create a Cloud Mobile** on [Minitap Platform](https://platform.minitap.ai/cloud-mobiles):
+   - Click **Create New Device**
+   - Choose platform (currently Android v11 / API level 30)
+   - Set a **Reference Name** (e.g., `my-dev-device`)
+
+2. **Configure the environment variable**:
+
+```bash
+# Using reference name (recommended)
+export CLOUD_MOBILE_NAME="my-dev-device"
+```
+
+3. **Start the server** (no local device needed):
+
+```bash
+minitap-mcp --server --api-key your_minitap_api_key
+```
+
+The server will:
+- Connect to your cloud mobile on startup
+- Maintain a keep-alive connection while running
+- Automatically disconnect when the server stops
+
+> ⚠️ **Important**: Cloud mobiles are billed while connected. The MCP server automatically stops the connection when you close your IDE or stop the server. Make sure to properly shut down the server to avoid unexpected charges.
+
+### Cloud vs Local Mode
+
+| Feature | Local Mode | Cloud Mode |
+|---------|------------|------------|
+| Device requirement | Physical/emulator | None |
+| Setup complexity | ADB setup required | Low (env var only) |
+| `CLOUD_MOBILE_NAME` | Not set | Set to device name/UUID |
+| Billing | None | Per-minute usage |
+
 ## Advanced Configuration
 
 ### Custom ADB Server

{minitap_mcp-0.5.2 → minitap_mcp-0.6.0}/minitap/mcp/core/agents/compare_screenshots/agent.py

@@ -1,4 +1,5 @@
 import asyncio
+import base64
 from pathlib import Path
 from uuid import uuid4
 
@@ -9,6 +10,7 @@ from pydantic import BaseModel
 from minitap.mcp.core.device import capture_screenshot, find_mobile_device
 from minitap.mcp.core.llm import get_minitap_llm
 from minitap.mcp.core.utils.images import get_screenshot_message_for_llm
+from minitap.mcp.server.cloud_mobile import get_cloud_mobile_id, get_cloud_screenshot
 
 
 class CompareScreenshotsOutput(BaseModel):
@@ -23,6 +25,8 @@ async def compare_screenshots(
     """
     Compare screenshots and return the comparison text along with both screenshots.
 
+    Supports both local devices (Android/iOS) and cloud devices.
+
     Returns:
         CompareScreenshotsOutput
     """
@@ -30,8 +34,14 @@ async def compare_screenshots(
         Path(__file__).parent.joinpath("prompt.md").read_text(encoding="utf-8")
     ).render()
 
-    device = find_mobile_device()
-    current_screenshot = capture_screenshot(device)
+    cloud_mobile_id = get_cloud_mobile_id()
+
+    if cloud_mobile_id:
+        screenshot_bytes = await get_cloud_screenshot(cloud_mobile_id)
+        current_screenshot = base64.b64encode(screenshot_bytes).decode("utf-8")
+    else:
+        device = find_mobile_device()
+        current_screenshot = capture_screenshot(device)
 
     messages: list[BaseMessage] = [
         SystemMessage(content=system_message),

minitap_mcp-0.6.0/minitap/mcp/core/cloud_apk.py

@@ -0,0 +1,109 @@
+"""Cloud APK deployment utilities for uploading and installing APKs on cloud mobiles."""
+
+import uuid
+from pathlib import Path
+
+import httpx
+
+from minitap.mcp.core.config import settings
+
+
+async def upload_apk_to_cloud_mobile(apk_path: str) -> str:
+    """
+    Upload an APK file via Platform storage API to user storage bucket.
+
+    Args:
+        apk_path: Path to the APK file
+
+    Returns:
+        Filename to use with install-apk endpoint
+
+    Raises:
+        FileNotFoundError: If APK file doesn't exist
+        httpx.HTTPError: If upload fails
+        ValueError: If MINITAP_API_KEY or MINITAP_API_BASE_URL is not configured
+    """
+    if not settings.MINITAP_API_KEY:
+        raise ValueError("MINITAP_API_KEY is not configured")
+    if not settings.MINITAP_API_BASE_URL:
+        raise ValueError("MINITAP_API_BASE_URL is not configured")
+
+    apk_file = Path(apk_path)
+    if not apk_file.exists():
+        raise FileNotFoundError(f"APK file not found: {apk_path}")
+
+    filename = f"app_{uuid.uuid4().hex[:6]}.apk"
+    api_key = settings.MINITAP_API_KEY.get_secret_value()
+    api_base_url = settings.MINITAP_API_BASE_URL.rstrip("/")
+
+    async with httpx.AsyncClient(timeout=300.0) as client:
+        # Step 1: Get signed upload URL from storage API
+        response = await client.get(
+            f"{api_base_url}/storage/signed-upload",
+            headers={"Authorization": f"Bearer {api_key}"},
+            params={"filenames": filename},
+        )
+        response.raise_for_status()
+        upload_data = response.json()
+
+        # Extract the signed URL for our file
+        signed_urls = upload_data.get("signed_urls", {})
+        if filename not in signed_urls:
+            raise ValueError(f"No signed URL returned for {filename}")
+
+        signed_url = signed_urls[filename]
+
+        # Step 2: Upload APK to signed URL
+        with open(apk_file, "rb") as f:
+            upload_response = await client.put(
+                signed_url,
+                content=f.read(),
+                headers={"Content-Type": "application/vnd.android.package-archive"},
+            )
+            upload_response.raise_for_status()
+
+        # Step 3: Return filename for install-apk call
+        return filename
+
+
+async def install_apk_on_cloud_mobile(filename: str) -> None:
+    """
+    Install an APK on a cloud mobile device via mobile-manager API.
+
+    Args:
+        filename: Filename returned from upload_apk_to_cloud_mobile
+
+    Raises:
+        httpx.HTTPError: If installation fails
+        ValueError: If required config settings are not configured
+    """
+    if not settings.MINITAP_API_KEY:
+        raise ValueError("MINITAP_API_KEY is not configured")
+    if not settings.MINITAP_DAAS_API:
+        raise ValueError("MINITAP_DAAS_API is not configured")
+    if not settings.CLOUD_MOBILE_NAME:
+        raise ValueError("CLOUD_MOBILE_NAME is not configured")
+
+    api_key = settings.MINITAP_API_KEY.get_secret_value()
+    base_url = settings.MINITAP_DAAS_API
+    cloud_mobile_name = settings.CLOUD_MOBILE_NAME
+
+    async with httpx.AsyncClient(timeout=120.0) as client:
+        cloud_mobile_response = await client.get(
+            f"{base_url}/virtual-mobiles/{cloud_mobile_name}",
+            headers={"Authorization": f"Bearer {api_key}"},
+        )
+        cloud_mobile_response.raise_for_status()
+        response_data = cloud_mobile_response.json()
+        cloud_mobile_uuid = response_data.get("id")
+        if not cloud_mobile_uuid:
+            raise ValueError(f"Cloud mobile '{cloud_mobile_name}' response missing 'id' field")
+        response = await client.post(
+            f"{base_url}/virtual-mobiles/{cloud_mobile_uuid}/install-apk",
+            headers={
+                "Authorization": f"Bearer {api_key}",
+                "Content-Type": "application/json",
+            },
+            json={"filename": filename},
+        )
+        response.raise_for_status()

{minitap_mcp-0.5.2 → minitap_mcp-0.6.0}/minitap/mcp/core/config.py

@@ -16,6 +16,7 @@ class MCPSettings(BaseSettings):
     # Minitap API configuration
     MINITAP_API_KEY: SecretStr | None = Field(default=None)
     MINITAP_API_BASE_URL: str = Field(default="https://platform.minitap.ai/api/v1")
+    MINITAP_DAAS_API: str = Field(default="https://platform.minitap.ai/api/daas")
     OPEN_ROUTER_API_KEY: SecretStr | None = Field(default=None)
 
     VISION_MODEL: str = Field(default="qwen/qwen-2.5-vl-7b-instruct")
@@ -27,5 +28,11 @@ class MCPSettings(BaseSettings):
     MCP_SERVER_HOST: str = Field(default="0.0.0.0")
     MCP_SERVER_PORT: int = Field(default=8000)
 
+    # Cloud Mobile configuration
+    # When set, the MCP server runs in cloud mode connecting to a Minitap cloud mobile
+    # instead of requiring a local device. Value can be a device name.
+    # Create cloud mobiles at https://platform.minitap.ai/cloud-mobiles
+    CLOUD_MOBILE_NAME: str | None = Field(default=None)
+
 
 settings = MCPSettings()  # type: ignore

{minitap_mcp-0.5.2 → minitap_mcp-0.6.0}/minitap/mcp/core/sdk_agent.py

@@ -3,6 +3,8 @@ import os
 from minitap.mobile_use.sdk import Agent
 from minitap.mobile_use.sdk.builders import Builders
 
+from minitap.mcp.core.config import settings
+
 # Lazy-initialized singleton agent
 _agent: Agent | None = None
 
@@ -23,5 +25,11 @@ def get_mobile_use_agent() -> Agent:
                 raise ValueError(f"Invalid ADB server socket: {custom_adb_socket}")
             _, host, port = parts
             config = config.with_adb_server(host=host, port=int(port))
+
+        # Add cloud mobile configuration if set
+        if settings.CLOUD_MOBILE_NAME:
+            config = config.for_cloud_mobile(cloud_mobile_id_or_ref=settings.CLOUD_MOBILE_NAME)
+
         _agent = Agent(config=config.build())
+
     return _agent

minitap_mcp-0.6.0/minitap/mcp/main.py

@@ -0,0 +1,276 @@
+"""MCP server for mobile-use with screen analysis capabilities."""
+
+import argparse
+import os
+import sys
+import threading
+from collections.abc import AsyncIterator
+from contextlib import asynccontextmanager
+from dataclasses import dataclass
+
+# Fix Windows console encoding for Unicode characters (emojis in logs)
+if sys.platform == "win32":
+    if hasattr(sys.stdout, "reconfigure"):
+        sys.stdout.reconfigure(encoding="utf-8")  # type: ignore[attr-defined]
+    if hasattr(sys.stderr, "reconfigure"):
+        sys.stderr.reconfigure(encoding="utf-8")  # type: ignore[attr-defined]
+    os.environ["PYTHONIOENCODING"] = "utf-8"
+
+    try:
+        import colorama
+
+        colorama.init(strip=False, convert=True, wrap=True)
+    except ImportError:
+        pass
+
+
+from fastmcp import FastMCP  # noqa: E402
+from minitap.mobile_use.config import settings as sdk_settings
+
+from minitap.mcp.core.config import settings  # noqa: E402
+from minitap.mcp.core.device import (
+    DeviceInfo,  # noqa: E402
+    list_available_devices,
+)
+from minitap.mcp.core.logging_config import (
+    configure_logging,  # noqa: E402
+    get_logger,
+)
+from minitap.mcp.server.cloud_mobile import CloudMobileService
+from minitap.mcp.server.middleware import LocalDeviceHealthMiddleware
+from minitap.mcp.server.poller import device_health_poller
+
+configure_logging(log_level=os.getenv("LOG_LEVEL", "INFO"))
+
+
+def main() -> None:
+    """Main entry point for the MCP server."""
+
+    parser = argparse.ArgumentParser(description="Mobile Use MCP Server")
+    parser.add_argument(
+        "--api-key",
+        type=str,
+        required=False,
+        default=None,
+        help="Minitap API key for authentication",
+    )
+    parser.add_argument(
+        "--cloud-mobile-name",
+        type=str,
+        required=False,
+        default=None,
+        help="Name of the cloud mobile device to connect to (enables cloud mode)",
+    )
+    parser.add_argument("--llm-profile", type=str, required=False, default=None)
+    parser.add_argument(
+        "--server",
+        action="store_true",
+        help="Run as network server (uses MCP_SERVER_HOST and MCP_SERVER_PORT from env)",
+    )
+    parser.add_argument(
+        "--port",
+        type=int,
+        required=False,
+        default=None,
+        help="Port to run the server on (overrides MCP_SERVER_PORT env variable)",
+    )
+
+    args = parser.parse_args()
+
+    if args.api_key:
+        os.environ["MINITAP_API_KEY"] = args.api_key
+        settings.__init__()
+        sdk_settings.__init__()
+
+    if args.cloud_mobile_name:
+        os.environ["CLOUD_MOBILE_NAME"] = args.cloud_mobile_name
+        settings.__init__()
+        sdk_settings.__init__()
+
+    if args.llm_profile:
+        os.environ["MINITAP_LLM_PROFILE_NAME"] = args.llm_profile
+        settings.__init__()
+        sdk_settings.__init__()
+
+    if args.port:
+        os.environ["MCP_SERVER_PORT"] = str(args.port)
+        settings.__init__()
+        sdk_settings.__init__()
+
+    if not settings.MINITAP_API_KEY:
+        raise ValueError("Minitap API key is required to run the MCP")
+
+    # Run MCP server with optional host/port for remote access
+    if args.server:
+        logger.info(f"Starting MCP server on {settings.MCP_SERVER_HOST}:{settings.MCP_SERVER_PORT}")
+        run_mcp_server(
+            transport="http",
+            host=settings.MCP_SERVER_HOST,
+            port=settings.MCP_SERVER_PORT,
+        )
+    else:
+        logger.info("Starting MCP server in local mode")
+        run_mcp_server()
+
+
+logger = get_logger(__name__)
+
+
+@dataclass
+class MCPLifespanContext:
+    """Context for MCP server lifespan.
+
+    Stores references to services that need cleanup on shutdown.
+    """
+
+    cloud_mobile_service: CloudMobileService | None = None
+    local_poller_stop_event: threading.Event | None = None
+    local_poller_thread: threading.Thread | None = None
+
+
+@asynccontextmanager
+async def mcp_lifespan(server: FastMCP) -> AsyncIterator[MCPLifespanContext]:
+    """Lifespan context manager for MCP server.
+
+    Handles startup/shutdown for both cloud and local modes.
+
+    Cloud mode (CLOUD_MOBILE_NAME set):
+    - Connects to cloud mobile on startup
+    - Maintains keep-alive polling
+    - Disconnects on shutdown (critical for billing!)
+
+    Local mode (CLOUD_MOBILE_NAME not set):
+    - Starts device health poller
+    - Monitors local device connection
+
+    Args:
+        server: The FastMCP server instance.
+
+    Yields:
+        MCPLifespanContext with references to running services.
+
+    Raises:
+        RuntimeError: If cloud mobile connection fails
+            (crashes MCP to prevent false "connected" state).
+    """
+    from minitap.mcp.core.sdk_agent import get_mobile_use_agent  # noqa: E402
+
+    context = MCPLifespanContext()
+    api_key = settings.MINITAP_API_KEY.get_secret_value() if settings.MINITAP_API_KEY else None
+
+    # Check if running in cloud mode
+    if settings.CLOUD_MOBILE_NAME:
+        # ==================== CLOUD MODE ====================
+        logger.info(f"Starting MCP in CLOUD mode with mobile: {settings.CLOUD_MOBILE_NAME}")
+
+        if not api_key:
+            logger.error("MINITAP_API_KEY is required")
+            raise RuntimeError(
+                "MINITAP_API_KEY is required when CLOUD_MOBILE_NAME is set. "
+                "Please set your API key via --api-key or MINITAP_API_KEY environment variable."
+            )
+
+        # Create and connect cloud mobile service
+        cloud_service = CloudMobileService(
+            cloud_mobile_name=settings.CLOUD_MOBILE_NAME,
+            api_key=api_key,
+        )
+
+        try:
+            await cloud_service.connect()
+        except Exception as e:
+            # CRITICAL: If cloud mobile not found, crash the MCP!
+            # This prevents the IDE from showing the server as "connected"
+            # when it actually can't do anything useful.
+            logger.error(f"Failed to connect to cloud mobile: {e}")
+            raise RuntimeError(
+                f"Cloud mobile connection failed. The MCP server cannot start.\n{e}"
+            ) from e
+
+        context.cloud_mobile_service = cloud_service
+        logger.info("Cloud mobile connected, MCP server ready")
+
+    else:
+        # ==================== LOCAL MODE ====================
+        logger.info("Starting MCP in LOCAL mode (no CLOUD_MOBILE_NAME set)")
+
+        agent = get_mobile_use_agent()
+        server.add_middleware(LocalDeviceHealthMiddleware(agent))
+
+        # Start device health poller in background
+        stop_event = threading.Event()
+        poller_thread = threading.Thread(
+            target=device_health_poller,
+            args=(stop_event, agent),
+            daemon=True,
+        )
+        poller_thread.start()
+
+        context.local_poller_stop_event = stop_event
+        context.local_poller_thread = poller_thread
+        logger.info("Device health poller started")
+
+    try:
+        yield context
+    finally:
+        # ==================== SHUTDOWN ====================
+        logger.info("MCP server shutting down, cleaning up resources...")
+
+        if context.cloud_mobile_service:
+            # CRITICAL: Stop cloud mobile connection to stop billing!
+            logger.info("Disconnecting cloud mobile (stopping billing)...")
+            try:
+                await context.cloud_mobile_service.disconnect()
+                logger.info("Cloud mobile disconnected successfully")
+            except Exception as e:
+                logger.error(f"Error disconnecting cloud mobile: {e}")
+
+        if context.local_poller_stop_event and context.local_poller_thread:
+            # Stop local device health poller
+            logger.info("Stopping device health poller...")
+            context.local_poller_stop_event.set()
+            context.local_poller_thread.join(timeout=10.0)
+
+            if context.local_poller_thread.is_alive():
+                logger.warning("Device health poller thread did not stop gracefully")
+            else:
+                logger.info("Device health poller stopped successfully")
+
+        logger.info("MCP server shutdown complete")
+
+
+# Create MCP server with lifespan handler
+mcp = FastMCP(
+    name="mobile-use-mcp",
+    instructions="""
+        This server provides analysis tools for connected
+        mobile devices (iOS or Android).
+        Call get_available_devices() to list them.
+    """,
+    lifespan=mcp_lifespan,
+)
+
+from minitap.mcp.tools import analyze_screen  # noqa: E402, F401
+from minitap.mcp.tools import compare_screenshot_with_figma  # noqa: E402, F401
+from minitap.mcp.tools import execute_mobile_command  # noqa: E402, F401
+from minitap.mcp.tools import save_figma_assets  # noqa: E402, F401
+
+
+@mcp.resource("data://devices")
+def get_available_devices() -> list[DeviceInfo]:
+    """Provides a list of connected mobile devices (iOS or Android)."""
+    return list_available_devices()
+
+
+def run_mcp_server(**mcp_run_kwargs):
+    """Run the MCP server with proper exception handling.
+
+    This wraps mcp.run() with exception handling for clean shutdown.
+    """
+    try:
+        mcp.run(**mcp_run_kwargs)
+    except KeyboardInterrupt:
+        logger.info("Keyboard interrupt received, shutting down...")
+    except Exception as e:
+        logger.error(f"Error running MCP server: {e}")
+        raise