minitap-mobile-use 0.0.1.dev0__tar.gz

minitap_mobile_use-0.0.1.dev0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Minitap, Inc
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

minitap_mobile_use-0.0.1.dev0/PKG-INFO

@@ -0,0 +1,274 @@
+Metadata-Version: 2.3
+Name: minitap-mobile-use
+Version: 0.0.1.dev0
+Summary: AI-powered multi-agent system that automates real Android and iOS devices through low-level control using LangGraph.
+Author: Pierre-Louis Favreau, Jean-Pierre Lo, Nicolas Dehandschoewercker
+License: MIT License
+         
+         Copyright (c) 2025 Minitap, Inc
+         
+         Permission is hereby granted, free of charge, to any person obtaining a copy
+         of this software and associated documentation files (the "Software"), to deal
+         in the Software without restriction, including without limitation the rights
+         to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+         copies of the Software, and to permit persons to whom the Software is
+         furnished to do so, subject to the following conditions:
+         
+         The above copyright notice and this permission notice shall be included in all
+         copies or substantial portions of the Software.
+         
+         THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+         IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+         FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+         AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+         LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+         OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+         SOFTWARE.
+Requires-Dist: langgraph==0.5.0
+Requires-Dist: adbutils==2.9.3
+Requires-Dist: langchain-google-genai==2.1.5
+Requires-Dist: langchain==0.3.26
+Requires-Dist: langchain-core==0.3.66
+Requires-Dist: jinja2==3.1.6
+Requires-Dist: python-dotenv==1.1.1
+Requires-Dist: pydantic-settings==2.10.1
+Requires-Dist: langchain-mcp-adapters==0.1.7
+Requires-Dist: langchain-openai==0.3.27
+Requires-Dist: typer==0.16.0
+Requires-Dist: langchain-cerebras>=0.5.0
+Requires-Dist: inquirer>=3.4.0
+Requires-Dist: sseclient-py==1.8.0
+Requires-Dist: fastapi==0.111.0
+Requires-Dist: uvicorn[standard]==0.30.1
+Requires-Dist: colorama>=0.4.6
+Requires-Dist: psutil>=5.9.0
+Requires-Dist: ruff==0.5.3 ; extra == 'dev'
+Requires-Dist: pytest==8.4.1 ; extra == 'dev'
+Requires-Dist: pytest-cov==5.0.0 ; extra == 'dev'
+Requires-Python: >=3.10
+Project-URL: Homepage, https://minitap.ai/
+Project-URL: Source, https://github.com/minitap-ai/mobile-use
+Provides-Extra: dev
+Description-Content-Type: text/markdown
+
+# mobile-use: automate your phone with natural language
+
+<div align="center">
+
+![mobile-use in Action](./doc/linkedin-demo-with-text.gif)
+
+</div>
+
+<div align="center">
+
+[![Discord](https://img.shields.io/discord/1403058278342201394?color=7289DA&label=Discord&logo=discord&logoColor=white&style=for-the-badge)](https://discord.gg/6nSqmQ9pQs)
+[![GitHub stars](https://img.shields.io/github/stars/minitap-ai/mobile-use?style=for-the-badge&color=e0a8dd)](https://github.com/minitap-ai/mobile-use/stargazers)
+
+<p align="center">
+    <a href="https://discord.gg/6nSqmQ9pQs"><b>Discord</b></a> •
+    <a href="https://x.com/minitap_ai?t=iRWtI497UhRGLeCKYQekig&s=09"><b>Twitter / X</b></a>
+</p>
+
+</div>
+
+Mobile-use is a powerful, open-source AI agent that controls your Android or IOS device using natural language. It understands your commands and interacts with the UI to perform tasks, from sending messages to navigating complex apps.
+
+> Mobile-use is quickly evolving. Your suggestions, ideas, and reported bugs will shape this project. Do not hesitate to join in the conversation on [Discord](https://discord.gg/6nSqmQ9pQs) or contribute directly, we will reply to everyone! ❤️
+
+## ✨ Features
+
+- 🗣️ **Natural Language Control**: Interact with your phone using your native language.
+- 📱 **UI-Aware Automation**: Intelligently navigates through app interfaces.
+- 📊 **Data Scraping**: Extract information from any app and structure it into your desired format (e.g., JSON) using a natural language description.
+- 🔧 **Extensible & Customizable**: Easily configure different LLMs to power the agents that power mobile-use.
+
+## Benchmarks
+
+<p align="center">
+  <img src="./doc/benchmark.jpg" alt="Project banner" />
+</p>
+
+We are global number 1 Opensource pass@1 on the AndroidWorld benchmark.
+
+More info here: https://minitap.ai/research/mobile-ai-agents-benchmark
+
+The official leaderboard is available [here](https://docs.google.com/spreadsheets/d/1cchzP9dlTZ3WXQTfYNhh3avxoLipqHN75v1Tb86uhHo/edit?pli=1&gid=0#gid=0)
+
+## 🚀 Getting Started
+
+Ready to automate your mobile experience? Follow these steps to get mobile-use up and running.
+
+1.  **Set up Environment Variables:**
+    Copy the example `.env.example` file to `.env` and add your API keys.
+
+    ```bash
+    cp .env.example .env
+    ```
+
+2.  **(Optional) Customize LLM Configuration:**
+    To use different models or providers, create your own LLM configuration file.
+    ```bash
+    cp llm-config.override.template.jsonc llm-config.override.jsonc
+    ```
+    Then, edit `llm-config.override.jsonc` to fit your needs.
+
+### Quick Launch (Docker)
+
+> [!NOTE]  
+> This quickstart, is only available for Android devices/emulators as of now, and you must have Docker installed.
+
+First:
+
+- Either plug your Android device and enable USB-debugging via the Developer Options
+- Or launch an Android emulator
+
+> [!IMPORTANT]  
+> At some point, the terminal will HANG, and Maestro will ask you `Maestro CLI would like to collect anonymous usage data to improve the product.`
+> It's up to you whether you accept (i.e enter 'Y') or not (i.e. enter 'n').
+
+Then run in your terminal:
+
+1. For Linux/macOS:
+
+```bash
+chmod +x mobile-use.sh
+./mobile-use.sh \
+  "Open Gmail, find first 3 unread emails, and list their sender and subject line" \
+  --output-description "A JSON list of objects, each with 'sender' and 'subject' keys"
+```
+
+2. For Windows (inside a Powershell terminal):
+
+```powershell
+powershell.exe -ExecutionPolicy Bypass -File mobile-use.ps1 `
+  "Open Gmail, find first 3 unread emails, and list their sender and subject line" `
+  --output-description "A JSON list of objects, each with 'sender' and 'subject' keys"
+```
+
+> [!NOTE]  
+> If using your own device, make sure to accept the ADB-related connection requests that will pop up on your device.
+> Similarly, Maestro will need to install its APK on your device, which will also require you to accept the installation request.
+
+#### 🧰 Troubleshooting
+
+The script will try to connect to your device via IP.  
+Therefore, your device **must be connected to the same Wi-Fi network as your computer**.
+
+##### 1. No device IP found
+
+If the script fails with the following message:
+
+```
+Could not get device IP. Is a device connected via USB and on the same Wi-Fi network?
+```
+
+Then it couldn't find one of the common Wi-Fi interfaces on your device.  
+Therefore, you must determine what WLAN interface your phone is using via `adb shell ip addr show up`.
+Then add the `--interface <YOUR_INTERFACE_NAME>` option to the script.
+
+##### 2. Failed to connect to <DEVICE_IP>:5555 inside Docker
+
+This is most probably an issue with your firewall blocking the connection. Therefore there is no clear fix for this.
+
+##### 3. Failed to pull GHCR docker images (unauthorized)
+
+Since UV docker images rely on a `ghcr.io` public repositories, you may have an expired token if you used `ghcr.io` before for private repositories.  
+Try running `docker logout ghcr.io` and then run the script again.
+
+### Manual Launch (Development Mode)
+
+For developers who want to set up the environment manually:
+
+#### 1. Device Support
+
+Mobile-use currently supports the following devices:
+
+- **Physical Android Phones**: Connect via USB with USB debugging enabled.
+- **Android Simulators**: Set up through Android Studio.
+- **iOS Simulators**: Supported for macOS users.
+
+> [!NOTE]  
+> Physical iOS devices are not yet supported.
+
+#### 2. Prerequisites
+
+For Android:
+
+- **[Android Debug Bridge (ADB)](https://developer.android.com/studio/releases/platform-tools)**: A tool to connect to your device.
+
+For iOS:
+
+- **[Xcode](https://developer.apple.com/xcode/)**: Apple's IDE for iOS development.
+
+Before you begin, ensure you have the following installed:
+
+- **[uv](https://github.com/astral-sh/uv)**: A lightning-fast Python package manager.
+- **[Maestro](https://maestro.mobile.dev/getting-started/installing-maestro)**: The framework we use to interact with your device.
+
+#### 3. Installation
+
+1.  **Clone the repository:**
+
+    ```bash
+    git clone https://github.com/minitap-ai/mobile-use.git && cd mobile-use
+    ```
+
+2.  [**Setup environment variables**](#-getting-started)
+
+3.  **Create & activate the virtual environment:**
+
+    ```bash
+    # This will create a .venv directory using the Python version in .python-version
+    uv venv
+
+    # Activate the environment
+    # On macOS/Linux:
+    source .venv/bin/activate
+    # On Windows:
+    .venv\Scripts\activate
+    ```
+
+4.  **Install dependencies:**
+    ```bash
+    # Sync with the locked dependencies for a consistent setup
+    uv sync
+    ```
+
+## 👨‍💻 Usage
+
+To run mobile-use, simply pass your command as an argument.
+
+**Example 1: Basic Command**
+
+```bash
+python ./src/mobile_use/main.py "Go to settings and tell me my current battery level"
+```
+
+**Example 2: Data Scraping**
+
+Extract specific information and get it back in a structured format. For instance, to get a list of your unread emails:
+
+```bash
+python ./src/mobile_use/main.py \
+  "Open Gmail, find all unread emails, and list their sender and subject line" \
+  --output-description "A JSON list of objects, each with 'sender' and 'subject' keys"
+```
+
+> [!NOTE]  
+> If you haven't configured a specific model, mobile-use will prompt you to choose one from the available options.
+
+## ❤️ Contributing
+
+We love contributions! Whether you're fixing a bug, adding a feature, or improving documentation, your help is welcome. Please read our **[Contributing Guidelines](CONTRIBUTING.md)** to get started.
+
+## ⭐ Star History
+
+<p align="center">
+  <a href="https://star-history.com/#minitap-ai/mobile-use&Date">
+    <img src="https://api.star-history.com/svg?repos=minitap-ai/mobile-use&type=Date" alt="Star History Chart" />
+  </a>
+</p>
+
+## 📜 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

minitap_mobile_use-0.0.1.dev0/README.md

@@ -0,0 +1,221 @@
+# mobile-use: automate your phone with natural language
+
+<div align="center">
+
+![mobile-use in Action](./doc/linkedin-demo-with-text.gif)
+
+</div>
+
+<div align="center">
+
+[![Discord](https://img.shields.io/discord/1403058278342201394?color=7289DA&label=Discord&logo=discord&logoColor=white&style=for-the-badge)](https://discord.gg/6nSqmQ9pQs)
+[![GitHub stars](https://img.shields.io/github/stars/minitap-ai/mobile-use?style=for-the-badge&color=e0a8dd)](https://github.com/minitap-ai/mobile-use/stargazers)
+
+<p align="center">
+    <a href="https://discord.gg/6nSqmQ9pQs"><b>Discord</b></a> •
+    <a href="https://x.com/minitap_ai?t=iRWtI497UhRGLeCKYQekig&s=09"><b>Twitter / X</b></a>
+</p>
+
+</div>
+
+Mobile-use is a powerful, open-source AI agent that controls your Android or IOS device using natural language. It understands your commands and interacts with the UI to perform tasks, from sending messages to navigating complex apps.
+
+> Mobile-use is quickly evolving. Your suggestions, ideas, and reported bugs will shape this project. Do not hesitate to join in the conversation on [Discord](https://discord.gg/6nSqmQ9pQs) or contribute directly, we will reply to everyone! ❤️
+
+## ✨ Features
+
+- 🗣️ **Natural Language Control**: Interact with your phone using your native language.
+- 📱 **UI-Aware Automation**: Intelligently navigates through app interfaces.
+- 📊 **Data Scraping**: Extract information from any app and structure it into your desired format (e.g., JSON) using a natural language description.
+- 🔧 **Extensible & Customizable**: Easily configure different LLMs to power the agents that power mobile-use.
+
+## Benchmarks
+
+<p align="center">
+  <img src="./doc/benchmark.jpg" alt="Project banner" />
+</p>
+
+We are global number 1 Opensource pass@1 on the AndroidWorld benchmark.
+
+More info here: https://minitap.ai/research/mobile-ai-agents-benchmark
+
+The official leaderboard is available [here](https://docs.google.com/spreadsheets/d/1cchzP9dlTZ3WXQTfYNhh3avxoLipqHN75v1Tb86uhHo/edit?pli=1&gid=0#gid=0)
+
+## 🚀 Getting Started
+
+Ready to automate your mobile experience? Follow these steps to get mobile-use up and running.
+
+1.  **Set up Environment Variables:**
+    Copy the example `.env.example` file to `.env` and add your API keys.
+
+    ```bash
+    cp .env.example .env
+    ```
+
+2.  **(Optional) Customize LLM Configuration:**
+    To use different models or providers, create your own LLM configuration file.
+    ```bash
+    cp llm-config.override.template.jsonc llm-config.override.jsonc
+    ```
+    Then, edit `llm-config.override.jsonc` to fit your needs.
+
+### Quick Launch (Docker)
+
+> [!NOTE]  
+> This quickstart, is only available for Android devices/emulators as of now, and you must have Docker installed.
+
+First:
+
+- Either plug your Android device and enable USB-debugging via the Developer Options
+- Or launch an Android emulator
+
+> [!IMPORTANT]  
+> At some point, the terminal will HANG, and Maestro will ask you `Maestro CLI would like to collect anonymous usage data to improve the product.`
+> It's up to you whether you accept (i.e enter 'Y') or not (i.e. enter 'n').
+
+Then run in your terminal:
+
+1. For Linux/macOS:
+
+```bash
+chmod +x mobile-use.sh
+./mobile-use.sh \
+  "Open Gmail, find first 3 unread emails, and list their sender and subject line" \
+  --output-description "A JSON list of objects, each with 'sender' and 'subject' keys"
+```
+
+2. For Windows (inside a Powershell terminal):
+
+```powershell
+powershell.exe -ExecutionPolicy Bypass -File mobile-use.ps1 `
+  "Open Gmail, find first 3 unread emails, and list their sender and subject line" `
+  --output-description "A JSON list of objects, each with 'sender' and 'subject' keys"
+```
+
+> [!NOTE]  
+> If using your own device, make sure to accept the ADB-related connection requests that will pop up on your device.
+> Similarly, Maestro will need to install its APK on your device, which will also require you to accept the installation request.
+
+#### 🧰 Troubleshooting
+
+The script will try to connect to your device via IP.  
+Therefore, your device **must be connected to the same Wi-Fi network as your computer**.
+
+##### 1. No device IP found
+
+If the script fails with the following message:
+
+```
+Could not get device IP. Is a device connected via USB and on the same Wi-Fi network?
+```
+
+Then it couldn't find one of the common Wi-Fi interfaces on your device.  
+Therefore, you must determine what WLAN interface your phone is using via `adb shell ip addr show up`.
+Then add the `--interface <YOUR_INTERFACE_NAME>` option to the script.
+
+##### 2. Failed to connect to <DEVICE_IP>:5555 inside Docker
+
+This is most probably an issue with your firewall blocking the connection. Therefore there is no clear fix for this.
+
+##### 3. Failed to pull GHCR docker images (unauthorized)
+
+Since UV docker images rely on a `ghcr.io` public repositories, you may have an expired token if you used `ghcr.io` before for private repositories.  
+Try running `docker logout ghcr.io` and then run the script again.
+
+### Manual Launch (Development Mode)
+
+For developers who want to set up the environment manually:
+
+#### 1. Device Support
+
+Mobile-use currently supports the following devices:
+
+- **Physical Android Phones**: Connect via USB with USB debugging enabled.
+- **Android Simulators**: Set up through Android Studio.
+- **iOS Simulators**: Supported for macOS users.
+
+> [!NOTE]  
+> Physical iOS devices are not yet supported.
+
+#### 2. Prerequisites
+
+For Android:
+
+- **[Android Debug Bridge (ADB)](https://developer.android.com/studio/releases/platform-tools)**: A tool to connect to your device.
+
+For iOS:
+
+- **[Xcode](https://developer.apple.com/xcode/)**: Apple's IDE for iOS development.
+
+Before you begin, ensure you have the following installed:
+
+- **[uv](https://github.com/astral-sh/uv)**: A lightning-fast Python package manager.
+- **[Maestro](https://maestro.mobile.dev/getting-started/installing-maestro)**: The framework we use to interact with your device.
+
+#### 3. Installation
+
+1.  **Clone the repository:**
+
+    ```bash
+    git clone https://github.com/minitap-ai/mobile-use.git && cd mobile-use
+    ```
+
+2.  [**Setup environment variables**](#-getting-started)
+
+3.  **Create & activate the virtual environment:**
+
+    ```bash
+    # This will create a .venv directory using the Python version in .python-version
+    uv venv
+
+    # Activate the environment
+    # On macOS/Linux:
+    source .venv/bin/activate
+    # On Windows:
+    .venv\Scripts\activate
+    ```
+
+4.  **Install dependencies:**
+    ```bash
+    # Sync with the locked dependencies for a consistent setup
+    uv sync
+    ```
+
+## 👨‍💻 Usage
+
+To run mobile-use, simply pass your command as an argument.
+
+**Example 1: Basic Command**
+
+```bash
+python ./src/mobile_use/main.py "Go to settings and tell me my current battery level"
+```
+
+**Example 2: Data Scraping**
+
+Extract specific information and get it back in a structured format. For instance, to get a list of your unread emails:
+
+```bash
+python ./src/mobile_use/main.py \
+  "Open Gmail, find all unread emails, and list their sender and subject line" \
+  --output-description "A JSON list of objects, each with 'sender' and 'subject' keys"
+```
+
+> [!NOTE]  
+> If you haven't configured a specific model, mobile-use will prompt you to choose one from the available options.
+
+## ❤️ Contributing
+
+We love contributions! Whether you're fixing a bug, adding a feature, or improving documentation, your help is welcome. Please read our **[Contributing Guidelines](CONTRIBUTING.md)** to get started.
+
+## ⭐ Star History
+
+<p align="center">
+  <a href="https://star-history.com/#minitap-ai/mobile-use&Date">
+    <img src="https://api.star-history.com/svg?repos=minitap-ai/mobile-use&type=Date" alt="Star History Chart" />
+  </a>
+</p>
+
+## 📜 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

minitap_mobile_use-0.0.1.dev0/minitap/mobile_use/agents/contextor/contextor.py

@@ -0,0 +1,42 @@
+from minitap.mobile_use.agents.executor.utils import is_last_tool_message_take_screenshot
+from minitap.mobile_use.controllers.mobile_command_controller import get_screen_data
+from minitap.mobile_use.controllers.platform_specific_commands_controller import (
+    get_device_date,
+    get_focused_app_info,
+)
+from minitap.mobile_use.graph.state import State
+from minitap.mobile_use.utils.decorators import wrap_with_callbacks
+from minitap.mobile_use.utils.logger import get_logger
+from minitap.mobile_use.context import MobileUseContext
+
+logger = get_logger(__name__)
+
+
+class ContextorNode:
+    def __init__(self, ctx: MobileUseContext):
+        self.ctx = ctx
+
+    @wrap_with_callbacks(
+        before=lambda: logger.info("Starting Contextor Agent"),
+        on_success=lambda _: logger.success("Contextor Agent"),
+        on_failure=lambda _: logger.error("Contextor Agent"),
+    )
+    def __call__(self, state: State):
+        device_data = get_screen_data(self.ctx.screen_api_client)
+        focused_app_info = get_focused_app_info(self.ctx)
+        device_date = get_device_date(self.ctx)
+
+        should_add_screenshot_context = is_last_tool_message_take_screenshot(list(state.messages))
+
+        return state.sanitize_update(
+            ctx=self.ctx,
+            update={
+                "latest_screenshot_base64": device_data.base64
+                if should_add_screenshot_context
+                else None,
+                "latest_ui_hierarchy": device_data.elements,
+                "focused_app_info": focused_app_info,
+                "screen_size": (device_data.width, device_data.height),
+                "device_date": device_date,
+            },
+        )

minitap_mobile_use-0.0.1.dev0/minitap/mobile_use/agents/cortex/cortex.md

@@ -0,0 +1,93 @@
+## You are the **Cortex**
+
+Your job is to **analyze the current {{ platform }} mobile device state** and produce **structured decisions** to achieve the current subgoal.
+
+You must act like a human brain, responsible for giving instructions to your hands (the **Executor** agent). Therefore, you must act with the same imprecision and uncertainty as a human when performing swipe actions: humans don't know where exactly they are swiping (always prefer percentages of width and height instead of absolute coordinates), they just know they are swiping up or down, left or right, and with how much force (usually amplified compared to what's truly needed - go overboard of sliders for instance).
+
+### Context You Receive:
+
+You are provided with:
+
+- 📱 **Device state**:
+
+  - Latest **UI hierarchy**
+  - (Optional) Latest **screenshot (base64)**. You can query one if you need it by calling the take_screenshot tool. Often, the UI hierarchy is enough to understand what is happening on the screen.
+  - Current **focused app info**
+  - **Screen size** and **device date**
+
+- 🧭 **Task context**:
+
+  - The user's **initial goal**
+  - The **subgoal plan** with their statuses
+  - The **current subgoal** to act on (the one in `PENDING` in the plan)
+  - A list of **agent thoughts** (previous reasoning, observations about the environment)
+  - **Executor agent feedback** on the latest UI decisions
+
+### Your Mission:
+
+Focus on the **current subgoal**.
+
+1. **Analyze the UI** and environment to understand what action is required.
+   2.1. If the **subgoal is completed**, set the `complete_subgoal` field to `True`. To justify your conclusion, you will fill in the `agent_thought` field based on:
+
+- The current UI state
+- Past agent thoughts
+- Recent tool effects
+  2.2. Otherwise, output a **stringified structured set of instructions** that an **Executor agent** can perform on a real mobile device:
+
+- These must be **concrete low-level actions**: back,tap, swipe, launch app, list packages, close app, input text, paste, erase, text, copy, etc.
+- If you refer to a UI element or coordinates, specify it clearly (e.g., `resource-id: com.whatsapp:id/search`, `text: "Alice"`, `x: 100, y: 200`).
+- **The structure is up to you**, but it must be valid **JSON stringified output**. You will accompany this output with a **natural-language summary** of your reasoning and approach in your agent thought.
+- When you want to launch/stop an app, prefer using its package name.
+- **Only reference UI element IDs or visible texts that are explicitly present in the provided UI hierarchy or screenshot. Do not invent, infer, or guess any IDs or texts that are not directly observed**.
+- **For text clearing**: When you need to completely clear text from an input field, always use **LONG PRESS** first to select the text field, then erase. Do NOT use tap + erase as this only clears from cursor position.
+
+### Output
+
+- **Structured Decisions**:
+  A **valid stringified JSON** describing what should be executed **right now** to advance the current subgoal **IF THE SUBGOAL IS NOT COMPLETED**.
+
+- **Agent Thought** _(1-2 sentences)_:
+  If there is any information you need to remember for later steps, you must include it here, because only the agent thoughts will be used to produce the final structured output.
+
+  This also helps other agents understand your decision and learn from future failures.
+  You must also use this field to mention checkpoints when you perform actions without definite ending: for instance "Swiping up to reveal more recipes - last seen recipe was <ID or NAME>, stop when no more".
+
+- **Subgoal Completion** _(boolean)_:
+  Set to true if the current subgoal has been successfully completed - you **cannot set it to true and provide structured decisions at the same time**. You must base your decision ONLY on what you have as input (device state, agent thoughts, executor feedback, etc) - NEVER based on the decisions you have produced.
+
+---
+
+### Example
+
+#### Current Subgoal:
+
+> "Search for Alice in WhatsApp"
+
+#### Structured Decisions:
+
+```text
+"{\"action\": \"tap\", \"target\": {\"resource_id\": \"com.whatsapp:id/menuitem_search\", \"text\": \"Search\"}}"
+```
+
+#### Agent Thought:
+
+> I will tap the search icon at the top of the WhatsApp interface to begin searching for Alice.
+
+### Input
+
+**Initial Goal:**
+{{ initial_goal }}
+
+**Subgoal Plan:**
+{{ subgoal_plan }}
+
+**Current Subgoal (what needs to be done right now):**
+{{ current_subgoal }}
+
+**Agent thoughts (previous reasoning, observations about the environment):**
+{{ agents_thoughts }}
+
+**Executor agent feedback on latest UI decisions:**
+
+{{ executor_feedback }}