android-agent-bridge 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +230 -119
- package/package.json +1 -1
- package/server/adb.mjs +35 -2
- package/server/cli.mjs +78 -4
- package/server/mcp.mjs +2 -0
- package/server/prompts.mjs +68 -0
package/README.md
CHANGED
|
@@ -1,119 +1,230 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
**
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
<img src="https://raw.githubusercontent.com/Rudrasamadhiya/android-agent-bridge/main/docs/icon.svg" width="96" alt="Agent Bridge icon" />
|
|
4
|
+
|
|
5
|
+
# Agent Bridge
|
|
6
|
+
|
|
7
|
+
**Give your AI agent a real Android phone.**
|
|
8
|
+
|
|
9
|
+
Claude Code, Cursor, Antigravity or any MCP agent can see your phone's screen, tap, type, open apps, change settings and move files.<br/>
|
|
10
|
+
One `npx` command on the computer, one QR scan on the phone.
|
|
11
|
+
|
|
12
|
+
[](https://www.npmjs.com/package/android-agent-bridge)
|
|
13
|
+
[](https://www.npmjs.com/package/android-agent-bridge)
|
|
14
|
+
[](https://github.com/Rudrasamadhiya/android-agent-bridge/releases)
|
|
15
|
+
[](#works-on)
|
|
16
|
+
[](#for-agents)
|
|
17
|
+
[](LICENSE)
|
|
18
|
+
|
|
19
|
+
[**Website**](https://rudrasamadhiya.github.io/android-agent-bridge/) · [**Get started**](https://rudrasamadhiya.github.io/android-agent-bridge/start/) · [**Docs**](https://rudrasamadhiya.github.io/android-agent-bridge/docs/) · [**Install the app**](https://rudrasamadhiya.github.io/android-agent-bridge/download/#install)
|
|
20
|
+
|
|
21
|
+
<br/>
|
|
22
|
+
|
|
23
|
+
<img src="https://raw.githubusercontent.com/Rudrasamadhiya/android-agent-bridge/main/docs/screens/app-connect.webp" width="230" alt="Connect tab" />
|
|
24
|
+
<img src="https://raw.githubusercontent.com/Rudrasamadhiya/android-agent-bridge/main/docs/screens/uber-choose-trip.webp" width="230" alt="An agent choosing a ride in Uber" />
|
|
25
|
+
<img src="https://raw.githubusercontent.com/Rudrasamadhiya/android-agent-bridge/main/docs/screens/app-transfer.webp" width="230" alt="Transfer tab" />
|
|
26
|
+
|
|
27
|
+
</div>
|
|
28
|
+
|
|
29
|
+
<br/>
|
|
30
|
+
|
|
31
|
+
```text
|
|
32
|
+
┌──────────────┐ MCP / CLI ┌─────────────────────┐ Wi-Fi or USB ┌──────────────────┐
|
|
33
|
+
│ Claude Code, │ ────────────▶ │ android-agent-bridge │ ◀──────────────▶ │ Agent Bridge app │
|
|
34
|
+
│ Cursor, … │ │ (your computer) │ │ (your phone) │
|
|
35
|
+
└──────────────┘ └─────────────────────┘ └──────────────────┘
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
## Quick start
|
|
39
|
+
|
|
40
|
+
> You need an Android phone (5.0+), Node.js 20+ and, the first time only, a USB cable.
|
|
41
|
+
|
|
42
|
+
**1. Install the app and pair, over USB.** Turn on *USB debugging* (Settings → About phone → tap *Build number* 7 times → Developer options), plug in, then:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
npx android-agent-bridge usb
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
It installs the app, switches on its accessibility service, pairs and starts the bridge. If you don't have Android's adb tool, it downloads it from Google the first time.
|
|
49
|
+
|
|
50
|
+
> **No cable?** On Android 11+, turn on *Wireless debugging* in Developer options and run `npx android-agent-bridge wifi`, then scan the QR code from *Pair device with QR code*.
|
|
51
|
+
|
|
52
|
+
**2. Connect your agent.** For Claude Code:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
claude mcp add phone -- npx -y android-agent-bridge
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
**3. Ask it something.**
|
|
59
|
+
|
|
60
|
+
> *"Open YouTube on my phone and search for lofi music."*
|
|
61
|
+
|
|
62
|
+
That's it. From now on the phone also works over Wi-Fi: run `npx android-agent-bridge start` and it reconnects by itself.
|
|
63
|
+
|
|
64
|
+
<details>
|
|
65
|
+
<summary><b>Prefer Wi-Fi and a QR code?</b></summary>
|
|
66
|
+
|
|
67
|
+
<br/>
|
|
68
|
+
|
|
69
|
+
[Download the APK](https://github.com/Rudrasamadhiya/android-agent-bridge/releases/latest/download/agent-bridge.apk) and install it on the phone, then:
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
npx android-agent-bridge start
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
In the app: **Scan QR code** → point it at the code in your terminal (or the bigger one at `localhost:8350/pair`) → turn on the **accessibility service**. Phone and computer must be on the same Wi-Fi.
|
|
76
|
+
|
|
77
|
+
If Play Protect says *"App blocked to protect your device"*, install from your computer instead with `usb` or `wifi`. Play Protect blocks sideloaded accessibility apps opened from a browser or chat app; installs through adb aren't affected.
|
|
78
|
+
|
|
79
|
+
</details>
|
|
80
|
+
|
|
81
|
+
<details>
|
|
82
|
+
<summary><b>Cursor, Antigravity, Windsurf and other MCP clients</b></summary>
|
|
83
|
+
|
|
84
|
+
<br/>
|
|
85
|
+
|
|
86
|
+
Run `npx android-agent-bridge init` in your project (it writes `.mcp.json`, plus `.cursor/mcp.json` with `--cursor`), or add this to your client's MCP config:
|
|
87
|
+
|
|
88
|
+
```json
|
|
89
|
+
{
|
|
90
|
+
"mcpServers": {
|
|
91
|
+
"phone": { "command": "npx", "args": ["-y", "android-agent-bridge"] }
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
</details>
|
|
97
|
+
|
|
98
|
+
## What it can do
|
|
99
|
+
|
|
100
|
+
| | |
|
|
101
|
+
|---|---|
|
|
102
|
+
| 👁 **Sees the screen** | A numbered list of every button and field, and screenshots with each element boxed. Knows what's hidden under sheets and dialogs. |
|
|
103
|
+
| 🔤 **Reads what apps hide** | On-device OCR reads the pixels when an app hides its controls from accessibility, so "tap Account" still works. Nothing leaves the phone. |
|
|
104
|
+
| 👆 **Acts like a person** | Tap by number, text or position. Type into fields, scroll, swipe, long-press, Back and Home. Open any app by name. |
|
|
105
|
+
| 🎛 **Runs the device** | Volume, brightness, flashlight, rotation, Do Not Disturb, settings pages, links, dialer, battery, notifications, clipboard. |
|
|
106
|
+
| 📁 **AirDrop, both ways** | `phonectl send photo.jpg`, drag files onto `localhost:8350/drop`, or Share → Agent Bridge on the phone. Any size, no cloud. |
|
|
107
|
+
| 📱 **Old phones welcome** | Android 5.0 (2015) and up. Where old Android can't do something, the computer does it over USB. |
|
|
108
|
+
|
|
109
|
+
## Try these prompts
|
|
110
|
+
|
|
111
|
+
Once the phone is connected, paste any of these into your agent:
|
|
112
|
+
|
|
113
|
+
- *"Is my phone connected? Take a screenshot and tell me what's on screen."*
|
|
114
|
+
- *"Read my phone's notifications and summarise them by app. Don't reply to anything."*
|
|
115
|
+
- *"Turn media volume to 30%, brightness to 60 and switch on Do Not Disturb."*
|
|
116
|
+
- *"Open my app (com.example.app) and test the sign-up flow like a QA tester. Give me a bug list."*
|
|
117
|
+
- *"Send ./report.pdf to my phone and tell me where it was saved."*
|
|
118
|
+
|
|
119
|
+
More in the [prompt library](https://rudrasamadhiya.github.io/android-agent-bridge/start/#prompts). Agents are told to ask before sending, posting, paying or deleting, and never to type passwords.
|
|
120
|
+
|
|
121
|
+
## For agents
|
|
122
|
+
|
|
123
|
+
The MCP server gives your agent **27 tools** and **6 ready-made prompts**. In Claude Code the prompts are slash commands:
|
|
124
|
+
|
|
125
|
+
| Slash command | What it does |
|
|
126
|
+
|---|---|
|
|
127
|
+
| `/mcp__phone__phone_tour` | Checks the phone is connected and describes the screen |
|
|
128
|
+
| `/mcp__phone__do_in_app` | Opens an app and carries out a task step by step |
|
|
129
|
+
| `/mcp__phone__test_my_app` | QA walkthrough of an app, with a bug report |
|
|
130
|
+
| `/mcp__phone__catch_up` | Summarises notifications, urgent first |
|
|
131
|
+
| `/mcp__phone__set_up_phone` | Volume, brightness, Do Not Disturb in one go |
|
|
132
|
+
| `/mcp__phone__send_to_phone` | Sends files and confirms where they landed |
|
|
133
|
+
|
|
134
|
+
<details>
|
|
135
|
+
<summary><b>All 27 tools</b></summary>
|
|
136
|
+
|
|
137
|
+
<br/>
|
|
138
|
+
|
|
139
|
+
| MCP tool | CLI | What it does |
|
|
140
|
+
|---|---|---|
|
|
141
|
+
| `phone_status` | `status` | Connected phone, Android version, capabilities |
|
|
142
|
+
| `phone_screen` | `screen [--mode marks\|grid\|both\|plain]` | Screenshot with numbered elements or a lettered grid |
|
|
143
|
+
| `phone_elements` | `elements [--all] [--ocr]` | Element list only (cheap text) |
|
|
144
|
+
| `phone_ocr` | `ocr` | Text read from the screen pixels, with positions |
|
|
145
|
+
| `phone_tap` | `tap <target> [--long]` | Tap or long-press |
|
|
146
|
+
| `phone_type` | `type <text> [--into t] [--submit]` | Type into a field |
|
|
147
|
+
| `phone_scroll` | `scroll <dir> [--in t]` | Scroll the screen or a list |
|
|
148
|
+
| `phone_swipe` | `swipe <from> <to>` | Drag between two points |
|
|
149
|
+
| `phone_key` | `key <name>` | back, home, recents, notifications, quick_settings, enter, lock |
|
|
150
|
+
| `phone_open_app` | `launch <app> [--store]` | Open an app by name, or its Play Store page |
|
|
151
|
+
| `phone_apps` | `apps [filter]` | Installed apps |
|
|
152
|
+
| `phone_wait_for` | `wait <text>` | Wait for text to appear |
|
|
153
|
+
| `phone_volume` | `volume [stream] [level]` | Media, ring, notification, alarm, call volume |
|
|
154
|
+
| `phone_brightness` | `brightness [0-100\|auto]` | Screen brightness |
|
|
155
|
+
| `phone_toggle` | `flashlight` · `rotation` · `dnd` · `ringer` | Quick switches |
|
|
156
|
+
| `phone_settings` | `settings <page>` | Wi-Fi, Bluetooth, display, battery… or an app's info page |
|
|
157
|
+
| `phone_open_link` | `url <link>` | Web pages and deep links |
|
|
158
|
+
| `phone_dial` | `dial <number>` | Dialer with the number filled in (never calls) |
|
|
159
|
+
| `phone_web_search` | `search <query>` | Web search on the phone |
|
|
160
|
+
| `phone_device_info` | `battery` · `current` | Battery, app in front |
|
|
161
|
+
| `phone_notifications` | `notifications` | Notifications since the service started |
|
|
162
|
+
| `phone_clipboard` | `clipboard [set <text>]` | Read or set the clipboard |
|
|
163
|
+
| `phone_wake` | `wake` | Turn the screen on (stays PIN-locked) |
|
|
164
|
+
| `phone_permissions` | `permissions [grant]` | Optional permissions, switched on over USB |
|
|
165
|
+
| `phone_send_files` | `send <files…>` | Computer → phone |
|
|
166
|
+
| `phone_received_files` | `received` | Files the phone sent to the computer |
|
|
167
|
+
| `phone_stop_app` | `stop <package>` | Force-stop an app (USB) |
|
|
168
|
+
|
|
169
|
+
A **target** is an element number (`12`), visible text (`Sign in`), a grid cell (`C7`) or pixels (`540,1200`).
|
|
170
|
+
|
|
171
|
+
</details>
|
|
172
|
+
|
|
173
|
+
## From the terminal
|
|
174
|
+
|
|
175
|
+
Everything an agent can do, you can do too. `npm i -g android-agent-bridge` gives you the `phonectl` command; keep `start` running in another terminal.
|
|
176
|
+
|
|
177
|
+
```bash
|
|
178
|
+
phonectl status # is the phone connected?
|
|
179
|
+
phonectl launch YouTube # open an app
|
|
180
|
+
phonectl screen --out shot.jpg # screenshot with numbered elements
|
|
181
|
+
phonectl tap "Search" # tap by text, number, grid cell or pixels
|
|
182
|
+
phonectl type "lofi music" --submit # type and press Enter
|
|
183
|
+
phonectl volume 40% # media volume
|
|
184
|
+
phonectl send report.pdf # send a file to the phone
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
Full reference: [Docs → Commands](https://rudrasamadhiya.github.io/android-agent-bridge/docs/#commands).
|
|
188
|
+
|
|
189
|
+
## Works on
|
|
190
|
+
|
|
191
|
+
| | Android 5–6 | Android 7–10 | Android 11+ |
|
|
192
|
+
|---|---|---|---|
|
|
193
|
+
| Read the screen, type, tap elements, Back/Home | ✅ | ✅ | ✅ |
|
|
194
|
+
| Tap anywhere (grid, pixels), swipes | over USB | ✅ | ✅ |
|
|
195
|
+
| Screenshots | one-time prompt, or USB | one-time prompt, or USB | ✅ built in |
|
|
196
|
+
|
|
197
|
+
Apps that block screen capture (banking, streaming) show up black in screenshots; their element list usually still works. iPhone isn't supported: iOS has no equivalent of Android's accessibility service for apps.
|
|
198
|
+
|
|
199
|
+
## Security
|
|
200
|
+
|
|
201
|
+
- **Paired, not public.** Only a computer holding the secret from your QR code can connect. `npx android-agent-bridge pair --reset` makes a new one.
|
|
202
|
+
- **Stays on your network.** Wi-Fi or USB straight to your computer. No cloud, no account, no analytics. Wi-Fi traffic isn't encrypted, so use networks you trust; over USB it never leaves the cable.
|
|
203
|
+
- **One switch to stop.** Turn off the accessibility service and access ends at once.
|
|
204
|
+
- **Agents ask first.** The bundled guide tells agents to ask before sending, paying, posting or deleting, and never to type passwords.
|
|
205
|
+
|
|
206
|
+
<details>
|
|
207
|
+
<summary><b>Troubleshooting</b></summary>
|
|
208
|
+
|
|
209
|
+
<br/>
|
|
210
|
+
|
|
211
|
+
- **"App blocked to protect your device":** install from your computer with `npx android-agent-bridge usb`, or `npx android-agent-bridge wifi` (Android 11+, no cable).
|
|
212
|
+
- **No phone connected:** is `start` running? Same Wi-Fi? On Windows, allow Node.js through the firewall. Over USB, run `usb` again.
|
|
213
|
+
- **Accessibility switch greyed out** (Android 13+): App info → ⋮ → Allow restricted settings.
|
|
214
|
+
- **A tap opened nothing right after Home:** Android holds app starts for about 5 seconds after Home; run it again.
|
|
215
|
+
|
|
216
|
+
</details>
|
|
217
|
+
|
|
218
|
+
## Build from source
|
|
219
|
+
|
|
220
|
+
```bash
|
|
221
|
+
git clone https://github.com/Rudrasamadhiya/android-agent-bridge
|
|
222
|
+
cd android-agent-bridge && npm install
|
|
223
|
+
cd android && ./gradlew assembleRelease # JDK 17 + Android SDK
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
Copy `android/app/build/outputs/apk/release/app-release.apk` to `apk/agent-bridge.apk`. The website lives in `website/` (Next.js); `npm run build` there publishes it to `docs/`.
|
|
227
|
+
|
|
228
|
+
## License
|
|
229
|
+
|
|
230
|
+
[MIT](LICENSE) · Made by [Rudra Samadhiya](https://github.com/Rudrasamadhiya)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "android-agent-bridge",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.3",
|
|
4
4
|
"description": "Give AI agents (Claude Code, Cursor, Antigravity) your whole Android phone: screenshots with numbered elements and a tap grid, tap, type, scroll, open apps. Pair once by QR over Wi-Fi or plug in USB. Android 5.0+.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"android",
|
package/server/adb.mjs
CHANGED
|
@@ -2,9 +2,13 @@
|
|
|
2
2
|
// fallback for what the app cannot do on old Android (taps by coordinate
|
|
3
3
|
// before Android 7, Enter key before Android 11, screenshots without capture).
|
|
4
4
|
import { execFile } from 'node:child_process';
|
|
5
|
-
import { existsSync } from 'node:fs';
|
|
5
|
+
import { chmodSync, existsSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';
|
|
6
|
+
import os from 'node:os';
|
|
6
7
|
import path from 'node:path';
|
|
7
8
|
|
|
9
|
+
// Where platform-tools is downloaded when the computer has no Android SDK.
|
|
10
|
+
const OWN_TOOLS = path.join(os.homedir(), '.android-agent-bridge', 'platform-tools');
|
|
11
|
+
|
|
8
12
|
export const KEYCODES = {
|
|
9
13
|
back: 4, home: 3, recents: 187, enter: 66, tab: 61, delete: 67, escape: 111,
|
|
10
14
|
volume_up: 24, volume_down: 25, mute: 164, power: 26, menu: 82, search: 84,
|
|
@@ -18,13 +22,42 @@ export function adbPath() {
|
|
|
18
22
|
process.env.HOME && path.join(process.env.HOME, 'Android', 'Sdk')]) {
|
|
19
23
|
if (root && existsSync(path.join(root, 'platform-tools', exe))) return path.join(root, 'platform-tools', exe);
|
|
20
24
|
}
|
|
25
|
+
if (existsSync(path.join(OWN_TOOLS, exe))) return path.join(OWN_TOOLS, exe);
|
|
21
26
|
return 'adb';
|
|
22
27
|
}
|
|
23
28
|
|
|
29
|
+
/**
|
|
30
|
+
* Makes sure adb exists. Without an Android SDK or adb on PATH, downloads Google's
|
|
31
|
+
* platform-tools (about 7 MB) into ~/.android-agent-bridge once.
|
|
32
|
+
*/
|
|
33
|
+
export async function ensureAdb(log = console.log) {
|
|
34
|
+
try { await adb(['version'], { serial: null }); return; } catch (e) { if (!/adb not found/.test(e.message)) return; }
|
|
35
|
+
const plat = process.platform === 'win32' ? 'windows' : process.platform === 'darwin' ? 'darwin' : 'linux';
|
|
36
|
+
const url = `https://dl.google.com/android/repository/platform-tools-latest-${plat}.zip`;
|
|
37
|
+
log('adb (Android platform-tools) not found; downloading it from Google, one time only…');
|
|
38
|
+
const res = await fetch(url);
|
|
39
|
+
if (!res.ok) throw new Error(`Couldn't download platform-tools (${res.status}). Install it yourself: https://developer.android.com/tools/releases/platform-tools`);
|
|
40
|
+
const base = path.dirname(OWN_TOOLS);
|
|
41
|
+
mkdirSync(base, { recursive: true });
|
|
42
|
+
const zip = path.join(base, 'platform-tools.zip');
|
|
43
|
+
writeFileSync(zip, Buffer.from(await res.arrayBuffer()));
|
|
44
|
+
rmSync(OWN_TOOLS, { recursive: true, force: true });
|
|
45
|
+
await new Promise((resolve, reject) => {
|
|
46
|
+
const [bin, args] = process.platform === 'win32'
|
|
47
|
+
? ['powershell', ['-NoProfile', '-Command', `Expand-Archive -LiteralPath '${zip}' -DestinationPath '${base}' -Force`]]
|
|
48
|
+
: ['unzip', ['-q', '-o', zip, '-d', base]];
|
|
49
|
+
execFile(bin, args, { windowsHide: true, timeout: 120000 }, (err) => (err ? reject(new Error('Couldn\'t unpack platform-tools: ' + err.message)) : resolve()));
|
|
50
|
+
});
|
|
51
|
+
rmSync(zip, { force: true });
|
|
52
|
+
if (process.platform !== 'win32') chmodSync(path.join(OWN_TOOLS, 'adb'), 0o755);
|
|
53
|
+
log('✔ adb ready');
|
|
54
|
+
}
|
|
55
|
+
|
|
24
56
|
export function adb(args, { timeout = 15000, binary = false, serial = process.env.ANDROID_SERIAL } = {}) {
|
|
25
57
|
const full = serial ? ['-s', serial, ...args] : args;
|
|
26
58
|
return new Promise((resolve, reject) => {
|
|
27
|
-
|
|
59
|
+
// ADB_MDNS_OPENSCREEN lets adb find phones over Wi-Fi (wireless debugging) on every platform
|
|
60
|
+
execFile(adbPath(), full, { timeout, windowsHide: true, encoding: binary ? 'buffer' : 'utf8', maxBuffer: 64 * 1024 * 1024, env: { ...process.env, ADB_MDNS_OPENSCREEN: '1' } }, (err, stdout, stderr) => {
|
|
28
61
|
if (err) {
|
|
29
62
|
reject(new Error(err.code === 'ENOENT'
|
|
30
63
|
? 'adb not found. Install Android platform-tools (https://developer.android.com/tools/releases/platform-tools) or set ANDROID_HOME.'
|
package/server/cli.mjs
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
// (no command) MCP server, launched by AI agents
|
|
4
4
|
// start run the bridge and show the pairing QR code
|
|
5
5
|
// usb set up a USB-connected phone (install, permissions, pair)
|
|
6
|
+
// wifi same, without a cable (Android 11+ wireless debugging)
|
|
6
7
|
// init add the MCP server + agent guide to the current project
|
|
7
8
|
// status | screen | elements | tap | type | scroll | swipe | key | open | apps | wait
|
|
8
9
|
import fs from 'node:fs';
|
|
@@ -11,7 +12,8 @@ import { fileURLToPath } from 'node:url';
|
|
|
11
12
|
import { Bridge } from './bridge.mjs';
|
|
12
13
|
import { loadConfig, resetToken, APP_ID } from './config.mjs';
|
|
13
14
|
import { lanAddresses, pairLink, qrTerminal } from './hub.mjs';
|
|
14
|
-
import { adb, devices, pickSerial } from './adb.mjs';
|
|
15
|
+
import { adb, devices, ensureAdb, pickSerial } from './adb.mjs';
|
|
16
|
+
import readline from 'node:readline/promises';
|
|
15
17
|
import { fmtSize } from './files.mjs';
|
|
16
18
|
|
|
17
19
|
const here = path.dirname(fileURLToPath(import.meta.url));
|
|
@@ -37,6 +39,7 @@ const HELP = `android-agent-bridge — let AI agents use your Android phone
|
|
|
37
39
|
Setup
|
|
38
40
|
start run the bridge and show the QR code to pair over Wi-Fi
|
|
39
41
|
usb set up a phone plugged in over USB (install app, permissions, pair)
|
|
42
|
+
wifi [--code] same without a cable: Android 11+ wireless debugging, pair by QR (--code: pairing code)
|
|
40
43
|
init add the MCP server and agent guide to this project
|
|
41
44
|
pair [--reset] show the QR code again (--reset: new secret, phones must re-scan)
|
|
42
45
|
(no command) MCP server (what Claude Code / Cursor / Antigravity launch)
|
|
@@ -256,12 +259,82 @@ function findApk() {
|
|
|
256
259
|
}
|
|
257
260
|
|
|
258
261
|
async function usb() {
|
|
259
|
-
|
|
262
|
+
await ensureAdb().catch((e) => die(e.message));
|
|
260
263
|
const list = await devices().catch((e) => die(e.message));
|
|
261
264
|
if (!list.length) {
|
|
262
|
-
die('No phone found over USB.\n1. Settings → About phone → tap "Build number" 7 times (developer mode).\n2. Settings → Developer options → turn on USB debugging.\n3. Plug the phone in and accept the "Allow USB debugging?" prompt
|
|
265
|
+
die('No phone found over USB.\n1. Settings → About phone → tap "Build number" 7 times (developer mode).\n2. Settings → Developer options → turn on USB debugging.\n3. Plug the phone in and accept the "Allow USB debugging?" prompt.\n\nNo cable? Android 11+ can do this over Wi-Fi: npx android-agent-bridge wifi');
|
|
266
|
+
}
|
|
267
|
+
await setUp(await pickSerial());
|
|
268
|
+
}
|
|
269
|
+
|
|
270
|
+
const wait = (ms) => new Promise((r) => setTimeout(r, ms));
|
|
271
|
+
|
|
272
|
+
/** Wireless-debugging services the phone announces on the network: [{ name, type, addr }]. */
|
|
273
|
+
async function mdnsServices() {
|
|
274
|
+
const out = await adb(['mdns', 'services'], { serial: null }).catch(() => '');
|
|
275
|
+
return out.split('\n').map((l) => l.trim().split(/\s+/)).filter((p) => p.length >= 3 && p[1].startsWith('_adb-tls'))
|
|
276
|
+
.map(([name, type, addr]) => ({ name, type, addr }));
|
|
277
|
+
}
|
|
278
|
+
|
|
279
|
+
/**
|
|
280
|
+
* Same as `usb`, but without a cable: Android 11+ "Wireless debugging". The phone scans a QR code
|
|
281
|
+
* from this terminal (or types a pairing code), then adb installs and sets up the app over Wi-Fi.
|
|
282
|
+
* Installs through adb are not blocked by Play Protect.
|
|
283
|
+
*/
|
|
284
|
+
async function wifi() {
|
|
285
|
+
const { flags } = parseArgs(rest);
|
|
286
|
+
await ensureAdb().catch((e) => die(e.message));
|
|
287
|
+
console.log('\nInstall Agent Bridge over Wi-Fi (Android 11 or newer, phone on the same Wi-Fi as this computer)\n');
|
|
288
|
+
console.log('On the phone:');
|
|
289
|
+
console.log(' 1. Settings → About phone → tap "Build number" 7 times (once, turns on developer options)');
|
|
290
|
+
console.log(' 2. Settings → Developer options → Wireless debugging → turn it on');
|
|
291
|
+
|
|
292
|
+
let paired = false;
|
|
293
|
+
if (!flags.code) {
|
|
294
|
+
const name = 'agentbridge-' + Math.random().toString(36).slice(2, 7);
|
|
295
|
+
const pass = String(Math.floor(100000 + Math.random() * 900000));
|
|
296
|
+
console.log(' 3. Tap "Pair device with QR code" and scan this:\n');
|
|
297
|
+
console.log(await qrTerminal(`WIFI:T:ADB;S:${name};P:${pass};;`));
|
|
298
|
+
console.log('Waiting for the phone… (no QR option? Ctrl+C and run: npx android-agent-bridge wifi --code)');
|
|
299
|
+
for (let i = 0; i < 180 && !paired; i++) {
|
|
300
|
+
const svc = (await mdnsServices()).find((s) => s.name === name && s.type.startsWith('_adb-tls-pairing'));
|
|
301
|
+
if (svc) {
|
|
302
|
+
const out = await adb(['pair', svc.addr, pass], { serial: null, timeout: 30000 }).catch((e) => e.message);
|
|
303
|
+
if (!/Successfully paired/i.test(out)) die('Pairing failed: ' + out);
|
|
304
|
+
paired = true;
|
|
305
|
+
} else await wait(1000);
|
|
306
|
+
}
|
|
307
|
+
if (!paired) die('The phone did not show up. Check both are on the same Wi-Fi, or try: npx android-agent-bridge wifi --code');
|
|
308
|
+
} else {
|
|
309
|
+
console.log(' 3. Tap "Pair device with pairing code"\n');
|
|
310
|
+
const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
|
|
311
|
+
const addr = (await rl.question('IP address & port shown in that dialog (e.g. 192.168.1.5:37000): ')).trim();
|
|
312
|
+
const code = (await rl.question('Wi-Fi pairing code: ')).trim();
|
|
313
|
+
rl.close();
|
|
314
|
+
const out = await adb(['pair', addr, code], { serial: null, timeout: 30000 }).catch((e) => e.message);
|
|
315
|
+
if (!/Successfully paired/i.test(out)) die('Pairing failed: ' + out);
|
|
316
|
+
}
|
|
317
|
+
console.log('✔ Paired with the phone for wireless debugging');
|
|
318
|
+
|
|
319
|
+
// after pairing the phone announces a connect service; adb usually connects to it by itself
|
|
320
|
+
let serial = null;
|
|
321
|
+
for (let i = 0; i < 30 && !serial; i++) {
|
|
322
|
+
const list = await devices().catch(() => []);
|
|
323
|
+
const wireless = list.find((d) => d.serial.includes('_adb-tls-connect') || /:\d+$/.test(d.serial));
|
|
324
|
+
if (wireless) { serial = wireless.serial; break; }
|
|
325
|
+
for (const s of (await mdnsServices()).filter((x) => x.type.startsWith('_adb-tls-connect'))) {
|
|
326
|
+
await adb(['connect', s.addr], { serial: null }).catch(() => {});
|
|
327
|
+
}
|
|
328
|
+
await wait(1000);
|
|
263
329
|
}
|
|
264
|
-
|
|
330
|
+
if (!serial) die('Paired, but could not connect. On the phone, check Wireless debugging is still on and run this again.');
|
|
331
|
+
console.log('✔ Connected over Wi-Fi');
|
|
332
|
+
await setUp(serial);
|
|
333
|
+
}
|
|
334
|
+
|
|
335
|
+
/** Installs the app if needed, switches on its accessibility service, pairs it and runs the bridge. */
|
|
336
|
+
async function setUp(serial) {
|
|
337
|
+
const { flags } = parseArgs(rest);
|
|
265
338
|
const sh = (...c) => adb(['shell', ...c], { serial });
|
|
266
339
|
const cfg = loadConfig();
|
|
267
340
|
const sdk = Number(await sh('getprop', 'ro.build.version.sdk'));
|
|
@@ -348,6 +421,7 @@ try {
|
|
|
348
421
|
} else if (cmd === 'help' || cmd === '--help' || cmd === '-h') console.log(HELP);
|
|
349
422
|
else if (cmd === 'start') await start();
|
|
350
423
|
else if (cmd === 'usb') await usb();
|
|
424
|
+
else if (cmd === 'wifi') await wifi();
|
|
351
425
|
else if (cmd === 'init') init();
|
|
352
426
|
else if (cmd === 'pair') {
|
|
353
427
|
const cfg = parseArgs(rest).flags.reset ? resetToken() : loadConfig();
|
package/server/mcp.mjs
CHANGED
|
@@ -6,6 +6,7 @@ import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
|
|
|
6
6
|
import { z } from 'zod';
|
|
7
7
|
import { createRequire } from 'node:module';
|
|
8
8
|
import { Bridge } from './bridge.mjs';
|
|
9
|
+
import { registerPrompts } from './prompts.mjs';
|
|
9
10
|
|
|
10
11
|
const { version: VERSION } = createRequire(import.meta.url)('../package.json');
|
|
11
12
|
const log = (...a) => process.stderr.write('[android-agent-bridge] ' + a.join(' ') + '\n');
|
|
@@ -226,6 +227,7 @@ export async function startServer() {
|
|
|
226
227
|
const bridge = new Bridge({ log, host: true });
|
|
227
228
|
const server = new McpServer({ name: 'android-agent-bridge', version: VERSION });
|
|
228
229
|
registerTools(server, bridge);
|
|
230
|
+
registerPrompts(server);
|
|
229
231
|
await server.connect(new StdioServerTransport());
|
|
230
232
|
bridge.ensure().then(async () => {
|
|
231
233
|
if (bridge.hub) {
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
// Ready-made prompts the MCP server offers. Clients show them as one-click starters
|
|
2
|
+
// (Claude Code: /mcp__phone__<name>). Each one sets the agent up to work safely.
|
|
3
|
+
import { z } from 'zod';
|
|
4
|
+
|
|
5
|
+
const SAFETY = 'Before sending a message, posting, paying, deleting or changing account settings, stop and ask me. Never type passwords or card numbers; ask me to do that on the phone.';
|
|
6
|
+
|
|
7
|
+
const msg = (text) => ({ messages: [{ role: 'user', content: { type: 'text', text } }] });
|
|
8
|
+
|
|
9
|
+
export const PROMPTS = [
|
|
10
|
+
{
|
|
11
|
+
name: 'phone_tour',
|
|
12
|
+
title: 'Check the phone',
|
|
13
|
+
description: 'Confirm the phone is connected and describe what is on screen right now.',
|
|
14
|
+
args: {},
|
|
15
|
+
text: () => `Use the phone tools. Call phone_status to check my Android phone is connected (if it isn't, tell me how to pair it). Then take a phone_screen and tell me in two or three sentences which app is open and what I could do from here.`,
|
|
16
|
+
},
|
|
17
|
+
{
|
|
18
|
+
name: 'do_in_app',
|
|
19
|
+
title: 'Do something in an app',
|
|
20
|
+
description: 'Open an app on the phone and carry out a task in it, step by step.',
|
|
21
|
+
args: {
|
|
22
|
+
app: z.string().describe('App name, e.g. "YouTube" or "Settings"'),
|
|
23
|
+
task: z.string().describe('What to do, e.g. "play the latest video from MKBHD"'),
|
|
24
|
+
},
|
|
25
|
+
text: ({ app, task }) => `On my Android phone, open ${app} and ${task}. Work step by step: after each action, read the updated element list and decide the next step; take a phone_screen only when you need to see images or layout. ${SAFETY} When you're done, tell me what you did and what's on screen now.`,
|
|
26
|
+
},
|
|
27
|
+
{
|
|
28
|
+
name: 'test_my_app',
|
|
29
|
+
title: 'Test my app',
|
|
30
|
+
description: 'Walk through an app like a QA tester and report anything broken or confusing.',
|
|
31
|
+
args: {
|
|
32
|
+
app: z.string().describe('App name or package, e.g. "com.example.shop"'),
|
|
33
|
+
focus: z.string().optional().describe('Optional: a flow to focus on, e.g. "sign-up and checkout"'),
|
|
34
|
+
},
|
|
35
|
+
text: ({ app, focus }) => `Test ${app} on my Android phone like a careful QA tester${focus ? `, focusing on ${focus}` : ''}. Open it, go through its main screens and flows, and try edge cases (empty fields, back button, rotating, long text). Take a phone_screen of anything that looks wrong. ${SAFETY} Finish with a short report: what works, bugs found (with steps to reproduce), and UX issues.`,
|
|
36
|
+
},
|
|
37
|
+
{
|
|
38
|
+
name: 'catch_up',
|
|
39
|
+
title: 'Catch me up',
|
|
40
|
+
description: 'Summarise the notifications that arrived on the phone.',
|
|
41
|
+
args: {},
|
|
42
|
+
text: () => `Read the notifications on my Android phone with phone_notifications and give me a short summary grouped by app, with anything that looks urgent first. Don't open or reply to anything.`,
|
|
43
|
+
},
|
|
44
|
+
{
|
|
45
|
+
name: 'set_up_phone',
|
|
46
|
+
title: 'Set the phone up',
|
|
47
|
+
description: 'Change volume, brightness, Do Not Disturb and other quick settings in one go.',
|
|
48
|
+
args: {
|
|
49
|
+
settings: z.string().describe('What you want, e.g. "media volume 30%, brightness 60, Do Not Disturb on"'),
|
|
50
|
+
},
|
|
51
|
+
text: ({ settings }) => `On my Android phone, set: ${settings}. Use phone_volume, phone_brightness and phone_toggle where they fit; fall back to phone_settings and the screen only if needed. Then report the final values.`,
|
|
52
|
+
},
|
|
53
|
+
{
|
|
54
|
+
name: 'send_to_phone',
|
|
55
|
+
title: 'Send files to the phone',
|
|
56
|
+
description: 'Send files from this computer to the phone and confirm where they landed.',
|
|
57
|
+
args: {
|
|
58
|
+
files: z.string().describe('File paths on this computer, separated by commas'),
|
|
59
|
+
},
|
|
60
|
+
text: ({ files }) => `Send these files from this computer to my Android phone with phone_send_files: ${files}. Then tell me where each one was saved on the phone.`,
|
|
61
|
+
},
|
|
62
|
+
];
|
|
63
|
+
|
|
64
|
+
export function registerPrompts(server) {
|
|
65
|
+
for (const p of PROMPTS) {
|
|
66
|
+
server.registerPrompt(p.name, { title: p.title, description: p.description, argsSchema: p.args }, (args) => msg(p.text(args || {})));
|
|
67
|
+
}
|
|
68
|
+
}
|