@mobileaidev/ai-app-bridge 0.2.10 → 0.2.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +105 -89
- package/bin/ai-app-bridge.js +4558 -4504
- package/bin/ios-provider.js +1183 -0
- package/bin/mcp-server.js +1153 -1053
- package/package.json +4 -2
- package/skills/ai-app-bridge-use/SKILL.md +38 -2
- package/skills/ai-app-bridge-use/agents/openai.yaml +2 -2
package/README.md
CHANGED
|
@@ -1,71 +1,77 @@
|
|
|
1
|
-
# AI App Bridge CLI
|
|
2
|
-
|
|
3
|
-
```bash
|
|
4
|
-
npm install -g @mobileaidev/ai-app-bridge
|
|
5
|
-
|
|
6
|
-
ai-app-bridge status --package-name io.github.mobileaidev.aiappbridge.sample
|
|
7
|
-
ai-app-bridge tree --package-name io.github.mobileaidev.aiappbridge.sample
|
|
8
|
-
ai-app-bridge install-apk --package-name io.github.mobileaidev.aiappbridge.sample --apk-path app-debug.apk
|
|
9
|
-
ai-app-bridge clear-app-data --package-name io.github.mobileaidev.aiappbridge.sample
|
|
10
|
-
ai-app-bridge launch-app --package-name io.github.mobileaidev.aiappbridge.sample
|
|
11
|
-
ai-app-bridge launch-activity --package-name io.github.mobileaidev.aiappbridge.sample --activity .MainActivity --extra route=/home
|
|
12
|
-
ai-app-bridge screenshot --package-name io.github.mobileaidev.aiappbridge.sample
|
|
13
|
-
ai-app-bridge input-text --package-name io.github.mobileaidev.aiappbridge.sample --text "中文输入" --hide-keyboard
|
|
14
|
-
ai-app-bridge network --package-name io.github.mobileaidev.aiappbridge.sample --compact --url-filter /api/
|
|
15
|
-
ai-app-bridge webview-network --package-name io.github.mobileaidev.aiappbridge.sample --duration-ms 3000
|
|
16
|
-
ai-app-bridge
|
|
17
|
-
ai-app-bridge
|
|
18
|
-
ai-app-bridge-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
and
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
}
|
|
67
|
-
|
|
68
|
-
|
|
1
|
+
# AI App Bridge CLI
|
|
2
|
+
|
|
3
|
+
```bash
|
|
4
|
+
npm install -g @mobileaidev/ai-app-bridge
|
|
5
|
+
|
|
6
|
+
ai-app-bridge status --package-name io.github.mobileaidev.aiappbridge.sample
|
|
7
|
+
ai-app-bridge tree --package-name io.github.mobileaidev.aiappbridge.sample
|
|
8
|
+
ai-app-bridge install-apk --package-name io.github.mobileaidev.aiappbridge.sample --apk-path app-debug.apk
|
|
9
|
+
ai-app-bridge clear-app-data --package-name io.github.mobileaidev.aiappbridge.sample
|
|
10
|
+
ai-app-bridge launch-app --package-name io.github.mobileaidev.aiappbridge.sample
|
|
11
|
+
ai-app-bridge launch-activity --package-name io.github.mobileaidev.aiappbridge.sample --activity .MainActivity --extra route=/home
|
|
12
|
+
ai-app-bridge screenshot --package-name io.github.mobileaidev.aiappbridge.sample
|
|
13
|
+
ai-app-bridge input-text --package-name io.github.mobileaidev.aiappbridge.sample --text "中文输入" --hide-keyboard
|
|
14
|
+
ai-app-bridge network --package-name io.github.mobileaidev.aiappbridge.sample --compact --url-filter /api/
|
|
15
|
+
ai-app-bridge webview-network --package-name io.github.mobileaidev.aiappbridge.sample --duration-ms 3000
|
|
16
|
+
ai-app-bridge ios-devices
|
|
17
|
+
ai-app-bridge ios-doctor --device-id <device-or-udid> --bundle-id <ios.bundle.id>
|
|
18
|
+
ai-app-bridge ios-setup --device-id <device-or-udid> --bundle-id <ios.bundle.id> --team-id <APPLE_TEAM_ID> --start-wda
|
|
19
|
+
ai-app-bridge ios-status --device-id <device-or-udid> --bundle-id <ios.bundle.id>
|
|
20
|
+
ai-app-bridge ios-tap --bundle-id <ios.bundle.id> --tap-x 120 --tap-y 360 --wda-url <wda-url-from-setup>
|
|
21
|
+
ai-app-bridge ios-input --bundle-id <ios.bundle.id> --accessibility-id sample_text_field --clear-first --text "hello" --wda-url <wda-url-from-setup>
|
|
22
|
+
ai-app-bridge thaw-app --package-name io.github.mobileaidev.aiappbridge.sample
|
|
23
|
+
ai-app-bridge freeze-app --package-name io.github.mobileaidev.aiappbridge.sample
|
|
24
|
+
ai-app-bridge-mcp
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
AI agents, model clients, automation frameworks, and IDE MCP clients reading
|
|
28
|
+
this README should load `skills/ai-app-bridge-use/SKILL.md` before operating
|
|
29
|
+
apps through MCP. If the client supports skills or project instructions, load
|
|
30
|
+
that file as the dedicated usage guide. If it does not, include the file content
|
|
31
|
+
in the system/developer/project instructions for the task. Codex-compatible
|
|
32
|
+
clients can copy the repo folder `skills/ai-app-bridge-use` into their skills
|
|
33
|
+
directory, for example `$CODEX_HOME/skills/ai-app-bridge-use` on macOS/Linux or
|
|
34
|
+
`%USERPROFILE%\.codex\skills\ai-app-bridge-use` on Windows, then restart or
|
|
35
|
+
refresh the session.
|
|
36
|
+
|
|
37
|
+
MCP defaults to a compact tool surface to avoid loading every command schema
|
|
38
|
+
into the model context:
|
|
39
|
+
|
|
40
|
+
- `capabilities` lists the bridge domains and command names.
|
|
41
|
+
- `run` executes a selected command with command-specific arguments.
|
|
42
|
+
|
|
43
|
+
This keeps install, data reset, launch, UI, Flutter, WebView, logcat, network,
|
|
44
|
+
permission, iOS, and web capabilities discoverable without exposing dozens of full schemas
|
|
45
|
+
at session start. Set `AI_APP_BRIDGE_MCP_SURFACE=full` before launching
|
|
46
|
+
`ai-app-bridge-mcp` only when a client needs the legacy one-tool-per-command
|
|
47
|
+
surface.
|
|
48
|
+
|
|
49
|
+
The MCP server accepts both standard `Content-Length` framed JSON-RPC messages
|
|
50
|
+
and single-line JSON messages. Responses use the format of the first request on
|
|
51
|
+
that connection, so standard MCP clients keep framed responses while local
|
|
52
|
+
Node REPL scripts can send and read one JSON object per line.
|
|
53
|
+
|
|
54
|
+
For multi-step app automation, call `run` with `command: "batch"`. Batch steps
|
|
55
|
+
run serially in one MCP call, so a failed step can stop and mark the remaining
|
|
56
|
+
steps as skipped without mixing results from different commands:
|
|
57
|
+
|
|
58
|
+
```json
|
|
59
|
+
{
|
|
60
|
+
"command": "batch",
|
|
61
|
+
"arguments": {
|
|
62
|
+
"defaults": {
|
|
63
|
+
"packageName": "io.github.mobileaidev.aiappbridge.sample"
|
|
64
|
+
},
|
|
65
|
+
"steps": [
|
|
66
|
+
{ "id": "launch", "command": "launch-app" },
|
|
67
|
+
{ "id": "wait-home", "command": "wait-text", "arguments": { "targetText": "Home" } },
|
|
68
|
+
{ "id": "capture-logs", "command": "logs", "arguments": { "limit": 20 } }
|
|
69
|
+
],
|
|
70
|
+
"stopOnError": true
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
```
|
|
74
|
+
|
|
69
75
|
For dynamic or transient screens, MCP agents can use `freeze-app`/`thaw-app` as
|
|
70
76
|
an optional stabilization control: thaw before reads, actions, waits, or
|
|
71
77
|
captures; freeze after evidence capture only when a changing UI would make
|
|
@@ -75,24 +81,34 @@ flows usually do not need freezing.
|
|
|
75
81
|
For visible state changes such as panels, dialogs, page transitions, tabs, or
|
|
76
82
|
button-triggered content, verify with both `screenshot` and `tree`/`uia-tree`;
|
|
77
83
|
do not conclude success from UI tree alone.
|
|
78
|
-
|
|
79
|
-
WebView network and console capture use Android WebView DevTools/CDP when the
|
|
80
|
-
target app is debuggable and WebView debugging is enabled.
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
or
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
`
|
|
98
|
-
|
|
84
|
+
|
|
85
|
+
WebView network and console capture use Android WebView DevTools/CDP when the
|
|
86
|
+
target app is debuggable and WebView debugging is enabled.
|
|
87
|
+
|
|
88
|
+
iOS commands use Xcode `devicectl` for device/app/screenshot operations, the
|
|
89
|
+
AiAppBridgeIOS runtime for in-app evidence, and WebDriverAgent/XCUITest for
|
|
90
|
+
full-control taps, input, swipes, and external UI tree reads. `ios-setup`
|
|
91
|
+
can start the vendored `appium-webdriveragent` project when `--start-wda` and
|
|
92
|
+
`--team-id` are supplied. On physical devices, reuse the returned WDA URL for
|
|
93
|
+
later WDA commands; it may be a CoreDevice tunnel such as
|
|
94
|
+
`http://[fdxx::1]:8100`. It returns explicit blockers for Developer Mode,
|
|
95
|
+
device preparation, signing, or WDA reachability instead of silently
|
|
96
|
+
downgrading iOS capability.
|
|
97
|
+
|
|
98
|
+
`input-text` first uses the app bridge native text endpoint. This is required
|
|
99
|
+
for Chinese and other Unicode text because `adb shell input text` is ASCII-only
|
|
100
|
+
on many Android 16 devices; ASCII text can still fall back to ADB when an older
|
|
101
|
+
bridge runtime is running.
|
|
102
|
+
|
|
103
|
+
When `screenshot` or `smoke` runs without `--out-file`, the CLI writes a unique
|
|
104
|
+
PNG under `build/ai_app_bridge_artifacts` instead of reusing a stable filename
|
|
105
|
+
or creating files in the project root.
|
|
106
|
+
It keeps the newest 20 generated screenshots for each command prefix. Use
|
|
107
|
+
`--artifact-dir` to choose that directory, or `--out-file` when a fixed path is
|
|
108
|
+
intentional.
|
|
109
|
+
|
|
110
|
+
`launch-app` queries Android LAUNCHER activities before starting the app. If a
|
|
111
|
+
debug dependency exposes multiple launcher entries, it returns
|
|
112
|
+
`launcher_ambiguous` with the candidates instead of guessing. Use
|
|
113
|
+
`launch-activity` or `launch-app --activity/--component` to choose the intended
|
|
114
|
+
entry point explicitly.
|