@mobileaidev/ai-app-bridge 0.2.10 → 0.2.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. package/README.md +105 -89
  2. package/bin/ai-app-bridge.js +4558 -4504
  3. package/bin/ios-provider.js +1183 -0
  4. package/bin/mcp-server.js +1153 -1053
  5. package/package.json +4 -2
  6. package/skills/ai-app-bridge-use/SKILL.md +38 -2
  7. package/skills/ai-app-bridge-use/agents/openai.yaml +2 -2
package/README.md CHANGED
@@ -1,71 +1,77 @@
1
- # AI App Bridge CLI
2
-
3
- ```bash
4
- npm install -g @mobileaidev/ai-app-bridge
5
-
6
- ai-app-bridge status --package-name io.github.mobileaidev.aiappbridge.sample
7
- ai-app-bridge tree --package-name io.github.mobileaidev.aiappbridge.sample
8
- ai-app-bridge install-apk --package-name io.github.mobileaidev.aiappbridge.sample --apk-path app-debug.apk
9
- ai-app-bridge clear-app-data --package-name io.github.mobileaidev.aiappbridge.sample
10
- ai-app-bridge launch-app --package-name io.github.mobileaidev.aiappbridge.sample
11
- ai-app-bridge launch-activity --package-name io.github.mobileaidev.aiappbridge.sample --activity .MainActivity --extra route=/home
12
- ai-app-bridge screenshot --package-name io.github.mobileaidev.aiappbridge.sample
13
- ai-app-bridge input-text --package-name io.github.mobileaidev.aiappbridge.sample --text "中文输入" --hide-keyboard
14
- ai-app-bridge network --package-name io.github.mobileaidev.aiappbridge.sample --compact --url-filter /api/
15
- ai-app-bridge webview-network --package-name io.github.mobileaidev.aiappbridge.sample --duration-ms 3000
16
- ai-app-bridge thaw-app --package-name io.github.mobileaidev.aiappbridge.sample
17
- ai-app-bridge freeze-app --package-name io.github.mobileaidev.aiappbridge.sample
18
- ai-app-bridge-mcp
19
- ```
20
-
21
- AI agents, model clients, automation frameworks, and IDE MCP clients reading
22
- this README should load `skills/ai-app-bridge-use/SKILL.md` before operating
23
- apps through MCP. If the client supports skills or project instructions, load
24
- that file as the dedicated usage guide. If it does not, include the file content
25
- in the system/developer/project instructions for the task. Codex-compatible
26
- clients can copy the repo folder `skills/ai-app-bridge-use` into their skills
27
- directory, for example `$CODEX_HOME/skills/ai-app-bridge-use` on macOS/Linux or
28
- `%USERPROFILE%\.codex\skills\ai-app-bridge-use` on Windows, then restart or
29
- refresh the session.
30
-
31
- MCP defaults to a compact tool surface to avoid loading every command schema
32
- into the model context:
33
-
34
- - `capabilities` lists the bridge domains and command names.
35
- - `run` executes a selected command with command-specific arguments.
36
-
37
- This keeps install, data reset, launch, UI, Flutter, WebView, logcat, network,
38
- and permission capabilities discoverable without exposing dozens of full schemas
39
- at session start. Set `AI_APP_BRIDGE_MCP_SURFACE=full` before launching
40
- `ai-app-bridge-mcp` only when a client needs the legacy one-tool-per-command
41
- surface.
42
-
43
- The MCP server accepts both standard `Content-Length` framed JSON-RPC messages
44
- and single-line JSON messages. Responses use the format of the first request on
45
- that connection, so standard MCP clients keep framed responses while local
46
- Node REPL scripts can send and read one JSON object per line.
47
-
48
- For multi-step app automation, call `run` with `command: "batch"`. Batch steps
49
- run serially in one MCP call, so a failed step can stop and mark the remaining
50
- steps as skipped without mixing results from different commands:
51
-
52
- ```json
53
- {
54
- "command": "batch",
55
- "arguments": {
56
- "defaults": {
57
- "packageName": "io.github.mobileaidev.aiappbridge.sample"
58
- },
59
- "steps": [
60
- { "id": "launch", "command": "launch-app" },
61
- { "id": "wait-home", "command": "wait-text", "arguments": { "targetText": "Home" } },
62
- { "id": "capture-logs", "command": "logs", "arguments": { "limit": 20 } }
63
- ],
64
- "stopOnError": true
65
- }
66
- }
67
- ```
68
-
1
+ # AI App Bridge CLI
2
+
3
+ ```bash
4
+ npm install -g @mobileaidev/ai-app-bridge
5
+
6
+ ai-app-bridge status --package-name io.github.mobileaidev.aiappbridge.sample
7
+ ai-app-bridge tree --package-name io.github.mobileaidev.aiappbridge.sample
8
+ ai-app-bridge install-apk --package-name io.github.mobileaidev.aiappbridge.sample --apk-path app-debug.apk
9
+ ai-app-bridge clear-app-data --package-name io.github.mobileaidev.aiappbridge.sample
10
+ ai-app-bridge launch-app --package-name io.github.mobileaidev.aiappbridge.sample
11
+ ai-app-bridge launch-activity --package-name io.github.mobileaidev.aiappbridge.sample --activity .MainActivity --extra route=/home
12
+ ai-app-bridge screenshot --package-name io.github.mobileaidev.aiappbridge.sample
13
+ ai-app-bridge input-text --package-name io.github.mobileaidev.aiappbridge.sample --text "中文输入" --hide-keyboard
14
+ ai-app-bridge network --package-name io.github.mobileaidev.aiappbridge.sample --compact --url-filter /api/
15
+ ai-app-bridge webview-network --package-name io.github.mobileaidev.aiappbridge.sample --duration-ms 3000
16
+ ai-app-bridge ios-devices
17
+ ai-app-bridge ios-doctor --device-id <device-or-udid> --bundle-id <ios.bundle.id>
18
+ ai-app-bridge ios-setup --device-id <device-or-udid> --bundle-id <ios.bundle.id> --team-id <APPLE_TEAM_ID> --start-wda
19
+ ai-app-bridge ios-status --device-id <device-or-udid> --bundle-id <ios.bundle.id>
20
+ ai-app-bridge ios-tap --bundle-id <ios.bundle.id> --tap-x 120 --tap-y 360 --wda-url <wda-url-from-setup>
21
+ ai-app-bridge ios-input --bundle-id <ios.bundle.id> --accessibility-id sample_text_field --clear-first --text "hello" --wda-url <wda-url-from-setup>
22
+ ai-app-bridge thaw-app --package-name io.github.mobileaidev.aiappbridge.sample
23
+ ai-app-bridge freeze-app --package-name io.github.mobileaidev.aiappbridge.sample
24
+ ai-app-bridge-mcp
25
+ ```
26
+
27
+ AI agents, model clients, automation frameworks, and IDE MCP clients reading
28
+ this README should load `skills/ai-app-bridge-use/SKILL.md` before operating
29
+ apps through MCP. If the client supports skills or project instructions, load
30
+ that file as the dedicated usage guide. If it does not, include the file content
31
+ in the system/developer/project instructions for the task. Codex-compatible
32
+ clients can copy the repo folder `skills/ai-app-bridge-use` into their skills
33
+ directory, for example `$CODEX_HOME/skills/ai-app-bridge-use` on macOS/Linux or
34
+ `%USERPROFILE%\.codex\skills\ai-app-bridge-use` on Windows, then restart or
35
+ refresh the session.
36
+
37
+ MCP defaults to a compact tool surface to avoid loading every command schema
38
+ into the model context:
39
+
40
+ - `capabilities` lists the bridge domains and command names.
41
+ - `run` executes a selected command with command-specific arguments.
42
+
43
+ This keeps install, data reset, launch, UI, Flutter, WebView, logcat, network,
44
+ permission, iOS, and web capabilities discoverable without exposing dozens of full schemas
45
+ at session start. Set `AI_APP_BRIDGE_MCP_SURFACE=full` before launching
46
+ `ai-app-bridge-mcp` only when a client needs the legacy one-tool-per-command
47
+ surface.
48
+
49
+ The MCP server accepts both standard `Content-Length` framed JSON-RPC messages
50
+ and single-line JSON messages. Responses use the format of the first request on
51
+ that connection, so standard MCP clients keep framed responses while local
52
+ Node REPL scripts can send and read one JSON object per line.
53
+
54
+ For multi-step app automation, call `run` with `command: "batch"`. Batch steps
55
+ run serially in one MCP call, so a failed step can stop and mark the remaining
56
+ steps as skipped without mixing results from different commands:
57
+
58
+ ```json
59
+ {
60
+ "command": "batch",
61
+ "arguments": {
62
+ "defaults": {
63
+ "packageName": "io.github.mobileaidev.aiappbridge.sample"
64
+ },
65
+ "steps": [
66
+ { "id": "launch", "command": "launch-app" },
67
+ { "id": "wait-home", "command": "wait-text", "arguments": { "targetText": "Home" } },
68
+ { "id": "capture-logs", "command": "logs", "arguments": { "limit": 20 } }
69
+ ],
70
+ "stopOnError": true
71
+ }
72
+ }
73
+ ```
74
+
69
75
  For dynamic or transient screens, MCP agents can use `freeze-app`/`thaw-app` as
70
76
  an optional stabilization control: thaw before reads, actions, waits, or
71
77
  captures; freeze after evidence capture only when a changing UI would make
@@ -75,24 +81,34 @@ flows usually do not need freezing.
75
81
  For visible state changes such as panels, dialogs, page transitions, tabs, or
76
82
  button-triggered content, verify with both `screenshot` and `tree`/`uia-tree`;
77
83
  do not conclude success from UI tree alone.
78
-
79
- WebView network and console capture use Android WebView DevTools/CDP when the
80
- target app is debuggable and WebView debugging is enabled.
81
-
82
- `input-text` first uses the app bridge native text endpoint. This is required
83
- for Chinese and other Unicode text because `adb shell input text` is ASCII-only
84
- on many Android 16 devices; ASCII text can still fall back to ADB when an older
85
- bridge runtime is running.
86
-
87
- When `screenshot` or `smoke` runs without `--out-file`, the CLI writes a unique
88
- PNG under `build/ai_app_bridge_artifacts` instead of reusing a stable filename
89
- or creating files in the project root.
90
- It keeps the newest 20 generated screenshots for each command prefix. Use
91
- `--artifact-dir` to choose that directory, or `--out-file` when a fixed path is
92
- intentional.
93
-
94
- `launch-app` queries Android LAUNCHER activities before starting the app. If a
95
- debug dependency exposes multiple launcher entries, it returns
96
- `launcher_ambiguous` with the candidates instead of guessing. Use
97
- `launch-activity` or `launch-app --activity/--component` to choose the intended
98
- entry point explicitly.
84
+
85
+ WebView network and console capture use Android WebView DevTools/CDP when the
86
+ target app is debuggable and WebView debugging is enabled.
87
+
88
+ iOS commands use Xcode `devicectl` for device/app/screenshot operations, the
89
+ AiAppBridgeIOS runtime for in-app evidence, and WebDriverAgent/XCUITest for
90
+ full-control taps, input, swipes, and external UI tree reads. `ios-setup`
91
+ can start the vendored `appium-webdriveragent` project when `--start-wda` and
92
+ `--team-id` are supplied. On physical devices, reuse the returned WDA URL for
93
+ later WDA commands; it may be a CoreDevice tunnel such as
94
+ `http://[fdxx::1]:8100`. It returns explicit blockers for Developer Mode,
95
+ device preparation, signing, or WDA reachability instead of silently
96
+ downgrading iOS capability.
97
+
98
+ `input-text` first uses the app bridge native text endpoint. This is required
99
+ for Chinese and other Unicode text because `adb shell input text` is ASCII-only
100
+ on many Android 16 devices; ASCII text can still fall back to ADB when an older
101
+ bridge runtime is running.
102
+
103
+ When `screenshot` or `smoke` runs without `--out-file`, the CLI writes a unique
104
+ PNG under `build/ai_app_bridge_artifacts` instead of reusing a stable filename
105
+ or creating files in the project root.
106
+ It keeps the newest 20 generated screenshots for each command prefix. Use
107
+ `--artifact-dir` to choose that directory, or `--out-file` when a fixed path is
108
+ intentional.
109
+
110
+ `launch-app` queries Android LAUNCHER activities before starting the app. If a
111
+ debug dependency exposes multiple launcher entries, it returns
112
+ `launcher_ambiguous` with the candidates instead of guessing. Use
113
+ `launch-activity` or `launch-app --activity/--component` to choose the intended
114
+ entry point explicitly.