@mobileaidev/ai-app-bridge 0.2.6 → 0.2.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/README.md +85 -65
  2. package/bin/ai-app-bridge.js +561 -465
  3. package/bin/mcp-server.js +755 -741
  4. package/package.json +7 -6
  5. package/skills/ai-app-bridge-use/SKILL.md +183 -0
  6. package/skills/ai-app-bridge-use/agents/openai.yaml +4 -0
package/README.md CHANGED
@@ -3,74 +3,94 @@
3
3
  ```bash
4
4
  npm install -g @mobileaidev/ai-app-bridge
5
5
 
6
- ai-app-bridge status --package-name io.github.mobileaidev.aiappbridge.sample
7
- ai-app-bridge tree --package-name io.github.mobileaidev.aiappbridge.sample
8
- ai-app-bridge install-apk --package-name io.github.mobileaidev.aiappbridge.sample --apk-path app-debug.apk
9
- ai-app-bridge clear-app-data --package-name io.github.mobileaidev.aiappbridge.sample
10
- ai-app-bridge launch-app --package-name io.github.mobileaidev.aiappbridge.sample
11
- ai-app-bridge launch-activity --package-name io.github.mobileaidev.aiappbridge.sample --activity .MainActivity --extra route=/home
12
- ai-app-bridge screenshot --package-name io.github.mobileaidev.aiappbridge.sample
13
- ai-app-bridge input-text --package-name io.github.mobileaidev.aiappbridge.sample --text "中文输入" --hide-keyboard
6
+ ai-app-bridge status --package-name io.github.mobileaidev.aiappbridge.sample
7
+ ai-app-bridge tree --package-name io.github.mobileaidev.aiappbridge.sample
8
+ ai-app-bridge install-apk --package-name io.github.mobileaidev.aiappbridge.sample --apk-path app-debug.apk
9
+ ai-app-bridge clear-app-data --package-name io.github.mobileaidev.aiappbridge.sample
10
+ ai-app-bridge launch-app --package-name io.github.mobileaidev.aiappbridge.sample
11
+ ai-app-bridge launch-activity --package-name io.github.mobileaidev.aiappbridge.sample --activity .MainActivity --extra route=/home
12
+ ai-app-bridge screenshot --package-name io.github.mobileaidev.aiappbridge.sample
13
+ ai-app-bridge input-text --package-name io.github.mobileaidev.aiappbridge.sample --text "中文输入" --hide-keyboard
14
14
  ai-app-bridge network --package-name io.github.mobileaidev.aiappbridge.sample --compact --url-filter /api/
15
- ai-app-bridge webview-network --package-name io.github.mobileaidev.aiappbridge.sample --duration-ms 3000
16
- ai-app-bridge-mcp
17
- ```
18
-
19
- MCP defaults to a compact tool surface to avoid loading every command schema
20
- into the model context:
21
-
22
- - `capabilities` lists the bridge domains and command names.
23
- - `run` executes a selected command with command-specific arguments.
24
-
25
- This keeps install, data reset, launch, UI, Flutter, WebView, logcat, network,
26
- and permission capabilities discoverable without exposing dozens of full schemas
27
- at session start. Set `AI_APP_BRIDGE_MCP_SURFACE=full` before launching
28
- `ai-app-bridge-mcp` only when a client needs the legacy one-tool-per-command
29
- surface.
30
-
31
- The MCP server accepts both standard `Content-Length` framed JSON-RPC messages
32
- and single-line JSON messages. Responses use the format of the first request on
33
- that connection, so standard MCP clients keep framed responses while local
34
- Node REPL scripts can send and read one JSON object per line.
35
-
36
- For multi-step app automation, call `run` with `command: "batch"`. Batch steps
37
- run serially in one MCP call, so a failed step can stop and mark the remaining
38
- steps as skipped without mixing results from different commands:
39
-
40
- ```json
41
- {
42
- "command": "batch",
43
- "arguments": {
44
- "defaults": {
45
- "packageName": "io.github.mobileaidev.aiappbridge.sample"
46
- },
47
- "steps": [
48
- { "id": "launch", "command": "launch-app" },
49
- { "id": "wait-home", "command": "wait-text", "arguments": { "targetText": "Home" } },
50
- { "id": "capture-logs", "command": "logs", "arguments": { "limit": 20 } }
51
- ],
52
- "stopOnError": true
53
- }
54
- }
55
- ```
56
-
57
- WebView network and console capture use Android WebView DevTools/CDP when the
58
- target app is debuggable and WebView debugging is enabled.
59
-
60
- `input-text` first uses the app bridge native text endpoint. This is required
61
- for Chinese and other Unicode text because `adb shell input text` is ASCII-only
62
- on many Android 16 devices; ASCII text can still fall back to ADB when an older
63
- bridge runtime is running.
15
+ ai-app-bridge webview-network --package-name io.github.mobileaidev.aiappbridge.sample --duration-ms 3000
16
+ ai-app-bridge thaw-app --package-name io.github.mobileaidev.aiappbridge.sample
17
+ ai-app-bridge freeze-app --package-name io.github.mobileaidev.aiappbridge.sample
18
+ ai-app-bridge-mcp
19
+ ```
20
+
21
+ AI agents, model clients, automation frameworks, and IDE MCP clients reading
22
+ this README should load `skills/ai-app-bridge-use/SKILL.md` before operating
23
+ apps through MCP. If the client supports skills or project instructions, load
24
+ that file as the dedicated usage guide. If it does not, include the file content
25
+ in the system/developer/project instructions for the task. Codex-compatible
26
+ clients can copy the repo folder `skills/ai-app-bridge-use` into their skills
27
+ directory, for example `$CODEX_HOME/skills/ai-app-bridge-use` on macOS/Linux or
28
+ `%USERPROFILE%\.codex\skills\ai-app-bridge-use` on Windows, then restart or
29
+ refresh the session.
30
+
31
+ MCP defaults to a compact tool surface to avoid loading every command schema
32
+ into the model context:
33
+
34
+ - `capabilities` lists the bridge domains and command names.
35
+ - `run` executes a selected command with command-specific arguments.
36
+
37
+ This keeps install, data reset, launch, UI, Flutter, WebView, logcat, network,
38
+ and permission capabilities discoverable without exposing dozens of full schemas
39
+ at session start. Set `AI_APP_BRIDGE_MCP_SURFACE=full` before launching
40
+ `ai-app-bridge-mcp` only when a client needs the legacy one-tool-per-command
41
+ surface.
42
+
43
+ The MCP server accepts both standard `Content-Length` framed JSON-RPC messages
44
+ and single-line JSON messages. Responses use the format of the first request on
45
+ that connection, so standard MCP clients keep framed responses while local
46
+ Node REPL scripts can send and read one JSON object per line.
47
+
48
+ For multi-step app automation, call `run` with `command: "batch"`. Batch steps
49
+ run serially in one MCP call, so a failed step can stop and mark the remaining
50
+ steps as skipped without mixing results from different commands:
51
+
52
+ ```json
53
+ {
54
+ "command": "batch",
55
+ "arguments": {
56
+ "defaults": {
57
+ "packageName": "io.github.mobileaidev.aiappbridge.sample"
58
+ },
59
+ "steps": [
60
+ { "id": "launch", "command": "launch-app" },
61
+ { "id": "wait-home", "command": "wait-text", "arguments": { "targetText": "Home" } },
62
+ { "id": "capture-logs", "command": "logs", "arguments": { "limit": 20 } }
63
+ ],
64
+ "stopOnError": true
65
+ }
66
+ }
67
+ ```
68
+
69
+ For dynamic screens, MCP agents should follow the `ai-app-bridge-use` cadence:
70
+ thaw immediately before reading app content or performing an app action, freeze
71
+ immediately after evidence or action results are captured, keep the app frozen
72
+ while reasoning/planning, and thaw once more before finishing the overall task.
73
+ For visible state changes such as panels, dialogs, page transitions, tabs, or
74
+ button-triggered content, verify with both `screenshot` and `tree`/`uia-tree`;
75
+ do not conclude success from UI tree alone.
76
+
77
+ WebView network and console capture use Android WebView DevTools/CDP when the
78
+ target app is debuggable and WebView debugging is enabled.
79
+
80
+ `input-text` first uses the app bridge native text endpoint. This is required
81
+ for Chinese and other Unicode text because `adb shell input text` is ASCII-only
82
+ on many Android 16 devices; ASCII text can still fall back to ADB when an older
83
+ bridge runtime is running.
64
84
 
65
85
  When `screenshot` or `smoke` runs without `--out-file`, the CLI writes a unique
66
86
  PNG under `build/ai_app_bridge_artifacts` instead of reusing a stable filename
67
87
  or creating files in the project root.
68
88
  It keeps the newest 20 generated screenshots for each command prefix. Use
69
- `--artifact-dir` to choose that directory, or `--out-file` when a fixed path is
70
- intentional.
71
-
72
- `launch-app` queries Android LAUNCHER activities before starting the app. If a
73
- debug dependency exposes multiple launcher entries, it returns
74
- `launcher_ambiguous` with the candidates instead of guessing. Use
75
- `launch-activity` or `launch-app --activity/--component` to choose the intended
76
- entry point explicitly.
89
+ `--artifact-dir` to choose that directory, or `--out-file` when a fixed path is
90
+ intentional.
91
+
92
+ `launch-app` queries Android LAUNCHER activities before starting the app. If a
93
+ debug dependency exposes multiple launcher entries, it returns
94
+ `launcher_ambiguous` with the candidates instead of guessing. Use
95
+ `launch-activity` or `launch-app --activity/--component` to choose the intended
96
+ entry point explicitly.