@mobileaidev/ai-app-bridge 0.2.12 → 0.2.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +33 -10
- package/bin/ai-app-bridge.js +33 -40
- package/bin/artifact-paths.js +129 -0
- package/bin/ios-provider.js +2 -10
- package/bin/mcp-server.js +72 -9
- package/package.json +3 -2
- package/skills/ai-app-bridge-use/SKILL.md +21 -2
- package/skills/ai-app-bridge-use/agents/openai.yaml +2 -2
package/README.md
CHANGED
|
@@ -1,5 +1,23 @@
|
|
|
1
1
|
# AI App Bridge CLI
|
|
2
2
|
|
|
3
|
+
AI App Bridge CLI/MCP supports Android native apps, Android WebView/H5/CDP,
|
|
4
|
+
Flutter apps on Android and iOS, iOS native apps via `AiAppBridgeIOS` plus
|
|
5
|
+
WebDriverAgent/XCUITest, WKWebView, and desktop Web Bridge sessions.
|
|
6
|
+
|
|
7
|
+
Command domains:
|
|
8
|
+
|
|
9
|
+
- `core`: `status`, `tree`, `uia-tree`, `screenshot`, `logs`, `network`, `state`, `events`
|
|
10
|
+
- `app`: `install-apk`, `clear-app-data`, `launch-*`, `freeze-app`, `thaw-app`, `permission-*`, `appops-set`
|
|
11
|
+
- `action`: `tap`, `tap-text`, `tap-uia-text`, `input-text`, `swipe`, `keyevent`, `wait-text`, `keyboard-state`, `hide-keyboard`
|
|
12
|
+
- `flutter`: `flutter-tree`, `flutter-nodes`, `flutter-action`, `tap-flutter-text`, `input-flutter-text`, `scroll-flutter`
|
|
13
|
+
- `webview`: `h5-*`, `flutter-h5-*`, `webview-pages`, `webview-network`, `webview-console`
|
|
14
|
+
- `ios`: `ios-devices`, `ios-doctor`, `ios-setup`, `ios-*` runtime evidence, WDA tree/tap/input/swipe, WKWebView, and Flutter iOS
|
|
15
|
+
- `web`: `web-session-start`, `web-sessions`, `web-status`, `web-dom`, `web-logs`, `web-network`, `web-state`, `web-events`, `web-command`, `web-click`, `web-input`, `web-wait`, `web-scroll`
|
|
16
|
+
- `diagnostics` / `advanced`: `logcat`, `smoke`, `batch`, `forward`, `remove-forward`
|
|
17
|
+
|
|
18
|
+
For MCP clients, the default surface is compact: call `capabilities` to discover
|
|
19
|
+
domains, commands, and options, then call `run` with the selected command.
|
|
20
|
+
|
|
3
21
|
```bash
|
|
4
22
|
npm install -g @mobileaidev/ai-app-bridge
|
|
5
23
|
|
|
@@ -19,9 +37,10 @@ ai-app-bridge ios-setup --device-id <device-or-udid> --bundle-id <ios.bundle.id>
|
|
|
19
37
|
ai-app-bridge ios-status --device-id <device-or-udid> --bundle-id <ios.bundle.id>
|
|
20
38
|
ai-app-bridge ios-tap --bundle-id <ios.bundle.id> --tap-x 120 --tap-y 360 --wda-url <wda-url-from-setup>
|
|
21
39
|
ai-app-bridge ios-input --bundle-id <ios.bundle.id> --accessibility-id sample_text_field --clear-first --text "hello" --wda-url <wda-url-from-setup>
|
|
40
|
+
ai-app-bridge-mcp # then run web-session-start / web-* commands through MCP
|
|
22
41
|
ai-app-bridge thaw-app --package-name io.github.mobileaidev.aiappbridge.sample
|
|
23
42
|
ai-app-bridge freeze-app --package-name io.github.mobileaidev.aiappbridge.sample
|
|
24
|
-
ai-app-bridge-mcp
|
|
43
|
+
ai-app-bridge-mcp --help
|
|
25
44
|
```
|
|
26
45
|
|
|
27
46
|
AI agents, model clients, automation frameworks, and IDE MCP clients reading
|
|
@@ -37,12 +56,13 @@ refresh the session.
|
|
|
37
56
|
MCP defaults to a compact tool surface to avoid loading every command schema
|
|
38
57
|
into the model context:
|
|
39
58
|
|
|
40
|
-
- `capabilities` lists
|
|
59
|
+
- `capabilities` lists supported targets, command domains, command names, and optional argument names.
|
|
41
60
|
- `run` executes a selected command with command-specific arguments.
|
|
42
61
|
|
|
43
|
-
This keeps install, data reset, launch, UI, Flutter, WebView,
|
|
44
|
-
|
|
45
|
-
|
|
62
|
+
This keeps install, data reset, launch, UI action, Flutter, WebView/H5/CDP,
|
|
63
|
+
iOS, Web Bridge, logcat, network, permission, smoke, batch, and port-forward
|
|
64
|
+
capabilities discoverable without exposing dozens of full schemas at session start.
|
|
65
|
+
Set `AI_APP_BRIDGE_MCP_SURFACE=full` before launching
|
|
46
66
|
`ai-app-bridge-mcp` only when a client needs the legacy one-tool-per-command
|
|
47
67
|
surface.
|
|
48
68
|
|
|
@@ -101,11 +121,14 @@ on many Android 16 devices; ASCII text can still fall back to ADB when an older
|
|
|
101
121
|
bridge runtime is running.
|
|
102
122
|
|
|
103
123
|
When `screenshot` or `smoke` runs without `--out-file`, the CLI writes a unique
|
|
104
|
-
PNG under
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
124
|
+
PNG under a git-ignored project artifact directory. Gradle, Android, and Flutter
|
|
125
|
+
projects normally use `build/ai_app_bridge_artifacts`; Node projects can use
|
|
126
|
+
`node_modules/.cache/ai_app_bridge_artifacts`; Swift projects can use
|
|
127
|
+
`.build/ai_app_bridge_artifacts`. If the current git worktree has no ignored
|
|
128
|
+
artifact candidate, generated defaults go under `.git/ai_app_bridge_artifacts`
|
|
129
|
+
so they cannot dirty the repository root. It keeps the newest 20 generated
|
|
130
|
+
screenshots for each command prefix. Use `--artifact-dir` to choose an ignored
|
|
131
|
+
directory, or `--out-file` when a fixed path is intentional.
|
|
109
132
|
|
|
110
133
|
`launch-app` queries Android LAUNCHER activities before starting the app. If a
|
|
111
134
|
debug dependency exposes multiple launcher entries, it returns
|
package/bin/ai-app-bridge.js
CHANGED
|
@@ -7,6 +7,13 @@ const net = require('net');
|
|
|
7
7
|
const os = require('os');
|
|
8
8
|
const path = require('path');
|
|
9
9
|
const { IOSBridgeProvider } = require('./ios-provider');
|
|
10
|
+
const {
|
|
11
|
+
artifactTimestamp,
|
|
12
|
+
defaultArtifactDirectory,
|
|
13
|
+
defaultArtifactPath,
|
|
14
|
+
sanitizeArtifactExtension,
|
|
15
|
+
sanitizeArtifactName,
|
|
16
|
+
} = require('./artifact-paths');
|
|
10
17
|
|
|
11
18
|
const generatedArtifactRetention = 20;
|
|
12
19
|
|
|
@@ -22,6 +29,15 @@ const defaults = {
|
|
|
22
29
|
|
|
23
30
|
const helpText = `Usage: ai-app-bridge <command> [options]
|
|
24
31
|
|
|
32
|
+
Supports:
|
|
33
|
+
Android native apps; Android WebView/H5/CDP; Flutter apps on Android and iOS;
|
|
34
|
+
iOS native apps via AiAppBridgeIOS + WebDriverAgent/XCUITest; WKWebView;
|
|
35
|
+
desktop Web Bridge sessions through ai-app-bridge-mcp.
|
|
36
|
+
|
|
37
|
+
MCP discovery:
|
|
38
|
+
ai-app-bridge-mcp exposes a compact capability index by default.
|
|
39
|
+
Call capabilities, then run a command from core/app/action/flutter/webview/ios/web/diagnostics/advanced.
|
|
40
|
+
|
|
25
41
|
Commands:
|
|
26
42
|
status Read bridge status and app/device metadata.
|
|
27
43
|
tree Read the Android View tree from the in-app bridge.
|
|
@@ -84,6 +100,23 @@ Flutter commands:
|
|
|
84
100
|
flutter-h5-wait Wait for Flutter H5 text or selector.
|
|
85
101
|
flutter-h5-scroll Scroll Flutter H5 content.
|
|
86
102
|
|
|
103
|
+
Web Bridge MCP commands:
|
|
104
|
+
web-provider-status Read desktop Web Bridge provider status.
|
|
105
|
+
web-session-start Start the Web Bridge WebSocket session server.
|
|
106
|
+
web-connect-info Read endpoint/token for browser SDK clients.
|
|
107
|
+
web-sessions List connected Web Bridge SDK sessions.
|
|
108
|
+
web-status Read Web Bridge session status.
|
|
109
|
+
web-dom Read or refresh Web Bridge DOM snapshots.
|
|
110
|
+
web-logs Read Web Bridge log records.
|
|
111
|
+
web-network Read Web Bridge network records.
|
|
112
|
+
web-state Read Web Bridge state records.
|
|
113
|
+
web-events Read Web Bridge event records.
|
|
114
|
+
web-command Run a whitelisted browser SDK command.
|
|
115
|
+
web-click Click a DOM element through the browser SDK.
|
|
116
|
+
web-input Set text in a DOM input through the browser SDK.
|
|
117
|
+
web-wait Wait for text or selector through the browser SDK.
|
|
118
|
+
web-scroll Scroll a Web Bridge DOM target.
|
|
119
|
+
|
|
87
120
|
Device/action commands:
|
|
88
121
|
tap Tap device coordinates through ADB.
|
|
89
122
|
tap-text Tap a visible node by exact text or content description.
|
|
@@ -2366,17 +2399,6 @@ function screenshotOutputPath(options = {}, prefix = 'ai_app_bridge_screenshot')
|
|
|
2366
2399
|
return defaultArtifactPath(prefix, 'png', { artifactDir: options.artifactDir });
|
|
2367
2400
|
}
|
|
2368
2401
|
|
|
2369
|
-
function defaultArtifactPath(prefix, extension, options = {}) {
|
|
2370
|
-
const directory = path.resolve(options.artifactDir || defaultArtifactDirectory());
|
|
2371
|
-
const suffix = [
|
|
2372
|
-
artifactTimestamp(options.now || new Date()),
|
|
2373
|
-
String(options.pid || process.pid),
|
|
2374
|
-
options.randomSuffix || Math.random().toString(36).slice(2, 8),
|
|
2375
|
-
].join('-');
|
|
2376
|
-
const name = `${sanitizeArtifactName(prefix)}-${suffix}.${sanitizeArtifactExtension(extension)}`;
|
|
2377
|
-
return path.join(directory, name);
|
|
2378
|
-
}
|
|
2379
|
-
|
|
2380
2402
|
async function pruneGeneratedArtifacts(options = {}) {
|
|
2381
2403
|
const keep = generatedArtifactRetention;
|
|
2382
2404
|
const result = {
|
|
@@ -2449,34 +2471,6 @@ async function pruneGeneratedArtifacts(options = {}) {
|
|
|
2449
2471
|
return result;
|
|
2450
2472
|
}
|
|
2451
2473
|
|
|
2452
|
-
function artifactTimestamp(date) {
|
|
2453
|
-
const value = date instanceof Date ? date : new Date(date);
|
|
2454
|
-
const pad = (number, size = 2) => String(number).padStart(size, '0');
|
|
2455
|
-
return [
|
|
2456
|
-
value.getUTCFullYear(),
|
|
2457
|
-
pad(value.getUTCMonth() + 1),
|
|
2458
|
-
pad(value.getUTCDate()),
|
|
2459
|
-
'-',
|
|
2460
|
-
pad(value.getUTCHours()),
|
|
2461
|
-
pad(value.getUTCMinutes()),
|
|
2462
|
-
pad(value.getUTCSeconds()),
|
|
2463
|
-
'-',
|
|
2464
|
-
pad(value.getUTCMilliseconds(), 3),
|
|
2465
|
-
].join('');
|
|
2466
|
-
}
|
|
2467
|
-
|
|
2468
|
-
function sanitizeArtifactName(value) {
|
|
2469
|
-
return String(value || 'artifact').replace(/[^a-zA-Z0-9_.-]+/g, '_').replace(/^_+|_+$/g, '') || 'artifact';
|
|
2470
|
-
}
|
|
2471
|
-
|
|
2472
|
-
function sanitizeArtifactExtension(value) {
|
|
2473
|
-
return sanitizeArtifactName(String(value || 'bin').replace(/^\.+/, '')) || 'bin';
|
|
2474
|
-
}
|
|
2475
|
-
|
|
2476
|
-
function defaultArtifactDirectory() {
|
|
2477
|
-
return path.join(process.cwd(), 'build', 'ai_app_bridge_artifacts');
|
|
2478
|
-
}
|
|
2479
|
-
|
|
2480
2474
|
function escapeRegExp(value) {
|
|
2481
2475
|
return String(value).replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
2482
2476
|
}
|
|
@@ -4555,4 +4549,3 @@ module.exports = {
|
|
|
4555
4549
|
waitTextConditionsMet,
|
|
4556
4550
|
withFileLock,
|
|
4557
4551
|
};
|
|
4558
|
-
|
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
const { execFileSync } = require('child_process');
|
|
2
|
+
const fs = require('fs');
|
|
3
|
+
const path = require('path');
|
|
4
|
+
|
|
5
|
+
const artifactDirectoryName = 'ai_app_bridge_artifacts';
|
|
6
|
+
|
|
7
|
+
function defaultArtifactDirectory(options = {}) {
|
|
8
|
+
const cwd = path.resolve(options.cwd || process.cwd());
|
|
9
|
+
const gitRoot = gitOutput(cwd, ['rev-parse', '--show-toplevel']);
|
|
10
|
+
if (!gitRoot) {
|
|
11
|
+
return path.join(cwd, 'build', artifactDirectoryName);
|
|
12
|
+
}
|
|
13
|
+
|
|
14
|
+
for (const directory of artifactDirectoryCandidates(gitRoot)) {
|
|
15
|
+
if (isGitIgnored(gitRoot, directory)) {
|
|
16
|
+
return directory;
|
|
17
|
+
}
|
|
18
|
+
}
|
|
19
|
+
|
|
20
|
+
const gitDirectory = gitOutput(cwd, ['rev-parse', '--absolute-git-dir']);
|
|
21
|
+
return path.join(gitDirectory || path.join(gitRoot, '.git'), artifactDirectoryName);
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
function defaultArtifactPath(prefix, extension, options = {}) {
|
|
25
|
+
const directory = path.resolve(options.artifactDir || defaultArtifactDirectory({ cwd: options.cwd }));
|
|
26
|
+
const suffix = [
|
|
27
|
+
artifactTimestamp(options.now || new Date()),
|
|
28
|
+
String(options.pid || process.pid),
|
|
29
|
+
options.randomSuffix || Math.random().toString(36).slice(2, 8),
|
|
30
|
+
].join('-');
|
|
31
|
+
const name = `${sanitizeArtifactName(prefix)}-${suffix}.${sanitizeArtifactExtension(extension)}`;
|
|
32
|
+
return path.join(directory, name);
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
function artifactDirectoryCandidates(gitRoot) {
|
|
36
|
+
const root = path.resolve(gitRoot);
|
|
37
|
+
const candidates = [];
|
|
38
|
+
|
|
39
|
+
if (hasAny(root, ['settings.gradle', 'settings.gradle.kts', 'build.gradle', 'build.gradle.kts', 'gradlew', 'pubspec.yaml'])) {
|
|
40
|
+
candidates.push(path.join(root, 'build', artifactDirectoryName));
|
|
41
|
+
}
|
|
42
|
+
if (hasAny(root, ['Package.swift'])) {
|
|
43
|
+
candidates.push(path.join(root, '.build', artifactDirectoryName));
|
|
44
|
+
}
|
|
45
|
+
if (hasAny(root, ['pubspec.yaml'])) {
|
|
46
|
+
candidates.push(path.join(root, '.dart_tool', artifactDirectoryName));
|
|
47
|
+
}
|
|
48
|
+
if (hasAny(root, ['package.json'])) {
|
|
49
|
+
candidates.push(path.join(root, 'node_modules', '.cache', artifactDirectoryName));
|
|
50
|
+
}
|
|
51
|
+
if (hasAny(root, ['Cargo.toml'])) {
|
|
52
|
+
candidates.push(path.join(root, 'target', artifactDirectoryName));
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
candidates.push(
|
|
56
|
+
path.join(root, 'build', artifactDirectoryName),
|
|
57
|
+
path.join(root, '.build', artifactDirectoryName),
|
|
58
|
+
path.join(root, '.dart_tool', artifactDirectoryName),
|
|
59
|
+
path.join(root, 'node_modules', '.cache', artifactDirectoryName),
|
|
60
|
+
path.join(root, 'target', artifactDirectoryName),
|
|
61
|
+
);
|
|
62
|
+
|
|
63
|
+
return [...new Set(candidates)];
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
function hasAny(root, fileNames) {
|
|
67
|
+
return fileNames.some((fileName) => fs.existsSync(path.join(root, fileName)));
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
function isGitIgnored(gitRoot, directory) {
|
|
71
|
+
const relative = path.relative(gitRoot, directory);
|
|
72
|
+
if (!relative || relative.startsWith('..') || path.isAbsolute(relative)) {
|
|
73
|
+
return false;
|
|
74
|
+
}
|
|
75
|
+
try {
|
|
76
|
+
execFileSync('git', ['-C', gitRoot, 'check-ignore', '-q', '--', relative], {
|
|
77
|
+
stdio: 'ignore',
|
|
78
|
+
timeout: 2000,
|
|
79
|
+
});
|
|
80
|
+
return true;
|
|
81
|
+
} catch (_) {
|
|
82
|
+
return false;
|
|
83
|
+
}
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
function gitOutput(cwd, args) {
|
|
87
|
+
try {
|
|
88
|
+
return execFileSync('git', ['-C', cwd, ...args], {
|
|
89
|
+
encoding: 'utf8',
|
|
90
|
+
stdio: ['ignore', 'pipe', 'ignore'],
|
|
91
|
+
timeout: 2000,
|
|
92
|
+
}).trim();
|
|
93
|
+
} catch (_) {
|
|
94
|
+
return '';
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
function artifactTimestamp(date) {
|
|
99
|
+
const value = date instanceof Date ? date : new Date(date);
|
|
100
|
+
const pad = (number, size = 2) => String(number).padStart(size, '0');
|
|
101
|
+
return [
|
|
102
|
+
value.getUTCFullYear(),
|
|
103
|
+
pad(value.getUTCMonth() + 1),
|
|
104
|
+
pad(value.getUTCDate()),
|
|
105
|
+
'-',
|
|
106
|
+
pad(value.getUTCHours()),
|
|
107
|
+
pad(value.getUTCMinutes()),
|
|
108
|
+
pad(value.getUTCSeconds()),
|
|
109
|
+
'-',
|
|
110
|
+
pad(value.getUTCMilliseconds(), 3),
|
|
111
|
+
].join('');
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
function sanitizeArtifactName(value) {
|
|
115
|
+
return String(value || 'artifact').replace(/[^a-zA-Z0-9_.-]+/g, '_').replace(/^_+|_+$/g, '') || 'artifact';
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
function sanitizeArtifactExtension(value) {
|
|
119
|
+
return sanitizeArtifactName(String(value || 'bin').replace(/^\.+/, '')) || 'bin';
|
|
120
|
+
}
|
|
121
|
+
|
|
122
|
+
module.exports = {
|
|
123
|
+
artifactDirectoryCandidates,
|
|
124
|
+
artifactTimestamp,
|
|
125
|
+
defaultArtifactDirectory,
|
|
126
|
+
defaultArtifactPath,
|
|
127
|
+
sanitizeArtifactExtension,
|
|
128
|
+
sanitizeArtifactName,
|
|
129
|
+
};
|
package/bin/ios-provider.js
CHANGED
|
@@ -4,6 +4,7 @@ const http = require('http');
|
|
|
4
4
|
const os = require('os');
|
|
5
5
|
const path = require('path');
|
|
6
6
|
const { URL } = require('url');
|
|
7
|
+
const { defaultArtifactPath } = require('./artifact-paths');
|
|
7
8
|
|
|
8
9
|
const defaultRuntimePort = 18080;
|
|
9
10
|
const runtimePortSearchCount = 50;
|
|
@@ -304,7 +305,7 @@ class IOSBridgeProvider {
|
|
|
304
305
|
const device = await this.requireDevice(args);
|
|
305
306
|
const outFile = args.outFile
|
|
306
307
|
? path.resolve(args.outFile)
|
|
307
|
-
: defaultArtifactPath('ios-screenshot', 'png', args.artifactDir);
|
|
308
|
+
: defaultArtifactPath('ios-screenshot', 'png', { artifactDir: args.artifactDir });
|
|
308
309
|
await fs.promises.mkdir(path.dirname(outFile), { recursive: true });
|
|
309
310
|
const command = [
|
|
310
311
|
'device',
|
|
@@ -1164,15 +1165,6 @@ function sleep(ms) {
|
|
|
1164
1165
|
return new Promise((resolve) => setTimeout(resolve, ms));
|
|
1165
1166
|
}
|
|
1166
1167
|
|
|
1167
|
-
function defaultArtifactPath(prefix, extension, artifactDir) {
|
|
1168
|
-
const dir = artifactDir ? path.resolve(artifactDir) : path.join(process.cwd(), 'build', 'ai_app_bridge_artifacts');
|
|
1169
|
-
return path.join(dir, `${prefix}-${artifactTimestamp()}.${extension}`);
|
|
1170
|
-
}
|
|
1171
|
-
|
|
1172
|
-
function artifactTimestamp(date = new Date()) {
|
|
1173
|
-
return date.toISOString().replace(/[-:]/g, '').replace(/\.\d{3}Z$/, 'Z');
|
|
1174
|
-
}
|
|
1175
|
-
|
|
1176
1168
|
module.exports = {
|
|
1177
1169
|
IOSBridgeProvider,
|
|
1178
1170
|
formatHostForUrl,
|
package/bin/mcp-server.js
CHANGED
|
@@ -6,18 +6,69 @@ const path = require('path');
|
|
|
6
6
|
const packageInfo = require('../package.json');
|
|
7
7
|
const { IOSBridgeProvider } = require('./ios-provider');
|
|
8
8
|
const { WebBridgeProvider } = require('./web-provider');
|
|
9
|
+
const { defaultArtifactDirectory } = require('./artifact-paths');
|
|
9
10
|
const bridgeDir = __dirname;
|
|
10
11
|
const cliScript = path.join(bridgeDir, 'ai-app-bridge.js');
|
|
11
12
|
const nodeBinary = process.env.AI_APP_BRIDGE_NODE || process.execPath;
|
|
12
13
|
const supportedProtocolVersions = ['2025-06-18', '2024-11-05'];
|
|
13
14
|
const defaultProtocolVersion = supportedProtocolVersions[0];
|
|
14
15
|
const mcpSurface = (process.env.AI_APP_BRIDGE_MCP_SURFACE || 'compact').toLowerCase();
|
|
16
|
+
const supportedTargets = [
|
|
17
|
+
'Android native apps',
|
|
18
|
+
'Android WebView/H5/CDP',
|
|
19
|
+
'Flutter apps on Android and iOS',
|
|
20
|
+
'iOS native apps via AiAppBridgeIOS + WebDriverAgent/XCUITest',
|
|
21
|
+
'WKWebView',
|
|
22
|
+
'desktop Web Bridge sessions',
|
|
23
|
+
];
|
|
24
|
+
const commandDomains = {
|
|
25
|
+
core: 'status, tree, uia-tree, screenshot, logs, network, state, events',
|
|
26
|
+
app: 'install-apk, clear-app-data, launch-*, freeze-app/thaw-app, permission-*, appops-set',
|
|
27
|
+
action: 'tap, tap-text, tap-uia-text, input-text, swipe, keyevent, wait-text, keyboard-state, hide-keyboard',
|
|
28
|
+
flutter: 'flutter-tree, flutter-nodes, flutter-action, tap/input/scroll Flutter controls',
|
|
29
|
+
webview: 'h5-*, flutter-h5-*, webview-pages, webview-network, webview-console',
|
|
30
|
+
ios: 'ios-devices, ios-doctor, ios-setup, ios runtime evidence, ios-uia-tree/tap/input/swipe, ios-h5-*, ios-flutter-*',
|
|
31
|
+
web: 'web-session-start, web-sessions, web-status, web-dom, web-logs, web-network, web-state, web-events, web-command, web-click, web-input, web-wait, web-scroll',
|
|
32
|
+
diagnostics: 'logcat, smoke',
|
|
33
|
+
advanced: 'batch, forward, remove-forward',
|
|
34
|
+
};
|
|
35
|
+
const supportedTargetsText = `AI App Bridge supports ${supportedTargets.join('; ')}.`;
|
|
36
|
+
const commandDomainsText = `Command domains: ${Object.entries(commandDomains).map(([domain, summary]) => `${domain}(${summary})`).join('; ')}.`;
|
|
37
|
+
const discoveryText = 'Default MCP surface is compact: call capabilities to discover domains, commands, and options, then call run with the selected command and arguments.';
|
|
15
38
|
const serverInstructions = [
|
|
16
|
-
|
|
17
|
-
|
|
39
|
+
supportedTargetsText,
|
|
40
|
+
commandDomainsText,
|
|
41
|
+
discoveryText,
|
|
42
|
+
'Prefer AI App Bridge over raw adb, devicectl, or browser-specific scripts when inspecting UI, text, WebView/WKWebView, logs, network, app install, launch, permissions, or app-level Web evidence.',
|
|
18
43
|
'Always pass packageName for Android app-specific commands, or pass an explicit port. For iOS, pass bundleId plus deviceId when more than one iPhone is connected.',
|
|
44
|
+
'For Web Bridge sessions, start the provider, connect the browser SDK, then pass sessionId and targetId when needed.',
|
|
19
45
|
'Use freeze-app/thaw-app only as an optional stabilization control for dynamic or transient screens: thaw before reads/actions/captures, freeze after evidence capture only when it helps reasoning, and thaw before the next operation or before finishing so the app is not left frozen.',
|
|
20
46
|
].join(' ');
|
|
47
|
+
const mcpHelpText = `Usage: ai-app-bridge-mcp [--help]
|
|
48
|
+
|
|
49
|
+
${supportedTargetsText}
|
|
50
|
+
|
|
51
|
+
${commandDomainsText}
|
|
52
|
+
|
|
53
|
+
MCP surface:
|
|
54
|
+
compact (default) exposes only capabilities and run.
|
|
55
|
+
full/legacy exposes one tool per command; set AI_APP_BRIDGE_MCP_SURFACE=full.
|
|
56
|
+
|
|
57
|
+
Discovery:
|
|
58
|
+
1. Call capabilities with optional domain or command filters.
|
|
59
|
+
2. Call run with a command name from capabilities.
|
|
60
|
+
3. Put command-specific options in arguments.
|
|
61
|
+
|
|
62
|
+
Target ids:
|
|
63
|
+
Android app commands require packageName or explicit port.
|
|
64
|
+
iOS app commands use bundleId; add deviceId when multiple devices exist and wdaUrl for WDA actions.
|
|
65
|
+
Web Bridge commands use sessionId; add targetId for multi-target pages.
|
|
66
|
+
|
|
67
|
+
Examples:
|
|
68
|
+
capabilities { "domain": "webview", "includeOptions": true }
|
|
69
|
+
run { "command": "screenshot", "packageName": "com.example.app" }
|
|
70
|
+
run { "command": "web-session-start", "arguments": { "webPort": 18180 } }
|
|
71
|
+
`;
|
|
21
72
|
const iosProvider = new IOSBridgeProvider();
|
|
22
73
|
const webProvider = new WebBridgeProvider();
|
|
23
74
|
|
|
@@ -185,18 +236,20 @@ function toolDefinitions() {
|
|
|
185
236
|
|
|
186
237
|
function compactToolDefinitions() {
|
|
187
238
|
return [
|
|
188
|
-
bridgeTool('capabilities',
|
|
189
|
-
domain: { type: 'string', description: 'Optional domain filter
|
|
190
|
-
command: { type: 'string', description: 'Optional command name for detailed arguments, such as install-apk, launch-app, tree, input-text, webview-network, ios-setup, or
|
|
239
|
+
bridgeTool('capabilities', `List AI App Bridge capability domains and commands across Android, iOS, Flutter, WebView/H5/CDP, and Web Bridge targets. ${commandDomainsText}`, {
|
|
240
|
+
domain: { type: 'string', description: 'Optional domain filter: core, app, action, flutter, webview, ios, web, diagnostics, or advanced.' },
|
|
241
|
+
command: { type: 'string', description: 'Optional command name for detailed arguments, such as install-apk, launch-app, tree, input-text, webview-network, ios-setup, ios-tap, web-session-start, or web-command.' },
|
|
191
242
|
includeOptions: { type: 'boolean', description: 'Include per-command argument names. Defaults to false to keep output compact.' },
|
|
192
243
|
}),
|
|
193
|
-
bridgeTool('run', 'Run an AI App Bridge command
|
|
194
|
-
command: { type: 'string', description: 'Command name from capabilities, using CLI form such as status, install-apk, launch-app, input-text, webview-network, ios-doctor, ios-setup, ios-status, ios-tap, or web-
|
|
244
|
+
bridgeTool('run', 'Run an AI App Bridge command from capabilities. Pass packageName/port for Android app commands, bundleId/deviceId for iOS app commands, and sessionId/targetId for Web Bridge commands.', {
|
|
245
|
+
command: { type: 'string', description: 'Command name from capabilities, using CLI form such as status, install-apk, launch-app, input-text, webview-network, ios-doctor, ios-setup, ios-status, ios-tap, web-session-start, web-status, or web-command.' },
|
|
195
246
|
packageName: { type: 'string', description: 'Target Android package for app-specific commands. Strongly recommended.' },
|
|
196
247
|
serial: { type: 'string', description: 'ADB serial when multiple devices are connected.' },
|
|
197
248
|
port: { type: 'number', description: 'Explicit bridge port when packageName discovery is not available.' },
|
|
198
249
|
bundleId: { type: 'string', description: 'Target iOS app bundle identifier for ios-* commands.' },
|
|
199
250
|
deviceId: { type: 'string', description: 'iOS devicectl identifier, UDID, serial number, or device name.' },
|
|
251
|
+
sessionId: { type: 'string', description: 'Target Web Bridge SDK session id for web-* commands.' },
|
|
252
|
+
targetId: { type: 'string', description: 'Optional Web Bridge target id for multi-target pages.' },
|
|
200
253
|
iosHost: { type: 'string', description: 'iOS runtime host or CoreDevice tunnel IP.' },
|
|
201
254
|
iosPort: { type: 'number', description: 'iOS runtime port when auto-discovery is unavailable.' },
|
|
202
255
|
runtimeUrl: { type: 'string', description: 'Explicit iOS runtime base URL.' },
|
|
@@ -635,7 +688,9 @@ function capabilityPayload(args = {}) {
|
|
|
635
688
|
return {
|
|
636
689
|
ok: true,
|
|
637
690
|
surface: mcpSurface === 'full' || mcpSurface === 'legacy' ? 'full' : 'compact',
|
|
638
|
-
|
|
691
|
+
supportedTargets,
|
|
692
|
+
commandDomains,
|
|
693
|
+
usage: `${supportedTargetsText} ${discoveryText} Use run with one of these command names. Prefer packageName for Android app commands, bundleId/deviceId for iOS, and sessionId/targetId for Web Bridge sessions.`,
|
|
639
694
|
domains,
|
|
640
695
|
};
|
|
641
696
|
}
|
|
@@ -1092,7 +1147,7 @@ async function runSmoke(args) {
|
|
|
1092
1147
|
function defaultArtifactDirFor(command, args) {
|
|
1093
1148
|
if (args.outFile) return '';
|
|
1094
1149
|
if (command !== 'screenshot' && command !== 'smoke') return '';
|
|
1095
|
-
return
|
|
1150
|
+
return defaultArtifactDirectory();
|
|
1096
1151
|
}
|
|
1097
1152
|
|
|
1098
1153
|
function addCommonArgs(cliArgs, args) {
|
|
@@ -1225,12 +1280,20 @@ function writeLog(text) {
|
|
|
1225
1280
|
}
|
|
1226
1281
|
|
|
1227
1282
|
if (require.main === module) {
|
|
1283
|
+
if (process.argv.includes('--help') || process.argv.includes('-h')) {
|
|
1284
|
+
process.stdout.write(`${mcpHelpText}\n`);
|
|
1285
|
+
process.exit(0);
|
|
1286
|
+
}
|
|
1228
1287
|
startServer();
|
|
1229
1288
|
}
|
|
1230
1289
|
|
|
1231
1290
|
module.exports = {
|
|
1232
1291
|
buildBridgeCliArgs,
|
|
1292
|
+
commandDomains,
|
|
1293
|
+
defaultArtifactDirFor,
|
|
1294
|
+
mcpHelpText,
|
|
1233
1295
|
readNextMessage,
|
|
1234
1296
|
runBatch,
|
|
1235
1297
|
startServer,
|
|
1298
|
+
supportedTargets,
|
|
1236
1299
|
};
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mobileaidev/ai-app-bridge",
|
|
3
|
-
"version": "0.2.
|
|
4
|
-
"description": "Desktop CLI and MCP server for AI App Bridge.",
|
|
3
|
+
"version": "0.2.14",
|
|
4
|
+
"description": "Desktop CLI and MCP server for AI App Bridge across Android, iOS, Flutter, WebView, and Web targets.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
7
7
|
"url": "git+https://github.com/mobileAiDev/ai-app-bridge.git",
|
|
@@ -17,6 +17,7 @@
|
|
|
17
17
|
},
|
|
18
18
|
"files": [
|
|
19
19
|
"bin/ai-app-bridge.js",
|
|
20
|
+
"bin/artifact-paths.js",
|
|
20
21
|
"bin/mcp-server.js",
|
|
21
22
|
"bin/ios-provider.js",
|
|
22
23
|
"bin/web-provider.js",
|
|
@@ -1,10 +1,28 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ai-app-bridge-use
|
|
3
|
-
description: 使用 AI App Bridge MCP 观察、操作、验证和调试 Android、
|
|
3
|
+
description: 使用 AI App Bridge MCP 观察、操作、验证和调试 Android native、Android WebView/H5/CDP、Flutter、iOS native + WDA/XCUITest、WKWebView 和桌面 Web Bridge。能力域包括 core/status/tree/screenshot/logs/network/state/events,app/install/clear-data/launch/freeze/thaw/permissions/appops,action/tap/input/swipe/keyevent/wait/keyboard,flutter/widget/action,webview/H5/CDP,ios/devices/setup/runtime/WDA/H5/flutter,web/session/DOM/logs/network/events/command,diagnostics/logcat/smoke,advanced/batch/port-forward。
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# AI App Bridge Use
|
|
7
7
|
|
|
8
|
+
## 能力总览
|
|
9
|
+
|
|
10
|
+
AI App Bridge 支持 Android native apps、Android WebView/H5/CDP、Flutter apps、iOS native apps via `AiAppBridgeIOS` + WebDriverAgent/XCUITest、WKWebView,以及桌面 Web Bridge sessions。
|
|
11
|
+
|
|
12
|
+
默认 MCP surface 只有 `capabilities` 和 `run`;先用 `capabilities` 查 domain、command 和 options,再用 `run` 执行。
|
|
13
|
+
|
|
14
|
+
目标标识:Android 用 `packageName` 或显式 `port`;iOS 用 `bundleId`,多设备加 `deviceId`,full-control 加 `wdaUrl`;Web 用 `sessionId`,多 target 加 `targetId`。
|
|
15
|
+
|
|
16
|
+
命令域:
|
|
17
|
+
- `core`: `status`/`tree`/`uia-tree`/`screenshot`/`logs`/`network`/`state`/`events`
|
|
18
|
+
- `app`: `install-apk`/`clear-app-data`/`launch-*`/`freeze-app`/`thaw-app`/`permission-*`/`appops-set`
|
|
19
|
+
- `action`: `tap`/`tap-text`/`tap-uia-text`/`input-text`/`swipe`/`keyevent`/`wait-text`/`keyboard-*`
|
|
20
|
+
- `flutter`: `flutter-tree`/`flutter-nodes`/`flutter-action`/`tap-flutter-text`/`input-flutter-text`/`scroll-flutter`
|
|
21
|
+
- `webview`: `h5-*`/`flutter-h5-*`/`webview-pages`/`webview-network`/`webview-console`
|
|
22
|
+
- `ios`: `ios-devices`/`ios-doctor`/`ios-setup`/`ios-status`/`ios-tree`/`ios-uia-tree`/`ios-tap`/`ios-input`/`ios-swipe`/`ios-h5-*`/`ios-flutter-*`
|
|
23
|
+
- `web`: `web-session-start`/`web-sessions`/`web-status`/`web-dom`/`web-logs`/`web-network`/`web-state`/`web-events`/`web-command`/`web-click`/`web-input`/`web-wait`/`web-scroll`
|
|
24
|
+
- `diagnostics`/`advanced`: `logcat`/`smoke`/`batch`/`forward`/`remove-forward`
|
|
25
|
+
|
|
8
26
|
## Agent 快速流程
|
|
9
27
|
|
|
10
28
|
1. 确认目标:Android 使用 `packageName`,iOS 使用 `bundleId`,真机多设备场景传 `deviceId`/UDID;Web 使用 `sessionId`,多 target 时加 `targetId`。没有目标 id 时先从上下文、构建配置或前台 app/session 线索推断。面向具体 app/session 的命令必须传目标 id,只有无法发现 bridge 端口时才传 `port`、`runtimeUrl` 或先启动 Web session。
|
|
@@ -12,7 +30,8 @@ description: 使用 AI App Bridge MCP 观察、操作、验证和调试 Android
|
|
|
12
30
|
3. 选择命令路径:按任务类型选 `core`、`app`、`action`、`flutter`、`webview`、`ios`、`web`、`diagnostics` 或 `advanced` 域;不要先退回原始 `adb`、浏览器脚本或坐标猜测。
|
|
13
31
|
4. 用 `batch` 串联相关步骤:观察、操作、等待、截图、tree 验证尽量放进一次 MCP 调用。
|
|
14
32
|
5. 验证可见结果:界面变化必须用 `screenshot` 加 `tree`/`uia-tree` 交叉确认。
|
|
15
|
-
6.
|
|
33
|
+
6. 生成截图等默认产物时,让 AI Bridge 自动选择会被当前项目 git 忽略的目录;只有确实需要固定路径时才传 `outFile`/`artifactDir`,且路径必须在 `build`、`.build`、`.dart_tool`、`node_modules/.cache`、`target` 或其他已忽略目录内。
|
|
34
|
+
7. 只在需要稳定动态画面时使用 `freeze-app`/`thaw-app`;如果本轮冻结过 app,最终回复前必须解冻。
|
|
16
35
|
|
|
17
36
|
## 能力发现和调用
|
|
18
37
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
interface:
|
|
2
2
|
display_name: "AI App Bridge Use"
|
|
3
|
-
short_description: "
|
|
4
|
-
default_prompt: "使用 $ai-app-bridge-use
|
|
3
|
+
short_description: "Android/iOS/Flutter/Web 全域观察、操作、验证能力索引。"
|
|
4
|
+
default_prompt: "使用 $ai-app-bridge-use 观察、操作和验证 Android native、Android WebView/H5/CDP、Flutter、iOS native/WKWebView/WDA/XCUITest 或桌面 Web Bridge;先用 capabilities 查 core/app/action/flutter/webview/ios/web/diagnostics/advanced,再用 run 执行。Android 传 packageName,iOS 传 bundleId/deviceId 并在 full-control 时使用 WDA,Web 先建立 sessionId,只有移动端动态画面需要稳定证据时才使用 freeze/thaw。"
|