@mobileaidev/ai-app-bridge 0.3.5 → 0.3.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -3
- package/bin/android-permissions.js +4 -2
- package/bin/command-discovery.js +8 -2
- package/bin/command-registry.js +18 -2
- package/bin/device-provider.js +53 -10
- package/bin/execution-host.js +18 -0
- package/bin/executors/android-host.js +201 -0
- package/bin/executors/android-port.js +114 -0
- package/bin/executors/automation-owner.js +42 -0
- package/bin/executors/command-schema.js +102 -0
- package/bin/executors/flutter-host.js +123 -0
- package/bin/executors/managed-runtime.js +85 -0
- package/bin/executors/playwright-host.js +146 -0
- package/bin/executors/playwright-worker.js +307 -0
- package/bin/executors/receipt-journal.js +56 -0
- package/bin/feedback-probe.js +27 -1
- package/bin/ios-provider.js +8 -3
- package/bin/ios-runtime-binding.js +1 -1
- package/bin/runtime-directory.js +1 -1
- package/bin/shared-kernel/device-ownership-recovery.js +8 -0
- package/bin/shared-kernel/native-target.js +11 -8
- package/bin/shared-kernel/uia-runtime-port.js +36 -0
- package/bin/ui-observation.js +29 -0
- package/bin/web-provider.js +6 -1
- package/docs/COMMAND_CONTRACT.md +38 -1
- package/docs/OPTIONAL_EXECUTORS.md +182 -0
- package/docs/RELEASE.md +35 -12
- package/package.json +5 -1
- package/runtime/executors/playwright/package-lock.json +45 -0
- package/runtime/executors/playwright/package.json +8 -0
package/docs/COMMAND_CONTRACT.md
CHANGED
|
@@ -1114,7 +1114,18 @@ metadata, not the raw transport bytes. Settlement proves the recorded execution
|
|
|
1114
1114
|
it does not turn an ambiguous action or unverified business assertion into a pass.
|
|
1115
1115
|
|
|
1116
1116
|
Native Intent `longPress`, `swipe` and `scroll` use `/v1/action/gesture-target`.
|
|
1117
|
-
The top visible
|
|
1117
|
+
The SDK selects the top visible window belonging to the current Activity on its
|
|
1118
|
+
display. Application windows use `WindowManager.LayoutParams.token` for Activity
|
|
1119
|
+
ownership; subwindows resolve that token through their parent window. Focus is
|
|
1120
|
+
not evidence of Activity ownership. A missing token, unresolved parent or unknown
|
|
1121
|
+
window type blocks selection with `native_window_metadata_unavailable`.
|
|
1122
|
+
Native tree snapshots publish this decision as `foregroundWindowId`, referring to
|
|
1123
|
+
exactly one entry in `windows`. Host summaries and semantic selection consume
|
|
1124
|
+
that identity, and SDK semantic, coordinate, gesture and H5 execution use the same
|
|
1125
|
+
window policy. Native window snapshots without this field require an updated SDK;
|
|
1126
|
+
Host does not reconstruct the decision from focus or array order.
|
|
1127
|
+
|
|
1128
|
+
A touchable, non-focusable
|
|
1118
1129
|
popup can use its focused owner with the same application window token; the SDK
|
|
1119
1130
|
does not select a background window when the popup blocks an action. Native/H5
|
|
1120
1131
|
window selection shares this rule. Native window observation carries
|
|
@@ -1640,3 +1651,29 @@ explicit state assertions, or request Agent help through `ctx.askAgent`.
|
|
|
1640
1651
|
`ios-setup --start-wda true` records the exact XCTest invocation and result bundle before starting the device test. A completed XCTest failure that explicitly says UI test initialization timed out while enabling automation is a settled failed startup (`ios_wda_automation_confirmation_required`). The Runner launch was dispatched; this does not undo its installation. Confirm Enable UI Automation on the device, then explicitly run setup again.
|
|
1641
1652
|
|
|
1642
1653
|
`ios-execution --operation reconcile --device-id DEVICE` reads the original result bundle after an interrupted Host. For a legacy generic `ios-setup` marker, supply `--setup-result-path ORIGINAL_RESPONSE.json`: recovery requires the original CLI JSON response (or its unwrapped value), its matching retained Host action in the current FactStore, the same device and startup interval, and the original XCTest bundle identified by that response. Missing, mismatched, or generic test failures remain unresolved. This option is accepted only for SDK/command reconciliation, never action cancellation or WDA session recovery. Lock files and original evidence are retained.
|
|
1654
|
+
|
|
1655
|
+
### Bounded SDK UI observation
|
|
1656
|
+
|
|
1657
|
+
`ui-observation`, `ios-ui-observation` and `web-ui-observation` expose
|
|
1658
|
+
`start`, `status` and `stop`. Start requires `durationMs` in 100–5000; stop
|
|
1659
|
+
requires the returned `leaseId`. Android/iOS accept `provider: native|flutter`.
|
|
1660
|
+
The capability is available to JS and Python Script under `capture.read`.
|
|
1661
|
+
There is no unbounded lease, automatic renewal or activation at SDK startup.
|
|
1662
|
+
Closing/reopening a window establishes a new baseline; an idle interval is not
|
|
1663
|
+
evidence of an unchanged UI. Lifecycle and explicit business captures remain
|
|
1664
|
+
independent of heavy UI observation.
|
|
1665
|
+
|
|
1666
|
+
Native and Flutter controls use `/v1/ui/observation` and
|
|
1667
|
+
`/v1/flutter/observation`. The response schema is `aab.ui-observation/v1`.
|
|
1668
|
+
`GET /v1/flutter/snapshot` explicitly pulls fresh Dart UI state. Flutter tree,
|
|
1669
|
+
node and selection commands use it. `GET /v1/status` does not pull a Flutter
|
|
1670
|
+
tree or publish cached layout as current UI. Query a provider's observation
|
|
1671
|
+
command for its live lease state. These endpoints require rebuilt SDKs;
|
|
1672
|
+
installing a new CLI cannot patch an installed application's old SDK.
|
|
1673
|
+
|
|
1674
|
+
Ordinary CLI/MCP `feedback=full` opens a window before the action and releases
|
|
1675
|
+
it in finally. If observation is unavailable, the action is rejected before
|
|
1676
|
+
dispatch; acquiring evidence does not mark the enclosing UI action dispatched.
|
|
1677
|
+
Launch retains its independent system feedback path. Script/Intent bypass this
|
|
1678
|
+
feedback wrapper and explicitly request windows only when their evidence needs
|
|
1679
|
+
them. SDK/page expiry handles a Host that disappears without sending stop.
|
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
# Optional UI executors (0.3.7)
|
|
2
|
+
|
|
3
|
+
Bridge keeps its existing SDK paths and exposes optional executors through `capabilities`, `run`, and JavaScript/Python Script. Select an executor explicitly. No command silently changes a touch into a setter, switches framework after failure, or repeats an uncertain action.
|
|
4
|
+
|
|
5
|
+
## What is installed
|
|
6
|
+
|
|
7
|
+
| Capability | Dependency location | Integration requirement |
|
|
8
|
+
| --- | --- | --- |
|
|
9
|
+
| Android UI Automator / Espresso | Application **androidTest** dependencies and its generated test APK | Small precompiled test entry; matching installed application and test packages |
|
|
10
|
+
| Android WebView H5 | Optional `ai-app-bridge-test-espresso-web` in androidTest | JavaScript already enabled by the application; explicit WebView and frame selection |
|
|
11
|
+
| Android Compose | Optional `ai-app-bridge-test-compose` in androidTest | Application-matched Compose test runtime and a JUnit rule surrounding the session |
|
|
12
|
+
| Flutter WidgetTester | `ai_app_bridge_test` in **dev_dependencies**, using Flutter SDK test packages | Debug APK built from `integration_test/bridge_test.dart` |
|
|
13
|
+
| Web Playwright | CLI-managed directory outside the page SDK | Explicitly prepare the pinned Playwright package and matching browser |
|
|
14
|
+
| iOS | Existing WDA / XCUITest integration | Existing signing, device and WDA prerequisites; no new iOS executor in this version |
|
|
15
|
+
|
|
16
|
+
These dependencies are isolated from ordinary production source sets. They are visible in build metadata and diagnostics. They do not disappear from the installation or compatibility requirements. Android does **not** need a separate business App, repository, or Gradle project. The standard test APK is an Android test artifact.
|
|
17
|
+
|
|
18
|
+
## Android: one entry in the existing application
|
|
19
|
+
|
|
20
|
+
Add the selected artifacts to the application's `androidTestImplementation` configuration. All Bridge artifacts use the same version:
|
|
21
|
+
|
|
22
|
+
```kotlin
|
|
23
|
+
android {
|
|
24
|
+
defaultConfig {
|
|
25
|
+
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
|
|
26
|
+
}
|
|
27
|
+
}
|
|
28
|
+
dependencies {
|
|
29
|
+
androidTestImplementation("com.github.mobileAiDev.ai-app-bridge:ai-app-bridge-test-instrumentation:0.3.7")
|
|
30
|
+
// Optional H5 adapter:
|
|
31
|
+
androidTestImplementation("com.github.mobileAiDev.ai-app-bridge:ai-app-bridge-test-espresso-web:0.3.7")
|
|
32
|
+
}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Place this tiny entry in the existing `app/src/androidTest/java/...` directory:
|
|
36
|
+
|
|
37
|
+
```java
|
|
38
|
+
package example.app;
|
|
39
|
+
import io.github.mobileaidev.aiappbridge.executor.instrumentation.AndroidExecutorTest;
|
|
40
|
+
public final class BridgeSessionTest extends AndroidExecutorTest {}
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
This is a precompiled, long-running JUnit test. Host passes the session identity, token, Activity and lease as Instrumentation arguments. The test thread executes incoming commands serially. Python/JS scripts do not become Kotlin/Java source and do not require rebuilding for each workflow. You may extend the test's `adapters()` and add ordinary JUnit rules or idling resources. Arbitrary business field reflection and arbitrary existing `@Test` method invocation are not exposed as remote commands.
|
|
44
|
+
|
|
45
|
+
Opening a session **restarts and instruments the target application**. Install the matching main and test APKs built from this application first:
|
|
46
|
+
|
|
47
|
+
```sh
|
|
48
|
+
./gradlew :app:assembleDebug :app:assembleDebugAndroidTest
|
|
49
|
+
adb -s DEVICE install -r -t app/build/outputs/apk/debug/app-debug.apk
|
|
50
|
+
adb -s DEVICE install -r -t app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
|
|
51
|
+
ai-app-bridge android-executor --operation status --serial DEVICE
|
|
52
|
+
ai-app-bridge android-executor --operation open --serial DEVICE \
|
|
53
|
+
--package-name example.app \
|
|
54
|
+
--instrumentation example.app.test/androidx.test.runner.AndroidJUnitRunner \
|
|
55
|
+
--test-class example.app.BridgeSessionTest --activity example.app.MainActivity
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Use the actual installed component names returned by `status`. Supply the returned `sessionId` and `runtimeEpoch` to subsequent operations. `observe` requires an `engine` listed in `open.capabilities.adapters`. `act` requires the current `snapshotId` and a `nodeId` from that observation. Changing adapters invalidates the previous snapshot.
|
|
59
|
+
|
|
60
|
+
The instrumentation artifact includes `uiautomator` and `espresso`. Add H5 in `adapters()` with `adapters.put("espresso-web", new EspressoWebExecutor())`, after obtaining `super.adapters()`. H5 observation accepts `webView: {by:"description",value:"..."}` or `resourceId`, and `framePath: [{name:"..."}]` or `{index:0}` entries. An unspecified WebView must match exactly one WebView. JavaScript is not automatically enabled. `webClick`, `webKeys`, `webClear` and `webScrollIntoView` use WebDriver JavaScript atoms, not Android IME or physical touch. Open Shadow DOM and cross-origin frame support must be checked for the concrete WebView; closed Shadow DOM is not advertised.
|
|
61
|
+
|
|
62
|
+
## Dependency checks and Compose
|
|
63
|
+
|
|
64
|
+
Apply `io.github.mobileaidev.aiappbridge.test` from the same Gradle plugin artifact. It only validates resolved dependencies; it adds no UI bytecode instrumentation and does not rewrite versions. With JitPack, map this plugin ID in `pluginManagement.resolutionStrategy.eachPlugin` to `com.github.mobileAiDev.ai-app-bridge:ai-app-bridge-gradle-plugin:<requested version>`, as for the existing `io.github.mobileaidev.aiappbridge.android` plugin. Include JitPack, Google Maven and Maven Central in the appropriate repositories.
|
|
65
|
+
|
|
66
|
+
The initial Android adapter profile is runner **1.7.0**, Espresso / Espresso-Web **3.7.0**, UI Automator **2.4.0**, Java **11**, minSdk **23**, compileSdk **35**. The source build uses AGP **8.9.1** and JDK **17**. This is a supported profile, not a promise that every old Gradle/AGP combination can consume the artifacts.
|
|
67
|
+
|
|
68
|
+
Compose is optional and its runtime is supplied by the consumer. The initial Compose adapter profile is **1.8.3**. Apply the application's **same Compose BOM** to its main/debug and androidTest dependencies, then add `ui-test-junit4` without an independent version. The plugin rejects a different main/test Compose runtime or an unvalidated adapter profile. It does not upgrade an application's Kotlin, Compose, AGP or compileSdk to make the check pass. An application outside this profile can keep the existing SDK/UI Automator route or provide a separately validated adapter build.
|
|
69
|
+
|
|
70
|
+
```java
|
|
71
|
+
@Rule public final ComposeTestRule compose =
|
|
72
|
+
AndroidComposeTestRule_androidKt.createEmptyComposeRule();
|
|
73
|
+
@Override protected Map<String, ExecutorAdapter> adapters() {
|
|
74
|
+
Map<String, ExecutorAdapter> adapters = super.adapters();
|
|
75
|
+
adapters.put("compose", new ComposeExecutor(compose));
|
|
76
|
+
return adapters;
|
|
77
|
+
}
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
The rule must start **before** Activity/composition creation. The adapter uses automatic test clock advancement and idle frame ticks; animation timing is not wall-clock fidelity. `click` uses Compose touch input; `composeInput`, `composeReplaceText`, `composeClearText`, scroll semantics and `semanticLongClick` are explicit semantics operations. WebView/platform Views and system dialogs require their corresponding adapters. No private Compose field reflection is used by Bridge.
|
|
81
|
+
|
|
82
|
+
## Flutter
|
|
83
|
+
|
|
84
|
+
Add `ai_app_bridge_test: 0.3.7` to the application's `dev_dependencies`. The helper takes `flutter_test` and `integration_test` from the **same Flutter SDK** as the application. It is a Dart test helper, not an additional Android plugin with its own AGP/Kotlin versions.
|
|
85
|
+
|
|
86
|
+
The helper declares Flutter **>=3.41.0** and Dart **>=3.11.0 <4.0.0**. This release was built and exercised with Flutter **3.41.9 on Android API 25** and **3.44.8 on Android API 36**. These are the verified combinations; newer SDK versions still need validation with the application's plugin graph.
|
|
87
|
+
|
|
88
|
+
```dart
|
|
89
|
+
// integration_test/bridge_test.dart
|
|
90
|
+
import 'package:ai_app_bridge_test/ai_app_bridge_test.dart';
|
|
91
|
+
import 'package:your_app/main.dart' as app;
|
|
92
|
+
void main() => aiAppBridgeTest(app.main);
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
```sh
|
|
96
|
+
flutter build apk --debug --target integration_test/bridge_test.dart \
|
|
97
|
+
--dart-define=INTEGRATION_TEST_SHOULD_REPORT_RESULTS_TO_NATIVE=false
|
|
98
|
+
adb -s DEVICE install -r -t build/app/outputs/flutter-apk/app-debug.apk
|
|
99
|
+
ai-app-bridge flutter-executor --operation open --serial DEVICE \
|
|
100
|
+
--package-name example.app --activity example.app.MainActivity
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
This version supports the standard **Android Flutter embedder**, with its normal application private cache directory. Launch configuration and original receipts use `no_backup/ai-app-bridge-integration`, outside evictable caches. Opening restarts the test-entrypoint app. Closing drains the tester queue and terminates only the original process, identified by boot ID, PID and process start time. The helper uses a fully live frame policy. `tap`, `longPress`, drag/fling, `ensureVisible`, `pageBack`, `pump` and `enterText` are exposed. `enterText` changes Flutter editing state and reads it back; it does not prove native IME input. Flutter platform views, WebView DOM and system dialogs require another applicable provider. Flutter iOS/desktop/web test-host launch is not implemented by this helper.
|
|
104
|
+
|
|
105
|
+
## Web
|
|
106
|
+
|
|
107
|
+
```sh
|
|
108
|
+
ai-app-bridge web-executor --operation status --browser chromium
|
|
109
|
+
ai-app-bridge web-executor --operation prepare --browser chromium --timeout-ms 300000
|
|
110
|
+
ai-app-bridge web-executor --operation open --url http://localhost:3000 --browser chromium
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
The CLI manages exact Playwright **1.63.0**, an included npm lock file and matching browser downloads. `prepare` is explicit; normal SDK usage does not download browsers. Cache keys include the dependency lock digest, OS and CPU architecture. Node **26.3.x** is the verified Host baseline (`>=26.3.0 <27` contract). Browser preparation is serialized and reports failures. `status` separates the static version from actual executable availability.
|
|
114
|
+
|
|
115
|
+
`observe` returns pages and documents. Pass the observed `targetId`, `frameId` and `documentId` to actions/waits. A navigation during automatic waiting cannot move an old command into the replacement document. Selectors must be unique; role names and text matching are exact. `css` accepts standard CSS syntax, not Playwright selector chains. Actions include click, doubleClick, hover, fill, type, press, check, select, drag, scrollIntoView and upload. Physical `wheel` is not exposed: Playwright's page mouse API cannot bind the event atomically to the observed document. It is never replaced with a synthetic DOM event. Dialogs default to dismiss; an action may explicitly request accept/dismiss. Browser dialogs, popup pages, open Shadow DOM and iframe observations are supported; native OS dialogs and closed Shadow DOM are outside this executor.
|
|
116
|
+
|
|
117
|
+
Control `text` contains the element's `innerText`, preserving an empty string. It is `null` for elements without `innerText`; hidden `textContent` is not substituted. Each action ID occupies one slot across all pages in the session. Reusing it on another page is rejected with `executor_receipt_identity_mismatch`; changing other action arguments is rejected with `idempotency_conflict`.
|
|
118
|
+
|
|
119
|
+
## Agent, Python and JS contract
|
|
120
|
+
|
|
121
|
+
Discover only the operation needed:
|
|
122
|
+
|
|
123
|
+
```json
|
|
124
|
+
{"command":"android-executor","operation":"act"}
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
This is a `capabilities` selection, not an action. Execute through the existing `run` tool, or `ctx.call` in Script with **`app.test`** permission. Script targets remain `platform:"android"` for Android/Flutter Android, and `platform:"web"` for browser sessions. Device/package targeting is inherited from the Script target; session identities are explicit inputs.
|
|
128
|
+
|
|
129
|
+
```javascript
|
|
130
|
+
// Within module.exports.main = async ctx => { ... }
|
|
131
|
+
const reply = await ctx.call('android-executor', {
|
|
132
|
+
...ctx.inputs.identity, operation: 'observe', engine: 'espresso'
|
|
133
|
+
});
|
|
134
|
+
if (!reply.ok || !reply.result.ok) throw new Error(JSON.stringify(reply));
|
|
135
|
+
const observation = reply.result.observation;
|
|
136
|
+
const buttons = observation.nodes.filter(n => n.text === 'Submit');
|
|
137
|
+
if (buttons.length !== 1) throw new Error('Expected exactly one Submit button');
|
|
138
|
+
const result = await ctx.call('android-executor', {
|
|
139
|
+
...ctx.inputs.identity, operation: 'act', snapshotId: observation.snapshotId,
|
|
140
|
+
actionId: 'submit-once', action: {type: 'click', nodeId: buttons[0].nodeId}
|
|
141
|
+
});
|
|
142
|
+
if (!result.ok || !result.result.ok) throw new Error(JSON.stringify(result));
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
```python
|
|
146
|
+
reply = ctx.call('flutter-executor', {**ctx.inputs['identity'], 'operation': 'observe'})
|
|
147
|
+
assert reply['ok'] and reply['result']['ok'], str(reply)
|
|
148
|
+
observation = reply['result']['observation']
|
|
149
|
+
nodes = [n for n in observation['nodes'] if n.get('key') == 'submit']
|
|
150
|
+
assert len(nodes) == 1
|
|
151
|
+
result = ctx.call('flutter-executor', {
|
|
152
|
+
**ctx.inputs['identity'], 'operation': 'act',
|
|
153
|
+
'snapshotId': observation['snapshotId'], 'actionId': 'submit-once',
|
|
154
|
+
'action': {'type': 'tap', 'nodeId': nodes[0]['nodeId']}
|
|
155
|
+
})
|
|
156
|
+
assert result['ok'] and result['result']['ok'], str(result)
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
An action receipt means the framework call ended, not that a payment/order/business transaction succeeded. Observe again and assert the independent business result. Do not reuse stale snapshots or change an action's mechanism after a failure without a new decision.
|
|
160
|
+
|
|
161
|
+
## Cancellation, recovery and lifecycle
|
|
162
|
+
|
|
163
|
+
- One test thread / browser worker serializes UI effects. Native device effects use the shared device ownership contract. UI Automator and the legacy UIA runtime cannot own Android UiAutomation simultaneously; the Host drains and hands over the connection explicitly.
|
|
164
|
+
- `timeoutMs` requests cancellation; it does not prove rollback. Already executing framework work must return its original receipt or the original process must be confirmed ended. An unsettled WebView atom ends the test session. No replacement action starts against an unresolved device effect.
|
|
165
|
+
- The same `actionId` with identical arguments returns its original receipt. A changed request is rejected. An unresolved receipt is never redispatched. Deduplication is session-scoped; it is not universal exactly-once business execution across process loss or new sessions.
|
|
166
|
+
- `receipt` remains readable after normal close. Host restart can recover Android/Flutter descriptors and read retained device records; a lost browser worker's original receipts remain readable, but its live browser session cannot be resumed.
|
|
167
|
+
- Use explicit `close` for normal teardown. Android/Flutter tests also have an idle lease (default 10 minutes, selectable 10 seconds–1 hour). Host loss preserves original state for recovery; it does not erase unresolved ownership. Run `device-ownership --operation reconcile --serial DEVICE` when required by the error. A closed Flutter test may require reconciliation/close to end its original app process.
|
|
168
|
+
- Each session permits at most 4096 action receipts, with bounded request/response and observation sizes. Receipts are retained for audit; storage is not automatically deleted across sessions. Operators must archive/remove closed session artifacts according to their retention policy. Never remove an unresolved session's records to bypass a device lock.
|
|
169
|
+
|
|
170
|
+
## Performance evidence
|
|
171
|
+
|
|
172
|
+
On the same API-36 device, sample APK, Activity, counter button and Instrumentation lifetime, 20 effective samples per path (two warmup rounds excluded) alternated Espresso touch and the existing SDK touch. Both included Host persistence and the same Espresso observations before/after every action. Every counter increment was verified.
|
|
173
|
+
|
|
174
|
+
| Metric (ms) | Espresso | Existing SDK touch |
|
|
175
|
+
| --- | ---: | ---: |
|
|
176
|
+
| Median action | 362 | 314 |
|
|
177
|
+
| Action p95 | 397 | 345 |
|
|
178
|
+
| Median action + before/after observation | 413 | 359 |
|
|
179
|
+
|
|
180
|
+
This does not establish a general speedup. Startup, dependency preparation, screenshots, model time and CLI process startup are excluded and must be reported separately. Semantics/setter input is a different operation from physical keyboard/IME input; its shorter duration cannot establish an equivalent-interaction speedup.
|
|
181
|
+
|
|
182
|
+
The version's local verification and publication status are recorded in the repository delivery report. Local artifact checks are separate from public registry publication and real customer workflow acceptance.
|
package/docs/RELEASE.md
CHANGED
|
@@ -1,8 +1,27 @@
|
|
|
1
|
-
# 0.3.
|
|
1
|
+
# 0.3.7 发行与接入交接
|
|
2
2
|
|
|
3
3
|
本文件记录正式版的依赖关系和出仓库交付入口。封版要求是同一提交的源码、发行包与公开接入合同一致;单个样本的测试进度不改变包版本或发布状态。推送 Git、创建远端标签及发布 npm/pub 包由维护者执行。
|
|
4
4
|
|
|
5
|
-
## 0.3.
|
|
5
|
+
## 0.3.7 修复
|
|
6
|
+
|
|
7
|
+
- Android Host 读取全部焦点记录,跳过 `null`,不再依赖记录排列顺序。只有一个有效窗口时直接使用;多个窗口通过系统 `mTopFocusedDisplayId` 选择,仍不能唯一定位时返回 `foreground_ambiguous`。
|
|
8
|
+
- 包名匹配、SDK 窗口与控件身份校验沿用既有路径。App SDK 只同步发行版本。
|
|
9
|
+
|
|
10
|
+
## 随本版包含的 0.3.6 能力
|
|
11
|
+
|
|
12
|
+
- Android、iOS、Flutter、Web 与内嵌 H5 的持续 UI 观察默认关闭;仅按需开启 100–5000 ms 的观察窗口,期限到达、主动停止或相应生命周期退出时清理观察任务。新增三个平台观察控制命令,公开命令共 121 个。
|
|
13
|
+
- Flutter 当前树改为显式读取 `/v1/flutter/snapshot`,状态请求不再触发树遍历;CLI/MCP 与 App 内 SDK 需要配套升级。Android 指纹编码减少 JNI 调用与临时分配。实测证据及边界见仓库中的 UI 观察性能评估记录。
|
|
14
|
+
|
|
15
|
+
- 可选 Android Instrumentation 会话,复用业务 androidTest;UI Automator、Espresso、Espresso-Web 和 Compose 可在同一会话内选择。
|
|
16
|
+
- 可选 Flutter integration_test/WidgetTester 测试入口,以及 Host 管理的 Playwright 1.63.0 浏览器执行器。
|
|
17
|
+
- 公开 capabilities + run、Python/JS Script 的 app.test 权限、观测身份、原始回执、取消和设备占用接线。
|
|
18
|
+
- Compose 主包/测试包版本检查;按测试配置隔离依赖,不自动升级业务 AGP/Kotlin/Compose。
|
|
19
|
+
- Android 7 权限输出的零 flags 省略及空格分隔格式支持,来源为实际 API-25 环境和 AOSP Settings 输出实现。
|
|
20
|
+
- 本地交付和外部发布分开记录;这份源码不表示 npm/JitPack/pub.dev 已发布 0.3.7。
|
|
21
|
+
|
|
22
|
+
接入合同、具体依赖和实测性能见 [OPTIONAL_EXECUTORS.md](OPTIONAL_EXECUTORS.md)。
|
|
23
|
+
|
|
24
|
+
## 随本版包含的 0.3.5 修复
|
|
6
25
|
|
|
7
26
|
- 修正 UIA nodeRef 原回执对账,正常重启和死进程后的显式启动可完成原 session 审计;未知回执和失效引用仍受保护。
|
|
8
27
|
- UIA node runtime 支持 Android API 25+,保留 POSIX 原子重命名、fsync、进程锁与原始 Binder 回调;不使用 dump 或坐标回退。Android 7 权限观察支持未初始化的权限状态以及旧 ActivityManager 的身份字段。
|
|
@@ -20,16 +39,20 @@
|
|
|
20
39
|
- CLI `--version`、录制目录、错误字段、Script 起步权限及 Reader UIA 只读重试修正;明确 freeze 与 MCP 重连语义。
|
|
21
40
|
- Android Gradle 无实现的历史开关发出弃用提示,旧 DSL 仍可构建;各端统一版本。
|
|
22
41
|
|
|
42
|
+
## 新增可选分发物
|
|
43
|
+
|
|
44
|
+
Android `ai-app-bridge-test-core`、`ai-app-bridge-test-uia`、`ai-app-bridge-test-espresso`、`ai-app-bridge-test-instrumentation`、`ai-app-bridge-test-espresso-web`、`ai-app-bridge-test-compose` 均使用 0.3.7,通过 androidTestImplementation 消费。Gradle 插件增加 `io.github.mobileaidev.aiappbridge.test` 依赖校验入口。新 Flutter 包 `ai_app_bridge_test` 使用 0.3.7,只作为 dev_dependency;它的发布不依赖 Android SDK 的 JitPack 坐标。
|
|
45
|
+
|
|
23
46
|
## 版本与消费方式
|
|
24
47
|
|
|
25
48
|
| 交付物 | 发行版本 | 独立消费入口 | 发布依赖 |
|
|
26
49
|
| --- | --- | --- | --- |
|
|
27
|
-
| Android SDK | `0.3.
|
|
28
|
-
| Android Gradle 插件 | `0.3.
|
|
29
|
-
| 原生 iOS SDK | Git tag `0.3.
|
|
30
|
-
| Flutter 插件 | `0.3.
|
|
31
|
-
| Desktop CLI/MCP | `0.3.
|
|
32
|
-
| Web SDK | `0.3.
|
|
50
|
+
| Android SDK | `0.3.7` | JitPack `com.github.mobileAiDev.ai-app-bridge:ai-app-bridge-android:0.3.7` | 同名 Git tag,JitPack 对该提交成功构建 |
|
|
51
|
+
| Android Gradle 插件 | `0.3.7` | JitPack `ai-app-bridge-gradle-plugin` 模块及插件 ID `io.github.mobileaidev.aiappbridge.android` | 与 SDK 相同的 Git tag;不再使用旧默认 `0.2.8` |
|
|
52
|
+
| 原生 iOS SDK | Git tag `0.3.7` | Git URL 的仓库根 `Package.swift`,产品 `AiAppBridgeIOS` | 根清单包含 Swift runtime、C adapter 和 segmented C store,无外部 C 包路径 |
|
|
53
|
+
| Flutter 插件 | `0.3.7` | pub `ai_app_bridge_flutter` | Android 固定依赖上述 SDK;iOS Swift/C 源码随插件分发 |
|
|
54
|
+
| Desktop CLI/MCP | `0.3.7` | npm `@mobileaidev/ai-app-bridge` | 包含 UIA bundle、WDA 模板和 native store 源码;WDA 上游固定 `14.1.1` |
|
|
55
|
+
| Web SDK | `0.3.7` | npm `@mobileaidev/ai-app-bridge-web` | 独立浏览器源码包,无 npm 对 CLI 的安装依赖 |
|
|
33
56
|
| Native store | `0.1.0` | 随 CLI 的 bundled dependency 安装 | 不要求另行发布到 npm;`file:../../native/segmented-fact-store` 是工作区构建入口,最终 tarball 必须包含该依赖源码 |
|
|
34
57
|
|
|
35
58
|
Flutter 的 podspec 是随 pub 插件消费的本地 podspec,不是独立 CocoaPods trunk 发布包;原生 iOS 使用根 Swift package。Flutter SwiftPM 的 `../FlutterFramework` 由 Flutter 的集成生成,不能当作本仓库的外部私有依赖,也不应将本机 Flutter framework 打包进插件。
|
|
@@ -38,11 +61,11 @@ Host 支持范围声明为 Node `>=26.3.0 <27`,本轮实际验证基线是 **2
|
|
|
38
61
|
|
|
39
62
|
## 发布顺序
|
|
40
63
|
|
|
41
|
-
1. 完成源码审阅并冻结一个提交,核对以下命令的产物确实来自它;包含当前 untracked 的实际源码、测试和文档,排除本机生成目录。所有对外发行版本使用同一个 `0.3.
|
|
42
|
-
2. 维护者推送提交与 `0.3.
|
|
64
|
+
1. 完成源码审阅并冻结一个提交,核对以下命令的产物确实来自它;包含当前 untracked 的实际源码、测试和文档,排除本机生成目录。所有对外发行版本使用同一个 `0.3.7`,若需要改版本,先同时更新上表涉及的 manifest 与固定依赖。
|
|
65
|
+
2. 维护者推送提交与 `0.3.7` 标签,让 JitPack 构建 Android SDK/插件。确认两条公开坐标可解析后,再发布依赖它们的 Flutter 包。本地 Gradle project/path/AAR 替换不能证明 JitPack 坐标可消费。
|
|
43
66
|
3. 原生 iOS 消费相同 Git tag 的根 package;完成根 package 的 iOS 构建,不仅构建 `ios/ai-app-bridge-ios/Package.swift`。Flutter iOS 则检查实际 pub 包内 Swift/C 源码与声明相符。
|
|
44
67
|
4. CLI 与 Web SDK 可分别发布到 npm 的 `latest` dist-tag。CLI 的 native store 已打包随行,不等待一个不存在的单独 registry 依赖。Flutter 包发布以第 2 步完成为前提。
|
|
45
|
-
5. 同步 npm `next` 指向 `0.3.
|
|
68
|
+
5. 同步 npm `next` 指向 `0.3.7`,让已有候选入口也使用本次正式版。将 GitHub `main` 与发行提交同步,并创建非预发布的 GitHub Release。
|
|
46
69
|
6. 从 registry/tag 安装刚发布的确切版本,读取 `capabilities` 和版本,核对来源及支持范围,确认默认安装入口指向本次发行版本。正式发布不自动等于全平台生产验收完成。
|
|
47
70
|
|
|
48
71
|
正式发布命令需在对应目录由维护者执行,例如 npm 使用 `npm publish --tag latest`;pub 使用 `flutter pub publish`。这些命令属于发布动作,不能混入本地验证脚本。
|
|
@@ -83,7 +106,7 @@ npm 升级不会替换已连接的 MCP 进程。用 `ai-app-bridge --version`
|
|
|
83
106
|
已有工作结束后显式停止旧 Runtime,并在 Cursor 等客户端重连 MCP,核对 initialize
|
|
84
107
|
中的 `serverInfo.version`。工具描述仍有旧 `batch`/`smoke` 时刷新客户端缓存。
|
|
85
108
|
|
|
86
|
-
实际发布状态与验证边界见仓库 `docs/RELEASE_HANDOFF_0.3.
|
|
109
|
+
实际发布状态与验证边界见仓库 `docs/RELEASE_HANDOFF_0.3.7_2026-09-15.md`。
|
|
87
110
|
Android Gradle 插件的 `webSocketCaptureEnabled`、`logInstrumentationEnabled`、
|
|
88
111
|
`webViewDebuggingEnabled` 没有对应插桩实现,现明确弃用并在显式设置时输出提示;
|
|
89
112
|
旧配置仍可构建。当前有效开关是 `enabled`、`okHttpCaptureEnabled`,以及可选
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mobileaidev/ai-app-bridge",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.7",
|
|
4
4
|
"description": "Desktop CLI and MCP server for AI App Bridge across Android, iOS, Flutter, WebView, and Web targets.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -26,6 +26,7 @@
|
|
|
26
26
|
"bin/fact-recorder.js",
|
|
27
27
|
"bin/fact-store.js",
|
|
28
28
|
"bin/feedback-probe.js",
|
|
29
|
+
"bin/ui-observation.js",
|
|
29
30
|
"bin/android-uia-xml.js",
|
|
30
31
|
"bin/android-permissions.js",
|
|
31
32
|
"bin/mmap-scan-index.js",
|
|
@@ -44,6 +45,8 @@
|
|
|
44
45
|
"bin/script",
|
|
45
46
|
"bin/intent",
|
|
46
47
|
"bin/shared-kernel",
|
|
48
|
+
"bin/executors",
|
|
49
|
+
"runtime/executors",
|
|
47
50
|
"runtime/uia",
|
|
48
51
|
"bin/observation-collector.js",
|
|
49
52
|
"bin/ios-provider.js",
|
|
@@ -63,6 +66,7 @@
|
|
|
63
66
|
"!bin/script/p9-*.js",
|
|
64
67
|
"!bin/intent/intent-device-adapter.js",
|
|
65
68
|
"docs/COMMAND_CONTRACT.md",
|
|
69
|
+
"docs/OPTIONAL_EXECUTORS.md",
|
|
66
70
|
"bin/ios-wda-port.js",
|
|
67
71
|
"bin/ios-wda-project.js",
|
|
68
72
|
"bin/ios-wda-execution.js",
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "ai-app-bridge-playwright-runtime",
|
|
3
|
+
"version": "0.3.7",
|
|
4
|
+
"lockfileVersion": 3,
|
|
5
|
+
"requires": true,
|
|
6
|
+
"packages": {
|
|
7
|
+
"": {
|
|
8
|
+
"name": "ai-app-bridge-playwright-runtime",
|
|
9
|
+
"version": "0.3.7",
|
|
10
|
+
"dependencies": {
|
|
11
|
+
"playwright": "1.63.0"
|
|
12
|
+
},
|
|
13
|
+
"engines": {
|
|
14
|
+
"node": ">=26.3.0 <27"
|
|
15
|
+
}
|
|
16
|
+
},
|
|
17
|
+
"node_modules/playwright": {
|
|
18
|
+
"version": "1.63.0",
|
|
19
|
+
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.63.0.tgz",
|
|
20
|
+
"integrity": "sha512-+7ziBLidS4NaNCdt57SUDT+wYmmd5fmiQejUic/kb+YsYSCPyOOE9sebzMjNmQrsnNpDJqd4WHvV/8lfKfUDUg==",
|
|
21
|
+
"license": "Apache-2.0",
|
|
22
|
+
"dependencies": {
|
|
23
|
+
"playwright-core": "1.63.0"
|
|
24
|
+
},
|
|
25
|
+
"bin": {
|
|
26
|
+
"playwright": "cli.js"
|
|
27
|
+
},
|
|
28
|
+
"engines": {
|
|
29
|
+
"node": ">=20"
|
|
30
|
+
}
|
|
31
|
+
},
|
|
32
|
+
"node_modules/playwright-core": {
|
|
33
|
+
"version": "1.63.0",
|
|
34
|
+
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.63.0.tgz",
|
|
35
|
+
"integrity": "sha512-rYCsBF/M5HjUch52bbtVONEFjv6Xu8sm8h72dNlR5bzIE1fvC/bxgspzkjSfU+MweEMmPM8KJebG6nnyxo5mCg==",
|
|
36
|
+
"license": "Apache-2.0",
|
|
37
|
+
"bin": {
|
|
38
|
+
"playwright-core": "cli.js"
|
|
39
|
+
},
|
|
40
|
+
"engines": {
|
|
41
|
+
"node": ">=20"
|
|
42
|
+
}
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
}
|