@mobileaidev/ai-app-bridge 0.2.15 → 0.3.0-rc.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +201 -0
- package/NOTICE +7 -0
- package/README.md +306 -53
- package/bin/ai-app-bridge.js +51 -4504
- package/bin/android-permissions.js +152 -0
- package/bin/android-uia-xml.js +74 -0
- package/bin/artifact-paths.js +127 -1
- package/bin/bridge-forward.js +56 -0
- package/bin/command-discovery.js +86 -0
- package/bin/command-errors.js +60 -0
- package/bin/command-registry.js +504 -0
- package/bin/command-request.js +14 -0
- package/bin/command-router.js +80 -0
- package/bin/connection-cache.js +119 -0
- package/bin/device-provider.js +2934 -0
- package/bin/execution-host.js +596 -0
- package/bin/execution-runtime.js +101 -0
- package/bin/fact-codec.js +321 -0
- package/bin/fact-recorder.js +691 -0
- package/bin/fact-store.js +226 -0
- package/bin/feedback-probe.js +285 -0
- package/bin/intent/install-intent.js +267 -0
- package/bin/intent/intent-action-executor.js +129 -0
- package/bin/intent/intent-autonomous-adapter.js +46 -0
- package/bin/intent/intent-capture-port.js +64 -0
- package/bin/intent/intent-entry.js +226 -0
- package/bin/intent/intent-errors.js +28 -0
- package/bin/intent/intent-evidence-store.js +121 -0
- package/bin/intent/intent-lifetime.js +62 -0
- package/bin/intent/intent-observation-target.js +33 -0
- package/bin/intent/intent-observer.js +194 -0
- package/bin/intent/intent-production-adapter.js +334 -0
- package/bin/intent/intent-provider.js +27 -0
- package/bin/intent/intent-runtime.js +63 -0
- package/bin/intent/intent-worker.js +432 -0
- package/bin/intent/ios-intent-adapter.js +90 -0
- package/bin/intent/permission-intent.js +239 -0
- package/bin/intent/web-intent-adapter.js +31 -0
- package/bin/ios-device-outcome.js +47 -0
- package/bin/ios-execution.js +108 -0
- package/bin/ios-provider.js +497 -583
- package/bin/ios-runtime-binding.js +54 -0
- package/bin/ios-wda-execution.js +70 -0
- package/bin/ios-wda-port.js +98 -0
- package/bin/ios-wda-project.js +79 -0
- package/bin/mcp-server.js +83 -1040
- package/bin/mmap-scan-index.js +329 -0
- package/bin/observation-collector.js +862 -0
- package/bin/runtime-client.js +154 -0
- package/bin/runtime-directory.js +107 -0
- package/bin/runtime-protocol.js +36 -0
- package/bin/script/bounded-script-registry.js +103 -0
- package/bin/script/node-runtime-adapter.js +233 -0
- package/bin/script/progress-projector.js +63 -0
- package/bin/script/python-runtime-adapter.js +111 -0
- package/bin/script/rolling-summary.js +134 -0
- package/bin/script/script-agent-port.js +40 -0
- package/bin/script/script-assert.js +95 -0
- package/bin/script/script-capture-port.js +76 -0
- package/bin/script/script-catalog.js +85 -0
- package/bin/script/script-durable-restore.js +195 -0
- package/bin/script/script-entry-code.js +26 -0
- package/bin/script/script-entry-route.js +38 -0
- package/bin/script/script-entry.js +3 -0
- package/bin/script/script-errors.js +29 -0
- package/bin/script/script-evidence-store.js +22 -0
- package/bin/script/script-format-removed.js +26 -0
- package/bin/script/script-host-port.js +397 -0
- package/bin/script/script-ledger.js +64 -0
- package/bin/script/script-result.js +57 -0
- package/bin/script/script-sdk.js +152 -0
- package/bin/script/script-sdk.py +153 -0
- package/bin/script/script-session-channel.js +127 -0
- package/bin/script/script-spec.js +86 -0
- package/bin/script/script-supervisor.js +919 -0
- package/bin/script/templates/checkpoint-reentry.js +13 -0
- package/bin/segment-index.js +481 -0
- package/bin/segmented-fact-store.js +1571 -0
- package/bin/shared-kernel/android-h5-target.js +10 -0
- package/bin/shared-kernel/android-install-execution.js +176 -0
- package/bin/shared-kernel/android-sdk-endpoint.js +42 -0
- package/bin/shared-kernel/android-shell-execution.js +195 -0
- package/bin/shared-kernel/argument-schema.js +117 -0
- package/bin/shared-kernel/canonical-path.js +17 -0
- package/bin/shared-kernel/device-acknowledgements.js +53 -0
- package/bin/shared-kernel/device-completion-history.js +52 -0
- package/bin/shared-kernel/device-mutation-lease.js +219 -0
- package/bin/shared-kernel/device-ownership-recovery.js +95 -0
- package/bin/shared-kernel/device-ownership-store.js +94 -0
- package/bin/shared-kernel/evidence-adapters.js +251 -0
- package/bin/shared-kernel/evidence-archive.js +329 -0
- package/bin/shared-kernel/evidence-recording.js +131 -0
- package/bin/shared-kernel/evidence-schema.js +194 -0
- package/bin/shared-kernel/evidence-store.js +190 -0
- package/bin/shared-kernel/execution-admission.js +22 -0
- package/bin/shared-kernel/execution-contracts.js +171 -0
- package/bin/shared-kernel/execution-io.js +106 -0
- package/bin/shared-kernel/execution-ledger.js +125 -0
- package/bin/shared-kernel/execution-scope.js +87 -0
- package/bin/shared-kernel/execution-target.js +115 -0
- package/bin/shared-kernel/flutter-execution.js +13 -0
- package/bin/shared-kernel/flutter-h5-port.js +60 -0
- package/bin/shared-kernel/flutter-h5-target.js +9 -0
- package/bin/shared-kernel/flutter-target.js +75 -0
- package/bin/shared-kernel/h5-execution.js +11 -0
- package/bin/shared-kernel/h5-target.js +31 -0
- package/bin/shared-kernel/host-fact-store.js +49 -0
- package/bin/shared-kernel/ios-h5-target.js +9 -0
- package/bin/shared-kernel/ios-native-target.js +71 -0
- package/bin/shared-kernel/live-capture-query.js +115 -0
- package/bin/shared-kernel/managed-sdk-execution.js +78 -0
- package/bin/shared-kernel/native-execution.js +13 -0
- package/bin/shared-kernel/native-target.js +156 -0
- package/bin/shared-kernel/provider-command-contracts.js +55 -0
- package/bin/shared-kernel/recorded-payload-archive.js +195 -0
- package/bin/shared-kernel/request-context.js +35 -0
- package/bin/shared-kernel/semantic-node.js +55 -0
- package/bin/shared-kernel/summary-transformer.js +352 -0
- package/bin/shared-kernel/target-lease-protocol.js +47 -0
- package/bin/shared-kernel/text-wait.js +111 -0
- package/bin/shared-kernel/uia-execution.js +96 -0
- package/bin/shared-kernel/uia-protocol.js +214 -0
- package/bin/shared-kernel/uia-runtime-port.js +377 -0
- package/bin/shared-kernel/uia-target.js +39 -0
- package/bin/shared-kernel/web-dom-target.js +44 -0
- package/bin/shared-kernel/xml-attributes.js +25 -0
- package/bin/target-execution.js +275 -0
- package/bin/web/command-schema.js +60 -0
- package/bin/web/session-store.js +157 -0
- package/bin/web-provider.js +334 -553
- package/docs/COMMAND_CONTRACT.md +1563 -0
- package/docs/EVIDENCE_ARCHIVE.md +214 -0
- package/docs/INTENT_FOREGROUND.md +71 -0
- package/docs/INTENT_NATIVE_EDITING.md +79 -0
- package/docs/RELEASE.md +59 -0
- package/docs/SCRIPT_AUTHORING.md +489 -0
- package/node_modules/@mobileaidev/segmented-fact-store-native/LICENSE +201 -0
- package/node_modules/@mobileaidev/segmented-fact-store-native/NOTICE +7 -0
- package/node_modules/@mobileaidev/segmented-fact-store-native/binding.gyp +36 -0
- package/node_modules/@mobileaidev/segmented-fact-store-native/bindings/node/sfs_node.c +597 -0
- package/node_modules/@mobileaidev/segmented-fact-store-native/include/sfs.h +178 -0
- package/node_modules/@mobileaidev/segmented-fact-store-native/index.js +5 -0
- package/node_modules/@mobileaidev/segmented-fact-store-native/package.json +24 -0
- package/node_modules/@mobileaidev/segmented-fact-store-native/src/sfs.c +2349 -0
- package/package.json +60 -5
- package/runtime/ios-wda/AABWDABinding.h +19 -0
- package/runtime/ios-wda/AABWDABinding.m +97 -0
- package/runtime/ios-wda/AABWDAExecution.h +26 -0
- package/runtime/ios-wda/AABWDAExecution.m +172 -0
- package/runtime/ios-wda/AABWDAIntegration.h +71 -0
- package/runtime/ios-wda/AABWDAManagedRoutes.h +392 -0
- package/runtime/ios-wda/AABWDAReceiptStore.h +10 -0
- package/runtime/ios-wda/AABWDAReceiptStore.m +116 -0
- package/runtime/uia/ai-app-bridge-uia.jar +0 -0
- package/runtime/uia/manifest.json +22 -0
- package/skills/ai-app-bridge-use/SKILL.md +23 -360
package/LICENSE
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
10
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
11
|
+
|
|
12
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
13
|
+
the copyright owner that is granting the License.
|
|
14
|
+
|
|
15
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
16
|
+
other entities that control, are controlled by, or are under common
|
|
17
|
+
control with that entity. For the purposes of this definition,
|
|
18
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
19
|
+
direction or management of such entity, whether by contract or
|
|
20
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
21
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
22
|
+
|
|
23
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
24
|
+
exercising permissions granted by this License.
|
|
25
|
+
|
|
26
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
27
|
+
including but not limited to software source code, documentation
|
|
28
|
+
source, and configuration files.
|
|
29
|
+
|
|
30
|
+
"Object" form shall mean any form resulting from mechanical
|
|
31
|
+
transformation or translation of a Source form, including but
|
|
32
|
+
not limited to compiled object code, generated documentation,
|
|
33
|
+
and conversions to other media types.
|
|
34
|
+
|
|
35
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
36
|
+
Object form, made available under the License, as indicated by a
|
|
37
|
+
copyright notice that is included in or attached to the work
|
|
38
|
+
(an example is provided in the Appendix below).
|
|
39
|
+
|
|
40
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
41
|
+
form, that is based on (or derived from) the Work and for which the
|
|
42
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
43
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
44
|
+
of this License, Derivative Works shall not include works that remain
|
|
45
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
46
|
+
the Work and Derivative Works thereof.
|
|
47
|
+
|
|
48
|
+
"Contribution" shall mean any work of authorship, including
|
|
49
|
+
the original version of the Work and any modifications or additions
|
|
50
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
51
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
52
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
53
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
54
|
+
means any form of electronic, verbal, or written communication sent
|
|
55
|
+
to the Licensor or its representatives, including but not limited to
|
|
56
|
+
communication on electronic mailing lists, source code control systems,
|
|
57
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
58
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
59
|
+
excluding communication that is conspicuously marked or otherwise
|
|
60
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
61
|
+
|
|
62
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
63
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
64
|
+
subsequently incorporated within the Work.
|
|
65
|
+
|
|
66
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
67
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
68
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
69
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
70
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
71
|
+
Work and such Derivative Works in Source or Object form.
|
|
72
|
+
|
|
73
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
74
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
75
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
76
|
+
(except as stated in this section) patent license to make, have made,
|
|
77
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
78
|
+
where such license applies only to those patent claims licensable
|
|
79
|
+
by such Contributor that are necessarily infringed by their
|
|
80
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
81
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
82
|
+
institute patent litigation against any entity (including a
|
|
83
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
84
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
85
|
+
or contributory patent infringement, then any patent licenses
|
|
86
|
+
granted to You under this License for that Work shall terminate
|
|
87
|
+
as of the date such litigation is filed.
|
|
88
|
+
|
|
89
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
90
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
91
|
+
modifications, and in Source or Object form, provided that You
|
|
92
|
+
meet the following conditions:
|
|
93
|
+
|
|
94
|
+
(a) You must give any other recipients of the Work or
|
|
95
|
+
Derivative Works a copy of this License; and
|
|
96
|
+
|
|
97
|
+
(b) You must cause any modified files to carry prominent notices
|
|
98
|
+
stating that You changed the files; and
|
|
99
|
+
|
|
100
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
101
|
+
that You distribute, all copyright, patent, trademark, and
|
|
102
|
+
attribution notices from the Source form of the Work,
|
|
103
|
+
excluding those notices that do not pertain to any part of
|
|
104
|
+
the Derivative Works; and
|
|
105
|
+
|
|
106
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
107
|
+
distribution, then any Derivative Works that You distribute must
|
|
108
|
+
include a readable copy of the attribution notices contained
|
|
109
|
+
within such NOTICE file, excluding those notices that do not
|
|
110
|
+
pertain to any part of the Derivative Works, in at least one
|
|
111
|
+
of the following places: within a NOTICE text file distributed
|
|
112
|
+
as part of the Derivative Works; within the Source form or
|
|
113
|
+
documentation, if provided along with the Derivative Works; or,
|
|
114
|
+
within a display generated by the Derivative Works, if and
|
|
115
|
+
wherever such third-party notices normally appear. The contents
|
|
116
|
+
of the NOTICE file are for informational purposes only and
|
|
117
|
+
do not modify the License. You may add Your own attribution
|
|
118
|
+
notices within Derivative Works that You distribute, alongside
|
|
119
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
120
|
+
that such additional attribution notices cannot be construed
|
|
121
|
+
as modifying the License.
|
|
122
|
+
|
|
123
|
+
You may add Your own copyright statement to Your modifications and
|
|
124
|
+
may provide additional or different license terms and conditions
|
|
125
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
126
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
127
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
128
|
+
the conditions stated in this License.
|
|
129
|
+
|
|
130
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
131
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
132
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
133
|
+
this License, without any additional terms or conditions.
|
|
134
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
135
|
+
the terms of any separate license agreement you may have executed
|
|
136
|
+
with Licensor regarding such Contributions.
|
|
137
|
+
|
|
138
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
139
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
140
|
+
except as required for reasonable and customary use in describing the
|
|
141
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
142
|
+
|
|
143
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
144
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
145
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
146
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
147
|
+
implied, including, without limitation, any warranties or conditions
|
|
148
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
149
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
150
|
+
appropriateness of using or redistributing the Work and assume any
|
|
151
|
+
risks associated with Your exercise of permissions under this License.
|
|
152
|
+
|
|
153
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
154
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
155
|
+
unless required by applicable law (such as deliberate and grossly
|
|
156
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
157
|
+
liable to You for damages, including any direct, indirect, special,
|
|
158
|
+
incidental, or consequential damages of any character arising as a
|
|
159
|
+
result of this License or out of the use or inability to use the
|
|
160
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
161
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
162
|
+
other commercial damages or losses), even if such Contributor
|
|
163
|
+
has been advised of the possibility of such damages.
|
|
164
|
+
|
|
165
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
166
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
167
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
168
|
+
or other liability obligations and/or rights consistent with this
|
|
169
|
+
License. However, in accepting such obligations, You may act only
|
|
170
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
171
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
172
|
+
defend, and hold each Contributor harmless for any liability
|
|
173
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
174
|
+
of your accepting any such warranty or additional liability.
|
|
175
|
+
|
|
176
|
+
END OF TERMS AND CONDITIONS
|
|
177
|
+
|
|
178
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
179
|
+
|
|
180
|
+
To apply the Apache License to your work, attach the following
|
|
181
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
182
|
+
replaced with your own identifying information. (Don't include
|
|
183
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
184
|
+
comment syntax for the file format. We also recommend that a
|
|
185
|
+
file or class name and description of purpose be included on the
|
|
186
|
+
same "printed page" as the copyright notice for easier
|
|
187
|
+
identification within third-party archives.
|
|
188
|
+
|
|
189
|
+
Copyright 2026 Mobile AI Dev
|
|
190
|
+
|
|
191
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
192
|
+
you may not use this file except in compliance with the License.
|
|
193
|
+
You may obtain a copy of the License at
|
|
194
|
+
|
|
195
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
196
|
+
|
|
197
|
+
Unless required by applicable law or agreed to in writing, software
|
|
198
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
199
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
200
|
+
See the License for the specific language governing permissions and
|
|
201
|
+
limitations under the License.
|
package/NOTICE
ADDED
package/README.md
CHANGED
|
@@ -1,45 +1,86 @@
|
|
|
1
1
|
# AI App Bridge CLI
|
|
2
2
|
|
|
3
|
+
Android SDK connections read the selected App's private endpoint descriptor and
|
|
4
|
+
bind an ADB forward to its exact `localabstract:aab-sdk-<runtimeEpoch>` socket.
|
|
5
|
+
`--port` selects an optional Host port; it cannot identify an App or bypass endpoint
|
|
6
|
+
discovery. Every request checks the mapping before dispatch. Mutating requests
|
|
7
|
+
are never replayed after a missing route or uncertain result. For manual cleanup,
|
|
8
|
+
pass the exact serial and returned Host port to `remove-forward`.
|
|
9
|
+
|
|
10
|
+
This release candidate is `0.3.0-rc.2`, distributed through the npm `next`
|
|
11
|
+
dist-tag. Install this exact version or `@next` to use the candidate Script/Intent
|
|
12
|
+
and capture contracts below; `@latest` remains the previous stable release.
|
|
13
|
+
The supported Node range is `>=26.3.0 <27`; this candidate was checked on 26.3.0.
|
|
14
|
+
See [the release guide](docs/RELEASE.md) for local packaging and coordinated publication.
|
|
15
|
+
|
|
3
16
|
AI App Bridge CLI/MCP supports Android native apps, Android WebView/H5/CDP,
|
|
4
17
|
Flutter apps on Android and iOS, iOS native apps via `AiAppBridgeIOS` plus
|
|
5
18
|
WebDriverAgent/XCUITest, WKWebView, and desktop Web Bridge sessions.
|
|
6
19
|
|
|
20
|
+
Intent and Script are first-class execution interfaces. The shared command registry supplies discovery, validation and provider access. See [the command contract](docs/COMMAND_CONTRACT.md) for dispatch rules and known limits.
|
|
21
|
+
|
|
22
|
+
CLI and MCP connect to one independent local execution runtime. Every registered
|
|
23
|
+
command is available from either entrypoint, including Intent, Script, Web,
|
|
24
|
+
installation and permission flows. A CLI command can start a task and a later MCP
|
|
25
|
+
connection can observe or decide the same operation ID. Disconnecting either
|
|
26
|
+
client leaves tasks running. Use `runtime --operation status` to inspect its
|
|
27
|
+
owner, `runtime --operation stop` to drain it, and task `cancel` to stop one task.
|
|
28
|
+
|
|
29
|
+
CLI output is `{kind: "json"|"text"|"bytes", value, history?}`. Read the command
|
|
30
|
+
payload from `value`; binary values are base64. Failure uses `value.ok:false` and
|
|
31
|
+
exit code 1. MCP wraps the same payload/history in its content format. See the
|
|
32
|
+
[shared lifecycle and configuration contract](docs/COMMAND_CONTRACT.md#shared-runtime-lifecycle).
|
|
33
|
+
|
|
34
|
+
Execution operations and nested controls use strict schemas. Supply an explicit
|
|
35
|
+
`operation`, use `script.target` and canonical `javascript`/`python` languages,
|
|
36
|
+
and handle call failures in source. Progress is read through `status`/`wait`.
|
|
37
|
+
Flutter expert payloads are JSON objects; Web App actions use the explicit
|
|
38
|
+
`action` wrapper. Unknown fields fail before storage or provider access.
|
|
39
|
+
|
|
7
40
|
Command domains:
|
|
8
41
|
|
|
42
|
+
- `execution`: `intent`, `script`, `runtime`, `device-ownership`
|
|
43
|
+
- `evidence`: archive export and offline verification
|
|
44
|
+
|
|
9
45
|
- `core`: `status`, `tree`, `uia-tree`, `screenshot`, `logs`, `network`, `state`, `events`
|
|
10
46
|
- `app`: `install-apk`, `clear-app-data`, `launch-*`, `freeze-app`, `thaw-app`, `permission-*`, `appops-set`
|
|
11
|
-
- `action`: `tap`, `tap-text`, `tap-uia-text`, `input-text`, `swipe`, `keyevent`, `wait-text`, `keyboard-state`, `hide-keyboard`
|
|
47
|
+
- `action`: `tap`, `tap-text`, `tap-native`, `tap-uia`, `tap-uia-text`, `input-text`, `swipe`, `native-gesture`, `keyevent`, `wait-text`, `keyboard-state`, `hide-keyboard`
|
|
12
48
|
- `flutter`: `flutter-tree`, `flutter-nodes`, `flutter-action`, `tap-flutter-text`, `input-flutter-text`, `scroll-flutter`
|
|
13
49
|
- `webview`: `h5-*`, `flutter-h5-*`, `webview-pages`, `webview-network`, `webview-console`
|
|
14
|
-
- `ios`: `ios-devices`, `ios-doctor`, `ios-setup`, `ios-*` runtime evidence, WDA tree/tap/input/swipe, WKWebView, and Flutter iOS
|
|
15
|
-
- `web`:
|
|
16
|
-
- `diagnostics` / `advanced`: `logcat`, `
|
|
50
|
+
- `ios`: `ios-devices`, `ios-doctor`, `ios-setup`, `ios-execution` status/cancel/result/reconcile, `ios-*` runtime evidence, explicit `ios-wda-session`, WDA tree/tap/input/swipe/orientation, WKWebView, and Flutter iOS
|
|
51
|
+
- `web`: provider/session lifecycle, `web-execution`, observations and capture, `web-command`, `web-click`, `web-input`, `web-key`, `web-wait`, `web-scroll`
|
|
52
|
+
- `diagnostics` / `advanced`: `logcat`, `uia-runtime`, `forward`, `remove-forward`
|
|
17
53
|
|
|
18
|
-
|
|
54
|
+
MCP exposes exactly two tools: call `capabilities` to discover
|
|
19
55
|
domains, commands, and options, then call `run` with the selected command.
|
|
20
56
|
|
|
21
57
|
```bash
|
|
22
|
-
|
|
58
|
+
# After candidate publication; see docs/RELEASE.md for local packaging.
|
|
59
|
+
npm install -g @mobileaidev/ai-app-bridge@0.3.0-rc.2
|
|
23
60
|
|
|
24
61
|
ai-app-bridge status --package-name io.github.mobileaidev.aiappbridge.sample
|
|
25
62
|
ai-app-bridge tree --package-name io.github.mobileaidev.aiappbridge.sample
|
|
26
|
-
ai-app-bridge
|
|
27
|
-
ai-app-bridge
|
|
28
|
-
ai-app-bridge launch-
|
|
29
|
-
ai-app-bridge launch-activity --package-name io.github.mobileaidev.aiappbridge.sample --activity .MainActivity --extra route=/home
|
|
63
|
+
ai-app-bridge clear-app-data --serial DEVICE --package-name io.github.mobileaidev.aiappbridge.sample
|
|
64
|
+
ai-app-bridge launch-app --serial DEVICE --package-name io.github.mobileaidev.aiappbridge.sample
|
|
65
|
+
ai-app-bridge launch-activity --serial DEVICE --package-name io.github.mobileaidev.aiappbridge.sample --activity .MainActivity --extra route=/home
|
|
30
66
|
ai-app-bridge screenshot --package-name io.github.mobileaidev.aiappbridge.sample
|
|
31
|
-
ai-app-bridge input-text --package-name io.github.mobileaidev.aiappbridge.sample --text "中文输入" --hide-keyboard
|
|
67
|
+
ai-app-bridge input-text --serial DEVICE --package-name io.github.mobileaidev.aiappbridge.sample --text "中文输入" --hide-keyboard
|
|
32
68
|
ai-app-bridge network --package-name io.github.mobileaidev.aiappbridge.sample --compact --url-filter /api/
|
|
33
69
|
ai-app-bridge webview-network --package-name io.github.mobileaidev.aiappbridge.sample --duration-ms 3000
|
|
34
70
|
ai-app-bridge ios-devices
|
|
35
|
-
ai-app-bridge ios-doctor --device-id <device-or-udid> --bundle-id <ios.bundle.id>
|
|
71
|
+
ai-app-bridge ios-doctor --device-id <device-or-udid> --bundle-id <ios.bundle.id> --wda-runner-bundle-id <runner-from-setup>
|
|
36
72
|
ai-app-bridge ios-setup --device-id <device-or-udid> --bundle-id <ios.bundle.id> --team-id <APPLE_TEAM_ID> --start-wda
|
|
37
73
|
ai-app-bridge ios-status --device-id <device-or-udid> --bundle-id <ios.bundle.id>
|
|
38
|
-
ai-app-bridge ios-
|
|
39
|
-
ai-app-bridge ios-
|
|
40
|
-
ai-app-bridge-
|
|
41
|
-
ai-app-bridge
|
|
42
|
-
ai-app-bridge
|
|
74
|
+
ai-app-bridge ios-execution --operation status --device-id <device-or-udid> --bundle-id <ios.bundle.id>
|
|
75
|
+
ai-app-bridge ios-execution --operation reconcile --device-id <device-or-udid> --bundle-id <original.ios.bundle.id>
|
|
76
|
+
ai-app-bridge ios-execution --operation reconcile --kind wda --device-id <device-or-udid> --wda-runner-bundle-id <runner.bundle.id>
|
|
77
|
+
ai-app-bridge ios-wda-session --operation create --device-id <device-or-udid> --wda-runner-bundle-id <runner-from-setup> --bundle-id <ios.bundle.id>
|
|
78
|
+
ai-app-bridge ios-tap --device-id <device-or-udid> --wda-runner-bundle-id <runner-from-setup> --bundle-id <ios.bundle.id> --wda-session-id <created-session> --tap-x 120 --tap-y 360
|
|
79
|
+
ai-app-bridge ios-input --device-id <device-or-udid> --wda-runner-bundle-id <runner-from-setup> --bundle-id <ios.bundle.id> --wda-session-id <created-session> --accessibility-id sample_text_field --clear-first --text "hello"
|
|
80
|
+
ai-app-bridge web-session-start --web-port 18180
|
|
81
|
+
ai-app-bridge runtime --operation status
|
|
82
|
+
ai-app-bridge thaw-app --serial DEVICE --package-name io.github.mobileaidev.aiappbridge.sample
|
|
83
|
+
ai-app-bridge freeze-app --serial DEVICE --package-name io.github.mobileaidev.aiappbridge.sample
|
|
43
84
|
ai-app-bridge-mcp --help
|
|
44
85
|
```
|
|
45
86
|
|
|
@@ -53,45 +94,250 @@ directory, for example `$CODEX_HOME/skills/ai-app-bridge-use` on macOS/Linux or
|
|
|
53
94
|
`%USERPROFILE%\.codex\skills\ai-app-bridge-use` on Windows, then restart or
|
|
54
95
|
refresh the session.
|
|
55
96
|
|
|
56
|
-
MCP
|
|
97
|
+
MCP uses a compact tool surface to avoid loading every command schema
|
|
57
98
|
into the model context:
|
|
58
99
|
|
|
59
|
-
- `capabilities` lists supported targets, command domains, command names, and
|
|
100
|
+
- `capabilities` lists supported targets, command domains, command names, and exact argument schemas, execution role and supported entrypoints.
|
|
60
101
|
- `run` executes a selected command with command-specific arguments.
|
|
61
102
|
|
|
103
|
+
The `install-apk` command through CLI or MCP starts an Intent installation; system buttons are chosen
|
|
104
|
+
from actual observations and the installed APK is independently verified. See
|
|
105
|
+
[the installation contract](docs/COMMAND_CONTRACT.md#installation-is-an-intent-operation).
|
|
106
|
+
|
|
62
107
|
This keeps install, data reset, launch, UI action, Flutter, WebView/H5/CDP,
|
|
63
|
-
iOS, Web Bridge, logcat, network, permission,
|
|
64
|
-
capabilities discoverable without exposing
|
|
65
|
-
|
|
66
|
-
`
|
|
67
|
-
|
|
108
|
+
iOS, Web Bridge, logcat, network, permission, port-forward,
|
|
109
|
+
and first-class `script`/`intent` capabilities discoverable without exposing
|
|
110
|
+
dozens of full schemas at session start. `script` is trusted-local-code
|
|
111
|
+
JavaScript or Python: `permissions` gate Bridge SDK calls only and are not
|
|
112
|
+
an OS sandbox. There is no explore, export-to-script, or assemble-report command.
|
|
113
|
+
Per-command MCP aliases and the old full surface were removed.
|
|
114
|
+
All command parameters belong exclusively in `run.arguments`.
|
|
68
115
|
|
|
69
116
|
The MCP server accepts both standard `Content-Length` framed JSON-RPC messages
|
|
70
117
|
and single-line JSON messages. Responses use the format of the first request on
|
|
71
118
|
that connection, so standard MCP clients keep framed responses while local
|
|
72
119
|
Node REPL scripts can send and read one JSON object per line.
|
|
73
120
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
121
|
+
## Persistent facts and action feedback
|
|
122
|
+
|
|
123
|
+
The shared execution runtime serializes mutations for the same physical Android serial
|
|
124
|
+
across direct commands, Intent and Script. A durable device lease also arbitrates
|
|
125
|
+
across Host processes. Phone `logs`/`network`/`state`/`events` live in
|
|
126
|
+
`MobileCaptureStore` on the device. Host live commands read the phone; the
|
|
127
|
+
collector does not copy those payloads into Host history. Android registration
|
|
128
|
+
does not open a background SDK connection or poll `status`; explicit reads and
|
|
129
|
+
Intent/Script capture still use the live provider. Android App logs
|
|
130
|
+
come from that App's in-process `logs` stream on an explicit live read. The
|
|
131
|
+
collector does not attribute device-wide logcat to the App. Device logs are
|
|
132
|
+
enabled only with the additive `deviceLogScope: "device"` option and are stored
|
|
133
|
+
as an explicit device target. One bounded stream is then reused per explicit
|
|
134
|
+
serial. Web Bridge commits incoming SDK evidence directly to FactStore before
|
|
135
|
+
acknowledgement; the observer does not poll or mirror those records.
|
|
136
|
+
The default buffers are `main`, `system`, and `crash`; sensitive `radio`,
|
|
137
|
+
`security`, and `kernel` buffers still require explicit `deviceLogBuffers`.
|
|
138
|
+
The observer keeps at most 32 targets and retires a target after 30 minutes
|
|
139
|
+
without an explicit operation. Android expiry uses a timer without device I/O
|
|
140
|
+
and stops an opted-in log stream when its last target expires. Its limit,
|
|
141
|
+
expirations, evictions, failures, and dropped-log counters are visible in
|
|
142
|
+
`_feedback.observer`. Android targets report `backgroundPolling: false`; null
|
|
143
|
+
poll timestamps and runtime epochs do not represent a successful health check.
|
|
144
|
+
For `launch-app` and `launch-activity`, full feedback runs the launch first,
|
|
145
|
+
then captures a system UIA tree and optional screenshot. It does not require
|
|
146
|
+
an active App SDK before launch. These snapshots show the current screen;
|
|
147
|
+
without an event baseline, they do not establish a semantic UI change.
|
|
148
|
+
|
|
149
|
+
Facts are stored once in an authoritative segmented mmap store. A bounded
|
|
150
|
+
SQLite WAL index is only a rebuildable query projection; it is not a second
|
|
151
|
+
fact-payload store. Legacy history, Script evidence, and Intent evidence use
|
|
152
|
+
the same runtime-owned FactStore with isolated adapters and namespaces. The
|
|
153
|
+
default `auto` profile selects 1 GB when total disk capacity is at least 32 GiB and at
|
|
154
|
+
least 8 GiB is available, 512 MB when total disk capacity is at least 16 GiB and at
|
|
155
|
+
least 4 GiB is available, 256 MB when total capacity is at least 4 GiB and at
|
|
156
|
+
least 2 GiB is available, and 64 MB otherwise. Explicitly select a profile with
|
|
157
|
+
`AI_APP_BRIDGE_FACT_CACHE_PROFILE=1gb`, `512mb`, `256mb`, or `64mb`. Override
|
|
158
|
+
the store directory with `AI_APP_BRIDGE_FACT_STORE_DIR`. The legacy
|
|
159
|
+
`AI_APP_BRIDGE_FACT_CACHE_PATH` setting remains accepted and places the new
|
|
160
|
+
`fact-store-v1` directory beside that path. `AI_APP_BRIDGE_FACT_CACHE=off`
|
|
161
|
+
continues to disable Legacy fact recording without weakening Script/Intent
|
|
162
|
+
evidence gates. The default macOS path is
|
|
163
|
+
`~/Library/Caches/ai-app-bridge/fact-store-v1`.
|
|
164
|
+
|
|
165
|
+
Partitions have independent shares of the mmap budget: network 25%, UI 18%,
|
|
166
|
+
App logs 10%, device logs 7%, state/events 10%, actions 13%, notes 5%, and index
|
|
167
|
+
metadata 2%; 10% remains reserved for manifests and recovery. The SQLite
|
|
168
|
+
projection has a separate bounded budget and cannot evict mmap facts. The
|
|
169
|
+
feedback reports the selected profile, disk selection inputs, mmap usage, and
|
|
170
|
+
projection health. The npm package bundles the native source and builds its
|
|
171
|
+
Node-API binding during installation. If only the SQLite projection fails, queries
|
|
172
|
+
scan the authoritative mmap records and report the projection degradation; persistent
|
|
173
|
+
execution evidence remains available. If the native store cannot initialize,
|
|
174
|
+
foreground commands report degraded history and Script/Intent fail their evidence
|
|
175
|
+
gate. No in-memory buffer substitutes for persistent facts.
|
|
176
|
+
|
|
177
|
+
Existing SQLite fact-cache files are left intact and are not silently imported;
|
|
178
|
+
opaque `fc1` cursors cannot be reused as segmented-store `fs1` cursors.
|
|
179
|
+
|
|
180
|
+
The automatic collector persists network metadata and redacted headers. It
|
|
181
|
+
stores raw body byte counts plus omission flags, not request or response body
|
|
182
|
+
content. An explicit live `network` read keeps its existing behavior. Each fact
|
|
183
|
+
also carries a canonical target key, App identity, runtime epoch, global
|
|
184
|
+
sequence, timestamps, and an action id when correlation is available.
|
|
185
|
+
|
|
186
|
+
On an Android runtime that supports persistent capture, phone
|
|
187
|
+
`logs`/`network`/`state`/`events` with `history: true` read the phone FactStore
|
|
188
|
+
while the device is connected. This also applies to capture forwarded by the
|
|
189
|
+
Flutter Android plugin when its embedded Android runtime has been updated.
|
|
190
|
+
If the phone or runtime is unavailable they return an explicit error and do not
|
|
191
|
+
fall back to a Host-copied payload. iOS capture uses its segmented device store
|
|
192
|
+
for live, `decision-window` and `connected-history` reads. Startup, flush and
|
|
193
|
+
storage failures return explicit errors and incomplete coverage. Web evidence
|
|
194
|
+
is committed on ingress to the Host FactStore; retained history pages
|
|
195
|
+
Host-owned facts through the existing command plus an opaque `factCursor`.
|
|
196
|
+
Mobile history does not merge Host execution records into mobile facts. Use
|
|
197
|
+
Script/Intent history to inspect execution records separately:
|
|
77
198
|
|
|
78
199
|
```json
|
|
79
200
|
{
|
|
80
|
-
"command": "
|
|
81
|
-
"arguments": {
|
|
82
|
-
"defaults": {
|
|
83
|
-
"packageName": "io.github.mobileaidev.aiappbridge.sample"
|
|
84
|
-
},
|
|
85
|
-
"steps": [
|
|
86
|
-
{ "id": "launch", "command": "launch-app" },
|
|
87
|
-
{ "id": "wait-home", "command": "wait-text", "arguments": { "targetText": "Home" } },
|
|
88
|
-
{ "id": "capture-logs", "command": "logs", "arguments": { "limit": 20 } }
|
|
89
|
-
],
|
|
90
|
-
"stopOnError": true
|
|
91
|
-
}
|
|
201
|
+
"command": "events",
|
|
202
|
+
"arguments": { "serial": "DEVICE", "packageName": "com.example.app", "history": true, "limit": 100 }
|
|
92
203
|
}
|
|
93
204
|
```
|
|
94
205
|
|
|
206
|
+
## Intent, Script and evidence
|
|
207
|
+
|
|
208
|
+
Intent supports daily observation and decisions; Script supports repeatable regression.
|
|
209
|
+
Individual `run` calls remain useful. A Script can be authored directly or from
|
|
210
|
+
observed Intent evidence. The duplicate batch executor has been removed. Start a Script
|
|
211
|
+
through CLI or MCP with `operation: "start"` and a `script`
|
|
212
|
+
object containing `schemaVersion: "aab.code-script/v1"`, `language`, one of
|
|
213
|
+
`source`/`sourcePath`, and an explicit Android, iOS or Web `target` for device work. Code exports
|
|
214
|
+
`async function main(ctx)`; Python defines its corresponding `main` entrypoint.
|
|
215
|
+
Each platform retains its own provider requirements and business acceptance evidence.
|
|
216
|
+
|
|
217
|
+
Read progress with `status`/`wait`; completion exposes a small `resultRef`.
|
|
218
|
+
Use `script --operation result --operation-id ID` to read the persisted final
|
|
219
|
+
JSON independently of event retention, including after runtime restart. Password
|
|
220
|
+
and token redaction is declared by the reference's `representation`; missing or
|
|
221
|
+
evicted results return an error. Full result storage is required before a Script
|
|
222
|
+
can report `completed`.
|
|
223
|
+
|
|
224
|
+
See [Script authoring](docs/SCRIPT_AUTHORING.md) for the executable source shape,
|
|
225
|
+
call envelope, assertion results, bounded UI waits and evidence reuse boundaries.
|
|
226
|
+
|
|
227
|
+
Use the public `evidence` command to export retained Intent/Script Host records
|
|
228
|
+
and verify their frozen archive offline. [Evidence archives](docs/EVIDENCE_ARCHIVE.md)
|
|
229
|
+
describes hashes, source binding, retention and the excluded external payloads.
|
|
230
|
+
|
|
231
|
+
- `completed` means that execution finished. Inspect the device assertion
|
|
232
|
+
results to determine which application outcomes were verified.
|
|
233
|
+
- `ctx.assert({scope: "code", name, condition})` checks local code. It cannot
|
|
234
|
+
claim device evidence and is counted separately in `rollingSummary`.
|
|
235
|
+
- Device assertions are the default. Pass the exact `evidence` object returned
|
|
236
|
+
by a current `ctx.call`; fabricated, missing, expired or pre-mutation
|
|
237
|
+
observations are `inconclusive`. UI tree and screenshot predicates must use
|
|
238
|
+
their own evidence. The Host keeps at most 128 observations / 256 KiB of
|
|
239
|
+
assertion metadata per Script and does not retain mobile capture bodies.
|
|
240
|
+
- For asynchronous mobile results, read the stream before the action. Save
|
|
241
|
+
`before.evidence.capture.watermarkCursor` and `.runtimeEpoch`; after the
|
|
242
|
+
action, pass them as `factCursor` and `runtimeEpoch` to the same stream.
|
|
243
|
+
The Host verifies that this boundary was observed before that action.
|
|
244
|
+
`afterActionId` is an optional association filter, not proof of business
|
|
245
|
+
causality; match the actual request or business fields in the predicate.
|
|
246
|
+
If startup dropped records before persistence attached, use a fresh device
|
|
247
|
+
`status.updatedAtMs` as the baseline `sinceMs` and preserve that same
|
|
248
|
+
`sinceMs` in later cursor reads. A cursor alone does not erase a recorded
|
|
249
|
+
loss fence. New loss inside the chosen window still makes it partial.
|
|
250
|
+
- Strong device assertions currently require a complete single page, actual
|
|
251
|
+
refs and `hasMore: false`. Multi-page reads are supported with `nextCursor`,
|
|
252
|
+
but there is no merged multi-page assertion contract yet. Partial windows,
|
|
253
|
+
missing refs, dropped facts and unknown capture backends stay inconclusive.
|
|
254
|
+
- `view: "decision-window"` queries the current epoch.
|
|
255
|
+
`view: "connected-history"` (or `history: true`) queries retained mobile
|
|
256
|
+
history. `mobileFactId` re-reads an exact ref while connected. Ref identity
|
|
257
|
+
survives App restart while the record remains retained; clear invalidates
|
|
258
|
+
the corresponding refs, and old epoch data cannot verify a new action.
|
|
259
|
+
Cursors and refs are opaque and must not be synthesized.
|
|
260
|
+
- Android `status.capturePersistence` reports attachment and lifecycle state,
|
|
261
|
+
including the actual storage operation error. Strong queries remain
|
|
262
|
+
unavailable while the durable backend is not attached. Startup records
|
|
263
|
+
that could not be committed are reported as a gap.
|
|
264
|
+
- `committed` means that the fact writer has made the record readable. The
|
|
265
|
+
mobile store uses group flushing; this does not promise survival of an
|
|
266
|
+
arbitrary power loss before flush. Cold disk reads and cache performance
|
|
267
|
+
are measured separately; this candidate has not passed the old hot-query
|
|
268
|
+
latency target on all devices and retained-store sizes.
|
|
269
|
+
|
|
270
|
+
Script defaults to `restartPolicy: "none"`. Opt into `"checkpoint"` only for
|
|
271
|
+
explicitly reentrant code that uses `ctx.checkpoint` and `ctx.resume`.
|
|
272
|
+
Recovery preserves the frozen source, target and permissions and uses the
|
|
273
|
+
real provider. It cannot restore an arbitrary JS/Python stack. An unmatched
|
|
274
|
+
prepare/receipt, a side effect after the last user checkpoint, or an uncertain
|
|
275
|
+
write requires reconciliation and is never automatically replayed. Completed
|
|
276
|
+
and cancelled operations cannot be resumed to repeat their effects.
|
|
277
|
+
|
|
278
|
+
The old declarative Script `steps` format is rejected with
|
|
279
|
+
`script_format_removed`; migrate it to an explicit code Script. All common
|
|
280
|
+
commands now use strict canonical arguments and physical Android serial arbitration
|
|
281
|
+
inside the same process. Old concurrency and alias contracts have been replaced. No fake
|
|
282
|
+
provider is selected by a missing production dependency.
|
|
283
|
+
|
|
284
|
+
Development checkout: [Script contract validation](scripts/validation/script-contract.md)
|
|
285
|
+
provides an explicit real-device MCP runner for assertion boundaries, cancellation,
|
|
286
|
+
checkpoint recovery and durable action receipts. Supply the server, device and
|
|
287
|
+
package explicitly; these runtime checks do not imply full application acceptance.
|
|
288
|
+
|
|
289
|
+
Every normal object result keeps its legacy fields and adds `_feedback` unless
|
|
290
|
+
`feedback: "off"` is requested. The default `auto` mode does not add post-action
|
|
291
|
+
UI polling, trees, or screenshots. `feedback: "full"` waits briefly for a correlated UI event; when no
|
|
292
|
+
change is observed, it returns an inconclusive result plus current tree and
|
|
293
|
+
screenshot references instead of claiming success. Observer health, runtime
|
|
294
|
+
epoch, dropped-log counters, and persisted fact references are reported in the
|
|
295
|
+
same feedback object.
|
|
296
|
+
|
|
297
|
+
For a coordinate tap on the foreground Android App, `auto`/`full` feedback uses
|
|
298
|
+
one App-local bridge request and reports the actual hit View, bounds, window,
|
|
299
|
+
and touch handling result. An explicit package mismatch stops the action. System UI uses an observed
|
|
300
|
+
system package and explicit device scope; an SDK failure does not trigger ADB retry.
|
|
301
|
+
The request id is carried into synchronous runtime log/network/state/event
|
|
302
|
+
records. Later asynchronous records are correlated by the bounded action
|
|
303
|
+
timeline instead of being presented as an exact runtime binding.
|
|
304
|
+
|
|
305
|
+
Generated Android and iOS screenshot artifacts share the same automatic
|
|
306
|
+
lifecycle: at most 20 files per screenshot prefix, no older than 24 hours, and
|
|
307
|
+
at most 64 MB total. An explicitly supplied `outFile` is user-owned and is not
|
|
308
|
+
automatically removed. Screenshot bytes remain files and are never copied into
|
|
309
|
+
the fact database.
|
|
310
|
+
|
|
311
|
+
For continuous automation, start a [code Script](docs/SCRIPT_AUTHORING.md).
|
|
312
|
+
Normal loops, `ctx.call`, `ctx.assert`, progress and cancellation share one
|
|
313
|
+
execution record. `tap-text` supports observed provider selection (`auto`) or
|
|
314
|
+
an explicit `native`, `flutter` or `uia` provider for repeatable execution.
|
|
315
|
+
For repeated labels or a semantic container, `tap-native` accepts the same exact
|
|
316
|
+
`text`, `contentDescription`, `resourceName` and optional `within` row scope as
|
|
317
|
+
Android Native Intent. CLI, MCP and Script use the same command and validated SDK
|
|
318
|
+
receipt; callers do not supply coordinates or fabricate a `targetRef`.
|
|
319
|
+
|
|
320
|
+
UIA reads and text actions use the bundled phone node runtime on Android API
|
|
321
|
+
33+. A selected node stays bound to its observation and original action ID;
|
|
322
|
+
ordinary commands, Intent and Script share the same executor and completion
|
|
323
|
+
recovery. `uia-runtime --serial DEVICE --operation status|start|stop` controls
|
|
324
|
+
its lifecycle. Fresh observation rotates a full, durably acknowledged session;
|
|
325
|
+
startup retires only confirmed history. Explicit start checks the phone's
|
|
326
|
+
process lock before reopening a dead runtime; unknown actions still block it.
|
|
327
|
+
Host crashes leave a durable pending-acknowledgement queue. `device-ownership
|
|
328
|
+
--operation reconcile --serial DEVICE` commits completion history, acknowledges
|
|
329
|
+
live or stopped phone records and retires that queue without replay.
|
|
330
|
+
For an exited phone owner, reconciliation can turn a matching committed
|
|
331
|
+
prepared/queued record into a durable non-dispatch receipt under the original
|
|
332
|
+
root's exclusive process lock. The receipt preserves original identity and
|
|
333
|
+
separately records the recovery boot/time; admitted/unknown or missing records
|
|
334
|
+
remain unresolved. Recovery never creates a UiAutomation connection.
|
|
335
|
+
`device-ownership --operation receipt --serial DEVICE --runtime-epoch UUID
|
|
336
|
+
--action-id ORIGINAL-ID` queries the retained FactStore representation, including
|
|
337
|
+
explicit original/redacted hashes. A busy FactStore preserves pending cleanup.
|
|
338
|
+
API 36 on OPPO PGFM10 and OnePlus PKR110 is the current real-device scope. See
|
|
339
|
+
the [command contract](docs/COMMAND_CONTRACT.md#semantic-targets-and-text-waits).
|
|
340
|
+
|
|
95
341
|
For dynamic or transient screens, MCP agents can use `freeze-app`/`thaw-app` as
|
|
96
342
|
an optional stabilization control: thaw before reads, actions, waits, or
|
|
97
343
|
captures; freeze after evidence capture only when a changing UI would make
|
|
@@ -107,20 +353,27 @@ target app is debuggable and WebView debugging is enabled.
|
|
|
107
353
|
|
|
108
354
|
iOS commands use Xcode `devicectl` for device/app/screenshot operations, the
|
|
109
355
|
AiAppBridgeIOS runtime for in-app evidence, and WebDriverAgent/XCUITest for
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
356
|
+
taps, input, swipes, and external UI tree reads. `ios-setup --start-wda --team-id`
|
|
357
|
+
prepares a separate copy of pinned WDA 14.1.1 with Bridge identity checks and
|
|
358
|
+
builds/signs that Runner. It returns the actual `wdaRunnerBundleId`; later WDA
|
|
359
|
+
commands require this ID and the selected `deviceId`. An optional `wdaUrl`
|
|
360
|
+
selects a forwarded endpoint without bypassing the Runner container check.
|
|
361
|
+
Create an explicit `ios-wda-session` for an already foreground App, then pass
|
|
362
|
+
its returned session ID with the App bundle ID. Reads do not create sessions
|
|
363
|
+
or launch Apps. See [the WDA contract](docs/COMMAND_CONTRACT.md#ios-wda-target-and-session).
|
|
364
|
+
WDA uses managed execution with queued cancellation, durable original completion
|
|
365
|
+
and `ios-execution --kind wda` recovery. Cancellation waits for the original
|
|
366
|
+
callback of an already submitted XCTest event. iOS Intent and Script support
|
|
367
|
+
native, H5 and Flutter providers with explicit target binding. Native Intent
|
|
368
|
+
`setOrientation` and CLI/MCP/Script `ios-set-orientation` share the same managed
|
|
369
|
+
rotation action. Reobserve the UI after rotation; capability support and individual
|
|
370
|
+
real-device results do not establish complete complex-App acceptance.
|
|
371
|
+
|
|
372
|
+
`input-text` uses the native SDK endpoint for Unicode text and returns its actual
|
|
373
|
+
failure without retrying through ADB. `clear-app-data` chooses `method:"pm-clear"`
|
|
374
|
+
(default) or `"runtime"` before dispatch; there is no retry through another method.
|
|
375
|
+
|
|
376
|
+
When `screenshot` runs without `--out-file`, the CLI writes a unique
|
|
124
377
|
PNG under a git-ignored project artifact directory. Gradle, Android, and Flutter
|
|
125
378
|
projects normally use `build/ai_app_bridge_artifacts`; Node projects can use
|
|
126
379
|
`node_modules/.cache/ai_app_bridge_artifacts`; Swift projects can use
|