@mobileaidev/ai-app-bridge 0.2.15 → 0.3.0-rc.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (156) hide show
  1. package/LICENSE +201 -0
  2. package/NOTICE +7 -0
  3. package/README.md +306 -53
  4. package/bin/ai-app-bridge.js +51 -4504
  5. package/bin/android-permissions.js +152 -0
  6. package/bin/android-uia-xml.js +74 -0
  7. package/bin/artifact-paths.js +127 -1
  8. package/bin/bridge-forward.js +56 -0
  9. package/bin/command-discovery.js +86 -0
  10. package/bin/command-errors.js +60 -0
  11. package/bin/command-registry.js +504 -0
  12. package/bin/command-request.js +14 -0
  13. package/bin/command-router.js +80 -0
  14. package/bin/connection-cache.js +119 -0
  15. package/bin/device-provider.js +2934 -0
  16. package/bin/execution-host.js +596 -0
  17. package/bin/execution-runtime.js +101 -0
  18. package/bin/fact-codec.js +321 -0
  19. package/bin/fact-recorder.js +691 -0
  20. package/bin/fact-store.js +226 -0
  21. package/bin/feedback-probe.js +285 -0
  22. package/bin/intent/install-intent.js +267 -0
  23. package/bin/intent/intent-action-executor.js +129 -0
  24. package/bin/intent/intent-autonomous-adapter.js +46 -0
  25. package/bin/intent/intent-capture-port.js +64 -0
  26. package/bin/intent/intent-entry.js +226 -0
  27. package/bin/intent/intent-errors.js +28 -0
  28. package/bin/intent/intent-evidence-store.js +121 -0
  29. package/bin/intent/intent-lifetime.js +62 -0
  30. package/bin/intent/intent-observation-target.js +33 -0
  31. package/bin/intent/intent-observer.js +194 -0
  32. package/bin/intent/intent-production-adapter.js +334 -0
  33. package/bin/intent/intent-provider.js +27 -0
  34. package/bin/intent/intent-runtime.js +63 -0
  35. package/bin/intent/intent-worker.js +432 -0
  36. package/bin/intent/ios-intent-adapter.js +90 -0
  37. package/bin/intent/permission-intent.js +239 -0
  38. package/bin/intent/web-intent-adapter.js +31 -0
  39. package/bin/ios-device-outcome.js +47 -0
  40. package/bin/ios-execution.js +108 -0
  41. package/bin/ios-provider.js +497 -583
  42. package/bin/ios-runtime-binding.js +54 -0
  43. package/bin/ios-wda-execution.js +70 -0
  44. package/bin/ios-wda-port.js +98 -0
  45. package/bin/ios-wda-project.js +79 -0
  46. package/bin/mcp-server.js +83 -1040
  47. package/bin/mmap-scan-index.js +329 -0
  48. package/bin/observation-collector.js +862 -0
  49. package/bin/runtime-client.js +154 -0
  50. package/bin/runtime-directory.js +107 -0
  51. package/bin/runtime-protocol.js +36 -0
  52. package/bin/script/bounded-script-registry.js +103 -0
  53. package/bin/script/node-runtime-adapter.js +233 -0
  54. package/bin/script/progress-projector.js +63 -0
  55. package/bin/script/python-runtime-adapter.js +111 -0
  56. package/bin/script/rolling-summary.js +134 -0
  57. package/bin/script/script-agent-port.js +40 -0
  58. package/bin/script/script-assert.js +95 -0
  59. package/bin/script/script-capture-port.js +76 -0
  60. package/bin/script/script-catalog.js +85 -0
  61. package/bin/script/script-durable-restore.js +195 -0
  62. package/bin/script/script-entry-code.js +26 -0
  63. package/bin/script/script-entry-route.js +38 -0
  64. package/bin/script/script-entry.js +3 -0
  65. package/bin/script/script-errors.js +29 -0
  66. package/bin/script/script-evidence-store.js +22 -0
  67. package/bin/script/script-format-removed.js +26 -0
  68. package/bin/script/script-host-port.js +397 -0
  69. package/bin/script/script-ledger.js +64 -0
  70. package/bin/script/script-result.js +57 -0
  71. package/bin/script/script-sdk.js +152 -0
  72. package/bin/script/script-sdk.py +153 -0
  73. package/bin/script/script-session-channel.js +127 -0
  74. package/bin/script/script-spec.js +86 -0
  75. package/bin/script/script-supervisor.js +919 -0
  76. package/bin/script/templates/checkpoint-reentry.js +13 -0
  77. package/bin/segment-index.js +481 -0
  78. package/bin/segmented-fact-store.js +1571 -0
  79. package/bin/shared-kernel/android-h5-target.js +10 -0
  80. package/bin/shared-kernel/android-install-execution.js +176 -0
  81. package/bin/shared-kernel/android-sdk-endpoint.js +42 -0
  82. package/bin/shared-kernel/android-shell-execution.js +195 -0
  83. package/bin/shared-kernel/argument-schema.js +117 -0
  84. package/bin/shared-kernel/canonical-path.js +17 -0
  85. package/bin/shared-kernel/device-acknowledgements.js +53 -0
  86. package/bin/shared-kernel/device-completion-history.js +52 -0
  87. package/bin/shared-kernel/device-mutation-lease.js +219 -0
  88. package/bin/shared-kernel/device-ownership-recovery.js +95 -0
  89. package/bin/shared-kernel/device-ownership-store.js +94 -0
  90. package/bin/shared-kernel/evidence-adapters.js +251 -0
  91. package/bin/shared-kernel/evidence-archive.js +329 -0
  92. package/bin/shared-kernel/evidence-recording.js +131 -0
  93. package/bin/shared-kernel/evidence-schema.js +194 -0
  94. package/bin/shared-kernel/evidence-store.js +190 -0
  95. package/bin/shared-kernel/execution-admission.js +22 -0
  96. package/bin/shared-kernel/execution-contracts.js +171 -0
  97. package/bin/shared-kernel/execution-io.js +106 -0
  98. package/bin/shared-kernel/execution-ledger.js +125 -0
  99. package/bin/shared-kernel/execution-scope.js +87 -0
  100. package/bin/shared-kernel/execution-target.js +115 -0
  101. package/bin/shared-kernel/flutter-execution.js +13 -0
  102. package/bin/shared-kernel/flutter-h5-port.js +60 -0
  103. package/bin/shared-kernel/flutter-h5-target.js +9 -0
  104. package/bin/shared-kernel/flutter-target.js +75 -0
  105. package/bin/shared-kernel/h5-execution.js +11 -0
  106. package/bin/shared-kernel/h5-target.js +31 -0
  107. package/bin/shared-kernel/host-fact-store.js +49 -0
  108. package/bin/shared-kernel/ios-h5-target.js +9 -0
  109. package/bin/shared-kernel/ios-native-target.js +71 -0
  110. package/bin/shared-kernel/live-capture-query.js +115 -0
  111. package/bin/shared-kernel/managed-sdk-execution.js +78 -0
  112. package/bin/shared-kernel/native-execution.js +13 -0
  113. package/bin/shared-kernel/native-target.js +156 -0
  114. package/bin/shared-kernel/provider-command-contracts.js +55 -0
  115. package/bin/shared-kernel/recorded-payload-archive.js +195 -0
  116. package/bin/shared-kernel/request-context.js +35 -0
  117. package/bin/shared-kernel/semantic-node.js +55 -0
  118. package/bin/shared-kernel/summary-transformer.js +352 -0
  119. package/bin/shared-kernel/target-lease-protocol.js +47 -0
  120. package/bin/shared-kernel/text-wait.js +111 -0
  121. package/bin/shared-kernel/uia-execution.js +96 -0
  122. package/bin/shared-kernel/uia-protocol.js +214 -0
  123. package/bin/shared-kernel/uia-runtime-port.js +377 -0
  124. package/bin/shared-kernel/uia-target.js +39 -0
  125. package/bin/shared-kernel/web-dom-target.js +44 -0
  126. package/bin/shared-kernel/xml-attributes.js +25 -0
  127. package/bin/target-execution.js +275 -0
  128. package/bin/web/command-schema.js +60 -0
  129. package/bin/web/session-store.js +157 -0
  130. package/bin/web-provider.js +334 -553
  131. package/docs/COMMAND_CONTRACT.md +1563 -0
  132. package/docs/EVIDENCE_ARCHIVE.md +214 -0
  133. package/docs/INTENT_FOREGROUND.md +71 -0
  134. package/docs/INTENT_NATIVE_EDITING.md +79 -0
  135. package/docs/RELEASE.md +59 -0
  136. package/docs/SCRIPT_AUTHORING.md +489 -0
  137. package/node_modules/@mobileaidev/segmented-fact-store-native/LICENSE +201 -0
  138. package/node_modules/@mobileaidev/segmented-fact-store-native/NOTICE +7 -0
  139. package/node_modules/@mobileaidev/segmented-fact-store-native/binding.gyp +36 -0
  140. package/node_modules/@mobileaidev/segmented-fact-store-native/bindings/node/sfs_node.c +597 -0
  141. package/node_modules/@mobileaidev/segmented-fact-store-native/include/sfs.h +178 -0
  142. package/node_modules/@mobileaidev/segmented-fact-store-native/index.js +5 -0
  143. package/node_modules/@mobileaidev/segmented-fact-store-native/package.json +24 -0
  144. package/node_modules/@mobileaidev/segmented-fact-store-native/src/sfs.c +2349 -0
  145. package/package.json +60 -5
  146. package/runtime/ios-wda/AABWDABinding.h +19 -0
  147. package/runtime/ios-wda/AABWDABinding.m +97 -0
  148. package/runtime/ios-wda/AABWDAExecution.h +26 -0
  149. package/runtime/ios-wda/AABWDAExecution.m +172 -0
  150. package/runtime/ios-wda/AABWDAIntegration.h +71 -0
  151. package/runtime/ios-wda/AABWDAManagedRoutes.h +392 -0
  152. package/runtime/ios-wda/AABWDAReceiptStore.h +10 -0
  153. package/runtime/ios-wda/AABWDAReceiptStore.m +116 -0
  154. package/runtime/uia/ai-app-bridge-uia.jar +0 -0
  155. package/runtime/uia/manifest.json +22 -0
  156. package/skills/ai-app-bridge-use/SKILL.md +23 -360
@@ -0,0 +1,1563 @@
1
+ # Command contract: aab.command/v1
2
+
3
+ Intent and Script are first-class execution interfaces. Intent observes a target,
4
+ accepts a decision tied to that observation, dispatches and records its receipt.
5
+ Script runs repeatable JavaScript/Python with progress, assertions and control.
6
+ Both use the device capabilities below and retained evidence. A single command
7
+ remains useful for observation, interaction, fixture setup and diagnosis.
8
+
9
+ ## Discovery and entrypoints
10
+
11
+ MCP exposes exactly `capabilities` and `run`. Use `capabilities {"command":"tap-text"}`
12
+ for its current `inputSchema`, platform, role and supported entrypoints. Domain
13
+ `execution` contains Intent, Script, runtime lifecycle and device ownership;
14
+ `evidence` contains archive operations.
15
+ `capabilities {"includeOptions":true}` returns every current command schema.
16
+
17
+ All parameters are under `run.arguments`:
18
+
19
+ ```json
20
+ {"command":"tap-text","arguments":{"serial":"DEVICE","packageName":"com.example.app","targetText":"设置","provider":"auto"}}
21
+ ```
22
+
23
+ Names are canonical and case-sensitive. Unknown arguments, aliases, invalid
24
+ JSON types, null numeric values and out-of-range coordinates fail before storage
25
+ or device access. Validation includes nested control objects and the selected
26
+ operation's fields. CLI flags convert textual numbers/booleans and parse typed
27
+ object flags as JSON; MCP accepts the actual JSON type. CLI flags use canonical
28
+ `--kebab-case` names followed by separate value tokens. Extra positional tokens
29
+ fail with `unexpected_argument`; repeated single-value flags fail with
30
+ `duplicate_argument` before device access, so a later value cannot replace the
31
+ original target. Only `--category` and `--extra` accept repeated values. Parse
32
+ failures return the same JSON error envelope and exit code 1 as validation errors.
33
+ The CLI's `--help` and `--help COMMAND` come from the same registry. All
34
+ registered commands are available through CLI and MCP, including Intent, Script,
35
+ installation, permission dialogs, evidence and Web sessions. Both adapters call
36
+ `runtime-client.js`; one independent `execution-runtime.js` owns the protocol-neutral
37
+ `execution-host.js`. MCP does not spawn the CLI for each request, and neither
38
+ adapter has a separate execution state machine.
39
+ Within the shared Host, `target-execution.js` still owns ordinary-command
40
+ queueing, deadlines and request-ID idempotency. Intent and Script use the same
41
+ provider access and device ownership through their execution-specific contracts.
42
+
43
+ ### Shared runtime lifecycle
44
+
45
+ The first executing command starts a local runtime. `runtime --operation start`
46
+ starts it explicitly; `runtime --operation status` inspects it without starting
47
+ one; `runtime --operation stop` cancels and drains active work before releasing
48
+ its ownership. CLI exit, MCP EOF and client SIGINT/SIGTERM close only that client.
49
+ Use `intent`/`script --operation cancel --operation-id ID` to cancel one task.
50
+ The same operation ID can be queried and controlled from either entrypoint.
51
+
52
+ A runtime namespace is keyed by the canonical FactStore directory. Configure it
53
+ with `AI_APP_BRIDGE_FACT_STORE_DIR`; symlinks to the same directory identify the
54
+ same store. Control files default to `~/.ai-app-bridge/runtimes/v1`; the explicit
55
+ `AI_APP_BRIDGE_RUNTIME_HOME` override must match in clients that share a runtime.
56
+ A private endpoint record and token authenticate loopback requests. An OS-held
57
+ SQLite lock controls ownership; a stale PID or failed health probe never permits
58
+ replacing a live owner. Startup candidates can repeat the ownership election
59
+ only after proving the lock is free and before dispatching any command.
60
+
61
+ Code, bundled runtime artifacts, native-store binary, Node version, FactStore
62
+ profile and provider configuration must match the owner. Mismatch rejects
63
+ execution with `runtime_code_mismatch` or `runtime_configuration_mismatch`.
64
+ `runtime status` reports the owner and compatibility; explicit `runtime stop`
65
+ remains available from a different build/configuration. An unresponsive owner
66
+ returns `runtime_unresponsive` without killing or replacing it. Lost execution
67
+ connections report an unknown outcome after request submission; execution is
68
+ never automatically retried. Runtime crashes retain the existing durable
69
+ `runtime_lost` and original-device-receipt recovery rules.
70
+
71
+ The runtime uses the environment, including PATH, that started it. PATH is not
72
+ silently replaced by later clients. Use explicit command/target executable paths
73
+ or stop and start the runtime to change its environment. Each request separately
74
+ carries the caller's working directory. Contract-defined local paths resolve
75
+ there; Script freezes its `cwd`, source and target before starting. Subsequent
76
+ clients cannot change those paths. `evidence verify` is offline in both adapters
77
+ and requires neither a running runtime nor an available FactStore.
78
+
79
+ CLI results are one JSON envelope: `{kind: "json"|"text"|"bytes", value, history?}`.
80
+ JSON false, zero and null remain values; text is a string and bytes are base64.
81
+ `history` contains the same recording outcome that MCP exposes as `_history`.
82
+ Command failures have `value.ok:false` and exit code 1. MCP uses its normal
83
+ content and `isError` representation. These are wire-format differences;
84
+ validation, dispatch, task control and evidence semantics are shared.
85
+
86
+ `batch`, `smoke`, `launch-native-test` and `launch-flutter` were removed. Use a
87
+ code Script for sequences, `launch-app` for a Flutter app's actual launcher, and
88
+ `launch-activity` with a supplied component/extra for app-specific test routes.
89
+ Full MCP surface, underscore tool aliases and duplicate outer parameters were
90
+ removed. There is no automatic migration or hidden command substitution.
91
+
92
+ ## Android SDK transport
93
+
94
+ App commands require `packageName`. `port` selects only an optional Host TCP port;
95
+ it cannot identify an App or skip endpoint discovery. Device capabilities such as
96
+ screenshot and keyevent still work without an SDK package.
97
+
98
+ The SDK listens on `localabstract:aab-sdk-<runtimeEpoch>` and atomically publishes
99
+ `files/ai_app_bridge_endpoint.json` in its App-private directory. The
100
+ `ai-app-bridge.android-endpoint.v1` record contains `ok`, `packageName`,
101
+ `runtimeEpoch`, `transport:"localabstract"`, `socketName`, `version`, `updatedAtMs`
102
+ and `error` (null when ready). Each Host request reads that file through `run-as`
103
+ on the resolved serial, validates its package and runtime socket, then creates or
104
+ reuses the exact ADB mapping and checks it before HTTP dispatch. Reads and actions
105
+ are each attempted once. No cached device port, port scan, TCP listener or old
106
+ port-file fallback remains in the Android SDK path.
107
+
108
+ An older SDK without this endpoint must be rebuilt. Discovery failure is
109
+ `bridge_endpoint_discovery_failed`; malformed identity is `bridge_endpoint_invalid`
110
+ or `bridge_package_mismatch`; an unready endpoint is `bridge_not_ready`.
111
+ `remove-forward` requires `serial` and the Host `port` and verifies the mapping's
112
+ serial before removal.
113
+
114
+ ## iOS SDK transport
115
+
116
+ SDK commands (`ios-status/tree/logs/network/state/events/h5-dom/h5-eval/h5-click/h5-input/h5-scroll` and
117
+ `ios-flutter-tree/nodes/action`, plus typed Flutter controls) require explicit `deviceId` and `bundleId`.
118
+
119
+ iOS Intent supports explicit `provider:"native"`, `provider:"h5"` and `provider:"flutter"` with an iOS target.
120
+ Native observation uses the bound WDA tree; tap and inputText require the native
121
+ selector below and an explicit WDA session in the target. Android native selectors,
122
+ keyevent, back and native gestures are not accepted on the iOS branch.
123
+ Native `setOrientation` requires `orientation` and no element selector; it binds
124
+ the observed Runner epoch, App process and WDA session before rotating.
125
+ An App transition currently requires an explicit session close/create and a new
126
+ Intent target; a system dialog disappearing does not silently retarget the old
127
+ Intent. The original action receipt remains recorded if its next observation fails.
128
+
129
+ For Flutter,
130
+ Observation, tap, text replacement, scrollBy, back and hideKeyboard use the production SDK
131
+ adapter and the existing managed execution port. Before a selected action,
132
+ the adapter reobserves and verifies the original Element identity; the SDK
133
+ validates the Element reference again at execution. H5 observation and typed actions
134
+ use the document/element contract below; the providers do not switch implicitly.
135
+
136
+ WKWebView DOM observations include `viewport` (`width`, `height`, `scrollX`,
137
+ `scrollY`), each control's standard text-editing eligibility in `editable`, and
138
+ DOM `interaction.status`: `ready`, `outside-viewport`, `obscured`, `hidden` or
139
+ `disabled`. Intent summaries preserve these facts, including empty editors.
140
+ `visible` describes CSS rendering; it does not mean a control is inside the
141
+ viewport or passes UIKit hit testing. Native hit testing still runs before an
142
+ input or click. A scroll receipt includes the resulting DOM interaction state;
143
+ requesting a scroll does not guarantee the control moved into view. `text` is
144
+ rendered DOM text and can contain an editor's own placeholder decorations.
145
+ Text input accepts text-like input types, textarea and editable content; it
146
+ does not treat date, color, checkbox or file controls as text fields.
147
+
148
+ JS/Python Script can call the catalog's iOS observations, capture queries,
149
+ launch, WDA session/actions and typed Flutter/H5 controls. `ios-tap-flutter`,
150
+ `ios-input-flutter-text` and `ios-scroll-flutter` require an exact `selector`
151
+ (`text` or `nodeId`); input also requires `text`, and scroll requires `delta`
152
+ in logical pixels. `ios-flutter-back` requests one navigation back action.
153
+ `ios-flutter-hide-keyboard` requests explicit focus/keyboard dismissal; observe
154
+ `viewport.viewInsets.bottom === 0` before acting on newly exposed controls.
155
+ These share the ordinary iOS provider's physical-UDID admission and original
156
+ completion receipts. A Script's public `requestId`, or Intent's internal
157
+ `runtimeActionId`, is preserved as the SDK/WDA action ID. Catalog availability
158
+ states implementation support, not completed complex-App or platform acceptance.
159
+ `deviceId` is the selected CoreDevice identifier or UDID. The device must expose
160
+ a connected developer tunnel with developer mode and DDI services ready.
161
+
162
+ Each command copies `Documents/ai_app_bridge_port.json` through devicectl from
163
+ that device's App data container. The atomically published descriptor declares
164
+ `schemaVersion:"aab.ios-runtime/v1"`, `bundleId`, `runtimeEpoch`, `processId`, `port`
165
+ and `ok:true`. The listener asks the OS for an available port and publishes the
166
+ actual bound port only when ready. It does not scan a fixed port range. A
167
+ starting, waiting or failed descriptor has `ok:false`, `port:0` and its reason;
168
+ it cannot authorize HTTP access. Ready descriptors and responses require a
169
+ port from 1 through 65535. An old, absent or invalid descriptor fails before HTTP access.
170
+ There is no port scan or hostname fallback. An optional `runtimeUrl` or
171
+ `iosHost`/`iosPort` selects the transport endpoint; it never replaces container
172
+ verification. A forwarded Host port may differ from the SDK's bound native port.
173
+
174
+ Host requests carry `X-AAB-Runtime-Schema`, `X-AAB-Bundle-Id`,
175
+ `X-AAB-Runtime-Epoch`, `X-AAB-Process-Id` and `X-AAB-Runtime-Port`. Every response
176
+ carries the matching `runtimeBinding`. Control requests first check bound status
177
+ so an old SDK cannot ignore headers and execute a write; the SDK rechecks all
178
+ five fields before dispatch. Missing, duplicate or mismatched identity headers
179
+ are rejected. These are process-routing credentials, not authentication.
180
+ Flutter snapshot ingress retains its existing contract.
181
+
182
+ Capture writes made immediately after SDK `start()` wait for persistent-store
183
+ attachment. The startup queue holds at most 256 records and 1 MiB of serialized
184
+ payload plus identity strings; it never supplies query results. `ios-status`
185
+ exposes `capture.pendingRecords` and `capture.pendingBytes`. SDK, HTTP capture
186
+ POSTs and Flutter capture ingress use the same append path. HTTP/Flutter replies
187
+ wait until the backend returns an actual receipt with its original `mobileFactId`;
188
+ `accepted:true, committed:false` still means queued, not durable proof. Queries
189
+ flush and read the original store before returning committed coverage.
190
+
191
+ Opening failure, disabled persistence, startup overflow and stop reject pending
192
+ writes explicitly (`capture_store_open_failed`, `capture_store_disabled`,
193
+ `capture_startup_queue_full`, `capture_store_unavailable`). A cancelled attachment
194
+ cannot replay its pending captures after restart. Dropped captures retain a gap;
195
+ successful startup does not erase earlier losses. A bounded decision window or
196
+ an exact original reference must carry its own coverage.
197
+
198
+ iOS capture queries scan the requested storage partition: logs use app-log,
199
+ network uses network, and state/events share state-event. Sequence-only seeks
200
+ exclude earlier records in that partition; returned page cursors keep the
201
+ existing global sequence and fixed upper watermark. Global and partition seeks
202
+ retain only bounded read positions, and still verify the original frame bytes.
203
+ Exact-reference lookup within state-event remains a paged scan; the current
204
+ implementation does not provide an indexed constant-time lookup for that case.
205
+
206
+ The default SDK command deadline is 30000 ms across device discovery, container
207
+ copy, preflight and response; `timeoutMs` replaces that total budget. Individual
208
+ HTTP requests default to 5000 ms within it. Descriptor reads are bounded to
209
+ 4096 bytes and HTTP responses to 8 MiB. HTTPS verifies certificates. Invalid or
210
+ non-object JSON and responses without a boolean `ok` fail explicitly. Cancellation
211
+ waits for local child/socket close. A lost control response remains
212
+ `dispatched:null, ambiguous:true, settled:false` until the original SDK completion
213
+ is recovered; local cancellation does not prove that the App stopped executing.
214
+
215
+ ### iOS WKWebView targets and DOM controls
216
+
217
+ `ios-h5-dom` requires one visible WebView, or an explicit `webViewId` previously
218
+ returned in `webViews` when selection is ambiguous. It returns `pageRef` with
219
+ `aab.ios-h5-target/v1`, SDK epoch, bundle/PID, WebView ID, document ID and URL.
220
+ `dom.controls` exposes stable element IDs, visible text, ariaLabel, values and bounds.
221
+ Password values are redacted. The first 1000 controls and first 20000 body characters
222
+ are retained with explicit truncation flags. Text selection rejects a truncated
223
+ snapshot; an explicit observed element ID remains usable within the renderer's
224
+ 5000-control scan bound.
225
+
226
+ `ios-h5-click`, `ios-h5-input` and `ios-h5-scroll` require `selector` with exactly
227
+ one of `elementId`, exact `text`, or exact `ariaLabel`; optional `tag` narrows it.
228
+ Input requires `text` (including empty text to clear). Scroll brings that element
229
+ into view. Click/input require the element to be in the viewport and unobscured.
230
+ These are DOM operations, not trusted physical touch or keyboard events. Native
231
+ hit testing checks the DOM action point before click/input, then the renderer
232
+ checks the original identity and unchanged geometry, including DOM scroll offsets,
233
+ in its final action turn. The native point is mapped from document coordinates
234
+ using UIScrollView zoom and coordinate conversion; CSS innerHeight need not equal
235
+ the native area remaining between toolbars. `nativeHit` records the mapped point,
236
+ scroll offset, scale and actual hit view. Active scrolling/zooming requires a new
237
+ observation. Non-default WKWebView pageZoom returns `ios_h5_page_zoom_unsupported`;
238
+ invalid geometry, a point outside the native viewport and native occlusion have
239
+ distinct errors.
240
+
241
+ Script commands select from a fresh snapshot. Optional `expectedTarget` binds the
242
+ original `{pageRef,element}` as well. H5 Intent always carries that original target
243
+ from its committed observation, and offers `tap`, `inputText`, and `scroll`.
244
+ A replaced node, document navigation, history route change or BFCache restore
245
+ requires a new observation. Multiple simultaneously visible WebViews require explicit selection; an
246
+ App must be active and its selected view attached and visible. Nested frames,
247
+ shadow DOM, zoom/transformed viewport coverage and simultaneous multi-WebView
248
+ selection are not yet accepted as production coverage. Kiwix real-device tab
249
+ creation/closure has verified cached WebView identity, same-URL editor isolation,
250
+ and rejection of hidden, closed and stale targets; see the
251
+ [business evidence](../../../docs/IOS_H5_EDITOR_BUSINESS_2026-09-11.md#真实双标签隔离).
252
+ Intent selects a WebView through `start.observationTarget` or
253
+ `observe.observationTarget`, independently of its frozen device/App target.
254
+ Use `{"webViewId":"observed ID"}` for an explicit selection, or `null` to require
255
+ one visible WebView. `start.target.webViewId` is no longer accepted for Intent;
256
+ Script target defaults retain their separate contract. See Intent operations below
257
+ for selection, failure evidence and revision semantics.
258
+
259
+ Expert `ios-h5-eval` now requires `expectedPage` from `ios-h5-dom`; unbound scripts
260
+ are rejected. All iOS H5 actions use `/v1/h5/action` with a typed payload and the
261
+ existing managed action ID/deadline/receipt. Old SDKs return
262
+ `ios_h5_target_schema_required` before dispatch. Script uses the typed commands;
263
+ expert eval is not exposed as a Script capability.
264
+
265
+ ### iOS SDK execution and recovery
266
+
267
+ `ios-h5-eval`, typed iOS H5 controls and `ios-flutter-action` require a matching SDK that advertises
268
+ managed execution. Host supplies the action ID, original runtime epoch and remaining
269
+ deadline. H5 uses the SDK process epoch; Flutter uses its Dart engine epoch.
270
+ The two kinds share one SDK admission slot. Queued work must obtain permission
271
+ immediately before a mutation. A queued cancellation revokes that permission;
272
+ after permission is issued, only the original execution callback can settle it.
273
+ The iOS Flutter MethodChannel is asynchronous and rejects unmanaged `runAction`.
274
+
275
+ Before reporting a settled result, the SDK synchronously persists an
276
+ `aab.ios-completion/v1` record in its internal action partition. A write failure
277
+ keeps admission closed. Retrying cancellation can persist that same completed
278
+ result; it cannot execute the action again. Queries read the segmented disk store,
279
+ including after an App restart. Opaque cursors retain a fixed committed upper
280
+ bound. Missing, evicted or invalid records cannot release unknown ownership.
281
+
282
+ All public iOS mutation commands share Host ownership keyed by the physical
283
+ UDID (`ios:<UDID>`), including different Apps and CoreDevice/UDID aliases.
284
+ The pending action is durable before dispatch. Host termination or response loss
285
+ therefore blocks subsequent mutations until the original completion is proven.
286
+ This coordination covers one OS user sharing the same ownership directory.
287
+
288
+ Use `ios-execution` through CLI or MCP:
289
+
290
+ - `status`: requires `deviceId` and `bundleId`; returns Host ownership and SDK status.
291
+ - `result`: additionally requires `kind:"h5"|"flutter"`, `actionId` and
292
+ `runtimeEpoch`; queries that original completion through bounded disk pages.
293
+ - `cancel`: has the same identity fields as `result`; requests cancellation and
294
+ reports the original completion or an unresolved result.
295
+ - `reconcile`: requires `deviceId` and the original `bundleId`; reads the saved
296
+ pending identity and releases Host ownership only with its durable SDK receipt.
297
+
298
+ An explicit `runtimeUrl` or `iosHost`/`iosPort` can select the new connection used
299
+ for recovery. Container verification still binds the same physical device and
300
+ App. A new serving process does not replace the original action epoch. `status`
301
+ showing an idle runtime and `cancel` receiving an acknowledgement are insufficient;
302
+ run `reconcile` to resolve a retained Host reservation.
303
+
304
+ These receipts prove the execution callback ended, not business success or all
305
+ asynchronous work triggered by arbitrary App code. WDA has its separate Runner
306
+ execution namespace below. Generic lost install/launch replies, simultaneous multi-WebView
307
+ acceptance and complete complex iOS business coverage remain open. Native, H5 and
308
+ Flutter Intent/Script are available within their implemented command contracts;
309
+ availability does not establish production acceptance.
310
+
311
+ ### iOS WDA target and session
312
+
313
+ WDA commands require explicit `deviceId` and `wdaRunnerBundleId`. The latter is
314
+ the Runner application that owns the data container, not the App under test or
315
+ the test bundle. Every connection copies `Documents/ai_app_bridge_wda.json` from
316
+ that exact device/Runner container. It declares `schemaVersion:"aab.ios-wda/v1"`,
317
+ `bundleId`, `runtimeEpoch`, `processId`, `port` and `ok:true`. Status preflight,
318
+ request headers and every response must match all five identity fields. A WDA
319
+ URL, hostname, `/status` response or vendor UUID alone cannot establish a
320
+ physical-device binding. These fields protect routing; they are not authentication.
321
+
322
+ `ios-setup --start-wda` builds a prepared copy of pinned `appium-webdriveragent`
323
+ 14.1.1; the dependency's installed source is not modified. Signing requires
324
+ `teamId` or the configured development team. `wdaTestBundleId` defaults to
325
+ `io.github.mobileaidev.aiappbridge.wda`; its Runner ID is the test bundle ID plus
326
+ `.xctrunner`. Setup returns `wdaRunnerBundleId`. Unmodified WDA is rejected.
327
+ Setup first runs `build-for-testing` on the Host, then starts the device with
328
+ `test-without-building`. A local compilation failure reports `ios_wda_build_failed`
329
+ without marking a device dispatch. Device-test interruption still requires original
330
+ device completion proof. After WDA is ready, setup launches the target App and
331
+ checks a fresh SDK response because starting the Runner changes the foreground App.
332
+ `wdaProjectPath`, `wdaBundleId`, unbound URL discovery and log/port scans are
333
+ removed. An optional `wdaUrl` can select an explicitly forwarded HTTP(S) endpoint;
334
+ without it, use only the selected developer tunnel IP and descriptor port.
335
+
336
+ `ios-wda-session` operations are explicit:
337
+
338
+ - `status`: requires the device and Runner IDs; returns foreground App and current
339
+ session. It does not accept `bundleId` or `wdaSessionId`.
340
+ - `create`: additionally requires `bundleId`. The App must already be foreground;
341
+ its actual bundle ID and process ID are frozen into the returned session.
342
+ It cannot launch an App or replace an existing session.
343
+ - `close`: requires `bundleId` and `wdaSessionId`. It closes exactly that session
344
+ without terminating the App, including after the foreground App changes.
345
+
346
+ `ios-uia-tree`, `ios-tap`, `ios-input` and `ios-swipe` require all four IDs:
347
+ `deviceId`, `wdaRunnerBundleId`, `bundleId`, `wdaSessionId`. The prepared Runner
348
+ checks its own identity and the session's foreground App/PID on its main route
349
+ queue before dispatch. Old Runner epochs, switched Apps and restarted processes
350
+ fail explicitly. A tree read never creates a session. Tap and swipe remain
351
+ coordinate operations scoped to this foreground App; they are not semantic
352
+ element actions.
353
+
354
+ `ios-set-orientation` requires the same four IDs plus `orientation`: `portrait`,
355
+ `landscapeLeft`, `landscapeRight` or `portraitUpsideDown`. These are App interface
356
+ directions; the Runner explicitly converts the opposite landscape device direction.
357
+ The Runner must advertise `orientationSchema:"aab.ios-orientation/v1"`.
358
+ The command uses the managed `set-orientation` action and the original XCTest
359
+ orientation callback. Its result reports `requestedOrientation` and
360
+ `observedInterfaceOrientation` (null if unknown); a different observed direction
361
+ returns `ios_wda_orientation_not_observed`, even when XCTest accepted the request.
362
+ Reobserve the tree/DOM and verify the intended controls after rotation.
363
+
364
+ Optional `expectedSession` has `{schemaVersion:"aab.ios-native-session/v1",
365
+ runnerEpoch,bundleId,processId,sessionId}` from the preceding `ios-uia-tree`.
366
+ Intent supplies it automatically for
367
+ `{action:"setOrientation",orientation:"landscapeLeft"}`. A changed observation
368
+ binding returns `reobserve_required` before dispatch; the Runner also checks
369
+ foreground identity immediately before submission. CLI/MCP use this same command,
370
+ and Script uses `ctx.call('ios-set-orientation',{orientation:'portrait'})` with
371
+ `app.interact`. No session is implicitly created or replaced.
372
+
373
+ `ios-tap-native` and `ios-input-native-text` require those same four target IDs
374
+ and a `selector` with exactly one of `accessibilityId`, `label`, or `elementId`;
375
+ optional `type` distinguishes repeated labels (for example a Button from a
376
+ StaticText). Input also requires `text` and replaces the selected editor's value.
377
+ The Runner must advertise `aab.ios-native-target/v1`. Missing, ambiguous, hidden,
378
+ disabled or changed controls fail explicitly; there is no coordinate fallback.
379
+ Intent passes its observed `expectedTarget`, binding the original element UID,
380
+ raw identifier, label, type, App process, session and Runner epoch. Script can
381
+ pass the same expectedTarget, or resolve a fresh exact selector on each call.
382
+ An unlabeled icon can use an elementId read from the current tree, not a UID
383
+ retained across App relaunches. The Runner revalidates immediately before the
384
+ original XCTest event submission. Managed action HTTP waits use the admitted
385
+ action deadline; ordinary reads retain their short request budget.
386
+
387
+ When an iOS native Intent summary exceeds its byte budget, visible buttons and
388
+ inputs take priority over long WKWebView text, followed by other visible nodes.
389
+ The returned nodes retain the original preorder, sourceIndex, element IDs and
390
+ visibility facts. This preserves native toolbar/sheet controls without inferring
391
+ that they are clickable or unoccluded. A truncated summary cannot prove absence;
392
+ the original observation tree remains available through the evidence export.
393
+
394
+ `ios-input` requires exactly one `elementId` or `accessibilityId`, plus `text`
395
+ (0–16384 characters). An accessibility ID must resolve to exactly one W3C element;
396
+ zero or multiple matches fail. Host submits one managed input action. Runner
397
+ clicks that editor, optionally sends one keyboard-clear HID event for
398
+ `clearFirst:true`, observes the empty value, and sends the text. Each event checks
399
+ App/PID/session and the selected editor; keyboard events also require its focus
400
+ immediately before permission. Missing, stale or unfocused editors fail; a
401
+ nonempty or unsupported raw value after clearing fails with `ios_wda_clear_not_observed`.
402
+ An empty XCTest value can be `nil` or an empty string. The displayed placeholder
403
+ is not used as proof of an empty editor. The clear event settles only from the
404
+ original XCTest daemon callback, followed by a fresh editor/focus check.
405
+ There are no coordinate/global-key endpoint replacements or clearing retries.
406
+ XCTest synthesizes keyboard events using the current keyboard focus: these checks
407
+ cannot atomically prevent an App from switching focus during an already submitted
408
+ event. Input, replacement and clearing were verified on the iPhone 17 Pro Max
409
+ with iOS 27.0 in the 2026-09-10 device checkpoint. Reentrant focus behavior and
410
+ other editor/keyboard/system combinations remain separate acceptance gaps.
411
+
412
+ `value.error` is a command failure even on HTTP 200. Missing or mismatched
413
+ response identity cannot pass. Normal WDA command deadlines default to 30000 ms
414
+ across discovery and HTTP. Cancellation closes local requests, and failed setup
415
+ waits for its own xcodebuild process to close; it never kills another Runner's
416
+ process group. A lost WDA write response retains physical-device ownership and
417
+ is never replayed through another route. `doctor.ready` requires a connected device,
418
+ developer services, the App SDK and bound WDA; it is a connectivity result, not
419
+ production acceptance.
420
+
421
+ ### iOS WDA execution and recovery
422
+
423
+ The prepared Runner advertises `aab.wda-execution/v1` at `/aab/status`. Session
424
+ create/close, tap, swipe and input all use one managed admission slot. Control
425
+ requests run outside the UI queue, so queued work can be cancelled while main is
426
+ busy. Every event requires permission immediately before submission. Queued
427
+ cancellation ends without an event; once an event is submitted, cancellation
428
+ prevents later steps but retains occupancy until the original XCTest completion
429
+ callback. It does not actively abort an in-flight XCTest event. A missing callback
430
+ or an exception during submission remains unresolved.
431
+
432
+ Runner synchronously commits `aab.wda-completion/v1` into the real segmented
433
+ action store before reporting settlement. Records bind Runner, operation,
434
+ App/PID, session, action ID and original Runner epoch. Persistence failure keeps
435
+ admission closed; cancellation may retry storing the same finished result, never
436
+ replay its event. Disk reads are paged at 64 records/2 MiB with a fixed committed
437
+ upper bound and scope-bound cursors. Missing or evicted records cannot release
438
+ ownership, even when a new Runner is idle.
439
+
440
+ Use the existing `ios-execution` command with `kind:"wda"`:
441
+
442
+ - `status`: requires `deviceId` and `wdaRunnerBundleId`; returns Host ownership
443
+ and Runner execution status.
444
+ - `result`/`cancel`: also require the original `actionId` and `runtimeEpoch`.
445
+ - `reconcile`: reads the durable Host pending identity and queries only that
446
+ original Runner/action/epoch/target. A committed matching receipt releases the
447
+ physical UDID reservation without replay. A new serving Runner epoch does not
448
+ replace the original completion epoch.
449
+
450
+ The WDA branch rejects SDK-only `bundleId`, `runtimeUrl`, `iosHost` and `iosPort`.
451
+ An explicit `wdaUrl` can select the current forwarded connection; the exact
452
+ device/Runner container must still match. `status` showing idle or `cancel`
453
+ acknowledging the request is insufficient to release retained Host ownership.
454
+ Use `reconcile`. Software fault checks and unsigned arm64 compilation qualify
455
+ this contract; physical-device execution remains a separate gate.
456
+
457
+ ## Execution operation contracts
458
+
459
+ `operation` is required. Each operation accepts only its own fields. For Intent,
460
+ use `start/status/observe/decide/pause/resume/cancel/intervene`; intervention
461
+ requires a reason. For Script, use `start/status/wait/result/pause/resume/decide/cancel/runtime-status`.
462
+ Read progress through `status` or event pages from `wait`; `ctx.progress()` remains
463
+ available inside the program. `progress`, Script `intervene`, Intent `reobserve`,
464
+ and `start.spec` are removed aliases.
465
+
466
+ Intent start requires `goal` and an explicit Android, iOS or Web target. Android uses
467
+ `platform:"android"`, `serial`, and `packageName`; optional fields include `adb`,
468
+ `port` and a `foregroundPackages` allowlist. iOS uses `platform:"ios"`, `deviceId`
469
+ and `bundleId`, plus the SDK/WDA binding described above. Web uses `platform:"web"`,
470
+ `sessionId`, `runtimeEpoch` and the observed `targetId`, with provider `h5`. Supervised
471
+ mode is the default. Autonomous mode requires `mode:"autonomous"` and `agentModule`;
472
+ its `budget` defaults to 30 actions, 30 Agent calls, 120000 ms and `allowlist:["tap"]`.
473
+ The allowlist uses the public action vocabulary, including Web `pressKey` and iOS
474
+ `setOrientation`; it never bypasses the selected provider/platform action schema.
475
+ The last permitted Agent reply is validated and may act or finish. A spent action
476
+ or call budget prevents another Agent call. Malformed replies remain
477
+ `waiting_for_decision` with a field error and require explicit correction; they
478
+ neither dispatch nor trigger another Agent request automatically.
479
+
480
+ Every decision requires `decisionId`, positive integer `basedOnRevision`, and
481
+ `agentDecision`. `act` requires a provider-specific `action`; terminal decisions
482
+ `complete/fail/inconclusive` forbid one. Tap uses an exact `selector` object, with
483
+ one identity: Native/UIA `text`, `resourceName` or `contentDescription`; Flutter
484
+ `text` or `nodeId`. Only Native supports a scoped `within` selector. Actions inherit
485
+ the provider of the committed observation; an explicit provider must match it.
486
+ To change providers within a supervised Intent, call
487
+ `{"operation":"observe","operationId":"…","provider":"h5","basedOnRevision":3}`.
488
+ `provider` and `basedOnRevision` are optional. Android accepts `native/uia/flutter/h5`;
489
+ iOS accepts `native/h5/flutter`; Web accepts `h5`. The device/App target and any WDA session stay
490
+ frozen. Switching to iOS native requires the original target to contain its WDA
491
+ Runner and session. Installation and permission workflows retain their required
492
+ `uia` provider and reject another provider or any explicit `observationTarget`.
493
+
494
+ For Android and iOS H5, `start` and `observe` accept `observationTarget:{webViewId:"observed ID"}`
495
+ or `null`. Start defaults to `null`; an explicit selection is configured at start.
496
+ On `observe`, omitting it retains the committed selection when the provider is
497
+ unchanged. Changing provider without a selection clears it to `null`. An explicit
498
+ `null` requires one visible WebView; ambiguity never selects the first candidate.
499
+ An explicit ID never selects a replacement if its view becomes hidden or closes.
500
+ The object accepts only `webViewId` and is supported by Android and iOS H5. Invalid
501
+ selections return a field error before provider I/O or revision changes.
502
+
503
+ Subsequent provider and WebView selections change only after their observation and
504
+ summary are committed.
505
+ Use the returned revision for the next decision; a decision from before the switch
506
+ returns `reobserve_required` without dispatch. Provider acquisition failure enters
507
+ `waiting_for_observation` and blocks decisions on the retained old summary. Retrying
508
+ `observe` with neither selection reads the last successfully selected provider and
509
+ WebView; specify the desired selection again to retry a failed switch. A rejected
510
+ provider observation is retained as an `observation-failed` checkpoint before
511
+ `observationFailure` exposes its evidence ID, attempted selection and original
512
+ structured response (including available `webViews`). A successful observation
513
+ clears the active failure; its checkpoint remains in history and exported evidence.
514
+ Failure details are not exposed if their evidence cannot be persisted. A pending
515
+ observation cannot accept another observation or action, and cancellation drains
516
+ it before finalization. H5 actions still bind the page and element from the
517
+ committed observation, including after an explicit WebView selection.
518
+
519
+ Native/UIA scrolling is `scroll` with required `direction:"up"|"down"`; Native also
520
+ requires the exact scroll-container `selector`. Flutter uses
521
+ `scrollBy` with required container `selector` and `delta`. Native and Flutter
522
+ `inputText` require `selector` and `value`, including an empty value to clear.
523
+ `keyevent` requires `keyCode` and accepts 0;
524
+ `back` has no key parameter. See the discovered schema for native editing gestures.
525
+
526
+ Intent capture requirements use `require.streams`, a nonempty unique array of
527
+ `logs/network/state/events`, plus the documented query window/limit options.
528
+ These options cannot replace the operation's target. Business JSON is open only
529
+ where declared, such as `script.inputs`, Script Agent answers and registered Web
530
+ App action data; control objects reject unknown fields and invalid values.
531
+
532
+ Intent expands `require.streams` into individual capture reads; the streams list
533
+ and foreground-routing options never become command arguments. A historical
534
+ partial page retains its gap. Its committed SDK watermark can bound the next
535
+ action window; only that later read can establish complete coverage for new
536
+ facts. Capture failures retain their machine code, field and message in the
537
+ observation and portable recording.
538
+
539
+ Script start accepts `script`, optional operation ID, `recordingDir`, and
540
+ `pythonPath` for Python only. Supply `script.target` for device work. Language is
541
+ exactly `javascript` or `python`; source is exactly one of `source` or `sourcePath`.
542
+ Policy supports `timeoutMs`, `restartPolicy`, `maxOutputBytes`, `maxProgressBytes`.
543
+ `policy.onFailure` was unused and has been removed: source checks call results
544
+ and assertion verdicts, uncaught exceptions fail execution, and pause is explicit.
545
+ Recording currently requires `restartPolicy:"none"`. Frozen checkpoint recovery
546
+ uses its saved program and never blends missing caller fields with stored fields.
547
+
548
+ Script completion persists its returned JSON before exposing `completed`. Status,
549
+ wait responses and terminal events carry only
550
+ `resultRef:{evidenceId,bytes,sha256,originalSha256,representation}`; the full value
551
+ is read with `script {operation:"result",operationId:"…"}`. This reads and checks
552
+ the retained FactStore record, including after runtime restart, independently of
553
+ the bounded progress ring. `maxOutputBytes` defaults to 1 MiB and accepts at most
554
+ 64 MiB; the store must retain the whole result or report failure. A failed result
555
+ write ends the operation as `failed` with `result_not_persisted`.
556
+
557
+ Results use the same password/token redaction as other persistent evidence.
558
+ `representation` is `original-json` or `redacted-json`; `sha256` and `bytes`
559
+ describe the persisted canonical JSON, and `originalSha256` describes the original
560
+ canonical return value. Successful reads return `result`, `resultRef` and
561
+ `persisted:true`. An unfinished operation returns `result_not_ready`; failed or
562
+ cancelled work returns `result_unavailable`. Missing result persistence, retained
563
+ checkpoint with evicted result, and inconsistent content return
564
+ `result_not_persisted`, `result_not_retained` and `result_checksum_mismatch`
565
+ respectively. An evicted or unknown operation may return `unknown_operation`;
566
+ store/read errors remain errors. No missing result is reconstructed from events.
567
+
568
+ `flutter-action` and `ios-flutter-action` accept a typed `payload` object for the
569
+ documented SDK actions. Payload `actionId` is owned by the Host. These remain expert
570
+ commands; raw SDK actions do not acquire Intent's observation contract. `web-command`
571
+ uses a builtin name and its typed `arguments`; registered App actions use
572
+ `name:"action", arguments:{name:"registered.name", arguments:{...}}`. Web target
573
+ commands require `sessionId`.
574
+
575
+ Flutter semantic taps resolve against the complete operable tree before DOWN.
576
+ While the pointer is held, the SDK revalidates the original Element, semantics,
577
+ action ancestors and the actual touch position. A removed, changed, covered or
578
+ moved-away target receives CANCEL instead of UP. Automatic whole-App snapshots
579
+ wait until this short pointer terminates, then resume; they cannot add tree
580
+ inspection work to the held gesture. App callbacks can still block the Flutter
581
+ isolate, and a mechanical tap receipt still requires an observed business outcome.
582
+
583
+ ## Target and dispatch
584
+
585
+ Every Android mutation requires explicit `serial`. App-specific commands require
586
+ `packageName` or, for compatible SDK reads, an explicit bridge port. Discovery
587
+ currently exposes common transport options; command-specific semantic and nested
588
+ runtime schemas are still being tightened. The current contract does not claim
589
+ that every advertised optional parameter is already equally meaningful on all
590
+ platforms. Read `entrypoints.script` independently of direct MCP availability.
591
+
592
+ `tap` uses physical pixels. An explicit package must match the foreground even
593
+ with `feedback:"off"`. App scope uses the SDK; `scope:"device"` chooses physical
594
+ ADB input. Unknown/mismatched foreground is a non-dispatched failure.
595
+ `tap-flutter` uses Flutter logical pixels, with no DPR multiplication.
596
+
597
+ `tree` and `uia-tree` compact reads accept `maxDepth` from 0 to 200 and
598
+ `maxNodes` from 1 to 1000. These parameters constrain actual returned nodes;
599
+ values outside the implemented limits fail validation instead of being clamped.
600
+
601
+ `tap-text` retains automatic provider selection. `provider:"auto"` inspects
602
+ Native, Flutter, then UIAutomator until one exact operable match is selected.
603
+ Read-only discovery failures are recorded and another provider may be inspected.
604
+ Ambiguous matches stop selection. The returned `provider`, `coordinateSpace`,
605
+ `selected` and `observations` explain the choice. Specify `native`, `flutter`
606
+ or `uia` to pin replay. Foreground changes invalidate the choice. After dispatch,
607
+ an error or unknown result is returned without another provider attempt.
608
+ This convenience command does not provide Intent's complete revision and page
609
+ identity contract; flows needing that contract should use Intent decisions.
610
+
611
+ `input-text` is SDK native Unicode input and accepts an empty string to clear
612
+ an editor. An optional exact Native `selector`, including `within`, binds a
613
+ unique editable View using the same target protocol as Intent. A selector and
614
+ coordinates are mutually exclusive. Without either, it requires a visible
615
+ focused EditText in the focused foreground window; it never chooses an arbitrary
616
+ first editor. An SDK failure is returned without ASCII ADB retry.
617
+ `clear-app-data` selects `method:"pm-clear"` (default) or
618
+ `"runtime"` before one attempt. WDA tap/input/swipe use their selected endpoint
619
+ once; a response timeout is ambiguous and is not retried through another endpoint.
620
+
621
+ Android mutations share physical-serial ownership across cooperating Host
622
+ processes and package installations under the same OS user. Different packages
623
+ on one serial contend; different serials remain independent. The shared directory
624
+ is `~/.ai-app-bridge/device-ownership/v1`, independent of FactStore and working
625
+ directory. `AI_APP_BRIDGE_DEVICE_OWNERSHIP_DIR` explicitly configures a shared
626
+ namespace; all cooperating processes must use the same directory. This does not
627
+ arbitrate unrelated ADB clients, other OS users, remote hosts, or two serial
628
+ aliases that address one phone.
629
+
630
+ An OS-managed exclusive lock has no heartbeat expiry. A separately synced journal
631
+ records pending work before dispatch. An idle dead owner can be replaced; an
632
+ unresolved action survives Host death, timeout and restart and blocks new writes
633
+ with `device_ownership_unresolved`. Long-running installation retains its own
634
+ record while observed installer UI actions execute under the same owner. Ending
635
+ the local ADB process does not by itself confirm a cancelled install has settled.
636
+
637
+ Use `device-ownership {operation:"status",serial:"DEVICE"}` to inspect ownership,
638
+ or `operation:"reconcile"` to query the recorded action through its original
639
+ package and transport configuration. Reconciliation exclusively owns the device
640
+ while checking; there is no force-release, caller-supplied replacement target,
641
+ expiry, or action replay. Android Native, Flutter, H5 and managed shell actions can
642
+ recover from an identity-matching `aab.native-execution/v1`,
643
+ `aab.flutter-execution/v1`, `aab.h5-execution/v1` or
644
+ `aab.android-shell-execution/v1` terminal receipt. UIA node actions use
645
+ `aab.uia.execution.v1`, additionally matching the boot ID, original request
646
+ bytes/hash, snapshot and node binding. Recovery queries the original runtime or
647
+ reads that same action's durable phone record; it never starts a replacement action.
648
+ Successful reconciliation returns `executionReceipt` and records it
649
+ in its own execution history; the original unknown result remains unchanged.
650
+ A missing action, idle SDK, changed runtime or failed connection is insufficient.
651
+ Other transports without a matching completion protocol remain unresolved.
652
+ Installation uses the original phone job and PackageInstaller session contract
653
+ described below.
654
+
655
+ The ownership journal is `aab.device-ownership/v2`, under the **same physical
656
+ lock directory** above. Reading v1 preserves every unresolved identity and
657
+ reservation; the next commit upgrades its format. v1 had no pending-ack queue,
658
+ so the upgrade cannot invent cleanup obligations for already forgotten actions.
659
+ Older writers reject v2 instead of dropping its new records.
660
+
661
+ UIA settlement atomically saves an acknowledgement obligation with the exact
662
+ request, receipt and original FactStore destination. At most eight may remain;
663
+ `device_acknowledgements_full` rejects further UIA preparation until reconciliation.
664
+ Acknowledgement first synchronously commits and reads back a completion record
665
+ in the same segmented FactStore used by MCP/Intent/Script. Only then may the phone
666
+ confirm its copy and the Host remove the obligation. Failures preserve the queue
667
+ without changing the known effect or replaying it. A busy native FactStore reports
668
+ `fact_store_writer_busy`: reconcile from its owning Host, or close that Host before
669
+ retrying. Changing the new Host's recording directory does not redirect an old
670
+ obligation. Ownership journals are bounded to 4 MiB.
671
+
672
+ `device-ownership {operation:"receipt",serial:"DEVICE",runtimeEpoch:"UUID",actionId:"ORIGINAL-ID"}`
673
+ reads retained UIA completion history from the configured Host FactStore without
674
+ contacting the phone. The returned request and completion each have `json`,
675
+ `originalSha256`, `storedSha256`, and `representation` (`original-json` or
676
+ `redacted-json`). `originalCompletionAvailable:false` explicitly means redaction
677
+ changed the original bytes. Missing or evicted history returns
678
+ `device_completion_not_retained`; it is never interpreted as an action outcome.
679
+ This query does not reconcile or release unknown ownership. History remains subject
680
+ to FactStore retention quotas, independent of later overwrites of `lastSettlement`.
681
+
682
+ Reconciliation also drains pending acknowledgements when there is no unresolved
683
+ effect, and reports each `acknowledgements[].disposition` with its durable history
684
+ reference. A stopped original session is acknowledged by a maintenance process
685
+ holding the phone root lock, without opening UiAutomation. A newer live runtime
686
+ may acknowledge an older epoch. `not_retained` only says its phone copy is already
687
+ absent; it is accepted for cleanup solely after the Host's matching completion
688
+ representation has been durably stored. An active epoch uses its original engine.
689
+ Another unresolved action remains occupied throughout cleanup of known history.
690
+
691
+ Android shell mutations, including explicit physical input, app launch,
692
+ PackageManager permission changes, app-ops, process signals and `logcat` clear,
693
+ use a detached phone worker. The Host persists the original action ID, job ID,
694
+ boot ID, command hash and monotonic admission deadline before starting it. The
695
+ phone records completion after the shell command exits. A pre-admission cancel
696
+ or expired deadline prevents the command from starting. After admission, the
697
+ command drains naturally; cancelling or killing the Host does not stop it or
698
+ release unknown ownership. Recovery checks this original job without replaying it.
699
+
700
+ The command process ending does not prove that navigation, PackageInstaller or
701
+ another asynchronous Android service completed its business operation. A shell
702
+ proof does not provide SDK target binding or App event correlation. UIA semantic
703
+ clicks use the separate node runtime described below.
704
+
705
+ Phone jobs are stored under `/data/local/tmp/ai-app-bridge-shell/v1`. Each stdout
706
+ and stderr file is limited to 64 KiB on the verified Android shell; exceeding
707
+ the limit fails the command and retains its exit code and completion proof.
708
+ At most 512 unacknowledged job directories may remain. A new preparation removes
709
+ only acknowledged directories; capacity exhaustion returns
710
+ `shell_execution_store_full` before dispatch. Unknown jobs are never evicted to
711
+ make room. A normal Host acknowledges only after reading the output and syncing
712
+ the matching proof to its ownership journal. Unread output, abandoned preparation
713
+ and recovered jobs remain retained; automatic retirement of those records is
714
+ not implemented. A missing directory or changed boot cannot release ownership.
715
+
716
+ `logcat` reads retain their text result. `logcat {clear:true}` returns an object
717
+ with `ok`, `text` and the verified execution fields because it mutates the phone.
718
+
719
+ A common command's `requestId` reuses the result only for the same target and
720
+ canonical command/arguments. Different content returns `idempotency_conflict`.
721
+ That cache is bounded and process-local, not a durable exactly-once guarantee.
722
+
723
+ ## Script access and evidence
724
+
725
+ The catalog exposes executable Android, iOS and desktop Web capabilities.
726
+ Script calls use the same provider and target ownership as direct CLI/MCP calls;
727
+ catalog support does not establish complete platform business acceptance. Unsupported Script target
728
+ fields are rejected instead of being silently discarded. Recovery hashes include
729
+ target and permissions as well as source/policy.
730
+
731
+ `script decide` answers `ctx.askAgent` with a JSON value, including structured
732
+ objects, arrays and primitive values such as null or false. Supply the question's
733
+ requestId and revision. Repeating the same answer by canonical JSON content is
734
+ idempotent; changing an already answered question's value is rejected. Omitting
735
+ the answer is invalid and is distinct from explicitly answering null.
736
+
737
+ Default permissions are `app.read`, `capture.read`, `app.interact`. The trusted
738
+ caller may explicitly include `app.lifecycle` for clear data and
739
+ `app.permissions` for permission/app-op fixture changes. The host freezes this
740
+ selection into the operation; Script source cannot change it through `ctx.call`.
741
+ These are capability declarations inside a trusted local execution model, not
742
+ an OS sandbox or a multi-tenant authorization system. Raw eval and provider
743
+ management remain explicit direct expert commands. Capture-only calls cannot
744
+ supply CDP eval code or clear device logs.
745
+
746
+ Android mobile capture reads the device FactStore after durable attachment; Host
747
+ execution/observation records use the Host FactStore. iOS public capture now uses
748
+ the device segmented store as described below. Web capture is committed on ingress
749
+ to the Host FactStore; retained history and live barrier coverage have distinct
750
+ contracts described in the Web section. Strong mobile evidence
751
+ requires actual refs, matching epoch/target, the requested window and valid
752
+ coverage. Archive validity, command completion and business assertions are
753
+ separate results. See [Script authoring](SCRIPT_AUTHORING.md) and
754
+ [evidence archives](EVIDENCE_ARCHIVE.md).
755
+
756
+ ## iOS persistent capture
757
+
758
+ The iOS SDK has one write/read path for logs, network, events and state. Before
759
+ persistent attachment, capture returns `ok:false` with `capture_store_opening`
760
+ or `capture_store_unavailable`; no in-memory payload history substitutes for it.
761
+ A successful POST means its record was queued. The returned `receipt` has
762
+ `accepted:true`, `committed:false` and the original `mobileFactId`. A later
763
+ successful persistent query proves the matching record exists. Flutter's iOS
764
+ MethodChannel forwards the complete payload, including `actionId`, and returns
765
+ the same receipt. App-log/NSLog/H5 console capture also uses this store; the
766
+ separate device-log partition remains separate.
767
+
768
+ HTTP GET accepts `view`, `sinceId`, `sinceMs`, `limit`, `runtimeEpoch`,
769
+ `afterActionId`, `factCursor`, `mobileFactId` and `targetKey`. Unknown names,
770
+ empty identifiers and invalid numbers fail explicitly. `limit` is 1–1000,
771
+ default 200; `sinceId` is exclusive, `sinceMs` inclusive. The old `since-id`
772
+ and `since-ms` HTTP aliases and silent limit clamping are removed.
773
+
774
+ - `legacy-live` is a bounded current-runtime projection; state keeps the latest
775
+ value per key. It reads persistent facts and reports projection limits.
776
+ - `decision-window` reads the current runtime. An action filter requires a
777
+ pre-action cursor, capture ID or timestamp boundary.
778
+ - `connected-history` reads retained records across runtimes. A capture ID
779
+ filter also requires `runtimeEpoch`, since IDs restart in each runtime.
780
+ An exact `mobileFactId` can resolve its original record after restart.
781
+
782
+ Queries flush and freeze the durable upper sequence in one writer operation;
783
+ new writes cannot enter that committed boundary. Each response carries refs,
784
+ coverage, the target/runtime, `nextCursor` and `watermarkCursor`. Page cursors
785
+ freeze the upper boundary; a watermark starts a subsequent window. A page scans
786
+ at most 1024 logical records or 8 MiB, with a two-second scan budget in addition
787
+ to bounded writer waits. An empty filtered page can have `hasMore:true`; callers
788
+ must use its `nextCursor`. Each writer read batch is at most 64 logical records
789
+ and 2 MiB. Query projection and page memory have explicit limits; payload
790
+ history is not hydrated into a second cache.
791
+
792
+ Clear persists stream generations and invalidates old cursors. Retention or
793
+ known write loss produces partial coverage. An unknown pre-attachment loss
794
+ cannot be cleared by `sinceId:0`. Metadata corruption, flush errors and changed
795
+ store bindings never return committed success. Capture status counts are bounded
796
+ accepted counts since attachment, not totals for retained disk history.
797
+
798
+ This storage contract has real-disk Swift integration coverage and iOS compilation
799
+ proof. Storage proof and complex business acceptance are separate. iOS Intent
800
+ and Script now use explicit native/H5/Flutter providers and bound execution
801
+ receipts described above; each real-App scenario retains its own evidence limits.
802
+
803
+ ## Semantic targets and text waits
804
+
805
+ `tap-uia` takes an explicit package and either a selector or the complete
806
+ `targetRef` described below. A selector has exactly one field:
807
+ `{"text":"All files"}`, `{"contentDescription":"All files"}` or
808
+ `{"resourceName":"com.example:id/files"}`. A text selector matches only the
809
+ text attribute. A parent accessibility description with the same wording does
810
+ not make that child text ambiguous. Two actual text matches remain an error.
811
+ The command observes the current UIA runtime, binds the selected node, checks
812
+ the foreground again and sends the same original action ID through the existing
813
+ UIA execution journal. CLI, MCP and Script share this implementation.
814
+
815
+ For duplicate or unlabelled controls, `uia-tree` with `compact:true` exposes
816
+ `node.targetRef`; Intent UIA summaries expose the same reference. It contains
817
+ `{schemaVersion:"aab.uia.target/v1",bootId,runtimeEpoch,snapshotId,nodeRef}`.
818
+ Pass that complete object as `tap-uia.targetRef`, mutually exclusive with
819
+ `selector`. An Intent uses `selector:{nodeRef:node.targetRef.nodeRef}` from its
820
+ current revision, which supplies the original observation identity.
821
+
822
+ Reference actions use the retained node from that exact snapshot. The phone
823
+ locates the same accessibility source in the same focused window and checks all
824
+ original node and clickable-ancestor attributes before requesting the click.
825
+ Duplicate text does not choose a different node. Expired, changed or foreign
826
+ references fail before dispatch; they are never replaced with a fresh text
827
+ match. UIA retains at most eight snapshots for 60 seconds, so Scripts observe
828
+ immediately before selecting and acting. This is a runtime reference, not a
829
+ persistent selector to save between runs.
830
+
831
+ The convenience text commands below match display text and, for Native/UIA,
832
+ accessibility descriptions. When this yields several candidates, use `tap-uia`
833
+ or an Intent with a precise selector to state which attribute is intended.
834
+
835
+ `tap-text`, `tap-uia-text` and `tap-flutter-text` share exact, unique selection,
836
+ fresh revalidation and foreground checks. Auto discovery happens only before a
837
+ provider is selected. Revalidation failure never dispatches through a different
838
+ provider. Native selection is limited to the top non-hidden window; a dialog,
839
+ unknown root or disabled foreground root cannot expose a background target.
840
+ Flutter text targeting also checks that the native foreground is the activity.
841
+ Coordinates returned by Flutter remain logical pixels.
842
+
843
+ Native Intent tap/input/longPress/swipe/scroll and Flutter tap/input/scrollBy re-read before input.
844
+ UIA revalidates its saved node reference on the phone before Binder admission.
845
+ Changed identity, ambiguity or invalid bounds reject before dispatch. Native
846
+ input requires the SDK's explicit `editable:true`, including standard EditText.
847
+ Native tree nodes expose `checked:true|false` for Android `Checkable` controls
848
+ and `checked:null` for other Views. The checked state participates in the SDK
849
+ target guard; a state change after observation rejects the old target before
850
+ dispatch, so a stale checkbox action cannot toggle a newly changed value.
851
+ Semantic summaries retain both checked and unchecked controls, including
852
+ unlabelled preference switches. UIAutomator's `checked` value is meaningful only
853
+ when its node declares `checkable:true`; other UIA nodes have `checked:null`.
854
+
855
+ `tap-native` exposes a precise Native selector to CLI, MCP and Script callers:
856
+
857
+ ```js
858
+ await ctx.call('tap-native', { selector: {
859
+ resourceName: 'com.example.app:id/confirm',
860
+ within: { text: 'Test item', ancestor: { resourceName: 'com.example.app:id/row' } }
861
+ } });
862
+ ```
863
+
864
+ It requires an explicit App target and one unique match, revalidates the same View
865
+ before dispatch, and retains the original Native execution receipt. Its selector
866
+ uses the shared Native contract, including `within`; ambiguous, replaced and
867
+ unbound targets fail without dispatch. Passive descendants inside a selected
868
+ semantic container may receive its gesture; an interactive child or unrelated
869
+ overlay still prevents that container tap.
870
+
871
+ `input-text` with a `selector` uses the same foreground and View revalidation,
872
+ requires `editable:true`, and sends the exact editor reference to the SDK.
873
+ CLI, MCP and Script share this entry; missing, ambiguous, replaced or noneditable
874
+ targets fail before input. For example:
875
+
876
+ ```js
877
+ await ctx.call('input-text', {
878
+ selector: { resourceName: 'com.example.app:id/search' }, text: 'Monaco'
879
+ });
880
+ ```
881
+
882
+ Native Intent `tap`/`inputText`, `tap-native`, selector-based `input-text` and the Native branch of `tap-text` additionally
883
+ require `targetRef.schemaVersion:"aab.native-target/v1"` from the SDK tree. The
884
+ Host sends the selector, reference and action ID to `/v1/action/tap-target` or
885
+ `/v1/action/input-target`, without coordinates. In one UI-thread task the SDK
886
+ checks the runtime, focused window, View instance, observed semantic ancestry,
887
+ uniqueness, editability and clipping, then uses the current bounds/selected editor.
888
+ Same-View layout movement is allowed; replacing a View with an identically named
889
+ View is rejected. Input rechecks after synchronous focus/connection callbacks
890
+ and again after the input connection sets the selection, before committing text.
891
+ A rejection after requesting focus reports `dispatched:true`, since that effect
892
+ already occurred; it does not commit text to a replacement editor.
893
+
894
+ Coordinate and focused `input-text` also retain the selected editor and window
895
+ across those callbacks. A detached/replaced editor returns
896
+ `native_target_replaced`; changed focus returns `input_focus_changed`. Neither
897
+ path redirects text to the new focus. This binds the SDK's editor selection and
898
+ commit call, without making custom App input-connection behavior transactional.
899
+
900
+ Missing references or an unsupported endpoint return
901
+ `native_atomic_target_unavailable`; this selected Native attempt never downgrades
902
+ to coordinate or device input. A queued SDK task that times out is cancelled;
903
+ a mutation already running at timeout returns an ambiguous outcome. These checks
904
+ do not make app touch handlers or business outcomes transactional.
905
+ Explicit coordinate commands retain their primitive role.
906
+
907
+ UIA observation and semantic clicks require Android API 33 or newer and the
908
+ bundled, hash-verified `runtime/uia` module. `uia-tree` opens or reuses one
909
+ UiAutomation connection on the explicit device. XML snapshots carry boot,
910
+ runtime and snapshot IDs; each node has an opaque reference. Ordinary UIA text
911
+ commands, Intent (including installer and permission choices), and JavaScript /
912
+ Python Script calls send that reference and their original action ID to the
913
+ same runtime. They do not convert text matches into physical coordinates.
914
+
915
+ The runtime checks the focused default-display window, unique selector match,
916
+ node attributes and clickable ancestor before one node action. Its guarantee
917
+ is `same_connection_node_and_reobserved_attributes`, not a transaction across
918
+ the App's content changes and Android window management. Original Binder
919
+ callbacks distinguish handled/rejected outcomes; an admitted action without a
920
+ matching callback remains unknown. Host timeout, cancellation or process exit
921
+ does not release that action's device ownership. Pre-admission cancellation
922
+ persists a tombstone, so a delayed start cannot execute afterward.
923
+
924
+ For a dead UIA owner, `device-ownership --operation reconcile` can recover a
925
+ committed `prepared` or `queued` record. One-shot phone maintenance must acquire
926
+ the original root's exclusive OS lock and validate the original request, epoch,
927
+ executable hash, zero interaction ID and empty receipt. It writes a durable
928
+ `recovered_before_admission` receipt with `dispatched:false` and
929
+ `uia_owner_exited_before_admission`; it never starts UiAutomation. The receipt
930
+ separately identifies the recovery boot/elapsed time, original preparation time,
931
+ prior record SHA-256 and executable hashes. It has no original callback or
932
+ `completedAtElapsedMs`. Original request bytes and deadline stay unchanged.
933
+ Already admitted/unknown actions and absent original records remain unresolved.
934
+ The first valid recovery receipt is retained on retry, persisted to FactStore,
935
+ and acknowledged through the same durable cleanup queue as other UIA receipts.
936
+
937
+ `uia-runtime --serial DEVICE --operation status|start|stop` exposes expert
938
+ lifecycle control. `status` is read-only; `start` and `stop` share device
939
+ ownership with actions. Stop requires committed terminal actions. Explicit start
940
+ checks the phone's OS-managed process lock. A dead process can be reopened only
941
+ after the new process acquires that lock and audits all original action records;
942
+ any nonterminal or corrupt record blocks reopening, including after a reboot.
943
+ An HTTP timeout or a stale `running` descriptor does not authorize replacement.
944
+ Authentication tokens
945
+ stay in private phone descriptors and are absent from public status and Host
946
+ ownership records. Each runtime retains at most 256 actions; the root retains
947
+ at most 64 sessions. A fresh observation automatically rotates a full session
948
+ only when every action is durably terminal and acknowledged. An already bound
949
+ action never triggers rotation; a full journal rejects it before Host preparation.
950
+ Old epoch references become invalid and require a new observation.
951
+
952
+ At startup, fully acknowledged sessions move atomically into `retired/`; both
953
+ parent directories are synced before any contents are deleted. Interrupted
954
+ deletion resumes there. Unacknowledged original receipts remain byte-exact in
955
+ `sessions/`. Reclamation never evicts unknown actions or trusts temporary files
956
+ as completion proof. Host acknowledgement follows durable ownership settlement,
957
+ so long-term evidence must be read from Host storage/archives after phone-side
958
+ retirement. A stopped session's unacknowledged receipt is retained and can be
959
+ acknowledged through exclusive-lock maintenance. Current real-device evidence
960
+ covers API 36 on OPPO PGFM10 and OnePlus PKR110 with distinct checkpoint scopes.
961
+
962
+ Nonsecret JSON captured as text keeps its original bytes during evidence
963
+ persistence, so Android's escaped solidus characters do not invalidate receipt
964
+ hashes. Credential redaction still applies, including duplicate/escaped JSON
965
+ keys. A redacted representation is not the original cryptographic credential.
966
+
967
+ Flutter semantic actions require `aab.flutter-target/v1` references from the
968
+ operable tree. IDs identify live Elements across snapshots; the reference also
969
+ binds the runtime, observed semantics, action ancestor, editor controller/focus
970
+ node and scroll container. The SDK rebuilds its current targets before dispatch,
971
+ rejects changed/replaced/covered targets, and uses current logical geometry.
972
+ A missing reference returns `flutter_atomic_target_unavailable` without a
973
+ coordinate fallback. Truncated trees cannot establish a unique target.
974
+
975
+ Flutter Intent compares target identities structurally; JSON object key order
976
+ does not identify an Element. Runtime, Element, semantic or guard changes still
977
+ require a fresh observation. The operable viewport reports logical-pixel
978
+ `viewInsets`; visible geometry intersects the actual View viewport excluding the
979
+ native keyboard. A partially visible scroll container keeps its exposed bounds;
980
+ a fully covered control cannot receive a semantic action. This does not detect
981
+ every native system overlay outside Flutter's hit-test tree.
982
+
983
+ Flutter editor observations and Intent summaries preserve the standard field's
984
+ optional `label`, `hint` and `errorText`. Material declarations come from that
985
+ editor's InputDecorator; a Cupertino placeholder is a hint, not a label. Custom
986
+ label/error widgets are not inferred from nearby text. Select the observed node
987
+ by its metadata, then submit its exact `nodeId`; no label selector is implied.
988
+ Label or hint changes invalidate the observed editor identity, while a validation
989
+ message alone does not. Transparent TextSpan content and controls under
990
+ zero-opacity Opacity/FadeTransition are excluded from the operable tree;
991
+ diagnostic strings are not used as visible text. Arbitrary canvas/shader paint
992
+ still needs App semantics.
993
+
994
+ The diagnostic widget-inspector JSON is separately bounded to 64 levels and
995
+ 4000 entries and reports `truncated`. Its truncation does not imply that the
996
+ operable tree is truncated or that unreported diagnostic widgets are absent.
997
+
998
+ `tap-flutter` accepts either `selector:{text|nodeId}` or the explicit logical
999
+ coordinate pair `tapX/tapY`. `input-flutter-text` accepts an optional selector or
1000
+ coordinate pair; when both are omitted it selects one focused editor, or one
1001
+ unique visible editor. Several unfocused editors return
1002
+ `flutter_selector_not_unique`. After the tap and awaited frames, the SDK verifies
1003
+ the original EditableText, controller and focus before addressing that exact
1004
+ TextInputClient. `flutter_input_focus_changed` never redirects text to the new
1005
+ focus. App input formatters and callbacks retain their normal behavior.
1006
+
1007
+ `scroll-flutter` accepts a container selector with `delta` or `targetText`.
1008
+ Omitting the selector requires one visible Scrollable. Scrolling until text
1009
+ uses exact unique matching and retains that container across frames; there is
1010
+ no last-container choice, hidden keyboard action or fallback physical swipe.
1011
+ `flutter_scroll_boundary` reports no movement. These commands retain the
1012
+ existing Script `app.interact` permission; raw `flutter-action` remains expert
1013
+ only. Raw target references/action IDs are not public payload parameters.
1014
+
1015
+ The Dart runtime rejects overlapping action requests with `flutter_action_busy`.
1016
+ Target removal/change during a bound tap sends CANCEL before UP; restarting the
1017
+ runtime invalidates old references. Android additionally advertises
1018
+ `executionSchema:aab.flutter-execution/v1`. Host freezes its runtime epoch and
1019
+ action ID and sends the remaining execution budget, without restarting it.
1020
+ Missing capability fails with `flutter_execution_unavailable`; there is no
1021
+ old-protocol retry. Intent, Script and direct actions share this transport.
1022
+
1023
+ Android receives Flutter actions asynchronously, leaving its HTTP listener free
1024
+ for cancellation. Dart obtains native admission before starting a pointer
1025
+ sequence, writing the bound editor, or making another scroll jump. An admitted
1026
+ pointer sequence handles its timing and terminal events locally; waiting on a
1027
+ channel while holding DOWN must not turn a short tap into a long press. Local
1028
+ stop signals and a monotonic deadline terminate Bridge-owned waits with CANCEL.
1029
+ Opaque App futures remain owned until they actually finish.
1030
+
1031
+ On a lost response or Host cancellation, Host sends one bounded cancellation
1032
+ request to the same connection and runtime, without replaying the action. Only
1033
+ the original action's matching terminal receipt proves `settled:true`.
1034
+ Queued requests whose admission was revoked prove `dispatched:false`. A stopped
1035
+ request with previously granted admission remains SDK-busy until its original
1036
+ completion. After the 1500 ms cleanup grace period, missing confirmation returns
1037
+ `dispatched:null, ambiguous:true, settled:false`; it does not free SDK admission.
1038
+ A repeated cancel can retrieve the single retained terminal receipt. This is
1039
+ bounded cleanup evidence, not a persistent idempotency or rollback service.
1040
+
1041
+ Cancellation does not undo delivered input or App callbacks; an action may
1042
+ finish before its stop signal is observed. SDK settlement means no further
1043
+ Bridge-controlled continuation for that action. Cross-process device ownership
1044
+ retains unresolved work across Host death and can reconcile the original SDK
1045
+ receipt as described above. iOS
1046
+ still uses its distinct transport and does not advertise this Android contract.
1047
+
1048
+ Native Android tap, text input and gestures share one SDK execution coordinator.
1049
+ `status.debugBridge.nativeExecutionSchema` advertises `aab.native-execution/v1`;
1050
+ `nativeAction` identifies the active operation or is null. Host sends its action
1051
+ ID, the observed runtime epoch and remaining timeout in `execution`. Missing
1052
+ capability is `native_execution_unavailable`; there is no old-protocol retry.
1053
+ The shared `/v1/action/cancel` requires the original action ID and runtime epoch.
1054
+ This replaces the former gesture-only cancellation endpoint and status field.
1055
+
1056
+ Cancellation before main-thread dispatch prevents a late tap or input. Once an
1057
+ App callback has begun, the SDK retains occupation until it actually returns;
1058
+ the 1500 ms cleanup grace only bounds the reply, not callback lifetime. A blocked
1059
+ callback returns `settled:false, dispatched:null, ambiguous:true`. Tap stops
1060
+ with CANCEL before a late UP; input checks cancellation before subsequent writes.
1061
+ Input already committed and App side effects are not rolled back.
1062
+
1063
+ The coordinator retains one immutable terminal receipt in memory. Repeated
1064
+ cancellation can retrieve that receipt; reusing its action ID for another action
1065
+ is rejected. This is not durable deduplication: SDK restart or a missing cached
1066
+ receipt cannot prove an unresolved prior action settled. Terminal
1067
+ `native.action.settled` events carry the original action identity without input
1068
+ text. Native, Flutter, H5 evaluation and runtime clear-data cannot overlap an
1069
+ active Native operation through this SDK.
1070
+
1071
+ Native/Flutter/H5/managed-shell action results expose a compact `executionReceipt` when verified
1072
+ settlement or explicit non-dispatch is known, and null when it remains unknown.
1073
+ Ordinary execution history, Intent action receipts and Script action receipts
1074
+ preserve it. Commands containing several physical operations can return
1075
+ `executionReceipts`; ordinary history and Script action receipts preserve that
1076
+ array. Each response hash covers the validated protocol result before Host
1077
+ metadata, not the raw transport bytes. Settlement proves the recorded execution ended;
1078
+ it does not turn an ambiguous action or unverified business assertion into a pass.
1079
+
1080
+ Native Intent `longPress`, `swipe` and `scroll` use `/v1/action/gesture-target`.
1081
+ The top visible SDK window owns pointer selection. A touchable, non-focusable
1082
+ popup can use its focused owner with the same application window token; the SDK
1083
+ does not select a background window when the popup blocks an action. Native/H5
1084
+ window selection shares this rule. Native window observation carries
1085
+ `focused`, `focusable`, `touchable` and nullable
1086
+ `focusOwnerWindowId`. Native input requires actual focus in the selected window.
1087
+ Native touch events keep screen-space `rawX/rawY` and window-local `x/y` distinct,
1088
+ so App outside-touch interceptors receive the same coordinate spaces as normal input.
1089
+ Script and direct callers use the shared `native-gesture` command, requiring
1090
+ `serial`, `packageName` and a typed `payload` with the same selector/action fields:
1091
+
1092
+ ```javascript
1093
+ await ctx.call('native-gesture', {
1094
+ payload: { action: 'longPress', selector: { text: 'Observed note' }, durationMs: 700 }
1095
+ });
1096
+ await ctx.call('native-gesture', {
1097
+ payload: { action: 'scroll', selector: { resourceName: 'example.app:id/list' }, direction: 'down' }
1098
+ });
1099
+ ```
1100
+
1101
+ The shared command observes a fresh target; Intent binds its decision to the
1102
+ committed observation. Both send an SDK reference and Host-owned action ID.
1103
+ Callers cannot supply `targetRef` or `actionId` in the public payload. Long press
1104
+ requires `durationMs:500..10000`; swipe requires `durationMs:1..10000` and numeric
1105
+ `deltaX/deltaY` relative to the selected node's current center. The endpoint must
1106
+ remain in the current window. Scroll uses the selected container's visible area,
1107
+ with `durationMs` defaulting to 400; a container that cannot move in the requested
1108
+ direction returns `native_scroll_boundary` before DOWN. Rounded zero movement
1109
+ and out-of-window endpoints also reject before DOWN. Public `swipe` remains the
1110
+ separate ADB device-coordinate primitive.
1111
+
1112
+ The SDK checks the reference immediately before DOWN, then runs a timed stream
1113
+ on the UI handler so actual long-click callbacks and frames can run. Android
1114
+ owns routing inside the original window until UP/CANCEL; moving/recycling list
1115
+ children does not cause semantic retargeting during the stream. Window/focus loss
1116
+ sends CANCEL to that original window and reports `native_gesture_window_changed`.
1117
+ This can occur after a long-click has already opened a dialog; inspect the result.
1118
+
1119
+ After Host cancellation or response loss, Host sends one bounded cancellation to
1120
+ the same forwarded endpoint with the exact action ID and runtime epoch, outside
1121
+ the aborted execution scope. It waits for that reply, never retries the gesture,
1122
+ and preserves uncertainty if acknowledgement is missing or invalid. The SDK
1123
+ keeps a blocked active touch reserved until CANCEL actually runs. Native tap/input,
1124
+ Flutter action, H5 eval and runtime clear-data reject `native_action_busy` during
1125
+ that interval. This SDK reservation does not arbitrate external device inputs.
1126
+ Queued cancellation proves `dispatched:false`; blocked delivery may return
1127
+ `dispatched:null, ambiguous:true`. CANCEL stops the stream, not prior App effects.
1128
+
1129
+ Gesture receipts include actual duration, event count, start/end coordinates and
1130
+ completion. Started/terminal `ui.interaction` events carry the action ID. A capture
1131
+ callback failure is exposed as `evidenceError` separately from touch delivery;
1132
+ it cannot trigger another terminal touch. Arbitrary asynchronous App callbacks
1133
+ are not all causally tagged. Verify fresh phone events and independent App state.
1134
+
1135
+ `wait-text` accepts `timeoutMs` (default 10000), `intervalMs` (default 500),
1136
+ `provider:auto|native|flutter|uia`, optional `targetText`, `requireText` and
1137
+ `absentText` arrays, and `requireActivity` (full class name). All text matches are
1138
+ exact and come from one fresh visible provider tree; metadata, hidden windows
1139
+ and text from another provider cannot complete a condition. For example:
1140
+
1141
+ ```json
1142
+ {"command":"wait-text","arguments":{"serial":"DEVICE","packageName":"com.example.app","provider":"native","targetText":"Save, draft","requireText":["Editor"],"absentText":["Loading"],"timeoutMs":5000}}
1143
+ ```
1144
+
1145
+ At least one condition is required. Pure absence or Activity-only waits require
1146
+ an explicit provider. An unreadable/invalid tree returns `observation_unavailable`,
1147
+ never successful absence. A deadline returns `deadline_exceeded`; the last
1148
+ available check reports missing/unexpected labels. Cancellation remains
1149
+ `cancelled`. `timeoutSec` and CSV strings are rejected for this command. In the
1150
+ CLI, pass JSON arrays to `--require-text`/`--absent-text`; commas remain part of the label.
1151
+ The H5 wait contract is described below; other platform waits remain separate.
1152
+
1153
+ ## Android H5 execution and DOM operations
1154
+
1155
+ Android Intent supports `provider: "h5"` through the same SDK and typed commands
1156
+ used by CLI, MCP and Script. `intent start/observe` accepts
1157
+ `observationTarget: { webViewId }`; `null` requires exactly one visible WebView.
1158
+ Ambiguity returns candidate IDs. An explicit ID never selects a replacement.
1159
+
1160
+ `h5-dom` returns `h5TargetSchema: "aab.android-h5-target/v1"`, `pageRef` and
1161
+ `dom.controls`. The page reference binds the runtime epoch, package, process,
1162
+ Activity, foreground window, WebView, document generation and URL. Each control
1163
+ has an `elementId` tied to that DOM object. Navigation, including history return
1164
+ to the same URL, and replacing an element invalidate the original action.
1165
+
1166
+ `h5-click`, `h5-input`, `h5-scroll` and `h5-wait` use an object `selector` with
1167
+ exactly one of `elementId`, exact `text` or exact `ariaLabel`, optionally qualified
1168
+ by `tag`. CSS strings and the old `targetText`/`exact` aliases are removed from
1169
+ these Android commands. A truncated snapshot requires an explicit observed
1170
+ `elementId`; duplicate visible matches are rejected. `expectedTarget` on typed
1171
+ mutations binds the original `{ pageRef, element }`. Intent supplies this binding
1172
+ automatically from its committed observation.
1173
+
1174
+ `h5-input` uses string `text`, including `""` to clear an editor. It validates
1175
+ editability and rechecks the same element and value after focus callbacks.
1176
+ Controls retain separate label, text and value; sensitive values are redacted.
1177
+ Snapshots contain at most 1000 controls, with the actual count and truncation
1178
+ flag, and at most 20000 body characters. Display text and values are capped at
1179
+ 500 characters. These semantic DOM operations do not promise trusted browser
1180
+ input events or business acceptance.
1181
+
1182
+ Click/input first obtain a DOM action point, then check Android clipping and
1183
+ native occlusion. The final renderer turn must still match the original element,
1184
+ document and geometry. Standard Android WebView coordinates account for scale
1185
+ and scroll offsets. Non-standard WebView geometry and transformed native views
1186
+ return explicit unsupported errors. Snapshot adapters may still observe them;
1187
+ there is no automatic provider fallback. Frame selection remains unimplemented.
1188
+
1189
+ `h5-scroll` accepts either a selector to bring an element into view, or both
1190
+ `deltaX` and `deltaY` to scroll the page in CSS pixels. Mixing these forms or
1191
+ supplying two zero deltas is rejected. Page scrolling can include `expectedPage`.
1192
+ Expert `h5-eval` always requires `expectedPage`; it executes synchronous JavaScript
1193
+ in that observed page. Script does not permit this expert escape hatch.
1194
+
1195
+ Mutations use `/v1/h5/action` and `aab.h5-execution/v1`. The SDK advertises
1196
+ `h5ExecutionSchema`, `h5TargetSchema` and the active `h5Action` in
1197
+ `status.debugBridge`. `/v1/h5/cancel` accepts only the original action ID and
1198
+ runtime epoch. Cancellation before admission prevents submission; cancellation
1199
+ between the read-only geometry probe and final action prevents the mutation.
1200
+ After submission, WebView offers no cancellation acknowledgement: occupancy is
1201
+ retained until the original callback and Java invocation have both completed.
1202
+ Uncertain effects continue to block other device mutations. Native, Flutter and
1203
+ H5 share coordination while retaining their own identities and receipts.
1204
+
1205
+ `h5-wait` only polls observations. It keeps the original WebView ID and retries
1206
+ only a readable snapshot with an absent selector. It uses `timeoutMs` (default
1207
+ 10000) and `intervalMs` (default 500), returns `h5_wait_timeout` on local expiry,
1208
+ and respects the encompassing Host deadline and cancellation. It has no mutation
1209
+ action ID. Missing WebViews, ambiguity and protocol errors return immediately.
1210
+
1211
+ Flutter H5 uses an explicit adapter contract. Registration requires
1212
+ `isVisible: bool Function()` backed by the App's actual route/widget state;
1213
+ it does not infer native occlusion. `flutter-h5-dom` may omit `adapterId` only
1214
+ when exactly one registered adapter is visible. Multiple visible adapters require
1215
+ an explicit ID. Duplicate registration is rejected; unregister before replacing
1216
+ an ID, and discard its old page references.
1217
+
1218
+ `pageRef` uses `aab.flutter-h5-target/v1` and binds `runtimeEpoch`, `adapterId`,
1219
+ `adapterGeneration`, `documentId` and `url`. `flutter-h5-click/input/wait` use
1220
+ `selector:{elementId|text|ariaLabel,tag?}`; CSS strings, `targetText` and `exact`
1221
+ are removed. Input uses string `text`, including an empty string. Typed mutations
1222
+ may carry `expectedTarget:{pageRef,element}` from the observation. Scroll accepts
1223
+ either a selector or both `deltaX`/`deltaY`; page scrolling may bind `expectedPage`.
1224
+ Expert eval requires `expectedPage` and retains managed original completion and
1225
+ cancellation. Wait only observes the same adapter and document; navigation or
1226
+ replacement fails instead of selecting another document. The shared renderer
1227
+ checks DOM occlusion; platform/App visibility and business acceptance remain
1228
+ separate evidence requirements.
1229
+
1230
+ Android and iOS share the generated DOM identity renderer
1231
+ in `shared/h5/renderer.js`; native window/geometry checks remain platform-owned.
1232
+ Regenerate with `python3 shared/h5/generate.py` and check mirrors with `--check`.
1233
+
1234
+ ## Web DOM, Intent and Script
1235
+
1236
+ Web commands run on the shared runtime that owns the browser's SDK
1237
+ connection, accessible from both CLI and MCP. Select a live `sessionId`, `runtimeEpoch` and `targetId: "main"`
1238
+ from `web-sessions`. A Web Intent target adds `platform: "web"` and uses
1239
+ `provider: "h5"`; other providers are rejected. Script supports the catalog's
1240
+ typed Web commands with the same target. Neither entrypoint creates a second
1241
+ provider or reconnects a different page behind the original target.
1242
+
1243
+ `web-dom` returns `webTargetSchema: "aab.web-dom-target/v1"`, `pageRef` and
1244
+ `dom.controls`. The page reference contains the schema, session, runtime, target,
1245
+ navigation ID and URL. Navigation IDs change even when a page returns to the same
1246
+ URL. Each control has an `elementId` tied to the actual DOM element in that
1247
+ document; replacing a node with identical HTML creates a different identity.
1248
+
1249
+ Every control includes `checked: true | false | "mixed" | null` and the raw
1250
+ `ariaChecked` attribute (`string | null`). Native checkbox/radio state comes
1251
+ from its live property, with an indeterminate checkbox reported as `"mixed"`.
1252
+ Supported ARIA controls use their declared state; missing, invalid or unsupported
1253
+ state remains `null`. For `radio`, `menuitemradio` and `switch`, an ARIA `mixed`
1254
+ value means false under [WAI-ARIA 1.2](https://www.w3.org/TR/wai-aria-1.2/#aria-checked);
1255
+ the raw attribute is still preserved. Checkbox/menuitemcheckbox retain `"mixed"`.
1256
+ Intent summaries preserve these checkable roles and state, including textless
1257
+ controls. A hidden companion input and its visible custom checkbox remain
1258
+ separate nodes with separate interaction readiness.
1259
+
1260
+ Use the matching current Web SDK: a snapshot without `checked`, or with a value
1261
+ outside this contract, fails as `invalid_web_dom_control`. The Host does not
1262
+ substitute false for missing state. Checked state is observation data, not part
1263
+ of element identity. `web-click` remains one click, not an idempotent set-state
1264
+ operation; reobserve and verify the business result after it.
1265
+
1266
+ Typed actions take an object `selector` with exactly one identity field:
1267
+ `elementId`, `text`, `ariaLabel` or `css`, optionally restricted by `tag`/`role`.
1268
+ They require one visible match. String selectors, mixed identity fields and
1269
+ ambiguous text matches are rejected. `web-dom.selector` is a CSS string for a
1270
+ bounded read; it is a different argument contract from an action selector.
1271
+ Intent decisions select observed controls by `elementId`, `text` or `ariaLabel`;
1272
+ they do not run an unobserved CSS query. A truncated Intent snapshot is rejected
1273
+ for action selection, rather than assuming the missing controls cannot match.
1274
+
1275
+ An action may bind its original observation using
1276
+ `expectedTarget: { pageRef, element }`. Copy the element fields `elementId`,
1277
+ `tag`, `id`, `name`, `type`, `role`, `ariaLabel`, `placeholder`, `href` and `text`
1278
+ from that observation. Intent supplies this binding automatically. The SDK
1279
+ rechecks the original page and element before dispatch, including after focus
1280
+ callbacks. A changed page or element requires a new observation; it never selects
1281
+ a replacement using the old text or CSS selector.
1282
+
1283
+ | Command | Behavior |
1284
+ | --- | --- |
1285
+ | `web-click` | Invoke one semantic DOM click on the unique visible, enabled, unobscured control. |
1286
+ | `web-input` | Replace `value` (a string, including empty) in a supported text input, textarea or contenteditable. Native form setters and browser input events reach framework handlers; rich-text insertion uses the browser editing path. |
1287
+ | `web-key` | Deliver one bubbling, cancellable `keydown` for `Enter` or `Escape` to the selected control. The receipt reports `trusted: false` and `defaultPrevented`; keyup, trusted keyboard defaults and general shortcuts are not promised. |
1288
+ | `web-scroll` | `mode: "into-view"` requires a selector and no deltas. `mode: "by"` requires at least one explicit `deltaX`/`deltaY`, with an optional element selector. |
1289
+ | `web-wait` | Wait for exactly one of a typed selector or body `targetText`, within the command deadline. |
1290
+
1291
+ DOM observations include `visible`, `disabled`, `editable` and
1292
+ `interaction.status`: `ready`, `hidden`, `disabled`, `outside-viewport` or
1293
+ `obscured`. A ready/obscured observation includes the checked point; an obscured
1294
+ one includes the hit element's diagnostic identity. Interaction state explains
1295
+ the observation and is checked again at execution. Errors retain their original
1296
+ receipt and dispatch status, including `web_element_ambiguous`,
1297
+ `web_element_changed`, `web_element_obscured`, `web_element_outside_viewport`
1298
+ and `reobserve_required`. Successful dispatch does not establish asynchronous
1299
+ business completion; observe and assert the actual resulting state.
1300
+
1301
+ Script admits `web-status`/`web-dom` under `app.read` and these typed controls
1302
+ under `app.interact`. Raw `web-command` remains outside its catalog. DOM reads
1303
+ produce Host tree references for device assertions. Ordinary
1304
+ `web-logs/network/state/events` query committed Host FactStore records. They are
1305
+ available under Script `capture.read` and Intent `require.streams`.
1306
+ The default `view: "decision-window"` requires the exact connected document.
1307
+ Each query obtains an SDK capture barrier; its upper FactStore cursor is frozen
1308
+ when that response reaches the Host, before later frames can enter the window.
1309
+ The `wc1:` watermark and its SDK/Host counters are persisted in FactStore.
1310
+
1311
+ Observe before acting and pass the returned `watermarkCursor` as `factCursor`
1312
+ after the action. `afterActionId` additionally filters explicitly associated
1313
+ records and requires that lower watermark. In Intent requirements, omitting
1314
+ `afterActionId` selects the current action; explicit `null` retains all records
1315
+ in the temporal window. Repeated observations of one action keep its original
1316
+ lower watermark so a delayed response remains observable. This temporal window
1317
+ does not assert that every included request was caused by the action.
1318
+
1319
+ `coverage.scope: "sdk-captures-through-barrier"` describes recorded SDK captures.
1320
+ Queue loss, sequence gaps, rejected captures and connection changes retain partial
1321
+ coverage; pending requests also prevent complete coverage. The window does not
1322
+ cover uncaptured traffic or future asynchronous work. Each record identifies
1323
+ `association: "explicit" | "synchronous" | "unattributed"`; unknown origins keep
1324
+ `actionId: null`. Request identity is captured when a request starts, never taken
1325
+ from the action active when its response arrives. Script dispatch now supplies
1326
+ the same original action ID to the provider, receipt and synchronous UI event.
1327
+
1328
+ Pages scan at most 16 records. Keep the original `factCursor`, filters and returned
1329
+ `throughCursor` with `nextCursor` for continuation. The upper bound stays fixed.
1330
+ A continuation page alone cannot pass a whole-window Script assertion; automatic
1331
+ aggregation of evidence across pages remains open. `history: true` or
1332
+ `view: "connected-history"` reads retained Host records, including while offline,
1333
+ with partial `retained-host-history` coverage. History forbids `factCursor` and
1334
+ `afterActionId`; explicit `history: true` conflicts with `view: "decision-window"`.
1335
+
1336
+ Optional request/response bodies carry `BodyState` and `BodyEncoding` fields.
1337
+ `utf8` bodies contain text; `base64` bodies preserve binary protocol bytes. Decode
1338
+ only complete bodies using the declared encoding and the App's protocol schema.
1339
+ Missing, disabled, oversized or timed-out bodies cannot prove submitted content.
1340
+ Full payload exports preserve Web Host references, document identity, action
1341
+ association and barrier diagnostics, and can be checked offline.
1342
+
1343
+ A browser harness screenshot is a separate artifact, not a Script-owned
1344
+ screenshot reference or an archive attachment by default. See [Script authoring](SCRIPT_AUTHORING.md) and the
1345
+ [Memos real-App result](../../../docs/WEB_MEMOS_BUSINESS_2026-09-11.md) for the
1346
+ implemented scope and remaining evidence gaps.
1347
+
1348
+ ## Errors and remaining work
1349
+
1350
+ Ordinary Android commands share one Host deadline from queue entry through
1351
+ provider calls and optional feedback. `timeoutMs` defaults to 30000; a timed
1352
+ logcat collection defaults to its requested duration plus 1000 ms. An explicit
1353
+ `adbTimeoutMs` caps each subprocess within that same deadline. Nested calls cannot
1354
+ extend the deadline. ADB/HTTP work and local polling stop cooperatively, and local
1355
+ subprocesses/sockets close before the call settles. `deadline_exceeded` before an
1356
+ action has `dispatched:false`; after submission it retains `ambiguous:true`.
1357
+ This does not undo work already accepted by Android or the App SDK. Install and
1358
+ permission Intent workflows retain the separate lifecycle rules below; iOS/Web
1359
+ provider cancellation still needs its own redesign.
1360
+
1361
+ Common command failures use `ok:false`, a stable string `error`, human `message`,
1362
+ optional `field`/`details`, and `dispatched`/`ambiguous`. `dispatched:null` means
1363
+ unknown. Argument rejections have `dispatched:false, ambiguous:false`. MCP
1364
+ `isError` follows command failure; CLI returns exit code 1. Runtime operations
1365
+ retain their own status/control and evidence envelopes; a completed Script may
1366
+ contain failed or inconclusive assertions.
1367
+
1368
+ The implemented contracts above define the supported command scope. Frame
1369
+ selection and merged multi-page strong assertions remain unsupported. Platform
1370
+ capability, execution completion and acceptance of a particular App workflow
1371
+ must be assessed separately from its retained evidence.
1372
+
1373
+ ## Ordinary Intent lifetime and recovery
1374
+
1375
+ `intent start.timeoutMs` is a total lifetime budget (default 300000 ms). It starts
1376
+ before the first observation and includes decision waits, paused time, Agent
1377
+ replies, subsequent observations and actions. An autonomous budget can shorten
1378
+ this deadline. Nested provider work cannot extend it. Pause invalidates the
1379
+ pending Agent reply; resume obtains a new decision. Agent adapters receive an
1380
+ `AbortSignal`; late replies cannot dispatch through the stopped Intent. The
1381
+ adapter must cooperate to stop any external computation it owns.
1382
+
1383
+ Cancel first enters `cancelling`, aborts owned provider work and waits for that
1384
+ work and its evidence writes. Timeout similarly stops new dispatch with
1385
+ `deadline_exceeded`. Other terminal outcomes pass through `finishing`. A terminal
1386
+ checkpoint is committed after the pending work settles and before the terminal
1387
+ status is exposed. Final persistence can extend beyond the execution deadline.
1388
+ `blocked_evidence_store` means this commit failed; `terminalEvidenceId` is absent.
1389
+ `pendingOperations` counts owned execution calls, not the final checkpoint write.
1390
+ Cancel cannot rewrite an already settled outcome. An accepted but unconfirmed
1391
+ action retains `lastAction.dispatched` and `lastAction.ambiguous`; cancellation
1392
+ does not imply rollback. Complete/fail/inconclusive decisions also require the
1393
+ current `basedOnRevision`.
1394
+
1395
+ After execution-runtime restart, `intent status` reads checksum-verified Host evidence and
1396
+ returns `live:false, recovered:true, restartPolicy:"none"`. A durable terminal
1397
+ checkpoint restores the outcome. Retained evidence without a terminal checkpoint
1398
+ returns `interrupted/runtime_restarted`, including unmatched action markers.
1399
+ There is no automatic replay, and a retained operation ID cannot be reused.
1400
+ Store read failures are errors, not an empty history. Explicit runtime stop or a
1401
+ signal to the runtime closes Intent intake, drains pending start/decide work and
1402
+ persists terminal evidence before closing Host storage. CLI exit and MCP EOF
1403
+ only disconnect the client. Mobile capture queries continue to use the live mobile
1404
+ store; this recovery contract applies to Host execution evidence.
1405
+
1406
+ ## Installation is an Intent operation
1407
+
1408
+ `install-apk` starts an ordinary supervised Intent operation through CLI or MCP. It
1409
+ returns `command:"intent"`, an `operationId` and `installation` status. Supply
1410
+ `serial` and `apkPath`; optional `packageName` must match the APK manifest.
1411
+ Android SDK `aapt` and `apksigner` are required, found under `ANDROID_SDK_ROOT`
1412
+ or supplied as `aaptPath`/`apksignerPath`.
1413
+
1414
+ The Host freezes and inspects the APK, stages those exact bytes on the phone,
1415
+ and commits the original action/job/boot identity before launching one detached
1416
+ installation worker. The worker rechecks the APK SHA-256, creates one non-staged
1417
+ PackageInstaller session, writes the base APK, and commits that session. The
1418
+ session ID and original PackageManager CLI response are retained on the phone.
1419
+ `allowDowngrade` is opt-in. `streaming` has been removed and is rejected.
1420
+
1421
+ `timeoutMs` bounds device staging, admission and Host waiting (default 180000).
1422
+ APK inspection, initial/final installed-identity reads and UI observation still
1423
+ have separate bounded calls; this is not yet one end-to-end deadline. Inspectors
1424
+ accept verified signer output from `Signer #N` and SDK `V1`, `V2`, `V3`, `V3.0`,
1425
+ `V3.1`, `V4` signer formats; unsupported formats reject without submitting an APK.
1426
+
1427
+ When system interaction is needed, the calling Agent uses `intent observe`, reads
1428
+ the actual foreground tree, then submits `intent decide` with its revision and
1429
+ one exact selector. The adapter has no ROM package table, positive-button labels
1430
+ or automatic click loop. Observation and decision may need to repeat as the
1431
+ system page changes. Without an Agent decision, no system button is clicked.
1432
+ The operation holds the phone's mutation lease through package verification.
1433
+
1434
+ For example, after receiving a fresh observation:
1435
+
1436
+ ```json
1437
+ {"command":"intent","arguments":{"operation":"decide","operationId":"FROM_START","decision":{"decisionId":"choice-1","basedOnRevision":2,"agentDecision":"act","action":{"action":"tap","selector":{"resourceName":"ID_FROM_ACTUAL_OBSERVATION"}}}}}
1438
+ ```
1439
+
1440
+ Completion requires the matching original shell-job receipt, a successful
1441
+ PackageInstaller commit response, and an independent `pm path`/`sha256sum` check
1442
+ of the installed bytes. The proof binds actionId, jobId, Android boot, command
1443
+ hash, admission deadline, installId, package and APK hash, and identifies the
1444
+ PackageInstaller session. The install receipt response hashes use canonical JSON
1445
+ with sorted object keys, so archive serialization preserves their verification.
1446
+ A matching old APK, a local ADB exit, a missing session
1447
+ or a mismatched receipt cannot settle this request. Caller `complete` decisions
1448
+ cannot override verification. Split APK layouts are explicitly unsupported.
1449
+
1450
+ The supported PackageManager CLI protocol recognizes exact `Success` and final
1451
+ `INSTALL_FAILED_*` / `INSTALL_PARSE_FAILED_*` failure responses. This is parsing
1452
+ machine command output, not recognizing installer button text. Pending-user-action,
1453
+ warnings, unfamiliar OEM output, truncated output, process death and changed boot
1454
+ remain unresolved. A terminal install rejection releases ownership but fails the
1455
+ workflow, even when the old APK still matches.
1456
+
1457
+ `intent cancel` stops Host waiting and UI decisions. It can prevent admission;
1458
+ once the phone worker is admitted it drains the original request, without a
1459
+ second install or a rollback promise. If its original result is unavailable,
1460
+ physical-device ownership remains blocked after cancellation, timeout, EOF or
1461
+ Host death. `device-ownership reconcile` reads that same job and response; it
1462
+ never repeats the install. It does not resume the lost Intent decision loop.
1463
+ A still-waiting PackageInstaller interaction must finish before it can provide
1464
+ completion proof. `pause` only suspends UI decisions.
1465
+
1466
+ The phone staging area admits at most 64 retained installations. APK staging is
1467
+ removed and the shell receipt acknowledged only after a known result is settled;
1468
+ normal completion and cross-Host recovery persist the exact ownership proof first.
1469
+ Cleanup failure is reported separately and cannot erase a known execution result.
1470
+ Unknown/orphaned staging, eviction/reboot recovery, warning/OEM variants, active
1471
+ PackageInstaller cancellation and multi-ROM verification remain separate gates.
1472
+
1473
+ ## Ordinary MCP command history
1474
+
1475
+ Ordinary command execution and auxiliary history persistence have separate
1476
+ outcomes. `ok`, dispatch state and `executionReceipt` describe the original
1477
+ provider operation. A history failure never retries or rewrites that operation.
1478
+ Intent/Script required evidence commits retain their existing strict admission
1479
+ and terminal-evidence contracts.
1480
+
1481
+ Object results carry `_history` with schema `aab.command-history/v1`, including
1482
+ when `feedback:"off"`. Every ordinary CLI/MCP result also carries the same value in
1483
+ `_meta["ai-app-bridge/history"]`, so raw text/image-result consumers can inspect
1484
+ history without rewriting the original content. This is MCP history metadata;
1485
+ CLI and MCP use the same history store; direct internal provider calls do not acquire an auxiliary one.
1486
+
1487
+ - `stored`: this invocation's action and returned Host evidence references were
1488
+ committed. It is not a claim of complete mobile history or business correctness.
1489
+ - `partial`: one or more writes failed. `action` and `evidence` identify which
1490
+ references were stored; `errors` preserves their failure codes.
1491
+ - `unavailable`: history initialization or recording setup failed.
1492
+ - `disabled`: auxiliary history was explicitly disabled or no recorder was supplied.
1493
+
1494
+ A failed action reference has `stored:false`; unavailable `globalSeq` is `null`.
1495
+ `replayed:false` makes the no-retry rule explicit. Use read-only verification of
1496
+ the original action when history is partial/unavailable. Do not repeat a mutation
1497
+ just to obtain a stored reference. A failed evidence append is not added to the
1498
+ in-process dedupe set, so a later observation may persist the same evidence.
1499
+ Repeated request IDs reuse execution without appending references to the cached
1500
+ feedback object; each delivery reports its own history-write attempt.
1501
+ Background observer health remains separate from this foreground write report.
1502
+
1503
+ ## Runtime permission requests use Intent
1504
+
1505
+ Trigger the runtime permission request through the App's existing flow, then call:
1506
+
1507
+ ```json
1508
+ {"command":"permission-dialog","arguments":{"serial":"DEVICE","packageName":"com.example.app","permission":"android.permission.RECORD_AUDIO","outcome":"allow-once"}}
1509
+ ```
1510
+
1511
+ This MCP entry returns an ordinary Intent ID, the actual UI summary, and
1512
+ `permissionDialog`. Choose an exact selector from that summary through `intent
1513
+ decide`. The Host has no permission-button label or resource-ID table. The
1514
+ request's App, Android user, UID and Activity instance must still match at input.
1515
+ The UIA node runtime revalidates the saved reference and current selector;
1516
+ changed text, window identity or duplicate matches stop dispatch. Android ActivityManager currently exposes the requesting App,
1517
+ not the requested permission list. The Agent must match the dialog's meaning to
1518
+ `permission`; the independent PackageManager query verifies that named permission.
1519
+
1520
+ | outcome | Required result after an owned action and original Activity closure |
1521
+ | --- | --- |
1522
+ | `allow` | `granted:true`, `ONE_TIME` absent |
1523
+ | `allow-once` | `granted:true`, `ONE_TIME` present |
1524
+ | `deny` | `granted:false`, `USER_SET` present |
1525
+ | `dismiss` | Grant and all permission flags unchanged from the initial state |
1526
+
1527
+ `deny` accepts only an observed tap. Its state check does not distinguish first
1528
+ refusal from "don't ask again"; the selected visible control is retained in the
1529
+ receipt. `dismiss` accepts an observed tap or `{ "action":"back" }`. A named
1530
+ permission alone does not verify every permission in a grouped request, an app-op,
1531
+ or background access. These require their own observations and state assertions.
1532
+
1533
+ When Android has closed the window but is still retiring its Activity record,
1534
+ `status:waiting_for_observation`, `permissionDialog.pending:activity_closure`
1535
+ means verification is pending. Use `intent observe` to refresh the proof; no
1536
+ second button click is required. Missing/changed evidence never becomes success.
1537
+ Caller `complete` cannot override verification. Export the Intent's plan, UI,
1538
+ decision, receipt and final permission checkpoint with `evidence export`.
1539
+
1540
+ `intent cancel` stops further dispatch, waits for any submitted input to settle,
1541
+ then records the actual state. It does not close the dialog or undo an Android
1542
+ grant. `timeoutMs` defaults to 60000: after that deadline no new input is sent;
1543
+ in-flight bounded ADB calls and final verification can finish later.
1544
+ `adbTimeoutMs` bounds individual ADB calls (default 15000). Pause suspends Agent
1545
+ input while the deadline continues. EOF and signal shutdown finalize active
1546
+ workflows before closing FactStore. Ownership is still within one Host process;
1547
+ external input and abrupt process death are not an exactly-once guarantee.
1548
+
1549
+ The requester probe currently requires one top-resumed Activity and its detailed
1550
+ ActivityManager record. Unsupported/multi-display layouts return explicit errors;
1551
+ there is no ROM-name fallback. These Android contracts do not imply iOS parity.
1552
+
1553
+ `permission-state`, `permission-grant` and `permission-revoke` remain direct
1554
+ CLI/MCP capabilities and Script calls. All require `serial`, `packageName` and
1555
+ `permission`; optional `userId` selects an Android user, otherwise the actual
1556
+ current user is resolved and frozen. They read the runtime-permission section of
1557
+ that exact package/user, never a Host cache or SDK port. `grant`/`revoke` verify the
1558
+ result after one `pm` submission and retain the before state. Failure distinguishes
1559
+ missing runtime permission, unavailable query, rejected change and unverified
1560
+ result. OEM restrictions may reject the shell identity with `permission_change_denied`; the result includes the platform reason and the independent before/after states. No automatic root or UI fallback is attempted. Script requires `app.permissions` for these fixture mutations and
1561
+ `app.read` for queries. `permission-dialog` is a supervised operation and is not
1562
+ a synchronous Script primitive; fixed regression can replay known selectors with
1563
+ explicit state assertions, or request Agent help through `ctx.askAgent`.