dsh-apk-vm 0.10.2 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +38 -0
- package/README.md +46 -3
- package/client.js +1 -1
- package/index.js +376 -8
- package/lib/android.js +258 -6
- package/lib/cdp.js +1166 -0
- package/lib/instances.js +10 -2
- package/lib/json-safe.js +39 -0
- package/lib/png-annotate.js +114 -0
- package/lib/video-stream.js +8 -0
- package/package.json +6 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,43 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.11.0
|
|
4
|
+
|
|
5
|
+
WebView/CDP control, reliable tool JSON, CSS-coordinate input, Unicode typing and reboot-safe network state.
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- **Native WebView control over Chrome DevTools Protocol.** New `lib/cdp.js` discovers
|
|
10
|
+
`webview_devtools_remote` sockets through adb, forwards them on loopback, and exposes
|
|
11
|
+
`webview list|eval|click|wait_for|insert_text|info`. Selectors support ordinary CSS and
|
|
12
|
+
`testid:VALUE` for `[data-testid="VALUE"]`, including nested **open** shadow roots.
|
|
13
|
+
Each Agent call owns a short-lived CDP connection; targets are sanitized and WebSocket URLs
|
|
14
|
+
are never returned to the model.
|
|
15
|
+
- `tap_css` clicks by CSS viewport coordinates, applying `info.coordinateMap`
|
|
16
|
+
(`device = round(css * dpr) + offset`) so callers do not hand-calculate a status bar.
|
|
17
|
+
- `tap_selector` resolves a DOM selector to a CDP mouse click.
|
|
18
|
+
- `type_text` sends UTF-8/Unicode text through the existing scrcpy control stream (Chinese,
|
|
19
|
+
emoji and other non-ASCII input no longer depend on ADBKeyboard).
|
|
20
|
+
- `wait_for` supports selector, HTTP, UI-text and seconds conditions with polling instead of
|
|
21
|
+
blind sleeps.
|
|
22
|
+
- `screenshot` accepts `overlay:true` (and `/apk-vm/debug/screenshot?overlay=1`) to draw
|
|
23
|
+
UIAutomator interactive-node boxes on the PNG and return the matching boxes.
|
|
24
|
+
- `networkAction=reverse-add|reverse-remove|reverse-list` manages guest→host `adb reverse`.
|
|
25
|
+
- `networkAction=validate` reports Android's `VALIDATED`/`INTERNET` state and gives a clear
|
|
26
|
+
WebView hint when the captive-portal probe has not passed.
|
|
27
|
+
- `networkInfo` now includes reverse forwards and connectivity validation.
|
|
28
|
+
- `env_snapshot` and `env_restore` capture/re-apply network mode, proxy, DNS intent, hosts and
|
|
29
|
+
forwards. `reboot` schedules this automatically after `sys.boot_completed`.
|
|
30
|
+
- `adb` runs a raw adb argv and returns structured `stdout`/`stderr`/`exit`.
|
|
31
|
+
- `info` now reports adb path, external-device flag, physical/override size, density, `dpr`,
|
|
32
|
+
stable-inset status bar, rotation and a ready-to-use coordinate map.
|
|
33
|
+
|
|
34
|
+
### Fixed
|
|
35
|
+
|
|
36
|
+
- `apk_vm` no longer returns invalid/lossless-JSON errors: `lib/json-safe.js` normalizes
|
|
37
|
+
`undefined`, `NaN`/`Infinity`, bigint, Buffer/TypedArray, Date, Error and cyclic values before
|
|
38
|
+
every tool result.
|
|
39
|
+
- Tools relying on scrcpy can type Unicode without changing the input protocol.
|
|
40
|
+
|
|
3
41
|
## 0.10.2
|
|
4
42
|
|
|
5
43
|
Panel administrator mode and owner-header compatibility.
|
package/README.md
CHANGED
|
@@ -45,7 +45,7 @@ same-origin, loopback-only debug HTTP API and an `apk_vm` Agent tool.
|
|
|
45
45
|
> 当前 **Windows x64 与 Linux x86_64(headless)为端到端已验证平台**;macOS 仅做探测/CI
|
|
46
46
|
> 覆盖,详见 `docs/LINUX-E2E.md`。
|
|
47
47
|
|
|
48
|
-
**Status:** `0.
|
|
48
|
+
**Status:** `0.11.0` · Apache-2.0 · no npm runtime dependencies · no build step.
|
|
49
49
|
|
|
50
50
|
This is an open-source DSH plugin published on npm as **`dsh-apk-vm`**. Repository:
|
|
51
51
|
[github.com/yuxiaole-bili/dsh-apk-vm](https://github.com/yuxiaole-bili/dsh-apk-vm).
|
|
@@ -272,6 +272,36 @@ key {"text":"HOME"} -> returned to launcher
|
|
|
272
272
|
- `POST /apk-vm/debug/ca` accepts a host `.pem`/`.crt` path and installs it under the guest's
|
|
273
273
|
user CA directory with the correct subject hash.
|
|
274
274
|
|
|
275
|
+
## WebView and Chrome DevTools Protocol
|
|
276
|
+
|
|
277
|
+
H5 pages and hybrid Apps often expose nothing useful to UIAutomator. The plugin now speaks CDP
|
|
278
|
+
to the running WebView: it discovers `webview_devtools_remote` sockets, forwards each over
|
|
279
|
+
`adb forward` on loopback, and keeps the connection internal to the Host process.
|
|
280
|
+
|
|
281
|
+
- `apk_vm` actions: `webview` (`webviewAction=list|eval|click|wait_for|insert_text|info`),
|
|
282
|
+
`tap_selector`, `tap_css`, `type_text`, `wait_for`.
|
|
283
|
+
- `webviewAction=list` returns sanitized target metadata (`id`, `title`, `url`, `type`), never a
|
|
284
|
+
DevTools WebSocket URL or token. Select a target with `targetId` or `webviewTarget`.
|
|
285
|
+
- `webviewAction=eval` evaluates a JSON-returning expression; `click` and `wait_for` accept
|
|
286
|
+
ordinary CSS and `testid:VALUE` (which maps to `[data-testid="VALUE"]`). **Open** shadow roots
|
|
287
|
+
are traversed; closed shadow roots and iframes are not.
|
|
288
|
+
- `tap_selector` uses the same deep query and clicks through CDP. `tap_css` is for canvas or
|
|
289
|
+
non-DOM UI: it converts CSS viewport coordinates with `info.coordinateMap`.
|
|
290
|
+
- `type_text` injects Unicode through the scrcpy control stream; it does not require ADBKeyboard.
|
|
291
|
+
- `wait_for` accepts a selector, an HTTP URL, a UI text condition, or a number of seconds. It
|
|
292
|
+
polls and returns the elapsed time instead of forcing fixed sleeps.
|
|
293
|
+
- CDP is diagnostic/automation only and is never exposed to the network: the bridge binds to
|
|
294
|
+
loopback and closes its forwards when the Agent call/plugin disposes.
|
|
295
|
+
|
|
296
|
+
Example Agent sequence for a hybrid App:
|
|
297
|
+
|
|
298
|
+
```text
|
|
299
|
+
apk_vm { action: "webview", webviewAction: "list" }
|
|
300
|
+
apk_vm { action: "webview", webviewAction: "click", selector: "testid:login" }
|
|
301
|
+
apk_vm { action: "type_text", text: "中文角色名" }
|
|
302
|
+
apk_vm { action: "wait_for", selector: "[data-testid=home]" }
|
|
303
|
+
```
|
|
304
|
+
|
|
275
305
|
## App-operation Skills and Control Codebook
|
|
276
306
|
|
|
277
307
|
App-operation Skills are short, replayable navigation recipes for tests. Instead of recording
|
|
@@ -397,6 +427,9 @@ port forwards and pcap status. `POST /apk-vm/network` accepts `action`:
|
|
|
397
427
|
{ "action": "dns", "servers": ["1.1.1.1"] }
|
|
398
428
|
{ "action": "hosts", "entries": ["198.51.100.10 api.example.test"] }
|
|
399
429
|
{ "action": "test", "host": "198.51.100.10", "port": 8080 }
|
|
430
|
+
{ "action": "reverse-add", "localPort": 8080, "hostPort": 3000 }
|
|
431
|
+
{ "action": "reverse-remove", "localPort": 8080 }
|
|
432
|
+
{ "action": "validate" }
|
|
400
433
|
{ "action": "pcap", "path": "D:\\apk-vm-capture.pcap" }
|
|
401
434
|
```
|
|
402
435
|
|
|
@@ -410,6 +443,11 @@ port forwards and pcap status. `POST /apk-vm/network` accepts `action`:
|
|
|
410
443
|
QEMU NAT itself remains isolated.
|
|
411
444
|
- pcap capture is passed to the emulator as `-tcpdump` and starts on the next emulator start;
|
|
412
445
|
download via `GET /apk-vm/network/pcap`.
|
|
446
|
+
- `reverse-add` maps guest `127.0.0.1:<localPort>` to host `127.0.0.1:<hostPort>` with
|
|
447
|
+
`adb reverse`; `reboot` re-adds it during environment restore.
|
|
448
|
+
- `validate` reports Android's `VALIDATED`/`INTERNET` state. An isolated QEMU NAT cannot pass
|
|
449
|
+
the captive-portal probe, so WebView may reject non-loopback HTTP even when `nc`/`curl`
|
|
450
|
+
succeed; the action returns an explicit hint instead of pretending the network is validated.
|
|
413
451
|
|
|
414
452
|
|
|
415
453
|
A public, dependency-free demo lives in [examples](examples/README.md):
|
|
@@ -766,14 +804,19 @@ Host half prepares a rooted debug environment and captures it in the quick-boot
|
|
|
766
804
|
| POST | `/apk-vm/debug/record` | `{"seconds":1–60}` (default 5) → `screenrecord` MP4 pulled to the host temp dir; returns `localPath` and byte size; the panel auto-downloads it. |
|
|
767
805
|
| GET | `/apk-vm/artifacts?name=…` | Download a screenshot/recording previously written by the plugin into the OS temp dir (basename whitelist, no arbitrary paths). |
|
|
768
806
|
| GET | `/apk-vm/debug/perf` | load average, `/proc/meminfo` totals/free/available and the first lines of `dumpsys cpuinfo`. |
|
|
769
|
-
| GET/HEAD | `/apk-vm/debug/screenshot` | PNG download (`Content-Disposition: attachment`). |
|
|
807
|
+
| GET/HEAD | `/apk-vm/debug/screenshot` | PNG download (`Content-Disposition: attachment`). Add `?overlay=1` to draw the current UIAutomator interactive-node boxes on the image. |
|
|
808
|
+
| POST | `/apk-vm/debug/env` | `{"action":"capture"\|"restore","envSnapshot":{…}}` — snapshot or re-apply network mode, proxy, DNS intent, hosts and port/reverse forwards. |
|
|
770
809
|
|
|
771
810
|
### Agent tool `apk_vm`
|
|
772
811
|
|
|
773
812
|
Registered globally through `ctx.tools`, so every DSH agent can operate the rooted VM directly:
|
|
774
813
|
`info`, `shell`, `logcat`, `getprop`, `setprop`, `packages`, `install`, `push`, `pull`,
|
|
775
814
|
`reboot`, `root`, `force_stop`, `clear_data`, `uninstall`, `rotate`, `ui_dump`, `record`, `perf`,
|
|
776
|
-
`screenshot
|
|
815
|
+
`screenshot` (add `overlay:true` for UIAutomator boxes), high-level input `tap` / `swipe` / `long_press` / `key` / `text`
|
|
816
|
+
(device coordinates), **CSS-coordinate input** `tap_css` / `tap_selector`, Unicode `type_text`,
|
|
817
|
+
**WebView CDP** `webview` (`list|eval|click|wait_for|insert_text|info`), conditional `wait_for`,
|
|
818
|
+
network `reverse-add` / `reverse-remove` / `reverse-list` / `validate`, environment
|
|
819
|
+
`env_snapshot` / `env_restore` and raw `adb` argv,
|
|
777
820
|
UI automation `find_ui` / `assert_text` / `tap_text` (UIAutomator text/resource-id/content-desc/class),
|
|
778
821
|
clipboard `clipboard_get` / `clipboard_set`,
|
|
779
822
|
static `apk_info` (metadata without booting a VM),
|
package/client.js
CHANGED
|
@@ -30,7 +30,7 @@ window.__ModuleLoader__.load({
|
|
|
30
30
|
var SWIPE_THRESHOLD_PX = 24;
|
|
31
31
|
var FRAME_INTERVAL_MS = 500;
|
|
32
32
|
var STATE_INTERVAL_MS = 2000;
|
|
33
|
-
var CLIENT_VERSION = '0.
|
|
33
|
+
var CLIENT_VERSION = '0.11.0';
|
|
34
34
|
|
|
35
35
|
/* ------------------------------------------------------------------ *
|
|
36
36
|
* Optional dsh-better-sidebar integration.
|