dsh-apk-vm 0.10.2 → 0.11.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,56 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.11.1
4
+
5
+ Clipboard/text tools no longer need a panel to already have a control stream; network proxy has an alias.
6
+
7
+ ### Fixed
8
+
9
+ - `clipboard_get` / `clipboard_set` and Unicode `type_text` now start the scrcpy control
10
+ stream on demand when no panel client is connected, then release it through the normal
11
+ idle timer. Clipboard reads time out with `504` instead of misreporting a stale stream
12
+ as a missing one (`409`).
13
+ - `action=network, networkAction=proxy` is accepted as an alias for the top-level `proxy`
14
+ action, including `enabled:false` / `clear:true` to disable it.
15
+
16
+ ## 0.11.0
17
+
18
+ WebView/CDP control, reliable tool JSON, CSS-coordinate input, Unicode typing and reboot-safe network state.
19
+
20
+ ### Added
21
+
22
+ - **Native WebView control over Chrome DevTools Protocol.** New `lib/cdp.js` discovers
23
+ `webview_devtools_remote` sockets through adb, forwards them on loopback, and exposes
24
+ `webview list|eval|click|wait_for|insert_text|info`. Selectors support ordinary CSS and
25
+ `testid:VALUE` for `[data-testid="VALUE"]`, including nested **open** shadow roots.
26
+ Each Agent call owns a short-lived CDP connection; targets are sanitized and WebSocket URLs
27
+ are never returned to the model.
28
+ - `tap_css` clicks by CSS viewport coordinates, applying `info.coordinateMap`
29
+ (`device = round(css * dpr) + offset`) so callers do not hand-calculate a status bar.
30
+ - `tap_selector` resolves a DOM selector to a CDP mouse click.
31
+ - `type_text` sends UTF-8/Unicode text through the existing scrcpy control stream (Chinese,
32
+ emoji and other non-ASCII input no longer depend on ADBKeyboard).
33
+ - `wait_for` supports selector, HTTP, UI-text and seconds conditions with polling instead of
34
+ blind sleeps.
35
+ - `screenshot` accepts `overlay:true` (and `/apk-vm/debug/screenshot?overlay=1`) to draw
36
+ UIAutomator interactive-node boxes on the PNG and return the matching boxes.
37
+ - `networkAction=reverse-add|reverse-remove|reverse-list` manages guest→host `adb reverse`.
38
+ - `networkAction=validate` reports Android's `VALIDATED`/`INTERNET` state and gives a clear
39
+ WebView hint when the captive-portal probe has not passed.
40
+ - `networkInfo` now includes reverse forwards and connectivity validation.
41
+ - `env_snapshot` and `env_restore` capture/re-apply network mode, proxy, DNS intent, hosts and
42
+ forwards. `reboot` schedules this automatically after `sys.boot_completed`.
43
+ - `adb` runs a raw adb argv and returns structured `stdout`/`stderr`/`exit`.
44
+ - `info` now reports adb path, external-device flag, physical/override size, density, `dpr`,
45
+ stable-inset status bar, rotation and a ready-to-use coordinate map.
46
+
47
+ ### Fixed
48
+
49
+ - `apk_vm` no longer returns invalid/lossless-JSON errors: `lib/json-safe.js` normalizes
50
+ `undefined`, `NaN`/`Infinity`, bigint, Buffer/TypedArray, Date, Error and cyclic values before
51
+ every tool result.
52
+ - Tools relying on scrcpy can type Unicode without changing the input protocol.
53
+
3
54
  ## 0.10.2
4
55
 
5
56
  Panel administrator mode and owner-header compatibility.
package/README.md CHANGED
@@ -45,7 +45,7 @@ same-origin, loopback-only debug HTTP API and an `apk_vm` Agent tool.
45
45
  > 当前 **Windows x64 与 Linux x86_64(headless)为端到端已验证平台**;macOS 仅做探测/CI
46
46
  > 覆盖,详见 `docs/LINUX-E2E.md`。
47
47
 
48
- **Status:** `0.10.2` · Apache-2.0 · no npm runtime dependencies · no build step.
48
+ **Status:** `0.11.1` · Apache-2.0 · no npm runtime dependencies · no build step.
49
49
 
50
50
  This is an open-source DSH plugin published on npm as **`dsh-apk-vm`**. Repository:
51
51
  [github.com/yuxiaole-bili/dsh-apk-vm](https://github.com/yuxiaole-bili/dsh-apk-vm).
@@ -272,6 +272,36 @@ key {"text":"HOME"} -> returned to launcher
272
272
  - `POST /apk-vm/debug/ca` accepts a host `.pem`/`.crt` path and installs it under the guest's
273
273
  user CA directory with the correct subject hash.
274
274
 
275
+ ## WebView and Chrome DevTools Protocol
276
+
277
+ H5 pages and hybrid Apps often expose nothing useful to UIAutomator. The plugin now speaks CDP
278
+ to the running WebView: it discovers `webview_devtools_remote` sockets, forwards each over
279
+ `adb forward` on loopback, and keeps the connection internal to the Host process.
280
+
281
+ - `apk_vm` actions: `webview` (`webviewAction=list|eval|click|wait_for|insert_text|info`),
282
+ `tap_selector`, `tap_css`, `type_text`, `wait_for`.
283
+ - `webviewAction=list` returns sanitized target metadata (`id`, `title`, `url`, `type`), never a
284
+ DevTools WebSocket URL or token. Select a target with `targetId` or `webviewTarget`.
285
+ - `webviewAction=eval` evaluates a JSON-returning expression; `click` and `wait_for` accept
286
+ ordinary CSS and `testid:VALUE` (which maps to `[data-testid="VALUE"]`). **Open** shadow roots
287
+ are traversed; closed shadow roots and iframes are not.
288
+ - `tap_selector` uses the same deep query and clicks through CDP. `tap_css` is for canvas or
289
+ non-DOM UI: it converts CSS viewport coordinates with `info.coordinateMap`.
290
+ - `type_text` injects Unicode through the scrcpy control stream; it does not require ADBKeyboard.
291
+ - `wait_for` accepts a selector, an HTTP URL, a UI text condition, or a number of seconds. It
292
+ polls and returns the elapsed time instead of forcing fixed sleeps.
293
+ - CDP is diagnostic/automation only and is never exposed to the network: the bridge binds to
294
+ loopback and closes its forwards when the Agent call/plugin disposes.
295
+
296
+ Example Agent sequence for a hybrid App:
297
+
298
+ ```text
299
+ apk_vm { action: "webview", webviewAction: "list" }
300
+ apk_vm { action: "webview", webviewAction: "click", selector: "testid:login" }
301
+ apk_vm { action: "type_text", text: "中文角色名" }
302
+ apk_vm { action: "wait_for", selector: "[data-testid=home]" }
303
+ ```
304
+
275
305
  ## App-operation Skills and Control Codebook
276
306
 
277
307
  App-operation Skills are short, replayable navigation recipes for tests. Instead of recording
@@ -397,6 +427,9 @@ port forwards and pcap status. `POST /apk-vm/network` accepts `action`:
397
427
  { "action": "dns", "servers": ["1.1.1.1"] }
398
428
  { "action": "hosts", "entries": ["198.51.100.10 api.example.test"] }
399
429
  { "action": "test", "host": "198.51.100.10", "port": 8080 }
430
+ { "action": "reverse-add", "localPort": 8080, "hostPort": 3000 }
431
+ { "action": "reverse-remove", "localPort": 8080 }
432
+ { "action": "validate" }
400
433
  { "action": "pcap", "path": "D:\\apk-vm-capture.pcap" }
401
434
  ```
402
435
 
@@ -410,6 +443,11 @@ port forwards and pcap status. `POST /apk-vm/network` accepts `action`:
410
443
  QEMU NAT itself remains isolated.
411
444
  - pcap capture is passed to the emulator as `-tcpdump` and starts on the next emulator start;
412
445
  download via `GET /apk-vm/network/pcap`.
446
+ - `reverse-add` maps guest `127.0.0.1:<localPort>` to host `127.0.0.1:<hostPort>` with
447
+ `adb reverse`; `reboot` re-adds it during environment restore.
448
+ - `validate` reports Android's `VALIDATED`/`INTERNET` state. An isolated QEMU NAT cannot pass
449
+ the captive-portal probe, so WebView may reject non-loopback HTTP even when `nc`/`curl`
450
+ succeed; the action returns an explicit hint instead of pretending the network is validated.
413
451
 
414
452
 
415
453
  A public, dependency-free demo lives in [examples](examples/README.md):
@@ -766,14 +804,19 @@ Host half prepares a rooted debug environment and captures it in the quick-boot
766
804
  | POST | `/apk-vm/debug/record` | `{"seconds":1–60}` (default 5) → `screenrecord` MP4 pulled to the host temp dir; returns `localPath` and byte size; the panel auto-downloads it. |
767
805
  | GET | `/apk-vm/artifacts?name=…` | Download a screenshot/recording previously written by the plugin into the OS temp dir (basename whitelist, no arbitrary paths). |
768
806
  | GET | `/apk-vm/debug/perf` | load average, `/proc/meminfo` totals/free/available and the first lines of `dumpsys cpuinfo`. |
769
- | GET/HEAD | `/apk-vm/debug/screenshot` | PNG download (`Content-Disposition: attachment`). |
807
+ | GET/HEAD | `/apk-vm/debug/screenshot` | PNG download (`Content-Disposition: attachment`). Add `?overlay=1` to draw the current UIAutomator interactive-node boxes on the image. |
808
+ | POST | `/apk-vm/debug/env` | `{"action":"capture"\|"restore","envSnapshot":{…}}` — snapshot or re-apply network mode, proxy, DNS intent, hosts and port/reverse forwards. |
770
809
 
771
810
  ### Agent tool `apk_vm`
772
811
 
773
812
  Registered globally through `ctx.tools`, so every DSH agent can operate the rooted VM directly:
774
813
  `info`, `shell`, `logcat`, `getprop`, `setprop`, `packages`, `install`, `push`, `pull`,
775
814
  `reboot`, `root`, `force_stop`, `clear_data`, `uninstall`, `rotate`, `ui_dump`, `record`, `perf`,
776
- `screenshot`, high-level input `tap` / `swipe` / `long_press` / `key` / `text` (device coordinates),
815
+ `screenshot` (add `overlay:true` for UIAutomator boxes), high-level input `tap` / `swipe` / `long_press` / `key` / `text`
816
+ (device coordinates), **CSS-coordinate input** `tap_css` / `tap_selector`, Unicode `type_text`,
817
+ **WebView CDP** `webview` (`list|eval|click|wait_for|insert_text|info`), conditional `wait_for`,
818
+ network `reverse-add` / `reverse-remove` / `reverse-list` / `validate`, environment
819
+ `env_snapshot` / `env_restore` and raw `adb` argv,
777
820
  UI automation `find_ui` / `assert_text` / `tap_text` (UIAutomator text/resource-id/content-desc/class),
778
821
  clipboard `clipboard_get` / `clipboard_set`,
779
822
  static `apk_info` (metadata without booting a VM),
package/client.js CHANGED
@@ -30,7 +30,7 @@ window.__ModuleLoader__.load({
30
30
  var SWIPE_THRESHOLD_PX = 24;
31
31
  var FRAME_INTERVAL_MS = 500;
32
32
  var STATE_INTERVAL_MS = 2000;
33
- var CLIENT_VERSION = '0.10.2';
33
+ var CLIENT_VERSION = '0.11.1';
34
34
 
35
35
  /* ------------------------------------------------------------------ *
36
36
  * Optional dsh-better-sidebar integration.