dsh-apk-vm 0.10.1 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,62 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.11.0
4
+
5
+ WebView/CDP control, reliable tool JSON, CSS-coordinate input, Unicode typing and reboot-safe network state.
6
+
7
+ ### Added
8
+
9
+ - **Native WebView control over Chrome DevTools Protocol.** New `lib/cdp.js` discovers
10
+ `webview_devtools_remote` sockets through adb, forwards them on loopback, and exposes
11
+ `webview list|eval|click|wait_for|insert_text|info`. Selectors support ordinary CSS and
12
+ `testid:VALUE` for `[data-testid="VALUE"]`, including nested **open** shadow roots.
13
+ Each Agent call owns a short-lived CDP connection; targets are sanitized and WebSocket URLs
14
+ are never returned to the model.
15
+ - `tap_css` clicks by CSS viewport coordinates, applying `info.coordinateMap`
16
+ (`device = round(css * dpr) + offset`) so callers do not hand-calculate a status bar.
17
+ - `tap_selector` resolves a DOM selector to a CDP mouse click.
18
+ - `type_text` sends UTF-8/Unicode text through the existing scrcpy control stream (Chinese,
19
+ emoji and other non-ASCII input no longer depend on ADBKeyboard).
20
+ - `wait_for` supports selector, HTTP, UI-text and seconds conditions with polling instead of
21
+ blind sleeps.
22
+ - `screenshot` accepts `overlay:true` (and `/apk-vm/debug/screenshot?overlay=1`) to draw
23
+ UIAutomator interactive-node boxes on the PNG and return the matching boxes.
24
+ - `networkAction=reverse-add|reverse-remove|reverse-list` manages guest→host `adb reverse`.
25
+ - `networkAction=validate` reports Android's `VALIDATED`/`INTERNET` state and gives a clear
26
+ WebView hint when the captive-portal probe has not passed.
27
+ - `networkInfo` now includes reverse forwards and connectivity validation.
28
+ - `env_snapshot` and `env_restore` capture/re-apply network mode, proxy, DNS intent, hosts and
29
+ forwards. `reboot` schedules this automatically after `sys.boot_completed`.
30
+ - `adb` runs a raw adb argv and returns structured `stdout`/`stderr`/`exit`.
31
+ - `info` now reports adb path, external-device flag, physical/override size, density, `dpr`,
32
+ stable-inset status bar, rotation and a ready-to-use coordinate map.
33
+
34
+ ### Fixed
35
+
36
+ - `apk_vm` no longer returns invalid/lossless-JSON errors: `lib/json-safe.js` normalizes
37
+ `undefined`, `NaN`/`Infinity`, bigint, Buffer/TypedArray, Date, Error and cyclic values before
38
+ every tool result.
39
+ - Tools relying on scrcpy can type Unicode without changing the input protocol.
40
+
41
+ ## 0.10.2
42
+
43
+ Panel administrator mode and owner-header compatibility.
44
+
45
+ ### Added
46
+
47
+ - The human Web panel is now an **administrator** owner: it may operate any VM even when an
48
+ Agent conversation holds the lease. Admin requests bypass the `409 vm-busy` guard but do not
49
+ steal or refresh the agent lease.
50
+ - `x-apk-vm-role: admin` is accepted only from loopback requests; non-loopback callers are
51
+ downgraded to `user`. `ownerStatus`, `ownerLabel` and the panel usage card expose the role.
52
+
53
+ ### Fixed
54
+
55
+ - Browser `fetch` no longer rejects the panel owner label with `String contains non
56
+ ISO-8859-1 code point`; labels and workspace paths are percent-encoded and decoded on Host.
57
+ - Panel **Claim** / **Release** actions force ownership for the local admin, so the human can
58
+ recover control of a VM held by an Agent.
59
+
3
60
  ## 0.10.1
4
61
 
5
62
  Fix panel requests failing with non-ISO-8859-1 header values.
package/README.md CHANGED
@@ -45,7 +45,7 @@ same-origin, loopback-only debug HTTP API and an `apk_vm` Agent tool.
45
45
  > 当前 **Windows x64 与 Linux x86_64(headless)为端到端已验证平台**;macOS 仅做探测/CI
46
46
  > 覆盖,详见 `docs/LINUX-E2E.md`。
47
47
 
48
- **Status:** `0.10.1` · Apache-2.0 · no npm runtime dependencies · no build step.
48
+ **Status:** `0.11.0` · Apache-2.0 · no npm runtime dependencies · no build step.
49
49
 
50
50
  This is an open-source DSH plugin published on npm as **`dsh-apk-vm`**. Repository:
51
51
  [github.com/yuxiaole-bili/dsh-apk-vm](https://github.com/yuxiaole-bili/dsh-apk-vm).
@@ -272,6 +272,36 @@ key {"text":"HOME"} -> returned to launcher
272
272
  - `POST /apk-vm/debug/ca` accepts a host `.pem`/`.crt` path and installs it under the guest's
273
273
  user CA directory with the correct subject hash.
274
274
 
275
+ ## WebView and Chrome DevTools Protocol
276
+
277
+ H5 pages and hybrid Apps often expose nothing useful to UIAutomator. The plugin now speaks CDP
278
+ to the running WebView: it discovers `webview_devtools_remote` sockets, forwards each over
279
+ `adb forward` on loopback, and keeps the connection internal to the Host process.
280
+
281
+ - `apk_vm` actions: `webview` (`webviewAction=list|eval|click|wait_for|insert_text|info`),
282
+ `tap_selector`, `tap_css`, `type_text`, `wait_for`.
283
+ - `webviewAction=list` returns sanitized target metadata (`id`, `title`, `url`, `type`), never a
284
+ DevTools WebSocket URL or token. Select a target with `targetId` or `webviewTarget`.
285
+ - `webviewAction=eval` evaluates a JSON-returning expression; `click` and `wait_for` accept
286
+ ordinary CSS and `testid:VALUE` (which maps to `[data-testid="VALUE"]`). **Open** shadow roots
287
+ are traversed; closed shadow roots and iframes are not.
288
+ - `tap_selector` uses the same deep query and clicks through CDP. `tap_css` is for canvas or
289
+ non-DOM UI: it converts CSS viewport coordinates with `info.coordinateMap`.
290
+ - `type_text` injects Unicode through the scrcpy control stream; it does not require ADBKeyboard.
291
+ - `wait_for` accepts a selector, an HTTP URL, a UI text condition, or a number of seconds. It
292
+ polls and returns the elapsed time instead of forcing fixed sleeps.
293
+ - CDP is diagnostic/automation only and is never exposed to the network: the bridge binds to
294
+ loopback and closes its forwards when the Agent call/plugin disposes.
295
+
296
+ Example Agent sequence for a hybrid App:
297
+
298
+ ```text
299
+ apk_vm { action: "webview", webviewAction: "list" }
300
+ apk_vm { action: "webview", webviewAction: "click", selector: "testid:login" }
301
+ apk_vm { action: "type_text", text: "中文角色名" }
302
+ apk_vm { action: "wait_for", selector: "[data-testid=home]" }
303
+ ```
304
+
275
305
  ## App-operation Skills and Control Codebook
276
306
 
277
307
  App-operation Skills are short, replayable navigation recipes for tests. Instead of recording
@@ -345,6 +375,10 @@ or `project`), session id, workspace, and `since` / `lastSeenAt` timestamps; lea
345
375
  - The VM instances card shows the current owner/scope. The **VM usage** card in the panel
346
376
  shows a readable owner label and provides **Claim**, **Release** and a `shared` /
347
377
  `conversation` / `project` scope control for the human user.
378
+ - The Web panel is an administrator: it can operate a VM held by an Agent conversation and may
379
+ force **Claim** / **Release**, while ordinary Agent calls still get `409 vm-busy`. The panel
380
+ sends `x-apk-vm-role: admin`; the Host accepts that role only from loopback requests and
381
+ downgrades non-local callers to `user`.
348
382
  - `owner_status` lists default/extra VM leases; `claim_vm` and `release_vm` manage them.
349
383
  HTTP: `GET|POST /apk-vm/ownership` (`status`, `claim`, `release`, `touch`), while
350
384
  `GET /apk-vm/state` and `GET /apk-vm/instances` include owner metadata.
@@ -393,6 +427,9 @@ port forwards and pcap status. `POST /apk-vm/network` accepts `action`:
393
427
  { "action": "dns", "servers": ["1.1.1.1"] }
394
428
  { "action": "hosts", "entries": ["198.51.100.10 api.example.test"] }
395
429
  { "action": "test", "host": "198.51.100.10", "port": 8080 }
430
+ { "action": "reverse-add", "localPort": 8080, "hostPort": 3000 }
431
+ { "action": "reverse-remove", "localPort": 8080 }
432
+ { "action": "validate" }
396
433
  { "action": "pcap", "path": "D:\\apk-vm-capture.pcap" }
397
434
  ```
398
435
 
@@ -406,6 +443,11 @@ port forwards and pcap status. `POST /apk-vm/network` accepts `action`:
406
443
  QEMU NAT itself remains isolated.
407
444
  - pcap capture is passed to the emulator as `-tcpdump` and starts on the next emulator start;
408
445
  download via `GET /apk-vm/network/pcap`.
446
+ - `reverse-add` maps guest `127.0.0.1:<localPort>` to host `127.0.0.1:<hostPort>` with
447
+ `adb reverse`; `reboot` re-adds it during environment restore.
448
+ - `validate` reports Android's `VALIDATED`/`INTERNET` state. An isolated QEMU NAT cannot pass
449
+ the captive-portal probe, so WebView may reject non-loopback HTTP even when `nc`/`curl`
450
+ succeed; the action returns an explicit hint instead of pretending the network is validated.
409
451
 
410
452
 
411
453
  A public, dependency-free demo lives in [examples](examples/README.md):
@@ -762,14 +804,19 @@ Host half prepares a rooted debug environment and captures it in the quick-boot
762
804
  | POST | `/apk-vm/debug/record` | `{"seconds":1–60}` (default 5) → `screenrecord` MP4 pulled to the host temp dir; returns `localPath` and byte size; the panel auto-downloads it. |
763
805
  | GET | `/apk-vm/artifacts?name=…` | Download a screenshot/recording previously written by the plugin into the OS temp dir (basename whitelist, no arbitrary paths). |
764
806
  | GET | `/apk-vm/debug/perf` | load average, `/proc/meminfo` totals/free/available and the first lines of `dumpsys cpuinfo`. |
765
- | GET/HEAD | `/apk-vm/debug/screenshot` | PNG download (`Content-Disposition: attachment`). |
807
+ | GET/HEAD | `/apk-vm/debug/screenshot` | PNG download (`Content-Disposition: attachment`). Add `?overlay=1` to draw the current UIAutomator interactive-node boxes on the image. |
808
+ | POST | `/apk-vm/debug/env` | `{"action":"capture"\|"restore","envSnapshot":{…}}` — snapshot or re-apply network mode, proxy, DNS intent, hosts and port/reverse forwards. |
766
809
 
767
810
  ### Agent tool `apk_vm`
768
811
 
769
812
  Registered globally through `ctx.tools`, so every DSH agent can operate the rooted VM directly:
770
813
  `info`, `shell`, `logcat`, `getprop`, `setprop`, `packages`, `install`, `push`, `pull`,
771
814
  `reboot`, `root`, `force_stop`, `clear_data`, `uninstall`, `rotate`, `ui_dump`, `record`, `perf`,
772
- `screenshot`, high-level input `tap` / `swipe` / `long_press` / `key` / `text` (device coordinates),
815
+ `screenshot` (add `overlay:true` for UIAutomator boxes), high-level input `tap` / `swipe` / `long_press` / `key` / `text`
816
+ (device coordinates), **CSS-coordinate input** `tap_css` / `tap_selector`, Unicode `type_text`,
817
+ **WebView CDP** `webview` (`list|eval|click|wait_for|insert_text|info`), conditional `wait_for`,
818
+ network `reverse-add` / `reverse-remove` / `reverse-list` / `validate`, environment
819
+ `env_snapshot` / `env_restore` and raw `adb` argv,
773
820
  UI automation `find_ui` / `assert_text` / `tap_text` (UIAutomator text/resource-id/content-desc/class),
774
821
  clipboard `clipboard_get` / `clipboard_set`,
775
822
  static `apk_info` (metadata without booting a VM),
package/client.js CHANGED
@@ -30,7 +30,7 @@ window.__ModuleLoader__.load({
30
30
  var SWIPE_THRESHOLD_PX = 24;
31
31
  var FRAME_INTERVAL_MS = 500;
32
32
  var STATE_INTERVAL_MS = 2000;
33
- var CLIENT_VERSION = '0.10.1';
33
+ var CLIENT_VERSION = '0.11.0';
34
34
 
35
35
  /* ------------------------------------------------------------------ *
36
36
  * Optional dsh-better-sidebar integration.
@@ -126,6 +126,7 @@ window.__ModuleLoader__.load({
126
126
  'owner.busy': 'in use by {label}',
127
127
  'owner.scope': 'Scope',
128
128
  'owner.forceHint': 'Another owner holds this VM; release it first.',
129
+ 'owner.admin': 'Administrator — can operate every VM',
129
130
  'action.start': 'Start VM',
130
131
  'action.starting': 'Starting…',
131
132
  'action.stop': 'Stop',
@@ -443,6 +444,7 @@ window.__ModuleLoader__.load({
443
444
  'owner.busy': '被 {label} 占用',
444
445
  'owner.scope': '范围',
445
446
  'owner.forceHint': '其他使用者正在占用,请先释放。',
447
+ 'owner.admin': '管理员 · 可操作所有虚拟机',
446
448
  'action.start': '启动虚拟机',
447
449
  'action.starting': '正在启动…',
448
450
  'action.stop': '停止',
@@ -850,7 +852,8 @@ window.__ModuleLoader__.load({
850
852
  return {
851
853
  'x-apk-vm-owner': panelOwnerKey(),
852
854
  'x-apk-vm-scope': panelOwnerScope(),
853
- 'x-apk-vm-label': encodeURIComponent('桌面面板'),
855
+ 'x-apk-vm-role': 'admin',
856
+ 'x-apk-vm-label': encodeURIComponent('桌面管理员'),
854
857
  };
855
858
  }
856
859
 
@@ -4692,6 +4695,7 @@ window.__ModuleLoader__.load({
4692
4695
  h(
4693
4696
  'div',
4694
4697
  { className: 'apkvm-ownerbody' },
4698
+ h('div', { className: 'apkvm-owneradmin' }, T('owner.admin')),
4695
4699
  h(
4696
4700
  'div',
4697
4701
  { className: 'apkvm-ownerrow' },