dsh-apk-vm 0.8.1 → 0.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +68 -0
- package/README.md +112 -8
- package/client.js +551 -9
- package/index.js +904 -15
- package/lib/app-skills.js +592 -0
- package/lib/control-codebook.js +374 -0
- package/lib/http-owner.js +80 -0
- package/lib/image-features.js +329 -0
- package/lib/instances.js +225 -43
- package/lib/ui-inspector.js +61 -2
- package/lib/vm-ownership.js +254 -0
- package/package.json +8 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,73 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.10.1
|
|
4
|
+
|
|
5
|
+
Fix panel requests failing with non-ISO-8859-1 header values.
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
- Percent-encode the panel owner label before sending it as `x-apk-vm-label`, so Chromium
|
|
10
|
+
`fetch` no longer rejects requests with `String contains non ISO-8859-1 code point`.
|
|
11
|
+
- Decode percent-encoded owner HTTP headers/query values on the Host, including Chinese
|
|
12
|
+
workspace paths and labels.
|
|
13
|
+
|
|
14
|
+
## 0.10.0
|
|
15
|
+
|
|
16
|
+
VM ownership and dedicated conversation/project VMs.
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- **Renewable VM owner leases** — each conversation owns the VM it drives; a second conversation
|
|
21
|
+
gets its own instance instead of silently taking over the default VM.
|
|
22
|
+
- **Ownership scopes** — `shared` (first come), `conversation` (dedicated per DSH session), and
|
|
23
|
+
`project` (dedicated per workspace). Leases expire after 30 minutes and renew on `apk_vm`
|
|
24
|
+
calls that route to the VM.
|
|
25
|
+
- **Conflict protection** — an explicit foreign `instance` returns `vm-busy` with the current
|
|
26
|
+
owner; `ownerForce=true` is an explicit takeover.
|
|
27
|
+
- **Agent actions** — `owner_status`, `claim_vm`, `release_vm`, plus `ownerScope` / `ownerLabel`
|
|
28
|
+
on normal VM actions.
|
|
29
|
+
- **HTTP API** — `GET|POST /apk-vm/ownership` (`status`, `claim`, `release`, `touch`); owner
|
|
30
|
+
metadata in `GET /apk-vm/state` and `/apk-vm/instances`.
|
|
31
|
+
- **Panel display** — the VM instances card shows the current owner and ownership scope.
|
|
32
|
+
- **Panel usage card** — human-readable owner label plus Claim / Release buttons and a
|
|
33
|
+
`shared` / `conversation` / `project` scope control; mutating panel HTTP requests carry
|
|
34
|
+
the matching owner headers.
|
|
35
|
+
- **HTTP owner validation** — state-changing `/apk-vm/*` requests return `409 vm-busy` with
|
|
36
|
+
the current owner when their `x-apk-vm-owner` header does not match; GET routes stay open.
|
|
37
|
+
The panel also sends `x-apk-vm-scope` / `x-apk-vm-session` / `x-apk-vm-workspace` /
|
|
38
|
+
`x-apk-vm-label`, and a matching owner header auto-claims a free VM.
|
|
39
|
+
- **Automatic release** — the `session/disposed` Host event releases every lease held by a
|
|
40
|
+
closed conversation, with the lease TTL as fallback.
|
|
41
|
+
- **Readable labels** — `ownerLabelFor()` now shows `对话 …`, `项目 <folder>` or `共享 …`
|
|
42
|
+
instead of a raw key.
|
|
43
|
+
- **Port presets** — one-click Metro 8081, HTTP 8080, Frida 27042 and DevTools 9222 forwards.
|
|
44
|
+
|
|
45
|
+
## 0.9.0
|
|
46
|
+
|
|
47
|
+
App-operation Skills and the Control Codebook.
|
|
48
|
+
|
|
49
|
+
### Added
|
|
50
|
+
|
|
51
|
+
- **App-operation Skills** — record a semantic navigation once, then replay it fast to a target
|
|
52
|
+
page for tests. Skills embed their own Control Codebook, so they keep working when UIAutomator
|
|
53
|
+
node order changes. Stored under the workspace `.apk-vm/skills/` directory.
|
|
54
|
+
- **Control Codebook** — numeric (`A01`) and semantic (`LOGIN_BTN`) codes plus aliases, matched
|
|
55
|
+
by structural features (`resourceId`, `text`, `textAny`, `contentDesc`, `className`,
|
|
56
|
+
`clickable`, `enabled`, bounds ratio) with deterministic ambiguity errors.
|
|
57
|
+
- **Image features** — optional PNG dHash plus average/dominant colour for icon-only controls;
|
|
58
|
+
`control_features` returns a ready-to-save entry with a human image name.
|
|
59
|
+
- **Agent actions** — `skill_list/get/save/delete/start/stop/cancel/run/codebook` and
|
|
60
|
+
`control_ui/features/codebook/tap/input/long_press/swipe/assert/wait/launch`.
|
|
61
|
+
- **Panel Skills card** — start/stop recording, name a target code, run or delete a Skill.
|
|
62
|
+
- Pure-Node PNG decoding for the image features, so there are still no runtime dependencies.
|
|
63
|
+
|
|
64
|
+
### Notes
|
|
65
|
+
|
|
66
|
+
- Skills use UI automation only; arbitrary in-app function invocation is intentionally not
|
|
67
|
+
supported.
|
|
68
|
+
- Runtime `snapshotId` numbers are not persisted in Skills. Codebook codes are the stable
|
|
69
|
+
interface.
|
|
70
|
+
|
|
3
71
|
## 0.8.1
|
|
4
72
|
|
|
5
73
|
Remote VM over SSH.
|
package/README.md
CHANGED
|
@@ -45,7 +45,7 @@ same-origin, loopback-only debug HTTP API and an `apk_vm` Agent tool.
|
|
|
45
45
|
> 当前 **Windows x64 与 Linux x86_64(headless)为端到端已验证平台**;macOS 仅做探测/CI
|
|
46
46
|
> 覆盖,详见 `docs/LINUX-E2E.md`。
|
|
47
47
|
|
|
48
|
-
**Status:** `0.
|
|
48
|
+
**Status:** `0.10.1` · Apache-2.0 · no npm runtime dependencies · no build step.
|
|
49
49
|
|
|
50
50
|
This is an open-source DSH plugin published on npm as **`dsh-apk-vm`**. Repository:
|
|
51
51
|
[github.com/yuxiaole-bili/dsh-apk-vm](https://github.com/yuxiaole-bili/dsh-apk-vm).
|
|
@@ -163,14 +163,18 @@ key {"text":"HOME"} -> returned to launcher
|
|
|
163
163
|
**bootstrap a completely empty host** (download Google command-line tools, extract, accept
|
|
164
164
|
licenses, install platform-tools/emulator/build-tools/platform).
|
|
165
165
|
- **Multi-agent isolation / multi-instance.** Several DSH Agents can drive separate VMs at the
|
|
166
|
-
same time:
|
|
167
|
-
|
|
166
|
+
same time: each conversation owns a renewable lease on the VM it uses, so a second
|
|
167
|
+
conversation never silently steals the first one's emulator. The first conversation gets the
|
|
168
|
+
default VM; a competing conversation gets an isolated instance with its own AVD, emulator
|
|
169
|
+
port, snapshot, debug state and **H.264/WebCodecs stream**
|
|
168
170
|
(`maxInstances`, default 3 extra; extra VMs use a 720×1280 preview display).
|
|
169
171
|
Extra instances expose `owner` / `lastUsedAt` / `idleMs`, and are **stopped automatically**
|
|
170
172
|
after `instanceIdleMs` without panel frames, stream clients or Agent commands (AVD kept for
|
|
171
173
|
quick reuse). The panel selector and **VM instances** card show owner/idle state and offer
|
|
172
174
|
Stop/Destroy. The `apk_vm` tool accepts
|
|
173
|
-
`instance`
|
|
175
|
+
`instance`, `ownerScope` (`shared` / `conversation` / `project`), `ownerLabel` and exposes
|
|
176
|
+
`list_instances` / `create_instance` / `destroy_instance` / `owner_status` / `claim_vm` /
|
|
177
|
+
`release_vm`.
|
|
174
178
|
- **Multi-AVD switching.** The Environment card lists the AVDs visible to the emulator and can
|
|
175
179
|
switch (`GET /apk-vm/avds`, `POST /apk-vm/avd`); the current AVD is stopped, switched and
|
|
176
180
|
restarted automatically.
|
|
@@ -200,6 +204,11 @@ key {"text":"HOME"} -> returned to launcher
|
|
|
200
204
|
and free-disk footprint and offers explicit cleanups: plugin temp files, quick-boot
|
|
201
205
|
snapshots for a stopped AVD, and orphan `dsh_apk_*` AVDs. Destructive actions need a second
|
|
202
206
|
click and never touch the active VM.
|
|
207
|
+
- **App-operation Skills + Control Codebook.** Record a navigation once, then replay it fast
|
|
208
|
+
to a target page. Skills use a persisted **Control Codebook** (stable numeric or semantic
|
|
209
|
+
codes, plus optional image features) instead of raw coordinates; each Skill embeds its own
|
|
210
|
+
codebook so it stays reproducible. The Agent tool gets `skill_*` and `control_*` actions;
|
|
211
|
+
the panel has a small Skills card for recording, running and deleting them.
|
|
203
212
|
- **Installed-app management.** The panel's **Installed apps** card lists third-party packages
|
|
204
213
|
for the active instance and offers force-stop, clear-data and uninstall; it uses the same
|
|
205
214
|
rooted `debug/packages` + `debug/action` API as the Agent tool.
|
|
@@ -263,6 +272,93 @@ key {"text":"HOME"} -> returned to launcher
|
|
|
263
272
|
- `POST /apk-vm/debug/ca` accepts a host `.pem`/`.crt` path and installs it under the guest's
|
|
264
273
|
user CA directory with the correct subject hash.
|
|
265
274
|
|
|
275
|
+
## App-operation Skills and Control Codebook
|
|
276
|
+
|
|
277
|
+
App-operation Skills are short, replayable navigation recipes for tests. Instead of recording
|
|
278
|
+
raw coordinates, the Agent works through a **Control Codebook**: a small table that gives stable
|
|
279
|
+
codes to controls it has identified. A Skill embeds a copy of that codebook, so it can be replayed
|
|
280
|
+
later even if the live UI tree changes its node order.
|
|
281
|
+
|
|
282
|
+
1. Start a recording with `skill_start` (optionally pass a codebook), then use
|
|
283
|
+
`control_ui` / `control_features` to inspect controls.
|
|
284
|
+
2. Save or refine codes with `control_codebook`, then act with `control_tap`,
|
|
285
|
+
`control_input`, `control_long_press`, `control_swipe`, `control_assert` or `control_wait`.
|
|
286
|
+
Each semantic action is appended to the recording.
|
|
287
|
+
3. Stop with `skill_stop`. The Skill stores its embedded codebook and steps under the workspace
|
|
288
|
+
`.apk-vm/skills/` directory (gitignored).
|
|
289
|
+
4. Replay with `skill_run`, optionally using `untilCode` to stop at a page and `speed` to scale
|
|
290
|
+
optional waits. Steps use `waitFor` polling, not only fixed sleeps.
|
|
291
|
+
|
|
292
|
+
A codebook entry looks like this:
|
|
293
|
+
|
|
294
|
+
```json
|
|
295
|
+
{
|
|
296
|
+
"codebook": {
|
|
297
|
+
"version": 1,
|
|
298
|
+
"entries": [
|
|
299
|
+
{ "code": "A01", "name": "USERNAME", "features": { "resourceId": "com.example:id/user" } },
|
|
300
|
+
{ "code": "LOGIN_BTN", "aliases": ["A02"], "features": { "text": "Sign in", "clickable": true } },
|
|
301
|
+
{ "code": "STAR", "features": { "image": { "name": "star-icon", "hash": "…", "maxDistance": 10 } } }
|
|
302
|
+
]
|
|
303
|
+
}
|
|
304
|
+
}
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
- Codes may be numeric (`A01`) or semantic (`LOGIN_BTN`); aliases are accepted too.
|
|
308
|
+
- A matcher combines structural features (`resourceId`, `text`, `textAny`, `contentDesc`,
|
|
309
|
+
`className`, `clickable`, `enabled`, bounds ratio). Multiple matches are an explicit error with
|
|
310
|
+
candidates, never a silent first match.
|
|
311
|
+
- `control_features` can capture optional image features (PNG dHash + average/dominant colour)
|
|
312
|
+
for a control; give the image a name and merge the suggested entry into the codebook.
|
|
313
|
+
Image matching is only used when the entry declares `features.image`.
|
|
314
|
+
- `skill_run` accepts `variables` for `{{placeholder}}` interpolation, `dryRun` to resolve
|
|
315
|
+
controls without touching the device, and `timeoutMs`. A failed step returns a screenshot path,
|
|
316
|
+
UI XML, candidates and the step index.
|
|
317
|
+
- The raw temporal flow recorder (`flow_*`) is still available for compatibility; Skills are the
|
|
318
|
+
semantic, code-based option.
|
|
319
|
+
|
|
320
|
+
HTTP routes: `GET|POST /apk-vm/skills` and `POST /apk-vm/controls` (with the same
|
|
321
|
+
`/apk-vm/instances/<id>/...` variants). `apk_vm` actions: `skill_list`, `skill_get`,
|
|
322
|
+
`skill_save`, `skill_delete`, `skill_start`, `skill_stop`, `skill_cancel`, `skill_run`,
|
|
323
|
+
`skill_codebook`, `control_ui`, `control_features`, `control_codebook`, `control_tap`, `control_input`,
|
|
324
|
+
`control_long_press`, `control_swipe`, `control_assert`, `control_wait`, `control_launch`.
|
|
325
|
+
|
|
326
|
+
This feature deliberately does **not** call arbitrary functions inside the target app. It drives
|
|
327
|
+
the app only through its UI, so it keeps working for third-party APKs and does not require app
|
|
328
|
+
instrumentation.
|
|
329
|
+
|
|
330
|
+
## VM ownership and dedicated conversation/project VMs
|
|
331
|
+
|
|
332
|
+
Every running VM has a renewable **owner lease**, so two DSH conversations cannot silently drive
|
|
333
|
+
the same emulator. A lease records the owner key, display label, scope (`shared`, `conversation`
|
|
334
|
+
or `project`), session id, workspace, and `since` / `lastSeenAt` timestamps; leases expire after
|
|
335
|
+
30 minutes (`leaseTtlMs`) and are renewed by `apk_vm` calls that route to that VM.
|
|
336
|
+
|
|
337
|
+
- The first conversation that needs a VM claims the default instance. A different conversation
|
|
338
|
+
gets a dedicated extra instance instead of taking the default one.
|
|
339
|
+
- `ownerScope=conversation` gives the current DSH session a VM that follows it across turns;
|
|
340
|
+
`ownerScope=project` does the same for the current workspace. `ownerScope=shared` keeps the
|
|
341
|
+
first-come default behaviour.
|
|
342
|
+
- If an `instance` is explicitly requested and owned by another conversation, the call fails with
|
|
343
|
+
`vm-busy` and the owner details. `ownerForce=true` is an explicit takeover and should be used
|
|
344
|
+
only after checking `owner_status`.
|
|
345
|
+
- The VM instances card shows the current owner/scope. The **VM usage** card in the panel
|
|
346
|
+
shows a readable owner label and provides **Claim**, **Release** and a `shared` /
|
|
347
|
+
`conversation` / `project` scope control for the human user.
|
|
348
|
+
- `owner_status` lists default/extra VM leases; `claim_vm` and `release_vm` manage them.
|
|
349
|
+
HTTP: `GET|POST /apk-vm/ownership` (`status`, `claim`, `release`, `touch`), while
|
|
350
|
+
`GET /apk-vm/state` and `GET /apk-vm/instances` include owner metadata.
|
|
351
|
+
- When the panel makes a mutating HTTP request it sends
|
|
352
|
+
`x-apk-vm-owner`, `x-apk-vm-scope`, `x-apk-vm-session`, `x-apk-vm-workspace` and
|
|
353
|
+
`x-apk-vm-label`. Header values may be percent-encoded (UTF-8); the Host decodes them, so
|
|
354
|
+
non-ASCII labels and workspace paths remain valid for browser `fetch`. A request that
|
|
355
|
+
supplies an owner header auto-claims a free VM; on a leased VM
|
|
356
|
+
it must match the current owner key. `POST /apk-vm/*` control routes return `409 vm-busy` with
|
|
357
|
+
the current owner when the header is missing or does not match. Read-only `GET` routes are
|
|
358
|
+
never blocked.
|
|
359
|
+
- Closing a DSH conversation (the `session/disposed` event) automatically releases its leases,
|
|
360
|
+
with the 30-minute TTL as a fallback if that event is unavailable.
|
|
361
|
+
|
|
266
362
|
## Storage management
|
|
267
363
|
|
|
268
364
|
`GET /apk-vm/storage` reports the logical size of every AVD, its quick-boot snapshots, the
|
|
@@ -300,7 +396,9 @@ port forwards and pcap status. `POST /apk-vm/network` accepts `action`:
|
|
|
300
396
|
{ "action": "pcap", "path": "D:\\apk-vm-capture.pcap" }
|
|
301
397
|
```
|
|
302
398
|
|
|
303
|
-
- Port forwards use `adb forward tcp:<hostPort> tcp:<guestPort>`.
|
|
399
|
+
- Port forwards use `adb forward tcp:<hostPort> tcp:<guestPort>`. The panel offers one-click
|
|
400
|
+
presets for **Metro 8081**, **HTTP 8080**, **Frida 27042** and **DevTools 9222**, plus the
|
|
401
|
+
manual host/guest fields.
|
|
304
402
|
- DNS is applied by the emulator `-dns-server` flag on the next start.
|
|
305
403
|
- Hosts entries are bind-mounted over the guest `/etc/hosts` (root required) and reapplied
|
|
306
404
|
after a plugin-managed boot.
|
|
@@ -433,6 +531,7 @@ config:
|
|
|
433
531
|
maxInstances: 3 # extra VMs (default VM is always available)
|
|
434
532
|
instanceIdleMs: 900000 # stop idle extra VMs after 15 min (AVD kept)
|
|
435
533
|
instanceReapIntervalMs: 60000
|
|
534
|
+
leaseTtlMs: 1800000 # VM owner lease lifetime; renewed by apk_vm calls
|
|
436
535
|
allowRemote: false # reject non-loopback VM API clients
|
|
437
536
|
allowSetupDownloads: false # gate the optional system-image download action
|
|
438
537
|
scrcpy: true
|
|
@@ -495,6 +594,7 @@ All keys are optional; defaults are shown in the **Default** column.
|
|
|
495
594
|
| `maxInstances` | `3` | Maximum number of extra instances (in addition to the default VM). Each running instance costs guest RAM/CPU. |
|
|
496
595
|
| `instanceIdleMs` | `900000` | Stop an extra instance after this long without panel frames, a video client, or an Agent command (AVD is kept). |
|
|
497
596
|
| `instanceReapIntervalMs` | `60000` | How often the idle reaper checks extra instances. |
|
|
597
|
+
| `leaseTtlMs` | `1800000` | Lifetime of a VM owner lease. Every `apk_vm` call that routes to the VM renews it; set `0` to disable expiry. |
|
|
498
598
|
| `allowRemote` | `false` | Allow non-loopback clients to reach `/apk-vm/*` and `/apk-vm/ws` (only behind your own authenticated proxy). |
|
|
499
599
|
| `allowSetupDownloads` | `false` | Allow the Environment card's `install-image` action to run `sdkmanager --install` (large download). |
|
|
500
600
|
| `frameFps` | `2` | PNG fallback target frame rate, 1–10 (the H.264 path is not affected). |
|
|
@@ -614,7 +714,8 @@ JSON bodies are bounded to 64 KiB; the raw APK upload is bounded to 500 MiB.
|
|
|
614
714
|
| POST | `/apk-vm/stream` | `{"fps":1–10}` — set the PNG fallback target frame rate. |
|
|
615
715
|
| POST | `/apk-vm/stream/quality` | `{"preset":"low\|balanced\|high"}` — change H.264 resolution/bitrate/fps; the scrcpy session reconnects. |
|
|
616
716
|
| GET (upgrade) | `/apk-vm/ws` | WebSocket H.264 stream (see above). |
|
|
617
|
-
| GET | `/apk-vm/
|
|
717
|
+
| GET/POST | `/apk-vm/ownership` | `GET` lists VM leases; `POST {"action":"claim\|release\|touch","instanceId":"default","ownerScope":"shared\|conversation\|project","sessionId"?,"workspace"?,"force"?}` manages the owner lease. |
|
|
718
|
+
| GET | `/apk-vm/instances` | List default + extra VM instances, including `owner`, `ownerInfo` (`key`, `label`, `scope`, `sessionId`, `workspace`, `since`, `lastSeenAt`), `lastUsedAt`, `idleMs`, `idleTimeoutMs`, `videoClients`. |
|
|
618
719
|
| POST | `/apk-vm/instances` | `{"name":"agent-a","start":true}` — create an isolated AVD/VM; every extra instance has its own H.264 stream. |
|
|
619
720
|
| DELETE | `/apk-vm/instances/:id` | Stop/dispose an extra instance (the default cannot be destroyed). |
|
|
620
721
|
| GET/POST | `/apk-vm/instances/:id/...` | Per-instance mirror of `state`, `frame`, `vm`, `input`, `install`, `upload`, `debug/*`. |
|
|
@@ -678,8 +779,11 @@ plus `start` / `stop` and instance management (`list_instances`, `create_instanc
|
|
|
678
779
|
duplicated logic).
|
|
679
780
|
|
|
680
781
|
For concurrent Agents, pass `instance` to target a specific VM; when omitted, the tool resolves
|
|
681
|
-
the calling Agent's assigned instance (first
|
|
682
|
-
instances created on demand until `maxInstances`).
|
|
782
|
+
the calling Agent's lease and assigned instance (first conversation → default VM, later
|
|
783
|
+
conversations → isolated instances created on demand until `maxInstances`). Set
|
|
784
|
+
`ownerScope=conversation` or `ownerScope=project` to request a dedicated VM for the current DSH
|
|
785
|
+
session/workspace. `owner_status` lists leases, `claim_vm` / `release_vm` manage them, and
|
|
786
|
+
`ownerForce=true` is required for an explicit takeover. Each extra instance has its own H.264
|
|
683
787
|
WebSocket (`/apk-vm/instances/<id>/ws`) and falls back to PNG if WebCodecs is unavailable.
|
|
684
788
|
|
|
685
789
|
UI automation is live-verified against a multi-instance VM: `find_ui {"text":"Phone"}` returned
|