dsh-apk-vm 0.8.0 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,80 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.10.0
4
+
5
+ VM ownership and dedicated conversation/project VMs.
6
+
7
+ ### Added
8
+
9
+ - **Renewable VM owner leases** — each conversation owns the VM it drives; a second conversation
10
+ gets its own instance instead of silently taking over the default VM.
11
+ - **Ownership scopes** — `shared` (first come), `conversation` (dedicated per DSH session), and
12
+ `project` (dedicated per workspace). Leases expire after 30 minutes and renew on `apk_vm`
13
+ calls that route to the VM.
14
+ - **Conflict protection** — an explicit foreign `instance` returns `vm-busy` with the current
15
+ owner; `ownerForce=true` is an explicit takeover.
16
+ - **Agent actions** — `owner_status`, `claim_vm`, `release_vm`, plus `ownerScope` / `ownerLabel`
17
+ on normal VM actions.
18
+ - **HTTP API** — `GET|POST /apk-vm/ownership` (`status`, `claim`, `release`, `touch`); owner
19
+ metadata in `GET /apk-vm/state` and `/apk-vm/instances`.
20
+ - **Panel display** — the VM instances card shows the current owner and ownership scope.
21
+ - **Panel usage card** — human-readable owner label plus Claim / Release buttons and a
22
+ `shared` / `conversation` / `project` scope control; mutating panel HTTP requests carry
23
+ the matching owner headers.
24
+ - **HTTP owner validation** — state-changing `/apk-vm/*` requests return `409 vm-busy` with
25
+ the current owner when their `x-apk-vm-owner` header does not match; GET routes stay open.
26
+ The panel also sends `x-apk-vm-scope` / `x-apk-vm-session` / `x-apk-vm-workspace` /
27
+ `x-apk-vm-label`, and a matching owner header auto-claims a free VM.
28
+ - **Automatic release** — the `session/disposed` Host event releases every lease held by a
29
+ closed conversation, with the lease TTL as fallback.
30
+ - **Readable labels** — `ownerLabelFor()` now shows `对话 …`, `项目 <folder>` or `共享 …`
31
+ instead of a raw key.
32
+ - **Port presets** — one-click Metro 8081, HTTP 8080, Frida 27042 and DevTools 9222 forwards.
33
+
34
+ ## 0.9.0
35
+
36
+ App-operation Skills and the Control Codebook.
37
+
38
+ ### Added
39
+
40
+ - **App-operation Skills** — record a semantic navigation once, then replay it fast to a target
41
+ page for tests. Skills embed their own Control Codebook, so they keep working when UIAutomator
42
+ node order changes. Stored under the workspace `.apk-vm/skills/` directory.
43
+ - **Control Codebook** — numeric (`A01`) and semantic (`LOGIN_BTN`) codes plus aliases, matched
44
+ by structural features (`resourceId`, `text`, `textAny`, `contentDesc`, `className`,
45
+ `clickable`, `enabled`, bounds ratio) with deterministic ambiguity errors.
46
+ - **Image features** — optional PNG dHash plus average/dominant colour for icon-only controls;
47
+ `control_features` returns a ready-to-save entry with a human image name.
48
+ - **Agent actions** — `skill_list/get/save/delete/start/stop/cancel/run/codebook` and
49
+ `control_ui/features/codebook/tap/input/long_press/swipe/assert/wait/launch`.
50
+ - **Panel Skills card** — start/stop recording, name a target code, run or delete a Skill.
51
+ - Pure-Node PNG decoding for the image features, so there are still no runtime dependencies.
52
+
53
+ ### Notes
54
+
55
+ - Skills use UI automation only; arbitrary in-app function invocation is intentionally not
56
+ supported.
57
+ - Runtime `snapshotId` numbers are not persisted in Skills. Codebook codes are the stable
58
+ interface.
59
+
60
+ ## 0.8.1
61
+
62
+ Remote VM over SSH.
63
+
64
+ ### Added
65
+
66
+ - **SSH adb tunnel** for a remote Android VM: forward a local loopback port to the remote
67
+ emulator's adb port with `ssh -N -L`, then attach the normal adb device. The key path is
68
+ supplied per connection and never stored.
69
+ - Remote SSH card with host/user/key/remote-port fields, connect/disconnect state, and automatic
70
+ tunnel cleanup when the plugin unloads.
71
+
72
+ ### Notes
73
+
74
+ - SSH forwarding requires an SSH server and an adb port reachable from that server
75
+ (for example an emulator started with its console port forwarded).
76
+ - The remote host/user/key are not persisted.
77
+
3
78
  ## 0.8.0
4
79
 
5
80
  Sidebar workspace, APK debugging, network and regression tools.
package/README.md CHANGED
@@ -45,7 +45,7 @@ same-origin, loopback-only debug HTTP API and an `apk_vm` Agent tool.
45
45
  > 当前 **Windows x64 与 Linux x86_64(headless)为端到端已验证平台**;macOS 仅做探测/CI
46
46
  > 覆盖,详见 `docs/LINUX-E2E.md`。
47
47
 
48
- **Status:** `0.8.0` · Apache-2.0 · no npm runtime dependencies · no build step.
48
+ **Status:** `0.10.0` · Apache-2.0 · no npm runtime dependencies · no build step.
49
49
 
50
50
  This is an open-source DSH plugin published on npm as **`dsh-apk-vm`**. Repository:
51
51
  [github.com/yuxiaole-bili/dsh-apk-vm](https://github.com/yuxiaole-bili/dsh-apk-vm).
@@ -163,14 +163,18 @@ key {"text":"HOME"} -> returned to launcher
163
163
  **bootstrap a completely empty host** (download Google command-line tools, extract, accept
164
164
  licenses, install platform-tools/emulator/build-tools/platform).
165
165
  - **Multi-agent isolation / multi-instance.** Several DSH Agents can drive separate VMs at the
166
- same time: the first Agent gets the default VM, later Agents are assigned isolated instances
167
- with their own AVD, emulator port, snapshot, debug state and **H.264/WebCodecs stream**
166
+ same time: each conversation owns a renewable lease on the VM it uses, so a second
167
+ conversation never silently steals the first one's emulator. The first conversation gets the
168
+ default VM; a competing conversation gets an isolated instance with its own AVD, emulator
169
+ port, snapshot, debug state and **H.264/WebCodecs stream**
168
170
  (`maxInstances`, default 3 extra; extra VMs use a 720×1280 preview display).
169
171
  Extra instances expose `owner` / `lastUsedAt` / `idleMs`, and are **stopped automatically**
170
172
  after `instanceIdleMs` without panel frames, stream clients or Agent commands (AVD kept for
171
173
  quick reuse). The panel selector and **VM instances** card show owner/idle state and offer
172
174
  Stop/Destroy. The `apk_vm` tool accepts
173
- `instance` and exposes `list_instances` / `create_instance` / `destroy_instance`.
175
+ `instance`, `ownerScope` (`shared` / `conversation` / `project`), `ownerLabel` and exposes
176
+ `list_instances` / `create_instance` / `destroy_instance` / `owner_status` / `claim_vm` /
177
+ `release_vm`.
174
178
  - **Multi-AVD switching.** The Environment card lists the AVDs visible to the emulator and can
175
179
  switch (`GET /apk-vm/avds`, `POST /apk-vm/avd`); the current AVD is stopped, switched and
176
180
  restarted automatically.
@@ -200,6 +204,11 @@ key {"text":"HOME"} -> returned to launcher
200
204
  and free-disk footprint and offers explicit cleanups: plugin temp files, quick-boot
201
205
  snapshots for a stopped AVD, and orphan `dsh_apk_*` AVDs. Destructive actions need a second
202
206
  click and never touch the active VM.
207
+ - **App-operation Skills + Control Codebook.** Record a navigation once, then replay it fast
208
+ to a target page. Skills use a persisted **Control Codebook** (stable numeric or semantic
209
+ codes, plus optional image features) instead of raw coordinates; each Skill embeds its own
210
+ codebook so it stays reproducible. The Agent tool gets `skill_*` and `control_*` actions;
211
+ the panel has a small Skills card for recording, running and deleting them.
203
212
  - **Installed-app management.** The panel's **Installed apps** card lists third-party packages
204
213
  for the active instance and offers force-stop, clear-data and uninstall; it uses the same
205
214
  rooted `debug/packages` + `debug/action` API as the Agent tool.
@@ -263,6 +272,91 @@ key {"text":"HOME"} -> returned to launcher
263
272
  - `POST /apk-vm/debug/ca` accepts a host `.pem`/`.crt` path and installs it under the guest's
264
273
  user CA directory with the correct subject hash.
265
274
 
275
+ ## App-operation Skills and Control Codebook
276
+
277
+ App-operation Skills are short, replayable navigation recipes for tests. Instead of recording
278
+ raw coordinates, the Agent works through a **Control Codebook**: a small table that gives stable
279
+ codes to controls it has identified. A Skill embeds a copy of that codebook, so it can be replayed
280
+ later even if the live UI tree changes its node order.
281
+
282
+ 1. Start a recording with `skill_start` (optionally pass a codebook), then use
283
+ `control_ui` / `control_features` to inspect controls.
284
+ 2. Save or refine codes with `control_codebook`, then act with `control_tap`,
285
+ `control_input`, `control_long_press`, `control_swipe`, `control_assert` or `control_wait`.
286
+ Each semantic action is appended to the recording.
287
+ 3. Stop with `skill_stop`. The Skill stores its embedded codebook and steps under the workspace
288
+ `.apk-vm/skills/` directory (gitignored).
289
+ 4. Replay with `skill_run`, optionally using `untilCode` to stop at a page and `speed` to scale
290
+ optional waits. Steps use `waitFor` polling, not only fixed sleeps.
291
+
292
+ A codebook entry looks like this:
293
+
294
+ ```json
295
+ {
296
+ "codebook": {
297
+ "version": 1,
298
+ "entries": [
299
+ { "code": "A01", "name": "USERNAME", "features": { "resourceId": "com.example:id/user" } },
300
+ { "code": "LOGIN_BTN", "aliases": ["A02"], "features": { "text": "Sign in", "clickable": true } },
301
+ { "code": "STAR", "features": { "image": { "name": "star-icon", "hash": "…", "maxDistance": 10 } } }
302
+ ]
303
+ }
304
+ }
305
+ ```
306
+
307
+ - Codes may be numeric (`A01`) or semantic (`LOGIN_BTN`); aliases are accepted too.
308
+ - A matcher combines structural features (`resourceId`, `text`, `textAny`, `contentDesc`,
309
+ `className`, `clickable`, `enabled`, bounds ratio). Multiple matches are an explicit error with
310
+ candidates, never a silent first match.
311
+ - `control_features` can capture optional image features (PNG dHash + average/dominant colour)
312
+ for a control; give the image a name and merge the suggested entry into the codebook.
313
+ Image matching is only used when the entry declares `features.image`.
314
+ - `skill_run` accepts `variables` for `{{placeholder}}` interpolation, `dryRun` to resolve
315
+ controls without touching the device, and `timeoutMs`. A failed step returns a screenshot path,
316
+ UI XML, candidates and the step index.
317
+ - The raw temporal flow recorder (`flow_*`) is still available for compatibility; Skills are the
318
+ semantic, code-based option.
319
+
320
+ HTTP routes: `GET|POST /apk-vm/skills` and `POST /apk-vm/controls` (with the same
321
+ `/apk-vm/instances/<id>/...` variants). `apk_vm` actions: `skill_list`, `skill_get`,
322
+ `skill_save`, `skill_delete`, `skill_start`, `skill_stop`, `skill_cancel`, `skill_run`,
323
+ `skill_codebook`, `control_ui`, `control_features`, `control_codebook`, `control_tap`, `control_input`,
324
+ `control_long_press`, `control_swipe`, `control_assert`, `control_wait`, `control_launch`.
325
+
326
+ This feature deliberately does **not** call arbitrary functions inside the target app. It drives
327
+ the app only through its UI, so it keeps working for third-party APKs and does not require app
328
+ instrumentation.
329
+
330
+ ## VM ownership and dedicated conversation/project VMs
331
+
332
+ Every running VM has a renewable **owner lease**, so two DSH conversations cannot silently drive
333
+ the same emulator. A lease records the owner key, display label, scope (`shared`, `conversation`
334
+ or `project`), session id, workspace, and `since` / `lastSeenAt` timestamps; leases expire after
335
+ 30 minutes (`leaseTtlMs`) and are renewed by `apk_vm` calls that route to that VM.
336
+
337
+ - The first conversation that needs a VM claims the default instance. A different conversation
338
+ gets a dedicated extra instance instead of taking the default one.
339
+ - `ownerScope=conversation` gives the current DSH session a VM that follows it across turns;
340
+ `ownerScope=project` does the same for the current workspace. `ownerScope=shared` keeps the
341
+ first-come default behaviour.
342
+ - If an `instance` is explicitly requested and owned by another conversation, the call fails with
343
+ `vm-busy` and the owner details. `ownerForce=true` is an explicit takeover and should be used
344
+ only after checking `owner_status`.
345
+ - The VM instances card shows the current owner/scope. The **VM usage** card in the panel
346
+ shows a readable owner label and provides **Claim**, **Release** and a `shared` /
347
+ `conversation` / `project` scope control for the human user.
348
+ - `owner_status` lists default/extra VM leases; `claim_vm` and `release_vm` manage them.
349
+ HTTP: `GET|POST /apk-vm/ownership` (`status`, `claim`, `release`, `touch`), while
350
+ `GET /apk-vm/state` and `GET /apk-vm/instances` include owner metadata.
351
+ - When the panel makes a mutating HTTP request it sends
352
+ `x-apk-vm-owner`, `x-apk-vm-scope`, `x-apk-vm-session`, `x-apk-vm-workspace` and
353
+ `x-apk-vm-label`. A request that supplies an owner header auto-claims a free VM; on a leased VM
354
+ it must match the current owner key. `POST /apk-vm/*` control routes return `409 vm-busy` with
355
+ the current owner when the header is missing or does not match. Read-only `GET` routes are
356
+ never blocked.
357
+ - Closing a DSH conversation (the `session/disposed` event) automatically releases its leases,
358
+ with the 30-minute TTL as a fallback if that event is unavailable.
359
+
266
360
  ## Storage management
267
361
 
268
362
  `GET /apk-vm/storage` reports the logical size of every AVD, its quick-boot snapshots, the
@@ -300,7 +394,9 @@ port forwards and pcap status. `POST /apk-vm/network` accepts `action`:
300
394
  { "action": "pcap", "path": "D:\\apk-vm-capture.pcap" }
301
395
  ```
302
396
 
303
- - Port forwards use `adb forward tcp:<hostPort> tcp:<guestPort>`.
397
+ - Port forwards use `adb forward tcp:<hostPort> tcp:<guestPort>`. The panel offers one-click
398
+ presets for **Metro 8081**, **HTTP 8080**, **Frida 27042** and **DevTools 9222**, plus the
399
+ manual host/guest fields.
304
400
  - DNS is applied by the emulator `-dns-server` flag on the next start.
305
401
  - Hosts entries are bind-mounted over the guest `/etc/hosts` (root required) and reapplied
306
402
  after a plugin-managed boot.
@@ -433,6 +529,7 @@ config:
433
529
  maxInstances: 3 # extra VMs (default VM is always available)
434
530
  instanceIdleMs: 900000 # stop idle extra VMs after 15 min (AVD kept)
435
531
  instanceReapIntervalMs: 60000
532
+ leaseTtlMs: 1800000 # VM owner lease lifetime; renewed by apk_vm calls
436
533
  allowRemote: false # reject non-loopback VM API clients
437
534
  allowSetupDownloads: false # gate the optional system-image download action
438
535
  scrcpy: true
@@ -495,6 +592,7 @@ All keys are optional; defaults are shown in the **Default** column.
495
592
  | `maxInstances` | `3` | Maximum number of extra instances (in addition to the default VM). Each running instance costs guest RAM/CPU. |
496
593
  | `instanceIdleMs` | `900000` | Stop an extra instance after this long without panel frames, a video client, or an Agent command (AVD is kept). |
497
594
  | `instanceReapIntervalMs` | `60000` | How often the idle reaper checks extra instances. |
595
+ | `leaseTtlMs` | `1800000` | Lifetime of a VM owner lease. Every `apk_vm` call that routes to the VM renews it; set `0` to disable expiry. |
498
596
  | `allowRemote` | `false` | Allow non-loopback clients to reach `/apk-vm/*` and `/apk-vm/ws` (only behind your own authenticated proxy). |
499
597
  | `allowSetupDownloads` | `false` | Allow the Environment card's `install-image` action to run `sdkmanager --install` (large download). |
500
598
  | `frameFps` | `2` | PNG fallback target frame rate, 1–10 (the H.264 path is not affected). |
@@ -600,6 +698,7 @@ JSON bodies are bounded to 64 KiB; the raw APK upload is bounded to 500 MiB.
600
698
  | GET/HEAD | `/apk-vm/apk-icon?path=…` | Extract the raster launcher icon from the APK ZIP (PNG/WebP/JPEG). |
601
699
  | GET | `/apk-vm/devices` | List adb devices (`serial`, `state`, `kind`, `model`, `product`, transport) plus current/managed-device metadata. |
602
700
  | POST | `/apk-vm/device` | `{"serial":"<adb-serial>"}` attach an online physical/network device; `{"serial":""}` detach and restore the managed AVD. |
701
+ | GET/POST | `/apk-vm/remote` | SSH adb tunnel: `POST {"action":"connect","host","user","keyPath"?,"remotePort":5558}` or `{"action":"disconnect"}`; `GET` returns tunnel status. |
603
702
  | POST | `/apk-vm/vm` | `{"action":"start"\|"stop"\|"restart"\|"snapshot"}`. |
604
703
  | POST | `/apk-vm/install` | `{"path":"C:/path/app.apk"}` → install + launch. |
605
704
  | POST | `/apk-vm/upload` | Raw APK/APKS/XAPK/AAB bytes (≤ 500 MiB) with the file name in the `x-apk-name` header → validate ZIP/APK magic, write to the OS temp dir, install + launch. |
@@ -613,7 +712,8 @@ JSON bodies are bounded to 64 KiB; the raw APK upload is bounded to 500 MiB.
613
712
  | POST | `/apk-vm/stream` | `{"fps":1–10}` — set the PNG fallback target frame rate. |
614
713
  | POST | `/apk-vm/stream/quality` | `{"preset":"low\|balanced\|high"}` — change H.264 resolution/bitrate/fps; the scrcpy session reconnects. |
615
714
  | GET (upgrade) | `/apk-vm/ws` | WebSocket H.264 stream (see above). |
616
- | GET | `/apk-vm/instances` | List default + extra VM instances, including `owner`, `lastUsedAt`, `idleMs`, `idleTimeoutMs`, `videoClients`. |
715
+ | GET/POST | `/apk-vm/ownership` | `GET` lists VM leases; `POST {"action":"claim\|release\|touch","instanceId":"default","ownerScope":"shared\|conversation\|project","sessionId"?,"workspace"?,"force"?}` manages the owner lease. |
716
+ | GET | `/apk-vm/instances` | List default + extra VM instances, including `owner`, `ownerInfo` (`key`, `label`, `scope`, `sessionId`, `workspace`, `since`, `lastSeenAt`), `lastUsedAt`, `idleMs`, `idleTimeoutMs`, `videoClients`. |
617
717
  | POST | `/apk-vm/instances` | `{"name":"agent-a","start":true}` — create an isolated AVD/VM; every extra instance has its own H.264 stream. |
618
718
  | DELETE | `/apk-vm/instances/:id` | Stop/dispose an extra instance (the default cannot be destroyed). |
619
719
  | GET/POST | `/apk-vm/instances/:id/...` | Per-instance mirror of `state`, `frame`, `vm`, `input`, `install`, `upload`, `debug/*`. |
@@ -677,8 +777,11 @@ plus `start` / `stop` and instance management (`list_instances`, `create_instanc
677
777
  duplicated logic).
678
778
 
679
779
  For concurrent Agents, pass `instance` to target a specific VM; when omitted, the tool resolves
680
- the calling Agent's assigned instance (first Agent → default VM, later Agents → isolated
681
- instances created on demand until `maxInstances`). Each extra instance has its own H.264
780
+ the calling Agent's lease and assigned instance (first conversation → default VM, later
781
+ conversations → isolated instances created on demand until `maxInstances`). Set
782
+ `ownerScope=conversation` or `ownerScope=project` to request a dedicated VM for the current DSH
783
+ session/workspace. `owner_status` lists leases, `claim_vm` / `release_vm` manage them, and
784
+ `ownerForce=true` is required for an explicit takeover. Each extra instance has its own H.264
682
785
  WebSocket (`/apk-vm/instances/<id>/ws`) and falls back to PNG if WebCodecs is unavailable.
683
786
 
684
787
  UI automation is live-verified against a multi-instance VM: `find_ui {"text":"Phone"}` returned