dsh-apk-vm 0.8.1 → 0.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,73 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.10.1
4
+
5
+ Fix panel requests failing with non-ISO-8859-1 header values.
6
+
7
+ ### Fixed
8
+
9
+ - Percent-encode the panel owner label before sending it as `x-apk-vm-label`, so Chromium
10
+ `fetch` no longer rejects requests with `String contains non ISO-8859-1 code point`.
11
+ - Decode percent-encoded owner HTTP headers/query values on the Host, including Chinese
12
+ workspace paths and labels.
13
+
14
+ ## 0.10.0
15
+
16
+ VM ownership and dedicated conversation/project VMs.
17
+
18
+ ### Added
19
+
20
+ - **Renewable VM owner leases** — each conversation owns the VM it drives; a second conversation
21
+ gets its own instance instead of silently taking over the default VM.
22
+ - **Ownership scopes** — `shared` (first come), `conversation` (dedicated per DSH session), and
23
+ `project` (dedicated per workspace). Leases expire after 30 minutes and renew on `apk_vm`
24
+ calls that route to the VM.
25
+ - **Conflict protection** — an explicit foreign `instance` returns `vm-busy` with the current
26
+ owner; `ownerForce=true` is an explicit takeover.
27
+ - **Agent actions** — `owner_status`, `claim_vm`, `release_vm`, plus `ownerScope` / `ownerLabel`
28
+ on normal VM actions.
29
+ - **HTTP API** — `GET|POST /apk-vm/ownership` (`status`, `claim`, `release`, `touch`); owner
30
+ metadata in `GET /apk-vm/state` and `/apk-vm/instances`.
31
+ - **Panel display** — the VM instances card shows the current owner and ownership scope.
32
+ - **Panel usage card** — human-readable owner label plus Claim / Release buttons and a
33
+ `shared` / `conversation` / `project` scope control; mutating panel HTTP requests carry
34
+ the matching owner headers.
35
+ - **HTTP owner validation** — state-changing `/apk-vm/*` requests return `409 vm-busy` with
36
+ the current owner when their `x-apk-vm-owner` header does not match; GET routes stay open.
37
+ The panel also sends `x-apk-vm-scope` / `x-apk-vm-session` / `x-apk-vm-workspace` /
38
+ `x-apk-vm-label`, and a matching owner header auto-claims a free VM.
39
+ - **Automatic release** — the `session/disposed` Host event releases every lease held by a
40
+ closed conversation, with the lease TTL as fallback.
41
+ - **Readable labels** — `ownerLabelFor()` now shows `对话 …`, `项目 <folder>` or `共享 …`
42
+ instead of a raw key.
43
+ - **Port presets** — one-click Metro 8081, HTTP 8080, Frida 27042 and DevTools 9222 forwards.
44
+
45
+ ## 0.9.0
46
+
47
+ App-operation Skills and the Control Codebook.
48
+
49
+ ### Added
50
+
51
+ - **App-operation Skills** — record a semantic navigation once, then replay it fast to a target
52
+ page for tests. Skills embed their own Control Codebook, so they keep working when UIAutomator
53
+ node order changes. Stored under the workspace `.apk-vm/skills/` directory.
54
+ - **Control Codebook** — numeric (`A01`) and semantic (`LOGIN_BTN`) codes plus aliases, matched
55
+ by structural features (`resourceId`, `text`, `textAny`, `contentDesc`, `className`,
56
+ `clickable`, `enabled`, bounds ratio) with deterministic ambiguity errors.
57
+ - **Image features** — optional PNG dHash plus average/dominant colour for icon-only controls;
58
+ `control_features` returns a ready-to-save entry with a human image name.
59
+ - **Agent actions** — `skill_list/get/save/delete/start/stop/cancel/run/codebook` and
60
+ `control_ui/features/codebook/tap/input/long_press/swipe/assert/wait/launch`.
61
+ - **Panel Skills card** — start/stop recording, name a target code, run or delete a Skill.
62
+ - Pure-Node PNG decoding for the image features, so there are still no runtime dependencies.
63
+
64
+ ### Notes
65
+
66
+ - Skills use UI automation only; arbitrary in-app function invocation is intentionally not
67
+ supported.
68
+ - Runtime `snapshotId` numbers are not persisted in Skills. Codebook codes are the stable
69
+ interface.
70
+
3
71
  ## 0.8.1
4
72
 
5
73
  Remote VM over SSH.
package/README.md CHANGED
@@ -45,7 +45,7 @@ same-origin, loopback-only debug HTTP API and an `apk_vm` Agent tool.
45
45
  > 当前 **Windows x64 与 Linux x86_64(headless)为端到端已验证平台**;macOS 仅做探测/CI
46
46
  > 覆盖,详见 `docs/LINUX-E2E.md`。
47
47
 
48
- **Status:** `0.8.1` · Apache-2.0 · no npm runtime dependencies · no build step.
48
+ **Status:** `0.10.1` · Apache-2.0 · no npm runtime dependencies · no build step.
49
49
 
50
50
  This is an open-source DSH plugin published on npm as **`dsh-apk-vm`**. Repository:
51
51
  [github.com/yuxiaole-bili/dsh-apk-vm](https://github.com/yuxiaole-bili/dsh-apk-vm).
@@ -163,14 +163,18 @@ key {"text":"HOME"} -> returned to launcher
163
163
  **bootstrap a completely empty host** (download Google command-line tools, extract, accept
164
164
  licenses, install platform-tools/emulator/build-tools/platform).
165
165
  - **Multi-agent isolation / multi-instance.** Several DSH Agents can drive separate VMs at the
166
- same time: the first Agent gets the default VM, later Agents are assigned isolated instances
167
- with their own AVD, emulator port, snapshot, debug state and **H.264/WebCodecs stream**
166
+ same time: each conversation owns a renewable lease on the VM it uses, so a second
167
+ conversation never silently steals the first one's emulator. The first conversation gets the
168
+ default VM; a competing conversation gets an isolated instance with its own AVD, emulator
169
+ port, snapshot, debug state and **H.264/WebCodecs stream**
168
170
  (`maxInstances`, default 3 extra; extra VMs use a 720×1280 preview display).
169
171
  Extra instances expose `owner` / `lastUsedAt` / `idleMs`, and are **stopped automatically**
170
172
  after `instanceIdleMs` without panel frames, stream clients or Agent commands (AVD kept for
171
173
  quick reuse). The panel selector and **VM instances** card show owner/idle state and offer
172
174
  Stop/Destroy. The `apk_vm` tool accepts
173
- `instance` and exposes `list_instances` / `create_instance` / `destroy_instance`.
175
+ `instance`, `ownerScope` (`shared` / `conversation` / `project`), `ownerLabel` and exposes
176
+ `list_instances` / `create_instance` / `destroy_instance` / `owner_status` / `claim_vm` /
177
+ `release_vm`.
174
178
  - **Multi-AVD switching.** The Environment card lists the AVDs visible to the emulator and can
175
179
  switch (`GET /apk-vm/avds`, `POST /apk-vm/avd`); the current AVD is stopped, switched and
176
180
  restarted automatically.
@@ -200,6 +204,11 @@ key {"text":"HOME"} -> returned to launcher
200
204
  and free-disk footprint and offers explicit cleanups: plugin temp files, quick-boot
201
205
  snapshots for a stopped AVD, and orphan `dsh_apk_*` AVDs. Destructive actions need a second
202
206
  click and never touch the active VM.
207
+ - **App-operation Skills + Control Codebook.** Record a navigation once, then replay it fast
208
+ to a target page. Skills use a persisted **Control Codebook** (stable numeric or semantic
209
+ codes, plus optional image features) instead of raw coordinates; each Skill embeds its own
210
+ codebook so it stays reproducible. The Agent tool gets `skill_*` and `control_*` actions;
211
+ the panel has a small Skills card for recording, running and deleting them.
203
212
  - **Installed-app management.** The panel's **Installed apps** card lists third-party packages
204
213
  for the active instance and offers force-stop, clear-data and uninstall; it uses the same
205
214
  rooted `debug/packages` + `debug/action` API as the Agent tool.
@@ -263,6 +272,93 @@ key {"text":"HOME"} -> returned to launcher
263
272
  - `POST /apk-vm/debug/ca` accepts a host `.pem`/`.crt` path and installs it under the guest's
264
273
  user CA directory with the correct subject hash.
265
274
 
275
+ ## App-operation Skills and Control Codebook
276
+
277
+ App-operation Skills are short, replayable navigation recipes for tests. Instead of recording
278
+ raw coordinates, the Agent works through a **Control Codebook**: a small table that gives stable
279
+ codes to controls it has identified. A Skill embeds a copy of that codebook, so it can be replayed
280
+ later even if the live UI tree changes its node order.
281
+
282
+ 1. Start a recording with `skill_start` (optionally pass a codebook), then use
283
+ `control_ui` / `control_features` to inspect controls.
284
+ 2. Save or refine codes with `control_codebook`, then act with `control_tap`,
285
+ `control_input`, `control_long_press`, `control_swipe`, `control_assert` or `control_wait`.
286
+ Each semantic action is appended to the recording.
287
+ 3. Stop with `skill_stop`. The Skill stores its embedded codebook and steps under the workspace
288
+ `.apk-vm/skills/` directory (gitignored).
289
+ 4. Replay with `skill_run`, optionally using `untilCode` to stop at a page and `speed` to scale
290
+ optional waits. Steps use `waitFor` polling, not only fixed sleeps.
291
+
292
+ A codebook entry looks like this:
293
+
294
+ ```json
295
+ {
296
+ "codebook": {
297
+ "version": 1,
298
+ "entries": [
299
+ { "code": "A01", "name": "USERNAME", "features": { "resourceId": "com.example:id/user" } },
300
+ { "code": "LOGIN_BTN", "aliases": ["A02"], "features": { "text": "Sign in", "clickable": true } },
301
+ { "code": "STAR", "features": { "image": { "name": "star-icon", "hash": "…", "maxDistance": 10 } } }
302
+ ]
303
+ }
304
+ }
305
+ ```
306
+
307
+ - Codes may be numeric (`A01`) or semantic (`LOGIN_BTN`); aliases are accepted too.
308
+ - A matcher combines structural features (`resourceId`, `text`, `textAny`, `contentDesc`,
309
+ `className`, `clickable`, `enabled`, bounds ratio). Multiple matches are an explicit error with
310
+ candidates, never a silent first match.
311
+ - `control_features` can capture optional image features (PNG dHash + average/dominant colour)
312
+ for a control; give the image a name and merge the suggested entry into the codebook.
313
+ Image matching is only used when the entry declares `features.image`.
314
+ - `skill_run` accepts `variables` for `{{placeholder}}` interpolation, `dryRun` to resolve
315
+ controls without touching the device, and `timeoutMs`. A failed step returns a screenshot path,
316
+ UI XML, candidates and the step index.
317
+ - The raw temporal flow recorder (`flow_*`) is still available for compatibility; Skills are the
318
+ semantic, code-based option.
319
+
320
+ HTTP routes: `GET|POST /apk-vm/skills` and `POST /apk-vm/controls` (with the same
321
+ `/apk-vm/instances/<id>/...` variants). `apk_vm` actions: `skill_list`, `skill_get`,
322
+ `skill_save`, `skill_delete`, `skill_start`, `skill_stop`, `skill_cancel`, `skill_run`,
323
+ `skill_codebook`, `control_ui`, `control_features`, `control_codebook`, `control_tap`, `control_input`,
324
+ `control_long_press`, `control_swipe`, `control_assert`, `control_wait`, `control_launch`.
325
+
326
+ This feature deliberately does **not** call arbitrary functions inside the target app. It drives
327
+ the app only through its UI, so it keeps working for third-party APKs and does not require app
328
+ instrumentation.
329
+
330
+ ## VM ownership and dedicated conversation/project VMs
331
+
332
+ Every running VM has a renewable **owner lease**, so two DSH conversations cannot silently drive
333
+ the same emulator. A lease records the owner key, display label, scope (`shared`, `conversation`
334
+ or `project`), session id, workspace, and `since` / `lastSeenAt` timestamps; leases expire after
335
+ 30 minutes (`leaseTtlMs`) and are renewed by `apk_vm` calls that route to that VM.
336
+
337
+ - The first conversation that needs a VM claims the default instance. A different conversation
338
+ gets a dedicated extra instance instead of taking the default one.
339
+ - `ownerScope=conversation` gives the current DSH session a VM that follows it across turns;
340
+ `ownerScope=project` does the same for the current workspace. `ownerScope=shared` keeps the
341
+ first-come default behaviour.
342
+ - If an `instance` is explicitly requested and owned by another conversation, the call fails with
343
+ `vm-busy` and the owner details. `ownerForce=true` is an explicit takeover and should be used
344
+ only after checking `owner_status`.
345
+ - The VM instances card shows the current owner/scope. The **VM usage** card in the panel
346
+ shows a readable owner label and provides **Claim**, **Release** and a `shared` /
347
+ `conversation` / `project` scope control for the human user.
348
+ - `owner_status` lists default/extra VM leases; `claim_vm` and `release_vm` manage them.
349
+ HTTP: `GET|POST /apk-vm/ownership` (`status`, `claim`, `release`, `touch`), while
350
+ `GET /apk-vm/state` and `GET /apk-vm/instances` include owner metadata.
351
+ - When the panel makes a mutating HTTP request it sends
352
+ `x-apk-vm-owner`, `x-apk-vm-scope`, `x-apk-vm-session`, `x-apk-vm-workspace` and
353
+ `x-apk-vm-label`. Header values may be percent-encoded (UTF-8); the Host decodes them, so
354
+ non-ASCII labels and workspace paths remain valid for browser `fetch`. A request that
355
+ supplies an owner header auto-claims a free VM; on a leased VM
356
+ it must match the current owner key. `POST /apk-vm/*` control routes return `409 vm-busy` with
357
+ the current owner when the header is missing or does not match. Read-only `GET` routes are
358
+ never blocked.
359
+ - Closing a DSH conversation (the `session/disposed` event) automatically releases its leases,
360
+ with the 30-minute TTL as a fallback if that event is unavailable.
361
+
266
362
  ## Storage management
267
363
 
268
364
  `GET /apk-vm/storage` reports the logical size of every AVD, its quick-boot snapshots, the
@@ -300,7 +396,9 @@ port forwards and pcap status. `POST /apk-vm/network` accepts `action`:
300
396
  { "action": "pcap", "path": "D:\\apk-vm-capture.pcap" }
301
397
  ```
302
398
 
303
- - Port forwards use `adb forward tcp:<hostPort> tcp:<guestPort>`.
399
+ - Port forwards use `adb forward tcp:<hostPort> tcp:<guestPort>`. The panel offers one-click
400
+ presets for **Metro 8081**, **HTTP 8080**, **Frida 27042** and **DevTools 9222**, plus the
401
+ manual host/guest fields.
304
402
  - DNS is applied by the emulator `-dns-server` flag on the next start.
305
403
  - Hosts entries are bind-mounted over the guest `/etc/hosts` (root required) and reapplied
306
404
  after a plugin-managed boot.
@@ -433,6 +531,7 @@ config:
433
531
  maxInstances: 3 # extra VMs (default VM is always available)
434
532
  instanceIdleMs: 900000 # stop idle extra VMs after 15 min (AVD kept)
435
533
  instanceReapIntervalMs: 60000
534
+ leaseTtlMs: 1800000 # VM owner lease lifetime; renewed by apk_vm calls
436
535
  allowRemote: false # reject non-loopback VM API clients
437
536
  allowSetupDownloads: false # gate the optional system-image download action
438
537
  scrcpy: true
@@ -495,6 +594,7 @@ All keys are optional; defaults are shown in the **Default** column.
495
594
  | `maxInstances` | `3` | Maximum number of extra instances (in addition to the default VM). Each running instance costs guest RAM/CPU. |
496
595
  | `instanceIdleMs` | `900000` | Stop an extra instance after this long without panel frames, a video client, or an Agent command (AVD is kept). |
497
596
  | `instanceReapIntervalMs` | `60000` | How often the idle reaper checks extra instances. |
597
+ | `leaseTtlMs` | `1800000` | Lifetime of a VM owner lease. Every `apk_vm` call that routes to the VM renews it; set `0` to disable expiry. |
498
598
  | `allowRemote` | `false` | Allow non-loopback clients to reach `/apk-vm/*` and `/apk-vm/ws` (only behind your own authenticated proxy). |
499
599
  | `allowSetupDownloads` | `false` | Allow the Environment card's `install-image` action to run `sdkmanager --install` (large download). |
500
600
  | `frameFps` | `2` | PNG fallback target frame rate, 1–10 (the H.264 path is not affected). |
@@ -614,7 +714,8 @@ JSON bodies are bounded to 64 KiB; the raw APK upload is bounded to 500 MiB.
614
714
  | POST | `/apk-vm/stream` | `{"fps":1–10}` — set the PNG fallback target frame rate. |
615
715
  | POST | `/apk-vm/stream/quality` | `{"preset":"low\|balanced\|high"}` — change H.264 resolution/bitrate/fps; the scrcpy session reconnects. |
616
716
  | GET (upgrade) | `/apk-vm/ws` | WebSocket H.264 stream (see above). |
617
- | GET | `/apk-vm/instances` | List default + extra VM instances, including `owner`, `lastUsedAt`, `idleMs`, `idleTimeoutMs`, `videoClients`. |
717
+ | GET/POST | `/apk-vm/ownership` | `GET` lists VM leases; `POST {"action":"claim\|release\|touch","instanceId":"default","ownerScope":"shared\|conversation\|project","sessionId"?,"workspace"?,"force"?}` manages the owner lease. |
718
+ | GET | `/apk-vm/instances` | List default + extra VM instances, including `owner`, `ownerInfo` (`key`, `label`, `scope`, `sessionId`, `workspace`, `since`, `lastSeenAt`), `lastUsedAt`, `idleMs`, `idleTimeoutMs`, `videoClients`. |
618
719
  | POST | `/apk-vm/instances` | `{"name":"agent-a","start":true}` — create an isolated AVD/VM; every extra instance has its own H.264 stream. |
619
720
  | DELETE | `/apk-vm/instances/:id` | Stop/dispose an extra instance (the default cannot be destroyed). |
620
721
  | GET/POST | `/apk-vm/instances/:id/...` | Per-instance mirror of `state`, `frame`, `vm`, `input`, `install`, `upload`, `debug/*`. |
@@ -678,8 +779,11 @@ plus `start` / `stop` and instance management (`list_instances`, `create_instanc
678
779
  duplicated logic).
679
780
 
680
781
  For concurrent Agents, pass `instance` to target a specific VM; when omitted, the tool resolves
681
- the calling Agent's assigned instance (first Agent → default VM, later Agents → isolated
682
- instances created on demand until `maxInstances`). Each extra instance has its own H.264
782
+ the calling Agent's lease and assigned instance (first conversation → default VM, later
783
+ conversations → isolated instances created on demand until `maxInstances`). Set
784
+ `ownerScope=conversation` or `ownerScope=project` to request a dedicated VM for the current DSH
785
+ session/workspace. `owner_status` lists leases, `claim_vm` / `release_vm` manage them, and
786
+ `ownerForce=true` is required for an explicit takeover. Each extra instance has its own H.264
683
787
  WebSocket (`/apk-vm/instances/<id>/ws`) and falls back to PNG if WebCodecs is unavailable.
684
788
 
685
789
  UI automation is live-verified against a multi-instance VM: `find_ui {"text":"Phone"}` returned